haltija 1.1.0

package/docs/CI-INTEGRATION.md

@@ -0,0 +1,230 @@
+# E2E Testing with Haltija in CI
+
+Run browser tests in GitHub Actions using Haltija. Your tests run against a real Electron browser — same engine your users run, no headless quirks.
+
+## Quick Start
+
+```yaml
+name: E2E Tests
+on: [push, pull_request]
+
+jobs:
+  e2e:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: oven-sh/setup-bun@v2
+
+      - name: Install Haltija
+        run: bun install && bun run build
+
+      - name: Install display deps
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y xvfb libnss3 libatk1.0-0 libatk-bridge2.0-0 \
+            libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \
+            libxfixes3 libxrandr2 libgbm1 libpango-1.0-0 libcairo2 libasound2
+
+      - name: Launch Haltija
+        run: |
+          cd apps/desktop && npm install --omit=dev
+          xvfb-run --auto-servernum npx electron . &
+          # Wait for ready
+          until hj status 2>/dev/null | grep -q '"ok":true'; do sleep 1; done
+
+      - name: Run tests
+        run: hj test-run tests/my-test.json
+```
+
+## The hj CLI
+
+The `hj` command is the simplest way to interact with Haltija:
+
+```bash
+# Check status
+hj status
+
+# See the page
+hj tree
+
+# Interact
+hj click "#submit"
+hj type "#email" user@example.com
+hj key Enter
+
+# Run tests
+hj test-run tests/login.json
+hj test-suite tests/           # Run all tests in directory
+```
+
+Run `hj --help` for all commands. The CLI auto-starts the server if needed.
+
+## Writing Tests
+
+Tests are JSON files with steps:
+
+```json
+{
+  "version": 1,
+  "name": "Login flow",
+  "url": "http://localhost:3000/login",
+  "steps": [
+    {"action": "type", "selector": "#email", "text": "user@example.com"},
+    {"action": "type", "selector": "#password", "text": "secret123"},
+    {"action": "click", "selector": "button[type=submit]"},
+    {"action": "wait", "selector": ".dashboard"},
+    {"action": "assert", "assertion": {"type": "url", "pattern": "/dashboard"}}
+  ]
+}
+```
+
+### Step Types
+
+| Action | Example | What It Does |
+|--------|---------|--------------|
+| `navigate` | `{"action": "navigate", "url": "..."}` | Load a URL |
+| `click` | `{"action": "click", "selector": "#btn"}` | Click element |
+| `type` | `{"action": "type", "selector": "#input", "text": "..."}` | Type text |
+| `key` | `{"action": "key", "key": "Enter"}` | Press key |
+| `wait` | `{"action": "wait", "selector": ".loaded"}` | Wait for element |
+| `assert` | `{"action": "assert", "assertion": {...}}` | Check condition |
+| `eval` | `{"action": "eval", "code": "..."}` | Run JavaScript |
+
+### Assertions
+
+```json
+{"type": "exists", "selector": ".modal"}
+{"type": "not-exists", "selector": ".error"}
+{"type": "text", "selector": "h1", "text": "Welcome"}
+{"type": "url", "pattern": "/dashboard"}
+{"type": "visible", "selector": "#content"}
+```
+
+### Step Metadata
+
+Add context to improve failure messages:
+
+```json
+{
+  "action": "click",
+  "selector": "#checkout",
+  "description": "Click checkout button",
+  "purpose": "Button may be disabled if cart is empty"
+}
+```
+
+## Running Tests
+
+### Single test
+```bash
+hj test-run tests/login.json
+```
+
+### Multiple tests
+```bash
+hj test-suite tests/
+```
+
+### With options
+```bash
+# JSON output (default)
+hj test-run tests/login.json
+
+# Human-readable output
+hj test-run tests/login.json --format human
+
+# GitHub Actions annotations
+hj test-run tests/login.json --format github
+```
+
+## Handling Flaky Tests
+
+Use the patience model for CI environments with variable timing:
+
+```json
+{
+  "test": {...},
+  "patience": 5,
+  "patienceStreak": 3,
+  "timeout": 8000
+}
+```
+
+- `patience: 5` — allow up to 5 step failures before bailing
+- `patienceStreak: 3` — 3 consecutive failures bails immediately
+- `timeout: 8000` — per-step timeout in ms
+
+## Debugging Failures
+
+### Capture state on failure
+```yaml
+- name: Debug info
+  if: failure()
+  run: |
+    hj screenshot > failure-screenshot.json
+    hj console > console-logs.json
+    hj snapshot > failure-state.json
+
+- uses: actions/upload-artifact@v4
+  if: failure()
+  with:
+    name: debug
+    path: "*.json"
+```
+
+### Check what's on the page
+```bash
+hj tree                    # DOM structure
+hj console                 # Browser console
+hj screenshot              # Visual capture
+```
+
+## Platform Notes
+
+### Linux (GitHub Actions)
+Requires xvfb and Electron dependencies:
+```bash
+sudo apt-get install -y xvfb libnss3 libatk1.0-0 libatk-bridge2.0-0 \
+  libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \
+  libxfixes3 libxrandr2 libgbm1 libpango-1.0-0 libcairo2 libasound2
+
+xvfb-run --auto-servernum npx electron . &
+```
+
+### macOS
+No xvfb needed:
+```bash
+npx electron . &
+```
+
+## Testing Your Own App
+
+Start your app before running tests:
+
+```yaml
+- name: Start app
+  run: |
+    npm start &
+    until curl -sf http://localhost:3000 > /dev/null; do sleep 1; done
+
+- name: Run tests
+  run: hj test-run tests/app-tests.json
+```
+
+## Tips
+
+- **Start simple** — one test, few steps, then expand
+- **Use `purpose`** on steps that might fail — explains intent on failure
+- **Navigate explicitly** — don't rely on default page state
+- **Upload artifacts** — screenshots and snapshots are invaluable for debugging
+
+## REST API
+
+For direct HTTP integration (scripts, other languages), see [REST-API.md](REST-API.md).
+
+## Examples
+
+See test files in this repo:
+- [`tests/playground.json`](../tests/playground.json)
+- [`tests/homepage.json`](../tests/homepage.json)
+- [`.github/workflows/test-qa.yml`](../.github/workflows/test-qa.yml)

package/docs/EXECUTIVE-SUMMARY.md

@@ -0,0 +1,213 @@
+# Haltija Executive Summary
+
+Haltija gives AI agents eyes and hands in the browser. Instead of parsing screenshots or guessing at page structure, agents see the actual DOM, click elements, type text, and watch for changes. One script tag makes any web app AI-controllable.
+
+---
+
+## For the CEO
+
+Haltija is infrastructure for AI-powered quality assurance. It replaces brittle, expensive end-to-end test suites with an AI QA engineer that explores applications, generates test plans, runs tests, and reports failures in plain English.
+
+The business case: E2E test maintenance is a significant engineering cost. Tests break when UI changes, produce cryptic errors, and require constant human attention. Haltija shifts that burden to AI, freeing engineering time for product work.
+
+- **Reduces QA engineering overhead** by automating test creation and maintenance
+- **Improves bug reports** with human-readable failure explanations, not stack traces
+- **Runs in CI** like existing test infrastructure, no workflow changes required
+- **Aligns with core AI strategy** as a practical application of AI agents doing real work
+- **Potential product opportunity** beyond internal tooling
+
+**Limitations**: Requires investment to productionize. Not yet proven at scale. Would need dedicated support if deployed broadly.
+
+**Missing**: Cloud-hosted offering, enterprise authentication integrations, usage analytics dashboard.
+
+---
+
+## For the CTO
+
+Haltija is a WebSocket bridge between a browser widget and a REST API. AI agents make HTTP calls; the widget executes them in a real browser context. The architecture is simple: no browser binaries to manage, no protocol complexity, no version mismatches.
+
+The technical differentiator is semantic events. Rather than exposing raw DOM events, Haltija aggregates user intent: "user typed email@example.com" not eighteen keydown events. This makes AI reasoning about browser state tractable.
+
+- **Zero-dependency deployment**: `bunx haltija` starts the server, one script tag injects the widget
+- **Schema-driven API** with self-documenting endpoints and type-safe handlers
+- **Real browser rendering** via Electron, not headless browser quirks
+- **Extensible**: new endpoints require schema definition + handler, router handles the rest
+- **Built on Bun** for fast startup and minimal resource footprint
+
+**Limitations**: Shadow DOM support is workable but not seamless. No iframe traversal for cross-origin content.
+
+**Missing**: Node.js support for the server (Bun-only currently), browser extension for persistent injection, pre-built binaries for all platforms (build script works on macOS/Linux, Windows untested).
+
+---
+
+## For the UI Engineer
+
+Haltija lets you test your components with an AI that actually sees what users see. Point it at your dev server, and it can explore your app, find interactive elements, and verify behavior without you writing selectors or maintaining test fixtures.
+
+Integration is one line: `<script src="http://localhost:4000/component.js"></script>`. The widget connects to a local server, and any AI agent with HTTP access can now control your browser tab.
+
+- **DOM tree inspection** with configurable depth, text content, and attribute filtering
+- **Visibility heuristics** that match user perception: hidden, off-screen, transparent, and disabled states
+- **Input value tracking** shows current form state without querying each field
+- **Matched CSS rules** option shows which stylesheets affect an element and why
+- **Cursor visualization** shows exactly where clicks land, useful for debugging interaction issues
+
+**Limitations**: Cannot pierce cross-origin iframes. Custom elements with closed shadow roots are opaque. Very dynamic UIs (heavy animation, virtual scrolling) may report stale state.
+
+**Missing**: React/Vue/Angular devtools integration, component-level boundaries in tree output, performance profiling hooks.
+
+---
+
+## For the QA Engineer
+
+Haltija is a QA engineer in a box. It explores your application, identifies interactive elements, generates test plans, executes them, and reports failures in terms you can act on. When tests break because UI changed, it can often fix them automatically.
+
+Unlike Selenium or Playwright, you don't write selectors. You describe intent: "log in as test user, add item to cart, verify checkout total." The AI figures out how to do it on your actual UI.
+
+- **Natural language test specs** that survive UI refactors
+- **Failure explanations** like "button not visible because parent has display:none" not "timeout after 30000ms"
+- **Semantic event recording** captures what the user did, not raw browser events
+- **Test JSON format** for version control and CI integration
+- **Actionable summary** lists all buttons, links, inputs on a page with their current state
+
+**Limitations**: AI interpretation adds latency compared to direct selector tests. Novel or highly custom UI patterns may confuse the AI. Not a replacement for unit tests or integration tests.
+
+**Missing**: Visual regression comparison, accessibility audit integration, performance budget assertions, test coverage reporting.
+
+---
+
+## For the Security Consultant
+
+Haltija runs a WebSocket server on localhost that accepts commands to control browser tabs. The widget self-identifies when active (no silent operation), and users can pause or kill the connection at any time.
+
+The threat model assumes a trusted local environment. The server binds to localhost by default. Cross-origin stylesheets cannot be inspected due to browser security. The widget cannot access cross-origin iframe content.
+
+- **Localhost-only by default**, no remote connections without explicit configuration
+- **Visible indicator** in browser when agent is connected and operating
+- **User kill switch** to immediately disconnect and remove widget
+- **No credential storage**, authentication is handled by the browser normally
+- **CSP-aware**: widget injection respects Content-Security-Policy where enforced
+
+**Limitations**: Bookmarklet injection bypasses CSP on the injecting page. Desktop app strips CSP headers for universal compatibility. No audit logging of commands executed. No authentication on the REST API.
+
+**Missing**: API authentication/authorization, command audit log, rate limiting, configurable command allowlists, SOC2 compliance documentation.
+
+---
+
+## For the Hobbyist / Vibe Coder
+
+Haltija lets you tell Claude to browse the web for you. Run the server, open the desktop app or inject the widget, and Claude can see pages, click buttons, fill forms, and tell you what happened.
+
+Setup takes two minutes: install Bun, run `bunx haltija`, open the Haltija app, and paste the agent prompt into your conversation. Now Claude has a browser.
+
+- **Copy-paste prompt** gets Claude controlling your browser immediately (simpler than MCP)
+- **Visual feedback** shows cursor movement and action subtitles as Claude operates
+- **Explore any website** with the included Electron browser
+- **Record your actions** and let Claude replay or modify them
+- **No coding required** for basic browsing automation
+
+**Limitations**: Some sites block the widget (strict CSP). CAPTCHAs and bot detection will stop automation. Sites requiring login need manual authentication first.
+
+**Missing**: One-click installer, browser extension for easier injection, mobile support, saved session/cookie management.
+
+---
+
+## For the AI Enthusiast
+
+Haltija is what browser MCP tools should be. Instead of sending screenshots and hoping vision models figure it out, Haltija gives agents structured DOM access. The AI sees elements, attributes, text content, and visibility state directly.
+
+The semantic event system is particularly interesting: instead of raw DOM events, Haltija aggregates meaningful actions. This makes it feasible for AI to understand user sessions without drowning in event noise.
+
+- **DOM over screenshots**: structured data beats pixel parsing for reliability
+- **Semantic events**: "user typed 'hello'" not 17 keystrokes
+- **Hindsight buffer**: review what happened without recording everything upfront
+- **Mutation watching** with noise filtering for framework-specific chatter
+- **Tool-use optimized**: API returns exactly what agents need, nothing more
+
+**Limitations**: Text-heavy UIs work best. Canvas, WebGL, and video content are opaque. Very large DOMs may need pagination or focused queries.
+
+**Missing**: Vision model fallback for non-DOM content, multi-modal event capture (audio, video), agent memory/persistence across sessions.
+
+---
+
+## Efficiency & Performance
+
+Haltija is designed for efficiency - reducing the data agents need to process while preserving the information they need to act.
+
+### Event Reduction: 99%+
+
+Raw DOM events are noisy. A user typing "hello@example.com" generates dozens of keydown, keypress, input, and keyup events. Haltija's semantic event system aggregates these into a single `input:typed` event with the final value. Typical reduction: **99%+ fewer events** while preserving user intent.
+
+### DOM Reduction
+
+Full DOM trees are massive. Haltija filters to what matters:
+- Interactive elements (buttons, inputs, links)
+- Visible content (hidden elements filtered)
+- Interesting attributes (ARIA, data-*, roles)
+- Configurable depth limits
+
+A 10,000-node DOM might reduce to 200 relevant nodes for a form-filling task.
+
+### Ref IDs: Efficient Re-targeting
+
+Every element in `/tree` output includes a ref ID (e.g., `1`, `42`). Agents can use these refs instead of CSS selectors for subsequent commands:
+
+```bash
+# First, get the tree
+hj tree
+# Response includes: 42: button "Submit" [interactive]
+
+# Later, click by ref - no selector matching needed
+hj click 42
+```
+
+Refs survive DOM changes better than selectors (which break when classes change) and are faster to resolve (direct lookup vs. CSS matching).
+
+### Measuring Efficiency
+
+Use the `/stats` endpoint or click the 📊 button in the widget to see real metrics:
+
+```json
+{
+  "events": { "raw": 1847, "semantic": 23, "reductionPercent": 98.8 },
+  "dom": { "processed": 3420, "inTree": 156, "reductionPercent": 95.4 },
+  "refs": { "assigned": 156, "resolved": 42, "stale": 3, "hitRate": 93.3 }
+}
+```
+
+Console access: `haltija.copyStats()` copies full stats to clipboard.
+
+---
+
+## For Existing Puppeteer MCP / Browser Automation Users
+
+If you're using Puppeteer MCP, Playwright MCP, or similar tools, Haltija offers a different philosophy: user-centric rather than developer-centric.
+
+Puppeteer exposes browser internals. You think in selectors, wait conditions, and protocol commands. When tests fail, you get stack traces. Haltija exposes user-visible state. You think in elements, actions, and outcomes. When tests fail, you get explanations.
+
+- **No browser binary management**: widget runs in any browser, server is a single command
+- **Human-readable failures**: "element hidden by ancestor with display:none" vs "timeout"
+- **Semantic events**: understand user intent, not DOM mutations
+- **Real browser rendering**: Electron app has full engine, not headless quirks
+- **Designed for AI agents**: API returns structured, actionable data
+
+**Limitations**: Less low-level control than Puppeteer. Cannot intercept network requests or modify browser behavior. No protocol-level access for advanced debugging.
+
+**Missing**: Network interception, request mocking, browser console forwarding to agent, multi-browser parallel execution.
+
+---
+
+## Summary
+
+| Audience | Primary Value | Key Limitation |
+|----------|--------------|----------------|
+| CEO | Reduced QA costs, AI-native testing | Needs investment to productionize |
+| CTO | Clean architecture, semantic events | Bun-only server, Windows untested |
+| UI Engineer | One-line integration, real DOM access | No cross-origin iframe support |
+| QA Engineer | Natural language tests, auto-fixing | AI latency vs direct selectors |
+| Security | Localhost-only, visible operation | No API auth, CSP bypassed in app |
+| Hobbyist | Easy setup, visual feedback | Some sites block widget |
+| AI Enthusiast | Structured DOM, semantic events | Canvas/WebGL opaque |
+| Puppeteer User | User-centric, human-readable | Less low-level control |
+
+Haltija is production-ready for local development and testing workflows. Cloud deployment and enterprise features would require additional investment.

package/docs/README.md

@@ -0,0 +1,67 @@
+# Haltija Documentation
+
+## Quick Start
+
+1. **[Getting Started: Service](getting-started/service.md)** - Run `bunx haltija` and inject the widget
+2. **[Getting Started: App](getting-started/app.md)** - Add one script tag to your app
+3. **[Getting Started: Playground](getting-started/playground.md)** - Interactive testing environment
+4. **[CI Integration](CI-INTEGRATION.md)** - Run Haltija in GitHub Actions and other CI systems
+
+## Recipes
+
+**[Recipes](recipes.md)** - Common workflows with copy-paste examples:
+- Testing login flows
+- Exploring unfamiliar UIs
+- Recording bug reproductions
+- Generating tests from manual exploration
+- Debugging customer issues
+- Accessibility auditing
+- Multi-tab testing (OAuth, admin/user)
+- Waiting for dynamic content
+- User selection ("point at the problem")
+
+## Reference
+
+- **[API Reference](../API.md)** - Complete endpoint documentation (auto-generated from schema)
+- **[Agent Prompt](agent-prompt.md)** - System prompt for AI agents using Haltija
+- **[UX Crimes](UX-CRIMES.md)** - Anti-patterns Haltija detects automatically
+
+## Architecture
+
+- **[CLAUDE.md](../CLAUDE.md)** - Build commands, architecture overview, code structure
+- **[Component Patterns](../COMPONENT-PATTERNS.md)** - Design patterns used in the widget
+
+## Planning
+
+- **[Executive Summary](EXECUTIVE-SUMMARY.md)** - What Haltija is, who it's for
+- **[Roadmap to 10/10](ROADMAP-TO-10.md)** - Where we're going
+- **[Development Roadmap](../ROADMAP.md)** - Completed and planned phases
+- **[TODO](../TODO.md)** - Outstanding issues and ideas
+
+## Desktop App
+
+- **[Desktop README](../apps/desktop/README.md)** - Electron app setup and building
+
+---
+
+## Documentation Map
+
+```
+README.md                    # Main entry point, 30-second pitch
+docs/
+  README.md                  # This file - documentation index
+  getting-started/
+    service.md               # Start the server
+    app.md                   # Add to your app
+    playground.md            # Interactive testing
+  CI-INTEGRATION.md          # GitHub Actions, CI/CD setup
+  recipes.md                 # Common workflows with examples
+  EXECUTIVE-SUMMARY.md       # For stakeholders
+  ROADMAP.md                 # Product roadmap (to 11/10)
+  agent-prompt.md            # AI agent system prompt
+  UX-CRIMES.md               # Anti-pattern detection
+API.md                       # Auto-generated API reference
+CLAUDE.md                    # Developer guide (for AI and humans)
+COMPONENT-PATTERNS.md        # Widget architecture patterns
+TODO.md                      # Issues and ideas
+```

package/docs/REST-API.md

@@ -0,0 +1,123 @@
+# Haltija REST API Reference
+
+> For most use cases, use the `hj` CLI instead. This document is for direct HTTP integration.
+
+The Haltija server exposes a REST API at `http://localhost:8700`. All POST endpoints accept JSON bodies and return JSON responses.
+
+## Quick Reference
+
+```bash
+# Health / readiness
+GET  /status                       # Server up?
+GET  /windows                      # Browser connected?
+
+# Navigation
+POST /navigate  {"url": "..."}     # Go to URL
+GET  /location                     # Current URL + title
+
+# Interaction
+POST /click     {"selector": "..."} 
+POST /type      {"selector": "...", "text": "..."}
+POST /key       {"key": "Enter"}
+
+# Inspection
+POST /tree      {"depth": 3}       # DOM tree with ref IDs
+POST /query     {"selector": "..."} # Find element details
+
+# Testing
+POST /test/run  {"test": {...}}    # Run one test
+POST /test/suite {"tests": [...]}  # Run multiple tests
+
+# Debugging
+GET  /console                      # Browser console output
+POST /screenshot                   # Page capture (base64 PNG)
+POST /snapshot                     # Full debug state dump
+
+# Tabs
+POST /tabs/open  {"url": "..."}    # New tab
+POST /tabs/close {"window": "..."}  # Close tab
+```
+
+## Response Format
+
+All POST endpoints return:
+```json
+{"success": true, "data": ...}
+```
+or on error:
+```json
+{"success": false, "error": "..."}
+```
+
+## Targeting Specific Tabs
+
+Add `?window=<id>` to any endpoint or include `"window": "id"` in the POST body.
+
+Get window IDs from `GET /windows`.
+
+## Full API Documentation
+
+Run `hj api` or visit `http://localhost:8700/api` for complete endpoint documentation with all parameters and examples.
+
+## curl Examples
+
+### Check server status
+```bash
+curl http://localhost:8700/status
+```
+
+### See page structure
+```bash
+curl -X POST http://localhost:8700/tree \
+  -H "Content-Type: application/json" \
+  -d '{"depth": 3, "mode": "actionable"}'
+```
+
+### Click an element
+```bash
+curl -X POST http://localhost:8700/click \
+  -H "Content-Type: application/json" \
+  -d '{"selector": "#submit"}'
+```
+
+### Type text
+```bash
+curl -X POST http://localhost:8700/type \
+  -H "Content-Type: application/json" \
+  -d '{"selector": "#email", "text": "user@example.com"}'
+```
+
+### Navigate
+```bash
+curl -X POST http://localhost:8700/navigate \
+  -H "Content-Type: application/json" \
+  -d '{"url": "https://example.com"}'
+```
+
+### Take screenshot
+```bash
+curl -X POST http://localhost:8700/screenshot \
+  -H "Content-Type: application/json" \
+  -d '{"maxWidth": 800}'
+```
+
+### Run a test
+```bash
+curl -X POST http://localhost:8700/test/run \
+  -H "Content-Type: application/json" \
+  -d @tests/my-test.json
+```
+
+## hj Equivalents
+
+Every curl command above has a simpler `hj` equivalent:
+
+| curl | hj |
+|------|-----|
+| `curl localhost:8700/status` | `hj status` |
+| `curl -X POST localhost:8700/tree -d '{...}'` | `hj tree` |
+| `curl -X POST localhost:8700/click -d '{"selector":"#btn"}'` | `hj click "#btn"` |
+| `curl -X POST localhost:8700/type -d '{"selector":"#email","text":"..."}' | `hj type "#email" user@example.com` |
+| `curl -X POST localhost:8700/navigate -d '{"url":"..."}'` | `hj navigate example.com` |
+
+Use `hj --help` for the full command list.