@intent-systems/nexus 2026.1.5-3

package/skills/agent-browser/SKILL.md

@@ -0,0 +1,450 @@
+---
+name: agent-browser
+description: Headless browser automation using the agent-browser CLI. Use when tasks require real browser navigation, clicking, form filling, or accessibility-tree snapshots of dynamic pages.
+homepage: https://github.com/vercel-labs/agent-browser
+type: tool
+capabilities: [browser-automation]
+license: Apache-2.0
+source:
+  repo: https://github.com/vercel-labs/agent-browser
+  commit: main
+security:
+  openSource: true
+  audit: required
+requires:
+  bins: [agent-browser, node]
+install:
+  - id: npm
+    kind: node
+    package: agent-browser
+    bins: [agent-browser]
+    label: Install agent-browser (npm)
+  - id: chromium
+    kind: shell
+    script: "agent-browser install"
+    label: Download Chromium for agent-browser
+metadata: {"nexus":{"emoji":"🌐","os":["darwin","linux","win32"],"requires":{"bins":["agent-browser","node"]},"install":[{"id":"npm","kind":"node","package":"agent-browser","bins":["agent-browser"],"label":"Install agent-browser (npm)"},{"id":"chromium","kind":"shell","script":"agent-browser install","label":"Download Chromium for agent-browser"}]}}
+---
+
+# agent-browser
+
+Headless browser automation CLI for AI agents. Use it when you need real browser navigation on dynamic sites, not just HTML fetching.
+
+## Quick start
+
+```bash
+agent-browser open https://example.com
+agent-browser snapshot -i --json
+agent-browser click @e2
+agent-browser fill @e3 "test@example.com"
+agent-browser get text @e1 --json
+agent-browser close
+```
+
+## Recommended workflow
+
+1. `open` the URL.
+2. `snapshot -i --json` to capture interactive elements with refs (`@e1`, `@e2`, ...).
+3. Use refs for actions (`click`, `fill`, `get text`).
+4. Re-snapshot after page changes.
+5. `close` when done.
+
+## Speed / model choice
+
+- Default to Claude Haiku 4.5 (use the latest Haiku in Claude Code).
+- Claude Code is the preferred dispatcher for fast runs.
+- Escalate only for complex reasoning or ambiguous flows.
+- The CLI is deterministic; keep planning short and execute quickly.
+
+## Targeting elements
+
+- **Refs (preferred):** Use refs from `snapshot`.
+- **Semantic finders:** `find role button click --name "Submit"`.
+- **CSS selectors:** `click "#submit"`, `fill "input[name=email]" "..."`.
+
+## Sessions and auth
+
+- Isolate work with sessions: `agent-browser --session login open https://...`.
+- Reuse login state:
+  - `agent-browser state save auth.json`
+  - `agent-browser state load auth.json`
+- Prefer header-based auth over UI logins:
+  - `agent-browser open https://api.example.com --headers '{"Authorization":"Bearer TOKEN"}'`
+
+## Tips
+
+- Use `wait` when pages are dynamic: `agent-browser wait --text "Welcome"`.
+- For debugging, run headed: `agent-browser open https://... --headed`.
+- If elements are off-screen, use `scroll` or `scrollintoview`.
+
+## Guardrails
+
+- Avoid typing or storing user passwords unless explicitly instructed.
+- Expect CAPTCHAs/2FA to require human intervention.
+- Close sessions after completing the task.
+
+## Demo playbook (embedded)
+
+This playbook gives two demos: a fast visible check and a harder "solver" flow.
+Both avoid API keys and OAuth.
+
+### Immediate value
+
+- Real browser automation on dynamic sites (not just HTML fetches).
+- Deterministic element refs from snapshots (stable, less brittle selectors).
+- Quick smoke tests for links, forms, and navigation.
+
+### Speed / model choice
+
+- Default to Claude Haiku 4.5 (use the latest Haiku in Claude Code).
+- Claude Code is the preferred dispatcher for fast runs.
+- Wordle uses a local word list; no answer lookup.
+
+### Prereqs
+
+```bash
+agent-browser --version
+agent-browser install
+```
+
+### Test 1: Hacker News headline (easy, visible)
+
+Goal: show a real page open, click, and read content.
+
+```bash
+# Visible browser window
+agent-browser --session demo open https://news.ycombinator.com --headed
+
+# Get interactive elements (refs)
+agent-browser --session demo snapshot -i -c -d 3
+
+# Pick the first story link ref from snapshot (example: @e1)
+agent-browser --session demo click @e1
+
+# Read the new page title and URL
+agent-browser --session demo get title
+agent-browser --session demo get url
+
+# Optional screenshot
+agent-browser --session demo screenshot /tmp/agent-browser-hn.png
+
+agent-browser --session demo close
+```
+
+Success looks like:
+- A headed browser opens.
+- Snapshot lists interactive links with refs.
+- Title/URL change after the click.
+
+### Test 2: NYT Wordle solver (harder, real site)
+
+Goal: solve Wordle using only a local word list and math, plus browser control.
+
+#### Approach
+
+1. Keep a local candidate list of valid words.
+2. After each guess, read tile evaluations (green/yellow/gray).
+3. Filter candidates by the feedback.
+4. Pick the next guess by maximizing letter information.
+
+#### One-time setup: word list + solver
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/tabatkins/wordle-list/main/words \
+  -o /tmp/wordle-words.txt
+```
+
+Create `~/nexus/home/projects/wordle-solver.mjs`:
+
+```js
+import fs from "fs";
+
+const WORDS_PATH = process.env.WORDLE_WORDS ?? "/tmp/wordle-words.txt";
+const STATE_PATH = process.env.WORDLE_STATE ?? "/tmp/wordle-state.json";
+
+if (!fs.existsSync(WORDS_PATH)) {
+  console.error(
+    "Missing word list. Download with: curl -fsSL https://raw.githubusercontent.com/tabatkins/wordle-list/main/words -o /tmp/wordle-words.txt",
+  );
+  process.exit(1);
+}
+
+const words = fs.readFileSync(WORDS_PATH, "utf8").trim().split(/\s+/);
+
+function score(guess, answer) {
+  const result = Array(5).fill("b");
+  const counts = {};
+  for (let i = 0; i < 5; i++) {
+    const g = guess[i];
+    const a = answer[i];
+    if (g === a) {
+      result[i] = "g";
+    } else {
+      counts[a] = (counts[a] ?? 0) + 1;
+    }
+  }
+  for (let i = 0; i < 5; i++) {
+    if (result[i] === "g") continue;
+    const g = guess[i];
+    if ((counts[g] ?? 0) > 0) {
+      result[i] = "y";
+      counts[g]--;
+    }
+  }
+  return result.join("");
+}
+
+function bestGuess(candidates) {
+  const freq = Array.from({ length: 5 }, () => ({}));
+  for (const w of candidates) {
+    for (let i = 0; i < 5; i++) {
+      const c = w[i];
+      freq[i][c] = (freq[i][c] ?? 0) + 1;
+    }
+  }
+  let best = candidates[0];
+  let bestScore = -1;
+  for (const w of candidates) {
+    let scoreSum = 0;
+    for (let i = 0; i < 5; i++) {
+      scoreSum += freq[i][w[i]] ?? 0;
+    }
+    if (scoreSum > bestScore) {
+      bestScore = scoreSum;
+      best = w;
+    }
+  }
+  return best;
+}
+
+function loadState() {
+  if (!fs.existsSync(STATE_PATH)) return { candidates: words };
+  return JSON.parse(fs.readFileSync(STATE_PATH, "utf8"));
+}
+
+function saveState(state) {
+  fs.writeFileSync(STATE_PATH, JSON.stringify(state, null, 2));
+}
+
+const args = new Map(
+  process.argv.slice(2).map((arg) => {
+    const [k, v] = arg.replace(/^--/, "").split("=");
+    return [k, v];
+  }),
+);
+
+const guess = args.get("guess");
+const pattern = args.get("pattern");
+
+const state = loadState();
+if (guess && pattern) {
+  state.candidates = state.candidates.filter((w) => score(guess, w) === pattern);
+}
+
+const suggestion = bestGuess(state.candidates);
+saveState(state);
+
+console.log(`remaining=${state.candidates.length}`);
+console.log(`suggested=${suggestion}`);
+```
+
+### Run flow
+
+```bash
+agent-browser --session wordle open https://www.nytimes.com/games/wordle/index.html --headed
+
+# Close intro modal / cookie banner
+agent-browser --session wordle find role button click --name "Play"
+agent-browser --session wordle find role button click --name "Continue"
+agent-browser --session wordle find role button click --name "Close"
+agent-browser --session wordle press Escape
+
+# If needed, inspect refs
+agent-browser --session wordle snapshot -i -c -d 4
+```
+
+Submit a guess:
+
+```bash
+node ~/nexus/home/projects/wordle-solver.mjs
+# example output: suggested=slate
+
+agent-browser --session wordle click "body"
+for c in s l a t e; do agent-browser --session wordle press "$c"; done
+agent-browser --session wordle press Enter
+agent-browser --session wordle wait 1200
+```
+
+Extract the last row feedback:
+
+```bash
+# Use the NYT eval helper to avoid shell-escaping issues
+agent-browser --session wordle eval "$(cat {baseDir}/docs/wordle-nyt-eval.js)" --json
+```
+
+Feed the pattern back into the solver and repeat:
+
+```bash
+node ~/nexus/home/projects/wordle-solver.mjs --guess=slate --pattern=bybgb
+```
+
+Stop when the pattern is `ggggg`, then:
+
+```bash
+agent-browser --session wordle screenshot /tmp/agent-browser-wordle.png
+agent-browser --session wordle close
+```
+
+Notes:
+- If `pattern` includes `?`, wait longer and re-run the eval.
+- If a guess is rejected, your word list is wrong; re-download the Wordle list.
+
+### Fast path (Claude Code + Haiku 4.5)
+
+Recommended for demos and LLM-in-loop runs. This uses the optimized runner flags
+that minimize wait time and block ads.
+
+```bash
+claude --model haiku --print --permission-mode bypassPermissions \
+  "Run: bash ~/nexus/home/projects/wordle-nyt-run.sh \
+  --session demo-haiku \
+  --timing end-to-end \
+  --block-ads \
+  --wait-ms 0 \
+  --key-delay-ms 0 \
+  --poll-delay-ms 50 \
+  --max-poll-tries 14 \
+  --no-screenshot"
+```
+
+Notes:
+- The speedup comes from the runner flags, not model reasoning.
+- For loop-only timing, change to `--timing guess-only`.
+
+### Fast benchmarking (no cheating)
+
+Use the automation script to compare models or tune speed:
+
+```bash
+bash ~/nexus/home/projects/wordle-nyt-run.sh \
+  --session bench-haiku \
+  --first-guess crane \
+  --timing end-to-end \
+  --block-ads \
+  --wait-ms 0 \
+  --key-delay-ms 0 \
+  --poll-delay-ms 50 \
+  --max-poll-tries 14 \
+  --no-screenshot
+```
+
+Notes:
+- `--timing guess-only` excludes page load and modal dismissal.
+- Lower `--wait-ms` and `--key-delay-ms` reduces latency.
+- Lower `--poll-delay-ms` reduces time between visual update and action.
+- `--block-ads` can reduce page load time (best-effort).
+- Use unique `--session` names per run to avoid Wordle state collisions.
+
+## Browser-use skills eval (embedded)
+
+This is a lightweight eval matrix for the browser-automation skills in this
+workspace. All tests are **no-cheating** and operate on the real internet.
+
+### Goals
+
+- Measure end-to-end latency and success rate.
+- Compare deterministic vs LLM-in-loop behaviors.
+- Keep tasks simple and reproducible.
+
+### Which is best?
+
+**Short answer:** use `agent-browser` for speed and determinism; use
+`browser-use/agent-sdk` when the task is unknown, ambiguous, or needs
+LLM reasoning and recovery.
+
+#### Key differences
+
+- **Control plane:** `agent-browser` is a deterministic CLI; `browser-use` is an
+  LLM-driven agent framework.
+- **Inputs:** `agent-browser` uses DOM refs/snapshots; `browser-use` typically
+  relies on screenshots + LLM decisions.
+- **Speed:** `agent-browser` is usually faster because it avoids vision + long
+  reasoning loops.
+- **Flexibility:** `browser-use` can handle novel UI flows better, at the cost
+  of latency and cost.
+
+### Performance tips (browser-use)
+
+- **Default model:** use Claude Haiku 4.5 via Claude Code for fast LLM-in-loop runs.
+- **Minimize perception:** avoid full-page screenshots when possible; keep the
+  viewport consistent and avoid repeated OCR.
+- **Wait for minimum readiness:** poll for interactive elements instead of
+  sleeping for full page load.
+- **Block ads/trackers:** reduces layout shifts and network churn (best-effort).
+- **Input timing:** use zero delays for benchmarks; add 20-80ms jitter for real
+  flows to reduce bot-detection risk.
+- **Keep prompts short:** one action per step; avoid long chain-of-thought.
+
+### Common setup
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/tabatkins/wordle-list/main/words \
+  -o /tmp/wordle-words.txt
+```
+
+### Skill: agent-browser
+
+#### Wordle (scripted)
+
+```bash
+bash ~/nexus/home/projects/wordle-nyt-run.sh \
+  --session eval-agent-browser \
+  --first-guess crane \
+  --timing guess-only \
+  --wait-ms 300 \
+  --key-delay-ms 10 \
+  --no-screenshot
+```
+
+#### HN click (smoke)
+
+```bash
+agent-browser --session eval-hn open https://news.ycombinator.com --headed
+agent-browser --session eval-hn snapshot -i -c -d 3
+agent-browser --session eval-hn click @e1
+agent-browser --session eval-hn get title
+agent-browser --session eval-hn close
+```
+
+### Skill: computer-use (Peekaboo / interpreter)
+
+#### HN click (visual)
+
+```bash
+peekaboo app switch --to "Google Chrome"
+peekaboo hotkey --keys "cmd,l"
+peekaboo paste --text "https://news.ycombinator.com"
+peekaboo press return
+sleep 2
+peekaboo see --annotate --path /tmp/hn.png
+```
+
+### Skill: browser-use/agent-sdk
+
+#### Wordle (LLM-in-loop)
+
+```bash
+LLM_MODEL=haiku python /tmp/bu_wordle_eval.py
+```
+
+Notes:
+- See `skills/browser-use-agent-sdk/SKILL.md` for the eval harness.
+- This wraps the existing Wordle script as a tool to keep the test stable.
+
+## References
+
+- Demo playbook: `docs/first-tests.md`
+- NYT Wordle eval helper: `docs/wordle-nyt-eval.js`
+- Browser-use eval matrix: `docs/browser-use-eval.md`
+- CLI docs: https://github.com/vercel-labs/agent-browser#commands
+- Install and setup: https://github.com/vercel-labs/agent-browser#installation

package/skills/agent-browser/docs/browser-use-eval.md

@@ -0,0 +1,95 @@
+# Browser-use skills eval
+
+This is a lightweight eval matrix for the browser‑automation skills in this
+workspace. All tests are **no‑cheating** and operate on the real internet.
+
+## Goals
+
+- Measure end‑to‑end latency and success rate.
+- Compare deterministic vs LLM‑in‑loop behaviors.
+- Keep tasks simple and reproducible.
+
+## Which is best?
+
+**Short answer:** use `agent-browser` for speed and determinism; use
+`browser-use/agent-sdk` when the task is unknown, ambiguous, or needs
+LLM reasoning and recovery.
+
+### Key differences
+
+- **Control plane:** `agent-browser` is a deterministic CLI; `browser-use` is an
+  LLM‑driven agent framework.
+- **Inputs:** `agent-browser` uses DOM refs/snapshots; `browser-use` typically
+  relies on screenshots + LLM decisions.
+- **Speed:** `agent-browser` is usually faster because it avoids vision + long
+  reasoning loops.
+- **Flexibility:** `browser-use` can handle novel UI flows better, at the cost
+  of latency and cost.
+
+## Performance tips (browser-use)
+
+- **Default model:** use Claude Haiku 4.5 via Claude Code for fast LLM-in-loop runs.
+- **Minimize perception:** avoid full-page screenshots when possible; keep the
+  viewport consistent and avoid repeated OCR.
+- **Wait for minimum readiness:** poll for interactive elements instead of
+  sleeping for full page load.
+- **Block ads/trackers:** reduces layout shifts and network churn (best‑effort).
+- **Input timing:** use zero delays for benchmarks; add 20–80ms jitter for real
+  flows to reduce bot‑detection risk.
+- **Keep prompts short:** one action per step; avoid long chain‑of‑thought.
+
+## Common setup
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/tabatkins/wordle-list/main/words \
+  -o /tmp/wordle-words.txt
+```
+
+## Skill: agent-browser
+
+### Wordle (scripted)
+
+```bash
+bash ~/nexus/home/projects/wordle-nyt-run.sh \
+  --session eval-agent-browser \
+  --first-guess crane \
+  --timing guess-only \
+  --wait-ms 300 \
+  --key-delay-ms 10 \
+  --no-screenshot
+```
+
+### HN click (smoke)
+
+```bash
+agent-browser --session eval-hn open https://news.ycombinator.com --headed
+agent-browser --session eval-hn snapshot -i -c -d 3
+agent-browser --session eval-hn click @e1
+agent-browser --session eval-hn get title
+agent-browser --session eval-hn close
+```
+
+## Skill: computer-use (Peekaboo / interpreter)
+
+### HN click (visual)
+
+```bash
+peekaboo app switch --to "Google Chrome"
+peekaboo hotkey --keys "cmd,l"
+peekaboo paste --text "https://news.ycombinator.com"
+peekaboo press return
+sleep 2
+peekaboo see --annotate --path /tmp/hn.png
+```
+
+## Skill: browser-use/agent-sdk
+
+### Wordle (LLM‑in‑loop)
+
+```bash
+LLM_MODEL=haiku python /tmp/bu_wordle_eval.py
+```
+
+Notes:
+- See `skills/browser-use-agent-sdk/SKILL.md` for the eval harness.
+- This wraps the existing Wordle script as a tool to keep the test stable.