alvin-bot 4.8.8 → 4.8.9

package/CHANGELOG.md

@@ -2,6 +2,34 @@
 
 All notable changes to Alvin Bot are documented here.
 
+## [4.8.9] — 2026-04-11
+
+### 🐛 Browser automation: dead `browse-server.cjs` path removed, 3-tier router now the source of truth
+
+The `browse` skill used to instruct the agent to start `node scripts/browse-server.cjs` on port 3800 for every browser task. That file was deleted in an earlier cleanup (see `20283c9` for the original 577-line version — now gone), but `skills/browse/SKILL.md` was never updated. Result: any browser-related user message on Telegram — or any cron job that hit the skill — got a system-prompt injection telling it to call a gateway that didn't exist, producing half-failed runs like the "Daily Job Alert" cron that couldn't load LinkedIn or StepStone.
+
+**What changed:**
+
+- **`skills/browse/SKILL.md` — full rewrite.** Now documents the hub 3-tier router at `~/.claude/hub/SCRIPTS/browser.sh`:
+  - **Tier 0** — WebFetch / `curl` for static pages and APIs
+  - **Tier 1** — `browser.sh stealth <url>` (Playwright + stealth plugin, headless, Cloudflare-masking)
+  - **Tier 2** — `browser.sh cdp {start|goto|shot|tabs|stop}` (real Chrome with persistent profile at `~/.claude/hub/BROWSER/profile/`, login cookies survive restarts)
+  - **Tier 3** — Claude-in-Chrome extension via MCP tools (interactive CLI only)
+  - Explicit escalation ladder (WebFetch → stealth → CDP → ask Ali to log in) and a `NIEMALS browse-server.cjs nutzen` anti-rule.
+  - Concrete working targets (StepStone ✅, Michael Page ✅, LinkedIn ✅ with login, Indeed ❌) so the agent knows what to try where.
+
+- **`src/services/browser-manager.ts` — hardened fallback chain.** The multi-strategy manager already had the right *shape* (`gateway → cdp → hub-stealth → cli`) but several ops silently broke or hung:
+  - **`gatewayRequest` now has a 15 s timeout** (`req.destroy` on elapse). Previously a hung gateway would wedge the caller forever.
+  - **CDP fallback for interactive ops.** `click`, `fill`, `type`, `press`, `scroll`, `evaluate`, `info`, and `getTree` used to hard-throw `"requires gateway"` when `browse-server.cjs` wasn't running. They now try the gateway first, then a short-lived `chromium.connectOverCDP()` via a new `withCdpPage()` helper that reuses Ali's live Chrome on port 9222. Refs are interpreted as CSS selectors when gateway is absent.
+  - **Explicit PNG extension** on auto-generated screenshot filenames (`shot_<ts>.png`) so Playwright's format inference is unambiguous.
+  - **Better error messages** — every "needs interactive" throw now includes the exact command to start CDP Chrome (`~/.claude/hub/SCRIPTS/browser.sh cdp start headless`).
+
+- **`src/paths.ts` — `HUB_BROWSER_SH` constant.** New absolute path to `~/.claude/hub/SCRIPTS/browser.sh` so the manager can shell out without hard-coding `os.homedir()` inline.
+
+**Why this matters:** `browser-manager.ts` is still not wired into any bot code path (it's future-proofing), so the production fix for user-interactive flows is `SKILL.md`. The manager hardening ensures that when it does eventually get wired into a sub-agent tool, it won't hang on missing gateways or lose all interactive capability when only CDP is available.
+
+**Testing:** Tier 1 stealth end-to-end against `stepstone.de/jobs/it-delivery-director` → 1.2 MB HTML, title parsed. Module-level integration test: `navigate('https://example.com')` via auto-selected hub-stealth → correct title/URL. `resolveStrategy('gateway')` → cascades to CDP with visible warning. `info()` via CDP fallback → returns live Chrome state without throwing. Skills reload picks up the new SKILL.md (5977 chars), `matchSkills("browse linkedin")` hits the browse skill, `buildSkillContext("open stepstone.de")` injects the 3-tier guidance block.
+
 ## [4.8.8] — 2026-04-11
 
 ### ✨ Unlimited sub-agent & cron timeouts (user-configurable)

package/dist/paths.js

@@ -86,6 +86,8 @@ export const AGENTS_FILE = resolve(DATA_DIR, "AGENTS.md");
 export const HOOKS_DIR = resolve(DATA_DIR, "hooks");
 /** scripts/browse-server.cjs — HTTP gateway for persistent browser sessions */
 export const BROWSE_SERVER_SCRIPT = resolve(BOT_ROOT, "scripts", "browse-server.cjs");
+/** ~/.claude/hub/SCRIPTS/browser.sh — Hub 3-tier browser router (stealth, CDP, ext) */
+export const HUB_BROWSER_SH = resolve(os.homedir(), ".claude", "hub", "SCRIPTS", "browser.sh");
 /** data/exec-allowlist.json — User-defined exec allowlist */
 export const EXEC_ALLOWLIST_FILE = resolve(DATA_DIR, "exec-allowlist.json");
 /** assets/ — User asset files (CVs, cover letters, legal docs, photos) */

package/dist/services/browser-manager.js

@@ -1,34 +1,166 @@
 /**
- * Multi-Strategy Browser Manager
+ * Multi-Strategy Browser Manager — with automatic fallback chain.
  *
- * Auto-selects between three browser strategies:
- * - CLI: Headless Playwright, one-shot (screenshots, text extraction, PDF)
- * - Gateway: Persistent HTTP browser server (interactive browsing, form-filling)
- * - CDP: Attach to user's live Chrome via DevTools Protocol
+ * Strategy priority:
+ *   1. Gateway (browse-server.cjs HTTP server) — if script exists and is running
+ *   2. CDP (Chrome DevTools Protocol) — via hub browser.sh cdp, persistent cookies
+ *   3. Hub Stealth (Playwright + stealth plugin) — via hub browser.sh stealth
+ *   4. Raw CLI (bare Playwright) — last resort, easily blocked
+ *
+ * If a strategy is unavailable, we automatically cascade to the next one
+ * and log a warning so failures are visible, not silent.
  */
-import { spawn } from "child_process";
+import { execSync, spawn } from "child_process";
 import http from "http";
 import fs from "fs";
 import { config } from "../config.js";
-import { BROWSE_SERVER_SCRIPT } from "../paths.js";
+import { BROWSE_SERVER_SCRIPT, HUB_BROWSER_SH } from "../paths.js";
 import { screenshotUrl, extractText, generatePdf } from "./browser.js";
-/** Auto-select the best browser strategy for a task */
+const CDP_PORT = 9222;
+const EXEC_TIMEOUT = 60_000; // 60s for page loads via shell
+// ── Logging ──────────────────────────────────────────────────────────
+function log(msg) {
+    console.warn(`[browser-manager] ${msg}`);
+}
+// ── Availability Checks ──────────────────────────────────────────────
+function isGatewayScriptPresent() {
+    return fs.existsSync(BROWSE_SERVER_SCRIPT);
+}
+async function isGatewayRunning() {
+    try {
+        const health = await gatewayRequest("/health");
+        return !!health?.ok;
+    }
+    catch {
+        return false;
+    }
+}
+function isHubBrowserAvailable() {
+    return fs.existsSync(HUB_BROWSER_SH);
+}
+async function isCDPAvailable() {
+    return new Promise((resolve) => {
+        const req = http.get(`http://127.0.0.1:${CDP_PORT}/json/version`, (res) => {
+            let data = "";
+            res.on("data", (chunk) => (data += chunk));
+            res.on("end", () => resolve(res.statusCode === 200));
+        });
+        req.on("error", () => resolve(false));
+        req.setTimeout(3000, () => {
+            req.destroy();
+            resolve(false);
+        });
+    });
+}
+// ── Strategy Selection with Fallback ─────────────────────────────────
+/** Pick the preferred strategy based on task type */
 export function selectStrategy(task = {}) {
     if (task.useUserBrowser || config.cdpUrl)
         return "cdp";
     if (task.interactive || task.multiStep)
         return "gateway";
+    return "hub-stealth";
+}
+/**
+ * Resolve the preferred strategy to one that's actually available.
+ * Cascades: gateway → cdp → hub-stealth → cli
+ */
+export async function resolveStrategy(preferred) {
+    const chain = [];
+    // Build fallback chain starting from preferred
+    switch (preferred) {
+        case "gateway":
+            chain.push("gateway", "cdp", "hub-stealth", "cli");
+            break;
+        case "cdp":
+            chain.push("cdp", "hub-stealth", "cli");
+            break;
+        case "hub-stealth":
+            chain.push("hub-stealth", "cli");
+            break;
+        case "cli":
+            chain.push("cli");
+            break;
+    }
+    for (const strategy of chain) {
+        switch (strategy) {
+            case "gateway":
+                if (isGatewayScriptPresent() && (await isGatewayRunning()))
+                    return "gateway";
+                if (!isGatewayScriptPresent()) {
+                    log("Gateway unavailable: browse-server.cjs not found. Falling back.");
+                }
+                else {
+                    log("Gateway not running. Falling back.");
+                }
+                break;
+            case "cdp":
+                if (await isCDPAvailable())
+                    return "cdp";
+                // Try starting CDP via hub script
+                if (isHubBrowserAvailable()) {
+                    try {
+                        log("CDP Chrome not running — attempting to start via hub browser.sh...");
+                        execSync(`"${HUB_BROWSER_SH}" cdp start headless`, {
+                            stdio: "pipe",
+                            timeout: 15_000,
+                        });
+                        // Give it a moment to spin up
+                        await new Promise((r) => setTimeout(r, 3000));
+                        if (await isCDPAvailable()) {
+                            log("CDP Chrome started successfully.");
+                            return "cdp";
+                        }
+                    }
+                    catch (err) {
+                        log(`Failed to start CDP Chrome: ${err.message}`);
+                    }
+                }
+                log("CDP unavailable. Falling back.");
+                break;
+            case "hub-stealth":
+                if (isHubBrowserAvailable())
+                    return "hub-stealth";
+                log("Hub browser.sh not found. Falling back to raw Playwright.");
+                break;
+            case "cli":
+                return "cli"; // Always available as last resort
+        }
+    }
     return "cli";
 }
-// ── Gateway Management ────────────────────────────────────────────────
+function execHub(args) {
+    try {
+        const result = execSync(`"${HUB_BROWSER_SH}" ${args}`, {
+            stdio: "pipe",
+            timeout: EXEC_TIMEOUT,
+            env: { ...process.env, PATH: process.env.PATH },
+        });
+        const stdout = result.toString().trim();
+        // Try to parse as JSON (stealth outputs JSON)
+        try {
+            return JSON.parse(stdout);
+        }
+        catch {
+            // Not JSON — return as raw text
+            return { title: "", url: "", raw: stdout };
+        }
+    }
+    catch (err) {
+        const error = err;
+        log(`Hub script failed: ${error.stderr?.toString()?.trim() || error.message}`);
+        return null;
+    }
+}
+// ── Gateway Management ───────────────────────────────────────────────
 let gatewayProcess = null;
-async function gatewayRequest(path, params = {}) {
+async function gatewayRequest(urlPath, params = {}, timeoutMs = 15_000) {
     const query = new URLSearchParams(params).toString();
-    const url = `http://127.0.0.1:${config.browseServerPort}${path}${query ? "?" + query : ""}`;
+    const url = `http://127.0.0.1:${config.browseServerPort}${urlPath}${query ? "?" + query : ""}`;
     return new Promise((resolve, reject) => {
-        http.get(url, (res) => {
+        const req = http.get(url, (res) => {
             let data = "";
-            res.on("data", chunk => data += chunk);
+            res.on("data", (chunk) => (data += chunk));
             res.on("end", () => {
                 try {
                     resolve(JSON.parse(data));
@@ -37,107 +169,270 @@ async function gatewayRequest(path, params = {}) {
                     reject(new Error(`Invalid JSON from gateway: ${data.slice(0, 200)}`));
                 }
             });
-        }).on("error", reject);
+        });
+        req.on("error", reject);
+        req.setTimeout(timeoutMs, () => {
+            req.destroy(new Error(`Gateway request timed out after ${timeoutMs}ms: ${urlPath}`));
+        });
     });
 }
 async function ensureGateway() {
     // Check if already running
-    try {
-        const health = await gatewayRequest("/health");
-        if (health.ok)
-            return true;
-    }
-    catch { /* not running */ }
-    // Start it
-    if (!fs.existsSync(BROWSE_SERVER_SCRIPT))
+    if (await isGatewayRunning())
+        return true;
+    // Try to start it
+    if (!isGatewayScriptPresent()) {
+        log("Cannot start gateway: browse-server.cjs not found.");
         return false;
+    }
     gatewayProcess = spawn("node", [BROWSE_SERVER_SCRIPT, String(config.browseServerPort)], {
         stdio: "pipe",
         detached: false,
     });
-    gatewayProcess.on("exit", () => { gatewayProcess = null; });
+    gatewayProcess.on("exit", () => {
+        gatewayProcess = null;
+    });
     // Wait for startup (max 10s)
     for (let i = 0; i < 20; i++) {
-        await new Promise(r => setTimeout(r, 500));
-        try {
-            const health = await gatewayRequest("/health");
-            if (health.ok)
-                return true;
-        }
-        catch { /* still starting */ }
+        await new Promise((r) => setTimeout(r, 500));
+        if (await isGatewayRunning())
+            return true;
     }
+    log("Gateway failed to start within 10s.");
     return false;
 }
-// ── Unified Operations ────────────────────────────────────────────────
-/** Navigate to URL using best strategy */
+// ── Unified Operations ───────────────────────────────────────────────
+/** Navigate to URL using best available strategy */
 export async function navigate(url, task = {}) {
-    const strategy = selectStrategy(task);
-    if (strategy === "gateway") {
-        await ensureGateway();
-        return gatewayRequest("/navigate", { url });
-    }
-    if (strategy === "cdp") {
-        // CDP: use playwright connectOverCDP
-        const { chromium } = await import("playwright");
-        const browser = await chromium.connectOverCDP(config.cdpUrl);
-        const contexts = browser.contexts();
-        const page = contexts[0]?.pages()[0] || await contexts[0]?.newPage() || await browser.newPage();
-        await page.goto(url, { waitUntil: "networkidle", timeout: 30000 });
-        const title = await page.title();
-        return { title, url: page.url() };
-    }
-    // CLI: simple text extraction
-    const text = await extractText(url);
-    return { title: url, url, tree: [text.slice(0, 500)] };
+    const strategy = await resolveStrategy(selectStrategy(task));
+    log(`navigate(${url}) using strategy: ${strategy}`);
+    switch (strategy) {
+        case "gateway": {
+            await ensureGateway();
+            return gatewayRequest("/navigate", { url });
+        }
+        case "cdp": {
+            // Try hub CDP first
+            if (isHubBrowserAvailable()) {
+                const result = execHub(`cdp goto "${url}"`);
+                if (result && !result.error) {
+                    return { title: result.title || "", url: result.url || url };
+                }
+            }
+            // Fallback: direct Playwright CDP
+            try {
+                const { chromium } = await import("playwright");
+                const browser = await chromium.connectOverCDP(config.cdpUrl || `http://127.0.0.1:${CDP_PORT}`);
+                const contexts = browser.contexts();
+                const page = contexts[0]?.pages()[0] || (await contexts[0]?.newPage()) || (await browser.newPage());
+                await page.goto(url, { waitUntil: "networkidle", timeout: 30000 });
+                const title = await page.title();
+                return { title, url: page.url() };
+            }
+            catch (err) {
+                log(`Direct CDP failed: ${err.message}`);
+                // Last resort: try stealth
+                if (isHubBrowserAvailable()) {
+                    const stealthResult = execHub(`stealth "${url}"`);
+                    if (stealthResult) {
+                        return { title: stealthResult.title || "", url: stealthResult.url || url };
+                    }
+                }
+                throw err;
+            }
+        }
+        case "hub-stealth": {
+            const result = execHub(`stealth "${url}"`);
+            if (result && !result.error) {
+                return { title: result.title || "", url: result.url || url };
+            }
+            // Fallback to raw CLI
+            log("Hub stealth failed, falling back to raw Playwright.");
+            const text = await extractText(url);
+            return { title: url, url, tree: [text.slice(0, 500)] };
+        }
+        case "cli":
+        default: {
+            const text = await extractText(url);
+            return { title: url, url, tree: [text.slice(0, 500)] };
+        }
+    }
 }
 /** Take a screenshot */
 export async function screenshot(url, options = {}) {
-    const strategy = selectStrategy();
-    if (strategy === "gateway") {
-        await ensureGateway();
-        if (url)
-            await gatewayRequest("/navigate", { url });
-        const result = await gatewayRequest("/screenshot", options.fullPage ? { full: "true" } : {});
-        return result.path;
-    }
-    // CLI fallback
-    return screenshotUrl(url, { fullPage: options.fullPage });
-}
-/** Get accessibility tree (gateway only) */
+    const strategy = await resolveStrategy(selectStrategy());
+    log(`screenshot(${url}) using strategy: ${strategy}`);
+    switch (strategy) {
+        case "gateway": {
+            await ensureGateway();
+            if (url)
+                await gatewayRequest("/navigate", { url });
+            const result = await gatewayRequest("/screenshot", options.fullPage ? { full: "true" } : {});
+            return result.path;
+        }
+        case "cdp": {
+            if (isHubBrowserAvailable()) {
+                const tmpName = `shot_${Date.now()}.png`;
+                const result = execHub(`cdp shot "${url}" ${tmpName}`);
+                if (result?.screenshot)
+                    return result.screenshot;
+            }
+            // Fallback to raw Playwright
+            return screenshotUrl(url, { fullPage: options.fullPage });
+        }
+        case "hub-stealth": {
+            const tmpName = `shot_${Date.now()}.png`;
+            const result = execHub(`stealth "${url}" --screenshot=${tmpName}`);
+            if (result?.screenshot)
+                return result.screenshot;
+            // Fallback
+            return screenshotUrl(url, { fullPage: options.fullPage });
+        }
+        case "cli":
+        default:
+            return screenshotUrl(url, { fullPage: options.fullPage });
+    }
+}
+// ── CDP Direct-Playwright Helper ─────────────────────────────────────
+// Used as fallback when the gateway isn't running but CDP Chrome is.
+// Each call opens a short-lived CDP connection, operates on the newest
+// existing page in the current context (keeps Chrome itself alive), and
+// disconnects. Safe for sub-agents that need a single op at a time.
+async function withCdpPage(fn) {
+    const { chromium } = await import("playwright");
+    const browser = await chromium.connectOverCDP(config.cdpUrl || `http://127.0.0.1:${CDP_PORT}`);
+    try {
+        const context = browser.contexts()[0];
+        if (!context)
+            throw new Error("No CDP contexts available — is Chrome CDP running?");
+        const pages = context.pages();
+        const page = pages[pages.length - 1] || (await context.newPage());
+        return await fn(page);
+    }
+    finally {
+        await browser.close(); // Closes CDP connection, not Chrome itself
+    }
+}
+const NEEDS_INTERACTIVE_HINT = "Start CDP Chrome: ~/.claude/hub/SCRIPTS/browser.sh cdp start headless";
+/**
+ * Get accessibility tree (gateway preferred, CDP fallback returns outerHTML).
+ * The @eN ref model only exists in the gateway; under CDP we return a
+ * best-effort DOM snippet instead so callers can still see what's there.
+ */
 export async function getTree(limit = 100) {
-    await ensureGateway();
-    return gatewayRequest("/tree", { limit: String(limit) });
-}
-/** Click element by ref (gateway only) */
-export async function click(ref) {
-    await ensureGateway();
-    return gatewayRequest("/click", { ref });
-}
-/** Fill input (gateway only) */
-export async function fill(ref, value) {
-    await ensureGateway();
-    await gatewayRequest("/fill", { ref, value });
-}
-/** Type text (gateway only) */
-export async function type(ref, text) {
-    await ensureGateway();
-    await gatewayRequest("/type", { ref, text });
-}
-/** Press key (gateway only) */
-export async function press(key, ref) {
-    await ensureGateway();
-    await gatewayRequest("/press", ref ? { key, ref } : { key });
-}
-/** Scroll page (gateway only) */
+    if (await isGatewayRunning()) {
+        return gatewayRequest("/tree", { limit: String(limit) });
+    }
+    if (await isCDPAvailable()) {
+        return withCdpPage(async (page) => {
+            const elements = await page.$$eval("a, button, input, select, textarea, [role=button], [role=link]", (els, max) => els.slice(0, max).map((el, i) => {
+                const tag = el.tagName.toLowerCase();
+                const text = (el.textContent || "").trim().slice(0, 60);
+                const id = el.id ? `#${el.id}` : "";
+                const name = el.name
+                    ? `[name=${el.name}]`
+                    : "";
+                return `@e${i + 1} <${tag}${id}${name}> "${text}"`;
+            }), limit);
+            return { tree: elements, total: elements.length };
+        });
+    }
+    throw new Error(`[browser-manager] getTree requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
+}
+/**
+ * Click an element. Accepts a gateway ref (@eN → "eN") when gateway is
+ * running, or a CSS selector when only CDP is available.
+ */
+export async function click(refOrSelector) {
+    if (await isGatewayRunning()) {
+        return gatewayRequest("/click", { ref: refOrSelector });
+    }
+    if (await isCDPAvailable()) {
+        return withCdpPage(async (page) => {
+            await page.click(refOrSelector, { timeout: 10_000 });
+            return { title: await page.title(), url: page.url() };
+        });
+    }
+    throw new Error(`[browser-manager] click() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
+}
+/** Fill an input. refOrSelector semantics match click(). */
+export async function fill(refOrSelector, value) {
+    if (await isGatewayRunning()) {
+        await gatewayRequest("/fill", { ref: refOrSelector, value });
+        return;
+    }
+    if (await isCDPAvailable()) {
+        await withCdpPage(async (page) => {
+            await page.fill(refOrSelector, value, { timeout: 10_000 });
+        });
+        return;
+    }
+    throw new Error(`[browser-manager] fill() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
+}
+/** Type text character-by-character (for inputs that reject page.fill). */
+export async function type(refOrSelector, text) {
+    if (await isGatewayRunning()) {
+        await gatewayRequest("/type", { ref: refOrSelector, text });
+        return;
+    }
+    if (await isCDPAvailable()) {
+        await withCdpPage(async (page) => {
+            await page.type(refOrSelector, text, { timeout: 10_000 });
+        });
+        return;
+    }
+    throw new Error(`[browser-manager] type() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
+}
+/** Press a keyboard key (page-level if no ref, element-level with ref). */
+export async function press(key, refOrSelector) {
+    if (await isGatewayRunning()) {
+        await gatewayRequest("/press", refOrSelector ? { key, ref: refOrSelector } : { key });
+        return;
+    }
+    if (await isCDPAvailable()) {
+        await withCdpPage(async (page) => {
+            if (refOrSelector) {
+                await page.locator(refOrSelector).press(key, { timeout: 10_000 });
+            }
+            else {
+                await page.keyboard.press(key);
+            }
+        });
+        return;
+    }
+    throw new Error(`[browser-manager] press() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
+}
+/** Scroll page. CDP fallback uses window.scrollBy. */
 export async function scroll(direction, amount = 600) {
-    await ensureGateway();
-    return gatewayRequest("/scroll", { direction, amount: String(amount) });
+    if (await isGatewayRunning()) {
+        return gatewayRequest("/scroll", { direction, amount: String(amount) });
+    }
+    if (await isCDPAvailable()) {
+        return withCdpPage(async (page) => {
+            const delta = direction === "up" ? -amount :
+                direction === "top" ? -1e9 :
+                    direction === "bottom" ? 1e9 :
+                        amount;
+            await page.evaluate((d) => window.scrollBy(0, d), delta);
+            return { title: await page.title(), url: page.url() };
+        });
+    }
+    throw new Error(`[browser-manager] scroll() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
 }
-/** Evaluate JS (gateway only) */
+/** Evaluate JS in the page context. */
 export async function evaluate(js) {
-    await ensureGateway();
-    const result = await gatewayRequest("/eval", { js });
-    return result.result;
+    if (await isGatewayRunning()) {
+        const result = await gatewayRequest("/eval", { js });
+        return result.result;
+    }
+    if (await isCDPAvailable()) {
+        return withCdpPage(async (page) => {
+            // `page.evaluate(fn)` wraps a function — we need eval of a raw
+            // expression string, so wrap in an IIFE.
+            return page.evaluate(new Function(`return (${js})`));
+        });
+    }
+    throw new Error(`[browser-manager] evaluate() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
 }
 /** Generate PDF from URL */
 export async function pdf(url) {
@@ -154,8 +449,16 @@ export async function close() {
         gatewayProcess = null;
     }
 }
-/** Get current page info (gateway) */
+/** Get current page info (gateway preferred, CDP fallback reads newest page). */
 export async function info() {
-    await ensureGateway();
-    return gatewayRequest("/info");
+    if (await isGatewayRunning()) {
+        return gatewayRequest("/info");
+    }
+    if (await isCDPAvailable()) {
+        return withCdpPage(async (page) => ({
+            title: await page.title(),
+            url: page.url(),
+        }));
+    }
+    throw new Error(`[browser-manager] info() requires gateway or CDP. ${NEEDS_INTERACTIVE_HINT}`);
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "alvin-bot",
-  "version": "4.8.8",
+  "version": "4.8.9",
   "description": "Alvin Bot — Your personal AI agent on Telegram, WhatsApp, Discord, Signal, and Web.",
   "type": "module",
   "main": "dist/index.js",

package/skills/browse/SKILL.md

@@ -1,136 +1,161 @@
 ---
 name: Browser Automation
-description: Interactive browser control — navigate, click, fill forms, screenshot, test web apps
-triggers: browse, browser, test webapp, test app, test website, screenshot page, interact with, click on, fill form, visual test, qa test, check page, open page, test my app, browse to, open url, puppeteer, playwright, browser automation, test die seite, teste die app, schau dir an, öffne die seite, teste mal, visual check, check the ui, check the page
+description: 3-tier browser control — stealth scraping, CDP with persistent cookies, visual oversight. Navigate, screenshot, extract text, interact with logged-in pages.
+triggers: browse, browser, test webapp, test app, test website, screenshot page, interact with, click on, fill form, visual test, qa test, check page, open page, test my app, browse to, open url, puppeteer, playwright, browser automation, linkedin, stepstone, indeed, scrape, fetch page, crawl, teste die seite, teste die app, schau dir an, öffne die seite, teste mal, visual check, check the ui, check the page, webseite öffnen, seite abrufen
 priority: 8
 category: automation
 ---
 
-# Browser Automation — Playwright Interactive
+# Browser Automation — 3-Tier Router
 
-## Browser Strategies
+Du hast drei Browser-Strategien plus WebFetch. **Wähle die billigste passende Stufe** und eskaliere nur wenn nötig.
 
-Alvin Bot auto-selects the best browser approach:
+## Entscheidungsregel (in dieser Reihenfolge)
 
-| Strategy | When | How |
-|----------|------|-----|
-| **CLI** (default) | Simple screenshots, text extraction, PDF | Headless Playwright, one-shot |
-| **HTTP Gateway** | Interactive browsing, form-filling, QA testing | Persistent browser server on port 3800 |
-| **CDP** | Attach to user's Chrome (with login state) | Chrome DevTools Protocol via CDP_URL |
+| Task | Tool | Warum |
+|------|------|-------|
+| Einzelne öffentliche Seite, nur Text | WebFetch oder `curl` | Am schnellsten, keine Browser-Engine |
+| Öffentliche Seite mit JS / Cloudflare | **Tier 1 Stealth** | Headless + Fingerprint-Masking |
+| Login-pflichtige Seite (LinkedIn, Gmail, …) | **Tier 2 CDP** | Echtes Chrome, persistente Cookies |
+| Komplexer Multi-Step-Flow, User soll zusehen | **Tier 3 Extension** | Visuelle Kontrolle |
 
-The gateway starts automatically when needed and shuts down after 5 min idle.
-For CDP: Launch Chrome with `--remote-debugging-port=9222` and set `CDP_URL=http://localhost:9222`.
+**NIEMALS** `scripts/browse-server.cjs` nutzen — existiert nicht mehr. **NIEMALS** nacktes `node -e "const {chromium}…"` für externe Seiten — wird sofort geblockt.
 
 ---
 
-You have a persistent Playwright browser server that gives you **eyes** and **hands** to interact with web pages. You can navigate, see screenshots, read the accessibility tree, click buttons, fill forms, and test running web apps.
+## Tier 0 — WebFetch / curl (schnellster Pfad)
 
-## Quick Start
+Für statische Seiten oder APIs, die keine JS-Rendering brauchen:
 
 ```bash
-# 1. Ensure server is running (auto-shuts down after 5 min idle)
-curl -s http://127.0.0.1:3800/health 2>/dev/null | grep -q '"ok":true' || \
-  (BOT_DIR=$(node -e "console.log(require('path').resolve(require.resolve('alvin-bot/package.json'), '..'))" 2>/dev/null || echo ".") && cd "$BOT_DIR" && node scripts/browse-server.cjs &) && sleep 3
+# Direkter curl
+curl -sL -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36" \
+  "https://www.michaelpage.de/jobs/it-director" | htmlq -t "h1, .job-title"
 
-# 2. Navigate to a page
-curl -s "http://127.0.0.1:3800/navigate?url=https://example.com" | jq
+# Oder das WebFetch-Tool, wenn verfügbar (interpretiert Inhalt direkt)
+```
 
-# 3. Take a screenshot (view it with Read tool)
-SHOT=$(curl -s "http://127.0.0.1:3800/screenshot" | jq -r '.path')
-# Then use Read tool on $SHOT to see the image
+Wenn das einen 403/Captcha gibt → eskaliere auf Tier 1.
 
-# 4. Get interactive elements
-curl -s "http://127.0.0.1:3800/tree" | jq '.tree[]' -r
+---
 
-# 5. Click something
-curl -s "http://127.0.0.1:3800/click?ref=e5" | jq
-```
+## Tier 1 — Playwright Stealth (headless, schnell, maskiert)
+
+**Router-Script:** `~/.claude/hub/SCRIPTS/browser.sh`
+
+```bash
+# Seite laden, JSON-Metadata zurück (title, url, html_length)
+~/.claude/hub/SCRIPTS/browser.sh stealth "https://www.stepstone.de/jobs/it-delivery"
 
-## All Routes
-
-| Route | Params | What it does |
-|-------|--------|-------------|
-| `/navigate` | `url` | Open a URL, returns title + accessibility tree |
-| `/screenshot` | `full=true` (optional) | Take screenshot, returns file path |
-| `/tree` | `limit=N` (optional) | Get all interactive elements with @eN refs |
-| `/click` | `ref=eN` | Click element by ref |
-| `/fill` | `ref=eN`, `value=text` | Fill input field |
-| `/type` | `ref=eN`, `text=chars` | Type character by character (for special inputs) |
-| `/press` | `key=Enter`, `ref=eN` (opt) | Press keyboard key |
-| `/select` | `ref=eN`, `value=opt` | Select dropdown option |
-| `/hover` | `ref=eN` | Hover over element |
-| `/scroll` | `direction=down/up/top/bottom`, `amount=600` | Scroll page |
-| `/eval` | `js=expression` | Run JavaScript on page |
-| `/wait` | `ms=2000` or `selector=.class` | Wait for time or element |
-| `/viewport` | `device=mobile/tablet` or `width=W&height=H` | Change viewport |
-| `/cookies` | `set=[{...}]` (optional) | Get or set cookies |
-| `/back` | — | Browser back |
-| `/forward` | — | Browser forward |
-| `/reload` | — | Reload page |
-| `/network` | `limit=20` | Recent network requests |
-| `/info` | — | Current page info |
-| `/close` | — | Close browser + shutdown server |
-| `/health` | — | Server status check |
-
-## Element Refs (@eN)
-
-The accessibility tree assigns **refs** like `@e1`, `@e2`, `@e3` to every interactive element (links, buttons, inputs, etc.). Use these refs for all interactions — they're more robust than CSS selectors.
-
-Example tree:
+# Mit Screenshot (PNG in ~/.claude/hub/BROWSER/screenshots/)
+~/.claude/hub/SCRIPTS/browser.sh stealth "https://example.com" --screenshot=page.png
 ```
-@e1 <a href="/"> "Home"
-@e2 <a href="/dashboard"> "Dashboard"
-@e3 <input type="email" name="email" placeholder="Enter email">
-@e4 <input type="password" name="password" placeholder="Password">
-@e5 <button> "Sign In"
-@e6 <a href="/forgot"> "Forgot password?"
+
+**Was du bekommst:** JSON mit `{title, url, html_length, screenshot}`. Der volle HTML liegt nicht in stdout — zum Parsen den `stealth.js` direkt als Modul importieren oder `/tmp/`-File lesen.
+
+**Wann blockt das:** reCAPTCHA v3, aggressive Cloudflare, Login-Walls.
+
+**Konkrete funktionierende Targets (Stand 2026):**
+- StepStone (alle Job-Suchen) ✅
+- Michael Page ✅
+- Hays ✅
+- Öffentliche Blog-Posts, News-Sites ✅
+- LinkedIn (ohne Login) ❌ → Tier 2
+- Indeed / Glassdoor ❌ (403 Scraping-Block) → nur über E-Mail-Alerts
+
+---
+
+## Tier 2 — Chrome CDP (persistent Profile, echte Cookies)
+
+Echtes Chrome mit Profil unter `~/.claude/hub/BROWSER/profile/`. Login-Cookies für LinkedIn/Gmail/etc. bleiben über Sessions erhalten.
+
+```bash
+# Einmal starten (checkt ob schon läuft)
+~/.claude/hub/SCRIPTS/browser.sh cdp start headless   # headless — für Cron/Daemon
+~/.claude/hub/SCRIPTS/browser.sh cdp start headful    # sichtbar — wenn User zusehen soll
+
+# Navigieren
+~/.claude/hub/SCRIPTS/browser.sh cdp goto "https://www.linkedin.com/jobs/search/?keywords=IT+Director"
+
+# Screenshot
+~/.claude/hub/SCRIPTS/browser.sh cdp shot "https://www.linkedin.com/feed/" linkedin_feed.png
+
+# Tabs auflisten
+~/.claude/hub/SCRIPTS/browser.sh cdp tabs
+
+# Stoppen (meistens nicht nötig, Chrome läuft persistent)
+~/.claude/hub/SCRIPTS/browser.sh cdp stop
 ```
 
-To login:
+**Login-Setup (einmalig):** Falls LinkedIn ausgeloggt ist, Ali per Telegram fragen:
+> "Bitte einmal in Chrome (Hub-Profil) bei LinkedIn einloggen. Cookies bleiben dann dauerhaft erhalten."
+
+Starten mit `cdp start headful` und Chrome öffnet sichtbar → Ali loggt ein → ab dann bleiben Cookies im Profil.
+
+**Wie teste ich ob eingeloggt:** nach `cdp goto` die URL prüfen — wenn `/authwall` oder `/login` im Pfad steht, bist du ausgeloggt.
+
+---
+
+## Tier 3 — Claude-in-Chrome Extension (visuelle Kontrolle)
+
+Nur in interaktiven CLI-Sessions, nicht im Cron/Daemon.
+
 ```bash
-curl -s "http://127.0.0.1:3800/fill?ref=e3&value=user@example.com"
-curl -s "http://127.0.0.1:3800/fill?ref=e4&value=mypassword"
-curl -s "http://127.0.0.1:3800/click?ref=e5"
+# Check ob Extension verbunden
+~/.claude/hub/SCRIPTS/browser.sh ext check
+
+# Dann MCP-Tools über ToolSearch laden:
+#   mcp__claude-in-chrome__tabs_context_mcp
+#   mcp__claude-in-chrome__navigate
+#   mcp__claude-in-chrome__computer
 ```
 
-## Standard Workflow: Test a Web App
+**Wann nutzen:** Drag&Drop, komplexe UI, User soll live zusehen und eingreifen können.
 
-1. **Start** the browse server if not running
-2. **Navigate** to the app URL
-3. **Screenshot** → view with Read tool to see current state
-4. **Tree** → see all interactive elements
-5. **Interact** (click, fill, press) using @eN refs
-6. **Screenshot** again to verify the result
-7. **Repeat** for each test step
-8. **Report** findings to the user
-9. **Close** when done
+---
 
-## Mobile Testing
+## Eskalations-Regel (PFLICHT)
+
+```
+Öffentliche Text-Seite → Tier 0 (WebFetch/curl)
+  ↓ 403/Cloudflare/leerer HTML?
+Tier 1 (stealth) → browser.sh stealth <url>
+  ↓ Captcha/Login-Wall?
+Tier 2 (CDP) → cdp start headless/headful + cdp goto <url>
+  ↓ Cookies fehlen?
+Ali fragen: "Bitte einmal in Chrome bei X einloggen, dann kann ich weitermachen."
+```
+
+**NIEMALS aufgeben mit "Browser funktioniert nicht"** — es gibt immer einen nächsten Schritt. Lieber ehrlich melden "Tier 1 blockt mit Captcha, versuche Tier 2" als "Failed to load".
+
+## Status-Checks
 
 ```bash
-# Switch to mobile viewport
-curl -s "http://127.0.0.1:3800/viewport?device=mobile"
-curl -s "http://127.0.0.1:3800/screenshot" | jq -r '.path'
-# Switch back to desktop
-curl -s "http://127.0.0.1:3800/viewport?width=1280&height=720"
+# Übersicht aller Tiers + Health
+~/.claude/hub/SCRIPTS/browser.sh status
+
+# Ist CDP Chrome gerade auf Port 9222?
+curl -s http://127.0.0.1:9222/json/version | head -c 200
 ```
 
-## Auth / Cookie Injection
+## Screenshot-Ausgabe ansehen
+
+Screenshots werden gespeichert unter `~/.claude/hub/BROWSER/screenshots/` (relativ) oder dem absoluten Pfad, den du angibst. Read-Tool auf den Pfad zeigt dir das Bild direkt an.
+
+## Interaktive Ops (Klicken, Formular füllen)
+
+Für einfache Fälle: `cdp eval` mit JavaScript, das in der Seite ausgeführt wird:
 
-For pages that need authentication:
 ```bash
-# Set cookies manually
-curl -s 'http://127.0.0.1:3800/cookies?set=[{"name":"session","value":"abc123","domain":"example.com","path":"/"}]'
-# Then navigate to the authenticated page
-curl -s "http://127.0.0.1:3800/navigate?url=https://example.com/dashboard"
+~/.claude/hub/SCRIPTS/browser.sh cdp eval "https://example.com/login" \
+  "document.querySelector('#username').value='test'; document.querySelector('#password').value='pw'; document.querySelector('form').submit();"
 ```
 
-## Important Notes
+Für komplexere Flows (sequentielles Klicken nach DOM-Updates) → Tier 3 (Extension) nutzen.
+
+## Wichtige Notes
 
-- **Server auto-shuts down** after 5 min idle — restart if needed
-- **One page at a time** — navigation replaces the current page
-- **Screenshots** are saved to `/tmp/alvin-bot/browse/` — view with Read tool
-- **127.0.0.1 only** — not accessible from outside
-- **URL-encode** values with special chars: `value=hello%20world`
-- **Refs reset** on every navigation/click — always get fresh /tree after page changes
-- For **local dev servers**: use `http://localhost:PORT` as the URL
+- **CDP-Profil-Konflikt:** Chrome kann `~/.claude/hub/BROWSER/profile/` nicht doppelt öffnen. Wenn Ali es lokal auf hatte, Port 9222 checken und `cdp stop` + `cdp start` machen.
+- **Headless vs Headful:** Im Cron/Daemon (launchd) IMMER `headless` — sonst scheitert Chrome an fehlendem Display.
+- **Nach Seiten-Navigation** (`cdp goto`) neue Tabs legt Playwright standardmäßig an — reuseTab ist nicht exponiert. Das ist OK für einzelne Scrapes, kann aber zu Tab-Explosion führen. `cdp stop` & Neustart räumt auf.
+- **Persistenz:** Cookies, LocalStorage, IndexedDB, alles in `~/.claude/hub/BROWSER/profile/`. Komplett persistiert zwischen Bot-Restarts.