barebrowse 0.7.0 → 0.7.1

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.7.1
+
+Fix: timeout now triggers auto-retry instead of bypassing it.
+
+### Bug fix (`mcp-server.js`)
+- **Root cause:** The 30s timeout was a `Promise.race` *outside* `withRetry()`. When a page timed out, the race rejected immediately — `withRetry` never got a chance to reset the session and retry. Timeouts also didn't match `isCdpDead()`, so even if they did reach `withRetry`, they wouldn't be retried.
+- **Fix:** Moved per-attempt timeout *inside* `withRetry()`. Each attempt gets its own 30s deadline. On timeout or CDP death, the session resets and a fresh attempt runs. The outer `Promise.race` is removed entirely.
+- `isCdpDead()` renamed to `isTransient()` — now also matches timeout errors (`"Timeout waiting for CDP event"`, `"timed out"`)
+- Non-transient errors (validation, unknown tool) are still not retried
+
+### Tests
+- 11 new unit tests in `test/unit/mcp.test.js`: `isTransient` detection (CDP death, timeouts, non-transient), `withRetry` behavior (success, CDP retry, timeout retry, no-retry for validation, double-failure, no-timeout mode)
+- 80 total tests (39 unit + 41 integration)
+
 ## 0.7.0
 
 MCP resilience: timeouts, auto-retry, LLM-friendly scroll, and click fallback for hidden elements.

package/barebrowse.context.md

@@ -1,7 +1,7 @@
 # barebrowse -- Integration Guide
 
 > For AI assistants and developers wiring barebrowse into a project.
-> v0.7.0 | Node.js >= 22 | 0 required deps | MIT
+> v0.7.1 | Node.js >= 22 | 0 required deps | MIT
 
 ## What this is
 
@@ -241,7 +241,7 @@ Action tools return `'ok'` -- the agent calls `snapshot` explicitly to observe.
 
 Session runs in hybrid mode (headless with automatic headed fallback on bot detection). `goto` injects cookies from the user's browser before navigation for authenticated access.
 
-Session tools share a singleton page, lazy-created on first use. All session tools have auto-retry on transient CDP failures (browser crash, WebSocket close) — session resets and retries once automatically. 30s timeout on all tools (60s for `browse`/`assess`). Scroll accepts `direction: "up"/"down"` in addition to numeric `deltaY`. Click falls back to JS `.click()` when elements have no layout. Assess tries headless first; if bot-blocked, retries headed. Browser OOM/crash auto-recovers (session resets, server stays alive).
+Session tools share a singleton page, lazy-created on first use. All session tools have auto-retry on transient failures (browser crash, WebSocket close, navigation timeout) — each attempt gets its own 30s deadline, session resets between attempts, retries once automatically. Scroll accepts `direction: "up"/"down"` in addition to numeric `deltaY`. Click falls back to JS `.click()` when elements have no layout. `browse` has a 60s timeout (no retry — stateless). Assess tries headless first; if bot-blocked, retries headed. Browser OOM/crash auto-recovers (session resets, server stays alive).
 
 ## Architecture
 

package/mcp-server.js

@@ -20,20 +20,37 @@ try {
 } catch {}
 
 
-function isCdpDead(err) {
+function isTransient(err) {
   const m = err.message || '';
-  return m.includes('WebSocket') || m.includes('Target closed') || m.includes('Session closed') || m.includes('CDP');
+  return m.includes('WebSocket') || m.includes('Target closed') || m.includes('Session closed')
+    || m.includes('CDP') || m.includes('Timeout waiting for CDP event') || m.includes('timed out');
 }
 
-/** Retry-once wrapper for transient CDP failures. Resets session and retries. */
-async function withRetry(fn) {
+/**
+ * Retry-once wrapper with per-attempt timeout.
+ * On transient failure (CDP death OR timeout), resets session and retries once.
+ * @param {Function} fn - async function to execute
+ * @param {number} timeoutMs - per-attempt timeout in ms
+ */
+async function withRetry(fn, timeoutMs) {
+  async function attempt() {
+    if (!timeoutMs) return await fn();
+    let timer;
+    const result = await Promise.race([
+      fn(),
+      new Promise((_, rej) => { timer = setTimeout(() => rej(new Error(`timed out after ${timeoutMs / 1000}s`)), timeoutMs); }),
+    ]);
+    clearTimeout(timer);
+    return result;
+  }
+
   try {
-    return await fn();
+    return await attempt();
   } catch (err) {
-    if (!isCdpDead(err)) throw err;
-    // CDP died — reset session and retry once
+    if (!isTransient(err)) throw err;
+    // Transient failure — reset session and retry once
     _page = null;
-    return await fn();
+    return await attempt();
   }
 }
 
@@ -226,7 +243,12 @@ if (assessFn) {
 async function handleToolCall(name, args) {
   switch (name) {
     case 'browse': {
-      const text = await browse(args.url, { mode: args.mode });
+      let timer;
+      const text = await Promise.race([
+        browse(args.url, { mode: args.mode }),
+        new Promise((_, rej) => { timer = setTimeout(() => rej(new Error('browse timed out after 60s')), 60000); }),
+      ]);
+      clearTimeout(timer);
       const limit = args.maxChars ?? MAX_CHARS_DEFAULT;
       if (text.length > limit) {
         const file = saveSnapshot(text);
@@ -239,7 +261,7 @@ async function handleToolCall(name, args) {
       try { await page.injectCookies(args.url); } catch {}
       await page.goto(args.url);
       return 'ok';
-    });
+    }, 30000);
     case 'snapshot': return withRetry(async () => {
       const page = await getPage();
       const text = await page.snapshot();
@@ -249,22 +271,22 @@ async function handleToolCall(name, args) {
         return `Snapshot (${text.length} chars) saved to ${file}`;
       }
       return text;
-    });
+    }, 30000);
     case 'click': return withRetry(async () => {
       const page = await getPage();
       await page.click(args.ref);
       return 'ok';
-    });
+    }, 30000);
     case 'type': return withRetry(async () => {
       const page = await getPage();
       await page.type(args.ref, args.text, { clear: args.clear });
       return 'ok';
-    });
+    }, 30000);
     case 'press': return withRetry(async () => {
       const page = await getPage();
       await page.press(args.key);
       return 'ok';
-    });
+    }, 30000);
     case 'scroll': return withRetry(async () => {
       const page = await getPage();
       let dy = args.deltaY;
@@ -276,31 +298,31 @@ async function handleToolCall(name, args) {
       }
       await page.scroll(dy);
       return 'ok';
-    });
+    }, 30000);
     case 'back': return withRetry(async () => {
       const page = await getPage();
       await page.goBack();
       return 'ok';
-    });
+    }, 30000);
     case 'forward': return withRetry(async () => {
       const page = await getPage();
       await page.goForward();
       return 'ok';
-    });
+    }, 30000);
     case 'drag': return withRetry(async () => {
       const page = await getPage();
       await page.drag(args.fromRef, args.toRef);
       return 'ok';
-    });
+    }, 30000);
     case 'upload': return withRetry(async () => {
       const page = await getPage();
       await page.upload(args.ref, args.files);
       return 'ok';
-    });
+    }, 30000);
     case 'pdf': return withRetry(async () => {
       const page = await getPage();
       return await page.pdf({ landscape: args.landscape });
-    });
+    }, 30000);
     case 'assess': {
       if (!assessFn) throw new Error('wearehere is not installed. Run: npm install wearehere');
       const releaseSlot = await acquireAssessSlot();
@@ -342,7 +364,7 @@ async function handleToolCall(name, args) {
         } catch (err) {
           clearTimeout(timer);
           await tab.close().catch(() => {});
-          if (isCdpDead(err)) _page = null;
+          if (isTransient(err)) _page = null;
           throw err;
         }
       } finally {
@@ -369,7 +391,7 @@ async function handleMessage(msg) {
     return jsonrpcResponse(id, {
       protocolVersion: '2024-11-05',
       capabilities: { tools: {} },
-      serverInfo: { name: 'barebrowse', version: '0.7.0' },
+      serverInfo: { name: 'barebrowse', version: '0.7.1' },
     });
   }
 
@@ -383,19 +405,13 @@ async function handleMessage(msg) {
 
   if (method === 'tools/call') {
     const { name, arguments: args } = params;
-    const TOOL_TIMEOUT = name === 'browse' || name === 'assess' ? 60000 : 30000;
     try {
-      let timer;
-      const result = await Promise.race([
-        handleToolCall(name, args || {}),
-        new Promise((_, rej) => { timer = setTimeout(() => rej(new Error(`Tool "${name}" timed out after ${TOOL_TIMEOUT / 1000}s`)), TOOL_TIMEOUT); }),
-      ]);
-      clearTimeout(timer);
+      const result = await handleToolCall(name, args || {});
       return jsonrpcResponse(id, {
         content: [{ type: 'text', text: typeof result === 'string' ? result : JSON.stringify(result) }],
       });
     } catch (err) {
-      if (isCdpDead(err)) _page = null;
+      if (isTransient(err)) _page = null;
       return jsonrpcResponse(id, {
         content: [{ type: 'text', text: `Error: ${err.message}` }],
         isError: true,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "barebrowse",
-  "version": "0.7.0",
+  "version": "0.7.1",
   "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
   "type": "module",
   "main": "src/index.js",

package/.aurora/plans/active/harden-assess/design.md

@@ -1,68 +0,0 @@
-# Design: Harden assess tool
-
-## Architecture
-
-### Current flow (broken)
-```
-assess(url) → connect({ mode: 'hybrid' })  ← NEW browser, no cookies
-            → assessFn(page, url)
-            → page.close()                  ← kills browser
-```
-
-### New flow
-```
-assess(url) → getPage()                     ← reuse session browser
-            → page.createTab()              ← new tab in same browser
-            → tab.injectCookies(url)        ← cookie injection
-            → assessFn(tab, url)            ← assess uses tab
-            → tab.close()                   ← close tab only
-            timeout guard wraps entire flow
-            retry wraps entire flow
-```
-
-## Key design decisions
-
-### Why a new tab, not the session page?
-wearehere's `assess()` calls:
-- `session.send('Page.addScriptToEvaluateOnNewDocument', ...)` — injects fingerprint detection scripts
-- `networkSession.on('Network.requestWillBeSent', ...)` — monitors all network traffic
-
-These would pollute the session page. A separate tab has its own CDP session with isolated Page/Network domains.
-
-### createTab() page-like interface
-wearehere expects a page object with: `goto()`, `cdp` (raw session), `waitForNetworkIdle()`. createTab() returns exactly this interface:
-
-```javascript
-{
-  goto(url, timeout)        // navigate tab
-  cdp                       // raw CDP session for this tab
-  waitForNetworkIdle(opts)  // reuses existing waitForNetworkIdle()
-  injectCookies(url)        // cookie injection for this tab
-  close()                   // close tab, NOT the browser
-}
-```
-
-### Retry strategy
-```
-attempt 1: assess with 45s timeout
-  fail → wait 2s
-attempt 2: assess with 45s timeout (if browser crashed, reset _page first)
-  fail → return { error: "assessment_failed", ... }
-```
-
-### Timeout implementation
-```javascript
-const result = await Promise.race([
-  doAssess(page, url, opts),
-  new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 45000))
-]);
-```
-On timeout, the tab is closed in a finally block.
-
-## Files changed
-| File | Change |
-|------|--------|
-| `src/index.js` | Add `createTab()` and tab's `close()` to connect() return object |
-| `mcp-server.js` | Rewrite assess case: use getPage().createTab(), add retry + timeout |
-| `README.md` | Update assess description, add self-healing mention |
-| `barebrowse.context.md` | Update assess section, document createTab() |

package/.aurora/plans/active/harden-assess/plan.md

@@ -1,71 +0,0 @@
-# Plan: Harden assess tool — session reuse + self-healing
-
-## Plan ID
-`harden-assess`
-
-## Summary
-Make the assess tool reuse the MCP session's browser instance (cookies, headed fallback) instead of spawning throwaway browsers, add retry logic for transient failures, and add a timeout guard so no single assessment can hang forever.
-
-## Problem Statement
-The `assess` tool in `mcp-server.js` calls `connect({ mode: 'hybrid' })` directly — creating a fresh headless browser per call with no cookies, no session state, and no headed fallback. Every other MCP tool uses the `getPage()` singleton. This causes:
-
-1. **No cookies** — assess browses as a stranger, getting blocked by consent walls and bot detection that cookies would bypass
-2. **No headed fallback reuse** — if the singleton already fell back to headed mode, assess still starts fresh headless and hits the same blocks
-3. **No retry** — any failure (browser crash, navigation timeout, CDP disconnect) kills the assessment with no recovery
-4. **No timeout guard** — if `wearehere`'s `assess()` hangs (e.g. network idle never resolves), the MCP call blocks indefinitely
-
-However, assess can't simply use the shared `_page` singleton directly because `wearehere` injects init scripts (`addScriptToEvaluateOnNewDocument`) and network listeners that would pollute the session page for subsequent calls. The solution is to create a **second page tab** within the same browser instance.
-
-## Proposed Solution
-1. **Session-aware page creation** — Instead of `connect()`, assess opens a new CDP tab within the existing browser. This shares the browser process (same cookies, same headed/headless state) but isolates assess's script injections to its own tab.
-2. **Retry with backoff** — Wrap the assess call in a retry loop (max 2 attempts, 2s backoff). On browser crash, reconnect the singleton.
-3. **Timeout guard** — Wrap each assess call in `Promise.race` with a 45s hard deadline. If exceeded, return an error result (not hang).
-
-## Benefits
-- Assess gets cookies and headed fallback for free — no separate browser instance
-- Failed assessments auto-retry instead of dying
-- Hanging assessments time out gracefully instead of blocking the MCP server forever
-- Eliminates the 10+ second cold-start per assessment (browser launch)
-
-## Scope
-### In Scope
-- Modify `mcp-server.js` assess handler to create tabs within existing browser
-- Add `createTab()` / `closeTab()` helper to `src/index.js` connect() page handle
-- Add retry wrapper in mcp-server.js
-- Add timeout guard in mcp-server.js
-- Update docs: README.md, barebrowse.context.md, CLAUDE.md
-
-### Out of Scope
-- Changing wearehere's internal logic
-- Adding retry/self-healing to other MCP tools (future work)
-- Batch/queue mode for multiple assessments
-- Changing the assess tool's MCP interface (same inputs/outputs)
-
-## Dependencies
-- `wearehere` package (assess function signature unchanged)
-- `src/index.js` connect() API (adding createTab/closeTab methods)
-- `src/cdp.js` (Target.createTarget / closeTarget already available)
-
-## Implementation Strategy
-Phase 1: Add tab management to connect() page handle (createTab, closeTab)
-Phase 2: Rewrite assess handler to use session tab + retry + timeout
-Phase 3: Update documentation
-
-## Risks and Mitigations
-| Risk | Impact | Mitigation |
-|------|--------|------------|
-| Init script injection leaks across tabs | Pollutes session page | Each tab gets its own Page domain; addScriptToEvaluateOnNewDocument is per-target |
-| Browser crash during assess kills session too | Session page lost | getPage() already handles reconnection lazily (set _page = null, next call recreates) |
-| wearehere expects full page handle, not raw CDP session | API mismatch | createTab() returns a page-like object with goto, cdp, waitForNetworkIdle — same interface |
-
-## Success Criteria
-- [ ] `assess` reuses the session browser (no separate `connect()` call)
-- [ ] `assess` inherits cookies from the session
-- [ ] `assess` works when session is in headed mode (hybrid fallback already triggered)
-- [ ] Failed assessment retries once before returning error
-- [ ] Assessment hanging > 45s returns timeout error, doesn't block server
-- [ ] All existing tests pass
-- [ ] Documentation updated
-
-## Open Questions
-1. Should createTab() also inject cookies? — **Recommendation**: Yes, call `authenticate()` for the target URL before navigation, same as `goto` does.

package/.aurora/plans/active/harden-assess/prd.md

@@ -1,38 +0,0 @@
-# PRD: Harden assess tool
-
-## Overview
-The `assess` MCP tool must reuse the session browser, retry on failure, and time out gracefully.
-
-## Requirements
-
-### R1: Session-aware tab creation
-The assess tool MUST create a new browser tab within the existing MCP session browser instead of spawning a separate browser via `connect()`. The tab MUST:
-- Share the same browser process (inheriting headless/headed state)
-- Have access to the browser's cookie jar
-- Isolate its CDP domains (Page, Network, DOM) from the session page
-- Be closed after each assessment completes or fails
-
-### R2: Cookie injection
-Before navigating, the assess tab MUST inject cookies from the user's browser for the target URL, using the same `authenticate()` mechanism as `goto`.
-
-### R3: Retry on failure
-If an assessment fails (navigation timeout, CDP error, browser crash), the tool MUST:
-- Retry once after a 2-second delay
-- If the browser crashed, reset the session singleton (`_page = null`) so getPage() reconnects
-- If retry also fails, return a structured error result (not throw)
-
-### R4: Timeout guard
-Each assessment MUST have a hard timeout of 45 seconds. If exceeded:
-- The tab is force-closed
-- A structured error result is returned: `{ site, url, error: "timeout", scanned_at }`
-- The session page is NOT affected
-
-### R5: Backwards compatibility
-- The assess tool's MCP interface (inputs/outputs) MUST NOT change
-- Successful assessments return the same JSON format as before
-- The tool appears/disappears based on wearehere availability (unchanged)
-
-## Non-functional
-- No new dependencies
-- No changes to wearehere package
-- createTab()/closeTab() exposed on connect() page handle for library users too