browserclaw-agent 0.3.0 → 0.3.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Idan Rubin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,218 @@
+# browserclaw-agent
+
+<p align="center">
+  <a href="https://browserclaw.org"><img src="https://img.shields.io/badge/Live-browserclaw.org-orange" alt="Live" /></a>
+  <a href="https://www.npmjs.com/package/browserclaw-agent"><img src="https://img.shields.io/npm/v/browserclaw-agent.svg" alt="npm" /></a>
+  <a href="./LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT" /></a>
+</p>
+
+The AI agent for [browserclaw](https://github.com/idan-rubin/browserclaw).
+
+## Layered, not bundled
+
+Three separate layers, not one monolith:
+
+- ⚡ **LLM** — the electricity. Your choice: Claude, GPT, Gemini, local. No lock-in.
+- 😎 **The agent** — the driver. Reasoning, obstacle recovery, learned skills. _That's this project._
+- 🏎️ **[BrowserClaw](https://github.com/idan-rubin/browserclaw)** — the vehicle. Snapshots, element refs, browser control. Standalone npm library.
+
+browser-use welds these into one package. We keep them as separate, swappable layers — drop the library into your own agent, pair this agent with any LLM, or run the whole stack as-is.
+
+## vs browser-use
+
+**Different lineage, different design.** [OpenClaw](https://openclaw.ai) took the [Playwright MCP](https://github.com/microsoft/playwright-mcp) approach — Microsoft's snapshot-and-ref pattern — implemented it locally, and refined it into [browserclaw](https://github.com/idan-rubin/browserclaw), a standalone npm library. This agent rides on that library. browser-use rolled its own Python stack as one bundled package.
+
+|                             |    browserclaw     |    browser-use     |
+| :-------------------------- | :----------------: | :----------------: |
+| Standalone browser library  | :white_check_mark: |        :x:         |
+| Works with any agent        | :white_check_mark: | :heavy_minus_sign: |
+| Auto-learns skills per site | :white_check_mark: |        :x:         |
+| Solves anti-bot in OSS      | :white_check_mark: | :heavy_minus_sign: |
+| TypeScript / Node           | :white_check_mark: |        :x:         |
+
+:white_check_mark: = Yes&ensp; :heavy_minus_sign: = Partial&ensp; :x: = No
+
+## What the agent does
+
+The agent reads an accessibility snapshot of the page, decides what to do next, and executes the action. Up to 100 steps per run. It maintains a memory scratchpad across steps and evaluates whether each action succeeded before deciding the next move.
+
+```
+snapshot → LLM → action → repeat
+```
+
+### Built-in skills
+
+When the agent hits common obstacles, built-in skills take over automatically — no prompting needed:
+
+- **Anti-bot bypass** — Detects and solves "hold to verify" overlays and press-and-hold challenges via CDP
+- **Cloudflare Turnstile** — Solves "Verify you are human" checkboxes by locating and clicking the Turnstile iframe via CDP
+- **Popup dismissal** — Closes cookie banners, consent dialogs, and modals using multi-strategy detection
+- **Loop detection** — Detects when the agent is stuck repeating the same action and nudges it toward a different approach
+- **Tab manager** — Detects and switches to new tabs opened during automation
+
+### Skill catalog
+
+Every successful run generates a skill file — steps and tips for that domain, stored in MinIO. On the next run against the same domain, the agent loads the skill as a playbook instead of exploring from scratch. If the new run completes in fewer steps, the skill is replaced. One domain, one skill, always improving.
+
+The first user to automate a domain pays the exploration cost. Every subsequent run benefits from the learned playbook — and refines it further.
+
+## Quick start
+
+### Local (dev mode)
+
+Chrome opens on your desktop. No containers, no VNC.
+
+**Requires:** Node.js 22+, Chrome installed
+
+```bash
+cd src/Services/Browser
+cp .env.example .env.local
+# Set LLM_PROVIDER and at least one API key (see LLM providers below)
+npm install
+npm run dev
+```
+
+Start a run:
+
+```bash
+curl -X POST http://localhost:5040/api/v1/sessions \
+  -H "Content-Type: application/json" \
+  -d '{"prompt": "Find apartments in NYC under $3000"}'
+```
+
+Stream progress:
+
+```bash
+curl http://localhost:5040/api/v1/sessions/{id}/stream
+```
+
+### Docker (full stack)
+
+Runs the frontend, browser service (headless Chrome + VNC), MinIO (skill storage), and Traefik. Same setup as [browserclaw.org](https://browserclaw.org).
+
+```bash
+git clone https://github.com/idan-rubin/browserclaw-agent.git
+cd browserclaw-agent
+cp src/Services/Browser/.env.example src/Services/Browser/.env.local
+# Set LLM_PROVIDER and at least one API key
+# Set BROWSER_INTERNAL_TOKEN to a random secret (used for frontend → browser service auth):
+echo "BROWSER_INTERNAL_TOKEN=$(openssl rand -hex 32)" >> .env
+docker compose up
+```
+
+Open [localhost](http://localhost).
+
+### LLM providers
+
+Add at least one API key to `.env.local` and set `LLM_PROVIDER`:
+
+| Provider                      | Env var              | `LLM_PROVIDER` | Free tier         |
+| ----------------------------- | -------------------- | -------------- | ----------------- |
+| Groq                          | `GROQ_API_KEY`       | `groq`         | Yes               |
+| Google Gemini                 | `GEMINI_API_KEY`     | `gemini`       | Yes               |
+| OpenAI                        | `OPENAI_API_KEY`     | `openai`       | No                |
+| OpenAI (ChatGPT subscription) | `OPENAI_OAUTH_TOKEN` | `openai-oauth` | No (subscription) |
+| Anthropic                     | `ANTHROPIC_API_KEY`  | `anthropic`    | No                |
+
+Set `LLM_MODEL` to override the default model for your provider.
+
+### Other features
+
+- **BYOK** — Users can pass their own LLM API key per session for multi-tenant deployments
+- **User interaction** — The agent can pause mid-run to ask for information (MFA codes, credentials)
+- **SSE streaming** — Real-time step-by-step progress events
+- **Content moderation** — Rejects harmful prompts before execution
+- **SSRF protection** — Private network access blocked by default
+
+## Bring your own agent
+
+Don't want this agent? Use the [browserclaw](https://github.com/idan-rubin/browserclaw) library directly with any LLM.
+
+```bash
+npm install browserclaw
+```
+
+Requires Chrome, Brave, Edge, or Chromium installed on your machine.
+
+```typescript
+import { BrowserClaw } from "browserclaw";
+
+const browser = await BrowserClaw.launch({ headless: false });
+const page = await browser.open("https://example.com");
+
+const { snapshot, refs } = await page.snapshot();
+// snapshot: text tree of the page
+// refs: { "e1": { role: "link", name: "More info" }, ... }
+
+await page.click("e1");
+await page.type("e3", "hello");
+await browser.stop();
+```
+
+`snapshot()` returns a text representation of the page with numbered refs. Pass it to any LLM, get back a ref, call the action. Here's a minimal agent loop:
+
+```typescript
+import { BrowserClaw } from "browserclaw";
+import Anthropic from "@anthropic-ai/sdk";
+
+const client = new Anthropic();
+const browser = await BrowserClaw.launch({ headless: false });
+const page = await browser.open("https://news.ycombinator.com");
+const history = [];
+
+for (let step = 0; step < 20; step++) {
+  const { snapshot } = await page.snapshot();
+  const response = await client.messages.create({
+    model: "claude-sonnet-4-20250514",
+    max_tokens: 1024,
+    system:
+      "You control a browser. Given a page snapshot, return JSON: { action, ref?, text?, url?, reasoning }. Actions: click, type, navigate, done.",
+    messages: [
+      ...history,
+      {
+        role: "user",
+        content: `Task: Find the top 3 AI posts.\n\nPage:\n${snapshot}`,
+      },
+    ],
+  });
+
+  const action = JSON.parse(response.content[0].text);
+  history.push(
+    { role: "user", content: `Page:\n${snapshot}` },
+    { role: "assistant", content: JSON.stringify(action) },
+  );
+  if (action.action === "done") break;
+
+  switch (action.action) {
+    case "click":
+      await page.click(action.ref);
+      break;
+    case "type":
+      await page.type(action.ref, action.text);
+      break;
+    case "navigate":
+      await page.goto(action.url);
+      break;
+  }
+}
+await browser.stop();
+```
+
+Swap Anthropic for OpenAI, Groq, Gemini, or a local model. See the full [browserclaw API docs](https://github.com/idan-rubin/browserclaw) for `fill()`, `select()`, `drag()`, `screenshot()`, `pdf()`, `waitFor()`, and more.
+
+## Why browserclaw?
+
+- **Built for TypeScript** — native to the JS ecosystem. First-class Node.js support, not a Python port.
+- **Accessibility tree, not DOM** — snapshots use the browser's accessibility tree — the same structure screen readers use. Semantic roles, names, and states instead of raw tags and attributes. Cleaner, smaller, and more meaningful to an LLM.
+- **Layered, not bundled** — the engine, the agent, and the LLM are separate, swappable pieces. See the [comparison above](#vs-browser-use).
+- **Gets smarter with use** — the skill catalog learns from every successful run. Other browser agents start from scratch each time. browserclaw-agent builds a playbook per domain and improves it on every run.
+- **Handles the real world** — Cloudflare Turnstile, press-and-hold anti-bot overlays, cookie banners, tab management — handled automatically via CDP. These are the things that make browser agents fail in production.
+
+## Read more
+
+- [The Intelligence Gap](https://mrrubin.substack.com/p/the-knowledge-gap-why-ai-browser) — why AI browser agents keep failing, and what we're doing about it
+
+## Built with
+
+- [BrowserClaw](https://github.com/idan-rubin/browserclaw) — the browser automation library
+- [OpenClaw](https://github.com/openclaw/openclaw) — the community behind it

package/dist/advisor.d.ts

@@ -0,0 +1,8 @@
+import type { AgentLoopResult } from './types.js';
+interface AdvisorReview {
+    success: boolean;
+    reasoning: string;
+}
+export declare function advisorReview(prompt: string, result: AgentLoopResult): Promise<AdvisorReview>;
+export {};
+//# sourceMappingURL=advisor.d.ts.map

package/dist/advisor.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"advisor.d.ts","sourceRoot":"","sources":["../src/advisor.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,YAAY,CAAC;AAGlD,UAAU,aAAa;IACrB,OAAO,EAAE,OAAO,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;CACnB;AA2CD,wBAAsB,aAAa,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,eAAe,GAAG,OAAO,CAAC,aAAa,CAAC,CAanG"}

package/dist/{judge.js → advisor.js}

@@ -1,8 +1,8 @@
 import { llmJson } from './llm.js';
 import { logger } from './logger.js';
-const JUDGE_PROMPT = `You are a strict evaluator of browser automation runs. Your job is to determine whether the agent actually completed its task successfully.
+const ADVISOR_PROMPT = `You are a quality advisor for browser automation runs. You review the agent's completed trace and flag quality issues as advisory notes — you do NOT gate success.
 
-Be initially doubtful of the agent's self-reported success. Check for:
+Review for:
 1. Did the agent's actions actually achieve the goal, or did it just claim success?
 2. Is the answer grounded in data from actual page snapshots, or does it contain fabricated/hallucinated data?
 3. Did the agent get blocked by a login wall, paywall, CAPTCHA, or error page and report success anyway?
@@ -12,11 +12,11 @@ Be initially doubtful of the agent's self-reported success. Check for:
 Respond with JSON:
 {
   "success": true/false,
-  "reasoning": "why you believe the run succeeded or failed"
+  "reasoning": "one-paragraph quality note — what you'd tell a reviewer to check"
 }
 
-Set success=false if ANY of the above checks fail.`;
-function buildJudgeMessage(prompt, result) {
+Set success=false when you'd want a human reviewer to double-check the answer. Callers treat this as advisory metadata, not a blocking verdict.`;
+function buildAdvisorMessage(prompt, result) {
     let message = `Task: ${prompt}\n`;
     message += `Agent reported: ${result.success ? 'SUCCESS' : 'FAILURE'}\n`;
     if (result.answer !== undefined) {
@@ -39,19 +39,19 @@ function buildJudgeMessage(prompt, result) {
     }
     return message;
 }
-export async function judgeRun(prompt, result) {
+export async function advisorReview(prompt, result) {
     try {
-        const verdict = await llmJson({
-            system: JUDGE_PROMPT,
-            message: buildJudgeMessage(prompt, result),
+        const review = await llmJson({
+            system: ADVISOR_PROMPT,
+            message: buildAdvisorMessage(prompt, result),
             maxTokens: 256,
         });
-        logger.info({ judge_ran: true, verdict_success: verdict.success }, 'Judge verdict');
-        return verdict;
+        logger.info({ advisor_ran: true, quality_ok: review.success }, 'Advisor review');
+        return review;
     }
     catch {
-        logger.warn({ judge_failed: true }, 'Judge evaluation failed — defaulting to agent result');
-        return { success: result.success, reasoning: 'Judge evaluation failed' };
+        logger.warn({ advisor_failed: true }, 'Advisor review failed — treating run as clean');
+        return { success: true, reasoning: 'Advisor review unavailable' };
     }
 }
-//# sourceMappingURL=judge.js.map
+//# sourceMappingURL=advisor.js.map

package/dist/advisor.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"advisor.js","sourceRoot":"","sources":["../src/advisor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,UAAU,CAAC;AAEnC,OAAO,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAOrC,MAAM,cAAc,GAAG;;;;;;;;;;;;;;;gJAeyH,CAAC;AAEjJ,SAAS,mBAAmB,CAAC,MAAc,EAAE,MAAuB;IAClE,IAAI,OAAO,GAAG,SAAS,MAAM,IAAI,CAAC;IAClC,OAAO,IAAI,mBAAmB,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,SAAS,IAAI,CAAC;IACzE,IAAI,MAAM,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;QAChC,OAAO,IAAI,iBAAiB,MAAM,CAAC,MAAM,IAAI,CAAC;IAChD,CAAC;IACD,IAAI,MAAM,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;QAC/B,OAAO,IAAI,gBAAgB,MAAM,CAAC,KAAK,IAAI,CAAC;IAC9C,CAAC;IACD,OAAO,IAAI,sBAAsB,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,YAAY,CAAC;IAEzE,KAAK,MAAM,IAAI,IAAI,MAAM,CAAC,KAAK,EAAE,CAAC;QAChC,IAAI,IAAI,GAAG,UAAU,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,IAAI,CAAC,MAAM,CAAC,MAAM,KAAK,IAAI,CAAC,MAAM,CAAC,SAAS,EAAE,CAAC;QAC3F,IAAI,IAAI,CAAC,GAAG,KAAK,SAAS;YAAE,IAAI,IAAI,KAAK,IAAI,CAAC,GAAG,GAAG,CAAC;QACrD,IAAI,IAAI,CAAC,MAAM,CAAC,cAAc,KAAK,SAAS;YAAE,IAAI,IAAI,cAAc,IAAI,CAAC,MAAM,CAAC,cAAc,EAAE,CAAC;QACjG,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,KAAK,SAAS,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,KAAK,EAAE,EAAE,CAAC;YAClE,IAAI,IAAI,iBAAiB,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAAC;QAClE,CAAC;QACD,OAAO,IAAI,GAAG,IAAI,IAAI,CAAC;IACzB,CAAC;IAED,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,MAAc,EAAE,MAAuB;IACzE,IAAI,CAAC;QACH,MAAM,MAAM,GAAG,MAAM,OAAO,CAAgB;YAC1C,MAAM,EAAE,cAAc;YACtB,OAAO,EAAE,mBAAmB,CAAC,MAAM,EAAE,MAAM,CAAC;YAC5C,SAAS,EAAE,GAAG;SACf,CAAC,CAAC;QACH,MAAM,CAAC,IAAI,CAAC,EAAE,WAAW,EAAE,IAAI,EAAE,UAAU,EAAE,MAAM,CAAC,OAAO,EAAE,EAAE,gBAAgB,CAAC,CAAC;QACjF,OAAO,MAAM,CAAC;IAChB,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,CAAC,IAAI,CAAC,EAAE,cAAc,EAAE,IAAI,EAAE,EAAE,+CAA+C,CAAC,CAAC;QACvF,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,4BAA4B,EAAE,CAAC;IACpE,CAAC;AACH,CAAC"}

package/dist/agent-loop.d.ts

@@ -45,6 +45,7 @@ export interface AgentLoopOptions {
     } | undefined>>;
     buildTask?: (prompt: string) => string | Promise<string>;
     getLesson?: (prompt: string) => Promise<TaskLesson | null>;
+    getSkill?: (domain: string) => Promise<CatalogSkill | null>;
     config?: Partial<AgentConfig>;
 }
 export interface UserChatHooks {
@@ -53,5 +54,5 @@ export interface UserChatHooks {
     /** Per-session random token used to wrap user-interjection content so the LLM can't confuse user data for system instructions. */
     nonce: string;
 }
-export declare function runAgentLoop(prompt: string, pageOrHolder: CrawlPage | PageHolder, emit: (event: string, data: unknown) => void, signal: AbortSignal, waitForUser?: () => Promise<string>, browser?: BrowserClaw, domainSkill?: CatalogSkill | null, maxSteps?: number, userChat?: UserChatHooks, options?: AgentLoopOptions): Promise<AgentLoopResult>;
+export declare function runAgentLoop(prompt: string, pageOrHolder: CrawlPage | PageHolder, emit: (event: string, data: unknown) => void, signal: AbortSignal, waitForUser?: () => Promise<string>, browser?: BrowserClaw, initialDomainSkill?: CatalogSkill | null, maxSteps?: number, userChat?: UserChatHooks, options?: AgentLoopOptions): Promise<AgentLoopResult>;
 //# sourceMappingURL=agent-loop.d.ts.map

package/dist/agent-loop.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"agent-loop.d.ts","sourceRoot":"","sources":["../src/agent-loop.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAW1D,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAG9C,OAAO,KAAK,EAAE,WAAW,EAAa,eAAe,EAAiB,YAAY,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AACnH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAY/C;;;;;;GAMG;AACH,wBAAgB,sBAAsB,CAAC,QAAQ,EAAE,WAAW,EAAE,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAiC5F;AAirBD,wBAAgB,wBAAwB,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAsB1D;AAMD,wBAAgB,8BAA8B,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAIvE;AA2PD,MAAM,MAAM,mBAAmB,GAAG;IAAE,KAAK,EAAE,IAAI,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GAAG;IAAE,KAAK,EAAE,KAAK,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,CAAC;AAEtG;;;GAGG;AACH,wBAAgB,sBAAsB,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAG5D;AA2CD,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,SAAS,CAAC;CACjB;AAED;;;;;;;;;;GAUG;AACH,MAAM,WAAW,gBAAgB;IAC/B,YAAY,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,aAAa,EAAE,MAAM,KAAK,MAAM,CAAC,CAAC;IAC5D,aAAa,CAAC,EAAE,MAAM,CACpB,MAAM,EACN,CAAC,MAAM,EAAE,WAAW,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,CAAC,EAAE,WAAW,KAAK,OAAO,CAAC;QAAE,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,SAAS,CAAC,CAC3G,CAAC;IACF,SAAS,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC;IACzD,SAAS,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,OAAO,CAAC,UAAU,GAAG,IAAI,CAAC,CAAC;IAC3D,MAAM,CAAC,EAAE,OAAO,CAAC,WAAW,CAAC,CAAC;CAC/B;AAED,MAAM,WAAW,aAAa;IAC5B,4EAA4E;IAC5E,aAAa,EAAE,MAAM,WAAW,EAAE,CAAC;IACnC,kIAAkI;IAClI,KAAK,EAAE,MAAM,CAAC;CACf;AAED,wBAAsB,YAAY,CAChC,MAAM,EAAE,MAAM,EACd,YAAY,EAAE,SAAS,GAAG,UAAU,EACpC,IAAI,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,KAAK,IAAI,EAC5C,MAAM,EAAE,WAAW,EACnB,WAAW,CAAC,EAAE,MAAM,OAAO,CAAC,MAAM,CAAC,EACnC,OAAO,CAAC,EAAE,WAAW,EACrB,WAAW,CAAC,EAAE,YAAY,GAAG,IAAI,EACjC,QAAQ,CAAC,EAAE,MAAM,EACjB,QAAQ,CAAC,EAAE,aAAa,EACxB,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,eAAe,CAAC,CA2/B1B"}
+{"version":3,"file":"agent-loop.d.ts","sourceRoot":"","sources":["../src/agent-loop.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAa1D,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAG9C,OAAO,KAAK,EAAE,WAAW,EAAa,eAAe,EAAiB,YAAY,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AACnH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAY/C;;;;;;GAMG;AACH,wBAAgB,sBAAsB,CAAC,QAAQ,EAAE,WAAW,EAAE,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAiC5F;AAgrBD,wBAAgB,wBAAwB,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAsB1D;AAMD,wBAAgB,8BAA8B,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAIvE;AA8QD,MAAM,MAAM,mBAAmB,GAAG;IAAE,KAAK,EAAE,IAAI,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GAAG;IAAE,KAAK,EAAE,KAAK,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,CAAC;AAEtG;;;GAGG;AACH,wBAAgB,sBAAsB,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAG5D;AA2CD,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,SAAS,CAAC;CACjB;AAED;;;;;;;;;;GAUG;AACH,MAAM,WAAW,gBAAgB;IAC/B,YAAY,CAAC,EAAE,MAAM,GAAG,CAAC,CAAC,aAAa,EAAE,MAAM,KAAK,MAAM,CAAC,CAAC;IAC5D,aAAa,CAAC,EAAE,MAAM,CACpB,MAAM,EACN,CAAC,MAAM,EAAE,WAAW,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,CAAC,EAAE,WAAW,KAAK,OAAO,CAAC;QAAE,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,SAAS,CAAC,CAC3G,CAAC;IACF,SAAS,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC;IACzD,SAAS,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,OAAO,CAAC,UAAU,GAAG,IAAI,CAAC,CAAC;IAC3D,QAAQ,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,OAAO,CAAC,YAAY,GAAG,IAAI,CAAC,CAAC;IAC5D,MAAM,CAAC,EAAE,OAAO,CAAC,WAAW,CAAC,CAAC;CAC/B;AAED,MAAM,WAAW,aAAa;IAC5B,4EAA4E;IAC5E,aAAa,EAAE,MAAM,WAAW,EAAE,CAAC;IACnC,kIAAkI;IAClI,KAAK,EAAE,MAAM,CAAC;CACf;AAED,wBAAsB,YAAY,CAChC,MAAM,EAAE,MAAM,EACd,YAAY,EAAE,SAAS,GAAG,UAAU,EACpC,IAAI,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,KAAK,IAAI,EAC5C,MAAM,EAAE,WAAW,EACnB,WAAW,CAAC,EAAE,MAAM,OAAO,CAAC,MAAM,CAAC,EACnC,OAAO,CAAC,EAAE,WAAW,EACrB,kBAAkB,CAAC,EAAE,YAAY,GAAG,IAAI,EACxC,QAAQ,CAAC,EAAE,MAAM,EACjB,QAAQ,CAAC,EAAE,aAAa,EACxB,OAAO,CAAC,EAAE,gBAAgB,GACzB,OAAO,CAAC,eAAe,CAAC,CAw/B1B"}

package/dist/agent-loop.js

@@ -7,6 +7,8 @@ import { diagnoseStuckAgent, formatRecovery } from './skills/recovery.js';
 import { TabManager } from './skills/tab-manager.js';
 import { getCdpBaseUrl, activateCdpTarget } from './skills/cdp-utils.js';
 import { extractItems, extractItemsFromUrls } from './skills/extract-items.js';
+import { extractDomain } from './skill-store.js';
+import { extractSchema, validateAnswer } from './answer-validator.js';
 import { webSearch } from './web-search.js';
 import { llmJson, llmVision, sanitizeErrorText, getTokenUsage } from './llm.js';
 import { LlmParseError } from './types.js';
@@ -377,7 +379,6 @@ async function isBrowserAlive(page) {
         return false;
     }
 }
-const SKILL_INJECT_MAX_STEP = 2;
 const PLAN_INJECT_MAX_STEP = 8;
 const HISTORY_RECENT_WINDOW = 8;
 const MAX_ACTIONS_PER_STEP = 4;
@@ -789,6 +790,24 @@ function computeStateSig(url, snapshotText) {
     }
     return `${String(payload.length)}:${String(h)}`;
 }
+const REF_FAILURE_KEYWORDS = [
+    'not found',
+    'no element',
+    'intercept',
+    'covered',
+    'obscured',
+    'detach',
+    'stale',
+    'disabled',
+    'not interactable',
+];
+function isRefFailure(action, err) {
+    if (action.action !== 'click' && action.action !== 'type' && action.action !== 'select')
+        return false;
+    const raw = err instanceof Error ? err.message : '';
+    const lower = raw.toLowerCase();
+    return REF_FAILURE_KEYWORDS.some((kw) => lower.includes(kw));
+}
 function describeActionError(action, err) {
     const raw = err instanceof Error ? err.message : 'Action execution failed';
     const lower = raw.toLowerCase();
@@ -943,7 +962,7 @@ Use ONLY data that appears in the memory or steps below — never invent values.
         return { ready: false, missing: 'judgment unavailable' };
     }
 }
-export async function runAgentLoop(prompt, pageOrHolder, emit, signal, waitForUser, browser, domainSkill, maxSteps, userChat, options) {
+export async function runAgentLoop(prompt, pageOrHolder, emit, signal, waitForUser, browser, initialDomainSkill, maxSteps, userChat, options) {
     const cfg = defaultAgentConfig(options?.config);
     maxSteps ??= cfg.maxSteps;
     // Accept either a bare CrawlPage or a PageHolder. When a PageHolder is
@@ -997,7 +1016,8 @@ export async function runAgentLoop(prompt, pageOrHolder, emit, signal, waitForUs
             final_url: currentUrl,
         };
     };
-    // ── Load task lessons ──────────────────────────────────────────────────────
+    let domainSkill = initialDomainSkill ?? null;
+    let answerSchema = null;
     let taskLesson = null;
     if (options?.getLesson !== undefined) {
         try {
@@ -1012,7 +1032,6 @@ export async function runAgentLoop(prompt, pageOrHolder, emit, signal, waitForUs
             logger.error({ err }, 'Failed to load task lessons');
         }
     }
-    // ── Planning + goal refinement (single LLM call) ──────────────────────────
     // If the prompt is vague (e.g. "find apartments in Chelsea"), the planner
     // also produces a SMART task with clear scope and stopping criteria.
     let refinedPrompt = prompt;
@@ -1020,7 +1039,7 @@ export async function runAgentLoop(prompt, pageOrHolder, emit, signal, waitForUs
     try {
         const taskInput = options?.buildTask !== undefined ? await options.buildTask(prompt) : prompt;
         let planMessage = `User prompt: ${taskInput}`;
-        if (domainSkill !== undefined && domainSkill !== null) {
+        if (domainSkill !== null) {
             planMessage += `\n\nWe have a proven skill for this site: "${domainSkill.skill.title}" — ${domainSkill.skill.description}`;
             planMessage += '\nLeverage it — no need to rediscover what already works.';
         }
@@ -1079,10 +1098,8 @@ Respond with JSON: {"task": "the SMART task", "plan": "your action plan"}`,
     let lastStateSig = null;
     let noOpStreak = 0;
     let baselineModalSigs = null;
+    let lastLoadedSkillDomain = domainSkill?.domain ?? null;
     const seenFilterTokens = new Set();
-    const visitedListingUrls = new Set();
-    let lastListExtractStep = -1;
-    let lastListExtractUrls = [];
     while (step < maxSteps) {
         if (signal.aborted) {
             return {
@@ -1110,6 +1127,27 @@ Respond with JSON: {"task": "the SMART task", "plan": "your action plan"}`,
         let snapshot = await safeSnapshot(holder.page);
         const url = await holder.page.url();
         const title = await holder.page.title();
+        if (options?.getSkill !== undefined) {
+            const currentDomain = extractDomain(url);
+            if (currentDomain !== '' && currentDomain !== lastLoadedSkillDomain) {
+                try {
+                    const loaded = await options.getSkill(currentDomain);
+                    if (loaded !== null) {
+                        domainSkill = loaded;
+                        emit('skills_loaded', {
+                            count: 1,
+                            domain: currentDomain,
+                            title: loaded.skill.title,
+                            run_count: loaded.run_count,
+                        });
+                    }
+                }
+                catch (err) {
+                    logger.warn({ domain: currentDomain, err: err instanceof Error ? err.message : err }, 'Lazy skill fetch failed');
+                }
+                lastLoadedSkillDomain = currentDomain;
+            }
+        }
         const domText = await getPageText(holder.page);
         const antiBotType = detectAntiBot(domText);
         if (antiBotType !== null) {
@@ -1127,7 +1165,6 @@ Respond with JSON: {"task": "the SMART task", "plan": "your action plan"}`,
             snapshot = await screenshotFallback(holder.page, snapshot);
         }
         emit('thinking', { step, message: `Analyzing page: ${title}` });
-        // --- Context compression ---
         if (step > 0 && step % CONTEXT_COMPRESS_INTERVAL === 0) {
             contextSummary = await compressContext(refinedPrompt, history, lastProgress);
             if (contextSummary !== '') {
@@ -1135,7 +1172,6 @@ Respond with JSON: {"task": "the SMART task", "plan": "your action plan"}`,
                 emit('context_compressed', { step, summary_length: contextSummary.length });
             }
         }
-        // --- Domain change detection (proactive replanning trigger) ---
         let domainChanged = false;
         try {
             const currentDomain = new URL(url).hostname;
@@ -1147,7 +1183,6 @@ Respond with JSON: {"task": "the SMART task", "plan": "your action plan"}`,
         catch {
             // URL may not be parseable (about:blank etc.)
         }
-        // --- Re-planning checkpoint ---
         // Uses adaptive intervals: extends when healthy, contracts on failures.
         // Also triggers on domain changes (proactive).
         const atReplanCheckpoint = step > 0 && step >= nextReplanStep;
@@ -1194,7 +1229,6 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                 logger.warn('Re-planning failed');
             }
         }
-        // --- Recovery diagnosis (only when the agent is struggling) ---
         let recoveryMessage = null;
         if (step > 0 && step % 4 === 0 && recentFailureCount > 0) {
             const recovery = diagnoseStuckAgent(history, url);
@@ -1299,7 +1333,7 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                 logger.warn({ error: sanitizeErrorText(err instanceof Error ? err.message : 'unknown') }, 'Failed to get tab count');
             }
         }
-        const skillForStep = step <= SKILL_INJECT_MAX_STEP ? domainSkill : undefined;
+        const skillForStep = extractDomain(url) === domainSkill?.domain ? domainSkill : undefined;
         const lessonForStep = step <= PLAN_INJECT_MAX_STEP ? taskLesson : null;
         // Keep plan available for longer — it's cheap context and prevents drift
         const planForStep = step <= PLAN_INJECT_MAX_STEP ? planText : null;
@@ -1485,22 +1519,6 @@ Respond with JSON: {"plan": "your revised plan here"}`,
             }
             continue;
         }
-        if (lastListExtractUrls.length >= 5 && step - lastListExtractStep <= 3 && actions[0]?.action === 'click') {
-            const urls = lastListExtractUrls.filter((u) => !visitedListingUrls.has(u)).slice(0, 8);
-            if (urls.length >= 3) {
-                logger.warn({ step, count: urls.length }, 'batch-urls preempt');
-                actions = [
-                    {
-                        action: 'extract',
-                        reasoning: `Batch-fetching ${String(urls.length)} unseen candidate detail pages in parallel.`,
-                        urls,
-                    },
-                ];
-                for (const u of urls)
-                    visitedListingUrls.add(u);
-                lastListExtractUrls = [];
-            }
-        }
         // Execute batch of actions sequentially
         for (let actionIdx = 0; actionIdx < actions.length; actionIdx++) {
             if (step >= maxSteps)
@@ -1526,7 +1544,6 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                 page_title: agentStep.page_title,
             });
             emit('tokens', getTokenUsage());
-            // --- Terminal actions ---
             if (action.action === 'done') {
                 const doneBlockReason = shouldBlockDone(pageState, history.length, action.answer);
                 if (doneBlockReason !== null) {
@@ -1535,12 +1552,21 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                     step++;
                     break;
                 }
+                const answerText = action.answer ?? '';
+                answerSchema ??= await extractSchema(prompt);
+                const validation = validateAnswer(answerSchema, answerText);
+                const validationWarnings = validation.valid ? undefined : validation.errors;
+                if (validationWarnings !== undefined) {
+                    logger.info({ step, warnings: validationWarnings }, 'Done has validation warnings');
+                    emit('done_validation_warnings', { step, warnings: validationWarnings });
+                }
                 return {
                     success: true,
                     steps: history,
                     answer: action.answer,
                     duration_ms: Date.now() - startTime,
                     final_url: agentStep.url,
+                    validation_warnings: validationWarnings,
                 };
             }
             if (action.action === 'fail') {
@@ -1580,7 +1606,6 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                 step++;
                 break; // Need new snapshot after user response
             }
-            // --- Special actions (break batch — need new snapshot) ---
             if (action.action === 'press_and_hold') {
                 const solved = await pressAndHold(holder.page, { holdMs: action.hold_ms });
                 if (!solved) {
@@ -1654,18 +1679,6 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                     logger.info({ step, source: items.source, count: items.count }, 'extract items auto');
                     if (items.count > 0)
                         extractProducedData = true;
-                    const urls = [];
-                    for (const r of items.records) {
-                        const u = typeof r.url === 'string' ? r.url : '';
-                        if (u !== '' && !urls.includes(u))
-                            urls.push(u);
-                        if (urls.length >= 10)
-                            break;
-                    }
-                    if (urls.length >= 5) {
-                        lastListExtractUrls = urls;
-                        lastListExtractStep = step;
-                    }
                 }
                 if (extractProducedData)
                     extractsWithDataCount++;
@@ -1735,7 +1748,6 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                 step++;
                 break;
             }
-            // --- Normal actions ---
             const preActionUrl = await holder.page.url();
             try {
                 const customHandler = options?.customActions?.[action.action];
@@ -1813,7 +1825,7 @@ Respond with JSON: {"plan": "your revised plan here"}`,
                 emit('step_error', { step, action: action.action, error: rawMessage });
                 agentStep.action.error_feedback = feedback;
                 recentFailureCount++;
-                if (action.ref !== undefined && action.ref !== '') {
+                if (action.ref !== undefined && action.ref !== '' && isRefFailure(action, err)) {
                     const entry = bannedRefs.get(action.ref) ?? { action: action.action, failures: 0 };
                     entry.failures += 1;
                     entry.action = action.action;