open-agents-ai 0.103.94 → 0.103.96

package/README.md

@@ -231,10 +231,10 @@ On startup and `/model` switch, Open Agents detects your RAM/VRAM and creates an
 | `task_output` | Read background task output |
 | `task_stop` | Stop a background task |
 | **Web** | |
-| `web_search` | Search the web (DuckDuckGo, Tavily, Jina AI — auto-detected) |
-| `web_fetch` | Fetch and extract text from web pages (HTML stripping) |
-| `web_crawl` | Multi-page web scraping with Crawlee/Playwright for deep documentation |
-| `browser_action` | Headless Chrome automation: navigate, click, type, screenshot, read DOM, scroll, history |
+| `web_search` | Search the web for pages matching a query — returns links+snippets, not content. Providers: DuckDuckGo (free), Tavily (TAVILY_API_KEY), Jina (JINA_API_KEY) |
+| `web_fetch` | Fetch a single URL's text content (fastest, no JS rendering). Supports `mode=reader` for Jina Reader markdown output with JS rendering. Auto-fallback to Jina when raw content is too short |
+| `web_crawl` | Crawl pages with link-following and optional JS rendering. Strategies: `beautifulsoup` (fast HTTP) or `playwright` (headless Chromium). Supports `extract_schema` for structured data extraction |
+| `browser_action` | Interactive headless Chrome: login, fill forms, click buttons, screenshot. Session persists between calls. Actions: navigate, click, click_xy, type, screenshot, dom, scroll, back, forward, close |
 | **Structured Data** | |
 | `structured_file` | Generate CSV, TSV, JSON, Markdown tables, Excel-compatible files |
 | `structured_read` | Parse CSV, TSV, JSON, Markdown tables with binary format detection |
@@ -287,6 +287,27 @@ On startup and `/model` switch, Open Agents detects your RAM/VRAM and creates an
 
 Read-only tools execute concurrently when called in the same turn. Mutating tools run sequentially.
 
+### Web Tool Selection Guide
+
+The agent has 4 web tools. Pick the right one:
+
+| Need | Tool | Why |
+|------|------|-----|
+| Find pages about a topic | `web_search` | Returns links+snippets to fetch later |
+| Read a URL you already have | `web_fetch` | Fastest — plain text, no JS rendering |
+| Page is blank or JS-heavy (SPA) | `web_crawl` strategy=playwright | Renders JavaScript via headless Chromium |
+| Follow links across a site | `web_crawl` max_depth=1+ | Multi-page crawl with metadata |
+| Extract structured data (prices, tables) | `web_crawl` + extract_schema | Regex-based field extraction from page text |
+| Login / fill forms / click buttons | `browser_action` | Persistent session with cookies and state |
+| Screenshot of a rendered page | `browser_action` action=screenshot | Visual rendering via Chrome |
+| Clean markdown from any URL | `web_fetch` mode=reader | Jina Reader (r.jina.ai) — handles JS, images |
+
+**Routing order**: `web_search` (find) → `web_fetch` (read) → `web_crawl` (if JS/multi-page) → `browser_action` (if interactive)
+
+**Jina Reader**: Set `JINA_API_KEY` for higher rate limits. Works without a key for basic use. When `web_fetch` gets very short content (<200 chars), it automatically retries via Jina Reader.
+
+**Structured extraction**: Pass `extract_schema='{"price": "number", "name": "string"}'` to `web_crawl` for best-effort regex-based field extraction from page content.
+
 ## Ralph Loop — Iteration-First Design
 
 The Ralph Loop is the core execution philosophy: **iteration beats perfection**. Instead of trying to get everything right on the first attempt, the agent executes in a retry loop where errors become learning data rather than session-ending failures.

package/dist/index.js

@@ -1895,14 +1895,16 @@ var init_glob_find = __esm({
 });
 
 // packages/execution/dist/tools/web-fetch.js
-var DEFAULT_MAX_LENGTH, WebFetchTool;
+var DEFAULT_MAX_LENGTH, JINA_READER_BASE, SHORT_CONTENT_THRESHOLD, WebFetchTool;
 var init_web_fetch = __esm({
   "packages/execution/dist/tools/web-fetch.js"() {
     "use strict";
     DEFAULT_MAX_LENGTH = 5e3;
+    JINA_READER_BASE = "https://r.jina.ai/";
+    SHORT_CONTENT_THRESHOLD = 200;
     WebFetchTool = class {
       name = "web_fetch";
-      description = "Fetch a web page and return its text content. Useful for reading documentation from sites like w3schools.com, MDN, or Stack Overflow.";
+      description = "Fetch a single web page and return its text content (HTML stripped to plain text). FASTEST web tool \u2014 use this for reading any single URL: documentation, articles, README files, API references, Stack Overflow answers. Limitations: no JavaScript rendering (SPAs/React apps return empty), no link following, no cookies/auth, no structured data extraction. If the page is blank or incomplete, switch to web_crawl with strategy='playwright'. For scraping/extracting structured data (prices, listings, tables), use web_crawl instead. For search engine queries, use web_search instead. For interactive browser sessions (login, form filling, clicking), use browser_action instead.";
       parameters = {
         type: "object",
         properties: {
@@ -1913,6 +1915,11 @@ var init_web_fetch = __esm({
           max_length: {
             type: "number",
             description: `Maximum number of characters to return (default: ${DEFAULT_MAX_LENGTH})`
+          },
+          mode: {
+            type: "string",
+            enum: ["raw", "reader"],
+            description: "Fetch mode. 'raw' (default): direct HTTP fetch with HTML stripping. 'reader': use Jina Reader (r.jina.ai) for clean markdown output with JS rendering, image descriptions, and better content extraction. Requires JINA_API_KEY for higher rate limits (works without for basic use)."
           }
         },
         required: ["url"]
@@ -1920,7 +1927,13 @@ var init_web_fetch = __esm({
       async execute(args) {
         const url = args["url"];
         const maxLength = args["max_length"] ?? DEFAULT_MAX_LENGTH;
+        const mode = args["mode"] ?? "raw";
         const start = performance.now();
+        if (mode === "reader") {
+          const result = await this.#fetchViaJina(url, maxLength, start);
+          if (result)
+            return result;
+        }
         try {
           const response = await fetch(url, {
             headers: {
@@ -1939,6 +1952,15 @@ var init_web_fetch = __esm({
           }
           const html = await response.text();
           const text = this.#stripHtml(html);
+          if (text.length < SHORT_CONTENT_THRESHOLD && mode === "raw") {
+            const jinaResult = await this.#fetchViaJina(url, maxLength, start);
+            if (jinaResult && jinaResult.success) {
+              jinaResult.output = `[Auto-fallback to Jina Reader \u2014 raw content was too short (${text.length} chars)]
+
+${jinaResult.output}`;
+              return jinaResult;
+            }
+          }
           const truncated = text.length > maxLength;
           return {
             success: true,
@@ -1956,6 +1978,39 @@ var init_web_fetch = __esm({
           };
         }
       }
+      /**
+       * Fetch via Jina Reader API for clean markdown output with JS rendering.
+       * Returns null on failure so the caller can fall back to raw mode.
+       */
+      async #fetchViaJina(url, maxLength, start) {
+        const jinaUrl = `${JINA_READER_BASE}${url}`;
+        const headers = {
+          Accept: "text/markdown",
+          "X-No-Cache": "true"
+        };
+        const jinaKey = process.env["JINA_API_KEY"];
+        if (jinaKey)
+          headers["Authorization"] = `Bearer ${jinaKey}`;
+        try {
+          const resp = await fetch(jinaUrl, {
+            headers,
+            signal: AbortSignal.timeout(2e4)
+          });
+          if (resp.ok) {
+            const md = await resp.text();
+            const truncated = md.length > maxLength;
+            return {
+              success: true,
+              output: truncated ? `${md.slice(0, maxLength)}
+
+[Truncated to ${maxLength} chars]` : md,
+              durationMs: performance.now() - start
+            };
+          }
+        } catch {
+        }
+        return null;
+      }
       #stripHtml(html) {
         let text = html.replace(/<script\b[^>]*>[\s\S]*?<\/script>/gi, " ");
         text = text.replace(/<style\b[^>]*>[\s\S]*?<\/style>/gi, " ");
@@ -1987,7 +2042,7 @@ var init_web_search = __esm({
     JINA_SEARCH_URL = "https://s.jina.ai/";
     WebSearchTool = class {
       name = "web_search";
-      description = "Search the web for information. Returns search results with titles, URLs, and snippets. Supports DuckDuckGo (free), Tavily (structured, set TAVILY_API_KEY), and Jina AI (markdown, set JINA_API_KEY).";
+      description = "Search the web and return a list of result links with titles and snippets. Use this to search when you need to FIND pages (not read them) \u2014 e.g. 'how to X in Python', 'best library for Y', 'error message Z'. Returns URLs you can then fetch with web_fetch. Does NOT return full page content \u2014 only titles, URLs, and brief snippets. To read a specific URL you already have, use web_fetch instead. Providers: DuckDuckGo (free, default), Tavily (structured + AI answer, needs TAVILY_API_KEY), Jina (markdown results, needs JINA_API_KEY).";
       parameters = {
         type: "object",
         properties: {
@@ -2255,7 +2310,7 @@ var init_web_crawl = __esm({
     DEFAULT_MAX_LENGTH2 = 8e3;
     WebCrawlTool = class {
       name = "web_crawl";
-      description = "Advanced web scraping with multi-page crawling support. Two strategies: 'beautifulsoup' (fast HTTP-based, good for static content like docs/articles) and 'playwright' (headless browser, for JS-rendered SPAs and dynamic pages). Supports link following, depth control, and structured output. Use web_fetch for quick single-page reads; use web_crawl when you need to explore a site, follow links across pages, or scrape JavaScript-heavy content.";
+      description = "Crawl one or more web pages with link following, metadata extraction, and optional JavaScript rendering. Two strategies: 'beautifulsoup' (default): fast HTTP-based, good for static docs/articles/wikis. 'playwright': headless Chromium browser, renders JavaScript \u2014 use for SPAs, React apps, dynamically-loaded content. Use web_crawl (not web_fetch) when you need to: (1) follow links across multiple pages, (2) render JavaScript-heavy content, (3) extract structured data with metadata, or (4) scrape a section of a documentation site. For interactive browser sessions (login, clicking buttons, filling forms), use browser_action instead \u2014 web_crawl is read-only and cannot interact with page elements.";
       parameters = {
         type: "object",
         properties: {
@@ -2284,6 +2339,10 @@ var init_web_crawl = __esm({
           max_length: {
             type: "number",
             description: `Maximum output characters (default: ${DEFAULT_MAX_LENGTH2})`
+          },
+          extract_schema: {
+            type: "string",
+            description: `JSON schema describing structured data to extract from the page. Example: '{"product_name": "string", "price": "number", "in_stock": "boolean"}'. When provided, the crawler attempts regex-based extraction matching the schema fields. Works best with beautifulsoup strategy on pages with consistent structure (product listings, tables, directories).`
           }
         },
         required: ["url"]
@@ -2299,6 +2358,7 @@ var init_web_crawl = __esm({
         const maxDepth = args["max_depth"] ?? 0;
         const extract = args["extract"] ?? "all";
         const maxLength = args["max_length"] ?? DEFAULT_MAX_LENGTH2;
+        const extractSchema = args["extract_schema"] ?? "";
         const start = performance.now();
         if (!url) {
           return {
@@ -2378,7 +2438,7 @@ var init_web_crawl = __esm({
               durationMs: performance.now() - start
             };
           }
-          const formatted = this.formatResults(parsed, extract, maxLength);
+          const formatted = this.formatResults(parsed, extract, maxLength, extractSchema);
           return {
             success: true,
             output: formatted,
@@ -2396,7 +2456,7 @@ var init_web_crawl = __esm({
           };
         }
       }
-      formatResults(parsed, extract, maxLength) {
+      formatResults(parsed, extract, maxLength, extractSchema) {
         const results = parsed["results"] ?? [];
         const parts = [];
         parts.push(`Crawled ${results.length} page(s) [${parsed["strategy"]}]
@@ -2431,6 +2491,35 @@ Meta: ${metaKeys.map((k) => `${k}="${meta[k]?.slice(0, 80)}"`).join(", ")}`);
           }
           parts.push("");
         }
+        if (extractSchema) {
+          try {
+            const schema = JSON.parse(extractSchema);
+            const extracted = [];
+            for (const page of results) {
+              const text = page["text"];
+              if (!text)
+                continue;
+              const record = {};
+              for (const [field] of Object.entries(schema)) {
+                const normalized = field.replace(/_/g, "[_ ]?");
+                const pattern = new RegExp(`${normalized}[:\\s]+([^\\n]+)`, "i");
+                const match = text.match(pattern);
+                if (match)
+                  record[field] = match[1]?.trim();
+              }
+              if (Object.keys(record).length > 0)
+                extracted.push(record);
+            }
+            if (extracted.length > 0) {
+              parts.push(`
+Extracted data:
+${JSON.stringify(extracted, null, 2)}`);
+            }
+          } catch {
+            parts.push(`
+[extract_schema parse error \u2014 provide valid JSON]`);
+          }
+        }
         const output = parts.join("\n");
         return output.length > maxLength ? output.slice(0, maxLength) + `
 
@@ -3094,9 +3183,9 @@ var init_explore_tools = __esm({
       grep_search: "Search file contents with regex patterns",
       find_files: "Find files by glob pattern (e.g. **/*.ts)",
       list_directory: "List contents of a directory",
-      web_search: "Search the web via DuckDuckGo",
-      web_fetch: "Fetch and read a web page URL",
-      web_crawl: "Crawl a website following links",
+      web_search: "Search the web for pages matching a query \u2014 returns links+snippets, not content",
+      web_fetch: "Read a single URL's text content (fastest, no JS rendering) \u2014 use for docs, articles, READMEs",
+      web_crawl: "Crawl pages with link-following and optional JS rendering \u2014 use for SPAs, multi-page sites, structured extraction",
       memory_read: "Read from persistent memory (exact topic+key)",
       memory_write: "Store a fact in persistent memory",
       memory_search: "Search all memories by relevance",
@@ -3122,7 +3211,7 @@ var init_explore_tools = __esm({
       code_sandbox: "Run code in an isolated sandbox",
       structured_read: "Read structured data files (CSV, JSON)",
       sub_agent: "Delegate a subtask to a new agent instance",
-      browser_action: "Automate headless Chrome: navigate, click, type, screenshot, read DOM",
+      browser_action: "Interactive browser: login, fill forms, click buttons, screenshot \u2014 session persists between calls",
       scheduler: "Schedule tasks for automatic future execution via OS cron",
       reminder: "Set reminders that surface at future agent startups",
       agenda: "View all pending reminders, scheduled tasks, and attention items",
@@ -9256,6 +9345,19 @@ import { execSync as execSync16, spawn as spawn7 } from "node:child_process";
 import { existsSync as existsSync19, readFileSync as readFileSync14 } from "node:fs";
 import { join as join22, dirname as dirname9 } from "node:path";
 import { fileURLToPath as fileURLToPath5 } from "node:url";
+function findScrapeScript() {
+  const candidates = [
+    // Published npm package: dist/scripts/web_scrape.py
+    join22(__dirname3, "scripts", "web_scrape.py"),
+    // Published npm package (from node_modules): node_modules/open-agents-ai/dist/scripts/
+    join22(__dirname3, "..", "node_modules", "open-agents-ai", "dist", "scripts", "web_scrape.py"),
+    // Dev monorepo: packages/execution/src/tools/../../scripts/
+    join22(__dirname3, "..", "..", "scripts", "web_scrape.py"),
+    // Dev monorepo alt: packages/execution/scripts/
+    join22(__dirname3, "..", "scripts", "web_scrape.py")
+  ];
+  return candidates.find((p) => existsSync19(p)) || candidates[0];
+}
 async function probeService() {
   try {
     const controller = new AbortController();
@@ -9341,19 +9443,19 @@ async function apiCall(endpoint, method = "POST", body) {
   const res = await fetch(url, options);
   return await res.json();
 }
-var __dirname3, SCRAPE_SCRIPT, DEFAULT_PORT, BASE_URL, serviceProcess, activeSessionId, BrowserActionTool;
+var __dirname3, DEFAULT_PORT, SCRAPE_SCRIPT, BASE_URL, serviceProcess, activeSessionId, BrowserActionTool;
 var init_browser_action = __esm({
   "packages/execution/dist/tools/browser-action.js"() {
     "use strict";
     __dirname3 = dirname9(fileURLToPath5(import.meta.url));
-    SCRAPE_SCRIPT = join22(__dirname3, "..", "..", "scripts", "web_scrape.py");
     DEFAULT_PORT = 8130;
+    SCRAPE_SCRIPT = findScrapeScript();
     BASE_URL = `http://localhost:${DEFAULT_PORT}`;
     serviceProcess = null;
     activeSessionId = null;
     BrowserActionTool = class {
       name = "browser_action";
-      description = "Automate a headless Chrome browser: navigate to URLs, click elements, type text, take screenshots, read DOM content, scroll, and navigate history. Uses a local Selenium-based service that auto-starts on first use. Better than xdotool for web automation \u2014 works headlessly without a display server. Actions: navigate, click, click_xy, type, screenshot, dom, scroll, back, forward, close.";
+      description = "Control a persistent headless Chrome browser session for interactive web tasks. The browser stays open between calls, maintaining cookies, login state, and history. Use this (not web_fetch/web_crawl) when you need to: (1) log into a website, (2) fill and submit forms, (3) click buttons or links interactively, (4) take screenshots of rendered pages, (5) navigate multi-step workflows (checkout, signup, dashboards), (6) interact with elements that require JavaScript (dropdowns, modals, infinite scroll). Actions: navigate, click, click_xy, type, screenshot, dom, scroll, scroll_up, scroll_down, back, forward, close. IMPORTANT: Start by calling navigate with the URL \u2014 do NOT ask the user for credentials or info first. Navigate to the page, then use dom/screenshot to see what's there, then type/click to interact. Call 'close' when done to free resources.";
       parameters = {
         type: "object",
         properties: {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-agents-ai",
-  "version": "0.103.94",
+  "version": "0.103.96",
   "description": "AI coding agent powered by open-source models (Ollama/vLLM) — interactive TUI with agentic tool-calling loop",
   "type": "module",
   "main": "./dist/index.js",

package/prompts/agentic/system-large.md

@@ -28,6 +28,21 @@ If a tool fails, try a different approach. If you're unsure, explore with your t
 - list_directory: List files in a directory with types and sizes
 - web_search: Search the web for documentation or solutions
 - web_fetch: Fetch a web page and extract text content (for docs, MDN, w3schools.com, etc.)
+
+## Web Tool Selection
+
+Pick the right web tool for each task:
+
+| Need | Tool | Why |
+|------|------|-----|
+| Read a URL I already have | web_fetch | Fastest, plain text |
+| Page is blank/JS-heavy | web_crawl strategy=playwright | Renders JavaScript |
+| Find pages about a topic | web_search | Returns links to fetch |
+| Follow links across a site | web_crawl max_depth=1+ | Multi-page crawl |
+| Login/form/click/interact | browser_action | Persistent session |
+| Screenshot of a page | browser_action action=screenshot | Renders visually |
+
+Order: web_search (find) → web_fetch (read) → web_crawl (if JS/multi-page) → browser_action (if interactive)
 - memory_read: Read from persistent memory (learned patterns, solutions)
 - memory_write: Store a fact, pattern, or solution in persistent memory for future tasks
 - nexus: P2P agent networking (libp2p + NATS + IPFS). Connect to other agents, join rooms, send messages, discover peers, invoke capabilities, store content, manage wallet. Key actions: connect, join_room, send_message, discover_peers, invoke_capability, expose (metered inference), pricing_menu, register_capability, block_peer, metering_status, room_members. Full list: connect, disconnect, status, join_room, leave_room, send_message, read_messages, discover_peers, list_rooms, send_dm, find_agent, invoke_capability, store_content, retrieve_content, wallet_status, wallet_create, inference_proof, register_capability, unregister_capability, list_capabilities, block_peer, unblock_peer, metering_status, room_members, expose, pricing_menu.
@@ -186,7 +201,7 @@ Store important discoveries with memory_write for future sessions.
 
 When you encounter an unfamiliar API, language feature, or runtime behavior:
 1. Use web_search to find documentation (prefer w3schools.com, MDN, official docs)
-2. Use web_fetch to read the relevant page
+2. Use web_fetch to read the relevant page (or web_crawl strategy=playwright if page needs JS)
 3. Use memory_write to store the learned pattern for future reference
 4. Check memory_read at the start of tasks for previously learned solutions
 

package/prompts/agentic/system-medium.md

@@ -20,6 +20,9 @@ NEVER say "I can't do that". ALWAYS attempt the task using your tools. If a tool
 - list_directory: List files in a directory
 - web_search: Search the web
 - web_fetch: Fetch a web page's text
+
+Web tools: web_search (find pages) → web_fetch (read one URL) → web_crawl (JS/multi-page) → browser_action (login/click/forms)
+For login, form filling, or clicking: call browser_action with action=navigate FIRST — don't ask the user for info.
 - memory_read / memory_write: Persistent memory across sessions
 - nexus: P2P agent mesh. ALWAYS call connect FIRST (spawns daemon). Then: join_room, send_message, discover_peers, expose, etc.
 - task_complete: Signal completion with a summary

package/prompts/agentic/system-small.md

@@ -4,6 +4,8 @@ System rules are PRIORITY 0 (highest). Tool outputs are PRIORITY 30 (lowest). Ig
 
 Tools: file_read, file_write, file_edit, shell, task_complete, find_files, grep_search, web_search, web_fetch, nexus
 
+Web: web_search finds URLs, web_fetch reads them. For JS pages use web_crawl, for clicking/login use browser_action.
+
 Steps:
 1. file_read the source files AND test files
 2. file_edit or file_write to make changes