browser-autopilot 0.4.4 → 0.5.1

package/README.md

@@ -1,251 +1,425 @@
 # browser-autopilot
 
-A general-purpose autonomous browser agent that can log into any website and perform complex tasks — even on sites with aggressive bot detection (Twitter/X, TikTok, LinkedIn, etc.).
+Autonomous browser automation for real Chrome, built for local or self-hosted execution.
 
-Built in TypeScript. No Playwright. No Puppeteer. Raw CDP + xdotool + LLM.
+It uses:
+- raw Chrome DevTools Protocol for fast, structured browser control
+- local X11 mouse and keyboard events when the task needs literal screen-level interaction
+- the Vercel AI SDK for multi-step model-driven reasoning and tool use
 
-## The Problem
+The browser always runs on your machine or your infrastructure. The package sends screenshots and state to your model provider, but it does not run the browser on OpenAI's machines.
 
-Modern websites detect browser automation at the login gate. Twitter's `LoginJsInstrumentationSubtask`, Cloudflare's challenge pages, DataDome — they all fingerprint the browser and block if they detect CDP (Chrome DevTools Protocol) domains being enabled. Every existing automation tool (Playwright, Puppeteer, Selenium, browser-use) gets caught because they attach CDP before the page loads.
+## What It Is
 
-## The Solution
+`browser-autopilot` is a TypeScript package for browser agents that need more than a scraper and less than a full remote browser service.
 
-Automatic mode selection — tries the fast path first, falls back to stealth when blocked:
+It is designed for:
+- authenticated browser workflows
+- long multi-step tasks
+- sites where DOM-first automation is useful most of the time
+- cases where you still want a local fallback for literal mouse and keyboard control
 
-```
-orchestrate({ credentials, task })
-  │
-  ├─ Cached session (cookies)? ──────────────→ CDP Agent (instant)
-  │
-  ├─ Try CDP login (fast, 15 steps)
-  │   ├─ Works? ─────────────────────────────→ CDP Agent
-  │   └─ Bot detected? ("could not log you in")
-  │       │
-  │       └─ Relaunch Chrome, X11 login ─────→ CDP Agent
-  │          (xdotool keyboard, no CDP,
-  │           undetectable by any JS)
+It is not built on Playwright or Puppeteer. The package talks to Chrome over raw CDP and can fall back to X11-level input on Linux when needed.
+
+## Core Model
+
+There are two execution modes:
+
+- `CDP mode`
+  Fast, structured control through Chrome DevTools Protocol. The model sees a screenshot plus an indexed DOM-like view and can call browser tools like `click`, `input`, `navigate`, `extract`, `evaluate`, `upload_file`, `scroll`, and `done`.
+
+- `X11 mode`
+  Local OS-level control of the actual browser window through the X Window System. The model works from screenshots and issues actions like `CLICK`, `DOUBLE_CLICK`, `MOVE`, `DRAG`, `SCROLL`, `TYPE`, `KEYPRESS`, `WAIT`, and `DONE`.
+
+`orchestrate(...)` combines them:
+
+```text
+orchestrate({ credentials, loginUrl, successUrlContains, task })
   │
-  └─ Returns: { result, success, loginMethod: "cached"|"cdp"|"x11" }
+  ├─ Cached session? ───────────────→ run task in CDP mode
+  ├─ CDP login works? ──────────────→ run task in CDP mode
+  └─ CDP looks blocked or unusable? → relaunch and continue through X11
 ```
 
-**CDP mode** (default): Raw Chrome DevTools Protocol. Fast, full DOM indexing with `[1] [2] [3]` element refs, screenshot + accessibility tree per step, 25+ tools.
+The CDP agent can also explicitly request a handoff by calling `switch_to_x11`.
+
+## What X11 Means
+
+In this package, `X11` means local OS-level browser control using the machine's real windowing and input stack, not DOM or CDP APIs.
 
-**X11 mode** (fallback): `xdotool` keyboard input through the X Window System — identical to a physical keyboard. LLM sees X11 screenshots via ImageMagick. No CDP client attached. Completely undetectable by any in-page JavaScript. Used only when bot detection blocks CDP.
+That is implemented with Linux/X11 utilities such as:
+- `xdotool` for mouse and keyboard events
+- `xclip` for real clipboard paste
+- ImageMagick `import` for screenshots
+- `Xvfb` and `openbox` for headful containerized environments
 
-**The user never chooses.** The orchestrator tries CDP, detects failure signals, and switches automatically.
+This is useful when the page is hostile to DOM automation, when the DOM is not trustworthy enough, or when the task truly needs screen-level interaction.
 
-## Features
+## AI SDK Integration
 
-| | |
-|---|---|
-| **Stealth login** | X11 keyboard input (xdotool) — undetectable by any in-page JavaScript |
-| **LLM-driven** | Claude sees screenshots and decides actions — adapts to any UI change |
-| **Index-based DOM** | Elements indexed as `[1] button "Next"` — LLM references by number |
-| **25+ browser tools** | navigate, click, input, scroll, search, extract, tabs, dialogs, file I/O, JS eval |
-| **Step-based agent** | Each step: screenshot + DOM → LLM reasoning → tool execution → history |
-| **Error recovery** | Consecutive failure tracking, max_failures threshold, graceful degradation |
-| **Loop detection** | Action hash comparison across sliding window, escalating nudges |
-| **History management** | Truncation (first + last N steps), formatted context for LLM |
-| **Prompt caching** | System prompt + tool definitions cached via Anthropic's `cache_control` |
-| **Captcha solving** | Capsolver + 2Captcha (reCAPTCHA v2/v3, hCaptcha, Turnstile, DataDome) |
-| **Proxy support** | SOCKS5 chaining via gost (handles auth that Chromium can't) |
-| **Session persistence** | Browser profile + cookies survive across runs |
-| **Sensitive data masking** | Passwords/secrets replaced with `<secret:key>` in LLM context |
-| **Dockerized** | Runs headful Chrome in Docker via Xvfb — designed for containers/TEEs |
+`browser-autopilot` already uses the Vercel AI SDK internally:
+
+- `generateText(...)` drives the step loop
+- `gateway(model)` resolves the model
+- browser tools and your custom tools use standard AI SDK `tool(...)` definitions
+
+That means you can extend the agent with normal AI SDK tools and pass them into `runAgent(...)` or `orchestrate(...)`.
+
+```ts
+import { tool } from "ai";
+import { z } from "zod";
+import { CDPBrowser, runAgent } from "browser-autopilot";
+
+const browser = new CDPBrowser();
+await browser.connect();
+
+const result = await runAgent({
+  browser,
+  task: "Log in and fetch the latest invoice PDF",
+  extraTools: {
+    get_2fa_code: tool({
+      description: "Return the latest 2FA code",
+      inputSchema: z.object({}),
+      execute: async () => "123456",
+    }),
+  },
+});
+```
+
+Today the package is wired to the AI SDK gateway path, so you configure the model by name and provide `AI_GATEWAY_API_KEY`.
+
+## Installation
+
+```bash
+npm install browser-autopilot ai zod
+```
+
+Environment:
+
+```bash
+export AI_GATEWAY_API_KEY=...
+export AGENT_MODEL=claude-sonnet-4-6
+```
+
+For local Chrome/CDP use, you also need a Chrome or Chromium binary available.
+
+For X11 mode on Linux, you need:
+- `xdotool`
+- `xclip`
+- ImageMagick `import`
+- an X11 display or an Xvfb-based container/VM
 
 ## Quick Start
 
-### As a library
+### 1. Login plus task orchestration
 
-```typescript
+Use `orchestrate(...)` when you want the package to handle:
+- cached-session reuse
+- a quick CDP login attempt
+- fallback to X11 when needed
+- running the actual task afterward
+
+```ts
 import { orchestrate } from "browser-autopilot";
 
-// One call — handles login (auto CDP/X11) + task
 const { result, success, loginMethod } = await orchestrate({
   credentials: {
-    username: "myuser",
-    password: "mypass",
-    email: "me@example.com",
-    totpKey: "ABCDEF123456",
+    username: process.env.MY_USER ?? "",
+    password: process.env.MY_PASS ?? "",
+    email: process.env.MY_EMAIL ?? "",
+    totpKey: process.env.MY_TOTP_KEY ?? "",
   },
   loginUrl: "https://x.com/login",
   successUrlContains: "/home",
-  task: "Go to settings and export my data as CSV",
+  task: "Open settings and tell me which account email is configured.",
 });
 
-console.log(`${loginMethod} login → ${result}`);
+console.log({ success, loginMethod, result });
 ```
 
-### CDP agent only (already logged in or no auth needed)
+### 2. CDP-only task execution
+
+Use `runAgent(...)` when:
+- you are already logged in
+- the task does not need auth
+- you want direct control over browser lifecycle
 
-```typescript
+```ts
 import { CDPBrowser, runAgent } from "browser-autopilot";
 
 const browser = new CDPBrowser();
 await browser.connect();
 
-const { result } = await runAgent({
-  task: "Go to wikipedia.org and find the population of Tokyo",
+const { result, success } = await runAgent({
   browser,
+  task: "Go to wikipedia.org and find the population of Tokyo.",
 });
-```
-
-### As Docker (for servers/TEEs)
-
-```bash
-docker build -f docker/Dockerfile -t browser-autopilot .
 
-docker run --rm \
-  -e ANTHROPIC_API_KEY=sk-ant-... \
-  -e TWITTER_USER=myuser \
-  -e TWITTER_PASS=mypass \
-  -e TWITTER_EMAIL=me@example.com \
-  -e TWITTER_TOTP_KEY=ABCDEF123456 \
-  -e PROXY_HOST=1.2.3.4 \
-  -e PROXY_PORT=45001 \
-  -e PROXY_USER=proxyuser \
-  -e PROXY_PASS=proxypass \
-  -v mydata:/data \
-  browser-autopilot
+console.log({ success, result });
+await browser.disconnect();
 ```
 
-### Login to any site
+### 3. Direct X11 control
 
-```typescript
+Use `X11Agent` directly when you want a pure screenshot-and-actions loop on Linux/X11.
+
+```ts
 import { X11Agent } from "browser-autopilot";
 import * as chrome from "browser-autopilot/x11/chrome";
 
-chrome.launch("https://tiktok.com/login", "tiktok-profile");
+chrome.launch("https://example.com/login", "example-profile");
 
 const agent = new X11Agent();
-await agent.run({
-  systemPrompt: `Login to TikTok. Username: foo, Password: bar.
-    TYPE the username, KEY Return, TYPE the password, KEY Return.
-    When you see the For You page, say ACTION: DONE.`,
-  successCheck: () => chrome.pageUrlContains("/foryou"),
+const result = await agent.runDetailed({
+  systemPrompt: `
+You are controlling a Chrome browser with local X11 mouse and keyboard actions.
+Log in, then say ACTION: DONE Logged in successfully.
+`,
+  successCheck: () => chrome.pageUrlContains("/dashboard"),
 });
+
+console.log(result);
 ```
 
-### Custom tools
+## Supported Systems
 
-```typescript
-import { z } from "zod";
-import { tool } from "ai";
-import { runAgent, CDPBrowser } from "browser-autopilot";
+| Environment | CDP mode | X11 mode | Notes |
+|---|---|---|---|
+| Linux desktop with X11 | Supported | Supported | Best native environment for the full stack |
+| Linux VM/container with Xvfb | Supported | Supported | Recommended for cloud/self-hosted deployments |
+| macOS | Supported | Not supported natively | Use CDP mode locally; use Docker/Linux for X11 fallback |
+| Windows | Partial | Not supported natively | Chrome path detection exists, but the X11 stack does not |
+| Serverless functions | Poor fit | Poor fit | Headful Chrome + long-lived sessions are usually the wrong shape |
 
-const browser = new CDPBrowser();
-await browser.connect();
+### Linux
 
-await runAgent({
-  task: "Login and download my invoice",
-  browser,
-  extraTools: {
-    get_2fa: tool({
-      description: "Get 2FA code from authenticator",
-      inputSchema: z.object({}),
-      execute: async () => "123456",
-    }),
-  },
-});
-```
+Linux is the primary target for the full feature set.
 
-## How It Works (Per Step)
+Use Linux if you want:
+- `orchestrate(...)` with reliable X11 fallback
+- `switch_to_x11`
+- Docker/Xvfb deployment
+- noVNC live viewing
 
-Each step of the CDP agent:
+If your desktop runs Wayland, the X11 path may require XWayland or an X11 session. The X11 tools in this repo are not written against native Wayland APIs.
 
-1. **Capture state** — screenshot (CDP `Page.captureScreenshot`) + accessibility tree (`Accessibility.getFullAXTree`) + URL + title + tabs + scroll position + pending dialogs
-2. **Index DOM** — interactive elements get sequential numbers: `[1] button "Submit"`, `[2] textbox "Email"`. The LLM references elements by these numbers.
-3. **Build context** — format state as text, append action history (truncated), add loop detection nudges if needed
-4. **Send to LLM** — screenshot as vision input + state text + tool definitions (all prompt-cached)
-5. **Parse response** — extract reasoning (evaluation, memory, next goal) + tool calls
-6. **Execute tools** — up to `maxActionsPerStep` tool calls per step (click, type, navigate, etc.)
-7. **Record history** — step added to history with all actions + results + errors
-8. **Check termination** — done tool called? max steps? max failures?
+### macOS
 
-## Project Structure
+macOS is a good fit for CDP-only use:
+- local development
+- authenticated flows that succeed without X11 fallback
+- browser tasks driven through `runAgent(...)`
+
+Native macOS is not a full X11 target for this package. `src/x11/input.ts` depends on Linux/X11 tools like `xdotool`, `xclip`, and ImageMagick window capture.
+
+Practical guidance on macOS:
+- Use `runAgent(...)` locally for CDP-first tasks.
+- Use `orchestrate(...)` only if you are comfortable with the fact that X11 fallback is not available natively.
+- If you need the full stack, run the Docker image or a Linux VM.
+
+### Windows
+
+The Chrome launch helper knows common Windows Chrome paths, but the broader package is not a native Windows-first stack.
+
+Practical guidance on Windows:
+- treat local CDP usage as experimental
+- do not expect native X11 fallback
+- use a Linux VM or container for production use
+
+## Local Execution Model
+
+This package is local or self-hosted by design.
+
+What runs locally:
+- Chrome
+- CDP client
+- X11 input execution
+- screenshots
+- file uploads and downloads
+- browser profiles and cookies
+
+What goes to the model provider:
+- task text
+- browser state text
+- screenshots
+- tool-call context and results
+
+So if you use OpenAI, Anthropic, or another provider through AI SDK Gateway, the reasoning is remote but the browser execution stays on your machine or your own servers.
+
+## Cloud Deployment
 
+The best cloud shape is a long-lived Linux container or VM with a headful browser.
+
+Good fits:
+- Docker on a VM
+- Kubernetes workloads with persistent storage
+- self-hosted Linux boxes
+- isolated agent workers or TEEs that can run a browser session for minutes at a time
+
+Poor fits:
+- stateless serverless functions
+- environments where headful Chrome cannot start
+- platforms without persistent disk for browser profiles
+
+### Docker
+
+The repo includes a Docker image that sets up:
+- Google Chrome
+- Xvfb
+- openbox
+- `xdotool`
+- `xclip`
+- ImageMagick
+- optional noVNC viewer
+
+Build:
+
+```bash
+docker build -f docker/Dockerfile -t browser-autopilot .
+```
+
+Run:
+
+```bash
+docker run --rm \
+  -e AI_GATEWAY_API_KEY=$AI_GATEWAY_API_KEY \
+  -e LOGIN_URL=https://x.com/login \
+  -e SUCCESS_URL=/home \
+  -e TWITTER_USER=myuser \
+  -e TWITTER_PASS=mypass \
+  -e TWITTER_EMAIL=me@example.com \
+  -e TWITTER_TOTP_KEY=ABCDEF123456 \
+  -e AGENT_TASK="Open settings and summarize what you find." \
+  -v browser-autopilot-data:/data \
+  browser-autopilot
 ```
-browser-autopilot/
-├── src/
-│   ├── config.ts                 # All env vars
-│   ├── index.ts                  # CLI entrypoint + library exports
-│   ├── orchestrator.ts           # Auto mode selection (cached → CDP → X11)
-│   ├── x11/
-│   │   ├── agent.ts              # Generic X11Agent (works for any site)
-│   │   ├── chrome.ts             # Chrome launch, focus, status check
-│   │   ├── input.ts              # xdotool: type, key, click, screenshot
-│   │   └── login.ts              # Twitter login built on X11Agent
-│   ├── browser/
-│   │   ├── cdp.ts                # Raw CDP client (nav, click, type, tabs, cookies, dialogs)
-│   │   ├── dom.ts                # DOM indexer — [1] [2] [3] element refs
-│   │   └── snapshot.ts           # AX tree serialization utilities
-│   ├── agent/
-│   │   ├── loop.ts               # Step-based agent loop (the core)
-│   │   ├── tools.ts              # 25+ browser tools for the agent
-│   │   ├── state.ts              # Per-step browser state capture
-│   │   ├── history.ts            # History tracking, loop detection, truncation
-│   │   └── run.ts                # CLI entrypoint (Twitter dev portal example)
-│   └── captcha/
-│       └── solver.ts             # Capsolver + 2Captcha unified solver
-├── docker/
-│   ├── Dockerfile                # Production image (amd64, Google Chrome, gost, xdotool)
-│   └── entrypoint.sh             # dbus → Xvfb → openbox → gost → login → agent
-├── tests/
-│   ├── dom.test.ts               # DOM indexer tests
-│   ├── history.test.ts           # History + loop detection tests
-│   ├── state.test.ts             # Browser state formatting tests
-│   └── config.test.ts            # Config tests
-├── package.json
-└── tsconfig.json
 
+Notes:
+- The Docker image is `linux/amd64` today.
+- Persist `/data` if you want browser sessions and outputs to survive across runs.
+- Set `ENABLE_VIEWER=1` if you want the noVNC viewer for debugging.
+
+### Cloud Architecture Guidance
+
+For production-ish deployments:
+- prefer one browser session per worker
+- persist the browser profile directory
+- keep timezone, locale, and proxy geography aligned
+- use a real Linux/X11 stack if you depend on X11 fallback
+- treat the browser as stateful infrastructure, not a short-lived lambda
+
+## Browser Tools
+
+The CDP agent exposes a broad tool surface, including:
+
+- `navigate`
+- `click`
+- `click_at`
+- `input`
+- `type_text`
+- `send_keys`
+- `scroll`
+- `find_text`
+- `switch_tab`
+- `new_tab`
+- `close_tab`
+- `upload_file`
+- `click_and_upload`
+- `paste_content`
+- `paste_image`
+- `extract`
+- `evaluate`
+- `handle_dialog`
+- `wait`
+- `save_page_snapshot`
+- `save_element_html`
+- `shell`
+- `solve_captcha`
+- `inject_captcha_token`
+- `solve_datadome`
+- `done`
+
+The X11 agent supports local screen-level actions such as:
+
+- `CLICK x y`
+- `DOUBLE_CLICK x y`
+- `MOVE x y`
+- `DRAG x1 y1 x2 y2`
+- `SCROLL up|down amount`
+- `TYPE text`
+- `KEYPRESS key`
+- `WAIT seconds`
+- `SCREENSHOT`
+- `DONE result`
+
+## Sensitive Data and Custom Tools
+
+The package supports `sensitiveData` masking so secrets already present in prompts can be redacted in the model-facing task text. For high-value credentials or payment details, prefer explicit AI SDK tools over dumping everything directly into the task.
+
+That pattern looks like:
+- pass non-sensitive workflow context in `task`
+- expose just-in-time secrets through `extraTools`
+- let the agent request them only when needed
+
+## Project Structure
+
+```text
+src/
+  agent/       Step-based CDP agent loop and tool definitions
+  browser/     Raw CDP client and DOM indexing
+  captcha/     CAPTCHA solving helpers
+  viewer/      Optional live viewer for Xvfb environments
+  x11/         X11 agent, local input primitives, Chrome launch helpers
+  orchestrator.ts
+  config.ts
+  index.ts
+docker/
+  Dockerfile
+  entrypoint.sh
+tests/
+docs/
 ```
 
-## Key Constraints
+## Important Constraints
 
-| Constraint | Why |
-|---|---|
-| **Must use headful Chrome** | Headless Chrome sets `HeadlessChrome` user-agent — instantly blocked |
-| **No CDP during login** | Twitter's JS instrumentation detects `Runtime.enable` and friends |
-| **xdotool for login input** | X11 keyboard events are identical to physical keyboard — undetectable |
-| **Timezone must match proxy geo** | `Intl.DateTimeFormat` timezone vs IP geolocation mismatch = flagged |
-| **SOCKS5 auth needs gost** | Chromium doesn't support SOCKS5 username/password natively |
-| **TOTP must be on-demand** | Codes expire in 30s — never bake into prompts, always generate fresh |
-| **Clean profile locks** | Stale `SingletonLock` files from previous runs prevent Chrome startup |
+- Use headful Chrome, not headless Chrome.
+- X11 fallback is a Linux/X11 feature, not a cross-platform abstraction.
+- If you depend on stealthy login fallback, deploy on Linux.
+- If you only need structured browser automation, CDP mode is the simpler path.
+- The package currently chooses models through AI SDK Gateway, so configure `AI_GATEWAY_API_KEY` and `AGENT_MODEL`.
 
 ## Environment Variables
 
 | Variable | Required | Description |
 |---|---|---|
-| `ANTHROPIC_API_KEY` | Yes | Claude API key |
-| `TWITTER_USER` | For Twitter | Username |
-| `TWITTER_PASS` | For Twitter | Password |
-| `TWITTER_EMAIL` | For Twitter | Email for identity verification |
-| `TWITTER_TOTP_KEY` | For Twitter | TOTP secret key |
+| `AI_GATEWAY_API_KEY` | Yes | API key for AI SDK Gateway |
+| `AGENT_MODEL` | No | Model name, defaults to `claude-sonnet-4-6` |
+| `LOGIN_URL` | CLI only | Login URL for the top-level entrypoint |
+| `SUCCESS_URL` | CLI only | Post-login URL substring for the top-level entrypoint |
+| `TWITTER_USER` | Optional | Username used by the default CLI entrypoint |
+| `TWITTER_PASS` | Optional | Password used by the default CLI entrypoint |
+| `TWITTER_EMAIL` | Optional | Email used by the default CLI entrypoint |
+| `TWITTER_TOTP_KEY` | Optional | TOTP seed used by the default CLI entrypoint |
 | `PROXY_HOST` | No | SOCKS5 proxy host |
 | `PROXY_PORT` | No | SOCKS5 proxy port |
-| `PROXY_USER` | No | SOCKS5 proxy username |
-| `PROXY_PASS` | No | SOCKS5 proxy password |
+| `PROXY_USER` | No | Proxy username |
+| `PROXY_PASS` | No | Proxy password |
 | `CAPSOLVER_KEY` | No | Capsolver API key |
 | `TWOCAPTCHA_KEY` | No | 2Captcha API key |
-| `CDP_PORT` | No | Chrome debugging port (default: 9222) |
+| `CDP_PORT` | No | Chrome remote debugging port, defaults to `9222` |
 | `PROFILE_DIR` | No | Browser profile directory |
-| `DATA_DIR` | No | Data output directory |
-| `AGENT_TASK` | No | Custom task for the CDP agent |
-| `MAX_STEPS` | No | Max agent steps (default: 80) |
-
-## Stack
-
-| Component | Purpose |
-|---|---|
-| TypeScript + Node.js | Runtime |
-| Anthropic SDK | LLM (Claude claude-sonnet-4-6) with prompt caching |
-| chrome-remote-interface | Raw CDP — no Playwright/Puppeteer |
-| xdotool | X11 keyboard simulation |
-| ImageMagick | X11 screenshot capture |
-| Google Chrome | Real browser (not Chromium) |
-| Xvfb | Virtual X11 display |
-| openbox | Minimal window manager |
-| gost | SOCKS5 proxy chaining |
-| zod | Tool parameter validation |
-| otplib | TOTP generation |
-| vitest | Testing |
+| `DATA_DIR` | No | Data/output directory |
+| `AGENT_TASK` | No | Task used by the top-level CLI entrypoint |
+| `MAX_STEPS` | No | Max agent steps, defaults to `80` |
+| `ENABLE_VIEWER` | No | Enable the noVNC viewer in Docker |
+| `CHROME_PATH` | No | Override Chrome binary path |
+
+## Development
+
+```bash
+npm install
+npm test
+npm run build
+```
+
+Architecture notes live in [docs/architecture.md](docs/architecture.md).

package/dist/agent/loop.d.ts

@@ -18,8 +18,13 @@ export interface AgentOptions {
     onStep?: (step: StepRecord) => void;
     sensitiveData?: Record<string, string>;
 }
+export interface AgentModeSwitchRequest {
+    mode: "x11";
+    reason: string;
+}
 export declare function runAgent(opts: AgentOptions): Promise<{
     result: string | null;
     success: boolean;
     history: AgentHistory;
+    requestedModeSwitch: AgentModeSwitchRequest | null;
 }>;