pi-agent-browser-native 0.9.1 → 0.9.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,18 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ ## 0.9.2 - 2026-10-04
6
+
7
+ ### Fixed
8
+
9
+ - Treat `--engine` as launch-scoped so switching an active managed browser uses the existing fresh-session recovery instead of replacing its page in place.
10
+ - Label positively identified Lightpanda images as text-rendered in direct, native batch and selected browser-code output; retain image bytes while refusing CSS/mouse-coordinate geometry even when synthetic dimensions match.
11
+ - Omit wrapper-supplied Chrome profile/executable bootstrap defaults when CLI, environment or native config selects Lightpanda, without changing caller-selected launch settings.
12
+
13
+ ### Documentation
14
+
15
+ - Keep Chrome default and Lightpanda opt-in. Document separate installation, single-target/visual/authentication limits, native unsupported commands, CORS and telemetry; add no driver, browser bundle or automatic fallback.
16
+
5
17
  ## 0.9.1 - 2026-10-02
6
18
 
7
19
  ### Removed
package/README.md CHANGED
@@ -321,7 +321,7 @@ npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-conf
321
321
  npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config browser executable set "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
322
322
  ```
323
323
 
324
- Profile hints with `authenticated-only` remain advisory. Global/override profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers; active roots and unrelated explicit sessions retain their native settings. Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `agent_browser` with `args: ["profiles"]` and `args: ["doctor"]` when profile resolution fails. The upstream `profiles` command lists Chrome profiles from Chrome's user data directory; `Default` is not canonical on every machine. Use the displayed profile directory name, a full profile/user-data directory path when upstream accepts one, or a configured `browser.executablePath` plus `sessionMode: "fresh"` for a different Chromium-compatible browser.
324
+ Profile hints with `authenticated-only` remain advisory. Global/override profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers only when the effective engine is Chrome; active roots and unrelated explicit sessions retain their native settings. Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `agent_browser` with `args: ["profiles"]` and `args: ["doctor"]` when profile resolution fails. The upstream `profiles` command lists Chrome profiles from Chrome's user data directory; `Default` is not canonical on every machine. Use the displayed profile directory name, a full profile/user-data directory path when upstream accepts one, or a configured `browser.executablePath` plus `sessionMode: "fresh"` for a different Chromium-compatible browser.
325
325
 
326
326
  ## Common agent calls
327
327
 
@@ -338,6 +338,21 @@ Chromium launch switches belong in the value of `--args`, for example `{ "args":
338
338
 
339
339
  Local Chrome startup uses the stock binary's `--args --no-startup-window` capability to suppress Chrome's extra startup window. Headless remains the default. Caller launch arguments keep native CLI → environment → config precedence and follow this default; browsers started before this change with custom arguments may restart once when their native launch configuration changes. Ordinary follow-ups do not resend bootstrap settings. CDP, providers, Electron, and Lightpanda remain native-owned; no tabs are deleted or profiles edited by this behavior.
340
340
 
341
+ ### Opt-in Lightpanda
342
+
343
+ Chrome remains the default. Install [Lightpanda](https://github.com/lightpanda-io/browser/releases/tag/1.0.0) separately on supported macOS/Linux hosts, put `lightpanda` on `PATH`, then select it explicitly:
344
+
345
+ ```json
346
+ { "args": ["--engine", "lightpanda", "open", "https://example.com"], "sessionMode": "fresh" }
347
+ { "args": ["snapshot", "-i"] }
348
+ ```
349
+
350
+ `--engine` is launch-scoped; changing an active managed browser uses the existing fresh-session recovery. `AGENT_BROWSER_ENGINE` and native config also select the engine. Wrapper-supplied Chrome profile/executable defaults are omitted for Lightpanda; caller arguments, environment, and native config remain unchanged.
351
+
352
+ Lightpanda screenshots render page text, not graphical layout. Positively identified captures carry `imageObservations[].rendering: "text"` and unknown coordinate geometry; keep using refs, not image-derived mouse coordinates. Lightpanda is not Chrome-parity visual QA, profile login, multi-tab, or WebMCP support, and this wrapper never falls back automatically. See [capabilities and limits](docs/COMMAND_REFERENCE.md#lightpanda-engine).
353
+
354
+ ### Native navigation and interaction
355
+
341
356
  `open` without a URL uses native `get url`: it launches when needed and otherwise preserves the current page. This also applies to effective batch rows and code calls. `details.args` preserves the request, while `effectiveArgs`, command, URL data, and lifecycle describe the actual native operation. Use `open about:blank` only when you intend to navigate there. Help/version requests remain unchanged.
342
357
 
343
358
  Watch a browser window during a demo, QA run, or user-completed login by adding upstream's global `--headed` flag on the first launch. Use `sessionMode: "fresh"` if a managed session may already exist, because headed/headless state is launch-scoped. A successful first/fresh local wrapper-managed headed launch, including a launch inside `batch`, returns `details.browserWindow = { mode: "headed", ownership: "wrapper-managed", sessionName, visibility: "unverified" }` and one visible handoff sentence; CDP, auto-connect, provider, and Electron attachments do not. This proves that the wrapper requested and upstream launched headed mode, not that the OS window is visible on the user's display; remote, container, or virtual-display setups can still hide it. After the user finishes in the window, continue with `sessionMode: "auto"`.
@@ -505,7 +520,7 @@ Use the exact returned recovery payload after errors. Timeout partial progress r
505
520
 
506
521
  Both JSON and prose results expose a bounded canonical observation with recovery actions and artifact receipts. Oversized observations name an `observationPath` for the complete redacted result; audit-only `details` is not the model's sole recovery channel. Inner code calls avoid full model rendering, spills, and image encoding until output is selected.
507
522
 
508
- Screenshots expose measured image dimensions and capture geometry when supported by matching before/after samples. A full-page or element image is not a viewport screenshot. Native mouse coordinates are viewport CSS pixels; use the [geometry contract](docs/TOOL_CONTRACT.md#image-observations) and current scroll/frame state before converting image pixels. Unknown geometry stays unknown, including ambiguous or scrolled element captures and unmeasured batch captures. Pi image-resize notes only map sent pixels to original image pixels.
523
+ Screenshots expose measured image dimensions and capture geometry when supported by matching before/after samples. A full-page or element image is not a viewport screenshot. Native mouse coordinates are viewport CSS pixels; use the [geometry contract](docs/TOOL_CONTRACT.md#image-observations) and current scroll/frame state before converting image pixels. Unknown geometry stays unknown, including ambiguous or scrolled element captures, unmeasured batch captures, and Lightpanda text-rendered images (`rendering: "text"`). Pi image-resize notes only map sent pixels to original image pixels.
509
524
 
510
525
  Save artifacts to the user's exact paths and check `artifactVerification` before claiming they exist or are fresh. Conditional `changed: false` screenshots produce no new attachment. `outputPath` is a separate result-data destination and must not alias a screenshot, download, or recording. Close ends the browser session; explicit files remain host-owned. Recording starts remain pending until stop and receipt/file verification.
511
526
 
@@ -29,6 +29,10 @@ export const LAUNCH_SCOPED_FLAG_DEFINITIONS = [
29
29
  flag: "--enable",
30
30
  reason: "selects built-in page init scripts before the upstream browser session is launched",
31
31
  },
32
+ {
33
+ flag: "--engine",
34
+ reason: "selects the browser engine used for the upstream launch",
35
+ },
32
36
  {
33
37
  flag: "--executable-path",
34
38
  reason: "selects the browser executable used for the upstream launch",
@@ -104,9 +104,10 @@ async function runAgentBrowserToolInContext(options) {
104
104
  const output = await processBrowserOutput({ ...options, artifactRunStartedAtMs, prepared, processResult });
105
105
  applyBrowserRunStatePatch(options.state, output.statePatch);
106
106
  if (isRecord(output.result.details) && Array.isArray(output.result.details.imageObservations) && prepared.commandTokens[0] === "screenshot") {
107
- output.result.details.imageObservations = redactSensitiveValue(output.result.details.imageObservations.map(image => ({
108
- ...image, geometry: buildScreenshotGeometry({ capture: image.capture, pixels: image.pixels, before, after }),
109
- })));
107
+ output.result.details.imageObservations = redactSensitiveValue(output.result.details.imageObservations.map(image => {
108
+ const rendering = image.rendering ?? before?.rendering ?? after?.rendering;
109
+ return { ...image, ...(rendering ? { rendering } : {}), geometry: buildScreenshotGeometry({ rendering, capture: image.capture, pixels: image.pixels, before, after }) };
110
+ }));
110
111
  }
111
112
  return output.result;
112
113
  }
@@ -1,4 +1,5 @@
1
1
  import { isRecord } from "../../parsing.js";
2
+ import { LIGHTPANDA_IMAGE_REASON } from "../../results/presentation/common.js";
2
3
  import { getScreenshotPathTokenIndex, getScreenshotPositionalIndices } from "./artifact-paths.js";
3
4
  import { runSessionCommandData } from "./session-state.js";
4
5
  export function getScreenshotCapture(command) {
@@ -28,7 +29,8 @@ export async function collectScreenshotSample(options) {
28
29
  element = { x: rect.x, y: rect.y, width: rect.width, height: rect.height };
29
30
  }
30
31
  } catch {}
31
- return { url: location.href, frame: window === window.top ? "main" : "child", childFrameCount: window.frames.length,
32
+ const textRendered = navigator.userAgentData?.brands?.some(b => b.brand === "Lightpanda") || /\\bLightpanda\\//i.test(navigator.userAgent);
33
+ return { ...(textRendered ? { rendering: "text" } : {}), url: location.href, frame: window === window.top ? "main" : "child", childFrameCount: window.frames.length,
32
34
  viewport: { width: innerWidth, height: innerHeight },
33
35
  document: { width: Math.max(root.scrollWidth, body?.scrollWidth || 0, innerWidth), height: Math.max(root.scrollHeight, body?.scrollHeight || 0, innerHeight) },
34
36
  scroll: { x: scrollX, y: scrollY }, dpr: devicePixelRatio,
@@ -37,7 +39,7 @@ export async function collectScreenshotSample(options) {
37
39
  try {
38
40
  const data = await runSessionCommandData({ ...options, args: ["eval", "--stdin"], stdin: script, timeoutMs: 2_000 });
39
41
  const sample = isRecord(data) ? data.result : undefined;
40
- if (!isRecord(sample) || typeof sample.url !== "string" || !["main", "child"].includes(String(sample.frame))
42
+ if (!isRecord(sample) || (sample.rendering !== undefined && sample.rendering !== "text") || typeof sample.url !== "string" || !["main", "child"].includes(String(sample.frame))
41
43
  || !finiteRecord(sample.viewport, ["width", "height"]) || !finiteRecord(sample.document, ["width", "height"])
42
44
  || !finiteRecord(sample.scroll, ["x", "y"]) || !finiteRecord(sample.visualViewport, ["x", "y", "scale"])
43
45
  || typeof sample.childFrameCount !== "number" || !Number.isInteger(sample.childFrameCount) || sample.childFrameCount < 0
@@ -53,6 +55,8 @@ export async function collectScreenshotSample(options) {
53
55
  export function buildScreenshotGeometry(options) {
54
56
  const { before, after, capture, pixels } = options;
55
57
  const unknown = (reason) => ({ status: "unknown", reason, before, after });
58
+ if (options.rendering === "text" || before?.rendering === "text" || after?.rendering === "text")
59
+ return unknown(LIGHTPANDA_IMAGE_REASON);
56
60
  if (!before || !after)
57
61
  return unknown("Capture was not bracketed by browser geometry samples; coordinates are unknown.");
58
62
  if (JSON.stringify(before) !== JSON.stringify(after))
@@ -100,11 +100,12 @@ export async function withNativeSessionDefaults(input, cwd, signal, run, root) {
100
100
  const configuredSession = env.AGENT_BROWSER_SESSION ?? identity.session;
101
101
  const launchArgs = scanUpstreamGlobalFlagOccurrences(input.toolArgs, "--args").at(-1)?.value ?? env.AGENT_BROWSER_ARGS ?? identity.args;
102
102
  const engine = scanUpstreamGlobalFlagOccurrences(input.toolArgs, "--engine").at(-1)?.value ?? env.AGENT_BROWSER_ENGINE ?? identity.engine;
103
+ const chromeEngine = engine === undefined || engine === "chrome";
103
104
  const provider = scanUpstreamGlobalFlagOccurrences(input.toolArgs, "--provider").at(-1)?.value ?? scanUpstreamGlobalFlagOccurrences(input.toolArgs, "-p").at(-1)?.value ?? env.AGENT_BROWSER_PROVIDER ?? identity.provider;
104
105
  const tokens = parseArgvDescriptor(input.toolArgs).upstreamCommandTokens;
105
106
  const batchAttaches = requestsConnection(tokens, input.toolStdin);
106
107
  const localChrome = browserCommand && !attachment && !batchAttaches && provider === undefined
107
- && (engine === undefined || engine === "chrome");
108
+ && chromeEngine;
108
109
  const chromeStartupArgs = localChrome ? ["--no-startup-window", ...(typeof launchArgs === "string" ? [launchArgs] : [])].join(",") : undefined;
109
110
  const persistentChromeArgs = hasLocalLaunchDefaults(identity, env) ? chromeStartupArgs : undefined;
110
111
  const configuredChromeLaunch = persistentChromeArgs !== undefined || [...LOCAL_VALUE_DEFAULTS, ...LOCAL_BOOLEAN_DEFAULTS, ...LOCAL_ARRAY_DEFAULTS, "hideScrollbars", "clearCaCert"].some(key => {
@@ -147,9 +148,9 @@ export async function withNativeSessionDefaults(input, cwd, signal, run, root) {
147
148
  if (restoreEligible && bootstrap)
148
149
  restore = rootDefault;
149
150
  const profile = scanUpstreamGlobalFlagOccurrences(args, "--profile").at(-1)?.value
150
- ?? (bootstrap && restoreEligible && identity.profile === undefined && env.AGENT_BROWSER_PROFILE === undefined ? root?.profile : undefined);
151
+ ?? (bootstrap && chromeEngine && restoreEligible && identity.profile === undefined && env.AGENT_BROWSER_PROFILE === undefined ? root?.profile : undefined);
151
152
  const executablePath = scanUpstreamGlobalFlagOccurrences(args, "--executable-path").at(-1)?.value
152
- ?? (bootstrap && identity.executablePath === undefined && env.AGENT_BROWSER_EXECUTABLE_PATH === undefined ? root?.executablePath : undefined);
153
+ ?? (bootstrap && chromeEngine && identity.executablePath === undefined && env.AGENT_BROWSER_EXECUTABLE_PATH === undefined ? root?.executablePath : undefined);
153
154
  return withAgentBrowserProcessEnvironment({
154
155
  ...(restore !== undefined ? { AGENT_BROWSER_RESTORE: restore } : {}),
155
156
  ...(profile !== undefined ? { AGENT_BROWSER_PROFILE: profile } : {}),
@@ -14,7 +14,7 @@ export const QUICK_START_GUIDELINES = [
14
14
  "Example agent_browser_code: { code: 'const result = await browser({args:[\"get\",\"title\"]}); if (!result.success) { emit(result); } else { emit({title:result.data}); }' }. Native batch still avoids extra CLI processes for fixed segments inside code. Explicit session/namespace controls select the browser for this cell without changing the default browser.",
15
15
  "Call agent_browser_tools with no fields for inventory, or { enable: [\"action\", \"qa\", \"electron\", \"source\", \"network\"] } for advanced native tools. It adds agent_browser_action, agent_browser_qa, agent_browser_electron, agent_browser_source, and agent_browser_network_source without disabling other tools. The browser instructions cover these capabilities; activation reveals their selected tool declarations.",
16
16
  "Use read <url> for readable web text; get title/url, get text/html/value/count <selector>, or get attr <selector> <name> for targeted state. For eval --stdin, put JavaScript in stdin and return the desired value. outputPath saves result data and must differ from screenshot/download/recording paths. Use --json only when JSON text is needed.",
17
- "For artifacts, verify artifactVerification and artifacts before reporting success. A pending recording is not verified output; install ffmpeg before starting and pair start with stop. Native download owns the click and saves Blob/redirect downloads. Close keeps explicitly saved files. Save promptGuard-required paths before close. Screenshot pixels are not browser CSS coordinates: use imageObservations geometry (dimensions, viewport, crop/origin, scroll and DPR) before any visual-coordinate action; full-page and element captures are not current-viewport coordinates, and Pi resizing only maps sent pixels to original image pixels.",
17
+ "For artifacts, verify artifactVerification and artifacts before reporting success. A pending recording is not verified output; install ffmpeg before starting and pair start with stop. Native download owns the click and saves Blob/redirect downloads. Close keeps explicitly saved files. Save promptGuard-required paths before close. Screenshot pixels are not browser CSS coordinates: use imageObservations geometry (dimensions, viewport, crop/origin, scroll and DPR) before any visual-coordinate action; full-page and element captures are not current-viewport coordinates, and Pi resizing only maps sent pixels to original image pixels. Lightpanda images are text-rendered, not graphical layout; rendering:text never supports CSS/mouse-coordinate input.",
18
18
  "Follow model-visible nextActions when present, including identity and stdin, instead of guessing commands. Treat native pending confirmations and unknown mutation outcomes as inspection boundaries. For dense snapshots, inspect Omitted high-value controls before reading a full spill.",
19
19
  ];
20
20
  export const ADVANCED_TOOL_PROMPT_GUIDELINES = {
@@ -61,6 +61,7 @@ export const SHARED_BROWSER_PLAYBOOK_GUIDELINES = [
61
61
  "Cooperating Pi processes serialize each complete browser operation and code cell by native socket context, namespace, and session, including target/ref helpers and state updates. Different identities remain concurrent; namespace-wide close waits for matching operations. All participating Pi instances must load this release; humans, third-party clients, and older extensions are outside that coordination. It is not a transaction or rollback. For raw batches whose later content depends on navigation, use exact batch --bail or split the calls.",
62
62
  "After a successful `connect`, `--cdp`, or enabled `--auto-connect` call, verify with get url and keep using the resulting session without repeating the attach flag. The wrapper remembers that attachment across active-branch reload/resume and live-checks the URL before later page reads/interactions because an attached browser can drift externally; caller config, file access, launch arguments, and environment pass through unchanged. A successful close clears the marker. When several named sessions share one Chrome, pass --pin-tab once (AGENT_BROWSER_PIN_TAB) so a closed bound tab fails as tab_gone instead of acting on a neighbor; recover with tab new or tab list. --no-pin-tab turns the sticky pin off. tab list includes each tab's CDP targetId, accepted as a tab ref.",
63
63
  `If you already used the implicit session and now need launch-scoped flags (${LAUNCH_SCOPED_FLAG_LABEL}), retry with top-level sessionMode set to fresh or pass an explicit --session for the new launch; never pass --session-mode inside args. After a successful unnamed fresh launch, later auto calls follow that new session.`,
64
+ "Chrome is the default. Use separately installed Lightpanda only when explicitly selected with --engine lightpanda, environment or native config; --engine is launch-scoped, so use sessionMode:fresh to change a managed browser. Lightpanda text-rendered screenshots do not prove graphical layout/visibility or mouse coordinates. Chrome profiles, headed mode, multi-tab, WebGPU and native WebMCP are not parity capabilities; choose Chrome explicitly when needed, never assume automatic fallback.",
64
65
  "For WebGPU pages, use args [\"--webgpu\", \"open\", \"<url>\"] on a fresh local browser launch; use doctor --webgpu (or --headed on Linux/Windows capture paths) to prove rendering before trusting a non-black screenshot. WebGPU cannot be combined with --cdp, --auto-connect, or provider launches unless --webgpu false overrides an enabled config/environment default.",
65
66
  "For experimental WebMCP page tools, use webmcp list, then webmcp invoke <tool> with --params and optional --frame/--detach/--timeout; use webmcp result or cancel for detached calls. Locally managed Chrome enables WebMCP by default; a positive navigation hint means the page has tools to list. --no-webmcp is launch-scoped and requires a fresh session; invoke/result/cancel can mutate or navigate, so refresh snapshot refs afterward.",
66
67
  "For --allowed-domains, use a fresh local Chrome context. Upstream rejects CDP/auto-connect, profiles, restore/state replay, direct-page providers, iOS/Safari, and startup/profile Chrome args because they cannot guarantee containment; Chromium also disables RTCPeerConnection while the allowlist is active.",
@@ -110,20 +111,20 @@ export const RUNTIME_PROMPT_GUIDELINES = [
110
111
  "For agent_browser, use open → snapshot -i → current @refs; re-snapshot after changes. Batch same-snapshot fills before submit; split on rerenders/navigation. Native args, profiles, auth, providers, paths, config and environment pass through. Use requested/configured profiles and verify authentication after restart.",
111
112
  "agent_browser bare calls share a root Pi browser with descendants, not unrelated roots; coordinate navigation within the group. Native explicit sessions win. sessionMode:fresh intentionally launches a separate managed browser. Never close an unrelated browser.",
112
113
  "For agent_browser and agent_browser_code, follow visible nextActions. Inspect unknown mutation outcomes before retry. Use exact user paths; verify artifactVerification/artifacts before success claims. Save promptGuard-required files before close; close keeps files; waited:timeout proves nothing.",
113
- "For agent_browser screenshot coordinates, use imageObservations dimensions/viewport/crop/origin/scroll/DPR; full-page and element images are not viewport CSS coordinates. Pi image resizing maps sent pixels to original image pixels only. Prefer refs; inspect screenshots only when the task needs vision.",
114
+ "For agent_browser screenshot coordinates, use imageObservations dimensions/viewport/crop/origin/scroll/DPR; full-page and element images are not viewport CSS coordinates. Pi image resizing maps sent pixels to original image pixels only. Lightpanda rendering:text images never establish CSS/mouse coordinates. Prefer refs; inspect screenshots only when the task needs vision.",
114
115
  "In agent_browser workflows, ordinary requested non-destructive submissions may proceed in authenticated unattended/auto-approved flows. Honor explicit stops; require explicit authorization for purchases, production-control, destructive/irreversible, or account/security/privacy changes.",
115
116
  "agent_browser: read <url> for text, get title/url, get text/html/value/count <selector>, get attr <selector> <name>; heed visibility warnings and Omitted high-value controls. Verify nested scrolling via snapshot/screenshot. Use --json only for JSON text; recording needs ffmpeg before start and a verified stop.",
116
117
  ];
117
118
  export function buildBrowserExecutablePathGuideline(executablePath) {
118
119
  if (!executablePath)
119
120
  return undefined;
120
- return `agent_browser config sets browser.executablePath to ${JSON.stringify(executablePath)}. Global/override values bootstrap inactive automatic root browsers; bare calls preserve active native launch settings. Project-only values remain guidance for explicit --executable-path launches. Use sessionMode:fresh only for an intentionally separate managed browser, not routine root-group browsing. Upstream profiles lists Chrome profiles only; use a supported explicit profile path for other Chromium browsers.`;
121
+ return `agent_browser config sets browser.executablePath to ${JSON.stringify(executablePath)}. Global/override values bootstrap inactive automatic root Chrome browsers, not Lightpanda; bare calls preserve active native launch settings. Project-only values remain guidance for explicit --executable-path launches. Use sessionMode:fresh only for an intentionally separate managed browser, not routine root-group browsing. Upstream profiles lists Chrome profiles only; use a supported explicit profile path for other Chromium browsers.`;
121
122
  }
122
123
  export function buildBrowserDefaultProfileGuideline(profile) {
123
124
  if (!profile || profile.policy === "explicit-only")
124
125
  return undefined;
125
126
  if (profile.policy === "always") {
126
- return `agent_browser config sets browser.defaultProfile.name to ${JSON.stringify(profile.name)} with policy always. Global/override Chrome profile names bootstrap automatic root browsers through independent native copies with per-root restore; bare calls and helpers preserve the active native launch settings without resending bootstrap options. Paths and project-only settings stay advisory. Explicit unrelated sessions are unchanged. Verify actual app auth after restart: native JSON restore omits new IndexedDB and memory-only credentials; use a supported per-root persistent profile if needed. Profile content is model-visible user data.`;
127
+ return `agent_browser config sets browser.defaultProfile.name to ${JSON.stringify(profile.name)} with policy always. Global/override Chrome profile names bootstrap automatic root Chrome browsers, not Lightpanda, through independent native copies with per-root restore; bare calls and helpers preserve the active native launch settings without resending bootstrap options. Paths and project-only settings stay advisory. Explicit unrelated sessions are unchanged. Verify actual app auth after restart: native JSON restore omits new IndexedDB and memory-only credentials; use a supported per-root persistent profile if needed. Profile content is model-visible user data.`;
127
128
  }
128
129
  return `agent_browser config sets browser.defaultProfile.name to ${JSON.stringify(profile.name)}; for signed-in/account-specific browser tasks, reuse a configured native shared session with bare calls, otherwise start with --profile ${JSON.stringify(profile.name)} plus sessionMode:fresh unless the user asks for a different profile.`;
129
130
  }
@@ -6,6 +6,7 @@ import { getExplicitArtifactDestination } from "../../orchestration/browser-run/
6
6
  import { isRecord, parsePositiveInteger } from "../../parsing.js";
7
7
  import { formatSessionArtifactRetentionSummary, getSessionArtifactManifestEntryKey, isPendingRecordingArtifact, isPendingRecordingCommand, mergeSessionArtifactManifest, } from "../artifact-manifest.js";
8
8
  import { classifyAgentBrowserSuccessCategory } from "../categories.js";
9
+ import { LIGHTPANDA_IMAGE_REASON } from "./common.js";
9
10
  import { formatRecordingReceipt, getRecordingReceipt } from "../recording.js";
10
11
  const PNG_HEADER = Buffer.from("89504e470d0a1a0a0000000d49484452", "hex");
11
12
  const INLINE_IMAGE_MAX_BYTES_ENV = "PI_AGENT_BROWSER_INLINE_IMAGE_MAX_BYTES";
@@ -587,11 +588,16 @@ export async function attachInlineImage(presentation, imagePath, modelVisible =
587
588
  return presentation;
588
589
  const dimensions = getImageDimensions(header.toString("base64"), mimeType);
589
590
  presentation.imagePath = imagePath;
591
+ const lifecycle = isRecord(presentation.data) && isRecord(presentation.data.lifecycle) ? presentation.data.lifecycle : undefined;
592
+ const textRendered = isRecord(lifecycle?.effectiveLaunch) && lifecycle.effectiveLaunch.engine === "lightpanda";
590
593
  presentation.imageObservations = [{ path: imagePath, mimeType,
594
+ ...(textRendered ? { rendering: "text" } : {}),
591
595
  pixels: dimensions ? { width: dimensions.widthPx, height: dimensions.heightPx } : undefined,
592
- capture: "unknown", geometry: { status: "unknown", reason: dimensions ? "Capture geometry was not observed." : "Image dimensions could not be read from the bounded header." } }];
596
+ capture: "unknown", geometry: { status: "unknown", reason: textRendered ? LIGHTPANDA_IMAGE_REASON : dimensions ? "Capture geometry was not observed." : "Image dimensions could not be read from the bounded header." } }];
593
597
  if (!modelVisible)
594
598
  return presentation;
599
+ if (textRendered)
600
+ appendPresentationNotice(presentation, LIGHTPANDA_IMAGE_REASON);
595
601
  const inlineImageMaxBytes = getInlineImageMaxBytes();
596
602
  if (fileStats.size > inlineImageMaxBytes) {
597
603
  appendPresentationNotice(presentation, `Image attachment skipped: ${formatByteCount(fileStats.size)} exceeds the inline limit of ${formatByteCount(inlineImageMaxBytes)}.`);
@@ -2,6 +2,7 @@ import { isRecord } from "../../parsing.js";
2
2
  import { redactSensitiveText, redactSensitiveValue } from "../../runtime.js";
3
3
  import { stringifyUnknown, truncateText } from "../text.js";
4
4
  const UNTITLED_PAGE_SUMMARY = "(untitled page)";
5
+ export const LIGHTPANDA_IMAGE_REASON = "Lightpanda screenshot is a text-rendered page representation, not a graphical capture; CSS/mouse coordinate mapping is unknown.";
5
6
  export function stringifyModelFacing(value) {
6
7
  return stringifyUnknown(redactSensitiveValue(value));
7
8
  }
@@ -64,7 +64,7 @@ All participating Pi processes must restart with the updated package. The lock i
64
64
 
65
65
  Native tool declarations share Browser namespace metadata and `outputSchema`; `structuredContent` reuses the bounded redacted observation or its existing compact/spill object. Loader inventory and normalized provider search keep their existing shapes. Internal ownership/replay details are not exported wholesale. This interoperates with host-selected native codemode without auto-enabling it or replacing the Browser VM. A path/handle is not native image data; verified content image blocks remain separate. Final results return native `isError` and retain prose failure notices while explicit JSON stays parseable and unchanged. Per-row TUI memoization skips unchanged formatting/Text updates without caching browser state or observations across calls.
66
66
 
67
- `orchestration/browser-run/screenshot-observation.ts` brackets standalone screenshots with CSS viewport/document/scroll/DPR/frame samples. Existing Pi image-dimension primitives measure saved pixels. Matching samples and dimensions can establish a crop mapping; changed, unmeasured batch, scrolled/ambiguous element, zoom, or frame-uncertain captures remain unknown. These samples are not an atomic capture guarantee. Native mouse input uses current viewport CSS pixels, not image pixels; full-page/element origin and any Pi resize must be accounted for separately. Unchanged conditional screenshots attach nothing. See [image observations](TOOL_CONTRACT.md#image-observations).
67
+ `orchestration/browser-run/screenshot-observation.ts` brackets standalone screenshots with CSS viewport/document/scroll/DPR/frame samples. Existing Pi image-dimension primitives measure saved pixels. Matching samples and dimensions can establish a crop mapping; changed, unmeasured batch, scrolled/ambiguous element, zoom, or frame-uncertain captures remain unknown. These samples are not an atomic capture guarantee. Native mouse input uses current viewport CSS pixels, not image pixels; full-page/element origin and any Pi resize must be accounted for separately. Unchanged conditional screenshots attach nothing. Native Lightpanda engine metadata labels verified images as `rendering: "text"` through shared attachment handling, including batch/code. The existing standalone samples also recognize the Lightpanda browser brand without another probe or engine cache. Text-rendered images retain unknown geometry and no crop/scale mapping, even when synthetic dimensions match. See [image observations](TOOL_CONTRACT.md#image-observations).
68
68
 
69
69
  ### Agent-first UX
70
70
 
@@ -89,7 +89,7 @@ Config layers merge in that order: global, project, override. The shared policy
89
89
 
90
90
  `agent_browser_web_search` availability is conditional. Startup registration uses global, override, and environment fallback config without reading project-local config before Pi trust context exists; trusted project config can register the companion tool on `session_start`, and every execution reloads the final session config so `webSearch.enabled: false` still prevents a request even if a startup credential made the tool visible. A global disable is the normal user default and can still be overridden by project config or `PI_AGENT_BROWSER_CONFIG`; a project disable applies to one repo; an explicit `PI_AGENT_BROWSER_CONFIG` file with `webSearch.enabled: false` is the highest-priority hard disable for that run. Literal and env-backed sources must resolve before they make the tool available; command-backed sources are considered configured without running the command until tool execution, so secret managers do not slow startup or prompt unexpectedly. The tool resolves the selected key lazily, chooses Exa or Brave from available credentials (preferring Exa by default unless `webSearch.preferredProvider` says otherwise), then follows one provider-agnostic execution path through provider adapters for request building, HTTP JSON fetch, response normalization, and provider-specific detail fields. Exa search-type precedence is per-call value, merged config default, then `auto`; bounded Exa filters stay on the Exa adapter and explicit Exa-only options fail before a Brave request. The tool calls Exa `/search` with highlights plus a fixed primary-source/version/date/distinct-result system prompt, or Brave Search, then removes later exact normalized-URL duplicates while retaining provider order. It does not guess across distinct paths/query URLs or overfetch replacements; provider page dates remain clues rather than claimed crawl/version evidence. Compact result details report any `duplicatesRemoved` without exposing keys.
91
91
 
92
- Global/override package profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers through scoped native environment. Active roots retain native settings; paths, other policies, and project-only values remain advisory. Direct and code calls selecting that root share the same configuration. Unrelated explicit sessions and Electron launches retain their separate native/owned rules.
92
+ Global/override package profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers through scoped native environment only when CLI → environment → native config resolves to Chrome. Lightpanda receives no wrapper Chrome defaults; explicit native settings are untouched. Active roots retain native settings; paths, other policies, and project-only values remain advisory. Direct and code calls selecting that root share the same configuration. Unrelated explicit sessions and Electron launches retain their separate native/owned rules.
93
93
 
94
94
  ### Prompt guidance budget
95
95
 
@@ -138,7 +138,7 @@ The optional `pi-change-working-dir` event protocol supplies one invocation exec
138
138
 
139
139
  An explicit native `session` default (user/project JSON or `AGENT_BROWSER_SESSION`) selects a caller-owned browser across Pi sessions. Per-call session/namespace flags win. `lib/orchestration/native-session-defaults.ts` reads session/namespace and contributing launch defaults from native config paths and uses sessionless native `--config <path> --json session` to validate each contributing file and resolve its session. Native config continues to own launch/profile/storage settings; the adapter does not duplicate its schema. Explicit config and per-call idle settings are scoped to helpers through `lib/process-environment.ts`. No startup probe, persistent config cache, or new lifecycle service is added.
140
140
 
141
- When no native session, attachment, or explicit fresh launch is selected, `sessionMode: "auto"` uses `pi-root-<sha256(root Pi ID)[0:24]>` through the existing caller-owned named-session path. Descendants receive `PI_SUBAGENT_ROOT_SESSION_ID` from pi-subagents; root processes use `ctx.sessionManager.getSessionId()`. No cwd, human fork ancestry, run registry, or process-global assignment participates. The same name is the default native restore key. Existing native daemon inspection runs inside the existing identity queue: inactive roots receive bootstrap defaults, while active roots retain native launch settings and their current restore key. Scoped subprocess environment keeps helper settings consistent; no launch-settings journal is added. Root-name explicit follow-ups retain those defaults; unrelated explicit names do not. Global/override package `browser.defaultProfile` with `policy: "always"` supplies native Chrome profile names only, plus the configured executable. Paths and project-only profile settings stay advisory. Native config/environment and explicit launch settings win. The communal source is copied by native Chrome launch, never by the wrapper, and each root's restore writes remain separate. Native JSON restore omits IndexedDB, service workers, and page-memory credentials; application-level restart verification is required. Parent and child quit leave the root browser alive; shared execution locks coordinate updated cooperating processes without a broker. An active older/fresh managed session keeps its existing lifecycle. A configured session has the same precedence over `fresh` as a literal `--session`; code follows the selected ordinary browser lifecycle; Electron launch retains its owned lifecycle. Electron launch suppresses the native session environment default only while allocating its generated session; ordinary browser follow-ups still honor the configured native default, so use the returned Electron session explicitly.
141
+ When no native session, attachment, or explicit fresh launch is selected, `sessionMode: "auto"` uses `pi-root-<sha256(root Pi ID)[0:24]>` through the existing caller-owned named-session path. Descendants receive `PI_SUBAGENT_ROOT_SESSION_ID` from pi-subagents; root processes use `ctx.sessionManager.getSessionId()`. No cwd, human fork ancestry, run registry, or process-global assignment participates. The same name is the default native restore key. Existing native daemon inspection runs inside the existing identity queue: inactive roots receive bootstrap defaults, while active roots retain native launch settings and their current restore key. Scoped subprocess environment keeps helper settings consistent; no launch-settings journal is added. Root-name explicit follow-ups retain those defaults; unrelated explicit names do not. Global/override package `browser.defaultProfile` with `policy: "always"` supplies native Chrome profile names only, plus the configured executable, only when the effective engine is Chrome. Paths and project-only profile settings stay advisory. Native config/environment and explicit launch settings win. The communal source is copied by native Chrome launch, never by the wrapper, and each root's restore writes remain separate. Native JSON restore omits IndexedDB, service workers, and page-memory credentials; application-level restart verification is required. Parent and child quit leave the root browser alive; shared execution locks coordinate updated cooperating processes without a broker. An active older/fresh managed session keeps its existing lifecycle. A configured session has the same precedence over `fresh` as a literal `--session`; code follows the selected ordinary browser lifecycle; Electron launch retains its owned lifecycle. Electron launch suppresses the native session environment default only while allocating its generated session; ordinary browser follow-ups still honor the configured native default, so use the returned Electron session explicitly.
142
142
 
143
143
  Why:
144
144
  - works out of the box
@@ -224,7 +224,7 @@ Input resolution replaces URL-less `open` with native `get url` before planning,
224
224
  `agent-browser` startup flags are sticky once a session is already running.
225
225
  The extension should surface that clearly and avoid hidden restart behavior.
226
226
 
227
- That means explicit startup-scoping flags like `--allowed-domains`, `--auto-connect`, `--args`, `--user-agent`, `--cdp`, `--enable`, `--executable-path`, `--webgpu`, `--no-webmcp`, `--headed`, `--init-script`, `--device`, `--namespace`, `--profile`, `--provider`, `-p`, `--restore`, `--restore-save`, restore check flags, `--session-name`, and `--state` should remain explicit upstream argv choices instead of being wrapped in extra hidden restart or cloning logic. The one deliberate exception is the env-only managed-session `AGENT_BROWSER_RESTORE` key above, which does not inject `--restore` into argv and therefore does not trip launch-scoped `sessionMode: "fresh"` recovery.
227
+ That means explicit startup-scoping flags like `--allowed-domains`, `--auto-connect`, `--args`, `--user-agent`, `--cdp`, `--enable`, `--engine`, `--executable-path`, `--webgpu`, `--no-webmcp`, `--headed`, `--init-script`, `--device`, `--namespace`, `--profile`, `--provider`, `-p`, `--restore`, `--restore-save`, restore check flags, `--session-name`, and `--state` should remain explicit upstream argv choices instead of being wrapped in extra hidden restart or cloning logic. The one deliberate exception is the env-only managed-session `AGENT_BROWSER_RESTORE` key above, which does not inject `--restore` into argv and therefore does not trip launch-scoped `sessionMode: "fresh"` recovery.
228
228
 
229
229
  The wrapper may still apply narrow compatibility normalizations when observed behavior justifies them and the result remains thin, local, and opt-out. For example, OpenAI web properties and `dash.cloudflare.com` reject the default local `HeadlessChrome` user agent while the same flow works with a normal Chrome UA, so the extension injects a domain-specific fallback only when the caller did not already choose raw Chrome arguments, a custom user agent, headed mode, CDP, auto-connect, a provider-backed launch, or a non-Chrome engine through argv or matching upstream environment. Managed sessions retain the injected value as per-session wrapper state across helper calls and branch reload/resume. Active daemons omit both launch forms so upstream does not replace a launch-configured browser; a session proven inactive receives the retained compatibility launch values, including the same fixed compatibility value as a comma-safe Chrome launch argument covering tabs and SSO popups on supported native versions that do not inherit per-page CDP overrides. Wrapper-owned headed launches also default upstream periodic restore autosave off because agent-browser 0.33.2 collects non-current origins through visible temporary targets while holding the daemon state lock; save-on-close remains enabled, and an explicit `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` value opts in when the daemon launches. The effective interval is retained in owned-session state and transcript results; changing it in either direction on a running wrapper-owned headed daemon is rejected until close plus a fresh launch.
230
230
 
@@ -237,6 +237,7 @@ Implementation detail lives in `extensions/agent-browser/lib/launch-scoped-flags
237
237
  - **Command discovery:** Leading argv is scanned with a value-taking allowlist so known global flags and documented command flags consume their values before the upstream command word is identified. Missing-value prevalidation is intentionally limited to upstream global value flags; command-scoped flags and literal text are left to upstream parsing so values like `fill #field --password` are not rejected by wrapper heuristics before the CLI sees them. Upstream 0.35.0 and newer accept `--restore=<key>` but reject other global `--flag=value` tokens during normal command execution, so the wrapper rejects those unsupported assignments before they can be mistaken for command or artifact operands. Plain help/version inspection preserves exact caller argv because upstream accepts those top-level shapes. Nested batch rows do not parse global flags, so their equals forms never receive the top-level inspection or restore exceptions. When upstream adds new global flags that take values ahead of the command, extend both the command-discovery and prevalidation allowlists; when it adds command-specific flags, extend only command discovery/redaction as needed. A smaller set of global boolean flags may be followed by an optional `true`/`false` literal; when present, that literal is consumed as the flag value before command discovery continues.
238
238
  - **`--state` disambiguation:** Persisted browser `--state` before the command participates in launch-scoped validation and tab-correction hints. The same flag spelling after a `wait` command is excluded from startup-scoped detection so upstream help examples such as `wait @ref --state hidden` do not spuriously require `sessionMode: "fresh"` while an implicit session is active. As of the current upstream baseline, the parser still does not implement those `wait --state` examples as distinct wait modes, so agent-facing docs recommend `wait --fn` predicates for disappearance checks instead.
239
239
  - **`--auto-connect`:** Treated as launch-scoped only when enabled (`--auto-connect` bare or `true`). `--auto-connect false` is ignored for startup-scoped blocking so disabled attach hints do not force a fresh launch.
240
+ - **`--engine`:** Treated as launch-scoped for Chrome and Lightpanda, using the same fresh-session recovery instead of letting native engine changes replace a running managed page. Caller-owned sessions remain native-owned. Chrome stays default; Lightpanda is opt-in with no bundle, second driver, emulation or automatic fallback. See [engine limits](COMMAND_REFERENCE.md#lightpanda-engine).
240
241
  - **`--webgpu`:** Treated as launch-scoped for both enabled and explicit `false` values. Enabled WebGPU selects upstream's platform-specific local-launch preset; explicit false can override an environment/config default and still belongs to a fresh browser launch. Upstream rejects enabled WebGPU with CDP, auto-connect, or provider launches.
241
242
  - **`--no-webmcp`:** Treated as launch-scoped for bare/`true` and explicit `false` values because it selects whether upstream 0.36.0 enables experimental WebMCP for a locally managed Chrome launch.
242
243
  - **`--headed`:** Treated as launch-scoped for both enabled and explicit `false` values so a visible-window choice cannot be silently ignored by an already-running managed session.
@@ -516,7 +516,7 @@ The upstream screenshot aliases are `screenshot --full` for full-page capture an
516
516
 
517
517
  Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. Native download always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper never substitutes a fetch of the anchor's `href`. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
518
518
 
519
- Screenshot `imageObservations` includes verified pixel dimensions and capture kind. Matching pre/post CSS viewport, scroll, DPR, and crop samples may establish a measured document-space mapping; otherwise geometry remains unknown, including unmeasured batches and uncertain scrolled/iframe element captures. Native mouse uses current viewport CSS pixels. Account separately for crop origin, current scroll/frame, and any Pi resize before using coordinates; prefer refs. See [image geometry](TOOL_CONTRACT.md#image-observations). Code only attaches images selected with `emitImage` from a current-cell handle.
519
+ Screenshot `imageObservations` includes verified pixel dimensions and capture kind. Positively identified Lightpanda images also carry `rendering: "text"`: their text representation is not graphical layout and never establishes CSS/mouse coordinates. Matching pre/post CSS viewport, scroll, DPR, and crop samples may establish a measured document-space mapping; otherwise geometry remains unknown, including unmeasured batches and uncertain scrolled/iframe element captures. Native mouse uses current viewport CSS pixels. Account separately for crop origin, current scroll/frame, and any Pi resize before using coordinates; prefer refs. See [image geometry](TOOL_CONTRACT.md#image-observations). Code only attaches images selected with `emitImage` from a current-cell handle.
520
520
 
521
521
  For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
522
522
 
@@ -989,7 +989,7 @@ Example config:
989
989
  }
990
990
  ```
991
991
 
992
- Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
992
+ Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers when the effective engine is Chrome. Lightpanda omits these wrapper defaults, without changing caller-selected launch settings. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
993
993
 
994
994
  ## Important global flags, config, and environment
995
995
 
@@ -1032,13 +1032,33 @@ The extension composes stock `--args --no-startup-window` for local Chrome start
1032
1032
  - `--cdp <port|url>`: connect through Chrome DevTools Protocol. Use it, `--auto-connect`, or `connect <port|url>` once on a named/fresh session, verify with `get url`, then reuse that session without repeating the attach flag. After a successful attachment the wrapper avoids re-emitting its own compatibility launch argument; caller launch/config/file-access settings remain unchanged. Content-bearing first use is blocked until URL verification; later page reads/interactions live-check the URL because the attached browser can drift externally. `close` clears attachment state.
1033
1033
  - `--color-scheme <scheme>`: `dark`, `light`, or `no-preference`. Environment: `AGENT_BROWSER_COLOR_SCHEME`.
1034
1034
  - `--download-path <path>`: default browser download directory. Environment: `AGENT_BROWSER_DOWNLOAD_PATH`.
1035
- - `--engine <name>`: browser engine, `chrome` by default or `lightpanda`. Environment: `AGENT_BROWSER_ENGINE`.
1035
+ - `--engine <name>`: browser engine, `chrome` by default or `lightpanda`. Environment: `AGENT_BROWSER_ENGINE`. Launch-scoped; use `sessionMode: "fresh"` when changing a managed browser. See [Lightpanda](#lightpanda-engine).
1036
1036
 
1037
1037
  On Android/Termux, follow the README setup to install the packaged Linux-musl arm64 upstream binary, install Termux's `which`, and expose its launcher as `$PREFIX/bin/chromium`. Prefer that upstream system-browser discovery over ambient `AGENT_BROWSER_EXECUTABLE_PATH`: it survives isolated `HOME` values and works for ordinary native calls. Code retains the selected browser's normal launch configuration. Wrapper-generated Android managed identities use a compact 80-bit digest so ordinary namespaces and fresh rotations fit upstream's Unix socket path.
1038
1038
 
1039
1039
  - `--no-auto-dialog`: disable automatic dismissal of alert/beforeunload dialogs. Environment: `AGENT_BROWSER_NO_AUTO_DIALOG`.
1040
1040
  - `--idle-timeout <ms>`: native background browser lifecycle setting (also accepts `10s`, `3m`, `1h`). Caller-owned sessions retain native idle policy; an explicit flag is carried to every helper in that call. Keep it consistent between calls to avoid a native daemon restart. Only wrapper-owned sessions receive the implicit timeout and numeric mismatch check against `PI_AGENT_BROWSER_IMPLICIT_SESSION_IDLE_TIMEOUT_MS`.
1041
1041
 
1042
+ ### Lightpanda engine
1043
+
1044
+ [Lightpanda 1.0](https://lightpanda.io/blog/posts/lightpanda-1-0) is an optional headless engine, installed separately from this package and `agent-browser`. The [native engine guide](https://github.com/vercel-labs/agent-browser/blob/v0.38.2/docs/src/app/engines/lightpanda/page.mdx) describes setup. Put its `lightpanda` executable on `PATH`; select it with `--engine lightpanda`, `AGENT_BROWSER_ENGINE=lightpanda`, or native config `engine`. Chrome remains the default.
1045
+
1046
+ ```json
1047
+ { "args": ["--engine", "lightpanda", "open", "https://example.com"], "sessionMode": "fresh" }
1048
+ { "args": ["snapshot", "-i"] }
1049
+ { "args": ["screenshot", "/tmp/lightpanda-text.png"] }
1050
+ { "args": ["close"] }
1051
+ ```
1052
+
1053
+ - Engine selection belongs on launch, not routine follow-ups. Changing an active wrapper-managed browser returns the existing fresh-session recovery; caller-owned sessions retain native lifecycle semantics.
1054
+ - Package `browser.defaultProfile` / `browser.executablePath` Chrome bootstrap defaults are omitted when CLI → environment → native config selects Lightpanda. Caller-selected profile, executable, config and environment values remain untouched; upstream validates combinations.
1055
+ - `open`, snapshots/refs, form interaction, getters, `eval --stdin`, native batch and browser code can work, but successful dispatch is not Chrome-parity visibility or layout evidence. By default Lightpanda skips external stylesheets, images, iframes and workers. Snapshot/hidden-element and overlay/scroll assumptions differ from Chrome.
1056
+ - Screenshots are PNGs rendering page markdown/text. Native engine metadata identifies them for direct, batch and selected code output; standalone geometry samples can also recognize the browser's Lightpanda brand on attachments or older native results. Identified images carry `rendering: "text"` with unknown geometry and no crop/scale mapping. They remain verified image attachments, not graphical captures or a basis for mouse coordinates. Missing provenance stays unknown.
1057
+ - Lightpanda currently permits one target per browser context; `tab new` can fail. Use separate native sessions for independent pages, or Chrome when the task needs multiple tabs. Do not infer broad authentication/restore parity from a single-origin storage result.
1058
+ - Native Lightpanda launch rejects Chrome profiles/extensions, headed mode, launch `--state`, WebGPU, file-access switches, Chromium `--args`, and private CA trust. Native WebMCP currently rejects this engine even though Lightpanda advertises its own WebMCP support. There is no wrapper workaround or automatic Chrome fallback; choose a fresh Chrome launch explicitly when those capabilities are needed.
1059
+ - Lightpanda 1.0 enforces CORS by default, so cross-origin page `fetch`/XHR can fail where earlier Lightpanda did not. Preserve that protection; navigation is a separate capability.
1060
+ - The 1.0 release provides macOS/Linux x86_64/ARM64 binaries (Linux glibc), not native Windows/Android. Remote CDP or WSL is a separate setup, not native-platform qualification. Telemetry is upstream-owned; `LIGHTPANDA_DISABLE_TELEMETRY=true` opts out.
1061
+
1042
1062
  ### Output, provider, policy, and AI flags
1043
1063
 
1044
1064
  - `--json`: JSON output. The wrapper injects this automatically for normal tool execution. Environment: `AGENT_BROWSER_JSON`.
@@ -1080,7 +1100,7 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
1080
1100
  ## Wrapper-specific behavior worth knowing
1081
1101
 
1082
1102
  - The extension may keep following one implicit managed session across later tool calls.
1083
- - If launch-scoped flags like `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `--provider` / `-p`, or provider device flags like `--device` would replace or be ignored by an already-active managed session, retry with `sessionMode: "fresh"`. When the call explicitly names the current managed session, the structured recovery payload removes that `--session` so the fresh rotation can succeed.
1103
+ - If launch-scoped flags like `--engine`, `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `--provider` / `-p`, or provider device flags like `--device` would replace or be ignored by an already-active managed session, retry with `sessionMode: "fresh"`. When the call explicitly names the current managed session, the structured recovery payload removes that `--session` so the fresh rotation can succeed.
1084
1104
  - If a `sessionMode: "fresh"` call fails (including upstream failure, timeout, missing binary, or **`qa`** reclassification after a nominally successful batch), read `details.managedSessionOutcome` before assuming where the next default call will go: `preserved` means the prior managed session remains current, while `abandoned` means no managed session became current. When the failure reason is not the fresh launch itself—for example `failureCategory: "qa-failure"`—`status`/`summary` may still describe the managed-session transition while `succeeded` on this object matches the final tool outcome.
1085
1105
  <!-- agent-browser-playbook:start wrapper-tab-recovery -->
1086
1106
  <!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
@@ -72,8 +72,9 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
72
72
  - Hold a cooperating-process execution lock keyed by actual socket context, canonical namespace, and session across direct helper/action work or the entire code cell. Use ordered multi-identity acquisition for replacement/cleanup and a namespace barrier for direct close-all. This does not coordinate humans, third-party clients, or older versions, guarantee atomic browser actions, or provide rollback.
73
73
  - Persist one canonical begin/finish event format for direct/code/host work through native Pi custom entries and a shared reducer. Define full captures once, preserve exact targets/native identity, and persist artifact-row changes rather than unchanged views. Interrupted outcomes remain unknown until inspection; failed begin blocks dispatch and failed finish stops dependent work without mutation retry. Require actual journal publication before browser dispatch. Legacy browser/lease decoders belong only to validated offline stopped-copy conversion; preserve original bytes, all ancestry and owner UUIDs. A fork must never acquire its parent's cleanup rights. Bound extension replay with token projection and winning definitions on both hosts, while disclosing stock official Pi's eager native loader and individual-value memory ceilings; see [`SESSION_STORAGE.md`](SESSION_STORAGE.md).
74
74
  - Use one canonical model-visible observation for JSON/prose/code with success/data/error, exact recovery, artifact receipts, and image observations. Native `outputSchema`/`structuredContent` under the Browser tool namespace expose that same bounded redacted projection, not raw internal details. Loader inventory and normalized search data keep their existing shapes. Details is audit/replay metadata, not the sole source of actionable failures. Return canonical native `isError` without a result-repair hook; preserve prose notices and explicit JSON. Inner calls avoid full model rendering/spills/image encoding. Bound outer output and name a complete redacted observation spill rather than truncating executable payloads.
75
- - Attach only verified real images; code selects current-cell image handles explicitly. Report pixel dimensions and CSS viewport/DPR/crop/scroll/frame evidence conservatively. Unknown geometry remains unknown; neither full-page nor element captures are assumed viewport coordinates. Do not inject model-specific image-detail/provider payload changes.
75
+ - Attach only verified real images; code selects current-cell image handles explicitly. Report pixel dimensions and CSS viewport/DPR/crop/scroll/frame evidence conservatively. Unknown geometry remains unknown; neither full-page nor element captures are assumed viewport coordinates. Positively identified Lightpanda images are text-rendered, never graphical CSS/mouse-coordinate evidence. Do not inject model-specific image-detail/provider payload changes.
76
76
  - Direct upstream capability remains available: explicit sessions, state/restore paths, profiles, config, file access, launch arguments, environment, local pages, output paths, and close arguments pass through under ordinary native semantics. Keep all session/state rows visible. Wrapper validation concerns input shape, lifecycle integrity, and truthful evidence, not authorization of resources.
77
+ - Keep Chrome default and separately installed Lightpanda opt-in. Treat `--engine` as launch-scoped through existing managed fresh-launch recovery. Omit only wrapper-supplied Chrome profile/executable defaults when Lightpanda is selected; preserve native caller settings and unsupported-command errors. No new driver, engine cache, browser bundle, or automatic Chrome fallback.
77
78
  - Native download owns clicking and saving, including loopback, Blob exports, and redirects. Create artifact directories before dispatch and verify saved files; output-data writes must not alias browser artifacts. Pending recording is not saved video.
78
79
  - Maintain a deliberate 0.7 migration map and package-level rollback that preserves saved authentication, explicit artifacts, and Pi transcripts. Restart every cooperating Pi participant on the updated package before relying on shared ordering. No universal performance, sandbox, authentication-persistence, or Windows qualification claims.
79
80
 
@@ -40,6 +40,7 @@ The compact direct/code/loader surface replaces public multimode inputs while re
40
40
  | RQ-0142 code | Fresh permissioned Node child over persistent parent-selected browser; explicit JSON/current-cell image output, visible failures/counts, ordinary auth/config/environment, ordered Pi intent/completion transitions, child cancellation/reaping, no legacy public execution. |
41
41
  | Hotpath boundaries | Unrelated tools/harmless Bash return before prompt policy. Latest raw user intent is seeded on start/tree, updated by `message_end` and refreshed after native content replacement. Captured admission uses a validated append-only prefix; one queued replay restores branch/lease state and folds per-session launch settings once. Per-row renderer memoization skips unchanged formatting. No broad config/trust/history cache, new wait/timer or budget increase. |
42
42
  | Shared ordering | Actual socket/namespace/session lock spans helper/action or entire cell; sorted identity sets for replacement/cleanup and namespace barrier for direct close-all. Requires every participating Pi to run the updated package; no human/third-party/older-version guarantee or transaction. |
43
+ | Lightpanda engine | Chrome remains default; separately installed Lightpanda is opt-in. `--engine` uses existing managed fresh-launch recovery; effective CLI/env/config Lightpanda omits wrapper Chrome profile/executable defaults while preserving native choices. Positively identified images are text-rendered with unknown geometry, not visual/coordinate parity. No automatic fallback. `test/agent-browser.runtime.test.ts`, `test/agent-browser.shared-defaults.test.ts`, and `test/agent-browser.screenshot-geometry.test.ts` own the regressions; [limits](COMMAND_REFERENCE.md#lightpanda-engine). |
43
44
  | Observation and geometry | Canonical bounded JSON/prose/code/native `structuredContent` observations expose exact recovery and receipts, with native namespace/output-schema metadata and canonical returned `isError`; complete observation spills preserve large output. Image dimensions/CSS viewport/DPR/crop/scroll/frame evidence is measured or explicitly unknown; no viewport-coordinate assumptions for full-page/element images. |
44
45
  | Pi integration | Pi 1.0 floor with independently qualified official/fork graphs. Five advanced direct tools use native `defaultActive: false`; a narrow additive initial-resume fallback preserves available transcript selections without overriding explicit/defaultTools/filtered host selection. Loader/tree/reload retain native history; public `pi:instruction-groups` registration supplies one full `browser` instruction source. Managed hosts own deferred delivery; unmanaged/stock hosts receive identical full eager guidance even for generic prompts. No permission widening or legacy discovery metadata. `test/agent-browser.discovery.test.ts` covers full-text parity, dynamic ownership, and startup/late registration; tool-surface tests preserve host selections, with pipeline coverage in `test/agent-browser.pi-pipeline.test.ts`. No checkpoint or fork metadata/revision APIs, provider payload edits or image-detail overrides. |
45
46
 
@@ -115,6 +116,8 @@ Batch/ref fidelity (#122/#123/#125/#126): tab recovery uses verified native sele
115
116
 
116
117
  Overlay recovery (RQ-0073, #147) recommends only `inspect-overlay-state` after an upstream covered-click rejection, preserving `upstream-error` and the exact session/namespace. Direct, semantic, raw `find` (including `nth` and default-click), and failed native-batch paths share the matcher; hover and generic errors do not qualify. `test/agent-browser.overlay-click-recovery.test.ts` covers presentation and error envelopes, `test/agent-browser.pi-pipeline.test.ts` covers persisted errors and parseable JSON, and the opt-in `test/agent-browser.overlay-real-upstream.test.ts` checks actual covered clicks, executable inspection, and unchanged target/blocker click counters. This is not a fix for silent input dispatch (#62).
117
118
 
119
+ Lightpanda support preserves native command access rather than promising Chrome parity. Registered-tool image controls cover native metadata in direct/JSON, batch and selected code attachments plus standalone browser-brand evidence when legacy/attached native output omits or mislabels the engine. Verified image bytes remain unchanged and coordinate claims stay unknown even when dimensions match synthetic browser geometry. Root-default controls cover CLI/env/native-config precedence and untouched caller settings; engine planner controls preserve fresh/new/explicit launches. See [engine limits](COMMAND_REFERENCE.md#lightpanda-engine). Release evidence below remains separate from these focused regressions.
120
+
118
121
  Shared native defaults are covered by `test/agent-browser.shared-defaults.test.ts`: environment/file/argv precedence, caller-owned reuse across independent Pi contexts, quit ownership, native helper idle settings, unchanged implicit cleanup, and an opt-in real local-profile fixture proving shared state and code follows the selected ordinary browser rather than a disposable isolated session. Code tests cover the fresh child language boundary and persistent browser behavior; the local Electron handoff regression in `test/agent-browser.extension-electron-lifecycle.test.ts` keeps generated launch ownership despite a native session environment default; native config validation uses upstream rather than a second schema. See [shared browser defaults](../README.md#shared-browser-defaults). These tests do not promise permanent third-party authentication.
119
122
 
120
123
  Browser correctness regressions cover credential-bearing filtered network details/exports, native generated-Blob downloads with actual saved bytes, accessible-name click target identity and abstention, smooth CSS-container movement, and parseable early-result JSON. Native fixture coverage complements deterministic dispatch/cleanup and output-shape checks; release qualification remains separate.
@@ -223,10 +223,11 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
223
223
  - Do not assume Playwright selector dialects such as text=Close or button:has-text('Close') are supported wrapper syntax unless current upstream agent-browser behavior has been verified.
224
224
  - For authenticated or user-specific content explicitly requested by the user, such as feeds, inboxes, account pages, or private dashboards, use a real profile only when the user/config asks for it or profiles have been inspected; do not assume --profile Default exists on every machine. Do not use a real profile for public pages just because they are dashboards. Treat visible page content from real profiles as model-visible transcript data. On macOS, copied Chrome profiles may omit encrypted cookies, so profile selection alone is not proof of authentication; verify the target page and use a user-approved headed login once when needed. Use --auto-connect only if profile-based reuse is unavailable or the task is specifically about attaching to a running debug-enabled browser. If profile/user-data-dir resolution fails, stop retrying opens, run profiles and/or doctor through agent_browser, then report what the user needs to configure.
225
225
  - Ordinary bare calls share one native browser within your root Pi parent/subagent group; unrelated roots browse independently. Coordinate multi-call navigation inside your group with existing subagent/intercom tools. Explicit native session defaults still win. Pi quit does not close group browsers; never close an unrelated browser.
226
- - When using launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), put them on the first command for that session. If you intentionally use an explicit --session, keep using that same explicit session for follow-ups.
226
+ - When using launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --engine, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), put them on the first command for that session. If you intentionally use an explicit --session, keep using that same explicit session for follow-ups.
227
227
  - Cooperating Pi processes serialize each complete browser operation and code cell by native socket context, namespace, and session, including target/ref helpers and state updates. Different identities remain concurrent; namespace-wide close waits for matching operations. All participating Pi instances must load this release; humans, third-party clients, and older extensions are outside that coordination. It is not a transaction or rollback. For raw batches whose later content depends on navigation, use exact batch --bail or split the calls.
228
228
  - After a successful `connect`, `--cdp`, or enabled `--auto-connect` call, verify with get url and keep using the resulting session without repeating the attach flag. The wrapper remembers that attachment across active-branch reload/resume and live-checks the URL before later page reads/interactions because an attached browser can drift externally; caller config, file access, launch arguments, and environment pass through unchanged. A successful close clears the marker. When several named sessions share one Chrome, pass --pin-tab once (AGENT_BROWSER_PIN_TAB) so a closed bound tab fails as tab_gone instead of acting on a neighbor; recover with tab new or tab list. --no-pin-tab turns the sticky pin off. tab list includes each tab's CDP targetId, accepted as a tab ref.
229
- - If you already used the implicit session and now need launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), retry with top-level sessionMode set to fresh or pass an explicit --session for the new launch; never pass --session-mode inside args. After a successful unnamed fresh launch, later auto calls follow that new session.
229
+ - If you already used the implicit session and now need launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --engine, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), retry with top-level sessionMode set to fresh or pass an explicit --session for the new launch; never pass --session-mode inside args. After a successful unnamed fresh launch, later auto calls follow that new session.
230
+ - Chrome is the default. Use separately installed Lightpanda only when explicitly selected with --engine lightpanda, environment or native config; --engine is launch-scoped, so use sessionMode:fresh to change a managed browser. Lightpanda text-rendered screenshots do not prove graphical layout/visibility or mouse coordinates. Chrome profiles, headed mode, multi-tab, WebGPU and native WebMCP are not parity capabilities; choose Chrome explicitly when needed, never assume automatic fallback.
230
231
  - For WebGPU pages, use args ["--webgpu", "open", "<url>"] on a fresh local browser launch; use doctor --webgpu (or --headed on Linux/Windows capture paths) to prove rendering before trusting a non-black screenshot. WebGPU cannot be combined with --cdp, --auto-connect, or provider launches unless --webgpu false overrides an enabled config/environment default.
231
232
  - For experimental WebMCP page tools, use webmcp list, then webmcp invoke <tool> with --params and optional --frame/--detach/--timeout; use webmcp result or cancel for detached calls. Locally managed Chrome enables WebMCP by default; a positive navigation hint means the page has tools to list. --no-webmcp is launch-scoped and requires a fresh session; invoke/result/cancel can mutate or navigate, so refresh snapshot refs afterward.
232
233
  - For --allowed-domains, use a fresh local Chrome context. Upstream rejects CDP/auto-connect, profiles, restore/state replay, direct-page providers, iOS/Safari, and startup/profile Chrome args because they cannot guarantee containment; Chromium also disables RTCPeerConnection while the allowlist is active.
@@ -480,7 +481,7 @@ Example:
480
481
 
481
482
  ### `sessionMode`
482
483
 
483
- A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` selects a caller-owned browser for direct, code, action, QA, and source tools. Per-call flags still win. Like literal `--session`, this selection takes precedence over `fresh`; it reports `usedImplicitSession: false` and is not closed when Pi quits. Without a selected native session, attachment, or explicit fresh launch, auto calls select a stable root-Pi named session and root-specific native restore key. `PI_SUBAGENT_ROOT_SESSION_ID` carries the parent identity to descendants; ordinary new/fork/clone roots differ, while resume/cwd changes keep identity. Root sessions report `usedImplicitSession: false` and survive parent/child quit. Global/override package profile names with `policy: "always"` and executable defaults bootstrap only inactive automatic roots. Active roots preserve native launch settings/current restore key, including matching root-name follow-ups; unrelated explicit browsers are unchanged. Profile paths and project-only profile config remain advisory. Native restore covers cookies and web storage, not newly created IndexedDB or memory-only authentication. See the README for communal bootstrap and persistent-profile fallback. Active older/fresh managed sessions retain the lifecycle below. `agent_browser_qa` with `attached: true` can inspect this shared current session using the same live target checks. See [shared browser defaults](../README.md#shared-browser-defaults).
484
+ A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` selects a caller-owned browser for direct, code, action, QA, and source tools. Per-call flags still win. Like literal `--session`, this selection takes precedence over `fresh`; it reports `usedImplicitSession: false` and is not closed when Pi quits. Without a selected native session, attachment, or explicit fresh launch, auto calls select a stable root-Pi named session and root-specific native restore key. `PI_SUBAGENT_ROOT_SESSION_ID` carries the parent identity to descendants; ordinary new/fork/clone roots differ, while resume/cwd changes keep identity. Root sessions report `usedImplicitSession: false` and survive parent/child quit. Global/override package profile names with `policy: "always"` and executable defaults bootstrap only inactive automatic roots whose effective engine is Chrome. Lightpanda receives no wrapper-supplied Chrome profile/executable defaults; caller native args, environment and config remain authoritative. Active roots preserve native launch settings/current restore key, including matching root-name follow-ups; unrelated explicit browsers are unchanged. Profile paths and project-only profile config remain advisory. Native restore covers cookies and web storage, not newly created IndexedDB or memory-only authentication. See the README for communal bootstrap and persistent-profile fallback. Active older/fresh managed sessions retain the lifecycle below. `agent_browser_qa` with `attached: true` can inspect this shared current session using the same live target checks. See [shared browser defaults](../README.md#shared-browser-defaults).
484
485
 
485
486
  - type: `"auto" | "fresh"`
486
487
  - optional
@@ -490,7 +491,7 @@ Behavior:
490
491
  - if `args` already include `--session` (including argv compiled from `agent_browser_action.session`), upstream session choice wins
491
492
  - `"auto"` prepends the current extension-managed active session when appropriate
492
493
  - after resume, a confirmed inactive wrapper-owned daemon with automatic managed restore enabled reopens its complete recorded URL, including its fragment, before the first current-page operation, including `get url` and `reload`. Non-page calls such as `tab list` or explicit HTTP `read <url>` can start a daemon without consuming that pending reopen, even across branch/reload replay. The wrapper verifies the observed tab and invalidates old refs; native `open` resets frame scope. This reloads the page with restored cookies/storage, not unsaved forms, JavaScript memory, or history. Live wrong-tab recovery does not navigate. Explicit URL reads, URL `a11y`/`vitals`, `diff url`, `window new`, URL-bearing recording commands, and explicit navigation/context changes do not require the old tab. Caller-owned/attached and restore-disabled sessions are not auto-reopened
493
- - `"fresh"` rotates that managed session to a fresh upstream launch so startup-scoped flags like `--profile`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--headed`, `--enable`, `-p` / `--provider`, or iOS `--device` apply and later default calls follow the new browser; `--idle-timeout` must equal the Pi process's configured managed idle timeout or the wrapper rejects it with restart guidance
494
+ - `"fresh"` rotates that managed session to a fresh upstream launch so startup-scoped flags like `--engine`, `--profile`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--headed`, `--enable`, `-p` / `--provider`, or iOS `--device` apply and later default calls follow the new browser; `--idle-timeout` must equal the Pi process's configured managed idle timeout or the wrapper rejects it with restart guidance
494
495
  - upstream `--webgpu` is a launch-scoped optional boolean: both enabled and explicit `false` values require a fresh managed launch once an implicit session exists; enabled WebGPU is local-launch-only and upstream rejects combinations with CDP, auto-connect, or providers
495
496
  - upstream `--no-webmcp` is also launch-scoped for bare/`true` and explicit `false` values because it selects whether locally managed Chrome enables the experimental feature
496
497
  - upstream restore sessions may periodically save cookies/localStorage while open; an explicit `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` passes through unchanged when a daemon launches (`30000` upstream default, `0` disables periodic saves but keeps native close saves), while wrapper-owned headed launches default it to `0` when unset to avoid upstream 0.33.2's visible temporary collector tabs; direct window close can lose newer state because headed browsers are exempt from idle shutdown, the effective interval persists across resume and changing it in either direction on a running wrapper-owned headed daemon requires close plus a fresh launch, and those restore files remain upstream-owned rather than wrapper artifacts
@@ -540,10 +541,12 @@ Inner code calls use `modelVisible: false` internally: no full model prose rende
540
541
 
541
542
  ### Image observations
542
543
 
543
- `imageObservations[]` carries verified capture metadata: `path`, optional `id` (cell-local code handle), `mimeType`, optional image `pixels` dimensions, `capture` (`viewport`, `full-page`, `element`, or `unknown`), and `geometry`. `geometry.status` is `measured` or `unknown`, with a `reason` and available before/after samples. Samples include CSS viewport/document dimensions, scroll offsets, device pixel ratio, visual viewport, frame evidence, and a uniquely measured CSS element box when available.
544
+ `imageObservations[]` carries verified capture metadata: `path`, optional `id` (cell-local code handle), `mimeType`, optional image `pixels` dimensions, `capture` (`viewport`, `full-page`, `element`, or `unknown`), optional `rendering: "text"` for positively identified Lightpanda text representations, and `geometry`. `geometry.status` is `measured` or `unknown`, with a `reason` and available before/after samples. Samples include CSS viewport/document dimensions, scroll offsets, device pixel ratio, visual viewport, frame evidence, and a uniquely measured CSS element box when available.
544
545
 
545
546
  A measured mapping requires matching pre/post samples and image dimensions that exactly match the CSS crop at the sampled DPR. It includes document-space `crop: { x, y, width, height }` and independent `pixelsPerCssPixel: { x, y }`. This is bracketed measurement, not an atomic capture guarantee. Viewport crop starts at sampled document scroll; full-page crop starts at document origin. Scrolled/ambiguous/non-CSS element captures, child-frame uncertainty, changed geometry, zoom/offset visual viewports, and captures without samples (including native batch captures) remain unknown rather than borrowing final-page geometry.
546
547
 
548
+ Lightpanda renders page text into an image rather than capturing graphical layout. Native engine metadata labels verified direct/batch/code images; standalone samples also recognize its browser brand when native engine metadata is absent or describes a CDP attachment as Chrome. Identified images remain attached but always have unknown geometry with no crop/scale mapping, even if synthetic dimensions match. Missing provenance does not justify coordinates. See [engine limits](COMMAND_REFERENCE.md#lightpanda-engine).
549
+
547
550
  Native mouse coordinates are **current viewport CSS pixels**. For measured geometry, original image pixel `(px, py)` maps to document CSS `(crop.x + px / scale.x, crop.y + py / scale.y)`; subtract current document scroll for viewport coordinates only after rechecking frame/scroll context. Prefer current refs/locators. Pi resize notes map sent-image pixels to original-image pixels only, not CSS coordinates. No provider image-detail override or model-specific coordinate rewrite is applied.
548
551
 
549
552
  Conditional `changed: false` captures have no new image/artifact attachment. A path in JSON is not vision input. Ordinary screenshot attachments use existing verification/size checks; code explicitly selects a handle from its current call. Unknown geometry permits inspection, but cannot justify coordinate input.
@@ -855,7 +858,7 @@ If `agent-browser` is not on `PATH`, fail with a message that:
855
858
  - keep wrapper-spawned commands bounded by clamping `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented default of 25 seconds while the default wrapper child-process watchdog is 35 seconds (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides it, and top-level `timeoutMs` overrides it per call for browser CLI subprocesses). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a per-call subprocess watchdog from the requested command duration plus a small grace window. Dialog commands use `PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS` (default 5000 ms), and click/tap/find refs or tokens plus `eval --stdin` snippets whose text looks like alert/confirm/prompt/dialog triggers use `PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS` (default 8000 ms). Timed-out advanced QA or native batch calls may add `details.timeoutPartialProgress` and visible `Timeout partial progress` evidence with per-step status, retry payloads, current page title/URL, and declared artifact path checks; timed-out dialog-like commands may add dialog status/dismiss/fresh-session recovery next actions
856
859
  - interactive or long-running upstream families such as `chat` without a prompt, `dashboard start`, `stream enable`, `trace start`, `profiler start`, `record start`, `inspect`, `install`, `upgrade`, `doctor --fix`, and `confirm-interactive` are passed through thinly but remain bounded by the same wrapper timeout/session planning rules; prefer explicit arguments, single-shot `chat <message>`, non-interactive flags like `doctor --offline --quick` or `doctor --json`, and cleanup pairs such as `dashboard stop`, `stream disable`, `trace stop`, `profiler stop`, and `record stop`
857
860
  - treat successful plain-text inspection commands like `--help` and `--version` as stateless: do not inject the implicit managed session and do not let those calls claim the managed-session slot
858
- - if startup-scoped flags like `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--enable`, `-p` / `--provider`, or iOS `--device` target the current active managed session, return a validation error with a structured recovery hint that recommends `sessionMode: "fresh"`; when the call explicitly names the current managed session, remove that `--session` from the recovery payload so rotation is actionable. If daemon inspection proves an explicitly targeted older wrapper-owned session active, reject its startup-scoped flags with close-first or fresh-rotation guidance before spawn
861
+ - if startup-scoped flags like `--engine`, `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--enable`, `-p` / `--provider`, or iOS `--device` target the current active managed session, return a validation error with a structured recovery hint that recommends `sessionMode: "fresh"`; when the call explicitly names the current managed session, remove that `--session` from the recovery payload so rotation is actionable. If daemon inspection proves an explicitly targeted older wrapper-owned session active, reject its startup-scoped flags with close-first or fresh-rotation guidance before spawn
859
862
  - for direct headless local Chrome launches to `chat.com` / `chatgpt.com` / `chat.openai.com` or `dash.cloudflare.com`, allow a narrow compatibility fallback that injects a normal Chrome `--user-agent` only when the caller did not explicitly provide one and did not choose raw Chrome arguments, headed, CDP, auto-connect, provider-backed, custom-UA, or non-Chrome behavior through argv or matching upstream environment. Wrapper-managed sessions retain that wrapper-owned user agent as per-session state across follow-up calls, failed replacement closes, and branch reload/resume. Active daemons omit both launch forms so upstream does not replace a launch-configured browser; a session proven inactive receives the retained compatibility launch values, including a fixed, comma-safe Chrome launch argument because the per-page override does not propagate to new tabs or SSO popups.
860
863
 
861
864
  ## Shared execution coordination
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-agent-browser-native",
3
- "version": "0.9.1",
3
+ "version": "0.9.2",
4
4
  "description": "pi extension that exposes agent-browser as a native tool for browser automation",
5
5
  "type": "module",
6
6
  "author": "Mitch Fultz (https://github.com/fitchmultz)",