pi-agent-browser-native 0.9.2 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +91 -5
- package/README.md +120 -86
- package/dist/extensions/agent-browser/index.js +43065 -2016
- package/dist/extensions/agent-browser/lib/argv-descriptor.js +49 -26
- package/dist/extensions/agent-browser/lib/argv-grammar.js +82 -93
- package/dist/extensions/agent-browser/lib/bash-guard.js +129 -126
- package/dist/extensions/agent-browser/lib/batch-lifecycle.js +113 -49
- package/dist/extensions/agent-browser/lib/browser-execution-claim-codec.js +61 -0
- package/dist/extensions/agent-browser/lib/browser-execution-claim-storage.js +232 -0
- package/dist/extensions/agent-browser/lib/browser-execution-claims.js +202 -0
- package/dist/extensions/agent-browser/lib/browser-execution-identity.js +77 -0
- package/dist/extensions/agent-browser/lib/browser-journal-branch.js +98 -0
- package/dist/extensions/agent-browser/lib/browser-journal-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/browser-journal-projection.js +221 -0
- package/dist/extensions/agent-browser/lib/browser-journal-reader.js +89 -0
- package/dist/extensions/agent-browser/lib/browser-journal-replay.js +246 -0
- package/dist/extensions/agent-browser/lib/browser-journal.js +103 -394
- package/dist/extensions/agent-browser/lib/browser-legacy-pages.js +220 -0
- package/dist/extensions/agent-browser/lib/browser-legacy-projection.js +282 -0
- package/dist/extensions/agent-browser/lib/browser-legacy-types.js +1 -0
- package/dist/extensions/agent-browser/lib/browser-session-conversion-stream.js +217 -0
- package/dist/extensions/agent-browser/lib/browser-session-conversion.js +80 -335
- package/dist/extensions/agent-browser/lib/browser-transcript-contracts.js +35 -0
- package/dist/extensions/agent-browser/lib/browser-transcript-validation.js +166 -0
- package/dist/extensions/agent-browser/lib/browser-transcript.js +137 -81
- package/dist/extensions/agent-browser/lib/browser-value-parsing.js +15 -0
- package/dist/extensions/agent-browser/lib/command-capabilities.js +340 -0
- package/dist/extensions/agent-browser/lib/command-policy.js +107 -88
- package/dist/extensions/agent-browser/lib/command-taxonomy.js +49 -322
- package/dist/extensions/agent-browser/lib/config-policy.js +138 -343
- package/dist/extensions/agent-browser/lib/config-providers.js +71 -0
- package/dist/extensions/agent-browser/lib/config-validation.js +218 -0
- package/dist/extensions/agent-browser/lib/config.js +19 -7
- package/dist/extensions/agent-browser/lib/electron/cdp.js +15 -7
- package/dist/extensions/agent-browser/lib/electron/cleanup.js +134 -55
- package/dist/extensions/agent-browser/lib/electron/discovery-darwin.js +109 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-desktop.js +136 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-evidence.js +89 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-linux.js +93 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-packages.js +162 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-types.js +2 -0
- package/dist/extensions/agent-browser/lib/electron/discovery.js +140 -549
- package/dist/extensions/agent-browser/lib/electron/launch-monitor.js +102 -0
- package/dist/extensions/agent-browser/lib/electron/launch-process.js +209 -0
- package/dist/extensions/agent-browser/lib/electron/launch-types.js +2 -0
- package/dist/extensions/agent-browser/lib/electron/launch.js +249 -354
- package/dist/extensions/agent-browser/lib/executable-path.js +21 -12
- package/dist/extensions/agent-browser/lib/execution-cwd.js +35 -18
- package/dist/extensions/agent-browser/lib/input-modes/electron.js +162 -121
- package/dist/extensions/agent-browser/lib/input-modes/job.js +116 -313
- package/dist/extensions/agent-browser/lib/input-modes/lookup-network-evidence.js +104 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-plan.js +132 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-redaction.js +57 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-source-evidence.js +151 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-workspace.js +142 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookups.js +57 -412
- package/dist/extensions/agent-browser/lib/input-modes/params.js +63 -17
- package/dist/extensions/agent-browser/lib/input-modes/qa-analysis.js +268 -0
- package/dist/extensions/agent-browser/lib/input-modes/qa-presentation.js +117 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-child.js +396 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-protocol.js +86 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-types.js +1 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-validation.js +60 -0
- package/dist/extensions/agent-browser/lib/input-modes/script.js +60 -409
- package/dist/extensions/agent-browser/lib/input-modes/semantic-action.js +218 -128
- package/dist/extensions/agent-browser/lib/input-modes/shared.js +15 -4
- package/dist/extensions/agent-browser/lib/input-modes/types.js +56 -7
- package/dist/extensions/agent-browser/lib/json-schema.js +22 -10
- package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +17 -5
- package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +92 -425
- package/dist/extensions/agent-browser/lib/managed-session-posix-storage.js +58 -0
- package/dist/extensions/agent-browser/lib/managed-session-project-identity.js +244 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-context.js +84 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-policy.js +119 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-state.js +83 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore.js +168 -300
- package/dist/extensions/agent-browser/lib/managed-session-snapshot-records.js +277 -0
- package/dist/extensions/agent-browser/lib/managed-session-snapshots.js +128 -305
- package/dist/extensions/agent-browser/lib/managed-session-storage.js +56 -253
- package/dist/extensions/agent-browser/lib/native-confirmation.js +140 -0
- package/dist/extensions/agent-browser/lib/native-output.js +44 -9
- package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +107 -66
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-merge.js +14 -6
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +109 -57
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/browser-observation-result.js +82 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +171 -57
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostic-values.js +31 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +152 -781
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-observation.js +202 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-text-diagnostics.js +109 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/fill-diagnostics.js +119 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-action-recovery.js +147 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-actions.js +144 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-content.js +123 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-details.js +156 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-electron.js +179 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-evidence.js +39 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-missing-binary.js +89 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-recovery.js +97 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-redaction.js +106 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-timeout.js +110 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +41 -538
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +204 -108
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/interaction-diagnostics.js +290 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-close.js +167 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-inspection.js +108 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +127 -199
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/native-helper-observation.js +65 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/observation-types.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/overlay-diagnostics.js +120 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/network-page-filter.js +113 -30
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +109 -28
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter-data.js +150 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +138 -253
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-rendered-text.js +147 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/wait-timeouts.js +59 -24
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-artifacts.js +159 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-command.js +131 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-daemon.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-failure.js +42 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-guards.js +242 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-input.js +75 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-observations.js +92 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-page-selection.js +254 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-page-state.js +121 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-refs.js +130 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-resources.js +114 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-semantic.js +121 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-session-plan.js +222 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-timeouts.js +68 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +176 -1114
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-analysis.js +191 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-confirmation.js +152 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-diagnostics.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-electron.js +150 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-phase.js +115 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-managed-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-managed.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native-phase.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native.js +113 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-navigation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-navigation.js +316 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-ownership.js +28 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-page-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-page-phase.js +142 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-presentation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-presentation.js +333 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication-phase.js +71 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication.js +201 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-receipts.js +183 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-refs.js +200 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-tabs.js +142 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +18 -910
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prompt-guards.js +23 -13
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-receipt-reconciliation.js +198 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery-presentation.js +154 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +68 -151
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-capture.js +14 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +169 -46
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-commands.js +130 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-electron.js +210 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-observations.js +61 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-outcomes.js +187 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-refs.js +171 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-tabs.js +228 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +77 -753
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/text-visibility-diagnostics.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/timeout-diagnostics.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/timeout-progress.js +108 -0
- package/dist/extensions/agent-browser/lib/orchestration/code-artifact-observation.js +39 -0
- package/dist/extensions/agent-browser/lib/orchestration/code-image-observation.js +119 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/branch.js +92 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/cleanup-result.js +58 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/cleanup.js +78 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/discovery.js +106 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +48 -816
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/policy.js +86 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-collection.js +69 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-data.js +167 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-presentation.js +164 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe.js +161 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/status.js +165 -0
- package/dist/extensions/agent-browser/lib/orchestration/execution-queue.js +21 -9
- package/dist/extensions/agent-browser/lib/orchestration/extension-artifact-preflight.js +211 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-artifacts.js +23 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-attachment-replay.js +99 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-branch-restore.js +275 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-code.js +280 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-admission.js +303 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-output.js +36 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-result.js +335 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-run.js +227 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-state.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-electron-ownership.js +204 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-execution-locks.js +139 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-execution-queue.js +24 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-host-coordination.js +98 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-host-run.js +295 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-intent-replay.js +41 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-invocation.js +7 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-lifecycle.js +102 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-managed-ownership.js +133 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-observation.js +120 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-prompt.js +180 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-recording.js +290 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-resource-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-resource-replay.js +269 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-result-state.js +113 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-runtime.js +146 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-script-leases.js +58 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-tool-boundary.js +62 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-version-check.js +54 -0
- package/dist/extensions/agent-browser/lib/orchestration/input-plan-types.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +261 -185
- package/dist/extensions/agent-browser/lib/orchestration/input-redaction.js +62 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-config.js +144 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-root-launch.js +77 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +210 -139
- package/dist/extensions/agent-browser/lib/orchestration/operation-paths.js +124 -58
- package/dist/extensions/agent-browser/lib/orchestration/output-file.js +198 -98
- package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +199 -96
- package/dist/extensions/agent-browser/lib/owned-managed-session-context.js +55 -0
- package/dist/extensions/agent-browser/lib/page-target-batch.js +118 -0
- package/dist/extensions/agent-browser/lib/page-target-guards.js +50 -0
- package/dist/extensions/agent-browser/lib/page-target-navigation.js +104 -0
- package/dist/extensions/agent-browser/lib/page-target-validation.js +59 -243
- package/dist/extensions/agent-browser/lib/parsing.js +13 -4
- package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +134 -68
- package/dist/extensions/agent-browser/lib/playbook.js +18 -14
- package/dist/extensions/agent-browser/lib/process-child.js +284 -0
- package/dist/extensions/agent-browser/lib/process-environment.js +22 -6
- package/dist/extensions/agent-browser/lib/process-errors.js +23 -0
- package/dist/extensions/agent-browser/lib/process-identity.js +47 -15
- package/dist/extensions/agent-browser/lib/process-socket-storage.js +256 -0
- package/dist/extensions/agent-browser/lib/process-stdout.js +74 -0
- package/dist/extensions/agent-browser/lib/process.js +146 -537
- package/dist/extensions/agent-browser/lib/prompt-artifacts.js +287 -0
- package/dist/extensions/agent-browser/lib/prompt-policy.js +15 -194
- package/dist/extensions/agent-browser/lib/read-command.js +41 -0
- package/dist/extensions/agent-browser/lib/read-confirmation.js +138 -35
- package/dist/extensions/agent-browser/lib/recording-reservations.js +118 -45
- package/dist/extensions/agent-browser/lib/redaction-fields.js +75 -0
- package/dist/extensions/agent-browser/lib/redaction-json.js +149 -0
- package/dist/extensions/agent-browser/lib/redaction-url.js +88 -0
- package/dist/extensions/agent-browser/lib/results/action-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/action-recommendations.js +28 -271
- package/dist/extensions/agent-browser/lib/results/artifact-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/artifact-manifest-validation.js +133 -0
- package/dist/extensions/agent-browser/lib/results/artifact-manifest.js +96 -94
- package/dist/extensions/agent-browser/lib/results/artifact-recommendations.js +73 -0
- package/dist/extensions/agent-browser/lib/results/categories.js +111 -54
- package/dist/extensions/agent-browser/lib/results/confirmation.js +29 -10
- package/dist/extensions/agent-browser/lib/results/editable-ref-evidence.js +62 -27
- package/dist/extensions/agent-browser/lib/results/electron-recommendations.js +83 -0
- package/dist/extensions/agent-browser/lib/results/envelope-decoding.js +48 -0
- package/dist/extensions/agent-browser/lib/results/envelope.js +93 -66
- package/dist/extensions/agent-browser/lib/results/evidence-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/failure-recommendations.js +180 -0
- package/dist/extensions/agent-browser/lib/results/network-routes.js +86 -45
- package/dist/extensions/agent-browser/lib/results/network.js +35 -12
- package/dist/extensions/agent-browser/lib/results/next-actions.js +49 -19
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-files.js +320 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-format.js +138 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-images.js +113 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-verification.js +145 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +99 -552
- package/dist/extensions/agent-browser/lib/results/presentation/batch-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-recording.js +118 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-redaction.js +68 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-rendering.js +135 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-step.js +227 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-summary.js +91 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch.js +91 -460
- package/dist/extensions/agent-browser/lib/results/presentation/browser-profile-recovery.js +9 -5
- package/dist/extensions/agent-browser/lib/results/presentation/command-presenters.js +216 -0
- package/dist/extensions/agent-browser/lib/results/presentation/common.js +26 -9
- package/dist/extensions/agent-browser/lib/results/presentation/content.js +86 -29
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-a11y.js +67 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-health.js +79 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-identity.js +199 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-logs.js +75 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-network-actions.js +216 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-network.js +124 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-redaction.js +64 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-storage.js +209 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-stream.js +69 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-summary.js +222 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +103 -1019
- package/dist/extensions/agent-browser/lib/results/presentation/early-result.js +70 -0
- package/dist/extensions/agent-browser/lib/results/presentation/errors.js +130 -60
- package/dist/extensions/agent-browser/lib/results/presentation/input-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +60 -145
- package/dist/extensions/agent-browser/lib/results/presentation/navigation.js +147 -88
- package/dist/extensions/agent-browser/lib/results/presentation/notices.js +75 -0
- package/dist/extensions/agent-browser/lib/results/presentation/observation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/observation-rendering.js +127 -0
- package/dist/extensions/agent-browser/lib/results/presentation/outcomes.js +120 -0
- package/dist/extensions/agent-browser/lib/results/presentation/output-completion.js +77 -0
- package/dist/extensions/agent-browser/lib/results/presentation/output-spill.js +51 -0
- package/dist/extensions/agent-browser/lib/results/presentation/registry.js +117 -237
- package/dist/extensions/agent-browser/lib/results/presentation/semantic-action.js +35 -21
- package/dist/extensions/agent-browser/lib/results/presentation/skills.js +104 -73
- package/dist/extensions/agent-browser/lib/results/presentation/source.js +109 -0
- package/dist/extensions/agent-browser/lib/results/presentation.js +17 -248
- package/dist/extensions/agent-browser/lib/results/recommendation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/recording.js +52 -14
- package/dist/extensions/agent-browser/lib/results/recovery-actions.js +14 -5
- package/dist/extensions/agent-browser/lib/results/recovery-next-actions.js +30 -22
- package/dist/extensions/agent-browser/lib/results/selector-recovery-target.js +83 -0
- package/dist/extensions/agent-browser/lib/results/selector-recovery.js +123 -123
- package/dist/extensions/agent-browser/lib/results/snapshot-compact.js +266 -0
- package/dist/extensions/agent-browser/lib/results/snapshot-content.js +32 -0
- package/dist/extensions/agent-browser/lib/results/snapshot-high-value-controls.js +104 -71
- package/dist/extensions/agent-browser/lib/results/snapshot-refs.js +31 -11
- package/dist/extensions/agent-browser/lib/results/snapshot-segments.js +125 -66
- package/dist/extensions/agent-browser/lib/results/snapshot-spill.js +26 -7
- package/dist/extensions/agent-browser/lib/results/snapshot.js +28 -248
- package/dist/extensions/agent-browser/lib/results/text.js +16 -5
- package/dist/extensions/agent-browser/lib/runtime-args-validation.js +133 -0
- package/dist/extensions/agent-browser/lib/runtime-compatibility.js +81 -0
- package/dist/extensions/agent-browser/lib/runtime-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/runtime-redaction.js +118 -0
- package/dist/extensions/agent-browser/lib/runtime-replay-events.js +144 -0
- package/dist/extensions/agent-browser/lib/runtime-session-identity.js +76 -0
- package/dist/extensions/agent-browser/lib/runtime-session-plan.js +114 -0
- package/dist/extensions/agent-browser/lib/runtime-session-replay.js +123 -0
- package/dist/extensions/agent-browser/lib/runtime-tab-correction.js +65 -0
- package/dist/extensions/agent-browser/lib/runtime-timeouts.js +28 -0
- package/dist/extensions/agent-browser/lib/runtime.js +137 -1037
- package/dist/extensions/agent-browser/lib/session-identity.js +51 -0
- package/dist/extensions/agent-browser/lib/session-page-observation.js +357 -0
- package/dist/extensions/agent-browser/lib/session-page-state.js +188 -324
- package/dist/extensions/agent-browser/lib/session-page-types.js +1 -0
- package/dist/extensions/agent-browser/lib/temp-artifact-budget.js +78 -0
- package/dist/extensions/agent-browser/lib/temp-mutation-queue.js +7 -0
- package/dist/extensions/agent-browser/lib/temp-root-ownership.js +234 -0
- package/dist/extensions/agent-browser/lib/temp-root.js +207 -0
- package/dist/extensions/agent-browser/lib/temp.js +33 -446
- package/dist/extensions/agent-browser/lib/tool-surface.js +94 -24
- package/dist/extensions/agent-browser/lib/upstream-version.js +3 -2
- package/dist/extensions/agent-browser/lib/web-search-providers.js +229 -0
- package/dist/extensions/agent-browser/lib/web-search-request.js +108 -0
- package/dist/extensions/agent-browser/lib/web-search-results.js +281 -0
- package/dist/extensions/agent-browser/lib/web-search-schema.js +82 -0
- package/dist/extensions/agent-browser/lib/web-search.js +109 -617
- package/dist/extensions/agent-browser/lib/webmcp-observation.js +13 -6
- package/dist/extensions/agent-browser/lib/windows-stock-launcher.js +63 -37
- package/dist/extensions/agent-browser/script-worker.js +80 -43
- package/dist/scripts/agent-browser-target.mjs +4 -2
- package/docs/ARCHITECTURE.md +22 -10
- package/docs/CODE_QUALITY.md +242 -0
- package/docs/COMMAND_REFERENCE.md +264 -229
- package/docs/ELECTRON.md +53 -44
- package/docs/RELEASE.md +21 -17
- package/docs/REQUIREMENTS.md +2 -0
- package/docs/SESSION_STORAGE.md +1 -1
- package/docs/SUPPORT_MATRIX.md +84 -77
- package/docs/TOOL_CONTRACT.md +165 -130
- package/docs/platform-smoke.md +5 -5
- package/package.json +143 -120
- package/scripts/agent-browser-capability-baseline.mjs +361 -957
- package/scripts/agent-browser-target.mjs +6 -2
- package/scripts/build.mjs +37 -8
- package/scripts/capability-core-commands.mjs +193 -0
- package/scripts/capability-options.mjs +283 -0
- package/scripts/capability-samples.mjs +133 -0
- package/scripts/capability-section.mjs +11 -0
- package/scripts/config.mjs +213 -114
- package/scripts/convert-browser-session.mjs +30 -9
- package/scripts/doctor-sources.mjs +195 -0
- package/scripts/doctor.mjs +92 -247
- package/scripts/platform-smoke/artifacts.mjs +38 -11
- package/scripts/platform-smoke/commands.mjs +104 -0
- package/scripts/platform-smoke/crabbox-runner.mjs +167 -80
- package/scripts/platform-smoke/doctor-support.mjs +157 -0
- package/scripts/platform-smoke/doctor-windows.mjs +230 -0
- package/scripts/platform-smoke/doctor.mjs +212 -320
- package/scripts/platform-smoke/lease-evidence.mjs +134 -0
- package/scripts/platform-smoke/suite-checks.mjs +98 -0
- package/scripts/platform-smoke/suite-evidence.mjs +281 -0
- package/scripts/platform-smoke/targets.mjs +158 -472
- package/scripts/platform-smoke.mjs +55 -40
- package/scripts/prepare.mjs +16 -7
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# Agent Browser command reference
|
|
2
2
|
|
|
3
3
|
Related docs:
|
|
4
|
+
|
|
4
5
|
- [`../README.md`](../README.md)
|
|
5
6
|
- [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md)
|
|
6
7
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md)
|
|
@@ -26,6 +27,7 @@ SDK hosts can supply an awaited [`beforeExecute` callback](TOOL_CONTRACT.md#host
|
|
|
26
27
|
|
|
27
28
|
<!-- agent-browser-capability-baseline:start upstream-baseline -->
|
|
28
29
|
<!-- Generated from scripts/agent-browser-capability-baseline.mjs. Run `npm run docs -- command-reference write` to update. Do not edit manually. -->
|
|
30
|
+
|
|
29
31
|
This reference is baselined to the locally installed `agent-browser 0.38.1` command/help surface, audited against vercel-labs/agent-browser@aff6125c023b810ea3f2e5deec5379e9a4270bdc. Upstream `agent-browser` remains the source of truth for command semantics; this file is the local fallback for Pi agent sessions where direct binary help is blocked or discouraged.
|
|
30
32
|
|
|
31
33
|
The lightweight drift check is `npm run verify -- command-reference`. Run it whenever the installed upstream `agent-browser` version changes or this reference is edited.
|
|
@@ -194,7 +196,7 @@ The 0.27.3 rebaseline is an install-only compatibility update: upstream changed
|
|
|
194
196
|
|
|
195
197
|
Use `agent_browser` for one native command and `batch --bail` with JSON-array stdin for a fixed sequence. Use `agent_browser_code` for loops, branches, and aggregation over the same persistent browser. Return to the model when observation needs judgment. All native command families remain available; there is no public job DSL or isolated script input.
|
|
196
198
|
|
|
197
|
-
`agent_browser` accepts `{ args, stdin?, outputPath?, timeoutMs?, sessionMode? }`. Tokens exclude the binary and shell syntax. Stdin is accepted for native batch, `eval --stdin`, and `auth save --password-stdin`. Omit `--json` for prose; include
|
|
199
|
+
`agent_browser` accepts `{ args, stdin?, outputPath?, timeoutMs?, sessionMode? }`. Tokens exclude the binary and shell syntax. Stdin is accepted for native batch, `eval --stdin`, and `auth save --password-stdin`. Omit `--json` for prose backed by structured upstream JSON; include `--json` or `--json true` for a parseable canonical observation. Explicit `--json false` preserves native text for standalone commands and batch, including leading/trailing whitespace and line breaks in data, complete spills and exports, without trusted row/confirmation/page-target/artifact receipts. `outputPath` success/failure notices append prose without reparsing JSON-looking page text. Repeated flags follow upstream's last-wins semantics; batch-row flags remain native operands and do not select outer output mode. See [the output contract](TOOL_CONTRACT.md#wrapper-json). Help/version remains native text. Empty stdin batch rows (`[]`) pass through unchanged and native execution skips them; `[""]` still fails as an unknown command. `--bail` stops at that failure, while a non-bail batch retains later native results. A nonzero process exit that supplies stderr instead of JSON retains that native error and redaction, with parse diagnostics secondary. Zero-exit malformed JSON remains a strict failure.
|
|
198
200
|
|
|
199
201
|
```json
|
|
200
202
|
{ "args": ["open", "https://example.com"] }
|
|
@@ -202,7 +204,7 @@ Use `agent_browser` for one native command and `batch --bail` with JSON-array st
|
|
|
202
204
|
{ "args": ["batch", "--bail"], "stdin": "[[\"get\",\"title\"],[\"get\",\"url\"]]" }
|
|
203
205
|
```
|
|
204
206
|
|
|
205
|
-
Ordinary calls share the root Pi parent/subagent browser unless native config, an explicit identity, or an active managed/fresh browser selects otherwise. `sessionMode: "fresh"` intentionally rotates the managed browser; explicit native session choice still wins. Keep profile/provider/attachment launch flags on first launch. After a failed fresh call, inspect `managedSessionOutcome` rather than guessing which browser is current.
|
|
207
|
+
Ordinary calls share the root Pi parent/subagent browser unless native config, an explicit identity, or an active managed/fresh browser selects otherwise. `sessionMode: "fresh"` intentionally rotates the managed browser; explicit native session choice still wins. Keep profile/provider/attachment launch flags on first launch. Native launch environment/config can submit the full launch configuration again on each invocation: mixing an executable default with first-call-only profile/user-agent flags can replace the browser when those flags disappear. Use a consistent native launch configuration; the wrapper passes it through unchanged and omits automatic Chrome startup args when native environment/config already supplies local launch defaults. After a failed fresh call, inspect `managedSessionOutcome` rather than guessing which browser is current.
|
|
206
208
|
|
|
207
209
|
A cooperating-process lock covers each direct helper/action sequence or whole code cell, keyed by actual socket context, canonical namespace, and session. Independent identities remain concurrent; direct namespace-wide close barriers matching work. Restart all participants onto the new version. This is not rollback or protection against human/external-client navigation, and separate tool calls still need coordination.
|
|
208
210
|
|
|
@@ -214,7 +216,7 @@ Call `agent_browser_code` with `{ code, session?, namespace?, timeoutMs?, output
|
|
|
214
216
|
|
|
215
217
|
```json
|
|
216
218
|
{
|
|
217
|
-
|
|
219
|
+
"code": "const titles = []; for (const url of ['https://example.com', 'https://example.org']) { const opened = await browser({ args: ['open', url] }); if (!opened.success) throw new Error(opened.error); const title = await browser({ args: ['get', 'title'] }); if (!title.success) throw new Error(title.error); titles.push({ url, title: title.data.title ?? title.data.result }); } emit(titles);"
|
|
218
220
|
}
|
|
219
221
|
```
|
|
220
222
|
|
|
@@ -236,13 +238,13 @@ On hosts supporting instruction groups, first enable `browser` with `discover_to
|
|
|
236
238
|
|
|
237
239
|
Activation is additive and uses native Pi selected-tool history. Omit CLI `--tools` for normal lazy activation, or include desired advanced tools in that explicit selection; excluded tools remain unavailable. The tools take flat input:
|
|
238
240
|
|
|
239
|
-
| Tool
|
|
240
|
-
|
|
|
241
|
-
| `agent_browser_action`
|
|
242
|
-
| `agent_browser_qa`
|
|
243
|
-
| `agent_browser_electron`
|
|
244
|
-
| `agent_browser_source`
|
|
245
|
-
| `agent_browser_network_source` | `{ "requestId": "req-1", "url": "/api/fail" }`
|
|
241
|
+
| Tool | Example |
|
|
242
|
+
| ------------------------------ | ------------------------------------------------------------------------------------------------------- |
|
|
243
|
+
| `agent_browser_action` | `{ "action": "select", "locator": "role", "role": "combobox", "name": "Flavor", "value": "chocolate" }` |
|
|
244
|
+
| `agent_browser_qa` | `{ "url": "https://example.com", "expectedText": "Learn more" }` |
|
|
245
|
+
| `agent_browser_electron` | `{ "action": "list", "query": "code" }` |
|
|
246
|
+
| `agent_browser_source` | `{ "selector": "#save", "componentName": "SaveButton" }` |
|
|
247
|
+
| `agent_browser_network_source` | `{ "requestId": "req-1", "url": "/api/fail" }` |
|
|
246
248
|
|
|
247
249
|
Native commands remain in `agent_browser`; advanced objects are neither nested batch rows nor nested fields on that tool. If a recovery action names an inactive advanced tool, enable it before calling its exact flat payload.
|
|
248
250
|
|
|
@@ -340,7 +342,7 @@ Examples:
|
|
|
340
342
|
{ "args": ["snapshot", "-i"] }
|
|
341
343
|
```
|
|
342
344
|
|
|
343
|
-
The optional `agent_browser_action` tool preserves stable accessible click/check/fill and unique visible dropdown selection. Enable `action`, then pass flat locator/selector fields. Role/name or label select resolves one current visible native combobox/listbox;
|
|
345
|
+
The optional `agent_browser_action` tool preserves stable accessible click/check/fill and unique visible dropdown selection. Enable `action`, then pass flat locator/selector fields. Role/name or label select resolves one current visible native combobox/listbox from a fresh snapshot or the eligible next-call confirmed capture; missing or ambiguous select targets still fail. It does not assume native `find ... select`. A `session` field targets a named browser. `compiledSemanticAction` remains the audit metadata name; `effectiveArgs` identifies the executed ref/command. `uncheck` remains native argv. Selector failures retain exact ref/role candidates and session-aware recovery.
|
|
344
346
|
|
|
345
347
|
For desktop, contenteditable, or host-controlled rich inputs, treat a semantic `fill` miss or mismatch differently. Active-session role/name fills can execute through one exact current editable `combobox`, `searchbox`, or `textbox` ref before upstream `find` runs. If a later selector miss still finds an exact current editable ref (`searchbox` or `textbox`), `details.richInputRecovery` and visible `Rich input recovery` describe the candidate and append `focus-current-editable-ref*` / `click-current-editable-ref*` next actions. Those actions deliberately do **not** copy the fill text and never press `Enter` or submit. Direct `fill @ref <text>` on contenteditable refs may also append/prepend instead of replacing; when the latest snapshot proves the target is contenteditable, the wrapper verifies `get text` after a successful fill and appends `details.fillVerification` plus `inspect-after-fill-verification` / `verify-filled-value` if the visible text does not match. Use the safe ladder instead: refresh refs, choose the current editable `@ref`, focus or click it, then use `keyboard type` for framework-controlled editors that require real key events. `keyboard inserttext` is paste-like: it can change a DOM value without updating application state, so use it only when later application-state evidence proves the edit was accepted. Do not auto-submit unless the user flow explicitly calls for it.
|
|
346
348
|
|
|
@@ -348,7 +350,7 @@ Do not assume Playwright selector dialects such as `text=Close` or `button:has-t
|
|
|
348
350
|
|
|
349
351
|
Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `agent_browser_action` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
|
|
350
352
|
|
|
351
|
-
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
|
|
353
|
+
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. Probe listeners and the window marker self-expire after the call's process-timeout budget plus the existing two-second cleanup grace if a pending decision blocks normal cleanup; collection cancels the timer. This uses no extra browser command while pending. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
|
|
352
354
|
|
|
353
355
|
Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a native batch tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close. A click that upstream rejects because another element covers the target's click point (`is covered by` … `at its click point`) remains `failureCategory: "upstream-error"`, but receives the same session-aware `inspect-overlay-state` snapshot action. This includes direct `click`, `agent_browser_action` click, raw `find` clicks (including `nth` and omitted default-click actions), and failed native batch rows. That failed-click path does not retry the original click or synthesize a dismiss candidate before a fresh snapshot provides evidence; it is separate from silent input-dispatch failures.
|
|
354
356
|
|
|
@@ -368,7 +370,10 @@ Use `read [url]` for documentation and other unstructured text. `read <url> --ra
|
|
|
368
370
|
When you already know several visible refs or selectors, extract them in one `batch` call instead of many serial getter calls. When a prior snapshot and session are available and the same-page freshness checks apply, ref-consuming calls add one extra `snapshot -i` preflight per top-level call or batch. Batching shares that probe across rows; it does not remove it:
|
|
369
371
|
|
|
370
372
|
```json
|
|
371
|
-
{
|
|
373
|
+
{
|
|
374
|
+
"args": ["batch"],
|
|
375
|
+
"stdin": "[[\"get\",\"text\",\"@e64\"],[\"get\",\"text\",\"@e65\"],[\"get\",\"text\",\"@e66\"]]"
|
|
376
|
+
}
|
|
372
377
|
```
|
|
373
378
|
|
|
374
379
|
Prefer `get` and scoped `eval --stdin` for read-only extraction. Getter names are grouped under `get`: use `get title`, `get url`, or `get text <selector>`, not shortcut commands such as `title` or `url`. When upstream reports an unknown command, unknown subcommand, or unrecognized command for a single-token shortcut (`attr`, `count`, `html`, `text`, `title`, `url`, or `value`), the wrapper adds a visible grouped-`get` hint; only `title` and `url` also get exact read-only `details.nextActions` (`use-get-title` / `use-get-url`, with `--session` preserved when the failed call named a session). If another `Agent-browser hint:` (selector dialect or stale-ref recovery) was already appended to the same error text, the getter hint is omitted.
|
|
@@ -389,15 +394,18 @@ For fixed flows, keep native batch rows short around navigation and rerender bou
|
|
|
389
394
|
|
|
390
395
|
```json
|
|
391
396
|
{
|
|
392
|
-
|
|
393
|
-
|
|
397
|
+
"args": ["batch", "--bail"],
|
|
398
|
+
"stdin": "[[\"open\",\"https://shop.example/checkout\"],[\"fill\",\"#email\",\"user@example.com\"],[\"click\",\"#continue\"],[\"wait\",\"--url\",\"**/shipping\"],[\"wait\",\"--text\",\"Shipping address\"],[\"screenshot\",\"/tmp/shipping.png\"]]"
|
|
394
399
|
}
|
|
395
400
|
```
|
|
396
401
|
|
|
397
402
|
Paced input uses explicit native rows, preserving the former job's behavior without assuming `--delay` equivalence:
|
|
398
403
|
|
|
399
404
|
```json
|
|
400
|
-
{
|
|
405
|
+
{
|
|
406
|
+
"args": ["batch", "--bail"],
|
|
407
|
+
"stdin": "[[\"focus\",\"#prompt\"],[\"keyboard\",\"type\",\"h\"],[\"wait\",\"20\"],[\"keyboard\",\"type\",\"i\"]]"
|
|
408
|
+
}
|
|
401
409
|
```
|
|
402
410
|
|
|
403
411
|
For longer text, code can construct the rows and call native batch; keep the former 200-character pacing bound when porting a job. Add a final `press` only when submission is intended. Fixed waits are not postconditions. Timeout partial progress remains available through the same batch path. See [all job step equivalents](TOOL_CONTRACT.md#07-migration).
|
|
@@ -409,7 +417,11 @@ For failed QA, inspect `details.qaPreset.failedChecks` separately from `notRunCh
|
|
|
409
417
|
The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `agent_browser_network_source` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. Filtered data receives the same credential redaction as ordinary network diagnostics before entering `details.data` or an `outputPath` export. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
|
|
410
418
|
|
|
411
419
|
```json
|
|
412
|
-
{
|
|
420
|
+
{
|
|
421
|
+
"url": "https://example.com",
|
|
422
|
+
"expectedText": "Learn more",
|
|
423
|
+
"screenshotPath": ".dogfood/qa-example.png"
|
|
424
|
+
}
|
|
413
425
|
```
|
|
414
426
|
|
|
415
427
|
Optional `loadState`, `checkNetwork`, `checkConsole`, and `checkErrors` default to `"domcontentloaded"`, `true`, `true`, and `true` for URL-opening QA; set a check to `false` to skip that diagnostic. For `qa.attached`, the diagnostic checks default to `false` because upstream buffers may predate the current check; opt in with `checkNetwork`, `checkConsole`, or `checkErrors` when preserved-buffer failures are desired. Omit `expectedText` and `expectedSelector` when you only need load plus diagnostics.
|
|
@@ -497,7 +509,10 @@ Use `wait --download [path]` after an earlier action has already started a brows
|
|
|
497
509
|
For one-call flows, put the click and wait in `batch`; the wait step keeps the saved-file metadata in `details.batchSteps[n].savedFilePath` and `details.batchSteps[n].savedFile`:
|
|
498
510
|
|
|
499
511
|
```json
|
|
500
|
-
{
|
|
512
|
+
{
|
|
513
|
+
"args": ["batch"],
|
|
514
|
+
"stdin": "[[\"click\",\"@e5\"],[\"wait\",\"--download\",\"/tmp/report.csv\"]]"
|
|
515
|
+
}
|
|
501
516
|
```
|
|
502
517
|
|
|
503
518
|
A successful wait-based download renders a readable summary such as `Download completed: /tmp/report.csv` and exposes top-level `details.savedFilePath` plus `details.savedFile` for non-batch calls. With current upstream `agent-browser`, `wait --download <path>` may report the requested path before this environment can verify that the file was persisted there. Treat `details.savedFilePath` as upstream-reported metadata unless `details.artifacts[].exists` is true. Upstream tracking: [vercel-labs/agent-browser#1300](https://github.com/vercel-labs/agent-browser/issues/1300).
|
|
@@ -521,9 +536,10 @@ Screenshot `imageObservations` includes verified pixel dimensions and capture ki
|
|
|
521
536
|
For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
|
|
522
537
|
|
|
523
538
|
Wrapper result rendering is metadata-first for saved files. Image MIME types come from a bounded header read for PNG, JPEG, GIF and WebP, never from a filename suffix; missing, unreadable, unknown or truncated headers omit `mediaType`. This identifies a format, not full image validity. Inline screenshots use the same byte check and existing size limit, so a PNG saved as `.webm` still attaches as `image/png`; other artifact kinds are not auto-inlined. An artifact-producing command fails as `artifact-missing` with artifact `status: "stale"` when the reported path's `mtimeMs` falls outside the command's bounded start/end window (with two seconds of filesystem precision tolerance), including a previous recording that `record restart` claims to finalize; clearly old or future-dated evidence is never accepted as a fresh capture. A batch, whether supplied through stdin arrays or argument command strings, must use distinct explicit artifact destinations; preflight canonicalizes existing path ancestry, compares existing file identities to catch hardlinks, and applies full Unicode plus platform case folding on macOS/Windows so aliases cannot satisfy another step's verification. The same preflight prevents `outputPath` from aliasing a same-call browser artifact, follows upstream's forward option consumption and final effective `-o` / `--output` for `diff screenshot`, and treats the optional path on `network har stop` as an artifact destination; upstream ignores positional paths on `network har start`. Outer CLI artifact parsing removes upstream global flags, so direct forms such as `record --json start <path>` and `pdf --quiet <path>` retain their native destinations. Native batch rows do not run that cleanup: `pdf --quick ignored.pdf` writes to the literal path `--quick`, and `download #link --quiet ignored.bin` writes to `--quiet`. Preflight, directory preparation, presentation, and timeout evidence use those same operands, not the ignored trailing tokens. Screenshot destination parsing mirrors upstream's exact flag matching and `[selector] [path]` positional order: `--` is positional, `true` / `false` after screenshot-only `--full` / `-f` remain positional, extra positionals are ignored after the path slot, selector-prefixed (`.`, `#`, `@`) or uppercase-extension single arguments remain selectors, and lowercase image extensions or slash-bearing arguments are paths. The wrapper deliberately keeps its existing slash-bearing hidden-workspace path normalization (for example `.dogfood/run/foo.png`) before launch. `wait --download` is observational and may verify a download that completed just before the wait began, so it is exempt from the command-window mtime gate; an explicit wait destination, in long `--download <path>` or short `-d <path>` form (including `wait --download --timeout 30000 capture.csv`), still participates in active-recording reservation preflight; the path is the next retained operand after the first timeout pair is removed; unsupported `--download=<path>` fails with split-argument guidance:
|
|
539
|
+
|
|
524
540
|
- screenshots return a saved-path summary, visible artifact metadata, structured `details.artifacts` metadata, and an inline image attachment when safe; the visible block includes artifact type, requested path, absolute path, existence, size, cwd, session, and repair/copy status when applicable
|
|
525
541
|
- downloads, PDFs, `wait --download` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed video recordings from `record stop`, and path-bearing HAR captures return concise saved-path summaries plus structured `details.artifacts` metadata without inlining large files
|
|
526
|
-
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification;
|
|
542
|
+
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification; without that receipt, only an acknowledged native `restarted: true` retires the previous take, whose file remains unverified, not saved; rejected or pending restarts keep the active take's reservation and stop action; a missing or stale prior file fails as `artifact-missing` while the new recording remains visible as pending and the prior manifest row is retired. Within one Pi extension process, an unbounded transcript-backed index reserves active recording destinations independently of the bounded artifact manifest. Artifact lifecycle calls and result `outputPath` writes serialize around that global check; reservations use canonical namespace/session identity, survive manifest eviction and branch replay, and retire after direct, ordered nested-batch, fresh-replacement, code, Electron, or shutdown close; the newest pending row per identity is authoritative. Legacy batch replay retires a pending manifest only when the ordered close lifecycle leaves recording closed; a later successful browser reactivation plus `record start` keeps the new pending reservation. Lexical, hardlink, existing/dangling symlink, full Unicode-fold, and macOS/Windows case aliases are rejected, so `record restart` must use a distinct new path. Do not place `record start` or `record restart` after `close` / `quit` / `exit` in one batch: wrapper preflight rejects it because upstream can report success without starting a recording; split the close and recording into separate calls. A `No recording in progress` stop failure checks the matching native receipt once and preserves checked file metadata instead of assuming the path is missing; a later successful batch recording row opens its new pending path normally. Recovery offers an exact status query, and a stop only when the matching take is still current and pending. The target remains unverified until recording stops. Native 0.37 checks `ffmpeg` before starting; older supported natives may defer failure. If a successful start/restart reports pending output without `ffmpeg`, the wrapper appends `Recording dependency warning: ffmpeg not found on PATH` and `details.recordingDependencyWarning`; stop, check the result, then install the dependency before starting a new recording.
|
|
527
543
|
- `batch` keeps each step's artifacts in `details.batchSteps[].artifacts`; top-level `details.artifacts` and `details.artifactManifest` coalesce an earlier pending recording into the later saved, missing, or stale terminal result for the same namespace/session identity; a successful later close retires an unfinalized recording as `close-abandoned`, uses `missing` only after a filesystem check proves absence (otherwise unverified), removes its stop action, and resets earlier ref/page/network-route batch state; a later successful `record stop` replaces that intermediate abandoned row with its verified saved artifact, and later rows—including failed rows—whose lifecycle reports a browser launch may rebuild state without triggering stale pre-close `about:blank` recovery; failed-step `batchSteps[]` retains only the bounded `lifecycle.effectiveLaunch.browserLaunched` boolean for replay, explicitly non-launching diagnostics leave the close terminal, missing lifecycle evidence remains conservatively active even on the first managed call, every successful close clears wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and any later same-session failure before recording stops keeps exact `stop-pending-recording` args alongside its normal recovery
|
|
528
544
|
|
|
529
545
|
`diff screenshot` follows the file-artifact path above for the **diff** image: model-visible text and `details.artifacts` focus on that output, while baseline paths stay out of the artifact summary block, and Pi does **not** auto-inline the diff the way it inlines trusted `screenshot` captures. `state load` may print the loaded path in prose but does not add a saved-file artifact entry the way `state save` does.
|
|
@@ -563,15 +579,20 @@ Oversized snapshots and oversized generic outputs are different: when a persiste
|
|
|
563
579
|
|
|
564
580
|
```json
|
|
565
581
|
{
|
|
566
|
-
|
|
567
|
-
|
|
582
|
+
"args": ["--profile", "Profile 1", "open", "https://mail.google.com"],
|
|
583
|
+
"sessionMode": "fresh"
|
|
568
584
|
}
|
|
569
585
|
```
|
|
570
586
|
|
|
571
587
|
```json
|
|
572
588
|
{
|
|
573
|
-
|
|
574
|
-
|
|
589
|
+
"args": [
|
|
590
|
+
"--executable-path",
|
|
591
|
+
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser",
|
|
592
|
+
"open",
|
|
593
|
+
"https://mail.google.com"
|
|
594
|
+
],
|
|
595
|
+
"sessionMode": "fresh"
|
|
575
596
|
}
|
|
576
597
|
```
|
|
577
598
|
|
|
@@ -589,7 +610,9 @@ Use `tab list` and `tab <tab-id-or-label>` when a profile restore, pop-up, or cl
|
|
|
589
610
|
|
|
590
611
|
### Recover from guarded-action confirmations
|
|
591
612
|
|
|
592
|
-
|
|
613
|
+
Configure `--confirm-actions` on the session's first command. The extension retains that setting for the same canonical namespace/session across helpers, bare follow-ups, code and advanced calls, and transcript resume. Follow the exact returned confirm/deny action without repeating the policy. First launch and new explicit `--confirm-actions` / `--config` selections use native argv → environment → config precedence. Bare follow-ups retain an established string, including explicit `""`, over ambient environment/discovered-config edits; those defaults apply again after close or for a new identity. Native unset is not pinned and continues ordinary native default resolution. Explicit policy changes still use native restart behavior and can discard pending confirmations. Successful close, namespace-wide close, and successfully closed managed replacements retire the setting with that session. Replacement cleanup uses the old identity's setting; failed or pending close retains its continuity state. Electron launch and host probes use the selected setting too. A definite wrapper/native CLI parse/config/local-inspection rejection restores the previous setting unless an earlier browser helper could have established the request; interrupted work retains selected intent.
|
|
614
|
+
|
|
615
|
+
When upstream requires confirmation, the native tool result prints the pending confirmation id and both recovery calls. Confirming one action in a compound command can return another native confirmation; follow its returned action too. Pending navigation is intent, never an observed page target, including pending native batch rows. A completed compound confirmation reconciles its native resulting target before ordinary follow-ups. A tab-creation transport can add `after_confirmation: ["open", <requested URL>]` plus nonempty `guidance` when the URL matches the original `tab new` request, including a preceding `--label`. Follow that visible continuation only after approval successfully creates the blank/labeled tab, in the same session and new tab—never after denial. The extension preserves the exact decision and conditional guidance but does not execute the continuation. Confirmed full snapshots install returned refs in direct and batch row order. A final confirmed capture supplies the next same-identity call's freshness sample once, including role click/check/fill and unique accessible select resolution; an intervening call or later row consumes the opportunity. A pending snapshot is displayed as confirmation required, never as an empty capture. Use refs from that capture: membership, target and invalidation checks still apply. On this one use, live verification compares the complete capture URL, including fragments, and rejects refs when it changes—including wrapper-owned managed sessions and warm sessions without resume. Ordinary ref/tab URL comparisons remain fragment-insensitive. DOM-only renames on the same page after the sample are not resampled on this one use. Further ref actions use normal freshness checks. Optional click-dispatch eval probes are skipped when the effective native confirmation list guards `evaluate`; native click confirmation still applies. Use the same `agent_browser` tool; do not switch to bash.
|
|
593
616
|
|
|
594
617
|
```json
|
|
595
618
|
{ "args": ["--confirm-actions", "click", "click", "@danger"] }
|
|
@@ -602,7 +625,7 @@ If the result says `Pending confirmation id: c_8f3a1234`, choose one follow-up:
|
|
|
602
625
|
{ "args": ["deny", "c_8f3a1234"] }
|
|
603
626
|
```
|
|
604
627
|
|
|
605
|
-
|
|
628
|
+
Use the exact returned actions for a genuine native control: they preserve its ID and canonical namespace/session across resume/branch replay. Matching guarded decisions skip page/pinning helpers that could overwrite the native pending slot, then reconcile completed browser actions normally; this is not proof of native ID validation or browser independence. Only a matching first effective guarded decision gets that suppression in native batch; native raw-argument-over-stdin selection and later page/ref checks remain unchanged. Page content and unrelated/nonmatching decisions gain no exemption. Explicit URL reads remain separate: only `capabilities.readRequiresConfirmation: true` grants browser independence with native ID checking; legacy URL reads keep normal checks. Explicit caller identities and the code cell's fixed identity still win. A pending cold-reopen helper exposes its exact action without running the requested getter or replaying open. A failed inner confirmed action fails the tool; a completed confirmed close retires session state.
|
|
606
629
|
|
|
607
630
|
Confirmation context may be redacted when it contains credentials, tokens, cookies, or auth-bearing URLs. Replacements are marked `[REDACTED]` (URL-encoded in parsed URLs); ordinary technical phrases such as `bearer token` and `bearer authentication` stay intact outside credential fields and headers. URL scrubbing covers `code`, SAMLRequest, SAMLResponse, RelayState, `authorization_session_id`, and auth-context `state` / `nonce` while retaining ordinary non-auth query values and the spelling of URLs needing no redaction. Visible text, structured details, persisted spills, and `outputPath` exports use the same redaction; exact internal page-target URLs remain available to browser state logic. Use the id exactly as printed.
|
|
608
631
|
|
|
@@ -636,7 +659,7 @@ Operational notes:
|
|
|
636
659
|
|
|
637
660
|
## Full supported surface
|
|
638
661
|
|
|
639
|
-
The
|
|
662
|
+
The command groups below intentionally list more than the recommended workflow. Rare commands are included so agents can discover that the installed upstream supports them without direct `agent-browser --help` access.
|
|
640
663
|
|
|
641
664
|
### Built-in skills
|
|
642
665
|
|
|
@@ -644,58 +667,56 @@ Native-tool note: upstream skills are written for the standalone `agent-browser`
|
|
|
644
667
|
|
|
645
668
|
Session note: `skills list`, `skills get …`, and `skills path …` are **stateless** in this wrapper. Even with default `sessionMode: "auto"`, the extension does not prepend the implicit managed `--session` for those commands, so reading bundled skills does not attach to or rotate the active browser session (same intent as plain-text `--help` / `--version` inspection). Other `skills` subcommands follow normal session rules until explicitly allowlisted in `extensions/agent-browser/lib/runtime.ts` alongside regression coverage in `test/agent-browser.runtime.test.ts`.
|
|
646
669
|
|
|
647
|
-
| Command
|
|
648
|
-
|
|
|
649
|
-
| `skills list`
|
|
650
|
-
| `skills get core`
|
|
651
|
-
| `skills get core --full`
|
|
652
|
-
| `skills get <name>`
|
|
653
|
-
| `skills get <name> --full` | Include a skill's supplementary references/templates when present.
|
|
654
|
-
| `skills get --all`
|
|
655
|
-
| `skills path [name]`
|
|
670
|
+
| Command | Purpose |
|
|
671
|
+
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
672
|
+
| `skills list` | List available CLI-bundled skills. |
|
|
673
|
+
| `skills get core` | Print the core usage guide. |
|
|
674
|
+
| `skills get core --full` | Print the full version-matched core command reference and templates. |
|
|
675
|
+
| `skills get <name>` | Load a specialized skill such as `electron` or `slack`. Common specialized calls include `skills get electron`, `skills get slack`, `skills get dogfood`, `skills get vercel-sandbox`, `skills get agentcore`, `skills get derive-client` (HAR-to-API-client workflow), and `skills get webmcp-gen` (create and validate page tools). |
|
|
676
|
+
| `skills get <name> --full` | Include a skill's supplementary references/templates when present. |
|
|
677
|
+
| `skills get --all` | Print all visible bundled skills for broad audit/debug work. |
|
|
678
|
+
| `skills path [name]` | Print a skill directory path. |
|
|
656
679
|
|
|
657
680
|
Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an override for bundled skill discovery. Normal agents should not need it, but it is useful when validating package layout or upstream skill packaging.
|
|
658
681
|
|
|
659
682
|
### Core page and element commands
|
|
660
683
|
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
| `tap <selector>` | Touch-oriented tap alias for iOS/provider workflows. |
|
|
698
|
-
| `swipe <direction> [distance]` | Touch-oriented swipe for iOS/provider workflows. |
|
|
684
|
+
- `open [url]` — Navigate when given a URL. URL-less `open` runs native `get url`, launching if needed and otherwise retaining the current page. Use explicit `about:blank` when navigation to blank is intended.
|
|
685
|
+
- `open <url>` — Navigate to a URL; `goto <url>` and `navigate <url>` are equivalent navigation aliases when a URL is present.
|
|
686
|
+
- `read [url]` — Fetch agent-readable text from an explicit URL without requiring a Chrome page, or omit the URL to read rendered active-tab DOM. Supports `--raw`, `--require-md`, `--llms <index|full>`, `--outline`, `--filter <text>`, and `--timeout <ms>`.
|
|
687
|
+
- `click <sel>` — Click an element or `@ref`.
|
|
688
|
+
- `click <sel> --new-tab` — Click a link/control while requesting a new tab.
|
|
689
|
+
- `dblclick <sel>` — Double-click an element.
|
|
690
|
+
- `type <sel> <text>` — Type into an element; both selector and text are required. For the focused element without a selector, use `keyboard type <text>`.
|
|
691
|
+
- `fill <sel> <text>` — Clear and fill an element.
|
|
692
|
+
- `press <key>` — Press a key such as `Enter`, `Tab`, or `Control+a`. `key <key>` is the upstream alias.
|
|
693
|
+
- `key <key>` — Alias for `press <key>`.
|
|
694
|
+
- `keydown <key>` — Hold a key down without releasing it, useful for modifiers.
|
|
695
|
+
- `keyup <key>` — Release a key previously held by `keydown <key>`. Common modifier examples are `keydown Shift` and `keyup Shift`.
|
|
696
|
+
- `keyboard type <text>` — Type text with real keystrokes and no selector.
|
|
697
|
+
- `keyboard inserttext <text>` — Insert text without key events.
|
|
698
|
+
- `hover <sel>` — Hover an element.
|
|
699
|
+
- `focus <sel>` — Focus an element.
|
|
700
|
+
- `check <sel>` — Check a checkbox.
|
|
701
|
+
- `uncheck <sel>` — Uncheck a checkbox.
|
|
702
|
+
- `select <sel> <val...>` — Select one or more dropdown options.
|
|
703
|
+
- `drag <src> <dst>` — Drag and drop.
|
|
704
|
+
- `upload <sel> <files...>` — Upload one or more files.
|
|
705
|
+
- `download <sel> <path>` — Download a file by clicking an element.
|
|
706
|
+
- `scroll <dir> [px]` — Scroll `up`, `down`, `left`, or `right`.
|
|
707
|
+
- `scroll <dir> [px] --selector <sel>` — Scroll a specific scrollable element/container instead of the page.
|
|
708
|
+
- `scrollintoview <sel>` — Scroll an element into view; `scrollinto <sel>` is the upstream alias.
|
|
709
|
+
- `scrollinto <sel>` — Alias for `scrollintoview <sel>`.
|
|
710
|
+
- `wait <sel|ms>` — Wait for an element or a duration.
|
|
711
|
+
- `screenshot [selector] [path]` — Take a full-page or element-scoped screenshot; a single selector-like argument scopes, while a path-like argument saves to that path.
|
|
712
|
+
- `screenshot [path]` — Take a screenshot and optionally save it to a path.
|
|
713
|
+
- `pdf <path>` — Save the page as a PDF.
|
|
714
|
+
- `snapshot` — Print an accessibility tree with refs for AI interaction. Common options include `snapshot --interactive`, `snapshot --urls`, `snapshot --compact`, `snapshot --depth <n>`, `snapshot --selector <sel>`, and native `snapshot --delta` / `snapshot --delta --full`. Cursor-interactive elements are included by default; `snapshot --cursor` / `snapshot -C` are deprecated no-ops.
|
|
715
|
+
- `eval <js>` — Run JavaScript. Use `eval --stdin` through this wrapper for larger snippets, or `eval -b <base64>` for shell-escaping-safe one-liners.
|
|
716
|
+
- `connect <port|url>` — Connect to a browser through CDP.
|
|
717
|
+
- `close [--all]` — Close the current browser or all sessions; `quit` and `exit` are upstream close aliases.
|
|
718
|
+
- `tap <selector>` — Touch-oriented tap alias for iOS/provider workflows.
|
|
719
|
+
- `swipe <direction> [distance]` — Touch-oriented swipe for iOS/provider workflows.
|
|
699
720
|
|
|
700
721
|
On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
|
|
701
722
|
|
|
@@ -703,39 +724,38 @@ Comboboxes vary by app. For native `<select>` controls, prefer raw `select <sele
|
|
|
703
724
|
|
|
704
725
|
### Navigation
|
|
705
726
|
|
|
706
|
-
| Command
|
|
707
|
-
|
|
|
708
|
-
| `back`
|
|
709
|
-
| `forward` | Go forward.
|
|
710
|
-
| `reload`
|
|
727
|
+
| Command | Purpose |
|
|
728
|
+
| --------- | ------------------------ |
|
|
729
|
+
| `back` | Go back. |
|
|
730
|
+
| `forward` | Go forward. |
|
|
731
|
+
| `reload` | Reload the current page. |
|
|
711
732
|
|
|
712
733
|
### Session, state, frames, dialogs, windows, and inspection commands
|
|
713
734
|
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
720
|
-
|
|
721
|
-
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
| `close` | Close the current browser session. |
|
|
733
|
-
| `close --all` | Close every session. |
|
|
735
|
+
- `session` — Show current session name.
|
|
736
|
+
- `session list` — List active sessions.
|
|
737
|
+
- `state save <path>` — Save cookies, local storage, and session storage to a state file.
|
|
738
|
+
- `state load <path>` — Load cookies and storage from a state file.
|
|
739
|
+
- `state list` — List saved state files.
|
|
740
|
+
- `state show <filename>` — Show saved-state metadata without dumping cookie or storage values.
|
|
741
|
+
- `state rename <old-name> <new-name>` — Rename a saved state file.
|
|
742
|
+
- `state clear [session-name] [--all]` — Clear saved states for one name or all names; `state clear -a` is the upstream short alias for clearing all names.
|
|
743
|
+
- `session id --scope worktree --prefix <name>` — Generate a stable session id for agent/worktree-scoped browser state.
|
|
744
|
+
- `session info --json` — One read-only preflight: daemon activity/PID versus native browser liveness, Chrome PID, exact profile, tabs and launched/attached ownership, plus separate Pi cleanup ownership. Missing native fields remain unknown; no browser launch or tab changes.
|
|
745
|
+
- `state clean --older-than <days>` — Delete expired saved-state files.
|
|
746
|
+
- `frame <selector|main>` — Switch iframe context by selector/ref/name/URL, or return to the main frame.
|
|
747
|
+
- `dialog accept [text]` — Accept an alert, confirm, or prompt dialog, optionally supplying prompt text.
|
|
748
|
+
- `dialog dismiss` — Dismiss or cancel the current dialog.
|
|
749
|
+
- `dialog status` — Check whether a dialog is pending.
|
|
750
|
+
- `window new` — Open and activate a new blank window. The wrapper keeps that intentional `about:blank` target rather than selecting the old tab. Old refs are invalid; take a fresh snapshot before further interaction.
|
|
751
|
+
- `close` — Close the current browser session.
|
|
752
|
+
- `close --all` — Close every session.
|
|
734
753
|
|
|
735
754
|
A canceled cold-session reopen returns an aborted result with its exact session identity and the consumed reopen marker once the CLI starts. Reload does not repeat that navigation. Cancellation before the attempt leaves the remembered URL pending for the next current-page operation.
|
|
736
755
|
|
|
737
756
|
<!-- agent-browser-playbook:start inspection -->
|
|
738
757
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
758
|
+
|
|
739
759
|
Native inspection calls use the `agent_browser` tool shape, not shell-like direct-binary commands:
|
|
740
760
|
|
|
741
761
|
- { "args": ["--help"] }
|
|
@@ -746,37 +766,35 @@ These calls return plain text and stay stateless: the extension does not inject
|
|
|
746
766
|
|
|
747
767
|
### Page state, finding, mouse, settings, network, and storage
|
|
748
768
|
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
| `cookies [get|set|clear]` | Manage cookies. Full set form: `cookies set <name> <value> --url <url> --domain <domain> --path <path> --httpOnly --secure --sameSite <Strict|Lax|None> --expires <timestamp>`; also supports `cookies set --curl <file>` for JSON, cURL, or bare Cookie-header bulk imports. |
|
|
760
|
-
| `storage <local|session>` | Manage web storage. |
|
|
769
|
+
- `get title`, `get url`, `get cdp-url` — Read page/browser metadata without a selector. Upstream root help summarizes this family as `get <what> [selector]`, but the selector is not optional for DOM getters.
|
|
770
|
+
- `get text/html/value/count <selector>` — Read matched elements; use `get text body` for whole-page text.
|
|
771
|
+
- `get attr <selector> <name>`, `get box <selector>`, `get styles <selector>` — Read an attribute, bounding box, or computed styles from matched elements.
|
|
772
|
+
- `is <what> <selector>` — Check `visible`, `enabled`, or `checked`.
|
|
773
|
+
- `find <locator> <value> <action> [text]` — Locator types include `role`, `text`, `label`, `placeholder`, `alt`, `title`, and `testid`; selector helpers include `find first <sel>`, `find last <sel>`, and `find nth <n> <sel>`. Role/text filters include `find role <role> --name <name>` and `find ... --exact`. Actions are `click, fill, check, hover, text` only. Prefer `find role` for semantic elements: implicit roles work (`find role heading text --name` for `<h2>`, list/banner landmarks, and similar). Default name matching is a case-insensitive substring; `--exact` makes the accessible name case-sensitive. On misses, upstream 0.32.4+ keeps locator detail such as `Names seen: …` or `No element found: getByRole(...)` instead of a generic flatten.
|
|
774
|
+
- `mouse <action> [args]` — `move <x> <y>`, `down [btn]`, `up [btn]`, `wheel <dy> [dx]`.
|
|
775
|
+
- `set <setting> [value]` — `viewport <w> <h>`, `device <name>`, `geo <lat> <lng>`, `offline [on|off]`, `headers <json>`, `credentials <user> <pass>`, and `set media <features>` (`dark`, `light`, and/or `reduced-motion`).
|
|
776
|
+
- `network <action>` — `network route <url> [--abort|--body <json>] [--resource-type <csv>]`, `network unroute [url]`, `network requests [--clear] [--filter <pattern>] [--type <csv>] [--method <method>] [--status <code|range>]`, `network request <requestId>`, `network har start`, `network har start --content text` (default; embeds text bodies), `network har start --content all`, `network har start --content none`, and `network har stop [path]`. `--resource-type` filters intercepted requests by CDP resource type, such as `script`, `image`, `font`, `xhr`, or `fetch`; request listing filters accept resource types (`xhr,fetch`), methods (`POST`), and statuses (`2xx`, `400-499`). HAR files can include auth headers and bodies—do not share them unredacted. For turning a recording into a reusable API client, load `skills get derive-client`.
|
|
777
|
+
- `cookies [get|set|clear]` — Manage cookies. Full set form: `cookies set <name> <value> --url <url> --domain <domain> --path <path> --httpOnly --secure --sameSite <Strict|Lax|None> --expires <timestamp>`; also supports `cookies set --curl <file>` for JSON, cURL, or bare Cookie-header bulk imports.
|
|
778
|
+
- `storage <local|session>` — Manage web storage.
|
|
761
779
|
|
|
762
780
|
`cookies set --curl <file>` selects native file-import mode even when `--curl` follows other set operands. Ordinary import paths remain visible in invocation echoes and failed batch diagnostics; inline cookie values, sensitive flags, and secret-bearing path text still redact. File contents are not echoed by this exception.
|
|
763
781
|
|
|
782
|
+
With explicit `--json false`, cookie `name=value` (including empty names) and storage `key: value` (including empty and colon-containing keys) output use command-aware redaction before presentation and export; benign preferences remain visible. Standalone `storage local|session get <key>` and shorthand `storage local|session <key>` redact the whole sensitive value, including embedded newlines, even when invocation display redaction changes the supplied key. All-entry/raw-batch text has unescaped multiline values and no row provenance, so use structured JSON for unambiguous sensitive storage. See [the output contract](TOOL_CONTRACT.md#wrapper-json).
|
|
783
|
+
|
|
764
784
|
Privacy note: `cookies get` can expose real profile cookies. Do not run it against `--profile Default` or other authenticated profiles unless the user explicitly needs cookie inspection; prefer task-specific page actions and storage checks.
|
|
765
785
|
|
|
766
786
|
### WebMCP page tools
|
|
767
787
|
|
|
768
788
|
WebMCP support is experimental and browser-dependent. Locally managed Chrome enables it by default; use a fresh launch with `--no-webmcp` or set `AGENT_BROWSER_NO_WEBMCP=1` to disable it. Page tool metadata and results come from the page itself. On 0.37, successful navigation with native `webmcp.available: true` and a positive tool count shows a `webmcp list` hint; the raw object stays in `details.data`. No hint is added for absent, unavailable or empty metadata.
|
|
769
789
|
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
|
|
778
|
-
| `webmcp result <id>` | Wait for or read a detached invocation result; accepts `--timeout <ms>`. |
|
|
779
|
-
| `webmcp cancel <id>` | Cancel an active detached invocation. |
|
|
790
|
+
- `webmcp list` — List tools registered by the current page, including each tool's frame id, origin, schema, and annotations.
|
|
791
|
+
- `webmcp invoke <tool>` — Invoke a page tool with an empty input object.
|
|
792
|
+
- `webmcp invoke <tool> --params <json|@file>` — Pass a JSON object inline or read it from a caller-selected file.
|
|
793
|
+
- `webmcp invoke <tool> --frame <frame-id>` — Select the registering frame when a tool name is ambiguous.
|
|
794
|
+
- `webmcp invoke <tool> --detach` — Start the call and return its invocation id without waiting.
|
|
795
|
+
- `webmcp invoke <tool> --timeout <ms>` — Bound a blocking invocation in milliseconds.
|
|
796
|
+
- `webmcp result <id>` — Wait for or read a detached invocation result; accepts `--timeout <ms>`.
|
|
797
|
+
- `webmcp cancel <id>` — Cancel an active detached invocation.
|
|
780
798
|
|
|
781
799
|
`webmcp invoke`, `webmcp result`, and `webmcp cancel` can run page code that changes or navigates the document. The wrapper refreshes page-target evidence and invalidates prior `@e…` refs after these commands. A detached invocation that still reports `pending`, or fails to settle while its target is unknown, leaves the target unverified rather than trusting the immediate URL probe; `result`, `cancel`, `get url`, and explicit navigation remain available for recovery. `details.nextActions` replaces the blocked snapshot suggestion with `verify-page-target-after-pending-webmcp` (`get url`) and warns that the detached tool remains unsettled. After a completed WebMCP mutation, run `snapshot -i` before reusing refs; inside `batch --bail`, put `get url` before that snapshot. `webmcp list` does not invalidate refs. Attached browsers, providers, Lightpanda, Safari/iOS, and Chrome builds without the experimental CDP domain may return `webmcp_unsupported`.
|
|
782
800
|
|
|
@@ -784,14 +802,12 @@ WebMCP support is experimental and browser-dependent. Locally managed Chrome ena
|
|
|
784
802
|
|
|
785
803
|
Stable tab ids look like `t1`, `t2`, and `t3`. Optional user labels such as `docs` or `app` are interchangeable with ids wherever a tab reference is accepted. Upstream help may refer to numeric tab positions, but this wrapper guidance uses stable `t<N>` ids because positional integers are not accepted by current upstream `agent-browser`.
|
|
786
804
|
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
| `tab <t<N>|label>` | Switch to a tab by id or label. CDP target ids from `tab list --json` are also accepted and stay stable across daemon restarts. |
|
|
794
|
-
| `tab close [t<N>|label|target]` | Close the current tab or a referenced tab. Generic references in workflows may say `tab close [target]`; use a stable `t<N>` id, label, or CDP target id when you have one. |
|
|
805
|
+
- `tab` — List open tabs by default.
|
|
806
|
+
- `tab list` — List open tabs with ids and labels.
|
|
807
|
+
- `tab new [url]` — Open a new tab.
|
|
808
|
+
- `tab new --label <name> [url]` — Open a new tab with a user label.
|
|
809
|
+
- `tab <t<N>|label>` — Switch to a tab by id or label. CDP target ids from `tab list --json` are also accepted and stay stable across daemon restarts.
|
|
810
|
+
- `tab close [t<N>|label|target]` — Close the current tab or a referenced tab. Generic references in workflows may say `tab close [target]`; use a stable `t<N>` id, label, or CDP target id when you have one.
|
|
795
811
|
|
|
796
812
|
The wrapper persists native CDP `targetId` when available so resume/branch recovery distinguishes tabs with the same URL and title. A missing known target requires explicit tab recovery; older transcripts without IDs still use URL matching, and a cold managed reopen adopts its new ID. Failed reached batch navigations retire earlier targets and recheck the live page, including Chrome error pages, before subsequent inspection.
|
|
797
813
|
|
|
@@ -801,17 +817,17 @@ With `--pin-tab`, a closed bound tab fails as `tab_gone` (`data.targetId`, optio
|
|
|
801
817
|
|
|
802
818
|
### Snapshot
|
|
803
819
|
|
|
804
|
-
| Option
|
|
805
|
-
|
|
|
806
|
-
| `snapshot`
|
|
807
|
-
| `snapshot -i` / `snapshot --interactive`
|
|
808
|
-
| `snapshot -i --urls`
|
|
809
|
-
| `snapshot -u` / `snapshot --urls`
|
|
810
|
-
| `snapshot -C` / `snapshot --cursor`
|
|
811
|
-
| `snapshot -c` / `snapshot --compact`
|
|
812
|
-
| `snapshot -d <n>` / `snapshot --depth <n>`
|
|
813
|
-
| `snapshot -s <sel>` / `snapshot --selector <sel>` | Scope to a CSS selector.
|
|
814
|
-
| `snapshot --delta` / `snapshot --delta --full`
|
|
820
|
+
| Option | Purpose |
|
|
821
|
+
| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
822
|
+
| `snapshot` | Full accessibility tree with refs. |
|
|
823
|
+
| `snapshot -i` / `snapshot --interactive` | Include only interactive elements. |
|
|
824
|
+
| `snapshot -i --urls` | Include only interactive elements and link hrefs. |
|
|
825
|
+
| `snapshot -u` / `snapshot --urls` | Include href URLs for link elements. |
|
|
826
|
+
| `snapshot -C` / `snapshot --cursor` | Deprecated no-ops; cursor-interactive elements are included by default. |
|
|
827
|
+
| `snapshot -c` / `snapshot --compact` | Remove empty structural elements. |
|
|
828
|
+
| `snapshot -d <n>` / `snapshot --depth <n>` | Limit tree depth. |
|
|
829
|
+
| `snapshot -s <sel>` / `snapshot --selector <sel>` | Scope to a CSS selector. |
|
|
830
|
+
| `snapshot --delta` / `snapshot --delta --full` | Native revision-relative observation or a forced full baseline; see [0.38.1](#upstream-0381-rebaseline). |
|
|
815
831
|
|
|
816
832
|
When a snapshot is too large for inline output, the Pi wrapper renders a compact view before spilling the full redacted snapshot to `details.fullOutputPath`. Compact snapshots are main-content-first, but dense pages and desktop host screens can still hide actionable controls in omitted content; scan `Omitted high-value controls` before opening the spill file. That bounded section favors editable/searchbox/textbox/combobox controls, named tab/surface controls, primary action buttons, and named action links such as row/navigation links and repository-style result links, then includes other useful controls such as checkboxes, radios, options, and menuitems that were not already listed under key refs or other refs. When that section appears, `details.data.highValueControlRefIds` repeats the same visible ref ids for programmatic follow-up alongside fields such as `previewMode`, `previewSections`, and counts on `details.data` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)).
|
|
817
833
|
|
|
@@ -819,48 +835,46 @@ For dense pages, the wrapper also accepts `snapshot -i --search <text>` and `sna
|
|
|
819
835
|
|
|
820
836
|
### Wait
|
|
821
837
|
|
|
822
|
-
| Mode
|
|
823
|
-
|
|
|
824
|
-
| `wait <selector>`
|
|
825
|
-
| `wait <ms>`
|
|
826
|
-
| `wait --url <pattern>`
|
|
827
|
-
| `wait --load <state>`
|
|
828
|
-
| `wait --fn <expression>`
|
|
829
|
-
| `wait --text <text>`
|
|
830
|
-
| `wait --download [path]`
|
|
831
|
-
| `wait --download [path] --timeout <ms>` | Set download-start timeout in milliseconds. The native Pi wrapper forwards explicit wait timeouts and extends the subprocess watchdog unless the caller supplies top-level `timeoutMs`.
|
|
838
|
+
| Mode | Purpose |
|
|
839
|
+
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
840
|
+
| `wait <selector>` | Wait for an element to appear. |
|
|
841
|
+
| `wait <ms>` | Wait for a fixed number of milliseconds; the duration is positional, not `--time <ms>`. The native Pi wrapper now forwards long waits and derives a subprocess watchdog from the explicit wait duration when the caller does not provide top-level `timeoutMs`. |
|
|
842
|
+
| `wait --url <pattern>` | Wait for the URL to match a pattern. On timeout the wrapper appends a `fresh-session-after-url-wait-timeout` next action (`sessionMode: "fresh"` + `open about:blank`, after the inspect action): if a preceding click or submit reported success but the page never navigated, upstream click dispatch may have silently missed, so replace about:blank with the target URL and replay the flow as one batch in a fresh session instead of retrying the wait. |
|
|
843
|
+
| `wait --load <state>` | Wait for load state: `load`, `domcontentloaded`, or `networkidle`. |
|
|
844
|
+
| `wait --fn <expression>` | Wait for a JavaScript expression to become truthy. |
|
|
845
|
+
| `wait --text <text>` | Wait for text to appear on the page; failures may include `inspect-after-text-assertion-failure` with a session-scoped `snapshot -i` payload. |
|
|
846
|
+
| `wait --download [path]` | Wait for a download started by a previous action and optionally save it to `path`; successful wrapper results include upstream-reported `savedFilePath`/`savedFile`, while `details.artifacts[].exists` is the wrapper's on-disk verification signal. |
|
|
847
|
+
| `wait --download [path] --timeout <ms>` | Set download-start timeout in milliseconds. The native Pi wrapper forwards explicit wait timeouts and extends the subprocess watchdog unless the caller supplies top-level `timeoutMs`. |
|
|
832
848
|
|
|
833
849
|
Current upstream still does not parse `wait <selector> --state hidden` / `wait <selector> --state detached` as distinct wait modes even though upstream help mentions those examples. Use `wait --fn "!document.querySelector('#spinner')"` or another explicit JavaScript predicate for disappearance/detach checks until upstream parser support exists.
|
|
834
850
|
|
|
835
851
|
### Diff, debug, and streaming
|
|
836
852
|
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
| `pushstate <url>` | Perform SPA client-side navigation; detects Next.js router pushes and falls back to history navigation events. |
|
|
863
|
-
| `removeinitscript <id>` | Remove an init script registered through upstream init-script mechanisms. |
|
|
853
|
+
- `diff snapshot` — Compare current versus last snapshot. Use `diff snapshot --baseline <file> --selector <sel> --compact --depth <n>` when you need a saved baseline, scoped subtree, compact output, or depth bound.
|
|
854
|
+
- `diff screenshot --baseline` — Compare current screenshot versus a baseline image. Use `diff screenshot --baseline <file> --output <file> --threshold <0-1> --selector <sel> --full` when you need a saved diff image, threshold tuning, element scope, or full-page capture.
|
|
855
|
+
- `diff url <u1> <u2>` — Navigate to both pages and compare them, leaving the second destination active. The wrapper observes the final URL, including redirects to `about:blank`, and invalidates old refs without recovering the old tab; direct and reached batch rows use the same rule. If the URL cannot be observed, run `get url` before taking a fresh snapshot. Use `diff url <u1> <u2> --screenshot --wait-until <strategy> --selector <sel> --compact --depth <n>` when you need screenshot comparison, navigation wait control, or scoped/compact snapshot comparison.
|
|
856
|
+
- `trace start`, `trace stop [path]` — Record a Chrome DevTools trace.
|
|
857
|
+
- `profiler start|stop [path]` — Record a Chrome DevTools profile.
|
|
858
|
+
- `record start <path> [url]` — Record the active page; an optional URL navigates first. Use `.webm` or `.mp4` and optional `--fps <n>` (1–60, default 30); native validates startup and requires `ffmpeg` on `PATH`. Verify output after `record stop`.
|
|
859
|
+
- `record stop` — Finalize video and inspect its native receipt plus wrapper file verification. A failed or recovered stop retains original attempt evidence; use a distinct top-level `outputPath` to save its receipt.
|
|
860
|
+
- `record restart <path> [url]` — Stop any current recording and start a new video. Supports the same formats and `--fps` option; without a URL it keeps the page and refs.
|
|
861
|
+
- `console [--clear]` — View or clear console logs.
|
|
862
|
+
- `errors [--clear]` — View or clear page errors.
|
|
863
|
+
- `highlight <sel>` — Highlight an element.
|
|
864
|
+
- `inspect` — Open Chrome DevTools for the active page.
|
|
865
|
+
- `clipboard <op> [text]` — Read/write clipboard: `clipboard read`, `clipboard write <text>`, `clipboard copy`, and `clipboard paste`. Clipboard access is environment-dependent; `NotAllowedError` / permission-denied failures are common in headless, managed-profile, remote, or `file://` sessions.
|
|
866
|
+
- `stream enable [--port <n>]` — Start runtime WebSocket streaming for this session. If upstream reports that streaming is already enabled, the wrapper treats it as an idempotent success and adds status/disable follow-ups.
|
|
867
|
+
- `stream disable` — Stop runtime WebSocket streaming.
|
|
868
|
+
- `stream status` — Show streaming status and active port.
|
|
869
|
+
- `react tree` — Print the full React component tree. Requires the page to have been launched with `--enable react-devtools`.
|
|
870
|
+
- `react inspect <id>` — Inspect one React fiber's props, hooks, state, and source.
|
|
871
|
+
- `react renders start` — Start recording React render activity.
|
|
872
|
+
- `react renders stop [--json]` — Stop render recording and print mount/re-render counts and changed details.
|
|
873
|
+
- `react suspense [--only-dynamic] [--json]` — Classify Suspense boundaries with grouped root-cause recommendations.
|
|
874
|
+
- `a11y [url]` — Run an embedded axe-core accessibility audit on the current page, or navigate to `url` first. Options: `a11y --tags wcag2a,wcag2aa`, `a11y --selector "#main"`. CDP browsers only (not Safari/iOS WebDriver). Model-facing text summarizes violation/incomplete counts and top rules; full node targets stay in `details.data`.
|
|
875
|
+
- `vitals [url] [--json]` — Report Core Web Vitals: LCP, CLS, TTFB, FCP, INP, plus React hydration timing when available. `web-vitals [url] [--json]` is the upstream alias.
|
|
876
|
+
- `pushstate <url>` — Perform SPA client-side navigation; detects Next.js router pushes and falls back to history navigation events.
|
|
877
|
+
- `removeinitscript <id>` — Remove an init script registered through upstream init-script mechanisms.
|
|
864
878
|
|
|
865
879
|
Recording destinations are reserved within one Pi process, not across processes. Use unique paths for concurrent Pi processes: different explicit sessions can overwrite one file even when both `record stop` results are verified. Upstream’s same-session `record start` guard does not reserve the filename across other sessions.
|
|
866
880
|
|
|
@@ -872,37 +886,37 @@ Long-running or lifecycle commands should be explicitly paired with cleanup call
|
|
|
872
886
|
|
|
873
887
|
### Batch, auth, confirmations, sessions, chat, dashboard, devices, and setup
|
|
874
888
|
|
|
875
|
-
| Command
|
|
876
|
-
|
|
|
877
|
-
| `batch [--bail] ["cmd" ...]`
|
|
878
|
-
| `auth save <name> [opts]`
|
|
879
|
-
| `auth login <name>`
|
|
880
|
-
| `auth list`
|
|
881
|
-
| `auth show <name>`
|
|
882
|
-
| `auth delete <name>`
|
|
883
|
-
| `confirm <id>`
|
|
884
|
-
| `deny <id>`
|
|
885
|
-
| `session`
|
|
886
|
-
| `session list`
|
|
887
|
-
| `chat <message>`
|
|
888
|
-
| `chat`
|
|
889
|
-
| `dashboard [start]`
|
|
890
|
-
| `dashboard start --port <n>`
|
|
891
|
-
| `dashboard start --allowed-origins <origins>`
|
|
892
|
-
| `dashboard stop`
|
|
893
|
-
| `device list`
|
|
894
|
-
| `install`
|
|
895
|
-
| `install --with-deps`
|
|
896
|
-
| `upgrade`
|
|
897
|
-
| `doctor [--fix]`
|
|
898
|
-
| `plugin add <ref>`
|
|
899
|
-
| `plugin [list]`
|
|
900
|
-
| `plugin show <name>`
|
|
901
|
-
| `plugin run <name> <type>`
|
|
902
|
-
| `auth login <name> --credential-provider <plugin>` | Resolve credentials just-in-time from a configured credential plugin (e.g. a vault) instead of saved passwords; pair with `--item <ref>` and optional selector overrides. Credentials are not stored locally.
|
|
903
|
-
| `mcp --help`
|
|
904
|
-
| `mcp`
|
|
905
|
-
| `profiles`
|
|
889
|
+
| Command | Purpose |
|
|
890
|
+
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
891
|
+
| `batch [--bail] ["cmd" ...]` | Execute multiple commands sequentially from args or stdin. |
|
|
892
|
+
| `auth save <name> [opts]` | Save an auth profile. Full credential form: `auth save <name> --url <url> --username <user> --password <pass>`; selector override form: `auth save <name> --username-selector <s> --password-selector <s> --submit-selector <s>`. Prefer `auth save <name> --password-stdin` with the tool `stdin` field; avoid putting passwords in `args`. |
|
|
893
|
+
| `auth login <name>` | Login using saved credentials; add `--no-navigate` for a prepared, same-origin login page. |
|
|
894
|
+
| `auth list` | List saved auth profiles. |
|
|
895
|
+
| `auth show <name>` | Show auth profile metadata. |
|
|
896
|
+
| `auth delete <name>` | Delete an auth profile; `auth remove <name>` is the upstream alias. |
|
|
897
|
+
| `confirm <id>` | Approve a pending action. |
|
|
898
|
+
| `deny <id>` | Deny a pending action. |
|
|
899
|
+
| `session` | Show current session name. |
|
|
900
|
+
| `session list` | List active sessions. |
|
|
901
|
+
| `chat <message>` | Send a natural-language instruction. |
|
|
902
|
+
| `chat` | Start interactive chat when stdin is a TTY. |
|
|
903
|
+
| `dashboard [start]` | Start the dashboard server on the default port `4848`. |
|
|
904
|
+
| `dashboard start --port <n>` | Start the dashboard on a specific port. |
|
|
905
|
+
| `dashboard start --allowed-origins <origins>` | Allow comma-separated exact HTTPS reverse-proxy origins. Environment: `AGENT_BROWSER_DASHBOARD_ALLOWED_ORIGINS`. |
|
|
906
|
+
| `dashboard stop` | Stop the dashboard server. |
|
|
907
|
+
| `device list` | List available iOS simulators. Use with `-p ios` when exercising iOS provider flows. |
|
|
908
|
+
| `install` | Install browser binaries. |
|
|
909
|
+
| `install --with-deps` | Install browser binaries plus Linux system dependencies; exits nonzero when required libraries cannot be installed. |
|
|
910
|
+
| `upgrade` | Upgrade `agent-browser` using its detected package manager. Native output is text, even with `--json`; the wrapper displays it with surrounding whitespace trimmed and normal redaction, and exposes it in `details.data`. Caller-requested `--json` stays a parseable result with the text in `data`. Nonzero exits, spawn failures, timeout and cancellation remain failures; failed upgrade stdout and stderr remain available as diagnostics. |
|
|
911
|
+
| `doctor [--fix]` | Diagnose install issues and optionally auto-clean stale files. Use `doctor --offline --quick` for a fast local-only check and `doctor --json` for structured output. |
|
|
912
|
+
| `plugin add <ref>` | Add a plugin from npm or GitHub (`<owner>/<repo>` or `@scope/<name>`); writes `agent-browser.json`. Flags such as `--name`, `--capability`, `--global`, and `--no-manifest` shape discovery. |
|
|
913
|
+
| `plugin [list]` | List configured plugins (default subcommand); `{ "plugins": [...] }` is a successful sessionless result. |
|
|
914
|
+
| `plugin show <name>` | Show one configured plugin; `{ "plugin": {...} }` is a successful sessionless result. |
|
|
915
|
+
| `plugin run <name> <type>` | Run a `command.run` or custom plugin request over the agent-browser plugin stdio protocol. |
|
|
916
|
+
| `auth login <name> --credential-provider <plugin>` | Resolve credentials just-in-time from a configured credential plugin (e.g. a vault) instead of saved passwords; pair with `--item <ref>` and optional selector overrides. Credentials are not stored locally. |
|
|
917
|
+
| `mcp --help` | Show MCP server help through the native tool. |
|
|
918
|
+
| `mcp` | Start a local MCP stdio server for external MCP clients; bare native-tool calls are rejected before spawn. External clients can opt into experimental page tools with `mcp --tools core,webmcp`. |
|
|
919
|
+
| `profiles` | List available Chrome profiles. |
|
|
906
920
|
|
|
907
921
|
When these commands are invoked through the native `agent_browser` tool, structured diagnostic/status outputs are rendered as compact summaries. `session list` and `state list` keep every upstream row and restore identifier visible. Explicit sessions, state/restore paths, broad state lifecycle commands, config, local files, and launch environment pass through unchanged. Local inspection/setup calls remain sessionless unless you explicitly pass `--session`; browser-backed or context-dependent calls keep normal managed-session behavior when no explicit session is supplied. List-like outputs such as sessions, Chrome profiles, auth profiles, network requests, console messages, and page errors include counts and key fields; large outputs are previewed with a `Full output path:` spill file instead of dumping the entire payload into context. For `network requests`, the wrapper shows a failed-request summary split into actionable versus benign low-impact rows, then status, method, URL, resource/mime type, request id, and, when the installed upstream output includes body-like fields, bounded redacted payload, response, and failure/error snippets. Safe request IDs also produce `details.nextActions` for exact request details, actionable failed-request source lookup candidates, filtered request lists, or starting HAR capture before a repro. If the same session has active wrapper-observed network routes, failed/pending/CORS-looking matched request rows add `details.networkRouteDiagnostics` and executable route-mock next actions before the generic request actions. `data:image` artifact rows are omitted from compact request previews but remain in raw `details.data.requests`. `network request <requestId>` can expose upstream full-detail body fields such as response bodies using the same bounded model-facing preview; its request URL stays diagnostic-only and does not overwrite `details.sessionTabTarget` for later ref guards. Clipboard failures that mention `NotAllowedError` or permission denial are usually browser/OS capability limits, not proof that a read, paste, or page mutation happened; prefer page-native reads (`snapshot -i`, `get text`, `eval --stdin`) or direct typing (`keyboard inserttext` / `keyboard type`) when the workflow allows it, and retry true clipboard flows only from an allowed profile/session on a normal `http(s)` page. Header, cookie, auth, token, and other secret-like fields are not expanded in model-facing text or `details.data`; low-risk primitive storage values may remain visible, while command echoes still redact `--body`, `--headers`, `--password`, proxy credentials, auth-bearing URLs, `clipboard write` text, cookie/storage set values, and bearer/basic credential text in positional arguments. Raw batch row strings receive the same command-aware redaction in visible output and ordered code journals without changing execution argv. Use upstream HAR or full raw details only when complete data is required.
|
|
908
922
|
|
|
@@ -957,9 +971,9 @@ For Exa, effective search type precedence is per-call `searchType` → `webSearc
|
|
|
957
971
|
|
|
958
972
|
```json
|
|
959
973
|
{
|
|
960
|
-
|
|
961
|
-
|
|
962
|
-
|
|
974
|
+
"query": "pi-agent-browser-native agent_browser_web_search searchType defaults",
|
|
975
|
+
"searchType": "deep-lite",
|
|
976
|
+
"count": 5
|
|
963
977
|
}
|
|
964
978
|
```
|
|
965
979
|
|
|
@@ -971,21 +985,21 @@ Example config:
|
|
|
971
985
|
|
|
972
986
|
```json
|
|
973
987
|
{
|
|
974
|
-
|
|
975
|
-
|
|
976
|
-
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
|
|
981
|
-
|
|
982
|
-
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
988
|
+
"version": 1,
|
|
989
|
+
"webSearch": {
|
|
990
|
+
"enabled": true,
|
|
991
|
+
"preferredProvider": "exa",
|
|
992
|
+
"defaultSearchType": "deep-lite",
|
|
993
|
+
"exaApiKey": "$EXA_API_KEY",
|
|
994
|
+
"braveApiKey": "$BRAVE_API_KEY"
|
|
995
|
+
},
|
|
996
|
+
"browser": {
|
|
997
|
+
"defaultProfile": {
|
|
998
|
+
"name": "Profile 1",
|
|
999
|
+
"policy": "authenticated-only"
|
|
1000
|
+
},
|
|
1001
|
+
"executablePath": "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
|
|
1002
|
+
}
|
|
989
1003
|
}
|
|
990
1004
|
```
|
|
991
1005
|
|
|
@@ -1013,7 +1027,7 @@ Browser defaults are scoped: global/override `defaultProfile` with `policy: "alw
|
|
|
1013
1027
|
|
|
1014
1028
|
### Browser launch and runtime flags
|
|
1015
1029
|
|
|
1016
|
-
The extension composes stock `--args --no-startup-window` for local Chrome startup
|
|
1030
|
+
The extension composes stock `--args --no-startup-window` for local Chrome startup only when native environment/config does not already supply local launch defaults. Otherwise no automatic Chrome startup arguments are injected; include `--no-startup-window` in native `args` if a single headed window is required. Headless remains the default, external engines/attachments stay native-owned, and caller settings retain native precedence. Plain active follow-ups do not repeat wrapper bootstrap settings; changes to launch settings can trigger native's normal restart. URL-less `open` runs native `get url` to lazily launch or preserve the active page, including batch/code calls; the result exposes the actual command and URL, not a fabricated launch receipt. See [the startup contract](TOOL_CONTRACT.md#headed-and-local-fixture-limits).
|
|
1017
1031
|
|
|
1018
1032
|
- `--executable-path <path>`: custom Chromium-compatible browser executable, such as Brave/Edge/Arc/Vivaldi when upstream can launch that binary. Environment: `AGENT_BROWSER_EXECUTABLE_PATH`.
|
|
1019
1033
|
- `--extension <path>`: load browser extensions; repeatable. Environment: `AGENT_BROWSER_EXTENSIONS`.
|
|
@@ -1036,7 +1050,7 @@ The extension composes stock `--args --no-startup-window` for local Chrome start
|
|
|
1036
1050
|
|
|
1037
1051
|
On Android/Termux, follow the README setup to install the packaged Linux-musl arm64 upstream binary, install Termux's `which`, and expose its launcher as `$PREFIX/bin/chromium`. Prefer that upstream system-browser discovery over ambient `AGENT_BROWSER_EXECUTABLE_PATH`: it survives isolated `HOME` values and works for ordinary native calls. Code retains the selected browser's normal launch configuration. Wrapper-generated Android managed identities use a compact 80-bit digest so ordinary namespaces and fresh rotations fit upstream's Unix socket path.
|
|
1038
1052
|
|
|
1039
|
-
- `--no-auto-dialog`: disable automatic dismissal of alert/beforeunload dialogs. Environment: `AGENT_BROWSER_NO_AUTO_DIALOG`.
|
|
1053
|
+
- `--no-auto-dialog`: disable automatic dismissal of alert/beforeunload dialogs. Environment: `AGENT_BROWSER_NO_AUTO_DIALOG`. Explicit `true`/`false` reaches all helpers in that call; keep the same selection on follow-ups to avoid native daemon replacement.
|
|
1040
1054
|
- `--idle-timeout <ms>`: native background browser lifecycle setting (also accepts `10s`, `3m`, `1h`). Caller-owned sessions retain native idle policy; an explicit flag is carried to every helper in that call. Keep it consistent between calls to avoid a native daemon restart. Only wrapper-owned sessions receive the implicit timeout and numeric mismatch check against `PI_AGENT_BROWSER_IMPLICIT_SESSION_IDLE_TIMEOUT_MS`.
|
|
1041
1055
|
|
|
1042
1056
|
### Lightpanda engine
|
|
@@ -1069,7 +1083,7 @@ On Android/Termux, follow the README setup to install the packaged Linux-musl ar
|
|
|
1069
1083
|
- `--content-boundaries`: wrap page output in boundary markers. Environment: `AGENT_BROWSER_CONTENT_BOUNDARIES`.
|
|
1070
1084
|
- `--max-output <chars>`: truncate page output to N characters. Environment: `AGENT_BROWSER_MAX_OUTPUT`.
|
|
1071
1085
|
- `--allowed-domains <list>`: restrict browser and `read` traffic to exact or `*.` wildcard domain patterns. Environment: `AGENT_BROWSER_ALLOWED_DOMAINS`. Use a fresh local Chrome context; upstream 0.32.0 owns containment and incompatible-mode rejection and disables Chromium `RTCPeerConnection` while active. The wrapper passes the setting and result through unchanged.
|
|
1072
|
-
- `--action-policy <path>`: action policy JSON file. Environment: `AGENT_BROWSER_ACTION_POLICY`.
|
|
1086
|
+
- `--action-policy <path>`: action policy JSON file. Environment: `AGENT_BROWSER_ACTION_POLICY`. Explicit policy values also reach internal helpers for the current call. Keep the action-policy path consistent on follow-ups; changing or omitting it can restart the native daemon and discard pending confirmations. The established `--confirm-actions` setting is retained for same-session follow-ups.
|
|
1073
1087
|
- `--confirm-actions <list>`: action categories requiring confirmation. Environment: `AGENT_BROWSER_CONFIRM_ACTIONS`.
|
|
1074
1088
|
- `--confirm-interactive`: interactive confirmations; auto-denies when stdin is not a TTY. Environment: `AGENT_BROWSER_CONFIRM_INTERACTIVE`.
|
|
1075
1089
|
- `-p, --provider <name>`: provider such as `ios`, `browserbase`, `kernel`, `browseruse`, `browserless`, or `agentcore`. Environment: `AGENT_BROWSER_PROVIDER`.
|
|
@@ -1078,7 +1092,7 @@ On Android/Termux, follow the README setup to install the packaged Linux-musl ar
|
|
|
1078
1092
|
- `--model <name>`: AI model for `chat`. Environment: `AI_GATEWAY_MODEL`.
|
|
1079
1093
|
- `-v, --verbose`: show tool commands and raw output.
|
|
1080
1094
|
- `-q, --quiet`: show only AI text responses.
|
|
1081
|
-
- `--debug`: debug output. Environment: `AGENT_BROWSER_DEBUG`.
|
|
1095
|
+
- `--debug`: debug output. Environment: `AGENT_BROWSER_DEBUG`. Explicit `true`/`false` reaches all helpers in that call, including overriding true config defaults; `false` removes the debug variable so native presence checks cannot enable logging. Inherited env/config behavior is unchanged.
|
|
1082
1096
|
- `AGENT_BROWSER_PLUGINS`: JSON plugin registry override for the upstream `plugin` commands.
|
|
1083
1097
|
- `--version`, `-V`: show version.
|
|
1084
1098
|
|
|
@@ -1102,15 +1116,19 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
1102
1116
|
- The extension may keep following one implicit managed session across later tool calls.
|
|
1103
1117
|
- If launch-scoped flags like `--engine`, `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `--provider` / `-p`, or provider device flags like `--device` would replace or be ignored by an already-active managed session, retry with `sessionMode: "fresh"`. When the call explicitly names the current managed session, the structured recovery payload removes that `--session` so the fresh rotation can succeed.
|
|
1104
1118
|
- If a `sessionMode: "fresh"` call fails (including upstream failure, timeout, missing binary, or **`qa`** reclassification after a nominally successful batch), read `details.managedSessionOutcome` before assuming where the next default call will go: `preserved` means the prior managed session remains current, while `abandoned` means no managed session became current. When the failure reason is not the fresh launch itself—for example `failureCategory: "qa-failure"`—`status`/`summary` may still describe the managed-session transition while `succeeded` on this object matches the final tool outcome.
|
|
1119
|
+
|
|
1105
1120
|
<!-- agent-browser-playbook:start wrapper-tab-recovery -->
|
|
1106
1121
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
1122
|
+
|
|
1107
1123
|
- After open/goto/navigate calls with --profile, --restore, --session-name, or --state, agent_browser best-effort re-selects the tab whose URL matches the returned page when restored tabs steal focus during launch or reconnect.
|
|
1108
1124
|
- After confirmed shutdown of an automatically restored managed session, the wrapper retains its complete recorded URL, including the fragment, until the first current-page operation (including get url and reload). Non-page calls such as tab list may start a daemon without fulfilling that reopen; explicit URL reads leave the managed browser and pending reopen untouched. The wrapper uses native open once, verifies the observed tab, and discards old refs/frame scope; it does not restore unsaved forms, JavaScript memory, or history. Explicit navigation, caller-owned/attached sessions, and restore-disabled sessions are not auto-reopened.
|
|
1109
1125
|
- For a still-live browser after tab drift or resume, the wrapper verifies/selects the intended tab before ref/semantic helpers and page commands; failed selection stops the call without navigating. Local commands, read <url>, URL a11y/vitals, diff url, window new, and explicit tab/navigation/connection/state recovery do not require the prior tab. Batch checks follow effective rows past non-page prefixes and stop at explicit context changes, preserving caller argv/stdin and continue-on-error behavior. Same-tab reselection is avoided because it clears refs. Use exact batch --bail for fail-fast, not --bail=<value>. Routine same-session calls skip tab-list preflights.
|
|
1110
1126
|
- For sessions with observed tab-drift risk, after a successful command on a known target tab, agent_browser also best-effort restores that intended tab if a restored/background tab steals focus after the command completes. Routine same-session commands skip focus correction; observed pages without a native targetId use tab list to establish their identity.
|
|
1111
1127
|
- If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
|
|
1112
1128
|
- If upstream reports tab_gone, the pinned bound tab is gone; use details.nextActions (tab list / tab new) instead of assuming another tab is yours.
|
|
1129
|
+
|
|
1113
1130
|
<!-- agent-browser-playbook:end wrapper-tab-recovery -->
|
|
1131
|
+
|
|
1114
1132
|
- Wrapper-spawned commands clamp `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented 25-second default and use a 35-second child-process watchdog (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides the default 35s budget; top-level `timeoutMs` overrides it per browser CLI call). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a subprocess watchdog from the requested command duration plus a small grace window. Batch budgeting and timeout recovery read the same effective source as upstream: raw command strings when present, otherwise stdin rows. Dialog commands are additionally bounded to 5 seconds (`PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS`), and click/tap/find refs or tokens plus `eval --stdin` snippets that look like alert/confirm/prompt/dialog triggers are bounded to 8 seconds (`PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS`). A `session info` timeout returns only an exact-session `retry-session-info` status action, without page probes, liveness claims, or changes to existing page/ref state. When a browser-operation watchdog fires, `details.timeoutPartialProgress` may include a planned step list with `status: "unknown"` (including `generatedFrom` labels for generated QA rows). Current URLs and existing files do not establish which steps ran, failed, or remained pending. Multi-step timeouts offer `inspect-current-page-after-timeout` when the target is already verified, rather than guessing a retry position. A single read-only or idempotent step may offer a `retry-timeout-step` candidate with a one-row native batch (`args: ["batch"]` plus `stdin`); inspect the current state before retrying because its outcome is still unknown. If the target is unknown, standalone snapshots are removed and visible failure text plus `details.nextActions` show `verify-page-target-after-timeout`, including its session-scoped `batch --bail` args and short stdin for fail-fast `get url` then `snapshot -i`; dialog status/accept/dismiss actions remain allowed for blocking-dialog recovery. It also includes current page URL from best-effort session `get url`, followed by `get title` only after a URL is recovered (or a planned URL inferred from the step list when the session cannot answer), and declared artifact paths such as `screenshot`, `pdf`, `download`, or `wait --download` outputs with existence/state checks; the same evidence is appended under `Timeout partial progress` in visible text with URL/path redaction.
|
|
1115
1133
|
- Oversized snapshots and oversized generic outputs may be compacted in tool content, with the full redacted output written to a spill file path shown directly in the tool result. Recent artifact metadata is bounded by `PI_AGENT_BROWSER_SESSION_ARTIFACT_MANIFEST_MAX_ENTRIES` (default 100); persisted spill files have a separate `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES` budget (default 32 MiB; `0` disables automatic eviction).
|
|
1116
1134
|
- The wrapper keeps `--help` and `--version` stateless so they do not consume the implicit managed-session slot.
|
|
@@ -1125,6 +1143,7 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
1125
1143
|
This generated block is review data for maintainers. The human-authored reference sections above remain the readable command guide.
|
|
1126
1144
|
|
|
1127
1145
|
#### Source evidence
|
|
1146
|
+
|
|
1128
1147
|
- repository: `vercel-labs/agent-browser`
|
|
1129
1148
|
- upstream HEAD: `aff6125c023b810ea3f2e5deec5379e9a4270bdc`
|
|
1130
1149
|
- upstream package version: `0.38.1`
|
|
@@ -1187,6 +1206,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1187
1206
|
- inspected: `packages/@agent-browser/sandbox/src/eve.ts`
|
|
1188
1207
|
|
|
1189
1208
|
#### Upstream help commands sampled
|
|
1209
|
+
|
|
1190
1210
|
- root help: `agent-browser --help`
|
|
1191
1211
|
- skills help: `agent-browser skills --help`
|
|
1192
1212
|
- skills list: `agent-browser skills list`
|
|
@@ -1248,6 +1268,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1248
1268
|
- plugin help: `agent-browser plugin --help`
|
|
1249
1269
|
|
|
1250
1270
|
#### Inventory sections
|
|
1271
|
+
|
|
1251
1272
|
- Built-in skills: 19 human-doc token(s), 24 upstream token(s)
|
|
1252
1273
|
- Core page, element, navigation, and extraction commands: 89 human-doc token(s), 94 upstream token(s)
|
|
1253
1274
|
- Sessions, state, tabs, frames, dialogs, and windows: 28 human-doc token(s), 26 upstream token(s)
|
|
@@ -1256,7 +1277,9 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1256
1277
|
- Global flags, config, providers, policy, and environment: 153 human-doc token(s), 120 upstream token(s)
|
|
1257
1278
|
|
|
1258
1279
|
#### Human-authored doc tokens required
|
|
1280
|
+
|
|
1259
1281
|
##### Built-in skills
|
|
1282
|
+
|
|
1260
1283
|
- `skills list`
|
|
1261
1284
|
- `skills get core`
|
|
1262
1285
|
- `skills get core --full`
|
|
@@ -1278,6 +1301,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1278
1301
|
- `AGENT_BROWSER_SKILLS_DIR`
|
|
1279
1302
|
|
|
1280
1303
|
##### Core page, element, navigation, and extraction commands
|
|
1304
|
+
|
|
1281
1305
|
- `open [url]`
|
|
1282
1306
|
- `open <url>`
|
|
1283
1307
|
- `goto <url>`
|
|
@@ -1369,6 +1393,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1369
1393
|
- `swipe <direction> [distance]`
|
|
1370
1394
|
|
|
1371
1395
|
##### Sessions, state, tabs, frames, dialogs, and windows
|
|
1396
|
+
|
|
1372
1397
|
- `session`
|
|
1373
1398
|
- `session id`
|
|
1374
1399
|
- `session id --scope worktree --prefix <name>`
|
|
@@ -1399,6 +1424,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1399
1424
|
- `window new`
|
|
1400
1425
|
|
|
1401
1426
|
##### Network, storage, artifacts, diagnostics, and performance
|
|
1427
|
+
|
|
1402
1428
|
- `network <action>`
|
|
1403
1429
|
- `network route <url> [--abort|--body <json>] [--resource-type <csv>]`
|
|
1404
1430
|
- `network unroute [url]`
|
|
@@ -1463,6 +1489,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1463
1489
|
- `removeinitscript <id>`
|
|
1464
1490
|
|
|
1465
1491
|
##### Batch, auth, confirmations, setup, dashboard, devices, and AI commands
|
|
1492
|
+
|
|
1466
1493
|
- `batch [--bail]`
|
|
1467
1494
|
- `auth save <name>`
|
|
1468
1495
|
- `auth save <name> --url <url> --username <user> --password <pass>`
|
|
@@ -1502,6 +1529,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1502
1529
|
- `profiles`
|
|
1503
1530
|
|
|
1504
1531
|
##### Global flags, config, providers, policy, and environment
|
|
1532
|
+
|
|
1505
1533
|
- `--profile <name|path>`
|
|
1506
1534
|
- `AGENT_BROWSER_PROFILE`
|
|
1507
1535
|
- `--session <name>`
|
|
@@ -1657,7 +1685,9 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1657
1685
|
- `AWS_DEFAULT_REGION`
|
|
1658
1686
|
|
|
1659
1687
|
#### Upstream help tokens expected
|
|
1688
|
+
|
|
1660
1689
|
##### Built-in skills
|
|
1690
|
+
|
|
1661
1691
|
- root help: `skills get core --full`
|
|
1662
1692
|
- skills help: `get <name> --full`
|
|
1663
1693
|
- skills help: `get --all`
|
|
@@ -1684,6 +1714,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1684
1714
|
- core skill full: `implicit roles work`
|
|
1685
1715
|
|
|
1686
1716
|
##### Core page, element, navigation, and extraction commands
|
|
1717
|
+
|
|
1687
1718
|
- open help: `open [url]`
|
|
1688
1719
|
- open help: `aliases still require a URL.`
|
|
1689
1720
|
- open help: `agent-browser webmcp list`
|
|
@@ -1780,6 +1811,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1780
1811
|
- swipe help: `swipe <direction> [distance]`
|
|
1781
1812
|
|
|
1782
1813
|
##### Sessions, state, tabs, frames, dialogs, and windows
|
|
1814
|
+
|
|
1783
1815
|
- root help: `session list`
|
|
1784
1816
|
- session help: `id`
|
|
1785
1817
|
- session help: `info`
|
|
@@ -1808,6 +1840,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1808
1840
|
- window help: `window <operation>`
|
|
1809
1841
|
|
|
1810
1842
|
##### Network, storage, artifacts, diagnostics, and performance
|
|
1843
|
+
|
|
1811
1844
|
- root help: `network <action>`
|
|
1812
1845
|
- root help: `--resource-type <csv>`
|
|
1813
1846
|
- network help: `unroute [url]`
|
|
@@ -1881,6 +1914,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1881
1914
|
- errors help: `--clear`
|
|
1882
1915
|
|
|
1883
1916
|
##### Batch, auth, confirmations, setup, dashboard, devices, and AI commands
|
|
1917
|
+
|
|
1884
1918
|
- root help: `batch [--bail]`
|
|
1885
1919
|
- root help: `auth save <name>`
|
|
1886
1920
|
- root help: `auth login <name>`
|
|
@@ -1924,6 +1958,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1924
1958
|
- plugin help: `credential.read`
|
|
1925
1959
|
|
|
1926
1960
|
##### Global flags, config, providers, policy, and environment
|
|
1961
|
+
|
|
1927
1962
|
- root help: `--profile <name|path>`
|
|
1928
1963
|
- root help: `AGENT_BROWSER_PROFILE`
|
|
1929
1964
|
- root help: `--session <name>`
|