pi-agent-browser-native 0.9.3 → 0.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +106 -5
- package/README.md +58 -791
- package/dist/extensions/agent-browser/index.js +43101 -2031
- package/dist/extensions/agent-browser/lib/argv-descriptor.js +49 -26
- package/dist/extensions/agent-browser/lib/argv-grammar.js +82 -93
- package/dist/extensions/agent-browser/lib/bash-guard.js +129 -126
- package/dist/extensions/agent-browser/lib/batch-lifecycle.js +112 -51
- package/dist/extensions/agent-browser/lib/browser-execution-claim-codec.js +61 -0
- package/dist/extensions/agent-browser/lib/browser-execution-claim-storage.js +232 -0
- package/dist/extensions/agent-browser/lib/browser-execution-claims.js +202 -0
- package/dist/extensions/agent-browser/lib/browser-execution-identity.js +77 -0
- package/dist/extensions/agent-browser/lib/browser-journal-branch.js +98 -0
- package/dist/extensions/agent-browser/lib/browser-journal-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/browser-journal-projection.js +221 -0
- package/dist/extensions/agent-browser/lib/browser-journal-reader.js +89 -0
- package/dist/extensions/agent-browser/lib/browser-journal-replay.js +246 -0
- package/dist/extensions/agent-browser/lib/browser-journal.js +103 -394
- package/dist/extensions/agent-browser/lib/browser-legacy-pages.js +220 -0
- package/dist/extensions/agent-browser/lib/browser-legacy-projection.js +282 -0
- package/dist/extensions/agent-browser/lib/browser-legacy-types.js +1 -0
- package/dist/extensions/agent-browser/lib/browser-session-conversion-stream.js +217 -0
- package/dist/extensions/agent-browser/lib/browser-session-conversion.js +80 -335
- package/dist/extensions/agent-browser/lib/browser-transcript-contracts.js +35 -0
- package/dist/extensions/agent-browser/lib/browser-transcript-validation.js +166 -0
- package/dist/extensions/agent-browser/lib/browser-transcript.js +137 -82
- package/dist/extensions/agent-browser/lib/browser-value-parsing.js +15 -0
- package/dist/extensions/agent-browser/lib/command-capabilities.js +340 -0
- package/dist/extensions/agent-browser/lib/command-policy.js +107 -88
- package/dist/extensions/agent-browser/lib/command-taxonomy.js +49 -322
- package/dist/extensions/agent-browser/lib/config-policy.js +138 -343
- package/dist/extensions/agent-browser/lib/config-providers.js +71 -0
- package/dist/extensions/agent-browser/lib/config-validation.js +231 -0
- package/dist/extensions/agent-browser/lib/config.js +20 -8
- package/dist/extensions/agent-browser/lib/electron/cdp.js +20 -8
- package/dist/extensions/agent-browser/lib/electron/cleanup.js +128 -83
- package/dist/extensions/agent-browser/lib/electron/discovery-darwin.js +109 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-desktop.js +136 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-evidence.js +90 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-linux.js +93 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-packages.js +162 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-types.js +2 -0
- package/dist/extensions/agent-browser/lib/electron/discovery.js +140 -549
- package/dist/extensions/agent-browser/lib/electron/launch-monitor.js +102 -0
- package/dist/extensions/agent-browser/lib/electron/launch-process.js +209 -0
- package/dist/extensions/agent-browser/lib/electron/launch-types.js +2 -0
- package/dist/extensions/agent-browser/lib/electron/launch.js +261 -355
- package/dist/extensions/agent-browser/lib/executable-path.js +21 -12
- package/dist/extensions/agent-browser/lib/execution-cwd.js +35 -18
- package/dist/extensions/agent-browser/lib/fs-utils.js +1 -9
- package/dist/extensions/agent-browser/lib/input-modes/electron.js +162 -121
- package/dist/extensions/agent-browser/lib/input-modes/job.js +123 -317
- package/dist/extensions/agent-browser/lib/input-modes/lookup-network-evidence.js +116 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-plan.js +132 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-redaction.js +57 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-source-evidence.js +151 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-workspace.js +142 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookups.js +58 -412
- package/dist/extensions/agent-browser/lib/input-modes/params.js +86 -40
- package/dist/extensions/agent-browser/lib/input-modes/qa-analysis.js +268 -0
- package/dist/extensions/agent-browser/lib/input-modes/qa-presentation.js +117 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-child.js +396 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-protocol.js +86 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-types.js +1 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-validation.js +60 -0
- package/dist/extensions/agent-browser/lib/input-modes/script.js +60 -409
- package/dist/extensions/agent-browser/lib/input-modes/semantic-action.js +218 -128
- package/dist/extensions/agent-browser/lib/input-modes/shared.js +15 -4
- package/dist/extensions/agent-browser/lib/input-modes/types.js +56 -7
- package/dist/extensions/agent-browser/lib/json-schema.js +22 -10
- package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +17 -5
- package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +92 -425
- package/dist/extensions/agent-browser/lib/managed-session-posix-storage.js +58 -0
- package/dist/extensions/agent-browser/lib/managed-session-project-identity.js +244 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-context.js +84 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-policy.js +119 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-state.js +83 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore.js +168 -300
- package/dist/extensions/agent-browser/lib/managed-session-snapshot-records.js +277 -0
- package/dist/extensions/agent-browser/lib/managed-session-snapshots.js +128 -305
- package/dist/extensions/agent-browser/lib/managed-session-storage.js +56 -253
- package/dist/extensions/agent-browser/lib/native-confirmation.js +140 -0
- package/dist/extensions/agent-browser/lib/native-output.js +43 -9
- package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +106 -59
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-merge.js +14 -6
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +109 -57
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/browser-observation-result.js +82 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +164 -60
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostic-values.js +31 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +152 -781
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-observation.js +203 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-text-diagnostics.js +109 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/fill-diagnostics.js +119 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-action-recovery.js +147 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-actions.js +144 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-content.js +123 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-details.js +156 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-electron.js +187 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-evidence.js +39 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-missing-binary.js +89 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-recovery.js +97 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-redaction.js +106 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-timeout.js +110 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +41 -538
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +199 -154
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/interaction-diagnostics.js +290 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-close.js +167 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-inspection.js +108 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +127 -205
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/native-helper-observation.js +65 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/observation-types.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/overlay-diagnostics.js +120 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/network-page-filter.js +113 -30
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +109 -28
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter-data.js +150 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +147 -253
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-rendered-text.js +147 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/wait-timeouts.js +59 -24
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-artifacts.js +159 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-command.js +131 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-daemon.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-failure.js +42 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-guards.js +242 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-input.js +75 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-observations.js +92 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-page-selection.js +254 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-page-state.js +121 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-refs.js +130 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-resources.js +114 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-semantic.js +121 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-session-plan.js +222 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-timeouts.js +68 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +176 -1123
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-analysis.js +191 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-confirmation.js +152 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-diagnostics.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-electron.js +150 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-phase.js +115 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-managed-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-managed.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native-phase.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native.js +113 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-navigation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-navigation.js +316 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-ownership.js +28 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-page-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-page-phase.js +142 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-presentation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-presentation.js +333 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication-phase.js +71 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication.js +201 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-receipts.js +183 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-refs.js +200 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-tabs.js +142 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +18 -968
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prompt-guards.js +23 -13
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-receipt-reconciliation.js +198 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery-presentation.js +154 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +68 -151
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-capture.js +14 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +169 -46
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-commands.js +130 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-electron.js +212 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-observations.js +61 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-outcomes.js +187 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-refs.js +169 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-tabs.js +228 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +77 -765
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/text-visibility-diagnostics.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/timeout-diagnostics.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/timeout-progress.js +108 -0
- package/dist/extensions/agent-browser/lib/orchestration/code-artifact-observation.js +39 -0
- package/dist/extensions/agent-browser/lib/orchestration/code-image-observation.js +119 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/branch.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/cleanup-result.js +58 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/cleanup.js +78 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/discovery.js +106 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +48 -821
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/policy.js +86 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-collection.js +68 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-data.js +169 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-presentation.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe.js +161 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/status.js +167 -0
- package/dist/extensions/agent-browser/lib/orchestration/execution-queue.js +22 -10
- package/dist/extensions/agent-browser/lib/orchestration/extension-artifact-preflight.js +211 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-artifacts.js +23 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-attachment-replay.js +99 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-branch-restore.js +275 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-code.js +280 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-admission.js +303 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-output.js +36 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-result.js +335 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-run.js +227 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-state.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-electron-ownership.js +204 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-execution-locks.js +139 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-execution-queue.js +24 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-host-coordination.js +98 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-host-run.js +295 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-intent-replay.js +41 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-invocation.js +7 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-lifecycle.js +102 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-managed-ownership.js +133 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-observation.js +120 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-prompt.js +180 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-recording.js +290 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-resource-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-resource-replay.js +269 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-result-state.js +113 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-runtime.js +146 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-script-leases.js +58 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-tool-boundary.js +62 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-version-check.js +54 -0
- package/dist/extensions/agent-browser/lib/orchestration/input-plan-types.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +261 -185
- package/dist/extensions/agent-browser/lib/orchestration/input-redaction.js +62 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-config.js +144 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-root-launch.js +77 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +210 -157
- package/dist/extensions/agent-browser/lib/orchestration/operation-paths.js +124 -58
- package/dist/extensions/agent-browser/lib/orchestration/output-file.js +197 -99
- package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +199 -96
- package/dist/extensions/agent-browser/lib/owned-managed-session-context.js +55 -0
- package/dist/extensions/agent-browser/lib/page-target-batch.js +118 -0
- package/dist/extensions/agent-browser/lib/page-target-guards.js +50 -0
- package/dist/extensions/agent-browser/lib/page-target-navigation.js +104 -0
- package/dist/extensions/agent-browser/lib/page-target-validation.js +55 -246
- package/dist/extensions/agent-browser/lib/parsing.js +13 -4
- package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +133 -68
- package/dist/extensions/agent-browser/lib/playbook.js +16 -12
- package/dist/extensions/agent-browser/lib/process-child.js +284 -0
- package/dist/extensions/agent-browser/lib/process-environment.js +13 -6
- package/dist/extensions/agent-browser/lib/process-errors.js +23 -0
- package/dist/extensions/agent-browser/lib/process-identity.js +47 -15
- package/dist/extensions/agent-browser/lib/process-socket-storage.js +256 -0
- package/dist/extensions/agent-browser/lib/process-stdout.js +74 -0
- package/dist/extensions/agent-browser/lib/process.js +145 -536
- package/dist/extensions/agent-browser/lib/prompt-artifacts.js +287 -0
- package/dist/extensions/agent-browser/lib/prompt-policy.js +15 -194
- package/dist/extensions/agent-browser/lib/read-command.js +41 -0
- package/dist/extensions/agent-browser/lib/read-confirmation.js +131 -68
- package/dist/extensions/agent-browser/lib/recording-reservations.js +118 -45
- package/dist/extensions/agent-browser/lib/redaction-fields.js +75 -0
- package/dist/extensions/agent-browser/lib/redaction-json.js +149 -0
- package/dist/extensions/agent-browser/lib/redaction-url.js +88 -0
- package/dist/extensions/agent-browser/lib/results/action-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/action-recommendations.js +28 -271
- package/dist/extensions/agent-browser/lib/results/artifact-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/artifact-manifest-validation.js +133 -0
- package/dist/extensions/agent-browser/lib/results/artifact-manifest.js +96 -94
- package/dist/extensions/agent-browser/lib/results/artifact-recommendations.js +73 -0
- package/dist/extensions/agent-browser/lib/results/categories.js +111 -54
- package/dist/extensions/agent-browser/lib/results/confirmation.js +29 -13
- package/dist/extensions/agent-browser/lib/results/editable-ref-evidence.js +62 -27
- package/dist/extensions/agent-browser/lib/results/electron-recommendations.js +83 -0
- package/dist/extensions/agent-browser/lib/results/envelope-decoding.js +48 -0
- package/dist/extensions/agent-browser/lib/results/envelope.js +90 -67
- package/dist/extensions/agent-browser/lib/results/evidence-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/failure-recommendations.js +180 -0
- package/dist/extensions/agent-browser/lib/results/network-routes.js +86 -45
- package/dist/extensions/agent-browser/lib/results/network.js +35 -12
- package/dist/extensions/agent-browser/lib/results/next-actions.js +49 -19
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-files.js +320 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-format.js +138 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-images.js +113 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-verification.js +145 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +99 -555
- package/dist/extensions/agent-browser/lib/results/presentation/batch-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-recording.js +118 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-redaction.js +68 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-rendering.js +135 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-step.js +227 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-summary.js +91 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch.js +91 -460
- package/dist/extensions/agent-browser/lib/results/presentation/browser-profile-recovery.js +9 -5
- package/dist/extensions/agent-browser/lib/results/presentation/command-presenters.js +216 -0
- package/dist/extensions/agent-browser/lib/results/presentation/common.js +26 -9
- package/dist/extensions/agent-browser/lib/results/presentation/content.js +86 -29
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-a11y.js +67 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-health.js +79 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-identity.js +199 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-logs.js +75 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-network-actions.js +216 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-network.js +124 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-redaction.js +64 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-storage.js +209 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-stream.js +69 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-summary.js +222 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +103 -1052
- package/dist/extensions/agent-browser/lib/results/presentation/early-result.js +70 -0
- package/dist/extensions/agent-browser/lib/results/presentation/errors.js +130 -60
- package/dist/extensions/agent-browser/lib/results/presentation/input-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +60 -145
- package/dist/extensions/agent-browser/lib/results/presentation/navigation.js +147 -88
- package/dist/extensions/agent-browser/lib/results/presentation/notices.js +75 -0
- package/dist/extensions/agent-browser/lib/results/presentation/observation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/observation-rendering.js +127 -0
- package/dist/extensions/agent-browser/lib/results/presentation/outcomes.js +120 -0
- package/dist/extensions/agent-browser/lib/results/presentation/output-completion.js +77 -0
- package/dist/extensions/agent-browser/lib/results/presentation/output-spill.js +51 -0
- package/dist/extensions/agent-browser/lib/results/presentation/registry.js +116 -240
- package/dist/extensions/agent-browser/lib/results/presentation/semantic-action.js +35 -21
- package/dist/extensions/agent-browser/lib/results/presentation/skills.js +109 -73
- package/dist/extensions/agent-browser/lib/results/presentation/source.js +109 -0
- package/dist/extensions/agent-browser/lib/results/presentation.js +17 -261
- package/dist/extensions/agent-browser/lib/results/recommendation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/recording.js +52 -14
- package/dist/extensions/agent-browser/lib/results/recovery-actions.js +14 -5
- package/dist/extensions/agent-browser/lib/results/recovery-next-actions.js +30 -22
- package/dist/extensions/agent-browser/lib/results/selector-recovery-target.js +83 -0
- package/dist/extensions/agent-browser/lib/results/selector-recovery.js +119 -119
- package/dist/extensions/agent-browser/lib/results/snapshot-compact.js +266 -0
- package/dist/extensions/agent-browser/lib/results/snapshot-content.js +32 -0
- package/dist/extensions/agent-browser/lib/results/snapshot-high-value-controls.js +104 -71
- package/dist/extensions/agent-browser/lib/results/snapshot-refs.js +31 -11
- package/dist/extensions/agent-browser/lib/results/snapshot-segments.js +125 -66
- package/dist/extensions/agent-browser/lib/results/snapshot-spill.js +26 -7
- package/dist/extensions/agent-browser/lib/results/snapshot.js +28 -248
- package/dist/extensions/agent-browser/lib/results/text.js +16 -5
- package/dist/extensions/agent-browser/lib/runtime-args-validation.js +133 -0
- package/dist/extensions/agent-browser/lib/runtime-compatibility.js +81 -0
- package/dist/extensions/agent-browser/lib/runtime-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/runtime-redaction.js +118 -0
- package/dist/extensions/agent-browser/lib/runtime-replay-events.js +144 -0
- package/dist/extensions/agent-browser/lib/runtime-session-identity.js +76 -0
- package/dist/extensions/agent-browser/lib/runtime-session-plan.js +114 -0
- package/dist/extensions/agent-browser/lib/runtime-session-replay.js +123 -0
- package/dist/extensions/agent-browser/lib/runtime-tab-correction.js +65 -0
- package/dist/extensions/agent-browser/lib/runtime-timeouts.js +28 -0
- package/dist/extensions/agent-browser/lib/runtime.js +137 -1041
- package/dist/extensions/agent-browser/lib/session-identity.js +51 -0
- package/dist/extensions/agent-browser/lib/session-page-observation.js +357 -0
- package/dist/extensions/agent-browser/lib/session-page-state.js +178 -333
- package/dist/extensions/agent-browser/lib/session-page-types.js +1 -0
- package/dist/extensions/agent-browser/lib/temp-artifact-budget.js +81 -0
- package/dist/extensions/agent-browser/lib/temp-mutation-queue.js +7 -0
- package/dist/extensions/agent-browser/lib/temp-root-ownership.js +234 -0
- package/dist/extensions/agent-browser/lib/temp-root.js +211 -0
- package/dist/extensions/agent-browser/lib/temp.js +33 -446
- package/dist/extensions/agent-browser/lib/tool-surface.js +94 -24
- package/dist/extensions/agent-browser/lib/upstream-version.js +3 -2
- package/dist/extensions/agent-browser/lib/web-search-providers.js +193 -0
- package/dist/extensions/agent-browser/lib/web-search-request.js +102 -0
- package/dist/extensions/agent-browser/lib/web-search-results.js +281 -0
- package/dist/extensions/agent-browser/lib/web-search-schema.js +82 -0
- package/dist/extensions/agent-browser/lib/web-search.js +124 -614
- package/dist/extensions/agent-browser/lib/webmcp-observation.js +13 -6
- package/dist/extensions/agent-browser/lib/windows-stock-launcher.js +63 -37
- package/dist/extensions/agent-browser/script-worker.js +80 -43
- package/dist/scripts/agent-browser-target.mjs +4 -2
- package/docs/ARCHITECTURE.md +19 -6
- package/docs/CODE_QUALITY.md +267 -0
- package/docs/COMMAND_REFERENCE.md +272 -241
- package/docs/ELECTRON.md +77 -46
- package/docs/REFERENCE.md +869 -0
- package/docs/RELEASE.md +25 -22
- package/docs/REQUIREMENTS.md +2 -1
- package/docs/SUPPORT_MATRIX.md +81 -79
- package/docs/TOOL_CONTRACT.md +158 -125
- package/docs/platform-smoke.md +8 -8
- package/package.json +144 -120
- package/scripts/agent-browser-capability-baseline.mjs +361 -957
- package/scripts/agent-browser-target.mjs +6 -2
- package/scripts/build.mjs +37 -8
- package/scripts/capability-core-commands.mjs +193 -0
- package/scripts/capability-options.mjs +283 -0
- package/scripts/capability-samples.mjs +133 -0
- package/scripts/capability-section.mjs +11 -0
- package/scripts/config.mjs +235 -132
- package/scripts/convert-browser-session.mjs +30 -9
- package/scripts/doctor-sources.mjs +195 -0
- package/scripts/doctor.mjs +94 -249
- package/scripts/platform-smoke/artifacts.mjs +38 -11
- package/scripts/platform-smoke/commands.mjs +104 -0
- package/scripts/platform-smoke/crabbox-runner.mjs +167 -80
- package/scripts/platform-smoke/doctor-support.mjs +157 -0
- package/scripts/platform-smoke/doctor-windows.mjs +230 -0
- package/scripts/platform-smoke/doctor.mjs +212 -320
- package/scripts/platform-smoke/lease-evidence.mjs +132 -0
- package/scripts/platform-smoke/linux-image/Dockerfile +8 -2
- package/scripts/platform-smoke/suite-checks.mjs +98 -0
- package/scripts/platform-smoke/suite-evidence.mjs +281 -0
- package/scripts/platform-smoke/targets.mjs +154 -472
- package/scripts/platform-smoke.mjs +55 -40
- package/scripts/prepare.mjs +16 -7
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# Agent Browser command reference
|
|
2
2
|
|
|
3
3
|
Related docs:
|
|
4
|
+
|
|
4
5
|
- [`../README.md`](../README.md)
|
|
5
6
|
- [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md)
|
|
6
7
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md)
|
|
@@ -26,6 +27,7 @@ SDK hosts can supply an awaited [`beforeExecute` callback](TOOL_CONTRACT.md#host
|
|
|
26
27
|
|
|
27
28
|
<!-- agent-browser-capability-baseline:start upstream-baseline -->
|
|
28
29
|
<!-- Generated from scripts/agent-browser-capability-baseline.mjs. Run `npm run docs -- command-reference write` to update. Do not edit manually. -->
|
|
30
|
+
|
|
29
31
|
This reference is baselined to the locally installed `agent-browser 0.38.1` command/help surface, audited against vercel-labs/agent-browser@aff6125c023b810ea3f2e5deec5379e9a4270bdc. Upstream `agent-browser` remains the source of truth for command semantics; this file is the local fallback for Pi agent sessions where direct binary help is blocked or discouraged.
|
|
30
32
|
|
|
31
33
|
The lightweight drift check is `npm run verify -- command-reference`. Run it whenever the installed upstream `agent-browser` version changes or this reference is edited.
|
|
@@ -202,7 +204,7 @@ Use `agent_browser` for one native command and `batch --bail` with JSON-array st
|
|
|
202
204
|
{ "args": ["batch", "--bail"], "stdin": "[[\"get\",\"title\"],[\"get\",\"url\"]]" }
|
|
203
205
|
```
|
|
204
206
|
|
|
205
|
-
Ordinary calls share the root Pi parent/subagent browser unless native config, an explicit identity, or an active managed/fresh browser selects otherwise. `sessionMode: "fresh"` intentionally rotates the managed browser; explicit native session choice still wins. Keep profile/provider/attachment launch flags on first launch. After a failed fresh call, inspect `managedSessionOutcome` rather than guessing which browser is current.
|
|
207
|
+
Ordinary calls share the root Pi parent/subagent browser unless native config, an explicit identity, or an active managed/fresh browser selects otherwise. `sessionMode: "fresh"` intentionally rotates the managed browser; explicit native session choice still wins. Keep profile/provider/attachment launch flags on first launch. Native launch environment/config can submit the full launch configuration again on each invocation: mixing an executable default with first-call-only profile/user-agent flags can replace the browser when those flags disappear. Use a consistent native launch configuration; the wrapper passes it through unchanged and omits automatic Chrome startup args when native environment/config already supplies local launch defaults. After a failed fresh call, inspect `managedSessionOutcome` rather than guessing which browser is current.
|
|
206
208
|
|
|
207
209
|
A cooperating-process lock covers each direct helper/action sequence or whole code cell, keyed by actual socket context, canonical namespace, and session. Independent identities remain concurrent; direct namespace-wide close barriers matching work. Restart all participants onto the new version. This is not rollback or protection against human/external-client navigation, and separate tool calls still need coordination.
|
|
208
210
|
|
|
@@ -214,7 +216,7 @@ Call `agent_browser_code` with `{ code, session?, namespace?, timeoutMs?, output
|
|
|
214
216
|
|
|
215
217
|
```json
|
|
216
218
|
{
|
|
217
|
-
|
|
219
|
+
"code": "const titles = []; for (const url of ['https://example.com', 'https://example.org']) { const opened = await browser({ args: ['open', url] }); if (!opened.success) throw new Error(opened.error); const title = await browser({ args: ['get', 'title'] }); if (!title.success) throw new Error(title.error); titles.push({ url, title: title.data.title ?? title.data.result }); } emit(titles);"
|
|
218
220
|
}
|
|
219
221
|
```
|
|
220
222
|
|
|
@@ -236,13 +238,13 @@ On hosts supporting instruction groups, first enable `browser` with `discover_to
|
|
|
236
238
|
|
|
237
239
|
Activation is additive and uses native Pi selected-tool history. Omit CLI `--tools` for normal lazy activation, or include desired advanced tools in that explicit selection; excluded tools remain unavailable. The tools take flat input:
|
|
238
240
|
|
|
239
|
-
| Tool
|
|
240
|
-
|
|
|
241
|
-
| `agent_browser_action`
|
|
242
|
-
| `agent_browser_qa`
|
|
243
|
-
| `agent_browser_electron`
|
|
244
|
-
| `agent_browser_source`
|
|
245
|
-
| `agent_browser_network_source` | `{ "requestId": "req-1", "url": "/api/fail" }`
|
|
241
|
+
| Tool | Example |
|
|
242
|
+
| ------------------------------ | ------------------------------------------------------------------------------------------------------- |
|
|
243
|
+
| `agent_browser_action` | `{ "action": "select", "locator": "role", "role": "combobox", "name": "Flavor", "value": "chocolate" }` |
|
|
244
|
+
| `agent_browser_qa` | `{ "url": "https://example.com", "expectedText": "Learn more" }` |
|
|
245
|
+
| `agent_browser_electron` | `{ "action": "list", "query": "code" }` |
|
|
246
|
+
| `agent_browser_source` | `{ "selector": "#save", "componentName": "SaveButton" }` |
|
|
247
|
+
| `agent_browser_network_source` | `{ "requestId": "req-1", "url": "/api/fail" }` |
|
|
246
248
|
|
|
247
249
|
Native commands remain in `agent_browser`; advanced objects are neither nested batch rows nor nested fields on that tool. If a recovery action names an inactive advanced tool, enable it before calling its exact flat payload.
|
|
248
250
|
|
|
@@ -346,7 +348,7 @@ For desktop, contenteditable, or host-controlled rich inputs, treat a semantic `
|
|
|
346
348
|
|
|
347
349
|
Do not assume Playwright selector dialects such as `text=Close` or `button:has-text('Close')` are supported wrapper syntax. In particular, current upstream can report successful `scrollintoview text=...` without moving the page, so the wrapper rejects that form before dispatch—directly or in an effective raw/stdin batch row—and shows executable `find text <label> hover` plus snapshot/ref recovery payloads in visible failure text and `details.nextActions`. `scrollintoview ... --help` and `-h` remain native help calls. Use `scrollintoview` with CSS, `xpath=...`, or a current `@e…` ref; use `find` for semantic text targets.
|
|
348
350
|
|
|
349
|
-
Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch`
|
|
351
|
+
Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside effective raw-argument or stdin `batch` steps, the wrapper also walks the full plan in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. This scan continues after a snapshot even when no refs appear before it: `snapshot -i`, a non-form `click @e1`, then `get value @e2` is rejected. Put another `snapshot -i` after the click or split the batch and inspect before continuing. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `agent_browser_action` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
|
|
350
352
|
|
|
351
353
|
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. Probe listeners and the window marker self-expire after the call's process-timeout budget plus the existing two-second cleanup grace if a pending decision blocks normal cleanup; collection cancels the timer. This uses no extra browser command while pending. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
|
|
352
354
|
|
|
@@ -368,7 +370,10 @@ Use `read [url]` for documentation and other unstructured text. `read <url> --ra
|
|
|
368
370
|
When you already know several visible refs or selectors, extract them in one `batch` call instead of many serial getter calls. When a prior snapshot and session are available and the same-page freshness checks apply, ref-consuming calls add one extra `snapshot -i` preflight per top-level call or batch. Batching shares that probe across rows; it does not remove it:
|
|
369
371
|
|
|
370
372
|
```json
|
|
371
|
-
{
|
|
373
|
+
{
|
|
374
|
+
"args": ["batch"],
|
|
375
|
+
"stdin": "[[\"get\",\"text\",\"@e64\"],[\"get\",\"text\",\"@e65\"],[\"get\",\"text\",\"@e66\"]]"
|
|
376
|
+
}
|
|
372
377
|
```
|
|
373
378
|
|
|
374
379
|
Prefer `get` and scoped `eval --stdin` for read-only extraction. Getter names are grouped under `get`: use `get title`, `get url`, or `get text <selector>`, not shortcut commands such as `title` or `url`. When upstream reports an unknown command, unknown subcommand, or unrecognized command for a single-token shortcut (`attr`, `count`, `html`, `text`, `title`, `url`, or `value`), the wrapper adds a visible grouped-`get` hint; only `title` and `url` also get exact read-only `details.nextActions` (`use-get-title` / `use-get-url`, with `--session` preserved when the failed call named a session). If another `Agent-browser hint:` (selector dialect or stale-ref recovery) was already appended to the same error text, the getter hint is omitted.
|
|
@@ -389,27 +394,34 @@ For fixed flows, keep native batch rows short around navigation and rerender bou
|
|
|
389
394
|
|
|
390
395
|
```json
|
|
391
396
|
{
|
|
392
|
-
|
|
393
|
-
|
|
397
|
+
"args": ["batch", "--bail"],
|
|
398
|
+
"stdin": "[[\"open\",\"https://shop.example/checkout\"],[\"fill\",\"#email\",\"user@example.com\"],[\"click\",\"#continue\"],[\"wait\",\"--url\",\"**/shipping\"],[\"wait\",\"--text\",\"Shipping address\"],[\"screenshot\",\"/tmp/shipping.png\"]]"
|
|
394
399
|
}
|
|
395
400
|
```
|
|
396
401
|
|
|
397
402
|
Paced input uses explicit native rows, preserving the former job's behavior without assuming `--delay` equivalence:
|
|
398
403
|
|
|
399
404
|
```json
|
|
400
|
-
{
|
|
405
|
+
{
|
|
406
|
+
"args": ["batch", "--bail"],
|
|
407
|
+
"stdin": "[[\"focus\",\"#prompt\"],[\"keyboard\",\"type\",\"h\"],[\"wait\",\"20\"],[\"keyboard\",\"type\",\"i\"]]"
|
|
408
|
+
}
|
|
401
409
|
```
|
|
402
410
|
|
|
403
411
|
For longer text, code can construct the rows and call native batch; keep the former 200-character pacing bound when porting a job. Add a final `press` only when submission is intended. Fixed waits are not postconditions. Timeout partial progress remains available through the same batch path. See [all job step equivalents](TOOL_CONTRACT.md#07-migration).
|
|
404
412
|
|
|
405
|
-
For quick smoke/QA checks, enable `qa` and call `agent_browser_qa` with flat fields. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
|
|
413
|
+
For quick smoke/QA checks, enable `qa` and call `agent_browser_qa` with flat fields. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. Text and control values under `display: none` or `opacity: 0` ancestors, or whose computed visibility is hidden, are excluded even when the control itself has a visible layout box; `display: contents` wrappers and `visibility: visible` overrides still count. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
|
|
406
414
|
|
|
407
415
|
For failed QA, inspect `details.qaPreset.failedChecks` separately from `notRunChecks`: a reported fail-fast error leaves later checks not run, rather than proving their assertions failed. Failure prose leads with the redacted cause and reports received execution coverage; missing timeout receipts mean an unknown executed count. Incomplete receipts without a reported failure are unverified, never proof of a pass or of unreached checks.
|
|
408
416
|
|
|
409
417
|
The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `agent_browser_network_source` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. Filtered data receives the same credential redaction as ordinary network diagnostics before entering `details.data` or an `outputPath` export. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
|
|
410
418
|
|
|
411
419
|
```json
|
|
412
|
-
{
|
|
420
|
+
{
|
|
421
|
+
"url": "https://example.com",
|
|
422
|
+
"expectedText": "Learn more",
|
|
423
|
+
"screenshotPath": ".dogfood/qa-example.png"
|
|
424
|
+
}
|
|
413
425
|
```
|
|
414
426
|
|
|
415
427
|
Optional `loadState`, `checkNetwork`, `checkConsole`, and `checkErrors` default to `"domcontentloaded"`, `true`, `true`, and `true` for URL-opening QA; set a check to `false` to skip that diagnostic. For `qa.attached`, the diagnostic checks default to `false` because upstream buffers may predate the current check; opt in with `checkNetwork`, `checkConsole`, or `checkErrors` when preserved-buffer failures are desired. Omit `expectedText` and `expectedSelector` when you only need load plus diagnostics.
|
|
@@ -469,7 +481,7 @@ For local app debugging, enable `source`; `agent_browser_source` can gather cand
|
|
|
469
481
|
{ "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" }
|
|
470
482
|
```
|
|
471
483
|
|
|
472
|
-
Enable `network` and call `agent_browser_network_source` for failed browser requests. `requestId` adds `network request <requestId>`; `filter` or `url` adds `network requests --filter …`, with explicit filter winning. Optional `namespace` and `session` select the native target; `namespace: ""` overrides an ambient namespace. HTTP status ≥ 400, `failed: true`, and error fields identify failed rows.
|
|
484
|
+
Enable `network` and call `agent_browser_network_source` for failed browser requests. `requestId` adds `network request <requestId>`; `filter` or `url` adds `network requests --filter …`, with explicit filter winning. Optional `namespace` and `session` select the native target; `namespace: ""` overrides an ambient namespace. HTTP status ≥ 400, `failed: true`, and error fields identify failed rows. The same explicit-filter-wins query limits failed-request and initiator analysis by substring; unrelated failed-request initiators are ignored. Repeated request IDs count once, keeping the first matching failed row; anonymous rows and different IDs sharing one URL remain distinct. Workspace URL-literal search uses the captured execution directory and the source tool's bounds: 2,000 files by default, 5,000 maximum, eight needles and ten matches. Results retain `details.networkSourceLookup` with status, failed requests, candidate evidence, and limitations. They never assign definitive blame or replace the active page target and same-page refs.
|
|
473
485
|
|
|
474
486
|
```json
|
|
475
487
|
{ "requestId": "req-1", "url": "/api/fail" }
|
|
@@ -497,7 +509,10 @@ Use `wait --download [path]` after an earlier action has already started a brows
|
|
|
497
509
|
For one-call flows, put the click and wait in `batch`; the wait step keeps the saved-file metadata in `details.batchSteps[n].savedFilePath` and `details.batchSteps[n].savedFile`:
|
|
498
510
|
|
|
499
511
|
```json
|
|
500
|
-
{
|
|
512
|
+
{
|
|
513
|
+
"args": ["batch"],
|
|
514
|
+
"stdin": "[[\"click\",\"@e5\"],[\"wait\",\"--download\",\"/tmp/report.csv\"]]"
|
|
515
|
+
}
|
|
501
516
|
```
|
|
502
517
|
|
|
503
518
|
A successful wait-based download renders a readable summary such as `Download completed: /tmp/report.csv` and exposes top-level `details.savedFilePath` plus `details.savedFile` for non-batch calls. With current upstream `agent-browser`, `wait --download <path>` may report the requested path before this environment can verify that the file was persisted there. Treat `details.savedFilePath` as upstream-reported metadata unless `details.artifacts[].exists` is true. Upstream tracking: [vercel-labs/agent-browser#1300](https://github.com/vercel-labs/agent-browser/issues/1300).
|
|
@@ -521,6 +536,7 @@ Screenshot `imageObservations` includes verified pixel dimensions and capture ki
|
|
|
521
536
|
For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
|
|
522
537
|
|
|
523
538
|
Wrapper result rendering is metadata-first for saved files. Image MIME types come from a bounded header read for PNG, JPEG, GIF and WebP, never from a filename suffix; missing, unreadable, unknown or truncated headers omit `mediaType`. This identifies a format, not full image validity. Inline screenshots use the same byte check and existing size limit, so a PNG saved as `.webm` still attaches as `image/png`; other artifact kinds are not auto-inlined. An artifact-producing command fails as `artifact-missing` with artifact `status: "stale"` when the reported path's `mtimeMs` falls outside the command's bounded start/end window (with two seconds of filesystem precision tolerance), including a previous recording that `record restart` claims to finalize; clearly old or future-dated evidence is never accepted as a fresh capture. A batch, whether supplied through stdin arrays or argument command strings, must use distinct explicit artifact destinations; preflight canonicalizes existing path ancestry, compares existing file identities to catch hardlinks, and applies full Unicode plus platform case folding on macOS/Windows so aliases cannot satisfy another step's verification. The same preflight prevents `outputPath` from aliasing a same-call browser artifact, follows upstream's forward option consumption and final effective `-o` / `--output` for `diff screenshot`, and treats the optional path on `network har stop` as an artifact destination; upstream ignores positional paths on `network har start`. Outer CLI artifact parsing removes upstream global flags, so direct forms such as `record --json start <path>` and `pdf --quiet <path>` retain their native destinations. Native batch rows do not run that cleanup: `pdf --quick ignored.pdf` writes to the literal path `--quick`, and `download #link --quiet ignored.bin` writes to `--quiet`. Preflight, directory preparation, presentation, and timeout evidence use those same operands, not the ignored trailing tokens. Screenshot destination parsing mirrors upstream's exact flag matching and `[selector] [path]` positional order: `--` is positional, `true` / `false` after screenshot-only `--full` / `-f` remain positional, extra positionals are ignored after the path slot, selector-prefixed (`.`, `#`, `@`) or uppercase-extension single arguments remain selectors, and lowercase image extensions or slash-bearing arguments are paths. The wrapper deliberately keeps its existing slash-bearing hidden-workspace path normalization (for example `.dogfood/run/foo.png`) before launch. `wait --download` is observational and may verify a download that completed just before the wait began, so it is exempt from the command-window mtime gate; an explicit wait destination, in long `--download <path>` or short `-d <path>` form (including `wait --download --timeout 30000 capture.csv`), still participates in active-recording reservation preflight; the path is the next retained operand after the first timeout pair is removed; unsupported `--download=<path>` fails with split-argument guidance:
|
|
539
|
+
|
|
524
540
|
- screenshots return a saved-path summary, visible artifact metadata, structured `details.artifacts` metadata, and an inline image attachment when safe; the visible block includes artifact type, requested path, absolute path, existence, size, cwd, session, and repair/copy status when applicable
|
|
525
541
|
- downloads, PDFs, `wait --download` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed video recordings from `record stop`, and path-bearing HAR captures return concise saved-path summaries plus structured `details.artifacts` metadata without inlining large files
|
|
526
542
|
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification; without that receipt, only an acknowledged native `restarted: true` retires the previous take, whose file remains unverified, not saved; rejected or pending restarts keep the active take's reservation and stop action; a missing or stale prior file fails as `artifact-missing` while the new recording remains visible as pending and the prior manifest row is retired. Within one Pi extension process, an unbounded transcript-backed index reserves active recording destinations independently of the bounded artifact manifest. Artifact lifecycle calls and result `outputPath` writes serialize around that global check; reservations use canonical namespace/session identity, survive manifest eviction and branch replay, and retire after direct, ordered nested-batch, fresh-replacement, code, Electron, or shutdown close; the newest pending row per identity is authoritative. Legacy batch replay retires a pending manifest only when the ordered close lifecycle leaves recording closed; a later successful browser reactivation plus `record start` keeps the new pending reservation. Lexical, hardlink, existing/dangling symlink, full Unicode-fold, and macOS/Windows case aliases are rejected, so `record restart` must use a distinct new path. Do not place `record start` or `record restart` after `close` / `quit` / `exit` in one batch: wrapper preflight rejects it because upstream can report success without starting a recording; split the close and recording into separate calls. A `No recording in progress` stop failure checks the matching native receipt once and preserves checked file metadata instead of assuming the path is missing; a later successful batch recording row opens its new pending path normally. Recovery offers an exact status query, and a stop only when the matching take is still current and pending. The target remains unverified until recording stops. Native 0.37 checks `ffmpeg` before starting; older supported natives may defer failure. If a successful start/restart reports pending output without `ffmpeg`, the wrapper appends `Recording dependency warning: ffmpeg not found on PATH` and `details.recordingDependencyWarning`; stop, check the result, then install the dependency before starting a new recording.
|
|
@@ -528,7 +544,7 @@ Wrapper result rendering is metadata-first for saved files. Image MIME types com
|
|
|
528
544
|
|
|
529
545
|
`diff screenshot` follows the file-artifact path above for the **diff** image: model-visible text and `details.artifacts` focus on that output, while baseline paths stay out of the artifact summary block, and Pi does **not** auto-inline the diff the way it inlines trusted `screenshot` captures. `state load` may print the loaded path in prose but does not add a saved-file artifact entry the way `state save` does.
|
|
530
546
|
|
|
531
|
-
For screenshot paths under dot-directories such as `.dogfood/run/foo.png`, the wrapper normalizes the requested path to an absolute path before invoking upstream `agent-browser`, verifies the requested file exists, and repairs from an upstream temp screenshot when possible. For direct artifact commands and batch artifact steps (`download`, `pdf`, `screenshot`, `state save`, and `wait --download`), the wrapper creates missing parent directories before launch. A parent-directory failure returns `validation-error` with the attempted directory and `verify-artifact-path` guidance before the browser command runs. Prefer **absolute paths in raw batch artifact rows** because the daemon's working directory may differ from Pi's. When `pi-change-working-dir` selects a different execution root, the wrapper binds relative file operands in direct, stdin, and raw batch rows to that invocation's root; upstream's raw-argument precedence over stdin is unchanged. Otherwise raw strings stay unchanged and screenshot path normalization applies only to direct calls and stdin rows. See [working-directory changes](
|
|
547
|
+
For screenshot paths under dot-directories such as `.dogfood/run/foo.png`, the wrapper normalizes the requested path to an absolute path before invoking upstream `agent-browser`, verifies the requested file exists, and repairs from an upstream temp screenshot when possible. For direct artifact commands and batch artifact steps (`download`, `pdf`, `screenshot`, `state save`, and `wait --download`), the wrapper creates missing parent directories before launch. A parent-directory failure returns `validation-error` with the attempted directory and `verify-artifact-path` guidance before the browser command runs. Prefer **absolute paths in raw batch artifact rows** because the daemon's working directory may differ from Pi's. When `pi-change-working-dir` selects a different execution root, the wrapper binds relative file operands in direct, stdin, and raw batch rows to that invocation's root; upstream's raw-argument precedence over stdin is unchanged. Otherwise raw strings stay unchanged and screenshot path normalization applies only to direct calls and stdin rows. See [working-directory changes](REFERENCE.md#working-directory-changes). Known caller paths appear as `Requested path`; `Absolute path` is the resolved location checked on disk, and `Reported path` exposes a differing screenshot report (including a canonical `/private/tmp` alias) through the existing `tempPath` metadata. No extra canonicalization rewrites upstream arguments.
|
|
532
548
|
|
|
533
549
|
For annotated screenshots in `batch`, put `--annotate` in top-level args instead of inside the screenshot step:
|
|
534
550
|
|
|
@@ -563,15 +579,20 @@ Oversized snapshots and oversized generic outputs are different: when a persiste
|
|
|
563
579
|
|
|
564
580
|
```json
|
|
565
581
|
{
|
|
566
|
-
|
|
567
|
-
|
|
582
|
+
"args": ["--profile", "Profile 1", "open", "https://mail.google.com"],
|
|
583
|
+
"sessionMode": "fresh"
|
|
568
584
|
}
|
|
569
585
|
```
|
|
570
586
|
|
|
571
587
|
```json
|
|
572
588
|
{
|
|
573
|
-
|
|
574
|
-
|
|
589
|
+
"args": [
|
|
590
|
+
"--executable-path",
|
|
591
|
+
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser",
|
|
592
|
+
"open",
|
|
593
|
+
"https://mail.google.com"
|
|
594
|
+
],
|
|
595
|
+
"sessionMode": "fresh"
|
|
575
596
|
}
|
|
576
597
|
```
|
|
577
598
|
|
|
@@ -638,106 +659,103 @@ Operational notes:
|
|
|
638
659
|
|
|
639
660
|
## Full supported surface
|
|
640
661
|
|
|
641
|
-
The
|
|
662
|
+
The command groups below intentionally list more than the recommended workflow. Rare commands are included so agents can discover that the installed upstream supports them without direct `agent-browser --help` access.
|
|
642
663
|
|
|
643
664
|
### Built-in skills
|
|
644
665
|
|
|
645
|
-
Native-tool note: upstream skills are written for the standalone `agent-browser` CLI and may show bash/heredoc examples. In pi, convert those examples to `agent_browser` calls: pass CLI tokens in `args`, and pass heredoc/stdin bodies through the tool `stdin` field for `batch`, `eval --stdin`, or `auth save --password-stdin`.
|
|
666
|
+
Native-tool note: upstream skills are written for the standalone `agent-browser` CLI and may show bash/heredoc examples. In pi, convert those examples to `agent_browser` calls: pass CLI tokens in `args`, and pass heredoc/stdin bodies through the tool `stdin` field for `batch`, `eval --stdin`, or `auth save --password-stdin`. The wrapper's copyable skill calls preserve quoted empty and whitespace-only operands: `fill '#name' ""` becomes `args: ["fill", "#name", ""]`, not a missing value.
|
|
646
667
|
|
|
647
668
|
Session note: `skills list`, `skills get …`, and `skills path …` are **stateless** in this wrapper. Even with default `sessionMode: "auto"`, the extension does not prepend the implicit managed `--session` for those commands, so reading bundled skills does not attach to or rotate the active browser session (same intent as plain-text `--help` / `--version` inspection). Other `skills` subcommands follow normal session rules until explicitly allowlisted in `extensions/agent-browser/lib/runtime.ts` alongside regression coverage in `test/agent-browser.runtime.test.ts`.
|
|
648
669
|
|
|
649
|
-
| Command
|
|
650
|
-
|
|
|
651
|
-
| `skills list`
|
|
652
|
-
| `skills get core`
|
|
653
|
-
| `skills get core --full`
|
|
654
|
-
| `skills get <name>`
|
|
655
|
-
| `skills get <name> --full` | Include a skill's supplementary references/templates when present.
|
|
656
|
-
| `skills get --all`
|
|
657
|
-
| `skills path [name]`
|
|
670
|
+
| Command | Purpose |
|
|
671
|
+
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
672
|
+
| `skills list` | List available CLI-bundled skills. |
|
|
673
|
+
| `skills get core` | Print the core usage guide. |
|
|
674
|
+
| `skills get core --full` | Print the full version-matched core command reference and templates. |
|
|
675
|
+
| `skills get <name>` | Load a specialized skill such as `electron` or `slack`. Common specialized calls include `skills get electron`, `skills get slack`, `skills get dogfood`, `skills get vercel-sandbox`, `skills get agentcore`, `skills get derive-client` (HAR-to-API-client workflow), and `skills get webmcp-gen` (create and validate page tools). |
|
|
676
|
+
| `skills get <name> --full` | Include a skill's supplementary references/templates when present. |
|
|
677
|
+
| `skills get --all` | Print all visible bundled skills for broad audit/debug work. |
|
|
678
|
+
| `skills path [name]` | Print a skill directory path. |
|
|
658
679
|
|
|
659
680
|
Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an override for bundled skill discovery. Normal agents should not need it, but it is useful when validating package layout or upstream skill packaging.
|
|
660
681
|
|
|
661
682
|
### Core page and element commands
|
|
662
683
|
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
|
|
684
|
+
- `open [url]` — Navigate when given a URL. URL-less `open` runs native `get url`, launching if needed and otherwise retaining the current page. Use explicit `about:blank` when navigation to blank is intended.
|
|
685
|
+
- `open <url>` — Navigate to a URL; `goto <url>` and `navigate <url>` are equivalent navigation aliases when a URL is present.
|
|
686
|
+
- `read [url]` — Fetch agent-readable text from an explicit URL without requiring a Chrome page, or omit the URL to read rendered active-tab DOM. Supports `--raw`, `--require-md`, `--llms <index|full>`, `--outline`, `--filter <text>`, and `--timeout <ms>`.
|
|
687
|
+
- `click <sel>` — Click an element or `@ref`.
|
|
688
|
+
- `click <sel> --new-tab` — Click a link/control while requesting a new tab.
|
|
689
|
+
- `dblclick <sel>` — Double-click an element.
|
|
690
|
+
- `type <sel> <text>` — Type into an element; both selector and text are required. For the focused element without a selector, use `keyboard type <text>`.
|
|
691
|
+
- `fill <sel> <text>` — Clear and fill an element.
|
|
692
|
+
- `press <key>` — Press a key such as `Enter`, `Tab`, or `Control+a`. `key <key>` is the upstream alias.
|
|
693
|
+
- `key <key>` — Alias for `press <key>`.
|
|
694
|
+
- `keydown <key>` — Hold a key down without releasing it, useful for modifiers.
|
|
695
|
+
- `keyup <key>` — Release a key previously held by `keydown <key>`. Common modifier examples are `keydown Shift` and `keyup Shift`.
|
|
696
|
+
- `keyboard type <text>` — Type text with real keystrokes and no selector.
|
|
697
|
+
- `keyboard inserttext <text>` — Insert text without key events.
|
|
698
|
+
- `hover <sel>` — Hover an element.
|
|
699
|
+
- `focus <sel>` — Focus an element.
|
|
700
|
+
- `check <sel>` — Check a checkbox.
|
|
701
|
+
- `uncheck <sel>` — Uncheck a checkbox.
|
|
702
|
+
- `select <sel> <val...>` — Select one or more dropdown options.
|
|
703
|
+
- `drag <src> <dst>` — Drag and drop.
|
|
704
|
+
- `upload <sel> <files...>` — Upload one or more files.
|
|
705
|
+
- `download <sel> <path>` — Download a file by clicking an element.
|
|
706
|
+
- `scroll <dir> [px]` — Scroll `up`, `down`, `left`, or `right`.
|
|
707
|
+
- `scroll <dir> [px] --selector <sel>` — Scroll a specific scrollable element/container instead of the page.
|
|
708
|
+
- `scrollintoview <sel>` — Scroll an element into view; `scrollinto <sel>` is the upstream alias.
|
|
709
|
+
- `scrollinto <sel>` — Alias for `scrollintoview <sel>`.
|
|
710
|
+
- `wait <sel|ms>` — Wait for an element or a duration.
|
|
711
|
+
- `screenshot [selector] [path]` — Take a full-page or element-scoped screenshot; a single selector-like argument scopes, while a path-like argument saves to that path.
|
|
712
|
+
- `screenshot [path]` — Take a screenshot and optionally save it to a path.
|
|
713
|
+
- `pdf <path>` — Save the page as a PDF.
|
|
714
|
+
- `snapshot` — Print an accessibility tree with refs for AI interaction. Common options include `snapshot --interactive`, `snapshot --urls`, `snapshot --compact`, `snapshot --depth <n>`, `snapshot --selector <sel>`, and native `snapshot --delta` / `snapshot --delta --full`. Cursor-interactive elements are included by default; `snapshot --cursor` / `snapshot -C` are deprecated no-ops.
|
|
715
|
+
- `eval <js>` — Run JavaScript. Use `eval --stdin` through this wrapper for larger snippets, or `eval -b <base64>` for shell-escaping-safe one-liners.
|
|
716
|
+
- `connect <port|url>` — Connect to a browser through CDP.
|
|
717
|
+
- `close [--all]` — Close the current browser or all sessions; `quit` and `exit` are upstream close aliases.
|
|
718
|
+
- `tap <selector>` — Touch-oriented tap alias for iOS/provider workflows.
|
|
719
|
+
- `swipe <direction> [distance]` — Touch-oriented swipe for iOS/provider workflows.
|
|
720
|
+
|
|
721
|
+
On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. These shims intercept only complete recognized shapes: `scroll to end|top` has no extra operands, and CSS-container scroll accepts only the selector, direction, and optional amount. Surplus operands such as `scroll to end --selector #panel` or `scroll #panel down 500 extra` pass through to native agent-browser unchanged rather than being silently dropped. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
|
|
703
722
|
|
|
704
723
|
Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, flat `agent_browser_action` select fields or active-session semantic role/name/label select instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `agent_browser_action` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `agent_browser_action` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
|
|
705
724
|
|
|
706
725
|
### Navigation
|
|
707
726
|
|
|
708
|
-
| Command
|
|
709
|
-
|
|
|
710
|
-
| `back`
|
|
711
|
-
| `forward` | Go forward.
|
|
712
|
-
| `reload`
|
|
727
|
+
| Command | Purpose |
|
|
728
|
+
| --------- | ------------------------ |
|
|
729
|
+
| `back` | Go back. |
|
|
730
|
+
| `forward` | Go forward. |
|
|
731
|
+
| `reload` | Reload the current page. |
|
|
713
732
|
|
|
714
733
|
### Session, state, frames, dialogs, windows, and inspection commands
|
|
715
734
|
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
720
|
-
|
|
721
|
-
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
| `close` | Close the current browser session. |
|
|
735
|
-
| `close --all` | Close every session. |
|
|
735
|
+
- `session` — Show current session name.
|
|
736
|
+
- `session list` — List active sessions.
|
|
737
|
+
- `state save <path>` — Save cookies, local storage, and session storage to a state file.
|
|
738
|
+
- `state load <path>` — Load cookies and storage from a state file.
|
|
739
|
+
- `state list` — List saved state files.
|
|
740
|
+
- `state show <filename>` — Show saved-state metadata without dumping cookie or storage values.
|
|
741
|
+
- `state rename <old-name> <new-name>` — Rename a saved state file.
|
|
742
|
+
- `state clear [session-name] [--all]` — Clear saved states for one name or all names; `state clear -a` is the upstream short alias for clearing all names.
|
|
743
|
+
- `session id --scope worktree --prefix <name>` — Generate a stable session id for agent/worktree-scoped browser state.
|
|
744
|
+
- `session info --json` — One read-only preflight: daemon activity/PID versus native browser liveness, Chrome PID, exact profile, tabs and launched/attached ownership, plus separate Pi cleanup ownership. Missing native fields remain unknown; no browser launch or tab changes.
|
|
745
|
+
- `state clean --older-than <days>` — Delete expired saved-state files.
|
|
746
|
+
- `frame <selector|main>` — Switch iframe context by selector/ref/name/URL, or return to the main frame.
|
|
747
|
+
- `dialog accept [text]` — Accept an alert, confirm, or prompt dialog, optionally supplying prompt text.
|
|
748
|
+
- `dialog dismiss` — Dismiss or cancel the current dialog.
|
|
749
|
+
- `dialog status` — Check whether a dialog is pending.
|
|
750
|
+
- `window new` — Open and activate a new blank window. The wrapper keeps that intentional `about:blank` target rather than selecting the old tab. Old refs are invalid; take a fresh snapshot before further interaction.
|
|
751
|
+
- `close` — Close the current browser session.
|
|
752
|
+
- `close --all` — Close every session.
|
|
736
753
|
|
|
737
754
|
A canceled cold-session reopen returns an aborted result with its exact session identity and the consumed reopen marker once the CLI starts. Reload does not repeat that navigation. Cancellation before the attempt leaves the remembered URL pending for the next current-page operation.
|
|
738
755
|
|
|
739
756
|
<!-- agent-browser-playbook:start inspection -->
|
|
740
757
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
758
|
+
|
|
741
759
|
Native inspection calls use the `agent_browser` tool shape, not shell-like direct-binary commands:
|
|
742
760
|
|
|
743
761
|
- { "args": ["--help"] }
|
|
@@ -748,18 +766,16 @@ These calls return plain text and stay stateless: the extension does not inject
|
|
|
748
766
|
|
|
749
767
|
### Page state, finding, mouse, settings, network, and storage
|
|
750
768
|
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
| `cookies [get|set|clear]` | Manage cookies. Full set form: `cookies set <name> <value> --url <url> --domain <domain> --path <path> --httpOnly --secure --sameSite <Strict|Lax|None> --expires <timestamp>`; also supports `cookies set --curl <file>` for JSON, cURL, or bare Cookie-header bulk imports. |
|
|
762
|
-
| `storage <local|session>` | Manage web storage. |
|
|
769
|
+
- `get title`, `get url`, `get cdp-url` — Read page/browser metadata without a selector. Upstream root help summarizes this family as `get <what> [selector]`, but the selector is not optional for DOM getters.
|
|
770
|
+
- `get text/html/value/count <selector>` — Read matched elements; use `get text body` for whole-page text.
|
|
771
|
+
- `get attr <selector> <name>`, `get box <selector>`, `get styles <selector>` — Read an attribute, bounding box, or computed styles from matched elements.
|
|
772
|
+
- `is <what> <selector>` — Check `visible`, `enabled`, or `checked`.
|
|
773
|
+
- `find <locator> <value> <action> [text]` — Locator types include `role`, `text`, `label`, `placeholder`, `alt`, `title`, and `testid`; selector helpers include `find first <sel>`, `find last <sel>`, and `find nth <n> <sel>`. Role/text filters include `find role <role> --name <name>` and `find ... --exact`. Actions are `click, fill, check, hover, text` only. Prefer `find role` for semantic elements: implicit roles work (`find role heading text --name` for `<h2>`, list/banner landmarks, and similar). Default name matching is a case-insensitive substring; `--exact` makes the accessible name case-sensitive. On misses, upstream 0.32.4+ keeps locator detail such as `Names seen: …` or `No element found: getByRole(...)` instead of a generic flatten.
|
|
774
|
+
- `mouse <action> [args]` — `move <x> <y>`, `down [btn]`, `up [btn]`, `wheel <dy> [dx]`.
|
|
775
|
+
- `set <setting> [value]` — `viewport <w> <h>`, `device <name>`, `geo <lat> <lng>`, `offline [on|off]`, `headers <json>`, `credentials <user> <pass>`, and `set media <features>` (`dark`, `light`, and/or `reduced-motion`).
|
|
776
|
+
- `network <action>` — `network route <url> [--abort|--body <json>] [--resource-type <csv>]`, `network unroute [url]`, `network requests [--clear] [--filter <pattern>] [--type <csv>] [--method <method>] [--status <code|range>]`, `network request <requestId>`, `network har start`, `network har start --content text` (default; embeds text bodies), `network har start --content all`, `network har start --content none`, and `network har stop [path]`. `--resource-type` filters intercepted requests by CDP resource type, such as `script`, `image`, `font`, `xhr`, or `fetch`; request listing filters accept resource types (`xhr,fetch`), methods (`POST`), and statuses (`2xx`, `400-499`). HAR files can include auth headers and bodies—do not share them unredacted. For turning a recording into a reusable API client, load `skills get derive-client`.
|
|
777
|
+
- `cookies [get|set|clear]` — Manage cookies. Full set form: `cookies set <name> <value> --url <url> --domain <domain> --path <path> --httpOnly --secure --sameSite <Strict|Lax|None> --expires <timestamp>`; also supports `cookies set --curl <file>` for JSON, cURL, or bare Cookie-header bulk imports.
|
|
778
|
+
- `storage <local|session>` — Manage web storage.
|
|
763
779
|
|
|
764
780
|
`cookies set --curl <file>` selects native file-import mode even when `--curl` follows other set operands. Ordinary import paths remain visible in invocation echoes and failed batch diagnostics; inline cookie values, sensitive flags, and secret-bearing path text still redact. File contents are not echoed by this exception.
|
|
765
781
|
|
|
@@ -771,16 +787,14 @@ Privacy note: `cookies get` can expose real profile cookies. Do not run it again
|
|
|
771
787
|
|
|
772
788
|
WebMCP support is experimental and browser-dependent. Locally managed Chrome enables it by default; use a fresh launch with `--no-webmcp` or set `AGENT_BROWSER_NO_WEBMCP=1` to disable it. Page tool metadata and results come from the page itself. On 0.37, successful navigation with native `webmcp.available: true` and a positive tool count shows a `webmcp list` hint; the raw object stays in `details.data`. No hint is added for absent, unavailable or empty metadata.
|
|
773
789
|
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
|
|
778
|
-
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
|
|
782
|
-
| `webmcp result <id>` | Wait for or read a detached invocation result; accepts `--timeout <ms>`. |
|
|
783
|
-
| `webmcp cancel <id>` | Cancel an active detached invocation. |
|
|
790
|
+
- `webmcp list` — List tools registered by the current page, including each tool's frame id, origin, schema, and annotations.
|
|
791
|
+
- `webmcp invoke <tool>` — Invoke a page tool with an empty input object.
|
|
792
|
+
- `webmcp invoke <tool> --params <json|@file>` — Pass a JSON object inline or read it from a caller-selected file.
|
|
793
|
+
- `webmcp invoke <tool> --frame <frame-id>` — Select the registering frame when a tool name is ambiguous.
|
|
794
|
+
- `webmcp invoke <tool> --detach` — Start the call and return its invocation id without waiting.
|
|
795
|
+
- `webmcp invoke <tool> --timeout <ms>` — Bound a blocking invocation in milliseconds.
|
|
796
|
+
- `webmcp result <id>` — Wait for or read a detached invocation result; accepts `--timeout <ms>`.
|
|
797
|
+
- `webmcp cancel <id>` — Cancel an active detached invocation.
|
|
784
798
|
|
|
785
799
|
`webmcp invoke`, `webmcp result`, and `webmcp cancel` can run page code that changes or navigates the document. The wrapper refreshes page-target evidence and invalidates prior `@e…` refs after these commands. A detached invocation that still reports `pending`, or fails to settle while its target is unknown, leaves the target unverified rather than trusting the immediate URL probe; `result`, `cancel`, `get url`, and explicit navigation remain available for recovery. `details.nextActions` replaces the blocked snapshot suggestion with `verify-page-target-after-pending-webmcp` (`get url`) and warns that the detached tool remains unsettled. After a completed WebMCP mutation, run `snapshot -i` before reusing refs; inside `batch --bail`, put `get url` before that snapshot. `webmcp list` does not invalidate refs. Attached browsers, providers, Lightpanda, Safari/iOS, and Chrome builds without the experimental CDP domain may return `webmcp_unsupported`.
|
|
786
800
|
|
|
@@ -788,14 +802,12 @@ WebMCP support is experimental and browser-dependent. Locally managed Chrome ena
|
|
|
788
802
|
|
|
789
803
|
Stable tab ids look like `t1`, `t2`, and `t3`. Optional user labels such as `docs` or `app` are interchangeable with ids wherever a tab reference is accepted. Upstream help may refer to numeric tab positions, but this wrapper guidance uses stable `t<N>` ids because positional integers are not accepted by current upstream `agent-browser`.
|
|
790
804
|
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
797
|
-
| `tab <t<N>|label>` | Switch to a tab by id or label. CDP target ids from `tab list --json` are also accepted and stay stable across daemon restarts. |
|
|
798
|
-
| `tab close [t<N>|label|target]` | Close the current tab or a referenced tab. Generic references in workflows may say `tab close [target]`; use a stable `t<N>` id, label, or CDP target id when you have one. |
|
|
805
|
+
- `tab` — List open tabs by default.
|
|
806
|
+
- `tab list` — List open tabs with ids and labels.
|
|
807
|
+
- `tab new [url]` — Open a new tab.
|
|
808
|
+
- `tab new --label <name> [url]` — Open a new tab with a user label.
|
|
809
|
+
- `tab <t<N>|label>` — Switch to a tab by id or label. CDP target ids from `tab list --json` are also accepted and stay stable across daemon restarts.
|
|
810
|
+
- `tab close [t<N>|label|target]` — Close the current tab or a referenced tab. Generic references in workflows may say `tab close [target]`; use a stable `t<N>` id, label, or CDP target id when you have one.
|
|
799
811
|
|
|
800
812
|
The wrapper persists native CDP `targetId` when available so resume/branch recovery distinguishes tabs with the same URL and title. A missing known target requires explicit tab recovery; older transcripts without IDs still use URL matching, and a cold managed reopen adopts its new ID. Failed reached batch navigations retire earlier targets and recheck the live page, including Chrome error pages, before subsequent inspection.
|
|
801
813
|
|
|
@@ -805,17 +817,17 @@ With `--pin-tab`, a closed bound tab fails as `tab_gone` (`data.targetId`, optio
|
|
|
805
817
|
|
|
806
818
|
### Snapshot
|
|
807
819
|
|
|
808
|
-
| Option
|
|
809
|
-
|
|
|
810
|
-
| `snapshot`
|
|
811
|
-
| `snapshot -i` / `snapshot --interactive`
|
|
812
|
-
| `snapshot -i --urls`
|
|
813
|
-
| `snapshot -u` / `snapshot --urls`
|
|
814
|
-
| `snapshot -C` / `snapshot --cursor`
|
|
815
|
-
| `snapshot -c` / `snapshot --compact`
|
|
816
|
-
| `snapshot -d <n>` / `snapshot --depth <n>`
|
|
817
|
-
| `snapshot -s <sel>` / `snapshot --selector <sel>` | Scope to a CSS selector.
|
|
818
|
-
| `snapshot --delta` / `snapshot --delta --full`
|
|
820
|
+
| Option | Purpose |
|
|
821
|
+
| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
822
|
+
| `snapshot` | Full accessibility tree with refs. |
|
|
823
|
+
| `snapshot -i` / `snapshot --interactive` | Include only interactive elements. |
|
|
824
|
+
| `snapshot -i --urls` | Include only interactive elements and link hrefs. |
|
|
825
|
+
| `snapshot -u` / `snapshot --urls` | Include href URLs for link elements. |
|
|
826
|
+
| `snapshot -C` / `snapshot --cursor` | Deprecated no-ops; cursor-interactive elements are included by default. |
|
|
827
|
+
| `snapshot -c` / `snapshot --compact` | Remove empty structural elements. |
|
|
828
|
+
| `snapshot -d <n>` / `snapshot --depth <n>` | Limit tree depth. |
|
|
829
|
+
| `snapshot -s <sel>` / `snapshot --selector <sel>` | Scope to a CSS selector. |
|
|
830
|
+
| `snapshot --delta` / `snapshot --delta --full` | Native revision-relative observation or a forced full baseline; see [0.38.1](#upstream-0381-rebaseline). |
|
|
819
831
|
|
|
820
832
|
When a snapshot is too large for inline output, the Pi wrapper renders a compact view before spilling the full redacted snapshot to `details.fullOutputPath`. Compact snapshots are main-content-first, but dense pages and desktop host screens can still hide actionable controls in omitted content; scan `Omitted high-value controls` before opening the spill file. That bounded section favors editable/searchbox/textbox/combobox controls, named tab/surface controls, primary action buttons, and named action links such as row/navigation links and repository-style result links, then includes other useful controls such as checkboxes, radios, options, and menuitems that were not already listed under key refs or other refs. When that section appears, `details.data.highValueControlRefIds` repeats the same visible ref ids for programmatic follow-up alongside fields such as `previewMode`, `previewSections`, and counts on `details.data` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)).
|
|
821
833
|
|
|
@@ -823,48 +835,46 @@ For dense pages, the wrapper also accepts `snapshot -i --search <text>` and `sna
|
|
|
823
835
|
|
|
824
836
|
### Wait
|
|
825
837
|
|
|
826
|
-
| Mode
|
|
827
|
-
|
|
|
828
|
-
| `wait <selector>`
|
|
829
|
-
| `wait <ms>`
|
|
830
|
-
| `wait --url <pattern>`
|
|
831
|
-
| `wait --load <state>`
|
|
832
|
-
| `wait --fn <expression>`
|
|
833
|
-
| `wait --text <text>`
|
|
834
|
-
| `wait --download [path]`
|
|
835
|
-
| `wait --download [path] --timeout <ms>` | Set download-start timeout in milliseconds. The native Pi wrapper forwards explicit wait timeouts and extends the subprocess watchdog unless the caller supplies top-level `timeoutMs`.
|
|
838
|
+
| Mode | Purpose |
|
|
839
|
+
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
840
|
+
| `wait <selector>` | Wait for an element to appear. |
|
|
841
|
+
| `wait <ms>` | Wait for a fixed number of milliseconds; the duration is positional, not `--time <ms>`. The native Pi wrapper now forwards long waits and derives a subprocess watchdog from the explicit wait duration when the caller does not provide top-level `timeoutMs`. |
|
|
842
|
+
| `wait --url <pattern>` | Wait for the URL to match a pattern. On timeout the wrapper appends a `fresh-session-after-url-wait-timeout` next action (`sessionMode: "fresh"` + `open about:blank`, after the inspect action): if a preceding click or submit reported success but the page never navigated, upstream click dispatch may have silently missed, so replace about:blank with the target URL and replay the flow as one batch in a fresh session instead of retrying the wait. |
|
|
843
|
+
| `wait --load <state>` | Wait for load state: `load`, `domcontentloaded`, or `networkidle`. |
|
|
844
|
+
| `wait --fn <expression>` | Wait for a JavaScript expression to become truthy. |
|
|
845
|
+
| `wait --text <text>` | Wait for text to appear on the page; failures may include `inspect-after-text-assertion-failure` with a session-scoped `snapshot -i` payload. |
|
|
846
|
+
| `wait --download [path]` | Wait for a download started by a previous action and optionally save it to `path`; successful wrapper results include upstream-reported `savedFilePath`/`savedFile`, while `details.artifacts[].exists` is the wrapper's on-disk verification signal. |
|
|
847
|
+
| `wait --download [path] --timeout <ms>` | Set download-start timeout in milliseconds. The native Pi wrapper forwards explicit wait timeouts and extends the subprocess watchdog unless the caller supplies top-level `timeoutMs`. |
|
|
836
848
|
|
|
837
849
|
Current upstream still does not parse `wait <selector> --state hidden` / `wait <selector> --state detached` as distinct wait modes even though upstream help mentions those examples. Use `wait --fn "!document.querySelector('#spinner')"` or another explicit JavaScript predicate for disappearance/detach checks until upstream parser support exists.
|
|
838
850
|
|
|
839
851
|
### Diff, debug, and streaming
|
|
840
852
|
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
|
|
863
|
-
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
| `pushstate <url>` | Perform SPA client-side navigation; detects Next.js router pushes and falls back to history navigation events. |
|
|
867
|
-
| `removeinitscript <id>` | Remove an init script registered through upstream init-script mechanisms. |
|
|
853
|
+
- `diff snapshot` — Compare current versus last snapshot. Use `diff snapshot --baseline <file> --selector <sel> --compact --depth <n>` when you need a saved baseline, scoped subtree, compact output, or depth bound.
|
|
854
|
+
- `diff screenshot --baseline` — Compare current screenshot versus a baseline image. Use `diff screenshot --baseline <file> --output <file> --threshold <0-1> --selector <sel> --full` when you need a saved diff image, threshold tuning, element scope, or full-page capture.
|
|
855
|
+
- `diff url <u1> <u2>` — Navigate to both pages and compare them, leaving the second destination active. The wrapper observes the final URL, including redirects to `about:blank`, and invalidates old refs without recovering the old tab; direct and reached batch rows use the same rule. If the URL cannot be observed, run `get url` before taking a fresh snapshot. Use `diff url <u1> <u2> --screenshot --wait-until <strategy> --selector <sel> --compact --depth <n>` when you need screenshot comparison, navigation wait control, or scoped/compact snapshot comparison.
|
|
856
|
+
- `trace start`, `trace stop [path]` — Record a Chrome DevTools trace.
|
|
857
|
+
- `profiler start|stop [path]` — Record a Chrome DevTools profile.
|
|
858
|
+
- `record start <path> [url]` — Record the active page; an optional URL navigates first. Use `.webm` or `.mp4` and optional `--fps <n>` (1–60, default 30); native validates startup and requires `ffmpeg` on `PATH`. Verify output after `record stop`.
|
|
859
|
+
- `record stop` — Finalize video and inspect its native receipt plus wrapper file verification. A failed or recovered stop retains original attempt evidence; use a distinct top-level `outputPath` to save its receipt.
|
|
860
|
+
- `record restart <path> [url]` — Stop any current recording and start a new video. Supports the same formats and `--fps` option; without a URL it keeps the page and refs.
|
|
861
|
+
- `console [--clear]` — View or clear console logs.
|
|
862
|
+
- `errors [--clear]` — View or clear page errors.
|
|
863
|
+
- `highlight <sel>` — Highlight an element.
|
|
864
|
+
- `inspect` — Open Chrome DevTools for the active page.
|
|
865
|
+
- `clipboard <op> [text]` — Read/write clipboard: `clipboard read`, `clipboard write <text>`, `clipboard copy`, and `clipboard paste`. Clipboard access is environment-dependent; `NotAllowedError` / permission-denied failures are common in headless, managed-profile, remote, or `file://` sessions.
|
|
866
|
+
- `stream enable [--port <n>]` — Start runtime WebSocket streaming for this session. If upstream reports that streaming is already enabled, the wrapper treats it as an idempotent success and adds status/disable follow-ups.
|
|
867
|
+
- `stream disable` — Stop runtime WebSocket streaming.
|
|
868
|
+
- `stream status` — Show streaming status and active port.
|
|
869
|
+
- `react tree` — Print the full React component tree. Requires the page to have been launched with `--enable react-devtools`.
|
|
870
|
+
- `react inspect <id>` — Inspect one React fiber's props, hooks, state, and source.
|
|
871
|
+
- `react renders start` — Start recording React render activity.
|
|
872
|
+
- `react renders stop [--json]` — Stop render recording and print mount/re-render counts and changed details.
|
|
873
|
+
- `react suspense [--only-dynamic] [--json]` — Classify Suspense boundaries with grouped root-cause recommendations.
|
|
874
|
+
- `a11y [url]` — Run an embedded axe-core accessibility audit on the current page, or navigate to `url` first. Options: `a11y --tags wcag2a,wcag2aa`, `a11y --selector "#main"`. CDP browsers only (not Safari/iOS WebDriver). Model-facing text summarizes violation/incomplete counts and top rules; full node targets stay in `details.data`.
|
|
875
|
+
- `vitals [url] [--json]` — Report Core Web Vitals: LCP, CLS, TTFB, FCP, INP, plus React hydration timing when available. `web-vitals [url] [--json]` is the upstream alias.
|
|
876
|
+
- `pushstate <url>` — Perform SPA client-side navigation; detects Next.js router pushes and falls back to history navigation events.
|
|
877
|
+
- `removeinitscript <id>` — Remove an init script registered through upstream init-script mechanisms.
|
|
868
878
|
|
|
869
879
|
Recording destinations are reserved within one Pi process, not across processes. Use unique paths for concurrent Pi processes: different explicit sessions can overwrite one file even when both `record stop` results are verified. Upstream’s same-session `record start` guard does not reserve the filename across other sessions.
|
|
870
880
|
|
|
@@ -876,37 +886,37 @@ Long-running or lifecycle commands should be explicitly paired with cleanup call
|
|
|
876
886
|
|
|
877
887
|
### Batch, auth, confirmations, sessions, chat, dashboard, devices, and setup
|
|
878
888
|
|
|
879
|
-
| Command
|
|
880
|
-
|
|
|
881
|
-
| `batch [--bail] ["cmd" ...]`
|
|
882
|
-
| `auth save <name> [opts]`
|
|
883
|
-
| `auth login <name>`
|
|
884
|
-
| `auth list`
|
|
885
|
-
| `auth show <name>`
|
|
886
|
-
| `auth delete <name>`
|
|
887
|
-
| `confirm <id>`
|
|
888
|
-
| `deny <id>`
|
|
889
|
-
| `session`
|
|
890
|
-
| `session list`
|
|
891
|
-
| `chat <message>`
|
|
892
|
-
| `chat`
|
|
893
|
-
| `dashboard [start]`
|
|
894
|
-
| `dashboard start --port <n>`
|
|
895
|
-
| `dashboard start --allowed-origins <origins>`
|
|
896
|
-
| `dashboard stop`
|
|
897
|
-
| `device list`
|
|
898
|
-
| `install`
|
|
899
|
-
| `install --with-deps`
|
|
900
|
-
| `upgrade`
|
|
901
|
-
| `doctor [--fix]`
|
|
902
|
-
| `plugin add <ref>`
|
|
903
|
-
| `plugin [list]`
|
|
904
|
-
| `plugin show <name>`
|
|
905
|
-
| `plugin run <name> <type>`
|
|
906
|
-
| `auth login <name> --credential-provider <plugin>` | Resolve credentials just-in-time from a configured credential plugin (e.g. a vault) instead of saved passwords; pair with `--item <ref>` and optional selector overrides. Credentials are not stored locally.
|
|
907
|
-
| `mcp --help`
|
|
908
|
-
| `mcp`
|
|
909
|
-
| `profiles`
|
|
889
|
+
| Command | Purpose |
|
|
890
|
+
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
891
|
+
| `batch [--bail] ["cmd" ...]` | Execute multiple commands sequentially from args or stdin. |
|
|
892
|
+
| `auth save <name> [opts]` | Save an auth profile. Full credential form: `auth save <name> --url <url> --username <user> --password <pass>`; selector override form: `auth save <name> --username-selector <s> --password-selector <s> --submit-selector <s>`. Prefer `auth save <name> --password-stdin` with the tool `stdin` field; avoid putting passwords in `args`. |
|
|
893
|
+
| `auth login <name>` | Login using saved credentials; add `--no-navigate` for a prepared, same-origin login page. |
|
|
894
|
+
| `auth list` | List saved auth profiles. |
|
|
895
|
+
| `auth show <name>` | Show auth profile metadata. |
|
|
896
|
+
| `auth delete <name>` | Delete an auth profile; `auth remove <name>` is the upstream alias. |
|
|
897
|
+
| `confirm <id>` | Approve a pending action. |
|
|
898
|
+
| `deny <id>` | Deny a pending action. |
|
|
899
|
+
| `session` | Show current session name. |
|
|
900
|
+
| `session list` | List active sessions. |
|
|
901
|
+
| `chat <message>` | Send a natural-language instruction. |
|
|
902
|
+
| `chat` | Start interactive chat when stdin is a TTY. |
|
|
903
|
+
| `dashboard [start]` | Start the dashboard server on the default port `4848`. |
|
|
904
|
+
| `dashboard start --port <n>` | Start the dashboard on a specific port. |
|
|
905
|
+
| `dashboard start --allowed-origins <origins>` | Allow comma-separated exact HTTPS reverse-proxy origins. Environment: `AGENT_BROWSER_DASHBOARD_ALLOWED_ORIGINS`. |
|
|
906
|
+
| `dashboard stop` | Stop the dashboard server. |
|
|
907
|
+
| `device list` | List available iOS simulators. Use with `-p ios` when exercising iOS provider flows. |
|
|
908
|
+
| `install` | Install browser binaries. |
|
|
909
|
+
| `install --with-deps` | Install browser binaries plus Linux system dependencies; exits nonzero when required libraries cannot be installed. |
|
|
910
|
+
| `upgrade` | Upgrade `agent-browser` using its detected package manager. Native output is text, even with `--json`; the wrapper displays it with surrounding whitespace trimmed and normal redaction, and exposes it in `details.data`. Caller-requested `--json` stays a parseable result with the text in `data`. Nonzero exits, spawn failures, timeout and cancellation remain failures; failed upgrade stdout and stderr remain available as diagnostics. |
|
|
911
|
+
| `doctor [--fix]` | Diagnose install issues and optionally auto-clean stale files. Use `doctor --offline --quick` for a fast local-only check and `doctor --json` for structured output. |
|
|
912
|
+
| `plugin add <ref>` | Add a plugin from npm or GitHub (`<owner>/<repo>` or `@scope/<name>`); writes `agent-browser.json`. Flags such as `--name`, `--capability`, `--global`, and `--no-manifest` shape discovery. |
|
|
913
|
+
| `plugin [list]` | List configured plugins (default subcommand); `{ "plugins": [...] }` is a successful sessionless result. |
|
|
914
|
+
| `plugin show <name>` | Show one configured plugin; `{ "plugin": {...} }` is a successful sessionless result. |
|
|
915
|
+
| `plugin run <name> <type>` | Run a `command.run` or custom plugin request over the agent-browser plugin stdio protocol. |
|
|
916
|
+
| `auth login <name> --credential-provider <plugin>` | Resolve credentials just-in-time from a configured credential plugin (e.g. a vault) instead of saved passwords; pair with `--item <ref>` and optional selector overrides. Credentials are not stored locally. |
|
|
917
|
+
| `mcp --help` | Show MCP server help through the native tool. |
|
|
918
|
+
| `mcp` | Start a local MCP stdio server for external MCP clients; bare native-tool calls are rejected before spawn. External clients can opt into experimental page tools with `mcp --tools core,webmcp`. |
|
|
919
|
+
| `profiles` | List available Chrome profiles. |
|
|
910
920
|
|
|
911
921
|
When these commands are invoked through the native `agent_browser` tool, structured diagnostic/status outputs are rendered as compact summaries. `session list` and `state list` keep every upstream row and restore identifier visible. Explicit sessions, state/restore paths, broad state lifecycle commands, config, local files, and launch environment pass through unchanged. Local inspection/setup calls remain sessionless unless you explicitly pass `--session`; browser-backed or context-dependent calls keep normal managed-session behavior when no explicit session is supplied. List-like outputs such as sessions, Chrome profiles, auth profiles, network requests, console messages, and page errors include counts and key fields; large outputs are previewed with a `Full output path:` spill file instead of dumping the entire payload into context. For `network requests`, the wrapper shows a failed-request summary split into actionable versus benign low-impact rows, then status, method, URL, resource/mime type, request id, and, when the installed upstream output includes body-like fields, bounded redacted payload, response, and failure/error snippets. Safe request IDs also produce `details.nextActions` for exact request details, actionable failed-request source lookup candidates, filtered request lists, or starting HAR capture before a repro. If the same session has active wrapper-observed network routes, failed/pending/CORS-looking matched request rows add `details.networkRouteDiagnostics` and executable route-mock next actions before the generic request actions. `data:image` artifact rows are omitted from compact request previews but remain in raw `details.data.requests`. `network request <requestId>` can expose upstream full-detail body fields such as response bodies using the same bounded model-facing preview; its request URL stays diagnostic-only and does not overwrite `details.sessionTabTarget` for later ref guards. Clipboard failures that mention `NotAllowedError` or permission denial are usually browser/OS capability limits, not proof that a read, paste, or page mutation happened; prefer page-native reads (`snapshot -i`, `get text`, `eval --stdin`) or direct typing (`keyboard inserttext` / `keyboard type`) when the workflow allows it, and retry true clipboard flows only from an allowed profile/session on a normal `http(s)` page. Header, cookie, auth, token, and other secret-like fields are not expanded in model-facing text or `details.data`; low-risk primitive storage values may remain visible, while command echoes still redact `--body`, `--headers`, `--password`, proxy credentials, auth-bearing URLs, `clipboard write` text, cookie/storage set values, and bearer/basic credential text in positional arguments. Raw batch row strings receive the same command-aware redaction in visible output and ordered code journals without changing execution argv. Use upstream HAR or full raw details only when complete data is required.
|
|
912
922
|
|
|
@@ -941,29 +951,29 @@ JSON
|
|
|
941
951
|
`pi install` does not add package helper binaries to your shell `PATH`. Use direct JSON config edits, or run the helper only through `npm exec`:
|
|
942
952
|
|
|
943
953
|
```bash
|
|
944
|
-
npm exec --yes --package pi-agent-browser-native
|
|
945
|
-
npm exec --yes --package pi-agent-browser-native
|
|
946
|
-
npm exec --yes --package pi-agent-browser-native
|
|
947
|
-
npm exec --yes --package pi-agent-browser-native
|
|
948
|
-
npm exec --yes --package pi-agent-browser-native
|
|
949
|
-
npm exec --yes --package pi-agent-browser-native
|
|
950
|
-
npm exec --yes --package pi-agent-browser-native
|
|
951
|
-
npm exec --yes --package pi-agent-browser-native
|
|
952
|
-
npm exec --yes --package pi-agent-browser-native
|
|
953
|
-
printf '%s' "$EXA_API_KEY" | npm exec --yes --package pi-agent-browser-native
|
|
954
|
-
npm exec --yes --package pi-agent-browser-native
|
|
955
|
-
npm exec --yes --package pi-agent-browser-native
|
|
954
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config paths
|
|
955
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config show
|
|
956
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-env EXA_API_KEY --global
|
|
957
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-env BRAVE_API_KEY --global
|
|
958
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-env EXA_API_KEY --project
|
|
959
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search prefer brave --global
|
|
960
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search disable --global
|
|
961
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search disable --project
|
|
962
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-command "op read 'op://Private/Brave Search/API Key'" --provider brave --global
|
|
963
|
+
printf '%s' "$EXA_API_KEY" | npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-key --provider exa --stdin
|
|
964
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config browser profile set "Profile 1" --policy authenticated-only
|
|
965
|
+
npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config browser executable set "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
|
|
956
966
|
```
|
|
957
967
|
|
|
958
|
-
The optional `agent_browser_web_search` tool is available when Exa or Brave credentials are visible from startup config or trusted session config and the runtime config has not set `webSearch.enabled` to `false`. It is a separate custom tool, not an `agent_browser` input mode, and does not launch a browser. Prefer it for current/live external web facts and URL discovery; use `agent_browser` for browser interaction, screenshots, authenticated/profile pages, and DOM inspection after you have a target URL. Prefer it over driving public search-engine forms such as Google with browser typing flows, which can redirect headless automation to anti-bot or CAPTCHA pages; do not attempt CAPTCHA bypass. Disable scope is explicit: `web-search disable --global` sets the normal user default, `web-search disable --project` disables it for one repo, and a `PI_AGENT_BROWSER_CONFIG` override containing `{ "version": 1, "webSearch": { "enabled": false } }` wins over both for a hard per-run disable. Loaded config may use plaintext, custom env aliases, interpolation literals, malformed-or-late-bound `$` values, and command-backed web-search keys; the resolved secret reaches the provider request while model-facing tool output and status text stay redacted. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
|
|
968
|
+
The optional `agent_browser_web_search` tool is available when Exa or Brave credentials are visible from startup config or trusted session config and the runtime config has not set `webSearch.enabled` to `false`. It is a separate custom tool, not an `agent_browser` input mode, and does not launch a browser. Prefer it for current/live external web facts and URL discovery; use `agent_browser` for browser interaction, screenshots, authenticated/profile pages, and DOM inspection after you have a target URL. Prefer it over driving public search-engine forms such as Google with browser typing flows, which can redirect headless automation to anti-bot or CAPTCHA pages; do not attempt CAPTCHA bypass. Disable scope is explicit: `web-search disable --global` sets the normal user default, `web-search disable --project` disables it for one repo, and a `PI_AGENT_BROWSER_CONFIG` override containing `{ "version": 1, "webSearch": { "enabled": false } }` wins over both for a hard per-run disable. Loaded config may use plaintext, custom env aliases, interpolation literals, malformed-or-late-bound `$` values, and command-backed web-search keys; the resolved secret reaches the provider request while model-facing tool output and status text stay redacted. Malformed config files report their scope/path and `Invalid JSON`, plus the parser's numeric position/line/column when available; malformed provider JSON reports a provider-specific invalid-JSON error. Neither includes parser previews or parser causes that could echo partial credentials, and provider HTTP failures report only the status with a fixed hint. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
|
|
959
969
|
|
|
960
970
|
For Exa, effective search type precedence is per-call `searchType` → `webSearch.defaultSearchType` → `auto`, with regular `contents.highlights: true`. Typical latencies are `instant` ~250 ms, `fast` ~450 ms, `auto` ~1 second, `deep-lite` ~4 seconds, `deep` 4–15 seconds, and `deep-reasoning` 12–40 seconds. Prefer `deep-lite` for research before implementation, `deep` for hard multi-source work, and `deep-reasoning` only for exhaustive or still-thin research. Searches are serialized; do not launch several in parallel.
|
|
961
971
|
|
|
962
972
|
```json
|
|
963
973
|
{
|
|
964
|
-
|
|
965
|
-
|
|
966
|
-
|
|
974
|
+
"query": "pi-agent-browser-native agent_browser_web_search searchType defaults",
|
|
975
|
+
"searchType": "deep-lite",
|
|
976
|
+
"count": 5
|
|
967
977
|
}
|
|
968
978
|
```
|
|
969
979
|
|
|
@@ -975,25 +985,25 @@ Example config:
|
|
|
975
985
|
|
|
976
986
|
```json
|
|
977
987
|
{
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
|
|
981
|
-
|
|
982
|
-
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
991
|
-
|
|
992
|
-
|
|
988
|
+
"version": 1,
|
|
989
|
+
"webSearch": {
|
|
990
|
+
"enabled": true,
|
|
991
|
+
"preferredProvider": "exa",
|
|
992
|
+
"defaultSearchType": "deep-lite",
|
|
993
|
+
"exaApiKey": "$EXA_API_KEY",
|
|
994
|
+
"braveApiKey": "$BRAVE_API_KEY"
|
|
995
|
+
},
|
|
996
|
+
"browser": {
|
|
997
|
+
"defaultProfile": {
|
|
998
|
+
"name": "Profile 1",
|
|
999
|
+
"policy": "authenticated-only"
|
|
1000
|
+
},
|
|
1001
|
+
"executablePath": "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
|
|
1002
|
+
}
|
|
993
1003
|
}
|
|
994
1004
|
```
|
|
995
1005
|
|
|
996
|
-
Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers when the effective engine is Chrome. Lightpanda omits these wrapper defaults, without changing caller-selected launch settings. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](
|
|
1006
|
+
Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers when the effective engine is Chrome. Lightpanda omits these wrapper defaults, without changing caller-selected launch settings. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](REFERENCE.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
|
|
997
1007
|
|
|
998
1008
|
## Important global flags, config, and environment
|
|
999
1009
|
|
|
@@ -1017,7 +1027,7 @@ Browser defaults are scoped: global/override `defaultProfile` with `policy: "alw
|
|
|
1017
1027
|
|
|
1018
1028
|
### Browser launch and runtime flags
|
|
1019
1029
|
|
|
1020
|
-
The extension composes stock `--args --no-startup-window` for local Chrome startup
|
|
1030
|
+
The extension composes stock `--args --no-startup-window` for local Chrome startup only when native environment/config does not already supply local launch defaults. Otherwise no automatic Chrome startup arguments are injected; include `--no-startup-window` in native `args` if a single headed window is required. Headless remains the default, external engines/attachments stay native-owned, and caller settings retain native precedence. Plain active follow-ups do not repeat wrapper bootstrap settings; changes to launch settings can trigger native's normal restart. URL-less `open` runs native `get url` to lazily launch or preserve the active page, including batch/code calls; the result exposes the actual command and URL, not a fabricated launch receipt. See [the startup contract](TOOL_CONTRACT.md#headed-and-local-fixture-limits).
|
|
1021
1031
|
|
|
1022
1032
|
- `--executable-path <path>`: custom Chromium-compatible browser executable, such as Brave/Edge/Arc/Vivaldi when upstream can launch that binary. Environment: `AGENT_BROWSER_EXECUTABLE_PATH`.
|
|
1023
1033
|
- `--extension <path>`: load browser extensions; repeatable. Environment: `AGENT_BROWSER_EXTENSIONS`.
|
|
@@ -1106,15 +1116,19 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
1106
1116
|
- The extension may keep following one implicit managed session across later tool calls.
|
|
1107
1117
|
- If launch-scoped flags like `--engine`, `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `--provider` / `-p`, or provider device flags like `--device` would replace or be ignored by an already-active managed session, retry with `sessionMode: "fresh"`. When the call explicitly names the current managed session, the structured recovery payload removes that `--session` so the fresh rotation can succeed.
|
|
1108
1118
|
- If a `sessionMode: "fresh"` call fails (including upstream failure, timeout, missing binary, or **`qa`** reclassification after a nominally successful batch), read `details.managedSessionOutcome` before assuming where the next default call will go: `preserved` means the prior managed session remains current, while `abandoned` means no managed session became current. When the failure reason is not the fresh launch itself—for example `failureCategory: "qa-failure"`—`status`/`summary` may still describe the managed-session transition while `succeeded` on this object matches the final tool outcome.
|
|
1119
|
+
|
|
1109
1120
|
<!-- agent-browser-playbook:start wrapper-tab-recovery -->
|
|
1110
1121
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
1122
|
+
|
|
1111
1123
|
- After open/goto/navigate calls with --profile, --restore, --session-name, or --state, agent_browser best-effort re-selects the tab whose URL matches the returned page when restored tabs steal focus during launch or reconnect.
|
|
1112
1124
|
- After confirmed shutdown of an automatically restored managed session, the wrapper retains its complete recorded URL, including the fragment, until the first current-page operation (including get url and reload). Non-page calls such as tab list may start a daemon without fulfilling that reopen; explicit URL reads leave the managed browser and pending reopen untouched. The wrapper uses native open once, verifies the observed tab, and discards old refs/frame scope; it does not restore unsaved forms, JavaScript memory, or history. Explicit navigation, caller-owned/attached sessions, and restore-disabled sessions are not auto-reopened.
|
|
1113
1125
|
- For a still-live browser after tab drift or resume, the wrapper verifies/selects the intended tab before ref/semantic helpers and page commands; failed selection stops the call without navigating. Local commands, read <url>, URL a11y/vitals, diff url, window new, and explicit tab/navigation/connection/state recovery do not require the prior tab. Batch checks follow effective rows past non-page prefixes and stop at explicit context changes, preserving caller argv/stdin and continue-on-error behavior. Same-tab reselection is avoided because it clears refs. Use exact batch --bail for fail-fast, not --bail=<value>. Routine same-session calls skip tab-list preflights.
|
|
1114
1126
|
- For sessions with observed tab-drift risk, after a successful command on a known target tab, agent_browser also best-effort restores that intended tab if a restored/background tab steals focus after the command completes. Routine same-session commands skip focus correction; observed pages without a native targetId use tab list to establish their identity.
|
|
1115
1127
|
- If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
|
|
1116
1128
|
- If upstream reports tab_gone, the pinned bound tab is gone; use details.nextActions (tab list / tab new) instead of assuming another tab is yours.
|
|
1129
|
+
|
|
1117
1130
|
<!-- agent-browser-playbook:end wrapper-tab-recovery -->
|
|
1131
|
+
|
|
1118
1132
|
- Wrapper-spawned commands clamp `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented 25-second default and use a 35-second child-process watchdog (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides the default 35s budget; top-level `timeoutMs` overrides it per browser CLI call). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a subprocess watchdog from the requested command duration plus a small grace window. Batch budgeting and timeout recovery read the same effective source as upstream: raw command strings when present, otherwise stdin rows. Dialog commands are additionally bounded to 5 seconds (`PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS`), and click/tap/find refs or tokens plus `eval --stdin` snippets that look like alert/confirm/prompt/dialog triggers are bounded to 8 seconds (`PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS`). A `session info` timeout returns only an exact-session `retry-session-info` status action, without page probes, liveness claims, or changes to existing page/ref state. When a browser-operation watchdog fires, `details.timeoutPartialProgress` may include a planned step list with `status: "unknown"` (including `generatedFrom` labels for generated QA rows). Current URLs and existing files do not establish which steps ran, failed, or remained pending. Multi-step timeouts offer `inspect-current-page-after-timeout` when the target is already verified, rather than guessing a retry position. A single read-only or idempotent step may offer a `retry-timeout-step` candidate with a one-row native batch (`args: ["batch"]` plus `stdin`); inspect the current state before retrying because its outcome is still unknown. If the target is unknown, standalone snapshots are removed and visible failure text plus `details.nextActions` show `verify-page-target-after-timeout`, including its session-scoped `batch --bail` args and short stdin for fail-fast `get url` then `snapshot -i`; dialog status/accept/dismiss actions remain allowed for blocking-dialog recovery. It also includes current page URL from best-effort session `get url`, followed by `get title` only after a URL is recovered (or a planned URL inferred from the step list when the session cannot answer), and declared artifact paths such as `screenshot`, `pdf`, `download`, or `wait --download` outputs with existence/state checks; the same evidence is appended under `Timeout partial progress` in visible text with URL/path redaction.
|
|
1119
1133
|
- Oversized snapshots and oversized generic outputs may be compacted in tool content, with the full redacted output written to a spill file path shown directly in the tool result. Recent artifact metadata is bounded by `PI_AGENT_BROWSER_SESSION_ARTIFACT_MANIFEST_MAX_ENTRIES` (default 100); persisted spill files have a separate `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES` budget (default 32 MiB; `0` disables automatic eviction).
|
|
1120
1134
|
- The wrapper keeps `--help` and `--version` stateless so they do not consume the implicit managed-session slot.
|
|
@@ -1129,6 +1143,7 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
1129
1143
|
This generated block is review data for maintainers. The human-authored reference sections above remain the readable command guide.
|
|
1130
1144
|
|
|
1131
1145
|
#### Source evidence
|
|
1146
|
+
|
|
1132
1147
|
- repository: `vercel-labs/agent-browser`
|
|
1133
1148
|
- upstream HEAD: `aff6125c023b810ea3f2e5deec5379e9a4270bdc`
|
|
1134
1149
|
- upstream package version: `0.38.1`
|
|
@@ -1191,6 +1206,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1191
1206
|
- inspected: `packages/@agent-browser/sandbox/src/eve.ts`
|
|
1192
1207
|
|
|
1193
1208
|
#### Upstream help commands sampled
|
|
1209
|
+
|
|
1194
1210
|
- root help: `agent-browser --help`
|
|
1195
1211
|
- skills help: `agent-browser skills --help`
|
|
1196
1212
|
- skills list: `agent-browser skills list`
|
|
@@ -1252,6 +1268,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1252
1268
|
- plugin help: `agent-browser plugin --help`
|
|
1253
1269
|
|
|
1254
1270
|
#### Inventory sections
|
|
1271
|
+
|
|
1255
1272
|
- Built-in skills: 19 human-doc token(s), 24 upstream token(s)
|
|
1256
1273
|
- Core page, element, navigation, and extraction commands: 89 human-doc token(s), 94 upstream token(s)
|
|
1257
1274
|
- Sessions, state, tabs, frames, dialogs, and windows: 28 human-doc token(s), 26 upstream token(s)
|
|
@@ -1260,7 +1277,9 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1260
1277
|
- Global flags, config, providers, policy, and environment: 153 human-doc token(s), 120 upstream token(s)
|
|
1261
1278
|
|
|
1262
1279
|
#### Human-authored doc tokens required
|
|
1280
|
+
|
|
1263
1281
|
##### Built-in skills
|
|
1282
|
+
|
|
1264
1283
|
- `skills list`
|
|
1265
1284
|
- `skills get core`
|
|
1266
1285
|
- `skills get core --full`
|
|
@@ -1282,6 +1301,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1282
1301
|
- `AGENT_BROWSER_SKILLS_DIR`
|
|
1283
1302
|
|
|
1284
1303
|
##### Core page, element, navigation, and extraction commands
|
|
1304
|
+
|
|
1285
1305
|
- `open [url]`
|
|
1286
1306
|
- `open <url>`
|
|
1287
1307
|
- `goto <url>`
|
|
@@ -1373,6 +1393,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1373
1393
|
- `swipe <direction> [distance]`
|
|
1374
1394
|
|
|
1375
1395
|
##### Sessions, state, tabs, frames, dialogs, and windows
|
|
1396
|
+
|
|
1376
1397
|
- `session`
|
|
1377
1398
|
- `session id`
|
|
1378
1399
|
- `session id --scope worktree --prefix <name>`
|
|
@@ -1403,6 +1424,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1403
1424
|
- `window new`
|
|
1404
1425
|
|
|
1405
1426
|
##### Network, storage, artifacts, diagnostics, and performance
|
|
1427
|
+
|
|
1406
1428
|
- `network <action>`
|
|
1407
1429
|
- `network route <url> [--abort|--body <json>] [--resource-type <csv>]`
|
|
1408
1430
|
- `network unroute [url]`
|
|
@@ -1467,6 +1489,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1467
1489
|
- `removeinitscript <id>`
|
|
1468
1490
|
|
|
1469
1491
|
##### Batch, auth, confirmations, setup, dashboard, devices, and AI commands
|
|
1492
|
+
|
|
1470
1493
|
- `batch [--bail]`
|
|
1471
1494
|
- `auth save <name>`
|
|
1472
1495
|
- `auth save <name> --url <url> --username <user> --password <pass>`
|
|
@@ -1506,6 +1529,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1506
1529
|
- `profiles`
|
|
1507
1530
|
|
|
1508
1531
|
##### Global flags, config, providers, policy, and environment
|
|
1532
|
+
|
|
1509
1533
|
- `--profile <name|path>`
|
|
1510
1534
|
- `AGENT_BROWSER_PROFILE`
|
|
1511
1535
|
- `--session <name>`
|
|
@@ -1661,7 +1685,9 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1661
1685
|
- `AWS_DEFAULT_REGION`
|
|
1662
1686
|
|
|
1663
1687
|
#### Upstream help tokens expected
|
|
1688
|
+
|
|
1664
1689
|
##### Built-in skills
|
|
1690
|
+
|
|
1665
1691
|
- root help: `skills get core --full`
|
|
1666
1692
|
- skills help: `get <name> --full`
|
|
1667
1693
|
- skills help: `get --all`
|
|
@@ -1688,6 +1714,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1688
1714
|
- core skill full: `implicit roles work`
|
|
1689
1715
|
|
|
1690
1716
|
##### Core page, element, navigation, and extraction commands
|
|
1717
|
+
|
|
1691
1718
|
- open help: `open [url]`
|
|
1692
1719
|
- open help: `aliases still require a URL.`
|
|
1693
1720
|
- open help: `agent-browser webmcp list`
|
|
@@ -1784,6 +1811,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1784
1811
|
- swipe help: `swipe <direction> [distance]`
|
|
1785
1812
|
|
|
1786
1813
|
##### Sessions, state, tabs, frames, dialogs, and windows
|
|
1814
|
+
|
|
1787
1815
|
- root help: `session list`
|
|
1788
1816
|
- session help: `id`
|
|
1789
1817
|
- session help: `info`
|
|
@@ -1812,6 +1840,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1812
1840
|
- window help: `window <operation>`
|
|
1813
1841
|
|
|
1814
1842
|
##### Network, storage, artifacts, diagnostics, and performance
|
|
1843
|
+
|
|
1815
1844
|
- root help: `network <action>`
|
|
1816
1845
|
- root help: `--resource-type <csv>`
|
|
1817
1846
|
- network help: `unroute [url]`
|
|
@@ -1885,6 +1914,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1885
1914
|
- errors help: `--clear`
|
|
1886
1915
|
|
|
1887
1916
|
##### Batch, auth, confirmations, setup, dashboard, devices, and AI commands
|
|
1917
|
+
|
|
1888
1918
|
- root help: `batch [--bail]`
|
|
1889
1919
|
- root help: `auth save <name>`
|
|
1890
1920
|
- root help: `auth login <name>`
|
|
@@ -1928,6 +1958,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1928
1958
|
- plugin help: `credential.read`
|
|
1929
1959
|
|
|
1930
1960
|
##### Global flags, config, providers, policy, and environment
|
|
1961
|
+
|
|
1931
1962
|
- root help: `--profile <name|path>`
|
|
1932
1963
|
- root help: `AGENT_BROWSER_PROFILE`
|
|
1933
1964
|
- root help: `--session <name>`
|