pi-agent-browser-native 0.9.3 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +77 -5
- package/README.md +103 -81
- package/dist/extensions/agent-browser/index.js +43057 -2031
- package/dist/extensions/agent-browser/lib/argv-descriptor.js +49 -26
- package/dist/extensions/agent-browser/lib/argv-grammar.js +82 -93
- package/dist/extensions/agent-browser/lib/bash-guard.js +129 -126
- package/dist/extensions/agent-browser/lib/batch-lifecycle.js +112 -51
- package/dist/extensions/agent-browser/lib/browser-execution-claim-codec.js +61 -0
- package/dist/extensions/agent-browser/lib/browser-execution-claim-storage.js +232 -0
- package/dist/extensions/agent-browser/lib/browser-execution-claims.js +202 -0
- package/dist/extensions/agent-browser/lib/browser-execution-identity.js +77 -0
- package/dist/extensions/agent-browser/lib/browser-journal-branch.js +98 -0
- package/dist/extensions/agent-browser/lib/browser-journal-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/browser-journal-projection.js +221 -0
- package/dist/extensions/agent-browser/lib/browser-journal-reader.js +89 -0
- package/dist/extensions/agent-browser/lib/browser-journal-replay.js +246 -0
- package/dist/extensions/agent-browser/lib/browser-journal.js +103 -394
- package/dist/extensions/agent-browser/lib/browser-legacy-pages.js +220 -0
- package/dist/extensions/agent-browser/lib/browser-legacy-projection.js +282 -0
- package/dist/extensions/agent-browser/lib/browser-legacy-types.js +1 -0
- package/dist/extensions/agent-browser/lib/browser-session-conversion-stream.js +217 -0
- package/dist/extensions/agent-browser/lib/browser-session-conversion.js +80 -335
- package/dist/extensions/agent-browser/lib/browser-transcript-contracts.js +35 -0
- package/dist/extensions/agent-browser/lib/browser-transcript-validation.js +166 -0
- package/dist/extensions/agent-browser/lib/browser-transcript.js +137 -82
- package/dist/extensions/agent-browser/lib/browser-value-parsing.js +15 -0
- package/dist/extensions/agent-browser/lib/command-capabilities.js +340 -0
- package/dist/extensions/agent-browser/lib/command-policy.js +107 -88
- package/dist/extensions/agent-browser/lib/command-taxonomy.js +49 -322
- package/dist/extensions/agent-browser/lib/config-policy.js +138 -343
- package/dist/extensions/agent-browser/lib/config-providers.js +71 -0
- package/dist/extensions/agent-browser/lib/config-validation.js +218 -0
- package/dist/extensions/agent-browser/lib/config.js +19 -7
- package/dist/extensions/agent-browser/lib/electron/cdp.js +15 -7
- package/dist/extensions/agent-browser/lib/electron/cleanup.js +134 -55
- package/dist/extensions/agent-browser/lib/electron/discovery-darwin.js +109 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-desktop.js +136 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-evidence.js +89 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-linux.js +93 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-packages.js +162 -0
- package/dist/extensions/agent-browser/lib/electron/discovery-types.js +2 -0
- package/dist/extensions/agent-browser/lib/electron/discovery.js +140 -549
- package/dist/extensions/agent-browser/lib/electron/launch-monitor.js +102 -0
- package/dist/extensions/agent-browser/lib/electron/launch-process.js +209 -0
- package/dist/extensions/agent-browser/lib/electron/launch-types.js +2 -0
- package/dist/extensions/agent-browser/lib/electron/launch.js +249 -354
- package/dist/extensions/agent-browser/lib/executable-path.js +21 -12
- package/dist/extensions/agent-browser/lib/execution-cwd.js +35 -18
- package/dist/extensions/agent-browser/lib/input-modes/electron.js +162 -121
- package/dist/extensions/agent-browser/lib/input-modes/job.js +116 -313
- package/dist/extensions/agent-browser/lib/input-modes/lookup-network-evidence.js +104 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-plan.js +132 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-redaction.js +57 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-source-evidence.js +151 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookup-workspace.js +142 -0
- package/dist/extensions/agent-browser/lib/input-modes/lookups.js +57 -412
- package/dist/extensions/agent-browser/lib/input-modes/params.js +63 -17
- package/dist/extensions/agent-browser/lib/input-modes/qa-analysis.js +268 -0
- package/dist/extensions/agent-browser/lib/input-modes/qa-presentation.js +117 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-child.js +396 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-protocol.js +86 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-types.js +1 -0
- package/dist/extensions/agent-browser/lib/input-modes/script-validation.js +60 -0
- package/dist/extensions/agent-browser/lib/input-modes/script.js +60 -409
- package/dist/extensions/agent-browser/lib/input-modes/semantic-action.js +218 -128
- package/dist/extensions/agent-browser/lib/input-modes/shared.js +15 -4
- package/dist/extensions/agent-browser/lib/input-modes/types.js +56 -7
- package/dist/extensions/agent-browser/lib/json-schema.js +22 -10
- package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +17 -5
- package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +92 -425
- package/dist/extensions/agent-browser/lib/managed-session-posix-storage.js +58 -0
- package/dist/extensions/agent-browser/lib/managed-session-project-identity.js +244 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-context.js +84 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-policy.js +119 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore-state.js +83 -0
- package/dist/extensions/agent-browser/lib/managed-session-restore.js +168 -300
- package/dist/extensions/agent-browser/lib/managed-session-snapshot-records.js +277 -0
- package/dist/extensions/agent-browser/lib/managed-session-snapshots.js +128 -305
- package/dist/extensions/agent-browser/lib/managed-session-storage.js +56 -253
- package/dist/extensions/agent-browser/lib/native-confirmation.js +140 -0
- package/dist/extensions/agent-browser/lib/native-output.js +43 -9
- package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +106 -59
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-merge.js +14 -6
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +109 -57
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/browser-observation-result.js +82 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +164 -60
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostic-values.js +31 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +152 -781
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-observation.js +202 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-text-diagnostics.js +109 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/fill-diagnostics.js +119 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-action-recovery.js +147 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-actions.js +144 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-content.js +123 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-details.js +156 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-electron.js +179 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-evidence.js +39 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-missing-binary.js +89 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-recovery.js +97 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-redaction.js +106 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-timeout.js +110 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +41 -538
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +199 -154
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/interaction-diagnostics.js +290 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-close.js +167 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-inspection.js +108 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +127 -205
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/native-helper-observation.js +65 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/observation-types.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/overlay-diagnostics.js +120 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/network-page-filter.js +113 -30
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +109 -28
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter-data.js +150 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +138 -253
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-rendered-text.js +147 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/wait-timeouts.js +59 -24
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-artifacts.js +159 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-command.js +131 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-daemon.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-failure.js +42 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-guards.js +242 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-input.js +75 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-observations.js +92 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-page-selection.js +254 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-page-state.js +121 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-refs.js +130 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-resources.js +114 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-semantic.js +121 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-session-plan.js +222 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare-timeouts.js +68 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +176 -1123
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-analysis.js +191 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-confirmation.js +152 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-diagnostics.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-electron.js +150 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle-phase.js +115 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-lifecycle.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-managed-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-managed.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native-phase.js +93 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-native.js +113 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-navigation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-navigation.js +316 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-ownership.js +28 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-page-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-page-phase.js +142 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-presentation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-presentation.js +333 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication-phase-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication-phase.js +71 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-publication.js +201 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-receipts.js +183 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-refs.js +200 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output-tabs.js +142 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +18 -968
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prompt-guards.js +23 -13
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-receipt-reconciliation.js +198 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery-presentation.js +154 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +68 -151
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-capture.js +14 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +169 -46
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-commands.js +130 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-electron.js +210 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-observations.js +61 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-outcomes.js +187 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-refs.js +171 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-tabs.js +228 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +77 -765
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/text-visibility-diagnostics.js +182 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/timeout-diagnostics.js +256 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/timeout-progress.js +108 -0
- package/dist/extensions/agent-browser/lib/orchestration/code-artifact-observation.js +39 -0
- package/dist/extensions/agent-browser/lib/orchestration/code-image-observation.js +119 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/branch.js +92 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/cleanup-result.js +58 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/cleanup.js +78 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/discovery.js +106 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +48 -821
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/policy.js +86 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-collection.js +69 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-data.js +167 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-presentation.js +164 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe.js +161 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/status.js +165 -0
- package/dist/extensions/agent-browser/lib/orchestration/execution-queue.js +21 -9
- package/dist/extensions/agent-browser/lib/orchestration/extension-artifact-preflight.js +211 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-artifacts.js +23 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-attachment-replay.js +99 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-branch-restore.js +275 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-code.js +280 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-admission.js +303 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-output.js +36 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-result.js +335 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-run.js +227 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-command-state.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-electron-ownership.js +204 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-execution-locks.js +139 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-execution-queue.js +24 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-host-coordination.js +98 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-host-run.js +295 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-intent-replay.js +41 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-invocation.js +7 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-lifecycle.js +102 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-managed-ownership.js +133 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-observation.js +120 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-prompt.js +180 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-recording.js +290 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-resource-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-resource-replay.js +269 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-result-state.js +113 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-runtime.js +146 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-script-leases.js +58 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-tool-boundary.js +62 -0
- package/dist/extensions/agent-browser/lib/orchestration/extension-version-check.js +54 -0
- package/dist/extensions/agent-browser/lib/orchestration/input-plan-types.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +261 -185
- package/dist/extensions/agent-browser/lib/orchestration/input-redaction.js +62 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-config.js +144 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-root-launch.js +77 -0
- package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +210 -157
- package/dist/extensions/agent-browser/lib/orchestration/operation-paths.js +124 -58
- package/dist/extensions/agent-browser/lib/orchestration/output-file.js +197 -99
- package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +199 -96
- package/dist/extensions/agent-browser/lib/owned-managed-session-context.js +55 -0
- package/dist/extensions/agent-browser/lib/page-target-batch.js +118 -0
- package/dist/extensions/agent-browser/lib/page-target-guards.js +50 -0
- package/dist/extensions/agent-browser/lib/page-target-navigation.js +104 -0
- package/dist/extensions/agent-browser/lib/page-target-validation.js +55 -246
- package/dist/extensions/agent-browser/lib/parsing.js +13 -4
- package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +133 -68
- package/dist/extensions/agent-browser/lib/playbook.js +16 -12
- package/dist/extensions/agent-browser/lib/process-child.js +284 -0
- package/dist/extensions/agent-browser/lib/process-environment.js +13 -6
- package/dist/extensions/agent-browser/lib/process-errors.js +23 -0
- package/dist/extensions/agent-browser/lib/process-identity.js +47 -15
- package/dist/extensions/agent-browser/lib/process-socket-storage.js +256 -0
- package/dist/extensions/agent-browser/lib/process-stdout.js +74 -0
- package/dist/extensions/agent-browser/lib/process.js +145 -536
- package/dist/extensions/agent-browser/lib/prompt-artifacts.js +287 -0
- package/dist/extensions/agent-browser/lib/prompt-policy.js +15 -194
- package/dist/extensions/agent-browser/lib/read-command.js +41 -0
- package/dist/extensions/agent-browser/lib/read-confirmation.js +131 -68
- package/dist/extensions/agent-browser/lib/recording-reservations.js +118 -45
- package/dist/extensions/agent-browser/lib/redaction-fields.js +75 -0
- package/dist/extensions/agent-browser/lib/redaction-json.js +149 -0
- package/dist/extensions/agent-browser/lib/redaction-url.js +88 -0
- package/dist/extensions/agent-browser/lib/results/action-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/action-recommendations.js +28 -271
- package/dist/extensions/agent-browser/lib/results/artifact-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/artifact-manifest-validation.js +133 -0
- package/dist/extensions/agent-browser/lib/results/artifact-manifest.js +96 -94
- package/dist/extensions/agent-browser/lib/results/artifact-recommendations.js +73 -0
- package/dist/extensions/agent-browser/lib/results/categories.js +111 -54
- package/dist/extensions/agent-browser/lib/results/confirmation.js +29 -13
- package/dist/extensions/agent-browser/lib/results/editable-ref-evidence.js +62 -27
- package/dist/extensions/agent-browser/lib/results/electron-recommendations.js +83 -0
- package/dist/extensions/agent-browser/lib/results/envelope-decoding.js +48 -0
- package/dist/extensions/agent-browser/lib/results/envelope.js +90 -67
- package/dist/extensions/agent-browser/lib/results/evidence-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/failure-recommendations.js +180 -0
- package/dist/extensions/agent-browser/lib/results/network-routes.js +86 -45
- package/dist/extensions/agent-browser/lib/results/network.js +35 -12
- package/dist/extensions/agent-browser/lib/results/next-actions.js +49 -19
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-files.js +320 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-format.js +138 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-images.js +113 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifact-verification.js +145 -0
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +99 -555
- package/dist/extensions/agent-browser/lib/results/presentation/batch-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-recording.js +118 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-redaction.js +68 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-rendering.js +135 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-step.js +227 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch-summary.js +91 -0
- package/dist/extensions/agent-browser/lib/results/presentation/batch.js +91 -460
- package/dist/extensions/agent-browser/lib/results/presentation/browser-profile-recovery.js +9 -5
- package/dist/extensions/agent-browser/lib/results/presentation/command-presenters.js +216 -0
- package/dist/extensions/agent-browser/lib/results/presentation/common.js +26 -9
- package/dist/extensions/agent-browser/lib/results/presentation/content.js +86 -29
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-a11y.js +67 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-health.js +79 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-identity.js +199 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-logs.js +75 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-network-actions.js +216 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-network.js +124 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-redaction.js +64 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-storage.js +209 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-stream.js +69 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostic-summary.js +222 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +103 -1052
- package/dist/extensions/agent-browser/lib/results/presentation/early-result.js +70 -0
- package/dist/extensions/agent-browser/lib/results/presentation/errors.js +130 -60
- package/dist/extensions/agent-browser/lib/results/presentation/input-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +60 -145
- package/dist/extensions/agent-browser/lib/results/presentation/navigation.js +147 -88
- package/dist/extensions/agent-browser/lib/results/presentation/notices.js +75 -0
- package/dist/extensions/agent-browser/lib/results/presentation/observation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/observation-rendering.js +127 -0
- package/dist/extensions/agent-browser/lib/results/presentation/outcomes.js +120 -0
- package/dist/extensions/agent-browser/lib/results/presentation/output-completion.js +77 -0
- package/dist/extensions/agent-browser/lib/results/presentation/output-spill.js +51 -0
- package/dist/extensions/agent-browser/lib/results/presentation/registry.js +116 -240
- package/dist/extensions/agent-browser/lib/results/presentation/semantic-action.js +35 -21
- package/dist/extensions/agent-browser/lib/results/presentation/skills.js +104 -73
- package/dist/extensions/agent-browser/lib/results/presentation/source.js +109 -0
- package/dist/extensions/agent-browser/lib/results/presentation.js +17 -261
- package/dist/extensions/agent-browser/lib/results/recommendation-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/results/recording.js +52 -14
- package/dist/extensions/agent-browser/lib/results/recovery-actions.js +14 -5
- package/dist/extensions/agent-browser/lib/results/recovery-next-actions.js +30 -22
- package/dist/extensions/agent-browser/lib/results/selector-recovery-target.js +83 -0
- package/dist/extensions/agent-browser/lib/results/selector-recovery.js +119 -119
- package/dist/extensions/agent-browser/lib/results/snapshot-compact.js +266 -0
- package/dist/extensions/agent-browser/lib/results/snapshot-content.js +32 -0
- package/dist/extensions/agent-browser/lib/results/snapshot-high-value-controls.js +104 -71
- package/dist/extensions/agent-browser/lib/results/snapshot-refs.js +31 -11
- package/dist/extensions/agent-browser/lib/results/snapshot-segments.js +125 -66
- package/dist/extensions/agent-browser/lib/results/snapshot-spill.js +26 -7
- package/dist/extensions/agent-browser/lib/results/snapshot.js +28 -248
- package/dist/extensions/agent-browser/lib/results/text.js +16 -5
- package/dist/extensions/agent-browser/lib/runtime-args-validation.js +133 -0
- package/dist/extensions/agent-browser/lib/runtime-compatibility.js +81 -0
- package/dist/extensions/agent-browser/lib/runtime-contracts.js +1 -0
- package/dist/extensions/agent-browser/lib/runtime-redaction.js +118 -0
- package/dist/extensions/agent-browser/lib/runtime-replay-events.js +144 -0
- package/dist/extensions/agent-browser/lib/runtime-session-identity.js +76 -0
- package/dist/extensions/agent-browser/lib/runtime-session-plan.js +114 -0
- package/dist/extensions/agent-browser/lib/runtime-session-replay.js +123 -0
- package/dist/extensions/agent-browser/lib/runtime-tab-correction.js +65 -0
- package/dist/extensions/agent-browser/lib/runtime-timeouts.js +28 -0
- package/dist/extensions/agent-browser/lib/runtime.js +137 -1041
- package/dist/extensions/agent-browser/lib/session-identity.js +51 -0
- package/dist/extensions/agent-browser/lib/session-page-observation.js +357 -0
- package/dist/extensions/agent-browser/lib/session-page-state.js +178 -333
- package/dist/extensions/agent-browser/lib/session-page-types.js +1 -0
- package/dist/extensions/agent-browser/lib/temp-artifact-budget.js +78 -0
- package/dist/extensions/agent-browser/lib/temp-mutation-queue.js +7 -0
- package/dist/extensions/agent-browser/lib/temp-root-ownership.js +234 -0
- package/dist/extensions/agent-browser/lib/temp-root.js +207 -0
- package/dist/extensions/agent-browser/lib/temp.js +33 -446
- package/dist/extensions/agent-browser/lib/tool-surface.js +94 -24
- package/dist/extensions/agent-browser/lib/upstream-version.js +3 -2
- package/dist/extensions/agent-browser/lib/web-search-providers.js +229 -0
- package/dist/extensions/agent-browser/lib/web-search-request.js +108 -0
- package/dist/extensions/agent-browser/lib/web-search-results.js +281 -0
- package/dist/extensions/agent-browser/lib/web-search-schema.js +82 -0
- package/dist/extensions/agent-browser/lib/web-search.js +109 -617
- package/dist/extensions/agent-browser/lib/webmcp-observation.js +13 -6
- package/dist/extensions/agent-browser/lib/windows-stock-launcher.js +63 -37
- package/dist/extensions/agent-browser/script-worker.js +80 -43
- package/dist/scripts/agent-browser-target.mjs +4 -2
- package/docs/ARCHITECTURE.md +17 -5
- package/docs/CODE_QUALITY.md +242 -0
- package/docs/COMMAND_REFERENCE.md +251 -220
- package/docs/ELECTRON.md +53 -44
- package/docs/RELEASE.md +18 -15
- package/docs/REQUIREMENTS.md +1 -0
- package/docs/SUPPORT_MATRIX.md +76 -75
- package/docs/TOOL_CONTRACT.md +145 -116
- package/docs/platform-smoke.md +5 -5
- package/package.json +143 -120
- package/scripts/agent-browser-capability-baseline.mjs +361 -957
- package/scripts/agent-browser-target.mjs +6 -2
- package/scripts/build.mjs +37 -8
- package/scripts/capability-core-commands.mjs +193 -0
- package/scripts/capability-options.mjs +283 -0
- package/scripts/capability-samples.mjs +133 -0
- package/scripts/capability-section.mjs +11 -0
- package/scripts/config.mjs +213 -114
- package/scripts/convert-browser-session.mjs +30 -9
- package/scripts/doctor-sources.mjs +195 -0
- package/scripts/doctor.mjs +92 -247
- package/scripts/platform-smoke/artifacts.mjs +38 -11
- package/scripts/platform-smoke/commands.mjs +104 -0
- package/scripts/platform-smoke/crabbox-runner.mjs +167 -80
- package/scripts/platform-smoke/doctor-support.mjs +157 -0
- package/scripts/platform-smoke/doctor-windows.mjs +230 -0
- package/scripts/platform-smoke/doctor.mjs +212 -320
- package/scripts/platform-smoke/lease-evidence.mjs +134 -0
- package/scripts/platform-smoke/suite-checks.mjs +98 -0
- package/scripts/platform-smoke/suite-evidence.mjs +281 -0
- package/scripts/platform-smoke/targets.mjs +158 -472
- package/scripts/platform-smoke.mjs +55 -40
- package/scripts/prepare.mjs +16 -7
package/docs/TOOL_CONTRACT.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# Tool contract
|
|
2
2
|
|
|
3
3
|
Related docs:
|
|
4
|
+
|
|
4
5
|
- [`../README.md`](../README.md)
|
|
5
6
|
- [`REQUIREMENTS.md`](REQUIREMENTS.md)
|
|
6
7
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md)
|
|
@@ -23,6 +24,7 @@ Official Pi 1.0's initial SDK resume supplies an initial tool selection that byp
|
|
|
23
24
|
## Why this tool shape
|
|
24
25
|
|
|
25
26
|
This keeps the integration:
|
|
27
|
+
|
|
26
28
|
- thin
|
|
27
29
|
- powerful
|
|
28
30
|
- low-drift
|
|
@@ -43,9 +45,9 @@ An SDK host can import the package's compiled `dist/extensions/agent-browser/ind
|
|
|
43
45
|
|
|
44
46
|
```ts
|
|
45
47
|
agentBrowserExtension(pi, {
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
48
|
+
async beforeExecute(toolCallId, ctx) {
|
|
49
|
+
await saveHostState(toolCallId, ctx.signal);
|
|
50
|
+
},
|
|
49
51
|
});
|
|
50
52
|
```
|
|
51
53
|
|
|
@@ -65,13 +67,13 @@ Config shape:
|
|
|
65
67
|
|
|
66
68
|
```json
|
|
67
69
|
{
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
70
|
+
"webSearch": {
|
|
71
|
+
"enabled": true,
|
|
72
|
+
"preferredProvider": "exa",
|
|
73
|
+
"defaultSearchType": "deep-lite",
|
|
74
|
+
"exaApiKey": "$EXA_API_KEY",
|
|
75
|
+
"braveApiKey": "$BRAVE_API_KEY"
|
|
76
|
+
}
|
|
75
77
|
}
|
|
76
78
|
```
|
|
77
79
|
|
|
@@ -79,24 +81,25 @@ Schema:
|
|
|
79
81
|
|
|
80
82
|
```json
|
|
81
83
|
{
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
84
|
+
"query": "search text",
|
|
85
|
+
"provider": "auto",
|
|
86
|
+
"searchType": "deep-lite",
|
|
87
|
+
"includeDomains": ["docs.exa.ai"],
|
|
88
|
+
"excludeDomains": ["archive.example"],
|
|
89
|
+
"category": "publication",
|
|
90
|
+
"additionalQueries": ["Exa search API defaults"],
|
|
91
|
+
"highlightsDynamic": false,
|
|
92
|
+
"count": 5,
|
|
93
|
+
"offset": 0,
|
|
94
|
+
"country": "US",
|
|
95
|
+
"searchLang": "en-US",
|
|
96
|
+
"safesearch": "moderate",
|
|
97
|
+
"freshness": "pw"
|
|
96
98
|
}
|
|
97
99
|
```
|
|
98
100
|
|
|
99
101
|
Provider notes:
|
|
102
|
+
|
|
100
103
|
- `provider` is optional; `auto` uses available keys plus `webSearch.preferredProvider`.
|
|
101
104
|
- `offset` skips results, not pages, for both providers. Each fetches the first `count + offset` results and slices locally; the maximum request is 19 results (`count: 10`, `offset: 9`), within Brave's 20-result page limit.
|
|
102
105
|
- `searchType` applies to Exa only and supports `auto`, `fast`, `instant`, `deep-lite`, `deep`, and `deep-reasoning`. Effective precedence is the per-call field, then `webSearch.defaultSearchType`, then `auto`. Use `deep-lite` for implementation research, `deep` for hard multi-source work, and `deep-reasoning` only for the hardest or exhaustive work.
|
|
@@ -111,26 +114,26 @@ Result details:
|
|
|
111
114
|
|
|
112
115
|
```json
|
|
113
116
|
{
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
117
|
+
"provider": "exa",
|
|
118
|
+
"query": "search text",
|
|
119
|
+
"returnedQuery": "search text",
|
|
120
|
+
"count": 5,
|
|
121
|
+
"offset": 0,
|
|
122
|
+
"searchType": "deep-lite",
|
|
123
|
+
"requestId": "request-id-when-provider-returns-one",
|
|
124
|
+
"duplicatesRemoved": 1,
|
|
125
|
+
"fetchedAt": "2026-06-02T00:00:00.000Z",
|
|
126
|
+
"results": [
|
|
127
|
+
{
|
|
128
|
+
"title": "Result title",
|
|
129
|
+
"url": "https://example.com/",
|
|
130
|
+
"description": "Compact summary or first highlight",
|
|
131
|
+
"highlights": ["Relevant excerpt"],
|
|
132
|
+
"source": "Example",
|
|
133
|
+
"pageDate": "2026-06-02",
|
|
134
|
+
"language": "en"
|
|
135
|
+
}
|
|
136
|
+
]
|
|
134
137
|
}
|
|
135
138
|
```
|
|
136
139
|
|
|
@@ -138,17 +141,17 @@ For Exa, `details.searchType` is the effective requested type even when the prov
|
|
|
138
141
|
|
|
139
142
|
## Input mode chooser
|
|
140
143
|
|
|
141
|
-
| Need
|
|
142
|
-
|
|
|
143
|
-
| One native command
|
|
144
|
-
| Fixed sequence
|
|
145
|
-
| Loops, branches, aggregation
|
|
146
|
-
| Discover/enable advanced tools
|
|
147
|
-
| Stable locator or native select
|
|
148
|
-
| Diagnostic verdict
|
|
149
|
-
| Desktop app lifecycle
|
|
150
|
-
| UI/source candidates
|
|
151
|
-
| Failed-request/source candidates | `agent_browser_network_source` | Flat request/filter fields
|
|
144
|
+
| Need | Tool | Input |
|
|
145
|
+
| -------------------------------- | ------------------------------ | ---------------------------------------------------------------- |
|
|
146
|
+
| One native command | `agent_browser` | `{ args, stdin?, outputPath?, timeoutMs?, sessionMode? }` |
|
|
147
|
+
| Fixed sequence | `agent_browser` | `args: ["batch", "--bail"]` and native JSON-array `stdin` |
|
|
148
|
+
| Loops, branches, aggregation | `agent_browser_code` | `{ code, session?, namespace?, timeoutMs?, outputPath? }` |
|
|
149
|
+
| Discover/enable advanced tools | `agent_browser_tools` | `{ enable?: ["action", "qa", "electron", "source", "network"] }` |
|
|
150
|
+
| Stable locator or native select | `agent_browser_action` | Flat action/locator fields |
|
|
151
|
+
| Diagnostic verdict | `agent_browser_qa` | Flat URL or attached-check fields |
|
|
152
|
+
| Desktop app lifecycle | `agent_browser_electron` | Flat action/target fields |
|
|
153
|
+
| UI/source candidates | `agent_browser_source` | Flat selector/fiber/component fields |
|
|
154
|
+
| Failed-request/source candidates | `agent_browser_network_source` | Flat request/filter fields |
|
|
152
155
|
|
|
153
156
|
Run `agent_browser_tools {}` for inventory; `enable` adds selected capabilities without removing unrelated active tools. Pi's native selected-tool history handles reload/resume rather than a package-owned activation registry. If an exact recovery action names an inactive advanced tool, enable its capability before using that payload.
|
|
154
157
|
|
|
@@ -200,7 +203,7 @@ Upstream 0.35.2 adds `dashboard start --allowed-origins <origins>` and `AGENT_BR
|
|
|
200
203
|
|
|
201
204
|
## Headed and local fixture limits
|
|
202
205
|
|
|
203
|
-
Local Chrome uses stock `--args --no-startup-window` at bootstrap
|
|
206
|
+
Local Chrome uses stock `--args --no-startup-window` at bootstrap only when native environment/config does not own local launch defaults; caller CLI argument values then follow that default. With native environment/config launch defaults, no wrapper Chrome startup arguments are injected, keeping browser, local auth, and helper invocations consistent. Include `--no-startup-window` in native `args` when a single headed window is required. CLI still overrides environment, which overrides native config. Plain active-session follow-ups do not resend wrapper bootstrap settings. Changed launch settings can trigger native's normal session-identity restart; there is no migration or launch-settings journal. Headless remains the default. CDP/auto-connect, providers, Electron, and Lightpanda do not receive this Chrome default. The wrapper does not delete tabs or edit profiles.
|
|
204
207
|
|
|
205
208
|
URL-less `open` is executed as native `get url`, including effective raw/stdin batch rows and code calls. This lazily opens a missing browser without navigating an existing page or issuing a second incomplete launch. `details.args` retains the request; `effectiveArgs`, `command`, native URL data and lifecycle show what ran. No launch receipt is synthesized. Explicit URLs, required-URL errors for `goto`/`navigate`, and top-level help/version argv keep their native meanings. Ignored batch stdin remains unchanged.
|
|
206
209
|
|
|
@@ -218,6 +221,7 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
|
|
|
218
221
|
|
|
219
222
|
<!-- agent-browser-playbook:start shared-guidelines -->
|
|
220
223
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
224
|
+
|
|
221
225
|
- Choose agent_browser for one native command, native batch --bail with JSON-array stdin for a known fixed sequence, or agent_browser_code for loops, branches, and aggregation. Return to the model when a fresh observation needs judgment. Neither tool is a named recipe registry.
|
|
222
226
|
- agent_browser_code takes { code, session?, namespace?, timeoutMs?, outputPath? }. JavaScript globals are fresh for every call; the selected browser persists and is shared with agent_browser. Use await browser({ args, stdin?, timeoutMs? }), inspect result.success, and emit selected JSON instead of whole envelopes. Use emitImage(result.imageObservations[0]) only when image inspection is needed; image handles belong to the current code call. No imports or host filesystem/network/process APIs are exposed.
|
|
223
227
|
- Standard workflow: open the page, snapshot -i, interact using current @refs from that snapshot, and re-snapshot after navigation, scrolling, rerendering, or other major DOM changes because refs are page-scoped; the wrapper fails mutation-prone stale/recycled refs before upstream can silently target a different current-page element. On dense pages, use wrapper-side snapshot -i --search <text> or snapshot -i --filter role=<role> to render matching refs while preserving the full ref map in details.refSnapshot, add snapshot --viewport when scroll position or above/below-fold context matters, and add snapshot --diff when a quick before/after ref-map delta would prevent reading a full spill file.
|
|
@@ -260,6 +264,7 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
|
|
|
260
264
|
- Respect explicit user stop boundaries yourself. When the surrounding authenticated employee or automation context is explicitly unattended/auto-approved, ordinary non-destructive form submissions within the requested flow may proceed without separate confirmation. Still require explicit authorization for purchases, production-control actions, destructive or irreversible actions, and account, security, or privacy changes. The wrapper does not infer broad business intent from prompt text; details.promptGuard is reserved for concrete artifact-before-close checks.
|
|
261
265
|
- Recording needs ffmpeg on PATH before start. Current upstream checks it at startup; older natives may defer failure. A pending recording is not verified output.
|
|
262
266
|
- Do not call --help or other exploratory inspection commands unless the user explicitly asks for them or debugging the browser integration is necessary.
|
|
267
|
+
|
|
263
268
|
<!-- agent-browser-playbook:end shared-guidelines -->
|
|
264
269
|
|
|
265
270
|
## Parameters
|
|
@@ -316,13 +321,13 @@ The browser persists between code and direct calls. JavaScript variables do not.
|
|
|
316
321
|
|
|
317
322
|
```json
|
|
318
323
|
{
|
|
319
|
-
|
|
324
|
+
"code": "const page = await browser({ args: ['get', 'title'] }); if (!page.success) throw new Error(page.error); emit({ title: page.data.title ?? page.data.result });"
|
|
320
325
|
}
|
|
321
326
|
```
|
|
322
327
|
|
|
323
328
|
```json
|
|
324
329
|
{
|
|
325
|
-
|
|
330
|
+
"code": "const shot = await browser({ args: ['screenshot', '/tmp/page.png'] }); if (!shot.success) throw new Error(shot.error); emit(shot.artifactVerification); if (shot.imageObservations?.length) emitImage(shot.imageObservations[0]);"
|
|
326
331
|
}
|
|
327
332
|
```
|
|
328
333
|
|
|
@@ -342,16 +347,16 @@ Abort, timeout, branch change, reload, and shutdown stop/reap the child, settle
|
|
|
342
347
|
|
|
343
348
|
Enable `action` through `agent_browser_tools`, then call the tool with flat fields. Optional `outputPath` and `timeoutMs` apply to the ordinary executor; this tool has no `sessionMode` field.
|
|
344
349
|
|
|
345
|
-
| Field
|
|
346
|
-
|
|
|
347
|
-
| `action`
|
|
348
|
-
| `selector`
|
|
349
|
-
| `locator`
|
|
350
|
-
| `value`
|
|
351
|
-
| `values`
|
|
352
|
-
| `text`
|
|
353
|
-
| `role`, `name` | Accessible role and optional name; role may replace locator value and must agree if both are given
|
|
354
|
-
| `session`
|
|
350
|
+
| Field | Contract |
|
|
351
|
+
| -------------- | ---------------------------------------------------------------------------------------------------- |
|
|
352
|
+
| `action` | Required: `click`, `check`, `fill`, or `select` |
|
|
353
|
+
| `selector` | Direct CSS/selector/current-ref target; cannot combine with locator fields for click/check/fill |
|
|
354
|
+
| `locator` | `role`, `text`, `label`, `placeholder`, `alt`, `title`, or `testid`; select supports only role/label |
|
|
355
|
+
| `value` | Locator value or one select option; for label select, this is the label |
|
|
356
|
+
| `values` | One or more select options; required for label select; rejected on other actions |
|
|
357
|
+
| `text` | Fill text; required for fill, rejected on select |
|
|
358
|
+
| `role`, `name` | Accessible role and optional name; role may replace locator value and must agree if both are given |
|
|
359
|
+
| `session` | Explicit native session; preserved in compiled commands and recovery |
|
|
355
360
|
|
|
356
361
|
Locator click/check/fill compiles to native `find`; direct targets compile to their ordinary command. Select uses native `select <selector> <value...>`. Accessible select resolves exactly one current visible combobox/listbox through a fresh snapshot or the eligible one-use confirmed capture; missing or ambiguous targets fail before action. Use `role: "combobox"` or `"listbox"` plus `name`, or label plus option `values`. No native `find ... select` capability is assumed.
|
|
357
362
|
|
|
@@ -384,13 +389,13 @@ The verdict is `qaPreset: { passed, failedChecks, notRunChecks, warnings, summar
|
|
|
384
389
|
|
|
385
390
|
Enable `electron`. Inputs are flat `{ action, ...fields, outputPath? }`; caller `stdin` and `sessionMode` are not accepted. The [Electron guide](ELECTRON.md) covers workflows and ownership.
|
|
386
391
|
|
|
387
|
-
| Action
|
|
388
|
-
|
|
|
389
|
-
| `list`
|
|
390
|
-
| `launch`
|
|
391
|
-
| `status`
|
|
392
|
-
| `probe`
|
|
393
|
-
| `cleanup` | `launchId?` or `all: true`, `timeoutMs?`
|
|
392
|
+
| Action | Fields | Behavior |
|
|
393
|
+
| --------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
394
|
+
| `list` | `query?`, `maxResults?` | Host-only bounded discovery; default 50, cap 200. No timeout field. macOS/Linux scanning only; Windows returns unsupported discovery. |
|
|
395
|
+
| `launch` | Exactly one of `appPath`, `appName`, `bundleId`, `executablePath`; `appArgs?`, `handoff?`, `targetType?`, `timeoutMs?`, `allow?`, `deny?` | Verify Electron evidence, create a temporary isolated profile/OS-chosen debug port, attach through native connect, hand off, and record ownership. |
|
|
396
|
+
| `status` | `launchId?` or `all: true`, `timeoutMs?` | Inspect tracked PID/port/profile and targets; explicit cleaned IDs remain inspectable as historical records. |
|
|
397
|
+
| `probe` | `launchId?`, `timeoutMs?` | Verify URL then read title/focus/tabs/snapshot with owned session settings. Without an ID, inspect the current attached managed session. |
|
|
398
|
+
| `cleanup` | `launchId?` or `all: true`, `timeoutMs?` | Close the tracked native session, stop only the owned app, verify port shutdown, remove its temporary profile. Partial cleanup fails with exact retry guidance. |
|
|
394
399
|
|
|
395
400
|
```json
|
|
396
401
|
{ "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" }
|
|
@@ -450,7 +455,10 @@ For `eval --stdin`, put the script in the top-level `stdin` field. The wrapper n
|
|
|
450
455
|
```
|
|
451
456
|
|
|
452
457
|
```json
|
|
453
|
-
{
|
|
458
|
+
{
|
|
459
|
+
"args": ["auth", "save", "my-login", "--password-stdin"],
|
|
460
|
+
"stdin": "password from the user-approved secret source"
|
|
461
|
+
}
|
|
454
462
|
```
|
|
455
463
|
|
|
456
464
|
### `outputPath`
|
|
@@ -466,7 +474,11 @@ For `eval --stdin`, put the script in the top-level `stdin` field. The wrapper n
|
|
|
466
474
|
Example:
|
|
467
475
|
|
|
468
476
|
```json
|
|
469
|
-
{
|
|
477
|
+
{
|
|
478
|
+
"args": ["eval", "--stdin"],
|
|
479
|
+
"stdin": "({ title: document.title, url: location.href })",
|
|
480
|
+
"outputPath": "logs/page-state.json"
|
|
481
|
+
}
|
|
470
482
|
```
|
|
471
483
|
|
|
472
484
|
### `timeoutMs`
|
|
@@ -494,6 +506,7 @@ A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` sel
|
|
|
494
506
|
- default: `"auto"`
|
|
495
507
|
|
|
496
508
|
Behavior:
|
|
509
|
+
|
|
497
510
|
- if `args` already include `--session` (including argv compiled from `agent_browser_action.session`), upstream session choice wins
|
|
498
511
|
- `"auto"` prepends the current extension-managed active session when appropriate
|
|
499
512
|
- after resume, a confirmed inactive wrapper-owned daemon with automatic managed restore enabled reopens its complete recorded URL, including its fragment, before the first current-page operation, including `get url` and `reload`. Non-page calls such as `tab list` or explicit HTTP `read <url>` can start a daemon without consuming that pending reopen, even across branch/reload replay. The wrapper verifies the observed tab and invalidates old refs; native `open` resets frame scope. This reloads the page with restored cookies/storage, not unsaved forms, JavaScript memory, or history. Live wrong-tab recovery does not navigate. Explicit URL reads, URL `a11y`/`vitals`, `diff url`, `window new`, URL-bearing recording commands, and explicit navigation/context changes do not require the old tab. Caller-owned/attached and restore-disabled sessions are not auto-reopened
|
|
@@ -504,6 +517,7 @@ Behavior:
|
|
|
504
517
|
- sessionless paths skip that injection even under `"auto"`: plain-text help/version, read-only skills, local auth/profile/setup commands, `session list`, and syntactically local state lifecycle operations keep `effectiveArgs` free of the implicit managed `--session` unless the caller supplied one. All upstream rows and supported targets remain available. Browser-backed or context-dependent commands such as `auth login` and `state save/load` keep normal managed-session injection when the caller did not choose an explicit session (`extensions/agent-browser/lib/command-policy.ts`, `needsManagedSession`; `extensions/agent-browser/lib/runtime.ts`, `buildExecutionPlan`)
|
|
505
518
|
|
|
506
519
|
Recommended use:
|
|
520
|
+
|
|
507
521
|
- use `"auto"` for the common browse/snapshot/click flow inside one `pi` session
|
|
508
522
|
- use `"fresh"` when switching from an already-active implicit session to domain containment or a new profile/browser executable/debug/auth/provider launch without inventing a fixed explicit session name
|
|
509
523
|
- when a fresh launch fails or times out before becoming current, check `details.managedSessionOutcome`: it states whether the prior managed session was preserved or whether the attempted fresh session was abandoned because no prior managed session existed; when `sessionMode` is `"fresh"` and the tool ultimately fails, the model-visible result also appends `Managed session outcome: …` (see `#details` below). Failures under `sessionMode: "auto"` still expose the struct on `details` when the extension injects a managed `--session`, but they do not add that extra prose line.
|
|
@@ -513,6 +527,7 @@ Recommended use:
|
|
|
513
527
|
Caller `args` may omit `--json` for structured details and prose; explicit `--json false` selects opaque native text. See [Wrapper `--json`](#wrapper-json).
|
|
514
528
|
|
|
515
529
|
The extension should:
|
|
530
|
+
|
|
516
531
|
- inject `--json` when no caller output mode is selected
|
|
517
532
|
- invoke `agent-browser` directly, not through a shell
|
|
518
533
|
- parse JSON-mode output into tool details
|
|
@@ -522,6 +537,7 @@ The extension should:
|
|
|
522
537
|
|
|
523
538
|
<!-- agent-browser-playbook:start inspection -->
|
|
524
539
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
540
|
+
|
|
525
541
|
Native inspection calls use the `agent_browser` tool shape, not shell-like direct-binary commands:
|
|
526
542
|
|
|
527
543
|
- { "args": ["--help"] }
|
|
@@ -529,6 +545,7 @@ Native inspection calls use the `agent_browser` tool shape, not shell-like direc
|
|
|
529
545
|
|
|
530
546
|
These calls return plain text and stay stateless: the extension does not inject its implicit session and does not let inspection consume the managed-session slot needed for later profile, session, CDP, state, auto-connect, or provider-backed launches.
|
|
531
547
|
<!-- agent-browser-playbook:end inspection -->
|
|
548
|
+
|
|
532
549
|
- still describe normal browser workflows in guidance so models do not overuse inspection for routine tasks
|
|
533
550
|
- surface stderr and non-zero exits clearly
|
|
534
551
|
- attach images when the result points to a screenshot-like artifact
|
|
@@ -560,6 +577,7 @@ Conditional `changed: false` captures have no new image/artifact attachment. A p
|
|
|
560
577
|
### Content
|
|
561
578
|
|
|
562
579
|
Primary content should be:
|
|
580
|
+
|
|
563
581
|
- useful result text for the model, not just a status line
|
|
564
582
|
- an image attachment when relevant
|
|
565
583
|
- browser-aware compacting for oversized snapshots so the model gets a concise actionable view before raw page noise
|
|
@@ -568,6 +586,7 @@ Primary content should be:
|
|
|
568
586
|
- wrapper-side `snapshot -i --search <text>` and `snapshot -i --filter role=<role>` filters should strip those wrapper-only flags before upstream spawn, preserve the full latest ref map in `details.refSnapshot`, and render matching direct refs plus surrounding snapshot context with `details.snapshotFilter` counts so dense-page agents can find controls without opening raw spill files; search should also run one bounded read-only rendered-DOM probe across the full document so visible below-fold warnings and accessible labels omitted from the accessibility snapshot remain discoverable while hidden nodes stay excluded; the visible summary should distinguish direct ref matches from rendered/contextual matches to avoid apparent count mismatches; wrapper-side `--viewport` should also strip before upstream spawn, run one read-only viewport/scroll probe, and report `details.snapshotViewport`; wrapper-side `--diff` should strip before upstream spawn and report `details.snapshotDiff` against the previous wrapper-tracked ref map for that session
|
|
569
587
|
|
|
570
588
|
Examples:
|
|
589
|
+
|
|
571
590
|
- small `snapshot` results should include the actual snapshot text
|
|
572
591
|
- oversized `snapshot` results should switch to a compact view that preserves the primary content, nearby sections, a trimmed set of high-value refs, and a separate bounded list of omitted high-value controls when dense pages or desktop host screens would otherwise hide editable inputs, named surfaces/tabs, or primary action buttons, while exposing the full redacted snapshot path directly in the rendered tool text and via `details.fullOutputPath`
|
|
573
592
|
- successful navigation actions like `click`, `back`, `forward`, and `reload` should include a lightweight post-action title/url summary when the wrapper can address the active session
|
|
@@ -580,22 +599,22 @@ Recommended details:
|
|
|
580
599
|
|
|
581
600
|
```json
|
|
582
601
|
{
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
602
|
+
"args": ["snapshot", "-i"],
|
|
603
|
+
"effectiveArgs": ["--json", "--session", "pi-abc123", "snapshot", "-i"],
|
|
604
|
+
"command": "snapshot",
|
|
605
|
+
"sessionMode": "auto",
|
|
606
|
+
"sessionName": "pi-abc123",
|
|
607
|
+
"usedImplicitSession": true,
|
|
608
|
+
"resultCategory": "success",
|
|
609
|
+
"successCategory": "completed",
|
|
610
|
+
"data": {
|
|
611
|
+
"origin": "https://example.com/",
|
|
612
|
+
"refs": {
|
|
613
|
+
"e1": { "name": "Example Domain", "role": "heading" }
|
|
614
|
+
},
|
|
615
|
+
"snapshot": "- heading \"Example Domain\" [level=1, ref=e1]"
|
|
616
|
+
},
|
|
617
|
+
"summary": "Snapshot: 1 refs on https://example.com/"
|
|
599
618
|
}
|
|
600
619
|
```
|
|
601
620
|
|
|
@@ -703,6 +722,7 @@ Implementation and precedence:
|
|
|
703
722
|
- The main tool implementation merges these fields into Pi-facing `details` from `extensions/agent-browser/index.ts` and from `extensions/agent-browser/lib/results/presentation.ts` for presentation-time failures.
|
|
704
723
|
|
|
705
724
|
Additional structured fields can appear when relevant:
|
|
725
|
+
|
|
706
726
|
- `sessionTabReopenPending: boolean` persists a confirmed-cold managed session's outstanding URL reopen through non-page commands and branch/reload replay. `true` means a daemon may have started but the remembered page has not been reopened; `false` means a reopen attempt or an executed explicit context/navigation command consumed the obligation, not that navigation succeeded. Cancellation after the reopen CLI starts returns `failureCategory: "aborted"`, the exact `sessionName` / `namespace`, the consumed `false` marker and ref invalidation through the normal result path, so replay cannot repeat the navigation. Cancellation before the CLI starts leaves the obligation pending. Successful close clears it with the rest of that session's page state. Internal remembered URLs retain their complete fragment; comparison remains fragment-insensitive and presentation redaction is unchanged. Old persisted targets without a fragment cannot reconstruct it.
|
|
707
727
|
- `closeAllApplied: true` when a successful direct or nested `close` / `quit` / `exit --all` reached upstream. The marker makes live state and transcript replay clear every managed/attached/page/ref/route/trace/recording identity in the effective canonical namespace; a later batch row that proves browser reactivation may rebuild only the effective session.
|
|
708
728
|
- `attachedBrowserSession: true` on successful calls that establish or reuse a wrapper-tracked CDP/auto-connect/Electron attachment, and on a failed fresh attachment only when `managedSessionOutcome.activeAfter` proves its daemon remained active for cleanup. The marker restores attachment continuity from the active transcript branch, including that active-after-failure case; live state and transcript replay remove it after a terminal successful close/cleanup even when aggregate verification failed; a close followed by a later step whose lifecycle reports a browser launch preserves it, while a successful non-launching diagnostic leaves the close terminal. Caller config, environment, paths, and file-access settings remain upstream-owned; the marker only adds live-URL verification and lifecycle continuity.
|
|
@@ -750,8 +770,8 @@ Additional structured fields can appear when relevant:
|
|
|
750
770
|
- `imagePath` / `imagePaths` for Pi inline image attachments from the **`screenshot`** command (including batched screenshot steps). **`diff screenshot`** still records the diff output as an `image`-kind entry in `details.artifacts`, but it does **not** populate `imagePath` / `imagePaths` or attach an inline image: only plain `screenshot` is treated as a trusted live-capture path for automatic inlining (`isTrustedScreenshotOutput` in `extensions/agent-browser/lib/results/presentation/artifacts.ts`).
|
|
751
771
|
- `artifacts` for saved files such as screenshots, `state save` outputs, `diff screenshot` diff images, PDFs, downloads, `wait --download` / `wait -d` files, traces, CPU profiles, completed video recordings, path-bearing HAR captures, and future recording output paths reported by `record start` / `record restart`. Non-file URL payloads such as `data:` / `blob:` / `http(s):` values are not treated as verified local artifacts. For direct artifact commands and batch artifact steps, the wrapper creates parent directories for requested paths before spawning upstream. Filesystem `mkdir` failures at this shared preparation boundary return `validation-error`, `agentBrowserStarted: false`, the attempted directory and `verify-artifact-path` guidance. When execution cwd differs from the launch/project root, relative file operands in raw batch strings are bound to the captured invocation root; otherwise raw strings stay unchanged and absolute artifact paths are preferred. Each artifact includes the original saved or requested `path`, resolved `absolutePath`, `kind`/`artifactType`, optional `mediaType`, optional `extension`, best-effort disk metadata such as `exists`, `sizeBytes`, and `updatedAtMs`, plus `requestedPath`, `status`, `cwd`, `session`, `namespace`, and `tempPath` when applicable. `requestedPath` is retained only when known from the caller, separately from reported/resolved locations; a differing screenshot report remains in `tempPath` and is displayed as `Reported path`, whether it is a temporary file or a canonical path alias. Ordinary file `mediaType` values come from bounded PNG/JPEG/GIF/WebP header recognition, not suffixes; unknown, missing, unreadable or truncated headers leave it undefined. Header recognition is not full-file format validation. Inline screenshot attachments use the same byte classifier and existing size limit. For commands that create/update artifacts, a path that existed but was not updated during this command uses `status: "stale"`; observational `wait --download` may accept a file completed just before the wait began. Pending `record start` / `record restart` artifacts use `status: "pending"`, omit `exists` rather than reporting false, and include `recordingState: "openRecording"` / `willExistOnStop: true`. Within one Pi extension process, the wrapper keeps an unbounded transcript-backed active-recording reservation index separate from the bounded artifact manifest, keyed by canonical namespace plus session; still-live process-owned reservations survive branch switches, while known closures are appended after tree navigation and during shutdown/reload so a close on one branch cannot be resurrected after returning to an older branch. Persisted active reservations require absolute storage paths and cwd; their display paths may remain relative. If a journal append fails, the next serialized browser boundary, tree navigation, or shutdown retries all current reservations and known closures. `recordingPersistenceWarning` and visible warning text remain present while restart protection is not durable; successful recovery is quiet and cleanup still runs. Artifact lifecycle calls, explicit `wait --download <path>` / `wait -d <path>` destinations, and result `outputPath` writes serialize around the global destination check/update, every successful direct, ordered nested-batch, fresh-replacement, code, Electron, or shutdown close retires only its exact identity at that lifecycle point, and destination reuse is rejected through lexical, existing or dangling symlink, hardlink, full Unicode-fold, or macOS/Windows case aliases. Batch preflight rejects `record start` / `record restart` after a close row because upstream can report a recording that did not start; split those operations into separate calls. A `No recording in progress` stop failure, direct or nested, first checks the matching native receipt once. Retirement preserves the receipt and freshly checked file metadata; it never turns an existing file into a missing file merely because no recording is active. A later successful batch recording row opens its new pending path normally. Batch preflight applies the same distinct-destination rule to the steps upstream will execute: raw argument command strings exclusively when any exist, stdin arrays only otherwise; upstream-ignored stdin rows cannot fail artifact preflight, add pending recordings, or create parent directories. Parent directories are prepared for the effective steps in both modes; outside execution-directory binding, raw argument strings stay unchanged, so screenshot normalization and tracked path requests apply to stdin rows only. Outer CLI globals are removed before artifact parsing, but native batch row operands stay literal: `pdf --quick ignored.pdf` targets `--quick`, not `ignored.pdf`. Reservation checks, preparation, and requested-path presentation follow that same distinction. Recording path/URL consumers skip complete numeric `--fps` pairs without rewriting argv; native still validates rate, format and extra arguments. FPS-only calls keep the intended pinned tab.
|
|
752
772
|
|
|
753
|
-
|
|
754
773
|
Recording destinations are reserved within one Pi process, not across processes. Use unique paths for concurrent Pi processes: different explicit sessions can overwrite one file even when both `record stop` results are verified. Upstream’s same-session `record start` guard does not reserve the filename across other sessions.
|
|
774
|
+
|
|
755
775
|
- `savedFilePath` / `savedFile` for direct `download`, `pdf`, and `wait --download` / `wait -d` saved-file workflows when a host file path is reported or wrapper-verified. Batch results preserve the same fields on the relevant `batchSteps` entry. These fields are metadata only until `artifactVerification` verifies the file. Native `download <selector> <path>` always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper does not substitute a fetch of an anchor's `href`.
|
|
756
776
|
- `batchSteps[].artifacts` for per-step artifacts in `batch` output; top-level `artifacts` and `artifactManifest` coalesce an earlier pending recording into the later saved, missing, or stale terminal result for the same namespace/session identity. `record restart` includes both the previous recording it finalized (or an explicit missing/stale failure) and the new pending recording; missing/stale terminal rows retire the prior pending manifest row. A successful later `close` / `quit` / `exit` retires an earlier unfinalized pending recording as `subcommand: "close-abandoned"`, clears its stop action, and leaves its file unverified. Batch presentation checks the path: only a confirmed absent file becomes `status: "missing"`; a present or inaccessible file stays unverified. It updates aggregate verification/manifest state consistently; a later successful `record stop` replaces that intermediate abandoned row with its saved artifact. Close also resets ref/page/network-route state produced by earlier rows; later lifecycle-proven browser launches, including `record stop`, can rebuild that state without triggering stale pre-close `about:blank` recovery, explicitly non-launching diagnostics cannot, and unknown later rows stay conservatively active. Per-step history remains unchanged. When any later call on the same namespace/session fails while a recording remains pending, `nextActions` combines its normal recovery with exact `stop-pending-recording` args and visible cleanup guidance; the same applies at top level when a later batch step fails. After reload in a non-Git checkout or with managed restore disabled, a live daemon without current-instance provenance or a matching same-owner native daemon receipt cannot accept a stop. A tracked Electron attachment can rebuild that proof through the live debug-endpoint check described above; unproven restore-disabled sessions cannot rebuild provenance from a name or URL. That policy refusal includes `managedSessionCleanupOnlyReason: "restore-disabled-daemon-without-provenance"` plus the exact `sessionName`/`namespace`, including on implicit calls. It replaces the impossible stop with `close-pending-recording`, an exact close without `sessionMode: "fresh"`. Close retires the recording as `close-abandoned`; any file it leaves is unverified. Same-instance recordings and supported durable-Git reloads still use stop and normal WebM verification.
|
|
757
777
|
- `artifactVerification` for a normalized verification summary on the unified result and on each successful `batchSteps[]` row and on failed recording rows whose receipt identifies an artifact. Top-level `batch` verification rolls up all step file artifacts; each step’s summary reflects that step’s nested tool presentation (including its spill paths and manifest slice). It reports `verified`, `verifiedCount`, `missingCount`, `pendingCount`, `unverifiedCount`, and `artifacts[]` entries with `path`, optional `absolutePath`, optional `requestedPath`, `kind` (a normal file artifact kind or `"spill"` for manifest-backed rows), optional `mediaType`, optional `exists`, optional `sizeBytes`, optional `updatedAtMs`, optional `status`, optional `retentionState` / `storageScope` on manifest-derived rows, `state` (`verified`, `missing`, `pending`, or `unverified`), and optional `limitation` (human-readable lifecycle or retention context, for example pending `record start` / `record restart`, missing, stale, or otherwise unverified files, ephemeral spill files, or evicted persisted spills). The summary `verified` boolean is true only when every entry is `verified`. `record start` / `record restart` are `pending` until `record stop`; `state load` may mention a path in command output but is not a saved artifact row.
|
|
@@ -806,6 +826,7 @@ A failed inner native confirmed action is a tool failure, including inside a bat
|
|
|
806
826
|
The TUI renderer is user-facing only. It may compact or colorize what the human sees in the Pi transcript, but it must not further truncate, summarize, or remove the model-facing `content` returned by the tool. Use the existing `details.fullOutputPath` / spill-file contracts for content that is too large for the model.
|
|
807
827
|
|
|
808
828
|
Worth doing in v1:
|
|
829
|
+
|
|
809
830
|
- screenshots → saved-path summary, visible artifact metadata, `details.artifacts` metadata, and inline image attachment when safe; screenshot paths that upstream would treat ambiguously, such as `.dogfood/run/foo.png`, are normalized to absolute paths before launch and repaired from upstream temp output when possible
|
|
810
831
|
- file artifacts such as PDFs, downloads, `wait --download` / `wait -d` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed video recordings, and path-bearing HAR captures → concise saved-path summaries plus metadata in `details.artifacts` and changed/referenced invocation receipts in `details.artifactManifest`; `record start` / `record restart` report recording lifecycle state and the future output path without adding a missing manifest entry, and `record restart` can also report the previous wrapper-known recording that was finalized by the restart; native 0.37 checks `ffmpeg` before starting video capture, while older supported natives may defer failure; successful start/restart calls without ffmpeg expose `details.recordingDependencyWarning` and leave output unverified until checked after stop; direct saved-file workflows also expose `details.savedFilePath` / `details.savedFile`; large or binary artifacts are not inlined into model context; the recent manifest cap can age out explicit-file metadata but does not remove explicit saved files from disk
|
|
811
832
|
- `diff screenshot` → same file-artifact pattern as above for the **diff** image path only (summary text uses “Saved diff image” only when the diff output exists; missing output says “Diff image reported; file not verified” and fails as `artifact-missing`); baseline paths and other fields stay in the structured payload but are not echoed as separate saved artifacts in the visible artifact block, and there is no Pi inline image attachment for the diff output
|
|
@@ -825,6 +846,7 @@ Worth doing in v1:
|
|
|
825
846
|
## Missing binary behavior
|
|
826
847
|
|
|
827
848
|
If `agent-browser` is not on `PATH`, fail with a message that:
|
|
849
|
+
|
|
828
850
|
- says `agent-browser` is required
|
|
829
851
|
- says this project does not bundle it
|
|
830
852
|
- points to upstream install/docs
|
|
@@ -851,15 +873,19 @@ If `agent-browser` is not on `PATH`, fail with a message that:
|
|
|
851
873
|
- treat every explicit caller-provided `--session` as user-managed, including `piab-*` names. Wrapper-owned implicit sessions set a Pi-transcript- and Git-checkout-generation-scoped `AGENT_BROWSER_RESTORE` key automatically unless disabled with `PI_AGENT_BROWSER_MANAGED_SESSION_RESTORE=0`; explicit caller sessions do not receive that injection unless they exactly target the current wrapper-owned identity. Caller state/restore paths, profiles, upstream config, file access, launch arguments, environment variables, local file pages, `outputPath`, and close arguments pass through unchanged. `session list` and `state list` keep all upstream rows and restore identifiers visible. Automatic restore still validates and pins its own checkout/storage/namespace identity and coordinates same-daemon reuse so the wrapper cannot mix restore pools or corrupt managed lifecycle state. Ambiguous tab, attachment, history, code, or state-load transitions remain page-target correctness boundaries: content calls live-check `get url` or require explicit navigation before acting. Windows uses `cross-spawn` for native executable and `.cmd` argument transport, rather than PowerShell or wrapper-owned argument reordering. Empty operands such as `fill #field ""`, `--args ""`, and explicit default `--namespace ""`, literal doublequotes in fill text, and command/subcommand adjacency are retained. Upstream receives the empty namespace rather than a wrapper omission or environment workaround. The selected child `PATH` shim is not bypassed; POSIX keeps native Node `spawn`.
|
|
852
874
|
- before a DOM/content-bearing read or interaction against a caller-owned explicit session or established attachment, run a session-scoped `get url` probe so stale transcript state cannot target the wrong page. A failed or non-URL probe blocks the requested content command. The shared execution lock orders that probe, semantic snapshot resolution, and the main command across cooperating updated Pi processes. Nested `batch` steps remain unsupported; raw batch command strings mirror upstream's ASCII-space tokenizer, including quoting and backslash handling.
|
|
853
875
|
- pass explicit `--profile` straight through to upstream `agent-browser`; no profile-cloning or isolation layer is added in v1
|
|
876
|
+
|
|
854
877
|
<!-- agent-browser-playbook:start wrapper-tab-recovery -->
|
|
855
878
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
879
|
+
|
|
856
880
|
- After open/goto/navigate calls with --profile, --restore, --session-name, or --state, agent_browser best-effort re-selects the tab whose URL matches the returned page when restored tabs steal focus during launch or reconnect.
|
|
857
881
|
- After confirmed shutdown of an automatically restored managed session, the wrapper retains its complete recorded URL, including the fragment, until the first current-page operation (including get url and reload). Non-page calls such as tab list may start a daemon without fulfilling that reopen; explicit URL reads leave the managed browser and pending reopen untouched. The wrapper uses native open once, verifies the observed tab, and discards old refs/frame scope; it does not restore unsaved forms, JavaScript memory, or history. Explicit navigation, caller-owned/attached sessions, and restore-disabled sessions are not auto-reopened.
|
|
858
882
|
- For a still-live browser after tab drift or resume, the wrapper verifies/selects the intended tab before ref/semantic helpers and page commands; failed selection stops the call without navigating. Local commands, read <url>, URL a11y/vitals, diff url, window new, and explicit tab/navigation/connection/state recovery do not require the prior tab. Batch checks follow effective rows past non-page prefixes and stop at explicit context changes, preserving caller argv/stdin and continue-on-error behavior. Same-tab reselection is avoided because it clears refs. Use exact batch --bail for fail-fast, not --bail=<value>. Routine same-session calls skip tab-list preflights.
|
|
859
883
|
- For sessions with observed tab-drift risk, after a successful command on a known target tab, agent_browser also best-effort restores that intended tab if a restored/background tab steals focus after the command completes. Routine same-session commands skip focus correction; observed pages without a native targetId use tab list to establish their identity.
|
|
860
884
|
- If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
|
|
861
885
|
- If upstream reports tab_gone, the pinned bound tab is gone; use details.nextActions (tab list / tab new) instead of assuming another tab is yours.
|
|
886
|
+
|
|
862
887
|
<!-- agent-browser-playbook:end wrapper-tab-recovery -->
|
|
888
|
+
|
|
863
889
|
- caller-owned sessions honor native `AGENT_BROWSER_SOCKET_DIR` unless the wrapper-specific socket override is set, using the same integrity checks; on other local Unix launches, set a short private socket directory for wrapper-spawned `agent-browser` processes so extension-generated session names do not fail the upstream Unix socket-path length limit in longer cwd/session-name combinations; require an absolute non-symlink directory owned by the current uid with mode `0700`, otherwise fail before spawn. Socket checks trust the operating environment's actual `/`, not its reported UID; all non-root ownership, permission and alias-destination checks remain in force. This is not protection from the controller of the root filesystem; see [socket trust](ARCHITECTURE.md#ownership). Android/Termux uses a short directory under the owner-only `/data/data/<package>` app sandbox, compacts generated managed identities to one 80-bit digest so ordinary namespace plus fresh-session paths remain within the limit, stores policy-lock coordination under `os.tmpdir()`, and probes process identity with Termux's `ps` beside Node instead of unavailable `/bin/ps`
|
|
864
890
|
- keep wrapper-spawned commands bounded by clamping `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented default of 25 seconds while the default wrapper child-process watchdog is 35 seconds (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides it, and top-level `timeoutMs` overrides it per call for browser CLI subprocesses). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a per-call subprocess watchdog from the requested command duration plus a small grace window. Dialog commands use `PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS` (default 5000 ms), and click/tap/find refs or tokens plus `eval --stdin` snippets whose text looks like alert/confirm/prompt/dialog triggers use `PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS` (default 8000 ms). Timed-out advanced QA or native batch calls may add `details.timeoutPartialProgress` and visible `Timeout partial progress` evidence with per-step status, retry payloads, current page title/URL, and declared artifact path checks; timed-out dialog-like commands may add dialog status/dismiss/fresh-session recovery next actions
|
|
865
891
|
- interactive or long-running upstream families such as `chat` without a prompt, `dashboard start`, `stream enable`, `trace start`, `profiler start`, `record start`, `inspect`, `install`, `upgrade`, `doctor --fix`, and `confirm-interactive` are passed through thinly but remain bounded by the same wrapper timeout/session planning rules; prefer explicit arguments, single-shot `chat <message>`, non-interactive flags like `doctor --offline --quick` or `doctor --json`, and cleanup pairs such as `dashboard stop`, `stream disable`, `trace stop`, `profiler stop`, and `record stop`
|
|
@@ -877,35 +903,38 @@ This is neither a browser transaction nor rollback. It does not coordinate human
|
|
|
877
903
|
|
|
878
904
|
The 0.7 public contract removes old multimode `agent_browser` inputs. Native CLI coverage, authentication, recordings, artifacts, diagnostics, and specialized outcomes remain available.
|
|
879
905
|
|
|
880
|
-
| Previous 0.6 input
|
|
881
|
-
|
|
|
882
|
-
| `agent_browser { args, stdin?, outputPath?, timeoutMs?, sessionMode? }` | Same compact direct shape
|
|
883
|
-
| `agent_browser { script: source }`
|
|
884
|
-
| `agent_browser { job: { steps, failFast? } }`
|
|
885
|
-
| `agent_browser { semanticAction: fields }`
|
|
886
|
-
| `agent_browser { qa: fields }`
|
|
887
|
-
| `agent_browser { electron: fields }`
|
|
888
|
-
| `agent_browser { sourceLookup: fields }`
|
|
889
|
-
| `agent_browser { networkSourceLookup: fields }`
|
|
906
|
+
| Previous 0.6 input | 0.7 replacement |
|
|
907
|
+
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
|
908
|
+
| `agent_browser { args, stdin?, outputPath?, timeoutMs?, sessionMode? }` | Same compact direct shape |
|
|
909
|
+
| `agent_browser { script: source }` | `agent_browser_code { code: source, session?, namespace?, timeoutMs?, outputPath? }`; check `.success`, not `.ok`; browser now persists |
|
|
910
|
+
| `agent_browser { job: { steps, failFast? } }` | Native `batch --bail` with argv rows; omit `--bail` only for deliberate safe continuation, or use code for branching |
|
|
911
|
+
| `agent_browser { semanticAction: fields }` | Enable `action`, then `agent_browser_action { ...fields }` |
|
|
912
|
+
| `agent_browser { qa: fields }` | Enable `qa`, then `agent_browser_qa { ...fields }` |
|
|
913
|
+
| `agent_browser { electron: fields }` | Enable `electron`, then `agent_browser_electron { ...fields }` |
|
|
914
|
+
| `agent_browser { sourceLookup: fields }` | Enable `source`, then `agent_browser_source { ...fields }` |
|
|
915
|
+
| `agent_browser { networkSourceLookup: fields }` | Enable `network`, then `agent_browser_network_source { ...fields }` |
|
|
890
916
|
|
|
891
917
|
Move former outer `outputPath`/`timeoutMs` alongside advanced flat fields where supported. Action has optional `session` but no `sessionMode`; QA/source/network retain `sessionMode`; Electron `timeoutMs` is flat and action-specific (`list` rejects it). Code chooses session/namespace in the parent.
|
|
892
918
|
|
|
893
|
-
| Former job step
|
|
894
|
-
|
|
|
895
|
-
| `open` plus `loadState`
|
|
896
|
-
| Selector `click` / `fill`
|
|
897
|
-
| Locator click/fill
|
|
898
|
-
| `select`
|
|
899
|
-
| `wait`
|
|
900
|
-
| `assertText` / `assertUrl` | `wait --text <text>` / `wait --url <url-or-glob>`
|
|
901
|
-
| `waitForDownload`
|
|
902
|
-
| `snapshot` / `screenshot`
|
|
903
|
-
| Paced `type`
|
|
919
|
+
| Former job step | Native argv rows |
|
|
920
|
+
| -------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
|
921
|
+
| `open` plus `loadState` | `open <url>`, then `wait --load <state>` |
|
|
922
|
+
| Selector `click` / `fill` | Ordinary command with selector and optional text |
|
|
923
|
+
| Locator click/fill | `find <locator> <value> <action> ...` |
|
|
924
|
+
| `select` | `select <selector> <value...>` |
|
|
925
|
+
| `wait` | `wait <milliseconds>` |
|
|
926
|
+
| `assertText` / `assertUrl` | `wait --text <text>` / `wait --url <url-or-glob>` |
|
|
927
|
+
| `waitForDownload` | `wait --download <path>` |
|
|
928
|
+
| `snapshot` / `screenshot` | `snapshot -i` / `screenshot <path>` |
|
|
929
|
+
| Paced `type` | Optional `focus <selector>`, interleaved `keyboard type <character>` / `wait <delayMs>`, optional final `press <key>` |
|
|
904
930
|
|
|
905
931
|
For exact delayed-input behavior, build native rows instead of relying on unverified native `--delay` equivalence. Keep the former 200-character pacing bound when porting that workflow. Typing `hi` with a 20 ms gap:
|
|
906
932
|
|
|
907
933
|
```json
|
|
908
|
-
{
|
|
934
|
+
{
|
|
935
|
+
"args": ["batch", "--bail"],
|
|
936
|
+
"stdin": "[[\"focus\",\"#prompt\"],[\"keyboard\",\"type\",\"h\"],[\"wait\",\"20\"],[\"keyboard\",\"type\",\"i\"]]"
|
|
937
|
+
}
|
|
909
938
|
```
|
|
910
939
|
|
|
911
940
|
Fixed waits are elapsed time, not postconditions. Preserve explicit URL/text waits, same-snapshot fill ordering, fresh refs after mutations, and partial-progress inspection. There is no legacy public script/job runtime. Exact isolated-script cleanup leases are normalized only by offline conversion so canonical replay can retire their recorded pre-upgrade resources. Saved auth and original Pi transcripts are not rewritten or deleted; [rollback](RELEASE.md#07-upgrade-and-rollback) changes the package version, not browser side effects.
|
package/docs/platform-smoke.md
CHANGED
|
@@ -32,11 +32,11 @@ npm run verify -- platform-smoke run --target ubuntu --suite platform-build
|
|
|
32
32
|
|
|
33
33
|
## Targets
|
|
34
34
|
|
|
35
|
-
| Target
|
|
36
|
-
|
|
|
37
|
-
| `macos`
|
|
38
|
-
| `ubuntu`
|
|
39
|
-
| `windows-native` | `parallels`
|
|
35
|
+
| Target | Crabbox provider | Shell contract | Release status |
|
|
36
|
+
| ---------------- | ---------------------- | -------------------------------------------------- | ------------------------------------------------------------------- |
|
|
37
|
+
| `macos` | `ssh` static localhost | POSIX shell on macOS | Required |
|
|
38
|
+
| `ubuntu` | `local-container` | POSIX shell in a Docker-compatible local container | Required |
|
|
39
|
+
| `windows-native` | `parallels` | native Windows PowerShell over OpenSSH | Nonblocking for the owner-waived 0.6.16 rollout; otherwise required |
|
|
40
40
|
|
|
41
41
|
## Alternate native transports
|
|
42
42
|
|