@trusty-squire/mcp 1.1.16-rc.4 → 1.1.16-rc.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -2
- package/dist/bot/act/act.d.ts +15 -1
- package/dist/bot/act/act.d.ts.map +1 -1
- package/dist/bot/act/act.js +247 -9
- package/dist/bot/act/act.js.map +1 -1
- package/dist/bot/act/identity.d.ts +64 -0
- package/dist/bot/act/identity.d.ts.map +1 -0
- package/dist/bot/act/identity.js +198 -0
- package/dist/bot/act/identity.js.map +1 -0
- package/dist/bot/act/targets.d.ts +4 -0
- package/dist/bot/act/targets.d.ts.map +1 -1
- package/dist/bot/act/targets.js +6 -0
- package/dist/bot/act/targets.js.map +1 -1
- package/dist/bot/browser.d.ts +16 -0
- package/dist/bot/browser.d.ts.map +1 -1
- package/dist/bot/browser.js +72 -14
- package/dist/bot/browser.js.map +1 -1
- package/dist/bot/capture/verification.d.ts +23 -0
- package/dist/bot/capture/verification.d.ts.map +1 -1
- package/dist/bot/capture/verification.js +54 -17
- package/dist/bot/capture/verification.js.map +1 -1
- package/dist/bot/drive-act.d.ts +32 -23
- package/dist/bot/drive-act.d.ts.map +1 -1
- package/dist/bot/drive-act.js +163 -518
- package/dist/bot/drive-act.js.map +1 -1
- package/dist/bot/drive-snapshot.d.ts +8 -0
- package/dist/bot/drive-snapshot.d.ts.map +1 -1
- package/dist/bot/drive-snapshot.js +127 -15
- package/dist/bot/drive-snapshot.js.map +1 -1
- package/dist/bot/google-login.d.ts +6 -2
- package/dist/bot/google-login.d.ts.map +1 -1
- package/dist/bot/google-login.js +15 -6
- package/dist/bot/google-login.js.map +1 -1
- package/dist/bot/operate-drive.d.ts +2 -2
- package/dist/bot/operate-drive.d.ts.map +1 -1
- package/dist/bot/operate-drive.js +90 -97
- package/dist/bot/operate-drive.js.map +1 -1
- package/dist/bot/remote-login-display.d.ts +5 -1
- package/dist/bot/remote-login-display.d.ts.map +1 -1
- package/dist/bot/remote-login-display.js +29 -1
- package/dist/bot/remote-login-display.js.map +1 -1
- package/dist/bot/session/model.d.ts +8 -6
- package/dist/bot/session/model.d.ts.map +1 -1
- package/dist/bot/session/model.js.map +1 -1
- package/dist/install/cli.d.ts.map +1 -1
- package/dist/install/cli.js +11 -5
- package/dist/install/cli.js.map +1 -1
- package/dist/pairing-ttl.d.ts +4 -0
- package/dist/pairing-ttl.d.ts.map +1 -0
- package/dist/pairing-ttl.js +14 -0
- package/dist/pairing-ttl.js.map +1 -0
- package/dist/tools/provision-drive.d.ts +2 -2
- package/dist/tools/provision-drive.d.ts.map +1 -1
- package/dist/tools/provision-drive.js +4 -0
- package/dist/tools/provision-drive.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -87,7 +87,7 @@ npx @trusty-squire/mcp connect
|
|
|
87
87
|
|
|
88
88
|
`connect` signs you in with Google or GitHub, detects your coding agent, and merges the `squire` MCP server into its existing configuration. On an enrolled machine, sign-in opens a tab in Trusty Squire's shared browser. When that browser is on the machine's own screen and you are running `connect` there, you finish sign-in in the window in front of you — no URL and no virtual display. Otherwise — the shared browser started without a screen and sits on a private Linux display, or you are running `connect` over SSH or a TTY with no screen of your own — the ceremony prints a single-use noVNC URL; open that URL on another device. The URL shows and controls the whole display that browser runs on until the ceremony ends: other sessions' tabs, and on a machine with its own screen, everything else on that screen. macOS and Windows draw their windows natively, so the shared browser's tab is on the screen in front of you there and nothing is exposed over noVNC. If the display cannot be shown, connect stops with an explanation instead of waiting indefinitely. On a first connect where no broker can serve, connect launches its own sign-in browser, using a visible desktop when one still answers or a login-scoped Xvfb + noVNC stack on headless Linux (including SSH/TTY sessions with an inherited virtual display, and a display that has stopped answering since startup).
|
|
89
89
|
|
|
90
|
-
Ceremony completion, timeout, failure, or interruption tears down its quick tunnel and local exposure helpers. The shared browser and its display remain with the broker; a self-launched ceremony also closes its own browser and display. Operators may set both `TS_LOGIN_PUBLIC_HOSTNAME` and `TS_LOGIN_LOCAL_PORT` to reuse an externally managed named tunnel, which Trusty Squire never creates or stops. If that fixed local port is busy, login reports it and uses a one-off quick tunnel instead. Restart the agent and ask for the finished website outcome. Trusty Squire is free to start.
|
|
90
|
+
Ceremony completion, timeout, failure, or interruption tears down its quick tunnel and local exposure helpers. Those helpers also hold a bound of their own — the sign-in link's lifetime plus a short grace, counted from the moment they are stood up — so a `connect` that is abandoned, wedged, or never waits still tears the display down and exits non-zero instead of leaving a tunnel published. The shared browser and its display remain with the broker; a self-launched ceremony also closes its own browser and display. Operators may set both `TS_LOGIN_PUBLIC_HOSTNAME` and `TS_LOGIN_LOCAL_PORT` to reuse an externally managed named tunnel, which Trusty Squire never creates or stops. If that fixed local port is busy, login reports it and uses a one-off quick tunnel instead. Restart the agent and ask for the finished website outcome. Trusty Squire is free to start.
|
|
91
91
|
|
|
92
92
|
`connect` is also the only way to sign in again: `--force-relogin` switches the bound account and `--force-relogin=google` or `--force-relogin=github` refreshes one provider session. It re-checks provider sessions before reporting success; when the shared browser keeps the profile busy, that check uses committed cookies, which can outlive a server-side session. Use `--force-relogin` to refresh a stale session. The detailed [provider-probe and ceremony contract](docs/browser-broker.md) describes completion and failure handling.
|
|
93
93
|
|
|
@@ -127,7 +127,7 @@ Each report carries seven fields:
|
|
|
127
127
|
- `browser_location` — where the ceremony browser actually opened, reported by the code that placed it: `{kind:"host_screen",display?}` (the screen you are at), `{kind:"virtual",url}` (a display nobody is sitting at — `url` is the noVNC address that reaches it, reported on a non-terminal line the moment the browser lands there, while the tunnel is up), `{kind:"unreachable",reason}` (nothing here could be shown the page — also the answer when a ceremony ran but no path ever reported a placement, with `reason` naming the failure), `{kind:"none"}` (no browser was opened), or `{kind:"unknown",reason}` (a browser opened that Squire did not place — `--skip-browser` hands the link to the machine's default browser). Squire decides this; callers do not detect screens.
|
|
128
128
|
- `reason` — only what the five fields above cannot say: `provider_session_missing`, `requested_provider_missing`, `account_mismatch`, `profile_unverifiable`, `install_expired`, `cached_cookie_evidence`, `run_failed`. `null` otherwise.
|
|
129
129
|
|
|
130
|
-
Connect never waits longer than the sign-in link lives — the wait is the link's lifetime counted from the moment connect received it, so a skewed machine clock cannot shorten or overrun it. A `sign_in_url` it hands back is still open; a wait that reached that deadline, or a link the server reported lapsed, is `no-browser` with `reason: "install_expired"` and no URL at all.
|
|
130
|
+
Connect never waits longer than the sign-in link lives — the wait is the link's lifetime counted from the moment connect received it, so a skewed machine clock cannot shorten or overrun it. A `sign_in_url` it hands back is still open; a wait that reached that deadline, or a link the server reported lapsed, is `no-browser` with `reason: "install_expired"` and no URL at all. A ceremony display that outlives its own bound answers the same way, on a terminal line written before connect exits — so a run that wedged before it ever reached that wait still ends the machine channel.
|
|
131
131
|
|
|
132
132
|
Supported targets: `claude-code`, `cursor`, `codex`, `opencode`, `goose`, `cline`, `continue`, and `hermes`.
|
|
133
133
|
|
|
@@ -190,6 +190,12 @@ grant; host scoping, auditing, and revocation still apply. When a grant reaches
|
|
|
190
190
|
an explicit hourly limit, the proxy returns `429 rate_limited` with
|
|
191
191
|
`scope: "grant"`, `Retry-After`, and window/reset metadata.
|
|
192
192
|
|
|
193
|
+
The proxy passes the provider's response through as it arrives and forwards the
|
|
194
|
+
upstream response headers (cookies, hop-by-hop headers, and the upstream
|
|
195
|
+
content-length removed), so a streaming completion — `stream: true`,
|
|
196
|
+
`text/event-stream` — reaches the app event by event rather than in one block
|
|
197
|
+
once generation has finished. The body is not buffered and carries no size cap.
|
|
198
|
+
|
|
193
199
|
The result contains a host-scoped egress `base_url` and a `token`, not the Clerk secret key. The token is returned once through the MCP result and remains valid until revoked. That means the scoped grant token can enter agent context; it is not the provider key. Move it directly into backend-only deployment secret storage, never browser code, logs, or source control. If you need zero grant-token exposure to the model, use `use_credential` for agent-initiated requests instead. Trusty Squire removes the grant authorization at the boundary and injects the vaulted provider credential into the upstream request.
|
|
194
200
|
|
|
195
201
|
## Security and threat model
|
package/dist/bot/act/act.d.ts
CHANGED
|
@@ -5,6 +5,18 @@ import { type ObserveDetail } from "../observe/observe.js";
|
|
|
5
5
|
import { type CompactV2TargetAuthorization } from "./targets.js";
|
|
6
6
|
import type { Observation, ProvisionAction } from "../provision-session.js";
|
|
7
7
|
import type { Session } from "../session/model.js";
|
|
8
|
+
export type ActExecutorOptions = {
|
|
9
|
+
drive?: boolean;
|
|
10
|
+
};
|
|
11
|
+
export type DriveActResult = {
|
|
12
|
+
kind: "ok";
|
|
13
|
+
combobox: boolean;
|
|
14
|
+
} | {
|
|
15
|
+
kind: "stale";
|
|
16
|
+
reason: string;
|
|
17
|
+
} | {
|
|
18
|
+
kind: "unsupported";
|
|
19
|
+
};
|
|
8
20
|
export declare function runSerializedGoogleIdentityOperation<T>(session: Session, operation: (browser: BrowserController) => Promise<T>, options?: {
|
|
9
21
|
deadline?: OAuthActionDeadline;
|
|
10
22
|
}): Promise<{
|
|
@@ -17,9 +29,11 @@ export interface InternalActResult {
|
|
|
17
29
|
outcome: {
|
|
18
30
|
selectedOption?: string;
|
|
19
31
|
};
|
|
32
|
+
combobox?: boolean;
|
|
20
33
|
}
|
|
21
|
-
export declare function actInternally(sessionId: string, action: ProvisionAction, detail?: ObserveDetail, compactV2Authorization?: CompactV2TargetAuthorization, operationPage?: Page): Promise<InternalActResult>;
|
|
34
|
+
export declare function actInternally(sessionId: string, action: ProvisionAction, detail?: ObserveDetail, compactV2Authorization?: CompactV2TargetAuthorization, operationPage?: Page, options?: ActExecutorOptions): Promise<InternalActResult>;
|
|
22
35
|
export declare function act(sessionId: string, action: ProvisionAction, detail?: ObserveDetail, outputFormat?: "compact" | "full", compactMapEmitted?: boolean): Promise<Observation>;
|
|
36
|
+
export declare function dispatchDriveAct(sessionId: string, action: ProvisionAction): Promise<DriveActResult>;
|
|
23
37
|
export declare function compactV2SelectionFailureReason(error: unknown): string;
|
|
24
38
|
export declare function settleAfterDriveAction(page?: Page, combobox?: boolean): Promise<void>;
|
|
25
39
|
export declare function settleAfterStateChange(browser: BrowserController, page?: Page, options?: {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"act.d.ts","sourceRoot":"","sources":["../../../src/bot/act/act.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,
|
|
1
|
+
{"version":3,"file":"act.d.ts","sourceRoot":"","sources":["../../../src/bot/act/act.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAS,IAAI,EAAE,MAAM,YAAY,CAAC;AAC9C,OAAO,EAIL,KAAK,iBAAiB,EAEvB,MAAM,eAAe,CAAC;AACvB,OAAO,EAWL,KAAK,mBAAmB,EAIzB,MAAM,mBAAmB,CAAC;AAY3B,OAAO,EAOL,KAAK,aAAa,EAMnB,MAAM,uBAAuB,CAAC;AAU/B,OAAO,EAiBL,KAAK,4BAA4B,EAClC,MAAM,cAAc,CAAC;AAEtB,OAAO,KAAK,EAAE,WAAW,EAAE,eAAe,EAAE,MAAM,yBAAyB,CAAC;AAC5E,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAC;AAuBnD,MAAM,MAAM,kBAAkB,GAAG;IAC/B,KAAK,CAAC,EAAE,OAAO,CAAC;CACjB,CAAC;AAIF,MAAM,MAAM,cAAc,GACtB;IAAE,IAAI,EAAE,IAAI,CAAC;IAAC,QAAQ,EAAE,OAAO,CAAA;CAAE,GACjC;IAAE,IAAI,EAAE,OAAO,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GACjC;IAAE,IAAI,EAAE,aAAa,CAAA;CAAE,CAAC;AA4E5B,wBAAsB,oCAAoC,CAAC,CAAC,EAC1D,OAAO,EAAE,OAAO,EAChB,SAAS,EAAE,CAAC,OAAO,EAAE,iBAAiB,KAAK,OAAO,CAAC,CAAC,CAAC,EACrD,OAAO,GAAE;IAAE,QAAQ,CAAC,EAAE,mBAAmB,CAAA;CAAO,GAC/C,OAAO,CAAC;IAAE,OAAO,EAAE,iBAAiB,CAAC;IAAC,MAAM,EAAE,CAAC,CAAA;CAAE,CAAC,CAOpD;AA+GD,MAAM,WAAW,iBAAiB;IAChC,WAAW,EAAE,WAAW,CAAC;IACzB,aAAa,CAAC,EAAE,IAAI,CAAC;IACrB,OAAO,EAAE;QACP,cAAc,CAAC,EAAE,MAAM,CAAC;KACzB,CAAC;IACF,QAAQ,CAAC,EAAE,OAAO,CAAC;CACpB;AAqED,wBAAsB,aAAa,CACjC,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,eAAe,EACvB,MAAM,GAAE,aAAyB,EACjC,sBAAsB,CAAC,EAAE,4BAA4B,EACrD,aAAa,CAAC,EAAE,IAAI,EACpB,OAAO,CAAC,EAAE,kBAAkB,GAC3B,OAAO,CAAC,iBAAiB,CAAC,CA4D5B;AAED,wBAAsB,GAAG,CACvB,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,eAAe,EACvB,MAAM,GAAE,aAAyB,EACjC,YAAY,GAAE,SAAS,GAAG,MAAe,EACzC,iBAAiB,UAAO,GACvB,OAAO,CAAC,WAAW,CAAC,CAwGtB;AA6vBD,wBAAsB,gBAAgB,CACpC,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,eAAe,GACtB,OAAO,CAAC,cAAc,CAAC,CA+DzB;AAED,wBAAgB,+BAA+B,CAAC,KAAK,EAAE,OAAO,GAAG,MAAM,CAEtE;AA+FD,wBAAsB,sBAAsB,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,UAAQ,GAAG,OAAO,CAAC,IAAI,CAAC,CAwCzF;AAED,wBAAsB,sBAAsB,CAC1C,OAAO,EAAE,iBAAiB,EAC1B,IAAI,CAAC,EAAE,IAAI,EACX,OAAO,CAAC,EAAE;IAAE,KAAK,CAAC,EAAE,OAAO,CAAC;IAAC,QAAQ,CAAC,EAAE,OAAO,CAAA;CAAE,GAChD,OAAO,CAAC,IAAI,CAAC,CAUf"}
|
package/dist/bot/act/act.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { BrowserClickDispatchError, clickDispatchStatusForError, } from "../browser.js";
|
|
1
|
+
import { BrowserClickDispatchError, DRIVE_DISPATCH_PACING, clickDispatchStatusForError, } from "../browser.js";
|
|
2
2
|
import { loginWithOAuth, oauthActionDeadline, oauthActionRemainingMs, oauthAutomatedActionTimeoutMs, oauthHumanHandoffTimeoutMs, oauthLoginLeaseCooldownMs, resetOAuthActionDeadline, settleAfterOAuth, withOAuthActionLease, withinOAuthActionDeadline, OAuthAwaitingHumanError, OAuthFailedError, OAuthOnboardingRequiredError, } from "../oauth-login.js";
|
|
3
3
|
import { compactV2AuditUrl, safeStageV2 } from "../compact-observation-v2.js";
|
|
4
4
|
import { ProvenPreDispatchMutationError } from "../mutation-dispatch-evidence.js";
|
|
@@ -11,6 +11,10 @@ import { substituteCardTokens } from "../card-secret-tokens.js";
|
|
|
11
11
|
import { clickScreenshot, ScreenshotClickError } from "../screenshot-click.js";
|
|
12
12
|
import { composeOperatorSignals, currentOperatorRequestSignal, operatorMutationDispatchPhase, } from "../request-cancellation.js";
|
|
13
13
|
import { AmbiguousProvisionTargetError, compactV2AuthorizationForTarget, frameTargetFor, isRequiredShippingAddressLine1, parseLocatorTarget, preparedOAuthLoginTarget, resolveAuthorizedCompactV2Target, resolveTarget, staleTargetError, TargetStaleError, throwCompactV2StaleRef, CompactV2ActionFailureError, CompactV2StaleRefError, CompactV2UnresolvedLabelError, ProvisionTargetMissingError, ProvisionTargetNotAllowedError, } from "./targets.js";
|
|
14
|
+
import { resolveIdentityScope, resolveLiveControlIdentity, } from "./identity.js";
|
|
15
|
+
import { evaluateBound } from "../drive-evaluate.js";
|
|
16
|
+
import { clickCrossOriginFrameTarget, commitDriveListOption, selectDriveOption, overlayOptionLabels, waitForOpenedOverlay, waitForOverlayOptionsToChange, } from "../drive-act.js";
|
|
17
|
+
const DRIVE_DISPATCH = { drive: true };
|
|
14
18
|
async function withOAuthActionBoundary(session, provider, run, outputFormat) {
|
|
15
19
|
if (provider === "google") {
|
|
16
20
|
const gate = await googleSessionGateForSession(session.id);
|
|
@@ -206,13 +210,18 @@ function oauthOnboardingRequiredObservation(session, error) {
|
|
|
206
210
|
};
|
|
207
211
|
return compactV2PublicObservation(session, { stage: "form", guidance, oauth, url });
|
|
208
212
|
}
|
|
209
|
-
export async function actInternally(sessionId, action, detail = "compact", compactV2Authorization, operationPage) {
|
|
213
|
+
export async function actInternally(sessionId, action, detail = "compact", compactV2Authorization, operationPage, options) {
|
|
210
214
|
const session = sessionForCall(sessionId);
|
|
211
215
|
const capturedOperationPage = operationPage ?? (session === undefined ? undefined : operationPageForSession(session));
|
|
212
216
|
const oauthProvider = action.kind === "oauth_login" || action.kind === "oauth_click" ? action.provider : undefined;
|
|
213
217
|
try {
|
|
214
218
|
const execute = async (deadline) => {
|
|
215
|
-
const run = async () =>
|
|
219
|
+
const run = async () => {
|
|
220
|
+
const act = async () => await executeAct(sessionId, action, detail, true, compactV2Authorization, deadline, capturedOperationPage, false, undefined, "full", true, options);
|
|
221
|
+
return session === undefined || options?.drive !== true
|
|
222
|
+
? await act()
|
|
223
|
+
: await session.browser.withDispatchPacing(DRIVE_DISPATCH_PACING, act);
|
|
224
|
+
};
|
|
216
225
|
return (action.kind === "click" ||
|
|
217
226
|
action.kind === "js_click" ||
|
|
218
227
|
action.kind === "oauth_login" ||
|
|
@@ -240,7 +249,7 @@ export async function actInternally(sessionId, action, detail = "compact", compa
|
|
|
240
249
|
return { observation: oauthOnboardingRequiredObservation(session, error), outcome: {} };
|
|
241
250
|
}
|
|
242
251
|
if (session?.compactV2Active === true && !(error instanceof ProvisionTargetMissingError)) {
|
|
243
|
-
throw new CompactV2ActionFailureError(compactV2ActionFailureReason(error, action.kind));
|
|
252
|
+
throw new CompactV2ActionFailureError(compactV2ActionFailureReason(error, action.kind), clickDispatchStatusForError(error));
|
|
244
253
|
}
|
|
245
254
|
throw error;
|
|
246
255
|
}
|
|
@@ -341,10 +350,90 @@ function actDriverTarget(el) {
|
|
|
341
350
|
return { kind: "frame", frame, selector: el.selector };
|
|
342
351
|
return { kind: "selector", selector: el.selector };
|
|
343
352
|
}
|
|
353
|
+
function actsThroughOverlay(el, picker) {
|
|
354
|
+
if (picker)
|
|
355
|
+
return true;
|
|
356
|
+
const role = (el.role ?? "").toLowerCase();
|
|
357
|
+
const type = (el.type ?? "").toLowerCase();
|
|
358
|
+
return (role === "combobox" ||
|
|
359
|
+
role === "searchbox" ||
|
|
360
|
+
type === "search" ||
|
|
361
|
+
type === "date" ||
|
|
362
|
+
type === "datetime-local" ||
|
|
363
|
+
type === "month");
|
|
364
|
+
}
|
|
365
|
+
function submitsOnEnter(el, ariaLabel) {
|
|
366
|
+
const role = (el.role ?? "").toLowerCase();
|
|
367
|
+
const type = (el.type ?? "").toLowerCase();
|
|
368
|
+
return (role === "searchbox" ||
|
|
369
|
+
type === "search" ||
|
|
370
|
+
el.name === "q" ||
|
|
371
|
+
/search/i.test(`${ariaLabel} ${el.placeholder ?? ""}`));
|
|
372
|
+
}
|
|
373
|
+
function scopeForElement(page, el) {
|
|
374
|
+
if (el.framePath === undefined || el.framePath === null || el.framePath.length === 0)
|
|
375
|
+
return page;
|
|
376
|
+
let frame = page.mainFrame();
|
|
377
|
+
for (const part of el.framePath.split("/")) {
|
|
378
|
+
const child = frame.childFrames()[Number(part)];
|
|
379
|
+
if (child === undefined)
|
|
380
|
+
return page;
|
|
381
|
+
frame = child;
|
|
382
|
+
}
|
|
383
|
+
return frame;
|
|
384
|
+
}
|
|
385
|
+
// The occlusion guard the drive used to carry in its own dispatch: a cookie
|
|
386
|
+
// banner or sticky footer over the target swallows the click while the page
|
|
387
|
+
// still reports a dispatch, so the caller records a step that never landed.
|
|
388
|
+
// Absent a box (still loading) it says nothing and the actionability waits rule.
|
|
389
|
+
async function clickTargetOccluded(scope, selector) {
|
|
390
|
+
return await evaluateBound(scope, (sel) => {
|
|
391
|
+
const element = document.querySelector(sel);
|
|
392
|
+
if (element === null)
|
|
393
|
+
return false;
|
|
394
|
+
// The snapshot keeps offscreen fillables so the model can name them, and
|
|
395
|
+
// the dispatch scrolls before clicking — so measure where the click will
|
|
396
|
+
// actually land, not where the element sits right now. "instant" is
|
|
397
|
+
// load-bearing: a page with scroll-behavior:smooth would animate and the
|
|
398
|
+
// rect below would still be the pre-scroll one.
|
|
399
|
+
const before = element.getBoundingClientRect();
|
|
400
|
+
const inView = before.width > 0 &&
|
|
401
|
+
before.height > 0 &&
|
|
402
|
+
before.bottom > 0 &&
|
|
403
|
+
before.top < innerHeight &&
|
|
404
|
+
before.right > 0 &&
|
|
405
|
+
before.left < innerWidth;
|
|
406
|
+
if (!inView) {
|
|
407
|
+
element.scrollIntoView({ block: "center", inline: "nearest", behavior: "instant" });
|
|
408
|
+
}
|
|
409
|
+
const rect = element.getBoundingClientRect();
|
|
410
|
+
if (rect.width <= 0 || rect.height <= 0)
|
|
411
|
+
return false;
|
|
412
|
+
const hit = document.elementFromPoint(rect.x + rect.width / 2, rect.y + rect.height / 2);
|
|
413
|
+
return hit !== null && hit !== element && !element.contains(hit) && !hit.contains(element);
|
|
414
|
+
}, selector).catch(() => false);
|
|
415
|
+
}
|
|
344
416
|
// Re-resolve against FRESH elements — never trust a stale index. Shared by the
|
|
345
417
|
// type_secret / select / click-group ref paths; `internalLabel`/`noMatchPrefix`
|
|
346
418
|
// keep each caller's error wording, `withVisibleCandidates` its candidate list.
|
|
347
419
|
async function resolveFreshActTarget(session, browser, compactV2ActionPage, compactV2Authorization, resolutionTarget, internalAccess, internalLabel, noMatchPrefix, actionTarget, withVisibleCandidates) {
|
|
420
|
+
const driveIdentity = session.drive?.identities?.get(resolutionTarget);
|
|
421
|
+
const livePage = compactV2ActionPage ?? browser.page;
|
|
422
|
+
if (driveIdentity !== undefined && livePage !== null) {
|
|
423
|
+
const live = await resolveLiveControlIdentity(livePage, resolutionTarget, driveIdentity);
|
|
424
|
+
if (live !== null) {
|
|
425
|
+
return { el: live, fresh: session.lastElements, driveIdentity };
|
|
426
|
+
}
|
|
427
|
+
if (session.compactV2Active) {
|
|
428
|
+
if (!internalAccess)
|
|
429
|
+
throwCompactV2StaleRef();
|
|
430
|
+
throw new CompactV2StaleRefError("stale_ref");
|
|
431
|
+
}
|
|
432
|
+
const stale = staleTargetError(session, actionTarget, session.lastElements);
|
|
433
|
+
if (stale !== null)
|
|
434
|
+
throw stale;
|
|
435
|
+
throw new CompactV2StaleRefError("stale_ref");
|
|
436
|
+
}
|
|
348
437
|
const fresh = (await browser.extractBrowserUseObservation(compactV2ActionPage)).elements;
|
|
349
438
|
retainSessionElements(session, fresh);
|
|
350
439
|
// resolveTarget recomputes identities (incl. volatile positional-group
|
|
@@ -354,7 +443,7 @@ async function resolveFreshActTarget(session, browser, compactV2ActionPage, comp
|
|
|
354
443
|
? resolveTarget(fresh, resolutionTarget)
|
|
355
444
|
: resolveAuthorizedCompactV2Target(session, fresh, compactV2Authorization);
|
|
356
445
|
if (el !== null)
|
|
357
|
-
return { el, fresh };
|
|
446
|
+
return { el, fresh, driveIdentity: undefined };
|
|
358
447
|
if (session.compactV2Active) {
|
|
359
448
|
if (!internalAccess)
|
|
360
449
|
throwCompactV2StaleRef();
|
|
@@ -373,15 +462,18 @@ async function resolveFreshActTarget(session, browser, compactV2ActionPage, comp
|
|
|
373
462
|
: "";
|
|
374
463
|
throw new Error(`${prefix}no element matched target "${actionTarget}".${visible}`);
|
|
375
464
|
}
|
|
376
|
-
async function executeAct(sessionId, action, detail, internalAccess, internalAuthorization, oauthDeadline, operationPage, preparedOAuthDispatch = false, onScreenshotDispatched, outputFormat = "full", compactMapEmitted = true) {
|
|
465
|
+
async function executeAct(sessionId, action, detail, internalAccess, internalAuthorization, oauthDeadline, operationPage, preparedOAuthDispatch = false, onScreenshotDispatched, outputFormat = "full", compactMapEmitted = true, options) {
|
|
377
466
|
const session = sessionForCall(sessionId);
|
|
378
467
|
if (session === undefined)
|
|
379
468
|
throw new Error(`unknown provision session ${sessionId}`);
|
|
380
469
|
let browser = session.browser;
|
|
381
470
|
const compactV2ActionPage = operationPage ?? operationPageForSession(session);
|
|
382
471
|
const driveSettle = session.drive !== null && action.kind !== "oauth_login" && action.kind !== "oauth_click";
|
|
472
|
+
const skipToolSettle = options?.drive === true;
|
|
383
473
|
let settleMs = 0;
|
|
384
474
|
const settle = async (combobox = false) => {
|
|
475
|
+
if (skipToolSettle)
|
|
476
|
+
return;
|
|
385
477
|
const started = Date.now();
|
|
386
478
|
await settleAfterStateChange(browser, compactV2ActionPage, {
|
|
387
479
|
drive: driveSettle,
|
|
@@ -390,6 +482,7 @@ async function executeAct(sessionId, action, detail, internalAccess, internalAut
|
|
|
390
482
|
settleMs += Date.now() - started;
|
|
391
483
|
};
|
|
392
484
|
const actStarted = Date.now();
|
|
485
|
+
let actedCombobox = false;
|
|
393
486
|
let actionPageAfter = compactV2ActionPage;
|
|
394
487
|
let completedAction = action;
|
|
395
488
|
let resolutionTarget;
|
|
@@ -529,7 +622,10 @@ async function executeAct(sessionId, action, detail, internalAccess, internalAut
|
|
|
529
622
|
}
|
|
530
623
|
case "select": {
|
|
531
624
|
const el = (await resolveFreshActTarget(session, browser, compactV2ActionPage, compactV2Authorization, resolutionTarget, internalAccess, "select", "select", action.target, true)).el;
|
|
532
|
-
const
|
|
625
|
+
const driveSelectPage = compactV2ActionPage ?? browser.page;
|
|
626
|
+
const committedText = options?.drive === true && driveSelectPage !== null && driveSelectPage !== undefined
|
|
627
|
+
? await selectDriveOptionOrRefuse(scopeForElement(driveSelectPage, el), el.selector, action.text ?? "")
|
|
628
|
+
: await browser.select(actDriverTarget(el), action.text, compactV2ActionPage);
|
|
533
629
|
session.committedSelectValues.set(compactV2CommittedSelectKey(session, el.selector), compactV2CommittedSelectValue(session, committedText));
|
|
534
630
|
completedAction = { ...action, text: committedText };
|
|
535
631
|
await settle();
|
|
@@ -629,18 +725,79 @@ async function executeAct(sessionId, action, detail, internalAccess, internalAut
|
|
|
629
725
|
break;
|
|
630
726
|
}
|
|
631
727
|
// Re-resolve against FRESH elements every act — never trust a stale index.
|
|
632
|
-
const { el, fresh } = await resolveFreshActTarget(session, browser, compactV2ActionPage, compactV2Authorization, resolutionTarget, internalAccess, action.kind, "", action.target, true);
|
|
728
|
+
const { el, fresh, driveIdentity } = await resolveFreshActTarget(session, browser, compactV2ActionPage, compactV2Authorization, resolutionTarget, internalAccess, action.kind, "", action.target, true);
|
|
729
|
+
actedCombobox = actsThroughOverlay(el, driveIdentity?.picker === true);
|
|
730
|
+
const ariaLabelAttribute = driveIdentity === undefined ? (el.ariaLabel ?? "") : (driveIdentity.ariaLabel ?? "");
|
|
633
731
|
// Preserve frame identity (origin + path) for the frame-scoped fill.
|
|
634
732
|
if (action.kind === "click" || action.kind === "js_click") {
|
|
733
|
+
const clickPage = compactV2ActionPage ?? browser.page;
|
|
734
|
+
// Only the drive's extra dispatch arms consume a frame scope; the
|
|
735
|
+
// tools reach a frame through actDriverTarget's frame target, so
|
|
736
|
+
// walking the positional frame path for them would be dead work.
|
|
737
|
+
const clickScope = options?.drive !== true || clickPage === null || clickPage === undefined
|
|
738
|
+
? undefined
|
|
739
|
+
: scopeForElement(clickPage, el);
|
|
740
|
+
if (action.kind === "click" &&
|
|
741
|
+
clickScope !== undefined &&
|
|
742
|
+
(await clickTargetOccluded(clickScope, el.selector))) {
|
|
743
|
+
throw new BrowserClickDispatchError("not_dispatched", "click target is occluded by an overlay");
|
|
744
|
+
}
|
|
745
|
+
// One adoption arm/grace around whichever dispatch runs: an option row
|
|
746
|
+
// or a cross-origin coordinate click can open a target=_blank tab
|
|
747
|
+
// just as an ordinary click can, and paying the grace per candidate
|
|
748
|
+
// would charge every click for the branches that declined.
|
|
749
|
+
const commitsListOption = action.kind === "click" &&
|
|
750
|
+
clickPage !== null &&
|
|
751
|
+
clickPage !== undefined &&
|
|
752
|
+
clickScope !== undefined;
|
|
753
|
+
const crossOriginScope = action.kind === "click" && clickScope !== undefined && "parentFrame" in clickScope
|
|
754
|
+
? clickScope
|
|
755
|
+
: undefined;
|
|
635
756
|
actionPageAfter =
|
|
636
757
|
(await adoptTabOpenedByClick(session, browser, async () => {
|
|
758
|
+
if (commitsListOption &&
|
|
759
|
+
clickPage !== null &&
|
|
760
|
+
clickPage !== undefined &&
|
|
761
|
+
clickScope !== undefined &&
|
|
762
|
+
(await commitDriveListOption(clickPage, clickScope, el.selector))) {
|
|
763
|
+
return;
|
|
764
|
+
}
|
|
765
|
+
if (crossOriginScope !== undefined &&
|
|
766
|
+
clickPage !== null &&
|
|
767
|
+
clickPage !== undefined &&
|
|
768
|
+
(await clickCrossOriginFrameTarget(clickPage, crossOriginScope, el.selector))) {
|
|
769
|
+
return;
|
|
770
|
+
}
|
|
637
771
|
await actClick({ ...actDriverTarget(el), method: action.kind });
|
|
638
772
|
})) ?? actionPageAfter;
|
|
639
773
|
}
|
|
640
774
|
else if (action.kind === "type") {
|
|
641
775
|
clearCommittedSelectValue(session, el.selector);
|
|
642
776
|
const actTarget = actDriverTarget(el);
|
|
643
|
-
|
|
777
|
+
if (options?.drive === true && actedCombobox && compactV2ActionPage !== undefined) {
|
|
778
|
+
// The click may remount the field into an overlay that takes focus,
|
|
779
|
+
// so the text goes to whatever is focused — after an explicit
|
|
780
|
+
// select-all, because insertText alone APPENDS to a committed value.
|
|
781
|
+
// The suggestion baseline is read AFTER the overlay opens; reading
|
|
782
|
+
// it before would make the refresh wait return on the stale rows.
|
|
783
|
+
await actClick({ ...actTarget, method: "click" });
|
|
784
|
+
await waitForOpenedOverlay(compactV2ActionPage).catch(() => undefined);
|
|
785
|
+
const overlayBefore = await overlayOptionLabels(compactV2ActionPage);
|
|
786
|
+
await compactV2ActionPage.keyboard.press("ControlOrMeta+a");
|
|
787
|
+
await compactV2ActionPage.keyboard.insertText(typedText ?? "");
|
|
788
|
+
await waitForOverlayOptionsToChange(compactV2ActionPage, overlayBefore);
|
|
789
|
+
if (submitsOnEnter(el, ariaLabelAttribute)) {
|
|
790
|
+
await compactV2ActionPage.keyboard.press("Enter").catch(() => undefined);
|
|
791
|
+
}
|
|
792
|
+
}
|
|
793
|
+
else {
|
|
794
|
+
await actType(actTarget, typedText, false);
|
|
795
|
+
if (options?.drive === true && submitsOnEnter(el, ariaLabelAttribute)) {
|
|
796
|
+
await (compactV2ActionPage ?? browser.page)?.keyboard
|
|
797
|
+
.press("Enter")
|
|
798
|
+
.catch(() => undefined);
|
|
799
|
+
}
|
|
800
|
+
}
|
|
644
801
|
// #635 fix (not a gate on typing): Shopify only enables delivery-rate
|
|
645
802
|
// selection after the required address line is committed by
|
|
646
803
|
// blur/change, not merely after the raw keystrokes land.
|
|
@@ -755,6 +912,7 @@ async function executeAct(sessionId, action, detail, internalAccess, internalAut
|
|
|
755
912
|
if (session.drive !== null) {
|
|
756
913
|
const observeMs = Date.now() - observeStarted;
|
|
757
914
|
session.drive.lastActProfile = {
|
|
915
|
+
...session.drive.lastActProfile,
|
|
758
916
|
act_ms: Math.max(0, Date.now() - actStarted - settleMs - observeMs),
|
|
759
917
|
settle_ms: settleMs,
|
|
760
918
|
observe_ms: observeMs,
|
|
@@ -782,8 +940,88 @@ async function executeAct(sessionId, action, detail, internalAccess, internalAut
|
|
|
782
940
|
outcome: {
|
|
783
941
|
...(completedAction.kind === "select" ? { selectedOption: completedAction.text } : {}),
|
|
784
942
|
},
|
|
943
|
+
combobox: actedCombobox,
|
|
785
944
|
};
|
|
786
945
|
}
|
|
946
|
+
async function selectDriveOptionOrRefuse(scope, selector, text) {
|
|
947
|
+
const { outcome, committed } = await selectDriveOption(scope, selector, text);
|
|
948
|
+
if (outcome === "ok")
|
|
949
|
+
return committed;
|
|
950
|
+
if (outcome === "missing")
|
|
951
|
+
throw new CompactV2StaleRefError("stale_ref");
|
|
952
|
+
throw new CompactV2ActionFailureError(outcome);
|
|
953
|
+
}
|
|
954
|
+
async function guardDriveOauthTarget(sessionId, target) {
|
|
955
|
+
const session = sessionForCall(sessionId);
|
|
956
|
+
const identity = session?.drive?.identities?.get(target);
|
|
957
|
+
const page = session?.browser.page ?? null;
|
|
958
|
+
if (session === undefined || identity === undefined || page === null) {
|
|
959
|
+
return { kind: "unsupported" };
|
|
960
|
+
}
|
|
961
|
+
const scope = await resolveIdentityScope(page, target, identity);
|
|
962
|
+
if (scope === null)
|
|
963
|
+
return { kind: "stale", reason: "stale_ref" };
|
|
964
|
+
if (await clickTargetOccluded(scope, identity.selector)) {
|
|
965
|
+
return { kind: "stale", reason: "occluded" };
|
|
966
|
+
}
|
|
967
|
+
return { kind: "unsupported" };
|
|
968
|
+
}
|
|
969
|
+
export async function dispatchDriveAct(sessionId, action) {
|
|
970
|
+
// "unsupported" means the shared executor has no drive verb for this action
|
|
971
|
+
// and the caller should fall back to the tools path. A DISPATCH failure is a
|
|
972
|
+
// different thing — the act did not land on the control, which is stale.
|
|
973
|
+
if (action.kind !== "click" &&
|
|
974
|
+
action.kind !== "type" &&
|
|
975
|
+
action.kind !== "select" &&
|
|
976
|
+
action.kind !== "scroll") {
|
|
977
|
+
// The tools' oauth_login clicks by coordinate after a visibility wait only,
|
|
978
|
+
// so a banner over the provider button costs the whole OAuth deadline. Hand
|
|
979
|
+
// the drive the same refusal its own guard gave, and fall through to the
|
|
980
|
+
// tools path only when the target is clean.
|
|
981
|
+
if (action.kind === "oauth_login")
|
|
982
|
+
return await guardDriveOauthTarget(sessionId, action.target);
|
|
983
|
+
return { kind: "unsupported" };
|
|
984
|
+
}
|
|
985
|
+
// A drive ref names the identity its own snapshot recorded. A target with no
|
|
986
|
+
// such record never came from a drive snapshot — a row from a fallback
|
|
987
|
+
// observation the drive did not capture — so it is the tools' to resolve.
|
|
988
|
+
// Dispatching it here would fail resolution and report a live control stale.
|
|
989
|
+
if (action.kind !== "scroll" &&
|
|
990
|
+
sessionForCall(sessionId)?.drive?.identities?.get(action.target) === undefined) {
|
|
991
|
+
return { kind: "unsupported" };
|
|
992
|
+
}
|
|
993
|
+
try {
|
|
994
|
+
const result = await actInternally(sessionId, action, "none", undefined, undefined, DRIVE_DISPATCH);
|
|
995
|
+
return { kind: "ok", combobox: result.combobox === true };
|
|
996
|
+
}
|
|
997
|
+
catch (error) {
|
|
998
|
+
if (error instanceof CompactV2StaleRefError)
|
|
999
|
+
return { kind: "stale", reason: "stale_ref" };
|
|
1000
|
+
if (error instanceof TargetStaleError)
|
|
1001
|
+
return { kind: "stale", reason: "stale" };
|
|
1002
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
1003
|
+
// A tracked click that reached the element and only then failed did land —
|
|
1004
|
+
// telling the drive it never executed invites a second submit. The evidence
|
|
1005
|
+
// is a CLICK's own dispatch listener, so it says nothing about a type whose
|
|
1006
|
+
// focusing click landed before the keystrokes were written.
|
|
1007
|
+
const dispatchStatus = error instanceof CompactV2ActionFailureError
|
|
1008
|
+
? error.dispatchStatus
|
|
1009
|
+
: clickDispatchStatusForError(error);
|
|
1010
|
+
if (action.kind === "click" && dispatchStatus === "dispatched") {
|
|
1011
|
+
return { kind: "ok", combobox: false };
|
|
1012
|
+
}
|
|
1013
|
+
// A select that could not take the value was refused by the CONTROL; the
|
|
1014
|
+
// drive must not blame the ref and retire it. A ref that went stale before
|
|
1015
|
+
// the select was ever attempted stays stale.
|
|
1016
|
+
if (action.kind === "select" && !/stale_ref|reobserve_required/i.test(message)) {
|
|
1017
|
+
return { kind: "stale", reason: message === "not_select" ? "not_select" : "option_missing" };
|
|
1018
|
+
}
|
|
1019
|
+
return {
|
|
1020
|
+
kind: "stale",
|
|
1021
|
+
reason: /occluded|intercepts pointer/i.test(message) ? "occluded" : "stale_ref",
|
|
1022
|
+
};
|
|
1023
|
+
}
|
|
1024
|
+
}
|
|
787
1025
|
export function compactV2SelectionFailureReason(error) {
|
|
788
1026
|
return compactV2ActionFailureReason(error, "select");
|
|
789
1027
|
}
|