@53able/pi-agent-browser 0.4.1 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ja-JP.md +5 -3
- package/README.md +5 -3
- package/index.ts +100 -39
- package/package.json +1 -1
- package/skills/agent-browser/SKILL.md +8 -2
package/README.ja-JP.md
CHANGED
|
@@ -164,12 +164,12 @@ https://example.com をブラウザ経由で読み、outputs/browser/example.md
|
|
|
164
164
|
すべての操作を無人で任せるべきではありません。適切なタイミングで人間に操作を戻すためのツールが3つあります。
|
|
165
165
|
|
|
166
166
|
- **安全な境界線** — `agent_browser_handoff` は、AI自身では対応できない・すべきでない操作(2FA/OTPの入力、CAPTCHA、本人確認)の直前でエージェントを一時停止させます。スクリーンショットで状況を記録したあと、実際のUIダイアログでブロックし、人間が「対応完了」または「中止」を選ぶまで待ちます。`agent_browser_read` / `agent_browser_snapshot` も簡易なキーワード検知を行い、こうしたチェックポイントらしき内容を出力に警告として付加します(この警告自体は停止ではなく、`agent_browser_handoff` を呼ぶきっかけとして扱ってください)。
|
|
167
|
-
- **インタラクティブなログイン** — `agent_browser_login_handoff` は、Google や OAuth など多くのプロバイダで sign-in がブロックされる問題を解決します。これは、agent-browser が使われた時点でオートメーション検出が起きるため、ログイン中に拒否されるというもの。このツールは CDP が接続されていない無人 Chrome
|
|
167
|
+
- **インタラクティブなログイン** — `agent_browser_login_handoff` は、Google や OAuth など多くのプロバイダで sign-in がブロックされる問題を解決します。これは、agent-browser が使われた時点でオートメーション検出が起きるため、ログイン中に拒否されるというもの。このツールは CDP が接続されていない無人 Chrome プロセスを起動して人間が正常にログインできるようにし、ログイン成功後に、**そのブラウザはセッションに結合され、閉じられるまで生存し続けます**—リモートデバッグポートが開いたまま表示されます。Google や OAuth の sign-in フローが期待されるタスクでは、`agent_browser_open` ではなく必ずこちらを使ってください。
|
|
168
168
|
- **Commit前の停止** — `agent_browser_commit` は、決済・削除・投稿・送信といった「やり直しのきかない操作」専用の確認ゲートです。証跡を保存し、クリックの前に必ず人間の明示的な確認を求め、承認されない限りクリックは実行されません。対話的なUIが利用できない場合は、クリックせずにそのまま拒否します。こうした操作の最終確認クリックには `agent_browser_click` ではなく必ずこちらを使ってください。
|
|
169
169
|
|
|
170
170
|
3つのツールすべてが対話的なセッション(TUI、またはUI対応のRPC)を必要とします。無人・headless実行では、黙って進めるのではなく拒否します。
|
|
171
171
|
|
|
172
|
-
`agent_browser_open`は、各セッションが`headed: true`で起動されたかどうかを記録します。`agent_browser_handoff`が、headedで開かれていないセッションに対して呼ばれた場合、人間への警告に加えて、可視ウィンドウを開き直すための`agent_browser_open`の引数(セッション名、`headed: true`、`restore: true`、最後に開いていたURL)をそのまま提示します — どのコマンドを打ち直せばよいか推測する必要はありません。ログインや2FA、CAPTCHA、本人確認に差し掛かる可能性があるタスクでは、最初から`headed: true`で開くことを推奨します。一部の確認チャレンジは使い切り・期限付きのため、途中で開き直すとログインからやり直しになることがあります。`agent_browser_login_handoff
|
|
172
|
+
`agent_browser_open`は、各セッションが`headed: true`で起動されたかどうかを記録します。`agent_browser_handoff`が、headedで開かれていないセッションに対して呼ばれた場合、人間への警告に加えて、可視ウィンドウを開き直すための`agent_browser_open`の引数(セッション名、`headed: true`、`restore: true`、最後に開いていたURL)をそのまま提示します — どのコマンドを打ち直せばよいか推測する必要はありません。ログインや2FA、CAPTCHA、本人確認に差し掛かる可能性があるタスクでは、最初から`headed: true`で開くことを推奨します。一部の確認チャレンジは使い切り・期限付きのため、途中で開き直すとログインからやり直しになることがあります。`agent_browser_login_handoff`が作るセッションは`headed: true`として記録されます(ログインブラウザが表示・生存し続けるため、開き直す必要はありません)。
|
|
173
173
|
|
|
174
174
|
## Troubleshooting
|
|
175
175
|
|
|
@@ -177,7 +177,9 @@ https://example.com をブラウザ経由で読み、outputs/browser/example.md
|
|
|
177
177
|
agent-browser doctor
|
|
178
178
|
```
|
|
179
179
|
|
|
180
|
-
または Pi に `agent_browser_doctor`
|
|
180
|
+
または Pi に `agent_browser_doctor` を実行させてください。インストール、Chrome、daemon、security、provider、network、launch test を確認します。`agent_browser_login_handoff` で作ったセッションでまだ開いているログインブラウザも報告し、期限切れ追跡データを自動削除します。
|
|
181
|
+
|
|
182
|
+
`agent_browser_login_handoff` で作ったセッションの中で、まだ生存しているログインブラウザがあれば、`agent_browser_doctor` がそれらを報告します(セッション名、プロセス ID、デバッグポート、最後に開いていた URL を表示)。終わったら各セッションに対して `agent_browser_close` を呼んでください — これらのセッションはブラウザを閉じるまでリモートデバッグポートが開いたままです。期限切れエントリ(実行中でなくなったプロセスの追跡データ)は、何も通知することなく自動削除されます。
|
|
181
183
|
|
|
182
184
|
よくある修正:
|
|
183
185
|
|
package/README.md
CHANGED
|
@@ -164,12 +164,12 @@ Clear all saved states only when intentionally resetting everything:
|
|
|
164
164
|
Not every step should run unattended. Three tools hand control back to a human at the right moments:
|
|
165
165
|
|
|
166
166
|
- **Safe boundaries** — `agent_browser_handoff` pauses the agent for steps it cannot or should not perform itself: 2FA/OTP entry, CAPTCHA, identity verification. It captures a screenshot for context, then blocks on a real confirmation dialog until a human resumes or aborts. `agent_browser_read` and `agent_browser_snapshot` also run a lightweight keyword scan and flag likely checkpoints in their output as a nudge to call this tool.
|
|
167
|
-
- **Interactive login flows** — `agent_browser_login_handoff` solves the problem that Google, OAuth, and similar providers block sign-in when agent-browser is used, because automation detection happens during the login step itself. This tool spawns an unattended Chrome process (with no CDP attached) so the human can log in normally,
|
|
167
|
+
- **Interactive login flows** — `agent_browser_login_handoff` solves the problem that Google, OAuth, and similar providers block sign-in when agent-browser is used, because automation detection happens during the login step itself. This tool spawns an unattended Chrome process (with no CDP attached) so the human can log in normally. After login succeeds, the still-running browser is live-attached to the named session and **kept open as the permanent backing browser**—it remains visible on screen with an open remote-debugging port until closed with `agent_browser_close`. Use this instead of `agent_browser_open` whenever a task is expected to hit an interactive Google or OAuth sign-in flow.
|
|
168
168
|
- **Stop before commit** — `agent_browser_commit` is a dedicated gate for irreversible actions: payment, deletion, publishing, sending. It captures evidence, requires explicit human confirmation before clicking, and refuses outright (without clicking) if no interactive UI is available to ask. Use it instead of `agent_browser_click` for the final confirming step of any such action.
|
|
169
169
|
|
|
170
170
|
All three tools require an interactive session (TUI or RPC with UI support); in headless/unattended runs they refuse rather than proceed silently.
|
|
171
171
|
|
|
172
|
-
`agent_browser_open` remembers whether each named session was launched with `headed: true`. If `agent_browser_handoff` is called for a session that was not opened headed, it automatically warns the human and includes the exact `agent_browser_open` arguments (session name, `headed: true`, `restore: true`, and the last known URL) needed to reopen a visible window — no more guessing which command to rerun. Prefer `headed: true` up front for any task that might hit a login, 2FA, CAPTCHA, or identity-verification step, since some challenges are single-use and reopening mid-flow may require restarting the login. `agent_browser_login_handoff`
|
|
172
|
+
`agent_browser_open` remembers whether each named session was launched with `headed: true`. If `agent_browser_handoff` is called for a session that was not opened headed, it automatically warns the human and includes the exact `agent_browser_open` arguments (session name, `headed: true`, `restore: true`, and the last known URL) needed to reopen a visible window — no more guessing which command to rerun. Prefer `headed: true` up front for any task that might hit a login, 2FA, CAPTCHA, or identity-verification step, since some challenges are single-use and reopening mid-flow may require restarting the login. `agent_browser_login_handoff` records the resulting session as `headed: true` (since the login browser is visible and running); you don't need to reopen it to see it.
|
|
173
173
|
|
|
174
174
|
## Troubleshooting
|
|
175
175
|
|
|
@@ -179,7 +179,9 @@ Run:
|
|
|
179
179
|
agent-browser doctor
|
|
180
180
|
```
|
|
181
181
|
|
|
182
|
-
Or ask Pi to run `agent_browser_doctor`.
|
|
182
|
+
Or ask Pi to run `agent_browser_doctor`. This verifies installation, Chrome, daemon, security, providers, network, and launch tests. It also reports any live login-handoff browsers left open and cleans up stale tracking entries.
|
|
183
|
+
|
|
184
|
+
If you have sessions created with `agent_browser_login_handoff`, `agent_browser_doctor` will report any that still have live login browsers running, showing the session name, process ID, debugging port, and last known URL. Call `agent_browser_close` for each session when done — these hold an open remote-debugging port until closed. Stale entries (tracking data for processes no longer running) are cleaned up automatically with no notification.
|
|
183
185
|
|
|
184
186
|
Common fixes:
|
|
185
187
|
|
package/index.ts
CHANGED
|
@@ -43,7 +43,9 @@ const SESSION_STATE_DIR = ".pi-agent-browser";
|
|
|
43
43
|
const SESSION_STATE_FILE = "sessions.json";
|
|
44
44
|
const DEFAULT_SESSION_KEY = "__default__";
|
|
45
45
|
|
|
46
|
-
type SessionState = { headed: boolean; lastUrl?: string; updatedAt: string };
|
|
46
|
+
type SessionState = { headed: boolean; lastUrl?: string; loginBrowserPid?: number; loginDebugPort?: number; updatedAt: string };
|
|
47
|
+
|
|
48
|
+
type SessionStatePatch = Partial<Omit<SessionState, "updatedAt">> & { clearLoginBrowserPid?: boolean };
|
|
47
49
|
|
|
48
50
|
function sessionKey(params: JsonObject): string {
|
|
49
51
|
return typeof params.session === "string" && params.session.trim() ? params.session.trim() : DEFAULT_SESSION_KEY;
|
|
@@ -58,7 +60,7 @@ async function readSessionStates(cwd: string): Promise<Record<string, SessionSta
|
|
|
58
60
|
}
|
|
59
61
|
}
|
|
60
62
|
|
|
61
|
-
async function writeSessionState(cwd: string, session: string, patch:
|
|
63
|
+
async function writeSessionState(cwd: string, session: string, patch: SessionStatePatch): Promise<void> {
|
|
62
64
|
const dir = join(cwd, SESSION_STATE_DIR);
|
|
63
65
|
await mkdir(dir, { recursive: true });
|
|
64
66
|
const states = await readSessionStates(cwd);
|
|
@@ -66,6 +68,8 @@ async function writeSessionState(cwd: string, session: string, patch: Partial<Om
|
|
|
66
68
|
states[session] = {
|
|
67
69
|
headed: patch.headed ?? previous?.headed ?? false,
|
|
68
70
|
lastUrl: patch.lastUrl ?? previous?.lastUrl,
|
|
71
|
+
loginBrowserPid: patch.clearLoginBrowserPid ? undefined : (patch.loginBrowserPid ?? previous?.loginBrowserPid),
|
|
72
|
+
loginDebugPort: patch.clearLoginBrowserPid ? undefined : (patch.loginDebugPort ?? previous?.loginDebugPort),
|
|
69
73
|
updatedAt: new Date().toISOString(),
|
|
70
74
|
};
|
|
71
75
|
await writeFile(join(dir, SESSION_STATE_FILE), JSON.stringify(states, null, 2), "utf8");
|
|
@@ -266,6 +270,20 @@ async function findFreePort(): Promise<number> {
|
|
|
266
270
|
});
|
|
267
271
|
}
|
|
268
272
|
|
|
273
|
+
function killPidBestEffort(pid?: number): void {
|
|
274
|
+
if (!pid) return;
|
|
275
|
+
try { process.kill(pid); } catch { /* already gone / not ours */ }
|
|
276
|
+
}
|
|
277
|
+
|
|
278
|
+
function isPidAlive(pid: number): boolean {
|
|
279
|
+
try {
|
|
280
|
+
process.kill(pid, 0); // signal 0 = no-op existence/permission check, does not actually kill
|
|
281
|
+
return true;
|
|
282
|
+
} catch (err) {
|
|
283
|
+
return (err as NodeJS.ErrnoException).code !== "ESRCH"; // ESRCH = no such process (dead); any other error (e.g. EPERM) means it exists
|
|
284
|
+
}
|
|
285
|
+
}
|
|
286
|
+
|
|
269
287
|
function abortableDelay(ms: number, signal?: AbortSignal): Promise<void> {
|
|
270
288
|
return new Promise((resolveDelay) => {
|
|
271
289
|
if (signal?.aborted) {
|
|
@@ -673,12 +691,28 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
|
|
|
673
691
|
parameters: CloseSchema,
|
|
674
692
|
async execute(_id, rawParams, signal, onUpdate, ctx) {
|
|
675
693
|
const params = rawParams as JsonObject;
|
|
694
|
+
const states = await readSessionStates(ctx.cwd);
|
|
676
695
|
const args: string[] = [];
|
|
677
696
|
pushCommonArgs(args, params);
|
|
678
697
|
args.push("close");
|
|
679
698
|
if (params.all === true) args.push("--all");
|
|
680
699
|
pushExtraArgs(args, params);
|
|
681
700
|
const result = await runAgentBrowser(args, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
|
|
701
|
+
|
|
702
|
+
// agent-browser close only detaches an externally-spawned login Chrome; kill the tracked pid(s) ourselves.
|
|
703
|
+
if (params.all === true) {
|
|
704
|
+
for (const [name, st] of Object.entries(states)) {
|
|
705
|
+
if (!st.loginBrowserPid) continue;
|
|
706
|
+
killPidBestEffort(st.loginBrowserPid);
|
|
707
|
+
await writeSessionState(ctx.cwd, name, { clearLoginBrowserPid: true });
|
|
708
|
+
}
|
|
709
|
+
} else {
|
|
710
|
+
const session = sessionKey(params);
|
|
711
|
+
if (states[session]?.loginBrowserPid) {
|
|
712
|
+
killPidBestEffort(states[session]?.loginBrowserPid);
|
|
713
|
+
await writeSessionState(ctx.cwd, session, { clearLoginBrowserPid: true });
|
|
714
|
+
}
|
|
715
|
+
}
|
|
682
716
|
return buildTextResult(result);
|
|
683
717
|
},
|
|
684
718
|
});
|
|
@@ -764,12 +798,13 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
|
|
|
764
798
|
pi.registerTool({
|
|
765
799
|
name: "agent_browser_login_handoff",
|
|
766
800
|
label: "Agent Browser Login Handoff",
|
|
767
|
-
description: "Launch a real, non-CDP-attached Chrome for an interactive login (e.g. Google sign-in) that automation frameworks get blocked from. Once the human confirms login is complete,
|
|
801
|
+
description: "Launch a real, non-CDP-attached Chrome for an interactive login (e.g. Google sign-in) that automation frameworks get blocked from. Once the human confirms login is complete, live-attaches that still-running browser to the named agent-browser session and keeps it open as the session's backing browser (no cookie/state export). Blocks until the human confirms or aborts.",
|
|
768
802
|
promptSnippet: "Use agent_browser_login_handoff (not agent_browser_open) whenever a task requires an interactive Google/OAuth-style login.",
|
|
769
803
|
promptGuidelines: [
|
|
770
804
|
"Never attempt an interactive Google/OAuth-style login via agent_browser_open, agent_browser_click, or agent_browser_fill — those attach CDP immediately, and Google blocks sign-in for CDP-attached sessions regardless of profile or cookies.",
|
|
771
805
|
"Call this tool instead whenever a task is expected to require signing into a Google account or a similarly automation-hostile login flow.",
|
|
772
806
|
"After this tool reports success, take a fresh agent_browser_snapshot before continuing — do not assume page state.",
|
|
807
|
+
"On success the login browser stays open and IS the session's live backing browser; drive it with the normal agent_browser_* tools by session name, and call agent_browser_close when done to shut it down (it holds an open remote-debugging port until then).",
|
|
773
808
|
"If this tool reports no interactive UI, a decline, or a failure, stop the task and report back; do not retry via agent_browser_open.",
|
|
774
809
|
],
|
|
775
810
|
parameters: LoginHandoffSchema,
|
|
@@ -805,6 +840,10 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
|
|
|
805
840
|
|
|
806
841
|
const port = await findFreePort();
|
|
807
842
|
|
|
843
|
+
// Supersede an older live login browser for this session so re-running doesn't leak it.
|
|
844
|
+
const prior = (await readSessionStates(ctx.cwd))[session];
|
|
845
|
+
killPidBestEffort(prior?.loginBrowserPid);
|
|
846
|
+
|
|
808
847
|
let spawnError: Error | undefined;
|
|
809
848
|
const child = spawn(executablePath, [
|
|
810
849
|
`--remote-debugging-port=${port}`,
|
|
@@ -889,42 +928,27 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
|
|
|
889
928
|
};
|
|
890
929
|
}
|
|
891
930
|
|
|
892
|
-
//
|
|
893
|
-
//
|
|
894
|
-
//
|
|
895
|
-
//
|
|
896
|
-
|
|
897
|
-
//
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
const authStatePath = join(profileDir, "auth-state.json");
|
|
901
|
-
const saveArgs = ["--cdp", String(port), "state", "save", authStatePath];
|
|
902
|
-
const saveResult = await runAgentBrowser(saveArgs, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
|
|
903
|
-
try { child.kill(); } catch { /* best-effort; the login browser is disposable once we've extracted its auth state */ }
|
|
904
|
-
|
|
905
|
-
if (!saveResult.ok) {
|
|
906
|
-
return {
|
|
907
|
-
content: [{
|
|
908
|
-
type: "text",
|
|
909
|
-
text: [
|
|
910
|
-
"Login browser was confirmed by the human, but extracting its authenticated state via `state save` failed. Do not assume the session is usable.",
|
|
911
|
-
`Command: ${saveResult.command} ${saveResult.args.map((a) => JSON.stringify(a)).join(" ")}`,
|
|
912
|
-
].join("\n"),
|
|
913
|
-
}],
|
|
914
|
-
details: { ...saveResult, artifactPaths: [] },
|
|
915
|
-
terminate: true,
|
|
916
|
-
};
|
|
917
|
-
}
|
|
931
|
+
// Human confirmed login. Bind the STILL-RUNNING login Chrome into the named session
|
|
932
|
+
// via CDP, WITHOUT navigating (snapshot binds + verifies in one shot; open would
|
|
933
|
+
// re-navigate away from the authenticated landing page). Keep the browser ALIVE —
|
|
934
|
+
// it is now the backing browser this session drives from here on.
|
|
935
|
+
|
|
936
|
+
// Defensive: surface multi-tab situations (OAuth popups) without failing.
|
|
937
|
+
const tabArgs = ["--cdp", String(port), "tab", "list"];
|
|
938
|
+
const tabResult = await runAgentBrowser(tabArgs, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
|
|
918
939
|
|
|
919
|
-
const
|
|
920
|
-
const result = await runAgentBrowser(
|
|
940
|
+
const bindArgs = ["--session", session, "--cdp", String(port), "snapshot"];
|
|
941
|
+
const result = await runAgentBrowser(bindArgs, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
|
|
921
942
|
|
|
922
943
|
if (!result.ok) {
|
|
944
|
+
// Bind failed: the login browser is still alive but unusable by the session.
|
|
945
|
+
// Best-effort kill so we don't leak a debug-port Chrome, and report.
|
|
946
|
+
killPidBestEffort(child.pid);
|
|
923
947
|
return {
|
|
924
948
|
content: [{
|
|
925
949
|
type: "text",
|
|
926
950
|
text: [
|
|
927
|
-
"
|
|
951
|
+
"Human confirmed login, but attaching the live browser to the agent-browser session failed. Do not assume the session is usable.",
|
|
928
952
|
`Command: ${result.command} ${result.args.map((a) => JSON.stringify(a)).join(" ")}`,
|
|
929
953
|
].join("\n"),
|
|
930
954
|
}],
|
|
@@ -933,16 +957,25 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
|
|
|
933
957
|
};
|
|
934
958
|
}
|
|
935
959
|
|
|
936
|
-
|
|
960
|
+
// Persist backing-browser identity + the fact that a VISIBLE window exists (headed:true).
|
|
961
|
+
await writeSessionState(ctx.cwd, session, {
|
|
962
|
+
headed: true,
|
|
963
|
+
lastUrl: url,
|
|
964
|
+
loginBrowserPid: child.pid,
|
|
965
|
+
loginDebugPort: port,
|
|
966
|
+
});
|
|
937
967
|
|
|
968
|
+
const multiTab = /\bt2\b/.test(tabResult.stdout) || (tabResult.stdout.match(/\[t\d+\]/g)?.length ?? 0) > 1;
|
|
938
969
|
return {
|
|
939
970
|
content: [{
|
|
940
971
|
type: "text",
|
|
941
972
|
text: [
|
|
942
|
-
"Login handoff complete: the
|
|
973
|
+
"Login handoff complete: the browser you signed in with is now live-attached to the agent-browser session and stays open.",
|
|
943
974
|
`Session: ${session === DEFAULT_SESSION_KEY ? "(default)" : session}`,
|
|
944
|
-
|
|
945
|
-
|
|
975
|
+
multiTab ? `Note: multiple tabs are open — verify the active tab before acting:\n${tabResult.stdout.trim()}` : undefined,
|
|
976
|
+
"The session drives this real browser window directly. Take a fresh agent_browser_snapshot before continuing.",
|
|
977
|
+
"This login browser has an open remote-debugging port and will stay open until agent_browser_close is called for this session (or the window is closed manually).",
|
|
978
|
+
].filter(Boolean).join("\n"),
|
|
946
979
|
}],
|
|
947
980
|
details: { ...result, artifactPaths: [] },
|
|
948
981
|
terminate: true,
|
|
@@ -1028,15 +1061,43 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
|
|
|
1028
1061
|
pi.registerTool({
|
|
1029
1062
|
name: "agent_browser_doctor",
|
|
1030
1063
|
label: "Agent Browser Doctor",
|
|
1031
|
-
description: "Run agent-browser doctor to verify installation, Chrome, daemon, security, providers, network, and launch tests.",
|
|
1032
|
-
promptSnippet: "Use agent_browser_doctor when browser automation fails or before relying on agent-browser in a workflow.",
|
|
1064
|
+
description: "Run agent-browser doctor to verify installation, Chrome, daemon, security, providers, network, and launch tests. Also reports and cleans up tracked login-handoff browsers (see agent_browser_login_handoff).",
|
|
1065
|
+
promptSnippet: "Use agent_browser_doctor when browser automation fails or before relying on agent-browser in a workflow. It also flags any live login-handoff browsers left open and cleans up stale tracking entries.",
|
|
1033
1066
|
parameters: DoctorSchema,
|
|
1034
1067
|
async execute(_id, rawParams, signal, onUpdate, ctx) {
|
|
1035
1068
|
const params = rawParams as JsonObject;
|
|
1036
1069
|
const args = ["doctor"];
|
|
1037
1070
|
if (params.fix === true) args.push("--fix");
|
|
1038
1071
|
const result = await runAgentBrowser(args, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
|
|
1039
|
-
|
|
1072
|
+
const textResult = buildTextResult(result);
|
|
1073
|
+
|
|
1074
|
+
const states = await readSessionStates(ctx.cwd);
|
|
1075
|
+
const liveBrowsers: { session: string; pid: number; port: number | undefined; lastUrl?: string }[] = [];
|
|
1076
|
+
for (const [session, state] of Object.entries(states)) {
|
|
1077
|
+
if (!state.loginBrowserPid) continue;
|
|
1078
|
+
if (!isPidAlive(state.loginBrowserPid)) {
|
|
1079
|
+
await writeSessionState(ctx.cwd, session, { clearLoginBrowserPid: true });
|
|
1080
|
+
continue;
|
|
1081
|
+
}
|
|
1082
|
+
liveBrowsers.push({
|
|
1083
|
+
session: session === DEFAULT_SESSION_KEY ? "(default)" : session,
|
|
1084
|
+
pid: state.loginBrowserPid,
|
|
1085
|
+
port: state.loginDebugPort,
|
|
1086
|
+
lastUrl: state.lastUrl,
|
|
1087
|
+
});
|
|
1088
|
+
}
|
|
1089
|
+
|
|
1090
|
+
if (liveBrowsers.length > 0) {
|
|
1091
|
+
const lines = [
|
|
1092
|
+
"",
|
|
1093
|
+
"Live login browsers (from agent_browser_login_handoff) still open:",
|
|
1094
|
+
...liveBrowsers.map((b) => `- ${b.session}: pid ${b.pid}, debug port ${b.port ?? "?"}${b.lastUrl ? `, last url ${b.lastUrl}` : ""}`),
|
|
1095
|
+
"Call agent_browser_close for each session when done — these hold an open remote-debugging port until closed.",
|
|
1096
|
+
];
|
|
1097
|
+
textResult.content[0].text += lines.join("\n");
|
|
1098
|
+
}
|
|
1099
|
+
|
|
1100
|
+
return textResult;
|
|
1040
1101
|
},
|
|
1041
1102
|
});
|
|
1042
1103
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@53able/pi-agent-browser",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "Typed Pi tools for browser automation, rendered-page context extraction, screenshots, and saved session state via agent-browser.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -44,7 +44,7 @@ Two tools exist so the agent stops and hands control back to a human instead of
|
|
|
44
44
|
|
|
45
45
|
`agent_browser_login_handoff` solves a specific problem: Google and similar OAuth providers block sign-in when `agent-browser open` is used, because agent-browser attaches a remote debugging protocol (CDP) session the instant Chrome launches, and Google detects this automation marker and rejects the login attempt regardless of profile, cookies, or executable.
|
|
46
46
|
|
|
47
|
-
This tool spawns a plain, un-instrumented Chrome process (not through `agent-browser open`) with a remote debugging port and a dedicated persistent profile, so no CDP client is attached while the human logs in. Chrome behaves like an ordinary browser to Google during sign-in. After the human confirms login is complete, the
|
|
47
|
+
This tool spawns a plain, un-instrumented Chrome process (not through `agent-browser open`) with a remote debugging port and a dedicated persistent profile, so no CDP client is attached while the human logs in. Chrome behaves like an ordinary browser to Google during sign-in. After the human confirms login is complete, the still-running login browser is **live-attached to the named agent-browser session** (via `agent-browser --session <name> --cdp <port> snapshot`, not `open`, to avoid navigating away from the authenticated page) and **kept open as the permanent backing browser for that session**. All subsequent automated snapshot, read, click, and other operations continue to work against it via `--session <name>` alone, unchanged.
|
|
48
48
|
|
|
49
49
|
**Important:** this is an architecture fix, not detection evasion. The tool does not hide webdriver flags or automation fingerprints — it simply avoids having a CDP session attached during the interactive login moment itself.
|
|
50
50
|
|
|
@@ -60,7 +60,13 @@ Example call:
|
|
|
60
60
|
}
|
|
61
61
|
```
|
|
62
62
|
|
|
63
|
-
After the human confirms login is complete, subsequent `agent_browser_open`, `agent_browser_snapshot`, `agent_browser_click`, and other calls with the same `session` name will reuse the authenticated browser
|
|
63
|
+
After the human confirms login is complete, subsequent `agent_browser_open`, `agent_browser_snapshot`, `agent_browser_click`, and other calls with the same `session` name will reuse the authenticated browser and operate normally. **The login browser window remains visible and open indefinitely** — this is a real, user-visible Chrome window with an open remote-debugging port on `127.0.0.1` that any local process can access. The session is recorded as `headed: true` (since the authenticated browser is still running).
|
|
64
|
+
|
|
65
|
+
**Security note:** The login browser's `--remote-debugging-port` remains open on localhost for as long as the session is alive. This gives any local process full browser control — the same caveat agent-browser's own docs give about `--remote-debugging-port`. Call `agent_browser_close` when done with a session that went through login handoff, rather than leaving it open indefinitely. `agent_browser_close` will terminate the tracked login Chrome process and clean up its debugging port.
|
|
66
|
+
|
|
67
|
+
Run `agent_browser_doctor` to see which sessions still have live login browsers open. It reports any that are still running (session name, process ID, debugging port, and last known URL) with a reminder to call `agent_browser_close`. Stale tracking entries for processes no longer running are cleaned up silently with no user notification.
|
|
68
|
+
|
|
69
|
+
Re-running `agent_browser_login_handoff` for the same session name will terminate any previous live login browser for that session before spawning a new one.
|
|
64
70
|
|
|
65
71
|
## Common examples
|
|
66
72
|
|