@53able/pi-agent-browser 0.4.0 → 0.4.1

package/README.ja-JP.md

@@ -164,12 +164,12 @@ https://example.com をブラウザ経由で読み、outputs/browser/example.md
 すべての操作を無人で任せるべきではありません。適切なタイミングで人間に操作を戻すためのツールが3つあります。
 
 - **安全な境界線** — `agent_browser_handoff` は、AI自身では対応できない・すべきでない操作（2FA/OTPの入力、CAPTCHA、本人確認）の直前でエージェントを一時停止させます。スクリーンショットで状況を記録したあと、実際のUIダイアログでブロックし、人間が「対応完了」または「中止」を選ぶまで待ちます。`agent_browser_read` / `agent_browser_snapshot` も簡易なキーワード検知を行い、こうしたチェックポイントらしき内容を出力に警告として付加します（この警告自体は停止ではなく、`agent_browser_handoff` を呼ぶきっかけとして扱ってください）。
-- **インタラクティブなログイン** — `agent_browser_login_handoff` は、Google や OAuth など多くのプロバイダで sign-in がブロックされる問題を解決します。これは、agent-browser が使われた時点でオートメーション検出が起きるため、ログイン中に拒否されるというもの。このツールは CDP が接続されていない無人 Chrome プロセスを起動し、人間が正常にログインできるようにしてから、ログイン成功後に `agent-browser` を接続します。Google や OAuth の sign-in フローが期待されるタスクでは、`agent_browser_open` ではなく必ずこちらを使ってください。
+- **インタラクティブなログイン** — `agent_browser_login_handoff` は、Google や OAuth など多くのプロバイダで sign-in がブロックされる問題を解決します。これは、agent-browser が使われた時点でオートメーション検出が起きるため、ログイン中に拒否されるというもの。このツールは CDP が接続されていない無人 Chrome プロセスを起動して人間が正常にログインできるようにし、ログイン成功後にその認証済みCookie/storageを取り出して、新しい通常のagent-browserセッションに読み込ませます。Google や OAuth の sign-in フローが期待されるタスクでは、`agent_browser_open` ではなく必ずこちらを使ってください。
 - **Commit前の停止** — `agent_browser_commit` は、決済・削除・投稿・送信といった「やり直しのきかない操作」専用の確認ゲートです。証跡を保存し、クリックの前に必ず人間の明示的な確認を求め、承認されない限りクリックは実行されません。対話的なUIが利用できない場合は、クリックせずにそのまま拒否します。こうした操作の最終確認クリックには `agent_browser_click` ではなく必ずこちらを使ってください。
 
 3つのツールすべてが対話的なセッション（TUI、またはUI対応のRPC）を必要とします。無人・headless実行では、黙って進めるのではなく拒否します。
 
-`agent_browser_open` と `agent_browser_login_handoff` は、各セッションが`headed: true`で起動されたかどうかも記録します。`agent_browser_handoff`が、headedで開かれていないセッションに対して呼ばれた場合、人間への警告に加えて、可視ウィンドウを開き直すための`agent_browser_open`の引数（セッション名、`headed: true`、`restore: true`、最後に開いていたURL）をそのまま提示します — どのコマンドを打ち直せばよいか推測する必要はありません。ログインや2FA、CAPTCHA、本人確認に差し掛かる可能性があるタスクでは、最初から`headed: true`で開くことを推奨します。一部の確認チャレンジは使い切り・期限付きのため、途中で開き直すとログインからやり直しになることがあります。
+`agent_browser_open`は、各セッションが`headed: true`で起動されたかどうかを記録します。`agent_browser_handoff`が、headedで開かれていないセッションに対して呼ばれた場合、人間への警告に加えて、可視ウィンドウを開き直すための`agent_browser_open`の引数（セッション名、`headed: true`、`restore: true`、最後に開いていたURL）をそのまま提示します — どのコマンドを打ち直せばよいか推測する必要はありません。ログインや2FA、CAPTCHA、本人確認に差し掛かる可能性があるタスクでは、最初から`headed: true`で開くことを推奨します。一部の確認チャレンジは使い切り・期限付きのため、途中で開き直すとログインからやり直しになることがあります。`agent_browser_login_handoff`が作るセッションは常にheadlessです（使い捨てのログイン用ブラウザではなく、新規セッションのため）。見たい場合は`headed: true`で開き直してください。
 
 ## Troubleshooting
 

package/README.md

@@ -164,12 +164,12 @@ Clear all saved states only when intentionally resetting everything:
 Not every step should run unattended. Three tools hand control back to a human at the right moments:
 
 - **Safe boundaries** — `agent_browser_handoff` pauses the agent for steps it cannot or should not perform itself: 2FA/OTP entry, CAPTCHA, identity verification. It captures a screenshot for context, then blocks on a real confirmation dialog until a human resumes or aborts. `agent_browser_read` and `agent_browser_snapshot` also run a lightweight keyword scan and flag likely checkpoints in their output as a nudge to call this tool.
-- **Interactive login flows** — `agent_browser_login_handoff` solves the problem that Google, OAuth, and similar providers block sign-in when agent-browser is used, because automation detection happens during the login step itself. This tool spawns an unattended Chrome process (with no CDP attached) so the human can log in normally, then attaches `agent-browser` after login succeeds. Use this instead of `agent_browser_open` whenever a task is expected to hit an interactive Google or OAuth sign-in flow.
+- **Interactive login flows** — `agent_browser_login_handoff` solves the problem that Google, OAuth, and similar providers block sign-in when agent-browser is used, because automation detection happens during the login step itself. This tool spawns an unattended Chrome process (with no CDP attached) so the human can log in normally, then captures the authenticated cookies/storage and loads them into a fresh, ordinary agent-browser session after login succeeds. Use this instead of `agent_browser_open` whenever a task is expected to hit an interactive Google or OAuth sign-in flow.
 - **Stop before commit** — `agent_browser_commit` is a dedicated gate for irreversible actions: payment, deletion, publishing, sending. It captures evidence, requires explicit human confirmation before clicking, and refuses outright (without clicking) if no interactive UI is available to ask. Use it instead of `agent_browser_click` for the final confirming step of any such action.
 
 All three tools require an interactive session (TUI or RPC with UI support); in headless/unattended runs they refuse rather than proceed silently.
 
-`agent_browser_open` and `agent_browser_login_handoff` both remember whether each named session was launched with `headed: true`. If `agent_browser_handoff` is called for a session that was not opened headed, it automatically warns the human and includes the exact `agent_browser_open` arguments (session name, `headed: true`, `restore: true`, and the last known URL) needed to reopen a visible window — no more guessing which command to rerun. Prefer `headed: true` up front for any task that might hit a login, 2FA, CAPTCHA, or identity-verification step, since some challenges are single-use and reopening mid-flow may require restarting the login.
+`agent_browser_open` remembers whether each named session was launched with `headed: true`. If `agent_browser_handoff` is called for a session that was not opened headed, it automatically warns the human and includes the exact `agent_browser_open` arguments (session name, `headed: true`, `restore: true`, and the last known URL) needed to reopen a visible window — no more guessing which command to rerun. Prefer `headed: true` up front for any task that might hit a login, 2FA, CAPTCHA, or identity-verification step, since some challenges are single-use and reopening mid-flow may require restarting the login. `agent_browser_login_handoff` always leaves the resulting session headless (it's a fresh session, not the disposable login browser), so reopen with `headed: true` if you need to see it.
 
 ## Troubleshooting
 

package/index.ts

@@ -764,7 +764,7 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
   pi.registerTool({
     name: "agent_browser_login_handoff",
     label: "Agent Browser Login Handoff",
-    description: "Launch a real, non-CDP-attached Chrome for an interactive login (e.g. Google sign-in) that automation frameworks get blocked from, then attach agent-browser to the resulting session once the human confirms login is complete. Blocks until the human confirms or aborts.",
+    description: "Launch a real, non-CDP-attached Chrome for an interactive login (e.g. Google sign-in) that automation frameworks get blocked from. Once the human confirms login is complete, captures the authenticated cookies/storage and loads them into a fresh, ordinary agent-browser session. Blocks until the human confirms or aborts.",
     promptSnippet: "Use agent_browser_login_handoff (not agent_browser_open) whenever a task requires an interactive Google/OAuth-style login.",
     promptGuidelines: [
       "Never attempt an interactive Google/OAuth-style login via agent_browser_open, agent_browser_click, or agent_browser_fill — those attach CDP immediately, and Google blocks sign-in for CDP-attached sessions regardless of profile or cookies.",
@@ -889,15 +889,42 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
         };
       }
 
-      const attachArgs = ["--cdp", String(port), "--session", session, "open"];
-      const result = await runAgentBrowser(attachArgs, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
+      // Do NOT keep driving the live --cdp connection as the ongoing session: `agent-browser
+      // --cdp <port> ... open` (no url) opens a brand-new blank page/context in the connected
+      // browser rather than reusing the tab the human actually authenticated in, leaving the
+      // "attached" session on about:blank with no auth. Instead follow agent-browser's own
+      // documented pattern for this exact scenario (see its authentication.md "Import Auth
+      // from Your Browser" / "Two-Factor Authentication" sections): extract the authenticated
+      // cookies/storage from the login browser via `state save`, close the disposable login
+      // browser, then load that state into a normal, freshly-launched session via `--state`.
+      const authStatePath = join(profileDir, "auth-state.json");
+      const saveArgs = ["--cdp", String(port), "state", "save", authStatePath];
+      const saveResult = await runAgentBrowser(saveArgs, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
+      try { child.kill(); } catch { /* best-effort; the login browser is disposable once we've extracted its auth state */ }
+
+      if (!saveResult.ok) {
+        return {
+          content: [{
+            type: "text",
+            text: [
+              "Login browser was confirmed by the human, but extracting its authenticated state via `state save` failed. Do not assume the session is usable.",
+              `Command: ${saveResult.command} ${saveResult.args.map((a) => JSON.stringify(a)).join(" ")}`,
+            ].join("\n"),
+          }],
+          details: { ...saveResult, artifactPaths: [] },
+          terminate: true,
+        };
+      }
+
+      const loadArgs = ["--session", session, "--restore", "--state", authStatePath, "open", url];
+      const result = await runAgentBrowser(loadArgs, { signal, timeoutMs: params.timeoutMs as number | undefined, cwd: ctx.cwd, onUpdate });
 
       if (!result.ok) {
         return {
           content: [{
             type: "text",
             text: [
-              "Login browser was confirmed by the human, but attaching agent-browser via --cdp failed. Do not assume the session is usable.",
+              "Login state was captured from the human's browser, but loading it into a fresh agent-browser session failed. Do not assume the session is usable.",
               `Command: ${result.command} ${result.args.map((a) => JSON.stringify(a)).join(" ")}`,
             ].join("\n"),
           }],
@@ -906,13 +933,13 @@ export default function agentBrowserExtension(pi: ExtensionAPI) {
         };
       }
 
-      await writeSessionState(ctx.cwd, session, { headed: true, lastUrl: url });
+      await writeSessionState(ctx.cwd, session, { headed: false, lastUrl: url });
 
       return {
         content: [{
           type: "text",
           text: [
-            "Login handoff complete and agent-browser is now attached to the human's authenticated Chrome session.",
+            "Login handoff complete: the human's authenticated cookies/storage were captured and loaded into a fresh agent-browser session.",
             `Session: ${session === DEFAULT_SESSION_KEY ? "(default)" : session}`,
             "Take a fresh agent_browser_snapshot before continuing.",
           ].join("\n"),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@53able/pi-agent-browser",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "Typed Pi tools for browser automation, rendered-page context extraction, screenshots, and saved session state via agent-browser.",
   "type": "module",
   "license": "MIT",

package/skills/agent-browser/SKILL.md

@@ -44,7 +44,7 @@ Two tools exist so the agent stops and hands control back to a human instead of
 
 `agent_browser_login_handoff` solves a specific problem: Google and similar OAuth providers block sign-in when `agent-browser open` is used, because agent-browser attaches a remote debugging protocol (CDP) session the instant Chrome launches, and Google detects this automation marker and rejects the login attempt regardless of profile, cookies, or executable.
 
-This tool spawns a plain, un-instrumented Chrome process (not through `agent-browser open`) with a remote debugging port and a dedicated persistent profile, so no CDP client is attached while the human logs in. Chrome behaves like an ordinary browser to Google during sign-in. Only after the human confirms login is complete does the tool attach `agent-browser` via CDP to that already-authenticated browser for all subsequent automated snapshot, read, and click operations.
+This tool spawns a plain, un-instrumented Chrome process (not through `agent-browser open`) with a remote debugging port and a dedicated persistent profile, so no CDP client is attached while the human logs in. Chrome behaves like an ordinary browser to Google during sign-in. After the human confirms login is complete, the tool captures the authenticated cookies and storage from that browser (`agent-browser state save`), closes the disposable login browser, and loads that captured state into a fresh, ordinary agent-browser session (`agent-browser state load`) for all subsequent automated snapshot, read, and click operations. (An earlier version tried to keep driving the login browser live over CDP; that opened a disconnected blank tab instead of reusing the authenticated one, so state capture-and-load is used instead.)
 
 **Important:** this is an architecture fix, not detection evasion. The tool does not hide webdriver flags or automation fingerprints — it simply avoids having a CDP session attached during the interactive login moment itself.
 
@@ -60,7 +60,7 @@ Example call:
 }
 ```
 
-After the human confirms login is complete, subsequent `agent_browser_open`, `agent_browser_snapshot`, `agent_browser_click`, and other calls with the same `session` name will reuse the authenticated browser state and operate normally. The session is marked as headed in the session-state tracking, so later calls to `agent_browser_handoff` on the same session won't false-flag it as headless.
+After the human confirms login is complete, subsequent `agent_browser_open`, `agent_browser_snapshot`, `agent_browser_click`, and other calls with the same `session` name will reuse the authenticated browser state and operate normally. This new session runs headless by default (it is a fresh agent-browser session, not the disposable login browser, which is closed once its auth state is captured); reopen it with `headed: true` if you need to see it.
 
 ## Common examples