@youdie006/prodex 0.21.3 → 0.22.0

package/README.md

@@ -35,7 +35,7 @@ prodex ask --file src/auth.ts "Review this for security holes"
 
 `prodex ask` is the short form of `prodex pro browser ask`; the full form and every flag work identically. In an interactive terminal, `login` keeps watching the opened window and tells you exactly which manual step is still missing (log in, clear a check, open a chat) until it reports READY. If you skip `login` and the browser is not running, an interactive `ask` recovers on its own: it launches the dedicated browser, waits for your saved session to be READY, and retries the send once (disable with `--no-auto-login`; scripts opt in with `--auto-login`). While ChatGPT thinks, `prodex` prints progress to stderr (connecting, prompt sent, elapsed seconds while generating), so a multi-minute Pro answer never looks frozen.
 
-The answer prints to your terminal and is saved under `.bridge/` for later (`prodex pro latest` re-prints it). Add `--new-chat` to send into a fresh chat (recommended for repeated consults - long threads eventually confuse send detection). For a structured second-opinion debate between your coding agent and GPT Pro, `prodex pro debate-prompt --topic "..."` prints a ready-to-paste orchestration prompt. `prodex` drives the picker you can see and will not send into a tab it cannot read, so leave the dedicated window on a ChatGPT tab; it sends quietly in the background without stealing focus. Prefer no window at all? See [virtual display](#no-window-at-all-virtual-display-recommended). Pin per-repo defaults once - `prodex setup --model Pro --project "your-project"` - so every ask runs Pro (20-minute timeout) inside that project instead of whatever the ChatGPT UI last had selected; list exact sidebar project names with `prodex pro browser projects`. Pass `--file` more than once to inline several files. `--file` puts a text file's CONTENTS into the prompt; `--attach` uploads the file itself, which is the only way to hand ChatGPT a pdf, pptx, xlsx or image and let it parse the original (`prodex ask --attach deck.pptx "Review slides 40-60"`). Both are restricted to paths inside the repo, so an agent cannot upload `~/.ssh` by asking nicely. The upload happens before the prompt is submitted and prodex waits for ChatGPT to finish accepting the file - the browser process reads the path, so the file has to live on the machine running the browser. `--tool` turns on a ChatGPT composer tool for that send: `--tool deep-research` (starts a browsed research run and hands you back the thread URL - prodex does NOT return the report, because a finished deep research report does not render in prodex's browser session even after a hard reload; read it in your own browser), `--tool web-search` (current facts with sources), `--tool create-image`. Any other label the menu shows works too, so a tool ChatGPT adds later needs no prodex release. When the thread is still generating a previous answer (common right after a timed-out Pro send), the send automatically queues behind it up to the timeout budget; tune that with `--busy-wait-ms` (0 fails fast with a `response_in_progress` blocker). See [First Pro Login](#first-pro-login) for the full flow, and the [FAQ](#faq) if a send stops.
+The answer prints to your terminal and is saved under `.bridge/` for later (`prodex pro latest` re-prints it). Add `--new-chat` to send into a fresh chat (recommended for repeated consults - long threads eventually confuse send detection). For a structured second-opinion debate between your coding agent and GPT Pro, `prodex pro debate-prompt --topic "..."` prints a ready-to-paste orchestration prompt. `prodex` drives the picker you can see and will not send into a tab it cannot read, so leave the dedicated window on a ChatGPT tab; it sends quietly in the background without stealing focus. Prefer no window at all? See [virtual display](#no-window-at-all-virtual-display-recommended). Pin per-repo defaults once - `prodex setup --model Pro --project "your-project"` - so every ask runs Pro (20-minute timeout) inside that project instead of whatever the ChatGPT UI last had selected; list exact sidebar project names with `prodex pro browser projects`. Pass `--file` more than once to inline several files. `--file` puts a text file's CONTENTS into the prompt; `--attach` uploads the file itself, which is the only way to hand ChatGPT a pdf, pptx, xlsx or image and let it parse the original (`prodex ask --attach deck.pptx "Review slides 40-60"`). Both are restricted to paths inside the repo, so an agent cannot upload `~/.ssh` by asking nicely. The upload happens before the prompt is submitted and prodex waits for ChatGPT to finish accepting the file - the browser process reads the path, so the file has to live on the machine running the browser. `--tool` turns on a ChatGPT composer tool for that send: `--tool deep-research` (a browsed report - the timeout rises to 30 minutes automatically; prodex presses the start control, waits out the run and returns the full report. The report is read from the conversation transcript rather than the page, because deep research renders inside a widget iframe that leaves the thread looking empty - which also means `prodex pro browser recover --target-url <thread>` fetches a research report that finished after a timeout), `--tool web-search` (current facts with sources), `--tool create-image`. Any other label the menu shows works too, so a tool ChatGPT adds later needs no prodex release. When the thread is still generating a previous answer (common right after a timed-out Pro send), the send automatically queues behind it up to the timeout budget; tune that with `--busy-wait-ms` (0 fails fast with a `response_in_progress` blocker). See [First Pro Login](#first-pro-login) for the full flow, and the [FAQ](#faq) if a send stops.
 
 ## Core Shape
 

package/dist/chatgpt-browser.js

@@ -1672,6 +1672,30 @@ export async function recoverChatGptAnswerFromThread(options) {
         // In-tab navigation (location.assign, not Page.navigate which has crashed the
         // instance) so we read the requested thread, not whatever was open.
         await cdp.evaluate(`location.assign(${JSON.stringify(url)})`);
+        // A deep research thread has no assistant message to recover - its report
+        // lives in the widget state on the conversation transcript. Check that
+        // first so `recover` works on research threads at all.
+        const conversationId = conversationIdFromThreadUrl(url);
+        if (conversationId) {
+            try {
+                const report = await evaluateOnPage(page.page, deepResearchReportExpression(conversationId), {
+                    timeoutMs: 60_000
+                });
+                if (report.ok && report.report.trim().length > 0) {
+                    return {
+                        url,
+                        title: "",
+                        answer: report.report.trim(),
+                        modelHints: [],
+                        warnings: []
+                    };
+                }
+            }
+            catch {
+                // Not a research thread, or the transcript API is unavailable: fall
+                // through to the normal DOM recovery below.
+            }
+        }
         const deadline = Date.now() + timeoutMs;
         while (Date.now() < deadline) {
             await sleep(500);
@@ -1838,6 +1862,7 @@ export async function sendChatGptPrompt(options) {
     };
     let beforeSubmit;
     let submitButtonFound = false;
+    let wantsDeepResearch = false;
     const sendWarnings = [];
     const cdp = await connectCdp(page.webSocketDebuggerUrl);
     try {
@@ -1872,7 +1897,7 @@ export async function sendChatGptPrompt(options) {
             emitProgress("selecting", `attached ${uploaded.attached.join(", ")}`);
         }
         const toolLabels = (options.tools ?? []).map(resolveComposerToolLabel);
-        const wantsDeepResearch = toolLabels.includes(DEEP_RESEARCH_TOOL_LABEL);
+        wantsDeepResearch = toolLabels.includes(DEEP_RESEARCH_TOOL_LABEL);
         if (toolLabels.length > 0)
             emitProgress("selecting", `tools=${toolLabels.join(", ")}`);
         await insertComposerTextViaCdp(cdp, options.prompt, page, toolLabels);
@@ -1932,11 +1957,6 @@ export async function sendChatGptPrompt(options) {
                 }
                 await sleep(1_000);
             }
-            // Do not wait for a report this browser will never render.
-            const threadNow = await cdp.evaluate("location.href");
-            if (typeof threadNow === "string" && threadNow.length > 0) {
-                throw new ChatGptBrowserBlockerError(deepResearchUnreadableBlocker(threadNow));
-            }
             if (!pressed) {
                 sendWarnings.push("deep_research_start_not_found: no start control appeared for the deep research run. If ChatGPT asked a clarifying question instead, answer it with a follow-up consult in the same thread.");
             }
@@ -2009,6 +2029,48 @@ export async function sendChatGptPrompt(options) {
     // silently, with a receipt (caught live). Nothing about that is recoverable
     // after the fact, so the wait either stays on this thread or fails loudly.
     const pinnedThreadUrl = finalState?.url;
+    // Deep research never reaches the DOM answer wait below: the report is
+    // rendered by a widget app in an iframe, so the main frame stays empty even
+    // when the run has finished. Read the run out of the conversation transcript
+    // instead, which is where the widget keeps its state.
+    if (wantsDeepResearch) {
+        const conversationId = pinnedThreadUrl ? conversationIdFromThreadUrl(pinnedThreadUrl) : undefined;
+        if (!conversationId)
+            throw new ChatGptBrowserBlockerError(deepResearchUnreadableBlocker(pinnedThreadUrl ?? "https://chatgpt.com/"));
+        let lastState;
+        while (Date.now() - started < timeoutMs) {
+            try {
+                lastState = await evaluateOnPage(page, deepResearchReportExpression(conversationId), { timeoutMs: 60_000 });
+            }
+            catch {
+                // Transient CDP/network failure: keep polling until the budget runs out.
+                await sleep(5_000);
+                continue;
+            }
+            if (lastState.ok && lastState.report.trim().length > 0) {
+                emitProgress("answered", `deep research report (${lastState.chars} chars)`);
+                return {
+                    url: pinnedThreadUrl ?? "",
+                    title: finalState?.title ?? "",
+                    answer: lastState.report.trim(),
+                    modelHints: finalState?.modelHints ?? [],
+                    ...(finalState?.modelSlug ? { modelSlug: finalState.modelSlug } : {}),
+                    warnings: sendWarnings
+                };
+            }
+            emitProgress("waiting", `deep research ${lastState.status || lastState.reason} (${formatDurationMs(Date.now() - started)})`);
+            // Each poll pulls the whole transcript, which a research run grows into
+            // the hundreds of KB - so poll on a calm cadence, not a tight one.
+            await sleep(15_000);
+        }
+        throw new ChatGptBrowserBlockerError({
+            code: "deep_research_still_running",
+            message: `The deep research run was still ${lastState?.status || "in progress"} after ${formatDurationMs(timeoutMs)}.`,
+            retryable: true,
+            next_step: `Fetch the report once it finishes with \`prodex pro browser recover --target-url ${pinnedThreadUrl}\`, or read it in your browser: ${pinnedThreadUrl}`,
+            ...(pinnedThreadUrl ? { thread: pinnedThreadUrl } : {})
+        });
+    }
     let recoveredNavigations = 0;
     const answerIsStable = createChatGptAnswerStabilityTracker();
     while (Date.now() - started < timeoutMs) {
@@ -2883,23 +2945,69 @@ export function chunkComposerText(text, size = COMPOSER_INSERT_CHUNK_CHARS) {
  * (measured live), because nothing pressed it.
  */
 /**
- * Deep research reports are not readable from prodex's browser session.
- * Measured against a run the user finished themselves: the SAME thread, same
- * account, hard-reloaded here, renders only the prompt turn and an empty
- * result turn (roles ["user"], 72 characters of page) while their own browser
- * shows the completed report. Rather than spend a 30-minute budget waiting for
- * text that never arrives, prodex hands back the thread and says where to read
- * it.
+ * The report is read from the conversation transcript, which is keyed by the
+ * conversation id in the thread url. Without that id there is nothing to poll,
+ * so hand the run back rather than waiting on a page that never renders it -
+ * deep research draws into a widget iframe, leaving the thread DOM empty.
  */
 export function deepResearchUnreadableBlocker(threadUrl) {
     return {
         code: "deep_research_not_readable",
-        message: "The deep research run was started, but prodex cannot read deep research reports from its browser session - they do not render there.",
-        retryable: false,
-        next_step: `Open the run in your own browser and read it there: ${threadUrl}`,
+        message: "The deep research run was started, but prodex could not tell which conversation it landed in, so it cannot fetch the report.",
+        retryable: true,
+        next_step: `Read the run in your browser, or fetch it once it finishes with \`prodex pro browser recover --target-url ${threadUrl}\`: ${threadUrl}`,
         thread: threadUrl
     };
 }
+export function deepResearchReportExpression(conversationId) {
+    return `(async () => {
+  const fail = (reason, status) => ({ ok: false, reason, status: status || "", report: "", chars: 0 });
+  let token = "";
+  try {
+    const session = await fetch("/api/auth/session", { credentials: "include" });
+    if (!session.ok) return fail("session_http_" + session.status);
+    const parsed = await session.json();
+    token = (parsed && parsed.accessToken) || "";
+  } catch (error) {
+    return fail("session_error");
+  }
+  let conversation;
+  try {
+    const response = await fetch("/backend-api/conversation/" + ${JSON.stringify(conversationId)}, {
+      credentials: "include",
+      headers: token ? { Authorization: "Bearer " + token } : {}
+    });
+    if (!response.ok) return fail("conversation_http_" + response.status);
+    conversation = await response.json();
+  } catch (error) {
+    return fail("conversation_error");
+  }
+  const nodes = Object.keys((conversation && conversation.mapping) || {}).map((key) => conversation.mapping[key]);
+  const widgetNode = nodes.find(
+    (node) => node && node.message && node.message.metadata && node.message.metadata.chatgpt_sdk && node.message.metadata.chatgpt_sdk.widget_state
+  );
+  if (!widgetNode) return fail("no_widget_state");
+  let state;
+  try {
+    state = JSON.parse(widgetNode.message.metadata.chatgpt_sdk.widget_state);
+  } catch (error) {
+    return fail("widget_state_unparsable");
+  }
+  const status = (state && state.status) || "";
+  const parts = state && state.report_message && state.report_message.content && state.report_message.content.parts;
+  const report = Array.isArray(parts) ? parts.filter((part) => typeof part === "string").join("") : "";
+  if (!report) return fail("report_not_ready", status);
+  return { ok: true, reason: "", status, report, chars: report.length };
+})()`;
+}
+/**
+ * Thread urls come in a plain (`/c/<id>`) and a project (`/g/g-p-.../c/<id>`)
+ * shape; both end in the conversation id the backend API is keyed by.
+ */
+export function conversationIdFromThreadUrl(url) {
+    const match = /\/c\/([0-9a-fA-F-]{16,})/.exec(url);
+    return match ? match[1] : undefined;
+}
 export function deepResearchStartButtonRectExpression() {
     return `(() => {${CLICK_POINT_SNIPPET}
     const buttons = [...document.querySelectorAll('button,[role="button"]')];

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@youdie006/prodex",
-  "version": "0.21.3",
+  "version": "0.22.0",
   "description": "Local receipt bus for coordinating Codex execution with ChatGPT Pro/Projects consultation.",
   "author": "youdie006",
   "license": "MIT",