@youdie006/prodex 0.21.2 → 0.22.0

package/README.md

@@ -35,7 +35,7 @@ prodex ask --file src/auth.ts "Review this for security holes"
 
 `prodex ask` is the short form of `prodex pro browser ask`; the full form and every flag work identically. In an interactive terminal, `login` keeps watching the opened window and tells you exactly which manual step is still missing (log in, clear a check, open a chat) until it reports READY. If you skip `login` and the browser is not running, an interactive `ask` recovers on its own: it launches the dedicated browser, waits for your saved session to be READY, and retries the send once (disable with `--no-auto-login`; scripts opt in with `--auto-login`). While ChatGPT thinks, `prodex` prints progress to stderr (connecting, prompt sent, elapsed seconds while generating), so a multi-minute Pro answer never looks frozen.
 
-The answer prints to your terminal and is saved under `.bridge/` for later (`prodex pro latest` re-prints it). Add `--new-chat` to send into a fresh chat (recommended for repeated consults - long threads eventually confuse send detection). For a structured second-opinion debate between your coding agent and GPT Pro, `prodex pro debate-prompt --topic "..."` prints a ready-to-paste orchestration prompt. `prodex` drives the picker you can see and will not send into a tab it cannot read, so leave the dedicated window on a ChatGPT tab; it sends quietly in the background without stealing focus. Prefer no window at all? See [virtual display](#no-window-at-all-virtual-display-recommended). Pin per-repo defaults once - `prodex setup --model Pro --project "your-project"` - so every ask runs Pro (20-minute timeout) inside that project instead of whatever the ChatGPT UI last had selected; list exact sidebar project names with `prodex pro browser projects`. Pass `--file` more than once to inline several files. `--file` puts a text file's CONTENTS into the prompt; `--attach` uploads the file itself, which is the only way to hand ChatGPT a pdf, pptx, xlsx or image and let it parse the original (`prodex ask --attach deck.pptx "Review slides 40-60"`). Both are restricted to paths inside the repo, so an agent cannot upload `~/.ssh` by asking nicely. The upload happens before the prompt is submitted and prodex waits for ChatGPT to finish accepting the file - the browser process reads the path, so the file has to live on the machine running the browser. `--tool` turns on a ChatGPT composer tool for that send: `--tool deep-research` (a browsed report - the timeout rises to 30 minutes automatically, and ChatGPT often replies with a clarifying question first, which you answer with a normal follow-up in the same thread), `--tool web-search` (current facts with sources), `--tool create-image`. Any other label the menu shows works too, so a tool ChatGPT adds later needs no prodex release. When the thread is still generating a previous answer (common right after a timed-out Pro send), the send automatically queues behind it up to the timeout budget; tune that with `--busy-wait-ms` (0 fails fast with a `response_in_progress` blocker). See [First Pro Login](#first-pro-login) for the full flow, and the [FAQ](#faq) if a send stops.
+The answer prints to your terminal and is saved under `.bridge/` for later (`prodex pro latest` re-prints it). Add `--new-chat` to send into a fresh chat (recommended for repeated consults - long threads eventually confuse send detection). For a structured second-opinion debate between your coding agent and GPT Pro, `prodex pro debate-prompt --topic "..."` prints a ready-to-paste orchestration prompt. `prodex` drives the picker you can see and will not send into a tab it cannot read, so leave the dedicated window on a ChatGPT tab; it sends quietly in the background without stealing focus. Prefer no window at all? See [virtual display](#no-window-at-all-virtual-display-recommended). Pin per-repo defaults once - `prodex setup --model Pro --project "your-project"` - so every ask runs Pro (20-minute timeout) inside that project instead of whatever the ChatGPT UI last had selected; list exact sidebar project names with `prodex pro browser projects`. Pass `--file` more than once to inline several files. `--file` puts a text file's CONTENTS into the prompt; `--attach` uploads the file itself, which is the only way to hand ChatGPT a pdf, pptx, xlsx or image and let it parse the original (`prodex ask --attach deck.pptx "Review slides 40-60"`). Both are restricted to paths inside the repo, so an agent cannot upload `~/.ssh` by asking nicely. The upload happens before the prompt is submitted and prodex waits for ChatGPT to finish accepting the file - the browser process reads the path, so the file has to live on the machine running the browser. `--tool` turns on a ChatGPT composer tool for that send: `--tool deep-research` (a browsed report - the timeout rises to 30 minutes automatically; prodex presses the start control, waits out the run and returns the full report. The report is read from the conversation transcript rather than the page, because deep research renders inside a widget iframe that leaves the thread looking empty - which also means `prodex pro browser recover --target-url <thread>` fetches a research report that finished after a timeout), `--tool web-search` (current facts with sources), `--tool create-image`. Any other label the menu shows works too, so a tool ChatGPT adds later needs no prodex release. When the thread is still generating a previous answer (common right after a timed-out Pro send), the send automatically queues behind it up to the timeout budget; tune that with `--busy-wait-ms` (0 fails fast with a `response_in_progress` blocker). See [First Pro Login](#first-pro-login) for the full flow, and the [FAQ](#faq) if a send stops.
 
 ## Core Shape
 

package/dist/chatgpt-browser.js

@@ -1672,6 +1672,30 @@ export async function recoverChatGptAnswerFromThread(options) {
         // In-tab navigation (location.assign, not Page.navigate which has crashed the
         // instance) so we read the requested thread, not whatever was open.
         await cdp.evaluate(`location.assign(${JSON.stringify(url)})`);
+        // A deep research thread has no assistant message to recover - its report
+        // lives in the widget state on the conversation transcript. Check that
+        // first so `recover` works on research threads at all.
+        const conversationId = conversationIdFromThreadUrl(url);
+        if (conversationId) {
+            try {
+                const report = await evaluateOnPage(page.page, deepResearchReportExpression(conversationId), {
+                    timeoutMs: 60_000
+                });
+                if (report.ok && report.report.trim().length > 0) {
+                    return {
+                        url,
+                        title: "",
+                        answer: report.report.trim(),
+                        modelHints: [],
+                        warnings: []
+                    };
+                }
+            }
+            catch {
+                // Not a research thread, or the transcript API is unavailable: fall
+                // through to the normal DOM recovery below.
+            }
+        }
         const deadline = Date.now() + timeoutMs;
         while (Date.now() < deadline) {
             await sleep(500);
@@ -1838,6 +1862,7 @@ export async function sendChatGptPrompt(options) {
     };
     let beforeSubmit;
     let submitButtonFound = false;
+    let wantsDeepResearch = false;
     const sendWarnings = [];
     const cdp = await connectCdp(page.webSocketDebuggerUrl);
     try {
@@ -1872,7 +1897,7 @@ export async function sendChatGptPrompt(options) {
             emitProgress("selecting", `attached ${uploaded.attached.join(", ")}`);
         }
         const toolLabels = (options.tools ?? []).map(resolveComposerToolLabel);
-        const wantsDeepResearch = toolLabels.includes(DEEP_RESEARCH_TOOL_LABEL);
+        wantsDeepResearch = toolLabels.includes(DEEP_RESEARCH_TOOL_LABEL);
         if (toolLabels.length > 0)
             emitProgress("selecting", `tools=${toolLabels.join(", ")}`);
         await insertComposerTextViaCdp(cdp, options.prompt, page, toolLabels);
@@ -2004,6 +2029,48 @@ export async function sendChatGptPrompt(options) {
     // silently, with a receipt (caught live). Nothing about that is recoverable
     // after the fact, so the wait either stays on this thread or fails loudly.
     const pinnedThreadUrl = finalState?.url;
+    // Deep research never reaches the DOM answer wait below: the report is
+    // rendered by a widget app in an iframe, so the main frame stays empty even
+    // when the run has finished. Read the run out of the conversation transcript
+    // instead, which is where the widget keeps its state.
+    if (wantsDeepResearch) {
+        const conversationId = pinnedThreadUrl ? conversationIdFromThreadUrl(pinnedThreadUrl) : undefined;
+        if (!conversationId)
+            throw new ChatGptBrowserBlockerError(deepResearchUnreadableBlocker(pinnedThreadUrl ?? "https://chatgpt.com/"));
+        let lastState;
+        while (Date.now() - started < timeoutMs) {
+            try {
+                lastState = await evaluateOnPage(page, deepResearchReportExpression(conversationId), { timeoutMs: 60_000 });
+            }
+            catch {
+                // Transient CDP/network failure: keep polling until the budget runs out.
+                await sleep(5_000);
+                continue;
+            }
+            if (lastState.ok && lastState.report.trim().length > 0) {
+                emitProgress("answered", `deep research report (${lastState.chars} chars)`);
+                return {
+                    url: pinnedThreadUrl ?? "",
+                    title: finalState?.title ?? "",
+                    answer: lastState.report.trim(),
+                    modelHints: finalState?.modelHints ?? [],
+                    ...(finalState?.modelSlug ? { modelSlug: finalState.modelSlug } : {}),
+                    warnings: sendWarnings
+                };
+            }
+            emitProgress("waiting", `deep research ${lastState.status || lastState.reason} (${formatDurationMs(Date.now() - started)})`);
+            // Each poll pulls the whole transcript, which a research run grows into
+            // the hundreds of KB - so poll on a calm cadence, not a tight one.
+            await sleep(15_000);
+        }
+        throw new ChatGptBrowserBlockerError({
+            code: "deep_research_still_running",
+            message: `The deep research run was still ${lastState?.status || "in progress"} after ${formatDurationMs(timeoutMs)}.`,
+            retryable: true,
+            next_step: `Fetch the report once it finishes with \`prodex pro browser recover --target-url ${pinnedThreadUrl}\`, or read it in your browser: ${pinnedThreadUrl}`,
+            ...(pinnedThreadUrl ? { thread: pinnedThreadUrl } : {})
+        });
+    }
     let recoveredNavigations = 0;
     const answerIsStable = createChatGptAnswerStabilityTracker();
     while (Date.now() - started < timeoutMs) {
@@ -2877,6 +2944,70 @@ export function chunkComposerText(text, size = COMPOSER_INSERT_CHUNK_CHARS) {
  * on that: a run left waiting produced zero assistant messages for 30+ minutes
  * (measured live), because nothing pressed it.
  */
+/**
+ * The report is read from the conversation transcript, which is keyed by the
+ * conversation id in the thread url. Without that id there is nothing to poll,
+ * so hand the run back rather than waiting on a page that never renders it -
+ * deep research draws into a widget iframe, leaving the thread DOM empty.
+ */
+export function deepResearchUnreadableBlocker(threadUrl) {
+    return {
+        code: "deep_research_not_readable",
+        message: "The deep research run was started, but prodex could not tell which conversation it landed in, so it cannot fetch the report.",
+        retryable: true,
+        next_step: `Read the run in your browser, or fetch it once it finishes with \`prodex pro browser recover --target-url ${threadUrl}\`: ${threadUrl}`,
+        thread: threadUrl
+    };
+}
+export function deepResearchReportExpression(conversationId) {
+    return `(async () => {
+  const fail = (reason, status) => ({ ok: false, reason, status: status || "", report: "", chars: 0 });
+  let token = "";
+  try {
+    const session = await fetch("/api/auth/session", { credentials: "include" });
+    if (!session.ok) return fail("session_http_" + session.status);
+    const parsed = await session.json();
+    token = (parsed && parsed.accessToken) || "";
+  } catch (error) {
+    return fail("session_error");
+  }
+  let conversation;
+  try {
+    const response = await fetch("/backend-api/conversation/" + ${JSON.stringify(conversationId)}, {
+      credentials: "include",
+      headers: token ? { Authorization: "Bearer " + token } : {}
+    });
+    if (!response.ok) return fail("conversation_http_" + response.status);
+    conversation = await response.json();
+  } catch (error) {
+    return fail("conversation_error");
+  }
+  const nodes = Object.keys((conversation && conversation.mapping) || {}).map((key) => conversation.mapping[key]);
+  const widgetNode = nodes.find(
+    (node) => node && node.message && node.message.metadata && node.message.metadata.chatgpt_sdk && node.message.metadata.chatgpt_sdk.widget_state
+  );
+  if (!widgetNode) return fail("no_widget_state");
+  let state;
+  try {
+    state = JSON.parse(widgetNode.message.metadata.chatgpt_sdk.widget_state);
+  } catch (error) {
+    return fail("widget_state_unparsable");
+  }
+  const status = (state && state.status) || "";
+  const parts = state && state.report_message && state.report_message.content && state.report_message.content.parts;
+  const report = Array.isArray(parts) ? parts.filter((part) => typeof part === "string").join("") : "";
+  if (!report) return fail("report_not_ready", status);
+  return { ok: true, reason: "", status, report, chars: report.length };
+})()`;
+}
+/**
+ * Thread urls come in a plain (`/c/<id>`) and a project (`/g/g-p-.../c/<id>`)
+ * shape; both end in the conversation id the backend API is keyed by.
+ */
+export function conversationIdFromThreadUrl(url) {
+    const match = /\/c\/([0-9a-fA-F-]{16,})/.exec(url);
+    return match ? match[1] : undefined;
+}
 export function deepResearchStartButtonRectExpression() {
     return `(() => {${CLICK_POINT_SNIPPET}
     const buttons = [...document.querySelectorAll('button,[role="button"]')];
@@ -3002,7 +3133,17 @@ export function answerExpression() {
     });
     const assistantMessages = messages.filter((message) => message.role === "assistant");
     const userMessages = messages.filter((message) => message.role === "user");
-    const assistant = assistantMessages.at(-1);
+    // Deep research renders no assistant-role node at all: the thread is
+    // conversation-turn sections, the prompt in the first and the report in a
+    // later one (measured live - roles were ["user"] only while a research ran).
+    // Fall back to the last turn that is NOT the user's, so such an answer is
+    // readable instead of looking like "no answer" forever.
+    const turnAnswers = assistantMessages.length > 0 ? [] : [...document.querySelectorAll('[data-testid^="conversation-turn"]')]
+      .filter((turn) => !turn.querySelector('[data-message-author-role="user"]'))
+      .map((turn) => ({ role: "assistant", text: (turn.innerText || "").trim(), modelSlug: undefined }))
+      .filter((turn) => turn.text.length > 0);
+    const effectiveAssistants = assistantMessages.length > 0 ? assistantMessages : turnAnswers;
+    const assistant = effectiveAssistants.at(-1);
     const buttons = [...document.querySelectorAll('button,[role="button"]')]
       .filter((node) => !!(node.offsetWidth || node.offsetHeight || node.getClientRects().length))
       .filter((node) => !node.closest(excludedTextSelector))
@@ -3021,7 +3162,7 @@ export function answerExpression() {
       blockerScanTextSample: visibleTextOutsideMessages(blockerScanExcludedSelector).slice(0, 12000),
       visibleButtonLabels: buttons,
       generating: placeholder || Boolean(document.querySelector(${streamingSelector})) || buttons.some((label) => generatingControlPattern.test(label)),
-      assistantMessageCount: assistantMessages.length,
+      assistantMessageCount: effectiveAssistants.length,
       userMessageCount: userMessages.length,
       // ChatGPT tags each assistant message with the model that produced it -
       // the only ground truth for "did the Pro selection actually take".

package/dist/cli-pro.js

@@ -948,6 +948,10 @@ export async function runAskProCommand(rest, io) {
                 const name = selectionMetadata.project;
                 return name ? text.split(name).join("<project>") : text;
             };
+            // Where the prompt actually landed beats where the caller aimed: with
+            // --new-chat there is no target url, and a blocker that started a run
+            // still has a thread worth handing back.
+            const blockedThread = blocker.thread ?? normalizedTargetUrl;
             const persistedBlocker = {
                 ...blocker,
                 message: redactProject(blocker.message),
@@ -965,7 +969,7 @@ export async function runAskProCommand(rest, io) {
                     direction: "codex_to_chatgpt",
                     backend: "chatgpt-control",
                     task_id: task.id,
-                    thread: normalizedTargetUrl,
+                    thread: blockedThread,
                     status: "blocked",
                     blocker: persistedBlocker,
                     warnings: []
@@ -977,7 +981,7 @@ export async function runAskProCommand(rest, io) {
             // Keep stdout machine-parseable for --json consumers on the blocked
             // path too; the human-readable error still goes to stderr via throw.
             if (jsonOutput) {
-                io.stdout(JSON.stringify({ task_id: task.id, status: "blocked", thread: normalizedTargetUrl ?? null, answer: null, warnings: [], blocker }, null, 2));
+                io.stdout(JSON.stringify({ task_id: task.id, status: "blocked", thread: blockedThread ?? null, answer: null, warnings: [], blocker }, null, 2));
             }
             throw new Error(formatBlockedConsultRecordedMessage(message, task.id, sourceCli, { cwd: targetCwd }));
         }
@@ -1313,7 +1317,10 @@ export function browserSendBlockerFromError(error) {
             code: blocker.code,
             message: blocker.message,
             retryable: blocker.retryable,
-            ...("next_step" in blocker && typeof blocker.next_step === "string" ? { next_step: blocker.next_step } : {})
+            ...("next_step" in blocker && typeof blocker.next_step === "string" ? { next_step: blocker.next_step } : {}),
+            // A blocker that knows which thread the prompt landed in is the only
+            // place that URL exists - keep it so callers get a link, not prose.
+            ...("thread" in blocker && typeof blocker.thread === "string" ? { thread: blocker.thread } : {})
         };
     }
     const message = errorMessage(error);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@youdie006/prodex",
-  "version": "0.21.2",
+  "version": "0.22.0",
   "description": "Local receipt bus for coordinating Codex execution with ChatGPT Pro/Projects consultation.",
   "author": "youdie006",
   "license": "MIT",