@youdie006/prodex 0.22.0 → 0.23.1

package/README.md

@@ -35,7 +35,7 @@ prodex ask --file src/auth.ts "Review this for security holes"
 
 `prodex ask` is the short form of `prodex pro browser ask`; the full form and every flag work identically. In an interactive terminal, `login` keeps watching the opened window and tells you exactly which manual step is still missing (log in, clear a check, open a chat) until it reports READY. If you skip `login` and the browser is not running, an interactive `ask` recovers on its own: it launches the dedicated browser, waits for your saved session to be READY, and retries the send once (disable with `--no-auto-login`; scripts opt in with `--auto-login`). While ChatGPT thinks, `prodex` prints progress to stderr (connecting, prompt sent, elapsed seconds while generating), so a multi-minute Pro answer never looks frozen.
 
-The answer prints to your terminal and is saved under `.bridge/` for later (`prodex pro latest` re-prints it). Add `--new-chat` to send into a fresh chat (recommended for repeated consults - long threads eventually confuse send detection). For a structured second-opinion debate between your coding agent and GPT Pro, `prodex pro debate-prompt --topic "..."` prints a ready-to-paste orchestration prompt. `prodex` drives the picker you can see and will not send into a tab it cannot read, so leave the dedicated window on a ChatGPT tab; it sends quietly in the background without stealing focus. Prefer no window at all? See [virtual display](#no-window-at-all-virtual-display-recommended). Pin per-repo defaults once - `prodex setup --model Pro --project "your-project"` - so every ask runs Pro (20-minute timeout) inside that project instead of whatever the ChatGPT UI last had selected; list exact sidebar project names with `prodex pro browser projects`. Pass `--file` more than once to inline several files. `--file` puts a text file's CONTENTS into the prompt; `--attach` uploads the file itself, which is the only way to hand ChatGPT a pdf, pptx, xlsx or image and let it parse the original (`prodex ask --attach deck.pptx "Review slides 40-60"`). Both are restricted to paths inside the repo, so an agent cannot upload `~/.ssh` by asking nicely. The upload happens before the prompt is submitted and prodex waits for ChatGPT to finish accepting the file - the browser process reads the path, so the file has to live on the machine running the browser. `--tool` turns on a ChatGPT composer tool for that send: `--tool deep-research` (a browsed report - the timeout rises to 30 minutes automatically; prodex presses the start control, waits out the run and returns the full report. The report is read from the conversation transcript rather than the page, because deep research renders inside a widget iframe that leaves the thread looking empty - which also means `prodex pro browser recover --target-url <thread>` fetches a research report that finished after a timeout), `--tool web-search` (current facts with sources), `--tool create-image`. Any other label the menu shows works too, so a tool ChatGPT adds later needs no prodex release. When the thread is still generating a previous answer (common right after a timed-out Pro send), the send automatically queues behind it up to the timeout budget; tune that with `--busy-wait-ms` (0 fails fast with a `response_in_progress` blocker). See [First Pro Login](#first-pro-login) for the full flow, and the [FAQ](#faq) if a send stops.
+The answer prints to your terminal and is saved under `.bridge/` for later (`prodex pro latest` re-prints it). It is read from the conversation transcript rather than scraped off the page, so markdown tables and fenced code arrive intact, citations keep their links, and a tab that drifts to another conversation mid-wait no longer costs you the answer; the rendered page stays as a fallback. Add `--new-chat` to send into a fresh chat (recommended for repeated consults - long threads eventually confuse send detection). For a structured second-opinion debate between your coding agent and GPT Pro, `prodex pro debate-prompt --topic "..."` prints a ready-to-paste orchestration prompt. `prodex` drives the picker you can see and will not send into a tab it cannot read, so leave the dedicated window on a ChatGPT tab; it sends quietly in the background without stealing focus. Prefer no window at all? See [virtual display](#no-window-at-all-virtual-display-recommended). Pin per-repo defaults once - `prodex setup --model Pro --project "your-project"` - so every ask runs Pro (20-minute timeout) inside that project instead of whatever the ChatGPT UI last had selected; list exact sidebar project names with `prodex pro browser projects`. Pass `--file` more than once to inline several files. `--file` puts a text file's CONTENTS into the prompt; `--attach` uploads the file itself, which is the only way to hand ChatGPT a pdf, pptx, xlsx or image and let it parse the original (`prodex ask --attach deck.pptx "Review slides 40-60"`). Both are restricted to paths inside the repo, so an agent cannot upload `~/.ssh` by asking nicely. The upload happens before the prompt is submitted and prodex waits for ChatGPT to finish accepting the file - the browser process reads the path, so the file has to live on the machine running the browser. `--tool` turns on a ChatGPT composer tool for that send: `--tool deep-research` (a browsed report - the timeout rises to 30 minutes automatically; prodex presses the start control, waits out the run and returns the full report. The report is read from the conversation transcript rather than the page, because deep research renders inside a widget iframe that leaves the thread looking empty - which also means `prodex pro browser recover --target-url <thread>` fetches a research report that finished after a timeout), `--tool web-search` (current facts with sources), `--tool create-image`. Any other label the menu shows works too, so a tool ChatGPT adds later needs no prodex release. When the thread is still generating a previous answer (common right after a timed-out Pro send), the send automatically queues behind it up to the timeout budget; tune that with `--busy-wait-ms` (0 fails fast with a `response_in_progress` blocker). See [First Pro Login](#first-pro-login) for the full flow, and the [FAQ](#faq) if a send stops.
 
 ## Core Shape
 

package/dist/chatgpt-browser.js

@@ -1685,7 +1685,7 @@ export async function recoverChatGptAnswerFromThread(options) {
                     return {
                         url,
                         title: "",
-                        answer: report.report.trim(),
+                        answer: resolveTranscriptCitations(report.report, report.references).trim(),
                         modelHints: [],
                         warnings: []
                     };
@@ -1750,6 +1750,26 @@ export async function recoverChatGptAnswerFromThread(options) {
         warnings: []
     };
 }
+/**
+ * Read the answer from the conversation transcript, or undefined when it is not
+ * there yet. The transcript trails the rendered stream by a beat, so callers
+ * either poll it or fall back to the DOM text.
+ */
+async function readTranscriptAnswer(page, conversationId) {
+    let transcript;
+    try {
+        transcript = await evaluateOnPage(page, transcriptAnswerExpression(conversationId), { timeoutMs: 30_000 });
+    }
+    catch {
+        // Transcript unavailable (endpoint changed, transient failure): the DOM
+        // reader still runs, so this never blocks a send.
+        return undefined;
+    }
+    if (!transcript.ok || transcript.text.trim().length === 0)
+        return undefined;
+    const answer = resolveTranscriptCitations(transcript.text, transcript.references).trim();
+    return answer.length > 0 ? { answer, modelSlug: transcript.modelSlug } : undefined;
+}
 export async function sendChatGptPrompt(options) {
     const port = resolveCdpPort(options.port);
     const timeoutMs = options.timeoutMs ?? 90_000;
@@ -2029,6 +2049,23 @@ export async function sendChatGptPrompt(options) {
     // silently, with a receipt (caught live). Nothing about that is recoverable
     // after the fact, so the wait either stays on this thread or fails loudly.
     const pinnedThreadUrl = finalState?.url;
+    // Pin the CONVERSATION, not the tab. The transcript reader fetches by id, so
+    // it keeps working when the tab wanders off the thread - which is exactly how
+    // a finished answer was lost: the tab returned to the project page, the url
+    // still matched the pin taken before ChatGPT rewrote it, and the DOM reader
+    // sat on zero assistant messages until the budget ran out.
+    let transcriptConversationId = pinnedThreadUrl ? conversationIdFromThreadUrl(pinnedThreadUrl) : undefined;
+    const transcriptResult = (transcript) => {
+        emitProgress("answered", `transcript (${transcript.answer.length} chars)`);
+        return {
+            url: finalState?.url ?? pinnedThreadUrl ?? "",
+            title: finalState?.title ?? "",
+            answer: transcript.answer,
+            modelHints: finalState?.modelHints ?? [],
+            ...(transcript.modelSlug ? { modelSlug: transcript.modelSlug } : finalState?.modelSlug ? { modelSlug: finalState.modelSlug } : {}),
+            warnings: [...sendWarnings, modelSelectionWarning(options.model, transcript.modelSlug || finalState?.modelSlug)].filter((warning) => Boolean(warning))
+        };
+    };
     // Deep research never reaches the DOM answer wait below: the report is
     // rendered by a widget app in an iframe, so the main frame stays empty even
     // when the run has finished. Read the run out of the conversation transcript
@@ -2048,11 +2085,12 @@ export async function sendChatGptPrompt(options) {
                 continue;
             }
             if (lastState.ok && lastState.report.trim().length > 0) {
-                emitProgress("answered", `deep research report (${lastState.chars} chars)`);
+                const report = resolveTranscriptCitations(lastState.report, lastState.references).trim();
+                emitProgress("answered", `deep research report (${report.length} chars)`);
                 return {
                     url: pinnedThreadUrl ?? "",
                     title: finalState?.title ?? "",
-                    answer: lastState.report.trim(),
+                    answer: report,
                     modelHints: finalState?.modelHints ?? [],
                     ...(finalState?.modelSlug ? { modelSlug: finalState.modelSlug } : {}),
                     warnings: sendWarnings
@@ -2077,7 +2115,19 @@ export async function sendChatGptPrompt(options) {
         await sleep(1000);
         try {
             finalState = await evaluateOnPage(page, answerExpression());
-            if (pinnedThreadUrl && finalState?.url && !chatGptUrlsReferToSameTarget(finalState.url, pinnedThreadUrl)) {
+            if (!transcriptConversationId && finalState?.url)
+                transcriptConversationId = conversationIdFromThreadUrl(finalState.url);
+            // The transcript is the same data the page renders, minus the rendering:
+            // markdown instead of flattened innerText, an explicit finish state
+            // instead of caret heuristics, and the model that actually answered.
+            if (transcriptConversationId && !finalState.generating) {
+                const transcript = await readTranscriptAnswer(page, transcriptConversationId);
+                if (transcript)
+                    return transcriptResult(transcript);
+            }
+            // Only the DOM reader depends on which thread the tab is showing; once the
+            // conversation id is known, a wandering tab is harmless.
+            if (!transcriptConversationId && pinnedThreadUrl && finalState?.url && !chatGptUrlsReferToSameTarget(finalState.url, pinnedThreadUrl)) {
                 if (recoveredNavigations >= 2) {
                     throw new ChatGptBrowserBlockerError({
                         code: "thread_navigated_away",
@@ -2112,8 +2162,20 @@ export async function sendChatGptPrompt(options) {
         // mid-stream, and the streaming caret renders as a literal trailing
         // character that can outlive the stop button. The tracker requires extra
         // confirmations for caret-suspect tails (see its doc comment).
-        if (answerIsStable(finalState.answer, finalState.generating))
+        if (answerIsStable(finalState.answer, finalState.generating)) {
+            // The rendered answer settles a beat before the server transcript does.
+            // Give the transcript that beat: it carries markdown (tables and fenced
+            // code that innerText flattens) and the model that actually answered.
+            if (transcriptConversationId) {
+                for (let attempt = 0; attempt < 6; attempt += 1) {
+                    const transcript = await readTranscriptAnswer(page, transcriptConversationId);
+                    if (transcript)
+                        return transcriptResult(transcript);
+                    await sleep(1_500);
+                }
+            }
             break;
+        }
     }
     const completed = finalState;
     if (completed && hasFreshChatGptAnswer(beforeSubmit.assistantMessageCount, completed)) {
@@ -2959,9 +3021,103 @@ export function deepResearchUnreadableBlocker(threadUrl) {
         thread: threadUrl
     };
 }
+/**
+ * The transcript is the same data the UI renders, minus the rendering: it
+ * carries the answer as markdown (tables and fenced code survive, which
+ * innerText flattens), an explicit finish state, and the model that actually
+ * answered. Walk from `current_node` up the parents so a regenerated turn reads
+ * the branch the UI is on, not an abandoned sibling.
+ */
+export function transcriptAnswerExpression(conversationId) {
+    return `(async () => {
+  const fail = (reason, extra) => Object.assign({ ok: false, reason, status: "", endTurn: false, isComplete: false, text: "", modelSlug: "", references: [] }, extra || {});
+  let token = "";
+  try {
+    const session = await fetch("/api/auth/session", { credentials: "include" });
+    if (!session.ok) return fail("session_http_" + session.status);
+    const parsed = await session.json();
+    token = (parsed && parsed.accessToken) || "";
+  } catch (error) {
+    return fail("session_error");
+  }
+  let conversation;
+  try {
+    const response = await fetch("/backend-api/conversation/" + ${JSON.stringify(conversationId)}, {
+      credentials: "include",
+      headers: token ? { Authorization: "Bearer " + token } : {}
+    });
+    if (!response.ok) return fail("conversation_http_" + response.status);
+    conversation = await response.json();
+  } catch (error) {
+    return fail("conversation_error");
+  }
+  const mapping = (conversation && conversation.mapping) || {};
+  const chain = [];
+  let nodeId = conversation && conversation.current_node;
+  let guard = 0;
+  while (nodeId && mapping[nodeId] && guard < 2000) {
+    guard += 1;
+    if (mapping[nodeId].message) chain.push(mapping[nodeId].message);
+    nodeId = mapping[nodeId].parent;
+  }
+  const message = chain.find(
+    (entry) => entry && entry.author && entry.author.role === "assistant" && entry.content && entry.content.content_type === "text"
+  );
+  if (!message) return fail("no_assistant_message");
+  const parts = (message.content.parts || []).filter((part) => typeof part === "string");
+  const text = parts.join("");
+  const metadata = message.metadata || {};
+  const state = {
+    status: message.status || "",
+    endTurn: message.end_turn === true,
+    isComplete: metadata.is_complete === true,
+    text,
+    modelSlug: metadata.model_slug || "",
+    references: Array.isArray(metadata.content_references) ? metadata.content_references : []
+  };
+  if (state.status !== "finished_successfully" || !state.endTurn) return fail("answer_not_finished", state);
+  if (!text) return fail("answer_empty", state);
+  return Object.assign({ ok: true, reason: "" }, state);
+})()`;
+}
+// ChatGPT marks citations with private-use delimiters (U+E200 opens, U+E202
+// separates, U+E201 closes) and keeps the real sources in content_references.
+const CITATION_MARKER_PATTERN = /\uE200[^\uE200-\uE206]*(?:[\uE202\uE204-\uE206][^\uE200-\uE206]*)*[\uE201\uE203]/g;
+/**
+ * Turn those markers into ordinary markdown links, so a saved answer keeps the
+ * sources instead of the private-use noise (or, as in the rendered DOM, nothing
+ * at all). Markers with no matching reference are dropped.
+ */
+export function resolveTranscriptCitations(text, references = []) {
+    const byMarker = new Map();
+    for (const reference of references) {
+        if (reference && typeof reference.matched_text === "string" && reference.matched_text.length > 0) {
+            byMarker.set(reference.matched_text, reference);
+        }
+    }
+    const linksFor = (reference) => {
+        const seen = new Set();
+        const links = [];
+        for (const item of reference?.items ?? []) {
+            const url = item?.url;
+            if (!url || seen.has(url))
+                continue;
+            seen.add(url);
+            links.push(`[${(item.title || url).trim()}](${url})`);
+        }
+        return links.length > 0 ? ` ${links.join(" ")}` : "";
+    };
+    let resolved = text;
+    for (const [marker, reference] of byMarker) {
+        resolved = resolved.split(marker).join(linksFor(reference));
+    }
+    // Anything still delimited had no reference to restore: strip it so private-use
+    // characters never reach a receipt.
+    return resolved.replace(CITATION_MARKER_PATTERN, "");
+}
 export function deepResearchReportExpression(conversationId) {
     return `(async () => {
-  const fail = (reason, status) => ({ ok: false, reason, status: status || "", report: "", chars: 0 });
+  const fail = (reason, status) => ({ ok: false, reason, status: status || "", report: "", chars: 0, references: [] });
   let token = "";
   try {
     const session = await fetch("/api/auth/session", { credentials: "include" });
@@ -2994,10 +3150,12 @@ export function deepResearchReportExpression(conversationId) {
     return fail("widget_state_unparsable");
   }
   const status = (state && state.status) || "";
-  const parts = state && state.report_message && state.report_message.content && state.report_message.content.parts;
+  const message = (state && state.report_message) || null;
+  const parts = message && message.content && message.content.parts;
   const report = Array.isArray(parts) ? parts.filter((part) => typeof part === "string").join("") : "";
+  const references = message && message.metadata && Array.isArray(message.metadata.content_references) ? message.metadata.content_references : [];
   if (!report) return fail("report_not_ready", status);
-  return { ok: true, reason: "", status, report, chars: report.length };
+  return { ok: true, reason: "", status, report, chars: report.length, references };
 })()`;
 }
 /**

package/dist/cli-help.js

@@ -257,6 +257,14 @@ export function printProBrowserHelp(stdout, sourceCli) {
     const modelsUsage = sourceCli
         ? `${cli} pro browser models${sourceCliOption} [--port 9333] [--timeout-ms 15000]`
         : "prodex pro browser models [--source-cli /absolute/path/to/dist/cli.js] [--port 9333] [--timeout-ms 15000]";
+    const projectsUsage = sourceCli
+        ? `${cli} pro browser projects${sourceCliOption} [--port 9333] [--timeout-ms 15000]  # read-only: exact sidebar project names`
+        : "prodex pro browser projects [--source-cli /absolute/path/to/dist/cli.js] [--port 9333] [--timeout-ms 15000]  # read-only: exact sidebar project names";
+    // A send that outlives its budget is not a lost answer, but only if agents
+    // know this exists - and this help is where onboarding sends them.
+    const recoverUsage = sourceCli
+        ? `${cli} pro browser recover${sourceCliOption} [--cwd /absolute/path/to/repo] [--port 9333] --target-url <thread-url> [--timeout-ms 60000]  # fetch a finished answer (deep research reports too) from a thread whose send timed out`
+        : "prodex pro browser recover [--source-cli /absolute/path/to/dist/cli.js] [--cwd /absolute/path/to/repo] [--port 9333] --target-url <thread-url> [--timeout-ms 60000]  # fetch a finished answer (deep research reports too) from a thread whose send timed out";
     stdout(`${cli} pro browser
 
 Commands:
@@ -264,7 +272,9 @@ Commands:
   ${checkUsage}
   ${smokeUsage}
   ${modelsUsage}
+  ${projectsUsage}
   ${askUsage}
+  ${recoverUsage}
 
 Visible-browser sends require a manual browser session and stop on login, captcha, Cloudflare, permission, rate-limit, or usage-limit blockers.
 Model/project selection (ask):

package/dist/cli.js

@@ -322,6 +322,16 @@ repo: ${cwd}
    ${cli} pro browser check${sourceCliOption} --cwd ${quotedCwd}
    ${cli} pro browser smoke${sourceCliOption} --cwd ${quotedCwd}
    Sharing the browser with other agents? Sends queue behind an in-flight response automatically; pass --busy-wait-ms 0 to fail fast instead.
+   Hand ChatGPT a real file - the only way it can open a pdf, pptx, xlsx or image:
+   ${cli} ask --cwd ${quotedCwd} --attach deck.pptx "Review slides 40-60"  # uploads the file itself
+   ${cli} ask --cwd ${quotedCwd} --file notes.md "Summarize"  # pastes a text file's CONTENTS into the prompt
+   Both are restricted to paths inside the repo. Repeat either flag for several files.
+   Turn on a composer tool for one send:
+   ${cli} ask --cwd ${quotedCwd} --tool web-search "What shipped in Node 24?"  # current facts with sources
+   ${cli} ask --cwd ${quotedCwd} --tool deep-research "Compare managed Postgres providers"  # full browsed report; runs ~10 minutes, budget rises to 30
+   Any other label the composer menu shows works too, so a tool ChatGPT adds later needs no prodex release.
+   A send that outlives its budget did not lose the answer - ChatGPT usually finishes after prodex stops waiting:
+   ${cli} pro browser recover --target-url <thread-url> --cwd ${quotedCwd}  # fetches the finished answer (deep research reports too) and records it
 
 2. Let coding agents consult ChatGPT (stdio MCP: Claude, Codex, Cursor, ...):
    ${cli} claude config --cwd ${quotedCwd}${sourceCliOption}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@youdie006/prodex",
-  "version": "0.22.0",
+  "version": "0.23.1",
   "description": "Local receipt bus for coordinating Codex execution with ChatGPT Pro/Projects consultation.",
   "author": "youdie006",
   "license": "MIT",