@youdie006/prodex 0.40.6 → 0.40.9

package/README.md

@@ -14,7 +14,9 @@
 
 </div>
 
-You pay for ChatGPT Pro. The reasoning that makes it worth paying for lives behind a web page, and the coding agent you actually spend the day with cannot reach it. prodex closes that gap without an API key, a proxy, or a stealth bot: it drives a real, visible Chrome that you logged into once, types into the same composer you would, and reads the answer back from the conversation transcript.
+You pay for ChatGPT Pro. The reasoning that makes it worth paying for lives behind a web page, and the coding agent you actually spend the day with cannot reach it. prodex closes that gap without an API key, a proxy, or a stealth bot: it drives a real Chrome that you logged into once, types into the same composer you would, and reads the rendered answer from the page.
+
+The recording below is from 0.40.6, before internal transcript access was removed. Current builds use rendered page content only; they do not fetch hidden ChatGPT APIs or extract session tokens.
 
 ```console
 $ prodex ask --new-chat --effort Pro "A CLI drives a logged-in browser over the Chrome DevTools Protocol and holds a cross-process file lock while a send is in flight. What failure modes must the lock's expiry rule handle, and which single rule would you ship? Under 150 words."
@@ -38,11 +40,11 @@ That is a real run, timings included. Every consult lands as a task, a result an
 
 ## What you can do with it
 
-- **Ask from the terminal.** `prodex ask` with a question, a file's contents (`--file`), an uploaded pdf, deck, sheet or image (`--attach`), or anything piped in (`--stdin`). Deep research, web search and image creation are one flag away (`--tool`).
+- **Ask from the terminal.** `prodex ask` with a question, a file's contents (`--file`), an uploaded pdf, deck, sheet or image (`--attach`), or anything piped in (`--stdin`). Available rendered composer tools can be selected with `--tool`; unsupported result formats stop with a blocker.
 - **Let your agent ask.** `prodex mcp` is a stdio MCP server with a `pro_consult` tool; Claude Code, Codex, Cursor and Gemini CLI call it like any other tool. ChatGPT Projects can hand work back the other way over a loopback HTTP MCP bridge.
 - **Pick the model and effort per ask.** The picker ChatGPT shows is the picker prodex drives: `--effort Pro` reaches the top rung, `--project` sends inside a sidebar project, and `prodex setup` pins defaults per repo.
 - **Keep every answer.** Tasks, results, sessions and receipts are versioned JSON on disk, signed with a local key. Nothing is stored anywhere else.
-- **Run it with no window at all.** One headed login, then a virtual display: a real browser that Cloudflare treats as one, and nothing on your desktop.
+- **Run it with no desktop window.** One headed login, then a virtual display: a dedicated browser running off your desktop. Authentication and protective checks still require your attention.
 - **Stop where a person should.** Login, captcha, Cloudflare, usage limits and permission prompts halt the send with a named blocker and a next step. prodex solves none of them for you.
 
 ## Install
@@ -67,16 +69,19 @@ prodex pro latest                 # re-print the last answer
 
 `prodex ask` is the short form of `prodex pro browser ask`; every flag works on both. The login opens its own Chrome profile (`~/.local/share/prodex/chrome-chatgpt-pro`), never your daily browser, and in a terminal it keeps watching the window and names the manual step still missing (sign in, clear a check, open a chat) until it reports READY.
 
-While Pro thinks, progress goes to stderr: connecting, prompt sent, elapsed time while generating. A Pro selection raises the send budget to twenty minutes on its own; `--timeout-ms` overrides it. The answer is read from the conversation transcript rather than scraped off the page, so tables and fenced code arrive intact and citations keep their links. If the dedicated browser is not running, an interactive `ask` starts it, waits for your saved session, and retries once (`--no-auto-login` turns that off; scripts opt in with `--auto-login`).
+While Pro thinks, progress goes to stderr: connecting, prompt sent, elapsed time while generating. A Pro selection raises the send budget to twenty minutes on its own; `--timeout-ms` overrides it. Answers are read from the rendered page, so formatting can differ from the original message. If the dedicated browser is not running, an interactive `ask` starts it, waits for your saved session, and retries once (`--no-auto-login` turns that off; scripts opt in with `--auto-login`).
+
+If the browser stops responding after your question was sent, prodex stops without sending it again. Use the `thread` and `request_id` from the error with `prodex pro browser recover --target-url <thread-url> --request-id <32hex>` (MCP: `pro_recover`). The request ID verifies that the recovered assistant answer follows that exact marked user turn. Legacy recovery without it remains available but returns `request_verified: false` and a warning.
 
 Useful flags on every send:
 
 | Flag | What it does |
 |---|---|
-| `--new-chat` | Send into a fresh chat. Recommended for repeated consults; very long threads eventually confuse send detection. |
+| `--new-chat` | Explicitly request the default: ordinary consults start in a fresh chat. The shared current tab is never an implicit destination. |
+| `--session-key id` | Identify one caller for scoped `--continue`. Falls back to `PRODEX_SESSION_KEY`, then `CODEX_THREAD_ID`. |
 | `--file path` | Inline a text file's contents into the prompt. Repeatable. |
 | `--attach path` | Upload the file itself: the only way to hand ChatGPT a pdf, pptx, xlsx or image. Paths must live inside the repo. |
-| `--tool deep-research` | Run a browsed report; the budget rises to thirty minutes and the full report comes back through the transcript. Also `web-search`, `create-image`, or any label the menu shows. |
+| `--tool web-search` | Select a rendered composer tool. Automatic deep-research report retrieval is currently unsupported and is blocked before sending. |
 | `--project "name"` | Send inside an existing sidebar project. `--project-new` creates one first. `prodex pro browser projects` lists exact names. |
 | `--temporary` | A ChatGPT Temporary Chat: nothing in your chat list, but the answer is read off the page and cannot be recovered later. |
 | `--json` | Structured output on stdout, progress on stderr. |
@@ -84,6 +89,8 @@ Useful flags on every send:
 
 Prefer prompts to flags? `prodex ui` (or a bare `prodex` in a terminal) asks what to send and where, shows a progress bar, and prints the equivalent command so the flags are learnable.
 
+Conversation/project lists contain only entries exposed by the rendered UI, not the full account history. Automatic chat/project deletion and hidden transcript/report retrieval are unavailable; perform those operations yourself in ChatGPT. A successful recovery requires the requested conversation and a stable, finished answer.
+
 ## Agents over MCP
 
 **Claude Code, Codex, Cursor, Gemini CLI** talk to prodex over stdio. For Claude:
@@ -98,7 +105,9 @@ prints a token-free config that points Claude at `prodex mcp --cwd /absolute/pat
 { "mcpServers": { "prodex": { "command": "prodex", "args": ["mcp", "--cwd", "/absolute/path/to/your/repo"] } } }
 ```
 
-The server exposes `pro_consult` (a visible-browser send, with the same model, effort, project and tool choices as the CLI), `pro_recover` (fetch an answer that finished after a timeout), the bridge ledger tools (`bridge_create_task`, `bridge_list_tasks`, `bridge_fetch_result`, receipts, sessions), bounded `repo_read_file` and `repo_search`, and a receipt-gated write path: `repo_write_file_dry_run` first, `repo_write_file_apply` only while git HEAD and the file's preimage hash still match, `repo_stage_reviewed_paths` for applied receipts only. No shell tool, no ungated write. `prodex claude prompt` prints a paste-ready prompt that verifies the wiring. [docs/claude.md](docs/claude.md) covers Claude Desktop and Claude Code; [docs/clients.md](docs/clients.md) covers the others, including the per-call approval and `tool_timeout_sec` Codex needs.
+The server exposes `pro_consult` (a visible-browser send, with the same model, effort, project and tool choices as the CLI), `pro_recover` (fetch an answer that finished after a timeout), the bridge ledger tools (`bridge_create_task`, `bridge_list_tasks`, `bridge_fetch_result`, receipts, sessions), bounded `repo_read_file` and `repo_search`, and a receipt-gated write path: `repo_write_file_dry_run` first, `repo_write_file_apply` only while git HEAD and the file's preimage hash still match, `repo_stage_reviewed_paths` for applied receipts only. Each stdio MCP connection receives one default session key, ordinary consults start fresh, and `continue_thread` only searches that key and project. Logical agents sharing one MCP connection should pass distinct explicit `session_key` values and preserve them for follow-ups; an explicit key also preserves continuity across an MCP restart. No shell tool, no ungated write. `prodex claude prompt` prints a paste-ready prompt that verifies the wiring. [docs/claude.md](docs/claude.md) covers Claude Desktop and Claude Code; [docs/clients.md](docs/clients.md) covers the others, including the per-call approval and `tool_timeout_sec` Codex needs.
+
+Updating the installed npm package does not reload an MCP process that is already running. Reconnect the MCP server or restart the Codex/Claude client to load the new build. The dedicated browser profile is separate and remains signed in, so this does not require ChatGPT authentication again.
 
 An MCP server usually starts without `--cwd`, so a per-repo default can be missed. For defaults that apply from any directory, set `PRODEX_DEFAULT_PROJECT`, `PRODEX_DEFAULT_MODEL`, `PRODEX_DEFAULT_EFFORT` or `PRODEX_DEFAULT_PRO_MODE` in the agent's MCP `env` block; a per-repo config still wins field by field.
 
@@ -151,15 +160,22 @@ prodex status                     # shows the saved defaults
 ```sh
 prodex pro browser login                    # once, headed: sign in
 prodex pro browser login --virtual-display  # from then on: no window anywhere
+prodex pro browser login --headed            # visible login/captcha when a hidden mode was saved
 ```
 
-`--virtual-display` (or `PRODEX_VIRTUAL_DISPLAY=1`, which the MCP server and its auto-recovery honour too) starts an X virtual framebuffer and runs the dedicated Chrome on it. It is a real headed browser, so Cloudflare treats it as one: measured end to end, the signed-in profile loaded chatgpt.com with no challenge and a Pro send returned normally, with nothing on the desktop. Linux and WSL; needs `xvfb` and `xauth`, and the display is protected by a per-display xauth cookie rather than opened to every process.
+`--virtual-display` (or `PRODEX_VIRTUAL_DISPLAY=1`, which the MCP server and its auto-recovery honour too) starts an X virtual framebuffer and runs the dedicated Chrome on it. It uses ordinary headed Chrome without a desktop window, not Chrome's headless mode. Linux and WSL; needs `xvfb` and `xauth`. New displays use local abstract Unix sockets with per-display xauth authentication and no TCP listener. This does not bypass login or protection checks.
+
+Already-running browsers and legacy TCP X servers are not stopped or migrated by the update. To migrate, finish pending consults, stop the dedicated browser and its old X server, then launch with `--virtual-display` using the updated prodex. A new launch skips any display number still occupied by a TCP listener.
+
+The last recorded window mode is reused by later `login` commands and by CLI/MCP auto-recovery. One explicit mode flag (`--headed`, `--headless`, `--minimized`, or `--virtual-display`) overrides environment and saved state as a whole. With no mode flag, any non-empty mode environment value wins as a whole too, including `PRODEX_HEADLESS=0`, `false`, or `no`; otherwise the saved mode remains. The modes are mutually exclusive, and the normal first-run default remains a visible headed browser. If virtual-display setup fails, recovery stops instead of unexpectedly opening a desktop window.
 
 `--minimized` keeps a window but minimizes it. Under WSLg a minimized Chrome still reports itself visible and consults keep working; a normal Linux desktop marks it hidden, and prodex refuses to send into a tab it cannot read, restores the window, and tells you.
 
 `--headless` exists and is not usable against ChatGPT today: measured on a signed-in profile, headless Chrome stays on Cloudflare's interstitial past sixty seconds. Only the window is optional; the login is not.
 
-A browser that stops answering its control port mid-send is ended and started fresh before the send, and the receipt says so (`PRODEX_NO_AUTO_CLEAR=1` turns that off). A browser that is merely slow is left alone.
+If a hidden or virtual browser needs login, captcha, Cloudflare, or account verification, close that browser yourself and run `prodex pro browser login --headed` to complete the interactive step. Merely omitting `--headless` does not switch modes because the saved mode persists. prodex does not bypass the protection or kill a running browser to change its mode.
+
+Before a prompt is submitted, a browser confirmed to have stopped answering its control port can be ended and started fresh, and the receipt says so (`PRODEX_NO_AUTO_CLEAR=1` turns that off). A browser that is merely slow is left alone. After submission, prodex never auto-resends a lost prompt.
 
 ## Receipts
 
@@ -175,7 +191,9 @@ Everything a consult touches is written under `.bridge/` in the repo it ran from
   diagnostics/  screenshots and page-shape snapshots from failed sends, when enabled
 ```
 
-`prodex init` creates the ledger (a browser send creates it on first use too). `prodex pro latest`, `pro show`, `results show`, `results artifact`, `receipts show` and `sessions show` read them; `--json` on the list commands gives structured output. Result artifacts are checked against the sha256 recorded when they were finalized. A blocked consult is completed as blocked with its code and next step, so `pro latest` shows what happened even when nothing was sent. `prodex receipts rotate-key` signs new receipts with a fresh key while older ones stay verifiable; `prodex results reseal <task-id> --confirm-current-result` re-signs a legacy result you have reviewed.
+`prodex init` creates the ledger (a browser send creates it on first use too). `prodex pro latest`, `pro show`, `results show`, `results artifact`, `receipts show` and `sessions show` read them; `--json` on the list commands gives structured output. Newly finalized result artifacts are checked against their recorded sha256. Legacy artifacts without hashes remain readable with a `legacy_artifact_unverified` warning; signing result metadata does not verify those bytes. A blocked consult is completed as blocked with its code and next step, so `pro latest` shows what happened even when nothing was sent. `prodex receipts rotate-key` signs new receipts with a fresh key while older ones stay verifiable; `prodex results reseal <task-id> --confirm-current-result` re-signs a legacy result you have reviewed.
+
+`sessions cancel` clears a stale session record after an interrupted send; it does not stop a running browser request. Stop that request in the terminal or client that started it.
 
 Two sibling tools read the same ledger, found through the bridge registry prodex keeps in `~/.local/share/prodex/bridges.json`: [sessionwiki](https://github.com/youdie006/sessionwiki) indexes every consult as a searchable session, and [swapdex](https://github.com/youdie006/swapdex) lists recent consults after an account switch. Neither is required.
 
@@ -192,7 +210,7 @@ ChatGPT Projects ---------+        v
                                                                                 ChatGPT Pro
 ```
 
-The browser is a real Chrome launched with `--remote-debugging-port` on `127.0.0.1`, live only while that window is open. prodex checks the page state, confirms the tab is on a ChatGPT conversation it can read, applies the picker selection, types the prompt, waits for the answer to finish, and reads it from the transcript. Sends are paced to human speed (one every ten seconds by default, `PRODEX_MIN_SEND_INTERVAL_MS` tunes it) and take a cross-process lock, so two agents on one machine queue rather than fight over the composer.
+The browser is a real Chrome launched with `--remote-debugging-port` on `127.0.0.1`, live only while the browser process is running. prodex checks the page state, confirms the tab is on a ChatGPT conversation it can read, applies the picker selection, types the prompt, waits for the answer to finish, and reads its rendered content. Sends are paced to human speed (one every ten seconds by default, `PRODEX_MIN_SEND_INTERVAL_MS` tunes it) and take a cross-process lock, so two agents on one machine queue rather than fight over the composer.
 
 ### What it will not do
 
@@ -225,9 +243,9 @@ Reports are deduplicated by blocker code, so something that stays broken adds to
 
 **Why the pause before sending?** Pacing: `send_pacing: waiting Ns` on stderr. `PRODEX_MIN_SEND_INTERVAL_MS=0` disables it.
 
-**The answer timed out.** Pro can take many minutes; the `send_timeout` blocker prints a rerun command with a doubled budget, and a partial answer is kept with an `answer_incomplete` warning. If the thread finished after the timeout, `prodex pro browser recover --target-url <thread>` fetches it, deep research reports included.
+**The answer timed out.** Pro can take many minutes. Do not resend automatically. Recover the original marked turn with `prodex pro browser recover --target-url <thread> --request-id <request_id>` after it finishes.
 
-**Sends started failing after many consults in one chat.** Long threads confuse prompt-acceptance detection. Use `--new-chat` (`new_chat: true` on the MCP tool).
+**A consult returned an unrelated answer.** Current sends append a unique visible request marker and accept only the assistant turn following that marker. Ordinary calls also start fresh. A `request_mismatch` blocker is not retryable; inspect the named request instead of treating the returned page's last answer as the consult.
 
 **Every send says "still generating" and nothing is being written.** ChatGPT parked the thread on "which response do you prefer?". prodex reports `response_choice_pending` and names the buttons; pick one, or send with `--new-chat`.
 

package/dist/browser-send-lock.js

@@ -1,6 +1,6 @@
-import { link, mkdir, open, readFile, rm, stat } from "node:fs/promises";
 import os from "node:os";
 import path from "node:path";
+import { withCrossProcessFileLock } from "./safe-file.js";
 // One visible-browser send at a time per machine: the dedicated Chrome is a
 // single shared tab, and two concurrent prodex clients interleave composer
 // input and navigation, silently cross-contaminating each other's threads
@@ -12,94 +12,6 @@ function lockPath() {
         return override;
     return path.join(os.homedir(), ".local", "share", "prodex", "browser-send.lock");
 }
-function holderIsAlive(pid) {
-    try {
-        process.kill(pid, 0);
-        return true;
-    }
-    catch (error) {
-        // EPERM: the process exists but is owned by another user -> alive. Only
-        // ESRCH (no such process) means the holder is truly dead and reapable.
-        return error.code === "EPERM";
-    }
-}
-async function readHolder(file) {
-    try {
-        const parsed = JSON.parse(await readFile(file, "utf8"));
-        return {
-            pid: typeof parsed.pid === "number" ? parsed.pid : undefined,
-            started_at: typeof parsed.started_at === "string" ? parsed.started_at : undefined
-        };
-    }
-    catch {
-        return undefined;
-    }
-}
-// A holder whose process is alive but has held the lock far longer than any real
-// send (default 60 min, well beyond the 15-min Pro timeout) is treated as wedged
-// and reapable, so a hung browser cannot block every send on the machine forever.
-// Deliberately generous so it never reaps a genuinely in-flight send; override
-// with PRODEX_SEND_LOCK_STALE_MS.
-function staleMs() {
-    const raw = Number(process.env.PRODEX_SEND_LOCK_STALE_MS);
-    return Number.isFinite(raw) && raw > 0 ? raw : 3_600_000;
-}
-function holderIsStale(startedAt) {
-    if (!startedAt)
-        return false;
-    const started = Date.parse(startedAt);
-    if (Number.isNaN(started))
-        return false;
-    return Date.now() - started > staleMs();
-}
-// Release only our own lock: if ours was already reaped (e.g. as stale) and a
-// new holder took over, we must not delete their lock on the way out.
-async function releaseIfOwned(file) {
-    const holder = await readHolder(file);
-    if (holder?.pid === process.pid) {
-        await rm(file, { force: true }).catch(() => undefined);
-    }
-}
-let acquireSeq = 0;
-async function tryAcquire(file) {
-    await mkdir(path.dirname(file), { recursive: true, mode: 0o700 });
-    // Publish atomically: write the pid into a temp file, then hard-link it into
-    // place. link() fails with EEXIST when a holder already exists (our exclusivity
-    // check) and, unlike open("wx")+writeFile, the lock file is never observed
-    // empty - so a concurrent waiter can never mistake a mid-publish lock for a
-    // dead one and reap it out from under us (which let two clients send at once).
-    // The temp name is unique per acquisition (pid + seq) so two concurrent
-    // same-process acquires never share a temp inode and truncate each other.
-    const temp = `${file}.${process.pid}.${(acquireSeq += 1)}.tmp`;
-    const handle = await open(temp, "w", 0o600);
-    try {
-        await handle.writeFile(`${JSON.stringify({ pid: process.pid, started_at: new Date().toISOString() })}\n`);
-    }
-    finally {
-        await handle.close();
-    }
-    try {
-        await link(temp, file);
-        return true;
-    }
-    catch (error) {
-        if (error.code !== "EEXIST")
-            throw error;
-        return false;
-    }
-    finally {
-        await rm(temp, { force: true }).catch(() => undefined);
-    }
-}
-async function lockFileExists(file) {
-    try {
-        await stat(file);
-        return true;
-    }
-    catch {
-        return false;
-    }
-}
 /**
  * Serialize visible-browser sends across processes. Waits up to waitMs for a
  * live holder to finish (0 = fail fast); a lock whose holder process is dead
@@ -107,46 +19,13 @@ async function lockFileExists(file) {
  */
 export async function withBrowserSendLock(waitMs, onWait, fn) {
     const file = lockPath();
-    const deadline = Date.now() + Math.max(0, waitMs);
-    let waited = false;
-    for (;;) {
-        if (await tryAcquire(file))
-            break;
-        const holder = await readHolder(file);
-        const reapable = holder === undefined || holder.pid === undefined || !holderIsAlive(holder.pid) || holderIsStale(holder.started_at);
-        if (reapable) {
-            // Re-verify the same holder still owns the file before removing it, so we
-            // don't delete a fresh lock another reaper just acquired in between.
-            const current = await readHolder(file);
-            if (current?.pid === holder?.pid) {
-                await rm(file, { force: true }).catch(() => undefined);
-            }
-            // A reap that did not actually remove the file - another user's lock in a
-            // shared directory, or a directory sitting in its place - used to retry
-            // immediately, skipping both the sleep and the deadline: a hot loop that
-            // never returned and never timed out. Waiting here costs a reap that
-            // raced nothing; not waiting costs the process.
-            if (await lockFileExists(file)) {
-                if (Date.now() >= deadline) {
-                    throw new Error(`A prodex browser send lock at ${file} is held by nothing and could not be removed, so no send can start. Delete that file and retry.`);
-                }
-                await new Promise((resolve) => setTimeout(resolve, 2_000));
-            }
-            continue;
-        }
-        if (Date.now() >= deadline) {
-            throw new Error(`Another prodex browser send is in progress (pid ${holder.pid}) and did not finish within the wait budget. Retry once it finishes, or raise --timeout-ms (which is also the queue budget).`);
-        }
-        if (!waited) {
-            waited = true;
-            onWait(`another prodex send holds the browser (pid ${holder.pid}); waiting`);
-        }
-        await new Promise((resolve) => setTimeout(resolve, 2_000));
-    }
-    try {
-        return await fn();
-    }
-    finally {
-        await releaseIfOwned(file);
-    }
+    const reapClaim = path.join(path.dirname(file), `.${path.basename(file)}.reap`);
+    return withCrossProcessFileLock(file, {
+        waitMs,
+        retryMs: 2_000,
+        privateParent: true,
+        onWait: (holder) => onWait(`another prodex send holds the browser (pid ${holder.pid ?? "unknown"}); waiting`),
+        busyError: (holder) => new Error(`Another prodex browser send is in progress (pid ${holder.pid ?? "unknown"}) and did not finish within the wait budget. Retry once it finishes, or raise --timeout-ms (which is also the queue budget).`),
+        unavailableError: () => new Error(`A prodex browser send lock at ${file} is held by nothing and could not be removed, so no send can start. Stop all prodex senders and verify no request is active before deleting that file and the abandoned reaper claim at ${reapClaim}, then retry.`)
+    }, fn);
 }