codex-chatgpt-control 0.1.0-alpha.1 → 0.3.0-alpha.1

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.3.0-alpha.1
+
+- Hardens mode-menu detection and selection against thread/sidebar action menus, with locale-registry-backed thread-action vetoes and container-scoped menu enumeration.
+- Adds the `modes.get` primitive and post-selection verification warnings on `modes.set`.
+- Rewrites wait polling around a single combined DOM snapshot per poll; response text is fetched once at completion instead of every poll.
+- Adds Windows and Linux clipboard capture with DOM fallback.
+- Fixes report `createdAt` to honor the injected clock for deterministic fixtures.
+
+## 0.2.0-alpha.1
+
+- Adds Windows-safe host path validation and cross-platform backend gates.
+- Adds localized ChatGPT selector support through the locale-label registry.
+- Adds untrusted-output safety envelopes and integrity sidecar verification helpers.
+
 ## 0.1.0-alpha.1
 
 - Initial public alpha package metadata and source layout.

package/README.md

@@ -40,6 +40,59 @@ await chatgpt.askInThread({
 });
 ```
 
+Attach local files with host-local absolute paths:
+
+```ts
+const preflight = await chatgpt.files.preflight({
+  paths: ["/absolute/host/path/to/report.pdf"]
+});
+
+await chatgpt.askWithFiles({
+  files: ["/absolute/host/path/to/report.pdf"],
+  prompt: "Summarize this report.",
+  wait: true,
+  read: { format: "markdown" }
+});
+
+await chatgpt.askWithFiles({
+  files: [String.raw`C:\Users\you\Documents\report.pdf`],
+  prompt: "Summarize this report.",
+  wait: true,
+  read: { format: "markdown" }
+});
+```
+
+Use the second example only when the backend process itself is running on Windows. If the backend runs in WSL/Linux, pass the WSL/Linux path, such as `/home/you/Documents/report.pdf`.
+
+Plan append-only ChatGPT Project Sources changes before mutating a project:
+
+```ts
+const plan = await chatgpt.projects.sources.planAdd({
+  projectUrl: "https://chatgpt.com/g/g-p-example/project",
+  files: ["/absolute/host/path/to/source.md"]
+});
+
+const added = await chatgpt.projects.sources.add({
+  projectUrl: "https://chatgpt.com/g/g-p-example/project",
+  files: ["/absolute/host/path/to/source.md"],
+  confirmMutation: true
+});
+```
+
+`planAdd` validates explicit local file metadata without reading file contents or opening ChatGPT. `add` is append-only and returns `needs_confirmation` unless `confirmMutation: true` is supplied.
+
+Explain structured blockers before deciding whether to retry:
+
+```ts
+const result = await chatgpt.session.bootstrap({ existingTab: true });
+if (!result.ok) {
+  const explanation = chatgpt.explainBlocker(result, { command: "session.bootstrap" });
+  console.log(explanation.markdown);
+}
+```
+
+`explainBlocker` preserves the original `result.blocker` fields and adds conservative retry/resume guidance. Existing-tab blockers include metadata such as requested target, candidate tab IDs, URLs, titles, conversation IDs, and mismatch reason; they do not include page text or chat content.
+
 ## Validation
 
 Run deterministic gates:
@@ -50,5 +103,7 @@ npm run build
 npm run bundle
 npm run bundle:backend
 npm run contract:validate
+npm run docs:drift
 npm run parity:fixtures
+npm run parity:suite
 ```

package/contracts/v1/fixtures/backend-capabilities.json

@@ -35,9 +35,17 @@
     "messages.wait",
     "messages.readLatest",
     "messages.waitAndRead",
+    "artifacts.listLatest",
+    "artifacts.wait",
+    "artifacts.downloadLatest",
+    "files.preflight",
     "files.attach",
     "files.downloadLatest",
+    "projects.sources.list",
+    "projects.sources.planAdd",
+    "projects.sources.add",
     "modes.set",
+    "modes.get",
     "tools.select",
     "response.copy"
   ],

package/contracts/v1/fixtures/blocker-explanation-profiles.json

@@ -0,0 +1,114 @@
+{
+  "ok": true,
+  "requestId": "req_blocker_explanation_profiles",
+  "result": {
+    "profiles": [
+      {
+        "category": "environment",
+        "kind": "browser_bridge_unavailable",
+        "severity": "blocked",
+        "title": "Browser bridge unavailable",
+        "userActionRequired": false
+      },
+      {
+        "category": "auth",
+        "kind": "login_required",
+        "severity": "action_required",
+        "title": "Login required",
+        "userActionRequired": true
+      },
+      {
+        "category": "auth",
+        "kind": "captcha",
+        "severity": "action_required",
+        "title": "Captcha or human verification required",
+        "userActionRequired": true
+      },
+      {
+        "category": "auth",
+        "kind": "rate_limit",
+        "severity": "action_required",
+        "title": "Rate limited",
+        "userActionRequired": true
+      },
+      {
+        "category": "runtime",
+        "kind": "modal",
+        "severity": "action_required",
+        "title": "Modal is blocking the page",
+        "userActionRequired": true
+      },
+      {
+        "category": "permission",
+        "kind": "permission",
+        "severity": "action_required",
+        "title": "Permission required",
+        "userActionRequired": true
+      },
+      {
+        "category": "user_confirmation",
+        "kind": "confirmation",
+        "severity": "action_required",
+        "title": "Confirmation required",
+        "userActionRequired": true
+      },
+      {
+        "category": "ui_drift",
+        "kind": "selector_drift",
+        "severity": "blocked",
+        "title": "Selector drift",
+        "userActionRequired": false
+      },
+      {
+        "category": "artifact",
+        "kind": "artifact_unavailable",
+        "severity": "warning",
+        "title": "Artifact unavailable",
+        "userActionRequired": false
+      },
+      {
+        "category": "ui_drift",
+        "kind": "artifact_selector_drift",
+        "severity": "blocked",
+        "title": "Artifact selector drift",
+        "userActionRequired": false
+      },
+      {
+        "category": "download",
+        "kind": "artifact_download_unavailable",
+        "severity": "warning",
+        "title": "Artifact download unavailable",
+        "userActionRequired": false
+      },
+      {
+        "category": "download",
+        "kind": "download_unavailable",
+        "severity": "warning",
+        "title": "Download unavailable",
+        "userActionRequired": false
+      },
+      {
+        "category": "upload",
+        "kind": "upload_failed",
+        "severity": "action_required",
+        "title": "Upload failed",
+        "userActionRequired": true
+      },
+      {
+        "category": "not_found",
+        "kind": "not_found",
+        "severity": "warning",
+        "title": "Target not found",
+        "userActionRequired": false
+      },
+      {
+        "category": "unknown",
+        "kind": "unknown",
+        "severity": "blocked",
+        "title": "Unknown blocker",
+        "userActionRequired": false
+      }
+    ]
+  },
+  "schemaVersion": "chatgpt.browser_control.backend_response.v1"
+}

package/contracts/v1/fixtures/command-descriptors.json

@@ -67,7 +67,7 @@
       "args": {
         "existingTab": "true or explicit policy to claim a user-open Chrome tab instead of opening a replacement",
         "files": "absolute local file paths to attach before submitting",
-        "mode": "optional visible mode selection, e.g. { effort: \"Thinking\" }",
+        "mode": "optional visible mode selection, e.g. { model: \"Pro\" } or { intelligence: \"Pro\", modelVersion: \"5.4\" }",
         "prompt": "message to send after files are attached",
         "read": "true or read options such as { format: \"markdown\" }; defaults to Markdown",
         "report": "optional redacted report settings",
@@ -89,7 +89,7 @@
       },
       "defaultTimeoutMs": 120000,
       "examples": [
-        "await chatgpt.askWithFiles({ thread: { type: \"url\", url: \"https://chatgpt.com/c/<conversation-id>\" }, existingTab: true, mode: { effort: \"Thinking\" }, files: [\"/absolute/path/brief.md\"], prompt: \"Summarize this.\", wait: true, read: { format: \"markdown\" } });"
+        "await chatgpt.askWithFiles({ thread: { type: \"url\", url: \"https://chatgpt.com/c/<conversation-id>\" }, existingTab: true, mode: { model: \"Pro\" }, files: [\"/absolute/host/path/brief.md\"], prompt: \"Summarize this.\", wait: true, read: { format: \"markdown\" } });"
       ],
       "layer": "workflow",
       "name": "askWithFiles",
@@ -119,7 +119,7 @@
       },
       "defaultTimeoutMs": 120000,
       "examples": [
-        "await chatgpt.askAndDownload({ prompt: \"Create a CSV.\", download: { destDir: \"/tmp/out\" }, wait: true });"
+        "await chatgpt.askAndDownload({ prompt: \"Create a CSV.\", download: { destDir: \"/absolute/host/output\" }, wait: true });"
       ],
       "layer": "workflow",
       "name": "askAndDownload",
@@ -371,7 +371,7 @@
       },
       "defaultTimeoutMs": 120000,
       "examples": [
-        "await chatgpt.runPlan({ name: \"attach-ask-read\", input: { files: [\"/absolute/path.md\"], prompt: \"Summarize.\" } });"
+        "await chatgpt.runPlan({ name: \"attach-ask-read\", input: { files: [\"/absolute/host/path.md\"], prompt: \"Summarize.\" } });"
       ],
       "layer": "workflow",
       "name": "attach-ask-read",
@@ -399,7 +399,7 @@
       },
       "defaultTimeoutMs": 120000,
       "examples": [
-        "await chatgpt.runPlan({ name: \"ask-and-download\", input: { prompt: \"Create a CSV.\", destDir: \"/tmp/out\" } });"
+        "await chatgpt.runPlan({ name: \"ask-and-download\", input: { prompt: \"Create a CSV.\", destDir: \"/absolute/host/output\" } });"
       ],
       "layer": "workflow",
       "name": "ask-and-download",
@@ -476,7 +476,10 @@
     },
     {
       "args": {
-        "check": "optional list of readiness checks"
+        "check": "optional list of readiness checks",
+        "existingTab": "optional exact existing-tab policy for check: [\"existing_tab\"]",
+        "files": "optional file paths for check: [\"file_preflight\"]",
+        "report": "optional report output policy for check: [\"reports\"]"
       },
       "blockers": [
         "browser_bridge_unavailable",
@@ -486,13 +489,16 @@
       "defaults": {},
       "defaultTimeoutMs": 30000,
       "examples": [
-        "await chatgpt.doctor({ check: [\"bridge\", \"login\", \"upload\"] });"
+        "await chatgpt.doctor({ check: [\"bridge\", \"login\", \"upload\"] });",
+        "await chatgpt.doctor({ check: [\"existing_tab\"], existingTab: { target: { type: \"conversationId\", conversationId: \"<conversation-id>\" }, ifMissing: \"block\" } });",
+        "await chatgpt.doctor({ check: [\"file_preflight\"], files: [\"/absolute/host/path.md\"] });",
+        "await chatgpt.doctor({ check: [\"localization\", \"reports\"], report: { destDir: \"/absolute/host/reports\" } });"
       ],
       "layer": "diagnostic",
       "name": "doctor",
       "retryPolicy": "Return structured readiness checks; retry only after the reported blocker or permission setting changes.",
       "risk": "low",
-      "summary": "Preflight browser bridge, login, upload, download, clipboard, mode, and tool readiness."
+      "summary": "Preflight browser bridge, login, upload, local files, existing-tab, artifact, localization, report, and selector readiness."
     },
     {
       "args": {
@@ -508,7 +514,7 @@
       },
       "defaultTimeoutMs": 5000,
       "examples": [
-        "await chatgpt.createReport(result, { destDir: \"/tmp/reports\" });"
+        "await chatgpt.createReport(result, { destDir: \"/absolute/host/reports\" });"
       ],
       "layer": "report",
       "name": "createReport",
@@ -645,7 +651,13 @@
       "summary": "Compose, submit, optionally wait, and optionally read."
     },
     {
-      "args": {},
+      "args": {
+        "mode": "normal or deep_research",
+        "pollMs": "optional polling interval",
+        "responseContent": "include for current behavior, metadata to omit assistant text and return chars/hash only",
+        "stableMs": "optional stability window before completion",
+        "timeoutMs": "optional wait timeout"
+      },
       "blockers": [
         "browser_bridge_unavailable",
         "login_required",
@@ -703,6 +715,102 @@
     },
     {
       "args": {
+        "kind": "artifact kind; currently image",
+        "max": "maximum artifacts to return"
+      },
+      "blockers": [
+        "browser_bridge_unavailable",
+        "login_required",
+        "artifact_unavailable",
+        "artifact_selector_drift",
+        "artifact_download_unavailable"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 30000,
+      "examples": [
+        "await chatgpt.artifacts.downloadLatest({ destDir: \"/absolute/host/output\" });"
+      ],
+      "layer": "primitive",
+      "name": "artifacts.listLatest",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "medium",
+      "summary": "Detect the latest visible generated ChatGPT artifact, such as an image-only result."
+    },
+    {
+      "args": {
+        "afterArtifactCount": "baseline artifact count",
+        "kind": "artifact kind; currently image",
+        "requireDownload": "wait until a download affordance is visible"
+      },
+      "blockers": [
+        "browser_bridge_unavailable",
+        "login_required",
+        "artifact_unavailable",
+        "artifact_selector_drift",
+        "artifact_download_unavailable"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 120000,
+      "examples": [
+        "await chatgpt.artifacts.downloadLatest({ destDir: \"/absolute/host/output\" });"
+      ],
+      "layer": "primitive",
+      "name": "artifacts.wait",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "low",
+      "summary": "Wait for a visible generated ChatGPT artifact to appear and stabilize."
+    },
+    {
+      "args": {
+        "destDir": "download destination directory",
+        "prefer": "download_control or visible_image_source"
+      },
+      "blockers": [
+        "browser_bridge_unavailable",
+        "login_required",
+        "artifact_unavailable",
+        "artifact_selector_drift",
+        "artifact_download_unavailable"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 120000,
+      "examples": [
+        "await chatgpt.artifacts.downloadLatest({ destDir: \"/absolute/host/output\" });"
+      ],
+      "layer": "primitive",
+      "name": "artifacts.downloadLatest",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "medium",
+      "summary": "Download or save the latest visible generated ChatGPT artifact."
+    },
+    {
+      "args": {
+        "includeHashes": "optional SHA-256 metadata for diagnostics; file contents are not returned",
+        "maxBytesPerFile": "optional local per-file byte limit",
+        "maxTotalBytes": "optional local total byte limit",
+        "paths": "absolute local file paths"
+      },
+      "blockers": [
+        "not_found",
+        "permission",
+        "upload_failed"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 30000,
+      "examples": [
+        "await chatgpt.files.preflight({ paths: [\"/absolute/host/path.md\"] });",
+        "await chatgpt.files.preflight({ paths: [\"/absolute/host/path.md\"], includeHashes: true });"
+      ],
+      "layer": "primitive",
+      "name": "files.preflight",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "low",
+      "summary": "Validate local file paths, size limits, duplicates, zero-byte files, and extension-based MIME/category guesses without opening ChatGPT."
+    },
+    {
+      "args": {
+        "includeDiagnostics": "optional preflight and browser input file-size metadata",
+        "includeHashes": "optional SHA-256 metadata inside diagnostics; file contents are not returned",
         "paths": "absolute local file paths"
       },
       "blockers": [
@@ -714,7 +822,8 @@
       "defaults": {},
       "defaultTimeoutMs": 180000,
       "examples": [
-        "await chatgpt.files.attach({ paths: [\"/absolute/path.jpg\"] });"
+        "await chatgpt.files.attach({ paths: [\"/absolute/host/path.jpg\"] });",
+        "// On Windows backend hosts, use paths such as C:\\Users\\you\\Pictures\\image.jpg."
       ],
       "layer": "primitive",
       "name": "files.attach",
@@ -738,6 +847,79 @@
       "risk": "medium",
       "summary": "Download the latest visible ChatGPT file affordance."
     },
+    {
+      "args": {
+        "existingTab": "optional exact existing-tab policy",
+        "projectUrl": "ChatGPT Project URL such as https://chatgpt.com/g/g-p-.../project",
+        "timeoutMs": "optional browser timeout"
+      },
+      "blockers": [
+        "browser_bridge_unavailable",
+        "login_required",
+        "selector_drift",
+        "confirmation",
+        "permission",
+        "upload_failed"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 30000,
+      "examples": [
+        "await chatgpt.projects.sources.list({ projectUrl: \"https://chatgpt.com/g/g-p-example/project\" });"
+      ],
+      "layer": "primitive",
+      "name": "projects.sources.list",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "low",
+      "summary": "Open or claim a visible ChatGPT Project Sources tab and list source names/statuses without source contents."
+    },
+    {
+      "args": {
+        "batchSize": "optional upload batch size",
+        "files": "explicit absolute local file paths",
+        "projectUrl": "ChatGPT Project URL"
+      },
+      "blockers": [
+        "not_found",
+        "permission",
+        "upload_failed"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 30000,
+      "examples": [
+        "await chatgpt.projects.sources.planAdd({ projectUrl: \"https://chatgpt.com/g/g-p-example/project\", files: [\"/absolute/host/path.md\"] });"
+      ],
+      "layer": "primitive",
+      "name": "projects.sources.planAdd",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "low",
+      "summary": "Dry-run an append-only Project Sources file add from explicit local files without opening ChatGPT."
+    },
+    {
+      "args": {
+        "batchSize": "optional upload batch size",
+        "confirmMutation": "must be true to mutate Project Sources",
+        "files": "explicit absolute local file paths",
+        "projectUrl": "ChatGPT Project URL"
+      },
+      "blockers": [
+        "browser_bridge_unavailable",
+        "login_required",
+        "selector_drift",
+        "confirmation",
+        "permission",
+        "upload_failed"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 180000,
+      "examples": [
+        "await chatgpt.projects.sources.add({ projectUrl: \"https://chatgpt.com/g/g-p-example/project\", files: [\"/absolute/host/path.md\"], confirmMutation: true });"
+      ],
+      "layer": "primitive",
+      "name": "projects.sources.add",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "medium",
+      "summary": "Append explicit local files to a visible ChatGPT Project Sources list after confirmMutation: true."
+    },
     {
       "args": {
         "format": "markdown, normalized_text, visible_text, html, blocks, or all",
@@ -761,9 +943,12 @@
     },
     {
       "args": {
-        "effort": "visible effort label such as Thinking or Extended",
-        "model": "visible model label such as Instant, Pro, or another available model",
-        "timeoutMs": "optional timeout for opening and selecting the visible mode menu"
+        "effort": "visible legacy effort label such as Thinking or Extended",
+        "intelligence": "visible intelligence label such as Medium, High, Extra High, or Pro",
+        "model": "visible model label such as Pro, GPT-5.5, or another available model",
+        "modelVersion": "visible nested model version such as 5.5, 5.4, 4.5, or o3",
+        "timeoutMs": "optional timeout for opening and selecting the visible mode menu",
+        "version": "alias for modelVersion"
       },
       "blockers": [
         "browser_bridge_unavailable",
@@ -773,14 +958,37 @@
       "defaults": {},
       "defaultTimeoutMs": 30000,
       "examples": [
+        "await chatgpt.modes.set({ model: \"Pro\" });",
+        "await chatgpt.modes.set({ intelligence: \"Pro\", modelVersion: \"5.4\" });",
         "await chatgpt.modes.set({ effort: \"Thinking\" });",
-        "await chatgpt.askWithFiles({ mode: { effort: \"Thinking\" }, files: [\"/absolute/path.jpg\"], prompt: \"Describe this image.\", wait: true });"
+        "await chatgpt.askWithFiles({ mode: { model: \"Pro\" }, files: [\"/absolute/host/path.jpg\"], prompt: \"Describe this image.\", wait: true });"
       ],
       "layer": "primitive",
       "name": "modes.set",
       "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
       "risk": "medium",
-      "summary": "Select a visible model or effort candidate when unambiguous."
+      "summary": "Select a visible model, intelligence, effort, or nested model-version candidate when unambiguous."
+    },
+    {
+      "args": {
+        "timeoutMs": "optional timeout for attaching to the ChatGPT page"
+      },
+      "blockers": [
+        "browser_bridge_unavailable",
+        "login_required",
+        "selector_drift"
+      ],
+      "defaults": {},
+      "defaultTimeoutMs": 30000,
+      "examples": [
+        "await chatgpt.modes.get();",
+        "// Verify an expensive Pro consult before submitting: const current = await chatgpt.modes.get();"
+      ],
+      "layer": "primitive",
+      "name": "modes.get",
+      "retryPolicy": "Return structured CommandResult failures; retry only when the blocker is recoverable and no duplicate prompt will be submitted.",
+      "risk": "low",
+      "summary": "Read the mode labels shown on the visible composer controls without changing them."
     },
     {
       "args": {},

package/contracts/v1/fixtures/doctor-bridge-upload.json

@@ -6,7 +6,15 @@
     "data": {
       "checks": {
         "bridge": {
+          "blockerKind": "browser_bridge_unavailable",
+          "code": "codex_chrome_bridge_unavailable",
+          "details": {
+            "category": "environment",
+            "severity": "blocked",
+            "userActionRequired": true
+          },
           "message": "Codex cannot access the ChatGPT browser bridge from this backend process. In an ordinary shell this is expected; for a live Codex Chrome run, bootstrap the Chrome plugin runtime with setupBrowserRuntime({ globals: globalThis }) before using globalThis.agent.",
+          "nextCommand": "session.bootstrap",
           "remediation": [
             "Ordinary shell: Treat browser_bridge_unavailable from a plain shell as an expected protocol/blocker-path result, not proof that Chrome, ChatGPT, or the Codex extension is broken.",
             "Codex Chrome bootstrap: For a live run, initialize the Chrome plugin runtime in node_repl with setupBrowserRuntime({ globals: globalThis }), then set globalThis.browser = await agent.browsers.get(\"extension\") before calling createChatGPT({ agent: globalThis.agent }).",