codex-copilot-dx 0.4.23 → 0.4.26

package/README.md

@@ -15,7 +15,7 @@ Supports both HTTP SSE streaming and non-streaming.
 ## Prerequisites
 
 - GitHub Copilot subscription (Individual, Business, or Enterprise)
-- Node.js 18+
+- Node.js 18.17+
 - [Codex Desktop](https://openai.com/codex), [Claude Code](https://claude.com/claude-code), and/or Claude App installed
 
 ## Usage
@@ -28,7 +28,7 @@ npx codex-copilot-dx@latest
 
 On first run, it will:
 1. Authenticate with GitHub via device flow (if needed), after first trying compatible local Copilot token sources
-2. Print the local package version and warn when a newer npm release is available
+2. Print the local package version and check for a newer npm release in the background
 3. Start the adapter on loopback (`127.0.0.1:2026`)
 4. Configure Codex (`~/.codex/config.toml`) to use the adapter, including stale shell env base URLs if present
 5. Configure Claude Code (`~/.claude/settings.json`) to use the adapter; it creates the file when missing, otherwise backs up `settings.json.bak` before updating the local API env keys
@@ -38,6 +38,8 @@ Claude Code picks up the new `ANTHROPIC_BASE_URL` on its next launch.
 
 If an existing `codex-copilot-dx` adapter is already running on the configured host and port, a second launch reuses it instead of starting another proxy. The second launch still refreshes Codex and Claude Code config, then exits.
 
+The running adapter reports its package and protocol versions. After upgrading `codex-copilot-dx`, stop the old process before starting the new version; the new CLI refuses to silently reuse an incompatible adapter.
+
 Do not set Claude Code by manually exporting `ANTHROPIC_BASE_URL` or `ANTHROPIC_AUTH_TOKEN` in your shell. Let `codex-copilot-dx` write the local config files instead. If you previously exported those variables, remove them from shell startup files and restart the terminal before launching Claude Code.
 
 ### Diagnostics
@@ -48,9 +50,19 @@ Run a read-only config check without starting the adapter or changing files:
 codex-copilot-dx doctor
 ```
 
+The command exits with status `1` when it finds an invalid configuration and `0` when checks contain only OK or warning results.
+
 The doctor checks the GitHub token, Codex config, Claude Code settings, Claude App gateway profile, and whether the local adapter port is listening.
 
-When the saved token is missing, `codex-copilot-dx` first looks for compatible local Copilot GitHub tokens, validates them with GitHub and Copilot, and imports a valid one before starting device login. It checks explicit token sources (`CCDX_GITHUB_TOKEN`, `CCDX_GITHUB_TOKEN_PATH`, `CCDX_GITHUB_TOKEN_PATHS`) plus common local `auth.json` layouts under application config directories. It does not rely on a specific app name. If a saved token later fails with `401` or `403`, the adapter repeats that discovery and retries once before asking you to reauthenticate.
+For a read-only live check of the saved GitHub token, Copilot entitlement, and models endpoint, run:
+
+```bash
+codex-copilot-dx doctor --online
+```
+
+The online doctor never starts device flow, scans for replacement tokens, or changes the saved token.
+
+When the saved token is missing, `codex-copilot-dx` first looks for compatible local Copilot GitHub tokens, validates them with GitHub and Copilot, and imports a valid one before starting device login. It checks explicit token sources (`CCDX_GITHUB_TOKEN`, `CCDX_GITHUB_TOKEN_PATH`, `CCDX_GITHUB_TOKEN_PATHS`) plus common local `auth.json` layouts under application config directories. It does not rely on a specific app name. Generic discovery refuses to choose silently when valid tokens for multiple GitHub accounts are found. After an account is selected, automatic `401`/`403` recovery accepts only the same GitHub account. Explicit token variables remain the intentional way to switch accounts.
 
 If Copilot token refresh still fails with `401` or `403`, the saved GitHub token may be expired, revoked, or missing Copilot access. Delete the saved token and start the tool again to trigger GitHub device login:
 
@@ -89,10 +101,13 @@ Environment variables:
 | `CCDX_MAX_BODY_BYTES` | `134217728` | Maximum compressed/raw request body size |
 | `CCDX_MAX_DECODED_BODY_BYTES` | `268435456` | Maximum decoded request body size after decompression |
 | `CCDX_UPSTREAM_TIMEOUT_MS` | `120000` | Timeout for non-streaming upstream Copilot requests |
+| `CCDX_STREAM_HANDSHAKE_TIMEOUT_MS` | `120000` | Timeout while waiting for upstream streaming response headers |
+| `CCDX_STREAM_IDLE_TIMEOUT_MS` | `120000` | Maximum idle time between upstream streaming body chunks |
 | `CCDX_UPSTREAM_RETRIES` | `2` | Retries for transient Copilot upstream network errors; capped at `5` |
 | `CCDX_UPSTREAM_RETRY_DELAY_MS` | `300` | Initial upstream retry backoff in milliseconds; capped at `5000` |
 | `CCDX_LOG_PATH` | unset | Mirror terminal logs to a file; set to `1` for `~/.local/share/codex-copilot-dx/debug.log` |
 | `CCDX_LOG_LEVEL` | `info` | Set to `debug` to include upstream request attempts, status codes, retry causes, and timings |
+| `CCDX_LOG_MAX_BYTES` | `16777216` | Rotate the debug log at this size, retaining one `.1` backup; set to `0` to disable rotation |
 | `CCDX_IMG_MAX_DIM` | `2048` | Max long edge in pixels for image downscaling |
 | `CCDX_IMG_QUALITY` | `85` | WebP quality used when re-encoding images |
 | `CCDX_IMG_MIN_BYTES` | `100000` | Images smaller than this are left untouched |
@@ -104,12 +119,18 @@ Environment variables:
 | `CCDX_GITHUB_TOKEN` | unset | Explicit GitHub Copilot OAuth token to validate and import before device login |
 | `CCDX_GITHUB_TOKEN_PATH` | unset | Explicit file containing a GitHub Copilot OAuth token to validate and import before device login |
 | `CCDX_GITHUB_TOKEN_PATHS` | unset | Multiple token files separated by the platform path delimiter (`:` on macOS/Linux, `;` on Windows) |
+| `CCDX_GITHUB_LOGIN` | saved account | Require automatic discovery and recovery to use this GitHub login |
 | `CCDX_DISABLE_TOKEN_DISCOVERY` | unset | Set to `1` to skip local token discovery and go straight to the saved token or device flow |
 | `CCDX_TOKEN_LOCK_TIMEOUT_MS` | `600000` | Maximum time to wait for another local process to finish GitHub token login/import |
 | `CCDX_TOKEN_LOCK_STALE_MS` | `900000` | Age after which a stale GitHub token lock file can be removed |
 | `CCDX_EXISTING_ADAPTER_TIMEOUT_MS` | `500` | Timeout for detecting an already-running local adapter during startup |
+| `CCDX_MODEL_REFRESH_INTERVAL_MS` | `1800000` | Interval for refreshing Copilot model metadata; successful lists are cached locally as last-known-good data |
+| `CCDX_RESPONSE_HISTORY_MAX_BYTES` | `67108864` | Total in-memory byte budget for locally expanded Responses history |
+| `CCDX_RESPONSE_HISTORY_MAX_ENTRIES` | `4096` | Maximum stored incremental Responses history nodes |
 | `CCDX_USAGE_PATH` | `~/.local/share/codex-copilot-dx/usage.jsonl` | Local JSONL token usage log |
+| `CCDX_USAGE_MAX_BYTES` | `33554432` | Rotate the usage log at this size, retaining one `.1` backup; set to `0` to disable rotation |
 | `CCDX_DISABLE_USAGE` | unset | Set to `1` to disable usage logging |
+| `CCDX_SHUTDOWN_TIMEOUT_MS` | `5000` | Time to drain active HTTP connections before forcing shutdown |
 
 ### Usage logging
 
@@ -129,6 +150,17 @@ Long computer-use sessions can accumulate screenshots inside the conversation hi
 
 The adapter automatically downsamples embedded screenshots to long-edge <= 2048 px and re-encodes them as WebP before forwarding `/v1/responses`.
 
+Newer ChatGPT/Codex clients can advertise an `image_gen` namespace that already exists upstream. The adapter removes that exact conflicting client tool before forwarding and retries once only when Copilot explicitly reports an image namespace collision. Image inputs and screenshot optimization remain enabled.
+
 ## License
 
 MIT
+
+## Development
+
+```bash
+npm ci
+npm run verify
+```
+
+`npm test` runs the unit and handler-level suite. `npm run test:smoke` starts a real local HTTP adapter with fully injected offline upstreams. `npm run pack:check` verifies the npm tarball contents without publishing. The CI workflow runs all three checks on supported Node.js release lines.

package/bin/cli.mjs

@@ -5,35 +5,93 @@ import { ensureCodexConfig } from "../src/config.mjs";
 import { ensureClaudeConfig } from "../src/claude-config.mjs";
 import { applyClaudeDesktopConfig, formatClaudeDesktopApplyResult, generatedClaudeDesktopApiKey } from "../src/claude-desktop-config.mjs";
 import { startAdapter } from "../src/adapter.mjs";
-import { listModels, refreshVSCodeVersion } from "../src/copilot.mjs";
+import { cacheModelEndpoints, listModels, refreshVSCodeVersion } from "../src/copilot.mjs";
 import { claudeDesktopModelDefsFromCopilotModels, claudeDesktopModelIds, gptModelIdsFromCopilotModels, parseModelAliasEnv } from "../src/models.mjs";
 import { status } from "../src/status.mjs";
 import { configureLogging } from "../src/log.mjs";
-import { printUsageSummary } from "../src/usage.mjs";
+import { flushUsageWrites, printUsageSummary } from "../src/usage.mjs";
 import { checkForUpdate, localPackageVersion } from "../src/version.mjs";
 import { runDoctor } from "../src/doctor.mjs";
 import { checkRunningAdapter } from "../src/running-adapter.mjs";
 import { assertSafeAdapterHost, isLoopbackHost } from "../src/security.mjs";
+import { loadModelCache, saveModelCache } from "../src/model-cache.mjs";
+import { initializeModelRegistry, runInBackground } from "../src/startup.mjs";
+import { cliHelp, parseCliArgs, parseRuntimeOptions } from "../src/cli-options.mjs";
+import { closeHttpServer } from "../src/shutdown.mjs";
 
-const ADAPTER_PORT = parseInt(process.env.ADAPTER_PORT || "2026");
-const ADAPTER_HOST = process.env.ADAPTER_HOST || "127.0.0.1";
-const MODEL_REFRESH_TIMEOUT_MS = parseInt(process.env.CCDX_MODEL_REFRESH_TIMEOUT_MS || "5000", 10);
-const EXISTING_ADAPTER_TIMEOUT_MS = parseInt(process.env.CCDX_EXISTING_ADAPTER_TIMEOUT_MS || "500", 10);
 const LOCAL_VERSION = localPackageVersion();
-const command = process.argv[2];
-const CONFIGURE_CLAUDE_DESKTOP = command === "--configure-claude-desktop" || process.env.CCDX_CONFIGURE_CLAUDE_DESKTOP === "1";
+
+let CLI;
+try {
+  CLI = parseCliArgs(process.argv.slice(2));
+} catch (e) {
+  console.error(e.message);
+  console.error("Run codex-copilot-dx --help for usage.");
+  process.exit(2);
+}
+
+if (CLI.command === "help") {
+  console.log(cliHelp());
+  process.exit(0);
+}
+if (CLI.command === "version") {
+  console.log(`codex-copilot-dx v${LOCAL_VERSION}`);
+  process.exit(0);
+}
+if (CLI.command === "usage") {
+  await printUsageSummary();
+  process.exit(0);
+}
+
+let RUNTIME;
+try {
+  RUNTIME = parseRuntimeOptions(process.env);
+} catch (e) {
+  console.error(e.message);
+  process.exit(2);
+}
+
+const ADAPTER_PORT = RUNTIME.adapterPort;
+const ADAPTER_HOST = RUNTIME.adapterHost;
+const MODEL_REFRESH_TIMEOUT_MS = RUNTIME.modelRefreshTimeoutMs;
+const EXISTING_ADAPTER_TIMEOUT_MS = RUNTIME.existingAdapterTimeoutMs;
+const CONFIGURE_CLAUDE_DESKTOP = CLI.configureClaudeDesktop || process.env.CCDX_CONFIGURE_CLAUDE_DESKTOP === "1";
 const LOGGING = configureLogging();
+const MODEL_REGISTRY = { modelDefs: undefined, source: "built-in" };
+let activeServer = null;
+let modelRefreshTimer = null;
+let shuttingDown = false;
 
 if (LOGGING.filePath) {
   console.log(status("info", `Debug log: ${LOGGING.filePath}`));
   if (LOGGING.level === "debug") console.log(status("debug", "Debug logging enabled"));
 }
 
-async function refreshClaudeDesktopModelDefs() {
-  if (parseModelAliasEnv(process.env.CCDX_CLAUDE_MODEL_ALIASES).length) {
-    console.log(status("info", "Using CCDX_CLAUDE_MODEL_ALIASES for Claude models"));
-    return undefined;
+function applyModelsToRegistry(models, source, { updateClaudeModels = true } = {}) {
+  const modelDefs = updateClaudeModels
+    ? claudeDesktopModelDefsFromCopilotModels(models)
+    : MODEL_REGISTRY.modelDefs;
+  if (updateClaudeModels && !modelDefs.length) throw new Error("Copilot models response contained no Claude models");
+  cacheModelEndpoints(models);
+  if (updateClaudeModels) MODEL_REGISTRY.modelDefs = modelDefs;
+  MODEL_REGISTRY.source = source;
+  return modelDefs;
+}
+
+function loadCachedModelRegistry() {
+  const cached = loadModelCache();
+  if (!cached) return false;
+  try {
+    const customAliases = parseModelAliasEnv(process.env.CCDX_CLAUDE_MODEL_ALIASES).length > 0;
+    applyModelsToRegistry(cached, "cache", { updateClaudeModels: !customAliases });
+    return true;
+  } catch {
+    return false;
   }
+}
+
+async function refreshClaudeDesktopModelDefs() {
+  const customAliases = parseModelAliasEnv(process.env.CCDX_CLAUDE_MODEL_ALIASES).length > 0;
 
   const ctrl = new AbortController();
   const timer = setTimeout(() => ctrl.abort(), MODEL_REFRESH_TIMEOUT_MS);
@@ -50,13 +108,22 @@ async function refreshClaudeDesktopModelDefs() {
       console.log(status("warn", "Copilot models response contained no GPT models"));
     }
 
-    const modelDefs = claudeDesktopModelDefsFromCopilotModels(models);
-    if (!modelDefs.length) throw new Error("Copilot models response contained no Claude models");
-    console.log(status("ok", `Refreshed Claude models from GitHub Copilot: ${modelDefs.map((model) => model.id).join(", ")}`));
+    const modelDefs = applyModelsToRegistry(models, "live", { updateClaudeModels: !customAliases });
+    try {
+      saveModelCache(models);
+    } catch (e) {
+      console.log(status("warn", `Could not persist the Copilot model cache (${e.message})`));
+    }
+    if (customAliases) {
+      console.log(status("info", "Using CCDX_CLAUDE_MODEL_ALIASES for Claude models"));
+    } else {
+      console.log(status("ok", `Refreshed Claude models from GitHub Copilot: ${modelDefs.map((model) => model.id).join(", ")}`));
+    }
     return modelDefs;
   } catch (e) {
-    console.log(status("warn", `Could not refresh Claude models; using built-in model list (${e.message})`));
-    return undefined;
+    const fallback = MODEL_REGISTRY.modelDefs?.length ? `${MODEL_REGISTRY.source} model list` : "built-in model list";
+    console.log(status("warn", `Could not refresh Claude models; using ${fallback} (${e.message})`));
+    return MODEL_REGISTRY.modelDefs;
   } finally {
     clearTimeout(timer);
   }
@@ -72,6 +139,10 @@ async function reuseRunningAdapterIfAvailable() {
     host: ADAPTER_HOST,
     timeoutMs: EXISTING_ADAPTER_TIMEOUT_MS,
   });
+  if (running.incompatible) {
+    const found = running.data?.version || "unknown";
+    throw new Error(`Adapter ${found} is already running at ${running.baseUrl}, but this CLI is ${LOCAL_VERSION}. Stop the existing process and run codex-copilot-dx again.`);
+  }
   if (!running.ok) return false;
 
   console.log(status("ok", `Using existing adapter at ${running.baseUrl}`));
@@ -105,19 +176,13 @@ async function reuseRunningAdapterIfAvailable() {
   return true;
 }
 
-if (command === "--version" || command === "-v" || command === "version") {
-  console.log(`codex-copilot-dx v${LOCAL_VERSION}`);
-  process.exit(0);
-}
-
-if (command === "usage") {
-  await printUsageSummary();
-  process.exit(0);
-}
-
-if (command === "doctor" || command === "--doctor") {
-  await runDoctor({ port: ADAPTER_PORT, host: ADAPTER_HOST });
-  process.exit(0);
+if (CLI.command === "doctor") {
+  const checks = await runDoctor({
+    port: ADAPTER_PORT,
+    host: ADAPTER_HOST,
+    online: CLI.online,
+  });
+  process.exit(checks.some((check) => check.kind === "err") ? 1 : 0);
 }
 
 console.log(`
@@ -136,20 +201,22 @@ async function printUpdateNotice() {
   }
 }
 
-// Await the update check up front so the notice is shown even on the
-// "reuse existing adapter" path, which exits early below.
-await printUpdateNotice();
-
 try {
   assertSafeAdapterHost(ADAPTER_HOST, process.env);
+  void runInBackground(printUpdateNotice);
   if (await reuseRunningAdapterIfAvailable()) process.exit(0);
 
   // Ensure GitHub login, using device flow if no token exists.
   await ensureAuth();
 
   // Refresh the VS Code version in the background; fallback is non-blocking.
-  refreshVSCodeVersion();
-  const claudeDesktopModelDefs = await refreshClaudeDesktopModelDefs();
+  void refreshVSCodeVersion();
+  const modelInitialization = await initializeModelRegistry({
+    loadCached: loadCachedModelRegistry,
+    currentModelDefs: () => MODEL_REGISTRY.modelDefs,
+    refresh: refreshClaudeDesktopModelDefs,
+  });
+  const claudeDesktopModelDefs = modelInitialization.modelDefs;
 
   if (!isLoopbackHost(ADAPTER_HOST)) {
     console.log(status("warn", `ADAPTER_HOST=${ADAPTER_HOST} exposes the adapter beyond loopback because CCDX_ALLOW_LAN=1 is set. Use only on trusted networks.`));
@@ -160,7 +227,14 @@ try {
     : currentClaudeDesktopApiKey();
 
   // Start the in-process adapter.
-  await startAdapter(ADAPTER_PORT, ADAPTER_HOST, { claudeDesktopApiKey, claudeDesktopModelDefs });
+  activeServer = await startAdapter(ADAPTER_PORT, ADAPTER_HOST, {
+    claudeDesktopApiKey,
+    claudeDesktopModelDefs,
+    modelRegistry: MODEL_REGISTRY,
+    upstreamTimeoutMs: RUNTIME.upstreamTimeoutMs,
+    streamHandshakeTimeoutMs: RUNTIME.streamHandshakeTimeoutMs,
+    streamIdleTimeoutMs: RUNTIME.streamIdleTimeoutMs,
+  });
 
   // Point Codex and Claude Code at the adapter.
   ensureCodexConfig(ADAPTER_PORT);
@@ -183,12 +257,12 @@ try {
 
   // Periodically refresh the model list + endpoint cache so a long-running
   // adapter picks up newly released models without a restart.
-  const refreshIntervalMs = parseInt(process.env.CCDX_MODEL_REFRESH_INTERVAL_MS || String(30 * 60 * 1000), 10);
+  const refreshIntervalMs = RUNTIME.modelRefreshIntervalMs;
   if (Number.isFinite(refreshIntervalMs) && refreshIntervalMs > 0) {
-    const timer = setInterval(() => {
+    modelRefreshTimer = setInterval(() => {
       refreshClaudeDesktopModelDefs().catch(() => {});
     }, refreshIntervalMs);
-    if (typeof timer.unref === "function") timer.unref();
+    modelRefreshTimer.unref?.();
   }
 
   console.log(`
@@ -199,11 +273,30 @@ try {
   Press Ctrl+C to stop.
 `);
 
-  process.on("SIGINT", () => {
-    console.log(`\n${status("wait", "Shutting down...")}`);
-    process.exit(0);
-  });
+  const shutdown = async (signal) => {
+    if (shuttingDown) return;
+    shuttingDown = true;
+    console.log(`\n${status("wait", `Shutting down on ${signal}...`)}`);
+    let exitCode = 0;
+    try {
+      if (modelRefreshTimer) clearInterval(modelRefreshTimer);
+      const result = await closeHttpServer(activeServer, { timeoutMs: RUNTIME.shutdownTimeoutMs });
+      if (result.forced) console.warn(status("warn", "Forced remaining adapter connections closed"));
+      await flushUsageWrites();
+    } catch (e) {
+      exitCode = 1;
+      console.error(status("err", `Shutdown failed: ${e.message}`));
+    } finally {
+      LOGGING.cleanup();
+      process.exit(exitCode);
+    }
+  };
+  process.once("SIGINT", () => { void shutdown("SIGINT"); });
+  process.once("SIGTERM", () => { void shutdown("SIGTERM"); });
 } catch (e) {
   console.error(status("err", e.message));
+  await closeHttpServer(activeServer, { timeoutMs: RUNTIME.shutdownTimeoutMs }).catch(() => {});
+  await flushUsageWrites();
+  LOGGING.cleanup();
   process.exit(1);
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "codex-copilot-dx",
-  "version": "0.4.23",
+  "version": "0.4.26",
   "description": "Use Codex Desktop, Claude Code, and Claude App with GitHub Copilot",
   "type": "module",
   "bin": {
@@ -25,8 +25,15 @@
     "responses-api"
   ],
   "license": "MIT",
+  "engines": {
+    "node": ">=18.17"
+  },
   "scripts": {
-    "test": "node --test"
+    "test": "node --test",
+    "test:smoke": "node scripts/offline-smoke.mjs",
+    "test:e2e": "bash test/e2e.sh",
+    "pack:check": "npm pack --dry-run --json",
+    "verify": "npm test && npm run test:smoke && npm run pack:check"
   },
   "dependencies": {
     "gpt-tokenizer": "^3.4.0",