impel-cli 0.20.14 → 0.20.16

package/README.md

@@ -180,6 +180,21 @@ impel claude --model opus
 impel codex exec "review this repository"
 ```
 
+To run one explicitly selected tenant agent in a single Codex thread, select
+it by exact agent ID (preferred) or unique exact title:
+
+```sh
+impel codex --agent research-agent exec "answer this question"
+```
+
+This path loads a generated, tenant-bound profile with Luna/low reasoning, a
+read-only local sandbox, the exact fixed MCP binding, and only that binding's
+answer/run, resume, and recovery tools. It omits the normal parent specialist
+instructions, so there is no parent spawn, parent wait, or relay turn. Model,
+profile, developer-instruction, MCP, approval, sandbox, and feature overrides
+are rejected with `--agent`. The selected tenant and profile integrity must
+match before Codex starts. Ordinary `impel codex` passthrough is unchanged.
+
 ## Remote Fargate sessions
 
 Remote operations use the tenant broker. The laptop sends its PAT only to that
@@ -496,6 +511,7 @@ impel setup
 impel update
 impel claude
 impel codex
+impel codex --agent <id|exact-title> ...
 impel tenant list|current|use
 impel tasks ...
 impel pat ...

package/RELEASE_NOTES.md

@@ -1,5 +1,27 @@
 # Release notes
 
+## 0.20.16 — Correlated Codex transport timing
+
+- Derives native-agent MCP duration from correlated, timestamped Codex rollout
+  events when durable profiles correctly omit transient telemetry destinations.
+- Retries one unestablished tenant-preflight result while still requiring one
+  exact selected tenant before every profiled Codex attempt.
+
+## 0.20.15 — Direct Codex native agents
+
+- Classifies the exact `mcp__impel_agent` namespace as direct-only in generated
+  Codex 0.146.0 agents, removing child code cells and code-mode waits while
+  preserving the fixed tool allowlist and recovery contract.
+- Adds `impel codex --agent <id|exact-title>` with integrity-tracked top-level
+  profiles, removing the parent spawn/wait/relay thread for explicit CLI runs.
+- Bumps managed agent and app profile contracts so already-fresh isolated CLI
+  and ChatGPT profiles regenerate immediately after update.
+- Adds a private-by-default production profiler and structural analyzer with
+  exact-tenant checks, strict JSONL, process-tree cancellation, original versus
+  replacement cohorts, and prompt/answer-free aggregate reports.
+- Refreshes the exact Windows ChatGPT/Codex Store build after the production
+  manifest advanced during release qualification.
+
 ## 0.20.14 — Opaque direct-answer continuations
 
 - Returns a minimal local continuation locator for read-only direct answers,

package/docs/native-agent-host-capability-matrix.md

@@ -8,8 +8,8 @@ is treated as an eager host.
 | --- | --- | --- | --- | --- |
 | Claude Code | 2.1.212 | Yes, agent frontmatter `tools` | Eager | The bound run/answer, resume, and recovery tools are direct child tools. No catalog discovery or code-cell polling is required. |
 | Claude Desktop | 1.24012.9 (embedded Code 2.1.219) | Yes, agent frontmatter `tools` | Eager | Same direct-tool contract as Claude Code. |
-| Codex CLI | 0.146.0 | No; MCP tools remain code-mode-only | Durable fallback | `enabled_tools` narrows the server contract but the host still invokes through code mode. Discovery/polling overhead is measured separately and is not classified as eager completion. |
-| ChatGPT desktop Codex | 26.727.40816 (embedded Codex alpha) | No; MCP tools remain code-mode-only | Durable fallback | Uses the same bounded durable composite and recovery state as Codex CLI. |
+| Codex CLI through `impel codex` | 0.146.0 | Yes, with `features.code_mode.direct_only_tool_namespaces = ["mcp__impel_agent"]` | Eager; optional single-thread profile | Existing `@Agent` children call the fixed MCP namespace directly. `impel codex --agent` also removes the parent spawn/wait/relay thread. |
+| ChatGPT desktop Codex | 26.727.40816 (embedded Codex 0.146.0 contract) | Yes, with the same exact native namespace | Eager child, compatible parent | Keeps the app's supported `@Agent` parent/child route, but the child no longer needs code cells or code-mode waits. |
 
 ## Qualified contract
 
@@ -28,11 +28,19 @@ is treated as an eager host.
   decrypted provider material. Production measurements use correlation ids,
   phase timestamps, counts, durations, build identifiers, and terminal classes
   only.
+- Top-level direct profiles exist only in Impel-isolated Codex and managed
+  ChatGPT homes. Bare/native Codex homes do not receive the 0.146.0-only key or
+  the new top-level profiles. `impel codex` resolves only the reviewed pinned
+  binary; an older or newer unqualified build must be updated and re-qualified
+  rather than receiving an unknown configuration key.
+- `mcp__impel_agent` is the native tool namespace. The server id
+  `impel_agent` is not equivalent and does not qualify as direct exposure.
 
 ## Requalification
 
-Re-run temporary-home profile parsing, direct tool visibility, cancellation,
+Re-run temporary-home profile parsing (including active, direct-answer,
+durable, and recovery profiles), direct tool visibility, cancellation,
 typed continuation, byte-exact relay, and the 10/30/45/60/120-second bounded
-wait fixture after a pinned client changes. A build stays on durable fallback
-unless the child trace has zero `ALL_TOOLS`, `functions.exec`, and
-`functions.wait` entries.
+wait fixture after a pinned client changes. A build stays fail-closed without
+the direct-only key unless the child trace has zero `ALL_TOOLS`, `exec`, and
+`wait` entries.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "impel-cli",
-  "version": "0.20.14",
+  "version": "0.20.16",
   "description": "Prepare isolated Claude and Codex workspaces for every accessible Impel tenant",
   "type": "module",
   "bin": {
@@ -15,6 +15,7 @@
     "bin",
     "docs/experimental-managed-cursor.md",
     "docs/native-agent-host-capability-matrix.md",
+    "scripts",
     "src",
     "README.md"
   ],
@@ -31,6 +32,8 @@
     "url": "git+https://github.com/UseImpel/impel-cli.git"
   },
   "scripts": {
+    "analyze:native-codex": "node scripts/analyze-native-codex.mjs",
+    "profile:native-codex": "node scripts/profile-native-codex.mjs",
     "test": "node --test"
   }
 }

package/scripts/analyze-native-codex.mjs

@@ -0,0 +1,454 @@
+#!/usr/bin/env node
+
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const AGGREGATE_SCHEMA = "impel.native-codex-profile.v1";
+const REPORT_SCHEMA = "impel.native-codex-analysis.v1";
+const SAFE_MCP_TOOLS = new Set([
+  "answer_native_agent",
+  "run_native_agent",
+  "resume_native_agent_run",
+  "recover_native_agent_runs",
+]);
+const SAFE_TOOL_NAMES = new Set([
+  ...SAFE_MCP_TOOLS,
+  "spawn_agent",
+  "wait_agent",
+  "send_message",
+  "followup_task",
+  "interrupt_agent",
+  "exec",
+  "wait",
+  "tool_search",
+  "ALL_TOOLS",
+]);
+
+function finiteNonnegative(value) {
+  return Number.isFinite(value) && value >= 0 ? value : null;
+}
+
+function integerNonnegative(value) {
+  return Number.isSafeInteger(value) && value >= 0 ? value : null;
+}
+
+export function parseStrictJsonLines(text, label = "JSONL") {
+  const records = [];
+  for (const [index, line] of text.split(/\r?\n/u).entries()) {
+    if (!line.trim()) continue;
+    let value;
+    try {
+      value = JSON.parse(line);
+    } catch {
+      throw new Error(`${label} line ${index + 1} is not valid JSON`);
+    }
+    if (!value || typeof value !== "object" || Array.isArray(value)) {
+      throw new Error(`${label} line ${index + 1} is not a JSON object`);
+    }
+    records.push(value);
+  }
+  return records;
+}
+
+function safeEventType(value) {
+  return typeof value === "string" && /^[A-Za-z0-9_.:-]{1,120}$/u.test(value)
+    ? value
+    : null;
+}
+
+function safeToolName(value) {
+  if (typeof value !== "string") return null;
+  const unprefixed = value.replace(/^mcp__impel_agent__/u, "");
+  return SAFE_TOOL_NAMES.has(unprefixed) ? unprefixed : null;
+}
+
+function eventItem(event) {
+  return event.item && typeof event.item === "object" && !Array.isArray(event.item)
+    ? event.item
+    : event;
+}
+
+function eventTool(item) {
+  const server = item.server || item.server_name || item.mcp_server;
+  const tool = item.tool || item.tool_name || item.name;
+  if (server === "impel_agent" && SAFE_MCP_TOOLS.has(tool)) return tool;
+  return safeToolName(tool);
+}
+
+function increment(counts, key) {
+  if (key) counts[key] = (counts[key] || 0) + 1;
+}
+
+export function summarizeCodexEvents(events) {
+  const eventTypes = {};
+  const toolCounts = {};
+  const threads = new Set();
+  let inputTokens = 0;
+  let cachedInputTokens = 0;
+  let outputTokens = 0;
+  let turnCompleted = false;
+  let errorEvents = 0;
+  let codeCells = 0;
+  let codeWaits = 0;
+
+  for (const event of events) {
+    const type = safeEventType(event.type);
+    increment(eventTypes, type);
+    const threadId = event.thread_id || event.threadId;
+    if (typeof threadId === "string" && /^[A-Za-z0-9_.:-]{1,200}$/u.test(threadId)) threads.add(threadId);
+    if (type === "turn.completed") turnCompleted = true;
+    if (type?.includes("error") || event.error) errorEvents += 1;
+
+    const item = eventItem(event);
+    const itemType = safeEventType(item.type);
+    const tool = eventTool(item);
+    increment(toolCounts, tool);
+    if (itemType && /(?:code_mode|command_execution|code_cell)/u.test(itemType)) codeCells += 1;
+    if (tool === "exec") codeCells += 1;
+    if (tool === "wait") codeWaits += 1;
+
+    const usage = event.usage && typeof event.usage === "object" ? event.usage : null;
+    if (usage) {
+      inputTokens = Math.max(inputTokens, integerNonnegative(usage.input_tokens) || 0);
+      cachedInputTokens = Math.max(cachedInputTokens, integerNonnegative(usage.cached_input_tokens) || 0);
+      outputTokens = Math.max(outputTokens, integerNonnegative(usage.output_tokens) || 0);
+    }
+  }
+
+  return {
+    eventCount: events.length,
+    eventTypes,
+    toolCounts,
+    threadCount: threads.size,
+    turnCompleted,
+    errorEvents,
+    codeCells,
+    codeWaits,
+    inputTokens,
+    cachedInputTokens,
+    outputTokens,
+  };
+}
+
+function rolloutSessionMetadata(events) {
+  const metadata = events.find((event) => event.type === "session_meta")?.payload;
+  return metadata && typeof metadata === "object" ? {
+    threadId: typeof (metadata.session_id || metadata.id) === "string"
+      ? metadata.session_id || metadata.id
+      : null,
+    parentThreadId: typeof metadata.parent_thread_id === "string" ? metadata.parent_thread_id : null,
+  } : { threadId: null, parentThreadId: null };
+}
+
+export function summarizeCodexRollouts(rollouts) {
+  const toolCounts = {};
+  const parentToolCounts = {};
+  const childToolCounts = {};
+  let eventCount = 0;
+  let codeCells = 0;
+  let codeWaits = 0;
+  let inputTokens = 0;
+  let cachedInputTokens = 0;
+  let outputTokens = 0;
+  let mcpDurationMs = 0;
+  let mcpCompletedCalls = 0;
+
+  for (const events of rollouts) {
+    eventCount += events.length;
+    const metadata = rolloutSessionMetadata(events);
+    const scopedCounts = metadata.parentThreadId ? childToolCounts : parentToolCounts;
+    let latestUsage = null;
+    const mcpStarts = new Map();
+    for (const event of events) {
+      const payload = event.payload && typeof event.payload === "object" ? event.payload : {};
+      if (event.type === "response_item"
+        && ["function_call", "custom_tool_call"].includes(payload.type)) {
+        const tool = safeToolName(payload.name);
+        increment(toolCounts, tool);
+        increment(scopedCounts, tool);
+        if (tool === "exec" || payload.type === "custom_tool_call") codeCells += tool === "exec" ? 1 : 0;
+        if (tool === "wait") codeWaits += 1;
+        const startedAt = Date.parse(event.timestamp || "");
+        if (SAFE_MCP_TOOLS.has(tool)
+          && typeof payload.call_id === "string"
+          && Number.isFinite(startedAt)) {
+          mcpStarts.set(payload.call_id, startedAt);
+        }
+      }
+      if (event.type === "event_msg"
+        && payload.type === "mcp_tool_call_end"
+        && typeof payload.call_id === "string") {
+        const startedAt = mcpStarts.get(payload.call_id);
+        const endedAt = Date.parse(event.timestamp || "");
+        const duration = endedAt - startedAt;
+        if (Number.isFinite(startedAt)
+          && Number.isFinite(endedAt)
+          && duration >= 0
+          && duration <= 24 * 60 * 60 * 1000) {
+          mcpDurationMs += duration;
+          mcpCompletedCalls += 1;
+          mcpStarts.delete(payload.call_id);
+        }
+      }
+      if (event.type === "event_msg" && payload.type === "token_count") {
+        const usage = payload.info?.total_token_usage;
+        if (usage && typeof usage === "object") latestUsage = usage;
+      }
+    }
+    if (latestUsage) {
+      inputTokens += integerNonnegative(latestUsage.input_tokens) || 0;
+      cachedInputTokens += integerNonnegative(latestUsage.cached_input_tokens) || 0;
+      outputTokens += integerNonnegative(latestUsage.output_tokens) || 0;
+    }
+  }
+
+  return {
+    eventCount,
+    threadCount: rollouts.length,
+    toolCounts,
+    parentToolCounts,
+    childToolCounts,
+    codeCells,
+    codeWaits,
+    inputTokens,
+    cachedInputTokens,
+    outputTokens,
+    mcpDurationMs,
+    mcpCompletedCalls,
+  };
+}
+
+export function summarizeNativeTelemetry(events) {
+  const eventCounts = {};
+  const toolCounts = {};
+  const correlations = new Set();
+  const gatewayRequests = new Set();
+  let mcpDurationMs = 0;
+  let localFailures = 0;
+  let upstreamDurationMs = 0;
+  for (const event of events) {
+    if (event.schema !== "impel.native-agent-telemetry.v1") {
+      throw new Error("native-agent telemetry has an unexpected schema");
+    }
+    const type = safeEventType(event.event);
+    increment(eventCounts, type);
+    const tool = safeToolName(event.tool);
+    increment(toolCounts, tool);
+    if (typeof event.correlationId === "string") correlations.add(event.correlationId);
+    if (typeof event.gatewayRequestId === "string") gatewayRequests.add(event.gatewayRequestId);
+    const duration = integerNonnegative(event.durationMs) || 0;
+    if (type === "local_tool_completed" && SAFE_MCP_TOOLS.has(tool)) {
+      mcpDurationMs += duration;
+      if (["failed", "cancelled"].includes(event.outcome)) localFailures += 1;
+    }
+    if (type === "upstream_request_completed") upstreamDurationMs += duration;
+  }
+  return {
+    eventCount: events.length,
+    eventCounts,
+    toolCounts,
+    correlationCount: correlations.size,
+    gatewayRequestCount: gatewayRequests.size,
+    mcpDurationMs,
+    upstreamDurationMs,
+    localFailures,
+  };
+}
+
+export function summarizeRawAttempt({ stdoutText, telemetryText = "", rolloutTexts = [] }) {
+  const codexEvents = parseStrictJsonLines(stdoutText, "Codex stdout");
+  const telemetryEvents = telemetryText.trim()
+    ? parseStrictJsonLines(telemetryText, "native-agent telemetry")
+    : [];
+  const stdout = summarizeCodexEvents(codexEvents);
+  const rollouts = rolloutTexts.map((text, index) => parseStrictJsonLines(text, `Codex rollout ${index + 1}`));
+  const rollout = summarizeCodexRollouts(rollouts);
+  return {
+    codex: {
+      ...stdout,
+      ...(rollouts.length ? {
+        eventCount: rollout.eventCount,
+        toolCounts: rollout.toolCounts,
+        threadCount: rollout.threadCount,
+        codeCells: rollout.codeCells,
+        codeWaits: rollout.codeWaits,
+        inputTokens: rollout.inputTokens,
+        cachedInputTokens: rollout.cachedInputTokens,
+      outputTokens: rollout.outputTokens,
+        mcpDurationMs: rollout.mcpDurationMs,
+        mcpCompletedCalls: rollout.mcpCompletedCalls,
+      } : {}),
+      parentToolCounts: rollout.parentToolCounts,
+      childToolCounts: rollout.childToolCounts,
+    },
+    telemetry: summarizeNativeTelemetry(telemetryEvents),
+  };
+}
+
+function percentile(values, probability) {
+  if (values.length === 0) return null;
+  const sorted = [...values].sort((left, right) => left - right);
+  if (sorted.length === 1) return sorted[0];
+  const position = (sorted.length - 1) * probability;
+  const lower = Math.floor(position);
+  const upper = Math.ceil(position);
+  const weight = position - lower;
+  return Math.round((sorted[lower] * (1 - weight) + sorted[upper] * weight) * 1000) / 1000;
+}
+
+function distribution(attempts, field) {
+  const values = attempts.map((attempt) => finiteNonnegative(attempt[field])).filter((value) => value !== null);
+  return { samples: values.length, p50: percentile(values, 0.5), p95: percentile(values, 0.95) };
+}
+
+function mergedCounts(attempts, field) {
+  const counts = {};
+  for (const attempt of attempts) {
+    for (const [key, value] of Object.entries(attempt[field] || {})) {
+      if (SAFE_TOOL_NAMES.has(key) && Number.isSafeInteger(value) && value >= 0) {
+        counts[key] = (counts[key] || 0) + value;
+      }
+    }
+  }
+  return counts;
+}
+
+function groupKey(attempt) {
+  return `${attempt.tenantId}\0${attempt.sloClass}\0${attempt.mode}`;
+}
+
+function directGate(report) {
+  return {
+    reliability: report.original.total === 30 && report.original.successful === 30,
+    e2e: report.timings.durationMs.p50 !== null
+      && report.timings.durationMs.p50 <= (report.sloClass === "cibi" ? 40_000 : 90_000)
+      && report.timings.durationMs.p95 <= (report.sloClass === "cibi" ? 50_000 : 105_000),
+    overhead: report.timings.nonMcpOverheadMs.p50 !== null
+      && report.timings.nonMcpOverheadMs.p50 <= 25_000
+      && report.timings.nonMcpOverheadMs.p95 <= 35_000,
+    input: report.tokens.input.p50 !== null
+      && report.tokens.input.p50 <= 45_000
+      && report.tokens.input.p95 <= 55_000,
+    structure: report.structure.maximumThreadCount <= 1
+      && report.structure.totalCodeCells === 0
+      && report.structure.totalCodeWaits === 0
+      && ["spawn_agent", "wait_agent", "send_message", "exec", "wait", "tool_search", "ALL_TOOLS"]
+        .every((tool) => !report.structure.allToolCounts[tool])
+      && (report.structure.toolCounts.answer_native_agent || 0) === report.successfulSamples
+      && !report.structure.toolCounts.run_native_agent,
+  };
+}
+
+function compatibleGate(report) {
+  return {
+    reliability: report.original.total === 30 && report.original.successful >= 29,
+    e2e: report.timings.durationMs.p50 !== null
+      && report.timings.durationMs.p50 <= (report.sloClass === "cibi" ? 55_000 : 100_000),
+    childStructure: report.structure.totalCodeCells === 0
+      && report.structure.totalCodeWaits === 0
+      && ["exec", "wait", "send_message", "tool_search", "ALL_TOOLS"]
+        .every((tool) => !report.structure.childToolCounts[tool]),
+  };
+}
+
+export function analyzeAggregates(aggregates) {
+  for (const aggregate of aggregates) {
+    if (!aggregate || aggregate.schema !== AGGREGATE_SCHEMA || !Array.isArray(aggregate.attempts)) {
+      throw new Error("profile input has an unexpected schema");
+    }
+  }
+  const groups = new Map();
+  for (const attempt of aggregates.flatMap((aggregate) => aggregate.attempts)) {
+    if (!attempt || typeof attempt !== "object") throw new Error("profile input has an invalid attempt");
+    const key = groupKey(attempt);
+    if (!groups.has(key)) groups.set(key, []);
+    groups.get(key).push(attempt);
+  }
+
+  const reports = [...groups.values()].map((attempts) => {
+    const original = attempts.filter((attempt) => attempt.cohort === "original");
+    const replacements = attempts.filter((attempt) => attempt.cohort === "replacement");
+    const successful = attempts.filter((attempt) => attempt.success === true);
+    const report = {
+      tenantId: attempts[0].tenantId,
+      sloClass: attempts[0].sloClass,
+      mode: attempts[0].mode,
+      original: {
+        total: original.length,
+        successful: original.filter((attempt) => attempt.success === true).length,
+      },
+      replacements: {
+        total: replacements.length,
+        successful: replacements.filter((attempt) => attempt.success === true).length,
+      },
+      latencySamples: {
+        original: original.filter((attempt) => attempt.success === true).length,
+        replacement: replacements.filter((attempt) => attempt.success === true).length,
+      },
+      successfulSamples: successful.length,
+      timings: {
+        durationMs: distribution(successful, "durationMs"),
+        mcpDurationMs: distribution(successful, "mcpDurationMs"),
+        nonMcpOverheadMs: distribution(successful, "nonMcpOverheadMs"),
+      },
+      tokens: {
+        input: distribution(successful, "inputTokens"),
+        cachedInput: distribution(successful, "cachedInputTokens"),
+        output: distribution(successful, "outputTokens"),
+      },
+      structure: {
+        toolCounts: mergedCounts(successful, "toolCounts"),
+        allToolCounts: mergedCounts(attempts, "toolCounts"),
+        parentToolCounts: mergedCounts(successful, "parentToolCounts"),
+        childToolCounts: mergedCounts(successful, "childToolCounts"),
+        maximumThreadCount: successful.reduce((maximum, attempt) => Math.max(maximum, attempt.threadCount || 0), 0),
+        totalCodeCells: successful.reduce((total, attempt) => total + (attempt.codeCells || 0), 0),
+        totalCodeWaits: successful.reduce((total, attempt) => total + (attempt.codeWaits || 0), 0),
+      },
+      failures: {
+        timeout: attempts.filter((attempt) => attempt.timedOut).length,
+        invalidJson: attempts.filter((attempt) => !attempt.jsonValid).length,
+        descendantsRemaining: attempts.filter((attempt) => attempt.descendantsRemaining === true).length,
+      },
+    };
+    report.gates = report.mode === "direct" ? directGate(report) : compatibleGate(report);
+    report.gatesPassed = Object.values(report.gates).every(Boolean);
+    return report;
+  }).sort((left, right) => left.tenantId.localeCompare(right.tenantId) || left.mode.localeCompare(right.mode));
+
+  return { schema: REPORT_SCHEMA, generatedAt: new Date().toISOString(), reports };
+}
+
+function parseArguments(argv) {
+  const inputs = [];
+  for (let index = 0; index < argv.length; index += 1) {
+    if (argv[index] === "--input") {
+      const value = argv[index += 1];
+      if (!value) throw new Error("--input requires an aggregate JSON path");
+      inputs.push(value);
+    } else if (argv[index] === "--help" || argv[index] === "-h") {
+      return { help: true, inputs: [] };
+    } else {
+      inputs.push(argv[index]);
+    }
+  }
+  if (inputs.length === 0) throw new Error("provide at least one aggregate JSON path");
+  return { help: false, inputs };
+}
+
+async function main(argv) {
+  const parsed = parseArguments(argv);
+  if (parsed.help) {
+    process.stdout.write("Usage: node scripts/analyze-native-codex.mjs --input <aggregate.json> [...]\n");
+    return;
+  }
+  const aggregates = parsed.inputs.map((input) => JSON.parse(fs.readFileSync(path.resolve(input), "utf8")));
+  process.stdout.write(`${JSON.stringify(analyzeAggregates(aggregates), null, 2)}\n`);
+}
+
+const isMain = process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
+if (isMain) main(process.argv.slice(2)).catch((error) => {
+  process.stderr.write(`analyze-native-codex: ${error.message}\n`);
+  process.exitCode = 1;
+});