@link-assistant/hive-mind 2.13.4 → 2.14.0

package/CHANGELOG.md

@@ -1,5 +1,34 @@
 # @link-assistant/hive-mind
 
+## 2.14.0
+
+### Minor Changes
+
+- e796988: Add the experimental `--use-router` option, which stops handing a Docker-isolated task the operator's real subscription. With the flag, `~/.claude`, `~/.claude.json`, `~/.codex` and `~/.agents` are mounted only into a reference-counted `hive-mind-router` sidecar (pinned to `ghcr.io/link-assistant/router:0.119.0`) on an internal Docker network; each task is joined to that network and given its own `la_sk_…` token scoped to the one repository it works on, so every model request lands in that token's own redacted log. The sidecar runs while at least one task holds a lease and is stopped when the last one is released, with the Telegram bot reconciling leases against Docker so a killed bot cannot leave it running forever. At end of task the agent's session data is drained into the router's data volume for audit, and that volume is never removed by any code path.
+
+  GitHub is routed too: the sidecar terminates TLS itself with a certificate covering `api.github.com`, and the task container resolves that name to the router, so an unmodified `gh` is mediated without holding a GitHub credential of its own. `git` pushes through the router's `/git/*` proxy, which refuses remote-ref deletions and, from router `0.110.0`, non-fast-forward updates as well — it asks GitHub's compare API whether the proposed tip is ahead and fails closed when it cannot tell. A read-only `pre-push` hook in the task covers the same ground locally and for non-routed remotes. `--model formal-ai` is served through the router as a stored OpenAI-compatible provider, so it is audited like every other model.
+
+  Nothing changes without the flag. The remaining limits are documented and printed before every routed run: destructive GitHub API calls are blocked by HTTP method rather than by effect, so `PUT` branch protection, `PUT` rulesets, `POST` transfer and `PATCH` visibility/archived/default_branch are still forwarded (upstream `link-assistant/router#329`) — which puts branch protection itself within reach of a routed token; the Formal AI sidecar's own upstream calls are not routed; and the router resolves exact model ids only, not aliases, so the tier-shaped surface (`--plan`, `--escalate`, fallback chains) needs dated ids on a routed run. New docs: `docs/ROUTER.md`, `docs/COLLECTING-LOGS.md`, `docs/case-studies/issue-2164/` and `examples/collect-logs.mjs`.
+
+## 2.13.5
+
+### Patch Changes
+
+- 0d0b96a: Fix every error and warning reported by CI/CD (issue #2175).
+
+  - The release job now lands the version bump through a pull request when a
+    repository ruleset rejects the direct push to `main` (GH013), instead of
+    retrying it as a lost race and failing the release.
+  - The release gate asks the npm registry, not the `.changeset` folder, whether
+    the current version is published, so an interrupted release self-heals on the
+    next push instead of being silently skipped.
+  - Release assets are uploaded with the `gh` CLI, removing the last action
+    pinned to the deprecated Node 20 runtime.
+  - Eight source files that had drifted into the 1350-line warning band were
+    reduced by extracting cohesive modules; behaviour is unchanged.
+
+- 3422f03: Retry transient GitHub and git failures instead of aborting the run. A `gh pr create` died 3.1 seconds after it was issued on `GraphQL: Something went wrong while executing your query …` without a single retry: GitHub reports internal GraphQL faults as **HTTP 200 with an `errors[]` payload**, and the transient-error classifier only knew about TCP/TLS faults and the literal strings `http 502`/`503`/`504`, so the failure was treated as permanent. Classification now lives in one place (`src/transient-errors.lib.mjs`), recognises the GraphQL 200-with-errors family, gets its own larger budget (`HIVE_MIND_MAX_GITHUB_TRANSIENT_RETRIES`, default 6), and logs _why_ an error was or was not retried — including GitHub's support reference id, which was previously discarded. Network git commands (`push`/`fetch`/`pull`/`ls-remote`) are now retried too: the wrapper is installed on command-stream's `$` tag rather than at each of the ~36 call sites, `gh pr create` recovers the existing PR URL when a retry hits "a pull request already exists", and a new `gh-rate-limit/no-unretried-git-network` ESLint rule fails the build on any future unguarded git network call in `src/`.
+
 ## 2.13.4
 
 ### Patch Changes

package/README.hi.md

@@ -88,6 +88,8 @@ Hive Mind में औसत प्रोग्रामर से अलग
 
 Hive Mind अपने प्रबंधित terminal, log, telemetry, upload और GitHub publication paths में पहचाने गए credentials को sanitize करता है। Scanner विफल होने पर publication रोक दिया जाता है, लेकिन यह किसी autonomous tool को किसी असंबंधित exfiltration path का उपयोग करने से नहीं रोकता। Masking contract, supported formats, maintenance process और incident-response guidance के लिए [Credential sanitization](./docs/CREDENTIAL-SANITIZATION.hi.md) देखें।
 
+डिफ़ॉल्ट रूप से Docker में आइसोलेट किए गए कार्यों को ऑपरेटर के Claude/Codex क्रेडेंशियल मिल जाते हैं। `--use-router` उन्हें रोक देता है और मॉडल ट्रैफ़िक को `hive-mind-router` sidecar से भेजता है, जिससे हर कार्य को अपना सीमित दायरे वाला टोकन और अपना request लॉग मिलता है — देखें [राउटर आइसोलेशन](./docs/ROUTER.hi.md) (प्रयोगात्मक) और [लॉग एकत्र करना](./docs/COLLECTING-LOGS.hi.md)।
+
 `hive.mjs` चलाने के लिए न्यूनतम सिस्टम आवश्यकताएँ:
 
 ```

package/README.md

@@ -88,6 +88,8 @@ We strongly recommend:
 
 Hive Mind sanitizes recognized credentials in its maintained terminal, log, telemetry, upload, and GitHub publication paths. Publication is blocked if the scanner fails, but this does not prevent an autonomous tool from using an unrelated exfiltration path. See [Credential sanitization](./docs/CREDENTIAL-SANITIZATION.md) for the masking contract, supported formats, maintenance process, and incident-response guidance.
 
+Docker-isolated tasks receive the operator's Claude/Codex credentials by default. `--use-router` withholds them and routes model traffic through a `hive-mind-router` sidecar instead, giving each task its own scoped token and its own request log — see [Router isolation](./docs/ROUTER.md) (experimental) and [Collecting logs](./docs/COLLECTING-LOGS.md).
+
 Minimum system requirements to run `hive.mjs`:
 
 ```

package/README.ru.md

@@ -88,6 +88,8 @@ Hive Mind обладает высоким уровнем творчества,
 
 Hive Mind скрывает распознанные учётные данные в обслуживаемых им путях вывода терминала, журналов, телеметрии, загрузок и публикаций GitHub. При сбое сканера публикация блокируется, однако это не мешает автономному инструменту использовать посторонний канал утечки. Соглашение о маскировании, поддерживаемые форматы, процесс сопровождения и рекомендации по реагированию на инциденты приведены в разделе [Санитизация учётных данных](./docs/CREDENTIAL-SANITIZATION.ru.md).
 
+По умолчанию задачи, изолированные в Docker, получают учётные данные Claude/Codex оператора. `--use-router` их не передаёт и вместо этого направляет трафик модели через sidecar `hive-mind-router`, выдавая каждой задаче собственный ограниченный токен и собственный журнал запросов — см. [Изоляция через роутер](./docs/ROUTER.ru.md) (экспериментально) и [Сбор журналов](./docs/COLLECTING-LOGS.ru.md).
+
 Минимальные системные требования для запуска `hive.mjs`:
 
 ```

package/README.zh.md

@@ -88,6 +88,8 @@ Hive Mind 具备与普通程序员无异的高度创造力。当需求不明确
 
 Hive Mind 会在其维护的终端、日志、遥测、上传和 GitHub 发布路径中清理可识别的凭证。如果扫描器失败，发布会被阻止；但这无法阻止自主工具通过不相关的路径泄露数据。有关掩码约定、支持的格式、维护流程和事件响应指南，请参阅[凭证清理](./docs/CREDENTIAL-SANITIZATION.zh.md)。
 
+默认情况下，Docker 隔离的任务会拿到操作者的 Claude/Codex 凭据。`--use-router` 不再传入这些凭据，而是让模型流量经由 `hive-mind-router` sidecar 转发，使每个任务获得仅限自身的令牌和自己的请求日志——参见[路由器隔离](./docs/ROUTER.zh.md)（实验性）与[收集日志](./docs/COLLECTING-LOGS.zh.md)。
+
 运行 `hive.mjs` 的最低系统要求：
 
 ```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@link-assistant/hive-mind",
-  "version": "2.13.4",
+  "version": "2.14.0",
   "description": "AI-powered issue solver and hive mind for collaborative problem solving",
   "main": "src/hive.mjs",
   "type": "module",

package/src/agent.lib.mjs

@@ -8,7 +8,11 @@ if (typeof globalThis.use === 'undefined') {
   await ensureUseM();
 }
 
-const { $ } = await use('command-stream');
+const { $: __rawDollar$ } = await use('command-stream');
+// Issue #2168: retry transient git network failures (push/fetch/pull) the same
+// way `gh` calls are retried, for every command run through this module's `$`.
+const { wrapDollarWithGitRetry } = await import('./git-retry.lib.mjs');
+const $ = wrapDollarWithGitRetry(__rawDollar$);
 const fs = (await use('fs')).promises;
 const path = (await use('path')).default;
 const os = (await use('os')).default;

package/src/claude.lib.mjs

@@ -14,8 +14,7 @@ import { createInteractiveHandler } from './interactive-mode.lib.mjs';
 import { setupBidirectionalHandler, finalizeBidirectionalHandler, validateBidirectionalModeConfig, attachStreamingInput } from './bidirectional-interactive.lib.mjs';
 import { initProgressMonitoring } from './solve.progress-monitoring.lib.mjs';
 import { sanitizeObjectStrings } from './unicode-sanitization.lib.mjs';
-import Decimal from 'decimal.js-light';
-import { createEmptySubSessionUsage, accumulateModelUsage, mergeResultModelUsage, createSubAgentCallEntry, accumulateSubAgentUsage, getRawRequestInputTokens, displaySessionTokenUsage } from './claude.budget-stats.lib.mjs';
+import { createSubAgentCallEntry, accumulateSubAgentUsage, displaySessionTokenUsage } from './claude.budget-stats.lib.mjs';
 import { buildClaudeResumeCommand, buildClaudeAutonomousResumeCommand } from './claude.command-builder.lib.mjs';
 import { beginAnthropicCostScope, seedCumulativeAnthropicCost, addAnthropicRunCost, captureAnthropicResultCost } from './anthropic-cost-accumulator.lib.mjs'; // Issues #1886, #2056, #2119
 import { buildSolveResumeCommand } from './solve.resume-command.lib.mjs'; // Issue #942
@@ -152,162 +151,10 @@ export const executeClaude = async params => {
 // this file under the 1500-line repo cap (see check-file-line-limits CI job).
 import { calculateModelCost } from './claude.cost.lib.mjs';
 export { calculateModelCost };
-export const calculateSessionTokens = async (sessionId, tempDir, resultModelUsage = null, options = {}) => {
-  const os = (await use('os')).default;
-  const homeDir = options.homeDir || os.homedir();
-  const fetchModelInfoForUsage = options.fetchModelInfo || fetchModelInfo;
-  const projectDirName = tempDir.replace(/\//g, '-');
-  const sessionFile = path.join(homeDir, '.claude', 'projects', projectDirName, `${sessionId}.jsonl`);
-  try {
-    await fs.access(sessionFile);
-  } catch {
-    return null;
-  }
-  const modelUsage = {};
-  // Issue #1501: Deduplicate JSONL entries by message ID (stream-json splits responses)
-  const seenMessageIds = new Set();
-  let duplicateCount = 0;
-  const peakContextByModel = {};
-  let globalPeakContext = 0;
-  const subSessions = [];
-  let currentSubSession = createEmptySubSessionUsage();
-  const compactifications = [];
-  try {
-    const fileContent = await fs.readFile(sessionFile, 'utf8');
-    const lines = fileContent.trim().split('\n');
-    for (const line of lines) {
-      if (!line.trim()) continue;
-      try {
-        const entry = JSON.parse(line);
-        if (entry.type === 'system' && entry.subtype === 'compact_boundary') {
-          if (currentSubSession.messageCount > 0) {
-            subSessions.push(currentSubSession);
-          }
-          compactifications.push({
-            timestamp: entry.timestamp || null,
-            preTokens: entry.compactMetadata?.preTokens || null,
-            trigger: entry.compactMetadata?.trigger || 'unknown',
-          });
-          currentSubSession = createEmptySubSessionUsage();
-          continue;
-        }
-        if (entry.message && entry.message.usage && entry.message.model) {
-          // Issue #1501: Skip duplicate JSONL entries (same message ID = same API response)
-          const msgId = entry.message.id;
-          if (msgId) {
-            if (seenMessageIds.has(msgId)) {
-              duplicateCount++;
-              continue;
-            }
-            seenMessageIds.add(msgId);
-          }
-          accumulateModelUsage(modelUsage, entry);
-          // Issue #1737: Track peak restored-context input per request.
-          // Anthropic splits a request's input into input_tokens,
-          // cache_creation_input_tokens, and cache_read_input_tokens; all three
-          // count toward "how much context will be restored if I resume here".
-          const usage = entry.message.usage;
-          const requestContext = getRawRequestInputTokens(usage);
-          const model = entry.message.model;
-          if (requestContext > (peakContextByModel[model] || 0)) {
-            peakContextByModel[model] = requestContext;
-          }
-          if (requestContext > globalPeakContext) {
-            globalPeakContext = requestContext;
-          }
-          if (usage.input_tokens) currentSubSession.inputTokens += usage.input_tokens;
-          if (usage.cache_creation_input_tokens) currentSubSession.cacheCreationTokens += usage.cache_creation_input_tokens;
-          if (usage.cache_read_input_tokens) currentSubSession.cacheReadTokens += usage.cache_read_input_tokens;
-          if (usage.output_tokens) currentSubSession.outputTokens += usage.output_tokens;
-          currentSubSession.messageCount++;
-          // Issue #1501: Track peak context and output per sub-session
-          if (requestContext > currentSubSession.peakContextUsage) {
-            currentSubSession.peakContextUsage = requestContext;
-          }
-          if ((usage.output_tokens || 0) > currentSubSession.peakOutputUsage) {
-            currentSubSession.peakOutputUsage = usage.output_tokens || 0;
-          }
-        }
-      } catch {
-        // Skip lines that aren't valid JSON
-        continue;
-      }
-    }
-    if (currentSubSession.messageCount > 0) {
-      subSessions.push(currentSubSession);
-    }
-    mergeResultModelUsage(modelUsage, resultModelUsage);
-    if (Object.keys(modelUsage).length === 0) {
-      return null;
-    }
-    const modelInfoPromises = Object.keys(modelUsage).map(async modelId => {
-      const modelInfo = await fetchModelInfoForUsage(modelId);
-      return { modelId, modelInfo };
-    });
-    const modelInfoResults = await Promise.all(modelInfoPromises);
-    const modelInfoMap = {};
-    for (const { modelId, modelInfo } of modelInfoResults) {
-      if (modelInfo) {
-        modelInfoMap[modelId] = modelInfo;
-      }
-    }
-    for (const [modelId, usage] of Object.entries(modelUsage)) {
-      const modelInfo = modelInfoMap[modelId];
-      // Issue #1501: Attach peak context usage per model
-      usage.peakContextUsage = peakContextByModel[modelId] || 0;
-      // Calculate cost using pricing API
-      if (modelInfo) {
-        const costData = calculateModelCost(usage, modelInfo, true);
-        usage.costUSD = costData.total;
-        usage.costBreakdown = costData.breakdown;
-        usage.modelName = modelInfo.name || modelId;
-        usage.modelInfo = modelInfo;
-      } else {
-        usage.costUSD = usage._resultCostUSD ?? null;
-        usage.costBreakdown = null;
-        usage.modelName = modelId;
-        // Issue #1539: Use contextWindow/maxOutputTokens from result JSON as fallback model limits
-        const ctx = usage._resultContextWindow,
-          out = usage._resultMaxOutputTokens;
-        usage.modelInfo = ctx || out ? { limit: { context: ctx || null, output: out || null } } : null;
-      }
-    }
-    let totalInputTokens = 0;
-    let totalCacheCreationTokens = 0;
-    let totalCacheReadTokens = 0;
-    let totalOutputTokens = 0;
-    let totalCostDecimal = new Decimal(0);
-    let hasCostData = false;
-    for (const usage of Object.values(modelUsage)) {
-      totalInputTokens += usage.inputTokens;
-      totalCacheCreationTokens += usage.cacheCreationTokens;
-      totalCacheReadTokens += usage.cacheReadTokens;
-      totalOutputTokens += usage.outputTokens;
-      if (usage.costUSD !== null) {
-        totalCostDecimal = totalCostDecimal.plus(new Decimal(usage.costUSD));
-        hasCostData = true;
-      }
-    }
-    const totalTokens = totalInputTokens + totalCacheCreationTokens + totalOutputTokens;
-    return {
-      modelUsage,
-      inputTokens: totalInputTokens,
-      cacheCreationTokens: totalCacheCreationTokens,
-      cacheReadTokens: totalCacheReadTokens,
-      outputTokens: totalOutputTokens,
-      totalTokens,
-      totalCostUSD: hasCostData ? totalCostDecimal.toNumber() : null,
-      // Issue #1501: Peak context usage (max single-request fill) and dedup stats
-      peakContextUsage: globalPeakContext,
-      duplicateEntriesSkipped: duplicateCount,
-      // Issue #1491/#1501: Sub-session and compactification data (always include for display)
-      subSessions,
-      compactifications: compactifications.length > 0 ? compactifications : null,
-    };
-  } catch (readError) {
-    throw new Error(`Failed to read session file: ${readError.message}`, { cause: readError });
-  }
-};
+// Issue #2175: session token accounting lives in claude.session-tokens.lib.mjs
+// so this file stays under the 1350-line early-warning threshold (issue #1593).
+import { calculateSessionTokens } from './claude.session-tokens.lib.mjs';
+export { calculateSessionTokens };
 // Extracted to claude.stderr.lib.mjs (Issue #477, #1337)
 import { isStderrError } from './claude.stderr.lib.mjs';
 import { ensureAiToolScratchIgnored, filterAiToolScratchFromStatus } from './ai-tool-scratch.lib.mjs';

package/src/claude.session-tokens.lib.mjs

@@ -0,0 +1,180 @@
+/**
+ * Per-session token/cost accounting for Claude sessions.
+ *
+ * Reads the Claude Code session JSONL (`~/.claude/projects/<dir>/<id>.jsonl`),
+ * deduplicates the stream-json entries, splits the transcript into sub-sessions
+ * at each compact boundary, and prices the result through the model info API.
+ *
+ * Extracted from claude.lib.mjs (issue #2175) so that file stays under the
+ * 1350-line early-warning threshold of the CI file-headroom check (long files
+ * cause concurrent PR merge conflicts — issue #1593). Behaviour is unchanged;
+ * claude.lib.mjs re-exports this function.
+ *
+ * @see https://github.com/link-assistant/hive-mind/issues/2175
+ */
+
+import { promises as fs } from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+
+import Decimal from 'decimal.js-light';
+
+import { accumulateModelUsage, createEmptySubSessionUsage, getRawRequestInputTokens, mergeResultModelUsage } from './claude.budget-stats.lib.mjs';
+import { calculateModelCost } from './claude.cost.lib.mjs';
+import { fetchModelInfo } from './model-info.lib.mjs';
+
+export const calculateSessionTokens = async (sessionId, tempDir, resultModelUsage = null, options = {}) => {
+  const homeDir = options.homeDir || os.homedir();
+  const fetchModelInfoForUsage = options.fetchModelInfo || fetchModelInfo;
+  const projectDirName = tempDir.replace(/\//g, '-');
+  const sessionFile = path.join(homeDir, '.claude', 'projects', projectDirName, `${sessionId}.jsonl`);
+  try {
+    await fs.access(sessionFile);
+  } catch {
+    return null;
+  }
+  const modelUsage = {};
+  // Issue #1501: Deduplicate JSONL entries by message ID (stream-json splits responses)
+  const seenMessageIds = new Set();
+  let duplicateCount = 0;
+  const peakContextByModel = {};
+  let globalPeakContext = 0;
+  const subSessions = [];
+  let currentSubSession = createEmptySubSessionUsage();
+  const compactifications = [];
+  try {
+    const fileContent = await fs.readFile(sessionFile, 'utf8');
+    const lines = fileContent.trim().split('\n');
+    for (const line of lines) {
+      if (!line.trim()) continue;
+      try {
+        const entry = JSON.parse(line);
+        if (entry.type === 'system' && entry.subtype === 'compact_boundary') {
+          if (currentSubSession.messageCount > 0) {
+            subSessions.push(currentSubSession);
+          }
+          compactifications.push({
+            timestamp: entry.timestamp || null,
+            preTokens: entry.compactMetadata?.preTokens || null,
+            trigger: entry.compactMetadata?.trigger || 'unknown',
+          });
+          currentSubSession = createEmptySubSessionUsage();
+          continue;
+        }
+        if (entry.message && entry.message.usage && entry.message.model) {
+          // Issue #1501: Skip duplicate JSONL entries (same message ID = same API response)
+          const msgId = entry.message.id;
+          if (msgId) {
+            if (seenMessageIds.has(msgId)) {
+              duplicateCount++;
+              continue;
+            }
+            seenMessageIds.add(msgId);
+          }
+          accumulateModelUsage(modelUsage, entry);
+          // Issue #1737: Track peak restored-context input per request.
+          // Anthropic splits a request's input into input_tokens,
+          // cache_creation_input_tokens, and cache_read_input_tokens; all three
+          // count toward "how much context will be restored if I resume here".
+          const usage = entry.message.usage;
+          const requestContext = getRawRequestInputTokens(usage);
+          const model = entry.message.model;
+          if (requestContext > (peakContextByModel[model] || 0)) {
+            peakContextByModel[model] = requestContext;
+          }
+          if (requestContext > globalPeakContext) {
+            globalPeakContext = requestContext;
+          }
+          if (usage.input_tokens) currentSubSession.inputTokens += usage.input_tokens;
+          if (usage.cache_creation_input_tokens) currentSubSession.cacheCreationTokens += usage.cache_creation_input_tokens;
+          if (usage.cache_read_input_tokens) currentSubSession.cacheReadTokens += usage.cache_read_input_tokens;
+          if (usage.output_tokens) currentSubSession.outputTokens += usage.output_tokens;
+          currentSubSession.messageCount++;
+          // Issue #1501: Track peak context and output per sub-session
+          if (requestContext > currentSubSession.peakContextUsage) {
+            currentSubSession.peakContextUsage = requestContext;
+          }
+          if ((usage.output_tokens || 0) > currentSubSession.peakOutputUsage) {
+            currentSubSession.peakOutputUsage = usage.output_tokens || 0;
+          }
+        }
+      } catch {
+        // Skip lines that aren't valid JSON
+        continue;
+      }
+    }
+    if (currentSubSession.messageCount > 0) {
+      subSessions.push(currentSubSession);
+    }
+    mergeResultModelUsage(modelUsage, resultModelUsage);
+    if (Object.keys(modelUsage).length === 0) {
+      return null;
+    }
+    const modelInfoPromises = Object.keys(modelUsage).map(async modelId => {
+      const modelInfo = await fetchModelInfoForUsage(modelId);
+      return { modelId, modelInfo };
+    });
+    const modelInfoResults = await Promise.all(modelInfoPromises);
+    const modelInfoMap = {};
+    for (const { modelId, modelInfo } of modelInfoResults) {
+      if (modelInfo) {
+        modelInfoMap[modelId] = modelInfo;
+      }
+    }
+    for (const [modelId, usage] of Object.entries(modelUsage)) {
+      const modelInfo = modelInfoMap[modelId];
+      // Issue #1501: Attach peak context usage per model
+      usage.peakContextUsage = peakContextByModel[modelId] || 0;
+      // Calculate cost using pricing API
+      if (modelInfo) {
+        const costData = calculateModelCost(usage, modelInfo, true);
+        usage.costUSD = costData.total;
+        usage.costBreakdown = costData.breakdown;
+        usage.modelName = modelInfo.name || modelId;
+        usage.modelInfo = modelInfo;
+      } else {
+        usage.costUSD = usage._resultCostUSD ?? null;
+        usage.costBreakdown = null;
+        usage.modelName = modelId;
+        // Issue #1539: Use contextWindow/maxOutputTokens from result JSON as fallback model limits
+        const ctx = usage._resultContextWindow,
+          out = usage._resultMaxOutputTokens;
+        usage.modelInfo = ctx || out ? { limit: { context: ctx || null, output: out || null } } : null;
+      }
+    }
+    let totalInputTokens = 0;
+    let totalCacheCreationTokens = 0;
+    let totalCacheReadTokens = 0;
+    let totalOutputTokens = 0;
+    let totalCostDecimal = new Decimal(0);
+    let hasCostData = false;
+    for (const usage of Object.values(modelUsage)) {
+      totalInputTokens += usage.inputTokens;
+      totalCacheCreationTokens += usage.cacheCreationTokens;
+      totalCacheReadTokens += usage.cacheReadTokens;
+      totalOutputTokens += usage.outputTokens;
+      if (usage.costUSD !== null) {
+        totalCostDecimal = totalCostDecimal.plus(new Decimal(usage.costUSD));
+        hasCostData = true;
+      }
+    }
+    const totalTokens = totalInputTokens + totalCacheCreationTokens + totalOutputTokens;
+    return {
+      modelUsage,
+      inputTokens: totalInputTokens,
+      cacheCreationTokens: totalCacheCreationTokens,
+      cacheReadTokens: totalCacheReadTokens,
+      outputTokens: totalOutputTokens,
+      totalTokens,
+      totalCostUSD: hasCostData ? totalCostDecimal.toNumber() : null,
+      // Issue #1501: Peak context usage (max single-request fill) and dedup stats
+      peakContextUsage: globalPeakContext,
+      duplicateEntriesSkipped: duplicateCount,
+      // Issue #1491/#1501: Sub-session and compactification data (always include for display)
+      subSessions,
+      compactifications: compactifications.length > 0 ? compactifications : null,
+    };
+  } catch (readError) {
+    throw new Error(`Failed to read session file: ${readError.message}`, { cause: readError });
+  }
+};

package/src/codex.diagnostics.lib.mjs

@@ -0,0 +1,135 @@
+/**
+ * Codex diagnostic-line parsing and sub-session reconstruction.
+ *
+ * Extracted from src/codex.lib.mjs (issue #2175) so that file stays under the
+ * 1350-line early-warning threshold that protects concurrent merges (#1593).
+ * Behaviour is unchanged.
+ *
+ * Codex emits its context window, auto-compact limit and successful
+ * `/responses/compact` calls only as `codex_otel.log_only:` diagnostic lines on
+ * stderr (with RUST_LOG=debug). Those lines are the only evidence that a
+ * compactification happened, which is what lets the token usage of a single
+ * Codex run be split back into the sub-sessions the user actually experienced.
+ */
+
+import { getCumulativeContextInputTokens } from './context-fill.lib.mjs';
+
+const CODEX_COMPACT_API_ENDPOINT = '/responses/compact';
+const escapeRegExp = value => String(value).replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+const getCodexDiagnosticValue = (line, key) => {
+  const match = line.match(new RegExp(`${escapeRegExp(key)}=(?:"([^"]*)"|([^\\s")]+))`));
+  return match?.[1] ?? match?.[2] ?? null;
+};
+const getCodexDiagnosticInteger = (line, key) => {
+  const value = getCodexDiagnosticValue(line, key);
+  if (value === null) return null;
+  const parsed = Number.parseInt(value, 10);
+  return Number.isFinite(parsed) ? parsed : null;
+};
+const getCodexDiagnosticTimestamp = line => {
+  const eventTimestamp = getCodexDiagnosticValue(line, 'event.timestamp');
+  if (eventTimestamp) return eventTimestamp;
+  const logPrefixMatch = line.match(/^\[(\d{4}-\d{2}-\d{2}T[^\]]+Z)\]/u);
+  return logPrefixMatch?.[1] ?? null;
+};
+const isSuccessfulCodexCompactRequestLine = line => {
+  if (!line.includes('codex_otel.log_only:')) return false;
+  if (!line.includes('event.name="codex.api_request"')) return false;
+  if (!line.includes(`endpoint="${CODEX_COMPACT_API_ENDPOINT}"`)) return false;
+  const statusCode = getCodexDiagnosticInteger(line, 'http.response.status_code');
+  return statusCode === null || (statusCode >= 200 && statusCode < 300);
+};
+
+const splitTokenCountEvenly = (total, partCount) => {
+  const safeTotal = Math.max(0, Math.round(total || 0));
+  const safePartCount = Math.max(1, Math.round(partCount || 1));
+  const base = Math.floor(safeTotal / safePartCount);
+  let remainder = safeTotal % safePartCount;
+  return Array.from({ length: safePartCount }, () => {
+    const value = base + (remainder > 0 ? 1 : 0);
+    if (remainder > 0) remainder--;
+    return value;
+  });
+};
+const splitCodexSubSessionInputTokens = (total, partCount, autoCompactTokenLimit = null) => {
+  const safeTotal = Math.max(0, Math.round(total || 0));
+  const safePartCount = Math.max(1, Math.round(partCount || 1));
+  const safeLimit = Number.isFinite(autoCompactTokenLimit) && autoCompactTokenLimit > 0 ? Math.round(autoCompactTokenLimit) : null;
+  if (safePartCount <= 1) return [safeTotal];
+  if (safeLimit && safeTotal > safeLimit * (safePartCount - 1)) {
+    const chunks = [];
+    let remaining = safeTotal;
+    for (let i = 0; i < safePartCount - 1; i++) {
+      const chunk = Math.min(safeLimit, remaining);
+      chunks.push(chunk);
+      remaining -= chunk;
+    }
+    chunks.push(Math.max(0, remaining));
+    return chunks;
+  }
+  return splitTokenCountEvenly(safeTotal, safePartCount);
+};
+const splitTokenCountByWeights = (total, weights) => {
+  const safeTotal = Math.max(0, Math.round(total || 0));
+  const safeWeights = Array.isArray(weights) && weights.length > 0 ? weights.map(weight => Math.max(0, weight || 0)) : [1];
+  const weightTotal = safeWeights.reduce((sum, weight) => sum + weight, 0);
+  if (weightTotal <= 0) return splitTokenCountEvenly(safeTotal, safeWeights.length);
+  let allocated = 0;
+  return safeWeights.map((weight, index) => {
+    if (index === safeWeights.length - 1) return Math.max(0, safeTotal - allocated);
+    const value = Math.floor((safeTotal * weight) / weightTotal);
+    allocated += value;
+    return value;
+  });
+};
+export const rebuildCodexSubSessionsFromCompactifications = tokenUsage => {
+  const compactifications = Array.isArray(tokenUsage.compactifications) ? tokenUsage.compactifications : [];
+  if (compactifications.length === 0 || (tokenUsage.stepCount || 0) === 0) {
+    tokenUsage.subSessions = Array.isArray(tokenUsage.subSessions) ? tokenUsage.subSessions : [];
+    return;
+  }
+
+  const subSessionCount = compactifications.length + 1;
+  const inputChunks = splitCodexSubSessionInputTokens(tokenUsage.inputTokens || 0, subSessionCount, tokenUsage.autoCompactTokenLimit);
+  const cacheWriteChunks = splitTokenCountByWeights(tokenUsage.cacheWriteTokens || 0, inputChunks);
+  const cacheReadChunks = splitTokenCountByWeights(tokenUsage.cacheReadTokens || 0, inputChunks);
+  const outputChunks = splitTokenCountByWeights(tokenUsage.outputTokens || 0, inputChunks);
+  tokenUsage.subSessions = inputChunks.map((inputTokens, index) => {
+    const cacheCreationTokens = cacheWriteChunks[index] || 0;
+    const outputTokens = outputChunks[index] || 0;
+    return {
+      inputTokens,
+      cacheCreationTokens,
+      cacheReadTokens: cacheReadChunks[index] || 0,
+      outputTokens,
+      messageCount: null,
+      peakContextUsage: getCumulativeContextInputTokens({ inputTokens, cacheCreationTokens }),
+      peakOutputUsage: outputTokens,
+      estimated: true,
+      source: 'codex.compact-diagnostics',
+      compactBoundaryBefore: index === 0 ? null : compactifications[index - 1] || null,
+    };
+  });
+};
+const recordCodexCompactification = (line, tokenUsage) => {
+  if (!isSuccessfulCodexCompactRequestLine(line)) return;
+  const timestamp = getCodexDiagnosticTimestamp(line);
+  const conversationId = getCodexDiagnosticValue(line, 'conversation.id');
+  const existing = tokenUsage.compactifications.find(compact => compact.timestamp === timestamp && compact.conversationId === conversationId);
+  if (existing) return;
+  tokenUsage.compactifications.push({
+    timestamp,
+    preTokens: null,
+    trigger: 'auto',
+    source: 'codex.responses.compact',
+    conversationId: conversationId || null,
+  });
+};
+export const parseCodexDiagnosticLine = (line, tokenUsage) => {
+  const contextLimit = getCodexDiagnosticInteger(line, 'context_window') ?? getCodexDiagnosticInteger(line, 'model_context_window');
+  if (contextLimit !== null) tokenUsage.contextLimit = contextLimit;
+
+  const autoCompactTokenLimit = getCodexDiagnosticInteger(line, 'auto_compact_token_limit') ?? getCodexDiagnosticInteger(line, 'model_auto_compact_token_limit');
+  if (autoCompactTokenLimit !== null) tokenUsage.autoCompactTokenLimit = autoCompactTokenLimit;
+  recordCodexCompactification(line, tokenUsage);
+};