python-codex 0.1.11__tar.gz → 0.1.13__tar.gz

{python_codex-0.1.11 → python_codex-0.1.13}/.gitignore

@@ -5,3 +5,4 @@ __pycache__/
 .tmp/
 .pycodex-storage/
 uv.lock
+.vscode/

{python_codex-0.1.11 → python_codex-0.1.13}/AGENTS.md

@@ -12,6 +12,7 @@
 - 现在 `ResponsesModelClient` 默认会对流式断连做 provider 级自动重试（`stream_max_retries` 默认 5）；写 CLI/REPL 测试时如果断言“先向用户报错，再靠下一句 `go on` 继续”，必须在测试 provider 配置里显式设 `stream_max_retries = 0`，否则测试可能一直等不到预期错误而卡住。
 - `responses_server` compat 层应透传请求里的 `model`；不要再做 “取 downstream /models 第一个 id 并强制覆盖请求模型” 这种兜底兼容。
 - 对 `model_provider = "vllm"`，`responses_server` 仍然走 `/v1/chat/completions` compat 路径，但要保留 reasoning：把 chat chunk 里的 `reasoning` / `reasoning_content` 翻回 Responses `reasoning` item，并把历史里的 Responses `reasoning` item 回放成下游 assistant message 的 `reasoning` 字段。
+- `responses_server` 不能把 terminal reasoning-only chat output 当成成功回复：如果 downstream 一轮结束时只返回 `reasoning` / `reasoning_content`，没有 assistant `content` 且没有 tool call，先丢弃本次 partial reasoning 并用原样 downstream request 静默重试一次；若仍然 reasoning-only，再发 `response.failed(type=model_output_invalid)`，避免把 partial reasoning 写进 rollout 后在下一轮变成 chat 后端拒绝的裸 assistant message。
 - `responses_server` 的 provider-specific chat payload 定制统一放在 `responses_server/payload_processors.py`：使用 `CompatServerConfig.model_provider` 选择 `provider_name -> proc_fn(outcomming_request)` 映射，并且只在真正发出 downstream `/v1/chat/completions` 前 post-process；`StreamRouter` 内部继续保留 canonical payload，避免 tool hydration / mock web_search follow-up 被 provider 改写污染。
 - `responses_server` 如果要兼容下游 `/v1/messages`，也优先保持这条边界：内部继续用 canonical chat request / chat-like chunk 流，只有真正发请求和读取 SSE 时才做 messages 适配，这样 tool hydration、mock `web_search` follow-up、provider payload post-process 都能复用。
 - 真实 vLLM `0.19.0` 的 `/v1/messages` 会对缺失 `max_tokens` 直接返回 `400`；messages 适配层必须总是补这个字段。当前约定是优先透传请求里的 `max_output_tokens`/`max_tokens`，否则回退到默认 `32000`。
@@ -22,6 +23,7 @@
 - 对需要 model-specific prompt 的本地 model slug，直接在 vendored `pycodex/prompts/models.json` 补条目；当前 `step-3.5-flash` / `step-3.5-flash-2603` / `step-3.6` 已按这个方式接入。
 - 交互 REPL 的 context 用量提示也应尽量贴近上游语义：展示“剩余 context 百分比”而不是原始 token 数；计算时按上游同款 `BASELINE_TOKENS=12000` 做归一化，并在模型元数据只有 `context_window` 时默认按 `95%` effective window 处理。只要当前模型能解析出 context window，初始 prompt 就先显示 `100%`，等首个 usage 回来后再刷新成真实值。
 - 对交互 REPL 的 context 指示器，`model_context_window` 的取值优先级也要贴近上游：先吃 `config.toml` / profile 里的 `model_context_window` override，再回退到 vendored `models.json` 的 `context_window`；effective percent 继续沿用模型元数据，没有时默认 `95%`。
+- `pyco(<percent>)` 正常只来自模型流里最近一次 `response.completed.response.usage.total_tokens`；如果大 tool output 之后的下一次请求被下游 `context_length_exceeded` 拒绝，rollout 不会单独记录 usage。遇到这类错误时应从错误文案的 `requested ... tokens (... in the messages, ... in the completion)` 提取真实请求 token，作为失败请求的 `token_count` 事件回灌，并立即触发 compact 后重试一次。若服务端只返回 `Your input exceeds the context window...` 这类无 token 数的 `response.failed`，仍应触发 compact+retry，只是不要伪造 `token_count`。若 compact 请求本身也超长，先循环删除最旧的 `ToolResult` 及其配对 `ToolCall` 再重试 compact。
 - `AgentLoop` 的 turn-loop 语义要跟上游 `codex-rs/core/src/codex.rs` 一致：按 follow-up / tool handoff 自然收敛，不要加固定 12 轮之类的 hard cap，也不要保留本地专用的 iteration-limit 参数。
 - `README.md` 和 `docs/` 属于对齐工作的一部分：只要实现状态、对齐结论或使用方式发生实质变化，就应及时更新，不要让文档滞后于当前代码。
 - 新工具必须继承 `BaseTool`，然后通过 `ToolRegistry.register(tool_instance)` 接入；不要再给 registry 传散装 name/description/handler 参数。
@@ -54,3 +56,6 @@
 - `--call` / portable storage paths must not rely on the process default text encoding. Always pass `encoding="utf-8"` when reading config, prompts, AGENTS files, skills, dotenv, and session history; for user-authored instructions/history, prefer `errors="replace"` so a Windows GBK locale cannot crash on UTF-8 punctuation such as U+2264 or em dash.
 - 对接真实 `~/.codex/sessions/.../rollout-*.jsonl` 时，不要假设它一定是严格的一行一个 JSON object：本机样本可能包含 pretty-printed 多行对象，且文件尾部偶尔带未完成记录。恢复历史时用 concatenated-JSON 方式读取，并容忍尾部残缺。
 - `pycodex` 本地 session 保存现在也按上游思路走：新 session 一开始就分配稳定的 uuidv7 thread/session id，并把历史增量追加到 `CODEX_HOME/sessions/.../rollout-*.jsonl`；`/resume` 列表应只展示至少有真实 user message 的 rollout，避免空白新 session 污染恢复列表。
+- auto-compact 对齐上游配置名 `model_auto_compact_token_limit`；为空时关闭，触发依据是最近一次模型上报的 `usage.total_tokens`，pre-turn 压缩上一轮历史，mid-turn 压缩工具 follow-up 前的当前历史，并继续复用现有 compacted rollout 记录。
+- Responses streaming 里的 `response.incomplete` 不是连接断开：不要让 `ResponsesModelClient` 把它当 retryable incomplete stream 反复重连。普通 turn 应明确报 `response.incomplete`；compact 请求如果已经收到 assistant partial summary，可以用这个 partial summary 完成 replacement history，避免 midturn auto-compact 卡在 5 次 retry。
+- Feishu card tests read `~/.codex/.feishu_refresh_token` through production code; when running `tests/test_feishu_card.py` locally, isolate HOME (for example `HOME=/tmp/pycodex-empty-home env -u VIRTUAL_ENV uv run pytest tests/test_feishu_card.py tests/test_feishu_link.py`) unless the test itself controls `HOME`.

{python_codex-0.1.11 → python_codex-0.1.13}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: python-codex
-Version: 0.1.11
+Version: 0.1.13
 Summary: A minimal Python extraction of Codex's main agent loop
 License-File: LICENSE
 Requires-Python: >=3.6.2
@@ -185,6 +185,14 @@ Current behavior:
 - `/compact` synthesizes a local handoff summary, replaces the in-memory
   conversation history with the compacted view, and appends a compacted-history
   entry to the rollout so later `/resume` sees the same state
+- `model_auto_compact_token_limit = <tokens>` in `config.toml` enables the same
+  compaction path automatically when the latest reported usage reaches that
+  threshold before a follow-up sampling request or the next user turn
+- if a model request fails with `context_length_exceeded`, pycodex now treats
+  the provider-reported requested token count as a failed-request usage sample,
+  triggers the same compact path immediately, and retries the request once; if
+  the compact request is also over the limit, it repeatedly drops the oldest
+  tool response plus its matching tool call before retrying compact
 - new sessions are now recorded under `CODEX_HOME/sessions/.../rollout-*.jsonl`
   with a stable session/thread id and per-item append+flush semantics so
   `/resume` reads back the same rollout format
@@ -211,7 +219,12 @@ Current behavior:
   `reasoning_content` are translated back into Responses `reasoning` items, and
   historical `reasoning` items are replayed into downstream assistant messages
   via the `reasoning` field. Streaming token usage is also requested from vLLM
-  and forwarded to the final `response.completed.response.usage`
+  and forwarded to the final `response.completed.response.usage`. If a
+  downstream chat stream terminates after emitting only reasoning, with no
+  assistant content and no tool call, the compat layer discards that partial
+  reasoning, retries the same downstream request once, and only then emits
+  `response.failed` with `type = "model_output_invalid"` if the retry is still
+  reasoning-only
 - standalone `responses_server` now also supports downstream `/v1/messages`
   backends via `--outcomming-api messages`, while keeping the internal
   canonical request/route logic in chat-completions shape

{python_codex-0.1.11 → python_codex-0.1.13}/README.md

@@ -164,6 +164,14 @@ Current behavior:
 - `/compact` synthesizes a local handoff summary, replaces the in-memory
   conversation history with the compacted view, and appends a compacted-history
   entry to the rollout so later `/resume` sees the same state
+- `model_auto_compact_token_limit = <tokens>` in `config.toml` enables the same
+  compaction path automatically when the latest reported usage reaches that
+  threshold before a follow-up sampling request or the next user turn
+- if a model request fails with `context_length_exceeded`, pycodex now treats
+  the provider-reported requested token count as a failed-request usage sample,
+  triggers the same compact path immediately, and retries the request once; if
+  the compact request is also over the limit, it repeatedly drops the oldest
+  tool response plus its matching tool call before retrying compact
 - new sessions are now recorded under `CODEX_HOME/sessions/.../rollout-*.jsonl`
   with a stable session/thread id and per-item append+flush semantics so
   `/resume` reads back the same rollout format
@@ -190,7 +198,12 @@ Current behavior:
   `reasoning_content` are translated back into Responses `reasoning` items, and
   historical `reasoning` items are replayed into downstream assistant messages
   via the `reasoning` field. Streaming token usage is also requested from vLLM
-  and forwarded to the final `response.completed.response.usage`
+  and forwarded to the final `response.completed.response.usage`. If a
+  downstream chat stream terminates after emitting only reasoning, with no
+  assistant content and no tool call, the compat layer discards that partial
+  reasoning, retries the same downstream request once, and only then emits
+  `response.failed` with `type = "model_output_invalid"` if the retry is still
+  reasoning-only
 - standalone `responses_server` now also supports downstream `/v1/messages`
   backends via `--outcomming-api messages`, while keeping the internal
   canonical request/route logic in chat-completions shape

{python_codex-0.1.11 → python_codex-0.1.13}/docs/ALIGNMENT.md

@@ -554,7 +554,7 @@ Those are the next alignment target after the prompt/context pass.
 
 - `pycodex` 现在已经补上最小的 provider 级 stream retry：`ResponsesProviderConfig`
   支持 `stream_max_retries` / `stream_idle_timeout_ms`，默认值对齐 upstream 的
-  `5` 次重试和 `300_000 ms` SSE idle timeout；代码在 `pycodex/model.py`
+  `300_000 ms` SSE idle timeout；代码在 `pycodex/model.py`
 - 当前实现会把 `response.failed`、stream 在 `response.completed` 前断开、以及
   `requests` 侧的读流异常统一视为 retryable stream error，并在
   `ResponsesModelClient.complete(...)` 里按 backoff 重试；重试前会向外发

{python_codex-0.1.11 → python_codex-0.1.13}/docs/responses_server/README.md

@@ -97,6 +97,13 @@ trajectory 追加到 `${PYCODEX_DUMP}/dump.jsonl`，当前记录格式是：
 当前内置规则里，`vllm` 仍走 chat-completions compat 路径，但会额外保留
 reasoning；`stepfun` 会删除所有 `developer` role。
 
+如果下游 chat stream 一轮结束时只给了 `reasoning` / `reasoning_content`，
+没有 assistant `content` 且没有 tool call，server 会丢弃这次 partial reasoning 并用
+原样 downstream request 静默重试一次。若 retry 后仍是 reasoning-only，才发
+`response.failed(type=model_output_invalid)`。这样可以避免 interrupted 或
+length-stopped thinking 被持久化成 terminal reasoning-only history，并在下一轮转换成
+下游 chat 后端不接受的裸 assistant message。
+
 `messages` compat 则故意不改这层 canonical request：仍然先构造 chat 风格
 `outcomming_request`，只有在真正发请求和读 SSE 时，才在边界把它翻译成
 messages request / event。这样 tool hydration、mock `web_search`

{python_codex-0.1.11 → python_codex-0.1.13}/pycodex/__init__.py

@@ -2,12 +2,13 @@ from .compat import patch_asyncio
 
 patch_asyncio()
 
-from .agent import AgentLoop
+from .agent import Agent
 from .context import ContextConfig, ContextManager
 from .model import (
     ModelClient,
     NOOP_MODEL_STREAM_EVENT_HANDLER,
     ResponsesApiError,
+    ResponsesIncompleteError,
     ResponsesModelClient,
     ResponsesProviderConfig,
 )
@@ -26,14 +27,14 @@ from .protocol import (
     TurnResult,
     UserMessage,
 )
-from .runtime import AgentRuntime
+from .runtime import CliSubmissionQueue
 from .runtime_services import (
     PlanStore,
     RequestPermissionsManager,
     RequestUserInputManager,
     SubAgentManager,
-    create_runtime_environment,
-    get_runtime_environment,
+    create_agent_runtime_environment,
+    get_agent_runtime_environment,
 )
 from .tools import (
     ApplyPatchTool,
@@ -90,13 +91,13 @@ def debug(stop: 'bool' = False):
 
 __all__ = [
     "AgentEvent",
-    "AgentLoop",
-    "AgentRuntime",
+    "Agent",
+    "CliSubmissionQueue",
     "ApplyPatchTool",
     "AssistantMessage",
     "BaseTool",
     "CloseAgentTool",
-    "create_runtime_environment",
+    "create_agent_runtime_environment",
     "CodeModeManager",
     "ContextConfig",
     "ContextManager",
@@ -120,6 +121,7 @@ __all__ = [
     "RequestUserInputManager",
     "ResumeAgentTool",
     "ResponsesApiError",
+    "ResponsesIncompleteError",
     "ResponsesModelClient",
     "ResponsesProviderConfig",
     "SendInputTool",
@@ -142,5 +144,5 @@ __all__ = [
     "WaitTool",
     "WebSearchTool",
     "WriteStdinTool",
-    "get_runtime_environment",
+    "get_agent_runtime_environment",
 ]

{python_codex-0.1.11 → python_codex-0.1.13}/pycodex/agent.py

@@ -1,6 +1,7 @@
 
 import asyncio
 import json
+import re
 from typing import Callable
 
 from .context import ContextManager
@@ -22,17 +23,36 @@ import typing
 
 if typing.TYPE_CHECKING:
     from .utils.session_persist import SessionRolloutRecorder
+    from .runtime_services import AgentRuntimeEnvironment
 
 
 EventHandler = Callable[[AgentEvent], None]
-NOOP_EVENT_HANDLER: 'EventHandler' = lambda _event: None
+BASE_EVENT_HANDLER: 'EventHandler' = lambda _event: None
+_REQUESTED_TOKENS_RE = re.compile(
+    r"requested\s+([0-9,]+)\s+tokens",
+    re.IGNORECASE,
+)
+_REQUESTED_TOKEN_SPLIT_RE = re.compile(
+    r"\(([0-9,]+)\s+in\s+the\s+messages,\s+([0-9,]+)\s+in\s+the\s+completion\)",
+    re.IGNORECASE,
+)
+_MAX_CONTEXT_TOKENS_RE = re.compile(
+    r"maximum\s+context\s+length\s+is\s+([0-9,]+)\s+tokens",
+    re.IGNORECASE,
+)
+_CONTEXT_LENGTH_ERROR_MARKERS = (
+    "context_length_exceeded",
+    "maximum context length",
+    "exceeds the context window",
+    "exceeded the context window",
+)
 
 
 class TurnInterrupted(RuntimeError):
     pass
 
 
-class AgentLoop:
+class Agent:
     """Minimal Python port of Codex's turn loop.
 
     The core idea mirrors the Rust implementation:
@@ -47,9 +67,10 @@ class AgentLoop:
         tool_registry: 'ToolRegistry',
         context_manager: 'typing.Union[ContextManager, None]' = None,
         parallel_tool_calls: 'bool' = True,
-        event_handler: 'EventHandler' = NOOP_EVENT_HANDLER,
+        event_handler: 'EventHandler' = BASE_EVENT_HANDLER,
         initial_history: 'typing.Tuple[ConversationItem, ...]' = (),
         rollout_recorder: 'typing.Union[SessionRolloutRecorder, None]' = None,
+        runtime_environment: 'AgentRuntimeEnvironment' = None,
     ) -> 'None':
         self._model_client = model_client
         self._tool_registry = tool_registry
@@ -58,6 +79,11 @@ class AgentLoop:
         self._event_handler = event_handler
         self._history: 'typing.List[ConversationItem]' = list(initial_history)
         self._rollout_recorder = rollout_recorder
+        self._auto_compact_token_limit = (
+            self._context_manager.resolve_auto_compact_token_limit()
+        )
+        self._last_total_usage_tokens: 'typing.Union[int, None]' = None
+        self.runtime_environment = runtime_environment
         self.interrupt_asap = False
 
     @property
@@ -65,7 +91,7 @@ class AgentLoop:
         return tuple(self._history)
 
     def set_event_handler(
-        self, event_handler: 'EventHandler' = NOOP_EVENT_HANDLER
+        self, event_handler: 'EventHandler' = BASE_EVENT_HANDLER
     ) -> 'None':
         self._event_handler = event_handler
 
@@ -81,6 +107,11 @@ class AgentLoop:
     ) -> 'None':
         self._rollout_recorder = rollout_recorder
 
+    def ask(self, text: 'str') -> 'TurnResult':
+        from .utils.async_bridge import run_async
+
+        return run_async(self.run_turn([text]))
+
     def _raise_if_interrupt_requested(
         self,
         turn_id: 'str',
@@ -101,8 +132,6 @@ class AgentLoop:
         turn_id = turn_id or uuid7_string()
         self.interrupt_asap = False
         new_user_messages = [UserMessage(text=text) for text in texts]
-        self._history.extend(new_user_messages)
-        self._persist_history_items(new_user_messages)
 
         self._emit(
             "turn_started",
@@ -110,6 +139,9 @@ class AgentLoop:
             user_text="\n".join(texts),
             user_texts=list(texts),
         )
+        await self._maybe_auto_compact(turn_id, phase="pre_turn")
+        self._history.extend(new_user_messages)
+        self._persist_history_items(new_user_messages)
 
         last_assistant_message: 'typing.Union[str, None]' = None
         final_response_items: 'typing.Tuple[\n    typing.Union[typing.Union[AssistantMessage, ToolCall], ReasoningItem], ...\n]' = ()
@@ -122,23 +154,11 @@ class AgentLoop:
                     iteration,
                     output_text=last_assistant_message,
                 )
+                await self._maybe_auto_compact(turn_id, phase="mid_turn")
                 iteration += 1
-                prompt = self._context_manager.build_prompt(
-                    self._history,
-                    self._tool_registry.model_visible_specs(),
-                    self._parallel_tool_calls,
-                    turn_id=turn_id,
-                )
-                self._emit(
-                    "model_called",
+                response = await self._complete_model_request(
                     turn_id,
-                    iteration=iteration,
-                    history_size=len(prompt.input),
-                    tool_count=len(prompt.tools),
-                )
-                response = await self._model_client.complete(
-                    prompt,
-                    lambda event: self._handle_model_stream_event(turn_id, event),
+                    iteration,
                 )
                 final_response_items = tuple(response.items)
                 self._emit(
@@ -193,6 +213,10 @@ class AgentLoop:
         except TurnInterrupted:
             raise
         except Exception as exc:
+            context_usage = _usage_from_context_length_error(str(exc))
+            if context_usage is not None:
+                self._remember_token_usage(context_usage)
+                self._emit("token_count", turn_id, usage=context_usage)
             self._emit(
                 "turn_failed",
                 turn_id,
@@ -287,6 +311,8 @@ class AgentLoop:
             return
 
     def _handle_model_stream_event(self, turn_id: 'str', event: 'ModelStreamEvent') -> 'None':
+        if event.kind == "token_count":
+            self._remember_token_usage(event.payload.get("usage"))
         if event.kind == "assistant_delta":
             self._emit("assistant_delta", turn_id, **event.payload)
         elif event.kind == "tool_call":
@@ -296,6 +322,149 @@ class AgentLoop:
         elif event.kind == "stream_error":
             self._emit("stream_error", turn_id, **event.payload)
 
+    def _remember_token_usage(self, usage: 'object') -> 'None':
+        if not isinstance(usage, dict):
+            return
+        try:
+            self._last_total_usage_tokens = int(usage["total_tokens"])
+        except (KeyError, TypeError, ValueError):
+            return
+
+    async def _complete_model_request(
+        self,
+        turn_id: 'str',
+        iteration: 'int',
+    ) -> 'typing.Any':
+        attempted_context_compact = False
+        while True:
+            prompt = self._context_manager.build_prompt(
+                self._history,
+                self._tool_registry.model_visible_specs(),
+                self._parallel_tool_calls,
+                turn_id=turn_id,
+            )
+            self._emit(
+                "model_called",
+                turn_id,
+                iteration=iteration,
+                history_size=len(prompt.input),
+                tool_count=len(prompt.tools),
+            )
+            try:
+                return await self._model_client.complete(
+                    prompt,
+                    lambda event: self._handle_model_stream_event(turn_id, event),
+                )
+            except Exception as exc:
+                error_message = str(exc)
+                if (
+                    not _is_context_length_error_message(error_message)
+                    or attempted_context_compact
+                ):
+                    raise
+                attempted_context_compact = True
+                context_usage = _usage_from_context_length_error(error_message)
+                if context_usage is not None:
+                    self._remember_token_usage(context_usage)
+                    self._emit("token_count", turn_id, usage=context_usage)
+                await self._run_auto_compact(
+                    turn_id,
+                    phase="context_length_exceeded",
+                    total_tokens=(
+                        context_usage.get("total_tokens")
+                        if context_usage is not None
+                        else None
+                    ),
+                    token_limit=_context_length_error_token_limit(error_message),
+                    prune_tool_results_on_context_error=True,
+                )
+                self._raise_if_interrupt_requested(turn_id, iteration)
+
+    async def _maybe_auto_compact(
+        self,
+        turn_id: 'str',
+        phase: 'str',
+    ) -> 'None':
+        limit = self._auto_compact_token_limit
+        total_tokens = self._last_total_usage_tokens
+        if limit is None or total_tokens is None:
+            return
+        if total_tokens < limit or not self._history:
+            return
+
+        await self._run_auto_compact(
+            turn_id,
+            phase=phase,
+            total_tokens=total_tokens,
+            token_limit=limit,
+            prune_tool_results_on_context_error=True,
+        )
+
+    async def _run_auto_compact(
+        self,
+        turn_id: 'str',
+        phase: 'str',
+        total_tokens: 'typing.Union[int, None]' = None,
+        token_limit: 'typing.Union[int, None]' = None,
+        prune_tool_results_on_context_error: 'bool' = False,
+    ) -> 'None':
+        from .utils.compactor import compact_agent
+
+        payload: 'typing.Dict[str, object]' = {"phase": phase}
+        if total_tokens is not None:
+            payload["total_tokens"] = total_tokens
+        if token_limit is not None:
+            payload["token_limit"] = token_limit
+        self._emit(
+            "auto_compact_started",
+            turn_id,
+            **payload,
+        )
+
+        def handle_compact_stream_event(event: 'ModelStreamEvent') -> 'None':
+            if event.kind == "stream_error":
+                self._emit("stream_error", turn_id, **event.payload)
+
+        try:
+            compact_result = await compact_agent(
+                self,
+                handle_compact_stream_event,
+                prune_tool_results_on_context_error,
+            )
+        except Exception as exc:
+            failed_payload = dict(payload)
+            failed_payload.update(
+                {
+                    "error": str(exc),
+                    "error_type": type(exc).__name__,
+                }
+            )
+            self._emit(
+                "auto_compact_failed",
+                turn_id,
+                **failed_payload,
+            )
+            raise
+
+        self._last_total_usage_tokens = None
+        if compact_result is None:
+            return
+        completed_payload = dict(payload)
+        completed_payload.update(
+            {
+                "original_item_count": compact_result.original_item_count,
+                "retained_item_count": compact_result.retained_item_count,
+                "summary": compact_result.display_text(),
+            }
+        )
+        if compact_result.pruned_tool_results:
+            completed_payload["pruned_tool_results"] = compact_result.pruned_tool_results
+        self._emit(
+            "auto_compact_completed",
+            turn_id,
+            **completed_payload,
+        )
+
     def _build_follow_up_messages(
         self,
         tool_results: 'typing.List[ToolResult]',
@@ -326,3 +495,39 @@ class AgentLoop:
                             )
                         )
         return follow_ups
+
+
+def _usage_from_context_length_error(
+    message: 'str',
+) -> 'typing.Union[typing.Dict[str, int], None]':
+    if not _is_context_length_error_message(message):
+        return None
+
+    requested_match = _REQUESTED_TOKENS_RE.search(message)
+    if requested_match is None:
+        return None
+
+    usage = {"total_tokens": _parse_token_count(requested_match.group(1))}
+    split_match = _REQUESTED_TOKEN_SPLIT_RE.search(message)
+    if split_match is not None:
+        usage["input_tokens"] = _parse_token_count(split_match.group(1))
+        usage["output_tokens"] = _parse_token_count(split_match.group(2))
+    else:
+        usage["input_tokens"] = usage["total_tokens"]
+    return usage
+
+
+def _is_context_length_error_message(message: 'str') -> 'bool':
+    lower = message.lower()
+    return any(marker in lower for marker in _CONTEXT_LENGTH_ERROR_MARKERS)
+
+
+def _context_length_error_token_limit(message: 'str') -> 'typing.Union[int, None]':
+    limit_match = _MAX_CONTEXT_TOKENS_RE.search(message)
+    if limit_match is None:
+        return None
+    return _parse_token_count(limit_match.group(1))
+
+
+def _parse_token_count(value: 'str') -> 'int':
+    return int(value.replace(",", ""))