python-codex 0.1.11__tar.gz → 0.1.12__tar.gz

{python_codex-0.1.11 → python_codex-0.1.12}/AGENTS.md

@@ -12,6 +12,7 @@
 - 现在 `ResponsesModelClient` 默认会对流式断连做 provider 级自动重试（`stream_max_retries` 默认 5）；写 CLI/REPL 测试时如果断言“先向用户报错，再靠下一句 `go on` 继续”，必须在测试 provider 配置里显式设 `stream_max_retries = 0`，否则测试可能一直等不到预期错误而卡住。
 - `responses_server` compat 层应透传请求里的 `model`；不要再做 “取 downstream /models 第一个 id 并强制覆盖请求模型” 这种兜底兼容。
 - 对 `model_provider = "vllm"`，`responses_server` 仍然走 `/v1/chat/completions` compat 路径，但要保留 reasoning：把 chat chunk 里的 `reasoning` / `reasoning_content` 翻回 Responses `reasoning` item，并把历史里的 Responses `reasoning` item 回放成下游 assistant message 的 `reasoning` 字段。
+- `responses_server` 不能把 terminal reasoning-only chat output 当成成功回复：如果 downstream 一轮结束时只返回 `reasoning` / `reasoning_content`，没有 assistant `content` 且没有 tool call，先丢弃本次 partial reasoning 并用原样 downstream request 静默重试一次；若仍然 reasoning-only，再发 `response.failed(type=model_output_invalid)`，避免把 partial reasoning 写进 rollout 后在下一轮变成 chat 后端拒绝的裸 assistant message。
 - `responses_server` 的 provider-specific chat payload 定制统一放在 `responses_server/payload_processors.py`：使用 `CompatServerConfig.model_provider` 选择 `provider_name -> proc_fn(outcomming_request)` 映射，并且只在真正发出 downstream `/v1/chat/completions` 前 post-process；`StreamRouter` 内部继续保留 canonical payload，避免 tool hydration / mock web_search follow-up 被 provider 改写污染。
 - `responses_server` 如果要兼容下游 `/v1/messages`，也优先保持这条边界：内部继续用 canonical chat request / chat-like chunk 流，只有真正发请求和读取 SSE 时才做 messages 适配，这样 tool hydration、mock `web_search` follow-up、provider payload post-process 都能复用。
 - 真实 vLLM `0.19.0` 的 `/v1/messages` 会对缺失 `max_tokens` 直接返回 `400`；messages 适配层必须总是补这个字段。当前约定是优先透传请求里的 `max_output_tokens`/`max_tokens`，否则回退到默认 `32000`。
@@ -22,6 +23,7 @@
 - 对需要 model-specific prompt 的本地 model slug，直接在 vendored `pycodex/prompts/models.json` 补条目；当前 `step-3.5-flash` / `step-3.5-flash-2603` / `step-3.6` 已按这个方式接入。
 - 交互 REPL 的 context 用量提示也应尽量贴近上游语义：展示“剩余 context 百分比”而不是原始 token 数；计算时按上游同款 `BASELINE_TOKENS=12000` 做归一化，并在模型元数据只有 `context_window` 时默认按 `95%` effective window 处理。只要当前模型能解析出 context window，初始 prompt 就先显示 `100%`，等首个 usage 回来后再刷新成真实值。
 - 对交互 REPL 的 context 指示器，`model_context_window` 的取值优先级也要贴近上游：先吃 `config.toml` / profile 里的 `model_context_window` override，再回退到 vendored `models.json` 的 `context_window`；effective percent 继续沿用模型元数据，没有时默认 `95%`。
+- `pyco(<percent>)` 正常只来自模型流里最近一次 `response.completed.response.usage.total_tokens`；如果大 tool output 之后的下一次请求被下游 `context_length_exceeded` 拒绝，rollout 不会单独记录 usage。遇到这类错误时应从错误文案的 `requested ... tokens (... in the messages, ... in the completion)` 提取真实请求 token，作为失败请求的 `token_count` 事件回灌，并立即触发 compact 后重试一次。若 compact 请求本身也超长，先循环删除最旧的 `ToolResult` 及其配对 `ToolCall` 再重试 compact。
 - `AgentLoop` 的 turn-loop 语义要跟上游 `codex-rs/core/src/codex.rs` 一致：按 follow-up / tool handoff 自然收敛，不要加固定 12 轮之类的 hard cap，也不要保留本地专用的 iteration-limit 参数。
 - `README.md` 和 `docs/` 属于对齐工作的一部分：只要实现状态、对齐结论或使用方式发生实质变化，就应及时更新，不要让文档滞后于当前代码。
 - 新工具必须继承 `BaseTool`，然后通过 `ToolRegistry.register(tool_instance)` 接入；不要再给 registry 传散装 name/description/handler 参数。
@@ -54,3 +56,4 @@
 - `--call` / portable storage paths must not rely on the process default text encoding. Always pass `encoding="utf-8"` when reading config, prompts, AGENTS files, skills, dotenv, and session history; for user-authored instructions/history, prefer `errors="replace"` so a Windows GBK locale cannot crash on UTF-8 punctuation such as U+2264 or em dash.
 - 对接真实 `~/.codex/sessions/.../rollout-*.jsonl` 时，不要假设它一定是严格的一行一个 JSON object：本机样本可能包含 pretty-printed 多行对象，且文件尾部偶尔带未完成记录。恢复历史时用 concatenated-JSON 方式读取，并容忍尾部残缺。
 - `pycodex` 本地 session 保存现在也按上游思路走：新 session 一开始就分配稳定的 uuidv7 thread/session id，并把历史增量追加到 `CODEX_HOME/sessions/.../rollout-*.jsonl`；`/resume` 列表应只展示至少有真实 user message 的 rollout，避免空白新 session 污染恢复列表。
+- auto-compact 对齐上游配置名 `model_auto_compact_token_limit`；为空时关闭，触发依据是最近一次模型上报的 `usage.total_tokens`，pre-turn 压缩上一轮历史，mid-turn 压缩工具 follow-up 前的当前历史，并继续复用现有 compacted rollout 记录。

{python_codex-0.1.11 → python_codex-0.1.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: python-codex
-Version: 0.1.11
+Version: 0.1.12
 Summary: A minimal Python extraction of Codex's main agent loop
 License-File: LICENSE
 Requires-Python: >=3.6.2
@@ -185,6 +185,14 @@ Current behavior:
 - `/compact` synthesizes a local handoff summary, replaces the in-memory
   conversation history with the compacted view, and appends a compacted-history
   entry to the rollout so later `/resume` sees the same state
+- `model_auto_compact_token_limit = <tokens>` in `config.toml` enables the same
+  compaction path automatically when the latest reported usage reaches that
+  threshold before a follow-up sampling request or the next user turn
+- if a model request fails with `context_length_exceeded`, pycodex now treats
+  the provider-reported requested token count as a failed-request usage sample,
+  triggers the same compact path immediately, and retries the request once; if
+  the compact request is also over the limit, it repeatedly drops the oldest
+  tool response plus its matching tool call before retrying compact
 - new sessions are now recorded under `CODEX_HOME/sessions/.../rollout-*.jsonl`
   with a stable session/thread id and per-item append+flush semantics so
   `/resume` reads back the same rollout format
@@ -211,7 +219,12 @@ Current behavior:
   `reasoning_content` are translated back into Responses `reasoning` items, and
   historical `reasoning` items are replayed into downstream assistant messages
   via the `reasoning` field. Streaming token usage is also requested from vLLM
-  and forwarded to the final `response.completed.response.usage`
+  and forwarded to the final `response.completed.response.usage`. If a
+  downstream chat stream terminates after emitting only reasoning, with no
+  assistant content and no tool call, the compat layer discards that partial
+  reasoning, retries the same downstream request once, and only then emits
+  `response.failed` with `type = "model_output_invalid"` if the retry is still
+  reasoning-only
 - standalone `responses_server` now also supports downstream `/v1/messages`
   backends via `--outcomming-api messages`, while keeping the internal
   canonical request/route logic in chat-completions shape

{python_codex-0.1.11 → python_codex-0.1.12}/README.md

@@ -164,6 +164,14 @@ Current behavior:
 - `/compact` synthesizes a local handoff summary, replaces the in-memory
   conversation history with the compacted view, and appends a compacted-history
   entry to the rollout so later `/resume` sees the same state
+- `model_auto_compact_token_limit = <tokens>` in `config.toml` enables the same
+  compaction path automatically when the latest reported usage reaches that
+  threshold before a follow-up sampling request or the next user turn
+- if a model request fails with `context_length_exceeded`, pycodex now treats
+  the provider-reported requested token count as a failed-request usage sample,
+  triggers the same compact path immediately, and retries the request once; if
+  the compact request is also over the limit, it repeatedly drops the oldest
+  tool response plus its matching tool call before retrying compact
 - new sessions are now recorded under `CODEX_HOME/sessions/.../rollout-*.jsonl`
   with a stable session/thread id and per-item append+flush semantics so
   `/resume` reads back the same rollout format
@@ -190,7 +198,12 @@ Current behavior:
   `reasoning_content` are translated back into Responses `reasoning` items, and
   historical `reasoning` items are replayed into downstream assistant messages
   via the `reasoning` field. Streaming token usage is also requested from vLLM
-  and forwarded to the final `response.completed.response.usage`
+  and forwarded to the final `response.completed.response.usage`. If a
+  downstream chat stream terminates after emitting only reasoning, with no
+  assistant content and no tool call, the compat layer discards that partial
+  reasoning, retries the same downstream request once, and only then emits
+  `response.failed` with `type = "model_output_invalid"` if the retry is still
+  reasoning-only
 - standalone `responses_server` now also supports downstream `/v1/messages`
   backends via `--outcomming-api messages`, while keeping the internal
   canonical request/route logic in chat-completions shape

{python_codex-0.1.11 → python_codex-0.1.12}/docs/responses_server/README.md

@@ -97,6 +97,13 @@ trajectory 追加到 `${PYCODEX_DUMP}/dump.jsonl`，当前记录格式是：
 当前内置规则里，`vllm` 仍走 chat-completions compat 路径，但会额外保留
 reasoning；`stepfun` 会删除所有 `developer` role。
 
+如果下游 chat stream 一轮结束时只给了 `reasoning` / `reasoning_content`，
+没有 assistant `content` 且没有 tool call，server 会丢弃这次 partial reasoning 并用
+原样 downstream request 静默重试一次。若 retry 后仍是 reasoning-only，才发
+`response.failed(type=model_output_invalid)`。这样可以避免 interrupted 或
+length-stopped thinking 被持久化成 terminal reasoning-only history，并在下一轮转换成
+下游 chat 后端不接受的裸 assistant message。
+
 `messages` compat 则故意不改这层 canonical request：仍然先构造 chat 风格
 `outcomming_request`，只有在真正发请求和读 SSE 时，才在边界把它翻译成
 messages request / event。这样 tool hydration、mock `web_search`

{python_codex-0.1.11 → python_codex-0.1.12}/pycodex/agent.py

@@ -1,6 +1,7 @@
 
 import asyncio
 import json
+import re
 from typing import Callable
 
 from .context import ContextManager
@@ -26,6 +27,18 @@ if typing.TYPE_CHECKING:
 
 EventHandler = Callable[[AgentEvent], None]
 NOOP_EVENT_HANDLER: 'EventHandler' = lambda _event: None
+_REQUESTED_TOKENS_RE = re.compile(
+    r"requested\s+([0-9,]+)\s+tokens",
+    re.IGNORECASE,
+)
+_REQUESTED_TOKEN_SPLIT_RE = re.compile(
+    r"\(([0-9,]+)\s+in\s+the\s+messages,\s+([0-9,]+)\s+in\s+the\s+completion\)",
+    re.IGNORECASE,
+)
+_MAX_CONTEXT_TOKENS_RE = re.compile(
+    r"maximum\s+context\s+length\s+is\s+([0-9,]+)\s+tokens",
+    re.IGNORECASE,
+)
 
 
 class TurnInterrupted(RuntimeError):
@@ -58,6 +71,10 @@ class AgentLoop:
         self._event_handler = event_handler
         self._history: 'typing.List[ConversationItem]' = list(initial_history)
         self._rollout_recorder = rollout_recorder
+        self._auto_compact_token_limit = (
+            self._context_manager.resolve_auto_compact_token_limit()
+        )
+        self._last_total_usage_tokens: 'typing.Union[int, None]' = None
         self.interrupt_asap = False
 
     @property
@@ -101,8 +118,6 @@ class AgentLoop:
         turn_id = turn_id or uuid7_string()
         self.interrupt_asap = False
         new_user_messages = [UserMessage(text=text) for text in texts]
-        self._history.extend(new_user_messages)
-        self._persist_history_items(new_user_messages)
 
         self._emit(
             "turn_started",
@@ -110,6 +125,9 @@ class AgentLoop:
             user_text="\n".join(texts),
             user_texts=list(texts),
         )
+        await self._maybe_auto_compact(turn_id, phase="pre_turn")
+        self._history.extend(new_user_messages)
+        self._persist_history_items(new_user_messages)
 
         last_assistant_message: 'typing.Union[str, None]' = None
         final_response_items: 'typing.Tuple[\n    typing.Union[typing.Union[AssistantMessage, ToolCall], ReasoningItem], ...\n]' = ()
@@ -122,23 +140,11 @@ class AgentLoop:
                     iteration,
                     output_text=last_assistant_message,
                 )
+                await self._maybe_auto_compact(turn_id, phase="mid_turn")
                 iteration += 1
-                prompt = self._context_manager.build_prompt(
-                    self._history,
-                    self._tool_registry.model_visible_specs(),
-                    self._parallel_tool_calls,
-                    turn_id=turn_id,
-                )
-                self._emit(
-                    "model_called",
+                response = await self._complete_model_request(
                     turn_id,
-                    iteration=iteration,
-                    history_size=len(prompt.input),
-                    tool_count=len(prompt.tools),
-                )
-                response = await self._model_client.complete(
-                    prompt,
-                    lambda event: self._handle_model_stream_event(turn_id, event),
+                    iteration,
                 )
                 final_response_items = tuple(response.items)
                 self._emit(
@@ -193,6 +199,10 @@ class AgentLoop:
         except TurnInterrupted:
             raise
         except Exception as exc:
+            context_usage = _usage_from_context_length_error(str(exc))
+            if context_usage is not None:
+                self._remember_token_usage(context_usage)
+                self._emit("token_count", turn_id, usage=context_usage)
             self._emit(
                 "turn_failed",
                 turn_id,
@@ -287,6 +297,8 @@ class AgentLoop:
             return
 
     def _handle_model_stream_event(self, turn_id: 'str', event: 'ModelStreamEvent') -> 'None':
+        if event.kind == "token_count":
+            self._remember_token_usage(event.payload.get("usage"))
         if event.kind == "assistant_delta":
             self._emit("assistant_delta", turn_id, **event.payload)
         elif event.kind == "tool_call":
@@ -296,6 +308,140 @@ class AgentLoop:
         elif event.kind == "stream_error":
             self._emit("stream_error", turn_id, **event.payload)
 
+    def _remember_token_usage(self, usage: 'object') -> 'None':
+        if not isinstance(usage, dict):
+            return
+        try:
+            self._last_total_usage_tokens = int(usage["total_tokens"])
+        except (KeyError, TypeError, ValueError):
+            return
+
+    async def _complete_model_request(
+        self,
+        turn_id: 'str',
+        iteration: 'int',
+    ) -> 'typing.Any':
+        attempted_context_compact = False
+        while True:
+            prompt = self._context_manager.build_prompt(
+                self._history,
+                self._tool_registry.model_visible_specs(),
+                self._parallel_tool_calls,
+                turn_id=turn_id,
+            )
+            self._emit(
+                "model_called",
+                turn_id,
+                iteration=iteration,
+                history_size=len(prompt.input),
+                tool_count=len(prompt.tools),
+            )
+            try:
+                return await self._model_client.complete(
+                    prompt,
+                    lambda event: self._handle_model_stream_event(turn_id, event),
+                )
+            except Exception as exc:
+                context_usage = _usage_from_context_length_error(str(exc))
+                if context_usage is None or attempted_context_compact:
+                    raise
+                attempted_context_compact = True
+                self._remember_token_usage(context_usage)
+                self._emit("token_count", turn_id, usage=context_usage)
+                await self._run_auto_compact(
+                    turn_id,
+                    phase="context_length_exceeded",
+                    total_tokens=context_usage.get("total_tokens"),
+                    token_limit=_context_length_error_token_limit(str(exc)),
+                    prune_tool_results_on_context_error=True,
+                )
+                self._raise_if_interrupt_requested(turn_id, iteration)
+
+    async def _maybe_auto_compact(
+        self,
+        turn_id: 'str',
+        phase: 'str',
+    ) -> 'None':
+        limit = self._auto_compact_token_limit
+        total_tokens = self._last_total_usage_tokens
+        if limit is None or total_tokens is None:
+            return
+        if total_tokens < limit or not self._history:
+            return
+
+        await self._run_auto_compact(
+            turn_id,
+            phase=phase,
+            total_tokens=total_tokens,
+            token_limit=limit,
+            prune_tool_results_on_context_error=True,
+        )
+
+    async def _run_auto_compact(
+        self,
+        turn_id: 'str',
+        phase: 'str',
+        total_tokens: 'typing.Union[int, None]' = None,
+        token_limit: 'typing.Union[int, None]' = None,
+        prune_tool_results_on_context_error: 'bool' = False,
+    ) -> 'None':
+        from .utils.compactor import compact_agent_loop
+
+        payload: 'typing.Dict[str, object]' = {"phase": phase}
+        if total_tokens is not None:
+            payload["total_tokens"] = total_tokens
+        if token_limit is not None:
+            payload["token_limit"] = token_limit
+        self._emit(
+            "auto_compact_started",
+            turn_id,
+            **payload,
+        )
+
+        def handle_compact_stream_event(event: 'ModelStreamEvent') -> 'None':
+            if event.kind == "stream_error":
+                self._emit("stream_error", turn_id, **event.payload)
+
+        try:
+            compact_result = await compact_agent_loop(
+                self,
+                handle_compact_stream_event,
+                prune_tool_results_on_context_error,
+            )
+        except Exception as exc:
+            failed_payload = dict(payload)
+            failed_payload.update(
+                {
+                    "error": str(exc),
+                    "error_type": type(exc).__name__,
+                }
+            )
+            self._emit(
+                "auto_compact_failed",
+                turn_id,
+                **failed_payload,
+            )
+            raise
+
+        self._last_total_usage_tokens = None
+        if compact_result is None:
+            return
+        completed_payload = dict(payload)
+        completed_payload.update(
+            {
+                "original_item_count": compact_result.original_item_count,
+                "retained_item_count": compact_result.retained_item_count,
+                "summary": compact_result.display_text(),
+            }
+        )
+        if compact_result.pruned_tool_results:
+            completed_payload["pruned_tool_results"] = compact_result.pruned_tool_results
+        self._emit(
+            "auto_compact_completed",
+            turn_id,
+            **completed_payload,
+        )
+
     def _build_follow_up_messages(
         self,
         tool_results: 'typing.List[ToolResult]',
@@ -326,3 +472,38 @@ class AgentLoop:
                             )
                         )
         return follow_ups
+
+
+def _usage_from_context_length_error(
+    message: 'str',
+) -> 'typing.Union[typing.Dict[str, int], None]':
+    lower = message.lower()
+    if (
+        "context_length_exceeded" not in lower
+        and "maximum context length" not in lower
+    ):
+        return None
+
+    requested_match = _REQUESTED_TOKENS_RE.search(message)
+    if requested_match is None:
+        return None
+
+    usage = {"total_tokens": _parse_token_count(requested_match.group(1))}
+    split_match = _REQUESTED_TOKEN_SPLIT_RE.search(message)
+    if split_match is not None:
+        usage["input_tokens"] = _parse_token_count(split_match.group(1))
+        usage["output_tokens"] = _parse_token_count(split_match.group(2))
+    else:
+        usage["input_tokens"] = usage["total_tokens"]
+    return usage
+
+
+def _context_length_error_token_limit(message: 'str') -> 'typing.Union[int, None]':
+    limit_match = _MAX_CONTEXT_TOKENS_RE.search(message)
+    if limit_match is None:
+        return None
+    return _parse_token_count(limit_match.group(1))
+
+
+def _parse_token_count(value: 'str') -> 'int':
+    return int(value.replace(",", ""))

{python_codex-0.1.11 → python_codex-0.1.12}/pycodex/cli.py

@@ -594,6 +594,7 @@ async def run_interactive_session(
             compact_result = await compact_agent_loop(
                 agent_loop,
                 handle_compact_stream_event,
+                True,
             )
             if compact_result is None:
                 view.write_line("Nothing to compact.")

{python_codex-0.1.11 → python_codex-0.1.12}/pycodex/context.py

@@ -78,6 +78,7 @@ class ContextConfig:
     project_doc_max_bytes: 'typing.Union[int, None]' = None
     model: 'typing.Union[str, None]' = None
     model_context_window: 'typing.Union[int, None]' = None
+    model_auto_compact_token_limit: 'typing.Union[int, None]' = None
     personality: 'typing.Union[str, None]' = None
     approval_policy: 'typing.Union[str, None]' = None
     sandbox_mode: 'typing.Union[str, None]' = None
@@ -120,6 +121,9 @@ class ContextConfig:
             project_doc_max_bytes=_normalize_int(selected.get("project_doc_max_bytes")),
             model=_normalize_text(selected.get("model")),
             model_context_window=_normalize_int(selected.get("model_context_window")),
+            model_auto_compact_token_limit=_normalize_int(
+                selected.get("model_auto_compact_token_limit")
+            ),
             personality=_normalize_text(selected.get("personality")),
             approval_policy=_normalize_text(selected.get("approval_policy")),
             sandbox_mode=_normalize_text(selected.get("sandbox_mode")),
@@ -268,6 +272,18 @@ class ContextManager:
             effective_percent = DEFAULT_EFFECTIVE_CONTEXT_WINDOW_PERCENT
         return context_window * max(effective_percent, 0) // 100
 
+    def resolve_auto_compact_token_limit(self) -> 'typing.Union[int, None]':
+        if self._config.model_auto_compact_token_limit is not None:
+            return self._config.model_auto_compact_token_limit
+
+        model_slug = self._config.model
+        if model_slug is None:
+            return None
+        model_metadata = _load_models_by_slug().get(model_slug)
+        if model_metadata is None:
+            return None
+        return _normalize_int(model_metadata.get("auto_compact_token_limit"))
+
     def _resolve_model_instructions(self) -> 'typing.Union[str, None]':
         model_slug = self._config.model
         if model_slug is None:

{python_codex-0.1.11 → python_codex-0.1.12}/pycodex/model.py

@@ -287,6 +287,8 @@ class ResponsesModelClient:
                     event_handler,
                 )
             except ResponsesRetryableError as exc:
+                if _is_context_length_error_message(str(exc)):
+                    raise ResponsesApiError(str(exc)) from exc
                 if retries >= max_retries:
                     raise
                 retries += 1
@@ -780,11 +782,14 @@ class ResponsesModelClient:
             )
 
         message = str(error.get("message") or "responses stream failed")
-        code = str(error.get("code") or "").strip()
+        code = str(error.get("code") or error.get("type") or "").strip()
+        if _is_context_length_error_message(message):
+            raise ResponsesApiError(self._format_response_failed_error(message))
         if code in {
             "context_length_exceeded",
             "insufficient_quota",
             "invalid_prompt",
+            "model_output_invalid",
             "usage_not_included",
         }:
             raise ResponsesApiError(self._format_response_failed_error(message))
@@ -888,6 +893,14 @@ def _optional_int(value: 'object') -> 'typing.Union[int, None]':
     return int(value)
 
 
+def _is_context_length_error_message(message: 'str') -> 'bool':
+    lower = message.lower()
+    return (
+        "context_length_exceeded" in lower
+        or "maximum context length" in lower
+    )
+
+
 def _requests_verify_setting() -> 'typing.Union[typing.Union[str, bool], None]':
     for env_name in ("REQUESTS_CA_BUNDLE", "CURL_CA_BUNDLE", "SSL_CERT_FILE"):
         value = os.environ.get(env_name, "").strip()

{python_codex-0.1.11 → python_codex-0.1.12}/pycodex/utils/compactor.py

@@ -1,6 +1,13 @@
 from dataclasses import dataclass
 
-from ..protocol import AssistantMessage, ConversationItem, ModelStreamEvent, UserMessage
+from ..protocol import (
+    AssistantMessage,
+    ConversationItem,
+    ModelStreamEvent,
+    ToolCall,
+    ToolResult,
+    UserMessage,
+)
 from .random_ids import uuid7_string
 import typing
 
@@ -35,6 +42,7 @@ _SUBAGENT_NOTIFICATION_PREFIX = "<subagent_notification>\n"
 class CompactResult:
     history: 'typing.Tuple[ConversationItem, ...]'
     original_item_count: 'int'
+    pruned_tool_results: 'int' = 0
 
     @property
     def retained_item_count(self) -> 'int':
@@ -43,10 +51,14 @@ class CompactResult:
     def display_text(self) -> 'str':
         retained_label = _pluralize("item", self.retained_item_count)
         original_label = _pluralize("item", self.original_item_count)
-        return (
+        text = (
             f"compact({self.original_item_count} {original_label}) -> "
             f"{self.retained_item_count} {retained_label} + [summary]"
         )
+        if self.pruned_tool_results:
+            tool_label = _pluralize("tool response", self.pruned_tool_results)
+            text += f" (dropped {self.pruned_tool_results} old {tool_label})"
+        return text
 
 
 def compact(
@@ -60,24 +72,42 @@ def compact(
 async def compact_agent_loop(
     agent_loop: 'AgentLoop',
     stream_event_handler: 'typing.Union[typing.Callable[[ModelStreamEvent], None], None]' = None,
+    prune_tool_results_on_context_error: 'bool' = False,
 ) -> 'typing.Union[CompactResult, None]':
     history = agent_loop.history
     if not history:
         return None
     original_item_count = len(history)
+    pruned_tool_results = 0
 
-    compact_prompt = UserMessage(text=DEFAULT_COMPACT_PROMPT)
-    prompt = agent_loop._context_manager.build_prompt(
-        list(history) + [compact_prompt],
-        [],
-        False,
-        turn_id=uuid7_string(),
-    )
     noop_stream_event_handler = lambda _event: None
-    response = await agent_loop._model_client.complete(
-        prompt,
-        stream_event_handler or noop_stream_event_handler,
-    )
+    while True:
+        compact_prompt = UserMessage(text=DEFAULT_COMPACT_PROMPT)
+        prompt = agent_loop._context_manager.build_prompt(
+            list(history) + [compact_prompt],
+            [],
+            False,
+            turn_id=uuid7_string(),
+        )
+        try:
+            response = await agent_loop._model_client.complete(
+                prompt,
+                stream_event_handler or noop_stream_event_handler,
+            )
+            break
+        except Exception as exc:
+            if (
+                not prune_tool_results_on_context_error
+                or not _is_context_length_error(str(exc))
+            ):
+                raise
+            pruned_history = prune_oldest_tool_response(history)
+            if pruned_history is None:
+                raise
+            history = pruned_history
+            pruned_tool_results += 1
+            agent_loop.replace_history(history)
+
     compacted_history = compact(
         list(history) + [compact_prompt] + list(response.items)
     )
@@ -88,6 +118,32 @@ async def compact_agent_loop(
     return CompactResult(
         history=compacted_history,
         original_item_count=original_item_count,
+        pruned_tool_results=pruned_tool_results,
+    )
+
+
+def prune_oldest_tool_response(
+    history: 'typing.Sequence[ConversationItem]',
+) -> 'typing.Union[typing.Tuple[ConversationItem, ...], None]':
+    items = list(history)
+    tool_result_index = None
+    call_id = None
+    for index, item in enumerate(items):
+        if isinstance(item, ToolResult):
+            tool_result_index = index
+            call_id = item.call_id
+            break
+    if tool_result_index is None:
+        return None
+
+    indexes_to_remove = {tool_result_index}
+    for index, item in enumerate(items[:tool_result_index]):
+        if isinstance(item, ToolCall) and item.call_id == call_id:
+            indexes_to_remove.add(index)
+            break
+
+    return tuple(
+        item for index, item in enumerate(items) if index not in indexes_to_remove
     )
 
 
@@ -187,3 +243,11 @@ def _pluralize(noun: 'str', count: 'int') -> 'str':
 
 def _is_synthetic_user_message(text: 'str') -> 'bool':
     return text.startswith(_SUBAGENT_NOTIFICATION_PREFIX)
+
+
+def _is_context_length_error(message: 'str') -> 'bool':
+    lower = message.lower()
+    return (
+        "context_length_exceeded" in lower
+        or "maximum context length" in lower
+    )

{python_codex-0.1.11 → python_codex-0.1.12}/pycodex/utils/visualize.py

@@ -705,6 +705,59 @@ class CliSessionView:
                 self._spinner.set_label("reconnecting")
             return
 
+        if event.kind == "auto_compact_started":
+            self._finish_stream()
+            total_tokens = event.payload.get("total_tokens")
+            token_limit = event.payload.get("token_limit")
+            if total_tokens is not None and token_limit is not None:
+                message = f"[status] auto-compact: {total_tokens}/{token_limit} tokens"
+            else:
+                message = "[status] auto-compact"
+            self._print_line(
+                colorize_cli_message(message, "status", self._color_enabled)
+            )
+            if self._input_active:
+                self._spinner.pause()
+            else:
+                self._spinner.resume()
+                self._spinner.set_label("compacting context")
+                self._spinner.render_now()
+            return
+
+        if event.kind == "auto_compact_completed":
+            self._finish_stream()
+            summary = str(event.payload.get("summary", "")).strip()
+            message = f"[status] {summary}" if summary else "[status] context compacted"
+            self._print_line(
+                colorize_cli_message(message, "status", self._color_enabled)
+            )
+            if self._input_active:
+                self._spinner.pause()
+            else:
+                self._spinner.resume()
+                self._spinner.set_label("thinking")
+                self._spinner.render_now()
+            return
+
+        if event.kind == "auto_compact_failed":
+            self._finish_stream()
+            error = str(event.payload.get("error", "")).strip()
+            message = (
+                f"[error] auto-compact failed: {error}"
+                if error
+                else "[error] auto-compact failed"
+            )
+            self._print_line(
+                colorize_cli_message(message, "error", self._color_enabled)
+            )
+            if self._input_active:
+                self._spinner.pause()
+            else:
+                self._spinner.resume()
+                self._spinner.set_label("thinking")
+                self._spinner.render_now()
+            return
+
         if event.kind == "assistant_delta":
             delta = str(event.payload.get("delta", ""))
             if not delta: