codex-agent-framework 0.1.24__tar.gz → 0.1.26__tar.gz

{codex_agent_framework-0.1.24 → codex_agent_framework-0.1.26}/CHANGELOG.md

@@ -4,6 +4,18 @@ All notable changes to this project will be documented in this file.
 
 This project loosely follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and uses semantic versioning where practical.
 
+## [0.1.25] - 2026-05-12
+### Added
+- Add bundled plugin `requirements.txt` metadata for built-in capabilities and move image generation, realtime voice, and TTS into plugin-scoped packages.
+- Add a selectable Textual TUI conversation pane with mouse scrolling, Ctrl+C selection copy, lazy older-transcript loading, and bottom-aware auto-scroll during streaming.
+
+### Changed
+- Continue migrating built-in capability instructions and dependencies toward the plugin-first architecture.
+- Remove legacy voice and prompt files that are now owned by dedicated built-in plugins.
+
+### Tests
+- Expand regression coverage across agent/plugin/runtime/chat behavior; the current suite validates at 546 passed.
+
 ## [0.1.24] - 2026-05-10
 ### Added
 - Add dedicated slash-command lifecycle events, `SlashCommandStartEvent` and `SlashCommandDoneEvent`, with command name, arguments, status, result, and error metadata.

{codex_agent_framework-0.1.24/codex_agent_framework.egg-info → codex_agent_framework-0.1.26}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codex-agent-framework
-Version: 0.1.24
+Version: 0.1.26
 Summary: A lightweight event-driven Codex agent runtime.
 Author: Baptiste
 License-Expression: MIT
@@ -17,7 +17,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: audioop-lts; python_version >= "3.13"
 Requires-Dist: beautifulsoup4
-Requires-Dist: codex-backend-sdk
+Requires-Dist: codex-backend-sdk>=0.3.1
 Requires-Dist: fastapi
 Requires-Dist: filetype
 Requires-Dist: modict
@@ -176,17 +176,16 @@ agent = Agent(session="latest")
 agent.interact()
 ```
 
-Voice is disabled by default. Enable it explicitly for a voice session:
+Voice features are disabled by default. The realtime voice plugin uses Codex/ChatGPT auth through `codex-backend-sdk`; the legacy synchronized TTS stream still uses the OpenAI SDK speech endpoint when explicitly enabled.
 
 ```python
 from codex_agent import Agent
 
 agent = Agent(
     session="latest",
-    openai_api_key="sk-...",
-    voice_enabled=True,
-    voice="nova",
+    builtin_plugins=["realtime", "memory", "planner", "scheduler"],
 )
+agent.load_builtin_plugin("realtime")
 agent.interact()
 ```
 
@@ -473,7 +472,7 @@ The default agent includes practical local tools grouped as plugins:
 | Desktop | `desktop_start_session`, `desktop_run_commands` | Screenshot-backed Linux desktop automation. |
 | Subagents | `subagents_run` | Ephemeral child agents from predefined profiles. |
 
-Current built-in plugin modules include `files`, `content`, `bash`, `python`, `interface`, `environment`, `memory`, `planner`, `scheduler`, `browser`, `desktop`, and `subagents`.
+Current built-in plugin modules include `files`, `content`, `bash`, `python`, `interface`, `environment`, `context`, `memory`, `planner`, `scheduler`, `browser`, `desktop`, and `subagents`.
 
 The `subagents` plugin ships an `explorer` profile: a read-only codebase inspection agent.
 
@@ -482,7 +481,7 @@ Select built-ins explicitly when you want a smaller agent:
 ```python
 agent = Agent(
     session="latest",
-    builtin_plugins=["files", "content", "bash", "python", "memory", "planner"],
+    builtin_plugins=["files", "content", "bash", "python", "environment", "context", "memory", "planner"],
 )
 ```
 
@@ -534,8 +533,8 @@ from codex_agent import Agent
 agent = Agent(
     session="latest",
     model="gpt-5.5",
-    reasoning_effort="medium",
-    verbosity="medium",
+    reasoning={"effort": "medium", "summary": "auto"},
+    text={"verbosity": "medium"},
     input_token_limit=128000,
     auto_compact=True,
     web_search_enabled=False,
@@ -550,9 +549,8 @@ Or update saved config from the CLI:
 ```bash
 codex-agent config get
 codex-agent config set input_token_limit 128000
-codex-agent config set voice_enabled=true
-codex-agent config set builtin_plugins='["memory", "planner", "scheduler"]'
-codex-agent config set --no-save verbosity low
+codex-agent config set builtin_plugins='["environment", "context", "memory", "planner", "scheduler", "realtime"]'
+codex-agent config set --no-save text='{"verbosity":"low"}'
 ```
 
 Saved local state lives in `~/.codex-agent` by default:
@@ -567,7 +565,7 @@ logs/              detached server/tray logs
 memory.json        durable semantic memory entries
 planner.json       persistent named todos
 wakeups.json       scheduled autonomous wakeups
-agent_config.json  persisted runtime configuration
+agent.config.json  persisted agent configuration
 ```
 
 Override it with:
@@ -610,7 +608,7 @@ python -m pytest
 The current tree validates at:
 
 ```text
-507 passed
+546 passed
 ```
 
 Build distributions:
@@ -622,8 +620,14 @@ python -m build
 python -m twine check dist/*
 ```
 
+### Auth model for backend-backed features
+
+Core LLM calls, memory embeddings, non-streaming STT transcription, and realtime voice use the Codex backend SDK with Codex/ChatGPT auth. Legacy synchronized TTS playback and OpenAI input-token counting still use the OpenAI SDK and may require `openai_api_key` or `OPENAI_API_KEY` when those paths are enabled.
+
 ## Recent releases
 
+- `0.1.26`: align SDK-facing config with Responses API shapes, add the `context` management plugin with selective non-destructive wrapper pruning, keep wrappers out of memory retrieval and compaction payloads, move embeddings/STT/realtime onto Codex backend auth, streamline voice/TTS plugin config, and improve TUI streaming/windowing.
+- `0.1.25`: move built-in capabilities further into plugin-scoped modules, add bundled plugin requirement metadata, and replace the TUI conversation pane with a selectable rich chat area that preserves mouse scrolling, copy selection, lazy transcript loading, and bottom-aware auto-scroll.
 - `0.1.24`: add slash-command lifecycle events and TUI progress/result rendering so commands such as `/compact` and `/help` show running, done, failed, result, and error feedback instead of going silent.
 - `0.1.23`: add SDK event streaming with `Agent.stream()`, root `Agent.add_plugin()`, a faster README flow, the `~/.codex-agent` runtime default, text-first voice defaults, safer prompt templating limited to system messages, smarter `codex-agent` local-server attachment, and clearer TUI tool-call progress lines.
 - `0.1.22`: harden FastAPI/SSE and TUI streaming after a stabilization audit; add finite SSE read timeouts, stale-subscriber shutdown, persisted turn-event locking, synchronous replay delta flushes, process-runtime startup guards, isolated event handlers, and shared version metadata.

{codex_agent_framework-0.1.24 → codex_agent_framework-0.1.26}/README.md

@@ -127,17 +127,16 @@ agent = Agent(session="latest")
 agent.interact()
 ```
 
-Voice is disabled by default. Enable it explicitly for a voice session:
+Voice features are disabled by default. The realtime voice plugin uses Codex/ChatGPT auth through `codex-backend-sdk`; the legacy synchronized TTS stream still uses the OpenAI SDK speech endpoint when explicitly enabled.
 
 ```python
 from codex_agent import Agent
 
 agent = Agent(
     session="latest",
-    openai_api_key="sk-...",
-    voice_enabled=True,
-    voice="nova",
+    builtin_plugins=["realtime", "memory", "planner", "scheduler"],
 )
+agent.load_builtin_plugin("realtime")
 agent.interact()
 ```
 
@@ -424,7 +423,7 @@ The default agent includes practical local tools grouped as plugins:
 | Desktop | `desktop_start_session`, `desktop_run_commands` | Screenshot-backed Linux desktop automation. |
 | Subagents | `subagents_run` | Ephemeral child agents from predefined profiles. |
 
-Current built-in plugin modules include `files`, `content`, `bash`, `python`, `interface`, `environment`, `memory`, `planner`, `scheduler`, `browser`, `desktop`, and `subagents`.
+Current built-in plugin modules include `files`, `content`, `bash`, `python`, `interface`, `environment`, `context`, `memory`, `planner`, `scheduler`, `browser`, `desktop`, and `subagents`.
 
 The `subagents` plugin ships an `explorer` profile: a read-only codebase inspection agent.
 
@@ -433,7 +432,7 @@ Select built-ins explicitly when you want a smaller agent:
 ```python
 agent = Agent(
     session="latest",
-    builtin_plugins=["files", "content", "bash", "python", "memory", "planner"],
+    builtin_plugins=["files", "content", "bash", "python", "environment", "context", "memory", "planner"],
 )
 ```
 
@@ -485,8 +484,8 @@ from codex_agent import Agent
 agent = Agent(
     session="latest",
     model="gpt-5.5",
-    reasoning_effort="medium",
-    verbosity="medium",
+    reasoning={"effort": "medium", "summary": "auto"},
+    text={"verbosity": "medium"},
     input_token_limit=128000,
     auto_compact=True,
     web_search_enabled=False,
@@ -501,9 +500,8 @@ Or update saved config from the CLI:
 ```bash
 codex-agent config get
 codex-agent config set input_token_limit 128000
-codex-agent config set voice_enabled=true
-codex-agent config set builtin_plugins='["memory", "planner", "scheduler"]'
-codex-agent config set --no-save verbosity low
+codex-agent config set builtin_plugins='["environment", "context", "memory", "planner", "scheduler", "realtime"]'
+codex-agent config set --no-save text='{"verbosity":"low"}'
 ```
 
 Saved local state lives in `~/.codex-agent` by default:
@@ -518,7 +516,7 @@ logs/              detached server/tray logs
 memory.json        durable semantic memory entries
 planner.json       persistent named todos
 wakeups.json       scheduled autonomous wakeups
-agent_config.json  persisted runtime configuration
+agent.config.json  persisted agent configuration
 ```
 
 Override it with:
@@ -561,7 +559,7 @@ python -m pytest
 The current tree validates at:
 
 ```text
-507 passed
+546 passed
 ```
 
 Build distributions:
@@ -573,8 +571,14 @@ python -m build
 python -m twine check dist/*
 ```
 
+### Auth model for backend-backed features
+
+Core LLM calls, memory embeddings, non-streaming STT transcription, and realtime voice use the Codex backend SDK with Codex/ChatGPT auth. Legacy synchronized TTS playback and OpenAI input-token counting still use the OpenAI SDK and may require `openai_api_key` or `OPENAI_API_KEY` when those paths are enabled.
+
 ## Recent releases
 
+- `0.1.26`: align SDK-facing config with Responses API shapes, add the `context` management plugin with selective non-destructive wrapper pruning, keep wrappers out of memory retrieval and compaction payloads, move embeddings/STT/realtime onto Codex backend auth, streamline voice/TTS plugin config, and improve TUI streaming/windowing.
+- `0.1.25`: move built-in capabilities further into plugin-scoped modules, add bundled plugin requirement metadata, and replace the TUI conversation pane with a selectable rich chat area that preserves mouse scrolling, copy selection, lazy transcript loading, and bottom-aware auto-scroll.
 - `0.1.24`: add slash-command lifecycle events and TUI progress/result rendering so commands such as `/compact` and `/help` show running, done, failed, result, and error feedback instead of going silent.
 - `0.1.23`: add SDK event streaming with `Agent.stream()`, root `Agent.add_plugin()`, a faster README flow, the `~/.codex-agent` runtime default, text-first voice defaults, safer prompt templating limited to system messages, smarter `codex-agent` local-server attachment, and clearer TUI tool-call progress lines.
 - `0.1.22`: harden FastAPI/SSE and TUI streaming after a stabilization audit; add finite SSE read timeouts, stale-subscriber shutdown, persisted turn-event locking, synchronous replay delta flushes, process-runtime startup guards, isolated event handlers, and shared version metadata.

{codex_agent_framework-0.1.24 → codex_agent_framework-0.1.26}/codex_agent/__init__.py

@@ -13,6 +13,7 @@ from .event import (
     AssistantTurnStartEvent,
     AudioPlaybackEvent,
     CompactionCompletedEvent,
+    CompactionFailedEvent,
     CompactionRequestedEvent,
     Event,
     EventBus,
@@ -25,6 +26,13 @@ from .event import (
     ResponseReasoningDeltaEvent,
     ResponseStartEvent,
     ResponseToolCallsDeltaEvent,
+    RealtimeApiEvent,
+    RealtimeErrorEvent,
+    RealtimeSessionRefreshEvent,
+    RealtimeStartedEvent,
+    RealtimeStartingEvent,
+    RealtimeStoppedEvent,
+    RuntimeStateChangedEvent,
     ServerToolCallEvent,
     SessionDeletedEvent,
     SlashCommandDoneEvent,
@@ -50,14 +58,14 @@ from .message import (
 )
 from .image import ImageMessage
 from .mainloop import MainLoopManager
-from .plugin import Plugin, PluginsManager, event_handler, hook_handler
+from .plugin import Plugin, PluginsManager, event_handler, hook_handler, stream_processor
 from .command import AgentCommand, AgentFuture, AgentTurn, CommandManager, command
 from .stream import AgentStream
 from .provider import ProvidersManager, provider
 from .runtime import RuntimeManager
 from .sessions import AgentSession, SessionsManager
 from .status import StatusManager
-from .tool import ImageGenerationTool, ServerTool, Tool, ToolsManager, WebSearchTool, get_agent, get_current_call_id, tool
+from .tool import ServerTool, Tool, ToolsManager, WebSearchTool, get_agent, get_current_call_id, tool
 from .worker import SummarizerWorker, TurnSummaryWorker, Worker
 from .version import __version__
 
@@ -78,6 +86,7 @@ __all__ = [
     "AssistantTurnStartEvent",
     "AudioPlaybackEvent",
     "CompactionCompletedEvent",
+    "CompactionFailedEvent",
     "CompactionRequestedEvent",
     "Event",
     "EventBus",
@@ -90,6 +99,13 @@ __all__ = [
     "ResponseReasoningDeltaEvent",
     "ResponseStartEvent",
     "ResponseToolCallsDeltaEvent",
+    "RealtimeApiEvent",
+    "RealtimeErrorEvent",
+    "RealtimeSessionRefreshEvent",
+    "RealtimeStartedEvent",
+    "RealtimeStartingEvent",
+    "RealtimeStoppedEvent",
+    "RuntimeStateChangedEvent",
     "ServerToolCallEvent",
     "SessionDeletedEvent",
     "SlashCommandDoneEvent",
@@ -116,6 +132,7 @@ __all__ = [
     "PluginsManager",
     "event_handler",
     "hook_handler",
+    "stream_processor",
     "ProvidersManager",
     "AgentCommand",
     "AgentFuture",
@@ -128,7 +145,6 @@ __all__ = [
     "AgentSession",
     "SessionsManager",
     "StatusManager",
-    "ImageGenerationTool",
     "ServerTool",
     "Tool",
     "ToolsManager",

{codex_agent_framework-0.1.24 → codex_agent_framework-0.1.26}/codex_agent/agent.py

@@ -1,6 +1,7 @@
 from .utils import text_content, ensure_runtime_dir, upload_file_to_folder, project_python_env, runtime_join, root_join
 import os
-from .message import DeveloperMessage, ServerToolResponseMessage, SystemMessage, ToolResultMessage, UserMessage, message_from_data
+from modict import modict
+from .message import DeveloperMessage, ServerToolResponseMessage, ToolResultMessage, UserMessage, message_from_data
 from .image import ImageMessage
 from .command import CommandManager
 from .config import ConfigManager
@@ -13,11 +14,9 @@ from .runtime import RuntimeManager
 from .scheduler import AgentScheduler
 from .sessions import SessionsManager
 from .status import StatusManager
-from .tool import ImageGenerationTool, ToolsManager, reset_current_agent, set_current_agent
+from .tool import ToolsManager, reset_current_agent, set_current_agent
 from .ai import AIClient
-from .voice import VoiceProcessor
-from .latex import LaTeXProcessor
-from .event import AssistantInterruptRequestedEvent, AssistantInterrupted, AssistantInterruptedEvent, AudioPlaybackEvent, EventBus
+from .event import AssistantInterruptRequestedEvent, AssistantInterrupted, AssistantInterruptedEvent, EventBus
 from threading import Event as ThreadEvent
 
 class Agent:
@@ -51,7 +50,8 @@ class Agent:
         self.runtime=RuntimeManager(self)
         self.scheduler=AgentScheduler(self)
         self.hooks=HookManager(self)
-        self.on(AudioPlaybackEvent, self.handle_audio_playback)
+        self.stream_processors=modict()
+        self.stream_processor_owners={}
         self.mainloop_manager=MainLoopManager(self)
         self._interrupt_requested=ThreadEvent()
         self.current_session_id=None
@@ -63,9 +63,6 @@ class Agent:
         self.command_manager.init_folder()
         self.plugins_manager.init_folder()
         self.ai=AIClient(self)
-        self.voice=VoiceProcessor(self)
-        self.latex=LaTeXProcessor()
-        self.content_processors=[self.voice]
         self.command_manager.init_builtin()
         self.plugins_manager.init_builtin()
         self.plugins_manager.init_runtime()
@@ -74,13 +71,13 @@ class Agent:
         self.tools_manager.apply_config_exclusions()
         self.start_new_session() if session == "new" else self.load_session(session)
 
-    def on(self, event_type, handler=None):
+    def on(self, event_type, handler=None, owner=None):
         """Register an event handler.
 
         Can be used as ``agent.on(ResponseContentDeltaEvent, handler)`` or as
         ``@agent.on(ResponseContentDeltaEvent)``.
         """
-        return self.events.on(event_type, handler)
+        return self.events.on(event_type, handler, owner=owner)
 
     def emit(self, event_type, **data):
         event = self.events.emit(event_type, **data)
@@ -203,8 +200,13 @@ class Agent:
     def save_session(self):
         return self.sessions_manager.save()
 
-    def compact_session(self, client=None, model=None, instructions=None):
-        return self.sessions_manager.compact(client=client, model=model, instructions=instructions)
+    def compact_session(self, client=None, model=None, instructions=None, max_compacted_turns=None):
+        return self.sessions_manager.compact(
+            client=client,
+            model=model,
+            instructions=instructions,
+            max_compacted_turns=max_compacted_turns,
+        )
 
     def add_message(self,msg,pending=False):
         """Queue a message for the next pending flush, except tool results."""
@@ -238,7 +240,7 @@ class Agent:
             self.add_message(img,pending=pending)
             return img
 
-    def add_tool(self,func=None,name=None,description=None,parameters=None,required=None,tool_type=None,format=None):
+    def add_tool(self,func=None,name=None,description=None,parameters=None,required=None,tool_type=None,format=None,owner=None):
         return self.tools_manager.add(
             func=func,
             name=name,
@@ -247,18 +249,59 @@ class Agent:
             required=required,
             tool_type=tool_type,
             format=format,
+            owner=owner,
         )
 
-    def add_provider(self, func=None, name=None):
-        return self.providers_manager.add(func=func, name=name)
+    def add_provider(self, func=None, name=None, owner=None):
+        return self.providers_manager.add(func=func, name=name, owner=owner)
 
-    def add_command(self, func=None, name=None):
-        return self.command_manager.add(func, name=name)
+    def add_command(self, func=None, name=None, owner=None):
+        return self.command_manager.add(func, name=name, owner=owner)
 
     def add_plugin(self, plugin):
         return self.plugins_manager.add_plugin(plugin)
 
-    def add_hook(self, key, func=None):
+    def load_builtin_plugin(self, name):
+        return self.plugins_manager.load_builtin_plugin(name)
+
+    def load_runtime_plugin(self, name_or_path):
+        return self.plugins_manager.load_runtime_plugin(name_or_path)
+
+    def unload_plugin(self, plugin_or_name, timeout=None):
+        return self.plugins_manager.unload_plugin(plugin_or_name, timeout=timeout)
+
+    def reload_plugin(self, plugin_or_name, timeout=None):
+        return self.plugins_manager.reload_plugin(plugin_or_name, timeout=timeout)
+
+    def add_stream_processor(self, key, processor, owner=None):
+        processors = self.stream_processors.setdefault(str(key), [])
+        processors.append(processor)
+        if owner is not None:
+            self.stream_processor_owners.setdefault(str(key), {})[processor] = owner
+        return processor
+
+    def get_stream_processors(self, key):
+        return list(self.stream_processors.get(str(key), []))
+
+    def remove_stream_processors_by_owner(self, owner):
+        removed = modict()
+        for key, processors in list(self.stream_processors.items()):
+            owners = self.stream_processor_owners.get(key, {})
+            kept = []
+            for processor in processors:
+                if owners.get(processor) == owner:
+                    removed.setdefault(key, []).append(processor)
+                    owners.pop(processor, None)
+                else:
+                    kept.append(processor)
+            if kept:
+                self.stream_processors[key] = kept
+            else:
+                self.stream_processors.pop(key, None)
+                self.stream_processor_owners.pop(key, None)
+        return removed
+
+    def add_hook(self, key, func=None, owner=None):
         """Register a runtime hook.
 
         Hooks are callable extension points keyed by name. They are runtime
@@ -267,13 +310,13 @@ class Agent:
         """
         if func is None:
             def decorator(f):
-                return self.add_hook(key, f)
+                return self.add_hook(key, f, owner=owner)
             return decorator
 
         if hasattr(self, "mainloop_manager") and self.mainloop_manager.should_queue_state_command():
-            return self.submit("add_hook", key=key, func=func).wait().result
+            return self.submit("add_hook", key=key, func=func, owner=owner).wait().result
 
-        return self.hooks.add(key, func)
+        return self.hooks.add(key, func, owner=owner)
 
     def has_hook(self, key):
         return self.hooks.has(key)
@@ -298,44 +341,6 @@ class Agent:
     def get_status(self):
         return self.status_manager.get_status()
 
-    def create_image(
-        self,
-        prompt,
-        input_images=None,
-        model="gpt-5.4",
-        reasoning_effort="medium",
-        verbosity="medium",
-        output_format="png",
-        size=None,
-        quality=None,
-        background=None,
-        moderation=None,
-    ):
-        """Generate an image directly and return the saved image path."""
-        content = [{"type": "text", "text": prompt}]
-        for source in input_images or []:
-            content.extend(ImageMessage(content=source).content_to_response_parts())
-
-        system_prompt = text_content(os.path.join(os.path.dirname(__file__), "prompts", "image_generation_system_prompt.txt"))
-        message = self.ai.run(
-            messages=[SystemMessage(content=system_prompt), UserMessage(content=content)],
-            tools=[ImageGenerationTool(
-                output_format=output_format,
-                size=size,
-                quality=quality,
-                background=background,
-                moderation=moderation,
-            ).to_response_tool()],
-            model=model,
-            reasoning_effort=reasoning_effort,
-            verbosity=verbosity,
-        )
-        for item in message.output_items:
-            if item.get("type") == "image_generation_call":
-                return ServerToolResponseMessage(item=item).content
-        else:
-            raise RuntimeError("The image_generation tool did not return an image.")
-
     def get_response(self):
         """Run the model, emit response events, and store the final message.
 
@@ -358,7 +363,7 @@ class Agent:
             messages=messages,
             tools=self.tools_manager.get_response_tools(),
             prompt_cache_key=self.current_session_id,
-            **self.config.extract('model','reasoning_effort','verbosity','parallel_tool_calls')
+            **self.config.extract('model','reasoning','text','parallel_tool_calls')
         )
         self.sessions_manager.append_message(message)
         for item in message.output_items:
@@ -366,20 +371,6 @@ class Agent:
                 self.sessions_manager.append_message(ServerToolResponseMessage(item=item))
         return message
 
-    def speak(self,text,**kwargs):
-        audio=self.ai.text_to_audio(text,**kwargs)
-        if audio is not None:
-            self.emit(AudioPlaybackEvent, audio=audio)
-
-    def handle_audio_playback(self, event):
-        audio = event.get("audio")
-        if audio is None:
-            return None
-        if self.has_hook("audio_playback_hook"):
-            return self.run_hooks("audio_playback_hook", audio)
-        from .voice import silent_play
-        return silent_play(audio)
-
     def listen(self, source, language=None, **kwargs):
         """
         description: |