python-codex 0.4.7__py3-none-any.whl → 0.5.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,905 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: python-codex
3
- Version: 0.4.7
4
- Summary: A minimal Python extraction of Codex's main agent loop
5
- License-File: LICENSE
6
- Requires-Python: >=3.6.2
7
- Requires-Dist: dataclasses>=0.8; python_version < '3.7'
8
- Requires-Dist: importlib-metadata>=4.8.3; python_version < '3.8'
9
- Requires-Dist: loguru>=0.7.3
10
- Requires-Dist: prompt-toolkit>=3.0.36
11
- Requires-Dist: pyyaml>=6.0
12
- Requires-Dist: requests>=2.27.1
13
- Requires-Dist: starlette<0.20,>=0.19.1; python_version < '3.7'
14
- Requires-Dist: starlette>=0.19.1; python_version >= '3.7'
15
- Requires-Dist: tomli<2,>=1.2.3; python_version < '3.11'
16
- Requires-Dist: typing-extensions>=4.1.1; python_version < '3.8'
17
- Requires-Dist: uvicorn<0.17,>=0.16.0; python_version < '3.7'
18
- Requires-Dist: uvicorn>=0.16.0; python_version >= '3.7'
19
- Description-Content-Type: text/markdown
20
-
21
- # pycodex
22
-
23
- English README. Chinese version: `README_ZH.md`
24
-
25
- 0.4.0 core/workspace package split and dependency cleanup: [release notes](docs/RELEASE_0.4.0.md).
26
-
27
- 0.3.3 workspace viewers and reasoning stream fix: [release notes](docs/RELEASE_0.3.3.md).
28
-
29
- 0.3.2 startup fixes: [release notes](docs/RELEASE_0.3.2.md).
30
-
31
- 0.3.0 release preparation and Python API migration: `docs/RELEASE_0.3.0.md`.
32
-
33
- PyPI distributions:
34
-
35
- - Core package: `python-codex`
36
- - Workspace and integrations: `pycodex-ws`
37
-
38
- The import path remains `pycodex`; the CLI commands are `pycodex` and
39
- `pycodex-ws`.
40
-
41
- `python-codex` contains the Agent, runtime, CLI, standard tools, Responses
42
- compatibility server and portable storage support. `pycodex-ws` depends on the
43
- exact matching core version and contains the browser workspace, viewer assets
44
- and optional Feishu integration. It owns the
45
- `workspace_server` and `pycodex_ws` modules and the `pycodex-ws` command.
46
-
47
- The Feishu SDK is not installed automatically. Install it in the workspace's
48
- Python environment with `python3 -m pip install lark-oapi` to enable `/link`.
49
- If `lark_oapi` cannot be imported, `/link` reports that it is unavailable.
50
-
51
- Core does not require Pillow. Without it, `view_image` sends the original image
52
- bytes with `detail="original"`, including when `high` is requested. With Pillow
53
- installed, the default path retains resizing to a maximum dimension of 2048.
54
- `pycodex-ws` installs Pillow for image preview validation.
55
-
56
- Portable encryption uses a bundled pure Python AES-GCM subset of pyaes and
57
- tlslite-ng, under MIT and BSD licenses. Core does not require a crypto package
58
- or compiler and ships a `py3-none-any` wheel. Existing `PCX1` bundles and call
59
- tokens remain compatible; large bundles take longer to encrypt and decrypt.
60
- See the [source notes](pycodex/utils/_aes_gcm/README.md).
61
-
62
- The Responses compatibility server and workspace HTTP server use Starlette
63
- and Uvicorn directly. Neither package requires FastAPI, Pydantic or
64
- `pydantic-core`. Python 3.6 uses Starlette 0.19.x; newer interpreters resolve
65
- a compatible Starlette release through its Python version metadata.
66
-
67
- This repository extracts the core Codex agent loop from upstream Codex
68
- (`https://github.com/openai/codex`) into a deliberately small Python version,
69
- while preserving the two most important layers:
70
-
71
- - `submission_loop`: sequentially consumes submitted operations.
72
- - `run_turn`: keeps executing `model sample -> tool call -> feed tool result
73
- back into the model` inside a single turn until a final answer is reached.
74
-
75
- Relevant Rust reference points:
76
-
77
- - `codex-rs/core/src/codex.rs` -> `submission_loop`
78
- - `codex-rs/core/src/codex.rs` -> `run_turn`
79
- - `codex-rs/core/src/codex.rs` -> `run_sampling_request`
80
- - `codex-rs/core/src/tools/router.rs` -> `ToolRouter`
81
- - `codex-rs/core/src/stream_events_utils.rs` -> `handle_output_item_done`
82
-
83
- ## Quick Start
84
-
85
- Install the core CLI, or install the workspace package for the browser frontend:
86
-
87
- ```bash
88
- python3 -m pip install python-codex
89
- python3 -m pip install pycodex-ws
90
- ```
91
-
92
- The core supports Python 3.6.2 or newer. The workspace package requires Python
93
- 3.7 or newer. Commands are installed into
94
- that interpreter's scripts directory. On a machine with multiple Python
95
- installations, select the same interpreter explicitly for startup:
96
-
97
- ```bash
98
- python3 -m pycodex
99
- python3 -m workspace_server --help
100
- ```
101
-
102
- If `pycodex` is missing from `PATH` or resolves to an older installation, use these
103
- module entry points. `python3 -m pip --version` shows which Python environment
104
- received the installation.
105
-
106
- For an existing Web installation, use `python3 -m pip install --upgrade pycodex-ws`
107
- to upgrade the workspace and its matching core together. Installing only
108
- `python-codex` provides the CLI and core API.
109
-
110
- For development from this repository, install dependencies first:
111
-
112
- ```bash
113
- uv sync --dev
114
- ```
115
-
116
- Try the real entry points:
117
-
118
- ```bash
119
- uv run pycodex "Reply with exactly OK."
120
- uv run pycodex
121
- ```
122
-
123
- Tests are split by package. `uv run --dev pytest` runs both suites;
124
- `uv run --dev pytest packages/pycodex-ws/tests` runs workspace and integration
125
- tests. To check core without installing workspace, use a separate
126
- environment:
127
-
128
- ```bash
129
- UV_PROJECT_ENVIRONMENT=.tmp/core-tests uv sync --group test
130
- UV_PROJECT_ENVIRONMENT=.tmp/core-tests uv run --group test pytest tests
131
- ```
132
-
133
- See [test environments and coverage](tests/TESTS.md). Core tests support Python
134
- 3.6.2+; workspace tests require Python 3.7+.
135
-
136
- A VS Code frontend is available in [`packages/pycodex-ws/extensions/vscode`](packages/pycodex-ws/extensions/vscode/README.md).
137
- It belongs to the workspace frontend and is packaged separately as a VSIX.
138
- It reuses an existing `pycodex-ws>=0.4.0` installation. On first use, missing
139
- or older backends are installed/upgraded with pip in the selected Python
140
- environment on the workspace host; compatible versions need no network check.
141
- It starts a dedicated `pycodex-ws` workspace on the project's host and opens a
142
- chat in the left sidebar using VS Code's stable WebviewView API. Each chat
143
- creates a workspace session with no code context attached. You can insert code
144
- from editors, diffs or file-tree selections as editable tokens among your words.
145
- Each token expands at its position when you send the message.
146
- Previous chats fold into live status entries; each can be
147
- reopened or closed, and models can be changed with `/model`.
148
- Remote workspaces run Python on the remote host. A stdin pipe ties each backend
149
- to its extension host; normal closure or host failure triggers draining and exit.
150
- `pycodex-ws` always shuts down on stdin EOF, after accepted work finishes.
151
- Keep stdin open while running the server; redirecting it from `/dev/null`
152
- causes the server to shut down.
153
- Workspace configuration uses a temporary JSON file passed to `--workspace-config`;
154
- normal backend exit removes the temporary directory.
155
- The extension uses that host's `~/.codex/config.toml` by default.
156
- The chat settings menu selects a project venv or another Python interpreter
157
- and edits the Codex profile. Changes offer a window reload when a backend is
158
- already running.
159
- Its context ring shows the same remaining budget and usage details as the
160
- workspace web frontend.
161
- See the extension README for building a VSIX, server setup and Remote SSH usage.
162
-
163
- ## Design Tradeoffs
164
-
165
- This is not a 1:1 port of the Rust implementation. The current goal is a
166
- minimal reusable kernel that converges on the upstream behavior over time:
167
-
168
- 1. Use a thin `ModelClient` protocol to abstract the model side.
169
- 2. Use `ToolRegistry` to manage tool specs and executors.
170
- 3. Use `AgentLoop` to implement the core closed loop.
171
- 4. Use `AgentRuntime` to preserve the outer submission queue so it can keep
172
- converging toward Rust's `submission_loop` later.
173
-
174
- Intentionally not included yet:
175
-
176
- - TUI / streaming incremental rendering
177
- - MCP / connectors / sandbox / approvals
178
- - memory / compact / review mode
179
- - a full production OpenAI adapter surface
180
-
181
- All of those can be layered on later. For now, the project is focused on
182
- nailing the core tool-augmented reasoning loop first.
183
-
184
- ## Layout
185
-
186
- The top level of `pycodex/` contains the core components and package entry points.
187
- Supporting services live in `tools/`, `portable/` and `utils/`.
188
-
189
- - `pycodex/agent.py`: inner turn loop
190
- - `pycodex/runtime.py`: session commands, submission queues and frontend event subscriptions
191
- - `pycodex/model.py`: model client protocol and Responses API adapter
192
- - `pycodex/context.py`: configuration, prompt assembly and instruction loading
193
- - `pycodex/protocol.py`: minimal conversation item / prompt protocol
194
- - `pycodex/events.py`: typed events, plain-text views and stateful presentation
195
- - `pycodex/bootstrap.py`: frontend-independent model, tools, Agent and session assembly
196
- - `pycodex/cli.py`: single-turn/interactive entry points, terminal I/O executor and input loop
197
- - `pycodex/tools/base_tool.py`: `BaseTool`, `ToolRegistry`, `ToolContext`
198
- - `pycodex/tools/runtime_services.py`: plan, sub-agent, user-input and permissions services
199
- - `pycodex/tools/`: concrete tools and their execution managers
200
- - `pycodex/portable/`: Codex home storage client and server, both included in core
201
- - `pycodex/utils/compat.py`: supported-Python compatibility helpers
202
- - `pycodex/utils/doctor.py`: configuration and connectivity diagnostics
203
- - `pycodex/utils/model_metadata.py`: vendored model metadata lookup
204
- - `pycodex/utils/debug.py`: debug paths and the `pycodex.debug()` implementation
205
- - `pycodex/utils/event_helpers.py`: stateless text, color, summary and result-formatting helpers
206
- - `pycodex/utils/image_utils.py`: image loading, resizing and data-URL preparation
207
- - `pycodex/prompts/`: vendored prompts and model metadata
208
- - `packages/pycodex-ws/workspace_server/`: Web board, API and file/diff viewers
209
- - `packages/pycodex-ws/pycodex_ws/`: Feishu and workspace assembly
210
- - `packages/pycodex-ws/extensions/vscode/`: workspace's VS Code frontend, built separately as a VSIX
211
- - `tests/`: core tests
212
- - `packages/pycodex-ws/tests/`: workspace and cross-frontend integration tests
213
-
214
- See [package layout and import paths](docs/RUNTIME.md#package-layout).
215
-
216
- ## Current Alignment Status
217
-
218
- Current progress is easiest to read in layers:
219
-
220
- - prompt/context alignment:
221
- - the 2026-09-23 audit uses Codex CLI 0.153.4: shared context matches for
222
- `gpt-5.4` first/resume/tool-follow-up requests and suffixed Astra first/resume
223
- requests, with explicit exclusions recorded in `docs/CONTEXT.md`;
224
- - this layer is now mainly handled by `pycodex/context.py` plus vendored
225
- prompt data; model lookup uses the longest matching slug prefix without
226
- rewriting the requested model name.
227
- - turn-loop semantic alignment:
228
- - `AgentLoop` no longer uses a fixed 12-iteration cap by default;
229
- - like upstream, it now converges naturally based on whether there is still
230
- follow-up work or tool handoff to do;
231
- - the local iteration-limit parameter is gone.
232
- - request-level alignment:
233
- - full raw request parity is **not** claimed: client-generated message/result
234
- IDs, the tool catalog, new upstream telemetry, and some permission profiles
235
- still differ;
236
- - the default CLI keeps the `codex-tui` client identity, but intentionally
237
- omits collaboration-mode developer instructions;
238
- - `User-Agent` and rollout `cli_version` use the fixed upstream alignment
239
- version `0.153.4`; session creation does not run `codex --version`;
240
- - `tests/compare_context_requests.py` keeps raw differences separate from
241
- shared-context exclusions; older interactive captures are not a fresh
242
- certification of 0.153.4 interactive behavior.
243
- - structured user input:
244
- - `request_user_input` remains declared, but calls return
245
- `request_user_input is unavailable in Default mode` without opening a question;
246
- - attaching a frontend or input handler does not enable it. The generic runtime
247
- input and permission services remain available;
248
- - local tool-result `success` metadata is omitted from API requests, including
249
- when replaying answers from older rollouts.
250
-
251
- See `docs/ALIGNMENT.md` for more detailed notes.
252
-
253
- `PYCODEX_DUMP` records downstream `stream_completed` and `stream_error_type`
254
- alongside usage, finish reason, and exact token IDs. Failed attempts remain
255
- in the dump, including failures before the first chunk. Chat SSE `error`
256
- payloads propagate as `response.failed`, never as an empty completed response.
257
- Chained failures also retain `stream_error_cause_type` and, for HTTP errors,
258
- `stream_error_http_status`; exception messages, headers and URLs are not
259
- copied into these diagnostic fields.
260
-
261
- Downstream errors include elapsed time, timeout, failure stage, response-header
262
- and first-chunk timing, and received chunk count directly in the error message.
263
-
264
- ## Live Model Integration
265
-
266
- If this machine already has a Codex CLI configuration, `pycodex` can reuse the
267
- `model`, `model_provider`, `base_url`, and `env_key` from
268
- `~/.codex/config.toml` directly:
269
-
270
- ```python
271
- from pycodex import ResponsesModelClient
272
-
273
- client = ResponsesModelClient.from_codex_config()
274
- ```
275
-
276
- `model_provider` is optional and defaults to `openai`. The built-in provider
277
- does not require a `[model_providers.openai]` table: it uses
278
- `https://api.openai.com/v1` and reads `OPENAI_API_KEY`. Set `openai_base_url` to
279
- override this address. Explicit provider tables retain their configured
280
- `base_url` and optional `env_key`; unknown provider names raise a configuration
281
- error. The model still comes from `model` in the configuration or selected
282
- profile.
283
-
284
- Authentication currently uses API keys from environment variables or `.env`;
285
- Codex desktop/CLI ChatGPT sign-in credentials in `auth.json` or the OS keychain
286
- are not loaded.
287
-
288
- The current implementation uses the streaming OpenAI-compatible `/responses`
289
- endpoint. This path has already been validated against the local
290
- `~/.codex/config.toml` setup.
291
-
292
- When launched through the CLI, `pycodex` also loads `.env` from the same
293
- configuration directory before reading config (typically `~/.codex/.env`), so
294
- provider keys and similar environment variables can live there. To match
295
- upstream Codex, variables starting with `CODEX_` are not imported from `.env`.
296
-
297
- ## pycodex CLI
298
-
299
- `pycodex` now defaults to a minimal interactive entry point. Internally it uses
300
- `AgentRuntime` to drive the turn submission loop and reuses
301
- `~/.codex/config.toml` by default:
302
-
303
- ```bash
304
- pycodex
305
- pycodex "Summarize this repo in one sentence."
306
- printf 'Reply with exactly OK.' | pycodex
307
- pycodex --json "Reply with exactly OK."
308
- pycodex --profile model_proxy "Reply with exactly OK."
309
- pycodex --profile opus --use-messages "Reply with exactly OK."
310
- pycodex --vllm-endpoint http://127.0.0.1:18000 "Reply with exactly OK."
311
- pycodex --put @127.0.0.1:5577
312
- pycodex --put /data/.codex/@127.0.0.1:5577
313
- pycodex --call SECRET-CALLID@127.0.0.1:5577 "Reply with exactly OK."
314
- pycodex doctor
315
- pycodex-ws --listen 0.0.0.0:6007 --workspace-config ./workspaces.json
316
- pycodex-ws --listen 0.0.0.0:6007 --workspace-config ./workspaces.json --password 12345
317
- ```
318
-
319
- Current behavior:
320
-
321
- - with no argv prompt and a TTY stdin, enter interactive mode
322
- - with an argv prompt or piped stdin, run a single turn
323
- - `pycodex-ws` starts the standalone browser workspace manager and serves each
324
- workspace with a board pane and a pycodex session pane
325
- - the left pane is the board, containing an optional live HTML Canvas tab and closable read-only file/diff tabs that
326
- scroll together in one toolbar; a mouse wheel scrolls overflowing tab strips
327
- horizontally.
328
- Click Markdown file links such as `[source](src/app.py:42)` or
329
- `[source](/data/project/app.py#L42)` to open a view. Git-style diff links use
330
- `[changes](diff:HEAD:src/app.py)` for working changes,
331
- `[versions](diff:HEAD~1..HEAD:src/app.py)` for two revisions, and
332
- `[files](diff:old.py..new.py)` for two local files. Append `#L42` or `#R42`
333
- to a diff link to select a 1-based line on the left or right.
334
- Views open when the user clicks; model replies alone do not create tabs.
335
- Reopening the same file or comparison selects its existing tab in that
336
- browser, including when the line changes. Other pages keep their selection.
337
- Tabs show filenames first, adding a short directory suffix
338
- for duplicate names. Git versions use short commit IDs in a secondary label.
339
- Long labels use the same width limit as session tabs, shortening the secondary
340
- information before the filename;
341
- hover titles retain the complete paths and revisions. Text,
342
- Markdown, HTML, common raster images and text diffs are supported. Monaco
343
- provides syntax highlighting, find, folding and character-level diffs.
344
- Right-click code and choose **Insert into chat**, or select text and use the
345
- toolbar button, to insert context at the current chat draft's cursor.
346
- Context appears as a separate block among your words. Click it to edit or
347
- remove it; Backspace and Delete also remove a block as one element.
348
- Context identifies the full view label, including both revisions of a diff.
349
- Selections include only the selected text and range; otherwise it records the
350
- cursor location. Diff context also identifies the selected side.
351
- The draft stays editable until you send it.
352
- Its code and styles load only when a code or diff view opens, so Canvas,
353
- Markdown and HTML previews do not load the editor. Viewer assets use gzip
354
- when supported and revalidate cached content with ETags.
355
- Narrow content panes automatically use a unified diff; targeting the left
356
- side keeps both sides visible. Markdown and HTML links default to previews;
357
- line targets open their source. Clicking a link without a line returns that
358
- same tab to preview. Manual tab switches preserve the chosen mode and reading state.
359
- All views remain
360
- read-only, with text selection and copying available. Local files update
361
- automatically, including both sides of file diffs and the working side of
362
- Git diffs; Git revisions stay fixed. Updates keep the active editor and
363
- reading state without replaying an earlier line request.
364
- Canvas is for explanations and custom visuals; source and diffs use links.
365
- `[Canvas](workspace://canvas)` returns to the live page; an HTML file link opens
366
- a preview tab, and `page.html:1` opens its source. File links work even with an empty `--toolset`.
367
- Canvas and HTML previews share the same origin and script permissions;
368
- the browser loads their resources with the workspace login cookie.
369
- Open views survive reloads and server restarts. See
370
- [workspace content views](docs/RUNTIME.md#workspace-content-views) for the
371
- complete syntax and limits.
372
- - interactive mode exits through Ctrl+D on an empty prompt or a single Ctrl+C;
373
- accepted work finishes before cleanup and normal exit.
374
- A `[closing]` message explains the wait. Press Ctrl+C again while closing
375
- to immediately exit the CLI with status 130. This skips remaining cleanup;
376
- external tool processes may keep running.
377
- - interactive mode shows a compact event stream for user-visible phases such as
378
- tool execution and model follow-up after tool results
379
- - startup shows a welcome, model, effort, endpoint and available commands.
380
- - assistant text is printed from streaming deltas directly
381
- - interactive mode supports `/history`, `/title`, `/model`, `/effort`, `/resume`,
382
- `/compact`, and `/fork`
383
- - `/model <name>` switches the model used by later turns in the current
384
- interactive session; `/model` shows the current model and available choices
385
- - `/effort low` switches effort while idle; `/effort` shows the current value,
386
- and `/effort default` restores the model default. Frontend labels show
387
- `model:effort`.
388
- - `/resume` with no argument lists the currently resumable sessions by their
389
- first user-message preview; `/resume 1` resumes the first listed session
390
- - `/resume <session_id>` accepts a full session UUID, including sessions older
391
- than the displayed list. Both forms search `CODEX_HOME/sessions` and
392
- `CODEX_HOME/archived_sessions` and replace the in-memory history with the
393
- selected recorded Codex rollout. Tool calls without recorded results
394
- are omitted during restore; later saved messages and completed tools remain
395
- available, and the source file is left intact.
396
- - `/compact` synthesizes a local handoff summary, replaces the in-memory
397
- conversation history with the compacted view, and appends a compacted-history
398
- entry to the rollout so later `/resume` sees the same state. The handoff remains
399
- model context, but is omitted from frontend conversation blocks and `/history`;
400
- real replies after compaction remain visible, including after resume or fork.
401
- - `/fork` allocates a new Agent/provider session id and lazy rollout while
402
- preserving current history and the workspace tab; the original rollout stays intact
403
- - workspace tabs follow the fork's rollout once it is written. Before that,
404
- restart restores the source recording and creates a fresh fork, preserving
405
- history and title without appending to the source file.
406
- - `model_auto_compact_token_limit = <tokens>` in `config.toml` enables the same
407
- compaction path automatically when the latest reported usage reaches that
408
- threshold before a follow-up sampling request or the next user turn
409
- - `service_tier = "fast"` enables Fast mode for models whose vendored metadata
410
- advertises the `priority` service tier; pycodex follows upstream Codex by
411
- sending `service_tier = "priority"` on the Responses request, while
412
- `service_tier = "default"` or unsupported tiers are omitted
413
- - if a model request fails with `context_length_exceeded`, pycodex now treats
414
- the provider-reported requested token count as a failed-request usage sample,
415
- triggers the same compact path immediately, and retries the request once; if
416
- the compact request is also over the limit, it repeatedly drops the oldest
417
- tool response plus its matching tool call before retrying compact
418
- - new sessions are now recorded under `CODEX_HOME/sessions/.../rollout-*.jsonl`
419
- with a stable session/thread id and per-item append+flush semantics so
420
- `/resume` reads back the same rollout format
421
- - if `TURN_HOOK.md` exists in the workspace root and is non-empty, each
422
- completed turn also forks the just-finished history into a temporary,
423
- non-persisted follow-up session and submits the file contents as the next
424
- user instruction; this is intended for side-effect follow-ups such as
425
- Feishu notifications
426
- - `/link <feishu-email|open_id|chat_id>` in a workspace session attaches it
427
- to a Feishu card when the optional `lark-oapi` SDK can be imported;
428
- multiple sessions in the same workspace process share
429
- one Feishu long-connection listener and route card actions by message id
430
- - Feishu user OAuth stores its refresh token only in
431
- `~/.codex/.feishu_refresh_token` (mode `0600`). Run
432
- `python3 -m pycodex_ws.feishu_oauth` after installing `pycodex-ws` to authorize.
433
- From this repository, use `env -u VIRTUAL_ENV uv run --dev python -m pycodex_ws.feishu_oauth`.
434
- App credentials can stay in `~/.codex/.env`. Each refresh reads
435
- the latest token from the dedicated file and saves its replacement there.
436
- `FEISHU_REFRESH_TOKEN` in `.env` or the environment is no longer used and
437
- can be removed. Refresh is triggered by API calls, with no idle keepalive.
438
- - `pycodex-ws --workspace-config workspaces.json` serves workspace boards from
439
- one process. The JSON file can be either a list or
440
- `{"workspaces": [...]}`; each entry uses `state_path` and `work_dir`, with optional
441
- display/URL `id` and `enable_board` (default `true`), and is exposed at `/w/<id>/`.
442
- `state_path` is a filename prefix: `/data/demo` uses
443
- `/data/demo.pycodex-ws.json` for the session index,
444
- `/data/demo.pycodex-views.json` for file views and `/data/demo.html` for Canvas.
445
- Suffixes are appended, so prefixes containing dots remain distinct.
446
- The resolved `state_path` is the workspace identity; duplicates are rejected
447
- regardless of name or Canvas setting. Relative `state_path` and `work_dir` paths
448
- resolve from the config file directory. The root path `/` shows a workspaces
449
- management page with create/delete controls; omitted names become `workspace-1`,
450
- `workspace-2`, etc. Omitted `state_path` when creating a workspace generates a
451
- writable `/tmp/pcws-*` prefix. `enable_board: false` hides Canvas and disables its
452
- routes, polling and context injection; sessions and file views still persist.
453
- Existing `board: "demo.html"` and `board: false` configurations load with their
454
- original state paths and are saved in the new format. See the
455
- [workspace configuration example](packages/pycodex-ws/README.md).
456
- Add/delete actions and later session-state saves refresh the JSON file.
457
- Board HTML can reference local
458
- images beside the board (including nested paths) with relative URLs; only
459
- `image/*` files contained by the board directory are served.
460
- Assistant Markdown supports KaTeX formulas with `$...$`, `$$...$$`,
461
- `\(...\)`, and `\[...\]` delimiters.
462
- `--password <value>` enables a password-only login page for workspace pages,
463
- APIs, and websocket connections. After login, the browser returns to the
464
- requested workspace path with its query parameters.
465
- Questions and permission requests appear below the conversation only while
466
- waiting for input; they disappear when answered, cancelled, or timed out,
467
- leaving the continuing assistant output at the bottom.
468
- Queued enqueue/steer messages become user conversation blocks when their
469
- turns start, in execution order. Small capsules to the left of the status
470
- pill preview pending Steer (`↑`) and Queue (`◷`) text. Long text is truncated;
471
- hover shows the full pending content. Each disappears when that type has
472
- no waiting input.
473
- - steer is enabled by default in interactive mode: normal input goes into the
474
- runtime steer path, the current request stops at the next safe boundary, and
475
- later steer text is appended to the next model request's `input` in order;
476
- for explicit queueing, use `/queue <message>`, which prints
477
- `[steer] queued: ...` and later `[steer] inserted: ...`
478
- - the default local tool set includes the upstream-aligned subset plus the
479
- pycodex `clock` extension: `shell`, `shell_command`, `exec_command`,
480
- `write_stdin`, `clock`, `exec`, `wait`, `web_search`, `update_plan`,
481
- `request_user_input`, `request_permissions`, `spawn_agent`, `send_input`,
482
- `resume_agent`, `wait_agent`, `close_agent`, `apply_patch`, `grep_files`,
483
- `read_file`, `list_dir`, `view_image`
484
- - `clock(period_m)` sets one periodic clock for the current Agent session;
485
- `null` cancels it. The countdown restarts after each reply and wakes the
486
- Agent with a `<clock_tick>` message containing the current timezone-aware
487
- time when it expires.
488
- - while a background command or clock is pending, the idle status is
489
- `idle: sleeping`
490
- - only the active workspace session tab shows its close button; closing it restores
491
- the selected tab's conversation, draft, and scroll position immediately
492
- after its snapshot loads
493
- - `--vllm-endpoint http://host:port` automatically launches a local
494
- `responses_server` compatibility layer; when the URL path is empty it is
495
- normalized to `/v1`, and `/responses` requests are still forwarded to the
496
- downstream `/v1/chat/completions` endpoint. This local compat path always
497
- uses the canonical Responses request shape, even when the selected model's
498
- metadata enables `responses_lite`. With `--vllm-endpoint`, startup also reads
499
- `/v1/models` and uses the last returned model id for the downstream request.
500
- For `model_provider = "vllm"`, reasoning is preserved across this path:
501
- chat chunks with `reasoning` or `reasoning_content` are translated back into
502
- Responses `reasoning` items, and
503
- historical `reasoning` items are replayed into downstream assistant messages
504
- via the `reasoning` field. Streaming token usage is also requested from vLLM
505
- and forwarded to the final `response.completed.response.usage`. If a
506
- downstream chat stream terminates after emitting only reasoning, with no
507
- assistant content and no tool call, the compat layer discards that partial
508
- reasoning, retries the same downstream request once, and only then emits
509
- `response.failed` with `type = "model_output_invalid"` if the retry is still
510
- reasoning-only
511
- - compat reasoning fragments are forwarded immediately as
512
- `response.reasoning_text.delta`, including thinking from `/v1/messages`.
513
- Continuous reasoning keeps the client's read idle timeout from expiring
514
- while it waits for assistant text; only finalized reasoning items enter history
515
- - malformed JSON in ordinary downstream function-call arguments is detected by
516
- `responses_server` and returned as `response.failed` with `type = "server_error"`;
517
- the client's existing `stream_max_retries` policy retries the current request
518
- before its local tools execute
519
- - downstream connection failures before HTTP response headers or during SSE
520
- reading are converted to `response.failed` for provider retries. Downstream
521
- requests honor environment proxies; see the
522
- [connection troubleshooting notes](docs/responses_server/README.md#连接排查)
523
- for `RemoteDisconnected` and proxy bypass configuration
524
- - standalone `responses_server` now also supports downstream `/v1/messages`
525
- backends via `--outcomming-api messages`, while keeping the internal
526
- canonical request/route logic in chat-completions shape
527
- - `pycodex doctor` checks config, `.env`, API keys, DNS, TCP/TLS, and an
528
- optional live Responses API request
529
-
530
- Current primary uses:
531
-
532
- - verify provider / model / auth configuration
533
- - debug `ResponsesModelClient`
534
- - run minimal single-turn and multi-turn smoke tests
535
-
536
- `doctor` examples:
537
-
538
- ```bash
539
- pycodex doctor
540
- pycodex doctor --skip-live
541
- pycodex doctor --json
542
- ```
543
-
544
- ## Portable Mode
545
-
546
- `Portable Mode` is the quickest way to bring your usual `pycodex` setup into a
547
- fresh machine, container, or debug image.
548
-
549
- The storage client and server are both included in `python-codex`. To start a
550
- local storage service, run this in a separate terminal:
551
-
552
- ```bash
553
- python -m pycodex.portable.server --root .tmp/pycodex_storage --host 127.0.0.1 --port 5577
554
- ```
555
-
556
- Use it like this:
557
-
558
- ```bash
559
- pycodex --put @127.0.0.1:5577
560
- pycodex --put /data/.codex/@127.0.0.1:5577
561
- ```
562
-
563
- - `--put` prints a reusable `SECRET-CALLID@host:port` plus a final one-line
564
- `pycodex --call ...` command
565
- - on the new environment or image, run that printed `--call` command directly
566
- - quickly restoring your usual `config.toml`, `.env`, `AGENTS.md`, and
567
- `skills/` into a clean debug environment
568
- - keeping a new image focused on the bug you are debugging instead of spending
569
- time rebuilding local Codex setup by hand
570
- - bootstrapping `pycodex` even when the target environment does not already
571
- have a populated `~/.codex`
572
- - bare `--put` uses the current user's `~/.codex`
573
- - `--put /path/.codex/@host:port` lets you publish a different Codex home
574
-
575
- ## Example
576
-
577
- ```python
578
- import asyncio
579
- from pathlib import Path
580
-
581
- from pycodex import (
582
- Agent,
583
- BaseTool,
584
- ContextConfig,
585
- ResponsesModelClient,
586
- ToolRegistry,
587
- )
588
-
589
-
590
- class EchoTool(BaseTool):
591
- name = "echo"
592
- description = "Echo the provided text."
593
- input_schema = {
594
- "type": "object",
595
- "properties": {"text": {"type": "string"}},
596
- "required": ["text"],
597
- }
598
-
599
- async def run(self, context, args):
600
- del context
601
- return args["text"]
602
-
603
-
604
- async def main() -> None:
605
- config_path = Path.home() / ".codex" / "config.toml"
606
- model = ResponsesModelClient.from_codex_config(config_path)
607
- context_config = ContextConfig.from_codex_config(config_path)
608
-
609
- tools = ToolRegistry()
610
- tools.register(EchoTool())
611
-
612
- agent = Agent(model, tools, context_config)
613
- result = await agent.run_turn(
614
- ["Call the echo tool with text=hello, then tell me what it returned."]
615
- )
616
- print(result.output_text)
617
-
618
-
619
- asyncio.run(main())
620
- ```
621
-
622
- ### Turn lifecycle
623
-
624
- `Agent` requires a model client, tool registry, and `ContextConfig`. It creates
625
- its own `ContextManager`; cwd, instruction overrides and
626
- extra contextual messages belong in that config. The client's model identifier
627
- is authoritative for the Agent's context. Standalone context construction uses
628
- `ContextManager(config)`; load files through `ContextConfig.from_codex_config`.
629
-
630
- Collaboration modes and their prompt templates are removed. There is no mode
631
- switch in `ContextConfig`, `ToolContext` or `build_agent`; normal interaction,
632
- `update_plan`, structured user input and sub-agents remain independent features.
633
-
634
- Dependencies are ordinary attributes: `model_client`, `tool_registry`,
635
- `context_manager` and `event_handler`. The Agent has no Runtime/Queue reference;
636
- `AgentRuntime` calls its public methods and subscribes to its events.
637
- Pure getter/setter wrappers are removed. `history` remains a tuple snapshot,
638
- and `model_name` and lifecycle properties remain derived values.
639
-
640
- `session_file_path=None`, the default, means **keep history in memory without
641
- recording**. Passing a file path creates an internal rollout recorder. Every
642
- Agent still receives a stable UUIDv7 session id unless an explicit id is supplied.
643
- CLI, Web and Feishu use `build_agent`, which assigns an id and a matching path
644
- under the configured Codex home's `sessions/` directory. Sub-agents leave the path
645
- as `None`, so their turns, compaction, forks and reopening stay in memory.
646
-
647
- When a path is supplied, construction does not create the file or its parent
648
- directories. The first history append or successful compaction creates it,
649
- writing metadata and the current initial history before the new records.
650
- An existing destination raises `FileExistsError`; exclusive creation also
651
- prevents overwriting a file that appears before the first write.
652
- `agent.session_id` and the read-only `agent.session_file_path` expose the identity
653
- and optional recording path without exposing the recorder.
654
-
655
- ```python
656
- agent = Agent(model, tools, context_config) # In-memory session.
657
- agent.resume("~/.codex/sessions/2026/09/22/rollout-example.jsonl")
658
- ```
659
-
660
- `resume(path)` reads an existing file, restores history and session identity, and
661
- continues appending to that same file. Paths accept `~`; the existing
662
- concatenated-JSON and compact-checkpoint loader is reused. Constructing an Agent
663
- and immediately resuming another file leaves no unused rollout behind. Initial
664
- history is recorded only when the new recorded session first writes. `history`
665
- is read-only; there is no public `replace_history()` interface.
666
-
667
- An `Agent` executes one turn or manual compaction at a time.
668
- There are two public turn interfaces:
669
-
670
- - `agent.ask(text)` blocks and returns the turn result.
671
- - `await agent.run_turn(texts, turn_id=None)` executes the ordinary coroutine
672
- and returns the turn result.
673
-
674
- The Agent does not create, hold or return a per-turn Task. Creating a coroutine
675
- does not start it or mark the Agent busy. A host that needs concurrent UI/input
676
- handling can schedule the coroutine with `asyncio.create_task`; synchronous code
677
- can use `ask` or `asyncio.run(agent.run_turn([...]))`. `start_turn` is removed.
678
-
679
- `agent.is_running` reflects actual execution. One idle event backs both that
680
- state and `await agent.wait_until_idle()`; `finally` releases it on exit.
681
- Starting an overlapping operation raises `RuntimeError`; use `AgentRuntime`
682
- for queued or steer submissions. `maybe_invoke` skips a busy or closed Agent;
683
- when idle, it directly awaits the same `run_turn` path and propagates errors.
684
- It returns `True` after that turn finishes, not immediately after scheduling.
685
-
686
- `runtime.is_busy` includes active Agent work and pending submissions, including
687
- direct/background turns that bypass the queue. CLI, web, and Feishu use this
688
- state instead of tracking another busy flag. There is no Agent cancellation API:
689
- `agent.cancel()`, `runtime.cancel_current()` and Task listeners are removed.
690
-
691
- Steer asks `agent.stop_asap()` to end the current execution at a safe boundary.
692
- The in-flight request and all issued tools finish first; the Agent records their
693
- results, emits `turn_interrupted` and raises `TurnInterrupted`, without knowing
694
- why the stop was requested. Runtime alone settles submission futures and starts
695
- the next batch. Runtime-owned steer retains its logical turn id, but starts a new
696
- `run_turn` execution with a fresh iteration count. `/queue` does not request a stop.
697
- Checks after compaction prevent an extra sample when steer arrives during it.
698
- Direct Agent calls never consume Runtime queues; their caller observes
699
- `TurnInterrupted`, and pending input waits for the Runtime worker.
700
-
701
- The core loop has one threshold-compaction entry, then samples, commits output
702
- and executes tools or finishes. Before the first sample, compaction precedes new
703
- user input; later it includes completed tool results. Context-overflow recovery
704
- stays inside sampling: compact and retry once without counting another iteration
705
- or replaying tools. Stops are checked before each request attempt and after the
706
- issued tool batch, without a separate preparation wrapper. Network retries remain
707
- inside the provider.
708
-
709
- `await agent.compact()` computes a summary and commits the replacement history
710
- only after the summary and rollout checkpoint succeed. Failed compaction leaves
711
- the active history unchanged, including tool results pruned from retry prompts.
712
- History persistence failures are surfaced rather than silently ignored.
713
- Manual and automatic compaction share the same execution/observation path.
714
-
715
- Sub-agent status comes from Agent lifecycle events and current busy/queue state,
716
- including direct and background invocations. Bare `Agent.shutdown()` rejects new
717
- calls and disables background hooks, but lets the current turn finish. `runtime.close()`,
718
- `close_agent`, and workspace close wait for accepted work and child workers to
719
- finish naturally; a stuck request can therefore keep close waiting.
720
- `resume_agent` explicitly reopens the child. No turn-abort or synthetic
721
- interruption-result recovery is performed.
722
-
723
- Queue lifecycle uses only `start()` and `close()`. Closing switches off admission;
724
- the worker exits when the queues are empty, without a special shutdown request.
725
- Background invocations obey the same admission switch. Tool shutdown hooks run
726
- once, after draining accepted work. Concurrent or repeated close calls share the
727
- worker's completion and errors; cancelling a caller does not cancel its Agent turn.
728
-
729
- CLI, web and Feishu restore through the backend's `runtime.resume(session_file_path)`,
730
- which delegates to `agent.resume(session_file_path)`. It rejects
731
- active or queued work, loads history, restores supported provider session
732
- identity, switches its internal recorder, and clears stale usage in one
733
- synchronous operation. Load failures leave the existing session unchanged.
734
- The backend broadcasts restored history/title/identity to all attached views.
735
- `agent.resume()` without a path reopens the same in-memory Agent and rebinds its
736
- tool callbacks, without reading or writing a file or changing history, identity,
737
- recorder or usage. It returns `None` and also rejects active or queued work.
738
- Sub-agent services use this form to reopen the child's in-memory history.
739
- There is no separate `reopen()` method.
740
- Model switching and history replacement remain explicit operations because they
741
- enforce state consistency, not merely assign a field.
742
-
743
- ### Shared session backend
744
-
745
- CLI, Web and Feishu all use `AgentRuntime.submit_input`: not just `/model`
746
- and `/resume`, but every session command, steer/queue admission, interactive
747
- questions and permissions, state notifications and shutdown. Frontends only
748
- adapt input and render events. Web no longer imports or runs the CLI shell.
749
- Workspace assembly uses `pycodex_ws.build_runtime()` to add
750
- `/link` and `/unlink`; `pycodex.bootstrap.build_runtime()` provides core commands.
751
-
752
- ```python
753
- from pycodex.bootstrap import build_agent, build_model, build_runtime
754
-
755
- async def run_session():
756
- runtime = build_runtime(build_agent(build_model()))
757
- await runtime.start()
758
- observer = runtime.attach(lambda event: print(event.kind))
759
- try:
760
- receipt = await runtime.submit_input("/model", sender="application")
761
- result = await receipt.future
762
- return result
763
- finally:
764
- await runtime.close()
765
- runtime.detach(observer)
766
- ```
767
-
768
- Attach immediately supplies a state snapshot; subsequent events synchronize
769
- all views. Detach does not close the backend. Plain input steers at the next
770
- sampling boundary, `/queue` waits for its own turn, and busy Feishu cards still
771
- accept steer or question answers. Unknown slash commands report errors instead
772
- of silently entering model history. See [runtime contracts](docs/RUNTIME.md)
773
- for command receipts, structured answers and session ownership.
774
-
775
- Python observers receive typed `Event` dataclasses from `pycodex.events`, such
776
- as `TurnStartedEvent` and `ToolCompletedEvent`, not generic payload dictionaries.
777
- Custom model clients emit `ModelEvent` variants such as `AssistantDeltaEvent`.
778
- Events provide reusable plain text through `event.visualize()`, or render through
779
- `event.render(display)`. Each CLI view and Feishu card owns an `EventDisplay` with
780
- independent stream/queue state and log/status/prompt callbacks. Events own colors,
781
- flushing, status transitions and prompt text; frontends execute I/O or update their
782
- display fields. Feishu disables colors and shows a bounded recent transcript plus
783
- the live stream, rather than a separate last-answer/last-turn projection.
784
- There is no display-handler mapping. Web retains its own projection and unchanged
785
- event JSON format.
786
-
787
- ### Internal contracts
788
-
789
- - Tools implement async `BaseTool.run`; synchronous implementations are not
790
- implicitly accepted. Tool-specific follow-up messages come from
791
- `BaseTool.follow_up_messages`, not tool-name checks in the Agent loop.
792
- - Completed tool results are committed individually. Explicitly parallel tool
793
- batches wait for all their operations before propagating a commit failure;
794
- they do not cancel remaining operations or invent interruption results.
795
- - Background tools bind their own callbacks through `BaseTool.bind_agent`.
796
- The Agent does not reach into specific tools' private managers.
797
- - Event observers report failures to the asyncio exception handler without
798
- changing the execution result. Provider stream callbacks run on the owning
799
- event loop and stop delivering when their request is finished or cancelled.
800
- - `ModelClient.model` is a required read-only model identifier, also supplied by
801
- test clients. `Agent.model_name` reads it directly, without `getattr` fallback.
802
- `ModelClient.complete` returns a `ModelResponse` containing only supported
803
- output item types. Context overflow is reported as `ContextLengthExceeded`;
804
- provider-specific text classification stays inside the Responses client.
805
- - A terminal `response.incomplete` is not a disconnected stream and is not
806
- retried. For `max_output_tokens`, done assistant/reasoning items are retained
807
- for the next turn, but uncommitted text deltas and unresolved tool calls are not.
808
- - Interactive clients expose the `ModelControl` interface in `pycodex.model`.
809
- Use `agent.set_model(name)` to update provider metadata, context instructions,
810
- context limits, and usage state together.
811
- - Runtime services belong to the tool registry. Pass an explicit environment to
812
- `ToolRegistry(environment)` or `get_tools(environment)` when needed; the Agent
813
- uses `agent.tool_registry.runtime_environment`, not a forwarding Agent property.
814
- The global `get_agent_runtime_environment()` and
815
- separate `Agent(..., runtime_environment=...)` entry points are removed.
816
- - Each sub-agent gets its own context manager and runtime services. The sub-agent
817
- service retains terminal event status and queues one notification per terminal
818
- event. The parent consumes these notifications before its next normal model
819
- request, without requiring `wait_agent`; an idle parent waits for its next turn.
820
- - Every Agent records its session, including direct and child Agents; external
821
- callers no longer construct or inject `SessionRolloutRecorder`. The outer queue
822
- remains optional and no worker starts implicitly. Empty initial history and a
823
- no-op event callback remain valid defaults.
824
-
825
- See `docs/RUNTIME.md` for state ownership and failure semantics.
826
-
827
- ## Alignment Checklist
828
-
829
- See `docs/ALIGNMENT.md` for more detail. This section keeps a high-level
830
- checklist for quick status scanning.
831
-
832
- ### Tool Alignment
833
-
834
- Official upstream tools:
835
-
836
- - [x] `shell` - run shell commands in argv form.
837
- - [x] `shell_command` - run shell scripts in string form.
838
- - [x] `exec_command` - start long-running commands with a session.
839
- - [x] `write_stdin` - write stdin to an existing execution session or poll
840
- output.
841
- - [x] `web_search` - expose provider-native web search capability.
842
- - [x] `update_plan` - update the task plan and maintain step status.
843
- - [x] `request_user_input` - retain the declaration and return the upstream
844
- Default-mode unavailable response.
845
- - [x] `request_permissions` - request extra permissions before continuing.
846
- - [x] `spawn_agent` - create and start a sub-agent.
847
- - [x] `send_input` - continue feeding input to an existing sub-agent.
848
- - [x] `resume_agent` - reopen a closed sub-agent.
849
- - [x] `wait_agent` - wait for a sub-agent to reach a terminal state.
850
- - [x] `close_agent` - close a sub-agent that is no longer needed.
851
- - [x] `apply_patch` - edit files precisely with a freeform patch.
852
- - [x] `grep_files` - search file contents by pattern.
853
- - [x] `read_file` - read file slices while preserving line-number semantics.
854
- - [x] `list_dir` - list directory tree slices.
855
- - [x] `view_image` - turn a local image into model-visible input.
856
-
857
- Upstream low-frequency / special-mode tools not yet modeled separately:
858
-
859
- - [ ] `wait_infinite` - long blocking wait for external events or later input.
860
- - [ ] `spawn_agents_on_csv` - create sub-agent jobs in bulk from CSV.
861
- - [ ] `report_agent_job_result` - report batch agent job results.
862
- - [ ] `js_repl` - JavaScript REPL / code-mode primary entry point.
863
- - [ ] `js_repl_reset` - reset `js_repl` state.
864
- - [ ] `artifacts` - generate or manage structured artifact outputs.
865
- - [ ] `list_mcp_resources` - list MCP resources.
866
- - [ ] `list_mcp_resource_templates` - list MCP resource templates.
867
- - [ ] `read_mcp_resource` - read MCP resource contents.
868
- - [ ] `multi_tool_use.parallel` - parallel wrapper around multiple developer
869
- tool calls.
870
-
871
- Repository-specific compatibility / transition tools:
872
-
873
- - [x] `clock` - pycodex periodic Agent wake-up extension.
874
- - [x] `exec` - current local approximation of code mode.
875
- - [x] `wait` - current local approximation of code-mode waiting behavior.
876
-
877
- ### Behavior Alignment
878
-
879
- - [x] `AgentLoop` / `AgentRuntime` main loop skeleton - turn loop and submission
880
- queue are in place.
881
- - [x] non-interactive `exec` `instructions` alignment - base instructions match
882
- upstream.
883
- - [x] shared `exec` context content - the audited first/resume/tool-follow-up
884
- scenarios match after documented exclusions; raw item identity is not equal.
885
- - [x] developer/contextual-user message shape alignment - message/content shape
886
- matches upstream.
887
- - [x] `AGENTS.md` + `<environment_context>` injection alignment - context
888
- assembly order matches upstream.
889
- - [ ] complete 0.153.4 tool catalog alignment - local `clock`, legacy sub-agent
890
- tools, and upstream goal/deferred/code-mode-v2 surfaces still differ.
891
- - [x] `include = ["reasoning.encrypted_content"]` - reasoning include field is
892
- aligned.
893
- - [x] `prompt_cache_key` - request-level prompt cache key is implemented.
894
- - [x] `x-client-request-id` - request id header is implemented.
895
- - [x] `x-codex-turn-metadata` - turn id / sandbox header is implemented.
896
- - [x] `originator` - mode-aware originator header is implemented.
897
- - [ ] current identity/telemetry header parity - local identity remains
898
- unchanged; the audit records upstream's newer headers and `client_metadata`.
899
- - [x] field-by-field upstream exec-mode tool schema alignment - aligned tools
900
- use class-level specs; `clock` is documented separately as an extension.
901
- - [ ] full interactive-mode and non-`exec` behavior alignment - the non-exec
902
- first-turn context is now on the `codex-tui` path, but continuous REPL
903
- multi-turn behavior is not fully verified yet.
904
- - [ ] sandbox / approvals / compact / memory and other outer behavior alignment
905
- - these systems are still in later scope.