python-codex 0.0.1__tar.gz → 0.1.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (101) hide show
  1. {python_codex-0.0.1 → python_codex-0.1.1}/.github/workflows/publish.yml +2 -2
  2. python_codex-0.1.1/.gitignore +7 -0
  3. python_codex-0.1.1/AGENTS.md +44 -0
  4. python_codex-0.1.1/LICENSE +201 -0
  5. python_codex-0.1.1/PKG-INFO +355 -0
  6. python_codex-0.1.1/README.md +340 -0
  7. python_codex-0.1.1/README_ZH.md +277 -0
  8. python_codex-0.1.1/docs/ALIGNMENT.md +551 -0
  9. python_codex-0.1.1/docs/CONTEXT.md +466 -0
  10. python_codex-0.1.1/docs/responses_server/README.md +89 -0
  11. python_codex-0.1.1/pycodex/__init__.py +142 -0
  12. python_codex-0.1.1/pycodex/agent.py +290 -0
  13. python_codex-0.1.1/pycodex/cli.py +705 -0
  14. python_codex-0.1.1/pycodex/collaboration.py +21 -0
  15. python_codex-0.1.1/pycodex/context.py +580 -0
  16. python_codex-0.1.1/pycodex/doctor.py +360 -0
  17. python_codex-0.1.1/pycodex/model.py +533 -0
  18. python_codex-0.1.1/pycodex/portable.py +390 -0
  19. python_codex-0.1.1/pycodex/portable_server.py +205 -0
  20. python_codex-0.1.1/pycodex/prompts/collaboration_default.md +11 -0
  21. python_codex-0.1.1/pycodex/prompts/collaboration_plan.md +128 -0
  22. python_codex-0.1.1/pycodex/prompts/default_base_instructions.md +275 -0
  23. python_codex-0.1.1/pycodex/prompts/exec_tools.json +411 -0
  24. python_codex-0.1.1/pycodex/prompts/models.json +847 -0
  25. python_codex-0.1.1/pycodex/prompts/permissions/approval_policy/never.md +1 -0
  26. python_codex-0.1.1/pycodex/prompts/permissions/approval_policy/on_failure.md +1 -0
  27. python_codex-0.1.1/pycodex/prompts/permissions/approval_policy/on_request.md +57 -0
  28. python_codex-0.1.1/pycodex/prompts/permissions/approval_policy/on_request_rule_request_permission.md +33 -0
  29. python_codex-0.1.1/pycodex/prompts/permissions/approval_policy/unless_trusted.md +1 -0
  30. python_codex-0.1.1/pycodex/prompts/permissions/sandbox_mode/danger_full_access.md +1 -0
  31. python_codex-0.1.1/pycodex/prompts/permissions/sandbox_mode/read_only.md +1 -0
  32. python_codex-0.1.1/pycodex/prompts/permissions/sandbox_mode/workspace_write.md +1 -0
  33. python_codex-0.1.1/pycodex/prompts/subagent_tools.json +163 -0
  34. python_codex-0.1.1/pycodex/protocol.py +347 -0
  35. python_codex-0.1.1/pycodex/runtime.py +204 -0
  36. python_codex-0.1.1/pycodex/runtime_services.py +409 -0
  37. python_codex-0.1.1/pycodex/tools/__init__.py +58 -0
  38. python_codex-0.1.1/pycodex/tools/agent_tool_schemas.py +70 -0
  39. python_codex-0.1.1/pycodex/tools/apply_patch_tool.py +363 -0
  40. python_codex-0.1.1/pycodex/tools/base_tool.py +168 -0
  41. python_codex-0.1.1/pycodex/tools/close_agent_tool.py +55 -0
  42. python_codex-0.1.1/pycodex/tools/code_mode_manager.py +519 -0
  43. python_codex-0.1.1/pycodex/tools/exec_command_tool.py +96 -0
  44. python_codex-0.1.1/pycodex/tools/exec_runtime.js +161 -0
  45. python_codex-0.1.1/pycodex/tools/exec_tool.py +48 -0
  46. python_codex-0.1.1/pycodex/tools/grep_files_tool.py +150 -0
  47. python_codex-0.1.1/pycodex/tools/list_dir_tool.py +135 -0
  48. python_codex-0.1.1/pycodex/tools/read_file_tool.py +217 -0
  49. python_codex-0.1.1/pycodex/tools/request_permissions_tool.py +95 -0
  50. python_codex-0.1.1/pycodex/tools/request_user_input_tool.py +167 -0
  51. python_codex-0.1.1/pycodex/tools/resume_agent_tool.py +56 -0
  52. python_codex-0.1.1/pycodex/tools/send_input_tool.py +106 -0
  53. python_codex-0.1.1/pycodex/tools/shell_command_tool.py +107 -0
  54. python_codex-0.1.1/pycodex/tools/shell_tool.py +112 -0
  55. python_codex-0.1.1/pycodex/tools/spawn_agent_tool.py +97 -0
  56. python_codex-0.1.1/pycodex/tools/unified_exec_manager.py +380 -0
  57. python_codex-0.1.1/pycodex/tools/update_plan_tool.py +79 -0
  58. python_codex-0.1.1/pycodex/tools/view_image_tool.py +111 -0
  59. python_codex-0.1.1/pycodex/tools/wait_agent_tool.py +75 -0
  60. python_codex-0.1.1/pycodex/tools/wait_tool.py +68 -0
  61. python_codex-0.1.1/pycodex/tools/web_search_tool.py +30 -0
  62. python_codex-0.1.1/pycodex/tools/write_stdin_tool.py +75 -0
  63. python_codex-0.1.1/pycodex/utils/__init__.py +40 -0
  64. python_codex-0.1.1/pycodex/utils/dotenv.py +64 -0
  65. python_codex-0.1.1/pycodex/utils/get_env.py +218 -0
  66. python_codex-0.1.1/pycodex/utils/random_ids.py +19 -0
  67. python_codex-0.1.1/pycodex/utils/visualize.py +978 -0
  68. python_codex-0.1.1/pyproject.toml +41 -0
  69. python_codex-0.1.1/responses_server/__init__.py +17 -0
  70. python_codex-0.1.1/responses_server/__main__.py +5 -0
  71. python_codex-0.1.1/responses_server/app.py +217 -0
  72. python_codex-0.1.1/responses_server/config.py +63 -0
  73. python_codex-0.1.1/responses_server/payload_processors.py +86 -0
  74. python_codex-0.1.1/responses_server/server.py +63 -0
  75. python_codex-0.1.1/responses_server/session_store.py +37 -0
  76. python_codex-0.1.1/responses_server/stream_router.py +784 -0
  77. python_codex-0.1.1/responses_server/tools/__init__.py +4 -0
  78. python_codex-0.1.1/responses_server/tools/custom_adapter.py +235 -0
  79. python_codex-0.1.1/responses_server/tools/web_search.py +263 -0
  80. python_codex-0.1.1/tests/TESTS.md +191 -0
  81. python_codex-0.1.1/tests/__init__.py +2 -0
  82. python_codex-0.1.1/tests/compare_request_user_input_roundtrip.py +679 -0
  83. python_codex-0.1.1/tests/compare_steer_request_bodies.py +633 -0
  84. python_codex-0.1.1/tests/compare_tool_schemas.py +484 -0
  85. python_codex-0.1.1/tests/fake_responses_server.py +389 -0
  86. python_codex-0.1.1/tests/fakes.py +65 -0
  87. python_codex-0.1.1/tests/responses_server/fake_chat_completions_server.py +276 -0
  88. python_codex-0.1.1/tests/responses_server/test_server.py +1387 -0
  89. python_codex-0.1.1/tests/test_agent.py +302 -0
  90. python_codex-0.1.1/tests/test_builtin_tools.py +1028 -0
  91. python_codex-0.1.1/tests/test_cli.py +2616 -0
  92. python_codex-0.1.1/tests/test_context.py +290 -0
  93. python_codex-0.1.1/tests/test_doctor.py +154 -0
  94. python_codex-0.1.1/tests/test_fake_responses_server.py +104 -0
  95. python_codex-0.1.1/tests/test_model.py +737 -0
  96. python_codex-0.1.1/tests/test_portable.py +171 -0
  97. python_codex-0.0.1/.gitignore +0 -7
  98. python_codex-0.0.1/PKG-INFO +0 -30
  99. python_codex-0.0.1/README.md +0 -21
  100. python_codex-0.0.1/pycodex/__init__.py +0 -3
  101. python_codex-0.0.1/pyproject.toml +0 -18
@@ -48,7 +48,7 @@ jobs:
48
48
  id-token: write
49
49
  environment:
50
50
  name: testpypi
51
- url: https://test.pypi.org/p/python-codex
51
+ url: https://test.pypi.org/project/pycodex/
52
52
  steps:
53
53
  - name: Download distributions
54
54
  uses: actions/download-artifact@v4
@@ -69,7 +69,7 @@ jobs:
69
69
  id-token: write
70
70
  environment:
71
71
  name: pypi
72
- url: https://pypi.org/p/python-codex
72
+ url: https://pypi.org/project/pycodex/
73
73
  steps:
74
74
  - name: Download distributions
75
75
  uses: actions/download-artifact@v4
@@ -0,0 +1,7 @@
1
+ .venv/
2
+ .pytest_cache/
3
+ __pycache__/
4
+ *.pyc
5
+ .tmp/
6
+ .pycodex-storage/
7
+ uv.lock
@@ -0,0 +1,44 @@
1
+ # pycodex AGENTS
2
+
3
+ ## Repo Priors
4
+
5
+ - 这个仓库的目标是尽可能准确地复现上游 Codex(`https://github.com/openai/codex`),不是发明一个“类似 Codex”的简化替代品;凡是行为差异都应优先朝原版 Codex 收敛。
6
+ - 发布到 PyPI 时,distribution name 用 `python-codex`;代码包目录、`import` 路径和 CLI 命令仍保持 `pycodex`。
7
+ - 包结构直接放在 repo 根目录下的 `pycodex/`,不使用 `src/` layout。
8
+ - Rust 到 Python 的主映射:`submission_loop` -> `pycodex/runtime.py`,`run_turn` / `run_sampling_request` -> `pycodex/agent.py`,`ToolRouter` -> `pycodex/tools/base_tool.py` + `pycodex/tools/`.
9
+ - 会话协议先收敛到 4 类 item:`UserMessage`、`AssistantMessage`、`ToolCall`、`ToolResult`;只有这层稳定后再扩 richer event model。
10
+ - 优先保持主循环可测试、可替换:模型侧通过 `ModelClient` 协议接入;测试专用的 `ScriptedModelClient` 放在 `tests/fakes.py`,不要放进运行时包。
11
+ - `ResponsesModelClient` 直接复用 `~/.codex/config.toml` 的 provider 配置;当前已验证这里的 responses provider 需要 `stream = true`,否则会返回 `400` 和 `Stream must be set to true`。
12
+ - `responses_server` compat 层应透传请求里的 `model`;不要再做 “取 downstream /models 第一个 id 并强制覆盖请求模型” 这种兜底兼容。
13
+ - 对 `model_provider = "vllm"`,`responses_server` 仍然走 `/v1/chat/completions` compat 路径,但要保留 reasoning:把 chat chunk 里的 `reasoning` / `reasoning_content` 翻回 Responses `reasoning` item,并把历史里的 Responses `reasoning` item 回放成下游 assistant message 的 `reasoning` 字段。
14
+ - `responses_server` 的 provider-specific chat payload 定制统一放在 `responses_server/payload_processors.py`:使用 `CompatServerConfig.model_provider` 选择 `provider_name -> proc_fn(outcomming_request)` 映射,并且只在真正发出 downstream `/v1/chat/completions` 前 post-process;`StreamRouter` 内部继续保留 canonical payload,避免 tool hydration / mock web_search follow-up 被 provider 改写污染。
15
+ - `pycodex` 默认是最小交互 CLI;无 prompt 时进入 REPL,并通过 `AgentRuntime` 跑外层提交循环。当前会显示最小事件流、assistant 流式输出、简单 title/history(`/title`, `/history`),并默认注册一组与原版一一对应的本地工具子集。
16
+ - 交互 CLI 的事件流展示优先表达用户可感知的阶段(例如工具开始/完成、模型回看工具结果),不要直接把内部 `iteration` 计数暴露成主要状态文案;`iterations` 应继续保留在 `TurnResult` 等程序化结果里。
17
+ - prompt/context 相关逻辑统一放在 `pycodex/context.py`:`AgentLoop` 只维护真实会话历史;每轮请求前由 `ContextManager` 注入 base instructions、developer message、`AGENTS.md` 指令和 `<environment_context>`,且这些注入项不写回 history。
18
+ - `AgentLoop` 的 turn-loop 语义要跟上游 `codex-rs/core/src/codex.rs` 一致:按 follow-up / tool handoff 自然收敛,不要加固定 12 轮之类的 hard cap,也不要保留本地专用的 iteration-limit 参数。
19
+ - `README.md` 和 `docs/` 属于对齐工作的一部分:只要实现状态、对齐结论或使用方式发生实质变化,就应及时更新,不要让文档滞后于当前代码。
20
+ - 新工具必须继承 `BaseTool`,然后通过 `ToolRegistry.register(tool_instance)` 接入;不要再给 registry 传散装 name/description/handler 参数。
21
+ - request/tool schema 对齐应落在实际建模层(`ToolSpec` / `BaseTool` / request builder)本身,不要再引入 prompt 级别的 `serialized_tools` 旁路覆盖。
22
+ - 当前已接入的内置工具子集:`shell`、`shell_command`、`exec_command`、`write_stdin`、`exec`、`wait`、`web_search`、`update_plan`、`request_user_input`、`request_permissions`、`spawn_agent`、`send_input`、`resume_agent`、`wait_agent`、`close_agent`、`apply_patch`、`grep_files`、`read_file`、`list_dir`、`view_image`。后续继续新增时,仍然按原版 Codex 内置工具逐个补齐。
23
+ - `exec_command` / `write_stdin` 的 unified-exec 对齐要注意两层默认截断语义:省略 `max_output_tokens` 时也要按 upstream 默认 `10_000` token 预算裁剪 tool response;同时长时间未轮询的未读输出缓冲不能无限累积,需保留 upstream 同款 `1 MiB` head/tail。
24
+ - 当前协议层已经支持两类原版工具载荷:普通 function tools,以及 `apply_patch` 这类 freeform/custom tools;工具结果也支持结构化 `input_image` content items,用于 `view_image` 这类会把图片喂回模型的工具。
25
+ - 当前本地 runtime 已支持一个最小的 in-process sub-agent 管理层:`spawn_agent` / `send_input` / `resume_agent` / `wait_agent` / `close_agent` 通过共享的 `SubAgentManager` 驱动新的 `AgentRuntime` 实例,不依赖 CLI harness 自带的多 agent 基础设施。
26
+ - 当前交互工具 `request_user_input` / `request_permissions` 已接到 `pycodex` 交互 CLI:它们会在一次 turn 内直接复用同一个 stdin/input_fn 向用户提问,因此只有交互模式下有完整体验;非交互调用默认会返回取消/不可用类错误。
27
+ - `request_user_input` 对齐上游时要注意两层:Default mode 默认返回固定错误 `request_user_input is unavailable in Default mode`;Plan mode happy path 则要求每个问题都有非空 `options`、handler 会补 `isOther=true`,并把结构化答案序列化成 JSON 字符串加 `success=true` 塞回 `function_call_output`。
28
+ - 在 `/data/pycodex` 本机已安装的 `codex-cli 0.115.0` 上,用 `tests/compare_request_user_input_roundtrip.py` 做 Plan-mode deterministic proxy live capture 时,upstream 的 second request `function_call_output` 当前不带 `success`;而 GitHub `openai/codex` `main` 源码里的 `FunctionCallOutputPayload` / `request_user_input` handler 支持传递 `success`。写对齐结论时要明确区分“installed CLI live capture”与“upstream main 源码建模”。
29
+ - 当前 `exec` / `wait` 已有一个最小 code-mode 实现:底层通过 Node 子进程运行 JavaScript,自带 `text` / `image` / `store` / `load` / `exit` / `notify` / `yield_control` helper,并允许通过 `tools.<name>(...)` 调回本地已注册工具;`web_search` 当前作为 Responses API provider-native tool declaration 暴露给模型,不经过本地 ToolRegistry 执行。
30
+ - 我们支持的 tools 必须和原版 Codex 内置 tools 一一对应:名称、定位、参数形状、交互模型、输出语义都应尽量对齐;不要混合多个原版工具的语义做一个“折中工具”。
31
+ - 代码风格上不要使用 `*` 定义 keyword-only 参数;接口默认允许位置参数。
32
+ - 本仓库使用 `uv`;本地默认没有预装测试依赖,开始工作前先跑 `uv sync --dev`,验证用 `uv run pytest`。
33
+ - 上游 Codex 的 `steer` 目前是 TUI 交互层能力:开启后 `Enter` 会立即提交,`Tab` 才是排队。对齐时优先盯“下一次发出去的请求体”而不是内部控制流;当前 `pycodex` 已把 steer/queue 语义下沉到 `AgentRuntime`,并让 `AgentLoop.run_turn(texts)` 一次接收一批 user texts,这样下一次请求的 `input` 可以把多个 steer 文本按顺序并到 history 尾部。`CliSessionView.handle_event()` 把 spinner 再次拉起。要保证输入不被遮挡,需要让 view 知道“当前正在输入”,并在 input-active 期间抑制这些事件对 spinner 的 resume。
34
+ - steer 的最小交互反馈已经约定为两条显式状态文案:入队时打印 `[steer] queued: <prompt>`,该条 queued turn 真正开始执行时打印 `[steer] inserted: <prompt>`。后续如果补更复杂的 queue UI,也应保留这两个核心状态语义。
35
+ - 在当前 `pycodex` CLI 里,普通输入与 `/queue <message>` 只负责选择 runtime queue;真正的 steer/queue 差别由 `AgentRuntime.enqueue_user_turn(..., queue=...)` 决定。runtime 内部也应保持成两个同构 queue,而不是一个普通 queue 再叠一个 steer 专用旁路状态机。
36
+ - 对上游 steer 语义要非常谨慎:正常 active-turn steer 首先走的是 `inject_input(...)` + `pending_input`,不是立刻 `spawn_task(...)` / `TurnAbortReason::Replaced`。更准确的理解是“在最近一次 sampling 边界插入”,而不是“任意时刻硬打断当前模型/工具调用”。
37
+ - 用 `tests/fake_responses_server.py` 做 steer 时序对比时,不要把 proxy capture 文件的生成时刻当成“请求已到达 upstream”的信号;`build_proxy_handler(...)` 会等整条 upstream response 读完后才 `write_capture(...)`。如果要在第一条 request 仍未完成时注入 steer,应该同步等待 fake origin 自己收到第 1 条 POST。
38
+ - 在本机做 steer fake-server 对比时,不要把用户本地 `config.toml` 里的 `service_tier` / fast-mode 设置混进“默认 steer”结论。`tests/compare_steer_request_bodies.py` 现在会给 upstream 和 `pycodex` 都生成临时 config,并去掉顶层 `service_tier` 后再比较 request body。
39
+ - `x-codex-turn-metadata.workspaces` 的时机不是“整个 session 只发第一条请求”。当前对齐结论是:首个 turn 的后续 steer/follow-up request 也继续带 `workspaces`;切到后续新 turn 才省略。
40
+ - 远端 Codex home 存储模式当前仍刻意只挂在 `pycodex/cli.py` 启动前:`--put`/`--call` 只负责上传或落本地 `CODEX_HOME` 并重写 `args.config`,`model/context/runtime` 继续完全按 `config_path.parent` 读取 `.env`、`AGENTS.md`、`skills/`;后续扩展时优先保持这个隔离边界,不要把分支判断散到运行时各模块里。
41
+ - 当前存储服务原型契约固定为 `POST /v1/storage/put` 上传客户端本地加密后的 bundle,并返回/使用 `SECRET-CALLID@<host:port>` 这种 call token;`GET /v1/storage/call/<call_id>` 只下载密文,bundle 的解密和解压都在 client 侧完成。做 smoke 或排障时,优先先验证这两个 endpoint 和 call token 形状。
42
+ - `--put` 的 CLI UX 现在约定为“先打印打包文件清单和上传目标,再打印结果”,并且最后一行保留为可直接执行的 `pycodex --call SECRET-CALLID@<host:port>` 一键启动命令;后续如果再改输出,尽量保留这个末行语义。
43
+ - `--put` 现在不是“只上传就结束”:上传成功后还会立刻跑一次真实的 `--call` 下载/解包 round-trip 测试,只有这个测试也成功才算整个 `--put` 成功。排障时如果上传成功但 CLI 仍退出非零,要继续看 call 路径而不只看 put 端。
44
+ - `--put` 现在会先做 `/healthz` preflight,再开始扫描/压缩目录;如果用户报“卡死”,先看目标 `host:port` 是否真有 storage server 在监听。默认打包策略也已经切到白名单:只带 `config.toml`、`.env`、`AGENTS.md`、`AGENTS.override.md`、`skills/**`,以及 config 里相对引用的 `model_instructions_file`,所以像 `sessions/` 这类运行态目录不会再靠黑名单排除。
@@ -0,0 +1,201 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright 2025 OpenAI
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
@@ -0,0 +1,355 @@
1
+ Metadata-Version: 2.4
2
+ Name: python-codex
3
+ Version: 0.1.1
4
+ Summary: A minimal Python extraction of Codex's main agent loop
5
+ License-File: LICENSE
6
+ Requires-Python: >=3.10
7
+ Requires-Dist: cryptography>=3.4
8
+ Requires-Dist: fastapi>=0.115
9
+ Requires-Dist: loguru>=0.7.3
10
+ Requires-Dist: prompt-toolkit>=3.0
11
+ Requires-Dist: requests>=2.31
12
+ Requires-Dist: tomli>=2.0; python_version < '3.11'
13
+ Requires-Dist: uvicorn>=0.32
14
+ Description-Content-Type: text/markdown
15
+
16
+ # pycodex
17
+
18
+ English README. Chinese version: `README_ZH.md`
19
+
20
+ PyPI distribution name: `python-codex`
21
+ Import path and CLI command remain `pycodex`.
22
+
23
+ This repository extracts the core Codex agent loop from upstream Codex
24
+ (`https://github.com/openai/codex`) into a deliberately small Python version,
25
+ while preserving the two most important layers:
26
+
27
+ - `submission_loop`: sequentially consumes submitted operations.
28
+ - `run_turn`: keeps executing `model sample -> tool call -> feed tool result
29
+ back into the model` inside a single turn until a final answer is reached.
30
+
31
+ Relevant Rust reference points:
32
+
33
+ - `codex-rs/core/src/codex.rs` -> `submission_loop`
34
+ - `codex-rs/core/src/codex.rs` -> `run_turn`
35
+ - `codex-rs/core/src/codex.rs` -> `run_sampling_request`
36
+ - `codex-rs/core/src/tools/router.rs` -> `ToolRouter`
37
+ - `codex-rs/core/src/stream_events_utils.rs` -> `handle_output_item_done`
38
+
39
+ ## Quick Start
40
+
41
+ Install dependencies first:
42
+
43
+ ```bash
44
+ uv sync
45
+ ```
46
+
47
+ Try the real entry points:
48
+
49
+ ```bash
50
+ uv run pycodex "Reply with exactly OK."
51
+ uv run pycodex
52
+ ```
53
+
54
+ ## Design Tradeoffs
55
+
56
+ This is not a 1:1 port of the Rust implementation. The current goal is a
57
+ minimal reusable kernel that converges on the upstream behavior over time:
58
+
59
+ 1. Use a thin `ModelClient` protocol to abstract the model side.
60
+ 2. Use `ToolRegistry` to manage tool specs and executors.
61
+ 3. Use `AgentLoop` to implement the core closed loop.
62
+ 4. Use `AgentRuntime` to preserve the outer submission queue so it can keep
63
+ converging toward Rust's `submission_loop` later.
64
+
65
+ Intentionally not included yet:
66
+
67
+ - TUI / streaming incremental rendering
68
+ - MCP / connectors / sandbox / approvals
69
+ - memory / compact / hooks / review mode
70
+ - a full production OpenAI adapter surface
71
+
72
+ All of those can be layered on later. For now, the project is focused on
73
+ nailing the core tool-augmented reasoning loop first.
74
+
75
+ ## Layout
76
+
77
+ - `pycodex/protocol.py`: minimal conversation item / prompt / event protocol
78
+ - `pycodex/model.py`: model client protocol and Responses API adapter
79
+ - `pycodex/cli.py`: single-turn and interactive `pycodex` CLI entry points
80
+ - `pycodex/tools/base_tool.py`: `BaseTool`, `ToolRegistry`, `ToolContext`
81
+ - `pycodex/tools/`: concrete tool implementations
82
+ - `pycodex/agent.py`: inner turn loop
83
+ - `pycodex/runtime.py`: outer submission queue
84
+ - `tests/test_agent.py`: core behavior tests
85
+
86
+ ## Current Alignment Status
87
+
88
+ Current progress is easiest to read in layers:
89
+
90
+ - prompt/context alignment:
91
+ - on the non-interactive `exec` path, `instructions` and `input` already
92
+ match upstream Codex;
93
+ - this layer is now mainly handled by `pycodex/context.py` plus vendored
94
+ prompt data.
95
+ - turn-loop semantic alignment:
96
+ - `AgentLoop` no longer uses a fixed 12-iteration cap by default;
97
+ - like upstream, it now converges naturally based on whether there is still
98
+ follow-up work or tool handoff to do;
99
+ - the local iteration-limit parameter is gone.
100
+ - request-level alignment:
101
+ - the non-interactive `exec` request body is mostly aligned;
102
+ - the default CLI non-exec first request now also follows the upstream
103
+ `codex-tui` + `<collaboration_mode>` path;
104
+ - the default CLI two-turn main-thread request/header behavior has also been
105
+ captured and aligned, including omitting `workspaces` on later turns;
106
+ - the remaining work is now more about outer behavior branches than this
107
+ already-compared request/header path.
108
+ - tool round-trip alignment:
109
+ - the Default-mode unavailable path for `request_user_input` is aligned to
110
+ real upstream captures;
111
+ - the Plan-mode happy path is also aligned at the tool/protocol layer based
112
+ on upstream source: it forces `isOther=true`, requires non-empty `options`,
113
+ and returns structured answers as a JSON string plus `success=true`;
114
+ - there is now a deterministic round-trip comparison helper,
115
+ `tests/compare_request_user_input_roundtrip.py`, built on the proxy mode in
116
+ `tests/fake_responses_server.py`; against the locally installed
117
+ `codex-cli 0.115.0`, the only remaining Plan-mode live-capture schema
118
+ difference is that `pycodex` includes `success=true` in
119
+ `function_call_output`.
120
+
121
+ See `docs/ALIGNMENT.md` for more detailed notes.
122
+
123
+ ## Live Model Integration
124
+
125
+ If this machine already has a Codex CLI configuration, `pycodex` can reuse the
126
+ `model`, `model_provider`, `base_url`, and `env_key` from
127
+ `~/.codex/config.toml` directly:
128
+
129
+ ```python
130
+ from pycodex import ResponsesModelClient
131
+
132
+ client = ResponsesModelClient.from_codex_config()
133
+ ```
134
+
135
+ The current implementation uses the streaming OpenAI-compatible `/responses`
136
+ endpoint. This path has already been validated against the local
137
+ `~/.codex/config.toml` setup.
138
+
139
+ When launched through the CLI, `pycodex` also loads `.env` from the same
140
+ configuration directory before reading config (typically `~/.codex/.env`), so
141
+ provider keys and similar environment variables can live there. To match
142
+ upstream Codex, variables starting with `CODEX_` are not imported from `.env`.
143
+
144
+ ## pycodex CLI
145
+
146
+ `pycodex` now defaults to a minimal interactive entry point. Internally it uses
147
+ `AgentRuntime` to drive the turn submission loop and reuses
148
+ `~/.codex/config.toml` by default:
149
+
150
+ ```bash
151
+ pycodex
152
+ pycodex "Summarize this repo in one sentence."
153
+ printf 'Reply with exactly OK.' | pycodex
154
+ pycodex --json "Reply with exactly OK."
155
+ pycodex --profile model_proxy "Reply with exactly OK."
156
+ pycodex --vllm-endpoint http://127.0.0.1:18000 "Reply with exactly OK."
157
+ pycodex --put @127.0.0.1:5577
158
+ pycodex --put /data/.codex/@127.0.0.1:5577
159
+ pycodex --call SECRET-CALLID@127.0.0.1:5577 "Reply with exactly OK."
160
+ pycodex doctor
161
+ ```
162
+
163
+ Current behavior:
164
+
165
+ - with no argv prompt and a TTY stdin, enter interactive mode
166
+ - with an argv prompt or piped stdin, run a single turn
167
+ - interactive mode supports `/exit` and `/quit`
168
+ - interactive mode shows a compact event stream for user-visible phases such as
169
+ tool execution and model follow-up after tool results
170
+ - assistant text is printed from streaming deltas directly
171
+ - interactive mode supports `/history`, `/title`, and `/model`
172
+ - `/model <name>` switches the model used by later turns in the current
173
+ interactive session; `/model` shows the current model and available choices
174
+ - steer is enabled by default in interactive mode: normal input goes into the
175
+ runtime steer path, the current request stops at the next safe boundary, and
176
+ later steer text is appended to the next model request's `input` in order;
177
+ for explicit queueing, use `/queue <message>`, which prints
178
+ `[steer] queued: ...` and later `[steer] inserted: ...`
179
+ - the default built-in tool subset currently exposed as local tools is:
180
+ `shell`, `shell_command`, `exec_command`, `write_stdin`, `exec`, `wait`,
181
+ `web_search`, `update_plan`, `request_user_input`, `request_permissions`,
182
+ `spawn_agent`, `send_input`, `resume_agent`, `wait_agent`, `close_agent`,
183
+ `apply_patch`, `grep_files`, `read_file`, `list_dir`, `view_image`
184
+ - `--vllm-endpoint http://host:port` automatically launches a local
185
+ `responses_server` compatibility layer; when the URL path is empty it is
186
+ normalized to `/v1`, and `/responses` requests are still forwarded to the
187
+ downstream `/v1/chat/completions` endpoint. For `model_provider = "vllm"`,
188
+ reasoning is now preserved across this path: chat chunks with `reasoning` or
189
+ `reasoning_content` are translated back into Responses `reasoning` items, and
190
+ historical `reasoning` items are replayed into downstream assistant messages
191
+ via the `reasoning` field. Streaming token usage is also requested from vLLM
192
+ and forwarded to the final `response.completed.response.usage`
193
+ - `pycodex doctor` checks config, `.env`, API keys, DNS, TCP/TLS, and an
194
+ optional live Responses API request
195
+
196
+ Current primary uses:
197
+
198
+ - verify provider / model / auth configuration
199
+ - debug `ResponsesModelClient`
200
+ - run minimal single-turn and multi-turn smoke tests
201
+
202
+ `doctor` examples:
203
+
204
+ ```bash
205
+ pycodex doctor
206
+ pycodex doctor --skip-live
207
+ pycodex doctor --json
208
+ ```
209
+
210
+ ## Portable Mode
211
+
212
+ `Portable Mode` is the quickest way to bring your usual `pycodex` setup into a
213
+ fresh machine, container, or debug image.
214
+
215
+ Use it like this:
216
+
217
+ ```bash
218
+ pycodex --put @127.0.0.1:5577
219
+ pycodex --put /data/.codex/@127.0.0.1:5577
220
+ ```
221
+
222
+ - `--put` prints a reusable `SECRET-CALLID@host:port` plus a final one-line
223
+ `pycodex --call ...` command
224
+ - on the new environment or image, run that printed `--call` command directly
225
+ - quickly restoring your usual `config.toml`, `.env`, `AGENTS.md`, and
226
+ `skills/` into a clean debug environment
227
+ - keeping a new image focused on the bug you are debugging instead of spending
228
+ time rebuilding local Codex setup by hand
229
+ - bootstrapping `pycodex` even when the target environment does not already
230
+ have a populated `~/.codex`
231
+ - bare `--put` uses the current user's `~/.codex`
232
+ - `--put /path/.codex/@host:port` lets you publish a different Codex home
233
+
234
+ ## Example
235
+
236
+ ```python
237
+ import asyncio
238
+
239
+ from pycodex import (
240
+ AgentLoop,
241
+ BaseTool,
242
+ ContextManager,
243
+ ResponsesModelClient,
244
+ ToolRegistry,
245
+ )
246
+
247
+
248
+ class EchoTool(BaseTool):
249
+ name = "echo"
250
+ description = "Echo the provided text."
251
+ input_schema = {
252
+ "type": "object",
253
+ "properties": {"text": {"type": "string"}},
254
+ "required": ["text"],
255
+ }
256
+
257
+ async def run(self, context, args):
258
+ del context
259
+ return args["text"]
260
+
261
+
262
+ async def main() -> None:
263
+ model = ResponsesModelClient.from_codex_config()
264
+ context_manager = ContextManager.from_codex_config()
265
+
266
+ tools = ToolRegistry()
267
+ tools.register(EchoTool())
268
+
269
+ agent = AgentLoop(model, tools, context_manager)
270
+ result = await agent.run_turn(
271
+ ["Call the echo tool with text=hello, then tell me what it returned."]
272
+ )
273
+ print(result.output_text)
274
+
275
+
276
+ asyncio.run(main())
277
+ ```
278
+
279
+ ## Alignment Checklist
280
+
281
+ See `docs/ALIGNMENT.md` for more detail. This section keeps a high-level
282
+ checklist for quick status scanning.
283
+
284
+ ### Tool Alignment
285
+
286
+ Official upstream tools:
287
+
288
+ - [x] `shell` - run shell commands in argv form.
289
+ - [x] `shell_command` - run shell scripts in string form.
290
+ - [x] `exec_command` - start long-running commands with a session.
291
+ - [x] `write_stdin` - write stdin to an existing execution session or poll
292
+ output.
293
+ - [x] `web_search` - expose provider-native web search capability.
294
+ - [x] `update_plan` - update the task plan and maintain step status.
295
+ - [x] `request_user_input` - ask the user structured questions and wait for an
296
+ answer.
297
+ - [x] `request_permissions` - request extra permissions before continuing.
298
+ - [x] `spawn_agent` - create and start a sub-agent.
299
+ - [x] `send_input` - continue feeding input to an existing sub-agent.
300
+ - [x] `resume_agent` - reopen a closed sub-agent.
301
+ - [x] `wait_agent` - wait for a sub-agent to reach a terminal state.
302
+ - [x] `close_agent` - close a sub-agent that is no longer needed.
303
+ - [x] `apply_patch` - edit files precisely with a freeform patch.
304
+ - [x] `grep_files` - search file contents by pattern.
305
+ - [x] `read_file` - read file slices while preserving line-number semantics.
306
+ - [x] `list_dir` - list directory tree slices.
307
+ - [x] `view_image` - turn a local image into model-visible input.
308
+
309
+ Upstream low-frequency / special-mode tools not yet modeled separately:
310
+
311
+ - [ ] `wait_infinite` - long blocking wait for external events or later input.
312
+ - [ ] `spawn_agents_on_csv` - create sub-agent jobs in bulk from CSV.
313
+ - [ ] `report_agent_job_result` - report batch agent job results.
314
+ - [ ] `js_repl` - JavaScript REPL / code-mode primary entry point.
315
+ - [ ] `js_repl_reset` - reset `js_repl` state.
316
+ - [ ] `artifacts` - generate or manage structured artifact outputs.
317
+ - [ ] `list_mcp_resources` - list MCP resources.
318
+ - [ ] `list_mcp_resource_templates` - list MCP resource templates.
319
+ - [ ] `read_mcp_resource` - read MCP resource contents.
320
+ - [ ] `multi_tool_use.parallel` - parallel wrapper around multiple developer
321
+ tool calls.
322
+
323
+ Repository-specific compatibility / transition tools:
324
+
325
+ - [x] `exec` - current local approximation of code mode.
326
+ - [x] `wait` - current local approximation of code-mode waiting behavior.
327
+
328
+ ### Behavior Alignment
329
+
330
+ - [x] `AgentLoop` / `AgentRuntime` main loop skeleton - turn loop and submission
331
+ queue are in place.
332
+ - [x] non-interactive `exec` `instructions` alignment - base instructions match
333
+ upstream.
334
+ - [x] non-interactive `exec` `input` alignment - prompt input matches upstream.
335
+ - [x] developer/contextual-user message shape alignment - message/content shape
336
+ matches upstream.
337
+ - [x] `AGENTS.md` + `<environment_context>` injection alignment - context
338
+ assembly order matches upstream.
339
+ - [x] non-interactive `exec` tool subset alignment - the model-visible tool set
340
+ has converged.
341
+ - [x] `include = ["reasoning.encrypted_content"]` - reasoning include field is
342
+ aligned.
343
+ - [x] `prompt_cache_key` - request-level prompt cache key is implemented.
344
+ - [x] `x-client-request-id` - request id header is implemented.
345
+ - [x] `x-codex-turn-metadata` - turn id / sandbox header is implemented.
346
+ - [x] `originator` - mode-aware originator header is implemented.
347
+ - [x] exact `user-agent` string alignment - aligned on the non-interactive
348
+ `exec` path.
349
+ - [x] field-by-field exec-mode tool schema alignment - currently reuses the
350
+ upstream snapshot directly through the tool layer.
351
+ - [ ] full interactive-mode and non-`exec` behavior alignment - the non-exec
352
+ first-turn context is now on the `codex-tui` path, but continuous REPL
353
+ multi-turn behavior is not fully verified yet.
354
+ - [ ] sandbox / approvals / compact / memory and other outer behavior alignment
355
+ - these systems are still in later scope.