@criogaid/pi-codex-compaction 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -0
- package/README.md +26 -31
- package/package.json +7 -5
- package/src/capability.ts +16 -17
- package/src/checkpoint.ts +2 -3
- package/src/fallback-settings.ts +59 -25
- package/src/index.ts +43 -38
- package/src/protocol.ts +29 -1
- package/src/remote-retry.ts +277 -0
- package/src/remote.ts +145 -72
- package/src/request-snapshot.ts +109 -36
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,28 @@
|
|
|
2
2
|
|
|
3
3
|
## 未发布
|
|
4
4
|
|
|
5
|
+
## 0.5.0
|
|
6
|
+
|
|
7
|
+
- 检查点的普通重放和再次压缩遵守当前图片读取设置,关闭时过滤保留的原始图片。
|
|
8
|
+
- 保持 V2 的工具声明与历史加载记录;符合快照条件时复用普通请求的完整序列化前缀,包括中途隐藏工具的投影和 provider 钩子改写的历史,再追加新增内容,避免重新生成旧前缀破坏缓存。
|
|
9
|
+
- 修复本轮临时提示词在对话结束后使请求快照失效的问题;真正的提示词、工具、思考等级或界面以外的配置变化仍会阻止复用。
|
|
10
|
+
- 切换界面主题后直接压缩仍复用普通请求的工具声明和历史前缀,避免主题变化导致隐藏工具声明重新出现。
|
|
11
|
+
- 手动压缩或下一轮开始前的自动压缩不再因 Pi 在本轮结束后回到基础系统提示词而丢弃请求快照;扩展按轮调整系统提示词时,压缩仍复用普通请求的完整前缀和缓存参数。其他输入变化仍会使快照失效。
|
|
12
|
+
- 切换思考块显示、终端图片、编辑器边距、全屏和 Markdown 渲染等只影响 Pi 界面的设置后,压缩继续复用请求快照。
|
|
13
|
+
- 补齐 `parallel_tool_calls`、完整 `text` 输出设置和 `prompt_cache_options` 缓存模式/TTL 的复用,不复制预热操作或诊断响应 ID。服务端会话引用不被照搬,相关请求保留本地重建的完整历史。补充重复检查点、隐藏上下文保留和重试请求一致性的回归测试。
|
|
14
|
+
- 兼容 Codex provider 的成功 `response.done` 事件,继续拒绝失败和未完成的响应。
|
|
15
|
+
- 使用跨进程文件锁保护配置版本检查和原子替换,防止多个 Pi 进程保存同一旧版本时静默覆盖。
|
|
16
|
+
- 补充上述路径的真实 Pi 会话、provider 和跨进程回归测试,以及与 Codex V2 源码的对照记录。
|
|
17
|
+
- V2 重试覆盖整次 SSE 请求与响应收集,额外次数取 Pi provider 设置与 2 的较小值,关闭 provider 内层重试,避免次数相乘。只凭暂时 HTTP、网络或流错误的明确证据重试;永久错误、协议校验失败和取消不会重试。每次尝试保持请求内容和后端不变,重新认证并校验会话归属。
|
|
18
|
+
- V2 压缩前按 Pi 实际准备请求的模型窗口裁剪末尾工具输出;保留原生文字回退、普通请求和模型切换行为,HTTP 重定向沿用 provider 与 fetch。
|
|
19
|
+
- 压缩期间切换模型后仍保存原模型生成的结果;新检查点无法在当前后端重放时,在安装后按检查点提示一次。配置文字回退同样不因切换对话模型而取消,会话变化或主动取消仍丢弃结果。
|
|
20
|
+
- 补充整次重试及兼容性回归测试,覆盖已有检查点的文字回退、跨后端继续请求、普通模型路由和被动模型切换。
|
|
21
|
+
- 开发与测试依赖更新至 Pi 1.0.4 并通过完整测试;Pi 最低支持版本仍为 0.99.1。
|
|
22
|
+
|
|
23
|
+
## 0.4.1
|
|
24
|
+
|
|
25
|
+
- V2 模型识别改为显式 `compat.remoteCompaction` 配置优先;未配置时,`openai-responses` 或 `openai-codex-responses` 模型的 ID 包含 `gpt`(不区分大小写)即尝试 V2,不再限制服务商或官方地址;其他 API 的 GPT 模型未配置时直接使用 Pi 原生压缩,不显示失败警告。无效配置不会退回名称匹配。检查点支持保存和恢复自定义 API 标识,继续校验后端身份。
|
|
26
|
+
|
|
5
27
|
## 0.4.0
|
|
6
28
|
|
|
7
29
|
- 更新压缩历史的恢复提示,明确使用当前包 `@criogaid/pi-codex-compaction`。保留旧会话摘要的精确识别与消息编辑保护,无需迁移会话文件。
|
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
让 [Pi](https://github.com/earendil-works/pi) 在 Codex 模型上使用 Codex Remote Compaction V2 压缩聊天记录,失败时自动退回 Pi 原生压缩,并且可以为原生压缩单独指定一个摘要模型。
|
|
6
6
|
|
|
7
7
|
- **装上即用:** 对 Codex 模型默认启用 V2 远程压缩,无需任何配置。
|
|
8
|
-
- **失败自动退回:** 当前模型不支持 V2,或 V2 请求失败时,改用 Pi
|
|
8
|
+
- **失败自动退回:** 当前模型不支持 V2,或 V2 请求失败时,改用 Pi 原生的文字摘要压缩。
|
|
9
9
|
- **独立摘要模型(可选):** 退回 Pi 原生压缩时,可以使用你指定的模型和思考等级生成摘要。它只接管这一步,不影响 V2 远程压缩,也不改变当前对话模型。
|
|
10
10
|
|
|
11
11
|
[安装](#安装) · [工作方式](#工作方式) · [独立摘要模型](#独立摘要模型) · [配置](#配置) · [自定义网关](#自定义网关) · [注意事项](#注意事项)
|
|
@@ -31,26 +31,7 @@ pi install git:github.com/Criogaid/pi-codex-compaction
|
|
|
31
31
|
|
|
32
32
|
## 工作方式
|
|
33
33
|
|
|
34
|
-
|
|
35
|
-
触发压缩(/compact、自动压缩、超出上下文)
|
|
36
|
-
|
|
|
37
|
-
v
|
|
38
|
-
V2 开启,且当前模型支持 V2? -- 否 ------------------+
|
|
39
|
-
| 是 |
|
|
40
|
-
v |
|
|
41
|
-
用当前对话模型请求 V2 远程压缩 |
|
|
42
|
-
| |
|
|
43
|
-
v |
|
|
44
|
-
成功? -- 是 --> 完成 |
|
|
45
|
-
| 否 |
|
|
46
|
-
v |
|
|
47
|
-
Pi 原生压缩(文字摘要) <----------------------------+
|
|
48
|
-
|
|
|
49
|
-
+-- 已启用独立摘要模型 --> 用指定模型和思考等级生成摘要
|
|
50
|
-
| 失败则停止本次压缩
|
|
51
|
-
|
|
|
52
|
-
+-- 未启用 -------------> 用当前对话模型生成摘要
|
|
53
|
-
```
|
|
34
|
+
手动、自动或上下文溢出触发压缩时,扩展先按设置尝试 V2。V2 关闭、不支持或请求失败时,按配置使用 Pi 原生文字摘要。
|
|
54
35
|
|
|
55
36
|
| 情况 | 压缩方式 | 使用的模型 |
|
|
56
37
|
| --- | --- | --- |
|
|
@@ -60,8 +41,16 @@ Pi 原生压缩(文字摘要) <----------------------------+
|
|
|
60
41
|
|
|
61
42
|
压缩完成后继续原来的对话,当前对话模型和思考等级保持不变。V2 失败、改用独立摘要模型时,Pi 会显示提示及实际使用的模型。
|
|
62
43
|
|
|
44
|
+
已有 V2 检查点也允许文字摘要回退。Pi 的文字摘要只能使用可读取的摘要和近期消息,无法解读检查点的加密历史;回退成功后,新的文字摘要成为后续对话上下文。
|
|
45
|
+
|
|
63
46
|
`/compact` 后附加的要求(例如“重点保留尚未完成的任务”)只对 Pi 原生压缩生效。V2 不接受这些要求,扩展会提示它们被忽略。
|
|
64
47
|
|
|
48
|
+
### 请求失败与重试
|
|
49
|
+
|
|
50
|
+
V2 对一次完整的 SSE 请求及其响应收集进行有限重试,覆盖可确认的暂时 HTTP 错误、网络错误和有响应进展后提前结束的流。额外重试次数取 Pi 的 provider 重试设置与 2 的较小值;未设置时最多额外重试 2 次,总共最多 3 次请求,设置为 0 时不重试。每次重试使用同一份已准备的请求内容和同一后端,并重新通过 Pi 认证。
|
|
51
|
+
|
|
52
|
+
无效压缩结果、重复输出、权限或额度等永久错误、取消操作,以及无法确认属于暂时故障的错误,不会重试。服务端要求等待的时间超过允许上限时也会停止重试。重试耗尽后按上表转为 Pi 文字摘要。HTTP 重定向沿用 provider 和 fetch 的处理方式。
|
|
53
|
+
|
|
65
54
|
## 独立摘要模型
|
|
66
55
|
|
|
67
56
|
假设你平时用 `gpt-6-astra` 聊天,希望退回原生压缩时改用 `gpt-6.1-sol` 生成摘要:
|
|
@@ -135,15 +124,15 @@ Pi 原生压缩(文字摘要) <----------------------------+
|
|
|
135
124
|
- 文件无法读取、JSON 格式错误或 V2 设置无效时,停止压缩。
|
|
136
125
|
- `fallback` 部分只在需要原生压缩时校验,因此它填错不会影响成功的 V2 压缩。
|
|
137
126
|
- `fallback` 无效时菜单显示 `Invalid`。你仍可切换 V2,已填写的内容会原样保留;重新选择模型即可修复,之后需要打开 `Use separate summary model` 才会使用它。
|
|
138
|
-
-
|
|
127
|
+
- 多个 Pi 进程保存同一个配置时会互斥;同一旧版本只能有一个修改成功,其他菜单须重新打开。直接编辑文件时仍会在保存前检查版本;外部编辑器不参与 Pi 的保存锁。
|
|
139
128
|
|
|
140
129
|
</details>
|
|
141
130
|
|
|
142
131
|
## 自定义网关
|
|
143
132
|
|
|
144
|
-
|
|
133
|
+
模型是否尝试 V2 由 `compat.remoteCompaction` 决定:有配置时按配置判断;没有配置时,API 类型为 `openai-responses` 或 `openai-codex-responses` 且模型 ID 包含 `gpt`(不区分大小写)即尝试 V2。其他 API 的请求格式无法携带 V2 压缩,未配置时直接使用 Pi 原生压缩。服务商名称和是否使用官方地址不参与启用判断。显示名称不参与匹配。
|
|
145
134
|
|
|
146
|
-
|
|
135
|
+
显式配置须指定 `protocol: "v2"`,可同时指定 `endpoint`。协议或端点配置无效时不尝试 V2,也不会退回名称匹配。非 GPT 模型或需要覆盖端点时,可以在 Pi 的 `models.json` 中添加:
|
|
147
136
|
|
|
148
137
|
```json
|
|
149
138
|
{
|
|
@@ -154,7 +143,7 @@ Pi 原生压缩(文字摘要) <----------------------------+
|
|
|
154
143
|
"apiKey": "$CUSTOM_CODEX_API_KEY",
|
|
155
144
|
"models": [
|
|
156
145
|
{
|
|
157
|
-
"id": "
|
|
146
|
+
"id": "custom-model",
|
|
158
147
|
"compat": {
|
|
159
148
|
"remoteCompaction": {
|
|
160
149
|
"protocol": "v2"
|
|
@@ -171,13 +160,19 @@ Pi 原生压缩(文字摘要) <----------------------------+
|
|
|
171
160
|
|
|
172
161
|
网关的压缩请求使用其他路径时,可以在 `remoteCompaction` 内增加 `endpoint`。它的协议、主机和端口须与 Pi 认证后实际使用的 `baseUrl` 一致,且不能包含用户名、密码、查询参数(`?`)或片段(`#`)。
|
|
173
162
|
|
|
163
|
+
V2 总开关仍决定是否发送新的远程压缩请求。模型名匹配只决定是否尝试;Pi 的适配器和网关仍须实际支持 V2,请求失败时退回 Pi 原生压缩。默认端点按 API 推导:`openai-codex-responses` 使用 Codex 路径,其他 API 使用 `/responses`。检查点继续绑定实际 provider、API 和端点,防止跨后端重放。
|
|
164
|
+
|
|
174
165
|
## 注意事项
|
|
175
166
|
|
|
176
|
-
- **V2 压缩结果绑定服务商。** V2
|
|
177
|
-
- **同一服务内切换模型。**
|
|
178
|
-
- **关闭 V2
|
|
179
|
-
-
|
|
180
|
-
-
|
|
167
|
+
- **V2 压缩结果绑定服务商。** V2 压缩后的旧记录以加密数据保存,扩展无法还原成文字。重新打开会话或创建分支时可以继续使用,但须保持服务商、API 类型和请求地址不变。换到其他后端后,扩展跳过加密历史,继续使用 Pi 可读取的摘要和近期消息。
|
|
168
|
+
- **同一服务内切换模型。** 扩展继续重放检查点,服务端决定新模型是否接受。模型切换本身不会触发压缩;后续 V2 使用当前对话模型,窗口容量由 Pi 和 provider 处理。普通请求的模型路由沿用 Pi 及其他扩展的处理结果。压缩进行中切换模型不会取消压缩,结果仍由开始时的模型生成并保存,与 Pi 原生压缩一致;如果新模型属于其他后端,扩展会提示新结果无法在当前后端重放。
|
|
169
|
+
- **关闭 V2 不影响已有记录的重放。** 已压缩记录仍可在原后端照常发送;下一次压缩按配置使用文字摘要。
|
|
170
|
+
- **关闭图片读取。** Pi 的 `images.blockImages` 设置也适用于检查点中保留的原始图片;普通重放和再次压缩都会用文字占位替换这些图片。已有保存记录不被修改,但关闭期间生成的新检查点会保留本次发送的占位符,之后重新开启读取不保证能恢复更早的原始图片。该设置不能删除已经包含在加密压缩结果中的图像信息。
|
|
171
|
+
- **加载顺序。** 压缩后 Pi 会保留一部分近期消息。如果它们被其他扩展改写,本扩展可能无法确认其与已压缩记录对应,便跳过加密历史并继续普通请求。搭配会改写聊天内容的扩展时,请把本包放在 Pi 的 `packages` 列表靠前的位置。
|
|
172
|
+
- **工具声明。** V2 保留工具声明、历史中的工具加载记录及实际调用与结果。符合快照条件时复用最近一次普通请求的声明;没有可用快照时,由 Pi 序列化会话中的工具元数据。Pi 未公开工具隐藏状态的修订号;只改变隐藏状态而其他公开设置未变时,旧快照仍可能被复用。
|
|
173
|
+
- **缓存复用。** 会话、模型、后端、历史前缀、工具和设置匹配时,压缩请求复用最近一次普通请求中观察到的完整前缀,包括中途工具加载和历史改写,再追加新增内容。本轮结束后 Pi 回到基础提示词、本轮临时提示词结束,或只修改主题、思考块显示、终端和编辑器外观等界面设置,都不会使匹配的快照失效;真正的提示词或其他配置变化仍会失效。系统指令、思考、服务等级、并行工具调用、输出格式、verbosity、缓存键及缓存模式/TTL 随快照复用,诊断响应 ID 和预热操作不被复制。普通请求依赖服务端会话引用时,仅复用独立参数,历史由 Pi 从本地上下文重建。记录只覆盖本扩展收到的请求内容,之后执行的扩展仍可能改写它;服务端自行决定是否使用缓存,因此不保证命中或费用下降。
|
|
174
|
+
|
|
175
|
+
与 Codex 客户端的源码对照、已还原部分和差异见 [V2 实现对照](docs/verification/codex-v2-parity.md)。
|
|
181
176
|
|
|
182
177
|
## 开发
|
|
183
178
|
|
|
@@ -197,7 +192,7 @@ npm run pack:check
|
|
|
197
192
|
<details>
|
|
198
193
|
<summary><strong>发布流程</strong></summary>
|
|
199
194
|
|
|
200
|
-
[Publish](.github/workflows/publish.yml) 在推送 `v*` 标签时运行。标签须与 `package.json` 的版本一致,例如版本 `0.
|
|
195
|
+
[Publish](.github/workflows/publish.yml) 在推送 `v*` 标签时运行。标签须与 `package.json` 的版本一致,例如版本 `0.5.0` 对应标签 `v0.5.0`。目前只发布正式版本,不接受 `-beta`、`-rc` 等预发布后缀。
|
|
201
196
|
|
|
202
197
|
首次发布前,在仓库的 **Settings → Secrets and variables → Actions** 中添加 `NPM_TOKEN`。按 [npm 文档](https://docs.npmjs.com/creating-and-viewing-access-tokens)创建 granular access token,授予 `@criogaid` 作用域的发布权限(Read and write / publish and stage),并启用 Bypass two-factor authentication。包创建后,可以将 token 权限缩小到该包。
|
|
203
198
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@criogaid/pi-codex-compaction",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "Provider-aware Codex Remote Compaction V2 for Pi.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|
|
@@ -41,14 +41,16 @@
|
|
|
41
41
|
"@earendil-works/pi-tui": ">=0.99.1"
|
|
42
42
|
},
|
|
43
43
|
"dependencies": {
|
|
44
|
+
"proper-lockfile": "4.1.2",
|
|
44
45
|
"saxes": "6.0.0"
|
|
45
46
|
},
|
|
46
47
|
"devDependencies": {
|
|
47
|
-
"@earendil-works/pi-agent-core": "0.
|
|
48
|
-
"@earendil-works/pi-ai": "0.
|
|
49
|
-
"@earendil-works/pi-coding-agent": "0.
|
|
50
|
-
"@earendil-works/pi-tui": "0.
|
|
48
|
+
"@earendil-works/pi-agent-core": "1.0.4",
|
|
49
|
+
"@earendil-works/pi-ai": "1.0.4",
|
|
50
|
+
"@earendil-works/pi-coding-agent": "1.0.4",
|
|
51
|
+
"@earendil-works/pi-tui": "1.0.4",
|
|
51
52
|
"@types/node": "26.1.1",
|
|
53
|
+
"@types/proper-lockfile": "4.1.4",
|
|
52
54
|
"typescript": "7.0.2"
|
|
53
55
|
},
|
|
54
56
|
"pi": {
|
package/src/capability.ts
CHANGED
|
@@ -1,23 +1,20 @@
|
|
|
1
1
|
// Own model eligibility and endpoint identity; authentication supplies the effective base URL.
|
|
2
2
|
import type { Api, Model } from "@earendil-works/pi-ai";
|
|
3
|
-
import { hasApi } from "@earendil-works/pi-ai";
|
|
4
3
|
import { isObject } from "./protocol.js";
|
|
5
4
|
|
|
6
5
|
export const CODEX_API = "openai-codex-responses" as const;
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
const OFFICIAL_PROVIDER = "openai-codex";
|
|
10
|
-
const OFFICIAL_BASE_URL = "https://chatgpt.com/backend-api";
|
|
6
|
+
// Name-matched models need a Responses payload with an input array and a query-free request URL.
|
|
7
|
+
const NAME_MATCH_APIS: readonly Api[] = ["openai-responses", CODEX_API];
|
|
11
8
|
|
|
12
9
|
export interface ProviderIdentity {
|
|
13
10
|
readonly provider: string;
|
|
14
|
-
readonly api:
|
|
11
|
+
readonly api: Api;
|
|
15
12
|
readonly modelId: string;
|
|
16
13
|
readonly baseUrl: string;
|
|
17
14
|
readonly endpoint: string;
|
|
18
15
|
}
|
|
19
16
|
export interface CapableModel {
|
|
20
|
-
readonly model: Model<
|
|
17
|
+
readonly model: Model<Api>;
|
|
21
18
|
readonly identity: ProviderIdentity;
|
|
22
19
|
}
|
|
23
20
|
|
|
@@ -30,18 +27,15 @@ export function normalizeUrl(value: string): string {
|
|
|
30
27
|
url.pathname = url.pathname.replace(/\/+$/, "") || "/";
|
|
31
28
|
return url.toString().replace(/\/$/, "");
|
|
32
29
|
}
|
|
33
|
-
/**
|
|
34
|
-
export function deriveEndpoint(baseUrl: string, api:
|
|
35
|
-
if (api
|
|
30
|
+
/** Use Pi's Codex route for that adapter and the Responses route for other APIs. */
|
|
31
|
+
export function deriveEndpoint(baseUrl: string, api: Api): string {
|
|
32
|
+
if (api !== CODEX_API) return `${baseUrl}/responses`;
|
|
36
33
|
if (baseUrl.endsWith("/codex/responses")) return baseUrl;
|
|
37
34
|
if (baseUrl.endsWith("/codex")) return `${baseUrl}/responses`;
|
|
38
35
|
return `${baseUrl}/codex/responses`;
|
|
39
36
|
}
|
|
40
37
|
|
|
41
|
-
function configuredEndpoint(model: Model<
|
|
42
|
-
// Pi preserves extension metadata but its built-in compatibility type does not declare it.
|
|
43
|
-
const compat: unknown = model.compat;
|
|
44
|
-
const value = isObject(compat) ? compat.remoteCompaction : undefined;
|
|
38
|
+
function configuredEndpoint(value: unknown, model: Model<Api>, baseUrl: string): string | undefined {
|
|
45
39
|
if (!isObject(value) || value.protocol !== "v2" ||
|
|
46
40
|
(value.endpoint !== undefined && typeof value.endpoint !== "string")) return undefined;
|
|
47
41
|
try {
|
|
@@ -54,7 +48,12 @@ function configuredEndpoint(model: Model<RemoteCompactionApi>, baseUrl: string):
|
|
|
54
48
|
}
|
|
55
49
|
|
|
56
50
|
export function capableModel(model: Model<Api> | undefined, effectiveBaseUrl?: string): CapableModel | undefined {
|
|
57
|
-
if (!model
|
|
51
|
+
if (!model) return undefined;
|
|
52
|
+
// Pi preserves extension metadata but its built-in compatibility type does not declare it.
|
|
53
|
+
const compat: unknown = model.compat;
|
|
54
|
+
const configured = isObject(compat) ? compat.remoteCompaction : undefined;
|
|
55
|
+
if (configured === undefined &&
|
|
56
|
+
(!NAME_MATCH_APIS.includes(model.api) || !model.id.toLowerCase().includes("gpt"))) return undefined;
|
|
58
57
|
let configuredBaseUrl: string;
|
|
59
58
|
let baseUrl: string;
|
|
60
59
|
try {
|
|
@@ -63,8 +62,8 @@ export function capableModel(model: Model<Api> | undefined, effectiveBaseUrl?: s
|
|
|
63
62
|
} catch {
|
|
64
63
|
return undefined;
|
|
65
64
|
}
|
|
66
|
-
const endpoint =
|
|
67
|
-
? deriveEndpoint(baseUrl,
|
|
65
|
+
const endpoint = configured === undefined
|
|
66
|
+
? deriveEndpoint(baseUrl, model.api) : configuredEndpoint(configured, model, baseUrl);
|
|
68
67
|
return endpoint ? {
|
|
69
68
|
model,
|
|
70
69
|
identity: { provider: model.provider, api: model.api, modelId: model.id, baseUrl, endpoint },
|
package/src/checkpoint.ts
CHANGED
|
@@ -12,8 +12,6 @@ import {
|
|
|
12
12
|
} from "@earendil-works/pi-coding-agent";
|
|
13
13
|
import { isObject, type JsonObject, REMOTE_COMPACTION_PROTOCOL, validateCompactionItem } from "./protocol.js";
|
|
14
14
|
import {
|
|
15
|
-
CODEX_API,
|
|
16
|
-
OPENAI_RESPONSES_API,
|
|
17
15
|
normalizeUrl,
|
|
18
16
|
type ProviderIdentity,
|
|
19
17
|
} from "./capability.js";
|
|
@@ -102,7 +100,8 @@ export function parseCheckpointDetails(value: unknown): CodexCheckpointDetails |
|
|
|
102
100
|
value.checkpointId.length < 8 ||
|
|
103
101
|
typeof value.provider !== "string" ||
|
|
104
102
|
!value.provider ||
|
|
105
|
-
|
|
103
|
+
typeof value.api !== "string" ||
|
|
104
|
+
!value.api ||
|
|
106
105
|
typeof value.modelId !== "string" ||
|
|
107
106
|
!value.modelId ||
|
|
108
107
|
typeof value.baseUrl !== "string" ||
|
package/src/fallback-settings.ts
CHANGED
|
@@ -3,6 +3,7 @@ import { randomUUID } from "node:crypto";
|
|
|
3
3
|
import { lstat, mkdir, open, rename, rm, writeFile } from "node:fs/promises";
|
|
4
4
|
import { basename, dirname, join } from "node:path";
|
|
5
5
|
import { withFileMutationQueue } from "@earendil-works/pi-coding-agent";
|
|
6
|
+
import lockfile from "proper-lockfile";
|
|
6
7
|
import { isObject } from "./protocol.js";
|
|
7
8
|
|
|
8
9
|
export const COMPACTION_SETTINGS_RELATIVE_PATH = "extensions/pi-codex-compaction/config.json";
|
|
@@ -28,6 +29,35 @@ interface SettingsFile {
|
|
|
28
29
|
readonly mode: number;
|
|
29
30
|
}
|
|
30
31
|
|
|
32
|
+
async function withSettingsLock(path: string, save: (assertOwned: () => void) => Promise<void>): Promise<void> {
|
|
33
|
+
let compromised: Error | undefined;
|
|
34
|
+
const release = await lockfile.lock(path, {
|
|
35
|
+
// The target can be absent on its first save; the adjacent lock still serializes its creation.
|
|
36
|
+
realpath: false,
|
|
37
|
+
retries: { retries: 10, factor: 1.5, minTimeout: 20, maxTimeout: 100 },
|
|
38
|
+
onCompromised: (error) => { compromised = error; },
|
|
39
|
+
}).catch((error: unknown) => {
|
|
40
|
+
throw new Error(isObject(error) && error.code === "ELOCKED"
|
|
41
|
+
? `Compaction settings at ${path} are being edited by another Pi process; reopen /codex-compaction and try again`
|
|
42
|
+
: `Could not lock compaction settings at ${path}`, { cause: error });
|
|
43
|
+
});
|
|
44
|
+
const assertOwned = () => {
|
|
45
|
+
if (compromised) throw new Error(`Compaction settings lock was lost at ${path}; reopen /codex-compaction and try again`, { cause: compromised });
|
|
46
|
+
};
|
|
47
|
+
try {
|
|
48
|
+
assertOwned();
|
|
49
|
+
await save(assertOwned);
|
|
50
|
+
} finally {
|
|
51
|
+
try {
|
|
52
|
+
await release();
|
|
53
|
+
} catch (error) {
|
|
54
|
+
// A compromised lock is already released by proper-lockfile; surface that cause below.
|
|
55
|
+
if (!compromised) throw new Error(`Could not release compaction settings lock at ${path}`, { cause: error });
|
|
56
|
+
}
|
|
57
|
+
assertOwned();
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
|
|
31
61
|
async function readSettingsFile(path: string): Promise<SettingsFile> {
|
|
32
62
|
let file;
|
|
33
63
|
try {
|
|
@@ -133,7 +163,8 @@ function parseFallback(fallback: unknown): FallbackConfiguration {
|
|
|
133
163
|
|
|
134
164
|
/**
|
|
135
165
|
* Capture one revision; reading configuration.fallback validates it on demand and can throw.
|
|
136
|
-
* Saving
|
|
166
|
+
* Saving locks across Pi processes before checking the revision and atomically replacing the file.
|
|
167
|
+
* A selection without a fallback field keeps the file's fallback as written.
|
|
137
168
|
*/
|
|
138
169
|
export async function loadCompactionSettings(path: string) {
|
|
139
170
|
const original = await readSettingsFile(path);
|
|
@@ -150,31 +181,34 @@ export async function loadCompactionSettings(path: string) {
|
|
|
150
181
|
const fallback = "fallback" in selection ? selection.fallback : rawFallback?.value;
|
|
151
182
|
await withFileMutationQueue(path, async () => {
|
|
152
183
|
await mkdir(dirname(path), { recursive: true });
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
184
|
+
await withSettingsLock(path, async (assertOwned) => {
|
|
185
|
+
const temporary = join(dirname(path), `.${basename(path)}.${randomUUID()}.tmp`);
|
|
186
|
+
const bytes = Buffer.from(`${JSON.stringify({
|
|
187
|
+
version: SETTINGS_VERSION,
|
|
188
|
+
remoteCompaction: { enabled: selection.remoteCompactionEnabled },
|
|
189
|
+
...(fallback === undefined ? {} : { fallback }),
|
|
190
|
+
}, null, 2)}\n`);
|
|
191
|
+
if (bytes.length > MAX_SETTINGS_BYTES) throw new Error(`Compaction settings must not exceed ${MAX_SETTINGS_BYTES} bytes`);
|
|
192
|
+
try {
|
|
193
|
+
await writeFile(temporary, bytes, { flag: "wx", mode: original.mode });
|
|
194
|
+
const current = await readSettingsFile(path);
|
|
195
|
+
if (original.bytes === undefined ? current.bytes !== undefined : !current.bytes?.equals(original.bytes)) {
|
|
196
|
+
throw new Error("Compaction settings changed while the menu was open; reopen /codex-compaction and try again");
|
|
197
|
+
}
|
|
198
|
+
// Reading follows existing symlinks; an atomic replacement would remove the link itself.
|
|
199
|
+
const entry = await lstat(path).catch((error: unknown) => {
|
|
200
|
+
if (isObject(error) && error.code === "ENOENT") return undefined;
|
|
201
|
+
throw error;
|
|
202
|
+
});
|
|
203
|
+
if (entry?.isSymbolicLink()) {
|
|
204
|
+
throw new Error(`Compaction settings are a symbolic link; edit its target directly at ${path}`);
|
|
205
|
+
}
|
|
206
|
+
assertOwned();
|
|
207
|
+
await rename(temporary, path);
|
|
208
|
+
} finally {
|
|
209
|
+
await rm(temporary, { force: true });
|
|
165
210
|
}
|
|
166
|
-
|
|
167
|
-
const entry = await lstat(path).catch((error: unknown) => {
|
|
168
|
-
if (isObject(error) && error.code === "ENOENT") return undefined;
|
|
169
|
-
throw error;
|
|
170
|
-
});
|
|
171
|
-
if (entry?.isSymbolicLink()) {
|
|
172
|
-
throw new Error(`Compaction settings are a symbolic link; edit its target directly at ${path}`);
|
|
173
|
-
}
|
|
174
|
-
await rename(temporary, path);
|
|
175
|
-
} finally {
|
|
176
|
-
await rm(temporary, { force: true });
|
|
177
|
-
}
|
|
211
|
+
});
|
|
178
212
|
});
|
|
179
213
|
},
|
|
180
214
|
};
|
package/src/index.ts
CHANGED
|
@@ -30,7 +30,7 @@ import {
|
|
|
30
30
|
projectCheckpointContext,
|
|
31
31
|
projectCheckpointRequest,
|
|
32
32
|
} from "./checkpoint.js";
|
|
33
|
-
import { hasCheckpointMarker, type JsonObject, REMOTE_COMPACTION_PROTOCOL, rewriteCheckpointMarker } from "./protocol.js";
|
|
33
|
+
import { hasCheckpointMarker, type JsonObject, REMOTE_COMPACTION_PROTOCOL, rewriteCheckpointMarker, withoutInputImages } from "./protocol.js";
|
|
34
34
|
import { requestRemoteCompaction } from "./remote.js";
|
|
35
35
|
import { requestFallbackCompaction } from "./fallback.js";
|
|
36
36
|
import { COMPACTION_SETTINGS_RELATIVE_PATH, loadCompactionSettings, type CompactionConfiguration } from "./fallback-settings.js";
|
|
@@ -39,6 +39,8 @@ import { compactionRequest, providerRequestFor, RequestSnapshotTracker, type Pro
|
|
|
39
39
|
|
|
40
40
|
const STATUS_KEY = "codex-compaction";
|
|
41
41
|
const COMPLETION_ENTRY_TYPE = "pi-codex-compaction-completed";
|
|
42
|
+
const INCOMPATIBLE_BACKEND_WARNING =
|
|
43
|
+
"The active Codex checkpoint cannot replay on this provider backend; only its fallback marker and retained recent messages remain available.";
|
|
42
44
|
|
|
43
45
|
interface CompletionEntryData {
|
|
44
46
|
message: string;
|
|
@@ -103,7 +105,7 @@ function projectedCurrentMessages(
|
|
|
103
105
|
return { messages: projected, prior };
|
|
104
106
|
}
|
|
105
107
|
|
|
106
|
-
async function replayCheckpoint(payload: unknown, ctx: ExtensionContext): Promise<JsonObject | undefined> {
|
|
108
|
+
async function replayCheckpoint(payload: unknown, ctx: ExtensionContext, blockImages: boolean): Promise<JsonObject | undefined> {
|
|
107
109
|
const checkpoint = activeCheckpoint(ctx);
|
|
108
110
|
if (!checkpoint) return undefined;
|
|
109
111
|
const marker = checkpointMarker(checkpoint.details.checkpointId);
|
|
@@ -111,7 +113,8 @@ async function replayCheckpoint(payload: unknown, ctx: ExtensionContext): Promis
|
|
|
111
113
|
if (!await compatibleIdentity(checkpoint.details, ctx)) {
|
|
112
114
|
throw new Error("The active opaque checkpoint no longer matches the resolved provider endpoint");
|
|
113
115
|
}
|
|
114
|
-
|
|
116
|
+
const history = checkpoint.details.replacementHistory;
|
|
117
|
+
return rewriteCheckpointMarker(payload, marker, blockImages ? withoutInputImages(history) : history);
|
|
115
118
|
}
|
|
116
119
|
|
|
117
120
|
function errorMessage(error: unknown): string {
|
|
@@ -123,6 +126,7 @@ function notifyFailure(ctx: ExtensionContext, error: unknown): void {
|
|
|
123
126
|
ctx.ui.notify(`Codex remote compaction failed; using Pi compaction. ${errorMessage(error)}`, "warning");
|
|
124
127
|
}
|
|
125
128
|
|
|
129
|
+
/** A model switch never cancels compaction; it finishes with the model selected when it started, as Pi's does. */
|
|
126
130
|
function sessionStillOwned(ctx: ExtensionContext, sessionId: string, signal: AbortSignal): boolean {
|
|
127
131
|
return !signal.aborted && ctx.sessionManager.getSessionId() === sessionId;
|
|
128
132
|
}
|
|
@@ -133,7 +137,7 @@ async function compactFallback(
|
|
|
133
137
|
ctx: ExtensionContext,
|
|
134
138
|
sessionId: string,
|
|
135
139
|
configuration: CompactionConfiguration,
|
|
136
|
-
remoteError
|
|
140
|
+
remoteError: unknown,
|
|
137
141
|
) {
|
|
138
142
|
let announced = false;
|
|
139
143
|
try {
|
|
@@ -147,7 +151,7 @@ async function compactFallback(
|
|
|
147
151
|
signal: event.signal,
|
|
148
152
|
sessionId,
|
|
149
153
|
onPrepared: ({ model, thinkingLevel }) => {
|
|
150
|
-
if (!sessionStillOwned(ctx, sessionId, event.signal)) throw new Error("Compaction session
|
|
154
|
+
if (!sessionStillOwned(ctx, sessionId, event.signal)) throw new Error("Compaction session changed");
|
|
151
155
|
if (announced) return;
|
|
152
156
|
announced = true;
|
|
153
157
|
ctx.ui.setStatus(STATUS_KEY, "Pi fallback compaction...");
|
|
@@ -183,48 +187,51 @@ async function compactRemotely(
|
|
|
183
187
|
) {
|
|
184
188
|
const supported = capableModel(ctx.model);
|
|
185
189
|
const sessionId = ctx.sessionManager.getSessionId();
|
|
186
|
-
|
|
190
|
+
const owned = () => sessionStillOwned(ctx, sessionId, event.signal);
|
|
191
|
+
if (!owned()) return { cancel: true };
|
|
187
192
|
let configuration: CompactionConfiguration;
|
|
188
193
|
try {
|
|
189
194
|
configuration = (await loadCompactionSettings(compactionSettingsPath)).configuration;
|
|
190
195
|
} catch (error) {
|
|
191
|
-
if (
|
|
196
|
+
if (owned() && ctx.hasUI) {
|
|
192
197
|
ctx.ui.notify(`Could not read compaction settings; compaction stopped. ${errorMessage(error)}`, "error");
|
|
193
198
|
}
|
|
194
199
|
return { cancel: true };
|
|
195
200
|
}
|
|
196
|
-
if (!
|
|
197
|
-
if (!configuration.remoteCompactionEnabled || !supported) return compactFallback(pi, event, ctx, sessionId, configuration);
|
|
201
|
+
if (!owned()) return { cancel: true };
|
|
202
|
+
if (!configuration.remoteCompactionEnabled || !supported) return compactFallback(pi, event, ctx, sessionId, configuration, undefined);
|
|
198
203
|
const reasoning = pi.getThinkingLevel();
|
|
199
204
|
const settings = pi.getSettings();
|
|
200
205
|
let announced = false;
|
|
201
206
|
ctx.ui.setStatus(STATUS_KEY, "Codex remote compaction...");
|
|
202
207
|
try {
|
|
203
|
-
if (!
|
|
208
|
+
if (!owned()) return { cancel: true };
|
|
204
209
|
if (event.customInstructions?.trim() && ctx.hasUI) {
|
|
205
210
|
ctx.ui.notify("Codex Remote Compaction V2 does not accept custom instructions; they are ignored.", "warning");
|
|
206
211
|
}
|
|
207
212
|
const current = projectedCurrentMessages(event, supported.identity);
|
|
208
213
|
const inputs = providerRequestInputs(pi, ctx);
|
|
214
|
+
const providerRequest = providerRequestFor(snapshots, sessionId, supported, current.messages, inputs);
|
|
209
215
|
const request = compactionRequest(snapshots, sessionId, supported, current.messages, {
|
|
210
216
|
blockImages: settings.images?.blockImages ?? false,
|
|
211
217
|
systemPrompt: () => ctx.getSystemPrompt(),
|
|
212
218
|
tools: () => activeTools(pi),
|
|
213
|
-
});
|
|
219
|
+
}, providerRequest);
|
|
214
220
|
const response = await requestRemoteCompaction({
|
|
215
221
|
modelRegistry: ctx.modelRegistry,
|
|
216
222
|
model: supported.model,
|
|
217
223
|
context: request.context,
|
|
218
224
|
userItemOrigins: userItemOrigins(request.messages),
|
|
219
|
-
providerRequest
|
|
225
|
+
providerRequest,
|
|
220
226
|
reasoning,
|
|
221
227
|
sessionId,
|
|
222
228
|
thinkingBudgets: settings.thinkingBudgets,
|
|
229
|
+
blockImages: settings.images?.blockImages ?? false,
|
|
223
230
|
maxRetries: settings.retry?.provider?.maxRetries,
|
|
224
231
|
maxRetryDelayMs: settings.retry?.provider?.maxRetryDelayMs,
|
|
225
232
|
signal: event.signal,
|
|
226
233
|
onPrepared: () => {
|
|
227
|
-
if (!
|
|
234
|
+
if (!owned()) throw new Error("Compaction session changed");
|
|
228
235
|
// Provider retries prepare the payload again; announce the compaction once.
|
|
229
236
|
if (announced) return;
|
|
230
237
|
announced = true;
|
|
@@ -242,12 +249,12 @@ async function compactRemotely(
|
|
|
242
249
|
: undefined,
|
|
243
250
|
fetch,
|
|
244
251
|
});
|
|
245
|
-
if (!
|
|
252
|
+
if (!owned()) return { cancel: true };
|
|
246
253
|
const retention = await prepareRetention(response.promptInput, event.signal, {
|
|
247
254
|
images: response.images,
|
|
248
255
|
contextual: response.contextual,
|
|
249
256
|
});
|
|
250
|
-
if (!
|
|
257
|
+
if (!owned()) return { cancel: true };
|
|
251
258
|
const replacementHistory = buildReplacementHistory(retention, response.item);
|
|
252
259
|
const details = createCheckpointDetails({
|
|
253
260
|
identity: response.identity,
|
|
@@ -264,7 +271,7 @@ async function compactRemotely(
|
|
|
264
271
|
},
|
|
265
272
|
};
|
|
266
273
|
} catch (error) {
|
|
267
|
-
if (
|
|
274
|
+
if (!owned()) {
|
|
268
275
|
return { cancel: true };
|
|
269
276
|
}
|
|
270
277
|
return await compactFallback(pi, event, ctx, sessionId, configuration, error);
|
|
@@ -283,6 +290,12 @@ export function createCodexCompactionExtension(
|
|
|
283
290
|
registerCompactionCommand(pi, compactionSettingsPath);
|
|
284
291
|
const warnings = new Set<string>();
|
|
285
292
|
const snapshots = new RequestSnapshotTracker();
|
|
293
|
+
const warnOnce = (ctx: ExtensionContext, key: string, message: string) => {
|
|
294
|
+
const scoped = `${ctx.sessionManager.getSessionId()}:${key}`;
|
|
295
|
+
if (warnings.has(scoped)) return;
|
|
296
|
+
warnings.add(scoped);
|
|
297
|
+
if (ctx.hasUI) ctx.ui.notify(message, "warning");
|
|
298
|
+
};
|
|
286
299
|
|
|
287
300
|
pi.registerEntryRenderer<CompletionEntryData>(
|
|
288
301
|
COMPLETION_ENTRY_TYPE,
|
|
@@ -303,7 +316,7 @@ export function createCodexCompactionExtension(
|
|
|
303
316
|
compactRemotely(pi, event, ctx, compactionSettingsPath, options.fetch, snapshots.current()),
|
|
304
317
|
);
|
|
305
318
|
|
|
306
|
-
pi.on("session_compact", (event) => {
|
|
319
|
+
pi.on("session_compact", async (event, ctx) => {
|
|
307
320
|
if (!event.fromExtension) return;
|
|
308
321
|
const details = parseCheckpointDetails(event.compactionEntry.details);
|
|
309
322
|
if (!details) return;
|
|
@@ -312,6 +325,9 @@ export function createCodexCompactionExtension(
|
|
|
312
325
|
protocol: REMOTE_COMPACTION_PROTOCOL,
|
|
313
326
|
checkpointId: details.checkpointId,
|
|
314
327
|
});
|
|
328
|
+
// The model may have switched to another backend while compaction ran.
|
|
329
|
+
if (await compatibleIdentity(details, ctx)) return;
|
|
330
|
+
warnOnce(ctx, `${details.checkpointId}:backend`, INCOMPATIBLE_BACKEND_WARNING);
|
|
315
331
|
});
|
|
316
332
|
|
|
317
333
|
pi.on("context", async (event, ctx) => {
|
|
@@ -320,16 +336,8 @@ export function createCodexCompactionExtension(
|
|
|
320
336
|
if (!checkpoint || !await compatibleIdentity(checkpoint.details, ctx)) return undefined;
|
|
321
337
|
const messages = projectCheckpointContext(event.messages, checkpoint.details);
|
|
322
338
|
if (messages) return { messages };
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
warnings.add(key);
|
|
326
|
-
if (ctx.hasUI) {
|
|
327
|
-
ctx.ui.notify(
|
|
328
|
-
"The active Codex checkpoint no longer matches the retained messages, so its opaque history is not replayed.",
|
|
329
|
-
"warning",
|
|
330
|
-
);
|
|
331
|
-
}
|
|
332
|
-
}
|
|
339
|
+
warnOnce(ctx, `${checkpoint.details.checkpointId}:projection`,
|
|
340
|
+
"The active Codex checkpoint no longer matches the retained messages, so its opaque history is not replayed.");
|
|
333
341
|
return undefined;
|
|
334
342
|
});
|
|
335
343
|
|
|
@@ -344,29 +352,26 @@ export function createCodexCompactionExtension(
|
|
|
344
352
|
});
|
|
345
353
|
|
|
346
354
|
pi.on("before_provider_request", async (event, ctx) => {
|
|
347
|
-
const payload = await replayCheckpoint(event.payload, ctx);
|
|
355
|
+
const payload = await replayCheckpoint(event.payload, ctx, pi.getSettings().images?.blockImages ?? false);
|
|
348
356
|
snapshots.recordProviderRequest(
|
|
349
357
|
ctx.sessionManager.getSessionId(),
|
|
350
358
|
capableModel(ctx.model),
|
|
351
359
|
() => canonicalMessages(ctx),
|
|
352
360
|
() => ctx.getSystemPrompt(),
|
|
353
|
-
|
|
361
|
+
// Keep the wire marker in the snapshot; ordinary requests and V2 each replay it exactly once.
|
|
362
|
+
() => ({ payload: event.payload, inputs: providerRequestInputs(pi, ctx) }),
|
|
354
363
|
);
|
|
355
364
|
return payload;
|
|
356
365
|
});
|
|
357
366
|
|
|
367
|
+
pi.on("agent_settled", (_event, ctx) => {
|
|
368
|
+
snapshots.recordSettledRun(ctx.sessionManager.getSessionId(), capableModel(ctx.model), () => providerRequestInputs(pi, ctx));
|
|
369
|
+
});
|
|
370
|
+
|
|
358
371
|
pi.on("model_select", async (event, ctx) => {
|
|
359
372
|
const checkpoint = activeCheckpoint(ctx);
|
|
360
373
|
if (!checkpoint || await compatibleIdentity(checkpoint.details, ctx, event.model)) return;
|
|
361
|
-
|
|
362
|
-
if (warnings.has(key)) return;
|
|
363
|
-
warnings.add(key);
|
|
364
|
-
if (ctx.hasUI) {
|
|
365
|
-
ctx.ui.notify(
|
|
366
|
-
"The active Codex checkpoint cannot replay on this provider backend; only its fallback marker and retained recent messages remain available.",
|
|
367
|
-
"warning",
|
|
368
|
-
);
|
|
369
|
-
}
|
|
374
|
+
warnOnce(ctx, `${event.model.provider}:${event.model.id}`, INCOMPATIBLE_BACKEND_WARNING);
|
|
370
375
|
});
|
|
371
376
|
|
|
372
377
|
pi.on("session_shutdown", (_event, ctx) => {
|