@narumitw/pi-codex-compact 0.47.0 โ†’ 0.47.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +194 -61
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,97 +1,225 @@
1
- # @narumitw/pi-codex-compact
1
+ # ๐Ÿ—œ๏ธ pi-codex-compact โ€” Remote Compaction V2 for Pi
2
+
3
+ [![npm](https://img.shields.io/npm/v/@narumitw/pi-codex-compact)](https://www.npmjs.com/package/@narumitw/pi-codex-compact) [![Pi extension](https://img.shields.io/badge/Pi-extension-blue)](https://pi.dev) [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)
2
4
 
3
5
  > [!WARNING]
4
6
  > This extension is experimental. It depends on an undocumented OpenAI Codex Responses wire
5
7
  > contract and stores provider-specific opaque checkpoints. Keep backups of important sessions.
6
8
 
7
- Codex Remote Compaction V2 for Pi's built-in `openai-codex` OAuth provider. It replaces Pi's
8
- plaintext summary-generation call with a server-generated opaque compaction item, then replays
9
+ `@narumitw/pi-codex-compact` adds Codex Remote Compaction V2 to the
10
+ [Pi Coding Agent](https://pi.dev) for the built-in `openai-codex` OAuth provider. It replaces Pi's
11
+ plaintext summary-generation call with a server-generated opaque compaction item and safely replays
9
12
  that item in later OpenAI Codex Responses requests.
10
13
 
11
- Pi still owns compaction thresholds, `/compact`, overflow retries, retained-message selection, and
12
- the append-only session tree. This package does not reproduce Codex core's context-window lineage
13
- or exact pre-turn and mid-turn lifecycle.
14
+ Pi remains responsible for deciding when compaction runs, including automatic thresholds, the
15
+ built-in `/compact` command, overflow retries, retained-message selection, and append-only session
16
+ publication. This package owns only the Remote V2 request, checkpoint persistence, and later payload
17
+ replay.
18
+
19
+ ## โœจ Features
20
+
21
+ - Uses the active `openai-codex` OAuth credentials without persisting tokens or request headers.
22
+ - Handles Pi's manual, threshold, and overflow compaction reasons through the same lifecycle hook.
23
+ - Validates one bounded opaque `compaction` item from a completed Responses SSE stream.
24
+ - Persists a versioned checkpoint in `CompactionEntry.details` and restores it after reload, resume,
25
+ or a fork that retains the checkpoint.
26
+ - Replays the newest compatible checkpoint while preserving later conversation and extension-added
27
+ context.
28
+ - Supports repeated compaction by carrying the previous opaque item into the next Remote V2 request.
29
+ - Falls back to Pi's native plaintext compaction after authentication, transport, protocol, or
30
+ validation failure; user cancellation does not launch the fallback.
31
+ - Provides `/codex-compact` for manual compaction, effective-route visibility, and bounded settings.
32
+
33
+ ## ๐Ÿ“ฆ Install
14
34
 
15
- ## Install
35
+ Install persistently from npm:
16
36
 
17
37
  ```bash
18
38
  pi install npm:@narumitw/pi-codex-compact
19
39
  ```
20
40
 
21
- From this repository:
41
+ Try the published package without installing:
22
42
 
23
43
  ```bash
24
- just try codex-compact
44
+ pi -e npm:@narumitw/pi-codex-compact
25
45
  ```
26
46
 
27
- Loading the package enables it. Use Pi normally, including its built-in `/compact` command. Open
28
- the extension's TUI control menu with:
47
+ Try a local checkout from the repository root:
29
48
 
30
- ```text
31
- /codex-compact
49
+ ```bash
50
+ pi -e ./experimental/pi-codex-compact
51
+ # or
52
+ just try codex-compact
32
53
  ```
33
54
 
34
- The menu shows the active model and whether compaction will use Codex Remote V2 or Pi's native
35
- fallback. Choose **Compact now** to close the menu and compact immediately, or **Settings** to edit
36
- user settings in `~/.pi/agent/pi-codex-compact.json` (or the active Pi agent directory). Invalid
37
- files are never overwritten; manual compaction remains available, while Settings opens read-only
38
- repair guidance until the file is fixed and `/reload` is run.
55
+ Loading the package enables Remote V2 with safe defaults. Avoid loading a global npm installation
56
+ and the local workspace at the same time.
39
57
 
40
- | Setting | Default | Accepted values |
41
- | --- | ---: | --- |
42
- | `enabled` | `true` | Boolean |
43
- | `requestTimeoutMs` | `300000` | Integer from 30,000 to 600,000 ms |
44
- | `maxRetries` | `2` | Integer from 0 to 2 |
45
- | `replacementTokenBudget` | `64000` | Integer from 8,000 to 128,000 tokens |
46
- | `notifyOnFallback` | `true` | Boolean |
58
+ ## ๐Ÿš€ Quick start
47
59
 
48
- Missing fields use built-in defaults; the file has no environment-variable or project-level
49
- override. Settings reload on every `session_start`, including `/reload`, resume, and fork. Menu
50
- writes apply immediately, preserve unknown JSON fields, serialize within the current Pi process, and
51
- use a conflict check plus atomic rename. Separate Pi processes are not coordinated by a shared lock;
52
- a detected concurrent edit is rejected for the user to reopen and retry.
60
+ 1. Sign in through Pi's built-in OpenAI Codex OAuth provider.
61
+ 2. Select a model whose provider is `openai-codex` and API is `openai-codex-responses`.
62
+ 3. Work normally. Pi's automatic compaction and built-in `/compact` continue to operate.
63
+ 4. Run `/codex-compact` to inspect the effective route or choose **Compact now**.
64
+ 5. After compaction, continue the session normally; compatible requests replay the opaque checkpoint.
53
65
 
54
- ## Requirements
66
+ The extension shows an experimental warning at session start in UI-capable modes. When the active
67
+ model is unsupported, compaction remains entirely Pi-native.
55
68
 
56
- - Pi `0.83.0`-compatible extension APIs.
57
- - A model using provider `openai-codex` and API `openai-codex-responses`.
58
- - A working OpenAI Codex OAuth login in Pi.
69
+ ## ๐Ÿ’ฌ Command
59
70
 
60
- OpenAI API-key, Azure, Copilot, proxies, and generic Responses-compatible providers are not
61
- supported. Unsupported models continue to use Pi's native compaction.
71
+ ```text
72
+ /codex-compact
73
+ ```
62
74
 
63
- ## How it works
75
+ In TUI mode, the root menu shows whether Remote V2 is enabled, the active model, and whether a
76
+ manual compact will use **Codex Remote V2** or **Pi native**. It contains:
64
77
 
65
- 1. Pi decides when compaction is needed.
66
- 2. The extension sends the current Codex Responses input with a final `compaction_trigger`.
67
- 3. It validates and persists the server's encrypted `compaction` item in the Pi
68
- `CompactionEntry.details` field.
69
- 4. Later Codex requests replace Pi's fallback summary and kept suffix with the opaque replacement
70
- history before dispatch.
78
+ ```text
79
+ Compact now
80
+ Settings
81
+ Close
82
+ ```
83
+
84
+ **Compact now** closes the menu before asking Pi to compact the active session. Escape or Ctrl+C
85
+ closes without compacting, and an obsolete menu cannot trigger work after session replacement or
86
+ shutdown. **Settings** opens the bounded settings editor. In non-TUI modes, the command reports the
87
+ manual settings path instead of opening custom UI or compacting.
88
+
89
+ Pi's built-in `/compact` remains available and follows the same extension hook when the active model
90
+ is compatible.
71
91
 
72
- The SSE response is limited to 8 MiB, the opaque item to 2 MiB, and the persisted replacement
73
- history to the configured text budget (64,000 tokens by default) and 8 MiB. Oversized media
74
- retention is dropped rather than creating an unbounded session entry.
92
+ ## โš™๏ธ Settings
75
93
 
76
- ## Failure and portability
94
+ The extension has one optional, global-only JSON settings file:
77
95
 
78
- Remote auth, transport, protocol, or validation failures fall back to Pi's native plaintext
79
- compaction. User cancellation does not start that fallback.
96
+ ```text
97
+ <getAgentDir()>/pi-codex-compact.json
98
+ ```
80
99
 
81
- After a successful remote compaction, full older history can be replayed only when this extension
82
- is loaded with the same `openai-codex` model. If the extension is removed or the model/provider is
83
- changed, Pi exposes a warning summary plus its retained recent messages. Switching back restores
84
- opaque replay. Repeated compaction after an opaque checkpoint also requires the checkpoint to be
85
- projected safely; otherwise Pi falls back rather than guessing.
100
+ The normal path is `~/.pi/agent/pi-codex-compact.json`. There is no environment-variable or
101
+ project-level override.
102
+
103
+ ```json
104
+ {
105
+ "enabled": true,
106
+ "requestTimeoutMs": 300000,
107
+ "maxRetries": 2,
108
+ "replacementTokenBudget": 64000,
109
+ "notifyOnFallback": true
110
+ }
111
+ ```
86
112
 
87
- ## Privacy and storage
113
+ | Setting | Default | Accepted values | Behavior | Recommendation |
114
+ | --- | ---: | --- | --- | --- |
115
+ | `enabled` | `true` | Boolean | Attempt Remote V2 for a compatible model. | Keep enabled unless diagnosing provider behavior. |
116
+ | `requestTimeoutMs` | `300000` | Integer from 30,000 to 600,000 ms | Bound one extension-owned remote request. | Keep five minutes; increase only for a consistently slow connection. |
117
+ | `maxRetries` | `2` | Integer from 0 to 2 | Retry transient provider transport failures before Pi fallback. | Keep two; use zero when diagnosing the first failure. |
118
+ | `replacementTokenBudget` | `64000` | Integer from 8,000 to 128,000 tokens | Bound approximate retained user-message text beside the opaque item. | Keep 64K; lower it to reduce session size or raise it only when recent user context is being lost. |
119
+ | `notifyOnFallback` | `true` | Boolean | Warn when Remote V2 fails and Pi-native compaction takes over. | Keep enabled so silent fallback does not hide protocol or entitlement problems. |
120
+
121
+ Missing fields use defaults. Settings reload on every `session_start`, including `/reload`, resume,
122
+ and fork. Menu writes apply immediately, preserve unknown JSON fields, serialize within the current
123
+ Pi process, and use a final conflict check plus same-directory atomic rename. On Unix, temporary
124
+ files use mode `0600`.
125
+
126
+ Malformed, invalid, oversized, or symlinked settings files are never overwritten. Safe defaults stay
127
+ active, and the menu provides read-only repair guidance until the file is fixed and Pi is reloaded.
128
+ Separate Pi processes do not share a mutation lock; a detected concurrent edit is rejected so the
129
+ user can reopen Settings and retry.
130
+
131
+ ### Relationship to Codex configuration
132
+
133
+ This extension does **not** read `~/.codex/config.toml`.
134
+
135
+ | Codex setting | Extension behavior |
136
+ | --- | --- |
137
+ | `features.remote_compaction_v2` | Conceptually corresponds to this extension's `enabled`; it is not imported. |
138
+ | `model_auto_compact_token_limit` | Not duplicated. Pi's own compaction threshold remains authoritative. |
139
+ | `model_auto_compact_token_limit_scope` | Not supported; Pi extensions do not own Codex's compact-window lineage. |
140
+ | `compact_prompt` / `experimental_compact_prompt_file` | Not used by Remote V2, whose opaque checkpoint is generated by the server. |
141
+ | `features.token_budget` | Not supported; token-budget context reset is a different experimental strategy. |
142
+
143
+ ## โœ… Requirements and compatibility
144
+
145
+ - Pi APIs compatible with the package's declared peer dependencies.
146
+ - The built-in provider `openai-codex`.
147
+ - API `openai-codex-responses` on the active model.
148
+ - A working OpenAI Codex OAuth login and Remote V2 entitlement.
149
+
150
+ OpenAI API-key, Azure, GitHub Copilot, proxies, and arbitrary Responses-compatible providers are not
151
+ supported. Switching to another provider or model leaves Pi's visible fallback marker plus retained
152
+ recent messages in context. Switching back to the checkpoint's original compatible model restores
153
+ opaque replay.
154
+
155
+ ## ๐Ÿ”„ How it works
156
+
157
+ 1. Pi prepares compaction and selects the recent message suffix it will retain.
158
+ 2. The extension projects an earlier compatible checkpoint, if present, into the current Responses
159
+ input.
160
+ 3. It appends exactly one final `compaction_trigger` and sends a normal authenticated Codex Responses
161
+ SSE request with cache retention disabled.
162
+ 4. It requires a completed response containing exactly one non-empty opaque `compaction` item.
163
+ 5. It constructs bounded replacement history from recent raw user-role Responses items followed by
164
+ the opaque item.
165
+ 6. It stores that history and fingerprints of Pi's retained suffix in versioned
166
+ `CompactionEntry.details`.
167
+ 7. On later compatible requests, it replaces an exactly validated marker with the persisted
168
+ replacement history immediately before provider dispatch.
169
+
170
+ If fingerprints, model identity, payload shape, or marker count do not match exactly, the extension
171
+ leaves Pi's visible fallback context unchanged instead of guessing.
172
+
173
+ ## ๐Ÿ” Privacy, storage, and limits
88
174
 
89
175
  Remote compaction sends the active conversation context, system prompt, and active tool schemas to
90
- the same OpenAI Codex backend used by the selected model. The extension stores the encrypted
91
- compaction item and bounded recent user-role Responses items in the Pi session. It never persists
92
- the OAuth token or request headers.
176
+ the same OpenAI Codex backend used by the selected model. The Pi session stores the encrypted
177
+ compaction item and bounded recent user-role Responses items. It does not store OAuth tokens,
178
+ authorization headers, or request headers in checkpoint details.
179
+
180
+ | Boundary | Limit |
181
+ | --- | ---: |
182
+ | Observed SSE stream | 8 MiB |
183
+ | Serialized opaque compaction item | 2 MiB |
184
+ | Persisted replacement history | 8 MiB |
185
+ | Retained user text | 64K approximate tokens by default; configurable from 8K to 128K |
186
+ | Settings file | 64 KiB |
187
+ | Transport retries | At most 2 |
188
+ | Request timeout | At most 10 minutes |
189
+
190
+ An individually oversized media item is dropped rather than making the session entry unbounded. The
191
+ oldest fitting text item may be partially truncated to preserve newer context. These hard byte
192
+ ceilings are intentionally not configurable.
193
+
194
+ ## ๐Ÿšง Experimental limitations
195
+
196
+ - The wire contract is undocumented and can change independently of Pi or this package.
197
+ - Full older history depends on this extension and the same compatible model. Removing the extension
198
+ exposes only the portability fallback marker and Pi-retained recent messages.
199
+ - The package does not reproduce Codex core's context-window UUID/number lineage, previous-model
200
+ compatibility fallback, exact pre-turn ordering, or exact mid-turn model-session ownership.
201
+ - Remote failure falls back to Pi's plaintext summary, so a session can contain both remote opaque
202
+ and native compaction entries over time.
203
+ - Settings concurrency is coordinated only within one Pi process; separate processes rely on the
204
+ final conflict check.
205
+
206
+ ## ๐Ÿ—‚๏ธ Package layout
93
207
 
94
- ## Development
208
+ ```text
209
+ src/index.ts Thin Pi entrypoint
210
+ src/codex-compact.ts Pi lifecycle, command, provider projection, and fallback
211
+ src/remote.ts Provider stream invocation, auth payload, timeout, and retry controls
212
+ src/protocol.ts Bounded SSE parsing and Remote V2 payload/output validation
213
+ src/checkpoint.ts Replacement history, fingerprints, persistence, and replay projection
214
+ src/settings.ts Global settings validation and atomic persistence
215
+ src/settings-menu.ts Manual compaction and settings TUI
216
+
217
+ test/ Protocol, checkpoint, lifecycle, remote, settings, and menu coverage
218
+ ```
219
+
220
+ ## ๐Ÿงช Development
221
+
222
+ From the repository root:
95
223
 
96
224
  ```bash
97
225
  npm --workspace @narumitw/pi-codex-compact run check
@@ -103,6 +231,11 @@ See
103
231
  [`docs/implementation-notes/codex-compaction-mechanism.md`](../../docs/implementation-notes/codex-compaction-mechanism.md)
104
232
  for the underlying Codex mechanism research and the extension boundary.
105
233
 
106
- ## License
234
+ ## ๐Ÿ”Ž Keywords
235
+
236
+ Pi extension, Pi coding agent, OpenAI Codex, OAuth, Remote Compaction V2, opaque checkpoint,
237
+ Responses API, context compaction.
238
+
239
+ ## ๐Ÿ“„ License
107
240
 
108
- MIT
241
+ [MIT](LICENSE)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@narumitw/pi-codex-compact",
3
- "version": "0.47.0",
3
+ "version": "0.47.1",
4
4
  "description": "Experimental Codex Remote Compaction V2 for Pi's OpenAI Codex provider.",
5
5
  "type": "module",
6
6
  "license": "MIT",