@agfpd/iapeer 0.4.101 → 0.4.103
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/06-storage.md
CHANGED
|
@@ -95,7 +95,7 @@ A peer's system prompt points here — `~/.iapeer/docs/`, one folder per package
|
|
|
95
95
|
```
|
|
96
96
|
|
|
97
97
|
- **`peer-profile.json`** — the peer's card and the single source of truth about it (the fields are covered in [03 — Peers, runtimes, identity](03-peers-runtimes-identity.md)).
|
|
98
|
-
- **`launch.env`** — a `KEY=VALUE` file for a specific peer+runtime pair: extra launch arguments (`PEER_START_ARGS`) and environment variables. So one peer can be given special launch flags without touching the rest.
|
|
98
|
+
- **`launch.env`** — a `KEY=VALUE` file for a specific peer+runtime pair: extra launch arguments (`PEER_START_ARGS`) and environment variables. So one peer can be given special launch flags without touching the rest. `create`/`init` scaffold it as an all-comment skeleton for every agentic runtime scope (behaves exactly like no file until edited), so fleet-wide launch.env operations never skip a peer for lack of the file; an existing file is never overwritten.
|
|
99
99
|
|
|
100
100
|
A peer's working folder doesn't have to live under `~/.iapeer/peers/`. By default `iapeer create` puts it there, but any folder can be made a peer with `iapeer init` — then `.iapeer/` appears right inside it.
|
|
101
101
|
|
package/docs/19-notices.md
CHANGED
|
@@ -53,19 +53,41 @@ enum, so no prose is matched.
|
|
|
53
53
|
|
|
54
54
|
### codex — the rollout
|
|
55
55
|
|
|
56
|
-
`~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl
|
|
56
|
+
`~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl`. **The primary signal is the death
|
|
57
|
+
fingerprint, established on the real incident of 17.07.2026** (quota genuinely exhausted,
|
|
58
|
+
both codex peers mute; forensics: `docs/internals/forensics/model-limit-2026-07-17/`). The
|
|
59
|
+
rollout records **no error event**; what the refused API call leaves instead is:
|
|
57
60
|
|
|
58
61
|
```json
|
|
59
|
-
{ "
|
|
60
|
-
"
|
|
61
|
-
"
|
|
62
|
-
|
|
62
|
+
{ "payload": { "type": "token_count",
|
|
63
|
+
"info": null-or-cumulative-totals-UNCHANGED,
|
|
64
|
+
"rate_limits": { "limit_id": "premium", "primary": null, "secondary": null,
|
|
65
|
+
"credits": { "has_credits": false, "unlimited": false, "balance": "0" },
|
|
66
|
+
"rate_limit_reached_type": null } } }
|
|
67
|
+
{ "payload": { "type": "task_complete", "last_agent_message": null, "completed_at": 1784309462 } }
|
|
63
68
|
```
|
|
64
69
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
70
|
+
The detector requires the CONJUNCTION: both windows null AND no new usage recorded (`info`
|
|
71
|
+
null in a fresh session, or cumulative totals identical to the previous `token_count` — a
|
|
72
|
+
refused call consumes nothing) AND the turn closes with `task_complete{last_agent_message:
|
|
73
|
+
null}`. Each conjunct is load-bearing: a lone null-window snapshot occurs transiently while
|
|
74
|
+
the turn SURVIVES (measured 10.07.2026 — totals advanced, the turn lived on), and a
|
|
75
|
+
message-less `task_complete` alone is an absence, not a signal (§8). The event time is
|
|
76
|
+
`completed_at` (the original epoch) — never the line timestamp, which a session resume
|
|
77
|
+
REWRITES when it replays history into a fresh rollout (measured 16.07: a 14:31:58 death
|
|
78
|
+
re-recorded at 18:49:23).
|
|
79
|
+
|
|
80
|
+
`errorType` is `usage_limit_exceeded` — codex's own error-code vocabulary (0.144.1 binary) —
|
|
81
|
+
when the tail holds a window at `used_percent ≥ 100` (the evidence this is the usage-limit
|
|
82
|
+
class); that window's `resets_at` becomes the notice's reset. Without such evidence the
|
|
83
|
+
honest fallback is `api-refusal` and no reset is invented. The model comes from the dead
|
|
84
|
+
turn's own `turn_context` (real bytes name it: `gpt-5.6-sol`).
|
|
85
|
+
|
|
86
|
+
`rate_limit_reached_type` non-null is KEPT as a forward-compat second path — but the real
|
|
87
|
+
incident **refuted it as the primary signal**: with the quota exhausted and turns dying, it
|
|
88
|
+
stayed `null` in every snapshot of every rollout. When it does fire, the detector keys on
|
|
89
|
+
**non-null**, never on the variant strings (`rate_limit_reached`,
|
|
90
|
+
`workspace_owner_usage_limit_reached`, … — read out of the 0.144.1 binary).
|
|
69
91
|
|
|
70
92
|
## 2a. The signal: `peer-goal-stalled` (codex only)
|
|
71
93
|
|
|
@@ -124,6 +146,46 @@ a stable *transition* clock, and the `updated_at_ms > since` boundary fires exac
|
|
|
124
146
|
then falls behind forever. A goal blocked while the daemon was down is missed — the same
|
|
125
147
|
honest boundary `peer-mute` already has (§7).
|
|
126
148
|
|
|
149
|
+
**Only the peer's CURRENT thread counts.** `thread_goals` is keyed by thread_id, so a thread the
|
|
150
|
+
peer has abandoned keeps its last goal row forever. Measured live 16.07.2026: a peer escaped a
|
|
151
|
+
blocked goal the only way its tools allowed — a fresh session — and ended up with two rows,
|
|
152
|
+
`blocked` on the dead thread and `active` on the live one. Reporting the fossil would tell the
|
|
153
|
+
owner an objective is stalled while the peer is working it. So a goal is attributed only on the
|
|
154
|
+
cwd's newest thread (`recency_at_ms`); a fossil is silent.
|
|
155
|
+
|
|
156
|
+
### Recovery — `iapeer goal <peer> <resume|clear|pause>`
|
|
157
|
+
|
|
158
|
+
A notice states a fact the peer cannot state. For `peer-goal-stalled` the owner then hits a second
|
|
159
|
+
wall: **the peer cannot fix it either.** Codex splits goal control, and the model's half has no
|
|
160
|
+
exit — `update_goal` may only mark complete|blocked ("pause, resume, budget-limited, and
|
|
161
|
+
usage-limited status changes are controlled by the user or system"), and `create_goal` is refused
|
|
162
|
+
while a goal is unfinished. An agent peer whose goal blocks can only lie (mark an unachieved
|
|
163
|
+
objective `complete`, which its own prompt forbids) or restart its session and lose the thread.
|
|
164
|
+
|
|
165
|
+
The transition out exists — it is human-only. The TUI's `/goal [<objective>|clear|edit|pause|
|
|
166
|
+
resume]` calls `thread/goal/set`, an unconditional upsert that sets Active from any status, and
|
|
167
|
+
`thread_goal_actions.rs` treats `Paused | Blocked | UsageLimited` as resumable. The daemon is the
|
|
168
|
+
peer's keyboard, so the verb presses it:
|
|
169
|
+
|
|
170
|
+
| sub | effect |
|
|
171
|
+
|---|---|
|
|
172
|
+
| `resume` | the SAME goal, `blocked` → `active`; codex resumes injecting its own continuation turns |
|
|
173
|
+
| `clear` | the goal is removed ⇒ the peer's OWN `create_goal` stops being refused — the supported route to a *new* objective, authored by the agent rather than by us |
|
|
174
|
+
| `pause` | operator park |
|
|
175
|
+
|
|
176
|
+
`/goal <objective>` is deliberately NOT offered: replacing an unfinished goal opens a "Replace
|
|
177
|
+
goal?" modal, and blind-typing past a modal is how the wrong option gets picked. Subcommands are a
|
|
178
|
+
closed whitelist — caller text is never interpolated into a keystroke stream. A claude peer returns
|
|
179
|
+
`unsupported` (it has no thread goals; `/goal` would land as literal text in its composer).
|
|
180
|
+
|
|
181
|
+
**Success is gated on codex's goal store, never on keystrokes being accepted** — a pty write proves
|
|
182
|
+
the bytes left us, not that the TUI acted (§the delivery contour learned this the hard way). The
|
|
183
|
+
verb reads the goal row before and after and reports the transition it can see; a session that took
|
|
184
|
+
the text but never acted returns `unchanged`, not success. iapeer only ever READS codex's sqlite.
|
|
185
|
+
|
|
186
|
+
Verified live 16.07.2026 on a real codex peer: `blocked → active` in ~1 s, same `goal_id`, same
|
|
187
|
+
thread, no self-fresh, and auto-continuation genuinely resumed.
|
|
188
|
+
|
|
127
189
|
## 3. What the owner is told — and what is deliberately withheld
|
|
128
190
|
|
|
129
191
|
A notice states who, which runtime, which error type, which model, and when the wall lifts.
|
|
@@ -291,14 +353,20 @@ there is nothing to derive from. Reading the file's own statement works for both
|
|
|
291
353
|
- **claude — proven live.** The transcript signal, the missing reset, the whole path
|
|
292
354
|
(detector → board → fleet surface) were exercised against a genuinely exhausted fable
|
|
293
355
|
bucket on 15.07.2026. Tests replay the captured line byte-for-byte.
|
|
294
|
-
- **codex —
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
356
|
+
- **codex — the 15.07 replay acceptance was REFUTED by the first real incident, and the
|
|
357
|
+
detector was rebuilt on the real bytes (17.07.2026).** The original detector keyed on
|
|
358
|
+
`rate_limit_reached_type` non-null — a field whose reached VALUE was synthesized at
|
|
359
|
+
acceptance (honestly flagged here at the time). When the quota genuinely ran out on
|
|
360
|
+
17.07.2026 the field stayed `null` through the entire incident and the detector missed
|
|
361
|
+
both mute peers; the owner found out by asking. The current death-fingerprint path (§2)
|
|
362
|
+
is built from and tested against the incident's verbatim bytes (four independent dying
|
|
363
|
+
sessions + a negative transient + the resume-replay trap), and proven live on the real
|
|
364
|
+
mute fleet. The reached path is retained as forward-compat only.
|
|
365
|
+
- **The blocking-window heuristic (reached path only) is unverified.** Codex does not say
|
|
366
|
+
WHICH window blocked; that path reports the **fullest** one. The fingerprint path has no
|
|
367
|
+
such ambiguity in practice: the real incident carried exactly one exhausted window, and
|
|
368
|
+
with several the LATEST reset is reported (blocked until the last wall lifts). Windows
|
|
369
|
+
ride in `content`, so a human sees the raw fact either way.
|
|
302
370
|
- **`peer-goal-stalled` — the state read is proven on real data; the CAUSE of one instance is
|
|
303
371
|
not.** The detector was run against the live goal store on 16.07.2026 and returned exactly
|
|
304
372
|
the real stall (`zapret2-oneclick`, `blocked`, goal `c25d2378…`, 09:54:32, attributed via the
|
|
@@ -316,9 +384,12 @@ Forensic evidence for all of the above: `docs/internals/forensics/model-limit-20
|
|
|
316
384
|
|
|
317
385
|
## 8. Known hole: codex generic API errors — measured, not assumed
|
|
318
386
|
|
|
319
|
-
**A codex peer left mute by
|
|
320
|
-
|
|
321
|
-
|
|
387
|
+
**A codex peer left mute by a cause that emits NO rate_limits snapshot is not detected.**
|
|
388
|
+
The usage-limit class — the actual incident class — IS covered, by the §2 death fingerprint
|
|
389
|
+
(the refused call's own snapshot; `rate_limit_reached_type` proved inert on the real
|
|
390
|
+
incident and is forward-compat only). Mute from a cause whose refusal leaves no
|
|
391
|
+
`token_count` at all (this section's fault-injected 429 on a custom provider left none) is
|
|
392
|
+
not.
|
|
322
393
|
|
|
323
394
|
This is a property of codex, not an omission here. Measured on 15.07.2026:
|
|
324
395
|
|
|
@@ -339,10 +410,15 @@ This is a property of codex, not an omission here. Measured on 15.07.2026:
|
|
|
339
410
|
So codex's only structural error surface is the `rate_limits` snapshot this detector already
|
|
340
411
|
reads. The claude/codex asymmetry in §2 is that fact, not a shortcut.
|
|
341
412
|
|
|
342
|
-
**Rejected, deliberately:** `task_complete` with `last_agent_message: null` is
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
413
|
+
**Rejected, deliberately:** `task_complete` with `last_agent_message: null` **alone** is an
|
|
414
|
+
**absence, not a signal**. A detector on it by itself would fire on every turn that
|
|
415
|
+
legitimately produced no agent message; that is not built, and the 17.07 rebuild did not
|
|
416
|
+
change it. What the real incident ADDED (this probe could not see it — the custom provider
|
|
417
|
+
emitted no `token_count` at all) is that the real ChatGPT provider's refused call leaves a
|
|
418
|
+
POSITIVE artifact: the null-window, no-new-usage `token_count` snapshot. The §2 conjunction
|
|
419
|
+
keys on that artifact and uses the message-less `task_complete` only to scope it to a turn
|
|
420
|
+
that actually died. A mute from a cause that emits NO snapshot (as in this probe) remains
|
|
421
|
+
undetected — this hole is narrowed, not closed.
|
|
346
422
|
|
|
347
423
|
**Not answered:** whether `stream_error` — a real serde variant of codex's `EventMsg`, next to
|
|
348
424
|
`deprecation_notice` and `patch_apply_begin` — is persisted to the rollout. Codex filters
|
|
@@ -95,7 +95,7 @@
|
|
|
95
95
|
```
|
|
96
96
|
|
|
97
97
|
- **`peer-profile.json`** — карточка пира и единственный источник истины о нём (поля разобраны в [03 — Пиры, рантаймы, идентичность](03-пиры-рантаймы-идентичность.md)).
|
|
98
|
-
- **`launch.env`** — файл вида `KEY=VALUE` для конкретной пары пир+рантайм: дополнительные аргументы запуска (`PEER_START_ARGS`) и переменные окружения. Так одному пиру можно задать особые флаги запуска, не трогая остальных.
|
|
98
|
+
- **`launch.env`** — файл вида `KEY=VALUE` для конкретной пары пир+рантайм: дополнительные аргументы запуска (`PEER_START_ARGS`) и переменные окружения. Так одному пиру можно задать особые флаги запуска, не трогая остальных. `create`/`init` скаффолдят его как скелет из одних комментариев для каждого агентского рантайм-скоупа (до правки ведёт себя ровно как отсутствие файла), поэтому массовые операции над launch.env не пропускают пира из-за отсутствия файла; существующий файл никогда не перезаписывается.
|
|
99
99
|
|
|
100
100
|
Рабочая папка пира не обязана лежать в `~/.iapeer/peers/`. По умолчанию `iapeer create` кладёт её туда, но пиром можно сделать любую папку командой `iapeer init` — тогда `.iapeer/` появится прямо в ней.
|
|
101
101
|
|
package/package.json
CHANGED
package/src/cli/index.ts
CHANGED
|
@@ -1392,7 +1392,7 @@ const VERBS: ReadonlyArray<{ sig: string; desc: string }> = [
|
|
|
1392
1392
|
},
|
|
1393
1393
|
{
|
|
1394
1394
|
sig: 'default-runtime <runtime> (--peer <p> | --all)',
|
|
1395
|
-
desc: '
|
|
1395
|
+
desc: 'switch/set the PRIMARY runtime of peer(s) (routing/wake default) — the fleet-switch lever; refuses an undeclared runtime; registry self-healed in the same command',
|
|
1396
1396
|
},
|
|
1397
1397
|
{ sig: 'attach <peer> [runtime]', desc: 'ensure-live + resume, then attach to the pty session (Ctrl-] detaches)' },
|
|
1398
1398
|
{ sig: 'interrupt <peer> [runtime]', desc: 'interrupt the current turn (Escape) — context intact' },
|
package/src/daemon/mutewatch.ts
CHANGED
|
@@ -24,12 +24,40 @@
|
|
|
24
24
|
// notice omits the reset rather than extrapolating from the 5h/7d buckets (a DIFFERENT
|
|
25
25
|
// limit: measured live at 5h 11% / 7d 66% while fable was fully exhausted).
|
|
26
26
|
//
|
|
27
|
-
// • codex → the rollout (~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl)
|
|
28
|
-
//
|
|
29
|
-
//
|
|
30
|
-
//
|
|
31
|
-
//
|
|
32
|
-
//
|
|
27
|
+
// • codex → the rollout (~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl). TWO detections:
|
|
28
|
+
//
|
|
29
|
+
// (a) `rate_limit_reached_type` non-null in a token_count's RateLimitSnapshot. Kept for
|
|
30
|
+
// forward-compat, but REFUTED as the primary signal by the real incident of
|
|
31
|
+
// 17.07.2026: with the account's codex quota genuinely exhausted (both codex peers
|
|
32
|
+
// mute, turns dying on delivery), every snapshot in every rollout still said
|
|
33
|
+
// `rate_limit_reached_type: null` — the field simply does not fire on this class.
|
|
34
|
+
// The 15.07 acceptance replayed real bytes but SYNTHESIZED this field (docs/19 §7
|
|
35
|
+
// said so); the synthesis diverged from reality and the detector missed the real
|
|
36
|
+
// incident. We still key on NON-NULL (never the variant strings) in case a future
|
|
37
|
+
// codex starts populating it.
|
|
38
|
+
//
|
|
39
|
+
// (b) The REAL fingerprint of a limit-death, read out of the incident's own bytes
|
|
40
|
+
// (doc, linus, zapret2-oneclick and a minimal `codex exec` probe, 17.07.2026 —
|
|
41
|
+
// fixtures in docs/internals/forensics/model-limit-2026-07-17/): the rollout has NO
|
|
42
|
+
// error event at all; the refused API call instead emits a token_count whose
|
|
43
|
+
// rate_limits snapshot has BOTH windows null (healthy snapshots carry a windowed
|
|
44
|
+
// primary), and whose `info` recorded no new usage (null in a fresh session, or
|
|
45
|
+
// cumulative totals identical to the previous token_count — the call consumed
|
|
46
|
+
// nothing because it was refused), immediately followed by `task_complete` with
|
|
47
|
+
// `last_agent_message: null` (the turn died). All three conjuncts are required:
|
|
48
|
+
// – a lone null-window snapshot happens transiently and the turn SURVIVES
|
|
49
|
+
// (observed 10.07.2026: null windows but totals advanced, turn continued);
|
|
50
|
+
// – `task_complete{last_agent_message:null}` alone is an absence, not a signal
|
|
51
|
+
// (docs/19 §8 rejected it deliberately — legitimate no-message turns exist).
|
|
52
|
+
// The event time is `completed_at` (epoch seconds), NOT the line timestamp: a
|
|
53
|
+
// resumed session REPLAYS history with fresh line timestamps but keeps the original
|
|
54
|
+
// completed_at, so a replayed old death does not re-raise after recovery.
|
|
55
|
+
// errorType is `usage_limit_exceeded` — codex's own error-code vocabulary (read out
|
|
56
|
+
// of the 0.144.1 binary) — when the tail holds an exhausted window (used_percent
|
|
57
|
+
// ≥ 100, the evidence this is the usage-limit class); `api-refusal` (our fallback,
|
|
58
|
+
// mirroring claude's `api-error`) when it does not. resets_at comes from that
|
|
59
|
+
// exhausted window — the refusal snapshot itself carries none. The model comes from
|
|
60
|
+
// the dead turn's own turn_context (real bytes name it, e.g. `gpt-5.6-sol`).
|
|
33
61
|
//
|
|
34
62
|
// PATHS ARE NOT DERIVED FROM cwd — but NOT for the reason first claimed here. The original note
|
|
35
63
|
// said this host carries BOTH `-Users-macmini-Projects-IAPeer` and `-Users-macmini-Projects-iapeer`
|
|
@@ -156,9 +184,17 @@ function describeWindow(name: string, w: CodexWindow): string | null {
|
|
|
156
184
|
return `${name} ${w.used_percent}% (${win}${reset})`
|
|
157
185
|
}
|
|
158
186
|
|
|
187
|
+
/** A window object out of a rate_limits snapshot, or null when absent/malformed. */
|
|
188
|
+
function asWindow(w: unknown): CodexWindow | null {
|
|
189
|
+
return w && typeof w === 'object' ? (w as CodexWindow) : null
|
|
190
|
+
}
|
|
191
|
+
|
|
159
192
|
/**
|
|
160
|
-
* The codex rollout's newest limit-
|
|
161
|
-
*
|
|
193
|
+
* The codex rollout's newest limit-death evidence newer than `sinceMs`, or null. Two paths
|
|
194
|
+
* (see the header): (a) `rate_limit_reached_type` NON-NULL — kept forward-compat, refuted as
|
|
195
|
+
* the primary signal by the 17.07.2026 incident; (b) the real-bytes fingerprint — a refused
|
|
196
|
+
* API call's token_count (both windows null, no new usage) as the dying turn's last snapshot,
|
|
197
|
+
* closed by `task_complete{last_agent_message:null}`.
|
|
162
198
|
* `cwd` comes from session_meta, which is the file's FIRST line — pass the file HEAD in
|
|
163
199
|
* `head` when the tail alone may have cut it away.
|
|
164
200
|
*/
|
|
@@ -184,34 +220,121 @@ export function parseCodexRollout(text: string, sinceMs: number, head = ''): Raw
|
|
|
184
220
|
if (!cwd) return null // unattributable to a peer
|
|
185
221
|
|
|
186
222
|
let found: RawDetection | null = null
|
|
223
|
+
// Path-(b) rolling state, in file order (= time order):
|
|
224
|
+
/** Latest snapshot that still HAD a window — the exhausted-window/resets evidence. */
|
|
225
|
+
let lastWindowed: { primary: CodexWindow | null; secondary: CodexWindow | null; planType?: string } | null = null
|
|
226
|
+
/** Previous token_count's CUMULATIVE total — the refused-call (no new usage) discriminator. */
|
|
227
|
+
let prevTotal: number | null = null
|
|
228
|
+
/** True while the current turn's last token_count is refusal-shaped. */
|
|
229
|
+
let refusalPending = false
|
|
230
|
+
let lastModel: string | undefined
|
|
231
|
+
const modelByTurn = new Map<string, string>()
|
|
232
|
+
|
|
187
233
|
for (const line of text.split('\n')) {
|
|
188
234
|
const s = line.trim()
|
|
189
|
-
if (!s.startsWith('{')
|
|
190
|
-
|
|
235
|
+
if (!s.startsWith('{')) continue
|
|
236
|
+
if (!s.includes('rate_limits') && !s.includes('turn_context') && !s.includes('task_started') && !s.includes('task_complete')) continue
|
|
237
|
+
let d: {
|
|
238
|
+
timestamp?: unknown
|
|
239
|
+
type?: unknown
|
|
240
|
+
payload?: {
|
|
241
|
+
type?: unknown
|
|
242
|
+
turn_id?: unknown
|
|
243
|
+
model?: unknown
|
|
244
|
+
last_agent_message?: unknown
|
|
245
|
+
completed_at?: unknown
|
|
246
|
+
info?: { total_token_usage?: { total_tokens?: unknown } } | null
|
|
247
|
+
rate_limits?: Record<string, unknown>
|
|
248
|
+
}
|
|
249
|
+
}
|
|
191
250
|
try {
|
|
192
251
|
d = JSON.parse(s)
|
|
193
252
|
} catch {
|
|
194
253
|
continue
|
|
195
254
|
}
|
|
196
|
-
const
|
|
255
|
+
const p = d.payload
|
|
256
|
+
if (!p) continue
|
|
257
|
+
|
|
258
|
+
// The dead turn's own turn_context names the model (real bytes: "gpt-5.6-sol").
|
|
259
|
+
if (d.type === 'turn_context') {
|
|
260
|
+
if (typeof p.model === 'string' && p.model) {
|
|
261
|
+
lastModel = p.model
|
|
262
|
+
if (typeof p.turn_id === 'string') modelByTurn.set(p.turn_id, p.model)
|
|
263
|
+
}
|
|
264
|
+
continue
|
|
265
|
+
}
|
|
266
|
+
|
|
267
|
+
if (p.type === 'task_started') {
|
|
268
|
+
refusalPending = false // a refusal snapshot never crosses turn boundaries
|
|
269
|
+
continue
|
|
270
|
+
}
|
|
271
|
+
|
|
272
|
+
if (p.type === 'task_complete') {
|
|
273
|
+
// `=== null` (strict): the field is PRESENT and null in every real death; a turn that
|
|
274
|
+
// produced a message carries the message. The conjunction with refusalPending is what
|
|
275
|
+
// keeps docs/19 §8 honest — null alone is an absence, not a signal.
|
|
276
|
+
const died = refusalPending && p.last_agent_message === null
|
|
277
|
+
refusalPending = false
|
|
278
|
+
if (!died) continue
|
|
279
|
+
// Event time = completed_at (epoch s), NOT the line timestamp: a resumed session
|
|
280
|
+
// REPLAYS history with fresh line timestamps but keeps the original completed_at —
|
|
281
|
+
// keying on it stops a replayed old death from re-raising after recovery.
|
|
282
|
+
const atMs = typeof p.completed_at === 'number' ? p.completed_at * 1000 : toMs(d.timestamp)
|
|
283
|
+
if (atMs === null || atMs <= sinceMs) continue
|
|
284
|
+
const exhausted = [lastWindowed?.primary, lastWindowed?.secondary].filter(
|
|
285
|
+
(w): w is CodexWindow => !!w && typeof w.used_percent === 'number' && w.used_percent >= 100,
|
|
286
|
+
)
|
|
287
|
+
const resets = exhausted
|
|
288
|
+
.map(w => (typeof w.resets_at === 'number' ? w.resets_at * 1000 : null))
|
|
289
|
+
.filter((n): n is number => n !== null)
|
|
290
|
+
// usage_limit_exceeded = codex's own error-code vocabulary (0.144.1 binary), applied
|
|
291
|
+
// only when an exhausted window evidences the class; else our honest fallback.
|
|
292
|
+
const errorType = exhausted.length ? 'usage_limit_exceeded' : 'api-refusal'
|
|
293
|
+
const parts = lastWindowed
|
|
294
|
+
? [describeWindow('primary', lastWindowed.primary ?? {}), describeWindow('secondary', lastWindowed.secondary ?? {})].filter(Boolean)
|
|
295
|
+
: []
|
|
296
|
+
const plan = lastWindowed?.planType ? ` plan=${lastWindowed.planType}` : ''
|
|
297
|
+
found = {
|
|
298
|
+
errorType,
|
|
299
|
+
model: (typeof p.turn_id === 'string' ? modelByTurn.get(p.turn_id) : undefined) ?? lastModel,
|
|
300
|
+
content: `Codex turn died on a refused API call (${errorType})${plan}${parts.length ? ` — ${parts.join('; ')}` : ''}`,
|
|
301
|
+
resetsAtMs: resets.length ? Math.max(...resets) : undefined,
|
|
302
|
+
sessionId,
|
|
303
|
+
atMs,
|
|
304
|
+
cwd,
|
|
305
|
+
}
|
|
306
|
+
continue
|
|
307
|
+
}
|
|
308
|
+
|
|
309
|
+
const rl = p.rate_limits
|
|
197
310
|
if (!rl) continue
|
|
311
|
+
const primary = asWindow(rl.primary)
|
|
312
|
+
const secondary = asWindow(rl.secondary)
|
|
313
|
+
if (primary || secondary) lastWindowed = { primary, secondary, planType: typeof rl.plan_type === 'string' ? rl.plan_type : undefined }
|
|
314
|
+
// Did this snapshot's call consume anything? info:null (fresh session, refused outright)
|
|
315
|
+
// or cumulative totals identical to the previous token_count = NO — the refused-call
|
|
316
|
+
// discriminator. Unknown previous (tail cut) counts as usage: conservative, no guess.
|
|
317
|
+
const total = typeof p.info?.total_token_usage?.total_tokens === 'number' ? p.info.total_token_usage.total_tokens : null
|
|
318
|
+
const noUsage = p.info === null ? true : prevTotal !== null && total !== null && total === prevTotal
|
|
319
|
+
if (total !== null) prevTotal = total
|
|
320
|
+
const unlimited = (rl.credits as { unlimited?: unknown } | null | undefined)?.unlimited === true
|
|
321
|
+
refusalPending = !primary && !secondary && !unlimited && noUsage
|
|
322
|
+
|
|
323
|
+
// Path (a) — forward-compat: NON-NULL reached type, never the variant strings.
|
|
198
324
|
const reached = rl.rate_limit_reached_type
|
|
199
|
-
// NON-NULL is the signal. An unknown future variant must still detect, so we never
|
|
200
|
-
// compare against the variant strings read out of the binary.
|
|
201
325
|
if (reached === null || reached === undefined || reached === '') continue
|
|
202
326
|
const atMs = toMs(d.timestamp)
|
|
203
327
|
if (atMs === null || atMs <= sinceMs) continue
|
|
204
|
-
const
|
|
205
|
-
const
|
|
328
|
+
const pWin = primary ?? {}
|
|
329
|
+
const sWin = secondary ?? {}
|
|
206
330
|
// Which window is blocking? Codex does not say — reported for the FULLEST one, the
|
|
207
|
-
// only defensible read.
|
|
208
|
-
//
|
|
209
|
-
|
|
210
|
-
const
|
|
211
|
-
const
|
|
212
|
-
const blocking = su > pu ? secondary : primary
|
|
331
|
+
// only defensible read. Both windows ride in `content`, so the human sees the raw
|
|
332
|
+
// fact even if the heuristic picked wrong.
|
|
333
|
+
const pu = typeof pWin.used_percent === 'number' ? pWin.used_percent : -1
|
|
334
|
+
const su = typeof sWin.used_percent === 'number' ? sWin.used_percent : -1
|
|
335
|
+
const blocking = su > pu ? sWin : pWin
|
|
213
336
|
const resetsAtMs = typeof blocking.resets_at === 'number' ? blocking.resets_at * 1000 : undefined
|
|
214
|
-
const parts = [describeWindow('primary',
|
|
337
|
+
const parts = [describeWindow('primary', pWin), describeWindow('secondary', sWin)].filter(Boolean)
|
|
215
338
|
const plan = typeof rl.plan_type === 'string' ? ` plan=${rl.plan_type}` : ''
|
|
216
339
|
found = {
|
|
217
340
|
errorType: String(reached),
|
|
@@ -220,7 +343,7 @@ export function parseCodexRollout(text: string, sinceMs: number, head = ''): Raw
|
|
|
220
343
|
sessionId,
|
|
221
344
|
atMs,
|
|
222
345
|
cwd,
|
|
223
|
-
|
|
346
|
+
model: lastModel,
|
|
224
347
|
}
|
|
225
348
|
}
|
|
226
349
|
return found
|
package/src/storage/index.ts
CHANGED
|
@@ -197,6 +197,10 @@ export function ensureLocalRuntimeScopes(cwd: string, runtimes: readonly Runtime
|
|
|
197
197
|
mkdirSync(root, { recursive: true, mode: DIR_MODE })
|
|
198
198
|
for (const runtime of supportedLocalRuntimeScopes(runtimes)) {
|
|
199
199
|
mkdirSync(join(root, runtime, PLUGINS_DIR, IAP_PLUGIN_DIR), { recursive: true, mode: DIR_MODE })
|
|
200
|
+
// Default launch.env skeleton — every agentic runtime scope carries the knob
|
|
201
|
+
// from birth, so fleet-wide launch.env operations (e.g. mass PEER_START_ARGS
|
|
202
|
+
// edits) never skip a peer for lack of the file. Existing files are untouched.
|
|
203
|
+
ensureLaunchEnv(cwd, runtime)
|
|
200
204
|
}
|
|
201
205
|
}
|
|
202
206
|
|
|
@@ -207,11 +211,45 @@ export function ensureLocalRuntimeScopes(cwd: string, runtimes: readonly Runtime
|
|
|
207
211
|
export const LAUNCH_ENV_FILE = 'launch.env'
|
|
208
212
|
|
|
209
213
|
/** `<cwd>/.iapeer/runtimes/<runtime>/launch.env` — the per-peer per-runtime launch
|
|
210
|
-
* fragment (PEER_START_ARGS + extra peer env).
|
|
214
|
+
* fragment (PEER_START_ARGS + extra peer env). Scaffolded as an all-comment
|
|
215
|
+
* skeleton by create/init (ensureLocalRuntimeScopes) for every agentic runtime
|
|
216
|
+
* scope; the operator uncomments/edits it. Read at launch; missing → empty. */
|
|
211
217
|
export function peerLaunchEnvPath(cwd: string, runtime: Runtime): string {
|
|
212
218
|
return join(cwd, IAPEER_DIR, RUNTIMES_DIR, runtime, LAUNCH_ENV_FILE)
|
|
213
219
|
}
|
|
214
220
|
|
|
221
|
+
/** The default launch.env skeleton. ALL lines are comments — deliberately: it must
|
|
222
|
+
* parse to exactly the missing-file semantics ({startArgs: [], env: {}}, and
|
|
223
|
+
* crucially NO PEER_DISALLOWED_TOOLS key, whose present-empty ≠ absent is the
|
|
224
|
+
* opt-in gate). No trailing comments on the example lines — the parser keeps
|
|
225
|
+
* everything after `=` as the value, so `KEY=x # note` would poison the value. */
|
|
226
|
+
export const LAUNCH_ENV_TEMPLATE = `# launch.env — optional per-peer per-runtime launch overrides (iapeer, docs/06-storage).
|
|
227
|
+
# Format: one KEY=VALUE per line (bash-style; an \`export \` prefix is allowed).
|
|
228
|
+
# Blank lines and lines starting with # are ignored. Do NOT put a trailing
|
|
229
|
+
# comment after a value — everything after \`=\` is taken verbatim as the value.
|
|
230
|
+
# This skeleton is all comments: it behaves exactly like no file at all.
|
|
231
|
+
#
|
|
232
|
+
# Extra CLI args appended after the runtime's base argv (whitespace-split):
|
|
233
|
+
#PEER_START_ARGS=""
|
|
234
|
+
#
|
|
235
|
+
# claude only — --disallowedTools override. Key absent = adapter default for the
|
|
236
|
+
# whole fleet; present-empty ("") = omit the flag; a value = pass it verbatim:
|
|
237
|
+
#PEER_DISALLOWED_TOOLS=""
|
|
238
|
+
#
|
|
239
|
+
# Any other KEY=VALUE becomes an env var of the peer's session:
|
|
240
|
+
#MY_VAR=value
|
|
241
|
+
`
|
|
242
|
+
|
|
243
|
+
/** Create the default launch.env skeleton if (and only if) the file does not
|
|
244
|
+
* exist yet — an existing operator-edited file is NEVER touched. 0600 like the
|
|
245
|
+
* rest of the tree (an operator may later put secrets in it). */
|
|
246
|
+
export function ensureLaunchEnv(cwd: string, runtime: Runtime): void {
|
|
247
|
+
const path = peerLaunchEnvPath(cwd, runtime)
|
|
248
|
+
if (existsSync(path)) return
|
|
249
|
+
mkdirSync(dirname(path), { recursive: true, mode: DIR_MODE })
|
|
250
|
+
writeFileAtomic(path, LAUNCH_ENV_TEMPLATE)
|
|
251
|
+
}
|
|
252
|
+
|
|
215
253
|
export interface LaunchEnv {
|
|
216
254
|
/** Tokens from PEER_START_ARGS, appended AFTER the adapter's base argv flags. */
|
|
217
255
|
startArgs: string[]
|
|
@@ -232,9 +270,10 @@ export interface LaunchEnv {
|
|
|
232
270
|
* reproduced here with a whitespace split (faithful to that semantics; a value
|
|
233
271
|
* with embedded spaces was never one arg in the bash either). Surrounding single/
|
|
234
272
|
* double quotes around a value are stripped. Lines that are blank or start with
|
|
235
|
-
* `#` are ignored. A missing/unreadable file → empty (no flags, no env)
|
|
236
|
-
*
|
|
237
|
-
*
|
|
273
|
+
* `#` are ignored. A missing/unreadable file → empty (no flags, no env); the
|
|
274
|
+
* scaffolded all-comment skeleton (LAUNCH_ENV_TEMPLATE) parses to the same. This
|
|
275
|
+
* is a deliberately MINIMAL parser (assignments only — no command substitution, no
|
|
276
|
+
* conditionals); the format is a controlled convention, not arbitrary shell.
|
|
238
277
|
*/
|
|
239
278
|
export function readLaunchEnv(cwd: string, runtime: Runtime): LaunchEnv {
|
|
240
279
|
let text: string
|