@astrosheep/pi-context 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +12 -15
  2. package/package.json +1 -1
  3. package/src/index.ts +124 -98
package/README.md CHANGED
@@ -19,40 +19,37 @@ pi -e npm:@astrosheep/pi-context
19
19
  The extension composes Pi's public `session_before_compact` / `session_compact` hooks, custom session entries, and the `context` hook to approximate Codex's experimental context management:
20
20
 
21
21
  - **`new_context` tool** — the model requests a fresh context window. The extension waits for the current tool turn to end, compacts with a short deterministic reset message (old conversation is excluded from the new provider context but stays in the session), then sends exactly one hidden continuation turn.
22
- - **`<context_window>` boot block** — the head of every fresh window. For a reset it IS the summary returned from `session_before_compact` (position 0, persisted, no extra message); for the root window `session_start` persists it once as a visible custom message. It carries the agent name and first/current/previous window IDs, the recent-notes index, and a `<context_window_protocol>` teaching block. The notes index is a window-open snapshot with the same frozen-at-write semantics as the reminder count. Nothing is injected transiently per request: the boot block is static once-per-window content, so the head of the window stays cache-stable. Codex diverges here — its `<context_window>` block carries only the agent path and window IDs, while the notes index is our own addition.
22
+ - **`<context_window>` boot block** — the head of every fresh window. For a reset it IS the summary returned from `session_before_compact` (position 0, persisted, no extra message); for the root window `session_start` persists it once as a visible custom message. It carries the agent name and first/current/previous window IDs, the recent-notes index, and a `<context_window_protocol>` teaching block. The notes index lists up to three most-recent notes, each with its `X lines, Y UTF-8 bytes` metadata plus a local-time ISO 8601 `updated` timestamp (explicit UTC offset, never `Z`) and an inline preview. A note of 200 Unicode characters or fewer is shown whole; a longer one shows its first 120 and last 80 Unicode characters joined by an ellipsis, so the two ends never overlap and no text is repeated. The notes index is a window-open snapshot with the same frozen-at-write semantics as the reminder count. Nothing is injected transiently per request: the boot block is static once-per-window content, so the head of the window stays cache-stable. Codex diverges here — its `<context_window>` block carries only the agent path and window IDs, while the notes index is our own addition.
23
23
  - **Low-budget guidance** — when estimated remaining context first drops to the reminder threshold (by default **40,960 tokens**: Pi's default 16,384 `reserveTokens` plus a 24,576 reminder margin; see [Reminder timing](#reminder-timing)), a `<context_window_guidance>` reminder is **persisted once per window** into history (TUI-visible, no extra turn; `sendMessage` safely defers mid-stream). There is deliberately no transient copy: a bridge would make the model meet the same text twice at shifted positions, because history records the persisted copy after the crossing request's assistant reply. The reminder is an early warning, so arriving from the next request on costs nothing and keeps the model's view identical to recorded history. The measured remaining count is frozen into the text at the threshold crossing, so the persisted reminder is a snapshot true at write time; `get_context_remaining` remains the live source for the current figure. The text is appended rather than prepended; existing history is not rewritten.
24
- - **Direct automatic reset** — every automatic compaction immediately uses the same reset handler. No cancellation to obtain a fallback turn, no input interception or replay, and no special idle/streaming scheduling. The reminder asks the model to write notes early; if it misses that opportunity, old history remains searchable. Pi owns automatic continuation, queued inputs, and overflow retry.
25
- - **Graceful fallback** — when estimated remaining context reaches the fallback threshold (by default **24,576 tokens**: Pi's default 16,384 `reserveTokens` plus an 8,192 fallback margin; see [Reminder timing](#reminder-timing)), the extension inserts one final note-taking instruction once per window. Before a fresh user turn, it is persisted through `before_agent_start`; after a running tool turn (only while the agent is still streaming), it is sent at the ordinary `turn_end` boundary with `triggerTurn: true`, which Pi routes to `agent.steer()`: the message is drained after the turn end and injected before the next LLM call, extending the current run by one note-taking turn while a queued user prompt (follow-up) waits until the agent would stop. It never copies, handles, or replays user input and never cancels Pi's compaction. Pi's automatic compaction still performs the reset afterward.
24
+ - **Two-phase automatic fallback** — while Pi is streaming, the first automatic `threshold`/`overflow` crossing of the reserve line does not reset immediately. `session_before_compact` queues the final note-taking instruction with `pi.sendMessage(..., { triggerTurn: true })` — which Pi routes to `agent.steer()` while streaming, so the message is queued synchronously and reaches the model before any pending user input — and returns `{ cancel: true }`. Pi records that as an aborted compaction, spends no summary, and continues the same run with the borrowed turn; no user text or images are copied, intercepted, or replayed, and no `input` handler is registered. Once the borrowed run has finished, `agent_end` arms the real reset and `agent_settled` requests it through `ctx.compact()` for both `threshold` and `overflow`, unless another compaction has already completed. A threshold crossing does not guarantee another automatic check: the agent loop can end without preparing another assistant response. Overflow recovery also has a one-shot guard. Requesting the reset after the run settles avoids issuing it from `agent_end` while Pi is still finishing the run. A phase flag makes the cancel happen at most once per window — it re-arms only after a completed reset starts a fresh window — and a failed reset keeps retrying the real compaction instead of borrowing another turn. The borrow is skipped when the crossing arrives while Pi is idle (the pre-prompt check in `AgentSession.prompt()`, where `triggerTurn` would start a nested run and make the pending `Agent.prompt()` reject); that crossing resets directly. `manual` `/compact` and `new_context` never take the borrowed-turn path. The reminder asks the model to write notes early; if it misses that opportunity, old history remains searchable.
26
25
  - **Runtime toggle** — `/pi-context off` disables the boot block, guidance, and reset-style compaction (Pi's default compaction, including `keepRecentTokens`, applies again). `/pi-context on` re-enables; a bare `/pi-context` reports the current state.
27
26
  - **History tools** — the model searches pre-reset conversation with case-sensitive literal substring search, exactly like Codex's `history.*` namespace.
28
27
  - **Notes tools** — persistent, session-scoped virtual files that survive window resets.
29
28
 
30
29
  ## Reminder timing
31
30
 
32
- Reminder and fallback thresholds derive from Pi's compaction reserve plus margins configured in `settings.json` under the top-level `pi-context` key:
31
+ The reminder threshold derives from Pi's compaction reserve plus a margin configured in `settings.json` under the top-level `pi-context` key:
33
32
 
34
33
  ```json
35
34
  {
36
35
  "compaction": { "reserveTokens": 16384 },
37
36
  "pi-context": {
38
- "reminderMarginTokens": 24576,
39
- "fallbackMarginTokens": 8192
37
+ "reminderMarginTokens": 24576
40
38
  }
41
39
  }
42
40
  ```
43
41
 
44
- Both margins are measured in **remaining context tokens** added on top of Pi's `compaction.reserveTokens`:
42
+ The margin is measured in **remaining context tokens** added on top of Pi's `compaction.reserveTokens`:
45
43
 
46
- - `fallback = reserveTokens + fallbackMarginTokens` (default margin `8192`)
47
44
  - `reminder = reserveTokens + reminderMarginTokens` (default margin `24576`)
48
45
 
49
- Put the key in the global settings (`~/.pi/agent/settings.json`) or the project settings (`<cwd>/.pi/settings.json`); project values win per key, mirroring Pi's own settings merge. With Pi's default `reserveTokens: 16384` the defaults give reminder `40960` and fallback `24576`: the fallback sits one note-taking turn above Pi's reset line, and the reminder leaves another 16,384 tokens of working room above the fallback, no matter how you set `reserveTokens`.
46
+ Put the key in the global settings (`~/.pi/agent/settings.json`) or the project settings (`<cwd>/.pi/settings.json`); project values win per key, mirroring Pi's own settings merge. With Pi's default `reserveTokens: 16384` the default gives reminder `40960`, leaving 24,576 tokens of working room above Pi's reset line no matter how you set `reserveTokens`. The borrowed fallback turn needs no threshold of its own: it is driven by Pi's automatic `threshold`/`overflow` compaction request, so `before_agent_start`/`turn_end` send nothing and no `fallbackMarginTokens` setting exists.
50
47
 
51
- Pi's `reserveTokens` and the `pi-context` margins are re-read from disk at every `session_start` and cached for that session. Invalid values — a margin that is not a positive integer, or a `reminderMarginTokens` that does not clear `fallbackMarginTokens` are ignored per offending key with one TUI warning naming the key and the default used instead; session handling never throws. If `fallbackMarginTokens` still leaves the reminder below the fallback after the reminder's default is applied, that key degrades too, with its own warning.
48
+ Pi's `reserveTokens` and the `pi-context` reminder margin are re-read from disk at every `session_start` and cached for that session. An invalid margin not a positive integer — is ignored with one TUI warning naming the key and the default used instead; session handling never throws.
52
49
 
53
50
  This is a file-backed read through Pi's public `SettingsManager`. It sees committed `settings.json` only: SDK-level ephemeral `applyOverrides()` calls and the unreleased `compaction.modelOverrides` are not seen by this extension.
54
51
 
55
- The extension does not change Pi settings or reserve additional context. Large tool outputs or user inputs can jump over one or both reminders; overflow recovery still resets immediately rather than forcing a doomed extra turn. For small context windows, tune the margins (or Pi's reserve) to fit the model.
52
+ The extension does not change Pi settings or reserve additional context. Large tool outputs or user inputs can jump over the reminder. For small context windows, tune the reminder margin (or Pi's reserve) to fit the model.
56
53
 
57
54
  ## Tools
58
55
 
@@ -70,9 +67,9 @@ The nine Codex History/Notes actions are flattened because Pi tools have one glo
70
67
  | `notes.append_to_file` | `notes_append_to_file` |
71
68
  | `notes.write_file` | `notes_write_file` |
72
69
 
73
- `history_*` reads the current branch's actual Pi session entries, including entries hidden by earlier compaction. Window IDs are extension-owned: the root window is `pcw:<session-id>:root`, and each reset mints an 8-hex id baked into the compaction entry's `details.windowId` as `pcw:<session-id>:<minted>`. Pi-native compactions (extension toggled off) fall back to `pcw:<session-id>:<compaction-entry-id>`. Item IDs are the persisted Pi entry IDs. No transcript copy or volatile archive is maintained.
70
+ `history_*` reads the current branch's actual Pi session entries, including entries hidden by earlier compaction. Window IDs are extension-owned: the root window is `pcw:<session-id>:root`, and each reset mints an 8-hex id baked into the compaction entry's `details.windowId` as `pcw:<session-id>:<minted>`. Pi-native compactions (extension toggled off) fall back to `pcw:<session-id>:<compaction-entry-id>`. Item IDs are the persisted Pi entry IDs. No transcript copy or volatile archive is maintained. Ordering is newest-first by default: `recent_first` omitted or `true` returns newest-first, and only an explicit `false` returns oldest-first; `history_list_windows`, `history_list_items`, and `history_search_contents` share that switch.
74
71
 
75
- `notes_*` stores operation entries in the same append-only Pi session under `pi-context/note`. They are session-scoped, survive JSONL reload, never enter provider context, and use safe relative virtual paths only (no absolute paths, `..`, `.`, empty components, or backslashes). Searches are literal and case-sensitive. `notes_read_file` accepts inclusive 1-based line ranges; negative line numbers count from the last line. Writes are capped at 1,000,000 UTF-8 bytes.
72
+ `notes_*` stores operation entries in the same append-only Pi session under `pi-context/note`. They are session-scoped, survive JSONL reload, never enter provider context, and use safe relative virtual paths only (no absolute paths, `..`, `.`, empty components, or backslashes). Searches are literal and case-sensitive. `notes_read_file` accepts inclusive 1-based line ranges; negative line numbers count from the last line. `notes_read_file` success results and every matched file object from `notes_search_contents` also carry `created_at` and `updated_at`, the same fields `notes_list_files_by_prefix` returns. All note timestamps are local-time ISO 8601 strings with an explicit UTC offset (for example `2026-09-15T17:31:45.392+08:00`; a UTC host renders `+00:00`, never `Z`), while the persisted `NoteFile`/`NoteOperation` metadata keeps plain epoch milliseconds. Error results carry no timestamps. Writes are capped at 1,000,000 UTF-8 bytes.
76
73
 
77
74
  Unlike Codex, the history tools do not advertise `agent_name`: Pi has no cross-agent session routing, so the parameter is omitted from the schemas entirely (strict `additionalProperties: false` still rejects it) instead of costing schema tokens on every request.
78
75
 
@@ -85,7 +82,7 @@ Two extra controls compose Pi public APIs:
85
82
 
86
83
  On `session_before_compact`, the extension appends a persistent custom reset marker through public `pi.appendEntry`, reads that real marker ID from the readonly session manager, and returns it as `firstKeptEntryId`. Pi's `buildContextEntries()` then keeps the compaction envelope plus that custom marker; custom markers are excluded from LLM context. Thus the subsequent provider context contains the boot block summary and the marker, not old conversation messages. The boot summary also carries the extension-minted window id in its `details.windowId`, so the fresh window names itself with an id Pi could not have supplied at bake time. Only an explicit `new_context` adds a hidden continuation; automatic resets and user `/compact` keep Pi's native scheduling. The old entries remain only in the session tree for `history_*`.
87
84
 
88
- Pi's built-in “compacted into the following summary” envelope is left intact. Its content explicitly says: “Context window reset. No summary was generated. Retrieve prior details through history_* and notes_*.” No context filtering or TUI override is used to hide that envelope.
85
+ Pi's built-in “compacted into the following summary” envelope is left intact. Its content explicitly says: “Context window reset: this is a fresh window. The previous conversation is not included and no summary was generated. Notes and durable session history persist across windows.” No context filtering or TUI override is used to hide that envelope.
89
86
 
90
87
  The same handler is used for native automatic compaction. When Pi marks an overflow compaction `willRetry`, Pi core performs its single retry itself and this extension deliberately sends no second continuation. While the extension is enabled, its custom reset keeps nothing after the boundary marker, so Pi's `keepRecentTokens` setting has no effect; with `/pi-context off`, Pi's default compaction (and `keepRecentTokens`) applies again.
91
88
 
@@ -98,4 +95,4 @@ npm run typecheck
98
95
  npm test
99
96
  ```
100
97
 
101
- The integration harness uses the installed Pi `SessionManager` and `SettingsManager` (global and project settings fixtures in temp directories, so the real `~/.pi` is never touched), including an on-disk JSONL reload. It verifies note persistence/Unicode/path rules, the boot block contents and extension-minted window ids on reset and root paths, that the `context` hook never injects, provider context exclusion after the real `firstKeptEntryId` boundary while history remains searchable, completed tool-result boundary placement, settings-derived threshold resolution and margin validation, one early reminder and one final fallback per window, one continuation only, and cancellation/failure/no-double-retry behavior. It uses no model or network call.
98
+ The integration harness uses the installed Pi `SessionManager` and `SettingsManager` (global and project settings fixtures in temp directories, so the real `~/.pi` is never touched), including an on-disk JSONL reload. It verifies note persistence/Unicode/path rules, the boot block contents and extension-minted window ids on reset and root paths, that the `context` hook never injects, provider context exclusion after the real `firstKeptEntryId` boundary while history remains searchable, completed tool-result boundary placement, settings-derived reminder threshold resolution and margin validation, that the removed `before_agent_start`/`turn_end` fallback is gone, one early reminder per window, one continuation only, the two-phase automatic ordering (one cancel, one steer, one real reset, re-armed per window, `ctx.compact()` for both threshold and overflow, idle crossings and manual/`new_context` resets bypassing the borrow), and cancellation/failure/no-double-retry behavior. It uses no model or network call.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@astrosheep/pi-context",
3
- "version": "0.8.0",
3
+ "version": "0.9.0",
4
4
  "type": "module",
5
5
  "description": "Codex-style context windows for Pi: reset-style compaction, durable session history tools, and persistent notes.",
6
6
  "license": "MIT",
package/src/index.ts CHANGED
@@ -21,8 +21,11 @@ const GUIDANCE_CLOSE_TAG = "</context_window_guidance>";
21
21
  const PI_CONTEXT_SETTINGS_KEY = "pi-context";
22
22
  const DEFAULT_RESERVE_TOKENS = 16_384;
23
23
  const DEFAULT_REMINDER_MARGIN_TOKENS = 24_576;
24
- const DEFAULT_FALLBACK_MARGIN_TOKENS = 8_192;
25
- const RESET_SUMMARY = "Context window reset. No summary was generated. Retrieve prior details through history_* and notes_*.";
24
+ const RESET_SUMMARY =
25
+ "Context window reset: this is a fresh window. The previous conversation is not included and no summary was generated. Notes and durable session history persist across windows.";
26
+ const NOTE_PREVIEW_HEAD_CHARS = 120;
27
+ const NOTE_PREVIEW_TAIL_CHARS = 80;
28
+ const NOTE_PREVIEW_CHARS = NOTE_PREVIEW_HEAD_CHARS + NOTE_PREVIEW_TAIL_CHARS;
26
29
  const CONTINUATION = "This is a fresh context window. Recover only the details needed to continue with history_* and notes_*; then continue the task.";
27
30
 
28
31
  /**
@@ -43,8 +46,8 @@ ${CONTEXT_WINDOW_PROTOCOL_CLOSE_TAG}`;
43
46
  const FALLBACK_PROMPT =
44
47
  "Context budget is almost exhausted. This is the final fallback turn before the window resets automatically. Write task state, decisions, open issues, and next steps with notes_write_file now. Do not start new work; old conversation remains searchable through history_*.";
45
48
 
46
- type ResolvedThresholds = { reminder: number; fallback: number };
47
- type PiContextMargins = { reminderMarginTokens: unknown; fallbackMarginTokens: unknown };
49
+ type ResolvedThresholds = { reminder: number };
50
+ type PiContextMargins = { reminderMarginTokens: unknown };
48
51
 
49
52
  type NoteFile = { text: string; createdAt: number; updatedAt: number };
50
53
  type NoteOperation = {
@@ -196,7 +199,7 @@ function filteredItems(ctx: ExtensionContext, params: HistoryFilter): HistoryIte
196
199
  if (typeof params.role === "string") items = items.filter((item) => item.role === params.role);
197
200
  if (typeof params.tool_namespace === "string") items = items.filter((item) => item.toolNamespace === params.tool_namespace);
198
201
  if (typeof params.tool_name === "string") items = items.filter((item) => item.toolName === params.tool_name);
199
- if (params.recent_first === true) items.reverse();
202
+ if (params.recent_first !== false) items.reverse();
200
203
  return items;
201
204
  }
202
205
 
@@ -256,15 +259,29 @@ function identityBlock(agentName: string, firstWindowId: string, currentWindowId
256
259
  return `${CONTEXT_WINDOW_OPEN_TAG}\n${lines.join("\n")}\n${CONTEXT_WINDOW_CLOSE_TAG}`;
257
260
  }
258
261
 
259
- /** Recent-notes index with the existing wording; empty when the session has no notes. */
262
+ /**
263
+ * Recent-notes index: up to three most-recent notes. Each note shows its path, line count,
264
+ * UTF-8 byte count and local ISO update time, followed by an indented inline preview: the
265
+ * whole text when it fits in NOTE_PREVIEW_CHARS, otherwise its first NOTE_PREVIEW_HEAD_CHARS
266
+ * and last NOTE_PREVIEW_TAIL_CHARS Unicode characters joined by an explicit ellipsis. The
267
+ * two slices never overlap, so the preview never duplicates head content as tail content.
268
+ * Empty when the session has no notes.
269
+ */
260
270
  function notesIndex(ctx: ExtensionContext): string {
261
271
  const recentNotes = [...notesFromSession(ctx)]
262
272
  .sort((a, b) => b[1].updatedAt - a[1].updatedAt)
263
- .slice(0, 5);
273
+ .slice(0, 3);
264
274
  if (recentNotes.length === 0) return "";
265
- const lines = ["Recent notes (up to 5, most-recent first):"];
275
+ const lines = ["Recent notes at window open (up to 3, most-recent first):"];
266
276
  for (const [path, file] of recentNotes) {
267
- lines.push(`- ${path} (${file.text.split("\n").length} lines, ${Buffer.byteLength(file.text, "utf8")} UTF-8 bytes)`);
277
+ lines.push(`- ${path} (${file.text.split("\n").length} lines, ${Buffer.byteLength(file.text, "utf8")} UTF-8 bytes, updated ${localIso(file.updatedAt)})`);
278
+ const chars = Array.from(file.text);
279
+ // Short notes stay whole; long notes keep both ends. head + tail <= NOTE_PREVIEW_CHARS < chars.length,
280
+ // so the slices are disjoint and no character is shown twice.
281
+ const preview = chars.length <= NOTE_PREVIEW_CHARS
282
+ ? file.text
283
+ : `${chars.slice(0, NOTE_PREVIEW_HEAD_CHARS).join("")}…${chars.slice(chars.length - NOTE_PREVIEW_TAIL_CHARS).join("")}`;
284
+ lines.push(preview.split("\n").map((line) => ` ${line}`).join("\n"));
268
285
  }
269
286
  return lines.join("\n");
270
287
  }
@@ -319,9 +336,26 @@ function lineRange(text: string, startValue: unknown, stopValue: unknown) {
319
336
  return { start_line: start, stop_line: stop, content: start > stop ? "" : lines.slice(start - 1, stop).join("\n") };
320
337
  }
321
338
 
339
+ const pad2 = (value: number) => String(value).padStart(2, "0");
340
+
341
+ /**
342
+ * Format epoch milliseconds as an ISO 8601 string in the host's local time zone with an
343
+ * explicit numeric offset (e.g. 2026-09-15T17:31:45.392+08:00). A UTC host renders
344
+ * "+00:00"; the "Z" designator is never used, and Date.parse round-trips the value.
345
+ */
346
+ function localIso(epochMs: number): string {
347
+ const date = new Date(epochMs);
348
+ const offsetMinutes = -date.getTimezoneOffset();
349
+ const absOffset = Math.abs(offsetMinutes);
350
+ const offset = `${offsetMinutes < 0 ? "-" : "+"}${pad2(Math.floor(absOffset / 60))}:${pad2(absOffset % 60)}`;
351
+ const wallClock = `${date.getFullYear()}-${pad2(date.getMonth() + 1)}-${pad2(date.getDate())}T${pad2(date.getHours())}:${pad2(date.getMinutes())}:${pad2(date.getSeconds())}.${String(date.getMilliseconds()).padStart(3, "0")}`;
352
+ return `${wallClock}${offset}`;
353
+ }
354
+
322
355
  const nullableString = () => Type.Optional(Type.Union([Type.String(), Type.Null()]));
323
356
  const nullableInteger = () => Type.Optional(Type.Union([Type.Integer(), Type.Null()]));
324
357
  const positiveInteger = () => Type.Optional(Type.Integer({ minimum: 1 }));
358
+ const recentFirst = () => Type.Optional(Type.Boolean({ description: "Return newest-first. Only an explicit false returns oldest-first. Defaults to true." }));
325
359
  const role = Type.Union([Type.Literal("user"), Type.Literal("assistant"), Type.Literal("tool"), Type.Literal("system"), Type.Literal("developer"), Type.Null()]);
326
360
 
327
361
  function isSettingsObject(value: unknown): value is Record<string, unknown> {
@@ -338,7 +372,7 @@ function piContextSettings(settings: unknown): Record<string, unknown> {
338
372
  /** Merge the global and project "pi-context" objects per key; project wins, mirroring Pi's deep merge. */
339
373
  export function mergePiContextSettings(globalSettings: unknown, projectSettings: unknown): PiContextMargins {
340
374
  const merged = { ...piContextSettings(globalSettings), ...piContextSettings(projectSettings) };
341
- return { reminderMarginTokens: merged.reminderMarginTokens, fallbackMarginTokens: merged.fallbackMarginTokens };
375
+ return { reminderMarginTokens: merged.reminderMarginTokens };
342
376
  }
343
377
 
344
378
  /** A margin is usable only as a positive integer; anything else is ignored. */
@@ -348,49 +382,37 @@ function validMargin(raw: unknown): number | undefined {
348
382
  }
349
383
 
350
384
  /**
351
- * Pure derivation of the effective thresholds from Pi's reserve plus the pi-context
352
- * margins. Invalid margins and a reminder that does not clear the fallback degrade
353
- * to defaults per offending key and report one warning each.
385
+ * Pure derivation of the reminder threshold from Pi's reserve plus the pi-context
386
+ * reminder margin. An invalid margin degrades to the default and reports one warning.
387
+ * The borrowed fallback turn has no token threshold of its own: it is driven by Pi's
388
+ * automatic threshold/overflow compaction request (see session_before_compact).
354
389
  */
355
390
  export function deriveThresholds(reserveTokens: number, margins: PiContextMargins): { thresholds: ResolvedThresholds; warnings: string[] } {
356
391
  const warnings: string[] = [];
357
392
  const reminderKey = `${PI_CONTEXT_SETTINGS_KEY}.reminderMarginTokens`;
358
- const fallbackKey = `${PI_CONTEXT_SETTINGS_KEY}.fallbackMarginTokens`;
359
- const parsedReminder = validMargin(margins.reminderMarginTokens);
360
- const parsedFallback = validMargin(margins.fallbackMarginTokens);
361
-
362
393
  let reminderMargin: number;
363
394
  if (margins.reminderMarginTokens === undefined) reminderMargin = DEFAULT_REMINDER_MARGIN_TOKENS;
364
- else if (parsedReminder === undefined) {
365
- warnings.push(`pi-context: ${reminderKey} must be a positive integer; using default ${DEFAULT_REMINDER_MARGIN_TOKENS}.`);
366
- reminderMargin = DEFAULT_REMINDER_MARGIN_TOKENS;
367
- } else reminderMargin = parsedReminder;
368
-
369
- let fallbackMargin: number;
370
- if (margins.fallbackMarginTokens === undefined) fallbackMargin = DEFAULT_FALLBACK_MARGIN_TOKENS;
371
- else if (parsedFallback === undefined) {
372
- warnings.push(`pi-context: ${fallbackKey} must be a positive integer; using default ${DEFAULT_FALLBACK_MARGIN_TOKENS}.`);
373
- fallbackMargin = DEFAULT_FALLBACK_MARGIN_TOKENS;
374
- } else fallbackMargin = parsedFallback;
375
-
376
- if (reminderMargin <= fallbackMargin) {
377
- warnings.push(`pi-context: ${reminderKey} must exceed ${fallbackKey}; using default ${DEFAULT_REMINDER_MARGIN_TOKENS}.`);
378
- reminderMargin = DEFAULT_REMINDER_MARGIN_TOKENS;
379
- if (reminderMargin <= fallbackMargin) {
380
- warnings.push(`pi-context: ${fallbackKey} must be below ${reminderKey}; using default ${DEFAULT_FALLBACK_MARGIN_TOKENS}.`);
381
- fallbackMargin = DEFAULT_FALLBACK_MARGIN_TOKENS;
382
- }
395
+ else {
396
+ const parsed = validMargin(margins.reminderMarginTokens);
397
+ if (parsed === undefined) {
398
+ warnings.push(`pi-context: ${reminderKey} must be a positive integer; using default ${DEFAULT_REMINDER_MARGIN_TOKENS}.`);
399
+ reminderMargin = DEFAULT_REMINDER_MARGIN_TOKENS;
400
+ } else reminderMargin = parsed;
383
401
  }
384
-
385
- return { thresholds: { reminder: reserveTokens + reminderMargin, fallback: reserveTokens + fallbackMargin }, warnings };
402
+ return { thresholds: { reminder: reserveTokens + reminderMargin }, warnings };
386
403
  }
387
404
 
388
405
  export default function piContext(pi: ExtensionAPI) {
389
406
  let rollover: "idle" | "requested" | "compacting" = "idle";
390
407
  let enabled = true;
391
408
  let guidancePersistedInWindow: string | undefined;
392
- let fallbackPersistedInWindow: string | undefined;
393
409
  let handledCompactionId: string | undefined;
410
+ // Two-phase main-line fallback. The first automatic threshold/overflow compaction
411
+ // borrows one final note-taking turn (cancel + steer) instead of resetting at once;
412
+ // the next compaction request is allowed through. The phase only returns to "idle"
413
+ // after a completed reset, so the cancel happens at most once per window and a failed
414
+ // reset retries the real compaction instead of borrowing another turn.
415
+ let fallbackPhase: "idle" | "steered" | "allow" | "requested" = "idle";
394
416
  let thresholds: ResolvedThresholds | undefined;
395
417
 
396
418
  /**
@@ -411,10 +433,7 @@ export default function piContext(pi: ExtensionAPI) {
411
433
  thresholds = derived.thresholds;
412
434
  } catch (error) {
413
435
  ctx.ui.notify(`pi-context: could not read settings; using defaults (${String(error)}).`, "warning");
414
- thresholds = {
415
- reminder: DEFAULT_RESERVE_TOKENS + DEFAULT_REMINDER_MARGIN_TOKENS,
416
- fallback: DEFAULT_RESERVE_TOKENS + DEFAULT_FALLBACK_MARGIN_TOKENS,
417
- };
436
+ thresholds = { reminder: DEFAULT_RESERVE_TOKENS + DEFAULT_REMINDER_MARGIN_TOKENS };
418
437
  }
419
438
  return thresholds;
420
439
  };
@@ -458,10 +477,10 @@ export default function piContext(pi: ExtensionAPI) {
458
477
  name: "history_list_windows",
459
478
  label: "History list windows",
460
479
  description: "List durable Pi session-history windows.",
461
- parameters: Type.Object({ limit: positiveInteger(), recent_first: Type.Optional(Type.Boolean()) }, { additionalProperties: false }),
480
+ parameters: Type.Object({ limit: positiveInteger(), recent_first: recentFirst() }, { additionalProperties: false }),
462
481
  async execute(_id, params, _signal, _update, ctx) {
463
482
  let windows = historyFromSession(ctx);
464
- if (params.recent_first) windows = [...windows].reverse();
483
+ if (params.recent_first !== false) windows = [...windows].reverse();
465
484
  const limit = params.limit ?? windows.length;
466
485
  return output({ windows: windows.slice(0, limit).map((window) => ({ window_id: window.windowId, item_count: window.items.length })) });
467
486
  },
@@ -471,7 +490,7 @@ export default function piContext(pi: ExtensionAPI) {
471
490
  name: "history_list_items",
472
491
  label: "History list items",
473
492
  description: "List durable session items, including items before compaction, using opaque item and window IDs.",
474
- parameters: Type.Object({ limit: positiveInteger(), recent_first: Type.Optional(Type.Boolean()), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString(), max_chars_per_item: positiveInteger() }, { additionalProperties: false }),
493
+ parameters: Type.Object({ limit: positiveInteger(), recent_first: recentFirst(), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString(), max_chars_per_item: positiveInteger() }, { additionalProperties: false }),
475
494
  async execute(_id, params, _signal, _update, ctx) {
476
495
  const items = filteredItems(ctx, params);
477
496
  return output({ items: items.slice(0, params.limit ?? items.length).map((item) => visibleItem(item, params.max_chars_per_item ?? 1200)) });
@@ -497,7 +516,7 @@ export default function piContext(pi: ExtensionAPI) {
497
516
  name: "history_search_contents",
498
517
  label: "History search",
499
518
  description: "Case-sensitive literal substring search over durable Pi session history; no semantic search.",
500
- parameters: Type.Object({ limit: positiveInteger(), query: Type.String(), recent_first: Type.Optional(Type.Boolean()), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString() }, { additionalProperties: false }),
519
+ parameters: Type.Object({ limit: positiveInteger(), query: Type.String(), recent_first: recentFirst(), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString() }, { additionalProperties: false }),
501
520
  async execute(_id, params, _signal, _update, ctx) {
502
521
  const items = filteredItems(ctx, params);
503
522
  const matching = items.filter((item) => item.content.includes(params.query));
@@ -508,7 +527,7 @@ export default function piContext(pi: ExtensionAPI) {
508
527
  pi.registerTool(defineTool({
509
528
  name: "notes_list_files_by_prefix",
510
529
  label: "Notes list files",
511
- description: "List persistent, session-scoped virtual note files.",
530
+ description: "List persistent, session-scoped virtual note files. created_at and updated_at are local-time ISO 8601 strings with an explicit UTC offset.",
512
531
  parameters: Type.Object({ prefix: nullableString(), max_results: positiveInteger(), file_order_by: Type.Optional(Type.Union([Type.Literal("name"), Type.Literal("created_at"), Type.Literal("updated_at")])), file_order: Type.Optional(Type.Union([Type.Literal("ascending"), Type.Literal("descending")])) }, { additionalProperties: false }),
513
532
  async execute(_id, params, _signal, _update, ctx) {
514
533
  const prefix = assertVirtualPrefix(params.prefix);
@@ -516,34 +535,34 @@ export default function piContext(pi: ExtensionAPI) {
516
535
  const key = params.file_order_by ?? "name";
517
536
  files.sort(([aPath, a], [bPath, b]) => key === "name" ? aPath.localeCompare(bPath) : (key === "created_at" ? a.createdAt - b.createdAt : a.updatedAt - b.updatedAt));
518
537
  if (params.file_order === "descending") files.reverse();
519
- return output({ files: files.slice(0, params.max_results ?? files.length).map(([path, file]) => ({ path, size_bytes: Buffer.byteLength(file.text, "utf8"), created_at: file.createdAt, updated_at: file.updatedAt })) });
538
+ return output({ files: files.slice(0, params.max_results ?? files.length).map(([path, file]) => ({ path, size_bytes: Buffer.byteLength(file.text, "utf8"), created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt) })) });
520
539
  },
521
540
  }));
522
541
 
523
542
  pi.registerTool(defineTool({
524
543
  name: "notes_read_file",
525
544
  label: "Notes read file",
526
- description: "Read a virtual note file, optionally by inclusive 1-based line range; negative lines count from the end.",
545
+ description: "Read a virtual note file, optionally by inclusive 1-based line range; negative lines count from the end. Success results carry created_at and updated_at as local-time ISO 8601 strings with an explicit UTC offset.",
527
546
  parameters: Type.Object({ path: Type.String(), start_line: nullableInteger(), stop_line: nullableInteger() }, { additionalProperties: false }),
528
547
  async execute(_id, params, _signal, _update, ctx) {
529
548
  const path = assertVirtualPath(params.path);
530
549
  const file = notesFromSession(ctx).get(path);
531
550
  if (!file) return output({ error: "note file not found", path });
532
- return output({ path, ...lineRange(file.text, params.start_line, params.stop_line) });
551
+ return output({ path, ...lineRange(file.text, params.start_line, params.stop_line), created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt) });
533
552
  },
534
553
  }));
535
554
 
536
555
  pi.registerTool(defineTool({
537
556
  name: "notes_search_contents",
538
557
  label: "Notes search",
539
- description: "Case-sensitive literal substring search over virtual note lines; no semantic search.",
558
+ description: "Case-sensitive literal substring search over virtual note lines; no semantic search. Each matched file carries created_at and updated_at as local-time ISO 8601 strings with an explicit UTC offset.",
540
559
  parameters: Type.Object({ max_matches_per_file: positiveInteger(), query: Type.String(), recent_file_first: Type.Optional(Type.Boolean()), max_files: positiveInteger(), path_prefix: nullableString() }, { additionalProperties: false }),
541
560
  async execute(_id, params, _signal, _update, ctx) {
542
561
  const prefix = assertVirtualPrefix(params.path_prefix);
543
562
  let files = [...notesFromSession(ctx)].filter(([path]) => !prefix || path.startsWith(prefix));
544
563
  if (params.recent_file_first) files.sort((a, b) => b[1].createdAt - a[1].createdAt);
545
564
  const maxPerFile = params.max_matches_per_file ?? Number.POSITIVE_INFINITY;
546
- const result = files.map(([path, file]) => ({ path, matches: file.text.split("\n").flatMap((line, index) => line.includes(params.query) ? [{ line: index + 1, text: line }] : []).slice(0, maxPerFile) })).filter((file) => file.matches.length > 0);
565
+ const result = files.map(([path, file]) => ({ path, created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt), matches: file.text.split("\n").flatMap((line, index) => line.includes(params.query) ? [{ line: index + 1, text: line }] : []).slice(0, maxPerFile) })).filter((file) => file.matches.length > 0);
547
566
  return output({ files: result.slice(0, params.max_files ?? result.length) });
548
567
  },
549
568
  }));
@@ -594,47 +613,6 @@ export default function piContext(pi: ExtensionAPI) {
594
613
  return undefined;
595
614
  });
596
615
 
597
- // Graceful fallback without intercepting user input: before a fresh prompt, if
598
- // remaining context has entered the buffer between this threshold and Pi's
599
- // reserve line, append a persistent user-level final-call instruction. Pi then
600
- // runs that turn with the user's queued prompt still present and runs its own
601
- // automatic compaction before the following prompt. Overflow is excluded: Pi
602
- // already owns its one-shot compact-and-retry recovery.
603
- pi.on("before_agent_start", (event, ctx) => {
604
- if (!enabled) return undefined;
605
- const usage = ctx.getContextUsage();
606
- if (!usage || usage.tokens === null) return undefined;
607
- const remaining = Math.max(0, usage.contextWindow - usage.tokens);
608
- if (remaining > resolveThresholds(ctx).fallback) return undefined;
609
- const windowId = currentWindowId(ctx);
610
- if (fallbackPersistedInWindow === windowId) return undefined;
611
- fallbackPersistedInWindow = windowId;
612
- return { message: { customType: FALLBACK_TYPE, content: FALLBACK_PROMPT, display: true } };
613
- });
614
-
615
- pi.on("turn_end", (_event, ctx) => {
616
- if (!enabled) return undefined;
617
- // Streaming case only: while the agent is streaming, triggerTurn:true routes
618
- // to agent.steer() — Pi drains the steering queue after this turn_end and
619
- // injects the message before the next LLM call, extending the current run by
620
- // one note-taking turn. A queued user prompt (follow-up) drains only when the
621
- // agent would stop, so it is processed after the notes turn. (Defensive: in
622
- // v0.85.1 turn_end always fires inside an active run, so isIdle is never
623
- // true here; the idle pre-prompt case is owned by before_agent_start above.)
624
- if (ctx.isIdle()) return undefined;
625
- const usage = ctx.getContextUsage();
626
- if (!usage || usage.tokens === null) return;
627
- const remaining = Math.max(0, usage.contextWindow - usage.tokens);
628
- if (remaining > resolveThresholds(ctx).fallback) return;
629
- const windowId = currentWindowId(ctx);
630
- if (fallbackPersistedInWindow === windowId) return;
631
- if (rollover !== "idle") return;
632
- fallbackPersistedInWindow = windowId;
633
- // The steered message reaches the model before the pending user input and no
634
- // input text/images are copied or replayed.
635
- pi.sendMessage({ customType: FALLBACK_TYPE, content: fallbackGuidance(), display: true }, { triggerTurn: true });
636
- });
637
-
638
616
  pi.registerTool(defineTool({
639
617
  name: "get_context_remaining",
640
618
  label: "Get context remaining",
@@ -662,19 +640,65 @@ export default function piContext(pi: ExtensionAPI) {
662
640
  pi.on("agent_end", (_event, ctx) => {
663
641
  if (!enabled) {
664
642
  if (rollover === "requested") rollover = "idle";
643
+ fallbackPhase = "idle";
665
644
  return;
666
645
  }
646
+ // The borrowed fallback turn (if any) has just finished. Arm the allowance so the
647
+ // next compaction request performs the real reset instead of cancelling again.
648
+ if (fallbackPhase === "steered") fallbackPhase = "allow";
667
649
  if (rollover !== "requested") return;
668
650
  rollover = "compacting";
669
651
  ctx.compact({ onError: () => { if (rollover === "compacting") rollover = "idle"; } });
670
652
  });
671
653
 
654
+ // Request the reset after the borrowed run settles, unless another compaction has
655
+ // already completed. Both threshold and overflow need this path: the agent loop
656
+ // can end without another threshold check, and overflow has a one-shot recovery
657
+ // guard. Waiting for agent_settled avoids requesting this reset from agent_end
658
+ // while Pi is still finishing the active run.
659
+ pi.on("agent_settled", (_event, ctx) => {
660
+ if (!enabled) {
661
+ fallbackPhase = "idle";
662
+ return;
663
+ }
664
+ if (fallbackPhase === "steered") {
665
+ // The borrowed turn has not been observed yet. Stay armed: the next automatic
666
+ // request is still allowed through, and re-arming from idle here would let an
667
+ // undeliverable steer cancel forever.
668
+ return;
669
+ }
670
+ if (fallbackPhase !== "allow") return;
671
+ fallbackPhase = "requested";
672
+ ctx.compact({ onError: () => { /* the allowance stays armed so the real reset is retried */ } });
673
+ });
674
+
672
675
  pi.on("session_before_compact", async (event, ctx) => {
673
- if (!enabled) return undefined; // Default Pi compaction applies; keepRecentTokens is honored again.
676
+ if (!enabled) {
677
+ fallbackPhase = "idle";
678
+ return undefined; // Default Pi compaction applies; keepRecentTokens is honored again.
679
+ }
674
680
  // Never let an aborted or failed custom reset fall through to Pi's default summary.
675
681
  if (event.signal.aborted) return { cancel: true };
676
- // Every compaction uses the same reset path. Never cancel to borrow a
677
- // note-taking turn: Pi owns user input, queued work, and overflow recovery.
682
+ // Manual /compact and new_context bypass the borrowed-turn phase entirely.
683
+ const selfRequested = rollover === "requested" || rollover === "compacting";
684
+ const automatic = event.reason === "threshold" || event.reason === "overflow";
685
+ // Phase 1: the first automatic crossing of the reserve line borrows one final
686
+ // note-taking turn instead of resetting immediately. This is only safe while Pi is
687
+ // streaming: there `pi.sendMessage(..., triggerTurn)` routes to agent.steer() and is
688
+ // queued synchronously, so it reaches the model before pending user input with no
689
+ // text/images copied, intercepted, or replayed (no input hook is registered). While
690
+ // idle, the same call would instead start a nested agent run (AgentSession's
691
+ // sendCustomMessage -> _runAgentPrompt), and the prompt that triggered this idle
692
+ // pre-flight check would then fail with "Agent is already processing a prompt"
693
+ // (Agent.prompt rejects while activeRun exists). So idle crossings reset directly.
694
+ if (automatic && !selfRequested && fallbackPhase === "idle" && !ctx.isIdle()) {
695
+ fallbackPhase = "steered";
696
+ pi.sendMessage({ customType: FALLBACK_TYPE, content: fallbackGuidance(), display: true }, { triggerTurn: true });
697
+ return { cancel: true };
698
+ }
699
+ // Phase 2 (or manual/new_context): perform the real reset. fallbackPhase deliberately
700
+ // stays armed until session_compact confirms success, so a failed reset is retried
701
+ // without borrowing another turn.
678
702
  try {
679
703
  const sessionId = ctx.sessionManager.getSessionId();
680
704
  // Pi mints the compaction entry id only after this hook returns, so the
@@ -705,8 +729,10 @@ export default function piContext(pi: ExtensionAPI) {
705
729
  pi.on("session_compact", (event, ctx) => {
706
730
  if (!enabled) {
707
731
  rollover = "idle";
732
+ fallbackPhase = "idle";
708
733
  return;
709
734
  }
735
+ fallbackPhase = "idle"; // A completed reset re-arms the borrowed-turn phase for the next window.
710
736
  const entry = ctx.sessionManager.getEntry(event.compactionEntry.id);
711
737
  if (entry?.type !== "compaction" || resetV2WindowId(entry.details) === undefined) return;
712
738
  if (handledCompactionId === entry.id) return;
@@ -726,4 +752,4 @@ export default function piContext(pi: ExtensionAPI) {
726
752
  });
727
753
  }
728
754
 
729
- export const internal = { MAX_NOTE_BYTES, NOTE_TYPE, BOOT_TYPE, GUIDANCE_TYPE, FALLBACK_TYPE, FALLBACK_PROMPT, RESET_MARKER_TYPE, RESET_SUMMARY, CONTINUATION, CONTEXT_WINDOW_OPEN_TAG, CONTEXT_WINDOW_CLOSE_TAG, CONTEXT_WINDOW_PROTOCOL_OPEN_TAG, CONTEXT_WINDOW_PROTOCOL_CLOSE_TAG, GUIDANCE_OPEN_TAG, PI_CONTEXT_SETTINGS_KEY, DEFAULT_RESERVE_TOKENS, DEFAULT_REMINDER_MARGIN_TOKENS, DEFAULT_FALLBACK_MARGIN_TOKENS, deriveThresholds, mergePiContextSettings, lineRange, assertVirtualPath };
755
+ export const internal = { MAX_NOTE_BYTES, NOTE_TYPE, BOOT_TYPE, GUIDANCE_TYPE, FALLBACK_TYPE, FALLBACK_PROMPT, RESET_MARKER_TYPE, RESET_SUMMARY, CONTINUATION, CONTEXT_WINDOW_OPEN_TAG, CONTEXT_WINDOW_CLOSE_TAG, CONTEXT_WINDOW_PROTOCOL_OPEN_TAG, CONTEXT_WINDOW_PROTOCOL_CLOSE_TAG, GUIDANCE_OPEN_TAG, PI_CONTEXT_SETTINGS_KEY, DEFAULT_RESERVE_TOKENS, DEFAULT_REMINDER_MARGIN_TOKENS, deriveThresholds, mergePiContextSettings, lineRange, assertVirtualPath };