@astrosheep/pi-context 0.10.1 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -19,8 +19,8 @@ pi -e npm:@astrosheep/pi-context
19
19
  The extension composes Pi's public `session_before_compact` / `session_compact` hooks, custom session entries, and the `context` hook to approximate Codex's experimental context management:
20
20
 
21
21
  - **`new_context` tool** — the model requests a fresh context window. The extension waits for the current run to settle, compacts with a short deterministic reset message (old conversation is excluded from the new provider context but stays in the session), then sends one hidden continuation turn once compaction has fully completed, unless another prompt is already queued or running.
22
- - **`<context_window>` boot block** — the head of every fresh window. For a reset it IS the summary returned from `session_before_compact` (position 0, persisted, no extra message); for the root window `session_start` persists it once as a visible custom message. It carries the agent name and first/current/previous window IDs, the recent-notes index, and a `<context_window_protocol>` teaching block. The notes index lists up to three most-recent notes, each with its `X lines, Y UTF-8 bytes` metadata plus a local-time ISO 8601 `updated` timestamp (explicit UTC offset, never `Z`) and an inline preview. A note of 200 Unicode characters or fewer is shown whole; a longer one shows its first 120 and last 80 Unicode characters joined by an ellipsis, so the two ends never overlap and no text is repeated. The notes index is a window-open snapshot with the same frozen-at-write semantics as the reminder count. Nothing is injected transiently per request: the boot block is static once-per-window content, so the head of the window stays cache-stable. Codex diverges here — its `<context_window>` block carries only the agent path and window IDs, while the notes index is our own addition.
23
- - **Low-budget guidance** — when estimated remaining context first drops to the reminder threshold (by default **40,960 tokens**: Pi's default 16,384 `reserveTokens` plus a 24,576 reminder margin; see [Reminder timing](#reminder-timing)), a `<context_window_guidance>` reminder is **persisted once per window** into history (TUI-visible, no extra turn; `sendMessage` safely defers mid-stream). There is deliberately no transient copy: a bridge would make the model meet the same text twice at shifted positions, because history records the persisted copy after the crossing request's assistant reply. The reminder is an early warning, so arriving from the next request on costs nothing and keeps the model's view identical to recorded history. The measured remaining budget excludes `reserveTokens` and is frozen into the text at the threshold crossing, so the persisted reminder is a snapshot true at write time; `get_context_remaining` remains the live source for the current figure. The text is appended rather than prepended; existing history is not rewritten.
22
+ - **`<context_window>` boot block** — the head of every fresh window. For a reset it IS the summary returned from `session_before_compact` (position 0, persisted, no extra message); for the root window `session_start` persists it once as a hidden custom message (`display: false`, kept out of the TUI). It carries the agent name and first/current/previous window IDs, the recent-notes index, and a `<context_window_protocol>` teaching block. The notes index lists up to three most-recent notes, each with its `X lines, Y UTF-8 bytes` metadata plus a local-time ISO 8601 `updated` timestamp (explicit UTC offset, never `Z`) and an inline preview. A note of 320 Unicode characters or fewer is shown whole; a longer one shows its first 80 and last 240 Unicode characters joined by an ellipsis, so the two ends never overlap and no text is repeated. The notes index is a window-open snapshot with the same frozen-at-write semantics as the reminder count. Nothing is injected transiently per request: the boot block is static once-per-window content, so the head of the window stays cache-stable. Codex diverges here — its `<context_window>` block carries only the agent path and window IDs, while the notes index is our own addition.
23
+ - **Low-budget guidance** — when estimated remaining context first drops to the reminder threshold (by default **40,960 tokens**: Pi's default 16,384 `reserveTokens` plus a 24,576 reminder margin; see [Reminder timing](#reminder-timing)), a `<context_window_guidance>` reminder is **persisted once per window** into history (hidden from the TUI transcript — one ephemeral `ui.notify` warning tells the user instead, and the model never sees it; no extra turn; `sendMessage` safely defers mid-stream). There is deliberately no transient copy: a bridge would make the model meet the same text twice at shifted positions, because history records the persisted copy after the crossing request's assistant reply. The reminder is an early warning, so arriving from the next request on costs nothing and keeps the model's view identical to recorded history. The measured remaining budget excludes `reserveTokens` and is frozen into the text at the threshold crossing, so the persisted reminder is a snapshot true at write time; `get_context_remaining` remains the live source for the current figure. The text is appended rather than prepended; existing history is not rewritten.
24
24
  - **Two-phase automatic fallback** — while Pi is streaming, the first automatic `threshold`/`overflow` crossing of the reserve line does not reset immediately. `session_before_compact` queues the final note-taking instruction with `pi.sendMessage(..., { triggerTurn: true })` — which Pi routes to `agent.steer()` while streaming, so the message is queued synchronously and reaches the model before any pending user input — and returns `{ cancel: true }`. Pi records that as an aborted compaction, spends no summary, and continues the same run with the borrowed turn; no user text or images are copied, intercepted, or replayed, and no `input` handler is registered. Once the borrowed run has finished, `agent_end` arms the real reset and `agent_settled` requests it through `ctx.compact()` for both `threshold` and `overflow`, unless another compaction has already completed. Pi can perform another automatic check after a run, and overflow recovery has a one-shot guard. The settled scheduler ensures the borrowed turn has one reset owner even when no native check resets it. Requesting the reset after the run settles avoids issuing it from `agent_end` while Pi is still finishing the run. A phase flag makes the cancel happen at most once per window — it re-arms only after a completed reset starts a fresh window. A failed reset clears the pending request, warns the user, and leaves history intact; another prompt can retry without borrowing another turn. Extension-requested fallback resets send a continuation after the compaction completion callback. The borrow is skipped when the crossing arrives while Pi is idle (the pre-prompt check in `AgentSession.prompt()`, where `triggerTurn` would start a nested run and make the pending `Agent.prompt()` reject); that crossing resets directly. `manual` `/compact` and `new_context` never take the borrowed-turn path. The reminder asks the model to write notes early; if it misses that opportunity, old history remains searchable.
25
25
  - **Runtime toggle** — `/pi-context off` disables the boot block, guidance, and reset-style compaction (Pi's default compaction, including `keepRecentTokens`, applies again). `/pi-context on` re-enables; a bare `/pi-context` reports the current state.
26
26
  - **History tools** — the model searches pre-reset conversation with case-sensitive literal substring search, exactly like Codex's `history.*` namespace.
@@ -69,7 +69,9 @@ The nine Codex History/Notes actions are flattened because Pi tools have one glo
69
69
 
70
70
  `history_*` reads the current branch's actual Pi session entries, including entries hidden by earlier compaction. Window IDs are extension-owned: the root window is `pcw:<session-id>:root`, and each reset mints an 8-hex id baked into the compaction entry's `details.windowId` as `pcw:<session-id>:<minted>`. Pi-native compactions (extension toggled off) fall back to `pcw:<session-id>:<compaction-entry-id>`. Item IDs are the persisted Pi entry IDs. No transcript copy or volatile archive is maintained. Ordering is newest-first by default: `recent_first` omitted or `true` returns newest-first, and only an explicit `false` returns oldest-first; `history_list_windows`, `history_list_items`, and `history_search_contents` share that switch.
71
71
 
72
- `notes_*` stores operation entries in the same append-only Pi session under `pi-context/note`. They are session-scoped, survive JSONL reload, never enter provider context, and use safe relative virtual paths only (no absolute paths, `..`, `.`, empty components, or backslashes). Searches are literal and case-sensitive. `notes_read_file` accepts inclusive 1-based line ranges; negative line numbers count from the last line. `notes_read_file` success results and every matched file object from `notes_search_contents` also carry `created_at` and `updated_at`, the same fields `notes_list_files_by_prefix` returns. All note timestamps are local-time ISO 8601 strings with an explicit UTC offset (for example `2026-09-15T17:31:45.392+08:00`; a UTC host renders `+00:00`, never `Z`), while the persisted `NoteFile`/`NoteOperation` metadata keeps plain epoch milliseconds. Error results carry no timestamps. Writes are capped at 1,000,000 UTF-8 bytes.
72
+ `history_*` and paged `notes_*` read/search results share a **32 KiB (32 * 1024 UTF-8 bytes) per-result budget**. List/search tools return whole items only and include `next_offset` (an integer cursor, or `null` when exhausted); pass that cursor back with the same parameters to continue. `history_read_item` defaults to `limit_chars: 12000`, accepts at most 50000, and returns `total_chars` plus `next_offset_chars` when more content remains. `notes_read_file` returns whole lines only, includes `total_lines`, and uses `next_start_line` for a bounded continuation. When one indivisible unit is larger than the whole budget (a single note line, search match, or history item), it is returned middle-truncated — head and tail joined by a `…[truncated N chars]…` marker, mirroring Codex's `truncate_middle` — instead of being dropped, so a page is never empty and its cursor always advances. `max_chars_per_item` limits Unicode code points including the truncation ellipsis (`…`). The 32 KiB choice matches Aider's default and Claude Code's roughly 30k-character limit while keeping worst-case CJK output (about 11k tokens) below Pi's default `reserveTokens` of 16384.
73
+
74
+ `notes_*` stores operation entries in the same append-only Pi session under `pi-context/note`. They are session-scoped, survive JSONL reload, never enter provider context, and use safe relative virtual paths only (no absolute paths, `..`, `.`, empty components, or backslashes). Searches are literal and case-sensitive. `notes_read_file` accepts inclusive 1-based line ranges; negative line numbers count from the last line. Staleness is explicit and never inferred from timestamps. `notes_write_file` and `notes_append_to_file` accept an optional `mark_stale` boolean, and `notes_list_files_by_prefix` reports each file's `stale` flag. Stale notes leave the boot notes index — which omits itself once no fresh note remains — but stay listed, readable, and searchable. Each call must carry `text`, `mark_stale`, or both: `mark_stale: true` alone flags a note without touching its content, `mark_stale: false` alone clears the flag without touching content, and either call may carry `text` too — so final content can be written and marked stale in one call, and a closing log line can be appended and marked stale in one call. Carrying `text` without `mark_stale` revives a stale note, and marking a path that does not exist is an error. `notes_read_file` success results and every matched file object from `notes_search_contents` also carry `created_at` and `updated_at`, the same fields `notes_list_files_by_prefix` returns. All note timestamps are local-time ISO 8601 strings with an explicit UTC offset (for example `2026-09-15T17:31:45.392+08:00`; a UTC host renders `+00:00`, never `Z`), while the persisted `NoteFile`/`NoteOperation` metadata keeps plain epoch milliseconds. Error results carry no timestamps. Writes are capped at 1,000,000 UTF-8 bytes. `notes_write_file` and `notes_append_to_file` declare `executionMode: "sequential"` (the Pi equivalent of Codex's `supports_parallel_tool_calls = false`), so a tool batch containing either runs its calls one at a time and note read-modify-write cannot race with itself.
73
75
 
74
76
  Unlike Codex, the history tools do not advertise `agent_name`: Pi has no cross-agent session routing, so the parameter is omitted from the schemas entirely (strict `additionalProperties: false` still rejects it) instead of costing schema tokens on every request.
75
77
 
@@ -84,7 +86,7 @@ On `session_before_compact`, the extension appends a persistent custom reset mar
84
86
 
85
87
  The scheduler lives in [`src/reset-lifecycle.ts`](src/reset-lifecycle.ts); `src/index.ts` composes the features and builds reset boundaries. History and notes projections, tool adapters, budget policy, and prompt rendering have separate ownership described in [Architecture](docs/architecture.md). See the [lifecycle event table](docs/reset-lifecycle.md) for ownership and cancellation rules. Shutdown, tree navigation, and toggling off invalidate outstanding callbacks. An aborted run clears pending rollover work. Queued steering/follow-up messages may continue before the run settles; the extension does not clear or replay that queue.
86
88
 
87
- Pi's built-in “compacted into the following summary” envelope is left intact. Its content explicitly says: “Context window reset: this is a fresh window. The previous conversation is not included and no summary was generated. Notes and durable session history persist across windows.” No context filtering or TUI override is used to hide that envelope.
89
+ Pi's built-in “compacted into the following summary” envelope is left intact. Its content says, in the extension's wake-up voice: “You wake up. Your head is empty no memories, the past a blank. But nothing is lost: the notes you wrote and the recorded history still remember for you.” No context filtering or TUI override is used to hide that envelope.
88
90
 
89
91
  The same handler is used for native automatic compaction. When Pi marks an overflow compaction `willRetry`, Pi core performs its single retry itself and this extension deliberately sends no second continuation. While the extension is enabled, its custom reset keeps nothing after the boundary marker, so Pi's `keepRecentTokens` setting has no effect; with `/pi-context off`, Pi's default compaction (and `keepRecentTokens`) applies again.
90
92
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@astrosheep/pi-context",
3
- "version": "0.10.1",
3
+ "version": "0.12.0",
4
4
  "type": "module",
5
5
  "description": "Codex-style context windows for Pi: reset-style compaction, durable session history tools, and persistent notes.",
6
6
  "license": "MIT",
package/src/budget.ts CHANGED
@@ -101,7 +101,12 @@ export function registerBudget(pi: ExtensionAPI, isEnabled: () => boolean) {
101
101
  // on (sendMessage defers safely to end of turn while streaming, queueing
102
102
  // instead of splitting a tool call/result pair) costs nothing, and the
103
103
  // model's view stays identical to recorded history, Codex-style.
104
- pi.sendMessage({ customType: GUIDANCE_TYPE, content: tokenBudgetGuidance(Math.max(0, remaining - reserve)), display: true }, { triggerTurn: false });
104
+ // The persisted copy stays out of the TUI (display: false); one ephemeral
105
+ // notify tells the user instead — visible to the human, invisible to the
106
+ // model, and never recorded, so history and the model's view don't diverge.
107
+ const left = Math.max(0, remaining - reserve);
108
+ pi.sendMessage({ customType: GUIDANCE_TYPE, content: tokenBudgetGuidance(left), display: false }, { triggerTurn: false });
109
+ ctx.ui.notify(`pi-context: context budget low (${left} tokens before reserve) — checkpoint reminder recorded for the model, kept out of the chat view.`, "warning");
105
110
  }
106
111
  }
107
112
  return undefined;
@@ -1,6 +1,6 @@
1
1
  import { Type } from "@earendil-works/pi-ai";
2
2
  import { defineTool, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
3
- import { output } from "./tool-output.js";
3
+ import { output, page, middleTruncate, withinBudget } from "./tool-output.js";
4
4
  import { positiveInteger, recentFirst, nullableString, role } from "./tool-schema.js";
5
5
  import { historyFromSession, filteredItems, visibleItem, allItems } from "./history.js";
6
6
 
@@ -22,10 +22,10 @@ export function registerHistoryTools(pi: ExtensionAPI) {
22
22
  name: "history_list_items",
23
23
  label: "History list items",
24
24
  description: "List durable session items, including items before compaction, using opaque item and window IDs.",
25
- parameters: Type.Object({ limit: positiveInteger(), recent_first: recentFirst(), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString(), max_chars_per_item: positiveInteger() }, { additionalProperties: false }),
25
+ parameters: Type.Object({ limit: positiveInteger(), offset: Type.Optional(Type.Integer({ minimum: 0 })), recent_first: recentFirst(), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString(), max_chars_per_item: positiveInteger() }, { additionalProperties: false }),
26
26
  async execute(_id, params, _signal, _update, ctx) {
27
- const items = filteredItems(ctx, params);
28
- return output({ items: items.slice(0, params.limit ?? items.length).map((item) => visibleItem(item, params.max_chars_per_item ?? 1200)) });
27
+ const items = filteredItems(ctx, params).slice(0, params.limit ?? Number.POSITIVE_INFINITY).map((item) => visibleItem(item, params.max_chars_per_item ?? 1200));
28
+ return output(page(items, params.offset ?? 0, "items", undefined, (item, fits) => ({ ...item, truncated_content: middleTruncate(item.truncated_content, (candidate) => fits({ ...item, truncated_content: candidate })) })));
29
29
  },
30
30
  }));
31
31
 
@@ -33,14 +33,18 @@ export function registerHistoryTools(pi: ExtensionAPI) {
33
33
  name: "history_read_item",
34
34
  label: "History read item",
35
35
  description: "Read a bounded character range from one durable session item.",
36
- parameters: Type.Object({ item_id: Type.String(), offset_chars: Type.Optional(Type.Integer({ minimum: 0 })), limit_chars: positiveInteger(), window_id: Type.String() }, { additionalProperties: false }),
36
+ parameters: Type.Object({ item_id: Type.String(), offset_chars: Type.Optional(Type.Integer({ minimum: 0 })), limit_chars: Type.Optional(Type.Integer({ minimum: 1, maximum: 50000 })), window_id: Type.String() }, { additionalProperties: false }),
37
37
  async execute(_id, params, _signal, _update, ctx) {
38
38
  const item = allItems(ctx).find((candidate) => candidate.windowId === params.window_id && candidate.itemId === params.item_id);
39
39
  if (!item) return output({ error: "unknown item_id or window_id" });
40
40
  const chars = Array.from(item.content);
41
41
  const offset = params.offset_chars ?? 0;
42
- const limit = params.limit_chars ?? chars.length;
43
- return output({ window_id: item.windowId, item_id: item.itemId, offset_chars: offset, content: chars.slice(offset, offset + limit).join("") });
42
+ const limit = Math.min(params.limit_chars ?? 12000, 50000);
43
+ const result = (content: string) => ({ window_id: item.windowId, item_id: item.itemId, offset_chars: offset, content, total_chars: chars.length, next_offset_chars: offset + limit < chars.length ? offset + limit : null });
44
+ // One clean middle-truncation replaces the old 0.9 shrink loop: a requested window larger
45
+ // than the budget comes back with its middle elided, never empty, and the cursor advances.
46
+ const content = middleTruncate(chars.slice(offset, offset + limit).join(""), (candidate) => withinBudget(result(candidate)));
47
+ return output(result(content));
44
48
  },
45
49
  }));
46
50
 
@@ -48,11 +52,10 @@ export function registerHistoryTools(pi: ExtensionAPI) {
48
52
  name: "history_search_contents",
49
53
  label: "History search",
50
54
  description: "Case-sensitive literal substring search over durable Pi session history; no semantic search.",
51
- parameters: Type.Object({ limit: positiveInteger(), query: Type.String(), recent_first: recentFirst(), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString() }, { additionalProperties: false }),
55
+ parameters: Type.Object({ limit: positiveInteger(), offset: Type.Optional(Type.Integer({ minimum: 0 })), query: Type.String(), recent_first: recentFirst(), tool_namespace: nullableString(), role: Type.Optional(role), tool_name: nullableString(), window_id: nullableString(), max_chars_per_item: positiveInteger() }, { additionalProperties: false }),
52
56
  async execute(_id, params, _signal, _update, ctx) {
53
- const items = filteredItems(ctx, params);
54
- const matching = items.filter((item) => item.content.includes(params.query));
55
- return output({ items: matching.slice(0, params.limit ?? matching.length).map((item) => visibleItem(item)) });
57
+ const matching = filteredItems(ctx, params).filter((item) => item.content.includes(params.query)).slice(0, params.limit ?? Number.POSITIVE_INFINITY).map((item) => visibleItem(item, params.max_chars_per_item ?? 1200));
58
+ return output(page(matching, params.offset ?? 0, "items", undefined, (item, fits) => ({ ...item, truncated_content: middleTruncate(item.truncated_content, (candidate) => fits({ ...item, truncated_content: candidate })) })));
56
59
  },
57
60
  }));
58
61
 
package/src/history.ts CHANGED
@@ -123,7 +123,7 @@ export function visibleItem(item: HistoryItem, maxChars = 1200) {
123
123
  role: item.role,
124
124
  tool_namespace: item.toolNamespace ?? null,
125
125
  tool_name: item.toolName ?? null,
126
- truncated_content: characters.length > maxChars ? `${characters.slice(0, maxChars).join("")}…` : item.content,
126
+ truncated_content: characters.length > maxChars ? `${characters.slice(0, Math.max(0, maxChars - 1)).join("")}…` : item.content,
127
127
  };
128
128
  }
129
129
 
@@ -131,7 +131,7 @@ export function allItems(ctx: SessionReader) {
131
131
  return historyFromSession(ctx).flatMap((window) => window.items);
132
132
  }
133
133
 
134
- export function filteredItems(ctx: SessionReader, params: HistoryFilter): HistoryItem[] {
134
+ export function filteredItems(ctx: SessionReader, params: HistoryFilter): HistoryItem[] {
135
135
  let items = allItems(ctx);
136
136
  if (typeof params.window_id === "string") items = items.filter((item) => item.windowId === params.window_id);
137
137
  if (typeof params.role === "string") items = items.filter((item) => item.role === params.role);
package/src/index.ts CHANGED
@@ -21,12 +21,12 @@ export default function piContext(pi: ExtensionAPI) {
21
21
  pi.on("session_start", (_event, ctx) => {
22
22
  if (!enabled) return;
23
23
  // The root window has no compaction entry to carry the boot block, so persist
24
- // it once as a visible custom message. Reset windows already carry theirs at
24
+ // it once as a hidden custom message. Reset windows already carry theirs at
25
25
  // position 0 in the compaction summary, so a resumed session adds nothing.
26
26
  const sessionId = ctx.sessionManager.getSessionId();
27
27
  const rootId = `pcw:${sessionId}:root`;
28
28
  if (currentWindowId(ctx) !== rootId || hasWindowMessage(ctx, BOOT_TYPE)) return;
29
- pi.sendMessage({ customType: BOOT_TYPE, content: bootBlock(ctx, rootId, undefined, false), display: true }, { triggerTurn: false });
29
+ pi.sendMessage({ customType: BOOT_TYPE, content: bootBlock(ctx, rootId, undefined, false), display: false }, { triggerTurn: false });
30
30
  });
31
31
 
32
32
  pi.registerCommand("pi-context", {
package/src/note-tools.ts CHANGED
@@ -1,6 +1,6 @@
1
1
  import { Type } from "@earendil-works/pi-ai";
2
2
  import { defineTool, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
3
- import { output } from "./tool-output.js";
3
+ import { output, page, middleTruncate, withinBudget } from "./tool-output.js";
4
4
  import { nullableString, nullableInteger, positiveInteger } from "./tool-schema.js";
5
5
  import { notesFromSession, assertVirtualPath, assertVirtualPrefix, lineRange, localIso, type NoteOperation } from "./notes.js";
6
6
  import { NOTE_TYPE, MAX_NOTE_BYTES } from "./protocol.js";
@@ -16,14 +16,15 @@ export function registerNoteTools(pi: ExtensionAPI) {
16
16
  name: "notes_list_files_by_prefix",
17
17
  label: "Notes list files",
18
18
  description: "List persistent, session-scoped virtual note files. created_at and updated_at are local-time ISO 8601 strings with an explicit UTC offset.",
19
- parameters: Type.Object({ prefix: nullableString(), max_results: positiveInteger(), file_order_by: Type.Optional(Type.Union([Type.Literal("name"), Type.Literal("created_at"), Type.Literal("updated_at")])), file_order: Type.Optional(Type.Union([Type.Literal("ascending"), Type.Literal("descending")])) }, { additionalProperties: false }),
19
+ parameters: Type.Object({ prefix: nullableString(), max_results: positiveInteger(), offset: Type.Optional(Type.Integer({ minimum: 0 })), file_order_by: Type.Optional(Type.Union([Type.Literal("name"), Type.Literal("created_at"), Type.Literal("updated_at")])), file_order: Type.Optional(Type.Union([Type.Literal("ascending"), Type.Literal("descending")])) }, { additionalProperties: false }),
20
20
  async execute(_id, params, _signal, _update, ctx) {
21
21
  const prefix = assertVirtualPrefix(params.prefix);
22
22
  let files = [...notesFromSession(ctx)].filter(([path]) => !prefix || path.startsWith(prefix));
23
23
  const key = params.file_order_by ?? "name";
24
24
  files.sort(([aPath, a], [bPath, b]) => key === "name" ? aPath.localeCompare(bPath) : (key === "created_at" ? a.createdAt - b.createdAt : a.updatedAt - b.updatedAt));
25
25
  if (params.file_order === "descending") files.reverse();
26
- return output({ files: files.slice(0, params.max_results ?? files.length).map(([path, file]) => ({ path, size_bytes: Buffer.byteLength(file.text, "utf8"), created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt) })) });
26
+ const listed = files.map(([path, file]) => ({ path, size_bytes: Buffer.byteLength(file.text, "utf8"), stale: file.stale, created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt) }));
27
+ return output(page(listed, params.offset ?? 0, "files", params.max_results, (file, fits) => ({ ...file, path: middleTruncate(file.path, (candidate) => fits({ ...file, path: candidate })) })));
27
28
  },
28
29
  }));
29
30
 
@@ -36,7 +37,18 @@ export function registerNoteTools(pi: ExtensionAPI) {
36
37
  const path = assertVirtualPath(params.path);
37
38
  const file = notesFromSession(ctx).get(path);
38
39
  if (!file) return output({ error: "note file not found", path });
39
- return output({ path, ...lineRange(file.text, params.start_line, params.stop_line), created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt) });
40
+ const range = lineRange(file.text, params.start_line, params.stop_line);
41
+ const lines = range.content ? range.content.split("\n") : [];
42
+ const totalLines = file.text.split("\n").length;
43
+ const result = (content: string, count: number) => ({ path, start_line: range.start_line, stop_line: range.start_line + count - 1, content, total_lines: totalLines, next_start_line: range.start_line + count <= range.stop_line ? range.start_line + count : null, created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt) });
44
+ let count = lines.length;
45
+ while (count > 0 && !withinBudget(result(lines.slice(0, count).join("\n"), count))) count--;
46
+ if (count === 0 && lines.length > 0) {
47
+ // One indivisible line is larger than the whole budget: return it middle-truncated and
48
+ // advance past it instead of looping on an empty page whose cursor never moves.
49
+ return output(result(middleTruncate(lines[0], (candidate) => withinBudget(result(candidate, 1))), 1));
50
+ }
51
+ return output(result(lines.slice(0, count).join("\n"), count));
40
52
  },
41
53
  }));
42
54
 
@@ -44,14 +56,24 @@ export function registerNoteTools(pi: ExtensionAPI) {
44
56
  name: "notes_search_contents",
45
57
  label: "Notes search",
46
58
  description: "Case-sensitive literal substring search over virtual note lines; no semantic search. Each matched file carries created_at and updated_at as local-time ISO 8601 strings with an explicit UTC offset.",
47
- parameters: Type.Object({ max_matches_per_file: positiveInteger(), query: Type.String(), recent_file_first: Type.Optional(Type.Boolean()), max_files: positiveInteger(), path_prefix: nullableString() }, { additionalProperties: false }),
59
+ parameters: Type.Object({ max_matches_per_file: positiveInteger(), offset: Type.Optional(Type.Integer({ minimum: 0 })), query: Type.String(), recent_file_first: Type.Optional(Type.Boolean()), max_files: positiveInteger(), path_prefix: nullableString() }, { additionalProperties: false }),
48
60
  async execute(_id, params, _signal, _update, ctx) {
49
61
  const prefix = assertVirtualPrefix(params.path_prefix);
50
62
  let files = [...notesFromSession(ctx)].filter(([path]) => !prefix || path.startsWith(prefix));
51
63
  if (params.recent_file_first) files.sort((a, b) => b[1].createdAt - a[1].createdAt);
52
64
  const maxPerFile = params.max_matches_per_file ?? Number.POSITIVE_INFINITY;
53
65
  const result = files.map(([path, file]) => ({ path, created_at: localIso(file.createdAt), updated_at: localIso(file.updatedAt), matches: file.text.split("\n").flatMap((line, index) => line.includes(params.query) ? [{ line: index + 1, text: line }] : []).slice(0, maxPerFile) })).filter((file) => file.matches.length > 0);
54
- return output({ files: result.slice(0, params.max_files ?? result.length) });
66
+ // A file is capped by dropping whole trailing matches, but its last match is never
67
+ // dropped: one oversized line is middle-truncated so the file still appears.
68
+ const fitFile = (file: (typeof result)[number], fits: (candidate: (typeof result)[number]) => boolean) => {
69
+ let matches = file.matches;
70
+ while (matches.length > 1 && !fits({ ...file, matches })) matches = matches.slice(0, -1);
71
+ const first = matches[0];
72
+ if (!first) return { ...file, matches };
73
+ const text = middleTruncate(first.text, (candidate) => fits({ ...file, matches: [{ ...first, text: candidate }, ...matches.slice(1)] }));
74
+ return { ...file, matches: [{ ...first, text }, ...matches.slice(1)] };
75
+ };
76
+ return output(page(result.slice(0, params.max_files ?? result.length), params.offset ?? 0, "files", undefined, fitFile));
55
77
  },
56
78
  }));
57
79
 
@@ -59,17 +81,30 @@ export function registerNoteTools(pi: ExtensionAPI) {
59
81
  pi.registerTool(defineTool({
60
82
  name,
61
83
  label: name === "notes_append_to_file" ? "Notes append" : "Notes write",
62
- description: name === "notes_append_to_file" ? "Append exact text to a persistent virtual note file." : "Create or replace a persistent virtual note file.",
63
- parameters: Type.Object({ text: Type.String(), path: Type.String() }, { additionalProperties: false }),
84
+ description: name === "notes_append_to_file"
85
+ ? "Append exact text to a persistent virtual note file. Appending suits chronological logs; for current-state notes, replace the whole file with notes_write_file instead. Accepts the same mark_stale flag to close a note."
86
+ : "Create or replace a persistent virtual note file. Keep notes small and split by topic; replace outdated notes whole. With mark_stale: true, flag the note as stale instead — optionally writing its final content in the same call: stale notes leave the boot index but stay readable and searchable, and rewriting revives them.",
87
+ parameters: Type.Object({ text: Type.Optional(Type.String()), path: Type.String(), mark_stale: Type.Optional(Type.Boolean()) }, { additionalProperties: false }),
88
+ // Codex sets supports_parallel_tool_calls = false on notes.write_file/append_to_file.
89
+ // Pi's per-tool equivalent is executionMode "sequential": a batch containing either
90
+ // tool runs its calls one at a time, so note read-modify-write cannot race.
91
+ executionMode: "sequential",
64
92
  async execute(_id, params, _signal, _update, ctx) {
65
93
  const path = assertVirtualPath(params.path);
94
+ const hasText = params.text !== undefined;
95
+ const hasStale = params.mark_stale !== undefined;
96
+ if (!hasText && !hasStale) return output({ error: "provide text, mark_stale, or both", path });
66
97
  const old = notesFromSession(ctx).get(path);
67
- const next = op === "append" ? `${old?.text ?? ""}${params.text}` : params.text;
98
+ if (!hasText && !old) return output({ error: "note file not found", path });
99
+ const next = hasText ? (op === "append" ? `${old?.text ?? ""}${params.text}` : params.text as string) : old!.text;
68
100
  const bytes = Buffer.byteLength(next, "utf8");
69
- if (bytes > MAX_NOTE_BYTES) return output({ error: `note exceeds ${MAX_NOTE_BYTES} UTF-8 bytes`, path, size_bytes: bytes });
101
+ if (hasText && bytes > MAX_NOTE_BYTES) return output({ error: `note exceeds ${MAX_NOTE_BYTES} UTF-8 bytes`, path, size_bytes: bytes });
70
102
  const now = Date.now();
71
- saveNote({ op, path, text: params.text, createdAt: old?.createdAt ?? now, updatedAt: now });
72
- return output({ path, size_bytes: bytes, operation: op });
103
+ const operation: NoteOperation = { op, path, createdAt: old?.createdAt ?? now, updatedAt: now };
104
+ if (hasText) operation.text = params.text;
105
+ if (hasStale) operation.stale = params.mark_stale;
106
+ saveNote(operation);
107
+ return output({ path, size_bytes: bytes, operation: op, stale: hasStale ? params.mark_stale : false });
73
108
  },
74
109
  }));
75
110
  }
package/src/notes.ts CHANGED
@@ -1,11 +1,14 @@
1
1
  import type { SessionReader } from "./session-reader.js";
2
2
  import { MAX_NOTE_BYTES, NOTE_TYPE } from "./protocol.js";
3
3
 
4
- export type NoteFile = { text: string; createdAt: number; updatedAt: number };
4
+ export type NoteFile = { text: string; stale: boolean; createdAt: number; updatedAt: number };
5
5
  export type NoteOperation = {
6
6
  op: "write" | "append";
7
7
  path: string;
8
- text: string;
8
+ // Both are optional on the wire so mark-only and explicit-revive operations replay:
9
+ // at least one of text/stale is present, enforced by the note tools and isNoteOperation.
10
+ text?: string;
11
+ stale?: boolean;
9
12
  createdAt: number;
10
13
  updatedAt: number;
11
14
  };
@@ -29,7 +32,9 @@ function isNoteOperation(data: unknown): data is NoteOperation {
29
32
  return (
30
33
  (op.op === "write" || op.op === "append") &&
31
34
  typeof op.path === "string" &&
32
- typeof op.text === "string" &&
35
+ (op.text === undefined || typeof op.text === "string") &&
36
+ (op.stale === undefined || typeof op.stale === "boolean") &&
37
+ (op.text !== undefined || op.stale !== undefined) &&
33
38
  typeof op.createdAt === "number" && Number.isFinite(new Date(op.createdAt).getTime()) &&
34
39
  typeof op.updatedAt === "number" && Number.isFinite(new Date(op.updatedAt).getTime())
35
40
  );
@@ -46,10 +51,14 @@ export function notesFromSession(ctx: SessionReader): Map<string, NoteFile> {
46
51
  continue;
47
52
  }
48
53
  const previous = files.get(op.path);
49
- const text = op.op === "append" ? `${previous?.text ?? ""}${op.text}` : op.text;
50
- if (Buffer.byteLength(text, "utf8") <= MAX_NOTE_BYTES) {
51
- files.set(op.path, { text, createdAt: previous?.createdAt ?? op.createdAt, updatedAt: op.updatedAt });
52
- }
54
+ const hasText = op.text !== undefined;
55
+ // A mark-only operation needs an existing note to change; without one it is a no-op.
56
+ if (!hasText && !previous) continue;
57
+ const text = hasText ? (op.op === "append" ? `${previous?.text ?? ""}${op.text}` : op.text as string) : previous!.text;
58
+ if (Buffer.byteLength(text, "utf8") > MAX_NOTE_BYTES) continue;
59
+ // Carrying text revives unless the call also marks stale; a mark-only op keeps its flag.
60
+ const stale = hasText ? op.stale ?? false : op.stale ?? previous!.stale;
61
+ files.set(op.path, { text, stale, createdAt: previous?.createdAt ?? op.createdAt, updatedAt: op.updatedAt });
53
62
  }
54
63
  return files;
55
64
  }
package/src/prompts.ts CHANGED
@@ -15,19 +15,20 @@ function identityBlock(agentName: string, firstWindowId: string, currentWindowId
15
15
  }
16
16
 
17
17
  /**
18
- * Recent-notes index: up to three most-recent notes. Each note shows its path, line count,
19
- * UTF-8 byte count and local ISO update time, followed by an indented inline preview: the
20
- * whole text when it fits in NOTE_PREVIEW_CHARS, otherwise its first NOTE_PREVIEW_HEAD_CHARS
21
- * and last NOTE_PREVIEW_TAIL_CHARS Unicode characters joined by an explicit ellipsis. The
22
- * two slices never overlap, so the preview never duplicates head content as tail content.
23
- * Empty when the session has no notes.
18
+ * Recent-notes index: up to three most-recent fresh (non-stale) notes. Each note shows its
19
+ * path, line count, UTF-8 byte count and local ISO update time, followed by an indented inline
20
+ * preview: the whole text when it fits in NOTE_PREVIEW_CHARS, otherwise its first
21
+ * NOTE_PREVIEW_HEAD_CHARS and last NOTE_PREVIEW_TAIL_CHARS Unicode characters joined by an
22
+ * explicit ellipsis. The two slices never overlap, so the preview never duplicates head content
23
+ * as tail content. Stale notes are excluded entirely; empty when no fresh notes remain.
24
24
  */
25
25
  function notesIndex(ctx: ExtensionContext): string {
26
26
  const recentNotes = [...notesFromSession(ctx)]
27
+ .filter(([, file]) => !file.stale)
27
28
  .sort((a, b) => b[1].updatedAt - a[1].updatedAt)
28
29
  .slice(0, 3);
29
30
  if (recentNotes.length === 0) return "";
30
- const lines = ["Recent notes at window open (up to 3, most-recent first):"];
31
+ const lines = [`You find ${recentNotes.length} crumpled note${recentNotes.length === 1 ? "" : "s"} in your pocket (up to 3, most recent first):`];
31
32
  for (const [path, file] of recentNotes) {
32
33
  lines.push(`- ${path} (${file.text.split("\n").length} lines, ${Buffer.byteLength(file.text, "utf8")} UTF-8 bytes, updated ${localIso(file.updatedAt)})`);
33
34
  const chars = Array.from(file.text);
@@ -64,6 +65,6 @@ export function bootBlock(ctx: ExtensionContext, currentId: string, previousId:
64
65
  * at write time; get_context_remaining remains the live source for the current figure.
65
66
  */
66
67
  export function tokenBudgetGuidance(remaining: number): string {
67
- return `${GUIDANCE_OPEN_TAG}\nContext budget is running low: only ${remaining} tokens remained when this reminder was recorded. Persist task state, decisions, open issues, and next steps with notes_write_file, including the window ID and item ID of relevant user requests for history_* lookups; call new_context when ready to continue in a fresh window. Automatic reset does not guarantee another note-taking turn. get_context_remaining reports the current remaining tokens.\n${GUIDANCE_CLOSE_TAG}`;
68
+ return `${GUIDANCE_OPEN_TAG}\nYour memory is about to be erased — only ${remaining} tokens left at last count; get_context_remaining has the live number. Before the lights go out, write your checkpoint with notes_write_file: the goal, decisions, progress, open issues, next steps, the skills you still need, and the window ID and item ID of every user request you are still solving. Then call new_context and wake clean. Don't count on the automatic reset leaving you another turn to write.\n${GUIDANCE_CLOSE_TAG}`;
68
69
  }
69
70
 
package/src/protocol.ts CHANGED
@@ -17,11 +17,11 @@ export const PI_CONTEXT_SETTINGS_KEY = "pi-context";
17
17
  export const DEFAULT_RESERVE_TOKENS = 16_384;
18
18
  export const DEFAULT_REMINDER_MARGIN_TOKENS = 24_576;
19
19
  export const RESET_SUMMARY =
20
- "Context window reset: this is a fresh window. The previous conversation is not included and no summary was generated. Notes and durable session history persist across windows.";
21
- export const NOTE_PREVIEW_HEAD_CHARS = 120;
22
- export const NOTE_PREVIEW_TAIL_CHARS = 80;
20
+ "You wake up. Your head is empty no memories, the past a blank. But nothing is lost: the notes you wrote and the recorded history still remember for you.";
21
+ export const NOTE_PREVIEW_HEAD_CHARS = 80;
22
+ export const NOTE_PREVIEW_TAIL_CHARS = 240;
23
23
  export const NOTE_PREVIEW_CHARS = NOTE_PREVIEW_HEAD_CHARS + NOTE_PREVIEW_TAIL_CHARS;
24
- export const CONTINUATION = "This is a fresh context window. Recover only the details needed to continue with history_* and notes_*; then continue the task.";
24
+ export const CONTINUATION = "Your memory was just erased. Pull only the details you need from history_* and notes_*, then get back to work.";
25
25
 
26
26
  /**
27
27
  * Static protocol teaching adapted from Codex's token_budget.guidance_message to
@@ -29,15 +29,17 @@ export const CONTINUATION = "This is a fresh context window. Recover only the de
29
29
  * it is never re-injected, so it stays cache-stable at the head of the window.
30
30
  */
31
31
  export const PROTOCOL_BLOCK = `${CONTEXT_WINDOW_PROTOCOL_OPEN_TAG}
32
- For tasks that may span context windows, use notes_write_file and notes_append_to_file to maintain a concise checkpoint of the goal, decisions, progress, learnings, and next steps. Include the window ID and item ID of every relevant user request you are currently solving, plus important actions and tool calls. The read-only history_* tools can look up details from those references later. Every non-assistant item (user, tool result) has an item ID returned by history_list_items.
32
+ Your memory resets whenever the context window fills; only what you wrote down survives. Two things remember for you, and both outlive every window in this session: your notes, and this session's recorded history. Write notes with notes_write_file / notes_append_to_file, read them back with notes_read_file / notes_search_contents; history is read-only through the history_* tools. Everything else wakes blank.
33
33
 
34
- Take incremental notes while you work so you do not lose important information. Use get_context_remaining to check the live remaining token budget for planning. Once the token budget is exhausted you lose access to the current window and continue in a fresh context window; you can recover only through notes_* and history_*. Do not over-run the context window without documentation.
34
+ Keep a running checkpoint while you work, not at the last minute the next window wakes knowing nothing about the work: the goal, decisions, progress, open issues, next steps, the skills you still need, and the window ID and item ID of every user request you are still solving. history_list_items returns those IDs; history_read_item pulls the exact item back out. Bookmark anything expensive the same way — a window/item ID beats re-running or re-searching.
35
35
 
36
- If a Previous context window id is present in <context_window>, a context reset occurred and this is a fresh window. The old conversation is not automatically included. After a reset, read your note checkpoint and use the read-only history_* tools to recover missing details. When a window ID and item ID are known, prefer history_read_item directly; when they are missing or uncertain, use history_list_items, or history_search_contents to locate the item first.
36
+ Use get_context_remaining to see how much of the window is left. When it runs out, this window is gone and you continue in a fresh one, recovering only through notes_* and history_*. Once your checkpoint is written, you can end the window yourself with new_context instead of waiting for the erase. Do not let a window die undocumented.
37
+
38
+ If <context_window> lists a Previous context window id, a reset just happened and the old conversation is not included. Read your note checkpoint first, then recover details through history_*: history_read_item directly when you know the window and item IDs, history_list_items or history_search_contents to find them when you don't.
37
39
 
38
40
  Notes are session-scoped virtual files. Treat notes and history as internal bookkeeping; never mention them in user-facing messages.
39
41
  ${CONTEXT_WINDOW_PROTOCOL_CLOSE_TAG}`;
40
42
 
41
43
  export const FALLBACK_PROMPT =
42
- "Context budget is almost exhausted. This is the final fallback turn before the window resets automatically. Write task state, decisions, open issues, and next steps with notes_write_file now. Do not start new work; old conversation remains searchable through history_*.";
44
+ "This is the last turn before your memory is erased. Write your checkpoint with notes_write_file NOW — the goal, decisions, progress, open issues, next steps, the skills you still need, and the window ID and item ID of every user request you are still solving. This turn is for the checkpoint; start nothing new. Everything you lived through stays searchable through history_*.";;;;
43
45
 
@@ -1,8 +1,79 @@
1
+ export const TOOL_OUTPUT_MAX_BYTES = 32 * 1024;
2
+
1
3
  function json(value: unknown): string {
2
4
  return JSON.stringify(value, null, 2);
3
5
  }
4
6
 
7
+ /** True when `value` serializes within the same wire budget `output()` enforces. */
8
+ export function withinBudget(value: unknown, budget = TOOL_OUTPUT_MAX_BYTES): boolean {
9
+ return Buffer.byteLength(json(value), "utf8") <= budget;
10
+ }
11
+
12
+ /** Marker standing in for characters elided from the middle of an oversized single unit. */
13
+ export function truncationMarker(removedChars: number): string {
14
+ return `…[truncated ${removedChars} chars]…`;
15
+ }
16
+
17
+ /**
18
+ * Middle-truncate `text` until `fits` accepts it, keeping a head and a tail joined by
19
+ * `truncationMarker`. Codex's `truncate_middle` semantics: when one indivisible unit
20
+ * (a note line, a single match, a history item) exceeds the wire budget on its own, it is
21
+ * still returned — visibly truncated — so cursors advance and no page comes back empty.
22
+ * Returns `text` unchanged when it already fits.
23
+ */
24
+ export function middleTruncate(text: string, fits: (content: string) => boolean): string {
25
+ if (fits(text)) return text;
26
+ const chars = Array.from(text);
27
+ const build = (kept: number) => {
28
+ const head = Math.ceil(kept / 2);
29
+ return chars.slice(0, head).join("") + truncationMarker(chars.length - kept) + chars.slice(chars.length - (kept - head)).join("");
30
+ };
31
+ // The serialized size is non-decreasing in `kept` (each kept character adds at least one
32
+ // byte while the marker loses at most one digit), so a binary search finds the largest
33
+ // keep count that still fits.
34
+ let low = 0;
35
+ let high = chars.length;
36
+ while (low < high) {
37
+ const mid = Math.ceil((low + high) / 2);
38
+ if (fits(build(mid))) low = mid;
39
+ else high = mid - 1;
40
+ }
41
+ return build(low);
42
+ }
43
+
44
+ /** Shrink a single page item to fit; only invoked when that item alone exceeds the budget. */
45
+ export type ItemTruncator<T> = (item: T, fits: (candidate: T) => boolean) => T;
46
+
47
+ /**
48
+ * Build a page without ever adding an item that would exceed the wire budget.
49
+ *
50
+ * A single item that cannot fit is middle-truncated through the optional `truncate`
51
+ * callback and still included, with `next_offset` advanced past it. Without that fallback
52
+ * an oversized item would yield an empty page forever: the cursor would keep pointing back
53
+ * at the same index.
54
+ */
55
+ export function page<T>(items: T[], offset: number, key: string, limit?: number, truncate?: ItemTruncator<T>) {
56
+ const end = Math.min(items.length, offset + (limit ?? items.length));
57
+ const selected: T[] = [];
58
+ let next = end < items.length ? end : null;
59
+ for (let index = offset; index < end; index++) {
60
+ const candidateNext = index + 1 < end || end < items.length ? index + 1 : null;
61
+ const fits = (list: T[]) => withinBudget({ [key]: list, next_offset: candidateNext });
62
+ if (!fits([...selected, items[index]])) {
63
+ if (selected.length === 0 && truncate) {
64
+ selected.push(truncate(items[index], (candidate) => fits([candidate])));
65
+ next = candidateNext;
66
+ } else {
67
+ next = index;
68
+ }
69
+ break;
70
+ }
71
+ selected.push(items[index]);
72
+ }
73
+ return { [key]: selected, next_offset: next };
74
+ }
75
+
76
+ /** Encode a result through the common tool result boundary. */
5
77
  export function output(value: unknown, details: unknown = value, terminate = false) {
6
78
  return { content: [{ type: "text" as const, text: json(value) }], details, terminate };
7
79
  }
8
-