pi-posthorse 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -12
- package/index.ts +229 -114
- package/package.json +1 -1
- package/ui.ts +3 -3
package/README.md
CHANGED
|
@@ -8,8 +8,6 @@ No-summary context rollover for [Pi](https://github.com/earendil-works/pi), on o
|
|
|
8
8
|
|
|
9
9
|
Pi owns the persisted boundary. Posthorse owns the policy: stable window guidance, one best-effort checkpoint reminder, `new_context`, `get_context_remaining`, durable `notes`, and window-aware `history`. A rollover removes the old window from active model context while the JSONL transcript stays append-only and complete.
|
|
10
10
|
|
|
11
|
-
An isolated [Codex prototype](adapters/codex-posthorse/README.md) tests local notes and recovery with Codex's native context reset. It is not ready for everyday use and is not included in the Pi package.
|
|
12
|
-
|
|
13
11
|
## Requirements
|
|
14
12
|
|
|
15
13
|
- Node `>=24.12.0`.
|
|
@@ -49,16 +47,16 @@ Official Pi limits that the fork removes:
|
|
|
49
47
|
## How it works
|
|
50
48
|
|
|
51
49
|
1. **Stable guidance.** Window behavior lives in a native system-prompt section. There is no per-request meter. Posthorse preserves custom prompts and falls back to appending guidance when an earlier extension replaces the full prompt; that compatibility path can prevent Pi from preserving an additive prompt prefix.
|
|
52
|
-
2. **One best-effort checkpoint.** While Pi compaction is enabled, one reminder may appear shortly before Pi's rollover line
|
|
53
|
-
3. **`get_context_remaining`.** Reports the best available native estimate of tokens until Pi's automatic rollover line and until the configured context limit. That configured limit is not a measured provider rejection boundary. Pi's value is an estimate until the active model reports usage.
|
|
50
|
+
2. **One best-effort checkpoint.** While Pi compaction is enabled, one reminder may appear shortly before Pi's rollover line, asking the model to rewrite its current-state note in place and call `new_context`. A large turn, overflow, restart, or smaller model can reach rollover without it. Reminders are fingerprinted by window, context size, and reserve, so switching to a different context size gets a fresh reminder. Stale reminders and reminders left after disabling compaction are filtered from model input while remaining in history.
|
|
51
|
+
3. **`get_context_remaining`.** Reports the best available native estimate of tokens until Pi's automatic rollover line and until the configured context limit. That configured limit is not a measured provider rejection boundary. Pi's value is an estimate until the active model reports usage. The checkpoint reminder marks the deadline, so the guidance does not prompt routine budget checks; the tool is described for use before a large step late in a window.
|
|
54
52
|
4. **`new_context`.** Requests an atomic rollover after the current foreground (synchronous) tools succeed. Items 4–6 and the receipt bounding below describe the fork; official Pi differences are listed above. An optional handoff is persisted and becomes the first state of the fresh window. A failed synchronous checkpoint save or other foreground sibling prevents Pi from committing the boundary; the checkpoint reminder still applies. Native asynchronous background calls continue across the reset without delaying it. Pi preserves their call provenance and delivers their results in the fresh context.
|
|
55
53
|
5. **Automatic rollover without summaries.** With a supported context budget and room for a recovery record, Posthorse claims Pi's automatic threshold and overflow trigger through `session_before_auto_compact`, before Pi resolves summarization credentials or prepares a summary. Oversized first turns and tool results can then roll over even without summarization credentials. Native asynchronous background calls can remain in progress across automatic rollover too. When a recovery record cannot fit but Pi will retain native receipts in the next window, Posthorse claims a window without a handoff so Pi can prepare and bound those receipts or refuse the window. Already-consumed receipts do not trigger an empty rollover. Otherwise Pi's own compaction remains in control. Manual `/compact` is unchanged.
|
|
56
|
-
6. **Bounded recovery record.** The automatic handoff
|
|
57
|
-
7. **`notes` and `history`.** Notes are shared across linked worktrees, at the main checkout for conventional Git layouts or inside the common Git directory when metadata is stored separately. History searches normalized transcript text and returns stored images for a requested entry. Tool calls and results retain names, namespaces, and call ids; admitted execution arguments are labeled separately from requested arguments when present. When a handoff carries edited content, its entry ID points to the replacement; `history read` of that ID returns the replacement text or images. Reading the original entry ID still returns the raw journal content.
|
|
54
|
+
6. **Bounded recovery record.** The automatic handoff quotes the window's first owner request, its latest owner input (a direct user input or `ask_question` outcome), and its latest input of any kind, including visible coordination messages. It adds tool result evidence: tool names and namespaces, call ids, requested and admitted execution arguments when available, result excerpts of up to 1,500 characters, and entry ids to recover the rest. Every other current-window input gets one index line with its label, time, and entry id; owner inputs keep a one-line preview there. Native async results from the current projected window remain eligible even across later complete assistant responses, since a result can arrive during a response that never received it. A result Pi still projects is also eligible when its call was edited out or predates the window, without attributing it to another call. These results may already have been received or handled; they are not proof of current progress. Synchronous results with projected call provenance are kept only from the trailing batch without a later complete response. Pi context edits also apply to these inputs and results: omitted entries stay out, and replacements supersede their original content. A clearly labeled, possibly stale older checkpoint comes last. Older assistant prose is not treated as current state. Newly submitted input stays separate and is saved after the boundary, not copied into the handoff.
|
|
55
|
+
7. **`notes` and `history`.** Notes are shared across linked worktrees, at the main checkout for conventional Git layouts or inside the common Git directory when metadata is stored separately. History searches normalized transcript text, skips entries whose full text the model already sees in its active context (and says how many), and returns stored images for a requested entry. Edited-away originals and checkpoint reminders stay searchable. Tool calls and results retain names, namespaces, and call ids; admitted execution arguments are labeled separately from requested arguments when present. When a handoff carries edited content, its entry ID points to the replacement; `history read` of that ID returns the replacement text or images. Reading the original entry ID still returns the raw journal content.
|
|
58
56
|
|
|
59
57
|
Before each fresh window becomes active, Posthorse bounds oversized retained native receipts through append-only context edits. It shares the available capacity across all receipts after accounting for the prompt, tools, pending input, handoff, carried calls, Pi's configured reserve when automatic rollover is enabled and supported, and a safety margin. Pi separately clamps each provider request's output limit to its remaining context capacity. When receipts need shortening, excerpts use half the remaining capacity so the next response has working room. Excerpts retain the native call/result identity and error state, with `history read` references for complete recovery, including images. Existing replacements remain the recovery source across later windows; raw pre-edit content is never used for a new excerpt. Already-fitting receipts stay unchanged. If fixed input and recovery references cannot fit, rollover fails visibly without publishing a window marker or any proposed edits. Later prompts retry preparation against the unchanged window; correcting the configuration allows ordinary rollover to proceed. Pi independently refuses estimated native transcript input that exceeds physical context capacity, including when compaction is disabled. Its output clamp still permits fitting requests with reduced output, even above the compaction reserve line. Raw wire replacements through `onPayload` / `before_provider_request` and custom transport implementations retain responsibility for their own request sizing.
|
|
60
58
|
|
|
61
|
-
At turn end, Posthorse checks whether usage is in the reminder band before explicitly looking up the full branch. Context filtering skips its branch lookup when model input contains no `posthorse-reminder` messages. Complete history remains available through paged searches and reads.
|
|
59
|
+
At turn end, Posthorse checks whether usage is in the reminder band before explicitly looking up the full branch. Context filtering skips its branch lookup when model input contains no `posthorse-reminder` messages, and on the fork while the window's marker message is present. Complete history remains available through paged searches and reads.
|
|
62
60
|
|
|
63
61
|
Automatic recovery is an emergency input record, not proof of progress. The fresh model is told to restore notes and todo state, inspect history when needed, and verify live state before taking stateful or external action.
|
|
64
62
|
|
|
@@ -68,7 +66,7 @@ Posthorse follows Pi's effective `compaction` settings, including Pi's decision
|
|
|
68
66
|
|
|
69
67
|
The model's context window minus `compaction.reserveTokens` must leave at least 10,000 usable tokens. Below that (for example an 8K or 16K model with the default 16,384 reserve) Posthorse reports an unsupported configuration in the guidance and in `get_context_remaining`, turns automatic behavior off for that model, and leaves Pi's own compaction in place. Lower the reserve or use a larger model. The checkpoint reminder band is the last 10% of usable context, capped at 32,000 tokens, so a large reserve cannot trigger a reminder immediately in a fresh window.
|
|
70
68
|
|
|
71
|
-
Explicit and automatic handoffs are capped at 20,000 characters and half the active model's fresh operational capacity after prompt/tool overhead and any pending input, whichever is smaller. Oversized explicit handoffs are rejected with an instruction to save fuller state in notes; automatic rollover stays with Pi when no safe recovery record fits and no native receipts require fresh-window preparation.
|
|
69
|
+
Explicit and automatic handoffs are capped at 20,000 characters and half the active model's fresh operational capacity after prompt/tool overhead and any pending input, whichever is smaller. Oversized explicit handoffs are rejected with an instruction to save fuller state in notes; automatic rollover stays with Pi when no safe recovery record fits and no native receipts require fresh-window preparation. Notes and history pages are capped at 40,000 characters and shrink to the remaining budget; when usage is not known yet, they also leave half the fresh operational capacity free.
|
|
72
70
|
|
|
73
71
|
Only one automatic compaction or rollover policy extension should be enabled at a time. Pi keeps the last non-cancel result from multiple handlers of the same hook, so load order would otherwise decide which policy wins.
|
|
74
72
|
|
|
@@ -76,8 +74,8 @@ Only one automatic compaction or rollover policy extension should be enabled at
|
|
|
76
74
|
|
|
77
75
|
- `new_context({ handoff? })`
|
|
78
76
|
- `get_context_remaining()`
|
|
79
|
-
- `notes({ op, ... })`: `list`, `read`, and `search` are paged; continue with the returned character `offset`.
|
|
80
|
-
- `history({ op, ... })`: `search` continues with the returned `cursor` and the same `query` and `all` scope
|
|
77
|
+
- `notes({ op, ... })`: `list`, `read`, and `search` are paged; continue with the returned character `offset`. `list` shows notes newest first with size and modification time (UTC) and takes an optional folder `path`; `read` of a folder returns its listing. `search` covers text notes, skipping binary files (a NUL byte in the first 8,000 bytes, as Git decides), and its excerpts center on the match. `write` replaces content (empty content clears); `append` adds one newline-terminated record.
|
|
78
|
+
- `history({ op, ... })`: `search` skips entries whose full text is already in the active context, continues with the returned `cursor` and the same `query` and `all` scope, and treats `limit` as the maximum results per page. `read` pages text and stored images; continue with the returned character `offset` and `imageOffset`. An image-only continuation can use an `offset` equal to the text length. If a page needs fresh context, retry with both offsets unchanged. Results keep native entry and window ids.
|
|
81
79
|
|
|
82
80
|
In the TUI, tools use Pi's native expandable cards. Collapsed cards show the operation and target, a short content preview, and counts or page ranges with the next offset when more remains. Expand with Pi's tool-output shortcut (`Ctrl+O` by default), or click the card's header or body in fullscreen mode. Expanded cards show the complete returned page and its metadata, not content the tool has not fetched yet. Writes and context requests also show the submitted content or handoff.
|
|
83
81
|
|
|
@@ -87,7 +85,7 @@ Notes list/search and history search share the remaining context budget with rea
|
|
|
87
85
|
|
|
88
86
|
`history search` puts matching original content before recovery material: handoffs, compaction and branch summaries, checkpoint reminders, and `notes`, `new_context`, and `history` calls/results. Ordinary prose or another tool call in the same assistant entry keeps its priority when that content matches. Every entry remains searchable; `history read` returns the complete normalized entry, including any recovery content omitted from a search excerpt.
|
|
89
87
|
|
|
90
|
-
Within each group, current-branch matches are newest first. With `all: true`, Posthorse searches session files in the active Pi session directory whose saved working directory matches the current one, along with their nested subagent sessions and the selected current session file when it is in that directory. Files without a valid session header or an attributable working directory are excluded; the active branch remains available. Within that scope, sessions are ordered newest-modified first and entries newest first; this is not a global timestamp sort. All-session hits use a file-qualified ID (`native-id@file-key
|
|
88
|
+
Within each group, current-branch matches are newest first. With `all: true`, Posthorse searches session files in the active Pi session directory whose saved working directory matches the current one, along with their nested subagent sessions and the selected current session file when it is in that directory. Files without a valid session header or an attributable working directory are excluded; the active branch remains available. Within that scope, sessions are ordered newest-modified first and entries newest first; this is not a global timestamp sort. All-session hits use a file-qualified ID (`native-id@file-key`, with a 10-character key); pass the complete ID to `history read`. Full-length keys printed by earlier releases still read. Hits from nested subagent runs are tagged `[subagent]`, because their user-role input came from a parent agent. A read that resolves a bare ID in another session file prints that entry's qualified ID, so continuations reopen the same file. When ripgrep is available (Pi's managed copy, then `PATH`), all-session search and bare-ID reads skip session files that cannot contain the query or ID. A query that could match text session files store escaped (quotes, backslashes) or label text that exists only in normalized entries (brackets, parentheses, braces, colons, `$`, leading or trailing spaces, or phrases such as "Call ID" and "No handoff") scans every file instead. Image summaries come only from entries with an image block, so image-count and comma queries also search every file that has one. Either way, results match a full scan. The per-page result limit applies after priority, so newer echoes cannot displace older original matches. Each included session file contributes its own matches, including fork copies, so unrelated entries sharing a native ID are never hidden. Search cursors advance past the previous entry, including partial headers or excerpts, so appending lookup calls and results does not keep pagination alive forever. Searches are live: external session edits or file reordering can change results; start a new search to include newer sources.
|
|
91
89
|
|
|
92
90
|
Notes live in `.pi/notes/` at the main checkout for conventional Git layouts, or the current directory outside Git. When Git metadata is stored separately, including in submodules, all checkouts use `.pi/notes/` inside Git's common directory instead; no configuration is needed. If that metadata is unavailable, notes stay local to the checkout. Writes report the actual storage path. Add `.pi/notes/` to `.gitignore` when the project should not track it.
|
|
93
91
|
|
|
@@ -97,7 +95,7 @@ Notes live in `.pi/notes/` at the main checkout for conventional Git layouts, or
|
|
|
97
95
|
|
|
98
96
|
- Posthorse makes no network requests.
|
|
99
97
|
- Notes are plaintext files under `.pi/notes`. They survive package removal and may be committed unless ignored.
|
|
100
|
-
- `history` with `all: true` scans project-matching JSONL files and their nested subagent sessions in the active Pi session directory, not unrelated projects that share that directory.
|
|
98
|
+
- `history` with `all: true` scans project-matching JSONL files and their nested subagent sessions in the active Pi session directory, not unrelated projects that share that directory. It runs ripgrep locally over that directory when available.
|
|
101
99
|
- History can return user text, assistant text and thinking, assistant failure status and error messages, tool arguments and results, handoffs, custom messages, and images. Direct shell entries Pi marked `excludeFromContext` come back as a placeholder only.
|
|
102
100
|
- Returned history content enters the currently selected model and provider context.
|
|
103
101
|
- Removing Posthorse does not remove notes or Pi session history.
|
package/index.ts
CHANGED
|
@@ -7,6 +7,7 @@
|
|
|
7
7
|
* tools, durable notes, and history recovery.
|
|
8
8
|
*/
|
|
9
9
|
|
|
10
|
+
import { execFile } from "node:child_process";
|
|
10
11
|
import { createHash, randomUUID } from "node:crypto";
|
|
11
12
|
import {
|
|
12
13
|
appendFileSync,
|
|
@@ -20,15 +21,18 @@ import {
|
|
|
20
21
|
} from "node:fs";
|
|
21
22
|
import { access, constants, lstat, open, readlink, realpath, rename, stat, unlink } from "node:fs/promises";
|
|
22
23
|
import { basename, dirname, isAbsolute, join, normalize, relative, resolve, sep } from "node:path";
|
|
23
|
-
import {
|
|
24
|
+
import { promisify } from "node:util";
|
|
25
|
+
import { estimateTokens, formatSize, getAgentDir, SettingsManager, withFileMutationQueue } from "@earendil-works/pi-coding-agent";
|
|
24
26
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
25
27
|
import { Type } from "typebox";
|
|
26
28
|
import { registerPosthorseMessages, toolCards, type PosthorseDisplay } from "./ui.ts";
|
|
27
29
|
|
|
28
30
|
const REMINDER_BUFFER_TOKENS = 32_000;
|
|
29
|
-
/** Absolute
|
|
31
|
+
/** Absolute ceilings; handoffs and pages shrink to the live remaining budget. */
|
|
30
32
|
const MAX_HANDOFF_CHARS = 20_000;
|
|
33
|
+
const MAX_PAGE_CHARS = 40_000;
|
|
31
34
|
const MAX_RECOVERY_RECORD_CHARS = 4_000;
|
|
35
|
+
const MAX_RECOVERY_RESULT_CHARS = 1_500;
|
|
32
36
|
const HANDOFF_OVERHEAD_RESERVE = 1_000;
|
|
33
37
|
const PAGE_MARGIN_TOKENS = 1_000;
|
|
34
38
|
const MIN_PAGE_CHARS = 1_000;
|
|
@@ -82,6 +86,7 @@ type MessageLike = {
|
|
|
82
86
|
stopReason?: string;
|
|
83
87
|
errorMessage?: string;
|
|
84
88
|
content?: unknown;
|
|
89
|
+
summary?: string;
|
|
85
90
|
toolName?: string;
|
|
86
91
|
namespace?: string;
|
|
87
92
|
toolCallId?: string;
|
|
@@ -119,6 +124,7 @@ type RecoveryRecord = {
|
|
|
119
124
|
text: string;
|
|
120
125
|
};
|
|
121
126
|
type ReminderFingerprint = { windowId: string; contextWindow?: number; reserveTokens?: number };
|
|
127
|
+
type NoteFile = { path: string; size: number; mtimeMs: number };
|
|
122
128
|
type Budget = {
|
|
123
129
|
contextWindow: number;
|
|
124
130
|
reserveTokens: number;
|
|
@@ -349,8 +355,22 @@ function isRecoveryCall(part: unknown): boolean {
|
|
|
349
355
|
return block?.type === "toolCall" && isRecoveryTool(block.name);
|
|
350
356
|
}
|
|
351
357
|
|
|
352
|
-
|
|
353
|
-
|
|
358
|
+
/** Ten base64url characters of SHA-256; 0.6 file keys were the whole digest, which starts with these. */
|
|
359
|
+
function shortKey(text: string): string {
|
|
360
|
+
return createHash("sha256").update(text).digest("base64url").slice(0, 10);
|
|
361
|
+
}
|
|
362
|
+
|
|
363
|
+
/** Nested session files are subagent runs, whose user-role input came from a parent agent. */
|
|
364
|
+
function sourceTag(source: string): string {
|
|
365
|
+
return source.includes(sep) ? " [subagent]" : "";
|
|
366
|
+
}
|
|
367
|
+
|
|
368
|
+
function contextEdits(entries: readonly EntryLike[]): Map<string, EntryLike> {
|
|
369
|
+
const edits = new Map<string, EntryLike>();
|
|
370
|
+
for (const entry of entries) {
|
|
371
|
+
if (entry.type === "context_edit" && entry.targetId) edits.set(entry.targetId, entry);
|
|
372
|
+
}
|
|
373
|
+
return edits;
|
|
354
374
|
}
|
|
355
375
|
|
|
356
376
|
function historyHit(item: WindowedEntry, query: string, source = ""): HistoryHit | undefined {
|
|
@@ -391,8 +411,8 @@ function historyHit(item: WindowedEntry, query: string, source = ""): HistoryHit
|
|
|
391
411
|
matchIndex = text.toLowerCase().indexOf(query);
|
|
392
412
|
}
|
|
393
413
|
}
|
|
394
|
-
const id = source ? `${entry.id}@${
|
|
395
|
-
const header = `${
|
|
414
|
+
const id = source ? `${entry.id}@${shortKey(source)}` : entry.id!;
|
|
415
|
+
const header = `${entry.timestamp ?? ""}${sourceTag(source)} [window ${item.windowId}] [${id}] `;
|
|
396
416
|
return {
|
|
397
417
|
id,
|
|
398
418
|
priority,
|
|
@@ -419,8 +439,8 @@ function* windowEntries(entries: Iterable<EntryLike>): Generator<WindowedEntry>
|
|
|
419
439
|
}
|
|
420
440
|
}
|
|
421
441
|
|
|
422
|
-
/**
|
|
423
|
-
async function*
|
|
442
|
+
/** Split on LF only: JSONL strings and notes can contain Unicode separators that Node's readline also splits on. */
|
|
443
|
+
async function* lfLines(file: string, signal?: AbortSignal): AsyncGenerator<string> {
|
|
424
444
|
const stream = createReadStream(file, { encoding: "utf8", signal });
|
|
425
445
|
let pending: string[] = [];
|
|
426
446
|
for await (const chunk of stream) {
|
|
@@ -436,10 +456,56 @@ async function* jsonlLines(file: string, signal?: AbortSignal): AsyncGenerator<s
|
|
|
436
456
|
if (last) yield last;
|
|
437
457
|
}
|
|
438
458
|
|
|
459
|
+
/** Git's heuristic: a NUL byte in the first 8,000 bytes marks binary content. */
|
|
460
|
+
async function isBinaryFile(path: string): Promise<boolean> {
|
|
461
|
+
const file = await open(path, "r");
|
|
462
|
+
try {
|
|
463
|
+
const { buffer, bytesRead } = await file.read(Buffer.alloc(8_000), 0, 8_000, 0);
|
|
464
|
+
return buffer.subarray(0, bytesRead).includes(0);
|
|
465
|
+
} finally {
|
|
466
|
+
await file.close();
|
|
467
|
+
}
|
|
468
|
+
}
|
|
469
|
+
|
|
470
|
+
const execFileAsync = promisify(execFile);
|
|
471
|
+
/**
|
|
472
|
+
* ripgrep needles whose raw JSONL matches cover every normalized match, or undefined when only a full scan is exact.
|
|
473
|
+
* Raw JSONL escapes quotes, backslashes, and control characters; flattenEntry adds brackets, parentheses, braces,
|
|
474
|
+
* colons, `$`, spaces beside those labels, and label-only phrases. Image summaries ("3 images: image/png,
|
|
475
|
+
* image/jpeg") come only from entries with an image block, so their text needs that block instead.
|
|
476
|
+
*/
|
|
477
|
+
function prefilterNeedles(query: string): string[] | undefined {
|
|
478
|
+
if (/["\\[\](){}:$\p{Cc}]/u.test(query) || query.trim() !== query) return undefined;
|
|
479
|
+
if (["call id", "execution arguments", "no handoff", "excluded from model context by pi"].some((phrase) => phrase.includes(query))) return undefined;
|
|
480
|
+
const image = query.includes(",") || /^\d+ ?i?m?a?g?e?s?$/.test(query) || ["images", "unknown type"].some((phrase) => phrase.includes(query));
|
|
481
|
+
return image ? [query, '"type":"image"'] : [query];
|
|
482
|
+
}
|
|
483
|
+
|
|
484
|
+
/** Session files whose raw JSONL contains any needle in any case, via Pi's managed ripgrep or PATH; undefined means scan every file. */
|
|
485
|
+
async function filesContaining(dir: string, needles: string[], signal?: AbortSignal): Promise<Set<string> | undefined> {
|
|
486
|
+
const args = [
|
|
487
|
+
"--files-with-matches", "--fixed-strings", "--ignore-case", "--no-ignore", "--hidden", "--no-messages",
|
|
488
|
+
"--glob", "*.jsonl", ...needles.flatMap((needle) => ["--regexp", needle]), dir,
|
|
489
|
+
];
|
|
490
|
+
for (const rg of [join(getAgentDir(), "bin", "rg"), "rg"]) {
|
|
491
|
+
try {
|
|
492
|
+
const { stdout } = await execFileAsync(rg, args, { signal, maxBuffer: 64 * 1024 * 1024 });
|
|
493
|
+
return new Set(stdout.split("\n").filter(Boolean).map((file) => resolve(file)));
|
|
494
|
+
} catch (error) {
|
|
495
|
+
const { code } = error as { code?: unknown };
|
|
496
|
+
if (code === 1) return new Set();
|
|
497
|
+
if (code === "ENOENT") continue;
|
|
498
|
+
signal?.throwIfAborted();
|
|
499
|
+
return undefined;
|
|
500
|
+
}
|
|
501
|
+
}
|
|
502
|
+
return undefined;
|
|
503
|
+
}
|
|
504
|
+
|
|
439
505
|
async function* sessionWindowEntries(file: string, signal?: AbortSignal): AsyncGenerator<WindowedEntry> {
|
|
440
506
|
if (!existsSync(file)) return;
|
|
441
507
|
const windows = new Map<string, string>();
|
|
442
|
-
for await (const line of
|
|
508
|
+
for await (const line of lfLines(file, signal)) {
|
|
443
509
|
let entry: EntryLike;
|
|
444
510
|
try {
|
|
445
511
|
entry = JSON.parse(line) as EntryLike;
|
|
@@ -454,18 +520,15 @@ async function* sessionWindowEntries(file: string, signal?: AbortSignal): AsyncG
|
|
|
454
520
|
|
|
455
521
|
function sessionFiles(dir: string): string[] {
|
|
456
522
|
if (!existsSync(dir)) return [];
|
|
457
|
-
|
|
523
|
+
return readdirSync(dir, { recursive: true, withFileTypes: true })
|
|
458
524
|
.filter((entry) => entry.isFile() && entry.name.endsWith(".jsonl") && !entry.name.includes(".intent."))
|
|
459
525
|
.map((entry) => join(entry.parentPath, entry.name));
|
|
460
|
-
if (files.length < 2) return files;
|
|
461
|
-
return files
|
|
462
|
-
.map((file) => ({ file, mtime: statSync(file).mtimeMs }))
|
|
463
|
-
.sort((a, b) => b.mtime - a.mtime)
|
|
464
|
-
.map(({ file }) => file);
|
|
465
526
|
}
|
|
466
527
|
|
|
528
|
+
/** Project session files, newest-modified first, optionally limited to one file key or to prefiltered candidates. */
|
|
467
529
|
async function* scopedSessionFiles(
|
|
468
|
-
dir: string, cwd: string, currentFile: string | undefined, signal?: AbortSignal,
|
|
530
|
+
dir: string, cwd: string, currentFile: string | undefined, signal?: AbortSignal,
|
|
531
|
+
{ fileKey, candidates }: { fileKey?: string; candidates?: Set<string> } = {},
|
|
469
532
|
): AsyncGenerator<string> {
|
|
470
533
|
const files = sessionFiles(dir);
|
|
471
534
|
const fileSet = new Set(files);
|
|
@@ -474,7 +537,7 @@ async function* scopedSessionFiles(
|
|
|
474
537
|
const known = visible.get(file);
|
|
475
538
|
if (known !== undefined) return known;
|
|
476
539
|
let header: { cwd?: string } | null = null;
|
|
477
|
-
for await (const line of
|
|
540
|
+
for await (const line of lfLines(file, signal)) {
|
|
478
541
|
try {
|
|
479
542
|
const entry = JSON.parse(line) as { type?: string; id?: unknown; cwd?: unknown } | null;
|
|
480
543
|
if (!entry) continue;
|
|
@@ -501,8 +564,12 @@ async function* scopedSessionFiles(
|
|
|
501
564
|
visible.set(file, allowed);
|
|
502
565
|
return allowed;
|
|
503
566
|
};
|
|
504
|
-
|
|
505
|
-
|
|
567
|
+
const selected = files
|
|
568
|
+
.filter((file) => (!candidates || candidates.has(resolve(file))) &&
|
|
569
|
+
(fileKey === undefined || fileKey.startsWith(shortKey(relative(dir, file)))))
|
|
570
|
+
.map((file) => ({ file, mtime: statSync(file).mtimeMs }))
|
|
571
|
+
.sort((a, b) => b.mtime - a.mtime);
|
|
572
|
+
for (const { file } of selected) {
|
|
506
573
|
if (await belongs(file)) yield file;
|
|
507
574
|
}
|
|
508
575
|
}
|
|
@@ -571,6 +638,14 @@ function formatRecoveryRecord(record: RecoveryRecord, limit: number): string {
|
|
|
571
638
|
return boundedBlock(`[${record.label} | ${record.timestamp} | entry ${record.id}]`, record.text, limit);
|
|
572
639
|
}
|
|
573
640
|
|
|
641
|
+
/** Owner inputs keep a one-line preview: they are direct intent, and most are short. */
|
|
642
|
+
function indexLine(record: RecoveryRecord): string {
|
|
643
|
+
const header = formatRecoveryRecord(record, 0);
|
|
644
|
+
if (record.kind !== "owner") return header;
|
|
645
|
+
const preview = record.text.replace(/\s+/g, " ").trim();
|
|
646
|
+
return `${header} ${preview.length > 120 ? `${preview.slice(0, 119)}…` : preview}`;
|
|
647
|
+
}
|
|
648
|
+
|
|
574
649
|
function formatPriorCheckpoint(entry: EntryLike | undefined, limit: number): string | undefined {
|
|
575
650
|
const handoff = windowHandoff(entry)?.trim();
|
|
576
651
|
if (!entry || !handoff) return undefined;
|
|
@@ -645,10 +720,7 @@ function recoverableToolResults(
|
|
|
645
720
|
function buildAutoHandoff(entries: readonly EntryLike[], projected: readonly ProjectedEntry[], maxChars: number): string {
|
|
646
721
|
const { start, boundary: priorWindow } = contextStart(entries);
|
|
647
722
|
const current = entries.slice(start);
|
|
648
|
-
const edits =
|
|
649
|
-
for (const entry of entries) {
|
|
650
|
-
if (entry.type === "context_edit" && entry.targetId) edits.set(entry.targetId, entry);
|
|
651
|
-
}
|
|
723
|
+
const edits = contextEdits(entries);
|
|
652
724
|
const omittedProjectedIds = new Set(projected.filter(({ messages }) => !messages.length).map(({ sourceEntry }) => sourceEntry.id));
|
|
653
725
|
const edited = current.flatMap((entry) => {
|
|
654
726
|
if (entry.id && omittedProjectedIds.has(entry.id)) return [];
|
|
@@ -721,16 +793,10 @@ function buildAutoHandoff(entries: readonly EntryLike[], projected: readonly Pro
|
|
|
721
793
|
? Math.min(MAX_RECOVERY_RECORD_CHARS, Math.floor(fixedBudget / fixedCount))
|
|
722
794
|
: MAX_RECOVERY_RECORD_CHARS;
|
|
723
795
|
const prior = formatPriorCheckpoint(priorWindow, fixedLimit);
|
|
724
|
-
const
|
|
725
|
-
records.map((record) => [
|
|
726
|
-
record,
|
|
727
|
-
formatRecoveryRecord(record, selected.has(record) ? fixedLimit : MAX_RECOVERY_RECORD_CHARS),
|
|
728
|
-
]),
|
|
729
|
-
);
|
|
730
|
-
const fixedParts = () => [
|
|
796
|
+
const fixedParts = [
|
|
731
797
|
preamble,
|
|
732
798
|
currentHeader,
|
|
733
|
-
...records.filter((record) => selected.has(record)).map((record) =>
|
|
799
|
+
...records.filter((record) => selected.has(record)).map((record) => formatRecoveryRecord(record, fixedLimit)),
|
|
734
800
|
];
|
|
735
801
|
|
|
736
802
|
let batch: string | undefined;
|
|
@@ -738,11 +804,11 @@ function buildAutoHandoff(entries: readonly EntryLike[], projected: readonly Pro
|
|
|
738
804
|
const availableText = Math.max(
|
|
739
805
|
0,
|
|
740
806
|
maxChars -
|
|
741
|
-
joinedLength([...fixedParts
|
|
807
|
+
joinedLength([...fixedParts, bareBatch, prior]) -
|
|
742
808
|
HANDOFF_OVERHEAD_RESERVE -
|
|
743
809
|
batchBlocks.length,
|
|
744
810
|
);
|
|
745
|
-
const perBlock = Math.min(
|
|
811
|
+
const perBlock = Math.min(MAX_RECOVERY_RESULT_CHARS, Math.floor(availableText / batchBlocks.length));
|
|
746
812
|
batch = [
|
|
747
813
|
batchHeader,
|
|
748
814
|
batchOmission,
|
|
@@ -752,21 +818,22 @@ function buildAutoHandoff(entries: readonly EntryLike[], projected: readonly Pro
|
|
|
752
818
|
.join("\n\n");
|
|
753
819
|
}
|
|
754
820
|
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
821
|
+
// Other inputs are indexed, newest kept first when space runs out; history read recovers their full text.
|
|
822
|
+
const others = records.filter((record) => !selected.has(record));
|
|
823
|
+
const indexHeader = "Other current-window inputs, oldest first (recover with history read):";
|
|
824
|
+
let indexBudget = Math.max(0, maxChars - joinedLength([...fixedParts, indexHeader, batch, prior]) - HANDOFF_OVERHEAD_RESERVE);
|
|
825
|
+
const listed: string[] = [];
|
|
826
|
+
for (let i = others.length - 1; i >= 0; i--) {
|
|
827
|
+
const line = indexLine(others[i]);
|
|
828
|
+
if (line.length + 1 > indexBudget) break;
|
|
829
|
+
listed.unshift(line);
|
|
830
|
+
indexBudget -= line.length + 1;
|
|
763
831
|
}
|
|
764
|
-
|
|
765
|
-
const
|
|
766
|
-
|
|
767
|
-
? `Omitted ${omitted.length} current-window input(s) to stay within the handoff limit (${omitted.filter((record) => record.kind === "owner").length} owner, ${omitted.filter((record) => record.kind === "coordination").length} coordination; ${omitted[0].timestamp} through ${omitted.at(-1)!.timestamp}). Use history search/read to recover them.`
|
|
832
|
+
const unlisted = others.length - listed.length;
|
|
833
|
+
const index = others.length
|
|
834
|
+
? [indexHeader, ...(unlisted ? [`${unlisted} earlier input(s) not listed; use history search to find them.`] : []), ...listed].join("\n")
|
|
768
835
|
: undefined;
|
|
769
|
-
return [...fixedParts
|
|
836
|
+
return [...fixedParts, index, batch, prior].filter((part): part is string => Boolean(part)).join("\n\n");
|
|
770
837
|
}
|
|
771
838
|
|
|
772
839
|
function reminderMatches(details: unknown, fingerprint: ReminderFingerprint): boolean {
|
|
@@ -792,9 +859,9 @@ function budgetFor(ctx: PolicyContext, contextWindow = ctx.model?.contextWindow)
|
|
|
792
859
|
}
|
|
793
860
|
|
|
794
861
|
/** Half the fresh capacity after prompt/tool/input overhead remains for continued work. */
|
|
795
|
-
function freshPayloadChars(ctx: PolicyContext, toolTokens: number, pendingMessages: readonly MessageLike[] = []): number {
|
|
862
|
+
function freshPayloadChars(ctx: PolicyContext, toolTokens: number, pendingMessages: readonly MessageLike[] = [], cap = MAX_HANDOFF_CHARS): number {
|
|
796
863
|
const contextWindow = ctx.model?.contextWindow;
|
|
797
|
-
if (!contextWindow || contextWindow <= 0) return
|
|
864
|
+
if (!contextWindow || contextWindow <= 0) return cap;
|
|
798
865
|
const budget = budgetFor(ctx, contextWindow);
|
|
799
866
|
const line = budget?.enabled && budget.supported ? budget.rolloverAt : contextWindow;
|
|
800
867
|
const promptTokens = Math.ceil(ctx.getSystemPrompt().length / 4);
|
|
@@ -803,7 +870,7 @@ function freshPayloadChars(ctx: PolicyContext, toolTokens: number, pendingMessag
|
|
|
803
870
|
0,
|
|
804
871
|
);
|
|
805
872
|
return Math.min(
|
|
806
|
-
|
|
873
|
+
cap,
|
|
807
874
|
Math.max(0, Math.floor((line - PAGE_MARGIN_TOKENS - promptTokens - toolTokens - pendingTokens) / 2)) * 4,
|
|
808
875
|
);
|
|
809
876
|
}
|
|
@@ -830,10 +897,7 @@ function boundWindowReceipts(
|
|
|
830
897
|
pending.reduce((sum, message) => sum + estimate(message), 0);
|
|
831
898
|
const fixed = messages.filter((message) => message.role !== "system" && message.role !== "toolResult");
|
|
832
899
|
const available = ctx.model.contextWindow - overhead - fixed.reduce((sum, message) => sum + estimate(message), 0);
|
|
833
|
-
const edits =
|
|
834
|
-
for (const entry of ctx.sessionManager.getBranch() as EntryLike[]) {
|
|
835
|
-
if (entry.type === "context_edit" && entry.targetId) edits.set(entry.targetId, entry);
|
|
836
|
-
}
|
|
900
|
+
const edits = contextEdits(ctx.sessionManager.getBranch() as EntryLike[]);
|
|
837
901
|
const receipts = projected.flatMap(({ sourceEntry, messages }) =>
|
|
838
902
|
messages.filter((message) => message.role === "toolResult").map((message) => {
|
|
839
903
|
if (!sourceEntry.id) throw new Error("Posthorse: retained receipt has no history entry");
|
|
@@ -876,8 +940,9 @@ function boundWindowReceipts(
|
|
|
876
940
|
return drafts;
|
|
877
941
|
}
|
|
878
942
|
|
|
879
|
-
/** Official Pi cannot expose its live settings to extensions; say
|
|
943
|
+
/** Official Pi cannot expose its live settings to extensions; budget reports say when they used the persisted snapshot. */
|
|
880
944
|
const SNAPSHOT_NOTE = "Reminder and budget policy uses available settings (a persisted CLI snapshot by default); Pi's live settings control automatic compaction.";
|
|
945
|
+
const CHECKPOINT_STEPS = "rewrite the task's current-state note in place with notes write (goal, progress, decisions, next steps), then call new_context";
|
|
881
946
|
|
|
882
947
|
function unsupportedMessage(budget: Budget): string {
|
|
883
948
|
const n = (value: number) => value.toLocaleString("en-US");
|
|
@@ -887,7 +952,7 @@ function unsupportedMessage(budget: Budget): string {
|
|
|
887
952
|
/** Capacity before Pi's automatic line (or configured context limit), including page metadata and refusals. */
|
|
888
953
|
function pageCapacity(ctx: PolicyContext, toolTokens: number): number {
|
|
889
954
|
const usage = ctx.getContextUsage();
|
|
890
|
-
if (!usage || usage.tokens == null) return freshPayloadChars(ctx, toolTokens) / 4 + PAGE_MARGIN_TOKENS;
|
|
955
|
+
if (!usage || usage.tokens == null) return freshPayloadChars(ctx, toolTokens, [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
|
|
891
956
|
const budget = budgetFor(ctx, usage.contextWindow);
|
|
892
957
|
const line = budget?.enabled && budget.supported ? budget.rolloverAt : usage.contextWindow;
|
|
893
958
|
return line - usage.tokens;
|
|
@@ -907,13 +972,12 @@ function buildGuidance(ctx: PolicyContext, official: boolean): string {
|
|
|
907
972
|
const deadline = budget
|
|
908
973
|
? `${Math.max(1, Math.round((budget.rolloverAt / budget.contextWindow) * 100))}% used`
|
|
909
974
|
: "the configured Pi context limit";
|
|
910
|
-
automatic = `Automatic
|
|
975
|
+
automatic = `Automatic rollover follows Pi's compaction setting. At most one checkpoint reminder may arrive before the rollover line (${deadline}); when it does, stop normal work, ${CHECKPOINT_STEPS}.`;
|
|
911
976
|
}
|
|
912
977
|
return `## Context self-management (Posthorse)
|
|
913
|
-
Context windows are finite. Use get_context_remaining for the best available native estimate when it matters; routine turns do not include a changing meter.${official ? `\n${SNAPSHOT_NOTE}` : ""}
|
|
914
978
|
${automatic}
|
|
915
|
-
|
|
916
|
-
|
|
979
|
+
After a rollover, earlier conversation stays in history. Restore notes and todos, then verify live state before stateful or external work; automatic handoffs record inputs, not progress.
|
|
980
|
+
Write handoffs and notes as normal readable prose; notes and history search match literal text.`;
|
|
917
981
|
}
|
|
918
982
|
|
|
919
983
|
function persistedPolicy(ctx: ExtensionContext): CompactionPolicy {
|
|
@@ -969,7 +1033,7 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
969
1033
|
previousPageUsage = usage;
|
|
970
1034
|
const remaining = pageCapacity(ctx, activeToolTokens()) - pendingPageTokens;
|
|
971
1035
|
const chars = Math.min(
|
|
972
|
-
|
|
1036
|
+
MAX_PAGE_CHARS,
|
|
973
1037
|
Math.max(0, remaining - PAGE_MARGIN_TOKENS) * 4 - imageCount * ESTIMATED_IMAGE_CHARS,
|
|
974
1038
|
);
|
|
975
1039
|
if (chars < MIN_PAGE_CHARS) {
|
|
@@ -1000,11 +1064,12 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1000
1064
|
let end = Math.min(text.length, offset + chars);
|
|
1001
1065
|
let start = 0;
|
|
1002
1066
|
const starts: number[] = [];
|
|
1067
|
+
const maxRows = kind === "notes-search" ? 20 : 100;
|
|
1003
1068
|
for (const row of rows) {
|
|
1004
1069
|
if (start >= end) break;
|
|
1005
1070
|
if (start + row.length > offset) starts.push(start);
|
|
1006
1071
|
start += row.length + 1;
|
|
1007
|
-
if (
|
|
1072
|
+
if (starts.length === maxRows) { end = Math.min(end, start - 1); break; }
|
|
1008
1073
|
}
|
|
1009
1074
|
// A result-count cap can require continuation even when the character cap did not.
|
|
1010
1075
|
if (end < text.length) end = Math.min(end, offset + chars - footer(text.length).length);
|
|
@@ -1073,7 +1138,7 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1073
1138
|
pi.sendMessage(
|
|
1074
1139
|
{
|
|
1075
1140
|
customType: REMINDER_TYPE,
|
|
1076
|
-
content: `[posthorse] Checkpoint now: ${(budget.rolloverAt - usage.tokens).toLocaleString("en-US")} tokens remain before Pi's automatic rollover line. Stop normal work,
|
|
1141
|
+
content: `[posthorse] Checkpoint now: ${(budget.rolloverAt - usage.tokens).toLocaleString("en-US")} tokens remain before Pi's automatic rollover line. Stop normal work, ${CHECKPOINT_STEPS}.`,
|
|
1077
1142
|
display: true,
|
|
1078
1143
|
details: fingerprint,
|
|
1079
1144
|
},
|
|
@@ -1084,16 +1149,16 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1084
1149
|
// Filter active input only; raw reminders remain in history for deduplication and recovery.
|
|
1085
1150
|
pi.on("context", (event, ctx) => {
|
|
1086
1151
|
if (!event.messages.some((message) => message.role === "custom" && isReminderType(message.customType))) return;
|
|
1087
|
-
|
|
1088
|
-
|
|
1089
|
-
|
|
1090
|
-
|
|
1091
|
-
|
|
1092
|
-
|
|
1152
|
+
// The fork's window marker saves a branch walk, but a compaction inside the window drops it.
|
|
1153
|
+
const marker = event.messages.find((message) => message.role === "custom" && message.customType === "context-window");
|
|
1154
|
+
const windowId = marker?.role === "custom"
|
|
1155
|
+
? (marker.details as { windowId?: unknown } | undefined)?.windowId
|
|
1156
|
+
: currentWindowId(ctx.sessionManager.getBranch() as EntryLike[]);
|
|
1157
|
+
if (typeof windowId !== "string") return;
|
|
1093
1158
|
const host = policy(ctx);
|
|
1094
1159
|
const budget = budgetFor(host);
|
|
1095
1160
|
const fingerprint: ReminderFingerprint = {
|
|
1096
|
-
windowId
|
|
1161
|
+
windowId,
|
|
1097
1162
|
contextWindow: budget?.contextWindow,
|
|
1098
1163
|
reserveTokens: budget?.reserveTokens,
|
|
1099
1164
|
};
|
|
@@ -1157,15 +1222,12 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1157
1222
|
label: "New Context",
|
|
1158
1223
|
...toolCards("new_context"),
|
|
1159
1224
|
description:
|
|
1160
|
-
"
|
|
1161
|
-
promptSnippet: "request a fresh context window with an optional
|
|
1162
|
-
promptGuidelines: [
|
|
1163
|
-
"Before calling new_context, pass concise continuation state in handoff or save durable goal/progress/decisions/next-steps with notes",
|
|
1164
|
-
],
|
|
1225
|
+
"Start a fresh context window once the current foreground tools succeed; background work continues across the reset. Earlier conversation stays recoverable through history. Put continuation state in handoff, or save fuller state in notes first.",
|
|
1226
|
+
promptSnippet: "request a fresh context window with an optional handoff",
|
|
1165
1227
|
parameters: Type.Object({
|
|
1166
1228
|
handoff: Type.Optional(
|
|
1167
1229
|
Type.String({
|
|
1168
|
-
description: "
|
|
1230
|
+
description: "What the fresh window needs to continue, as readable prose",
|
|
1169
1231
|
maxLength: MAX_HANDOFF_CHARS,
|
|
1170
1232
|
}),
|
|
1171
1233
|
),
|
|
@@ -1193,8 +1255,8 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1193
1255
|
label: "Context Remaining",
|
|
1194
1256
|
...toolCards("get_context_remaining"),
|
|
1195
1257
|
description:
|
|
1196
|
-
"
|
|
1197
|
-
promptSnippet: "check the
|
|
1258
|
+
"Estimate tokens left before Pi's automatic rollover and the configured context limit. A checkpoint reminder already arrives before rollover; use this only before a large step late in a window.",
|
|
1259
|
+
promptSnippet: "check the context budget before a large step late in a window",
|
|
1198
1260
|
parameters: Type.Object({}),
|
|
1199
1261
|
async execute(_id, _params, _signal, _onUpdate, ctx) {
|
|
1200
1262
|
const host = policy(ctx);
|
|
@@ -1222,18 +1284,14 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1222
1284
|
label: "Notes",
|
|
1223
1285
|
...toolCards("notes"),
|
|
1224
1286
|
description:
|
|
1225
|
-
"
|
|
1287
|
+
"Durable notes in .pi/notes/, shared across Git worktrees and kept across context resets. Ops: list (newest first, with size and date; optional folder path), read, search (case-insensitive substring over text notes), write (replace; empty content clears), append (adds one line-terminated record). list/read/search are paged; continue with the returned offset. Keep one current-state note per task and rewrite it in place.",
|
|
1226
1288
|
promptSnippet: "save and recall durable state that survives context resets",
|
|
1227
|
-
promptGuidelines: [
|
|
1228
|
-
"Use notes for durable state too large for a new_context handoff",
|
|
1229
|
-
"Reload relevant notes after a context rollover",
|
|
1230
|
-
],
|
|
1231
1289
|
parameters: Type.Object({
|
|
1232
1290
|
op: Type.Union(
|
|
1233
1291
|
[Type.Literal("list"), Type.Literal("read"), Type.Literal("write"), Type.Literal("append"), Type.Literal("search")],
|
|
1234
1292
|
{ description: "Operation to perform" },
|
|
1235
1293
|
),
|
|
1236
|
-
path: Type.Optional(Type.String({ description: "Note path relative to .pi/notes/
|
|
1294
|
+
path: Type.Optional(Type.String({ description: "Note or folder path relative to .pi/notes/" })),
|
|
1237
1295
|
content: Type.Optional(Type.String({ description: "Full file content (write) or text to add (append)" })),
|
|
1238
1296
|
query: Type.Optional(Type.String({ description: "Substring to find in notes (search)" })),
|
|
1239
1297
|
offset: Type.Optional(Type.Integer({ description: "Character offset for list/read/search (default 0)", minimum: 0 })),
|
|
@@ -1247,7 +1305,7 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1247
1305
|
}
|
|
1248
1306
|
return join(dir, relative);
|
|
1249
1307
|
};
|
|
1250
|
-
const walk = (directory: string, output:
|
|
1308
|
+
const walk = (directory: string, output: NoteFile[], ancestors = new Set<string>()) => {
|
|
1251
1309
|
const root = realpathSync(directory);
|
|
1252
1310
|
if (ancestors.has(root)) return;
|
|
1253
1311
|
ancestors.add(root);
|
|
@@ -1256,21 +1314,37 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1256
1314
|
const info = statSync(path, { throwIfNoEntry: false });
|
|
1257
1315
|
if (!info) continue;
|
|
1258
1316
|
if (info.isDirectory()) walk(path, output, ancestors);
|
|
1259
|
-
else output.push(path);
|
|
1317
|
+
else output.push({ path, size: info.size, mtimeMs: info.mtimeMs });
|
|
1260
1318
|
}
|
|
1261
1319
|
ancestors.delete(root);
|
|
1262
1320
|
};
|
|
1321
|
+
const notesUnder = (target: string): NoteFile[] => {
|
|
1322
|
+
const info = statSync(target, { throwIfNoEntry: false });
|
|
1323
|
+
if (!info) return [];
|
|
1324
|
+
const files: NoteFile[] = [];
|
|
1325
|
+
if (info.isDirectory()) walk(target, files);
|
|
1326
|
+
else files.push({ path: target, size: info.size, mtimeMs: info.mtimeMs });
|
|
1327
|
+
return files.sort((a, b) => b.mtimeMs - a.mtimeMs);
|
|
1328
|
+
};
|
|
1329
|
+
const list = (folder?: string) => notesPage(
|
|
1330
|
+
policy(ctx),
|
|
1331
|
+
notesUnder(folder ? safeJoin(folder) : dir).map((file) =>
|
|
1332
|
+
`${file.path.slice(dir.length + 1)} ${formatSize(file.size)} ${new Date(file.mtimeMs).toISOString().slice(0, 16)}Z`),
|
|
1333
|
+
params.offset ?? 0,
|
|
1334
|
+
"notes-list",
|
|
1335
|
+
folder ? `(no notes in ${folder})` : "(no notes yet)",
|
|
1336
|
+
);
|
|
1263
1337
|
|
|
1264
1338
|
switch (params.op) {
|
|
1265
|
-
case "list":
|
|
1266
|
-
|
|
1267
|
-
if (existsSync(dir)) walk(dir, files);
|
|
1268
|
-
return notesPage(policy(ctx), files.map((file) => file.slice(dir.length + 1)), params.offset ?? 0, "notes-list", "(no notes yet)");
|
|
1269
|
-
}
|
|
1339
|
+
case "list":
|
|
1340
|
+
return list(params.path);
|
|
1270
1341
|
case "read": {
|
|
1271
1342
|
const relative = requireValue(params.path, "path", params.op);
|
|
1272
1343
|
const path = safeJoin(relative);
|
|
1273
|
-
|
|
1344
|
+
const info = statSync(path, { throwIfNoEntry: false });
|
|
1345
|
+
if (!info) throw new Error(`No note at ${relative}. Use op "list" to see available notes.`);
|
|
1346
|
+
if (info.isDirectory()) return list(relative);
|
|
1347
|
+
if (await isBinaryFile(path)) throw new Error(`${relative} is a binary file (${formatSize(info.size)}); notes read returns text only.`);
|
|
1274
1348
|
const text = readFileSync(path, "utf8");
|
|
1275
1349
|
const offset = params.offset ?? 0;
|
|
1276
1350
|
if (offset && offset >= text.length) {
|
|
@@ -1307,15 +1381,16 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1307
1381
|
}
|
|
1308
1382
|
case "search": {
|
|
1309
1383
|
const query = requireValue(params.query, "query", params.op).toLowerCase();
|
|
1310
|
-
const files: string[] = [];
|
|
1311
|
-
if (existsSync(dir)) walk(dir, files);
|
|
1312
1384
|
const hits: string[] = [];
|
|
1313
|
-
for (const file of
|
|
1314
|
-
|
|
1385
|
+
for (const { path: file } of notesUnder(dir)) {
|
|
1386
|
+
if (await isBinaryFile(file)) continue;
|
|
1387
|
+
let number = 0;
|
|
1388
|
+
for await (const line of lfLines(file, signal)) {
|
|
1389
|
+
number++;
|
|
1315
1390
|
const trimmed = line.trim();
|
|
1316
1391
|
const match = trimmed.toLowerCase().indexOf(query);
|
|
1317
1392
|
if (match !== -1) {
|
|
1318
|
-
hits.push(`${file.slice(dir.length + 1)}:${
|
|
1393
|
+
hits.push(`${file.slice(dir.length + 1)}:${number}: ${excerptAround(trimmed, match, 50, 200)}`);
|
|
1319
1394
|
}
|
|
1320
1395
|
}
|
|
1321
1396
|
}
|
|
@@ -1330,28 +1405,45 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1330
1405
|
label: "History",
|
|
1331
1406
|
...toolCards("history"),
|
|
1332
1407
|
description:
|
|
1333
|
-
"Search or read
|
|
1408
|
+
"Search or read earlier session entries, including previous context windows. search skips entries still in your active context and lists original content before recovery echoes, newest first; all=true also searches this project's other sessions and subagent runs. read takes an entry id exactly as search printed it and pages text and images.",
|
|
1334
1409
|
promptSnippet: "recover earlier conversation that left the active context window",
|
|
1335
|
-
promptGuidelines: ["Use history search first, then history read with the
|
|
1410
|
+
promptGuidelines: ["Use history search first, then history read with the entry id exactly as printed"],
|
|
1336
1411
|
parameters: Type.Object({
|
|
1337
1412
|
op: Type.Union([Type.Literal("search"), Type.Literal("read")], { description: "Operation to perform" }),
|
|
1338
1413
|
query: Type.Optional(Type.String({ description: "Case-insensitive text to find (search)" })),
|
|
1339
|
-
id: Type.Optional(Type.String({ description: "Entry id
|
|
1340
|
-
all: Type.Optional(Type.Boolean({ description: "
|
|
1414
|
+
id: Type.Optional(Type.String({ description: "Entry id exactly as search printed it (read)" })),
|
|
1415
|
+
all: Type.Optional(Type.Boolean({ description: "Also search this project's other sessions and subagent runs" })),
|
|
1341
1416
|
limit: Type.Optional(Type.Integer({ description: "Maximum search results per page (default 10, max 50)", minimum: 1, maximum: 50 })),
|
|
1342
|
-
cursor: Type.Optional(Type.String({ description: "
|
|
1417
|
+
cursor: Type.Optional(Type.String({ description: "Continuation from the previous search page; keep query and all unchanged", maxLength: 512 })),
|
|
1343
1418
|
offset: Type.Optional(Type.Integer({ description: "Character offset for read (default 0)", minimum: 0 })),
|
|
1344
|
-
imageOffset: Type.Optional(Type.Integer({ description: "First image to return
|
|
1419
|
+
imageOffset: Type.Optional(Type.Integer({ description: "First image to return (read); use the value from the continuation", minimum: 0 })),
|
|
1345
1420
|
}),
|
|
1346
1421
|
async execute(_id, params, signal, _onUpdate, ctx) {
|
|
1347
1422
|
const manager = ctx.sessionManager;
|
|
1348
1423
|
const cwd = resolve(ctx.cwd);
|
|
1349
1424
|
const currentFile = manager.getSessionFile?.();
|
|
1425
|
+
const dir = manager.getSessionDir();
|
|
1350
1426
|
|
|
1351
1427
|
if (params.op === "search") {
|
|
1352
1428
|
const query = requireValue(params.query, "query", params.op).toLowerCase();
|
|
1353
1429
|
const limit = params.limit ?? 10;
|
|
1354
|
-
const searchKey =
|
|
1430
|
+
const searchKey = shortKey(JSON.stringify([query, params.all === true]));
|
|
1431
|
+
// Projections show replacements and stripped async calls under the original id, so an entry is in
|
|
1432
|
+
// context only when all of its own text is visible there. Posthorse strips stale reminders from input.
|
|
1433
|
+
const seen = (message: MessageLike) => [
|
|
1434
|
+
textOf(message), assistantFailure(message), imageSummary(imagesOf(message.content)), message.summary, message.command, message.output,
|
|
1435
|
+
].filter(Boolean).join("\n").toLowerCase();
|
|
1436
|
+
const visible = new Map((manager.buildSessionProjection().entries as readonly ProjectedEntry[]).flatMap(({ sourceEntry, messages }) =>
|
|
1437
|
+
sourceEntry.id && messages.length ? [[sourceEntry.id, messages.map(seen).join("\n")] as const] : []));
|
|
1438
|
+
const inContext = (entry: EntryLike) => {
|
|
1439
|
+
if (entry.type === "custom_message" && isReminderType(entry.customType)) return false;
|
|
1440
|
+
const own = entry.type === "message" ? entry.message ?? {}
|
|
1441
|
+
: entry.type === "context_edit" ? { content: entry.replacement?.content }
|
|
1442
|
+
: entry.type === "context_window" ? { content: entry.handoff }
|
|
1443
|
+
: { content: entry.content, summary: entry.summary };
|
|
1444
|
+
return visible.get((entry.type === "context_edit" ? entry.targetId : entry.id) ?? "")?.includes(seen(own)) === true;
|
|
1445
|
+
};
|
|
1446
|
+
let skipped = 0;
|
|
1355
1447
|
let cursor: [string, 0 | 1, number, string] | undefined;
|
|
1356
1448
|
if (params.cursor) {
|
|
1357
1449
|
try {
|
|
@@ -1375,13 +1467,21 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1375
1467
|
};
|
|
1376
1468
|
|
|
1377
1469
|
if (params.all) {
|
|
1378
|
-
|
|
1470
|
+
const needles = prefilterNeedles(query);
|
|
1471
|
+
const candidates = needles && await filesContaining(dir, needles, signal);
|
|
1472
|
+
const current = currentFile && resolve(currentFile);
|
|
1473
|
+
for await (const file of scopedSessionFiles(dir, cwd, currentFile, signal, { candidates })) {
|
|
1379
1474
|
const recent: HistoryHit[][] = [[], []];
|
|
1380
1475
|
let anchorHere = false;
|
|
1381
|
-
const source = relative(
|
|
1476
|
+
const source = relative(dir, file);
|
|
1477
|
+
const inCurrent = resolve(file) === current;
|
|
1382
1478
|
for await (const item of sessionWindowEntries(file, signal)) {
|
|
1383
1479
|
const hit = historyHit(item, query, source);
|
|
1384
1480
|
if (!hit) continue;
|
|
1481
|
+
if (inCurrent && inContext(item.entry)) {
|
|
1482
|
+
skipped++;
|
|
1483
|
+
continue;
|
|
1484
|
+
}
|
|
1385
1485
|
const seeking = cursor && !found && hit.priority === cursor[1];
|
|
1386
1486
|
if (seeking && anchorHere) continue;
|
|
1387
1487
|
if (seeking && hit.id === cursor?.[0]) anchorHere = true;
|
|
@@ -1397,7 +1497,13 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1397
1497
|
} else {
|
|
1398
1498
|
const current = [...windowEntries(manager.getBranch() as EntryLike[])];
|
|
1399
1499
|
for (const item of current.reverse()) {
|
|
1400
|
-
|
|
1500
|
+
const hit = historyHit(item, query);
|
|
1501
|
+
if (!hit) continue;
|
|
1502
|
+
if (inContext(item.entry)) {
|
|
1503
|
+
skipped++;
|
|
1504
|
+
continue;
|
|
1505
|
+
}
|
|
1506
|
+
addHit(hit);
|
|
1401
1507
|
if (hits[0].length > limit) break;
|
|
1402
1508
|
}
|
|
1403
1509
|
}
|
|
@@ -1408,9 +1514,12 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1408
1514
|
const footer = (next: string) => `\n[More results; continue with cursor "${next}" and the same query/scope.]`;
|
|
1409
1515
|
const reserve = Math.max(0, ...results.map((hit) => footer(cursorFor(hit, hit.text.length)).length));
|
|
1410
1516
|
if (reserve >= chars) pageError(policy(ctx), "History entry id is too large for pagination.", params.cursor);
|
|
1517
|
+
const skippedNote = skipped && !params.cursor
|
|
1518
|
+
? `\n[Skipped ${skipped} match${skipped === 1 ? "" : "es"} already in your active context.]`
|
|
1519
|
+
: "";
|
|
1411
1520
|
const parts: string[] = [];
|
|
1412
1521
|
const spans: Array<{ headerLength: number; length: number }> = [];
|
|
1413
|
-
let available = chars - reserve;
|
|
1522
|
+
let available = chars - reserve - skippedNote.length;
|
|
1414
1523
|
let next: string | undefined;
|
|
1415
1524
|
for (const hit of results) {
|
|
1416
1525
|
const offset = cursor?.[0] === hit.id ? cursor[2] : 0;
|
|
@@ -1427,8 +1536,8 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1427
1536
|
if (available <= 0 || parts.length >= limit) break;
|
|
1428
1537
|
}
|
|
1429
1538
|
const body = parts.length ? parts.join("\n") : `No history matches "${excerpt(params.query!, 200)}".`;
|
|
1430
|
-
const
|
|
1431
|
-
return pageResult(body +
|
|
1539
|
+
const tail = `${skippedNote}${next ? footer(next) : ""}`;
|
|
1540
|
+
return pageResult(body + tail, [], { kind: "history-search", entries: spans, footerLength: tail.length, more: next !== undefined, skipped });
|
|
1432
1541
|
}
|
|
1433
1542
|
|
|
1434
1543
|
const id = requireValue(params.id, "id", params.op);
|
|
@@ -1436,6 +1545,8 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1436
1545
|
const entryId = separator < 0 ? id : id.slice(0, separator);
|
|
1437
1546
|
const fileKey = separator < 0 ? undefined : id.slice(separator + 1);
|
|
1438
1547
|
const formatEntry = (item: WindowedEntry, source = "") => {
|
|
1548
|
+
// Session-file reads print the qualified id, so continuations reopen the same file.
|
|
1549
|
+
const shownId = source ? `${item.entry.id}@${shortKey(source)}` : id;
|
|
1439
1550
|
const offset = params.offset ?? 0;
|
|
1440
1551
|
const imageOffset = params.imageOffset ?? (offset === 0 ? 0 : item.images.length);
|
|
1441
1552
|
if (imageOffset > item.images.length) {
|
|
@@ -1454,8 +1565,8 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1454
1565
|
offset + chars - (images.length - firstImage) * ESTIMATED_IMAGE_CHARS,
|
|
1455
1566
|
);
|
|
1456
1567
|
const more = end < item.text.length || imageEnd < item.images.length
|
|
1457
|
-
? `\nMore remains; call history read with id "${
|
|
1458
|
-
const header = `${
|
|
1568
|
+
? `\nMore remains; call history read with id "${shownId}" and offset ${end}${item.images.length ? ` and imageOffset ${imageEnd}` : ""}.` : "";
|
|
1569
|
+
const header = `${item.entry.timestamp ?? ""}${sourceTag(source)} [window ${item.windowId}] [${shownId}] [chars ${offset}-${end} of ${item.text.length}] `;
|
|
1459
1570
|
return pageResult(
|
|
1460
1571
|
`${header}${item.text.slice(offset, end)}${more}`,
|
|
1461
1572
|
images,
|
|
@@ -1468,13 +1579,17 @@ export const createPosthorse = (getPolicy: (ctx: ExtensionContext) => Compaction
|
|
|
1468
1579
|
if (item.entry.id === entryId) return formatEntry(item);
|
|
1469
1580
|
}
|
|
1470
1581
|
}
|
|
1471
|
-
|
|
1472
|
-
|
|
1582
|
+
// Bare ids from other sessions reach here through older notes and handoffs.
|
|
1583
|
+
const candidates = fileKey === undefined && /^[\w-]+$/.test(entryId)
|
|
1584
|
+
? await filesContaining(dir, [`"id":"${entryId}"`], signal)
|
|
1585
|
+
: undefined;
|
|
1586
|
+
for await (const file of scopedSessionFiles(dir, cwd, currentFile, signal, { fileKey, candidates })) {
|
|
1587
|
+
const fileSource = relative(dir, file);
|
|
1473
1588
|
for await (const item of sessionWindowEntries(file, signal)) {
|
|
1474
1589
|
if (item.entry.id === entryId) return formatEntry(item, fileSource);
|
|
1475
1590
|
}
|
|
1476
1591
|
}
|
|
1477
|
-
throw new Error(`No history entry
|
|
1592
|
+
throw new Error(`No history entry "${id}" in this session or this project's other sessions. Pass an id exactly as history search prints it, such as 1a2f07de, or 1a2f07de@Ab3dE5fG7h from another session; search first when you only have text or a timestamp.`);
|
|
1478
1593
|
},
|
|
1479
1594
|
});
|
|
1480
1595
|
};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-posthorse",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Posthorse: fresh context, same journey. No-summary context rollover for Pi, with sparse budget reminders, handoffs, notes, and history.",
|
|
6
6
|
"keywords": ["pi-package", "pi-extension", "context", "posthorse", "rollover"],
|
package/ui.ts
CHANGED
|
@@ -7,7 +7,7 @@ export type PosthorseDisplay =
|
|
|
7
7
|
| { kind: "notes-list" | "notes-search"; count: number; page?: { offset: number; end: number; total: number } }
|
|
8
8
|
| { kind: "note-read"; offset: number; end: number; total: number }
|
|
9
9
|
| { kind: "note-write" | "note-append" | "new-context" }
|
|
10
|
-
| { kind: "history-search"; entries: Array<{ headerLength: number; length: number }>; footerLength?: number }
|
|
10
|
+
| { kind: "history-search"; entries: Array<{ headerLength: number; length: number }>; footerLength?: number; more?: boolean; skipped?: number }
|
|
11
11
|
| { kind: "history-read"; headerLength: number; offset: number; end: number; total: number; imageOffset?: number; imageEnd?: number; imageTotal?: number };
|
|
12
12
|
|
|
13
13
|
type ToolName = "notes" | "history" | "get_context_remaining" | "new_context";
|
|
@@ -62,7 +62,7 @@ export function toolCards(name: ToolName): Pick<ToolDefinition, "renderCall" | "
|
|
|
62
62
|
const args = argsOf(rawArgs);
|
|
63
63
|
const action = name === "notes" || name === "history" ? ` · ${argument(args.op, "op")}` : name === "new_context" ? " · request" : "";
|
|
64
64
|
const status = context.isError ? " · error" : context.isPartial ? context.executionStarted ? " · running" : " · preparing" : "";
|
|
65
|
-
const target = args.op === "search" ? ` · “${argument(args.query, "query")}”` : name === "notes" && args.op !== "list" ? ` · ${argument(args.path, "path")}` : name === "history" && args.op === "read" ? ` · ${argument(args.id, "id")}` : "";
|
|
65
|
+
const target = args.op === "search" ? ` · “${argument(args.query, "query")}”` : name === "notes" && (args.op !== "list" || args.path !== undefined) ? ` · ${argument(args.path, "path")}` : name === "history" && args.op === "read" ? ` · ${argument(args.id, "id")}` : "";
|
|
66
66
|
const label = `${context.isPartial ? "…" : context.expanded ? "▾" : "▸"} ${titles[name]}${action}${status}${target}`;
|
|
67
67
|
return {
|
|
68
68
|
invalidate() {},
|
|
@@ -103,7 +103,7 @@ export function toolCards(name: ToolName): Pick<ToolDefinition, "renderCall" | "
|
|
|
103
103
|
case "note-read": summary = pageSummary(display); break;
|
|
104
104
|
case "note-write": summary = args.content === "" ? "Cleared note" : "Saved note"; break;
|
|
105
105
|
case "note-append": summary = "Appended to note"; break;
|
|
106
|
-
case "history-search": if (Array.isArray(display.entries) && (!display.entries.length || sections)) summary = `${display.entries.length ? `${n(display.entries.length)} matches` : "No matches"} · ${args.all === true ? "all sessions" : "current branch"}${display.footerLength ? "\nMore results; continue with cursor" : ""}`; break;
|
|
106
|
+
case "history-search": if (Array.isArray(display.entries) && (!display.entries.length || sections)) summary = `${display.entries.length ? `${n(display.entries.length)} matches` : "No matches"} · ${args.all === true ? "all sessions" : "current branch"}${display.skipped ? ` · ${n(display.skipped)} in context skipped` : ""}${(display.more ?? display.footerLength) ? "\nMore results; continue with cursor" : ""}`; break;
|
|
107
107
|
case "history-read":
|
|
108
108
|
summary = pageSummary(display);
|
|
109
109
|
if (display.imageTotal && display.imageEnd !== undefined) {
|