pi-posthorse 0.4.5 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -19
- package/index.ts +244 -101
- package/package.json +5 -3
- package/ui.ts +13 -9
package/README.md
CHANGED
|
@@ -8,10 +8,12 @@ Native, no-summary context windows for the [`fitchmultz/pi`](https://github.com/
|
|
|
8
8
|
|
|
9
9
|
Pi owns the persisted boundary. Posthorse owns the policy: stable window guidance, one best-effort checkpoint reminder, `new_context`, `get_context_remaining`, durable `notes`, and window-aware `history`. A rollover removes the old window from active model context while the JSONL transcript stays append-only and complete.
|
|
10
10
|
|
|
11
|
+
An isolated [Codex prototype](adapters/codex-posthorse/README.md) tests local notes and recovery with Codex's native context reset. It is not ready for everyday use and is not included in the Pi package.
|
|
12
|
+
|
|
11
13
|
## Requirements
|
|
12
14
|
|
|
13
15
|
- Node `>=22.19.0`.
|
|
14
|
-
- The `fitchmultz/pi` fork. The
|
|
16
|
+
- The `fitchmultz/pi` fork. The native qualification target is `8fb7886130ff1fedc415bdd6fea03aef8a8957d8` (fork package version `0.87.0`). Posthorse needs the fork's native `context_window` entries, its `session_before_auto_compact` hook, `ctx.getCompactionSettings()`, and the coding-agent SDK's `publishLocalFile` export.
|
|
15
17
|
- Official, unpatched Pi is unsupported. Posthorse reports a clear extension error at session start and cannot operate; Pi itself keeps running.
|
|
16
18
|
|
|
17
19
|
## Install
|
|
@@ -21,7 +23,7 @@ Build the fork:
|
|
|
21
23
|
```bash
|
|
22
24
|
git clone https://github.com/fitchmultz/pi.git
|
|
23
25
|
cd pi
|
|
24
|
-
git checkout
|
|
26
|
+
git checkout 8fb7886130ff1fedc415bdd6fea03aef8a8957d8
|
|
25
27
|
npm install --ignore-scripts
|
|
26
28
|
npm run build
|
|
27
29
|
```
|
|
@@ -46,15 +48,15 @@ Keep exactly one copy loaded. `pi list` shows every package source; if an older
|
|
|
46
48
|
|
|
47
49
|
## How it works
|
|
48
50
|
|
|
49
|
-
1. **Stable guidance.** Window behavior
|
|
50
|
-
2. **One best-effort checkpoint.** While Pi compaction is enabled, one reminder may appear shortly before Pi's rollover line. A large turn, overflow, restart, or smaller model can reach rollover without it. Reminders are fingerprinted by window, context size, and reserve, so switching to a different context size gets a fresh reminder and
|
|
51
|
-
3. **`get_context_remaining`.** Reports the best available native estimate of tokens until Pi's automatic rollover line and until the
|
|
51
|
+
1. **Stable guidance.** Window behavior lives in a native system-prompt section. There is no per-request meter. Posthorse preserves custom prompts and falls back to appending guidance when an earlier extension replaces the full prompt; that compatibility path can prevent Pi from preserving an additive prompt prefix.
|
|
52
|
+
2. **One best-effort checkpoint.** While Pi compaction is enabled, one reminder may appear shortly before Pi's rollover line. A large turn, overflow, restart, or smaller model can reach rollover without it. Reminders are fingerprinted by window, context size, and reserve, so switching to a different context size gets a fresh reminder. Stale reminders and reminders left after disabling compaction are filtered from model input while remaining in history.
|
|
53
|
+
3. **`get_context_remaining`.** Reports the best available native estimate of tokens until Pi's automatic rollover line and until the configured context limit. That configured limit is not a measured provider rejection boundary. Pi's value is an estimate until the active model reports usage.
|
|
52
54
|
4. **`new_context`.** Requests an atomic rollover after the complete tool batch succeeds. An optional handoff is persisted and becomes the first state of the fresh window. If a sibling tool in the same batch fails, Pi does not commit the boundary; the checkpoint reminder still applies.
|
|
53
55
|
5. **Automatic rollover without summaries.** With a supported context budget and room for a recovery record, Posthorse claims Pi's automatic threshold and overflow trigger through `session_before_auto_compact`, before Pi resolves summarization credentials or prepares a summary. Oversized first turns and tool results can then roll over even without summarization credentials. Otherwise Pi's own compaction remains in control. Manual `/compact` is unchanged.
|
|
54
|
-
6. **Bounded recovery record.** The automatic handoff keeps direct user inputs, `ask_question` outcomes, visible coordination messages, and the trailing tool batch
|
|
55
|
-
7. **`notes` and `history`.** Notes
|
|
56
|
+
6. **Bounded recovery record.** The automatic handoff keeps direct user inputs, `ask_question` outcomes, visible coordination messages, and the trailing tool batch without a later complete assistant response: call arguments, bounded result text, and entry ids to recover the rest. Failed or interrupted requests may already have received those results. A clearly labeled, possibly stale older checkpoint comes last. Older assistant prose and earlier tool batches are not treated as current state. Newly submitted input stays separate and is saved after the boundary, not copied into the handoff.
|
|
57
|
+
7. **`notes` and `history`.** Notes are shared across linked worktrees, at the main checkout for conventional Git layouts or inside the common Git directory when metadata is stored separately. History searches normalized transcript text and returns stored images for a requested entry.
|
|
56
58
|
|
|
57
|
-
At turn end, Posthorse checks whether usage is in the reminder band before explicitly looking up the full branch. Context filtering skips its branch lookup when model input contains neither `posthorse-reminder` nor legacy `headroom-reminder` messages.
|
|
59
|
+
At turn end, Posthorse checks whether usage is in the reminder band before explicitly looking up the full branch. Context filtering skips its branch lookup when model input contains neither `posthorse-reminder` nor legacy `headroom-reminder` messages. Complete history remains available through paged searches and reads.
|
|
58
60
|
|
|
59
61
|
Automatic recovery is an emergency input record, not proof of progress. The fresh model is told to restore notes and todo state, inspect history when needed, and verify live state before taking stateful or external action.
|
|
60
62
|
|
|
@@ -72,27 +74,29 @@ Only one automatic compaction or rollover policy extension should be enabled at
|
|
|
72
74
|
|
|
73
75
|
- `new_context({ handoff? })`
|
|
74
76
|
- `get_context_remaining()`
|
|
75
|
-
- `notes({ op, ... })`: `list`, `read`
|
|
76
|
-
- `history({ op, ... })`: `search
|
|
77
|
+
- `notes({ op, ... })`: `list`, `read`, and `search` are paged; continue with the returned character `offset`. Search excerpts center on the match. `write` replaces content (empty content clears); `append` adds one newline-terminated record.
|
|
78
|
+
- `history({ op, ... })`: `search` continues with the returned `cursor` and the same `query` and `all` scope; `limit` is the maximum results per page. `read` returns stored images with the first page and the next character `offset` when text remains. Results keep native entry and window ids.
|
|
77
79
|
|
|
78
80
|
In the TUI, tools use Pi's native expandable cards. Collapsed cards show the operation and target, a short content preview, and counts or page ranges with the next offset when more remains. Expand with Pi's tool-output shortcut (`Ctrl+O` by default), or click the card's header or body in fullscreen mode. Expanded cards show the complete returned page and its metadata, not content the tool has not fetched yet. Writes and context requests also show the submitted content or handoff.
|
|
79
81
|
|
|
80
82
|
Committed context-window messages and checkpoint reminders are compact, expandable cards too. The `new_context` tool card describes a request; only the committed context-window message says a fresh window has started.
|
|
81
83
|
|
|
82
|
-
|
|
84
|
+
Notes list/search and history search share the remaining context budget with read pages and returned images. Search/list headers, continuation text, no-match responses, and parallel sibling results count toward that budget; pages already counted by Pi are not counted twice. Before usage is known, pages reserve prompt/tool overhead and leave half the rest free. Unsafe pages are refused with the offset or cursor preserved in the call; call `new_context` and retry. Refusal text uses the same budget, so later failures omit repeated guidance once no more fits.
|
|
83
85
|
|
|
84
86
|
`history search` puts matching original content before recovery material: handoffs, compaction and branch summaries, checkpoint reminders, and `notes`, `new_context`, and `history` calls/results. Ordinary prose or another tool call in the same assistant entry keeps its priority when that content matches. Every entry remains searchable; `history read` returns the complete normalized entry, including any recovery content omitted from a search excerpt.
|
|
85
87
|
|
|
86
|
-
Within each group, current-branch matches are newest first. With `all: true`, Posthorse searches every session file in the active Pi session directory, newest-modified sessions first and newest entries within each session; this is not a global timestamp sort. The result limit applies after priority, so newer echoes cannot displace older original matches. Entries copied by a fork are reported once.
|
|
88
|
+
Within each group, current-branch matches are newest first. With `all: true`, Posthorse searches every session file in the active Pi session directory, newest-modified sessions first and newest entries within each session; this is not a global timestamp sort. The per-page result limit applies after priority, so newer echoes cannot displace older original matches. Entries copied by a fork are reported once. Search cursors advance past the previous entry, including partial headers or excerpts, so appending lookup calls and results does not keep pagination alive forever. Searches are live: external session edits or file reordering can change results; start a new search to include newer sources.
|
|
89
|
+
|
|
90
|
+
Notes live in `.pi/notes/` at the main checkout for conventional Git layouts, or the current directory outside Git. When Git metadata is stored separately, including in submodules, all checkouts use `.pi/notes/` inside Git's common directory instead; no configuration is needed. If that metadata is unavailable, notes stay local to the checkout. Old checkout-local notes are imported when that checkout is accessed, without overwriting shared files or deleting the originals. Writes report the actual storage path. Add `.pi/notes/` to `.gitignore` when the project should not track it.
|
|
87
91
|
|
|
88
|
-
|
|
92
|
+
`write` uses Pi's native complete-file replacement: a failed write before publication leaves the previous note intact. It follows symlink targets and preserves ordinary permissions and ownership, or fails before publication. Hardlink aliases and already-open handles keep the previous file; ACLs, extended attributes, and power-loss durability are not guaranteed. Full replacements do not merge concurrent appends or other replacements. `append` retains one `O_APPEND` write per newline-terminated record.
|
|
89
93
|
|
|
90
94
|
## Data and privacy
|
|
91
95
|
|
|
92
96
|
- Posthorse makes no network requests.
|
|
93
97
|
- Notes are plaintext files under `.pi/notes`. They survive package removal and may be committed unless ignored.
|
|
94
98
|
- `history` with `all: true` scans nested JSONL files in the active Pi session directory, including subagent sessions.
|
|
95
|
-
- History can return user text, assistant text and thinking, tool arguments and results, handoffs, custom messages, and images. Direct shell entries Pi marked `excludeFromContext` come back as a placeholder only.
|
|
99
|
+
- History can return user text, assistant text and thinking, assistant failure status and error messages, tool arguments and results, handoffs, custom messages, and images. Direct shell entries Pi marked `excludeFromContext` come back as a placeholder only.
|
|
96
100
|
- Returned history content enters the currently selected model and provider context.
|
|
97
101
|
- Removing Posthorse does not remove notes or Pi session history.
|
|
98
102
|
|
|
@@ -103,10 +107,15 @@ Reminders persisted by pi-headroom (`headroom-reminder`) are recognized alongsid
|
|
|
103
107
|
## Develop
|
|
104
108
|
|
|
105
109
|
```bash
|
|
106
|
-
npm ci
|
|
107
|
-
npm
|
|
108
|
-
|
|
109
|
-
|
|
110
|
+
npm ci --ignore-scripts
|
|
111
|
+
npm run check # public npm declarations can check types, but cannot run Posthorse
|
|
112
|
+
# After the compatibility runner installs the selected fork SDK/types cohort:
|
|
113
|
+
PI_COMPAT_HOST=fork npm run check:compat
|
|
114
|
+
PI_FORK=../pi scripts/integration.sh # additional full source-harness integration (disposable fork built)
|
|
110
115
|
```
|
|
111
116
|
|
|
112
|
-
`npm run check` type-checks the extension, renderers, and tests
|
|
117
|
+
`npm run check` type-checks the extension, renderers, and tests against the exact 0.87.0 npm declaration baseline. Runtime tests require the qualified fork SDK, including `publishLocalFile`, installed in this checkout's dependency graph. `npm test` covers reminder policy, notes/history behavior, failed note replacement, and real native card components, including width, expansion, and legacy results. Tests use the public SDK entry; official Pi is not an operating target.
|
|
118
|
+
|
|
119
|
+
`check:compat` adds `test:native`: real SDK loading, prompt composition, mixed recovery-output budgets, cursor progress, reminder filtering, interrupted-response recovery, oversized-result rollover without summary compaction, explicit `new_context`, and checkpoint restore without provider replay. It requires native fork capabilities; it never substitutes synthetic windows or silently skips on official. The runner must install the fork cohort in this checkout's `node_modules` so both `tsc` and SDK imports resolve that graph. `PI_HOST_INDEX` and `PI_COMPAT_EXPECTED_PACKAGE_DIR` are verified against it, and `PI_COMPAT_EXPECTED_VERSION` verifies the version. A same-version official graph is not a fork. Official qualification is a separate actual CLI startup/refusal probe, not `check:compat`.
|
|
120
|
+
|
|
121
|
+
For a focused read-only SDK diagnostic only, `PI_HOST_INDEX=/absolute/fork/dist/index.js node --test test/native.test.mjs` can inspect an immutable host; this does not qualify this checkout's types. The larger source integration suite remains available through `scripts/integration.sh`; use a disposable built fork checkout because it briefly copies a test into that host. All native fixtures must use an isolated HOME/agent directory with offline mode and no provider credentials. These checks do not claim native Windows support.
|
package/index.ts
CHANGED
|
@@ -6,24 +6,27 @@
|
|
|
6
6
|
* notes, and history recovery.
|
|
7
7
|
*/
|
|
8
8
|
|
|
9
|
+
import { createHash } from "node:crypto";
|
|
9
10
|
import {
|
|
10
11
|
appendFileSync,
|
|
12
|
+
constants,
|
|
13
|
+
copyFileSync,
|
|
11
14
|
createReadStream,
|
|
12
15
|
existsSync,
|
|
13
16
|
mkdirSync,
|
|
14
17
|
readdirSync,
|
|
15
18
|
readFileSync,
|
|
19
|
+
realpathSync,
|
|
16
20
|
statSync,
|
|
17
|
-
writeFileSync,
|
|
18
21
|
} from "node:fs";
|
|
19
|
-
import { dirname, isAbsolute, join, normalize, relative, resolve, sep } from "node:path";
|
|
20
|
-
import
|
|
22
|
+
import { basename, dirname, isAbsolute, join, normalize, relative, resolve, sep } from "node:path";
|
|
23
|
+
import * as codingAgent from "@earendil-works/pi-coding-agent";
|
|
21
24
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
22
25
|
import { Type } from "typebox";
|
|
23
26
|
import { registerPosthorseMessages, toolCards, type PosthorseDisplay } from "./ui.ts";
|
|
24
27
|
|
|
25
28
|
const REMINDER_BUFFER_TOKENS = 32_000;
|
|
26
|
-
/** Absolute ceiling for handoffs and
|
|
29
|
+
/** Absolute ceiling for handoffs and recovery pages; pages shrink to the live remaining budget. */
|
|
27
30
|
const MAX_HANDOFF_CHARS = 20_000;
|
|
28
31
|
const MAX_RECOVERY_RECORD_CHARS = 4_000;
|
|
29
32
|
const HANDOFF_OVERHEAD_RESERVE = 1_000;
|
|
@@ -41,6 +44,11 @@ const LEGACY_AUTO_HANDOFF_PREFIX =
|
|
|
41
44
|
|
|
42
45
|
type CompactionPolicy = { enabled: boolean; reserveTokens: number };
|
|
43
46
|
type ContextUsage = { tokens: number | null; contextWindow: number; percent: number | null };
|
|
47
|
+
/** Fork-only SDK export; the public npm declarations do not include it. */
|
|
48
|
+
const { publishLocalFile } = codingAgent as typeof codingAgent & {
|
|
49
|
+
publishLocalFile(absolutePath: string, content: string | Uint8Array, signal?: AbortSignal): Promise<void>;
|
|
50
|
+
};
|
|
51
|
+
|
|
44
52
|
/** Fork-only ExtensionContext members; the published Pi types do not declare them. */
|
|
45
53
|
type NativeContext = {
|
|
46
54
|
model?: { contextWindow: number };
|
|
@@ -64,6 +72,7 @@ type ImageLike = { type: "image"; data: string; mimeType: string };
|
|
|
64
72
|
type MessageLike = {
|
|
65
73
|
role?: string;
|
|
66
74
|
stopReason?: string;
|
|
75
|
+
errorMessage?: string;
|
|
67
76
|
content?: unknown;
|
|
68
77
|
toolName?: string;
|
|
69
78
|
toolCallId?: string;
|
|
@@ -110,9 +119,10 @@ function nativeContext<T>(ctx: T): T & NativeContext {
|
|
|
110
119
|
if (
|
|
111
120
|
typeof candidate.newContext !== "function" ||
|
|
112
121
|
typeof candidate.getCompactionSettings !== "function" ||
|
|
113
|
-
typeof candidate.getSystemPrompt !== "function"
|
|
122
|
+
typeof candidate.getSystemPrompt !== "function" ||
|
|
123
|
+
typeof publishLocalFile !== "function"
|
|
114
124
|
) {
|
|
115
|
-
throw new Error("Posthorse requires the fitchmultz/pi fork with native context windows (see README).");
|
|
125
|
+
throw new Error("Posthorse requires the fitchmultz/pi fork with native context windows and publishLocalFile (see README).");
|
|
116
126
|
}
|
|
117
127
|
return candidate as T & NativeContext;
|
|
118
128
|
}
|
|
@@ -164,21 +174,49 @@ function textResult(text: string, images: ImageLike[] = [], display?: PosthorseD
|
|
|
164
174
|
return { content: [{ type: "text" as const, text }, ...images], details: display };
|
|
165
175
|
}
|
|
166
176
|
|
|
167
|
-
/**
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
177
|
+
/** Import old checkout-local notes without overwriting shared notes, including concurrent writes. */
|
|
178
|
+
function importLegacyNotes(source: string, target: string): void {
|
|
179
|
+
if (!existsSync(source)) return;
|
|
180
|
+
if (existsSync(target) && (!statSync(target).isDirectory() || realpathSync(source) === realpathSync(target))) return;
|
|
181
|
+
mkdirSync(target, { recursive: true });
|
|
182
|
+
for (const name of readdirSync(source)) {
|
|
183
|
+
const from = join(source, name);
|
|
184
|
+
const to = join(target, name);
|
|
185
|
+
if (statSync(from).isDirectory()) {
|
|
186
|
+
importLegacyNotes(from, to);
|
|
187
|
+
} else {
|
|
188
|
+
try {
|
|
189
|
+
copyFileSync(from, to, constants.COPYFILE_EXCL);
|
|
190
|
+
} catch (error) {
|
|
191
|
+
if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error;
|
|
192
|
+
}
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
/** Conventional repos use the main checkout; separate Git directories are their own shared root. */
|
|
172
198
|
function notesRoot(cwd: string): string {
|
|
173
199
|
for (let dir = cwd; ; dir = dirname(dir)) {
|
|
174
200
|
const marker = join(dir, ".git");
|
|
175
201
|
if (existsSync(marker)) {
|
|
202
|
+
let root: string;
|
|
176
203
|
try {
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
204
|
+
let target = marker;
|
|
205
|
+
if (!statSync(marker).isDirectory()) {
|
|
206
|
+
const gitdir = readFileSync(marker, "utf8").match(/^gitdir:\s*(.+?)\s*$/m)?.[1];
|
|
207
|
+
if (!gitdir) return dir;
|
|
208
|
+
target = resolve(dir, gitdir);
|
|
209
|
+
}
|
|
210
|
+
target = realpathSync(target);
|
|
211
|
+
const commonFile = join(target, "commondir");
|
|
212
|
+
const common = realpathSync(existsSync(commonFile) ? resolve(target, readFileSync(commonFile, "utf8").trim()) : target);
|
|
213
|
+
root = basename(common) === ".git" ? dirname(common) : common;
|
|
214
|
+
} catch {
|
|
215
|
+
// Orphaned/copied worktrees can outlive their Git metadata; local notes still work.
|
|
216
|
+
return dir;
|
|
217
|
+
}
|
|
218
|
+
if (root !== dir) importLegacyNotes(join(dir, ".pi", "notes"), join(root, ".pi", "notes"));
|
|
219
|
+
return root;
|
|
182
220
|
}
|
|
183
221
|
if (dirname(dir) === dir) return cwd;
|
|
184
222
|
}
|
|
@@ -195,6 +233,12 @@ function excerptAround(text: string, index: number, before: number, length: numb
|
|
|
195
233
|
return `${start ? "…" : ""}${text.slice(start, end)}${end < text.length ? "…" : ""}`;
|
|
196
234
|
}
|
|
197
235
|
|
|
236
|
+
function assistantFailure(message: MessageLike): string {
|
|
237
|
+
if (message.role !== "assistant") return "";
|
|
238
|
+
if (message.stopReason !== "error" && message.stopReason !== "aborted" && !message.errorMessage) return "";
|
|
239
|
+
return `[${message.stopReason ?? "error"}]${message.errorMessage ? ` ${message.errorMessage}` : ""}`;
|
|
240
|
+
}
|
|
241
|
+
|
|
198
242
|
function flattenEntry(entry: EntryLike): string | undefined {
|
|
199
243
|
if (entry.type === "message") {
|
|
200
244
|
const message = entry.message ?? {};
|
|
@@ -203,7 +247,7 @@ function flattenEntry(entry: EntryLike): string | undefined {
|
|
|
203
247
|
if (message.excludeFromContext === true) return "[bashExecution] (excluded from model context by Pi)";
|
|
204
248
|
return `[bashExecution] $ ${message.command ?? ""}\n${message.output ?? ""}`;
|
|
205
249
|
}
|
|
206
|
-
return `[${message.role ?? "message"}] ${[textOf(message), imageSummary(imagesOf(message.content))].filter(Boolean).join("\n")}`;
|
|
250
|
+
return `[${message.role ?? "message"}] ${[textOf(message), assistantFailure(message), imageSummary(imagesOf(message.content))].filter(Boolean).join("\n")}`;
|
|
207
251
|
}
|
|
208
252
|
if (entry.type === "compaction" || entry.type === "branch_summary") {
|
|
209
253
|
return `[${entry.type}] ${entry.summary ?? ""}`;
|
|
@@ -254,8 +298,8 @@ function historyHit(item: WindowedEntry, query: string, source = ""): HistoryHit
|
|
|
254
298
|
if (partText) originals[originals.length - 1] += `${originals.at(-1) ? "\n" : ""}${partText}`;
|
|
255
299
|
}
|
|
256
300
|
}
|
|
257
|
-
const
|
|
258
|
-
if (
|
|
301
|
+
const metadata = [assistantFailure(entry.message), imageSummary(item.images)].filter(Boolean).join("\n");
|
|
302
|
+
if (metadata) originals[originals.length - 1] += `${originals.at(-1) ? "\n" : ""}${metadata}`;
|
|
259
303
|
if (originals[0]) originals[0] = `[assistant] ${originals[0]}`;
|
|
260
304
|
const original = originals.find((part) => part.toLowerCase().includes(query));
|
|
261
305
|
priority = original ? 0 : 1;
|
|
@@ -291,25 +335,30 @@ function* windowEntries(entries: Iterable<EntryLike>): Generator<WindowedEntry>
|
|
|
291
335
|
}
|
|
292
336
|
}
|
|
293
337
|
|
|
338
|
+
/** JSONL splits on LF, not the Unicode separators that Node's readline also recognizes. */
|
|
339
|
+
async function* jsonlLines(file: string, signal?: AbortSignal): AsyncGenerator<string> {
|
|
340
|
+
const stream = createReadStream(file, { encoding: "utf8", signal });
|
|
341
|
+
let pending = "";
|
|
342
|
+
for await (const chunk of stream) {
|
|
343
|
+
const lines = (pending + chunk).split("\n");
|
|
344
|
+
pending = lines.pop()!;
|
|
345
|
+
yield* lines;
|
|
346
|
+
}
|
|
347
|
+
if (pending) yield pending;
|
|
348
|
+
}
|
|
349
|
+
|
|
294
350
|
async function* sessionWindowEntries(file: string, signal?: AbortSignal): AsyncGenerator<WindowedEntry> {
|
|
295
351
|
if (!existsSync(file)) return;
|
|
296
|
-
const stream = createReadStream(file, { encoding: "utf8", signal });
|
|
297
|
-
const lines = createInterface({ input: stream, crlfDelay: Number.POSITIVE_INFINITY });
|
|
298
352
|
const windows = new Map<string, string>();
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
continue;
|
|
306
|
-
}
|
|
307
|
-
const item = toWindowedEntry(entry, windows);
|
|
308
|
-
if (item) yield item;
|
|
353
|
+
for await (const line of jsonlLines(file, signal)) {
|
|
354
|
+
let entry: EntryLike;
|
|
355
|
+
try {
|
|
356
|
+
entry = JSON.parse(line) as EntryLike;
|
|
357
|
+
} catch {
|
|
358
|
+
continue;
|
|
309
359
|
}
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
stream.destroy();
|
|
360
|
+
const item = toWindowedEntry(entry, windows);
|
|
361
|
+
if (item) yield item;
|
|
313
362
|
}
|
|
314
363
|
}
|
|
315
364
|
|
|
@@ -399,12 +448,8 @@ function formatPriorCheckpoint(entry: EntryLike | undefined, limit: number): str
|
|
|
399
448
|
return boundedBlock(header, handoff, limit);
|
|
400
449
|
}
|
|
401
450
|
|
|
402
|
-
/**
|
|
403
|
-
|
|
404
|
-
* its completed results (and non-message entries). Pi can roll over right after tools finish, so the
|
|
405
|
-
* result that triggered the rollover would otherwise vanish before any model saw it.
|
|
406
|
-
*/
|
|
407
|
-
function unconsumedToolBatch(
|
|
451
|
+
/** Trailing results without a later complete assistant response; interrupted requests may have received them. */
|
|
452
|
+
function trailingToolBatch(
|
|
408
453
|
entries: readonly EntryLike[],
|
|
409
454
|
): { callId: string; blocks: Array<{ header: string; text: string }> } | undefined {
|
|
410
455
|
const results: EntryLike[] = [];
|
|
@@ -478,9 +523,9 @@ function buildAutoHandoff(entries: readonly EntryLike[], maxChars: number): stri
|
|
|
478
523
|
const joinedLength = (parts: Array<string | undefined>) =>
|
|
479
524
|
parts.filter((part): part is string => Boolean(part)).join("\n\n").length;
|
|
480
525
|
|
|
481
|
-
const toolBatch =
|
|
526
|
+
const toolBatch = trailingToolBatch(current);
|
|
482
527
|
const batchHeader = toolBatch
|
|
483
|
-
? `
|
|
528
|
+
? `Trailing tool batch without a later complete assistant response (failed or interrupted requests may already have received these results). Tool-call entry ${toolBatch.callId}:`
|
|
484
529
|
: undefined;
|
|
485
530
|
const minimumParts = [
|
|
486
531
|
preamble,
|
|
@@ -636,26 +681,13 @@ function unsupportedMessage(budget: Budget): string {
|
|
|
636
681
|
return `Posthorse: unsupported configuration. The model's context window (${n(budget.contextWindow)} tokens) minus Pi's compaction.reserveTokens (${n(budget.reserveTokens)}) leaves ${n(budget.usable)} usable tokens; Posthorse needs at least ${n(MIN_USABLE_TOKENS)}. Automatic rollover and checkpoint reminders are off for this model. Lower compaction.reserveTokens in Pi settings or use a larger-context model. new_context remains available with a model-aware handoff limit.`;
|
|
637
682
|
}
|
|
638
683
|
|
|
639
|
-
/**
|
|
640
|
-
function
|
|
684
|
+
/** Capacity before Pi's automatic line (or configured context limit), including page metadata and refusals. */
|
|
685
|
+
function pageCapacity(ctx: NativeContext, toolTokens: number): number {
|
|
641
686
|
const usage = ctx.getContextUsage();
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
const budget = budgetFor(ctx, usage.contextWindow);
|
|
647
|
-
const line = budget?.enabled && budget.supported ? budget.rolloverAt : usage.contextWindow;
|
|
648
|
-
chars = Math.min(
|
|
649
|
-
MAX_HANDOFF_CHARS,
|
|
650
|
-
Math.max(0, line - usage.tokens - PAGE_MARGIN_TOKENS) * 4 - imageCount * ESTIMATED_IMAGE_CHARS,
|
|
651
|
-
);
|
|
652
|
-
}
|
|
653
|
-
if (chars < MIN_PAGE_CHARS) {
|
|
654
|
-
throw new Error(
|
|
655
|
-
`Too little context remains to read a page safely. Call new_context first, then retry with offset ${offset}.`,
|
|
656
|
-
);
|
|
657
|
-
}
|
|
658
|
-
return chars;
|
|
687
|
+
if (!usage || usage.tokens == null) return freshPayloadChars(ctx, toolTokens) / 4 + PAGE_MARGIN_TOKENS;
|
|
688
|
+
const budget = budgetFor(ctx, usage.contextWindow);
|
|
689
|
+
const line = budget?.enabled && budget.supported ? budget.rolloverAt : usage.contextWindow;
|
|
690
|
+
return line - usage.tokens;
|
|
659
691
|
}
|
|
660
692
|
|
|
661
693
|
function buildGuidance(ctx: NativeContext): string {
|
|
@@ -698,13 +730,77 @@ export default function (pi: ExtensionAPI) {
|
|
|
698
730
|
);
|
|
699
731
|
};
|
|
700
732
|
|
|
733
|
+
let pendingPageTokens = 0;
|
|
734
|
+
let previousPageUsage: number | null | undefined;
|
|
735
|
+
const pageSize = (ctx: NativeContext, offset: number, imageCount: number, cursor?: string) => {
|
|
736
|
+
const usage = ctx.getContextUsage()?.tokens;
|
|
737
|
+
// Parallel siblings are not in native usage yet; sequential results must not be counted twice.
|
|
738
|
+
if (usage != null && previousPageUsage != null) {
|
|
739
|
+
pendingPageTokens = Math.max(0, pendingPageTokens - Math.max(0, usage - previousPageUsage));
|
|
740
|
+
}
|
|
741
|
+
previousPageUsage = usage;
|
|
742
|
+
const remaining = pageCapacity(ctx, activeToolTokens()) - pendingPageTokens;
|
|
743
|
+
const chars = Math.min(
|
|
744
|
+
MAX_HANDOFF_CHARS,
|
|
745
|
+
Math.max(0, remaining - PAGE_MARGIN_TOKENS) * 4 - imageCount * ESTIMATED_IMAGE_CHARS,
|
|
746
|
+
);
|
|
747
|
+
if (chars < MIN_PAGE_CHARS) {
|
|
748
|
+
const message = `Too little context remains to read a page safely. Call new_context first, then retry ${cursor ? "with the same cursor" : `with offset ${offset}`}.`;
|
|
749
|
+
// Refusals consume context too. Once even the guidance no longer fits, omit repeated text;
|
|
750
|
+
// the failed call still carries its original offset and earlier refusals explain the retry.
|
|
751
|
+
const text = Math.ceil(message.length / 4) <= remaining ? message : "";
|
|
752
|
+
pendingPageTokens += Math.ceil(text.length / 4);
|
|
753
|
+
throw new Error(text);
|
|
754
|
+
}
|
|
755
|
+
return chars;
|
|
756
|
+
};
|
|
757
|
+
const pageResult = (text: string, images: ImageLike[], display: PosthorseDisplay) => {
|
|
758
|
+
pendingPageTokens += Math.ceil(text.length / 4) + images.length * (ESTIMATED_IMAGE_CHARS / 4);
|
|
759
|
+
return textResult(text, images, display);
|
|
760
|
+
};
|
|
761
|
+
const pageError = (ctx: NativeContext, message: string, cursor?: string): never => {
|
|
762
|
+
pageSize(ctx, 0, 0, cursor);
|
|
763
|
+
pendingPageTokens += Math.ceil(message.length / 4);
|
|
764
|
+
throw new Error(message);
|
|
765
|
+
};
|
|
766
|
+
const notesPage = (ctx: NativeContext, rows: string[], offset: number, kind: "notes-list" | "notes-search", empty: string) => {
|
|
767
|
+
const text = rows.length ? rows.join("\n") : empty;
|
|
768
|
+
const chars = pageSize(ctx, offset, 0);
|
|
769
|
+
if (offset && offset >= text.length) pageError(ctx, "Offset is past the end; restart with offset 0.");
|
|
770
|
+
// Reserve the complete continuation, even when one path or excerpt needs several pages.
|
|
771
|
+
const footer = (end: number) => `\n[chars ${offset}-${end} of ${text.length}; continue with offset ${end}]`;
|
|
772
|
+
let end = Math.min(text.length, offset + chars);
|
|
773
|
+
let start = 0;
|
|
774
|
+
const starts: number[] = [];
|
|
775
|
+
for (const row of rows) {
|
|
776
|
+
if (start >= end) break;
|
|
777
|
+
if (start + row.length > offset) starts.push(start);
|
|
778
|
+
start += row.length + 1;
|
|
779
|
+
if (kind === "notes-search" && starts.length === 20) { end = Math.min(end, start - 1); break; }
|
|
780
|
+
}
|
|
781
|
+
// A result-count cap can require continuation even when the character cap did not.
|
|
782
|
+
if (end < text.length) end = Math.min(end, offset + chars - footer(text.length).length);
|
|
783
|
+
return pageResult(text.slice(offset, end) + (end < text.length ? footer(end) : ""), [], { kind, count: starts.filter((start) => start < end).length, page: { offset, end, total: text.length } });
|
|
784
|
+
};
|
|
785
|
+
pi.on("turn_start", () => {
|
|
786
|
+
pendingPageTokens = 0;
|
|
787
|
+
previousPageUsage = undefined;
|
|
788
|
+
});
|
|
789
|
+
|
|
701
790
|
pi.on("session_start", (_event, ctx) => {
|
|
791
|
+
pendingPageTokens = 0;
|
|
792
|
+
previousPageUsage = undefined;
|
|
702
793
|
nativeContext(ctx);
|
|
703
794
|
});
|
|
704
795
|
|
|
705
|
-
pi.on("before_agent_start", (event, ctx) =>
|
|
706
|
-
|
|
707
|
-
|
|
796
|
+
pi.on("before_agent_start", (event, ctx) => {
|
|
797
|
+
const guidance = buildGuidance(nativeContext(ctx));
|
|
798
|
+
// An earlier full-prompt override makes section edits invisible to Pi.
|
|
799
|
+
if (event.systemPromptOptions.forceSystemPrompt !== undefined) {
|
|
800
|
+
return { systemPrompt: `${event.systemPrompt}\n\n${guidance}` };
|
|
801
|
+
}
|
|
802
|
+
event.systemPromptOptions.sections.posthorse = guidance;
|
|
803
|
+
});
|
|
708
804
|
|
|
709
805
|
pi.on("turn_end", (event, ctx) => {
|
|
710
806
|
const native = nativeContext(ctx);
|
|
@@ -746,7 +842,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
746
842
|
);
|
|
747
843
|
});
|
|
748
844
|
|
|
749
|
-
//
|
|
845
|
+
// Filter active input only; raw reminders remain in history for deduplication and recovery.
|
|
750
846
|
pi.on("context", (event, ctx) => {
|
|
751
847
|
const marker = event.messages.find(
|
|
752
848
|
(message) => message.role === "custom" && message.customType === "context-window",
|
|
@@ -765,7 +861,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
765
861
|
const stale = (message: (typeof event.messages)[number]) =>
|
|
766
862
|
message.role === "custom" &&
|
|
767
863
|
isReminderType(message.customType) &&
|
|
768
|
-
reminderIsStale(message.customType, message.details, fingerprint, branch);
|
|
864
|
+
(!native.getCompactionSettings().enabled || reminderIsStale(message.customType, message.details, fingerprint, branch));
|
|
769
865
|
if (event.messages.some(stale)) return { messages: event.messages.filter((message) => !stale(message)) };
|
|
770
866
|
});
|
|
771
867
|
|
|
@@ -825,7 +921,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
825
921
|
label: "Context Remaining",
|
|
826
922
|
...toolCards("get_context_remaining"),
|
|
827
923
|
description:
|
|
828
|
-
"Best available native estimate of the context budget: tokens until Pi's automatic rollover line and until the
|
|
924
|
+
"Best available native estimate of the context budget: tokens until Pi's automatic rollover line and until the configured context limit.",
|
|
829
925
|
promptSnippet: "check the remaining context budget only when needed",
|
|
830
926
|
parameters: Type.Object({}),
|
|
831
927
|
async execute(_id, _params, _signal, _onUpdate, ctx) {
|
|
@@ -839,11 +935,11 @@ export default function (pi: ExtensionAPI) {
|
|
|
839
935
|
rollover: !budget?.enabled ? "disabled" : budget.supported ? "enabled" : "unsupported",
|
|
840
936
|
rolloverAt: budget?.rolloverAt,
|
|
841
937
|
};
|
|
842
|
-
const
|
|
843
|
-
if (!budget?.enabled) return textResult(`Automatic rollover is disabled (Pi compaction.enabled=false). ${
|
|
844
|
-
if (!budget.supported) return textResult(`${unsupportedMessage(budget)} ${
|
|
938
|
+
const configured = `≈${n(Math.max(0, usage.contextWindow - usage.tokens))} tokens until the configured context limit (${n(usage.tokens)}/${n(usage.contextWindow)} used, ${Math.round(usage.percent ?? 0)}%). Best available native estimate.`;
|
|
939
|
+
if (!budget?.enabled) return textResult(`Automatic rollover is disabled (Pi compaction.enabled=false). ${configured}`, [], display);
|
|
940
|
+
if (!budget.supported) return textResult(`${unsupportedMessage(budget)} ${configured}`, [], display);
|
|
845
941
|
return textResult(
|
|
846
|
-
`≈${n(Math.max(0, budget.rolloverAt - usage.tokens))} tokens until automatic rollover (line at ${n(budget.rolloverAt)}); ${
|
|
942
|
+
`≈${n(Math.max(0, budget.rolloverAt - usage.tokens))} tokens until automatic rollover (line at ${n(budget.rolloverAt)}); ${configured}`,
|
|
847
943
|
[], display,
|
|
848
944
|
);
|
|
849
945
|
},
|
|
@@ -854,7 +950,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
854
950
|
label: "Notes",
|
|
855
951
|
...toolCards("notes"),
|
|
856
952
|
description:
|
|
857
|
-
"Persistent notes in .pi/notes/ that survive context resets. Ops: list
|
|
953
|
+
"Persistent notes in .pi/notes/ that survive context resets. Ops: list/read/search (paged; repeat with offset to continue), write (create/replace; empty content clears), append. Search matches case-insensitive substrings over note lines. Git worktrees share notes: at the main checkout for conventional .git layouts, or in the common Git directory when metadata is stored separately. Old checkout-local notes are imported without overwriting shared notes.",
|
|
858
954
|
promptSnippet: "save and recall durable state that survives context resets",
|
|
859
955
|
promptGuidelines: [
|
|
860
956
|
"Use notes for durable state too large for a new_context handoff",
|
|
@@ -868,9 +964,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
868
964
|
path: Type.Optional(Type.String({ description: "Note path relative to .pi/notes/ (read/write/append)" })),
|
|
869
965
|
content: Type.Optional(Type.String({ description: "Full file content (write) or text to add (append)" })),
|
|
870
966
|
query: Type.Optional(Type.String({ description: "Substring to find in notes (search)" })),
|
|
871
|
-
offset: Type.Optional(Type.Integer({ description: "Character offset for read (default 0)", minimum: 0 })),
|
|
967
|
+
offset: Type.Optional(Type.Integer({ description: "Character offset for list/read/search (default 0)", minimum: 0 })),
|
|
872
968
|
}),
|
|
873
|
-
async execute(_id, params,
|
|
969
|
+
async execute(_id, params, signal, _onUpdate, ctx) {
|
|
874
970
|
const dir = join(notesRoot(ctx.cwd), ".pi", "notes");
|
|
875
971
|
const safeJoin = (path: string) => {
|
|
876
972
|
const relative = normalize(path.replace(/^[/\\]+/, ""));
|
|
@@ -889,10 +985,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
889
985
|
|
|
890
986
|
switch (params.op) {
|
|
891
987
|
case "list": {
|
|
892
|
-
if (!existsSync(dir)) return textResult("(no notes yet)", [], { kind: "notes-list", count: 0 });
|
|
893
988
|
const files: string[] = [];
|
|
894
|
-
walk(dir, files);
|
|
895
|
-
return
|
|
989
|
+
if (existsSync(dir)) walk(dir, files);
|
|
990
|
+
return notesPage(nativeContext(ctx), files.map((file) => file.slice(dir.length + 1)), params.offset ?? 0, "notes-list", "(no notes yet)");
|
|
896
991
|
}
|
|
897
992
|
case "read": {
|
|
898
993
|
const relative = requireValue(params.path, "path", params.op);
|
|
@@ -903,18 +998,18 @@ export default function (pi: ExtensionAPI) {
|
|
|
903
998
|
if (offset && offset >= text.length) {
|
|
904
999
|
throw new Error(`Offset ${offset} is past the end of ${relative} (${text.length} chars).`);
|
|
905
1000
|
}
|
|
906
|
-
const end = Math.min(text.length, offset +
|
|
1001
|
+
const end = Math.min(text.length, offset + pageSize(nativeContext(ctx), offset, 0));
|
|
907
1002
|
const more =
|
|
908
1003
|
end < text.length ? `\n[chars ${offset}-${end} of ${text.length}; continue with offset ${end}]` : "";
|
|
909
|
-
return
|
|
1004
|
+
return pageResult(`${text.slice(offset, end)}${more}`, [], { kind: "note-read", offset, end, total: text.length });
|
|
910
1005
|
}
|
|
911
1006
|
case "write": {
|
|
912
1007
|
const relative = requireValue(params.path, "path", params.op);
|
|
913
1008
|
if (params.content === undefined) throw new Error(`"content" is required for op "write" (use "" to clear a note).`);
|
|
914
1009
|
const path = safeJoin(relative);
|
|
915
1010
|
mkdirSync(dirname(path), { recursive: true });
|
|
916
|
-
|
|
917
|
-
return textResult(`Wrote
|
|
1011
|
+
await publishLocalFile(path, params.content, signal);
|
|
1012
|
+
return textResult(`Wrote ${path}`, [], { kind: "note-write" });
|
|
918
1013
|
}
|
|
919
1014
|
case "append": {
|
|
920
1015
|
const relative = requireValue(params.path, "path", params.op);
|
|
@@ -927,17 +1022,15 @@ export default function (pi: ExtensionAPI) {
|
|
|
927
1022
|
const existing = existsSync(path) ? readFileSync(path, "utf8") : "";
|
|
928
1023
|
const separator = existing && !existing.endsWith("\n") ? "\n" : "";
|
|
929
1024
|
appendFileSync(path, `${separator}${content.replace(/\n?$/, "\n")}`);
|
|
930
|
-
return textResult(`Appended to
|
|
1025
|
+
return textResult(`Appended to ${path}`, [], { kind: "note-append" });
|
|
931
1026
|
}
|
|
932
1027
|
case "search": {
|
|
933
1028
|
const query = requireValue(params.query, "query", params.op).toLowerCase();
|
|
934
|
-
if (!existsSync(dir)) return textResult("(no notes yet)", [], { kind: "notes-search", count: 0 });
|
|
935
1029
|
const files: string[] = [];
|
|
936
|
-
walk(dir, files);
|
|
1030
|
+
if (existsSync(dir)) walk(dir, files);
|
|
937
1031
|
const hits: string[] = [];
|
|
938
1032
|
for (const file of files) {
|
|
939
1033
|
for (const [index, line] of readFileSync(file, "utf8").split("\n").entries()) {
|
|
940
|
-
if (hits.length >= 20) break;
|
|
941
1034
|
const trimmed = line.trim();
|
|
942
1035
|
const match = trimmed.toLowerCase().indexOf(query);
|
|
943
1036
|
if (match !== -1) {
|
|
@@ -945,7 +1038,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
945
1038
|
}
|
|
946
1039
|
}
|
|
947
1040
|
}
|
|
948
|
-
return
|
|
1041
|
+
return notesPage(nativeContext(ctx), hits, params.offset ?? 0, "notes-search", `No notes match "${excerpt(params.query!, 200)}".`);
|
|
949
1042
|
}
|
|
950
1043
|
}
|
|
951
1044
|
},
|
|
@@ -956,7 +1049,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
956
1049
|
label: "History",
|
|
957
1050
|
...toolCards("history"),
|
|
958
1051
|
description:
|
|
959
|
-
"Search or read normalized session entries, including earlier native context windows. Search prioritizes original content before recovery notes, handoffs, and history lookups; all remain searchable. Current branch by default; all=true searches every project session file. Within each group: newest-modified sessions first, newest entries per session. Reads return stored images and page long text with the next offset.",
|
|
1052
|
+
"Search or read normalized session entries, including earlier native context windows. Search prioritizes original content before recovery notes, handoffs, and history lookups; all remain searchable. Current branch by default; all=true searches every project session file. Within each group: newest-modified sessions first, newest entries per session. Continue searches with the returned cursor and the same query/scope; new searches see newer entries. Reads return stored images and page long text with the next offset.",
|
|
960
1053
|
promptSnippet: "recover earlier conversation that left the active context window",
|
|
961
1054
|
promptGuidelines: ["Use history search first, then history read with the returned entry id"],
|
|
962
1055
|
parameters: Type.Object({
|
|
@@ -964,7 +1057,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
964
1057
|
query: Type.Optional(Type.String({ description: "Case-insensitive text to find (search)" })),
|
|
965
1058
|
id: Type.Optional(Type.String({ description: "Entry id returned by search (read)" })),
|
|
966
1059
|
all: Type.Optional(Type.Boolean({ description: "Search all project sessions instead of the current branch" })),
|
|
967
|
-
limit: Type.Optional(Type.Integer({ description: "Maximum results (default 10, max 50)", minimum: 1, maximum: 50 })),
|
|
1060
|
+
limit: Type.Optional(Type.Integer({ description: "Maximum search results per page (default 10, max 50)", minimum: 1, maximum: 50 })),
|
|
1061
|
+
cursor: Type.Optional(Type.String({ description: "Search continuation returned by the previous page; keep query and all unchanged", maxLength: 512 })),
|
|
968
1062
|
offset: Type.Optional(Type.Integer({ description: "Character offset for read (default 0)", minimum: 0 })),
|
|
969
1063
|
}),
|
|
970
1064
|
async execute(_id, params, signal, _onUpdate, ctx) {
|
|
@@ -973,42 +1067,91 @@ export default function (pi: ExtensionAPI) {
|
|
|
973
1067
|
if (params.op === "search") {
|
|
974
1068
|
const query = requireValue(params.query, "query", params.op).toLowerCase();
|
|
975
1069
|
const limit = params.limit ?? 10;
|
|
1070
|
+
const searchKey = createHash("sha256").update(JSON.stringify([query, params.all === true])).digest("base64url");
|
|
1071
|
+
let cursor: [string, 0 | 1, number, string] | undefined;
|
|
1072
|
+
if (params.cursor) {
|
|
1073
|
+
try {
|
|
1074
|
+
const value = JSON.parse(Buffer.from(params.cursor, "base64url").toString());
|
|
1075
|
+
if (!Array.isArray(value) || value.length !== 4 || typeof value[0] !== "string" || !value[0] || ![0, 1].includes(value[1]) || !Number.isSafeInteger(value[2]) || value[2] < 0 || value[3] !== searchKey) throw new Error();
|
|
1076
|
+
cursor = value as typeof cursor;
|
|
1077
|
+
} catch { pageError(nativeContext(ctx), "Invalid history cursor or changed query/scope; restart the search without it.", params.cursor); }
|
|
1078
|
+
}
|
|
976
1079
|
const hits: HistoryHit[][] = [[], []];
|
|
977
|
-
//
|
|
1080
|
+
// Mark skipped ids too: fork copies must not reappear on subsequent pages.
|
|
978
1081
|
const seen = new Set<string>();
|
|
1082
|
+
let found = !cursor;
|
|
979
1083
|
const addHit = (hit: HistoryHit | undefined) => {
|
|
980
|
-
if (!hit || seen.has(hit.id)
|
|
1084
|
+
if (!hit || seen.has(hit.id)) return;
|
|
981
1085
|
seen.add(hit.id);
|
|
982
|
-
|
|
1086
|
+
if (cursor && hit.priority < cursor[1]) return;
|
|
1087
|
+
if (cursor && hit.priority === cursor[1] && !found) {
|
|
1088
|
+
if (hit.id !== cursor[0]) return;
|
|
1089
|
+
found = true;
|
|
1090
|
+
if (cursor[2] > hit.text.length) pageError(nativeContext(ctx), "History cursor is past the entry; restart the search without it.", params.cursor);
|
|
1091
|
+
if (cursor[2] === hit.text.length) return;
|
|
1092
|
+
}
|
|
1093
|
+
if (hits[hit.priority].length <= limit) hits[hit.priority].push(hit);
|
|
983
1094
|
};
|
|
984
1095
|
|
|
985
1096
|
if (params.all) {
|
|
986
1097
|
for (const file of sessionFiles(manager.getSessionDir())) {
|
|
987
1098
|
const recent: HistoryHit[][] = [[], []];
|
|
1099
|
+
const matchedIds = new Set<string>();
|
|
1100
|
+
let anchorHere = false;
|
|
988
1101
|
const source = relative(manager.getSessionDir(), file);
|
|
989
1102
|
for await (const item of sessionWindowEntries(file, signal)) {
|
|
990
1103
|
if (seen.has(item.entry.id!)) continue;
|
|
991
1104
|
const hit = historyHit(item, query, source);
|
|
992
1105
|
if (!hit) continue;
|
|
1106
|
+
matchedIds.add(hit.id);
|
|
1107
|
+
const seeking = cursor && !found && hit.priority === cursor[1];
|
|
1108
|
+
if (seeking && anchorHere) continue;
|
|
1109
|
+
if (seeking && hit.id === cursor?.[0]) anchorHere = true;
|
|
993
1110
|
const group = recent[hit.priority];
|
|
994
1111
|
group.push(hit);
|
|
995
|
-
|
|
1112
|
+
// Keep one page plus its anchor and lookahead, not every matching excerpt.
|
|
1113
|
+
if (group.length > limit + 2) group.shift();
|
|
996
1114
|
}
|
|
1115
|
+
if (cursor && !found && !anchorHere) recent[cursor[1]] = [];
|
|
997
1116
|
for (const group of recent) for (const hit of group.reverse()) addHit(hit);
|
|
998
|
-
|
|
1117
|
+
for (const id of matchedIds) seen.add(id);
|
|
1118
|
+
if (hits[0].length > limit) break;
|
|
999
1119
|
}
|
|
1000
1120
|
} else {
|
|
1001
1121
|
const current = [...windowEntries(manager.getBranch() as EntryLike[])];
|
|
1002
1122
|
for (const item of current.reverse()) {
|
|
1003
1123
|
addHit(historyHit(item, query));
|
|
1004
|
-
if (hits[0].length
|
|
1124
|
+
if (hits[0].length > limit) break;
|
|
1005
1125
|
}
|
|
1006
1126
|
}
|
|
1007
|
-
const
|
|
1008
|
-
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1127
|
+
const chars = pageSize(nativeContext(ctx), 0, 0, params.cursor);
|
|
1128
|
+
if (!found) pageError(nativeContext(ctx), "History cursor entry no longer matches; restart the search without it.", params.cursor);
|
|
1129
|
+
const results = hits.flat();
|
|
1130
|
+
const cursorFor = (hit: HistoryHit, offset: number) => Buffer.from(JSON.stringify([hit.id, hit.priority, offset, searchKey])).toString("base64url");
|
|
1131
|
+
const footer = (next: string) => `\n[More results; continue with cursor "${next}" and the same query/scope.]`;
|
|
1132
|
+
const reserve = Math.max(0, ...results.map((hit) => footer(cursorFor(hit, hit.text.length)).length));
|
|
1133
|
+
if (reserve >= chars) pageError(nativeContext(ctx), "History entry id is too large for pagination.", params.cursor);
|
|
1134
|
+
const parts: string[] = [];
|
|
1135
|
+
const spans: Array<{ headerLength: number; length: number }> = [];
|
|
1136
|
+
let available = chars - reserve;
|
|
1137
|
+
let next: string | undefined;
|
|
1138
|
+
for (const hit of results) {
|
|
1139
|
+
const offset = cursor?.[0] === hit.id ? cursor[2] : 0;
|
|
1140
|
+
if (parts.length && hit.text.length - offset > available) break;
|
|
1141
|
+
// A split header or excerpt must still identify the native entry on every page.
|
|
1142
|
+
const prefix = offset || hit.text.length > available ? `[entry ${hit.id}; from char ${offset}] ` : "";
|
|
1143
|
+
if (prefix.length >= available) break;
|
|
1144
|
+
const end = Math.min(hit.text.length, offset + available - prefix.length);
|
|
1145
|
+
const part = prefix + hit.text.slice(offset, end);
|
|
1146
|
+
parts.push(part);
|
|
1147
|
+
spans.push({ headerLength: prefix.length + Math.max(0, Math.min(hit.headerLength - offset, end - offset)), length: part.length });
|
|
1148
|
+
available -= part.length + 1;
|
|
1149
|
+
next = end < hit.text.length || parts.length < results.length ? cursorFor(hit, end) : undefined;
|
|
1150
|
+
if (available <= 0 || parts.length >= limit) break;
|
|
1151
|
+
}
|
|
1152
|
+
const body = parts.length ? parts.join("\n") : `No history matches "${excerpt(params.query!, 200)}".`;
|
|
1153
|
+
const more = next ? footer(next) : "";
|
|
1154
|
+
return pageResult(body + more, [], { kind: "history-search", entries: spans, footerLength: more.length });
|
|
1012
1155
|
}
|
|
1013
1156
|
|
|
1014
1157
|
const id = requireValue(params.id, "id", params.op);
|
|
@@ -1019,12 +1162,12 @@ export default function (pi: ExtensionAPI) {
|
|
|
1019
1162
|
}
|
|
1020
1163
|
const end = Math.min(
|
|
1021
1164
|
item.text.length,
|
|
1022
|
-
offset +
|
|
1165
|
+
offset + pageSize(nativeContext(ctx), offset, offset === 0 ? item.images.length : 0),
|
|
1023
1166
|
);
|
|
1024
1167
|
const more = end < item.text.length ? `\nMore remains; call history read with id "${id}" and offset ${end}.` : "";
|
|
1025
1168
|
const header = `${source ? `${source} ` : ""}${item.entry.timestamp ?? ""} [window ${item.windowId}] [${id}] [chars ${offset}-${end} of ${item.text.length}] `;
|
|
1026
1169
|
// Stored images ride along with the first page only.
|
|
1027
|
-
return
|
|
1170
|
+
return pageResult(
|
|
1028
1171
|
`${header}${item.text.slice(offset, end)}${more}`,
|
|
1029
1172
|
offset === 0 ? item.images : [],
|
|
1030
1173
|
{ kind: "history-read", headerLength: header.length, offset, end, total: item.text.length },
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-posthorse",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Posthorse: fresh context, same journey. Native no-summary context windows for the fitchmultz/pi fork, with sparse budget reminders, handoffs, notes, and history.",
|
|
6
6
|
"keywords": ["pi-package", "pi-extension", "context", "posthorse", "rollover"],
|
|
@@ -19,6 +19,8 @@
|
|
|
19
19
|
},
|
|
20
20
|
"scripts": {
|
|
21
21
|
"check": "tsc --noEmit",
|
|
22
|
+
"check:compat": "npm run check && npm test && npm run test:native",
|
|
23
|
+
"test:native": "node --import ./test/pi-loader.ts --test test/native.test.mjs",
|
|
22
24
|
"test": "node --import ./test/pi-loader.ts --test test/*.test.ts",
|
|
23
25
|
"prepublishOnly": "npm test && npm run check"
|
|
24
26
|
},
|
|
@@ -39,8 +41,8 @@
|
|
|
39
41
|
}
|
|
40
42
|
},
|
|
41
43
|
"devDependencies": {
|
|
42
|
-
"@earendil-works/pi-coding-agent": "0.
|
|
43
|
-
"@earendil-works/pi-tui": "0.
|
|
44
|
+
"@earendil-works/pi-coding-agent": "0.87.0",
|
|
45
|
+
"@earendil-works/pi-tui": "0.87.0",
|
|
44
46
|
"typebox": "1.3.7",
|
|
45
47
|
"typescript": "^5.5.0"
|
|
46
48
|
}
|
package/ui.ts
CHANGED
|
@@ -4,10 +4,10 @@ import { Box, MouseRegion, Text, stripTerminalSequences, truncateToWidth, type C
|
|
|
4
4
|
/** Display-only facts and offsets into content, never a second copy of a note or history page. */
|
|
5
5
|
export type PosthorseDisplay =
|
|
6
6
|
| { kind: "context"; usage?: { tokens: number | null; contextWindow: number; percent: number | null }; rollover?: "enabled" | "disabled" | "unsupported"; rolloverAt?: number }
|
|
7
|
-
| { kind: "notes-list" | "notes-search"; count: number }
|
|
7
|
+
| { kind: "notes-list" | "notes-search"; count: number; page?: { offset: number; end: number; total: number } }
|
|
8
8
|
| { kind: "note-read"; offset: number; end: number; total: number }
|
|
9
9
|
| { kind: "note-write" | "note-append" | "new-context" }
|
|
10
|
-
| { kind: "history-search"; entries: Array<{ headerLength: number; length: number }
|
|
10
|
+
| { kind: "history-search"; entries: Array<{ headerLength: number; length: number }>; footerLength?: number }
|
|
11
11
|
| { kind: "history-read"; headerLength: number; offset: number; end: number; total: number };
|
|
12
12
|
|
|
13
13
|
type ToolName = "notes" | "history" | "get_context_remaining" | "new_context";
|
|
@@ -43,14 +43,17 @@ function pageSummary(page: { offset: number; end: number; total: number }): stri
|
|
|
43
43
|
function historySections(raw: string, display: PosthorseDisplay | undefined): Array<{ body: string; header: string }> | undefined {
|
|
44
44
|
const spans = display?.kind === "history-search" ? display.entries : display?.kind === "history-read" ? [{ headerLength: display.headerLength, length: raw.length }] : undefined;
|
|
45
45
|
if (!Array.isArray(spans) || !spans.length) return undefined;
|
|
46
|
+
const footerLength = display?.kind === "history-search" ? display.footerLength ?? 0 : 0;
|
|
47
|
+
if (!Number.isInteger(footerLength) || footerLength < 0 || footerLength > raw.length) return undefined;
|
|
48
|
+
const bodyLength = raw.length - footerLength;
|
|
46
49
|
let start = 0;
|
|
47
50
|
const sections: Array<{ body: string; header: string }> = [];
|
|
48
51
|
for (const span of spans) {
|
|
49
|
-
if (!Number.isInteger(span.headerLength) || !Number.isInteger(span.length) || span.headerLength < 0 || span.headerLength > span.length || start + span.length >
|
|
52
|
+
if (!Number.isInteger(span.headerLength) || !Number.isInteger(span.length) || span.headerLength < 0 || span.headerLength > span.length || start + span.length > bodyLength) return undefined;
|
|
50
53
|
sections.push({ header: raw.slice(start, start + span.headerLength).trimEnd(), body: raw.slice(start + span.headerLength, start + span.length) });
|
|
51
54
|
start += span.length + 1;
|
|
52
55
|
}
|
|
53
|
-
return start ===
|
|
56
|
+
return start === bodyLength + 1 ? sections : undefined;
|
|
54
57
|
}
|
|
55
58
|
|
|
56
59
|
export function toolCards(name: ToolName): Pick<ToolDefinition, "renderCall" | "renderResult"> {
|
|
@@ -88,22 +91,23 @@ export function toolCards(name: ToolName): Pick<ToolDefinition, "renderCall" | "
|
|
|
88
91
|
case "context": {
|
|
89
92
|
const usage = display.usage;
|
|
90
93
|
if (!usage || usage.tokens == null) { summary = "Context usage unknown"; break; }
|
|
91
|
-
const
|
|
94
|
+
const configured = `≈${n(Math.max(0, usage.contextWindow - usage.tokens))} tokens to configured context limit`;
|
|
92
95
|
const rollover = display.rollover === "enabled" && display.rolloverAt !== undefined ? `≈${n(Math.max(0, display.rolloverAt - usage.tokens))} tokens to rollover` : `Automatic rollover ${display.rollover ?? "unknown"}`;
|
|
93
|
-
summary = `${rollover}\n${
|
|
96
|
+
summary = `${rollover}\n${configured}\n≈${n(usage.tokens)} / ${n(usage.contextWindow)} used${usage.percent == null ? "" : ` (≈${Math.round(usage.percent)}%)`}`;
|
|
94
97
|
if (display.rollover === "unsupported") color = "warning";
|
|
95
98
|
break;
|
|
96
99
|
}
|
|
97
|
-
case "notes-list": summary = display.count ? `${n(display.count)} note${display.count === 1 ? "" : "s"}` : "No notes yet"; break;
|
|
100
|
+
case "notes-list": summary = display.count ? `${n(display.count)} note${display.count === 1 ? "" : "s"} returned` : "No notes yet"; break;
|
|
98
101
|
case "notes-search": summary = display.count ? `${n(display.count)} matches returned` : "No matches"; break;
|
|
99
102
|
case "note-read": summary = pageSummary(display); break;
|
|
100
103
|
case "note-write": summary = args.content === "" ? "Cleared note" : "Saved note"; break;
|
|
101
104
|
case "note-append": summary = "Appended to note"; break;
|
|
102
|
-
case "history-search": if (Array.isArray(display.entries) && (!display.entries.length || sections)) summary = `${display.entries.length ? `${n(display.entries.length)} matches` : "No matches"} · ${args.all === true ? "all sessions" : "current branch"}`; break;
|
|
105
|
+
case "history-search": if (Array.isArray(display.entries) && (!display.entries.length || sections)) summary = `${display.entries.length ? `${n(display.entries.length)} matches` : "No matches"} · ${args.all === true ? "all sessions" : "current branch"}${display.footerLength ? "\nMore results; continue with cursor" : ""}`; break;
|
|
103
106
|
case "history-read": summary = pageSummary(display); break;
|
|
104
107
|
case "new-context": summary = "Requested for after the whole tool batch succeeds."; break;
|
|
105
108
|
}
|
|
106
109
|
}
|
|
110
|
+
if (!context.isError && !isPartial && (display?.kind === "notes-list" || display?.kind === "notes-search") && display.page && (display.page.offset || display.page.end < display.page.total)) summary += `\n${pageSummary(display.page)}`;
|
|
107
111
|
if (!body.trim()) body = context.isError ? "No error details returned." : isPartial ? "Running…" : name === "notes" && args.op === "read" ? "Empty note" : "No text returned.";
|
|
108
112
|
const submitted = name === "new_context" && typeof args.handoff === "string" ? `Handoff supplied:\n${args.handoff}` : name === "notes" && (args.op === "write" || args.op === "append") && typeof args.content === "string" ? `Submitted content:\n${args.content || "(empty)"}` : "";
|
|
109
113
|
const images = Array.isArray(result.content) ? result.content.filter((part) => part.type === "image") : [];
|
|
@@ -121,7 +125,7 @@ export function toolCards(name: ToolName): Pick<ToolDefinition, "renderCall" | "
|
|
|
121
125
|
invalidate() {},
|
|
122
126
|
render(width) {
|
|
123
127
|
if (expanded) {
|
|
124
|
-
return [...fullText.render(width), ...attachmentText.render(width), ...(submitted ? ["", ...submittedText.render(width)] : [])];
|
|
128
|
+
return [...fullText.render(width), ...attachmentText.render(width), ...(sections && display?.kind === "history-search" && display.footerLength ? textBlock(raw.slice(-display.footerLength), theme, "muted").render(width) : []), ...(submitted ? ["", ...submittedText.render(width)] : [])];
|
|
125
129
|
}
|
|
126
130
|
const summaryRows = summaryText.render(width).slice(0, 5);
|
|
127
131
|
const previewRows = summaryOnly && !context.isError && !isPartial ? [] : searchPreviews
|