pi-posthorse 0.10.0 → 0.10.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,11 +2,11 @@
2
2
 
3
3
  Posthorse starts fresh context windows for long [Pi](https://github.com/earendil-works/pi) tasks without an automatic model-generated summary. Stock Pi already preserves the session journal and retains a recent tail during compaction; Posthorse adds explicit notes and history retrieval, with a handoff or bounded recovery record instead of the automatic summary.
4
4
 
5
- ![A Pi conversation rolls into a fresh context window. Task notes and the complete session transcript remain available.](.github/readme/context-rollover.png)
5
+ ![A Pi conversation rolls into a fresh context window. Task notes and the complete session transcript remain available.](https://raw.githubusercontent.com/fitchmultz/pi-posthorse/main/.github/readme/context-rollover.png)
6
6
 
7
7
  ## Install and start
8
8
 
9
- Use Node 24.12.0 or later and Pi 1.3.0 or later. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
9
+ Check the [Node.js and Pi requirements](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#install) before installing. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
10
10
 
11
11
  Update an older Pi installation before you install Posthorse. Run these commands:
12
12
 
@@ -27,9 +27,13 @@ Posthorse can nudge the model to update task notes before rollover, then continu
27
27
 
28
28
  Automatic rollover retains a safe recent-message tail using Pi's `compaction.keepRecentTokens`, alongside a bounded recovery record for discarded inputs and tool evidence. It never calls a summarization model. The model must restore notes and check live state before it continues. Optional `new_context` deliberately retains no prior messages; manual `/compact` uses Pi's normal summarization.
29
29
 
30
- Safe recent-tail retention and the notes-only reminder are part of the [0.10.0 changes](CHANGELOG.md#0100). Match these docs to your installed version using the changelog; default-branch documentation may describe changes not yet published.
30
+ Posthorse does not schedule rollover when a provider cache expires. An idle session below the
31
+ rollover threshold keeps its context; use an explicit `new_context` when a deliberate clean reset
32
+ is useful. See [idle sessions and cache expiry](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#idle-sessions-and-cache-expiry).
31
33
 
32
- Recoverable history is not guaranteed model recall. See the [practical guarantees and limits](docs/context-recovery.md) for the stock-Pi comparison, search scope, checkpoints, privacy, concurrency and cost, or the [rollover reference](docs/reference.md#how-it-works) for implementation details.
34
+ See the [changelog](https://github.com/fitchmultz/pi-posthorse/blob/main/CHANGELOG.md) for release history. Default-branch documentation may describe changes not yet published.
35
+
36
+ Recoverable history is not guaranteed model recall. See the [practical guarantees and limits](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/context-recovery.md) for the stock-Pi comparison, search scope, checkpoints, privacy, concurrency and cost, or the [rollover reference](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#how-it-works) for implementation details.
33
37
 
34
38
  ## Tools
35
39
 
@@ -44,7 +48,7 @@ These tools are available to the model:
44
48
 
45
49
  Expand a tool card with Ctrl+O. In fullscreen mode, click a tool card to expand it. A `new_context` card shows a request. Pi's `Compacted from … tokens` card confirms the rollover.
46
50
 
47
- See [tool examples and paging details](docs/reference.md#tools).
51
+ See [tool examples and paging details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#tools).
48
52
 
49
53
  ## Settings and limits
50
54
 
@@ -60,10 +64,10 @@ Two Posthorse copies can register the same tools and compete for rollover. Keep
60
64
 
61
65
  Multiple compaction-policy extensions can make rollover depend on load order. Enable only one.
62
66
 
63
- Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Background work outside the foreground tool batch continues across rollover.
67
+ Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Posthorse's automatic and explicit rollovers do not stop background work outside the foreground tool batch. Manual `/compact` follows Pi's abort behavior instead.
64
68
 
65
- See [size limits and extension order](docs/reference.md#settings).
66
- Check the [legacy-session details](docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
69
+ See [size limits and extension order](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#settings).
70
+ Check the [legacy-session details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
67
71
 
68
72
  ## Update or remove
69
73
 
@@ -72,7 +76,7 @@ pi update npm:pi-posthorse
72
76
  pi remove npm:pi-posthorse
73
77
  ```
74
78
 
75
- Removal leaves notes and Pi session history in place. See the [installation reference](docs/reference.md#install) for Git sources.
79
+ Removal leaves notes and Pi session history in place. See the [installation reference](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#install) for Git sources.
76
80
 
77
81
  ## Data and privacy
78
82
 
@@ -80,14 +84,14 @@ Posthorse makes no network requests. Notes are plaintext files under `.pi/notes/
80
84
 
81
85
  History can include user messages, assistant thinking, tool arguments, errors, and images. Content you retrieve enters the context of your selected model and provider.
82
86
 
83
- See [storage and privacy details](docs/reference.md#data-and-privacy).
84
- For cost diagnosis, see the [cached-token investigation](ARTIFACTS/investigation-report.md).
87
+ See [storage and privacy details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#data-and-privacy).
88
+ For cost diagnosis, see the [cached-token investigation](https://github.com/fitchmultz/pi-posthorse/blob/main/ARTIFACTS/investigation-report.md).
85
89
  It does not establish the cause of a production bill or a measured cost improvement.
86
90
 
87
91
  ## Development
88
92
 
89
- See [development and host qualification](docs/development.md) for setup, tests, and releases.
90
- See [code quality](docs/quality.md) for the required checks.
93
+ See [development and host qualification](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/development.md) for setup, tests, and releases.
94
+ See [code quality](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/quality.md) for the required checks.
91
95
 
92
96
  ## License
93
97
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-posthorse",
3
- "version": "0.10.0",
3
+ "version": "0.10.2",
4
4
  "type": "module",
5
5
  "description": "Posthorse: fresh context, same journey. No-summary context rollover for Pi, with sparse budget reminders, handoffs, notes, and history.",
6
6
  "keywords": [
@@ -33,7 +33,7 @@
33
33
  "scripts": {
34
34
  "check": "tsc",
35
35
  "quality:prepare": "node scripts/quality-prepare.ts",
36
- "quality:policy": "node scripts/quality-policy.ts",
36
+ "quality:policy": "node scripts/quality-policy.ts && node scripts/config-policy.ts",
37
37
  "quality:probes": "npm run quality:prepare && node --test test/quality*.test.ts",
38
38
  "lint": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings .",
39
39
  "lint:agent": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings --format=agent .",
@@ -41,7 +41,7 @@
41
41
  "format": "oxfmt --write .",
42
42
  "format:check": "oxfmt --check .",
43
43
  "build": "npm pack --dry-run",
44
- "verify": "npm run lint && npm run format:check && npm run check:compat && npm run build",
44
+ "verify": "node scripts/toolchain-policy.ts && npm run lint && npm run format:check && npm run check:compat && npm run build",
45
45
  "check:compat": "npm run check && npm test",
46
46
  "test:native": "node --import ./test/pi-loader.ts --test test/host.test.mjs",
47
47
  "test": "npm run quality:prepare && node --import ./test/pi-loader.ts --test test/*.test.ts test/host.test.mjs",
@@ -73,6 +73,7 @@
73
73
  "oxlint": "1.87.0",
74
74
  "oxlint-tsgolint": "7.0.2003",
75
75
  "typebox": "1.3.27",
76
- "typescript": "7.0.2"
76
+ "typescript": "7.0.2",
77
+ "yaml": "2.9.1"
77
78
  }
78
79
  }
package/src/budget.ts CHANGED
@@ -137,10 +137,10 @@ export function freshPayloadChars(
137
137
  ) * 4,
138
138
  );
139
139
  }
140
- export function pageCapacity(ctx: PolicyContext, toolTokens: number): number {
140
+ export function pageCapacity(ctx: PolicyContext, toolTokens: () => number): number {
141
141
  const usage = ctx.getContextUsage();
142
142
  if (usage === undefined || usage.tokens === null) {
143
- return freshPayloadChars(ctx, toolTokens, [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
143
+ return freshPayloadChars(ctx, toolTokens(), [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
144
144
  }
145
145
  const budget = budgetFor(ctx, usage.contextWindow);
146
146
  const line =
package/src/contracts.ts CHANGED
@@ -38,9 +38,12 @@ export type MessageLike = {
38
38
  readonly content?: unknown;
39
39
  readonly summary?: string;
40
40
  readonly toolName?: string;
41
+ readonly customType?: string;
41
42
  readonly namespace?: string;
42
43
  readonly toolCallId?: string;
43
44
  readonly isError?: boolean;
45
+ readonly details?: unknown;
46
+ readonly nestedCalls?: unknown;
44
47
  readonly command?: string;
45
48
  readonly output?: string;
46
49
  readonly excludeFromContext?: boolean;
package/src/files.ts CHANGED
@@ -21,6 +21,7 @@ import {
21
21
  rename,
22
22
  stat,
23
23
  unlink,
24
+ type FileHandle,
24
25
  } from "node:fs/promises";
25
26
  import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
26
27
  import { getAgentDir } from "@earendil-works/pi-coding-agent";
@@ -123,31 +124,50 @@ export async function publishFile(
123
124
  throw error;
124
125
  }
125
126
  }
126
- /** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
127
- export async function* lfLines(
127
+ /** Bounded LF-delimited segments; end also closes an unterminated final line at EOF. */
128
+ export async function* lfChunks(
128
129
  file: string,
129
130
  signal?: AbortSignal,
130
131
  highWaterMark = 64 * 1024,
131
- ): AsyncGenerator<Buffer> {
132
- const stream = createReadStream(file, { signal, highWaterMark });
133
- let pending: Buffer[] = [];
132
+ handle?: Readonly<Pick<FileHandle, "createReadStream">>,
133
+ ): AsyncGenerator<{ readonly bytes: Buffer; readonly end: boolean }> {
134
+ const stream =
135
+ handle === undefined
136
+ ? createReadStream(file, { signal, highWaterMark })
137
+ : handle.createReadStream({ signal, highWaterMark, autoClose: false });
138
+ let openLine = false;
134
139
  for await (const chunk of stream) {
135
140
  if (!Buffer.isBuffer(chunk)) {
136
141
  throw new Error("Expected a byte stream.");
137
142
  }
138
143
  let start = 0;
139
144
  for (let end = chunk.indexOf(10); end !== -1; end = chunk.indexOf(10, start)) {
140
- const line = chunk.subarray(start, end);
141
- yield pending.length === 0 ? line : Buffer.concat([...pending, line]);
142
- pending = [];
145
+ yield { bytes: chunk.subarray(start, end), end: true };
146
+ openLine = false;
143
147
  start = end + 1;
144
148
  }
145
149
  if (start < chunk.length) {
146
- pending.push(chunk.subarray(start));
150
+ yield { bytes: chunk.subarray(start), end: false };
151
+ openLine = true;
147
152
  }
148
153
  }
149
- if (pending.length > 0) {
150
- yield Buffer.concat(pending);
154
+ if (openLine) {
155
+ yield { bytes: Buffer.alloc(0), end: true };
156
+ }
157
+ }
158
+ /** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
159
+ export async function* lfLines(
160
+ file: string,
161
+ signal?: AbortSignal,
162
+ highWaterMark = 64 * 1024,
163
+ ): AsyncGenerator<Buffer> {
164
+ let pending: Buffer[] = [];
165
+ for await (const { bytes, end } of lfChunks(file, signal, highWaterMark)) {
166
+ pending.push(bytes);
167
+ if (end) {
168
+ yield pending.length === 1 ? bytes : Buffer.concat(pending);
169
+ pending = [];
170
+ }
151
171
  }
152
172
  }
153
173
  /** Git's heuristic: a NUL in the first 8,000 bytes marks binary content. */
@@ -1,4 +1,5 @@
1
1
  import { relative, resolve } from "node:path";
2
+ import type { LineMatcher } from "./session-lines.ts";
2
3
  import {
3
4
  isReminderType,
4
5
  requireValue,
@@ -12,7 +13,6 @@ import { shortKey } from "./message-text.ts";
12
13
  import {
13
14
  needleLines,
14
15
  prefilterNeedles,
15
- type RawLine,
16
16
  scopedSessionFiles,
17
17
  sessionWindowEntries,
18
18
  } from "./session-files.ts";
@@ -82,21 +82,30 @@ function entryMessage(entry: EntryLike) {
82
82
  return { content: entry.content, summary: entry.summary };
83
83
  }
84
84
  }
85
- function contextVisibility(projected: readonly ProjectedEntry[]): (entry: EntryLike) => boolean {
86
- const visible = new Map(
87
- projected.flatMap(({ sourceEntry, messages }) =>
88
- sourceEntry.id !== undefined && sourceEntry.id !== "" && messages.length > 0
89
- ? [[sourceEntry.id, messages.map(seenText).join("\n")]]
90
- : [],
91
- ),
92
- );
85
+ function contextVisibility(project: HistorySource["projection"]): (entry: EntryLike) => boolean {
86
+ let visible: Map<string, ProjectedEntry["messages"] | string> | undefined;
93
87
  return (entry) => {
94
88
  if (entry.type === "custom_message" && isReminderType(entry.customType)) {
95
89
  return false;
96
90
  }
97
- const own = entryMessage(entry);
98
- const id = entry.type === "context_edit" ? entry.targetId : entry.id;
99
- return visible.get(id ?? "")?.includes(seenText(own)) === true;
91
+ // Pi owns edits and compaction; only matching candidates need their projected text.
92
+ visible ??= new Map(
93
+ project().flatMap(({ sourceEntry, messages }) =>
94
+ sourceEntry.id !== undefined && sourceEntry.id !== "" && messages.length > 0
95
+ ? [[sourceEntry.id, messages]]
96
+ : [],
97
+ ),
98
+ );
99
+ const id = (entry.type === "context_edit" ? entry.targetId : entry.id) ?? "";
100
+ let text = visible.get(id);
101
+ if (text === undefined) {
102
+ return false;
103
+ }
104
+ if (typeof text !== "string") {
105
+ text = text.map(seenText).join("\n");
106
+ visible.set(id, text);
107
+ }
108
+ return text.includes(seenText(entryMessage(entry)));
100
109
  };
101
110
  }
102
111
  type SearchProgress = {
@@ -193,14 +202,19 @@ async function scanArchive(
193
202
  readonly source: string;
194
203
  readonly query: string;
195
204
  readonly signal?: AbortSignal;
196
- readonly wanted?: (bytes: RawLine) => boolean;
205
+ readonly wanted?: LineMatcher;
197
206
  readonly inContext: (entry: EntryLike) => boolean;
198
207
  },
199
208
  accumulator: SearchProgress,
200
209
  ): Promise<{ readonly groups: readonly (readonly HistoryHit[])[]; readonly skipped: number }> {
201
210
  const collector = archiveCollector(accumulator);
202
211
  let skipped = 0;
203
- for await (const item of sessionWindowEntries(file, options.signal, options.wanted)) {
212
+ for await (const item of sessionWindowEntries(
213
+ file,
214
+ options.signal,
215
+ undefined,
216
+ options.wanted,
217
+ )) {
204
218
  const hit = historyHit(item, options.query, options.source);
205
219
  if (hit === undefined) {
206
220
  continue;
@@ -299,7 +313,9 @@ export async function historyMatches(
299
313
  fail("Invalid history cursor or changed query/scope; restart the search without it.");
300
314
  }
301
315
  const accumulator = new SearchAccumulator(limit, cursor, fail);
302
- const inContext = contextVisibility(source.projection());
316
+ // Archive scans await I/O; keep their visibility snapshot at invocation time.
317
+ const projected = params.all === true ? source.projection() : undefined;
318
+ const inContext = contextVisibility(() => projected ?? source.projection());
303
319
  const skipped =
304
320
  params.all === true
305
321
  ? await archivedMatches(source, { query, signal, inContext }, accumulator)
package/src/history.ts CHANGED
@@ -50,7 +50,7 @@ async function findEntry(
50
50
  for await (const item of sessionWindowEntries(
51
51
  file,
52
52
  signal,
53
- (_bytes, entry) => entry.id === entryId,
53
+ (entry) => entry.id === entryId,
54
54
  )) {
55
55
  return { item, source: relative(source.dir, file) };
56
56
  }
package/src/notes.ts CHANGED
@@ -1,13 +1,7 @@
1
- import {
2
- appendFileSync,
3
- existsSync,
4
- mkdirSync,
5
- readdirSync,
6
- readFileSync,
7
- realpathSync,
8
- statSync,
9
- } from "node:fs";
1
+ import { appendFileSync, existsSync, mkdirSync, readFileSync, statSync } from "node:fs";
2
+ import { readdir, realpath, stat } from "node:fs/promises";
10
3
  import { dirname, isAbsolute, join, normalize, sep } from "node:path";
4
+ import { setImmediate } from "node:timers/promises";
11
5
  import {
12
6
  formatSize,
13
7
  withFileMutationQueue,
@@ -18,6 +12,7 @@ import { Type } from "typebox";
18
12
  import { toolCards } from "../ui.ts";
19
13
  import {
20
14
  MAX_PAGE_CHARS,
15
+ isRecord,
21
16
  requireValue,
22
17
  textResult,
23
18
  type PolicyContext,
@@ -36,10 +31,25 @@ export type NotesParams = {
36
31
  };
37
32
  type NoteFile = { readonly path: string; readonly size: number; readonly mtimeMs: number };
38
33
  type NoteHits = { readonly rows: string[]; readonly more: boolean; readonly skipped: number };
34
+ /** Paged operations take a fresh policy snapshot after their filesystem awaits. */
35
+ type NotesAccess = {
36
+ readonly host: () => PolicyContext;
37
+ readonly pages: PageAccess;
38
+ readonly signal: AbortSignal | undefined;
39
+ };
39
40
  /** Larger files are logs or evidence rather than notes; reading them would dominate a search. */
40
41
  const SEARCH_FILE_BYTES = 1024 * 1024;
41
42
  /** Overlapping reads cut a full search of a 212,000-file notes tree from 39 s to 14 s. */
42
43
  const SEARCH_BATCH = 16;
44
+ /**
45
+ * Maximum explicit `stat` calls overlapped per batch. On a 212,751-note tree, 16, 64 and 256 took
46
+ * equal time within host noise. The walk enters one folder at a time, so this bounds our pending
47
+ * `stat` calls at any depth. On filesystems that report unknown entry types, Node 24's
48
+ * `readdir({ withFileTypes: true })` also issues its own `lstat` for every entry at once.
49
+ */
50
+ const WALK_BATCH = 64;
51
+ /** Formatting all 212,751 rows of one listing at once stalled Pi for 156-163 ms. */
52
+ const FORMAT_BATCH = 10_000;
43
53
  /** Search rescans up to the requested offset, so this also caps the matches one call keeps. */
44
54
  const SEARCH_OFFSET_LIMIT = 1_000_000;
45
55
  function safeJoin(dir: string, path: string): string {
@@ -49,46 +59,105 @@ function safeJoin(dir: string, path: string): string {
49
59
  }
50
60
  return join(dir, relative);
51
61
  }
52
- function walk(directory: string, ancestors: readonly string[] = []): NoteFile[] {
53
- const root = realpathSync(directory);
62
+ /** Like `statSync(path, { throwIfNoEntry: false })`: an entry deleted mid-walk is skipped. */
63
+ async function statIfPresent(path: string) {
64
+ try {
65
+ return await stat(path);
66
+ } catch (error) {
67
+ if (isRecord(error) && (error.code === "ENOENT" || error.code === "ENOTDIR")) {
68
+ return;
69
+ }
70
+ throw error;
71
+ }
72
+ }
73
+ /** A folder's present entries in listing order. */
74
+ async function* folderEntries(directory: string, signal: AbortSignal | undefined) {
75
+ const entries = await readdir(directory, { withFileTypes: true });
76
+ let start = 0;
77
+ while (start < entries.length) {
78
+ signal?.throwIfAborted();
79
+ const batch = entries.slice(start, start + WALK_BATCH);
80
+ const firstFolder = batch.findIndex(
81
+ (entry) => entry.isDirectory() || entry.isSymbolicLink(),
82
+ );
83
+ const paths = batch
84
+ .slice(0, firstFolder === -1 ? batch.length : firstFolder + 1)
85
+ .map((entry) => join(directory, entry.name));
86
+ // Stats overlap only up to the next entry that may be a folder.
87
+ // oxlint-disable-next-line no-await-in-loop
88
+ const infos = await Promise.all(paths.map(statIfPresent));
89
+ for (const [index, path] of paths.entries()) {
90
+ start += 1;
91
+ const info = infos[index];
92
+ if (info !== undefined) {
93
+ yield { path, info };
94
+ }
95
+ if (info?.isDirectory() === true) {
96
+ // Stat later entries only after this folder's subtree is walked, discarding any
97
+ // already taken, so entries removed meanwhile are skipped.
98
+ break;
99
+ }
100
+ }
101
+ }
102
+ }
103
+ /** Files depth-first in directory-listing order, yielding between filesystem calls. */
104
+ async function walk(
105
+ directory: string,
106
+ signal: AbortSignal | undefined,
107
+ ancestors: readonly string[] = [],
108
+ ): Promise<NoteFile[]> {
109
+ signal?.throwIfAborted();
110
+ const root = await realpath(directory);
54
111
  if (ancestors.includes(root)) {
55
112
  return [];
56
113
  }
57
114
  const next = [...ancestors, root];
58
- return readdirSync(directory).flatMap((file) => {
59
- const path = join(directory, file);
60
- const info = statSync(path, { throwIfNoEntry: false });
61
- if (info === undefined) {
62
- return [];
115
+ const files: NoteFile[] = [];
116
+ for await (const { path, info } of folderEntries(directory, signal)) {
117
+ if (info.isDirectory()) {
118
+ for (const file of await walk(path, signal, next)) {
119
+ files.push(file);
120
+ }
121
+ } else {
122
+ files.push({ path, size: info.size, mtimeMs: info.mtimeMs });
63
123
  }
64
- return info.isDirectory()
65
- ? walk(path, next)
66
- : [{ path, size: info.size, mtimeMs: info.mtimeMs }];
67
- });
124
+ }
125
+ return files;
68
126
  }
69
- function notesUnder(target: string): NoteFile[] {
70
- const info = statSync(target, { throwIfNoEntry: false });
127
+ async function notesUnder(target: string, signal: AbortSignal | undefined): Promise<NoteFile[]> {
128
+ const info = await statIfPresent(target);
129
+ signal?.throwIfAborted();
71
130
  if (info === undefined) {
72
131
  return [];
73
132
  }
74
133
  const files = info.isDirectory()
75
- ? walk(target)
134
+ ? await walk(target, signal)
76
135
  : [{ path: target, size: info.size, mtimeMs: info.mtimeMs }];
136
+ signal?.throwIfAborted();
77
137
  return files.sort((a, b) => b.mtimeMs - a.mtimeMs);
78
138
  }
79
- function listNotes(
139
+ async function listNotes(
80
140
  dir: string,
81
141
  params: NotesParams,
82
- host: PolicyContext,
83
- pages: PageAccess,
84
- ): TextResult {
142
+ access: NotesAccess,
143
+ ): Promise<TextResult> {
144
+ const { host, pages, signal } = access;
85
145
  const folder = params.path;
86
146
  const target = folder === undefined || folder === "" ? dir : safeJoin(dir, folder);
87
- const rows = notesUnder(target).map(
88
- (file) =>
89
- `${file.path.slice(dir.length + 1)} ${formatSize(file.size)} ${new Date(file.mtimeMs).toISOString().slice(0, 16)}Z`,
90
- );
91
- return pages.notes(host, {
147
+ const files = await notesUnder(target, signal);
148
+ const rows: string[] = [];
149
+ for (let start = 0; start < files.length; start += FORMAT_BATCH) {
150
+ // Pi gets an event-loop turn before each batch of rows.
151
+ // oxlint-disable-next-line no-await-in-loop
152
+ await setImmediate();
153
+ signal?.throwIfAborted();
154
+ for (const file of files.slice(start, start + FORMAT_BATCH)) {
155
+ rows.push(
156
+ `${file.path.slice(dir.length + 1)} ${formatSize(file.size)} ${new Date(file.mtimeMs).toISOString().slice(0, 16)}Z`,
157
+ );
158
+ }
159
+ }
160
+ return pages.notes(host(), {
92
161
  rows,
93
162
  offset: params.offset ?? 0,
94
163
  kind: "notes-list",
@@ -99,9 +168,9 @@ function listNotes(
99
168
  async function readNote(
100
169
  dir: string,
101
170
  params: NotesParams,
102
- getHost: () => PolicyContext,
103
- pages: PageAccess,
171
+ access: NotesAccess,
104
172
  ): Promise<TextResult> {
173
+ const { pages } = access;
105
174
  const relative = requireValue(params.path, "path", params.op);
106
175
  const path = safeJoin(dir, relative);
107
176
  const info = statSync(path, { throwIfNoEntry: false });
@@ -109,7 +178,7 @@ async function readNote(
109
178
  throw new Error(`No note at ${relative}. Use op "list" to see available notes.`);
110
179
  }
111
180
  if (info.isDirectory()) {
112
- return listNotes(dir, params, getHost(), pages);
181
+ return listNotes(dir, params, access);
113
182
  }
114
183
  if (await isBinaryFile(path)) {
115
184
  throw new Error(
@@ -122,7 +191,7 @@ async function readNote(
122
191
  throw new Error(`Offset ${offset} is past the end of ${relative} (${text.length} chars).`);
123
192
  }
124
193
  const header = `File: ${path}\n`;
125
- const host = getHost();
194
+ const host = access.host();
126
195
  const chars = pages.size(host, { offset }) - header.length;
127
196
  if (chars <= 0) {
128
197
  pages.error(
@@ -197,11 +266,13 @@ async function noteHits(
197
266
  limit: number,
198
267
  signal?: AbortSignal,
199
268
  ): Promise<NoteHits> {
200
- const notes = notesUnder(dir);
269
+ const notes = await notesUnder(dir, signal);
201
270
  const rows: string[] = [];
202
271
  let length = -1;
203
272
  let skipped = 0;
204
273
  for (let start = 0; start < notes.length; start += SEARCH_BATCH) {
274
+ // A batch of notes over 1 MiB reads nothing that would notice cancellation.
275
+ signal?.throwIfAborted();
205
276
  // Batches keep note order, so a search stops within one batch of the page limit.
206
277
  // oxlint-disable-next-line no-await-in-loop
207
278
  const batch = await Promise.all(
@@ -231,21 +302,20 @@ async function noteHits(
231
302
  async function searchNotes(
232
303
  dir: string,
233
304
  params: NotesParams,
234
- access: { readonly host: PolicyContext; readonly pages: PageAccess },
235
- signal?: AbortSignal,
305
+ access: NotesAccess,
236
306
  ): Promise<TextResult> {
237
307
  const { host, pages } = access;
238
308
  const query = requireValue(params.query, "query", params.op);
239
309
  const offset = params.offset ?? 0;
240
310
  if (offset >= SEARCH_OFFSET_LIMIT) {
241
311
  pages.error(
242
- host,
312
+ host(),
243
313
  `Notes search pages end before offset ${SEARCH_OFFSET_LIMIT}; search for something more specific.`,
244
314
  );
245
315
  }
246
- const hits = await noteHits(dir, query.toLowerCase(), offset + MAX_PAGE_CHARS, signal);
316
+ const hits = await noteHits(dir, query.toLowerCase(), offset + MAX_PAGE_CHARS, access.signal);
247
317
  const skipped = hits.skipped === 1 ? "1 note" : `${hits.skipped} notes`;
248
- return pages.notes(host, {
318
+ return pages.notes(host(), {
249
319
  rows: hits.rows,
250
320
  more: hits.more,
251
321
  offset,
@@ -297,17 +367,18 @@ export function registerNotes(
297
367
  // oxlint-disable-next-line max-params
298
368
  async execute(_id, params, signal, _onUpdate, ctx) {
299
369
  const dir = notesDir(ctx.cwd);
370
+ const access: NotesAccess = { host: () => policy(ctx), pages, signal };
300
371
  switch (params.op) {
301
372
  case "list":
302
- return listNotes(dir, params, policy(ctx), pages);
373
+ return listNotes(dir, params, access);
303
374
  case "read":
304
- return readNote(dir, params, () => policy(ctx), pages);
375
+ return readNote(dir, params, access);
305
376
  case "write":
306
377
  return writeNote(dir, params, signal);
307
378
  case "append":
308
379
  return appendNote(dir, params, signal);
309
380
  case "search":
310
- return searchNotes(dir, params, { host: policy(ctx), pages }, signal);
381
+ return searchNotes(dir, params, access);
311
382
  }
312
383
  },
313
384
  });
package/src/paging.ts CHANGED
@@ -63,7 +63,7 @@ export class PageReservations {
63
63
  );
64
64
  }
65
65
  this.previousUsage = usage;
66
- const remaining = pageCapacity(ctx, this.toolTokens()) - this.pendingTokens;
66
+ const remaining = pageCapacity(ctx, this.toolTokens) - this.pendingTokens;
67
67
  const chars = Math.min(
68
68
  MAX_PAGE_CHARS,
69
69
  Math.max(0, remaining - PAGE_MARGIN_TOKENS) * 4 -