pi-posthorse 0.9.3 → 0.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (17) hide show
  1. package/README.md +33 -26
  2. package/index.ts +1 -1
  3. package/package.json +10 -9
  4. package/src/budget.ts +5 -5
  5. package/src/context-tools.ts +3 -3
  6. package/src/contracts.ts +1 -0
  7. package/src/files.ts +87 -25
  8. package/src/history-search.ts +31 -15
  9. package/src/history.ts +1 -1
  10. package/src/notes.ts +116 -45
  11. package/src/paging.ts +1 -1
  12. package/src/recovery-results.ts +5 -2
  13. package/src/recovery.ts +59 -21
  14. package/src/retention.ts +203 -0
  15. package/src/rollover.ts +29 -20
  16. package/src/session-files.ts +37 -24
  17. package/src/session-lines.ts +108 -0
package/README.md CHANGED
@@ -1,12 +1,12 @@
1
1
  # Posthorse
2
2
 
3
- Posthorse starts fresh context windows for long [Pi](https://github.com/earendil-works/pi) tasks and keeps earlier conversation recoverable. Each rollover uses a handoff or recovery record without a model-generated summary.
3
+ Posthorse starts fresh context windows for long [Pi](https://github.com/earendil-works/pi) tasks without an automatic model-generated summary. Stock Pi already preserves the session journal and retains a recent tail during compaction; Posthorse adds explicit notes and history retrieval, with a handoff or bounded recovery record instead of the automatic summary.
4
4
 
5
- ![A Pi conversation rolls into a fresh context window. Task notes and the complete session transcript remain available.](.github/readme/context-rollover.png)
5
+ ![A Pi conversation rolls into a fresh context window. Task notes and the complete session transcript remain available.](https://raw.githubusercontent.com/fitchmultz/pi-posthorse/main/.github/readme/context-rollover.png)
6
6
 
7
7
  ## Install and start
8
8
 
9
- Use Node 24.12.0 or later and Pi 1.0.0 or later. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
9
+ Check the [Node.js and Pi requirements](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#install) before installing. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
10
10
 
11
11
  Update an older Pi installation before you install Posthorse. Run these commands:
12
12
 
@@ -23,44 +23,51 @@ Give Pi your task. For example:
23
23
 
24
24
  ## How rollover works
25
25
 
26
- Posthorse can remind the model to save task state before rollover. Large turns can reach rollover before this reminder. The reminder does not wake an agent after its final answer.
26
+ Posthorse can nudge the model to update task notes before rollover, then continue working. The nudge does not require `new_context` or wake an agent after its final answer. Large turns can reach rollover before the nudge.
27
27
 
28
- Automatic rollover creates a recovery record from session inputs and tool results. The model must restore notes and check live state before it continues. Manual `/compact` uses Pi's normal summarization.
28
+ Automatic rollover retains a safe recent-message tail using Pi's `compaction.keepRecentTokens`, alongside a bounded recovery record for discarded inputs and tool evidence. It never calls a summarization model. The model must restore notes and check live state before it continues. Optional `new_context` deliberately retains no prior messages; manual `/compact` uses Pi's normal summarization.
29
29
 
30
- See [rollover and recovery details](docs/reference.md#how-it-works).
30
+ Posthorse does not schedule rollover when a provider cache expires. An idle session below the
31
+ rollover threshold keeps its context; use an explicit `new_context` when a deliberate clean reset
32
+ is useful. See [idle sessions and cache expiry](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#idle-sessions-and-cache-expiry).
33
+
34
+ See the [changelog](https://github.com/fitchmultz/pi-posthorse/blob/main/CHANGELOG.md) for release history. Default-branch documentation may describe changes not yet published.
35
+
36
+ Recoverable history is not guaranteed model recall. See the [practical guarantees and limits](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/context-recovery.md) for the stock-Pi comparison, search scope, checkpoints, privacy, concurrency and cost, or the [rollover reference](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#how-it-works) for implementation details.
31
37
 
32
38
  ## Tools
33
39
 
34
40
  These tools are available to the model:
35
41
 
36
- | Tool | Use |
37
- | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
38
- | `new_context({ handoff? })` | Request a fresh window after every foreground tool succeeds. Aborted turns and failed tools prevent the reset. |
39
- | `get_context_remaining()` | Estimate tokens left until rollover and the configured context limit. Usage can be unknown before a model response. |
40
- | `notes({ op, ... })` | List, read, search, write, or append task notes. |
41
- | `history({ op, ... })` | Search earlier conversation and read entries, including images. `all: true` includes this project's other sessions and nested subagent runs. |
42
+ | Tool | Use |
43
+ | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
44
+ | `new_context({ handoff? })` | Optionally request a clean reset with no retained messages after every foreground tool succeeds. Aborted turns and failed tools prevent the reset. |
45
+ | `get_context_remaining()` | Estimate tokens left until rollover and the configured context limit. Usage can be unknown before a model response. |
46
+ | `notes({ op, ... })` | List, read, search, write, or append task notes. |
47
+ | `history({ op, ... })` | Search earlier conversation and read entries, including images. `all: true` includes this project's other sessions and nested subagent runs. |
42
48
 
43
49
  Expand a tool card with Ctrl+O. In fullscreen mode, click a tool card to expand it. A `new_context` card shows a request. Pi's `Compacted from … tokens` card confirms the rollover.
44
50
 
45
- See [tool examples and paging details](docs/reference.md#tools).
51
+ See [tool examples and paging details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#tools).
46
52
 
47
53
  ## Settings and limits
48
54
 
49
55
  Posthorse uses Pi's effective `compaction` settings. It has no separate settings.
50
56
 
51
- | Pi setting | Effect |
52
- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
53
- | `compaction.enabled` | Set to `false` to disable reminders and automatic rollover. `new_context` remains available. |
54
- | `compaction.reserveTokens` | Automatic Posthorse rollover needs at least 10,000 tokens after this reserve. Smaller budgets use Pi's own compaction. Lower the reserve or choose a larger model. |
57
+ | Pi setting | Effect |
58
+ | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
59
+ | `compaction.enabled` | Set to `false` to disable reminders and automatic rollover. `new_context` remains available. |
60
+ | `compaction.reserveTokens` | Automatic Posthorse rollover needs at least 10,000 tokens after this reserve. Smaller budgets use Pi's own compaction. Lower the reserve or choose a larger model. |
61
+ | `compaction.keepRecentTokens` | Approximate recent tail for automatic rollover (Pi's default: 20,000). Zero keeps no messages. Tail size shrinks when safe groups cannot fit the fresh budget. |
55
62
 
56
63
  Two Posthorse copies can register the same tools and compete for rollover. Keep one copy loaded. Check installed sources with `pi list`.
57
64
 
58
65
  Multiple compaction-policy extensions can make rollover depend on load order. Enable only one.
59
66
 
60
- Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Background work outside the foreground tool batch continues across rollover.
67
+ Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Posthorse's automatic and explicit rollovers do not stop background work outside the foreground tool batch. Manual `/compact` follows Pi's abort behavior instead.
61
68
 
62
- See [size limits and extension order](docs/reference.md#settings).
63
- Check the [legacy-session details](docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
69
+ See [size limits and extension order](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#settings).
70
+ Check the [legacy-session details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
64
71
 
65
72
  ## Update or remove
66
73
 
@@ -69,22 +76,22 @@ pi update npm:pi-posthorse
69
76
  pi remove npm:pi-posthorse
70
77
  ```
71
78
 
72
- Removal leaves notes and Pi session history in place. See the [installation reference](docs/reference.md#install) for Git sources.
79
+ Removal leaves notes and Pi session history in place. See the [installation reference](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#install) for Git sources.
73
80
 
74
81
  ## Data and privacy
75
82
 
76
- Posthorse makes no network requests. Notes are plaintext files under `.pi/notes/`; linked Git worktrees share them. Notes for the Git packages Pi installs in `~/.pi/agent/git/` live under `~/.pi/agent/notes/` instead, because `pi update` deletes untracked files in those checkouts. Add `.pi/notes/` to `.gitignore` if the notes must stay private.
83
+ Posthorse makes no network requests. Notes are plaintext files under `.pi/notes/`; linked Git worktrees share them. Notes for Pi-managed Git packages live outside their checkouts because `pi update` deletes untracked files there: global packages (including their descendants) use Pi's agent-directory `notes/`, and other project-local packages use the containing project's `.pi/package-notes/`, with project-local stores keyed by a full SHA-256 digest of the host and package path. Access existing checkout-local notes with this version before updating the package so they can migrate. Add `.pi/notes/` and `.pi/package-notes/` to the containing project's `.gitignore` if the notes must stay private.
77
84
 
78
85
  History can include user messages, assistant thinking, tool arguments, errors, and images. Content you retrieve enters the context of your selected model and provider.
79
86
 
80
- See [storage and privacy details](docs/reference.md#data-and-privacy).
81
- For cost diagnosis, see the [cached-token investigation](ARTIFACTS/investigation-report.md).
87
+ See [storage and privacy details](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/reference.md#data-and-privacy).
88
+ For cost diagnosis, see the [cached-token investigation](https://github.com/fitchmultz/pi-posthorse/blob/main/ARTIFACTS/investigation-report.md).
82
89
  It does not establish the cause of a production bill or a measured cost improvement.
83
90
 
84
91
  ## Development
85
92
 
86
- See [development and host qualification](docs/development.md) for setup, tests, and releases.
87
- See [code quality](docs/quality.md) for the required checks.
93
+ See [development and host qualification](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/development.md) for setup, tests, and releases.
94
+ See [code quality](https://github.com/fitchmultz/pi-posthorse/blob/main/docs/quality.md) for the required checks.
88
95
 
89
96
  ## License
90
97
 
package/index.ts CHANGED
@@ -1,7 +1,7 @@
1
1
  /**
2
2
  * Posthorse — fresh context, same journey.
3
3
  *
4
- * Pi owns persisted retain-none context boundaries. Posthorse owns reminder and
4
+ * Pi owns persisted context boundaries. Posthorse owns recent-tail retention, reminder and
5
5
  * rollover policy, bounded recovery, durable notes, and history retrieval.
6
6
  */
7
7
  import {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-posthorse",
3
- "version": "0.9.3",
3
+ "version": "0.10.1",
4
4
  "type": "module",
5
5
  "description": "Posthorse: fresh context, same journey. No-summary context rollover for Pi, with sparse budget reminders, handoffs, notes, and history.",
6
6
  "keywords": [
@@ -33,7 +33,7 @@
33
33
  "scripts": {
34
34
  "check": "tsc",
35
35
  "quality:prepare": "node scripts/quality-prepare.ts",
36
- "quality:policy": "node scripts/quality-policy.ts",
36
+ "quality:policy": "node scripts/quality-policy.ts && node scripts/config-policy.ts",
37
37
  "quality:probes": "npm run quality:prepare && node --test test/quality*.test.ts",
38
38
  "lint": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings .",
39
39
  "lint:agent": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings --format=agent .",
@@ -41,15 +41,15 @@
41
41
  "format": "oxfmt --write .",
42
42
  "format:check": "oxfmt --check .",
43
43
  "build": "npm pack --dry-run",
44
- "verify": "npm run lint && npm run format:check && npm run check:compat && npm run build",
44
+ "verify": "node scripts/toolchain-policy.ts && npm run lint && npm run format:check && npm run check:compat && npm run build",
45
45
  "check:compat": "npm run check && npm test",
46
46
  "test:native": "node --import ./test/pi-loader.ts --test test/host.test.mjs",
47
47
  "test": "npm run quality:prepare && node --import ./test/pi-loader.ts --test test/*.test.ts test/host.test.mjs",
48
48
  "prepublishOnly": "npm run verify"
49
49
  },
50
50
  "peerDependencies": {
51
- "@earendil-works/pi-coding-agent": "*",
52
- "@earendil-works/pi-tui": "*",
51
+ "@earendil-works/pi-coding-agent": ">=1.3.0",
52
+ "@earendil-works/pi-tui": ">=1.3.0",
53
53
  "typebox": "*"
54
54
  },
55
55
  "peerDependenciesMeta": {
@@ -64,15 +64,16 @@
64
64
  }
65
65
  },
66
66
  "devDependencies": {
67
- "@earendil-works/pi-ai": "1.0.0",
68
- "@earendil-works/pi-coding-agent": "1.0.0",
69
- "@earendil-works/pi-tui": "1.0.0",
67
+ "@earendil-works/pi-ai": "1.3.0",
68
+ "@earendil-works/pi-coding-agent": "1.3.0",
69
+ "@earendil-works/pi-tui": "1.3.0",
70
70
  "@types/node": "24.19.1",
71
71
  "oxc-parser": "0.153.0",
72
72
  "oxfmt": "0.72.0",
73
73
  "oxlint": "1.87.0",
74
74
  "oxlint-tsgolint": "7.0.2003",
75
75
  "typebox": "1.3.27",
76
- "typescript": "7.0.2"
76
+ "typescript": "7.0.2",
77
+ "yaml": "2.9.1"
77
78
  }
78
79
  }
package/src/budget.ts CHANGED
@@ -14,7 +14,7 @@ import { imagesOf, textOf } from "./message-text.ts";
14
14
 
15
15
  const MIN_USABLE_TOKENS = Math.ceil(MAX_HANDOFF_CHARS / 4) * 2;
16
16
  export const CHECKPOINT_STEPS =
17
- "update the task's current-state note (goal, progress, decisions, next steps) with available file-editing tools at its absolute path; use notes write only for creation, substantial restructuring, or when editing tools are unavailable. If the note is already current, leave it unchanged. Then call new_context";
17
+ "update the task's current-state note (goal, progress, decisions, next steps) with available file-editing tools at its absolute path; use notes write only for creation, substantial restructuring, or when editing tools are unavailable. If the note is already current, leave it unchanged. Then continue working; let automatic rollover happen without calling new_context";
18
18
  export type Budget = {
19
19
  readonly contextWindow: number;
20
20
  readonly reserveTokens: number;
@@ -84,7 +84,7 @@ function automaticGuidance(budget: Budget | undefined, enabled: boolean): string
84
84
  budget === undefined
85
85
  ? ""
86
86
  : ` The rollover line is ${budget.rolloverAt.toLocaleString("en-US")} tokens used under the available Pi settings.`;
87
- return `Automatic rollover follows Pi's compaction setting.${line} At most one checkpoint reminder may arrive at the end of a working turn before the rollover line (${deadline}), never on a turn that ends the run; when it does, stop normal work, ${CHECKPOINT_STEPS}.`;
87
+ return `Automatic rollover follows Pi's compaction setting and retains a safe recent-message tail using compaction.keepRecentTokens, without model summarization.${line} At most one notes reminder may arrive at the end of a working turn before the rollover line (${deadline}), never on a turn that ends the run; when it does, ${CHECKPOINT_STEPS}. new_context is an optional deliberate clean reset, not a checkpoint requirement.`;
88
88
  }
89
89
  export function buildGuidance(ctx: PolicyContext): string {
90
90
  const budget = budgetFor(ctx);
@@ -92,7 +92,7 @@ export function buildGuidance(ctx: PolicyContext): string {
92
92
  const capacity =
93
93
  budget === undefined
94
94
  ? "Configured context capacity is unknown."
95
- : `Configured context capacity: ${budget.contextWindow.toLocaleString("en-US")} tokens. This is Pi's best available native limit, not remaining space or a measured provider boundary. Fresh windows use the active configuration; system instructions, tools, the handoff, and new messages consume part of it.`;
95
+ : `Configured context capacity: ${budget.contextWindow.toLocaleString("en-US")} tokens. This is Pi's best available native limit, not remaining space or a measured provider boundary. Fresh windows use the active configuration; system instructions, tools, the handoff, retained recent messages, and new messages consume part of it.`;
96
96
  return `## Context self-management (Posthorse)
97
97
  ${capacity}
98
98
  Do not guess context capacity or remaining space from response-token limits or reasoning budgets. Before reporting a remaining-token count or changing your work plan because of context limits, call get_context_remaining. If usage is unknown, report it as unknown. No routine budget checks are needed.
@@ -137,10 +137,10 @@ export function freshPayloadChars(
137
137
  ) * 4,
138
138
  );
139
139
  }
140
- export function pageCapacity(ctx: PolicyContext, toolTokens: number): number {
140
+ export function pageCapacity(ctx: PolicyContext, toolTokens: () => number): number {
141
141
  const usage = ctx.getContextUsage();
142
142
  if (usage === undefined || usage.tokens === null) {
143
- return freshPayloadChars(ctx, toolTokens, [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
143
+ return freshPayloadChars(ctx, toolTokens(), [], MAX_PAGE_CHARS) / 4 + PAGE_MARGIN_TOKENS;
144
144
  }
145
145
  const budget = budgetFor(ctx, usage.contextWindow);
146
146
  const line =
package/src/context-tools.ts CHANGED
@@ -60,8 +60,8 @@ export function registerContextTools(
60
60
  label: "New Context",
61
61
  ...toolCards("new_context"),
62
62
  description:
63
- "Start a fresh context window once the current foreground tools succeed; background work continues across the reset. Earlier conversation stays recoverable through history. Put continuation state in handoff, or save fuller state in notes first.",
64
- promptSnippet: "request a fresh context window with an optional handoff",
63
+ "Optionally request a deliberate clean context reset once the current foreground tools succeed; retains no prior messages, unlike automatic rollover. Not required after a notes reminder. Background work continues. Earlier conversation stays recoverable through history. Put continuation state in handoff, or save fuller state in notes first.",
64
+ promptSnippet: "optionally request a clean context reset with an optional handoff",
65
65
  parameters: Type.Object({
66
66
  handoff: Type.Optional(
67
67
  Type.String({
@@ -94,7 +94,7 @@ export function registerContextTools(
94
94
  label: "Context Remaining",
95
95
  ...toolCards("get_context_remaining"),
96
96
  description:
97
- "Estimate tokens left before Pi's automatic rollover and the configured context limit. Check before reporting remaining space or changing your work plan because of context limits. A checkpoint reminder may arrive before rollover on a working turn; no routine checks are needed.",
97
+ "Estimate tokens left before Pi's automatic rollover and the configured context limit. Check before reporting remaining space or changing your work plan because of context limits. A notes-only reminder may arrive before rollover on a working turn; continue afterward and let rollover happen automatically. No routine checks are needed.",
98
98
  promptSnippet:
99
99
  "verify remaining context before reporting a token count or changing plans because of context limits",
100
100
  parameters: Type.Object({}),
package/src/contracts.ts CHANGED
@@ -4,6 +4,7 @@ import type { PosthorseDisplay } from "../ui.ts";
4
4
  export type CompactionPolicy = {
5
5
  readonly enabled: boolean;
6
6
  readonly reserveTokens: number;
7
+ readonly keepRecentTokens?: number;
7
8
  };
8
9
  export type ContextUsage = {
9
10
  readonly tokens: number | null;
package/src/files.ts CHANGED
@@ -1,4 +1,4 @@
1
- import { randomUUID } from "node:crypto";
1
+ import { createHash, randomUUID } from "node:crypto";
2
2
  import {
3
3
  createReadStream,
4
4
  existsSync,
@@ -21,6 +21,7 @@ import {
21
21
  rename,
22
22
  stat,
23
23
  unlink,
24
+ type FileHandle,
24
25
  } from "node:fs/promises";
25
26
  import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
26
27
  import { getAgentDir } from "@earendil-works/pi-coding-agent";
@@ -123,31 +124,50 @@ export async function publishFile(
123
124
  throw error;
124
125
  }
125
126
  }
126
- /** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
127
- export async function* lfLines(
127
+ /** Bounded LF-delimited segments; end also closes an unterminated final line at EOF. */
128
+ export async function* lfChunks(
128
129
  file: string,
129
130
  signal?: AbortSignal,
130
131
  highWaterMark = 64 * 1024,
131
- ): AsyncGenerator<Buffer> {
132
- const stream = createReadStream(file, { signal, highWaterMark });
133
- let pending: Buffer[] = [];
132
+ handle?: Readonly<Pick<FileHandle, "createReadStream">>,
133
+ ): AsyncGenerator<{ readonly bytes: Buffer; readonly end: boolean }> {
134
+ const stream =
135
+ handle === undefined
136
+ ? createReadStream(file, { signal, highWaterMark })
137
+ : handle.createReadStream({ signal, highWaterMark, autoClose: false });
138
+ let openLine = false;
134
139
  for await (const chunk of stream) {
135
140
  if (!Buffer.isBuffer(chunk)) {
136
141
  throw new Error("Expected a byte stream.");
137
142
  }
138
143
  let start = 0;
139
144
  for (let end = chunk.indexOf(10); end !== -1; end = chunk.indexOf(10, start)) {
140
- const line = chunk.subarray(start, end);
141
- yield pending.length === 0 ? line : Buffer.concat([...pending, line]);
142
- pending = [];
145
+ yield { bytes: chunk.subarray(start, end), end: true };
146
+ openLine = false;
143
147
  start = end + 1;
144
148
  }
145
149
  if (start < chunk.length) {
146
- pending.push(chunk.subarray(start));
150
+ yield { bytes: chunk.subarray(start), end: false };
151
+ openLine = true;
147
152
  }
148
153
  }
149
- if (pending.length > 0) {
150
- yield Buffer.concat(pending);
154
+ if (openLine) {
155
+ yield { bytes: Buffer.alloc(0), end: true };
156
+ }
157
+ }
158
+ /** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
159
+ export async function* lfLines(
160
+ file: string,
161
+ signal?: AbortSignal,
162
+ highWaterMark = 64 * 1024,
163
+ ): AsyncGenerator<Buffer> {
164
+ let pending: Buffer[] = [];
165
+ for await (const { bytes, end } of lfChunks(file, signal, highWaterMark)) {
166
+ pending.push(bytes);
167
+ if (end) {
168
+ yield pending.length === 1 ? bytes : Buffer.concat(pending);
169
+ pending = [];
170
+ }
151
171
  }
152
172
  }
153
173
  /** Git's heuristic: a NUL in the first 8,000 bytes marks binary content. */
@@ -197,36 +217,78 @@ function notesRoot(cwd: string): string {
197
217
  }
198
218
  }
199
219
  }
220
+ function packageRelativePath(installs: string, root: string): string | undefined {
221
+ const path = relative(installs, root);
222
+ return path !== "" && path !== ".." && !path.startsWith(`..${sep}`) && !isAbsolute(path)
223
+ ? path
224
+ : undefined;
225
+ }
226
+ function projectPackageNotes(root: string, cwd: string): string | undefined {
227
+ let moved: string | undefined;
228
+ let checkout = root;
229
+ for (let dir = cwd; dirname(dir) !== dir; dir = dirname(dir)) {
230
+ // Retain the logical checkout path when a package container or source segment is linked.
231
+ if (existsSync(join(dir, ".git")) && realpathSync(dir) === root) {
232
+ checkout = dir;
233
+ }
234
+ if (basename(dir) !== "git" || basename(dirname(dir)) !== ".pi") {
235
+ continue;
236
+ }
237
+ const path = packageRelativePath(dir, checkout);
238
+ if (path !== undefined) {
239
+ // Flat full keys keep arbitrary Git URL prefix paths out of one another's note stores.
240
+ const key = createHash("sha256").update(path.split(sep).join("/")).digest("hex");
241
+ moved = join(realpathSync(dirname(dir)), "package-notes", key, "notes");
242
+ }
243
+ }
244
+ // The outermost package container also protects packages installed inside another package.
245
+ return moved;
246
+ }
200
247
  /**
201
- * `pi update` resets the Git packages Pi installs in the agent directory and runs `git clean -fdx`
202
- * in them, deleting notes kept inside. Their notes live in the agent directory under the same host
203
- * and path instead.
204
- * ponytail: packages installed for one project (its `.pi/git/`) still keep notes in the checkout;
205
- * give them a per-project location if sessions start working inside them.
248
+ * Pi's package manager installs Git sources in <agent>/git or <project>/.pi/git and runs
249
+ * `git clean -fdx` inside their checkouts on update. Keep notes alongside, not inside, that tree.
206
250
  * ponytail: notes a package commits under `.pi/notes` return with each update and move into
207
251
  * another `from-checkout-*` folder; skip tracked files if a package ever commits its notes.
208
252
  */
209
- function packageNotes(root: string): string | undefined {
253
+ function packageNotes(root: string, cwd: string): string | undefined {
254
+ // Pi cleans only the checkouts; folders that hold several of them keep their own notes.
255
+ if (!existsSync(join(root, ".git"))) {
256
+ return;
257
+ }
210
258
  const agentDir = getAgentDir();
211
259
  const installs = join(agentDir, "git");
212
- const path = relative(existsSync(installs) ? realpathSync(installs) : installs, root);
213
- const inside =
214
- path !== "" && path !== ".." && !path.startsWith(`..${sep}`) && !isAbsolute(path);
215
- // Pi cleans only the checkouts; folders that hold several of them keep their own notes.
216
- return inside && existsSync(join(root, ".git")) ? join(agentDir, "notes", path) : undefined;
260
+ const path = packageRelativePath(
261
+ existsSync(installs) ? realpathSync(installs) : installs,
262
+ root,
263
+ );
264
+ return path === undefined
265
+ ? (projectPackageNotes(root, cwd) ?? projectPackageNotes(root, root))
266
+ : join(agentDir, "notes", path);
267
+ }
268
+ /** Preserve every existing entry, including empty directories and dangling links. */
269
+ function migrationTarget(moved: string): string {
270
+ if (lstatSync(moved, { throwIfNoEntry: false }) === undefined) {
271
+ return moved;
272
+ }
273
+ const base = join(moved, `from-checkout-${Date.now()}`);
274
+ let target = base;
275
+ for (let suffix = 1; lstatSync(target, { throwIfNoEntry: false }) !== undefined; suffix++) {
276
+ target = `${base}-${suffix}`;
277
+ }
278
+ return target;
217
279
  }
218
280
  /** The notes directory for `cwd`, moving any notes found inside a Pi package checkout. */
219
281
  export function notesDir(cwd: string): string {
220
282
  const root = notesRoot(cwd);
221
283
  const local = join(root, ".pi", "notes");
222
- const moved = packageNotes(root);
284
+ const moved = packageNotes(root, cwd);
223
285
  if (moved === undefined) {
224
286
  return local;
225
287
  }
226
288
  if (lstatSync(local, { throwIfNoEntry: false })?.isDirectory() === true) {
227
289
  mkdirSync(dirname(moved), { recursive: true });
228
290
  // Notes an older Posthorse wrote here after an earlier move keep a folder of their own.
229
- const target = existsSync(moved) ? join(moved, `from-checkout-${Date.now()}`) : moved;
291
+ const target = migrationTarget(moved);
230
292
  try {
231
293
  renameSync(local, target);
232
294
  // Sessions still running an older Posthorse keep reaching the notes until they reload.
package/src/history-search.ts CHANGED
@@ -1,4 +1,5 @@
1
1
  import { relative, resolve } from "node:path";
2
+ import type { LineMatcher } from "./session-lines.ts";
2
3
  import {
3
4
  isReminderType,
4
5
  requireValue,
@@ -12,7 +13,6 @@ import { shortKey } from "./message-text.ts";
12
13
  import {
13
14
  needleLines,
14
15
  prefilterNeedles,
15
- type RawLine,
16
16
  scopedSessionFiles,
17
17
  sessionWindowEntries,
18
18
  } from "./session-files.ts";
@@ -82,21 +82,30 @@ function entryMessage(entry: EntryLike) {
82
82
  return { content: entry.content, summary: entry.summary };
83
83
  }
84
84
  }
85
- function contextVisibility(projected: readonly ProjectedEntry[]): (entry: EntryLike) => boolean {
86
- const visible = new Map(
87
- projected.flatMap(({ sourceEntry, messages }) =>
88
- sourceEntry.id !== undefined && sourceEntry.id !== "" && messages.length > 0
89
- ? [[sourceEntry.id, messages.map(seenText).join("\n")]]
90
- : [],
91
- ),
92
- );
85
+ function contextVisibility(project: HistorySource["projection"]): (entry: EntryLike) => boolean {
86
+ let visible: Map<string, ProjectedEntry["messages"] | string> | undefined;
93
87
  return (entry) => {
94
88
  if (entry.type === "custom_message" && isReminderType(entry.customType)) {
95
89
  return false;
96
90
  }
97
- const own = entryMessage(entry);
98
- const id = entry.type === "context_edit" ? entry.targetId : entry.id;
99
- return visible.get(id ?? "")?.includes(seenText(own)) === true;
91
+ // Pi owns edits and compaction; only matching candidates need their projected text.
92
+ visible ??= new Map(
93
+ project().flatMap(({ sourceEntry, messages }) =>
94
+ sourceEntry.id !== undefined && sourceEntry.id !== "" && messages.length > 0
95
+ ? [[sourceEntry.id, messages]]
96
+ : [],
97
+ ),
98
+ );
99
+ const id = (entry.type === "context_edit" ? entry.targetId : entry.id) ?? "";
100
+ let text = visible.get(id);
101
+ if (text === undefined) {
102
+ return false;
103
+ }
104
+ if (typeof text !== "string") {
105
+ text = text.map(seenText).join("\n");
106
+ visible.set(id, text);
107
+ }
108
+ return text.includes(seenText(entryMessage(entry)));
100
109
  };
101
110
  }
102
111
  type SearchProgress = {
@@ -193,14 +202,19 @@ async function scanArchive(
193
202
  readonly source: string;
194
203
  readonly query: string;
195
204
  readonly signal?: AbortSignal;
196
- readonly wanted?: (bytes: RawLine) => boolean;
205
+ readonly wanted?: LineMatcher;
197
206
  readonly inContext: (entry: EntryLike) => boolean;
198
207
  },
199
208
  accumulator: SearchProgress,
200
209
  ): Promise<{ readonly groups: readonly (readonly HistoryHit[])[]; readonly skipped: number }> {
201
210
  const collector = archiveCollector(accumulator);
202
211
  let skipped = 0;
203
- for await (const item of sessionWindowEntries(file, options.signal, options.wanted)) {
212
+ for await (const item of sessionWindowEntries(
213
+ file,
214
+ options.signal,
215
+ undefined,
216
+ options.wanted,
217
+ )) {
204
218
  const hit = historyHit(item, options.query, options.source);
205
219
  if (hit === undefined) {
206
220
  continue;
@@ -299,7 +313,9 @@ export async function historyMatches(
299
313
  fail("Invalid history cursor or changed query/scope; restart the search without it.");
300
314
  }
301
315
  const accumulator = new SearchAccumulator(limit, cursor, fail);
302
- const inContext = contextVisibility(source.projection());
316
+ // Archive scans await I/O; keep their visibility snapshot at invocation time.
317
+ const projected = params.all === true ? source.projection() : undefined;
318
+ const inContext = contextVisibility(() => projected ?? source.projection());
303
319
  const skipped =
304
320
  params.all === true
305
321
  ? await archivedMatches(source, { query, signal, inContext }, accumulator)
package/src/history.ts CHANGED
@@ -50,7 +50,7 @@ async function findEntry(
50
50
  for await (const item of sessionWindowEntries(
51
51
  file,
52
52
  signal,
53
- (_bytes, entry) => entry.id === entryId,
53
+ (entry) => entry.id === entryId,
54
54
  )) {
55
55
  return { item, source: relative(source.dir, file) };
56
56
  }