pi-posthorse 0.9.0 → 0.9.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -95
- package/package.json +4 -4
- package/src/files.ts +21 -15
- package/src/history-search.ts +7 -7
- package/src/history.ts +8 -7
- package/src/rollover.ts +4 -0
- package/src/session-files.ts +205 -77
- package/src/ui/cards.ts +73 -78
- package/src/ui/text.ts +38 -0
- package/ui.ts +14 -7
package/README.md
CHANGED
|
@@ -1,135 +1,91 @@
|
|
|
1
1
|
# Posthorse
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Posthorse starts fresh context windows for long [Pi](https://github.com/earendil-works/pi) tasks and keeps earlier conversation recoverable. Each rollover uses a handoff or recovery record without a model-generated summary.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+

|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Install and start
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Use Node 24.12.0 or later and Pi 1.0.0 or later. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
- Node `>=24.12.0`.
|
|
14
|
-
- Pi `>=1.0.0`, official (`@earendil-works/pi-coding-agent`) or a maintained fork with the same public compaction contracts. No checkpoint, metadata iterator, or native context-window API is required.
|
|
15
|
-
|
|
16
|
-
Required qualification targets: latest stable official Pi and latest maintained fork `main`, resolved once per workflow run and retained with exact SDK/CLI identity evidence. Locked development dependencies are reproducible build snapshots, not validation targets. Historical official Pi `1.0.0` ([source `a13d35a`](https://github.com/earendil-works/pi/commit/a13d35a742c6ef8462812a28fbe1d8c8b7431c32)), TypeBox `1.3.27` and Node `24.21.0` evidence does not certify newer hosts. Qualification does not install, activate, or restart the live runtime.
|
|
17
|
-
|
|
18
|
-
## Install
|
|
19
|
-
|
|
20
|
-
Posthorse `0.9.0` requires Pi `>=1.0.0`. Update the host before installing this release; do not activate it on an older live fork.
|
|
11
|
+
Update an older Pi installation before you install Posthorse. Run these commands:
|
|
21
12
|
|
|
22
13
|
```bash
|
|
23
|
-
pi install npm:pi-posthorse
|
|
24
|
-
pi
|
|
25
|
-
pi -e npm:pi-posthorse@0.9.0 # try it for one run without installing
|
|
14
|
+
pi install npm:pi-posthorse
|
|
15
|
+
pi
|
|
26
16
|
```
|
|
27
17
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
Update an unpinned install with `pi update npm:pi-posthorse` or `pi update --extensions`. Move a pinned install with `pi install npm:pi-posthorse@<new version>` or `pi install git:github.com/fitchmultz/pi-posthorse@<new tag>`. Uninstall with `pi remove` followed by the source you installed. Removing the package leaves `.pi/notes` and Pi's session history in place.
|
|
31
|
-
|
|
32
|
-
Restart Pi after installing or updating Posthorse to load the new extension code.
|
|
33
|
-
|
|
34
|
-
Keep exactly one copy loaded. `pi list` shows every package source; if a second source or a local checkout is listed, `pi remove` it, otherwise two copies register the same tools and compete for the same rollover hook.
|
|
35
|
-
|
|
36
|
-
## Official Pi and the fork
|
|
37
|
-
|
|
38
|
-
Both hosts get the same tools, guidance, reminders, notes, history, and recovery records. Rollover is a public compaction entry retaining no earlier conversation, with the handoff as its summary; no summary model is called. `new_context` commits at turn end only after Posthorse's own callback succeeds and every sibling tool succeeds. A foreign tool with the same name cannot authorize a rollover. Automatic rollover handles `session_before_compact`; manual `/compact` stays Pi's own summarization.
|
|
39
|
-
|
|
40
|
-
- **Automatic eligibility on both hosts:** Pi's compaction preparation must succeed before the automatic hook runs. Posthorse's custom compaction result bypasses the default summarizer and its credential lookup. In fresh compaction windows without a subsequent projected turn-start input, Posthorse owns threshold rollover at completed turn end when known usage reaches the enabled, supported threshold; it preserves normal continuation and never requests another response after a final answer. Other threshold spans and overflow/truncated-response recovery stay Pi-owned. Tiny overflow/truncated responses can still miss native recovery, and oversized initial input can reach the provider before recovery.
|
|
41
|
-
- **Policy:** guidance, reminders and budget reports use Pi's live effective settings, including runtime and per-model compaction overrides, defaults and validation; the turn-end threshold fallback uses this same policy reader, while the native automatic hook uses the live settings in its event. Manual settings-file edits take effect when Pi refreshes its live settings, just like Pi's own compaction, not on Posthorse's next evaluation. SDK hosts can inject policy with `createPosthorse(getPolicy)`. Handoffs appear as compaction-summary messages. Background work owned outside the foreground batch is not serialized or stopped by Posthorse.
|
|
42
|
-
|
|
43
|
-
Legacy fork `context_window` entries remain readable for history recovery. New boundaries do not use the retired native-window/async ABI. Opening legacy fork journals in official Pi can reintroduce earlier context because official Pi does not interpret those old markers; do not assume cross-host session migration.
|
|
44
|
-
|
|
45
|
-
## How it works
|
|
18
|
+
Restart Pi after installation or an update. To try Posthorse for one session, run `pi -e npm:pi-posthorse`.
|
|
46
19
|
|
|
47
|
-
|
|
48
|
-
2. **One best-effort checkpoint.** While Pi compaction is enabled, one reminder may arrive at the end of a working turn inside the checkpoint band before Pi's rollover line, never on a turn that ends the run. It cannot wake a finished agent or replace its final answer. The reminder asks the model to update its current-state note with available file-editing tools and call `new_context`. Already-current notes need no rewrite; `notes write` remains the fallback for creation, substantial restructuring, or unavailable editing tools. A large turn, overflow, restart, or smaller model can reach rollover without it. Reminders are fingerprinted by window, context size, and reserve, so switching to a different context size gets a fresh reminder. Stale reminders and reminders left after disabling compaction are filtered from model input while remaining in history.
|
|
49
|
-
3. **`get_context_remaining`.** Reports the best available native estimate of tokens until Pi's automatic rollover line and until the configured context limit. That configured limit is not a measured provider rejection boundary. Pi's value is an estimate until the active model reports usage. Check before claiming a remaining-token count or changing plans because of context limits. The checkpoint reminder marks the deadline, so routine budget checks are unnecessary.
|
|
50
|
-
4. **`new_context`.** Requests an atomic retain-none rollover once the current foreground tool batch succeeds. An optional handoff becomes the first state of the fresh window. Failed sibling writes, aborted turns, or a handoff that no longer fits after queued input prevent commitment. The request card is not proof that the boundary committed. Own successful callback IDs and admitted handoffs are consumed once at settlement; transcript names alone cannot request a reset.
|
|
51
|
-
5. **Automatic rollover without summaries.** When Pi reaches the public automatic hook with a supported budget, Posthorse builds a bounded recovery record and retains only an invisible boundary sentinel. Cancellation or recovery construction failure leaves active context intact rather than silently invoking a summarizer. Unsupported small-model budgets retain Pi's native fallback. Manual `/compact` is unchanged. Native eligibility limits are listed above.
|
|
52
|
-
6. **Bounded recovery record.** The automatic handoff quotes the window's first owner request, its latest owner input (a direct user input or `ask_question` outcome), and its latest input of any kind, including visible coordination messages. Across consecutive automatic boundaries, a window with no owner inputs carries the same first/latest owner anchors from the nearest earlier owner window, labeled as earlier, possibly already handled, and not current progress. The walk stops at that window or an explicit/native compaction checkpoint; context edits still omit or replace carried inputs, and `ask_question` answers keep the same provenance checks. The newest non-automatic checkpoint reached replaces nested automatic text, with bounded history pointers to skipped automatic record ids. Carried anchors share the existing payload budget; earlier automatic tool evidence is never nested. It adds tool result evidence: tool names and namespaces, call ids, requested and admitted execution arguments when available, result excerpts of up to 1,500 characters, and entry ids to recover the rest. Every other current-window input gets one index line with its label, time, and entry id; owner inputs keep a one-line preview there. Only a successful result positively linked to a projected unnamespaced `ask_question` call **and** an effective registered unnamespaced `ask_question` definition is an owner answer. Providers can omit call namespaces, so call metadata alone is insufficient. Namespaced, unregistered or unresolved-provenance results remain tool evidence, never owner approval. Legacy async results still visible in a projected journal remain eligible even across later complete assistant responses; this is history compatibility, not a new native-async execution path. A result Pi still projects is also eligible when its call was edited out or predates the window, without attributing it to another call. These results may already have been received or handled; they are not proof of current progress. Synchronous results with projected call provenance are kept only from the trailing batch without a later complete response. Pi context edits also apply to these inputs and results: omitted entries stay out, and replacements supersede their original content. A clearly labeled, possibly stale older checkpoint comes last. Older assistant prose is not treated as current state. Newly submitted input stays separate and is saved after the boundary, not copied into the handoff.
|
|
53
|
-
7. **`notes` and `history`.** Notes are shared across linked worktrees, at the main checkout for conventional Git layouts or inside the common Git directory when metadata is stored separately. History searches normalized transcript text, skips entries whose full text the model already sees in its active context (and says how many), and returns stored images for a requested entry. Edited-away originals and checkpoint reminders stay searchable. Tool calls and results retain names, namespaces, and call ids; admitted execution arguments are labeled separately from requested arguments when present. When a handoff carries edited content, its entry ID points to the replacement; `history read` of that ID returns the replacement text or images. Reading the original entry ID still returns the raw journal content.
|
|
20
|
+
Give Pi your task. For example:
|
|
54
21
|
|
|
55
|
-
|
|
22
|
+
> Save task state in a note. Start a fresh context window with the next steps.
|
|
56
23
|
|
|
57
|
-
|
|
24
|
+
## How rollover works
|
|
58
25
|
|
|
59
|
-
|
|
26
|
+
Posthorse can remind the model to save task state before rollover. Large turns can reach rollover before this reminder. The reminder does not wake an agent after its final answer.
|
|
60
27
|
|
|
61
|
-
Automatic
|
|
28
|
+
Automatic rollover creates a recovery record from session inputs and tool results. The model must restore notes and check live state before it continues. Manual `/compact` uses Pi's normal summarization.
|
|
62
29
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
Posthorse follows Pi's effective `compaction` settings, including Pi's decision about whether project settings are trusted. Disabling `compaction.enabled` disables reminders and automatic rollover; `new_context` stays available.
|
|
66
|
-
|
|
67
|
-
The model's context window minus `compaction.reserveTokens` must leave at least 10,000 usable tokens. Below that (for example an 8K or 16K model with the default 16,384 reserve) Posthorse reports an unsupported configuration in the guidance and in `get_context_remaining`, turns automatic behavior off for that model, and leaves Pi's own compaction in place. Lower the reserve or use a larger model. The checkpoint reminder band is the last 10% of usable context, capped at 32,000 tokens, so a large reserve cannot trigger a reminder immediately in a fresh window.
|
|
68
|
-
|
|
69
|
-
Explicit and automatic handoffs are capped at 20,000 characters and half the native context limit's fresh operational capacity after prompt/tool overhead and any pending input, whichever is smaller. Oversized explicit handoffs are rejected with an instruction to save fuller state in notes; automatic rollover cancels visibly when a supported configuration has no room for a safe recovery record. Notes and history pages are capped at 40,000 characters and shrink to the remaining budget; when usage is not known yet, they also leave half the fresh operational capacity free. Payload budgets reserve Posthorse's request-local guidance too, because native estimates can omit it.
|
|
70
|
-
|
|
71
|
-
Only one automatic compaction or rollover policy extension should be enabled at a time. Pi keeps the last non-cancel result from multiple handlers of the same hook, so load order would otherwise decide which policy wins.
|
|
72
|
-
|
|
73
|
-
The fresh-window threshold fallback builds its recovery record from the boundary preview when its `turn_end` handler runs. Earlier turn-start drafts keep compaction Pi-owned; earlier context-edit drafts defer the fallback until usage can be re-evaluated after commit. Unlike the native automatic hook (which runs after all turn-end handlers), the eager fallback cannot observe later handlers' edits. Later projection-editing handlers must explicitly replace/remove its drafted compaction if they change its source evidence; ordering an editor before Posthorse lets its drafts defer rollover safely. A host post-boundary hook would be needed to include arbitrary later same-phase edits automatically.
|
|
30
|
+
See [rollover and recovery details](docs/reference.md#how-it-works).
|
|
74
31
|
|
|
75
32
|
## Tools
|
|
76
33
|
|
|
77
|
-
|
|
78
|
-
- `get_context_remaining()`
|
|
79
|
-
- `notes({ op, ... })`: `list`, `read`, and `search` are paged; continue with the returned character `offset`. `list` shows the absolute shared notes directory, then relative note names newest first with size and modification time (UTC), and takes an optional folder `path`; `read` of a folder returns its listing. File reads show the absolute file path before the content. Offsets refer to note content or listing rows, excluding path headers. `search` covers text notes, skipping binary files (a NUL byte in the first 8,000 bytes, as Git decides), and its excerpts center on the match. It also skips files over 1 MiB and says how many it skipped. Search stops reading once the requested page is decided, so a broad query reports that more matches follow instead of a total; its pages end before offset 1,000,000. `write` replaces content (empty content clears); `append` adds one newline-terminated record.
|
|
80
|
-
- `history({ op, ... })`: `search` skips entries whose full text is already in the active context, continues with the returned `cursor` and the same `query` and `all` scope, and treats `limit` as the maximum results per page. `read` pages text and stored images; continue with the returned character `offset` and `imageOffset`. An image-only continuation can use an `offset` equal to the text length. If a page needs fresh context, retry with both offsets unchanged. Results keep native entry and window ids.
|
|
81
|
-
|
|
82
|
-
In the TUI, tools use Pi's native expandable cards. Collapsed cards show the operation and target, a short content preview, and counts or page ranges with the next offset when more remains. Expand with Pi's tool-output shortcut (`Ctrl+O` by default), or click the card's header or body in fullscreen mode. Expanded cards show the complete returned page and its metadata, not content the tool has not fetched yet. Writes and context requests also show the submitted content or handoff.
|
|
34
|
+
These tools are available to the model:
|
|
83
35
|
|
|
84
|
-
|
|
36
|
+
| Tool | Use |
|
|
37
|
+
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
38
|
+
| `new_context({ handoff? })` | Request a fresh window after every foreground tool succeeds. Aborted turns and failed tools prevent the reset. |
|
|
39
|
+
| `get_context_remaining()` | Estimate tokens left until rollover and the configured context limit. Usage can be unknown before a model response. |
|
|
40
|
+
| `notes({ op, ... })` | List, read, search, write, or append task notes. |
|
|
41
|
+
| `history({ op, ... })` | Search earlier conversation and read entries, including images. `all: true` includes this project's other sessions and nested subagent runs. |
|
|
85
42
|
|
|
86
|
-
|
|
43
|
+
Expand a tool card with Ctrl+O. In fullscreen mode, click a tool card to expand it. A `new_context` card shows a request. Pi's `Compacted from … tokens` card confirms the rollover.
|
|
87
44
|
|
|
88
|
-
|
|
45
|
+
See [tool examples and paging details](docs/reference.md#tools).
|
|
89
46
|
|
|
90
|
-
|
|
47
|
+
## Settings and limits
|
|
91
48
|
|
|
92
|
-
|
|
49
|
+
Posthorse uses Pi's effective `compaction` settings. It has no separate settings.
|
|
93
50
|
|
|
94
|
-
|
|
51
|
+
| Pi setting | Effect |
|
|
52
|
+
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
53
|
+
| `compaction.enabled` | Set to `false` to disable reminders and automatic rollover. `new_context` remains available. |
|
|
54
|
+
| `compaction.reserveTokens` | Automatic Posthorse rollover needs at least 10,000 tokens after this reserve. Smaller budgets use Pi's own compaction. Lower the reserve or choose a larger model. |
|
|
95
55
|
|
|
96
|
-
|
|
56
|
+
Two Posthorse copies can register the same tools and compete for rollover. Keep one copy loaded. Check installed sources with `pi list`.
|
|
97
57
|
|
|
98
|
-
|
|
58
|
+
Multiple compaction-policy extensions can make rollover depend on load order. Enable only one.
|
|
99
59
|
|
|
100
|
-
|
|
101
|
-
- Posthorse makes no network requests.
|
|
102
|
-
- Notes are plaintext files under `.pi/notes`. They survive package removal and may be committed unless ignored.
|
|
103
|
-
- `history` with `all: true` scans project-matching JSONL files and their nested subagent sessions in the active Pi session directory, not unrelated projects that share that directory. It runs ripgrep locally over that directory when available.
|
|
104
|
-
- History can return user text, assistant text and thinking, assistant failure status and error messages, tool arguments and results, handoffs, custom messages, and images. Direct shell entries Pi marked `excludeFromContext` come back as a placeholder only.
|
|
105
|
-
- Returned history content enters the currently selected model and provider context.
|
|
106
|
-
- Removing Posthorse does not remove notes or Pi session history.
|
|
60
|
+
Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Background work outside the foreground tool batch continues across rollover.
|
|
107
61
|
|
|
108
|
-
|
|
62
|
+
See [size limits and extension order](docs/reference.md#settings).
|
|
63
|
+
Check the [legacy-session details](docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
|
|
109
64
|
|
|
110
|
-
##
|
|
65
|
+
## Update or remove
|
|
111
66
|
|
|
112
|
-
|
|
67
|
+
```bash
|
|
68
|
+
pi update npm:pi-posthorse
|
|
69
|
+
pi remove npm:pi-posthorse
|
|
70
|
+
```
|
|
113
71
|
|
|
114
|
-
|
|
72
|
+
Removal leaves notes and Pi session history in place. See the [installation reference](docs/reference.md#install) for Git sources.
|
|
115
73
|
|
|
116
|
-
##
|
|
74
|
+
## Data and privacy
|
|
117
75
|
|
|
118
|
-
|
|
119
|
-
npm ci --ignore-scripts
|
|
120
|
-
npm run verify # strict policy, lint, formatting, typecheck, tests, package build
|
|
121
|
-
npm run check:compat # typecheck, unit tests, and the real-SDK suite for the installed host
|
|
122
|
-
```
|
|
76
|
+
Posthorse makes no network requests. Notes are plaintext files under `.pi/notes/`; linked Git worktrees share them. Add `.pi/notes/` to `.gitignore` if the notes must stay private.
|
|
123
77
|
|
|
124
|
-
|
|
125
|
-
exception governance, editor setup, and the enforced acceptance workflow.
|
|
78
|
+
History can include user messages, assistant thinking, tool arguments, errors, and images. Content you retrieve enters the context of your selected model and provider.
|
|
126
79
|
|
|
127
|
-
|
|
80
|
+
See [storage and privacy details](docs/reference.md#data-and-privacy).
|
|
81
|
+
For cost diagnosis, see the [cached-token investigation](ARTIFACTS/investigation-report.md).
|
|
82
|
+
It does not establish the cause of a production bill or a measured cost improvement.
|
|
128
83
|
|
|
129
|
-
Development
|
|
84
|
+
## Development
|
|
130
85
|
|
|
131
|
-
|
|
86
|
+
See [development and host qualification](docs/development.md) for setup, tests, and releases.
|
|
87
|
+
See [code quality](docs/quality.md) for the required checks.
|
|
132
88
|
|
|
133
|
-
|
|
89
|
+
## License
|
|
134
90
|
|
|
135
|
-
|
|
91
|
+
[MIT](LICENSE).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-posthorse",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.2",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Posthorse: fresh context, same journey. No-summary context rollover for Pi, with sparse budget reminders, handoffs, notes, and history.",
|
|
6
6
|
"keywords": [
|
|
@@ -32,7 +32,7 @@
|
|
|
32
32
|
},
|
|
33
33
|
"scripts": {
|
|
34
34
|
"check": "tsc",
|
|
35
|
-
"quality:prepare": "node scripts/
|
|
35
|
+
"quality:prepare": "node scripts/quality-prepare.ts",
|
|
36
36
|
"quality:policy": "node scripts/quality-policy.ts",
|
|
37
37
|
"quality:probes": "npm run quality:prepare && node --test test/quality*.test.ts",
|
|
38
38
|
"lint": "npm run quality:prepare && npm run quality:policy && oxlint --deny-warnings .",
|
|
@@ -42,9 +42,9 @@
|
|
|
42
42
|
"format:check": "oxfmt --check .",
|
|
43
43
|
"build": "npm pack --dry-run",
|
|
44
44
|
"verify": "npm run lint && npm run format:check && npm run check:compat && npm run build",
|
|
45
|
-
"check:compat": "npm run check && npm test
|
|
45
|
+
"check:compat": "npm run check && npm test",
|
|
46
46
|
"test:native": "node --import ./test/pi-loader.ts --test test/host.test.mjs",
|
|
47
|
-
"test": "npm run quality:prepare && node --import ./test/pi-loader.ts --test test/*.test.ts",
|
|
47
|
+
"test": "npm run quality:prepare && node --import ./test/pi-loader.ts --test test/*.test.ts test/host.test.mjs",
|
|
48
48
|
"prepublishOnly": "npm run verify"
|
|
49
49
|
},
|
|
50
50
|
"peerDependencies": {
|
package/src/files.ts
CHANGED
|
@@ -112,25 +112,31 @@ export async function publishFile(
|
|
|
112
112
|
throw error;
|
|
113
113
|
}
|
|
114
114
|
}
|
|
115
|
-
/**
|
|
116
|
-
export async function* lfLines(
|
|
117
|
-
|
|
118
|
-
|
|
115
|
+
/** Raw LF-delimited lines; UTF-8 never encodes LF inside another character. */
|
|
116
|
+
export async function* lfLines(
|
|
117
|
+
file: string,
|
|
118
|
+
signal?: AbortSignal,
|
|
119
|
+
highWaterMark = 64 * 1024,
|
|
120
|
+
): AsyncGenerator<Buffer> {
|
|
121
|
+
const stream = createReadStream(file, { signal, highWaterMark });
|
|
122
|
+
let pending: Buffer[] = [];
|
|
119
123
|
for await (const chunk of stream) {
|
|
120
|
-
if (
|
|
121
|
-
throw new Error("Expected a
|
|
124
|
+
if (!Buffer.isBuffer(chunk)) {
|
|
125
|
+
throw new Error("Expected a byte stream.");
|
|
126
|
+
}
|
|
127
|
+
let start = 0;
|
|
128
|
+
for (let end = chunk.indexOf(10); end !== -1; end = chunk.indexOf(10, start)) {
|
|
129
|
+
const line = chunk.subarray(start, end);
|
|
130
|
+
yield pending.length === 0 ? line : Buffer.concat([...pending, line]);
|
|
131
|
+
pending = [];
|
|
132
|
+
start = end + 1;
|
|
122
133
|
}
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
if (lines.length > 1) {
|
|
126
|
-
yield pending.join("");
|
|
127
|
-
yield* lines.slice(1, -1);
|
|
128
|
-
pending = [lines.at(-1) ?? ""];
|
|
134
|
+
if (start < chunk.length) {
|
|
135
|
+
pending.push(chunk.subarray(start));
|
|
129
136
|
}
|
|
130
137
|
}
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
yield last;
|
|
138
|
+
if (pending.length > 0) {
|
|
139
|
+
yield Buffer.concat(pending);
|
|
134
140
|
}
|
|
135
141
|
}
|
|
136
142
|
/** Git's heuristic: a NUL in the first 8,000 bytes marks binary content. */
|
package/src/history-search.ts
CHANGED
|
@@ -10,8 +10,9 @@ import {
|
|
|
10
10
|
import { historyHit, seenText, reverseWindowEntries } from "./history-entries.ts";
|
|
11
11
|
import { shortKey } from "./message-text.ts";
|
|
12
12
|
import {
|
|
13
|
-
|
|
13
|
+
needleLines,
|
|
14
14
|
prefilterNeedles,
|
|
15
|
+
type RawLine,
|
|
15
16
|
scopedSessionFiles,
|
|
16
17
|
sessionWindowEntries,
|
|
17
18
|
} from "./session-files.ts";
|
|
@@ -192,13 +193,14 @@ async function scanArchive(
|
|
|
192
193
|
readonly source: string;
|
|
193
194
|
readonly query: string;
|
|
194
195
|
readonly signal?: AbortSignal;
|
|
196
|
+
readonly wanted?: (bytes: RawLine) => boolean;
|
|
195
197
|
readonly inContext: (entry: EntryLike) => boolean;
|
|
196
198
|
},
|
|
197
199
|
accumulator: SearchProgress,
|
|
198
200
|
): Promise<{ readonly groups: readonly (readonly HistoryHit[])[]; readonly skipped: number }> {
|
|
199
201
|
const collector = archiveCollector(accumulator);
|
|
200
202
|
let skipped = 0;
|
|
201
|
-
for await (const item of sessionWindowEntries(file, options.signal)) {
|
|
203
|
+
for await (const item of sessionWindowEntries(file, options.signal, options.wanted)) {
|
|
202
204
|
const hit = historyHit(item, options.query, options.source);
|
|
203
205
|
if (hit === undefined) {
|
|
204
206
|
continue;
|
|
@@ -221,10 +223,7 @@ async function archivedMatches(
|
|
|
221
223
|
accumulator: SearchProgress,
|
|
222
224
|
): Promise<number> {
|
|
223
225
|
const needles = prefilterNeedles(search.query);
|
|
224
|
-
const
|
|
225
|
-
needles === undefined
|
|
226
|
-
? undefined
|
|
227
|
-
: await filesContaining(source.dir, needles, search.signal);
|
|
226
|
+
const wanted = needles === undefined ? undefined : needleLines(needles);
|
|
228
227
|
const current =
|
|
229
228
|
source.currentFile === undefined || source.currentFile === ""
|
|
230
229
|
? undefined
|
|
@@ -235,7 +234,7 @@ async function archivedMatches(
|
|
|
235
234
|
cwd: source.cwd,
|
|
236
235
|
currentFile: source.currentFile,
|
|
237
236
|
signal: search.signal,
|
|
238
|
-
|
|
237
|
+
needles,
|
|
239
238
|
})) {
|
|
240
239
|
const inCurrent = resolve(file) === current;
|
|
241
240
|
const recent = await scanArchive(
|
|
@@ -244,6 +243,7 @@ async function archivedMatches(
|
|
|
244
243
|
source: relative(source.dir, file),
|
|
245
244
|
query: search.query,
|
|
246
245
|
signal: search.signal,
|
|
246
|
+
wanted,
|
|
247
247
|
inContext: (entry) => inCurrent && search.inContext(entry),
|
|
248
248
|
},
|
|
249
249
|
accumulator,
|
package/src/history.ts
CHANGED
|
@@ -7,7 +7,7 @@ import { toWindowedEntry, windowProjection } from "./history-entries.ts";
|
|
|
7
7
|
import { historyReadPage, searchPage } from "./history-pages.ts";
|
|
8
8
|
import { historyMatches, type HistorySource } from "./history-search.ts";
|
|
9
9
|
import type { PageAccess } from "./paging.ts";
|
|
10
|
-
import {
|
|
10
|
+
import { scopedSessionFiles, sessionWindowEntries } from "./session-files.ts";
|
|
11
11
|
|
|
12
12
|
type LocatedEntry = { readonly item: WindowedEntry; readonly source: string };
|
|
13
13
|
function entryReference(id: string): { readonly entryId: string; readonly fileKey?: string } {
|
|
@@ -38,19 +38,20 @@ async function findEntry(
|
|
|
38
38
|
if (current !== undefined) {
|
|
39
39
|
return { item: current, source: "" };
|
|
40
40
|
}
|
|
41
|
-
const candidates =
|
|
42
|
-
fileKey === undefined && /^[\w-]+$/.test(entryId)
|
|
43
|
-
? await filesContaining(source.dir, [`"id":"${entryId}"`], signal)
|
|
44
|
-
: undefined;
|
|
45
41
|
for await (const file of scopedSessionFiles({
|
|
46
42
|
dir: source.dir,
|
|
47
43
|
cwd: source.cwd,
|
|
48
44
|
currentFile: source.currentFile,
|
|
49
45
|
signal,
|
|
50
46
|
fileKey,
|
|
51
|
-
|
|
47
|
+
needles:
|
|
48
|
+
fileKey === undefined && /^[\w-]+$/.test(entryId) ? [`"id":"${entryId}"`] : undefined,
|
|
52
49
|
})) {
|
|
53
|
-
for await (const item of sessionWindowEntries(
|
|
50
|
+
for await (const item of sessionWindowEntries(
|
|
51
|
+
file,
|
|
52
|
+
signal,
|
|
53
|
+
(_bytes, entry) => entry.id === entryId,
|
|
54
|
+
)) {
|
|
54
55
|
return { item, source: relative(source.dir, file) };
|
|
55
56
|
}
|
|
56
57
|
}
|
package/src/rollover.ts
CHANGED
|
@@ -41,6 +41,10 @@ function registerGuidance(pi: ExtensionAPI, policy: SnapshotReader): void {
|
|
|
41
41
|
pi.on("before_agent_start", (event) => {
|
|
42
42
|
promptOptions = event.systemPromptOptions;
|
|
43
43
|
});
|
|
44
|
+
pi.on("agent_settled", () => {
|
|
45
|
+
// Older hosts skip the start hook for custom runs; do not reuse a settled run's options.
|
|
46
|
+
promptOptions = undefined;
|
|
47
|
+
});
|
|
44
48
|
pi.on("context_with_system", (event, ctx) => {
|
|
45
49
|
const guidance = buildGuidance(policy(ctx));
|
|
46
50
|
const system =
|
package/src/session-files.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { execFile } from "node:child_process";
|
|
2
|
-
import { existsSync
|
|
2
|
+
import { existsSync } from "node:fs";
|
|
3
|
+
import { readdir, stat } from "node:fs/promises";
|
|
3
4
|
import { join, relative, resolve, sep } from "node:path";
|
|
4
5
|
|
|
5
6
|
import { getAgentDir } from "@earendil-works/pi-coding-agent";
|
|
@@ -14,13 +15,21 @@ import { lfLines } from "./files.ts";
|
|
|
14
15
|
import { toWindowedEntry, windowProjection } from "./history-entries.ts";
|
|
15
16
|
import { shortKey } from "./message-text.ts";
|
|
16
17
|
|
|
18
|
+
/** Newest archive bytes searched before the rest; common queries finish within them. */
|
|
19
|
+
const FIRST_SEARCH_BYTES = 256 * 1024 * 1024;
|
|
20
|
+
/** File-list characters per ripgrep run, within Windows' 32,767 and POSIX ARG_MAX limits. */
|
|
21
|
+
const ARGUMENT_CHARS = process.platform === "win32" ? 30_000 : 256 * 1024;
|
|
22
|
+
/** Pi writes these keys first, so most lines yield window metadata without JSON.parse. */
|
|
23
|
+
const ENTRY_HEAD = /^\{"type":"([^"\\]*)","id":"([^"\\]*)","parentId":(null|"[^"\\]*")[,}]/u;
|
|
24
|
+
|
|
17
25
|
function execFileText(
|
|
18
26
|
file: string,
|
|
19
27
|
args: readonly string[],
|
|
28
|
+
cwd: string,
|
|
20
29
|
signal?: AbortSignal,
|
|
21
30
|
): Promise<string> {
|
|
22
31
|
return new Promise((resolveOutput, reject) => {
|
|
23
|
-
execFile(file, args, { signal, maxBuffer: 64 * 1024 * 1024 }, (error, stdout) => {
|
|
32
|
+
execFile(file, args, { cwd, signal, maxBuffer: 64 * 1024 * 1024 }, (error, stdout) => {
|
|
24
33
|
if (error !== null) {
|
|
25
34
|
const failure: Error = error;
|
|
26
35
|
reject(failure);
|
|
@@ -30,65 +39,127 @@ function execFileText(
|
|
|
30
39
|
});
|
|
31
40
|
});
|
|
32
41
|
}
|
|
33
|
-
/**
|
|
42
|
+
/** Label text normalization adds, and raw JSONL that every entry showing it contains. */
|
|
43
|
+
const LABEL_SOURCES: readonly (readonly [label: string, needles: readonly string[]])[] = [
|
|
44
|
+
["call id", ['"type":"toolCall"', '"toolCallId"']],
|
|
45
|
+
["execution arguments", ['"executionArguments"']],
|
|
46
|
+
["no handoff", ['"type":"context_window"']],
|
|
47
|
+
["excluded from model context by pi", ['"excludeFromContext":true']],
|
|
48
|
+
["images", ['"type":"image"']],
|
|
49
|
+
["unknown type", ['"type":"image"']],
|
|
50
|
+
["unknown", ['"type":"custom_message"']],
|
|
51
|
+
];
|
|
52
|
+
/**
|
|
53
|
+
* Raw JSONL needles, one of which every line rendering the lowercase query contains; undefined
|
|
54
|
+
* means every line must be scanned.
|
|
55
|
+
*/
|
|
34
56
|
export function prefilterNeedles(query: string): string[] | undefined {
|
|
35
|
-
|
|
57
|
+
// A match contains every fragment between characters that JSON escapes or labels may add.
|
|
58
|
+
const [fragment] = query
|
|
59
|
+
.split(/["\\[\](){}:$\p{Cc}\p{Cs}]/u)
|
|
60
|
+
.map((part) => part.trim())
|
|
61
|
+
.sort((a, b) => b.length - a.length);
|
|
62
|
+
if (fragment === "") {
|
|
36
63
|
return undefined;
|
|
37
64
|
}
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
)
|
|
42
|
-
|
|
43
|
-
|
|
65
|
+
const image = fragment.includes(",") || /^\d+ ?i?m?a?g?e?s?$/.test(fragment);
|
|
66
|
+
return [
|
|
67
|
+
fragment,
|
|
68
|
+
...LABEL_SOURCES.filter(([label]) => label.includes(fragment)).flatMap(
|
|
69
|
+
([, needles]) => needles,
|
|
70
|
+
),
|
|
71
|
+
...(image ? ['"type":"image"'] : []),
|
|
72
|
+
// İ lowercases to i + U+0307, which case-insensitive raw matching cannot see.
|
|
73
|
+
...(/i|\u0307/u.test(fragment) ? ["\u0130"] : []),
|
|
74
|
+
];
|
|
75
|
+
}
|
|
76
|
+
/** UTF-8 of the only non-ASCII characters that case-fold to ASCII letters, read as Latin-1. */
|
|
77
|
+
const ASCII_FOLDS: Readonly<Partial<Record<string, string>>> = {
|
|
78
|
+
k: "\u00e2\u0084\u00aa",
|
|
79
|
+
s: "\u00c5\u00bf",
|
|
80
|
+
};
|
|
81
|
+
const DOTTED_I = Buffer.from("\u0130");
|
|
82
|
+
/** The read capabilities line filters need from a raw journal line. */
|
|
83
|
+
export type RawLine = Readonly<Pick<Buffer, "includes" | "toString">>;
|
|
84
|
+
/** Whether a raw line contains one of prefilterNeedles' needles, ignoring case like its text would. */
|
|
85
|
+
export function needleLines(needles: readonly string[]): (bytes: RawLine) => boolean {
|
|
86
|
+
const others = needles.filter((needle) => needle !== "\u0130");
|
|
87
|
+
const dotted = others.length < needles.length;
|
|
88
|
+
if (others.every((needle) => /^[\0-\x7f]*$/u.test(needle))) {
|
|
89
|
+
// ASCII text matches its Latin-1 byte view, so most lines are never decoded as UTF-8.
|
|
90
|
+
const latin1 = others.map((needle) =>
|
|
91
|
+
needle
|
|
92
|
+
.split("")
|
|
93
|
+
.map((char) => {
|
|
94
|
+
const fold = ASCII_FOLDS[char.toLowerCase()];
|
|
95
|
+
return fold === undefined ? RegExp.escape(char) : `(?:${char}|${fold})`;
|
|
96
|
+
})
|
|
97
|
+
.join(""),
|
|
98
|
+
);
|
|
99
|
+
const pattern = new RegExp(latin1.join("|"), "i");
|
|
100
|
+
return (bytes) =>
|
|
101
|
+
(dotted && bytes.includes(DOTTED_I)) || pattern.test(bytes.toString("latin1"));
|
|
44
102
|
}
|
|
45
|
-
const
|
|
46
|
-
|
|
47
|
-
/^\d+ ?i?m?a?g?e?s?$/.test(query) ||
|
|
48
|
-
["images", "unknown type"].some((phrase) => phrase.includes(query));
|
|
49
|
-
return image ? [query, '"type":"image"'] : [query];
|
|
103
|
+
const pattern = new RegExp(needles.map((needle) => RegExp.escape(needle)).join("|"), "iu");
|
|
104
|
+
return (bytes) => pattern.test(bytes.toString("utf8"));
|
|
50
105
|
}
|
|
51
|
-
|
|
106
|
+
/** File lists that each fit one ripgrep command line. */
|
|
107
|
+
function commandGroups(paths: readonly string[]): string[][] {
|
|
108
|
+
const groups: string[][] = [];
|
|
109
|
+
let group: string[] = [];
|
|
110
|
+
let chars = 0;
|
|
111
|
+
for (const path of paths) {
|
|
112
|
+
if (group.length > 0 && chars + path.length > ARGUMENT_CHARS) {
|
|
113
|
+
groups.push(group);
|
|
114
|
+
group = [];
|
|
115
|
+
chars = 0;
|
|
116
|
+
}
|
|
117
|
+
group.push(path);
|
|
118
|
+
chars += path.length + 1;
|
|
119
|
+
}
|
|
120
|
+
return group.length > 0 ? [...groups, group] : groups;
|
|
121
|
+
}
|
|
122
|
+
/** Files containing a needle, per ripgrep; files of a failed run stay candidates. */
|
|
123
|
+
function ripgrepFilter(
|
|
52
124
|
dir: string,
|
|
53
125
|
needles: readonly string[],
|
|
54
126
|
signal?: AbortSignal,
|
|
55
|
-
): Promise<
|
|
56
|
-
|
|
127
|
+
): (files: readonly string[]) => Promise<string[]> {
|
|
128
|
+
let binaries = [join(getAgentDir(), "bin", "rg"), "rg"];
|
|
129
|
+
const flags = [
|
|
57
130
|
"--files-with-matches",
|
|
131
|
+
"--null",
|
|
58
132
|
"--fixed-strings",
|
|
59
133
|
"--ignore-case",
|
|
60
|
-
"--no-ignore",
|
|
61
|
-
"--hidden",
|
|
62
134
|
"--no-messages",
|
|
63
|
-
"--glob",
|
|
64
|
-
"*.jsonl",
|
|
65
135
|
...needles.flatMap((needle) => ["--regexp", needle]),
|
|
66
|
-
|
|
136
|
+
"--",
|
|
67
137
|
];
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
138
|
+
const matching = async (paths: readonly string[]): Promise<readonly string[]> => {
|
|
139
|
+
for (const binary of binaries) {
|
|
140
|
+
try {
|
|
141
|
+
// Try the host-managed binary first; PATH is a fallback only when it is unavailable.
|
|
142
|
+
// oxlint-disable-next-line no-await-in-loop
|
|
143
|
+
return (await execFileText(binary, [...flags, ...paths], dir, signal)).split("\0");
|
|
144
|
+
} catch (error) {
|
|
145
|
+
const code = isRecord(error) ? error.code : undefined;
|
|
146
|
+
if (code === 1) {
|
|
147
|
+
return [];
|
|
148
|
+
}
|
|
149
|
+
if (code !== "ENOENT") {
|
|
150
|
+
signal?.throwIfAborted();
|
|
151
|
+
return paths;
|
|
152
|
+
}
|
|
153
|
+
binaries = binaries.filter((candidate) => candidate !== binary);
|
|
83
154
|
}
|
|
84
|
-
if (code === "ENOENT") {
|
|
85
|
-
continue;
|
|
86
|
-
}
|
|
87
|
-
signal?.throwIfAborted();
|
|
88
|
-
return undefined;
|
|
89
155
|
}
|
|
90
|
-
|
|
91
|
-
|
|
156
|
+
return paths;
|
|
157
|
+
};
|
|
158
|
+
return async (files) => {
|
|
159
|
+
const paths = files.map((file) => relative(dir, file));
|
|
160
|
+
const found = new Set((await Promise.all(commandGroups(paths).map(matching))).flat());
|
|
161
|
+
return files.filter((_file, index) => found.has(paths[index]));
|
|
162
|
+
};
|
|
92
163
|
}
|
|
93
164
|
function messageFrom(value: unknown): MessageLike | undefined {
|
|
94
165
|
if (!isRecord(value)) {
|
|
@@ -138,40 +209,90 @@ function parsedEntry(line: string): EntryLike | undefined {
|
|
|
138
209
|
return undefined;
|
|
139
210
|
} // Pi skips malformed journal lines too.
|
|
140
211
|
}
|
|
212
|
+
/** Window metadata of a Pi-ordered line that cannot open a window, without parsing it. */
|
|
213
|
+
function entryHead(bytes: RawLine): EntryLike | undefined {
|
|
214
|
+
const head = ENTRY_HEAD.exec(bytes.toString("utf8", 0, 512));
|
|
215
|
+
if (head === null || head[1] === "compaction" || head[1] === "context_window") {
|
|
216
|
+
return undefined;
|
|
217
|
+
}
|
|
218
|
+
return {
|
|
219
|
+
type: head[1],
|
|
220
|
+
id: head[2],
|
|
221
|
+
parentId: head[3] === "null" ? null : head[3].slice(1, -1),
|
|
222
|
+
};
|
|
223
|
+
}
|
|
224
|
+
/** Whether a caller may need a line; other lines only advance window ancestry. */
|
|
225
|
+
type LineFilter = (bytes: RawLine, entry: EntryLike) => boolean;
|
|
141
226
|
export async function* sessionWindowEntries(
|
|
142
227
|
file: string,
|
|
143
228
|
signal?: AbortSignal,
|
|
144
|
-
|
|
229
|
+
wanted: LineFilter = () => true,
|
|
145
230
|
): AsyncGenerator<WindowedEntry> {
|
|
146
231
|
if (!existsSync(file)) {
|
|
147
232
|
return;
|
|
148
233
|
}
|
|
149
234
|
const project = windowProjection();
|
|
150
|
-
for await (const
|
|
151
|
-
const
|
|
152
|
-
if (
|
|
235
|
+
for await (const bytes of lfLines(file, signal, 1024 * 1024)) {
|
|
236
|
+
const head = entryHead(bytes);
|
|
237
|
+
if (head !== undefined && !wanted(bytes, head)) {
|
|
238
|
+
project(head);
|
|
153
239
|
continue;
|
|
154
240
|
}
|
|
155
|
-
const
|
|
156
|
-
if (
|
|
241
|
+
const entry = parsedEntry(bytes.toString("utf8"));
|
|
242
|
+
if (entry === undefined) {
|
|
157
243
|
continue;
|
|
158
244
|
}
|
|
159
|
-
const
|
|
245
|
+
const windowId = project(entry);
|
|
246
|
+
const item =
|
|
247
|
+
head !== undefined || wanted(bytes, entry)
|
|
248
|
+
? toWindowedEntry(entry, windowId)
|
|
249
|
+
: undefined;
|
|
160
250
|
if (item !== undefined) {
|
|
161
251
|
yield item;
|
|
162
252
|
}
|
|
163
253
|
}
|
|
164
254
|
}
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
255
|
+
/** Breadth-first, like readdirSync's recursive order, reading each level concurrently. */
|
|
256
|
+
async function sessionFiles(dir: string): Promise<string[]> {
|
|
257
|
+
const files: string[] = [];
|
|
258
|
+
let level = existsSync(dir) ? [dir] : [];
|
|
259
|
+
while (level.length > 0) {
|
|
260
|
+
// Each level lists the directories the previous level found.
|
|
261
|
+
// oxlint-disable-next-line no-await-in-loop
|
|
262
|
+
const listings = await Promise.all(
|
|
263
|
+
level.map((path) => readdir(path, { withFileTypes: true })),
|
|
264
|
+
);
|
|
265
|
+
level = [];
|
|
266
|
+
for (const entry of listings.flat()) {
|
|
267
|
+
const path = join(entry.parentPath, entry.name);
|
|
268
|
+
if (entry.isDirectory()) {
|
|
269
|
+
level.push(path);
|
|
270
|
+
} else if (
|
|
271
|
+
entry.isFile() &&
|
|
272
|
+
entry.name.endsWith(".jsonl") &&
|
|
273
|
+
!entry.name.includes(".intent.")
|
|
274
|
+
) {
|
|
275
|
+
files.push(path);
|
|
276
|
+
}
|
|
277
|
+
}
|
|
168
278
|
}
|
|
169
|
-
return
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
279
|
+
return files;
|
|
280
|
+
}
|
|
281
|
+
type SessionFile = { readonly file: string; readonly size: number; readonly mtime: number };
|
|
282
|
+
async function newestFirst(files: readonly string[]): Promise<SessionFile[]> {
|
|
283
|
+
const stats = await Promise.all(files.map((file) => stat(file)));
|
|
284
|
+
return files
|
|
285
|
+
.map((file, index) => ({ file, size: stats[index].size, mtime: stats[index].mtimeMs }))
|
|
286
|
+
.sort((a, b) => b.mtime - a.mtime);
|
|
287
|
+
}
|
|
288
|
+
/** The newest files up to FIRST_SEARCH_BYTES, then the rest. */
|
|
289
|
+
function searchPhases(files: readonly SessionFile[]): readonly (readonly string[])[] {
|
|
290
|
+
let count = 0;
|
|
291
|
+
for (let bytes = 0; count < files.length && bytes < FIRST_SEARCH_BYTES; count++) {
|
|
292
|
+
bytes += files[count].size;
|
|
293
|
+
}
|
|
294
|
+
const paths = files.map(({ file }) => file);
|
|
295
|
+
return [paths.slice(0, count), paths.slice(count)];
|
|
175
296
|
}
|
|
176
297
|
async function sessionHeader(
|
|
177
298
|
file: string,
|
|
@@ -179,7 +300,7 @@ async function sessionHeader(
|
|
|
179
300
|
): Promise<{ readonly cwd?: string } | undefined> {
|
|
180
301
|
for await (const line of lfLines(file, signal)) {
|
|
181
302
|
try {
|
|
182
|
-
const value: unknown = JSON.parse(line);
|
|
303
|
+
const value: unknown = JSON.parse(line.toString("utf8"));
|
|
183
304
|
if (value === null) {
|
|
184
305
|
continue;
|
|
185
306
|
}
|
|
@@ -212,7 +333,8 @@ export type SessionScope = {
|
|
|
212
333
|
readonly currentFile?: string;
|
|
213
334
|
readonly signal?: AbortSignal;
|
|
214
335
|
readonly fileKey?: string;
|
|
215
|
-
|
|
336
|
+
/** Raw-text needles; files without any of them are skipped. */
|
|
337
|
+
readonly needles?: readonly string[];
|
|
216
338
|
};
|
|
217
339
|
/** Scope memoization is owned by one enumeration; nested runs inherit their parent's project. */
|
|
218
340
|
function membership(
|
|
@@ -240,25 +362,31 @@ function membership(
|
|
|
240
362
|
};
|
|
241
363
|
return belongs;
|
|
242
364
|
}
|
|
243
|
-
/** Newest modified project session files, optionally limited by file key or
|
|
365
|
+
/** Newest modified project session files, optionally limited by file key or needles. */
|
|
244
366
|
export async function* scopedSessionFiles(scope: SessionScope): AsyncGenerator<string> {
|
|
245
|
-
const files = sessionFiles(scope.dir);
|
|
367
|
+
const files = await sessionFiles(scope.dir);
|
|
246
368
|
const fileSet = new Set(files);
|
|
247
369
|
const belongs = membership(scope, (file) => fileSet.has(file));
|
|
248
|
-
const selected =
|
|
249
|
-
.filter(
|
|
370
|
+
const selected = await newestFirst(
|
|
371
|
+
files.filter(
|
|
250
372
|
(file) =>
|
|
251
|
-
|
|
252
|
-
(scope.
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
.
|
|
257
|
-
|
|
258
|
-
|
|
373
|
+
scope.fileKey === undefined ||
|
|
374
|
+
scope.fileKey.startsWith(shortKey(relative(scope.dir, file))),
|
|
375
|
+
),
|
|
376
|
+
);
|
|
377
|
+
const prefilter =
|
|
378
|
+
scope.needles === undefined
|
|
379
|
+
? undefined
|
|
380
|
+
: ripgrepFilter(scope.dir, scope.needles, scope.signal);
|
|
381
|
+
for (const phase of searchPhases(selected)) {
|
|
382
|
+
// Older files are searched only when the newest ones did not satisfy the caller.
|
|
259
383
|
// oxlint-disable-next-line no-await-in-loop
|
|
260
|
-
|
|
261
|
-
|
|
384
|
+
for (const file of prefilter === undefined ? phase : await prefilter(phase)) {
|
|
385
|
+
// Preserve newest-first traversal while closing each header stream before opening another.
|
|
386
|
+
// oxlint-disable-next-line no-await-in-loop
|
|
387
|
+
if (await belongs(file)) {
|
|
388
|
+
yield file;
|
|
389
|
+
}
|
|
262
390
|
}
|
|
263
391
|
}
|
|
264
392
|
}
|
package/src/ui/cards.ts
CHANGED
|
@@ -9,7 +9,16 @@ import {
|
|
|
9
9
|
type HistorySection,
|
|
10
10
|
type PosthorseDisplay,
|
|
11
11
|
} from "./display.ts";
|
|
12
|
-
import {
|
|
12
|
+
import {
|
|
13
|
+
argsOf,
|
|
14
|
+
argument,
|
|
15
|
+
clean,
|
|
16
|
+
hint,
|
|
17
|
+
leadingRows,
|
|
18
|
+
textBlock,
|
|
19
|
+
textOf,
|
|
20
|
+
type OutputColor,
|
|
21
|
+
} from "./text.ts";
|
|
13
22
|
|
|
14
23
|
export type ToolName = "notes" | "history" | "get_context_remaining" | "new_context";
|
|
15
24
|
const titles: Readonly<Record<ToolName, string>> = {
|
|
@@ -197,45 +206,6 @@ function previewBody(facts: ResultFacts, sections: readonly HistorySection[] | u
|
|
|
197
206
|
: body;
|
|
198
207
|
}
|
|
199
208
|
|
|
200
|
-
interface ResultLayout {
|
|
201
|
-
readonly full: Component;
|
|
202
|
-
readonly summary: Component;
|
|
203
|
-
readonly preview: Component;
|
|
204
|
-
readonly submitted: Component;
|
|
205
|
-
readonly attachment: string;
|
|
206
|
-
readonly attachmentBlock: Component;
|
|
207
|
-
readonly footer: Component;
|
|
208
|
-
readonly searchPreviews: readonly string[] | undefined;
|
|
209
|
-
readonly hasDetails: boolean;
|
|
210
|
-
}
|
|
211
|
-
|
|
212
|
-
function resultLayout(
|
|
213
|
-
facts: ResultFacts,
|
|
214
|
-
sections: readonly HistorySection[] | undefined,
|
|
215
|
-
theme: Theme,
|
|
216
|
-
): ResultLayout {
|
|
217
|
-
const color = resultColor(facts);
|
|
218
|
-
const submission = submittedText(facts);
|
|
219
|
-
const attachment = attachmentSummary(facts);
|
|
220
|
-
const { display } = facts;
|
|
221
|
-
const footerLength = display?.kind === "history-search" ? (display.footerLength ?? 0) : 0;
|
|
222
|
-
return {
|
|
223
|
-
full: fullBody(facts, sections, theme),
|
|
224
|
-
summary: textBlock(resultSummary(facts, sections), theme, color),
|
|
225
|
-
preview: textBlock(previewBody(facts, sections), theme, color),
|
|
226
|
-
submitted: textBlock(submission, theme),
|
|
227
|
-
attachment,
|
|
228
|
-
attachmentBlock: textBlock(attachment, theme, "muted"),
|
|
229
|
-
footer: textBlock(
|
|
230
|
-
sections !== undefined && footerLength > 0 ? facts.raw.slice(-footerLength) : "",
|
|
231
|
-
theme,
|
|
232
|
-
"muted",
|
|
233
|
-
),
|
|
234
|
-
searchPreviews: searchPreviews(facts, sections, theme),
|
|
235
|
-
hasDetails: sections !== undefined || submission.length > 0 || noteHeaderLength(facts) > 0,
|
|
236
|
-
};
|
|
237
|
-
}
|
|
238
|
-
|
|
239
209
|
function attachmentSummary(facts: ResultFacts): string {
|
|
240
210
|
if (facts.imageCount === 0) {
|
|
241
211
|
return "";
|
|
@@ -275,58 +245,83 @@ function searchPreviews(
|
|
|
275
245
|
: undefined;
|
|
276
246
|
}
|
|
277
247
|
|
|
278
|
-
|
|
248
|
+
type RowView = (width: number) => string[];
|
|
249
|
+
|
|
250
|
+
function expandedView(
|
|
251
|
+
facts: ResultFacts,
|
|
252
|
+
sections: readonly HistorySection[] | undefined,
|
|
253
|
+
theme: Theme,
|
|
254
|
+
): RowView {
|
|
255
|
+
const full = fullBody(facts, sections, theme);
|
|
256
|
+
const attachment = textBlock(attachmentSummary(facts), theme, "muted");
|
|
257
|
+
const { display } = facts;
|
|
258
|
+
const footerLength = display?.kind === "history-search" ? (display.footerLength ?? 0) : 0;
|
|
259
|
+
const footer = textBlock(
|
|
260
|
+
sections !== undefined && footerLength > 0 ? facts.raw.slice(-footerLength) : "",
|
|
261
|
+
theme,
|
|
262
|
+
"muted",
|
|
263
|
+
);
|
|
264
|
+
const submission = submittedText(facts);
|
|
265
|
+
const submitted = textBlock(submission, theme);
|
|
266
|
+
return (width) => [
|
|
267
|
+
...full.render(width),
|
|
268
|
+
...attachment.render(width),
|
|
269
|
+
...footer.render(width),
|
|
270
|
+
...(submission.length > 0 ? ["", ...submitted.render(width)] : []),
|
|
271
|
+
];
|
|
272
|
+
}
|
|
273
|
+
|
|
274
|
+
function previewSource(
|
|
275
|
+
facts: ResultFacts,
|
|
276
|
+
sections: readonly HistorySection[] | undefined,
|
|
277
|
+
theme: Theme,
|
|
278
|
+
): (width: number) => readonly string[] {
|
|
279
279
|
if (summaryOnly(facts.display) && !facts.isError && !facts.isPartial) {
|
|
280
|
-
return [];
|
|
280
|
+
return () => [];
|
|
281
281
|
}
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
282
|
+
const searches = searchPreviews(facts, sections, theme);
|
|
283
|
+
if (searches !== undefined) {
|
|
284
|
+
return (width) => searches.map((line) => truncateToWidth(line, width));
|
|
285
|
+
}
|
|
286
|
+
return leadingRows(previewBody(facts, sections), theme, resultColor(facts), 3);
|
|
286
287
|
}
|
|
287
288
|
|
|
288
|
-
function
|
|
289
|
+
function collapsedView(
|
|
289
290
|
facts: ResultFacts,
|
|
290
|
-
|
|
291
|
+
sections: readonly HistorySection[] | undefined,
|
|
291
292
|
theme: Theme,
|
|
292
|
-
|
|
293
|
-
)
|
|
294
|
-
const
|
|
295
|
-
const
|
|
296
|
-
const
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
293
|
+
): RowView {
|
|
294
|
+
const summary = textBlock(resultSummary(facts, sections), theme, resultColor(facts));
|
|
295
|
+
const previews = previewSource(facts, sections, theme);
|
|
296
|
+
const attachment = attachmentSummary(facts);
|
|
297
|
+
const details =
|
|
298
|
+
summaryOnly(facts.display) ||
|
|
299
|
+
sections !== undefined ||
|
|
300
|
+
submittedText(facts).length > 0 ||
|
|
301
|
+
noteHeaderLength(facts) > 0;
|
|
302
|
+
return (width) => {
|
|
303
|
+
const summaryRows = summary.render(width).slice(0, 5);
|
|
304
|
+
const rows = previews(width);
|
|
305
|
+
const shown = rows.slice(
|
|
306
|
+
0,
|
|
307
|
+
Math.max(0, Math.min(3, 5 - summaryRows.length - (attachment.length > 0 ? 1 : 0))),
|
|
308
|
+
);
|
|
309
|
+
const attachments =
|
|
310
|
+
attachment.length > 0 ? [truncateToWidth(theme.fg("muted", attachment), width)] : [];
|
|
311
|
+
const hidden = details || rows.length > shown.length;
|
|
312
|
+
return [...summaryRows, ...shown, ...attachments, ...(hidden ? [hint(theme, width)] : [])];
|
|
313
|
+
};
|
|
307
314
|
}
|
|
308
315
|
|
|
309
316
|
function resultComponent(facts: ResultFacts, theme: Theme): Component {
|
|
310
317
|
const sections =
|
|
311
318
|
!facts.isError && !facts.isPartial ? historySections(facts.raw, facts.display) : undefined;
|
|
312
|
-
const
|
|
319
|
+
const render = (facts.expanded ? expandedView : collapsedView)(facts, sections, theme);
|
|
313
320
|
return {
|
|
314
321
|
invalidate() {
|
|
315
|
-
// Pi creates this presentation for each result
|
|
316
|
-
},
|
|
317
|
-
render(width) {
|
|
318
|
-
if (facts.expanded) {
|
|
319
|
-
const submitted =
|
|
320
|
-
submittedText(facts).length > 0 ? ["", ...layout.submitted.render(width)] : [];
|
|
321
|
-
return [
|
|
322
|
-
...layout.full.render(width),
|
|
323
|
-
...layout.attachmentBlock.render(width),
|
|
324
|
-
...layout.footer.render(width),
|
|
325
|
-
...submitted,
|
|
326
|
-
];
|
|
327
|
-
}
|
|
328
|
-
return collapsedRows(facts, layout, theme, width);
|
|
322
|
+
// Pi creates this presentation for each result, theme, and expansion update.
|
|
329
323
|
},
|
|
324
|
+
render,
|
|
330
325
|
};
|
|
331
326
|
}
|
|
332
327
|
|
package/src/ui/text.ts
CHANGED
|
@@ -44,6 +44,44 @@ export function textBlock(text: string, theme: Theme, color: OutputColor = "tool
|
|
|
44
44
|
return new Text(text.length > 0 ? theme.fg(color, clean(text)) : "", 0, 0);
|
|
45
45
|
}
|
|
46
46
|
|
|
47
|
+
const graphemes = new Intl.Segmenter(undefined, { granularity: "grapheme" });
|
|
48
|
+
|
|
49
|
+
/**
|
|
50
|
+
* Rows of `textBlock(text)` wrapped only as far as a preview needs, so resizing long text stays
|
|
51
|
+
* cheap: the first `count` rows are exact, and any further row means the full text has more.
|
|
52
|
+
*/
|
|
53
|
+
export function leadingRows(
|
|
54
|
+
text: string,
|
|
55
|
+
theme: Theme,
|
|
56
|
+
color: OutputColor,
|
|
57
|
+
count: number,
|
|
58
|
+
): (width: number) => readonly string[] {
|
|
59
|
+
// One line past `count` already wraps to more rows when the text continues.
|
|
60
|
+
const head = clean(text)
|
|
61
|
+
.split("\n", count + 1)
|
|
62
|
+
.join("\n");
|
|
63
|
+
const segments = graphemes.segment(head);
|
|
64
|
+
const wrap = (width: number): readonly string[] => {
|
|
65
|
+
for (let limit = (count + 1) * width + 1024; ; limit *= 2) {
|
|
66
|
+
// pi-tui wraps greedily by grapheme, so text cut between graphemes wraps like the
|
|
67
|
+
// whole text in every row but its last.
|
|
68
|
+
const end = segments.containing(limit)?.index ?? head.length;
|
|
69
|
+
const prefix = theme.fg(color, head.slice(0, end));
|
|
70
|
+
const rows = new Text(text.length > 0 ? prefix : "", 0, 0).render(width);
|
|
71
|
+
if (end === head.length || rows.length > count) {
|
|
72
|
+
return rows;
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
};
|
|
76
|
+
let cached: { readonly width: number; readonly rows: readonly string[] } | undefined;
|
|
77
|
+
return (width) => {
|
|
78
|
+
if (cached?.width !== width) {
|
|
79
|
+
cached = { width, rows: wrap(width) };
|
|
80
|
+
}
|
|
81
|
+
return cached.rows;
|
|
82
|
+
};
|
|
83
|
+
}
|
|
84
|
+
|
|
47
85
|
export function hint(theme: Theme, width: number): string {
|
|
48
86
|
const key = keyText("app.tools.expand");
|
|
49
87
|
return truncateToWidth(
|
package/ui.ts
CHANGED
|
@@ -1,6 +1,12 @@
|
|
|
1
1
|
import type { ExtensionAPI, MessageRenderer } from "@earendil-works/pi-coding-agent";
|
|
2
|
-
import {
|
|
3
|
-
|
|
2
|
+
import {
|
|
3
|
+
Box,
|
|
4
|
+
MouseRegion,
|
|
5
|
+
truncateToWidth,
|
|
6
|
+
type Component,
|
|
7
|
+
type Text,
|
|
8
|
+
} from "@earendil-works/pi-tui";
|
|
9
|
+
import { clean, hint, leadingRows, textBlock, textOf } from "./src/ui/text.ts";
|
|
4
10
|
|
|
5
11
|
export type { PosthorseDisplay } from "./src/ui/display.ts";
|
|
6
12
|
export { toolCards } from "./src/ui/cards.ts";
|
|
@@ -27,12 +33,13 @@ export function registerPosthorseMessages(pi: ExtensionAPI): void {
|
|
|
27
33
|
const preview = window
|
|
28
34
|
? raw.slice(handoffAt < 0 ? raw.length : handoffAt + handoffLabel.length)
|
|
29
35
|
: raw;
|
|
30
|
-
const
|
|
31
|
-
const previewText =
|
|
36
|
+
const previewRows = window ? 2 : 3;
|
|
37
|
+
const previewText = leadingRows(preview, theme, "toolOutput", previewRows);
|
|
38
|
+
let fullText: Text | undefined;
|
|
32
39
|
const historyText = textBlock("Earlier conversation remains in history.", theme, "muted");
|
|
33
40
|
const content: Component = {
|
|
34
41
|
invalidate() {
|
|
35
|
-
//
|
|
42
|
+
// Pi calls the renderer again on invalidation, so cached rows never outlive a theme.
|
|
36
43
|
},
|
|
37
44
|
render(width) {
|
|
38
45
|
const padding = Math.min(
|
|
@@ -46,10 +53,10 @@ export function registerPosthorseMessages(pi: ExtensionAPI): void {
|
|
|
46
53
|
inner,
|
|
47
54
|
);
|
|
48
55
|
const body = view.expanded
|
|
49
|
-
? fullText.render(inner)
|
|
56
|
+
? (fullText ??= textBlock(raw, theme)).render(inner)
|
|
50
57
|
: [
|
|
51
58
|
...(window ? historyText.render(inner).slice(0, 2) : []),
|
|
52
|
-
...previewText
|
|
59
|
+
...previewText(inner).slice(0, previewRows),
|
|
53
60
|
hint(theme, inner),
|
|
54
61
|
];
|
|
55
62
|
const box = new Box(padding, 1, (line) => theme.bg("customMessageBg", line));
|