pi-posthorse 0.8.2 → 0.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -93
- package/index.ts +12 -7
- package/package.json +1 -1
- package/src/budget.ts +2 -21
- package/src/context-tools.ts +3 -9
- package/src/recovery.ts +94 -17
- package/src/rollover.ts +137 -3
package/README.md
CHANGED
|
@@ -1,133 +1,91 @@
|
|
|
1
1
|
# Posthorse
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Posthorse starts fresh context windows for long [Pi](https://github.com/earendil-works/pi) tasks and keeps earlier conversation recoverable. Each rollover uses a handoff or recovery record without a model-generated summary.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+

|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Install and start
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Use Node 24.12.0 or later and Pi 1.0.0 or later. Posthorse supports official Pi and [fitchmultz/pi](https://github.com/fitchmultz/pi).
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
- Node `>=24.12.0`.
|
|
14
|
-
- Pi `>=1.0.0`, official (`@earendil-works/pi-coding-agent`) or a maintained fork with the same public compaction contracts. No checkpoint, metadata iterator, or native context-window API is required.
|
|
15
|
-
|
|
16
|
-
Required qualification targets: latest stable official Pi and latest maintained fork `main`, resolved once per workflow run and retained with exact SDK/CLI identity evidence. Locked development dependencies are reproducible build snapshots, not validation targets. Historical official Pi `1.0.0` ([source `a13d35a`](https://github.com/earendil-works/pi/commit/a13d35a742c6ef8462812a28fbe1d8c8b7431c32)), TypeBox `1.3.27` and Node `24.21.0` evidence does not certify newer hosts. Qualification does not install, activate, or restart the live runtime.
|
|
17
|
-
|
|
18
|
-
## Install
|
|
19
|
-
|
|
20
|
-
Posthorse `0.8.2` requires Pi `>=1.0.0`. Update the host before installing this release; do not activate it on an older live fork.
|
|
11
|
+
Update an older Pi installation before you install Posthorse. Run these commands:
|
|
21
12
|
|
|
22
13
|
```bash
|
|
23
|
-
pi install npm:pi-posthorse
|
|
24
|
-
pi
|
|
25
|
-
pi -e npm:pi-posthorse@0.8.2 # try it for one run without installing
|
|
14
|
+
pi install npm:pi-posthorse
|
|
15
|
+
pi
|
|
26
16
|
```
|
|
27
17
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
Update an unpinned install with `pi update npm:pi-posthorse` or `pi update --extensions`. Move a pinned install with `pi install npm:pi-posthorse@<new version>` or `pi install git:github.com/fitchmultz/pi-posthorse@<new tag>`. Uninstall with `pi remove` followed by the source you installed. Removing the package leaves `.pi/notes` and Pi's session history in place.
|
|
31
|
-
|
|
32
|
-
Restart Pi after installing or updating Posthorse to load the new extension code.
|
|
33
|
-
|
|
34
|
-
Keep exactly one copy loaded. `pi list` shows every package source; if a second source or a local checkout is listed, `pi remove` it, otherwise two copies register the same tools and compete for the same rollover hook.
|
|
35
|
-
|
|
36
|
-
## Official Pi and the fork
|
|
37
|
-
|
|
38
|
-
Both hosts get the same tools, guidance, reminders, notes, history, and recovery records. Rollover is a public compaction entry retaining no earlier conversation, with the handoff as its summary; no summary model is called. `new_context` commits at turn end only after Posthorse's own callback succeeds and every sibling tool succeeds. A foreign tool with the same name cannot authorize a rollover. Automatic rollover handles `session_before_compact`; manual `/compact` stays Pi's own summarization.
|
|
39
|
-
|
|
40
|
-
- **Native eligibility on both 1.0 targets:** Pi's compaction preparation must succeed before the automatic hook runs. Posthorse's custom compaction result bypasses the default summarizer and its credential lookup. With default `keepRecentTokens`, tiny overflow/truncated responses and a second large result without a new user turn can miss the hook. Lowering `keepRecentTokens` narrows, but does not eliminate, that gap. Oversized initial input can reach the provider before recovery.
|
|
41
|
-
- **Policy:** reminders and budget reports use persisted settings by default; automatic rollover uses the live settings in the event. SDK hosts can inject live policy with `createPosthorse(getPolicy)`. Handoffs appear as compaction-summary messages. Background work owned outside the foreground batch is not serialized or stopped by Posthorse.
|
|
42
|
-
|
|
43
|
-
Legacy fork `context_window` entries remain readable for history recovery. New boundaries do not use the retired native-window/async ABI. Opening legacy fork journals in official Pi can reintroduce earlier context because official Pi does not interpret those old markers; do not assume cross-host session migration.
|
|
44
|
-
|
|
45
|
-
## How it works
|
|
18
|
+
Restart Pi after installation or an update. To try Posthorse for one session, run `pi -e npm:pi-posthorse`.
|
|
46
19
|
|
|
47
|
-
|
|
48
|
-
2. **One best-effort checkpoint.** While Pi compaction is enabled, one reminder may appear shortly before Pi's rollover line, asking the model to update its current-state note with available file-editing tools and call `new_context`. Already-current notes need no rewrite; `notes write` remains the fallback for creation, substantial restructuring, or unavailable editing tools. A large turn, overflow, restart, or smaller model can reach rollover without it. Reminders are fingerprinted by window, context size, and reserve, so switching to a different context size gets a fresh reminder. Stale reminders and reminders left after disabling compaction are filtered from model input while remaining in history.
|
|
49
|
-
3. **`get_context_remaining`.** Reports the best available native estimate of tokens until Pi's automatic rollover line and until the configured context limit. That configured limit is not a measured provider rejection boundary. Pi's value is an estimate until the active model reports usage. Check before claiming a remaining-token count or changing plans because of context limits. The checkpoint reminder marks the deadline, so routine budget checks are unnecessary.
|
|
50
|
-
4. **`new_context`.** Requests an atomic retain-none rollover once the current foreground tool batch succeeds. An optional handoff becomes the first state of the fresh window. Failed sibling writes, aborted turns, or a handoff that no longer fits after queued input prevent commitment. The request card is not proof that the boundary committed. Own successful callback IDs and admitted handoffs are consumed once at settlement; transcript names alone cannot request a reset.
|
|
51
|
-
5. **Automatic rollover without summaries.** When Pi reaches the public automatic hook with a supported budget, Posthorse builds a bounded recovery record and retains only an invisible boundary sentinel. Cancellation or recovery construction failure leaves active context intact rather than silently invoking a summarizer. Unsupported small-model budgets retain Pi's native fallback. Manual `/compact` is unchanged. Native eligibility limits are listed above.
|
|
52
|
-
6. **Bounded recovery record.** The automatic handoff quotes the window's first owner request, its latest owner input (a direct user input or `ask_question` outcome), and its latest input of any kind, including visible coordination messages. It adds tool result evidence: tool names and namespaces, call ids, requested and admitted execution arguments when available, result excerpts of up to 1,500 characters, and entry ids to recover the rest. Every other current-window input gets one index line with its label, time, and entry id; owner inputs keep a one-line preview there. Only a successful result positively linked to a projected unnamespaced `ask_question` call **and** an effective registered unnamespaced `ask_question` definition is an owner answer. Providers can omit call namespaces, so call metadata alone is insufficient. Namespaced, unregistered or unresolved-provenance results remain tool evidence, never owner approval. Legacy async results still visible in a projected journal remain eligible even across later complete assistant responses; this is history compatibility, not a new native-async execution path. A result Pi still projects is also eligible when its call was edited out or predates the window, without attributing it to another call. These results may already have been received or handled; they are not proof of current progress. Synchronous results with projected call provenance are kept only from the trailing batch without a later complete response. Pi context edits also apply to these inputs and results: omitted entries stay out, and replacements supersede their original content. A clearly labeled, possibly stale older checkpoint comes last. Older assistant prose is not treated as current state. Newly submitted input stays separate and is saved after the boundary, not copied into the handoff.
|
|
53
|
-
7. **`notes` and `history`.** Notes are shared across linked worktrees, at the main checkout for conventional Git layouts or inside the common Git directory when metadata is stored separately. History searches normalized transcript text, skips entries whose full text the model already sees in its active context (and says how many), and returns stored images for a requested entry. Edited-away originals and checkpoint reminders stay searchable. Tool calls and results retain names, namespaces, and call ids; admitted execution arguments are labeled separately from requested arguments when present. When a handoff carries edited content, its entry ID points to the replacement; `history read` of that ID returns the replacement text or images. Reading the original entry ID still returns the raw journal content.
|
|
20
|
+
Give Pi your task. For example:
|
|
54
21
|
|
|
55
|
-
|
|
22
|
+
> Save task state in a note. Start a fresh context window with the next steps.
|
|
56
23
|
|
|
57
|
-
|
|
24
|
+
## How rollover works
|
|
58
25
|
|
|
59
|
-
|
|
26
|
+
Posthorse can remind the model to save task state before rollover. Large turns can reach rollover before this reminder. The reminder does not wake an agent after its final answer.
|
|
60
27
|
|
|
61
|
-
Automatic
|
|
28
|
+
Automatic rollover creates a recovery record from session inputs and tool results. The model must restore notes and check live state before it continues. Manual `/compact` uses Pi's normal summarization.
|
|
62
29
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
Posthorse follows Pi's effective `compaction` settings, including Pi's decision about whether project settings are trusted. Disabling `compaction.enabled` disables reminders and automatic rollover; `new_context` stays available.
|
|
66
|
-
|
|
67
|
-
The model's context window minus `compaction.reserveTokens` must leave at least 10,000 usable tokens. Below that (for example an 8K or 16K model with the default 16,384 reserve) Posthorse reports an unsupported configuration in the guidance and in `get_context_remaining`, turns automatic behavior off for that model, and leaves Pi's own compaction in place. Lower the reserve or use a larger model. The checkpoint reminder band is the last 10% of usable context, capped at 32,000 tokens, so a large reserve cannot trigger a reminder immediately in a fresh window.
|
|
68
|
-
|
|
69
|
-
Explicit and automatic handoffs are capped at 20,000 characters and half the native context limit's fresh operational capacity after prompt/tool overhead and any pending input, whichever is smaller. Oversized explicit handoffs are rejected with an instruction to save fuller state in notes; automatic rollover cancels visibly when a supported configuration has no room for a safe recovery record. Notes and history pages are capped at 40,000 characters and shrink to the remaining budget; when usage is not known yet, they also leave half the fresh operational capacity free. Payload budgets reserve Posthorse's request-local guidance too, because native estimates can omit it.
|
|
70
|
-
|
|
71
|
-
Only one automatic compaction or rollover policy extension should be enabled at a time. Pi keeps the last non-cancel result from multiple handlers of the same hook, so load order would otherwise decide which policy wins.
|
|
30
|
+
See [rollover and recovery details](docs/reference.md#how-it-works).
|
|
72
31
|
|
|
73
32
|
## Tools
|
|
74
33
|
|
|
75
|
-
|
|
76
|
-
- `get_context_remaining()`
|
|
77
|
-
- `notes({ op, ... })`: `list`, `read`, and `search` are paged; continue with the returned character `offset`. `list` shows the absolute shared notes directory, then relative note names newest first with size and modification time (UTC), and takes an optional folder `path`; `read` of a folder returns its listing. File reads show the absolute file path before the content. Offsets refer to note content or listing rows, excluding path headers. `search` covers text notes, skipping binary files (a NUL byte in the first 8,000 bytes, as Git decides), and its excerpts center on the match. It also skips files over 1 MiB and says how many it skipped. Search stops reading once the requested page is decided, so a broad query reports that more matches follow instead of a total; its pages end before offset 1,000,000. `write` replaces content (empty content clears); `append` adds one newline-terminated record.
|
|
78
|
-
- `history({ op, ... })`: `search` skips entries whose full text is already in the active context, continues with the returned `cursor` and the same `query` and `all` scope, and treats `limit` as the maximum results per page. `read` pages text and stored images; continue with the returned character `offset` and `imageOffset`. An image-only continuation can use an `offset` equal to the text length. If a page needs fresh context, retry with both offsets unchanged. Results keep native entry and window ids.
|
|
79
|
-
|
|
80
|
-
In the TUI, tools use Pi's native expandable cards. Collapsed cards show the operation and target, a short content preview, and counts or page ranges with the next offset when more remains. Expand with Pi's tool-output shortcut (`Ctrl+O` by default), or click the card's header or body in fullscreen mode. Expanded cards show the complete returned page and its metadata, not content the tool has not fetched yet. Writes and context requests also show the submitted content or handoff.
|
|
34
|
+
These tools are available to the model:
|
|
81
35
|
|
|
82
|
-
|
|
36
|
+
| Tool | Use |
|
|
37
|
+
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
38
|
+
| `new_context({ handoff? })` | Request a fresh window after every foreground tool succeeds. Aborted turns and failed tools prevent the reset. |
|
|
39
|
+
| `get_context_remaining()` | Estimate tokens left until rollover and the configured context limit. Usage can be unknown before a model response. |
|
|
40
|
+
| `notes({ op, ... })` | List, read, search, write, or append task notes. |
|
|
41
|
+
| `history({ op, ... })` | Search earlier conversation and read entries, including images. `all: true` includes this project's other sessions and nested subagent runs. |
|
|
83
42
|
|
|
84
|
-
|
|
43
|
+
Expand a tool card with Ctrl+O. In fullscreen mode, click a tool card to expand it. A `new_context` card shows a request. Pi's `Compacted from … tokens` card confirms the rollover.
|
|
85
44
|
|
|
86
|
-
|
|
45
|
+
See [tool examples and paging details](docs/reference.md#tools).
|
|
87
46
|
|
|
88
|
-
|
|
47
|
+
## Settings and limits
|
|
89
48
|
|
|
90
|
-
|
|
49
|
+
Posthorse uses Pi's effective `compaction` settings. It has no separate settings.
|
|
91
50
|
|
|
92
|
-
|
|
51
|
+
| Pi setting | Effect |
|
|
52
|
+
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
53
|
+
| `compaction.enabled` | Set to `false` to disable reminders and automatic rollover. `new_context` remains available. |
|
|
54
|
+
| `compaction.reserveTokens` | Automatic Posthorse rollover needs at least 10,000 tokens after this reserve. Smaller budgets use Pi's own compaction. Lower the reserve or choose a larger model. |
|
|
93
55
|
|
|
94
|
-
|
|
56
|
+
Two Posthorse copies can register the same tools and compete for rollover. Keep one copy loaded. Check installed sources with `pi list`.
|
|
95
57
|
|
|
96
|
-
|
|
58
|
+
Multiple compaction-policy extensions can make rollover depend on load order. Enable only one.
|
|
97
59
|
|
|
98
|
-
|
|
99
|
-
- Posthorse makes no network requests.
|
|
100
|
-
- Notes are plaintext files under `.pi/notes`. They survive package removal and may be committed unless ignored.
|
|
101
|
-
- `history` with `all: true` scans project-matching JSONL files and their nested subagent sessions in the active Pi session directory, not unrelated projects that share that directory. It runs ripgrep locally over that directory when available.
|
|
102
|
-
- History can return user text, assistant text and thinking, assistant failure status and error messages, tool arguments and results, handoffs, custom messages, and images. Direct shell entries Pi marked `excludeFromContext` come back as a placeholder only.
|
|
103
|
-
- Returned history content enters the currently selected model and provider context.
|
|
104
|
-
- Removing Posthorse does not remove notes or Pi session history.
|
|
60
|
+
Oversized input and some overflow or truncated responses can reach the provider before automatic recovery. Save task state in notes. Background work outside the foreground tool batch continues across rollover.
|
|
105
61
|
|
|
106
|
-
|
|
62
|
+
See [size limits and extension order](docs/reference.md#settings).
|
|
63
|
+
Check the [legacy-session details](docs/reference.md#official-pi-and-the-fork) before you move old fork sessions to official Pi.
|
|
107
64
|
|
|
108
|
-
##
|
|
65
|
+
## Update or remove
|
|
109
66
|
|
|
110
|
-
|
|
67
|
+
```bash
|
|
68
|
+
pi update npm:pi-posthorse
|
|
69
|
+
pi remove npm:pi-posthorse
|
|
70
|
+
```
|
|
111
71
|
|
|
112
|
-
|
|
72
|
+
Removal leaves notes and Pi session history in place. See the [installation reference](docs/reference.md#install) for Git sources.
|
|
113
73
|
|
|
114
|
-
##
|
|
74
|
+
## Data and privacy
|
|
115
75
|
|
|
116
|
-
|
|
117
|
-
npm ci --ignore-scripts
|
|
118
|
-
npm run verify # strict policy, lint, formatting, typecheck, tests, package build
|
|
119
|
-
npm run check:compat # typecheck, unit tests, and the real-SDK suite for the installed host
|
|
120
|
-
```
|
|
76
|
+
Posthorse makes no network requests. Notes are plaintext files under `.pi/notes/`; linked Git worktrees share them. Add `.pi/notes/` to `.gitignore` if the notes must stay private.
|
|
121
77
|
|
|
122
|
-
|
|
123
|
-
exception governance, editor setup, and the enforced acceptance workflow.
|
|
78
|
+
History can include user messages, assistant thinking, tool arguments, errors, and images. Content you retrieve enters the context of your selected model and provider.
|
|
124
79
|
|
|
125
|
-
|
|
80
|
+
See [storage and privacy details](docs/reference.md#data-and-privacy).
|
|
81
|
+
For cost diagnosis, see the [cached-token investigation](ARTIFACTS/investigation-report.md).
|
|
82
|
+
It does not establish the cause of a production bill or a measured cost improvement.
|
|
126
83
|
|
|
127
|
-
Development
|
|
84
|
+
## Development
|
|
128
85
|
|
|
129
|
-
|
|
86
|
+
See [development and host qualification](docs/development.md) for setup, tests, and releases.
|
|
87
|
+
See [code quality](docs/quality.md) for the required checks.
|
|
130
88
|
|
|
131
|
-
|
|
89
|
+
## License
|
|
132
90
|
|
|
133
|
-
|
|
91
|
+
[MIT](LICENSE).
|
package/index.ts
CHANGED
|
@@ -4,9 +4,13 @@
|
|
|
4
4
|
* Pi owns persisted retain-none context boundaries. Posthorse owns reminder and
|
|
5
5
|
* rollover policy, bounded recovery, durable notes, and history retrieval.
|
|
6
6
|
*/
|
|
7
|
-
import
|
|
7
|
+
import {
|
|
8
|
+
SettingsManager,
|
|
9
|
+
type ExtensionAPI,
|
|
10
|
+
type ExtensionContext,
|
|
11
|
+
} from "@earendil-works/pi-coding-agent";
|
|
8
12
|
import { registerPosthorseMessages } from "./ui.ts";
|
|
9
|
-
import {
|
|
13
|
+
import { policySnapshot } from "./src/budget.ts";
|
|
10
14
|
import type { CompactionPolicy, PolicyReader } from "./src/contracts.ts";
|
|
11
15
|
import { registerHistory } from "./src/history.ts";
|
|
12
16
|
import { safeJsonStringify } from "./src/message-text.ts";
|
|
@@ -32,13 +36,14 @@ function activeToolTokens(pi: ExtensionAPI): number {
|
|
|
32
36
|
0,
|
|
33
37
|
);
|
|
34
38
|
}
|
|
35
|
-
/**
|
|
36
|
-
export function createPosthorse(
|
|
37
|
-
getPolicy: PolicyReader = persistedPolicy,
|
|
38
|
-
): (pi: ExtensionAPI) => void {
|
|
39
|
+
/** Use Pi's live effective settings by default; SDK hosts can inject a policy reader. */
|
|
40
|
+
export function createPosthorse(getPolicy?: PolicyReader): (pi: ExtensionAPI) => void {
|
|
39
41
|
return (pi) => {
|
|
42
|
+
const readPolicy: PolicyReader =
|
|
43
|
+
getPolicy ??
|
|
44
|
+
((ctx) => SettingsManager.inMemory(pi.getSettings()).getCompactionSettings(ctx.model));
|
|
40
45
|
const policy = (ctx: ExtensionContext, livePolicy?: CompactionPolicy) =>
|
|
41
|
-
policySnapshot(ctx,
|
|
46
|
+
policySnapshot(ctx, readPolicy, livePolicy);
|
|
42
47
|
const toolTokens = () => activeToolTokens(pi);
|
|
43
48
|
const pages = new PageReservations(toolTokens);
|
|
44
49
|
registerPosthorseMessages(pi);
|
package/package.json
CHANGED
package/src/budget.ts
CHANGED
|
@@ -1,9 +1,4 @@
|
|
|
1
|
-
import {
|
|
2
|
-
estimateTokens,
|
|
3
|
-
getAgentDir,
|
|
4
|
-
SettingsManager,
|
|
5
|
-
type ExtensionContext,
|
|
6
|
-
} from "@earendil-works/pi-coding-agent";
|
|
1
|
+
import { estimateTokens, type ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
7
2
|
import {
|
|
8
3
|
ESTIMATED_IMAGE_CHARS,
|
|
9
4
|
MAX_HANDOFF_CHARS,
|
|
@@ -18,8 +13,6 @@ import {
|
|
|
18
13
|
import { imagesOf, textOf } from "./message-text.ts";
|
|
19
14
|
|
|
20
15
|
const MIN_USABLE_TOKENS = Math.ceil(MAX_HANDOFF_CHARS / 4) * 2;
|
|
21
|
-
export const SNAPSHOT_NOTE =
|
|
22
|
-
"Reminder and budget policy uses available settings (a persisted CLI snapshot by default); Pi's live settings control automatic compaction.";
|
|
23
16
|
export const CHECKPOINT_STEPS =
|
|
24
17
|
"update the task's current-state note (goal, progress, decisions, next steps) with available file-editing tools at its absolute path; use notes write only for creation, substantial restructuring, or when editing tools are unavailable. If the note is already current, leave it unchanged. Then call new_context";
|
|
25
18
|
export type Budget = {
|
|
@@ -30,18 +23,6 @@ export type Budget = {
|
|
|
30
23
|
readonly rolloverAt: number;
|
|
31
24
|
readonly supported: boolean;
|
|
32
25
|
};
|
|
33
|
-
export function persistedPolicy(ctx: ExtensionContext): CompactionPolicy {
|
|
34
|
-
const settings = SettingsManager.create(ctx.cwd, getAgentDir(), {
|
|
35
|
-
projectTrusted: ctx.isProjectTrusted(),
|
|
36
|
-
});
|
|
37
|
-
const errors = settings.drainErrors();
|
|
38
|
-
if (errors.length > 0) {
|
|
39
|
-
throw new Error(
|
|
40
|
-
`Posthorse could not read persisted Pi settings: ${errors.map((error) => error.error.message).join("; ")}`,
|
|
41
|
-
);
|
|
42
|
-
}
|
|
43
|
-
return settings.getCompactionSettings(ctx.model);
|
|
44
|
-
}
|
|
45
26
|
/** A synchronous evaluation owns one snapshot; never carry it across an await. */
|
|
46
27
|
export function policySnapshot(
|
|
47
28
|
ctx: ExtensionContext,
|
|
@@ -103,7 +84,7 @@ function automaticGuidance(budget: Budget | undefined, enabled: boolean): string
|
|
|
103
84
|
budget === undefined
|
|
104
85
|
? ""
|
|
105
86
|
: ` The rollover line is ${budget.rolloverAt.toLocaleString("en-US")} tokens used under the available Pi settings.`;
|
|
106
|
-
return `Automatic rollover follows Pi's compaction setting.${line} At most one checkpoint reminder may arrive before the rollover line (${deadline}); when it does, stop normal work, ${CHECKPOINT_STEPS}.`;
|
|
87
|
+
return `Automatic rollover follows Pi's compaction setting.${line} At most one checkpoint reminder may arrive at the end of a working turn before the rollover line (${deadline}), never on a turn that ends the run; when it does, stop normal work, ${CHECKPOINT_STEPS}.`;
|
|
107
88
|
}
|
|
108
89
|
export function buildGuidance(ctx: PolicyContext): string {
|
|
109
90
|
const budget = budgetFor(ctx);
|
package/src/context-tools.ts
CHANGED
|
@@ -1,13 +1,7 @@
|
|
|
1
1
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
2
2
|
import { Type } from "typebox";
|
|
3
3
|
import { toolCards, type PosthorseDisplay } from "../ui.ts";
|
|
4
|
-
import {
|
|
5
|
-
budgetFor,
|
|
6
|
-
freshPayloadChars,
|
|
7
|
-
SNAPSHOT_NOTE,
|
|
8
|
-
unsupportedMessage,
|
|
9
|
-
type Budget,
|
|
10
|
-
} from "./budget.ts";
|
|
4
|
+
import { budgetFor, freshPayloadChars, unsupportedMessage, type Budget } from "./budget.ts";
|
|
11
5
|
import {
|
|
12
6
|
EMPTY_HANDOFF,
|
|
13
7
|
MAX_HANDOFF_CHARS,
|
|
@@ -38,7 +32,7 @@ function contextReport(host: PolicyContext): TextResult {
|
|
|
38
32
|
rollover,
|
|
39
33
|
rolloverAt: budget?.rolloverAt,
|
|
40
34
|
};
|
|
41
|
-
const configured = `≈${n(Math.max(0, usage.contextWindow - usage.tokens))} tokens until the configured context limit (${n(usage.tokens)}/${n(usage.contextWindow)} used, ${Math.round(usage.percent ?? 0)}%). Best available native estimate
|
|
35
|
+
const configured = `≈${n(Math.max(0, usage.contextWindow - usage.tokens))} tokens until the configured context limit (${n(usage.tokens)}/${n(usage.contextWindow)} used, ${Math.round(usage.percent ?? 0)}%). Best available native estimate.`;
|
|
42
36
|
if (budget?.enabled !== true) {
|
|
43
37
|
return textResult(
|
|
44
38
|
`Automatic rollover is disabled in the available settings (Pi compaction.enabled=false). ${configured}`,
|
|
@@ -100,7 +94,7 @@ export function registerContextTools(
|
|
|
100
94
|
label: "Context Remaining",
|
|
101
95
|
...toolCards("get_context_remaining"),
|
|
102
96
|
description:
|
|
103
|
-
"Estimate tokens left before Pi's automatic rollover and the configured context limit. Check before reporting remaining space or changing your work plan because of context limits. A checkpoint reminder
|
|
97
|
+
"Estimate tokens left before Pi's automatic rollover and the configured context limit. Check before reporting remaining space or changing your work plan because of context limits. A checkpoint reminder may arrive before rollover on a working turn; no routine checks are needed.",
|
|
104
98
|
promptSnippet:
|
|
105
99
|
"verify remaining context before reporting a token count or changing plans because of context limits",
|
|
106
100
|
parameters: Type.Object({}),
|
package/src/recovery.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import {
|
|
2
2
|
AUTO_HANDOFF_PREFIX,
|
|
3
|
+
isRecord,
|
|
3
4
|
isReminderType,
|
|
4
5
|
type EntryLike,
|
|
5
6
|
type ProjectedEntry,
|
|
@@ -114,19 +115,13 @@ function priorCheckpoint(entry: EntryLike | undefined, limit: number): string |
|
|
|
114
115
|
}
|
|
115
116
|
const id = entry.id?.slice(0, 120) ?? "unknown";
|
|
116
117
|
const header = `[older checkpoint; possibly stale | context-window entry ${id}]`;
|
|
117
|
-
return boundedBlock(
|
|
118
|
-
header,
|
|
119
|
-
handoff.startsWith(AUTO_HANDOFF_PREFIX)
|
|
120
|
-
? `Prior automatic recovery text is not nested here. Use history read with entry ${id} if needed.`
|
|
121
|
-
: handoff,
|
|
122
|
-
limit,
|
|
123
|
-
);
|
|
118
|
+
return boundedBlock(header, handoff, limit);
|
|
124
119
|
}
|
|
125
120
|
function projectedCurrent(
|
|
126
121
|
entries: readonly EntryLike[],
|
|
127
122
|
projected: readonly ProjectedEntry[],
|
|
123
|
+
current: readonly EntryLike[] = entries.slice(contextStart(entries).start),
|
|
128
124
|
): EntryLike[] {
|
|
129
|
-
const current = entries.slice(contextStart(entries).start);
|
|
130
125
|
const edits = contextEdits(entries);
|
|
131
126
|
const omittedIds = new Set(
|
|
132
127
|
projected
|
|
@@ -198,6 +193,72 @@ function anchors(records: readonly RecoveryRecord[]): {
|
|
|
198
193
|
others: records.filter((record) => !selected.has(record)),
|
|
199
194
|
};
|
|
200
195
|
}
|
|
196
|
+
function automaticBoundary(entry: EntryLike | undefined): boolean {
|
|
197
|
+
if (
|
|
198
|
+
entry === undefined ||
|
|
199
|
+
windowHandoff(entry)?.trim().startsWith(AUTO_HANDOFF_PREFIX) !== true
|
|
200
|
+
) {
|
|
201
|
+
return false;
|
|
202
|
+
}
|
|
203
|
+
return (
|
|
204
|
+
entry.type === "context_window" ||
|
|
205
|
+
(isRecord(entry.details) &&
|
|
206
|
+
entry.details.posthorse === 1 &&
|
|
207
|
+
(entry.details.reason === "threshold" || entry.details.reason === "overflow"))
|
|
208
|
+
);
|
|
209
|
+
}
|
|
210
|
+
function recoveryAncestry(
|
|
211
|
+
entries: readonly EntryLike[],
|
|
212
|
+
projected: readonly ProjectedEntry[],
|
|
213
|
+
ownerQuestionRegistered: boolean,
|
|
214
|
+
): {
|
|
215
|
+
readonly checkpoint?: EntryLike;
|
|
216
|
+
readonly carried: readonly RecoveryRecord[];
|
|
217
|
+
readonly skipped: readonly string[];
|
|
218
|
+
} {
|
|
219
|
+
let prefix = entries;
|
|
220
|
+
let boundary = contextStart(prefix).boundary;
|
|
221
|
+
const skipped: string[] = [];
|
|
222
|
+
while (automaticBoundary(boundary) && boundary !== undefined) {
|
|
223
|
+
skipped.push(boundary.id?.slice(0, 120) ?? "unknown");
|
|
224
|
+
prefix = prefix.slice(0, prefix.indexOf(boundary));
|
|
225
|
+
const window = contextStart(prefix);
|
|
226
|
+
const restored = projectedCurrent(entries, projected, prefix.slice(window.start));
|
|
227
|
+
const historical = restored.map((sourceEntry) => ({
|
|
228
|
+
sourceEntry,
|
|
229
|
+
messages: sourceEntry.message === undefined ? [] : [sourceEntry.message],
|
|
230
|
+
}));
|
|
231
|
+
const owners = ownerRecords(restored, historical, ownerQuestionRegistered).filter(
|
|
232
|
+
(record) => record.kind === "owner",
|
|
233
|
+
);
|
|
234
|
+
if (owners.length > 0) {
|
|
235
|
+
return {
|
|
236
|
+
checkpoint: automaticBoundary(window.boundary) ? undefined : window.boundary,
|
|
237
|
+
carried: anchors(owners).fixed,
|
|
238
|
+
skipped,
|
|
239
|
+
};
|
|
240
|
+
}
|
|
241
|
+
boundary = window.boundary;
|
|
242
|
+
}
|
|
243
|
+
return { checkpoint: boundary, carried: [], skipped };
|
|
244
|
+
}
|
|
245
|
+
function skippedRecords(ids: readonly string[], limit: number): string | undefined {
|
|
246
|
+
if (ids.length === 0) {
|
|
247
|
+
return undefined;
|
|
248
|
+
}
|
|
249
|
+
let names = ids[0];
|
|
250
|
+
let count = 1;
|
|
251
|
+
while (count < ids.length) {
|
|
252
|
+
const next = `, ${ids[count]}`;
|
|
253
|
+
if (names.length + next.length > limit) {
|
|
254
|
+
break;
|
|
255
|
+
}
|
|
256
|
+
names += next;
|
|
257
|
+
count++;
|
|
258
|
+
}
|
|
259
|
+
const omitted = ids.length - count;
|
|
260
|
+
return `Earlier automatic records skipped (not nested; use history read): ${names}${omitted > 0 ? `; ${omitted} earlier record(s) not listed.` : ""}`;
|
|
261
|
+
}
|
|
201
262
|
function projectedToolEntries(
|
|
202
263
|
entries: readonly EntryLike[],
|
|
203
264
|
projected: readonly ProjectedEntry[],
|
|
@@ -324,25 +385,35 @@ export function buildAutoHandoff(
|
|
|
324
385
|
maxChars: number,
|
|
325
386
|
ownerQuestionRegistered: boolean,
|
|
326
387
|
): string {
|
|
327
|
-
const
|
|
388
|
+
const history = recoveryAncestry(entries, projected, ownerQuestionRegistered);
|
|
389
|
+
const priorWindow = history.checkpoint;
|
|
328
390
|
const records = ownerRecords(
|
|
329
391
|
projectedCurrent(entries, projected),
|
|
330
392
|
projected,
|
|
331
393
|
ownerQuestionRegistered,
|
|
332
394
|
);
|
|
333
395
|
const { fixed, others } = anchors(records);
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
396
|
+
if (!records.some((record) => record.kind === "owner")) {
|
|
397
|
+
fixed.unshift(
|
|
398
|
+
...history.carried.map((record) => ({
|
|
399
|
+
...record,
|
|
400
|
+
label: `${record.label}; carried from an earlier window, possibly already handled; not current progress`,
|
|
401
|
+
})),
|
|
402
|
+
);
|
|
403
|
+
}
|
|
404
|
+
const inputHeader =
|
|
405
|
+
fixed.length > 0
|
|
406
|
+
? "Selected owner and coordination inputs (chronological):"
|
|
337
407
|
: "No selected current-window owner or visible coordination inputs were found.";
|
|
338
408
|
const batch = recoverableToolResults(projectedToolEntries(entries, projected));
|
|
339
409
|
const header = batchHeader(batch);
|
|
340
410
|
const minimum = [
|
|
341
411
|
PREAMBLE,
|
|
342
|
-
|
|
412
|
+
inputHeader,
|
|
343
413
|
...fixed.map((record) => formatRecord(record, 0)),
|
|
344
414
|
header,
|
|
345
415
|
priorCheckpoint(priorWindow, 0),
|
|
416
|
+
skippedRecords(history.skipped, 0),
|
|
346
417
|
];
|
|
347
418
|
const plan = planBatch(
|
|
348
419
|
batch,
|
|
@@ -350,19 +421,25 @@ export function buildAutoHandoff(
|
|
|
350
421
|
Math.max(0, maxChars - joined(minimum).length - HANDOFF_OVERHEAD_RESERVE),
|
|
351
422
|
);
|
|
352
423
|
const handoff = windowHandoff(priorWindow)?.trim();
|
|
353
|
-
const count =
|
|
424
|
+
const count =
|
|
425
|
+
fixed.length +
|
|
426
|
+
(handoff === undefined || handoff === "" ? 0 : 1) +
|
|
427
|
+
(history.skipped.length > 0 ? 1 : 0);
|
|
354
428
|
const fixedBudget = Math.max(
|
|
355
429
|
0,
|
|
356
|
-
maxChars - joined([PREAMBLE,
|
|
430
|
+
maxChars - joined([PREAMBLE, inputHeader, plan.bare]).length - HANDOFF_OVERHEAD_RESERVE,
|
|
357
431
|
);
|
|
358
432
|
const fixedLimit =
|
|
359
433
|
count > 0
|
|
360
434
|
? Math.min(MAX_RECOVERY_RECORD_CHARS, Math.floor(fixedBudget / count))
|
|
361
435
|
: MAX_RECOVERY_RECORD_CHARS;
|
|
362
|
-
const prior =
|
|
436
|
+
const prior = joined([
|
|
437
|
+
priorCheckpoint(priorWindow, fixedLimit),
|
|
438
|
+
skippedRecords(history.skipped, fixedLimit),
|
|
439
|
+
]);
|
|
363
440
|
const fixedParts = [
|
|
364
441
|
PREAMBLE,
|
|
365
|
-
|
|
442
|
+
inputHeader,
|
|
366
443
|
...fixed.map((record) => formatRecord(record, fixedLimit)),
|
|
367
444
|
];
|
|
368
445
|
const renderedBatch = renderBatch(plan, fixedParts, prior, maxChars);
|
package/src/rollover.ts
CHANGED
|
@@ -18,6 +18,7 @@ import {
|
|
|
18
18
|
REMINDER_TYPE,
|
|
19
19
|
type CompactionPolicy,
|
|
20
20
|
type PolicyContext,
|
|
21
|
+
type ProjectedEntry,
|
|
21
22
|
type ReminderFingerprint,
|
|
22
23
|
} from "./contracts.ts";
|
|
23
24
|
import { registerContextTools } from "./context-tools.ts";
|
|
@@ -40,6 +41,10 @@ function registerGuidance(pi: ExtensionAPI, policy: SnapshotReader): void {
|
|
|
40
41
|
pi.on("before_agent_start", (event) => {
|
|
41
42
|
promptOptions = event.systemPromptOptions;
|
|
42
43
|
});
|
|
44
|
+
pi.on("agent_settled", () => {
|
|
45
|
+
// Older hosts skip the start hook for custom runs; do not reuse a settled run's options.
|
|
46
|
+
promptOptions = undefined;
|
|
47
|
+
});
|
|
43
48
|
pi.on("context_with_system", (event, ctx) => {
|
|
44
49
|
const guidance = buildGuidance(policy(ctx));
|
|
45
50
|
const system =
|
|
@@ -104,6 +109,18 @@ function explicitReset(
|
|
|
104
109
|
continue: true,
|
|
105
110
|
};
|
|
106
111
|
}
|
|
112
|
+
function endsRun(
|
|
113
|
+
event: TurnEndEvent,
|
|
114
|
+
ctx: { readonly hasPendingMessages?: () => boolean },
|
|
115
|
+
): boolean {
|
|
116
|
+
return (
|
|
117
|
+
event.message.role === "assistant" &&
|
|
118
|
+
!event.message.content.some((part) => part.type === "toolCall") &&
|
|
119
|
+
!event.continue &&
|
|
120
|
+
event.context.pendingMessages.length === 0 &&
|
|
121
|
+
ctx.hasPendingMessages?.() !== true
|
|
122
|
+
);
|
|
123
|
+
}
|
|
107
124
|
function reminderBudget(
|
|
108
125
|
host: PolicyContext,
|
|
109
126
|
): { readonly budget: Budget; readonly tokens: number } | undefined {
|
|
@@ -126,7 +143,15 @@ function reminderBudget(
|
|
|
126
143
|
function sendReminder(
|
|
127
144
|
pi: ExtensionAPI,
|
|
128
145
|
ctx: ExtensionContext,
|
|
129
|
-
available: {
|
|
146
|
+
available: {
|
|
147
|
+
readonly budget: Budget;
|
|
148
|
+
readonly tokens: number;
|
|
149
|
+
readonly pendingMessages: readonly {
|
|
150
|
+
readonly role: string;
|
|
151
|
+
readonly customType?: string;
|
|
152
|
+
readonly details?: unknown;
|
|
153
|
+
}[];
|
|
154
|
+
},
|
|
130
155
|
reminders: ReminderLookup,
|
|
131
156
|
): void {
|
|
132
157
|
const { budget, tokens } = available;
|
|
@@ -136,7 +161,15 @@ function sendReminder(
|
|
|
136
161
|
contextWindow: budget.contextWindow,
|
|
137
162
|
reserveTokens: budget.reserveTokens,
|
|
138
163
|
};
|
|
139
|
-
if (
|
|
164
|
+
if (
|
|
165
|
+
hasReminder(state.reminders, fingerprint) ||
|
|
166
|
+
available.pendingMessages.some(
|
|
167
|
+
(message) =>
|
|
168
|
+
message.role === "custom" &&
|
|
169
|
+
isReminderType(message.customType) &&
|
|
170
|
+
reminderMatches(message.details, fingerprint),
|
|
171
|
+
)
|
|
172
|
+
) {
|
|
140
173
|
return;
|
|
141
174
|
}
|
|
142
175
|
pi.sendMessage(
|
|
@@ -225,6 +258,92 @@ function reportAutomaticFailure(ctx: ExtensionContext, error: unknown): void {
|
|
|
225
258
|
console.error(message);
|
|
226
259
|
}
|
|
227
260
|
}
|
|
261
|
+
function freshWindowWithoutInput(projected: readonly ProjectedEntry[]): boolean {
|
|
262
|
+
const boundary = projected.findIndex(
|
|
263
|
+
(entry) => entry.sourceEntry.type === "compaction" && entry.messages.length > 0,
|
|
264
|
+
);
|
|
265
|
+
if (boundary < 0) {
|
|
266
|
+
return false;
|
|
267
|
+
}
|
|
268
|
+
const turnStarts = new Set([
|
|
269
|
+
"user",
|
|
270
|
+
"custom",
|
|
271
|
+
"bashExecution",
|
|
272
|
+
"branchSummary",
|
|
273
|
+
"compactionSummary",
|
|
274
|
+
]);
|
|
275
|
+
return !projected
|
|
276
|
+
.slice(boundary + 1)
|
|
277
|
+
.some(
|
|
278
|
+
(entry) =>
|
|
279
|
+
entry.sourceEntry.type !== "compaction" &&
|
|
280
|
+
entry.messages.some((message) => turnStarts.has(message.role ?? "")),
|
|
281
|
+
);
|
|
282
|
+
}
|
|
283
|
+
function reachedThreshold(host: PolicyContext): boolean {
|
|
284
|
+
const usage = host.getContextUsage();
|
|
285
|
+
const budget = budgetFor(host);
|
|
286
|
+
return (
|
|
287
|
+
budget?.supported === true &&
|
|
288
|
+
budget.enabled &&
|
|
289
|
+
usage !== undefined &&
|
|
290
|
+
usage.tokens !== null &&
|
|
291
|
+
usage.tokens >= budget.rolloverAt
|
|
292
|
+
);
|
|
293
|
+
}
|
|
294
|
+
function fallbackTurnEligible(event: TurnEndEvent, ctx: ExtensionContext): boolean {
|
|
295
|
+
return (
|
|
296
|
+
event.outcome === "completed" &&
|
|
297
|
+
!(event.message.role === "assistant" && event.message.stopReason === "length") &&
|
|
298
|
+
ctx.signal?.aborted !== true &&
|
|
299
|
+
!event.entries.some((entry) => entry.type === "compaction" || entry.type === "context_edit")
|
|
300
|
+
);
|
|
301
|
+
}
|
|
302
|
+
function thresholdFallback(
|
|
303
|
+
pi: ExtensionAPI,
|
|
304
|
+
event: TurnEndEvent,
|
|
305
|
+
ctx: ExtensionContext,
|
|
306
|
+
capacity: { readonly host: PolicyContext; readonly toolTokens: () => number },
|
|
307
|
+
): BoundaryResult | undefined {
|
|
308
|
+
if (!fallbackTurnEligible(event, ctx)) {
|
|
309
|
+
return undefined;
|
|
310
|
+
}
|
|
311
|
+
const { host, toolTokens } = capacity;
|
|
312
|
+
if (!reachedThreshold(host)) {
|
|
313
|
+
return undefined;
|
|
314
|
+
}
|
|
315
|
+
if (!freshWindowWithoutInput(event.context.contextEntries)) {
|
|
316
|
+
return undefined;
|
|
317
|
+
}
|
|
318
|
+
const projected = event.context.contextEntries;
|
|
319
|
+
const limit = freshPayloadChars(host, toolTokens(), event.context.pendingMessages);
|
|
320
|
+
const ownerQuestionRegistered = pi
|
|
321
|
+
.getAllTools()
|
|
322
|
+
.some((tool) => tool.name === "ask_question" && tool.namespace === undefined);
|
|
323
|
+
const summary = buildAutoHandoff(
|
|
324
|
+
ctx.sessionManager.getBranch(),
|
|
325
|
+
projected,
|
|
326
|
+
limit,
|
|
327
|
+
ownerQuestionRegistered,
|
|
328
|
+
);
|
|
329
|
+
if (limit < MIN_PAGE_CHARS || summary.length > limit) {
|
|
330
|
+
return undefined;
|
|
331
|
+
}
|
|
332
|
+
// ponytail: turn_end drafts are eager. Later projection editors must replace/remove this draft;
|
|
333
|
+
// upgrade to a host post-boundary hook if recovery must include later handler edits.
|
|
334
|
+
return {
|
|
335
|
+
entries: [
|
|
336
|
+
...event.entries,
|
|
337
|
+
{
|
|
338
|
+
type: "compaction",
|
|
339
|
+
summary,
|
|
340
|
+
firstKeptEntryId: null,
|
|
341
|
+
details: { posthorse: 1, reason: "threshold" },
|
|
342
|
+
},
|
|
343
|
+
],
|
|
344
|
+
continue: event.continue,
|
|
345
|
+
};
|
|
346
|
+
}
|
|
228
347
|
function registerAutomaticRollover(
|
|
229
348
|
pi: ExtensionAPI,
|
|
230
349
|
policy: SnapshotReader,
|
|
@@ -318,9 +437,24 @@ export function registerRollover(
|
|
|
318
437
|
freshPayloadChars(host, activeToolTokens(), event.context.pendingMessages),
|
|
319
438
|
);
|
|
320
439
|
}
|
|
440
|
+
const fallback = thresholdFallback(pi, event, ctx, {
|
|
441
|
+
host,
|
|
442
|
+
toolTokens: activeToolTokens,
|
|
443
|
+
});
|
|
444
|
+
if (fallback !== undefined) {
|
|
445
|
+
return fallback;
|
|
446
|
+
}
|
|
447
|
+
if (endsRun(event, ctx)) {
|
|
448
|
+
return;
|
|
449
|
+
}
|
|
321
450
|
const available = reminderBudget(host);
|
|
322
451
|
if (available !== undefined) {
|
|
323
|
-
sendReminder(
|
|
452
|
+
sendReminder(
|
|
453
|
+
pi,
|
|
454
|
+
ctx,
|
|
455
|
+
{ ...available, pendingMessages: event.context.pendingMessages },
|
|
456
|
+
reminders,
|
|
457
|
+
);
|
|
324
458
|
}
|
|
325
459
|
return;
|
|
326
460
|
});
|