@adeildo/pi-compact 5.7.1 → 5.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -7
- package/package.json +2 -2
- package/src/index.ts +84 -11
- package/src/prune.ts +83 -31
- package/src/record.ts +40 -6
- package/src/server.ts +45 -0
- package/src/settings.ts +14 -2
- package/src/summary.ts +1 -1
package/README.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
|
|
8
8
|
<p align="center">
|
|
9
9
|
<strong>Keeps a long session's context small, so each request costs less.</strong><br>
|
|
10
|
-
|
|
10
|
+
It hides old tool output and keeps the agent's words, your words and the decisions.
|
|
11
11
|
</p>
|
|
12
12
|
|
|
13
13
|
<p align="center"><code>pi install npm:@adeildo/pi-compact</code></p>
|
|
@@ -22,7 +22,9 @@ Output hidden to save context, about 4.2k tokens. It came from: read src/view.ts
|
|
|
22
22
|
|
|
23
23
|
Long arguments go the same way, like a script sent to bash or a file sent to write. Your messages, the agent's own words, errors and the last 10 turns stay whole. After a prune the agent gets the session's [tasks](../tasks) at the end of its context, so it keeps its goals.
|
|
24
24
|
|
|
25
|
-
|
|
25
|
+
On Anthropic's own models, Anthropic clears the old output on its side instead, with [`clear_tool_uses`](https://platform.claude.com/docs/en/build-with-claude/context-editing). Nothing edits the session, so the agent keeps its earlier thinking. Anthropic signs each thinking block with the conversation before it, and an edit drops the blocks of every later turn. The last 10 calls stay whole, and older ones keep their parameters, so the agent still sees what it read and ran. In a live test that cleared the parameters too, the agent took its own earlier reports for invented ones.
|
|
26
|
+
|
|
27
|
+
Each prune or clear leaves a line in the chat, with what rewriting the prompt cache costs and when that pays back. `Ctrl+O` lists the calls it hid.
|
|
26
28
|
|
|
27
29
|
<!-- docs:compact/prune -->
|
|
28
30
|
|
|
@@ -36,30 +38,31 @@ Each prune leaves a line in the chat, with what the next request pays to rewrite
|
|
|
36
38
|
|
|
37
39
|
When pi compacts anyway, pi's own prompt writes the summary on a small model, and the tasks and your last words go on top as they are, so no model rephrases a decision. Without tasks, pi writes the summary on the session's model.
|
|
38
40
|
|
|
39
|
-
On six recorded sessions, an agent that only had the summary answered 48 questions about what was dropped. A judge graded the answers against the truth
|
|
41
|
+
On six recorded sessions, an agent that only had the summary answered 48 questions about what was dropped. A judge graded the answers against the truth. pi's summary on Haiku scored as well as on Opus, within the judge's noise, for 1/40 of the price. A first version that wrote its own short summary from the tasks scored 2.6 of 5 against pi's 3.9, and was dropped. The method is [Factory's probes](https://factory.ai/news/evaluating-compression), and `bun packages/compact/eval/summary.ts` runs it.
|
|
40
42
|
|
|
41
43
|
## Why this way
|
|
42
44
|
|
|
43
45
|
- Hiding old tool output works as well as summarizing it and costs less. Summaries made agents run 13 to 15% longer ([JetBrains, NeurIPS 2025](https://arxiv.org/abs/2508.21433)).
|
|
44
46
|
- A model that picks what to keep cost 94% more with no gain in quality ([OpenHands, 2026](https://arxiv.org/abs/2605.18854)). The rules here are plain code.
|
|
45
|
-
-
|
|
47
|
+
- After a prune, the next request writes the whole conversation to the prompt cache again. Anthropic reuses cache only where an earlier request wrote it ([Anthropic](https://platform.claude.com/docs/en/build-with-claude/context-editing)), and the writes near old outputs have expired. So a prune waits until it frees at least two fifths of the context at once. Pruning a little every turn would cost 3.4 times as much.
|
|
46
48
|
- Errors stay and every placeholder keeps its call, after [Manus](https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus).
|
|
47
49
|
|
|
48
|
-
Replaying a week of real sessions
|
|
50
|
+
Replaying a week of real sessions, mostly Opus with a 1M context, costs 32% less with the prune and 27% less with Anthropic's clearing, which keeps the parameters and the thinking. The replay has each prune's next request write all of the context again, so the real saving is at least that. `bun packages/compact/eval/replay.ts [--server 10]` replays yours.
|
|
49
51
|
|
|
50
52
|
## With pi's compaction
|
|
51
53
|
|
|
52
|
-
|
|
54
|
+
The package turns off nothing of pi's. The prune runs at the end of each turn, right before pi checks its own compaction threshold, so pi rarely needs to compact. `/compact` and auto-compaction still work; this package only writes their summary. The prune is a `context_edit`, so the session file keeps every output, and `/tree` to an earlier point shows them again. Anthropic's clearing edits nothing in the session.
|
|
53
55
|
|
|
54
56
|
## Commands and settings
|
|
55
57
|
|
|
56
|
-
`/prune` hides old output at the end of the next turn, whatever the context's size.
|
|
58
|
+
`/prune` hides old output at the end of the next turn, whatever the context's size. It edits the context on Anthropic models too, so it drops the thinking there.
|
|
57
59
|
|
|
58
60
|
On the Compact tab of `Alt+S`, stored under `compact` in `~/.pi/agent/extensions/pi-harness/settings.json`:
|
|
59
61
|
|
|
60
62
|
| Setting | Default | Does |
|
|
61
63
|
| ----------------------- | -------- | ------------------------------------------------------------------------------------------------------ |
|
|
62
64
|
| Hide old tool output | on | The prune |
|
|
65
|
+
| On Anthropic's side | on | On Anthropic models, Anthropic clears instead of the prune, and the agent keeps its thinking |
|
|
63
66
|
| Above | `150000` | Context tokens that start a prune |
|
|
64
67
|
| Write it from the tasks | on | The tasks on top of pi's summary, written on the model below. Off, pi writes it on the session's model |
|
|
65
68
|
| Model | `auto` | Writes the summary. `auto` is the cheapest of the session's provider |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@adeildo/pi-compact",
|
|
3
|
-
"version": "5.
|
|
3
|
+
"version": "5.8.0",
|
|
4
4
|
"description": "Keeps a long pi session's context small: hides old tool output, keeping each call so the agent can run it again, and writes pi's compaction summary from the session's tasks.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"compaction",
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
"access": "public"
|
|
38
38
|
},
|
|
39
39
|
"dependencies": {
|
|
40
|
-
"@adeildo/pi-kit": "5.
|
|
40
|
+
"@adeildo/pi-kit": "5.8.0"
|
|
41
41
|
},
|
|
42
42
|
"peerDependencies": {
|
|
43
43
|
"@earendil-works/pi-ai": "*",
|
package/src/index.ts
CHANGED
|
@@ -13,11 +13,16 @@ import {
|
|
|
13
13
|
estimateTokens,
|
|
14
14
|
type ExtensionAPI,
|
|
15
15
|
type ExtensionContext,
|
|
16
|
+
type SessionBoundaryDraft,
|
|
17
|
+
type TurnEndEvent,
|
|
16
18
|
} from "@earendil-works/pi-coding-agent";
|
|
17
19
|
|
|
18
|
-
import { describeCall, KEEP_TURNS,
|
|
20
|
+
import { describeCall, KEEP_TURNS, planServerClear, pruneNow, sizeAfter } from "./prune.ts";
|
|
19
21
|
import {
|
|
22
|
+
type Billed,
|
|
23
|
+
clearedEntries,
|
|
20
24
|
hiddenCalls,
|
|
25
|
+
lastPromptOf,
|
|
21
26
|
NOTE_MESSAGE,
|
|
22
27
|
PRUNE_ENTRY,
|
|
23
28
|
priced,
|
|
@@ -29,9 +34,11 @@ import {
|
|
|
29
34
|
spentOn,
|
|
30
35
|
tokens,
|
|
31
36
|
} from "./record.ts";
|
|
37
|
+
import { clearsOnServer, KEEP_TOOL_USES, serverCleared, withServerClearing } from "./server.ts";
|
|
32
38
|
import {
|
|
33
39
|
COMPACT_SETTINGS,
|
|
34
40
|
model,
|
|
41
|
+
onServer,
|
|
35
42
|
prune,
|
|
36
43
|
PRUNE_SECTION,
|
|
37
44
|
pruneAbove,
|
|
@@ -50,12 +57,27 @@ export const compact = defineFeature({
|
|
|
50
57
|
setup(scope) {
|
|
51
58
|
let forced = false;
|
|
52
59
|
let hidden = new Set<string>();
|
|
60
|
+
let cleared = new Set<string>();
|
|
61
|
+
let lastPrompt: number | undefined;
|
|
53
62
|
|
|
54
63
|
const restore = (ctx: ExtensionContext) => {
|
|
55
|
-
|
|
64
|
+
const branch = ctx.sessionManager.getBranch();
|
|
65
|
+
hidden = hiddenCalls(branch);
|
|
66
|
+
cleared = clearedEntries(branch);
|
|
67
|
+
lastPrompt = lastPromptOf(branch);
|
|
56
68
|
};
|
|
57
69
|
scope.onSessionStart(restore);
|
|
58
70
|
scope.on("session_tree", (_event, ctx) => restore(ctx));
|
|
71
|
+
scope.on("session_compact", () => {
|
|
72
|
+
lastPrompt = undefined;
|
|
73
|
+
});
|
|
74
|
+
|
|
75
|
+
const onServerFor = (ctx: ExtensionContext) =>
|
|
76
|
+
prune.get(scope) && onServer.get(scope) && clearsOnServer(ctx.model);
|
|
77
|
+
|
|
78
|
+
scope.on("before_provider_request", (event, ctx) =>
|
|
79
|
+
onServerFor(ctx) ? withServerClearing(event.payload, pruneAbove.get(scope)) : undefined,
|
|
80
|
+
);
|
|
59
81
|
|
|
60
82
|
// Pi checks its own compaction right after turn_end, so pruning here usually spares it.
|
|
61
83
|
scope.on("turn_end", (event, ctx) => {
|
|
@@ -68,8 +90,18 @@ export const compact = defineFeature({
|
|
|
68
90
|
)
|
|
69
91
|
return undefined;
|
|
70
92
|
const { input, cacheRead, cacheWrite } = message.usage;
|
|
93
|
+
const prompt = input + cacheRead + cacheWrite;
|
|
94
|
+
const previous = lastPrompt;
|
|
95
|
+
lastPrompt = prompt;
|
|
96
|
+
if (!forced && onServerFor(ctx)) {
|
|
97
|
+
if (previous === undefined || !serverCleared(previous, prompt, pruneAbove.get(scope)))
|
|
98
|
+
return undefined;
|
|
99
|
+
return {
|
|
100
|
+
entries: [...event.entries, ...clearedOnServer(event, ctx, previous, message.usage)],
|
|
101
|
+
};
|
|
102
|
+
}
|
|
71
103
|
const outputs = event.toolResults.reduce((sum, result) => sum + estimateTokens(result), 0);
|
|
72
|
-
const before =
|
|
104
|
+
const before = prompt + outputs;
|
|
73
105
|
const asked = forced;
|
|
74
106
|
forced = false;
|
|
75
107
|
const plan = pruneNow(event.context.contextEntries, before, pruneAbove.get(scope), asked);
|
|
@@ -86,7 +118,8 @@ export const compact = defineFeature({
|
|
|
86
118
|
message.usage,
|
|
87
119
|
ctx.model?.cost.cacheRead ?? 0,
|
|
88
120
|
before - after,
|
|
89
|
-
|
|
121
|
+
// Only the system prompt stays cached, so the next request writes all of this again.
|
|
122
|
+
after,
|
|
90
123
|
);
|
|
91
124
|
const record: PruneRecord = {
|
|
92
125
|
before,
|
|
@@ -96,13 +129,13 @@ export const compact = defineFeature({
|
|
|
96
129
|
...cost,
|
|
97
130
|
};
|
|
98
131
|
for (const { call } of plan.hidden) hidden.add(call);
|
|
99
|
-
|
|
100
|
-
|
|
132
|
+
// The next request is smaller for the edit, not for a server clear.
|
|
133
|
+
lastPrompt = undefined;
|
|
101
134
|
return {
|
|
102
135
|
entries: [
|
|
103
136
|
...event.entries,
|
|
104
137
|
...plan.edits,
|
|
105
|
-
note(
|
|
138
|
+
note(tasksNow(ctx), false),
|
|
106
139
|
{ type: "custom", customType: PRUNE_ENTRY, data: record },
|
|
107
140
|
],
|
|
108
141
|
};
|
|
@@ -132,7 +165,12 @@ export const compact = defineFeature({
|
|
|
132
165
|
description: "Hide old tool output at the end of the next turn, whatever the context's size",
|
|
133
166
|
handler: async (_args, ctx) => {
|
|
134
167
|
forced = true;
|
|
135
|
-
ctx.ui.notify(
|
|
168
|
+
ctx.ui.notify(
|
|
169
|
+
onServerFor(ctx)
|
|
170
|
+
? "Old tool output goes at the end of the next turn. On Anthropic models this edit drops the agent's earlier thinking, which Anthropic's own clearing keeps."
|
|
171
|
+
: "Old tool output goes at the end of the next turn.",
|
|
172
|
+
"info",
|
|
173
|
+
);
|
|
136
174
|
},
|
|
137
175
|
});
|
|
138
176
|
|
|
@@ -142,13 +180,48 @@ export const compact = defineFeature({
|
|
|
142
180
|
});
|
|
143
181
|
|
|
144
182
|
registerScreen(scope);
|
|
183
|
+
|
|
184
|
+
/** Records what Anthropic cleared on the request that just answered, and recites the tasks. */
|
|
185
|
+
function clearedOnServer(
|
|
186
|
+
event: TurnEndEvent,
|
|
187
|
+
ctx: ExtensionContext,
|
|
188
|
+
previous: number,
|
|
189
|
+
usage: Billed,
|
|
190
|
+
): SessionBoundaryDraft[] {
|
|
191
|
+
const prompt = usage.input + usage.cacheRead + usage.cacheWrite;
|
|
192
|
+
const entries = event.context.contextEntries;
|
|
193
|
+
const end = entries.findIndex((item) => item.sourceEntry.id === event.messageEntryId);
|
|
194
|
+
const plan = planServerClear(
|
|
195
|
+
end === -1 ? entries : entries.slice(0, end),
|
|
196
|
+
KEEP_TOOL_USES,
|
|
197
|
+
cleared,
|
|
198
|
+
);
|
|
199
|
+
for (const { entry, call } of plan.hidden) {
|
|
200
|
+
cleared.add(entry);
|
|
201
|
+
hidden.add(call);
|
|
202
|
+
}
|
|
203
|
+
const record: PruneRecord = {
|
|
204
|
+
before: previous,
|
|
205
|
+
after: prompt,
|
|
206
|
+
hidden: plan.hidden,
|
|
207
|
+
server: true,
|
|
208
|
+
...priced(usage, ctx.model?.cost.cacheRead ?? 0, previous - prompt, usage.cacheWrite),
|
|
209
|
+
};
|
|
210
|
+
return [note(tasksNow(ctx), true), { type: "custom", customType: PRUNE_ENTRY, data: record }];
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
function tasksNow(ctx: ExtensionContext): SessionTasks {
|
|
214
|
+
return currentTasks(scope.events)?.state ?? tasksFromEntries(ctx.sessionManager.getBranch());
|
|
215
|
+
}
|
|
145
216
|
},
|
|
146
217
|
});
|
|
147
218
|
|
|
148
|
-
// Recites the tasks after a prune, so the agent keeps its goals.
|
|
149
|
-
function note(tasks: SessionTasks): CustomMessageEntryDraft {
|
|
219
|
+
// Recites the tasks after a prune or a clear, so the agent keeps its goals.
|
|
220
|
+
function note(tasks: SessionTasks, server: boolean): CustomMessageEntryDraft {
|
|
150
221
|
const parts = [
|
|
151
|
-
|
|
222
|
+
server
|
|
223
|
+
? "Anthropic cleared older tool output to save context. The calls are still there: run one again if you need its output."
|
|
224
|
+
: "Older tool output was hidden to save context. Each placeholder names the call it came from: run it again if you need it.",
|
|
152
225
|
];
|
|
153
226
|
const stands = describeTasks(tasks, { done: 10, decisions: 10 });
|
|
154
227
|
if (stands !== "") parts.push(`Where the session stands:\n${stands}`);
|
package/src/prune.ts
CHANGED
|
@@ -7,12 +7,13 @@ import {
|
|
|
7
7
|
// 10 turns did best on SWE-bench: arxiv.org/abs/2508.21433
|
|
8
8
|
export const KEEP_TURNS = 10;
|
|
9
9
|
// Each prune rewrites the prompt cache, so it waits to free this much at once.
|
|
10
|
-
export const MIN_FREED_SHARE = 0.
|
|
10
|
+
export const MIN_FREED_SHARE = 0.4;
|
|
11
11
|
const MIN_OUTPUT_TOKENS = 250;
|
|
12
12
|
const MIN_ARGUMENT_CHARS = 1500;
|
|
13
13
|
const LONG_STRING = 200;
|
|
14
14
|
const SHOWN_CHARS = 120;
|
|
15
|
-
const KEPT_TOOLS =
|
|
15
|
+
export const KEPT_TOOLS = ["ask_questions"];
|
|
16
|
+
const SERVER_PLACEHOLDER = "[cleared]";
|
|
16
17
|
|
|
17
18
|
export interface Hidden {
|
|
18
19
|
entry: string;
|
|
@@ -26,8 +27,6 @@ export interface PrunePlan {
|
|
|
26
27
|
/** Estimated, like `total`: compare them, never `freed` with tokens the API reported. */
|
|
27
28
|
freed: number;
|
|
28
29
|
total: number;
|
|
29
|
-
/** Estimated tokens before the first edit, which the prompt cache keeps. */
|
|
30
|
-
cachedPrefix: number;
|
|
31
30
|
}
|
|
32
31
|
|
|
33
32
|
export type AgentMessage = ProjectedSessionEntry["messages"][number];
|
|
@@ -43,37 +42,19 @@ interface Visible {
|
|
|
43
42
|
}
|
|
44
43
|
|
|
45
44
|
export function planPrune(entries: readonly ProjectedSessionEntry[]): PrunePlan {
|
|
46
|
-
const
|
|
47
|
-
const
|
|
48
|
-
sourceEntry.type === "message" && messages.length === 1 && messages[0] !== undefined
|
|
49
|
-
? [{ id: sourceEntry.id, message: messages[0] }]
|
|
50
|
-
: [],
|
|
51
|
-
);
|
|
52
|
-
plan.total = visible.reduce((sum, { message }) => sum + estimateTokens(message), 0);
|
|
45
|
+
const visible = visibleOf(entries);
|
|
46
|
+
const plan: PrunePlan = { edits: [], hidden: [], freed: 0, total: totalOf(visible) };
|
|
53
47
|
const turns = visible.flatMap((entry, index) =>
|
|
54
48
|
entry.message.role === "assistant" ? [index] : [],
|
|
55
49
|
);
|
|
56
50
|
const cutoff = turns.at(-KEEP_TURNS);
|
|
57
51
|
if (turns.length <= KEEP_TURNS || cutoff === undefined) return plan;
|
|
58
52
|
|
|
59
|
-
const calls =
|
|
60
|
-
visible.flatMap(({ message }) =>
|
|
61
|
-
message.role === "assistant"
|
|
62
|
-
? message.content.flatMap((part) =>
|
|
63
|
-
part.type === "toolCall" ? [[part.id, part] as const] : [],
|
|
64
|
-
)
|
|
65
|
-
: [],
|
|
66
|
-
),
|
|
67
|
-
);
|
|
53
|
+
const calls = callsOf(visible);
|
|
68
54
|
for (const { id, message } of visible.slice(0, cutoff)) {
|
|
69
55
|
if (message.role === "toolResult") hideOutput(plan, id, message, calls.get(message.toolCallId));
|
|
70
56
|
if (message.role === "assistant") cutArguments(plan, id, message);
|
|
71
57
|
}
|
|
72
|
-
const edited = new Set(plan.edits.map((edit) => edit.targetId));
|
|
73
|
-
const first = visible.findIndex(({ id }) => edited.has(id));
|
|
74
|
-
plan.cachedPrefix = visible
|
|
75
|
-
.slice(0, Math.max(0, first))
|
|
76
|
-
.reduce((sum, { message }) => sum + estimateTokens(message), 0);
|
|
77
58
|
return plan;
|
|
78
59
|
}
|
|
79
60
|
|
|
@@ -90,15 +71,86 @@ export function pruneNow(
|
|
|
90
71
|
return plan;
|
|
91
72
|
}
|
|
92
73
|
|
|
74
|
+
/**
|
|
75
|
+
* What Anthropic's `clear_tool_uses` takes out when it runs: every tool result but the last
|
|
76
|
+
* `keep`, errors too, leaving the calls. `cleared` are the entries it took out before.
|
|
77
|
+
*/
|
|
78
|
+
export function planServerClear(
|
|
79
|
+
entries: readonly ProjectedSessionEntry[],
|
|
80
|
+
keep: number,
|
|
81
|
+
cleared: ReadonlySet<string> = new Set(),
|
|
82
|
+
inputs = false,
|
|
83
|
+
): PrunePlan {
|
|
84
|
+
const visible = visibleOf(entries);
|
|
85
|
+
const plan: PrunePlan = { edits: [], hidden: [], freed: 0, total: totalOf(visible) };
|
|
86
|
+
const calls = callsOf(visible);
|
|
87
|
+
const results = visible.filter(
|
|
88
|
+
({ message }) => message.role === "toolResult" && !KEPT_TOOLS.includes(message.toolName),
|
|
89
|
+
);
|
|
90
|
+
for (const { id, message } of results.slice(0, Math.max(0, results.length - keep))) {
|
|
91
|
+
if (message.role !== "toolResult" || cleared.has(id)) continue;
|
|
92
|
+
const call = calls.get(message.toolCallId);
|
|
93
|
+
const tokens = estimateTokens(message);
|
|
94
|
+
const content = [{ type: "text" as const, text: SERVER_PLACEHOLDER }];
|
|
95
|
+
plan.edits.push({ type: "context_edit", targetId: id, replacement: { content } });
|
|
96
|
+
plan.hidden.push({
|
|
97
|
+
entry: id,
|
|
98
|
+
call: call === undefined ? message.toolName : describeCall(call.name, call.arguments),
|
|
99
|
+
tokens,
|
|
100
|
+
});
|
|
101
|
+
plan.freed += tokens - estimateTokens({ ...message, content });
|
|
102
|
+
// estimateTokens counts four characters a token.
|
|
103
|
+
if (inputs && call !== undefined) plan.freed += JSON.stringify(call.arguments).length / 4;
|
|
104
|
+
}
|
|
105
|
+
return plan;
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
/** The server clears only past `above`, and only when it frees `clearAtLeast` real tokens. */
|
|
109
|
+
export function serverClearNow(
|
|
110
|
+
entries: readonly ProjectedSessionEntry[],
|
|
111
|
+
contextTokens: number,
|
|
112
|
+
above: number,
|
|
113
|
+
keep: number,
|
|
114
|
+
cleared: ReadonlySet<string> = new Set(),
|
|
115
|
+
inputs = false,
|
|
116
|
+
): PrunePlan | undefined {
|
|
117
|
+
if (contextTokens < above) return undefined;
|
|
118
|
+
const plan = planServerClear(entries, keep, cleared, inputs);
|
|
119
|
+
if (plan.freed * scale(plan, contextTokens) < clearAtLeast(above)) return undefined;
|
|
120
|
+
return plan;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
export function clearAtLeast(above: number): number {
|
|
124
|
+
return Math.round(MIN_FREED_SHARE * above);
|
|
125
|
+
}
|
|
126
|
+
|
|
93
127
|
/** The context's size after `plan`, in the tokens the API reported for it before. */
|
|
94
128
|
export function sizeAfter(plan: PrunePlan, contextTokens: number): number {
|
|
95
129
|
return Math.round(contextTokens - plan.freed * scale(plan, contextTokens));
|
|
96
130
|
}
|
|
97
131
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
132
|
+
function visibleOf(entries: readonly ProjectedSessionEntry[]): Visible[] {
|
|
133
|
+
return entries.flatMap(({ sourceEntry, messages }): Visible[] =>
|
|
134
|
+
sourceEntry.type === "message" && messages.length === 1 && messages[0] !== undefined
|
|
135
|
+
? [{ id: sourceEntry.id, message: messages[0] }]
|
|
136
|
+
: [],
|
|
137
|
+
);
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
function totalOf(visible: readonly Visible[]): number {
|
|
141
|
+
return visible.reduce((sum, { message }) => sum + estimateTokens(message), 0);
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
function callsOf(visible: readonly Visible[]): Map<string, ToolCall> {
|
|
145
|
+
return new Map(
|
|
146
|
+
visible.flatMap(({ message }) =>
|
|
147
|
+
message.role === "assistant"
|
|
148
|
+
? message.content.flatMap((part) =>
|
|
149
|
+
part.type === "toolCall" ? [[part.id, part] as const] : [],
|
|
150
|
+
)
|
|
151
|
+
: [],
|
|
152
|
+
),
|
|
153
|
+
);
|
|
102
154
|
}
|
|
103
155
|
|
|
104
156
|
function scale(plan: PrunePlan, contextTokens: number): number {
|
|
@@ -112,7 +164,7 @@ function hideOutput(
|
|
|
112
164
|
call: ToolCall | undefined,
|
|
113
165
|
): void {
|
|
114
166
|
// Errors stay: without them the agent repeats what failed.
|
|
115
|
-
if (message.isError || KEPT_TOOLS.
|
|
167
|
+
if (message.isError || KEPT_TOOLS.includes(message.toolName)) return;
|
|
116
168
|
const tokens = estimateTokens(message);
|
|
117
169
|
if (tokens < MIN_OUTPUT_TOKENS) return;
|
|
118
170
|
const described = call === undefined ? message.toolName : describeCall(call.name, call.arguments);
|
|
@@ -171,7 +223,7 @@ function shortenedValue(value: JsonValue): JsonValue {
|
|
|
171
223
|
|
|
172
224
|
function clip(text: string): string {
|
|
173
225
|
const line = text.trim().split("\n")[0] ?? "";
|
|
174
|
-
return line.length <= SHOWN_CHARS ? line : `${line.slice(0, SHOWN_CHARS -
|
|
226
|
+
return line.length <= SHOWN_CHARS ? line : `${line.slice(0, SHOWN_CHARS - 3)}...`;
|
|
175
227
|
}
|
|
176
228
|
|
|
177
229
|
function rounded(tokens: number): string {
|
package/src/record.ts
CHANGED
|
@@ -17,7 +17,9 @@ export interface PruneRecord {
|
|
|
17
17
|
after: number;
|
|
18
18
|
hidden: Hidden[];
|
|
19
19
|
forced?: boolean;
|
|
20
|
-
/**
|
|
20
|
+
/** Anthropic cleared it on its side, on the request this was recorded after. */
|
|
21
|
+
server?: boolean;
|
|
22
|
+
/** Dollars the next request pays to write the cache again, or paid already when `server`. */
|
|
21
23
|
rewrite?: number;
|
|
22
24
|
/** Dollars each request after saves. */
|
|
23
25
|
saving?: number;
|
|
@@ -48,6 +50,7 @@ export function recordOf(entry: unknown): PruneRecord | undefined {
|
|
|
48
50
|
before: data.before,
|
|
49
51
|
after: data.after,
|
|
50
52
|
hidden: Array.isArray(data.hidden) ? (data.hidden as Hidden[]) : [],
|
|
53
|
+
...(data.server === true ? { server: true } : {}),
|
|
51
54
|
...(typeof data.rewrite === "number" ? { rewrite: data.rewrite } : {}),
|
|
52
55
|
...(typeof data.saving === "number" ? { saving: data.saving } : {}),
|
|
53
56
|
};
|
|
@@ -82,6 +85,34 @@ export function spentOn(branch: readonly unknown[]): Spent {
|
|
|
82
85
|
return spent;
|
|
83
86
|
}
|
|
84
87
|
|
|
88
|
+
/** Entries Anthropic cleared on its side, which the session itself still holds whole. */
|
|
89
|
+
export function clearedEntries(branch: readonly unknown[]): Set<string> {
|
|
90
|
+
const entries = new Set<string>();
|
|
91
|
+
for (const entry of branch) {
|
|
92
|
+
const record = recordOf(entry);
|
|
93
|
+
if (record?.server) for (const { entry: id } of record.hidden) entries.add(id);
|
|
94
|
+
}
|
|
95
|
+
return entries;
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
/**
|
|
99
|
+
* The last request's prompt, so the first clear after a reload is seen. Undefined after an edit or
|
|
100
|
+
* a compaction, which shrink the next request on their own.
|
|
101
|
+
*/
|
|
102
|
+
export function lastPromptOf(branch: readonly unknown[]): number | undefined {
|
|
103
|
+
for (const entry of branch.toReversed()) {
|
|
104
|
+
if (!isObject(entry) || entry.type === "compaction" || entry.type === "context_edit")
|
|
105
|
+
return undefined;
|
|
106
|
+
const message = entry.type === "message" && isObject(entry.message) ? entry.message : {};
|
|
107
|
+
const { role, stopReason, usage } = message;
|
|
108
|
+
if (role !== "assistant" || stopReason === "error" || stopReason === "aborted") continue;
|
|
109
|
+
if (!isObject(usage)) return undefined;
|
|
110
|
+
const { input, cacheRead, cacheWrite } = usage;
|
|
111
|
+
return Number(input) + Number(cacheRead) + Number(cacheWrite) || undefined;
|
|
112
|
+
}
|
|
113
|
+
return undefined;
|
|
114
|
+
}
|
|
115
|
+
|
|
85
116
|
export function hiddenCalls(branch: readonly unknown[]): Set<string> {
|
|
86
117
|
const calls = new Set<string>();
|
|
87
118
|
for (const entry of branch) {
|
|
@@ -110,10 +141,13 @@ export function costRows(record: PruneRecord): [string, string][] {
|
|
|
110
141
|
const { rewrite, saving } = record;
|
|
111
142
|
if (rewrite === undefined || saving === undefined || saving < 0.005) return [];
|
|
112
143
|
if (rewrite < 0.005) return [["saved on each request after", money(saving)]];
|
|
144
|
+
const paid = record.server
|
|
145
|
+
? "cache rewrite on this request"
|
|
146
|
+
: "cache rewrite on the next request, at most";
|
|
113
147
|
const requests = Math.ceil(rewrite / saving);
|
|
114
148
|
const back = requests === 1 ? "back in 1 request" : `back in ${requests} requests`;
|
|
115
149
|
return [
|
|
116
|
-
[
|
|
150
|
+
[paid, money(rewrite)],
|
|
117
151
|
[`saved on each request after, ${back}`, money(saving)],
|
|
118
152
|
];
|
|
119
153
|
}
|
|
@@ -138,10 +172,10 @@ export class PruneEntry implements Component {
|
|
|
138
172
|
|
|
139
173
|
render(width: number): string[] {
|
|
140
174
|
const theme = this.#theme;
|
|
141
|
-
const { before, after, hidden } = this.#record;
|
|
175
|
+
const { before, after, hidden, server } = this.#record;
|
|
142
176
|
// Glyph on the tool box's border column, text on its content column, numbers on its edge.
|
|
143
177
|
const edge = width - 1;
|
|
144
|
-
const head = ` ${theme.fg("accent", `${GLYPH} pruned`)} `;
|
|
178
|
+
const head = ` ${theme.fg("accent", `${GLYPH} ${server ? "cleared" : "pruned"}`)} `;
|
|
145
179
|
const stats = `${tokens(before)} \u2192 ${tokens(after)} tokens`;
|
|
146
180
|
const lines = [row(head, theme.fg("text", outputsHidden(hidden.length)), stats, edge, theme)];
|
|
147
181
|
for (const [label, amount] of costRows(this.#record))
|
|
@@ -153,13 +187,13 @@ export class PruneEntry implements Component {
|
|
|
153
187
|
const more = hidden.length - shown.length;
|
|
154
188
|
if (more > 0) lines.push(INDENT + theme.fg("dim", `and ${more} more`));
|
|
155
189
|
}
|
|
156
|
-
return lines.map((line) => truncateToWidth(line, width, "
|
|
190
|
+
return lines.map((line) => truncateToWidth(line, width, "..."));
|
|
157
191
|
}
|
|
158
192
|
}
|
|
159
193
|
|
|
160
194
|
function row(head: string, detail: string, stats: string, width: number, theme: Theme): string {
|
|
161
195
|
const room = width - visibleWidth(head) - visibleWidth(stats) - 1;
|
|
162
|
-
const shown = truncateToWidth(detail, Math.max(8, room), "
|
|
196
|
+
const shown = truncateToWidth(detail, Math.max(8, room), "...");
|
|
163
197
|
const gap = Math.max(1, width - visibleWidth(head) - visibleWidth(shown) - visibleWidth(stats));
|
|
164
198
|
return head + shown + " ".repeat(gap) + theme.fg("dim", stats);
|
|
165
199
|
}
|
package/src/server.ts
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
// On Anthropic models, Anthropic clears old tool output on its side with clear_tool_uses. Its
|
|
2
|
+
// clearing never drops the agent's thinking, where an edit to the context drops the thinking of
|
|
3
|
+
// every later turn: platform.claude.com/docs/en/build-with-claude/context-editing
|
|
4
|
+
// The calls stay: with their parameters cleared too, an agent in a test took its own earlier
|
|
5
|
+
// reports for invented.
|
|
6
|
+
import { isObject } from "@adeildo/pi-kit";
|
|
7
|
+
|
|
8
|
+
import { clearAtLeast, KEPT_TOOLS } from "./prune.ts";
|
|
9
|
+
|
|
10
|
+
export const KEEP_TOOL_USES = 10;
|
|
11
|
+
const BETA = "context-management-2025-06-27";
|
|
12
|
+
|
|
13
|
+
/** Anthropic's own API only: other endpoints that speak it may refuse the beta. */
|
|
14
|
+
export function clearsOnServer(model: { provider: string; api: string } | undefined): boolean {
|
|
15
|
+
return model?.provider === "anthropic" && model.api === "anthropic-messages";
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
/** Undefined leaves the payload as it is, like one another extension already manages. */
|
|
19
|
+
export function withServerClearing(payload: unknown, above: number): unknown {
|
|
20
|
+
if (!isObject(payload) || payload.context_management !== undefined) return undefined;
|
|
21
|
+
const betas = Array.isArray(payload.betas) ? payload.betas : [];
|
|
22
|
+
return {
|
|
23
|
+
...payload,
|
|
24
|
+
betas: betas.includes(BETA) ? betas : [...betas, BETA],
|
|
25
|
+
context_management: {
|
|
26
|
+
edits: [
|
|
27
|
+
{
|
|
28
|
+
type: "clear_tool_uses_20250919",
|
|
29
|
+
trigger: { type: "input_tokens", value: above },
|
|
30
|
+
keep: { type: "tool_uses", value: KEEP_TOOL_USES },
|
|
31
|
+
clear_at_least: { type: "input_tokens", value: clearAtLeast(above) },
|
|
32
|
+
exclude_tools: KEPT_TOOLS,
|
|
33
|
+
},
|
|
34
|
+
],
|
|
35
|
+
},
|
|
36
|
+
};
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* pi does not pass on what the server cleared, so a request whose context shrank by half the
|
|
41
|
+
* clearing minimum counts as cleared. A turn adds far less than that.
|
|
42
|
+
*/
|
|
43
|
+
export function serverCleared(previous: number, prompt: number, above: number): boolean {
|
|
44
|
+
return previous - prompt >= clearAtLeast(above) / 2;
|
|
45
|
+
}
|
package/src/settings.ts
CHANGED
|
@@ -15,6 +15,18 @@ export const prune = setting({
|
|
|
15
15
|
},
|
|
16
16
|
});
|
|
17
17
|
|
|
18
|
+
export const onServer = setting({
|
|
19
|
+
id: "compact.server",
|
|
20
|
+
default: true,
|
|
21
|
+
decoder: boolean,
|
|
22
|
+
ui: {
|
|
23
|
+
section: PRUNE_SECTION,
|
|
24
|
+
label: "On Anthropic's side",
|
|
25
|
+
description:
|
|
26
|
+
"On Anthropic models, Anthropic clears old tool output itself and keeps the last 10 calls whole, so the agent keeps its thinking. Off, the prune edits the context there too, which drops the thinking of every later turn.",
|
|
27
|
+
},
|
|
28
|
+
});
|
|
29
|
+
|
|
18
30
|
export const pruneAbove = setting({
|
|
19
31
|
id: "compact.pruneAbove",
|
|
20
32
|
default: 150_000,
|
|
@@ -23,7 +35,7 @@ export const pruneAbove = setting({
|
|
|
23
35
|
section: PRUNE_SECTION,
|
|
24
36
|
label: "Above",
|
|
25
37
|
description:
|
|
26
|
-
"Context size that starts a prune. Each prune rewrites the prompt cache once, so it waits until it can free at least
|
|
38
|
+
"Context size that starts a prune. Each prune rewrites the prompt cache once, so it waits until it can free at least two fifths of the context.",
|
|
27
39
|
control: { type: "number", min: 20_000, max: 2_000_000, step: 10_000, unit: " tokens" },
|
|
28
40
|
},
|
|
29
41
|
});
|
|
@@ -53,4 +65,4 @@ export const model = setting({
|
|
|
53
65
|
},
|
|
54
66
|
});
|
|
55
67
|
|
|
56
|
-
export const COMPACT_SETTINGS = [prune, pruneAbove, summary, model];
|
|
68
|
+
export const COMPACT_SETTINGS = [prune, onServer, pruneAbove, summary, model];
|
package/src/summary.ts
CHANGED
|
@@ -77,7 +77,7 @@ function yourWords(messages: readonly AgentMessage[]): string[] {
|
|
|
77
77
|
.replace(/\s+/g, " ")
|
|
78
78
|
.trim();
|
|
79
79
|
if (text === "") return [];
|
|
80
|
-
return [text.length <= MAX_ASK_CHARS ? text : `${text.slice(0, MAX_ASK_CHARS -
|
|
80
|
+
return [text.length <= MAX_ASK_CHARS ? text : `${text.slice(0, MAX_ASK_CHARS - 3)}...`];
|
|
81
81
|
});
|
|
82
82
|
return words.slice(-MAX_ASKS);
|
|
83
83
|
}
|