pi-tuck 1.0.10 → 1.0.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +49 -3
- package/README.md +32 -27
- package/dist/src/build-id.d.ts +1 -1
- package/dist/src/build-id.js +1 -1
- package/dist/src/clip.d.ts +8 -0
- package/dist/src/clip.js +13 -0
- package/dist/src/compaction.d.ts +3 -0
- package/dist/src/compaction.js +54 -15
- package/dist/src/diagnostic-replay.d.ts +1 -0
- package/dist/src/diagnostics.d.ts +1 -1
- package/dist/src/tuck.d.ts +4 -1
- package/dist/src/tuck.js +12 -4
- package/package.json +1 -1
- package/src/build-id.ts +1 -1
- package/src/clip.ts +13 -0
- package/src/compaction.ts +59 -14
- package/src/tuck.ts +15 -6
package/CHANGELOG.md
CHANGED
|
@@ -3,7 +3,52 @@
|
|
|
3
3
|
All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
|
|
4
4
|
[Semantic Versioning](https://semver.org/).
|
|
5
5
|
|
|
6
|
-
## [1.0.
|
|
6
|
+
## [1.0.11] - 2026-11-06
|
|
7
|
+
|
|
8
|
+
### Fixed
|
|
9
|
+
|
|
10
|
+
- A prompt that fits the room is no longer dropped when the previous summary alone fills it (the 1.0.10 known issue): a cut can now shrink a
|
|
11
|
+
summary that sits alone, when the cut is due. A prompt that fits beside the summary is sent as it stands.
|
|
12
|
+
- A fresh tool result is clipped in more of the cases where the request would otherwise be refused. The clip's budget is now counted in the
|
|
13
|
+
same units as the request and allows for what the smallest cut keeps, so a round that fits only when clipped no longer ends the run. This
|
|
14
|
+
is most of the refusals the settings stress test found.
|
|
15
|
+
- One wrong usage report no longer refuses later prompts: a reply that reports more than the model's window is not learned from.
|
|
16
|
+
- When a provider refuses a request as over its window, pi-tuck now says so in a notice, where the turn could read as a hang.
|
|
17
|
+
- Every refusal says why.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- A settings stress test (`test/stress`, run with `STRESS=1`): real Pi with a scripted fake model, across 272 cells of window, reserve,
|
|
22
|
+
keep-recent, output cap, setup size and usage reports. CI runs it on Pi 1.2.0 and the newest Pi. Rejection wordings and usage reports in
|
|
23
|
+
it are scripted: no hosted provider API is tested.
|
|
24
|
+
|
|
25
|
+
### Known issues
|
|
26
|
+
|
|
27
|
+
- Same settings stress test, this build: 4 of 272 cells report a failure (90 on 1.0.10), none stuck and none losing work. The next prompt runs.
|
|
28
|
+
- A fresh assistant message larger than the free room is refused: nothing can clip it yet. A short next prompt recovers. In an unattended
|
|
29
|
+
`pi -p` run, a refusal ends the run.
|
|
30
|
+
- With a setup near the band, cuts can come too often: with Pi's defaults, a 40k setup at a 65,536 window made 25 cuts in 60 tool rounds, each
|
|
31
|
+
freeing little. It costs extra work, not a task. One sampled cell at a 167,739 window also trips this count. The fix is planned for the next
|
|
32
|
+
patch.
|
|
33
|
+
- A wrong usage report at or under the window can still raise the estimate for a while: in the stress cell for it, a 6,000-token prompt is
|
|
34
|
+
refused in a 28,672-token room.
|
|
35
|
+
- If a provider keeps rejecting requests in a wording Pi does not class as a context overflow, or with a bare HTTP 413, pi-tuck does not recover
|
|
36
|
+
on its own: the provider's error reaches you, the session stays intact, and every prompt can fail until the request shrinks. Try `/compact`.
|
|
37
|
+
|
|
38
|
+
### Release gates
|
|
39
|
+
|
|
40
|
+
- **Shipped without a live gate.** Its replay matches 1.0.10, so the paired live test has nothing to measure on recorded sessions; the fixes
|
|
41
|
+
act only where a request would otherwise be refused. For that reason replay cannot show them working either, and they are shown by the stress
|
|
42
|
+
test alone, on a scripted model.
|
|
43
|
+
- Passed: tests; replay of the exact code on 29 recordings against 1.0.10 (13,946 requests, 0 differ, 0 dead tasks, the same 9 clips, all read
|
|
44
|
+
back); the stress test (no cell stuck, none losing work; no cell has a failure 1.0.10 does not. One cell makes 8 cuts where 1.0.10 made 6,
|
|
45
|
+
because its prompts now run instead of being refused).
|
|
46
|
+
- An earlier 1.0.11 build ran Pi's defaults on chain t06 with no lost task
|
|
47
|
+
([`benchmark/RESULTS-DEFAULTS-1010.md`](benchmark/RESULTS-DEFAULTS-1010.md)). One of its tasks, task 21, ran to the 30-minute wall cap on a
|
|
48
|
+
first request the local server never answered, and recorded no token. It is read as a server stall. It was counted as passing under the bar's
|
|
49
|
+
definition; the registered counter alone would count one dead task.
|
|
50
|
+
|
|
51
|
+
## [1.0.10] - 2026-11-05
|
|
7
52
|
|
|
8
53
|
### Fixed
|
|
9
54
|
|
|
@@ -28,8 +73,9 @@ All notable changes to pi-tuck. The format follows [Keep a Changelog](https://ke
|
|
|
28
73
|
|
|
29
74
|
- Passed: tests; replay of the exact build on 29 recordings (no dead task, no refused request, every oversize result clipped and read
|
|
30
75
|
back); the Pi-defaults rerun on chain t06, which lost 28 of 30 tasks on 1.0.8: 0 dead tasks and 0 stuck sessions
|
|
31
|
-
([`benchmark/RESULTS-DEFAULTS-1010.md`](benchmark/RESULTS-DEFAULTS-1010.md)).
|
|
32
|
-
|
|
76
|
+
([`benchmark/RESULTS-DEFAULTS-1010.md`](benchmark/RESULTS-DEFAULTS-1010.md)). It ran on a slow local server: 27 of 30 tasks hit the 30-minute
|
|
77
|
+
wall cap, and 2 of them sat on a first request the server never answered and recorded no token. Those 2 are read as server stalls, and a strict
|
|
78
|
+
no-token count would not meet the bar.
|
|
33
79
|
One task was refused mid-run: its round carried 9,020 tokens of reasoning, so the next request was about 17,801 against the 16,384-token room.
|
|
34
80
|
- **Failed as registered:** the paired 32k gate against 1.0.9 (chains t01 and t02, 60 tasks). No task was lost in either arm, but 1.0.10
|
|
35
81
|
resolved 49 tasks against 50, so "resolved no lower" failed by one task (−1.7 points, one-sided 95% lower bound −8.3;
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# pi-tuck
|
|
2
2
|
|
|
3
|
-
**Compaction for [Pi](https://pi.dev) with no model call, and a history the agent can search by keyword, and on Apple silicon by meaning, all on your machine.** In our live run, a cut took a median 0.
|
|
3
|
+
**Compaction for [Pi](https://pi.dev) with no model call, and a history the agent can search by keyword, and on Apple silicon by meaning, all on your machine.** In our 128k live run, a cut took a median 0.08 s (pi-tuck 1.0.1). It costs no tokens. The same session always gives the same summary. And nothing is lost: the agent can read back anything earlier, word for word, and on Apple silicon it can find it by meaning.
|
|
4
4
|
|
|
5
5
|
```sh
|
|
6
6
|
pi install npm:pi-tuck
|
|
@@ -8,22 +8,29 @@ pi install npm:pi-tuck
|
|
|
8
8
|
|
|
9
9
|
## pi-tuck against Pi's built-in compaction
|
|
10
10
|
|
|
11
|
-
Pi's built-in compaction asks a model to write a summary. pi-tuck builds the summary from the session itself. The
|
|
11
|
+
Pi's built-in compaction asks a model to write a summary. pi-tuck builds the summary from the session itself. The time, context and task rows compare pi-tuck 1.0.1 with Pi 1.4.0's built-in compaction, where a local model writes the summary. Each measured result in this README names the pi-tuck build or version it was measured on; the release you install may be newer.
|
|
12
12
|
|
|
13
13
|
| | Pi's built-in (local model writes the summary) | pi-tuck |
|
|
14
14
|
|---|---|---|
|
|
15
15
|
| Model call per cut | One, to a model you pay for or wait on | None |
|
|
16
|
-
| Time per cut | **
|
|
17
|
-
| Context carried after a cut |
|
|
16
|
+
| Time per cut | **235 s** (median) | **0.08 s** (median; an upper bound, see below) |
|
|
17
|
+
| Context carried after a cut | 27,750 tokens | 26,790 tokens (about the same) |
|
|
18
|
+
| Facts cut away, asked back at 128k (mechanical probes) | Answered from the summary alone, with no recall tool | Answered from the summary and pi-tuck's recall tools, cuts 1 to 4 only. Claim line below the table |
|
|
19
|
+
| Tasks resolved at 128k (300 SWE-bench Verified tasks per arm, pi-tuck 1.0.1) | 251 of 300 | 258 of 300. The registered claim is only "no loss larger than 0.4 points"; chain s09 matters, see below |
|
|
18
20
|
| Same session, same summary | Not guaranteed, since a model writes it | Yes |
|
|
19
21
|
| Can the agent get dropped detail back | No recall tool in Pi 1.4.0. The full history stays in the session file, but the agent has no tool made to search it | Yes. `tuck_recall` reads any original exactly |
|
|
20
22
|
| Search history by meaning | No | Yes, with `tuck_search`, on macOS with Apple silicon. It runs locally on your Mac's GPU, and no text leaves your machine. Elsewhere there is no `tuck_search`, and `tuck_recall` searches by keyword |
|
|
21
23
|
| Can the summary invent history | It can, since a model writes it | No, it holds only recorded text |
|
|
22
24
|
| What the summary leaves out | Whatever the model dropped | Tool output only; the agent reads it back exactly |
|
|
23
|
-
| Billed input at the default 60% cut point (simulated) | Not measured | 10–18% less than pi-tuck cutting at the full room. This is not a comparison with the built-in |
|
|
24
25
|
| Tasks resolved at a 32k window (150 SWE-bench Verified tasks, one model, pi-tuck 1.0.5) | 94 of 150 | 120 of 150 (+17.3 points, 95% lower bound +10.7). Limits below the table |
|
|
25
26
|
|
|
26
|
-
The time and context rows are
|
|
27
|
+
The time and context rows are from the 128k live run: 10 chains of 30 tasks, pi-tuck 1.0.1, medians over 46 pi-tuck cuts and 48 cuts by Pi's compaction. pi-tuck's cut time is an upper bound, read from the session file ([`benchmark/DEVIATIONS.md`](benchmark/DEVIATIONS.md)).
|
|
28
|
+
|
|
29
|
+
The retention row is a paired replay on mechanical probes only: files changed, test-runner summary, required tag, error class and the session's result-line rule. No probe judged how well an agent reasons. The claim line, as [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) words it: **pi-tuck, answering with its recall tools, retains probed facts at a rate not more than 3 points below that of "Pi's compaction prompts on mac-studio/main:xhigh" (20 recordings) or of "Pi's compaction prompts on GPT-6.1 Terra" (19 of 20 recordings, s09-A missing), each answering from its summary alone, on these probes.** This is not summary against summary: it compares pi-tuck's summary together with its recall tools against Pi's compaction without them, and the run has no pi-tuck arm without its recall tools. It says nothing about any other compactor, model or window.
|
|
30
|
+
|
|
31
|
+
In the replay behind pi-tuck's retention numbers, 52 of its 95 answer sets ended in an abort (0 of 95 for Pi's compaction prompts on the same local model), so those numbers cover cuts 1 to 4 only and no retention claim is made past cut 4; see [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md).
|
|
32
|
+
|
|
33
|
+
The 32k row is one model, one window and five chains. Observation masking, a third approach, resolved 117 of the same 150 tasks (pi-tuck +2.0 points, lower bound −2.7: no difference shown). In the 32k run only pi-tuck was held to the 16,384-token room: Pi's built-in compaction sent a request over it in 97 of 150 tasks. Every number comes from a file in this repository. Conditions, sample sizes and the full limits are in [Benchmarks](#benchmarks).
|
|
27
34
|
|
|
28
35
|
## Ask your history
|
|
29
36
|
|
|
@@ -89,14 +96,14 @@ On Apple silicon, both tools cost about 1,050 tokens of tool description per req
|
|
|
89
96
|
|
|
90
97
|
## What is proven, and what is not
|
|
91
98
|
|
|
92
|
-
**Measured so far.** In the live run
|
|
99
|
+
**Measured so far.** In the 128k live run (pi-tuck 1.0.1), a pi-tuck cut took a median 0.08 s, where Pi's compaction with a local model writing the summary took a median 235 s. A cut calls no model, so it costs no tokens. After a cut, both carried about the same context. On mechanical probes about facts cut away, pi-tuck with its recall tools was not more than 3 points below Pi's compaction prompts on mac-studio/main:xhigh, or below Pi's compaction prompts on GPT-6.1 Terra, each answering from its summary alone (claim line above; cuts 1 to 4 only). Over 300 tasks per arm, pi-tuck resolved 258 and Pi's compaction 251; the registered claim is only that pi-tuck lost no more than 0.4 points. In replay, answers held when pi-tuck cut at 60% of the room instead of 100%. At a 32k window, in one suite of 150 SWE-bench Verified tasks with one model, pi-tuck 1.0.5 resolved 120 tasks and Pi's built-in compaction 94 (+17.3 points, 95% lower bound +10.7). Observation masking resolved 117 (pi-tuck +2.0 points, lower bound −2.7: no difference shown). The limits of each result are in [Benchmarks](#benchmarks).
|
|
93
100
|
|
|
94
|
-
**Not claimed.** This README does not say that pi-tuck is as good as
|
|
101
|
+
**Not claimed.** This README does not say that pi-tuck is as good as or better than any compactor, model, window or kind of task other than those named, and it never calls it "the best LLM compactor". The retention tests compare pi-tuck with its recall tools against Pi's compaction without them. The tests were pre-registered before any result came in. The plans, and every result, including those that did not favour pi-tuck, are in [`benchmark/`](benchmark).
|
|
95
102
|
|
|
96
103
|
The pre-registered plan has three parts:
|
|
97
104
|
|
|
98
105
|
1. **A live run** at a 128k window on a local model, with pi-tuck 1.0.1. Ten chains of 30 SWE-bench Verified tasks per arm, with Pi's built-in compaction in one arm and pi-tuck in the other. Official grading.
|
|
99
|
-
2. **A paired replay with probe questions.** The same cuts, made by pi-tuck and by Pi's summarize prompts, then questions about facts that fell before the cut. Pre-registered rule: the claim holds only if pi-tuck is no more than 3 points worse than the better compactor, with a 95% bound. The comparators are named in [`benchmark/PINS.md`](benchmark/PINS.md).
|
|
106
|
+
2. **A paired replay with probe questions.** The same cuts, made by pi-tuck and by Pi's summarize prompts, then questions about facts that fell before the cut. Pre-registered rule: the claim holds only if pi-tuck is no more than 3 points worse than the better compactor, with a one-sided 95% bound. The comparators are named in [`benchmark/PINS.md`](benchmark/PINS.md) and above.
|
|
100
107
|
3. **A 32k suite**, where tasks are bigger than the room, so cuts must summarise work in progress. Its result is the 32k row above.
|
|
101
108
|
|
|
102
109
|
The plan: [`benchmark/PREREG.md`](benchmark/PREREG.md), [`benchmark/PREREG-32K.md`](benchmark/PREREG-32K.md), [`benchmark/PREREG-TERRA.md`](benchmark/PREREG-TERRA.md), and every change after the freeze in [`benchmark/DEVIATIONS.md`](benchmark/DEVIATIONS.md).
|
|
@@ -104,22 +111,29 @@ The plan: [`benchmark/PREREG.md`](benchmark/PREREG.md), [`benchmark/PREREG-32K.m
|
|
|
104
111
|
Known limits today:
|
|
105
112
|
|
|
106
113
|
- pi-tuck's summary does not explain why things happened. It holds the words that were said, not a model's reading of them. The agent must look things up. Each look-up costs a tool call.
|
|
107
|
-
- Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space
|
|
108
|
-
- Small window, after a stopped run (1.0.8 and 1.0.9): Pi's default `compaction.keepRecentTokens` (20,000) can exceed pi-tuck's input room, which is the window less the larger of `compaction.reserveTokens` and the model's maximum output
|
|
114
|
+
- Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space. pi-tuck then refuses to cut rather than overflow. This made 1.0.1 refuse at a 32k window with Pi's default 16,384 reserve. Version 1.0.4 added a notice that says when the provider counts far above pi-tuck's estimate. In 1.0.4, after an oversize tool result or a "Nothing to compact" error from Pi, a session could keep refusing later prompts; 1.0.5 fixed both. Version 1.0.8 clips a fresh tool result that is too large for the room (its head and tail, with a `tuck_recall` locator to the rest) instead of refusing the run. If a small window still refuses, pin a different upstream, lower the reserve, or use a larger window; see [Small windows](#small-windows).
|
|
115
|
+
- Small window, after a stopped run (1.0.8 and 1.0.9 only; fixed in 1.0.10): Pi's default `compaction.keepRecentTokens` (20,000) can exceed pi-tuck's input room, which is the window less the larger of `compaction.reserveTokens` and the model's maximum output (16,384 at a 32k window with the default reserve). After a run that is stopped (Esc, or a time limit) with the context above the room, every later prompt could be refused with "The request needs a cut and Pi has none to make (Nothing to compact (session too small))", and the session stayed stuck. In one check at these defaults (pi-tuck 1.0.8, 32k window, one session of 30 tasks, one local model), 28 tasks were lost, tasks 3 to 30 ([`benchmark/RESULTS-DEFAULTS.md`](benchmark/RESULTS-DEFAULTS.md)). If you are on 1.0.8 or 1.0.9, upgrade to 1.0.10. In 1.0.10 the cut a prompt needs is pi-tuck's own, so `keepRecentTokens` no longer leaves it with nothing to cut. A rerun of the same check on the 1.0.10 candidate lost no task and got stuck nowhere, with the caveat that two of its tasks had a first request the local server never answered, which a literal no-token count would call lost ([`benchmark/RESULTS-DEFAULTS-1010.md`](benchmark/RESULTS-DEFAULTS-1010.md)). Windows of 128k and up were not affected at the defaults.
|
|
116
|
+
- Small window, a large prompt after a summary-only cut (1.0.10): when the previous summary alone fills the room and a large prompt arrives, the prompt is dropped, because a cut cannot yet shrink a summary on its own. 1.0.10 drops it with no message, where 1.0.9 refused it with one. The next prompt runs normally. Seen once, on a 25,160-character prompt at a 32k window ([`benchmark/RESULTS-CUTLATER.md`](benchmark/RESULTS-CUTLATER.md), t03 task 24). A fix is planned for 1.0.11.
|
|
117
|
+
- Other extensions (1.0.10): a prompt's cut no longer goes through Pi's `session_before_compact`, so another extension's veto there no longer stops it. It never stopped a cut between tool rounds. Pi's own threshold and overflow cuts, and `/compact`, still ask other extensions as before.
|
|
109
118
|
- Semantic search (`tuck_search`) needs a Mac with Metal (Apple silicon). Elsewhere, compaction and `tuck_recall`'s word search work, there is no `tuck_search`, and nothing is downloaded.
|
|
110
119
|
|
|
111
120
|
## Benchmarks
|
|
112
121
|
|
|
113
|
-
Each row says what was measured, under which conditions, and where the data is.
|
|
122
|
+
Each row says what was measured, under which conditions, and where the data is.
|
|
114
123
|
|
|
115
124
|
| What | Result | Conditions | Source |
|
|
116
125
|
|---|---|---|---|
|
|
117
|
-
| Pi's default compaction settings, 32k window |
|
|
126
|
+
| Pi's default compaction settings, 32k window, pi-tuck 1.0.10 candidate | Passed as registered: 0 dead tasks and 0 stuck sessions on 30 tasks, on a slow local server (27 of 30 tasks hit the 30-minute wall cap, and 2 had a first request the server never answered). 6 resolved. A counter written later, which counts a task with no token as dead, would count those 2 | Candidate `a67ec30` of 1.0.10, `mac-studio/main`, window 32,768, output cap 16,384, Pi's default `reserveTokens` 16,384 and `keepRecentTokens` 20,000, one session of chain t06. One mid-run refusal (a round with 9,020 tokens of reasoning against the 16,384-token room). One session, one model; a pass shows no lost task in these 30 tasks, not that none can occur | [`benchmark/RESULTS-DEFAULTS-1010.md`](benchmark/RESULTS-DEFAULTS-1010.md) |
|
|
127
|
+
| Pi's default compaction settings, 32k window, pi-tuck 1.0.8 | 28 of 30 tasks lost (tasks 3 to 30, each refused at its prompt); 1 resolved. Fails the pre-registered bar of 0 lost tasks | pi-tuck 1.0.8 (`86935e9`), `mac-studio/main`, window 32,768, output cap 16,384, `reserveTokens` 16,384 and `keepRecentTokens` 20,000 (larger than the room), one session of chain t06. One session, one model, one build: it shows that the failure occurs at these settings, not how often. Not comparable with other rows | [`benchmark/RESULTS-DEFAULTS.md`](benchmark/RESULTS-DEFAULTS.md) |
|
|
118
128
|
| Tasks resolved, 32k window | Pi's built-in compaction 94 of 150; pi-tuck 120 of 150 (+17.3 points, 95% lower bound +10.7); observation masking 117 of 150 (pi-tuck +2.0, lower bound −2.7: no difference shown). Turns per task: 9.4 (built-in), 24.0 (pi-tuck), 11.9 (masking). Tokens per session: 6.10 M, 6.39 M, 7.02 M. Spend per arm for the five sessions: $0.87, $0.55, $0.94 | pi-tuck 1.0.5 (commit `aa30824`, the 1.0.5 code; 1.0.6 and 1.0.7 are not measured). DeepSeek V4.1 Flash pinned to one upstream, window 32,768, reserve 16,384, 5 chains of 30 SWE-bench Verified tasks, paired by task, one-sided bootstrap over tasks. **Limits:** only pi-tuck is held to the 16,384-token room (the built-in sent a request over it in 97 of 150 tasks, masking in 73, so the window is nominal for them). The difference by chain is +10.0, −3.3, +50.0, +16.7 and +13.3 points; leaving out chain t03 it is +9.2. The bootstrap resamples tasks, not chains. The pi-tuck arm ran at a different time from the others. One model, one window. pi-tuck refused to cut mid-run in 5 tasks, where one fresh tool result was larger than the room; they count as unresolved. In the same setup pi-tuck 1.0.3 resolved 72 of 150, with 52 dead tasks (tasks refused at their prompt after an earlier failure). In the pi-tuck arm of this row there was 1 dead task (t03 task 24: refused at its prompt with "Nothing to compact"; the next task ran normally), so this build missed the pre-registered acceptance bar of zero dead tasks by one task (the other part of the bar, no loss against 1.0.3 by more than 3 points, was met) | [`benchmark/RESULTS-32K-ACCEPT.md`](benchmark/RESULTS-32K-ACCEPT.md) |
|
|
129
|
+
| The 1.0.10 candidate against 1.0.9, 32k window | 0 dead tasks in both arms; no mid-run refusal. Resolved 50 of 60 (1.0.9) and 49 of 60 (1.0.10 candidate): −1.7 points, 95% lower bound −8.3. **The gate failed as registered, by one task**; 1.0.10 shipped as an exception because of the small-window defect. Turns per task 25.08 and 29.12; cuts 731 and 884; total tokens 13.07 M and 15.12 M | Pre-registered gate. pi-tuck `a67ec30` against 1.0.9 (`cd4626b`). DeepSeek V4.1 Flash pinned to one upstream, window 32,768, reserve 16,384, `keepRecentTokens` 3,333, chains t01 and t02, 60 paired tasks. One model, one window, one run per arm; 60 tasks cannot separate a one-task difference from run-to-run variation | [`benchmark/RESULTS-V110.md`](benchmark/RESULTS-V110.md) |
|
|
119
130
|
| The 1.0.8 candidate against 1.0.7, 32k window | 0 dead tasks in both arms. Resolved 46 of 60 in both (+0.0 points, 95% lower bound −6.7). No mid-run refusal in either arm. Turns per task 25.37 (1.0.7) and 26.10 (1.0.8) | Pre-registered gate, passed as registered. pi-tuck `a2e7d1a` (the tool-result clip and the cut at the end of a run, on 1.0.6) against 1.0.7. Released 1.0.8 is that build merged with 1.0.7's Metal-only recall; its replay requests are reported byte-identical to the gated build on 29 recordings (stated in the source file, not re-run for this README). DeepSeek V4.1 Flash pinned to one upstream, window 32,768, reserve 16,384, chains t01 and t02, 60 paired tasks, both arms run together. One model, one window, one run per arm; 60 tasks cannot show a difference smaller than the bound. The run shows that the change introduced no refusal; no refusal occurred in the 1.0.7 arm for the clip to prevent | [`benchmark/RESULTS-V108.md`](benchmark/RESULTS-V108.md) |
|
|
120
|
-
| Cut time, live | pi-tuck median 0.
|
|
121
|
-
| Cut time, paired replay | pi-tuck median 0.
|
|
122
|
-
|
|
|
131
|
+
| Cut time, live, 128k | pi-tuck median 0.081 s (46 cuts; 90th percentile 0.279 s). Pi's compaction median 235.3 s (48 cuts; 90th percentile 403.8 s). Context carried after a cut, median: 26,790 tokens (pi-tuck), 27,750 (Pi's compaction) | pi-tuck 1.0.1, Pi 1.4.0, SWE-bench Verified, 10 chains of 30 tasks, window 128,000, reply reserve 32,768, `mac-studio/main:xhigh` writing Pi's summaries. The built-in time depends on that model and on server load. pi-tuck's time is an upper bound, read from the session file: the time between the entry written just before the cut and the compaction entry, so it includes anything between the last message and the cut (pi-tuck's tool-loop cuts emit no compaction events). The 44 failed compaction attempts in chain s09-A have no compaction entry and are not in the counts | [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) test 3 |
|
|
132
|
+
| Cut time, paired replay, 128k | pi-tuck median 0.061 s (95 cuts). Pi's compaction prompts on mac-studio/main:xhigh 154.6 s (86 cuts). Pi's compaction prompts on GPT-6.1 Terra 53.2 s (87 cuts, 19 recordings). Context carried after a cut, median: 27,656, 25,790 and 22,784 tokens; median summary 85,561, 23,598 and 11,200 characters | The same recorded sessions cut at the same raw positions. pi-tuck 1.0.1. Terra's cuts ran through Codex, not on the local server | [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) test 3 |
|
|
133
|
+
| Retention, 128k, test 1a: pi-tuck with its recall tools against Pi's compaction prompts on mac-studio/main:xhigh | Registered claim: not more than 3 points below, on these probes (cuts 1 to 4 only) (supported). Accuracy on the probes each was asked: 89.7% (783 probes) and 61.8% (1,677). Difference over the 783 shared probes, which come from the 43 of pi-tuck's 95 answer sets that did not abort: +25.7 points, one-sided 95% lower bound +22.0 (+22.3 clustered by chain). By probe type, pi-tuck / comparator: error class 80.2 / 24.7, files changed 84.5 / 78.9, result-line rule 100.0 / 100.0, required tag 100.0 / 99.3, test-runner summary 90.1 / 31.6 | pi-tuck 1.0.1, 20 recordings, mechanical probes only, evidence at least 35,000 raw tokens before the cut. pi-tuck answered with its recall tools and the comparators with none, from the summary and the kept recent messages. Every one of the 43 asks behind the 783 probes made recall calls (1.25 per probe on average), so no pi-tuck answer lacked the tools; the run cannot show how much of the result comes from the summary and how much from recall. No pi-tuck arm without recall tools exists. The comparators' agents had unrestricted bash, and the logs cannot rule out that one read a recording; nothing shows it. pi-tuck's accuracy does not rise with more calls per probe (87.9%, 90.8%, 90.0% by thirds; post hoc, descriptive). The 783 shared probes are fewer than the 2,969 needed to certify a difference near the margin | [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) test 1a |
|
|
134
|
+
| Retention, 128k, test 1b-Terra: pi-tuck with its recall tools against Pi's compaction prompts on GPT-6.1 Terra | Registered claim: not more than 3 points below, on these probes (cuts 1 to 4 only) (supported). Accuracy on the probes each was asked: 89.7% (783) and 57.8% (1,685). Difference over the 728 shared probes, from the same 43 of pi-tuck's 95 answer sets: +33.7 points, one-sided 95% lower bound +30.4 (+30.1 clustered by chain). By probe type, pi-tuck / Terra: error class 80.2 / 14.7, files changed 84.5 / 72.3, result-line rule 100.0 / 100.0, required tag 100.0 / 93.3, test-runner summary 90.1 / 38.1 | As test 1a, on 19 of the 20 recordings: s09-A is missing because its Terra cut job hit a Codex usage limit, and the set was closed at 19 before any Terra result was read. The claim is non-inferiority. The same limits as test 1a apply | [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) test 1b-Terra |
|
|
135
|
+
| Tasks resolved, 128k, test 2 | Pi's compaction 251 of 300, pi-tuck 258 of 300. Registered claim: no loss larger than 0.4 points. All 10 chains: +2.3 points, lower bound +0.3. Without chain s09: +1.5 points, lower bound −0.4 | pi-tuck 1.0.1, 10 chains of 30 tasks, paired by task, bootstrap over chains. In s09-A Pi's compaction failed 44 times ("generation hit the token cap") from task 23 on, and no other chain has a failed compaction; that is why the bound changes without s09 | [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) test 2 |
|
|
136
|
+
| Tokens per session, input plus output without cache reads, 128k | 241,000 fewer for pi-tuck per session on average, one-sided 95% upper bound −21,000, with all 10 chains. Without chain s09 the mean is −174,000 and the upper bound +43,000 | pi-tuck 1.0.1, paired by chain, bootstrap over sessions. The line holds only with all chains | [`benchmark/RESULTS-MAIN.md`](benchmark/RESULTS-MAIN.md) test 3 |
|
|
123
137
|
| Answers after cutting at 60% instead of 100% | No more losses than the limit on all 4 sets. Losses / wins over n questions: 128k shared 3 / 7 of 99; 128k asymmetric 3 / 1 of 67; 32k shared 5 / 3 of 77; 32k asymmetric 9 / 5 of 343. In the 32k asymmetric set the losses lean against 60% (test_result 4 / 2, error_hit 3 / 1); it is the set to watch | Paired replay, same 5 sessions, DeepSeek flash as agent and judge. Limit, set in advance: losses minus wins at most max(2, 5% of n). Asymmetric = evidence the base keeps native and the variant has summarised | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md) |
|
|
124
138
|
| 1.0.2 changes (plain-text summary, shrinking dialogue view, time on user turns only) against 1.0.1 | No more losses than the limit on all 4 sets. Losses / wins: 128k 1 / 2 of 128; 128k time questions 0 / 0 of 48; 32k 4 / 2 of 127; 32k asymmetric 0 / 2 of 56 | Paired replay, 5 recorded live sessions, DeepSeek flash as agent and judge. 11 of 146 answer sessions hit a 15-minute wall limit and drop out of the pairing | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md), [`CHANGELOG.md`](CHANGELOG.md) |
|
|
125
139
|
| Summary size, plain-text format against the 1.0.1 JSON format | About 30% shorter where the agent writes little, about 14% shorter where it writes a lot | Same dialogue | [`CHANGELOG.md`](CHANGELOG.md) 1.0.2 |
|
|
@@ -128,9 +142,8 @@ Each row says what was measured, under which conditions, and where the data is.
|
|
|
128
142
|
|
|
129
143
|
Notes:
|
|
130
144
|
|
|
131
|
-
- The billing figures come from a simulation that replays real sessions through the real cut trigger and prices every request. They are not bills.
|
|
132
145
|
- The answer replays use one cheap hosted model (DeepSeek flash) as agent and judge. They compare pi-tuck with other pi-tuck settings and versions. They do not compare pi-tuck with an LLM compactor.
|
|
133
|
-
- The
|
|
146
|
+
- The 128k rows measure pi-tuck 1.0.1. The newer versions are listed in the [changelog](CHANGELOG.md). A cut still calls no model, but the 128k run has not been repeated on them.
|
|
134
147
|
- Everything here is from SWE-bench-style tasks on a small set of models. It may not carry to your work.
|
|
135
148
|
|
|
136
149
|
## Settings
|
|
@@ -141,7 +154,7 @@ pi-tuck needs none. Two are optional. Put them in `pi-tuck.json` in Pi's agent f
|
|
|
141
154
|
{ "cutAt": 0.6, "semanticRecall": true }
|
|
142
155
|
```
|
|
143
156
|
|
|
144
|
-
- **`cutAt`** (above 0, at most 1; default 0.6) is the part of the input room a request may fill before pi-tuck cuts. At 0.6, each cut leaves room to grow
|
|
157
|
+
- **`cutAt`** (above 0, at most 1; default 0.6) is the part of the input room a request may fill before pi-tuck cuts. At 0.6, each cut leaves room to grow. `{ "cutAt": 1 }` cuts at the full room. A lower value cuts earlier and smaller. The whole room still decides which request pi-tuck refuses.
|
|
145
158
|
- **`semanticRecall`** (`false` turns semantic search off, on Apple silicon). Nothing is then downloaded, and `tuck_search` uses keywords only.
|
|
146
159
|
|
|
147
160
|
A bad file is reported at session start, and pi-tuck uses the defaults.
|
|
@@ -156,14 +169,6 @@ pi-tuck keeps room for the model's reply: the larger of the model's output limit
|
|
|
156
169
|
|
|
157
170
|
The reserve is the larger of `reserveTokens` and the model's `maxTokens`, so lowering `reserveTokens` alone does nothing when `models.json` sets a larger `maxTokens` for that model. Lower both, and keep `maxTokens` above your largest expected reply.
|
|
158
171
|
|
|
159
|
-
If a session gets stuck after a stopped run (see Known limits), set `keepRecentTokens` well below the room, in `~/.pi/agent/settings.json` for all models, or for one model:
|
|
160
|
-
|
|
161
|
-
```json
|
|
162
|
-
{ "compaction": { "modelOverrides": { "my-provider/my-model": { "keepRecentTokens": 3333 } } } }
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
3,333 is the value the 32k benchmark used: the main run's 20,000 of a 98,304 room, scaled to a 16,384 room (`benchmark/PREREG-32K.md`).
|
|
166
|
-
|
|
167
172
|
If you reach the model through OpenRouter, you can also pin one upstream, so that every request is served by the same one. Put this in `models.json` in Pi's agent folder, and name the upstream you want:
|
|
168
173
|
|
|
169
174
|
```json
|
package/dist/src/build-id.d.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export declare const BUILD_ID = "
|
|
1
|
+
export declare const BUILD_ID = "828ac4821689607af006d54a41fd5960bb910bc65aa43adbd81c2f6aac25d180";
|
package/dist/src/build-id.js
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export const BUILD_ID = "
|
|
1
|
+
export const BUILD_ID = "828ac4821689607af006d54a41fd5960bb910bc65aa43adbd81c2f6aac25d180";
|
package/dist/src/clip.d.ts
CHANGED
|
@@ -4,6 +4,14 @@ export interface Clip {
|
|
|
4
4
|
toolCallId: string;
|
|
5
5
|
content: ToolResultMessage["content"];
|
|
6
6
|
}
|
|
7
|
+
/** The budgets to offer a fresh round the room cannot hold whole, in the units the round and the mandatory input are counted in, tried in order.
|
|
8
|
+
* The room is the provider's, so it is divided by the factor the provider's counts have been running at - the same conversion the planner's own
|
|
9
|
+
* reserve makes (budget.ts) - before the input no cut can remove is taken out; counting the provider's room against an estimate asks the round for
|
|
10
|
+
* more than the room holds at a factor above 1. The first try is half of what that leaves. `kept` is what the smallest cut leaves the request
|
|
11
|
+
* besides the fresh results - the summary floor and its quote of the prompt - measured from the planner's own preview, not a new formula: a half
|
|
12
|
+
* cannot spend it, so the second try is the whole free room less it. Omit it when the planner bailed before building a preview, and only the half
|
|
13
|
+
* is tried. Nothing is offered when the mandatory input alone leaves no room. */
|
|
14
|
+
export declare const clipBudgets: (room: number, factor: number, mandatory: number, kept?: number) => number[];
|
|
7
15
|
/** The round's text results within `budget` tokens together: each result smaller than an equal share of what is left stays whole, and the
|
|
8
16
|
* rest are clipped to that share, keeping the most head and tail characters that fit it. None when the round already fits, or when a share
|
|
9
17
|
* cannot hold even the locator. A round of one result gets the whole budget. Deterministic: the same round and budget always clip the same way. */
|
package/dist/src/clip.js
CHANGED
|
@@ -9,6 +9,19 @@ const clipped = (text, keep, entryId) => {
|
|
|
9
9
|
`\n[pi-tuck: characters ${head}-${text.length - tail} of ${text.length} are not shown, to fit the context window; read them with ` +
|
|
10
10
|
`tuck_recall {"entryId":"${entryId}","offset":${head}}, up to ${MAX_RECALL_CHARACTERS} characters a call]\n` + text.slice(text.length - tail) }];
|
|
11
11
|
};
|
|
12
|
+
/** The budgets to offer a fresh round the room cannot hold whole, in the units the round and the mandatory input are counted in, tried in order.
|
|
13
|
+
* The room is the provider's, so it is divided by the factor the provider's counts have been running at - the same conversion the planner's own
|
|
14
|
+
* reserve makes (budget.ts) - before the input no cut can remove is taken out; counting the provider's room against an estimate asks the round for
|
|
15
|
+
* more than the room holds at a factor above 1. The first try is half of what that leaves. `kept` is what the smallest cut leaves the request
|
|
16
|
+
* besides the fresh results - the summary floor and its quote of the prompt - measured from the planner's own preview, not a new formula: a half
|
|
17
|
+
* cannot spend it, so the second try is the whole free room less it. Omit it when the planner bailed before building a preview, and only the half
|
|
18
|
+
* is tried. Nothing is offered when the mandatory input alone leaves no room. */
|
|
19
|
+
export const clipBudgets = (room, factor, mandatory, kept) => {
|
|
20
|
+
const free = Math.floor(room / Math.max(1, factor)) - mandatory;
|
|
21
|
+
if (free <= 0)
|
|
22
|
+
return [];
|
|
23
|
+
return [...new Set(kept === undefined ? [Math.floor(free / 2)] : [Math.floor(free / 2), free - kept - 1])].filter(budget => budget > 0);
|
|
24
|
+
};
|
|
12
25
|
/** The round's text results within `budget` tokens together: each result smaller than an equal share of what is left stays whole, and the
|
|
13
26
|
* rest are clipped to that share, keeping the most head and tail characters that fit it. None when the round already fits, or when a share
|
|
14
27
|
* cannot hold even the locator. A round of one result gets the whole budget. Deterministic: the same round and budget always clip the same way. */
|
package/dist/src/compaction.d.ts
CHANGED
|
@@ -6,6 +6,9 @@ import type { RecallPolicy } from "./recall.js";
|
|
|
6
6
|
/** What a capacity refusal tells the operator. When the provider has counted far more input than pi-tuck's estimate, the refusal most
|
|
7
7
|
* likely comes from that counting (an upstream behind a router can spend much more context on tool schemas), so the notice says so. */
|
|
8
8
|
export declare const refusalNotice: (message: string, factor: number) => string;
|
|
9
|
+
/** What the operator hears when the provider refuses a request pi-tuck admitted as over its window: the bound that reply puts on the counting, and
|
|
10
|
+
* what pi-tuck did with it. A bound pi-tuck cannot carry (above the ceiling) says so rather than implying the next request is planned at it. */
|
|
11
|
+
export declare const overflowNotice: (contextWindow: number, bound: number, learned: boolean) => string;
|
|
9
12
|
/** The most a seed or an overflow raise may set the calibration factor to. A paired reply's own ratio is not bounded by it. */
|
|
10
13
|
export declare const CALIBRATION_CEILING = 1.5;
|
|
11
14
|
/** How many recent paired replies (and overflow bounds, and the seed while it is among them) the calibration factor is the maximum of. */
|
package/dist/src/compaction.js
CHANGED
|
@@ -5,7 +5,7 @@ import { exportSource } from "./source-export.js";
|
|
|
5
5
|
import { isDeepStrictEqual } from "node:util";
|
|
6
6
|
import { assertNativeToolRounds } from "./protocol.js";
|
|
7
7
|
import { projectedMessages } from "./omissions.js";
|
|
8
|
-
import { applyClips, clipFor } from "./clip.js";
|
|
8
|
+
import { applyClips, clipBudgets, clipFor } from "./clip.js";
|
|
9
9
|
import { DEFAULT_CUT_AT, bandOf, chooseCut, conversationTokens, decideCut, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, requestTokens, systemPart, validCutAt } from "./budget.js";
|
|
10
10
|
import { isContextOverflow } from "@earendil-works/pi-ai";
|
|
11
11
|
import { convertToLlm, SessionManager, SettingsManager } from "@earendil-works/pi-coding-agent";
|
|
@@ -52,6 +52,10 @@ class CapacityError extends Refusal {
|
|
|
52
52
|
* likely comes from that counting (an upstream behind a router can spend much more context on tool schemas), so the notice says so. */
|
|
53
53
|
export const refusalNotice = (message, factor) => message + "; shorten the input, use a model with a larger window, or lower Pi's compaction.reserveTokens, then retry." +
|
|
54
54
|
(factor >= 1.5 ? ` This provider has counted up to ${factor.toFixed(1)}x the input pi-tuck estimates; through a router such as OpenRouter, a different upstream may count less.` : "");
|
|
55
|
+
/** What the operator hears when the provider refuses a request pi-tuck admitted as over its window: the bound that reply puts on the counting, and
|
|
56
|
+
* what pi-tuck did with it. A bound pi-tuck cannot carry (above the ceiling) says so rather than implying the next request is planned at it. */
|
|
57
|
+
export const overflowNotice = (contextWindow, bound, learned) => `pi-tuck: the provider refused this request as over its ${contextWindow}-token window, so the input counted at least ${bound.toFixed(2)}x what pi-tuck estimated` +
|
|
58
|
+
(learned ? "; the next request is planned at that." : "; too far out to be a low estimate, so the factor is unchanged.");
|
|
55
59
|
/** The most a seed or an overflow raise may set the calibration factor to. A paired reply's own ratio is not bounded by it. */
|
|
56
60
|
export const CALIBRATION_CEILING = 1.5;
|
|
57
61
|
/** How many recent paired replies (and overflow bounds, and the seed while it is among them) the calibration factor is the maximum of. */
|
|
@@ -253,10 +257,14 @@ export async function installCompaction(pi, options = {}) {
|
|
|
253
257
|
return;
|
|
254
258
|
// The provider refused this exact request for exceeding the window: its true count was at least window + 1, a lower bound on the ratio. A
|
|
255
259
|
// bound above the ceiling says the estimate is not merely low, so it is not learned: the overflow ends the attempt as it does without it.
|
|
260
|
+
// Either way the operator hears it from pi-tuck. The estimate is what admitted the request, and a turn that ended in the provider's error
|
|
261
|
+
// with no word from pi-tuck read as a hang: every other refusal says why, and this is the path that skipped it.
|
|
256
262
|
if (overflow) {
|
|
257
|
-
const
|
|
258
|
-
|
|
259
|
-
|
|
263
|
+
const bound = (ctx.model.contextWindow + 1) / paired.tokens;
|
|
264
|
+
const learned = bound <= CALIBRATION_CEILING;
|
|
265
|
+
ctx.ui.notify(overflowNotice(ctx.model.contextWindow, bound, learned), "warning");
|
|
266
|
+
if (learned)
|
|
267
|
+
learn(bound);
|
|
260
268
|
return;
|
|
261
269
|
}
|
|
262
270
|
const usage = message.usage;
|
|
@@ -266,7 +274,10 @@ export async function installCompaction(pi, options = {}) {
|
|
|
266
274
|
if (!counts.every(n => Number.isSafeInteger(n) && n >= 0))
|
|
267
275
|
return;
|
|
268
276
|
const input = counts.reduce((a, b) => a + b, 0), ratio = input / paired.tokens;
|
|
269
|
-
|
|
277
|
+
// A request that was answered cannot have held more than the model's window, so a report above it describes something that never happened.
|
|
278
|
+
// Learning from one sets the factor for the requests after it: a 60,000-token cache read reported in a 32,768-token window made a
|
|
279
|
+
// 6,000-token prompt refuse. The overflow branch above learns from a refusal, where the count is a lower bound, and the ceiling is unchanged.
|
|
280
|
+
if (input > 0 && input <= ctx.model.contextWindow && Number.isSafeInteger(input) && Number.isFinite(ratio))
|
|
270
281
|
learn(ratio);
|
|
271
282
|
});
|
|
272
283
|
pi.on("agent_start", () => { admittedRun = true; pending = undefined; phase = "idle"; });
|
|
@@ -320,7 +331,9 @@ export async function installCompaction(pi, options = {}) {
|
|
|
320
331
|
const clips = new Map();
|
|
321
332
|
const fail = (error, ctx) => {
|
|
322
333
|
recorder.stop("native-attempt-failed");
|
|
323
|
-
|
|
334
|
+
// Every refusal says why, once per attempt: a capacity error with what the user can change, a cut Pi failed to commit with what that
|
|
335
|
+
// means for the session, anything else as it is (an attempt that ended with no word looked like a hang).
|
|
336
|
+
const notify = !failure;
|
|
324
337
|
// A CommitFault stays whatever fails after it.
|
|
325
338
|
if (!(failure instanceof CommitFault))
|
|
326
339
|
failure = error instanceof Error ? error : new Error(String(error));
|
|
@@ -333,7 +346,9 @@ export async function installCompaction(pi, options = {}) {
|
|
|
333
346
|
finally {
|
|
334
347
|
try {
|
|
335
348
|
if (notify)
|
|
336
|
-
ctx.ui.notify(refusalNotice(failure.message, factor)
|
|
349
|
+
ctx.ui.notify(failure instanceof CapacityError ? refusalNotice(failure.message, factor) : failure instanceof CommitFault
|
|
350
|
+
? `pi-tuck: ${failure.message}; the session holds a cut its file may lack, so start a new session.`
|
|
351
|
+
: `pi-tuck could not plan this request (${failure.message}); the next prompt is planned afresh.`, "error");
|
|
337
352
|
}
|
|
338
353
|
finally {
|
|
339
354
|
ctx.abort();
|
|
@@ -544,7 +559,9 @@ export async function installCompaction(pi, options = {}) {
|
|
|
544
559
|
unreduced: "Incoming prompt compaction did not reduce context within the estimated request budget",
|
|
545
560
|
plan: (target) => {
|
|
546
561
|
const reserve = planReserve(target, room, mandatory, ctx);
|
|
547
|
-
|
|
562
|
+
// Only a due cut re-renders a summary that sits alone in the session: inside the band a cut is opportunistic, and a smaller summary
|
|
563
|
+
// there buys nothing the request needed and costs history the next recall would have found.
|
|
564
|
+
const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: before, previousSummary, newPrompt: true, reRenderSummary: !gate.opportunistic }, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, false);
|
|
548
565
|
if (!draft.compaction)
|
|
549
566
|
return { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
|
|
550
567
|
const messages = withCut(ctx, { type: "compaction", ...draft.compaction, tokensBefore: before });
|
|
@@ -595,7 +612,13 @@ export async function installCompaction(pi, options = {}) {
|
|
|
595
612
|
const queued = event.context.pendingMessages.length ? conversationTokens(event.context.pendingMessages) : 0;
|
|
596
613
|
const mandatory = system + host + queued;
|
|
597
614
|
const previousSummary = saved.find(message => message.role === "compactionSummary")?.summary;
|
|
615
|
+
// The cut previews this decision has built, in the order it built them. The clip retry below measures them for what a cut leaves the
|
|
616
|
+
// request besides the fresh results. Reset per decision, so a retry sizes itself against the cut it is retrying, not an earlier attempt's.
|
|
617
|
+
let previews = [];
|
|
618
|
+
let floorKept;
|
|
598
619
|
const decide = () => {
|
|
620
|
+
previews = [];
|
|
621
|
+
floorKept = undefined;
|
|
599
622
|
const before = measure(saved);
|
|
600
623
|
return decideCut({
|
|
601
624
|
total: before + mandatory, mandatory, before, room, band, fits: (tokens, limit) => fits(tokens, limit, ctx),
|
|
@@ -603,8 +626,15 @@ export async function installCompaction(pi, options = {}) {
|
|
|
603
626
|
plan: (target) => {
|
|
604
627
|
const reserve = planReserve(target, room, mandatory, ctx);
|
|
605
628
|
const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: before, previousSummary }, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, true);
|
|
606
|
-
if (!draft.compaction)
|
|
629
|
+
if (!draft.compaction) {
|
|
630
|
+
// The planner's own floor, so a clip's second budget is measured against what the smallest cut really keeps.
|
|
631
|
+
if (draft.floorMessages && event.toolResults) {
|
|
632
|
+
const fresh = new Set(event.toolResults.map(result => result.toolCallId));
|
|
633
|
+
const kept = measure(draft.floorMessages.filter(message => !(message.role === "toolResult" && fresh.has(message.toolCallId))));
|
|
634
|
+
floorKept = Math.min(floorKept ?? kept, kept);
|
|
635
|
+
}
|
|
607
636
|
return { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
|
|
637
|
+
}
|
|
608
638
|
const header = ctx.sessionManager.getHeader();
|
|
609
639
|
if (!header)
|
|
610
640
|
throw new Error("Missing native session header at tool-loop boundary");
|
|
@@ -612,27 +642,36 @@ export async function installCompaction(pi, options = {}) {
|
|
|
612
642
|
preview.appendCompaction(draft.compaction.summary, draft.compaction.firstKeptEntryId, before, draft.compaction.details, true);
|
|
613
643
|
const messages = preview.buildSessionProjection().messages;
|
|
614
644
|
assertNativeToolRounds(messages);
|
|
645
|
+
previews.push(messages);
|
|
615
646
|
return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve };
|
|
616
647
|
},
|
|
617
648
|
});
|
|
618
649
|
};
|
|
619
650
|
let decision = decide();
|
|
620
|
-
// Only a request that would be refused is clipped. A fresh result is kept whole by every cut (it has not reached the model yet), so
|
|
621
|
-
//
|
|
622
|
-
//
|
|
651
|
+
// Only a request that would be refused is clipped. A fresh result is kept whole by every cut (it has not reached the model yet), so a round
|
|
652
|
+
// larger than the room refuses the run; in its place the request carries the result's head and tail with a tuck_recall locator to the rest.
|
|
653
|
+
// clipBudgets states the budgets tried and why they are counted in the estimate's units; `kept` is measured from the previews this decision
|
|
654
|
+
// built, so the second budget is the cut's own floor and not a new formula. When nothing is clipped or the decision still faults, the
|
|
655
|
+
// refusal stands.
|
|
623
656
|
if (decision.kind === "fault" && event.toolResults) {
|
|
624
657
|
const branch = ctx.sessionManager.getBranch();
|
|
625
658
|
const entryIdOf = (toolCallId) => branch.findLast(entry => entry.type === "message" && entry.message.role === "toolResult" && entry.message.toolCallId === toolCallId)?.id;
|
|
626
|
-
const
|
|
627
|
-
|
|
659
|
+
const fresh = new Set(event.toolResults.map(result => result.toolCallId));
|
|
660
|
+
const kept = previews.length ? Math.min(...previews.map(preview => measure(preview.filter(message => !(message.role === "toolResult" && fresh.has(message.toolCallId)))))) : floorKept;
|
|
661
|
+
for (const budget of clipBudgets(room, factor, mandatory, kept)) {
|
|
662
|
+
const round = clipFor(event.toolResults, entryIdOf, budget);
|
|
663
|
+
if (!round.length)
|
|
664
|
+
continue;
|
|
628
665
|
for (const clip of round)
|
|
629
666
|
clips.set(clip.toolCallId, clip);
|
|
630
667
|
const clippedDecision = decide();
|
|
631
668
|
if (clippedDecision.kind === "fault")
|
|
632
669
|
for (const clip of round)
|
|
633
670
|
clips.delete(clip.toolCallId);
|
|
634
|
-
else
|
|
671
|
+
else {
|
|
635
672
|
decision = clippedDecision;
|
|
673
|
+
break;
|
|
674
|
+
}
|
|
636
675
|
}
|
|
637
676
|
}
|
|
638
677
|
const before = measure(saved);
|
|
@@ -24,6 +24,7 @@ export declare function inspectIncident(records: IncidentRecord[]): {
|
|
|
24
24
|
mode?: never;
|
|
25
25
|
unavailable: string;
|
|
26
26
|
capacityRejected?: boolean;
|
|
27
|
+
floorMessages?: import("@earendil-works/pi-agent-core").AgentMessage[];
|
|
27
28
|
};
|
|
28
29
|
receiptId: string | null;
|
|
29
30
|
matchesRecordedProposal: boolean | null;
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { type RecallPolicy } from './recall.js';
|
|
2
2
|
import type { SessionEntry } from '@earendil-works/pi-coding-agent';
|
|
3
|
-
export declare const DIAGNOSTIC_BUILD = "
|
|
3
|
+
export declare const DIAGNOSTIC_BUILD = "828ac4821689607af006d54a41fd5960bb910bc65aa43adbd81c2f6aac25d180";
|
|
4
4
|
export interface Receipt {
|
|
5
5
|
id: string;
|
|
6
6
|
build: string;
|
package/dist/src/tuck.d.ts
CHANGED
|
@@ -21,6 +21,7 @@ type Draft = {
|
|
|
21
21
|
mode?: never;
|
|
22
22
|
unavailable: string;
|
|
23
23
|
capacityRejected?: boolean;
|
|
24
|
+
floorMessages?: AgentMessage[];
|
|
24
25
|
};
|
|
25
26
|
interface SearchHit {
|
|
26
27
|
index: number;
|
|
@@ -84,13 +85,15 @@ export declare function historicalAllowance(window: number, reserve: number, bas
|
|
|
84
85
|
export declare function tuckDraft(tuck: Tuck, event: SessionBeforeCompactEvent, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy?: RecallPolicy, retainCurrentInput?: boolean, operatorManual?: boolean, newPrompt?: boolean): Draft;
|
|
85
86
|
/** Shared compiler observes canonical branch and current token estimate; no native preparation utility. */
|
|
86
87
|
/** `newPrompt`: the cut is planned for a run that has not begun (a prompt Pi has not appended yet, an operator's manual cut, Pi's own cut before
|
|
87
|
-
* a run a custom message starts); the session has moved on from an ended run.
|
|
88
|
+
* a run a custom message starts); the session has moved on from an ended run.
|
|
89
|
+
* `reRenderSummary`: the cut is due, so its summary may be re-rendered smaller (see the `no_live_messages` branch below). */
|
|
88
90
|
export declare function tuckObservedDraft(tuck: Tuck, input: {
|
|
89
91
|
branchEntries: SessionEntry[];
|
|
90
92
|
tokensBefore: number;
|
|
91
93
|
previousSummary?: string;
|
|
92
94
|
operatorManual?: boolean;
|
|
93
95
|
newPrompt?: boolean;
|
|
96
|
+
reRenderSummary?: boolean;
|
|
94
97
|
}, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy?: RecallPolicy, retainCurrentInput?: boolean): Draft;
|
|
95
98
|
/** Numeric references use the original session index, then intersect with readable current-branch entries. */
|
|
96
99
|
export declare function tuckRecall(tuck: Tuck, entries: readonly SessionEntry[], branch: readonly SessionEntry[], input: RecallOptions & {
|
package/dist/src/tuck.js
CHANGED
|
@@ -31,7 +31,8 @@ export function tuckDraft(tuck, event, ctx, reserveTokens, measure, policy = {},
|
|
|
31
31
|
}
|
|
32
32
|
/** Shared compiler observes canonical branch and current token estimate; no native preparation utility. */
|
|
33
33
|
/** `newPrompt`: the cut is planned for a run that has not begun (a prompt Pi has not appended yet, an operator's manual cut, Pi's own cut before
|
|
34
|
-
* a run a custom message starts); the session has moved on from an ended run.
|
|
34
|
+
* a run a custom message starts); the session has moved on from an ended run.
|
|
35
|
+
* `reRenderSummary`: the cut is due, so its summary may be re-rendered smaller (see the `no_live_messages` branch below). */
|
|
35
36
|
export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, policy = {}, retainCurrentInput = true) {
|
|
36
37
|
const { isHistorySource, readable } = createRecall(policy);
|
|
37
38
|
const projected = ctx.sessionManager.buildSessionProjection();
|
|
@@ -62,6 +63,12 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
|
|
|
62
63
|
if (cut.ok)
|
|
63
64
|
cut = tuck.applyTailBudget(entries, cut, { charsPerToken, maxTokens,
|
|
64
65
|
oversizedFactor: Math.min(OVERSIZED_TAIL_FACTOR, inputRoom / Math.max(1, maxTokens)) });
|
|
66
|
+
// Nothing since the last cut: its summary alone is the conversation, and only a cut that is due can re-render it. A summary is compiled from
|
|
67
|
+
// the original sources, so the cut takes the whole history again under this cut's allowance, and a summary that leaves the request no room is
|
|
68
|
+
// made smaller. Inside the band a cut is opportunistic, and re-rendering there shrinks the summary and costs history for nothing (replays
|
|
69
|
+
// s04-B task 13, s08-B task 23: a 13,008-token first request became 3,336), so an opportunistic cut refuses as it did before.
|
|
70
|
+
else if (input.reRenderSummary && cut.reason === "no_live_messages" && entries.some(entry => entry.type === "compaction"))
|
|
71
|
+
cut = { ok: true, messages: [], selectedIds: [], firstKeptEntryId: "", compactAll: true, keptUserTurns: 0, keepFallbackToCompactAll: false };
|
|
65
72
|
if (!cut.ok)
|
|
66
73
|
return { unavailable: `pi-tuck cannot select a cut: ${cut.reason}` };
|
|
67
74
|
// The latest call's trailing receipts have not reached a subsequent response.
|
|
@@ -196,7 +203,8 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
|
|
|
196
203
|
const model = ctx.model;
|
|
197
204
|
const floorAt = (keep) => {
|
|
198
205
|
const ledger = [...actionRecords(actions, keep), ...ledgerRecords(items, keep)];
|
|
199
|
-
|
|
206
|
+
const messages = preview(serializeTranscript([...mandatory, ...ledger]) + DIALOGUE_SUFFIX);
|
|
207
|
+
return { ledger, messages, base: measure(messages) };
|
|
200
208
|
};
|
|
201
209
|
const fits = (base) => Boolean(model) && base + reserveTokens < model.contextWindow && base < before;
|
|
202
210
|
// When the whole floor cannot fit, the trail and the ledger keep only their headings and counts; only that can refuse the cut.
|
|
@@ -205,7 +213,7 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
|
|
|
205
213
|
floor = floorAt(0);
|
|
206
214
|
const { ledger, base } = floor;
|
|
207
215
|
if (!model || !fits(base))
|
|
208
|
-
return { unavailable: "pi-tuck's mandatory context cannot fit or reduce the request budget", capacityRejected: Boolean(model && base + reserveTokens >= model.contextWindow) };
|
|
216
|
+
return { unavailable: "pi-tuck's mandatory context cannot fit or reduce the request budget", capacityRejected: Boolean(model && base + reserveTokens >= model.contextWindow), floorMessages: floor.messages };
|
|
209
217
|
const maxBriefChars = historicalAllowance(model.contextWindow, reserveTokens, base, before);
|
|
210
218
|
const compileInput = { messages, sourceIndices, sourceEntryIds: originalCut.selectedIds, sourceTimes: originalCut.selectedIds.map(id => resolveSourceTime(entryById.get(id))), receiptContext,
|
|
211
219
|
ranking: { maxBriefChars, mandatorySourceIndices: new Set(mandatory.flatMap(r => r.sourceIndex === undefined ? [] : [r.sourceIndex])) }, };
|
|
@@ -244,7 +252,7 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
|
|
|
244
252
|
const { summary, after } = offer ?? assemble(tuck.compileShrunkDialogue(compileInput));
|
|
245
253
|
// Draft reduction concerns conversation. Final request admission owns the effective system/tools.
|
|
246
254
|
if (after + reserveTokens >= model.contextWindow || after >= before)
|
|
247
|
-
return { unavailable: "pi-tuck's draft does not reduce context within the request budget", capacityRejected: after + reserveTokens >= model.contextWindow };
|
|
255
|
+
return { unavailable: "pi-tuck's draft does not reduce context within the request budget", capacityRejected: after + reserveTokens >= model.contextWindow, floorMessages: floor.messages };
|
|
248
256
|
const mode = offer ? "dialogue" : "shrunk";
|
|
249
257
|
return { compaction: { summary, firstKeptEntryId: cut.firstKeptEntryId, tokensBefore: input.tokensBefore, details: { ...details, tokensAfter: after, view: offer ? "complete-dialogue" : "shrunk-dialogue", mode } }, after, mode };
|
|
250
258
|
};
|
package/package.json
CHANGED
package/src/build-id.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export const BUILD_ID = "
|
|
1
|
+
export const BUILD_ID = "828ac4821689607af006d54a41fd5960bb910bc65aa43adbd81c2f6aac25d180";
|
package/src/clip.ts
CHANGED
|
@@ -16,6 +16,19 @@ const clipped = (text: string, keep: number, entryId: string): ToolResultMessage
|
|
|
16
16
|
`tuck_recall {"entryId":"${entryId}","offset":${head}}, up to ${MAX_RECALL_CHARACTERS} characters a call]\n` + text.slice(text.length - tail) }];
|
|
17
17
|
};
|
|
18
18
|
|
|
19
|
+
/** The budgets to offer a fresh round the room cannot hold whole, in the units the round and the mandatory input are counted in, tried in order.
|
|
20
|
+
* The room is the provider's, so it is divided by the factor the provider's counts have been running at - the same conversion the planner's own
|
|
21
|
+
* reserve makes (budget.ts) - before the input no cut can remove is taken out; counting the provider's room against an estimate asks the round for
|
|
22
|
+
* more than the room holds at a factor above 1. The first try is half of what that leaves. `kept` is what the smallest cut leaves the request
|
|
23
|
+
* besides the fresh results - the summary floor and its quote of the prompt - measured from the planner's own preview, not a new formula: a half
|
|
24
|
+
* cannot spend it, so the second try is the whole free room less it. Omit it when the planner bailed before building a preview, and only the half
|
|
25
|
+
* is tried. Nothing is offered when the mandatory input alone leaves no room. */
|
|
26
|
+
export const clipBudgets = (room: number, factor: number, mandatory: number, kept?: number): number[] => {
|
|
27
|
+
const free = Math.floor(room / Math.max(1, factor)) - mandatory;
|
|
28
|
+
if (free <= 0) return [];
|
|
29
|
+
return [...new Set(kept === undefined ? [Math.floor(free / 2)] : [Math.floor(free / 2), free - kept - 1])].filter(budget => budget > 0);
|
|
30
|
+
};
|
|
31
|
+
|
|
19
32
|
/** The round's text results within `budget` tokens together: each result smaller than an equal share of what is left stays whole, and the
|
|
20
33
|
* rest are clipped to that share, keeping the most head and tail characters that fit it. None when the round already fits, or when a share
|
|
21
34
|
* cannot hold even the locator. A round of one result gets the whole budget. Deterministic: the same round and budget always clip the same way. */
|
package/src/compaction.ts
CHANGED
|
@@ -5,10 +5,10 @@ import { exportSource } from "./source-export.js";
|
|
|
5
5
|
import { isDeepStrictEqual } from "node:util";
|
|
6
6
|
import { assertNativeToolRounds } from "./protocol.js";
|
|
7
7
|
import { projectedMessages } from "./omissions.js";
|
|
8
|
-
import { applyClips, clipFor, type Clip } from "./clip.js";
|
|
8
|
+
import { applyClips, clipBudgets, clipFor, type Clip } from "./clip.js";
|
|
9
9
|
import { DEFAULT_CUT_AT, bandOf, chooseCut, conversationTokens, decideCut, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, requestTokens, systemPart, validCutAt,
|
|
10
10
|
type AgentMessage, type Messages, type Plan, type Target, type ToolShape } from "./budget.js";
|
|
11
|
-
import { isContextOverflow } from "@earendil-works/pi-ai";
|
|
11
|
+
import { isContextOverflow, type ToolResultMessage } from "@earendil-works/pi-ai";
|
|
12
12
|
import { convertToLlm, SessionManager, SettingsManager, type ExtensionAPI, type ExtensionContext, type SessionEntry } from "@earendil-works/pi-coding-agent";
|
|
13
13
|
import { Type } from "typebox";
|
|
14
14
|
import { Check } from "typebox/value";
|
|
@@ -54,6 +54,11 @@ class CapacityError extends Refusal {}
|
|
|
54
54
|
export const refusalNotice = (message: string, factor: number): string =>
|
|
55
55
|
message + "; shorten the input, use a model with a larger window, or lower Pi's compaction.reserveTokens, then retry." +
|
|
56
56
|
(factor >= 1.5 ? ` This provider has counted up to ${factor.toFixed(1)}x the input pi-tuck estimates; through a router such as OpenRouter, a different upstream may count less.` : "");
|
|
57
|
+
/** What the operator hears when the provider refuses a request pi-tuck admitted as over its window: the bound that reply puts on the counting, and
|
|
58
|
+
* what pi-tuck did with it. A bound pi-tuck cannot carry (above the ceiling) says so rather than implying the next request is planned at it. */
|
|
59
|
+
export const overflowNotice = (contextWindow: number, bound: number, learned: boolean): string =>
|
|
60
|
+
`pi-tuck: the provider refused this request as over its ${contextWindow}-token window, so the input counted at least ${bound.toFixed(2)}x what pi-tuck estimated` +
|
|
61
|
+
(learned ? "; the next request is planned at that." : "; too far out to be a low estimate, so the factor is unchanged.");
|
|
57
62
|
/** The most a seed or an overflow raise may set the calibration factor to. A paired reply's own ratio is not bounded by it. */
|
|
58
63
|
export const CALIBRATION_CEILING = 1.5;
|
|
59
64
|
/** How many recent paired replies (and overflow bounds, and the seed while it is among them) the calibration factor is the maximum of. */
|
|
@@ -232,13 +237,24 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
232
237
|
!isDeepStrictEqual(convertToLlm(paired.messages), paired.snapshot)) return;
|
|
233
238
|
// The provider refused this exact request for exceeding the window: its true count was at least window + 1, a lower bound on the ratio. A
|
|
234
239
|
// bound above the ceiling says the estimate is not merely low, so it is not learned: the overflow ends the attempt as it does without it.
|
|
235
|
-
|
|
240
|
+
// Either way the operator hears it from pi-tuck. The estimate is what admitted the request, and a turn that ended in the provider's error
|
|
241
|
+
// with no word from pi-tuck read as a hang: every other refusal says why, and this is the path that skipped it.
|
|
242
|
+
if (overflow) {
|
|
243
|
+
const bound = (ctx.model.contextWindow + 1) / paired.tokens;
|
|
244
|
+
const learned = bound <= CALIBRATION_CEILING;
|
|
245
|
+
ctx.ui.notify(overflowNotice(ctx.model.contextWindow, bound, learned), "warning");
|
|
246
|
+
if (learned) learn(bound);
|
|
247
|
+
return;
|
|
248
|
+
}
|
|
236
249
|
const usage = message.usage;
|
|
237
250
|
if (!usage) return;
|
|
238
251
|
const counts = [usage.input, usage.cacheRead, usage.cacheWrite];
|
|
239
252
|
if (!counts.every(n => Number.isSafeInteger(n) && n >= 0)) return;
|
|
240
253
|
const input = counts.reduce((a, b) => a + b, 0), ratio = input / paired.tokens;
|
|
241
|
-
|
|
254
|
+
// A request that was answered cannot have held more than the model's window, so a report above it describes something that never happened.
|
|
255
|
+
// Learning from one sets the factor for the requests after it: a 60,000-token cache read reported in a 32,768-token window made a
|
|
256
|
+
// 6,000-token prompt refuse. The overflow branch above learns from a refusal, where the count is a lower bound, and the ceiling is unchanged.
|
|
257
|
+
if (input > 0 && input <= ctx.model.contextWindow && Number.isSafeInteger(input) && Number.isFinite(ratio)) learn(ratio);
|
|
242
258
|
});
|
|
243
259
|
pi.on("agent_start", () => { admittedRun = true; pending = undefined; phase = "idle"; });
|
|
244
260
|
pi.on("agent_settled", () => { admittedRun = false; pending = undefined; phase = "idle"; forgetFailure(); clips.clear(); promptCut = undefined; });
|
|
@@ -295,14 +311,18 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
295
311
|
const clips = new Map<string, Clip>();
|
|
296
312
|
const fail = (error: unknown, ctx: ExtensionContext) => {
|
|
297
313
|
recorder.stop("native-attempt-failed");
|
|
298
|
-
|
|
314
|
+
// Every refusal says why, once per attempt: a capacity error with what the user can change, a cut Pi failed to commit with what that
|
|
315
|
+
// means for the session, anything else as it is (an attempt that ended with no word looked like a hang).
|
|
316
|
+
const notify = !failure;
|
|
299
317
|
// A CommitFault stays whatever fails after it.
|
|
300
318
|
if (!(failure instanceof CommitFault)) failure = error instanceof Error ? error : new Error(String(error));
|
|
301
319
|
pendingBoundaryCommit = false;
|
|
302
320
|
pending = undefined;
|
|
303
321
|
promptCut = undefined;
|
|
304
322
|
try { options.fail?.(error); } finally {
|
|
305
|
-
try { if (notify) ctx.ui.notify(refusalNotice(failure.message, factor)
|
|
323
|
+
try { if (notify) ctx.ui.notify(failure instanceof CapacityError ? refusalNotice(failure.message, factor) : failure instanceof CommitFault
|
|
324
|
+
? `pi-tuck: ${failure.message}; the session holds a cut its file may lack, so start a new session.`
|
|
325
|
+
: `pi-tuck could not plan this request (${failure.message}); the next prompt is planned afresh.`, "error"); }
|
|
306
326
|
finally { ctx.abort(); }
|
|
307
327
|
}
|
|
308
328
|
};
|
|
@@ -472,7 +492,9 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
472
492
|
unreduced: "Incoming prompt compaction did not reduce context within the estimated request budget",
|
|
473
493
|
plan: (target): Plan<{ compaction: NonNullable<ReturnType<typeof tuckObservedDraft>["compaction"]>; reserve: number }> => {
|
|
474
494
|
const reserve = planReserve(target, room, mandatory, ctx);
|
|
475
|
-
|
|
495
|
+
// Only a due cut re-renders a summary that sits alone in the session: inside the band a cut is opportunistic, and a smaller summary
|
|
496
|
+
// there buys nothing the request needed and costs history the next recall would have found.
|
|
497
|
+
const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: before, previousSummary, newPrompt: true, reRenderSummary: !gate.opportunistic }, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, false);
|
|
476
498
|
if (!draft.compaction) return { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
|
|
477
499
|
const messages = withCut(ctx, { type: "compaction", ...draft.compaction, tokensBefore: before });
|
|
478
500
|
assertNativeToolRounds(messages);
|
|
@@ -515,7 +537,13 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
515
537
|
const queued = event.context.pendingMessages.length ? conversationTokens(event.context.pendingMessages as Messages) : 0;
|
|
516
538
|
const mandatory = system + host + queued;
|
|
517
539
|
const previousSummary = saved.find(message => message.role === "compactionSummary")?.summary;
|
|
540
|
+
// The cut previews this decision has built, in the order it built them. The clip retry below measures them for what a cut leaves the
|
|
541
|
+
// request besides the fresh results. Reset per decision, so a retry sizes itself against the cut it is retrying, not an earlier attempt's.
|
|
542
|
+
let previews: Messages[] = [];
|
|
543
|
+
let floorKept: number | undefined;
|
|
518
544
|
const decide = () => {
|
|
545
|
+
previews = [];
|
|
546
|
+
floorKept = undefined;
|
|
519
547
|
const before = measure(saved);
|
|
520
548
|
return decideCut({
|
|
521
549
|
total: before + mandatory, mandatory, before, room, band, fits: (tokens, limit) => fits(tokens, limit, ctx),
|
|
@@ -523,29 +551,46 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
523
551
|
plan: (target): Plan<{ compaction: NonNullable<ReturnType<typeof tuckObservedDraft>["compaction"]>; reserve: number }> => {
|
|
524
552
|
const reserve = planReserve(target, room, mandatory, ctx);
|
|
525
553
|
const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: before, previousSummary }, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, true);
|
|
526
|
-
if (!draft.compaction)
|
|
554
|
+
if (!draft.compaction) {
|
|
555
|
+
// The planner's own floor, so a clip's second budget is measured against what the smallest cut really keeps.
|
|
556
|
+
if (draft.floorMessages && event.toolResults) {
|
|
557
|
+
const fresh = new Set(event.toolResults.map(result => result.toolCallId));
|
|
558
|
+
const kept = measure(draft.floorMessages.filter(message =>
|
|
559
|
+
!(message.role === "toolResult" && fresh.has((message as ToolResultMessage).toolCallId))));
|
|
560
|
+
floorKept = Math.min(floorKept ?? kept, kept);
|
|
561
|
+
}
|
|
562
|
+
return { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
|
|
563
|
+
}
|
|
527
564
|
const header = ctx.sessionManager.getHeader();
|
|
528
565
|
if (!header) throw new Error("Missing native session header at tool-loop boundary");
|
|
529
566
|
const preview = SessionManager.inMemory(ctx.cwd, undefined, [header, ...ctx.sessionManager.getBranch()]);
|
|
530
567
|
preview.appendCompaction(draft.compaction.summary, draft.compaction.firstKeptEntryId, before, draft.compaction.details, true);
|
|
531
568
|
const messages = preview.buildSessionProjection().messages;
|
|
532
569
|
assertNativeToolRounds(messages);
|
|
570
|
+
previews.push(messages);
|
|
533
571
|
return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve };
|
|
534
572
|
},
|
|
535
573
|
});
|
|
536
574
|
};
|
|
537
575
|
let decision = decide();
|
|
538
|
-
// Only a request that would be refused is clipped. A fresh result is kept whole by every cut (it has not reached the model yet), so
|
|
539
|
-
//
|
|
540
|
-
//
|
|
576
|
+
// Only a request that would be refused is clipped. A fresh result is kept whole by every cut (it has not reached the model yet), so a round
|
|
577
|
+
// larger than the room refuses the run; in its place the request carries the result's head and tail with a tuck_recall locator to the rest.
|
|
578
|
+
// clipBudgets states the budgets tried and why they are counted in the estimate's units; `kept` is measured from the previews this decision
|
|
579
|
+
// built, so the second budget is the cut's own floor and not a new formula. When nothing is clipped or the decision still faults, the
|
|
580
|
+
// refusal stands.
|
|
541
581
|
if (decision.kind === "fault" && event.toolResults) {
|
|
542
582
|
const branch = ctx.sessionManager.getBranch();
|
|
543
583
|
const entryIdOf = (toolCallId: string) => branch.findLast(entry => entry.type === "message" && entry.message.role === "toolResult" && entry.message.toolCallId === toolCallId)?.id;
|
|
544
|
-
const
|
|
545
|
-
|
|
584
|
+
const fresh = new Set(event.toolResults.map(result => result.toolCallId));
|
|
585
|
+
const kept = previews.length ? Math.min(...previews.map(preview => measure(preview.filter(message =>
|
|
586
|
+
!(message.role === "toolResult" && fresh.has((message as ToolResultMessage).toolCallId)))))) : floorKept;
|
|
587
|
+
for (const budget of clipBudgets(room, factor, mandatory, kept)) {
|
|
588
|
+
const round = clipFor(event.toolResults, entryIdOf, budget);
|
|
589
|
+
if (!round.length) continue;
|
|
546
590
|
for (const clip of round) clips.set(clip.toolCallId, clip);
|
|
547
591
|
const clippedDecision = decide();
|
|
548
|
-
if (clippedDecision.kind === "fault") for (const clip of round) clips.delete(clip.toolCallId);
|
|
592
|
+
if (clippedDecision.kind === "fault") for (const clip of round) clips.delete(clip.toolCallId);
|
|
593
|
+
else { decision = clippedDecision; break; }
|
|
549
594
|
}
|
|
550
595
|
}
|
|
551
596
|
const before = measure(saved);
|
package/src/tuck.ts
CHANGED
|
@@ -18,7 +18,8 @@ export type Measure = (messages: AgentMessage[]) => number;
|
|
|
18
18
|
/** The summary view that produced a draft: the whole pre-cut dialogue, or the same dialogue shrunk to the allowance. Both carry the file ledger. */
|
|
19
19
|
export type DraftMode = "dialogue" | "shrunk";
|
|
20
20
|
type Draft = { compaction: Pick<Extract<SessionEntry, { type: "compaction" }>, "summary" | "firstKeptEntryId" | "tokensBefore" | "details">; after: number; mode: DraftMode; unavailable?: never; capacityRejected?: never }
|
|
21
|
-
|
|
21
|
+
// The actual floor projection lets the caller measure it without the fresh results it may clip.
|
|
22
|
+
| { compaction?: never; mode?: never; unavailable: string; capacityRejected?: boolean; floorMessages?: AgentMessage[] };
|
|
22
23
|
interface SearchHit { index: number; role: string; summary: string }
|
|
23
24
|
export interface RecallContinuation { query: string; throughEntryId: string; offset: number }
|
|
24
25
|
/** Includes JSON escaping, hit metadata and continuation, not just excerpt text. */
|
|
@@ -63,8 +64,9 @@ export function tuckDraft(tuck: Tuck, event: SessionBeforeCompactEvent, ctx: Ext
|
|
|
63
64
|
}
|
|
64
65
|
/** Shared compiler observes canonical branch and current token estimate; no native preparation utility. */
|
|
65
66
|
/** `newPrompt`: the cut is planned for a run that has not begun (a prompt Pi has not appended yet, an operator's manual cut, Pi's own cut before
|
|
66
|
-
* a run a custom message starts); the session has moved on from an ended run.
|
|
67
|
-
|
|
67
|
+
* a run a custom message starts); the session has moved on from an ended run.
|
|
68
|
+
* `reRenderSummary`: the cut is due, so its summary may be re-rendered smaller (see the `no_live_messages` branch below). */
|
|
69
|
+
export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEntry[]; tokensBefore: number; previousSummary?: string; operatorManual?: boolean; newPrompt?: boolean; reRenderSummary?: boolean }, ctx: ExtensionContext,
|
|
68
70
|
reserveTokens: number, measure: Measure, policy: RecallPolicy = {}, retainCurrentInput = true): Draft {
|
|
69
71
|
const { isHistorySource, readable } = createRecall(policy);
|
|
70
72
|
const projected = ctx.sessionManager.buildSessionProjection();
|
|
@@ -92,6 +94,12 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
|
|
|
92
94
|
let cut = tuck.buildOwnCut(entries, 1);
|
|
93
95
|
if (cut.ok) cut = tuck.applyTailBudget(entries, cut, { charsPerToken, maxTokens,
|
|
94
96
|
oversizedFactor: Math.min(OVERSIZED_TAIL_FACTOR, inputRoom / Math.max(1, maxTokens)) });
|
|
97
|
+
// Nothing since the last cut: its summary alone is the conversation, and only a cut that is due can re-render it. A summary is compiled from
|
|
98
|
+
// the original sources, so the cut takes the whole history again under this cut's allowance, and a summary that leaves the request no room is
|
|
99
|
+
// made smaller. Inside the band a cut is opportunistic, and re-rendering there shrinks the summary and costs history for nothing (replays
|
|
100
|
+
// s04-B task 13, s08-B task 23: a 13,008-token first request became 3,336), so an opportunistic cut refuses as it did before.
|
|
101
|
+
else if (input.reRenderSummary && cut.reason === "no_live_messages" && entries.some(entry => entry.type === "compaction"))
|
|
102
|
+
cut = { ok: true, messages: [], selectedIds: [], firstKeptEntryId: "", compactAll: true, keptUserTurns: 0, keepFallbackToCompactAll: false };
|
|
95
103
|
if (!cut.ok) return { unavailable: `pi-tuck cannot select a cut: ${cut.reason}` };
|
|
96
104
|
// The latest call's trailing receipts have not reached a subsequent response.
|
|
97
105
|
// A preferred tail size must not erase this first exposure when the whole
|
|
@@ -213,7 +221,8 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
|
|
|
213
221
|
const model = ctx.model;
|
|
214
222
|
const floorAt = (keep: number) => {
|
|
215
223
|
const ledger = [...actionRecords(actions, keep), ...ledgerRecords(items, keep)];
|
|
216
|
-
|
|
224
|
+
const messages = preview(serializeTranscript([...mandatory, ...ledger]) + DIALOGUE_SUFFIX);
|
|
225
|
+
return { ledger, messages, base: measure(messages) };
|
|
217
226
|
};
|
|
218
227
|
const fits = (base: number) => Boolean(model) && base + reserveTokens < model!.contextWindow && base < before;
|
|
219
228
|
// When the whole floor cannot fit, the trail and the ledger keep only their headings and counts; only that can refuse the cut.
|
|
@@ -221,7 +230,7 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
|
|
|
221
230
|
if (!fits(floor.base) && (items.length || actions.length)) floor = floorAt(0);
|
|
222
231
|
const { ledger, base } = floor;
|
|
223
232
|
if (!model || !fits(base))
|
|
224
|
-
return { unavailable: "pi-tuck's mandatory context cannot fit or reduce the request budget", capacityRejected: Boolean(model && base + reserveTokens >= model.contextWindow) };
|
|
233
|
+
return { unavailable: "pi-tuck's mandatory context cannot fit or reduce the request budget", capacityRejected: Boolean(model && base + reserveTokens >= model.contextWindow), floorMessages: floor.messages };
|
|
225
234
|
const maxBriefChars = historicalAllowance(model.contextWindow, reserveTokens, base, before);
|
|
226
235
|
const compileInput = { messages, sourceIndices, sourceEntryIds: originalCut.selectedIds, sourceTimes: originalCut.selectedIds.map(id => resolveSourceTime(entryById.get(id)!)), receiptContext,
|
|
227
236
|
ranking: { maxBriefChars, mandatorySourceIndices: new Set(mandatory.flatMap(r => r.sourceIndex === undefined ? [] : [r.sourceIndex])) },
|
|
@@ -259,7 +268,7 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
|
|
|
259
268
|
const { summary, after } = offer ?? assemble(tuck.compileShrunkDialogue(compileInput));
|
|
260
269
|
// Draft reduction concerns conversation. Final request admission owns the effective system/tools.
|
|
261
270
|
if (after + reserveTokens >= model.contextWindow || after >= before)
|
|
262
|
-
return { unavailable: "pi-tuck's draft does not reduce context within the request budget", capacityRejected: after + reserveTokens >= model.contextWindow };
|
|
271
|
+
return { unavailable: "pi-tuck's draft does not reduce context within the request budget", capacityRejected: after + reserveTokens >= model.contextWindow, floorMessages: floor.messages };
|
|
263
272
|
const mode: DraftMode = offer ? "dialogue" : "shrunk";
|
|
264
273
|
return { compaction: { summary, firstKeptEntryId: cut.firstKeptEntryId, tokensBefore: input.tokensBefore, details: { ...details, tokensAfter: after, view: offer ? "complete-dialogue" : "shrunk-dialogue", mode } }, after, mode };
|
|
265
274
|
};
|