pi-tuck 1.0.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -0
- package/README.md +8 -5
- package/dist/index.d.ts +2 -1
- package/dist/index.js +7 -4
- package/dist/src/budget.d.ts +17 -4
- package/dist/src/budget.js +33 -10
- package/dist/src/build-id.d.ts +1 -1
- package/dist/src/build-id.js +1 -1
- package/dist/src/compaction.d.ts +4 -0
- package/dist/src/compaction.js +8 -5
- package/dist/src/diagnostics.d.ts +2 -1
- package/dist/src/omissions.d.ts +1 -1
- package/dist/src/omissions.js +8 -8
- package/dist/src/tuck/core/brief.d.ts +13 -43
- package/dist/src/tuck/core/brief.js +39 -255
- package/dist/src/tuck/core/format.d.ts +2 -6
- package/dist/src/tuck/core/format.js +4 -10
- package/dist/src/tuck/core/normalize.js +20 -43
- package/dist/src/tuck/core/shrink.d.ts +33 -0
- package/dist/src/tuck/core/shrink.js +221 -0
- package/dist/src/tuck/core/summarize.d.ts +15 -7
- package/dist/src/tuck/core/summarize.js +10 -23
- package/dist/src/tuck/core/transcript.d.ts +24 -13
- package/dist/src/tuck/core/transcript.js +243 -83
- package/dist/src/tuck/types.d.ts +4 -14
- package/dist/src/tuck.d.ts +8 -10
- package/dist/src/tuck.js +23 -25
- package/index.ts +8 -5
- package/package.json +1 -1
- package/src/budget.ts +29 -9
- package/src/build-id.ts +1 -1
- package/src/compaction.ts +11 -5
- package/src/diagnostics.ts +1 -1
- package/src/omissions.ts +8 -8
- package/src/tuck/core/brief.ts +46 -265
- package/src/tuck/core/format.ts +4 -11
- package/src/tuck/core/normalize.ts +20 -43
- package/src/tuck/core/shrink.ts +218 -0
- package/src/tuck/core/summarize.ts +20 -25
- package/src/tuck/core/transcript.ts +234 -82
- package/src/tuck/types.ts +7 -19
- package/src/tuck.ts +30 -32
- package/dist/src/tuck/core/brief-meter.d.ts +0 -42
- package/dist/src/tuck/core/brief-meter.js +0 -273
- package/dist/src/tuck/core/rank.d.ts +0 -42
- package/dist/src/tuck/core/rank.js +0 -292
- package/src/tuck/core/brief-meter.ts +0 -235
- package/src/tuck/core/rank.ts +0 -331
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,47 @@
|
|
|
3
3
|
All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
|
|
4
4
|
[Semantic Versioning](https://semver.org/).
|
|
5
5
|
|
|
6
|
+
## [1.0.3] - 2026-10-31
|
|
7
|
+
|
|
8
|
+
First npm release since 1.0.1: it also carries everything listed under 1.0.2, which was tagged but not published to npm. Releases are
|
|
9
|
+
now published from GitHub Actions by npm trusted publishing, with provenance.
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- `"cutAt"` in `pi-tuck.json` (and the `cutAt` option of `installCompaction`) sets the fraction of the input room at which a cut is triggered and
|
|
14
|
+
planned: a number above 0 and at most 1. The room still decides every refusal. A cut's receipt records `cutAt` when it is not 1.
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
|
|
18
|
+
- Cuts happen once a request passes 60% of the input room (`cutAt` 0.6) instead of just under the whole room. Each cut leaves room to grow,
|
|
19
|
+
so fewer tokens are carried per request: in paired replays against cutting at the full room, answers held and billed input fell about 17%
|
|
20
|
+
at a 128k window and 10-14% at 32k, with prompt caching automatic or by explicit breakpoints. `{"cutAt": 1}` restores the earlier point.
|
|
21
|
+
|
|
22
|
+
## [1.0.2] - 2026-10-31
|
|
23
|
+
|
|
24
|
+
Tagged, not published to npm; 1.0.3 ships it.
|
|
25
|
+
|
|
26
|
+
Measured before release with paired replays against 1.0.1 (same sessions, same model as agent and judge): no loss of answer accuracy at
|
|
27
|
+
a 128k or a 32k window, and time questions answered as before.
|
|
28
|
+
|
|
29
|
+
### Changed
|
|
30
|
+
|
|
31
|
+
- A dialogue that does not fit the summary is now shortened, oldest assistant text first, instead of replaced by a ranked selection of
|
|
32
|
+
records. The newest assistant text that fits a third of the allowance is kept until user prompts have given way; a clip that would keep
|
|
33
|
+
no text is counted in one omission notice instead.
|
|
34
|
+
- Records name the time of user turns only; assistant, tool and notice records no longer carry one. Recall still returns every
|
|
35
|
+
source's time.
|
|
36
|
+
- Summaries are headed `TUCK-TRANSCRIPT/2` and hold one line of plain text per record (`#12 user @2026-10-30T14:00:00Z: text`) instead
|
|
37
|
+
of one JSON object, which makes them about 30 percent shorter on sessions where the assistant writes little and about 14 percent on
|
|
38
|
+
prose-heavy ones. A passage a clip leaves out names the call that reads it back (`tuck_recall {"index":12,"offset":523}`). Summaries
|
|
39
|
+
saved by 1.0.x (`TUCK-TRANSCRIPT/1`, and `VCC-TRANSCRIPT/1` before them) are read as before and keep their format when a native
|
|
40
|
+
omission filters them.
|
|
41
|
+
|
|
42
|
+
### Fixed
|
|
43
|
+
|
|
44
|
+
- In small windows a session that writes little prose cut again almost every tool round (174 cuts in 300 rounds at 32k), because the
|
|
45
|
+
recent turns kept native could fill the whole input room. They now take at most half of it.
|
|
46
|
+
|
|
6
47
|
## [1.0.1] - 2026-10-30
|
|
7
48
|
|
|
8
49
|
First release on npm. 1.0.0 was tagged but never published; its notes below describe what 1.0.1 ships besides these changes.
|
|
@@ -62,6 +103,8 @@ Tagged, not published.
|
|
|
62
103
|
Source preview on GitHub, not published to npm. Deterministic compaction with `pi_tuck_recall` and `pi_tuck_search`; semantic search
|
|
63
104
|
opt-in, with a runtime the user had to supply.
|
|
64
105
|
|
|
106
|
+
[1.0.3]: https://github.com/galyavin/pi-tuck/releases/tag/v1.0.3
|
|
107
|
+
[1.0.2]: https://github.com/galyavin/pi-tuck/tree/v1.0.2
|
|
65
108
|
[1.0.1]: https://github.com/galyavin/pi-tuck/releases/tag/v1.0.1
|
|
66
109
|
[1.0.0]: https://github.com/galyavin/pi-tuck/tree/v1.0.0
|
|
67
110
|
[0.2.1]: https://github.com/galyavin/pi-tuck/tree/e00fa99
|
package/README.md
CHANGED
|
@@ -8,13 +8,13 @@ by meaning.
|
|
|
8
8
|
## Install
|
|
9
9
|
|
|
10
10
|
```sh
|
|
11
|
-
pi install npm:pi-tuck@1.0.
|
|
11
|
+
pi install npm:pi-tuck@1.0.3
|
|
12
12
|
```
|
|
13
13
|
|
|
14
14
|
or straight from GitHub:
|
|
15
15
|
|
|
16
16
|
```sh
|
|
17
|
-
pi install git:github.com/galyavin/pi-tuck@v1.0.
|
|
17
|
+
pi install git:github.com/galyavin/pi-tuck@v1.0.3
|
|
18
18
|
```
|
|
19
19
|
|
|
20
20
|
Requires Pi 1.1 or later (tested on 1.1.0 and 1.4.0) on macOS or Linux. Semantic search (`tuck_search` by meaning) runs only on macOS
|
|
@@ -30,9 +30,10 @@ Start a new Pi session after installing.
|
|
|
30
30
|
- the agent's other tool calls, such as shell commands, newest first (up to 30), without their output but with pointers to it;
|
|
31
31
|
- your first prompt in the session, word for word (up to 12,000 characters).
|
|
32
32
|
|
|
33
|
-
Tool output is left out of the summary; it stays in the session file. If the dialogue itself is too long for the room left,
|
|
34
|
-
|
|
35
|
-
and look things up rather than guess.
|
|
33
|
+
Tool output is left out of the summary; it stays in the session file. If the dialogue itself is too long for the room left, it is
|
|
34
|
+
shortened, oldest assistant text first, instead of being replaced by a ranked selection. Each summary ends with a short instruction to
|
|
35
|
+
the agent: re-read a file before editing it, and look things up rather than guess. The summary is plain text, one line per record
|
|
36
|
+
(`#12 user: ...`, where 12 is the index `tuck_recall` takes); summaries written by earlier versions are still read.
|
|
36
37
|
|
|
37
38
|
**`tuck_recall`** finds earlier text by keyword and reads any original message or tool result exactly, by the index shown in a summary.
|
|
38
39
|
It can also write an original back out to a file.
|
|
@@ -58,6 +59,8 @@ with a window of 32k tokens or less, lower the reserve for that model in Pi's `s
|
|
|
58
59
|
{ "compaction": { "modelOverrides": { "my-provider/my-model": { "reserveTokens": 4096 } } } }
|
|
59
60
|
```
|
|
60
61
|
|
|
62
|
+
**Cut point.** pi-tuck cuts once a request passes 60% of the input room, so each cut leaves room to grow and fewer tokens are carried (and billed) per request; measured against cutting at the full room, answers held and billed input fell 10-18%. Set `"cutAt"` in `pi-tuck.json` in Pi's agent folder (`~/.pi/agent`) to change it, a number above 0 and at most 1: `{ "cutAt": 1 }` restores cutting at the full room, a lower value cuts earlier and smaller. The guard is unaffected, because the whole room still decides which request is refused.
|
|
63
|
+
|
|
61
64
|
## Semantic search
|
|
62
65
|
|
|
63
66
|
Semantic search is on by default and runs entirely on your Mac: an embedding model,
|
package/dist/index.d.ts
CHANGED
|
@@ -6,5 +6,6 @@ export { createSemanticRecall, SemanticRecallController } from "./src/semantic-r
|
|
|
6
6
|
export type { SemanticRecallConfig, PriorResult, SemanticState } from "./src/semantic-recall.js";
|
|
7
7
|
export { ensureSemanticRuntime } from "./src/setup.js";
|
|
8
8
|
/** pi-tuck as a Pi extension: compaction and history recall always; semantic recall too, unless `pi-tuck.json` in Pi's agent folder
|
|
9
|
-
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts.
|
|
9
|
+
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts. The same file's `"cutAt"`
|
|
10
|
+
* (a number above 0 and at most 1) is the fraction of the input room a cut is triggered at, 1 by default. */
|
|
10
11
|
export default function (pi: ExtensionAPI): Promise<void>;
|
package/dist/index.js
CHANGED
|
@@ -2,6 +2,7 @@ import { getAgentDir } from "@earendil-works/pi-coding-agent";
|
|
|
2
2
|
import { readFileSync, realpathSync } from "node:fs";
|
|
3
3
|
import { join, sep } from "node:path";
|
|
4
4
|
import { installCompaction } from "./src/compaction.js";
|
|
5
|
+
import { validCutAt } from "./src/budget.js";
|
|
5
6
|
import { createSemanticRecall } from "./src/semantic-recall.js";
|
|
6
7
|
import { defaultRoot } from "./src/assets.js";
|
|
7
8
|
import { ensureSemanticRuntime } from "./src/setup.js";
|
|
@@ -10,13 +11,14 @@ export { inspectIncident, readableIncident } from "./src/diagnostic-replay.js";
|
|
|
10
11
|
export { createSemanticRecall, SemanticRecallController } from "./src/semantic-recall.js";
|
|
11
12
|
export { ensureSemanticRuntime } from "./src/setup.js";
|
|
12
13
|
/** pi-tuck as a Pi extension: compaction and history recall always; semantic recall too, unless `pi-tuck.json` in Pi's agent folder
|
|
13
|
-
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts.
|
|
14
|
+
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts. The same file's `"cutAt"`
|
|
15
|
+
* (a number above 0 and at most 1) is the fraction of the input room a cut is triggered at, 1 by default. */
|
|
14
16
|
export default async function (pi) {
|
|
15
17
|
const semanticRecall = createSemanticRecall();
|
|
16
18
|
// `/tuck-semantic off` holds for the rest of the session, including over a setup that finishes later.
|
|
17
19
|
let off = false;
|
|
18
|
-
const handle = await installCompaction(pi, { semanticRecall, onSemanticSetting: async (enabled) => { off = !enabled; } });
|
|
19
20
|
const settings = readSettings(join(getAgentDir(), "pi-tuck.json"));
|
|
21
|
+
const handle = await installCompaction(pi, { semanticRecall, cutAt: settings.cutAt, onSemanticSetting: async (enabled) => { off = !enabled; } });
|
|
20
22
|
if (settings.semanticRecall === false)
|
|
21
23
|
return;
|
|
22
24
|
let session = 0, said = "", setup;
|
|
@@ -74,11 +76,12 @@ function readSettings(path) {
|
|
|
74
76
|
}
|
|
75
77
|
try {
|
|
76
78
|
const value = JSON.parse(text);
|
|
77
|
-
if (value && typeof value === "object" && (value.semanticRecall === undefined || typeof value.semanticRecall === "boolean")
|
|
79
|
+
if (value && typeof value === "object" && (value.semanticRecall === undefined || typeof value.semanticRecall === "boolean") &&
|
|
80
|
+
(value.cutAt === undefined || validCutAt(value.cutAt)))
|
|
78
81
|
return value;
|
|
79
82
|
}
|
|
80
83
|
catch { /* reported below */ }
|
|
81
|
-
return { error: `${path} should be a JSON object like {"semanticRecall": false}` };
|
|
84
|
+
return { error: `${path} should be a JSON object like {"semanticRecall": false, "cutAt": 0.8}, with cutAt a number above 0 and at most 1` };
|
|
82
85
|
}
|
|
83
86
|
const reason = (error) => error instanceof Error ? error.message : String(error);
|
|
84
87
|
const within = (path, folder) => path === folder || path.startsWith(folder.endsWith(sep) ? folder : folder + sep);
|
package/dist/src/budget.d.ts
CHANGED
|
@@ -16,8 +16,17 @@ export interface ToolShape {
|
|
|
16
16
|
}
|
|
17
17
|
/** Triggers fire this fraction of the room early. The margin absorbs request growth a boundary cannot see; it never enters the calibration factor. */
|
|
18
18
|
export declare const TRIGGER_MARGIN_FRACTION = 0.02;
|
|
19
|
-
/**
|
|
20
|
-
export declare const
|
|
19
|
+
/** Whether `value` is a cut point: a fraction of the room above 0 and at most 1. */
|
|
20
|
+
export declare const validCutAt: (value: unknown) => value is number;
|
|
21
|
+
/**
|
|
22
|
+
* The one owner of the band: the limit a cut is triggered at and preferably planned to. It is the room less the margin, or the cut point
|
|
23
|
+
* `cutAt` (a fraction of the room) when that is lower. The exact room stays the limit of every throw, fault and the complete-context guard,
|
|
24
|
+
* whatever the cut point. With `cutAt` 1 the band is the room less the margin alone.
|
|
25
|
+
*/
|
|
26
|
+
/** The default cut point: a cut once a request passes 60% of the room. Measured against 1 with paired replays (no answer loss) and billed
|
|
27
|
+
* input under automatic and explicit prompt caching (about 17% less at 128k, 10-14% at 32k). */
|
|
28
|
+
export declare const DEFAULT_CUT_AT = 0.6;
|
|
29
|
+
export declare const bandOf: (room: number, cutAt?: number) => number;
|
|
21
30
|
/**
|
|
22
31
|
* Replayed reasoning that Pi's estimator does not see. On the OpenAI Responses APIs an assistant turn's reasoning item rides in its thinking
|
|
23
32
|
* block's `thinkingSignature` (encrypted) and is sent back with later requests (pi-ai openai-responses-shared; transform-messages skips
|
|
@@ -49,7 +58,7 @@ export declare const measurableConversationTokens: (messages: Messages, ctx: Ext
|
|
|
49
58
|
/** Raw reserve for the cut planner: the window less the raw tokens that fit the target room, plus the input no cut can remove. */
|
|
50
59
|
export declare const planningReserve: (window: number, targetRoom: number, factor: number, mandatory: number) => number;
|
|
51
60
|
export type Target = "band" | "room";
|
|
52
|
-
/** A planned cut: the conversation it leaves, and the summary view that produced it (whole dialogue or
|
|
61
|
+
/** A planned cut: the conversation it leaves, and the summary view that produced it (whole dialogue or shrunk). */
|
|
53
62
|
export type Plan<D extends object = {}> = ({
|
|
54
63
|
draft: true;
|
|
55
64
|
after: number;
|
|
@@ -98,12 +107,16 @@ export type Verdict<D extends object = {}> = Exclude<Decision<D>, {
|
|
|
98
107
|
}>;
|
|
99
108
|
/**
|
|
100
109
|
* Rule 4. The exact room is planned first. When its plan already lands below the band it is taken as it is (target "room"): it keeps the
|
|
101
|
-
* room plan's content exactly, because every allowance of the planner is derived from the reserve and the band's is
|
|
110
|
+
* room plan's content exactly, because every allowance of the planner is derived from the reserve and the band's is smaller. Only
|
|
102
111
|
* otherwise is the band planned. Inside the band a cut is opportunistic, so a failed plan is never a fault and never falls back to the
|
|
103
112
|
* room (a cut that still lands in the band would trigger again every round); the exact room would cut nothing there, so a band cut is
|
|
104
113
|
* taken only when the room can plan a cut too and with the same mode (a lossy view the room could not even plan is a quality change with
|
|
105
114
|
* no fault to prevent). At or above the room, where a cut is due, a band plan of another mode gives way to the room plan (today's
|
|
106
115
|
* view), and a band plan stands when the room has none (a cut instead of a fault).
|
|
116
|
+
* A lean band, one the cut point binds (`band < bandOf(room)`), is planned first and the room plan landing below it is not taken for that:
|
|
117
|
+
* the room plan keeps a native tail of up to half the room, so it can land just under a band that far below the room, and the next round
|
|
118
|
+
* would cut again, freeing little. The room is planned only where the rules above use it: to compare the mode, or when the band has no plan
|
|
119
|
+
* and a cut is due. With the default cut point the band is never lean and the order and the decisions are those above.
|
|
107
120
|
* The margin adds no new decision fault. Exceptions thrown by `plan` are not caught here: they are faults, and they now surface at
|
|
108
121
|
* boundaries inside the band too, where no planner ran before the margin.
|
|
109
122
|
*/
|
package/dist/src/budget.js
CHANGED
|
@@ -4,8 +4,20 @@ import { measurableMessages, projectedMessages } from "./omissions.js";
|
|
|
4
4
|
import { isDiscardedResponse } from "./protocol.js";
|
|
5
5
|
/** Triggers fire this fraction of the room early. The margin absorbs request growth a boundary cannot see; it never enters the calibration factor. */
|
|
6
6
|
export const TRIGGER_MARGIN_FRACTION = 0.02;
|
|
7
|
-
/**
|
|
8
|
-
export const
|
|
7
|
+
/** Whether `value` is a cut point: a fraction of the room above 0 and at most 1. */
|
|
8
|
+
export const validCutAt = (value) => typeof value === "number" && value > 0 && value <= 1;
|
|
9
|
+
/**
|
|
10
|
+
* The one owner of the band: the limit a cut is triggered at and preferably planned to. It is the room less the margin, or the cut point
|
|
11
|
+
* `cutAt` (a fraction of the room) when that is lower. The exact room stays the limit of every throw, fault and the complete-context guard,
|
|
12
|
+
* whatever the cut point. With `cutAt` 1 the band is the room less the margin alone.
|
|
13
|
+
*/
|
|
14
|
+
/** The default cut point: a cut once a request passes 60% of the room. Measured against 1 with paired replays (no answer loss) and billed
|
|
15
|
+
* input under automatic and explicit prompt caching (about 17% less at 128k, 10-14% at 32k). */
|
|
16
|
+
export const DEFAULT_CUT_AT = 0.6;
|
|
17
|
+
export const bandOf = (room, cutAt = 1) => {
|
|
18
|
+
const positive = Math.max(0, room);
|
|
19
|
+
return Math.min(room - Math.ceil(positive * TRIGGER_MARGIN_FRACTION), Math.floor(positive * cutAt));
|
|
20
|
+
};
|
|
9
21
|
/** The APIs whose request converter (pi-ai openai-responses-shared) sends each signed thinking block back as an input reasoning item. */
|
|
10
22
|
const REPLAYING_APIS = new Set(["openai-responses", "azure-openai-responses", "openai-codex-responses"]);
|
|
11
23
|
/**
|
|
@@ -59,29 +71,40 @@ export const measurableConversationTokens = (messages, ctx, policy) => conversat
|
|
|
59
71
|
export const planningReserve = (window, targetRoom, factor, mandatory) => window - Math.floor(targetRoom / factor) + mandatory;
|
|
60
72
|
/**
|
|
61
73
|
* Rule 4. The exact room is planned first. When its plan already lands below the band it is taken as it is (target "room"): it keeps the
|
|
62
|
-
* room plan's content exactly, because every allowance of the planner is derived from the reserve and the band's is
|
|
74
|
+
* room plan's content exactly, because every allowance of the planner is derived from the reserve and the band's is smaller. Only
|
|
63
75
|
* otherwise is the band planned. Inside the band a cut is opportunistic, so a failed plan is never a fault and never falls back to the
|
|
64
76
|
* room (a cut that still lands in the band would trigger again every round); the exact room would cut nothing there, so a band cut is
|
|
65
77
|
* taken only when the room can plan a cut too and with the same mode (a lossy view the room could not even plan is a quality change with
|
|
66
78
|
* no fault to prevent). At or above the room, where a cut is due, a band plan of another mode gives way to the room plan (today's
|
|
67
79
|
* view), and a band plan stands when the room has none (a cut instead of a fault).
|
|
80
|
+
* A lean band, one the cut point binds (`band < bandOf(room)`), is planned first and the room plan landing below it is not taken for that:
|
|
81
|
+
* the room plan keeps a native tail of up to half the room, so it can land just under a band that far below the room, and the next round
|
|
82
|
+
* would cut again, freeing little. The room is planned only where the rules above use it: to compare the mode, or when the band has no plan
|
|
83
|
+
* and a cut is due. With the default cut point the band is never lean and the order and the decisions are those above.
|
|
68
84
|
* The margin adds no new decision fault. Exceptions thrown by `plan` are not caught here: they are faults, and they now surface at
|
|
69
85
|
* boundaries inside the band too, where no planner ran before the margin.
|
|
70
86
|
*/
|
|
71
87
|
export function chooseCut(input) {
|
|
72
88
|
const { before, mandatory, room, band, fits, plan, unreduced } = input;
|
|
73
|
-
const
|
|
74
|
-
|
|
75
|
-
|
|
89
|
+
const lands = (p) => p.draft && p.after < before && fits(p.after + mandatory, band);
|
|
90
|
+
let roomPlan;
|
|
91
|
+
const exact = () => roomPlan ??= plan("room");
|
|
92
|
+
if (band >= bandOf(room)) {
|
|
93
|
+
const e = exact();
|
|
94
|
+
if (lands(e))
|
|
95
|
+
return { kind: "cut", target: "room", plan: e };
|
|
96
|
+
}
|
|
76
97
|
const preferred = plan("band");
|
|
77
98
|
if (input.opportunistic) {
|
|
78
|
-
if (!(preferred
|
|
99
|
+
if (!lands(preferred))
|
|
79
100
|
return { kind: "none" };
|
|
80
|
-
|
|
101
|
+
const e = exact();
|
|
102
|
+
return e.draft && e.mode === preferred.mode ? { kind: "cut", target: "band", plan: preferred } : { kind: "none" };
|
|
81
103
|
}
|
|
104
|
+
const e = exact();
|
|
82
105
|
let chosen = preferred, target = "band";
|
|
83
|
-
if (!preferred.draft || (
|
|
84
|
-
chosen =
|
|
106
|
+
if (!preferred.draft || (e.draft && e.mode !== preferred.mode)) {
|
|
107
|
+
chosen = e;
|
|
85
108
|
target = "room";
|
|
86
109
|
}
|
|
87
110
|
if (!chosen.draft)
|
package/dist/src/build-id.d.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export declare const BUILD_ID = "
|
|
1
|
+
export declare const BUILD_ID = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
|
package/dist/src/build-id.js
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export const BUILD_ID = "
|
|
1
|
+
export const BUILD_ID = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
|
package/dist/src/compaction.d.ts
CHANGED
|
@@ -27,6 +27,10 @@ export interface CompactionOptions {
|
|
|
27
27
|
/** Optional estimated complete-input ceiling, separate from physical output reservation.
|
|
28
28
|
* The host owns final serialization/transport counting; this is not an exact wire-token cap. */
|
|
29
29
|
activeInputTokens?: number | ((ctx: ExtensionContext) => number);
|
|
30
|
+
/** The cut point: a fraction of the input room, above 0 and at most 1 (default 1). A cut is triggered at, and preferably planned to, the band,
|
|
31
|
+
* which is the room less 2% of it, or this fraction of the room when that is lower; a lower value cuts earlier and leaves less. The exact room
|
|
32
|
+
* stays the limit of every throw, fault and the complete-context guard. Anything else is rejected when the compactor is installed. */
|
|
33
|
+
cutAt?: number;
|
|
30
34
|
/** The response reserve: the input room is the context window less this. By default the room never exceeds the window less the model's
|
|
31
35
|
* catalogue maximum output, because a transport may omit the output cap and the reply can then grow to that maximum. A host that knows its
|
|
32
36
|
* transport's behavior declares an exact reserve with `responseExact: true`, and the room is then the window less `response` exactly. */
|
package/dist/src/compaction.js
CHANGED
|
@@ -5,7 +5,7 @@ import { exportSource } from "./source-export.js";
|
|
|
5
5
|
import { isDeepStrictEqual } from "node:util";
|
|
6
6
|
import { assertNativeToolRounds } from "./protocol.js";
|
|
7
7
|
import { projectedMessages } from "./omissions.js";
|
|
8
|
-
import { bandOf, chooseCut, conversationTokens, decideCut, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, requestTokens, systemPart } from "./budget.js";
|
|
8
|
+
import { DEFAULT_CUT_AT, bandOf, chooseCut, conversationTokens, decideCut, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, requestTokens, systemPart, validCutAt } from "./budget.js";
|
|
9
9
|
import { isContextOverflow } from "@earendil-works/pi-ai";
|
|
10
10
|
import { convertToLlm, SessionManager, SettingsManager } from "@earendil-works/pi-coding-agent";
|
|
11
11
|
import { Type } from "typebox";
|
|
@@ -81,6 +81,9 @@ export function seedWindow(branch, key) {
|
|
|
81
81
|
return initialWindow(key);
|
|
82
82
|
}
|
|
83
83
|
export async function installCompaction(pi, options = {}) {
|
|
84
|
+
const cutAt = options.cutAt ?? DEFAULT_CUT_AT;
|
|
85
|
+
if (!validCutAt(cutAt))
|
|
86
|
+
throw new Error("Invalid cut point: cutAt must be a number above 0 and at most 1");
|
|
84
87
|
const semantic = options.semanticRecall ?? createSemanticRecall();
|
|
85
88
|
let semanticTransition = Promise.resolve();
|
|
86
89
|
let semanticRequest = 0;
|
|
@@ -101,7 +104,7 @@ export async function installCompaction(pi, options = {}) {
|
|
|
101
104
|
}
|
|
102
105
|
});
|
|
103
106
|
const captureDraft = (draft, ctx, reserve, reason, tokensBefore, retainCurrentInput, operatorManual, previousSummary, budget) => recorder.measureInline(() => {
|
|
104
|
-
const attached = attachReceipt(draft, reason, reserve, ctx.model?.contextWindow ?? 0, { responseReserveTokens: budget?.response, hostInputTokens: budget?.host, mandatoryInputTokens: budget?.mandatory, target: budget?.target, triggerMarginTokens: budget?.margin, calibrationFactor: factor, calibrationWindow: [...ratios], calibrationIdentity: identity, activeInputCeilingTokens: lastCeiling });
|
|
107
|
+
const attached = attachReceipt(draft, reason, reserve, ctx.model?.contextWindow ?? 0, { responseReserveTokens: budget?.response, hostInputTokens: budget?.host, mandatoryInputTokens: budget?.mandatory, target: budget?.target, triggerMarginTokens: budget?.margin, ...(cutAt === 1 ? {} : { cutAt }), calibrationFactor: factor, calibrationWindow: [...ratios], calibrationIdentity: identity, activeInputCeilingTokens: lastCeiling });
|
|
105
108
|
proposalId = receiptId(attached);
|
|
106
109
|
if (recorder.enabled) {
|
|
107
110
|
try {
|
|
@@ -204,7 +207,7 @@ export async function installCompaction(pi, options = {}) {
|
|
|
204
207
|
// The cut planner's raw reserve for the target limit, on the same factor as every other consumer.
|
|
205
208
|
const planReserve = (target, room, mandatory, ctx) => {
|
|
206
209
|
identityFor(ctx);
|
|
207
|
-
return planningReserve(ctx.model?.contextWindow ?? 0, target === "band" ? bandOf(room) : room, factor, mandatory);
|
|
210
|
+
return planningReserve(ctx.model?.contextWindow ?? 0, target === "band" ? bandOf(room, cutAt) : room, factor, mandatory);
|
|
208
211
|
};
|
|
209
212
|
pi.on("message_end", (event, ctx) => {
|
|
210
213
|
if (event.message.role !== "assistant")
|
|
@@ -268,10 +271,10 @@ export async function installCompaction(pi, options = {}) {
|
|
|
268
271
|
throw new Error("Invalid active-input token ceiling");
|
|
269
272
|
return Math.min((ctx.model?.contextWindow ?? 0) - response, ceiling ?? Infinity);
|
|
270
273
|
};
|
|
271
|
-
// The exact room, and the band a cut is triggered at (budget.ts); every throw and every fault uses the exact room.
|
|
274
|
+
// The exact room, and the band a cut is triggered at (budget.ts, from this session's cutAt); every throw and every fault uses the exact room.
|
|
272
275
|
const limits = (ctx) => {
|
|
273
276
|
const { response } = requestReserve(ctx), room = inputRoom(response, ctx);
|
|
274
|
-
return { response, room, band: bandOf(room) };
|
|
277
|
+
return { response, room, band: bandOf(room, cutAt) };
|
|
275
278
|
};
|
|
276
279
|
const fits = (tokens, limit, ctx) => corrected(tokens, ctx) < limit;
|
|
277
280
|
// The saved conversation and a cut's quantities are counted by the one function the guard counts with (strict, as the context hook is).
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { type RecallPolicy } from './recall.js';
|
|
2
2
|
import type { SessionEntry } from '@earendil-works/pi-coding-agent';
|
|
3
|
-
export declare const DIAGNOSTIC_BUILD = "
|
|
3
|
+
export declare const DIAGNOSTIC_BUILD = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
|
|
4
4
|
export interface Receipt {
|
|
5
5
|
id: string;
|
|
6
6
|
build: string;
|
|
@@ -15,6 +15,7 @@ export interface Receipt {
|
|
|
15
15
|
mandatoryInputTokens?: number;
|
|
16
16
|
target?: 'band' | 'room';
|
|
17
17
|
triggerMarginTokens?: number;
|
|
18
|
+
cutAt?: number;
|
|
18
19
|
calibrationFactor?: number;
|
|
19
20
|
calibrationWindow?: number[];
|
|
20
21
|
calibrationIdentity?: string;
|
package/dist/src/omissions.d.ts
CHANGED
|
@@ -2,7 +2,7 @@ import type { ExtensionContext, SessionEntry } from '@earendil-works/pi-coding-a
|
|
|
2
2
|
import { type RecallPolicy } from './recall.js';
|
|
3
3
|
/** Native branch edits supply exclusion; original global indices supply provenance. */
|
|
4
4
|
export declare function omittedSourceIndices(entries: readonly SessionEntry[], branch: readonly SessionEntry[]): ReadonlySet<number>;
|
|
5
|
-
/** Preserve surviving record bytes/order. No text matching or semantic inference. */
|
|
5
|
+
/** Preserve surviving record bytes/order, and the summary's format (a v1 summary stays v1). No text matching or semantic inference. */
|
|
6
6
|
export declare function filterOmittedSummary(summary: string, omitted: ReadonlySet<number>): string;
|
|
7
7
|
/** Custom quotes carry native entry identity; no text matching or legacy fallback. */
|
|
8
8
|
export declare function filterCustomQuotes(summary: string, entries: readonly SessionEntry[], branch: readonly SessionEntry[], policy?: RecallPolicy): string;
|
package/dist/src/omissions.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { buildGlobalIndexById } from './tuck/core/global-indices.js';
|
|
2
|
-
import {
|
|
2
|
+
import { parseRecords, splitSummary, transcriptHeader } from './tuck/core/transcript.js';
|
|
3
3
|
import { createRecall } from './recall.js';
|
|
4
4
|
/** Native branch edits supply exclusion; original global indices supply provenance. */
|
|
5
5
|
export function omittedSourceIndices(entries, branch) {
|
|
@@ -13,24 +13,24 @@ export function omittedSourceIndices(entries, branch) {
|
|
|
13
13
|
}
|
|
14
14
|
return new Set([...edits].flatMap(([id, omitted]) => omitted && indices.has(id) ? [indices.get(id)] : []));
|
|
15
15
|
}
|
|
16
|
-
/** Preserve surviving record bytes/order. No text matching or semantic inference. */
|
|
16
|
+
/** Preserve surviving record bytes/order, and the summary's format (a v1 summary stays v1). No text matching or semantic inference. */
|
|
17
17
|
export function filterOmittedSummary(summary, omitted) {
|
|
18
18
|
if (!omitted.size)
|
|
19
19
|
return summary;
|
|
20
20
|
const split = splitSummary(summary);
|
|
21
21
|
if (!split)
|
|
22
22
|
throw new Error('Cannot apply native source omission to an unsupported compaction summary');
|
|
23
|
-
const { body, closing } = split;
|
|
24
|
-
const records = parseRecords(body), lines = body ? body.split('\n') : [];
|
|
23
|
+
const { body, closing, version } = split;
|
|
24
|
+
const records = parseRecords(body, version), lines = body ? body.split('\n') : [];
|
|
25
25
|
const retained = records.flatMap((record, i) => (record.sourceIndex !== undefined && omitted.has(record.sourceIndex)) || record.sourceIndices?.some(index => omitted.has(index)) ? [] : [lines[i]]);
|
|
26
|
-
return
|
|
26
|
+
return transcriptHeader(version) + '\n' + retained.join('\n') + closing;
|
|
27
27
|
}
|
|
28
28
|
/** Custom quotes carry native entry identity; no text matching or legacy fallback. */
|
|
29
29
|
export function filterCustomQuotes(summary, entries, branch, policy = {}) {
|
|
30
30
|
const split = splitSummary(summary);
|
|
31
31
|
if (!split)
|
|
32
32
|
throw new Error('Cannot resolve custom sources in an unsupported compaction summary');
|
|
33
|
-
const { body, closing } = split;
|
|
33
|
+
const { body, closing, version } = split;
|
|
34
34
|
const omitted = new Set(branch.flatMap(entry => entry.type === 'context_edit' && entry.replacement === null ? [entry.targetId] : []));
|
|
35
35
|
const eligible = createRecall(policy).isHistorySource, indices = buildGlobalIndexById(entries);
|
|
36
36
|
const globalCounts = new Map(), branchSources = new Map();
|
|
@@ -42,7 +42,7 @@ export function filterCustomQuotes(summary, entries, branch, policy = {}) {
|
|
|
42
42
|
branchSources.set(entry.id, found);
|
|
43
43
|
}
|
|
44
44
|
const lines = body ? body.split('\n') : [];
|
|
45
|
-
const records = parseRecords(body);
|
|
45
|
+
const records = parseRecords(body, version);
|
|
46
46
|
const retained = records.flatMap((record, i) => {
|
|
47
47
|
if (record.role !== 'custom')
|
|
48
48
|
return [lines[i]];
|
|
@@ -56,7 +56,7 @@ export function filterCustomQuotes(summary, entries, branch, policy = {}) {
|
|
|
56
56
|
return [];
|
|
57
57
|
return [lines[i]];
|
|
58
58
|
});
|
|
59
|
-
return
|
|
59
|
+
return transcriptHeader(version) + '\n' + retained.join('\n') + closing;
|
|
60
60
|
}
|
|
61
61
|
/** The saved conversation as this compactor delivers it: native omissions and ineligible custom quotes removed from summaries. */
|
|
62
62
|
export function projectedMessages(messages, ctx, policy) {
|
|
@@ -1,49 +1,19 @@
|
|
|
1
1
|
import type { NormalizedBlock } from "../types.js";
|
|
2
|
-
import type { SourceTime } from "./source-time.js";
|
|
3
2
|
import { type TranscriptRecord } from "./transcript.js";
|
|
4
|
-
|
|
5
|
-
export
|
|
6
|
-
|
|
7
|
-
* If `lines[i]` opens a heredoc whose terminator actually appears on a later
|
|
8
|
-
* line, return that terminator's line index; otherwise -1. Callers use -1 to
|
|
9
|
-
* leave following lines intact instead of treating a stray `<<` (a shift op, a
|
|
10
|
-
* quoted string, or a truncated body) as a heredoc and skipping real commands.
|
|
11
|
-
*/
|
|
12
|
-
export declare const heredocCloseIndex: (lines: string[], i: number) => number;
|
|
13
|
-
export type CallRecord = Extract<TranscriptRecord, {
|
|
14
|
-
kind: "tool_call";
|
|
3
|
+
/** The user, assistant and notice blocks of a dialogue; tool blocks are never rendered. */
|
|
4
|
+
export type DialogueBlock = Extract<NormalizedBlock, {
|
|
5
|
+
kind: "user" | "assistant" | "notice";
|
|
15
6
|
}>;
|
|
16
|
-
|
|
17
|
-
export type ExcerptCache = WeakMap<NormalizedBlock, Map<string, TranscriptRecord[]>>;
|
|
18
|
-
/** The assistant texts that close a segment: no further text before the next user turn or the end of the chronology. */
|
|
19
|
-
export declare const segmentClosing: (chronology: NormalizedBlock[]) => Set<NormalizedBlock>;
|
|
7
|
+
export declare const isDialogue: (block: NormalizedBlock) => block is DialogueBlock;
|
|
20
8
|
/** A user or assistant text block the omission notices count when it is unselected: non-empty, and not rendered in the mandatory base. */
|
|
21
9
|
export declare const isOmittable: (block: NormalizedBlock, mandatorySources?: ReadonlySet<number>) => boolean;
|
|
22
|
-
/** `sources` are the distinct locators of the omitted blocks in ascending order; at most the first 9 decide the notice.
|
|
10
|
+
/** `sources` are the distinct locators of the omitted blocks in ascending order; at most the first 9 decide the notice. They are written
|
|
11
|
+
* as "#N" on the record's line. */
|
|
23
12
|
export declare const omissionRecord: (role: "user" | "assistant", count: number, sources: number[], unlocated: boolean) => TranscriptRecord;
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
export declare const
|
|
29
|
-
/**
|
|
30
|
-
export
|
|
31
|
-
full: boolean;
|
|
32
|
-
closing: boolean;
|
|
33
|
-
mandatory?: ReadonlySet<number>;
|
|
34
|
-
bounded: boolean;
|
|
35
|
-
covered: boolean;
|
|
36
|
-
}
|
|
37
|
-
/** The records of one block alone: before identical calls merge and before source times are attached. */
|
|
38
|
-
export declare const blockRecords: (b: NormalizedBlock, view: BlockView, cache?: ExcerptCache) => TranscriptRecord[];
|
|
39
|
-
/** Whether `record` is the same call as the `previous` record, shown a second time. */
|
|
40
|
-
export declare const repeatsCall: (previous: TranscriptRecord | undefined, record: TranscriptRecord) => previous is CallRecord & {
|
|
41
|
-
sourceIndex: number;
|
|
42
|
-
};
|
|
43
|
-
/** The earlier record of a repeated call, now also locating the later one. */
|
|
44
|
-
export declare const absorbCall: (previous: TranscriptRecord, record: TranscriptRecord) => TranscriptRecord;
|
|
45
|
-
/** A source time for each locator the record names; only briefs of times-carrying blocks have them. */
|
|
46
|
-
export declare const noticeTimes: (blocks: NormalizedBlock[]) => Map<string, SourceTime>;
|
|
47
|
-
export declare const decorateRecord: (record: TranscriptRecord, times: NormalizedBlock["sourceTimes"], entryTimes: ReadonlyMap<string, SourceTime>) => TranscriptRecord;
|
|
48
|
-
export declare const buildBriefRecords: (blocks: NormalizedBlock[], cache?: ExcerptCache) => TranscriptRecord[];
|
|
49
|
-
export declare const compileBrief: (blocks: NormalizedBlock[], cache?: ExcerptCache) => string;
|
|
13
|
+
/** The word budgets of a clipped excerpt: the passages kept from the start of the text, and from its end. */
|
|
14
|
+
export type Clip = readonly [headWords: number, tailWords: number];
|
|
15
|
+
/** The records of one dialogue block alone, before source times are attached: its whole original text, or, with a `clip`, the excerpts
|
|
16
|
+
* that keep their sanitized-block spans around an omission. A mandatory source renders nothing: the adapter quotes it in its base. */
|
|
17
|
+
export declare const blockRecords: (b: DialogueBlock, mandatory: ReadonlySet<number> | undefined, clip?: Clip) => TranscriptRecord[];
|
|
18
|
+
/** A source time for each locator a user record names; only briefs of times-carrying blocks have them. */
|
|
19
|
+
export declare const decorateRecord: (record: TranscriptRecord, times: NormalizedBlock["sourceTimes"]) => TranscriptRecord;
|