pi-context-kit 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +5 -2
- package/README.md +517 -51
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -2,11 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
## Unreleased
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
5
|
+
## 0.2.1
|
|
6
|
+
|
|
7
|
+
- Rewrite the README for users who do not know Pi internals: it explains the terms, the sequence of a session, and each setting with its default and an example.
|
|
7
8
|
|
|
8
9
|
## 0.2.0
|
|
9
10
|
|
|
11
|
+
- Send handoff note requests with the session's thinking level and tool choice, so Anthropic reuses the cached conversation instead of re-billing it; reject notes that call a tool.
|
|
12
|
+
- `soft` now writes Pi's summary at `thresholdTokens` or `thresholdPercent`, so an earlier threshold compacts even when no handoff note is available. Before, Pi compacted only at its own threshold.
|
|
10
13
|
- Keep older read results when a newer full or ranged read is truncated, count the replaced read in the stale suffix budget, and preserve user-role messages during shaking.
|
|
11
14
|
- Bound recall regex searches in a worker and propagate cancellation; keep regex syntax errors and result paging.
|
|
12
15
|
- Reject length-truncated or oversized handoff notes, validate retained-context budgets, and use only ready background notes during native, manual, and overflow compaction.
|
package/README.md
CHANGED
|
@@ -1,85 +1,551 @@
|
|
|
1
1
|
# pi-context-kit
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
pi-context-kit keeps long [Pi](https://github.com/earendil-works/pi) sessions small, so the AI model can continue to work.
|
|
4
|
+
You install it once. After that, it works automatically.
|
|
5
|
+
|
|
6
|
+
## The problem
|
|
7
|
+
|
|
8
|
+
An AI model can read only a limited amount of text at one time. This limit is the **context window**.
|
|
9
|
+
Text is measured in **tokens**. One token is approximately 3 to 4 characters of English text.
|
|
10
|
+
|
|
11
|
+
In a long session, the conversation fills the context window. File reads and command outputs are usually the largest parts.
|
|
12
|
+
When the window is almost full, Pi does a **compaction**: it replaces the old conversation with a summary.
|
|
13
|
+
A summary can lose details, and it costs time and money.
|
|
14
|
+
|
|
15
|
+
pi-context-kit does three things:
|
|
16
|
+
|
|
17
|
+
1. It removes text that the model does not need any more, before the window is full.
|
|
18
|
+
2. When a summary is necessary, it prepares a better summary in advance.
|
|
19
|
+
3. It keeps all removed text available. The model can read it again with the `recall` tool.
|
|
20
|
+
|
|
21
|
+
## Words used in this document
|
|
22
|
+
|
|
23
|
+
| Word | Meaning |
|
|
24
|
+
| --- | --- |
|
|
25
|
+
| Token | A small piece of text. The model counts its memory in tokens. |
|
|
26
|
+
| Context window | The maximum number of tokens that the model can read at one time. |
|
|
27
|
+
| Context | All the text that Pi sends to the model now: instructions, messages, and tool outputs. |
|
|
28
|
+
| Turn | One cycle: the model replies, uses its tools, and stops. |
|
|
29
|
+
| Tool output | The result of a tool, for example the content of a file or the output of a command. |
|
|
30
|
+
| Threshold | The number of tokens at which maintenance starts. |
|
|
31
|
+
| Compaction | Pi replaces the old conversation with a summary. The newest messages stay. |
|
|
32
|
+
| Handoff note | A summary that pi-context-kit writes with the same model as your session. |
|
|
33
|
+
| Pi's summary | The summary that Pi writes when no handoff note is available. |
|
|
4
34
|
|
|
5
35
|
## Install
|
|
6
36
|
|
|
7
|
-
|
|
8
|
-
|
|
37
|
+
1. Open a terminal.
|
|
38
|
+
2. Type this command and press Enter:
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
pi install npm:pi-context-kit
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
3. Restart Pi.
|
|
45
|
+
|
|
46
|
+
To check the installation, type `pi list`. The list must show `npm:pi-context-kit`.
|
|
47
|
+
|
|
48
|
+
To remove it, type `pi remove npm:pi-context-kit` and restart Pi.
|
|
49
|
+
|
|
50
|
+
## Recommended setup
|
|
51
|
+
|
|
52
|
+
The default setup works without changes. By default, maintenance starts at the same time as Pi's compaction.
|
|
53
|
+
For large context windows, start maintenance earlier. This example starts at 150,000 tokens:
|
|
54
|
+
|
|
55
|
+
1. Open the file `~/.pi/agent/settings.json` in a text editor. `~` is your home folder.
|
|
56
|
+
2. Add the `contextKit` block inside the outer `{ }`. Put a comma between this block and the other settings.
|
|
57
|
+
|
|
58
|
+
```json
|
|
59
|
+
{
|
|
60
|
+
"contextKit": {
|
|
61
|
+
"thresholdTokens": 150000
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
3. Save the file.
|
|
67
|
+
4. In Pi, type `/reload`, or restart Pi.
|
|
68
|
+
|
|
69
|
+
If a value is incorrect, Pi shows a warning that starts with `pi-context-kit: invalid settings`. The incorrect value is ignored and its default is used.
|
|
70
|
+
|
|
71
|
+
To use a setting only in one project, put the same block in the file `.pi/settings.json` in that project folder. Project settings replace global settings.
|
|
72
|
+
|
|
73
|
+
## What happens during a session
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
Every turn ends
|
|
77
|
+
|
|
|
78
|
+
v
|
|
79
|
+
1. Remove old file reads that a newer read replaced
|
|
80
|
+
|
|
|
81
|
+
v
|
|
82
|
+
Context near the threshold? --yes--> 2. Start a handoff note in the background
|
|
83
|
+
|
|
|
84
|
+
v
|
|
85
|
+
Context above the threshold? --no--> continue to work
|
|
86
|
+
|
|
|
87
|
+
yes
|
|
88
|
+
v
|
|
89
|
+
3. Prune old tool outputs
|
|
90
|
+
|
|
|
91
|
+
v
|
|
92
|
+
Small enough now? --yes--> continue to work
|
|
93
|
+
|
|
|
94
|
+
no
|
|
95
|
+
v
|
|
96
|
+
4. Try each method in "methodOrder": handoff, shake, soft
|
|
9
97
|
```
|
|
10
98
|
|
|
11
|
-
|
|
99
|
+
### 1. Old file reads are replaced
|
|
12
100
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
101
|
+
Sometimes the model reads the same file two times. The first copy is then old.
|
|
102
|
+
pi-context-kit replaces the old copy with this text: `[Superseded by a newer read of this file]`.
|
|
103
|
+
|
|
104
|
+
It does this only when the newer read covers all of the old read. If the newer read was cut short, the old copy stays.
|
|
105
|
+
It does this only for recent entries, limited by `staleSuffixTokens`. This limit tries to keep the prompt cache useful, but the provider decides about caching.
|
|
106
|
+
|
|
107
|
+
### 2. A handoff note is prepared in advance
|
|
20
108
|
|
|
21
|
-
|
|
109
|
+
Before the context reaches the threshold, the session model writes a handoff note in the background. Your work does not stop.
|
|
110
|
+
The note starts one eighth of the threshold early, with a maximum of 32,000 tokens. With a threshold of 150,000 tokens, it starts at approximately 131,250 tokens.
|
|
22
111
|
|
|
23
|
-
|
|
24
|
-
- At an earlier `thresholdTokens` or `thresholdPercent`, a handoff is appended only when the retained context and note fit the progress target. Otherwise maintenance continues in the configured order. The extension never calls `ctx.compact()`, which aborts the run.
|
|
25
|
-
- The kept part always holds the latest messages and tool results.
|
|
26
|
-
- An early automatic handoff can defer while a relevant background note is still being written. Native, manual, and overflow compaction never wait for an unfinished note.
|
|
112
|
+
The note uses the same sections as Pi's summary. It also lists the files that were read and changed.
|
|
27
113
|
|
|
28
|
-
|
|
114
|
+
### 3. Old tool outputs are pruned
|
|
29
115
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
116
|
+
Above the threshold, old tool outputs are replaced with a short notice, for example:
|
|
117
|
+
`[Output pruned: about 5000 tokens. Call recall with entryId "abc123" to read it.]`
|
|
118
|
+
|
|
119
|
+
The newest 40,000 tokens of tool outputs always stay. pi-context-kit keeps this change only if the context becomes small enough. "Small enough" means 80% of the threshold or less.
|
|
120
|
+
|
|
121
|
+
### 4. Other methods, in order
|
|
122
|
+
|
|
123
|
+
If pruning is not sufficient, pi-context-kit tries the methods in `methodOrder`, one after the other:
|
|
124
|
+
|
|
125
|
+
| Method | What it does | Uses the model? |
|
|
126
|
+
| --- | --- | --- |
|
|
127
|
+
| `handoff` | Replaces the old conversation with the handoff note. The newest messages stay. | Yes |
|
|
128
|
+
| `shake` | Replaces large old code blocks, XML blocks, and images with a short notice. Your messages are never shaken. | No |
|
|
129
|
+
| `soft` | Replaces the old conversation with Pi's summary. The newest messages stay. Stops here. | Yes |
|
|
130
|
+
|
|
131
|
+
Your work continues during these steps. pi-context-kit never stops a running task to compact.
|
|
132
|
+
|
|
133
|
+
### The `recall` tool
|
|
134
|
+
|
|
135
|
+
The model can always read removed text again. The `recall` tool can:
|
|
136
|
+
|
|
137
|
+
- search all messages of the current conversation branch, also the removed and summarized messages;
|
|
138
|
+
- read one full entry, with the ID given in a notice.
|
|
139
|
+
|
|
140
|
+
Images cannot be recovered. `recall` shows only their file type.
|
|
141
|
+
|
|
142
|
+
## What it never changes
|
|
143
|
+
|
|
144
|
+
- Your messages. Your instructions and your images always stay.
|
|
145
|
+
- The tool calls of the model, and the model's reasoning.
|
|
146
|
+
- The pairs of tool call and tool result. Each tool call keeps its result.
|
|
37
147
|
|
|
38
148
|
## Settings
|
|
39
149
|
|
|
40
|
-
|
|
150
|
+
This section lists every setting. Each setting has its allowed values, its default, and an example.
|
|
151
|
+
|
|
152
|
+
### Where to put the settings
|
|
153
|
+
|
|
154
|
+
Put all settings in a `contextKit` block, in one of these files:
|
|
155
|
+
|
|
156
|
+
| File | Applies to |
|
|
157
|
+
| --- | --- |
|
|
158
|
+
| `~/.pi/agent/settings.json` | All your projects. `~` is your home folder. |
|
|
159
|
+
| `.pi/settings.json` in a project folder | Only this project. These values replace the values of the global file. |
|
|
160
|
+
|
|
161
|
+
You do not need to set all settings. A missing setting uses its default.
|
|
162
|
+
After a change, type `/reload` in Pi, or restart Pi.
|
|
163
|
+
|
|
164
|
+
### All settings with their defaults
|
|
165
|
+
|
|
166
|
+
This block sets every setting to its default. It has the same effect as no `contextKit` block.
|
|
167
|
+
Copy only the lines that you want to change.
|
|
41
168
|
|
|
42
169
|
```json
|
|
43
170
|
{
|
|
44
171
|
"contextKit": {
|
|
45
|
-
"
|
|
46
|
-
"
|
|
172
|
+
"enabled": true,
|
|
173
|
+
"thresholdTokens": -1,
|
|
174
|
+
"thresholdPercent": -1,
|
|
175
|
+
"methodOrder": ["handoff", "shake", "soft"],
|
|
176
|
+
"asyncEnabled": true,
|
|
177
|
+
"supersedeReads": true,
|
|
178
|
+
"staleSuffixTokens": 8000,
|
|
179
|
+
"progressRatio": 0.8,
|
|
180
|
+
"prune": {
|
|
181
|
+
"enabled": true,
|
|
182
|
+
"protectTokens": 40000,
|
|
183
|
+
"minSavings": 20000,
|
|
184
|
+
"minTokens": 50
|
|
185
|
+
},
|
|
186
|
+
"shake": {
|
|
187
|
+
"protectTokens": 16000,
|
|
188
|
+
"minSavings": 4000,
|
|
189
|
+
"blockMinTokens": 400
|
|
190
|
+
}
|
|
47
191
|
}
|
|
48
192
|
}
|
|
49
193
|
```
|
|
50
194
|
|
|
51
|
-
|
|
195
|
+
### Summary table
|
|
196
|
+
|
|
197
|
+
| Setting | Allowed values | Default | Short description |
|
|
198
|
+
| --- | --- | --- | --- |
|
|
199
|
+
| [`enabled`](#enabled) | `true`, `false` | `true` | Turns all maintenance on or off. |
|
|
200
|
+
| [`thresholdTokens`](#thresholdtokens) | `-1`, or a whole number above 0 | `-1` | Starts maintenance at this number of tokens. |
|
|
201
|
+
| [`thresholdPercent`](#thresholdpercent) | `-1`, or a number above 0, up to 100 | `-1` | Starts maintenance at this percentage of the context window. |
|
|
202
|
+
| [`methodOrder`](#methodorder) | A list of `"handoff"`, `"shake"`, `"soft"` | `["handoff", "shake", "soft"]` | The methods to try, and their order. |
|
|
203
|
+
| [`asyncEnabled`](#asyncenabled) | `true`, `false` | `true` | Prepares the handoff note in the background. |
|
|
204
|
+
| [`supersedeReads`](#supersedereads) | `true`, `false` | `true` | Replaces old copies of a file that was read again. |
|
|
205
|
+
| [`staleSuffixTokens`](#stalesuffixtokens) | A whole number, 0 or more | `8000` | Limits which old copies can be replaced. |
|
|
206
|
+
| [`progressRatio`](#progressratio) | A number above 0, up to 1 | `0.8` | How small the context must become. |
|
|
207
|
+
| [`prune.enabled`](#pruneenabled) | `true`, `false` | `true` | Turns pruning of old tool outputs on or off. |
|
|
208
|
+
| [`prune.protectTokens`](#pruneprotecttokens) | A whole number, 0 or more | `40000` | The newest tool outputs that are never pruned. |
|
|
209
|
+
| [`prune.minSavings`](#pruneminsavings) | A whole number, 0 or more | `20000` | The minimum gain for pruning. |
|
|
210
|
+
| [`prune.minTokens`](#prunemintokens) | A whole number, 0 or more | `50` | Tool outputs smaller than this are never pruned. |
|
|
211
|
+
| [`shake.protectTokens`](#shakeprotecttokens) | A whole number, 0 or more | `16000` | The newest context that is never shaken. |
|
|
212
|
+
| [`shake.minSavings`](#shakeminsavings) | A whole number, 0 or more | `4000` | The minimum gain for shaking. |
|
|
213
|
+
| [`shake.blockMinTokens`](#shakeblockmintokens) | A whole number, 0 or more | `400` | Blocks smaller than this are never shaken. |
|
|
214
|
+
|
|
215
|
+
"Whole number" means a number without decimals, for example `150000`. Do not put spaces, dots, or commas in numbers: write `150000`, not `150,000`.
|
|
216
|
+
|
|
217
|
+
### When a setting is incorrect
|
|
218
|
+
|
|
219
|
+
pi-context-kit checks each value. If a value is incorrect, Pi shows a warning, and the setting uses its default. The other settings still apply.
|
|
220
|
+
|
|
221
|
+
Example: with `"thresholdTokens": "150000"` (a text in quotes, not a number), Pi shows:
|
|
222
|
+
|
|
223
|
+
```text
|
|
224
|
+
pi-context-kit: invalid settings, defaults used (contextKit.thresholdTokens must be -1 or a positive integer).
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
A setting name with a spelling mistake also shows a warning, for example `contextKit.treshold is not a setting`.
|
|
228
|
+
|
|
229
|
+
### `enabled`
|
|
230
|
+
|
|
231
|
+
Turns all maintenance on or off.
|
|
232
|
+
|
|
233
|
+
- `true` (default): pi-context-kit works as described in this document.
|
|
234
|
+
- `false`: pi-context-kit does nothing. Pi uses only its own summary. The `recall` tool stays available.
|
|
235
|
+
|
|
236
|
+
```json
|
|
237
|
+
{ "contextKit": { "enabled": false } }
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
### `thresholdTokens`
|
|
241
|
+
|
|
242
|
+
Maintenance starts when the context is larger than this number of tokens.
|
|
243
|
+
|
|
244
|
+
- `-1` (default): use `thresholdPercent`. If `thresholdPercent` is also `-1`, use Pi's limit.
|
|
245
|
+
- A whole number above 0: start at this number of tokens.
|
|
246
|
+
|
|
247
|
+
This setting has priority over `thresholdPercent`.
|
|
248
|
+
pi-context-kit can start before Pi's limit, never after. If your value is larger than Pi's limit, Pi's limit is used. See [Pi settings that also apply](#pi-settings-that-also-apply).
|
|
249
|
+
|
|
250
|
+
Example: start at 150,000 tokens.
|
|
251
|
+
|
|
252
|
+
```json
|
|
253
|
+
{ "contextKit": { "thresholdTokens": 150000 } }
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
| Model context window | Pi's limit (default reserve) | Threshold used |
|
|
257
|
+
| --- | --- | --- |
|
|
258
|
+
| 1,000,000 | 983,616 | 150,000 |
|
|
259
|
+
| 200,000 | 183,616 | 150,000 |
|
|
260
|
+
| 128,000 | 111,616 | 111,616 (Pi's limit is smaller) |
|
|
261
|
+
|
|
262
|
+
### `thresholdPercent`
|
|
263
|
+
|
|
264
|
+
Maintenance starts when the context is larger than this percentage of the model's context window.
|
|
265
|
+
|
|
266
|
+
- `-1` (default): no percentage. If `thresholdTokens` is also `-1`, use Pi's limit.
|
|
267
|
+
- A number above 0, up to 100: start at this percentage. Decimals are allowed, for example `72.5`. Values above 99 are used as 99.
|
|
268
|
+
|
|
269
|
+
pi-context-kit ignores this setting when `thresholdTokens` is not `-1`.
|
|
270
|
+
|
|
271
|
+
Example: start at 75% of the context window.
|
|
272
|
+
|
|
273
|
+
```json
|
|
274
|
+
{ "contextKit": { "thresholdPercent": 75 } }
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
| Model context window | Threshold used |
|
|
278
|
+
| --- | --- |
|
|
279
|
+
| 1,000,000 | 750,000 |
|
|
280
|
+
| 200,000 | 150,000 |
|
|
281
|
+
| 128,000 | 96,000 |
|
|
282
|
+
|
|
283
|
+
Use `thresholdPercent` if you use models with different context windows and you want the same proportion for all of them. Use `thresholdTokens` if you want the same number for all of them.
|
|
284
|
+
|
|
285
|
+
### `methodOrder`
|
|
286
|
+
|
|
287
|
+
The methods that pi-context-kit tries when pruning is not sufficient, in this order.
|
|
288
|
+
|
|
289
|
+
Allowed values in the list:
|
|
290
|
+
|
|
291
|
+
| Value | What it does | Calls the model? |
|
|
292
|
+
| --- | --- | --- |
|
|
293
|
+
| `"handoff"` | Replaces the old conversation with a handoff note. The newest messages stay. | Yes, the session model |
|
|
294
|
+
| `"shake"` | Replaces large old code blocks, XML blocks, and images with a short notice. | No |
|
|
295
|
+
| `"soft"` | Replaces the old conversation with Pi's summary. The newest messages stay. Stops the list. | Yes, the session model |
|
|
296
|
+
| `"remote"`, `"snapcompact"` | Accepted for oh-my-pi compatibility, but skipped. Pi does not have these methods. | No |
|
|
297
|
+
|
|
298
|
+
Rules:
|
|
299
|
+
|
|
300
|
+
- The order is important. pi-context-kit tries the first method, then the next one.
|
|
301
|
+
- `"soft"` ends the list. Methods after `"soft"` never run.
|
|
302
|
+
- If `"handoff"` is not in the list before `"soft"`, pi-context-kit writes no handoff note at all. This also applies to `/compact` and to background notes. Pi writes its own summary.
|
|
303
|
+
- If `"shake"` is not in the list before `"soft"`, nothing is shaken.
|
|
304
|
+
- A value that appears two times is used one time.
|
|
305
|
+
- An empty list `[]` is allowed. Then pi-context-kit only replaces old file reads and prunes old tool outputs.
|
|
306
|
+
|
|
307
|
+
Common choices:
|
|
308
|
+
|
|
309
|
+
| `methodOrder` | What happens above the threshold |
|
|
310
|
+
| --- | --- |
|
|
311
|
+
| `["handoff", "shake", "soft"]` (default) | Prune. Then a handoff note. If the note fails, shake. Then Pi's summary. |
|
|
312
|
+
| `["shake", "handoff", "soft"]` | Prune and shake together. If that is not sufficient, a handoff note. Then Pi's summary. |
|
|
313
|
+
| `["handoff", "soft"]` | Prune. Then a handoff note. Never shake. |
|
|
314
|
+
| `["shake", "soft"]` | Prune and shake. Then Pi's summary. No handoff note. |
|
|
315
|
+
| `["soft"]` | Prune. Then Pi's summary. |
|
|
316
|
+
| `[]` | Prune only. Pi writes its summary when the context reaches Pi's limit. |
|
|
317
|
+
|
|
318
|
+
Example: try shaking first, because it does not call the model.
|
|
319
|
+
|
|
320
|
+
```json
|
|
321
|
+
{ "contextKit": { "methodOrder": ["shake", "handoff", "soft"] } }
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
### `asyncEnabled`
|
|
325
|
+
|
|
326
|
+
Prepares the handoff note in the background, before the threshold.
|
|
327
|
+
|
|
328
|
+
- `true` (default): the note starts one eighth of the threshold early, with a maximum of 32,000 tokens. At the threshold, the note is often ready, so compaction is fast.
|
|
329
|
+
- `false`: no background note. At the threshold, pi-context-kit writes a new note. Your work waits for this note.
|
|
330
|
+
|
|
331
|
+
This setting has an effect only if `"handoff"` is in `methodOrder` before `"soft"`.
|
|
332
|
+
|
|
333
|
+
Example: no background requests.
|
|
334
|
+
|
|
335
|
+
```json
|
|
336
|
+
{ "contextKit": { "asyncEnabled": false } }
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
### `supersedeReads`
|
|
340
|
+
|
|
341
|
+
Replaces the old copy of a file when the model reads the same file again.
|
|
342
|
+
|
|
343
|
+
- `true` (default): the old copy becomes `[Superseded by a newer read of this file]`.
|
|
344
|
+
- `false`: all copies stay.
|
|
345
|
+
|
|
346
|
+
This works at every turn, also below the threshold.
|
|
347
|
+
|
|
348
|
+
```json
|
|
349
|
+
{ "contextKit": { "supersedeReads": false } }
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
### `staleSuffixTokens`
|
|
353
|
+
|
|
354
|
+
Limits which old file copies `supersedeReads` can replace.
|
|
355
|
+
pi-context-kit replaces an old copy only if the old copy and all newer text are smaller than this number of tokens. Thus only recent copies are replaced.
|
|
356
|
+
|
|
357
|
+
Why: the provider can keep the start of the conversation in a cache. A change in old text can make the provider read all the text after it again, which costs more.
|
|
358
|
+
|
|
359
|
+
- `8000` (default).
|
|
360
|
+
- A larger value replaces more copies, but can cause more cache misses.
|
|
361
|
+
- `0` replaces nothing. It has the same effect as `"supersedeReads": false`.
|
|
362
|
+
|
|
363
|
+
```json
|
|
364
|
+
{ "contextKit": { "staleSuffixTokens": 16000 } }
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
### `progressRatio`
|
|
368
|
+
|
|
369
|
+
How small the context must become after a method, as a fraction of the threshold.
|
|
370
|
+
If a method cannot make the context this small, pi-context-kit does not use its result and tries the next method.
|
|
371
|
+
|
|
372
|
+
- `0.8` (default): the context must become 80% of the threshold or less. With a threshold of 150,000 tokens, this is 120,000 tokens.
|
|
373
|
+
- A smaller value, for example `0.6`, gives more free space after maintenance. But each method must remove more, so it fails more often.
|
|
374
|
+
- A larger value, for example `0.95`, makes each method succeed more easily. But maintenance can start again soon after.
|
|
375
|
+
|
|
376
|
+
The value must be above 0 and not more than 1.
|
|
377
|
+
|
|
378
|
+
```json
|
|
379
|
+
{ "contextKit": { "progressRatio": 0.7 } }
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
### `prune.enabled`
|
|
383
|
+
|
|
384
|
+
Turns pruning on or off. Pruning replaces old tool outputs with a notice, for example `[Output pruned: about 5000 tokens. Call recall with entryId "abc123" to read it.]`.
|
|
385
|
+
|
|
386
|
+
- `true` (default).
|
|
387
|
+
- `false`: tool outputs are never pruned.
|
|
388
|
+
|
|
389
|
+
```json
|
|
390
|
+
{ "contextKit": { "prune": { "enabled": false } } }
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
### `prune.protectTokens`
|
|
394
|
+
|
|
395
|
+
The newest tool outputs, up to this number of tokens, are never pruned. Thus the model keeps its most recent results.
|
|
396
|
+
|
|
397
|
+
- `40000` (default).
|
|
398
|
+
- A larger value keeps more recent outputs, but removes less.
|
|
399
|
+
|
|
400
|
+
```json
|
|
401
|
+
{ "contextKit": { "prune": { "protectTokens": 60000 } } }
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
### `prune.minSavings`
|
|
405
|
+
|
|
406
|
+
Pruning occurs only if it removes at least this number of tokens. This prevents small changes with little gain.
|
|
407
|
+
|
|
408
|
+
- `20000` (default).
|
|
409
|
+
|
|
410
|
+
```json
|
|
411
|
+
{ "contextKit": { "prune": { "minSavings": 10000 } } }
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
### `prune.minTokens`
|
|
415
|
+
|
|
416
|
+
Tool outputs smaller than this number of tokens are never pruned. A notice would not be much smaller than these outputs.
|
|
417
|
+
|
|
418
|
+
- `50` (default).
|
|
419
|
+
|
|
420
|
+
```json
|
|
421
|
+
{ "contextKit": { "prune": { "minTokens": 200 } } }
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
### `shake.protectTokens`
|
|
425
|
+
|
|
426
|
+
The newest part of the context, up to this number of tokens, is never shaken. Your own messages are never shaken, also when they are old.
|
|
427
|
+
|
|
428
|
+
- `16000` (default).
|
|
429
|
+
|
|
430
|
+
```json
|
|
431
|
+
{ "contextKit": { "shake": { "protectTokens": 32000 } } }
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
### `shake.minSavings`
|
|
435
|
+
|
|
436
|
+
Shaking occurs only if it removes at least this number of tokens.
|
|
437
|
+
|
|
438
|
+
- `4000` (default).
|
|
439
|
+
|
|
440
|
+
```json
|
|
441
|
+
{ "contextKit": { "shake": { "minSavings": 8000 } } }
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
### `shake.blockMinTokens`
|
|
445
|
+
|
|
446
|
+
Code blocks and XML blocks smaller than this number of tokens are never shaken.
|
|
447
|
+
|
|
448
|
+
- `400` (default).
|
|
449
|
+
|
|
450
|
+
```json
|
|
451
|
+
{ "contextKit": { "shake": { "blockMinTokens": 1000 } } }
|
|
452
|
+
```
|
|
453
|
+
|
|
454
|
+
There is no `shake.enabled` setting. To stop shaking, remove `"shake"` from `methodOrder`.
|
|
455
|
+
|
|
456
|
+
### Pi settings that also apply
|
|
457
|
+
|
|
458
|
+
These settings belong to Pi, not to pi-context-kit. Put them in the `compaction` block, next to the `contextKit` block.
|
|
459
|
+
|
|
460
|
+
| Pi setting | Default | Effect on pi-context-kit |
|
|
52
461
|
| --- | --- | --- |
|
|
53
|
-
| `enabled` | `true` |
|
|
54
|
-
| `
|
|
55
|
-
| `
|
|
56
|
-
| `
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
462
|
+
| `compaction.enabled` | `true` | `false` stops Pi's compaction. Then pi-context-kit only replaces old file reads. |
|
|
463
|
+
| `compaction.reserveTokens` | `16384` | Pi's limit is the context window minus this number. pi-context-kit always starts at this limit or before. |
|
|
464
|
+
| `compaction.keepRecentTokens` | `20000` | After a handoff note, approximately this number of the newest tokens stays unchanged. |
|
|
465
|
+
| `compaction.modelOverrides` | none | Changes `reserveTokens` and `keepRecentTokens` for one model. |
|
|
466
|
+
|
|
467
|
+
pi-context-kit has no setting per model. To start earlier for one model only, make Pi's limit smaller for this model with `compaction.modelOverrides`. pi-context-kit always starts at Pi's limit or before, so it also starts earlier for this model.
|
|
468
|
+
|
|
469
|
+
Example: all models start at 150,000 tokens. One model with a 200,000-token window starts at 200,000 − 80,000 = 120,000 tokens. Replace `provider/model-id` with the ID of your model.
|
|
470
|
+
|
|
471
|
+
```json
|
|
472
|
+
{
|
|
473
|
+
"compaction": {
|
|
474
|
+
"modelOverrides": {
|
|
475
|
+
"provider/model-id": { "reserveTokens": 80000 }
|
|
476
|
+
}
|
|
477
|
+
},
|
|
478
|
+
"contextKit": {
|
|
479
|
+
"thresholdTokens": 150000
|
|
480
|
+
}
|
|
481
|
+
}
|
|
482
|
+
```
|
|
483
|
+
|
|
484
|
+
Pi also uses `reserveTokens` to limit the length of its own summary. See the Pi documentation about compaction.
|
|
485
|
+
|
|
486
|
+
The model key must be the exact `provider/modelId`. To see the IDs, type `pi --list-models`.
|
|
487
|
+
|
|
488
|
+
## Recipes
|
|
489
|
+
|
|
490
|
+
| I want to... | Settings |
|
|
491
|
+
| --- | --- |
|
|
492
|
+
| Start earlier with large models | `{ "contextKit": { "thresholdTokens": 150000 } }` |
|
|
493
|
+
| Use the same proportion for all models | `{ "contextKit": { "thresholdPercent": 75 } }` |
|
|
494
|
+
| Never write a handoff note | `{ "contextKit": { "methodOrder": ["shake", "soft"] } }` |
|
|
495
|
+
| Try the free method first, then a handoff note | `{ "contextKit": { "methodOrder": ["shake", "handoff", "soft"] } }` |
|
|
496
|
+
| Keep more recent tool outputs | `{ "contextKit": { "prune": { "protectTokens": 80000 } } }` |
|
|
497
|
+
| Keep all old file copies | `{ "contextKit": { "supersedeReads": false } }` |
|
|
498
|
+
| Use other settings in one project | Put the `contextKit` block in `.pi/settings.json` in that project. |
|
|
499
|
+
| Turn pi-context-kit off, but keep `recall` | `{ "contextKit": { "enabled": false } }` |
|
|
500
|
+
|
|
501
|
+
To combine recipes, put all the settings in one `contextKit` block:
|
|
502
|
+
|
|
503
|
+
```json
|
|
504
|
+
{
|
|
505
|
+
"contextKit": {
|
|
506
|
+
"thresholdTokens": 150000,
|
|
507
|
+
"methodOrder": ["shake", "handoff", "soft"],
|
|
508
|
+
"prune": { "protectTokens": 80000 }
|
|
509
|
+
}
|
|
510
|
+
}
|
|
511
|
+
```
|
|
512
|
+
|
|
513
|
+
## Limits
|
|
514
|
+
|
|
515
|
+
- **Pi still has its own limit.** Pi compacts when the context reaches the context window minus `compaction.reserveTokens` (default 16,384). pi-context-kit can only start earlier, not later. You set Pi's limit with `compaction.reserveTokens` and `compaction.modelOverrides`.
|
|
516
|
+
- **A threshold is not a guarantee.** At the threshold, pi-context-kit tries to make the context smaller. If one tool output is very large, or the summary request fails, the methods can fail. Then Pi compacts at its own limit.
|
|
517
|
+
- **Token counts are estimates.** pi-context-kit uses Pi's estimates. The real count of the provider can be different.
|
|
518
|
+
- **If Pi's compaction is off** (`compaction.enabled: false`), pi-context-kit only replaces old file reads.
|
|
519
|
+
- **Manual compaction and errors.** When you type `/compact`, when the context is too large for the model, or when Pi compacts before a turn, `shake` cannot run. pi-context-kit uses a ready handoff note, or Pi's summary.
|
|
520
|
+
- **`methodOrder` stops at `soft`.** The methods after `soft` do not run.
|
|
521
|
+
- **Subagents** use their own settings. They show no warnings, because they have no screen.
|
|
522
|
+
|
|
523
|
+
## Questions
|
|
524
|
+
|
|
525
|
+
**Does it slow down my work?**
|
|
526
|
+
Usually not. The handoff note is written in the background, and pruning and shaking do not use the model.
|
|
527
|
+
If no note is ready at the threshold, pi-context-kit can postpone the handoff until the note in progress is ready. It can also write a new note, which adds one model request.
|
|
528
|
+
|
|
529
|
+
**Does it cost more?**
|
|
530
|
+
A handoff note is one extra model request. It uses the same model, thinking level, and tool choice as your session. With Anthropic, this lets the provider reuse the cached conversation. Caching is not guaranteed.
|
|
531
|
+
|
|
532
|
+
**What happens if the handoff note fails?**
|
|
533
|
+
Pi shows a warning that starts with `pi-context-kit: handoff note failed`. Then pi-context-kit uses the next method, or Pi's summary. After a failed note at the threshold, pi-context-kit does not try a new note at the threshold again before the next compaction.
|
|
534
|
+
|
|
535
|
+
**Can I give a focus to a summary?**
|
|
536
|
+
Yes. Type `/compact` and the focus, for example `/compact keep the database migration details`. If enough space is available, pi-context-kit writes a new note with this focus. If not, Pi writes its summary.
|
|
537
|
+
|
|
538
|
+
**The model forgot a detail. What can I do?**
|
|
539
|
+
Ask the model to use the `recall` tool, for example: "Use recall to find the error message from the first test run."
|
|
74
540
|
|
|
75
541
|
## Development
|
|
76
542
|
|
|
77
543
|
```sh
|
|
78
|
-
npm
|
|
544
|
+
npm ci
|
|
79
545
|
npm run check
|
|
80
546
|
```
|
|
81
547
|
|
|
82
|
-
The tests run offline with Pi's faux provider.
|
|
548
|
+
The tests run offline with Pi's faux provider. They do not call a real model.
|
|
83
549
|
|
|
84
550
|
## Contributing and releases
|
|
85
551
|
|
package/package.json
CHANGED