pi-dynamic-context-pruning 0.1.10 → 0.1.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -43
- package/index.ts +2133 -1976
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -27,11 +27,23 @@ and why it hasn't been built yet.
|
|
|
27
27
|
key wins when replaying history, so a decision can be pruned, restored, and
|
|
28
28
|
re-pruned any number of times.
|
|
29
29
|
- **Recomputed every call, not on a timer.** All three automatic strategies run
|
|
30
|
-
fresh on every `context` event, over the current message array.
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
30
|
+
fresh on every `context` event, over the current message array. Pi 0.99's
|
|
31
|
+
`context` hook intentionally receives the mutable conversation without system
|
|
32
|
+
messages; Pi restores its system prompt and tool state after the handler. The
|
|
33
|
+
extension does not try to rebuild that request-only envelope.
|
|
34
|
+
- **Canonical picker input.** `/prune` uses Pi's canonical session projection when
|
|
35
|
+
available, so context omissions and replacements, branch selection, and tool
|
|
36
|
+
pairing match the provider-visible conversation. Older hosts retain the raw
|
|
37
|
+
message-entry fallback.
|
|
38
|
+
- **Nested calls stay metadata.** Pi records bounded `nestedCalls` metadata on a
|
|
39
|
+
parent tool result, but nested results are not transcript messages. Strategies
|
|
40
|
+
count and prune only the parent transcript tool result; they never materialize
|
|
41
|
+
child entries, double-count child output, or infer complete history from a
|
|
42
|
+
truncated/incomplete record.
|
|
43
|
+
- **Persisted decisions remain branch-local.** Nothing is pruned "once and
|
|
44
|
+
forgotten"; if a tool result would no longer qualify (e.g. it's now within the
|
|
45
|
+
protected recency window), it isn't proposed again, but an already-applied,
|
|
46
|
+
persisted decision still applies until explicitly restored.
|
|
35
47
|
|
|
36
48
|
## Strategies
|
|
37
49
|
|
|
@@ -133,7 +145,7 @@ recoup the one-time cache-bust cost via the recurring saving. The gate accepts
|
|
|
133
145
|
a batch of candidates only if `breakEvenCalls` is at most the effective
|
|
134
146
|
threshold for the current state (`idle=22`, `mid_loop=1` by default,
|
|
135
147
|
calibrated from the representative-corpus benchmark — see the calibration
|
|
136
|
-
note under [Configuration](#configuration)
|
|
148
|
+
note under [Configuration](#configuration)).
|
|
137
149
|
|
|
138
150
|
All candidates proposed in the same `context` call share one cache bust (since
|
|
139
151
|
a prompt cache is a single linear prefix), so they're evaluated and
|
|
@@ -250,7 +262,7 @@ Notes:
|
|
|
250
262
|
`idle` if the most recent relevant message is a user message (this is the
|
|
251
263
|
first LLM call of the turn), `mid_loop` if it's an assistant message or a
|
|
252
264
|
tool result (the agent is iterating mid-turn). The representative-corpus
|
|
253
|
-
benchmark
|
|
265
|
+
benchmark was re-derived against this same
|
|
254
266
|
runtime-observable definition and found the state-split REVERSED from an
|
|
255
267
|
earlier turn-end-based analysis: at r=0.1, `idle` candidates now carry
|
|
256
268
|
essentially all the realized net benefit and `mid_loop` candidates carry
|
|
@@ -265,8 +277,7 @@ Notes:
|
|
|
265
277
|
impossible. Set `mid_loop` back to `22` to restore the old parity behavior.
|
|
266
278
|
- `gate.breakEvenThreshold`'s default of `22` is calibrated from the
|
|
267
279
|
representative-corpus benchmark at `cachedPriceRatio` r=0.1 (aggressive
|
|
268
|
-
prompt caching, the common case)
|
|
269
|
-
evidence and its ratio-sensitivity caveats. It is **provider/ratio-
|
|
280
|
+
prompt caching, the common case). It is **provider/ratio-
|
|
270
281
|
dependent**: the optimal threshold rises as caching gets weaker (T=29 at
|
|
271
282
|
r=0.25, T=54 at r=0.5, T=58 at r=0.9 on the same corpus).
|
|
272
283
|
|
|
@@ -428,40 +439,9 @@ candidates — see the corpus-choice note above) produced:
|
|
|
428
439
|
more amortization runway than the earlier small-corpus run showed, because
|
|
429
440
|
this corpus is dominated by long, tool-heavy agent sessions rather than
|
|
430
441
|
short orchestrator sessions.
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
| ratio `r` | overall `T` (benefit) | mid_loop `T` (benefit) | idle `T` (benefit) |
|
|
435
|
-
| --- | --- | --- | --- |
|
|
436
|
-
| 0.1 | T=22 (~20.6k) | T=22 (~20.6k) | T=1 (0 — zero benefit) |
|
|
437
|
-
| 0.25 | T=29 (~1.54M) | T=29 (~1.54M) | T=12 (~4.0k) |
|
|
438
|
-
| 0.5 | T=54 (~8.06M) | T=54 (~8.06M) | T=27 (~37.2k) |
|
|
439
|
-
| 0.9 | T=58 (~24.7M) | T=58 (~24.2M) | T=21 (~459k) |
|
|
440
|
-
|
|
441
|
-
**Reading**: `mid_loop` candidates carry essentially all of the realized
|
|
442
|
-
benefit; `idle` candidates carry essentially none (literally zero at r=0.1).
|
|
443
|
-
At r=0.1 (aggressive prompt caching, the common Anthropic case) the total
|
|
444
|
-
realized benefit across the whole 1,390-session corpus is only ~20.6k
|
|
445
|
-
token-units — economically marginal, on the order of pennies. Savings only
|
|
446
|
-
become material at weaker caching, r>=0.25.
|
|
447
|
-
|
|
448
|
-
**pe-zy4s update (2026-07-08): this idle/mid_loop split used a turn-END
|
|
449
|
-
definition** ("idle" = this call IS the turn's final assistant message, only
|
|
450
|
-
knowable in hindsight via replay) **that was never runtime-observable.** Once
|
|
451
|
-
real runtime agent-state detection landed (turn-START definition: "idle" =
|
|
452
|
-
this is the FIRST LLM call of a turn; see the config-notes bullet above), the
|
|
453
|
-
benchmark's candidate labeling was aligned to that same definition and
|
|
454
|
-
re-derived on the representative corpus (460 gate-eligible candidates on the
|
|
455
|
-
current `~/.the-last-harness/agent/sessions` corpus). The split **reverses**:
|
|
456
|
-
at r=0.1, `idle` (T=22, ~20.6k realized net benefit) now carries essentially
|
|
457
|
-
all the value, and `mid_loop` (T=1, ~0 realized net benefit) carries
|
|
458
|
-
essentially none. Under that ticket's decision rule (stricter idle default
|
|
459
|
-
only if idle is shown ~worthless), parity (`22`/`22`) was retained at the time.
|
|
460
|
-
The 2026-09-08 follow-up supersedes that default with `idle=22` / `mid_loop=1`,
|
|
461
|
-
using the measured mid-loop result to choose a conservative threshold (see
|
|
462
|
-
[Configuration](#configuration) for the evidence and limitations). The table
|
|
463
|
-
above is retained as the historical turn-END-definition record; it no longer
|
|
464
|
-
reflects how the runtime classifies agent state.
|
|
442
|
+
|
|
443
|
+
Shipped gate defaults remain `idle=22` / `mid_loop=1` (see
|
|
444
|
+
[Configuration](#configuration)).
|
|
465
445
|
|
|
466
446
|
**This reframes, rather than weakens, the case for v2.** Small deterministic
|
|
467
447
|
removals (dedupe/error-purge/superseded-file-ops) structurally cannot beat
|