pi-fold 3.1.0 → 4.1.0

package/README.md

@@ -4,7 +4,7 @@
 
 **Rotate stale text out of the context window without losing it, and spend the prefix cache once per commit instead of continuously.** The oldest part of a session collapses into short briefs while the exact originals stay on disk, one tool call away. Folding is automatic and needs no agent participation. The shipped shape is deterministic briefs paired with a durable memory store, which is the cheapest configuration measured that kept recall intact.
 
-![pi-fold folding a session in place: marks outline completed spans as the session works, and the marks commit together at a fold event](https://raw.githubusercontent.com/shaneconner/pi-fold/main/media/fold-demo.gif)
+![pi-fold folding a session in place: marks outline completed spans as the session works, and the marks commit together at a fold event](https://raw.githubusercontent.com/shaneconner/fold/main/media/fold-demo.gif)
 
 *Marks outline spans as they complete and move nothing; the next commit applies every standing mark in one rewrite. Expanding a brief brings the exact messages back byte for byte. Illustrative animation drawn on the 2.x surface, where the agent laid the marks; since 3.0 the runtime lays them.*
 
@@ -16,15 +16,15 @@ pi install npm:pi-fold
 
 Node 22 or later, Pi 0.83 or later. MIT licensed.
 
-Folding does not require a memory store, but every best result measured came from pairing it with one. The campaigns used [pi-canon](https://github.com/shaneconner/pi-canon), installed separately. The two began as a single working setup and were split into separate extensions for modularity, and the experiments are what established that they work better as complements than either does alone. Only that pairing has been measured. Any durable long-term memory store is expected to compose the same way, though that is a suspicion rather than a result.
+Folding does not require a memory store, but every best result measured came from pairing it with one. The campaigns used [pi-canon](https://github.com/shaneconner/canon), installed separately. The two began as a single working setup and were split into separate extensions for modularity, and the experiments are what established that they work better as complements than either does alone. Only that pairing has been measured. Any durable long-term memory store is expected to compose the same way, though that is a suspicion rather than a result.
 
 ## What it does
 
 Compaction replaces the transcript with a summary and discards the originals. Folding replaces stale spans with briefs and keeps the originals retrievable.
 
 <picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/shaneconner/pi-fold/main/media/fold-vs-compaction-dark.svg">
-  <img alt="Compaction turns the transcript into a summary and discards the originals, so the session starts over. pi-fold turns it into a brief and keeps the exact originals on disk, one peek or expand away, so the session keeps going." src="https://raw.githubusercontent.com/shaneconner/pi-fold/main/media/fold-vs-compaction-light.svg">
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/shaneconner/fold/main/media/fold-vs-compaction-dark.svg">
+  <img alt="Compaction turns the transcript into a summary and discards the originals, so the session starts over. pi-fold turns it into a brief and keeps the exact originals on disk, one peek or expand away, so the session keeps going." src="https://raw.githubusercontent.com/shaneconner/fold/main/media/fold-vs-compaction-light.svg">
 </picture>
 
 A fold takes a contiguous span of session entries and replaces it, in the window only, with a short brief. The entries go to a fold store byte for byte, addressed by their SHA-256 hash. The brief carries a handle, the handle resolves to the original, and expansion restores the exact bytes after verifying the hash.
@@ -51,7 +51,7 @@ Compaction alone abstained nine times and was wrong three, so its dominant failu
 
 Agent-written briefs scored worst of anything measured, which is why that invitation ships off.
 
-Four papers with figure sources, redacted per-request ledgers and campaign logs are on Zenodo: [design and trace evaluation](https://doi.org/10.5281/zenodo.21856873), [working memory under context shedding](https://doi.org/10.5281/zenodo.21980746), [ephemeral retrieval](https://doi.org/10.5281/zenodo.22142454), and [rotation and retention](https://doi.org/10.5281/zenodo.22142456), which is the campaign above. Narrative versions on [Medium](https://medium.com/@shane.conner/a-compaction-summary-is-one-record-doing-a-stores-job-3f46212ef059), interactive versions at [shaneconner.com](https://shaneconner.com/projects/pi-fold/), and the full chronology in [the experiment log](https://github.com/shaneconner/pi-fold/blob/main/docs/fold_vs_compaction/experiment-log.md).
+Four papers with figure sources, redacted per-request ledgers and campaign logs are on Zenodo: [design and trace evaluation](https://doi.org/10.5281/zenodo.21856873), [working memory under context shedding](https://doi.org/10.5281/zenodo.21980746), [ephemeral retrieval](https://doi.org/10.5281/zenodo.22142454), and [rotation and retention](https://doi.org/10.5281/zenodo.22142456), which is the campaign above. Narrative versions on [Medium](https://medium.com/@shane.conner/a-compaction-summary-is-one-record-doing-a-stores-job-3f46212ef059), interactive versions at [shaneconner.com](https://shaneconner.com/projects/pi-fold/), and the full chronology in [the experiment log](https://github.com/shaneconner/fold/blob/main/docs/fold_vs_compaction/experiment-log.md).
 
 ## Limits
 
@@ -59,6 +59,25 @@ Four papers with figure sources, redacted per-request ledgers and campaign logs
 - **The benchmark is narrow by construction.** It was built to reproduce specific things compaction handles badly, one to four runs per condition, and it says nothing about production behavior.
 - **Folding is not a memory store.** It rotates transcript source out of the window; durable placement of what a session learned is a separate job, and the pairing is where the measured value sat.
 
+## Configuration
+
+`registerPiFold(pi)` works with no arguments, and the defaults are the configuration the campaign above actually ran: deterministic briefs, the invitation off, the tool-call diet at half the window, and a 0.80/0.20 band. Nothing here needs tuning.
+
+The one thing worth adding is a long-term memory store. Rotation and retention are different jobs: folding gets stale material out of the window, a store keeps what the session learned. A handoff summary tries to do both in one record, written once under time pressure, bounded by the window it has to fit in. Folding is the rotation half and only that half.
+
+Six options, and every other name is refused.
+
+| Option | Default | Effect |
+| --- | --- | --- |
+| `thresholds` | `{ maxTarget: 0.80, minTarget: 0.20, consolidateAfter: 10, minFoldChars: 8000 }` | The band, set whole or not at all. The first two are proportions of the serving budget. `consolidateAfter` is how many visible roots a parent is owed. `minFoldChars` is the size a fold has to reach to be worth making, and the floor a gap between folds has to clear to stay raw. Validated atomically; an impossible policy is refused by name rather than clamped. |
+| `providerInputBudget` | the transport's descriptor | The tokens this deployment may actually put in a request, already net of any output reservation. Every ratio, fence and budget on this page is a share of this one number. |
+| `blacklistAutoFoldTools` | `new Set()` | The exception list. Every completed tool batch is foldable without an agent mark; name a tool here and its results stay raw. |
+| `toolFoldThreshold` | `0.50` | The tool-call diet: a share in `[0, 1)` naming the oldest fraction of the window whose tool results are clipped in view. `0` turns it off. |
+| `preCommitNotice` | `false` | The notice switch. False is the shape that won the campaign above: no carrier reaches the projection and every fold ships the runtime's own words. `true` states the window's status to the agent once per approach to a commit: where the commit fires, how far away it is, what is staged, what that frees, what is pinned and what the pin costs, with the verbs listed and no argument made for using any of them. |
+| `noticeLeadShare` | `0.10` | How far below the commit trigger that notice speaks, as a share of the serving budget. A distance rather than a point, so it follows a band that moves. Read only when `preCommitNotice` is on. |
+
+Four commands ship for the human: `/fold-status`, `/fold` (commit every staged mark now), `/fold-editor` (the interactive map of the window) and `/fold-settings` (edit the options above; each change is saved to `~/.config/pi-fold/settings.json` and takes effect in the running session).
+
 ## How it works
 
 ### The cache invariant
@@ -70,8 +89,8 @@ Four papers with figure sources, redacted per-request ledgers and campaign logs
 ### Marks and commits
 
 <picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/shaneconner/pi-fold/main/media/marks-and-commits-dark.svg">
-  <img alt="A completed unit becomes a pending mark that leaves the window and its cached prefix untouched. Marks repeat for free below maxTarget; at maxTarget one commit applies every standing mark in a single rewrite. From the brief, peek appends the exact bytes and expand restores them in place, both after verifying the SHA-256." src="https://raw.githubusercontent.com/shaneconner/pi-fold/main/media/marks-and-commits-light.svg">
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/shaneconner/fold/main/media/marks-and-commits-dark.svg">
+  <img alt="A completed unit becomes a pending mark that leaves the window and its cached prefix untouched. Marks repeat for free below maxTarget; at maxTarget one commit applies every standing mark in a single rewrite. From the brief, peek appends the exact bytes and expand restores them in place, both after verifying the SHA-256." src="https://raw.githubusercontent.com/shaneconner/fold/main/media/marks-and-commits-light.svg">
 </picture>
 
 A mark moves nothing. The moment a tool batch or chapter closes, the runtime cuts it into a pending mark, up to eight cuts per pass, stalest first. The mark lives in durable state outside the window, so the projection stays byte-identical and the cached prefix survives. Marks accumulate for free until one commit applies all of them in a single rewrite.
@@ -110,27 +129,6 @@ Pi's automatic compaction stays enabled underneath and is intercepted by kind. A
 
 Evidence ingestion writes read-only artifacts under the session directory's `pi-fold-evidence/` at mode `0444`, under a 512 MB session cap. It is always on: the artifacts are what an oversized result folds against, so a deployment that stopped writing them would keep its folds and lose what makes them lossless.
 
-## Configuration
-
-`registerPiFold(pi)` works with no arguments, and the defaults are the configuration the campaign above actually ran: deterministic briefs, the invitation off, the tool-call diet at half the window, and a 0.80/0.20 band. Nothing here needs tuning.
-
-The one thing worth adding is a long-term memory store. Rotation and retention are different jobs: folding gets stale material out of the window, a store keeps what the session learned. A handoff summary tries to do both in one record, written once under time pressure, bounded by the window it has to fit in. Folding is the rotation half and only that half.
-
-Six options, and every other name is refused.
-
-| Option | Default | Effect |
-| --- | --- | --- |
-| `thresholds` | `{ maxTarget: 0.80, minTarget: 0.20, consolidateAfter: 10, minFoldChars: 8000 }` | The band, set whole or not at all. The first two are proportions of the serving budget. `consolidateAfter` is how many visible roots a parent is owed. `minFoldChars` is the size a fold has to reach to be worth making, and the floor a gap between folds has to clear to stay raw. Validated atomically; an impossible policy is refused by name rather than clamped. |
-| `providerInputBudget` | the transport's descriptor | The tokens this deployment may actually put in a request, already net of any output reservation. Every ratio, fence and budget on this page is a share of this one number. |
-| `blacklistAutoFoldTools` | `new Set()` | The exception list. Every completed tool batch is foldable without an agent mark; name a tool here and its results stay raw. |
-| `toolFoldThreshold` | `0.50` | The tool-call diet: a share in `[0, 1)` naming the oldest fraction of the window whose tool results are clipped in view. `0` turns it off. |
-| `postFoldNotice` | `false` | The invitation switch. False is the shape that won the campaign above: no carrier invites a brief and every fold ships the runtime's own words. `true` appends a standing invitation to improve a pending brief. |
-| `workingMemory` | off | An in-window keyed dictionary the agent maintains beside the fold index, with `remember` and `recall` added to the tool. Offered to one run, which never called it. |
-
-Four commands ship for the human: `/fold-status`, `/fold` (commit every staged mark now), `/fold-editor` (the interactive map of the window) and `/fold-settings`.
-
-Upgrading from 2.x: the `fold` action is gone from the tool surface, since the runtime stages every completed unit itself; `guidance` is refused by name and `postFoldNotice` replaces it; `thresholds.freshTail` is gone, and a settings file carrying it is migrated with every tuned value kept. Existing sessions stay readable, folds and briefs included.
-
 ## Prior work
 
 Two recent lines inform this one. Lossless hierarchical compaction, exemplified by LCM ([arXiv:2605.04050](https://arxiv.org/abs/2605.04050)) with a walkthrough at [losslesscontext.ai](https://www.losslesscontext.ai/), builds a summary DAG over older messages with lossless pointers to every original. Agentic context management, exemplified by [Letta](https://github.com/letta-ai/letta) and by [arXiv:2510.11967](https://arxiv.org/abs/2510.11967), gives the agent tools over its own memory. See also [arXiv:2607.00692](https://arxiv.org/abs/2607.00692).