pi-fold 1.0.1 → 1.0.2

package/README.md

@@ -1,5 +1,7 @@
 # pi-fold
 
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21856873.svg)](https://doi.org/10.5281/zenodo.21856873)
+
 Agent-governed lossless context folding for Pi. A session folds its own transcript instead of losing it: the stale end collapses into short briefs while the exact originals stay on disk, addressed and verified by SHA-256, one tool call away.
 
 The full write-up, with interactive versions of every mechanism on this page and the complete measurement story, is at [shaneconner.com/projects/pi-fold](https://shaneconner.com/projects/pi-fold/).
@@ -28,7 +30,7 @@ Everything else follows from one invariant, and it is the reason several otherwi
 
 **So the runtime may move a byte in the window only at a moment it is already rewriting the window**, which is a commit or a fold. Everything else is a pure append at the tail, and an appended byte is never later altered, shortened, removed, or repositioned. Stated plainly: never mutate the window, and never show something and then take it back. The second half is the one that costs features. Anything that displays state and then updates it in place is out, however useful the display was, because updating in place is a mid-prefix write with better manners.
 
-**Marks are free, because a mark moves nothing.** Under `foldScheduling: "epoch"` a `fold` action records a pending mark and returns immediately. The mark lives in durable session state, outside the window; the projection stays byte-identical and the cached prefix survives. The agent can therefore mark as it works, the moment a span stops being load bearing, without weighing each fold against a cache invalidation.
+**Marks are free, because a mark moves nothing.** Under `foldScheduling: "epoch"` a `fold` action records a pending mark and returns immediately. The mark lives in durable session state, outside the window; the projection stays byte-identical and the cached prefix survives. The agent can therefore mark as it works, the moment a span stops mattering, without weighing each fold against a cache invalidation.
 
 **A commit is the one moment bytes move, and every accumulated decision lands at once.** It goes through the same preparation and commit machinery an immediate fold uses, so a committed mark and an immediate fold produce the identical fold record. Committing is the runtime's call, not a verb the agent can spend: measured 2026-08-07, an agent given the verb called it twice and the runtime correctly held it both times, which is surface without function.
 
@@ -50,6 +52,8 @@ One 64-stage staged assignment over the curl C repository: one Pi session per ru
 
 There is one run per arm, and native did not finish: it ended its turn at stage 56 of 64. That leaves two pairings, and both are reported here rather than the flattering one.
 
+The measurement is written up as a paper, with the figure sources, the redacted per-request ledgers, and the campaign log deposited beside it: [doi.org/10.5281/zenodo.21856873](https://doi.org/10.5281/zenodo.21856873). The story of how the campaign got here is on [Medium](https://medium.com/@shane.conner/compaction-doesnt-have-to-mean-starting-over-89c0b319d1a6).
+
 ### Same seed, both cut at stage 56
 
 **Basis:** each arm is cut at the ledger record that delivered the stage 56 payload, and usage and cost are summed from the same billed records at or before that cut, including native's three compaction records, which all precede it. Same seed, same plan hash `6ed7c5f9e957`, both runs on 2026-08-07.
@@ -90,7 +94,7 @@ There is one run per arm, and native did not finish: it ended its turn at stage
 
 **Where the cost numbers come from.** Cost is Pi's own per-request ledger accounting, not a rate card chosen for the writeup. Every billed record's cost equals that record's own usage priced at the tier Pi applied: base rates of $5.00/M input, $0.50/M cache read and $30.00/M output, and a long-context tier of $10.00/M, $1.00/M and $45.00/M on any request whose context exceeds the model's 272,000-token window. That reproduces the ledger to the cent on both arms, including pi-fold's complete-run total of $13.89. Reasoning tokens sit inside output and are not billed separately. Native's three compaction summarizations are themselves billed calls, $0.58 at stage 56 and $0.54 on the complete run.
 
-**Caveats, because they are load bearing.**
+**Caveats, because the numbers rest on them.**
 
 - **Fresh input is essentially identical.** 0.95x at stage 56 and 1.06x on the complete pairing: neither runtime consistently reads less new material. The 6.4 million token gap at stage 56 is re-sent accumulated context, 6.29 million of it cache read alone, not a difference in how much material was consumed.
 - **No wall-clock claim on the complete pairing.** pi-fold was 5.6% slower there, 34.2 minutes against 32.4. The wall-clock win belongs to the stage-56 pairing only.
@@ -116,7 +120,7 @@ Below 80% occupancy of the serving budget the runtime is quiet: nothing folds au
 
 Chapter briefs are generated by the session's model by default, using one request per fold with a bounded input span, bounded output, and no tool calls. Every summarizer failure falls back to a deterministic brief; set `summarizer: "deterministic"` to disable model calls entirely. The default spends provider tokens, as native compaction does. A cheaper model from the same provider is the recommended summarizer: a session on GPT 5.6 Sol can delegate fold briefs to GPT 5.6 Luna at medium effort with `summarizer: { provider: "openai", model: "gpt-5.6-luna", effort: "medium" }`, using the provider id from your Pi model registry.
 
-The serving budget is the total window less the output reservation actually in force, because the per-request max-input descriptor a provider advertises assumes a full reservation and understates the real ceiling. With no `providerTotalWindow` supplied the runtime falls back to the transport's descriptor, which is the honest default for a model it has no total-window fact for; the descriptor is reported in status either way, so the gap stays auditable. A provider-window fence aborts an unsafe request before transmission if no lossless fold can make room, and says so out of band rather than by writing into the window.
+The serving budget is the total window less the output reservation actually in force, because the per-request max-input descriptor a provider advertises assumes a full reservation and understates the real ceiling. With no `providerTotalWindow` supplied the runtime falls back to the transport's descriptor, which is the conservative default for a model it has no total-window fact for; the descriptor is reported in status either way, so the gap stays auditable. A provider-window fence aborts an unsafe request before transmission if no lossless fold can make room, and says so out of band rather than by writing into the window.
 
 By default, evidence ingestion writes read-only artifact files under the session directory's `pi-fold-evidence/`, sets them to mode `0444`, and enforces a 512 MB session cap. These immutable files provide exact-recovery anchors for oversized tool results. Set `evidenceIngestion: false` to disable the hook and all evidence writes.
 
@@ -162,7 +166,7 @@ pi-fold combines two recent lines of work in one mechanism. Lossless hierarchica
 
 Context-Folding ([arXiv:2510.11967](https://arxiv.org/abs/2510.11967), ByteDance Seed) has the agent branch into a sub-trajectory and fold it on completion, with the behavior shaped by reinforcement learning; the fold keeps a summary of the outcome rather than a lossless path back. MemGPT and its successor [Letta](https://github.com/letta-ai/letta) are the canonical agent-held memory-verb systems: the agent edits its own memory blocks and archives while the window pages against external stores. Two things separate pi-fold from that line. First, what is governed: Letta manages traffic between the window and outside storage, and when its message queue overflows the in-window record becomes a lossy recursive summary; pi-fold restructures the transcript in place. Second, recovery: theirs is retrieval, a search that re-ingests whatever comes back as new tokens; ours is expansion, which restores the exact folded entries after SHA-256 verification.
 
-The honest placement is narrow. The lossless DAG is shared ground. The distinguishing claim is agent curation, and the measured run never exercised it, so the numbers above belong to the fallback. The part that is new is stated here and not yet measured.
+The placement is narrow. The lossless DAG is shared ground. The distinguishing claim is agent curation, and the measured run never exercised it, so the numbers above belong to the fallback. The part that is new is stated here and not yet measured.
 
 The design also pairs with Pi's session trees: a fold's collapsed-or-expanded state is a natural unit for curating exactly which context a subagent or workflow leg inherits.
 

package/extensions/active-context.ts

@@ -1385,7 +1385,7 @@ export function registerActiveContext(pi: any, options: {
     if (chars === null || chars < PROJECTION_CALIBRATION_MIN_CHARS) return;
     if (!Number.isFinite(measurement.tokens) || measurement.tokens < PROJECTION_CALIBRATION_MIN_TOKENS) return;
     // The signed error of the estimate we made for the projection this measurement
-    // describes. This is the only honest measure of how wrong the fence can be.
+    // describes. This is the only direct measure of how wrong the fence can be.
     // Only a CALIBRATED estimate's error belongs in the window. The bootstrap constant
     // is wrong by design -- 76% high in rep12 -- and letting that one reading set the
     // margin would keep every later request permanently inside it.

package/extensions/lib/advisory.ts

@@ -174,7 +174,7 @@ export function updateAdvisoryMilestone(
  * Wording rule for every dosage below: report what is HAPPENING and name the actions
  * that change it. The earlier phrasing asked the agent to fold proactively, which is
  * advice it can only act on by interrupting its own task; the runtime folds either way,
- * so the honest thing to say is what it will do and what reacting buys.
+ * so the useful thing to say is what it will do and what reacting buys.
  */
 function curationText(milestone: AdvisoryMilestone, percent: number, toolName: string): string {
   if (milestone === "orientation") {

package/extensions/lib/curation.ts

@@ -272,7 +272,7 @@ export function advanceCurationGate(input: {
 /**
  * The last-call notice.
  *
- * Shape rule, load-bearing: operational status plus available actions, with continuing
+ * Shape rule, chosen deliberately: operational status plus available actions, with continuing
  * work as the stated default. A message shaped like a question inviting a chat-style
  * reply can make a single-turn agent emit a final answer and end its own run, so this
  * never asks anything and never implies a reply is expected.

package/extensions/lib/scheduling.ts

@@ -77,7 +77,7 @@ export function pendingMarks(state: Pick<ActiveContextState, "pendingMarks">): P
 }
 
 /**
- * Property ORDER is load-bearing: the state digest is a stable stringify, so an
+ * Property ORDER matters here: the state digest is a stable stringify, so an
  * appended key would replay as digest drift. Rebuild the tail of the record in the
  * same order `parseActiveContextState` produces.
  */

package/extensions/lib/surfacing.ts

@@ -195,7 +195,7 @@ export function surfacingLog(state: Pick<ActiveContextState, "surfacing">): Surf
 }
 
 /**
- * Property ORDER is load-bearing: the state digest is a stable stringify, so a
+ * Property ORDER matters here: the state digest is a stable stringify, so a
  * `surfacing` key assigned onto a state that did not already carry one would land
  * after `pendingMarks`/`advisory`/`prepared` and replay as digest drift. Rebuild the
  * tail of the record in the same order `parseActiveContextState` produces.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-fold",
-  "version": "1.0.1",
+  "version": "1.0.2",
   "description": "Lossless, agent-governed context folding for the Pi coding agent: only the oldest context changes; the fresh window stays untouched.",
   "keywords": [
     "pi-package",