pi-fold 0.0.1 → 0.1.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shane Conner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,18 +1,142 @@
 # pi-fold
 
-Lossless, agent-governed context folding for the [Pi coding agent](https://pi.dev).
+pi-fold lets a Pi agent govern its own context window. The registered `active_context` tool gives the session's agent seven actions over its transcript: `status`, `peek`, `fold`, `expand`, `refold`, `protect`, and `unprotect`. Folding is lossless: a fold replaces a stale span with a short brief while keeping SHA-256-verified references to every original entry, so any fold can be expanded back to its exact source. Advisory milestones tell the agent what is eligible as measured context pressure rises; the agent decides what stays hot, what folds, and what is pinned.
 
-Long sessions never stop to compact and never reread what they already knew: tool
-results and closed chapters fold into content-addressed, SHA-verified briefs that the
-agent can expand back to the exact original bytes at any time. Only the oldest context
-ever changes — the fresh working window stays untouched, so work continues mid-stride.
+When the agent does not act, an autonomous ladder manages the window the same lossless way: it folds completed read-only tool batches, folds structurally closed chapters, and consolidates existing folds into a hierarchy only past explicit thresholds. Only the oldest eligible context changes; the fresh window is never touched.
+
+![pi-fold folding a live session](https://raw.githubusercontent.com/shaneconner/pi-fold/main/docs/demo.gif)
+
+*A live session at 5.5× speed: the autonomous ladder folds a completed read batch mid-conversation, no command issued, and `/context` reports the fold.*
 
 ## Status
 
-**This version is a name reservation while the first release is finalized** — the
-runtime exists and is in production hardening (autonomous fold ladder, deterministic
-brief floor, expand leases, width consolidation, cadence-gated folding, a 18-gate
-verification suite, and multi-hour soak testing).
+This package is pre-release. Its API may move until 0.1.0. Licensed under [MIT](./LICENSE).
+
+## Install
+
+After the first 0.1.0 release:
+
+```sh
+pi install npm:pi-fold
+```
+
+## How it works
+
+pi-fold maps canonical session entries into a fold lattice. A fold replaces an eligible stale span with a short placeholder while retaining ordered references to the exact source entries. Expansion checks the stored SHA-256 identities before restoring that source.
+
+Collapsed folds double as a browsable index of earlier work. The briefs sit in the stable prefix of the window, which providers typically serve from cache, so the agent can page through them at little cost, note which spans matter to its current task, expand those, and leave the rest folded.
+
+The briefs are the index, and reading a fold is not the same as committing to it. `peek` is the ephemeral read: it returns one fold's exact SHA-256-verified source as a tool result, at any depth and with the ancestors still collapsed, without touching the projection; the content lands in the fresh window like any other read and can later fold away as a completed read batch. `expand` is the commitment, restoring that source in place until the fold is refolded, and `status` with `detail: "tree"` lists every fold, nested ones included, in transcript order with its depth and parent. Expansion is outside-in, level by level: expanding a parent reveals its children's briefs, each expandable in turn, and collapsing a parent re-collapses its descendants.
+
+The agent's own actions come first. Through the context tool it folds any structurally closed span it judges stale, folds completed tool batches, consolidates two or more adjacent folds into a deeper one at any time, expands any fold whose detail is needed again, and protects entries that must stay raw. None of the ladder's pressure or width thresholds apply to these actions.
+
+Every fold, agent-made or automatic, obeys the same structural rules. Folds form a DAG whose leaves are the exact session entries, and every fold record keeps ordered references down to those leaves, so expansion at any depth restores the originals after SHA-256 verification. Folds nest rather than partially overlap. Chapter folds align to closed user/assistant/tool-batch units, consolidations take two or more adjacent folds, and protected or fresh evidence never folds.
+
+The autonomous ladder is deliberately more conservative than the agent. It folds only completed read-only tool batches and structurally closed chapters, absorbs only the tool-result folds inside a chapter's own span, and creates folds of folds only past explicit thresholds, taking adjacent folds oldest first. Those thresholds are more than ten visible collapsed folds, which keeps the brief index tidy long before pressure matters, or total measured context usage (all of it, not just briefs) reaching 85% of the provider window, at which point consolidation squeezes existing folds to reclaim room. Briefs themselves are bounded at 1,200 characters each, so the visible index normally costs about 1% of the window. Cadence and fresh-tail rules prevent repeated or premature automatic folding.
+
+Chapter briefs are generated by the session's model by default, using one request per fold with a bounded input span, bounded output, and no tool calls. Every summarizer failure falls back to a deterministic brief; set `summarizer: "deterministic"` to disable model calls entirely. The default spends provider tokens, as native compaction does. A cheaper model from the same provider is the recommended summarizer: for example, a session on GPT 5.6 Sol can delegate fold briefs to GPT 5.6 Luna at medium effort with `summarizer: { provider: "openai", model: "gpt-5.6-luna", effort: "medium" }`, using the provider id from your Pi model registry. Advisory milestones describe eligible actions as measured context pressure rises, and the provider-window fence blocks an unsafe request if no lossless fold can make room.
+
+Milestone wording comes in three guidance profiles over the same rungs, thresholds, and per-session budgets. The default `"pressure"` frames each milestone as rising context pressure; `"curation"` frames the same milestones as curating context for the current task and adds one early orientation advisory, at 25% of the window, that introduces the fold index; `"minimal"` delivers only the urgent fence advisory.
+
+By default, evidence ingestion writes read-only artifact files under the session directory's `pi-fold-evidence/`, sets them to mode `0444`, and enforces a 512 MB session cap. These immutable files provide exact-recovery anchors for oversized tool results. Set `evidenceIngestion: false` to disable the hook and all evidence writes.
+
+## Surfacing relevant folds
+
+Paging the fold index is something the agent can do at any moment. Surfacing is that same act performed on its behalf when it did not think to look. As a session grows, the material most worth reopening tends to sit in the oldest and deepest folds, which are exactly the ones the agent stops seeing. pi-fold scores every collapsed fold against what the session is working on right now and hands the best two or three back with the fold id and both affordances, so the agent can peek one ephemerally, expand it in place, or dismiss the whole slate and carry on. The judgment stays with the agent; the mechanism only puts the option where it can be acted on.
+
+The selector is deterministic and model free: no extra provider call, no embeddings, no randomness, no wall clock. A fold scores on the lexical overlap between its brief and the most recent user and assistant text, plus a small recency component read from transcript position and a small bonus for depth (a deeply nested fold is otherwise invisible). Those structural components alone can never clear the threshold, so a suggestion always rests on evidence that the span matches the task in hand, and when nothing clears the threshold, nothing is shown at all. A fold that was suggested and left alone is suppressed for a cooldown; a fold that is currently expanded or protected is never suggested.
+
+Suggestions ride the same ephemeral channel as the advisory milestones: appended at the tail of the window, after the stable prefix, so no cached prefix is invalidated and nothing durable is written to the transcript. They never share an advisory with urgent fence text, where the only useful next action is the fold that keeps the request transmissible.
+
+Every suggestion shown is recorded in durable session state with its source, item id, score and transcript ordinal. The record resolves to `accept` when the agent peeks or expands that item within a short window of turns, and to `reject` otherwise. `active_context {"action":"status"}` returns the live slate and the whole log under `automatic.surfacing`. Set `surfacing: false` to register no sources and emit nothing. Slate size, threshold, cooldown and character budget are internal constants rather than options: they are the knobs the accept and reject record exists to settle.
+
+### Suggestion sources
+
+pi-fold owns the carrier, not the content. Any extension can put its own items into the same slate, with the same ranking, the same budget, and the same accept and reject log. The built-in fold-brief source is registered through that identical interface.
+
+```js
+registerPiFold(pi, {
+  setSuggestionSourceRegistrar: (register) => {
+    const memory = register({
+      id: "my-memory",
+      candidates: ({ state, snapshot, toolName }) => [
+        {
+          id: "note-42",
+          text: "Durable note the current task may want back.",
+          route: "recall {\"address\":\"note-42\"}",
+        },
+      ],
+    });
+    // memory.accepted("note-42") when the agent acts on it; memory.unregister() to withdraw.
+  },
+});
+```
+
+A candidate needs an id, the text that is scored and shown, and a route string naming the action that retrieves it. Optional `position` (a transcript ordinal, where higher is more recent) and `depth` feed the recency and depth components. A source that throws, or an item that is malformed, costs its own items and nothing else: the carrier, the projection, and the ladder are never at risk.
+
+## Relation to prior work
+
+pi-fold combines two recent lines of work in one mechanism. Lossless hierarchical compaction, exemplified by LCM ([arXiv:2605.04050](https://arxiv.org/abs/2605.04050)) with an interactive walkthrough at [losslesscontext.ai](https://www.losslesscontext.ai/), builds a summary DAG over older messages with lossless pointers to every original, but the system alone decides when to compact. Self-GC ([arXiv:2607.00692](https://arxiv.org/abs/2607.00692)) treats context as indexed, recoverable objects with fold, mask, and prune actions, proposed by a side-channel planner under harness enforcement. In pi-fold the session agent itself holds the verbs, in-band: it can fold, expand, refold, protect, and consolidate on its own judgment, and the autonomous ladder is the fallback; a session that never touches the tool degrades gracefully into lossless hierarchical compaction.
+
+Context-Folding ([arXiv:2510.11967](https://arxiv.org/abs/2510.11967), ByteDance Seed) has the agent branch into a sub-trajectory and fold it on completion, with the behavior shaped by reinforcement learning; the fold keeps a summary of the outcome rather than a lossless path back. MemGPT and its successor [Letta](https://github.com/letta-ai/letta) are the canonical agent-held memory-verb systems: the agent edits its own memory blocks and archives while the window pages against external stores. Two things separate pi-fold from that line. First, what is governed: Letta manages traffic between the window and outside storage, and when its message queue overflows the in-window record becomes a lossy recursive summary; pi-fold restructures the transcript in place. Second, recovery: theirs is retrieval, a search that re-ingests whatever comes back as new tokens; ours is expansion, which restores the exact folded entries after SHA-256 verification. Further inspirations include Hindsight and Hermes.
+
+The design also pairs with Pi's session trees: a fold's collapsed-or-expanded state is a natural unit for curating exactly which context a subagent or workflow leg inherits.
+
+## Configuration
+
+The package entry calls `registerPiFold(pi)` with the defaults below. Hosts that call the named `registerPiFold` export directly may pass an options object. Replacement lists and sets replace the defaults; they are not additive.
+
+| Option | Default | Effect |
+| --- | --- | --- |
+| `toolName` | `"active_context"` | Name of the registered context tool. |
+| `toolLabel` | `"Active Context"` | Human-facing label for the registered context tool. |
+| `brandNoun` | `"active-context"` | Short noun used in fold placeholders, advisory headings, fence text, and native-compaction notices. |
+| `entryTypePrefix` | `"pi-fold-active-context"` | Prefix for durable state, fold-record, status, and advisory identifiers. Its namespace also derives measurement/compaction entry types, status source metadata, MCP evidence ownership, the evidence directory, and the unparseable MCP-server fallback. Changing it starts a separate state namespace. |
+| `commandPrefix` | `""` | Prefix for `/context` and `/fold-context`; trailing hyphens are removed. |
+| `commandNames` | `undefined` | Optional full-name overrides for the effective defaults `{ status: "context", fold: "fold-context" }`. Both names must be distinct kebab-case strings; supplied values take precedence over `commandPrefix`. |
+| `summarizer` | `"session"` | Brief generator: the live session model, `"deterministic"` for no model calls, or `{ provider, model, effort? }` resolved through Pi's model registry. Cannot be combined with `summarizeContextSpan`. |
+| `summarizeContextSpan` | `undefined` | Optional async brief generator. Results must include a useful bounded brief, provider/model/effort attribution, and `toolCalls: 0`; failure uses the deterministic brief. |
+| `guidance` | `"pressure"` | Advisory wording profile: `"pressure"`, `"curation"` for task-relevance self-curation plus an early orientation advisory, or `"minimal"` for the urgent advisory alone. Rungs, thresholds, and budgets are the same in every profile. |
+| `setProjectionProvider` | `undefined` | Optional host callback that receives the [projection-candidate provider](#projection-candidate-records). Normal Pi context events do not require it. |
+| `surfacing` | `true` | Score collapsed folds against the recent task context and offer the best few back through the ephemeral tail channel. Set `false` to register no suggestion sources and emit nothing. |
+| `setSuggestionSourceRegistrar` | `undefined` | Optional host callback that receives the [suggestion source](#suggestion-sources) registration function. |
+| `toolActions` | all seven actions | Replacement allowlist drawn from `status`, `peek`, `fold`, `expand`, `refold`, `protect`, and `unprotect`. |
+| `blockingTools` | `["Agent"]` | Replacement list of tool names that trigger one opportunistic stale-tool fold before the call. Use `[]` to disable it. |
+| `readOnlyTools` | `new Set(["read", "grep", "find", "ls"])` | Replacement set of tool names whose completed batches may fold automatically. Defaults to Pi's built-in read-only tools. Pass a `ReadonlySet<string>`. |
+| `isMcpTool` | `() => false` | Synchronous predicate enabling evidence ingestion for oversized structured MCP results. Names shaped as `mcp__<server>__<tool>` project the actual server; other names use the neutral namespace derived from `entryTypePrefix`. |
+| `evidenceIngestion` | `true` | Set to `false` to register no evidence-ingestion hook and write no evidence artifacts. |
+
+### Projection candidate records
+
+The provider emits one record per visible collapsed fold. The fields and current values are:
+
+| Field | Emitted value |
+| --- | --- |
+| `version` | `1` |
+| `key` | `"projection:<fold-id>"` |
+| `kind` | `"projection"` |
+| `domain` | `"system"` |
+| `horizon` | `"working"` |
+| `source_id` | Fold ID. |
+| `source_version` | Exact source SHA-256. |
+| `route` | `{ tool: <toolName>, arguments: { action: "expand", id: <fold-id> } }` |
+| `token_cost` | Estimated replacement tokens, rounded up with a minimum of 1. |
+| `expansion_cost` | Estimated exact-source tokens, rounded up with a minimum of 1. |
+| `rank` | Zero-based position among emitted candidates. |
+| `score` | Saved-byte fraction, capped at 1. |
+| `raw_score` | Exact bytes saved by the fold. |
+| `confidence` | `"exact"` |
+| `freshness` | `"current"` |
+| `locked_owner` | `false` |
+| `collapse_key` | `"projection:<fold-id>"` |
+| `generator` | `"projection-model"`, `"projection-deterministic"`, or `"projection-supplied"` from brief provenance. |
+| `generator_version` | `"memory-slate-generators-v2"` |
+| `recency` | `null` |
+
+## Development
 
-First release: days away, at [shaneconner/pi-fold](https://github.com/shaneconner/pi-fold),
-under an open-source license.
+```sh
+npm ci
+npm run lint
+npm test
+```