@tduyng/pi-context 0.0.1 → 0.0.3

package/README.md

@@ -1,112 +1,82 @@
 # @tduyng/pi-context
 
-See what fills the context window, summarize it from the warm prompt cache, retire a finished task, and recover what compaction dropped.
+See what fills the window, compact from the warm cache instead of a cold one, retire a finished turn, and read back what compaction dropped.
 
-## Commands and tools
+## What it costs
 
-### /context
+`history` is a deferred tool at 134 tokens in the sessions that load it and nothing in the rest. `/context`, `/collapse`, and `/context-cache` are commands, so they cost nothing until typed. The dedupe handler is an event hook and adds no schema.
 
-Opens a TUI panel with a colored pixel map and a dotted-leader category list. Every context consumer gets a row: system prompt, tools, user messages, assistant text, tool results, bash executions, compacted history, extension entries, free space. Categories break down by tool or custom type, sorted by token weight. Child items within a category are capped at 30 shown. The rest are summarized as a "+N more" line with their combined token count.
+Nothing here rewrites context between compactions. The panel, `history`, and the dedupe add no request bytes, and collapse and handoff act where a compaction boundary has already rewritten the prefix, so their work is paid for once.
 
-The panel fits the window it opens in. Its body takes at most 70% of the terminal rows, each line is truncated to the column width instead of wrapping, and a list too tall for the window keeps every category it can before spending the rest on child rows. A window too short for all categories says how many it hid. The map is hidden when the window is shorter than the map's six rows or narrower than 86 columns, so a small window keeps the numbers instead of a map no one can read. The pixel map uses proportional colored squares across 24 columns.
+## /context
 
-### Cached compaction
+A TUI panel with a colored pixel map and a category list. Every consumer of the window gets a row: system prompt, tools, user messages, assistant text, tool results, bash executions, compacted history, extension entries, free space. Categories break down by tool or custom type, sorted by token weight, and children are capped at 30 shown with a `+N more` line carrying their combined count.
 
-Compaction replays the conversation the provider already cached and appends one instruction, so the summary request reads almost all of its input from cache at a fraction of the price, and the summarizer sees the full tool output instead of pi's 2,000-character head cuts.
+The panel fits the window it opens in. Its body takes at most 70% of the rows, each line is truncated to the column width instead of wrapping, and a list too tall for the window keeps every category it can before spending the rest on child rows. A window too short says how many categories it hid. The map is hidden below six rows or 86 columns, so a small window keeps the numbers rather than a map nobody can read.
 
-The fallback. The branch is projected through pi's own entry conversion and the captured conversation must still be its prefix. A kept region that moved, a model change, a cold cache, a truncated reply or a provider error all fall back to pi's own compaction. Nothing is cancelled: a compaction pi asked for always runs, on the cached path when it can and on pi's path otherwise.
+## Cached compaction
 
-The capture is the full transcript of the last request, including the leading system message that carries the prompt and the tool declarations, and the summary request replays it unchanged. That is what keeps the prefix cacheable, and the request adds no second system prompt. `/context-cache` shows the state now: the declared lifetime, the last refresh and the warm capture. Idle cache warming is pi's own job (`cacheWarming` in settings), so this extension starts no compaction of its own.
+Compaction replays the conversation the provider already cached and appends one instruction, so the summary request reads almost all of its input from cache at a fraction of the price, and the summarizer sees full tool output rather than pi's 2,000-character head cuts.
 
-Every decision lands in `~/.pi/agent/cached-compaction.jsonl` with the reason it took.
+The fallback is narrow on purpose. The branch is projected through pi's own entry conversion and the captured conversation must still be its prefix. A kept region that moved, a model change, a cold cache, a truncated reply, or a provider error all fall back to pi's own compaction. Nothing is cancelled: a compaction pi asked for always runs, on the cached path when it can and on pi's path otherwise.
 
-### history
+The capture is the full transcript of the last request, including the leading system message that carries the prompt and the tool declarations, and the summary request replays it unchanged. That is what keeps the prefix cacheable, and the request adds no second system prompt. `/context-cache` shows the declared lifetime, the last refresh, and the warm capture. Idle cache warming is pi's own job, so this extension starts no compaction of its own.
 
-Search or read earlier conversation that left the context window: pruned results, compacted spans, or earlier sessions of the project.
+Every decision lands in `~/.pi/agent/cached-compaction.jsonl` with the reason it took.
 
-`search` walks the current branch newest-first. With `all: true`, it searches every `.jsonl` session file of the project, newest file first. Multi-word queries OR-match. Echoes (compaction summaries, verbatim input records, prior history results) rank below originals, so a copy never displaces what it copied. Default limit is 10, max 50. The ranking is lexical: a model rerank of the top eight was deleted, see the removal note below.
+## history
 
-`read` returns one entry in full, sliced to fit the remaining context. Page size: min 2,000, max 20,000 chars, sized to leave room for the model's reply. Pass `offset` for pagination.
+Search or read earlier conversation that left the window: pruned results, compacted spans, or earlier sessions of the project.
 
-### Compaction input record
+`search` walks the current branch newest first. With `all: true` it searches every session file of the project, newest file first. Multi-word queries OR-match. Echoes (compaction summaries, verbatim input records, prior history results) rank below originals, so a copy never displaces what it copied. The default limit is 10, the maximum 50, and the ranking is lexical.
 
-After automatic compaction (not manual `/compact`), the extension appends a hidden custom message listing the compacted span's verbatim user inputs, `ask_question` answers, and visible custom messages. Total cap: 20,000 chars. The first ask, the latest ask, and the latest input always survive. The rest fill newest-first. Anything omitted is recoverable with the `history` tool.
+`read` returns one entry in full, sliced to fit the remaining context. Page size is min 2,000, max 20,000 characters, sized to leave room for the reply. `offset` pages through.
 
+Session files are append-only JSONL, streamed line by line, so a large transcript never lands in memory.
 
-### /collapse
+## /collapse
 
-`/collapse [turns]` replaces finished turns with one branch summary. The default is one turn. The anchor is the entry before the oldest user prompt in the span, so nothing before the task moves.
+`/collapse [turns]` replaces finished turns with one branch summary, one turn by default. The anchor is the entry before the oldest user prompt in the span, so nothing before the task moves.
 
-The summary is built from the entries themselves. One walk collects the modified files, the files that were only read, the command count, the validation runs, and the failures with the command that failed. The last assistant text becomes the outcome. No model reads the span and no summary request goes out, which is where this differs from a compaction. This one costs nothing and it leaves the cache alone.
+The summary is built from the entries themselves. One walk collects the modified files, the files that were only read, the command count, the validation runs, and the failures with the command that failed. The last assistant text becomes the outcome. No model reads the span and no summary request goes out, which is where this differs from a compaction: it costs nothing and leaves the cache alone.
 
-Two cases refuse. Nothing sits before the first prompt of a session, so there is no entry to anchor to, and a prompt with no assistant turn after it is work in flight rather than a finished turn.
+Two cases refuse. Nothing sits before the first prompt of a session, so there is no anchor, and a prompt with no assistant turn after it is work in flight rather than a finished turn.
 
-What it drops is real. The reasoning inside a turn goes, the summary keeps one text block, and a file list is capped at twelve with the remainder counted. The raw entries stay in the session file, so `history` recovers any of them and `/tree` walks back to them. The collapsed messages' usage rides along on the branch summary entry, which is where pi reads totals from, so `/cost` still counts what those turns cost.
+What it drops is real. The reasoning inside a turn goes, the summary keeps one text block, and a file list is capped at twelve with the remainder counted. The raw entries stay in the session file, so `history` recovers any of them and `/tree` walks back to them. The collapsed messages' usage rides along on the branch summary entry, which is where pi reads totals, so `/cost` still counts what those turns cost.
 
 A hidden `context-collapse` message carries the summary into the next turn and tells the model not to go looking for the raw turns. It starts no turn of its own.
 
 Its reach is not measured yet. What decides whether it is worth running is how much front a finished task leaves behind, and `just retention` prints that for the read side of the same spans.
 
-## Reads a later edit made wrong
-
-A `read` puts a file's bytes in the transcript. When an `edit` or `write` then lands on that path, those bytes are stale, and nothing in the transcript says so. Over the recorded sessions the frozen benchmark fixture holds 459 read results, of which 359 are marked by this rule, 66 percent of read bytes, 9.5 percent of all tool-result bytes.
-
-Marking them in place costs more than it saves. Replacing a tool result in the `context` event rewrites the cached prefix from that point on, so the next request re-bills everything after it. On a 300k-token session that is one 200k to 300k re-bill against the 8KB the marker reclaims, and the rewrite recurs every time a later edit lands on a path already read. The rewrite was removed; `pi-context` now changes nothing the provider has already cached, and the spending on stale reads is left to compaction, which rewrites the whole prefix once and pays for it once.
-
-The session file keeps the original bytes, so `history` still recovers them.
-
-### Project context dedupe
+## Compaction input record
 
-When the agent dir and the working directory resolve to the same file through different paths (hardlinks, bind mounts, a symlinked ancestor), pi adds the same `<project_instructions>` block twice. A `before_agent_start` handler strips the second copy, matching on content, so two identical blocks are always redundant regardless of path. Blocks with different content stay.
+After automatic compaction, not manual `/compact`, the extension appends a hidden custom message listing the compacted span's verbatim user inputs, `ask_question` answers, and visible custom messages. The cap is 20,000 characters, and the first ask, the latest ask, and the latest input always survive. The rest fill newest first, and anything omitted is recoverable with `history`.
 
-## Tool-result views
-
-Not wired, and the reason is worth keeping because it closes the question rather than deferring it.
-
-A large tool result is rarely needed whole, so the idea was to build the small views of one: `full`, `head`, `tail`, `signals` (the error and failure lines) and `outline` (headings, definitions and `path:line` references), each built from the result's own lines with no model, and to ask the local model which one to keep.
-
-It used to run, on a `tool_execution_end` handler over every result past 2,000 characters. It cost 839 calls and 740,402 tokens, 43 percent of everything this suite sent to the local model, and it changed nothing: the handler returned no hint, nothing was rewritten, and the log entry held a `stateHash` rather than the result, so nothing could be scored from it afterwards. Asking the model directly does not work either, and returns near-uniform distributions over the five views.
-
-The offline fallback scored each view against the lines a later message actually quoted from the result. That label cannot fire. Across 1,045 replayed cases, not one tool-result line was ever quoted verbatim, at every bar from 4 to 80 characters. The model reads a result and acts on it rather than repeating it, so there is no signal in these sessions to score a view against, and no comparison can say which view is safe.
+## Reads a later edit made wrong
 
-The views and the benchmark are deleted. This section is what is left of them, so the next person does not build it again.
+A `read` puts a file's bytes in the transcript. When an `edit` or `write` then lands on that path, those bytes are stale, and nothing in the transcript says so. Over the recorded sessions the frozen benchmark fixture holds 459 read results, of which 359 are marked by this rule: 66 percent of read bytes, 9.5 percent of all tool-result bytes.
 
-## What actually reclaims context, measured
+Marking them in place costs more than it saves. Replacing a tool result in the `context` event rewrites the cached prefix from that point on, so the next request re-bills everything after it. On a 300k-token session that is one 200k to 300k re-bill against the 8 KB the marker reclaims, and the rewrite recurs every time a later edit lands on a path already read. The rewrite was removed: `pi-context` now changes nothing the provider has already cached, and the spending on stale reads is left to compaction, which rewrites the whole prefix once and pays for it once.
 
-Over 358 recorded sessions, pi's own compaction does the work. One session is enough to show
-the ratio: its context climbed to 48 percent, dropped to 7, climbed to 53, and dropped again.
-Both drops are pi's own compaction, `fromHook: False`, and the second reclaimed about 458,000
-tokens in the event.
+The session file keeps the original bytes, so `history` still recovers them.
 
-Nothing in this package rewrites context between compactions. The panel, `history` and dedupe
-add no request bytes, and collapse and handoff act where a compaction boundary has already
-rewritten the prefix, so their work is paid for once.
+## Project context dedupe
 
-Two local model gates are gone from this package, both because nothing measured them.
+When the agent dir and the working directory resolve to the same file through different paths, from hardlinks, bind mounts, or a symlinked ancestor, pi adds the same `<project_instructions>` block twice. A `before_agent_start` handler strips the second copy by content, so two identical blocks are always redundant regardless of path, and blocks with different content stay.
 
-The compaction deferral asked whether summarizing the older messages would lose something the
-work in flight still needs, and it had one live record in the whole history. Pi opens a
-threshold compaction at `contextWindow - reserveTokens`, a line at 1,032,192 tokens for this
-1,048,576-token model, against a session request that carries about 217,000, so the hook
-cannot fire at the mean context. Its label would be the compaction that happened anyway, and
-with no deferral there is no negative class, so no cut could be fitted. Three hand-written
-probes said the answer looked right, and three probes are not a corpus.
+## What reclaims context, measured
 
-The history rerank asked which of the top eight search hits answers the question. Over 17
-searches it promoted twice, and the same shape on file selection read 0.612 to 0.659 AUC
-against a lexical floor at 0.707, so the ranking it would have reordered was already the
-better one. Both are deleted rather than left waiting for a corpus that has not arrived.
+Over 358 recorded sessions, pi's own compaction does the work. One session is enough to show the ratio: its context climbed to 48 percent, dropped to 7, climbed to 53, and dropped again. Both drops are pi's own compaction, `fromHook: False`, and the second reclaimed about 458,000 tokens.
 
-## Compaction timing
+Two local model gates are gone from this package, both because nothing measured them. The compaction deferral had one live record in the whole history: pi opens a threshold compaction at `contextWindow - reserveTokens`, a line at 1,032,192 tokens for a 1,048,576-token model, against a session request carrying about 217,000, so the hook could not fire at the mean context. The history rerank promoted twice in 17 searches, and the same shape on file selection read 0.612 to 0.659 AUC against a lexical floor at 0.707, so the ranking it would have reordered was already the better one.
 
-Pi compacts at `contextWindow - compaction.reserveTokens`. Fixed line. It summarizes whole spans whether or not the work in flight is finished, and a half-applied edit survives that summary by luck.
+### Tool-result views, and why they are not here
 
-The deferral that used to sit here asked the model at that moment whether the summary would lose something the remaining work still needs, and let the space be reclaimed in place instead. It is gone, for the reasons above. What is left is pi's own timing, and cached compaction, which pays for a rewrite once instead of once per edit.
+This one is kept because it closes the question rather than deferring it.
 
-## Design
+A large tool result is rarely needed whole, so the idea was to build small views of one (`full`, `head`, `tail`, `signals`, `outline`) and ask the local model which to keep. It ran on a `tool_execution_end` handler over every result past 2,000 characters, cost 839 calls and 740,402 tokens, 43 percent of everything this suite sent to the local model, and changed nothing: the handler returned no hint, nothing was rewritten, and the log entry held a `stateHash` rather than the result, so nothing could be scored afterwards. Asking the model directly returns near-uniform distributions over the five views.
 
-- `history` registers and unregisters dynamically, so the tool appears only when there is an earlier transcript to search.
-- Session files are JSONL, append-only. `history` streams them line by line so large transcripts never land in memory.
+The offline fallback scored each view against the lines a later message quoted from the result, and that label cannot fire: across 1,045 replayed cases, not one tool-result line was ever quoted verbatim, at every bar from 4 to 80 characters. The model reads a result and acts on it rather than repeating it, so these sessions hold no signal to score a view against. The views and the benchmark are deleted.
 
 ## Install
 
@@ -116,6 +86,4 @@ pi install npm:@tduyng/pi-context
 
 Local checkout: `pi install /path/to/dotfiles/pi/agent/extensions/pi-context`
 
-## Configure
-
-None.
+No configuration.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@tduyng/pi-context",
-  "version": "0.0.1",
-  "description": "Context-window suite: see what fills it, prune spent tool results, retire reads an edit made wrong, and recover what compaction dropped.",
+  "version": "0.0.3",
+  "description": "See what fills the context window, compact from the warm prompt cache, collapse a finished turn, and recover what compaction dropped.",
   "keywords": [
     "pi",
     "pi-package",

package/src/cache-state.ts

@@ -54,6 +54,7 @@ function refreshAt(entry: SessionEntry): number | null {
     }
     if (entry.type !== 'message') return null
     if (entry.message.role !== 'assistant') return null
+    if (entry.message.stopReason === 'error' || entry.message.stopReason === 'aborted') return null
     const usage = entry.message.usage
     const read = usage.cacheRead ?? 0
     const write = usage.cacheWrite ?? 0

package/src/cached-compaction.ts

@@ -14,14 +14,23 @@ import {
 import type { Usage } from '@earendil-works/pi-ai'
 
 import { cacheState, formatExpiry, resolveRetention } from './cache-state.js'
-import { assistantText, forget, keptStartQuote, summaryInstruction, walkTrailing, type CapturedRequest, type CaptureStore, type TranscriptMessage } from './cached-request.js'
+import {
+    assistantText,
+    forget,
+    keptStartQuote,
+    summaryInstruction,
+    walkTrailing,
+    type CapturedRequest,
+    type CaptureStore,
+    type TranscriptMessage
+} from './cached-request.js'
 import { checkpointSummary } from './summary.js'
 
 const MAX_TOKENS_SHARE = 0.8
 
 type Preparation = SessionBeforeCompactEvent['preparation']
 
-type Decision = { kind: 'cached'; summary: string; tokens: number } | { kind: 'fallback'; reason: string }
+type Decision = { kind: 'cached'; summary: string; tokens: number; usage: Usage } | { kind: 'fallback'; reason: string }
 
 function traceFile(): string {
     return join(getAgentDir(), 'cached-compaction.jsonl')
@@ -51,11 +60,15 @@ async function cachedSummary(
     const keptStart = keptStartQuote(event.branchEntries, event.preparation.firstKeptEntryId)
     const instruction = summaryInstruction(keptStart, event.customInstructions)
     const transcript = convertToLlm([...captured.transcript, ...trailing, instruction])
-    const stream = ctx.modelRegistry.streamSimple(captured.model, { messages: transcript }, {
-        maxTokens: Math.min(Math.floor(event.preparation.settings.reserveTokens * MAX_TOKENS_SHARE), 32_000),
-        sessionId: ctx.sessionManager.getSessionId(),
-        cacheRetention: resolveRetention(process.env)
-    })
+    const stream = ctx.modelRegistry.streamSimple(
+        captured.model,
+        { messages: transcript },
+        {
+            maxTokens: Math.min(Math.floor(event.preparation.settings.reserveTokens * MAX_TOKENS_SHARE), 32_000),
+            sessionId: ctx.sessionManager.getSessionId(),
+            cacheRetention: resolveRetention(process.env)
+        }
+    )
     for await (const _ of stream) {
         // drain: the final assistant message carries text, usage and the stop reason
     }
@@ -65,7 +78,7 @@ async function cachedSummary(
     }
     const summary = assistantText(assistant)
     if (summary.trim() === '') return { kind: 'fallback', reason: 'the summary came back empty' }
-    return { kind: 'cached', summary, tokens: assistant.usage.input }
+    return { kind: 'cached', summary, tokens: assistant.usage.input, usage: assistant.usage }
 }
 
 function decideFallback(reason: string): SessionBeforeCompactResult {
@@ -94,7 +107,6 @@ let pendingTranscript: TranscriptMessage[] = []
 
 export default function cachedCompactionExtension(pi: ExtensionAPI): void {
     const store: CaptureStore = { captured: null }
-    let retentionApplied = false
 
     pi.on('context_with_system', (event) => {
         pendingTranscript = event.messages
@@ -109,10 +121,6 @@ export default function cachedCompactionExtension(pi: ExtensionAPI): void {
                 at: Date.now()
             }
         }
-        if (!retentionApplied && ctx.hasUI && process.env['PI_CACHE_RETENTION'] === undefined) {
-            retentionApplied = true
-            process.env['PI_CACHE_RETENTION'] = 'long'
-        }
     })
 
     pi.on('model_select', () => {
@@ -140,13 +148,9 @@ export default function cachedCompactionExtension(pi: ExtensionAPI): void {
             const decision = await cachedSummary(captured, event, ctx)
             if (decision.kind === 'fallback') return decideFallback(decision.reason)
             const previous = getLatestCompactionEntry(event.branchEntries)
-            const prepared = checkpointSummary(
-                decision.summary,
-                event.preparation.fileOps,
-                previous?.details
-            )
+            const prepared = checkpointSummary(decision.summary, event.preparation.fileOps, previous?.details)
             trace({ decision: 'cached', tokens: decision.tokens })
-            return runSummary(event.preparation, prepared.summary, prepared.details, undefined)
+            return runSummary(event.preparation, prepared.summary, prepared.details, decision.usage)
         } catch (error) {
             const reason = error instanceof Error ? error.message : String(error)
             return decideFallback(reason)

package/src/cached-request.ts

@@ -81,15 +81,13 @@ export function walkTrailing(captured: CapturedRequest, branchEntries: SessionEn
 
 export interface KeptStart {
     text: string
-    repeats: number
 }
 
 const QUOTE_CHARS = 120
 
 /**
  * The opening of the first message pi keeps verbatim, quoted back to the summarizer. The
- * quote marks where the kept window begins. A repeated opening is counted from the end,
- * so the quote names the last of them.
+ * quote marks where the kept window begins.
  */
 export function keptStartQuote(branchEntries: SessionEntry[], firstKeptEntryId: string): KeptStart | null {
     const start = branchEntries.findIndex((entry) => entry.id === firstKeptEntryId)
@@ -100,12 +98,7 @@ export function keptStartQuote(branchEntries: SessionEntry[], firstKeptEntryId:
         .filter((text) => text !== '')
     const first = quotes[0]
     if (first === undefined) return null
-    const text = first.slice(0, QUOTE_CHARS)
-    let repeats = 0
-    for (const quote of quotes) {
-        if (quote.startsWith(text)) repeats++
-    }
-    return { text, repeats }
+    return { text: first.slice(0, QUOTE_CHARS) }
 }
 
 /**
@@ -118,7 +111,6 @@ export function summaryInstruction(keptStart: KeptStart | null, customFocus: str
         keptStart === null
             ? 'Summarize the conversation so far.'
             : `Summarize the conversation only up to the message beginning "${keptStart.text}".\n` +
-              'A message with that opening may appear more than once: the boundary is the last of them.\n' +
               'Do not include anything after the boundary.'
     const message: BranchMessage = {
         role: 'user',