pi-smart-compact 7.7.0 → 7.9.0

package/README.md

@@ -1,248 +1,416 @@
-# Smart Compact
+# pi-smart-compact
 
 [![npm version](https://img.shields.io/npm/v/pi-smart-compact.svg)](https://www.npmjs.com/package/pi-smart-compact)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
 [![GitHub](https://img.shields.io/badge/GitHub-alpertarhan%2Fpi--smart--compact-blue)](https://github.com/alpertarhan/pi-smart-compact)
 
-> Intelligent, verification-oriented conversation compaction for the [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent).
+<p align="center">
+  <img src="./docs/assets/pi-smart-compact.png" alt="pi-smart-compact" width="760" />
+</p>
 
-**Smart Compact** is a Pi extension that compresses long coding sessions by preserving the *working state* of the conversation - not just the words. Instead of blindly truncating old messages, it extracts verified facts, explores ambiguous areas when needed, synthesizes a structured summary, and checks that the result still covers the important parts of the session.
+> Verification-oriented smart compaction for the [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent).
 
-In practice, that means your agent keeps the things that actually matter:
+`pi-smart-compact` replaces blind conversation trimming with a structured compaction pipeline that tries to preserve the agent's working state: goal, files, errors, decisions, constraints, open loops, and cross-compaction deltas.
 
-- the real goal
-- exact file paths
-- unresolved errors
-- decisions already made
-- constraints and preferences
-- follow-up work still pending
-- **open loops** — unresolved tasks that survive compaction
-- **delta since last compaction** — what changed, what resolved, what's new
-- **structured state** — machine-readable JSON alongside the Markdown summary
+It is built around an **EESV** pipeline:
+
+**Extract → Explore → Synthesize → Verify**
 
 ---
 
 ## Table of Contents
 
-- [Why this exists](#why-this-exists)
-- [What makes it different](#what-makes-it-different)
-- [Design philosophy](#design-philosophy)
-- [Inspiration](#inspiration)
-- [How it works](#how-it-works)
-- [Key capabilities](#key-capabilities)
+- [What this project is](#what-this-project-is)
+- [Current repository snapshot](#current-repository-snapshot)
+- [Visual identity](#visual-identity)
+- [Why it exists](#why-it-exists)
+- [Architecture flow](#architecture-flow)
+- [Execution model](#execution-model)
+- [Repository layout](#repository-layout)
+- [Runtime artifacts](#runtime-artifacts)
 - [Installation](#installation)
-- [Quick start](#quick-start)
+- [Usage](#usage)
 - [Configuration](#configuration)
-- [Output format](#output-format)
-- [Architecture details](#architecture-details)
-- [Quality and safety controls](#quality-and-safety-controls)
-- [Compatibility](#compatibility)
+- [Output contract](#output-contract)
+- [Quality controls](#quality-controls)
+- [Current caveats](#current-caveats)
 - [Development](#development)
-- [Limitations](#limitations)
-- [Contributing](#contributing)
 - [License](#license)
 
 ---
 
-## Why this exists
+## What this project is
 
-Large coding sessions have a very specific failure mode: the context window fills up, compaction happens, and the agent loses the operational memory required to continue well.
+This package is a **Pi extension** that registers three integration surfaces:
 
-Typical summaries often miss at least one of these:
+| Surface | Where | Purpose |
+| --- | --- | --- |
+| Slash command | `/smart-compact` | Manual compaction, interactive or direct |
+| Session hook | `session_before_compact` | Auto-trigger smart compaction before Pi's default compaction |
+| Tool | `smart_compact` | Agent-callable compaction that stages a pending summary |
 
-- which files were actually changed
-- whether an error was resolved or only retried
-- what the user explicitly asked for
-- which architectural decision already won
-- what still needs to happen next
+The extension keeps a short-lived pending compaction in memory, then hands that summary back to Pi when compaction is applied.
 
-For a coding agent, these omissions are expensive. They lead to redundant reads, repeated questions, contradictory edits, or unfinished follow-up work.
+---
 
-**Smart Compact** is built to reduce those failures.
+## Current repository snapshot
 
----
+**Observed from the current codebase (`README`, `src/`, `test/`, `package.json`)**
 
-## What makes it different
+- **Package version:** `7.8.0`
+- **Runtime entrypoint:** `dist/index.js`
+- **Source entrypoint:** `src/index.ts`
+- **Source modules:** 16 TypeScript files under `src/`
+- **Tests:** 9 test files, **93 passing tests**
+- **Approx repo footprint:** ~5,053 lines across `src/` + `test/`
+- **Documentation asset:** `docs/assets/pi-smart-compact.png`
+- **Published package files:** `dist/`, `docs/`, `README.md`, `LICENSE`, `CHANGELOG.md`
 
-Smart Compact is not just "another summary prompt." It is a **multi-stage compaction pipeline** with deterministic extraction, targeted exploration, structured synthesis, and verification.
+### Current command health
 
-That design gives it a few practical advantages over plain truncation or one-shot summarization:
+| Command | Status | Notes |
+| --- | --- | --- |
+| `bun test` | ✅ Pass | 93/93 tests passing |
+| `bun run build` | ✅ Pass | Bundles `src/index.ts` to `dist/index.js` |
+| `bun run typecheck` | ⚠️ Fails | Current repo has TypeScript compatibility issues against the installed Pi typings / `.ts` import style |
 
-- **Deterministic first, LLM second** - verified facts are extracted before any model call
-- **Cheaper when possible** - simple sessions skip exploration entirely
-- **Safer by default** - summaries are checked for missing files, errors, constraints, and fabricated paths
-- **More agent-friendly** - output is structured around goal, progress, decisions, files, next steps, and critical context
-- **Better continuity** - follow-up work survives compaction more reliably
+So the project is currently **buildable and tested**, but **not fully typecheck-clean**.
 
 ---
 
-## Design philosophy
+## Visual identity
 
-Smart Compact is built around a few core principles:
+The repository image used by this `README.md` now lives at:
 
-### 1. Accuracy over style
-A beautiful summary that invents a file path is worse than a plain summary that is correct.
+- `docs/assets/pi-smart-compact.png`
 
-### 2. Determinism before generation
-Anything we can extract mechanically from the conversation should not be guessed by an LLM.
+The asset is now a cleaned transparent `PNG` instead of a checkerboard/transparent-preview render, so it displays correctly inside GitHub `README.md`.
 
-### 3. Preserve working state, not transcript fidelity
-The goal is not to recreate the whole conversation. The goal is to preserve the information needed to continue the work correctly.
+This keeps documentation assets separate from `src/` implementation code and `test/` fixtures while still shipping the image with the package via `package.json` `files`.
 
-### 4. Spend tokens where they matter
-Easy sessions should stay cheap. Complex sessions should get deeper exploration and better synthesis.
+---
 
-### 5. Optimize for real coding sessions
-This extension is designed for implementation, debugging, review, and discussion workflows inside Pi - not for generic meeting notes.
+## Why it exists
+
+Default compaction usually loses exactly the things a coding agent needs most:
+
+- which files were actually modified
+- which errors are still unresolved
+- what the user explicitly asked for
+- what decisions already won
+- what still needs to happen next
+
+`pi-smart-compact` tries to preserve that operational state instead of producing a generic prose summary.
 
 ---
 
-## Inspiration
+## Architecture flow
+
+```mermaid
+flowchart TD
+    A["Entry points<br/>/smart-compact<br/>session_before_compact<br/>smart_compact tool"] --> B["Load config and resolve models<br/>loadConfig()<br/>summaryModel / segmentationModel"]
+    B --> C{"Context >= 5000 tokens?"}
+    C -- No --> C1["Exit early"]
+    C -- Yes --> D["Read current branch<br/>keep recent tail by profile<br/>smartKeepBoundary()"]
+    D --> E["Pre-processing<br/>pruneRedundant()<br/>serializeConversation()<br/>backupConversation()<br/>getPreviousCompactionContext()"]
+    E --> F["Phase 1: Extract<br/>extractStructured()<br/>files, errors, decisions, constraints,<br/>heuristic topics, timeline, goal"]
+    F --> G["Cross-session helpers<br/>load or save cached extraction<br/>deriveProjectId()<br/>loadProjectFingerprint()"]
+    G --> H{"Compacted text below singlePassMaxTokens?"}
+    H -- Yes --> I["Single-pass path<br/>singlePassCompact()"]
+    H -- No --> J{"shouldExplore()?"}
+    J -- Yes --> K["Phase 2: Explore<br/>exploreConversation()<br/>tool probe -> tool loop<br/>or direct JSON fallback"]
+    J -- No --> L["Skip exploration<br/>use heuristic boundaries only"]
+    K --> M["Merge LLM boundaries<br/>with heuristic boundaries"]
+    L --> N["chunkLlmMessages()"]
+    M --> N
+    N --> O["createBatches()<br/>provider-aware concurrency"]
+    O --> P["Phase 3: Synthesize<br/>summarizeBatch() per batch<br/>assembleLLM() or assembleFallback()"]
+    I --> Q["Phase 4: Verify<br/>verifySummary()"]
+    P --> Q
+    Q --> R{"Quality score below 85?"}
+    R -- No --> S["Accept summary"]
+    R -- Yes --> T["patchDeterministic()"]
+    T --> U{"Still below 75?"}
+    U -- No --> S
+    U -- Yes --> V["patchSummary() via LLM"]
+    V --> S
+    S --> W["Post-processing<br/>extractOpenLoops()<br/>buildCompactionState()<br/>computeDelta()<br/>inject sections"]
+    W --> X["Persist artifacts<br/>project fingerprint<br/>compaction state<br/>metrics log<br/>damage report"]
+    X --> Y["Store pending compaction<br/>in memory<br/>TTL: 5 minutes"]
+    Y --> Z["ctx.compact() now<br/>or next tree compact consumes it"]
+```
 
-This project is informed by the broader **context engineering** and **agentic context management** space.
+---
 
-In particular, the design is influenced by:
+## Execution model
 
-- long-context summarization patterns used in agent systems
-- deterministic-plus-LLM hybrid pipelines
-- structured memory preservation for coding workflows
-- ideas popularized in the ecosystem by people such as **Greg Kamradt** around context quality, retrieval discipline, and practical LLM memory design
+### 1. Entry and model resolution
 
-Smart Compact is **not** a copy of any single project. It is a Pi-native implementation focused specifically on coding-agent conversations, with strong emphasis on verified facts, exact code references, and safe continuation after compaction.
+`src/index.ts` is the extension boundary.
 
----
+It does four jobs:
 
-## How it works
+1. registers `/smart-compact`
+2. registers `session_before_compact`
+3. registers `smart_compact`
+4. keeps a shared in-memory `pendingRef` and `isRunning` lock
 
-Smart Compact uses an **EESV** pipeline:
+Model resolution order is effectively:
 
-```text
-Extract → Explore → Synthesize → Verify
-```
+- explicit command model argument, if provided
+- configured `summaryModel`, if resolvable
+- current session model
+- first available model in the registry
 
-| Phase | Purpose | Typical LLM cost |
-| --- | --- | --- |
-| **Extract** | Deterministically mine files, errors, decisions, constraints, and topic boundaries | **0 calls** |
-| **Explore** | Investigate ambiguous areas with tools and improve topic understanding | 0-8 calls |
-| **Synthesize** | Build batch summaries and merge them into one structured compaction summary | N+1 calls |
-| **Verify** | Check coverage, detect hallucinations, patch missing facts deterministically first | 0-1 calls |
+`segmentationModel` falls back to the summary model unless explicitly configured.
 
-### Before EESV
+### 2. Context gate and keep window
 
-Smart Compact first performs lightweight preprocessing:
+`src/core.ts` is the pipeline orchestrator.
 
-- **redundancy pruning**
-- **project fingerprint loading**
-- **incremental extraction cache lookup**
+Before compaction starts it:
 
-### After EESV
+- checks `MIN_TOKEN_THRESHOLD = 5000`
+- reads the current session branch
+- keeps the most recent tail according to the selected profile
+- nudges the keep boundary with `smartKeepBoundary()` if adjacent messages appear to reference the same file
 
-It can also record quality signals for future analysis:
+### 3. Pre-processing
 
-- re-reads after compaction
-- user complaints
-- weak continuity indicators
+Before any summarization, the pipeline performs:
 
----
+- **redundancy pruning** via `src/utils/pruning.ts`
+- **conversation backup** via `backupConversation()`
+- **previous compaction context injection** via `getPreviousCompactionContext()`
+- **incremental extraction cache lookup** via `src/utils/cache.ts`
+- **project fingerprint lookup** via `src/utils/fingerprint.ts`
 
-## Key capabilities
+### 4. Phase 1 — Extract
 
-### Deterministic extraction
-Before asking any model to summarize anything, Smart Compact extracts:
+`src/utils/extraction.ts` is the deterministic core.
 
-- modified, read, and deleted files
-- error chains and retry attempts
-- explicit and implicit decisions
-- user constraints and preferences
+It extracts, with zero LLM calls:
+
+- modified files
+- read files
+- deleted files
+- tool and bash-like errors
+- retry / resolution signals
+- explicit decisions from `ask_user`
+- implicit user choices
+- English and Turkish constraints
 - heuristic topic segments
-- main goal and recent user signals
+- timeline events
+- main goal
+- recent user messages
+- recent error snippets
+
+It also builds **open loops** from:
 
-### Adaptive exploration
-Not every session needs expensive model-driven exploration.
+- unresolved errors
+- follow-up language
+- blocked/waiting language
+- retried-but-unresolved failures
 
-Simple sessions can skip Phase 2 entirely when they have:
+### 5. Phase 2 — Explore
 
-- few topics
-- few unresolved errors
-- few decisions
-- limited cross-directory work
+`src/phases/explore.ts` adds targeted LLM exploration only when complexity justifies it.
 
-### Open Loops detection
-Every compaction identifies unresolved work and tracks it as **open loops**:
+Exploration is skipped for simple sessions when the extraction stays below these heuristics:
 
-- **bugfix** — unresolved errors from tool calls
-- **follow-up** — user mentions of pending next steps
-- **blocked** — dependencies waiting on external input
-- **retry** — retried but still-unresolved failures
+- `<= 3` topics
+- `<= 1` unresolved error
+- `<= 2` decisions
+- `<= 2` directory groups touched
 
-Each loop gets a stable ID, priority, and file references. They appear in both the Markdown summary and the structured JSON state.
+If exploration runs, it can use these tools:
 
-### Cross-compaction tracking
-Smart Compact persists structured state between compactions. On the next compaction, it loads the previous state and computes a **delta**:
+- `get_message_range`
+- `search_conversation`
+- `get_recent_user_messages`
+- `get_context_around`
+- `get_file_changes`
+- `get_error_chain`
 
-- which open loops were resolved
-- which are still persistent
-- which decisions carried over
-- which errors were fixed vs newly introduced
-- which files are newly modified
-- whether the goal shifted
+If the provider cannot or does not use tools, exploration falls back to a direct JSON analysis prompt.
 
-This means every compaction builds on the last one — not from scratch.
+### 6. Phase 3 — Synthesize
 
-### Structured JSON state output
-Alongside the human-readable Markdown summary, Smart Compact produces a machine-readable `CompactionState` JSON object:
+`src/phases/synthesize.ts` supports two paths:
 
-- goal, decisions, constraints
-- modified/read/deleted files
-- unresolved and resolved errors
-- open loops with stable IDs
-- next actions and critical context
-- session type and version
+#### Single-pass
+Used when the pruned conversation fits under the profile's `singlePassMaxTokens`.
 
-This structured state enables better verification, follow-up tracking, and future retrieval integration.
+#### Hierarchical
+Used for larger sessions:
+
+- merge heuristic and exploratory boundaries
+- chunk messages with `chunkLlmMessages()`
+- batch chunks with `createBatches()`
+- summarize each batch with `summarizeBatch()`
+- assemble a final summary with `assembleLLM()`
+- fall back to `assembleFallback()` if assembly fails
+
+Important synthesis behaviors already present in code:
+
+- **decision propagation** into later batch prompts
+- **session-type-specific prompting**
+- **topic-level budget hints** during assembly pre-processing
+- **provider-aware batch concurrency** from `src/utils/tokens.ts`
+
+### 7. Phase 4 — Verify
 
-### Decision propagation
-Batch summaries receive decisions from earlier segments, reducing a common failure mode where later summaries "forget" what was decided earlier.
+`src/phases/verify.ts` scores the summary against deterministic extraction data.
+
+It checks for:
+
+- missing modified files
+- missing unresolved errors
+- missing high-confidence constraints
+- missing goal coverage
+- missing required sections
+- suspicious fabricated file references
+- done/unresolved inconsistency
+- missing explicit decisions
+- missing open-loop coverage when unresolved errors exist
 
-### Verification-oriented synthesis
-The final summary is checked against extracted facts. If important information is missing, Smart Compact tries to patch it deterministically before spending another LLM call.
+Repair strategy is intentionally ordered:
 
-### Redundancy pruning
-Input is reduced by removing or compressing low-value patterns such as:
+1. no patch if score is acceptable
+2. deterministic patch first
+3. LLM patch only if deterministic patch is insufficient
 
-- duplicate file reads
-- repetitive failure chains
-- empty acknowledgments
-- oversized tool outputs
+### 8. Post-processing and persistence
 
-### Cross-session project context
-The extension keeps a small project fingerprint so later compactions can reuse context such as:
+After verification, the pipeline:
 
-- dominant language
-- likely framework
-- important directories
-- recently relevant files
+- extracts open loops
+- injects `## Open Loops`
+- builds a machine-readable `CompactionState`
+- loads previous compaction state
+- computes delta across compactions
+- injects `## Changes Since Last Compaction`
+- saves project fingerprint
+- saves compaction state
+- appends metrics log
+- attempts post-compaction damage detection
 
-### Auto-triggered compaction
-When enabled, Smart Compact hooks into Pi's `session_before_compact` event and can replace default blind compaction with a smarter summary.
+### 9. Applying compaction
+
+The extension stores the result in an in-memory pending object containing:
+
+- summary
+- first kept entry id
+- tokens before compaction
+- details payload
+- structured compaction state
+
+That pending summary is valid for **5 minutes** and is consumed by:
+
+- immediate `ctx.compact()` in the slash-command flow, or
+- the next `session_before_compact` hook call in tool-driven flows
+
+---
+
+## Repository layout
+
+### Top-level
+
+```text
+.
+├── CHANGELOG.md
+├── DEVPLAN.md
+├── LICENSE
+├── README.md
+├── dist/
+│   └── index.js
+├── docs/
+│   └── assets/
+│       └── pi-smart-compact.png
+├── package.json
+├── src/
+├── test/
+└── tsconfig.json
+```
+
+### Source modules
+
+| File | Role |
+| --- | --- |
+| `src/index.ts` | Extension registration: command, hook, tool |
+| `src/core.ts` | End-to-end pipeline orchestration |
+| `src/constants.ts` | Version, prompts, profiles, thresholds, config keys |
+| `src/types.ts` | Shared types and guards |
+| `src/phases/explore.ts` | Exploration phase, tool loop, JSON fallback |
+| `src/phases/synthesize.ts` | Chunking, batching, single-pass and hierarchical synthesis |
+| `src/phases/verify.ts` | Verification, deterministic patch, LLM patch |
+| `src/ui/overlays.ts` | 2-step picker UI, progress notices, result screen |
+| `src/utils/cache.ts` | Metrics, cache-aware LLM options, extraction cache |
+| `src/utils/damage.ts` | Post-compaction regression signal detection |
+| `src/utils/extraction.ts` | Deterministic extraction and open-loop detection |
+| `src/utils/fingerprint.ts` | Cross-session project fingerprinting |
+| `src/utils/helpers.ts` | Config loading, backups, batching, prompt helpers |
+| `src/utils/pruning.ts` | Redundancy pruning before compaction |
+| `src/utils/state.ts` | Compaction state persistence and delta logic |
+| `src/utils/tokens.ts` | Provider capabilities and token estimation |
+
+### Tests
+
+| Test file | Coverage |
+| --- | --- |
+| `test/extraction.test.ts` | deterministic extraction |
+| `test/exploration.test.ts` | exploration parsing and gating |
+| `test/eval.test.ts` | gold scenarios, delta evaluation, fabrication safety |
+| `test/fingerprint.test.ts` | project fingerprint helpers |
+| `test/pruning.test.ts` | redundancy pruning |
+| `test/semantic-compact.test.ts` | legacy-name regression coverage |
+| `test/state.test.ts` | open loops, state, delta, persistence |
+| `test/tokens.test.ts` | token estimation and provider caps |
+| `test/verify.test.ts` | verification and patching |
+
+---
+
+## Runtime artifacts
+
+The current code writes to these paths at runtime:
+
+| Artifact | Path |
+| --- | --- |
+| Settings file | `~/.pi/agent/settings.json` |
+| Conversation backups | `~/.pi/agent/compact-backups/` |
+| Extraction cache | `~/.pi/agent/.cache/compact-extraction-<session>.json` |
+| Metrics log | `~/.pi/agent/.cache/compact-metrics.jsonl` |
+| Project fingerprints | `~/.pi/agent/.cache/smart-compact/projects/<projectId>.json` |
+| Compaction states | `~/.pi/agent/.cache/smart-compact/states/<projectId>.json` |
+| Damage reports | `~/.pi/agent/.cache/smart-compact/damage-reports.jsonl` |
+
+### TTLs currently implemented
+
+| Item | TTL |
+| --- | --- |
+| pending in-memory compaction | 5 minutes |
+| exploration tool-support cache | 30 minutes |
+| extraction cache | 1 hour |
+| compaction state | 7 days |
+| project fingerprint | 30 days |
 
 ---
 
 ## Installation
 
-### Recommended: install as a Pi package
+### npm / Pi package
 
 ```bash
 pi install npm:pi-smart-compact
 ```
 
-### Or install from GitHub
+### GitHub
 
 ```bash
 pi install git:github.com/alpertarhan/pi-smart-compact
 ```
 
-### Or work on it locally
+### Local development
 
 ```bash
 cd ~/.pi/agent/extensions
@@ -252,51 +420,39 @@ bun install
 bun run build
 ```
 
-The published package loads the compiled extension entry at **`dist/index.js`**. Source code lives in `src/`, but the package manifest points Pi at `./dist/index.js` for distribution.
-
 ---
 
-## Quick start
+## Usage
 
-### Slash command
+### Interactive
 
 ```bash
 /smart-compact
 ```
 
-This opens the interactive picker and lets you choose:
+With no arguments, the extension opens a **2-step TUI**:
 
-- model
-- compression profile
+1. model selection
+2. profile selection
 
-### Direct usage
+### Direct command examples
 
 ```bash
 /smart-compact anthropic/claude-sonnet-4 balanced
-```
-
-### Dry run
-
-```bash
 /smart-compact dry-run
-```
-
-### Verbose mode
-
-```bash
 /smart-compact debug
+/smart-compact "focus on auth changes and unresolved follow-up work"
 ```
 
-### Add steering / follow-up emphasis
+Argument parsing in the current code supports:
 
-```bash
-/smart-compact "focus on auth changes and remaining follow-up work"
-```
+- model ids containing `/`
+- profiles: `light`, `balanced`, `aggressive`
+- flags: `verbose`, `debug`, `dry-run`
+- remaining free text as a user steering note
 
 ### Tool usage
 
-Smart Compact also registers an agent-callable tool:
-
 ```json
 {
   "name": "smart_compact",
@@ -308,6 +464,12 @@ Smart Compact also registers an agent-callable tool:
 }
 ```
 
+Tool behavior is slightly different from the slash command:
+
+- it generates a pending smart summary
+- it does **not** immediately compact when `skipCompact` is used internally
+- it expects the next tree compaction to consume the pending result within 5 minutes
+
 ---
 
 ## Configuration
@@ -332,28 +494,29 @@ Add this to `~/.pi/agent/settings.json`:
 }
 ```
 
-### Supported options
-
-| Option | Type | Default | Meaning |
-| --- | --- | --- | --- |
-| `profile` | `"light" \| "balanced" \| "aggressive"` | `"balanced"` | Default compaction profile |
-| `summaryModel` | `string \| null` | `null` | Override summarization model |
-| `segmentationModel` | `string \| null` | `null` | Override exploration model |
-| `autoTrigger` | `boolean` | `true` | Run automatically before Pi's built-in compaction |
-| `backupEnabled` | `boolean` | `true` | Save a backup before compaction |
-| `profiles` | `object` | built-in defaults | Override per-profile budgets |
-
-### Profiles
+### Supported keys
 
-| Profile | Summary budget | Keep recent | Best for |
-| --- | --- | --- | --- |
-| **light** | 10K | 30K | sessions where more detail should survive |
-| **balanced** | 6K | 20K | general daily development |
-| **aggressive** | 3K | 10K | large contexts and faster reduction |
+| Key | Type | Default |
+| --- | --- | --- |
+| `profile` | `light \| balanced \| aggressive` | `balanced` |
+| `summaryModel` | `string \| null` | `null` |
+| `segmentationModel` | `string \| null` | `null` |
+| `autoTrigger` | `boolean` | `true` |
+| `backupEnabled` | `boolean` | `true` |
+| `backupDir` | `string` | `~/.pi/agent/compact-backups` |
+| `profiles` | partial per-profile overrides | built-ins |
+
+### Profiles currently shipped
+
+| Profile | Summary budget | Keep recent | Min chunk | Max chunk | Single-pass max | Batch max |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| `light` | 10000 | 30000 | 800 | 12000 | 40000 | 30000 |
+| `balanced` | 6000 | 20000 | 500 | 8000 | 30000 | 24000 |
+| `aggressive` | 3000 | 10000 | 300 | 6000 | 20000 | 18000 |
 
 ### Backward compatibility
 
-For migration safety, the extension still accepts the older config key:
+The code still accepts the old config key:
 
 - `semanticCompact`
 
@@ -363,9 +526,9 @@ but the current key is:
 
 ---
 
-## Output format
+## Output contract
 
-Smart Compact produces structured Markdown designed to be both human-readable and useful to the agent:
+The generated Markdown is expected to follow this structure:
 
 ```markdown
 ## Goal
@@ -384,213 +547,91 @@ Smart Compact produces structured Markdown designed to be both human-readable an
 ## Topics Covered
 ```
 
-New sections:
-
-- **Open Loops** — unresolved tasks with priority and file references
-- **Changes Since Last Compaction** — delta from previous compaction state
-
-The format is intentionally opinionated. It is optimized to preserve:
-
-- actionable state
-- exact references
-- unresolved issues
-- open loops and follow-up integrity
-- state transitions across compactions
-- clear continuation paths
-
-### Structured JSON output
-
-In addition to Markdown, Smart Compact produces a structured `CompactionState` object accessible in compaction details:
-
-```json
-{
-  "goal": "Build auth module",
-  "decisions": [{ "id": "decision-1", "summary": "Use JWT", "type": "explicit" }],
-  "constraints": [{ "id": "constraint-1", "text": "Must use TypeScript", "category": "requirement" }],
-  "modifiedFiles": ["src/auth.ts"],
-  "unresolvedErrors": [],
-  "openLoops": [{ "id": "loop-1", "type": "follow-up", "priority": "normal", "summary": "add tests" }],
-  "resolvedErrors": [{ "id": "error-1", "message": "login returns undefined", "tool": "bash" }],
-  "nextActions": ["Add unit tests for auth"],
-  "sessionType": "implementation",
-  "compactionVersion": "7.7.0"
-}
-```
-
----
-
-## Architecture details
-
-## 1) Extract
-
-The extraction phase performs zero-LLM analysis on message structure.
+The extension also builds a structured `CompactionState` object containing:
 
-It identifies:
-
-- file operations from tool calls and tool results
-- no-op edits (`applied: 0`, `no changes`)
-- tool errors and bash-like failures
-- retries and likely resolutions
-- explicit `ask_user` decisions
-- implicit user choices such as "use X instead of Y"
-- English and Turkish constraint language
-- topic boundaries from file transitions, error density, and user shift cues
-
-## 2) Explore
-
-Exploration is only used when the session is complex enough to justify it.
-
-When active, the model can use tools such as:
-
-- `get_message_range`
-- `search_conversation`
-- `get_recent_user_messages`
-- `get_context_around`
-- `get_file_changes`
-- `get_error_chain`
-
-This phase helps refine:
-
-- topic boundaries
-- cross-topic relationships
-- missing constraints
-- completion state
-- narrative continuity
-
-## 3) Synthesize
-
-Synthesis supports two modes:
-
-### Single-pass
-Used when the compacted portion is small enough.
-
-### Hierarchical
-Used for larger sessions:
-
-- chunk messages into segments
-- summarize segments in batches
-- propagate prior decisions forward
-- merge summaries into one final structured summary
-
-## 4) Verify
-
-Verification checks the final summary against deterministic extraction data.
-
-It looks for issues such as:
-
-- missing modified files
-- missing unresolved errors
-- missing strong constraints
-- missing explicit decisions
-- missing Open Loops section when unresolved errors exist
-- suspicious file references not seen in the conversation
-- structural omissions
-
-If needed, Smart Compact applies:
+- goal
+- decisions
+- constraints
+- modified/read/deleted files
+- unresolved/resolved errors
+- open loops
+- topics
+- next actions
+- critical context
+- session type
+- compaction version
 
-1. **deterministic patching first**
-2. **LLM patching only if necessary**
+This state is persisted on disk and reused for delta tracking on later compactions.
 
 ---
 
-## Quality and safety controls
-
-### Exact-name discipline
-Prompts explicitly tell the model to preserve exact file paths, identifiers, and verified facts.
-
-### Immutable context framing
-Deterministically extracted facts are presented as ground truth during assembly.
+## Quality controls
 
-### Verification-first fallback strategy
-The extension prefers:
+The current codebase includes these safeguards:
 
-- no patch
-- deterministic patch
-- LLM patch as last resort
-
-### Backups
-Conversation backups can be written before compaction.
-
-### Metrics
-The pipeline tracks:
-
-- call counts
-- input/output token volume
-- cache hit rate
-- average latency
-
-### Incremental extraction cache
-Structured extraction results are cached per session to avoid reprocessing unchanged history.
-
-### Cross-compaction state persistence
-After each compaction, the structured state is persisted to disk. On the next compaction, Smart Compact:
-
-1. loads the previous state
-2. computes a delta (resolved loops, new errors, goal shifts, etc.)
-3. injects `## Changes Since Last Compaction` into the summary
-4. saves the updated state for the next cycle
-
-This creates a **compaction memory chain** — every compaction builds on the last.
+- deterministic extraction before any summarization
+- adaptive exploration skip for simple sessions
+- project fingerprint reuse across sessions
+- incremental extraction cache
+- provider-aware token estimation and concurrency
+- deterministic verification scoring
+- deterministic patch before LLM patch
+- hallucinated file-reference detection
+- open-loop injection
+- cross-compaction delta injection
+- post-compaction damage detection
+- backup creation before compaction
+- metrics logging for LLM cost/latency/cache usage
 
 ---
 
-## Compatibility
-
-Smart Compact is designed as a standalone Pi extension, but it is also intended to fit naturally into richer Pi setups.
+## Current caveats
 
-It should be a good conceptual fit alongside workflow-oriented extensions and packages such as:
+To keep this README aligned with the repository's **actual** current state:
 
-- `pi-agent-flow`
-- `pi-simplify`
-- `pi-lens`
-
-As always with Pi packages, review interactions in your own environment if you combine multiple extensions that hook into related session flows.
+1. **`bun run typecheck` is currently failing.** The repo builds with Bun and passes tests, but strict TypeScript compatibility is not clean right now.
+2. **One legacy test filename remains:** `test/semantic-compact.test.ts`.
+3. **The package is published from `dist/`, not directly from `src/`.** Source and tests are not included in the package tarball.
+4. **The extension depends on Pi runtime APIs and peer packages** (`@earendil-works/pi-ai`, `@earendil-works/pi-coding-agent`, `@earendil-works/pi-tui`, `typebox`).
 
 ---
 
 ## Development
 
-### Project structure
+### Install
 
-```text
-src/         TypeScript source
- dist/        compiled package entry for distribution
- test/        Bun tests
- README.md    package documentation
+```bash
+bun install
 ```
 
-### Test suite
-
-Smart Compact has **91 tests** across 9 files, including:
-
-- **Unit tests** — extraction, tokens, verification, pruning, fingerprint, exploration
-- **State tests** — open loops, compaction state, delta computation, state persistence
-- **Evaluation harness** — 5 gold conversation scenarios with expected extraction results, delta evaluation across compactions, and fabrication safety checks
+### Test
 
 ```bash
-bun test                # run all 91 tests
-bun test test/eval.test.ts  # evaluation harness only
+bun test
+bun test test/eval.test.ts
 ```
 
-### Local commands
+### Build
 
 ```bash
-bun install
 bun run build
-bun run typecheck
 ```
 
-### Build output
+Current build command:
 
-Published builds use:
+```bash
+rm -rf dist && mkdir dist && bun build ./src/index.ts --outdir ./dist --target bun --external '@earendil-works/*' --external 'typebox'
+```
 
-- `dist/index.js`
+### Typecheck
 
-This keeps the distributed package aligned with common npm packaging expectations while preserving a TypeScript-first source layout during development.
+```bash
+bun run typecheck
+```
 
-### Local package path
+At the moment, this command is expected to report errors.
 
-If you are developing inside Pi directly, this project commonly lives at:
+### Typical local path inside Pi
 
 ```text
 ~/.pi/agent/extensions/pi-smart-compact
@@ -598,42 +639,6 @@ If you are developing inside Pi directly, this project commonly lives at:
 
 ---
 
-## Limitations
-
-Smart Compact is strong, but it is not magic.
-
-A few honest limitations:
-
-- it still depends on model quality during exploration and synthesis
-- very noisy sessions can still produce weaker summaries than ideal
-- verification is strong for extracted facts, but not a formal proof system
-- project fingerprints are intentionally lightweight, not a full memory database
-- token estimates are calibrated heuristics, not exact provider tokenizers
-
----
-
-## Contributing
-
-Issues, suggestions, and pull requests are welcome.
-
-If you contribute, the best changes tend to be:
-
-- measurable
-- easy to validate with tests
-- explicit about quality/cost trade-offs
-- careful about hallucination risk and continuation quality
-
-Typical flow:
-
-```bash
-git checkout -b feat/my-change
-bun test
-bun run build
-git commit -m "feat: ..."
-```
-
----
-
 ## License
 
 MIT © [Alper Tarhan](https://github.com/alpertarhan)