pi-smart-compact 7.15.1 → 7.18.0

package/CHANGELOG.md

@@ -1,5 +1,80 @@
 # Changelog
 
+## [7.18.0] - 2026-06-26
+
+### Fixed
+- **`mergeExtractions` mainGoal corruption** — incremental extraction replaced the original goal with a delta-suffix message; now prefers `base.mainGoal`.
+- **`mergeExtractions` boundary gap** — `lastUserMessages`/`lastErrors` now span the cache boundary instead of dropping base's tail.
+- **`catalogErrors` id-less retry** — retry resolution falls back to tool-name match when the retry call lacks an id.
+- **Pruning hides errors** — short error-containing tool outputs preserved from 800-char truncation (shared `LIKELY_ERROR_RE`).
+- **`computeDelta` collision** — prefix-slice keys collided different items sharing an opening; now full-normalized text.
+- **`smartKeepBoundary` dead regex** — `path="..."` never matched agent output; replaced with toolCall-arg file extraction.
+- **`buildCompactionState` basename** — raw basename → `buildPathNeedles` (generic-basename gate, consistent with `extractOpenLoops`).
+- **`extractNextActions`/`Critical`** — case-sensitive regex → canonical `findSection`.
+- **`CONSTRAINT_PATTERNS`** — `\b` → lookarounds so Turkish-leading chars (`önemli`, `şart`) match.
+- **`verifySummary` fuzzy** — salient-token keyword extraction; error snippet whitespace-normalized.
+- **`estimateTokens` JSON** — density fallback for non-brace-first JSON (capped at 8KB).
+
+### Added
+- **`domain/keywords.ts`** — `extractCheckKeywords` (shared by verify + damage).
+- **Centralized constants** — `TRUNC` (truncation budgets), `ID_PREFIX`, `TUNING` (EMA/clamp/confidence), shared durations, `EXTRACTION_CACHE_PREFIX`, `LIKELY_ERROR_RE`.
+
+### Changed
+- **Zero inline magic values** — ~70 truncation literals, ID prefixes, confidence scores, TTL duplications (7d×3, 1h×2), and tuning factors centralized into named constants.
+- **`makeCompactSessionId`** — single source (services.ts); cache.ts fallback deduped.
+- **`damage.ts` re-question** — shared `extractCheckKeywords` (threshold relaxed for high-precision salient tokens).
+
+## [7.17.0] - 2026-06-26
+
+### Added
+- **Name-agnostic tool classification** (`src/domain/tool-semantics.ts`) — a pure `classifyTool(args)` that classifies a tool call by its argument shape (`mutates` / `accesses` / `executes` / `other`) plus `extractToolPath(args)`. A tool is a write because its arguments carry a content payload, not because its name contains "write" — so this auto-adapts to tools the code has never seen (`hypa_*`, MCP servers, custom extensions) with no name list to maintain.
+
+### Changed
+- **Extraction is now name-agnostic.** `trackFileOps`, `catalogErrors`, `segmentTopicsHeuristic` (`utils/extraction.ts`), `get_file_changes` (`phases/explore.ts`), and re-read detection (`utils/damage.ts`) all classify via `classifyTool` instead of substring name matching. Removes the `WRITE/DELETE/READ_TOOL_HINTS` lists, `hasToolHint`, and the hardcoded `=== "bash"` gate. Shell-like tools (`hypa_shell`, …) and path-bearing readers (`hypa_grep`/`find`/`ls`) are now detected correctly by argument shape.
+- **Model resolution deduplicated** (`index.ts`) — the `provider/id` split + `modelRegistry.find` pattern (duplicated three times) is now a single `findModelById` helper; `resolveModelArg` removed.
+- **`SHIFT_RE` Turkish coverage** — the topic-shift cue regex was ASCII-only and silently missed natural Turkish spellings (`şimdi`, `geçelim`, `bakalım`, `yapalım`, `başka`); both spellings now match, consistent with `FOLLOWUP_RE`'s dual-spelling convention.
+- **`CONSTRAINT_PATTERNS` readability** — the Turkish regexes use raw UTF-8 characters instead of `\u00f6`/`\u015f` escapes (the source is UTF-8; the escapes added no value and hurt readability).
+
+### Fixed
+- **`trackFileOps` duplicated branch** — the `isTruncated` and `!NO_OP_RE` arms had identical bodies; collapsed into one condition (`isTruncated(resultText) || !NO_OP_RE.test(resultText)`).
+- **Dead locals in `segmentTopicsHeuristic`** — `type` and `primaryFile` were assigned every iteration but never read (the topic push uses `currentType`/`currentPrimaryFile`); removed. Path lookup now uses `extractToolPath`, matching the other consumers.
+
+### Tests
+- New `test/tool-semantics.test.ts` covering all four tool classes, the path-only delete/read ambiguity, and `extractToolPath` across key variants. Extraction fixtures updated to realistic content payloads; a regression case proves an unknown tool name (`totally_unknown_mcp_tool`) is still classified correctly. Suite: 501 tests across 43 files.
+
+## [7.16.0] - 2026-06-18
+
+### Added
+- **Pinned never-compact context** (`smartCompact.pinPaths`) — file paths that must always survive compaction regardless of what the LLM summary includes. Surfaced in the summary's Files Read via a deterministic, LLM-free `ensurePinnedPaths` step in `buildState`.
+- **Damage auto-remediation** — `detectDamage` now collects the files the agent re-reads after a compaction (`reReadFiles`), persists them as remediation hints, and the *next* compaction re-preserves them (merged with `pinPaths`) so lost context stops being lost twice. Closes the detect → remediate loop.
+- **`/smart-compact restore`** — list, view, and restore backups. `listBackups`/`readBackupContent` make the previously write-only backups browsable; `showRestorePicker` + `showRestoreAction` + `showBackupViewer` provide a TUI; and a true restore forks from the current leaf and re-injects the pre-compaction content as context via `sendMessage` (graceful fallback to view on any failure).
+- `asBranchMessage` / `asSerializableMessages` boundary adapters in `src/infra/ai-messages.ts`, documenting why each cross-package upcast is sound.
+
+### Fixed
+- **session-log timestamp** — `normalizeLogMessage` stamped `Date.now()` (the recovery wall-clock) gated on a nonsensical content-shape condition, and dropped `toolName`. Now parses the log entry's real timestamp and preserves `toolName`.
+- **`synthesize` empty-batch guard** — `batches[0]` could be dereferenced when the chunk list was empty; now guarded with a deterministic fallback.
+- **`backupDir` config validation** — the one config key without type validation now rejects non-string values.
+- **result-screen timer** — the `setTimeout` used in the result-screen `Promise.race` is now cleared in a `finally` instead of lingering up to 5s.
+- **negative exploration boundary clamp** — `normalizeBoundaries` now lower-clamps `afterIndex` to 0 (LLMs occasionally emit negative values) and guards `confidence` against non-numeric values.
+- **`computeToolCharPercentage` dead branch** — removed the unreachable `block.content` path (text blocks carry `.text`).
+- **`ctx.ui.notify` invalid type** — restore used `"success"`, which `ctx.ui.notify` does not accept; corrected to `"info"`.
+- **state.ts basename recompute** — the per-error file-attribution basename is now precomputed once instead of recomputed for every (error × file) pair.
+
+### Changed
+- **Message cast normalization** — the explore feedback loop now builds native `Message[]` (assistant turns are the real `AssistantMessage` from `trackedComplete`, no longer downcast to `LlmMessage`); `recover`/`persist`/`extract` route through the documented `asBranchMessage`/`asSerializableMessages` adapters. Removes the lossy `as unknown as Message[]` casts and the silent dropping of `usage`/`api`/`provider`/`model`/`stopReason`.
+- **Tools cast normalization** — `EXPLORATION_TOOLS` is now declared as native `Tool[]` using typebox schemas, removing both `as unknown as Parameters<...>["tools"]` casts. Behavior-preserving: every pi-ai provider only serializes the schema, so real typebox schemas are wire-identical to the previous plain JSON-schema objects.
+- `failedChunkSummary` co-located with `assembleFallback` in `phases/synthesize.ts` and exported (was an untested module-private in the step module).
+- `buildExplorationReportFromParsed` parameter narrowed `any` → `unknown` with proper field validation.
+
+### Build
+- **pi runtime peers resolved 0.79.4 → 0.79.6** and **typebox 1.2.11 → 1.2.16** in the lockfile. `peerDependencies`/`devDependencies` keep their `*` wildcard ranges per the forward-compatibility policy from 7.15.0.
+
+### Tests
+- **+79 tests (414 → 493):** type-guards validators (`isValidSmartCompactDetails`/`sanitizeSmartCompactDetails`), synthesize fallback contracts (`assembleFallback`/`failedChunkSummary`), `ai-messages` adapters, pinned-paths injection, remediation-hints round-trip, backup restore (list/read/build-restore-message), and exploration boundary normalization (negative clamp, confidence guard, non-string mainGoal).
+
+### Docs
+- README, ARCHITECTURE, CONTRIBUTING, SECURITY, SUPPORT, RELEASE, and CODE_OF_CONDUCT redesigned; new `docs/assets/banner.svg` hero. CONTRIBUTING drift fixed (`core.ts`/`DEVPLAN.md`/`ROADMAP.md` references removed; repo map updated to the layered architecture).
+
 ## [7.15.1] - 2026-06-15
 
 ### Fixed

package/README.md

@@ -1,396 +1,277 @@
-# pi-smart-compact
+<div align="center">
 
-[CI](https://github.com/alpertarhan/pi-smart-compact/actions/workflows/ci.yml) · [npm](https://www.npmjs.com/package/pi-smart-compact) · [MIT License](./LICENSE) · [Pi extension](https://github.com/earendil-works/pi)
+<img src="./docs/assets/banner.svg" alt="pi-smart-compact" width="860" />
 
-<p align="center">
-  <img src="./docs/assets/pi-smart-compact.png" alt="pi-smart-compact" width="420" />
-</p>
+[![CI](https://github.com/alpertarhan/pi-smart-compact/actions/workflows/ci.yml/badge.svg)](https://github.com/alpertarhan/pi-smart-compact/actions/workflows/ci.yml)
+[![npm version](https://img.shields.io/npm/v/pi-smart-compact?color=60a5fa)](https://www.npmjs.com/package/pi-smart-compact)
+[![license](https://img.shields.io/npm/l/pi-smart-compact?color=22c55e)](./LICENSE)
+[![Pi extension](https://img.shields.io/badge/Pi-extension-fbbf24)](https://github.com/earendil-works/pi)
 
-> Verification-oriented smart compaction for the [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent).
+**Verification-oriented smart compaction for the [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent).**
 
-`pi-smart-compact` replaces blind conversation trimming with a structured compaction pipeline that tries to preserve what an agent actually needs to continue working: the goal, modified files, unresolved errors, decisions, constraints, and open follow-up loops.
+</div>
 
-It uses an **EESV** pipeline:
+Default compaction trims your conversation blind. `pi-smart-compact` keeps what
+the agent actually needs to continue — the **goal, changed files, unresolved
+errors, decisions, constraints, and open loops** — through a verified
+**Extract → Explore → Synthesize → Verify** pipeline.
 
-**Extract → Explore → Synthesize → Verify**
-
-## Highlights
-
-- **Pi-native integration** — `/smart-compact`, `smart_compact`, and `session_before_compact` support.
-- **Verification-oriented output** — deterministic extraction and repair before trusting LLM synthesis.
-- **Adaptive cost profile** — skips unnecessary work on small sessions and uses chunking only when useful.
-- **Operational safety** — pending summaries expire, backups are available, and metrics make regressions visible.
-- **Companion-friendly** — designed to coexist with context hygiene tools such as `pi-toolkit`.
-
-Under the hood, the design is grounded in two core ideas:
-
-- **agentic compaction**: let the system inspect and reason about the session instead of collapsing everything into generic prose
-- **Kamradt-style chunking**: break large conversations into more coherent segments before synthesis
+> Facts first, synthesis second, verification last.
 
 ---
 
-## What this project is
+## Why
 
-This package is a **Pi extension** with three integration surfaces:
+Default compaction produces a vague recap and quietly drops the operational
+context that matters most during coding. The result is the classic
+*"didn't we already fix this?"* loop.
 
-| Surface | Purpose |
+| Default compaction | `pi-smart-compact` |
 | --- | --- |
-| `/smart-compact` | manual compaction from the chat UI |
-| `session_before_compact` | auto-run before Pi's default compaction |
-| `smart_compact` tool | agent-callable compaction for long sessions |
-
-The extension stages a short-lived pending summary in memory, then hands it back to Pi when compaction is applied.
-
----
-
-## Project status
+| Trims by token count | Extracts facts deterministically (zero LLM) |
+| Generic prose recap | Structured working-state summary |
+| Loses files / errors / decisions | Preserves them, then **verifies** they survived |
+| No regression signal | Damage detection + metrics dashboard |
 
-This is an actively maintained Pi extension. The public API is intentionally small, but the internals are still evolving as Pi's compaction lifecycle and extension APIs mature. Pin versions in production workflows if compaction behavior is mission-critical.
+## Install
 
-## Compatibility note
-
-`pi-smart-compact` sits close to Pi's compaction path: it registers `session_before_compact`, reads the active branch and session log, stages a pending summary, and may call Pi's native compaction flow from `/smart-compact`.
-
-Because of that, use extra care with extensions that also manipulate:
-
-- **compaction hooks** — especially extensions that return a custom result from `session_before_compact`
-- **session / branch history** — rewriting, pruning, reordering, or replacing entries before compaction
-- **message identity** — removing entry IDs, tool-call IDs, or tool-result metadata used to align log entries
-- **tool output content** — truncating or rewriting `toolResult` messages before extraction
-- **compaction boundaries** — moving the keep/discard split or splitting `toolCall` / `toolResult` pairs
-- **session log storage** — replacing or deleting Pi's `.jsonl` logs under `~/.pi/agent/sessions`
-
-It is intentionally compatible with, and recommended alongside, [`pi-toolkit`](https://github.com/ersintarhan/pi-toolkit): pi-toolkit handles everyday context hygiene such as anchors, pivots, status lines, and old tool-output trimming; `pi-smart-compact` handles high-pressure verified compaction. The integration protects recent pi-toolkit anchors and can recover original tool outputs from the session log when older tool results were trimmed.
+```bash
+pi install npm:pi-smart-compact
+```
 
-If you use another automatic compaction or context-rewriting extension, prefer enabling only one `session_before_compact` owner unless the hook order and returned values are explicitly coordinated.
+From GitHub:
 
----
+```bash
+pi install git:github.com/alpertarhan/pi-smart-compact
+```
 
-## Why it exists
+## Quick start
 
-Default compaction often loses the parts that matter most during coding work:
+```bash
+/smart-compact                              # interactive — pick model + profile
+/smart-compact anthropic/claude-sonnet-4 balanced
+/smart-compact "focus on auth + unresolved follow-ups"
+/smart-compact metrics                      # profile / provider comparison
+/smart-compact dashboard                    # interactive TUI dashboard
+/smart-compact restore                      # list + view + restore backups
+```
 
-- which files were actually changed
-- which errors are still unresolved
-- what the user explicitly asked for
-- what decisions already won
-- what should happen next
+Or let the agent call it as a tool on long sessions:
 
-`pi-smart-compact` is built to preserve that operational context instead of producing a vague recap.
+```jsonc
+{
+  "name": "smart_compact",
+  "parameters": { "profile": "balanced", "dashboard": false }
+}
+```
 
----
+Auto-compaction also runs before Pi's native compact once context pressure
+crosses your threshold (default 60% **actual** context usage).
 
 ## How it works
 
 ```mermaid
 flowchart LR
-    A[Extract<br/>deterministic facts] --> B[Explore<br/>optional targeted analysis]
-    B --> C[Synthesize<br/>single-pass or chunked summary]
-    C --> D[Verify<br/>score gaps and repair]
-    D --> E[Return smart compaction to Pi]
+    A["Extract<br/>deterministic facts<br/><i>0 LLM calls</i>"] --> B["Explore<br/>optional deep<br/>analysis"]
+    B --> C["Synthesize<br/>single-pass or<br/>chunked summary"]
+    C --> D["Verify<br/>score gaps,<br/>repair"]
+    D --> E["Return verified<br/>summary to Pi"]
 ```
 
-### Pipeline summary
-
-1. **Extract**
-   - deterministically pulls files, errors, decisions, constraints, topics, and open loops from the session
-2. **Explore**
-   - optionally inspects the conversation more deeply when the session is complex
-3. **Synthesize**
-   - creates either a single-pass summary or a chunked hierarchical summary
-4. **Verify**
-   - checks the result against extracted facts and patches missing critical details
-
-In short: **facts first, synthesis second, verification last**.
-
----
-
-## What it tries to preserve
-
-- user goal
-- constraints and preferences
-- modified / read / deleted files
-- unresolved and resolved errors
-- key decisions
-- open follow-up work
-- critical context needed for the next turn
-- delta from the previous compaction
-
----
+| Stage | What it does |
+| --- | --- |
+| **Extract** | Deterministically pulls files, errors, decisions, constraints, topics, and open loops — no LLM, the ground truth. |
+| **Explore** | Optionally inspects the conversation more deeply when the session is complex. |
+| **Synthesize** | Single-pass for short sessions, Kamradt-style chunked + assembled for long ones. |
+| **Verify** | Scores the result against extracted facts and patches missing critical details. |
 
-## Installation
+**What it preserves:** user goal · constraints & preferences · modified / read /
+deleted files · unresolved & resolved errors · key decisions · open follow-up
+work · critical next-turn context · the delta since the previous compaction.
 
-### npm / Pi package
+## Integration surfaces
 
-```bash
-pi install npm:pi-smart-compact
-```
+| Surface | When | Detail |
+| --- | --- | --- |
+| `/smart-compact` | Manual | Interactive picker or direct args; bypasses the adaptive gate. |
+| `session_before_compact` | Auto | Runs before Pi's native compaction when context pressure is high. |
+| `smart_compact` tool | Agent | Prepares a pending summary; Pi applies it on the next natural compact. |
 
-### GitHub
+A short-lived pending summary is staged in memory (5-minute TTL) and handed to
+Pi when compaction is applied.
 
-```bash
-pi install git:github.com/alpertarhan/pi-smart-compact
-```
+## Example output
 
-### Local development
+A generated summary follows a stable, structured contract:
 
-```bash
-cd ~/.pi/agent/extensions
-git clone https://github.com/alpertarhan/pi-smart-compact.git
-cd pi-smart-compact
-bun install
-bun run build
-```
+```markdown
+## Goal
+Add retry/backoff to the LLM client so transient 429/5xx don't abort compaction.
 
----
+## Constraints & Preferences
+- [requirement] never compact mid-turn from the tool path
 
-## Quick start
+## Progress
+### Done
+- [x] Added `withRetry` wrapper in src/infra/llm-retry.ts
+### In Progress
+- [ ] Wire retry client into the services container
+### Blocked
+- None
 
-### Interactive
+## Key Decisions
+- **Honor Retry-After verbatim**: providers that set it know their limits best.
 
-```bash
-/smart-compact
-```
+## Files Modified
+- src/infra/llm-retry.ts
+- src/infra/llm-client.ts
 
-With no arguments, the extension opens a small picker for:
+## Files Read
+- src/app/run-smart-compact.ts
 
-1. model
-2. profile
+## Open Loops
+- [high] Retried but unresolved: AbortSignal ignored by some providers
 
-### Direct command examples
+## Changes Since Last Compaction
+- New files touched: src/infra/llm-retry.ts
+- New loops: AbortSignal ignored by some providers
 
-```bash
-/smart-compact anthropic/claude-sonnet-4 balanced
-/smart-compact dry-run
-/smart-compact debug
-/smart-compact metrics
-/smart-compact dashboard
-/smart-compact "focus on auth changes and unresolved follow-up work"
-```
+## Next Steps
+1. Add an outer hard-timeout as a second line of defense
 
-### Tool usage
+## Critical Context
+- 408/425/429/5xx are retriable; 4xx (other) fails fast
 
-```json
-{
-  "name": "smart_compact",
-  "parameters": {
-    "profile": "balanced",
-    "verbose": false,
-    "dry_run": false,
-    "report": false,
-    "dashboard": false
-  }
-}
+## Topics Covered
+- LLM retry wrapper (high)
 ```
 
-The tool prepares a pending smart summary and lets Pi consume it on the next natural compaction.
-
----
-
-## Usage notes
-
-- auto/tool compaction is skipped when the context is still small enough (default: below 60% actual context usage)
-- explicit manual `/smart-compact` commands bypass the 60% adaptive gate because the user intentionally requested compaction
-- pi-toolkit `tool=XX%` status means tool-output ratio, **not** context fullness; smart-compact uses actual `context=XX%`
-- the tool path does **not** compact the conversation mid-turn
-- pending summaries are kept in memory for **5 minutes**
-- exploration is adaptive and may be skipped for simple sessions
-- use `/smart-compact metrics` for profile/provider comparisons
-- use `/smart-compact dashboard` to open the interactive TUI dashboard (overview, latest run, current session, recent runs, or write HTML)
-
-This keeps the extension helpful without forcing extra work when it is not needed.
-
----
+A machine-readable `CompactionState` is built alongside it for reuse across
+later compactions (delta tracking, damage detection).
 
 ## Configuration
 
-Add this to `~/.pi/agent/settings.json`:
+Add to `~/.pi/agent/settings.json`:
 
 ```json
 {
   "smartCompact": {
     "profile": "balanced",
     "summaryModel": "anthropic/claude-sonnet-4",
-    "segmentationModel": "anthropic/claude-haiku-3",
     "autoTrigger": true,
-    "autoTriggerTimeoutMs": 120000,
     "minContextPercent": 60,
-    "backupEnabled": true,
-    "profiles": {
-      "balanced": {
-        "summaryBudgetTokens": 6000,
-        "keepRecentTokens": 20000
-      }
-    }
+    "backupEnabled": true
   }
 }
 ```
 
-### Supported keys
-
 | Key | Type | Default |
 | --- | --- | --- |
 | `profile` | `light \| balanced \| aggressive` | `balanced` |
-| `summaryModel` | `string \| null` | `null` |
+| `summaryModel` | `string \| null` | `null` (uses session model) |
 | `segmentationModel` | `string \| null` | `null` |
 | `autoTrigger` | `boolean` | `true` |
 | `autoTriggerTimeoutMs` | `number` | `120000` |
 | `minContextPercent` | `number` | `60` |
 | `backupEnabled` | `boolean` | `true` |
 | `backupDir` | `string` | `~/.pi/agent/compact-backups` |
-| `profiles` | partial per-profile overrides | built-ins |
+| `profiles` | per-profile overrides | built-ins |
+| `pinPaths` | `string[]` | `[]` (paths always preserved) |
 
 ### Profiles
 
-| Profile | Summary budget | Keep recent | Typical use |
+| Profile | Summary budget | Keep recent | Use when |
 | --- | ---: | ---: | --- |
 | `light` | 10000 | 30000 | preserve more detail |
-| `balanced` | 6000 | 20000 | default general use |
+| `balanced` | 6000 | 20000 | default, general use |
 | `aggressive` | 3000 | 10000 | tighter summaries |
 
-### Backward compatibility
+The legacy config key `semanticCompact` is still accepted.
 
-The extension still accepts the old config key `semanticCompact`, but `smartCompact` is the current key.
+## Safeguards
 
----
+**Correctness**
 
-## Output contract
+- Deterministic extraction before any synthesis
+- Verification scoring with deterministic patching **before** LLM patching
+- Hallucinated file-reference detection (SemVer-aware)
+- Open-loop injection + cross-compaction delta tracking
+- Pinned-path preservation (`pinPaths`) — a deterministic, LLM-free guarantee
+- Damage auto-remediation — re-read files feed forward and get re-preserved next compaction
 
-Generated summaries are expected to use this structure:
+**Safety**
 
-```markdown
-## Goal
-## Constraints & Preferences
-## Progress
-### Done
-### In Progress
-### Blocked
-## Key Decisions
-## Files Modified
-## Files Read
-## Open Loops
-## Changes Since Last Compaction
-## Next Steps
-## Critical Context
-## Topics Covered
-```
+- Conversation backups before compaction (retention-pruned), browsable + restorable via `/smart-compact restore`
+- `toolCall` / `toolResult` pair integrity at the compaction boundary
+- Cross-session leak guard on the pending summary
+- Session-log recovery that bypasses truncation of older tool results
 
-The extension also builds a structured `CompactionState` for reuse across later compactions.
+**Observability**
 
----
+- Metrics logging with profile / provider comparison
+- Post-compaction damage (regression) detection
+- Interactive TUI + HTML dashboards
 
-## Safeguards
-
-The current design includes:
-
-- deterministic extraction before summarization
-- adaptive exploration
-- chunked synthesis for larger sessions
-- deterministic verification scoring
-- deterministic patching before LLM patching
-- hallucinated file-reference detection
-- open-loop injection
-- project fingerprinting and delta tracking
-- provider-specific timeout and single-pass strategies
-- multimodal attachment metadata preservation
-- backup creation before compaction
-- metrics logging, profile/provider comparison, and damage detection
+## Usage notes
 
----
+- Auto / tool compaction is skipped while context is small (below 60% actual usage).
+- Manual `/smart-compact` bypasses that gate — you asked for it.
+- `pi-toolkit`'s `tool=XX%` means tool-output ratio, **not** context fullness;
+  smart-compact uses actual `context=XX%`.
+- The tool path does **not** compact mid-turn (it stages a pending summary).
+- Exploration is adaptive and may be skipped for simple sessions.
 
-## Runtime artifacts
+## Companions & compatibility
 
-At runtime, the extension writes to paths under `~/.pi/agent/`, including:
+`pi-smart-compact` is designed to coexist with — and recommended alongside —
+[`pi-toolkit`](https://github.com/ersintarhan/pi-toolkit). pi-toolkit handles
+everyday context hygiene (anchors, pivots, status lines, old tool-output
+trimming); smart-compact handles high-pressure verified compaction. The
+integration protects recent pi-toolkit anchors and recovers original tool
+outputs from the session log.
 
-- `settings.json`
-- `compact-backups/`
-- `.cache/compact-extraction-<session>.json`
-- `.cache/compact-metrics.jsonl`
-- `.cache/smart-compact-report.html`
-- `.cache/smart-compact/projects/<projectId>.json`
-- `.cache/smart-compact/states/<projectId>.json`
-- `.cache/smart-compact/damage-reports.jsonl`
+Because smart-compact sits close to Pi's compaction path, take extra care with
+extensions that also rewrite compaction hooks, branch history, entry IDs, tool
+output, or the compaction boundary. If you run another automatic compaction /
+context-rewriting extension, prefer a single `session_before_compact` owner
+unless hook order is explicitly coordinated.
 
----
-
-## Repository layout
-
-```text
-.
-├── src/
-│   ├── index.ts              # extension registration + command routing
-│   ├── constants.ts          # version, thresholds, prompts
-│   ├── types.ts              # shared types
-│   ├── app/                  # orchestration layer
-│   │   ├── run-smart-compact.ts   # pipeline orchestrator (was core.ts)
-│   │   ├── run-context.ts         # typed stage chain
-│   │   ├── pending-slot.ts        # encapsulated pending-compaction state cell
-│   │   ├── explore-wrap.ts        # explore re-export shim
-│   │   └── steps/                 # 10 stage modules
-│   │       ├── prepare.ts   →   resolves config + auth
-│   │       ├── window.ts    →   picks compaction window
-│   │       ├── recover.ts   →   recovers truncated messages
-│   │       ├── tier.ts      →   chooses compaction tier
-│   │       ├── extract.ts   →   pruning + extraction + cache
-│   │       ├── synthesize.ts→   single-pass / EESV summarization
-│   │       ├── verify.ts    →   structural verify + repair
-│   │       ├── state.ts     →   state machine + open loops
-│   │       ├── persist.ts   →   apply compaction
-│   │       └── metrics.ts   →   success / failure metrics
-│   ├── domain/               # pure semantics (no I/O)
-│   │   ├── summary-schema.ts
-│   │   └── summary-parse.ts
-│   ├── phases/               # algorithms
-│   │   ├── explore.ts
-│   │   ├── synthesize.ts
-│   │   └── verify.ts
-│   ├── infra/                # external-world interaction
-│   │   ├── fs.ts                # atomic writes, advisory locks
-│   │   ├── paths.ts             # canonical paths
-│   │   ├── git.ts               # git-root discovery (cached)
-│   │   ├── clock.ts             # injectable clock
-│   │   ├── llm-client.ts        # LLM client seam
-│   │   ├── llm-retry.ts         # 429/5xx backoff
-│   │   ├── services.ts          # per-run services container
-│   │   └── session-identity.ts # robust session-id resolution
-│   ├── ui/                   # TUI overlays + dashboard
-│   │   ├── overlays.ts
-│   │   └── dashboard-format.ts
-│   └── utils/                # 15 focused utility modules
-├── test/                     # 414 tests across 36 files
-├── docs/
-├── dist/
-└── package.json
-```
+## Runtime artifacts
 
-See `ARCHITECTURE.md` for the full responsibility breakdown of each layer.
+The extension writes under `~/.pi/agent/`:
 
----
+| Path | Purpose |
+| --- | --- |
+| `settings.json` | configuration (read) |
+| `compact-backups/` | conversation backups (retention-pruned) |
+| `.cache/compact-extraction-<session>.json` | incremental extraction cache |
+| `.cache/compact-metrics.jsonl` | metrics log |
+| `.cache/smart-compact-report.html` | HTML dashboard |
+| `.cache/smart-compact/projects/<projectId>.json` | project fingerprint |
+| `.cache/smart-compact/states/<projectId>.json` | reusable compaction state |
+| `.cache/smart-compact/damage-reports.jsonl` | regression signals |
+| `.cache/smart-compact/remediation-<projectId>.json` | files to re-preserve after damage |
 
 ## Development
 
 ```bash
 bun install
-bun test
-bun run build
-bun run typecheck
+bun run typecheck   # tsc --noEmit
+bun test            # full test suite
+bun run build       # dist/ output for publishing
 ```
 
-Build output is published from `dist/`. Pull requests are expected to pass the same verification in GitHub Actions before merge.
+Pull requests run the same verification in GitHub Actions before merge.
 
----
-
-## Project docs
+## Documentation
 
+- [`ARCHITECTURE.md`](./ARCHITECTURE.md) — system design, execution model, layer responsibilities
 - [`CHANGELOG.md`](./CHANGELOG.md) — release history
-- [`ARCHITECTURE.md`](./ARCHITECTURE.md) — system design and execution model
 - [`CONTRIBUTING.md`](./CONTRIBUTING.md) — contributor workflow and expectations
 - [`SECURITY.md`](./SECURITY.md) — vulnerability reporting and data-handling notes
 - [`SUPPORT.md`](./SUPPORT.md) — where to ask for help
 - [`docs/RELEASE.md`](./docs/RELEASE.md) — release checklist
 
----
-
 ## License
 
 MIT © [Alper Tarhan](https://github.com/alpertarhan)