pi-blackhole 0.5.0 → 0.5.2

package/CHANGELOG.md

@@ -2,6 +2,38 @@
 
 ---
 
+## [0.5.2] - 2026-09-07
+
+### Added
+
+- **One-time migration notice for pinned-threshold users.** On the first session start of the 0.5.2 release, users whose config pins a flat `compactAfterTokens` threshold (the pre-curve legacy behavior) get a single TUI notification explaining that auto-compaction can now derive the threshold from the model's context window and how to switch (`/blackhole settings` → Compaction, or `/blackhole changelog` for details). Users already on a derived knob, the preset curve, manual/off mode, or Pi-default engine are never notified; no state is written to disk (read-only filesystems unaffected), and the notice self-disables from 0.5.3 on. ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+
+- **Context-window-aware auto-compaction threshold (opt-in).** ([#60](https://github.com/k0valik/pi-blackhole/issues/60)) New `compactAfterRatio` and `compactReserveTokens` config keys let auto-compaction derive its trigger threshold from the **active model's context window** instead of the fixed `compactAfterTokens` default: `ratio` compacts at `floor(window × ratio)` (e.g. 0.65 → ~83k @128k, ~130k @200k, ~650k @1M), `reserve` compacts when only `window − reserve` tokens of headroom remain. The threshold is re-derived on every evaluation, so mid-session `/model` switches take effect on the next check automatically. Precedence: explicit (non-default) `compactAfterTokens` > `compactAfterRatio` > `compactReserveTokens`; when a derived knob is set and `compactAfterTokens` is unset or at its 81000 default, the default is dropped so the derived knob governs (a scaffolded/modal-written default no longer blocks derived mode). Window resolution honors the per-model `contextWindow` override, then Pi's model registry, then a 128k fallback. Env overrides: `PI_BLACKHOLE_COMPACT_AFTER_RATIO`, `PI_BLACKHOLE_COMPACT_RESERVE_TOKENS`. `/blackhole-memory` status shows the effective threshold and its basis; the settings modal exposes both knobs under Compaction (0 = not set).
+- **Window-scaled auto-compaction preset curves (out of the box).** Auto-compaction's no-knob default is no longer a fixed 81,000-token threshold: a built-in `default` preset curve derives the trigger from the active session model's context window — compact at `floor(window × ratio)`, 0.90 @ 32,768, 0.80 @ 131,072, 0.70 @ 262,144, 0.40 @ 1,048,576 (piecewise-linear between anchors, constant outside; e.g. 32k → 29,491, 128k → 104,857, 1M → ~0.42 / ~418,530, ≥1,048,576 → exactly 0.40). New `compactAfterPreset` selects the curve (default `"default"`; settings-modal select under Compaction; env `PI_BLACKHOLE_COMPACT_AFTER_PRESET`); new `compactAfterPresets` holds hand-edited preset _definitions_ — name → sorted `{ window, ratio }` anchors (`window` int > 0, `0 < ratio ≤ 1`). Same-name definitions override the built-in curve, new names extend it; invalid anchors are dropped with a warning; an unknown selected name warns once and falls back to the built-in `default`. Spec: `work_docs/proposal-ratio-presets-by-context-window.md`.
+
+### Changed
+
+- **Ledger `entryIndexById` map is cached per entry-ID list** instead of being rebuilt on every call, removing a full-branch O(n) rebuild from the per-turn consolidation and compaction trigger paths. Includes a canary test documenting why upstream OM PR #57's zero-chunk observer backoff (fix 3) is unnecessary in our architecture ([upstream OM `#57`](https://github.com/elpapi42/pi-observational-memory/pull/57)). ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+
+- **No-config auto-compaction now follows the built-in `default` preset curve (behavior change on upgrade).** Users who never set a numeric knob previously auto-compacted at a flat 81,000 tokens; they now compact at `floor(window × ratio)` under the `default` preset, whose ratio _falls_ as the window grows — small 32k/64k local windows fill to ~90% and now auto-compact mid-session where the flat 81k was unreachable, while 1M-class windows compact early (~0.40) instead of at ~8% full. Config files scaffolded or modal-written before this change that literally contain `"compactAfterTokens": 81000` are auto-migrated: exactly `81000` is treated as legacy scaffold residue (never a deliberate pin) and dropped, so the preset curve / derived knobs govern — unless the value came from the `PI_BLACKHOLE_COMPACT_AFTER_TOKENS` env var, which stays explicit — an exported env var signals deliberate intent to keep the flat threshold, so unset/remove it to take the new curve. Any other explicit value (e.g. 80,000 or 180,000) still pins a fixed threshold; the flat-81k behavior can no longer be pinned by writing exactly `81000`. Spec: `work_docs/proposal-ratio-presets-by-context-window.md`.
+
+### Fixed
+
+- **Compact-all compactions no longer silently drop every OM observation and reflection.** pi-core's compact-all sentinel (`firstKeptEntryId === ""`) resolved the projection boundary to index −1, producing an empty OM summary on single-prompt and no-user-message sessions; the OM fold now covers the whole branch up to the tip. ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+- **Consolidation is cancelled across session reloads.** Reloading or replacing a session during active consolidation could append late observer/reflector output through the stale extension instance while reflection work was lost instead of retried. Observer/reflector/dropper stages, model resolution, and deferred compaction are now guarded by a runtime generation + AbortSignal on `session_start`/`session_shutdown`, and a fresh runtime retries the work without accepting stale output ([upstream OM `#58`](https://github.com/elpapi42/pi-observational-memory/pull/58)). ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+- **Cooldown reasons no longer store HTML error pages and skip toasts no longer dump them.** Cooldown reasons are sanitized to a short `HTTP <status>` line (capped at 200 chars) so an HTML WAF block page never lands in `pi-blackhole-cooldown.json` or the skip toast, which now points at the cooldown log only. ([#80](https://github.com/k0valik/pi-blackhole/issues/80))
+- **OM workers resolve `streamSimple` through the model registry for custom providers.** Observer/reflector/dropper were hard-wired to the pi-ai compat `streamSimple`, which cannot dispatch `pi.registerProvider` streams (cursor-sdk, CLIProxyAPI, …), so custom-provider-only setups crashed after a successful turn. Resolution chain: `modelRegistry.streamSimple` (Pi [`#8964`](https://github.com/earendil-works/pi/issues/8964)) → `getRegisteredProviderConfig()` matching `model.provider`, then `model.api` → global `Symbol.for` map → compat fallback. Custom-provider-only setups can now leave `observational-memory.model` unset ([upstream OM `#60`](https://github.com/elpapi42/pi-observational-memory/pull/60), [`#30`](https://github.com/elpapi42/pi-observational-memory/issues/30)). ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+
+---
+
+## [0.5.1] - 2026-09-06
+
+### Fixed
+
+- **Host inline-compaction adapter now prefers pi's already-loaded bundled runtime chunk over its `dist/index.js` barrel.** Probe on pi 0.85.1 showed the barrel import cost 484ms (fresh graph) while the bundled chunk pi already loaded cost 5ms (cache-hit); the adapter tried the barrel first and kept iterating after a successful chunk patch. Reordered candidates so the chunk is tried first per host path, with the barrel only as a fallback for roots that never resolve a chunk, while still patching every unique host identity. Adapter total 506ms → 16ms, factory ~490ms saved.
+
+---
+
 ## [0.5.0] - 2026-09-06
 
 ### Added

package/README.md

@@ -31,20 +31,16 @@ Then `/reload` or restart Pi. The config file at `~/.pi/agent/pi-blackhole/pi-bl
 
 ## ✨ What's new
 
-> **Latest release: [0.5.0](CHANGELOG.md#050---2026-09-06)**
+> **Latest release: [0.5.2](CHANGELOG.md)**
 >
-> - **Recall & export ranking upgrade** — BM25+, SimHash64, c-TF-IDF, technical density scoring, and surface-form-preserving topic labels for sharper dedup and more readable exports; export preamble now carries a best-effort heuristic warning.
-> - **`/blackhole-export` — distilled project-memory export** — Export for long-term agent memory tools - scans all project sessions + pending buffers, fuzzy-dedupes, and writes one import-ready Markdown (`Reflections → Critical → High → Medium → Low`) with topic badges and orphan-gated pending. `out:<path>.md` supported.
-> - **Append compaction mode** (`compactionSummaryMode: "append"`) — better prompt caching - keep every auto-compaction summary as an immutable segment visible to the model (`S1 | S2 | …`) instead of rewriting a single summary. `/blackhole` rebases the chain. Opt-in.
-> - **Mid-run auto-compaction** (`midRunCompaction: "resume"` | `"pause"`) — **good for goal/task** opt into transparent compaction during long tool loops without interrupting the agent. Default is `"off"`.
-> - **Robust compaction-failure handling** — unified `session_compact_failed` (pi >=0.84.3) with correct attribution, overflow-retry visibility, and noise filtering; plus bundled-CLI `AgentSession` resolution so inline compaction works from `dist/bundle/cli.js` ([#62](https://github.com/k0valik/pi-blackhole/pull/62)).
+> - **⚠️ Auto-compaction now scales to your model's context window** — the no-knob default no longer fires at a flat 81,000 tokens: a built-in **`default` preset curve** derives the trigger as `floor(window × ratio)` — 0.90 @ 32,768, 0.80 @ 131,072, 0.70 @ 262,144, 0.40 @ 1,048,576 (piecewise-linear between anchors). Small local windows fill near-full (cheap to resend); 1M-class windows compact early. **Behavior change on upgrade:** a config file containing the literal scaffold value `"compactAfterTokens": 81000` is auto-migrated (treated as residue, never a deliberate pin) so the curve governs — any _other_ explicit value still pins a flat threshold. Prefer a different shape? `compactAfterPreset`, `compactAfterRatio`, and `compactReserveTokens` live under **Compaction** in `/blackhole settings`; thresholds re-derive on every check, so mid-session `/model` switches apply automatically. ([#79](https://github.com/k0valik/pi-blackhole/pull/79))
+> - **Observational memory now works with custom providers** — observer/reflector/dropper resolve their streams through the model registry (`pi.registerProvider` providers like cursor-sdk, CLIProxyAPI, …) instead of the built-in compat path. Custom-provider-only setups no longer crash after a turn, and `observational-memory.model` can stay unset — no second built-in provider required. ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+> - **Reload-safe memory pipeline** — reloading, forking, or switching a session mid-consolidation now cancels the in-flight observer/reflector/dropper work cleanly instead of appending through the stale session; the fresh session retries the work without losing reflections. ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+> - **Compact-all compactions keep your memory** — fixed a silent drop of _all_ observations and reflections when compaction fires on a single-prompt or no-user-message session (the compact-all sentinel resolved to an empty fold). ([#74](https://github.com/k0valik/pi-blackhole/pull/74))
+> - **Faster session startup** — the host inline-compaction adapter now prefers Pi's already-loaded bundled runtime chunk over the slow barrel import: adapter setup drops from ~506ms to ~16ms, saving ~490ms per session start.
 
 See [`CHANGELOG.md`](CHANGELOG.md) for the full history.
 
-### ⚠️ Upcoming change
-
-> **Default compaction thresholds will become model-context-window-aware** in an upcoming release. Instead of static absolute tokens (`compactAfterTokens: 81000`), default thresholds will derive from your model's effective context window — keeping the same approximate cadence regardless of model size. Existing explicitly-set values will continue to be respected verbatim. If you're using the defaults, no action is needed; the migration is automatic.
-
 ---
 
 ## What it does
@@ -103,14 +99,14 @@ The `/blackhole-recall` command exposes the same engine to the user. Results are
 
 Two modes, one shared goal: keep your agent's context sharp without manual housekeeping. (`compaction: "off"` is a third escape hatch that hands everything back to Pi.)
 
-|                             | Auto (default)                                | Manual (`compaction: "manual"`)                  | Off (`compaction: "off"`)                                  |
-| --------------------------- | --------------------------------------------- | ------------------------------------------------ | ---------------------------------------------------------- |
-| Workers run?                | Yes                                           | Yes                                              | Yes (unless `memory: false`)                               |
-| Observations go to          | Conversation markers (invisible in TUI)       | Per-session disk buffers                         | Conversation markers                                       |
-| Auto-compact on `agent_end` | Yes — blackhole fires at `compactAfterTokens` | No                                               | No (Pi handles it)                                         |
-| `/compact` (Pi built-in)    | Replaced by blackhole                         | Pi handles                                       | Pi handles                                                 |
-| `/blackhole`                | Optional                                      | **Required** to flush + compact                  | Optional, but works                                        |
-| Use case                    | "Install and forget"                          | "I want to control when context gets compressed" | "Let Pi handle it, but I want `/blackhole` when I need it" |
+|                             | Auto (default)                                                         | Manual (`compaction: "manual"`)                  | Off (`compaction: "off"`)                                  |
+| --------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------ | ---------------------------------------------------------- |
+| Workers run?                | Yes                                                                    | Yes                                              | Yes (unless `memory: false`)                               |
+| Observations go to          | Conversation markers (invisible in TUI)                                | Per-session disk buffers                         | Conversation markers                                       |
+| Auto-compact on `agent_end` | Yes — fires at the auto-compaction threshold (preset curve by default) | No                                               | No (Pi handles it)                                         |
+| `/compact` (Pi built-in)    | Replaced by blackhole                                                  | Pi handles                                       | Pi handles                                                 |
+| `/blackhole`                | Optional                                                               | **Required** to flush + compact                  | Optional, but works                                        |
+| Use case                    | "Install and forget"                                                   | "I want to control when context gets compressed" | "Let Pi handle it, but I want `/blackhole` when I need it" |
 
 Manual mode is the maintainer's daily driver: workers still run, but observations accumulate in `<sessionId>-pending.json` files instead of cluttering the conversation. `/blackhole` flushes the buffer, runs algorithmic compaction, and injects durable reflections in one shot.