pi-observational-memory-extension 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nikita Nosov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,100 @@
+# Pi Observational Memory (pi-observational-memory)
+
+A Mastra-style **Observational Memory (OM)** extension for Pi.
+
+This extension completely replaces Pi's legacy, lossy, single-block conversation summarization (`/compact`) with a transparent, highly-structured, two-layer memory model: **Mastra-style Observational Memory + Recall-driven retrieval**.
+
+By running an Observer-Reflector agent pipeline, this package allows Pi to prune old, verbose raw message history from the active context while preserving structured observations, current tasks, and continuation guidelines. When precise details are needed, the main agent uses the `om_recall` tool to fetch raw conversations from a scrollable history index.
+
+---
+
+## 🧠 Philosophy and the 3-Agent Model
+
+Legacy compaction compresses raw history into a single monolithic block of text, losing timestamps, status signals, temporal connections, and distinct user vs. agent traits. Observational Memory breaks memory down into a distinct psychological subsystem:
+
+1. **The Actor (Main Agent):** Interacts with the user, performs tool calls, and references the current Observations block. It is only presented with active observations, current-task instructions, suggested-response guidelines, and a trailing window of raw, unobserved messages.
+2. **The Observer (Extraction Agent):** Runs automatically when unobserved messages exceed a token threshold. It converts raw message blocks into chronological, priority-labeled (🔴, 🟡, 🟢, ✅) observations.
+3. **The Reflector (Consolidation Agent):** Runs when the accumulated Observations block exceeds a memory threshold. It reorganizes, cleans up, and condenses older observations into high-density facts, scaling up through 5 instruction levels (0–4) if the output size does not meet the targeted compression ratio.
+
+---
+
+## 🛠 Features
+
+- **No Silent Fallbacks:** Observer, Reflector, or model failures throw loud, descriptive errors (`Observational Memory Observer failed: ...`) rather than failing silently or leaving the context in an ambiguous state.
+- **Context Injection & Pruning:** Injects structured observations, `current-task`, `suggested-response`, retrieval guidelines, and a continuation hint into the context. Prunes observed messages from the LLM context automatically to save tokens and avoid redundant text.
+- **Async Buffering:** Extracts observations in the background at regular intervals (every 20% of the threshold) so that when the main threshold is hit, the accumulated memory is swapped/activated instantly with zero LLM-latency overhead.
+- **Adaptive Thresholds:** The active message-observation threshold expands dynamically based on remaining memory capacity in the reflection pool, maximizing raw context usage safely.
+- **Recall-driven Retrieval:** Exposes an `om_recall` tool to the Actor so it can retrieve full, raw message payloads from observed history when exact code, quotes, or numbers are needed.
+- **Custom Compaction Hook:** Plugs directly into Pi's `session_before_compact` lifecycle event. Typing `/compact` or triggering auto-compaction launches the Observer/Reflector memory consolidation flow instead of Pi's legacy summary compaction.
+- **TUI Overlay Panel:** Fully custom, responsive, multi-tab overlay (`/om-memory`) in interactive mode for inspecting memory. Features tabbed navigation, smooth scroll, and precise border framing.
+- **Durable Persistence:** Serializes and loads state files safely under `.pi/om/<session-id>.json`. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
+
+---
+
+## 📋 Configuration & Settings
+
+You can customize Observational Memory behaviors by specifying fields inside your project's `.pi/settings.json` or globally inside `~/.pi/agent/settings.json`.
+
+```json
+{
+  "compaction": {
+    "enabled": true,
+    "reserveTokens": 16384,
+    "keepRecentTokens": 20000
+  }
+}
+```
+
+- **Observation Trigger Threshold:** ~`30,000` raw message tokens.
+- **Reflection Trigger Threshold:** ~`40,000` observation tokens.
+- **Default Models:** `google/gemini-2.5-flash` with 0.3 temperature for Observer, and 0.0 temperature for Reflector.
+
+---
+
+## 🕹 Commands
+
+This extension registers the following slash commands in Pi:
+
+- `/om-status` — Shows a detailed breakdown of pending/observation tokens, active locks, thresholds, and last operation results.
+- `/om-memory` — Opens the interactive multi-tab overlay panel (observations, status stats, and background debug details).
+- `/om-observe` — Forces an immediate Observation pass on all pending raw message history.
+- `/om-reflect` — Forces an immediate Reflection/compression run on all existing observations. Supports optional guidance (e.g. `/om-reflect consolidate auth files`).
+- `/om-enable` — Enables Observational Memory for the current session.
+- `/om-disable` — Disables Observational Memory (falls back to legacy summarizing).
+- `/om-compact` — Starts the standard Pi compaction pipeline, routed through pi-observational-memory.
+
+---
+
+## 💻 TUI Overlay Panel Controls
+
+Type `/om-memory` in Pi's interactive terminal mode to open the fullscreen-ish overlay panel:
+
+- `← / →` (Arrow keys) — Switch tabs:
+  - **Memory:** Displays the active Observations list, Current Task, and Suggested Response blocks.
+  - **Status:** Shows live token meters, ratios, and model state.
+  - **Debug:** Lists background buffered chunks, filenames, state paths, and latest errors.
+- `↑ / ↓` (Arrow keys) — Scroll content.
+- `Home / End` — Jump to top/bottom of scroll.
+- `q / Esc` — Close overlay and return to chat.
+
+---
+
+## 🧪 Development and Validation
+
+This package is structured to comply with Pi's strict peer dependency models and TypeScript validation rules.
+
+### Check Types and Manifests
+Run the typechecker and validation scripts before packaging or releasing:
+```bash
+npm run typecheck
+npm run validate
+```
+
+### Commit Guidelines
+We enforce Conventional Commits. Ensure all commits use standard prefixes (`feat(om):`, `fix(tui):`, `refactor(memory):`, etc.).
+
+---
+
+## 📄 License
+
+MIT © Nikita Nosov