pi-observational-memory-extension 0.1.3 → 0.2.1

package/CHANGELOG.md

@@ -5,6 +5,62 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+---
+
+### 📖 How to Work with This Changelog
+
+When preparing a new release:
+1. **Always use Semantic Versioning** (`[Major.Minor.Patch]`).
+2. **Group changes under standard subheadings**:
+   - `Added` for new features.
+   - `Changed` for changes in existing functionality.
+   - `Deprecated` for soon-to-be removed features.
+   - `Removed` for now removed features.
+   - `Fixed` for any bug fixes.
+   - `Security` in case of vulnerabilities or security updates.
+3. **Keep descriptions concise, technical, and objective** (lowcase, founder-builder voice).
+4. **Link references or commit highlights** if appropriate.
+
+---
+
+## [0.2.1] - 2026-06-23
+
+### Fixed
+- **status**: Restored the colored OM footer metrics after reloads and session restarts.
+- **status**: Prevented failed OM states from permanently disabling future observation attempts.
+- **status**: Clamped footer percentages to readable `100%+` overflow instead of noisy `999%` values.
+- **observer**: Forced/block-after observation now processes bounded chunks instead of sending the entire backlog to the Observer.
+- **observer**: Automatically recovers from oversized stale backlogs by retaining the latest tail-window and recording a diagnostic observation.
+- **observer**: Treats empty observer output and stale reload races as non-fatal skips that advance the cursor safely.
+- **ui**: Added resilient status bootstrapping on `session_start`, `agent_start`, and `turn_start`.
+- **recovery**: Shows an explicit OM footer error if state recovery fails instead of leaving the status area empty.
+- **recovery**: Removed stdout/stderr recovery logging so internal extension diagnostics do not leak into chat transcripts.
+
+## [0.2.0] - 2026-06-23
+
+### Added
+- **retrieval**: Added vector retrieval for observations before context injection.
+- **retrieval**: Added local hash/BOW retrieval as the default offline provider.
+- **retrieval**: Added configurable Gemini embedding retrieval with `gemini-embedding-001`.
+- **scope**: Added `session` and `project` memory scopes with separate durable storage paths.
+- **attachments**: Added `auto`, `on`, and `off` attachment observation modes with image and size safeguards.
+- **caveman**: Replaced the simplified terse mode with Mastra-style caveman compression instructions.
+- **dedupe**: Added exact-hash and similarity-based duplicate suppression before appending observations.
+
+### Changed
+- **state**: Migrated OM state schema to v3 for retrieval, vectors, attachment modes, and scoped storage.
+- **settings**: Extended `/om set` with `scope`, `retrieval`, `retrieval-model`, `retrieval-top-k`, and `retrieval-threshold`.
+
+### Fixed
+- **state**: Resolved JSON state file corruption caused by running regex redactions after serialization.
+- **redaction**: Refined token/secret sanitization to avoid over-redacting non-sensitive metrics (e.g., `observationTokens`).
+- **scope**: Added durable session pointer resolution, ensuring project scope selection persists across separate run processes.
+- **durability**: Added automatic `.corrupted` rename recovery for unparseable state files instead of crashing.
+
+### Tested
+- **regression**: Added tests for scope settings, attachment gates, duplicate suppression, and local retrieval.
+- **provider**: Smoke-tested Gemini embeddings against `gemini-embedding-001` with valid 3072-dimensional normalized vectors.
+
 ## [0.1.3] - 2026-06-23
 
 ### Added

package/README.md

@@ -25,9 +25,13 @@ Legacy compaction compresses raw history into a single monolithic block of text,
 - **Async Buffering:** Extracts observations in the background at regular intervals (every 20% of the threshold) so that when the main threshold is hit, the accumulated memory is swapped/activated instantly with zero LLM-latency overhead.
 - **Adaptive Thresholds:** The active message-observation threshold expands dynamically based on remaining memory capacity in the reflection pool, maximizing raw context usage safely.
 - **Recall-driven Retrieval:** Exposes an `om_recall` tool to the Actor so it can retrieve full, raw message payloads from observed history when exact code, quotes, or numbers are needed.
+- **Vector Retrieval:** Ranks relevant observations before context injection. Local hash/BOW retrieval is enabled by default; Gemini embeddings (`gemini-embedding-001`) are available through `/om set retrieval gemini`.
+- **Session or Project Scope:** Stores memory either per session head or shared project memory via `/om set scope session|project`.
+- **Attachment Observation Modes:** Supports `auto`, `on`, and `off` with image-only auto mode and size limits to avoid unsafe prompt bloat.
+- **Duplicate Suppression:** Skips exact and near-duplicate observations before appending new memory.
 - **Custom Compaction Hook:** Plugs directly into Pi's `session_before_compact` lifecycle event. Typing `/compact` or triggering auto-compaction launches the Observer/Reflector memory consolidation flow instead of Pi's legacy summary compaction.
 - **TUI Overlay Panel:** Fully custom, responsive, multi-tab overlay (`/om-memory`) in interactive mode for inspecting memory. Features tabbed navigation, smooth scroll, and precise border framing.
-- **Durable Persistence:** Serializes and loads state files safely under `.pi/om/<session-id>.json` using atomic writes and `.bak` recovery. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
+- **Durable Persistence:** Serializes and loads state files safely under `.pi/om/sessions/<session-id>.json` or `.pi/om/projects/project.json` using atomic writes and `.bak` recovery. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
 - **Secret Redaction:** Redacts common API keys, npm/GitHub tokens, bearer tokens, passwords, and secret fields before writing state/debug artifacts or observer text.
 - **Stale Lock Recovery:** Recovers old interrupted OM operation locks automatically so a crashed or killed process does not block future memory runs.
 - **Bounded Observer Context:** Sends only a safe tail of previous observations to the Observer, preventing recursive prompt bloat while preserving full memory in the main runtime.
@@ -36,7 +40,7 @@ Legacy compaction compresses raw history into a single monolithic block of text,
 
 ## 📋 Configuration & Settings
 
-Use `/om` in Pi to inspect and update runtime settings. Settings are persisted in the session OM state file under `.pi/om/<session-id>.json`.
+Use `/om` in Pi to inspect and update runtime settings. Settings are persisted in the OM state file under `.pi/om/sessions/<session-id>.json` or `.pi/om/projects/project.json`.
 
 ```bash
 /om
@@ -48,7 +52,13 @@ Use `/om` in Pi to inspect and update runtime settings. Settings are persisted i
 /om set observation-model google/gemini-2.5-flash
 /om set reflection-model google/gemini-2.5-flash
 /om set caveman on
-/om set attachments off
+/om set attachments auto
+/om set scope project
+/om set retrieval local
+/om set retrieval gemini
+/om set retrieval-model gemini-embedding-001
+/om set retrieval-top-k 6
+/om set retrieval-threshold 12%
 /om reset
 ```
 
@@ -56,7 +66,9 @@ Use `/om` in Pi to inspect and update runtime settings. Settings are persisted i
 - **Reflection Trigger Threshold:** default `40,000` observation tokens.
 - **Default Models:** `google/gemini-2.5-flash` with 0.3 temperature for Observer, and 0.0 temperature for Reflector.
 - **Caveman Mode:** optional terse compression style for denser memory.
-- **Attachment Observation Toggle:** controls whether image/attachment placeholders are exposed to observation text.
+- **Attachment Observation Mode:** `auto` observes small images only, `on` allows supported attachments, and `off` omits them.
+- **Scope:** `session` keeps each session isolated; `project` shares one memory file for the current project.
+- **Retrieval:** `local` is the default and works offline; `gemini` uses Gemini embeddings when `GEMINI_API_KEY` or `GOOGLE_GENERATIVE_AI_API_KEY` is available; `off` disables vector retrieval.
 
 ---