pi-observational-memory 1.0.3 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,80 +1,54 @@
1
1
  # pi-observational-memory
2
2
 
3
- **Make Pi sessions feel endless.**
3
+ > **Make long sessions feel endless.** A Pi extension that keeps your agent in hour six knowing what you decided in hour one.
4
4
 
5
- Every session has a cliff. You're three hours in, the context window fills up, compaction runs, and suddenly the agent doesn't remember what you decided in hour one. You start repeating yourself. The session that was flowing now feels like a new conversation with an amnesiac.
5
+ ---
6
6
 
7
- pi-observational-memory pushes that cliff out far enough that you stop thinking about it. It replaces Pi's compaction summary with a two-tier memory system — **observations** (a timestamped, priority-tagged event log) and **reflections** (stable long-term facts) — so the agent carries forward *what* you decided, *when*, *why*, and what's already done. Not as prose that degrades with each compaction cycle, but as structured memory that stays sharp.
7
+ ## The cliff
8
8
 
9
- ```
10
- <reflections>
11
- - User works at Acme Corp, building "Acme Dashboard"
12
- - Stack: Next.js 15, Supabase auth, server components with client-side hydration
13
- - Hard constraint: ship by January 22nd 2026
14
- </reflections>
15
-
16
- <observations>
17
- Date: 2026-01-15
18
- - 🔴 14:30 User decided to switch from REST to GraphQL for the public API
19
- - 🟡 14:32 Motivation: reduce over-fetching on mobile clients
20
- - 🟢 14:35 Agent scaffolded GraphQL schema in src/schema.ts
21
- - ✅ 14:50 GraphQL migration completed — user confirmed queries working
22
- - 🔴 15:10 User wants rate limiting on all public endpoints
23
- - 🟡 15:12 Prefers token bucket algorithm, 100 req/min per API key
24
- </observations>
25
- ```
9
+ Every long AI session has a cliff.
26
10
 
27
- Hour six should feel like hour one. The agent knows who you are, what you've built together, and what's left to do.
11
+ You're three hours in. The context window fills up. Compaction runs. Suddenly the agent doesn't remember what you decided in hour one. You start repeating yourself. The session that was flowing now feels like a new conversation with an amnesiac.
28
12
 
29
- Pi's built-in compaction handles most sessions well it tracks file operations, manages split turns, and keeps recent messages intact. This extension is for the sessions where "most" isn't enough: long builds, multi-feature sprints, and the kind of deep work where breaking flow to start a new session costs you more than the tokens.
13
+ Worse, compaction often hits *mid-work*right when you were about to ask the next question. Whatever the summary captured (and didn't) becomes what the agent knows from now on.
30
14
 
31
- Inspired by [Mastra's Observational Memory](https://mastra.ai/blog/observational-memory) research (94.87% on LongMemEval). This is an independent implementation built for Pi's extension system and compaction model.
15
+ This is a universal problem for AI agents. Context windows are finite, sessions aren't, and the bridge between the two is fragile. Compactions are necessary, but they're also where memory degrades — the more you do, the further you drift from what was actually said and decided early in the session.
32
16
 
33
- ## Why this matters
17
+ ## What this gives you
34
18
 
35
- Pi's default compaction summarizes old messages into prose and tracks which files were read and modified. This works well for short-to-medium sessions. But prose summaries are inherently lossy in ways that compound over time the third compaction summarizes a summary of a summary, and specific decisions, timestamps, and completion states get flattened.
19
+ `pi-observational-memory` runs an **observer** silently in the background while you work, summarizing the conversation in ~1k token chunks into a structured event log. When compaction runs, the extension assembles that logplus stable long-term **reflections** crystallized from it into the new summary.
36
20
 
37
- Observational memory uses a different format that's designed to survive repeated compaction cycles:
21
+ What the agent sees after compaction looks like this:
38
22
 
39
- | What you get | Why it matters |
40
- |---|---|
41
- | 🔴 Priority tags | Agent knows what's important vs. what's noise |
42
- | Timestamps | Temporal reasoning agent knows *when* things happened |
43
- | Completion markers | Agent won't redo finished work |
44
- | State change tracking | "Switched from A to B" — no stale decisions |
45
- | User quotes preserved | Your exact words survive compression |
46
- | Reflections tier | Identity and constraints never get pruned |
23
+ ```
24
+ ## Reflections
25
+ User works at Acme Corp building Acme Dashboard on Next.js 15 with Supabase auth.
26
+ Hard constraint: ship by January 22nd 2026.
27
+ Public API uses GraphQL (switched from REST to reduce mobile over-fetching).
28
+
29
+ ## Observations
30
+ 2026-01-15 14:30 [high] User decided to switch from REST to GraphQL for the public API; motivation was reducing over-fetching on mobile clients.
31
+ 2026-01-15 14:35 [medium] Agent scaffolded GraphQL schema in src/schema.ts.
32
+ 2026-01-15 14:50 [medium] GraphQL migration completed; user confirmed queries working.
33
+ 2026-01-15 15:10 [critical] User wants rate limiting on all public endpoints; prefers token bucket algorithm at 100 req/min per API key.
34
+ ```
47
35
 
48
- ## How it works
36
+ Two layers of memory, two different jobs:
49
37
 
50
- Two LLM passes intercept Pi's compaction to produce structured output instead of prose:
38
+ - **Reflections** are durable patterns — who you are, what you've decided, hard constraints. Plain prose, no timestamps. They crystallize once and persist across every future compaction.
39
+ - **Observations** are timestamped events with a per-entry relevance tier (`low` / `medium` / `high` / `critical`). They're written near-real-time, then pruned over time — but never paraphrased.
51
40
 
52
- ```
53
- Raw messages accumulate
54
-
55
- ▼ exceeds observationThreshold (default: 50k tokens)
56
- ┌─────────┐
57
- │ Observer │──▶ Compresses messages into timestamped,
58
- └─────────┘ priority-tagged observations. Append-only.
59
-
60
- ▼ observations exceed reflectionThreshold (default: 30k tokens)
61
- ┌───────────┐
62
- │ Reflector │──▶ Promotes stable facts to reflections.
63
- └───────────┘ Prunes only what it's certain is dead.
64
-
65
-
66
- ┌──────────────────────────────┐
67
- │ Agent context: │
68
- │ 1. System prompt │
69
- │ 2. Reflections (stable) │
70
- │ 3. Observations (event log) │
71
- │ 4. Recent raw messages │
72
- └──────────────────────────────┘
73
- ```
41
+ Hour six should feel like hour one. The agent knows who you are, what you've built together, and what's left to do.
74
42
 
75
- The observer is aggressive it compresses everything into dense observations. The reflector is conservative — it only prunes what's clearly dead. Between reflector runs, no information is lost.
43
+ ## What you actually get from it
76
44
 
77
- For the full technical breakdown compaction lifecycle, state persistence, configuration interactions see **[docs/how-it-works.md](docs/how-it-works.md)**.
45
+ - **Continuity that survives many compactions.** The summary is built by mechanical concatenation, not an LLM rewrite. What survives one compaction survives all of them, byte-identicalthere's no compounding drift across cycles.
46
+ - **Temporal reasoning.** Every observation carries a per-minute timestamp. The agent can reason about *when* something happened, not just *that* it happened.
47
+ - **Relevance-aware pruning.** Four relevance tiers drive what gets dropped first when the observation pool grows. Trivia goes; user assertions, decisions, and verbatim errors stay.
48
+ - **Reflections that crystallize.** Identity, constraints, and durable preferences settle into a separate layer that doesn't get re-paraphrased on each compaction.
49
+ - **Predictable token cost.** Properly configured for your use case, this can save real money. The reflector + pruner only run above a configurable gate, so most compactions cost **zero LLM calls** — just bookkeeping. The observer can be pointed at a cheap fast model independently of your main coding model.
50
+ - **Cache-friendly by design.** Memory updates are batched at compaction boundaries instead of injected into every turn, so prompt prefix caching keeps working between compactions.
51
+ - **Fewer mid-work surprises.** The extension proactively triggers compaction when the agent is idle, and this will not affect your current work as after compaction you still keep the tail of your session intact.
78
52
 
79
53
  ## Install
80
54
 
@@ -88,79 +62,80 @@ Or from GitHub:
88
62
  pi install git:github.com/elpapi42/pi-observational-memory
89
63
  ```
90
64
 
91
- That's it. The extension hooks into Pi's compaction lifecycle automatically. No config file needed to startdefaults work well for most sessions.
65
+ That's it. The extension hooks into Pi's lifecycle automatically. Defaults work well for most sessionsno config file needed to start.
92
66
 
93
- ## Configuration
67
+ ## How it works (60-second version)
94
68
 
95
- ### Extension settings
69
+ Three tiers, two of them mostly asynchronous:
96
70
 
97
- Settings live under the `observational-memory` key in Pi's `settings.json` — globally at `~/.pi/agent/settings.json`, or per-project at `.pi/settings.json`. Project values override global.
71
+ ```mermaid
72
+ flowchart TD
73
+ Conv([Conversation accumulates])
74
+ Obs[Observer<br/>async, fire-and-forget<br/>compresses each chunk into timestamped,<br/>relevance-tagged observations<br/>stored as silent tree entries]
75
+ Comp[Compaction<br/>extension-owned; merges accumulated<br/>observations with prior compaction state]
76
+ RP[Reflector + Pruner<br/>Reflector appends new reflections<br/>crystallized from the pool<br/>Pruner drops observations by id<br/>across up to 5 passes]
77
+ Sum[Summary mechanically assembled<br/>## Reflections<br/>&nbsp;&nbsp;plain prose lines<br/>## Observations<br/>&nbsp;&nbsp;YYYY-MM-DD HH:MM relevance ...<br/>Becomes the compactionSummary<br/>the agent sees on the next turn]
98
78
 
99
- ```json
100
- {
101
- "observational-memory": {
102
- "observationThreshold": 50000,
103
- "reflectionThreshold": 30000
104
- }
105
- }
79
+ Conv -->|every ~1k raw tokens since last bound| Obs
80
+ Obs -->|every ~50k raw tokens since last compaction| Comp
81
+ Comp -->|observation pool ≥ 30k tokens| RP
82
+ RP --> Sum
83
+ Comp -.->|pool below gate — skip LLM calls| Sum
106
84
  ```
107
85
 
108
- | Setting | Default | What it controls |
109
- |---|---|---|
110
- | `observationThreshold` | `50,000` tokens | How much raw conversation accumulates before the observer runs |
111
- | `reflectionThreshold` | `30,000` tokens | How large observations grow before the reflector promotes and prunes |
112
- | `compactionModel` | session model | Optional — use a cheaper model for observer/reflector passes |
86
+ - **Observer** runs in the background as turns complete. The user never waits on it.
87
+ - **Compaction** is owned by the extension. The summary is *mechanically concatenated* from current reflections + current observations — never an LLM rewrite. This is what eliminates the summary-of-a-summary problem.
88
+ - **Reflector + Pruner** run as an inseparable pair, and only when there's enough material to crystallize. Below the gate, compaction does **zero LLM calls**.
113
89
 
114
- ### Using a cheaper model for compaction
90
+ The agent only ever sees the most recent compaction summary, packaged as a normal `compactionSummary` message. Observations and reflections are never injected into the live message stream — that would invalidate prefix caching with every observation. By batching memory updates at compaction boundaries, the prefix stays stable between compactions and prefix caching keeps working.
115
91
 
116
- The observer and reflector don't need the same capabilities as your coding agent. Offload them to something fast and cheap:
92
+ For the full picture, read on:
117
93
 
118
- ```json
119
- {
120
- "observational-memory": {
121
- "compactionModel": { "provider": "openrouter", "id": "google/gemma-4-31b-it" }
122
- }
123
- }
124
- ```
94
+ - **[docs/concepts.md](docs/concepts.md)** — vocabulary and mental model. Start here if you're new.
95
+ - **[docs/how-it-works.md](docs/how-it-works.md)** — the full lifecycle, data shapes, and async-race handling.
96
+ - **[docs/configuration.md](docs/configuration.md)** — every setting, what it trades off, and tuning recipes.
125
97
 
126
- ### Pi compaction settings
98
+ ## Configuration in 30 seconds
127
99
 
128
- The extension works alongside Pi's built-in compaction settings in the same `settings.json`:
100
+ Settings live in Pi's `settings.json` globally at `~/.pi/agent/settings.json` or per-project at `.pi/settings.json` (project values override global).
129
101
 
130
102
  ```json
131
103
  {
104
+ "observational-memory": {
105
+ "observationThresholdTokens": 1000,
106
+ "compactionThresholdTokens": 50000,
107
+ "reflectionThresholdTokens": 30000
108
+ },
132
109
  "compaction": {
133
110
  "keepRecentTokens": 20000
134
111
  }
135
112
  }
136
113
  ```
137
114
 
115
+ The five settings most worth knowing:
116
+
138
117
  | Setting | Default | What it controls |
139
118
  |---|---|---|
140
- | `keepRecentTokens` | `20,000` | Tokens of recent conversation kept verbatim (not summarized) |
141
- | `reserveTokens` | `16,384` | Headroom for LLM response; Pi auto-compacts when context exceeds `window - reserveTokens` |
119
+ | `observationThresholdTokens` | `1,000` | How often the observer fires in the background |
120
+ | `compactionThresholdTokens` | `50,000` | How often the extension proactively triggers compaction |
121
+ | `reflectionThresholdTokens` | `30,000` | The observation pool size at which reflector + pruner engage |
122
+ | `compactionModel` | session model | Which model runs the observer / reflector / pruner — point at a cheaper one to save cost |
123
+ | `compaction.keepRecentTokens` | `20,000` | How much recent conversation Pi keeps verbatim post-compaction (Pi setting; structural to the extension) |
142
124
 
143
- **How they interact:** Pi decides *when* to compact and *how many recent messages to keep raw*. The extension decides *how* to compact (observations + reflections instead of a flat summary) and *when to proactively trigger* compaction before the window fills. Both paths end up at the same `session_before_compact` hook.
125
+ For the full list and tuning recipes, see **[docs/configuration.md](docs/configuration.md)**.
126
+
127
+ > **Upgrading from `pi-observational-memory@1.x`?** The config keys changed: v1's `observationThreshold` is now `compactionThresholdTokens`, v1's `reflectionThreshold` is now `reflectionThresholdTokens`, and `observationThresholdTokens` is new. Old v1 keys are silently ignored — update your `settings.json`.
144
128
 
145
129
  ## Commands
146
130
 
147
- | Command | Description |
131
+ | Command | What it does |
148
132
  |---|---|
149
- | `/om-status` | Token counts, thresholds, and when the next observer/reflector pass will trigger |
150
- | `/om-view` | Print current reflections and observations |
151
- | `/om-view --full` | Same as above, plus the raw kept messages |
152
-
153
- ## Design decisions
154
-
155
- **Why observations are append-only.** Between reflector runs, nothing is lost. The observer only compresses new messages — it doesn't decide what to keep. This keeps the observer simple and predictable.
156
-
157
- **Why the reflector is conservative.** It only prunes what it's certain is dead: completed tasks no longer referenced, superseded information, exact duplicates. Being old or low-priority is not a reason to prune. If in doubt, it keeps.
158
-
159
- **Why user messages are captured near-verbatim.** When the context window shrinks, observations become the only record of what you said. Short messages are preserved exactly; long ones are summarized with key phrases quoted.
133
+ | `/om-status` | Memory totals, percent-to-threshold for each gate, and in-flight flags for observer and compaction |
134
+ | `/om-view` | Full dump of memory state: every reflection, every committed observation, every pending observation. Each observation line is `[id] YYYY-MM-DD HH:MM [relevance] content` |
160
135
 
161
- **Why state changes are explicit.** When you say "switching from A to B," the observation notes both the new state and what it replaces. This prevents the agent from acting on stale information after compaction.
136
+ ## Credits
162
137
 
163
- **Why memory lives in the session.** State is stored in Pi's session entries as compaction `details` — no external database, no filesystem state, no separate sync. On session resume, the extension walks backward through entries to restore memory. If you use `/tree` to branch, each branch gets its own memory state.
138
+ Inspired by [Mastra's Observational Memory](https://mastra.ai/blog/observational-memory) research (94.87% on LongMemEval). This is an independent implementation built for Pi's extension system.
164
139
 
165
140
  ## License
166
141
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-observational-memory",
3
- "version": "1.0.3",
3
+ "version": "2.1.0",
4
4
  "description": "Observational memory extension for pi — cache-friendly tiered compaction with observations and reflections.",
5
5
  "type": "module",
6
6
  "license": "MIT",
package/src/branch.ts ADDED
@@ -0,0 +1,190 @@
1
+ import {
2
+ OBSERVATION_CUSTOM_TYPE,
3
+ isMemoryDetails,
4
+ isObservationEntryData,
5
+ type MemoryDetails,
6
+ type ObservationEntryData,
7
+ type ObservationRecord,
8
+ type Reflection,
9
+ } from "./types.js";
10
+ import { estimateEntryTokens } from "./tokens.js";
11
+
12
+ type Entry = {
13
+ type: string;
14
+ id: string;
15
+ timestamp?: string;
16
+ message?: unknown;
17
+ content?: unknown;
18
+ customType?: string;
19
+ summary?: unknown;
20
+ fromId?: string;
21
+ data?: unknown;
22
+ details?: unknown;
23
+ firstKeptEntryId?: string;
24
+ };
25
+
26
+ const RAW_TYPES = new Set(["message", "custom_message", "branch_summary"]);
27
+
28
+ function isObservationEntry(entry: Entry): boolean {
29
+ return entry.type === "custom" && entry.customType === OBSERVATION_CUSTOM_TYPE;
30
+ }
31
+
32
+ export function findLastCompactionIndex(entries: Entry[]): number {
33
+ for (let i = entries.length - 1; i >= 0; i--) {
34
+ if (entries[i].type === "compaction") return i;
35
+ }
36
+ return -1;
37
+ }
38
+
39
+ export function lastObservationCoverEndIdx(entries: Entry[]): number {
40
+ const idToIdx = new Map<string, number>();
41
+ for (let i = 0; i < entries.length; i++) idToIdx.set(entries[i].id, i);
42
+ let maxIdx = -1;
43
+ for (let i = 0; i < entries.length; i++) {
44
+ const entry = entries[i];
45
+ if (!isObservationEntry(entry)) continue;
46
+ if (!isObservationEntryData(entry.data)) continue;
47
+ const coverIdx = idToIdx.get(entry.data.coversUpToId);
48
+ if (coverIdx !== undefined && coverIdx > maxIdx) maxIdx = coverIdx;
49
+ }
50
+ return maxIdx;
51
+ }
52
+
53
+ function rawTokensFromIndex(entries: Entry[], startIndex: number): number {
54
+ let total = 0;
55
+ for (let i = Math.max(0, startIndex); i < entries.length; i++) {
56
+ if (RAW_TYPES.has(entries[i].type)) total += estimateEntryTokens(entries[i]);
57
+ }
58
+ return total;
59
+ }
60
+
61
+ export function rawTokensSinceLastBound(entries: Entry[]): number {
62
+ return rawTokensFromIndex(entries, lastObservationCoverEndIdx(entries) + 1);
63
+ }
64
+
65
+ export function rawTokensSinceLastCompaction(entries: Entry[]): number {
66
+ const compactionIdx = findLastCompactionIndex(entries);
67
+ if (compactionIdx === -1) return rawTokensFromIndex(entries, 0);
68
+ return rawTokensFromIndex(entries, liveTailStartIndex(entries));
69
+ }
70
+
71
+ function liveTailStartIndex(entries: Entry[]): number {
72
+ const compactionIdx = findLastCompactionIndex(entries);
73
+ if (compactionIdx === -1) return 0;
74
+ const firstKept = entries[compactionIdx].firstKeptEntryId;
75
+ if (!firstKept) throw new Error("compaction entry missing firstKeptEntryId");
76
+ const firstKeptIdx = entries.findIndex((e) => e.id === firstKept);
77
+ if (firstKeptIdx === -1) throw new Error(`firstKeptEntryId "${firstKept}" not found in entries`);
78
+ return firstKeptIdx;
79
+ }
80
+
81
+ export function firstRawIdAfter(entries: Entry[], afterIndex: number): string | undefined {
82
+ for (let i = Math.max(0, afterIndex + 1); i < entries.length; i++) {
83
+ if (RAW_TYPES.has(entries[i].type)) return entries[i].id;
84
+ }
85
+ return undefined;
86
+ }
87
+
88
+ export function gapRawEntries(entries: Entry[], newFirstKeptEntryId: string): Entry[] {
89
+ const lastBoundIdx = lastObservationCoverEndIdx(entries);
90
+ const newKeptIdx = entries.findIndex((e) => e.id === newFirstKeptEntryId);
91
+ if (newKeptIdx === -1) return [];
92
+ const result: Entry[] = [];
93
+ for (let i = lastBoundIdx + 1; i < newKeptIdx; i++) {
94
+ if (RAW_TYPES.has(entries[i].type)) result.push(entries[i]);
95
+ }
96
+ return result;
97
+ }
98
+
99
+ export function rawTailEntriesBetween(entries: Entry[], fromId: string, untilId: string): Entry[] {
100
+ const fromIdx = entries.findIndex((e) => e.id === fromId);
101
+ const untilIdx = entries.findIndex((e) => e.id === untilId);
102
+ if (fromIdx === -1 || untilIdx === -1 || untilIdx < fromIdx) return [];
103
+
104
+ const result: Entry[] = [];
105
+ for (let i = fromIdx; i <= untilIdx; i++) {
106
+ if (RAW_TYPES.has(entries[i].type)) result.push(entries[i]);
107
+ }
108
+ return result;
109
+ }
110
+
111
+ function getPriorMemoryDetails(entries: Entry[]): MemoryDetails | undefined {
112
+ const idx = findLastCompactionIndex(entries);
113
+ if (idx === -1) return undefined;
114
+ const details = entries[idx].details;
115
+ return isMemoryDetails(details) ? details : undefined;
116
+ }
117
+
118
+ export function collectObservationsByCoverage(
119
+ entries: Entry[],
120
+ priorFirstKeptEntryId: string | undefined,
121
+ newFirstKeptEntryId: string,
122
+ ): ObservationEntryData[] {
123
+ const idToIdx = new Map<string, number>();
124
+ for (let i = 0; i < entries.length; i++) idToIdx.set(entries[i].id, i);
125
+
126
+ const newFKIIdx = idToIdx.get(newFirstKeptEntryId);
127
+ if (newFKIIdx === undefined) return [];
128
+
129
+ let priorFKIIdx: number;
130
+ if (priorFirstKeptEntryId === undefined) {
131
+ priorFKIIdx = -1;
132
+ } else {
133
+ const idx = idToIdx.get(priorFirstKeptEntryId);
134
+ if (idx === undefined) throw new Error(`priorFirstKeptEntryId "${priorFirstKeptEntryId}" not found in entries`);
135
+ priorFKIIdx = idx;
136
+ }
137
+
138
+ const result: ObservationEntryData[] = [];
139
+ for (const entry of entries) {
140
+ if (!isObservationEntry(entry)) continue;
141
+ if (!isObservationEntryData(entry.data)) continue;
142
+ const fromIdx = idToIdx.get(entry.data.coversFromId);
143
+ if (fromIdx === undefined) continue;
144
+ if (fromIdx >= priorFKIIdx && fromIdx < newFKIIdx) result.push(entry.data);
145
+ }
146
+ return result;
147
+ }
148
+
149
+ function collectObservationsPendingNextCompaction(entries: Entry[]): ObservationEntryData[] {
150
+ const idToIdx = new Map<string, number>();
151
+ for (let i = 0; i < entries.length; i++) idToIdx.set(entries[i].id, i);
152
+
153
+ const priorCompactionIdx = findLastCompactionIndex(entries);
154
+ let thresholdIdx: number;
155
+ if (priorCompactionIdx === -1) {
156
+ thresholdIdx = -1;
157
+ } else {
158
+ const priorFirstKept = entries[priorCompactionIdx].firstKeptEntryId;
159
+ if (!priorFirstKept) throw new Error("prior compaction entry missing firstKeptEntryId");
160
+ const idx = idToIdx.get(priorFirstKept);
161
+ if (idx === undefined) throw new Error(`prior firstKeptEntryId "${priorFirstKept}" not found in entries`);
162
+ thresholdIdx = idx;
163
+ }
164
+
165
+ const result: ObservationEntryData[] = [];
166
+ for (const entry of entries) {
167
+ if (!isObservationEntry(entry)) continue;
168
+ if (!isObservationEntryData(entry.data)) continue;
169
+ const fromIdx = idToIdx.get(entry.data.coversFromId);
170
+ if (fromIdx === undefined) continue;
171
+ if (fromIdx >= thresholdIdx) result.push(entry.data);
172
+ }
173
+ return result;
174
+ }
175
+
176
+ export interface MemoryState {
177
+ reflections: Reflection[];
178
+ committedObs: ObservationRecord[];
179
+ pendingObs: ObservationRecord[];
180
+ }
181
+
182
+ export function getMemoryState(entries: Entry[]): MemoryState {
183
+ const priorDetails = getPriorMemoryDetails(entries);
184
+ const pendingData = collectObservationsPendingNextCompaction(entries);
185
+ return {
186
+ reflections: priorDetails?.reflections ?? [],
187
+ committedObs: priorDetails?.observations ?? [],
188
+ pendingObs: pendingData.flatMap((d) => d.records),
189
+ };
190
+ }
@@ -0,0 +1,80 @@
1
+ import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
2
+ import { SettingsManager } from "@mariozechner/pi-coding-agent";
3
+ import {
4
+ getMemoryState,
5
+ rawTokensSinceLastBound,
6
+ rawTokensSinceLastCompaction,
7
+ } from "../branch.js";
8
+ import { countByRelevance, formatRelevanceHistogram } from "../relevance.js";
9
+ import type { Runtime } from "../runtime.js";
10
+ import { estimateStringTokens } from "../tokens.js";
11
+
12
+ export function registerStatusCommand(pi: ExtensionAPI, runtime: Runtime): void {
13
+ pi.registerCommand("om-status", {
14
+ description: "Show observational memory status",
15
+ handler: async (_args, ctx) => {
16
+ runtime.ensureConfig(ctx.cwd);
17
+ const entries = ctx.sessionManager.getBranch() as Parameters<typeof rawTokensSinceLastBound>[0];
18
+ const sinceBound = rawTokensSinceLastBound(entries);
19
+ const sinceCompaction = rawTokensSinceLastCompaction(entries);
20
+
21
+ const { reflections: committedRefs, committedObs, pendingObs } = getMemoryState(entries);
22
+ const committedObsTokens = committedObs.reduce((s, r) => s + estimateStringTokens(r.content), 0);
23
+ const committedObsCount = committedObs.length;
24
+ const committedRefsTokens = committedRefs.reduce((s, r) => s + estimateStringTokens(r), 0);
25
+ const committedRefsCount = committedRefs.length;
26
+
27
+ const pendingObsTokens = pendingObs.reduce((s, r) => s + estimateStringTokens(r.content), 0);
28
+ const pendingObsCount = pendingObs.length;
29
+
30
+ const relevanceHistogram = countByRelevance([...committedObs, ...pendingObs]);
31
+
32
+ const keepRecentTokens = SettingsManager.create(ctx.cwd).getCompactionKeepRecentTokens();
33
+
34
+ const obsThreshold = runtime.config.observationThresholdTokens;
35
+ const compThreshold = runtime.config.compactionThresholdTokens;
36
+ const refThreshold = runtime.config.reflectionThresholdTokens;
37
+ // Approximation: the real compaction gate excludes pending obs whose coversFromId falls inside
38
+ // the new keep-recent tail (deferred to next cycle) and adds any sync-catch-up gap obs produced
39
+ // at compaction entry. We over-count by the tail slice and can't predict the gap obs here.
40
+ // Precise version would simulate the new firstKeptEntryId by walking back keepRecentTokens from
41
+ // the branch tail and split pending into pre-tail vs tail-covering.
42
+ const observationPoolTokens = committedObsTokens + pendingObsTokens;
43
+ const obsPct = Math.min(100, Math.round((sinceBound / obsThreshold) * 100));
44
+ const compPct = Math.min(100, Math.round((sinceCompaction / compThreshold) * 100));
45
+ const refPct = Math.min(100, Math.round((observationPoolTokens / refThreshold) * 100));
46
+
47
+ const refLabel = committedRefsCount === 1 ? "entry" : "entries";
48
+ const cObsLabel = committedObsCount === 1 ? "observation" : "observations";
49
+ const pObsLabel = pendingObsCount === 1 ? "observation" : "observations";
50
+
51
+ const lines = [
52
+ "── Memory ──",
53
+ `Reflections: ~${committedRefsTokens.toLocaleString()} tokens (${committedRefsCount} ${refLabel}) — durable insights`,
54
+ `Observations:`,
55
+ ` committed ~${committedObsTokens.toLocaleString()} tokens (${committedObsCount} ${cObsLabel}) — folded into last compaction`,
56
+ ` pending ~${pendingObsTokens.toLocaleString()} tokens (${pendingObsCount} ${pObsLabel}) — waiting for next compaction`,
57
+ ` relevance ${formatRelevanceHistogram(relevanceHistogram)}`,
58
+ "",
59
+ "── Activity ──",
60
+ `Next observation: ~${sinceBound.toLocaleString()} / ${obsThreshold.toLocaleString()} tokens (${obsPct}%)`,
61
+ ` → at ${obsThreshold.toLocaleString()} tokens, recent conversation is compressed into new observations`,
62
+ `Next compaction: ~${sinceCompaction.toLocaleString()} / ${compThreshold.toLocaleString()} tokens (${compPct}%)`,
63
+ ` → at ${compThreshold.toLocaleString()} tokens, raw history is replaced by the updated reflections and`,
64
+ ` observations, keeping only the last ${keepRecentTokens.toLocaleString()} tokens of conversation verbatim`,
65
+ `Next reflection: ~${observationPoolTokens.toLocaleString()} / ${refThreshold.toLocaleString()} tokens (${refPct}%)`,
66
+ ` → if observations exceed ${refThreshold.toLocaleString()} tokens when compaction runs, reflections are`,
67
+ ` distilled from them and redundant observations are pruned away`,
68
+ ];
69
+
70
+ if (runtime.observerInFlight || runtime.compactInFlight) {
71
+ lines.push("");
72
+ lines.push("── In flight ──");
73
+ if (runtime.observerInFlight) lines.push("Observer: running");
74
+ if (runtime.compactInFlight) lines.push("Compaction: running");
75
+ }
76
+
77
+ ctx.ui.notify(lines.join("\n"), "info");
78
+ },
79
+ });
80
+ }
@@ -0,0 +1,79 @@
1
+ import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
2
+ import { getMemoryState } from "../branch.js";
3
+ import { countByRelevance, formatRelevanceHistogram } from "../relevance.js";
4
+ import type { Runtime } from "../runtime.js";
5
+ import { estimateStringTokens } from "../tokens.js";
6
+ import type { ObservationRecord } from "../types.js";
7
+
8
+ export function registerViewCommand(pi: ExtensionAPI, runtime: Runtime): void {
9
+ pi.registerCommand("om-view", {
10
+ description: "Print observational memory details (reflections + observations)",
11
+ handler: async (_args, ctx) => {
12
+ runtime.ensureConfig(ctx.cwd);
13
+ const entries = ctx.sessionManager.getBranch() as Parameters<typeof getMemoryState>[0];
14
+ const { reflections: committedRefs, committedObs, pendingObs } = getMemoryState(entries);
15
+
16
+ const committedRefTokens = committedRefs.reduce((s, r) => s + estimateStringTokens(r), 0);
17
+ const committedRefCount = committedRefs.length;
18
+
19
+ const committedObsTokens = committedObs.reduce((s, r) => s + estimateStringTokens(r.content), 0);
20
+ const committedObsCount = committedObs.length;
21
+
22
+ const pendingObsTokens = pendingObs.reduce((s, r) => s + estimateStringTokens(r.content), 0);
23
+ const pendingObsCount = pendingObs.length;
24
+
25
+ const totalObsCount = committedObsCount + pendingObsCount;
26
+ const totalTokens = committedRefTokens + committedObsTokens + pendingObsTokens;
27
+ const relevanceHistogram = countByRelevance([...committedObs, ...pendingObs]);
28
+
29
+ const plural = (n: number, singular: string, plural: string) => (n === 1 ? singular : plural);
30
+ const renderObs = (r: ObservationRecord) =>
31
+ `[${r.id}] ${r.timestamp} [${r.relevance}] ${r.content}`;
32
+
33
+ const sections: string[] = [];
34
+
35
+ sections.push(
36
+ `Memory: ${committedRefCount} ${plural(committedRefCount, "reflection", "reflections")} · ` +
37
+ `${totalObsCount} ${plural(totalObsCount, "observation", "observations")} ` +
38
+ `(${committedObsCount} committed, ${pendingObsCount} pending) · ` +
39
+ `~${totalTokens.toLocaleString()} tokens · ` +
40
+ `relevance ${formatRelevanceHistogram(relevanceHistogram)}`,
41
+ );
42
+ sections.push("");
43
+
44
+ sections.push(
45
+ `── Reflections (${committedRefCount} ${plural(committedRefCount, "entry", "entries")}, ~${committedRefTokens.toLocaleString()} tokens) ──`,
46
+ );
47
+ if (committedRefs.length > 0) {
48
+ sections.push(committedRefs.join("\n\n"));
49
+ } else {
50
+ sections.push("(none)");
51
+ }
52
+
53
+ sections.push("");
54
+ sections.push(
55
+ `── Observations — committed (${committedObsCount} ${plural(committedObsCount, "observation", "observations")}, ~${committedObsTokens.toLocaleString()} tokens) ──`,
56
+ );
57
+ if (committedObs.length > 0) {
58
+ sections.push(committedObs.map(renderObs).join("\n"));
59
+ } else {
60
+ sections.push("(none)");
61
+ }
62
+
63
+ sections.push("");
64
+ sections.push(
65
+ `── Observations — pending (${pendingObsCount} ${plural(pendingObsCount, "observation", "observations")}, ~${pendingObsTokens.toLocaleString()} tokens) ──`,
66
+ );
67
+ if (pendingObs.length > 0) {
68
+ sections.push(pendingObs.map(renderObs).join("\n"));
69
+ } else {
70
+ sections.push("(none)");
71
+ }
72
+
73
+ sections.push("");
74
+ sections.push("Tip: use /tree to browse the raw messages still live in the session.");
75
+
76
+ ctx.ui.notify(sections.join("\n"), "info");
77
+ },
78
+ });
79
+ }