pi-observational-memory-extension 0.1.1 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,40 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [0.1.3] - 2026-06-23
9
+
10
+ ### Added
11
+ - **settings**: Added the unified `/om` command with `status`, `set`, `reset`, `enable`, `disable`, `observe`, `reflect`, and `memory` actions.
12
+ - **settings**: Persisted model overrides, thresholds, caveman compression mode, attachment observation toggle, and scope configuration in OM state.
13
+ - **tests**: Added a focused regression test suite for redaction, prompt limiting, stale lock recovery, settings parsing, v1→v2 migration, and atomic writes.
14
+
15
+ ### Fixed
16
+ - **durability**: State files now use atomic temp-file writes with `.bak` recovery support.
17
+ - **stability**: Stale operation locks are recovered automatically instead of blocking future OM runs forever.
18
+ - **security**: State, debug logs, recall/observer serialization, and model outputs now redact common API keys, npm tokens, GitHub tokens, bearer tokens, passwords, and secret fields.
19
+ - **context**: Observer prompts now cap previous observations to a bounded tail to prevent observation-context bloat.
20
+
21
+ ## [0.1.2] - 2026-06-22
22
+
23
+ ### Fixed
24
+ - **om**: Resolved "stale context" errors during asynchronous `session_shutdown` handlers and background buffer tasks.
25
+ - **om**: Safely caught uncaught promise rejections within internal `bufferObservation` tasks when contexts are deactivated by Pi.
26
+
27
+ ## [0.1.1] - 2026-06-22
28
+
29
+ ### Added
30
+ - **om**: Added capability to force immediate observational compaction on `session_shutdown` if pending message tokens exist. This guarantees subagent and short-lived loopflow sessions persist their memories before exiting.
31
+
32
+ ## [0.1.0] - 2026-06-22
33
+
34
+ ### Added
35
+ - **om**: First release of the Mastra-style **Observational Memory (OM)** extension.
36
+ - **om**: Implemented the 3-Agent psychological memory model: Actor (Main Agent), Observer (Extraction Agent), and Reflector (Consolidation/Compression Agent).
37
+ - **om**: Automatically intercepts `/compact` and `session_before_compact` events.
38
+ - **om**: Real-time TUI status bar panel underneath the input showing memory, message, and token thresholds.
39
+ - **om**: Color-coded, responsive status indicators for high (🔴), medium (🟡), and low (🟢) priority observations.
40
+ - **om**: Dedicated interactive, fullscreen overlay `/om-memory` with interactive tab switching (`Memory`, `Status`, `Debug`) and ANSI word wrapping.
package/README.md CHANGED
@@ -27,27 +27,36 @@ Legacy compaction compresses raw history into a single monolithic block of text,
27
27
  - **Recall-driven Retrieval:** Exposes an `om_recall` tool to the Actor so it can retrieve full, raw message payloads from observed history when exact code, quotes, or numbers are needed.
28
28
  - **Custom Compaction Hook:** Plugs directly into Pi's `session_before_compact` lifecycle event. Typing `/compact` or triggering auto-compaction launches the Observer/Reflector memory consolidation flow instead of Pi's legacy summary compaction.
29
29
  - **TUI Overlay Panel:** Fully custom, responsive, multi-tab overlay (`/om-memory`) in interactive mode for inspecting memory. Features tabbed navigation, smooth scroll, and precise border framing.
30
- - **Durable Persistence:** Serializes and loads state files safely under `.pi/om/<session-id>.json`. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
30
+ - **Durable Persistence:** Serializes and loads state files safely under `.pi/om/<session-id>.json` using atomic writes and `.bak` recovery. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
31
+ - **Secret Redaction:** Redacts common API keys, npm/GitHub tokens, bearer tokens, passwords, and secret fields before writing state/debug artifacts or observer text.
32
+ - **Stale Lock Recovery:** Recovers old interrupted OM operation locks automatically so a crashed or killed process does not block future memory runs.
33
+ - **Bounded Observer Context:** Sends only a safe tail of previous observations to the Observer, preventing recursive prompt bloat while preserving full memory in the main runtime.
31
34
 
32
35
  ---
33
36
 
34
37
  ## 📋 Configuration & Settings
35
38
 
36
- You can customize Observational Memory behaviors by specifying fields inside your project's `.pi/settings.json` or globally inside `~/.pi/agent/settings.json`.
39
+ Use `/om` in Pi to inspect and update runtime settings. Settings are persisted in the session OM state file under `.pi/om/<session-id>.json`.
37
40
 
38
- ```json
39
- {
40
- "compaction": {
41
- "enabled": true,
42
- "reserveTokens": 16384,
43
- "keepRecentTokens": 20000
44
- }
45
- }
41
+ ```bash
42
+ /om
43
+ /om set observation-threshold 30000
44
+ /om set reflection-threshold 40000
45
+ /om set block-after 36000
46
+ /om set buffer-tokens 6000
47
+ /om set buffer-activation 80%
48
+ /om set observation-model google/gemini-2.5-flash
49
+ /om set reflection-model google/gemini-2.5-flash
50
+ /om set caveman on
51
+ /om set attachments off
52
+ /om reset
46
53
  ```
47
54
 
48
- - **Observation Trigger Threshold:** ~`30,000` raw message tokens.
49
- - **Reflection Trigger Threshold:** ~`40,000` observation tokens.
55
+ - **Observation Trigger Threshold:** default `30,000` raw message tokens.
56
+ - **Reflection Trigger Threshold:** default `40,000` observation tokens.
50
57
  - **Default Models:** `google/gemini-2.5-flash` with 0.3 temperature for Observer, and 0.0 temperature for Reflector.
58
+ - **Caveman Mode:** optional terse compression style for denser memory.
59
+ - **Attachment Observation Toggle:** controls whether image/attachment placeholders are exposed to observation text.
51
60
 
52
61
  ---
53
62
 
@@ -55,6 +64,7 @@ You can customize Observational Memory behaviors by specifying fields inside you
55
64
 
56
65
  This extension registers the following slash commands in Pi:
57
66
 
67
+ - `/om` — Unified settings/status command. Supports `set`, `reset`, `enable`, `disable`, `observe`, `reflect`, and `memory`.
58
68
  - `/om-status` — Shows a detailed breakdown of pending/observation tokens, active locks, thresholds, and last operation results.
59
69
  - `/om-memory` — Opens the interactive multi-tab overlay panel (observations, status stats, and background debug details).
60
70
  - `/om-observe` — Forces an immediate Observation pass on all pending raw message history.
@@ -88,6 +98,7 @@ Run the typechecker and validation scripts before packaging or releasing:
88
98
  ```bash
89
99
  npm run typecheck
90
100
  npm run validate
101
+ npm test
91
102
  ```
92
103
 
93
104
  ### Commit Guidelines
@@ -7,12 +7,12 @@ import {
7
7
  } from "@earendil-works/pi-coding-agent";
8
8
  import { Key, matchesKey, truncateToWidth, wrapTextWithAnsi } from "@earendil-works/pi-tui";
9
9
  import { Type } from "typebox";
10
- import { mkdir, readFile, writeFile } from "node:fs/promises";
10
+ import { copyFile, mkdir, readFile, rename, unlink, writeFile } from "node:fs/promises";
11
11
  import { existsSync } from "node:fs";
12
12
  import { basename, dirname, join } from "node:path";
13
13
 
14
14
  const EXTENSION_ID = "pi-observational-memory";
15
- const STATE_VERSION = 1;
15
+ const STATE_VERSION = 2;
16
16
  const DEFAULT_OBSERVATION_MODEL = "google/gemini-2.5-flash";
17
17
  const DEFAULT_REFLECTION_MODEL = "google/gemini-2.5-flash";
18
18
  const OBSERVATION_THRESHOLD = 30_000;
@@ -25,6 +25,9 @@ const TOOL_RESULT_MAX_CHARS = 8_000;
25
25
  const MESSAGE_PART_MAX_CHARS = 20_000;
26
26
  const MAX_OBSERVATION_LINE_CHARS = 10_000;
27
27
  const MAX_RESTART_ERRORS = 3;
28
+ const STALE_OPERATION_LOCK_MS = 15 * 60 * 1000;
29
+ const PREVIOUS_OBSERVATIONS_MAX_TOKENS = 2_000;
30
+ const ATOMIC_BACKUP_SUFFIX = ".bak";
28
31
 
29
32
  const OBSERVATION_CONTEXT_PROMPT = `The following observations block contains your memory of past conversations with this user.`;
30
33
  const OBSERVATION_CONTEXT_INSTRUCTIONS = `IMPORTANT: When responding, reference specific details from these observations. Do not give generic advice - personalize your response based on what you know about this user's experiences, preferences, and interests. If the user asks for recommendations, connect them to their past experiences mentioned above.
@@ -165,7 +168,8 @@ const OBSERVER_GUIDELINES = `- Be specific enough for the assistant to act on
165
168
  - Observe WHAT the agent did and WHAT it means
166
169
  - If the user provides detailed messages or code snippets, observe all important details`;
167
170
 
168
- function buildObserverSystemPrompt(): string {
171
+ function buildObserverSystemPrompt(caveman = false): string {
172
+ const cavemanInstruction = caveman ? `\n\nCAVEMAN MODE: Write brutally short, dense observations. Remove filler. Preserve facts, decisions, dates, paths, and errors only.` : "";
169
173
  return `You are the memory consciousness of an AI assistant. Your observations will be the ONLY information the assistant has about past interactions with this user.
170
174
 
171
175
  Extract observations that will help the assistant remember:
@@ -188,20 +192,22 @@ Do NOT add thread identifiers, thread IDs, or <thread> tags to your observations
188
192
 
189
193
  Remember: These observations are the assistant's ONLY memory. Make them count.
190
194
 
191
- User messages are extremely important. If the user asks a question or gives a new task, make it clear in <current-task> that this is the priority. If the assistant needs to respond to the user, indicate in <suggested-response> that it should pause for user reply before continuing other tasks.`;
195
+ User messages are extremely important. If the user asks a question or gives a new task, make it clear in <current-task> that this is the priority. If the assistant needs to respond to the user, indicate in <suggested-response> that it should pause for user reply before continuing other tasks.${cavemanInstruction}`;
192
196
  }
193
197
 
194
198
  function buildObserverTaskPrompt(existingObservations: string | undefined, opts: { priorCurrentTask?: string; priorSuggestedResponse?: string; wasTruncated?: boolean } = {}): string {
195
199
  let prompt = "";
196
- if (existingObservations?.trim()) {
197
- prompt += `## Previous Observations\n\n${existingObservations}\n\n---\n\nDo not repeat these existing observations. Your new observations will be appended to the existing observations.\n\n`;
200
+ const limitedExisting = limitTextByTokens(existingObservations, PREVIOUS_OBSERVATIONS_MAX_TOKENS);
201
+ const previousWasTruncated = Boolean(existingObservations?.trim()) && limitedExisting !== existingObservations;
202
+ if (limitedExisting?.trim()) {
203
+ prompt += `## Previous Observations\n\n${limitedExisting}\n\n---\n\nDo not repeat these existing observations. Your new observations will be appended to the existing observations. Previous observations may be truncated to the most recent/relevant tail for context budget safety.\n\n`;
198
204
  }
199
205
  const metadata: string[] = [];
200
206
  if (opts.priorCurrentTask) metadata.push(`- prior current-task: ${opts.priorCurrentTask}`);
201
207
  if (opts.priorSuggestedResponse) metadata.push(`- prior suggested-response: ${opts.priorSuggestedResponse}`);
202
208
  if (metadata.length) {
203
209
  prompt += `## Prior Thread Metadata\n\n${metadata.join("\n")}\n\n`;
204
- if (opts.wasTruncated) {
210
+ if (opts.wasTruncated || previousWasTruncated) {
205
211
  prompt += `Previous observations were truncated for context budget reasons. The main agent still has full memory context outside this observer window.\n`;
206
212
  }
207
213
  prompt += `Use prior current-task and suggested-response as continuity hints, then update them based on the new messages.\n\n---\n\n`;
@@ -210,7 +216,8 @@ function buildObserverTaskPrompt(existingObservations: string | undefined, opts:
210
216
  return prompt;
211
217
  }
212
218
 
213
- function buildReflectorSystemPrompt(): string {
219
+ function buildReflectorSystemPrompt(caveman = false): string {
220
+ const cavemanInstruction = caveman ? `\n\nCAVEMAN MODE: Compress aggressively. Prefer terse facts over prose. Preserve only actionable, durable memory.` : "";
214
221
  return `You are the memory consciousness of an AI assistant. Your memory observation reflections will be the ONLY information the assistant has about past interactions with this user.
215
222
 
216
223
  The following instructions were given to another part of your psyche (the observer) to create memories.
@@ -258,7 +265,7 @@ State current task(s) explicitly: primary and secondary pending tasks. Mark wait
258
265
  Hint for the agent's immediate next message.
259
266
  </suggested-response>
260
267
 
261
- User messages are extremely important. If the user asks a question or gives a new task, make it clear in <current-task> that this is the priority. If the assistant needs to respond, indicate in <suggested-response> that it should pause for user reply before continuing other tasks.`;
268
+ User messages are extremely important. If the user asks a question or gives a new task, make it clear in <current-task> that this is the priority. If the assistant needs to respond, indicate in <suggested-response> that it should pause for user reply before continuing other tasks.${cavemanInstruction}`;
262
269
  }
263
270
 
264
271
  type CompressionLevel = 0 | 1 | 2 | 3 | 4;
@@ -292,6 +299,13 @@ type BufferedChunk = {
292
299
  createdAt: string;
293
300
  };
294
301
 
302
+ type PiOMSettings = {
303
+ observationModel: string;
304
+ reflectionModel: string;
305
+ caveman: boolean;
306
+ observeAttachments: boolean;
307
+ };
308
+
295
309
  type PiOMRecord = {
296
310
  version: number;
297
311
  enabled: boolean;
@@ -307,6 +321,7 @@ type PiOMRecord = {
307
321
  pendingMessageTokens: number;
308
322
  observationTokens: number;
309
323
  thresholds: { observation: number; reflection: number; blockAfter: number; bufferTokens: number; bufferActivation: number };
324
+ settings: PiOMSettings;
310
325
  buffered: { observations: BufferedChunk[]; reflection?: BufferedChunk };
311
326
  operationLock?: { type: OperationType; startedAt: string };
312
327
  lastOperation?: { type: OperationType; startedAt: string; endedAt?: string; inputTokens: number; outputTokens?: number; error?: string; model?: string; compressionLevel?: number };
@@ -380,7 +395,13 @@ export default function (pi: ExtensionAPI) {
380
395
  runtime.currentOperation?.abort();
381
396
  if (runtime.state) {
382
397
  try {
383
- if (runtime.state.enabled && runtime.state.pendingMessageTokens > 0) {
398
+ let isStale = false;
399
+ try {
400
+ const _ = ctx.cwd;
401
+ } catch {
402
+ isStale = true;
403
+ }
404
+ if (!isStale && runtime.state.enabled && runtime.state.pendingMessageTokens > 0) {
384
405
  await observeNow(ctx, { force: true, reason: "session_shutdown" });
385
406
  }
386
407
  } catch (error) {
@@ -388,8 +409,12 @@ export default function (pi: ExtensionAPI) {
388
409
  }
389
410
  await saveState(runtime.state);
390
411
  }
391
- ctx.ui.setStatus("om", undefined);
392
- ctx.ui.setWidget("om", undefined);
412
+ try {
413
+ ctx.ui.setStatus("om", undefined);
414
+ ctx.ui.setWidget("om", undefined);
415
+ } catch (e) {
416
+ // Ignore if UI or context is already stale
417
+ }
393
418
  });
394
419
 
395
420
  pi.on("model_select", async (event, ctx) => {
@@ -555,6 +580,71 @@ export default function (pi: ExtensionAPI) {
555
580
  },
556
581
  });
557
582
 
583
+ pi.registerCommand("om", {
584
+ description: "Manage Observational Memory settings. Usage: /om, /om set <key> <value>, /om enable|disable|observe|reflect|memory",
585
+ handler: async (args, ctx) => {
586
+ const state = await ensureState(ctx);
587
+ const input = (args || "").trim();
588
+ if (!input || input === "status") {
589
+ await refreshCounts(ctx);
590
+ ctx.ui.notify(formatSettingsText(state), state.status === "failed" ? "error" : "info");
591
+ updateStatus(ctx);
592
+ return;
593
+ }
594
+ const [cmd, ...rest] = input.split(/\s+/);
595
+ if (cmd === "enable") {
596
+ state.enabled = true;
597
+ state.status = "idle";
598
+ await saveState(state);
599
+ updateStatus(ctx);
600
+ ctx.ui.notify("Observational Memory enabled", "info");
601
+ return;
602
+ }
603
+ if (cmd === "disable") {
604
+ state.enabled = false;
605
+ state.status = "disabled";
606
+ await saveState(state);
607
+ updateStatus(ctx);
608
+ ctx.ui.notify("Observational Memory disabled", "warning");
609
+ return;
610
+ }
611
+ if (cmd === "observe") {
612
+ await observeNow(ctx, { force: true, reason: "manual" });
613
+ updateStatus(ctx);
614
+ ctx.ui.notify("OM observation complete", "info");
615
+ return;
616
+ }
617
+ if (cmd === "reflect") {
618
+ await reflectNow(ctx, { reason: "manual", manualPrompt: rest.join(" ") || undefined });
619
+ updateStatus(ctx);
620
+ ctx.ui.notify("OM reflection complete", "info");
621
+ return;
622
+ }
623
+ if (cmd === "memory") {
624
+ ctx.ui.notify(formatMemoryText(state), "info");
625
+ return;
626
+ }
627
+ if (cmd === "set") {
628
+ const key = rest[0];
629
+ const value = rest.slice(1).join(" ");
630
+ applySetting(state, key, value);
631
+ await saveState(state);
632
+ updateStatus(ctx);
633
+ ctx.ui.notify(`OM setting updated: ${key}=${value}`, "info");
634
+ return;
635
+ }
636
+ if (cmd === "reset") {
637
+ state.thresholds = defaultThresholds();
638
+ state.settings = defaultSettings();
639
+ await saveState(state);
640
+ updateStatus(ctx);
641
+ ctx.ui.notify("OM settings reset to defaults", "info");
642
+ return;
643
+ }
644
+ throw new Error(`Unknown /om command: ${cmd}. Use /om, /om set <key> <value>, /om enable|disable|observe|reflect|memory|reset`);
645
+ },
646
+ });
647
+
558
648
  pi.registerCommand("om-compact", {
559
649
  description: "Run Pi compaction; pi-observational-memory will replace the summary with OM.",
560
650
  handler: async (args, ctx) => {
@@ -568,6 +658,14 @@ export default function (pi: ExtensionAPI) {
568
658
  }
569
659
 
570
660
  async function safeBoundary(ctx: any, boundary: string): Promise<void> {
661
+ let isStale = false;
662
+ try {
663
+ const _ = ctx.cwd;
664
+ } catch {
665
+ isStale = true;
666
+ }
667
+ if (isStale) return;
668
+
571
669
  const state = await ensureState(ctx);
572
670
  if (!state.enabled || state.status === "failed") return;
573
671
  await refreshCounts(ctx);
@@ -586,17 +684,40 @@ async function safeBoundary(ctx: any, boundary: string): Promise<void> {
586
684
  }
587
685
  if (shouldBuffer(state)) {
588
686
  void bufferObservation(ctx, `${boundary}:buffer`).catch(async (error) => {
589
- const s = await ensureState(ctx);
590
- s.status = "failed";
591
- s.lastError = `OM buffer failed: ${errorMessage(error)}`;
592
- await saveState(s);
593
- updateStatus(ctx);
594
- ctx.ui.notify(s.lastError, "error");
687
+ try {
688
+ let innerStale = false;
689
+ try {
690
+ const _ = ctx.cwd;
691
+ } catch {
692
+ innerStale = true;
693
+ }
694
+ if (innerStale) return;
695
+
696
+ const s = await ensureState(ctx);
697
+ s.status = "failed";
698
+ s.lastError = `OM buffer failed: ${errorMessage(error)}`;
699
+ await saveState(s);
700
+ updateStatus(ctx);
701
+ ctx.ui.notify(s.lastError, "error");
702
+ } catch (innerError) {
703
+ // Prevent any uncaught exceptions from background handler
704
+ }
595
705
  });
596
706
  }
597
707
  }
598
708
 
599
709
  async function ensureState(ctx: any): Promise<PiOMRecord> {
710
+ let isStale = false;
711
+ try {
712
+ const _ = ctx.cwd;
713
+ } catch {
714
+ isStale = true;
715
+ }
716
+ if (isStale) {
717
+ if (runtime.state) return runtime.state;
718
+ throw new Error("Cannot ensure state because the extension context is stale.");
719
+ }
720
+
600
721
  if (runtime.state && runtime.state.cwd === ctx.cwd && runtime.state.sessionFile === ctx.sessionManager.getSessionFile()) return runtime.state;
601
722
  const sessionFile = ctx.sessionManager.getSessionFile?.();
602
723
  const sessionId = ctx.sessionManager.getSessionId?.() || (sessionFile ? basename(sessionFile, ".jsonl") : "in-memory");
@@ -609,48 +730,141 @@ async function ensureState(ctx: any): Promise<PiOMRecord> {
609
730
  let state: PiOMRecord | undefined;
610
731
  if (existsSync(statePath)) {
611
732
  try {
612
- state = JSON.parse(await readFile(statePath, "utf8")) as PiOMRecord;
733
+ state = normalizeState(JSON.parse(await readFile(statePath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
613
734
  } catch (error) {
614
- throw new Error(`Failed to load OM state ${statePath}: ${errorMessage(error)}`);
735
+ const backupPath = `${statePath}${ATOMIC_BACKUP_SUFFIX}`;
736
+ if (existsSync(backupPath)) {
737
+ try {
738
+ state = normalizeState(JSON.parse(await readFile(backupPath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
739
+ } catch {
740
+ throw new Error(`Failed to load OM state ${statePath} and backup ${backupPath}: ${errorMessage(error)}`);
741
+ }
742
+ } else {
743
+ throw new Error(`Failed to load OM state ${statePath}: ${errorMessage(error)}`);
744
+ }
615
745
  }
616
746
  }
617
- if (!state || state.version !== STATE_VERSION) {
618
- state = {
619
- version: STATE_VERSION,
620
- enabled: true,
621
- sessionId,
622
- sessionFile,
623
- cwd: ctx.cwd,
624
- scope: "session",
625
- status: "idle",
626
- observations: "",
627
- pendingMessageTokens: 0,
628
- observationTokens: 0,
629
- thresholds: {
630
- observation: OBSERVATION_THRESHOLD,
631
- reflection: REFLECTION_THRESHOLD,
632
- blockAfter: Math.round(OBSERVATION_THRESHOLD * DEFAULT_BLOCK_AFTER_MULTIPLIER),
633
- bufferTokens: Math.round(OBSERVATION_THRESHOLD * BUFFER_TOKENS_RATIO),
634
- bufferActivation: BUFFER_ACTIVATION_RATIO,
635
- },
636
- buffered: { observations: [] },
637
- updatedAt: new Date().toISOString(),
638
- };
747
+ if (!state) {
748
+ state = createDefaultState({ sessionId, sessionFile, cwd: ctx.cwd });
639
749
  await saveState(state);
640
750
  }
751
+ const recovered = recoverStaleOperationLock(state);
641
752
  runtime.state = state;
753
+ if (recovered || state.version !== STATE_VERSION) await saveState(state);
642
754
  return state;
643
755
  }
644
756
 
757
+ function defaultSettings(): PiOMSettings {
758
+ return { observationModel: DEFAULT_OBSERVATION_MODEL, reflectionModel: DEFAULT_REFLECTION_MODEL, caveman: false, observeAttachments: true };
759
+ }
760
+
761
+ function defaultThresholds(): PiOMRecord["thresholds"] {
762
+ return {
763
+ observation: OBSERVATION_THRESHOLD,
764
+ reflection: REFLECTION_THRESHOLD,
765
+ blockAfter: Math.round(OBSERVATION_THRESHOLD * DEFAULT_BLOCK_AFTER_MULTIPLIER),
766
+ bufferTokens: Math.round(OBSERVATION_THRESHOLD * BUFFER_TOKENS_RATIO),
767
+ bufferActivation: BUFFER_ACTIVATION_RATIO,
768
+ };
769
+ }
770
+
771
+ function createDefaultState(identity: { sessionId: string; sessionFile?: string; cwd: string }): PiOMRecord {
772
+ return {
773
+ version: STATE_VERSION,
774
+ enabled: true,
775
+ sessionId: identity.sessionId,
776
+ sessionFile: identity.sessionFile,
777
+ cwd: identity.cwd,
778
+ scope: "session",
779
+ status: "idle",
780
+ observations: "",
781
+ pendingMessageTokens: 0,
782
+ observationTokens: 0,
783
+ thresholds: defaultThresholds(),
784
+ settings: defaultSettings(),
785
+ buffered: { observations: [] },
786
+ updatedAt: new Date().toISOString(),
787
+ };
788
+ }
789
+
790
+ function normalizeState(raw: any, identity: { sessionId: string; sessionFile?: string; cwd: string }): PiOMRecord {
791
+ const defaults = createDefaultState(identity);
792
+ const state = {
793
+ ...defaults,
794
+ ...raw,
795
+ version: STATE_VERSION,
796
+ sessionId: raw?.sessionId || identity.sessionId,
797
+ sessionFile: raw?.sessionFile || identity.sessionFile,
798
+ cwd: raw?.cwd || identity.cwd,
799
+ thresholds: { ...defaults.thresholds, ...(raw?.thresholds ?? {}) },
800
+ settings: { ...defaults.settings, ...(raw?.settings ?? {}) },
801
+ buffered: { observations: [], ...(raw?.buffered ?? {}) },
802
+ } as PiOMRecord;
803
+ state.observations = redactSecrets(String(state.observations ?? ""));
804
+ state.currentTask = state.currentTask ? redactSecrets(state.currentTask) : undefined;
805
+ state.suggestedResponse = state.suggestedResponse ? redactSecrets(state.suggestedResponse) : undefined;
806
+ state.lastError = state.lastError ? redactSecrets(state.lastError) : undefined;
807
+ state.observationTokens = estimateTokens(state.observations);
808
+ return state;
809
+ }
810
+
811
+ function isStaleOperationLock(lock: PiOMRecord["operationLock"], now = Date.now()): boolean {
812
+ if (!lock) return false;
813
+ const started = Date.parse(lock.startedAt);
814
+ return Number.isFinite(started) && now - started > STALE_OPERATION_LOCK_MS;
815
+ }
816
+
817
+ function recoverStaleOperationLock(state: PiOMRecord, now = Date.now()): boolean {
818
+ if (!isStaleOperationLock(state.operationLock, now)) return false;
819
+ const lock = state.operationLock!;
820
+ state.lastOperation = { type: lock.type, startedAt: lock.startedAt, endedAt: new Date(now).toISOString(), inputTokens: 0, error: `Recovered stale OM operation lock after ${Math.round((now - Date.parse(lock.startedAt)) / 1000)}s` };
821
+ state.lastError = state.lastOperation.error;
822
+ state.operationLock = undefined;
823
+ state.status = state.enabled ? "idle" : "disabled";
824
+ return true;
825
+ }
826
+
645
827
  async function saveState(state: PiOMRecord): Promise<void> {
646
828
  if (!runtime.statePath) return;
647
829
  state.updatedAt = new Date().toISOString();
648
- await mkdir(dirname(runtime.statePath), { recursive: true });
649
- await writeFile(runtime.statePath, JSON.stringify(state, null, 2) + "\n", "utf8");
830
+ state.observations = redactSecrets(state.observations);
831
+ if (state.currentTask) state.currentTask = redactSecrets(state.currentTask);
832
+ if (state.suggestedResponse) state.suggestedResponse = redactSecrets(state.suggestedResponse);
833
+ if (state.lastError) state.lastError = redactSecrets(state.lastError);
834
+ await atomicWriteJson(runtime.statePath, state);
650
835
  runtime.overlayHandle?.requestRender();
651
836
  }
652
837
 
838
+ async function atomicWriteJson(filePath: string, value: unknown): Promise<void> {
839
+ await mkdir(dirname(filePath), { recursive: true });
840
+ const tmpPath = `${filePath}.${process.pid}.${Date.now()}.tmp`;
841
+ const backupPath = `${filePath}${ATOMIC_BACKUP_SUFFIX}`;
842
+ const json = redactSecrets(JSON.stringify(value, null, 2)) + "\n";
843
+ await writeFile(tmpPath, json, "utf8");
844
+ if (existsSync(filePath)) {
845
+ try {
846
+ await copyFile(filePath, backupPath);
847
+ } catch {
848
+ // Backup is best-effort; rename below is the durability boundary.
849
+ }
850
+ }
851
+ try {
852
+ await rename(tmpPath, filePath);
853
+ } catch (error) {
854
+ try { await unlink(tmpPath); } catch {}
855
+ throw error;
856
+ }
857
+ }
858
+
653
859
  async function refreshCounts(ctx: any): Promise<void> {
860
+ let isStale = false;
861
+ try {
862
+ const _ = ctx.cwd;
863
+ } catch {
864
+ isStale = true;
865
+ }
866
+ if (isStale) return;
867
+
654
868
  const state = await ensureState(ctx);
655
869
  const branch = ctx.sessionManager.getBranch() as SessionEntry[];
656
870
  const pending = entriesAfter(branch, state.lastObservedEntryId).filter(isMessageLikeEntry);
@@ -662,11 +876,23 @@ async function refreshCounts(ctx: any): Promise<void> {
662
876
  function updateStatus(ctx: any): void {
663
877
  const state = runtime.state;
664
878
  if (!state) return;
665
- ctx.ui.setStatus("om", formatShortStatusColored(state));
879
+ try {
880
+ ctx.ui.setStatus("om", formatShortStatusColored(state));
881
+ } catch (e) {
882
+ // Context might be stale or UI disposed
883
+ }
666
884
  runtime.overlayHandle?.requestRender();
667
885
  }
668
886
 
669
887
  async function observeNow(ctx: any, opts: { force: boolean; reason: string; signal?: AbortSignal; manualText?: string }): Promise<void> {
888
+ let isStale = false;
889
+ try {
890
+ const _ = ctx.cwd;
891
+ } catch {
892
+ isStale = true;
893
+ }
894
+ if (isStale) return;
895
+
670
896
  const state = await ensureState(ctx);
671
897
  assertNoOperation(state);
672
898
  const branch = ctx.sessionManager.getBranch() as SessionEntry[];
@@ -742,7 +968,7 @@ async function bufferObservation(ctx: any, reason: string): Promise<void> {
742
968
  });
743
969
  state.operationLock = undefined;
744
970
  state.status = "idle";
745
- state.lastOperation = { type: "buffer", startedAt, endedAt: new Date().toISOString(), inputTokens: estimateTokens(inputText), outputTokens: estimateTokens(observations), model: DEFAULT_OBSERVATION_MODEL };
971
+ state.lastOperation = { type: "buffer", startedAt, endedAt: new Date().toISOString(), inputTokens: estimateTokens(inputText), outputTokens: estimateTokens(observations), model: state.settings.observationModel || DEFAULT_OBSERVATION_MODEL };
746
972
  await writeDebug(ctx, "buffer", { startedAt, reason, inputText, rawOutput: result.rawOutput, parsed: result });
747
973
  await saveState(state);
748
974
  } catch (error) {
@@ -758,6 +984,14 @@ async function bufferObservation(ctx: any, reason: string): Promise<void> {
758
984
  }
759
985
 
760
986
  async function activateBuffered(ctx: any, reason: string): Promise<void> {
987
+ let isStale = false;
988
+ try {
989
+ const _ = ctx.cwd;
990
+ } catch {
991
+ isStale = true;
992
+ }
993
+ if (isStale) return;
994
+
761
995
  const state = await ensureState(ctx);
762
996
  if (state.buffered.observations.length === 0) return;
763
997
  assertNoOperation(state);
@@ -794,6 +1028,14 @@ async function activateBuffered(ctx: any, reason: string): Promise<void> {
794
1028
  }
795
1029
 
796
1030
  async function reflectNow(ctx: any, opts: { reason: string; signal?: AbortSignal; manualPrompt?: string }): Promise<void> {
1031
+ let isStale = false;
1032
+ try {
1033
+ const _ = ctx.cwd;
1034
+ } catch {
1035
+ isStale = true;
1036
+ }
1037
+ if (isStale) return;
1038
+
797
1039
  const state = await ensureState(ctx);
798
1040
  assertNoOperation(state);
799
1041
  if (!state.observations.trim()) throw new Error("No observations to reflect");
@@ -826,7 +1068,7 @@ async function reflectNow(ctx: any, opts: { reason: string; signal?: AbortSignal
826
1068
  state.operationLock = undefined;
827
1069
  state.status = "idle";
828
1070
  state.lastError = undefined;
829
- state.lastOperation = { type: "reflection", startedAt, endedAt: new Date().toISOString(), inputTokens: originalTokens, outputTokens: reflectedTokens, model: DEFAULT_REFLECTION_MODEL, compressionLevel: level };
1071
+ state.lastOperation = { type: "reflection", startedAt, endedAt: new Date().toISOString(), inputTokens: originalTokens, outputTokens: reflectedTokens, model: state.settings.reflectionModel || DEFAULT_REFLECTION_MODEL, compressionLevel: level };
830
1072
  await saveState(state);
831
1073
  return;
832
1074
  }
@@ -846,9 +1088,11 @@ async function reflectNow(ctx: any, opts: { reason: string; signal?: AbortSignal
846
1088
  }
847
1089
 
848
1090
  async function runObserver(ctx: any, historyText: string, signal: AbortSignal | undefined, opts: { source: string; existingObservations?: string }): Promise<ObserverResult> {
849
- const model = resolveModel(ctx, DEFAULT_OBSERVATION_MODEL);
1091
+ const state = await ensureState(ctx);
1092
+ const modelId = state.settings.observationModel || DEFAULT_OBSERVATION_MODEL;
1093
+ const model = resolveModel(ctx, modelId);
850
1094
  const response = await runModel(ctx, model, [
851
- { role: "user", content: [{ type: "text", text: buildObserverSystemPrompt() }] },
1095
+ { role: "user", content: [{ type: "text", text: buildObserverSystemPrompt(state.settings.caveman) }] },
852
1096
  { role: "user", content: [{ type: "text", text: `## New Message History to Observe\n\n${historyText}\n\n---\n\n${buildObserverTaskPrompt(opts.existingObservations, { priorCurrentTask: runtime.state?.currentTask, priorSuggestedResponse: runtime.state?.suggestedResponse })}` }] },
853
1097
  ], { temperature: 0.3, maxTokens: 100_000, signal });
854
1098
  const text = responseText(response);
@@ -859,9 +1103,11 @@ async function runObserver(ctx: any, historyText: string, signal: AbortSignal |
859
1103
  }
860
1104
 
861
1105
  async function runReflector(ctx: any, observations: string, level: CompressionLevel, signal: AbortSignal | undefined, manualPrompt?: string): Promise<ReflectorResult> {
862
- const model = resolveModel(ctx, DEFAULT_REFLECTION_MODEL);
1106
+ const state = await ensureState(ctx);
1107
+ const modelId = state.settings.reflectionModel || DEFAULT_REFLECTION_MODEL;
1108
+ const model = resolveModel(ctx, modelId);
863
1109
  const response = await runModel(ctx, model, [
864
- { role: "user", content: [{ type: "text", text: buildReflectorSystemPrompt() }] },
1110
+ { role: "user", content: [{ type: "text", text: buildReflectorSystemPrompt(state.settings.caveman) }] },
865
1111
  { role: "user", content: [{ type: "text", text: buildReflectorPrompt(observations, level, manualPrompt) }] },
866
1112
  ], { temperature: 0, maxTokens: 100_000, signal });
867
1113
  const text = responseText(response);
@@ -909,7 +1155,7 @@ function extractListItemsOnly(content: string): string {
909
1155
  }
910
1156
 
911
1157
  function sanitizeObservationLines(observations: string): string {
912
- return observations.split("\n").map(line => line.length > MAX_OBSERVATION_LINE_CHARS ? `${line.slice(0, MAX_OBSERVATION_LINE_CHARS)} … [truncated]` : line).join("\n").trim();
1158
+ return redactSecrets(observations).split("\n").map(line => line.length > MAX_OBSERVATION_LINE_CHARS ? `${line.slice(0, MAX_OBSERVATION_LINE_CHARS)} … [truncated]` : line).join("\n").trim();
913
1159
  }
914
1160
 
915
1161
  function detectDegenerateRepetition(text: string): boolean {
@@ -937,7 +1183,7 @@ function appendObservations(state: PiOMRecord, result: ObserverResult, inputToke
937
1183
  state.currentTask = result.currentTask ?? state.currentTask;
938
1184
  state.suggestedResponse = result.suggestedContinuation ?? state.suggestedResponse;
939
1185
  state.observationTokens = estimateTokens(state.observations);
940
- state.lastOperation = { type: "observation", startedAt: state.operationLock?.startedAt ?? new Date().toISOString(), endedAt: new Date().toISOString(), inputTokens, outputTokens: estimateTokens(observations), model: DEFAULT_OBSERVATION_MODEL };
1186
+ state.lastOperation = { type: "observation", startedAt: state.operationLock?.startedAt ?? new Date().toISOString(), endedAt: new Date().toISOString(), inputTokens, outputTokens: estimateTokens(observations), model: state.settings.observationModel || DEFAULT_OBSERVATION_MODEL };
941
1187
  }
942
1188
 
943
1189
  function buildOMContextMessage(state: PiOMRecord): AgentMessage {
@@ -1059,15 +1305,50 @@ function formatAgentMessage(msg: any, mode: "observer" | "recall" = "observer",
1059
1305
  }
1060
1306
  }
1061
1307
 
1308
+
1309
+ function limitTextByTokens(text: string | undefined, maxTokens: number): string | undefined {
1310
+ if (!text) return text;
1311
+ if (estimateTokens(text) <= maxTokens) return text;
1312
+ const maxChars = Math.max(0, maxTokens * 4);
1313
+ const tail = text.slice(Math.max(0, text.length - maxChars));
1314
+ const lineBoundary = tail.indexOf("\n");
1315
+ const trimmedTail = lineBoundary >= 0 ? tail.slice(lineBoundary + 1) : tail;
1316
+ return `[Earlier observations truncated for observer prompt safety: kept last ~${maxTokens} tokens.]\n${trimmedTail}`;
1317
+ }
1318
+
1319
+ function redactSecrets(input: string): string {
1320
+ if (!input) return input;
1321
+ return input
1322
+ .replace(/npm_[A-Za-z0-9]{16,}/g, "[REDACTED_NPM_TOKEN]")
1323
+ .replace(/github_pat_[A-Za-z0-9_]{20,}/g, "[REDACTED_GITHUB_TOKEN]")
1324
+ .replace(/gh[pousr]_[A-Za-z0-9_]{20,}/g, "[REDACTED_GITHUB_TOKEN]")
1325
+ .replace(/sk-[A-Za-z0-9_-]{20,}/g, "[REDACTED_API_KEY]")
1326
+ .replace(/(?<=(?:\b|["'])(?:api[_-]?key|token|secret|password)(?:\b|["'])\s*[:=]\s*["']?)[^"'\s,;}]{8,}/gi, "[REDACTED_SECRET]")
1327
+ .replace(/\bBearer\s+[A-Za-z0-9._~+/=-]{16,}/gi, "Bearer [REDACTED_TOKEN]");
1328
+ }
1329
+
1330
+ function redactDeep<T>(value: T): T {
1331
+ if (typeof value === "string") return redactSecrets(value) as T;
1332
+ if (Array.isArray(value)) return value.map(item => redactDeep(item)) as T;
1333
+ if (value && typeof value === "object") {
1334
+ const out: Record<string, unknown> = {};
1335
+ for (const [key, child] of Object.entries(value as Record<string, unknown>)) {
1336
+ out[key] = /apiKey|authorization|token|secret|password/i.test(key) ? "[REDACTED_SECRET]" : redactDeep(child);
1337
+ }
1338
+ return out as T;
1339
+ }
1340
+ return value;
1341
+ }
1342
+
1062
1343
  function formatContent(content: any, cap: number): string {
1063
- if (typeof content === "string") return truncateText(content, cap);
1064
- if (!Array.isArray(content)) return truncateText(JSON.stringify(content), cap);
1344
+ if (typeof content === "string") return truncateText(redactSecrets(content), cap);
1345
+ if (!Array.isArray(content)) return truncateText(redactSecrets(JSON.stringify(redactDeep(content))), cap);
1065
1346
  return content.map(part => {
1066
- if (part.type === "text") return part.text;
1067
- if (part.type === "thinking") return `[Thinking]: ${part.thinking}`;
1068
- if (part.type === "image") return `[Image: ${part.mimeType ?? "unknown"}]`;
1069
- if (part.type === "toolCall") return `[Tool Call ${part.name}: ${truncateText(JSON.stringify(part.arguments ?? {}), 2_000)}]`;
1070
- return `[${part.type}: ${truncateText(JSON.stringify(part), 2_000)}]`;
1347
+ if (part.type === "text") return redactSecrets(part.text);
1348
+ if (part.type === "thinking") return `[Thinking]: ${redactSecrets(part.thinking ?? "")}`;
1349
+ if (part.type === "image") return runtime.state?.settings.observeAttachments === false ? "[Image omitted: attachment observation disabled]" : `[Image: ${part.mimeType ?? "unknown"}]`;
1350
+ if (part.type === "toolCall") return `[Tool Call ${part.name}: ${truncateText(redactSecrets(JSON.stringify(redactDeep(part.arguments ?? {}))), 2_000)}]`;
1351
+ return `[${part.type}: ${truncateText(redactSecrets(JSON.stringify(redactDeep(part))), 2_000)}]`;
1071
1352
  }).join("\n").slice(0, cap);
1072
1353
  }
1073
1354
 
@@ -1102,9 +1383,9 @@ function truncateText(text: string, maxChars: number): string {
1102
1383
  }
1103
1384
 
1104
1385
  function responseText(response: any): string {
1105
- if (typeof response?.text === "string") return response.text;
1106
- if (Array.isArray(response?.content)) return response.content.filter((c: any) => c?.type === "text").map((c: any) => c.text).join("\n");
1107
- if (typeof response === "string") return response;
1386
+ if (typeof response?.text === "string") return redactSecrets(response.text);
1387
+ if (Array.isArray(response?.content)) return redactSecrets(response.content.filter((c: any) => c?.type === "text").map((c: any) => c.text).join("\n"));
1388
+ if (typeof response === "string") return redactSecrets(response);
1108
1389
  return "";
1109
1390
  }
1110
1391
 
@@ -1142,6 +1423,70 @@ function formatShortStatusColored(state: PiOMRecord): string {
1142
1423
  return `om: ${statusColor}${state.status}\x1b[0m \x1b[2;37m|\x1b[0m msg \x1b[1;33m${formatTokens(state.pendingMessageTokens)}\x1b[0m/\x1b[2;37m${formatTokens(effectiveObservationThreshold(state))}\x1b[0m [${msgPctStr}] \x1b[2;37m|\x1b[0m mem \x1b[1;36m${formatTokens(state.observationTokens)}\x1b[0m/\x1b[2;37m${formatTokens(state.thresholds.reflection)}\x1b[0m [${memPctStr}]${buffer}${err}`;
1143
1424
  }
1144
1425
 
1426
+ function parsePositiveIntSetting(key: string, value: string): number {
1427
+ const n = Number(value);
1428
+ if (!Number.isFinite(n) || n <= 0) throw new Error(`OM setting ${key} must be a positive number`);
1429
+ return Math.round(n);
1430
+ }
1431
+
1432
+ function parseRatioSetting(key: string, value: string): number {
1433
+ const raw = value.trim();
1434
+ const n = raw.endsWith("%") ? Number(raw.slice(0, -1)) / 100 : Number(raw);
1435
+ if (!Number.isFinite(n) || n <= 0 || n > 1) throw new Error(`OM setting ${key} must be a ratio between 0 and 1, or percent like 80%`);
1436
+ return n;
1437
+ }
1438
+
1439
+ function parseBooleanSetting(key: string, value: string): boolean {
1440
+ const normalized = value.trim().toLowerCase();
1441
+ if (["1", "true", "yes", "on", "enabled"].includes(normalized)) return true;
1442
+ if (["0", "false", "no", "off", "disabled"].includes(normalized)) return false;
1443
+ throw new Error(`OM setting ${key} must be on/off or true/false`);
1444
+ }
1445
+
1446
+ function applySetting(state: PiOMRecord, key: string | undefined, value: string): void {
1447
+ if (!key) throw new Error("Missing OM setting key");
1448
+ if (!value) throw new Error(`Missing OM setting value for ${key}`);
1449
+ const normalized = key.toLowerCase().replace(/_/g, "-");
1450
+ if (["observation", "observation-threshold", "observe-threshold"].includes(normalized)) state.thresholds.observation = parsePositiveIntSetting(key, value);
1451
+ else if (["reflection", "reflection-threshold", "reflect-threshold"].includes(normalized)) state.thresholds.reflection = parsePositiveIntSetting(key, value);
1452
+ else if (["block-after", "blockafter"].includes(normalized)) state.thresholds.blockAfter = parsePositiveIntSetting(key, value);
1453
+ else if (["buffer", "buffer-tokens"].includes(normalized)) state.thresholds.bufferTokens = parsePositiveIntSetting(key, value);
1454
+ else if (["buffer-activation", "activation-ratio"].includes(normalized)) state.thresholds.bufferActivation = parseRatioSetting(key, value);
1455
+ else if (["observation-model", "observer-model", "observe-model"].includes(normalized)) state.settings.observationModel = value.trim();
1456
+ else if (["reflection-model", "reflector-model", "reflect-model"].includes(normalized)) state.settings.reflectionModel = value.trim();
1457
+ else if (normalized === "caveman") state.settings.caveman = parseBooleanSetting(key, value);
1458
+ else if (["attachments", "observe-attachments", "attachment-observation"].includes(normalized)) state.settings.observeAttachments = parseBooleanSetting(key, value);
1459
+ else if (normalized === "scope") {
1460
+ const scope = value.trim().toLowerCase();
1461
+ if (scope !== "session" && scope !== "project") throw new Error("OM scope must be session or project");
1462
+ state.scope = scope;
1463
+ } else {
1464
+ throw new Error(`Unknown OM setting: ${key}`);
1465
+ }
1466
+ if (state.thresholds.blockAfter < state.thresholds.observation) state.thresholds.blockAfter = Math.round(state.thresholds.observation * DEFAULT_BLOCK_AFTER_MULTIPLIER);
1467
+ state.observationTokens = estimateTokens(state.observations);
1468
+ }
1469
+
1470
+ function formatSettingsText(state: PiOMRecord): string {
1471
+ return [
1472
+ formatDetailedStatus(state),
1473
+ "",
1474
+ "settings:",
1475
+ ` observation-model: ${state.settings.observationModel}`,
1476
+ ` reflection-model: ${state.settings.reflectionModel}`,
1477
+ ` caveman: ${state.settings.caveman ? "on" : "off"}`,
1478
+ ` attachments: ${state.settings.observeAttachments ? "on" : "off"}`,
1479
+ ` scope: ${state.scope}`,
1480
+ "",
1481
+ "usage:",
1482
+ " /om set observation-threshold 30000",
1483
+ " /om set reflection-threshold 40000",
1484
+ " /om set observation-model google/gemini-2.5-flash",
1485
+ " /om set caveman on",
1486
+ " /om reset",
1487
+ ].join("\n");
1488
+ }
1489
+
1145
1490
  function formatDetailedStatus(state: PiOMRecord): string {
1146
1491
  return `${formatShortStatus(state)}\nlastObservedEntryId: ${state.lastObservedEntryId ?? "none"}\ncurrentTask: ${state.currentTask ?? "none"}\nsuggestedResponse: ${state.suggestedResponse ?? "none"}\nlastOperation: ${state.lastOperation ? JSON.stringify(state.lastOperation, null, 2) : "none"}\nstatePath: ${runtime.statePath ?? "unknown"}`;
1147
1492
  }
@@ -1320,11 +1665,15 @@ function formatTokens(tokens: number): string {
1320
1665
  }
1321
1666
 
1322
1667
  async function writeDebug(ctx: any, name: string, payload: unknown): Promise<void> {
1323
- const state = await ensureState(ctx);
1324
- const dir = runtime.debugDir ?? join(ctx.cwd, CONFIG_DIR_NAME, "om", "debug");
1325
- await mkdir(dir, { recursive: true });
1326
- const file = join(dir, `${new Date().toISOString().replace(/[:.]/g, "-")}-${sanitizeFileName(name)}.json`);
1327
- await writeFile(file, JSON.stringify({ extension: EXTENSION_ID, sessionId: state.sessionId, ...payload as any }, null, 2) + "\n", "utf8");
1668
+ try {
1669
+ const state = await ensureState(ctx);
1670
+ const dir = runtime.debugDir || join(ctx.cwd, CONFIG_DIR_NAME, "om", "debug");
1671
+ await mkdir(dir, { recursive: true });
1672
+ const file = join(dir, `${new Date().toISOString().replace(/[:.]/g, "-")}-${sanitizeFileName(name)}.json`);
1673
+ await writeFile(file, redactSecrets(JSON.stringify(redactDeep({ extension: EXTENSION_ID, sessionId: state.sessionId, ...payload as any }), null, 2)) + "\n", "utf8");
1674
+ } catch (error) {
1675
+ // Ignore debug write errors if context is stale
1676
+ }
1328
1677
  }
1329
1678
 
1330
1679
  function sanitizeFileName(name: string): string {
@@ -1346,6 +1695,22 @@ function mergeAbortSignals(a?: AbortSignal, b?: AbortSignal): AbortSignal | unde
1346
1695
  return controller.signal;
1347
1696
  }
1348
1697
 
1698
+ export const __test = {
1699
+ redactSecrets,
1700
+ redactDeep,
1701
+ limitTextByTokens,
1702
+ buildObserverTaskPrompt,
1703
+ defaultSettings,
1704
+ defaultThresholds,
1705
+ createDefaultState,
1706
+ normalizeState,
1707
+ isStaleOperationLock,
1708
+ recoverStaleOperationLock,
1709
+ atomicWriteJson,
1710
+ applySetting,
1711
+ formatSettingsText,
1712
+ };
1713
+
1349
1714
  class OMOverlay {
1350
1715
  private scroll = 0;
1351
1716
  private tab: "memory" | "status" | "debug" = "memory";
package/package.json CHANGED
@@ -1,13 +1,39 @@
1
1
  {
2
2
  "name": "pi-observational-memory-extension",
3
- "version": "0.1.1",
3
+ "version": "0.1.3",
4
4
  "description": "Mastra-style Observational Memory extension for Pi compaction and runtime context.",
5
5
  "license": "MIT",
6
6
  "author": "Nikita Nosov <20nik.nosov21@gmail.com>",
7
7
  "type": "module",
8
- "keywords": ["pi-package", "pi", "pi-coding-agent", "observational-memory", "memory", "compaction"],
9
- "files": ["extensions", "docs", "scripts", "README.md", "LICENSE", "CHANGELOG.md"],
10
- "pi": { "extensions": ["./extensions"] },
8
+ "keywords": [
9
+ "pi-package",
10
+ "pi",
11
+ "pi-coding-agent",
12
+ "observational-memory",
13
+ "memory",
14
+ "compaction"
15
+ ],
16
+ "homepage": "https://github.com/nik1t7n/pi-observational-memory-extension#readme",
17
+ "repository": {
18
+ "type": "git",
19
+ "url": "git+https://github.com/nik1t7n/pi-observational-memory-extension.git"
20
+ },
21
+ "bugs": {
22
+ "url": "https://github.com/nik1t7n/pi-observational-memory-extension/issues"
23
+ },
24
+ "files": [
25
+ "extensions",
26
+ "docs",
27
+ "scripts",
28
+ "README.md",
29
+ "LICENSE",
30
+ "CHANGELOG.md"
31
+ ],
32
+ "pi": {
33
+ "extensions": [
34
+ "./extensions"
35
+ ]
36
+ },
11
37
  "peerDependencies": {
12
38
  "@earendil-works/pi-coding-agent": "*",
13
39
  "@earendil-works/pi-ai": "*",
@@ -24,6 +50,7 @@
24
50
  "typecheck": "tsc --noEmit",
25
51
  "validate": "node scripts/validate.mjs",
26
52
  "pack:check": "npm pack --dry-run",
27
- "prepublishOnly": "npm run validate && npm run typecheck && npm run pack:check"
53
+ "prepublishOnly": "npm run validate && npm run typecheck && npm test && npm run pack:check",
54
+ "test": "node scripts/test.mjs"
28
55
  }
29
56
  }
@@ -0,0 +1,70 @@
1
+ #!/usr/bin/env node
2
+ import assert from "node:assert/strict";
3
+ import { execFileSync } from "node:child_process";
4
+ import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
5
+ import { existsSync } from "node:fs";
6
+ import { join } from "node:path";
7
+ import { tmpdir } from "node:os";
8
+
9
+ const dist = ".tmp-test-dist";
10
+ await rm(dist, { recursive: true, force: true });
11
+ execFileSync("npx", ["tsc", "--outDir", dist, "--declaration", "false", "--noEmit", "false", "--rootDir", "."], { stdio: "inherit" });
12
+ const { __test } = await import(`../${dist}/extensions/index.js?${Date.now()}`);
13
+
14
+ // Secret redaction: state/debug/observer text must not persist common tokens.
15
+ const redacted = __test.redactSecrets("npm_abcdefghijklmnopqrstuvwxyz sk-abcdefghijklmnopqrstuvwxyz Bearer abcdefghijklmnopqrstuvwxyz token=supersecrettoken");
16
+ assert(!redacted.includes("npm_abcdefghijklmnopqrstuvwxyz"));
17
+ assert(!redacted.includes("sk-abcdefghijklmnopqrstuvwxyz"));
18
+ assert(!redacted.includes("Bearer abcdefghijklmnopqrstuvwxyz"));
19
+ assert(!redacted.includes("supersecrettoken"));
20
+ assert(redacted.includes("[REDACTED_NPM_TOKEN]"));
21
+
22
+ // Observer prompt limiting: previous observations are capped to a safe tail.
23
+ const huge = Array.from({ length: 5000 }, (_, i) => `* 🔴 old observation ${i}`).join("\n");
24
+ const prompt = __test.buildObserverTaskPrompt(huge, { priorCurrentTask: "continue tests" });
25
+ assert(prompt.length < huge.length / 2, "observer prompt should not include full previous observations");
26
+ assert(prompt.includes("truncated"));
27
+ assert(prompt.includes("continue tests"));
28
+
29
+ // Stale lock recovery: old operation locks are cleared and recorded as errors.
30
+ const state = __test.createDefaultState({ sessionId: "s", cwd: process.cwd() });
31
+ state.operationLock = { type: "observation", startedAt: new Date(Date.now() - 30 * 60 * 1000).toISOString() };
32
+ state.status = "observing";
33
+ assert.equal(__test.recoverStaleOperationLock(state), true);
34
+ assert.equal(state.operationLock, undefined);
35
+ assert.equal(state.status, "idle");
36
+ assert.match(state.lastError, /Recovered stale OM operation lock/);
37
+
38
+ // Settings parser: full /om set surface mutates persisted config safely.
39
+ __test.applySetting(state, "observation-model", "google/gemini-2.5-flash");
40
+ __test.applySetting(state, "reflection-threshold", "12345");
41
+ __test.applySetting(state, "caveman", "on");
42
+ __test.applySetting(state, "attachments", "off");
43
+ assert.equal(state.settings.observationModel, "google/gemini-2.5-flash");
44
+ assert.equal(state.thresholds.reflection, 12345);
45
+ assert.equal(state.settings.caveman, true);
46
+ assert.equal(state.settings.observeAttachments, false);
47
+ assert.throws(() => __test.applySetting(state, "buffer-activation", "2"), /ratio/);
48
+
49
+ // Migration/normalization: v1 state keeps observations and receives v2 settings.
50
+ const migrated = __test.normalizeState({ version: 1, observations: "token=do-not-keep-this-secret", thresholds: { observation: 10 } }, { sessionId: "m", cwd: process.cwd() });
51
+ assert.equal(migrated.version, 2);
52
+ assert.equal(migrated.thresholds.observation, 10);
53
+ assert.equal(migrated.settings.caveman, false);
54
+ assert(!migrated.observations.includes("do-not-keep-this-secret"));
55
+
56
+ // Atomic write: writes valid JSON, creates backup on subsequent write, removes temp files.
57
+ const dir = await mkdtemp(join(tmpdir(), "pi-om-test-"));
58
+ try {
59
+ const file = join(dir, "state.json");
60
+ await __test.atomicWriteJson(file, { a: 1, token: "supersecrettoken" });
61
+ assert.equal(JSON.parse(await readFile(file, "utf8")).token, "[REDACTED_SECRET]");
62
+ await __test.atomicWriteJson(file, { a: 2 });
63
+ assert.equal(JSON.parse(await readFile(file, "utf8")).a, 2);
64
+ assert.equal(existsSync(`${file}.bak`), true);
65
+ } finally {
66
+ await rm(dir, { recursive: true, force: true });
67
+ await rm(dist, { recursive: true, force: true });
68
+ }
69
+
70
+ console.log("pi-observational-memory-extension tests passed");