pi-session-memory 0.2.1 → 0.3.0

package/README.md

@@ -17,7 +17,7 @@ A local-first Pi extension that saves completed conversations to SQLite and give
 - Ranks results by literal relevance and recency, boosts an explicitly scoped project, and limits results to two turns per session for diversity.
 - Supports explicit durable memories, which remain after their source transcript turns are deleted.
 - Suppresses a raw turn from recall when an active durable memory contains the same unchanged pinned source evidence; other turns in that session remain eligible.
-- Flags a recalled durable memory when a newer matching turn exists in its source session, so users can explicitly confirm or supersede it without silent updates.
+- Associates recalled durable memories with later source-session activity and newer query-relevant evidence from the same or another session, so users can explicitly compare, confirm, or supersede them without silent updates.
 - Includes status, direct search, and permanent deletion commands so users can inspect and control local memory.
 - Uses only Node.js built-ins and SQLite (`node:sqlite`); no external runtime dependencies.
 
@@ -46,7 +46,7 @@ pi -e npm:pi-session-memory
 To intentionally pin a known version (which `pi update --extensions` skips), add its version explicitly:
 
 ```bash
-pi install npm:pi-session-memory@0.2.1
+pi install npm:pi-session-memory@0.3.0
 ```
 
 ## Usage
@@ -108,7 +108,7 @@ What did we decide about LangGraph last time?
 
 `recall_memory` is the discovery step: it searches and ranks the complete active durable-memory and raw-turn match set using the original request plus important entities. It supports optional exact project-directory, source, and time-window filters. Each tool response deliberately renders five results and reports `totalResults` and `nextOffset`; when more candidates are needed, Pi repeats the exact same query and filters with that explicit offset. This pages model context without silently limiting the local search. Raw transcript candidates contain a short excerpt plus a session ID and turn index, rather than the entire turn context. When surrounding conversation is needed to answer accurately, Pi calls `fetch_session` with that session ID and the smallest useful turn-index range. When an initial literal search is empty, Pi may make up to two additional local searches using reasoned alternatives—such as abbreviations, expansions, aliases, translations, or likely task wording—while retaining the original filters.
 
-When recall returns a durable memory, Pi is instructed to naturally communicate a relevant remembered conclusion and provenance when useful. If newer matching evidence makes that memory a freshness candidate, Pi explains the discrepancy and asks whether you want to keep, confirm, or replace it. It never claims a memory was updated or superseded without your explicit choice.
+When recall returns a durable memory, Pi is instructed to naturally communicate a relevant remembered conclusion and provenance when useful. It reports later activity in the memory's source session separately from newer query-relevant evidence to compare; that evidence can come from the original session or another newer session. Pi compares the old memory with the evidence as a possible confirmation, supplement, conflict, or replacement, then asks whether you want to keep, confirm, or replace it. It never claims a memory was updated or superseded without your explicit choice.
 
 ### Inspect and control memory
 
@@ -132,7 +132,7 @@ Recall and freshness explanations are automatic model behavior. The commands bel
 - `/remember <text>` saves an explicit durable `fact` scoped to the current project.
 - `/memory-pin <turn-id>` promotes a historical turn to a durable fact and records its source session, source turn ID, and a hash of the pinned evidence.
 - `/memory-list [kind]` displays durable memories, optionally limited to `preference`, `decision`, `fact`, `project_state`, `task`, or `lesson`.
-- Recall marks a durable memory as a freshness candidate when a newer query-matching turn appears in its source session; this is a review signal, not an automatic update.
+- Recall distinguishes later activity in a memory's source session from newer query-relevant evidence to compare. That evidence may come from the original session or another newer session; it is a review signal, not an automatic update.
 - `/memory-confirm <memory-id>` records that an active memory remains current by updating `last_confirmed_at`.
 - `/memory-supersede <old-memory-id> <new-memory-id>` explicitly replaces an active memory while retaining the old record for history; superseded memories are excluded from normal recall.
 - `/memory-history <memory-id>` displays the complete oldest-to-newest supersession chain.
@@ -168,6 +168,7 @@ Conversation data is stored and queried locally. This package does not add a rem
 
 | Version | Highlights |
 | --- | --- |
+| `0.3.0` | Replaces source-session-only freshness hints with provenance-linked evidence comparison across newer same-session and cross-session turns. This changes recall output and `freshness_candidate` semantics, but keeps tool inputs, slash commands, SQLite data, and explicit user-controlled memory mutation compatible; no migration is required. |
 | `0.2.1` | Pages `recall_memory` results in explicit five-result `offset` windows while still evaluating the complete local match set; npm publishing now uses a runtime-file allowlist. |
 | `0.2.0` | Added cross-client SQLite recall and durable-memory controls, plus native current-project Codex-to-Pi session migration for `/resume`. |
 | `0.1.4` | Automatically syncs new or changed Pi, Claude Code, and Codex history when Pi starts; `/memory-backfill` forces a full rescan. |
@@ -176,6 +177,18 @@ Conversation data is stored and queried locally. This package does not add a rem
 | `0.1.1` | Added repository and package metadata for public distribution. |
 | `0.1.0` | Initial release: local SQLite memory, Pi live persistence, historical import, and `recall_memory` retrieval. |
 
+## Release compatibility review
+
+Before every significant release, review these compatibility surfaces and record any migration or versioning decision:
+
+1. **Install/package:** package name, Pi manifest, runtime dependencies, and published file allowlist.
+2. **Persistent data:** SQLite schema/migrations, JSONL-import compatibility, and any data rewrite.
+3. **Agent tools and commands:** tool names, input schemas, result/details contracts, and slash commands.
+4. **Retrieval and agent behavior:** ranking, pagination, freshness/evidence semantics, prompt policy, and automatic side effects.
+5. **Public TypeScript/module API:** exported types/functions and required result fields.
+
+The `0.3.0` review found no installation, SQLite, command, or tool-input breaking change. It intentionally changes recall result semantics and adds evidence fields, so it is released as a minor `0.x` version rather than a patch.
+
 ## Development
 
 ```bash

package/extensions/index.ts

@@ -211,9 +211,9 @@ Invocation policy:
 
 Memory-aware response policy:
 1. Treat returned durable memories as reusable evidence, not as invisible context. In your natural-language answer, briefly state the relevant remembered conclusion and identify its source turn/session when that provenance matters to the answer.
-2. If a durable memory is marked **Freshness**, explain that newer matching evidence exists. Compare the old memory and the newer raw evidence; do not claim that the memory was updated, confirmed, or superseded unless the user explicitly chose that action.
-3. After explaining a meaningful freshness conflict or decision change, offer the user clear control: keep the current memory, confirm that it remains current, or create/pin a replacement and supersede the old memory. Ask which outcome they want before any persistent memory-management action.
-4. When an existing durable memory resolves the question and has no freshness signal, use it directly and avoid repeating its identical source turn. Do not mention memory mechanics unless provenance or freshness is useful to the user.
+2. A durable memory can report `source_session_changed` when its original session has later activity; this alone does not mean the memory is stale. When it includes **Newer evidence to compare**, compare that evidence with the memory: it may confirm, supplement, conflict with, or replace the old conclusion. Evidence can come from another newer session as well as the original session. Do not claim that the memory was updated, confirmed, or superseded unless the user explicitly chose that action.
+3. After explaining a meaningful comparison, offer clear control: keep the current memory, confirm that it remains current, or create/pin a replacement and supersede the old memory. Ask which outcome they want before any persistent memory-management action.
+4. When an existing durable memory resolves the question and has no comparison evidence, use it directly and avoid repeating its identical source turn. Do not mention memory mechanics unless provenance or evidence comparison is useful to the user.
 5. Slash commands are user-controlled management actions. Do not instruct the user to execute a command merely to answer their question; mention the relevant command only when they want to inspect, confirm, replace, or delete a memory.
 
 Session-expansion policy:

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "pi-session-memory",
-  "version": "0.2.1",
-  "description": "Persistent, local-first cross-session memory for Pi, with SQLite-backed paginated recall across Pi, Claude Code, and Codex conversations",
+  "version": "0.3.0",
+  "description": "Persistent, local-first cross-session memory for Pi, with SQLite-backed paginated recall and freshness-evidence comparison across Pi, Claude Code, and Codex conversations",
   "keywords": [
     "pi-package"
   ],

package/src/retriever.ts

@@ -26,6 +26,16 @@ export interface RecallTurnResult {
   score: number;
 }
 
+export interface FreshnessEvidence {
+  turn_id: string;
+  session_id: string;
+  turn_index: number;
+  source: MemorySource;
+  ts: number;
+  relation: "same_source_session_later" | "newer_cross_session";
+  excerpt: string;
+}
+
 export interface RecallDurableMemoryResult {
   type: "memory";
   memory_id: string;
@@ -36,7 +46,9 @@ export interface RecallDurableMemoryResult {
   source_session_id: string | null;
   source_content_hash: string | null;
   source_turn_index: number | null;
+  source_session_changed: boolean;
   freshness_candidate: boolean;
+  freshness_evidence: FreshnessEvidence[];
   created_at: number;
   last_confirmed_at: number;
   importance: number;
@@ -67,13 +79,15 @@ export function recallTurns(entities: string[]): RecallTurnResult[] {
 export function recallMemories(options: RecallOptions): RecallResult[] {
   const durableMemories = _recallDurableMemories(options);
   const recalledTurns = _recallTurns(options);
-  const freshnessCandidates = new Set(
-    durableMemories
-      .filter((memory) => memory.source_session_id && memory.source_turn_index !== null)
-      .filter((memory) => recalledTurns.some((turn) => turn.session_id === memory.source_session_id && turn.turn_index > memory.source_turn_index!))
-      .map((memory) => memory.memory_id),
-  );
-  const memories = durableMemories.map((memory) => ({ ...memory, freshness_candidate: freshnessCandidates.has(memory.memory_id) }));
+  const memories = durableMemories.map((memory) => {
+    const freshness_evidence = _freshnessEvidence(memory, recalledTurns);
+    return {
+      ...memory,
+      source_session_changed: _sourceSessionChanged(memory),
+      freshness_candidate: freshness_evidence.length > 0,
+      freshness_evidence,
+    };
+  });
   const coveredSourceHashes = new Map(
     memories
       .filter((memory) => memory.source_turn_id && memory.source_content_hash)
@@ -105,7 +119,14 @@ export function formatRecallResults(results: RecallResult[], options?: Pick<Reca
       lines.push(`**Memory ID:** ${result.memory_id}`);
       if (result.source_turn_id) lines.push(`**Source turn:** ${result.source_turn_id}`);
       if (result.source_session_id) lines.push(`**Source session:** ${result.source_session_id}`);
-      if (result.freshness_candidate) lines.push("**Freshness:** newer matching turn exists in the source session; confirm or supersede this memory.");
+      if (result.source_session_changed) lines.push("**Source session changed:** later turns exist; this alone does not mean the memory is stale.");
+      if (result.freshness_candidate) {
+        lines.push("**Newer evidence to compare:**");
+        for (const evidence of result.freshness_evidence) {
+          lines.push(`- [${evidence.relation.replaceAll("_", " ")} · ${evidence.source} · ${new Date(evidence.ts).toLocaleString()} · ${evidence.session_id} · turn ${evidence.turn_index}] ${evidence.excerpt}`);
+        }
+        lines.push("Compare this evidence with the durable memory; it may confirm, supplement, conflict with, or replace it. Do not change the memory without the user's explicit choice.");
+      }
     } else {
       const date = new Date(result.ts).toLocaleString();
       lines.push(`### [${result.source} · ${date}]`);
@@ -167,7 +188,45 @@ function _recallDurableMemories(options: RecallOptions): RecallDurableMemoryResu
     WHERE ${filters.join(" AND ")}
     ORDER BY hits DESC, importance DESC, last_confirmed_at DESC
   `).all(...parameters, ...filterParameters)
-    .map((memory) => ({ ...memory, type: "memory" as const, freshness_candidate: false, score: memory.hits + memory.importance })) as RecallDurableMemoryResult[];
+    .map((memory) => ({ ...memory, type: "memory" as const, source_session_changed: false, freshness_candidate: false, freshness_evidence: [], score: memory.hits + memory.importance })) as RecallDurableMemoryResult[];
+}
+
+/** Detect later activity in the original session independently of this recall query and ranking. */
+function _sourceSessionChanged(memory: RecallDurableMemoryResult): boolean {
+  if (!memory.source_session_id || memory.source_turn_index === null) return false;
+  return getDb().prepare(`
+    SELECT 1 FROM turns
+    WHERE session_id = ? AND turn_index > ?
+    LIMIT 1
+  `).get(memory.source_session_id, memory.source_turn_index) !== undefined;
+}
+
+/** Associate each memory with all newer query-relevant turns without deciding their semantic relationship. */
+function _freshnessEvidence(memory: RecallDurableMemoryResult, recalledTurns: RecallTurnResult[]): FreshnessEvidence[] {
+  const baselineTs = _memoryEvidenceTimestamp(memory);
+  return recalledTurns
+    .filter((turn) => {
+      if (memory.source_session_id === turn.session_id && memory.source_turn_index !== null) {
+        return turn.turn_index > memory.source_turn_index;
+      }
+      return turn.ts > baselineTs;
+    })
+    .map((turn) => ({
+      turn_id: turn.turn_id,
+      session_id: turn.session_id,
+      turn_index: turn.turn_index,
+      source: turn.source,
+      ts: turn.ts,
+      relation: memory.source_session_id === turn.session_id ? "same_source_session_later" as const : "newer_cross_session" as const,
+      excerpt: _excerpt(turn.user_text || turn.reply_text),
+    }));
+}
+
+/** Use source-turn time when available; explicit memories become comparable from their creation time. */
+function _memoryEvidenceTimestamp(memory: RecallDurableMemoryResult): number {
+  if (!memory.source_turn_id) return memory.created_at;
+  const source = getDb().prepare("SELECT ts FROM turns WHERE turn_id = ?").get(memory.source_turn_id) as { ts: number } | undefined;
+  return source?.ts ?? memory.created_at;
 }
 
 /** Retrieve and rank locally stored turns using literal query terms and optional scopes. */