pi-session-memory 0.2.0 → 0.3.0

package/README.md

@@ -17,20 +17,36 @@ A local-first Pi extension that saves completed conversations to SQLite and give
 - Ranks results by literal relevance and recency, boosts an explicitly scoped project, and limits results to two turns per session for diversity.
 - Supports explicit durable memories, which remain after their source transcript turns are deleted.
 - Suppresses a raw turn from recall when an active durable memory contains the same unchanged pinned source evidence; other turns in that session remain eligible.
-- Flags a recalled durable memory when a newer matching turn exists in its source session, so users can explicitly confirm or supersede it without silent updates.
+- Associates recalled durable memories with later source-session activity and newer query-relevant evidence from the same or another session, so users can explicitly compare, confirm, or supersede them without silent updates.
 - Includes status, direct search, and permanent deletion commands so users can inspect and control local memory.
 - Uses only Node.js built-ins and SQLite (`node:sqlite`); no external runtime dependencies.
 
 ## Installation
 
 ```bash
-pi install npm:pi-session-memory@0.2.0
+pi install npm:pi-session-memory
 ```
 
-To try the package without installing it permanently:
+This unpinned source can receive package-update checks at Pi startup. After a new release, update it explicitly with:
 
 ```bash
-pi -e npm:pi-session-memory@0.2.0
+pi update npm:pi-session-memory
+# or update every unpinned Pi extension
+pi update --extensions
+```
+
+Restart Pi after the update to load the new extension code.
+
+To try the latest package without installing it permanently:
+
+```bash
+pi -e npm:pi-session-memory
+```
+
+To intentionally pin a known version (which `pi update --extensions` skips), add its version explicitly:
+
+```bash
+pi install npm:pi-session-memory@0.3.0
 ```
 
 ## Usage
@@ -90,9 +106,9 @@ The extension instructs Pi to call `recall_memory` when a user explicitly asks a
 What did we decide about LangGraph last time?
 ```
 
-`recall_memory` is the discovery step: it searches active durable memories first, then ranks prior user prompts and assistant responses using the original request plus important entities. It supports optional exact project-directory, source, and time-window filters. Raw transcript candidates contain a short excerpt plus a session ID and turn index, rather than the entire turn context. When surrounding conversation is needed to answer accurately, Pi calls `fetch_session` with that session ID and the smallest useful turn-index range. When an initial literal search is empty, Pi may make up to two additional local searches using reasoned alternatives—such as abbreviations, expansions, aliases, translations, or likely task wording—while retaining the original filters.
+`recall_memory` is the discovery step: it searches and ranks the complete active durable-memory and raw-turn match set using the original request plus important entities. It supports optional exact project-directory, source, and time-window filters. Each tool response deliberately renders five results and reports `totalResults` and `nextOffset`; when more candidates are needed, Pi repeats the exact same query and filters with that explicit offset. This pages model context without silently limiting the local search. Raw transcript candidates contain a short excerpt plus a session ID and turn index, rather than the entire turn context. When surrounding conversation is needed to answer accurately, Pi calls `fetch_session` with that session ID and the smallest useful turn-index range. When an initial literal search is empty, Pi may make up to two additional local searches using reasoned alternatives—such as abbreviations, expansions, aliases, translations, or likely task wording—while retaining the original filters.
 
-When recall returns a durable memory, Pi is instructed to naturally communicate a relevant remembered conclusion and provenance when useful. If newer matching evidence makes that memory a freshness candidate, Pi explains the discrepancy and asks whether you want to keep, confirm, or replace it. It never claims a memory was updated or superseded without your explicit choice.
+When recall returns a durable memory, Pi is instructed to naturally communicate a relevant remembered conclusion and provenance when useful. It reports later activity in the memory's source session separately from newer query-relevant evidence to compare; that evidence can come from the original session or another newer session. Pi compares the old memory with the evidence as a possible confirmation, supplement, conflict, or replacement, then asks whether you want to keep, confirm, or replace it. It never claims a memory was updated or superseded without your explicit choice.
 
 ### Inspect and control memory
 
@@ -116,7 +132,7 @@ Recall and freshness explanations are automatic model behavior. The commands bel
 - `/remember <text>` saves an explicit durable `fact` scoped to the current project.
 - `/memory-pin <turn-id>` promotes a historical turn to a durable fact and records its source session, source turn ID, and a hash of the pinned evidence.
 - `/memory-list [kind]` displays durable memories, optionally limited to `preference`, `decision`, `fact`, `project_state`, `task`, or `lesson`.
-- Recall marks a durable memory as a freshness candidate when a newer query-matching turn appears in its source session; this is a review signal, not an automatic update.
+- Recall distinguishes later activity in a memory's source session from newer query-relevant evidence to compare. That evidence may come from the original session or another newer session; it is a review signal, not an automatic update.
 - `/memory-confirm <memory-id>` records that an active memory remains current by updating `last_confirmed_at`.
 - `/memory-supersede <old-memory-id> <new-memory-id>` explicitly replaces an active memory while retaining the old record for history; superseded memories are excluded from normal recall.
 - `/memory-history <memory-id>` displays the complete oldest-to-newest supersession chain.
@@ -152,6 +168,8 @@ Conversation data is stored and queried locally. This package does not add a rem
 
 | Version | Highlights |
 | --- | --- |
+| `0.3.0` | Replaces source-session-only freshness hints with provenance-linked evidence comparison across newer same-session and cross-session turns. This changes recall output and `freshness_candidate` semantics, but keeps tool inputs, slash commands, SQLite data, and explicit user-controlled memory mutation compatible; no migration is required. |
+| `0.2.1` | Pages `recall_memory` results in explicit five-result `offset` windows while still evaluating the complete local match set; npm publishing now uses a runtime-file allowlist. |
 | `0.2.0` | Added cross-client SQLite recall and durable-memory controls, plus native current-project Codex-to-Pi session migration for `/resume`. |
 | `0.1.4` | Automatically syncs new or changed Pi, Claude Code, and Codex history when Pi starts; `/memory-backfill` forces a full rescan. |
 | `0.1.3` | Improved package documentation and installation guidance. |
@@ -159,6 +177,18 @@ Conversation data is stored and queried locally. This package does not add a rem
 | `0.1.1` | Added repository and package metadata for public distribution. |
 | `0.1.0` | Initial release: local SQLite memory, Pi live persistence, historical import, and `recall_memory` retrieval. |
 
+## Release compatibility review
+
+Before every significant release, review these compatibility surfaces and record any migration or versioning decision:
+
+1. **Install/package:** package name, Pi manifest, runtime dependencies, and published file allowlist.
+2. **Persistent data:** SQLite schema/migrations, JSONL-import compatibility, and any data rewrite.
+3. **Agent tools and commands:** tool names, input schemas, result/details contracts, and slash commands.
+4. **Retrieval and agent behavior:** ranking, pagination, freshness/evidence semantics, prompt policy, and automatic side effects.
+5. **Public TypeScript/module API:** exported types/functions and required result fields.
+
+The `0.3.0` review found no installation, SQLite, command, or tool-input breaking change. It intentionally changes recall result semantics and adds evidence fields, so it is released as a minor `0.x` version rather than a patch.
+
 ## Development
 
 ```bash

package/extensions/index.ts

@@ -2,7 +2,7 @@ import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-a
 import { Type } from "typebox";
 import { writeTurn } from "../src/writer.ts";
 import { confirmMemory, createMemory, deleteMemory, deleteTurn, getMemoryHistory, getMemoryStats, getSession, listMemories, pinTurnAsMemory, supersedeMemory, type MemoryKind } from "../src/db.ts";
-import { recallMemories, formatRecallResults } from "../src/retriever.ts";
+import { recallMemories, formatRecallResults, paginateRecallResults } from "../src/retriever.ts";
 import { backfillAll, syncChangedHistory, type BackfillStats } from "../src/backfill.ts";
 import { migrateCodexProjectSessions, type ProjectSessionMigrationStats } from "../src/session-migration.ts";
 import { SESSION_MEMORY_HELP } from "../src/helper.ts";
@@ -57,13 +57,13 @@ export default function (pi: ExtensionAPI) {
   });
 
   pi.registerCommand("memory-search", {
-    description: "Search local memory with a literal query",
-    /** Search stored memory from a literal command query. */
+    description: "Search local memory with a literal query and show its first five results",
+    /** Search stored memory from a literal command query without rendering the entire match set. */
     handler: async (args, ctx) => {
       const query = args.trim();
       if (!query) throw new Error("Usage: /memory-search <query>");
-      const results = recallMemories({ query, topK: 5 });
-      ctx.ui.notify(formatRecallResults(results), "info");
+      const page = paginateRecallResults(recallMemories({ query }));
+      ctx.ui.notify(formatRecallResults(page.results, { query }, page), "info");
     },
   });
 
@@ -211,9 +211,9 @@ Invocation policy:
 
 Memory-aware response policy:
 1. Treat returned durable memories as reusable evidence, not as invisible context. In your natural-language answer, briefly state the relevant remembered conclusion and identify its source turn/session when that provenance matters to the answer.
-2. If a durable memory is marked **Freshness**, explain that newer matching evidence exists. Compare the old memory and the newer raw evidence; do not claim that the memory was updated, confirmed, or superseded unless the user explicitly chose that action.
-3. After explaining a meaningful freshness conflict or decision change, offer the user clear control: keep the current memory, confirm that it remains current, or create/pin a replacement and supersede the old memory. Ask which outcome they want before any persistent memory-management action.
-4. When an existing durable memory resolves the question and has no freshness signal, use it directly and avoid repeating its identical source turn. Do not mention memory mechanics unless provenance or freshness is useful to the user.
+2. A durable memory can report `source_session_changed` when its original session has later activity; this alone does not mean the memory is stale. When it includes **Newer evidence to compare**, compare that evidence with the memory: it may confirm, supplement, conflict with, or replace the old conclusion. Evidence can come from another newer session as well as the original session. Do not claim that the memory was updated, confirmed, or superseded unless the user explicitly chose that action.
+3. After explaining a meaningful comparison, offer clear control: keep the current memory, confirm that it remains current, or create/pin a replacement and supersede the old memory. Ask which outcome they want before any persistent memory-management action.
+4. When an existing durable memory resolves the question and has no comparison evidence, use it directly and avoid repeating its identical source turn. Do not mention memory mechanics unless provenance or evidence comparison is useful to the user.
 5. Slash commands are user-controlled management actions. Do not instruct the user to execute a command merely to answer their question; mention the relevant command only when they want to inspect, confirm, replace, or delete a memory.
 
 Session-expansion policy:
@@ -221,6 +221,10 @@ Session-expansion policy:
 2. Call \`fetch_session\` only when a candidate's surrounding conversation is necessary to answer accurately, verify a conclusion, resolve a conflict, or inspect context around a matched turn. Use its turn bounds to request the smallest useful range.
 3. Do not fetch a session when a durable memory or returned excerpt already answers the question. Do not fetch unrelated sessions merely because they were listed.
 
+Pagination policy:
+1. Each invocation returns five results. Local retrieval still evaluates every match before selecting that page.
+2. When the result reports a \`nextOffset\`, call \`recall_memory\` again with the exact same query and filters plus that offset only when more candidates are needed. Do not request pages merely to exhaust the result set.
+
 Extract 2–5 specific entities from the user's topic: project names, tool names, technologies, domain terms, or identifiers.`,
     promptSnippet: "Search cross-client Pi, Claude Code, and Codex history when the user asks about prior discussions or work.",
 
@@ -240,15 +244,17 @@ Extract 2–5 specific entities from the user's topic: project names, tool names
       cwd: Type.Optional(Type.String({ minLength: 1, description: "Exact project working directory to restrict results." })),
       after: Type.Optional(Type.Number({ description: "Inclusive Unix timestamp in milliseconds." })),
       before: Type.Optional(Type.Number({ description: "Inclusive Unix timestamp in milliseconds." })),
+      offset: Type.Optional(Type.Integer({ minimum: 0, description: "Zero-based result offset. Each call returns five results; use the returned nextOffset with identical search and filter inputs only when more candidates are needed." })),
     }),
 
-    /** Resolve an agent memory request into a scoped set of locally ranked conversations. */
-    async execute(_toolCallId, { query, entities, sources, cwd, after, before }) {
-      const results = recallMemories({ query, entities, sources, cwd, after, before, topK: 5 });
-      const text = formatRecallResults(results, { query, entities, sources, cwd, after, before });
+    /** Resolve an agent memory request into one explicit page of a fully evaluated local result set. */
+    async execute(_toolCallId, { query, entities, sources, cwd, after, before, offset }) {
+      const results = recallMemories({ query, entities, sources, cwd, after, before });
+      const page = paginateRecallResults(results, offset);
+      const text = formatRecallResults(page.results, { query, entities, sources, cwd, after, before }, page);
       return {
         content: [{ type: "text" as const, text }],
-        details: { query, entities, sources, cwd, after, before, resultCount: results.length },
+        details: { query, entities, sources, cwd, after, before, offset: page.offset, pageSize: page.results.length, totalResults: page.totalResults, nextOffset: page.nextOffset },
       };
     },
   });

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "pi-session-memory",
-  "version": "0.2.0",
-  "description": "Persistent, local-first cross-session memory for Pi, with SQLite-backed recall across Pi, Claude Code, and Codex conversations",
+  "version": "0.3.0",
+  "description": "Persistent, local-first cross-session memory for Pi, with SQLite-backed paginated recall and freshness-evidence comparison across Pi, Claude Code, and Codex conversations",
   "keywords": [
     "pi-package"
   ],
@@ -15,6 +15,10 @@
     "url": "https://github.com/shengxiao20/pi-session-memory/issues"
   },
   "type": "module",
+  "files": [
+    "extensions/",
+    "src/"
+  ],
   "scripts": {
     "test": "tsx tests/core.test.ts"
   },

package/src/retriever.ts

@@ -6,12 +6,10 @@ export type MemorySource = "pi" | "claude" | "codex";
 export interface RecallOptions {
   query: string;
   entities?: string[];
-  topK?: number;
   sources?: MemorySource[];
   cwd?: string;
   after?: number;
   before?: number;
-  diversify?: boolean;
 }
 
 export interface RecallTurnResult {
@@ -28,6 +26,16 @@ export interface RecallTurnResult {
   score: number;
 }
 
+export interface FreshnessEvidence {
+  turn_id: string;
+  session_id: string;
+  turn_index: number;
+  source: MemorySource;
+  ts: number;
+  relation: "same_source_session_later" | "newer_cross_session";
+  excerpt: string;
+}
+
 export interface RecallDurableMemoryResult {
   type: "memory";
   memory_id: string;
@@ -38,7 +46,9 @@ export interface RecallDurableMemoryResult {
   source_session_id: string | null;
   source_content_hash: string | null;
   source_turn_index: number | null;
+  source_session_changed: boolean;
   freshness_candidate: boolean;
+  freshness_evidence: FreshnessEvidence[];
   created_at: number;
   last_confirmed_at: number;
   importance: number;
@@ -48,41 +58,59 @@ export interface RecallDurableMemoryResult {
 
 export type RecallResult = RecallDurableMemoryResult | RecallTurnResult;
 
-// Avoid allowing one long conversation to fill every raw-transcript recall slot.
-const MAX_TURNS_PER_SESSION = 2;
+export const RECALL_PAGE_SIZE = 5;
+
+export interface RecallPage {
+  results: RecallResult[];
+  offset: number;
+  totalResults: number;
+  nextOffset: number | null;
+}
+
 // Recency is a bounded tie-breaker, not a replacement for literal relevance.
 const RECENCY_WINDOW_MS = 90 * 24 * 60 * 60 * 1000;
 
 /** Retained for compatibility with the v0.1 public retrieval helper. */
-export function recallTurns(entities: string[], topK = 5): RecallTurnResult[] {
-  return _recallTurns({ query: entities.join(" "), entities, topK, diversify: false });
+export function recallTurns(entities: string[]): RecallTurnResult[] {
+  return _recallTurns({ query: entities.join(" "), entities });
 }
 
 /** Retrieve active durable memories, then raw turns not already represented by unchanged source evidence. */
 export function recallMemories(options: RecallOptions): RecallResult[] {
   const durableMemories = _recallDurableMemories(options);
   const recalledTurns = _recallTurns(options);
-  const freshnessCandidates = new Set(
-    durableMemories
-      .filter((memory) => memory.source_session_id && memory.source_turn_index !== null)
-      .filter((memory) => recalledTurns.some((turn) => turn.session_id === memory.source_session_id && turn.turn_index > memory.source_turn_index!))
-      .map((memory) => memory.memory_id),
-  );
-  const memories = durableMemories.map((memory) => ({ ...memory, freshness_candidate: freshnessCandidates.has(memory.memory_id) }));
+  const memories = durableMemories.map((memory) => {
+    const freshness_evidence = _freshnessEvidence(memory, recalledTurns);
+    return {
+      ...memory,
+      source_session_changed: _sourceSessionChanged(memory),
+      freshness_candidate: freshness_evidence.length > 0,
+      freshness_evidence,
+    };
+  });
   const coveredSourceHashes = new Map(
     memories
       .filter((memory) => memory.source_turn_id && memory.source_content_hash)
       .map((memory) => [memory.source_turn_id!, memory.source_content_hash!]),
   );
   const rawTurns = recalledTurns.filter((turn) => coveredSourceHashes.get(turn.turn_id) !== _turnContentHash(turn));
-  return [...memories, ...rawTurns].slice(0, options.topK ?? 5);
+  return [...memories, ...rawTurns];
+}
+
+/** Select one fixed-size recall page without limiting the complete local retrieval result. */
+export function paginateRecallResults(results: RecallResult[], offset = 0): RecallPage {
+  if (!Number.isInteger(offset) || offset < 0) throw new Error("Recall offset must be a non-negative integer");
+  const pageResults = results.slice(offset, offset + RECALL_PAGE_SIZE);
+  const nextOffset = offset + pageResults.length < results.length ? offset + pageResults.length : null;
+  return { results: pageResults, offset, totalResults: results.length, nextOffset };
 }
 
 /** Render the exact query inputs and recall results as concise Markdown for a command notification or tool response. */
-export function formatRecallResults(results: RecallResult[], options?: Pick<RecallOptions, "query" | "entities" | "sources" | "cwd" | "after" | "before">): string {
+export function formatRecallResults(results: RecallResult[], options?: Pick<RecallOptions, "query" | "entities" | "sources" | "cwd" | "after" | "before">, page?: Omit<RecallPage, "results">): string {
   const lines = options ? [_formatRecallQuery(options), ""] : [];
-  if (results.length === 0) return [...lines, "No relevant past conversations found."].join("\n");
+  if (results.length === 0) return [...lines, page && page.totalResults > 0 ? `No results at offset ${page.offset}; the matching result set contains ${page.totalResults} result(s).` : "No relevant past conversations found."].join("\n");
 
+  if (page) lines.push(`**Results:** ${page.offset + 1}–${page.offset + results.length} of ${page.totalResults} (five results per page)\n`);
   lines.push("## Relevant past memories\n");
   for (const result of results) {
     if (result.type === "memory") {
@@ -91,7 +119,14 @@ export function formatRecallResults(results: RecallResult[], options?: Pick<Reca
       lines.push(`**Memory ID:** ${result.memory_id}`);
       if (result.source_turn_id) lines.push(`**Source turn:** ${result.source_turn_id}`);
       if (result.source_session_id) lines.push(`**Source session:** ${result.source_session_id}`);
-      if (result.freshness_candidate) lines.push("**Freshness:** newer matching turn exists in the source session; confirm or supersede this memory.");
+      if (result.source_session_changed) lines.push("**Source session changed:** later turns exist; this alone does not mean the memory is stale.");
+      if (result.freshness_candidate) {
+        lines.push("**Newer evidence to compare:**");
+        for (const evidence of result.freshness_evidence) {
+          lines.push(`- [${evidence.relation.replaceAll("_", " ")} · ${evidence.source} · ${new Date(evidence.ts).toLocaleString()} · ${evidence.session_id} · turn ${evidence.turn_index}] ${evidence.excerpt}`);
+        }
+        lines.push("Compare this evidence with the durable memory; it may confirm, supplement, conflict with, or replace it. Do not change the memory without the user's explicit choice.");
+      }
     } else {
       const date = new Date(result.ts).toLocaleString();
       lines.push(`### [${result.source} · ${date}]`);
@@ -101,6 +136,9 @@ export function formatRecallResults(results: RecallResult[], options?: Pick<Reca
     }
     lines.push("");
   }
+  if (page?.nextOffset !== null && page?.nextOffset !== undefined) {
+    lines.push(`More matching results exist. To retrieve the next five, call \`recall_memory\` again with every same search/filter parameter and \`offset: ${page.nextOffset}\`.`);
+  }
   return lines.join("\n");
 }
 
@@ -149,9 +187,46 @@ function _recallDurableMemories(options: RecallOptions): RecallDurableMemoryResu
     FROM memories
     WHERE ${filters.join(" AND ")}
     ORDER BY hits DESC, importance DESC, last_confirmed_at DESC
-    LIMIT ?
-  `).all(...parameters, ...filterParameters, Math.max(options.topK ?? 5, 1))
-    .map((memory) => ({ ...memory, type: "memory" as const, freshness_candidate: false, score: memory.hits + memory.importance })) as RecallDurableMemoryResult[];
+  `).all(...parameters, ...filterParameters)
+    .map((memory) => ({ ...memory, type: "memory" as const, source_session_changed: false, freshness_candidate: false, freshness_evidence: [], score: memory.hits + memory.importance })) as RecallDurableMemoryResult[];
+}
+
+/** Detect later activity in the original session independently of this recall query and ranking. */
+function _sourceSessionChanged(memory: RecallDurableMemoryResult): boolean {
+  if (!memory.source_session_id || memory.source_turn_index === null) return false;
+  return getDb().prepare(`
+    SELECT 1 FROM turns
+    WHERE session_id = ? AND turn_index > ?
+    LIMIT 1
+  `).get(memory.source_session_id, memory.source_turn_index) !== undefined;
+}
+
+/** Associate each memory with all newer query-relevant turns without deciding their semantic relationship. */
+function _freshnessEvidence(memory: RecallDurableMemoryResult, recalledTurns: RecallTurnResult[]): FreshnessEvidence[] {
+  const baselineTs = _memoryEvidenceTimestamp(memory);
+  return recalledTurns
+    .filter((turn) => {
+      if (memory.source_session_id === turn.session_id && memory.source_turn_index !== null) {
+        return turn.turn_index > memory.source_turn_index;
+      }
+      return turn.ts > baselineTs;
+    })
+    .map((turn) => ({
+      turn_id: turn.turn_id,
+      session_id: turn.session_id,
+      turn_index: turn.turn_index,
+      source: turn.source,
+      ts: turn.ts,
+      relation: memory.source_session_id === turn.session_id ? "same_source_session_later" as const : "newer_cross_session" as const,
+      excerpt: _excerpt(turn.user_text || turn.reply_text),
+    }));
+}
+
+/** Use source-turn time when available; explicit memories become comparable from their creation time. */
+function _memoryEvidenceTimestamp(memory: RecallDurableMemoryResult): number {
+  if (!memory.source_turn_id) return memory.created_at;
+  const source = getDb().prepare("SELECT ts FROM turns WHERE turn_id = ?").get(memory.source_turn_id) as { ts: number } | undefined;
+  return source?.ts ?? memory.created_at;
 }
 
 /** Retrieve and rank locally stored turns using literal query terms and optional scopes. */
@@ -185,11 +260,11 @@ function _recallTurns(options: RecallOptions): RecallTurnResult[] {
       (${scoreExpression}) AS hits
     FROM turns JOIN sessions ON sessions.session_id = turns.session_id
     WHERE ${filters.join(" AND ")}
-    ORDER BY hits DESC, turns.ts DESC LIMIT ?
-  `).all(...scoreParameters, ...filterParameters, Math.max(options.topK ?? 5, 1) * 10) as Array<Omit<RecallTurnResult, "type" | "score">>;
+    ORDER BY hits DESC, turns.ts DESC
+  `).all(...scoreParameters, ...filterParameters) as Array<Omit<RecallTurnResult, "type" | "score">>;
   const newestTs = candidates.reduce((newest, result) => Math.max(newest, result.ts), 0);
   const results = candidates.map((result) => ({ ...result, type: "turn" as const, score: result.hits + _recencyScore(result.ts, newestTs) + (options.cwd === result.cwd ? 0.8 : 0) })).sort((left, right) => right.score - left.score || right.ts - left.ts);
-  return options.diversify === false ? results.slice(0, options.topK ?? 5) : _diversify(results, options.topK ?? 5);
+  return results;
 }
 
 /** Hash the current raw turn evidence using the same representation captured during pinning. */
@@ -217,14 +292,3 @@ function _likePattern(term: string): string {
 function _recencyScore(ts: number, newestTs: number): number {
   return Math.max(0, 0.5 * (1 - (newestTs - ts) / RECENCY_WINDOW_MS));
 }
-
-/** Limit ranked output to prevent any one session from dominating the recall window. */
-function _diversify(results: RecallTurnResult[], topK: number): RecallTurnResult[] {
-  const counts = new Map<string, number>();
-  return results.filter((result) => {
-    const count = counts.get(result.session_id) ?? 0;
-    if (count >= MAX_TURNS_PER_SESSION) return false;
-    counts.set(result.session_id, count + 1);
-    return true;
-  }).slice(0, topK);
-}

package/AGENTS.md

@@ -1,5 +0,0 @@
-# Project Instructions
-
-## Command and tool discoverability
-
-Every user-facing command must have a precise `description` explaining what it does and when to use it. Commands exist so users can invoke them manually, but an equivalent capability that an agent may need must also be exposed as a clearly described Pi tool. Tool descriptions must state the appropriate invocation conditions so Pi can discover and call them without relying on hidden knowledge.

package/spec.md

@@ -1,28 +0,0 @@
-# Native Codex-to-Pi Project Session Migration — Spec
-
-## Goal
-
-Allow a user to convert each historical Codex session for the active project into a separate, native Pi session that can be selected through Pi `/resume` and continued normally.
-
-This is distinct from SQLite historical import and `recall_memory`:
-
-- Native migration writes Pi session JSONL files for direct continuation in Pi.
-- `recall_memory` searches local SQLite excerpts and does not restore a client session.
-
-## Design
-
-- Export `migrateCodexProjectSessions(cwd)` from `src/session-migration.ts`.
-- Scan Codex JSONL sessions and select only sessions with `session.cwd === cwd`.
-- Create one Pi v3 session JSONL per Codex session under Pi's default session directory for that cwd.
-- Write a `Migrated from Codex: <session-id>` session name so it is recognizable in `/resume`.
-- Convert user and assistant textual messages only. Do not represent Codex system/developer prompts, tool calls, or tool results as Pi conversation messages.
-- Use deterministic output file names and skip an already migrated Codex session, making reruns idempotent.
-- Register `/project-session-migration` for users and `migrate_codex_project_sessions` for Pi agents. Both descriptions must state that this is native Pi continuation, not ordinary recall.
-
-## Acceptance criteria
-
-1. Each current-project Codex fixture produces one independently resumable Pi session with the expected user/assistant message sequence.
-2. Another project's Codex session is not migrated.
-3. A second migration skips existing output session files.
-4. A malformed source file becomes an isolated issue and does not block other sessions.
-5. `npm test` and `git diff --check` pass.

package/tests/core.test.ts

@@ -1,287 +0,0 @@
-import assert from "node:assert/strict";
-import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
-import { tmpdir } from "node:os";
-import { join } from "node:path";
-
-const dbPath = join(tmpdir(), `pi-session-memory-${process.pid}.db`);
-const historyHome = join(tmpdir(), `pi-session-memory-home-${process.pid}`);
-process.env.MEMORY_DB_PATH = dbPath;
-process.env.HOME = historyHome;
-
-const { confirmMemory, createMemory, deleteMemory, deleteTurn, getDb, getMemoryHistory, getMemoryStats, getSession, insertTurn, listMemories, pinTurnAsMemory, supersedeMemory, upsertSession } = await import("../src/db.ts");
-const { formatRecallResults, recallMemories, recallTurns } = await import("../src/retriever.ts");
-const { HISTORY_SCHEMA_REFERENCE_VERSIONS, backfillAll } = await import("../src/backfill.ts");
-const { migrateCodexProjectSessions } = await import("../src/session-migration.ts");
-const { SessionManager } = await import("@earendil-works/pi-coding-agent");
-
-/** Remove the temporary SQLite database and its WAL sidecar files after this test. */
-function cleanup(): void {
-  for (const suffix of ["", "-wal", "-shm"]) {
-    const path = `${dbPath}${suffix}`;
-    if (existsSync(path)) rmSync(path);
-  }
-  if (existsSync(historyHome)) rmSync(historyHome, { recursive: true });
-}
-
-cleanup();
-
-upsertSession({
-  session_id: "pi:test",
-  source: "pi",
-  cwd: "/tmp",
-  started_at: 1,
-  model_id: null,
-  jsonl_path: "/tmp/test.jsonl",
-});
-
-assert.equal(insertTurn({
-  turn_id: "pi:test:user-1",
-  session_id: "pi:test",
-  turn_index: 0,
-  ts: 1,
-  user_text: "literal 100% and a_b",
-  reply_text: "first reply",
-  tool_names: null,
-  user_message_id: "user-1",
-}), true);
-
-assert.equal(insertTurn({
-  turn_id: "pi:test:user-2",
-  session_id: "pi:test",
-  turn_index: 1,
-  ts: 2,
-  user_text: "wildcard 100x and acb",
-  reply_text: "second reply",
-  tool_names: null,
-  user_message_id: "user-2",
-}), true);
-
-assert.equal(insertTurn({
-  turn_id: "pi:test:user-1",
-  session_id: "pi:test",
-  turn_index: 0,
-  ts: 1,
-  user_text: "duplicate",
-  reply_text: "duplicate",
-  tool_names: null,
-  user_message_id: "user-1",
-}), false);
-
-assert.throws(() => insertTurn({
-  turn_id: "pi:test:invalid-message-id",
-  session_id: "pi:test",
-  turn_index: 2,
-  ts: 3,
-  user_text: "invalid SQLite parameter",
-  reply_text: "",
-  tool_names: null,
-  user_message_id: undefined as unknown as string,
-}), /SQLite parameter 8 must be string, number, bigint, Uint8Array, or null; received undefined/);
-
-mkdirSync(join(historyHome, ".pi", "agent", "sessions"), { recursive: true });
-mkdirSync(join(historyHome, ".claude", "projects"), { recursive: true });
-mkdirSync(join(historyHome, ".codex", "sessions"), { recursive: true });
-writeFileSync(join(historyHome, ".pi", "agent", "sessions", "invalid.jsonl"), [
-  JSON.stringify({ type: "session", id: "pi-invalid", timestamp: "2026-01-01T00:00:00.000Z", cwd: "/tmp" }),
-  JSON.stringify({ type: "message", message: { role: "user", timestamp: 1, content: [{ type: "text", text: "invalid Pi id" }] } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".pi", "agent", "sessions", "valid.jsonl"), [
-  JSON.stringify({ type: "session", id: "pi-compatible", timestamp: "2026-01-01T00:00:00.000Z", cwd: "/tmp" }),
-  JSON.stringify({ type: "message", id: "pi-user", message: { role: "user", timestamp: 1, content: [{ type: "text", text: "schema normal Pi request" }] } }),
-  JSON.stringify({ type: "message", id: "pi-assistant", message: { role: "assistant", timestamp: 2, content: [{ type: "text", text: "schema normal Pi reply" }] } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".claude", "projects", "invalid.jsonl"), [
-  JSON.stringify({ type: "user", sessionId: "claude-invalid", cwd: "/tmp", timestamp: "2026-01-01T00:00:00.000Z", message: { role: "user", content: "invalid Claude id" } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".claude", "projects", "valid-old-schema.jsonl"), [
-  JSON.stringify({ type: "user", id: "claude-user", sessionId: "claude-compatible", cwd: "/tmp", timestamp: "2026-01-01T00:00:00.000Z", message: { role: "user", content: "schema normal Claude request" } }),
-  JSON.stringify({ type: "assistant", id: "claude-assistant", sessionId: "claude-compatible", cwd: "/tmp", timestamp: "2026-01-01T00:00:01.000Z", message: { role: "assistant", content: [{ type: "text", text: "schema normal Claude reply" }] } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".codex", "sessions", "invalid.jsonl"), [
-  JSON.stringify({ type: "session_meta", payload: { session_id: "codex-invalid", cwd: "/tmp", timestamp: "2026-01-01T00:00:00.000Z" } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-01-01T00:00:00.000Z", payload: { type: "message", role: "user", content: [{ type: "input_text", text: "invalid Codex id" }] } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".codex", "sessions", "valid.jsonl"), [
-  JSON.stringify({ type: "session_meta", payload: { session_id: "codex-compatible", cwd: "/tmp", timestamp: "2026-01-01T00:00:00.000Z" } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-01-01T00:00:00.000Z", payload: { type: "message", id: "codex-user", role: "user", content: [{ type: "input_text", text: "schema normal Codex request" }] } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-01-01T00:00:01.000Z", payload: { type: "message", id: "codex-assistant", role: "assistant", content: [{ type: "output_text", text: "schema normal Codex reply" }] } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".codex", "sessions", "valid-legacy.jsonl"), [
-  JSON.stringify({ type: "session_meta", payload: { session_id: "codex-legacy", cwd: "/tmp", timestamp: "2026-07-17T03:03:23.000Z" } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-07-17T03:03:24.000Z", payload: { type: "message", role: "user", content: [{ type: "input_text", text: "schema legacy Codex request" }], internal_chat_message_metadata_passthrough: { turn_id: "legacy-codex-turn" } } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-07-17T03:03:25.000Z", payload: { type: "message", role: "assistant", content: [{ type: "output_text", text: "schema legacy Codex reply" }], internal_chat_message_metadata_passthrough: { turn_id: "legacy-codex-turn" } } }),
-].join("\n"));
-writeFileSync(join(historyHome, ".codex", "sessions", "other-project.jsonl"), [
-  JSON.stringify({ type: "session_meta", payload: { session_id: "codex-other-project", cwd: "/other-project", timestamp: "2026-01-01T00:00:00.000Z" } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-01-01T00:00:00.000Z", payload: { type: "message", id: "codex-other-user", role: "user", content: [{ type: "input_text", text: "other project request" }] } }),
-  JSON.stringify({ type: "response_item", timestamp: "2026-01-01T00:00:01.000Z", payload: { type: "message", id: "codex-other-assistant", role: "assistant", content: [{ type: "output_text", text: "other project reply" }] } }),
-].join("\n"));
-const migrationStats = migrateCodexProjectSessions("/tmp");
-assert.deepEqual({ scannedFiles: migrationStats.scannedFiles, migratedSessions: migrationStats.migratedSessions, skippedSessions: migrationStats.skippedSessions, migratedMessages: migrationStats.migratedMessages, issues: migrationStats.issues.length }, { scannedFiles: 4, migratedSessions: 2, skippedSessions: 0, migratedMessages: 4, issues: 1 });
-assert.match(migrationStats.issues[0].error, /no stable native ID/);
-const migratedSessionFiles = [...(await SessionManager.list("/tmp"))].filter((session) => session.name?.startsWith("Migrated from Codex:"));
-assert.deepEqual(migratedSessionFiles.map((session) => session.name).sort(), ["Migrated from Codex: codex-compatible", "Migrated from Codex: codex-legacy"]);
-const migrated = SessionManager.open(migratedSessionFiles.find((session) => session.name === "Migrated from Codex: codex-compatible")!.path);
-assert.deepEqual(migrated.getBranch().filter((entry) => entry.type === "message").map((entry) => entry.message.role), ["user", "assistant"]);
-assert.equal(migrateCodexProjectSessions("/tmp").skippedSessions, 2);
-assert.equal((await SessionManager.list("/other-project")).some((session) => session.name === "Migrated from Codex: codex-other-project"), false);
-const backfillStats = backfillAll();
-assert.deepEqual({ pi: backfillStats.pi, claude: backfillStats.claude, codex: backfillStats.codex, turns: backfillStats.turns }, { pi: 3, claude: 1, codex: 3, turns: 7 });
-assert.deepEqual(backfillStats.issues.map((issue) => issue.source), ["pi", "claude", "codex"]);
-for (const issue of backfillStats.issues) {
-  assert.match(issue.error, new RegExp(`${issue.source} history import failed[\\s\\S]*supported reference ${HISTORY_SCHEMA_REFERENCE_VERSIONS[issue.source]}`));
-}
-assert.deepEqual(HISTORY_SCHEMA_REFERENCE_VERSIONS, { pi: "0.85.1", claude: "2.1.234", codex: "0.154.0" });
-assert.deepEqual(recallTurns(["schema normal"], 10).map((result) => result.turn_id).filter((turnId) => !turnId.startsWith("pi:") || turnId === "pi:pi-compatible:pi-user"), [
-  "claude:claude-compatible:claude-user",
-  "codex:codex-compatible:codex-user",
-  "pi:pi-compatible:pi-user",
-]);
-assert.equal(recallTurns(["schema legacy Codex"], 10).some((result) => result.turn_id === "codex:codex-legacy:legacy-codex-turn"), true);
-
-assert.deepEqual(recallTurns(["100%"], 10).map((result) => result.turn_id), ["pi:test:user-1"]);
-assert.deepEqual(recallTurns(["a_b"], 10).map((result) => result.turn_id), ["pi:test:user-1"]);
-const thinRecall = formatRecallResults(recallMemories({ query: "literal", topK: 1 }));
-assert.match(thinRecall, /\*\*Session:\*\* pi:test · \*\*Turn:\*\* 0[\s\S]*\*\*Excerpt:\*\* literal 100% and a_b[\s\S]*Use `fetch_session`/);
-assert.doesNotMatch(thinRecall, /\*\*Assistant:\*\* first reply/);
-assert.match(
-  formatRecallResults([], { query: "SAP BTP", entities: ["BTP", "Business Technology Platform"], sources: ["pi"], cwd: "/tmp" }),
-  /\*\*Search query:\*\* `SAP BTP`[\s\S]*\*\*Filters:\*\* entities: `BTP`, `Business Technology Platform` · sources: pi · cwd: `\/tmp`[\s\S]*No relevant past conversations found\./,
-);
-upsertSession({
-  session_id: "claude:project-a",
-  source: "claude",
-  cwd: "/workspace/project-a",
-  started_at: 10,
-  model_id: null,
-  jsonl_path: "/tmp/project-a.jsonl",
-});
-upsertSession({
-  session_id: "codex:project-b",
-  source: "codex",
-  cwd: "/workspace/project-b",
-  started_at: 20,
-  model_id: null,
-  jsonl_path: "/tmp/project-b.jsonl",
-});
-for (const [turnId, sessionId, index, ts, text] of [
-  ["claude:project-a:user-1", "claude:project-a", 0, 1_000, "deploy memory ranking"],
-  ["claude:project-a:user-2", "claude:project-a", 1, 2_000, "deploy memory testing"],
-  ["claude:project-a:user-3", "claude:project-a", 2, 3_000, "deploy memory release"],
-  ["codex:project-b:user-1", "codex:project-b", 0, 4_000, "deploy memory ranking"],
-] as Array<[string, string, number, number, string]>) {
-  assert.equal(insertTurn({
-    turn_id: turnId,
-    session_id: sessionId,
-    turn_index: index,
-    ts,
-    user_text: text,
-    reply_text: "confirmed",
-    tool_names: null,
-    user_message_id: turnId.split(":").at(-1)!,
-  }), true);
-}
-
-assert.deepEqual(
-  recallMemories({ query: "deploy memory", cwd: "/workspace/project-a", topK: 10 }).map((result) => result.turn_id),
-  ["claude:project-a:user-3", "claude:project-a:user-2"],
-);
-assert.deepEqual(
-  recallMemories({ query: "deploy memory", sources: ["codex"], after: 4_000, topK: 10 }).map((result) => result.turn_id),
-  ["codex:project-b:user-1"],
-);
-assert.deepEqual(
-  recallMemories({ query: "deploy memory", topK: 10 }).map((result) => result.type === "turn" ? result.turn_id : result.memory_id),
-  ["codex:project-b:user-1", "claude:project-a:user-3", "claude:project-a:user-2"],
-);
-
-const explicitMemory = createMemory({
-  memory_id: "memory:explicit",
-  kind: "decision",
-  content: "Use SQLite durable memory for deploy decisions.",
-  project_key: "/workspace/project-a",
-  source_turn_id: null,
-  importance: 2,
-  created_at: 5_000,
-});
-const pinnedMemory = pinTurnAsMemory("claude:project-a:user-1");
-assert.equal(pinnedMemory.source_turn_id, "claude:project-a:user-1");
-assert.equal(pinnedMemory.source_session_id, "claude:project-a");
-assert.ok(pinnedMemory.source_content_hash);
-assert.deepEqual(listMemories("decision").map((memory) => memory.memory_id), [explicitMemory.memory_id]);
-const deployRecall = recallMemories({ query: "deploy", cwd: "/workspace/project-a", topK: 10 });
-assert.deepEqual(
-  deployRecall.map((result) => result.type === "memory" ? result.memory_id : result.turn_id),
-  [explicitMemory.memory_id, pinnedMemory.memory_id, "claude:project-a:user-3", "claude:project-a:user-2"],
-);
-assert.equal(deployRecall.find((result) => result.type === "memory" && result.memory_id === pinnedMemory.memory_id)?.freshness_candidate, true);
-const confirmedMemory = confirmMemory(pinnedMemory.memory_id);
-assert.ok(confirmedMemory.last_confirmed_at >= pinnedMemory.last_confirmed_at);
-const replacementMemory = createMemory({
-  memory_id: "memory:replacement",
-  kind: "decision",
-  content: "Use reviewed SQLite durable memory for deploy decisions.",
-  project_key: "/workspace/project-a",
-  source_turn_id: null,
-  importance: 2,
-});
-supersedeMemory(explicitMemory.memory_id, replacementMemory.memory_id);
-assert.deepEqual(getMemoryHistory(replacementMemory.memory_id).map((memory) => memory.memory_id), [explicitMemory.memory_id, replacementMemory.memory_id]);
-assert.deepEqual(
-  recallMemories({ query: "SQLite durable memory", cwd: "/workspace/project-a", topK: 10 }).filter((result) => result.type === "memory").map((result) => result.memory_id),
-  [replacementMemory.memory_id],
-);
-assert.throws(() => supersedeMemory(explicitMemory.memory_id, replacementMemory.memory_id), /already superseded/);
-upsertSession({
-  session_id: "pi:provenance",
-  source: "pi",
-  cwd: "/workspace/provenance",
-  started_at: 6_000,
-  model_id: null,
-  jsonl_path: "/tmp/provenance.jsonl",
-});
-assert.equal(insertTurn({
-  turn_id: "pi:provenance:user-1",
-  session_id: "pi:provenance",
-  turn_index: 0,
-  ts: 6_000,
-  user_text: "provenance deduplication",
-  reply_text: "original source evidence",
-  tool_names: null,
-  user_message_id: "user-1",
-}), true);
-const provenanceMemory = pinTurnAsMemory("pi:provenance:user-1");
-assert.deepEqual(
-  recallMemories({ query: "provenance deduplication", cwd: "/workspace/provenance", topK: 10 }).map((result) => result.type === "memory" ? result.memory_id : result.turn_id),
-  [provenanceMemory.memory_id],
-);
-getDb().prepare("UPDATE turns SET reply_text = ? WHERE turn_id = ?").run("changed source evidence", "pi:provenance:user-1");
-assert.deepEqual(
-  recallMemories({ query: "provenance deduplication", cwd: "/workspace/provenance", topK: 10 }).map((result) => result.type === "memory" ? result.memory_id : result.turn_id),
-  [provenanceMemory.memory_id, "pi:provenance:user-1"],
-);
-assert.deepEqual(
-  getSession("claude:project-a", 1, 2).turns.map((turn) => [turn.turn_index, turn.turn_id]),
-  [[1, "claude:project-a:user-2"], [2, "claude:project-a:user-3"]],
-);
-assert.throws(() => getSession("missing:session"), /Memory session not found/);
-assert.equal(deleteTurn("claude:project-a:user-1"), true);
-assert.equal(listMemories().some((memory) => memory.memory_id === pinnedMemory.memory_id), true);
-assert.equal(deleteMemory(pinnedMemory.memory_id), true);
-assert.equal(deleteMemory(pinnedMemory.memory_id), false);
-
-const stats = getMemoryStats();
-assert.equal(stats.sessions, 11);
-assert.equal(stats.turns, 13);
-assert.deepEqual([...stats.sources].map(({ source, sessions, turns }) => ({ source, sessions, turns })), [
-  { source: "claude", sessions: 2, turns: 3 },
-  { source: "codex", sessions: 4, turns: 4 },
-  { source: "pi", sessions: 5, turns: 6 },
-]);
-assert.equal(deleteTurn("codex:project-b:user-1"), true);
-assert.equal(deleteTurn("codex:project-b:user-1"), false);
-assert.equal(getDb().prepare("SELECT count(*) AS count FROM sessions WHERE session_id = 'codex:project-b'").get().count, 0);
-assert.equal(getDb().prepare("SELECT count(*) AS count FROM turns").get().count, 12);
-
-cleanup();
-console.log("core.test.ts: passed");

package/tsconfig.json

@@ -1,10 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "ES2022",
-    "module": "ESNext",
-    "moduleResolution": "bundler",
-    "strict": true,
-    "allowImportingTsExtensions": true,
-    "noEmit": true
-  }
-}