@chendpoc/pi-memory 0.2.4 → 0.3.2

package/doc/ROADMAP-zh.md

@@ -0,0 +1,114 @@
+# @chendpoc/pi-memory 路线图
+
+<p align="center">
+  <a href="ROADMAP.md">English</a> |
+  <a href="ROADMAP-zh.md">简体中文</a>
+</p>
+
+这是 `@chendpoc/pi-memory` 的公开产品路线图。它聚焦用户可感知的结果，不链接未随 npm 包发布的内部 `dev-doc/` 设计笔记。
+
+## 版本规划
+
+| 版本 | 主题 | 用户可感知结果 |
+| --- | --- | --- |
+| **0.3.x** | 信任与安全 | 记忆写入更安全、有边界，也更容易诊断。 |
+| **0.4.x** | 召回质量 | Pi 在 Preflight 预算内更常召回正确记忆。 |
+| **0.5.x** | 记忆生命周期 | 用户能审查、淘汰、纠正长期记忆，减少手动清理。 |
+| **0.6.x** | 可观测 + 控制面 | 用户能检查、排障、编辑、审查，并在本地迁移 Pi 记忆。 |
+| **0.7.x** | 可选同步 | 在不强制上云的前提下，支持多 agent / 多设备协同记忆。 |
+
+进入 **0.7** 之前，必须先完成三件事：debug/trace、memory edit/review、本地 import/export。Cloud 或多 agent 同步应保持可选，并默认 local-first。
+
+## 产品原则
+
+- `MEMORY.md` 继续作为可编辑的 Ground Truth。
+- derived index、sidecar、trace、未来 sync state 都必须能从本地记忆工件重建。
+- 召回应改善普通 agent 行为，但不能给每轮对话加入多秒级反思。
+- 安全能力对记忆写入应 fail closed；对 agent 回答应 fail open。
+- `pi-memory` 不应变成完整 transcript search 产品；旧会话搜索应由专门的 session-search 包负责。
+
+## 当前基础
+
+- 带 overflow 的 `MEMORY.md` Ground Truth。
+- `/remember` 和 `/memory-status`。
+- 基于 UDS JSONL 的 sidecar。
+- `memory.vec.sqlite` 向量索引。
+- QueryIntent + raw-query fallback。
+- 800ms 共享 Preflight 预算，sidecar query 支持 AbortSignal 取消。
+- sidecar warm、intent cache、query cache。
+- dual-purpose compaction summary。
+- Shutdown Queue + `maintenance`。
+- Consolidate + reindex。
+- Subagent Memory Cap + Compact Delta。
+- Ground Truth 写入前 secret 脱敏。
+- `MemoryRuntime` 扩展生命周期与重构后的 store/sidecar 模块。
+
+## P0：信任与安全
+
+**目标版本：0.3.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/1)
+
+用户目标：记忆能力足够安全，可以默认开启。
+
+- ✅ durable memory 写入前做 secret/token redaction — **0.3.0 已交付**。
+- ✅ 拆小并澄清 store、ingest、status、lifecycle、sidecar 等运行模块 — **0.3.0 已交付**。
+- 为 LLM 生成的 Memory Export 增加 prompt-injection guardrails。
+- 针对用户明确纠正做 correction detector，优先限定在 `/remember` 链路。
+- 为跳过写入和 fallback 决策提供轻量 reason code；完整 trace UX 放在 **0.6**。
+
+## P1：召回质量
+
+**目标版本：0.4.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/2)
+
+用户目标：Pi 更常召回正确的稳定事实，同时保持交互轮次足够快。
+
+- 对 `MEMORY.md` 条目做 lexical + vector 混合召回：SQLite FTS5 + 现有 vector chunks，RRF merge 后再 MMR。
+- 为常见 coding-agent 问题建立 recall eval fixtures。
+- debug metrics 拆分 intent、embed、scan、MMR、render、fallback、总 Preflight 时间。
+- 改进 Ollama 之外的 embedding provider，在可行时支持免费/开源 embedding API。
+- sidecar 检索优化：增量 reindex、batch embed、更清晰的 reindex/query 隔离。
+- 只有在延迟预算允许时，才在 MMR 后加入可选 reranker。
+
+## P2：记忆生命周期
+
+**目标版本：0.5.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/3)
+
+用户目标：长期记忆保持有用，而不是变成不断增长的过期事实堆。
+
+- 增加 failure 和 tool-quirk 类别。
+- 晋升前提供人类可审查的 memory draft 或 diary。
+- 基于使用信号的晋升和裁剪。
+- rewrite 前提供更安全的 consolidate preview。
+- 显式处理 stale / superseded fact，而不是只做简单 TODO 清理。
+- lifecycle 事件进入共享 debug/trace event schema；面向用户的 trace surface 在 **0.6** 交付。
+
+## P3：可观测 + 控制面
+
+**目标版本：0.6.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/4)
+
+用户目标：当记忆行为不符合预期时，用户能看懂原因并修正，而不必阅读底层内部状态。
+
+必须完成的出口标准：
+
+- 本地 trace ring buffer，例如 `logs/trace.jsonl`，包含稳定 event name 和 reason code。
+- `pi-memory status --verbose` 与 `/memory-status` 能总结最近 skip、fallback、write、reindex、recall 事件。
+- memory edit/review 命令，并配合 **0.5** 的 stale/supersede 语义。
+- 不同 Pi 安装之间的本地 import/export，并提供 dry-run，为未来 sync 做准备。
+
+0.6 可选 backlog：
+
+- 更好的 TUI status panel。
+- 常见 Pi 配置的文档 recipes。
+- 剩余 sidecar tuning，例如并行 raw+intent recall、MMR lambda 调参、consolidate 后 recall 抽检，由 p99 延迟和 eval 数据驱动。
+
+## P4：可选同步
+
+**目标版本：0.7.x**
+
+用户目标：跨多个 Pi agent 或设备工作时，可以协同记忆，但不放弃本地控制权。
+
+- 多 agent 跨 session 记忆协同。
+- 可选 cloud Pi agent <-> local Pi agent session-memory sync。
+- `MEMORY.md` 仍是 source of truth；vector index 在各设备本地 rebuild。
+- 默认不自动上传。
+- sync 设计必须先定义冲突处理、trust boundary、redaction guarantee、rollback，再进入实现。
+- 前置条件：injection + redaction 闭环、stale/edit 语义、本地 import/export、debug/trace 能排障跨进程写入。

package/doc/ROADMAP.md

@@ -0,0 +1,114 @@
+# @chendpoc/pi-memory Roadmap
+
+<p align="center">
+  <a href="ROADMAP.md">English</a> |
+  <a href="ROADMAP-zh.md">简体中文</a>
+</p>
+
+This is the public product roadmap for `@chendpoc/pi-memory`. It focuses on user-visible outcomes and intentionally avoids linking to internal `dev-doc/` design notes that are not shipped in the npm package.
+
+## Version Map
+
+| Version | Theme | User-facing outcome |
+| --- | --- | --- |
+| **0.3.x** | Trust and safety | Memory writes are safer, bounded, and easier to diagnose. |
+| **0.4.x** | Recall quality | Pi retrieves the right memory more often under the Preflight budget. |
+| **0.5.x** | Memory lifecycle | Users can review, age out, and correct long-lived memory with less manual cleanup. |
+| **0.6.x** | Observability + controls | Users can inspect, debug, edit, review, and locally move memory between Pi installs. |
+| **0.7.x** | Optional sync | Local-first memory can coordinate across agents or devices without making cloud sync mandatory. |
+
+Before **0.7**, three things must be real: debug/trace, memory edit/review, and local import/export. Cloud or multi-agent sync should remain optional and local-first by default.
+
+## Product Principles
+
+- `MEMORY.md` remains the editable Ground Truth.
+- Derived indexes, sidecars, traces, and future sync state must be rebuildable from local memory artifacts.
+- Recall should improve ordinary agent behavior without adding multi-second thinking to every turn.
+- Safety features should fail closed for memory writes and fail open for agent turns.
+- `pi-memory` should not become a full transcript search product; that belongs in a dedicated session-search package.
+
+## Current Foundation
+
+- `MEMORY.md` Ground Truth with overflow.
+- `/remember` and `/memory-status`.
+- Sidecar over UDS JSONL.
+- `memory.vec.sqlite` vector index.
+- QueryIntent with raw-query fallback.
+- 800ms shared Preflight budget with AbortSignal-aware sidecar query.
+- Warm sidecar, intent cache, query cache.
+- Dual-purpose compaction summary.
+- Shutdown Queue + `maintenance`.
+- Consolidate + reindex.
+- Subagent Memory Cap + Compact Delta.
+- Secret redaction before Ground Truth writes.
+- `MemoryRuntime` extension lifecycle and refactored store/sidecar modules.
+
+## P0: Trust And Safety
+
+**Target: 0.3.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/1)
+
+User-facing goal: memory should be safe enough to keep enabled by default.
+
+- ✅ Secret and token redaction before durable memory writes — **shipped in 0.3.0**.
+- ✅ Smaller, clearer runtime modules for store, ingest, status, lifecycle, and sidecar behavior — **shipped in 0.3.0**.
+- Prompt-injection guardrails for LLM-generated Memory Export.
+- Correction detector for explicit user corrections, scoped first to the `/remember` path.
+- Lightweight reason codes for skipped writes and fallback decisions; full trace UX is planned for **0.6**.
+
+## P1: Recall Quality
+
+**Target: 0.4.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/2)
+
+User-facing goal: Pi should recall the right stable facts more often, while staying fast enough for interactive turns.
+
+- Hybrid lexical + vector recall for `MEMORY.md` entries: SQLite FTS5 plus existing vector chunks, RRF merge, then MMR.
+- Recall eval fixtures for common coding-agent questions.
+- Debug metrics split by intent, embed, scan, MMR, render, fallback, and total Preflight time.
+- Embedding provider improvements beyond Ollama, including free/open-source embedding APIs where practical.
+- Sidecar retrieval optimizations: incremental reindex, batch embed, and clearer reindex/query isolation.
+- Optional reranker after MMR only when it fits the latency budget.
+
+## P2: Memory Lifecycle
+
+**Target: 0.5.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/3)
+
+User-facing goal: long-lived memory should stay useful instead of becoming a growing pile of stale facts.
+
+- Failure and tool-quirk categories.
+- Human-reviewable memory draft or diary before promotion.
+- Usage-weighted promotion and pruning.
+- Safer consolidate preview before rewrite.
+- Explicit stale/superseded fact handling instead of simple TODO pruning.
+- Lifecycle events feed the shared debug/trace event schema; the user-facing trace surfaces ship in **0.6**.
+
+## P3: Observability + Controls
+
+**Target: 0.6.x** · [GitHub milestone](https://github.com/chendpoc/pi-memory/milestone/4)
+
+User-facing goal: when memory behaves unexpectedly, users can see why and fix it without reading raw internals.
+
+Required exit criteria:
+
+- Local trace ring buffer, for example `logs/trace.jsonl`, with stable event names and reason codes.
+- `pi-memory status --verbose` and `/memory-status` surfaces that summarize recent skip, fallback, write, reindex, and recall events.
+- Memory edit/review commands paired with stale/supersede semantics from **0.5**.
+- Local import/export between Pi installations, with a dry-run mode before any future sync work.
+
+Optional backlog for 0.6:
+
+- Better TUI status panel.
+- Documentation recipes for common Pi setups.
+- Remaining sidecar tuning such as parallel raw+intent recall, MMR lambda tuning, and post-consolidate recall spot checks, driven by p99 latency and eval data.
+
+## P4: Optional Sync
+
+**Target: 0.7.x**
+
+User-facing goal: users who work across multiple Pi agents or devices can coordinate memory without giving up local control.
+
+- Multi-agent cross-session memory coordination.
+- Optional cloud Pi agent <-> local Pi agent session-memory sync.
+- `MEMORY.md` remains the source of truth; vector indexes are rebuilt per device.
+- No automatic upload by default.
+- Sync design must define conflict handling, trust boundaries, redaction guarantees, and rollback before implementation.
+- Prerequisites: injection + redaction closed loop, stale/edit semantics, local import/export, and debug/trace for cross-process write troubleshooting.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@chendpoc/pi-memory",
-  "version": "0.2.4",
+  "version": "0.3.2",
   "description": "Cross-session episodic memory for Pi agent — MEMORY.md ground truth, JSONL sidecar retrieval, and Preflight injection",
   "type": "module",
   "main": "./dist/index.js",
@@ -38,7 +38,15 @@
   },
   "files": [
     "dist",
+    "doc/README-zh.md",
+    "doc/ROADMAP.md",
+    "doc/ROADMAP-zh.md",
+    "doc/LAUNCH-KIT.md",
     "templates/MEMORY.md.example",
+    "templates/com.pi.memory.consolidate.plist.example",
+    "templates/consolidate.cmd.example",
+    "templates/crontab.example",
+    "templates/schtasks.example.txt",
     "scripts/path-utils.mjs",
     "scripts/init-memory-workspace.mjs",
     "scripts/platform-fs.mjs",
@@ -55,6 +63,11 @@
     "pi-extension",
     "pi-agent",
     "episodic-memory",
+    "agent-memory",
+    "local-memory",
+    "markdown-memory",
+    "coding-agent",
+    "ai-memory",
     "recent-memory",
     "memory-recall"
   ],
@@ -69,7 +82,7 @@
     "extensions": [
       "./dist/pi-extension.js"
     ],
-    "image": "https://raw.githubusercontent.com/chendpoc/pi-memory/v0.2.4/assets/gallery/pi-memory-gallery.png"
+    "image": "https://raw.githubusercontent.com/chendpoc/pi-memory/v0.3.1/assets/gallery/pi-memory-gallery.png"
   },
   "peerDependencies": {
     "@earendil-works/pi-agent-core": "*",
@@ -89,7 +102,6 @@
     "@earendil-works/pi-coding-agent": "*",
     "@earendil-works/pi-tui": "*",
     "@types/better-sqlite3": "^7.6.0",
-    "@types/lodash": "^4.17.24",
     "@types/node": "^24.0.0",
     "typebox": "^1.3.3",
     "typescript": "^5.7.2",
@@ -102,8 +114,8 @@
     "chalk": "^5.6.2",
     "dayjs": "^1.11.21",
     "dotenv": "^17.4.2",
+    "es-toolkit": "^1.49.0",
     "execa": "^9.6.1",
-    "lodash": "^4.18.1",
     "lru-cache": "^11.5.1",
     "proper-lockfile": "^4.1.2"
   }

package/scripts/postinstall.mjs

@@ -1,6 +1,7 @@
 #!/usr/bin/env node
 /**
  * Prefer compiled `pi-memory init` when dist exists; otherwise pre-build JS fallback.
+ * Scheduler sync is best-effort: failures must not fail npm install.
  */
 import { existsSync } from "node:fs";
 import { spawnSync } from "node:child_process";
@@ -10,8 +11,17 @@ import { fileURLToPath } from "node:url";
 const packageRoot = join(dirname(fileURLToPath(import.meta.url)), "..");
 const cli = join(packageRoot, "dist", "cli.js");
 
+function runBestEffort(args) {
+  try {
+    spawnSync(process.execPath, args, { cwd: packageRoot, stdio: "ignore" });
+  } catch {
+    // ignore — workspace init / launchd sync must not block install
+  }
+}
+
 if (existsSync(cli)) {
-  spawnSync(process.execPath, [cli, "init"], { cwd: packageRoot, stdio: "ignore" });
+  runBestEffort([cli, "init"]);
+  runBestEffort([cli, "scheduler", "sync"]);
 } else {
   await import("./init-memory-workspace.mjs");
 }

package/templates/com.pi.memory.consolidate.plist.example

@@ -0,0 +1,41 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
+  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<!--
+  Managed automatically by pi-memory (`scheduler sync` on postinstall, init, session_start).
+  Reference copy — normally do not install by hand.
+
+  Manual test after sync:
+    launchctl start com.pi.memory.maintenance
+
+  Logs: __HOME__/.pi/pi-memory-data/logs/maintenance.{log,err.log}
+-->
+<plist version="1.0">
+<dict>
+  <key>Label</key>
+  <string>com.pi.memory.maintenance</string>
+
+  <key>ProgramArguments</key>
+  <array>
+    <string>/bin/sh</string>
+    <string>-c</string>
+    <string>mkdir -p '__HOME__/.pi/pi-memory-data/logs' &amp;&amp; exec '__NODE__' '__CLI__' maintenance --cron --verbose >> '__HOME__/.pi/pi-memory-data/logs/maintenance.log' 2>> '__HOME__/.pi/pi-memory-data/logs/maintenance.err.log'</string>
+  </array>
+
+  <key>StartCalendarInterval</key>
+  <dict>
+    <key>Hour</key>
+    <integer>3</integer>
+    <key>Minute</key>
+    <integer>0</integer>
+  </dict>
+
+  <key>EnvironmentVariables</key>
+  <dict>
+    <key>PI_MEMORY_ENV_FILE</key>
+    <string>__HOME__/.pi/agent/pi-memory.env</string>
+    <key>PI_MEMORY_AGENT_DIR</key>
+    <string>__HOME__/.pi/pi-memory-data</string>
+  </dict>
+</dict>
+</plist>

package/templates/consolidate.cmd.example

@@ -0,0 +1,15 @@
+@echo off
+REM Windows wrapper for daily maintenance (Task Scheduler).
+REM Replace __NODE__ and __CLI__ before use.
+REM
+REM Install (elevated cmd):
+REM   schtasks /Create /TN "PiMemoryMaintenance" /TR "\"%CD%\consolidate.cmd\"" /SC DAILY /ST 03:00 /F
+REM
+REM Or point /TR directly at this script after editing paths below.
+
+set PI_MEMORY_ENV_FILE=%USERPROFILE%\.pi\agent\pi-memory.env
+set PI_MEMORY_AGENT_DIR=%USERPROFILE%\.pi\pi-memory-data
+
+if not exist "%USERPROFILE%\.pi\pi-memory-data\logs" mkdir "%USERPROFILE%\.pi\pi-memory-data\logs"
+
+"__NODE__" "__CLI__" maintenance --cron --verbose >> "%USERPROFILE%\.pi\pi-memory-data\logs\maintenance.log" 2>> "%USERPROFILE%\.pi\pi-memory-data\logs\maintenance.err.log"

package/templates/crontab.example

@@ -0,0 +1,14 @@
+# Daily 03:00 maintenance via OS cron (Linux/macOS crontab)
+#
+# Runs in one window:
+#   1. consolidate — dedupe / merge MEMORY.md
+#   2. drain-shutdown-queue — ingest facts from sessions that never compacted
+#
+# Install:
+#   1. Replace paths below, then: crontab -e
+#   2. `logs/` is created by Pi session start or `pi-memory init`; the mkdir below covers cron-only installs.
+#
+# __NODE__ = which node
+# __CLI__  = path to pi-memory/dist/cli.js (after pnpm build)
+#
+0 3 * * * mkdir -p $HOME/.pi/pi-memory-data/logs && PI_MEMORY_ENV_FILE=$HOME/.pi/agent/pi-memory.env PI_MEMORY_AGENT_DIR=$HOME/.pi/pi-memory-data __NODE__ __CLI__ maintenance --cron --verbose >> $HOME/.pi/pi-memory-data/logs/maintenance.log 2>&1

package/templates/schtasks.example.txt

@@ -0,0 +1,34 @@
+# Windows Task Scheduler — daily 03:00 maintenance
+#
+# Runs consolidate, then drain-shutdown-queue in one invocation.
+#
+# Prerequisites:
+#   1. pnpm build
+#   2. Edit templates/consolidate.cmd.example (__NODE__, __CLI__)
+#   3. `logs/` is created by Pi session start or `pi-memory init`; consolidate.cmd also mkdirs before redirect.
+#
+# Option A — schtasks with wrapper script (recommended):
+#
+#   schtasks /Create ^
+#     /TN "PiMemoryMaintenance" ^
+#     /TR "C:\path\to\pi-memory\templates\consolidate.cmd" ^
+#     /SC DAILY /ST 03:00 /F
+#
+# Option B — schtasks inline (replace paths):
+#
+#   schtasks /Create ^
+#     /TN "PiMemoryMaintenance" ^
+#     /TR "\"C:\Program Files\nodejs\node.exe\" \"C:\path\to\pi-memory\dist\cli.js\" maintenance --cron --verbose" ^
+#     /SC DAILY /ST 03:00 /F
+#
+# Set env vars for the task (Task Scheduler → Properties → Environment):
+#   PI_MEMORY_ENV_FILE = %USERPROFILE%\.pi\agent\pi-memory.env
+#   PI_MEMORY_AGENT_DIR = %USERPROFILE%\.pi\pi-memory-data
+#
+# Manual test:
+#   schtasks /Run /TN "PiMemoryMaintenance"
+#
+# Remove:
+#   schtasks /Delete /TN "PiMemoryMaintenance" /F
+#
+# Logs: %USERPROFILE%\.pi\pi-memory-data\logs\maintenance.log

package/dist/consolidate/entryKey.d.ts

@@ -1,5 +0,0 @@
-/** Stable dedupe key for memory entries (section + trimmed content). */
-export declare function entryDedupeKey(entry: {
-    section: string;
-    content: string;
-}): string;

package/dist/consolidate/entryKey.js

@@ -1,4 +0,0 @@
-/** Stable dedupe key for memory entries (section + trimmed content). */
-export function entryDedupeKey(entry) {
-    return `${entry.section}\0${entry.content.trim()}`;
-}