@chendpoc/pi-memory 0.1.13 → 0.2.1

package/README.md

@@ -1,297 +1,365 @@
 # @chendpoc/pi-memory
 
-[![npm](https://img.shields.io/npm/v/@chendpoc/pi-memory)](https://www.npmjs.com/package/@chendpoc/pi-memory)
+<p align="center">
+  <a href="README.md">English</a> |
+  <a href="doc/README-zh.md">简体中文</a>
+</p>
 
-Local long-term memory for [Pi](https://pi.dev) coding agent. It closes the Write → Consolidate → Retrieve loop so past sessions, preferences, projects, and decisions can re-enter future conversations automatically.
+Cross-session episodic memory for the [Pi coding agent](https://pi.dev).
 
-**No external binaries required** --- pure TypeScript, runs entirely inside Pi.
+`pi-memory` gives Pi a local, auditable memory layer across sessions. It keeps durable facts in **`MEMORY.md` as the source of truth**, derives a vector index in `memory.vec.sqlite`, and injects relevant private context through Preflight before the main model answers.
 
-## Features
+## What It Does
 
-- **Implicit memory preflight** --- automatically detects memory-relevant questions (Chinese/English/Japanese) and injects `<private_memory>` context before each LLM call
-- **`memory_recall` tool** --- LLM can explicitly query the knowledge graph by entity/relationship
-- **`memory_append` tool** --- queues explicit user memories into stage1 so Phase2 can merge them safely
-- **Offline consolidation** --- `session_shutdown` only enqueues metadata; `pi-memory consolidate` drains queue, trains graph, and updates memory files
-- **MEMORY.md index** --- session-level cap injection plus scoped project memory files
-- **Local trainer** --- extracts entities, relations, and events from Pi session history (regex or LLM-powered)
-- **FTS5 search** --- SQLite full-text index for keyword fallback
-- **LLM rerank** --- reranks keyword search results using `deepseek-v4-flash` for relevance scoring
-- **Session dedup** --- content-hash deduplication in both trainer and indexer
-- **Zero config** --- works out of the box with `pi install`
+Pi already has compaction for long sessions. That solves "this conversation is too long"; it does not solve "a new session forgot my preferences, project conventions, prior decisions, and unresolved todos."
 
-## Install
+`pi-memory` fills that gap:
+
+```text
+durable facts -> MEMORY.md -> derived vector index -> per-turn Preflight recall
+```
+
+It provides:
+
+- **Explicit memory** through `/remember`.
+- **Automatic durable fact export** from Pi compaction.
+- **Shutdown queue recovery** for short or missed sessions.
+- **Per-turn private recall** before the main model runs.
+- **Human-editable storage** in Markdown, with vector search as a rebuildable index.
+- **Offline maintenance** for consolidation, dedupe, overflow handling, and queue draining.
+
+## Installation
+
+Requirements:
+
+- Node.js `>=24 <25`
+- pnpm
+- Pi extension runtime packages supplied by Pi
+
+Install as a package in the Pi extension environment:
 
 ```bash
-pi install npm:@chendpoc/pi-memory
+pnpm add @chendpoc/pi-memory
+```
+
+For local development from this repository:
+
+```bash
+pnpm install
+pnpm build
+pnpm typecheck
+pnpm test
 ```
 
-Or for local development:
+Enable the extension through Pi's extension loading mechanism. This package declares:
 
 ```json
 {
-  "packages": ["./extensions/pi-memory"]
+  "pi": {
+    "extensions": ["./src/pi-extension.ts"]
+  }
 }
 ```
 
-Pi reads `package.json` > `pi.extensions` and auto-loads the extension via [jiti](https://github.com/unjs/jiti) (TypeScript, no build step needed).
-
-## Quick Start
+Initialize memory explicitly when needed:
 
 ```bash
-# 1. Install the package
-pi install npm:@chendpoc/pi-memory
+pi-memory init
+```
 
-# 2. Train a memory bundle from your session history
-npx pi-memory train --full
+Initialization never overwrites a non-empty `MEMORY.md`.
 
-# 3. Run first consolidation and install the daily scheduler
-npx pi-memory consolidate
-npx pi-memory setup-schedule
+## Why Choose `pi-memory`
 
-# 4. Restart Pi --- memory is now active
-# Try asking: "What projects have I worked on?"
-```
+### Agent Before / After
+
+| Situation | Without `pi-memory` | With `pi-memory` |
+| --- | --- | --- |
+| New session asks "continue the plan from last time" | Agent has to ask for context or guess from the current repo. | Preflight recalls matching `MEMORY.md` facts and injects private reference context. |
+| User says "remember that this repo uses Vitest" | The fact may stay only in the current session summary. | `/remember` writes a `[user]` entry that consolidate must preserve. |
+| Long session compacts | Compaction helps continue that session but does not create durable cross-session facts. | One dual-purpose compact summary keeps session context and exports durable facts. |
+| Subagent is spawned | It may over-recall or duplicate the parent session's memory writes. | Subagents get Memory Cap only and write Compact Delta facts. |
+| Vector sidecar is down | A hard dependency would break the turn. | Preflight silently falls back to Markdown or injects nothing; the model still runs. |
+| Memory grows | A file can become noisy and unbounded. | 150-line `MEMORY.md` cap, `auto-*.md` overflow, consolidate merge/dedupe. |
+
+### Key Advantages
+
+- **Markdown Ground Truth**: `MEMORY.md` and `auto-*.md` can be opened, reviewed, edited, grepped, copied, or versioned.
+- **Derived index, not hidden state**: `memory.vec.sqlite` can be deleted and rebuilt from Markdown.
+- **Preflight recall**: Memory is injected before the main model answers instead of hoping the model calls a search tool.
+- **Hot-path budget**: Default Preflight budget is **800ms**, with QueryIntent, sidecar query, and fallback all bounded.
+- **Protected user notes**: `/remember` writes `[user]` entries that consolidate must not remove or rewrite.
+- **Sidecar isolation**: embedding, vector scan, MMR, stats, and reindex run outside the extension process, while writes stay owned by `MemoryStore`.
+- **Subagent policy**: root sessions get Memory Cap + Episodic Preflight; subagents get Memory Cap only by default.
+- **Graceful fallback**: if sidecar recall is empty, timed out, or unavailable, the turn still runs.
+
+### Comparison
 
-After training, Pi will:
-- Show `status: ready, mode: local_graph` when you type `/memory`
-- Show queue/index details when you type `/memory --verbose`
-- Automatically inject past context when you ask relationship questions
-- Let the LLM call `memory_recall` for explicit lookups
+`pi-memory` is not trying to be every memory system. The value is a specific Pi-native loop: Markdown ground truth, Preflight injection, sidecar retrieval, compaction export, and offline maintenance.
+
+| System | Strength | Difference From `@chendpoc/pi-memory` |
+| --- | --- | --- |
+| Cursor Rules / OpenCode `AGENTS.md` | Static project instructions, predictable injection. | Mostly user-authored rules; no automatic durable fact extraction or per-turn episodic Preflight. |
+| Claude Code Auto Memory | Agent can write local memory files. | File-based memory, but no sidecar vector recall or Pi compact/shutdown integration. |
+| `pi-hermes-memory` | Rich Pi package with FTS5, failure memory, correction learning, security scanning. | More automated and feature-heavy; no `<private_memory>` Preflight loop or sidecar-derived vector index. |
+| OpenClaw memory-core | Mature file + index design, dreaming, hybrid search, local embeddings. | Broader memory platform; `pi-memory` is narrower and Pi-extension focused. |
+| Mem0 / Zep | Managed memory APIs with hybrid search, graph, temporal modeling. | Stronger retrieval infrastructure, but external service/database oriented and not Markdown-ground-truth first. |
+| Letta | Context engineering with git-backed memory repos and sleep-time compute. | Powerful for autonomous memory management; heavier mental model than Pi's extension lifecycle. |
+| Cognee | Knowledge engine with graph/vector/relational stores and many retrieval modes. | Better for knowledge graphs; overkill for lightweight coding-agent preferences and conventions. |
+
+Where other systems are stronger:
+
+- `pi-hermes-memory`: failure memory, correction detector, tool quirks, secret scanning.
+- OpenClaw: dreaming stages, memory wiki, hybrid FTS/vector search, local embedding providers.
+- Zep/Cognee: temporal graph reasoning and multi-hop graph retrieval.
+- Mem0: hosted multi-tenant memory API.
+- Letta: autonomous context repositories and sleep-time memory work.
 
 ## How It Works
 
+### Architecture
+
+```text
+Pi extension process
+  |- session_start
+  |    |- initialize MEMORY.md
+  |    |- start/warm sidecar
+  |    |- reindex derived vector index
+  |    `- preload Memory Cap
+  |
+  |- before_agent_start / context
+  |    `- Preflight recall -> <private_memory> injection
+  |
+  |- /remember
+  |    `- append [user] Memory Entry
+  |
+  |- session_before_compact / session_compact
+  |    `- dual-purpose summary -> Memory Export ingest
+  |
+  |- session_shutdown
+  |    `- append shutdown metadata only
+  |
+  `- consolidate scheduler
+       `- merge/dedupe -> rewrite Ground Truth -> reindex
+
+Sidecar process over UDS JSONL
+  |- ping
+  |- stats
+  |- query: embed -> cosine scan -> MMR
+  `- reindex: upsert chunks into memory.vec.sqlite
 ```
-Pi session files (.jsonl)
-    | session_shutdown
-    v
-pending_sessions queue (memories.sqlite)
-    | daily / manual consolidate
-    v
-stage1 outputs + graph train
-    |
-    v
-MEMORY.md / project memory + graph.json + sessions.db
-    |
-    v
-memory_recall tool / implicit preflight injection / MEMORY.md cap
+
+### Read Path
+
+Root session:
+
+```text
+Memory Cap from Ground Truth
+  + Episodic Preflight for the current user message
+  -> merged <private_memory>
 ```
 
-### Query Backends
+Subagent session:
 
-The service automatically selects the best available backend:
+```text
+Memory Cap only
+  -> no episodic QueryIntent / sidecar query by default
+```
 
-| Priority | Backend | Requires |
-|----------|---------|----------|
-| 1 | **LocalGraphQuerier** (in-process) | bundle only |
-| 2 | **FTS5 + keyword fallback** | `better-sqlite3` (optional) |
+Fallback chain:
 
-The default backend is LocalGraphQuerier --- pure TypeScript, no external dependencies.
+```text
+Sidecar results
+  -> if empty/error/timeout: MEMORY.md fallback
+  -> if empty: no injection
+```
 
-## Commands
+### Write Paths
 
-| Command | Description |
-|---------|-------------|
-| `/memory` | Show service status, query mode, and bundle info |
-| `/memory --verbose` | Show consolidation queue, memory index usage, and recent job logs |
+| Path | Trigger | LLM? | Blocking? | Purpose |
+| --- | --- | --- | --- | --- |
+| `/remember` | User command | No | Yes | Explicit durable note |
+| Compaction | `session_before_compact` + `session_compact` | One summary call | Summary blocks; ingest is background | Continue current session and export durable facts |
+| Shutdown Queue | `session_shutdown` + `pi-memory maintenance` | Only offline, when no compaction summary exists | No during shutdown | Recover facts from short or missed sessions |
+| Consolidate | overflow >= 12, 7 days, or daily cron | Optional | Offline/background | Dedupe, merge, prune obsolete todos |
 
-## CLI
+## Data And Memory Format
 
-```bash
-# Train a bundle from session history
-npx pi-memory train --full                    # full rebuild
-npx pi-memory train                           # incremental (only new sessions)
-npx pi-memory train --extractor llm           # use LLM for deeper extraction
-npx pi-memory train --extractor llm --model deepseek/deepseek-v4-flash
-
-# Query the knowledge graph
-npx pi-memory query '{"mode":"direct_relation","anchor_mentions":["Alice"]}'
-
-# Service diagnostics
-npx pi-memory health
-npx pi-memory status
-npx pi-memory memory-status
-
-# Offline consolidation
-npx pi-memory consolidate
-npx pi-memory consolidate --dry-run
-npx pi-memory consolidate --phase1-only
-npx pi-memory consolidate --phase2-only
-
-# OS scheduler
-npx pi-memory setup-schedule
-npx pi-memory setup-schedule --hour 4
-npx pi-memory setup-schedule --status
-npx pi-memory setup-schedule --remove
-
-# Manage bundles
-npx pi-memory install-bundle ./path/to/bundle
-
-# Rebuild FTS5 search index
-npx pi-memory index
-
-# Continuous training (watch mode)
-npx pi-memory train --watch
-```
+All artifacts live under one memory agent directory.
 
-`train --watch` is kept for compatibility. The long-term memory path is `pi-memory consolidate` plus the OS scheduler.
+Resolution order:
 
-## Flags
+1. `--agent-dir` CLI flag
+2. `PI_MEMORY_AGENT_DIR`
+3. default `~/.pi/pi-memory-data`
 
-| Flag | Description | Default |
-|------|-------------|---------|
-| `--memory-helper-model` | LLM model for intent detection and rerank | `deepseek/deepseek-v4-flash` |
+| File | Role |
+| --- | --- |
+| `MEMORY.md` | Ground Truth file |
+| `auto-*.md` | Overflow files after the 150-line cap |
+| `.memory_gc` | Last consolidate timestamp |
+| `.memory_compactions.json` | Compaction idempotency state |
+| `.memory_shutdown_queue.jsonl` | Append-only shutdown metadata |
+| `.memory_shutdown_processed.json` | Drain idempotency state |
+| `memory.vec.sqlite` | Derived Vector Index |
+| `memory.sock` | Sidecar Unix domain socket |
 
-```bash
-pi --memory-helper-model deepseek/deepseek-v4-flash
-```
+Canonical scaffold: [`templates/MEMORY.md.example`](./templates/MEMORY.md.example)
 
-## Data Layout
+```markdown
+# Memory
 
-```
-~/.pi/
-├── memory/
-│   ├── current/              # active bundle (symlink)
-│   │   ├── manifest.json
-│   │   └── graph.json        # entities, edges, events
-│   ├── bundles/              # historical bundles
-│   ├── sessions.db           # FTS5 search index
-│   ├── memories.sqlite       # pending queue + stage1 outputs
-│   ├── workspace/            # raw memories + rollout summaries
-│   ├── projects/<hash>/      # project-scoped MEMORY.md files
-│   ├── consolidation.log     # structured consolidation logs
-│   └── .train_marker         # incremental training timestamp
-├── agent/sessions/           # Pi session files (.jsonl)
-└── MEMORY.md                 # global memory index
+## Preferences
+
+## Conventions
+
+## Findings
+
+## Todos
 ```
 
-## Programmatic API
-
-```typescript
-import {
-  MemoryService,
-  defaultMemoryConfig,
-  trainBundle,
-  LocalGraphQuerier,
-  createLLMFactExtractor,
-  createStandaloneLLMClient,
-  rerankWithLLM,
-} from "@chendpoc/pi-memory";
-
-// --- Query the graph directly ---
-const querier = new LocalGraphQuerier("/Users/you/.pi/memory");
-querier.load();
-const result = querier.query({
-  mode: "direct_relation",
-  anchor_mentions: ["Alice"],
-});
-
-// --- Train a bundle ---
-const result = await trainBundle({
-  sessionsDir: "~/.pi/agent/sessions",
-  bundleRoot: "~/.pi/memory",
-  full: true,
-});
-
-// --- Train with LLM extraction ---
-const client = createStandaloneLLMClient("deepseek/deepseek-v4-flash");
-const extractor = createLLMFactExtractor({ client, batchSize: 10 });
-await trainBundle({ extractOpts: { llmExtractor: extractor } });
-
-// --- Use as Pi extension ---
-import piMemory from "@chendpoc/pi-memory/extension";
-export default piMemory;
+Entries are single Markdown bullets:
+
+```markdown
+- [user] Prefer pnpm over npm <!-- id:abc123 user ts:2026-07-04T09:00:00.000+08:00 -->
+- Project tests use Vitest <!-- id:def456 ts:2026-07-04T09:05:00.000+08:00 -->
 ```
 
-## Architecture
+Rules:
 
-| Module | Purpose |
-|--------|---------|
-| `src/pi-extension.ts` | Pi ExtensionAPI entry point |
-| `src/local/graphQuery.ts` | In-process graph query engine |
-| `src/adapters/piComplete.ts` | LLM adapter via `@earendil-works/pi-ai/compat` |
-| `src/service.ts` | Service lifecycle and query routing |
-| `src/consolidation/` | Queue, stage1, Phase2, scheduler, memory index, scope |
-| `src/trainer/` | Session loader, fact extraction, entity resolution, bundle builder |
-| `src/fallback/` | FTS5 index, keyword search, LLM rerank |
-| `src/preflight/` | Intent detection, private memory render/inject/strip |
-| `src/tools/` | `memory_recall` and `memory_append` tool definitions |
+- `/remember` writes `[user]` entries.
+- Consolidate must not remove or rewrite `[user]` entries.
+- `MEMORY.md` is capped at 150 lines.
+- Overflow entries spill to `auto-*.md`, with a pointer in `MEMORY.md`.
+- Vector chunks are derived from entries; by default long entries split beyond `PI_MEMORY_CHUNK_MAX_CHARS=512`.
 
-## Implicit Preflight
+## Configuration
 
-On every LLM call, the `context` event:
+Optional env file locations are loaded in this order:
 
-1. Detects memory-relevant intents via regex (Chinese/English/Japanese relationship patterns)
-2. Optionally calls a helper LLM (`compile_memory_intents` forced tool_use)
-3. Queries the graph for matching entities/relations and greps scoped MEMORY.md in parallel
-4. Injects the session memory index and dynamic recall into `<private_memory>` blocks (deep copy, not persisted to session)
-5. Results are cached per agent loop to avoid redundant queries during multi-tool turns
+1. `PI_MEMORY_ENV_FILE`
+2. project `.env`
+3. project `.env.local`
+4. `~/.pi/agent/pi-memory.env`
 
-## Peer Dependencies
+Common variables:
 
-These are provided by the Pi host and should not be bundled:
+| Variable | Default | Purpose |
+| --- | --- | --- |
+| `PI_MEMORY_AGENT_DIR` | `~/.pi/pi-memory-data` | Memory data root |
+| `PI_MEMORY_EMBEDDER` | `hash` | `hash`, `ollama`, or `openai` |
+| `PI_MEMORY_HELPER_MODEL` | `deepseek/deepseek-v4-flash` | Helper model spec for QueryIntent and consolidate |
+| `PI_MEMORY_PREFLIGHT_BUDGET_MS` | `800` | Shared Preflight budget, clamped to 250-1500ms |
+| `PI_MEMORY_INTENT_RETRIES` | `0` | Helper LLM retries after the first attempt |
+| `PI_MEMORY_WARM_SIDECAR` | `1` | Warm sidecar at `session_start` |
+| `PI_MEMORY_INTENT_CACHE` | `1` | Cache QueryIntent per session |
+| `PI_MEMORY_REINDEX_DEBOUNCE_MS` | `500` | Debounce sidecar reindex after writes |
+| `PI_MEMORY_TOP_K` | `3` | Vector recall result count |
+| `PI_MEMORY_MMR_LAMBDA` | `0.8` | MMR relevance/diversity balance |
+| `PI_MEMORY_MIN_RELEVANCE` | `0.4` | Minimum cosine similarity |
+| `PI_MEMORY_CHUNK_MAX_CHARS` | `512` | Split long entries for indexing; `0` disables |
+| `PI_MEMORY_DEBUG` | unset | `1` prints debug timing logs |
 
-- `@earendil-works/pi-agent-core`
-- `@earendil-works/pi-ai`
-- `@earendil-works/pi-coding-agent`
-- `typebox`
+See [`.env.example`](./.env.example) for the full list.
 
-## Optional Dependencies
+### Embedders
 
-- `better-sqlite3` --- enables FTS5 session search index (graceful degradation if absent)
+| Embedder | Use When | Notes |
+| --- | --- | --- |
+| `hash` | Zero-config local development | Offline, deterministic, lower semantic quality |
+| `ollama` | Local semantic embeddings | Uses `PI_MEMORY_OLLAMA_BASE_URL` and `PI_MEMORY_OLLAMA_EMBED_MODEL` |
+| `openai` | Higher-quality cloud embeddings | Requires `PI_MEMORY_OPENAI_API_KEY` or `OPENAI_API_KEY` |
 
-## Releasing
+The Vector Index stores embedding provider, model, and dimension metadata. When they change, old chunks are cleared and rebuilt.
 
-This repo uses [Changesets](https://github.com/changesets/changesets) for semver bumps and `CHANGELOG.md`, and a **tag-triggered** workflow for GitHub Releases (and optional npm publish).
+## Commands
+
+Inside Pi:
+
+```text
+/remember [section] <content>
+/memory-status [refresh|expand|collapse|hide]
+```
 
-1. After a feature/fix PR, add a changeset: `pnpm changeset`
-2. Merge to `main` — GitHub Actions opens a **Version Packages** PR (bumps version + updates `CHANGELOG.md`)
-3. Merge that PR, then tag and push:
+CLI:
 
 ```bash
-git pull origin main
-git tag v0.1.13
-git push origin v0.1.13
+pi-memory init
+pi-memory status
+pi-memory maintenance --cron --verbose
+pi-memory consolidate --force --verbose
+pi-memory drain-shutdown-queue --verbose
 ```
 
-4. The **Tag Release** workflow (`tag-release.yml`) runs on `v*` tags: creates a GitHub Release from `CHANGELOG.md`, then publishes to npm via [OIDC trusted publishing](https://docs.npmjs.com/trusted-publishers/) (no long-lived `NPM_TOKEN`).
+`maintenance` is the recommended scheduler entrypoint:
 
-### npm trusted publishing (recommended)
+```text
+consolidate -> drain-shutdown-queue
+```
 
-npm now recommends **Trusted Publishing** over Granular "bypass 2FA" tokens for CI. GitHub Actions proves its identity with a short-lived OIDC credential; no stored token, no OTP (`EOTP`).
+Scheduler templates:
 
-**One-time setup on npmjs.com** (package must exist first — see below):
+- [`templates/com.pi.memory.consolidate.plist.example`](./templates/com.pi.memory.consolidate.plist.example)
+- [`templates/crontab.example`](./templates/crontab.example)
+- [`templates/consolidate.cmd.example`](./templates/consolidate.cmd.example)
+- [`templates/schtasks.example.txt`](./templates/schtasks.example.txt)
 
-1. Open `https://www.npmjs.com/package/@chendpoc/pi-memory/access` (after first publish)
-2. **Trusted Publisher** → GitHub Actions
-3. Set exactly:
-   - Organization or user: `chendpoc`
-   - Repository: `pi-memory`
-   - Workflow filename: `tag-release.yml`
-   - Environment: *(leave blank unless you add a GitHub Environment)*
+## Diagnostics
 
-**First publish:** npm requires the package to exist before you can configure trusted publishing. Either:
+Use `/memory-status` or `pi-memory status` to inspect:
 
-- Publish once locally with `npm login` + `npm publish --access public`, or
-- Use [setup-npm-trusted-publish](https://github.com/azu/setup-npm-trusted-publish) to create a placeholder, configure OIDC, then publish real versions from CI.
+- memory agent directory
+- `MEMORY.md` line count
+- entry count
+- overflow count
+- last consolidate timestamp
+- sidecar socket status
+- vector index generation and chunk count
+- configured embedder
+- index embedder mismatch
 
-The workflow already sets `id-token: write` and uses Node 24 (npm ≥ 11.5.1). Provenance is automatic with trusted publishing.
+Use `PI_MEMORY_DEBUG=1` to log Preflight timings:
 
-**Re-run without deleting a tag:** GitHub → Actions → **Tag Release** → **Run workflow**
+```json
+{
+  "phase": "preflight",
+  "event": "recall",
+  "intent_ms": 0,
+  "intent_skipped": true,
+  "intent_cache_hit": false,
+  "sidecar_ms": 42,
+  "cache_hit": true,
+  "total_ms": 45,
+  "fallback": false,
+  "results": 3
+}
+```
+
+## Non-Goals
 
-- `tag`: `v0.1.13`
-- `skip_github_release`: ✅ (release already exists)
-- `publish_npm`: ✅
+- Replacing Pi compaction.
+- Replacing session search; use a dedicated session-search extension for old conversations.
+- Maintaining a graph database inside this package.
+- Making the sidecar authoritative.
+- Storing full chat transcripts as memory.
+- Adding multi-second reflection to every user turn.
 
 ## Development
 
 ```bash
-pnpm install
-pnpm build
-pnpm test
 pnpm typecheck
-pnpm changeset    # before opening a release PR
+pnpm test
+pnpm build
 ```
 
+The sidecar IPC test opens a Unix domain socket. If it fails with `listen EPERM` inside a restricted sandbox, run the test in a normal local shell.
+
+## Docs
+
+- [Chinese README](./doc/README-zh.md)
+- [Roadmap](./doc/ROADMAP.md)
+- [UBIQUITOUS_LANGUAGE.md](./UBIQUITOUS_LANGUAGE.md) - domain glossary
+
 ## License
 
 MIT

package/UBIQUITOUS_LANGUAGE.md

@@ -0,0 +1,81 @@
+# Ubiquitous Language
+
+> pi-memory 扩展的领域术语表。实现与文档以本表 canonical 名称为准。
+
+## Storage
+
+| Term | Definition | Aliases to avoid |
+| ---- | ---------- | ---------------- |
+| **Ground Truth** | The durable cross-session fact store: `MEMORY.md` plus overflow `auto-*.md` files. | memory.db, chat backup |
+| **Vector Index** | A derived SQLite file (`memory.vec.sqlite`) holding embeddings for episodic retrieval. | memory.db, sqlite-vec ANN |
+| **Memory Entry** | One bullet fact in Ground Truth under Preferences, Conventions, Findings, or Todos. | note, fact, bullet |
+| **Overflow File** | An `auto-*.md` spill file created when `MEMORY.md` exceeds the line cap. | auto file, spill |
+
+## Read path
+
+| Term | Definition | Aliases to avoid |
+| ---- | ---------- | ---------------- |
+| **Preflight** | Best-effort retrieval and injection before the main model sees each user message. | memory recall, pre-fetch |
+| **QueryIntent** | Structured retrieval hints (what/who/where or raw_query) extracted by a helper LLM. | search intent |
+| **Memory Cap** | Session-scoped static summary of Ground Truth injected without episodic query. | turnMemoryIndex, static cap |
+| **Episodic Preflight** | QueryIntent → Sidecar query → optional Fallback for query-dependent recall. | full preflight |
+| **Fallback** | Silent read of Ground Truth when Sidecar returns nothing or errors. | md fallback |
+| **Private Memory** | The `<private_memory>` block prefixed to in-flight user messages only. | injected context |
+
+## Write path
+
+| Term | Definition | Aliases to avoid |
+| ---- | ---------- | ---------------- |
+| **Memory Export** | The cross-session facts section inside a dual-purpose compact summary. | memoryQueue, export block |
+| **Compact Delta** | New Memory Export facts filtered against existing Ground Truth before append (subagent). | Stage1, shutdown extract |
+| **Dual-Purpose Summary** | One LLM compact output with Session Context and Memory Export sections. | compact summary |
+| **Consolidate** | Periodic LLM merge/dedupe of Ground Truth; not a source of new facts. | GC, merge |
+| **Shutdown Queue** | Append-only JSONL file (`.memory_shutdown_queue.jsonl`) of session metadata on shutdown; drained by `pi-memory maintenance`. | session enqueue pipeline |
+
+## Infrastructure
+
+| Term | Definition | Aliases to avoid |
+| ---- | ---------- | ---------------- |
+| **Sidecar** | Separate Node process for embed, vector scan, MMR, and reindex over UDS. | tlm, daemon |
+| **JSONL Framing** | One JSON object per line over UDS IPC. | NDJSON, ndjson |
+| **Index Generation** | Monotonic counter bumped on reindex; invalidates query cache. | indexGeneration |
+| **onSyncToSidecar** | Callback fired after Ground Truth writes to schedule debounced reindex. | onDirty |
+| **onConsolidateCheck** | Callback fired after writes to schedule debounced consolidate evaluation. | onDirty |
+
+## Session types
+
+| Term | Definition | Aliases to avoid |
+| ---- | ---------- | ---------------- |
+| **Root Session** | A top-level Pi agent session without a parent session header. | main session |
+| **Subagent Session** | A forked session whose header includes `parentSession` or `parent_session`. | child session, fork |
+
+## Relationships
+
+- **Ground Truth** is the source; the **Vector Index** is derived and may lag.
+- **Preflight** reads **Ground Truth** via **Sidecar** first, then **Fallback**.
+- **Root Session** runs **Episodic Preflight** plus **Memory Cap** each turn; **Subagent Session** runs **Memory Cap** only.
+- **Memory Export** from compaction flows through **Compact Delta** for **Subagent Session**, then **appendIfAbsent**.
+- **Shutdown Queue** records metadata only; it does not write **Ground Truth**.
+- **Consolidate** rewrites **Ground Truth** and triggers **onSyncToSidecar**.
+
+## Example dialogue
+
+> **Dev:** "When a **Subagent Session** compacts, do we run **Episodic Preflight** on its turns?"
+>
+> **Domain expert:** "No. Subagents get **Memory Cap** at **session_start** and skip **Episodic Preflight**. New durable facts still arrive via **Memory Export**, but **Compact Delta** filters out anything already in parent **Ground Truth**."
+>
+> **Dev:** "What happens on **session_shutdown**?"
+>
+> **Domain expert:** "We append one line to the **Shutdown Queue**—session file, parent path, reason. No LLM on shutdown. **`pi-memory maintenance`** drains the queue offline (after consolidate), reusing **Compact Delta** rules for subagents."
+>
+> **Dev:** "If **Sidecar** is down, does **Preflight** fail?"
+>
+> **Domain expert:** "It **Fallback**s to **Ground Truth** or **Private Memory** stays empty. The user never sees an error."
+
+## Flagged ambiguities
+
+- **memory.db** was used for the vector file — canonical name is **Vector Index** file `memory.vec.sqlite`.
+- **sqlite-vec** implied ANN search — MVP uses **better-sqlite3 + full-table cosine scan**; ANN is future work.
+- **NDJSON / ndjson** — canonical term is **JSONL Framing** (one JSON object per line).
+- **memoryQueue / onDirty / Stage1** — replaced by **appendFromCompaction**, **onSyncToSidecar**, and **Compact Delta**.
+- **MVP chunking**：**1 Memory Entry = 1+ vector chunks**；超过 `PI_MEMORY_CHUNK_MAX_CHARS`（默认 512）按段落/句界拆分，`chunk_id = entry.id` 或 `entry.id#N`；embed 文本带 `[Section]` 前缀。Ground Truth 仍为 1 bullet = 1 entry。