@ppagent/memory 0.3.1 → 0.4.0

package/llms.txt

@@ -1,862 +1,908 @@
-# @ppagent/memory
-
-> `@ppagent/memory` is an independent TypeScript/Node.js long-term memory package for AI agents. It stores conversation memory in a pluggable vector store (LanceDB or SQLite/sqlite-vec, auto-detected per platform), extracts and searches entity relationships with Grafeo, supports user/chat/session scoped facts, compresses chat history into multi-resolution topics, and ingests Markdown documents as a searchable knowledge base.
-
-This file is written for AI coding agents and assistants. Use it as the primary package guide when generating code that depends on `@ppagent/memory`.
-
-## Package Identity
-
-- Package name: `@ppagent/memory`
-- Module format: ESM
-- Main entry: `dist/index.js`
-- Type entry: `dist/index.d.ts`
-- Source entry during local workspace development: `src/index.ts`
-- Runtime target: Node.js with global `fetch`
-- Storage engines:
-  - Vector store (dual backend, auto-detected): LanceDB or SQLite (better-sqlite3 + sqlite-vec + FTS5 with jieba Chinese tokenization) for messages, topics, facts, sessions, documents, and chunks. LanceDB requires native prebuilt binaries (unavailable on Intel macOS since 0.23.0); SQLite works everywhere and is the automatic fallback. RRF hybrid-search fusion runs in the shared domain layer, so search behavior is identical across backends.
-  - Grafeo for entity/relation graph storage and graph search
-- Configuration principle: the package NEVER reads environment variables, config files, or any implicit global state. Every parameter enters through the `MemoryConfig` object passed to `MemoryManager`/`MemoryStore`. Mapping env vars to config fields (as shown in the examples below) is entirely the host application's responsibility.
-- LLM APIs: OpenAI-compatible `/chat/completions`
-- Embedding APIs: OpenAI-compatible `/embeddings`
-
-## When To Use This Package
-
-Use `@ppagent/memory` when an AI agent needs:
-
-- Long-term conversation memory across sessions.
-- Recent message storage plus compressed historical context windows.
-- User-level or chat-level facts such as preferences, profile information, project decisions, or instructions that should persist.
-- Hybrid semantic/full-text search over remembered topics and messages.
-- Optional graph search for entity-relationship questions.
-- Markdown knowledge-base ingestion with chunking, vector search, document-level coarse recall, and optional knowledge graph construction.
-- Session management APIs for agent dashboards or memory administration pages.
-- A standalone memory layer that does not depend on the larger PPAgent runtime.
-
-Do not use it as a full agent framework by itself. It does not send chat messages, call tools for users, route platform webhooks, or run a frontend. It is a memory and retrieval module.
-
-## Install
-
-```bash
-pnpm add @ppagent/memory
-```
-
-```bash
-npm install @ppagent/memory
-```
-
-## Imports
-
-```ts
-import {
-  MemoryManager,
-  MemoryStore,
-  DEFAULT_NODE_TYPES,
-  DEFAULT_RELATION_TYPES,
-  KIND_CONVERSATION,
-  KIND_KNOWLEDGE,
-} from "@ppagent/memory";
-
-import type {
-  MemoryConfig,
-  RawMessage,
-  SearchOptions,
-  SearchResult,
-  AddDocumentOptions,
-  KnowledgeSearchOptions,
+# @ppagent/memory
+
+> `@ppagent/memory` is an independent TypeScript/Node.js long-term memory package for AI agents. It stores complete raw conversation messages in a pluggable vector store (LanceDB or SQLite/sqlite-vec, auto-detected per platform), builds importance-aware Topic summaries, returns fixed-budget Topics plus recent uncompressed messages as one context window, extracts and searches entity relationships with Grafeo, supports cached user/chat facts, and ingests Markdown documents as a searchable knowledge base.
+
+This file is written for AI coding agents and assistants. Use it as the primary package guide when generating code that depends on `@ppagent/memory`.
+
+## Package Identity
+
+- Package name: `@ppagent/memory`
+- Module format: ESM
+- Main entry: `dist/index.js`
+- Type entry: `dist/index.d.ts`
+- Source entry during local workspace development: `src/index.ts`
+- Runtime target: Node.js with global `fetch`
+- Storage engines:
+  - Vector store (dual backend, auto-detected): LanceDB or SQLite (better-sqlite3 + sqlite-vec + FTS5 with jieba Chinese tokenization) for messages, topics, facts, sessions, documents, and chunks. LanceDB requires native prebuilt binaries (unavailable on Intel macOS since 0.23.0); SQLite works everywhere and is the automatic fallback. RRF hybrid-search fusion runs in the shared domain layer, so search behavior is identical across backends.
+  - Grafeo for entity/relation graph storage and graph search
+- Runtime configuration principle: `MemoryManager`/`MemoryStore` never read environment variables, config files, or implicit global state; every parameter enters through `MemoryConfig`. The separate manual migration CLI reads only the file explicitly supplied with `--config`.
+- LLM APIs: OpenAI-compatible `/chat/completions`
+- Embedding APIs: OpenAI-compatible `/embeddings`
+
+## When To Use This Package
+
+Use `@ppagent/memory` when an AI agent needs:
+
+- Long-term conversation memory across sessions.
+- Recent message storage plus compressed historical context windows.
+- User-level or chat-level facts such as preferences, profile information, project decisions, or instructions that should persist.
+- Hybrid semantic/full-text search over remembered topics and messages.
+- Optional graph search for entity-relationship questions.
+- Markdown knowledge-base ingestion with chunking, vector search, document-level coarse recall, and optional knowledge graph construction.
+- Session management APIs for agent dashboards or memory administration pages.
+- A standalone memory layer that does not depend on the larger PPAgent runtime.
+
+Do not use it as a full agent framework by itself. It does not send chat messages, call tools for users, route platform webhooks, or run a frontend. It is a memory and retrieval module.
+
+## Install
+
+```bash
+pnpm add @ppagent/memory
+```
+
+```bash
+npm install @ppagent/memory
+```
+
+## Imports
+
+```ts
+import {
+  MemoryManager,
+  MemoryStore,
+  DEFAULT_NODE_TYPES,
+  DEFAULT_RELATION_TYPES,
+  KIND_CONVERSATION,
+  KIND_KNOWLEDGE,
+} from "@ppagent/memory";
+
+import type {
+  MemoryConfig,
+  RawMessage,
+  SearchOptions,
+  SearchResult,
+  AddDocumentOptions,
+  KnowledgeSearchOptions,
   KnowledgeSearchResult,
+  MemoryContextWindow,
 } from "@ppagent/memory";
-```
-
-Most consumers should only instantiate `MemoryManager`. `MemoryStore` (plus the `VectorStoreProvider` / `Filter` types) is exported for advanced storage tests or custom infrastructure work. Breaking change in 0.2.0: `LanceService` was replaced by `MemoryStore` + provider abstraction; string SQL filters were replaced by structured `Filter` objects.
-
-## Minimal Example
-
-```ts
-import { MemoryManager } from "@ppagent/memory";
-
-const memory = new MemoryManager({
-  lancedbPath: "./data/memory/lance",
-  grafeoPath: "./data/memory/grafeo",
-
-  llmBaseUrl: process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
-  llmApiKey: process.env.LLM_API_KEY ?? "",
-  llmModel: process.env.LLM_MODEL ?? "gpt-4o-mini",
-
-  embeddingBaseUrl: process.env.EMBED_BASE_URL ?? process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
-  embeddingApiKey: process.env.EMBED_API_KEY ?? process.env.LLM_API_KEY ?? "",
-  embeddingModel: process.env.EMBED_MODEL ?? "text-embedding-3-small",
-  embeddingDimension: Number(process.env.EMBED_DIMENSION ?? 1536),
-});
-
-await memory.init();
-
-await memory.updateChat(
-  [
-    { talkerId: "user", content: "我叫 Bob，正在做一个 Rust 项目。" },
-    { talkerId: "assistant", content: "好的，我会记住你在做 Rust 项目。" },
-  ],
-  {
-    userId: "user-bob",
-    chatId: "agent-dev",
-    sessionId: "session-001",
-    sessionTitle: "Bob 的开发对话",
-  }
-);
-
-const results = await memory.search({
-  query: "Bob 正在做什么项目？",
-  scope: "user",
-  scopeId: "user-bob",
-  mode: "auto",
-});
-
-console.log(results);
-
-const answer = await memory.ask({
-  query: "Bob 正在做什么项目？",
-  scope: "user",
-  scopeId: "user-bob",
-  maxChars: 120,
-});
-
-console.log(answer);
-
-memory.destroy();
-```
-
-## Complete Example
-
-```ts
-import { MemoryManager } from "@ppagent/memory";
-import type { Entity, Relation } from "@ppagent/memory";
-
-const memory = new MemoryManager({
-  lancedbPath: "./data/memory/lance",
-  grafeoPath: "./data/memory/grafeo",
-
-  llmBaseUrl: process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
-  llmApiKey: process.env.LLM_API_KEY ?? "",
-  llmModel: process.env.LLM_MODEL ?? "gpt-4o-mini",
-
-  embeddingBaseUrl: process.env.EMBED_BASE_URL ?? "",
-  embeddingApiKey: process.env.EMBED_API_KEY ?? "",
-  embeddingModel: process.env.EMBED_MODEL ?? "text-embedding-3-small",
-  embeddingDimension: 1536,
-
-  httpTimeoutMs: 60_000,
-  httpMaxRetries: 2,
-  embeddingBatchSize: 20,
-  embeddingConcurrency: 2,
-
-  sessionTokenLimit: 16_386,
-  historyWindowTokenLimit: 10_240,
-  topicRatio: [1, 5, 20],
-  detailMaxTokens: 2048,
-  summaryMaxTokens: 512,
-  conciseMaxTokens: 128,
+```
+
+Most consumers should only instantiate `MemoryManager`. `MemoryStore` (plus the `VectorStoreProvider` / `Filter` types) is exported for advanced storage tests or custom infrastructure work. Breaking change in 0.2.0: `LanceService` was replaced by `MemoryStore` + provider abstraction; string SQL filters were replaced by structured `Filter` objects.
+
+## Minimal Example
+
+```ts
+import { MemoryManager } from "@ppagent/memory";
+
+const memory = new MemoryManager({
+  lancedbPath: "./data/memory/lance",
+  grafeoPath: "./data/memory/grafeo",
+
+  llmBaseUrl: process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
+  llmApiKey: process.env.LLM_API_KEY ?? "",
+  llmModel: process.env.LLM_MODEL ?? "gpt-4o-mini",
+
+  embeddingBaseUrl: process.env.EMBED_BASE_URL ?? process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
+  embeddingApiKey: process.env.EMBED_API_KEY ?? process.env.LLM_API_KEY ?? "",
+  embeddingModel: process.env.EMBED_MODEL ?? "text-embedding-3-small",
+  embeddingDimension: Number(process.env.EMBED_DIMENSION ?? 1536),
+});
+
+await memory.init();
+
+await memory.updateChat(
+  [
+    { talkerId: "user", content: "我叫 Bob，正在做一个 Rust 项目。" },
+    { talkerId: "assistant", content: "好的，我会记住你在做 Rust 项目。" },
+  ],
+  {
+    userId: "user-bob",
+    chatId: "agent-dev",
+    sessionId: "session-001",
+    sessionTitle: "Bob 的开发对话",
+  }
+);
+
+const results = await memory.search({
+  query: "Bob 正在做什么项目？",
+  scope: "user",
+  scopeId: "user-bob",
+  mode: "auto",
+});
+
+console.log(results);
+
+const answer = await memory.ask({
+  query: "Bob 正在做什么项目？",
+  scope: "user",
+  scopeId: "user-bob",
+  maxChars: 120,
+});
+
+console.log(answer);
+
+await memory.destroy();
+```
+
+## Complete Example
+
+```ts
+import { MemoryManager } from "@ppagent/memory";
+import type { Entity, Relation } from "@ppagent/memory";
+
+const memory = new MemoryManager({
+  lancedbPath: "./data/memory/lance",
+  grafeoPath: "./data/memory/grafeo",
+
+  llmBaseUrl: process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
+  llmApiKey: process.env.LLM_API_KEY ?? "",
+  llmModel: process.env.LLM_MODEL ?? "gpt-4o-mini",
+
+  embeddingBaseUrl: process.env.EMBED_BASE_URL ?? "",
+  embeddingApiKey: process.env.EMBED_API_KEY ?? "",
+  embeddingModel: process.env.EMBED_MODEL ?? "text-embedding-3-small",
+  embeddingDimension: 1536,
+
+  httpTimeoutMs: 60_000,
+  httpMaxRetries: 2,
+  embeddingBatchSize: 20,
+  embeddingConcurrency: 2,
+
+  compressedContextTokenLimit: 16 * 1024,
+  contextUsageRatio: 0.75,
+  precompressionRatio: 0.75,
+  compressionBatchRatio: 0.5,
+  compressionBatchTokenLimit: 0,
+  topicSummaryMaxTokens: 2048,
+  defaultModelContextTokens: 256 * 1024,
+  maxHistoryAgeMs: 0,
+  sessionIdleTtlMs: 30 * 60_000,
+  sessionSweepIntervalMs: 60_000,
   maxConcurrentCompressions: 3,
-  entitySimilarityThreshold: 0.92,
-  defaultSearchLimit: 10,
-  recallBoostMs: 3_600_000,
-
-  chunkStrategy: "markdown-heading",
-  chunkMaxTokens: 800,
-  chunkOverlap: 0,
-  knowledgeTopK: 8,
-  docCoarseTopK: 5,
-  buildGraphDefault: "auto",
-  chunkRedundantIds: true,
-  knowledgeGraphTriggerScore: 0.78,
-  knowledgeGraphEntityTopK: 10,
-  knowledgeGraphAnchorTopK: 3,
-  knowledgeGraphHopLimit: 10,
-  graphExtractConcurrency: 2,
-  graphBuildTimeoutMs: 120_000,
-});
-
-await memory.init();
-
-const userId = "u-001";
-const chatId = "agent-work";
-const sessionId = "s-2026-06-12";
-
-await memory.updateChat(
-  [
-    {
-      talkerId: "user",
-      content: "我叫 Alice，在 ChatMe 项目中负责后端架构。",
-      metadata: { source: "chat" },
-    },
-    {
-      talkerId: "assistant",
-      content: "明白，我会记住你负责 ChatMe 后端架构。",
-    },
-  ],
-  {
-    userId,
-    chatId,
-    sessionId,
-    sessionTitle: "ChatMe 架构讨论",
-    sessionMetadata: { product: "ChatMe" },
-  }
-);
-
-await memory.updateFacts(
-  "Alice 负责 ChatMe 项目的后端架构",
-  "user",
-  userId,
-  chatId,
-  sessionId
-);
-
-const entities: Entity[] = [
-  { name: "Alice", type: "Person", meta: { role: "backend architect", userId, chatId, sessionId } },
-  { name: "ChatMe", type: "Project", meta: { userId, chatId, sessionId } },
-  { name: "TypeScript", type: "Technology", meta: { userId, chatId, sessionId } },
-];
-
-const relations: Relation[] = [
-  { from: "Alice", to: "ChatMe", type: "works_on", meta: { userId, chatId, sessionId } },
-  { from: "ChatMe", to: "TypeScript", type: "built_with", meta: { userId, chatId, sessionId } },
-];
-
-await memory.updateEntity(entities, relations, { userId, chatId, sessionId });
-
-await memory.flushChat(sessionId, { wait: true });
-
-const remembered = await memory.search({
-  query: "Alice 在 ChatMe 中负责什么？",
-  scope: "user",
-  scopeId: userId,
-  mode: "all",
-  limit: 5,
-});
-
-console.log(remembered);
-
-const doc = await memory.addDocument({
-  content: `# ChatMe 技术说明
-
-ChatMe 使用 TypeScript 构建，后端包含长期记忆系统。
-
-## Memory
-
-长期记忆系统使用向量存储（LanceDB 或 SQLite，自动探测）存储向量，使用 Grafeo 存储知识图谱。`,
-  userId,
-  chatId,
-  sessionId,
-  sourceName: "chatme-memory.md",
-  metadata: { product: "ChatMe" },
-  buildGraph: true,
-  wait: true,
-  waitGraph: true,
-});
-
-const knowledge = await memory.searchKnowledge({
-  query: "ChatMe 的长期记忆系统用了什么存储？",
-  scope: "chat",
-  scopeId: chatId,
-  mode: "auto",
-});
-
-console.log(doc.docId, knowledge);
-
-const answer = await memory.ask({
-  query: "ChatMe 的长期记忆系统用了什么存储？",
-  scope: "chat",
-  scopeId: chatId,
-  includeKnowledge: true,
-  maxChars: 200,
-});
-
-console.log(answer);
-
-memory.destroy();
-```
-
-## Configuration Reference
-
-`MemoryManager` accepts a `MemoryConfig` object.
-
-Required storage fields:
-
-- `lancedbPath: string` - local LanceDB directory (used when the lancedb backend is active; its parent directory also hosts the backend marker file `.memory-provider`).
-- `grafeoPath: string` - Grafeo database path.
-
-Optional storage fields:
-
-- `provider?: "auto" | "lancedb" | "sqlite"` - vector store backend. Default `"auto"`: probes the @lancedb/lancedb native binding at init and falls back to sqlite when unavailable (e.g. Intel macOS, musl without prebuilds). The first resolved backend is persisted in a marker file; auto mode never silently switches an existing data directory to a different backend (it throws with migration guidance instead).
-- `sqlitePath?: string` - SQLite database file for the sqlite backend. Default: `path.join(path.dirname(lancedbPath), "memory.sqlite3")`, i.e. a `memory.sqlite3` file in the parent directory of `lancedbPath`. Parent directories are created automatically on init.
-
-Storage layout example with `lancedbPath: "./data/memory/lance"` and defaults:
-
-```
-data/memory/
-├── .memory-provider   # backend marker written on first init ("lancedb" or "sqlite")
-├── lance/             # LanceDB tables (lancedb backend only)
-└── memory.sqlite3     # SQLite database (sqlite backend only; plus -wal/-shm files while open)
-```
-
-Backend data files are not interchangeable; switching backends requires an explicit export/import migration.
-
-Required LLM fields:
-
-- `llmBaseUrl: string` - OpenAI-compatible base URL, for example `https://api.openai.com/v1`.
-- `llmApiKey: string` - API key. Never hard-code real secrets in examples.
-- `llmModel: string` - chat model used for summaries, graph decisions, entity extraction, and final `ask` answers.
-
-Required embedding fields:
-
-- `embeddingBaseUrl: string` - OpenAI-compatible embedding base URL.
-- `embeddingApiKey: string` - embedding API key.
-- `embeddingModel: string` - embedding model name.
-- `embeddingDimension: number` - vector dimension. This must match the embedding model and the existing vector store schema.
-
-Embedding fallback behavior:
-
-- If `embeddingBaseUrl` is empty, it falls back to `llmBaseUrl` and logs a warning.
-- If `embeddingApiKey` is empty, it falls back to `llmApiKey` and logs a warning.
-- This is convenient when one provider serves both chat and embeddings.
-
-HTTP and retry fields:
-
-- `httpTimeoutMs?: number` - timeout per LLM or embedding HTTP attempt. Default: `60000`.
-- `httpMaxRetries?: number` - retry count after the first attempt for network errors, timeout/abort, HTTP `429`, and HTTP `5xx`. Default: `2`.
-- `embeddingBatchSize?: number` - max texts per embedding request. Default: `20`.
-- `embeddingConcurrency?: number` - concurrent embedding batches. Default: `2`.
-
+  entitySimilarityThreshold: 0.92,
+  defaultSearchLimit: 10,
+
+  chunkStrategy: "markdown-heading",
+  chunkMaxTokens: 800,
+  chunkOverlap: 0,
+  knowledgeTopK: 8,
+  docCoarseTopK: 5,
+  buildGraphDefault: "auto",
+  chunkRedundantIds: true,
+  knowledgeGraphTriggerScore: 0.78,
+  knowledgeGraphEntityTopK: 10,
+  knowledgeGraphAnchorTopK: 3,
+  knowledgeGraphHopLimit: 10,
+  graphExtractConcurrency: 2,
+  graphBuildTimeoutMs: 120_000,
+});
+
+await memory.init();
+
+const userId = "u-001";
+const chatId = "agent-work";
+const sessionId = "s-2026-06-12";
+
+await memory.updateChat(
+  [
+    {
+      talkerId: "user",
+      content: "我叫 Alice，在 ChatMe 项目中负责后端架构。",
+      metadata: { source: "chat" },
+    },
+    {
+      talkerId: "assistant",
+      content: "明白，我会记住你负责 ChatMe 后端架构。",
+    },
+  ],
+  {
+    userId,
+    chatId,
+    sessionId,
+    sessionTitle: "ChatMe 架构讨论",
+    sessionMetadata: { product: "ChatMe" },
+  }
+);
+
+await memory.updateFacts(
+  "Alice 负责 ChatMe 项目的后端架构",
+  "user",
+  userId,
+  chatId,
+  sessionId
+);
+
+const entities: Entity[] = [
+  { name: "Alice", type: "Person", meta: { role: "backend architect", userId, chatId, sessionId } },
+  { name: "ChatMe", type: "Project", meta: { userId, chatId, sessionId } },
+  { name: "TypeScript", type: "Technology", meta: { userId, chatId, sessionId } },
+];
+
+const relations: Relation[] = [
+  { from: "Alice", to: "ChatMe", type: "works_on", meta: { userId, chatId, sessionId } },
+  { from: "ChatMe", to: "TypeScript", type: "built_with", meta: { userId, chatId, sessionId } },
+];
+
+await memory.updateEntity(entities, relations, { userId, chatId, sessionId });
+
+await memory.flushChat(sessionId, { wait: true });
+
+const remembered = await memory.search({
+  query: "Alice 在 ChatMe 中负责什么？",
+  scope: "user",
+  scopeId: userId,
+  mode: "all",
+  limit: 5,
+});
+
+console.log(remembered);
+
+const doc = await memory.addDocument({
+  content: `# ChatMe 技术说明
+
+ChatMe 使用 TypeScript 构建，后端包含长期记忆系统。
+
+## Memory
+
+长期记忆系统使用向量存储（LanceDB 或 SQLite，自动探测）存储向量，使用 Grafeo 存储知识图谱。`,
+  userId,
+  chatId,
+  sessionId,
+  sourceName: "chatme-memory.md",
+  metadata: { product: "ChatMe" },
+  buildGraph: true,
+  wait: true,
+  waitGraph: true,
+});
+
+const knowledge = await memory.searchKnowledge({
+  query: "ChatMe 的长期记忆系统用了什么存储？",
+  scope: "chat",
+  scopeId: chatId,
+  mode: "auto",
+});
+
+console.log(doc.docId, knowledge);
+
+const answer = await memory.ask({
+  query: "ChatMe 的长期记忆系统用了什么存储？",
+  scope: "chat",
+  scopeId: chatId,
+  includeKnowledge: true,
+  maxChars: 200,
+});
+
+console.log(answer);
+
+await memory.destroy();
+```
+
+## Configuration Reference
+
+`MemoryManager` accepts a `MemoryConfig` object.
+
+Required storage fields:
+
+- `lancedbPath: string` - local LanceDB directory (used when the lancedb backend is active; its parent directory also hosts the backend marker file `.memory-provider`).
+- `grafeoPath: string` - Grafeo database path.
+
+Optional storage fields:
+
+- `provider?: "auto" | "lancedb" | "sqlite"` - vector store backend. Default `"auto"`: probes the @lancedb/lancedb native binding at init and falls back to sqlite when unavailable (e.g. Intel macOS, musl without prebuilds). The first resolved backend is persisted in a marker file; auto mode never silently switches an existing data directory to a different backend (it throws with migration guidance instead).
+- `sqlitePath?: string` - SQLite database file for the sqlite backend. Default: `path.join(path.dirname(lancedbPath), "memory.sqlite3")`, i.e. a `memory.sqlite3` file in the parent directory of `lancedbPath`. Parent directories are created automatically on init.
+
+Storage layout example with `lancedbPath: "./data/memory/lance"` and defaults:
+
+```
+data/memory/
+├── .memory-provider   # backend marker written on first init ("lancedb" or "sqlite")
+├── lance/             # LanceDB tables (lancedb backend only)
+└── memory.sqlite3     # SQLite database (sqlite backend only; plus -wal/-shm files while open)
+```
+
+Backend data files are not interchangeable; switching backends requires an explicit export/import migration.
+
+Required LLM fields:
+
+- `llmBaseUrl: string` - OpenAI-compatible base URL, for example `https://api.openai.com/v1`.
+- `llmApiKey: string` - API key. Never hard-code real secrets in examples.
+- `llmModel: string` - chat model used for summaries, graph decisions, entity extraction, and final `ask` answers.
+
+Required embedding fields:
+
+- `embeddingBaseUrl: string` - OpenAI-compatible embedding base URL.
+- `embeddingApiKey: string` - embedding API key.
+- `embeddingModel: string` - embedding model name.
+- `embeddingDimension: number` - vector dimension. This must match the embedding model and the existing vector store schema.
+
+Embedding fallback behavior:
+
+- If `embeddingBaseUrl` is empty, it falls back to `llmBaseUrl` and logs a warning.
+- If `embeddingApiKey` is empty, it falls back to `llmApiKey` and logs a warning.
+- This is convenient when one provider serves both chat and embeddings.
+
+HTTP and retry fields:
+
+- `httpTimeoutMs?: number` - timeout per LLM or embedding HTTP attempt. Default: `60000`.
+- `httpMaxRetries?: number` - retry count after the first attempt for network errors, timeout/abort, HTTP `429`, and HTTP `5xx`. Default: `2`.
+- `embeddingBatchSize?: number` - max texts per embedding request. Default: `20`.
+- `embeddingConcurrency?: number` - concurrent embedding batches. Default: `2`.
+
 Conversation memory fields:
 
-- `sessionTokenLimit?: number` - when a session cache reaches this token count, compression is triggered. Default: `16386`.
-- `historyWindowTokenLimit?: number` - maximum token budget for the rebuilt compressed history window. Default: `10240`.
-- `topicRatio?: [number, number, number]` - number ratio for detail/summary/concise topics in the history window. Default: `[1, 5, 20]`.
-- `detailMaxTokens?: number` - max detailed summary tokens requested from the LLM. Default: `2048`.
-- `summaryMaxTokens?: number` - max medium summary tokens requested from the LLM. Default: `512`.
-- `conciseMaxTokens?: number` - max concise summary tokens requested from the LLM. Default: `128`.
+- `compressedContextTokenLimit?: number` - fixed token budget for compressed Topics. Default: `16384`; startup logs a warning above `32768`.
+- `contextUsageRatio?: number` - fraction of the current model context available to conversation history. Default: `0.75`.
+- `precompressionRatio?: number` - start background raw-message compression when raw usage reaches this fraction of its dynamic budget. Default: `0.75`.
+- `compressionBatchRatio?: number` - target fraction of the raw-message budget compressed in one batch. Default: `0.5`.
+- `compressionBatchTokenLimit?: number` - optional hard limit for one compression batch. Default: `0` (ratio only).
+- `topicSummaryMaxTokens?: number` - hard maximum for one Topic summary. The LLM uses less for low-value chat and more for facts, experience, decisions, preferences, constraints, and reusable knowledge. Default: `2048`.
+- `defaultModelContextTokens?: number` - used when `getHistoryWindow` receives no model size; a warning is logged. Default: `262144`.
+- `maxHistoryAgeMs?: number` - maximum Topic and raw-message age restored during lazy cold start. Default: `0` (all history).
+- `sessionIdleTtlMs?: number` - idle time before a session cache is released. Persistent data is not deleted. Default: `1800000`.
+- `sessionSweepIntervalMs?: number` - idle cache sweep interval. Default: `60000`.
 - `maxConcurrentCompressions?: number` - global semaphore limit for concurrent compression and knowledge ingestion tasks. Default: `3`.
 - `entitySimilarityThreshold?: number` - graph entity similarity threshold. Default: `0.92`.
 - `defaultSearchLimit?: number` - default `search` result limit. Default: `10`.
-- `recallBoostMs?: number` - each topic recall count acts like this many milliseconds of freshness boost. Default: `3_600_000`.
-
-Knowledge-base fields:
-
-- `chunkStrategy?: "markdown-heading"` - current chunking strategy. Default: `"markdown-heading"`.
-- `chunkMaxTokens?: number` - max tokens per chunk; long heading sections are split again. Default: `800`.
-- `chunkOverlap?: number` - overlap tokens between chunks. Default: `0`.
-- `knowledgeTopK?: number` - default `searchKnowledge` chunk return count. Default: `8`.
-- `docCoarseTopK?: number` - document-level coarse recall count before chunk search. Set `0` to disable coarse recall. Default: `5`.
-- `buildGraphDefault?: boolean | "auto"` - default graph build behavior for `addDocument`. Default: `"auto"`.
-- `chunkRedundantIds?: boolean` - whether chunks redundantly store `userId`, `chatId`, and `sessionId` for direct filtering. Default: `true`.
-- `knowledgeGraphTriggerScore?: number` - in `mode: "auto"`, graph expansion only starts when top chunk cosine score reaches this threshold. Default: `0.78`.
-- `knowledgeGraphEntityTopK?: number` - entity vector search candidate count for knowledge graph expansion. Default: `10`.
-- `knowledgeGraphAnchorTopK?: number` - top entity anchors used for one-hop expansion. Default: `3`.
-- `knowledgeGraphHopLimit?: number` - max one-hop relations per anchor. Default: `10`.
-- `graphExtractConcurrency?: number` - concurrent chunk-level entity extraction tasks during document graph construction. Default: `2`.
-- `graphBuildTimeoutMs?: number` - timeout used only when a caller explicitly waits for graph construction with `waitGraph: true`. Default: `120000`.
-
-## Critical Async And Wait Semantics
-
-The package intentionally separates durable base writes from expensive graph construction.
 
+`sessionTokenLimit`, `historyWindowTokenLimit`, `topicRatio`, `detailMaxTokens`, and `conciseMaxTokens` remain only as deprecated 0.3.x configuration/read compatibility fields. New code should not use them.
+
+Knowledge-base fields:
+
+- `chunkStrategy?: "markdown-heading"` - current chunking strategy. Default: `"markdown-heading"`.
+- `chunkMaxTokens?: number` - max tokens per chunk; long heading sections are split again. Default: `800`.
+- `chunkOverlap?: number` - overlap tokens between chunks. Default: `0`.
+- `knowledgeTopK?: number` - default `searchKnowledge` chunk return count. Default: `8`.
+- `docCoarseTopK?: number` - document-level coarse recall count before chunk search. Set `0` to disable coarse recall. Default: `5`.
+- `buildGraphDefault?: boolean | "auto"` - default graph build behavior for `addDocument`. Default: `"auto"`.
+- `chunkRedundantIds?: boolean` - whether chunks redundantly store `userId`, `chatId`, and `sessionId` for direct filtering. Default: `true`.
+- `knowledgeGraphTriggerScore?: number` - in `mode: "auto"`, graph expansion only starts when top chunk cosine score reaches this threshold. Default: `0.78`.
+- `knowledgeGraphEntityTopK?: number` - entity vector search candidate count for knowledge graph expansion. Default: `10`.
+- `knowledgeGraphAnchorTopK?: number` - top entity anchors used for one-hop expansion. Default: `3`.
+- `knowledgeGraphHopLimit?: number` - max one-hop relations per anchor. Default: `10`.
+- `graphExtractConcurrency?: number` - concurrent chunk-level entity extraction tasks during document graph construction. Default: `2`.
+- `graphBuildTimeoutMs?: number` - timeout used only when a caller explicitly waits for graph construction with `waitGraph: true`. Default: `120000`.
+
+## Critical Async And Wait Semantics
+
+The package intentionally separates durable base writes from expensive graph construction.
+
 `updateChat(messages, opts)`:
 
-- Embeds and writes raw messages to the vector store before resolving.
+- Registers its write synchronously, so a caller may intentionally fire-and-forget it and a following `getHistoryWindow` will still wait for that write.
+- Calculates tokens over `content`, `parts`, `payload`, and `metadata`, embeds the searchable text, and upserts raw messages by stable `messageId` before resolving.
 - Creates or updates the session record.
 - Updates the in-memory session cache.
-- If `sessionTokenLimit` is exceeded, compression is triggered in the background.
-- Background compression may create topics/history and graph entities later.
-
+- Once `getHistoryWindow` has supplied the current model size, reaching the precompression threshold starts background compression.
+- Topic storage and cache replacement finish before conversation graph extraction; graph work remains asynchronous.
+
 `flushChat(sessionId?, opts?)`:
-
-- Forces compression for the current cached messages even if `sessionTokenLimit` was not reached.
-- `wait: true` waits for topic creation, cache clearing, and history-window rebuild.
-- `wait: true` does not wait for graph extraction/persistence.
-- `waitGraph: true` waits for graph extraction/persistence too.
+
+- Forces compression for the current cached raw messages even if the dynamic precompression threshold was not reached.
+- `wait: true` waits for topic creation, cache clearing, and history-window rebuild.
+- `wait: true` does not wait for graph extraction/persistence.
+- `waitGraph: true` waits for graph extraction/persistence too.
 - When `waitGraph: true`, graph waiting is bounded by `graphBuildTimeoutMs` and may throw a timeout error.
 
-`addDocument(opts)`:
-
-- Writes the document row first.
-- Chunk embedding and chunk insertion run through the ingestion pipeline.
-- Graph construction, when enabled, runs after chunks are ready.
-- With neither `wait` nor `waitGraph`, the method returns after document storage; chunks and graph are background work.
-- `wait: true` waits until chunks are inserted and searchable.
-- `wait: true` does not wait for graph construction.
-- `waitGraph: true` implies waiting for chunks and then waits for graph construction.
-- When `waitGraph: true`, graph waiting is bounded by `graphBuildTimeoutMs` and may throw a timeout error.
-
-Use `wait: true` when the next line of code must immediately call `searchKnowledge` and find chunks. Use `waitGraph: true` only when `hasGraph` or graph hits must be ready immediately.
-
-## Data Model
-
-`RawMessage` is input to `updateChat`:
-
-- `messageId?: string` - generated with UUID if omitted.
-- `talkerId?: string` - speaker id. Default: `"user"`.
-- `chatId?: string` - conversation/chat/agent id.
-- `userId?: string` - owner user id.
-- `sessionId?: string` - session id.
-- `type?: "text" | "image" | "file"` - default: `"text"`.
-- `content: string` - required plain text used for embedding and retrieval.
+`getHistoryWindow(sessionId, modelContextTokens?)`:
+
+- Lazily hydrates the session from persistent Topics plus raw messages after the newest Topic boundary.
+- Waits for registered message writes. If the hard raw budget is exceeded, it also waits for or starts compression until the returned context fits.
+- Returns `{ compressedContext, recentMessages, usage }`. `recentMessages` preserves the `RawMessage` input shape, including `messageId`, `parts`, `payload`, `metadata`, and `createdAt`.
+- Topic budget is fixed by manager configuration; the raw-message budget is recalculated from the supplied model context size.
+- If accumulated Topics exceed their fixed budget, the oldest approximately half are summarized into one Topic. This Topic-to-Topic rollup does not rebuild graph data.
+
+`addDocument(opts)`:
+
+- Writes the document row first.
+- Chunk embedding and chunk insertion run through the ingestion pipeline.
+- Graph construction, when enabled, runs after chunks are ready.
+- With neither `wait` nor `waitGraph`, the method returns after document storage; chunks and graph are background work.
+- `wait: true` waits until chunks are inserted and searchable.
+- `wait: true` does not wait for graph construction.
+- `waitGraph: true` implies waiting for chunks and then waits for graph construction.
+- When `waitGraph: true`, graph waiting is bounded by `graphBuildTimeoutMs` and may throw a timeout error.
+
+Use `wait: true` when the next line of code must immediately call `searchKnowledge` and find chunks. Use `waitGraph: true` only when `hasGraph` or graph hits must be ready immediately.
+
+## Data Model
+
+`RawMessage` is input to `updateChat`:
+
+- `messageId?: string` - generated with UUID if omitted.
+- `talkerId?: string` - speaker id. Default: `"user"`.
+- `chatId?: string` - conversation/chat/agent id.
+- `userId?: string` - owner user id.
+- `sessionId?: string` - session id.
+- `type?: "text" | "image" | "file"` - default: `"text"`.
+- `content: string` - required plain text used for embedding and retrieval.
 - `parts?: ContentPart[]` - optional multimodal content parts; stored serialized.
+- `payload?: unknown` - host-framework message payload stored and returned without interpretation.
 - `usage?: number` - token count; estimated with `tiktoken` if omitted.
-- `metadata?: Record<string, unknown>` - custom metadata stored as JSON.
-- `createdAt?: number` - Unix milliseconds; default is current time.
-
-`ContentPart` aligns with common multimodal content shapes:
-
-- Text: `{ type: "text", text: "..." }`
-- Image: `{ type: "image_url", image_url: { url: "https://..." } }`
-- File: `{ type: "file_url", file_url: { url: "https://...", name: "file.pdf" } }`
-
-`StoredMessage` is persisted message output with required ids, serialized `parts`, serialized `metadata`, generated vector, and `createdAt`.
-
+- `metadata?: Record<string, unknown>` - custom metadata stored as JSON.
+- `createdAt?: number` - Unix milliseconds; default is current time.
+
+`ContentPart` aligns with common multimodal content shapes:
+
+- Text: `{ type: "text", text: "..." }`
+- Image: `{ type: "image_url", image_url: { url: "https://..." } }`
+- File: `{ type: "file_url", file_url: { url: "https://...", name: "file.pdf" } }`
+
+`StoredMessage` is persisted message output with required ids, serialized `parts`, serialized `metadata`, generated vector, and `createdAt`.
+
 `Topic` is a compressed memory item:
 
 - `title` - short topic title.
-- `detail` - rich summary.
-- `summary` - medium summary.
-- `concise` - short summary.
+- `summary` - the only compressed body; its actual length is importance-aware and bounded by `topicSummaryMaxTokens`.
+- `tokens` - summary token count stored when the Topic is created.
+- `startMessageId` / `endMessageId` - exact raw-message boundaries, in addition to timestamps.
 - `startTime` and `endTime` - covered message time range.
-- `recallCount` - affects freshness ranking through `recallBoostMs`.
-
+- `recallCount` - incremented best-effort when search recalls the Topic; it is not used to reorder the continuous history window.
+
 `Fact` is a user/chat scoped manual memory:
-
-- `level: "user" | "chat"`
-- `userId`, `chatId`, optional `sessionId`
+
+- `level: "user" | "chat"`
+- `userId`, `chatId`, optional `sessionId`
 - `content`
-- `createdAt`
-
-`Entity` and `Relation` are graph records:
-
-- Entity: `{ name, type, meta }`
-- Relation: `{ from, to, type, happenedAt?, meta }`
-- `meta` should usually include `userId`, `chatId`, `sessionId`, and source details.
-
-`SessionView` is the in-memory session representation:
-
-- `sessionId`
-- `chatId`
-- `userId`
-- `title`
-- `metadata: Record<string, unknown>`
+- optional stable `key` for immediate cache + database upsert
 - `createdAt`
 - `updatedAt`
-
-`Document` is a knowledge-base document row:
-
-- `docId`
-- `userId`, `chatId`, `sessionId`
-- `title`, `sourceName`
-- `fullContent`
-- `contentHash`
-- `summary`
-- `summaryVector`
-- `chunkCount`
-- `hasGraph`
-- `metadata`
-- `createdAt`, `updatedAt`
-
-`Chunk` is a knowledge-base chunk row:
-
-- `chunkId`
-- `docId`
-- `content`
-- `headingPath`
-- `ordinal`
-- `tokens`
-- `vector`
-- optional redundant domain ids depending on `chunkRedundantIds`
-
-## Scope And Search Modes
-
-Scopes:
-
-- `scope: "session"` - filter by `sessionId`; pass `scopeId`.
-- `scope: "chat"` - filter by `chatId`; pass `scopeId`.
-- `scope: "user"` - filter by `userId`; pass `scopeId`.
-- `scope: "all"` - no domain filter; `scopeId` is ignored.
-
+
+`Entity` and `Relation` are graph records:
+
+- Entity: `{ name, type, meta }`
+- Relation: `{ from, to, type, happenedAt?, meta }`
+- `meta` should usually include `userId`, `chatId`, `sessionId`, and source details.
+
+`SessionView` is the in-memory session representation:
+
+- `sessionId`
+- `chatId`
+- `userId`
+- `title`
+- `metadata: Record<string, unknown>`
+- `createdAt`
+- `updatedAt`
+
+`Document` is a knowledge-base document row:
+
+- `docId`
+- `userId`, `chatId`, `sessionId`
+- `title`, `sourceName`
+- `fullContent`
+- `contentHash`
+- `summary`
+- `summaryVector`
+- `chunkCount`
+- `hasGraph`
+- `metadata`
+- `createdAt`, `updatedAt`
+
+`Chunk` is a knowledge-base chunk row:
+
+- `chunkId`
+- `docId`
+- `content`
+- `headingPath`
+- `ordinal`
+- `tokens`
+- `vector`
+- optional redundant domain ids depending on `chunkRedundantIds`
+
+## Scope And Search Modes
+
+Scopes:
+
+- `scope: "session"` - filter by `sessionId`; pass `scopeId`.
+- `scope: "chat"` - filter by `chatId`; pass `scopeId`.
+- `scope: "user"` - filter by `userId`; pass `scopeId`.
+- `scope: "all"` - no domain filter; `scopeId` is ignored.
+
 Search modes:
-
-- `mode: "fast"` - only vector-store hybrid/vector retrieval. No graph search.
-- `mode: "auto"` - conversation search asks the LLM whether graph search is needed; knowledge search triggers graph expansion only when top chunk score is high enough.
+
+- `mode: "fast"` - only vector-store hybrid/vector retrieval. No graph search.
+- `mode: "auto"` - conversation search asks the LLM whether graph search is needed; knowledge search triggers graph expansion only when top chunk score is high enough.
 - `mode: "all"` - include graph search and merge graph results with vector/full-text results.
 
-For deterministic low-latency calls, use `fast`. For richer entity relationship answers, use `auto` or `all`.
-
-## API Reference
-
-### `new MemoryManager(config)`
-
-Creates the manager and resolves defaults. It does not connect to storage until `init()`.
-
+Conversation search always searches both compressed Topics and raw messages, including raw rows already represented by a Topic. For deterministic low-latency calls, use `fast`. For richer entity relationship answers, use `auto` or `all`.
+
+## API Reference
+
+### `new MemoryManager(config)`
+
+Creates the manager and resolves defaults. It does not connect to storage until `init()`.
+
 ### `init(): Promise<void>`
 
-Initializes tiktoken, the vector store (backend auto-detection happens here), Grafeo, facts cache, sessions cache, and restored history windows. Always call before using read/write APIs.
-
-### `updateChat(messages, opts?): Promise<void>`
-
-Stores new chat messages.
-
-Options:
-
-- `userId?: string`
-- `chatId?: string`
-- `sessionId?: string`
-- `sessionTitle?: string`
-- `sessionMetadata?: Record<string, unknown>`
-
-Defaults for ids are `"default"`.
-
-Use for every new user/assistant message that should become retrievable memory.
-
-### `flushChat(sessionId?, opts?): Promise<void>`
-
-Forces compression of a session cache.
-
-Options:
-
-- `wait?: boolean` - wait for topic/history work.
-- `waitGraph?: boolean` - also wait for graph extraction/persistence, bounded by `graphBuildTimeoutMs`.
-
-If `sessionId` is omitted, `"default"` is used.
-
-### `search(opts): Promise<SearchResult[]>`
-
-Searches conversation memory.
-
-Options:
-
-- `query: string`
-- `scope?: "session" | "chat" | "user" | "all"`
-- `scopeId?: string`
-- `mode?: "fast" | "auto" | "all"`
-- `limit?: number`
-
-Returns results with:
-
-- `type: "message" | "topic" | "entity" | "relation"`
-- `content`
-- `score`
-- `meta`
-
-### `ask(opts): Promise<string>`
-
-Searches memory and asks the configured LLM to summarize the hits into a natural-language answer.
-
-Options are `SearchOptions` plus:
-
-- `maxChars?: number`
-- `includeKnowledge?: boolean`
-
-When `includeKnowledge: true`, the method also calls `searchKnowledge` and merges knowledge chunks and graph hits into the answer context.
-
-### `updateFacts(content, level, userId, chatId, sessionId?): Promise<void>`
-
-Adds a fact to persistent storage and in-memory cache.
-
-- `level` is `"user"` or `"chat"`.
-- User facts apply across chats for that user.
-- Chat facts apply to a specific chat/agent context.
-
-### `getFacts(level, id): Promise<string>`
-
-Returns facts as newline-separated text in `ISO_TIME：content` format. Use `level: "user"` with `userId`, or `level: "chat"` with `chatId`.
-
-### `getFactsForContext(userId, chatId): Promise<string>`
-
-Returns merged user-level facts for `userId` and chat-level facts for `chatId`, sorted by time. This is useful for injecting stable facts into an agent prompt.
-
-### `addFact(content, level, userId, chatId, sessionId?): Promise<void>`
-
-Alias-style management API for adding a fact manually.
-
-### `deleteFact(factId): Promise<boolean>`
-
-Deletes one fact and returns whether it was found.
-
-### `updateEntity(entities, relations, context?): Promise<void>`
-
-Upserts graph entities and relations. The method embeds entity names and stores graph data in Grafeo.
-
-Context:
-
-- `sessionId?: string`
-- `chatId?: string`
-- `userId?: string`
-
-### `getHistoryWindow(sessionId): string`
-
-Returns the compressed history window for a session. This is prompt-ready text built from recent `Topic` records using `topicRatio` and `historyWindowTokenLimit`.
-
-### `getRecentMessages(sessionId, limit): Promise<StoredMessage[]>`
-
-Returns the globally latest `limit` messages for a session in chronological order (oldest to newest within the returned window). The storage query orders by `createdAt` and uses `messageId` as a deterministic same-timestamp tiebreak before applying the limit, so long sessions do not return an early candidate window.
-
-### Session APIs
-
-- `getSession(sessionId): SessionView | null`
-- `getSessionsByUserId(userId, opts?): SessionView[]`
-- `getSessionsByChatId(chatId, opts?): SessionView[]`
-- `searchSessions(opts): SessionView[]`
-- `updateSession(sessionId, opts): Promise<SessionView | null>`
-- `deleteSession(sessionId): Promise<boolean>`
-- `listSessions(filter?): SessionView[]`
-
-Session update options:
-
-- `title?: string`
+Initializes tiktoken, the vector store (backend auto-detection happens here), Grafeo, and the facts cache. Conversation sessions are hydrated lazily on first access and evicted after the configured idle timeout. Always call before using read/write APIs.
+
+### `updateChat(messages, opts?): Promise<void>`
+
+Stores new chat messages.
+
+Options:
+
+- `userId?: string`
+- `chatId?: string`
+- `sessionId?: string`
+- `sessionTitle?: string`
+- `sessionMetadata?: Record<string, unknown>`
+
+Defaults for ids are `"default"`.
+
+Use for every new user/assistant message that should become retrievable memory.
+
+### `flushChat(sessionId?, opts?): Promise<void>`
+
+Forces compression of a session cache.
+
+Options:
+
+- `wait?: boolean` - wait for topic/history work.
+- `waitGraph?: boolean` - also wait for graph extraction/persistence, bounded by `graphBuildTimeoutMs`.
+
+If `sessionId` is omitted, `"default"` is used.
+
+### `search(opts): Promise<SearchResult[]>`
+
+Searches conversation memory.
+
+Options:
+
+- `query: string`
+- `scope?: "session" | "chat" | "user" | "all"`
+- `scopeId?: string`
+- `mode?: "fast" | "auto" | "all"`
+- `limit?: number`
+
+Returns results with:
+
+- `type: "message" | "topic" | "entity" | "relation"`
+- `content`
+- `score`
+- `meta`
+
+### `ask(opts): Promise<string>`
+
+Searches memory and asks the configured LLM to summarize the hits into a natural-language answer.
+
+Options are `SearchOptions` plus:
+
+- `maxChars?: number`
+- `includeKnowledge?: boolean`
+
+When `includeKnowledge: true`, the method also calls `searchKnowledge` and merges knowledge chunks and graph hits into the answer context.
+
+### `updateFacts(content, level, userId, chatId, sessionId?): Promise<void>`
+
+Adds a fact to persistent storage and in-memory cache.
+
+- `level` is `"user"` or `"chat"`.
+- User facts apply across chats for that user.
+- Chat facts apply to a specific chat/agent context.
+
+### `getFacts(level, id): Promise<string>`
+
+Returns facts as newline-separated text in `ISO_TIME：content` format. Use `level: "user"` with `userId`, or `level: "chat"` with `chatId`.
+
+### `getFactsForContext(userId, chatId): Promise<string>`
+
+Returns merged user-level facts for `userId` and chat-level facts for `chatId`, sorted by time. This is useful for injecting stable facts into an agent prompt.
+
+### `addFact(content, level, userId, chatId, sessionId?): Promise<void>`
+
+Alias-style management API for adding a fact manually.
+
+### `deleteFact(factId): Promise<boolean>`
+
+Deletes one fact and returns whether it was found.
+
+### `updateEntity(entities, relations, context?): Promise<void>`
+
+Upserts graph entities and relations. The method embeds entity names and stores graph data in Grafeo.
+
+Context:
+
+- `sessionId?: string`
+- `chatId?: string`
+- `userId?: string`
+
+### `getHistoryWindow(sessionId, modelContextTokens?): Promise<MemoryContextWindow>`
+
+Returns the complete model-history window:
+
+- `compressedContext: string` - chronological Topic summaries within the fixed compressed budget.
+- `recentMessages: MemoryRawMessage[]` - all currently uncompressed raw messages in chronological order, preserving the host payload and metadata.
+- `usage` - model size, usable history size, compressed usage, and dynamic raw-message budget/usage.
+
+Pass the active model's context size on every read. Omitting it uses `defaultModelContextTokens` (256K by default) and logs a warning.
+
+### `getRecentMessages(sessionId, limit): Promise<StoredMessage[]>`
+
+Returns the globally latest `limit` messages for a session in chronological order (oldest to newest within the returned window). The storage query orders by `createdAt` and uses `messageId` as a deterministic same-timestamp tiebreak before applying the limit, so long sessions do not return an early candidate window.
+
+### Session APIs
+
+- `getSession(sessionId): SessionView | null`
+- `getSessionsByUserId(userId, opts?): SessionView[]`
+- `getSessionsByChatId(chatId, opts?): SessionView[]`
+- `searchSessions(opts): SessionView[]`
+- `updateSession(sessionId, opts): Promise<SessionView | null>`
+- `deleteSession(sessionId): Promise<boolean>`
+- `listSessions(filter?): SessionView[]`
+
+Session update options:
+
+- `title?: string`
 - `metadata?: Record<string, unknown>`
-
-Session search options:
-
-- `title?: string`
-- `userId?: string`
-- `chatId?: string`
-- `limit?: number`
-
-### Management APIs
-
-- `stats(): Promise<{ sessions; messages; topics; facts; entities; relations; documents; chunks }>` - total counts.
-- `trend(days?): Promise<Array<{ date; sessions; messages; facts }>>` - daily activity trend. `days` defaults to `30` and is clamped to `1..365`.
-- `listMessages(sessionId, limit?): Promise<StoredMessage[]>`
-- `listTopics(filter?): Promise<Topic[]>`
-- `listFacts(filter?): Fact[]`
-- `listEntities(): Promise<Entity[]>`
-- `listRelations(): Promise<Relation[]>`
-
-### `addDocument(opts): Promise<{ docId: string }>`
-
-Ingests a Markdown document into the knowledge base.
-
-Options:
-
-- `content: string` - required Markdown content.
-- `userId?: string`
-- `chatId?: string`
-- `sessionId?: string`
-- `title?: string` - inferred from first Markdown heading or first non-empty line when omitted.
-- `sourceName?: string`
-- `metadata?: Record<string, unknown>`
-- `buildGraph?: boolean | "auto"` - default from `config.buildGraphDefault`.
-- `wait?: boolean` - wait for chunks to be embedded and inserted.
-- `waitGraph?: boolean` - wait for chunks and graph construction.
-
-Deduplication:
-
-- Documents are deduplicated by `sha256(content)` within the same `userId + chatId + sessionId` domain.
-- If a duplicate exists, `addDocument` returns the existing `docId`.
-
-Graph behavior:
-
-- `buildGraph: false` disables graph construction.
-- `buildGraph: true` builds a graph.
-- `buildGraph: "auto"` uses the LLM document summary call to decide whether the document has valuable entity relationships.
-
-### `searchKnowledge(opts): Promise<KnowledgeSearchResult>`
-
-Searches Markdown knowledge-base chunks.
-
-Options:
-
-- `query: string`
-- `scope?: "session" | "chat" | "user" | "all"`
-- `scopeId?: string`
-- `mode?: "fast" | "auto" | "all"`
-- `limit?: number`
-
-Returns:
-
-- `chunks: Array<{ chunkId; docId; content; headingPath; score }>`
-- `documents: Array<{ docId; title; matchedChunkCount }>`
-- `graphHits: SearchResult[]`
-
-The method uses document-level coarse recall when `docCoarseTopK > 0`, then chunk-level search. In `auto` mode, knowledge graph expansion only happens when the top chunk score reaches `knowledgeGraphTriggerScore`.
-
-### `getDocument(docId): Promise<Document | null>`
-
-Returns a document including `fullContent`.
-
-### `deleteDocument(docId): Promise<boolean>`
-
-Deletes the document, its chunks, and best-effort Grafeo knowledge graph entries for that document.
-
-### `listDocuments(filter?): Promise<Document[]>`
-
-Lists documents, optionally filtered by:
-
-- `userId`
-- `chatId`
-- `sessionId`
-
-### `destroy(): void`
-
-Closes the Grafeo connection. It also closes the vector store connection. It does not delete vector store or Grafeo data.
-
+- `scope?: "session" | "chat" | "user" | "all"` - content-hash deduplication boundary; default is `session`.
+
+Session search options:
+
+- `title?: string`
+- `userId?: string`
+- `chatId?: string`
+- `limit?: number`
+
+### Management APIs
+
+- `stats(): Promise<{ sessions; messages; topics; facts; entities; relations; documents; chunks }>` - total counts.
+- `trend(days?): Promise<Array<{ date; sessions; messages; facts }>>` - daily activity trend. `days` defaults to `30` and is clamped to `1..365`.
+- `listMessages(sessionId, limit?): Promise<StoredMessage[]>`
+- `listTopics(filter?): Promise<Topic[]>`
+- `listFacts(filter?): Fact[]`
+- `listEntities(): Promise<Entity[]>`
+- `listRelations(): Promise<Relation[]>`
+
+### `addDocument(opts): Promise<{ docId: string }>`
+
+Ingests a Markdown document into the knowledge base.
+
+Options:
+
+- `content: string` - required Markdown content.
+- `userId?: string`
+- `chatId?: string`
+- `sessionId?: string`
+- `title?: string` - inferred from first Markdown heading or first non-empty line when omitted.
+- `sourceName?: string`
+- `metadata?: Record<string, unknown>`
+- `buildGraph?: boolean | "auto"` - default from `config.buildGraphDefault`.
+- `wait?: boolean` - wait for chunks to be embedded and inserted.
+- `waitGraph?: boolean` - wait for chunks and graph construction.
+
+Deduplication:
+
+- Documents are deduplicated by `sha256(content)` within the requested scope (`session` by default, or `chat` / `user` / `all`).
+- If a duplicate exists, `addDocument` returns the existing `docId`.
+
+Graph behavior:
+
+- `buildGraph: false` disables graph construction.
+- `buildGraph: true` builds a graph.
+- `buildGraph: "auto"` uses the LLM document summary call to decide whether the document has valuable entity relationships.
+
+### `searchKnowledge(opts): Promise<KnowledgeSearchResult>`
+
+Searches Markdown knowledge-base chunks.
+
+Options:
+
+- `query: string`
+- `scope?: "session" | "chat" | "user" | "all"`
+- `scopeId?: string`
+- `mode?: "fast" | "auto" | "all"`
+- `limit?: number`
+
+Returns:
+
+- `chunks: Array<{ chunkId; docId; content; headingPath; score }>`
+- `documents: Array<{ docId; title; matchedChunkCount }>`
+- `graphHits: SearchResult[]`
+
+The method uses document-level coarse recall when `docCoarseTopK > 0`, then chunk-level search. In `auto` mode, knowledge graph expansion only happens when the top chunk score reaches `knowledgeGraphTriggerScore`.
+
+### `getDocument(docId): Promise<Document | null>`
+
+Returns a document including `fullContent`.
+
+### `deleteDocument(docId): Promise<boolean>`
+
+Deletes the document, its chunks, and best-effort Grafeo knowledge graph entries for that document.
+
+### `listDocuments(filter?): Promise<Document[]>`
+
+Lists documents, optionally filtered by:
+
+- `userId`
+- `chatId`
+- `sessionId`
+
+### `destroy(): Promise<void>`
+
+Waits for registered writes, compression, graph work, and storage maintenance, then closes Grafeo and the vector store. It does not delete data.
+
 ## Common Recipes
 
-### Store User And Assistant Messages
+### Migrate A 0.3.x Store
 
-```ts
-await memory.updateChat(
-  [
-    { talkerId: "user", content: "我喜欢 TypeScript，不太喜欢写纯 JavaScript。" },
-    { talkerId: "assistant", content: "记住了，你偏好 TypeScript。" },
-  ],
-  { userId: "u1", chatId: "assistant-main", sessionId: "s1" }
-);
+Stop all processes that write the store, then run the manual, idempotent migration. It backfills single-field Topic summaries/tokens/boundaries, raw-message usage, and fact timestamps without deleting raw messages or rerunning the LLM.
+
+```bash
+ppagent-memory migrate --config ./config.json --dry-run
+ppagent-memory migrate --config ./config.json --backup ./memory-backup
 ```
 
+The config loader accepts a direct `MemoryConfig`, `memory`, `app.memory.auto`, or `service.memory` object. `--dry-run` migrates a temporary copy. `--backup` copies both supported vector-store files/directories plus Grafeo data and writes rollback guidance before mutating the live store.
+
+### Store User And Assistant Messages
+
+```ts
+await memory.updateChat(
+  [
+    { talkerId: "user", content: "我喜欢 TypeScript，不太喜欢写纯 JavaScript。" },
+    { talkerId: "assistant", content: "记住了，你偏好 TypeScript。" },
+  ],
+  { userId: "u1", chatId: "assistant-main", sessionId: "s1" }
+);
+```
+
 ### Force A Prompt-Ready History Window
 
 ```ts
 await memory.flushChat("s1", { wait: true });
-const history = memory.getHistoryWindow("s1");
-
-const systemPrompt = `你是助手。以下是长期记忆摘要：\n${history}`;
-```
-
-### Wait For Conversation Graph Data
-
-```ts
-await memory.flushChat("s1", {
-  wait: true,
-  waitGraph: true,
-});
-```
-
-Use this only when graph data must be ready immediately. It can throw if graph construction exceeds `graphBuildTimeoutMs`.
-
-### Store Stable Facts
-
-```ts
-await memory.updateFacts("用户偏好 TypeScript", "user", "u1", "assistant-main", "s1");
-
-const facts = await memory.getFactsForContext("u1", "assistant-main");
-```
-
-### Search Fast Without Graph
-
-```ts
-const hits = await memory.search({
-  query: "用户偏好什么语言？",
-  scope: "user",
-  scopeId: "u1",
-  mode: "fast",
-});
-```
-
-### Search With Graph
-
-```ts
-const hits = await memory.search({
-  query: "Alice 负责哪个项目，项目用了什么技术？",
-  scope: "chat",
-  scopeId: "work-agent",
-  mode: "all",
-});
-```
-
-### Ingest Markdown And Search Immediately
-
-```ts
-const { docId } = await memory.addDocument({
-  content: "# Rust\n\nRust 的所有权系统保证内存安全。",
-  userId: "u1",
-  chatId: "dev-agent",
-  sourceName: "rust.md",
-  wait: true,
-});
-
-const result = await memory.searchKnowledge({
-  query: "Rust 如何保证内存安全？",
-  scope: "chat",
-  scopeId: "dev-agent",
-  mode: "fast",
-});
-```
-
-### Ingest Markdown And Wait For Knowledge Graph
-
-```ts
-await memory.addDocument({
-  content: markdown,
-  userId: "u1",
-  chatId: "work-agent",
-  sourceName: "project.md",
-  buildGraph: true,
-  waitGraph: true,
-});
-```
-
-### Ask With Conversation Memory And Knowledge Base
-
-```ts
-const answer = await memory.ask({
-  query: "这个项目的技术栈是什么？结合我之前说过的内容回答。",
-  scope: "chat",
-  scopeId: "work-agent",
-  mode: "auto",
-  includeKnowledge: true,
-  maxChars: 300,
-});
+const history = await memory.getHistoryWindow("s1", 256 * 1024);
+
+const promptMessages = [
+  { role: "system", content: `更早对话摘要：\n${history.compressedContext}` },
+  ...history.recentMessages.map((message) => ({
+    role: message.talkerId === "assistant" ? "assistant" : "user",
+    content: message.payload ?? message.content,
+  })),
+];
 ```
-
-## Node And Relation Types
-
-Default node types:
-
-`Person`, `Group`, `Organization`, `Project`, `Task`, `Decision`, `Plan`, `Event`, `Product`, `Technology`, `Data`, `Document`, `Topic`, `Concept`, `Preference`, `Habit`, `Goal`, `Skill`, `Attribute`, `Value`, `Status`, `Time`, `Location`, `Resource`, `Relationship`.
-
-Default relation types:
-
-`is_a`, `part_of`, `belongs_to`, `contains`, `has_member`, `mentioned_in`, `refers_to`, `same_as`, `alias_of`, `related_to`, `associated_with`, `uses`, `creates`, `updates`, `buys`, `owns`, `consumes`, `works_on`, `prefers`, `likes`, `dislikes`, `interested_in`, `favorite`, `plans`, `decides`, `habit_of`, `tends_to`, `avoids`, `skilled_in`, `learning`, `knows`, `friends_with`, `married_to`, `parent_of`, `child_of`, `lives_with`, `depends_on`, `built_with`, `integrates_with`, `deployed_on`, `inputs`, `outputs`, `trained_on`, `predicts`, `happens_at`, `started_at`, `ended_at`, `affects`, `causes`, `leads_to`, `improves`, `reduces`, `assigned_to`, `executed_by`, `blocks`, `completes`, `describes`, `explains`, `references`.
-
-Graph kind constants:
-
-- `KIND_CONVERSATION = "conversation"`
-- `KIND_KNOWLEDGE = "knowledge"`
-
-Conversation graph search only targets conversation graph data. Knowledge graph search only targets knowledge graph data.
-
-## Best Practices
-
-- Always call `await memory.init()` before using the manager.
-- Always call `memory.destroy()` during shutdown to close Grafeo resources.
-- Use stable `userId`, `chatId`, and `sessionId` values. They define memory isolation and retrieval scope.
-- Use `scope: "user"` for personal memory, `scope: "chat"` for one agent/conversation channel, and `scope: "session"` for one short-lived conversation.
-- Use `getFactsForContext` for stable facts and `getHistoryWindow` for compressed session history.
-- Use `updateFacts` for explicit facts that should not depend on LLM extraction.
-- Use `updateEntity` when you already know structured entities and relations.
-- Use `flushChat(..., { wait: true })` before reading `getHistoryWindow` in tests or immediate workflows.
-- Use `addDocument(..., { wait: true })` before immediate knowledge search.
-- Use `waitGraph: true` sparingly because graph extraction depends on LLM calls and can be slower.
-- Keep `embeddingDimension` unchanged for an existing vector store unless you rebuild the database.
-- Keep examples free of real API keys, tokens, personal data, and proprietary customer content.
-
-## Common Pitfalls
-
-- Forgetting `init()` before `updateChat` or search.
-- Expecting `updateChat` to finish compression or graph extraction. It only waits for raw message storage.
-- Expecting `flushChat(..., { wait: true })` to wait for graph data. Use `waitGraph: true`.
-- Expecting `addDocument(..., { wait: true })` to wait for graph data. Use `waitGraph: true`.
-- Setting `waitGraph: true` without allowing enough `graphBuildTimeoutMs` for large documents.
-- Using `scope: "session"` with a `chatId`, or `scope: "chat"` with a `userId`. `scopeId` must match the selected scope.
-- Searching immediately after `addDocument` without `wait: true`; chunks may still be ingesting.
-- Mixing embedding models with different vector dimensions in the same existing vector store.
-- Treating `metadata` as indexed arbitrary JSON. Store important filter dimensions in stable top-level ids where possible.
-- Using `mode: "all"` for every query. It is richer but can be slower than `fast`.
-
-## Security And Privacy
-
-- Never hard-code real `llmApiKey` or `embeddingApiKey`.
-- Treat vector store (LanceDB/SQLite) and Grafeo files as sensitive data stores.
-- `content`, `metadata`, facts, and graph `meta` may contain private information.
-- `destroy()` closes resources; it does not scrub data.
-- Implement your own retention/deletion policy around `deleteSession`, `deleteDocument`, `deleteFact`, and storage directory cleanup.
-- If exposing memory search through an API, enforce user authorization before passing `scope` and `scopeId`.
-
-## Publication Notes
-
-This file is included in the npm package root through the `files` field in `packages/memory/package.json`. When public APIs, config defaults, wait behavior, search semantics, or examples change, update this `llms.txt` in the same change.
+
+### Wait For Conversation Graph Data
+
+```ts
+await memory.flushChat("s1", {
+  wait: true,
+  waitGraph: true,
+});
+```
+
+Use this only when graph data must be ready immediately. It can throw if graph construction exceeds `graphBuildTimeoutMs`.
+
+### Store Stable Facts
+
+```ts
+await memory.updateFacts("用户偏好 TypeScript", "user", "u1", "assistant-main", "s1");
+
+const facts = await memory.getFactsForContext("u1", "assistant-main");
+```
+
+### Search Fast Without Graph
+
+```ts
+const hits = await memory.search({
+  query: "用户偏好什么语言？",
+  scope: "user",
+  scopeId: "u1",
+  mode: "fast",
+});
+```
+
+### Search With Graph
+
+```ts
+const hits = await memory.search({
+  query: "Alice 负责哪个项目，项目用了什么技术？",
+  scope: "chat",
+  scopeId: "work-agent",
+  mode: "all",
+});
+```
+
+### Ingest Markdown And Search Immediately
+
+```ts
+const { docId } = await memory.addDocument({
+  content: "# Rust\n\nRust 的所有权系统保证内存安全。",
+  userId: "u1",
+  chatId: "dev-agent",
+  sourceName: "rust.md",
+  wait: true,
+});
+
+const result = await memory.searchKnowledge({
+  query: "Rust 如何保证内存安全？",
+  scope: "chat",
+  scopeId: "dev-agent",
+  mode: "fast",
+});
+```
+
+### Ingest Markdown And Wait For Knowledge Graph
+
+```ts
+await memory.addDocument({
+  content: markdown,
+  userId: "u1",
+  chatId: "work-agent",
+  sourceName: "project.md",
+  buildGraph: true,
+  waitGraph: true,
+});
+```
+
+### Ask With Conversation Memory And Knowledge Base
+
+```ts
+const answer = await memory.ask({
+  query: "这个项目的技术栈是什么？结合我之前说过的内容回答。",
+  scope: "chat",
+  scopeId: "work-agent",
+  mode: "auto",
+  includeKnowledge: true,
+  maxChars: 300,
+});
+```
+
+## Node And Relation Types
+
+Default node types:
+
+`Person`, `Group`, `Organization`, `Project`, `Task`, `Decision`, `Plan`, `Event`, `Product`, `Technology`, `Data`, `Document`, `Topic`, `Concept`, `Preference`, `Habit`, `Goal`, `Skill`, `Attribute`, `Value`, `Status`, `Time`, `Location`, `Resource`, `Relationship`.
+
+Default relation types:
+
+`is_a`, `part_of`, `belongs_to`, `contains`, `has_member`, `mentioned_in`, `refers_to`, `same_as`, `alias_of`, `related_to`, `associated_with`, `uses`, `creates`, `updates`, `buys`, `owns`, `consumes`, `works_on`, `prefers`, `likes`, `dislikes`, `interested_in`, `favorite`, `plans`, `decides`, `habit_of`, `tends_to`, `avoids`, `skilled_in`, `learning`, `knows`, `friends_with`, `married_to`, `parent_of`, `child_of`, `lives_with`, `depends_on`, `built_with`, `integrates_with`, `deployed_on`, `inputs`, `outputs`, `trained_on`, `predicts`, `happens_at`, `started_at`, `ended_at`, `affects`, `causes`, `leads_to`, `improves`, `reduces`, `assigned_to`, `executed_by`, `blocks`, `completes`, `describes`, `explains`, `references`.
+
+Graph kind constants:
+
+- `KIND_CONVERSATION = "conversation"`
+- `KIND_KNOWLEDGE = "knowledge"`
+
+Conversation graph search only targets conversation graph data. Knowledge graph search only targets knowledge graph data.
+
+## Best Practices
+
+- Always call `await memory.init()` before using the manager.
+- Always call `await memory.destroy()` during shutdown to drain background work and close resources.
+- Use stable `userId`, `chatId`, and `sessionId` values. They define memory isolation and retrieval scope.
+- Use `scope: "user"` for personal memory, `scope: "chat"` for one agent/conversation channel, and `scope: "session"` for one short-lived conversation.
+- Use `getFactsForContext` for stable facts and `getHistoryWindow` for compressed Topics plus recent raw session history.
+- Use `updateFacts` for explicit facts that should not depend on LLM extraction.
+- Use `updateEntity` when you already know structured entities and relations.
+- Use `flushChat(..., { wait: true })` before reading `getHistoryWindow` in tests or immediate workflows.
+- Use `addDocument(..., { wait: true })` before immediate knowledge search.
+- Use `waitGraph: true` sparingly because graph extraction depends on LLM calls and can be slower.
+- Keep `embeddingDimension` unchanged for an existing vector store unless you rebuild the database.
+- Keep examples free of real API keys, tokens, personal data, and proprietary customer content.
+
+## Common Pitfalls
+
+- Forgetting `init()` before `updateChat` or search.
+- Treating `getHistoryWindow` as synchronous or string-only. Since 0.4 it is async and returns a structured complete window.
+- Omitting stable `messageId` when the host can update an existing assistant message; without it, an update becomes another raw message.
+- Expecting `flushChat(..., { wait: true })` to wait for graph data. Use `waitGraph: true`.
+- Expecting `addDocument(..., { wait: true })` to wait for graph data. Use `waitGraph: true`.
+- Setting `waitGraph: true` without allowing enough `graphBuildTimeoutMs` for large documents.
+- Using `scope: "session"` with a `chatId`, or `scope: "chat"` with a `userId`. `scopeId` must match the selected scope.
+- Searching immediately after `addDocument` without `wait: true`; chunks may still be ingesting.
+- Mixing embedding models with different vector dimensions in the same existing vector store.
+- Treating `metadata` as indexed arbitrary JSON. Store important filter dimensions in stable top-level ids where possible.
+- Using `mode: "all"` for every query. It is richer but can be slower than `fast`.
+
+## Security And Privacy
+
+- Never hard-code real `llmApiKey` or `embeddingApiKey`.
+- Treat vector store (LanceDB/SQLite) and Grafeo files as sensitive data stores.
+- `content`, `metadata`, facts, and graph `meta` may contain private information.
+- `destroy()` closes resources; it does not scrub data.
+- Implement your own retention/deletion policy around `deleteSession`, `deleteDocument`, `deleteFact`, and storage directory cleanup.
+- If exposing memory search through an API, enforce user authorization before passing `scope` and `scopeId`.
+
+## Publication Notes
+
+This file is included in the npm package root through the `files` field in `packages/memory/package.json`. When public APIs, config defaults, wait behavior, search semantics, or examples change, update this `llms.txt` in the same change.