recallnest 2.2.0 → 2.3.0

package/docs/plans/2026-04-01-recall-boost.md

@@ -1,100 +0,0 @@
-# RecallNest 召回优化实施清单
-
-> 2026-04-01 发现问题：高频信息（如"轮巡仓库"）召回失败
-> 根因：短查询 vs 长文档 embedding 距离过大 + 无频率加权
-> 与已有 P1-P5 互补，本清单聚焦**检索召回**环节
-
----
-
-## P0: 短查询召回增强（优先级最高）
-
-### P0.1 双层索引 — 原文 + 检索锚点
-
-- [ ] `capture-engine.ts` / `ingest.ts`：存储时同时生成一句话摘要（≤80 chars），存入 `metadata.anchor`
-- [ ] `search.ts`：检索时同时搜原文向量和 anchor 向量，取 max score
-- [ ] 效果：短 query "轮巡仓库" 匹配短 anchor "每日轮巡4个win4r仓库"，距离大幅缩短
-- **预估:** ~120 LOC
-- **风险:** 低——不改原文存储，纯增量字段
-- **依赖:** 无
-
-### P0.2 高频 boost — 重复提及自动提权
-
-- [ ] 新增 `frequency-tracker.ts`：记录 query→memory 的命中次数
-- [ ] 评分公式：`final_score = base_score × (1 + log2(hit_count) × boost_factor)`
-- [ ] hit_count ≥ 3 → peripheral 自动升为 core tier
-- [ ] 持久化：写入 `data/frequency-stats.json`，跨 session 累积
-- **预估:** ~100 LOC
-- **风险:** 中——需要平衡 boost 幅度，避免旧高频记忆永远压过新记忆
-- **依赖:** 无
-
-### P0.3 短查询自动扩展
-
-- [ ] `search.ts`：检测 query ≤ 6 字符时，调用 LLM 扩展为 3-5 个同义关键词
-- [ ] 扩展结果缓存到 `data/query-expansion-cache.json`，相同 query 不重复调用
-- [ ] 备选：不用 LLM，维护一个 `data/alias-map.json` 手动/自动映射（"轮巡" → "轮巡 仓库 patrol repo 每日检查"）
-- **预估:** ~60 LOC（alias-map 方案）/ ~100 LOC（LLM 方案）
-- **风险:** alias-map 低；LLM 方案增加延迟 ~200ms
-- **依赖:** 无
-
----
-
-## 高频 vs 查重 平衡机制（P0.2 的关键设计）
-
-### 问题
-
-高频 boost 和去重是一对矛盾：
-- **boost 说**：提到 10 次的东西一定重要，提权！
-- **dedup 说**：同一条信息存了 10 遍，应该合并！
-
-### 设计方案：分层计数，不是分层存储
-
-```
-存储层：dedup 照做，相同内容只保留 1 条（现有 consolidation 逻辑不变）
-计数层：新增 frequency-tracker，记录的是 **query 命中次数**，不是存储条数
-```
-
-**关键区分：**
-
-| 维度 | 存储去重 | 召回加权 |
-|------|---------|---------|
-| 触发时机 | 写入时（store/ingest） | 检索时（search） |
-| 操作对象 | 重复的 memory 条目 | query→memory 的命中频率 |
-| 目标 | 数据库不膨胀 | 高频信息排前面 |
-| 互相影响 | 不影响——去重后只剩 1 条，但那 1 条的 hit_count 持续累积 |
-
-### 边界情况处理
-
-1. **旧高频 vs 新相关**
-   - 时间衰减：`effective_hits = hit_count × decay(days_since_last_hit)`
-   - 30 天没被命中 → hit_count 等效减半，不会永远霸榜
-
-2. **高频但已过时**
-   - 用户说"轮巡仓库从4个变成3个了" → store_memory 更新
-   - 旧条目被 consolidation 标记为 superseded → hit_count 不继承
-   - 新条目从 0 开始计，但因为用户接下来会频繁触发，很快追上
-
-3. **频率统计粒度**
-   - 按 memory_id 计数，不按 query 文本
-   - "轮巡" "轮巡仓库" "patrol repo" 命中同一条 memory → 同一个 counter +1
-   - 避免同义词分散计数
-
----
-
-## 与现有 P1-P5 的关系
-
-| 现有项 | 关系 |
-|--------|------|
-| P1 摘要保真 | P0.1 的 anchor 生成需要 P1 的保真约束，但可并行开发 |
-| P2 聚类摘要 | 互补——P2 解决注入时省 token，P0 解决检索时找得到 |
-| P3 增量 ingest | P0.1 的 anchor 字段需要对已有数据 backfill，P3 的增量逻辑可复用 |
-| P4 data-checkup | 可加一项检查：anchor 字段覆盖率（多少 memory 已有 anchor） |
-| P5 大文件门控 | 无直接关系 |
-
----
-
-## 验证标准
-
-- [ ] `search_memory("轮巡")` → 返回轮巡仓库相关记忆，score ≥ 70%
-- [ ] `search_memory("轮巡仓库")` → 返回结果，score ≥ 80%
-- [ ] 高频记忆（命中 ≥5 次）自动出现在 `resume_context` 的 core 区
-- [ ] 去重后数据库条目数不增长（frequency 只在 tracker 层累积）

package/docs/plans/mcp-tiering-plan.md

@@ -1,126 +0,0 @@
-# MCP Tool Surface Tiering Plan
-
-## Goal
-Reduce MCP tool surface from 24 to ~10 for daily agents, moving governance tools to CLI-only.
-
-## Current Tools (24)
-
-### Tier 1: Core (5) — Always Exposed
-- `resume_context` — startup continuity
-- `search_memory` — proactive recall
-- `store_memory` — durable write
-- `checkpoint_session` — work handoff
-- `latest_checkpoint` — checkpoint read
-
-### Tier 2: Advanced (10) — Exposed by Default
-- `store_case` — structured case write
-- `store_workflow_pattern` — structured pattern write
-- `promote_memory` — evidence → durable
-- `explain_memory` — retrieval explain
-- `distill_memory` — compact briefing
-- `brief_memory` — create brief asset
-- `pin_memory` — pin for reuse
-- `list_assets` — list assets
-- `list_pins` — list pins
-- `memory_stats` — db stats
-
-### Tier 3: Governance (9) — CLI Only, Not MCP
-- `workflow_observe`
-- `workflow_health`
-- `workflow_evidence`
-- `list_conflicts`
-- `resolve_conflict`
-- `audit_conflicts`
-- `escalate_conflicts`
-- `list_dirty_briefs`
-- `clean_dirty_briefs`
-
-## Implementation
-
-### Step 1: Add Tier Config
-Add to `mcp-server.ts`:
-
-```typescript
-// Environment: RECALLNEST_MCP_TIER=core|advanced|full (default: advanced)
-const MCP_TIER = process.env.RECALLNEST_MCP_TIER || "advanced";
-
-const TOOL_TIERS: Record<string, "core" | "advanced" | "governance"> = {
-  // Core (always)
-  resume_context: "core",
-  search_memory: "core",
-  store_memory: "core",
-  checkpoint_session: "core",
-  latest_checkpoint: "core",
-
-  // Advanced
-  store_case: "advanced",
-  store_workflow_pattern: "advanced",
-  promote_memory: "advanced",
-  explain_memory: "advanced",
-  distill_memory: "advanced",
-  brief_memory: "advanced",
-  pin_memory: "advanced",
-  list_assets: "advanced",
-  list_pins: "advanced",
-  memory_stats: "advanced",
-  export_memory: "advanced",
-
-  // Governance (CLI-only)
-  workflow_observe: "governance",
-  workflow_health: "governance",
-  workflow_evidence: "governance",
-  list_conflicts: "governance",
-  resolve_conflict: "governance",
-  audit_conflicts: "governance",
-  escalate_conflicts: "governance",
-  list_dirty_briefs: "governance",
-  clean_dirty_briefs: "governance",
-};
-
-function shouldRegisterTool(toolName: string): boolean {
-  const tier = TOOL_TIERS[toolName];
-  if (!tier) return true; // unknown tools always register
-  if (MCP_TIER === "full") return true;
-  if (MCP_TIER === "advanced") return tier !== "governance";
-  if (MCP_TIER === "core") return tier === "core";
-  return true;
-}
-```
-
-### Step 2: Wrap Tool Registrations
-Replace `server.tool()` calls with conditional registration:
-
-```typescript
-function registerTool(...) {
-  if (!shouldRegisterTool(name)) {
-    console.log(`[MCP] Skipping ${name} (tier: ${TOOL_TIERS[name]})`);
-    return;
-  }
-  server.tool(name, description, schema, handler);
-}
-```
-
-### Step 3: Update CLI for Governance Tools
-Ensure governance tools are fully accessible via CLI:
-- `recallnest workflow-observe ...` → already exists
-- `recallnest conflicts list ...` → already exists
-- Add `recallnest dirty-briefs list/clean` if missing
-
-## Testing
-
-1. Default run: `bun src/mcp-server.ts` → should register ~15 tools (core + advanced)
-2. Tier=core: `RECALLNEST_MCP_TIER=core bun src/mcp-server.ts` → 5 tools
-3. Tier=full: `RECALLNEST_MCP_TIER=full bun src/mcp-server.ts` → 24 tools (backward compatible)
-4. Smoke test: MCP import works for all tiers
-
-## Files to Modify
-
-- `src/mcp-server.ts` — add tier config + conditional registration
-
-## Verification
-
-```bash
-cd ~/recallnest
-RECALLNEST_MCP_TIER=core bun src/mcp-server.ts &
-# Check logs for registered tools
-```

package/docs/research/recall-quality-audit-2026-03-22.md

@@ -1,153 +0,0 @@
-# RecallNest Recall Quality Audit — 2026-03-22
-
-## Executive Summary
-
-This audit assessed three dimensions of RecallNest's memory quality: **ingest noise ratio**, **case/pattern recall relevance**, and **checkpoint content quality**. The system has strong retrieval-side relevance mechanisms (multi-stage scoring, reranking, noise filtering) and robust checkpoint sanitization, but suffers from weak ingest-side validation for evidence-layer data and a critical gap in case recall — 5,853 stored cases are effectively invisible due to over-aggressive filtering. The top 5 issues are ranked below with concrete fix suggestions.
-
----
-
-## Top 5 Highest-Impact Quality Issues
-
-### 1. Case Recall is Effectively Broken (CRITICAL)
-
-**Finding**: 0 out of 5 probes returned any case results, despite 5,853 cases in the store. Vector search finds relevant cases (similarity 0.70–0.74), but the selection pipeline filters them all out.
-
-**Root cause**: `isTaskCandidateUseful` in `src/ranking.ts:168` rejects non-structured cases (`if (category === "cases" && !structured) return false`). Most stored cases are raw transcript chunks in `cc:` scopes, not structured `Case: problem/solution` format. Additionally, `selectTaskResults` in `src/context-composer-task-selection.ts:298-300` filters non-durable candidates when durable ones exist.
-
-**Impact**: The entire case recall subsystem provides zero value. Users who stored debugging solutions, fix recipes, or incident resolutions never see them recalled.
-
-**Fix suggestion**:
-- **File**: `src/ranking.ts` — `isTaskCandidateUseful` function (~line 168)
-- **Approach**: Relax the structured-only filter for cases with high vector similarity (>0.72). Add a fallback path: if zero structured cases pass, allow top-N unstructured cases with high similarity scores through. This preserves precision for normal recall but prevents the "zero results" cliff.
-- **Classification**: Quick win (localized change, testable in isolation)
-
-### 2. Noise Filtering Only at Retrieval, Not Ingest (MODERATE-HIGH)
-
-**Finding**: The noise filter in `src/noise-filter.ts` only runs during `hybridRetrieval()` and `vectorOnlyRetrieval()` — NOT during ingest. Noisy data (bash progress output, screenshot base64, tool-use tags, system messages, permission instructions) is stored permanently in the evidence layer. The noise occupies embedding space, dilutes vector search quality, and wastes MMR diversity budget on filtering out junk.
-
-**Root cause**: The ingest pipeline in `src/ingest.ts` only validates on text length (> 8 chars). No content quality check, no protocol filtering, no diagnostic artifact detection. 59.2% of all 39,580 entries are category "fact" — many are low-signal transcript snippets. 84.5% are from ephemeral `cc:*` session scopes.
-
-**Impact**: The store is bloated with low-value entries. Retrieval-side filtering is a band-aid — the noisy entries still consume storage, slow vector search, and can occasionally leak through retrieval filters into recall results.
-
-**Fix suggestion**:
-- **File**: `src/ingest.ts` — add pre-storage filtering after chunking
-- **File**: `src/noise-filter.ts` — extract shared noise patterns into reusable functions
-- **Approach**: Apply the existing noise filter patterns at ingest time as a hard gate. Add additional ingest-specific patterns: reject chunks under 30 chars matching boilerplate, base64-heavy content, bash progress output, and tool-use protocol tags. Score chunk information density and skip purely routine/acknowledgment content.
-- **Classification**: Deeper refactor (needs heuristic set + testing against real transcripts to avoid false negatives)
-
-### 3. Checkpoint Scope Normalization Gap (MODERATE)
-
-**Finding**: Three different scope strings exist for the same project: `project:recallnest` (114 checkpoints), `recallnest` (24), and `project:RecallNest` (4). A `resume_context` call with one variant misses checkpoints stored under another.
-
-**Root cause**: No scope normalization at checkpoint save time. The 24 `recallnest` checkpoints are from March 16–17 before conventions solidified. The 4 `project:RecallNest` checkpoints have a case mismatch.
-
-**Impact**: `getLatest()` by scope can return a stale checkpoint instead of the truly latest one if the most recent was stored under a non-canonical scope string. This directly affects continuity recovery quality.
-
-**Fix suggestion**:
-- **File**: `src/session-engine.ts` — checkpoint save path
-- **Approach**: Add a `normalizeScope()` function that lowercases and ensures `project:` prefix. Apply it both at save time and at query time. For historical data, run a one-time migration script to normalize existing checkpoint scopes.
-- **Classification**: Quick win (small utility function + migration script)
-
-### 4. Empty/Fallback Checkpoints Displace Useful Recovery Data (MODERATE)
-
-**Finding**: ~44% of checkpoints (69 of 156) have sanitization artifacts. Of these, ~10% (16) have only fallback summary text ("Checkpoint captured current task state without repo-state details.") plus empty structured fields. These provide zero recovery value but can be returned by `getLatest()`, displacing useful checkpoints.
-
-**Additional concern**: No checkpoint expiry mechanism exists. All 156 checkpoints persist indefinitely (156 files in ~6 days = ~9,500/year projected). No age-based pruning, no size limits, no archive mechanism. File-system based storage means O(n) scan on every retrieval.
-
-**Root cause**: The checkpoint API accepts and stores checkpoints even when all meaningful fields are empty or sanitized away.
-
-**Impact**: If a fallback-only checkpoint is the most recent for a scope, `resume_context` returns useless continuity data. Combined with no expiry, checkpoint directory will grow unbounded.
-
-**Fix suggestion**:
-- **File**: `src/session-engine.ts` — `checkpoint_session` handler
-- **Approach**: (a) Add a quality gate before saving: if summary would be fallback AND decisions/openLoops/nextActions are all empty, either reject with a warning or mark as `quality: "minimal"` so `getLatest()` can prefer higher-quality alternatives. (b) Add checkpoint age check in `resume_context` — warn if > 24h old. (c) Implement expiry: archive checkpoints older than 30 days.
-- **Classification**: Quick win for quality gate; medium effort for expiry
-
-### 5. Importance Defaults Too Permissive + Weak Tier Inference (MODERATE)
-
-**Finding**: Most ingested transcript memories get default importance 0.7 — no distinction between high-value technical insights and casual chat. Tier assignment is purely category-based (`profile`/`patterns` -> "working", everything else -> "peripheral") with no semantic quality analysis.
-
-**Root cause**: No importance scoring at ingest time. The decay engine (`src/decay-engine.ts`) uses a 3-tier system (core/working/peripheral) with different Weibull decay curves, but tier promotion requires 3+ accesses or importance >= 0.5 (peripheral -> working) or 10+ accesses or importance >= 0.8 (working -> core). Since everything starts at 0.7 importance, most evidence-layer memories sit at "peripheral" tier forever.
-
-**Impact**: Low-value transcript entries compete fairly with high-value durable memories in retrieval scoring. The importance weight formula `0.7 + 0.3 * importance` gives a 0.7-importance entry 91% the score of a 1.0-importance entry — barely distinguishable.
-
-**Fix suggestion**:
-- **File**: `src/ingest.ts` — post-parse stage; `src/decay-engine.ts` — tier logic
-- **Approach**: Use LLM (already integrated via `smartExtractBatch` in ingest.ts:328) to assign importance during ingestion. For non-LLM path, use heuristics: code-heavy content -> importance 0.8, error resolution -> 0.85, boilerplate -> 0.3. Widen the importance multiplier range (e.g., `0.5 + 0.5 * importance`) to make it more discriminating.
-- **Classification**: Deeper refactor (needs scoring model + tuning)
-
----
-
-## Additional Findings
-
-### What's Working Well
-
-| Area | Rating | Evidence |
-|---|---|---|
-| Tool output filtering | Excellent | 98.7% clean — `parseCCTranscript` type filter is effective |
-| Hybrid retrieval pipeline | Excellent | 8-stage scoring with RRF fusion, Weibull decay, MMR diversity |
-| Checkpoint schema design | Good | Strict Zod validation, appropriate field limits, well-structured |
-| Repo-state sanitizer (post-deploy) | Excellent | Zero leaks since March 18 deployment |
-| Checkpoint summary quality | Good | 75% of checkpoints have rich, actionable content |
-| Durable memory dedup | Good | Two-stage vector + LLM semantic dedup with canonical key matching |
-| Cross-encoder reranking | Good | Jina/Pinecone/Voyage reranking available for semantic re-ranking |
-| Retrieval profiles | Good | 4 task-tuned profiles (default, writing, debug, fact-check) with appropriate tuning |
-| Cue coverage diversity | Good | Greedy set-cover algorithm prevents duplicate pattern recall |
-| Entity capture | Good | Specific, useful entities (file paths, tool names, PR refs). 8-entity limit sufficient |
-
-### Pattern vs Case Recall Asymmetry
-
-Patterns recall well because they tend to be durable and structured (967 eligible entries, 2.5% of store). Cases fail because the same filtering criteria are applied but cases in the store are overwhelmingly unstructured transcript chunks. This is a design asymmetry — the system was built for curated patterns but cases accumulated organically.
-
-### Dedup Gap When LLM Unavailable
-
-Evidence-layer transcript ingest skips LLM semantic dedup if no embedder is available (`src/ingest.ts:817-821`) — it queues for later processing but still ingests the raw chunk with only vector dedup. When LLM is unavailable, borderline-similar chunks (similarity 0.68-0.80) are stored as separate entries, leading to near-duplicate pollution.
-
-### Checkpoint Coverage Details
-
-- 156 checkpoints across 94 sessions (March 16-22)
-- March 21 has only 3 checkpoints vs 28-46 on other days — potential gap
-- `resume_context` correctly uses checkpoint as first line of recovery with entity supplementation
-- 220-char headline truncation loses detail from rich summaries, but full checkpoint is available in response body
-- **No checkpoint-to-durable promotion pipeline**: High-quality checkpoint decisions never automatically become durable memories
-- **Limited coverage in resume_context**: Only 3-5 lines from checkpoint extracted (<5% of max checkpoint content)
-
-### Distillation Opportunity
-
-23,441 "fact" entries could be periodically distilled into fewer, higher-quality durable memories. This would improve the signal-to-noise ratio of the entire store and make vector search more effective. The existing `distill_memory` tool could be leveraged for this.
-
----
-
-## Quick Wins vs Deeper Refactors
-
-### Quick Wins (1-2 days each)
-
-1. **Relax case recall filter** — Allow high-similarity unstructured cases when zero structured cases match (`src/ranking.ts`)
-2. **Scope normalization** — `normalizeScope()` at save + query time (`src/session-engine.ts`)
-3. **Empty checkpoint gate** — Reject or flag checkpoints with all-fallback content (`src/session-engine.ts`)
-4. **Historical data cleanup** — Re-sanitize 9 pre-guard checkpoint files (one-time script)
-5. **Checkpoint staleness warning** — Warn in `resume_context` when checkpoint is >24h old (`src/context-composer.ts`)
-
-### Deeper Refactors (1-2 weeks each)
-
-1. **Ingest-time noise filtering** — Apply noise patterns + content quality scoring at ingest (`src/ingest.ts` + `src/noise-filter.ts`)
-2. **Importance scoring at ingest** — LLM or heuristic-based importance assignment during ingestion (`src/ingest.ts`)
-3. **Case promotion pipeline** — Auto-promote high-quality `cc:` cases to `memory:` scope based on structure detection and retrieval frequency
-4. **Fact distillation pipeline** — Periodic batch job to consolidate 23,441 low-value fact entries into curated durable memories
-5. **Checkpoint expiry and indexing** — Archive old checkpoints + migrate from filesystem scan to indexed storage
-6. **Checkpoint-to-durable promotion** — Auto-promote high-quality decisions/nextActions to `events` or `cases` memory category
-
----
-
-## Methodology
-
-- **Architecture review**: Full codebase analysis of 89 source files (~19,500 LOC TypeScript) across all subsystems — ingestion, retrieval, checkpoint, conflict, workflow observation.
-- **Ingest noise analysis**: Traced the parser pipeline through `src/ingest.ts`, analyzed noise filter patterns in `src/noise-filter.ts`, reviewed dedup thresholds and LLM fallback behavior. Parsed 10 real CC transcripts, classified 792 text blocks for noise type.
-- **Case/pattern recall analysis**: Ran 5 `composeResumeContext` probes with different task prompts against `project:recallnest` scope. Traced the selection pipeline through `src/ranking.ts` and `src/context-composer-task-selection.ts`. Compared vector search results vs final pipeline output.
-- **Checkpoint audit**: Schema analysis of `src/session-schema.ts` + `src/session-engine.ts`, then quality review of all 156 checkpoint files from March 16-22. Field-level emptiness analysis, scrubbing effectiveness verification, scope distribution analysis.
-- **Retrieval pipeline audit**: Full trace through `src/retriever.ts` 8-stage scoring pipeline, `src/decay-engine.ts` Weibull decay, `src/embedder.ts` multi-provider embedding, and `src/retrieval-profiles.ts` task-tuned profiles.
-
----
-
-*Report generated 2026-03-22 by recallnest-quality-8ad2b7 team (coordinator + 3 research agents)*
-*Supersedes earlier draft by recallnest-quality-69722c team*

package/docs/self-evolution.md

@@ -1,198 +0,0 @@
-# Self-Evolution: How RecallNest Memories and Workflows Evolve
-
-> 自我进化原理：RecallNest 不只让记忆自己整合、衰减、上浮，也开始观察 workflow 什么时候做对、什么时候做错。
-
-## Overview
-
-Traditional memory systems are append-only: store everything, search later, hope for the best. Over time, they bloat with duplicates, outdated entries, and noise.
-
-RecallNest takes a different approach: **memories evolve**. They consolidate, decay, and surface based on usage — just like biological memory.
-
-```
-Store → Classify → Decay → Access → Consolidate → Promote/Demote
-  │        │         │        │          │              │
-  ▼        ▼         ▼        ▼          ▼              ▼
- New    6 types   Weibull   "Use it    Merge         Core ↔
-memory  auto-     curve     or lose    similar       Working ↔
-        assigned  over      it"        entries       Peripheral
-                  time
-```
-
----
-
-## Workflow Observation Loop
-
-RecallNest now separates self-evolution signals from regular memory.
-
-Instead of turning every workflow failure into another `events` row, RecallNest records workflow observations in a dedicated append-only store:
-
-```
-Observe → Inspect → Health → Evidence → Fix / Test
-   │         │         │         │          │
-   ▼         ▼         ▼         ▼          ▼
- record   aggregate  degrade   package   repair the
- success/ 7d / 30d   or stay   top       workflow,
- failure   health    healthy   signals   then observe again
-```
-
-- **`workflow_observe`** records whether `resume_context`, `checkpoint_session`, or another workflow primitive succeeded, failed, was corrected, or was missed
-- **`workflow_health`** aggregates recent observations into a health report or degraded-workflow dashboard
-- **`workflow_evidence`** packages recent issue observations and suggestions so tests, rules, and prompts can be tightened
-- these records live outside the 6 memory categories and are not composed into `resume_context`
-- managed MCP / HTTP continuity calls now append `resume_context` / `checkpoint_session` observations automatically, and checkpoint repo-state sanitization is recorded as `corrected`
-
-That boundary is deliberate: self-evolution needs operational telemetry, not more noisy durable memory.
-
----
-
-## The Three-Tier Lifecycle
-
-Every memory lives in one of three tiers:
-
-| Tier | Description | Typical Size |
-|------|-------------|--------------|
-| **Core** | Essential knowledge, accessed frequently | ~5% of total |
-| **Working** | Actively useful, moderate access | ~25% of total |
-| **Peripheral** | Rarely accessed, candidates for archival | ~70% of total |
-
-> 三层生命周期：核心层（常用的 5%）→ 工作层（中等频率 25%）→ 边缘层（低频 70%）
-
-### Promotion and Demotion
-
-Memories move between tiers based on access patterns:
-
-- **Promotion** (peripheral → working → core): Memory accessed ≥ N times within a window
-- **Demotion** (core → working → peripheral): Memory not accessed for extended period
-- **Archival**: Peripheral memories with zero access beyond decay threshold get archived (not deleted)
-
----
-
-## Weibull Decay
-
-RecallNest uses the [Weibull distribution](https://en.wikipedia.org/wiki/Weibull_distribution) for time-based memory decay, inspired by human forgetting curves.
-
-> Weibull 衰减模型比简单的线性衰减更符合人类遗忘曲线：开始衰减慢，后面加速。
-
-### Why Weibull, not Exponential?
-
-| Model | Behavior | Problem |
-|-------|----------|---------|
-| Linear | Constant decay rate | Too aggressive early on |
-| Exponential | Fast initial decay | Important but old memories vanish too quickly |
-| **Weibull** | Slow start, accelerating decay | Matches human memory: recent stays fresh, old fades naturally |
-
-### Formula
-
-```
-decay(t) = exp(-(t / λ)^k)
-
-where:
-  t = days since last access
-  λ = scale parameter (half-life, default: 120 days)
-  k = shape parameter (default: 1.5, gives the "slow then fast" curve)
-```
-
-### Impact on Search
-
-Decay factor multiplies the retrieval score:
-
-```
-final_score = relevance_score × decay(days_since_access)
-```
-
-A highly relevant but stale memory can still surface if the semantic match is strong enough.
-
----
-
-## Memory Consolidation
-
-Over time, you accumulate many memories about the same topic. Consolidation merges them.
-
-> 记忆整合：找到主题相近的记忆簇，按策略合并或去重，保持记忆库精简。
-
-### Merge vs. Append Strategies
-
-| Category | Strategy | Why |
-|----------|----------|-----|
-| `profile` | **Merge** | User identity evolves; latest version wins |
-| `preferences` | **Merge** | Preferences update within the same slot; concrete item preferences stay separate |
-| `entities` | **Merge** | Entity info accumulates; consolidate |
-| `patterns` | **Merge** | Patterns refine over time |
-| `events` | **Append** | Events are facts; dedup but keep distinct entries |
-| `cases` | **Append** | Case studies are unique; dedup only exact matches |
-
-### How It Works
-
-1. For each category, compute pairwise cosine similarity
-2. Cluster entries with similarity > threshold (default: 0.85)
-3. For **merge** categories: LLM combines the cluster into one entry, archives originals
-4. For **append** categories: remove exact duplicates, keep distinct entries
-5. Update access counts (merged entry inherits the sum)
-
-`preferences` needs one extra guard: do not collapse same-brand, different-item preferences into one topic-level memory. A summary can exist, but it should not replace the atomic preference facts. The same idea now also applies to slot-aware reply-style and tool-choice preferences.
-
-### Safety
-
-- Default mode is **dry-run**: preview changes before applying
-- Original entries are **archived**, never deleted
-- Consolidation is triggered manually or via scheduled task, never automatic
-
----
-
-## Gap Detection
-
-RecallNest tracks what you search for and notices when searches consistently return poor results.
-
-> 缺口检测：分析搜索日志，找出你经常问但记忆库覆盖不足的主题。
-
-### How It Works
-
-1. Log every search query and its top result score
-2. Periodically analyze: which topics have avg score < threshold?
-3. Cluster low-score queries by topic
-4. Generate a gap report with suggestions
-
-### Example Gap Report
-
-```
-Topic: "deployment pipeline"
-  Searches: 8 in last 30 days
-  Avg top score: 0.12
-  Suggestion: Consider documenting your deployment workflow
-
-Topic: "API rate limiting"
-  Searches: 5 in last 30 days
-  Avg top score: 0.18
-  Suggestion: Store your rate limiting strategy and past incidents
-```
-
----
-
-## Access-Based Surfacing ("Use It or Lose It")
-
-> 用进废退：经常被搜到的记忆权重上升，长期无人问津的权重下降。
-
-The access tracker records every time a memory appears in search results. This creates a feedback loop:
-
-```
-Accessed often → higher tier → higher base score → more likely to surface
-Never accessed → lower tier → lower base score → gradually fades
-```
-
-This is the "use it or lose it" principle applied to AI memory.
-
----
-
-## Putting It All Together
-
-A typical memory lifecycle:
-
-1. **Day 0**: New memory stored → classified as `events` → tier: `peripheral`
-2. **Day 1-30**: Searched 5 times → promoted to `working`
-3. **Day 30-90**: Searched 15 more times → promoted to `core`
-4. **Day 120**: Consolidated with 3 similar memories → single refined entry
-5. **Day 180+**: Access drops to zero → demoted to `working`
-6. **Day 365+**: Still no access → demoted to `peripheral`, Weibull decay significant
-7. **Archival**: If importance < threshold after extended inactivity → archived
-
-The result: your memory index stays lean, relevant, and self-maintaining.