memento-mcp-server 1.6.0 → 1.7.1

package/docs/en/Search-Ranking-Memory-Decay-Formulas.md

@@ -1,177 +0,0 @@
-# Formulas for Search Ranking and Memory Decay
-
-## 1) Final Ranking Score
-
-$$S = \alpha \cdot \text{relevance} + \beta \cdot \text{recency} + \gamma \cdot \text{importance} + \delta \cdot \text{usage} - \varepsilon \cdot \text{duplication\_penalty}$$
-
-**Default coefficients (M1 recommended)**: $\alpha=0.50$, $\beta=0.20$, $\gamma=0.20$, $\delta=0.10$, $\varepsilon=0.15$
-
-**Meaning**: For candidate m, boost relevance, recency, importance, and usage, while reducing duplication penalty.
-
-**Practical tip**: Start with the above default values → adjust $\alpha$, $\beta$ through A/B testing, and increase $\varepsilon$ if result diversity is insufficient.
-
-## 2) Relevance Composition
-
-$$\text{relevance} = w_e \cdot \text{sim}_e + w_k \cdot \text{bm25\_norm} + w_t \cdot \text{tag\_match} + w_p \cdot \text{title\_hit}$$
-
-**Default weights (M1)**: $w_e=0.60$, $w_k=0.30$, $w_t=0.05$, $w_p=0.05$
-
-### (1) Embedding Similarity (Cosine)
-
-$$\text{sim}_e = \max(0, \cos(\mathbf{E}(q), \mathbf{E}(m)))$$
-
-$\mathbf{E}(x)$ is the embedding function. Clamp cosine values to 0 if negative to prevent negative scores.
-
-### (2) Keyword (BM25 Normalization)
-
-$$\text{bm25\_norm} = \frac{\text{BM25}}{\text{BM25} + k_{\text{norm}}}$$
-
-Recommended $k_{\text{norm}}=2.0$. Maps any query to [0,1] for easy combination with other features.
-
-### (3) Tag Matching (Jaccard Similarity)
-
-$$\text{tag\_match} = \frac{|T_q \cap T_m|}{|T_q \cup T_m|}$$
-
-Overlap between query tags ($T_q$) and document tags ($T_m$). Actively using tags significantly improves search quality.
-
-### (4) Title Match (Exact·Prefix·N-gram Addition)
-
-$$\text{title\_hit} = I_{\text{exact}} \cdot 1.0 + I_{\text{prefix}} \cdot 0.5 + I_{\text{ngram}} \cdot 0.2$$
-
-$I_*$ is 1 when condition is met, 0 otherwise. Fine-tuning section.
-
-### Candidate Generation Tips (2-stage search recommended)
-
-Stage 1 (ANN): Top $N_e$ embeddings, Stage 2 (BM25): Top $N_k$ keywords → Create union as candidate set and re-rank with above relevance.
-
-## 3) Recency = Half-life Based Exponential Decay
-
-$$\text{recency} = \exp\left(-\frac{\ln(2) \cdot \text{age\_days}}{\text{half\_life}(\text{type})}\right)$$
-
-- $\text{half\_life}(\text{working})=2$
-- $\text{half\_life}(\text{episodic})=30$ 
-- $\text{half\_life}(\text{semantic})=180$ (unit: days)
-
-**Meaning**: Different half-lives for each memory type so working memory fades quickly while semantic memory lasts long.
-
-**Tuning**: Shorter half_life for fast domains (e.g., issue response notes), longer for knowledge bases.
-
-## 4) Importance = User Weight + System Weight
-
-$$\text{importance} = \text{clamp}(\text{imp\_user} + 0.20 \cdot \text{pinned} + \text{type\_boost}, 0, 1)$$
-
-- $\text{imp\_user} \in [0,1]$: User-provided importance when storing (default 0.5)
-- $\text{pinned} \in \{0,1\}$: 1 when pinned (protected from forgetting·decay)
-- $\text{type\_boost}$: semantic=+0.10, episodic=0.00, working=−0.05 (example)
-- $\text{clamp}(a,0,1)$: Clamp to 0~1 range
-
-**Tip**: Instead of using imp_user directly, add project-specific policies (e.g., project:accounting +0.05).
-
-## 5) Usage = Normalization of Log-scale Aggregation
-
-$$\text{usage\_raw} = \log(1 + \text{view\_cnt}) + 2 \cdot \log(1 + \text{cite\_cnt}) + 0.5 \cdot \log(1 + \text{edit\_cnt})$$
-
-$$\text{usage} = \frac{\text{usage\_raw} - \text{min\_batch}}{\text{max\_batch} - \text{min\_batch} + \varepsilon}$$
-
-$\text{cite\_cnt}$ (reuse/citation) has highest weight. $\varepsilon$ is small value like $1e-6$.
-
-**Reason for batch normalization**: Reduces absolute value deviation per document to ensure stable scale per query·session.
-
-## 6) Duplication Penalty = MMR-style Diversity Control
-
-$$\text{duplication\_penalty} = \max_{j \in R} \text{sim}_e(m, j)$$
-
-$R$: Set of items already selected in current result list
-
-**Interpretation**: Give penalty as the degree of similarity between new candidate m and most similar existing selected result.
-
-If result diversity is insufficient, increase $\varepsilon$ (coefficient in final formula) or use MMR post-processing below.
-
-### MMR Post-processing (Optional)
-
-$$\text{MMR}(m) = \lambda \cdot S(m) - (1 - \lambda) \cdot \max_{j \in R} \text{sim}_e(m, j)$$
-
-Recommended $\lambda=0.8$. When extracting final Top-K, add diversity term on top of S score.
-
-## 7) Forgetting (garbage collection)·Reminder (spaced review)
-
-Forgetting includes not just "deletion" but also "re-exposure (review)". Using score-based deletion risk and review schedule together is most stable.
-
-### 7.1 Deletion Risk (ForgetScore)
-
-$$\text{ForgetScore} = u_1 \cdot (1 - \text{recency}) + u_2 \cdot (1 - \text{usage}) + u_3 \cdot \text{dup\_ratio} - u_4 \cdot \text{importance} - u_5 \cdot \text{pinned}$$
-
-**Default coefficients (M1)**: $u_1=0.35$, $u_2=0.25$, $u_3=0.20$, $u_4=0.15$, $u_5=0.30$
-
-$\text{dup\_ratio}$: Proportion of similar items in cluster (if none, normalize duplication_penalty to 0~1 as substitute)
-
-**Interpretation**: Easier to delete when old ($1-\text{recency}↑$), unused ($1-\text{usage}↑$), and many duplicates ($\text{dup\_ratio}↑$). Protect important or pinned items.
-
-### Deletion Policy (Example)
-
-- **Soft deletion candidates**: $\text{ForgetScore} \geq \theta_{\text{soft}}$ AND $\text{age\_days} \geq \text{TTL}_{\text{soft}}(\text{type})$
-- **Hard deletion**: $\text{ForgetScore} \geq \theta_{\text{hard}}$ AND $\text{age\_days} \geq \text{TTL}_{\text{hard}}(\text{type})$
-
-**Recommended values (M1)**:
-- $\theta_{\text{soft}}=0.6$, $\theta_{\text{hard}}=0.8$
-- $\text{TTL}_{\text{soft}}(\text{working})=2$, $\text{TTL}_{\text{soft}}(\text{episodic})=30$, $\text{TTL}_{\text{soft}}(\text{semantic})=\infty$
-- $\text{TTL}_{\text{hard}}(\text{working})=7$, $\text{TTL}_{\text{hard}}(\text{episodic})=180$, $\text{TTL}_{\text{hard}}(\text{semantic})=\infty$
-
-### 7.2 Spaced Review Schedule
-
-$$p_{\text{recall}}(t) = \exp\left(-\frac{t}{\tau}\right)$$
-
-$$\tau_{\text{next}} = \tau_{\text{prev}} \cdot (1 + a_1 \cdot \text{importance} + a_2 \cdot \text{usage} + a_3 \cdot \text{feedback\_helpful} - a_4 \cdot \text{feedback\_bad})$$
-
-**Recommended coefficients**: $a_1=0.6$, $a_2=0.4$, $a_3=0.5$, $a_4=0.7$
-
-**Meaning**: Memories that are important, frequently used, and marked as helpful have **increased memory half-life ($\tau$)** so review cycle becomes longer.
-
-**Scheduling condition (example)**: Set next review time when $p_{\text{recall}}(t) \leq 0.7$ to issue reminder cards.
-
-### Simple Preset (Light version instead of complex SM-2)
-
-After first storage: 1 day, 6 days, then $\text{interval\_next} = \lceil \text{interval\_prev} \cdot (1 + 0.5 \cdot \text{importance} + 0.3 \cdot \text{usage}) \rceil$
-
-## 8) Candidate Generation, Re-ranking, Diversification Pipeline (Recommended)
-
-### CandidateGen
-$$C = \text{TopN}_e(\text{ANN by sim}_e) \cup \text{TopN}_k(\text{BM25})$$
-(Recommended $N_e=50$, $N_k=50$)
-
-### Feature Compute
-Calculate relevance, recency, importance, usage, duplication_penalty for each $m \in C$
-
-### Primary Score
-$$S(m) = \alpha \cdot \text{relevance} + \beta \cdot \text{recency} + \gamma \cdot \text{importance} + \delta \cdot \text{usage} - \varepsilon \cdot \text{duplication\_penalty}$$
-
-### Diversification (Optional)
-Apply MMR greedily to generate Top-K
-
-### Thresholding
-Keep only $S(m) \geq \theta_{\text{keep}}$ (e.g., $\theta_{\text{keep}}=0.35$)
-
-## 9) Normalization and Calibration
-
-Maintaining [0,1] scale for all partial scores makes combination easy. Recommend batch normalization for features with high variance per batch.
-
-$$x_{\text{norm}} = \frac{x - \text{min\_batch}}{\text{max\_batch} - \text{min\_batch} + \varepsilon}$$
-
-Example: Apply to usage, duplication_penalty, bm25_norm
-
-**Advantage**: Reduces query bias and collection deviation to increase score stability↑
-
-## 10) M1 Default Parameter Table (Summary)
-
-| Item | Value |
-|------|-------|
-| **Final formula** | $\alpha=0.50$, $\beta=0.20$, $\gamma=0.20$, $\delta=0.10$, $\varepsilon=0.15$ |
-| **relevance weights** | $w_e=0.60$, $w_k=0.30$, $w_t=0.05$, $w_p=0.05$ |
-| **recency** | $\text{half\_life}(\text{working})=2$, $\text{episodic}=30$, $\text{semantic}=180$ |
-| **importance** | $\text{pinned}=+0.20$, $\text{type\_boost}(\text{semantic}=+0.10, \text{working}=-0.05)$ |
-| **usage** | log-based, batch normalization |
-| **duplication_penalty** | max sim to selected |
-| **forgetting** | $\theta_{\text{soft}}=0.6$, $\theta_{\text{hard}}=0.8$, TTL_soft/hard see table above |
-| **review** | $p_{\text{recall}}(t)=\exp(-t/\tau)$, first 1day→6days→variable multiplier |
-
-Implementing this way can be used immediately in **M1 (personal use)**, and only coefficient re-tuning is needed when going to M2/M3.