math-skill 3.0.1 → 3.1.1

package/knowledge-base/optimization/riemannian-optimization.en.md

@@ -9,10 +9,11 @@ Optimization on smooth manifolds $\mathcal{M}$ (e.g., the orthogonal group $O(n)
 - Riemannian gradient: $\text{grad} f(x) = \text{proj}_{T_x\mathcal{M}}(\nabla f(x))$ (Euclidean gradient projected onto the tangent space)
 - Riemannian gradient descent: $x_{k+1} = R_{x_k}(-\alpha_k \cdot \text{grad} f(x_k))$, where $R$ is a retraction
 - Tangent space of the orthogonal group $O(n)$: $T_Q O(n) = \{Q\Omega : \Omega^T = -\Omega\}$ (left multiplication by skew-symmetric matrices)
-- Cayley retraction for the orthogonal group: $R_Q(\xi) = Q(I + \frac{1}{2}\Omega)^{-1}(I - \frac{1}{2}\Omega)$, $\xi = Q\Omega$
+- Cayley retraction for the orthogonal group: $R_Q(\xi) = Q(I + \frac{1}{2}\Omega)^{-1}(I - \frac{1}{2}\Omega)$, $\xi = Q\Omega$; equivalently in $A$-form: let $G = \nabla f(Q)$ be the Euclidean gradient and $A = GQ^T - QG^T$ (skew-symmetric), then $R_Q(t) = (I + \frac{t}{2}A)^{-1}(I - \frac{t}{2}A)Q$, with step size $t > 0$ moving along the negative gradient (descent direction)
 - Polar decomposition retraction: $R_Q(\xi) = (Q + \xi)(I + \xi^T\xi)^{-1/2}$ (projection onto the nearest orthogonal matrix)
 - Newton-Schulz orthogonalization: $X_{k+1} = \frac{1}{2}X_k(3I - X_k^T X_k)$, converging to the nearest orthogonal matrix
 - Hyperbolic space (Poincaré ball): $\text{grad}_{\mathcal{H}} f = \frac{(1-\|x\|^2)^2}{4} \nabla f(x)$
+- Riemannian gradient on the Stiefel manifold $St(n,p) = \{W : W^TW = I_p\}$: $\text{grad} f(W) = G - W \cdot \text{sym}(W^TG)$, where $G = \nabla f(W)$ is the Euclidean gradient and $\text{sym}(A) = \frac{A + A^T}{2}$ is the symmetric correction term. Note: one cannot simply use $G - WW^TG$ (orthogonal projection); the symmetric correction is essential to ensure the gradient lies in the tangent space.
 
 ## Applicable Problems
 
@@ -24,8 +25,8 @@ Optimization on smooth manifolds $\mathcal{M}$ (e.g., the orthogonal group $O(n)
 
 ## AI Design Translation
 
-- **Muon optimizer (orthogonalized gradient updates)**: Projects the momentum matrix $M$ onto the nearest orthogonal matrix via Newton-Schulz iteration: $X_{k+1} = \frac{1}{2}X_k(3I - X_k^TX_k)$, converging in 5 steps. Update $W \leftarrow W - \alpha \cdot U$ ($U$ being the orthogonalized result). The core is a pure matmul chain, 2 matmul operations per step, fully tensor-core-friendly, stable under bf16.
-- **Manifold perspective on spectral normalization**: The constraint $\sigma_{\max}(W) = 1$ is equivalent to projection onto a tangent direction of the Stiefel manifold. Power iteration for direction estimation + normalization = an approximate Riemannian gradient projection. Implementation is identical to standard spectral norm.
+- **Muon optimizer (orthogonalized gradient updates)**: Approximates the orthogonal polar factor of a scaled momentum matrix $M$ via Newton-Schulz / polar iteration, e.g. $X_{k+1} = \frac{1}{2}X_k(3I - X_k^TX_k)$ (after normalization and under spectral conditions). A fixed 5-step iteration is an engineering approximation, not an unconditional convergence guarantee. Update $W \leftarrow W - \alpha \cdot U$ ($U$ being the orthogonalized result). The core is a matmul chain, tensor-core-friendly, with scaling and residual checks needed in low precision.
+- **Manifold perspective on spectral normalization**: The common constraint $\sigma_{\max}(W) \leq 1$ is enforced by scaling weights by the largest singular value, i.e. a reparameterization / normalization. It is not a Stiefel tangent projection and not the exact nearest projection onto the spectral-norm ball. Power iteration only estimates the top singular direction.
 - **Orthogonal RNN**: Hidden state recurrence $h_t = \sigma(W h_{t-1} + U x_t)$, constraining $W \in O(n)$ to avoid vanishing/exploding gradients. During training, uses the Cayley parameterization $W = (I-A)(I+A)^{-1}$ ($A$ skew-symmetric), with backpropagation computing unconstrained gradients with respect to $A$. The core is matrix inversion $O(n^3)$ (acceptable for small layer dimensions).
 - **Poincaré embeddings (hyperbolic space)**: Embeds hierarchical data into the Poincaré ball $\mathcal{B}^n = \{x : \|x\| < 1\}$. Distance $d(x,y) = \text{arcosh}(1 + 2\|x-y\|^2 / ((1-\|x\|^2)(1-\|y\|^2)))$. The gradient is simply scaled by the metric factor $(1-\|x\|^2)^2/4$. Implemented as elementwise scaling, $O(d)$.
 - **Subspace learning on the Grassmann manifold**: Treats low-rank subspaces as points on the Grassmann manifold, updated online via Riemannian SGD. More suitable for streaming data than SVD. The projection $P = QQ^T$ update is implemented via QR decomposition, with the core being matmul + thin QR.
@@ -47,5 +48,18 @@ Optimization on smooth manifolds $\mathcal{M}$ (e.g., the orthogonal group $O(n)
 
 ## Further References
 
-- Distilled notes: references/books/matrix-analysis.md (Section 7.3 Polar Decomposition, Newton-Schulz Iteration, Section 2.6 SVD and Orthogonal Factors)
+- Distilled notes: ../../references/books/matrix-analysis.en.md (Section 7.3 Polar Decomposition, Newton-Schulz Iteration, Section 2.6 SVD and Orthogonal Factors)
 - Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 7 Section 7.3 (Polar Decomposition) + Absil, Mahony, Sepulchre, *Optimization Algorithms on Matrix Manifolds*, Princeton University Press, 2008
+
+
+## Routing Extensions
+- If local linearization is needed -> `../differential-geometry/tangent-space.en.md` (gradient computation in tangent space)
+- If retraction choice is needed -> `../differential-geometry/metric-tensor.en.md` (metric determines retraction map)
+- If the metric comes from Fisher information -> `../information-geometry/natural-gradient.en.md` (natural gradient under Fisher metric)
+
+## Extensible Directions
+- Retraction types: exponential map, projection retraction, Cayley transform
+- Vector transport: transporting vectors between different tangent spaces on a manifold
+- Riemannian conjugate gradient: conjugate gradient method on manifolds
+- Riemannian trust region: trust region methods on manifolds
+- Stochastic Riemannian optimization: SGD variants on manifolds

package/knowledge-base/optimization/riemannian-optimization.md

@@ -9,10 +9,11 @@
 - 黎曼梯度：$\text{grad} f(x) = \text{proj}_{T_x\mathcal{M}}(\nabla f(x))$（欧氏梯度投影到切空间）
 - 黎曼梯度下降：$x_{k+1} = R_{x_k}(-\alpha_k \cdot \text{grad} f(x_k))$，$R$ 为收缩映射（retraction）
 - 正交群 $O(n)$ 的切空间：$T_Q O(n) = \{Q\Omega : \Omega^T = -\Omega\}$（反对称矩阵左乘）
-- 正交群收缩映射（Cayley）：$R_Q(\xi) = Q(I + \frac{1}{2}\Omega)^{-1}(I - \frac{1}{2}\Omega)$，$\xi = Q\Omega$
+- 正交群收缩映射（Cayley）：$R_Q(\xi) = Q(I + \frac{1}{2}\Omega)^{-1}(I - \frac{1}{2}\Omega)$，$\xi = Q\Omega$；等价地用 $A$-形式：设 $G = \nabla f(Q)$ 为欧氏梯度，$A = GQ^T - QG^T$（反对称），则 $R_Q(t) = (I + \frac{t}{2}A)^{-1}(I - \frac{t}{2}A)Q$，步长 $t > 0$ 沿负梯度方向移动（下降方向）
 - 极分解收缩：$R_Q(\xi) = (Q + \xi)(I + \xi^T\xi)^{-1/2}$（投影到最近正交矩阵）
 - Newton-Schulz 正交化：$X_{k+1} = \frac{1}{2}X_k(3I - X_k^T X_k)$，收敛到最近正交矩阵
 - 双曲空间（Poincaré ball）：$\text{grad}_{\mathcal{H}} f = \frac{(1-\|x\|^2)^2}{4} \nabla f(x)$
+- Stiefel 流形 $St(n,p) = \{W : W^TW = I_p\}$ 的黎曼梯度：$\text{grad} f(W) = G - W \cdot \text{sym}(W^TG)$，其中 $G = \nabla f(W)$ 为欧氏梯度，$\text{sym}(A) = \frac{A + A^T}{2}$ 为对称修正项。注意：不能简单用 $G - WW^TG$（正交投影），必须包含对称修正才能保证梯度在切空间中。
 
 ## 适用问题
 
@@ -24,8 +25,8 @@
 
 ## AI 设计翻译
 
-- **Muon 优化器 (正交化梯度更新)**：将动量矩阵 $M$ 通过 Newton-Schulz 迭代投影到最近正交矩阵：$X_{k+1} = \frac{1}{2}X_k(3I - X_k^TX_k)$，5 步收敛。更新 $W \leftarrow W - \alpha \cdot U$（$U$ 为正交化结果）。核心是纯 matmul 链，每步 2 次 matmul，完全 tensor core 友好，bf16 稳定。
-- **谱归一化的流形视角**：约束 $\sigma_{\max}(W) = 1$ 等价于在 Stiefel 流形的某个切方向上做投影。Power iteration 估计方向 + 归一化 = 一种近似的黎曼梯度投影。实现同标准 spectral norm。
+- **Muon 优化器 (正交化梯度更新)**：将缩放后的动量矩阵 $M$ 通过 Newton-Schulz / polar 迭代近似其正交极因子，例如 $X_{k+1} = \frac{1}{2}X_k(3I - X_k^TX_k)$（需先归一化并满足谱条件）。固定 5 步是工程近似而非无条件收敛保证。更新 $W \leftarrow W - \alpha \cdot U$（$U$ 为正交化结果）。核心是 matmul 链，tensor core 友好，低精度下需缩放和残差监控。
+- **谱归一化的流形视角**：约束 $\sigma_{\max}(W) \leq 1$ 的常用做法是按最大奇异值缩放权重，属于重参数化/归一化；它不是 Stiefel 流形切空间投影，也不是到谱范数球的精确最近投影。Power iteration 只用于估计最大奇异方向。
 - **正交 RNN (orthogonal RNN)**：隐藏状态递推 $h_t = \sigma(W h_{t-1} + U x_t)$，约束 $W \in O(n)$ 避免梯度消失/爆炸。训练时用 Cayley 参数化 $W = (I-A)(I+A)^{-1}$（$A$ 反对称），反向传播走 $A$ 的无约束梯度。核心是矩阵求逆 $O(n^3)$（层维度小，可接受）。
 - **Poincaré 嵌入 (双曲空间)**：将层次数据嵌入 Poincaré ball $\mathcal{B}^n = \{x : \|x\| < 1\}$。距离 $d(x,y) = \text{arcosh}(1 + 2\|x-y\|^2 / ((1-\|x\|^2)(1-\|y\|^2)))$。梯度乘以度量因子 $(1-\|x\|^2)^2/4$ 即可。实现为 elementwise 缩放，$O(d)$。
 - **Grassmann 流形上的子空间学习**：将低秩子空间视为 Grassmann 流形上的点，用黎曼 SGD 在线更新。比 SVD 更适合 streaming 数据。投影 $P = QQ^T$ 的更新通过 QR 分解实现，核心是 matmul + thin QR。
@@ -47,5 +48,18 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/matrix-analysis.md（§7.3 极分解、Newton-Schulz 迭代、§2.6 SVD 与正交因子）
+- 蒸馏稿：../../references/books/matrix-analysis.md（§7.3 极分解、Newton-Schulz 迭代、§2.6 SVD 与正交因子）
 - 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 7 §7.3 (Polar Decomposition) + Absil, Mahony, Sepulchre, *Optimization Algorithms on Matrix Manifolds*, Princeton University Press, 2008
+
+
+## 路由扩展
+- 若需要局部线性化 → `../differential-geometry/tangent-space.md`（切空间上的梯度计算）
+- 若需要收缩映射的具体选择 → `../differential-geometry/metric-tensor.md`（度量决定收缩映射）
+- 若度量来自 Fisher 信息 → `../information-geometry/natural-gradient.md`（Fisher 度量下的自然梯度）
+
+## 可扩展方向
+- 收缩映射类型（retraction types）：指数映射、投影收缩、Cayley 变换
+- 向量传输（vector transport）：流形上向量在不同切空间间的传输
+- 黎曼共轭梯度（Riemannian conjugate gradient）：流形上的共轭梯度法
+- 黎曼信赖域（Riemannian trust region）：流形上的信赖域方法
+- 随机黎曼优化（stochastic Riemannian optimization）：流形上的 SGD 变体

package/knowledge-base/overview.en.md

@@ -1,6 +1,6 @@
-# Knowledge Base Navigation
+# Mathematical Activation Anchor Index
 
-> This file is the v3 knowledge base index, helping you find specific knowledge cards from problem types.
+> knowledge-base/ is not a closed encyclopedia but an activation entry point for high-frequency mathematical structures. Each card is an activation anchor that answers: what mathematical concept to activate, what deeper knowledge it connects to, what AI design actions it translates to, and where to extend when it is insufficient.
 
 ## Knowledge Base Structure
 
@@ -28,12 +28,24 @@ The knowledge base is organized by mathematical domain, with 7 domains and 31 kn
 | Need information compression | information-bottleneck, entropy, kl-divergence |
 | Need topological regularization | persistent-homology, euler-characteristic |
 
-## Relationship to Thinking Lenses
+## When Anchors Are Not Enough
 
-Thinking lenses (`../lenses/`) handle "what perspective to use"; the knowledge base provides "concrete mathematical tools." Typical chain:
+The current 31 anchors cover the most commonly used mathematical structures in AI research. When a problem requires mathematical tools not among them:
 
-```
-Lens diagnosis → Knowledge card provides tools → Design pattern translates to AI module
-```
+1. Check the corresponding domain's `*/index.en.md` (e.g., `topology/index.en.md`) for extended concepts and reference book directions
+2. Enter the **Knowledge Gap Protocol** defined in `../skills/math-research-activator/SKILL.en.md` to generate a temporary knowledge card
+3. Never respond with "knowledge base does not cover this" or force-fit the closest card
 
-For deeper study, `../references/books/*.md` provides 7 book distillations.
+## Domain Extension Index
+
+Each domain has an `*/index.en.md` that lists: domain trigger signals, core anchors, extended concepts, reference book directions, and temporary activation rules.
+
+| Domain | Extension Index |
+|--------|----------------|
+| Matrix Analysis | `matrix-analysis/index.en.md` |
+| Optimization | `optimization/index.en.md` |
+| Differential Geometry | `differential-geometry/index.en.md` |
+| Lie Theory | `lie-theory/index.en.md` |
+| Topology | `topology/index.en.md` |
+| Probability & Information | `probability/index.en.md` |
+| Information Geometry | `information-geometry/index.en.md` |

package/knowledge-base/overview.md

@@ -1,6 +1,6 @@
-# 数学知识库导航 / Knowledge Base Navigation
+# 数学激活锚点索引 / Mathematical Activation Anchor Index
 
-> 本文件是 v3 知识库的索引，帮助你从问题类型找到具体的知识卡片。
+> knowledge-base/ 不是封闭百科，而是高频数学结构的激活入口。每张卡片是一个激活锚点，回答：激活什么数学概念、连接哪些更深知识、可翻译成哪些 AI 设计动作、不足时应该往哪里扩展。
 
 ## 知识库结构
 
@@ -28,12 +28,24 @@
 | 需要信息压缩 | information-bottleneck, entropy, kl-divergence |
 | 需要拓扑正则 | persistent-homology, euler-characteristic |
 
-## 与思想透镜的关系
+## 当锚点不够时
 
-思想透镜（`../lenses/`）负责"用什么视角看问题"，知识库负责"提供具体数学工具"。典型链路：
+现有 31 个锚点覆盖 AI 研究中最常用的数学结构。当问题需要的数学工具不在其中时：
 
-```
-透镜诊断 → 知识卡片提供工具 → 设计模式翻译成 AI 模块
-```
+1. 查看对应领域的 `*/index.md`（如 `topology/index.md`），获取扩展概念和参考书方向
+2. 进入 SKILL.md 中定义的**知识缺口协议**，生成临时知识卡
+3. 不得回答"知识库未覆盖"或强行套用最相近卡片
 
-深入查阅时，`../references/books/*.md` 提供 7 本书的蒸馏稿。
+## 领域扩展索引
+
+每个领域有一个 `*/index.md`，列出：领域触发信号、核心锚点、扩展概念、参考书方向、临时激活规则。
+
+| 领域 | 扩展索引 |
+|------|---------|
+| 矩阵分析 | `matrix-analysis/index.md` |
+| 最优化 | `optimization/index.md` |
+| 微分几何 | `differential-geometry/index.md` |
+| 李理论 | `lie-theory/index.md` |
+| 拓扑 | `topology/index.md` |
+| 概率与信息 | `probability/index.md` |
+| 信息几何 | `information-geometry/index.md` |

package/knowledge-base/probability/concentration-inequality.en.md

@@ -39,7 +39,21 @@ $$P(|f(X_1,\ldots,X_n) - \mathbb{E}[f]| \geq t) \leq 2\exp\left(-\frac{2t^2}{\su
 - **Boundedness assumption violated**: Under heavy-tailed distributions (e.g., power laws), Hoeffding does not apply; Bernstein or truncation tricks are needed. In LLM training, when gradients occasionally take extreme values, naive concentration bounds yield a false sense of security.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated probability distillation draft at present; this card is based on standard probability theory textbooks
+- Distillation draft: `../../references/books/` — no dedicated probability distillation draft at present; this card is based on standard probability theory textbooks
 - Boucheron, Lugosi, Massart. *Concentration Inequalities: A Nonasymptotic Theory of Independence*. Oxford, 2013
 - Vershynin. *High-Dimensional Probability*. Cambridge, 2018 (Chapters 2-3)
-- Related knowledge cards: `probability/entropy.md`, `probability/fisher-information.md`
+- Related knowledge cards: `entropy.en.md`, `fisher-information.en.md`
+
+
+## Routing Extensions
+- If information-theoretic bounds are needed -> `entropy.en.md` (entropy-based concentration inequalities)
+- If Donsker-Varadhan representation is needed -> `kl-divergence.en.md` (variational representation of KL divergence)
+- If random matrix bounds are involved -> `../matrix-analysis/matrix-perturbation.md` (concentration behavior of random matrices)
+
+## Extensible Directions
+- Martingale concentration (Azuma / Freedman): concentration inequalities for martingale differences
+- Log-Sobolev inequalities: implying hypercontractivity and concentration
+- Transportation inequalities: relationship between Wasserstein distance and relative entropy
+- Talagrand's inequality: dimension-free concentration on product spaces
+- Concentration on product spaces: concentration of functions of independent variables
+- Dimension-free bounds: concentration bounds that do not degrade with dimension

package/knowledge-base/probability/concentration-inequality.md

@@ -39,7 +39,21 @@ $$P(|f(X_1,\ldots,X_n) - \mathbb{E}[f]| \geq t) \leq 2\exp\left(-\frac{2t^2}{\su
 - **有界性假设违反**：重尾分布（如幂律）下 Hoeffding 不适用，需改用 Bernstein 或截断技巧。在 LLM 训练中梯度偶尔出现极大值时，朴素浓度界给出虚假安全感。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用概率蒸馏稿，本卡基于标准概率论教材
+- 蒸馏稿：`../../references/books/` 暂无专用概率蒸馏稿，本卡基于标准概率论教材
 - Boucheron, Lugosi, Massart. *Concentration Inequalities: A Nonasymptotic Theory of Independence*. Oxford, 2013
 - Vershynin. *High-Dimensional Probability*. Cambridge, 2018（第 2-3 章）
-- 关联知识卡：`probability/entropy.md`、`probability/fisher-information.md`
+- 关联知识卡：`entropy.md`、`fisher-information.md`
+
+
+## 路由扩展
+- 若需要信息论界 → `entropy.md`（基于熵的集中不等式）
+- 若需要 Donsker-Varadhan 表示 → `kl-divergence.md`（KL 散度的变分表示）
+- 若涉及随机矩阵界 → `../matrix-analysis/matrix-perturbation.md`（随机矩阵的集中行为）
+
+## 可扩展方向
+- 鞅集中（Azuma / Freedman）：鞅差的集中不等式
+- 对数 Sobolev 不等式（log-Sobolev inequalities）：蕴含超收缩和集中
+- 传输不等式（transportation inequalities）：Wasserstein 距离与相对熵的关系
+- Talagrand 不等式：乘积空间上的维度无关集中
+- 乘积空间集中（concentration on product spaces）：独立变量函数的集中
+- 维度无关界（dimension-free bounds）：不随维度退化的集中界

package/knowledge-base/probability/entropy.en.md

@@ -26,7 +26,7 @@ $$I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = \sum_{x,y} p(x,y) \log \frac{p(x,y)}
 
 ## AI Design Translation
 - **Cross-Entropy Loss**: $H(p, q) = -\sum p(x)\log q(x)$, the default loss function for classification tasks; it is essentially the "coding redundancy" between the true distribution $p$ and the model distribution $q$
-- **KL Divergence** (see `kl-divergence.md`): $D_{KL}(p\|q) = H(p,q) - H(p)$, i.e., the difference between cross-entropy and entropy
+- **KL Divergence** (see `kl-divergence.en.md`): $D_{KL}(p\|q) = H(p,q) - H(p)$, i.e., the difference between cross-entropy and entropy
 - **Variational Autoencoder (VAE)**: ELBO = reconstruction likelihood $-$ KL regularization term; it fundamentally balances information compression (low $H(Z)$) against reconstruction fidelity
 
 ## Engineering Feasibility
@@ -41,7 +41,23 @@ $$I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = \sum_{x,y} p(x,y) \log \frac{p(x,y)}
 - **Sensitive to vocabulary size**: With large vocabularies (e.g., 128K tokenizers for LLMs), the peak memory usage of softmax + cross-entropy can reach tens of GB, requiring chunked/online softmax or label smoothing for mitigation.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated information theory distillation draft at present
+- Distillation draft: `../../references/books/` — no dedicated information theory distillation draft at present
 - Cover & Thomas. *Elements of Information Theory*, 2nd Edition. Wiley, 2006
 - MacKay. *Information Theory, Inference, and Learning Algorithms*. Cambridge, 2003
-- Related knowledge cards: `probability/kl-divergence.md`, `probability/information-bottleneck.md`
+- Related knowledge cards: `kl-divergence.en.md`, `information-bottleneck.en.md`
+
+
+## Routing Extensions
+- If relative entropy is needed -> `kl-divergence.en.md` (KL divergence is relative entropy)
+- If information compression is involved -> `information-bottleneck.en.md` (information bottleneck uses entropy and mutual information)
+- If entropy-power inequality is involved -> `fisher-information.en.md` (relationship between Fisher information and entropy)
+
+## Extensible Directions
+- Renyi entropy: parameterized family of generalized entropies
+- Tsallis entropy: entropy for non-extensive statistical mechanics
+- Conditional / mutual information: multi-variable information measures
+- Entropy rate: asymptotic entropy of stochastic processes
+- Maximum entropy principle: distribution selection under minimal assumptions
+- Entropy estimation: methods for estimating entropy from samples
+- Differential entropy: entropy for continuous distributions
+- Entropy power inequality: lower bound on entropy of independent sums

package/knowledge-base/probability/entropy.md

@@ -41,7 +41,23 @@ $$I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = \sum_{x,y} p(x,y) \log \frac{p(x,y)}
 - **对 vocab 大小敏感**：大 vocab（如 LLM 的 128K tokenizer）下 softmax + 交叉熵的显存峰值可达数十 GB，需 chunked/online softmax 或 label smoothing 缓解。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用信息论蒸馏稿
+- 蒸馏稿：`../../references/books/` 暂无专用信息论蒸馏稿
 - Cover & Thomas. *Elements of Information Theory*, 2nd Edition. Wiley, 2006
 - MacKay. *Information Theory, Inference, and Learning Algorithms*. Cambridge, 2003
-- 关联知识卡：`probability/kl-divergence.md`、`probability/information-bottleneck.md`
+- 关联知识卡：`kl-divergence.md`、`information-bottleneck.md`
+
+
+## 路由扩展
+- 若需要相对熵 → `kl-divergence.md`（KL 散度即相对熵）
+- 若涉及信息压缩 → `information-bottleneck.md`（信息瓶颈使用熵和互信息）
+- 若涉及熵-功率不等式 → `fisher-information.md`（Fisher 信息与熵的关系）
+
+## 可扩展方向
+- Renyi 熵（Renyi entropy）：广义熵的参数族
+- Tsallis 熵（Tsallis entropy）：非广延统计力学的熵
+- 条件/互信息（conditional / mutual information）：多变量信息度量
+- 熵率（entropy rate）：随机过程的渐近熵
+- 最大熵原理（maximum entropy principle）：最少假设下的分布选择
+- 熵估计（entropy estimation）：从样本估计熵的方法
+- 微分熵（differential entropy）：连续分布的熵
+- 熵功率不等式（entropy power inequality）：独立和的熵下界

package/knowledge-base/probability/fisher-information.en.md

@@ -19,7 +19,7 @@ $$D_{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} d\theta^T \mathca
 
 ## Applicable Problems
 - **Parameter estimation efficiency assessment**: The Cramér-Rao bound gives the theoretical limit on the precision of any unbiased estimator
-- **Natural gradient descent**: Use $\mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}$ instead of the naive gradient, updating along geodesic directions on the statistical manifold (see `information-geometry/natural-gradient.md`)
+- **Natural gradient descent**: Use $\mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}$ instead of the naive gradient, updating along geodesic directions on the statistical manifold (see `../information-geometry/natural-gradient.en.md`)
 - **Experimental design / active learning**: Select data points that maximize Fisher information, maximizing the information gain for parameter learning
 
 ## AI Design Translation
@@ -41,8 +41,22 @@ $$D_{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} d\theta^T \mathca
 - **Empirical Fisher ≠ True Fisher**: Replacing the expectation with a training set average introduces significant estimation bias when the sample size is insufficient, potentially causing the natural gradient direction to point in the wrong direction. This should be paired with learning rate warmup.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Distillation draft: `../../references/books/` — no dedicated information geometry distillation draft at present
 - Amari. *Information Geometry and Its Applications*. Springer, 2016
 - Amari & Nagaoka. *Methods of Information Geometry*. AMS, 2000
 - Martens. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
-- Related knowledge cards: `information-geometry/natural-gradient.md`, `information-geometry/fisher-metric.md`
+- Related knowledge cards: `../information-geometry/natural-gradient.en.md`, `../information-geometry/fisher-metric.en.md`
+
+
+## Routing Extensions
+- If a geometric perspective is needed -> `../information-geometry/fisher-metric.en.md` (Fisher information as Riemannian metric)
+- If Fisher-based optimization is needed -> `../information-geometry/natural-gradient.en.md` (natural gradient driven by Fisher information)
+- If Cramer-Rao bounds are needed -> `concentration-inequality.en.md` (Fisher information and estimation accuracy bounds)
+
+## Extensible Directions
+- Observed vs expected Fisher: two types of Fisher information matrices
+- Fisher information matrix properties: positive definiteness, chain rule, sufficient statistics
+- Jeffreys prior: non-informative prior defined by Fisher information
+- Fisher information distance: Fisher metric distance between distributions
+- Mutual information and Fisher: relationship between Fisher information and mutual information
+- Fisher in deep learning: approximation methods such as K-FAC, Shampoo

package/knowledge-base/probability/fisher-information.md

@@ -19,7 +19,7 @@ $$D_{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} d\theta^T \mathca
 
 ## 适用问题
 - **参数估计效率评估**：Cramér-Rao 界给出任何无偏估计器精度的理论极限
-- **自然梯度下降**：用 $\mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}$ 替代朴素梯度，沿统计流形的测地线方向更新（详见 `information-geometry/natural-gradient.md`）
+- **自然梯度下降**：用 $\mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}$ 替代朴素梯度，沿统计流形的测地线方向更新（详见 `../information-geometry/natural-gradient.md`）
 - **实验设计 / 主动学习**：选择使 Fisher 信息最大的数据点采集，最大化参数学习的信息增益
 
 ## AI 设计翻译
@@ -41,8 +41,22 @@ $$D_{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} d\theta^T \mathca
 - **经验 Fisher ≠ 真实 Fisher**：用训练集均值替代期望，在样本量不足时估计偏差大，自然梯度方向可能指向错误方向。需与 learning rate warmup 配合。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- 蒸馏稿：`../../references/books/` 暂无专用信息几何蒸馏稿
 - Amari. *Information Geometry and Its Applications*. Springer, 2016
 - Amari & Nagaoka. *Methods of Information Geometry*. AMS, 2000
 - Martens. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
-- 关联知识卡：`information-geometry/natural-gradient.md`、`information-geometry/fisher-metric.md`
+- 关联知识卡：`../information-geometry/natural-gradient.md`、`../information-geometry/fisher-metric.md`
+
+
+## 路由扩展
+- 若需要几何视角 → `../information-geometry/fisher-metric.md`（Fisher 信息作为黎曼度量）
+- 若需要基于 Fisher 的优化 → `../information-geometry/natural-gradient.md`（Fisher 信息驱动的自然梯度）
+- 若需要 Cramer-Rao 界 → `concentration-inequality.md`（Fisher 信息与估计精度界）
+
+## 可扩展方向
+- 观测 vs 期望 Fisher（observed vs expected Fisher）：两种 Fisher 信息矩阵
+- Fisher 信息矩阵性质：正定性、链式法则、充分统计量
+- Jeffreys 先验（Jeffreys prior）：Fisher 信息定义的无信息先验
+- Fisher 信息距离（Fisher information distance）：分布间的 Fisher 度量距离
+- 互信息与 Fisher（mutual information and Fisher）：Fisher 信息与互信息的关系
+- 深度学习中的 Fisher（Fisher in deep learning）：K-FAC, Shampoo 等近似方法

package/knowledge-base/probability/index.en.md

@@ -0,0 +1,51 @@
+# Probability & Information Activation Index
+
+## Domain Signals
+Activate this domain direction when the problem involves:
+- Uncertainty quantification: need to quantify the concentration or tail behavior of random variables
+- Generalization bounds: need to derive theoretical upper bounds on model generalization
+- Distribution distance: need to measure the difference between two distributions
+- Information compression: need to compress representations while preserving information
+- Sample efficiency: need to analyze the ability to learn from finite samples
+- Tail control: need to control the probability of extreme deviations of random variables
+
+## Core Anchors
+- `concentration-inequality.en.md` — Concentration inequalities
+- `entropy.en.md` — Entropy
+- `kl-divergence.en.md` — KL divergence
+- `information-bottleneck.en.md` — Information bottleneck
+- `fisher-information.en.md` — Fisher information
+
+## Extended Concepts
+When core anchors are insufficient, the following concepts may need temporary activation:
+- optimal transport (Wasserstein distance, Sinkhorn): optimal transport and Wasserstein distance
+- total variation distance: total variation distance
+- f-divergence family: f-divergence family (chi-squared, Hellinger, Jensen-Shannon, etc.)
+- mutual information estimation (MINE / NWJ): neural estimation methods for mutual information
+- variational inference (ELBO / VI): variational inference and evidence lower bound
+- Markov chain Monte Carlo: Markov chain Monte Carlo methods
+- stochastic process (martingale, Brownian motion, SDE): stochastic process fundamentals
+- PAC-Bayes bounds: PAC-Bayes generalization bounds
+- Rademacher complexity: Rademacher complexity
+- VC dimension: VC dimension and hypothesis space capacity
+- generalization via compression: compression-based generalization theory
+- differential privacy: differential privacy
+- normalizing flow theory: normalizing flow theory
+- score matching: score matching
+- diffusion process theory: diffusion process theory
+
+## Reference Book Directions
+- `../../references/books/optimization-ml.en.md`: variational methods and probabilistic inference chapters
+
+## AI Translation Directions
+- concentration inequality → generalization bounds / confidence-aware predictions / tail-risk loss
+- entropy → entropy regularization / uncertainty estimation / exploration bonus
+- kl-divergence → distribution matching loss / knowledge distillation / policy regularization
+- information bottleneck → representation compression / VIB loss / uncertainty routing
+- fisher information → natural gradient / parameter sensitivity monitor / active learning
+
+## Temporary Activation Rules
+When the problem requires mathematics not in the core anchors:
+1. First check whether extended concepts contain a match
+2. If yes, generate a temporary knowledge card based on the lens
+3. If no, enter the Knowledge Gap Protocol

package/knowledge-base/probability/index.md

@@ -0,0 +1,51 @@
+# 概率与信息 激活索引 / Probability & Information Activation Index
+
+## 领域信号
+当问题涉及以下信号时，激活本领域方向：
+- 不确定性量化：需要量化随机变量的集中程度或尾部行为
+- 泛化界：需要推导模型泛化的理论上界
+- 分布距离：需要度量两个分布之间的差异
+- 信息压缩：需要在保持信息的前提下压缩表示
+- 样本效率：需要分析从有限样本中学习的能力
+- 尾部控制：需要控制随机变量的极端偏差概率
+
+## 核心锚点
+- `concentration-inequality.md` — 集中不等式
+- `entropy.md` — 熵
+- `kl-divergence.md` — KL 散度
+- `information-bottleneck.md` — 信息瓶颈
+- `fisher-information.md` — Fisher 信息
+
+## 扩展概念
+当核心锚点不够时，以下概念可能需要临时激活：
+- optimal transport（Wasserstein distance, Sinkhorn）：最优传输与 Wasserstein 距离
+- total variation distance：全变差距离
+- f-divergence family：f-散度族（chi-squared, Hellinger, Jensen-Shannon 等）
+- mutual information estimation（MINE / NWJ）：互信息的神经估计方法
+- variational inference（ELBO / VI）：变分推断与证据下界
+- Markov chain Monte Carlo：马尔可夫链蒙特卡罗方法
+- stochastic process（martingale, Brownian motion, SDE）：随机过程基础
+- PAC-Bayes bounds：PAC-Bayes 泛化界
+- Rademacher complexity：Rademacher 复杂度
+- VC dimension：VC 维与假设空间容量
+- generalization via compression：基于压缩的泛化理论
+- differential privacy：差分隐私
+- normalizing flow theory：规范化流理论
+- score matching：得分匹配
+- diffusion process theory：扩散过程理论
+
+## 参考书方向
+- `../../references/books/optimization-ml.md`：变分方法和概率推断相关章节
+
+## AI 翻译方向
+- concentration inequality → generalization bounds / confidence-aware predictions / tail-risk loss
+- entropy → entropy regularization / uncertainty estimation / exploration bonus
+- kl-divergence → distribution matching loss / knowledge distillation / policy regularization
+- information bottleneck → representation compression / VIB loss / uncertainty routing
+- fisher information → natural gradient / parameter sensitivity monitor / active learning
+
+## 临时激活规则
+当问题需要的数学不在核心锚点中时：
+1. 先检查扩展概念中是否有匹配
+2. 若有，根据透镜生成临时知识卡
+3. 若无，进入 Knowledge Gap Protocol

package/knowledge-base/probability/information-bottleneck.en.md

@@ -42,8 +42,24 @@ where $q_\phi(y|z)$ is the classifier/decoder, $r(z)$ is the prior distribution
 - **Sensitive to $\beta$ tuning**: If $\beta$ is too large, excessive compression leads to underfitting; if too small, the objective degenerates to standard ERM (no compression effect). Information plane analysis or adaptive $\beta$ scheduling is required.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated IB distillation draft at present
+- Distillation draft: `../../references/books/` — no dedicated IB distillation draft at present
 - Tishby, Pereira, Bialek. "The Information Bottleneck Method." *arXiv:physics/0004057*, 2000
 - Alemi, Poole, Fischer, Dillon, Suresh, Murphy. "Deep Variational Information Bottleneck." *ICLR*, 2017
 - Shwartz-Ziv, Tishby. "Opening the Black Box of Deep Neural Networks via Information." *arXiv:1703.00810*, 2017
-- Related knowledge cards: `probability/entropy.md`, `probability/kl-divergence.md`
+- Related knowledge cards: `entropy.en.md`, `kl-divergence.en.md`
+
+
+## Routing Extensions
+- If the KL component in IB objective is needed -> `kl-divergence.en.md` (KL component of IB objective)
+- If rate-distortion theory is involved -> `entropy.en.md` (relationship between IB and rate-distortion theory)
+- If used for VIB loss design -> `variational-loss` (design pattern layer for variational information bottleneck loss)
+
+## Extensible Directions
+- Rate-distortion theory: optimal compression bounds from information theory
+- Deterministic IB: IB variant with deterministic encoding
+- Geometric IB: information bottleneck under geometric structure
+- IB for representation learning: feature learning under IB framework
+- IB for clustering: IB-driven clustering algorithms
+- Deep IB: information bottleneck in deep networks
+- IB with multiple bottlenecks: multi-layer information constraints
+- IB generalization bounds: theoretical connection between IB and generalization

package/knowledge-base/probability/information-bottleneck.md

@@ -42,8 +42,24 @@ $$\mathcal{L}_{VIB} = \mathbb{E}_{p(x,y)}[-\log q_\phi(y|z)] + D_{KL}(p_\theta(z
 - **$\beta$ 调参敏感**：$\beta$ 过大会过度压缩（欠拟合），过小则退化为标准 ERM（无压缩效果）。需要信息平面分析或自适应 $\beta$ 调度。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用 IB 蒸馏稿
+- 蒸馏稿：`../../references/books/` 暂无专用 IB 蒸馏稿
 - Tishby, Pereira, Bialek. "The Information Bottleneck Method." *arXiv:physics/0004057*, 2000
 - Alemi, Poole, Fischer, Dillon, Suresh, Murphy. "Deep Variational Information Bottleneck." *ICLR*, 2017
 - Shwartz-Ziv, Tishby. "Opening the Black Box of Deep Neural Networks via Information." *arXiv:1703.00810*, 2017
-- 关联知识卡：`probability/entropy.md`、`probability/kl-divergence.md`
+- 关联知识卡：`entropy.md`、`kl-divergence.md`
+
+
+## 路由扩展
+- 若需要 IB 目标中的 KL → `kl-divergence.md`（IB 目标的 KL 分量）
+- 若涉及速率-失真理论 → `entropy.md`（IB 与率失真理论的关系）
+- 若用于 VIB 损失设计 → `variational-loss`（设计模式层的变分信息瓶颈损失）
+
+## 可扩展方向
+- 速率-失真理论（rate-distortion theory）：信息论的最优压缩界
+- 确定性 IB（deterministic IB）：确定性编码的 IB 变体
+- 几何 IB（geometric IB）：几何结构下的信息瓶颈
+- IB 用于表示学习（IB for representation learning）：IB 框架下的特征学习
+- IB 用于聚类（IB for clustering）：IB 驱动的聚类算法
+- 深度 IB（deep IB）：深度网络中的信息瓶颈
+- 多瓶颈 IB（IB with multiple bottlenecks）：多层信息约束
+- IB 泛化界（IB generalization bounds）：IB 与泛化能力的理论联系

package/knowledge-base/probability/kl-divergence.en.md

@@ -45,7 +45,20 @@ $$D_{KL}(p \| q) = H(p, q) - H(p)$$
 - **High gradient variance**: In RL (PPO/RLHF), KL estimation relies on sampling; high variance can lead to training instability. A clipped + linear approximation $\mathbb{E}[\log p - \log q]$ is commonly used in place of the exact KL.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated information theory distillation draft at present
+- Distillation draft: `../../references/books/` — no dedicated information theory distillation draft at present
 - Cover & Thomas. *Elements of Information Theory*, Ch. 2-3. Wiley, 2006
 - Murphy. *Probabilistic Machine Learning: Advanced Topics*, Ch. 6. MIT Press, 2023
-- Related knowledge cards: `probability/entropy.md`, `probability/information-bottleneck.md`, `probability/fisher-information.md`
+- Related knowledge cards: `entropy.en.md`, `information-bottleneck.en.md`, `fisher-information.en.md`
+
+
+## Routing Extensions
+- If used for IB objective -> `information-bottleneck.en.md` (IB defines objective using KL)
+- If the absolute version is needed -> `entropy.en.md` (KL divergence reduces to entropy)
+- If local KL geometry is needed -> `fisher-information.en.md` (Fisher information is local curvature of KL)
+
+## Extensible Directions
+- f-divergence family (chi-squared, Hellinger, Jensen-Shannon, total variation): unified divergence framework
+- Reverse KL (mode-seeking): mode-seeking behavior in variational inference
+- Bregman divergence: divergence family generated by convex functions
+- Variational representations (Donsker-Varadhan, Barber-Agakov): variational upper/lower bounds of KL divergence
+- KL in exponential families: geometry of sufficient statistics

package/knowledge-base/probability/kl-divergence.md

@@ -45,7 +45,20 @@ $$D_{KL}(p \| q) = H(p, q) - H(p)$$
 - **梯度方差大**：在 RL（PPO/RLHF）中，KL 估计依赖采样，高方差可导致训练不稳定。常用 clip + 线性近似 $\mathbb{E}[\log p - \log q]$ 替代精确 KL。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用信息论蒸馏稿
+- 蒸馏稿：`../../references/books/` 暂无专用信息论蒸馏稿
 - Cover & Thomas. *Elements of Information Theory*, Ch. 2-3. Wiley, 2006
 - Murphy. *Probabilistic Machine Learning: Advanced Topics*, Ch. 6. MIT Press, 2023
-- 关联知识卡：`probability/entropy.md`、`probability/information-bottleneck.md`、`probability/fisher-information.md`
+- 关联知识卡：`entropy.md`、`information-bottleneck.md`、`fisher-information.md`
+
+
+## 路由扩展
+- 若用于 IB 目标函数 → `information-bottleneck.md`（IB 使用 KL 定义目标）
+- 若需要绝对版本 → `entropy.md`（KL 散度退化为熵）
+- 若需要局部 KL 几何 → `fisher-information.md`（Fisher 信息是 KL 的局部曲率）
+
+## 可扩展方向
+- f-散度族（chi-squared, Hellinger, Jensen-Shannon, total variation）：统一的散度框架
+- 逆 KL（reverse KL / mode-seeking）：变分推断中的模式搜索行为
+- Bregman 散度（Bregman divergence）：凸函数生成的散度族
+- 变分表示（Donsker-Varadhan, Barber-Agakov）：KL 散度的变分上/下界
+- 指数族中的 KL（KL in exponential families）：充分统计量的几何

package/knowledge-base/topology/euler-characteristic.en.md

@@ -48,7 +48,20 @@ High GPU friendliness. Computing the Euler characteristic is extremely cheap:
 
 ## Further References
 
-- Distillation notes: references/books/smooth-manifolds.md (Ch 17--18 De Rham Cohomology, Betti numbers and cohomology)
-- Distillation notes: references/books/differential-geometry.md (Ch 4 Curves and Hypersurfaces, intuition source for Gauss curvature)
+- Distillation notes: ../../references/books/smooth-manifolds.en.md (Ch 17--18 De Rham Cohomology, Betti numbers and cohomology)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 4 Curves and Hypersurfaces, intuition source for Gauss curvature)
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 17--18 (de Rham cohomology and topological invariants)
 - Extended reading: Hatcher, *Algebraic Topology*, Ch 2 (standard treatment of simplicial homology and the Euler characteristic)
+
+
+## Routing Extensions
+- If multi-scale topology analysis is needed -> `persistent-homology.en.md` (persistent homology provides scale-dependent topology)
+- If curvature-topology connection is involved -> `../differential-geometry/curvature.md` (Gauss-Bonnet theorem connects curvature and Euler characteristic)
+
+## Extensible Directions
+- Betti numbers: counting independent loops at each dimension
+- Poincare polynomial: generating function of Betti numbers
+- Lefschetz fixed point theorem: Euler characteristic and map fixed points
+- Morse inequalities: relationship between critical points and Betti numbers
+- Discrete Morse theory: Morse functions on complexes
+- Euler characteristic curve: Euler characteristic variation across thresholds

package/knowledge-base/topology/euler-characteristic.md

@@ -48,7 +48,20 @@ GPU 友好度高。欧拉示性数的计算极其廉价：
 
 ## 深入参考
 
-- 蒸馏稿：references/books/smooth-manifolds.md（Ch 17-18 De Rham Cohomology，Betti 数与上同调）
-- 蒸馏稿：references/books/differential-geometry.md（Ch 4 Curves and Hypersurfaces, Gauss curvature 直觉来源）
+- 蒸馏稿：../../references/books/smooth-manifolds.md（Ch 17-18 De Rham Cohomology，Betti 数与上同调）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 4 Curves and Hypersurfaces, Gauss curvature 直觉来源）
 - 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 17-18（de Rham 上同调与拓扑不变量）
 - 延伸：Hatcher, *Algebraic Topology*, Ch 2（单纯同调与欧拉示性数的标准处理）
+
+
+## 路由扩展
+- 若需要多尺度拓扑分析 → `persistent-homology.md`（持续同调提供尺度依赖的拓扑）
+- 若涉及曲率-拓扑联系 → `../differential-geometry/curvature.md`（Gauss-Bonnet 定理连接曲率与 Euler 示性数）
+
+## 可扩展方向
+- Betti 数（Betti numbers）：各维度的独立环路计数
+- Poincare 多项式（Poincare polynomial）：Betti 数的生成函数
+- Lefschetz 不动点定理：Euler 示性数与映射不动点
+- Morse 不等式（Morse inequalities）：临界点与 Betti 数的关系
+- 离散 Morse 理论（discrete Morse theory）：复形上的 Morse 函数
+- Euler 示性数曲线（Euler characteristic curve）：多阈值的 Euler 示性数变化