math-skill 2.0.1 → 3.0.0

package/knowledge-base/probability/entropy.md

@@ -0,0 +1,47 @@
+# 熵 / Entropy
+
+## 最小定义
+Shannon 熵度量一个随机变量的**不确定性总量**——即描述该变量所需的最少平均比特数。它是信息论的基石，也是最大似然、变分推断、正则化等 AI 方法的底层统一量。
+
+## 核心公式
+
+**Shannon 熵**（离散）：
+$$H(X) = -\sum_{x} p(x) \log p(x)$$
+
+**微分熵**（连续）：
+$$h(X) = -\int p(x) \log p(x)\, dx$$
+
+**联合熵与条件熵**：
+$$H(X, Y) = H(X) + H(Y|X), \quad H(Y|X) = -\sum_{x,y} p(x,y) \log p(y|x)$$
+
+**互信息**（两个变量共享的信息量）：
+$$I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = \sum_{x,y} p(x,y) \log \frac{p(x,y)}{p(x)p(y)}$$
+
+**最大熵原理**：在满足约束条件 $\mathbb{E}[f_i(X)] = c_i$ 的所有分布中，使 $H(X)$ 最大的分布为指数族 $p(x) \propto \exp\left(\sum \lambda_i f_i(x)\right)$。
+
+## 适用问题
+- **特征选择**：用互信息 $I(X; Y)$ 筛选对目标变量最有信息量的特征
+- **模型压缩与量化**：熵给出无损压缩的理论下界（Shannon 编码定理）
+- **正则化设计**：最大熵正则化鼓励模型输出"不确定但公平"的分布，防止过自信
+
+## AI 设计翻译
+- **交叉熵损失 Cross-Entropy Loss**：$H(p, q) = -\sum p(x)\log q(x)$，分类任务的默认损失函数，本质是真实分布 $p$ 与模型分布 $q$ 之间的"编码冗余"
+- **KL 散度**（详见 `kl-divergence.md`）：$D_{KL}(p\|q) = H(p,q) - H(p)$，即交叉熵与熵之差
+- **变分自编码器 VAE**：ELBO = 重构似然 $-$ KL 正则项，本质是在信息压缩（低 $H(Z)$）与重构保真之间取平衡
+
+## 工程可行性
+- **维度 1 张量化 ✅**：$-\sum p \log p$ 是逐元素运算，完美向量化
+- **维度 2 GEMM 可映射 ⚠️**：熵本身不是 GEMM，但交叉熵损失的梯度计算涉及 softmax → matmul 链
+- **维度 3 复杂度 ✅**：$O(|\mathcal{X}|)$ 线性，vocab 级计算可接受
+- **维度 5 低精度 ✅**：$\log$ 和 exp 在 bf16 下稳定，softmax 有 log-sum-exp 技巧
+- **维度 8 算子融合 ✅**：softmax + cross-entropy 是经典融合算子（FusedSoftmaxCrossEntropy）
+
+## 风险与失效条件
+- **连续熵可为负**：微分熵 $h(X)$ 不受 $H(X) \geq 0$ 约束，直接比较不同量纲的微分熵可产生误导。应改用互信息或 KL 散度（非负）。
+- **对 vocab 大小敏感**：大 vocab（如 LLM 的 128K tokenizer）下 softmax + 交叉熵的显存峰值可达数十 GB，需 chunked/online softmax 或 label smoothing 缓解。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息论蒸馏稿
+- Cover & Thomas. *Elements of Information Theory*, 2nd Edition. Wiley, 2006
+- MacKay. *Information Theory, Inference, and Learning Algorithms*. Cambridge, 2003
+- 关联知识卡：`probability/kl-divergence.md`、`probability/information-bottleneck.md`

package/knowledge-base/probability/fisher-information.en.md

@@ -0,0 +1,48 @@
+# Fisher Information
+
+## Minimal Definition
+Fisher information measures the **sensitivity of a parametric family of probability distributions to its parameters** — that is, how much information observed data provides about the parameter $\theta$. It defines a natural Riemannian metric on the statistical manifold (the Fisher information matrix = the metric tensor), and is the cornerstone of information geometry.
+
+## Core Formulas
+
+**Fisher Information (scalar parameter)**:
+$$\mathcal{I}(\theta) = \mathbb{E}_\theta\left[\left(\frac{\partial}{\partial \theta} \log p(X|\theta)\right)^2\right] = -\mathbb{E}_\theta\left[\frac{\partial^2}{\partial \theta^2} \log p(X|\theta)\right]$$
+
+**Fisher Information Matrix (vector parameter)**:
+$$[\mathcal{I}(\theta)]_{ij} = \mathbb{E}_\theta\left[\frac{\partial \log p(X|\theta)}{\partial \theta_i} \frac{\partial \log p(X|\theta)}{\partial \theta_j}\right] = -\mathbb{E}_\theta\left[\frac{\partial^2 \log p(X|\theta)}{\partial \theta_i \partial \theta_j}\right]$$
+
+**Cramér-Rao Lower Bound** (variance lower bound for unbiased estimators):
+$$\text{Var}(\hat{\theta}) \geq \frac{1}{\mathcal{I}(\theta)}$$
+
+**Relationship to KL Divergence** (Fisher information = second-order expansion coefficient of KL divergence):
+$$D_{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} d\theta^T \mathcal{I}(\theta) d\theta$$
+
+## Applicable Problems
+- **Parameter estimation efficiency assessment**: The Cramér-Rao bound gives the theoretical limit on the precision of any unbiased estimator
+- **Natural gradient descent**: Use $\mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}$ instead of the naive gradient, updating along geodesic directions on the statistical manifold (see `information-geometry/natural-gradient.md`)
+- **Experimental design / active learning**: Select data points that maximize Fisher information, maximizing the information gain for parameter learning
+
+## AI Design Translation
+- **Natural Gradient / K-FAC Optimizer**: Use a Kronecker approximation of the Fisher information matrix $\mathcal{I} \approx A \otimes B$ in place of the Hessian, enabling approximate second-order optimization
+- **Elastic Weight Consolidation (EWC)**: $\mathcal{L}_{\text{EWC}} = \mathcal{L}_{\text{new}} + \frac{\lambda}{2} \sum_i \mathcal{I}_i (\theta_i - \theta_i^*)^2$, using Fisher information to measure the importance of each parameter to previous tasks, preventing catastrophic forgetting
+- **Sensitivity analysis for pretraining-finetuning**: Parameter directions with high Fisher information = parameters sensitive to data; these should be handled more carefully during fine-tuning
+
+## Engineering Feasibility
+- **Dimension 1 Tensorization ⚠️**: The full FIM is a $d \times d$ matrix ($d$ = number of parameters); direct materialization is infeasible (LLM parameter counts $10^{10}+$). Approximations are required.
+- **Dimension 2 GEMM-mappability ✅**: K-FAC uses Kronecker factors $A \otimes B$; $A$ and $B$ can each be computed and inverted using GEMM
+- **Dimension 3 Complexity ⚠️**: Exact FIM computation is $O(nd^2)$; K-FAC reduces this to $O(d)$ scale but requires per-layer maintenance
+- **Dimension 4 Memory ⚠️**: K-FAC's Kronecker factors require additional memory, though significantly compressed compared to the full FIM
+- **Dimension 5 Low Precision ✅**: FIM estimation can use fp32; fp64 is not required
+- **Dimension 6 Parallelism ✅**: K-FAC's Kronecker factors naturally decompose by layer, enabling parallel computation
+- **Dimension 8 Operator Fusion ✅**: The EWC penalty term is element-wise and can be fused into the parameter update kernel
+
+## Risks and Failure Conditions
+- **Full FIM is intractable**: For LLM-scale parameter counts ($d > 10^9$), even K-FAC's Kronecker approximation may be too costly. In practice, diagonal Fisher ($O(d)$) or low-rank approximations are commonly used.
+- **Empirical Fisher ≠ True Fisher**: Replacing the expectation with a training set average introduces significant estimation bias when the sample size is insufficient, potentially causing the natural gradient direction to point in the wrong direction. This should be paired with learning rate warmup.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Amari. *Information Geometry and Its Applications*. Springer, 2016
+- Amari & Nagaoka. *Methods of Information Geometry*. AMS, 2000
+- Martens. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
+- Related knowledge cards: `information-geometry/natural-gradient.md`, `information-geometry/fisher-metric.md`

package/knowledge-base/probability/fisher-information.md

@@ -0,0 +1,48 @@
+# Fisher 信息 / Fisher Information
+
+## 最小定义
+Fisher 信息度量**概率分布族对参数的敏感度**——即观测数据对参数 $\theta$ 提供了多少信息。它在统计流形上定义了一个自然的黎曼度量（Fisher 信息矩阵 = 度量张量），是信息几何的基石。
+
+## 核心公式
+
+**Fisher 信息（标量参数）**：
+$$\mathcal{I}(\theta) = \mathbb{E}_\theta\left[\left(\frac{\partial}{\partial \theta} \log p(X|\theta)\right)^2\right] = -\mathbb{E}_\theta\left[\frac{\partial^2}{\partial \theta^2} \log p(X|\theta)\right]$$
+
+**Fisher 信息矩阵（向量参数）**：
+$$[\mathcal{I}(\theta)]_{ij} = \mathbb{E}_\theta\left[\frac{\partial \log p(X|\theta)}{\partial \theta_i} \frac{\partial \log p(X|\theta)}{\partial \theta_j}\right] = -\mathbb{E}_\theta\left[\frac{\partial^2 \log p(X|\theta)}{\partial \theta_i \partial \theta_j}\right]$$
+
+**Cramér-Rao 下界**（无偏估计的方差下界）：
+$$\text{Var}(\hat{\theta}) \geq \frac{1}{\mathcal{I}(\theta)}$$
+
+**与 KL 散度的关系**（Fisher 信息 = KL 散度的二阶展开系数）：
+$$D_{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} d\theta^T \mathcal{I}(\theta) d\theta$$
+
+## 适用问题
+- **参数估计效率评估**：Cramér-Rao 界给出任何无偏估计器精度的理论极限
+- **自然梯度下降**：用 $\mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}$ 替代朴素梯度，沿统计流形的测地线方向更新（详见 `information-geometry/natural-gradient.md`）
+- **实验设计 / 主动学习**：选择使 Fisher 信息最大的数据点采集，最大化参数学习的信息增益
+
+## AI 设计翻译
+- **自然梯度 / K-FAC 优化器**：用 Fisher 信息矩阵的 Kronecker 近似 $\mathcal{I} \approx A \otimes B$ 替代 Hessian，实现近似二阶优化
+- **弹性权重巩固 EWC**：$\mathcal{L}_{\text{EWC}} = \mathcal{L}_{\text{new}} + \frac{\lambda}{2} \sum_i \mathcal{I}_i (\theta_i - \theta_i^*)^2$，用 Fisher 信息衡量每个参数对旧任务的重要性，防止灾难性遗忘
+- **预训练-微调的敏感度分析**：Fisher 信息大的参数方向 = 对数据敏感的参数，微调时应更谨慎
+
+## 工程可行性
+- **维度 1 张量化 ⚠️**：完整 FIM 是 $d \times d$ 矩阵（$d$ = 参数量），直接物化不可行（LLM 参数量 $10^{10}+$）。必须用近似。
+- **维度 2 GEMM 可映射 ✅**：K-FAC 用 Kronecker 因子 $A \otimes B$，$A$ 和 $B$ 各自可用 GEMM 计算和求逆
+- **维度 3 复杂度 ⚠️**：精确 FIM 计算 $O(nd^2)$，K-FAC 降至 $O(d)$ 量级但需逐层维护
+- **维度 4 显存 ⚠️**：K-FAC 的 Kronecker 因子需额外显存，但相比完整 FIM 已大幅压缩
+- **维度 5 低精度 ✅**：FIM 估计用 fp32 即可，不需 fp64
+- **维度 6 并行 ✅**：K-FAC 的 Kronecker 因子天然按层分解，可并行计算
+- **维度 8 算子融合 ✅**：EWC 惩罚项为逐元素运算，可融入参数更新 kernel
+
+## 风险与失效条件
+- **完整 FIM 不可计算**：对于 LLM 级别的参数规模（$d > 10^9$），即使 K-FAC 的 Kronecker 近似也可能代价过高。实践中常用对角 Fisher（$O(d)$）或低秩近似。
+- **经验 Fisher ≠ 真实 Fisher**：用训练集均值替代期望，在样本量不足时估计偏差大，自然梯度方向可能指向错误方向。需与 learning rate warmup 配合。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- Amari. *Information Geometry and Its Applications*. Springer, 2016
+- Amari & Nagaoka. *Methods of Information Geometry*. AMS, 2000
+- Martens. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
+- 关联知识卡：`information-geometry/natural-gradient.md`、`information-geometry/fisher-metric.md`

package/knowledge-base/probability/information-bottleneck.en.md

@@ -0,0 +1,49 @@
+# Information Bottleneck
+
+## Minimal Definition
+The Information Bottleneck (IB) is a theoretical framework for representation learning: given input $X$ and target $Y$, find a compressed representation $Z$ that **discards as much information in $X$ that is irrelevant to $Y$ as possible, while retaining information useful for predicting $Y$**. It recasts the learning problem as an information-theoretic constrained optimization.
+
+## Core Formulas
+
+**IB Objective**:
+$$\min_{p(z|x)} \; I(X; Z) - \beta \cdot I(Z; Y)$$
+
+where $\beta > 0$ is a Lagrange multiplier controlling the **compression-prediction** trade-off:
+- $I(X; Z)$: the amount of information about input $X$ retained in representation $Z$ (smaller = stronger compression)
+- $I(Z; Y)$: the amount of information about target $Y$ contained in representation $Z$ (larger = better prediction)
+
+**Variational Lower Bound** (practically computable version):
+$$\mathcal{L}_{VIB} = \mathbb{E}_{p(x,y)}[-\log q_\phi(y|z)] + D_{KL}(p_\theta(z|x) \| r(z))$$
+
+where $q_\phi(y|z)$ is the classifier/decoder, $r(z)$ is the prior distribution (typically $\mathcal{N}(0,I)$), and $p_\theta(z|x)$ is the encoder.
+
+**IB Curve**: In the $(I(X;Z), I(Z;Y))$ plane, the Pareto-optimal solutions form a concave curve, with inflection points corresponding to optimal compression rates.
+
+## Applicable Problems
+- **Understanding the learning dynamics of deep networks**: Information Plane analysis — tracking the trajectory of $(I(X;Z_l), I(Z_l;Y))$ for each layer during training
+- **Theoretical guidance for representation learning**: Why regularization (dropout, weight decay) works — they implicitly compress redundant information
+- **Feature selection and dimensionality reduction**: Finding Pareto-optimal points between compression rate and predictive performance
+
+## AI Design Translation
+- **VIB Layer (Variational Information Bottleneck)**: Encoder $p_\theta(z|x)$ + KL regularization + decoder $q_\phi(y|z)$; structurally identical to a VAE but with different objective semantics (VAE reconstructs $X$, VIB predicts $Y$)
+- **Unified perspective on $\beta$-VAE**: The $\beta$-VAE objective is formally identical to the VIB objective, where $\beta$ is the IB Lagrange multiplier
+- **Information-theoretic interpretation of attention sparsification / routing**: Sparse Attention and MoE routing can be understood as implicit information bottlenecks — selectively allowing "useful" tokens to pass while discarding noise
+
+## Engineering Feasibility
+- **Dimension 1 Tensorization ✅**: The VIB encoder/decoder are standard networks; $D_{KL}$ is computed element-wise
+- **Dimension 2 GEMM-mappability ✅**: The main computation is a standard feedforward network + GEMM
+- **Dimension 3 Complexity ✅**: Only adds $O(d)$ computation for the KL term compared to the original network
+- **Dimension 4 Memory ⚠️**: Requires additional parameters for the prior distribution $r(z)$ and intermediate quantities for KL computation
+- **Dimension 5 Low Precision ✅**: Reparameterization trick + analytical KL solution are stable in bf16
+- **Dimension 8 Operator Fusion ✅**: No conflict with the standard training pipeline; normal fusion applies
+
+## Risks and Failure Conditions
+- **Accurate estimation of $I(X;Z)$ is difficult**: Mutual information estimation between high-dimensional continuous variables is itself an open problem (estimators such as MINE and NWJ have high variance). In practice, the VIB variational lower bound is used as a workaround, but the bound may be loose.
+- **Sensitive to $\beta$ tuning**: If $\beta$ is too large, excessive compression leads to underfitting; if too small, the objective degenerates to standard ERM (no compression effect). Information plane analysis or adaptive $\beta$ scheduling is required.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated IB distillation draft at present
+- Tishby, Pereira, Bialek. "The Information Bottleneck Method." *arXiv:physics/0004057*, 2000
+- Alemi, Poole, Fischer, Dillon, Suresh, Murphy. "Deep Variational Information Bottleneck." *ICLR*, 2017
+- Shwartz-Ziv, Tishby. "Opening the Black Box of Deep Neural Networks via Information." *arXiv:1703.00810*, 2017
+- Related knowledge cards: `probability/entropy.md`, `probability/kl-divergence.md`

package/knowledge-base/probability/information-bottleneck.md

@@ -0,0 +1,49 @@
+# 信息瓶颈 / Information Bottleneck
+
+## 最小定义
+信息瓶颈（IB）是一种表示学习理论框架：给定输入 $X$ 和目标 $Y$，寻找压缩表示 $Z$，使其**尽可能丢弃 $X$ 中与 $Y$ 无关的信息，同时保留对预测 $Y$ 有用的信息**。它将学习问题重新表述为信息论约束优化。
+
+## 核心公式
+
+**IB 目标函数**：
+$$\min_{p(z|x)} \; I(X; Z) - \beta \cdot I(Z; Y)$$
+
+其中 $\beta > 0$ 为拉格朗日乘子，控制**压缩-预测**的权衡：
+- $I(X; Z)$：表示 $Z$ 中保留的关于输入 $X$ 的信息量（越小 = 压缩越强）
+- $I(Z; Y)$：表示 $Z$ 中关于目标 $Y$ 的信息量（越大 = 预测越好）
+
+**变分下界**（实际可计算版本）：
+$$\mathcal{L}_{VIB} = \mathbb{E}_{p(x,y)}[-\log q_\phi(y|z)] + D_{KL}(p_\theta(z|x) \| r(z))$$
+
+其中 $q_\phi(y|z)$ 为分类器/解码器，$r(z)$ 为先验分布（通常为 $\mathcal{N}(0,I)$），$p_\theta(z|x)$ 为编码器。
+
+**IB 曲线**：在 $(I(X;Z), I(Z;Y))$ 平面上，帕累托最优解构成一条凹曲线，拐点处对应最优压缩率。
+
+## 适用问题
+- **理解深度网络的学习动态**：信息平面（Information Plane）分析——每层的 $(I(X;Z_l), I(Z_l;Y))$ 随训练的变化轨迹
+- **表示学习的理论指导**：为什么正则化（dropout、weight decay）有效——它们在隐式压缩冗余信息
+- **特征选择与降维**：在压缩率和预测性能之间寻找帕累托最优点
+
+## AI 设计翻译
+- **VIB 层 Variational Information Bottleneck**：编码器 $p_\theta(z|x)$ + KL 正则 + 解码器 $q_\phi(y|z)$，本质与 VAE 结构相同但目标语义不同（VAE 重构 $X$，VIB 预测 $Y$）
+- **$\beta$-VAE 的统一视角**：$\beta$-VAE 的目标函数与 VIB 形式一致，$\beta$ 即为 IB 拉格朗日乘子
+- **注意力稀疏化 / 路由的信息论解释**：Sparse Attention 和 MoE 路由可理解为隐式信息瓶颈——选择性地让"有用" token 通过，丢弃噪声
+
+## 工程可行性
+- **维度 1 张量化 ✅**：VIB 的编码器/解码器均为标准网络，$D_{KL}$ 为逐元素计算
+- **维度 2 GEMM 可映射 ✅**：主体计算为标准前向网络 + GEMM
+- **维度 3 复杂度 ✅**：相比原始网络仅增加 KL 项的 $O(d)$ 计算
+- **维度 4 显存 ⚠️**：需要额外维护先验分布 $r(z)$ 的参数和 KL 计算中间量
+- **维度 5 低精度 ✅**：重参数化技巧 + KL 解析解在 bf16 下稳定
+- **维度 8 算子融合 ✅**：与标准训练流程无冲突，可正常融合
+
+## 风险与失效条件
+- **$I(X;Z)$ 的精确估计困难**：高维连续变量间的互信息估计本身是开放问题（MINE、NWJ 等估计器方差大），实际用 VIB 变分下界绕过，但下界可能很松。
+- **$\beta$ 调参敏感**：$\beta$ 过大会过度压缩（欠拟合），过小则退化为标准 ERM（无压缩效果）。需要信息平面分析或自适应 $\beta$ 调度。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用 IB 蒸馏稿
+- Tishby, Pereira, Bialek. "The Information Bottleneck Method." *arXiv:physics/0004057*, 2000
+- Alemi, Poole, Fischer, Dillon, Suresh, Murphy. "Deep Variational Information Bottleneck." *ICLR*, 2017
+- Shwartz-Ziv, Tishby. "Opening the Black Box of Deep Neural Networks via Information." *arXiv:1703.00810*, 2017
+- 关联知识卡：`probability/entropy.md`、`probability/kl-divergence.md`

package/knowledge-base/probability/kl-divergence.en.md

@@ -0,0 +1,51 @@
+# KL Divergence
+
+## Minimal Definition
+Kullback-Leibler divergence measures the **information loss** when a probability distribution $q$ is used to approximate the true distribution $p$ — that is, the average number of extra bits spent when encoding $p$ using $q$. It is not a metric (it is asymmetric and does not satisfy the triangle inequality), but it defines a natural "directional distance" on the probability simplex.
+
+## Core Formulas
+
+**Definition**:
+$$D_{KL}(p \| q) = \sum_x p(x) \log \frac{p(x)}{q(x)} = \mathbb{E}_{p}\left[\log \frac{p(X)}{q(X)}\right]$$
+
+**Continuous version**:
+$$D_{KL}(p \| q) = \int p(x) \log \frac{p(x)}{q(x)}\, dx$$
+
+**Fundamental properties**:
+- $D_{KL}(p \| q) \geq 0$ (Gibbs' inequality), with equality if and only if $p = q$
+- **Asymmetric**: $D_{KL}(p \| q) \neq D_{KL}(q \| p)$, hence it is not a metric
+
+**Relationship to cross-entropy and entropy**:
+$$D_{KL}(p \| q) = H(p, q) - H(p)$$
+
+**Semantic difference between the two directions**:
+- **Forward KL** $D_{KL}(p \| q)$: $q$ tends to cover all modes of $p$ (mean-seeking)
+- **Reverse KL** $D_{KL}(q \| p)$: $q$ tends to lock onto a single mode of $p$ (mode-seeking)
+
+## Applicable Problems
+- **Variational inference**: Minimize the reverse KL $D_{KL}(q \| p)$ to find an approximate posterior distribution
+- **Knowledge distillation**: Minimize the information loss from the teacher distribution $p$ to the student distribution $q$
+- **Regularization**: Constrain the model distribution to remain close to a prior (e.g., the KL regularization term in VAEs)
+
+## AI Design Translation
+- **Knowledge Distillation Loss**: $\mathcal{L} = (1-\alpha) \cdot CE(y, q_s) + \alpha \cdot T^2 \cdot D_{KL}(p_t \| q_s)$, where $T$ is the temperature parameter
+- **VAE Regularization Term**: $D_{KL}(q_\phi(z|x) \| p(z))$, typically with $p(z) = \mathcal{N}(0, I)$, which admits an analytical solution
+- **PPO / RLHF**: $D_{KL}(\pi_\theta \| \pi_{\text{ref}})$ serves as a penalty term for the policy deviating from the reference policy
+
+## Engineering Feasibility
+- **Dimension 1 Tensorization ✅**: Element-wise $p \log(p/q)$ is fully vectorizable
+- **Dimension 2 GEMM-mappability ⚠️**: KL itself is not a GEMM, but its inputs (logits) come from GEMM layers
+- **Dimension 3 Complexity ✅**: $O(|\mathcal{X}|)$ linear
+- **Dimension 4 Memory ⚠️**: For large vocabularies, both $p$ and $q$ probability vectors must be held simultaneously; chunked computation is possible
+- **Dimension 5 Low Precision ✅**: Log-softmax differences are stable in bf16; note that $\log q$ diverges as $q \to 0$, requiring clamping
+- **Dimension 8 Operator Fusion ✅**: Can be fused with softmax into FusedKLDivLoss
+
+## Risks and Failure Conditions
+- **KL diverges to infinity when $q(x)=0$ but $p(x)>0$**: In practice, label smoothing or temperature scaling must be applied to $q$ to avoid zero probabilities. The mode-seeking behavior of reverse KL can exacerbate this issue — the student model "drops" low-probability regions of the teacher distribution.
+- **High gradient variance**: In RL (PPO/RLHF), KL estimation relies on sampling; high variance can lead to training instability. A clipped + linear approximation $\mathbb{E}[\log p - \log q]$ is commonly used in place of the exact KL.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated information theory distillation draft at present
+- Cover & Thomas. *Elements of Information Theory*, Ch. 2-3. Wiley, 2006
+- Murphy. *Probabilistic Machine Learning: Advanced Topics*, Ch. 6. MIT Press, 2023
+- Related knowledge cards: `probability/entropy.md`, `probability/information-bottleneck.md`, `probability/fisher-information.md`

package/knowledge-base/probability/kl-divergence.md

@@ -0,0 +1,51 @@
+# KL 散度 / KL Divergence
+
+## 最小定义
+Kullback-Leibler 散度度量一个概率分布 $q$ 相对于真实分布 $p$ 的**信息损失**——即用 $q$ 编码 $p$ 时多花的平均比特数。它不是度量（不对称、不满足三角不等式），但在概率单纯形上定义了自然的"方向性距离"。
+
+## 核心公式
+
+**定义**：
+$$D_{KL}(p \| q) = \sum_x p(x) \log \frac{p(x)}{q(x)} = \mathbb{E}_{p}\left[\log \frac{p(X)}{q(X)}\right]$$
+
+**连续版本**：
+$$D_{KL}(p \| q) = \int p(x) \log \frac{p(x)}{q(x)}\, dx$$
+
+**基本性质**：
+- $D_{KL}(p \| q) \geq 0$（Gibbs 不等式），等号当且仅当 $p = q$
+- **不对称**：$D_{KL}(p \| q) \neq D_{KL}(q \| p)$，因此不是度量
+
+**与交叉熵/熵的关系**：
+$$D_{KL}(p \| q) = H(p, q) - H(p)$$
+
+**两种方向的语义差异**：
+- **前向 KL** $D_{KL}(p \| q)$：$q$ 倾向于覆盖 $p$ 的所有模式（mean-seeking）
+- **反向 KL** $D_{KL}(q \| p)$：$q$ 倾向于锁定 $p$ 的某一个模式（mode-seeking）
+
+## 适用问题
+- **变分推断**：最小化反向 KL $D_{KL}(q \| p)$ 以寻找近似后验分布
+- **知识蒸馏**：教师分布 $p$ 到学生分布 $q$ 的信息损失最小化
+- **正则化**：约束模型分布不偏离先验太远（VAE 中的 KL 正则项）
+
+## AI 设计翻译
+- **知识蒸馏 Loss**：$\mathcal{L} = (1-\alpha) \cdot CE(y, q_s) + \alpha \cdot T^2 \cdot D_{KL}(p_t \| q_s)$，其中 $T$ 为温度参数
+- **VAE 正则项**：$D_{KL}(q_\phi(z|x) \| p(z))$，通常取 $p(z) = \mathcal{N}(0, I)$，解析可算
+- **PPO / RLHF**：$D_{KL}(\pi_\theta \| \pi_{\text{ref}})$ 作为策略偏离参考策略的惩罚项
+
+## 工程可行性
+- **维度 1 张量化 ✅**：逐元素 $p \log(p/q)$ 完全向量化
+- **维度 2 GEMM 可映射 ⚠️**：KL 本身不是 GEMM，但输入（logits）来自 GEMM 层
+- **维度 3 复杂度 ✅**：$O(|\mathcal{X}|)$ 线性
+- **维度 4 显存 ⚠️**：大 vocab 下需同时保留 $p$ 和 $q$ 的完整概率向量，可 chunk 计算
+- **维度 5 低精度 ✅**：log-softmax 差值在 bf16 下稳定；注意 $q \to 0$ 时 $\log q$ 发散，需 clamp
+- **维度 8 算子融合 ✅**：可与 softmax 融合为 FusedKLDivLoss
+
+## 风险与失效条件
+- **$q(x)=0$ 但 $p(x)>0$ 时 KL 发散为无穷**：实践中必须对 $q$ 做 label smoothing 或温度缩放，避免零概率。反向 KL 的 mode-seeking 行为可加剧此问题——学生模型"丢弃"教师分布的低概率区域。
+- **梯度方差大**：在 RL（PPO/RLHF）中，KL 估计依赖采样，高方差可导致训练不稳定。常用 clip + 线性近似 $\mathbb{E}[\log p - \log q]$ 替代精确 KL。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息论蒸馏稿
+- Cover & Thomas. *Elements of Information Theory*, Ch. 2-3. Wiley, 2006
+- Murphy. *Probabilistic Machine Learning: Advanced Topics*, Ch. 6. MIT Press, 2023
+- 关联知识卡：`probability/entropy.md`、`probability/information-bottleneck.md`、`probability/fisher-information.md`

package/knowledge-base/topology/euler-characteristic.en.md

@@ -0,0 +1,54 @@
+# Euler Characteristic
+
+## Minimal Definition
+
+The Euler characteristic $\chi$ is the most fundamental integer invariant of a topological space, defined as the alternating sum of the number of cells (or simplices) of each dimension: $\chi = \sum_{k=0}^d (-1)^k c_k$. It is invariant under continuous deformation and equals the alternating sum of Betti numbers $\chi = \sum_k (-1)^k \beta_k$.
+
+## Core Formulas
+
+- Cell formula: $\chi = c_0 - c_1 + c_2 - c_3 + \cdots = \sum_{k=0}^d (-1)^k c_k$
+- Betti number formula: $\chi = \beta_0 - \beta_1 + \beta_2 - \cdots = \sum_{k=0}^d (-1)^k \beta_k$
+- Gauss-Bonnet theorem: $\chi(M) = \frac{1}{2\pi} \int_M K \, dA$ (for 2-dimensional surfaces), connecting topology and curvature
+- Higher-dimensional Gauss-Bonnet-Chern theorem: $\chi(M^{2n}) = \int_M \text{Pf}(\Omega / 2\pi)$
+- Product formula: $\chi(X \times Y) = \chi(X) \cdot \chi(Y)$
+- Common values: $\chi(S^2) = 2$, $\chi(T^2) = 0$, $\chi(\text{genus-}g) = 2 - 2g$
+
+## Applicable Problems
+
+- Rapid topological diagnostics: a single integer distinguishes spheres from tori from high-genus surfaces
+- Mesh/graph quality inspection: the $\chi$ of a triangle mesh should be 2 (sphere-homeomorphic); anomalous values indicate topological errors
+- Loss landscape analysis: in Morse theory of critical points, $\chi$ constrains the number and type of critical points
+- Quick summary of persistent homology: $\chi = \sum (-1)^k \beta_k$ can be quickly computed from the persistence diagram
+
+## AI Design Translation
+
+- **Topological diagnostic metric**: Monitor the $\chi$ of the latent/feature space during training; sudden changes in $\chi$ indicate topological phase transitions (e.g., mode collapse)
+- **Gauss-Bonnet regularization**: $\int K \, dA$ can be approximated by the trace of the Hessian, using curvature integrals as a regularization term to constrain loss landscape topology
+- **Mesh quality loss**: For 3D generative models, penalize $\chi \neq \chi_{\text{target}}$ to ensure topological correctness of generated meshes
+- **Morse-theoretic critical point analysis**: $\chi = \sum (-1)^{\text{index}} (\text{number of critical points})$, using critical point indices to diagnose optimization landscape
+- **Euler characteristic curve**: $\chi(\epsilon) = \chi(VR_\epsilon)$ as a function of scale $\epsilon$, providing richer information than a single $\chi$ value
+
+## Engineering Feasibility
+
+High GPU friendliness. Computing the Euler characteristic is extremely cheap:
+- **Cell counting**: $c_k$ is an integer count, $O(n)$ summation, perfectly GPU-friendly
+- **From Betti numbers**: $\chi = \sum (-1)^k \beta_k$; if Betti numbers are already available, $O(d)$ summation
+- **From persistence diagram**: $\beta_k(\epsilon) = |\{(b,d) \in D_k \mid b \leq \epsilon < d\}|$, a counting operation, $O(|D_k|)$
+- **Gauss-Bonnet integral**: For 2D surfaces $\int K \, dA \approx \sum K_i A_i$, per-face summation, parallelizable
+- **Euler characteristic curve**: Scanning $\chi(\epsilon)$ along $\epsilon$, implementable with sorting + cumulative sum, $O(n \log n)$
+- Overall complexity: linear or sub-quadratic, fully computable in real time during the training loop
+
+## Risks and Failure Conditions
+
+- **Extreme information compression**: $\chi$ is a single integer; many distinct topological spaces share the same $\chi$ value ($\chi = 0$ can correspond to a torus, Klein bottle, etc.)
+- **Sensitivity to noise**: Small perturbations of a point cloud can add/remove simplices, changing $c_k$ and hence $\chi$; should be combined with scale analysis from persistent homology
+- **Discrete approximation error in Gauss-Bonnet**: Discrete curvature definitions are not unique; different discretizations yield different $\chi$ estimates
+- **Degeneracy in high dimensions**: The Euler characteristic of odd-dimensional closed manifolds is $\chi = 0$, losing discriminative power; high-dimensional Betti numbers are expensive to compute
+- **Captures only the global, not the local**: $\chi$ is a global invariant; local topological changes may cancel out
+
+## Further References
+
+- Distillation notes: references/books/smooth-manifolds.md (Ch 17--18 De Rham Cohomology, Betti numbers and cohomology)
+- Distillation notes: references/books/differential-geometry.md (Ch 4 Curves and Hypersurfaces, intuition source for Gauss curvature)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 17--18 (de Rham cohomology and topological invariants)
+- Extended reading: Hatcher, *Algebraic Topology*, Ch 2 (standard treatment of simplicial homology and the Euler characteristic)

package/knowledge-base/topology/euler-characteristic.md

@@ -0,0 +1,54 @@
+# 欧拉示性数 (Euler Characteristic)
+
+## 最小定义
+
+欧拉示性数 $\chi$ 是拓扑空间最基本的整数不变量，定义为各维胞腔（或单纯形）数的交错和：$\chi = \sum_{k=0}^d (-1)^k c_k$。它在连续变形下不变，等于各阶 Betti 数的交错和 $\chi = \sum_k (-1)^k \beta_k$。
+
+## 核心公式
+
+- 胞腔公式：$\chi = c_0 - c_1 + c_2 - c_3 + \cdots = \sum_{k=0}^d (-1)^k c_k$
+- Betti 数公式：$\chi = \beta_0 - \beta_1 + \beta_2 - \cdots = \sum_{k=0}^d (-1)^k \beta_k$
+- Gauss-Bonnet 定理：$\chi(M) = \frac{1}{2\pi} \int_M K \, dA$（2 维曲面），连接拓扑与曲率
+- 高维 Gauss-Bonnet-Chern：$\chi(M^{2n}) = \int_M \text{Pf}(\Omega / 2\pi)$
+- 乘积公式：$\chi(X \times Y) = \chi(X) \cdot \chi(Y)$
+- 常见值：$\chi(S^2) = 2$，$\chi(T^2) = 0$，$\chi(\text{genus-}g) = 2 - 2g$
+
+## 适用问题
+
+- 快速拓扑诊断：一个整数即可区分球面 vs 环面 vs 高亏格曲面
+- 网格/图的质量检验：三角形网格的 $\chi$ 应为 2（球面同胚），异常值指示拓扑错误
+- 损失地形分析：临界点的 Morse 理论中 $\chi$ 约束临界点的数量和类型
+- 持续同调的快速摘要：$\chi = \sum (-1)^k \beta_k$ 可从持续图快速算出
+
+## AI 设计翻译
+
+- **拓扑诊断指标**：训练过程中监控隐空间/特征空间的 $\chi$，$\chi$ 突变指示拓扑相变（如模式坍塌）
+- **Gauss-Bonnet 正则化**：$\int K \, dA$ 可用 Hessian 的迹近似，将曲率积分作为正则项约束 loss landscape 拓扑
+- **网格质量损失**：对 3D 生成模型，惩罚 $\chi \neq \chi_{\text{target}}$ 确保生成网格的拓扑正确
+- **Morse 理论的临界点分析**：$\chi = \sum (-1)^{\text{index}} (\text{临界点数})$，用临界点指标诊断优化地形
+- **Euler characteristic curve**：$\chi(\epsilon) = \chi(VR_\epsilon)$ 作为尺度 $\epsilon$ 的函数，比单一 $\chi$ 信息更丰富
+
+## 工程可行性
+
+GPU 友好度高。欧拉示性数的计算极其廉价：
+- **胞腔计数**：$c_k$ 是整数计数，$O(n)$ 求和，完美 GPU 友好
+- **从 Betti 数计算**：$\chi = \sum (-1)^k \beta_k$，若已有 Betti 数则 $O(d)$ 求和
+- **从持续图计算**：$\beta_k(\epsilon) = |\{(b,d) \in D_k \mid b \leq \epsilon < d\}|$，计数操作，$O(|D_k|)$
+- **Gauss-Bonnet 积分**：2 维曲面 $\int K \, dA \approx \sum K_i A_i$，逐面片求和，可并行
+- **Euler characteristic curve**：沿 $\epsilon$ 扫描计算 $\chi(\epsilon)$，可用排序 + 累积和实现，$O(n \log n)$
+- 整体复杂度：线性或亚二次，完全可在训练循环中实时计算
+
+## 风险与失效条件
+
+- **信息极度压缩**：$\chi$ 是单一整数，大量不同拓扑空间共享同一 $\chi$ 值（$\chi = 0$ 可以是环面、Klein 瓶等）
+- **对噪声敏感**：点云的小扰动可能添加/删除单纯形，改变 $c_k$ 从而改变 $\chi$；需配合持续同调的尺度分析
+- **Gauss-Bonnet 的离散近似误差**：离散曲率的定义不唯一，不同离散化给出不同的 $\chi$ 估计
+- **高维退化**：奇数维闭流形的 $\chi = 0$，失去区分能力；高维 Betti 数计算昂贵
+- **只捕捉"整体"不捕捉"局部"**：$\chi$ 是全局不变量，局部拓扑变化可能被抵消
+
+## 深入参考
+
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 17-18 De Rham Cohomology，Betti 数与上同调）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 4 Curves and Hypersurfaces, Gauss curvature 直觉来源）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 17-18（de Rham 上同调与拓扑不变量）
+- 延伸：Hatcher, *Algebraic Topology*, Ch 2（单纯同调与欧拉示性数的标准处理）

package/knowledge-base/topology/fundamental-group.en.md

@@ -0,0 +1,52 @@
+# Fundamental Group
+
+## Minimal Definition
+
+The fundamental group $\pi_1(X, x_0)$ is the group of equivalence classes of loops in a topological space $X$ based at a point $x_0$, under homotopy equivalence. The group operation is loop concatenation (first traverse $\alpha$, then $\beta$). It is the first homotopy group, detecting "non-contractible holes" in the space -- whether loops can be continuously shrunk to a point.
+
+## Core Formulas
+
+- Fundamental group: $\pi_1(X, x_0) = \{[\gamma] \mid \gamma: [0,1] \to X, \gamma(0)=\gamma(1)=x_0\}$
+- Group operation: $[\alpha] \cdot [\beta] = [\alpha * \beta]$, $(\alpha * \beta)(t) = \begin{cases} \alpha(2t) & t \leq 1/2 \\ \beta(2t-1) & t \geq 1/2 \end{cases}$
+- Simply connected: $\pi_1(X) = \{e\}$ (all loops can be contracted to a point)
+- Common fundamental groups: $\pi_1(S^1) = \mathbb{Z}$, $\pi_1(T^2) = \mathbb{Z}^2$, $\pi_1(S^n) = 0 \, (n \geq 2)$
+- Covering space correspondence: subgroups of $\pi_1$ $\leftrightarrow$ covering spaces (Galois correspondence)
+- Seifert-van Kampen theorem: $\pi_1(X) = \pi_1(U) *_{\pi_1(U \cap V)} \pi_1(V)$ (computation via cover decomposition)
+
+## Applicable Problems
+
+- Detecting "holes" in a space: $\pi_1 \neq 0$ implies the existence of non-contractible loops
+- Covering spaces and multi-valued functions: subgroups of $\pi_1$ determine the Riemann surface structure of multi-valued functions
+- Robotic path planning: $\pi_1$ of the configuration space determines the number of topological equivalence classes of paths
+- Molecular configuration analysis: the topology of the configuration space of cyclic molecules affects conformational sampling
+
+## AI Design Translation
+
+- **Topological regularization (non-contractible loop penalty)**: In the latent space, if the data has a loop-like structure (e.g., periodic signals), penalize the collapse of $\pi_1$ to maintain loop connectivity in the latent space
+- **Covering-space-inspired multi-hypothesis tracking**: Non-trivial subgroups of $\pi_1$ correspond to multiple "covers," which can be used for parallel multi-hypothesis tracking in multi-modal prediction
+- **Path equivalence class classifier**: In robotics/motion planning, label paths of different topological classes using elements of $\pi_1$, training a classifier to distinguish path topology types
+- **Simply-connected regularization**: Enforce simple connectivity of the latent space ($\pi_1 = 0$), ensuring that interpolation paths between any two points can be continuously deformed, preventing "tearing" in the latent space
+
+## Engineering Feasibility
+
+Low GPU friendliness. Computation of the fundamental group is inherently combinatorial/algebraic rather than numerically linear:
+- **Computing the fundamental group itself**: For a simplicial complex, $\pi_1$ is isomorphic to the edge group modulo face relations (Wirtinger presentation); requires a group presentation, **not tensorizable**
+- **Loop detection**: Finding non-contractible loops on a graph/mesh is equivalent to finding non-trivial cycles, solvable via BFS/DFS, $O(V+E)$, but unsuitable for GPU parallelism (data-dependent graph traversal)
+- **Differentiable alternatives**: Convert "loop contractibility" into a differentiable proxy -- such as loop integrals $\oint \omega$ (de Rham cohomology); a non-zero closed-form integral implies non-contractibility, which is differentiable
+- **Numerical implementation of covering spaces**: For known group structures (e.g., $\mathbb{Z}^n$), covering spaces can be explicitly parameterized as periodic identifications, which is tensorizable
+- Overall assessment: exact computation of $\pi_1$ belongs to the "beautiful but incomputable" category; differentiable proxies (loop integrals/cohomology) are required
+
+## Risks and Failure Conditions
+
+- **Not tensorizable**: The group presentation of $\pi_1$ is symbolic computation with no GEMM representation; it cannot be directly inserted into a training forward pass
+- **Base-point dependence**: $\pi_1(X, x_0)$ is independent of the base point (up to isomorphism) in path-connected spaces, but different components of disconnected spaces have different $\pi_1$
+- **Non-Abelianness**: $\pi_1$ is generally a non-Abelian group (e.g., free groups); the group operation is non-commutative, increasing computational and representational complexity
+- **High-dimensional blind spot**: $\pi_1$ only detects 1-dimensional holes (non-contractible loops); higher-dimensional holes (e.g., "voids" in $S^2$) require higher homotopy groups $\pi_k$ ($k \geq 2$)
+- **Robustness of numerical loop detection**: The definition of a "loop" in discretized/sampled point clouds is ambiguous and requires scale analysis from persistent homology
+
+## Further References
+
+- Distillation notes: references/books/smooth-manifolds.md (Ch 17--18 De Rham Cohomology, cohomology and topological invariants)
+- Distillation notes: references/books/algebraic-geometry-rising-sea.md (Section 18 Cech Cohomology, local gluing and global obstructions)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 17 (de Rham cohomology, the loop integral perspective)
+- Extended reading: Hatcher, *Algebraic Topology*, Ch 1 (standard textbook treatment of the fundamental group, including the Seifert-van Kampen theorem)

package/knowledge-base/topology/fundamental-group.md

@@ -0,0 +1,52 @@
+# 基本群 (Fundamental Group)
+
+## 最小定义
+
+基本群 $\pi_1(X, x_0)$ 是拓扑空间 $X$ 中以基点 $x_0$ 为起终点的环路（loop）在同伦等价下的等价类构成的群。群运算是环路的拼接（先走 $\alpha$ 再走 $\beta$）。它是第一个同伦群，检测空间中"不可收缩的洞"——环路能否连续缩为一点。
+
+## 核心公式
+
+- 基本群：$\pi_1(X, x_0) = \{[\gamma] \mid \gamma: [0,1] \to X, \gamma(0)=\gamma(1)=x_0\}$
+- 群运算：$[\alpha] \cdot [\beta] = [\alpha * \beta]$，$(\alpha * \beta)(t) = \begin{cases} \alpha(2t) & t \leq 1/2 \\ \beta(2t-1) & t \geq 1/2 \end{cases}$
+- 单连通：$\pi_1(X) = \{e\}$（所有环路可缩为一点）
+- 常见基本群：$\pi_1(S^1) = \mathbb{Z}$，$\pi_1(T^2) = \mathbb{Z}^2$，$\pi_1(S^n) = 0 \, (n \geq 2)$
+- 覆盖空间对应：$\pi_1$ 的子群 $\leftrightarrow$ 覆盖空间（Galois 对应）
+- Seifert-van Kampen 定理：$\pi_1(X) = \pi_1(U) *_{\pi_1(U \cap V)} \pi_1(V)$（由覆盖拼接计算）
+
+## 适用问题
+
+- 检测空间中的"洞"：$\pi_1 \neq 0$ 意味着存在不可收缩的环路
+- 覆盖空间与多值函数：$\pi_1$ 的子群决定多值函数的黎曼面结构
+- 机器人路径规划：构型空间的 $\pi_1$ 决定路径的拓扑等价类数
+- 分子构型分析：环状分子的构型空间拓扑影响构象采样
+
+## AI 设计翻译
+
+- **拓扑正则化（环路不可缩惩罚）**：在隐空间中，如果数据有环状结构（如周期信号），惩罚 $\pi_1$ 的坍塌，保持隐空间的环路连通性
+- **覆盖空间启发的多假设跟踪**：$\pi_1$ 的非平凡子群对应多个"覆盖"，可用于多模态预测中的多假设并行跟踪
+- **路径等价类的分类器**：在机器人/运动规划中，用 $\pi_1$ 的元素标记不同拓扑类的路径，训练分类器区分路径的拓扑类型
+- **单连通正则化**：强制隐空间为单连通（$\pi_1 = 0$），确保任意两点间的插值路径可连续变形，避免隐空间"撕裂"
+
+## 工程可行性
+
+GPU 友好度低。基本群的计算本质上是组合/代数的，而非数值线性的：
+- **基本群本身的计算**：对于单纯复形，$\pi_1$ 同构于边的群模去面的关系（Wirtinger 表示），需要群的展示（presentation），**不可张量化**
+- **环路检测**：在图/网格上找不可缩环路等价于找非平凡循环，可用 BFS/DFS，$O(V+E)$，但不适合 GPU 并行（数据依赖的图遍历）
+- **可微替代**：将"环路可缩性"转化为可微代理——如环路积分 $\oint \omega$（de Rham 上同调），闭形式积分非零即不可缩，这是可微的
+- **覆盖空间的数值实现**：对于已知群结构（如 $\mathbb{Z}^n$），覆盖空间可以显式参数化为周期性拼接，可张量化
+- 整体评估：精确计算 $\pi_1$ 属于"美但不可算"类别，需要可微代理（环路积分/上同调）替代
+
+## 风险与失效条件
+
+- **不可张量化**：$\pi_1$ 的群展示是符号计算，无 GEMM 表达，不能直接塞进训练 forward
+- **基点依赖**：$\pi_1(X, x_0)$ 在路径连通空间中不依赖基点选择（同构），但不连通空间中不同分量有不同的 $\pi_1$
+- **非阿贝尔性**：$\pi_1$ 一般是非阿贝尔群（如自由群），群运算不可交换，增加了计算和表示的复杂度
+- **高维盲区**：$\pi_1$ 只检测 1 维洞（不可缩环路），高维洞（如 $S^2$ 的"空腔"）需要高阶同伦群 $\pi_k (k \geq 2)$
+- **数值环路检测的鲁棒性**：离散化/采样的点云中"环路"的定义模糊，需要配合持续同调的尺度分析
+
+## 深入参考
+
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 17-18 De Rham Cohomology，上同调与拓扑不变量）
+- 蒸馏稿：references/books/algebraic-geometry-rising-sea.md（§18 Cech Cohomology，局部黏合与全局障碍）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 17（de Rham 上同调，环路积分的视角）
+- 延伸：Hatcher, *Algebraic Topology*, Ch 1（基本群的标准教材处理，含 Seifert-van Kampen 定理）