math-skill 2.1.0 → 3.0.1

package/design-patterns/routing/spectral-clustering-routing.md

@@ -0,0 +1,90 @@
+# Spectral Clustering Routing（谱聚类路由）
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行，需工程验证。
+
+## 适用问题
+当需要基于 token/样本的内在相似性结构进行分组路由时使用。典型场景：
+(1) 无监督专家分配——没有路由标签时，用谱聚类自动发现 token 的自然分簇；
+(2) 自适应专家初始化——训练初期用谱聚类结果初始化专家参数；
+(3) 输入感知的动态聚类——不同 batch 的 token 分布不同，路由应自适应；
+(4) 多粒度聚类——不同层使用不同粒度的谱聚类（粗→细）。
+核心诉求：**发现数据内在的簇结构，用于路由或专家初始化**。
+
+## 数学思想来源
+- 透镜：lenses/geometric.md（谱图论、拉普拉斯特征映射）、lenses/variational.md（松弛与近似）
+- 知识：knowledge-base/matrix-analysis/projection.md（特征值分解、Rayleigh 商）、
+  knowledge-base/differential-geometry/manifold.md（流形学习、图割）
+
+## 需要的数学知识
+- **谱聚类 (Ng-Jordan-Weiss)**：
+  1. 构造相似度图 W_{ij} = exp(-‖x_i - x_j‖² / 2σ²)
+  2. 计算归一化拉普拉斯 L_sym = I - D^{-1/2} W D^{-1/2}
+  3. 取前 k 个最小特征向量 U_k ∈ R^{N×k}
+  4. 对 U_k 的行做 k-means 得到 k 个簇
+- **Nyström 近似**：当 N 太大无法计算完整 W 时，采样 m<<N 个点
+  W ≈ C · W_m^{-1} · C^T，将特征分解降维到 m×m
+- **谱松弛连续化**：离散聚类分配 → 连续特征向量 → 可微路由
+  用 softmax(U_k · W_proj) 替代硬 k-means 分配
+- **幂迭代加速**：不需完整特征分解，只需前 k 个特征向量
+  用 Lanczos/Arnoldi 迭代 O(N²·k·iter) 或 randomized SVD O(N·k·log k)
+
+## AI 模块形式
+```
+模块：SpectralClusterRouter
+输入：X ∈ R^{N×d}，簇数 K
+
+方法1 - 在线谱聚类路由（训练时周期性更新）：
+  // 每 M 步更新一次聚类中心，推理时用最近邻
+  W = exp(-cdist(X_sample, X_sample) / (2σ²))  // m×m 采样相似度
+  L = I - D^{-1/2} W D^{-1/2}                    // 归一化拉普拉斯
+  U_k = eigsh(L, k=K, which='SM')                // 前 K 个最小特征向量
+  centers = kmeans(U_k, K)                        // K 个聚类中心
+  // 路由：将新 token 投影到谱空间后分配
+  proj = X @ W_proj                               // N→K 维投影（可学习）
+  assignment = argmin(cdist(proj, centers))        // 最近中心分配
+
+方法2 - 可微谱路由（端到端）：
+  // 用 softmax 松弛替代硬分配
+  sim_matrix = X @ X^T                             // N×N（或采样 m×m）
+  A = exp(sim_matrix / τ)                          // 相似度图（可学习 τ）
+  D_inv_sqrt = diag(1 / sqrt(sum(A, dim=1) + ε))
+  L_norm = I - D_inv_sqrt @ A @ D_inv_sqrt         // 归一化拉普拉斯
+  // 近似前 K 个特征向量（幂迭代 + 正交化）
+  U_k = power_iteration_approx(L_norm, K, steps=5)  // N×K
+  // 软分配
+  cluster_logits = U_k @ W_cluster                  // N×K（可学习投影）
+  route_probs = softmax(cluster_logits / τ_route)    // 软路由概率
+
+方法3 - 锚点谱聚类（大规模）：
+  anchors = kmeans_pp(X, m)                         // m 个锚点，m << N
+  Z = exp(-cdist(X, anchors) / (2σ²))               // N×m 亲和矩阵
+  L_anchor = I - D_z^{-1/2} Z^T Z D_z^{-1/2}        // m×m 拉普拉斯
+  U_k = eigsh(L_anchor, K)                          // m×K 特征向量
+  route = Z @ U_k @ W_proj                           // N×K 路由分数
+```
+
+## 可实现结构
+- **周期性离线聚类**：每 N_step 步收集 token 表示 → 离线谱聚类 → 更新路由表
+- **Nyström 采样**：随机采样 m=1024 个代表点，将 N×N 问题降为 m×m
+- **幂迭代实现**：5~10 步幂迭代 + Gram-Schmidt 正交化，GPU 友好
+- **渐进式训练**：初期用 k-means 粗路由 → 中期谱聚类精化 → 后期可微调路由网络
+
+## GPU 可行性
+- **张量化**：相似度矩阵 X@X^T 为 GEMM；拉普拉斯构造为 element-wise + 对角矩阵
+- **GEMM 可映射**：方法3 的 Z^T@Z 为 GEMM (m×N)@(N×m)；Z@U_k 为 GEMM (N×m)@(m×K)
+- **复杂度**：完整谱聚类 O(N²·K) 不可扩展；Nyström O(N·m·K+m³)；幂迭代 O(N²·K·T)
+- **显存与 KV-Cache**：N×N 相似度矩阵在 N>4096 时 >64MB，必须采样降维
+- **低精度稳定**：特征分解建议 fp32；exp(-dist/σ²) 在 fp16 下需 clip distance
+- **并行与通信**：幂迭代的 matvec 高度并行；k-means 的 assign+update 可批并行
+- **稀疏结构**：k-NN 图替代全连接图，W 稀疏度 >95%，SpMM 加速
+- **算子融合**：D^{-1/2}@A@D^{-1/2} 的对角缩放可融合；cdist+exp+normalize 可融合
+
+## 论文表述方式
+"利用谱聚类的连续松弛实现可微路由：构造 token 相似度图的归一化拉普拉斯，
+通过 Nyström 近似将 O(N²) 特征分解降为 O(Nm+K³)，配合幂迭代实现 GPU 友好的
+在线谱聚类，聚类质量以 Normalized Cut 衡量保证 O(√(log N/K)) 的近似比。"
+
+## 风险
+- N×N 相似度矩阵的显存和计算在长序列下不可扩展，必须采样或 k-NN 稀疏化
+- 特征分解不可微（特征值重合时梯度未定义），端到端训练需松弛或 stop-gradient
+- 簇数 K 需先验指定，且 K 变化时需重新聚类
+- σ（带宽参数）对聚类质量敏感，过小导致孤立点，过大导致合并

package/knowledge-base/differential-geometry/connection.en.md

@@ -0,0 +1,51 @@
+# Connection
+
+## Minimal Definition
+
+A connection $\nabla$ is a rule specifying "how to transport a vector along a curve from one point to another without extraneous rotation." Formally, $\nabla: \mathfrak{X}(M) \times \mathfrak{X}(M) \to \mathfrak{X}(M)$ satisfies the Leibniz rule. The Levi-Civita connection is the unique connection that is metric-compatible ($\nabla g = 0$) and torsion-free ($T = 0$).
+
+## Core Formulas
+
+- Covariant derivative: $\nabla_X Y = \left(X^i \partial_i Y^k + X^i Y^j \Gamma^k_{ij}\right) \partial_k$
+- Christoffel symbols (Levi-Civita): $\Gamma^k_{ij} = \frac{1}{2} g^{kl}(\partial_i g_{jl} + \partial_j g_{il} - \partial_l g_{ij})$
+- Parallel transport equation: $\frac{D V^k}{dt} = \dot V^k + \Gamma^k_{ij} \dot\gamma^i V^j = 0$
+- Connection form (on a principal bundle): $\omega \in \Omega^1(P, \mathfrak{g})$; the gauge field $A_\mu$ is a local connection form
+- Curvature = non-commutativity of the connection: $R(X,Y) = [\nabla_X, \nabla_Y] - \nabla_{[X,Y]}$
+
+## Applicable Problems
+
+- Cross-point vector comparison: tangent spaces at different points cannot be directly summed; a connection specifies the "transport rule"
+- Momentum/state transport in optimization: historical gradients in Riemannian Adam must be carried across steps via parallel transport
+- Gauge-equivariant networks: the freedom in choosing local coordinate frames (gauges) is aligned by the connection
+- Physically constrained systems: the electromagnetic field equals the curvature of a U(1) connection; Yang-Mills equals the curvature of a non-Abelian connection
+
+## AI Design Translation
+
+- **Vector transport module**: In Riemannian optimizers, transport the momentum $m_k \in T_{x_k}M$ to $T_{x_{k+1}}M$; closed-form transport (e.g., projection on the Stiefel manifold) can be expressed as GEMM
+- **Gauge-equivariant CNN**: On manifolds/meshes, each edge carries a $G$-connection element that aligns the local frames of adjacent points, making convolution kernels invariant to local coordinate choices
+- **Parallel transport regularization**: Penalize non-parallelism of the feature field under the connection $\|\nabla_X f\|^2$, enforcing smooth feature variation along the manifold
+- **Connection learning parameterization**: Parameterize Christoffel symbols as neural network outputs, learning the "optimal transport rule" on the data manifold
+
+## Engineering Feasibility
+
+GPU friendliness: the core challenge of connections is "serial ODE integration."
+- **Closed-form parallel transport** (specific manifolds such as SO(3), Stiefel): single-step matrix operations, batchable, GPU-friendly
+- **Parallel transport for general connections**: integrating $\dot{V} + \Gamma \dot\gamma V = 0$ along a curve is a serial ODE with poor parallelism
+- **Christoffel symbol computation**: involves partial derivatives of the metric $g$ and $g^{-1}$; if $g$ has a closed form, $O(n^3)$; otherwise more expensive
+- **Connections in gauge-equivariant CNNs**: one $G$-element action per edge (matrix-times-feature-vector), expressible as sparse matmul or batched small GEMM
+- Key adaptation: use a single-step retraction/closed-form transport instead of step-by-step ODE integration
+
+## Risks and Failure Conditions
+
+- **Step-by-step ODE integration for parallel transport**: Serial recurrence kills parallelism; closed-form transport or single-step approximations must be used
+- **Inconsistent left/right connection conventions**: Mixing left-invariant and right-invariant connections leads to misaligned gradients
+- **Improper handling of gauge freedom**: If the connection parameterization in gauge-equivariant networks is incomplete, equivariance silently breaks
+- **Numerical derivatives of Christoffel symbols**: Finite-difference estimation of $\partial_i g_{jk}$ is noisy; analytical formulas or autodiff are preferable
+- **Connection does not imply metric compatibility**: A connection need not admit a compatible metric (non-metric connections); incorrectly assuming compatibility leads to inconsistencies
+
+## Further References
+
+- Distillation notes: references/books/differential-geometry.md (Ch 12 Connections and Covariant Derivatives, Section 12.2 Connection Forms, Section 12.4 Ehresmann, Section 12.12 G-Connections)
+- Distillation notes: references/books/differential-geometry.md (Section 6.8 Principal Bundles, Section 9.8 Electromagnetism)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 12 (Section 12.1--Section 12.12, complete connection theory)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 13.1 Levi-Civita Connection

package/knowledge-base/differential-geometry/connection.md

@@ -0,0 +1,51 @@
+# 联络 (Connection)
+
+## 最小定义
+
+联络 $\nabla$ 是流形上"如何把一个向量沿曲线搬到另一点而不额外旋转"的规则。形式化地，$\nabla: \mathfrak{X}(M) \times \mathfrak{X}(M) \to \mathfrak{X}(M)$ 满足 Leibniz 法则。Levi-Civita 联络是唯一与度量相容（$\nabla g = 0$）且无挠（$T = 0$）的联络。
+
+## 核心公式
+
+- 协变导数：$\nabla_X Y = \left(X^i \partial_i Y^k + X^i Y^j \Gamma^k_{ij}\right) \partial_k$
+- Christoffel 符号（Levi-Civita）：$\Gamma^k_{ij} = \frac{1}{2} g^{kl}(\partial_i g_{jl} + \partial_j g_{il} - \partial_l g_{ij})$
+- 平行移动方程：$\frac{D V^k}{dt} = \dot V^k + \Gamma^k_{ij} \dot\gamma^i V^j = 0$
+- 联络形式（主丛）：$\omega \in \Omega^1(P, \mathfrak{g})$，规范场 $A_\mu$ 即局部联络形式
+- 曲率 = 联络的不交换性：$R(X,Y) = [\nabla_X, \nabla_Y] - \nabla_{[X,Y]}$
+
+## 适用问题
+
+- 跨点向量比较：不同点的切空间无法直接相加，需要联络规定的"搬运规则"
+- 优化中的动量/状态搬运：Riemannian Adam 中历史梯度需要通过 parallel transport 跨步搬运
+- 规范等变网络：局部坐标系（gauge）的选择自由由联络对齐
+- 物理约束系统：电磁场 = U(1) 联络曲率，Yang-Mills = 非阿贝尔联络曲率
+
+## AI 设计翻译
+
+- **Vector transport 模块**：Riemannian 优化器中，将动量 $m_k \in T_{x_k}M$ 搬到 $T_{x_{k+1}}M$；闭式 transport（如 Stiefel 上的投影）可 GEMM 化
+- **Gauge-equivariant CNN**：在流形/网格上，每条边携带一个 $G$-联络元素对齐相邻点的局部 frame，使卷积核对局部坐标选择不变
+- **Parallel transport 正则化**：惩罚特征场在联络下的非平行性 $\|\nabla_X f\|^2$，强制特征沿流形平滑变化
+- **联络学习的参数化**：将 Christoffel 符号参数化为神经网络输出，学习数据流形上的"最优搬运规则"
+
+## 工程可行性
+
+GPU 友好度：联络的核心挑战是"串行 ODE 积分"。
+- **闭式 parallel transport**（特定流形如 SO(3)、Stiefel）：单步矩阵运算，可 batched，GPU 友好
+- **一般联络的平行移动**：沿曲线积分 $\dot{V} + \Gamma \dot\gamma V = 0$ 是串行 ODE，并行性差
+- **Christoffel 符号计算**：涉及度量 $g$ 的偏导和 $g^{-1}$，若 $g$ 有闭式则 $O(n^3)$，否则更贵
+- **规范等变 CNN 中的联络**：每边一个 $G$-元素作用（矩阵乘特征向量），可表达为 sparse matmul 或 batched small GEMM
+- 关键改造：用一步 retraction/closed-form transport 替代逐步 ODE 积分
+
+## 风险与失效条件
+
+- **逐步 ODE 积分做平行移动**：串行递推杀死并行度，必须用闭式 transport 或一步近似
+- **左右联络约定不统一**：左不变 vs 右不变联络的选择不一致导致梯度错位
+- **规范自由度未正确处理**：gauge-equivariant 网络中若联络参数化不完备，等变性会悄悄破缺
+- **Christoffel 符号的数值导数**：用有限差分估计 $\partial_i g_{jk}$ 时噪声大，最好有解析式或 autodiff
+- **联络 ≠ 度量**：有联络不一定有相容度量（非度量联络），错误假设相容性会导致不一致
+
+## 深入参考
+
+- 蒸馏稿：references/books/differential-geometry.md（Ch 12 Connections and Covariant Derivatives, §12.2 联络形式, §12.4 Ehresmann, §12.12 G-联络）
+- 蒸馏稿：references/books/differential-geometry.md（§6.8 Principal Bundles, §9.8 Electromagnetism）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 12（§12.1-§12.12 完整联络理论）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.1 Levi-Civita Connection

package/knowledge-base/differential-geometry/curvature.en.md

@@ -0,0 +1,49 @@
+# Curvature
+
+## Minimal Definition
+
+Curvature measures whether a manifold "bends" and whether "paths are path-dependent." The Riemann curvature tensor $R(X,Y)Z = \nabla_X\nabla_Y Z - \nabla_Y\nabla_X Z - \nabla_{[X,Y]}Z$ describes the deflection of a vector after parallel transport around an infinitesimal parallelogram. The sectional curvature $K(\sigma)$ is its simplest scalar extraction.
+
+## Core Formulas
+
+- Riemann curvature tensor: $R^l_{ijk} = \partial_i \Gamma^l_{jk} - \partial_j \Gamma^l_{ik} + \Gamma^l_{im}\Gamma^m_{jk} - \Gamma^l_{jm}\Gamma^m_{ik}$
+- Ricci curvature (contraction): $R_{ij} = \sum_k R^k_{ikj}$
+- Scalar curvature: $S = \sum_{ij} g^{ij} R_{ij}$
+- Sectional curvature: $K(X,Y) = \frac{\langle R(X,Y)Y, X\rangle}{\|X\|^2\|Y\|^2 - \langle X,Y\rangle^2}$
+- Jacobi equation: $\frac{D^2 J}{dt^2} + R(J, \dot\gamma)\dot\gamma = 0$ (describes divergence/convergence of geodesics)
+- Hessian-vector product: $Hv = \nabla(\nabla L \cdot v)$, $O(N)$ estimation of curvature information
+
+## Applicable Problems
+
+- Loss landscape analysis: curvature determines conditioning and sharpness, distinguishing sharp minima from flat minima
+- Optimization trajectory stability: Jacobi fields describe the divergence/convergence of neighboring optimization trajectories
+- Generalization diagnostics: flat minima (low curvature) tend to generalize better
+- Manifold learning: the curvature of the data manifold guides latent space dimension and metric selection
+
+## AI Design Translation
+
+- **Curvature regularization (geometric perspective on SAM)**: Use Hessian-vector products to estimate $\max_v v^T H v$, penalizing sharp minima and preferring flat minima
+- **HVP-based diagnostic**: $\|Hv\|/\|v\|$ as a cheap proxy for loss landscape curvature, used for learning rate adaptation and early stopping
+- **Jacobi field trajectory monitoring**: Track the distance evolution between two neighboring optimization trajectories, the discrete analog of $J''(t) + R(J,\dot\gamma)\dot\gamma = 0$, to detect divergence/convergence
+- **Ricci-flow-inspired graph rewiring**: Use discrete Ricci curvature to guide dynamic adjustment of graph/attention structure (negative-curvature edges indicate bottlenecks that need additional connections)
+
+## Engineering Feasibility
+
+GPU friendliness: the core difficulty of curvature is that "the full tensor cannot be materialized."
+- **Riemann tensor**: 4th order, $n^4$ components, materializing it exhausts memory -- **explicit computation is prohibited**
+- **Hessian-vector product (HVP)**: Via Pearlmutter's algorithm, one forward pass + one backward pass yields $Hv$, $O(N)$ time and $O(N)$ memory, GPU-friendly
+- **Monte Carlo estimation of Ricci/scalar curvature**: Randomly sample directions $v$, $\mathbb{E}[v^T H v] = \text{tr}(H)$, using Hutchinson's estimator, GPU-friendly
+- **Jacobi fields**: Require integrating a second-order ODE along a trajectory, serial recurrence, GPU-unfriendly; in practice, discrete finite-difference approximations are used
+- Low precision: second-order derivatives in HVP are noisy under fp16, requiring fp32 accumulation
+
+## Risks and Failure Conditions
+
+- **Materializing the full Riemann/Hessian tensor**: $O(N^2)$ to $O(N^4)$ memory, impossible when $N \sim 10^9$
+- **Low signal-to-noise ratio in curvature estimation**: Monte Carlo estimation of HVP has high variance; with small batches, the signal may be drowned in noise
+- **Treating curvature regularization as a panacea**: Curvature estimation itself is expensive (requiring additional forward and backward passes); benefits should be validated at small scale first when uncertain
+- **Discrete approximation errors**: When using finite differences to approximate Jacobi fields/HVP, the step size is sensitive -- too large causes truncation error, too small causes floating-point cancellation
+
+## Further References
+
+- Distillation notes: references/books/differential-geometry.md (Ch 12 Section 12.5/Section 12.10 Curvature, Ch 13 Section 13.2 Riemann Curvature, Section 13.7 Jacobi Fields, Section 13.11 Rauch Comparison)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 13.2 Riemann Curvature Tensor, Section 13.7 Jacobi Fields

package/knowledge-base/differential-geometry/curvature.md

@@ -0,0 +1,49 @@
+# 曲率 (Curvature)
+
+## 最小定义
+
+曲率度量流形"弯不弯"以及"路径依不依赖"。Riemann 曲率张量 $R(X,Y)Z = \nabla_X\nabla_Y Z - \nabla_Y\nabla_X Z - \nabla_{[X,Y]}Z$ 描述向量绕无穷小平行四边形平行移动一圈后的偏转。截面曲率 $K(\sigma)$ 是其最简标量提取。
+
+## 核心公式
+
+- Riemann 曲率张量：$R^l_{ijk} = \partial_i \Gamma^l_{jk} - \partial_j \Gamma^l_{ik} + \Gamma^l_{im}\Gamma^m_{jk} - \Gamma^l_{jm}\Gamma^m_{ik}$
+- Ricci 曲率（缩并）：$R_{ij} = \sum_k R^k_{ikj}$
+- 标量曲率：$S = \sum_{ij} g^{ij} R_{ij}$
+- 截面曲率：$K(X,Y) = \frac{\langle R(X,Y)Y, X\rangle}{\|X\|^2\|Y\|^2 - \langle X,Y\rangle^2}$
+- Jacobi 方程：$\frac{D^2 J}{dt^2} + R(J, \dot\gamma)\dot\gamma = 0$（描述测地线的发散/汇聚）
+- Hessian-vector product：$Hv = \nabla(\nabla L \cdot v)$，$O(N)$ 估计曲率信息
+
+## 适用问题
+
+- 损失地形分析：曲率决定条件数与尖锐度，尖锐极小 vs 平坦极小的区分
+- 优化轨迹稳定性：Jacobi 场描述相邻优化轨迹的发散/汇聚
+- 泛化性诊断：平坦极小（低曲率）往往泛化更好
+- 流形学习：数据流形的曲率指导隐空间维数和度量选择
+
+## AI 设计翻译
+
+- **曲率正则化（SAM 的几何视角）**：用 Hessian-vector product 估计 $\max_v v^T H v$，惩罚尖锐极小，偏好 flat minima
+- **HVP-based 诊断器**：$\|Hv\|/\|v\|$ 作为 loss landscape 曲率的廉价代理，用于学习率自适应和早停
+- **Jacobi 场轨迹监控**：跟踪两条相邻优化轨迹的距离变化，$J''(t) + R(J,\dot\gamma)\dot\gamma = 0$ 的离散版，检测发散/收敛
+- **Ricci-flow 启发式图重连**：用离散 Ricci 曲率指导图/注意力结构的动态调整（负曲率边=瓶颈，需增连）
+
+## 工程可行性
+
+GPU 友好度：曲率的核心难点是"不能物化全张量"。
+- **Riemann 张量**：4 阶，$n^4$ 个分量，物化即爆显存，**禁止显式计算**
+- **Hessian-vector product (HVP)**：通过 Pearlmutter 算法，一次前向 + 一次反向即可得到 $Hv$，$O(N)$ 时间 $O(N)$ 显存，GPU 友好
+- **Ricci/标量曲率的 Monte Carlo 估计**：随机采样方向 $v$，$\mathbb{E}[v^T H v] = \text{tr}(H)$，用 Hutchinson 估计，GPU 友好
+- **Jacobi 场**：需要沿轨迹积分二阶 ODE，串行递推，GPU 不友好；工程上用离散差分近似
+- 低精度：HVP 中的二阶导在 fp16 下噪声大，需 fp32 累加
+
+## 风险与失效条件
+
+- **物化全 Riemann/Hessian 张量**：$O(N^2)$~$O(N^4)$ 显存，$N \sim 10^9$ 时不可能
+- **曲率估计信噪比低**：HVP 的 Monte Carlo 估计方差大，小 batch 下信号可能被噪声淹没
+- **把曲率正则当万能药**：曲率估计本身昂贵（每次需额外前向+反向），收益不确定时需先小规模验证
+- **离散近似误差**：用有限差分近似 Jacobi 场/HVP 时，步长选择敏感——太大截断误差大，太小浮点抵消
+
+## 深入参考
+
+- 蒸馏稿：references/books/differential-geometry.md（Ch 12 §12.5/§12.10 Curvature, Ch 13 §13.2 Riemann Curvature, §13.7 Jacobi Fields, §13.11 Rauch Comparison）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.2 Riemann Curvature Tensor, §13.7 Jacobi Fields

package/knowledge-base/differential-geometry/geodesic.en.md

@@ -0,0 +1,51 @@
+# Geodesic
+
+## Minimal Definition
+
+A geodesic is a locally length-minimizing (or extremal-length) path $\gamma: [0,1] \to M$ on a manifold, satisfying the auto-parallel condition $\nabla_{\dot\gamma} \dot\gamma = 0$ -- that is, the tangent vector is parallel-transported unchanged along itself. The exponential map $\exp_p(v)$ maps a tangent vector $v \in T_pM$ to the endpoint at $t=1$ of the geodesic starting at $p$ with initial velocity $v$.
+
+## Core Formulas
+
+- Geodesic equation: $\ddot\gamma^k + \sum_{ij} \Gamma^k_{ij} \dot\gamma^i \dot\gamma^j = 0$
+- Exponential map: $\exp_p(v) = \gamma_v(1)$, where $\gamma_v$ is the geodesic with $\gamma(0)=p, \dot\gamma(0)=v$
+- Logarithmic map: $\log_p(q) = v \in T_pM$ such that $\exp_p(v) = q$ (the inverse of the exponential map)
+- Retraction (engineering approximation): $R_p(v) \approx \exp_p(v)$, requiring only a first-order approximation for optimization
+- Closed-form on the sphere: $\exp_p(v) = \cos(\|v\|) p + \sin(\|v\|) \frac{v}{\|v\|}$
+
+## Applicable Problems
+
+- Constrained optimization: performing SGD on SPD/Stiefel/Grassmann/hyperbolic manifolds, where update steps must move along geodesics
+- Latent space interpolation: geodesics between two points in the latent space respect the data manifold structure better than Euclidean straight lines
+- Distance computation on manifolds: $d(p,q) = \|\log_p(q)\|_g$
+- Data augmentation: sampling along geodesics to generate new training samples
+
+## AI Design Translation
+
+- **Retraction-based optimizer**: Each step performs $x_{k+1} = R_{x_k}(-\eta \cdot \text{grad})$, using closed-form retractions instead of ODE integration; rotations for the sphere, QR/Cayley for Stiefel, Rodrigues for SO(3)
+- **Geodesic interpolation layer**: Use closed-form geodesics in spherical/hyperbolic latent spaces for mixup and interpolation, $\gamma(t) = \exp_p(t \cdot \log_p(q))$
+- **Momentum on manifolds**: Transport the momentum vector from $T_{x_k}M$ to $T_{x_{k+1}}M$ via vector transport (the discrete analog of parallel transport), then combine with the new gradient
+- **Exponential map output head**: The network makes unconstrained predictions in the tangent space $\mathbb{R}^n$, then projects back to the valid manifold via $\exp_p$, naturally satisfying constraints
+
+## Engineering Feasibility
+
+GPU friendliness depends on whether a closed-form retraction exists:
+- **Manifolds with closed forms** (sphere, hyperbolic, SO(3), Stiefel-QR): $\exp_p(v)$ is a finite algebraic expression, $O(1)$/sample, tensorizable in batches, GPU-friendly
+- **Manifolds without closed forms**: require numerical integration of the geodesic equation (second-order ODE), serial recurrence, GPU-unfriendly
+- Closed-form retractions as substitutes for exact exp: QR decomposition, Cayley transform, and other first-order approximations trade a small amount of precision for significant speedup
+- Small-matrix exp for 3x3/4x4 (SO(3)/SE(3)) can be fused into a single kernel, but cannot fully saturate Tensor Cores
+- **Low-precision critical issue**: $\sin\theta/\theta$ and $(1-\cos\theta)/\theta^2$ produce division by zero as $\theta \to 0$; $\log$ is singular as $\theta \to \pi$; fp16 yields NaN directly
+
+## Risks and Failure Conditions
+
+- **Step-by-step ODE integration for geodesics**: Serial recurrence kills GPU parallelism; closed-form retractions must be used instead
+- **Small-angle / large-angle singularities**: Numerical instability at $\theta \to 0$ and $\theta \to \pi$ is catastrophically amplified at low precision; Taylor expansion fallbacks are essential
+- **Retraction error accumulation**: First-order approximations may accumulate errors over multiple iterations, requiring occasional exact projection corrections
+- **Cut locus problem**: Beyond the cut locus, the exponential map no longer yields the shortest path; $\log_p(q)$ may not exist or may not be unique
+- **Forced manifold structure for geometric aesthetics**: Applying geodesics to tasks where Euclidean approximations suffice adds complexity and singularity risk
+
+## Further References
+
+- Distillation notes: references/books/differential-geometry.md (Ch 13 Section 13.4 Geodesics, Section 13.11 Rauch Comparison)
+- Distillation notes: references/books/smooth-manifolds.md (Ch 20 The Exponential Map)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 13.4 Geodesics
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 20 (exponential map, retraction prototype)

package/knowledge-base/differential-geometry/geodesic.md

@@ -0,0 +1,51 @@
+# 测地线 (Geodesic)
+
+## 最小定义
+
+测地线是流形上局部最短（或极值长度）的路径 $\gamma: [0,1] \to M$，满足自平行条件 $\nabla_{\dot\gamma} \dot\gamma = 0$——即切向量沿自身平行移动不变。指数映射 $\exp_p(v)$ 将切向量 $v \in T_pM$ 映为从 $p$ 出发、初速为 $v$ 的测地线在 $t=1$ 处的终点。
+
+## 核心公式
+
+- 测地方程：$\ddot\gamma^k + \sum_{ij} \Gamma^k_{ij} \dot\gamma^i \dot\gamma^j = 0$
+- 指数映射：$\exp_p(v) = \gamma_v(1)$，其中 $\gamma_v$ 是 $\gamma(0)=p, \dot\gamma(0)=v$ 的测地线
+- 对数映射：$\log_p(q) = v \in T_pM$ 使得 $\exp_p(v) = q$（指数映射的逆）
+- Retraction（工程近似）：$R_p(v) \approx \exp_p(v)$，只需一阶近似即可用于优化
+- 球面闭式：$\exp_p(v) = \cos(\|v\|) p + \sin(\|v\|) \frac{v}{\|v\|}$
+
+## 适用问题
+
+- 约束优化：在 SPD/Stiefel/Grassmann/双曲流形上做 SGD，更新步需要沿测地线移动
+- 隐空间插值：latent space 中两点间的测地线比欧氏直线更尊重数据流形结构
+- 流形上的距离计算：$d(p,q) = \|\log_p(q)\|_g$
+- 数据增广：沿测地线采样生成新训练样本
+
+## AI 设计翻译
+
+- **Retraction-based 优化器**：每步做 $x_{k+1} = R_{x_k}(-\eta \cdot \text{grad})$，用闭式 retraction 替代 ODE 积分；球面用旋转、Stiefel 用 QR/Cayley、SO(3) 用 Rodrigues
+- **测地线插值层**：在球面/双曲 latent space 中用闭式测地线做 mixup 和插值，$\gamma(t) = \exp_p(t \cdot \log_p(q))$
+- **流形上的 momentum**：将动量向量通过 vector transport（平行移动的离散版）从 $T_{x_k}M$ 搬到 $T_{x_{k+1}}M$，再与新梯度合成
+- **指数映射输出头**：网络在切空间 $\mathbb{R}^n$ 中自由预测，经 $\exp_p$ 投回合法流形，天然满足约束
+
+## 工程可行性
+
+GPU 友好度取决于是否有闭式 retraction：
+- **有闭式的流形**（球面、双曲、SO(3)、Stiefel-QR）：$\exp_p(v)$ 是有限项代数表达式，$O(1)$/样本，可 batched 张量化，GPU 友好
+- **无闭式的流形**：需要数值积分测地方程（二阶 ODE），串行递推，GPU 不友好
+- 闭式 retraction 替代精确 exp：QR 分解、Cayley 变换等一阶近似，牺牲少量精度换取大幅加速
+- 3x3/4x4 小矩阵 exp（SO(3)/SE(3)）可融进单 kernel，但吃不满 Tensor Core
+- **低精度致命点**：$\sin\theta/\theta$、$(1-\cos\theta)/\theta^2$ 在 $\theta \to 0$ 时除零，$\theta \to \pi$ 时 log 奇异，fp16 直接 NaN
+
+## 风险与失效条件
+
+- **逐步 ODE 积分做测地线**：串行递推杀死 GPU 并行度，必须用闭式 retraction 替代
+- **小角/大角奇异点**：$\theta \to 0$ 和 $\theta \to \pi$ 处的数值不稳定在低精度下被灾难性放大，必须做 Taylor 展开兜底
+- **Retraction 误差累积**：一阶近似在多步迭代中误差可能累积，需偶尔做一次精确投影校正
+- **Cut locus 问题**：指数映射在 cut locus 以外不再是最短路径，$\log_p(q)$ 可能不存在或不唯一
+- **为几何美强行流形化**：欧氏近似已足够的任务硬上测地线，增加复杂度和奇异点风险
+
+## 深入参考
+
+- 蒸馏稿：references/books/differential-geometry.md（Ch 13 §13.4 Geodesics, §13.11 Rauch Comparison）
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 20 The Exponential Map）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.4 Geodesics
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 20（指数映射、retraction 原型）

package/knowledge-base/differential-geometry/manifold.en.md

@@ -0,0 +1,47 @@
+# Manifold
+
+## Minimal Definition
+
+A manifold is a topological space that is locally homeomorphic to Euclidean space $\mathbb{R}^n$. A smooth manifold further requires that the transition maps $\phi_\beta \circ \phi_\alpha^{-1}$ between coordinate charts are $C^\infty$-smooth, enabling calculus to be performed on curved spaces.
+
+## Core Formulas
+
+- Coordinate chart: $(U_\alpha, \phi_\alpha)$, where $\phi_\alpha: U_\alpha \to \mathbb{R}^n$ is a homeomorphism
+- Smooth transition map: $\phi_\beta \circ \phi_\alpha^{-1}: \phi_\alpha(U_\alpha \cap U_\beta) \to \phi_\beta(U_\alpha \cap U_\beta) \in C^\infty$
+- Whitney Embedding Theorem: An $n$-dimensional manifold can be embedded in $\mathbb{R}^{2n}$
+
+## Applicable Problems
+
+- Data naturally resides in non-Euclidean spaces: rotations SO(3), covariance matrices SPD(n), directional data $S^2$, graphs and meshes
+- Parameters have geometric constraints (orthogonality, unit norm, low rank) that require identifying the constraint set as a submanifold
+- Latent space geometric modeling: interpolation, clustering, and nearest-neighbor search must respect the intrinsic curved structure of the data
+- Dimensionality reduction and embedding: the manifold hypothesis assumes high-dimensional data lies on a low-dimensional manifold
+
+## AI Design Translation
+
+- **Manifold optimizer (Riemannian SGD/Adam)**: Project gradients onto the tangent space and retract back to the manifold, replacing the ad-hoc patchwork of projected gradient descent
+- **Latent space geometry module**: Use manifold structure in the VAE/GAN latent space for geodesic interpolation, replacing Euclidean linear interpolation
+- **Constraint reparameterization layer**: Encode orthogonality/SPD/unit-norm constraints as manifold parameterizations (e.g., Cayley transform, matrix exponential), so outputs naturally satisfy constraints
+- **Manifold-hypothesis-driven architecture design**: Use manifold dimension estimation to guide latent space dimension selection, avoiding the curse of dimensionality
+
+## Engineering Feasibility
+
+Moderate GPU friendliness. Coordinate chart transformations are element-wise maps (parallelizable), but the core bottleneck lies in:
+- Transition maps with closed-form expressions (sphere, hyperbolic space): can be directly tensorized as batched per-sample computations, GPU-friendly
+- Transition maps requiring iterative solvers (general manifolds): serial dependencies, not tensorizable, GPU-unfriendly
+- Partition of unity involves locally weighted sums, expressible as sparse matmul
+- Key operation complexity depends on the specific manifold: simple manifolds $O(1)$/sample, complex manifolds may be $O(n^3)$
+
+## Risks and Failure Conditions
+
+- **Global chart illusion**: Attempting to cover the entire manifold with a single parameterization inevitably introduces singularities (e.g., gimbal lock with Euler angles); an atlas or redundant parameterization is required
+- **Manifold hypothesis abuse**: Applying manifold structure to data that actually lives in flat Euclidean space is pure over-engineering
+- **Low-precision instability**: Matrix exp/log/eig in coordinate transformations becomes catastrophically unstable under fp16/bf16, often silently diverging
+- **Dimension estimation errors**: The Whitney Embedding Theorem provides an upper bound of $2n$; practical embedding dimension selection lacks theoretical guidance
+
+## Further References
+
+- Distillation notes: references/books/smooth-manifolds.md (Ch 1-2 Smooth Manifolds / Smooth Maps)
+- Distillation notes: references/books/differential-geometry.md (Ch 1-2 Differentiable Manifolds / The Tangent Structure)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2 (topological manifolds, smooth structures, partition of unity)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2

package/knowledge-base/differential-geometry/manifold.md

@@ -0,0 +1,47 @@
+# 流形 (Manifold)
+
+## 最小定义
+
+流形是局部同胚于欧氏空间 $\mathbb{R}^n$ 的拓扑空间。光滑流形进一步要求坐标卡之间的转移映射 $\phi_\beta \circ \phi_\alpha^{-1}$ 是 $C^\infty$ 光滑的，使得微积分可以在弯曲空间上进行。
+
+## 核心公式
+
+- 坐标卡：$(U_\alpha, \phi_\alpha)$，其中 $\phi_\alpha: U_\alpha \to \mathbb{R}^n$ 是同胚
+- 光滑转移映射：$\phi_\beta \circ \phi_\alpha^{-1}: \phi_\alpha(U_\alpha \cap U_\beta) \to \phi_\beta(U_\alpha \cap U_\beta) \in C^\infty$
+- Whitney 嵌入定理：$n$ 维流形可嵌入 $\mathbb{R}^{2n}$
+
+## 适用问题
+
+- 数据本身住在非欧空间：旋转 SO(3)、协方差矩阵 SPD(n)、方向数据 $S^2$、图与网格
+- 参数有几何约束（正交、单位范数、低秩），需要把约束集识别为子流形
+- 隐空间几何建模：插值、聚类、最近邻需要尊重数据的内在弯曲结构
+- 降维与嵌入：高维数据的低维流形假设（manifold hypothesis）
+
+## AI 设计翻译
+
+- **流形优化器（Riemannian SGD/Adam）**：梯度投影到切空间 + retraction 回流形，替代投影梯度下降的 ad-hoc 修补
+- **隐空间几何模块**：在 VAE/GAN 的 latent space 中用流形结构做测地插值，替代欧氏线性插值
+- **约束重参数化层**：将正交/SPD/单位范数约束编码为流形参数化（如 Cayley 变换、矩阵指数），输出天然满足约束
+- **流形假设驱动的架构设计**：用流形维数估计指导隐空间维度选择，避免维度诅咒
+
+## 工程可行性
+
+GPU 友好度中等。坐标卡变换本身是逐元素的映射（可并行），但核心瓶颈在于：
+- 转移映射若为闭式（球面、双曲空间）：可直接张量化，batched 逐样本计算，GPU 友好
+- 转移映射若需迭代求解（一般流形）：串行依赖、不可张量化，GPU 不友好
+- 单位分解（partition of unity）涉及局部加权求和，可表达为稀疏 matmul
+- 关键操作复杂度取决于具体流形：简单流形 $O(1)$/样本，复杂流形可能 $O(n^3)$
+
+## 风险与失效条件
+
+- **全局坐标卡幻觉**：试图用单一参数化覆盖整个流形必有奇点（如欧拉角的 gimbal lock），需要 atlas / 冗余参数化
+- **流形假设滥用**：数据实际分布在平坦欧氏空间时硬套流形结构，纯属过度工程
+- **低精度不稳定**：坐标变换中的矩阵 exp/log/eig 在 fp16/bf16 下灾难性不稳定，常静默发散
+- **维数估计错误**：Whitney 嵌入定理给出上界 $2n$，实际嵌入维数选择缺乏理论指导
+
+## 深入参考
+
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 1-2 Smooth Manifolds / Smooth Maps）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 1-2 Differentiable Manifolds / The Tangent Structure）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2（拓扑流形、光滑结构、单位分解）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2

package/knowledge-base/differential-geometry/metric-tensor.en.md

@@ -0,0 +1,50 @@
+# Metric Tensor
+
+## Minimal Definition
+
+The metric tensor $g$ is a positive-definite symmetric bilinear form $g_p: T_pM \times T_pM \to \mathbb{R}$ on the tangent space $T_pM$ at each point $p$ of a manifold, defining inner products, lengths, angles, and distances on the manifold. It is the precise quantification of "what is close to what" and "which direction is steepest."
+
+## Core Formulas
+
+- Inner product: $\langle u, v \rangle_g = u^T g_p v = \sum_{ij} g_{ij} u^i v^j$
+- Arc length element: $ds^2 = \sum_{ij} g_{ij} dx^i dx^j$
+- Geodesic distance: $d(p,q) = \inf_\gamma \int_0^1 \sqrt{g_{\gamma(t)}(\dot\gamma, \dot\gamma)} \, dt$
+- Musical isomorphism (index raising/lowering): $v^\flat = gv$ (tangent to cotangent), $\omega^\sharp = g^{-1}\omega$ (cotangent to tangent)
+- Fisher-Rao metric: $g_{ij}(\theta) = \mathbb{E}_{p_\theta}\left[\frac{\partial \log p_\theta}{\partial \theta^i} \frac{\partial \log p_\theta}{\partial \theta^j}\right]$
+
+## Applicable Problems
+
+- Parameter space is non-flat and non-Euclidean: the natural metric on families of probability distributions is the Fisher information matrix
+- Slow optimization convergence: poor conditioning arises from metric mismatch, remedied by preconditioning with the natural gradient $g^{-1}\nabla L$
+- Distance/similarity must adapt to data geometry: metric learning is essentially learning a $g$
+- Volume computation and density estimation: $\sqrt{\det g}$ gives the volume form on the manifold
+
+## AI Design Translation
+
+- **Natural gradient / K-FAC optimizer**: $F^{-1}\nabla L$ where $F$ is the Fisher metric; K-FAC uses the Kronecker factorization $F \approx A \otimes B$ to reduce inversion to two small matrix inversions, and the preconditioning reduces to a GEMM chain
+- **Learnable metric layer**: Parameterize $g = L^T L$ (Cholesky) to learn a task-specific Riemannian metric for metric learning and contrastive learning
+- **Information-geometric regularization**: Replace Euclidean $\|d\theta\|^2$ with the Fisher-Rao distance $\|d\theta\|_F^2 = d\theta^T F d\theta$, making regularization invariant to reparameterization
+- **Fisher-aware learning rate scheduling**: Use $\|g^{-1}\nabla L\|_g$ as the "geometrically correct" gradient norm to guide learning rate selection
+
+## Engineering Feasibility
+
+GPU friendliness depends on the structure of the metric:
+- **Diagonal metric** $g = \text{diag}(g_1, \ldots, g_n)$: element-wise multiply/divide, $O(n)$, perfectly GPU-friendly
+- **Kronecker-factored** $g = A \otimes B$: $(A\otimes B)^{-1} = A^{-1}\otimes B^{-1}$, small matrix inversions + GEMM chain, the core trick of K-FAC, GPU-feasible
+- **Block-diagonal metric**: per-block independent inversion, batched small matrix operations, GPU-friendly
+- **Full dense metric**: $n \times n$ matrix inversion $O(n^3)$ + memory $O(n^2)$; with parameter count $N \sim 10^9$, this is immediately ruled out
+- Low-precision risk: the Fisher matrix is often ill-conditioned; inversion under fp16 catastrophically amplifies errors -- **must add damping $F + \lambda I$ and invert in fp32**
+
+## Risks and Failure Conditions
+
+- **Ill-conditioned metric matrix**: The condition number of the Fisher matrix can exceed $10^6$; low-precision inversion results are entirely noise
+- **Materializing the full metric matrix**: An $N \times N$ matrix ($N \sim 10^9$) requires $\sim 4$ PB of memory, making materialization impossible
+- **Metric-task mismatch**: The Fisher metric assumes the probabilistic model is correct; under model misspecification, the natural gradient can perform worse than SGD
+- **Dynamic metric update overhead**: The Fisher matrix changes with the parameters; statistical noise from re-estimation at each step may offset preconditioning benefits
+
+## Further References
+
+- Distillation notes: references/books/differential-geometry.md (Ch 7 Section 7.6 Metric Tensors, Ch 13 Section 13.1 Levi-Civita)
+- Distillation notes: references/books/smooth-manifolds.md (Ch 13 Riemannian Metrics)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 7.6 Metric Tensors
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 13 (Riemannian metrics, musical isomorphisms sharp/flat)

package/knowledge-base/differential-geometry/metric-tensor.md

@@ -0,0 +1,50 @@
+# 度量张量 (Metric Tensor)
+
+## 最小定义
+
+度量张量 $g$ 是流形上每点 $p$ 处切空间 $T_pM$ 上的一个正定对称双线性形式 $g_p: T_pM \times T_pM \to \mathbb{R}$，定义了流形上的内积、长度、角度和距离。它是"谁离谁近、什么方向最陡"的精确量化。
+
+## 核心公式
+
+- 内积：$\langle u, v \rangle_g = u^T g_p v = \sum_{ij} g_{ij} u^i v^j$
+- 弧长：$ds^2 = \sum_{ij} g_{ij} dx^i dx^j$
+- 测地距离：$d(p,q) = \inf_\gamma \int_0^1 \sqrt{g_{\gamma(t)}(\dot\gamma, \dot\gamma)} \, dt$
+- 升降指标（musical isomorphism）：$v^\flat = gv$（切→余切），$\omega^\sharp = g^{-1}\omega$（余切→切）
+- Fisher-Rao 度量：$g_{ij}(\theta) = \mathbb{E}_{p_\theta}\left[\frac{\partial \log p_\theta}{\partial \theta^i} \frac{\partial \log p_\theta}{\partial \theta^j}\right]$
+
+## 适用问题
+
+- 参数空间非平坦欧氏：概率分布族上的自然度量是 Fisher 信息矩阵
+- 优化收敛缓慢：条件数差源于度量不匹配，用自然梯度 $g^{-1}\nabla L$ 预条件
+- 距离/相似度需要适应数据几何：度量学习本质是学习一个 $g$
+- 体积计算与密度估计：$\sqrt{\det g}$ 给出流形上的体积形式
+
+## AI 设计翻译
+
+- **自然梯度/K-FAC 优化器**：$F^{-1}\nabla L$ 其中 $F$ 是 Fisher 度量；K-FAC 用 Kronecker 因子化 $F \approx A \otimes B$ 使求逆变为两个小矩阵求逆，预条件作用退化为 GEMM 链
+- **可学习度量层**：参数化 $g = L^T L$（Cholesky），学习任务相关的黎曼度量，用于度量学习和对比学习
+- **信息几何正则化**：用 Fisher-Rao 距离 $\|d\theta\|_F^2 = d\theta^T F d\theta$ 替代欧氏 $\|d\theta\|^2$，使正则化对重参数化不变
+- **Fisher 感知的学习率调度**：$\|g^{-1}\nabla L\|_g$ 作为"几何正确"的梯度范数，指导学习率选择
+
+## 工程可行性
+
+GPU 友好度取决于度量的结构化程度：
+- **对角度量** $g = \text{diag}(g_1, \ldots, g_n)$：逐元素乘除，$O(n)$，完美 GPU 友好
+- **Kronecker 因子化** $g = A \otimes B$：$(A\otimes B)^{-1} = A^{-1}\otimes B^{-1}$，小矩阵求逆 + GEMM 链，K-FAC 的核心 trick，GPU 可行
+- **块对角度量**：逐块独立求逆，batched 小矩阵运算，GPU 友好
+- **全稠密度量**：$n \times n$ 矩阵求逆 $O(n^3)$ + 显存 $O(n^2)$，参数量 $N \sim 10^9$ 时直接出局
+- 低精度风险：Fisher 矩阵常病态，fp16 下求逆灾难性放大误差，**必须加阻尼 $F + \lambda I$ 并在 fp32 下求逆**
+
+## 风险与失效条件
+
+- **度量矩阵病态**：Fisher 矩阵的条件数可达 $10^6$ 以上，低精度求逆结果全是噪声
+- **物化全度量矩阵**：$N \times N$ 矩阵（$N \sim 10^9$）需 $\sim 4$ PB 显存，不可能物化
+- **度量与任务不匹配**：Fisher 度量假设概率模型正确，模型误设定时自然梯度可能比 SGD 更差
+- **动态度量更新开销**：Fisher 矩阵随参数变化，每步重新估计的统计噪声可能抵消预条件收益
+
+## 深入参考
+
+- 蒸馏稿：references/books/differential-geometry.md（Ch 7 §7.6 Metric Tensors, Ch 13 §13.1 Levi-Civita）
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 13 Riemannian Metrics）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §7.6 Metric Tensors
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 13（黎曼度量、升降指标 (sharp)/(flat)）