math-skill 2.1.0 → 3.0.0

package/knowledge-base/differential-geometry/geodesic.md

@@ -0,0 +1,51 @@
+# 测地线 (Geodesic)
+
+## 最小定义
+
+测地线是流形上局部最短（或极值长度）的路径 $\gamma: [0,1] \to M$，满足自平行条件 $\nabla_{\dot\gamma} \dot\gamma = 0$——即切向量沿自身平行移动不变。指数映射 $\exp_p(v)$ 将切向量 $v \in T_pM$ 映为从 $p$ 出发、初速为 $v$ 的测地线在 $t=1$ 处的终点。
+
+## 核心公式
+
+- 测地方程：$\ddot\gamma^k + \sum_{ij} \Gamma^k_{ij} \dot\gamma^i \dot\gamma^j = 0$
+- 指数映射：$\exp_p(v) = \gamma_v(1)$，其中 $\gamma_v$ 是 $\gamma(0)=p, \dot\gamma(0)=v$ 的测地线
+- 对数映射：$\log_p(q) = v \in T_pM$ 使得 $\exp_p(v) = q$（指数映射的逆）
+- Retraction（工程近似）：$R_p(v) \approx \exp_p(v)$，只需一阶近似即可用于优化
+- 球面闭式：$\exp_p(v) = \cos(\|v\|) p + \sin(\|v\|) \frac{v}{\|v\|}$
+
+## 适用问题
+
+- 约束优化：在 SPD/Stiefel/Grassmann/双曲流形上做 SGD，更新步需要沿测地线移动
+- 隐空间插值：latent space 中两点间的测地线比欧氏直线更尊重数据流形结构
+- 流形上的距离计算：$d(p,q) = \|\log_p(q)\|_g$
+- 数据增广：沿测地线采样生成新训练样本
+
+## AI 设计翻译
+
+- **Retraction-based 优化器**：每步做 $x_{k+1} = R_{x_k}(-\eta \cdot \text{grad})$，用闭式 retraction 替代 ODE 积分；球面用旋转、Stiefel 用 QR/Cayley、SO(3) 用 Rodrigues
+- **测地线插值层**：在球面/双曲 latent space 中用闭式测地线做 mixup 和插值，$\gamma(t) = \exp_p(t \cdot \log_p(q))$
+- **流形上的 momentum**：将动量向量通过 vector transport（平行移动的离散版）从 $T_{x_k}M$ 搬到 $T_{x_{k+1}}M$，再与新梯度合成
+- **指数映射输出头**：网络在切空间 $\mathbb{R}^n$ 中自由预测，经 $\exp_p$ 投回合法流形，天然满足约束
+
+## 工程可行性
+
+GPU 友好度取决于是否有闭式 retraction：
+- **有闭式的流形**（球面、双曲、SO(3)、Stiefel-QR）：$\exp_p(v)$ 是有限项代数表达式，$O(1)$/样本，可 batched 张量化，GPU 友好
+- **无闭式的流形**：需要数值积分测地方程（二阶 ODE），串行递推，GPU 不友好
+- 闭式 retraction 替代精确 exp：QR 分解、Cayley 变换等一阶近似，牺牲少量精度换取大幅加速
+- 3x3/4x4 小矩阵 exp（SO(3)/SE(3)）可融进单 kernel，但吃不满 Tensor Core
+- **低精度致命点**：$\sin\theta/\theta$、$(1-\cos\theta)/\theta^2$ 在 $\theta \to 0$ 时除零，$\theta \to \pi$ 时 log 奇异，fp16 直接 NaN
+
+## 风险与失效条件
+
+- **逐步 ODE 积分做测地线**：串行递推杀死 GPU 并行度，必须用闭式 retraction 替代
+- **小角/大角奇异点**：$\theta \to 0$ 和 $\theta \to \pi$ 处的数值不稳定在低精度下被灾难性放大，必须做 Taylor 展开兜底
+- **Retraction 误差累积**：一阶近似在多步迭代中误差可能累积，需偶尔做一次精确投影校正
+- **Cut locus 问题**：指数映射在 cut locus 以外不再是最短路径，$\log_p(q)$ 可能不存在或不唯一
+- **为几何美强行流形化**：欧氏近似已足够的任务硬上测地线，增加复杂度和奇异点风险
+
+## 深入参考
+
+- 蒸馏稿：references/books/differential-geometry.md（Ch 13 §13.4 Geodesics, §13.11 Rauch Comparison）
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 20 The Exponential Map）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.4 Geodesics
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 20（指数映射、retraction 原型）

package/knowledge-base/differential-geometry/manifold.en.md

@@ -0,0 +1,47 @@
+# Manifold
+
+## Minimal Definition
+
+A manifold is a topological space that is locally homeomorphic to Euclidean space $\mathbb{R}^n$. A smooth manifold further requires that the transition maps $\phi_\beta \circ \phi_\alpha^{-1}$ between coordinate charts are $C^\infty$-smooth, enabling calculus to be performed on curved spaces.
+
+## Core Formulas
+
+- Coordinate chart: $(U_\alpha, \phi_\alpha)$, where $\phi_\alpha: U_\alpha \to \mathbb{R}^n$ is a homeomorphism
+- Smooth transition map: $\phi_\beta \circ \phi_\alpha^{-1}: \phi_\alpha(U_\alpha \cap U_\beta) \to \phi_\beta(U_\alpha \cap U_\beta) \in C^\infty$
+- Whitney Embedding Theorem: An $n$-dimensional manifold can be embedded in $\mathbb{R}^{2n}$
+
+## Applicable Problems
+
+- Data naturally resides in non-Euclidean spaces: rotations SO(3), covariance matrices SPD(n), directional data $S^2$, graphs and meshes
+- Parameters have geometric constraints (orthogonality, unit norm, low rank) that require identifying the constraint set as a submanifold
+- Latent space geometric modeling: interpolation, clustering, and nearest-neighbor search must respect the intrinsic curved structure of the data
+- Dimensionality reduction and embedding: the manifold hypothesis assumes high-dimensional data lies on a low-dimensional manifold
+
+## AI Design Translation
+
+- **Manifold optimizer (Riemannian SGD/Adam)**: Project gradients onto the tangent space and retract back to the manifold, replacing the ad-hoc patchwork of projected gradient descent
+- **Latent space geometry module**: Use manifold structure in the VAE/GAN latent space for geodesic interpolation, replacing Euclidean linear interpolation
+- **Constraint reparameterization layer**: Encode orthogonality/SPD/unit-norm constraints as manifold parameterizations (e.g., Cayley transform, matrix exponential), so outputs naturally satisfy constraints
+- **Manifold-hypothesis-driven architecture design**: Use manifold dimension estimation to guide latent space dimension selection, avoiding the curse of dimensionality
+
+## Engineering Feasibility
+
+Moderate GPU friendliness. Coordinate chart transformations are element-wise maps (parallelizable), but the core bottleneck lies in:
+- Transition maps with closed-form expressions (sphere, hyperbolic space): can be directly tensorized as batched per-sample computations, GPU-friendly
+- Transition maps requiring iterative solvers (general manifolds): serial dependencies, not tensorizable, GPU-unfriendly
+- Partition of unity involves locally weighted sums, expressible as sparse matmul
+- Key operation complexity depends on the specific manifold: simple manifolds $O(1)$/sample, complex manifolds may be $O(n^3)$
+
+## Risks and Failure Conditions
+
+- **Global chart illusion**: Attempting to cover the entire manifold with a single parameterization inevitably introduces singularities (e.g., gimbal lock with Euler angles); an atlas or redundant parameterization is required
+- **Manifold hypothesis abuse**: Applying manifold structure to data that actually lives in flat Euclidean space is pure over-engineering
+- **Low-precision instability**: Matrix exp/log/eig in coordinate transformations becomes catastrophically unstable under fp16/bf16, often silently diverging
+- **Dimension estimation errors**: The Whitney Embedding Theorem provides an upper bound of $2n$; practical embedding dimension selection lacks theoretical guidance
+
+## Further References
+
+- Distillation notes: references/books/smooth-manifolds.md (Ch 1-2 Smooth Manifolds / Smooth Maps)
+- Distillation notes: references/books/differential-geometry.md (Ch 1-2 Differentiable Manifolds / The Tangent Structure)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2 (topological manifolds, smooth structures, partition of unity)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2

package/knowledge-base/differential-geometry/manifold.md

@@ -0,0 +1,47 @@
+# 流形 (Manifold)
+
+## 最小定义
+
+流形是局部同胚于欧氏空间 $\mathbb{R}^n$ 的拓扑空间。光滑流形进一步要求坐标卡之间的转移映射 $\phi_\beta \circ \phi_\alpha^{-1}$ 是 $C^\infty$ 光滑的，使得微积分可以在弯曲空间上进行。
+
+## 核心公式
+
+- 坐标卡：$(U_\alpha, \phi_\alpha)$，其中 $\phi_\alpha: U_\alpha \to \mathbb{R}^n$ 是同胚
+- 光滑转移映射：$\phi_\beta \circ \phi_\alpha^{-1}: \phi_\alpha(U_\alpha \cap U_\beta) \to \phi_\beta(U_\alpha \cap U_\beta) \in C^\infty$
+- Whitney 嵌入定理：$n$ 维流形可嵌入 $\mathbb{R}^{2n}$
+
+## 适用问题
+
+- 数据本身住在非欧空间：旋转 SO(3)、协方差矩阵 SPD(n)、方向数据 $S^2$、图与网格
+- 参数有几何约束（正交、单位范数、低秩），需要把约束集识别为子流形
+- 隐空间几何建模：插值、聚类、最近邻需要尊重数据的内在弯曲结构
+- 降维与嵌入：高维数据的低维流形假设（manifold hypothesis）
+
+## AI 设计翻译
+
+- **流形优化器（Riemannian SGD/Adam）**：梯度投影到切空间 + retraction 回流形，替代投影梯度下降的 ad-hoc 修补
+- **隐空间几何模块**：在 VAE/GAN 的 latent space 中用流形结构做测地插值，替代欧氏线性插值
+- **约束重参数化层**：将正交/SPD/单位范数约束编码为流形参数化（如 Cayley 变换、矩阵指数），输出天然满足约束
+- **流形假设驱动的架构设计**：用流形维数估计指导隐空间维度选择，避免维度诅咒
+
+## 工程可行性
+
+GPU 友好度中等。坐标卡变换本身是逐元素的映射（可并行），但核心瓶颈在于：
+- 转移映射若为闭式（球面、双曲空间）：可直接张量化，batched 逐样本计算，GPU 友好
+- 转移映射若需迭代求解（一般流形）：串行依赖、不可张量化，GPU 不友好
+- 单位分解（partition of unity）涉及局部加权求和，可表达为稀疏 matmul
+- 关键操作复杂度取决于具体流形：简单流形 $O(1)$/样本，复杂流形可能 $O(n^3)$
+
+## 风险与失效条件
+
+- **全局坐标卡幻觉**：试图用单一参数化覆盖整个流形必有奇点（如欧拉角的 gimbal lock），需要 atlas / 冗余参数化
+- **流形假设滥用**：数据实际分布在平坦欧氏空间时硬套流形结构，纯属过度工程
+- **低精度不稳定**：坐标变换中的矩阵 exp/log/eig 在 fp16/bf16 下灾难性不稳定，常静默发散
+- **维数估计错误**：Whitney 嵌入定理给出上界 $2n$，实际嵌入维数选择缺乏理论指导
+
+## 深入参考
+
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 1-2 Smooth Manifolds / Smooth Maps）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 1-2 Differentiable Manifolds / The Tangent Structure）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2（拓扑流形、光滑结构、单位分解）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2

package/knowledge-base/differential-geometry/metric-tensor.en.md

@@ -0,0 +1,50 @@
+# Metric Tensor
+
+## Minimal Definition
+
+The metric tensor $g$ is a positive-definite symmetric bilinear form $g_p: T_pM \times T_pM \to \mathbb{R}$ on the tangent space $T_pM$ at each point $p$ of a manifold, defining inner products, lengths, angles, and distances on the manifold. It is the precise quantification of "what is close to what" and "which direction is steepest."
+
+## Core Formulas
+
+- Inner product: $\langle u, v \rangle_g = u^T g_p v = \sum_{ij} g_{ij} u^i v^j$
+- Arc length element: $ds^2 = \sum_{ij} g_{ij} dx^i dx^j$
+- Geodesic distance: $d(p,q) = \inf_\gamma \int_0^1 \sqrt{g_{\gamma(t)}(\dot\gamma, \dot\gamma)} \, dt$
+- Musical isomorphism (index raising/lowering): $v^\flat = gv$ (tangent to cotangent), $\omega^\sharp = g^{-1}\omega$ (cotangent to tangent)
+- Fisher-Rao metric: $g_{ij}(\theta) = \mathbb{E}_{p_\theta}\left[\frac{\partial \log p_\theta}{\partial \theta^i} \frac{\partial \log p_\theta}{\partial \theta^j}\right]$
+
+## Applicable Problems
+
+- Parameter space is non-flat and non-Euclidean: the natural metric on families of probability distributions is the Fisher information matrix
+- Slow optimization convergence: poor conditioning arises from metric mismatch, remedied by preconditioning with the natural gradient $g^{-1}\nabla L$
+- Distance/similarity must adapt to data geometry: metric learning is essentially learning a $g$
+- Volume computation and density estimation: $\sqrt{\det g}$ gives the volume form on the manifold
+
+## AI Design Translation
+
+- **Natural gradient / K-FAC optimizer**: $F^{-1}\nabla L$ where $F$ is the Fisher metric; K-FAC uses the Kronecker factorization $F \approx A \otimes B$ to reduce inversion to two small matrix inversions, and the preconditioning reduces to a GEMM chain
+- **Learnable metric layer**: Parameterize $g = L^T L$ (Cholesky) to learn a task-specific Riemannian metric for metric learning and contrastive learning
+- **Information-geometric regularization**: Replace Euclidean $\|d\theta\|^2$ with the Fisher-Rao distance $\|d\theta\|_F^2 = d\theta^T F d\theta$, making regularization invariant to reparameterization
+- **Fisher-aware learning rate scheduling**: Use $\|g^{-1}\nabla L\|_g$ as the "geometrically correct" gradient norm to guide learning rate selection
+
+## Engineering Feasibility
+
+GPU friendliness depends on the structure of the metric:
+- **Diagonal metric** $g = \text{diag}(g_1, \ldots, g_n)$: element-wise multiply/divide, $O(n)$, perfectly GPU-friendly
+- **Kronecker-factored** $g = A \otimes B$: $(A\otimes B)^{-1} = A^{-1}\otimes B^{-1}$, small matrix inversions + GEMM chain, the core trick of K-FAC, GPU-feasible
+- **Block-diagonal metric**: per-block independent inversion, batched small matrix operations, GPU-friendly
+- **Full dense metric**: $n \times n$ matrix inversion $O(n^3)$ + memory $O(n^2)$; with parameter count $N \sim 10^9$, this is immediately ruled out
+- Low-precision risk: the Fisher matrix is often ill-conditioned; inversion under fp16 catastrophically amplifies errors -- **must add damping $F + \lambda I$ and invert in fp32**
+
+## Risks and Failure Conditions
+
+- **Ill-conditioned metric matrix**: The condition number of the Fisher matrix can exceed $10^6$; low-precision inversion results are entirely noise
+- **Materializing the full metric matrix**: An $N \times N$ matrix ($N \sim 10^9$) requires $\sim 4$ PB of memory, making materialization impossible
+- **Metric-task mismatch**: The Fisher metric assumes the probabilistic model is correct; under model misspecification, the natural gradient can perform worse than SGD
+- **Dynamic metric update overhead**: The Fisher matrix changes with the parameters; statistical noise from re-estimation at each step may offset preconditioning benefits
+
+## Further References
+
+- Distillation notes: references/books/differential-geometry.md (Ch 7 Section 7.6 Metric Tensors, Ch 13 Section 13.1 Levi-Civita)
+- Distillation notes: references/books/smooth-manifolds.md (Ch 13 Riemannian Metrics)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 7.6 Metric Tensors
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 13 (Riemannian metrics, musical isomorphisms sharp/flat)

package/knowledge-base/differential-geometry/metric-tensor.md

@@ -0,0 +1,50 @@
+# 度量张量 (Metric Tensor)
+
+## 最小定义
+
+度量张量 $g$ 是流形上每点 $p$ 处切空间 $T_pM$ 上的一个正定对称双线性形式 $g_p: T_pM \times T_pM \to \mathbb{R}$，定义了流形上的内积、长度、角度和距离。它是"谁离谁近、什么方向最陡"的精确量化。
+
+## 核心公式
+
+- 内积：$\langle u, v \rangle_g = u^T g_p v = \sum_{ij} g_{ij} u^i v^j$
+- 弧长：$ds^2 = \sum_{ij} g_{ij} dx^i dx^j$
+- 测地距离：$d(p,q) = \inf_\gamma \int_0^1 \sqrt{g_{\gamma(t)}(\dot\gamma, \dot\gamma)} \, dt$
+- 升降指标（musical isomorphism）：$v^\flat = gv$（切→余切），$\omega^\sharp = g^{-1}\omega$（余切→切）
+- Fisher-Rao 度量：$g_{ij}(\theta) = \mathbb{E}_{p_\theta}\left[\frac{\partial \log p_\theta}{\partial \theta^i} \frac{\partial \log p_\theta}{\partial \theta^j}\right]$
+
+## 适用问题
+
+- 参数空间非平坦欧氏：概率分布族上的自然度量是 Fisher 信息矩阵
+- 优化收敛缓慢：条件数差源于度量不匹配，用自然梯度 $g^{-1}\nabla L$ 预条件
+- 距离/相似度需要适应数据几何：度量学习本质是学习一个 $g$
+- 体积计算与密度估计：$\sqrt{\det g}$ 给出流形上的体积形式
+
+## AI 设计翻译
+
+- **自然梯度/K-FAC 优化器**：$F^{-1}\nabla L$ 其中 $F$ 是 Fisher 度量；K-FAC 用 Kronecker 因子化 $F \approx A \otimes B$ 使求逆变为两个小矩阵求逆，预条件作用退化为 GEMM 链
+- **可学习度量层**：参数化 $g = L^T L$（Cholesky），学习任务相关的黎曼度量，用于度量学习和对比学习
+- **信息几何正则化**：用 Fisher-Rao 距离 $\|d\theta\|_F^2 = d\theta^T F d\theta$ 替代欧氏 $\|d\theta\|^2$，使正则化对重参数化不变
+- **Fisher 感知的学习率调度**：$\|g^{-1}\nabla L\|_g$ 作为"几何正确"的梯度范数，指导学习率选择
+
+## 工程可行性
+
+GPU 友好度取决于度量的结构化程度：
+- **对角度量** $g = \text{diag}(g_1, \ldots, g_n)$：逐元素乘除，$O(n)$，完美 GPU 友好
+- **Kronecker 因子化** $g = A \otimes B$：$(A\otimes B)^{-1} = A^{-1}\otimes B^{-1}$，小矩阵求逆 + GEMM 链，K-FAC 的核心 trick，GPU 可行
+- **块对角度量**：逐块独立求逆，batched 小矩阵运算，GPU 友好
+- **全稠密度量**：$n \times n$ 矩阵求逆 $O(n^3)$ + 显存 $O(n^2)$，参数量 $N \sim 10^9$ 时直接出局
+- 低精度风险：Fisher 矩阵常病态，fp16 下求逆灾难性放大误差，**必须加阻尼 $F + \lambda I$ 并在 fp32 下求逆**
+
+## 风险与失效条件
+
+- **度量矩阵病态**：Fisher 矩阵的条件数可达 $10^6$ 以上，低精度求逆结果全是噪声
+- **物化全度量矩阵**：$N \times N$ 矩阵（$N \sim 10^9$）需 $\sim 4$ PB 显存，不可能物化
+- **度量与任务不匹配**：Fisher 度量假设概率模型正确，模型误设定时自然梯度可能比 SGD 更差
+- **动态度量更新开销**：Fisher 矩阵随参数变化，每步重新估计的统计噪声可能抵消预条件收益
+
+## 深入参考
+
+- 蒸馏稿：references/books/differential-geometry.md（Ch 7 §7.6 Metric Tensors, Ch 13 §13.1 Levi-Civita）
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 13 Riemannian Metrics）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §7.6 Metric Tensors
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 13（黎曼度量、升降指标 ♯/♭）

package/knowledge-base/differential-geometry/tangent-space.en.md

@@ -0,0 +1,48 @@
+# Tangent Space
+
+## Minimal Definition
+
+The tangent space $T_pM$ is the $n$-dimensional vector space of all tangent vectors to a manifold $M$ at a point $p$, serving as the local linearization of the manifold. The differential (pushforward) of a smooth map $f: M \to N$ at $p$, denoted $df_p: T_pM \to T_{f(p)}N$, is a linear map between tangent spaces.
+
+## Core Formulas
+
+- Tangent vector as a derivation: $v(f) = \sum_i v^i \frac{\partial f}{\partial x^i}\bigg|_p$
+- Pushforward (differential): $df_p: T_pM \to T_{f(p)}N$, with coordinate representation given by the Jacobian matrix $J_f(p)$
+- Tangent bundle: $TM = \bigsqcup_{p \in M} T_pM$
+- Cotangent space (the true gradient): $df \in T_p^*M$, which requires the metric to raise the index via $\sharp$ to obtain a tangent vector
+
+## Applicable Problems
+
+- Geometric interpretation of backpropagation: the chain rule equals pushforward along composed maps
+- Correct gradient direction computation: autodiff outputs are covectors (1-forms), requiring a metric to convert them into descent directions
+- Gradient projection in constrained optimization: projecting the Euclidean gradient onto the tangent space of the constraint submanifold
+- Linearized approximations on manifolds: using linear methods within the tangent space to handle local problems
+
+## AI Design Translation
+
+- **Natural gradient layer**: $\tilde{\nabla} L = g^{-1} \nabla L$, using the Fisher metric to raise the covector (autodiff output) to a tangent vector, invariant under reparameterization
+- **Tangent space projection module**: Under orthogonality/Stiefel constraints, project the gradient onto the tangent space $W\Omega$ (where $\Omega$ is skew-symmetric) to maintain constraints
+- **Jacobian-vector product (JVP) acceleration**: The pushforward $df_p(v)$ naturally corresponds to the JVP, serving as the geometric prototype for forward-mode AD
+- **Tangent space feature representation**: In manifold optimization, store momentum/historical gradients in the tangent space and transport them across points via vector transport
+
+## Engineering Feasibility
+
+High GPU friendliness. The core operations of the tangent space are linear algebra:
+- Pushforward $df_p(v) = Jv$: matrix-vector multiplication, $O(n^2)$, naturally batched GEMM
+- Pullback $df_p^*(\omega) = J^T \omega$: transposed matrix-vector multiplication, which is backpropagation itself
+- Tangent space projection $P = I - WW^T$ (Stiefel): a chain of matrix multiplications, GPU-friendly
+- Metric index raising $g^{-1}\nabla L$: depends on the structure of $g$ -- diagonal/Kronecker-factored yields $O(n)$ to $O(n^2)$, full matrix $O(n^3)$ is infeasible
+
+## Risks and Failure Conditions
+
+- **Confusing gradient with descent direction**: Forgetting the metric index-raising and directly using the raw autodiff output (covector) as a descent direction (tangent vector) leads to incorrect directions in curved spaces
+- **Large matrix inversion**: Natural gradients require $g^{-1}$; the full Fisher matrix $O(N^3)$ is infeasible and necessitates Kronecker/block-diagonal/low-rank factorization
+- **Confusing tangent space with the ambient space**: Performing tangent-space vector addition directly on a curved manifold ignores nonlinear deviations caused by curvature
+- **Missing vector transport**: Tangent spaces at different points cannot be directly summed; momentum/Adam states require parallel transport to be carried across steps
+
+## Further References
+
+- Distillation notes: references/books/smooth-manifolds.md (Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle)
+- Distillation notes: references/books/differential-geometry.md (Ch 1-2, Ch 7 Tensors)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 3 (tangent spaces, pushforward, tangent bundle)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 11 (cotangent bundle, 1-forms, pullback)

package/knowledge-base/differential-geometry/tangent-space.md

@@ -0,0 +1,48 @@
+# 切空间 (Tangent Space)
+
+## 最小定义
+
+切空间 $T_pM$ 是流形 $M$ 在点 $p$ 处的所有切向量构成的 $n$ 维向量空间，是流形的局部线性化。光滑映射 $f: M \to N$ 在 $p$ 处的微分（pushforward）$df_p: T_pM \to T_{f(p)}N$ 是切空间之间的线性映射。
+
+## 核心公式
+
+- 切向量作为导子：$v(f) = \sum_i v^i \frac{\partial f}{\partial x^i}\bigg|_p$
+- Pushforward（微分）：$df_p: T_pM \to T_{f(p)}N$，坐标表示为 Jacobian 矩阵 $J_f(p)$
+- 切丛：$TM = \bigsqcup_{p \in M} T_pM$
+- 余切空间（梯度真身）：$df \in T_p^*M$，需用度量升指标 $\sharp$ 变为切向量
+
+## 适用问题
+
+- 反向传播的几何理解：链式法则 = 沿复合映射做 pushforward
+- 梯度方向的正确计算：autodiff 输出是余向量（1-form），需要度量才能变成下降方向
+- 约束优化中梯度投影：将欧氏梯度投影到约束子流形的切空间
+- 流形上的线性化近似：在切空间中用线性方法处理局部问题
+
+## AI 设计翻译
+
+- **自然梯度层**：$\tilde{\nabla} L = g^{-1} \nabla L$，用 Fisher 度量将余向量（autodiff 输出）升为切向量，对重参数化不变
+- **切空间投影模块**：在正交/Stiefel 约束下，将梯度投影到切空间 $W\Omega$（$\Omega$ 反对称），维持约束
+- **Jacobian-vector product (JVP) 加速**：pushforward $df_p(v)$ 天然对应 JVP，是 forward-mode AD 的几何原型
+- **切空间特征表示**：在流形优化中，将动量/历史梯度存放在切空间中，通过 vector transport 跨点搬运
+
+## 工程可行性
+
+GPU 友好度高。切空间的核心操作是线性代数：
+- Pushforward $df_p(v) = Jv$：矩阵-向量乘法，$O(n^2)$，天然 batched GEMM
+- Pullback $df_p^*(\omega) = J^T \omega$：转置矩阵-向量乘法，即反向传播本身
+- 切空间投影 $P = I - WW^T$（Stiefel）：矩阵乘法链，GPU 友好
+- 度量升指标 $g^{-1}\nabla L$：取决于 $g$ 的结构——对角/Kronecker 因子化时 $O(n)$~$O(n^2)$，全矩阵 $O(n^3)$ 不可行
+
+## 风险与失效条件
+
+- **混淆梯度与下降方向**：忘记度量升指标，把 raw autodiff 输出（余向量）直接当下降方向（切向量），在弯曲空间中方向错误
+- **大矩阵求逆**：自然梯度需要 $g^{-1}$，全 Fisher 矩阵 $O(N^3)$ 不可行，必须做 Kronecker/块对角/低秩因子化
+- **切空间与原空间混淆**：在弯曲流形上直接做切空间的向量加法，忽略曲率导致的非线性偏差
+- **Vector transport 缺失**：不同点的切空间无法直接相加，动量/Adam 状态需要 parallel transport 才能跨步搬运
+
+## 深入参考
+
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 1-2, Ch 7 Tensors）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 3（切空间、pushforward、切丛）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 11（余切丛、1-form、pullback）

package/knowledge-base/information-geometry/fisher-metric.en.md

@@ -0,0 +1,56 @@
+# Fisher-Rao Metric
+
+## Minimal Definition
+The Fisher-Rao metric is a **Riemannian metric** on the parameter space of a family of probability distributions (statistical model), whose metric tensor is precisely the Fisher information matrix. It endows the parameter space with an intrinsic geometric structure, enabling the "distance" between probability distributions to be described using geometric language (geodesics, curvature, connections). It is the core structure of information geometry.
+
+## Core Formulas
+
+**Metric Tensor (i.e., Fisher Information Matrix)**:
+$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right]$$
+
+**Line Element (infinitesimal distance between distributions)**:
+$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j = 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta})$$
+
+**Geodesic Distance (finite distance between distributions)**:
+$$d(p_{\theta_1}, p_{\theta_2}) = \inf_{\gamma} \int_0^1 \sqrt{\dot{\gamma}^T \mathcal{I}(\gamma(t)) \dot{\gamma}} \, dt$$
+
+**$\alpha$-Connection Family** (different notions of "straight lines"):
+$$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \frac{1-\alpha}{2} \partial_i \ell \, \partial_j \ell\right) \partial_k \ell\right]$$
+- $\alpha = 0$: Levi-Civita connection (metric-compatible), corresponding to "midpoint" symmetry
+- $\alpha = 1$: $e$-connection (exponential connection), corresponding to straight lines in the natural parameters of the exponential family
+- $\alpha = -1$: $m$-connection (mixture connection), corresponding to straight lines in the expectation parameters of the mixture family
+
+**Dually Flat Structure**: $(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ forms a dually flat manifold — the $e$-connection and $m$-connection are dual to each other with respect to $g$, and the generalized Pythagorean theorem holds.
+
+## Applicable Problems
+- **Geometric distance between distributions**: Comparing the "intrinsic difference" between two probabilistic models (e.g., output distributions of two language models)
+- **Statistical model complexity measurement**: The volume element $\sqrt{\det \mathcal{I}(\theta)} \, d\theta$ induced by the Fisher metric is used for model complexity penalties in MDL/BIC
+- **Parameterization-invariant optimization**: Ensuring that the behavior of optimization algorithms does not depend on the specific choice of parameterization (reparameterization invariance)
+
+## AI Design Translation
+- **Wasserstein vs. Fisher-Rao in generative models**: GANs use the Wasserstein distance to measure distributional differences; the Fisher-Rao metric provides an alternative — performing geometry-aware optimization in the parameter space of the distribution family
+- **Geometry of pretrained model space**: Treating different checkpoints as points on the statistical manifold, Fisher geodesic distances can be used for model selection, model merging, and interpolation path planning
+- **Geometric analysis of MoE expert distributions**: The degree of separation between the output distributions of different experts under the Fisher metric can quantify expert diversity
+
+## Engineering Feasibility
+- **Dimension 1 Tensorization ❌**: The full metric tensor $g_{ij}$ is $d \times d$; infeasible to materialize when $d \sim 10^{10}$
+- **Dimension 2 GEMM-mappability ⚠️**: Kronecker/diagonal approximations can be mapped; the exact metric cannot
+- **Dimension 3 Complexity ❌**: Geodesic computation requires solving a second-order ODE; exact computation is intractable
+- **Dimension 4 Memory ❌**: Full metric tensor storage is $O(d^2)$; completely impossible at LLM scale
+- **Dimension 5 Low Precision ⚠️**: The condition number of the metric tensor may be very large, leading to instability under low precision
+- **Dimension 6 Parallelism ⚠️**: Approximate versions (K-FAC, diagonal) can be parallelized; exact versions cannot
+- **Dimension 7 Sparsity ⚠️**: The Fisher information matrix is typically dense; block-diagonal approximations (inter-layer independence) introduce structured sparsity
+- **Dimension 8 Operator Fusion ✅**: Approximate versions can be fused into optimizer updates
+
+**Conclusion**: The exact Fisher metric is infeasible at LLM scale, but **approximate versions** (K-FAC, diagonal Fisher, low-rank) are engineering-viable. The theoretical value of information geometry lies primarily in **guiding design** rather than direct computation.
+
+## Risks and Failure Conditions
+- **Computational complexity is prohibitive**: The metric tensor in a $d$-dimensional parameter space has $O(d^2)$ independent components, which is unaffordable at LLM scale. All practical approaches must use approximations (diagonal, Kronecker, low-rank), and the quality of the approximation determines the practical effectiveness.
+- **Singularities of the statistical manifold**: In certain regions of the parameter space (e.g., degenerate points of mixture distributions), the Fisher metric may degenerate ($\det \mathcal{I} = 0$), causing geodesic distances to be undefined. This type of degeneracy arises in MoE when an expert's weight is zero.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Amari & Nagaoka. *Methods of Information Geometry*. AMS/Oxford, 2000
+- Amari. *Information Geometry and Its Applications*. Springer, 2016
+- Ay, Jost, Le, Schwachhofer. *Information Geometry*. Springer, 2017
+- Related knowledge cards: `probability/fisher-information.md`, `information-geometry/natural-gradient.md`

package/knowledge-base/information-geometry/fisher-metric.md

@@ -0,0 +1,56 @@
+# Fisher 度量 / Fisher-Rao Metric
+
+## 最小定义
+Fisher-Rao 度量是概率分布族（统计模型）参数空间上的**黎曼度量**，其度量张量恰好是 Fisher 信息矩阵。它赋予参数空间一个内蕴几何结构，使得概率分布之间的"距离"可以用几何语言（测地线、曲率、联络）来描述，是信息几何的核心结构。
+
+## 核心公式
+
+**度量张量（即 Fisher 信息矩阵）**：
+$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right]$$
+
+**线元（分布间的无穷小距离）**：
+$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j = 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta})$$
+
+**测地线距离（有限分布间距离）**：
+$$d(p_{\theta_1}, p_{\theta_2}) = \inf_{\gamma} \int_0^1 \sqrt{\dot{\gamma}^T \mathcal{I}(\gamma(t)) \dot{\gamma}} \, dt$$
+
+**$\alpha$-联络族**（定义"直线"的不同方式）：
+$$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \frac{1-\alpha}{2} \partial_i \ell \, \partial_j \ell\right) \partial_k \ell\right]$$
+- $\alpha = 0$：Levi-Civita 联络（度量兼容），对应"中点"对称
+- $\alpha = 1$：$e$-联络（指数联络），对应指数族的自然参数直线
+- $\alpha = -1$：$m$-联络（混合联络），对应混合族的期望参数直线
+
+**对偶平坦性**：$(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ 构成对偶平坦流形——$e$-联络和 $m$-联络关于 $g$ 互为对偶，广义勾股定理成立。
+
+## 适用问题
+- **分布间的几何距离**：比较两个概率模型（如两个语言模型的输出分布）的"本质差异"
+- **统计模型复杂度度量**：Fisher 度量诱导的体积元 $\sqrt{\det \mathcal{I}(\theta)} \, d\theta$ 用于 MDL/BIC 中的模型复杂度惩罚
+- **参数化无关优化**：确保优化算法的行为不依赖于参数的具体选取方式（reparameterization invariance）
+
+## AI 设计翻译
+- **Wasserstein vs. Fisher-Rao 在生成模型中**：GAN 用 Wasserstein 距离度量分布差异；Fisher-Rao 度量提供替代方案——在分布族的参数空间上做几何感知优化
+- **预训练模型空间的几何**：将不同 checkpoint 视为统计流形上的点，Fisher 测地线距离可用于模型选择、合并（model merging）和插值路径规划
+- **MoE 专家分布的几何分析**：不同专家的输出分布在 Fisher 度量下的分离度可量化专家多样性
+
+## 工程可行性
+- **维度 1 张量化 ❌**：完整度量张量 $g_{ij}$ 是 $d \times d$，$d \sim 10^{10}$ 时不可物化
+- **维度 2 GEMM 可映射 ⚠️**：Kronecker/对角近似可，精确度量不可
+- **维度 3 复杂度 ❌**：测地线计算需解二阶 ODE，精确计算不可行
+- **维度 4 显存 ❌**：完整度量张量存储 $O(d^2)$，LLM 级完全不可能
+- **维度 5 低精度 ⚠️**：度量张量的条件数可能很大，低精度下不稳定
+- **维度 6 并行 ⚠️**：近似版本（K-FAC、对角）可并行，精确版本不可
+- **维度 7 稀疏 ⚠️**：Fisher 信息矩阵通常稠密；块对角近似（层间独立）引入结构化稀疏
+- **维度 8 算子融合 ✅**：近似版本可融入优化器更新
+
+**结论**：精确 Fisher 度量在 LLM 规模下不可行，但**近似版本**（K-FAC、对角 Fisher、低秩）在工程上可落地。信息几何的理论价值主要在**指导设计**而非直接计算。
+
+## 风险与失效条件
+- **计算复杂度致命**：$d$ 维参数空间的度量张量有 $O(d^2)$ 个独立分量，LLM 级无法承受。所有实用方案必须做近似（对角、Kronecker、低秩），近似质量决定实际效果。
+- **统计流形的奇点**：在参数空间的某些区域（如混合分布的退化点），Fisher 度量可能退化（$\det \mathcal{I} = 0$），导致测地线距离不定义。在 MoE 中某专家权重为零时即出现此类退化。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- Amari & Nagaoka. *Methods of Information Geometry*. AMS/Oxford, 2000
+- Amari. *Information Geometry and Its Applications*. Springer, 2016
+- Ay, Jost, Le, Schwachhofer. *Information Geometry*. Springer, 2017
+- 关联知识卡：`probability/fisher-information.md`、`information-geometry/natural-gradient.md`

package/knowledge-base/information-geometry/natural-gradient.en.md

@@ -0,0 +1,54 @@
+# Natural Gradient
+
+## Minimal Definition
+The natural gradient is the steepest descent direction with respect to the **Fisher information metric** on the parameter space. Unlike the naive gradient defined under the Euclidean metric, the natural gradient is defined under the Riemannian metric of the statistical manifold, making it invariant to the choice of parameterization (reparameterization invariant) and automatically adaptive to the curvature structure of the loss surface.
+
+## Core Formulas
+
+**Naive Gradient Descent** (Euclidean metric):
+$$\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$$
+
+**Natural Gradient Descent** (Fisher metric):
+$$\tilde{\nabla} \mathcal{L}(\theta) = \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+$$\theta_{t+1} = \theta_t - \eta \, \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+
+where $\mathcal{I}(\theta)$ is the Fisher information matrix (see `probability/fisher-information.md`).
+
+**Equivalent Derivation (Constrained Optimization Perspective)**: The natural gradient is the solution to the following constrained optimization problem —
+$$\min_{\Delta\theta} \mathcal{L}(\theta + \Delta\theta) \quad \text{s.t.} \quad D_{KL}(p_\theta \| p_{\theta+\Delta\theta}) \leq \epsilon$$
+
+Using the second-order expansion $D_{KL} \approx \frac{1}{2} \Delta\theta^T \mathcal{I} \Delta\theta$, solving via Lagrangian yields the natural gradient.
+
+**K-FAC Approximation** (Kronecker-Factored Approximate Curvature):
+$$\mathcal{I}_l \approx A_l \otimes B_l$$
+where $A_l = \mathbb{E}[a_l a_l^T]$ (activation covariance) and $B_l = \mathbb{E}[g_l g_l^T]$ (gradient covariance), computed and inverted independently per layer.
+
+## Applicable Problems
+- **Ill-conditioned loss surface optimization**: When the Hessian condition number is large (narrow canyon), the natural gradient updates along the canyon floor, avoiding oscillation
+- **Distributional parameter learning**: Updates to posterior parameters in variational inference; the natural gradient automatically handles the curvature of the Fisher-Rao manifold
+- **Policy gradient (RL)**: The trust-region constraint in TRPO/PPO is equivalent to a step-size-limited version of the natural gradient
+
+## AI Design Translation
+- **K-FAC Optimizer**: Uses Kronecker decomposition to approximate the FIM, enabling approximate second-order optimization. Each layer maintains $(A_l, B_l)$, with inverse $A_l^{-1} \otimes B_l^{-1}$, reducing matrix inversion complexity from $O(d^3)$ to $O(d_A^3 + d_B^3)$
+- **TRPO Trust-Region Policy Gradient**: Policy updates under the constraint $D_{KL}(\pi_{\theta_{\text{old}}} \| \pi_\theta) \leq \delta$, essentially natural gradient + line search
+- **Natural Gradient in Variational Inference**: For exponential family parameters, the natural gradient equals the difference in expected sufficient statistics, avoiding Fisher matrix inversion
+
+## Engineering Feasibility
+- **Dimension 1 Tensorization ⚠️**: The Kronecker factors of the FIM are dense matrices and can be tensorized; the full FIM cannot
+- **Dimension 2 GEMM-mappability ✅**: K-FAC's $A_l^{-1} (\nabla W_l) B_l^{-1}$ is two matrix multiplications, naturally GEMM
+- **Dimension 3 Complexity ⚠️**: K-FAC adds $O(d_A^2 + d_B^2)$ per-layer covariance estimation + $O(d_A^3 + d_B^3)$ matrix inversion; diagonal approximation is $O(d)$
+- **Dimension 4 Memory ⚠️**: Requires additional storage of $A_l$ and $B_l$ per layer ($O(d_A^2 + d_B^2)$); acceptable for LLMs but non-trivial
+- **Dimension 5 Low Precision ⚠️**: Matrix inversion may be unstable in fp16; fp32 or Tikhonov regularization $(A + \epsilon I)^{-1}$ is needed
+- **Dimension 6 Parallelism ✅**: Kronecker factors for each layer are computed independently; fully parallel across layers
+- **Dimension 8 Operator Fusion ✅**: Natural gradient updates can be fused into the parameter update kernel
+
+## Risks and Failure Conditions
+- **K-FAC's inter-layer independence assumption is overly strong**: It assumes the Fisher information is block-diagonal across layers, ignoring inter-layer correlations. In deep networks, this may underestimate the effective curvature, leading to excessively large steps. Line search or trust-region safeguards are needed.
+- **Burn-in problem for covariance estimation**: In early training, $A_l, B_l$ estimates are inaccurate and the natural gradient direction may be wrong. The standard practice is to warm up with Adam/SGD for the first few hundred steps, then switch to K-FAC.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Amari. *Natural Gradient Works Efficiently in Learning.* Neural Computation, 1998
+- Martens & Grosse. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
+- Schulman et al. "Trust Region Policy Optimization." *ICML*, 2015
+- Related knowledge cards: `probability/fisher-information.md`, `information-geometry/fisher-metric.md`

package/knowledge-base/information-geometry/natural-gradient.md

@@ -0,0 +1,54 @@
+# 自然梯度 / Natural Gradient
+
+## 最小定义
+自然梯度是参数空间上关于**Fisher 信息度量**的最速下降方向。与朴素梯度在欧氏度量下定义不同，自然梯度在统计流形的黎曼度量下定义，因此对参数化方式的选择不敏感（reparameterization invariant），且能自动适应损失面的曲率结构。
+
+## 核心公式
+
+**朴素梯度下降**（欧氏度量）：
+$$\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$$
+
+**自然梯度下降**（Fisher 度量）：
+$$\tilde{\nabla} \mathcal{L}(\theta) = \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+$$\theta_{t+1} = \theta_t - \eta \, \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+
+其中 $\mathcal{I}(\theta)$ 为 Fisher 信息矩阵（详见 `probability/fisher-information.md`）。
+
+**等价推导（约束优化视角）**：自然梯度是以下约束优化问题的解——
+$$\min_{\Delta\theta} \mathcal{L}(\theta + \Delta\theta) \quad \text{s.t.} \quad D_{KL}(p_\theta \| p_{\theta+\Delta\theta}) \leq \epsilon$$
+
+用二阶展开 $D_{KL} \approx \frac{1}{2} \Delta\theta^T \mathcal{I} \Delta\theta$，拉格朗日求解即得自然梯度。
+
+**K-FAC 近似**（Kronecker-Factored Approximate Curvature）：
+$$\mathcal{I}_l \approx A_l \otimes B_l$$
+其中 $A_l = \mathbb{E}[a_l a_l^T]$（激活协方差），$B_l = \mathbb{E}[g_l g_l^T]$（梯度协方差），逐层独立计算和求逆。
+
+## 适用问题
+- **病态损失面优化**：当 Hessian 条件数很大（狭长峡谷）时，自然梯度沿峡谷底部方向更新，避免震荡
+- **分布参数学习**：变分推断中后验参数的更新，自然梯度自动处理 Fisher-Rao 流形的曲率
+- **策略梯度（RL）**：TRPO/PPO 的信任域约束等价于自然梯度的步长限制版本
+
+## AI 设计翻译
+- **K-FAC 优化器**：用 Kronecker 分解近似 FIM，实现近似二阶优化。每层维护 $(A_l, B_l)$，逆为 $A_l^{-1} \otimes B_l^{-1}$，矩阵逆复杂度从 $O(d^3)$ 降至 $O(d_A^3 + d_B^3)$
+- **TRPO 信任域策略梯度**：$D_{KL}(\pi_{\theta_{\text{old}}} \| \pi_\theta) \leq \delta$ 约束下的策略更新，本质是自然梯度 + 线搜索
+- **变分推断中的自然梯度**：指数族参数的自然梯度 = 充分统计量的期望差，避免 Fisher 矩阵求逆
+
+## 工程可行性
+- **维度 1 张量化 ⚠️**：FIM 的 Kronecker 因子为稠密矩阵，可张量化；完整 FIM 不可
+- **维度 2 GEMM 可映射 ✅**：K-FAC 的 $A_l^{-1} (\nabla W_l) B_l^{-1}$ 是两次矩阵乘，天然 GEMM
+- **维度 3 复杂度 ⚠️**：K-FAC 每层额外 $O(d_A^2 + d_B^2)$ 协方差估计 + $O(d_A^3 + d_B^3)$ 矩阵逆；对角近似 $O(d)$
+- **维度 4 显存 ⚠️**：需额外存储每层的 $A_l$ 和 $B_l$（$O(d_A^2 + d_B^2)$），对 LLM 可接受但非零
+- **维度 5 低精度 ⚠️**：矩阵求逆在 fp16 下可能不稳定，需 fp32 或 Tikhonov 正则化 $(A + \epsilon I)^{-1}$
+- **维度 6 并行 ✅**：各层 Kronecker 因子独立计算，层间完全并行
+- **维度 8 算子融合 ✅**：自然梯度更新可融入参数更新 kernel
+
+## 风险与失效条件
+- **K-FAC 的层间独立性假设过强**：假设各层 Fisher 信息块对角，忽略层间相关性。在深网络中可能低估有效曲率，导致步长过大。需搭配线搜索或 trust-region 安全机制。
+- **协方差估计的 burn-in 问题**：训练初期 $A_l, B_l$ 估计不准确，自然梯度方向可能错误。标准做法是前几百步用 Adam/SGD warmup，再切换到 K-FAC。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- Amari. *Natural Gradient Works Efficiently in Learning.* Neural Computation, 1998
+- Martens & Grosse. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
+- Schulman et al. "Trust Region Policy Optimization." *ICML*, 2015
+- 关联知识卡：`probability/fisher-information.md`、`information-geometry/fisher-metric.md`