math-skill 2.1.0 → 3.0.1

package/knowledge-base/differential-geometry/tangent-space.en.md

@@ -0,0 +1,48 @@
+# Tangent Space
+
+## Minimal Definition
+
+The tangent space $T_pM$ is the $n$-dimensional vector space of all tangent vectors to a manifold $M$ at a point $p$, serving as the local linearization of the manifold. The differential (pushforward) of a smooth map $f: M \to N$ at $p$, denoted $df_p: T_pM \to T_{f(p)}N$, is a linear map between tangent spaces.
+
+## Core Formulas
+
+- Tangent vector as a derivation: $v(f) = \sum_i v^i \frac{\partial f}{\partial x^i}\bigg|_p$
+- Pushforward (differential): $df_p: T_pM \to T_{f(p)}N$, with coordinate representation given by the Jacobian matrix $J_f(p)$
+- Tangent bundle: $TM = \bigsqcup_{p \in M} T_pM$
+- Cotangent space (the true gradient): $df \in T_p^*M$, which requires the metric to raise the index via $\sharp$ to obtain a tangent vector
+
+## Applicable Problems
+
+- Geometric interpretation of backpropagation: the chain rule equals pushforward along composed maps
+- Correct gradient direction computation: autodiff outputs are covectors (1-forms), requiring a metric to convert them into descent directions
+- Gradient projection in constrained optimization: projecting the Euclidean gradient onto the tangent space of the constraint submanifold
+- Linearized approximations on manifolds: using linear methods within the tangent space to handle local problems
+
+## AI Design Translation
+
+- **Natural gradient layer**: $\tilde{\nabla} L = g^{-1} \nabla L$, using the Fisher metric to raise the covector (autodiff output) to a tangent vector, invariant under reparameterization
+- **Tangent space projection module**: Under orthogonality/Stiefel constraints, project the gradient onto the tangent space $W\Omega$ (where $\Omega$ is skew-symmetric) to maintain constraints
+- **Jacobian-vector product (JVP) acceleration**: The pushforward $df_p(v)$ naturally corresponds to the JVP, serving as the geometric prototype for forward-mode AD
+- **Tangent space feature representation**: In manifold optimization, store momentum/historical gradients in the tangent space and transport them across points via vector transport
+
+## Engineering Feasibility
+
+High GPU friendliness. The core operations of the tangent space are linear algebra:
+- Pushforward $df_p(v) = Jv$: matrix-vector multiplication, $O(n^2)$, naturally batched GEMM
+- Pullback $df_p^*(\omega) = J^T \omega$: transposed matrix-vector multiplication, which is backpropagation itself
+- Tangent space projection $P = I - WW^T$ (Stiefel): a chain of matrix multiplications, GPU-friendly
+- Metric index raising $g^{-1}\nabla L$: depends on the structure of $g$ -- diagonal/Kronecker-factored yields $O(n)$ to $O(n^2)$, full matrix $O(n^3)$ is infeasible
+
+## Risks and Failure Conditions
+
+- **Confusing gradient with descent direction**: Forgetting the metric index-raising and directly using the raw autodiff output (covector) as a descent direction (tangent vector) leads to incorrect directions in curved spaces
+- **Large matrix inversion**: Natural gradients require $g^{-1}$; the full Fisher matrix $O(N^3)$ is infeasible and necessitates Kronecker/block-diagonal/low-rank factorization
+- **Confusing tangent space with the ambient space**: Performing tangent-space vector addition directly on a curved manifold ignores nonlinear deviations caused by curvature
+- **Missing vector transport**: Tangent spaces at different points cannot be directly summed; momentum/Adam states require parallel transport to be carried across steps
+
+## Further References
+
+- Distillation notes: references/books/smooth-manifolds.md (Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle)
+- Distillation notes: references/books/differential-geometry.md (Ch 1-2, Ch 7 Tensors)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 3 (tangent spaces, pushforward, tangent bundle)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 11 (cotangent bundle, 1-forms, pullback)

package/knowledge-base/differential-geometry/tangent-space.md

@@ -0,0 +1,48 @@
+# 切空间 (Tangent Space)
+
+## 最小定义
+
+切空间 $T_pM$ 是流形 $M$ 在点 $p$ 处的所有切向量构成的 $n$ 维向量空间，是流形的局部线性化。光滑映射 $f: M \to N$ 在 $p$ 处的微分（pushforward）$df_p: T_pM \to T_{f(p)}N$ 是切空间之间的线性映射。
+
+## 核心公式
+
+- 切向量作为导子：$v(f) = \sum_i v^i \frac{\partial f}{\partial x^i}\bigg|_p$
+- Pushforward（微分）：$df_p: T_pM \to T_{f(p)}N$，坐标表示为 Jacobian 矩阵 $J_f(p)$
+- 切丛：$TM = \bigsqcup_{p \in M} T_pM$
+- 余切空间（梯度真身）：$df \in T_p^*M$，需用度量升指标 $\sharp$ 变为切向量
+
+## 适用问题
+
+- 反向传播的几何理解：链式法则 = 沿复合映射做 pushforward
+- 梯度方向的正确计算：autodiff 输出是余向量（1-form），需要度量才能变成下降方向
+- 约束优化中梯度投影：将欧氏梯度投影到约束子流形的切空间
+- 流形上的线性化近似：在切空间中用线性方法处理局部问题
+
+## AI 设计翻译
+
+- **自然梯度层**：$\tilde{\nabla} L = g^{-1} \nabla L$，用 Fisher 度量将余向量（autodiff 输出）升为切向量，对重参数化不变
+- **切空间投影模块**：在正交/Stiefel 约束下，将梯度投影到切空间 $W\Omega$（$\Omega$ 反对称），维持约束
+- **Jacobian-vector product (JVP) 加速**：pushforward $df_p(v)$ 天然对应 JVP，是 forward-mode AD 的几何原型
+- **切空间特征表示**：在流形优化中，将动量/历史梯度存放在切空间中，通过 vector transport 跨点搬运
+
+## 工程可行性
+
+GPU 友好度高。切空间的核心操作是线性代数：
+- Pushforward $df_p(v) = Jv$：矩阵-向量乘法，$O(n^2)$，天然 batched GEMM
+- Pullback $df_p^*(\omega) = J^T \omega$：转置矩阵-向量乘法，即反向传播本身
+- 切空间投影 $P = I - WW^T$（Stiefel）：矩阵乘法链，GPU 友好
+- 度量升指标 $g^{-1}\nabla L$：取决于 $g$ 的结构——对角/Kronecker 因子化时 $O(n)$~$O(n^2)$，全矩阵 $O(n^3)$ 不可行
+
+## 风险与失效条件
+
+- **混淆梯度与下降方向**：忘记度量升指标，把 raw autodiff 输出（余向量）直接当下降方向（切向量），在弯曲空间中方向错误
+- **大矩阵求逆**：自然梯度需要 $g^{-1}$，全 Fisher 矩阵 $O(N^3)$ 不可行，必须做 Kronecker/块对角/低秩因子化
+- **切空间与原空间混淆**：在弯曲流形上直接做切空间的向量加法，忽略曲率导致的非线性偏差
+- **Vector transport 缺失**：不同点的切空间无法直接相加，动量/Adam 状态需要 parallel transport 才能跨步搬运
+
+## 深入参考
+
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 1-2, Ch 7 Tensors）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 3（切空间、pushforward、切丛）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 11（余切丛、1-form、pullback）

package/knowledge-base/information-geometry/fisher-metric.en.md

@@ -0,0 +1,56 @@
+# Fisher-Rao Metric
+
+## Minimal Definition
+The Fisher-Rao metric is a **Riemannian metric** on the parameter space of a family of probability distributions (statistical model), whose metric tensor is precisely the Fisher information matrix. It endows the parameter space with an intrinsic geometric structure, enabling the "distance" between probability distributions to be described using geometric language (geodesics, curvature, connections). It is the core structure of information geometry.
+
+## Core Formulas
+
+**Metric Tensor (i.e., Fisher Information Matrix)**:
+$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right]$$
+
+**Line Element (infinitesimal distance between distributions)**:
+$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j = 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta})$$
+
+**Geodesic Distance (finite distance between distributions)**:
+$$d(p_{\theta_1}, p_{\theta_2}) = \inf_{\gamma} \int_0^1 \sqrt{\dot{\gamma}^T \mathcal{I}(\gamma(t)) \dot{\gamma}} \, dt$$
+
+**$\alpha$-Connection Family** (different notions of "straight lines"):
+$$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \frac{1-\alpha}{2} \partial_i \ell \, \partial_j \ell\right) \partial_k \ell\right]$$
+- $\alpha = 0$: Levi-Civita connection (metric-compatible), corresponding to "midpoint" symmetry
+- $\alpha = 1$: $e$-connection (exponential connection), corresponding to straight lines in the natural parameters of the exponential family
+- $\alpha = -1$: $m$-connection (mixture connection), corresponding to straight lines in the expectation parameters of the mixture family
+
+**Dually Flat Structure**: $(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ forms a dually flat manifold — the $e$-connection and $m$-connection are dual to each other with respect to $g$, and the generalized Pythagorean theorem holds.
+
+## Applicable Problems
+- **Geometric distance between distributions**: Comparing the "intrinsic difference" between two probabilistic models (e.g., output distributions of two language models)
+- **Statistical model complexity measurement**: The volume element $\sqrt{\det \mathcal{I}(\theta)} \, d\theta$ induced by the Fisher metric is used for model complexity penalties in MDL/BIC
+- **Parameterization-invariant optimization**: Ensuring that the behavior of optimization algorithms does not depend on the specific choice of parameterization (reparameterization invariance)
+
+## AI Design Translation
+- **Wasserstein vs. Fisher-Rao in generative models**: GANs use the Wasserstein distance to measure distributional differences; the Fisher-Rao metric provides an alternative — performing geometry-aware optimization in the parameter space of the distribution family
+- **Geometry of pretrained model space**: Treating different checkpoints as points on the statistical manifold, Fisher geodesic distances can be used for model selection, model merging, and interpolation path planning
+- **Geometric analysis of MoE expert distributions**: The degree of separation between the output distributions of different experts under the Fisher metric can quantify expert diversity
+
+## Engineering Feasibility
+- **D1[x]**: The full metric tensor $g_{ij}$ is $d \times d$; infeasible to materialize when $d \sim 10^{10}$
+- **D2[~]**: Kronecker/diagonal approximations can be mapped; the exact metric cannot
+- **D3[x]**: Geodesic computation requires solving a second-order ODE; exact computation is intractable
+- **D4[x]**: Full metric tensor storage is $O(d^2)$; completely impossible at LLM scale
+- **D5[~]**: The condition number of the metric tensor may be very large, leading to instability under low precision
+- **D6[~]**: Approximate versions (K-FAC, diagonal) can be parallelized; exact versions cannot
+- **D7[~]**: The Fisher information matrix is typically dense; block-diagonal approximations (inter-layer independence) introduce structured sparsity
+- **D8[v]**: Approximate versions can be fused into optimizer updates
+
+**Conclusion**: The exact Fisher metric is infeasible at LLM scale, but **approximate versions** (K-FAC, diagonal Fisher, low-rank) are engineering-viable. The theoretical value of information geometry lies primarily in **guiding design** rather than direct computation.
+
+## Risks and Failure Conditions
+- **Computational complexity is prohibitive**: The metric tensor in a $d$-dimensional parameter space has $O(d^2)$ independent components, which is unaffordable at LLM scale. All practical approaches must use approximations (diagonal, Kronecker, low-rank), and the quality of the approximation determines the practical effectiveness.
+- **Singularities of the statistical manifold**: In certain regions of the parameter space (e.g., degenerate points of mixture distributions), the Fisher metric may degenerate ($\det \mathcal{I} = 0$), causing geodesic distances to be undefined. This type of degeneracy arises in MoE when an expert's weight is zero.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Amari & Nagaoka. *Methods of Information Geometry*. AMS/Oxford, 2000
+- Amari. *Information Geometry and Its Applications*. Springer, 2016
+- Ay, Jost, Le, Schwachhofer. *Information Geometry*. Springer, 2017
+- Related knowledge cards: `probability/fisher-information.md`, `information-geometry/natural-gradient.md`

package/knowledge-base/information-geometry/fisher-metric.md

@@ -0,0 +1,56 @@
+# Fisher 度量 / Fisher-Rao Metric
+
+## 最小定义
+Fisher-Rao 度量是概率分布族（统计模型）参数空间上的**黎曼度量**，其度量张量恰好是 Fisher 信息矩阵。它赋予参数空间一个内蕴几何结构，使得概率分布之间的"距离"可以用几何语言（测地线、曲率、联络）来描述，是信息几何的核心结构。
+
+## 核心公式
+
+**度量张量（即 Fisher 信息矩阵）**：
+$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right]$$
+
+**线元（分布间的无穷小距离）**：
+$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j = 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta})$$
+
+**测地线距离（有限分布间距离）**：
+$$d(p_{\theta_1}, p_{\theta_2}) = \inf_{\gamma} \int_0^1 \sqrt{\dot{\gamma}^T \mathcal{I}(\gamma(t)) \dot{\gamma}} \, dt$$
+
+**$\alpha$-联络族**（定义"直线"的不同方式）：
+$$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \frac{1-\alpha}{2} \partial_i \ell \, \partial_j \ell\right) \partial_k \ell\right]$$
+- $\alpha = 0$：Levi-Civita 联络（度量兼容），对应"中点"对称
+- $\alpha = 1$：$e$-联络（指数联络），对应指数族的自然参数直线
+- $\alpha = -1$：$m$-联络（混合联络），对应混合族的期望参数直线
+
+**对偶平坦性**：$(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ 构成对偶平坦流形——$e$-联络和 $m$-联络关于 $g$ 互为对偶，广义勾股定理成立。
+
+## 适用问题
+- **分布间的几何距离**：比较两个概率模型（如两个语言模型的输出分布）的"本质差异"
+- **统计模型复杂度度量**：Fisher 度量诱导的体积元 $\sqrt{\det \mathcal{I}(\theta)} \, d\theta$ 用于 MDL/BIC 中的模型复杂度惩罚
+- **参数化无关优化**：确保优化算法的行为不依赖于参数的具体选取方式（reparameterization invariance）
+
+## AI 设计翻译
+- **Wasserstein vs. Fisher-Rao 在生成模型中**：GAN 用 Wasserstein 距离度量分布差异；Fisher-Rao 度量提供替代方案——在分布族的参数空间上做几何感知优化
+- **预训练模型空间的几何**：将不同 checkpoint 视为统计流形上的点，Fisher 测地线距离可用于模型选择、合并（model merging）和插值路径规划
+- **MoE 专家分布的几何分析**：不同专家的输出分布在 Fisher 度量下的分离度可量化专家多样性
+
+## 工程可行性
+- **D1[x]**：完整度量张量 $g_{ij}$ 是 $d \times d$，$d \sim 10^{10}$ 时不可物化
+- **D2[~]**：Kronecker/对角近似可，精确度量不可
+- **D3[x]**：测地线计算需解二阶 ODE，精确计算不可行
+- **D4[x]**：完整度量张量存储 $O(d^2)$，LLM 级完全不可能
+- **D5[~]**：度量张量的条件数可能很大，低精度下不稳定
+- **D6[~]**：近似版本（K-FAC、对角）可并行，精确版本不可
+- **D7[~]**：Fisher 信息矩阵通常稠密；块对角近似（层间独立）引入结构化稀疏
+- **D8[v]**：近似版本可融入优化器更新
+
+**结论**：精确 Fisher 度量在 LLM 规模下不可行，但**近似版本**（K-FAC、对角 Fisher、低秩）在工程上可落地。信息几何的理论价值主要在**指导设计**而非直接计算。
+
+## 风险与失效条件
+- **计算复杂度致命**：$d$ 维参数空间的度量张量有 $O(d^2)$ 个独立分量，LLM 级无法承受。所有实用方案必须做近似（对角、Kronecker、低秩），近似质量决定实际效果。
+- **统计流形的奇点**：在参数空间的某些区域（如混合分布的退化点），Fisher 度量可能退化（$\det \mathcal{I} = 0$），导致测地线距离不定义。在 MoE 中某专家权重为零时即出现此类退化。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- Amari & Nagaoka. *Methods of Information Geometry*. AMS/Oxford, 2000
+- Amari. *Information Geometry and Its Applications*. Springer, 2016
+- Ay, Jost, Le, Schwachhofer. *Information Geometry*. Springer, 2017
+- 关联知识卡：`probability/fisher-information.md`、`information-geometry/natural-gradient.md`

package/knowledge-base/information-geometry/natural-gradient.en.md

@@ -0,0 +1,54 @@
+# Natural Gradient
+
+## Minimal Definition
+The natural gradient is the steepest descent direction with respect to the **Fisher information metric** on the parameter space. Unlike the naive gradient defined under the Euclidean metric, the natural gradient is defined under the Riemannian metric of the statistical manifold, making it invariant to the choice of parameterization (reparameterization invariant) and automatically adaptive to the curvature structure of the loss surface.
+
+## Core Formulas
+
+**Naive Gradient Descent** (Euclidean metric):
+$$\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$$
+
+**Natural Gradient Descent** (Fisher metric):
+$$\tilde{\nabla} \mathcal{L}(\theta) = \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+$$\theta_{t+1} = \theta_t - \eta \, \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+
+where $\mathcal{I}(\theta)$ is the Fisher information matrix (see `probability/fisher-information.md`).
+
+**Equivalent Derivation (Constrained Optimization Perspective)**: The natural gradient is the solution to the following constrained optimization problem —
+$$\min_{\Delta\theta} \mathcal{L}(\theta + \Delta\theta) \quad \text{s.t.} \quad D_{KL}(p_\theta \| p_{\theta+\Delta\theta}) \leq \epsilon$$
+
+Using the second-order expansion $D_{KL} \approx \frac{1}{2} \Delta\theta^T \mathcal{I} \Delta\theta$, solving via Lagrangian yields the natural gradient.
+
+**K-FAC Approximation** (Kronecker-Factored Approximate Curvature):
+$$\mathcal{I}_l \approx A_l \otimes B_l$$
+where $A_l = \mathbb{E}[a_l a_l^T]$ (activation covariance) and $B_l = \mathbb{E}[g_l g_l^T]$ (gradient covariance), computed and inverted independently per layer.
+
+## Applicable Problems
+- **Ill-conditioned loss surface optimization**: When the Hessian condition number is large (narrow canyon), the natural gradient updates along the canyon floor, avoiding oscillation
+- **Distributional parameter learning**: Updates to posterior parameters in variational inference; the natural gradient automatically handles the curvature of the Fisher-Rao manifold
+- **Policy gradient (RL)**: The trust-region constraint in TRPO/PPO is equivalent to a step-size-limited version of the natural gradient
+
+## AI Design Translation
+- **K-FAC Optimizer**: Uses Kronecker decomposition to approximate the FIM, enabling approximate second-order optimization. Each layer maintains $(A_l, B_l)$, with inverse $A_l^{-1} \otimes B_l^{-1}$, reducing matrix inversion complexity from $O(d^3)$ to $O(d_A^3 + d_B^3)$
+- **TRPO Trust-Region Policy Gradient**: Policy updates under the constraint $D_{KL}(\pi_{\theta_{\text{old}}} \| \pi_\theta) \leq \delta$, essentially natural gradient + line search
+- **Natural Gradient in Variational Inference**: For exponential family parameters, the natural gradient equals the difference in expected sufficient statistics, avoiding Fisher matrix inversion
+
+## Engineering Feasibility
+- **D1[~]**: The Kronecker factors of the FIM are dense matrices and can be tensorized; the full FIM cannot
+- **D2[v]**: K-FAC's $A_l^{-1} (\nabla W_l) B_l^{-1}$ is two matrix multiplications, naturally GEMM
+- **D3[~]**: K-FAC adds $O(d_A^2 + d_B^2)$ per-layer covariance estimation + $O(d_A^3 + d_B^3)$ matrix inversion; diagonal approximation is $O(d)$
+- **D4[~]**: Requires additional storage of $A_l$ and $B_l$ per layer ($O(d_A^2 + d_B^2)$); acceptable for LLMs but non-trivial
+- **D5[~]**: Matrix inversion may be unstable in fp16; fp32 or Tikhonov regularization $(A + \epsilon I)^{-1}$ is needed
+- **D6[v]**: Kronecker factors for each layer are computed independently; fully parallel across layers
+- **D8[v]**: Natural gradient updates can be fused into the parameter update kernel
+
+## Risks and Failure Conditions
+- **K-FAC's inter-layer independence assumption is overly strong**: It assumes the Fisher information is block-diagonal across layers, ignoring inter-layer correlations. In deep networks, this may underestimate the effective curvature, leading to excessively large steps. Line search or trust-region safeguards are needed.
+- **Burn-in problem for covariance estimation**: In early training, $A_l, B_l$ estimates are inaccurate and the natural gradient direction may be wrong. The standard practice is to warm up with Adam/SGD for the first few hundred steps, then switch to K-FAC.
+
+## Further References
+- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Amari. *Natural Gradient Works Efficiently in Learning.* Neural Computation, 1998
+- Martens & Grosse. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
+- Schulman et al. "Trust Region Policy Optimization." *ICML*, 2015
+- Related knowledge cards: `probability/fisher-information.md`, `information-geometry/fisher-metric.md`

package/knowledge-base/information-geometry/natural-gradient.md

@@ -0,0 +1,54 @@
+# 自然梯度 / Natural Gradient
+
+## 最小定义
+自然梯度是参数空间上关于**Fisher 信息度量**的最速下降方向。与朴素梯度在欧氏度量下定义不同，自然梯度在统计流形的黎曼度量下定义，因此对参数化方式的选择不敏感（reparameterization invariant），且能自动适应损失面的曲率结构。
+
+## 核心公式
+
+**朴素梯度下降**（欧氏度量）：
+$$\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$$
+
+**自然梯度下降**（Fisher 度量）：
+$$\tilde{\nabla} \mathcal{L}(\theta) = \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+$$\theta_{t+1} = \theta_t - \eta \, \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
+
+其中 $\mathcal{I}(\theta)$ 为 Fisher 信息矩阵（详见 `probability/fisher-information.md`）。
+
+**等价推导（约束优化视角）**：自然梯度是以下约束优化问题的解——
+$$\min_{\Delta\theta} \mathcal{L}(\theta + \Delta\theta) \quad \text{s.t.} \quad D_{KL}(p_\theta \| p_{\theta+\Delta\theta}) \leq \epsilon$$
+
+用二阶展开 $D_{KL} \approx \frac{1}{2} \Delta\theta^T \mathcal{I} \Delta\theta$，拉格朗日求解即得自然梯度。
+
+**K-FAC 近似**（Kronecker-Factored Approximate Curvature）：
+$$\mathcal{I}_l \approx A_l \otimes B_l$$
+其中 $A_l = \mathbb{E}[a_l a_l^T]$（激活协方差），$B_l = \mathbb{E}[g_l g_l^T]$（梯度协方差），逐层独立计算和求逆。
+
+## 适用问题
+- **病态损失面优化**：当 Hessian 条件数很大（狭长峡谷）时，自然梯度沿峡谷底部方向更新，避免震荡
+- **分布参数学习**：变分推断中后验参数的更新，自然梯度自动处理 Fisher-Rao 流形的曲率
+- **策略梯度（RL）**：TRPO/PPO 的信任域约束等价于自然梯度的步长限制版本
+
+## AI 设计翻译
+- **K-FAC 优化器**：用 Kronecker 分解近似 FIM，实现近似二阶优化。每层维护 $(A_l, B_l)$，逆为 $A_l^{-1} \otimes B_l^{-1}$，矩阵逆复杂度从 $O(d^3)$ 降至 $O(d_A^3 + d_B^3)$
+- **TRPO 信任域策略梯度**：$D_{KL}(\pi_{\theta_{\text{old}}} \| \pi_\theta) \leq \delta$ 约束下的策略更新，本质是自然梯度 + 线搜索
+- **变分推断中的自然梯度**：指数族参数的自然梯度 = 充分统计量的期望差，避免 Fisher 矩阵求逆
+
+## 工程可行性
+- **D1[~]**：FIM 的 Kronecker 因子为稠密矩阵，可张量化；完整 FIM 不可
+- **D2[v]**：K-FAC 的 $A_l^{-1} (\nabla W_l) B_l^{-1}$ 是两次矩阵乘，天然 GEMM
+- **D3[~]**：K-FAC 每层额外 $O(d_A^2 + d_B^2)$ 协方差估计 + $O(d_A^3 + d_B^3)$ 矩阵逆；对角近似 $O(d)$
+- **D4[~]**：需额外存储每层的 $A_l$ 和 $B_l$（$O(d_A^2 + d_B^2)$），对 LLM 可接受但非零
+- **D5[~]**：矩阵求逆在 fp16 下可能不稳定，需 fp32 或 Tikhonov 正则化 $(A + \epsilon I)^{-1}$
+- **D6[v]**：各层 Kronecker 因子独立计算，层间完全并行
+- **D8[v]**：自然梯度更新可融入参数更新 kernel
+
+## 风险与失效条件
+- **K-FAC 的层间独立性假设过强**：假设各层 Fisher 信息块对角，忽略层间相关性。在深网络中可能低估有效曲率，导致步长过大。需搭配线搜索或 trust-region 安全机制。
+- **协方差估计的 burn-in 问题**：训练初期 $A_l, B_l$ 估计不准确，自然梯度方向可能错误。标准做法是前几百步用 Adam/SGD warmup，再切换到 K-FAC。
+
+## 深入参考
+- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- Amari. *Natural Gradient Works Efficiently in Learning.* Neural Computation, 1998
+- Martens & Grosse. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
+- Schulman et al. "Trust Region Policy Optimization." *ICML*, 2015
+- 关联知识卡：`probability/fisher-information.md`、`information-geometry/fisher-metric.md`

package/knowledge-base/lie-theory/equivariance.en.md

@@ -0,0 +1,54 @@
+# Equivariance
+
+## Minimal Definition
+
+A map $f: X \to Y$ is equivariant with respect to the action of a group $G$ if $f(g \cdot x) = g \cdot f(x)$ holds for all $g \in G, x \in X$. Equivariance is a finer structure-preserving property than invariance ($f(g \cdot x) = f(x)$): the output "co-moves" with the input under the same group action.
+
+## Core Formulas
+
+- Equivariance condition: $f(\rho_X(g) x) = \rho_Y(g) f(x), \quad \forall g \in G$
+- Invariance = equivariance to the trivial representation: $f(g \cdot x) = f(x)$ ($\rho_Y = \text{id}$)
+- Translation equivariance of convolution: $f(T_a x) = T_a f(x)$, where $T_a$ is the translation operator
+- Gauge equivariance: $f(\alpha \cdot_\omega x) = \alpha \cdot_{f(\omega)} f(x)$, where $\alpha$ is a local gauge transformation and $\omega$ is a connection
+- Adjoint equivariance: $f(X \oplus \tau) = f(X) \oplus (\text{Ad}_X \tau)$ (in the Lie group setting)
+
+## Applicable Problems
+
+- 3D point clouds/molecules: when the input is rotated, the output (segmentation/forces/pose) should rotate accordingly
+- Signal processing on spheres/manifolds: the choice of local coordinates should not affect prediction results
+- Multi-view/multi-sensor: when camera orientation changes, features should covary rather than require relearning
+- Physical simulation: vector quantities such as forces and velocities should rotate correctly under coordinate transformations
+
+## AI Design Translation
+
+- **E(n)-equivariant GNN**: Node features + coordinates; message passing simultaneously updates scalar features and equivariantly updates coordinates $x_i \to x_i + \sum_j \phi(r_{ij}) \cdot (x_i - x_j)$
+- **Gauge-equivariant CNN**: Each edge carries a $G$-connection to align local frames; convolution kernels are invariant to local coordinate choices; applicable to meshes/spheres/graphs
+- **Steerable CNN**: Feature fields are direct sums of group representations $\bigoplus_l \rho_l$; convolution kernels are constrained by Schur's lemma to block structure, yielding few parameters with strict equivariance
+- **Adjoint-equiv output head**: Pose regression $f: X \to SE(3)$ satisfying $f(g \cdot X) = g \cdot f(X)$, implemented using $\exp$ and the Lie algebra
+- **Equivariance verification loss**: $L_{\text{eq}} = \|f(g \cdot x) - g \cdot f(x)\|^2$ as an auxiliary regularizer, enforcing approximate equivariance
+
+## Engineering Feasibility
+
+GPU friendliness depends on the degree of group discretization:
+- **Discrete groups ($C_n$, octahedral group, etc.)**: Group convolution can be expanded into GEMM; equivariance constraints block-diagonalize weights (reducing parameters), GPU-friendly
+- **Translation group (standard CNN)**: Naturally equivariant; weight sharing is the engineering realization of equivariance, perfectly GPU-friendly
+- **Continuous groups SO(3)/SE(3)**: Require frequency-domain expansion (spherical harmonics) or sampling-based discretization; fast algorithms exist for spherical harmonic transforms but implementation is complex
+- **Gauge-equivariant**: One $G$-element action per edge equals small matrix-times-feature-vector, expressible as sparse matmul
+- **Approximate equivariance (regularization)**: The equivariance loss $L_{\text{eq}}$ is a standard MSE, fully GPU-friendly, but equivariance is not exact
+- Key trade-off: strict equivariance (structural constraints) vs. approximate equivariance (regularization) -- the former has fewer parameters but complex implementation, the latter is simple but not guaranteed
+
+## Risks and Failure Conditions
+
+- **Continuous group discretization error**: Improper sampling causes equivariance to silently break, passing verification but failing at inference
+- **Equivariance-expressiveness trade-off**: Strict equivariance constraints reduce the parameter space, potentially insufficient for fitting complex functions
+- **Combinatorial explosion of multiple group actions**: Simultaneously requiring rotation + translation + permutation equivariance leads to complex cross-constraints
+- **Data noise breaking equivariance**: Sensor noise makes the exact computation of $g \cdot x$ unreliable, invalidating the equivariance premise
+- **Numerical precision in equivariant layers**: Floating-point errors in spherical harmonics/CG coefficients at large $l$ break equivariance, requiring fp32 accumulation
+- **Over-constraining with equivariance**: When a task only requires approximate symmetry, enforcing strict equivariance is inferior to using soft regularization constraints
+
+## Further References
+
+- Distillation notes: references/books/micro-lie-theory.md (Section II-F Adjoint $\text{Ad}_X$, algebraic realization of equivariance)
+- Distillation notes: references/books/differential-geometry.md (Section 6.8 Principal Bundles, Section 12.12 G-Connections, gauge equivariance)
+- Distillation notes: references/books/differential-geometry.md (Ch 5 Lie Groups, continuous symmetry as prior)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 6.8 + Section 12.12 (geometric foundations of gauge equivariance)

package/knowledge-base/lie-theory/equivariance.md

@@ -0,0 +1,54 @@
+# 等变性 (Equivariance)
+
+## 最小定义
+
+映射 $f: X \to Y$ 关于群 $G$ 的作用等变，若 $f(g \cdot x) = g \cdot f(x)$ 对所有 $g \in G, x \in X$ 成立。等变性是比不变性（$f(g \cdot x) = f(x)$）更精细的结构保持：输出随输入按同一群作用"协动"。
+
+## 核心公式
+
+- 等变性条件：$f(\rho_X(g) x) = \rho_Y(g) f(x), \quad \forall g \in G$
+- 不变性 = 等变到平凡表示：$f(g \cdot x) = f(x)$（$\rho_Y = \text{id}$）
+- 卷积的平移等变性：$f(T_a x) = T_a f(x)$，其中 $T_a$ 是平移算子
+- 规范等变性（gauge equivariance）：$f(\alpha \cdot_\omega x) = \alpha \cdot_{f(\omega)} f(x)$，$\alpha$ 是局部规范变换，$\omega$ 是联络
+- 伴随等变性：$f(X \oplus \tau) = f(X) \oplus (\text{Ad}_X \tau)$（李群场景）
+
+## 适用问题
+
+- 3D 点云/分子：输入旋转后，输出（分割/力/位姿）应同样旋转
+- 球面/流形上的信号处理：局部坐标选择不应影响预测结果
+- 多视角/多传感器：相机朝向变化时，特征应协变而非重新学习
+- 物理模拟：力、速度等矢量应随坐标系变换正确旋转
+
+## AI 设计翻译
+
+- **E(n)-等变 GNN**：节点特征 + 坐标，消息传递同时更新标量特征和等变更新坐标 $x_i \to x_i + \sum_j \phi(r_{ij}) \cdot (x_i - x_j)$
+- **Gauge-equivariant CNN**：每条边携带 $G$-联络对齐局部 frame，卷积核对局部坐标选择不变；适用于 mesh/球面/图
+- **Steerable CNN**：特征场是群表示的直和 $\bigoplus_l \rho_l$，卷积核被 Schur 引理约束为块结构，参数少但严格等变
+- **伴随等变输出头**：位姿回归 $f: X \to SE(3)$ 满足 $f(g \cdot X) = g \cdot f(X)$，用 $\exp$ 和李代数实现
+- **等变性验证损失**：$L_{\text{eq}} = \|f(g \cdot x) - g \cdot f(x)\|^2$ 作为辅助正则，强制近似等变
+
+## 工程可行性
+
+GPU 友好度取决于群的离散化程度：
+- **离散群（$C_n$, 八面体群等）**：群卷积可展开为 GEMM，等变约束使权重块对角化（参数减少），GPU 友好
+- **平移群（标准 CNN）**：天然等变，weight sharing 就是等变性的工程实现，完美 GPU 友好
+- **连续群 SO(3)/SE(3)**：需频域展开（球谐）或采样离散化；球谐变换有快速算法但实现复杂
+- **Gauge-equivariant**：每边一个 $G$-元素作用 = 小矩阵乘特征向量，可表达为 sparse matmul
+- **近似等变（正则化）**：等变性损失 $L_{\text{eq}}$ 是普通的 MSE，完全 GPU 友好，但等变性不严格
+- 关键权衡：严格等变（结构约束）vs 近似等变（正则化）——前者参数少但实现复杂，后者简单但不保证
+
+## 风险与失效条件
+
+- **连续群离散化误差**：采样不当导致等变性悄悄破缺，验证时通过但推理时失败
+- **等变性与表达力的权衡**：严格等变约束减少参数空间，可能不足以拟合复杂函数
+- **多群作用的组合爆炸**：同时要求旋转 + 平移 + 置换等变时，约束交叉复杂
+- **数据噪声破坏等变性**：传感器噪声使 $g \cdot x$ 的精确计算不可靠，等变性前提失效
+- **等变层的数值精度**：球谐/CG 系数在大 $l$ 下的浮点误差会破坏等变性，需 fp32 累加
+- **过度等变约束**：任务只需近似对称时硬上严格等变，不如用正则化软约束
+
+## 深入参考
+
+- 蒸馏稿：references/books/micro-lie-theory.md（§II-F 伴随 Ad_X，等变性的代数实现）
+- 蒸馏稿：references/books/differential-geometry.md（§6.8 Principal Bundles, §12.12 G-Connections, 规范等变）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 5 Lie Groups, 连续对称作为先验）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §6.8 + §12.12（规范等变的几何基础）

package/knowledge-base/lie-theory/group-action.en.md

@@ -0,0 +1,51 @@
+# Group Action
+
+## Minimal Definition
+
+A group $G$ acting on a set $X$ is a homomorphism $\rho: G \to \text{Bij}(X)$ satisfying $\rho(e) = \text{id}$ and $\rho(g_1 g_2) = \rho(g_1) \circ \rho(g_2)$. It converts group elements into transformations on the set, providing the mathematical realization of "symmetry": the algebraic structure of the group determines the structure of geometric transformations.
+
+## Core Formulas
+
+- Group action: $g \cdot x = \rho(g)(x)$, satisfying $e \cdot x = x$ and $(gh)\cdot x = g \cdot (h \cdot x)$
+- Orbit: $\text{Orb}(x) = \{g \cdot x \mid g \in G\}$
+- Stabilizer subgroup: $\text{Stab}(x) = \{g \in G \mid g \cdot x = x\}$
+- Orbit-stabilizer theorem: $|G| = |\text{Orb}(x)| \cdot |\text{Stab}(x)|$
+- Invariant function: $f(g \cdot x) = f(x), \forall g \in G$
+- Equivariant map: $\phi(g \cdot x) = g \cdot \phi(x)$
+
+## Applicable Problems
+
+- Data possesses known symmetries: rotations, translations, permutations, scale transformations; the model must respect these symmetries
+- Output should covary with input: in pose estimation, when the object rotates, the output pose should rotate accordingly
+- Theoretical foundation for data augmentation: sampling along group orbits is equivalent to traversing the group action
+- Quotient space construction: modding out by the stabilizer subgroup yields an invariant feature space
+
+## AI Design Translation
+
+- **Equivariant network layer**: $f(g \cdot x) = g \cdot f(x)$, hard-coding the group action into the network structure to obtain equivariance without data augmentation
+- **Invariant pooling layer**: Average/max over orbits $\frac{1}{|G|}\sum_g f(g \cdot x)$ to extract invariants from equivariant features
+- **Group convolution**: $(f * h)(g) = \sum_{g'} f(g') h(g'^{-1} g)$, performing convolution directly on the group itself, applicable when signals are defined on the group
+- **Orbit-sampling data augmentation**: Use group actions to generate symmetry-equivalent training samples, effectively enlarging the training set
+
+## Engineering Feasibility
+
+GPU friendliness depends on the type of group:
+- **Finite/discrete groups**: Group convolution can be expanded into batched GEMM or sparse matmul, $O(|G|^2)$ or $O(|G| \cdot d)$, GPU-friendly
+- **Continuous compact groups SO(n)/SU(n)**: Require discrete sampling or frequency-domain expansion (Peter-Weyl theorem); fast algorithms exist for spherical harmonic transforms
+- **Permutation group $S_n$**: Order $n!$, cannot be enumerated; use sort pooling, symmetric functions, and other approximate invariants
+- **Fourier-accelerated group convolution**: The FFT on finite groups reduces convolution from $O(|G|^2)$ to $O(|G| \log |G|)$, but implementation is complex
+- Key bottleneck: if the discretization of a continuous group is not exact, equivariance silently breaks
+
+## Risks and Failure Conditions
+
+- **Naive discretization of continuous groups**: Improper sampling leads to broken equivariance and irregular gather/scatter patterns, GPU-unfriendly
+- **Incorrect group action definition**: Confusing left and right actions or inconsistent group multiplication order causes equivariance verification to pass but inference to fail
+- **Infeasible orbit enumeration**: Orbits of large/continuous groups cannot be fully enumerated; approximate invariants introduce bias
+- **Over-constraining**: Not all tasks require strict equivariance; enforcing group actions on weakly symmetric tasks may sacrifice expressiveness
+
+## Further References
+
+- Distillation notes: references/books/micro-lie-theory.md (Section II-B Group Actions)
+- Distillation notes: references/books/smooth-manifolds.md (Ch 7 Lie Groups)
+- Original text: Joan Sola et al., *A micro Lie theory*, Section II-B (group action definition and applications in robotics)
+- Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 7 (Lie groups and group actions)

package/knowledge-base/lie-theory/group-action.md

@@ -0,0 +1,51 @@
+# 群作用 (Group Action)
+
+## 最小定义
+
+群 $G$ 作用在集合 $X$ 上是一个同态 $\rho: G \to \text{Bij}(X)$，满足 $\rho(e) = \text{id}$ 和 $\rho(g_1 g_2) = \rho(g_1) \circ \rho(g_2)$。它将群元素转化为集合上的变换，是"对称性"的数学实现：群的代数结构决定了几何变换的结构。
+
+## 核心公式
+
+- 群作用：$g \cdot x = \rho(g)(x)$，满足 $e \cdot x = x$，$(gh)\cdot x = g \cdot (h \cdot x)$
+- 轨道（orbit）：$\text{Orb}(x) = \{g \cdot x \mid g \in G\}$
+- 稳定子群（stabilizer）：$\text{Stab}(x) = \{g \in G \mid g \cdot x = x\}$
+- 轨道-稳定子定理：$|G| = |\text{Orb}(x)| \cdot |\text{Stab}(x)|$
+- 不变函数：$f(g \cdot x) = f(x), \forall g \in G$
+- 等变映射：$\phi(g \cdot x) = g \cdot \phi(x)$
+
+## 适用问题
+
+- 数据具有已知对称性：旋转、平移、置换、尺度变换，需要模型尊重这些对称
+- 输出应随输入协变：姿态估计中，物体旋转后输出位姿也应相应旋转
+- 数据增广的理论基础：群轨道上的采样等价于群作用的遍历
+- 商空间构造：模去稳定子群得到不变特征空间
+
+## AI 设计翻译
+
+- **等变网络层**：$f(g \cdot x) = g \cdot f(x)$，将群作用硬编码进网络结构，无需数据增广即获得等变性
+- **不变池化层**：对轨道求平均/最大 $\frac{1}{|G|}\sum_g f(g \cdot x)$，从等变特征中提取不变量
+- **群卷积**：$(f * h)(g) = \sum_{g'} f(g') h(g'^{-1} g)$，在群本身上做卷积，适用于信号定义在群上的场景
+- **轨道采样数据增广**：用群作用生成训练样本的对称等价类，扩大有效训练集
+
+## 工程可行性
+
+GPU 友好度取决于群的类型：
+- **有限群/离散群**：群卷积可展开为 batched GEMM 或稀疏 matmul，$O(|G|^2)$ 或 $O(|G| \cdot d)$，GPU 友好
+- **连续紧群 SO(n)/SU(n)**：需离散采样或频域展开（Peter-Wigner 定理），球谐变换有快速算法
+- **置换群 $S_n$**：$n!$ 阶，不可遍历；用排序池化、对称函数等近似不变化
+- **群卷积的 Fourier 加速**：有限群的 FFT 将卷积从 $O(|G|^2)$ 降至 $O(|G| \log |G|)$，但实现复杂
+- 关键瓶颈：连续群的离散化若不精确，等变性会悄悄破缺
+
+## 风险与失效条件
+
+- **连续群无脑离散化**：采样不当导致等变性破缺 + 不规则 gather/scatter，GPU 不友好
+- **群作用定义错误**：左右作用混淆、群乘法顺序不一致导致等变性验证通过但推理失败
+- **轨道遍历不可行**：大群/连续群的轨道无法完整遍历，近似不变化引入偏差
+- **过度约束**：不是所有任务都需要严格等变，弱对称任务硬上群作用可能牺牲表达力
+
+## 深入参考
+
+- 蒸馏稿：references/books/micro-lie-theory.md（§II-B 群作用）
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 7 Lie Groups）
+- 原书：Joan Sola et al., *A micro Lie theory*, §II-B（群作用定义与机器人中的应用）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 7（李群与群作用）

package/knowledge-base/lie-theory/lie-algebra.en.md

@@ -0,0 +1,51 @@
+# Lie Algebra
+
+## Minimal Definition
+
+The Lie algebra $\mathfrak{g}$ is the tangent space $T_e G$ of a Lie group $G$ at the identity element $e$, equipped with a Lie bracket $[\cdot, \cdot]: \mathfrak{g} \times \mathfrak{g} \to \mathfrak{g}$ satisfying bilinearity, antisymmetry, and the Jacobi identity. It is the linear space of "infinitesimal generators" -- locally linearizing the curved, nonlinear group into a flat vector space.
+
+## Core Formulas
+
+- Lie bracket: $[X, Y] = XY - YX$ (for matrix groups), satisfying $[X,[Y,Z]] + [Y,[Z,X]] + [Z,[X,Y]] = 0$
+- $\text{so}(3)$: skew-symmetric matrices, $[\omega]_\times = \begin{pmatrix} 0 & -\omega_3 & \omega_2 \\ \omega_3 & 0 & -\omega_1 \\ -\omega_2 & \omega_1 & 0 \end{pmatrix}$
+- hat/vee operators: $\hat{\cdot}: \mathbb{R}^n \to \mathfrak{g}$ (vector to skew-symmetric matrix), $\check{\cdot}: \mathfrak{g} \to \mathbb{R}^n$ (inverse)
+- Baker-Campbell-Hausdorff formula: $\log(\exp(X)\exp(Y)) = X + Y + \frac{1}{2}[X,Y] + \cdots$
+- Adjoint representation: $\text{ad}_X(Y) = [X,Y]$, which is the differential of $\text{Ad}$
+
+## Applicable Problems
+
+- Local linearization of rotations/poses: approximate nonlinear group variations in a small neighborhood using Lie algebra vectors $\delta \in \mathbb{R}^n$
+- Reparameterization for constrained optimization: convert orthogonality/rotation constraints into unconstrained Lie algebra parameters + exp map
+- Infinitesimal description of symmetries: continuous symmetry groups are fully characterized by a small set of generators
+- Error-state estimation: covariances are defined on the tangent space $\mathbb{R}^n$ rather than on the group
+
+## AI Design Translation
+
+- **Lie algebra parameterization layer**: The network outputs $\delta \in \mathbb{R}^3$ (so(3)), then applies $\exp$ to obtain a valid rotation matrix; replaces quaternion normalization or 6D representations
+- **Error-state EKF/optimization layer**: Parameterize the error around a nominal state $X$ as $\delta \in \mathfrak{g}$ with $X_{\text{true}} = X \oplus \delta$; Kalman filtering proceeds in the linear tangent space
+- **Lie bracket regularization**: Penalize $[\xi_i, \xi_j] \neq 0$ to constrain the commutativity of generators, or use as a symmetry consistency loss
+- **Generator learning**: Learn a Lie algebra basis $\{E_1, \ldots, E_n\}$ as trainable parameters, enabling data-driven symmetry discovery
+
+## Engineering Feasibility
+
+High GPU friendliness. The core advantage of the Lie algebra is that it is a "linear space":
+- **hat/vee maps**: Pure index operations + sign flips, $O(n)$, perfectly GPU-friendly
+- **Lie bracket $[X,Y] = XY - YX$**: Two small matrix multiplications + subtraction, $O(n^3)$ for small matrices, batchable
+- **Linear combination $\sum c_i E_i$**: Vector addition + scalar multiplication, $O(nd)$, perfectly GPU-friendly
+- **BCH approximation**: The first few terms suffice for engineering precision; $[X,Y]$ computation is a small matrix multiplication
+- Main cost lies in $\exp$ rather than algebra operations: algebra operations are all linear; the bottleneck is the subsequent exp map
+
+## Risks and Failure Conditions
+
+- **BCH series truncation error**: Higher-order terms are non-negligible at large angles; the first-order approximation $X+Y$ is valid only for small perturbations
+- **Misinterpreting non-zero Lie brackets**: $[X,Y] \neq 0$ for non-commutative groups means group composition is order-sensitive; operations cannot be freely interchanged
+- **Basis selection affects optimization**: The choice of Lie algebra basis is not unique; poor conditioning leads to optimization difficulties
+- **Using the Lie algebra as global coordinates**: The exp map is only a local diffeomorphism; global coverage requires multiple charts (an atlas)
+- **Inconsistent left/right conventions**: Right Jacobian vs. left Jacobian, local vs. global frame -- inconsistency leads to misaligned gradients and covariances
+
+## Further References
+
+- Distillation notes: references/books/micro-lie-theory.md (Section II-C Tangent Space and Lie Algebra, hat/vee operators)
+- Distillation notes: references/books/differential-geometry.md (Ch 5 Lie Groups, Lie algebra section)
+- Original text: Joan Sola et al., *A micro Lie theory*, Section II-C (Lie algebra definition and hat/vee), Section II-D (exp/log bridge)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 5 (Lie groups and Lie algebras)