math-skill 3.0.0 → 3.1.0

package/knowledge-base/differential-geometry/index.en.md

@@ -0,0 +1,52 @@
+# Differential Geometry Activation Index
+
+## Domain Signals
+Activate this domain direction when the problem involves:
+- Parameter space curvature: geometric curvature of parameter space affects optimization or inference
+- Manifold constraints: parameters must remain on a specific manifold
+- Geodesic distance: need to compute shortest paths or distances along a manifold
+- Connection / parallel transport: need to compare tangent vectors at different points on a manifold
+- Riemannian metric: need to define inner products and lengths on a manifold
+- Index raising/lowering: need to convert between tangent and cotangent spaces
+
+## Core Anchors
+- `manifold.en.md` — Fundamental concepts of manifolds
+- `tangent-space.en.md` — Tangent space and tangent map
+- `metric-tensor.en.md` — Metric tensor and Riemannian metric
+- `geodesic.en.md` — Geodesics
+- `curvature.en.md` — Curvature
+- `connection.en.md` — Connection and covariant derivative
+
+## Extended Concepts
+When core anchors are insufficient, the following concepts may need temporary activation:
+- Lie derivative: Lie derivative and rate of change along a vector field
+- exterior calculus (differential forms, wedge product, exterior derivative): exterior differentiation and differential form operations
+- Hodge decomposition: Hodge decomposition theorem
+- fiber bundle: fiber bundles and sections
+- principal bundle: principal bundles and gauge theory
+- holonomy: holonomy group and closure of parallel transport
+- Riemannian submersion: Riemannian submersions and quotient metrics
+- symmetric space: symmetric spaces
+- homogeneous space: homogeneous spaces G/H
+- Cartan geometry: Cartan geometry and moving frames
+- spin structure: spin structures
+- Dirac operator: Dirac operator
+- Ricci flow: Ricci flow
+
+## Reference Book Directions
+- `../../references/books/differential-geometry.en.md`: core content of Riemannian geometry
+- `../../references/books/smooth-manifolds.en.md`: smooth manifolds, vector bundles, and differential forms
+
+## AI Translation Directions
+- manifold → latent space geometry / manifold-valued predictions / hyperbolic embeddings
+- tangent space → local linearization of parameter updates / tangent-space features
+- metric tensor → learned distance metrics / Fisher-Rao natural gradient / metric-aware loss
+- geodesic → interpolation on manifolds / geodesic regularization / shortest-path loss
+- curvature → loss landscape analysis / curvature-aware optimization / flatness seeking
+- connection → parallel transport of features / gauge-equivariant layers
+
+## Temporary Activation Rules
+When the problem requires mathematics not in the core anchors:
+1. First check whether extended concepts contain a match
+2. If yes, generate a temporary knowledge card based on the lens
+3. If no, enter the Knowledge Gap Protocol

package/knowledge-base/differential-geometry/index.md

@@ -0,0 +1,52 @@
+# 微分几何 激活索引 / Differential Geometry Activation Index
+
+## 领域信号
+当问题涉及以下信号时，激活本领域方向：
+- 参数空间曲率：参数空间的几何弯曲影响优化或推断
+- 流形约束：参数必须保持在特定流形上
+- 测地线距离：需要沿流形计算最短路径或距离
+- 联络/平行移动：需要在流形上比较不同点的切向量
+- 黎曼度量：需要定义流形上的内积和长度
+- 指标升降：需要在切空间与余切空间之间转换
+
+## 核心锚点
+- `manifold.md` — 流形的基本概念
+- `tangent-space.md` — 切空间与切映射
+- `metric-tensor.md` — 度量张量与黎曼度量
+- `geodesic.md` — 测地线
+- `curvature.md` — 曲率
+- `connection.md` — 联络与协变导数
+
+## 扩展概念
+当核心锚点不够时，以下概念可能需要临时激活：
+- Lie derivative：Lie 导数与沿向量场的变化率
+- exterior calculus（differential forms, wedge product, exterior derivative）：外微分与微分形式运算
+- Hodge decomposition：Hodge 分解定理
+- fiber bundle：纤维丛与截面
+- principal bundle：主丛与规范理论
+- holonomy：和乐群与平行移动的闭合性
+- Riemannian submersion：黎曼浸没与商度量
+- symmetric space：对称空间
+- homogeneous space：齐性空间 G/H
+- Cartan geometry：Cartan 几何与活动标架
+- spin structure：旋量结构
+- Dirac operator：Dirac 算子
+- Ricci flow：Ricci 流
+
+## 参考书方向
+- `../../references/books/differential-geometry.md`：黎曼几何的核心内容
+- `../../references/books/smooth-manifolds.md`：光滑流形、向量丛和微分形式
+
+## AI 翻译方向
+- manifold → latent space geometry / manifold-valued predictions / hyperbolic embeddings
+- tangent space → local linearization of parameter updates / tangent-space features
+- metric tensor → learned distance metrics / Fisher-Rao natural gradient / metric-aware loss
+- geodesic → interpolation on manifolds / geodesic regularization / shortest-path loss
+- curvature → loss landscape analysis / curvature-aware optimization / flatness seeking
+- connection → parallel transport of features / gauge-equivariant layers
+
+## 临时激活规则
+当问题需要的数学不在核心锚点中时：
+1. 先检查扩展概念中是否有匹配
+2. 若有，根据透镜生成临时知识卡
+3. 若无，进入 Knowledge Gap Protocol

package/knowledge-base/differential-geometry/manifold.en.md

@@ -41,7 +41,20 @@ Moderate GPU friendliness. Coordinate chart transformations are element-wise map
 
 ## Further References
 
-- Distillation notes: references/books/smooth-manifolds.md (Ch 1-2 Smooth Manifolds / Smooth Maps)
-- Distillation notes: references/books/differential-geometry.md (Ch 1-2 Differentiable Manifolds / The Tangent Structure)
+- Distillation notes: ../../references/books/smooth-manifolds.en.md (Ch 1-2 Smooth Manifolds / Smooth Maps)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 1-2 Differentiable Manifolds / The Tangent Structure)
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2 (topological manifolds, smooth structures, partition of unity)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2
+
+
+## Routing Extensions
+- If local structure analysis is needed -> `tangent-space.en.md` (tangent space provides local linear approximation)
+- If distance definition is needed -> `metric-tensor.en.md` (metric tensor defines distance on manifolds)
+- If optimization on manifolds is needed -> `../optimization/riemannian-optimization.md` (Riemannian optimization methods)
+
+## Extensible Directions
+- Submanifold: embedded and immersed submanifolds
+- Product manifold: direct product construction of multiple manifolds
+- Quotient manifold: quotient space under equivalence relations
+- Stiefel / Grassmann manifolds: orthogonal matrix and subspace manifolds
+- Manifold learning (Isomap / LLE / diffusion maps): discovering low-dimensional manifolds from high-dimensional data

package/knowledge-base/differential-geometry/manifold.md

@@ -41,7 +41,20 @@ GPU 友好度中等。坐标卡变换本身是逐元素的映射（可并行）
 
 ## 深入参考
 
-- 蒸馏稿：references/books/smooth-manifolds.md（Ch 1-2 Smooth Manifolds / Smooth Maps）
-- 蒸馏稿：references/books/differential-geometry.md（Ch 1-2 Differentiable Manifolds / The Tangent Structure）
+- 蒸馏稿：../../references/books/smooth-manifolds.md（Ch 1-2 Smooth Manifolds / Smooth Maps）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 1-2 Differentiable Manifolds / The Tangent Structure）
 - 原书：John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2（拓扑流形、光滑结构、单位分解）
 - 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2
+
+
+## 路由扩展
+- 若需要局部结构分析 → `tangent-space.md`（切空间提供局部线性近似）
+- 若需要距离定义 → `metric-tensor.md`（度量张量定义流形上的距离）
+- 若需要在流形上做优化 → `../optimization/riemannian-optimization.md`（黎曼优化方法）
+
+## 可扩展方向
+- 子流形（submanifold）：嵌入子流形与浸没子流形
+- 积流形（product manifold）：多个流形的直积构造
+- 商流形（quotient manifold）：等价关系下的商空间
+- Stiefel / Grassmann 流形：正交矩阵与子空间流形
+- 流形学习（Isomap / LLE / diffusion maps）：从高维数据发现低维流形

package/knowledge-base/differential-geometry/metric-tensor.en.md

@@ -44,7 +44,21 @@ GPU friendliness depends on the structure of the metric:
 
 ## Further References
 
-- Distillation notes: references/books/differential-geometry.md (Ch 7 Section 7.6 Metric Tensors, Ch 13 Section 13.1 Levi-Civita)
-- Distillation notes: references/books/smooth-manifolds.md (Ch 13 Riemannian Metrics)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 7 Section 7.6 Metric Tensors, Ch 13 Section 13.1 Levi-Civita)
+- Distillation notes: ../../references/books/smooth-manifolds.en.md (Ch 13 Riemannian Metrics)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 7.6 Metric Tensors
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 13 (Riemannian metrics, musical isomorphisms sharp/flat)
+
+
+## Routing Extensions
+- If the metric comes from Fisher information -> `../information-geometry/natural-gradient.md` (natural gradient under Fisher metric)
+- If Riemannian gradient computation is needed -> `../optimization/riemannian-optimization.md` (metric determines gradient direction)
+- If curvature analysis is needed -> `curvature.en.md` (metric determines curvature tensor)
+
+## Extensible Directions
+- Finsler metric: generalized non-quadratic metrics
+- Sub-Riemannian metric: metrics under distribution constraints
+- Information metric: Fisher-Rao metric on statistical manifolds
+- Pullback metric: metric induced by a mapping
+- Metric learning: learning optimal metrics from data
+- Distance metric learning: supervised distance learning

package/knowledge-base/differential-geometry/metric-tensor.md

@@ -44,7 +44,21 @@ GPU 友好度取决于度量的结构化程度：
 
 ## 深入参考
 
-- 蒸馏稿：references/books/differential-geometry.md（Ch 7 §7.6 Metric Tensors, Ch 13 §13.1 Levi-Civita）
-- 蒸馏稿：references/books/smooth-manifolds.md（Ch 13 Riemannian Metrics）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 7 §7.6 Metric Tensors, Ch 13 §13.1 Levi-Civita）
+- 蒸馏稿：../../references/books/smooth-manifolds.md（Ch 13 Riemannian Metrics）
 - 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §7.6 Metric Tensors
-- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 13（黎曼度量、升降指标 ♯/♭）
+- 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 13（黎曼度量、升降指标 (sharp)/(flat)）
+
+
+## 路由扩展
+- 若度量来自 Fisher 信息 → `../information-geometry/natural-gradient.md`（Fisher 度量下的自然梯度）
+- 若需要黎曼梯度计算 → `../optimization/riemannian-optimization.md`（度量决定的梯度方向）
+- 若需要曲率分析 → `curvature.md`（度量决定的曲率张量）
+
+## 可扩展方向
+- Finsler 度量（Finsler metric）：非二次型的广义度量
+- 亚黎曼度量（sub-Riemannian metric）：分布约束下的度量
+- 信息度量（information metric）：统计流形上的 Fisher-Rao 度量
+- 拉回度量（pullback metric）：映射诱导的度量
+- 度量学习（metric learning）：从数据学习最优度量
+- 距离度量学习（distance metric learning）：监督距离学习

package/knowledge-base/differential-geometry/tangent-space.en.md

@@ -13,7 +13,7 @@ The tangent space $T_pM$ is the $n$-dimensional vector space of all tangent vect
 
 ## Applicable Problems
 
-- Geometric interpretation of backpropagation: the chain rule equals pushforward along composed maps
+- Geometric interpretation of backpropagation: the chain rule equals pullback on the cotangent bundle, i.e., VJP (vector-Jacobian product) = pullback of covectors $df_p^*(\omega) = J^T \omega$. Note: pushforward corresponds to JVP (forward-mode AD), not backpropagation
 - Correct gradient direction computation: autodiff outputs are covectors (1-forms), requiring a metric to convert them into descent directions
 - Gradient projection in constrained optimization: projecting the Euclidean gradient onto the tangent space of the constraint submanifold
 - Linearized approximations on manifolds: using linear methods within the tangent space to handle local problems
@@ -42,7 +42,21 @@ High GPU friendliness. The core operations of the tangent space are linear algeb
 
 ## Further References
 
-- Distillation notes: references/books/smooth-manifolds.md (Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle)
-- Distillation notes: references/books/differential-geometry.md (Ch 1-2, Ch 7 Tensors)
+- Distillation notes: ../../references/books/smooth-manifolds.en.md (Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 1-2, Ch 7 Tensors)
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 3 (tangent spaces, pushforward, tangent bundle)
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 11 (cotangent bundle, 1-forms, pullback)
+
+
+## Routing Extensions
+- If gradient computation on manifolds is needed -> `../optimization/riemannian-optimization.md` (gradient descent on manifolds)
+- If the tangent space of a group structure is involved -> `../lie-theory/lie-algebra.md` (tangent space of a Lie group is its Lie algebra)
+- If covariant derivative is needed -> `connection.en.md` (connection defines covariant differentiation)
+
+## Extensible Directions
+- Cotangent space: dual space and differential forms
+- Differential / pushforward: tangent map of smooth maps
+- Vector field: smooth vector fields on manifolds
+- Lie bracket: commutation relations of vector fields
+- Integral curve: integral curves and flows of vector fields
+- Exponential map: mapping from tangent space to manifold

package/knowledge-base/differential-geometry/tangent-space.md

@@ -13,7 +13,7 @@
 
 ## 适用问题
 
-- 反向传播的几何理解：链式法则 = 沿复合映射做 pushforward
+- 反向传播的几何理解：链式法则 = 余切丛上的 pullback（拉回），即 VJP（vector-Jacobian product）= 余向量的拉回 $df_p^*(\omega) = J^T \omega$。注意：pushforward 对应的是 JVP（前向模式 AD），不是反向传播
 - 梯度方向的正确计算：autodiff 输出是余向量（1-form），需要度量才能变成下降方向
 - 约束优化中梯度投影：将欧氏梯度投影到约束子流形的切空间
 - 流形上的线性化近似：在切空间中用线性方法处理局部问题
@@ -42,7 +42,21 @@ GPU 友好度高。切空间的核心操作是线性代数：
 
 ## 深入参考
 
-- 蒸馏稿：references/books/smooth-manifolds.md（Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle）
-- 蒸馏稿：references/books/differential-geometry.md（Ch 1-2, Ch 7 Tensors）
+- 蒸馏稿：../../references/books/smooth-manifolds.md（Ch 3 Tangent Vectors, Ch 11 The Cotangent Bundle）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 1-2, Ch 7 Tensors）
 - 原书：John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 3（切空间、pushforward、切丛）
 - 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 11（余切丛、1-form、pullback）
+
+
+## 路由扩展
+- 若需要在流形上计算梯度 → `../optimization/riemannian-optimization.md`（流形上的梯度下降）
+- 若涉及群结构的切空间 → `../lie-theory/lie-algebra.md`（李群的切空间即李代数）
+- 若需要协变导数 → `connection.md`（联络定义协变微分）
+
+## 可扩展方向
+- 余切空间（cotangent space）：对偶空间与微分形式
+- 微分/前推算子（differential / pushforward）：光滑映射的切映射
+- 向量场（vector field）：流形上的光滑向量场
+- Lie 括号（Lie bracket）：向量场的对易关系
+- 积分曲线（integral curve）：向量场的积分曲线与流
+- 指数映射（exponential map）：从切空间到流形的映射

package/knowledge-base/information-geometry/fisher-metric.en.md

@@ -6,10 +6,14 @@ The Fisher-Rao metric is a **Riemannian metric** on the parameter space of a fam
 ## Core Formulas
 
 **Metric Tensor (i.e., Fisher Information Matrix)**:
-$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right]$$
+$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right] = -\mathbb{E}_\theta\left[\frac{\partial^2 \log p}{\partial \theta_i \partial \theta_j}\right]$$
+
+**Second-Order Taylor Expansion of KL Divergence**: The Fisher metric is precisely the Hessian of KL divergence at $\theta' = \theta$:
+$$g_{ij}(\theta) = \frac{\partial^2}{\partial \theta_i' \partial \theta_j'} D_{KL}(p_\theta \| p_{\theta'})\bigg|_{\theta'=\theta}$$
+That is, $D_{KL}(p_\theta \| p_{\theta+d\theta}) = \frac{1}{2} \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j + O(\|d\theta\|^3)$.
 
 **Line Element (infinitesimal distance between distributions)**:
-$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j = 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta})$$
+$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j \approx 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta}) \quad \text{(second-order local expansion, asymptotically exact as } d\theta \to 0\text{)}$$
 
 **Geodesic Distance (finite distance between distributions)**:
 $$d(p_{\theta_1}, p_{\theta_2}) = \inf_{\gamma} \int_0^1 \sqrt{\dot{\gamma}^T \mathcal{I}(\gamma(t)) \dot{\gamma}} \, dt$$
@@ -20,7 +24,7 @@ $$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \
 - $\alpha = 1$: $e$-connection (exponential connection), corresponding to straight lines in the natural parameters of the exponential family
 - $\alpha = -1$: $m$-connection (mixture connection), corresponding to straight lines in the expectation parameters of the mixture family
 
-**Dually Flat Structure**: $(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ forms a dually flat manifold — the $e$-connection and $m$-connection are dual to each other with respect to $g$, and the generalized Pythagorean theorem holds.
+**Dually Flat Structure (conditional)**: Exponential families, mixture families, and related structured statistical models can make $(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ a dually flat manifold, where the $e$-connection and $m$-connection are dual with respect to $g$ and the generalized Pythagorean theorem holds. A general Fisher-Rao statistical manifold is not automatically dually flat; the model-family structure must be verified.
 
 ## Applicable Problems
 - **Geometric distance between distributions**: Comparing the "intrinsic difference" between two probabilistic models (e.g., output distributions of two language models)
@@ -33,14 +37,14 @@ $$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \
 - **Geometric analysis of MoE expert distributions**: The degree of separation between the output distributions of different experts under the Fisher metric can quantify expert diversity
 
 ## Engineering Feasibility
-- **Dimension 1 Tensorization ❌**: The full metric tensor $g_{ij}$ is $d \times d$; infeasible to materialize when $d \sim 10^{10}$
-- **Dimension 2 GEMM-mappability ⚠️**: Kronecker/diagonal approximations can be mapped; the exact metric cannot
-- **Dimension 3 Complexity ❌**: Geodesic computation requires solving a second-order ODE; exact computation is intractable
-- **Dimension 4 Memory ❌**: Full metric tensor storage is $O(d^2)$; completely impossible at LLM scale
-- **Dimension 5 Low Precision ⚠️**: The condition number of the metric tensor may be very large, leading to instability under low precision
-- **Dimension 6 Parallelism ⚠️**: Approximate versions (K-FAC, diagonal) can be parallelized; exact versions cannot
-- **Dimension 7 Sparsity ⚠️**: The Fisher information matrix is typically dense; block-diagonal approximations (inter-layer independence) introduce structured sparsity
-- **Dimension 8 Operator Fusion ✅**: Approximate versions can be fused into optimizer updates
+- **D1[x]**: The full metric tensor $g_{ij}$ is $d \times d$; infeasible to materialize when $d \sim 10^{10}$
+- **D2[~]**: Kronecker/diagonal approximations can be mapped; the exact metric cannot
+- **D3[x]**: Geodesic computation requires solving a second-order ODE; exact computation is intractable
+- **D4[x]**: Full metric tensor storage is $O(d^2)$; completely impossible at LLM scale
+- **D5[~]**: The condition number of the metric tensor may be very large, leading to instability under low precision
+- **D6[~]**: Approximate versions (K-FAC, diagonal) can be parallelized; exact versions cannot
+- **D7[~]**: The Fisher information matrix is typically dense; block-diagonal approximations (inter-layer independence) introduce structured sparsity
+- **D8[v]**: Approximate versions can be fused into optimizer updates
 
 **Conclusion**: The exact Fisher metric is infeasible at LLM scale, but **approximate versions** (K-FAC, diagonal Fisher, low-rank) are engineering-viable. The theoretical value of information geometry lies primarily in **guiding design** rather than direct computation.
 
@@ -49,8 +53,23 @@ $$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \
 - **Singularities of the statistical manifold**: In certain regions of the parameter space (e.g., degenerate points of mixture distributions), the Fisher metric may degenerate ($\det \mathcal{I} = 0$), causing geodesic distances to be undefined. This type of degeneracy arises in MoE when an expert's weight is zero.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Distillation draft: `../../references/books/` — no dedicated information geometry distillation draft at present
 - Amari & Nagaoka. *Methods of Information Geometry*. AMS/Oxford, 2000
 - Amari. *Information Geometry and Its Applications*. Springer, 2016
 - Ay, Jost, Le, Schwachhofer. *Information Geometry*. Springer, 2017
-- Related knowledge cards: `probability/fisher-information.md`, `information-geometry/natural-gradient.md`
+- Related knowledge cards: `../probability/fisher-information.en.md`, `natural-gradient.en.md`
+
+
+## Routing Extensions
+- If optimization application is needed -> `natural-gradient.en.md` (natural gradient descent under Fisher metric)
+- If a general Riemannian metric is needed -> `../differential-geometry/metric-tensor.md` (Fisher metric is a special case of Riemannian metric)
+- If local KL analysis is needed -> `../probability/kl-divergence.md` (local KL divergence equals Fisher metric)
+
+## Extensible Directions
+- Rao's distance: geodesic distance under Fisher-Rao metric
+- Fisher-Rao geodesic: shortest paths in distribution space
+- Alpha-connection family: Amari's alpha-connections
+- Dual flatness: e-flat and m-flat structures
+- Chentsov's uniqueness theorem: uniqueness of Fisher metric
+- Infinite-dimensional Fisher metric: Fisher metric on function spaces
+- Fisher metric in function space: Fisher metric for neural networks

package/knowledge-base/information-geometry/fisher-metric.md

@@ -6,10 +6,14 @@ Fisher-Rao 度量是概率分布族（统计模型）参数空间上的**黎曼
 ## 核心公式
 
 **度量张量（即 Fisher 信息矩阵）**：
-$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right]$$
+$$g_{ij}(\theta) = \mathcal{I}_{ij}(\theta) = \mathbb{E}_\theta\left[\frac{\partial \log p}{\partial \theta_i} \frac{\partial \log p}{\partial \theta_j}\right] = -\mathbb{E}_\theta\left[\frac{\partial^2 \log p}{\partial \theta_i \partial \theta_j}\right]$$
+
+**KL 散度的二阶 Taylor 展开**：Fisher 度量恰好是 KL 散度在 $\theta' = \theta$ 处的 Hessian：
+$$g_{ij}(\theta) = \frac{\partial^2}{\partial \theta_i' \partial \theta_j'} D_{KL}(p_\theta \| p_{\theta'})\bigg|_{\theta'=\theta}$$
+即 $D_{KL}(p_\theta \| p_{\theta+d\theta}) = \frac{1}{2} \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j + O(\|d\theta\|^3)$。
 
 **线元（分布间的无穷小距离）**：
-$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j = 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta})$$
+$$ds^2 = \sum_{i,j} g_{ij}(\theta) \, d\theta_i \, d\theta_j \approx 2 \, D_{KL}(p_\theta \| p_{\theta+d\theta}) \quad \text{（二阶局部展开，当 } d\theta \to 0 \text{ 时渐近成立）}$$
 
 **测地线距离（有限分布间距离）**：
 $$d(p_{\theta_1}, p_{\theta_2}) = \inf_{\gamma} \int_0^1 \sqrt{\dot{\gamma}^T \mathcal{I}(\gamma(t)) \dot{\gamma}} \, dt$$
@@ -20,7 +24,7 @@ $$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \
 - $\alpha = 1$：$e$-联络（指数联络），对应指数族的自然参数直线
 - $\alpha = -1$：$m$-联络（混合联络），对应混合族的期望参数直线
 
-**对偶平坦性**：$(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ 构成对偶平坦流形——$e$-联络和 $m$-联络关于 $g$ 互为对偶，广义勾股定理成立。
+**对偶平坦性（有条件）**：指数族、混合族等特定统计模型可使 $(\mathcal{M}, g, \nabla^{(e)}, \nabla^{(m)})$ 构成对偶平坦流形，$e$-联络和 $m$-联络关于 $g$ 互为对偶，广义勾股定理成立。一般 Fisher-Rao 统计流形不自动对偶平坦，需验证模型族结构。
 
 ## 适用问题
 - **分布间的几何距离**：比较两个概率模型（如两个语言模型的输出分布）的"本质差异"
@@ -33,14 +37,14 @@ $$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \
 - **MoE 专家分布的几何分析**：不同专家的输出分布在 Fisher 度量下的分离度可量化专家多样性
 
 ## 工程可行性
-- **维度 1 张量化 ❌**：完整度量张量 $g_{ij}$ 是 $d \times d$，$d \sim 10^{10}$ 时不可物化
-- **维度 2 GEMM 可映射 ⚠️**：Kronecker/对角近似可，精确度量不可
-- **维度 3 复杂度 ❌**：测地线计算需解二阶 ODE，精确计算不可行
-- **维度 4 显存 ❌**：完整度量张量存储 $O(d^2)$，LLM 级完全不可能
-- **维度 5 低精度 ⚠️**：度量张量的条件数可能很大，低精度下不稳定
-- **维度 6 并行 ⚠️**：近似版本（K-FAC、对角）可并行，精确版本不可
-- **维度 7 稀疏 ⚠️**：Fisher 信息矩阵通常稠密；块对角近似（层间独立）引入结构化稀疏
-- **维度 8 算子融合 ✅**：近似版本可融入优化器更新
+- **D1[x]**：完整度量张量 $g_{ij}$ 是 $d \times d$，$d \sim 10^{10}$ 时不可物化
+- **D2[~]**：Kronecker/对角近似可，精确度量不可
+- **D3[x]**：测地线计算需解二阶 ODE，精确计算不可行
+- **D4[x]**：完整度量张量存储 $O(d^2)$，LLM 级完全不可能
+- **D5[~]**：度量张量的条件数可能很大，低精度下不稳定
+- **D6[~]**：近似版本（K-FAC、对角）可并行，精确版本不可
+- **D7[~]**：Fisher 信息矩阵通常稠密；块对角近似（层间独立）引入结构化稀疏
+- **D8[v]**：近似版本可融入优化器更新
 
 **结论**：精确 Fisher 度量在 LLM 规模下不可行，但**近似版本**（K-FAC、对角 Fisher、低秩）在工程上可落地。信息几何的理论价值主要在**指导设计**而非直接计算。
 
@@ -49,8 +53,23 @@ $$\Gamma_{ijk}^{(\alpha)} = \mathbb{E}\left[\left(\partial_i \partial_j \ell + \
 - **统计流形的奇点**：在参数空间的某些区域（如混合分布的退化点），Fisher 度量可能退化（$\det \mathcal{I} = 0$），导致测地线距离不定义。在 MoE 中某专家权重为零时即出现此类退化。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- 蒸馏稿：`../../references/books/` 暂无专用信息几何蒸馏稿
 - Amari & Nagaoka. *Methods of Information Geometry*. AMS/Oxford, 2000
 - Amari. *Information Geometry and Its Applications*. Springer, 2016
 - Ay, Jost, Le, Schwachhofer. *Information Geometry*. Springer, 2017
-- 关联知识卡：`probability/fisher-information.md`、`information-geometry/natural-gradient.md`
+- 关联知识卡：`../probability/fisher-information.md`、`natural-gradient.md`
+
+
+## 路由扩展
+- 若需要优化应用 → `natural-gradient.md`（Fisher 度量下的自然梯度下降）
+- 若需要一般黎曼度量 → `../differential-geometry/metric-tensor.md`（Fisher 度量是黎曼度量的特例）
+- 若需要局部 KL 分析 → `../probability/kl-divergence.md`（局部 KL 散度等于 Fisher 度量）
+
+## 可扩展方向
+- Rao 距离（Rao's distance）：Fisher-Rao 度量下的测地距离
+- Fisher-Rao 测地线（Fisher-Rao geodesic）：分布空间中的最短路径
+- alpha-联络族（alpha-connection family）：Amari 的 alpha-联络
+- 对偶平坦性（dual flatness）：e-平坦与 m-平坦结构
+- Chentsov 唯一性定理（Chentsov's uniqueness theorem）：Fisher 度量的唯一性
+- 无穷维 Fisher 度量（infinite-dimensional Fisher metric）：函数空间上的 Fisher 度量
+- 函数空间 Fisher 度量（Fisher metric in function space）：神经网络的 Fisher 度量

package/knowledge-base/information-geometry/index.en.md

@@ -0,0 +1,45 @@
+# Information Geometry Activation Index
+
+## Domain Signals
+Activate this domain direction when the problem involves:
+- Distribution family geometry: probability distribution families have intrinsic geometric structure
+- Parameterization-independent optimization: optimization methods should not depend on specific parameterization
+- Statistical model complexity: need to measure the intrinsic complexity of statistical models
+- Distance between distributions: need to define geometric distance in distribution space
+- Natural gradient: gradient direction must account for the geometric structure of parameter space
+
+## Core Anchors
+- `natural-gradient.en.md` — Natural gradient
+- `fisher-metric.en.md` — Fisher metric
+
+## Extended Concepts
+When core anchors are insufficient, the following concepts may need temporary activation:
+- alpha-divergence: alpha-divergence family
+- Amari-Chentsov tensor: Amari-Chentsov tensor and cubic differential structure
+- dually flat manifold: dually flat manifolds
+- e-connection / m-connection: exponential connection and mixture connection
+- Pythagorean theorem for KL: generalized Pythagorean theorem for KL divergence
+- Bregman divergence: Bregman divergence and convex duality
+- mirror descent as natural gradient: equivalence between mirror descent and natural gradient
+- EM algorithm geometry: geometric interpretation of the EM algorithm
+- variational Bayes geometry: geometric structure of variational Bayes
+- information geometry of neural networks (loss landscape curvature): information geometry of neural networks and loss landscape curvature
+- neural tangent kernel as metric: neural tangent kernel as a metric
+- Fisher-Rao gradient flow: Fisher-Rao gradient flow
+- Wasserstein gradient flow: Wasserstein gradient flow
+
+## Reference Book Directions
+- `../../references/books/smooth-manifolds.en.md`: Chapter 13 on Riemannian metrics, providing differential geometry foundations for information geometry
+
+## AI Translation Directions
+- natural gradient → parameter-space-aware optimizer / K-FAC / natural policy gradient
+- fisher metric → distribution distance in parameter space / model merging geometry / loss landscape metric
+- α-divergence → generalized variational inference / robust posterior approximation
+- Bregman divergence → mirror descent / online learning updates / exponential family fitting
+- dual flatness → efficient projection in distribution space / EM algorithm geometry
+
+## Temporary Activation Rules
+When the problem requires mathematics not in the core anchors:
+1. First check whether extended concepts contain a match
+2. If yes, generate a temporary knowledge card based on the lens
+3. If no, enter the Knowledge Gap Protocol

package/knowledge-base/information-geometry/index.md

@@ -0,0 +1,45 @@
+# 信息几何 激活索引 / Information Geometry Activation Index
+
+## 领域信号
+当问题涉及以下信号时，激活本领域方向：
+- 分布族几何：概率分布族本身具有几何结构
+- 参数化无关优化：优化方法应不依赖于具体参数化方式
+- 统计模型复杂度：需要度量统计模型的内在复杂度
+- 分布间距离：需要在分布空间中定义几何距离
+- 自然梯度：梯度方向需要考虑参数空间的几何结构
+
+## 核心锚点
+- `natural-gradient.md` — 自然梯度
+- `fisher-metric.md` — Fisher 度量
+
+## 扩展概念
+当核心锚点不够时，以下概念可能需要临时激活：
+- alpha-divergence：alpha-散度族
+- Amari-Chentsov tensor：Amari-Chentsov 张量与三次微分结构
+- dually flat manifold：对偶平坦流形
+- e-connection / m-connection：指数联络与混合联络
+- Pythagorean theorem for KL：KL 散度的广义勾股定理
+- Bregman divergence：Bregman 散度与凸对偶
+- mirror descent as natural gradient：镜像下降与自然梯度的等价性
+- EM algorithm geometry：EM 算法的几何解释
+- variational Bayes geometry：变分贝叶斯的几何结构
+- information geometry of neural networks（loss landscape curvature）：神经网络的信息几何与损失景观曲率
+- neural tangent kernel as metric：神经切线核作为度量
+- Fisher-Rao gradient flow：Fisher-Rao 梯度流
+- Wasserstein gradient flow：Wasserstein 梯度流
+
+## 参考书方向
+- `../../references/books/smooth-manifolds.md`：第 13 章黎曼度量，为信息几何提供微分几何基础
+
+## AI 翻译方向
+- natural gradient → parameter-space-aware optimizer / K-FAC / natural policy gradient
+- fisher metric → distribution distance in parameter space / model merging geometry / loss landscape metric
+- α-divergence → generalized variational inference / robust posterior approximation
+- Bregman divergence → mirror descent / online learning updates / exponential family fitting
+- dual flatness → efficient projection in distribution space / EM algorithm geometry
+
+## 临时激活规则
+当问题需要的数学不在核心锚点中时：
+1. 先检查扩展概念中是否有匹配
+2. 若有，根据透镜生成临时知识卡
+3. 若无，进入 Knowledge Gap Protocol

package/knowledge-base/information-geometry/natural-gradient.en.md

@@ -12,7 +12,7 @@ $$\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$$
 $$\tilde{\nabla} \mathcal{L}(\theta) = \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
 $$\theta_{t+1} = \theta_t - \eta \, \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
 
-where $\mathcal{I}(\theta)$ is the Fisher information matrix (see `probability/fisher-information.md`).
+where $\mathcal{I}(\theta)$ is the Fisher information matrix (see `../probability/fisher-information.en.md`).
 
 **Equivalent Derivation (Constrained Optimization Perspective)**: The natural gradient is the solution to the following constrained optimization problem —
 $$\min_{\Delta\theta} \mathcal{L}(\theta + \Delta\theta) \quad \text{s.t.} \quad D_{KL}(p_\theta \| p_{\theta+\Delta\theta}) \leq \epsilon$$
@@ -34,21 +34,35 @@ where $A_l = \mathbb{E}[a_l a_l^T]$ (activation covariance) and $B_l = \mathbb{E
 - **Natural Gradient in Variational Inference**: For exponential family parameters, the natural gradient equals the difference in expected sufficient statistics, avoiding Fisher matrix inversion
 
 ## Engineering Feasibility
-- **Dimension 1 Tensorization ⚠️**: The Kronecker factors of the FIM are dense matrices and can be tensorized; the full FIM cannot
-- **Dimension 2 GEMM-mappability ✅**: K-FAC's $A_l^{-1} (\nabla W_l) B_l^{-1}$ is two matrix multiplications, naturally GEMM
-- **Dimension 3 Complexity ⚠️**: K-FAC adds $O(d_A^2 + d_B^2)$ per-layer covariance estimation + $O(d_A^3 + d_B^3)$ matrix inversion; diagonal approximation is $O(d)$
-- **Dimension 4 Memory ⚠️**: Requires additional storage of $A_l$ and $B_l$ per layer ($O(d_A^2 + d_B^2)$); acceptable for LLMs but non-trivial
-- **Dimension 5 Low Precision ⚠️**: Matrix inversion may be unstable in fp16; fp32 or Tikhonov regularization $(A + \epsilon I)^{-1}$ is needed
-- **Dimension 6 Parallelism ✅**: Kronecker factors for each layer are computed independently; fully parallel across layers
-- **Dimension 8 Operator Fusion ✅**: Natural gradient updates can be fused into the parameter update kernel
+- **D1[~]**: The Kronecker factors of the FIM are dense matrices and can be tensorized; the full FIM cannot
+- **D2[v]**: K-FAC's $A_l^{-1} (\nabla W_l) B_l^{-1}$ is two matrix multiplications, naturally GEMM
+- **D3[~]**: K-FAC adds $O(d_A^2 + d_B^2)$ per-layer covariance estimation + $O(d_A^3 + d_B^3)$ matrix inversion; diagonal approximation is $O(d)$
+- **D4[~]**: Requires additional storage of $A_l$ and $B_l$ per layer ($O(d_A^2 + d_B^2)$); acceptable for LLMs but non-trivial
+- **D5[~]**: Matrix inversion may be unstable in fp16; fp32 or Tikhonov regularization $(A + \epsilon I)^{-1}$ is needed
+- **D6[v]**: Kronecker factors for each layer are computed independently; fully parallel across layers
+- **D8[v]**: Natural gradient updates can be fused into the parameter update kernel
 
 ## Risks and Failure Conditions
 - **K-FAC's inter-layer independence assumption is overly strong**: It assumes the Fisher information is block-diagonal across layers, ignoring inter-layer correlations. In deep networks, this may underestimate the effective curvature, leading to excessively large steps. Line search or trust-region safeguards are needed.
 - **Burn-in problem for covariance estimation**: In early training, $A_l, B_l$ estimates are inaccurate and the natural gradient direction may be wrong. The standard practice is to warm up with Adam/SGD for the first few hundred steps, then switch to K-FAC.
 
 ## Further References
-- Distillation draft: `references/books/` — no dedicated information geometry distillation draft at present
+- Distillation draft: `../../references/books/` — no dedicated information geometry distillation draft at present
 - Amari. *Natural Gradient Works Efficiently in Learning.* Neural Computation, 1998
 - Martens & Grosse. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
 - Schulman et al. "Trust Region Policy Optimization." *ICML*, 2015
-- Related knowledge cards: `probability/fisher-information.md`, `information-geometry/fisher-metric.md`
+- Related knowledge cards: `../probability/fisher-information.en.md`, `fisher-metric.en.md`
+
+
+## Routing Extensions
+- If metric definition is needed -> `fisher-metric.en.md` (Fisher metric is the foundation of natural gradient)
+- If a general Riemannian optimization framework is needed -> `../optimization/riemannian-optimization.md` (natural gradient is a special case of Riemannian gradient)
+- If an information-theoretic perspective is needed -> `../probability/fisher-information.en.md` (statistical interpretation of Fisher information)
+
+## Extensible Directions
+- Mirror descent as natural gradient: equivalence on dual spaces
+- Amari's alpha-geometry: alpha-connection family
+- Natural policy gradient (RL): natural gradient in reinforcement learning
+- Natural evolution strategies (NES): NES optimizer
+- Practical natural gradient: efficient implementations such as K-FAC, diagonal approximations
+- Adaptive natural gradient: methods for dynamically estimating Fisher information

package/knowledge-base/information-geometry/natural-gradient.md

@@ -12,7 +12,7 @@ $$\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$$
 $$\tilde{\nabla} \mathcal{L}(\theta) = \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
 $$\theta_{t+1} = \theta_t - \eta \, \mathcal{I}(\theta)^{-1} \nabla_\theta \mathcal{L}(\theta)$$
 
-其中 $\mathcal{I}(\theta)$ 为 Fisher 信息矩阵（详见 `probability/fisher-information.md`）。
+其中 $\mathcal{I}(\theta)$ 为 Fisher 信息矩阵（详见 `../probability/fisher-information.md`）。
 
 **等价推导（约束优化视角）**：自然梯度是以下约束优化问题的解——
 $$\min_{\Delta\theta} \mathcal{L}(\theta + \Delta\theta) \quad \text{s.t.} \quad D_{KL}(p_\theta \| p_{\theta+\Delta\theta}) \leq \epsilon$$
@@ -34,21 +34,35 @@ $$\mathcal{I}_l \approx A_l \otimes B_l$$
 - **变分推断中的自然梯度**：指数族参数的自然梯度 = 充分统计量的期望差，避免 Fisher 矩阵求逆
 
 ## 工程可行性
-- **维度 1 张量化 ⚠️**：FIM 的 Kronecker 因子为稠密矩阵，可张量化；完整 FIM 不可
-- **维度 2 GEMM 可映射 ✅**：K-FAC 的 $A_l^{-1} (\nabla W_l) B_l^{-1}$ 是两次矩阵乘，天然 GEMM
-- **维度 3 复杂度 ⚠️**：K-FAC 每层额外 $O(d_A^2 + d_B^2)$ 协方差估计 + $O(d_A^3 + d_B^3)$ 矩阵逆；对角近似 $O(d)$
-- **维度 4 显存 ⚠️**：需额外存储每层的 $A_l$ 和 $B_l$（$O(d_A^2 + d_B^2)$），对 LLM 可接受但非零
-- **维度 5 低精度 ⚠️**：矩阵求逆在 fp16 下可能不稳定，需 fp32 或 Tikhonov 正则化 $(A + \epsilon I)^{-1}$
-- **维度 6 并行 ✅**：各层 Kronecker 因子独立计算，层间完全并行
-- **维度 8 算子融合 ✅**：自然梯度更新可融入参数更新 kernel
+- **D1[~]**：FIM 的 Kronecker 因子为稠密矩阵，可张量化；完整 FIM 不可
+- **D2[v]**：K-FAC 的 $A_l^{-1} (\nabla W_l) B_l^{-1}$ 是两次矩阵乘，天然 GEMM
+- **D3[~]**：K-FAC 每层额外 $O(d_A^2 + d_B^2)$ 协方差估计 + $O(d_A^3 + d_B^3)$ 矩阵逆；对角近似 $O(d)$
+- **D4[~]**：需额外存储每层的 $A_l$ 和 $B_l$（$O(d_A^2 + d_B^2)$），对 LLM 可接受但非零
+- **D5[~]**：矩阵求逆在 fp16 下可能不稳定，需 fp32 或 Tikhonov 正则化 $(A + \epsilon I)^{-1}$
+- **D6[v]**：各层 Kronecker 因子独立计算，层间完全并行
+- **D8[v]**：自然梯度更新可融入参数更新 kernel
 
 ## 风险与失效条件
 - **K-FAC 的层间独立性假设过强**：假设各层 Fisher 信息块对角，忽略层间相关性。在深网络中可能低估有效曲率，导致步长过大。需搭配线搜索或 trust-region 安全机制。
 - **协方差估计的 burn-in 问题**：训练初期 $A_l, B_l$ 估计不准确，自然梯度方向可能错误。标准做法是前几百步用 Adam/SGD warmup，再切换到 K-FAC。
 
 ## 深入参考
-- 蒸馏稿：`references/books/` 暂无专用信息几何蒸馏稿
+- 蒸馏稿：`../../references/books/` 暂无专用信息几何蒸馏稿
 - Amari. *Natural Gradient Works Efficiently in Learning.* Neural Computation, 1998
 - Martens & Grosse. "Optimizing Neural Networks with Kronecker-Factored Approximate Curvature." *ICML*, 2015
 - Schulman et al. "Trust Region Policy Optimization." *ICML*, 2015
-- 关联知识卡：`probability/fisher-information.md`、`information-geometry/fisher-metric.md`
+- 关联知识卡：`../probability/fisher-information.md`、`fisher-metric.md`
+
+
+## 路由扩展
+- 若需要度量的定义 → `fisher-metric.md`（Fisher 度量是自然梯度的基础）
+- 若需要一般黎曼优化框架 → `../optimization/riemannian-optimization.md`（自然梯度是黎曼梯度的特例）
+- 若需要信息论视角 → `../probability/fisher-information.md`（Fisher 信息的统计解释）
+
+## 可扩展方向
+- 镜像下降即自然梯度（mirror descent as natural gradient）：对偶空间上的等价性
+- Amari 的 alpha-几何（Amari's alpha-geometry）：alpha-联络族
+- 自然策略梯度（natural policy gradient / RL）：强化学习中的自然梯度
+- 自然进化策略（natural evolution strategies）：NES 优化器
+- 实用自然梯度（practical natural gradient）：K-FAC, 对角近似等高效实现
+- 自适应自然梯度（adaptive natural gradient）：动态估计 Fisher 信息的方法