math-skill 3.0.1 → 3.1.0

package/design-patterns/routing/moe-routing.en.md

@@ -9,9 +9,9 @@ Typical scenarios: (1) Sparse MoE layers -- each token selects top-k experts (k
 Core requirement: **sparse activation, load balancing, end-to-end trainability**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/variational.md (discrete variational relaxation, Gumbel-Softmax), lenses/probabilistic.md (information-theoretic routing)
-- Knowledge: knowledge-base/optimization/lagrangian-duality.md (combinatorial variational, integer programming relaxation),
-  knowledge-base/probability/entropy.md (entropy regularization, information bottleneck)
+- Lenses: ../../lenses/variational.en.md (discrete variational relaxation, Gumbel-Softmax), ../../lenses/probabilistic.en.md (information-theoretic routing)
+- Knowledge: ../../knowledge-base/optimization/lagrangian-duality.en.md (combinatorial variational, integer programming relaxation),
+  ../../knowledge-base/probability/entropy.en.md (entropy regularization, information bottleneck)
 
 ## Required Mathematical Background
 - **Mixture Model EM**: p(y|x) = sum_k pi_k(x) * p(y|x, theta_k)
@@ -66,17 +66,17 @@ Auxiliary loss:
 - **Router Z-loss**: L_z = alpha * mean(logsumexp(logits)^2) stabilizes logit magnitudes
 
 ## GPU Feasibility
-- **Tensorization**: Gate logits = X @ W_gate is standard GEMM (N x d) @ (d x K); expert computation is batched GEMM
-- **GEMM-mappable**: Gate requires 1 GEMM; each expert internally is a standard FFN (2 GEMMs + activation)
-- **Complexity**: Gate O(N * d * K); per expert O(N * d * d_ff / k); total FLOPs approximately standard FFN x k
-- **Memory & KV-Cache**: All K expert parameters stored but only k activated; activation memory approximately standard FFN x k
-- **Low-precision stability**: Gate softmax + top-k is safe in fp16; Router Z-loss requires fp32 logsumexp
-- **Parallelism & Communication**: Expert parallelism requires all-to-all communication (each GPU sends/receives N/K tokens); bandwidth-sensitive
-- **Sparse structure**: Top-k routing is naturally sparse (activates k/K fraction of parameters); sparsity = 1 - k/K
-- **Operator fusion**: Gate -> top-k -> softmax -> weighted-sum can be fused; intra-expert FFN can be fused
+- **D1[v]**: Gate logits = X @ W_gate is standard GEMM (N x d) @ (d x K); expert computation is batched GEMM
+- **D2[v]**: Gate requires 1 GEMM; each expert internally is a standard FFN (2 GEMMs + activation)
+- **D3[v]**: Gate O(N * d * K); per expert O(N * d * d_ff / k); total FLOPs approximately standard FFN x k
+- **D4[v]**: All K expert parameters stored but only k activated; activation memory approximately standard FFN x k
+- **D5[v]**: Gate softmax + top-k is safe in fp16; Router Z-loss requires fp32 logsumexp
+- **D6[~]**: Expert parallelism requires all-to-all communication (each GPU sends/receives N/K tokens); bandwidth-sensitive
+- **D7[v]**: Top-k routing is naturally sparse (activates k/K fraction of parameters); sparsity = 1 - k/K
+- **D8[v]**: Gate -> top-k -> softmax -> weighted-sum can be fused; intra-expert FFN can be fused
 
 ## Paper-Worthy Formulation
-"We employ noisy top-k gating for sparse mixture-of-experts routing, activating a k/K fraction of parameters to achieve O(K) parameter capacity at O(k) inference cost. Combined with the load-balancing auxiliary loss L_aux = K * <f, P> and Router Z-loss, expert utilization exceeds 95%, yielding X% performance improvement over dense models under the same FLOPs budget."
+"We employ noisy top-k gating for sparse mixture-of-experts routing, activating only k experts per token to reduce activated compute while using the load-balancing auxiliary loss L_aux = K * <f, P> and Router Z-loss to stabilize routing logits. A paper should report measured expert utilization, overflow rate, all-to-all communication share, and quality deltas against dense baselines under matched FLOPs / parameter budgets; do not use unmeasured fixed-percentage placeholders."
 
 ## Risks
 - Load imbalance: A few experts are over-selected (Matthew effect), while remaining experts receive insufficient training

package/design-patterns/routing/moe-routing.md

@@ -9,9 +9,9 @@
 核心诉求：**稀疏激活、负载均衡、端到端可训练**。
 
 ## 数学思想来源
-- 透镜：lenses/variational.md（离散优化松弛、Gumbel-Softmax）、lenses/probabilistic.md（信息论路由）
-- 知识：knowledge-base/optimization/lagrangian-duality.md（对偶分解、拉格朗日松弛）、
-  knowledge-base/probability/entropy.md（熵正则化、信息瓶颈）
+- 透镜：../../lenses/variational.md（离散优化松弛、Gumbel-Softmax）、../../lenses/probabilistic.md（信息论路由）
+- 知识：../../knowledge-base/optimization/lagrangian-duality.md（对偶分解、拉格朗日松弛）、
+  ../../knowledge-base/probability/entropy.md（熵正则化、信息瓶颈）
 
 ## 需要的数学知识
 - **混合模型 EM**：p(y|x) = Σ_k π_k(x) · p(y|x,θ_k)
@@ -66,19 +66,17 @@
 - **Router Z-loss**：L_z = α·mean(logsumexp(logits)²) 稳定 logits 幅度
 
 ## GPU 可行性
-- **张量化**：gate logits = X@W_gate 为标准 GEMM (N×d)@(d×K)；专家计算为 batched GEMM
-- **GEMM 可映射**：gate 1 次 GEMM；每个专家内部为标准 FFN（2 次 GEMM + activation）
-- **复杂度**：gate O(N·d·K)；每专家 O(N·d·d_ff/k)；总 FLOPs ≈ 标准 FFN × k
-- **显存与 KV-Cache**：K 个专家参数全存储但仅 k 个激活，激活显存 ≈ 标准 FFN × k
-- **低精度稳定**：gate 的 softmax + top-k 在 fp16 安全；Router Z-loss 需 fp32 logsumexp
-- **并行与通信**：专家并行需 all-to-all 通信（每个 GPU 收发 N/K 个 token），带宽敏感
-- **稀疏结构**：top-k 路由天然稀疏（激活 k/K 的参数），稀疏度 = 1 - k/K
-- **算子融合**：gate → top-k → softmax → weighted-sum 可融合；专家内 FFN 可 fuse
+- **D1[v]**：gate logits = X@W_gate 为标准 GEMM (N×d)@(d×K)；专家计算为 batched GEMM
+- **D2[v]**：gate 1 次 GEMM；每个专家内部为标准 FFN（2 次 GEMM + activation）
+- **D3[v]**：gate O(N·d·K)；每专家 O(N·d·d_ff/k)；总 FLOPs ≈ 标准 FFN × k
+- **D4[v]**：K 个专家参数全存储但仅 k 个激活，激活显存 ≈ 标准 FFN × k
+- **D5[v]**：gate 的 softmax + top-k 在 fp16 安全；Router Z-loss 需 fp32 logsumexp
+- **D6[~]**：专家并行需 all-to-all 通信（每个 GPU 收发 N/K 个 token），带宽敏感
+- **D7[v]**：top-k 路由天然稀疏（激活 k/K 的参数），稀疏度 = 1 - k/K
+- **D8[v]**：gate → top-k → softmax → weighted-sum 可融合；专家内 FFN 可 fuse
 
 ## 论文表述方式
-"采用 noisy top-k 门控实现稀疏混合专家路由，激活 k/K 比例的参数以 O(k) 推理代价
-获得 O(K) 参数容量，配合负载均衡辅助损失 L_aux = K·⟨f,P⟩ 和 Router Z-loss 保证
-专家利用率 >95%，在同等 FLOPs 预算下相比 dense 模型提升 X% 性能。"
+"采用 noisy top-k 门控实现稀疏混合专家路由，每个 token 仅激活 k 个专家以降低激活计算，同时用负载均衡辅助损失 L_aux = K·⟨f,P⟩ 和 Router Z-loss 稳定路由 logits。论文中应报告专家利用率、溢出率、all-to-all 通信占比，以及同等 FLOPs/参数预算下相对 dense baseline 的实测质量差异；不得用未实测的固定百分比占位。"
 
 ## 风险
 - 负载不均衡：少数专家被过度选择（马太效应），其余专家得不到训练

package/design-patterns/routing/optimal-transport-routing.en.md

@@ -7,9 +7,9 @@ Typical scenarios: (1) Load-balanced MoE routing -- assigning N tokens to K expe
 Core requirement: **globally optimal assignment, rather than greedy per-point decisions**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/variational.md (convex variational, duality theory), lenses/geometric.md (Wasserstein distance)
-- Knowledge: knowledge-base/optimization/lagrangian-duality.md (duality theory, constrained optimization),
-  knowledge-base/probability/entropy.md (entropy regularization, marginal constraints)
+- Lenses: ../../lenses/variational.en.md (convex variational, duality theory), ../../lenses/geometric.en.md (Wasserstein distance)
+- Knowledge: ../../knowledge-base/optimization/lagrangian-duality.en.md (duality theory, constrained optimization),
+  ../../knowledge-base/probability/entropy.en.md (entropy regularization, marginal constraints)
 
 ## Required Mathematical Background
 - **Discrete Optimal Transport**: min_{P in Pi(mu,nu)} <C, P> = sum_{ij} C_{ij} P_{ij}
@@ -32,7 +32,7 @@ Sinkhorn routing (entropy-regularized):
   for t = 1..T:                       // T = 5 ~ 20 iterations
     u = a / (K_mat @ v)              // row scaling, a = 1/N
     v = b / (K_mat^T @ u)            // column scaling, b = cap / sum(cap)
-  P = diag(u) @ K_mat @ diag(v)     // optimal transport plan, doubly stochastic matrix
+  P = diag(u) @ K_mat @ diag(v)     // optimal transport plan (satisfies marginal constraints; doubly stochastic only when N = K with uniform marginals)
   assignment = argmax(P, dim=1)      // hard assignment (at inference)
   // At training: weighted_features = P @ E  (soft assignment, differentiable)
 
@@ -59,7 +59,7 @@ Capacity constraint (b vector):
 - **Operator fusion**: The exp -> matvec -> division pipeline in a single Sinkhorn step can be fused into a CUDA kernel
 
 ## Paper-Worthy Formulation
-"We formulate token-to-expert routing as an entropy-regularized optimal transport problem, obtaining an epsilon-approximate doubly stochastic assignment matrix via the Sinkhorn-Knopp algorithm within T = 10 iterations. The transport cost is guaranteed to converge to the global optimum within O(eps * log N), while marginal constraints b precisely control the load upper bound per expert."
+"We formulate token-to-expert routing as an entropy-regularized optimal transport problem, obtaining an approximate transport plan via the Sinkhorn-Knopp algorithm within T = 10 iterations. Finite Sinkhorn iterations yield an approximate solution to the entropy-regularized problem (not the exact global optimum); approximation quality depends on the iteration count T and regularization parameter eps. Marginal constraints b control the load upper bound per expert."
 
 ## Risks
 - Excessively small eps causes numerical instability in Sinkhorn (exp overflow); requires log-domain or increasing eps

package/design-patterns/routing/optimal-transport-routing.md

@@ -9,9 +9,9 @@
 核心诉求：**全局最优分配，而非贪心逐点决策**。
 
 ## 数学思想来源
-- 透镜：lenses/variational.md（凸优化、对偶理论）、lenses/geometric.md（Wasserstein 距离）
-- 知识：knowledge-base/optimization/lagrangian-duality.md（对偶理论、约束优化）、
-  knowledge-base/probability/entropy.md（熵正则化、边际约束）
+- 透镜：../../lenses/variational.md（凸优化、对偶理论）、../../lenses/geometric.md（Wasserstein 距离）
+- 知识：../../knowledge-base/optimization/lagrangian-duality.md（对偶理论、约束优化）、
+  ../../knowledge-base/probability/entropy.md（熵正则化、边际约束）
 
 ## 需要的数学知识
 - **离散最优传输**：min_{P∈Π(μ,ν)} ⟨C, P⟩ = Σ_{ij} C_{ij} P_{ij}
@@ -34,7 +34,7 @@ Sinkhorn 路由（熵正则化）：
   for t = 1..T:                     // T=5~20 次迭代
     u = a / (K_mat @ v)            // 行缩放，a = 1/N
     v = b / (K_mat^T @ u)          // 列缩放，b = cap/sum(cap)
-  P = diag(u) @ K_mat @ diag(v)   // 最优传输计划，双随机矩阵
+  P = diag(u) @ K_mat @ diag(v)   // 最优传输计划（满足边际约束；仅当 N=K 且边际均匀时才是双随机矩阵）
   assignment = argmax(P, dim=1)    // 硬分配（推理时）
   // 训练时：weighted_features = P @ E  (软分配，可微)
 
@@ -62,8 +62,8 @@ Sinkhorn 路由（熵正则化）：
 
 ## 论文表述方式
 "将 token-to-expert 路由建模为熵正则化最优传输问题，通过 Sinkhorn-Knopp 算法在 T=10 次
-迭代内求得 ε-近似最优的双随机分配矩阵，理论保证传输代价在 O(ε·log N) 内收敛到全局最优，
-同时通过边际约束 b 精确控制各专家的负载上限。"
+迭代内求得近似最优的传输计划矩阵。有限次迭代给出熵正则化问题的近似解（非精确全局最优），
+近似质量取决于迭代次数 T 和正则化参数 ε；同时通过边际约束 b 控制各专家的负载上限。"
 
 ## 风险
 - ε 过小导致 Sinkhorn 数值不稳定（exp 溢出），需用 log-domain 或增大 ε

package/design-patterns/routing/spectral-clustering-routing.en.md

@@ -10,9 +10,9 @@ Use when routing needs to be based on the intrinsic similarity structure of toke
 Core requirement: **discover the intrinsic cluster structure of data for routing or expert initialization**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/geometric.md (spectral graph theory, Laplacian eigenmaps), lenses/variational.md (relaxation and approximation)
-- Knowledge: knowledge-base/matrix-analysis/projection.md (eigendecomposition, Rayleigh quotient),
-  knowledge-base/differential-geometry/manifold.md (manifold learning, graph cuts)
+- Lenses: ../../lenses/geometric.en.md (spectral graph theory, Laplacian eigenmaps), ../../lenses/variational.en.md (relaxation and approximation)
+- Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (eigendecomposition, Rayleigh quotient),
+  ../../knowledge-base/differential-geometry/manifold.en.md (manifold learning, graph cuts)
 
 ## Required Mathematical Background
 - **Spectral Clustering (Ng-Jordan-Weiss)**:
@@ -24,8 +24,12 @@ Core requirement: **discover the intrinsic cluster structure of data for routing
   W approximately C * W_m^{-1} * C^T, reducing eigendecomposition to m x m
 - **Spectral Relaxation Continuation**: Discrete cluster assignment => continuous eigenvectors => differentiable routing
   Use softmax(U_k * W_proj) instead of hard k-means assignment
-- **Power Iteration Acceleration**: Full eigendecomposition is unnecessary; only the top k eigenvectors are needed
-  Use Lanczos/Arnoldi iteration O(N^2 * k * iter) or randomized SVD O(N * k * log k)
+- **Power Iteration Acceleration**: Full eigendecomposition is unnecessary; only the $k$ **smallest** eigenvectors of Laplacian $L$ are needed
+  ⚠ Power iteration naturally finds the **largest** eigenvectors, so it cannot be applied directly to $L$!
+  Correct approaches: (1) apply power iteration to the similarity matrix $W$ (or normalized $D^{-1/2}WD^{-1/2}$)
+  to find its largest eigenvectors (corresponding to $L$'s smallest); (2) use Lanczos on $L$ with which='SM';
+  (3) shifted inverse iteration $(L - \sigma I)^{-1}$
+  Lanczos/Arnoldi iteration O(N^2 * k * iter) or randomized SVD O(N^2 * k)
 
 ## AI Module Form
 ```
@@ -34,13 +38,16 @@ Input: X in R^{N x d}, number of clusters K
 
 Method 1 - Online Spectral Clustering Routing (periodic updates during training):
   // Update cluster centers every M steps; use nearest neighbor at inference
-  W = exp(-cdist(X_sample, X_sample) / (2 sigma^2))  // m x m sampled similarity
+  W = exp(-(cdist(X_sample, X_sample)**2) / (2 sigma^2))  // m x m RBF similarity
   L = I - D^{-1/2} W D^{-1/2}                         // normalized Laplacian
   U_k = eigsh(L, k=K, which='SM')                     // K smallest eigenvectors
   centers = kmeans(U_k, K)                             // K cluster centers
-  // Routing: project new tokens into spectral space and assign
-  proj = X @ W_proj                                    // N -> K dimensional projection (learnable)
-  assignment = argmin(cdist(proj, centers))             // nearest center assignment
+  // Routing: embed new tokens into spectral space and assign
+  // ⚠ X @ W_proj is just a learnable linear projection, NOT Nystrom extension!
+  // True Nystrom extension: v_new = (1/λ) * W(x_new, X_sample) @ v, where v is eigenvector, λ is eigenvalue
+  proj_nystrom = (1/λ_k) * W_new_sample @ U_k    // Nystrom extension: similarity from x_new to m samples × eigenvectors
+  proj = X @ W_proj                               // alternative: learnable linear projection (not Nystrom, but end-to-end trainable)
+  assignment = argmin(cdist(proj_nystrom, centers))  // nearest center assignment
 
 Method 2 - Differentiable Spectral Routing (end-to-end):
   // Use softmax relaxation instead of hard assignment
@@ -48,18 +55,22 @@ Method 2 - Differentiable Spectral Routing (end-to-end):
   A = exp(sim_matrix / tau)                            // similarity graph (learnable tau)
   D_inv_sqrt = diag(1 / sqrt(sum(A, dim=1) + eps))
   L_norm = I - D_inv_sqrt @ A @ D_inv_sqrt             // normalized Laplacian
-  // Approximate top K eigenvectors (power iteration + orthogonalization)
-  U_k = power_iteration_approx(L_norm, K, steps=5)    // N x K
+  // ⚠ Power iteration finds largest eigenvectors, but spectral clustering needs L_norm's smallest!
+  // Apply power iteration to the normalized similarity matrix: its largest eigenvectors = L_norm's smallest
+  W_norm = D_inv_sqrt @ A @ D_inv_sqrt                 // = I - L_norm
+  U_k = power_iteration_approx(W_norm, K, steps=5)    // N x K (W_norm's largest eigvecs = L_norm's smallest)
   // Soft assignment
   cluster_logits = U_k @ W_cluster                     // N x K (learnable projection)
   route_probs = softmax(cluster_logits / tau_route)     // soft routing probabilities
 
 Method 3 - Anchor Spectral Clustering (large-scale):
   anchors = kmeans_pp(X, m)                             // m anchor points, m << N
-  Z = exp(-cdist(X, anchors) / (2 sigma^2))             // N x m affinity matrix
+  Z = exp(-(cdist(X, anchors)**2) / (2 sigma^2))        // N x m RBF affinity matrix
   L_anchor = I - D_z^{-1/2} Z^T Z D_z^{-1/2}            // m x m Laplacian
   U_k = eigsh(L_anchor, K)                             // m x K eigenvectors
-  route = Z @ U_k @ W_proj                              // N x K routing scores
+  // Nystrom extension: Z @ U_k extends anchor eigenvectors to all N points (non-learnable part)
+  embedding_nystrom = Z @ U_k                          // N x K Nystrom extension embedding
+  route = embedding_nystrom @ W_proj                   // N x K routing scores after learnable projection
 ```
 
 ## Implementable Structures
@@ -75,11 +86,11 @@ Method 3 - Anchor Spectral Clustering (large-scale):
 - **Memory & KV-Cache**: N x N similarity matrix exceeds 64 MB when N > 4096; sampling-based dimensionality reduction is essential
 - **Low-precision stability**: Eigendecomposition recommended in fp32; exp(-dist/sigma^2) in fp16 requires distance clipping
 - **Parallelism & Communication**: Power iteration matvec is highly parallel; k-means assign + update steps can be batch-parallelized
-- **Sparse structure**: k-NN graph replaces fully connected graph; W sparsity > 95%, enabling SpMM acceleration
+- **Sparse structure**: k-NN graph replaces the fully connected graph; W sparsity is about $1-k_{\text{nn}}/N$, enabling SpMM acceleration
 - **Operator fusion**: Diagonal scaling in D^{-1/2} @ A @ D^{-1/2} can be fused; cdist + exp + normalize can be fused
 
 ## Paper-Worthy Formulation
-"We achieve differentiable routing via continuous relaxation of spectral clustering: constructing the normalized Laplacian of the token similarity graph, reducing the O(N^2) eigendecomposition to O(Nm + K^3) through Nystrom approximation, and enabling GPU-friendly online spectral clustering via power iteration. Clustering quality, measured by Normalized Cut, guarantees an O(sqrt(log(N/K))) approximation ratio."
+"We implement routing through a continuous relaxation of spectral clustering: construct the normalized Laplacian of the token similarity graph, use Nystrom / anchor approximations plus power iteration to avoid a full O(N^3) eigendecomposition, and express the main work as GEMM, matvecs, and k-means. Normalized Cut can be reported as a clustering-quality metric, but approximation ratios depend on graph-model, sampling, and solver assumptions and should not be claimed unconditionally."
 
 ## Risks
 - The memory and computation cost of the N x N similarity matrix does not scale for long sequences; sampling or k-NN sparsification is mandatory

package/design-patterns/routing/spectral-clustering-routing.md

@@ -10,9 +10,9 @@
 核心诉求：**发现数据内在的簇结构，用于路由或专家初始化**。
 
 ## 数学思想来源
-- 透镜：lenses/geometric.md（谱图论、拉普拉斯特征映射）、lenses/variational.md（松弛与近似）
-- 知识：knowledge-base/matrix-analysis/projection.md（特征值分解、Rayleigh 商）、
-  knowledge-base/differential-geometry/manifold.md（流形学习、图割）
+- 透镜：../../lenses/geometric.md（谱图论、拉普拉斯特征映射）、../../lenses/variational.md（松弛与近似）
+- 知识：../../knowledge-base/matrix-analysis/projection.md（特征值分解、Rayleigh 商）、
+  ../../knowledge-base/differential-geometry/manifold.md（流形学习、图割）
 
 ## 需要的数学知识
 - **谱聚类 (Ng-Jordan-Weiss)**：
@@ -24,8 +24,11 @@
   W ≈ C · W_m^{-1} · C^T，将特征分解降维到 m×m
 - **谱松弛连续化**：离散聚类分配 → 连续特征向量 → 可微路由
   用 softmax(U_k · W_proj) 替代硬 k-means 分配
-- **幂迭代加速**：不需完整特征分解，只需前 k 个特征向量
-  用 Lanczos/Arnoldi 迭代 O(N²·k·iter) 或 randomized SVD O(N·k·log k)
+- **幂迭代加速**：不需完整特征分解，只需拉普拉斯 $L$ 的前 $k$ 个**最小**特征向量
+  ⚠ 幂迭代天然找**最大**特征向量，因此不能直接对 $L$ 做幂迭代！
+  正确做法：(1) 对相似度矩阵 $W$（或归一化 $D^{-1/2}WD^{-1/2}$）做幂迭代找最大特征向量
+  （对应 $L$ 的最小特征向量）；(2) 对 $L$ 用 Lanczos + which='SM'；(3) 移位逆迭代 $(L - \sigma I)^{-1}$
+  Lanczos/Arnoldi 迭代 O(N²·k·iter) 或 randomized SVD O(N²·k)
 
 ## AI 模块形式
 ```
@@ -34,13 +37,16 @@
 
 方法1 - 在线谱聚类路由（训练时周期性更新）：
   // 每 M 步更新一次聚类中心，推理时用最近邻
-  W = exp(-cdist(X_sample, X_sample) / (2σ²))  // m×m 采样相似度
+  W = exp(-(cdist(X_sample, X_sample)**2) / (2σ²))  // m×m RBF 相似度
   L = I - D^{-1/2} W D^{-1/2}                    // 归一化拉普拉斯
   U_k = eigsh(L, k=K, which='SM')                // 前 K 个最小特征向量
   centers = kmeans(U_k, K)                        // K 个聚类中心
-  // 路由：将新 token 投影到谱空间后分配
-  proj = X @ W_proj                               // N→K 维投影（可学习）
-  assignment = argmin(cdist(proj, centers))        // 最近中心分配
+  // 路由：将新 token 嵌入到谱空间后分配
+  // ⚠ X @ W_proj 仅为可学习线性投影，不是 Nyström 扩展！
+  // 真正的 Nyström 扩展：v_new = (1/λ) * W(x_new, X_sample) @ v，其中 v 为特征向量，λ 为特征值
+  proj_nystrom = (1/λ_k) * W_new_sample @ U_k    // Nyström 扩展：x_new 到 m 个采样点的相似度 × 特征向量
+  proj = X @ W_proj                               // 替代方案：可学习线性投影（非 Nyström，但可端到端训练）
+  assignment = argmin(cdist(proj_nystrom, centers))  // 最近中心分配
 
 方法2 - 可微谱路由（端到端）：
   // 用 softmax 松弛替代硬分配
@@ -48,18 +54,22 @@
   A = exp(sim_matrix / τ)                          // 相似度图（可学习 τ）
   D_inv_sqrt = diag(1 / sqrt(sum(A, dim=1) + ε))
   L_norm = I - D_inv_sqrt @ A @ D_inv_sqrt         // 归一化拉普拉斯
-  // 近似前 K 个特征向量（幂迭代 + 正交化）
-  U_k = power_iteration_approx(L_norm, K, steps=5)  // N×K
+  // ⚠ 幂迭代找最大特征向量，但谱聚类需要 L_norm 的最小特征向量！
+  // 对归一化相似度矩阵做幂迭代：其最大特征向量 = L_norm 的最小特征向量
+  W_norm = D_inv_sqrt @ A @ D_inv_sqrt             // = I - L_norm
+  U_k = power_iteration_approx(W_norm, K, steps=5)  // N×K（W_norm 的最大特征向量 = L_norm 的最小）
   // 软分配
   cluster_logits = U_k @ W_cluster                  // N×K（可学习投影）
   route_probs = softmax(cluster_logits / τ_route)    // 软路由概率
 
 方法3 - 锚点谱聚类（大规模）：
   anchors = kmeans_pp(X, m)                         // m 个锚点，m << N
-  Z = exp(-cdist(X, anchors) / (2σ²))               // N×m 亲和矩阵
+  Z = exp(-(cdist(X, anchors)**2) / (2σ²))          // N×m RBF 亲和矩阵
   L_anchor = I - D_z^{-1/2} Z^T Z D_z^{-1/2}        // m×m 拉普拉斯
   U_k = eigsh(L_anchor, K)                          // m×K 特征向量
-  route = Z @ U_k @ W_proj                           // N×K 路由分数
+  // Nyström 扩展：Z @ U_k 将锚点特征向量扩展到全部 N 个点（非可学习部分）
+  embedding_nystrom = Z @ U_k                       // N×K Nyström 扩展嵌入
+  route = embedding_nystrom @ W_proj                 // N×K 可学习投影后的路由分数
 ```
 
 ## 可实现结构
@@ -75,13 +85,12 @@
 - **显存与 KV-Cache**：N×N 相似度矩阵在 N>4096 时 >64MB，必须采样降维
 - **低精度稳定**：特征分解建议 fp32；exp(-dist/σ²) 在 fp16 下需 clip distance
 - **并行与通信**：幂迭代的 matvec 高度并行；k-means 的 assign+update 可批并行
-- **稀疏结构**：k-NN 图替代全连接图，W 稀疏度 >95%，SpMM 加速
+- **稀疏结构**：k-NN 图替代全连接图，W 稀疏度约为 $1-k_{\text{nn}}/N$，可用 SpMM 加速
 - **算子融合**：D^{-1/2}@A@D^{-1/2} 的对角缩放可融合；cdist+exp+normalize 可融合
 
 ## 论文表述方式
-"利用谱聚类的连续松弛实现可微路由：构造 token 相似度图的归一化拉普拉斯，
-通过 Nyström 近似将 O(N²) 特征分解降为 O(Nm+K³)，配合幂迭代实现 GPU 友好的
-在线谱聚类，聚类质量以 Normalized Cut 衡量保证 O(√(log N/K)) 的近似比。"
+"利用谱聚类的连续松弛实现路由：构造 token 相似度图的归一化拉普拉斯，
+通过 Nyström / 锚点近似和幂迭代避免完整 O(N³) 特征分解，并将主要计算转化为 GEMM、matvec 与 k-means。Normalized Cut 可作为聚类质量指标，但近似比依赖图模型、采样策略和求解器假设，不能无条件宣称。"
 
 ## 风险
 - N×N 相似度矩阵的显存和计算在长序列下不可扩展，必须采样或 k-NN 稀疏化

package/knowledge-base/differential-geometry/connection.en.md

@@ -45,7 +45,22 @@ GPU friendliness: the core challenge of connections is "serial ODE integration."
 
 ## Further References
 
-- Distillation notes: references/books/differential-geometry.md (Ch 12 Connections and Covariant Derivatives, Section 12.2 Connection Forms, Section 12.4 Ehresmann, Section 12.12 G-Connections)
-- Distillation notes: references/books/differential-geometry.md (Section 6.8 Principal Bundles, Section 9.8 Electromagnetism)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 12 Connections and Covariant Derivatives, Section 12.2 Connection Forms, Section 12.4 Ehresmann, Section 12.12 G-Connections)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Section 6.8 Principal Bundles, Section 9.8 Electromagnetism)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 12 (Section 12.1--Section 12.12, complete connection theory)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 13.1 Levi-Civita Connection
+
+
+## Routing Extensions
+- If curvature definition is needed -> `curvature.en.md` (curvature tensor derived from connection)
+- If parallel transport and geodesics are needed -> `geodesic.en.md` (geodesics are auto-parallel curves of parallel transport)
+- If covariant derivative computation is needed -> `tangent-space.en.md` (covariant differentiation on tangent space)
+
+## Extensible Directions
+- Levi-Civita connection: unique torsion-free metric connection on Riemannian manifolds
+- Christoffel symbols: connection components in coordinate basis
+- Holonomy: effect of parallel transport around closed curves
+- Torsion: antisymmetric part of a connection
+- Affine connection: general affine connection theory
+- Ehresmann connection: horizontal distributions on fiber bundles
+- Gauge connection: gauge fields as connections in physics

package/knowledge-base/differential-geometry/connection.md

@@ -45,7 +45,22 @@ GPU 友好度：联络的核心挑战是"串行 ODE 积分"。
 
 ## 深入参考
 
-- 蒸馏稿：references/books/differential-geometry.md（Ch 12 Connections and Covariant Derivatives, §12.2 联络形式, §12.4 Ehresmann, §12.12 G-联络）
-- 蒸馏稿：references/books/differential-geometry.md（§6.8 Principal Bundles, §9.8 Electromagnetism）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 12 Connections and Covariant Derivatives, §12.2 联络形式, §12.4 Ehresmann, §12.12 G-联络）
+- 蒸馏稿：../../references/books/differential-geometry.md（§6.8 Principal Bundles, §9.8 Electromagnetism）
 - 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 12（§12.1-§12.12 完整联络理论）
 - 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.1 Levi-Civita Connection
+
+
+## 路由扩展
+- 若需要曲率的定义 → `curvature.md`（曲率张量由联络导出）
+- 若需要平行移动与测地线 → `geodesic.md`（测地线是平行移动的自平行曲线）
+- 若需要协变导数的具体计算 → `tangent-space.md`（切空间上的协变微分）
+
+## 可扩展方向
+- Levi-Civita 联络：黎曼流形的唯一无挠度量联络
+- Christoffel 符号：联络在坐标基下的分量
+- 和乐（holonomy）：平行移动绕闭曲线的效果
+- 挠率（torsion）：联络的非对称部分
+- 仿射联络（affine connection）：一般的仿射联络理论
+- Ehresmann 联络：纤维丛上的水平分布
+- 规范联络（gauge connection）：物理中的规范场作为联络

package/knowledge-base/differential-geometry/curvature.en.md

@@ -45,5 +45,18 @@ GPU friendliness: the core difficulty of curvature is that "the full tensor cann
 
 ## Further References
 
-- Distillation notes: references/books/differential-geometry.md (Ch 12 Section 12.5/Section 12.10 Curvature, Ch 13 Section 13.2 Riemann Curvature, Section 13.7 Jacobi Fields, Section 13.11 Rauch Comparison)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 12 Section 12.5/Section 12.10 Curvature, Ch 13 Section 13.2 Riemann Curvature, Section 13.7 Jacobi Fields, Section 13.11 Rauch Comparison)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 13.2 Riemann Curvature Tensor, Section 13.7 Jacobi Fields
+
+
+## Routing Extensions
+- If local geometric analysis is needed -> `metric-tensor.en.md` (metric determines local curvature)
+- If global topological analysis is needed -> `../topology/persistent-homology.md` (persistent homology captures global topology)
+- If curvature's effect on stability is involved -> `../matrix-analysis/matrix-perturbation.md` (curvature-induced perturbation amplification)
+
+## Extensible Directions
+- Sectional / Ricci / scalar curvature: curvature concepts at different dimensions
+- Gauss-Bonnet theorem: relationship between curvature and topological invariants
+- Comparison theorems (Toponogov, Bishop-Gromov): geometric comparison under curvature constraints
+- Curvature flow: mean curvature flow and Ricci flow
+- Cartan-Hadamard theorem: global structure of non-positively curved manifolds

package/knowledge-base/differential-geometry/curvature.md

@@ -45,5 +45,18 @@ GPU 友好度：曲率的核心难点是"不能物化全张量"。
 
 ## 深入参考
 
-- 蒸馏稿：references/books/differential-geometry.md（Ch 12 §12.5/§12.10 Curvature, Ch 13 §13.2 Riemann Curvature, §13.7 Jacobi Fields, §13.11 Rauch Comparison）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 12 §12.5/§12.10 Curvature, Ch 13 §13.2 Riemann Curvature, §13.7 Jacobi Fields, §13.11 Rauch Comparison）
 - 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.2 Riemann Curvature Tensor, §13.7 Jacobi Fields
+
+
+## 路由扩展
+- 若需要局部几何分析 → `metric-tensor.md`（度量决定局部曲率）
+- 若需要全局拓扑分析 → `../topology/persistent-homology.md`（持续同调捕捉全局拓扑）
+- 若涉及曲率对稳定性的影响 → `../matrix-analysis/matrix-perturbation.md`（曲率导致的扰动放大）
+
+## 可扩展方向
+- 截面/Ricci/标量曲率（sectional / Ricci / scalar curvature）：不同维度的曲率概念
+- Gauss-Bonnet 定理：曲率与拓扑不变量的关系
+- 比较定理（Toponogov, Bishop-Gromov）：曲率约束下的几何比较
+- 曲率流（curvature flow）：平均曲率流与 Ricci 流
+- Cartan-Hadamard 定理：非正曲率流形的全局结构

package/knowledge-base/differential-geometry/geodesic.en.md

@@ -45,7 +45,21 @@ GPU friendliness depends on whether a closed-form retraction exists:
 
 ## Further References
 
-- Distillation notes: references/books/differential-geometry.md (Ch 13 Section 13.4 Geodesics, Section 13.11 Rauch Comparison)
-- Distillation notes: references/books/smooth-manifolds.md (Ch 20 The Exponential Map)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 13 Section 13.4 Geodesics, Section 13.11 Rauch Comparison)
+- Distillation notes: ../../references/books/smooth-manifolds.en.md (Ch 20 The Exponential Map)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Section 13.4 Geodesics
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, Ch 20 (exponential map, retraction prototype)
+
+
+## Routing Extensions
+- If the distance definition is needed -> `metric-tensor.en.md` (metric tensor determines geodesics)
+- If used as a retraction -> `../optimization/riemannian-optimization.md` (exponential map as retraction)
+- If deviation from flat space is needed -> `curvature.en.md` (curvature controls geodesic deviation)
+
+## Extensible Directions
+- Conjugate points: zeros of Jacobi fields along geodesics
+- Cut locus: critical points where geodesics lose optimality
+- Hopf-Rinow theorem: completeness and geodesic existence
+- Geodesic convexity: convex sets and convex functions on manifolds
+- Jacobi field: linearization of geodesic variation
+- Geodesic regression: regression analysis on manifolds

package/knowledge-base/differential-geometry/geodesic.md

@@ -45,7 +45,21 @@ GPU 友好度取决于是否有闭式 retraction：
 
 ## 深入参考
 
-- 蒸馏稿：references/books/differential-geometry.md（Ch 13 §13.4 Geodesics, §13.11 Rauch Comparison）
-- 蒸馏稿：references/books/smooth-manifolds.md（Ch 20 The Exponential Map）
+- 蒸馏稿：../../references/books/differential-geometry.md（Ch 13 §13.4 Geodesics, §13.11 Rauch Comparison）
+- 蒸馏稿：../../references/books/smooth-manifolds.md（Ch 20 The Exponential Map）
 - 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, §13.4 Geodesics
 - 原书：John M. Lee, *Introduction to Smooth Manifolds*, Ch 20（指数映射、retraction 原型）
+
+
+## 路由扩展
+- 若需要距离的具体定义 → `metric-tensor.md`（度量张量决定测地线）
+- 若用作收缩映射 → `../optimization/riemannian-optimization.md`（指数映射作为收缩映射）
+- 若需要偏离平坦空间的程度 → `curvature.md`（曲率控制测地线偏差）
+
+## 可扩展方向
+- 共轭点（conjugate points）：测地线上 Jacobi 场的零点
+- 割迹（cut locus）：测地线失去最优性的临界点
+- Hopf-Rinow 定理：完备性与测地线存在性
+- 测地凸性（geodesic convexity）：流形上的凸集与凸函数
+- Jacobi 场（Jacobi field）：测地线变分的线性化
+- 测地回归（geodesic regression）：流形上的回归分析

package/knowledge-base/differential-geometry/index.en.md

@@ -0,0 +1,52 @@
+# Differential Geometry Activation Index
+
+## Domain Signals
+Activate this domain direction when the problem involves:
+- Parameter space curvature: geometric curvature of parameter space affects optimization or inference
+- Manifold constraints: parameters must remain on a specific manifold
+- Geodesic distance: need to compute shortest paths or distances along a manifold
+- Connection / parallel transport: need to compare tangent vectors at different points on a manifold
+- Riemannian metric: need to define inner products and lengths on a manifold
+- Index raising/lowering: need to convert between tangent and cotangent spaces
+
+## Core Anchors
+- `manifold.en.md` — Fundamental concepts of manifolds
+- `tangent-space.en.md` — Tangent space and tangent map
+- `metric-tensor.en.md` — Metric tensor and Riemannian metric
+- `geodesic.en.md` — Geodesics
+- `curvature.en.md` — Curvature
+- `connection.en.md` — Connection and covariant derivative
+
+## Extended Concepts
+When core anchors are insufficient, the following concepts may need temporary activation:
+- Lie derivative: Lie derivative and rate of change along a vector field
+- exterior calculus (differential forms, wedge product, exterior derivative): exterior differentiation and differential form operations
+- Hodge decomposition: Hodge decomposition theorem
+- fiber bundle: fiber bundles and sections
+- principal bundle: principal bundles and gauge theory
+- holonomy: holonomy group and closure of parallel transport
+- Riemannian submersion: Riemannian submersions and quotient metrics
+- symmetric space: symmetric spaces
+- homogeneous space: homogeneous spaces G/H
+- Cartan geometry: Cartan geometry and moving frames
+- spin structure: spin structures
+- Dirac operator: Dirac operator
+- Ricci flow: Ricci flow
+
+## Reference Book Directions
+- `../../references/books/differential-geometry.en.md`: core content of Riemannian geometry
+- `../../references/books/smooth-manifolds.en.md`: smooth manifolds, vector bundles, and differential forms
+
+## AI Translation Directions
+- manifold → latent space geometry / manifold-valued predictions / hyperbolic embeddings
+- tangent space → local linearization of parameter updates / tangent-space features
+- metric tensor → learned distance metrics / Fisher-Rao natural gradient / metric-aware loss
+- geodesic → interpolation on manifolds / geodesic regularization / shortest-path loss
+- curvature → loss landscape analysis / curvature-aware optimization / flatness seeking
+- connection → parallel transport of features / gauge-equivariant layers
+
+## Temporary Activation Rules
+When the problem requires mathematics not in the core anchors:
+1. First check whether extended concepts contain a match
+2. If yes, generate a temporary knowledge card based on the lens
+3. If no, enter the Knowledge Gap Protocol

package/knowledge-base/differential-geometry/index.md

@@ -0,0 +1,52 @@
+# 微分几何 激活索引 / Differential Geometry Activation Index
+
+## 领域信号
+当问题涉及以下信号时，激活本领域方向：
+- 参数空间曲率：参数空间的几何弯曲影响优化或推断
+- 流形约束：参数必须保持在特定流形上
+- 测地线距离：需要沿流形计算最短路径或距离
+- 联络/平行移动：需要在流形上比较不同点的切向量
+- 黎曼度量：需要定义流形上的内积和长度
+- 指标升降：需要在切空间与余切空间之间转换
+
+## 核心锚点
+- `manifold.md` — 流形的基本概念
+- `tangent-space.md` — 切空间与切映射
+- `metric-tensor.md` — 度量张量与黎曼度量
+- `geodesic.md` — 测地线
+- `curvature.md` — 曲率
+- `connection.md` — 联络与协变导数
+
+## 扩展概念
+当核心锚点不够时，以下概念可能需要临时激活：
+- Lie derivative：Lie 导数与沿向量场的变化率
+- exterior calculus（differential forms, wedge product, exterior derivative）：外微分与微分形式运算
+- Hodge decomposition：Hodge 分解定理
+- fiber bundle：纤维丛与截面
+- principal bundle：主丛与规范理论
+- holonomy：和乐群与平行移动的闭合性
+- Riemannian submersion：黎曼浸没与商度量
+- symmetric space：对称空间
+- homogeneous space：齐性空间 G/H
+- Cartan geometry：Cartan 几何与活动标架
+- spin structure：旋量结构
+- Dirac operator：Dirac 算子
+- Ricci flow：Ricci 流
+
+## 参考书方向
+- `../../references/books/differential-geometry.md`：黎曼几何的核心内容
+- `../../references/books/smooth-manifolds.md`：光滑流形、向量丛和微分形式
+
+## AI 翻译方向
+- manifold → latent space geometry / manifold-valued predictions / hyperbolic embeddings
+- tangent space → local linearization of parameter updates / tangent-space features
+- metric tensor → learned distance metrics / Fisher-Rao natural gradient / metric-aware loss
+- geodesic → interpolation on manifolds / geodesic regularization / shortest-path loss
+- curvature → loss landscape analysis / curvature-aware optimization / flatness seeking
+- connection → parallel transport of features / gauge-equivariant layers
+
+## 临时激活规则
+当问题需要的数学不在核心锚点中时：
+1. 先检查扩展概念中是否有匹配
+2. 若有，根据透镜生成临时知识卡
+3. 若无，进入 Knowledge Gap Protocol

package/knowledge-base/differential-geometry/manifold.en.md

@@ -41,7 +41,20 @@ Moderate GPU friendliness. Coordinate chart transformations are element-wise map
 
 ## Further References
 
-- Distillation notes: references/books/smooth-manifolds.md (Ch 1-2 Smooth Manifolds / Smooth Maps)
-- Distillation notes: references/books/differential-geometry.md (Ch 1-2 Differentiable Manifolds / The Tangent Structure)
+- Distillation notes: ../../references/books/smooth-manifolds.en.md (Ch 1-2 Smooth Manifolds / Smooth Maps)
+- Distillation notes: ../../references/books/differential-geometry.en.md (Ch 1-2 Differentiable Manifolds / The Tangent Structure)
 - Original text: John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Ch 1-2 (topological manifolds, smooth structures, partition of unity)
 - Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 1-2
+
+
+## Routing Extensions
+- If local structure analysis is needed -> `tangent-space.en.md` (tangent space provides local linear approximation)
+- If distance definition is needed -> `metric-tensor.en.md` (metric tensor defines distance on manifolds)
+- If optimization on manifolds is needed -> `../optimization/riemannian-optimization.md` (Riemannian optimization methods)
+
+## Extensible Directions
+- Submanifold: embedded and immersed submanifolds
+- Product manifold: direct product construction of multiple manifolds
+- Quotient manifold: quotient space under equivalence relations
+- Stiefel / Grassmann manifolds: orthogonal matrix and subspace manifolds
+- Manifold learning (Isomap / LLE / diffusion maps): discovering low-dimensional manifolds from high-dimensional data