math-skill 3.0.0 → 3.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (183) hide show
  1. package/README.en-US.md +50 -30
  2. package/README.md +26 -6
  3. package/agents/math-critic.en.md +30 -28
  4. package/agents/math-critic.md +30 -30
  5. package/commands/ask.en.md +15 -0
  6. package/commands/ask.md +6 -1
  7. package/design-patterns/attention/equivariant-attention.en.md +17 -11
  8. package/design-patterns/attention/equivariant-attention.md +16 -10
  9. package/design-patterns/attention/geometry-aware-attention.en.md +11 -11
  10. package/design-patterns/attention/geometry-aware-attention.md +10 -10
  11. package/design-patterns/attention/information-bottleneck-attention.en.md +40 -20
  12. package/design-patterns/attention/information-bottleneck-attention.md +40 -20
  13. package/design-patterns/attention/projection-attention.en.md +12 -12
  14. package/design-patterns/attention/projection-attention.md +11 -11
  15. package/design-patterns/attention/spectral-attention.en.md +27 -13
  16. package/design-patterns/attention/spectral-attention.md +26 -12
  17. package/design-patterns/compression/leverage-score-selection.en.md +14 -14
  18. package/design-patterns/compression/leverage-score-selection.md +14 -14
  19. package/design-patterns/compression/low-rank-kv-cache.en.md +40 -13
  20. package/design-patterns/compression/low-rank-kv-cache.md +39 -13
  21. package/design-patterns/compression/spectral-token-pruning.en.md +17 -15
  22. package/design-patterns/compression/spectral-token-pruning.md +17 -15
  23. package/design-patterns/compression/topology-preserving-compression.en.md +11 -11
  24. package/design-patterns/compression/topology-preserving-compression.md +11 -11
  25. package/design-patterns/loss/constraint-penalty.en.md +9 -5
  26. package/design-patterns/loss/constraint-penalty.md +10 -7
  27. package/design-patterns/loss/contrastive-loss.en.md +38 -12
  28. package/design-patterns/loss/contrastive-loss.md +39 -13
  29. package/design-patterns/loss/information-bottleneck-loss.en.md +6 -6
  30. package/design-patterns/loss/information-bottleneck-loss.md +7 -7
  31. package/design-patterns/loss/orthogonality-loss.en.md +25 -18
  32. package/design-patterns/loss/orthogonality-loss.md +25 -19
  33. package/design-patterns/loss/variational-loss.en.md +5 -5
  34. package/design-patterns/loss/variational-loss.md +6 -6
  35. package/design-patterns/overview.en.md +36 -0
  36. package/design-patterns/overview.md +36 -0
  37. package/design-patterns/representation/equivariant-split.en.md +5 -5
  38. package/design-patterns/representation/equivariant-split.md +5 -6
  39. package/design-patterns/representation/manifold-representation.en.md +13 -13
  40. package/design-patterns/representation/manifold-representation.md +14 -15
  41. package/design-patterns/representation/shared-private-decomposition.en.md +13 -8
  42. package/design-patterns/representation/shared-private-decomposition.md +13 -10
  43. package/design-patterns/representation/subspace-alignment.en.md +13 -13
  44. package/design-patterns/representation/subspace-alignment.md +13 -13
  45. package/design-patterns/routing/graph-routing.en.md +12 -12
  46. package/design-patterns/routing/graph-routing.md +12 -12
  47. package/design-patterns/routing/moe-routing.en.md +13 -13
  48. package/design-patterns/routing/moe-routing.md +13 -15
  49. package/design-patterns/routing/optimal-transport-routing.en.md +6 -6
  50. package/design-patterns/routing/optimal-transport-routing.md +7 -7
  51. package/design-patterns/routing/spectral-clustering-routing.en.md +27 -16
  52. package/design-patterns/routing/spectral-clustering-routing.md +27 -18
  53. package/knowledge-base/differential-geometry/connection.en.md +17 -2
  54. package/knowledge-base/differential-geometry/connection.md +17 -2
  55. package/knowledge-base/differential-geometry/curvature.en.md +14 -1
  56. package/knowledge-base/differential-geometry/curvature.md +14 -1
  57. package/knowledge-base/differential-geometry/geodesic.en.md +16 -2
  58. package/knowledge-base/differential-geometry/geodesic.md +16 -2
  59. package/knowledge-base/differential-geometry/index.en.md +52 -0
  60. package/knowledge-base/differential-geometry/index.md +52 -0
  61. package/knowledge-base/differential-geometry/manifold.en.md +15 -2
  62. package/knowledge-base/differential-geometry/manifold.md +15 -2
  63. package/knowledge-base/differential-geometry/metric-tensor.en.md +16 -2
  64. package/knowledge-base/differential-geometry/metric-tensor.md +17 -3
  65. package/knowledge-base/differential-geometry/tangent-space.en.md +17 -3
  66. package/knowledge-base/differential-geometry/tangent-space.md +17 -3
  67. package/knowledge-base/information-geometry/fisher-metric.en.md +32 -13
  68. package/knowledge-base/information-geometry/fisher-metric.md +32 -13
  69. package/knowledge-base/information-geometry/index.en.md +45 -0
  70. package/knowledge-base/information-geometry/index.md +45 -0
  71. package/knowledge-base/information-geometry/natural-gradient.en.md +24 -10
  72. package/knowledge-base/information-geometry/natural-gradient.md +24 -10
  73. package/knowledge-base/lie-theory/equivariance.en.md +18 -3
  74. package/knowledge-base/lie-theory/equivariance.md +18 -3
  75. package/knowledge-base/lie-theory/group-action.en.md +16 -2
  76. package/knowledge-base/lie-theory/group-action.md +16 -2
  77. package/knowledge-base/lie-theory/index.en.md +52 -0
  78. package/knowledge-base/lie-theory/index.md +52 -0
  79. package/knowledge-base/lie-theory/lie-algebra.en.md +17 -2
  80. package/knowledge-base/lie-theory/lie-algebra.md +17 -2
  81. package/knowledge-base/lie-theory/lie-group.en.md +17 -3
  82. package/knowledge-base/lie-theory/lie-group.md +17 -3
  83. package/knowledge-base/lie-theory/representation.en.md +17 -2
  84. package/knowledge-base/lie-theory/representation.md +17 -2
  85. package/knowledge-base/matrix-analysis/index.en.md +49 -0
  86. package/knowledge-base/matrix-analysis/index.md +49 -0
  87. package/knowledge-base/matrix-analysis/low-rank-approximation.en.md +20 -7
  88. package/knowledge-base/matrix-analysis/low-rank-approximation.md +20 -7
  89. package/knowledge-base/matrix-analysis/matrix-perturbation.en.md +13 -1
  90. package/knowledge-base/matrix-analysis/matrix-perturbation.md +13 -1
  91. package/knowledge-base/matrix-analysis/positive-semidefinite.en.md +20 -6
  92. package/knowledge-base/matrix-analysis/positive-semidefinite.md +20 -6
  93. package/knowledge-base/matrix-analysis/projection.en.md +14 -1
  94. package/knowledge-base/matrix-analysis/projection.md +14 -1
  95. package/knowledge-base/matrix-analysis/spectral-decomposition.en.md +14 -1
  96. package/knowledge-base/matrix-analysis/spectral-decomposition.md +14 -1
  97. package/knowledge-base/optimization/constrained-optimization.en.md +17 -3
  98. package/knowledge-base/optimization/constrained-optimization.md +17 -3
  99. package/knowledge-base/optimization/convex-optimization.en.md +15 -2
  100. package/knowledge-base/optimization/convex-optimization.md +15 -2
  101. package/knowledge-base/optimization/index.en.md +50 -0
  102. package/knowledge-base/optimization/index.md +50 -0
  103. package/knowledge-base/optimization/lagrangian-duality.en.md +18 -4
  104. package/knowledge-base/optimization/lagrangian-duality.md +18 -4
  105. package/knowledge-base/optimization/proximal-method.en.md +15 -1
  106. package/knowledge-base/optimization/proximal-method.md +15 -1
  107. package/knowledge-base/optimization/riemannian-optimization.en.md +18 -4
  108. package/knowledge-base/optimization/riemannian-optimization.md +18 -4
  109. package/knowledge-base/overview.en.md +20 -8
  110. package/knowledge-base/overview.md +20 -8
  111. package/knowledge-base/probability/concentration-inequality.en.md +21 -7
  112. package/knowledge-base/probability/concentration-inequality.md +21 -7
  113. package/knowledge-base/probability/entropy.en.md +24 -8
  114. package/knowledge-base/probability/entropy.md +23 -7
  115. package/knowledge-base/probability/fisher-information.en.md +24 -10
  116. package/knowledge-base/probability/fisher-information.md +24 -10
  117. package/knowledge-base/probability/index.en.md +51 -0
  118. package/knowledge-base/probability/index.md +51 -0
  119. package/knowledge-base/probability/information-bottleneck.en.md +24 -8
  120. package/knowledge-base/probability/information-bottleneck.md +24 -8
  121. package/knowledge-base/probability/kl-divergence.en.md +21 -8
  122. package/knowledge-base/probability/kl-divergence.md +21 -8
  123. package/knowledge-base/topology/euler-characteristic.en.md +15 -2
  124. package/knowledge-base/topology/euler-characteristic.md +15 -2
  125. package/knowledge-base/topology/fundamental-group.en.md +16 -2
  126. package/knowledge-base/topology/fundamental-group.md +16 -2
  127. package/knowledge-base/topology/index.en.md +48 -0
  128. package/knowledge-base/topology/index.md +48 -0
  129. package/knowledge-base/topology/persistent-homology.en.md +17 -2
  130. package/knowledge-base/topology/persistent-homology.md +17 -2
  131. package/lenses/algorithmic.en.md +1 -1
  132. package/lenses/algorithmic.md +1 -1
  133. package/lenses/axiomatization.en.md +1 -1
  134. package/lenses/axiomatization.md +1 -1
  135. package/lenses/categorical.en.md +1 -1
  136. package/lenses/categorical.md +1 -1
  137. package/lenses/causal.en.md +1 -1
  138. package/lenses/causal.md +1 -1
  139. package/lenses/duality.en.md +1 -1
  140. package/lenses/duality.md +1 -1
  141. package/lenses/game.en.md +2 -2
  142. package/lenses/game.md +2 -2
  143. package/lenses/geometric.en.md +1 -1
  144. package/lenses/geometric.md +1 -1
  145. package/lenses/local-to-global.en.md +1 -1
  146. package/lenses/local-to-global.md +1 -1
  147. package/lenses/perturbation.en.md +1 -1
  148. package/lenses/perturbation.md +1 -1
  149. package/lenses/probabilistic.en.md +1 -1
  150. package/lenses/probabilistic.md +1 -1
  151. package/lenses/projection.en.md +1 -1
  152. package/lenses/projection.md +1 -1
  153. package/lenses/spectral.en.md +2 -2
  154. package/lenses/spectral.md +2 -2
  155. package/lenses/symmetry.en.md +1 -1
  156. package/lenses/symmetry.md +1 -1
  157. package/lenses/topological.en.md +2 -2
  158. package/lenses/topological.md +2 -2
  159. package/lenses/variational.en.md +1 -1
  160. package/lenses/variational.md +1 -1
  161. package/package.json +5 -5
  162. package/references/agentic-workflow.en.md +3 -3
  163. package/references/agentic-workflow.md +1 -1
  164. package/references/books/abstract-algebra.en.md +124 -0
  165. package/references/books/abstract-algebra.md +17 -17
  166. package/references/books/algebraic-geometry-rising-sea.en.md +171 -0
  167. package/references/books/algebraic-geometry-rising-sea.md +7 -7
  168. package/references/books/differential-geometry.en.md +140 -0
  169. package/references/books/differential-geometry.md +23 -23
  170. package/references/books/matrix-analysis.en.md +146 -0
  171. package/references/books/matrix-analysis.md +32 -32
  172. package/references/books/micro-lie-theory.en.md +116 -0
  173. package/references/books/micro-lie-theory.md +16 -16
  174. package/references/books/optimization-ml.en.md +163 -0
  175. package/references/books/optimization-ml.md +20 -21
  176. package/references/books/smooth-manifolds.en.md +105 -0
  177. package/references/books/smooth-manifolds.md +15 -15
  178. package/references/gpu-friendly-math.en.md +35 -9
  179. package/references/gpu-friendly-math.md +33 -5
  180. package/references/skill-index.en.md +105 -0
  181. package/references/skill-index.md +105 -0
  182. package/skills/math-research-activator/SKILL.en.md +72 -88
  183. package/skills/math-research-activator/SKILL.md +61 -78
@@ -1,12 +1,12 @@
1
1
  # Variational Loss
2
- > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as verified / ⚠️ retrofittable (needs validation) / infeasible. Unmarked claims are theoretically possible but require engineering validation.
2
+ > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
3
3
 
4
4
  ## Applicable Problems
5
5
  When sampling from latent variable distributions is required to generate diverse outputs. Typical scenarios: (1) Expert selection introduces discrete latent variables $z$ that need end-to-end variational; (2) Representation spaces need to model uncertainty; (3) Generative routing requires sampling from posterior distributions $p(z|x)$; (4) Bayesian mixture of experts. Core objective: **model distributions rather than point estimates in latent space, enabling uncertainty awareness and diversity**.
6
6
 
7
7
  ## Mathematical Inspiration
8
- - Lenses: lenses/probabilistic.md (variational inference and ELBO), lenses/probabilistic.md (posterior and prior)
9
- - Knowledge: knowledge-base/probability/entropy.md (KL divergence, variational families), knowledge-base/probability/kl-divergence.md (ELBO derivation)
8
+ - Lenses: ../../lenses/probabilistic.en.md (variational inference and ELBO), ../../lenses/probabilistic.en.md (posterior and prior)
9
+ - Knowledge: ../../knowledge-base/probability/entropy.en.md (KL divergence, variational families), ../../knowledge-base/probability/kl-divergence.en.md (ELBO derivation)
10
10
 
11
11
  ## Required Mathematical Knowledge
12
12
  - **ELBO (Evidence Lower Bound)**: $\log p(x) \geq \mathbb{E}_{q(z|x)}[\log p(x|z)] - \text{KL}(q(z|x) \| p(z))$ -- the first term is reconstruction likelihood, the second regularizes the posterior toward the prior
@@ -31,8 +31,8 @@ Method 2 - Gumbel-Softmax (discrete expert selection):
31
31
  L_gumbel = CE(task_head(z_soft * features), y) // straight-through estimator backprop
32
32
 
33
33
  Method 3 - Variational Information Bottleneck:
34
- L_IB = -I(X; Z) + beta * I(Z; Y) // maximize MI between Z and X, minimize redundancy between Z and Y
35
- ~ E[-log q(x|z)] + beta * KL(q(z|x) || p(z)) // variational approximation
34
+ L_IB = I(X; Z) - beta * I(Z; Y) // minimize information redundancy of Z about X, maximize predictive power of Z about Y
35
+ ~ E[-log q(y|z)] + beta * KL(q(z|x) || p(z)) // variational approximation (first term is prediction loss, second is compression penalty)
36
36
  ```
37
37
 
38
38
  ## Implementable Architectures
@@ -1,5 +1,5 @@
1
1
  # Variational Loss(变分损失)
2
- > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / 不可行」。未标注的视为理论可行,需工程验证。
2
+ > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行,需工程验证。
3
3
 
4
4
  ## 适用问题
5
5
  当需要从隐变量分布中采样并生成多样化输出时使用。典型场景:
@@ -8,9 +8,9 @@
8
8
  核心诉求:**在潜空间中建模分布而非点估计,获得不确定性感知与多样性**。
9
9
 
10
10
  ## 数学思想来源
11
- - 透镜:lenses/probabilistic.md(变分推断与 ELBO)、lenses/probabilistic.md(后验与先验)
12
- - 知识:knowledge-base/probability/entropy.md(KL 散度、变分族)、
13
- knowledge-base/probability/kl-divergence.md(ELBO 推导)
11
+ - 透镜:../../lenses/probabilistic.md(变分推断与 ELBO)、../../lenses/probabilistic.md(后验与先验)
12
+ - 知识:../../knowledge-base/probability/entropy.md(KL 散度、变分族)、
13
+ ../../knowledge-base/probability/kl-divergence.md(ELBO 推导)
14
14
 
15
15
  ## 需要的数学知识
16
16
  - **ELBO(证据下界)**:log p(x) ≥ E_{q(z|x)}[log p(x|z)] - KL(q(z|x) ‖ p(z))
@@ -39,8 +39,8 @@
39
39
  L_gumbel = CE(task_head(z_soft ⊙ features), y) // 直通估计器反向传播
40
40
 
41
41
  方法3 - 信息瓶颈变分:
42
- L_IB = -I(X; Z) + β · I(Z; Y) // 最大化 Z X 的互信息,最小化 Z Y 的冗余
43
- ≈ E[-log q(x|z)] + β · KL(q(z|x) ‖ p(z)) // 变分近似
42
+ L_IB = I(X; Z) - β · I(Z; Y) // 最小化 Z X 的信息冗余,最大化 Z Y 的预测能力
43
+ ≈ E[-log q(y|z)] + β · KL(q(z|x) ‖ p(z)) // 变分近似(第一项为预测损失,第二项为压缩惩罚)
44
44
  ```
45
45
 
46
46
  ## 可实现结构
@@ -0,0 +1,36 @@
1
+ # Design Translation Pattern Library
2
+
3
+ > The design pattern library is not a complete model repository but a collection of "math → AI design" translation prototypes.
4
+ > When existing patterns are insufficient, generate a temporary design candidate from the mathematical source and label it as a temporary design pattern, rather than refusing or force-fitting existing patterns.
5
+ > Rigor convention: claims about complexity, memory, FlashAttention / Tensor Core / KV-Cache, and other engineering properties should be marked [v]/[~]/[x]; unmarked claims are theoretical possibilities requiring engineering validation.
6
+
7
+ ## Translation Paradigms
8
+
9
+ | Mathematical Structure | AI Design Direction |
10
+ |----------------------|-------------------|
11
+ | Projection / Decomposition | subspace split / conflict removal / low-rank attention |
12
+ | Spectral structure | token pruning / stability monitor / spectral filter |
13
+ | Information theory | bottleneck loss / entropy gate / uncertainty routing |
14
+ | Geometry / Metric | manifold representation / metric-aware update |
15
+ | Topology | topology-preserving compression / obstruction loss |
16
+ | Duality | constrained optimization / primal-dual training |
17
+ | Symmetry / Group | equivariant features / weight sharing / orbit aggregation |
18
+ | Variational | energy minimization / variational regularization |
19
+
20
+ ## Relationship to Activation Anchors
21
+
22
+ ```
23
+ Activation anchors provide math tools → Design patterns translate into AI modules
24
+ ```
25
+
26
+ When an activation anchor triggers the Knowledge Gap Protocol to generate a temporary knowledge card, a corresponding temporary design candidate should also be generated here.
27
+
28
+ ## By Component Type
29
+
30
+ | Component | Directory | Count |
31
+ |-----------|-----------|-------|
32
+ | Attention | `attention/` | 5 |
33
+ | Loss Functions | `loss/` | 5 |
34
+ | Routing | `routing/` | 4 |
35
+ | Representation | `representation/` | 4 |
36
+ | Compression | `compression/` | 4 |
@@ -0,0 +1,36 @@
1
+ # 设计翻译原型库 / Design Translation Pattern Library
2
+
3
+ > 设计模式库不是完整模型仓库,而是"数学 → AI 设计"的翻译原型集合。
4
+ > 当现有模式不足时,应根据数学来源生成临时设计草案,并标记为 temporary design pattern,而不是拒绝或强行套用已有模式。
5
+ > 严谨性约定:复杂度、显存、FlashAttention/Tensor Core/KV-Cache 等工程结论需用 [v]/[~]/[x] 标注;未标注结论视为理论可行但需工程验证。
6
+
7
+ ## 翻译范式
8
+
9
+ | 数学结构 | AI 设计方向 |
10
+ |---------|-----------|
11
+ | 投影/分解 | subspace split / conflict removal / low-rank attention |
12
+ | 谱结构 | token pruning / stability monitor / spectral filter |
13
+ | 信息论 | bottleneck loss / entropy gate / uncertainty routing |
14
+ | 几何/度量 | manifold representation / metric-aware update |
15
+ | 拓扑 | topology-preserving compression / obstruction loss |
16
+ | 对偶 | constrained optimization / primal-dual training |
17
+ | 对称/群 | equivariant features / weight sharing / orbit aggregation |
18
+ | 变分 | energy minimization / variational regularization |
19
+
20
+ ## 与知识锚点的关系
21
+
22
+ ```
23
+ 知识锚点提供数学工具 → 设计模式翻译成 AI 模块
24
+ ```
25
+
26
+ 当知识锚点触发 Knowledge Gap Protocol 生成临时知识卡时,此处也应生成对应的临时设计草案。
27
+
28
+ ## 按组件类型
29
+
30
+ | 组件 | 目录 | 模式数 |
31
+ |------|------|--------|
32
+ | 注意力 | `attention/` | 5 |
33
+ | 损失函数 | `loss/` | 5 |
34
+ | 路由 | `routing/` | 4 |
35
+ | 表示 | `representation/` | 4 |
36
+ | 压缩 | `compression/` | 4 |
@@ -1,5 +1,5 @@
1
1
  # Equivariant Split
2
- > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as verified / ⚠️ retrofittable (needs validation) / infeasible. Unmarked claims are theoretically possible but require engineering validation.
2
+ > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
3
3
 
4
4
  ## Applicable Problems
5
5
  Use when the input possesses symmetries (e.g., permutations, rotations, translations) and the representation should preserve or reflect those symmetries.
@@ -10,9 +10,9 @@ Typical scenarios: (1) Token permutation equivariance -- representation should c
10
10
  Core requirement: **encode symmetry priors into network architecture to reduce learning burden and improve generalization**.
11
11
 
12
12
  ## Mathematical Inspiration
13
- - Lenses: lenses/geometric.md (group actions, invariant/equivariant maps), lenses/probabilistic.md (symmetry and information redundancy)
14
- - Knowledge: knowledge-base/matrix-analysis/projection.md (group representation theory, irreducible representations),
15
- knowledge-base/differential-geometry/manifold.md (Lie groups, homogeneous spaces)
13
+ - Lenses: ../../lenses/geometric.en.md (group actions, invariant/equivariant maps), ../../lenses/probabilistic.en.md (symmetry and information redundancy)
14
+ - Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (group representation theory, irreducible representations),
15
+ ../../knowledge-base/differential-geometry/manifold.en.md (Lie groups, homogeneous spaces)
16
16
 
17
17
  ## Required Mathematical Background
18
18
  - **Group Action and Equivariance**: A map f is equivariant with respect to group G if and only if f(g * x) = g * f(x) for all g in G
@@ -78,7 +78,7 @@ Method 3 - Group convolution / group pooling:
78
78
  - **Operator fusion**: Split -> batched matmul -> concat can be fused; group pooling scatter + reduce can be fused
79
79
 
80
80
  ## Paper-Worthy Formulation
81
- "Leveraging Schur's lemma from group representation theory, we decompose the d-dimensional feature space into a direct sum of irreducible representations of symmetry group G, with each component independently processed by equivariance-constrained linear layers. This achieves a |G| / sum(m_i^2) improvement in parameter efficiency while guaranteeing network equivariance under group actions, reducing generalization error by O(1/sqrt(|G|)) on symmetric data."
81
+ "Leveraging Schur's lemma from group representation theory, we decompose the d-dimensional feature space into a direct sum of irreducible representations of the symmetry group G, with each component processed by equivariance-constrained linear layers. Under the assumed group action this enforces equivariance and reduces learnable degrees of freedom; the exact parameter savings depend on the representation decomposition and channel multiplicities. Generalization gains must be measured on tasks with the corresponding symmetry and should not be stated as an unconditional O(1/sqrt(|G|)) rate."
82
82
 
83
83
  ## Risks
84
84
  - Improper group selection (too large constrains cause underfitting, too small fails to capture symmetries)
@@ -1,5 +1,5 @@
1
1
  # Equivariant Split(等变分割)
2
- > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / 不可行」。未标注的视为理论可行,需工程验证。
2
+ > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行,需工程验证。
3
3
 
4
4
  ## 适用问题
5
5
  当输入具有对称性(如排列、旋转、平移),且表示应保持或反映这些对称性时使用。
@@ -10,9 +10,9 @@
10
10
  核心诉求:**让网络结构编码对称性先验,减少学习负担,提升泛化**。
11
11
 
12
12
  ## 数学思想来源
13
- - 透镜:lenses/geometric.md(群作用、不变/等变映射)、lenses/probabilistic.md(对称性与信息冗余)
14
- - 知识:knowledge-base/matrix-analysis/projection.md(群表示论、不可约表示)、
15
- knowledge-base/differential-geometry/manifold.md(李群、齐性空间)
13
+ - 透镜:../../lenses/geometric.md(群作用、不变/等变映射)、../../lenses/probabilistic.md(对称性与信息冗余)
14
+ - 知识:../../knowledge-base/matrix-analysis/projection.md(群表示论、不可约表示)、
15
+ ../../knowledge-base/differential-geometry/manifold.md(李群、齐性空间)
16
16
 
17
17
  ## 需要的数学知识
18
18
  - **群作用与等变性**:映射 f 对群 G 等变 ⟺ f(g·x) = g·f(x), ∀g∈G
@@ -79,8 +79,7 @@
79
79
 
80
80
  ## 论文表述方式
81
81
  "利用群表示论的 Schur 引理,将 d 维特征空间按对称群 G 的不可约表示分解为直和 ⊕mᵢVᵢ,
82
- 每个分量使用受等变约束的线性层独立处理,参数效率提升 |G|/Σmᵢ² 倍,
83
- 同时保证网络对群作用的等变性,在对称数据上泛化误差降低 O(1/√|G|)。"
82
+ 每个分量使用受等变约束的线性层独立处理,从而在满足群作用假设时严格保持等变性并减少可学习自由度。参数节省量取决于具体表示分解与通道重数;泛化收益需在具有相应对称性的任务上实测,不能无条件写成 O(1/√|G|)。"
84
83
 
85
84
  ## 风险
86
85
  - 群的选择不当(过大约束导致欠拟合,过小无法捕获对称性)
@@ -1,5 +1,5 @@
1
1
  # Manifold Representation
2
- > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as verified / ⚠️ retrofittable (needs validation) / infeasible. Unmarked claims are theoretically possible but require engineering validation.
2
+ > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
3
3
 
4
4
  ## Applicable Problems
5
5
  Use when input data resides in a high-dimensional space but is actually distributed on a low-dimensional manifold. Typical scenarios:
@@ -10,9 +10,9 @@ Use when input data resides in a high-dimensional space but is actually distribu
10
10
  Core requirement: **leverage the low-dimensional manifold structure of data to improve representation efficiency and generalization**.
11
11
 
12
12
  ## Mathematical Inspiration
13
- - Lenses: lenses/geometric.md (Riemannian geometry, geodesics, curvature), lenses/probabilistic.md (intrinsic dimensionality)
14
- - Knowledge: knowledge-base/differential-geometry/manifold.md (manifolds, tangent spaces, exponential maps),
15
- knowledge-base/matrix-analysis/projection.md (SVD, low-rank approximation, PCA)
13
+ - Lenses: ../../lenses/geometric.en.md (Riemannian geometry, geodesics, curvature), ../../lenses/probabilistic.en.md (intrinsic dimensionality)
14
+ - Knowledge: ../../knowledge-base/differential-geometry/manifold.en.md (manifolds, tangent spaces, exponential maps),
15
+ ../../knowledge-base/matrix-analysis/projection.en.md (SVD, low-rank approximation, PCA)
16
16
 
17
17
  ## Required Mathematical Background
18
18
  - **Manifold Hypothesis**: Data x in R^D actually lies on a smooth d-dimensional manifold M with d << D
@@ -66,17 +66,17 @@ Method 3 - Riemannian Optimization (optimize directly on the manifold):
66
66
  - **Adaptive d**: Local dimensionality varies across regions; estimate locally via PCA
67
67
 
68
68
  ## GPU Feasibility
69
- - **Tensorization**: Local linear projections are GEMM (d x D) @ (D x N); graph Laplacian regularization is SpMM
70
- - **GEMM-mappable**: K linear projections in Chart MoE form a batched GEMM (K x d x D) @ (D x N)
71
- - **Complexity**: k-NN construction O(N * D * log N) requires FAISS; manifold regularization O(N^2) requires sampling approximation
72
- - **Memory & KV-Cache**: K chart parameters K * d * D typically < 10 MB; k-NN graph N * k * 4 bytes
73
- - **Low-precision stability**: Distance computations and exp in fp16 require attention to numerical range; Riemannian retract recommended in fp32
74
- - **Parallelism & Communication**: K charts computed independently, perfectly parallel; k-NN search accelerated with FAISS GPU
75
- - **Sparse structure**: k-NN graph is naturally sparse; manifold regularization L is a sparse matrix, enabling SpMM acceleration
76
- - **Operator fusion**: Matmul + bias + activation within a chart can be fused; gating softmax + weighted-sum can be fused
69
+ - **D1[v]**: Local linear projections are GEMM (d x D) @ (D x N); graph Laplacian regularization is SpMM
70
+ - **D2[v]**: K linear projections in Chart MoE form a batched GEMM (K x d x D) @ (D x N)
71
+ - **D3[~]**: k-NN construction O(N * D * log N) requires FAISS; manifold regularization O(N^2) requires sampling approximation
72
+ - **D4[v]**: K chart parameters K * d * D typically < 10 MB; k-NN graph N * k * 4 bytes
73
+ - **D5[~]**: Distance computations and exp in fp16 require attention to numerical range; Riemannian retract recommended in fp32
74
+ - **D6[v]**: K charts computed independently, perfectly parallel; k-NN search accelerated with FAISS GPU
75
+ - **D7[v]**: k-NN graph is naturally sparse; manifold regularization L is a sparse matrix, enabling SpMM acceleration
76
+ - **D8[v]**: Matmul + bias + activation within a chart can be fused; gating softmax + weighted-sum can be fused
77
77
 
78
78
  ## Paper-Worthy Formulation
79
- "Based on the manifold hypothesis, we constrain D-dimensional token representations to a d*-dimensional (intrinsic dimension estimate) submanifold, achieving piecewise-linear approximation via K local coordinate charts (Chart MoE) and preserving global geodesic structure through graph Laplacian manifold regularization, with theoretical guarantees that embedding error converges at rate O(N^{-2/d*})."
79
+ "Based on the manifold hypothesis, we model D-dimensional token representations as a low-intrinsic-dimensional structure, approximate it with K local coordinate charts (Chart MoE), and use graph-Laplacian regularization to encourage local-neighborhood consistency. Embedding-error or geodesic-preservation bounds require assumptions on sampling density, manifold smoothness, graph construction, and estimator choice; in practice report neighborhood preservation, reconstruction error, and downstream metrics."
80
80
 
81
81
  ## Risks
82
82
  - Inaccurate intrinsic dimension d* estimation leads to over-compression or dimension waste
@@ -1,5 +1,5 @@
1
1
  # Manifold Representation(流形表示)
2
- > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / 不可行」。未标注的视为理论可行,需工程验证。
2
+ > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行,需工程验证。
3
3
 
4
4
  ## 适用问题
5
5
  当输入数据虽然在高维空间中,但实际分布在低维流形上时使用。典型场景:
@@ -10,9 +10,9 @@
10
10
  核心诉求:**利用数据的低维流形结构,提高表示效率和泛化能力**。
11
11
 
12
12
  ## 数学思想来源
13
- - 透镜:lenses/geometric.md(黎曼几何、测地线、曲率)、lenses/probabilistic.md(内在维度)
14
- - 知识:knowledge-base/differential-geometry/manifold.md(流形、切空间、指数映射)、
15
- knowledge-base/matrix-analysis/projection.md(SVD、低秩近似、PCA)
13
+ - 透镜:../../lenses/geometric.md(黎曼几何、测地线、曲率)、../../lenses/probabilistic.md(内在维度)
14
+ - 知识:../../knowledge-base/differential-geometry/manifold.md(流形、切空间、指数映射)、
15
+ ../../knowledge-base/matrix-analysis/projection.md(SVD、低秩近似、PCA)
16
16
 
17
17
  ## 需要的数学知识
18
18
  - **流形假设**:数据 x ∈ R^D 实际分布在 d << D 维的光滑流形 M 上
@@ -66,19 +66,18 @@
66
66
  - **自适应 d**:不同区域的局部维度不同,用 PCA 局部估计
67
67
 
68
68
  ## GPU 可行性
69
- - **张量化**:局部线性投影为 GEMM (d×D)@(D×N);图拉普拉斯正则为 SpMM
70
- - **GEMM 可映射**:Chart MoE 的 K 个线性投影为 batched GEMM (K×d×D)@(D×N)
71
- - **复杂度**:k-NN 构建 O(N·D·log N) 需 FAISS;流形正则 O(N²) 需采样近似
72
- - **显存与 KV-Cache**:K 个 chart 参数 K·d·D 通常 <10MB;k-NN 图 N·k·4 bytes
73
- - **低精度稳定**:距离计算和 exp 在 fp16 下需注意数值范围;黎曼 retract 建议 fp32
74
- - **并行与通信**:K 个 chart 独立计算,完美并行;k-NN 搜索用 FAISS GPU 加速
75
- - **稀疏结构**:k-NN 图天然稀疏,流形正则 L 为稀疏矩阵,SpMM 加速
76
- - **算子融合**:chart 内的 matmul+bias+activation 可融合;门控 softmax+weighted-sum 可融合
69
+ - **D1[v]**:局部线性投影为 GEMM (d×D)@(D×N);图拉普拉斯正则为 SpMM
70
+ - **D2[v]**:Chart MoE 的 K 个线性投影为 batched GEMM (K×d×D)@(D×N)
71
+ - **D3[~]**:k-NN 构建 O(N·D·log N) 需 FAISS;流形正则 O(N²) 需采样近似
72
+ - **D4[v]**:K 个 chart 参数 K·d·D 通常 <10MB;k-NN 图 N·k·4 bytes
73
+ - **D5[~]**:距离计算和 exp 在 fp16 下需注意数值范围;黎曼 retract 建议 fp32
74
+ - **D6[v]**:K 个 chart 独立计算,完美并行;k-NN 搜索用 FAISS GPU 加速
75
+ - **D7[v]**:k-NN 图天然稀疏,流形正则 L 为稀疏矩阵,SpMM 加速
76
+ - **D8[v]**:chart 内的 matmul+bias+activation 可融合;门控 softmax+weighted-sum 可融合
77
77
 
78
78
  ## 论文表述方式
79
- "基于流形假设将 D 维 token 表示约束到 d*(内在维度估计)维子流形上,
80
- 通过 K 个局部坐标卡(Chart MoE)实现分段线性近似,配合图拉普拉斯流形正则
81
- 保持全局测地线结构,理论上保证嵌入误差以 O(N^{-2/d*}) 收敛。"
79
+ "基于流形假设将 D 维 token 表示建模为低内在维度结构,
80
+ 通过 K 个局部坐标卡(Chart MoE)实现分段近似,并用图拉普拉斯正则鼓励局部邻域一致性。嵌入误差或测地线保持只在采样密度、流形光滑性、图构造和估计器假设满足时才有理论界,实际应报告邻域保持、重构误差和下游指标。"
82
81
 
83
82
  ## 风险
84
83
  - 内在维度 d* 估计不准确导致过度压缩或维度浪费
@@ -1,5 +1,5 @@
1
1
  # Shared-Private Decomposition
2
- > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as verified / ⚠️ retrofittable (needs validation) / infeasible. Unmarked claims are theoretically possible but require engineering validation.
2
+ > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
3
3
 
4
4
  ## Applicable Problems
5
5
  Use in multi-task/multi-domain learning when representations need to be decomposed into a "cross-task common component" and a "task-specific component."
@@ -9,16 +9,16 @@ Typical scenarios: (1) Multi-task MoE -- Shared experts handle linguistic common
9
9
  Core requirement: **explicitly separate commonality from individuality to prevent negative transfer and catastrophic forgetting**.
10
10
 
11
11
  ## Mathematical Inspiration
12
- - Lenses: lenses/projection.md (subspace decomposition, direct sum decomposition), lenses/probabilistic.md (information decomposition)
13
- - Knowledge: knowledge-base/matrix-analysis/projection.md (direct sum decomposition V = U + W, projection operators),
14
- knowledge-base/probability/kl-divergence.md (information decomposition: shared/synergy/unique)
12
+ - Lenses: ../../lenses/projection.en.md (subspace decomposition, direct sum decomposition), ../../lenses/probabilistic.en.md (information decomposition)
13
+ - Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (direct sum decomposition V = U + W, projection operators),
14
+ ../../knowledge-base/probability/kl-divergence.en.md (information decomposition: shared/synergy/unique)
15
15
 
16
16
  ## Required Mathematical Background
17
17
  - **Direct Sum Decomposition**: R^d = S + P, where S intersect P = {0}, and every x = x_S + x_P is unique
18
18
  Projection matrices P_S + P_P = I, P_S * P_P = 0
19
19
  - **Information Decomposition (Williams & Beer PID)**:
20
20
  I(X; Y_1, Y_2) = Shared + Unique_1 + Unique_2 + Synergy
21
- Shared = the redundant information component min(I(X; Y_1), I(X; Y_2))
21
+ Shared is the redundant-information term; min(I(X;Y_1), I(X;Y_2)) is only an early/simplified proxy, not the general Williams-Beer redundancy definition
22
22
  - **Low-Rank + Sparse Decomposition (RPCA)**: M = L + S, where L is low-rank (common) + S is sparse (specific)
23
23
  Solved via nuclear norm + L1 norm convex relaxation
24
24
  - **CCA (Canonical Correlation Analysis)**: max corr(W_1^T X, W_2^T Y), extracting shared variation between two sets of variables
@@ -42,12 +42,17 @@ Method 2 - Gated Decomposition (dynamic weighting):
42
42
  z = gate * z_shared + (1 - gate) * z_private
43
43
  // Gating allows per-dimension selection of shared/private contribution ratios
44
44
 
45
- Method 3 - Adversarial Decomposition (information-theoretic guarantee):
45
+ Method 3 - Adversarial Decomposition (information-theoretic proxy):
46
46
  z_shared = E_shared(X)
47
47
  z_private = E_private[t](X)
48
48
  // Shared should be indistinguishable across tasks (adversarial gradient):
49
+ // ⚠ Sign correctness is critical! Original formula L_adv = -CE with flip_gradient creates a double sign reversal:
50
+ // Discriminator minimizes -CE → gets worse at prediction; Encoder (sees +CE after reversal) → helps discriminator → shared becomes MORE task-specific!
51
+ // Correct approach: L_adv = +CE, combined with flip_gradient:
52
+ // Discriminator minimizes +CE → learns to predict task from shared
53
+ // Encoder (gradient reversed) sees -CE → maximizes discriminator loss → shared becomes task-independent
49
54
  task_pred = classifier(z_shared.flip_gradient())
50
- L_adv = -CE(task_pred, t) // Shared contains no task information
55
+ L_adv = CE(task_pred, t) // Discriminator: minimize CE to predict task; Encoder: sees -CE after reversal, maximizes CE, making shared task-independent
51
56
  // Private should be discriminative across tasks:
52
57
  L_private = CE(classifier(z_private), t)
53
58
  L = L_task + lambda_adv * L_adv + lambda_priv * L_private
@@ -75,7 +80,7 @@ Dimension Allocation Principle:
75
80
  - **Operator fusion**: Additive fusion is trivial; gated fusion sigmoid -> multiply -> add can be fused
76
81
 
77
82
  ## Paper-Worthy Formulation
78
- "We decompose the multi-task representation space R^d into a direct sum S + P. The Shared subspace ensures task-invariance through adversarial training (H(T|Z_s) -> log T), while the Private subspace guarantees information complementarity with Shared via orthogonal regularization. Theoretical analysis shows that negative transfer decays as O(||P_S * P_P||_F) with increasing orthogonality."
83
+ "We decompose the multi-task representation space R^d into a shared subspace S and task-private subspaces P: the shared branch reduces task identifiability through adversarial training, while the private branches use orthogonality / decorrelation regularization to reduce linear overlap with S. Information complementarity and reduced negative transfer must be validated with task-transfer matrices, mutual-information / PID proxies, and ablations; they are not automatically guaranteed by orthogonality alone."
79
84
 
80
85
  ## Risks
81
86
  - Min-max variational in adversarial training is unstable; the gradient reversal scale and lambda_adv require careful tuning
@@ -1,5 +1,5 @@
1
1
  # Shared-Private Decomposition(共享-私有分解)
2
- > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / 不可行」。未标注的视为理论可行,需工程验证。
2
+ > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行,需工程验证。
3
3
 
4
4
  ## 适用问题
5
5
  多任务/多领域学习中,需要将表示分解为"跨任务共性部分"和"任务特异部分"。
@@ -9,16 +9,16 @@
9
9
  核心诉求:**显式分离共性与个性,避免负迁移和灾难性遗忘**。
10
10
 
11
11
  ## 数学思想来源
12
- - 透镜:lenses/projection.md(子空间分解、直和分解)、lenses/probabilistic.md(信息分解)
13
- - 知识:knowledge-base/matrix-analysis/projection.md(直和分解 V = U ⊕ W、投影算子)、
14
- knowledge-base/probability/kl-divergence.md(信息分解:shared/synergy/unique)
12
+ - 透镜:../../lenses/projection.md(子空间分解、直和分解)、../../lenses/probabilistic.md(信息分解)
13
+ - 知识:../../knowledge-base/matrix-analysis/projection.md(直和分解 V = U ⊕ W、投影算子)、
14
+ ../../knowledge-base/probability/kl-divergence.md(信息分解:shared/synergy/unique)
15
15
 
16
16
  ## 需要的数学知识
17
17
  - **直和分解**:R^d = S ⊕ P,其中 S ∩ P = {0},每个 x = x_S + x_P 唯一
18
18
  投影矩阵 P_S + P_P = I,P_S · P_P = 0
19
19
  - **信息分解 (Williams & Beer PID)**:
20
20
  I(X;Y₁,Y₂) = Shared + Unique₁ + Unique₂ + Synergy
21
- Shared = min(I(X;Y₁), I(X;Y₂)) 的冗余信息部分
21
+ Shared 为冗余信息项;min(I(X;Y₁), I(X;Y₂)) 只是早期/简化代理,不是 Williams-Beer 冗余的一般定义
22
22
  - **低秩+稀疏分解 (RPCA)**:M = L + S,L 低秩(共性)+ S 稀疏(特异)
23
23
  通过核范数 + L1 范数凸松弛求解
24
24
  - **CCA (典型相关分析)**:max corr(W₁^T X, W₂^T Y),提取两组变量的共享变异
@@ -42,12 +42,17 @@
42
42
  z = gate ⊙ z_shared + (1 - gate) ⊙ z_private
43
43
  // 门控允许逐维度选择 shared/private 的贡献比例
44
44
 
45
- 方法3 - 对抗分解(信息论保证):
45
+ 方法3 - 对抗分解(信息论代理):
46
46
  z_shared = E_shared(X)
47
47
  z_private = E_private[t](X)
48
48
  // Shared 应无法区分任务(对抗梯度):
49
+ // ⚠ 符号正确性至关重要!原公式 L_adv = -CE 与 flip_gradient 形成双重符号反转:
50
+ // 判别器最小化 -CE → 变差;编码器(梯度反转后看到 +CE)→ 帮助判别器 → shared 变得更任务相关!
51
+ // 正确做法:L_adv = +CE,配合 flip_gradient:
52
+ // 判别器最小化 +CE → 学会从 shared 预测任务
53
+ // 编码器(梯度反转)看到 -CE → 最大化判别器损失 → shared 不含任务信息
49
54
  task_pred = classifier(z_shared.flip_gradient())
50
- L_adv = -CE(task_pred, t) // Shared 不含任务信息
55
+ L_adv = CE(task_pred, t) // 判别器:最小化 CE 预测任务;编码器:梯度反转后最大化 CE,使 shared 任务无关
51
56
  // Private 应能区分任务:
52
57
  L_private = CE(classifier(z_private), t)
53
58
  L = L_task + λ_adv·L_adv + λ_priv·L_private
@@ -75,9 +80,7 @@
75
80
  - **算子融合**:加法融合 trivial;门控融合的 sigmoid→multiply→add 可融合
76
81
 
77
82
  ## 论文表述方式
78
- "将多任务表示空间 R^d 分解为直和 S P,Shared 子空间通过对抗训练确保任务无关性
79
- (H(T|Z_s)→log T),Private 子空间通过正交正则保证与 Shared 的信息互补,
80
- 理论分析表明负迁移随正交性以 O(‖P_S·P_P‖_F) 衰减。"
83
+ "将多任务表示空间 R^d 分解为共享子空间 S 与任务私有子空间 P:共享分支通过对抗训练降低任务可识别性,私有分支通过正交/去相关正则减少与共享分支的线性重叠。信息互补与负迁移降低需通过任务间迁移矩阵、互信息/PID 代理和消融实验验证,不能仅由正交性自动保证。"
81
84
 
82
85
  ## 风险
83
86
  - 对抗训练的 min-max 优化不稳定,梯度反转的 scale 和 λ_adv 需精心调节
@@ -1,5 +1,5 @@
1
1
  # Subspace Alignment
2
- > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as verified / ⚠️ retrofittable (needs validation) / infeasible. Unmarked claims are theoretically possible but require engineering validation.
2
+ > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
3
3
 
4
4
  ## Applicable Problems
5
5
  Use when two or more representation spaces need to be aligned to a common subspace. Typical scenarios:
@@ -10,9 +10,9 @@ Use when two or more representation spaces need to be aligned to a common subspa
10
10
  Core requirement: **find the optimal linear/nonlinear mapping between two spaces such that corresponding semantics are aligned**.
11
11
 
12
12
  ## Mathematical Inspiration
13
- - Lenses: lenses/geometric.md (Grassmann manifold, principal angles), lenses/variational.md (Procrustes problem)
14
- - Knowledge: knowledge-base/matrix-analysis/projection.md (SVD, orthogonal Procrustes),
15
- knowledge-base/differential-geometry/manifold.md (Grassmann distance, geodesics)
13
+ - Lenses: ../../lenses/geometric.en.md (Grassmann manifold, principal angles), ../../lenses/variational.en.md (Procrustes problem)
14
+ - Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (SVD, orthogonal Procrustes),
15
+ ../../knowledge-base/differential-geometry/manifold.en.md (Grassmann distance, geodesics)
16
16
 
17
17
  ## Required Mathematical Background
18
18
  - **Orthogonal Procrustes Problem**: min_{W in O(d)} ||AW - B||_F^2
@@ -71,17 +71,17 @@ Online Subspace Tracking (adaptation at inference):
71
71
  - **Chunked SVD**: Use randomized SVD approximation at large scale; sufficient precision and faster
72
72
 
73
73
  ## GPU Feasibility
74
- - **Tensorization**: A^T @ B is standard GEMM (d x N) @ (N x d); SVD has cuSOLVER implementations
75
- - **GEMM-mappable**: Procrustes core requires 1 GEMM + 1 SVD; CCA requires 2 GEMMs + 1 SVD
76
- - **Complexity**: GEMM O(N * d^2); SVD O(d^3) (d typically < 1024, acceptable); online tracking O(d^2)
77
- - **Memory & KV-Cache**: Storing covariance matrix d x d (~4 MB for d = 1024); no KV-Cache overhead
78
- - **Low-precision stability**: SVD strongly recommended in fp32; whitening matrix inversion requires fp32 + epsilon regularization
79
- - **Parallelism & Communication**: Subspace angle computations for multiple expert pairs are independent and parallel; CCA GEMMs are highly parallel
80
- - **Sparse structure**: When source/target representations are sparse, covariance matrix Sigma is sparse, enabling sparse SVD
81
- - **Operator fusion**: Whitening (mean-sub -> cov -> inv_sqrt -> transform) can be partially fused
74
+ - **D1[v]**: A^T @ B is standard GEMM (d x N) @ (N x d); SVD has cuSOLVER implementations
75
+ - **D2[v]**: Procrustes core requires 1 GEMM + 1 SVD; CCA requires 2 GEMMs + 1 SVD
76
+ - **D3[v]**: GEMM O(N * d^2); SVD O(d^3) (d typically < 1024, acceptable); online tracking O(d^2)
77
+ - **D4[v]**: Storing covariance matrix d x d (~4 MB for d = 1024); no KV-Cache overhead
78
+ - **D5[~]**: SVD strongly recommended in fp32; whitening matrix inversion requires fp32 + epsilon regularization
79
+ - **D6[v]**: Subspace angle computations for multiple expert pairs are independent and parallel; CCA GEMMs are highly parallel
80
+ - **D7[~]**: When source/target representations are sparse, covariance matrix Sigma is sparse, enabling sparse SVD
81
+ - **D8[~]**: Whitening (mean-sub -> cov -> inv_sqrt -> transform) can be partially fused
82
82
 
83
83
  ## Paper-Worthy Formulation
84
- "Based on orthogonal Procrustes theory, we obtain the optimal isometric mapping W* = UV^T (where USV^T = SVD(A^T B)) between source and target representations. Extending this to Deep CCA for nonlinear subspace alignment, principal angle analysis on the Grassmann manifold shows that post-alignment subspace distance converges at rate O(r/sqrt(N)), while the Barlow Twins redundancy reduction objective ensures feature dimension decorrelation."
84
+ "Based on orthogonal Procrustes theory, we obtain the optimal isometric mapping W* = UV^T (where USV^T = SVD(A^T B)) between source and target representations. Extending this to Deep CCA for nonlinear subspace alignment, principal angles on the Grassmann manifold can measure pre/post alignment distance. Convergence rates depend on sample independence, spectral gaps, and covariance-estimation assumptions; the Barlow Twins objective penalizes cross-correlation but does not by itself guarantee semantic disentanglement, so report principal angles, CCA correlations, and downstream transfer metrics."
85
85
 
86
86
  ## Risks
87
87
  - SVD during backpropagation produces undefined gradients when singular values coincide; epsilon regularization is needed
@@ -1,5 +1,5 @@
1
1
  # Subspace Alignment(子空间对齐)
2
- > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / 不可行」。未标注的视为理论可行,需工程验证。
2
+ > **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行,需工程验证。
3
3
 
4
4
  ## 适用问题
5
5
  当两个或多个表示空间需要对齐到共同的子空间时使用。典型场景:
@@ -10,9 +10,9 @@
10
10
  核心诉求:**找到两个空间之间的最优线性/非线性映射,使对应语义对齐**。
11
11
 
12
12
  ## 数学思想来源
13
- - 透镜:lenses/geometric.md(Grassmann 流形、主角度)、lenses/variational.md(Procrustes 问题)
14
- - 知识:knowledge-base/matrix-analysis/projection.md(SVD、正交 Procrustes)、
15
- knowledge-base/differential-geometry/manifold.md(Grassmann 距离、测地线)
13
+ - 透镜:../../lenses/geometric.md(Grassmann 流形、主角度)、../../lenses/variational.md(Procrustes 问题)
14
+ - 知识:../../knowledge-base/matrix-analysis/projection.md(SVD、正交 Procrustes)、
15
+ ../../knowledge-base/differential-geometry/manifold.md(Grassmann 距离、测地线)
16
16
 
17
17
  ## 需要的数学知识
18
18
  - **正交 Procrustes 问题**:min_{W∈O(d)} ‖AW - B‖_F²
@@ -71,19 +71,19 @@
71
71
  - **分块 SVD**:大规模时用 randomized SVD 近似,精度足够且更快
72
72
 
73
73
  ## GPU 可行性
74
- - **张量化**:A^T@B 为标准 GEMM (d×N)@(N×d);SVD 有 cuSOLVER 实现
75
- - **GEMM 可映射**:Procrustes 核心 1 次 GEMM + 1 次 SVD;CCA 2 次 GEMM + 1 次 SVD
76
- - **复杂度**:GEMM O(N·d²);SVD O(d³)(d 通常 <1024,可接受);在线追踪 O(d²)
77
- - **显存与 KV-Cache**:存储协方差矩阵 d×d(~4MB for d=1024),不增加 KV-Cache
78
- - **低精度稳定**:SVD 强烈建议 fp32;白化的矩阵逆需 fp32 + ε 正则化
79
- - **并行与通信**:多专家对的子空间角度计算独立并行;CCA 的 GEMM 高度并行
80
- - **稀疏结构**:当源/目标表示稀疏时,协方差矩阵 Σ 稀疏,可用稀疏 SVD
81
- - **算子融合**:白化 (mean-sub → cov → inv_sqrt → transform) 可部分融合
74
+ - **D1[v]**:A^T@B 为标准 GEMM (d×N)@(N×d);SVD 有 cuSOLVER 实现
75
+ - **D2[v]**:Procrustes 核心 1 次 GEMM + 1 次 SVD;CCA 2 次 GEMM + 1 次 SVD
76
+ - **D3[v]**:GEMM O(N·d²);SVD O(d³)(d 通常 <1024,可接受);在线追踪 O(d²)
77
+ - **D4[v]**:存储协方差矩阵 d×d(~4MB for d=1024),不增加 KV-Cache
78
+ - **D5[~]**:SVD 强烈建议 fp32;白化的矩阵逆需 fp32 + ε 正则化
79
+ - **D6[v]**:多专家对的子空间角度计算独立并行;CCA 的 GEMM 高度并行
80
+ - **D7[~]**:当源/目标表示稀疏时,协方差矩阵 Σ 稀疏,可用稀疏 SVD
81
+ - **D8[~]**:白化 (mean-sub → cov → inv_sqrt → transform) 可部分融合
82
82
 
83
83
  ## 论文表述方式
84
84
  "基于正交 Procrustes 理论求得源-目标表示间的最优等距映射 W*=UV^T(USV^T=SVD(A^TB)),
85
85
  将其扩展为深度 CCA 实现非线性子空间对齐,Grassmann 流形上的主角度分析表明
86
- 对齐后子空间距离以 O(r/√N) 收敛,同时 Barlow Twins 冗余减少目标保证特征维度解耦。"
86
+ 对齐前后的子空间距离可由主角度度量。收敛速率依赖样本独立性、谱间隙和协方差估计条件;Barlow Twins 目标只惩罚交叉相关,不能单独保证语义解耦,应报告主角度、CCA 相关系数和下游迁移指标。"
87
87
 
88
88
  ## 风险
89
89
  - SVD 在反向传播时奇异值重合导致梯度未定义,需添加 ε 正则化
@@ -1,5 +1,5 @@
1
1
  # Graph Routing
2
- > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as verified / ⚠️ retrofittable (needs validation) / infeasible. Unmarked claims are theoretically possible but require engineering validation.
2
+ > **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
3
3
 
4
4
  ## Applicable Problems
5
5
  Use when there exists a known or learnable topological structure among modules/experts. Typical scenarios:
@@ -10,9 +10,9 @@ Use when there exists a known or learnable topological structure among modules/e
10
10
  Core requirement: **leverage structural priors to constrain routing decisions and reduce the search space**.
11
11
 
12
12
  ## Mathematical Inspiration
13
- - Lenses: lenses/geometric.md (graph Laplacian, spectral graph theory), lenses/probabilistic.md (message passing, information flow)
14
- - Knowledge: knowledge-base/matrix-analysis/projection.md (adjacency matrix, spectral decomposition),
15
- knowledge-base/optimization/lagrangian-duality.md (variational on graphs, diffusion processes)
13
+ - Lenses: ../../lenses/geometric.en.md (graph Laplacian, spectral graph theory), ../../lenses/probabilistic.en.md (message passing, information flow)
14
+ - Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (adjacency matrix, spectral decomposition),
15
+ ../../knowledge-base/optimization/lagrangian-duality.en.md (variational on graphs, diffusion processes)
16
16
 
17
17
  ## Required Mathematical Background
18
18
  - **Graph Laplacian**: L = D - A (combinatorial) or L_sym = D^{-1/2} L D^{-1/2} (normalized)
@@ -60,14 +60,14 @@ Method 3 - Hierarchical Tree Routing (O(log K) complexity):
60
60
  - **Hierarchical tree implementation**: Represented as a complete binary tree array with level-wise vectorization
61
61
 
62
62
  ## GPU Feasibility
63
- - **Tensorization**: GCN layers perform sparse matrix times dense matrix (SpMM), supported by both PyTorch and cuSPARSE
64
- - **GEMM-mappable**: score_init @ P_t in Method 1 is standard GEMM (N x K) @ (K x K)
65
- - **Complexity**: Method 1 O(N * K^2) diffusion + O(N * K * d) gate; Method 3 O(N * d * log K) significantly better than O(N * d * K)
66
- - **Memory & KV-Cache**: A matrix stored sparsely at K x K; hierarchical tree parameters d x log K are negligible
67
- - **Low-precision stability**: Probability matrix P and softmax in fp16 require attention to normalization precision
68
- - **Parallelism & Communication**: GNN message passing can be batched in parallel; hierarchical tree nodes at the same level can be evaluated independently in parallel
69
- - **Sparse structure**: Graph adjacency matrices are naturally sparse (degree << K); SpMM reduces from O(K^2) to O(K * avg_deg)
70
- - **Operator fusion**: L_norm @ H @ W in GCN can be fused into a single sparse GEMM
63
+ - **D1[v]**: GCN layers perform sparse matrix times dense matrix (SpMM), supported by both PyTorch and cuSPARSE
64
+ - **D2[v]**: score_init @ P_t in Method 1 is standard GEMM (N x K) @ (K x K)
65
+ - **D3[v]**: Method 1 O(N * K^2) diffusion + O(N * K * d) gate; Method 3 O(N * d * log K) significantly better than O(N * d * K)
66
+ - **D4[v]**: A matrix stored sparsely at K x K; hierarchical tree parameters d x log K are negligible
67
+ - **D5[v]**: Probability matrix P and softmax in fp16 require attention to normalization precision
68
+ - **D6[v]**: GNN message passing can be batched in parallel; hierarchical tree nodes at the same level can be evaluated independently in parallel
69
+ - **D7[v]**: Graph adjacency matrices are naturally sparse (degree << K); SpMM reduces from O(K^2) to O(K * avg_deg)
70
+ - **D8[v]**: L_norm @ H @ W in GCN can be fused into a single sparse GEMM
71
71
 
72
72
  ## Paper-Worthy Formulation
73
73
  "Leveraging hierarchical tree/graph topology among experts, we compress routing decisions from a flat O(N * K) search to O(N * log K) tree traversal or O(N * K * avg_deg) graph diffusion. Fiedler spectral analysis reveals that the graph's algebraic connectivity lambda_2 directly governs the diversity-coherence trade-off in routing."