math-skill 3.0.1 → 3.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en-US.md +72 -47
- package/README.md +40 -15
- package/agents/math-critic.en.md +6 -6
- package/agents/math-critic.md +6 -6
- package/commands/ask.en.md +6 -1
- package/commands/ask.md +6 -1
- package/design-patterns/attention/equivariant-attention.en.md +8 -2
- package/design-patterns/attention/equivariant-attention.md +7 -1
- package/design-patterns/attention/geometry-aware-attention.en.md +2 -2
- package/design-patterns/attention/geometry-aware-attention.md +1 -1
- package/design-patterns/attention/information-bottleneck-attention.en.md +32 -12
- package/design-patterns/attention/information-bottleneck-attention.md +32 -12
- package/design-patterns/attention/projection-attention.en.md +3 -3
- package/design-patterns/attention/projection-attention.md +2 -2
- package/design-patterns/attention/spectral-attention.en.md +19 -5
- package/design-patterns/attention/spectral-attention.md +18 -4
- package/design-patterns/compression/leverage-score-selection.en.md +13 -13
- package/design-patterns/compression/leverage-score-selection.md +13 -13
- package/design-patterns/compression/low-rank-kv-cache.en.md +39 -12
- package/design-patterns/compression/low-rank-kv-cache.md +38 -12
- package/design-patterns/compression/spectral-token-pruning.en.md +16 -14
- package/design-patterns/compression/spectral-token-pruning.md +16 -14
- package/design-patterns/compression/topology-preserving-compression.en.md +10 -10
- package/design-patterns/compression/topology-preserving-compression.md +10 -10
- package/design-patterns/loss/constraint-penalty.en.md +8 -4
- package/design-patterns/loss/constraint-penalty.md +9 -6
- package/design-patterns/loss/contrastive-loss.en.md +37 -11
- package/design-patterns/loss/contrastive-loss.md +38 -12
- package/design-patterns/loss/information-bottleneck-loss.en.md +5 -5
- package/design-patterns/loss/information-bottleneck-loss.md +6 -6
- package/design-patterns/loss/orthogonality-loss.en.md +24 -17
- package/design-patterns/loss/orthogonality-loss.md +24 -18
- package/design-patterns/loss/variational-loss.en.md +4 -4
- package/design-patterns/loss/variational-loss.md +5 -5
- package/design-patterns/overview.en.md +36 -0
- package/design-patterns/overview.md +36 -0
- package/design-patterns/representation/equivariant-split.en.md +4 -4
- package/design-patterns/representation/equivariant-split.md +4 -5
- package/design-patterns/representation/manifold-representation.en.md +12 -12
- package/design-patterns/representation/manifold-representation.md +13 -14
- package/design-patterns/representation/shared-private-decomposition.en.md +12 -7
- package/design-patterns/representation/shared-private-decomposition.md +12 -9
- package/design-patterns/representation/subspace-alignment.en.md +12 -12
- package/design-patterns/representation/subspace-alignment.md +12 -12
- package/design-patterns/routing/graph-routing.en.md +11 -11
- package/design-patterns/routing/graph-routing.md +11 -11
- package/design-patterns/routing/moe-routing.en.md +12 -12
- package/design-patterns/routing/moe-routing.md +12 -14
- package/design-patterns/routing/optimal-transport-routing.en.md +5 -5
- package/design-patterns/routing/optimal-transport-routing.md +6 -6
- package/design-patterns/routing/spectral-clustering-routing.en.md +26 -15
- package/design-patterns/routing/spectral-clustering-routing.md +26 -17
- package/knowledge-base/differential-geometry/connection.en.md +17 -2
- package/knowledge-base/differential-geometry/connection.md +17 -2
- package/knowledge-base/differential-geometry/curvature.en.md +14 -1
- package/knowledge-base/differential-geometry/curvature.md +14 -1
- package/knowledge-base/differential-geometry/geodesic.en.md +16 -2
- package/knowledge-base/differential-geometry/geodesic.md +16 -2
- package/knowledge-base/differential-geometry/index.en.md +52 -0
- package/knowledge-base/differential-geometry/index.md +52 -0
- package/knowledge-base/differential-geometry/manifold.en.md +15 -2
- package/knowledge-base/differential-geometry/manifold.md +15 -2
- package/knowledge-base/differential-geometry/metric-tensor.en.md +16 -2
- package/knowledge-base/differential-geometry/metric-tensor.md +16 -2
- package/knowledge-base/differential-geometry/tangent-space.en.md +17 -3
- package/knowledge-base/differential-geometry/tangent-space.md +17 -3
- package/knowledge-base/information-geometry/fisher-metric.en.md +24 -5
- package/knowledge-base/information-geometry/fisher-metric.md +24 -5
- package/knowledge-base/information-geometry/index.en.md +45 -0
- package/knowledge-base/information-geometry/index.md +45 -0
- package/knowledge-base/information-geometry/natural-gradient.en.md +17 -3
- package/knowledge-base/information-geometry/natural-gradient.md +17 -3
- package/knowledge-base/lie-theory/equivariance.en.md +18 -3
- package/knowledge-base/lie-theory/equivariance.md +18 -3
- package/knowledge-base/lie-theory/group-action.en.md +16 -2
- package/knowledge-base/lie-theory/group-action.md +16 -2
- package/knowledge-base/lie-theory/index.en.md +52 -0
- package/knowledge-base/lie-theory/index.md +52 -0
- package/knowledge-base/lie-theory/lie-algebra.en.md +17 -2
- package/knowledge-base/lie-theory/lie-algebra.md +17 -2
- package/knowledge-base/lie-theory/lie-group.en.md +17 -3
- package/knowledge-base/lie-theory/lie-group.md +17 -3
- package/knowledge-base/lie-theory/representation.en.md +17 -2
- package/knowledge-base/lie-theory/representation.md +17 -2
- package/knowledge-base/matrix-analysis/index.en.md +49 -0
- package/knowledge-base/matrix-analysis/index.md +49 -0
- package/knowledge-base/matrix-analysis/low-rank-approximation.en.md +20 -7
- package/knowledge-base/matrix-analysis/low-rank-approximation.md +20 -7
- package/knowledge-base/matrix-analysis/matrix-perturbation.en.md +13 -1
- package/knowledge-base/matrix-analysis/matrix-perturbation.md +13 -1
- package/knowledge-base/matrix-analysis/positive-semidefinite.en.md +20 -6
- package/knowledge-base/matrix-analysis/positive-semidefinite.md +20 -6
- package/knowledge-base/matrix-analysis/projection.en.md +14 -1
- package/knowledge-base/matrix-analysis/projection.md +14 -1
- package/knowledge-base/matrix-analysis/spectral-decomposition.en.md +14 -1
- package/knowledge-base/matrix-analysis/spectral-decomposition.md +14 -1
- package/knowledge-base/optimization/constrained-optimization.en.md +17 -3
- package/knowledge-base/optimization/constrained-optimization.md +17 -3
- package/knowledge-base/optimization/convex-optimization.en.md +15 -2
- package/knowledge-base/optimization/convex-optimization.md +15 -2
- package/knowledge-base/optimization/index.en.md +50 -0
- package/knowledge-base/optimization/index.md +50 -0
- package/knowledge-base/optimization/lagrangian-duality.en.md +18 -4
- package/knowledge-base/optimization/lagrangian-duality.md +18 -4
- package/knowledge-base/optimization/proximal-method.en.md +15 -1
- package/knowledge-base/optimization/proximal-method.md +15 -1
- package/knowledge-base/optimization/riemannian-optimization.en.md +18 -4
- package/knowledge-base/optimization/riemannian-optimization.md +18 -4
- package/knowledge-base/overview.en.md +20 -8
- package/knowledge-base/overview.md +20 -8
- package/knowledge-base/probability/concentration-inequality.en.md +16 -2
- package/knowledge-base/probability/concentration-inequality.md +16 -2
- package/knowledge-base/probability/entropy.en.md +19 -3
- package/knowledge-base/probability/entropy.md +18 -2
- package/knowledge-base/probability/fisher-information.en.md +17 -3
- package/knowledge-base/probability/fisher-information.md +17 -3
- package/knowledge-base/probability/index.en.md +51 -0
- package/knowledge-base/probability/index.md +51 -0
- package/knowledge-base/probability/information-bottleneck.en.md +18 -2
- package/knowledge-base/probability/information-bottleneck.md +18 -2
- package/knowledge-base/probability/kl-divergence.en.md +15 -2
- package/knowledge-base/probability/kl-divergence.md +15 -2
- package/knowledge-base/topology/euler-characteristic.en.md +15 -2
- package/knowledge-base/topology/euler-characteristic.md +15 -2
- package/knowledge-base/topology/fundamental-group.en.md +16 -2
- package/knowledge-base/topology/fundamental-group.md +16 -2
- package/knowledge-base/topology/index.en.md +48 -0
- package/knowledge-base/topology/index.md +48 -0
- package/knowledge-base/topology/persistent-homology.en.md +17 -2
- package/knowledge-base/topology/persistent-homology.md +17 -2
- package/lenses/spectral.en.md +1 -1
- package/lenses/spectral.md +1 -1
- package/lenses/topological.en.md +1 -1
- package/lenses/topological.md +1 -1
- package/package.json +6 -6
- package/references/agentic-workflow.en.md +2 -2
- package/references/books/abstract-algebra.en.md +12 -12
- package/references/books/abstract-algebra.md +9 -9
- package/references/books/algebraic-geometry-rising-sea.en.md +2 -2
- package/references/books/differential-geometry.en.md +16 -16
- package/references/books/differential-geometry.md +14 -14
- package/references/books/matrix-analysis.en.md +3 -3
- package/references/books/matrix-analysis.md +24 -24
- package/references/books/micro-lie-theory.en.md +1 -1
- package/references/books/optimization-ml.en.md +15 -15
- package/references/books/optimization-ml.md +14 -14
- package/references/books/smooth-manifolds.en.md +3 -3
- package/references/books/smooth-manifolds.md +6 -6
- package/references/gpu-friendly-math.en.md +21 -8
- package/references/gpu-friendly-math.md +19 -4
- package/references/skill-index.en.md +106 -0
- package/references/skill-index.md +106 -0
- package/skills/math-research-activator/SKILL.en.md +76 -118
- package/skills/math-research-activator/SKILL.md +69 -109
|
@@ -69,39 +69,39 @@
|
|
|
69
69
|
|
|
70
70
|
> 每条标注**八维落点**(对应 `../gpu-friendly-math.md` 维度编号),方便直接进 GPU 验收门。
|
|
71
71
|
|
|
72
|
-
1. **随机化数值线代(randomized NLA)**:用随机投影 + QR(§2.1)做随机 SVD,把 O(n³) 完整分解降到亚二次,对超大权重/激活做低秩草图(sketching
|
|
73
|
-
2. **低秩注意力 / KV 压缩**:以 Eckart–Young(§7.4)保证截断 SVD 是最优低秩逼近;用**核范数(谱范数对偶,§5.5)**作低秩正则,把 KV-Cache
|
|
74
|
-
3. **谱归一化(spectral normalization)**:power iteration 估最大奇异值(算子范数 §5.6),约束每层 Lipschitz——GAN
|
|
75
|
-
4. **Newton–Schulz 正交化(Muon 式)**:极分解(§7.3)把梯度矩阵投到最近正交矩阵,迭代只含矩阵乘——当前最 GPU 友好的"二阶味"
|
|
76
|
-
5. **预条件 / Shampoo / K-FAC**:条件数(§5.8)诊断病态,用 PSD Kronecker 因子(Ch 7)近似 Hessian
|
|
77
|
-
6. **Geršgorin 廉价谱半径门(§6.1)**:训练循环里用圆盘界 O(n²) 快速估谱半径,做低成本稳定性 gate,不必跑完整 EVD
|
|
78
|
-
7. **PSD 核工程(Schur 积定理 §7.5)**:用 Hadamard 积组合多个 PSD
|
|
79
|
-
8. **Perron–Frobenius 诊断(§8.2–8.5)**:把行随机 attention 当 Markov 算子,用谱 gap 量化 over-smoothing / rank collapse
|
|
80
|
-
9. **块化 / communication-avoiding 分解**:把 QR、Cholesky(§3.5)写成分块版本,用 GEMM
|
|
72
|
+
1. **随机化数值线代(randomized NLA)**:用随机投影 + QR(§2.1)做随机 SVD,把 O(n³) 完整分解降到亚二次,对超大权重/激活做低秩草图(sketching)。*落点:D2/D3——全 GEMM、复杂度可控。*
|
|
73
|
+
2. **低秩注意力 / KV 压缩**:以 Eckart–Young(§7.4)保证截断 SVD 是最优低秩逼近;用**核范数(谱范数对偶,§5.5)**作低秩正则,把 KV-Cache 投影到低维子空间。*落点:D2/D4——GEMM 链 + 显存压缩。*
|
|
74
|
+
3. **谱归一化(spectral normalization)**:power iteration 估最大奇异值(算子范数 §5.6),约束每层 Lipschitz——GAN/扩散/稳定训练。*落点:D1/D6——matvec,但迭代串行需块化。*
|
|
75
|
+
4. **Newton–Schulz 正交化(Muon 式)**:极分解(§7.3)把梯度矩阵投到最近正交矩阵,迭代只含矩阵乘——当前最 GPU 友好的"二阶味"更新。*落点:D2/D6/D8——纯 GEMM、可融合、bf16 稳。*
|
|
76
|
+
5. **预条件 / Shampoo / K-FAC**:条件数(§5.8)诊断病态,用 PSD Kronecker 因子(Ch 7)近似 Hessian 做预条件,把病态损失面拉圆。*落点:D2/D5——小矩阵 GEMM,注意逆运算的精度。*
|
|
77
|
+
6. **Geršgorin 廉价谱半径门(§6.1)**:训练循环里用圆盘界 O(n²) 快速估谱半径,做低成本稳定性 gate,不必跑完整 EVD。*落点:D1/D3——逐行求和,极廉价。*
|
|
78
|
+
7. **PSD 核工程(Schur 积定理 §7.5)**:用 Hadamard 积组合多个 PSD 核,保证可学习相似度矩阵始终半正定。*落点:D1——逐元素张量积,天然友好。*
|
|
79
|
+
8. **Perron–Frobenius 诊断(§8.2–8.5)**:把行随机 attention 当 Markov 算子,用谱 gap 量化 over-smoothing / rank collapse,指导残差与温度设计。*落点:D1/D3——谱估计廉价,避免深层坍缩。*
|
|
80
|
+
9. **块化 / communication-avoiding 分解**:把 QR、Cholesky(§3.5)写成分块版本,用 GEMM 替代逐列消元,跨设备减少通信轮次。*落点:D2/D6——串行递推改造成并行 + overlap。*
|
|
81
81
|
|
|
82
82
|
## GPU 友好性警告
|
|
83
83
|
|
|
84
84
|
> 评分维度引用 `../gpu-friendly-math.md` 的**八维检查**(张量化 / GEMM 可映射 / 复杂度 / 显存 / 低精度 / 并行 / 稀疏 / 算子融合),此处不重复定义。
|
|
85
85
|
|
|
86
86
|
**天然友好(math beautiful × GPU friendly):**
|
|
87
|
-
- **SVD 截断 / 低秩**:写成 GEMM
|
|
88
|
-
- **Frobenius / 谱范数、Gram 矩阵、Hadamard
|
|
89
|
-
- **极分解 Newton–Schulz
|
|
90
|
-
- **Geršgorin 圆盘**:O(n²)
|
|
91
|
-
- **良态 PSD 的 blocked Cholesky / Gram 构造**:分块后是 GEMM
|
|
87
|
+
- **SVD 截断 / 低秩**:写成 GEMM 链(D2),压缩 KV-Cache/权重(D4)。
|
|
88
|
+
- **Frobenius / 谱范数、Gram 矩阵、Hadamard 积**:批量张量代数(D1/D2)。
|
|
89
|
+
- **极分解 Newton–Schulz**:纯矩阵乘迭代(D2/D6/D8 可融合),bf16 下稳健。
|
|
90
|
+
- **Geršgorin 圆盘**:O(n²) 逐行求和(D1),廉价稳定性估计。
|
|
91
|
+
- **良态 PSD 的 blocked Cholesky / Gram 构造**:分块后是 GEMM 链(D2),核方法与协方差预条件常用。
|
|
92
92
|
|
|
93
93
|
**美但不可算(beautiful, not computable):**
|
|
94
|
-
- **Jordan 标准型(§3.1
|
|
95
|
-
- **完整 EVD / SVD 的 O(n³)
|
|
96
|
-
- **非正规矩阵(non-normal,§2.5 之外)**——特征值不能反映真实行为,需 pseudospectra
|
|
97
|
-
- **病态 / 高条件数(§5.8)**——会出现灾难性抵消,要 fp64 才对,与 bf16/fp8
|
|
98
|
-
- **QR / Cholesky 的串行依赖(§2.1, §3.5
|
|
99
|
-
- **power iteration 的串行迭代**——单向量迭代并行度低;需块化(block / subspace iteration)才吃满 SM
|
|
94
|
+
- **Jordan 标准型(§3.1)**——经典反例:特征值重数对扰动极度敏感,浮点下根本无法可靠计算,**永远不要当数值工具**(违反 D5)。Weyr 型同理。
|
|
95
|
+
- **完整 EVD / SVD 的 O(n³)**——大矩阵直接爆(违反 D3),必须换随机化/迭代法。
|
|
96
|
+
- **非正规矩阵(non-normal,§2.5 之外)**——特征值不能反映真实行为,需 pseudospectra;低精度下谱失真(D5)。
|
|
97
|
+
- **病态 / 高条件数(§5.8)**——会出现灾难性抵消,要 fp64 才对,与 bf16/fp8 训练冲突(D5)。
|
|
98
|
+
- **QR / Cholesky 的串行依赖(§2.1, §3.5)**——朴素实现是长串行递推(违反 D6),需 blocked / communication-avoiding 变体。
|
|
99
|
+
- **power iteration 的串行迭代**——单向量迭代并行度低;需块化(block / subspace iteration)才吃满 SM(D6)。
|
|
100
100
|
|
|
101
101
|
**改造手法(呼应 `../gpu-friendly-math.md` 的 Make-It-Computable Toolkit):**
|
|
102
|
-
- 完整 EVD/SVD → **随机化 +
|
|
103
|
-
- 精确分解 → **块化 / GEMM
|
|
104
|
-
- 病态/非正规 → **重参数化 +
|
|
102
|
+
- 完整 EVD/SVD → **随机化 + 截断**降复杂度(D3);
|
|
103
|
+
- 精确分解 → **块化 / GEMM 化**消除串行(D2/D6);
|
|
104
|
+
- 病态/非正规 → **重参数化 + 谱归一化**稳低精度(D5);
|
|
105
105
|
- Jordan 等不可算标准型 → 只留作**理论证明**,数值上换 Schur/SVD。
|
|
106
106
|
|
|
107
107
|
## 该调用哪个思想透镜
|
|
@@ -125,7 +125,7 @@
|
|
|
125
125
|
- **数值上假设精确正定**:浮点下 Gram/协方差可能丢正定性,要加 jitter(对角抖动)或用 Cholesky 带 pivot。
|
|
126
126
|
- **用 Frobenius 范数当低秩正则**:Frobenius / 权重衰减压的是"能量"不是"秩";要低秩得用核范数或显式低秩参数化(如 LoRA)。
|
|
127
127
|
- **对非对称矩阵谈"特征值大小"**:度量能量/范数/稳定裕度该看**奇异值**;非正规矩阵的特征值模长会严重误导(瞬态增长远超谱半径预测)。
|
|
128
|
-
- **把 O(n²) Gram 矩阵全量物化**:attention/核矩阵不分块就直接撑爆显存;应走 FlashAttention 式融合 + 分块(呼应 GPU
|
|
128
|
+
- **把 O(n²) Gram 矩阵全量物化**:attention/核矩阵不分块就直接撑爆显存;应走 FlashAttention 式融合 + 分块(呼应 GPU 八维 D4/D8)。
|
|
129
129
|
- **堆定理而不诊断瓶颈**:先问"算法瓶颈是谱、低秩还是稳定性",再选结构,别一上来灌矩阵理论。
|
|
130
130
|
|
|
131
131
|
## 深挖入口
|
|
@@ -62,7 +62,7 @@ Two additional directly usable "components":
|
|
|
62
62
|
|
|
63
63
|
## GPU Friendliness Warning
|
|
64
64
|
|
|
65
|
-
> The sole authority for the acceptance gate: [`../gpu-friendly-math.md`](../gpu-friendly-math.md) (eight dimensions: D1 Tensorization, D2 GEMM-mappability, D3 Complexity, D4 Memory/KV, D5 Low-precision stability, D6 Parallelism & communication, D7 Sparse structure, D8 Operator fusion). Dimension-by-dimension scoring follows.
|
|
65
|
+
> The sole authority for the acceptance gate: [`../gpu-friendly-math.en.md`](../gpu-friendly-math.en.md) (eight dimensions: D1 Tensorization, D2 GEMM-mappability, D3 Complexity, D4 Memory/KV, D5 Low-precision stability, D6 Parallelism & communication, D7 Sparse structure, D8 Operator fusion). Dimension-by-dimension scoring follows.
|
|
66
66
|
|
|
67
67
|
| # | Dimension | Rating | Notes |
|
|
68
68
|
|---|---|---|---|
|
|
@@ -102,32 +102,32 @@ Actual chapter map:
|
|
|
102
102
|
|
|
103
103
|
## GPU Friendliness Warning
|
|
104
104
|
|
|
105
|
-
> The following item-by-item evaluation uses the **eight-dimension scorecard** from `../gpu-friendly-math.md` (read that first).
|
|
105
|
+
> The following item-by-item evaluation uses the **eight-dimension scorecard** from `../gpu-friendly-math.en.md` (read that first).
|
|
106
106
|
> Conclusion: The book's first-order / stochastic / low-rank content is naturally GPU-friendly; second-order methods and search-based methods require heavy adaptation or elimination.
|
|
107
107
|
|
|
108
108
|
- **Full Hessian in second-order methods (Ch 9, 11)**
|
|
109
|
-
- Violates **
|
|
110
|
-
- Violates **
|
|
111
|
-
- Violates **
|
|
112
|
-
- Adaptation: Matrix-free HVP + CG, L-BFGS low-rank, K-FAC block-diagonal -> restores **
|
|
109
|
+
- Violates **D4**: Dense Hessian is O(d^2), impossible to materialize for hundreds of millions of parameters.
|
|
110
|
+
- Violates **D2**: Inversion / factorization is not batched-GEMM friendly.
|
|
111
|
+
- Violates **D5**: Ill-conditioned inversion causes catastrophic cancellation under bf16/fp16.
|
|
112
|
+
- Adaptation: Matrix-free HVP + CG, L-BFGS low-rank, K-FAC block-diagonal -> restores **D2/D4** friendliness.
|
|
113
113
|
- **Line search (Ch 7 golden section / Armijo S24.4.4)**
|
|
114
|
-
- Violates **
|
|
115
|
-
- Violates **
|
|
114
|
+
- Violates **D1**: Essentially a scalar loop + data-dependent branching (step-by-step comparison of function values).
|
|
115
|
+
- Violates **D8**: Frequent small kernels, control-flow divergence.
|
|
116
116
|
- Adaptation: Large models almost universally use scheduled step sizes (warmup + cosine) instead of step-by-step line search.
|
|
117
117
|
- **First-order / SGD (Ch 8, 27)**
|
|
118
|
-
- **
|
|
119
|
-
- **
|
|
118
|
+
- **D1/D2 friendly**: Gradients = tensor algebra, expressible as GEMM.
|
|
119
|
+
- **D6** is good: Data-parallel all-reduce can overlap with backprop -- the fundamental reason this is the default approach for large models.
|
|
120
120
|
- **Global search (Ch 14: GA/SA/PSO)**
|
|
121
|
-
- Violates **
|
|
122
|
-
- **
|
|
121
|
+
- Violates **D1**: Non-differentiable, discrete stochastic search, blocks end-to-end gradient training.
|
|
122
|
+
- **D6** is also poor: Inter-population dependencies, hard to parallelize. Only suitable for black-box hyperparameter search, not for the training inner loop.
|
|
123
123
|
- **LP simplex / interior-point methods (Ch 16, 18)**
|
|
124
|
-
- Simplex row operations are acceptable; interior-point methods require solving a linear system at each step, **
|
|
124
|
+
- Simplex row operations are acceptable; interior-point methods require solving a linear system at each step, **D2/D6** limited by factorization and communication.
|
|
125
125
|
- Suitable for medium-scale constrained subproblems, not for placement inside every training step.
|
|
126
126
|
- **Projections (S24.2)**
|
|
127
|
-
- Norm-ball / box projections are cheap elementwise operations (**
|
|
128
|
-
- General polyhedral projections require solving a QP (**
|
|
127
|
+
- Norm-ball / box projections are cheap elementwise operations (**D1/D8** friendly).
|
|
128
|
+
- General polyhedral projections require solving a QP (**D2** degrades).
|
|
129
129
|
- **Convergence depends on condition number kappa**
|
|
130
|
-
- Large kappa -> **
|
|
130
|
+
- Large kappa -> **D5** is further amplified at low precision (ill-conditioned gradients).
|
|
131
131
|
- Must pair with normalization / preconditioning; otherwise bf16 training diverges.
|
|
132
132
|
|
|
133
133
|
## Which Thinking Lens to Invoke
|
|
@@ -106,28 +106,28 @@
|
|
|
106
106
|
> 结论:本书的一阶 / 随机 / 低秩部分天然友好;二阶法与搜索类方法需重度改造或淘汰。
|
|
107
107
|
|
|
108
108
|
- **二阶法的全 Hessian(Ch 9, 11)**
|
|
109
|
-
-
|
|
110
|
-
-
|
|
111
|
-
-
|
|
112
|
-
- 改造:matrix-free HVP + CG、L-BFGS 低秩、K-FAC 块对角 →
|
|
109
|
+
- 违反**D4**:稠密 Hessian 是 O(d²),亿级参数无法物化。
|
|
110
|
+
- 违反**D2**:求逆 / 分解非 batched-GEMM 友好。
|
|
111
|
+
- 违反**D5**:病态求逆在 bf16/fp16 灾难性抵消。
|
|
112
|
+
- 改造:matrix-free HVP + CG、L-BFGS 低秩、K-FAC 块对角 → 重回**D2/D4** 友好。
|
|
113
113
|
- **线搜索(Ch 7 黄金分割 / Armijo §24.4.4)**
|
|
114
|
-
-
|
|
115
|
-
-
|
|
114
|
+
- 违反**D1**:本质是标量循环 + 数据相关分支(逐步比较函数值)。
|
|
115
|
+
- 违反**D8**:频繁小 kernel、控制流发散。
|
|
116
116
|
- 改造:大模型几乎都用调度式步长(warmup + cosine)替代逐步线搜索。
|
|
117
117
|
- **一阶 / SGD(Ch 8, 27)**
|
|
118
|
-
-
|
|
119
|
-
-
|
|
118
|
+
- **D1/D2 友好**:梯度 = 张量代数、可写成 GEMM。
|
|
119
|
+
- **D6**良好:数据并行 all-reduce 可与反向 overlap——大模型默认路线的根本原因。
|
|
120
120
|
- **全局搜索(Ch 14:GA/SA/PSO)**
|
|
121
|
-
-
|
|
122
|
-
-
|
|
121
|
+
- 违反**D1**:不可微、离散随机搜索,阻断端到端梯度训练。
|
|
122
|
+
- **D6** 也差:种群间依赖、难并行映射。仅适合黑盒超参搜索,不进训练内环。
|
|
123
123
|
- **LP 单纯形 / 内点法(Ch 16, 18)**
|
|
124
|
-
-
|
|
124
|
+
- 单纯形的行运算尚可;内点法每步要解线性系统,**D2/D6** 受限于分解与通信。
|
|
125
125
|
- 适合中等规模约束子问题,不适合放进每个训练 step。
|
|
126
126
|
- **投影(§24.2)**
|
|
127
|
-
- norm-ball / box 投影是廉价 elementwise
|
|
128
|
-
- 一般多面体投影要解 QP
|
|
127
|
+
- norm-ball / box 投影是廉价 elementwise(**D1/D8** 友好)。
|
|
128
|
+
- 一般多面体投影要解 QP(**D2** 退化)。
|
|
129
129
|
- **收敛依赖条件数 κ**
|
|
130
|
-
- κ 大 →
|
|
130
|
+
- κ 大 → **D5** 在低精度下被进一步放大(病态梯度)。
|
|
131
131
|
- 必须配归一化 / 预条件,否则 bf16 训练发散。
|
|
132
132
|
|
|
133
133
|
## 该调用哪个思想透镜
|
|
@@ -25,7 +25,7 @@ Actual chapter map (2nd ed., chapter numbers match the book):
|
|
|
25
25
|
|
|
26
26
|
## Core Structures Transferable to AI/Infra
|
|
27
27
|
|
|
28
|
-
- **Tangent space = local linearization of parameter/latent space**. `df_p: T_pM -> T_{f(p)}N` is the Jacobian / pushforward; backpropagation =
|
|
28
|
+
- **Tangent space = local linearization of parameter/latent space**. `df_p: T_pM -> T_{f(p)}N` is the Jacobian / pushforward (corresponding to JVP / forward-mode AD); backpropagation = pullback on the cotangent bundle (VJP = vector-Jacobian product = pullback of covectors), i.e., pullback along composed maps (the geometric version of the chain rule). All first-order methods live in the tangent space.
|
|
29
29
|
- **Gradients are covectors, not vectors**. Autodiff produces 1-forms (elements of the cotangent space); to obtain a descent direction (tangent vector) one must use the **metric to raise indices** (sharp). Euclidean metric -> ordinary gradient; Fisher metric -> natural gradient. **This is the manifold-level root cause of natural gradients / mirror descent**.
|
|
30
30
|
- **Constraint sets = submanifolds**. Regular level set theorem: when g is a submersion, the solution set `g(x)=c` is a smooth submanifold; constrained optimization = unconstrained optimization on a submanifold.
|
|
31
31
|
- **Lie groups = differentiable symmetry groups**. SO(n)/U(n)/Stiefel/Grassmannian are all manifolds; their Lie algebras (e.g., skew-symmetric matrices so(n)) are linear spaces, mapped back to the group via `exp` -> **reparameterize "constrained weights" as "unconstrained Lie algebra + exp"**.
|
|
@@ -53,11 +53,11 @@ Actual chapter map (2nd ed., chapter numbers match the book):
|
|
|
53
53
|
|
|
54
54
|
## GPU Friendliness Warning
|
|
55
55
|
|
|
56
|
-
> The sole authority for the acceptance gate: the **eight dimensions** in `../gpu-friendly-math.md`. The success or failure of manifold methods almost entirely hinges on one point: **whether the retraction / exponential map can be tensorized and expressed as GEMM, or must be solved iteratively.**
|
|
56
|
+
> The sole authority for the acceptance gate: the **eight dimensions** in `../gpu-friendly-math.en.md`. The success or failure of manifold methods almost entirely hinges on one point: **whether the retraction / exponential map can be tensorized and expressed as GEMM, or must be solved iteratively.**
|
|
57
57
|
|
|
58
58
|
Dimension-by-dimension comparison:
|
|
59
59
|
|
|
60
|
-
- **D1-D2 Tensorization / GEMM**: tangent-space operations (pushforward/pullback, Jacobian-vector products, projecting gradients to the tangent space) are **naturally batched GEMM** [v] -- backpropagation itself is
|
|
60
|
+
- **D1-D2 Tensorization / GEMM**: tangent-space operations (pushforward/pullback, Jacobian-vector products, projecting gradients to the tangent space) are **naturally batched GEMM** [v] -- backpropagation itself is pullback (VJP), which is extremely GPU-friendly. **However**, retractions/exp typically require QR, eigendecomposition, matrix exponentials, or small-matrix inverses: QR/eig are **not clean GEMM**, they are decompositions with serial dependencies (cuSOLVER batched small matrices are acceptable, but large matrices are O(n^3) with poor parallelism) -> **reformable** rather than natively friendly.
|
|
61
61
|
- **D3 Complexity**: geodesic distances, parallel transport, general `log|det J|` are all O(n^3) or worse. **Fix**: restrict to manifolds with closed-form geodesics (spheres/hyperbolic/SO(3)); use triangular/coupling layers in normalizing flows so that log-det reduces to a diagonal sum (O(n)).
|
|
62
62
|
- **D5 Low-precision**: [~] **biggest pitfall**. Matrix `exp / log / sqrt`, eigendecompositions, and the affine-invariant metric for SPD are **catastrophically unstable** under bf16/fp16, often silently requiring fp32/fp64. Manifold primitives frequently "appear to run but have numerically diverged long ago."
|
|
63
63
|
- **D6 Parallelism & communication**: the squaring chain in scaling-and-squaring, ODE integration steps, Householder/QR all have **serial recurrences**, making cross-SM/device overlap difficult. Positive counter-example: explicit symplectic integrators (leapfrog) are highly parallel [v].
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
|
|
26
26
|
## 可迁移到 AI/Infra 的核心结构
|
|
27
27
|
|
|
28
|
-
- **切空间 = 参数/隐空间的局部线性化(local linearization)**。`df_p: T_pM → T_{f(p)}N` 就是 Jacobian / pushforward
|
|
28
|
+
- **切空间 = 参数/隐空间的局部线性化(local linearization)**。`df_p: T_pM → T_{f(p)}N` 就是 Jacobian / pushforward(前推,对应 JVP / 前向模式 AD);反向传播 = 余切丛上的拉回(pullback on cotangent bundle,VJP = 向量-Jacobian 积 = 余向量的拉回),即沿复合映射做 pullback(链式法则的几何版)。一切一阶方法都活在切空间里。
|
|
29
29
|
- **梯度是余向量(covector),不是向量**。autodiff 给出的是 1-form(余切空间元素);要变成可下降的方向(切向量)必须用 **度量升指标**((sharp))。欧氏度量 → 普通梯度;Fisher 度量 → 自然梯度(natural gradient)。**这是自然梯度 / 镜像下降的流形根因**。
|
|
30
30
|
- **约束集 = 子流形(submanifold)**。正则水平集定理:当 g 是 submersion 时 `g(x)=c` 的解集是光滑子流形;约束优化 = 在子流形上做无约束优化。
|
|
31
31
|
- **李群 = 可微的对称群**。SO(n)/U(n)/Stiefel/Grassmann 都是流形;其李代数(如反对称矩阵 so(n))是线性空间,用 `exp` 映射回群 → **把"约束权重"重参数化为"无约束李代数 + exp"**。
|
|
@@ -57,11 +57,11 @@
|
|
|
57
57
|
|
|
58
58
|
逐维对照:
|
|
59
59
|
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
-
|
|
63
|
-
-
|
|
64
|
-
-
|
|
60
|
+
- **D1–D2** 切空间运算(pushforward/pullback、Jacobian-向量积、把梯度投影到切空间)**天然是 batched GEMM** [v]——反向传播本就是 pullback(VJP),这部分对 GPU 极友好。**但** retraction/exp 多半要 QR、特征分解、矩阵指数或小矩阵求逆:QR/eig **不是干净的 GEMM**,是带串行依赖的分解(cuSOLVER 批量小矩阵尚可,大矩阵 O(n³) 且并行差)→ **可改造** 而非天然友好。
|
|
61
|
+
- **D3 复杂度**:测地线距离、平行移动、一般 `log|det J|` 都是 O(n³) 起。**改造**:限定有闭式测地线的流形(球面/双曲/SO(3));归一化流用三角/耦合层让 log-det 退化成对角和(O(n))。
|
|
62
|
+
- **D5 低精度**:[~] **最大坑**。矩阵 `exp / log / sqrt`、特征分解、SPD 的仿射不变度量在 bf16/fp16 下 **灾难性不稳定**,常静默地需要 fp32/fp64。流形原语经常"表面能跑、数值早已发散"。
|
|
63
|
+
- **D6 并行与通信**:scaling-and-squaring 的平方链、ODE 积分步、Householder/QR 都有 **串行递推**,难跨 SM/设备 overlap。反例向好:显式辛积分器(leapfrog)高并行 [v]。
|
|
64
|
+
- **D4/D7/D8 显存 / 稀疏 / 融合**:李代数/旋转参数化若限制在 **小矩阵或块对角**(如逐头旋转、SO(3) 的 Rodrigues 闭式),可融进 kernel、走 Tensor Core;大稠密流形算子则要物化大中间张量。
|
|
65
65
|
|
|
66
66
|
**结论与改造手法(呼应 gpu-friendly-math.md 工具箱)**:
|
|
67
67
|
|
|
@@ -1,9 +1,7 @@
|
|
|
1
1
|
# GPU-Friendly Math Checklist
|
|
2
2
|
|
|
3
3
|
> This file is the **single source of truth** for the "GPU-feasibility" acceptance gate.
|
|
4
|
-
> The activator, the 15 thinking lenses, `books/*.md`, and `../agents/math-critic.md` all reference this file; no duplicate definitions elsewhere.
|
|
5
|
-
>
|
|
6
|
-
> This file is the single source of truth for the "GPU-feasibility" acceptance gate. The activator, the 15 thinking lenses, the book references, and the math-critic all point here.
|
|
4
|
+
> The activator, the 15 thinking lenses, `books/*.md`, and `../agents/math-critic.en.md` all reference this file; no duplicate definitions elsewhere.
|
|
7
5
|
|
|
8
6
|
## Eight-Dimension Abbreviations
|
|
9
7
|
|
|
@@ -18,6 +16,21 @@
|
|
|
18
16
|
| D7 | Sparse structure |
|
|
19
17
|
| D8 | Operator fusion |
|
|
20
18
|
|
|
19
|
+
## Quantitative Checklist
|
|
20
|
+
|
|
21
|
+
When evaluating each dimension, provide concrete numbers rather than just labels:
|
|
22
|
+
|
|
23
|
+
| Dimension | Quantitative Questions to Answer |
|
|
24
|
+
|-----------|--------------------------------|
|
|
25
|
+
| D1 Tensorization | What are the tensor shapes of core operations? Where is the batch dimension? |
|
|
26
|
+
| D2 GEMM-mappability | How many GEMM/matmul operations? What are the (M,N,K) dimensions of each? |
|
|
27
|
+
| D3 Complexity | Total FLOPs? Ratio vs. baseline (standard attention/MLP)? |
|
|
28
|
+
| D4 Memory | Peak memory (bytes)? Is an n×n matrix materialized? KV-Cache overhead? |
|
|
29
|
+
| D5 Low-precision | Numerical error magnitude under bf16/fp8? Mixed-precision strategy needed? |
|
|
30
|
+
| D6 Parallelism | Theoretical parallelism degree? Communication volume (bytes/step)? All-reduce required? |
|
|
31
|
+
| D7 Sparsity | Sparsity ratio (%)? Sparse format (CSR/BSR/block-sparse)? Dedicated kernel available? |
|
|
32
|
+
| D8 Operator Fusion | Number of fusible kernels? Reduction in kernel launches and memory transfers after fusion? |
|
|
33
|
+
|
|
21
34
|
## Core Proposition
|
|
22
35
|
|
|
23
36
|
**Mathematical beauty ≠ computability.** For a structure to truly enter training and inference on modern GPU clusters, it must simultaneously satisfy two requirements:
|
|
@@ -66,13 +79,13 @@ Common techniques for transforming "beautiful but non-computable" into "both bea
|
|
|
66
79
|
|
|
67
80
|
## Worked Example: Tropical Sheaf Attention
|
|
68
81
|
|
|
69
|
-
Drawn from the auto-research directions cited in `agentic-workflow.md`, demonstrating how a **candidate design enters the 8-dimension verification**:
|
|
82
|
+
Drawn from the auto-research directions cited in `agentic-workflow.en.md`, demonstrating how a **candidate design enters the 8-dimension verification**:
|
|
70
83
|
|
|
71
84
|
| Component | Mathematical Source | GPU Friendliness |
|
|
72
85
|
|-----------|-------------------|-----------------|
|
|
73
|
-
| Tropical Gating | Tropical semiring, piecewise-linear | Replaces Top-K: element-wise max-plus —
|
|
74
|
-
| Cellular Sheaf Diffusion | Algebraic geometry / topology (sheaves, restriction maps) | Each edge is a low-rank linear transform = small GEMM (
|
|
75
|
-
| Čech Cohomology Regularization | Algebraic topology (first cohomology $H^1$) | Local, inexpensive; serves as an algebraic criterion for hallucination (
|
|
76
|
-
| Low-Rank Basis KV Compression (Plücker/Grassmannian perspective) | Projective geometry | Store the basis rather than Plücker coordinates (the latter expands when low-rank); block-summary candidate — compression ratio / error / throughput must be benchmarked (
|
|
86
|
+
| Tropical Gating | Tropical semiring, piecewise-linear | Replaces Top-K: element-wise max-plus — D1[v] / D2[x] Not a Tensor Core GEMM (runs on CUDA cores) / D3[v] Per-token gating only, sub-quadratic (min-plus matmul is APSP-hard, not sub-quadratic); sub-differentiable, kinks require LogSumExp smoothing (smoothing recovers standard softmax) |
|
|
87
|
+
| Cellular Sheaf Diffusion | Algebraic geometry / topology (sheaves, restriction maps) | Each edge is a low-rank linear transform = small GEMM (D2/D4) |
|
|
88
|
+
| Čech Cohomology Regularization | Algebraic topology (first cohomology $H^1$) | Local, inexpensive; serves as an algebraic criterion for hallucination (D3/D8) |
|
|
89
|
+
| Low-Rank Basis KV Compression (Plücker/Grassmannian perspective) | Projective geometry | Store the basis rather than Plücker coordinates (the latter expands when low-rank); block-summary candidate — compression ratio / error / throughput must be benchmarked (D4) |
|
|
77
90
|
|
|
78
91
|
Do not treat the table above as validated conclusions. The correct approach is to enter each component into the test plan: prove or estimate complexity, measure peak memory and throughput, check bf16/fp8 stability, and confirm whether it can be mapped to GEMM / batched GEMM / fused kernels. Only after both empirical benchmarks and theoretical derivations pass should a component be labeled "math beautiful × GPU friendly."
|
|
@@ -20,6 +20,21 @@
|
|
|
20
20
|
| D7 | 稀疏结构 |
|
|
21
21
|
| D8 | 算子融合 |
|
|
22
22
|
|
|
23
|
+
## 量化检查清单
|
|
24
|
+
|
|
25
|
+
评估每个维度时,应填写具体数值而非仅给标签:
|
|
26
|
+
|
|
27
|
+
| 维度 | 需回答的量化问题 |
|
|
28
|
+
|------|----------------|
|
|
29
|
+
| D1 张量化 | 核心操作的 tensor shape 是什么?batch 维在哪? |
|
|
30
|
+
| D2 GEMM 可映射 | 操作可分解为几次 GEMM/matmul?每次的 (M,N,K) 是多少? |
|
|
31
|
+
| D3 复杂度 | FLOPs 总量?与 baseline (标准 attention/MLP) 的比值? |
|
|
32
|
+
| D4 显存 | 峰值显存 (bytes)?是否物化 n×n 矩阵?KV-Cache 额外开销? |
|
|
33
|
+
| D5 低精度 | bf16/fp8 下的数值误差量级?是否需要混合精度策略? |
|
|
34
|
+
| D6 并行 | 理论并行度?通信量 (bytes/step)?是否需要 all-reduce? |
|
|
35
|
+
| D7 稀疏 | 稀疏度 (%)?稀疏格式 (CSR/BSR/block-sparse)?是否有专用 kernel? |
|
|
36
|
+
| D8 算子融合 | 可融合的 kernel 数?融合后减少的 kernel launch 次数和显存搬运量? |
|
|
37
|
+
|
|
23
38
|
## 核心命题 / Core Proposition
|
|
24
39
|
|
|
25
40
|
**数学美 ≠ 可算。** 一个结构要真正进入现代 GPU 集群的训练与推理,必须同时满足两件事:
|
|
@@ -72,9 +87,9 @@
|
|
|
72
87
|
|
|
73
88
|
| 组件 | 数学来源 | GPU 友好性 |
|
|
74
89
|
|------|---------|-----------|
|
|
75
|
-
| 热带门控 Tropical Gating | 热带半环分段线性 | 替代 Top-K:逐元素 max-plus——
|
|
76
|
-
| 胞腔层扩散 Cellular Sheaf Diffusion | 代数几何/拓扑(层、限制映射)| 每边低秩线性变换 = 小 GEMM
|
|
77
|
-
| Čech 上同调正则 | 代数拓扑(一阶上同调 H¹)|
|
|
78
|
-
| 低秩基底 KV 压缩(Plücker/Grassmannian 视角) | 射影几何 | 存基底而非 Plücker
|
|
90
|
+
| 热带门控 Tropical Gating | 热带半环分段线性 | 替代 Top-K:逐元素 max-plus——D1[v] / D2[x] 非 Tensor Core GEMM(落 CUDA core)/ D3[v] 仅逐 token 门控亚二次(min-plus matmul 则 APSP-hard 非亚二次);次可微,折点需 LogSumExp 软化(软化退回标准 softmax)|
|
|
91
|
+
| 胞腔层扩散 Cellular Sheaf Diffusion | 代数几何/拓扑(层、限制映射)| 每边低秩线性变换 = 小 GEMM(D2/D4)|
|
|
92
|
+
| Čech 上同调正则 | 代数拓扑(一阶上同调 H¹)| 局部、廉价;作幻觉的代数判据(D3/D8)|
|
|
93
|
+
| 低秩基底 KV 压缩(Plücker/Grassmannian 视角) | 射影几何 | 存基底而非 Plücker 坐标(后者低秩时反扩张);块摘要候选,压缩率/误差/吞吐需实测(D4)|
|
|
79
94
|
|
|
80
95
|
使用时不要把上表当作已验证结论。正确做法是把每个组件写进 testplan:证明或估计复杂度,测峰值显存和吞吐,检查 bf16/fp8 稳定性,确认能否落到 GEMM / batched GEMM / fused kernel。只有实测和推导都通过后,才标为 "math beautiful × GPU friendly"。
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# Skill Index: Lens Library, Activation Anchors, Design Translation Prototypes, Workflow Examples
|
|
2
|
+
|
|
3
|
+
> This file is extracted from `../skills/math-research-activator/SKILL.en.md` for reference when the full catalogs are needed. SKILL.en.md retains condensed summaries with links to this file.
|
|
4
|
+
|
|
5
|
+
## Language Routing & Mixed-Input Rules (Full Version)
|
|
6
|
+
|
|
7
|
+
Language routing only determines "which language version to read" and "what language to respond in." It does NOT affect whether the math system triggers or the A/B/C/D/E scenario classification.
|
|
8
|
+
|
|
9
|
+
### Decision Rules
|
|
10
|
+
|
|
11
|
+
1. **Judge the natural-language frame first**
|
|
12
|
+
- If the user's sentence structure, verbs, and mood particles are primarily Chinese, treat as Chinese even if English technical terms are interspersed.
|
|
13
|
+
- E.g., "帮我 design 一个 attention" / "这个 loss 有没有理论问题" / "能不能用 manifold 做 routing" → Chinese.
|
|
14
|
+
|
|
15
|
+
2. **English technical terms do not count as English primary language**
|
|
16
|
+
- attention, loss, routing, embedding, manifold, operator, kernel, KV-cache, transformer, MoE, and similar AI/math/engineering terms are domain terms and do not trigger a switch to English.
|
|
17
|
+
|
|
18
|
+
3. **Code, paths, and formulas are excluded from language detection**
|
|
19
|
+
- File paths, function names, variable names, LaTeX formulas, and CLI arguments do not count toward language ratio.
|
|
20
|
+
|
|
21
|
+
4. **When primary language is unclear, follow the user's last clear language**
|
|
22
|
+
- If the CN/EN ratio is close and indeterminate, use the user's most recent unambiguous natural language.
|
|
23
|
+
- If there is no prior context, default to Chinese.
|
|
24
|
+
|
|
25
|
+
5. **Output language matches primary language**
|
|
26
|
+
- Chinese primary → read `../skills/math-research-activator/SKILL.md`, respond in Chinese, retaining necessary English terms.
|
|
27
|
+
- English primary → read `../skills/math-research-activator/SKILL.en.md`, respond in English.
|
|
28
|
+
- If the user explicitly requests "in English" / "in Chinese," follow the explicit request.
|
|
29
|
+
|
|
30
|
+
## Lens Library (15 Mathematical Perspectives)
|
|
31
|
+
|
|
32
|
+
Each lens answers: What is this perspective? What kinds of problems is it suited to diagnose? Which knowledge domains does it route to?
|
|
33
|
+
|
|
34
|
+
| Lens | File | Core Perspective |
|
|
35
|
+
|------|------|-----------------|
|
|
36
|
+
| Axiomatization | `../lenses/axiomatization.en.md` | Examine consistency/independence/completeness of assumptions |
|
|
37
|
+
| Duality | `../lenses/duality.en.md` | Transform to the dual space to expose constraints and invariants |
|
|
38
|
+
| Symmetry | `../lenses/symmetry.en.md` | Invariants and conservation laws under transformations |
|
|
39
|
+
| Spectral Decomposition | `../lenses/spectral.en.md` | Eigenvalues/singular values reveal dominant structure |
|
|
40
|
+
| Geometric | `../lenses/geometric.en.md` | Metric/curvature/spatial structure on manifolds |
|
|
41
|
+
| Projection & Decomposition | `../lenses/projection.en.md` | Orthogonal decomposition, subspace separation, conflict elimination |
|
|
42
|
+
| Variational | `../lenses/variational.en.md` | Constrained extrema, energy minimization |
|
|
43
|
+
| Local-to-Global | `../lenses/local-to-global.en.md` | Patching local properties into global ones, cohomological obstructions |
|
|
44
|
+
| Topological | `../lenses/topological.en.md` | Invariants under continuous deformation, connectedness, holes |
|
|
45
|
+
| Categorical | `../lenses/categorical.en.md` | Universal properties, functors, natural transformations |
|
|
46
|
+
| Perturbation | `../lenses/perturbation.en.md` | Propagation of small perturbations, stability, robustness |
|
|
47
|
+
| Causal | `../lenses/causal.en.md` | Correlation ≠ causation, interventions, counterfactuals |
|
|
48
|
+
| Game-Theoretic | `../lenses/game.en.md` | Multi-agent strategic interaction, equilibria, mechanism design |
|
|
49
|
+
| Probabilistic & Statistical | `../lenses/probabilistic.en.md` | Quantifying uncertainty, Bayesian updating |
|
|
50
|
+
| Algorithmic | `../lenses/algorithmic.en.md` | Complexity, feasibility, parallelizability |
|
|
51
|
+
|
|
52
|
+
## Activation Anchors (Organized by Mathematical Domain)
|
|
53
|
+
|
|
54
|
+
Each anchor is not a closed knowledge card but answers: what math structure to activate, what deeper knowledge it connects to, what AI design actions it translates to, and how to extend when insufficient.
|
|
55
|
+
|
|
56
|
+
| Domain | Directory | Anchors |
|
|
57
|
+
|--------|-----------|----------------|
|
|
58
|
+
| Matrix Analysis | `../knowledge-base/matrix-analysis/` | projection, spectral-decomposition, low-rank-approximation, positive-semidefinite, matrix-perturbation |
|
|
59
|
+
| Optimization | `../knowledge-base/optimization/` | lagrangian-duality, convex-optimization, constrained-optimization, riemannian-optimization, proximal-method |
|
|
60
|
+
| Differential Geometry | `../knowledge-base/differential-geometry/` | manifold, tangent-space, metric-tensor, geodesic, curvature, connection |
|
|
61
|
+
| Lie Theory | `../knowledge-base/lie-theory/` | group-action, lie-group, lie-algebra, representation, equivariance |
|
|
62
|
+
| Topology | `../knowledge-base/topology/` | persistent-homology, euler-characteristic, fundamental-group |
|
|
63
|
+
| Probability & Information | `../knowledge-base/probability/` | concentration-inequality, entropy, kl-divergence, information-bottleneck, fisher-information |
|
|
64
|
+
| Information Geometry | `../knowledge-base/information-geometry/` | natural-gradient, fisher-metric |
|
|
65
|
+
|
|
66
|
+
## Design Pattern Library (Organized by AI Component)
|
|
67
|
+
|
|
68
|
+
Each design pattern answers: Mathematical origin, AI module form, implementable architecture, GPU feasibility, paper-level exposition, risks.
|
|
69
|
+
|
|
70
|
+
| Component Type | Directory | Patterns |
|
|
71
|
+
|---------------|-----------|----------|
|
|
72
|
+
| Attention | `../design-patterns/attention/` | projection-attention, spectral-attention, equivariant-attention, geometry-aware-attention, information-bottleneck-attention |
|
|
73
|
+
| Loss Functions | `../design-patterns/loss/` | orthogonality-loss, contrastive-loss, variational-loss, information-bottleneck-loss, constraint-penalty |
|
|
74
|
+
| Routing | `../design-patterns/routing/` | optimal-transport-routing, graph-routing, moe-routing, spectral-clustering-routing |
|
|
75
|
+
| Representation | `../design-patterns/representation/` | shared-private-decomposition, manifold-representation, equivariant-split, subspace-alignment |
|
|
76
|
+
| Compression | `../design-patterns/compression/` | low-rank-kv-cache, spectral-token-pruning, topology-preserving-compression, leverage-score-selection |
|
|
77
|
+
|
|
78
|
+
## Workflow Example
|
|
79
|
+
|
|
80
|
+
**User**: "Design a new KV Cache compression method that preserves long-range dependencies — I don't want to just do top-k."
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
Step 1 — Diagnosis: Scenario B (Mechanism Design)
|
|
84
|
+
Problem type: Sequence memory compression + information preservation + long-range structure
|
|
85
|
+
Core tension: Compressing token count vs. preserving long-range dependencies
|
|
86
|
+
|
|
87
|
+
Step 2 — Lens Selection:
|
|
88
|
+
1. Spectral Decomposition (preserve dominant subspace)
|
|
89
|
+
2. Information Theory (retain states with maximum mutual information)
|
|
90
|
+
3. Topological (preserve critical connectivity points of sequential structure)
|
|
91
|
+
|
|
92
|
+
Step 3 — Activation Anchors:
|
|
93
|
+
→ low-rank-approximation (Matrix Analysis anchor)
|
|
94
|
+
→ leverage-score-selection (Design Pattern: compression)
|
|
95
|
+
→ information-bottleneck (Probability & Information anchor)
|
|
96
|
+
If existing anchors are insufficient, enter Knowledge Gap Protocol to generate a temporary knowledge card.
|
|
97
|
+
|
|
98
|
+
Step 4 — Design Translation:
|
|
99
|
+
Candidate A: Spectral KV Compression (low-rank + leverage score)
|
|
100
|
+
Candidate B: Information-Preserving Cache (query sensitivity)
|
|
101
|
+
Candidate C: Topology-Preserving Cache (graph bridge-node retention)
|
|
102
|
+
|
|
103
|
+
Step 5 — Critic Review:
|
|
104
|
+
A is most GPU-friendly; B requires estimating future queries with inherent uncertainty; C has prohibitive graph construction cost
|
|
105
|
+
Recommendation: Prioritize A; use B as a lightweight gate
|
|
106
|
+
```
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# Skill 索引:透镜库、激活锚点、设计翻译原型、工作流范例
|
|
2
|
+
|
|
3
|
+
> 本文件从 `../skills/math-research-activator/SKILL.md` 提取,供需要查阅完整目录时使用。SKILL.md 保留精简摘要和指向本文件的链接。
|
|
4
|
+
|
|
5
|
+
## 语言路由与混合输入规则(完整版)
|
|
6
|
+
|
|
7
|
+
语言路由只决定"读取哪个语言版本的说明"和"最终用什么语言回答",不参与数学系统是否触发、也不参与 A/B/C/D/E 场景判断。
|
|
8
|
+
|
|
9
|
+
### 判定规则
|
|
10
|
+
|
|
11
|
+
1. **先判断自然语言主框架**
|
|
12
|
+
- 如果用户的请求句式、动词、语气词主要是中文,即使夹杂英文技术词,也按中文处理。
|
|
13
|
+
- 例如:"帮我 design 一个 attention""这个 loss 有没有理论问题""能不能用 manifold 做 routing"均按中文处理。
|
|
14
|
+
|
|
15
|
+
2. **英文技术词不计入英文主语言**
|
|
16
|
+
- attention、loss、routing、embedding、manifold、operator、kernel、KV-cache、transformer、MoE 等 AI/数学/工程术语视为领域术语,不作为切换到英文的依据。
|
|
17
|
+
|
|
18
|
+
3. **代码、路径、公式不参与语言判定**
|
|
19
|
+
- 文件路径、函数名、变量名、LaTeX 公式、命令行参数不计入语言比例。
|
|
20
|
+
|
|
21
|
+
4. **主语言不明显时,沿用用户上一轮主要语言**
|
|
22
|
+
- 若中英文比例接近且无法判断,以用户最近一次明确使用的自然语言为准。
|
|
23
|
+
- 若没有上下文,默认中文。
|
|
24
|
+
|
|
25
|
+
5. **输出语言与主语言一致**
|
|
26
|
+
- 中文主语言 → 读取中文 `../skills/math-research-activator/SKILL.md`,用中文回答,保留必要英文术语。
|
|
27
|
+
- 英文主语言 → 读取 `../skills/math-research-activator/SKILL.en.md`,用英文回答。
|
|
28
|
+
- 用户明确要求"用英文/用中文"时,以用户显式要求为准。
|
|
29
|
+
|
|
30
|
+
## 透镜库(15 个数学视角)
|
|
31
|
+
|
|
32
|
+
每个透镜回答:这是什么视角?适合诊断什么问题?会路由到哪些知识域?
|
|
33
|
+
|
|
34
|
+
| 透镜 | 文件 | 核心视角 |
|
|
35
|
+
|------|------|---------|
|
|
36
|
+
| 公理化 | `../lenses/axiomatization.md` | 审查假设的相容性/独立性/完备性 |
|
|
37
|
+
| 对偶 | `../lenses/duality.md` | 转换到对偶空间暴露约束与不变量 |
|
|
38
|
+
| 对称性 | `../lenses/symmetry.md` | 变换下的不变量与守恒律 |
|
|
39
|
+
| 谱分解 | `../lenses/spectral.md` | 特征值/奇异值揭示主导结构 |
|
|
40
|
+
| 几何 | `../lenses/geometric.md` | 度量/曲率/流形上的空间结构 |
|
|
41
|
+
| 投影与分解 | `../lenses/projection.md` | 正交分解、子空间分离、冲突消除 |
|
|
42
|
+
| 变分 | `../lenses/variational.md` | 约束下极值、能量最小化 |
|
|
43
|
+
| 局部到整体 | `../lenses/local-to-global.md` | 局部性质拼接为全局、层上同调障碍 |
|
|
44
|
+
| 拓扑 | `../lenses/topological.md` | 连续变形不变量、连通性、空洞 |
|
|
45
|
+
| 范畴化 | `../lenses/categorical.md` | 泛性质、函子、自然变换 |
|
|
46
|
+
| 扰动 | `../lenses/perturbation.md` | 小扰动的传播、稳定性、鲁棒性 |
|
|
47
|
+
| 因果 | `../lenses/causal.md` | 相关≠因果、干预、反事实 |
|
|
48
|
+
| 博弈 | `../lenses/game.md` | 多方策略互动、均衡、机制设计 |
|
|
49
|
+
| 概率统计 | `../lenses/probabilistic.md` | 量化不确定性、贝叶斯更新 |
|
|
50
|
+
| 算法 | `../lenses/algorithmic.md` | 复杂度、可行性、并行性 |
|
|
51
|
+
|
|
52
|
+
## 激活锚点(按数学领域组织)
|
|
53
|
+
|
|
54
|
+
每个锚点不是封闭知识卡,而是回答:激活什么数学结构、连接哪些更深知识、可翻译成哪些 AI 设计动作、不足时如何扩展。
|
|
55
|
+
|
|
56
|
+
| 领域 | 目录 | 锚点 |
|
|
57
|
+
|------|------|---------|
|
|
58
|
+
| 矩阵分析 | `../knowledge-base/matrix-analysis/` | projection, spectral-decomposition, low-rank-approximation, positive-semidefinite, matrix-perturbation |
|
|
59
|
+
| 最优化 | `../knowledge-base/optimization/` | lagrangian-duality, convex-optimization, constrained-optimization, riemannian-optimization, proximal-method |
|
|
60
|
+
| 微分几何 | `../knowledge-base/differential-geometry/` | manifold, tangent-space, metric-tensor, geodesic, curvature, connection |
|
|
61
|
+
| 李理论 | `../knowledge-base/lie-theory/` | group-action, lie-group, lie-algebra, representation, equivariance |
|
|
62
|
+
| 拓扑 | `../knowledge-base/topology/` | persistent-homology, euler-characteristic, fundamental-group |
|
|
63
|
+
| 概率与信息 | `../knowledge-base/probability/` | concentration-inequality, entropy, kl-divergence, information-bottleneck, fisher-information |
|
|
64
|
+
| 信息几何 | `../knowledge-base/information-geometry/` | natural-gradient, fisher-metric |
|
|
65
|
+
|
|
66
|
+
## 设计模式库(按 AI 组件组织)
|
|
67
|
+
|
|
68
|
+
每个设计模式回答:数学来源、AI 模块形式、可实现结构、GPU 可行性、论文表述、风险。
|
|
69
|
+
|
|
70
|
+
| 组件类型 | 目录 | 模式 |
|
|
71
|
+
|---------|------|------|
|
|
72
|
+
| 注意力 | `../design-patterns/attention/` | projection-attention, spectral-attention, equivariant-attention, geometry-aware-attention, information-bottleneck-attention |
|
|
73
|
+
| 损失函数 | `../design-patterns/loss/` | orthogonality-loss, contrastive-loss, variational-loss, information-bottleneck-loss, constraint-penalty |
|
|
74
|
+
| 路由 | `../design-patterns/routing/` | optimal-transport-routing, graph-routing, moe-routing, spectral-clustering-routing |
|
|
75
|
+
| 表示 | `../design-patterns/representation/` | shared-private-decomposition, manifold-representation, equivariant-split, subspace-alignment |
|
|
76
|
+
| 压缩 | `../design-patterns/compression/` | low-rank-kv-cache, spectral-token-pruning, topology-preserving-compression, leverage-score-selection |
|
|
77
|
+
|
|
78
|
+
## 工作流范例
|
|
79
|
+
|
|
80
|
+
**用户**:"设计新的 KV Cache 压缩方法,保留长期依赖,不想只做 top-k"
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
第一步 诊断:场景 B(机制设计)
|
|
84
|
+
问题类型:序列记忆压缩 + 信息保留 + 长程结构
|
|
85
|
+
核心张力:压缩 token 数量 vs 不破坏长期依赖
|
|
86
|
+
|
|
87
|
+
第二步 透镜选择:
|
|
88
|
+
1. 谱分解(保留主导子空间)
|
|
89
|
+
2. 信息论(保留最大互信息状态)
|
|
90
|
+
3. 拓扑(保留序列结构关键连接点)
|
|
91
|
+
|
|
92
|
+
第三步 激活锚点:
|
|
93
|
+
→ low-rank-approximation(矩阵分析锚点)
|
|
94
|
+
→ leverage-score-selection(压缩设计模式)
|
|
95
|
+
→ information-bottleneck(概率与信息锚点)
|
|
96
|
+
若现有锚点不足,进入 Knowledge Gap Protocol 生成临时知识卡。
|
|
97
|
+
|
|
98
|
+
第四步 设计翻译:
|
|
99
|
+
候选 A:Spectral KV Compression(低秩 + leverage score)
|
|
100
|
+
候选 B:Information-Preserving Cache(query sensitivity)
|
|
101
|
+
候选 C:Topology-Preserving Cache(图桥接节点保留)
|
|
102
|
+
|
|
103
|
+
第五步 Critic 审查:
|
|
104
|
+
A 最 GPU 友好,B 需估计未来 query 有不确定性,C 图构建成本过高
|
|
105
|
+
建议:优先 A,B 作轻量 gate
|
|
106
|
+
```
|