math-skill 3.0.0 → 3.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en-US.md +50 -30
- package/README.md +26 -6
- package/agents/math-critic.en.md +30 -28
- package/agents/math-critic.md +30 -30
- package/commands/ask.en.md +15 -0
- package/commands/ask.md +6 -1
- package/design-patterns/attention/equivariant-attention.en.md +17 -11
- package/design-patterns/attention/equivariant-attention.md +16 -10
- package/design-patterns/attention/geometry-aware-attention.en.md +11 -11
- package/design-patterns/attention/geometry-aware-attention.md +10 -10
- package/design-patterns/attention/information-bottleneck-attention.en.md +40 -20
- package/design-patterns/attention/information-bottleneck-attention.md +40 -20
- package/design-patterns/attention/projection-attention.en.md +12 -12
- package/design-patterns/attention/projection-attention.md +11 -11
- package/design-patterns/attention/spectral-attention.en.md +27 -13
- package/design-patterns/attention/spectral-attention.md +26 -12
- package/design-patterns/compression/leverage-score-selection.en.md +14 -14
- package/design-patterns/compression/leverage-score-selection.md +14 -14
- package/design-patterns/compression/low-rank-kv-cache.en.md +40 -13
- package/design-patterns/compression/low-rank-kv-cache.md +39 -13
- package/design-patterns/compression/spectral-token-pruning.en.md +17 -15
- package/design-patterns/compression/spectral-token-pruning.md +17 -15
- package/design-patterns/compression/topology-preserving-compression.en.md +11 -11
- package/design-patterns/compression/topology-preserving-compression.md +11 -11
- package/design-patterns/loss/constraint-penalty.en.md +9 -5
- package/design-patterns/loss/constraint-penalty.md +10 -7
- package/design-patterns/loss/contrastive-loss.en.md +38 -12
- package/design-patterns/loss/contrastive-loss.md +39 -13
- package/design-patterns/loss/information-bottleneck-loss.en.md +6 -6
- package/design-patterns/loss/information-bottleneck-loss.md +7 -7
- package/design-patterns/loss/orthogonality-loss.en.md +25 -18
- package/design-patterns/loss/orthogonality-loss.md +25 -19
- package/design-patterns/loss/variational-loss.en.md +5 -5
- package/design-patterns/loss/variational-loss.md +6 -6
- package/design-patterns/overview.en.md +36 -0
- package/design-patterns/overview.md +36 -0
- package/design-patterns/representation/equivariant-split.en.md +5 -5
- package/design-patterns/representation/equivariant-split.md +5 -6
- package/design-patterns/representation/manifold-representation.en.md +13 -13
- package/design-patterns/representation/manifold-representation.md +14 -15
- package/design-patterns/representation/shared-private-decomposition.en.md +13 -8
- package/design-patterns/representation/shared-private-decomposition.md +13 -10
- package/design-patterns/representation/subspace-alignment.en.md +13 -13
- package/design-patterns/representation/subspace-alignment.md +13 -13
- package/design-patterns/routing/graph-routing.en.md +12 -12
- package/design-patterns/routing/graph-routing.md +12 -12
- package/design-patterns/routing/moe-routing.en.md +13 -13
- package/design-patterns/routing/moe-routing.md +13 -15
- package/design-patterns/routing/optimal-transport-routing.en.md +6 -6
- package/design-patterns/routing/optimal-transport-routing.md +7 -7
- package/design-patterns/routing/spectral-clustering-routing.en.md +27 -16
- package/design-patterns/routing/spectral-clustering-routing.md +27 -18
- package/knowledge-base/differential-geometry/connection.en.md +17 -2
- package/knowledge-base/differential-geometry/connection.md +17 -2
- package/knowledge-base/differential-geometry/curvature.en.md +14 -1
- package/knowledge-base/differential-geometry/curvature.md +14 -1
- package/knowledge-base/differential-geometry/geodesic.en.md +16 -2
- package/knowledge-base/differential-geometry/geodesic.md +16 -2
- package/knowledge-base/differential-geometry/index.en.md +52 -0
- package/knowledge-base/differential-geometry/index.md +52 -0
- package/knowledge-base/differential-geometry/manifold.en.md +15 -2
- package/knowledge-base/differential-geometry/manifold.md +15 -2
- package/knowledge-base/differential-geometry/metric-tensor.en.md +16 -2
- package/knowledge-base/differential-geometry/metric-tensor.md +17 -3
- package/knowledge-base/differential-geometry/tangent-space.en.md +17 -3
- package/knowledge-base/differential-geometry/tangent-space.md +17 -3
- package/knowledge-base/information-geometry/fisher-metric.en.md +32 -13
- package/knowledge-base/information-geometry/fisher-metric.md +32 -13
- package/knowledge-base/information-geometry/index.en.md +45 -0
- package/knowledge-base/information-geometry/index.md +45 -0
- package/knowledge-base/information-geometry/natural-gradient.en.md +24 -10
- package/knowledge-base/information-geometry/natural-gradient.md +24 -10
- package/knowledge-base/lie-theory/equivariance.en.md +18 -3
- package/knowledge-base/lie-theory/equivariance.md +18 -3
- package/knowledge-base/lie-theory/group-action.en.md +16 -2
- package/knowledge-base/lie-theory/group-action.md +16 -2
- package/knowledge-base/lie-theory/index.en.md +52 -0
- package/knowledge-base/lie-theory/index.md +52 -0
- package/knowledge-base/lie-theory/lie-algebra.en.md +17 -2
- package/knowledge-base/lie-theory/lie-algebra.md +17 -2
- package/knowledge-base/lie-theory/lie-group.en.md +17 -3
- package/knowledge-base/lie-theory/lie-group.md +17 -3
- package/knowledge-base/lie-theory/representation.en.md +17 -2
- package/knowledge-base/lie-theory/representation.md +17 -2
- package/knowledge-base/matrix-analysis/index.en.md +49 -0
- package/knowledge-base/matrix-analysis/index.md +49 -0
- package/knowledge-base/matrix-analysis/low-rank-approximation.en.md +20 -7
- package/knowledge-base/matrix-analysis/low-rank-approximation.md +20 -7
- package/knowledge-base/matrix-analysis/matrix-perturbation.en.md +13 -1
- package/knowledge-base/matrix-analysis/matrix-perturbation.md +13 -1
- package/knowledge-base/matrix-analysis/positive-semidefinite.en.md +20 -6
- package/knowledge-base/matrix-analysis/positive-semidefinite.md +20 -6
- package/knowledge-base/matrix-analysis/projection.en.md +14 -1
- package/knowledge-base/matrix-analysis/projection.md +14 -1
- package/knowledge-base/matrix-analysis/spectral-decomposition.en.md +14 -1
- package/knowledge-base/matrix-analysis/spectral-decomposition.md +14 -1
- package/knowledge-base/optimization/constrained-optimization.en.md +17 -3
- package/knowledge-base/optimization/constrained-optimization.md +17 -3
- package/knowledge-base/optimization/convex-optimization.en.md +15 -2
- package/knowledge-base/optimization/convex-optimization.md +15 -2
- package/knowledge-base/optimization/index.en.md +50 -0
- package/knowledge-base/optimization/index.md +50 -0
- package/knowledge-base/optimization/lagrangian-duality.en.md +18 -4
- package/knowledge-base/optimization/lagrangian-duality.md +18 -4
- package/knowledge-base/optimization/proximal-method.en.md +15 -1
- package/knowledge-base/optimization/proximal-method.md +15 -1
- package/knowledge-base/optimization/riemannian-optimization.en.md +18 -4
- package/knowledge-base/optimization/riemannian-optimization.md +18 -4
- package/knowledge-base/overview.en.md +20 -8
- package/knowledge-base/overview.md +20 -8
- package/knowledge-base/probability/concentration-inequality.en.md +21 -7
- package/knowledge-base/probability/concentration-inequality.md +21 -7
- package/knowledge-base/probability/entropy.en.md +24 -8
- package/knowledge-base/probability/entropy.md +23 -7
- package/knowledge-base/probability/fisher-information.en.md +24 -10
- package/knowledge-base/probability/fisher-information.md +24 -10
- package/knowledge-base/probability/index.en.md +51 -0
- package/knowledge-base/probability/index.md +51 -0
- package/knowledge-base/probability/information-bottleneck.en.md +24 -8
- package/knowledge-base/probability/information-bottleneck.md +24 -8
- package/knowledge-base/probability/kl-divergence.en.md +21 -8
- package/knowledge-base/probability/kl-divergence.md +21 -8
- package/knowledge-base/topology/euler-characteristic.en.md +15 -2
- package/knowledge-base/topology/euler-characteristic.md +15 -2
- package/knowledge-base/topology/fundamental-group.en.md +16 -2
- package/knowledge-base/topology/fundamental-group.md +16 -2
- package/knowledge-base/topology/index.en.md +48 -0
- package/knowledge-base/topology/index.md +48 -0
- package/knowledge-base/topology/persistent-homology.en.md +17 -2
- package/knowledge-base/topology/persistent-homology.md +17 -2
- package/lenses/algorithmic.en.md +1 -1
- package/lenses/algorithmic.md +1 -1
- package/lenses/axiomatization.en.md +1 -1
- package/lenses/axiomatization.md +1 -1
- package/lenses/categorical.en.md +1 -1
- package/lenses/categorical.md +1 -1
- package/lenses/causal.en.md +1 -1
- package/lenses/causal.md +1 -1
- package/lenses/duality.en.md +1 -1
- package/lenses/duality.md +1 -1
- package/lenses/game.en.md +2 -2
- package/lenses/game.md +2 -2
- package/lenses/geometric.en.md +1 -1
- package/lenses/geometric.md +1 -1
- package/lenses/local-to-global.en.md +1 -1
- package/lenses/local-to-global.md +1 -1
- package/lenses/perturbation.en.md +1 -1
- package/lenses/perturbation.md +1 -1
- package/lenses/probabilistic.en.md +1 -1
- package/lenses/probabilistic.md +1 -1
- package/lenses/projection.en.md +1 -1
- package/lenses/projection.md +1 -1
- package/lenses/spectral.en.md +2 -2
- package/lenses/spectral.md +2 -2
- package/lenses/symmetry.en.md +1 -1
- package/lenses/symmetry.md +1 -1
- package/lenses/topological.en.md +2 -2
- package/lenses/topological.md +2 -2
- package/lenses/variational.en.md +1 -1
- package/lenses/variational.md +1 -1
- package/package.json +5 -5
- package/references/agentic-workflow.en.md +3 -3
- package/references/agentic-workflow.md +1 -1
- package/references/books/abstract-algebra.en.md +124 -0
- package/references/books/abstract-algebra.md +17 -17
- package/references/books/algebraic-geometry-rising-sea.en.md +171 -0
- package/references/books/algebraic-geometry-rising-sea.md +7 -7
- package/references/books/differential-geometry.en.md +140 -0
- package/references/books/differential-geometry.md +23 -23
- package/references/books/matrix-analysis.en.md +146 -0
- package/references/books/matrix-analysis.md +32 -32
- package/references/books/micro-lie-theory.en.md +116 -0
- package/references/books/micro-lie-theory.md +16 -16
- package/references/books/optimization-ml.en.md +163 -0
- package/references/books/optimization-ml.md +20 -21
- package/references/books/smooth-manifolds.en.md +105 -0
- package/references/books/smooth-manifolds.md +15 -15
- package/references/gpu-friendly-math.en.md +35 -9
- package/references/gpu-friendly-math.md +33 -5
- package/references/skill-index.en.md +105 -0
- package/references/skill-index.md +105 -0
- package/skills/math-research-activator/SKILL.en.md +72 -88
- package/skills/math-research-activator/SKILL.md +61 -78
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#
|
|
1
|
+
# 光滑流形 / Smooth Manifolds
|
|
2
2
|
|
|
3
3
|
> **书目**:John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition. Graduate Texts in Mathematics 218, Springer, 2013. ISBN 978-1-4419-9981-8 / DOI 10.1007/978-1-4419-9982-5. MSC 53-01 / 58-01 / 57-01.
|
|
4
4
|
> **定位**:把"局部像欧氏空间、整体可弯曲"的对象(manifold)配上微积分(切空间、向量场、微分形式、流、李导数),是 **流形优化、隐空间几何、可微结构** 的数学母体。
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
- **Ch 7 Lie Groups**:既是群又是流形(SO(n), U(n), GL(n), Stiefel…),李代数 = 单位元处切空间。→ 正交/酉权重约束、等变。
|
|
17
17
|
- **Ch 8–9 Vector Fields / Integral Curves and Flows**:向量场、积分曲线、流(flow,单参数微分同胚群)、李导数与李括号 [X,Y]。→ **Neural ODE / 扩散 / 连续归一化流** 的母结构。
|
|
18
18
|
- **Ch 10–12 Vector Bundles / Cotangent Bundle / Tensors**:丛、余向量场(1-form)、拉回(pullback)、张量。→ 梯度的真身是余向量。
|
|
19
|
-
- **Ch 13 Riemannian Metrics**:每点内积、长度/距离/体积、切-余切同构(musical
|
|
19
|
+
- **Ch 13 Riemannian Metrics**:每点内积、长度/距离/体积、切-余切同构(musical (sharp)/(flat),升降指标)。→ **自然梯度 / 黎曼优化的度量来源**。
|
|
20
20
|
- **Ch 14–16 Differential Forms / Orientations / Integration**:k-形式、楔积、外微分 d(d²=0)、定向、体积形式、流形上积分与变量替换。→ 归一化流的 log-det-Jacobian = 体积形式拉回。
|
|
21
21
|
- **Ch 17–18 De Rham Cohomology / de Rham Theorem**:闭形式模去恰当形式 = 从微分数据读出的拓扑不变量。→ 全局障碍 / 上同调正则。
|
|
22
22
|
- **Ch 19–22 Distributions & Foliations / Exponential Map / Quotient Manifolds / Symplectic Manifolds**:可积分布(Frobenius)、指数映射(retraction 原型)、商流形(Grassmann 等)、辛形式与 Hamilton 流。→ retraction、商空间约束、辛积分器 / HMC。
|
|
@@ -25,8 +25,8 @@
|
|
|
25
25
|
|
|
26
26
|
## 可迁移到 AI/Infra 的核心结构
|
|
27
27
|
|
|
28
|
-
- **切空间 = 参数/隐空间的局部线性化(local linearization)**。`df_p: T_pM → T_{f(p)}N` 就是 Jacobian / pushforward
|
|
29
|
-
- **梯度是余向量(covector),不是向量**。autodiff 给出的是 1-form(余切空间元素);要变成可下降的方向(切向量)必须用
|
|
28
|
+
- **切空间 = 参数/隐空间的局部线性化(local linearization)**。`df_p: T_pM → T_{f(p)}N` 就是 Jacobian / pushforward(前推,对应 JVP / 前向模式 AD);反向传播 = 余切丛上的拉回(pullback on cotangent bundle,VJP = 向量-Jacobian 积 = 余向量的拉回),即沿复合映射做 pullback(链式法则的几何版)。一切一阶方法都活在切空间里。
|
|
29
|
+
- **梯度是余向量(covector),不是向量**。autodiff 给出的是 1-form(余切空间元素);要变成可下降的方向(切向量)必须用 **度量升指标**((sharp))。欧氏度量 → 普通梯度;Fisher 度量 → 自然梯度(natural gradient)。**这是自然梯度 / 镜像下降的流形根因**。
|
|
30
30
|
- **约束集 = 子流形(submanifold)**。正则水平集定理:当 g 是 submersion 时 `g(x)=c` 的解集是光滑子流形;约束优化 = 在子流形上做无约束优化。
|
|
31
31
|
- **李群 = 可微的对称群**。SO(n)/U(n)/Stiefel/Grassmann 都是流形;其李代数(如反对称矩阵 so(n))是线性空间,用 `exp` 映射回群 → **把"约束权重"重参数化为"无约束李代数 + exp"**。
|
|
32
32
|
- **流(flow)= 时间参数化的微分同胚族**。学一个向量场 + 沿它积分 = Neural ODE / 连续归一化流 / 扩散采样。流的可逆性、保体积性直接对应模型性质。
|
|
@@ -57,11 +57,11 @@
|
|
|
57
57
|
|
|
58
58
|
逐维对照:
|
|
59
59
|
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
-
|
|
63
|
-
-
|
|
64
|
-
-
|
|
60
|
+
- **D1–D2** 切空间运算(pushforward/pullback、Jacobian-向量积、把梯度投影到切空间)**天然是 batched GEMM** [v]——反向传播本就是 pullback(VJP),这部分对 GPU 极友好。**但** retraction/exp 多半要 QR、特征分解、矩阵指数或小矩阵求逆:QR/eig **不是干净的 GEMM**,是带串行依赖的分解(cuSOLVER 批量小矩阵尚可,大矩阵 O(n³) 且并行差)→ **可改造** 而非天然友好。
|
|
61
|
+
- **D3 复杂度**:测地线距离、平行移动、一般 `log|det J|` 都是 O(n³) 起。**改造**:限定有闭式测地线的流形(球面/双曲/SO(3));归一化流用三角/耦合层让 log-det 退化成对角和(O(n))。
|
|
62
|
+
- **D5 低精度**:[~] **最大坑**。矩阵 `exp / log / sqrt`、特征分解、SPD 的仿射不变度量在 bf16/fp16 下 **灾难性不稳定**,常静默地需要 fp32/fp64。流形原语经常"表面能跑、数值早已发散"。
|
|
63
|
+
- **D6 并行与通信**:scaling-and-squaring 的平方链、ODE 积分步、Householder/QR 都有 **串行递推**,难跨 SM/设备 overlap。反例向好:显式辛积分器(leapfrog)高并行 [v]。
|
|
64
|
+
- **D4/D7/D8 显存 / 稀疏 / 融合**:李代数/旋转参数化若限制在 **小矩阵或块对角**(如逐头旋转、SO(3) 的 Rodrigues 闭式),可融进 kernel、走 Tensor Core;大稠密流形算子则要物化大中间张量。
|
|
65
65
|
|
|
66
66
|
**结论与改造手法(呼应 gpu-friendly-math.md 工具箱)**:
|
|
67
67
|
|
|
@@ -73,11 +73,11 @@
|
|
|
73
73
|
|
|
74
74
|
## 该调用哪个思想透镜
|
|
75
75
|
|
|
76
|
-
- **
|
|
77
|
-
- **symmetry
|
|
76
|
+
- **variational(变分透镜)**:主透镜——约束下寻最优、黎曼/流形优化、retraction 选型。
|
|
77
|
+
- **symmetry(对称与不变性)**:李群、等变、商流形、群作用下的不变量。
|
|
78
78
|
- **duality(对偶透镜)**:坐标卡变换、pushforward/pullback、归一化流的变量替换、微分同胚。
|
|
79
|
-
- **topological
|
|
80
|
-
- **
|
|
79
|
+
- **topological(拓扑透镜)**:de Rham 上同调、全局障碍、隐空间的"洞"与连通性。
|
|
80
|
+
- **categorical(范畴化透镜)**:从高维杂乱的环境数据中抽出"局部线性 + 光滑拼接"的流形骨架(流形假设)。
|
|
81
81
|
|
|
82
82
|
## 反模式
|
|
83
83
|
|
|
@@ -90,7 +90,7 @@
|
|
|
90
90
|
|
|
91
91
|
## 深挖入口
|
|
92
92
|
|
|
93
|
-
>
|
|
93
|
+
> **书目信息**:John M. Lee, *Introduction to Smooth Manifolds*, 2nd Edition, Graduate Texts in Mathematics 218, Springer, 2013. ISBN 978-1-4419-9981-8.
|
|
94
94
|
>
|
|
95
95
|
> **启用方式**:将 `Introduction to Smooth Manifolds.pdf` 放入项目根目录的 `math_book/` 文件夹,Agent 即可自动搜索原文。PDF 不随 npm/git 分发(版权原因),需自行获取。
|
|
96
96
|
|
|
@@ -98,7 +98,7 @@
|
|
|
98
98
|
|
|
99
99
|
- **Ch 3 Tangent Vectors** — 切空间、微分/pushforward、切丛:局部线性化与反传的几何原型。
|
|
100
100
|
- **Ch 11 The Cotangent Bundle** — 余向量场(1-form)、`df` 作为余向量、pullback:梯度真身 = 余向量。
|
|
101
|
-
- **Ch 13 Riemannian Metrics** —
|
|
101
|
+
- **Ch 13 Riemannian Metrics** — 度量、切-余切同构((sharp)/(flat))、距离:自然梯度 / 黎曼优化的根。
|
|
102
102
|
- **Ch 9 Integral Curves and Flows** — 流、积分曲线、李导数/李括号:Neural ODE / 扩散 / 保结构动力学。
|
|
103
103
|
- **Ch 20 The Exponential Map** — 指数映射:retraction 原型,也是 GPU 可行性的主瓶颈。
|
|
104
104
|
|
|
@@ -1,9 +1,35 @@
|
|
|
1
1
|
# GPU-Friendly Math Checklist
|
|
2
2
|
|
|
3
3
|
> This file is the **single source of truth** for the "GPU-feasibility" acceptance gate.
|
|
4
|
-
> The activator, the 15 thinking lenses, `books/*.md`, and `../agents/math-critic.md` all reference this file; no duplicate definitions elsewhere.
|
|
5
|
-
|
|
6
|
-
|
|
4
|
+
> The activator, the 15 thinking lenses, `books/*.md`, and `../agents/math-critic.en.md` all reference this file; no duplicate definitions elsewhere.
|
|
5
|
+
|
|
6
|
+
## Eight-Dimension Abbreviations
|
|
7
|
+
|
|
8
|
+
| Abbrev. | Full Name |
|
|
9
|
+
|---------|-----------|
|
|
10
|
+
| D1 | Tensorization |
|
|
11
|
+
| D2 | GEMM-mappability |
|
|
12
|
+
| D3 | Complexity |
|
|
13
|
+
| D4 | Memory & KV-Cache |
|
|
14
|
+
| D5 | Low-precision stability |
|
|
15
|
+
| D6 | Parallelism & communication |
|
|
16
|
+
| D7 | Sparse structure |
|
|
17
|
+
| D8 | Operator fusion |
|
|
18
|
+
|
|
19
|
+
## Quantitative Checklist
|
|
20
|
+
|
|
21
|
+
When evaluating each dimension, provide concrete numbers rather than just labels:
|
|
22
|
+
|
|
23
|
+
| Dimension | Quantitative Questions to Answer |
|
|
24
|
+
|-----------|--------------------------------|
|
|
25
|
+
| D1 Tensorization | What are the tensor shapes of core operations? Where is the batch dimension? |
|
|
26
|
+
| D2 GEMM-mappability | How many GEMM/matmul operations? What are the (M,N,K) dimensions of each? |
|
|
27
|
+
| D3 Complexity | Total FLOPs? Ratio vs. baseline (standard attention/MLP)? |
|
|
28
|
+
| D4 Memory | Peak memory (bytes)? Is an n×n matrix materialized? KV-Cache overhead? |
|
|
29
|
+
| D5 Low-precision | Numerical error magnitude under bf16/fp8? Mixed-precision strategy needed? |
|
|
30
|
+
| D6 Parallelism | Theoretical parallelism degree? Communication volume (bytes/step)? All-reduce required? |
|
|
31
|
+
| D7 Sparsity | Sparsity ratio (%)? Sparse format (CSR/BSR/block-sparse)? Dedicated kernel available? |
|
|
32
|
+
| D8 Operator Fusion | Number of fusible kernels? Reduction in kernel launches and memory transfers after fusion? |
|
|
7
33
|
|
|
8
34
|
## Core Proposition
|
|
9
35
|
|
|
@@ -18,7 +44,7 @@ Many "beautiful on paper" modern mathematical structures cannot run at high perf
|
|
|
18
44
|
|
|
19
45
|
For any candidate structure (operator, attention variant, routing mechanism, regularization term, compression scheme…) rate each dimension as `Friendly / Retrofittable / Unfriendly` and provide adaptation recommendations.
|
|
20
46
|
|
|
21
|
-
| # | Dimension | Key Question | Friendly
|
|
47
|
+
| # | Dimension | Key Question | Friendly [v] | Unfriendly [x] |
|
|
22
48
|
|---|-----------|-------------|------------|--------------|
|
|
23
49
|
| 1 | **Tensorization** | Can it be expressed as dense tensor operations, avoiding element-wise irregular control flow? | Batched tensor algebra | Scalar loops, data-dependent branches |
|
|
24
50
|
| 2 | **GEMM-mappability** | Can it be reduced to matrix multiplication / batched GEMM / convolution to fully utilize Tensor Cores? | Expressible as a GEMM chain | Irregular computations that cannot be expressed as matrix operations |
|
|
@@ -53,13 +79,13 @@ Common techniques for transforming "beautiful but non-computable" into "both bea
|
|
|
53
79
|
|
|
54
80
|
## Worked Example: Tropical Sheaf Attention
|
|
55
81
|
|
|
56
|
-
Drawn from the auto-research directions cited in `agentic-workflow.md`, demonstrating how a **candidate design enters the 8-dimension verification**:
|
|
82
|
+
Drawn from the auto-research directions cited in `agentic-workflow.en.md`, demonstrating how a **candidate design enters the 8-dimension verification**:
|
|
57
83
|
|
|
58
84
|
| Component | Mathematical Source | GPU Friendliness |
|
|
59
85
|
|-----------|-------------------|-----------------|
|
|
60
|
-
| Tropical Gating | Tropical semiring, piecewise-linear | Replaces Top-K: element-wise max-plus —
|
|
61
|
-
| Cellular Sheaf Diffusion | Algebraic geometry / topology (sheaves, restriction maps) | Each edge is a low-rank linear transform = small GEMM (
|
|
62
|
-
| Čech Cohomology Regularization | Algebraic topology (first cohomology $H^1$) | Local, inexpensive; serves as an algebraic criterion for hallucination (
|
|
63
|
-
| Low-Rank Basis KV Compression (Plücker/Grassmannian perspective) | Projective geometry | Store the basis rather than Plücker coordinates (the latter expands when low-rank); block-summary candidate — compression ratio / error / throughput must be benchmarked (
|
|
86
|
+
| Tropical Gating | Tropical semiring, piecewise-linear | Replaces Top-K: element-wise max-plus — D1[v] / D2[x] Not a Tensor Core GEMM (runs on CUDA cores) / D3[v] Per-token gating only, sub-quadratic (min-plus matmul is APSP-hard, not sub-quadratic); sub-differentiable, kinks require LogSumExp smoothing (smoothing recovers standard softmax) |
|
|
87
|
+
| Cellular Sheaf Diffusion | Algebraic geometry / topology (sheaves, restriction maps) | Each edge is a low-rank linear transform = small GEMM (D2/D4) |
|
|
88
|
+
| Čech Cohomology Regularization | Algebraic topology (first cohomology $H^1$) | Local, inexpensive; serves as an algebraic criterion for hallucination (D3/D8) |
|
|
89
|
+
| Low-Rank Basis KV Compression (Plücker/Grassmannian perspective) | Projective geometry | Store the basis rather than Plücker coordinates (the latter expands when low-rank); block-summary candidate — compression ratio / error / throughput must be benchmarked (D4) |
|
|
64
90
|
|
|
65
91
|
Do not treat the table above as validated conclusions. The correct approach is to enter each component into the test plan: prove or estimate complexity, measure peak memory and throughput, check bf16/fp8 stability, and confirm whether it can be mapped to GEMM / batched GEMM / fused kernels. Only after both empirical benchmarks and theoretical derivations pass should a component be labeled "math beautiful × GPU friendly."
|
|
@@ -7,6 +7,34 @@
|
|
|
7
7
|
>
|
|
8
8
|
> This file is the single source of truth for the "GPU-feasibility" acceptance gate. The activator, the 15 thinking lenses, the book references, and the math-critic all point here.
|
|
9
9
|
|
|
10
|
+
## 八维缩写速查
|
|
11
|
+
|
|
12
|
+
| 缩写 | 全称 |
|
|
13
|
+
|------|------|
|
|
14
|
+
| D1 | 张量化 |
|
|
15
|
+
| D2 | GEMM 可映射 |
|
|
16
|
+
| D3 | 复杂度 |
|
|
17
|
+
| D4 | 显存与 KV-Cache |
|
|
18
|
+
| D5 | 低精度稳定 |
|
|
19
|
+
| D6 | 并行与通信 |
|
|
20
|
+
| D7 | 稀疏结构 |
|
|
21
|
+
| D8 | 算子融合 |
|
|
22
|
+
|
|
23
|
+
## 量化检查清单
|
|
24
|
+
|
|
25
|
+
评估每个维度时,应填写具体数值而非仅给标签:
|
|
26
|
+
|
|
27
|
+
| 维度 | 需回答的量化问题 |
|
|
28
|
+
|------|----------------|
|
|
29
|
+
| D1 张量化 | 核心操作的 tensor shape 是什么?batch 维在哪? |
|
|
30
|
+
| D2 GEMM 可映射 | 操作可分解为几次 GEMM/matmul?每次的 (M,N,K) 是多少? |
|
|
31
|
+
| D3 复杂度 | FLOPs 总量?与 baseline (标准 attention/MLP) 的比值? |
|
|
32
|
+
| D4 显存 | 峰值显存 (bytes)?是否物化 n×n 矩阵?KV-Cache 额外开销? |
|
|
33
|
+
| D5 低精度 | bf16/fp8 下的数值误差量级?是否需要混合精度策略? |
|
|
34
|
+
| D6 并行 | 理论并行度?通信量 (bytes/step)?是否需要 all-reduce? |
|
|
35
|
+
| D7 稀疏 | 稀疏度 (%)?稀疏格式 (CSR/BSR/block-sparse)?是否有专用 kernel? |
|
|
36
|
+
| D8 算子融合 | 可融合的 kernel 数?融合后减少的 kernel launch 次数和显存搬运量? |
|
|
37
|
+
|
|
10
38
|
## 核心命题 / Core Proposition
|
|
11
39
|
|
|
12
40
|
**数学美 ≠ 可算。** 一个结构要真正进入现代 GPU 集群的训练与推理,必须同时满足两件事:
|
|
@@ -20,7 +48,7 @@
|
|
|
20
48
|
|
|
21
49
|
对任何候选结构(算子、注意力变体、路由、正则项、压缩方案……)逐项评 `友好 / 可改造 / 不友好`,并给出改造建议。
|
|
22
50
|
|
|
23
|
-
| # | 维度 | 关键问题 | 友好
|
|
51
|
+
| # | 维度 | 关键问题 | 友好 [v] | 不友好 [x] |
|
|
24
52
|
|---|------|---------|--------|----------|
|
|
25
53
|
| 1 | **张量化 / Tensorization** | 能否表达为稠密张量运算,避免逐元素不规则控制流? | 批量张量代数 | 标量循环、数据相关分支 |
|
|
26
54
|
| 2 | **GEMM 可映射 / GEMM mappability** | 能否落到 矩阵乘 / batched GEMM / 卷积,吃满 Tensor Core? | 可写成 GEMM 链 | 无法矩阵化的不规则运算 |
|
|
@@ -59,9 +87,9 @@
|
|
|
59
87
|
|
|
60
88
|
| 组件 | 数学来源 | GPU 友好性 |
|
|
61
89
|
|------|---------|-----------|
|
|
62
|
-
| 热带门控 Tropical Gating | 热带半环分段线性 | 替代 Top-K:逐元素 max-plus——
|
|
63
|
-
| 胞腔层扩散 Cellular Sheaf Diffusion | 代数几何/拓扑(层、限制映射)| 每边低秩线性变换 = 小 GEMM
|
|
64
|
-
| Čech 上同调正则 | 代数拓扑(一阶上同调 H¹)|
|
|
65
|
-
| 低秩基底 KV 压缩(Plücker/Grassmannian 视角) | 射影几何 | 存基底而非 Plücker
|
|
90
|
+
| 热带门控 Tropical Gating | 热带半环分段线性 | 替代 Top-K:逐元素 max-plus——D1[v] / D2[x] 非 Tensor Core GEMM(落 CUDA core)/ D3[v] 仅逐 token 门控亚二次(min-plus matmul 则 APSP-hard 非亚二次);次可微,折点需 LogSumExp 软化(软化退回标准 softmax)|
|
|
91
|
+
| 胞腔层扩散 Cellular Sheaf Diffusion | 代数几何/拓扑(层、限制映射)| 每边低秩线性变换 = 小 GEMM(D2/D4)|
|
|
92
|
+
| Čech 上同调正则 | 代数拓扑(一阶上同调 H¹)| 局部、廉价;作幻觉的代数判据(D3/D8)|
|
|
93
|
+
| 低秩基底 KV 压缩(Plücker/Grassmannian 视角) | 射影几何 | 存基底而非 Plücker 坐标(后者低秩时反扩张);块摘要候选,压缩率/误差/吞吐需实测(D4)|
|
|
66
94
|
|
|
67
95
|
使用时不要把上表当作已验证结论。正确做法是把每个组件写进 testplan:证明或估计复杂度,测峰值显存和吞吐,检查 bf16/fp8 稳定性,确认能否落到 GEMM / batched GEMM / fused kernel。只有实测和推导都通过后,才标为 "math beautiful × GPU friendly"。
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Skill Index: Lens Library, Knowledge Base, Design Pattern Library, Workflow Examples
|
|
2
|
+
|
|
3
|
+
> This file is extracted from `../skills/math-research-activator/SKILL.en.md` for reference when the full catalogs are needed. SKILL.en.md retains condensed summaries with links to this file.
|
|
4
|
+
|
|
5
|
+
## Language Routing & Mixed-Input Rules (Full Version)
|
|
6
|
+
|
|
7
|
+
Language routing only determines "which language version to read" and "what language to respond in." It does NOT affect whether the math system triggers or the A/B/C/D/E scenario classification.
|
|
8
|
+
|
|
9
|
+
### Decision Rules
|
|
10
|
+
|
|
11
|
+
1. **Judge the natural-language frame first**
|
|
12
|
+
- If the user's sentence structure, verbs, and mood particles are primarily Chinese, treat as Chinese even if English technical terms are interspersed.
|
|
13
|
+
- E.g., "帮我 design 一个 attention" / "这个 loss 有没有理论问题" / "能不能用 manifold 做 routing" → Chinese.
|
|
14
|
+
|
|
15
|
+
2. **English technical terms do not count as English primary language**
|
|
16
|
+
- attention, loss, routing, embedding, manifold, operator, kernel, KV-cache, transformer, MoE, and similar AI/math/engineering terms are domain terms and do not trigger a switch to English.
|
|
17
|
+
|
|
18
|
+
3. **Code, paths, and formulas are excluded from language detection**
|
|
19
|
+
- File paths, function names, variable names, LaTeX formulas, and CLI arguments do not count toward language ratio.
|
|
20
|
+
|
|
21
|
+
4. **When primary language is unclear, follow the user's last clear language**
|
|
22
|
+
- If the CN/EN ratio is close and indeterminate, use the user's most recent unambiguous natural language.
|
|
23
|
+
- If there is no prior context, default to Chinese.
|
|
24
|
+
|
|
25
|
+
5. **Output language matches primary language**
|
|
26
|
+
- Chinese primary → read `../skills/math-research-activator/SKILL.md`, respond in Chinese, retaining necessary English terms.
|
|
27
|
+
- English primary → read `../skills/math-research-activator/SKILL.en.md`, respond in English.
|
|
28
|
+
- If the user explicitly requests "in English" / "in Chinese," follow the explicit request.
|
|
29
|
+
|
|
30
|
+
## Lens Library (15 Mathematical Perspectives)
|
|
31
|
+
|
|
32
|
+
Each lens answers: What is this perspective? What kinds of problems is it suited to diagnose? Which knowledge domains does it route to?
|
|
33
|
+
|
|
34
|
+
| Lens | File | Core Perspective |
|
|
35
|
+
|------|------|-----------------|
|
|
36
|
+
| Axiomatization | `../lenses/axiomatization.en.md` | Examine consistency/independence/completeness of assumptions |
|
|
37
|
+
| Duality | `../lenses/duality.en.md` | Transform to the dual space to expose constraints and invariants |
|
|
38
|
+
| Symmetry | `../lenses/symmetry.en.md` | Invariants and conservation laws under transformations |
|
|
39
|
+
| Spectral Decomposition | `../lenses/spectral.en.md` | Eigenvalues/singular values reveal dominant structure |
|
|
40
|
+
| Geometric | `../lenses/geometric.en.md` | Metric/curvature/spatial structure on manifolds |
|
|
41
|
+
| Projection & Decomposition | `../lenses/projection.en.md` | Orthogonal decomposition, subspace separation, conflict elimination |
|
|
42
|
+
| Variational | `../lenses/variational.en.md` | Constrained extrema, energy minimization |
|
|
43
|
+
| Local-to-Global | `../lenses/local-to-global.en.md` | Patching local properties into global ones, cohomological obstructions |
|
|
44
|
+
| Topological | `../lenses/topological.en.md` | Invariants under continuous deformation, connectedness, holes |
|
|
45
|
+
| Categorical | `../lenses/categorical.en.md` | Universal properties, functors, natural transformations |
|
|
46
|
+
| Perturbation | `../lenses/perturbation.en.md` | Propagation of small perturbations, stability, robustness |
|
|
47
|
+
| Causal | `../lenses/causal.en.md` | Correlation ≠ causation, interventions, counterfactuals |
|
|
48
|
+
| Game-Theoretic | `../lenses/game.en.md` | Multi-agent strategic interaction, equilibria, mechanism design |
|
|
49
|
+
| Probabilistic & Statistical | `../lenses/probabilistic.en.md` | Quantifying uncertainty, Bayesian updating |
|
|
50
|
+
| Algorithmic | `../lenses/algorithmic.en.md` | Complexity, feasibility, parallelizability |
|
|
51
|
+
|
|
52
|
+
## Knowledge Base (Organized by Mathematical Domain)
|
|
53
|
+
|
|
54
|
+
Each knowledge card answers: Minimal definition, core formulas, applicable problems, AI design translation, engineering feasibility, risks.
|
|
55
|
+
|
|
56
|
+
| Domain | Directory | Knowledge Cards |
|
|
57
|
+
|--------|-----------|----------------|
|
|
58
|
+
| Matrix Analysis | `../knowledge-base/matrix-analysis/` | projection, spectral-decomposition, low-rank-approximation, positive-semidefinite, matrix-perturbation |
|
|
59
|
+
| Optimization | `../knowledge-base/optimization/` | lagrangian-duality, convex-optimization, constrained-optimization, riemannian-optimization, proximal-method |
|
|
60
|
+
| Differential Geometry | `../knowledge-base/differential-geometry/` | manifold, tangent-space, metric-tensor, geodesic, curvature, connection |
|
|
61
|
+
| Lie Theory | `../knowledge-base/lie-theory/` | group-action, lie-group, lie-algebra, representation, equivariance |
|
|
62
|
+
| Topology | `../knowledge-base/topology/` | persistent-homology, euler-characteristic, fundamental-group |
|
|
63
|
+
| Probability & Information | `../knowledge-base/probability/` | concentration-inequality, entropy, kl-divergence, information-bottleneck, fisher-information |
|
|
64
|
+
| Information Geometry | `../knowledge-base/information-geometry/` | natural-gradient, fisher-metric |
|
|
65
|
+
|
|
66
|
+
## Design Pattern Library (Organized by AI Component)
|
|
67
|
+
|
|
68
|
+
Each design pattern answers: Mathematical origin, AI module form, implementable architecture, GPU feasibility, paper-level exposition, risks.
|
|
69
|
+
|
|
70
|
+
| Component Type | Directory | Patterns |
|
|
71
|
+
|---------------|-----------|----------|
|
|
72
|
+
| Attention | `../design-patterns/attention/` | projection-attention, spectral-attention, equivariant-attention, geometry-aware-attention, information-bottleneck-attention |
|
|
73
|
+
| Loss Functions | `../design-patterns/loss/` | orthogonality-loss, contrastive-loss, variational-loss, information-bottleneck-loss, constraint-penalty |
|
|
74
|
+
| Routing | `../design-patterns/routing/` | optimal-transport-routing, graph-routing, moe-routing, spectral-clustering-routing |
|
|
75
|
+
| Representation | `../design-patterns/representation/` | shared-private-decomposition, manifold-representation, equivariant-split, subspace-alignment |
|
|
76
|
+
| Compression | `../design-patterns/compression/` | low-rank-kv-cache, spectral-token-pruning, topology-preserving-compression, leverage-score-selection |
|
|
77
|
+
|
|
78
|
+
## Workflow Example
|
|
79
|
+
|
|
80
|
+
**User**: "Design a new KV Cache compression method that preserves long-range dependencies — I don't want to just do top-k."
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
Step 1 — Diagnosis: Scenario B (Mechanism Design)
|
|
84
|
+
Problem type: Sequence memory compression + information preservation + long-range structure
|
|
85
|
+
Core tension: Compressing token count vs. preserving long-range dependencies
|
|
86
|
+
|
|
87
|
+
Step 2 — Lens Selection:
|
|
88
|
+
1. Spectral Decomposition (preserve dominant subspace)
|
|
89
|
+
2. Information Theory (retain states with maximum mutual information)
|
|
90
|
+
3. Topological (preserve critical connectivity points of sequential structure)
|
|
91
|
+
|
|
92
|
+
Step 3 — Knowledge Query:
|
|
93
|
+
→ low-rank-approximation (Matrix Analysis)
|
|
94
|
+
→ leverage-score-selection (Design Pattern: compression)
|
|
95
|
+
→ information-bottleneck (Probability & Information)
|
|
96
|
+
|
|
97
|
+
Step 4 — Design Translation:
|
|
98
|
+
Candidate A: Spectral KV Compression (low-rank + leverage score)
|
|
99
|
+
Candidate B: Information-Preserving Cache (query sensitivity)
|
|
100
|
+
Candidate C: Topology-Preserving Cache (graph bridge-node retention)
|
|
101
|
+
|
|
102
|
+
Step 5 — Critic Review:
|
|
103
|
+
A is most GPU-friendly; B requires estimating future queries with inherent uncertainty; C has prohibitive graph construction cost
|
|
104
|
+
Recommendation: Prioritize A; use B as a lightweight gate
|
|
105
|
+
```
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Skill 索引:透镜库、知识库、设计模式库、工作流范例
|
|
2
|
+
|
|
3
|
+
> 本文件从 `../skills/math-research-activator/SKILL.md` 提取,供需要查阅完整目录时使用。SKILL.md 保留精简摘要和指向本文件的链接。
|
|
4
|
+
|
|
5
|
+
## 语言路由与混合输入规则(完整版)
|
|
6
|
+
|
|
7
|
+
语言路由只决定"读取哪个语言版本的说明"和"最终用什么语言回答",不参与数学系统是否触发、也不参与 A/B/C/D/E 场景判断。
|
|
8
|
+
|
|
9
|
+
### 判定规则
|
|
10
|
+
|
|
11
|
+
1. **先判断自然语言主框架**
|
|
12
|
+
- 如果用户的请求句式、动词、语气词主要是中文,即使夹杂英文技术词,也按中文处理。
|
|
13
|
+
- 例如:"帮我 design 一个 attention""这个 loss 有没有理论问题""能不能用 manifold 做 routing"均按中文处理。
|
|
14
|
+
|
|
15
|
+
2. **英文技术词不计入英文主语言**
|
|
16
|
+
- attention、loss、routing、embedding、manifold、operator、kernel、KV-cache、transformer、MoE 等 AI/数学/工程术语视为领域术语,不作为切换到英文的依据。
|
|
17
|
+
|
|
18
|
+
3. **代码、路径、公式不参与语言判定**
|
|
19
|
+
- 文件路径、函数名、变量名、LaTeX 公式、命令行参数不计入语言比例。
|
|
20
|
+
|
|
21
|
+
4. **主语言不明显时,沿用用户上一轮主要语言**
|
|
22
|
+
- 若中英文比例接近且无法判断,以用户最近一次明确使用的自然语言为准。
|
|
23
|
+
- 若没有上下文,默认中文。
|
|
24
|
+
|
|
25
|
+
5. **输出语言与主语言一致**
|
|
26
|
+
- 中文主语言 → 读取中文 `../skills/math-research-activator/SKILL.md`,用中文回答,保留必要英文术语。
|
|
27
|
+
- 英文主语言 → 读取 `../skills/math-research-activator/SKILL.en.md`,用英文回答。
|
|
28
|
+
- 用户明确要求"用英文/用中文"时,以用户显式要求为准。
|
|
29
|
+
|
|
30
|
+
## 透镜库(15 个数学视角)
|
|
31
|
+
|
|
32
|
+
每个透镜回答:这是什么视角?适合诊断什么问题?会路由到哪些知识域?
|
|
33
|
+
|
|
34
|
+
| 透镜 | 文件 | 核心视角 |
|
|
35
|
+
|------|------|---------|
|
|
36
|
+
| 公理化 | `../lenses/axiomatization.md` | 审查假设的相容性/独立性/完备性 |
|
|
37
|
+
| 对偶 | `../lenses/duality.md` | 转换到对偶空间暴露约束与不变量 |
|
|
38
|
+
| 对称性 | `../lenses/symmetry.md` | 变换下的不变量与守恒律 |
|
|
39
|
+
| 谱分解 | `../lenses/spectral.md` | 特征值/奇异值揭示主导结构 |
|
|
40
|
+
| 几何 | `../lenses/geometric.md` | 度量/曲率/流形上的空间结构 |
|
|
41
|
+
| 投影与分解 | `../lenses/projection.md` | 正交分解、子空间分离、冲突消除 |
|
|
42
|
+
| 变分 | `../lenses/variational.md` | 约束下极值、能量最小化 |
|
|
43
|
+
| 局部到整体 | `../lenses/local-to-global.md` | 局部性质拼接为全局、层上同调障碍 |
|
|
44
|
+
| 拓扑 | `../lenses/topological.md` | 连续变形不变量、连通性、空洞 |
|
|
45
|
+
| 范畴化 | `../lenses/categorical.md` | 泛性质、函子、自然变换 |
|
|
46
|
+
| 扰动 | `../lenses/perturbation.md` | 小扰动的传播、稳定性、鲁棒性 |
|
|
47
|
+
| 因果 | `../lenses/causal.md` | 相关≠因果、干预、反事实 |
|
|
48
|
+
| 博弈 | `../lenses/game.md` | 多方策略互动、均衡、机制设计 |
|
|
49
|
+
| 概率统计 | `../lenses/probabilistic.md` | 量化不确定性、贝叶斯更新 |
|
|
50
|
+
| 算法 | `../lenses/algorithmic.md` | 复杂度、可行性、并行性 |
|
|
51
|
+
|
|
52
|
+
## 知识库(按数学领域组织)
|
|
53
|
+
|
|
54
|
+
每个知识卡片回答:最小定义、核心公式、适用问题、AI 设计翻译、工程可行性、风险。
|
|
55
|
+
|
|
56
|
+
| 领域 | 目录 | 知识卡片 |
|
|
57
|
+
|------|------|---------|
|
|
58
|
+
| 矩阵分析 | `../knowledge-base/matrix-analysis/` | projection, spectral-decomposition, low-rank-approximation, positive-semidefinite, matrix-perturbation |
|
|
59
|
+
| 最优化 | `../knowledge-base/optimization/` | lagrangian-duality, convex-optimization, constrained-optimization, riemannian-optimization, proximal-method |
|
|
60
|
+
| 微分几何 | `../knowledge-base/differential-geometry/` | manifold, tangent-space, metric-tensor, geodesic, curvature, connection |
|
|
61
|
+
| 李理论 | `../knowledge-base/lie-theory/` | group-action, lie-group, lie-algebra, representation, equivariance |
|
|
62
|
+
| 拓扑 | `../knowledge-base/topology/` | persistent-homology, euler-characteristic, fundamental-group |
|
|
63
|
+
| 概率与信息 | `../knowledge-base/probability/` | concentration-inequality, entropy, kl-divergence, information-bottleneck, fisher-information |
|
|
64
|
+
| 信息几何 | `../knowledge-base/information-geometry/` | natural-gradient, fisher-metric |
|
|
65
|
+
|
|
66
|
+
## 设计模式库(按 AI 组件组织)
|
|
67
|
+
|
|
68
|
+
每个设计模式回答:数学来源、AI 模块形式、可实现结构、GPU 可行性、论文表述、风险。
|
|
69
|
+
|
|
70
|
+
| 组件类型 | 目录 | 模式 |
|
|
71
|
+
|---------|------|------|
|
|
72
|
+
| 注意力 | `../design-patterns/attention/` | projection-attention, spectral-attention, equivariant-attention, geometry-aware-attention, information-bottleneck-attention |
|
|
73
|
+
| 损失函数 | `../design-patterns/loss/` | orthogonality-loss, contrastive-loss, variational-loss, information-bottleneck-loss, constraint-penalty |
|
|
74
|
+
| 路由 | `../design-patterns/routing/` | optimal-transport-routing, graph-routing, moe-routing, spectral-clustering-routing |
|
|
75
|
+
| 表示 | `../design-patterns/representation/` | shared-private-decomposition, manifold-representation, equivariant-split, subspace-alignment |
|
|
76
|
+
| 压缩 | `../design-patterns/compression/` | low-rank-kv-cache, spectral-token-pruning, topology-preserving-compression, leverage-score-selection |
|
|
77
|
+
|
|
78
|
+
## 工作流范例
|
|
79
|
+
|
|
80
|
+
**用户**:"设计新的 KV Cache 压缩方法,保留长期依赖,不想只做 top-k"
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
第一步 诊断:场景 B(机制设计)
|
|
84
|
+
问题类型:序列记忆压缩 + 信息保留 + 长程结构
|
|
85
|
+
核心张力:压缩 token 数量 vs 不破坏长期依赖
|
|
86
|
+
|
|
87
|
+
第二步 透镜选择:
|
|
88
|
+
1. 谱分解(保留主导子空间)
|
|
89
|
+
2. 信息论(保留最大互信息状态)
|
|
90
|
+
3. 拓扑(保留序列结构关键连接点)
|
|
91
|
+
|
|
92
|
+
第三步 知识查询:
|
|
93
|
+
→ low-rank-approximation(矩阵分析)
|
|
94
|
+
→ leverage-score-selection(设计模式:压缩)
|
|
95
|
+
→ information-bottleneck(概率与信息)
|
|
96
|
+
|
|
97
|
+
第四步 设计翻译:
|
|
98
|
+
候选 A:Spectral KV Compression(低秩 + leverage score)
|
|
99
|
+
候选 B:Information-Preserving Cache(query sensitivity)
|
|
100
|
+
候选 C:Topology-Preserving Cache(图桥接节点保留)
|
|
101
|
+
|
|
102
|
+
第五步 Critic 审查:
|
|
103
|
+
A 最 GPU 友好,B 需估计未来 query 有不确定性,C 图构建成本过高
|
|
104
|
+
建议:优先 A,B 作轻量 gate
|
|
105
|
+
```
|