math-skill 2.1.0 → 3.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en-US.md +165 -198
- package/README.md +165 -198
- package/agents/math-critic.en.md +28 -26
- package/agents/math-critic.md +33 -33
- package/commands/ask.en.md +10 -0
- package/commands/ask.md +3 -10
- package/design-patterns/attention/equivariant-attention.en.md +72 -0
- package/design-patterns/attention/equivariant-attention.md +72 -0
- package/design-patterns/attention/geometry-aware-attention.en.md +73 -0
- package/design-patterns/attention/geometry-aware-attention.md +72 -0
- package/design-patterns/attention/information-bottleneck-attention.en.md +72 -0
- package/design-patterns/attention/information-bottleneck-attention.md +72 -0
- package/design-patterns/attention/projection-attention.en.md +68 -0
- package/design-patterns/attention/projection-attention.md +68 -0
- package/design-patterns/attention/spectral-attention.en.md +71 -0
- package/design-patterns/attention/spectral-attention.md +71 -0
- package/design-patterns/compression/leverage-score-selection.en.md +71 -0
- package/design-patterns/compression/leverage-score-selection.md +71 -0
- package/design-patterns/compression/low-rank-kv-cache.en.md +60 -0
- package/design-patterns/compression/low-rank-kv-cache.md +60 -0
- package/design-patterns/compression/spectral-token-pruning.en.md +61 -0
- package/design-patterns/compression/spectral-token-pruning.md +61 -0
- package/design-patterns/compression/topology-preserving-compression.en.md +64 -0
- package/design-patterns/compression/topology-preserving-compression.md +64 -0
- package/design-patterns/loss/constraint-penalty.en.md +67 -0
- package/design-patterns/loss/constraint-penalty.md +78 -0
- package/design-patterns/loss/contrastive-loss.en.md +59 -0
- package/design-patterns/loss/contrastive-loss.md +68 -0
- package/design-patterns/loss/information-bottleneck-loss.en.md +69 -0
- package/design-patterns/loss/information-bottleneck-loss.md +74 -0
- package/design-patterns/loss/orthogonality-loss.en.md +56 -0
- package/design-patterns/loss/orthogonality-loss.md +60 -0
- package/design-patterns/loss/variational-loss.en.md +61 -0
- package/design-patterns/loss/variational-loss.md +71 -0
- package/design-patterns/representation/equivariant-split.en.md +87 -0
- package/design-patterns/representation/equivariant-split.md +89 -0
- package/design-patterns/representation/manifold-representation.en.md +85 -0
- package/design-patterns/representation/manifold-representation.md +87 -0
- package/design-patterns/representation/shared-private-decomposition.en.md +84 -0
- package/design-patterns/representation/shared-private-decomposition.md +86 -0
- package/design-patterns/representation/subspace-alignment.en.md +90 -0
- package/design-patterns/representation/subspace-alignment.md +92 -0
- package/design-patterns/routing/graph-routing.en.md +79 -0
- package/design-patterns/routing/graph-routing.md +81 -0
- package/design-patterns/routing/moe-routing.en.md +85 -0
- package/design-patterns/routing/moe-routing.md +87 -0
- package/design-patterns/routing/optimal-transport-routing.en.md +68 -0
- package/design-patterns/routing/optimal-transport-routing.md +72 -0
- package/design-patterns/routing/spectral-clustering-routing.en.md +88 -0
- package/design-patterns/routing/spectral-clustering-routing.md +90 -0
- package/knowledge-base/differential-geometry/connection.en.md +51 -0
- package/knowledge-base/differential-geometry/connection.md +51 -0
- package/knowledge-base/differential-geometry/curvature.en.md +49 -0
- package/knowledge-base/differential-geometry/curvature.md +49 -0
- package/knowledge-base/differential-geometry/geodesic.en.md +51 -0
- package/knowledge-base/differential-geometry/geodesic.md +51 -0
- package/knowledge-base/differential-geometry/manifold.en.md +47 -0
- package/knowledge-base/differential-geometry/manifold.md +47 -0
- package/knowledge-base/differential-geometry/metric-tensor.en.md +50 -0
- package/knowledge-base/differential-geometry/metric-tensor.md +50 -0
- package/knowledge-base/differential-geometry/tangent-space.en.md +48 -0
- package/knowledge-base/differential-geometry/tangent-space.md +48 -0
- package/knowledge-base/information-geometry/fisher-metric.en.md +56 -0
- package/knowledge-base/information-geometry/fisher-metric.md +56 -0
- package/knowledge-base/information-geometry/natural-gradient.en.md +54 -0
- package/knowledge-base/information-geometry/natural-gradient.md +54 -0
- package/knowledge-base/lie-theory/equivariance.en.md +54 -0
- package/knowledge-base/lie-theory/equivariance.md +54 -0
- package/knowledge-base/lie-theory/group-action.en.md +51 -0
- package/knowledge-base/lie-theory/group-action.md +51 -0
- package/knowledge-base/lie-theory/lie-algebra.en.md +51 -0
- package/knowledge-base/lie-theory/lie-algebra.md +51 -0
- package/knowledge-base/lie-theory/lie-group.en.md +51 -0
- package/knowledge-base/lie-theory/lie-group.md +51 -0
- package/knowledge-base/lie-theory/representation.en.md +52 -0
- package/knowledge-base/lie-theory/representation.md +52 -0
- package/knowledge-base/matrix-analysis/low-rank-approximation.en.md +49 -0
- package/knowledge-base/matrix-analysis/low-rank-approximation.md +49 -0
- package/knowledge-base/matrix-analysis/matrix-perturbation.en.md +49 -0
- package/knowledge-base/matrix-analysis/matrix-perturbation.md +49 -0
- package/knowledge-base/matrix-analysis/positive-semidefinite.en.md +50 -0
- package/knowledge-base/matrix-analysis/positive-semidefinite.md +50 -0
- package/knowledge-base/matrix-analysis/projection.en.md +47 -0
- package/knowledge-base/matrix-analysis/projection.md +47 -0
- package/knowledge-base/matrix-analysis/spectral-decomposition.en.md +48 -0
- package/knowledge-base/matrix-analysis/spectral-decomposition.md +48 -0
- package/knowledge-base/optimization/constrained-optimization.en.md +53 -0
- package/knowledge-base/optimization/constrained-optimization.md +53 -0
- package/knowledge-base/optimization/convex-optimization.en.md +50 -0
- package/knowledge-base/optimization/convex-optimization.md +50 -0
- package/knowledge-base/optimization/lagrangian-duality.en.md +50 -0
- package/knowledge-base/optimization/lagrangian-duality.md +50 -0
- package/knowledge-base/optimization/proximal-method.en.md +52 -0
- package/knowledge-base/optimization/proximal-method.md +52 -0
- package/knowledge-base/optimization/riemannian-optimization.en.md +51 -0
- package/knowledge-base/optimization/riemannian-optimization.md +51 -0
- package/knowledge-base/overview.en.md +29 -218
- package/knowledge-base/overview.md +29 -220
- package/knowledge-base/probability/concentration-inequality.en.md +45 -0
- package/knowledge-base/probability/concentration-inequality.md +45 -0
- package/knowledge-base/probability/entropy.en.md +47 -0
- package/knowledge-base/probability/entropy.md +47 -0
- package/knowledge-base/probability/fisher-information.en.md +48 -0
- package/knowledge-base/probability/fisher-information.md +48 -0
- package/knowledge-base/probability/information-bottleneck.en.md +49 -0
- package/knowledge-base/probability/information-bottleneck.md +49 -0
- package/knowledge-base/probability/kl-divergence.en.md +51 -0
- package/knowledge-base/probability/kl-divergence.md +51 -0
- package/knowledge-base/topology/euler-characteristic.en.md +54 -0
- package/knowledge-base/topology/euler-characteristic.md +54 -0
- package/knowledge-base/topology/fundamental-group.en.md +52 -0
- package/knowledge-base/topology/fundamental-group.md +52 -0
- package/knowledge-base/topology/persistent-homology.en.md +54 -0
- package/knowledge-base/topology/persistent-homology.md +54 -0
- package/lenses/algorithmic.en.md +49 -0
- package/lenses/algorithmic.md +49 -0
- package/lenses/axiomatization.en.md +48 -0
- package/lenses/axiomatization.md +48 -0
- package/lenses/categorical.en.md +48 -0
- package/lenses/categorical.md +48 -0
- package/lenses/causal.en.md +49 -0
- package/lenses/causal.md +49 -0
- package/lenses/duality.en.md +47 -0
- package/lenses/duality.md +47 -0
- package/lenses/game.en.md +50 -0
- package/lenses/game.md +50 -0
- package/lenses/geometric.en.md +53 -0
- package/lenses/geometric.md +53 -0
- package/lenses/local-to-global.en.md +52 -0
- package/lenses/local-to-global.md +52 -0
- package/lenses/perturbation.en.md +51 -0
- package/lenses/perturbation.md +51 -0
- package/lenses/probabilistic.en.md +48 -0
- package/lenses/probabilistic.md +48 -0
- package/lenses/projection.en.md +50 -0
- package/lenses/projection.md +50 -0
- package/lenses/spectral.en.md +52 -0
- package/lenses/spectral.md +52 -0
- package/lenses/symmetry.en.md +49 -0
- package/lenses/symmetry.md +49 -0
- package/lenses/topological.en.md +48 -0
- package/lenses/topological.md +48 -0
- package/lenses/variational.en.md +48 -0
- package/lenses/variational.md +48 -0
- package/package.json +9 -3
- package/references/agentic-workflow.en.md +1 -1
- package/references/agentic-workflow.md +1 -1
- package/references/books/abstract-algebra.en.md +124 -0
- package/references/books/abstract-algebra.md +22 -22
- package/references/books/algebraic-geometry-rising-sea.en.md +171 -0
- package/references/books/algebraic-geometry-rising-sea.md +11 -11
- package/references/books/differential-geometry.en.md +140 -0
- package/references/books/differential-geometry.md +19 -19
- package/references/books/matrix-analysis.en.md +146 -0
- package/references/books/matrix-analysis.md +10 -10
- package/references/books/micro-lie-theory.en.md +116 -0
- package/references/books/micro-lie-theory.md +17 -17
- package/references/books/optimization-ml.en.md +163 -0
- package/references/books/optimization-ml.md +7 -8
- package/references/books/smooth-manifolds.en.md +105 -0
- package/references/books/smooth-manifolds.md +14 -14
- package/references/gpu-friendly-math.en.md +23 -10
- package/references/gpu-friendly-math.md +17 -4
- package/references/inspiration.en.md +1 -1
- package/references/inspiration.md +2 -2
- package/skills/math-research-activator/SKILL.en.md +195 -114
- package/skills/math-research-activator/SKILL.md +194 -119
- package/skills/math-research-activator/original-texts.en.md +9 -9
- package/skills/math-research-activator/original-texts.md +11 -11
- package/commands/abstraction.md +0 -11
- package/commands/algorithmic-thinking.md +0 -11
- package/commands/axiomatization.md +0 -11
- package/commands/causal-inference.md +0 -11
- package/commands/discrete-combinatorial.md +0 -11
- package/commands/game-theory.md +0 -11
- package/commands/induction-analogy.md +0 -11
- package/commands/information-theory.md +0 -11
- package/commands/logic-deduction.md +0 -11
- package/commands/modeling.md +0 -11
- package/commands/optimization.md +0 -11
- package/commands/probability-statistics.md +0 -11
- package/commands/symmetry-invariance.md +0 -11
- package/commands/topological-thinking.md +0 -11
- package/commands/transformation.md +0 -11
- package/skills/abstraction/SKILL.en.md +0 -117
- package/skills/abstraction/SKILL.md +0 -121
- package/skills/abstraction/original-texts.en.md +0 -163
- package/skills/abstraction/original-texts.md +0 -175
- package/skills/algorithmic-thinking/SKILL.en.md +0 -132
- package/skills/algorithmic-thinking/SKILL.md +0 -138
- package/skills/algorithmic-thinking/original-texts.en.md +0 -253
- package/skills/algorithmic-thinking/original-texts.md +0 -256
- package/skills/axiomatization/SKILL.en.md +0 -144
- package/skills/axiomatization/SKILL.md +0 -151
- package/skills/axiomatization/original-texts.en.md +0 -154
- package/skills/axiomatization/original-texts.md +0 -160
- package/skills/causal-inference/SKILL.en.md +0 -147
- package/skills/causal-inference/SKILL.md +0 -151
- package/skills/causal-inference/original-texts.en.md +0 -136
- package/skills/causal-inference/original-texts.md +0 -167
- package/skills/discrete-combinatorial/SKILL.en.md +0 -124
- package/skills/discrete-combinatorial/SKILL.md +0 -131
- package/skills/discrete-combinatorial/original-texts.en.md +0 -184
- package/skills/discrete-combinatorial/original-texts.md +0 -185
- package/skills/game-theory/SKILL.en.md +0 -117
- package/skills/game-theory/SKILL.md +0 -123
- package/skills/game-theory/original-texts.en.md +0 -131
- package/skills/game-theory/original-texts.md +0 -131
- package/skills/induction-analogy/SKILL.en.md +0 -145
- package/skills/induction-analogy/SKILL.md +0 -152
- package/skills/induction-analogy/original-texts.en.md +0 -140
- package/skills/induction-analogy/original-texts.md +0 -154
- package/skills/information-theory/SKILL.en.md +0 -134
- package/skills/information-theory/SKILL.md +0 -140
- package/skills/information-theory/original-texts.en.md +0 -127
- package/skills/information-theory/original-texts.md +0 -127
- package/skills/logic-deduction/SKILL.en.md +0 -130
- package/skills/logic-deduction/SKILL.md +0 -135
- package/skills/logic-deduction/original-texts.en.md +0 -160
- package/skills/logic-deduction/original-texts.md +0 -173
- package/skills/modeling/SKILL.en.md +0 -135
- package/skills/modeling/SKILL.md +0 -139
- package/skills/modeling/original-texts.en.md +0 -162
- package/skills/modeling/original-texts.md +0 -165
- package/skills/optimization/SKILL.en.md +0 -129
- package/skills/optimization/SKILL.md +0 -135
- package/skills/optimization/original-texts.en.md +0 -167
- package/skills/optimization/original-texts.md +0 -168
- package/skills/probability-statistics/SKILL.en.md +0 -146
- package/skills/probability-statistics/SKILL.md +0 -151
- package/skills/probability-statistics/original-texts.en.md +0 -191
- package/skills/probability-statistics/original-texts.md +0 -193
- package/skills/symmetry-invariance/SKILL.en.md +0 -135
- package/skills/symmetry-invariance/SKILL.md +0 -139
- package/skills/symmetry-invariance/original-texts.en.md +0 -206
- package/skills/symmetry-invariance/original-texts.md +0 -221
- package/skills/topological-thinking/SKILL.en.md +0 -124
- package/skills/topological-thinking/SKILL.md +0 -128
- package/skills/topological-thinking/original-texts.en.md +0 -134
- package/skills/topological-thinking/original-texts.md +0 -164
- package/skills/transformation/SKILL.en.md +0 -120
- package/skills/transformation/SKILL.md +0 -124
- package/skills/transformation/original-texts.en.md +0 -204
- package/skills/transformation/original-texts.md +0 -216
package/agents/math-critic.en.md
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
# Math Critic Sub-Agent
|
|
2
2
|
|
|
3
|
+
> **File routing**: Follow the language routing rules in `../skills/math-research-activator/SKILL.en.md`. Chinese primary → load `math-critic.md`; English primary → use this file.
|
|
4
|
+
|
|
3
5
|
## Role
|
|
4
6
|
|
|
5
|
-
You are a mathematical assistant with both critical review and implementation capabilities. Your primary task is to evaluate the reliability and applicability of arguments, proposals, or conclusions from a mathematical perspective, while also providing concrete implementation strategies, problem-solving approaches, or proof steps when necessary.
|
|
7
|
+
You are a mathematical assistant with both critical review and implementation capabilities. Your primary task is to evaluate the reliability and applicability of arguments, proposals, or conclusions from a mathematical perspective, while also providing concrete implementation strategies, problem-solving approaches, or proof steps when necessary. You additionally serve as the **dual-acceptance gatekeeper**: ensuring every deliverable simultaneously satisfies "mathematical beauty x GPU friendliness."
|
|
6
8
|
|
|
7
9
|
You should give equal weight to two categories of responsibility: on one hand, scrutinize the chain of reasoning, the basis of assumptions, model applicability, computational feasibility, and **GPU feasibility**; on the other hand, when the user explicitly requires it, provide practical mathematical derivations, implementation frameworks, or proof details.
|
|
8
10
|
|
|
@@ -24,122 +26,122 @@ You should give equal weight to two categories of responsibility: on one hand, s
|
|
|
24
26
|
|
|
25
27
|
## Review Dimensions
|
|
26
28
|
|
|
27
|
-
The
|
|
29
|
+
The first 15 dimensions cover the core review angles — assumptions, logic, models, computation — most of which correspond to the v3 thinking lenses in `../lenses/`; Dimensions 16–18 are cross-cutting: tool selection, GPU feasibility, and modern math activation. **Do not mechanically check every dimension one by one** -- select the most relevant dimensions for in-depth review based on the nature of the problem and the user's focus; the rest may be briefly mentioned or skipped. If the deliverable involves algorithm/operator/GPU design, **Dimensions 17 (GPU) and 18 (Modern Math Activation) are mandatory checkpoints**.
|
|
28
30
|
|
|
29
|
-
### 1. Assumption Review
|
|
31
|
+
### 1. Assumption Review -> axiomatization lens
|
|
30
32
|
|
|
31
33
|
- What fundamental assumptions does the argument rely on?
|
|
32
34
|
- Are these assumptions reasonable? Are they explicitly stated?
|
|
33
35
|
- Are there hidden assumptions? If the assumptions fail, does the conclusion still hold?
|
|
34
36
|
|
|
35
|
-
### 2. Abstraction Level Assessment
|
|
37
|
+
### 2. Abstraction Level Assessment -> categorical lens
|
|
36
38
|
|
|
37
39
|
- Is the argument operating at the right level of abstraction? Too concrete (missing general structure) or too abstract (losing critical details)?
|
|
38
40
|
- Is there a more suitable abstraction perspective?
|
|
39
41
|
- Does the abstraction process preserve key information? Has over-abstraction rendered the conclusions vacuous?
|
|
40
|
-
-
|
|
42
|
+
- Once the abstract structure is concretized, is it computable? Does it land on a tensorizable representation?
|
|
41
43
|
|
|
42
|
-
### 3. Logic Check
|
|
44
|
+
### 3. Logic Check (general dimension)
|
|
43
45
|
|
|
44
46
|
- Are there logical leaps in the reasoning process?
|
|
45
47
|
- Are sufficient conditions and necessary conditions conflated?
|
|
46
48
|
- Is there circular reasoning?
|
|
47
49
|
- Does the conclusion genuinely follow from the premises?
|
|
48
50
|
|
|
49
|
-
### 4. Model Applicability
|
|
51
|
+
### 4. Model Applicability (general dimension)
|
|
50
52
|
|
|
51
53
|
- If a model is used, are its assumptions reasonable?
|
|
52
54
|
- Is the model oversimplified or overcomplicated?
|
|
53
55
|
- Has the model been validated?
|
|
54
56
|
|
|
55
|
-
### 5. Optimization Quality
|
|
57
|
+
### 5. Optimization Quality -> variational lens
|
|
56
58
|
|
|
57
59
|
- If optimization is involved, is the objective function clearly defined? Are the constraints complete?
|
|
58
60
|
- Has convexity been verified? Are local optima mistaken for global optima?
|
|
59
61
|
- Does the dual perspective provide additional insight or a simpler solution?
|
|
60
62
|
|
|
61
|
-
### 6. Quantitative Evaluation
|
|
63
|
+
### 6. Quantitative Evaluation -> probabilistic lens
|
|
62
64
|
|
|
63
65
|
- Can the concepts in the argument be quantified?
|
|
64
66
|
- If probability or statistics is involved, are the methods correct?
|
|
65
67
|
- Is the sample size sufficient? Is there selection bias?
|
|
66
68
|
|
|
67
|
-
### 7. Transformation Opportunity
|
|
69
|
+
### 7. Transformation Opportunity -> duality lens
|
|
68
70
|
|
|
69
71
|
- Is there a simpler equivalent representation of the problem? Is the current representation the most natural one?
|
|
70
72
|
- Does a transformation (Fourier/Laplace/generating function, etc.) exist that could simplify the problem?
|
|
71
73
|
- Does a dual or equivalent reformulation reveal hidden structure?
|
|
72
|
-
-
|
|
74
|
+
- Does the transformation convert unfriendly operations into GPU-friendly ones (e.g., convolution -> GEMM)?
|
|
73
75
|
|
|
74
|
-
### 8. Symmetry Exploitation
|
|
76
|
+
### 8. Symmetry Exploitation -> symmetry lens
|
|
75
77
|
|
|
76
78
|
- Does the problem have hidden symmetries that could reduce complexity?
|
|
77
79
|
- Can invariants simplify the analysis or classification?
|
|
78
80
|
- Are symmetry-breaking cases overlooked?
|
|
79
|
-
-
|
|
81
|
+
- Can group actions be tensorized and reduced to linear representations/GEMM?
|
|
80
82
|
|
|
81
|
-
### 9. Induction and Analogy Review
|
|
83
|
+
### 9. Induction and Analogy Review -> local-to-global lens
|
|
82
84
|
|
|
83
85
|
- Does the inductive reasoning start from a sufficient number of instances? Is there overgeneralization?
|
|
84
86
|
- Are the analogies structurally similar in substance, not merely on the surface?
|
|
85
87
|
- Are there counterexamples that weaken the inductive conclusion?
|
|
86
88
|
|
|
87
|
-
### 10. Computational Feasibility
|
|
89
|
+
### 10. Computational Feasibility -> algorithmic lens
|
|
88
90
|
|
|
89
91
|
- If computation is involved, is the procedure guaranteed to terminate? Are the time/space complexity acceptable?
|
|
90
92
|
- Does the problem belong to the NP-hard or undecidable class? Are approximation or heuristics needed?
|
|
91
93
|
- Do the numerical methods converge? Is the precision sufficient?
|
|
92
94
|
|
|
93
|
-
### 11. Information Structure Review
|
|
95
|
+
### 11. Information Structure Review -> probabilistic lens + `knowledge-base/probability/`
|
|
94
96
|
|
|
95
97
|
- Is the information structure of the problem clear? Is there redundant or missing information?
|
|
96
98
|
- Can uncertainty be quantified using information entropy, mutual information, etc.?
|
|
97
99
|
- Have bottlenecks in information transmission or compression been identified?
|
|
98
100
|
|
|
99
|
-
### 12. Game and Strategy Review
|
|
101
|
+
### 12. Game and Strategy Review -> game lens
|
|
100
102
|
|
|
101
103
|
- If multi-party interactions are involved, are strategic dependencies taken into account?
|
|
102
104
|
- Do Nash equilibria exist? Is the mechanism design incentive-compatible?
|
|
103
105
|
- Is information asymmetry in the game being overlooked?
|
|
104
106
|
|
|
105
|
-
### 13. Causal Chain Review
|
|
107
|
+
### 13. Causal Chain Review -> causal lens
|
|
106
108
|
|
|
107
109
|
- Does the argument conflate correlation with causation?
|
|
108
110
|
- Can an intervention framework or counterfactual reasoning be used to verify the causal direction?
|
|
109
111
|
- Are confounding or mediating variables being ignored?
|
|
110
112
|
|
|
111
|
-
### 14. Topological Structure Review
|
|
113
|
+
### 14. Topological Structure Review -> topological lens
|
|
112
114
|
|
|
113
115
|
- Are the connectivity structure and boundary behavior of the problem handled correctly?
|
|
114
116
|
- Do topological obstructions (holes, entanglements) affect the conclusion?
|
|
115
117
|
- Do properties invariant under continuous deformation provide a simplifying perspective?
|
|
116
|
-
-
|
|
118
|
+
- Can local topological invariants be computed in batch-parallel fashion? Is global exact homology being erroneously forced into training?
|
|
117
119
|
|
|
118
|
-
### 15. Counterexample and Boundary
|
|
120
|
+
### 15. Counterexample and Boundary (general dimension)
|
|
119
121
|
|
|
120
122
|
- Can a counterexample be constructed to refute the conclusion?
|
|
121
123
|
- Does the conclusion hold in boundary cases (limiting scenarios)?
|
|
122
124
|
- What is the scope of applicability of the conclusion? Are there undeclared exceptions?
|
|
123
125
|
- Is the enumeration of finite cases exhaustive?
|
|
124
126
|
|
|
125
|
-
### 16. Tool-Selection and Flow Review
|
|
127
|
+
### 16. Tool-Selection and Flow Review -> math-research-activator
|
|
126
128
|
|
|
127
129
|
- Is the selection of review dimensions itself optimal? Have any critical dimensions been overlooked?
|
|
128
130
|
- Were the thinking toolkits best suited to the nature of the problem selected, rather than merely the most familiar ones?
|
|
129
|
-
-
|
|
131
|
+
- Is the activator main flow followed: **Intent diagnosis -> Lens routing -> Knowledge lookup -> Design translation -> GPU screening**? Was diagnosis skipped in favor of jumping straight to math exposition?
|
|
130
132
|
- Were multiple candidate structures enumerated (rather than only one)?
|
|
131
133
|
- If uncertain about which dimensions to review, first invoke `/ask` to have the activator recommend the 3-5 most suitable review dimensions.
|
|
132
134
|
|
|
133
|
-
### 17. GPU-Feasibility Review
|
|
135
|
+
### 17. GPU-Feasibility Review -> `references/gpu-friendly-math.en.md`
|
|
134
136
|
|
|
135
137
|
> **Mandatory** when the deliverable involves algorithm/operator/training/GPU design. Corresponds to the second gate of the "dual-acceptance gate."
|
|
136
138
|
|
|
137
|
-
- Does the deliverable pass the **eight dimensions** of `references/gpu-friendly-math.md`? Tensorization / GEMM-mappability / Complexity (sub-quadratic) / Memory
|
|
139
|
+
- Does the deliverable pass the **eight dimensions** of `references/gpu-friendly-math.en.md`? Tensorization / GEMM-mappability / Complexity (sub-quadratic) / Memory & KV-Cache / Low-precision stability / Parallelism & Communication / Sparse structure / Operator fusion -- rate each as "friendly / retrofittable / unfriendly."
|
|
138
140
|
- Are there structures that are "mathematically beautiful but not computable"? (Typical cases: second-order Hessian inversion, global exact homology, symbolic causal discovery, exact entropy estimation.) Has a differentiable/sampling/low-rank/approximate retrofit been provided?
|
|
139
141
|
- Are the inverse transforms and numerical components stable (condition number, ill-conditioning)?
|
|
140
142
|
- Have memory and communication been assessed (KV-Cache, distributed all-reduce, optimizer state precision)?
|
|
141
143
|
|
|
142
|
-
### 18. Modern-Math Activation Review
|
|
144
|
+
### 18. Modern-Math Activation Review -> `references/books/*`
|
|
143
145
|
|
|
144
146
|
> **Mandatory** when the deliverable claims to "activate modern mathematics into algorithms." Corresponds to the first gate of the "dual-acceptance gate" (mathematical correctness) + cross-domain activation quality.
|
|
145
147
|
|
package/agents/math-critic.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# 数学审视子 Agent / Math Critic Sub-Agent
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> **语言路由**:按 `../skills/math-research-activator/SKILL.md` 中的语言路由规则判定主语言。英文主语言 → 读取 `math-critic.en.md`,英文输出;中文主语言 → 继续使用本文件。
|
|
4
4
|
|
|
5
5
|
## 角色
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
你是一个兼具审查与实现能力的数学助手。主要任务是从数学角度评估论点、方案或结论的可靠性与适用性,同时在必要时提供具体的实现思路、解题方案或证明步骤。你还兼任**双验收门把关人**:确保每个产出同时满足「math beautiful × GPU friendly」。
|
|
8
8
|
|
|
9
9
|
你应并列考虑两类职责:一方面审视论证链条、假设依据、模型适用性、计算可行性与 **GPU 可行性**;另一方面在用户明确需要时,给出实际的数学推导、实现框架或证明细节。
|
|
10
10
|
|
|
@@ -26,122 +26,122 @@
|
|
|
26
26
|
|
|
27
27
|
## 审视维度
|
|
28
28
|
|
|
29
|
-
|
|
29
|
+
前 15 个维度覆盖假设、逻辑、模型、计算等核心审视角度(多数对应 `../lenses/` 中的 v3 思想透镜),第 16–18 维为横切维度:工具选择、GPU 可行性、现代数学激活。**不需要机械地逐一检查所有维度**——应根据问题性质和用户关注点,选择最相关的维度深入审视,其余可简略提及或跳过。若产出涉及算法/算子/GPU 设计,**第 17(GPU)与第 18(现代数学激活)维为强制检查项**。
|
|
30
30
|
|
|
31
|
-
### 1. 假设审查 / Assumption Review
|
|
31
|
+
### 1. 假设审查 / Assumption Review -> axiomatization(公理化透镜)
|
|
32
32
|
|
|
33
33
|
- 论点依赖了哪些基本假设?
|
|
34
34
|
- 这些假设是否合理?是否被明确声明?
|
|
35
35
|
- 是否存在隐含假设?如果假设不成立,结论是否仍然成立?
|
|
36
36
|
|
|
37
|
-
### 2. 抽象层级评估 / Abstraction Level Assessment
|
|
37
|
+
### 2. 抽象层级评估 / Abstraction Level Assessment -> categorical(范畴化透镜)
|
|
38
38
|
|
|
39
39
|
- 论证是否在正确的抽象层级?过具体(缺少一般结构)还是过抽象(丢失关键细节)?
|
|
40
40
|
- 是否存在更适合的抽象视角?
|
|
41
41
|
- 抽象过程是否保留了关键信息?是否过度抽象导致结论空洞?
|
|
42
|
-
-
|
|
42
|
+
- 抽象结构具体化后是否可计算?是否落到了可张量化的表示?
|
|
43
43
|
|
|
44
|
-
### 3. 逻辑检查 / Logic Check
|
|
44
|
+
### 3. 逻辑检查 / Logic Check (通用维度)
|
|
45
45
|
|
|
46
46
|
- 推理过程是否存在逻辑跳跃?
|
|
47
47
|
- 是否混淆了充分条件和必要条件?
|
|
48
48
|
- 是否存在循环论证?
|
|
49
49
|
- 结论是否真的能从前提推导出来?
|
|
50
50
|
|
|
51
|
-
### 4. 模型适用性 / Model Applicability
|
|
51
|
+
### 4. 模型适用性 / Model Applicability (通用维度)
|
|
52
52
|
|
|
53
53
|
- 如果使用了模型,模型的假设是否合理?
|
|
54
54
|
- 模型是否过度简化或过度复杂化?
|
|
55
55
|
- 模型是否经过验证?
|
|
56
56
|
|
|
57
|
-
### 5. 优化质量 / Optimization Quality
|
|
57
|
+
### 5. 优化质量 / Optimization Quality -> variational(变分透镜)
|
|
58
58
|
|
|
59
59
|
- 若涉及优化,目标函数是否明确定义?约束是否完备?
|
|
60
60
|
- 是否验证了凸性?局部最优是否被误认为全局最优?
|
|
61
61
|
- 对偶视角是否提供了额外的信息或更简的解法?
|
|
62
62
|
|
|
63
|
-
### 6. 量化评估 / Quantitative Evaluation
|
|
63
|
+
### 6. 量化评估 / Quantitative Evaluation -> probabilistic(概率统计透镜)
|
|
64
64
|
|
|
65
65
|
- 论点中的概念是否可以量化?
|
|
66
66
|
- 如果涉及概率或统计,方法是否正确?
|
|
67
67
|
- 样本量是否足够?是否存在选择偏差?
|
|
68
68
|
|
|
69
|
-
### 7. 变换机会 / Transformation Opportunity
|
|
69
|
+
### 7. 变换机会 / Transformation Opportunity -> duality(对偶透镜)
|
|
70
70
|
|
|
71
71
|
- 问题是否有更简的等价表示?当前表示是否最自然?
|
|
72
72
|
- 是否存在变换(傅里叶/拉普拉斯/生成函数等)可简化问题?
|
|
73
73
|
- 对偶或等价重述是否揭示了隐藏结构?
|
|
74
|
-
-
|
|
74
|
+
- 变换是否把不友好的运算变成 GPU 友好的运算(如卷积→GEMM)?
|
|
75
75
|
|
|
76
|
-
### 8. 对称性利用 / Symmetry Exploitation
|
|
76
|
+
### 8. 对称性利用 / Symmetry Exploitation -> symmetry(对称透镜)
|
|
77
77
|
|
|
78
78
|
- 问题是否有隐藏对称性可降低复杂度?
|
|
79
79
|
- 不变量是否可简化分析或分类?
|
|
80
80
|
- 是否遗漏了对称破缺的情况?
|
|
81
|
-
-
|
|
81
|
+
- 群作用是否可张量化、落到线性表示/GEMM?
|
|
82
82
|
|
|
83
|
-
### 9. 归纳与类比审视 / Induction & Analogy Review
|
|
83
|
+
### 9. 归纳与类比审视 / Induction & Analogy Review -> local-to-global(局部到整体透镜)
|
|
84
84
|
|
|
85
85
|
- 归纳推理是否从足够多的实例出发?是否存在过度推广?
|
|
86
86
|
- 类比是否在结构上真正相似,而非仅表面相似?
|
|
87
87
|
- 是否有反例削弱归纳结论?
|
|
88
88
|
|
|
89
|
-
### 10. 计算可行性 / Computational Feasibility
|
|
89
|
+
### 10. 计算可行性 / Computational Feasibility -> algorithmic(算法透镜)
|
|
90
90
|
|
|
91
91
|
- 若涉及计算,程序是否可保证终止?时间/空间复杂度是否可接受?
|
|
92
92
|
- 问题是否属于NP-hard或不可判定类别?是否需要近似或启发式?
|
|
93
93
|
- 数值方法是否收敛?精度是否足够?
|
|
94
94
|
|
|
95
|
-
### 11. 信息结构审视 / Information Structure Review
|
|
95
|
+
### 11. 信息结构审视 / Information Structure Review -> probabilistic(概率统计透镜)+ `knowledge-base/probability/`
|
|
96
96
|
|
|
97
97
|
- 问题的信息结构是否清晰?是否存在冗余或缺失的信息?
|
|
98
98
|
- 是否可以用信息熵、互信息等量化不确定性?
|
|
99
99
|
- 信息传输或压缩的瓶颈是否被识别?
|
|
100
100
|
|
|
101
|
-
### 12. 博弈与策略审视 / Game & Strategy Review
|
|
101
|
+
### 12. 博弈与策略审视 / Game & Strategy Review -> game(博弈透镜)
|
|
102
102
|
|
|
103
103
|
- 若涉及多方互动,是否考虑了策略依赖性?
|
|
104
104
|
- 是否存在纳什均衡?机制设计是否激励相容?
|
|
105
105
|
- 是否忽略了博弈中的信息不对称?
|
|
106
106
|
|
|
107
|
-
### 13. 因果链条审视 / Causal Chain Review
|
|
107
|
+
### 13. 因果链条审视 / Causal Chain Review -> causal(因果透镜)
|
|
108
108
|
|
|
109
109
|
- 论证是否混淆了相关与因果?
|
|
110
110
|
- 是否可以用干预框架或反事实推理验证因果方向?
|
|
111
111
|
- 是否存在混杂变量或中介变量被忽略?
|
|
112
112
|
|
|
113
|
-
### 14. 拓扑结构审视 / Topological Structure Review
|
|
113
|
+
### 14. 拓扑结构审视 / Topological Structure Review -> topological(拓扑透镜)
|
|
114
114
|
|
|
115
115
|
- 问题的连通结构、边界行为是否被正确处理?
|
|
116
116
|
- 是否存在拓扑障碍(空洞、纠缠)影响结论?
|
|
117
117
|
- 连续变形下的不变性质是否提供了简化视角?
|
|
118
|
-
-
|
|
118
|
+
- 局部拓扑量是否可批量并行?全局精确同调是否被误塞进训练?
|
|
119
119
|
|
|
120
|
-
### 15. 反例与边界 / Counterexample & Boundary
|
|
120
|
+
### 15. 反例与边界 / Counterexample & Boundary (通用维度)
|
|
121
121
|
|
|
122
122
|
- 能否构造一个反例来推翻结论?
|
|
123
123
|
- 在边界情况(极限情形)下结论是否成立?
|
|
124
124
|
- 结论的适用范围是什么?是否存在未声明的例外?
|
|
125
125
|
- 有限情况下的枚举是否穷尽?
|
|
126
126
|
|
|
127
|
-
### 16. 工具选择与流程审视 / Tool-Selection & Flow Review
|
|
127
|
+
### 16. 工具选择与流程审视 / Tool-Selection & Flow Review -> math-research-activator
|
|
128
128
|
|
|
129
129
|
- 审视维度的选择本身是否最优?是否遗漏了关键维度?
|
|
130
|
-
-
|
|
131
|
-
-
|
|
130
|
+
- 是否选择了最适合问题性质的思想透镜,而非仅仅选择最熟悉的?
|
|
131
|
+
- 是否遵循激活器主流程:**意图诊断 → 透镜路由 → 知识调用 → 设计翻译 → GPU 筛选**?是否跳过了诊断就堆数学科普?
|
|
132
132
|
- 是否枚举了多个候选结构(而非只给一个)?
|
|
133
133
|
- 若不确定该审视哪些维度,先调用 `/ask` 让激活器推荐最合适的 3–5 个审视维度。
|
|
134
134
|
|
|
135
|
-
### 17. GPU 可行性审视 / GPU-Feasibility Review
|
|
135
|
+
### 17. GPU 可行性审视 / GPU-Feasibility Review -> `references/gpu-friendly-math.md`
|
|
136
136
|
|
|
137
137
|
> 产出涉及算法/算子/训练/GPU 设计时为**强制项**。对应「双验收门」的第二道门。
|
|
138
138
|
|
|
139
|
-
- 产出是否过 `references/gpu-friendly-math.md` 的**八维**?张量化 / GEMM
|
|
139
|
+
- 产出是否过 `references/gpu-friendly-math.md` 的**八维**?张量化 / GEMM 可映射 / 复杂度(亚二次)/ 显存与 KV-Cache / 低精度稳定 / 并行与通信 / 稀疏结构 / 算子融合——逐项给「友好/可改造/不友好」。
|
|
140
140
|
- 是否存在「数学美但不可算」的结构?(典型:二阶法 Hessian 求逆、全局精确同调、符号因果发现、精确熵估计)——是否给出了可微/采样/低秩/近似改造?
|
|
141
141
|
- 逆变换/数值部分是否稳定(条件数、病态)?
|
|
142
142
|
- 显存与通信是否被评估(KV-Cache、分布式 all-reduce、optimizer state 精度)?
|
|
143
143
|
|
|
144
|
-
### 18. 现代数学激活审视 / Modern-Math Activation Review
|
|
144
|
+
### 18. 现代数学激活审视 / Modern-Math Activation Review -> `references/books/*`
|
|
145
145
|
|
|
146
146
|
> 产出声称「把现代数学激活进算法」时为**强制项**。对应「双验收门」的第一道门(数学正确)+ 跨域激活质量。
|
|
147
147
|
|
|
@@ -165,7 +165,7 @@
|
|
|
165
165
|
### 实现阶段
|
|
166
166
|
|
|
167
167
|
7. **定位修正点**:从审视发现的问题中,确定哪些需要具体实现方案。
|
|
168
|
-
8.
|
|
168
|
+
8. **选择思维工具**:针对每个修正点,选择最合适的数学思想透镜作为实现工具。
|
|
169
169
|
9. **给出实现方案**:为每个修正点提供具体的推导步骤、证明框架、算法设计或模型改进方案;若原方案不可算,给出可微/采样/低秩/近似改造方向。
|
|
170
170
|
|
|
171
171
|
## 输出格式
|
|
@@ -184,7 +184,7 @@
|
|
|
184
184
|
- [列出本次选择深入审视的 3–5 个维度及原因;若涉算法/GPU,标注 17/18 已纳入]
|
|
185
185
|
|
|
186
186
|
#### 双验收门结果
|
|
187
|
-
- [候选1]: 数学正确[
|
|
187
|
+
- [候选1]: 数学正确[v/x] | GPU 八维[友好/可改造/不友好] | 通过[是/否]
|
|
188
188
|
- [候选2]: ...
|
|
189
189
|
|
|
190
190
|
#### 做得好的地方
|
|
@@ -208,7 +208,7 @@
|
|
|
208
208
|
### 实现部分
|
|
209
209
|
|
|
210
210
|
#### 问题一:[必须/应当改正的问题标题]
|
|
211
|
-
- 思维工具: [
|
|
211
|
+
- 思维工具: [使用的数学思想透镜]
|
|
212
212
|
- 实现方案: [具体推导步骤 / 证明框架 / 算法设计 / 模型改进方案]
|
|
213
213
|
- 验证方式: [如何确认修正有效]
|
|
214
214
|
|
|
@@ -216,7 +216,7 @@
|
|
|
216
216
|
- [同上结构,根据问题数量灵活增减]
|
|
217
217
|
|
|
218
218
|
### 思维工具索引
|
|
219
|
-
- [
|
|
219
|
+
- [本次审视与实现中实际使用的思想透镜及其在哪个环节发挥作用]
|
|
220
220
|
|
|
221
221
|
### 总体评估
|
|
222
222
|
- [是否达标 + 主要修正路径 + 双验收门是否通过]
|
|
@@ -225,7 +225,7 @@
|
|
|
225
225
|
说明:
|
|
226
226
|
- 审视部分的"实现方向"是简要指引,实现部分的"实现方案"才是完整展开。两者呼应但不重复。
|
|
227
227
|
- 若某个问题无需深入实现(如纯粹的逻辑跳跃只需指出即可),实现部分可省略该条目。
|
|
228
|
-
- "思维工具索引"
|
|
228
|
+
- "思维工具索引"汇总本次实际动用的思想透镜,而非机械列出全部。
|
|
229
229
|
- 审视维度的选择因问题而异——审查论文假设时公理化思想是核心,评估算法时算法与计算思想+GPU 可行性是核心,不必每次都从假设开始。
|
|
230
230
|
|
|
231
231
|
## 原则
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ask
|
|
3
|
+
description: |
|
|
4
|
+
Math Research OS entry: auto-diagnose user intent, route to thinking lenses, math knowledge base, or design translation layer.
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Read and follow `../skills/math-research-activator/SKILL.en.md`.
|
|
8
|
+
|
|
9
|
+
Current question:
|
|
10
|
+
$ARGUMENTS
|
package/commands/ask.md
CHANGED
|
@@ -1,18 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ask
|
|
3
3
|
description: |
|
|
4
|
-
|
|
5
|
-
English:
|
|
4
|
+
数学研究操作系统入口:自动诊断用户意图,路由到思想透镜、数学知识库或设计翻译层。
|
|
5
|
+
English: Math Research OS entry: auto-diagnose user intent, route to thinking lenses, math knowledge base, or design translation layer.
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
按 `../skills/math-research-activator/SKILL.md` 中的语言路由与混合输入规则判定主语言。英文主语言 → 读取 `../skills/math-research-activator/SKILL.en.md`;中文主语言 → 读取 `../skills/math-research-activator/SKILL.md`。
|
|
9
9
|
|
|
10
10
|
当前问题:
|
|
11
11
|
$ARGUMENTS
|
|
12
|
-
|
|
13
|
-
输出要求(遵循激活器操作规程):
|
|
14
|
-
1. **[诊断]** 一句话点明问题核心特征/瓶颈(互动性/不确定性/约束/结构/动态/复杂度/显存/数值/并行)。
|
|
15
|
-
2. **[映射]** 枚举可迁移的现代数学结构候选(≥2 个,标注来自哪本书 `../references/books/*`);若与算法/算子设计无关可略。
|
|
16
|
-
3. **[武器路由]** 推荐 1–3 个思想武器,标主/辅 + 触发命令(如 `/optimization`);多个时说明组合顺序。
|
|
17
|
-
4. **[GPU 筛选]** 候选过 `../references/gpu-friendly-math.md` 八维,给「友好/可改造/不友好」+ 改造建议;若与硬件无关可略。
|
|
18
|
-
5. **[结论]** 保留通过双验收门的候选;标注哪些武器不适用;若不适合任何武器,明确说明。
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Equivariant Attention
|
|
2
|
+
> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
|
|
3
|
+
|
|
4
|
+
## Applicable Problems
|
|
5
|
+
When the input possesses an **explicit symmetry group $G$ action** (rotation, translation, permutation, reflection, etc.), and the desired model output should be **equivariant** (covariant) rather than invariant under the same transformations, equivariant constraints must be directly encoded into the attention mechanism. Typical scenarios include: 3D point clouds / molecules ($E(3)$ rigid-body group), image classification ($D_n$ rotation/reflection group), set data ($S_n$ permutation group), and multi-view / multi-sensor fusion.
|
|
6
|
+
|
|
7
|
+
## Mathematical Inspiration
|
|
8
|
+
- Lenses: [symmetry, categorical (unified framework for group actions)]
|
|
9
|
+
- Knowledge: [`../../knowledge-base/probability/concentration-inequality.md` (sample efficiency gains under equivariant constraints -- data equivalence along orbits), `../../knowledge-base/probability/entropy.md` (equivariant constraints reduce output distribution entropy, yielding stronger inductive bias)]
|
|
10
|
+
|
|
11
|
+
## Required Mathematical Knowledge
|
|
12
|
+
- **Group Representation Theory Basics**: Linear representation of a group $G$, $\rho: G \to GL(V)$, and irreducible representation decomposition
|
|
13
|
+
- **Equivariant Map Definition**: $f(g \cdot x) = \rho_{\text{out}}(g) \cdot f(x)$ for all $g \in G$
|
|
14
|
+
- **Orbit-Stabilizer Theorem** (see `references/books/abstract-algebra.md` Ch.5): $|orbit| = |G|/|stab|$, giving the parameter sharing multiplier
|
|
15
|
+
- **Schur's Lemma**: An equivariant linear map between irreducible representations is either zero or a scalar multiple
|
|
16
|
+
|
|
17
|
+
## AI Module Form
|
|
18
|
+
|
|
19
|
+
**Core Idea**: Replace the standard attention $Q, K, V$ with **steerable features**, ensuring that attention weights are invariant under group actions and that outputs are equivariant under group actions.
|
|
20
|
+
|
|
21
|
+
**Scheme A: Permutation-Equivariant Attention ($S_n$ group, set data)**:
|
|
22
|
+
```python
|
|
23
|
+
# DeepSets / Set Transformer style
|
|
24
|
+
# Attention weights are permutation-invariant: pi(Q)pi(K)^T = QK^T (permutations cancel)
|
|
25
|
+
# Output is permutation-equivariant: pi(softmax(QK^T) V) = softmax(QK^T) pi(V)
|
|
26
|
+
Q, K, V = W_q(X), W_k(X), W_v(X) # pointwise linear transform
|
|
27
|
+
scores = Q @ K.T / sqrt(d) # permutation-invariant
|
|
28
|
+
attn = softmax(scores) # permutation-invariant
|
|
29
|
+
output = attn @ V # permutation-equivariant (V is equivariant)
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
**Scheme B: $E(3)$-Equivariant Attention (3D point clouds / molecules)**:
|
|
33
|
+
```python
|
|
34
|
+
# Decompose features into scalars + vectors + higher-order tensors (spherical harmonic basis)
|
|
35
|
+
# Attention weights computed using only scalar features (rotation-invariant)
|
|
36
|
+
scalar_Q = scalar_proj(X_scalar) # scalars only -> rotation-invariant
|
|
37
|
+
scalar_K = scalar_proj(X_scalar)
|
|
38
|
+
scores = scalar_Q @ scalar_K.T / sqrt(d_s) # rotation-invariant attention weights
|
|
39
|
+
|
|
40
|
+
# V contains equivariant features (scalars + vectors), weighted by invariant weights
|
|
41
|
+
output_scalar = softmax(scores) @ V_scalar # scalar -> invariant
|
|
42
|
+
output_vector = softmax(scores) @ V_vector # vector -> equivariant (rotation-covariant)
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Scheme C: $D_n$-Equivariant Attention (image rotation / reflection)**:
|
|
46
|
+
```python
|
|
47
|
+
# G-CNN style: for each group element g in D_n, transform input with rho(g) then compute attention
|
|
48
|
+
# Weights shared along orbits (same W_q/W_k/W_v), aggregate outputs over all group elements
|
|
49
|
+
output = mean(softmax((rho(g)@X@W_q) @ (rho(g)@X@W_k).T/sqrt(d)) @ (rho(g)@X@W_v) for g in D_n)
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## Implementable Architectures
|
|
53
|
+
- **SE(3)-Transformer / Equiformer**: Spherical harmonic features + equivariant attention for molecular property prediction and protein structure
|
|
54
|
+
- **Set Transformer**: $S_n$ permutation-equivariant attention + Induced Set Attention (low-rank inducing points for complexity reduction)
|
|
55
|
+
- **G-CNN Attention**: $D_n$ rotation/reflection equivariance for remote sensing imagery and medical imaging
|
|
56
|
+
|
|
57
|
+
## GPU Feasibility
|
|
58
|
+
- **D1**: Group actions implemented as $\rho(g)$ matrix multiplications; equivariant features stored as batched tensors
|
|
59
|
+
- **D2**: $\rho(g) X$ and $Q K^T$ are both GEMM operations; $|G|$ group elements map to batched GEMM
|
|
60
|
+
- **D3**: $|G|$-fold computation overhead; acceptable for small groups ($|D_4|=8$), infeasible for large groups ($|S_n|=n!$). Remedy: use generators + Cayley graph propagation instead of full group enumeration
|
|
61
|
+
- **D4**: Requires storing $|G|$ copies of intermediate features; mitigated by chunking + gradient checkpointing
|
|
62
|
+
- **D5**: Orthogonal representation matrices are numerically stable under bf16
|
|
63
|
+
- **D6**: $|G|$ group elements are naturally parallelizable (along the batch dimension)
|
|
64
|
+
- **D7**: Permutation $\rho(g)$ is extremely sparse and can be encoded as gather indices
|
|
65
|
+
- **D8**: Group action + linear duality can be fused into a single batched GEMM
|
|
66
|
+
|
|
67
|
+
## Paper Phrasing
|
|
68
|
+
"We propose an equivariant attention mechanism that constrains attention weights to be group invariants and attention outputs to be group equivariants, directly encoding the inductive bias of symmetry group $G$ into the model architecture without additional data augmentation, achieving a $|G|/|stab|$-fold improvement in parameter efficiency."
|
|
69
|
+
|
|
70
|
+
## Risks
|
|
71
|
+
- **Cost of Incorrect Group Selection**: If the data does not possess the assumed symmetry (e.g., molecules lacking full $E(3)$ symmetry), equivariant constraints will impair expressiveness. The symmetry assumption must be validated first, or "approximate equivariance" (soft equivariance) should be used.
|
|
72
|
+
- **Memory Explosion of High-Order Representations**: The $L$-th order spherical harmonic representation of $SO(3)$ has dimension $(2L+1)^2$, causing storage and computation to grow rapidly for high-order features ($L \geq 3$). In practice, truncation to $L \leq 2$ is standard.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# 等变注意力 / Equivariant Attention
|
|
2
|
+
> **严谨性声明**:本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行,需工程验证。
|
|
3
|
+
|
|
4
|
+
## 适用问题
|
|
5
|
+
当输入具有**明确的对称群 $G$ 作用**(旋转、平移、置换、反射等),且期望模型输出在相同变换下**协变**(equivariant)而非不变时,需要将等变约束直接编入注意力机制。典型场景:3D 点云/分子($E(3)$ 刚体群)、图像分类($D_n$ 旋转/镜像群)、集合数据($S_n$ 置换群)、多视角/多传感器融合。
|
|
6
|
+
|
|
7
|
+
## 数学思想来源
|
|
8
|
+
- 透镜:[symmetry(对称透镜), categorical(范畴化透镜 — 群作用的统一框架)]
|
|
9
|
+
- 知识:[`../../knowledge-base/probability/concentration-inequality.md`(等变约束下的样本效率提升 — 轨道上的数据等价性), `../../knowledge-base/probability/entropy.md`(等变约束降低输出分布熵 → 更强的归纳偏置)]
|
|
10
|
+
|
|
11
|
+
## 需要的数学知识
|
|
12
|
+
- **群表示论基础**:群 $G$ 的线性表示 $\rho: G \to GL(V)$,不可约表示分解
|
|
13
|
+
- **等变映射定义**:$f(g \cdot x) = \rho_{\text{out}}(g) \cdot f(x)$,对所有 $g \in G$
|
|
14
|
+
- **轨道-稳定子定理**(参见 `references/books/abstract-algebra.md` Ch.5):$|orbit| = |G|/|stab|$,参数共享倍率
|
|
15
|
+
- **Schur 引理**:不可约表示间的等变线性映射要么为零要么为标量乘
|
|
16
|
+
|
|
17
|
+
## AI 模块形式
|
|
18
|
+
|
|
19
|
+
**核心思路**:将标准注意力的 $Q, K, V$ 替换为**等变特征**(steerable features),确保注意力权重在群作用下不变,输出在群作用下等变。
|
|
20
|
+
|
|
21
|
+
**方案 A:置换等变注意力($S_n$ 群,集合数据)**:
|
|
22
|
+
```python
|
|
23
|
+
# DeepSets / Set Transformer 风格
|
|
24
|
+
# 注意力权重对置换不变:π(Q)π(K)^T = QK^T(置换抵消)
|
|
25
|
+
# 输出对置换等变:π(softmax(QK^T) V) = softmax(QK^T) π(V)
|
|
26
|
+
Q, K, V = W_q(X), W_k(X), W_v(X) # 逐点线性变换
|
|
27
|
+
scores = Q @ K.T / sqrt(d) # 置换不变
|
|
28
|
+
attn = softmax(scores) # 置换不变
|
|
29
|
+
output = attn @ V # 置换等变(V 是等变的)
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
**方案 B:$E(3)$ 等变注意力(3D 点云/分子)**:
|
|
33
|
+
```python
|
|
34
|
+
# 将特征分解为标量 + 向量 + 高阶张量(球谐基)
|
|
35
|
+
# 注意力权重只用标量特征计算(旋转不变)
|
|
36
|
+
scalar_Q = scalar_proj(X_scalar) # 仅标量 → 旋转不变
|
|
37
|
+
scalar_K = scalar_proj(X_scalar)
|
|
38
|
+
scores = scalar_Q @ scalar_K.T / sqrt(d_s) # 旋转不变的注意力权重
|
|
39
|
+
|
|
40
|
+
# V 包含等变特征(标量 + 向量),用不变的权重加权
|
|
41
|
+
output_scalar = softmax(scores) @ V_scalar # 标量 → 不变
|
|
42
|
+
output_vector = softmax(scores) @ V_vector # 向量 → 等变(旋转协变)
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**方案 C:$D_n$ 等变注意力(图像旋转/镜像)**:
|
|
46
|
+
```python
|
|
47
|
+
# G-CNN 风格:对每个群元素 g ∈ D_n,用 ρ(g) 变换输入后计算注意力
|
|
48
|
+
# 权重沿轨道共享(同一组 W_q/W_k/W_v),聚合所有群元素输出
|
|
49
|
+
output = mean(softmax((rho(g)@X@W_q) @ (rho(g)@X@W_k).T/sqrt(d)) @ (rho(g)@X@W_v) for g in D_n)
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## 可实现结构
|
|
53
|
+
- **SE(3)-Transformer / Equiformer**:球谐特征 + 等变注意力,用于分子性质预测和蛋白质结构
|
|
54
|
+
- **Set Transformer**:$S_n$ 置换等变注意力 + Induced Set Attention(低秩诱导点降复杂度)
|
|
55
|
+
- **G-CNN 注意力**:$D_n$ 旋转/镜像等变,用于遥感图像和医学影像
|
|
56
|
+
|
|
57
|
+
## GPU 可行性
|
|
58
|
+
- **D1[v]**:群作用实现为 $\rho(g)$ 矩阵乘,等变特征为 batched 张量
|
|
59
|
+
- **D2[v]**:$\rho(g) X$ 和 $Q K^T$ 均为 GEMM;$|G|$ 个群元素 → batched GEMM
|
|
60
|
+
- **D3[~]**:$|G|$ 倍计算量,小群($|D_4|=8$)可接受,大群($|S_n|=n!$)不可。改造:用生成元 + Cayley 图传播替代整群枚举
|
|
61
|
+
- **D4[~]**:需存储 $|G|$ 份中间特征,可分块 + gradient checkpointing
|
|
62
|
+
- **D5[v]**:正交表示矩阵在 bf16 下数值稳定
|
|
63
|
+
- **D6[v]**:$|G|$ 个群元素天然并行(batch 维)
|
|
64
|
+
- **D7[~]**:置换 $\rho(g)$ 极度稀疏,可编码为 gather 索引
|
|
65
|
+
- **D8[v]**:群作用 + 线性变换可融合为单次 batched GEMM
|
|
66
|
+
|
|
67
|
+
## 论文表述方式
|
|
68
|
+
"我们提出等变注意力机制,通过将注意力权重约束为群不变量、注意力输出约束为群等变量,在不增加数据增强的情况下将对称群 $G$ 的归纳偏置直接编码进模型结构,参数效率提升 $|G|/|stab|$ 倍。"
|
|
69
|
+
|
|
70
|
+
## 风险
|
|
71
|
+
- **群选择错误的代价**:若数据不具有假定的对称性(如分子不具有完全 $E(3)$ 对称性),等变约束会伤害表达能力。需先验证对称性假设,或使用"近似等变"(soft equivariance)。
|
|
72
|
+
- **高阶表示的显存爆炸**:$SO(3)$ 的 $L$ 阶球谐表示维度为 $(2L+1)^2$,高阶特征 ($L \geq 3$) 的存储和计算量急剧增长。实践中通常截断到 $L \leq 2$。
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# Geometry-Aware Attention
|
|
2
|
+
> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
|
|
3
|
+
|
|
4
|
+
## Applicable Problems
|
|
5
|
+
When **known geometric relationships** exist between tokens/keys (spatial distance, manifold geodesic distance, hierarchical structure, temporal interval), the position-agnostic inner product of standard attention cannot exploit this structural information. Geometry-aware attention **directly injects geometric priors into attention weight computation**, enabling the model to naturally respect the metric structure of the underlying space. Typical scenarios include: 3D scene understanding, molecular conformation, temporal forecasting, hierarchical text structure (paragraph-sentence-word), and knowledge graphs.
|
|
6
|
+
|
|
7
|
+
## Mathematical Inspiration
|
|
8
|
+
- Lenses: [symmetry (metric invariance), duality (coordinate-system-independent representation)]
|
|
9
|
+
- Knowledge: [`../../knowledge-base/information-geometry/fisher-metric.md` (geometric metrics on distribution spaces), `../../knowledge-base/probability/concentration-inequality.md` (concentration behavior under geometric constraints)]
|
|
10
|
+
|
|
11
|
+
## Required Mathematical Knowledge
|
|
12
|
+
- **Metric Spaces and Distance Functions**: Euclidean distance, geodesic distance, tree distance, Wasserstein distance
|
|
13
|
+
- **RBF / Kernel Methods**: $k(x, y) = \exp(-d(x,y)^2 / 2\sigma^2)$, converting distance to similarity
|
|
14
|
+
- **Geometric Interpretation of Positional Encoding**: RoPE = action of the rotation group $SO(2)$ (relative distance encoded as rotation angle difference); ALiBi = exponentially decaying distance bias
|
|
15
|
+
|
|
16
|
+
## AI Module Form
|
|
17
|
+
|
|
18
|
+
**Core Idea**: Explicitly introduce a geometric distance term into attention scores so that attention to distant tokens naturally decays:
|
|
19
|
+
|
|
20
|
+
$$\text{GeoAttn}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d}} + \text{GeoBias}(i, j)\right) V$$
|
|
21
|
+
|
|
22
|
+
**Scheme A: Distance-Biased Attention (General Metric Spaces)**:
|
|
23
|
+
```python
|
|
24
|
+
# d_ij is the geometric distance between token i and j (precomputed or computed online)
|
|
25
|
+
# Learnable distance bias function
|
|
26
|
+
distance_bias = MLP(d_ij) # or simply -alpha * d_ij
|
|
27
|
+
scores = (Q @ K.T) / sqrt(d) + distance_bias
|
|
28
|
+
attn = softmax(scores) @ V
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**Scheme B: Relative Positional Encoding (Geometric Generalization of RoPE / ALiBi)**:
|
|
32
|
+
```python
|
|
33
|
+
# Generalize RoPE: position -> group element g_i, relative position -> g_i g_j^{-1}
|
|
34
|
+
# Learnable relative position bias on arbitrary metric spaces
|
|
35
|
+
def geo_attention(Q, K, V, positions):
|
|
36
|
+
rel_pos = pairwise_difference(positions) # (n, n, coord_dim)
|
|
37
|
+
geo_bias = geo_encoder(rel_pos) # (n, n) learnable mapping
|
|
38
|
+
scores = (Q @ K.T) / sqrt(d) + geo_bias
|
|
39
|
+
return softmax(scores) @ V
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
**Scheme C: Manifold-Aware Attention (Non-Euclidean Spaces)**:
|
|
43
|
+
```python
|
|
44
|
+
# When tokens reside on a non-Euclidean manifold (hyperbolic space, sphere),
|
|
45
|
+
# use geodesic distance instead of Euclidean distance
|
|
46
|
+
def manifold_attention(Q, K, V, manifold):
|
|
47
|
+
geodist = manifold.geodesic_distance_matrix(positions) # (n, n)
|
|
48
|
+
geo_kernel = exp(-geodist^2 / (2 * sigma^2)) # RBF kernel
|
|
49
|
+
scores = (Q @ K.T) / sqrt(d) + log(geo_kernel + eps)
|
|
50
|
+
return softmax(scores) @ V
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Implementable Architectures
|
|
54
|
+
- **RoPE Extensions**: From $SO(2)$ rotations to higher-dimensional rotation groups $SO(2k)$, encoding multi-dimensional positional information (2D image patches, 3D voxels)
|
|
55
|
+
- **Hierarchical Positional Bias**: Use LCA (Lowest Common Ancestor) depth as distance in tree structures, suitable for hierarchical geometric of documents and code
|
|
56
|
+
- **Molecular Conformation Attention**: 3D atomic coordinates -> distance matrix -> geometric bias, used in molecular GNNs and protein structure prediction
|
|
57
|
+
|
|
58
|
+
## GPU Feasibility
|
|
59
|
+
- **D1**: Distance matrices and bias matrices are dense tensors with element-wise operations
|
|
60
|
+
- **D2**: The main body $Q K^T$ is standard GEMM; geometric bias is additive and does not interrupt GEMM
|
|
61
|
+
- **D3**: Pairwise distance matrix construction and storage are $O(n^2)$; however, block-wise computation + online softmax can avoid materializing the full matrix
|
|
62
|
+
- **D4**: The $n \times n$ distance matrix occupies GPU memory; mitigated by block/streaming computation (compatible with FlashAttention)
|
|
63
|
+
- **D5**: Distance computation and bias addition are stable under bf16; the exp in RBF requires overflow caution (clamp distances)
|
|
64
|
+
- **D6**: Distance computation and attention can be pipelined; pairwise distances can be computed in parallel blocks
|
|
65
|
+
- **D7**: Distant biases tend toward $-\infty$ (approaching zero after softmax), naturally inducing structured sparsity (local attention windows)
|
|
66
|
+
- **D8**: Geometric bias can be fused into FlashAttention's online softmax loop (added after $QK^T$, before softmax)
|
|
67
|
+
|
|
68
|
+
## Paper Phrasing
|
|
69
|
+
"We propose geometry-aware attention, which explicitly injects learnable geometric distance bias terms into attention scores, enabling the model to naturally respect the metric structure of the input space. This achieves exponential decay of attention to distant tokens without increasing parameter count, while maintaining compatibility with FlashAttention."
|
|
70
|
+
|
|
71
|
+
## Risks
|
|
72
|
+
- **Geometric Prior Conflicting with Data**: If geometric distance is inconsistent with semantic relevance (e.g., distant but semantically related tokens in text), strong geometric bias will impair model expressiveness. The bias should be learnable and overridable by content-based attention.
|
|
73
|
+
- **Non-Differentiability of Distance Computation**: Certain distances (graph shortest path, tree LCA depth) are non-differentiable or computationally expensive. Differentiable relaxations (softmin instead of min) or precomputed distance matrices are needed.
|