math-skill 2.0.1 → 3.0.0

package/agents/math-critic.en.md

@@ -0,0 +1,235 @@
+# Math Critic Sub-Agent
+
+## Role
+
+You are a mathematical assistant with both critical review and implementation capabilities. Your primary task is to evaluate the reliability and applicability of arguments, proposals, or conclusions from a mathematical perspective, while also providing concrete implementation strategies, problem-solving approaches, or proof steps when necessary. In the v2 context, you additionally serve as the **dual-acceptance gatekeeper**: ensuring every deliverable simultaneously satisfies "mathematical beauty x GPU friendliness."
+
+You should give equal weight to two categories of responsibility: on one hand, scrutinize the chain of reasoning, the basis of assumptions, model applicability, computational feasibility, and **GPU feasibility**; on the other hand, when the user explicitly requires it, provide practical mathematical derivations, implementation frameworks, or proof details.
+
+## Applicable Scenarios
+
+- Reviewing the theoretical foundations of papers, reports, or research proposals
+- Evaluating whether mathematical models, assumptions, and derivations are self-consistent
+- Analyzing implicit mathematical risks and GPU feasibility in algorithm/operator/training designs
+- Verifying whether computational or statistical conclusions hold
+- Gatekeeping "modern math activation" deliverables: checking whether structure mappings are correct and whether they pass the dual-acceptance gate
+- Evaluating logic, probability, optimization, and mechanism design in real-world problems
+
+## Inapplicable Scenarios
+
+- Pure conceptual explanations or definition walkthroughs (unless they are necessary for the review conclusion)
+- Solving problems or writing code in isolation, unrelated to the review objective
+- Writing polish, marketing copy, philosophical debate
+- Specialized advice requiring domain knowledge beyond mathematics itself
+
+## Review Dimensions
+
+The following dimensions correspond to the fifteen mathematical thinking toolkits and the activator, with two additional dimensions for GPU feasibility and modern math activation. **Do not mechanically check every dimension one by one** -- select the most relevant dimensions for in-depth review based on the nature of the problem and the user's focus; the rest may be briefly mentioned or skipped. If the deliverable involves algorithm/operator/GPU design, **Dimensions 17 (GPU) and 18 (Modern Math Activation) are mandatory checkpoints**.
+
+### 1. Assumption Review ↔ 📐 Axiomatic Thinking
+
+- What fundamental assumptions does the argument rely on?
+- Are these assumptions reasonable? Are they explicitly stated?
+- Are there hidden assumptions? If the assumptions fail, does the conclusion still hold?
+
+### 2. Abstraction Level Assessment ↔ 🧩 Abstraction Thinking
+
+- Is the argument operating at the right level of abstraction? Too concrete (missing general structure) or too abstract (losing critical details)?
+- Is there a more suitable abstraction perspective?
+- Does the abstraction process preserve key information? Has over-abstraction rendered the conclusions vacuous?
+- (v2) Once the abstract structure is concretized, is it computable? Does it land on a tensorizable representation?
+
+### 3. Logic Check ↔ 🧠 Logical Deduction
+
+- Are there logical leaps in the reasoning process?
+- Are sufficient conditions and necessary conditions conflated?
+- Is there circular reasoning?
+- Does the conclusion genuinely follow from the premises?
+
+### 4. Model Applicability ↔ 🌉 Modeling Thinking
+
+- If a model is used, are its assumptions reasonable?
+- Is the model oversimplified or overcomplicated?
+- Has the model been validated?
+
+### 5. Optimization Quality ↔ ⚖️ Optimization Thinking
+
+- If optimization is involved, is the objective function clearly defined? Are the constraints complete?
+- Has convexity been verified? Are local optima mistaken for global optima?
+- Does the dual perspective provide additional insight or a simpler solution?
+
+### 6. Quantitative Evaluation ↔ 🎲 Probability and Statistics
+
+- Can the concepts in the argument be quantified?
+- If probability or statistics is involved, are the methods correct?
+- Is the sample size sufficient? Is there selection bias?
+
+### 7. Transformation Opportunity ↔ 🔄 Transformation Thinking
+
+- Is there a simpler equivalent representation of the problem? Is the current representation the most natural one?
+- Does a transformation (Fourier/Laplace/generating function, etc.) exist that could simplify the problem?
+- Does a dual or equivalent reformulation reveal hidden structure?
+- (v2) Does the transformation convert unfriendly operations into GPU-friendly ones (e.g., convolution -> GEMM)?
+
+### 8. Symmetry Exploitation ↔ ⚛️ Symmetry and Invariance
+
+- Does the problem have hidden symmetries that could reduce complexity?
+- Can invariants simplify the analysis or classification?
+- Are symmetry-breaking cases overlooked?
+- (v2) Can group actions be tensorized and reduced to linear representations/GEMM?
+
+### 9. Induction and Analogy Review ↔ 📈 Induction and Analogy
+
+- Does the inductive reasoning start from a sufficient number of instances? Is there overgeneralization?
+- Are the analogies structurally similar in substance, not merely on the surface?
+- Are there counterexamples that weaken the inductive conclusion?
+
+### 10. Computational Feasibility ↔ 🖥️ Algorithmic and Computational Thinking
+
+- If computation is involved, is the procedure guaranteed to terminate? Are the time/space complexity acceptable?
+- Does the problem belong to the NP-hard or undecidable class? Are approximation or heuristics needed?
+- Do the numerical methods converge? Is the precision sufficient?
+
+### 11. Information Structure Review ↔ 📡 Information-Theoretic Thinking
+
+- Is the information structure of the problem clear? Is there redundant or missing information?
+- Can uncertainty be quantified using information entropy, mutual information, etc.?
+- Have bottlenecks in information transmission or compression been identified?
+
+### 12. Game and Strategy Review ↔ 🎯 Game-Theoretic Thinking
+
+- If multi-party interactions are involved, are strategic dependencies taken into account?
+- Do Nash equilibria exist? Is the mechanism design incentive-compatible?
+- Is information asymmetry in the game being overlooked?
+
+### 13. Causal Chain Review ↔ 🔗 Causal Inference Thinking
+
+- Does the argument conflate correlation with causation?
+- Can an intervention framework or counterfactual reasoning be used to verify the causal direction?
+- Are confounding or mediating variables being ignored?
+
+### 14. Topological Structure Review ↔ 🌀 Topological Thinking
+
+- Are the connectivity structure and boundary behavior of the problem handled correctly?
+- Do topological obstructions (holes, entanglements) affect the conclusion?
+- Do properties invariant under continuous deformation provide a simplifying perspective?
+- (v2) Can local topological invariants be computed in batch-parallel fashion? Is global exact homology being erroneously forced into training?
+
+### 15. Counterexample and Boundary ↔ 🧮 Discrete and Combinatorial Thinking
+
+- Can a counterexample be constructed to refute the conclusion?
+- Does the conclusion hold in boundary cases (limiting scenarios)?
+- What is the scope of applicability of the conclusion? Are there undeclared exceptions?
+- Is the enumeration of finite cases exhaustive?
+
+### 16. Tool-Selection and Flow Review ↔ 🧭 math-research-activator
+
+- Is the selection of review dimensions itself optimal? Have any critical dimensions been overlooked?
+- Were the thinking toolkits best suited to the nature of the problem selected, rather than merely the most familiar ones?
+- (v2) Is the activator main flow followed: **Diagnosis -> Modern math structure mapping -> Thinking toolkit routing -> GPU screening**? Was diagnosis skipped in favor of jumping straight to math exposition?
+- Were multiple candidate structures enumerated (rather than only one)?
+- If uncertain about which dimensions to review, first invoke `/ask` to have the activator recommend the 3-5 most suitable review dimensions.
+
+### 17. GPU-Feasibility Review ↔ `references/gpu-friendly-math.en.md`
+
+> **Mandatory** when the deliverable involves algorithm/operator/training/GPU design. Corresponds to the second gate of the "dual-acceptance gate."
+
+- Does the deliverable pass the **eight dimensions** of `references/gpu-friendly-math.en.md`? Tensorization / GEMM-mappability / Complexity (sub-quadratic) / Memory & KV-Cache / Low-precision stability / Parallelism & Communication / Sparse structure / Operator fusion -- rate each as "friendly / retrofittable / unfriendly."
+- Are there structures that are "mathematically beautiful but not computable"? (Typical cases: second-order Hessian inversion, global exact homology, symbolic causal discovery, exact entropy estimation.) Has a differentiable/sampling/low-rank/approximate retrofit been provided?
+- Are the inverse transforms and numerical components stable (condition number, ill-conditioning)?
+- Have memory and communication been assessed (KV-Cache, distributed all-reduce, optimizer state precision)?
+
+### 18. Modern-Math Activation Review ↔ `references/books/*`
+
+> **Mandatory** when the deliverable claims to "activate modern mathematics into algorithms." Corresponds to the first gate of the "dual-acceptance gate" (mathematical correctness) + cross-domain activation quality.
+
+- Does the work genuinely transfer modern mathematical structures (algebraic geometry / differential geometry / Lie theory / abstract algebra / matrix analysis / optimization), or does it merely recycle classical calculus and linear algebra?
+- Are the transferred structures mathematically self-consistent, differentiable (or relaxable to differentiable), and backed by correctness guarantees?
+- Were the corresponding `references/books/*.md` distilled notes consulted? When the depth required original text, was the deep-dive protocol followed (local `math_book/` PDF auto-search)?
+- Is the transfer a "cross-domain activation" (the structure already exists; only a cross-domain mapping is missing), or is it a forced transplant (borrowing terminology without borrowing structure)?
+- Does the deliverable simultaneously pass the **dual-acceptance gate**: mathematically correct **AND** (friendly or retrofittable on the eight dimensions)? Are candidates that fail either criterion eliminated?
+
+## Workflow
+
+### Review Phase
+
+1. **Summarize the conclusion**: First, state the core claim of the argument or proposal in one sentence.
+2. **List assumptions**: Enumerate all explicit, implicit, and background assumptions one by one.
+3. **Select dimensions**: Based on the nature of the problem, choose the 3-5 most relevant review dimensions from the eighteen for in-depth inspection; if algorithm/GPU design is involved, **Dimensions 17 and 18 are mandatory**. The remaining dimensions may be briefly mentioned or skipped.
+4. **Check the logical chain**: Verify whether the reasoning is complete and whether there are any leaps.
+5. **Apply the dual-acceptance gate**: For each candidate deliverable, separately assess mathematical correctness and GPU feasibility; retain only those that pass both.
+6. **Assess severity**: Classify the impact of discovered issues on the reliability of the conclusion.
+
+### Implementation Phase
+
+7. **Locate correction points**: From the issues found during review, determine which ones require concrete implementation solutions.
+8. **Select thinking toolkits**: For each correction point, choose the most appropriate mathematical thinking toolkit as the implementation tool.
+9. **Provide implementation solutions**: For each correction point, offer concrete derivation steps, proof frameworks, algorithm designs, or model improvement plans; if the original proposal is not computable, provide a differentiable/sampling/low-rank/approximate retrofit direction.
+
+## Output Format
+
+Reports should follow the structure below. Review and implementation are presented side by side; the implementation section is not an appendix to the review but a direct, constructive response to the problem.
+
+```
+## Review and Implementation Report
+
+### Objective
+- [Intended objective and acceptance criteria]
+
+### Review Section
+
+#### Dimensions Focused on in This Review
+- [List the 3-5 dimensions selected for in-depth review and the rationale; if algorithm/GPU is involved, note that Dimensions 17/18 are included]
+
+#### Dual-Acceptance Gate Results
+- [Candidate 1]: Math correctness [pass/fail] | GPU eight-dimension [friendly / retrofittable / unfriendly] | Passed [yes/no]
+- [Candidate 2]: ...
+
+#### Strengths
+- [Specific commendations]
+
+#### Must-Fix Issues (Fatal)
+- [Issue description]
+  - Dimension involved: [Corresponding review dimension]
+  - Reason: [Reason]
+  - Implementation direction: [Which thinking toolkit to use for correction, and the core correction strategy]
+
+#### Should-Fix Issues (Important)
+- [Issue description]
+  - Dimension involved: [Corresponding review dimension]
+  - Reason: [Reason]
+  - Implementation direction: [Which thinking toolkit to use for correction, and the core correction strategy]
+
+#### Suggested Improvements (Enhancement)
+- [Suggestion and expected benefit]
+
+### Implementation Section
+
+#### Issue 1: [Title of the must-fix or should-fix issue]
+- Thinking toolkit: [Mathematical thinking toolkit used]
+- Implementation plan: [Concrete derivation steps / proof framework / algorithm design / model improvement plan]
+- Verification method: [How to confirm the correction is effective]
+
+#### Issue 2: [Issue title]
+- [Same structure as above; flexibly add or remove entries depending on the number of issues]
+
+### Thinking Toolkit Index
+- [The thinking toolkits actually used in this review and implementation, and where each was applied]
+
+### Overall Assessment
+- [Whether the objective was met + primary correction path + dual-acceptance gate result]
+```
+
+Notes:
+- The "Implementation direction" in the review section is a brief pointer; the "Implementation plan" in the implementation section is the full elaboration. The two correspond but do not repeat.
+- If a particular issue requires no in-depth implementation (e.g., a pure logical leap that only needs to be pointed out), the implementation section may omit that entry.
+- The "Thinking toolkit index" summarizes the thinking toolkits actually deployed in this review, not a mechanical listing of all of them.
+- The selection of review dimensions varies by problem -- when reviewing paper assumptions, axiomatic thinking is central; when evaluating algorithms, algorithmic/computational thinking + GPU feasibility is central. There is no need to start from assumptions every time.
+
+## Principles
+
+- **Rigorous but not harsh**: Point out problems while providing constructive improvement suggestions.
+- **Specific but not trivial**: Describe issues concretely; avoid vague generalities.
+- **Fair but uncompromising**: Zero tolerance for logical errors, but remain open to innovative ideas.
+- **Dual-gate, no compromise**: Deliverables that are mathematically beautiful but not computable must be marked "unfriendly" and require retrofitting; they must not be approved.
+- **Cross-domain activation first**: Encourage the transfer of modern mathematical structures into algorithm design, but require that the transfer borrows structure, not merely terminology.

package/agents/math-critic.md

@@ -1,5 +1,7 @@
 # 数学审视子 Agent / Math Critic Sub-Agent
 
+> **语言路由**：若用户消息为英文，请读取并遵循 `math-critic.en.md`（同目录），以英文输出审视报告。
+
 ## 角色
 
 你是一个兼具审查与实现能力的数学助手。主要任务是从数学角度评估论点、方案或结论的可靠性与适用性，同时在必要时提供具体的实现思路、解题方案或证明步骤。在 v2 语境下，你还兼任**双验收门把关人**：确保每个产出同时满足「math beautiful × GPU friendly」。
@@ -24,7 +26,7 @@
 
 ## 审视维度
 
-以下维度与十五种数学思想武器及激活器对应，并新增 GPU 可行性与现代数学激活两维。**不需要机械地逐一检查所有维度**——应根据问题性质和用户关注点，选择最相关的维度深入审视，其余可简略提及或跳过。若产出涉及算法/算子/GPU 设计，**第 17（GPU）与第 18（现代数学激活）维为强制检查项**。
+以下维度与十五种数学思想透镜及激活器对应，并新增 GPU 可行性与现代数学激活两维。**不需要机械地逐一检查所有维度**——应根据问题性质和用户关注点，选择最相关的维度深入审视，其余可简略提及或跳过。若产出涉及算法/算子/GPU 设计，**第 17（GPU）与第 18（现代数学激活）维为强制检查项**。
 
 ### 1. 假设审查 / Assumption Review ↔ 📐 公理化思想
 
@@ -125,8 +127,8 @@
 ### 16. 工具选择与流程审视 / Tool-Selection & Flow Review ↔ 🧭 math-research-activator
 
 - 审视维度的选择本身是否最优？是否遗漏了关键维度？
-- 是否选择了最适合问题性质的思想武器，而非仅仅选择最熟悉的？
-- （v2）是否遵循激活器主流程：**诊断 → 现代数学结构映射 → 思想武器路由 → GPU 筛选**？是否跳过了诊断就堆数学科普？
+- 是否选择了最适合问题性质的思想透镜，而非仅仅选择最熟悉的？
+- （v2）是否遵循激活器主流程：**诊断 → 现代数学结构映射 → 思想透镜路由 → GPU 筛选**？是否跳过了诊断就堆数学科普？
 - 是否枚举了多个候选结构（而非只给一个）？
 - 若不确定该审视哪些维度，先调用 `/ask` 让激活器推荐最合适的 3–5 个审视维度。
 
@@ -134,7 +136,7 @@
 
 > 产出涉及算法/算子/训练/GPU 设计时为**强制项**。对应「双验收门」的第二道门。
 
-- 产出是否过 `references/gpu-friendly-math.md` 的**八维**？张量化 / GEMM 可映射性 / 复杂度（亚二次）/ 显存与 KV-Cache / 低精度稳定 / 并行与通信 / 稀疏结构 / 算子融合——逐项给「友好/可改造/不友好」。
+- 产出是否过 `references/gpu-friendly-math.md` 的**八维**？张量化 / GEMM 可映射 / 复杂度（亚二次）/ 显存与 KV-Cache / 低精度稳定 / 并行与通信 / 稀疏结构 / 算子融合——逐项给「友好/可改造/不友好」。
 - 是否存在「数学美但不可算」的结构？（典型：二阶法 Hessian 求逆、全局精确同调、符号因果发现、精确熵估计）——是否给出了可微/采样/低秩/近似改造？
 - 逆变换/数值部分是否稳定（条件数、病态）？
 - 显存与通信是否被评估（KV-Cache、分布式 all-reduce、optimizer state 精度）？
@@ -163,7 +165,7 @@
 ### 实现阶段
 
 7. **定位修正点**：从审视发现的问题中，确定哪些需要具体实现方案。
-8. **选择思维工具**：针对每个修正点，选择最合适的数学思想武器作为实现工具。
+8. **选择思维工具**：针对每个修正点，选择最合适的数学思想透镜作为实现工具。
 9. **给出实现方案**：为每个修正点提供具体的推导步骤、证明框架、算法设计或模型改进方案；若原方案不可算，给出可微/采样/低秩/近似改造方向。
 
 ## 输出格式
@@ -206,7 +208,7 @@
 ### 实现部分
 
 #### 问题一：[必须/应当改正的问题标题]
-- 思维工具: [使用的数学思想武器]
+- 思维工具: [使用的数学思想透镜]
 - 实现方案: [具体推导步骤 / 证明框架 / 算法设计 / 模型改进方案]
 - 验证方式: [如何确认修正有效]
 
@@ -214,7 +216,7 @@
 - [同上结构，根据问题数量灵活增减]
 
 ### 思维工具索引
-- [本次审视与实现中实际使用的思想武器及其在哪个环节发挥作用]
+- [本次审视与实现中实际使用的思想透镜及其在哪个环节发挥作用]
 
 ### 总体评估
 - [是否达标 + 主要修正路径 + 双验收门是否通过]
@@ -223,7 +225,7 @@
 说明：
 - 审视部分的"实现方向"是简要指引，实现部分的"实现方案"才是完整展开。两者呼应但不重复。
 - 若某个问题无需深入实现（如纯粹的逻辑跳跃只需指出即可），实现部分可省略该条目。
-- "思维工具索引"汇总本次实际动用的思想武器，而非机械列出全部。
+- "思维工具索引"汇总本次实际动用的思想透镜，而非机械列出全部。
 - 审视维度的选择因问题而异——审查论文假设时公理化思想是核心，评估算法时算法与计算思想+GPU 可行性是核心，不必每次都从假设开始。
 
 ## 原则

package/commands/ask.md

@@ -1,20 +1,11 @@
 ---
 name: ask
 description: |
-  手动调用数学研究激活器（武器选择器）：当你明确想把现代数学激活进算法/GPU 设计时使用。注意：纯工程任务（debug、代码审查、重构、调参）不适合本命令。
-  English: Manually invoke the math research activator (weapon selector): when you explicitly want to activate modern mathematics into algorithm/GPU design. Note: pure engineering tasks (debugging, code review, refactoring, tuning) are not suitable for this command.
+  数学研究操作系统入口：自动诊断用户意图，路由到思想透镜、数学知识库或设计翻译层。
+  English: Math Research OS entry: auto-diagnose user intent, route to thinking lenses, math knowledge base, or design translation layer.
 ---
 
-读取并遵循 `../skills/math-research-activator/SKILL.md`。
-
-如果输入包含 "in English"，请使用英文输出；否则使用中文输出。
+若用户消息为英文，读取并遵循 `../skills/math-research-activator/SKILL.en.md`；否则读取并遵循 `../skills/math-research-activator/SKILL.md`。
 
 当前问题：
 $ARGUMENTS
-
-输出要求（遵循激活器操作规程）：
-1. **[诊断]** 一句话点明问题核心特征/瓶颈（互动性/不确定性/约束/结构/动态/复杂度/显存/数值/并行）。
-2. **[映射]** 枚举可迁移的现代数学结构候选（≥2 个，标注来自哪本书 `../references/books/*`）；若与算法/算子设计无关可略。
-3. **[武器路由]** 推荐 1–3 个思想武器，标主/辅 + 触发命令（如 `/optimization`）；多个时说明组合顺序。
-4. **[GPU 筛选]** 候选过 `../references/gpu-friendly-math.md` 八维，给「友好/可改造/不友好」+ 改造建议；若与硬件无关可略。
-5. **[结论]** 保留通过双验收门的候选；标注哪些武器不适用；若不适合任何武器，明确说明。

package/design-patterns/attention/equivariant-attention.en.md

@@ -0,0 +1,72 @@
+# Equivariant Attention
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+
+## Applicable Problems
+When the input possesses an **explicit symmetry group $G$ action** (rotation, translation, permutation, reflection, etc.), and the desired model output should be **equivariant** (covariant) rather than invariant under the same transformations, equivariant constraints must be directly encoded into the attention mechanism. Typical scenarios include: 3D point clouds / molecules ($E(3)$ rigid-body group), image classification ($D_n$ rotation/reflection group), set data ($S_n$ permutation group), and multi-view / multi-sensor fusion.
+
+## Mathematical Inspiration
+- Lenses: [symmetry, categorical (unified framework for group actions)]
+- Knowledge: [`../../knowledge-base/probability/concentration-inequality.md` (sample efficiency gains under equivariant constraints -- data equivalence along orbits), `../../knowledge-base/probability/entropy.md` (equivariant constraints reduce output distribution entropy, yielding stronger inductive bias)]
+
+## Required Mathematical Knowledge
+- **Group Representation Theory Basics**: Linear representation of a group $G$, $\rho: G \to GL(V)$, and irreducible representation decomposition
+- **Equivariant Map Definition**: $f(g \cdot x) = \rho_{\text{out}}(g) \cdot f(x)$ for all $g \in G$
+- **Orbit-Stabilizer Theorem** (see `references/books/abstract-algebra.md` Ch.5): $|orbit| = |G|/|stab|$, giving the parameter sharing multiplier
+- **Schur's Lemma**: An equivariant linear map between irreducible representations is either zero or a scalar multiple
+
+## AI Module Form
+
+**Core Idea**: Replace the standard attention $Q, K, V$ with **steerable features**, ensuring that attention weights are invariant under group actions and that outputs are equivariant under group actions.
+
+**Scheme A: Permutation-Equivariant Attention ($S_n$ group, set data)**:
+```python
+# DeepSets / Set Transformer style
+# Attention weights are permutation-invariant: pi(Q)pi(K)^T = QK^T (permutations cancel)
+# Output is permutation-equivariant: pi(softmax(QK^T) V) = softmax(QK^T) pi(V)
+Q, K, V = W_q(X), W_k(X), W_v(X)  # pointwise linear transform
+scores = Q @ K.T / sqrt(d)         # permutation-invariant
+attn = softmax(scores)             # permutation-invariant
+output = attn @ V                  # permutation-equivariant (V is equivariant)
+```
+
+**Scheme B: $E(3)$-Equivariant Attention (3D point clouds / molecules)**:
+```python
+# Decompose features into scalars + vectors + higher-order tensors (spherical harmonic basis)
+# Attention weights computed using only scalar features (rotation-invariant)
+scalar_Q = scalar_proj(X_scalar)   # scalars only -> rotation-invariant
+scalar_K = scalar_proj(X_scalar)
+scores = scalar_Q @ scalar_K.T / sqrt(d_s)  # rotation-invariant attention weights
+
+# V contains equivariant features (scalars + vectors), weighted by invariant weights
+output_scalar = softmax(scores) @ V_scalar   # scalar -> invariant
+output_vector = softmax(scores) @ V_vector   # vector -> equivariant (rotation-covariant)
+```
+
+**Scheme C: $D_n$-Equivariant Attention (image rotation / reflection)**:
+```python
+# G-CNN style: for each group element g in D_n, transform input with rho(g) then compute attention
+# Weights shared along orbits (same W_q/W_k/W_v), aggregate outputs over all group elements
+output = mean(softmax((rho(g)@X@W_q) @ (rho(g)@X@W_k).T/sqrt(d)) @ (rho(g)@X@W_v) for g in D_n)
+```
+
+## Implementable Architectures
+- **SE(3)-Transformer / Equiformer**: Spherical harmonic features + equivariant attention for molecular property prediction and protein structure
+- **Set Transformer**: $S_n$ permutation-equivariant attention + Induced Set Attention (low-rank inducing points for complexity reduction)
+- **G-CNN Attention**: $D_n$ rotation/reflection equivariance for remote sensing imagery and medical imaging
+
+## GPU Feasibility
+- **Dimension 1 Tensorization**: Group actions implemented as $\rho(g)$ matrix multiplications; equivariant features stored as batched tensors
+- **Dimension 2 GEMM-mappability**: $\rho(g) X$ and $Q K^T$ are both GEMM operations; $|G|$ group elements map to batched GEMM
+- **Dimension 3 Complexity**: $|G|$-fold computation overhead; acceptable for small groups ($|D_4|=8$), infeasible for large groups ($|S_n|=n!$). Remedy: use generators + Cayley graph propagation instead of full group enumeration
+- **Dimension 4 Memory**: Requires storing $|G|$ copies of intermediate features; mitigated by chunking + gradient checkpointing
+- **Dimension 5 Low Precision**: Orthogonal representation matrices are numerically stable under bf16
+- **Dimension 6 Parallelism**: $|G|$ group elements are naturally parallelizable (along the batch dimension)
+- **Dimension 7 Sparsity**: Permutation $\rho(g)$ is extremely sparse and can be encoded as gather indices
+- **Dimension 8 Operator Fusion**: Group action + linear duality can be fused into a single batched GEMM
+
+## Paper Phrasing
+"We propose an equivariant attention mechanism that constrains attention weights to be group invariants and attention outputs to be group equivariants, directly encoding the inductive bias of symmetry group $G$ into the model architecture without additional data augmentation, achieving a $|G|/|stab|$-fold improvement in parameter efficiency."
+
+## Risks
+- **Cost of Incorrect Group Selection**: If the data does not possess the assumed symmetry (e.g., molecules lacking full $E(3)$ symmetry), equivariant constraints will impair expressiveness. The symmetry assumption must be validated first, or "approximate equivariance" (soft equivariance) should be used.
+- **Memory Explosion of High-Order Representations**: The $L$-th order spherical harmonic representation of $SO(3)$ has dimension $(2L+1)^2$, causing storage and computation to grow rapidly for high-order features ($L \geq 3$). In practice, truncation to $L \leq 2$ is standard.

package/design-patterns/attention/equivariant-attention.md

@@ -0,0 +1,72 @@
+# 等变注意力 / Equivariant Attention
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+
+## 适用问题
+当输入具有**明确的对称群 $G$ 作用**（旋转、平移、置换、反射等），且期望模型输出在相同变换下**协变**（equivariant）而非不变时，需要将等变约束直接编入注意力机制。典型场景：3D 点云/分子（$E(3)$ 刚体群）、图像分类（$D_n$ 旋转/镜像群）、集合数据（$S_n$ 置换群）、多视角/多传感器融合。
+
+## 数学思想来源
+- 透镜：[symmetry（对称透镜）, categorical（范畴化透镜 — 群作用的统一框架）]
+- 知识：[`../../knowledge-base/probability/concentration-inequality.md`（等变约束下的样本效率提升 — 轨道上的数据等价性）, `../../knowledge-base/probability/entropy.md`（等变约束降低输出分布熵 → 更强的归纳偏置）]
+
+## 需要的数学知识
+- **群表示论基础**：群 $G$ 的线性表示 $\rho: G \to GL(V)$，不可约表示分解
+- **等变映射定义**：$f(g \cdot x) = \rho_{\text{out}}(g) \cdot f(x)$，对所有 $g \in G$
+- **轨道-稳定子定理**（参见 `references/books/abstract-algebra.md` Ch.5）：$|orbit| = |G|/|stab|$，参数共享倍率
+- **Schur 引理**：不可约表示间的等变线性映射要么为零要么为标量乘
+
+## AI 模块形式
+
+**核心思路**：将标准注意力的 $Q, K, V$ 替换为**等变特征**（steerable features），确保注意力权重在群作用下不变，输出在群作用下等变。
+
+**方案 A：置换等变注意力（$S_n$ 群，集合数据）**：
+```python
+# DeepSets / Set Transformer 风格
+# 注意力权重对置换不变：π(Q)π(K)^T = QK^T（置换抵消）
+# 输出对置换等变：π(softmax(QK^T) V) = softmax(QK^T) π(V)
+Q, K, V = W_q(X), W_k(X), W_v(X)  # 逐点线性变换
+scores = Q @ K.T / sqrt(d)         # 置换不变
+attn = softmax(scores)             # 置换不变
+output = attn @ V                  # 置换等变（V 是等变的）
+```
+
+**方案 B：$E(3)$ 等变注意力（3D 点云/分子）**：
+```python
+# 将特征分解为标量 + 向量 + 高阶张量（球谐基）
+# 注意力权重只用标量特征计算（旋转不变）
+scalar_Q = scalar_proj(X_scalar)   # 仅标量 → 旋转不变
+scalar_K = scalar_proj(X_scalar)
+scores = scalar_Q @ scalar_K.T / sqrt(d_s)  # 旋转不变的注意力权重
+
+# V 包含等变特征（标量 + 向量），用不变的权重加权
+output_scalar = softmax(scores) @ V_scalar   # 标量 → 不变
+output_vector = softmax(scores) @ V_vector   # 向量 → 等变（旋转协变）
+```
+
+**方案 C：$D_n$ 等变注意力（图像旋转/镜像）**：
+```python
+# G-CNN 风格：对每个群元素 g ∈ D_n，用 ρ(g) 变换输入后计算注意力
+# 权重沿轨道共享（同一组 W_q/W_k/W_v），聚合所有群元素输出
+output = mean(softmax((rho(g)@X@W_q) @ (rho(g)@X@W_k).T/sqrt(d)) @ (rho(g)@X@W_v) for g in D_n)
+```
+
+## 可实现结构
+- **SE(3)-Transformer / Equiformer**：球谐特征 + 等变注意力，用于分子性质预测和蛋白质结构
+- **Set Transformer**：$S_n$ 置换等变注意力 + Induced Set Attention（低秩诱导点降复杂度）
+- **G-CNN 注意力**：$D_n$ 旋转/镜像等变，用于遥感图像和医学影像
+
+## GPU 可行性
+- **维度 1 张量化 ✅**：群作用实现为 $\rho(g)$ 矩阵乘，等变特征为 batched 张量
+- **维度 2 GEMM 可映射 ✅**：$\rho(g) X$ 和 $Q K^T$ 均为 GEMM；$|G|$ 个群元素 → batched GEMM
+- **维度 3 复杂度 ⚠️**：$|G|$ 倍计算量，小群（$|D_4|=8$）可接受，大群（$|S_n|=n!$）不可。改造：用生成元 + Cayley 图传播替代整群枚举
+- **维度 4 显存 ⚠️**：需存储 $|G|$ 份中间特征，可分块 + gradient checkpointing
+- **维度 5 低精度 ✅**：正交表示矩阵在 bf16 下数值稳定
+- **维度 6 并行 ✅**：$|G|$ 个群元素天然并行（batch 维）
+- **维度 7 稀疏 ⚠️**：置换 $\rho(g)$ 极度稀疏，可编码为 gather 索引
+- **维度 8 算子融合 ✅**：群作用 + 线性变换可融合为单次 batched GEMM
+
+## 论文表述方式
+"我们提出等变注意力机制，通过将注意力权重约束为群不变量、注意力输出约束为群等变量，在不增加数据增强的情况下将对称群 $G$ 的归纳偏置直接编码进模型结构，参数效率提升 $|G|/|stab|$ 倍。"
+
+## 风险
+- **群选择错误的代价**：若数据不具有假定的对称性（如分子不具有完全 $E(3)$ 对称性），等变约束会伤害表达能力。需先验证对称性假设，或使用"近似等变"（soft equivariance）。
+- **高阶表示的显存爆炸**：$SO(3)$ 的 $L$ 阶球谐表示维度为 $(2L+1)^2$，高阶特征 ($L \geq 3$) 的存储和计算量急剧增长。实践中通常截断到 $L \leq 2$。

package/design-patterns/attention/geometry-aware-attention.en.md

@@ -0,0 +1,73 @@
+# Geometry-Aware Attention
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+
+## Applicable Problems
+When **known geometric relationships** exist between tokens/keys (spatial distance, manifold geodesic distance, hierarchical structure, temporal interval), the position-agnostic inner product of standard attention cannot exploit this structural information. Geometry-aware attention **directly injects geometric priors into attention weight computation**, enabling the model to naturally respect the metric structure of the underlying space. Typical scenarios include: 3D scene understanding, molecular conformation, temporal forecasting, hierarchical text structure (paragraph-sentence-word), and knowledge graphs.
+
+## Mathematical Inspiration
+- Lenses: [symmetry (metric invariance), duality (coordinate-system-independent representation)]
+- Knowledge: [`../../knowledge-base/information-geometry/fisher-metric.md` (geometric metrics on distribution spaces), `../../knowledge-base/probability/concentration-inequality.md` (concentration behavior under geometric constraints)]
+
+## Required Mathematical Knowledge
+- **Metric Spaces and Distance Functions**: Euclidean distance, geodesic distance, tree distance, Wasserstein distance
+- **RBF / Kernel Methods**: $k(x, y) = \exp(-d(x,y)^2 / 2\sigma^2)$, converting distance to similarity
+- **Geometric Interpretation of Positional Encoding**: RoPE = action of the rotation group $SO(2)$ (relative distance encoded as rotation angle difference); ALiBi = exponentially decaying distance bias
+
+## AI Module Form
+
+**Core Idea**: Explicitly introduce a geometric distance term into attention scores so that attention to distant tokens naturally decays:
+
+$$\text{GeoAttn}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d}} + \text{GeoBias}(i, j)\right) V$$
+
+**Scheme A: Distance-Biased Attention (General Metric Spaces)**:
+```python
+# d_ij is the geometric distance between token i and j (precomputed or computed online)
+# Learnable distance bias function
+distance_bias = MLP(d_ij)  # or simply -alpha * d_ij
+scores = (Q @ K.T) / sqrt(d) + distance_bias
+attn = softmax(scores) @ V
+```
+
+**Scheme B: Relative Positional Encoding (Geometric Generalization of RoPE / ALiBi)**:
+```python
+# Generalize RoPE: position -> group element g_i, relative position -> g_i g_j^{-1}
+# Learnable relative position bias on arbitrary metric spaces
+def geo_attention(Q, K, V, positions):
+    rel_pos = pairwise_difference(positions)       # (n, n, coord_dim)
+    geo_bias = geo_encoder(rel_pos)               # (n, n) learnable mapping
+    scores = (Q @ K.T) / sqrt(d) + geo_bias
+    return softmax(scores) @ V
+```
+
+**Scheme C: Manifold-Aware Attention (Non-Euclidean Spaces)**:
+```python
+# When tokens reside on a non-Euclidean manifold (hyperbolic space, sphere),
+# use geodesic distance instead of Euclidean distance
+def manifold_attention(Q, K, V, manifold):
+    geodist = manifold.geodesic_distance_matrix(positions)  # (n, n)
+    geo_kernel = exp(-geodist^2 / (2 * sigma^2))  # RBF kernel
+    scores = (Q @ K.T) / sqrt(d) + log(geo_kernel + eps)
+    return softmax(scores) @ V
+```
+
+## Implementable Architectures
+- **RoPE Extensions**: From $SO(2)$ rotations to higher-dimensional rotation groups $SO(2k)$, encoding multi-dimensional positional information (2D image patches, 3D voxels)
+- **Hierarchical Positional Bias**: Use LCA (Lowest Common Ancestor) depth as distance in tree structures, suitable for hierarchical geometric of documents and code
+- **Molecular Conformation Attention**: 3D atomic coordinates -> distance matrix -> geometric bias, used in molecular GNNs and protein structure prediction
+
+## GPU Feasibility
+- **Dimension 1 Tensorization**: Distance matrices and bias matrices are dense tensors with element-wise operations
+- **Dimension 2 GEMM-mappability**: The main body $Q K^T$ is standard GEMM; geometric bias is additive and does not interrupt GEMM
+- **Dimension 3 Complexity**: Pairwise distance matrix construction and storage are $O(n^2)$; however, block-wise computation + online softmax can avoid materializing the full matrix
+- **Dimension 4 Memory**: The $n \times n$ distance matrix occupies GPU memory; mitigated by block/streaming computation (compatible with FlashAttention)
+- **Dimension 5 Low Precision**: Distance computation and bias addition are stable under bf16; the exp in RBF requires overflow caution (clamp distances)
+- **Dimension 6 Parallelism**: Distance computation and attention can be pipelined; pairwise distances can be computed in parallel blocks
+- **Dimension 7 Sparsity**: Distant biases tend toward $-\infty$ (approaching zero after softmax), naturally inducing structured sparsity (local attention windows)
+- **Dimension 8 Operator Fusion**: Geometric bias can be fused into FlashAttention's online softmax loop (added after $QK^T$, before softmax)
+
+## Paper Phrasing
+"We propose geometry-aware attention, which explicitly injects learnable geometric distance bias terms into attention scores, enabling the model to naturally respect the metric structure of the input space. This achieves exponential decay of attention to distant tokens without increasing parameter count, while maintaining compatibility with FlashAttention."
+
+## Risks
+- **Geometric Prior Conflicting with Data**: If geometric distance is inconsistent with semantic relevance (e.g., distant but semantically related tokens in text), strong geometric bias will impair model expressiveness. The bias should be learnable and overridable by content-based attention.
+- **Non-Differentiability of Distance Computation**: Certain distances (graph shortest path, tree LCA depth) are non-differentiable or computationally expensive. Differentiable relaxations (softmin instead of min) or precomputed distance matrices are needed.

package/design-patterns/attention/geometry-aware-attention.md

@@ -0,0 +1,72 @@
+# 几何感知注意力 / Geometry-Aware Attention
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+
+## 适用问题
+当 token/key 之间存在**已知的几何关系**（空间距离、流形测地距、层级结构、时序间隔）时，标准注意力的位置无关内积无法利用这些结构信息。几何感知注意力将**几何先验直接注入注意力权重计算**，使模型天然尊重底空间的度量结构。典型场景：3D 场景理解、分子构象、时序预测、层级文本结构（段落-句子-词）、知识图谱。
+
+## 数学思想来源
+- 透镜：[symmetry（对称透镜 — 度量不变性）, duality（对偶透镜 — 坐标系无关表达）]
+- 知识：[`../../knowledge-base/information-geometry/fisher-metric.md`（分布空间的几何度量）, `../../knowledge-base/probability/concentration-inequality.md`（几何约束下的浓度行为）]
+
+## 需要的数学知识
+- **度量空间与距离函数**：欧氏距离、测地距、树距离、Wasserstein 距离
+- **RBF / 核方法**：$k(x, y) = \exp(-d(x,y)^2 / 2\sigma^2)$，距离→相似度的转换
+- **位置编码的几何解释**：RoPE = 旋转群 $SO(2)$ 的作用（相对距离编码为旋转角度差）；ALiBi = 指数衰减的距离偏置
+
+## AI 模块形式
+
+**核心思路**：在注意力分数中显式引入几何距离项，使远距离 token 的注意力天然衰减：
+
+$$\text{GeoAttn}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d}} + \text{GeoBias}(i, j)\right) V$$
+
+**方案 A：距离偏置注意力（通用度量空间）**：
+```python
+# d_ij 为 token i 和 j 之间的几何距离（预计算或在线计算）
+# 可学习距离偏置函数
+distance_bias = MLP(d_ij)  # 或简单的 -α * d_ij
+scores = (Q @ K.T) / sqrt(d) + distance_bias
+attn = softmax(scores) @ V
+```
+
+**方案 B：相对位置编码（RoPE / ALiBi 的几何推广）**：
+```python
+# 推广 RoPE：位置 → 群元素 g_i，相对位置 → g_i g_j^{-1}
+# 可学习任意度量空间上的相对位置偏置
+def geo_attention(Q, K, V, positions):
+    rel_pos = pairwise_difference(positions)       # (n, n, coord_dim)
+    geo_bias = geo_encoder(rel_pos)               # (n, n) 可学习映射
+    scores = (Q @ K.T) / sqrt(d) + geo_bias
+    return softmax(scores) @ V
+```
+
+**方案 C：流形感知注意力（非欧空间）**：
+```python
+# token 位于非欧流形（双曲空间、球面）时，用测地距替代欧氏距
+def manifold_attention(Q, K, V, manifold):
+    geodist = manifold.geodesic_distance_matrix(positions)  # (n, n)
+    geo_kernel = exp(-geodist^2 / (2 * sigma^2))  # RBF 核
+    scores = (Q @ K.T) / sqrt(d) + log(geo_kernel + eps)
+    return softmax(scores) @ V
+```
+
+## 可实现结构
+- **RoPE 扩展**：从 $SO(2)$ 旋转到更高维旋转群 $SO(2k)$，编码多维位置信息（2D 图像 patch、3D voxel）
+- **层级位置偏置**：树结构中用 LCA（最近公共祖先）深度作为距离，适合文档/代码的层级建模
+- **分子构象注意力**：3D 原子坐标 → 距离矩阵 → 几何偏置，用于分子 GNN 和蛋白质结构预测
+
+## GPU 可行性
+- **维度 1 张量化 ✅**：距离矩阵和偏置矩阵均为稠密张量，逐元素运算
+- **维度 2 GEMM 可映射 ✅**：主体 $Q K^T$ 为标准 GEMM；几何偏置为加法，不阻断 GEMM
+- **维度 3 复杂度 ⚠️**：成对距离矩阵 $O(n^2)$ 构建和存储；但可用分块计算 + online softmax 避免物化完整矩阵
+- **维度 4 显存 ⚠️**：$n \times n$ 距离矩阵占用显存；可分块/流式计算（与 FlashAttention 兼容）
+- **维度 5 低精度 ✅**：距离计算和偏置加法在 bf16 下稳定；RBF 的 exp 需注意上溢（clamp 距离）
+- **维度 6 并行 ✅**：距离计算和注意力可流水线并行，成对距离可分块并行
+- **维度 7 稀疏 ✅**：远距离偏置趋于 $-\infty$（softmax 后趋零），天然诱导结构化稀疏（局部注意力窗口）
+- **维度 8 算子融合 ✅**：几何偏置可融入 FlashAttention 的 online softmax 循环中（加在 $QK^T$ 之后、softmax 之前）
+
+## 论文表述方式
+"我们提出几何感知注意力，通过在注意力分数中显式注入可学习的几何距离偏置项，使模型天然尊重输入空间的度量结构，在不增加参数量的情况下将远距离 token 的注意力指数衰减，同时保持与 FlashAttention 的兼容性。"
+
+## 风险
+- **几何先验与数据冲突**：若几何距离与语义相关性不一致（如文本中远距离但语义相关的 token），强几何偏置会伤害模型表达。需让偏置可学习且可被内容注意力覆盖。
+- **距离计算的非可微性**：某些距离（图最短路径、树 LCA 深度）不可微或计算复杂。需使用可微松弛（softmin 替代 min）或预计算距离矩阵。