math-skill 3.0.1 → 3.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (154) hide show
  1. package/README.en-US.md +42 -30
  2. package/README.md +18 -6
  3. package/agents/math-critic.en.md +6 -6
  4. package/agents/math-critic.md +6 -6
  5. package/commands/ask.en.md +6 -1
  6. package/commands/ask.md +6 -1
  7. package/design-patterns/attention/equivariant-attention.en.md +8 -2
  8. package/design-patterns/attention/equivariant-attention.md +7 -1
  9. package/design-patterns/attention/geometry-aware-attention.en.md +2 -2
  10. package/design-patterns/attention/geometry-aware-attention.md +1 -1
  11. package/design-patterns/attention/information-bottleneck-attention.en.md +32 -12
  12. package/design-patterns/attention/information-bottleneck-attention.md +32 -12
  13. package/design-patterns/attention/projection-attention.en.md +3 -3
  14. package/design-patterns/attention/projection-attention.md +2 -2
  15. package/design-patterns/attention/spectral-attention.en.md +19 -5
  16. package/design-patterns/attention/spectral-attention.md +18 -4
  17. package/design-patterns/compression/leverage-score-selection.en.md +13 -13
  18. package/design-patterns/compression/leverage-score-selection.md +13 -13
  19. package/design-patterns/compression/low-rank-kv-cache.en.md +39 -12
  20. package/design-patterns/compression/low-rank-kv-cache.md +38 -12
  21. package/design-patterns/compression/spectral-token-pruning.en.md +16 -14
  22. package/design-patterns/compression/spectral-token-pruning.md +16 -14
  23. package/design-patterns/compression/topology-preserving-compression.en.md +10 -10
  24. package/design-patterns/compression/topology-preserving-compression.md +10 -10
  25. package/design-patterns/loss/constraint-penalty.en.md +8 -4
  26. package/design-patterns/loss/constraint-penalty.md +9 -6
  27. package/design-patterns/loss/contrastive-loss.en.md +37 -11
  28. package/design-patterns/loss/contrastive-loss.md +38 -12
  29. package/design-patterns/loss/information-bottleneck-loss.en.md +5 -5
  30. package/design-patterns/loss/information-bottleneck-loss.md +6 -6
  31. package/design-patterns/loss/orthogonality-loss.en.md +24 -17
  32. package/design-patterns/loss/orthogonality-loss.md +24 -18
  33. package/design-patterns/loss/variational-loss.en.md +4 -4
  34. package/design-patterns/loss/variational-loss.md +5 -5
  35. package/design-patterns/overview.en.md +36 -0
  36. package/design-patterns/overview.md +36 -0
  37. package/design-patterns/representation/equivariant-split.en.md +4 -4
  38. package/design-patterns/representation/equivariant-split.md +4 -5
  39. package/design-patterns/representation/manifold-representation.en.md +12 -12
  40. package/design-patterns/representation/manifold-representation.md +13 -14
  41. package/design-patterns/representation/shared-private-decomposition.en.md +12 -7
  42. package/design-patterns/representation/shared-private-decomposition.md +12 -9
  43. package/design-patterns/representation/subspace-alignment.en.md +12 -12
  44. package/design-patterns/representation/subspace-alignment.md +12 -12
  45. package/design-patterns/routing/graph-routing.en.md +11 -11
  46. package/design-patterns/routing/graph-routing.md +11 -11
  47. package/design-patterns/routing/moe-routing.en.md +12 -12
  48. package/design-patterns/routing/moe-routing.md +12 -14
  49. package/design-patterns/routing/optimal-transport-routing.en.md +5 -5
  50. package/design-patterns/routing/optimal-transport-routing.md +6 -6
  51. package/design-patterns/routing/spectral-clustering-routing.en.md +26 -15
  52. package/design-patterns/routing/spectral-clustering-routing.md +26 -17
  53. package/knowledge-base/differential-geometry/connection.en.md +17 -2
  54. package/knowledge-base/differential-geometry/connection.md +17 -2
  55. package/knowledge-base/differential-geometry/curvature.en.md +14 -1
  56. package/knowledge-base/differential-geometry/curvature.md +14 -1
  57. package/knowledge-base/differential-geometry/geodesic.en.md +16 -2
  58. package/knowledge-base/differential-geometry/geodesic.md +16 -2
  59. package/knowledge-base/differential-geometry/index.en.md +52 -0
  60. package/knowledge-base/differential-geometry/index.md +52 -0
  61. package/knowledge-base/differential-geometry/manifold.en.md +15 -2
  62. package/knowledge-base/differential-geometry/manifold.md +15 -2
  63. package/knowledge-base/differential-geometry/metric-tensor.en.md +16 -2
  64. package/knowledge-base/differential-geometry/metric-tensor.md +16 -2
  65. package/knowledge-base/differential-geometry/tangent-space.en.md +17 -3
  66. package/knowledge-base/differential-geometry/tangent-space.md +17 -3
  67. package/knowledge-base/information-geometry/fisher-metric.en.md +24 -5
  68. package/knowledge-base/information-geometry/fisher-metric.md +24 -5
  69. package/knowledge-base/information-geometry/index.en.md +45 -0
  70. package/knowledge-base/information-geometry/index.md +45 -0
  71. package/knowledge-base/information-geometry/natural-gradient.en.md +17 -3
  72. package/knowledge-base/information-geometry/natural-gradient.md +17 -3
  73. package/knowledge-base/lie-theory/equivariance.en.md +18 -3
  74. package/knowledge-base/lie-theory/equivariance.md +18 -3
  75. package/knowledge-base/lie-theory/group-action.en.md +16 -2
  76. package/knowledge-base/lie-theory/group-action.md +16 -2
  77. package/knowledge-base/lie-theory/index.en.md +52 -0
  78. package/knowledge-base/lie-theory/index.md +52 -0
  79. package/knowledge-base/lie-theory/lie-algebra.en.md +17 -2
  80. package/knowledge-base/lie-theory/lie-algebra.md +17 -2
  81. package/knowledge-base/lie-theory/lie-group.en.md +17 -3
  82. package/knowledge-base/lie-theory/lie-group.md +17 -3
  83. package/knowledge-base/lie-theory/representation.en.md +17 -2
  84. package/knowledge-base/lie-theory/representation.md +17 -2
  85. package/knowledge-base/matrix-analysis/index.en.md +49 -0
  86. package/knowledge-base/matrix-analysis/index.md +49 -0
  87. package/knowledge-base/matrix-analysis/low-rank-approximation.en.md +20 -7
  88. package/knowledge-base/matrix-analysis/low-rank-approximation.md +20 -7
  89. package/knowledge-base/matrix-analysis/matrix-perturbation.en.md +13 -1
  90. package/knowledge-base/matrix-analysis/matrix-perturbation.md +13 -1
  91. package/knowledge-base/matrix-analysis/positive-semidefinite.en.md +20 -6
  92. package/knowledge-base/matrix-analysis/positive-semidefinite.md +20 -6
  93. package/knowledge-base/matrix-analysis/projection.en.md +14 -1
  94. package/knowledge-base/matrix-analysis/projection.md +14 -1
  95. package/knowledge-base/matrix-analysis/spectral-decomposition.en.md +14 -1
  96. package/knowledge-base/matrix-analysis/spectral-decomposition.md +14 -1
  97. package/knowledge-base/optimization/constrained-optimization.en.md +17 -3
  98. package/knowledge-base/optimization/constrained-optimization.md +17 -3
  99. package/knowledge-base/optimization/convex-optimization.en.md +15 -2
  100. package/knowledge-base/optimization/convex-optimization.md +15 -2
  101. package/knowledge-base/optimization/index.en.md +50 -0
  102. package/knowledge-base/optimization/index.md +50 -0
  103. package/knowledge-base/optimization/lagrangian-duality.en.md +18 -4
  104. package/knowledge-base/optimization/lagrangian-duality.md +18 -4
  105. package/knowledge-base/optimization/proximal-method.en.md +15 -1
  106. package/knowledge-base/optimization/proximal-method.md +15 -1
  107. package/knowledge-base/optimization/riemannian-optimization.en.md +18 -4
  108. package/knowledge-base/optimization/riemannian-optimization.md +18 -4
  109. package/knowledge-base/overview.en.md +20 -8
  110. package/knowledge-base/overview.md +20 -8
  111. package/knowledge-base/probability/concentration-inequality.en.md +16 -2
  112. package/knowledge-base/probability/concentration-inequality.md +16 -2
  113. package/knowledge-base/probability/entropy.en.md +19 -3
  114. package/knowledge-base/probability/entropy.md +18 -2
  115. package/knowledge-base/probability/fisher-information.en.md +17 -3
  116. package/knowledge-base/probability/fisher-information.md +17 -3
  117. package/knowledge-base/probability/index.en.md +51 -0
  118. package/knowledge-base/probability/index.md +51 -0
  119. package/knowledge-base/probability/information-bottleneck.en.md +18 -2
  120. package/knowledge-base/probability/information-bottleneck.md +18 -2
  121. package/knowledge-base/probability/kl-divergence.en.md +15 -2
  122. package/knowledge-base/probability/kl-divergence.md +15 -2
  123. package/knowledge-base/topology/euler-characteristic.en.md +15 -2
  124. package/knowledge-base/topology/euler-characteristic.md +15 -2
  125. package/knowledge-base/topology/fundamental-group.en.md +16 -2
  126. package/knowledge-base/topology/fundamental-group.md +16 -2
  127. package/knowledge-base/topology/index.en.md +48 -0
  128. package/knowledge-base/topology/index.md +48 -0
  129. package/knowledge-base/topology/persistent-homology.en.md +17 -2
  130. package/knowledge-base/topology/persistent-homology.md +17 -2
  131. package/lenses/spectral.en.md +1 -1
  132. package/lenses/spectral.md +1 -1
  133. package/lenses/topological.en.md +1 -1
  134. package/lenses/topological.md +1 -1
  135. package/package.json +5 -5
  136. package/references/agentic-workflow.en.md +2 -2
  137. package/references/books/abstract-algebra.en.md +12 -12
  138. package/references/books/abstract-algebra.md +9 -9
  139. package/references/books/algebraic-geometry-rising-sea.en.md +2 -2
  140. package/references/books/differential-geometry.en.md +16 -16
  141. package/references/books/differential-geometry.md +14 -14
  142. package/references/books/matrix-analysis.en.md +3 -3
  143. package/references/books/matrix-analysis.md +24 -24
  144. package/references/books/micro-lie-theory.en.md +1 -1
  145. package/references/books/optimization-ml.en.md +15 -15
  146. package/references/books/optimization-ml.md +14 -14
  147. package/references/books/smooth-manifolds.en.md +3 -3
  148. package/references/books/smooth-manifolds.md +6 -6
  149. package/references/gpu-friendly-math.en.md +21 -8
  150. package/references/gpu-friendly-math.md +19 -4
  151. package/references/skill-index.en.md +105 -0
  152. package/references/skill-index.md +105 -0
  153. package/skills/math-research-activator/SKILL.en.md +70 -112
  154. package/skills/math-research-activator/SKILL.md +60 -100
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "math-skill",
3
- "version": "3.0.1",
3
+ "version": "3.1.0",
4
4
  "description": "面向 AI 架构创新的数学研究操作系统 v3:三层正交架构(思想透镜 / 数学知识库 / 设计翻译层),自动诊断用户意图并路由。Math Research OS for AI architecture design: thinking lenses + math knowledge base + design translation layer with auto intent diagnosis.",
5
5
  "keywords": [
6
6
  "math",
@@ -71,10 +71,10 @@
71
71
  "references/"
72
72
  ],
73
73
  "scripts": {
74
- "validate": "bash tests/validate.sh || powershell -ExecutionPolicy Bypass -File tests/validate.ps1",
75
- "validate:sh": "bash tests/validate.sh",
76
- "validate:win": "powershell -ExecutionPolicy Bypass -File tests/validate.ps1",
74
+ "validate": "node -e \"const fs=require('fs'),cp=require('child_process');if(fs.existsSync('tests/validate.sh'))process.exit(cp.spawnSync('bash',['tests/validate.sh'],{stdio:'inherit'}).status||0);if(fs.existsSync('tests/validate.ps1'))process.exit(cp.spawnSync('powershell',['-ExecutionPolicy','Bypass','-File','tests/validate.ps1'],{stdio:'inherit'}).status||0);console.log('tests/ not available in published package');\"",
75
+ "validate:sh": "test -f tests/validate.sh && bash tests/validate.sh || echo 'tests/ not available in published package'",
76
+ "validate:win": "powershell -ExecutionPolicy Bypass -Command \"if (Test-Path tests/validate.ps1) { & tests/validate.ps1 } else { Write-Host 'tests/ not available in published package' }\"",
77
77
  "pack": "npm pack",
78
78
  "prepublishOnly": "npm run validate"
79
79
  }
80
- }
80
+ }
@@ -7,7 +7,7 @@
7
7
 
8
8
  1. **The mathematical foundation of this AI revolution is the first time 20th-century mathematics has stepped onto the commercial computing stage** — category theory, algebraic topology, algebraic geometry. The mathematical foundations of most mainstream algorithms still largely rest on the calculus and linear algebra of 1800–1900. Activating modern mathematics in algorithm design is the most important task during the algorithm exploration phase.
9
9
  2. **Humility**: The breadth of knowledge from the pre-training stage already surpasses any individual. **More knowledge is learned from the model than taught to it.** Letting go of the bias toward "teaching the model" usually yields better results.
10
- 3. **Mathematical beauty ≠ computability**: Many mathematically elegant constructs break down in the face of GPU parallelism and low-precision arithmetic errors. Every deliverable must pass the acceptance gate defined in `gpu-friendly-math.md`.
10
+ 3. **Mathematical beauty ≠ computability**: Many mathematically elegant constructs break down in the face of GPU parallelism and low-precision arithmetic errors. Every deliverable must pass the acceptance gate defined in `gpu-friendly-math.en.md`.
11
11
 
12
12
  ## Division of Labor
13
13
 
@@ -28,7 +28,7 @@
28
28
 
29
29
  1. **Diagnosis**: What is the algorithmic structure / bottleneck of the problem? (Do not start by piling on math tutorials.)
30
30
  2. **Cross-domain activation**: Scan `books/*.md` for transferable modern mathematical structures, **enumerate multiple candidates** (leveraging the model's large-context advantage).
31
- 3. **Dual acceptance gate**: Each candidate passes two gates — (1) Mathematically correct (differentiable, self-consistent, with correctness guarantees); (2) GPU-feasible (the eight dimensions of `gpu-friendly-math.md`).
31
+ 3. **Dual acceptance gate**: Each candidate passes two gates — (1) Mathematically correct (differentiable, self-consistent, with correctness guarantees); (2) GPU-feasible (the eight dimensions of `gpu-friendly-math.en.md`).
32
32
  4. **Iterative tracking**: Record candidates, scores, and status in a markdown test plan, converging step by step.
33
33
 
34
34
  ## Markdown Test Plan Template / Tracking Template
@@ -51,7 +51,7 @@ Actual chapter map (grouped by topic; chapter numbers taken from the book's tabl
51
51
  ## Possible Algorithmic Inspirations
52
52
 
53
53
  1. **G-equivariant layer** (source: 5, 6, 10): Implement the symmetry group G's action as permutation/rotation matrices rho(g), with weights shared along orbits, enforcing f(g x)=rho(g) f(x). In practice, for each g in G the input is transformed by rho(g) then passed through the same set of weights, and the results are aggregated -- equivalent to expanding the convolution kernel into a [\|G\|, C_out, C_in] batched GEMM, reducing independent parameters by a factor of \|G\|. Deployed models: CNN (translation), G-CNN (D_n steerable filters), DeepSets/Set Transformer (S_n), E(3)-equivariant GNN (point clouds/molecules).
54
- 2. **Tropical / min-plus attention and routing** (source: 12-14 ring axiom relaxation): Replace softmax's (x,)+exp with (+, min/max) min-plus matmul: score(+) = min_k(Q_ik + K_kj). Hard Top-K routing -> tropical semiring **piecewise-linear gating** (sub-differentiable -- breakpoints need LogSumExp smoothing, which recovers standard softmax; an alternative to the non-differentiable argmax). Viterbi, DTW, and shortest paths are all instances of this min-plus matmul -- see the Tropical Gating example in `../gpu-friendly-math.md` (used only for low-dimensional gating and **not on Tensor Cores** -- max/min runs on CUDA cores, while the main trunk still uses standard (x,+) GEMM).
54
+ 2. **Tropical / min-plus attention and routing** (source: 12-14 ring axiom relaxation): Replace softmax's (x,)+exp with (+, min/max) min-plus matmul: score(+) = min_k(Q_ik + K_kj). Hard Top-K routing -> tropical semiring **piecewise-linear gating** (sub-differentiable -- breakpoints need LogSumExp smoothing, which recovers standard softmax; an alternative to the non-differentiable argmax). Viterbi, DTW, and shortest paths are all instances of this min-plus matmul -- see the Tropical Gating example in `../gpu-friendly-math.en.md` (used only for low-dimensional gating and **not on Tensor Cores** -- max/min runs on CUDA cores, while the main trunk still uses standard (x,+) GEMM).
55
55
  3. **Permutation-invariant aggregation** (source: 9, 29): Quotienting by the S_n action yields sum/max/mean pooling -- naturally O(n) and parallel-friendly, the algebraic foundation for DeepSets rho(Sigma phi(x_i)) and GNN message passing. Burnside counting (Ch 29) can estimate "how many essentially distinct configurations exist" at design time, for predicting parameter savings and data deduplication benefits.
56
56
  4. **Finite-field coding for Infra** (source: 22, 31): Linear codes c=mG (generator matrix G over GF(q)) for fault-tolerant storage and gradient compression, with Hamming distance providing a lower bound on error-correcting capability; Shamir secret sharing (polynomial interpolation over GF(p), recoverable from t shards) for secure aggregation in federated learning; finite-field hashing/CRC for deduplication and consistency verification. Commonality: pure int/bitwise operations, placed in pre/post-processing on data flows.
57
57
  5. **Cyclic group representations -> Frequency-domain token mixing** (source: 4, 11): The irreducible representations of Z_n are the DFT basis, giving rise to FFT convolution (O(n log n) replacing O(n^2)), RoPE (encoding positions as SO(2) rotations, i.e., unitary representations of Z), and cyclic/relative positional encodings. The finite abelian group classification theorem (Ch 11) guarantees that any such periodic structure can be decomposed into cyclic components for FFT.
@@ -59,22 +59,22 @@ Actual chapter map (grouped by topic; chapter numbers taken from the book's tabl
59
59
 
60
60
  ## GPU Friendliness Warning
61
61
 
62
- > The sole source for eight-dimension criteria and scoring rules: `../gpu-friendly-math.md` (Tensorization / GEMM-mappability / Complexity / Memory / Low-precision / Parallelism / Sparsity / Operator fusion). Here we give only per-dimension verdicts for the book's structures.
62
+ > The sole source for eight-dimension criteria and scoring rules: `../gpu-friendly-math.en.md` (Tensorization / GEMM-mappability / Complexity / Memory / Low-precision / Parallelism / Sparsity / Operator fusion). Here we give only per-dimension verdicts for the book's structures.
63
63
 
64
64
  **Focus A: Can semiring GEMM run on Tensor Cores? -- Not by default.**
65
65
 
66
- - **Dimension 2 GEMM-mappability ([x] unfriendly)**: Tensor Core hardware only performs (x,+) MAC (fp16/bf16/fp8 accumulated into fp32). Min-plus / max-plus uses (+, min/max), which is **not natively supported**; naive tropical GEMM degenerates to CUDA core scalar comparisons and cannot leverage Tensor Cores.
67
- - **Dimension 3 Complexity ([x])**: Min-plus matrix multiplication has no Strassen-style sub-cubic speedup (equivalent to the APSP hard problem); naive O(n^3).
68
- - **Dimension 5 Low-precision ([v])**: Tropical uses max/min, no exp overflow risk; numerically robust.
69
- - **Dimension 8 Differentiability / Operator fusion ([~] adaptable)**: min/max is non-differentiable; requires relaxation.
70
- - **Adaptations (-> eight-dimension friendly)**: (1) log-sum-exp smoothing of min/max recovers (x,+) stable softmax, back on Tensor Cores; (2) confine tropical operations **to low-dimensional gating only**, keeping the main trunk on standard (x,+) GEMM (the Tropical Gating example in the reference file = Dimension 1 [v] Tensorization / Dimension 2 [x] not Tensor Core GEMM / Dimension 3 [v] per-token gating sub-quadratic, Dimension 8 needs LogSumExp smoothing); (3) blocking -- standard GEMM within blocks, min-plus reduction between blocks.
66
+ - **D2 GEMM-mappability ([x] unfriendly)**: Tensor Core hardware only performs (x,+) MAC (fp16/bf16/fp8 accumulated into fp32). Min-plus / max-plus uses (+, min/max), which is **not natively supported**; naive tropical GEMM degenerates to CUDA core scalar comparisons and cannot leverage Tensor Cores.
67
+ - **D3 Complexity ([x])**: Min-plus matrix multiplication has no Strassen-style sub-cubic speedup (equivalent to the APSP hard problem); naive O(n^3).
68
+ - **D5 Low-precision ([v])**: Tropical uses max/min, no exp overflow risk; numerically robust.
69
+ - **D8 Differentiability / Operator fusion ([~] adaptable)**: min/max is non-differentiable; requires relaxation.
70
+ - **Adaptations (-> eight-dimension friendly)**: (1) log-sum-exp smoothing of min/max recovers (x,+) stable softmax, back on Tensor Cores; (2) confine tropical operations **to low-dimensional gating only**, keeping the main trunk on standard (x,+) GEMM (the Tropical Gating example in the reference file = D1 [v] Tensorization / D2 [x] not Tensor Core GEMM / D3 [v] per-token gating sub-quadratic, D8 needs LogSumExp smoothing); (3) blocking -- standard GEMM within blocks, min-plus reduction between blocks.
71
71
 
72
72
  **Focus B: Can group operations be tensorized? -- Small groups yes, large groups and exact arithmetic no.**
73
73
 
74
- - **Finite group actions (Dimension 1/2 [v])**: Permutation matrices / dense representation matrices -> batched GEMM; G-CNN has been engineered. Orthogonal rotation/permutation matrices are numerically stable under bf16 (Dimension 5 [v]).
75
- - **Large group enumeration (Dimension 3/4 [x])**: \|S_n\|=n! explodes; explicitly enumerating group elements -> memory and compute blow up. **Adaptation**: Use only generators (Ch 26 Generators and Relations) + Cayley graphs (Ch 30) for local propagation, without materializing the entire group.
76
- - **Permutation = gather/scatter (Dimension 7/1/8 [x])**: Implementation as irregular gather/scatter causes warp divergence. **Adaptation**: Pre-compile fixed permutation patterns into structured sparsity or dense indexing.
77
- - **Finite fields / modular arithmetic (Dimension 2 [x], Dimension 1/6 [v])**: mod p, GF(2^n), XOR, table lookups **are not floating-point MAC** and cannot use Tensor Cores, but are highly parallel on int/bitwise kernels. Conclusion: **suitable for pre/post-processing in coding/hashing, never to be placed in the training main-trunk GEMM**. Galois / exact-field arithmetic requires int, is non-differentiable, violating Dimension 8.
74
+ - **Finite group actions (D1/D2 [v])**: Permutation matrices / dense representation matrices -> batched GEMM; G-CNN has been engineered. Orthogonal rotation/permutation matrices are numerically stable under bf16 (D5 [v]).
75
+ - **Large group enumeration (D3/D4 [x])**: \|S_n\|=n! explodes; explicitly enumerating group elements -> memory and compute blow up. **Adaptation**: Use only generators (Ch 26 Generators and Relations) + Cayley graphs (Ch 30) for local propagation, without materializing the entire group.
76
+ - **Permutation = gather/scatter (D7/D1/D8 [x])**: Implementation as irregular gather/scatter causes warp divergence. **Adaptation**: Pre-compile fixed permutation patterns into structured sparsity or dense indexing.
77
+ - **Finite fields / modular arithmetic (D2 [x], D1/D6 [v])**: mod p, GF(2^n), XOR, table lookups **are not floating-point MAC** and cannot use Tensor Cores, but are highly parallel on int/bitwise kernels. Conclusion: **suitable for pre/post-processing in coding/hashing, never to be placed in the training main-trunk GEMM**. Galois / exact-field arithmetic requires int, is non-differentiable, violating D8.
78
78
 
79
79
  **Scoring conclusion**: Group equivariance (small groups, dense representations) and frequency-domain structures = mathematical beauty x GPU friendliness, can enter the main trunk; semirings, finite fields, exact Galois = require relaxation/isolation first, otherwise can only serve as auxiliary operators or offline tools.
80
80
 
@@ -96,7 +96,7 @@ Actual chapter map (grouped by topic; chapter numbers taken from the book's tabl
96
96
  - **Secondary: `duality`** -- homomorphisms/isomorphisms as equivalence transformations, FFT/frequency-domain transforms to simplify problems.
97
97
  - **Secondary: `algorithmic`** -- finite group counting (Burnside, Ch 29), coding (Ch 31), finite field enumeration (Ch 22).
98
98
 
99
- Typical combination chain: First `symmetry` to identify the group and invariants in the problem -> `categorical` to extract the common algebraic structure -> `axiomatization` to verify axioms truly hold (guard against false symmetry, guard against misusing subtraction) -> `duality` to realize as a learnable linear map -> finally pass through the `../gpu-friendly-math.md` eight-dimension acceptance gate.
99
+ Typical combination chain: First `symmetry` to identify the group and invariants in the problem -> `categorical` to extract the common algebraic structure -> `axiomatization` to verify axioms truly hold (guard against false symmetry, guard against misusing subtraction) -> `duality` to realize as a learnable linear map -> finally pass through the `../gpu-friendly-math.en.md` eight-dimension acceptance gate.
100
100
 
101
101
  ## Anti-patterns
102
102
 
@@ -63,18 +63,18 @@
63
63
 
64
64
  **焦点 A:半环 GEMM 能上 Tensor Core 吗?——默认不能。**
65
65
 
66
- - **维度 2 GEMM 可映射([x]不友好)**:Tensor Core 硬件只做 (×,+) 的 MAC(fp16/bf16/fp8 累加到 fp32)。min-plus / max-plus 用的是 (+, min/max),**非原生**,朴素 tropical GEMM 退化到 CUDA core 标量比较,吃不到 Tensor Core。
67
- - **维度 3 复杂度([x])**:min-plus 矩阵乘没有 Strassen 式亚立方加速(等价于 APSP 难题),朴素 O(n³)。
68
- - **维度 5 低精度([v])**:tropical 用 max/min,无 exp 上溢,反而数值稳健。
69
- - **维度 8 可微/融合([~]可改造)**:min/max 不可微,需松弛。
70
- - **改造(→ 八维转友好)**:① log-sum-exp 软化 min/max 退回 (×,+) 稳定 softmax,重上 Tensor Core;② 把热带运算**只放在低维门控**,主干仍走标准 (×,+) GEMM(范例文件中 Tropical Gating = dim 1[v]张量化 / 2[x]非 Tensor Core GEMM / 3[v]逐 token 门控亚二次,dim 8 需 LogSumExp 软化);③ 分块——块内标准 GEMM、块间 min-plus 归约。
66
+ - **D2 GEMM 可映射([x]不友好)**:Tensor Core 硬件只做 (×,+) 的 MAC(fp16/bf16/fp8 累加到 fp32)。min-plus / max-plus 用的是 (+, min/max),**非原生**,朴素 tropical GEMM 退化到 CUDA core 标量比较,吃不到 Tensor Core。
67
+ - **D3 复杂度([x])**:min-plus 矩阵乘没有 Strassen 式亚立方加速(等价于 APSP 难题),朴素 O(n³)。
68
+ - **D5 低精度([v])**:tropical 用 max/min,无 exp 上溢,反而数值稳健。
69
+ - **D8 可微/融合([~]可改造)**:min/max 不可微,需松弛。
70
+ - **改造(→ 八维转友好)**:① log-sum-exp 软化 min/max 退回 (×,+) 稳定 softmax,重上 Tensor Core;② 把热带运算**只放在低维门控**,主干仍走标准 (×,+) GEMM(范例文件中 Tropical Gating = D1[v] / D2[x] 非 Tensor Core GEMM / D3[v] 逐 token 门控亚二次,D8 需 LogSumExp 软化);③ 分块——块内标准 GEMM、块间 min-plus 归约。
71
71
 
72
72
  **焦点 B:群操作能张量化吗?——小群能,大群和精确算术不能。**
73
73
 
74
- - **有限群作用(维度 1/2 [v])**:置换矩阵 / 稠密表示矩阵 → batched GEMM,G-CNN 已工程化。正交的旋转/置换矩阵在 bf16 下数值稳定(维度 5 [v])。
75
- - **大群枚举(维度 3/4 [x])**:\|Sₙ\|=n! 爆炸,显式枚举群元素 → 显存与算力爆。**改造**:只用生成元(26 Generators and Relations)+ Cayley 图(30)做局部传播,不物化整群。
76
- - **置换 = gather/scatter(维度 7/1/8 [x])**:实现为不规则 gather/scatter 会 warp divergence。**改造**:固定置换模式预编译为结构化稀疏或 dense 索引。
77
- - **有限域 / 模算术(维度 2 [x]、维度 1/6 [v])**:mod p、GF(2ⁿ)、XOR、查表**不是浮点 MAC**,吃不到 Tensor Core,但在 int/bitwise kernel 上高并行。结论:**适合放编码/哈希的前后处理,绝不放进训练主干 GEMM**。Galois/精确域算术要求 int、不可微,违反维度 8
74
+ - **有限群作用(D1/D2 [v])**:置换矩阵 / 稠密表示矩阵 → batched GEMM,G-CNN 已工程化。正交的旋转/置换矩阵在 bf16 下数值稳定(D5 [v])。
75
+ - **大群枚举(D3/D4 [x])**:\|Sₙ\|=n! 爆炸,显式枚举群元素 → 显存与算力爆。**改造**:只用生成元(26 Generators and Relations)+ Cayley 图(30)做局部传播,不物化整群。
76
+ - **置换 = gather/scatter(D7/D1/D8 [x])**:实现为不规则 gather/scatter 会 warp divergence。**改造**:固定置换模式预编译为结构化稀疏或 dense 索引。
77
+ - **有限域 / 模算术(D2 [x]、D1/D6 [v])**:mod p、GF(2ⁿ)、XOR、查表**不是浮点 MAC**,吃不到 Tensor Core,但在 int/bitwise kernel 上高并行。结论:**适合放编码/哈希的前后处理,绝不放进训练主干 GEMM**。Galois/精确域算术要求 int、不可微,违反 D8
78
78
 
79
79
  **判分结论**:群等变(小群、稠密表示)与频域结构 = 数学美 × GPU 友好,可进主干;半环、有限域、精确 Galois = 需先松弛/隔离,否则只能当辅助算子或离线工具。
80
80
 
@@ -92,7 +92,7 @@ First the overview mapping, then detailed expansion:
92
92
 
93
93
  ## Possible Algorithmic Inspirations
94
94
 
95
- **Tropical Sheaf Attention Trilogy** (consistent with the candidate validation examples in `../gpu-friendly-math.md`):
95
+ **Tropical Sheaf Attention Trilogy** (consistent with the candidate validation examples in `../gpu-friendly-math.en.md`):
96
96
 
97
97
  1. **Tropical Gating**
98
98
  - Replace hard Top-K routing with **piecewise-linear** scoring on the max-plus semiring.
@@ -112,7 +112,7 @@ First the overview mapping, then detailed expansion:
112
112
 
113
113
  ## GPU Friendliness Warning
114
114
 
115
- > **Required reading and sole authority**: `../gpu-friendly-math.md`
115
+ > **Required reading and sole authority**: `../gpu-friendly-math.en.md`
116
116
  > Eight-dimension scorecard: D1 Tensorization, D2 GEMM-mappability, D3 Complexity, D4 Memory/KV, D5 Low-precision stability, D6 Parallelism & communication, D7 Sparse structure, D8 Operator fusion.
117
117
  > **Mathematical beauty != computable**; any dimension that is "unfriendly and not reformable" means elimination.
118
118
 
@@ -74,27 +74,27 @@ Each entry follows **geometric concept -> mathematical core -> AI transfer**, ma
74
74
 
75
75
  ## GPU Friendliness Warning
76
76
 
77
- > Evaluate item by item using the **eight-dimension scorecard** from `../gpu-friendly-math.md`. The biggest pitfall in differential geometry is "inversion and materialization of metric / curvature matrices."
77
+ > Evaluate item by item using the **eight-dimension scorecard** from `../gpu-friendly-math.en.md`. The biggest pitfall in differential geometry is "inversion and materialization of metric / curvature matrices."
78
78
 
79
- - **Dimension 2 GEMM-mappability / Dimension 3 Complexity -- Inversion is the make-or-break line.** Naive natural gradient requires inverting the N x N Fisher, where N is the parameter count (~10^9); O(N^3) inversion + O(N^2) memory means **immediate disqualification**.
79
+ - **D2/D3**: Inversion is the make-or-break line. Naive natural gradient requires inverting the N x N Fisher, where N is the parameter count (~10^9); O(N^3) inversion + O(N^2) memory means **immediate disqualification**.
80
80
  - **Adaptable [v]**: **K-FAC** blocks F into a Kronecker product A (x) B, exploiting (A (x) B)^{-1} = A^{-1} (x) B^{-1}; only two small factors need inversion, and applying the preconditioner to the gradient **is a GEMM**. This is the affirmative answer to "can Kronecker factorization reduce to GEMM" -- yes, and this is the only form that can scale to a cluster.
81
- - **Dimension 4 Memory -- Do not materialize the full Hessian / full curvature tensor.** The Riemann curvature tensor is order 4; full materialization blows up memory. Use **Hessian-vector products (HVP)** to extract curvature information in O(N) via a single backward pass, avoiding N x N.
82
- - **Dimension 5 Low-precision -- Fisher / metric matrices are often ill-conditioned.** Inversion under bf16/fp16 catastrophically amplifies errors; **must add damping (Tikhonov, F + lambda I)** and keep inversion in fp32; otherwise it violates "low-precision stability."
83
- - **Dimension 6 Parallelism -- Parallel transport / geodesics are serial ODEs.** Integrating the connection equation along a curve is a long serial recurrence with poor parallelism; in practice, one uses **single-step retractions / closed-form parallel transport** (analytic formulas exist for specific manifolds) instead of step-by-step integration.
84
- - **Dimension 1 Tensorization / Dimension 2 GEMM-mappability -- Group convolution can be friendly, but continuous groups require caution.** Discrete groups (e.g., C_n, octahedral group) admit group convolutions that expand into GEMM [v]; continuous Lie groups require discretization by sampling, and improper sampling can break equivariance + produce irregular gather/scatter (Dimension 7 Sparsity unfriendly).
81
+ - **D4**: Do not materialize the full Hessian / full curvature tensor. The Riemann curvature tensor is order 4; full materialization blows up memory. Use **Hessian-vector products (HVP)** to extract curvature information in O(N) via a single backward pass, avoiding N x N.
82
+ - **D5**: Fisher / metric matrices are often ill-conditioned. Inversion under bf16/fp16 catastrophically amplifies errors; **must add damping (Tikhonov, F + lambda I)** and keep inversion in fp32; otherwise it violates "low-precision stability."
83
+ - **D6**: Parallel transport / geodesics are serial ODEs. Integrating the connection equation along a curve is a long serial recurrence with poor parallelism; in practice, one uses **single-step retractions / closed-form parallel transport** (analytic formulas exist for specific manifolds) instead of step-by-step integration.
84
+ - **D1/D2**: Group convolution can be friendly, but continuous groups require caution. Discrete groups (e.g., C_n, octahedral group) admit group convolutions that expand into GEMM [v]; continuous Lie groups require discretization by sampling, and improper sampling can break equivariance + produce irregular gather/scatter (D7 Sparsity unfriendly).
85
85
 
86
86
  **Natural gradient / K-FAC eight-dimension scorecard (worked example):**
87
87
 
88
- | Dimension | Naive Natural Gradient (full Fisher inversion) | K-FAC (Kronecker factorization) |
88
+ | D | Naive Natural Gradient (full Fisher inversion) | K-FAC (Kronecker factorization) |
89
89
  |-----------|------------------------------------------------|--------------------------------|
90
- | Dimension 1 Tensorization | [x] explicit large-matrix inverse | [v] batched small-matrix algebra |
91
- | Dimension 2 GEMM-mappability | [x] N x N inversion, not GEMM-able | [v] A^{-1} (x) B^{-1} application = GEMM chain |
92
- | Dimension 3 Complexity | [x] O(N^3) | [v] two small factors, sub-cubic |
93
- | Dimension 4 Memory | [x] materialize N x N | [v] store only two small factors |
94
- | Dimension 5 Low-precision | [x] ill-conditioned, needs fp64 | [~] add damping + fp32 inversion, adaptable |
95
- | Dimension 6 Parallelism | [~] single large inversion hard to parallelize | [v] per-layer factors independent, parallelizable |
96
- | Dimension 7 Sparsity | -- | [v] block-diagonal structure |
97
- | Dimension 8 Operator fusion | [x] | [v] preconditioner can be fused into optimizer kernel |
90
+ | D1 | [x] explicit large-matrix inverse | [v] batched small-matrix algebra |
91
+ | D2 | [x] N x N inversion, not GEMM-able | [v] A^{-1} (x) B^{-1} application = GEMM chain |
92
+ | D3 | [x] O(N^3) | [v] two small factors, sub-cubic |
93
+ | D4 | [x] materialize N x N | [v] store only two small factors |
94
+ | D5 | [x] ill-conditioned, needs fp64 | [~] add damping + fp32 inversion, adaptable |
95
+ | D6 | [~] single large inversion hard to parallelize | [v] per-layer factors independent, parallelizable |
96
+ | D7 | -- | [v] block-diagonal structure |
97
+ | D8 | [x] | [v] preconditioner can be fused into optimizer kernel |
98
98
 
99
99
  **Conclusion**: Riemannian / information-geometric methods **become GPU-feasible only when the metric is structurally factored (Kronecker / block-diagonal / low-rank)**; exact inversion and exact parallel transport are "beautiful but incomputable" and must be adapted or eliminated.
100
100
 
@@ -106,7 +106,7 @@ Each entry follows **geometric concept -> mathematical core -> AI transfer**, ma
106
106
  - **geometric**: Explicitly model parameter / data spaces as manifolds, then translate back to algorithms.
107
107
  - **topological**: Auxiliary -- de Rham cohomology / global invariants for conservation laws and integrability diagnostics.
108
108
 
109
- Recommended combination: First `symmetry` to establish the symmetry structure -> `variational` to arrive at natural gradient / Riemannian optimization -> `duality` to handle retractions -> pass through the `../gpu-friendly-math.md` acceptance gate.
109
+ Recommended combination: First `symmetry` to establish the symmetry structure -> `variational` to arrive at natural gradient / Riemannian optimization -> `duality` to handle retractions -> pass through the `../gpu-friendly-math.en.md` acceptance gate.
110
110
 
111
111
  ## Anti-patterns
112
112
 
@@ -76,25 +76,25 @@
76
76
 
77
77
  > 用 `../gpu-friendly-math.md` 的**八维记分卡**逐项评判。微分几何最大的坑是"度量 / 曲率矩阵的求逆与物化"。
78
78
 
79
- - **维度 2/3(GEMM 可映射 / 复杂度)—— 求逆是生死线。** 朴素自然梯度要算 N×N Fisher 的逆,N 是参数量(~10⁹),O(N³) 求逆 + O(N²) 显存,**直接出局**。
79
+ - **D2/D3**:求逆是生死线。 朴素自然梯度要算 N×N Fisher 的逆,N 是参数量(~10⁹),O(N³) 求逆 + O(N²) 显存,**直接出局**。
80
80
  - **可改造 [v]**:**K-FAC** 把 F 分块为 Kronecker 积 A⊗B,利用 (A⊗B)⁻¹ = A⁻¹⊗B⁻¹,只需对两个小因子求逆,且预条件作用到梯度上**就是 GEMM**。这就是"能否 Kronecker 因子化为 GEMM"的肯定答案——能,且这是它唯一可上集群的形态。
81
- - **维度 4(显存)—— 不要物化全 Hessian / 全曲率张量。** Riemann 曲率张量是 4 阶,全物化爆显存。用 **Hessian-vector product(HVP)** 经一次反向以 O(N) 拿到曲率信息,避免 N×N。
82
- - **维度 5(低精度)—— Fisher / 度量矩阵常病态。** bf16/fp16 下求逆灾难性放大误差,**必须加阻尼(damping / Tikhonov,F+λI)** 并把求逆留在 fp32;否则违反"低精度稳定"。
83
- - **维度 6(并行)—— 平行移动 / 测地线是串行 ODE。** 沿曲线积分联络方程是长串行递推,并行性差;工程上用**一步 retraction / 闭式平行移动**(特定流形有解析公式)替代逐步积分。
84
- - **维度 1/2(张量化 / GEMM)—— 群卷积可友好,但连续群要当心。** 离散群(如 C_n、八面体群)的群卷积可展开成 GEMM [v];连续李群需采样离散化,采样不当会导致等变性破缺 + 不规则 gather/scatter(维度 7 稀疏不友好)。
81
+ - **D4**:不要物化全 Hessian / 全曲率张量。 Riemann 曲率张量是 4 阶,全物化爆显存。用 **Hessian-vector product(HVP)** 经一次反向以 O(N) 拿到曲率信息,避免 N×N。
82
+ - **D5**:Fisher / 度量矩阵常病态。 bf16/fp16 下求逆灾难性放大误差,**必须加阻尼(damping / Tikhonov,F+λI)** 并把求逆留在 fp32;否则违反"低精度稳定"。
83
+ - **D6**:平行移动 / 测地线是串行 ODE 沿曲线积分联络方程是长串行递推,并行性差;工程上用**一步 retraction / 闭式平行移动**(特定流形有解析公式)替代逐步积分。
84
+ - **D1/D2**:群卷积可友好,但连续群要当心。 离散群(如 C_n、八面体群)的群卷积可展开成 GEMM [v];连续李群需采样离散化,采样不当会导致等变性破缺 + 不规则 gather/scatter(D7 稀疏不友好)。
85
85
 
86
86
  **自然梯度 / K-FAC 八维记分卡(worked example):**
87
87
 
88
- | 维度 | 朴素自然梯度(全 Fisher 求逆)| K-FAC(Kronecker 因子化)|
88
+ | D | 朴素自然梯度(全 Fisher 求逆)| K-FAC(Kronecker 因子化)|
89
89
  |------|------------------------------|--------------------------|
90
- | 1 张量化 | [x] 大矩阵显式逆 | [v] 批量小矩阵代数 |
91
- | 2 GEMM | [x] N×N 求逆无法 GEMM | [v] A⁻¹⊗B⁻¹ 作用 = GEMM 链 |
92
- | 3 复杂度 | [x] O(N³) | [v] 两个小因子,亚立方 |
93
- | 4 显存 | [x] 物化 N×N | [v] 只存两个小因子 |
94
- | 5 低精度 | [x] 病态、需 fp64 | [~] 加阻尼 + fp32 求逆可改造 |
95
- | 6 并行 | [~] 单次大求逆难并行 | [v] 各层因子独立、可并行 |
96
- | 7 稀疏 | — | [v] 块对角结构化 |
97
- | 8 融合 | [x] | [v] 预条件可融进优化器 kernel |
90
+ | D1 | [x] 大矩阵显式逆 | [v] 批量小矩阵代数 |
91
+ | D2 | [x] N×N 求逆无法 GEMM | [v] A⁻¹⊗B⁻¹ 作用 = GEMM 链 |
92
+ | D3 | [x] O(N³) | [v] 两个小因子,亚立方 |
93
+ | D4 | [x] 物化 N×N | [v] 只存两个小因子 |
94
+ | D5 | [x] 病态、需 fp64 | [~] 加阻尼 + fp32 求逆可改造 |
95
+ | D6 | [~] 单次大求逆难并行 | [v] 各层因子独立、可并行 |
96
+ | D7 | — | [v] 块对角结构化 |
97
+ | D8 | [x] | [v] 预条件可融进优化器 kernel |
98
98
 
99
99
  **结论**:黎曼 / 信息几何方法**只有在度量被结构化因子化(Kronecker / 块对角 / 低秩)后才 GPU 可行**;精确求逆与精确平行移动都属于"美但不可算",须改造或淘汰。
100
100
 
@@ -67,7 +67,7 @@ When "activating" this book into algorithms, these are the most frequently used
67
67
 
68
68
  ## Possible Algorithmic Inspirations
69
69
 
70
- > Each item is tagged with the **eight-dimension touchpoint** (corresponding to the dimension numbers in `../gpu-friendly-math.md`) for direct entry into the GPU acceptance gate.
70
+ > Each item is tagged with the **eight-dimension touchpoint** (corresponding to the dimension numbers in `../gpu-friendly-math.en.md`) for direct entry into the GPU acceptance gate.
71
71
 
72
72
  1. **Randomized numerical linear algebra (randomized NLA)**: using random projections + QR (Sec. 2.1) for randomized SVD, reducing the O(n^3) full decomposition to sub-quadratic, producing low-rank sketches of very large weights/activations. *Touchpoint: D2/D3 -- all GEMM, manageable complexity.*
73
73
  2. **Low-rank attention / KV compression**: using Eckart-Young (Sec. 7.4) to guarantee truncated SVD is the optimal low-rank approximation; using the **nuclear norm (dual of spectral norm, Sec. 5.5)** as low-rank regularization, projecting the KV-Cache into a low-dimensional subspace. *Touchpoint: D2/D4 -- GEMM chains + memory compression.*
@@ -81,7 +81,7 @@ When "activating" this book into algorithms, these are the most frequently used
81
81
 
82
82
  ## GPU Friendliness Warning
83
83
 
84
- > The scoring dimensions reference the **eight-dimension checklist** in `../gpu-friendly-math.md` (Tensorization / GEMM-mappability / Complexity / Memory / Low-precision / Parallelism / Sparsity / Operator fusion); definitions are not repeated here.
84
+ > The scoring dimensions reference the **eight-dimension checklist** in `../gpu-friendly-math.en.md` (Tensorization / GEMM-mappability / Complexity / Memory / Low-precision / Parallelism / Sparsity / Operator fusion); definitions are not repeated here.
85
85
 
86
86
  **Natively friendly (math beautiful x GPU friendly):**
87
87
  - **Truncated SVD / low-rank**: expressed as GEMM chains (D2), compressing KV-Cache/weights (D4).
@@ -98,7 +98,7 @@ When "activating" this book into algorithms, these are the most frequently used
98
98
  - **Serial dependencies in QR / Cholesky (Sec. 2.1, 3.5)** -- naive implementations are long serial recurrences (violates D6), requiring blocked / communication-avoiding variants.
99
99
  - **Serial iteration in power iteration** -- single-vector iteration has low parallelism; must be blocked (block / subspace iteration) to saturate SMs (D6).
100
100
 
101
- **Reform strategies (echoing the Make-It-Computable Toolkit in `../gpu-friendly-math.md`):**
101
+ **Reform strategies (echoing the Make-It-Computable Toolkit in `../gpu-friendly-math.en.md`):**
102
102
  - Full EVD/SVD -> **randomized + truncated** to reduce complexity (D3);
103
103
  - Exact decompositions -> **blocked / GEMM-ified** to eliminate serial dependencies (D2/D6);
104
104
  - Ill-conditioned/non-normal -> **reparameterization + spectral normalization** to stabilize low precision (D5);
@@ -69,39 +69,39 @@
69
69
 
70
70
  > 每条标注**八维落点**(对应 `../gpu-friendly-math.md` 维度编号),方便直接进 GPU 验收门。
71
71
 
72
- 1. **随机化数值线代(randomized NLA)**:用随机投影 + QR(§2.1)做随机 SVD,把 O(n³) 完整分解降到亚二次,对超大权重/激活做低秩草图(sketching)。*落点:维度 2/3——全 GEMM、复杂度可控。*
73
- 2. **低秩注意力 / KV 压缩**:以 Eckart–Young(§7.4)保证截断 SVD 是最优低秩逼近;用**核范数(谱范数对偶,§5.5)**作低秩正则,把 KV-Cache 投影到低维子空间。*落点:维度 2/4——GEMM 链 + 显存压缩。*
74
- 3. **谱归一化(spectral normalization)**:power iteration 估最大奇异值(算子范数 §5.6),约束每层 Lipschitz——GAN/扩散/稳定训练。*落点:维度 1/6——matvec,但迭代串行需块化。*
75
- 4. **Newton–Schulz 正交化(Muon 式)**:极分解(§7.3)把梯度矩阵投到最近正交矩阵,迭代只含矩阵乘——当前最 GPU 友好的"二阶味"更新。*落点:维度 2/6/8——纯 GEMM、可融合、bf16 稳。*
76
- 5. **预条件 / Shampoo / K-FAC**:条件数(§5.8)诊断病态,用 PSD Kronecker 因子(Ch 7)近似 Hessian 做预条件,把病态损失面拉圆。*落点:维度 2/5——小矩阵 GEMM,注意逆运算的精度。*
77
- 6. **Geršgorin 廉价谱半径门(§6.1)**:训练循环里用圆盘界 O(n²) 快速估谱半径,做低成本稳定性 gate,不必跑完整 EVD。*落点:维度 1/3——逐行求和,极廉价。*
78
- 7. **PSD 核工程(Schur 积定理 §7.5)**:用 Hadamard 积组合多个 PSD 核,保证可学习相似度矩阵始终半正定。*落点:维度 1——逐元素张量积,天然友好。*
79
- 8. **Perron–Frobenius 诊断(§8.2–8.5)**:把行随机 attention 当 Markov 算子,用谱 gap 量化 over-smoothing / rank collapse,指导残差与温度设计。*落点:维度 1/3——谱估计廉价,避免深层坍缩。*
80
- 9. **块化 / communication-avoiding 分解**:把 QR、Cholesky(§3.5)写成分块版本,用 GEMM 替代逐列消元,跨设备减少通信轮次。*落点:维度 2/6——串行递推改造成并行 + overlap。*
72
+ 1. **随机化数值线代(randomized NLA)**:用随机投影 + QR(§2.1)做随机 SVD,把 O(n³) 完整分解降到亚二次,对超大权重/激活做低秩草图(sketching)。*落点:D2/D3——全 GEMM、复杂度可控。*
73
+ 2. **低秩注意力 / KV 压缩**:以 Eckart–Young(§7.4)保证截断 SVD 是最优低秩逼近;用**核范数(谱范数对偶,§5.5)**作低秩正则,把 KV-Cache 投影到低维子空间。*落点:D2/D4——GEMM 链 + 显存压缩。*
74
+ 3. **谱归一化(spectral normalization)**:power iteration 估最大奇异值(算子范数 §5.6),约束每层 Lipschitz——GAN/扩散/稳定训练。*落点:D1/D6——matvec,但迭代串行需块化。*
75
+ 4. **Newton–Schulz 正交化(Muon 式)**:极分解(§7.3)把梯度矩阵投到最近正交矩阵,迭代只含矩阵乘——当前最 GPU 友好的"二阶味"更新。*落点:D2/D6/D8——纯 GEMM、可融合、bf16 稳。*
76
+ 5. **预条件 / Shampoo / K-FAC**:条件数(§5.8)诊断病态,用 PSD Kronecker 因子(Ch 7)近似 Hessian 做预条件,把病态损失面拉圆。*落点:D2/D5——小矩阵 GEMM,注意逆运算的精度。*
77
+ 6. **Geršgorin 廉价谱半径门(§6.1)**:训练循环里用圆盘界 O(n²) 快速估谱半径,做低成本稳定性 gate,不必跑完整 EVD。*落点:D1/D3——逐行求和,极廉价。*
78
+ 7. **PSD 核工程(Schur 积定理 §7.5)**:用 Hadamard 积组合多个 PSD 核,保证可学习相似度矩阵始终半正定。*落点:D1——逐元素张量积,天然友好。*
79
+ 8. **Perron–Frobenius 诊断(§8.2–8.5)**:把行随机 attention 当 Markov 算子,用谱 gap 量化 over-smoothing / rank collapse,指导残差与温度设计。*落点:D1/D3——谱估计廉价,避免深层坍缩。*
80
+ 9. **块化 / communication-avoiding 分解**:把 QR、Cholesky(§3.5)写成分块版本,用 GEMM 替代逐列消元,跨设备减少通信轮次。*落点:D2/D6——串行递推改造成并行 + overlap。*
81
81
 
82
82
  ## GPU 友好性警告
83
83
 
84
84
  > 评分维度引用 `../gpu-friendly-math.md` 的**八维检查**(张量化 / GEMM 可映射 / 复杂度 / 显存 / 低精度 / 并行 / 稀疏 / 算子融合),此处不重复定义。
85
85
 
86
86
  **天然友好(math beautiful × GPU friendly):**
87
- - **SVD 截断 / 低秩**:写成 GEMM 链(维度 2),压缩 KV-Cache/权重(维度 4)。
88
- - **Frobenius / 谱范数、Gram 矩阵、Hadamard 积**:批量张量代数(维度 1、2)。
89
- - **极分解 Newton–Schulz**:纯矩阵乘迭代(维度 2、6、8 可融合),bf16 下稳健。
90
- - **Geršgorin 圆盘**:O(n²) 逐行求和(维度 1),廉价稳定性估计。
91
- - **良态 PSD 的 blocked Cholesky / Gram 构造**:分块后是 GEMM 链(维度 2),核方法与协方差预条件常用。
87
+ - **SVD 截断 / 低秩**:写成 GEMM 链(D2),压缩 KV-Cache/权重(D4)。
88
+ - **Frobenius / 谱范数、Gram 矩阵、Hadamard 积**:批量张量代数(D1/D2)。
89
+ - **极分解 Newton–Schulz**:纯矩阵乘迭代(D2/D6/D8 可融合),bf16 下稳健。
90
+ - **Geršgorin 圆盘**:O(n²) 逐行求和(D1),廉价稳定性估计。
91
+ - **良态 PSD 的 blocked Cholesky / Gram 构造**:分块后是 GEMM 链(D2),核方法与协方差预条件常用。
92
92
 
93
93
  **美但不可算(beautiful, not computable):**
94
- - **Jordan 标准型(§3.1)**——经典反例:特征值重数对扰动极度敏感,浮点下根本无法可靠计算,**永远不要当数值工具**(违反维度 5 低精度稳定)。Weyr 型同理。
95
- - **完整 EVD / SVD 的 O(n³)**——大矩阵直接爆(违反维度 3),必须换随机化/迭代法。
96
- - **非正规矩阵(non-normal,§2.5 之外)**——特征值不能反映真实行为,需 pseudospectra;低精度下谱失真(维度 5)。
97
- - **病态 / 高条件数(§5.8)**——会出现灾难性抵消,要 fp64 才对,与 bf16/fp8 训练冲突(维度 5)。
98
- - **QR / Cholesky 的串行依赖(§2.1, §3.5)**——朴素实现是长串行递推(违反维度 6),需 blocked / communication-avoiding 变体。
99
- - **power iteration 的串行迭代**——单向量迭代并行度低;需块化(block / subspace iteration)才吃满 SM(维度 6)。
94
+ - **Jordan 标准型(§3.1)**——经典反例:特征值重数对扰动极度敏感,浮点下根本无法可靠计算,**永远不要当数值工具**(违反 D5)。Weyr 型同理。
95
+ - **完整 EVD / SVD 的 O(n³)**——大矩阵直接爆(违反 D3),必须换随机化/迭代法。
96
+ - **非正规矩阵(non-normal,§2.5 之外)**——特征值不能反映真实行为,需 pseudospectra;低精度下谱失真(D5)。
97
+ - **病态 / 高条件数(§5.8)**——会出现灾难性抵消,要 fp64 才对,与 bf16/fp8 训练冲突(D5)。
98
+ - **QR / Cholesky 的串行依赖(§2.1, §3.5)**——朴素实现是长串行递推(违反 D6),需 blocked / communication-avoiding 变体。
99
+ - **power iteration 的串行迭代**——单向量迭代并行度低;需块化(block / subspace iteration)才吃满 SM(D6)。
100
100
 
101
101
  **改造手法(呼应 `../gpu-friendly-math.md` 的 Make-It-Computable Toolkit):**
102
- - 完整 EVD/SVD → **随机化 + 截断**降复杂度(维度 3);
103
- - 精确分解 → **块化 / GEMM 化**消除串行(维度 2/6);
104
- - 病态/非正规 → **重参数化 + 谱归一化**稳低精度(维度 5);
102
+ - 完整 EVD/SVD → **随机化 + 截断**降复杂度(D3);
103
+ - 精确分解 → **块化 / GEMM 化**消除串行(D2/D6);
104
+ - 病态/非正规 → **重参数化 + 谱归一化**稳低精度(D5);
105
105
  - Jordan 等不可算标准型 → 只留作**理论证明**,数值上换 Schur/SVD。
106
106
 
107
107
  ## 该调用哪个思想透镜
@@ -125,7 +125,7 @@
125
125
  - **数值上假设精确正定**:浮点下 Gram/协方差可能丢正定性,要加 jitter(对角抖动)或用 Cholesky 带 pivot。
126
126
  - **用 Frobenius 范数当低秩正则**:Frobenius / 权重衰减压的是"能量"不是"秩";要低秩得用核范数或显式低秩参数化(如 LoRA)。
127
127
  - **对非对称矩阵谈"特征值大小"**:度量能量/范数/稳定裕度该看**奇异值**;非正规矩阵的特征值模长会严重误导(瞬态增长远超谱半径预测)。
128
- - **把 O(n²) Gram 矩阵全量物化**:attention/核矩阵不分块就直接撑爆显存;应走 FlashAttention 式融合 + 分块(呼应 GPU 八维维度 4/8)。
128
+ - **把 O(n²) Gram 矩阵全量物化**:attention/核矩阵不分块就直接撑爆显存;应走 FlashAttention 式融合 + 分块(呼应 GPU 八维 D4/D8)。
129
129
  - **堆定理而不诊断瓶颈**:先问"算法瓶颈是谱、低秩还是稳定性",再选结构,别一上来灌矩阵理论。
130
130
 
131
131
  ## 深挖入口
@@ -62,7 +62,7 @@ Two additional directly usable "components":
62
62
 
63
63
  ## GPU Friendliness Warning
64
64
 
65
- > The sole authority for the acceptance gate: [`../gpu-friendly-math.md`](../gpu-friendly-math.md) (eight dimensions: D1 Tensorization, D2 GEMM-mappability, D3 Complexity, D4 Memory/KV, D5 Low-precision stability, D6 Parallelism & communication, D7 Sparse structure, D8 Operator fusion). Dimension-by-dimension scoring follows.
65
+ > The sole authority for the acceptance gate: [`../gpu-friendly-math.en.md`](../gpu-friendly-math.en.md) (eight dimensions: D1 Tensorization, D2 GEMM-mappability, D3 Complexity, D4 Memory/KV, D5 Low-precision stability, D6 Parallelism & communication, D7 Sparse structure, D8 Operator fusion). Dimension-by-dimension scoring follows.
66
66
 
67
67
  | # | Dimension | Rating | Notes |
68
68
  |---|---|---|---|
@@ -102,32 +102,32 @@ Actual chapter map:
102
102
 
103
103
  ## GPU Friendliness Warning
104
104
 
105
- > The following item-by-item evaluation uses the **eight-dimension scorecard** from `../gpu-friendly-math.md` (read that first).
105
+ > The following item-by-item evaluation uses the **eight-dimension scorecard** from `../gpu-friendly-math.en.md` (read that first).
106
106
  > Conclusion: The book's first-order / stochastic / low-rank content is naturally GPU-friendly; second-order methods and search-based methods require heavy adaptation or elimination.
107
107
 
108
108
  - **Full Hessian in second-order methods (Ch 9, 11)**
109
- - Violates **Dimension 4 (Memory)**: Dense Hessian is O(d^2), impossible to materialize for hundreds of millions of parameters.
110
- - Violates **Dimension 2 (GEMM-mappability)**: Inversion / factorization is not batched-GEMM friendly.
111
- - Violates **Dimension 5 (Low-precision stability)**: Ill-conditioned inversion causes catastrophic cancellation under bf16/fp16.
112
- - Adaptation: Matrix-free HVP + CG, L-BFGS low-rank, K-FAC block-diagonal -> restores **Dimension 2/4** friendliness.
109
+ - Violates **D4**: Dense Hessian is O(d^2), impossible to materialize for hundreds of millions of parameters.
110
+ - Violates **D2**: Inversion / factorization is not batched-GEMM friendly.
111
+ - Violates **D5**: Ill-conditioned inversion causes catastrophic cancellation under bf16/fp16.
112
+ - Adaptation: Matrix-free HVP + CG, L-BFGS low-rank, K-FAC block-diagonal -> restores **D2/D4** friendliness.
113
113
  - **Line search (Ch 7 golden section / Armijo S24.4.4)**
114
- - Violates **Dimension 1 (Tensorization)**: Essentially a scalar loop + data-dependent branching (step-by-step comparison of function values).
115
- - Violates **Dimension 8 (Operator fusion)**: Frequent small kernels, control-flow divergence.
114
+ - Violates **D1**: Essentially a scalar loop + data-dependent branching (step-by-step comparison of function values).
115
+ - Violates **D8**: Frequent small kernels, control-flow divergence.
116
116
  - Adaptation: Large models almost universally use scheduled step sizes (warmup + cosine) instead of step-by-step line search.
117
117
  - **First-order / SGD (Ch 8, 27)**
118
- - **Dimension 1/2 friendly**: Gradients = tensor algebra, expressible as GEMM.
119
- - **Dimension 6 (Parallelism and communication)** is good: Data-parallel all-reduce can overlap with backprop -- the fundamental reason this is the default approach for large models.
118
+ - **D1/D2 friendly**: Gradients = tensor algebra, expressible as GEMM.
119
+ - **D6** is good: Data-parallel all-reduce can overlap with backprop -- the fundamental reason this is the default approach for large models.
120
120
  - **Global search (Ch 14: GA/SA/PSO)**
121
- - Violates **Dimension 1**: Non-differentiable, discrete stochastic search, blocks end-to-end gradient training.
122
- - **Dimension 6** is also poor: Inter-population dependencies, hard to parallelize. Only suitable for black-box hyperparameter search, not for the training inner loop.
121
+ - Violates **D1**: Non-differentiable, discrete stochastic search, blocks end-to-end gradient training.
122
+ - **D6** is also poor: Inter-population dependencies, hard to parallelize. Only suitable for black-box hyperparameter search, not for the training inner loop.
123
123
  - **LP simplex / interior-point methods (Ch 16, 18)**
124
- - Simplex row operations are acceptable; interior-point methods require solving a linear system at each step, **Dimension 2/6** limited by factorization and communication.
124
+ - Simplex row operations are acceptable; interior-point methods require solving a linear system at each step, **D2/D6** limited by factorization and communication.
125
125
  - Suitable for medium-scale constrained subproblems, not for placement inside every training step.
126
126
  - **Projections (S24.2)**
127
- - Norm-ball / box projections are cheap elementwise operations (**Dimension 1/8** friendly).
128
- - General polyhedral projections require solving a QP (**Dimension 2** degrades).
127
+ - Norm-ball / box projections are cheap elementwise operations (**D1/D8** friendly).
128
+ - General polyhedral projections require solving a QP (**D2** degrades).
129
129
  - **Convergence depends on condition number kappa**
130
- - Large kappa -> **Dimension 5** is further amplified at low precision (ill-conditioned gradients).
130
+ - Large kappa -> **D5** is further amplified at low precision (ill-conditioned gradients).
131
131
  - Must pair with normalization / preconditioning; otherwise bf16 training diverges.
132
132
 
133
133
  ## Which Thinking Lens to Invoke
@@ -106,28 +106,28 @@
106
106
  > 结论:本书的一阶 / 随机 / 低秩部分天然友好;二阶法与搜索类方法需重度改造或淘汰。
107
107
 
108
108
  - **二阶法的全 Hessian(Ch 9, 11)**
109
- - 违反**维度 4(显存)**:稠密 Hessian 是 O(d²),亿级参数无法物化。
110
- - 违反**维度 2(GEMM 可映射)**:求逆 / 分解非 batched-GEMM 友好。
111
- - 违反**维度 5(低精度稳定)**:病态求逆在 bf16/fp16 灾难性抵消。
112
- - 改造:matrix-free HVP + CG、L-BFGS 低秩、K-FAC 块对角 → 重回**维度 2/4** 友好。
109
+ - 违反**D4**:稠密 Hessian 是 O(d²),亿级参数无法物化。
110
+ - 违反**D2**:求逆 / 分解非 batched-GEMM 友好。
111
+ - 违反**D5**:病态求逆在 bf16/fp16 灾难性抵消。
112
+ - 改造:matrix-free HVP + CG、L-BFGS 低秩、K-FAC 块对角 → 重回**D2/D4** 友好。
113
113
  - **线搜索(Ch 7 黄金分割 / Armijo §24.4.4)**
114
- - 违反**维度 1(张量化)**:本质是标量循环 + 数据相关分支(逐步比较函数值)。
115
- - 违反**维度 8(算子融合)**:频繁小 kernel、控制流发散。
114
+ - 违反**D1**:本质是标量循环 + 数据相关分支(逐步比较函数值)。
115
+ - 违反**D8**:频繁小 kernel、控制流发散。
116
116
  - 改造:大模型几乎都用调度式步长(warmup + cosine)替代逐步线搜索。
117
117
  - **一阶 / SGD(Ch 8, 27)**
118
- - **维度 1/2 友好**:梯度 = 张量代数、可写成 GEMM。
119
- - **维度 6(并行与通信)**良好:数据并行 all-reduce 可与反向 overlap——大模型默认路线的根本原因。
118
+ - **D1/D2 友好**:梯度 = 张量代数、可写成 GEMM。
119
+ - **D6**良好:数据并行 all-reduce 可与反向 overlap——大模型默认路线的根本原因。
120
120
  - **全局搜索(Ch 14:GA/SA/PSO)**
121
- - 违反**维度 1**:不可微、离散随机搜索,阻断端到端梯度训练。
122
- - **维度 6** 也差:种群间依赖、难并行映射。仅适合黑盒超参搜索,不进训练内环。
121
+ - 违反**D1**:不可微、离散随机搜索,阻断端到端梯度训练。
122
+ - **D6** 也差:种群间依赖、难并行映射。仅适合黑盒超参搜索,不进训练内环。
123
123
  - **LP 单纯形 / 内点法(Ch 16, 18)**
124
- - 单纯形的行运算尚可;内点法每步要解线性系统,**维度 2/6** 受限于分解与通信。
124
+ - 单纯形的行运算尚可;内点法每步要解线性系统,**D2/D6** 受限于分解与通信。
125
125
  - 适合中等规模约束子问题,不适合放进每个训练 step。
126
126
  - **投影(§24.2)**
127
- - norm-ball / box 投影是廉价 elementwise(**维度 1/8** 友好)。
128
- - 一般多面体投影要解 QP(**维度 2** 退化)。
127
+ - norm-ball / box 投影是廉价 elementwise(**D1/D8** 友好)。
128
+ - 一般多面体投影要解 QP(**D2** 退化)。
129
129
  - **收敛依赖条件数 κ**
130
- - κ 大 → **维度 5** 在低精度下被进一步放大(病态梯度)。
130
+ - κ 大 → **D5** 在低精度下被进一步放大(病态梯度)。
131
131
  - 必须配归一化 / 预条件,否则 bf16 训练发散。
132
132
 
133
133
  ## 该调用哪个思想透镜
@@ -25,7 +25,7 @@ Actual chapter map (2nd ed., chapter numbers match the book):
25
25
 
26
26
  ## Core Structures Transferable to AI/Infra
27
27
 
28
- - **Tangent space = local linearization of parameter/latent space**. `df_p: T_pM -> T_{f(p)}N` is the Jacobian / pushforward; backpropagation = pushforward along composed maps (the geometric version of the chain rule). All first-order methods live in the tangent space.
28
+ - **Tangent space = local linearization of parameter/latent space**. `df_p: T_pM -> T_{f(p)}N` is the Jacobian / pushforward (corresponding to JVP / forward-mode AD); backpropagation = pullback on the cotangent bundle (VJP = vector-Jacobian product = pullback of covectors), i.e., pullback along composed maps (the geometric version of the chain rule). All first-order methods live in the tangent space.
29
29
  - **Gradients are covectors, not vectors**. Autodiff produces 1-forms (elements of the cotangent space); to obtain a descent direction (tangent vector) one must use the **metric to raise indices** (sharp). Euclidean metric -> ordinary gradient; Fisher metric -> natural gradient. **This is the manifold-level root cause of natural gradients / mirror descent**.
30
30
  - **Constraint sets = submanifolds**. Regular level set theorem: when g is a submersion, the solution set `g(x)=c` is a smooth submanifold; constrained optimization = unconstrained optimization on a submanifold.
31
31
  - **Lie groups = differentiable symmetry groups**. SO(n)/U(n)/Stiefel/Grassmannian are all manifolds; their Lie algebras (e.g., skew-symmetric matrices so(n)) are linear spaces, mapped back to the group via `exp` -> **reparameterize "constrained weights" as "unconstrained Lie algebra + exp"**.
@@ -53,11 +53,11 @@ Actual chapter map (2nd ed., chapter numbers match the book):
53
53
 
54
54
  ## GPU Friendliness Warning
55
55
 
56
- > The sole authority for the acceptance gate: the **eight dimensions** in `../gpu-friendly-math.md`. The success or failure of manifold methods almost entirely hinges on one point: **whether the retraction / exponential map can be tensorized and expressed as GEMM, or must be solved iteratively.**
56
+ > The sole authority for the acceptance gate: the **eight dimensions** in `../gpu-friendly-math.en.md`. The success or failure of manifold methods almost entirely hinges on one point: **whether the retraction / exponential map can be tensorized and expressed as GEMM, or must be solved iteratively.**
57
57
 
58
58
  Dimension-by-dimension comparison:
59
59
 
60
- - **D1-D2 Tensorization / GEMM**: tangent-space operations (pushforward/pullback, Jacobian-vector products, projecting gradients to the tangent space) are **naturally batched GEMM** [v] -- backpropagation itself is pushforward, which is extremely GPU-friendly. **However**, retractions/exp typically require QR, eigendecomposition, matrix exponentials, or small-matrix inverses: QR/eig are **not clean GEMM**, they are decompositions with serial dependencies (cuSOLVER batched small matrices are acceptable, but large matrices are O(n^3) with poor parallelism) -> **reformable** rather than natively friendly.
60
+ - **D1-D2 Tensorization / GEMM**: tangent-space operations (pushforward/pullback, Jacobian-vector products, projecting gradients to the tangent space) are **naturally batched GEMM** [v] -- backpropagation itself is pullback (VJP), which is extremely GPU-friendly. **However**, retractions/exp typically require QR, eigendecomposition, matrix exponentials, or small-matrix inverses: QR/eig are **not clean GEMM**, they are decompositions with serial dependencies (cuSOLVER batched small matrices are acceptable, but large matrices are O(n^3) with poor parallelism) -> **reformable** rather than natively friendly.
61
61
  - **D3 Complexity**: geodesic distances, parallel transport, general `log|det J|` are all O(n^3) or worse. **Fix**: restrict to manifolds with closed-form geodesics (spheres/hyperbolic/SO(3)); use triangular/coupling layers in normalizing flows so that log-det reduces to a diagonal sum (O(n)).
62
62
  - **D5 Low-precision**: [~] **biggest pitfall**. Matrix `exp / log / sqrt`, eigendecompositions, and the affine-invariant metric for SPD are **catastrophically unstable** under bf16/fp16, often silently requiring fp32/fp64. Manifold primitives frequently "appear to run but have numerically diverged long ago."
63
63
  - **D6 Parallelism & communication**: the squaring chain in scaling-and-squaring, ODE integration steps, Householder/QR all have **serial recurrences**, making cross-SM/device overlap difficult. Positive counter-example: explicit symplectic integrators (leapfrog) are highly parallel [v].
@@ -25,7 +25,7 @@
25
25
 
26
26
  ## 可迁移到 AI/Infra 的核心结构
27
27
 
28
- - **切空间 = 参数/隐空间的局部线性化(local linearization)**。`df_p: T_pM → T_{f(p)}N` 就是 Jacobian / pushforward;反向传播 = 沿复合映射做 pushforward(链式法则的几何版)。一切一阶方法都活在切空间里。
28
+ - **切空间 = 参数/隐空间的局部线性化(local linearization)**。`df_p: T_pM → T_{f(p)}N` 就是 Jacobian / pushforward(前推,对应 JVP / 前向模式 AD);反向传播 = 余切丛上的拉回(pullback on cotangent bundle,VJP = 向量-Jacobian 积 = 余向量的拉回),即沿复合映射做 pullback(链式法则的几何版)。一切一阶方法都活在切空间里。
29
29
  - **梯度是余向量(covector),不是向量**。autodiff 给出的是 1-form(余切空间元素);要变成可下降的方向(切向量)必须用 **度量升指标**((sharp))。欧氏度量 → 普通梯度;Fisher 度量 → 自然梯度(natural gradient)。**这是自然梯度 / 镜像下降的流形根因**。
30
30
  - **约束集 = 子流形(submanifold)**。正则水平集定理:当 g 是 submersion 时 `g(x)=c` 的解集是光滑子流形;约束优化 = 在子流形上做无约束优化。
31
31
  - **李群 = 可微的对称群**。SO(n)/U(n)/Stiefel/Grassmann 都是流形;其李代数(如反对称矩阵 so(n))是线性空间,用 `exp` 映射回群 → **把"约束权重"重参数化为"无约束李代数 + exp"**。
@@ -57,11 +57,11 @@
57
57
 
58
58
  逐维对照:
59
59
 
60
- - **维度 12 张量化 / GEMM**:切空间运算(pushforward/pullback、Jacobian-向量积、把梯度投影到切空间)**天然是 batched GEMM** [v]——反向传播本就是 pushforward,这部分对 GPU 极友好。**但** retraction/exp 多半要 QR、特征分解、矩阵指数或小矩阵求逆:QR/eig **不是干净的 GEMM**,是带串行依赖的分解(cuSOLVER 批量小矩阵尚可,大矩阵 O(n³) 且并行差)→ **可改造** 而非天然友好。
61
- - **维度 3 复杂度**:测地线距离、平行移动、一般 `log|det J|` 都是 O(n³) 起。**改造**:限定有闭式测地线的流形(球面/双曲/SO(3));归一化流用三角/耦合层让 log-det 退化成对角和(O(n))。
62
- - **维度 5 低精度**:[~] **最大坑**。矩阵 `exp / log / sqrt`、特征分解、SPD 的仿射不变度量在 bf16/fp16 下 **灾难性不稳定**,常静默地需要 fp32/fp64。流形原语经常"表面能跑、数值早已发散"。
63
- - **维度 6 并行与通信**:scaling-and-squaring 的平方链、ODE 积分步、Householder/QR 都有 **串行递推**,难跨 SM/设备 overlap。反例向好:显式辛积分器(leapfrog)高并行 [v]。
64
- - **维度 4/7/8 显存 / 稀疏 / 融合**:李代数/旋转参数化若限制在 **小矩阵或块对角**(如逐头旋转、SO(3) 的 Rodrigues 闭式),可融进 kernel、走 Tensor Core;大稠密流形算子则要物化大中间张量。
60
+ - **D1D2** 切空间运算(pushforward/pullback、Jacobian-向量积、把梯度投影到切空间)**天然是 batched GEMM** [v]——反向传播本就是 pullback(VJP),这部分对 GPU 极友好。**但** retraction/exp 多半要 QR、特征分解、矩阵指数或小矩阵求逆:QR/eig **不是干净的 GEMM**,是带串行依赖的分解(cuSOLVER 批量小矩阵尚可,大矩阵 O(n³) 且并行差)→ **可改造** 而非天然友好。
61
+ - **D3 复杂度**:测地线距离、平行移动、一般 `log|det J|` 都是 O(n³) 起。**改造**:限定有闭式测地线的流形(球面/双曲/SO(3));归一化流用三角/耦合层让 log-det 退化成对角和(O(n))。
62
+ - **D5 低精度**:[~] **最大坑**。矩阵 `exp / log / sqrt`、特征分解、SPD 的仿射不变度量在 bf16/fp16 下 **灾难性不稳定**,常静默地需要 fp32/fp64。流形原语经常"表面能跑、数值早已发散"。
63
+ - **D6 并行与通信**:scaling-and-squaring 的平方链、ODE 积分步、Householder/QR 都有 **串行递推**,难跨 SM/设备 overlap。反例向好:显式辛积分器(leapfrog)高并行 [v]。
64
+ - **D4/D7/D8 显存 / 稀疏 / 融合**:李代数/旋转参数化若限制在 **小矩阵或块对角**(如逐头旋转、SO(3) 的 Rodrigues 闭式),可融进 kernel、走 Tensor Core;大稠密流形算子则要物化大中间张量。
65
65
 
66
66
  **结论与改造手法(呼应 gpu-friendly-math.md 工具箱)**:
67
67