math-skill 3.0.0 → 3.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en-US.md +9 -1
- package/README.md +9 -1
- package/agents/math-critic.en.md +27 -25
- package/agents/math-critic.md +27 -27
- package/commands/ask.en.md +10 -0
- package/commands/ask.md +1 -1
- package/design-patterns/attention/equivariant-attention.en.md +9 -9
- package/design-patterns/attention/equivariant-attention.md +9 -9
- package/design-patterns/attention/geometry-aware-attention.en.md +9 -9
- package/design-patterns/attention/geometry-aware-attention.md +9 -9
- package/design-patterns/attention/information-bottleneck-attention.en.md +9 -9
- package/design-patterns/attention/information-bottleneck-attention.md +9 -9
- package/design-patterns/attention/projection-attention.en.md +9 -9
- package/design-patterns/attention/projection-attention.md +9 -9
- package/design-patterns/attention/spectral-attention.en.md +9 -9
- package/design-patterns/attention/spectral-attention.md +9 -9
- package/design-patterns/compression/leverage-score-selection.en.md +1 -1
- package/design-patterns/compression/leverage-score-selection.md +1 -1
- package/design-patterns/compression/low-rank-kv-cache.en.md +1 -1
- package/design-patterns/compression/low-rank-kv-cache.md +1 -1
- package/design-patterns/compression/spectral-token-pruning.en.md +1 -1
- package/design-patterns/compression/spectral-token-pruning.md +1 -1
- package/design-patterns/compression/topology-preserving-compression.en.md +1 -1
- package/design-patterns/compression/topology-preserving-compression.md +1 -1
- package/design-patterns/loss/constraint-penalty.en.md +1 -1
- package/design-patterns/loss/constraint-penalty.md +1 -1
- package/design-patterns/loss/contrastive-loss.en.md +1 -1
- package/design-patterns/loss/contrastive-loss.md +1 -1
- package/design-patterns/loss/information-bottleneck-loss.en.md +1 -1
- package/design-patterns/loss/information-bottleneck-loss.md +1 -1
- package/design-patterns/loss/orthogonality-loss.en.md +1 -1
- package/design-patterns/loss/orthogonality-loss.md +1 -1
- package/design-patterns/loss/variational-loss.en.md +1 -1
- package/design-patterns/loss/variational-loss.md +1 -1
- package/design-patterns/representation/equivariant-split.en.md +1 -1
- package/design-patterns/representation/equivariant-split.md +1 -1
- package/design-patterns/representation/manifold-representation.en.md +1 -1
- package/design-patterns/representation/manifold-representation.md +1 -1
- package/design-patterns/representation/shared-private-decomposition.en.md +1 -1
- package/design-patterns/representation/shared-private-decomposition.md +1 -1
- package/design-patterns/representation/subspace-alignment.en.md +1 -1
- package/design-patterns/representation/subspace-alignment.md +1 -1
- package/design-patterns/routing/graph-routing.en.md +1 -1
- package/design-patterns/routing/graph-routing.md +1 -1
- package/design-patterns/routing/moe-routing.en.md +1 -1
- package/design-patterns/routing/moe-routing.md +1 -1
- package/design-patterns/routing/optimal-transport-routing.en.md +1 -1
- package/design-patterns/routing/optimal-transport-routing.md +1 -1
- package/design-patterns/routing/spectral-clustering-routing.en.md +1 -1
- package/design-patterns/routing/spectral-clustering-routing.md +1 -1
- package/knowledge-base/differential-geometry/metric-tensor.md +1 -1
- package/knowledge-base/information-geometry/fisher-metric.en.md +8 -8
- package/knowledge-base/information-geometry/fisher-metric.md +8 -8
- package/knowledge-base/information-geometry/natural-gradient.en.md +7 -7
- package/knowledge-base/information-geometry/natural-gradient.md +7 -7
- package/knowledge-base/overview.en.md +2 -2
- package/knowledge-base/overview.md +2 -2
- package/knowledge-base/probability/concentration-inequality.en.md +5 -5
- package/knowledge-base/probability/concentration-inequality.md +5 -5
- package/knowledge-base/probability/entropy.en.md +5 -5
- package/knowledge-base/probability/entropy.md +5 -5
- package/knowledge-base/probability/fisher-information.en.md +7 -7
- package/knowledge-base/probability/fisher-information.md +7 -7
- package/knowledge-base/probability/information-bottleneck.en.md +6 -6
- package/knowledge-base/probability/information-bottleneck.md +6 -6
- package/knowledge-base/probability/kl-divergence.en.md +6 -6
- package/knowledge-base/probability/kl-divergence.md +6 -6
- package/lenses/algorithmic.en.md +1 -1
- package/lenses/algorithmic.md +1 -1
- package/lenses/axiomatization.en.md +1 -1
- package/lenses/axiomatization.md +1 -1
- package/lenses/categorical.en.md +1 -1
- package/lenses/categorical.md +1 -1
- package/lenses/causal.en.md +1 -1
- package/lenses/causal.md +1 -1
- package/lenses/duality.en.md +1 -1
- package/lenses/duality.md +1 -1
- package/lenses/game.en.md +2 -2
- package/lenses/game.md +2 -2
- package/lenses/geometric.en.md +1 -1
- package/lenses/geometric.md +1 -1
- package/lenses/local-to-global.en.md +1 -1
- package/lenses/local-to-global.md +1 -1
- package/lenses/perturbation.en.md +1 -1
- package/lenses/perturbation.md +1 -1
- package/lenses/probabilistic.en.md +1 -1
- package/lenses/probabilistic.md +1 -1
- package/lenses/projection.en.md +1 -1
- package/lenses/projection.md +1 -1
- package/lenses/spectral.en.md +1 -1
- package/lenses/spectral.md +1 -1
- package/lenses/symmetry.en.md +1 -1
- package/lenses/symmetry.md +1 -1
- package/lenses/topological.en.md +1 -1
- package/lenses/topological.md +1 -1
- package/lenses/variational.en.md +1 -1
- package/lenses/variational.md +1 -1
- package/package.json +1 -1
- package/references/agentic-workflow.en.md +1 -1
- package/references/agentic-workflow.md +1 -1
- package/references/books/abstract-algebra.en.md +124 -0
- package/references/books/abstract-algebra.md +17 -17
- package/references/books/algebraic-geometry-rising-sea.en.md +171 -0
- package/references/books/algebraic-geometry-rising-sea.md +7 -7
- package/references/books/differential-geometry.en.md +140 -0
- package/references/books/differential-geometry.md +18 -18
- package/references/books/matrix-analysis.en.md +146 -0
- package/references/books/matrix-analysis.md +8 -8
- package/references/books/micro-lie-theory.en.md +116 -0
- package/references/books/micro-lie-theory.md +16 -16
- package/references/books/optimization-ml.en.md +163 -0
- package/references/books/optimization-ml.md +6 -7
- package/references/books/smooth-manifolds.en.md +105 -0
- package/references/books/smooth-manifolds.md +12 -12
- package/references/gpu-friendly-math.en.md +15 -2
- package/references/gpu-friendly-math.md +15 -2
- package/skills/math-research-activator/SKILL.en.md +27 -1
- package/skills/math-research-activator/SKILL.md +26 -3
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
# Abstract Algebra
|
|
2
|
+
|
|
3
|
+
> **Contemporary Abstract Algebra**, Eighth Edition -- Joseph A. Gallian -- Brooks/Cole, Cengage Learning.
|
|
4
|
+
> This file is an "activation" reference: mapping the book's group/ring/field structures to ML/algorithms/Infra, not reproducing the original text. For full-fidelity lookups see the "Deep-dive Entry" at the end.
|
|
5
|
+
|
|
6
|
+
## Overview
|
|
7
|
+
|
|
8
|
+
This book is a standard undergraduate abstract algebra textbook whose main thread is a three-part progression: **groups -> rings -> fields**. Starting from axioms, it emphasizes **symmetry, structure-preserving maps (homomorphisms/isomorphisms), quotient structures, and the classification and counting of finite objects**. The value for AI lies not in the theorems themselves but in the **language of "remaining invariant under some transformation"** that it provides -- this is the algebraic foundation for equivariant networks, permutation-invariant aggregation, coding, and hashing.
|
|
9
|
+
|
|
10
|
+
One-sentence activation insight: **Symmetry is a free inductive bias**. Rather than forcing the model to learn from data that "a rotated cat is still the same cat," one can build the symmetry group G's action directly into the network architecture, sharing parameters along orbits -- saving parameters while providing generalization guarantees. The main line -- groups (the transformations themselves) -> rings (algebra with two operations, whose relaxation yields generalized matrix multiplication) -> fields (arithmetically closed and exact, yielding coding and hashing) -- exactly covers the three Infra needs of "equivariance / generalized GEMM / fault-tolerant computation."
|
|
11
|
+
|
|
12
|
+
**Boundaries of this book (to avoid misuse)**: Gallian is an elementary undergraduate textbook; it **covers rings but not semirings** (the tropical semiring requires axiom relaxation on your own); representation theory only reaches the level of finite abelian groups and **does not develop non-abelian representation theory or character theory**; it does not touch category theory, modules, or homological algebra. When these more modern tools are needed, this book serves only as an "activation starting point," and deeper study requires turning to specialized texts on representation theory / algebraic geometry.
|
|
13
|
+
|
|
14
|
+
Actual chapter map (grouped by topic; chapter numbers taken from the book's table of contents):
|
|
15
|
+
|
|
16
|
+
- **Group foundations**: 1 Introduction to Groups (dihedral group D_n as the symmetry group exemplar), 2 Groups, 3 Finite Groups; Subgroups, 4 Cyclic Groups.
|
|
17
|
+
- **Symmetry and actions**: 5 Permutation Groups (orbit / stabilizer), 6 Isomorphisms, 7 Cosets and Lagrange's Theorem, 8 External Direct Products.
|
|
18
|
+
- **Quotients and homomorphisms**: 9 Normal Subgroups and Factor Groups, 10 Group Homomorphisms, 11 Fundamental Theorem of Finite Abelian Groups.
|
|
19
|
+
- **Rings**: 12 Introduction to Rings, 13 Integral Domains, 14 Ideals and Factor Rings, 15 Ring Homomorphisms, 16 Polynomial Rings, 17 Factorization of Polynomials, 18 Divisibility in Integral Domains.
|
|
20
|
+
- **Fields and extensions**: 19 Vector Spaces, 20 Extension Fields, 21 Algebraic Extensions, **22 Finite Fields**, 23 Geometric Constructions.
|
|
21
|
+
- **Advanced group theory**: 24 Sylow Theorems, 25 Finite Simple Groups, 26 Generators and Relations.
|
|
22
|
+
- **Applications of symmetry**: 27 Symmetry Groups, 28 Frieze and Crystallographic Groups, **29 Symmetry and Counting** (Burnside counting), 30 Cayley Digraphs of Groups.
|
|
23
|
+
- **Coding and Galois**: **31 Introduction to Algebraic Coding Theory** (Hamming codes, linear codes over finite fields), **32 An Introduction to Galois Theory**, 33 Cyclotomic Extensions.
|
|
24
|
+
|
|
25
|
+
## Core Structures Transferable to AI/Infra
|
|
26
|
+
|
|
27
|
+
| Algebraic structure (Ch) | One-line essence | ML/Infra correspondence |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| **Group action** (5, 29) | G acts on input space; orbit-stabilizer theorem \|orbit\|=\|G\|/\|stab\| | Equivariant layers + **weight sharing**: parameters reused along an orbit |
|
|
30
|
+
| **Homomorphism** (10, 15) | Operation-preserving map phi(ab)=phi(a)phi(b) | **Representation** = homomorphism G->GL(V) (in the book GL(2,F)); learnable linear actions |
|
|
31
|
+
| **Quotient structure** (9, 14) | Quotienting by a normal subgroup/ideal = "forgetting" a symmetry direction | Pooling / coarsening / equivariant downsampling: quotienting by S_n = permutation-invariant aggregation |
|
|
32
|
+
| **Cyclic & finite abelian groups** (4, 11) | Z_n's periodic structure; any finite abelian group ~ product of cyclic groups | DFT/FFT, cyclic convolution, RoPE (representation of SO(2)/cyclic rotations) |
|
|
33
|
+
| **Ring** (12-14) -> axiom relaxation | Drop the additive inverse axiom -> **semiring** (the book only covers rings; semirings require axiom relaxation) | Generalized matrix multiply: replacing (x,+) with (+, min/max) yields **min-plus / tropical semiring** GEMM |
|
|
34
|
+
| **Finite field GF(p^n)** (22, 16, 17) | Finite elements, arithmetically closed and exact | Error-correcting codes, hashing/LSH, CRC, Shamir secret sharing, quantization codebooks |
|
|
35
|
+
| **Linear code** (31) | k-dimensional subspace over finite field F, generator matrix G, Hamming distance/weight | Robust storage/communication, gradient compression, fault-tolerant training, secure aggregation |
|
|
36
|
+
|
|
37
|
+
### Three Most Valuable Activated Mappings (Expanded)
|
|
38
|
+
|
|
39
|
+
- **Group action -> Equivariance + weight sharing**: The orbit-stabilizer theorem \|orbit\|=\|G\|/\|stab\| (Ch 5) is precisely the algebraic explanation for "weight sharing" -- inputs on the same orbit share a single set of parameters, and the number of independent parameters is compressed by a factor of \|G\|. The equivariance constraint f(g x)=rho(g) f(x) unifies a family of real models: CNN (translation group), Group Equivariant CNN (D_n rotation/reflection steerable filters), DeepSets / Set Transformer (S_n permutation-invariant), E(3)-equivariant GNN (rigid-body group, molecules/point clouds).
|
|
40
|
+
- **Ring axiom relaxation -> Semiring -> Generalized GEMM**: Chapters 12-14 of this book give the two operations (+,x) and axioms of a ring; **dropping the additive inverse** (i.e., no longer requiring subtraction to be invertible) yields a semiring. Abstracting the general matrix multiply C=A (x) B by replacing (x,+) with an arbitrary semiring ((.),(+)) allows a single codebase to express standard GEMM, Boolean reachability (AND,OR), shortest path/Viterbi/DTW (+, min). This is the unified perspective that puts "dynamic programming" inside matrix multiplication.
|
|
41
|
+
- **Finite fields -> Exactly reproducible arithmetic**: GF(p^n) (Ch 22, constructed from a polynomial ring modulo an irreducible polynomial, see Ch 16-17) has arithmetically closed and **floating-point-error-free** operations, making it naturally suited for scenarios requiring reproducibility, verifiability, and error correction: Reed-Solomon / Hamming error-correcting codes (Ch 31), Shamir secret sharing (polynomial interpolation over GF(p)), consistent hashing and LSH.
|
|
42
|
+
|
|
43
|
+
## Problem Types Suited for Activation
|
|
44
|
+
|
|
45
|
+
- Input has **clear symmetry**: translation, rotation, reflection (D_n), permutation (S_n), periodicity (Z_n) -- want equivariance/invariance rather than learning it from data augmentation.
|
|
46
|
+
- Need **hard selection / discrete routing** that remains end-to-end differentiable (Top-K, shortest path, alignment) -- can use semiring relaxation.
|
|
47
|
+
- Need **robust compression / communication / storage**: fault tolerance and compression for KV-Cache, gradients, checkpoints.
|
|
48
|
+
- Need to embed **periodic / cyclic structure** into positional encoding or token mixing.
|
|
49
|
+
- Need to express a **constraint / conservation law** as a learnable linear map that naturally maps to GEMM.
|
|
50
|
+
|
|
51
|
+
## Possible Algorithmic Inspirations
|
|
52
|
+
|
|
53
|
+
1. **G-equivariant layer** (source: 5, 6, 10): Implement the symmetry group G's action as permutation/rotation matrices rho(g), with weights shared along orbits, enforcing f(g x)=rho(g) f(x). In practice, for each g in G the input is transformed by rho(g) then passed through the same set of weights, and the results are aggregated -- equivalent to expanding the convolution kernel into a [\|G\|, C_out, C_in] batched GEMM, reducing independent parameters by a factor of \|G\|. Deployed models: CNN (translation), G-CNN (D_n steerable filters), DeepSets/Set Transformer (S_n), E(3)-equivariant GNN (point clouds/molecules).
|
|
54
|
+
2. **Tropical / min-plus attention and routing** (source: 12-14 ring axiom relaxation): Replace softmax's (x,)+exp with (+, min/max) min-plus matmul: score(+) = min_k(Q_ik + K_kj). Hard Top-K routing -> tropical semiring **piecewise-linear gating** (sub-differentiable -- breakpoints need LogSumExp smoothing, which recovers standard softmax; an alternative to the non-differentiable argmax). Viterbi, DTW, and shortest paths are all instances of this min-plus matmul -- see the Tropical Gating example in `../gpu-friendly-math.md` (used only for low-dimensional gating and **not on Tensor Cores** -- max/min runs on CUDA cores, while the main trunk still uses standard (x,+) GEMM).
|
|
55
|
+
3. **Permutation-invariant aggregation** (source: 9, 29): Quotienting by the S_n action yields sum/max/mean pooling -- naturally O(n) and parallel-friendly, the algebraic foundation for DeepSets rho(Sigma phi(x_i)) and GNN message passing. Burnside counting (Ch 29) can estimate "how many essentially distinct configurations exist" at design time, for predicting parameter savings and data deduplication benefits.
|
|
56
|
+
4. **Finite-field coding for Infra** (source: 22, 31): Linear codes c=mG (generator matrix G over GF(q)) for fault-tolerant storage and gradient compression, with Hamming distance providing a lower bound on error-correcting capability; Shamir secret sharing (polynomial interpolation over GF(p), recoverable from t shards) for secure aggregation in federated learning; finite-field hashing/CRC for deduplication and consistency verification. Commonality: pure int/bitwise operations, placed in pre/post-processing on data flows.
|
|
57
|
+
5. **Cyclic group representations -> Frequency-domain token mixing** (source: 4, 11): The irreducible representations of Z_n are the DFT basis, giving rise to FFT convolution (O(n log n) replacing O(n^2)), RoPE (encoding positions as SO(2) rotations, i.e., unitary representations of Z), and cyclic/relative positional encodings. The finite abelian group classification theorem (Ch 11) guarantees that any such periodic structure can be decomposed into cyclic components for FFT.
|
|
58
|
+
6. **Fourier / spectral methods on groups** (source: 10 homomorphism + 4/11): Extending the irreducible decomposition of representation theory to general finite groups yields the group convolution theorem -- spectral GNNs and spherical CNNs (harmonic analysis on SO(3)) are special cases; the cost is that fast transforms for non-abelian groups may not exist and complexity must be assessed.
|
|
59
|
+
|
|
60
|
+
## GPU Friendliness Warning
|
|
61
|
+
|
|
62
|
+
> The sole source for eight-dimension criteria and scoring rules: `../gpu-friendly-math.md` (Tensorization / GEMM-mappability / Complexity / Memory / Low-precision / Parallelism / Sparsity / Operator fusion). Here we give only per-dimension verdicts for the book's structures.
|
|
63
|
+
|
|
64
|
+
**Focus A: Can semiring GEMM run on Tensor Cores? -- Not by default.**
|
|
65
|
+
|
|
66
|
+
- **Dimension 2 GEMM-mappability ([x] unfriendly)**: Tensor Core hardware only performs (x,+) MAC (fp16/bf16/fp8 accumulated into fp32). Min-plus / max-plus uses (+, min/max), which is **not natively supported**; naive tropical GEMM degenerates to CUDA core scalar comparisons and cannot leverage Tensor Cores.
|
|
67
|
+
- **Dimension 3 Complexity ([x])**: Min-plus matrix multiplication has no Strassen-style sub-cubic speedup (equivalent to the APSP hard problem); naive O(n^3).
|
|
68
|
+
- **Dimension 5 Low-precision ([v])**: Tropical uses max/min, no exp overflow risk; numerically robust.
|
|
69
|
+
- **Dimension 8 Differentiability / Operator fusion ([~] adaptable)**: min/max is non-differentiable; requires relaxation.
|
|
70
|
+
- **Adaptations (-> eight-dimension friendly)**: (1) log-sum-exp smoothing of min/max recovers (x,+) stable softmax, back on Tensor Cores; (2) confine tropical operations **to low-dimensional gating only**, keeping the main trunk on standard (x,+) GEMM (the Tropical Gating example in the reference file = Dimension 1 [v] Tensorization / Dimension 2 [x] not Tensor Core GEMM / Dimension 3 [v] per-token gating sub-quadratic, Dimension 8 needs LogSumExp smoothing); (3) blocking -- standard GEMM within blocks, min-plus reduction between blocks.
|
|
71
|
+
|
|
72
|
+
**Focus B: Can group operations be tensorized? -- Small groups yes, large groups and exact arithmetic no.**
|
|
73
|
+
|
|
74
|
+
- **Finite group actions (Dimension 1/2 [v])**: Permutation matrices / dense representation matrices -> batched GEMM; G-CNN has been engineered. Orthogonal rotation/permutation matrices are numerically stable under bf16 (Dimension 5 [v]).
|
|
75
|
+
- **Large group enumeration (Dimension 3/4 [x])**: \|S_n\|=n! explodes; explicitly enumerating group elements -> memory and compute blow up. **Adaptation**: Use only generators (Ch 26 Generators and Relations) + Cayley graphs (Ch 30) for local propagation, without materializing the entire group.
|
|
76
|
+
- **Permutation = gather/scatter (Dimension 7/1/8 [x])**: Implementation as irregular gather/scatter causes warp divergence. **Adaptation**: Pre-compile fixed permutation patterns into structured sparsity or dense indexing.
|
|
77
|
+
- **Finite fields / modular arithmetic (Dimension 2 [x], Dimension 1/6 [v])**: mod p, GF(2^n), XOR, table lookups **are not floating-point MAC** and cannot use Tensor Cores, but are highly parallel on int/bitwise kernels. Conclusion: **suitable for pre/post-processing in coding/hashing, never to be placed in the training main-trunk GEMM**. Galois / exact-field arithmetic requires int, is non-differentiable, violating Dimension 8.
|
|
78
|
+
|
|
79
|
+
**Scoring conclusion**: Group equivariance (small groups, dense representations) and frequency-domain structures = mathematical beauty x GPU friendliness, can enter the main trunk; semirings, finite fields, exact Galois = require relaxation/isolation first, otherwise can only serve as auxiliary operators or offline tools.
|
|
80
|
+
|
|
81
|
+
**Worked example comparison (candidate x eight-dimension verdict):**
|
|
82
|
+
|
|
83
|
+
| Candidate design | Algebraic source (Ch) | Key dimension verdicts | Enter main trunk? |
|
|
84
|
+
|---|---|---|---|
|
|
85
|
+
| Equivariant layer (small groups D_n/S_n, permutation+rotation matrices) | 5, 6, 10 | 1[v] 2[v] 5[v], large groups 3/4[x] | [v] (limited to small groups/generators) |
|
|
86
|
+
| Tropical gating (low-dim min-plus replacing hard Top-K) | 12-14 relaxation | 1[v] 2[v] 8 needs relaxation, main trunk still (x,+) | [v] (gating only) |
|
|
87
|
+
| Pure min-plus main-trunk attention | 12-14 relaxation | 2[x] Tensor Core unsupported, 3[x] O(n^3) | [x] (needs log-sum-exp smoothing first) |
|
|
88
|
+
| Finite-field coding for KV/gradient compression | 22, 31 | 2[x] not floating-point MAC, 1/6[v] int parallel | [x] main trunk / [v] pre/post-processing |
|
|
89
|
+
| Cyclic group representations (FFT/RoPE token mixing) | 4, 11 | 1[v] 2[v] 3[v] (n log n) | [v] |
|
|
90
|
+
|
|
91
|
+
## Which Thinking Lens to Invoke
|
|
92
|
+
|
|
93
|
+
- **Primary: `symmetry` (symmetry and invariance)** -- group actions, equivariance/invariance, orbit-stabilizer are the largest interface between this book and ML.
|
|
94
|
+
- **Secondary: `categorical`** -- extract the common structure of groups/rings/fields, see through "different modules are actually the same algebraic object."
|
|
95
|
+
- **Secondary: `axiomatization`** -- relax ring axioms to get semirings, check one by one whether assumed algebraic properties truly hold (guard against false symmetry).
|
|
96
|
+
- **Secondary: `duality`** -- homomorphisms/isomorphisms as equivalence transformations, FFT/frequency-domain transforms to simplify problems.
|
|
97
|
+
- **Secondary: `algorithmic`** -- finite group counting (Burnside, Ch 29), coding (Ch 31), finite field enumeration (Ch 22).
|
|
98
|
+
|
|
99
|
+
Typical combination chain: First `symmetry` to identify the group and invariants in the problem -> `categorical` to extract the common algebraic structure -> `axiomatization` to verify axioms truly hold (guard against false symmetry, guard against misusing subtraction) -> `duality` to realize as a learnable linear map -> finally pass through the `../gpu-friendly-math.md` eight-dimension acceptance gate.
|
|
100
|
+
|
|
101
|
+
## Anti-patterns
|
|
102
|
+
|
|
103
|
+
- **Forcing "group elegance" into the main trunk without Tensor Core support**: tropical / finite-field arithmetic used as GEMM, actually runs as CUDA core scalar throughout.
|
|
104
|
+
- **Enumerating the entire group** instead of using generators (\|S_n\|=n! / \|GL\| blows memory).
|
|
105
|
+
- **Insisting on exact Galois / finite-field structures**: leads to non-differentiability, requires int, blocks end-to-end gradients.
|
|
106
|
+
- **Over-constraining with symmetry**: writing "approximately symmetric / pseudo-symmetric" patterns as hard equivariance, stifling expressiveness and model exploration (echoing the agentic-workflow principle of "don't hardcode subjective biases").
|
|
107
|
+
- **Confusing rings with semirings**: assuming additive inverses exist for subtraction/inversion, which fails under min-plus -> correctness errors.
|
|
108
|
+
- **Building a pile of modules for "group elegance"**: when existing equivariant libraries/FFT suffice, don't create new skills; first exhaust existing model capabilities, then add structure (echoing the agentic-workflow principle of "don't create a bunch of skills right away").
|
|
109
|
+
- **Treating homomorphisms as isomorphisms**: Homomorphisms can lose information (have a kernel); mistakenly treating them as invertible bijections leads to reconstruction/decoding errors.
|
|
110
|
+
|
|
111
|
+
## Deep-dive Entry
|
|
112
|
+
|
|
113
|
+
> **Bibliographic information**: Joseph A. Gallian, *Contemporary Abstract Algebra*, 8th Edition, Brooks/Cole, Cengage Learning, 2013. ISBN 978-1-133-59971-5.
|
|
114
|
+
>
|
|
115
|
+
> **Activation method**: Place `Contemporary Abstract Algebra.pdf` in the `math_book/` folder at the project root; the Agent can then automatically search the original text. The PDF is not distributed via npm/git (copyright reasons) and must be obtained separately.
|
|
116
|
+
|
|
117
|
+
> **Full-fidelity lookup = Agent automatically searches the local PDF**: `math_book/Contemporary Abstract Algebra.pdf`. When precise definitions/theorems/examples are needed, have the Agent do targeted skimming by the following actual chapter numbers; do not paraphrase from memory. If the deployment environment lacks `math_book/`, stop at this distillation level.
|
|
118
|
+
|
|
119
|
+
- **Ch 5 Permutation Groups** -- Orbit / stabilizer: the root of equivariant weight sharing.
|
|
120
|
+
- **Ch 22 Finite Fields** -- GF(p^n) construction and arithmetic: the foundation for coding/hashing/secure aggregation.
|
|
121
|
+
- **Ch 29 Symmetry and Counting** -- Burnside counting: estimating equivalence class count / parameter savings.
|
|
122
|
+
- **Ch 31 Introduction to Algebraic Coding Theory** -- Hamming codes, linear codes over finite fields (generator matrix, Hamming distance/weight): compression and fault tolerance.
|
|
123
|
+
- **Ch 32 An Introduction to Galois Theory** -- The symmetry group of field extensions: an exemplar of structured transformations (deployment requires passing the GPU gate first).
|
|
124
|
+
- (Semiring relaxation starting point: Ch 12-14 ring axioms -> drop additive inverse -> tropical semiring.)
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#
|
|
1
|
+
# 抽象代数 / Abstract Algebra
|
|
2
2
|
|
|
3
3
|
> **Contemporary Abstract Algebra**, Eighth Edition · Joseph A. Gallian · Brooks/Cole, Cengage Learning.
|
|
4
4
|
> 本文件是「激活」参考:把书里的群/环/域结构映射到 ML/算法/Infra,不复述原文。全保真回查见文末「深挖入口」。
|
|
@@ -63,18 +63,18 @@
|
|
|
63
63
|
|
|
64
64
|
**焦点 A:半环 GEMM 能上 Tensor Core 吗?——默认不能。**
|
|
65
65
|
|
|
66
|
-
- **维度 2 GEMM
|
|
67
|
-
- **维度 3
|
|
68
|
-
- **维度 5
|
|
69
|
-
- **维度 8
|
|
70
|
-
- **改造(→ 八维转友好)**:① log-sum-exp 软化 min/max 退回 (×,+) 稳定 softmax,重上 Tensor Core;② 把热带运算**只放在低维门控**,主干仍走标准 (×,+) GEMM(范例文件中 Tropical Gating = dim 1
|
|
66
|
+
- **维度 2 GEMM 可映射([x]不友好)**:Tensor Core 硬件只做 (×,+) 的 MAC(fp16/bf16/fp8 累加到 fp32)。min-plus / max-plus 用的是 (+, min/max),**非原生**,朴素 tropical GEMM 退化到 CUDA core 标量比较,吃不到 Tensor Core。
|
|
67
|
+
- **维度 3 复杂度([x])**:min-plus 矩阵乘没有 Strassen 式亚立方加速(等价于 APSP 难题),朴素 O(n³)。
|
|
68
|
+
- **维度 5 低精度([v])**:tropical 用 max/min,无 exp 上溢,反而数值稳健。
|
|
69
|
+
- **维度 8 可微/融合([~]可改造)**:min/max 不可微,需松弛。
|
|
70
|
+
- **改造(→ 八维转友好)**:① log-sum-exp 软化 min/max 退回 (×,+) 稳定 softmax,重上 Tensor Core;② 把热带运算**只放在低维门控**,主干仍走标准 (×,+) GEMM(范例文件中 Tropical Gating = dim 1[v]张量化 / 2[x]非 Tensor Core GEMM / 3[v]逐 token 门控亚二次,dim 8 需 LogSumExp 软化);③ 分块——块内标准 GEMM、块间 min-plus 归约。
|
|
71
71
|
|
|
72
72
|
**焦点 B:群操作能张量化吗?——小群能,大群和精确算术不能。**
|
|
73
73
|
|
|
74
|
-
- **有限群作用(维度 1/2
|
|
75
|
-
- **大群枚举(维度 3/4
|
|
76
|
-
- **置换 = gather/scatter(维度 7/1/8
|
|
77
|
-
- **有限域 / 模算术(维度 2
|
|
74
|
+
- **有限群作用(维度 1/2 [v])**:置换矩阵 / 稠密表示矩阵 → batched GEMM,G-CNN 已工程化。正交的旋转/置换矩阵在 bf16 下数值稳定(维度 5 [v])。
|
|
75
|
+
- **大群枚举(维度 3/4 [x])**:\|Sₙ\|=n! 爆炸,显式枚举群元素 → 显存与算力爆。**改造**:只用生成元(26 Generators and Relations)+ Cayley 图(30)做局部传播,不物化整群。
|
|
76
|
+
- **置换 = gather/scatter(维度 7/1/8 [x])**:实现为不规则 gather/scatter 会 warp divergence。**改造**:固定置换模式预编译为结构化稀疏或 dense 索引。
|
|
77
|
+
- **有限域 / 模算术(维度 2 [x]、维度 1/6 [v])**:mod p、GF(2ⁿ)、XOR、查表**不是浮点 MAC**,吃不到 Tensor Core,但在 int/bitwise kernel 上高并行。结论:**适合放编码/哈希的前后处理,绝不放进训练主干 GEMM**。Galois/精确域算术要求 int、不可微,违反维度 8。
|
|
78
78
|
|
|
79
79
|
**判分结论**:群等变(小群、稠密表示)与频域结构 = 数学美 × GPU 友好,可进主干;半环、有限域、精确 Galois = 需先松弛/隔离,否则只能当辅助算子或离线工具。
|
|
80
80
|
|
|
@@ -82,11 +82,11 @@
|
|
|
82
82
|
|
|
83
83
|
| 候选设计 | 代数来源(章) | 关键维度裁决 | 进主干? |
|
|
84
84
|
|---|---|---|---|
|
|
85
|
-
| 群等变层(小群 Dₙ/Sₙ,置换+旋转矩阵)| 5, 6, 10 | 1
|
|
86
|
-
| 热带门控(低维 min-plus 取代硬 Top-K)| 12–14 松弛 | 1
|
|
87
|
-
| 纯 min-plus 主干注意力 | 12–14 松弛 | 2
|
|
88
|
-
| 有限域编码压缩 KV/梯度 | 22, 31 | 2
|
|
89
|
-
| 循环群表示(FFT/RoPE token mixing)| 4, 11 | 1
|
|
85
|
+
| 群等变层(小群 Dₙ/Sₙ,置换+旋转矩阵)| 5, 6, 10 | 1[v] 2[v] 5[v],群大时 3/4[x] | [v](限小群/生成元)|
|
|
86
|
+
| 热带门控(低维 min-plus 取代硬 Top-K)| 12–14 松弛 | 1[v] 2[v] 8 需松弛,主干仍 (×,+) | [v](仅门控)|
|
|
87
|
+
| 纯 min-plus 主干注意力 | 12–14 松弛 | 2[x] Tensor Core 不支持、3[x] O(n³) | [x](先 log-sum-exp 软化)|
|
|
88
|
+
| 有限域编码压缩 KV/梯度 | 22, 31 | 2[x] 非浮点 MAC,1/6[v] int 并行 | [x]主干 / [v]前后处理 |
|
|
89
|
+
| 循环群表示(FFT/RoPE token mixing)| 4, 11 | 1[v] 2[v] 3[v](n log n)| [v] |
|
|
90
90
|
|
|
91
91
|
## 该调用哪个思想透镜
|
|
92
92
|
|
|
@@ -94,7 +94,7 @@
|
|
|
94
94
|
- **副:`categorical`**——提取群/环/域的公共结构,看穿"不同模块其实是同一代数对象"。
|
|
95
95
|
- **副:`axiomatization`**——放松环公理得半环、逐条核对所设代数性质是否真成立(防伪对称)。
|
|
96
96
|
- **副:`duality`**——同态/同构作等价转换、FFT/频域变换简化问题。
|
|
97
|
-
- **副:`
|
|
97
|
+
- **副:`algorithmic`**——有限群计数(Burnside, 29)、编码(31)、有限域枚举(22)。
|
|
98
98
|
|
|
99
99
|
典型组合链:先 `symmetry` 识别问题里的群与不变量 → `categorical` 抽出公共代数结构 → `axiomatization` 核对公理是否真成立(防伪对称、防误用减法)→ `duality` 落成可学习线性映射 → 最后过 `../gpu-friendly-math.md` 八维门。
|
|
100
100
|
|
|
@@ -110,7 +110,7 @@
|
|
|
110
110
|
|
|
111
111
|
## 深挖入口
|
|
112
112
|
|
|
113
|
-
>
|
|
113
|
+
> **书目信息**:Joseph A. Gallian, *Contemporary Abstract Algebra*, 8th Edition, Brooks/Cole, Cengage Learning, 2013. ISBN 978-1-133-59971-5.
|
|
114
114
|
>
|
|
115
115
|
> **启用方式**:将 `Contemporary Abstract Algebra.pdf` 放入项目根目录的 `math_book/` 文件夹,Agent 即可自动搜索原文。PDF 不随 npm/git 分发(版权原因),需自行获取。
|
|
116
116
|
|
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
# Algebraic Geometry (The Rising Sea)
|
|
2
|
+
|
|
3
|
+
> *The Rising Sea: Foundations of Algebraic Geometry*, Ravi Vakil, Princeton University Press, 2025 edition (ISBN 978-0-691-26866-8).
|
|
4
|
+
> The title is taken from Grothendieck's "rising tide" (la mer monte) metaphor: rather than attacking nut-like problems head-on, let the waters of abstraction (categories, sheaves, cohomology) slowly rise, silently submerging and dissolving the problems.
|
|
5
|
+
> This file distills this "rising-tide abstraction" into activatable algorithmic structures for AI/GPU -- taking only the most transferable skeleton, not restating the original proofs.
|
|
6
|
+
|
|
7
|
+
## Overview
|
|
8
|
+
|
|
9
|
+
This book builds modern Grothendieck-school algebraic geometry from scratch across roughly 30 chapters: the core is to **translate geometric objects into commutative algebra**, then use **sheaves + cohomology** to measure "whether local data can be glued into global data."
|
|
10
|
+
|
|
11
|
+
For AI, what is truly transferable is not elliptic curves or divisors, but three pieces of abstract machinery:
|
|
12
|
+
|
|
13
|
+
- **Sheaves = local data + consistent gluing** -- the ontology of geometry-aware aggregation on graphs.
|
|
14
|
+
- **Cohomology = obstruction measure for global consistency** -- an algebraic criterion for hallucination/inconsistency.
|
|
15
|
+
- **Categories = unified interface for different constructions** -- gathering heterogeneous operators into a single set of adjoint relationships.
|
|
16
|
+
|
|
17
|
+
In one sentence: **sheaves govern "how to assemble local into global," cohomology governs "what goes wrong when assembly fails," and categories govern "saying all of this in one unified language."**
|
|
18
|
+
|
|
19
|
+
### Actual Chapter Map
|
|
20
|
+
|
|
21
|
+
By the book's chapter numbers (`*`/`**` denote advanced sections):
|
|
22
|
+
|
|
23
|
+
- **Part I Preliminaries**
|
|
24
|
+
- **Sec. 1 Just Enough Category Theory to Be Dangerous**: Sec. 1.1 categories and functors, Sec. 1.2 universal properties, Sec. 1.3 limits and colimits, Sec. 1.4 adjoints, Sec. 1.5 abelian categories, Sec. 1.6* spectral sequences.
|
|
25
|
+
- **Sec. 2 Sheaves**: Sec. 2.1 motivation (sheaf of smooth functions), Sec. 2.2 sheaf and presheaf definitions, Sec. 2.3 morphisms, Sec. 2.4 stalks & sheafification, Sec. 2.5 sheaf on a base, Sec. 2.6 OX-modules form an abelian category, Sec. 2.7 inverse image sheaf.
|
|
26
|
+
- **Part II Schemes**: Sec. 3 sets and topological spaces of affine schemes (Zariski topology, generic point), Sec. 4 structure sheaf and scheme definition, Sec. 5 properties of schemes, Sec. 6 quasicoherent sheaves.
|
|
27
|
+
- **Part III Morphisms of Schemes**: Sec. 7 morphisms of schemes (Sec. 7.7 Grassmannian first construction), Sec. 8 various finiteness morphisms (Sec. 8.4 Chevalley's theorem and elimination theory), Sec. 9 closed embeddings, Sec. 10 fiber products and base change (Sec. 10.6 Segre embedding), Sec. 11 separated and proper morphisms, varieties.
|
|
28
|
+
- **Part IV "Geometric" Properties of Schemes**: Sec. 12 dimension, Sec. 13 regularity and smoothness (Sec. 13.1 Zariski tangent space).
|
|
29
|
+
- **Part V Quasicoherent Sheaves on Schemes and Applications**: Sec. 14 vector bundles "=" locally free sheaves, Sec. 15 line bundles, maps to projective space, and divisors (Sec. 15.4 line bundles and Weil divisors), Sec. 16 line bundle properties (Sec. 16.2 ample/very ample, Sec. 16.4 Grassmannian as moduli space), Sec. 17 projective morphisms and relative Spec/Proj.
|
|
30
|
+
- **Sec. 18 Cech Cohomology of Quasicoherent Sheaves**: Sec. 18.1 desired properties, Sec. 18.2 definition and proofs, Sec. 18.3 cohomology of line bundles on projective space, Sec. 18.4 Riemann-Roch and arithmetic genus, Sec. 18.5 Serre duality first glimpse.
|
|
31
|
+
- Sec. 19 applications: curves, Sec. 20* intersection theory glimpse, Sec. 21 differentials, Sec. 22 Riemann-Hurwitz formula.
|
|
32
|
+
- **Part VI More Cohomological Tools**: **Sec. 23 Derived functors (Sec. 23.5 Cech cohomology agrees with derived-functor cohomology)**, Sec. 24 flatness, Sec. 25 cohomology and base change.
|
|
33
|
+
|
|
34
|
+
> Note: **Tropical geometry** is the "skeletonization" of algebraic geometry over the tropical/min-plus semiring, the tropicalization of the projective varieties in Sec. 15-17; the book does not have a dedicated chapter for it. The "tropical gating" below draws on this as a mathematical source, and references are **not tied to specific chapter numbers** (to avoid fabrication).
|
|
35
|
+
|
|
36
|
+
## Core Structures Transferable to AI/Infra
|
|
37
|
+
|
|
38
|
+
First the overview mapping, then detailed expansion:
|
|
39
|
+
|
|
40
|
+
| Mathematical concept (book section) | AI/ML correspondence | Engineering implementation |
|
|
41
|
+
|---|---|---|
|
|
42
|
+
| Sheaf (Sec. 2) | Geometry-aware information aggregation on graphs | Node = section, edge = transformation in message passing |
|
|
43
|
+
| Restriction map (Sec. 2.3, 14) | Directional feature transformation on edges | One low-rank linear map per edge = small GEMM |
|
|
44
|
+
| Sheaf Laplacian (Sec. 2, 14) | Geometric attention/diffusion operator | Propagation on L = delta^T delta |
|
|
45
|
+
| Cohomology H^0/H^1 (Sec. 18, 23) | Global consistency / hallucination criterion | Cech H^1 regularizer |
|
|
46
|
+
| Category + adjoint (Sec. 1.3-1.4) | Unified operator interface | pullback = alignment, pushforward = aggregation |
|
|
47
|
+
| Grassmannian + Plucker (Sec. 7.7, 16.4) | Subspace compression encoding | Plucker coordinate block summaries to compress KV-Cache |
|
|
48
|
+
| Flatness (Sec. 24) | Smooth distribution transition criterion | Fiber without jumps -> no rank collapse signal |
|
|
49
|
+
|
|
50
|
+
**1. Sheaf -> geometry-aware information aggregation on graphs** (Sec. 2)
|
|
51
|
+
|
|
52
|
+
- **Ontology**: a sheaf systematically assigns "to each open set/node a data space (stalk/section) + restriction maps along inclusion relations + a gluing axiom (local consistency implies global gluing)."
|
|
53
|
+
- **Mapping**: node features = sections, edges = restriction maps, message passing = enforcing consistency of adjacent sections under restrictions.
|
|
54
|
+
- **Implementation**: this is precisely the ontology of **cellular sheaf diffusion / sheaf neural networks**; standard GNNs are the "trivial sheaf" special case -- the sheaf structure injects directional geometric inductive bias into every edge.
|
|
55
|
+
|
|
56
|
+
**2. Restriction map -> one low-rank linear transformation per edge** (Sec. 2.3, 14)
|
|
57
|
+
|
|
58
|
+
- **Ontology**: each edge carries a learned linear map F(U) -> F(V).
|
|
59
|
+
- **Operator**: this defines the **sheaf Laplacian** L = delta^T delta (where delta is the coboundary operator); diffusion/attention is propagation on L; when all restriction maps are identity, this reduces to the standard graph Laplacian.
|
|
60
|
+
- **Implementation**: each edge map is low-rank -> a sequence of **small GEMMs**, naturally landing on Tensor Cores; the rank is a tunable expressivity knob.
|
|
61
|
+
|
|
62
|
+
**3. Cohomology -> global consistency / hallucination criterion** (Sec. 18, 23)
|
|
63
|
+
|
|
64
|
+
- **H^0** = global sections: all locally consistent data that can truly be glued into a global solution.
|
|
65
|
+
- **H^1** = **gluing obstruction**: pairwise locally consistent yet unable to assemble globally -- i.e., "self-consistent contradictions." Formally H^1 = ker delta^1 / im delta^0.
|
|
66
|
+
- **Criterion**: gives hallucination an **algebraic criterion** -- H^1 != 0 <=> the model is locally confident but globally conflicting.
|
|
67
|
+
- **Computability boundary**: **Cech cohomology (Sec. 18.2)** is computed directly from the overlaps of a cover, locally and cheaply; Sec. 23.5 proves that under good conditions it agrees with the expensive derived-functor cohomology -- this boundary line determines "which cohomology can run on a GPU."
|
|
68
|
+
|
|
69
|
+
**4. Category + universal properties + adjoints -> unified abstract interface** (Sec. 1)
|
|
70
|
+
|
|
71
|
+
- **Ontology**: pullback / pushforward (f^*, f_*) are an adjoint pair (Sec. 1.4); products, coproducts, fiber products are unified as (co)limits (Sec. 1.3).
|
|
72
|
+
- **Implementation**: using one abstract interface to unify different operators -- **pullback = feature alignment/resampling, pushforward = aggregation/pooling** -- with the adjoint relationship automatically guaranteeing compatibility, reducing hyperparameters and alignment bugs.
|
|
73
|
+
|
|
74
|
+
**5. Proj / projective + Plucker coordinates -> subspace compression encoding** (Sec. 7.7, 16.4, 15)
|
|
75
|
+
|
|
76
|
+
- **Ontology**: the Grassmannian parameterizes "k-dimensional subspaces," embedded into projective space via the **Plucker embedding (explicitly appearing in this book)** using exterior product coordinates (Plucker coordinates).
|
|
77
|
+
- **Implementation**: the subspace spanned by a set of KV vectors can be summarized by a small number of Plucker/exterior-product coordinates as **block summaries**, thereby compressing the KV-Cache -- storing "the subspace" rather than individual vectors.
|
|
78
|
+
|
|
79
|
+
**6. Flatness -> geometric criterion for smooth distribution transitions** (Sec. 24)
|
|
80
|
+
|
|
81
|
+
- **Ontology**: fibers of a flat morphism vary continuously without jumps over the base.
|
|
82
|
+
- **Implementation**: can serve as a geometric correctness signal for "smooth distribution transfer, no rank collapse" during training/fine-tuning (somewhat theoretical; implementation requires careful validation).
|
|
83
|
+
|
|
84
|
+
## Problem Types Suited for Activation
|
|
85
|
+
|
|
86
|
+
- **Attention and message passing on graph/set structures**: nodes carry heterogeneous feature spaces, edges carry directional transformations -> sheaf diffusion is more expressive than plain GNNs.
|
|
87
|
+
- **Multi-source/multi-view consistency**: multimodal, multi-agent, retrieval-augmented (RAG) local evidence to be glued into global answers -> H^0/H^1 measures consistency.
|
|
88
|
+
- **Hallucination / self-consistency detection and regularization**: need a differentiable penalty for "locally self-consistent but globally contradictory" -> Cech H^1 regularizer.
|
|
89
|
+
- **Memory compression for long-context inference**: KV subspace redundancy is high -> Plucker-style block summaries.
|
|
90
|
+
- **Sparse routing / gating (MoE, Top-K)**: need differentiable approximation of discrete choices -> tropical semiring piecewise-linear gating.
|
|
91
|
+
- **Need a framework unifying heterogeneous operators**: using the pullback/pushforward adjoint pair to unify alignment and aggregation.
|
|
92
|
+
|
|
93
|
+
## Possible Algorithmic Inspirations
|
|
94
|
+
|
|
95
|
+
**Tropical Sheaf Attention Trilogy** (consistent with the candidate validation examples in `../gpu-friendly-math.md`):
|
|
96
|
+
|
|
97
|
+
1. **Tropical Gating**
|
|
98
|
+
- Replace hard Top-K routing with **piecewise-linear** scoring on the max-plus semiring.
|
|
99
|
+
- Sub-differentiable (kink points need LogSumExp softening; after softening it reverts to standard softmax), tensorizable but **not Tensor Core GEMM** (max/min falls on CUDA cores) -- replacing non-differentiable discrete choices.
|
|
100
|
+
2. **Cellular Sheaf Diffusion**
|
|
101
|
+
- Attention = diffusion on a learnable sheaf Laplacian.
|
|
102
|
+
- One **low-rank restriction map per edge (= small GEMM)**, injecting edge-direction geometric inductive bias into attention.
|
|
103
|
+
3. **Cech Cohomology Regularizer**
|
|
104
|
+
- Compute first-order Cech H^1 on a **fixed finite cover** of attention maps.
|
|
105
|
+
- Serves as a hallucination/inconsistency penalty term; local and cheap. Differentiability requires a proxy (e.g., H^1-component projection norm ||(I - P_{im delta^0}) c||^2, differentiable via SVD/pseudoinverse, non-smooth at rank jumps), not exact Betti numbers.
|
|
106
|
+
|
|
107
|
+
**Other point inspirations:**
|
|
108
|
+
|
|
109
|
+
- **Low-rank basis KV compression (Grassmannian/Plucker perspective)**: represent each block's KV subspace by its **low-rank basis** (low-rank decomposition, kn or k(n-k) parameters) to compress memory. Note: Plucker coordinates themselves, at low rank, number C(n,k) and actually **expand** rather than compress, so the true compression comes from the low-rank basis rather than Plucker coordinates -- "Plucker" is a borrowed name. Compression ratio depends on the block's original redundancy and must be measured empirically.
|
|
110
|
+
- **Tropical-semiring MoE routing**: routing logits are piecewise-linear on max-plus, yielding **structured sparse** and differentiable expert selection.
|
|
111
|
+
- **Adjoint Pull/Push operator pair**: implementing up/downsampling and alignment/aggregation as an adjoint pair, enforcing compatibility and reducing hyperparameters.
|
|
112
|
+
|
|
113
|
+
## GPU Friendliness Warning
|
|
114
|
+
|
|
115
|
+
> **Required reading and sole authority**: `../gpu-friendly-math.md`
|
|
116
|
+
> Eight-dimension scorecard: D1 Tensorization, D2 GEMM-mappability, D3 Complexity, D4 Memory/KV, D5 Low-precision stability, D6 Parallelism & communication, D7 Sparse structure, D8 Operator fusion.
|
|
117
|
+
> **Mathematical beauty != computable**; any dimension that is "unfriendly and not reformable" means elimination.
|
|
118
|
+
|
|
119
|
+
**Can land as GEMM / sub-quadratic (friendly [v]):**
|
|
120
|
+
|
|
121
|
+
| Construction | Dimensions hit | Notes |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| Low-rank restriction maps | D1, D2, D4 | Batched small GEMM, saturates Tensor Cores, low-rank saves memory |
|
|
124
|
+
| Tropical gating | D1, D3 (not D2) | Tensorizable, per-token gating is sub-quadratic; **not GEMM** (does not land on Tensor Cores); sub-differentiable, max kink points need LogSumExp softening |
|
|
125
|
+
| Cech H^1 regularizer (fixed cover) | D3, D8 | Locally cheap, fusible with attention kernel (FlashAttention-style) |
|
|
126
|
+
| Low-rank basis block summaries (Plucker perspective) | D4 | Low-rank decomposition compresses KV, large inference memory reduction (compression ratio needs empirical measurement) |
|
|
127
|
+
|
|
128
|
+
**Beautiful but not computable (unfriendly [x], prohibited from training forward pass):**
|
|
129
|
+
|
|
130
|
+
- **Derived functor cohomology for general sheaf cohomology (Sec. 23)**
|
|
131
|
+
- Requires injective resolutions + spectral sequences, belonging to **symbolic algebra**.
|
|
132
|
+
- No tensorization, no GEMM, not sub-quadratic, not differentiable -> violates D1, D2, D3, D5, D8.
|
|
133
|
+
- Can only be done offline, small-scale, for analysis; **use only the Cech version that agrees with it (Sec. 23.5) and restrict to fixed covers**.
|
|
134
|
+
- **Topological layers of Spec/Proj (Zariski topology, generic point, Sec. 3-4)**: discrete non-numerical structures, not differentiable -> violates D1. Serve only as conceptual scaffolding.
|
|
135
|
+
- **Ideal elimination / Groebner bases (Chevalley's theorem and elimination theory, Sec. 8.4)**: combinatorial explosion, serial, unstructured -> violates D3, D6, D7.
|
|
136
|
+
- **Dynamic / unstructured Cech covers**: if the cover changes with input or overlaps are irregular, H^1 computation degenerates into random gather/scatter -> violates D7. **Must use fixed finite covers + structured (block/banded) overlaps.**
|
|
137
|
+
|
|
138
|
+
## Which Design Lens to Invoke
|
|
139
|
+
|
|
140
|
+
- **topological (primary)**: cohomology, H^1 obstructions, invariants under continuous deformation -- precisely the motivating theme of sheaf/cohomology activation.
|
|
141
|
+
- **categorical**: extracting the essence of message passing via "sheaf = local data + gluing," abstracting engineering operators as restriction maps.
|
|
142
|
+
- **duality**: tropicalization, Plucker embedding, pullback/pushforward -- using equivalence transformations to move hard-to-compute problems into computable coordinate systems.
|
|
143
|
+
- **symmetry**: projective invariance, gauge symmetry, sheaf covariance, constraining the model's equivariant structure.
|
|
144
|
+
- **axiomatization**: treating the sheaf axioms (locality + gluing) and the cohomology long exact sequence as correctness constraints, auditing "whether the required consistency is violated."
|
|
145
|
+
|
|
146
|
+
## Anti-patterns
|
|
147
|
+
|
|
148
|
+
- **Putting abstract/derived-functor cohomology directly into the training forward pass**: symbolic computation, gradient-free, not computable -- the classic "beautiful but not computable," violating the GPU eight dimensions. Use Cech + fixed covers with differentiable proxies.
|
|
149
|
+
- **Introducing the full Scheme/Proj machinery to "look advanced"** when the task only needs a graph Laplacian: over-engineering, violating "Simplicity First." Ask first: "does a trivial sheaf (= standard GNN) suffice?"
|
|
150
|
+
- **Treating Zariski topology / generic points and other discrete structures as differentiable objects** to optimize: type error.
|
|
151
|
+
- **Making Cech covers dynamic with unstructured overlaps**: degenerates into random memory access, destroying GPU parallelism (D7).
|
|
152
|
+
- **Using the max-plus semiring as an exact semiring** without relaxing at non-differentiable points: gradient breaks, training stalls.
|
|
153
|
+
- **Blindly trusting H^1=0 to mean "no hallucination"**: it only guarantees local gluing consistency under the chosen cover, not factual correctness; it is a structural consistency signal, not a truth criterion.
|
|
154
|
+
|
|
155
|
+
## Deep Dive Entry
|
|
156
|
+
|
|
157
|
+
> **Bibliographic info**: Ravi Vakil, *The Rising Sea: Foundations of Algebraic Geometry*, Princeton University Press, 2025. ISBN 978-0-691-26866-8.
|
|
158
|
+
>
|
|
159
|
+
> **Activation method**: Place `The Rising Sea Foundations of Algebraic Geometry.pdf` in the `math_book/` folder at the project root; the Agent can then automatically search the original text. The PDF is not distributed via npm/git (copyright reasons); obtain it independently.
|
|
160
|
+
|
|
161
|
+
**Full-fidelity lookup**: when the summary is insufficient to support design, have the Agent automatically retrieve the local PDF
|
|
162
|
+
`math_book/The Rising Sea Foundations of Algebraic Geometry.pdf`
|
|
163
|
+
(using `pdftotext -f <start> -l <end>` for targeted pages, **do not dump the entire book**).
|
|
164
|
+
|
|
165
|
+
Actual chapter numbers worth deep reading:
|
|
166
|
+
|
|
167
|
+
1. **Sec. 2 Sheaves** (focus on Sec. 2.2 definitions, Sec. 2.4 stalks and sheafification, Sec. 2.5 sheaf on a base, Sec. 2.7 inverse image sheaf) -- the mathematical foundation for cellular sheaf diffusion and restriction maps.
|
|
168
|
+
2. **Sec. 18 Cech Cohomology of Quasicoherent Sheaves** (Sec. 18.1 desired properties, Sec. 18.2 definition and proofs, Sec. 18.4 Riemann-Roch, Sec. 18.5 Serre duality first glimpse) -- the source and computability boundary for H^1 hallucination regularization.
|
|
169
|
+
3. **Sec. 23 Derived Functors** (Sec. 23.5 Cech cohomology agrees with derived-functor cohomology) -- drawing the red line between "computable Cech vs. non-computable derived functors."
|
|
170
|
+
4. **Sec. 7.7 + Sec. 16.4 Grassmannian** (together with Sec. 15 line bundles and divisors; Plucker embedding appears within this framework) -- the geometric basis for Plucker KV compression.
|
|
171
|
+
5. **Sec. 1 Category Theory** (Sec. 1.3 limits/colimits, Sec. 1.4 adjoints) -- the pullback/pushforward adjoint pair and unified operator interfaces.
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#
|
|
1
|
+
# 代数几何(The Rising Sea)/ Algebraic Geometry
|
|
2
2
|
|
|
3
3
|
> 《The Rising Sea: Foundations of Algebraic Geometry》,Ravi Vakil 著,Princeton University Press,2025 年版(ISBN 978-0-691-26866-8)。
|
|
4
4
|
> 书名取自 Grothendieck 的"涨潮(la mer monte)"比喻:不正面强攻坚果般的难题,而是让抽象之水(范畴、层、上同调)缓缓上涨,悄无声息地淹没并瓦解问题。
|
|
@@ -116,7 +116,7 @@
|
|
|
116
116
|
> 八维记分卡:①张量化 ②GEMM 可映射 ③复杂度 ④显存/KV ⑤低精度稳定 ⑥并行与通信 ⑦稀疏结构 ⑧算子融合。
|
|
117
117
|
> **数学美 ≠ 可算**;任一维"不友好且不可改造"即淘汰。
|
|
118
118
|
|
|
119
|
-
**可落 GEMM / 亚二次(友好
|
|
119
|
+
**可落 GEMM / 亚二次(友好 [v]):**
|
|
120
120
|
|
|
121
121
|
| 构造 | 命中维度 | 说明 |
|
|
122
122
|
|---|---|---|
|
|
@@ -125,7 +125,7 @@
|
|
|
125
125
|
| Čech H¹ 正则(固定覆盖) | ③⑧ | 局部廉价,可与注意力 kernel 融合(FlashAttention 式)|
|
|
126
126
|
| 低秩基底块摘要(Plücker 视角) | ④ | 低秩分解压缩 KV,推理显存大降(压缩率需实测) |
|
|
127
127
|
|
|
128
|
-
**美但不可算(不友好
|
|
128
|
+
**美但不可算(不友好 [x],禁止塞进训练 forward):**
|
|
129
129
|
|
|
130
130
|
- **一般层上同调的导出函子计算(derived functor cohomology,§23)**
|
|
131
131
|
- 需内射分解(injective resolution)+ 谱序列(spectral sequence),属**符号代数**。
|
|
@@ -137,11 +137,11 @@
|
|
|
137
137
|
|
|
138
138
|
## 该调用哪个思想透镜
|
|
139
139
|
|
|
140
|
-
- **topological
|
|
140
|
+
- **topological(拓扑透镜)** — 主力:上同调、H¹ 障碍、连续变形下的不变量,正是层/上同调激活的母题。
|
|
141
141
|
- **categorical(范畴化透镜)**:用"层 = 局部数据 + 黏合"提取消息传递的本质,把工程算子抽象为限制映射。
|
|
142
142
|
- **duality(对偶透镜)**:tropicalization、Plücker 嵌入、pullback/pushforward —— 用等价转换把难算问题搬到可算坐标系。
|
|
143
|
-
- **symmetry
|
|
144
|
-
- **axiomatization
|
|
143
|
+
- **symmetry(对称与不变性)**:射影不变、规范(gauge)对称、层的协变性,约束模型的等变结构。
|
|
144
|
+
- **axiomatization(公理化)**:把层公理(局部性 + 黏合)、上同调长正合列当作正确性约束,审查"该有的一致性是否被违反"。
|
|
145
145
|
|
|
146
146
|
## 反模式
|
|
147
147
|
|
|
@@ -154,7 +154,7 @@
|
|
|
154
154
|
|
|
155
155
|
## 深挖入口
|
|
156
156
|
|
|
157
|
-
>
|
|
157
|
+
> **书目信息**:Ravi Vakil, *The Rising Sea: Foundations of Algebraic Geometry*, Princeton University Press, 2025. ISBN 978-0-691-26866-8.
|
|
158
158
|
>
|
|
159
159
|
> **启用方式**:将 `The Rising Sea Foundations of Algebraic Geometry.pdf` 放入项目根目录的 `math_book/` 文件夹,Agent 即可自动搜索原文。PDF 不随 npm/git 分发(版权原因),需自行获取。
|
|
160
160
|
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# Manifolds & Differential Geometry
|
|
2
|
+
|
|
3
|
+
> **Manifolds and Differential Geometry** -- Jeffrey M. Lee
|
|
4
|
+
> American Mathematical Society, *Graduate Studies in Mathematics*, Volume 107 (2009), ISBN 978-0-8218-4815-9.
|
|
5
|
+
> MSC: 58A05, 53C05, 22E15, 53C20, 53B30, 55R10. This file is an "activation" summary, not a verbatim transcription; for full-fidelity lookups see the "Deep-dive Entry" at the end.
|
|
6
|
+
|
|
7
|
+
## Overview
|
|
8
|
+
|
|
9
|
+
This book is a graduate-level textbook that builds from scratch: **smooth manifold -> tensors / differential forms -> connections and curvature -> Riemannian geometry**. Its central thread is not "computing in Euclidean space" but rather **doing calculus on curved spaces**: when there are no global coordinates and no "natural" vector addition, how does one define differentiation, compare vectors at different points, and measure distance and curvature? This is precisely the language deep learning needs once the default assumption that "parameter space is flat Euclidean" is abandoned.
|
|
10
|
+
|
|
11
|
+
A remark in the preface is critically important for AI research: **a connection on a fiber bundle and a gauge field in physics are the same concept, independently discovered by mathematicians and physicists** (preface, footnote 2). This is the archetype of the "cross-domain activation" that this skill pack repeatedly emphasizes -- the structure was already there; what was missing was someone connecting it to algorithm design (gauge-equivariant networks are the product of exactly that connection).
|
|
12
|
+
|
|
13
|
+
Actual chapter map (ordered by dependency; chapter and section numbers verified against the PDF table of contents):
|
|
14
|
+
|
|
15
|
+
| Ch | Title | Hook to AI |
|
|
16
|
+
|----|-------|-----------|
|
|
17
|
+
| 1-2 | Differentiable Manifolds / The Tangent Structure | Tangent space = local linearization, the space to which gradients belong |
|
|
18
|
+
| 3 | Immersion and Submersion | Submanifolds, dimensionality reduction / embedding |
|
|
19
|
+
| 4 | Curves and Hypersurfaces in Euclidean Space | Geometric intuition source for Gauss / mean curvature |
|
|
20
|
+
| 5 | Lie Groups | Continuous symmetry groups, exponential map, adjoint representation -> equivariant architectures |
|
|
21
|
+
| 6 | Fiber Bundles (S6.1 general bundles, S6.2 vector bundles, S6.8 principal and associated bundles) | **Geometric skeleton of gauge equivariance** |
|
|
22
|
+
| 7 | Tensors (S7.6 Metric Tensors) | Metric tensor g = inner-product field -> Fisher metric |
|
|
23
|
+
| 8 | Differential Forms (S8.5 bundle-valued forms) | Antisymmetric tensors, exterior derivative, gauge field strength |
|
|
24
|
+
| 9 | Integration and Stokes' Theorem (S9.8 Electromagnetism) | Maxwell = instantiation of U(1) connection curvature |
|
|
25
|
+
| 10 | De Rham Cohomology | Global topological invariants (integral conservation laws) |
|
|
26
|
+
| 11 | Distributions and Frobenius' Theorem | Integrability, constraint distributions |
|
|
27
|
+
| 12 | Connections and Covariant Derivatives (S12.2 connection forms, S12.4 Ehresmann, S12.5/S12.10 curvature, S12.12 G-connections) | **Parallel transport + curvature** |
|
|
28
|
+
| 13 | Riemannian & Semi-Riemannian Geometry (S13.1 Levi-Civita, S13.2 Riemann curvature, S13.4 geodesics, S13.7 Jacobi fields, S13.11 Rauch comparison) | **Natural gradient / optimization terrain** |
|
|
29
|
+
|
|
30
|
+
## Core Structures Transferable to AI/Infra
|
|
31
|
+
|
|
32
|
+
Each entry follows **geometric concept -> mathematical core -> AI transfer**, making it easy to plug directly into algorithm design.
|
|
33
|
+
|
|
34
|
+
- **Riemannian metric g (S7.6 / S13.1) -> Natural gradient and information geometry.**
|
|
35
|
+
- Core: The metric tensor assigns an inner product <u,v>_g = u^T g v to the tangent space at each point, determining "what is close to what" and which direction counts as "steepest."
|
|
36
|
+
- Transfer: The natural metric on a family of probability distributions is the **Fisher information matrix (Fisher-Rao metric)**; parameter space is therefore not flat Euclidean but a curved manifold. The steepest-descent direction is not nabla L but **g^{-1} nabla L (natural gradient)**, which is invariant under reparameterization.
|
|
37
|
+
- **Connection / covariant derivative (S12.1-S12.4) -> Parallel transport.**
|
|
38
|
+
- Core: Tangent spaces at different points cannot be directly added; a connection nabla specifies "how to carry a vector along a curve to another point without introducing extra rotation." The **Levi-Civita connection** is the unique one that is metric-compatible and torsion-free.
|
|
39
|
+
- Transfer: Correctly transporting **momentum / historical gradients / second-order state** on the parameter manifold is the origin of vector transport in Riemannian SGD/Adam.
|
|
40
|
+
- **Curvature (S12.5 / S13.2 / S13.7) -> Optimization terrain (loss landscape).**
|
|
41
|
+
- Core: Curvature = how much a vector rotates after parallel transport around a small loop; it measures "whether the space is curved and whether paths are path-dependent," and is essentially the geometric avatar of the Hessian.
|
|
42
|
+
- Transfer: The curvature of the loss surface determines the condition number and sharpness; **Jacobi fields / Rauch comparison theorem (S13.7 / S13.11)** describe geodesic divergence-convergence, equivalent to stability vs. divergence of optimization trajectories.
|
|
43
|
+
- **Geodesics and exponential map (S13.4) -> "Straight-line steps" on a manifold.**
|
|
44
|
+
- Core: Geodesics are locally shortest paths; the exponential map exp_p(v) maps a tangent vector v back to the endpoint of the corresponding shortest path on the manifold.
|
|
45
|
+
- Transfer: When doing **constrained optimization** on SPD matrices, Stiefel / Grassmann manifolds, exp_p is the exact version of a retraction; geodesic interpolation in latent space respects the data manifold better than Euclidean straight lines.
|
|
46
|
+
- **Fiber bundles / principal bundles + G-connections (S6.8 / S12.12 / S9.8) -> Gauge equivariance.**
|
|
47
|
+
- Core: A principal G-bundle packages "the arbitrary choice of local coordinate system / frame" as a group action on fibers; a connection on the bundle = gauge field, curvature = field strength.
|
|
48
|
+
- Transfer: Physical quantities should not depend on the choice of local frame; this "gauge freedom" is precisely the inductive bias behind **gauge-equivariant CNNs (convolution on spheres / meshes / general manifolds)**; S9.8 uses Maxwell's equations to provide a concrete example of U(1) connection curvature.
|
|
49
|
+
- **Lie groups and Lie algebras (S5) -> Continuous symmetry as prior.**
|
|
50
|
+
- Core: The exponential map exp: g -> G and the adjoint representation Ad provide the passage "infinitesimal generator -> finite transformation."
|
|
51
|
+
- Transfer: Encoding the symmetry group as a hard inductive bias of the network (equivariant layers, Lie-algebra-parameterized rotations / rigid-body transformations).
|
|
52
|
+
|
|
53
|
+
## Problem Types Suited for Activation
|
|
54
|
+
|
|
55
|
+
- Optimization is **ill-conditioned / converges slowly under Euclidean assumptions**, but the underlying parameters have a natural probabilistic or geometric structure (use a metric to redefine "distance").
|
|
56
|
+
- Data itself lives on **non-Euclidean manifolds**: covariance / SPD matrices, rotations SO(3), directional data, graphs and meshes, spherical signals.
|
|
57
|
+
- Need **rigorous symmetry / equivariance guarantees**: outputs transform predictably under rotations, translations, and local gauge transformations.
|
|
58
|
+
- Need to explicitly model **"arbitrariness of coordinate choice"** as a symmetry (multi-view, multi-frame, sensor-pose invariance).
|
|
59
|
+
- Want to use **geometric quantities (curvature / geodesic distance) for regularization or diagnostics**: sharpness, generalization, trajectory stability.
|
|
60
|
+
|
|
61
|
+
## Possible Algorithmic Inspirations
|
|
62
|
+
|
|
63
|
+
- **Natural gradient / K-FAC**: Use the Fisher metric as preconditioner; update direction = F^{-1} nabla L rather than nabla L.
|
|
64
|
+
- Key engineering: K-FAC approximates the per-layer Fisher as a Kronecker product **F ~ A (x) B** (A from input activations, B from output gradients); inversion reduces to inverting two small matrices, and the preconditioner application becomes a small GEMM (see scorecard below).
|
|
65
|
+
- **Information-geometric optimization**: View training as moving along Fisher-Rao geodesics on the distribution manifold.
|
|
66
|
+
- Mirror descent, Bregman divergences, and dual coordinates of exponential families are all special cases of this Hessian-metric framework; can be used to design optimizers insensitive to parameterization.
|
|
67
|
+
- **Riemannian optimization**: SGD/Adam on SPD / Stiefel / Grassmann / hyperbolic manifolds.
|
|
68
|
+
- The toolkit of three: retraction (a cheap approximation of exp), vector transport (a discrete version of parallel transport), momentum on manifolds; commonly used in metric learning, orthogonality constraints, hierarchical structure embeddings.
|
|
69
|
+
- **Gauge-equivariant CNN**: Introduce local gauge frames when convolving on manifolds / meshes.
|
|
70
|
+
- Use G-connections to align frames at neighboring points so that features are invariant to local coordinate choices; applicable to spherical signals, meshes, lattices, and other domains without global coordinates.
|
|
71
|
+
- **Curvature regularization / geometric perspective on SAM**: Use Hessian-vector products to estimate curvature.
|
|
72
|
+
- Penalize sharp minima (flat-minima preference), or use Jacobi fields to characterize trajectory divergence, providing a geometric interpretation for sharpness-aware training.
|
|
73
|
+
- **Geodesic interpolation and manifold augmentation**: Interpolate and sample along geodesics in latent / embedding space; respects the data manifold better than Euclidean straight lines; applicable to data augmentation and controllable generation.
|
|
74
|
+
|
|
75
|
+
## GPU Friendliness Warning
|
|
76
|
+
|
|
77
|
+
> Evaluate item by item using the **eight-dimension scorecard** from `../gpu-friendly-math.md`. The biggest pitfall in differential geometry is "inversion and materialization of metric / curvature matrices."
|
|
78
|
+
|
|
79
|
+
- **Dimension 2 GEMM-mappability / Dimension 3 Complexity -- Inversion is the make-or-break line.** Naive natural gradient requires inverting the N x N Fisher, where N is the parameter count (~10^9); O(N^3) inversion + O(N^2) memory means **immediate disqualification**.
|
|
80
|
+
- **Adaptable [v]**: **K-FAC** blocks F into a Kronecker product A (x) B, exploiting (A (x) B)^{-1} = A^{-1} (x) B^{-1}; only two small factors need inversion, and applying the preconditioner to the gradient **is a GEMM**. This is the affirmative answer to "can Kronecker factorization reduce to GEMM" -- yes, and this is the only form that can scale to a cluster.
|
|
81
|
+
- **Dimension 4 Memory -- Do not materialize the full Hessian / full curvature tensor.** The Riemann curvature tensor is order 4; full materialization blows up memory. Use **Hessian-vector products (HVP)** to extract curvature information in O(N) via a single backward pass, avoiding N x N.
|
|
82
|
+
- **Dimension 5 Low-precision -- Fisher / metric matrices are often ill-conditioned.** Inversion under bf16/fp16 catastrophically amplifies errors; **must add damping (Tikhonov, F + lambda I)** and keep inversion in fp32; otherwise it violates "low-precision stability."
|
|
83
|
+
- **Dimension 6 Parallelism -- Parallel transport / geodesics are serial ODEs.** Integrating the connection equation along a curve is a long serial recurrence with poor parallelism; in practice, one uses **single-step retractions / closed-form parallel transport** (analytic formulas exist for specific manifolds) instead of step-by-step integration.
|
|
84
|
+
- **Dimension 1 Tensorization / Dimension 2 GEMM-mappability -- Group convolution can be friendly, but continuous groups require caution.** Discrete groups (e.g., C_n, octahedral group) admit group convolutions that expand into GEMM [v]; continuous Lie groups require discretization by sampling, and improper sampling can break equivariance + produce irregular gather/scatter (Dimension 7 Sparsity unfriendly).
|
|
85
|
+
|
|
86
|
+
**Natural gradient / K-FAC eight-dimension scorecard (worked example):**
|
|
87
|
+
|
|
88
|
+
| Dimension | Naive Natural Gradient (full Fisher inversion) | K-FAC (Kronecker factorization) |
|
|
89
|
+
|-----------|------------------------------------------------|--------------------------------|
|
|
90
|
+
| Dimension 1 Tensorization | [x] explicit large-matrix inverse | [v] batched small-matrix algebra |
|
|
91
|
+
| Dimension 2 GEMM-mappability | [x] N x N inversion, not GEMM-able | [v] A^{-1} (x) B^{-1} application = GEMM chain |
|
|
92
|
+
| Dimension 3 Complexity | [x] O(N^3) | [v] two small factors, sub-cubic |
|
|
93
|
+
| Dimension 4 Memory | [x] materialize N x N | [v] store only two small factors |
|
|
94
|
+
| Dimension 5 Low-precision | [x] ill-conditioned, needs fp64 | [~] add damping + fp32 inversion, adaptable |
|
|
95
|
+
| Dimension 6 Parallelism | [~] single large inversion hard to parallelize | [v] per-layer factors independent, parallelizable |
|
|
96
|
+
| Dimension 7 Sparsity | -- | [v] block-diagonal structure |
|
|
97
|
+
| Dimension 8 Operator fusion | [x] | [v] preconditioner can be fused into optimizer kernel |
|
|
98
|
+
|
|
99
|
+
**Conclusion**: Riemannian / information-geometric methods **become GPU-feasible only when the metric is structurally factored (Kronecker / block-diagonal / low-rank)**; exact inversion and exact parallel transport are "beautiful but incomputable" and must be adapted or eliminated.
|
|
100
|
+
|
|
101
|
+
## Which Thinking Lens to Invoke
|
|
102
|
+
|
|
103
|
+
- **symmetry (symmetry and invariance) -- primary.** Gauge equivariance, Lie group symmetry, fiber bundles = encoding "frame / coordinate-choice invariance" as symmetry; the strongest interface between this book and DL.
|
|
104
|
+
- **variational -- co-primary.** Natural gradient, Riemannian SGD, curvature regularization are all "finding optima in curved constrained spaces."
|
|
105
|
+
- **duality**: Exponential / logarithmic maps, retractions, coordinate transformations to simplify problems.
|
|
106
|
+
- **geometric**: Explicitly model parameter / data spaces as manifolds, then translate back to algorithms.
|
|
107
|
+
- **topological**: Auxiliary -- de Rham cohomology / global invariants for conservation laws and integrability diagnostics.
|
|
108
|
+
|
|
109
|
+
Recommended combination: First `symmetry` to establish the symmetry structure -> `variational` to arrive at natural gradient / Riemannian optimization -> `duality` to handle retractions -> pass through the `../gpu-friendly-math.md` acceptance gate.
|
|
110
|
+
|
|
111
|
+
## Anti-patterns
|
|
112
|
+
|
|
113
|
+
Each entry gives **anti-pattern -> correct approach**:
|
|
114
|
+
|
|
115
|
+
- **Materializing and exactly inverting the full Fisher / full Hessian**: O(N^3) / O(N^2), infeasible on a cluster.
|
|
116
|
+
- Correct approach: First apply Kronecker / block-diagonal / low-rank factorization, then invert; without factorization, don't use natural gradient.
|
|
117
|
+
- **Directly inverting ill-conditioned metric matrices in fp16**: Catastrophic cancellation, results are noise.
|
|
118
|
+
- Correct approach: Add damping F + lambda I, keep inversion in fp32, use CG / Woodbury for implicit solves when necessary.
|
|
119
|
+
- **Step-by-step ODE integration for parallel transport / geodesics**: Serial recurrence kills parallelism.
|
|
120
|
+
- Correct approach: Use closed-form retractions / vector transport (analytic formulas exist for specific manifolds), replacing the recurrence with a single step.
|
|
121
|
+
- **Blindly discretizing continuous groups**: Equivariance silently breaks, and irregular gather/scatter patterns are introduced.
|
|
122
|
+
- Correct approach: Choose discrete subgroups that can be exactly represented, or use quadrature / frequency-domain (spherical harmonics) schemes with provable error bounds.
|
|
123
|
+
- **Forcing manifold structure for "geometric beauty"**: For the vast majority of tasks, Euclidean approximations suffice.
|
|
124
|
+
- Correct approach: Before introducing Riemannian machinery, first demonstrate experimentally that the Euclidean approach is genuinely ill-conditioned (condition number / convergence curves).
|
|
125
|
+
- **Treating curvature regularization as a panacea**: Curvature estimation is itself expensive and noisy.
|
|
126
|
+
- Correct approach: First validate the benefit at small scale using HVPs, confirm the signal-to-noise ratio, then scale up.
|
|
127
|
+
|
|
128
|
+
## Deep-dive Entry
|
|
129
|
+
|
|
130
|
+
> **Bibliographic information**: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Graduate Studies in Mathematics Vol. 107, American Mathematical Society, 2009. ISBN 978-0-8218-4815-9.
|
|
131
|
+
>
|
|
132
|
+
> **Activation method**: Place `Manifolds and Differential Geometry.pdf` in the `math_book/` folder at the project root; the Agent can then automatically search the original text. The PDF is not distributed via npm/git (copyright reasons) and must be obtained separately.
|
|
133
|
+
|
|
134
|
+
> Full-fidelity lookup = have the Agent **automatically search the local PDF** `math_book/Manifolds and Differential Geometry.pdf` (locate by chapter / section number; do not paraphrase from memory). This summary provides only coordinates, not a substitute for the original text.
|
|
135
|
+
|
|
136
|
+
- **S6.8 Principal and Associated Bundles** + **S12.12 G-Connections**: Geometric foundations of gauge equivariance (principal bundle + connection = gauge field).
|
|
137
|
+
- **S7.6 Metric Tensors** + **S13.1 Levi-Civita Connection**: The origin of metric tensors and natural gradient / Fisher metric.
|
|
138
|
+
- **S13.2 Riemann Curvature Tensor** + **S13.7 Jacobi Fields**: Curvature, optimization terrain, and trajectory stability.
|
|
139
|
+
- **S13.4 Geodesics** + **S13.11 Rauch's Comparison Theorem**: Geodesics / retractions and convergence-divergence comparison.
|
|
140
|
+
- **S9.8 Electromagnetism**: Gauge field as a concrete instance of U(1) connection curvature (a historical example of cross-domain activation).
|