math-skill 2.1.0 → 3.0.1

package/knowledge-base/lie-theory/lie-algebra.md

@@ -0,0 +1,51 @@
+# 李代数 (Lie Algebra)
+
+## 最小定义
+
+李代数 $\mathfrak{g}$ 是李群 $G$ 在单位元 $e$ 处的切空间 $T_e G$，配备李括号 $[\cdot, \cdot]: \mathfrak{g} \times \mathfrak{g} \to \mathfrak{g}$ 满足双线性、反对称性和 Jacobi 恒等式。它是"无穷小生成元"的线性空间——把曲的非线性群局部线性化为平的向量空间。
+
+## 核心公式
+
+- 李括号：$[X, Y] = XY - YX$（矩阵群），满足 $[X,[Y,Z]] + [Y,[Z,X]] + [Z,[X,Y]] = 0$
+- $\text{so}(3)$：反对称矩阵，$[\omega]_\times = \begin{pmatrix} 0 & -\omega_3 & \omega_2 \\ \omega_3 & 0 & -\omega_1 \\ -\omega_2 & \omega_1 & 0 \end{pmatrix}$
+- hat/vee 算子：$\hat{\cdot}: \mathbb{R}^n \to \mathfrak{g}$（向量→反对称矩阵），$\check{\cdot}: \mathfrak{g} \to \mathbb{R}^n$（逆）
+- Baker-Campbell-Hausdorff：$\log(\exp(X)\exp(Y)) = X + Y + \frac{1}{2}[X,Y] + \cdots$
+- 伴随表示：$\text{ad}_X(Y) = [X,Y]$，是 $\text{Ad}$ 的微分
+
+## 适用问题
+
+- 旋转/位姿的局部线性化：在小邻域内用李代数向量 $\delta \in \mathbb{R}^n$ 近似群上的非线性变化
+- 约束优化的重参数化：将正交/旋转约束转化为无约束李代数参数 + exp 映射
+- 对称性的无穷小描述：连续对称群由少数生成元完全刻画
+- 误差状态估计：协方差定义在切空间 $\mathbb{R}^n$ 上而非群上
+
+## AI 设计翻译
+
+- **李代数参数化层**：网络输出 $\delta \in \mathbb{R}^3$（so(3)），经 $\exp$ 得到合法旋转矩阵；替代四元数归一化或 6D 表示
+- **误差状态 EKF/优化层**：在标称状态 $X$ 附近用 $\delta \in \mathfrak{g}$ 参数化误差 $X_{\text{true}} = X \oplus \delta$，卡尔曼滤波在线性切空间进行
+- **李括号正则**：惩罚 $[\xi_i, \xi_j] \neq 0$ 来约束生成元的交换性，或用作对称性一致性损失
+- **生成元学习**：学习一组李代数基 $\{E_1, \ldots, E_n\}$ 作为可训练参数，实现数据驱动的对称性发现
+
+## 工程可行性
+
+GPU 友好度高。李代数的核心优势是"线性空间"：
+- **hat/vee 映射**：纯索引操作 + 符号翻转，$O(n)$，完美 GPU 友好
+- **李括号 $[X,Y] = XY - YX$**：两次小矩阵乘法 + 减法，$O(n^3)$ 小矩阵，可 batched
+- **线性组合 $\sum c_i E_i$**：向量加法 + 标量乘，$O(nd)$，完美 GPU 友好
+- **BCH 近似**：取前几项即够工程精度，$[X,Y]$ 计算为小矩阵乘法
+- 主要开销在 $\exp$ 而非代数运算本身：代数运算全部是线性操作，瓶颈在后续的 exp 映射
+
+## 风险与失效条件
+
+- **BCH 级数截断误差**：高阶项在大角度时不可忽略，一阶近似 $X+Y$ 仅对小扰动有效
+- **李括号非零的误读**：非交换群的 $[X,Y] \neq 0$ 意味着群合成顺序敏感，不能随意交换操作
+- **基的选择影响优化**：李代数基的选取不唯一，差的条件数会导致优化困难
+- **把李代数当全局坐标**：exp 映射只是局部微分同胚，大范围需要多张覆盖（atlas）
+- **左右约定不统一**：右雅可比 vs 左雅可比、局部 vs 全局坐标系，不统一则梯度与协方差错位
+
+## 深入参考
+
+- 蒸馏稿：references/books/micro-lie-theory.md（§II-C 切空间与李代数, hat/vee 算子）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 5 Lie Groups, 李代数部分）
+- 原书：Joan Sola et al., *A micro Lie theory*, §II-C（李代数定义与 hat/vee）、§II-D（exp/log 桥梁）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 5（李群与李代数）

package/knowledge-base/lie-theory/lie-group.en.md

@@ -0,0 +1,51 @@
+# Lie Group
+
+## Minimal Definition
+
+A Lie group $G$ is a space endowed with both a smooth manifold structure and a group structure, such that group multiplication $\mu: G \times G \to G$ and inversion $\iota: G \to G$ are smooth maps. It is the precise mathematical object of "continuous symmetry transformation groups" -- simultaneously a differentiable space and a multiplicative algebra.
+
+## Core Formulas
+
+- Exponential map: $\exp: \mathfrak{g} \to G$, $\exp(\xi) = \gamma_\xi(1)$, where $\gamma_\xi$ is the one-parameter subgroup
+- SO(3) Rodrigues formula: $\exp([\theta]_\times) = I + \frac{\sin\|\theta\|}{\|\theta\|}[\theta]_\times + \frac{1-\cos\|\theta\|}{\|\theta\|^2}[\theta]_\times^2$
+- SE(3): $\exp\begin{pmatrix} [\omega]_\times & v \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} \exp([\omega]_\times) & V(\theta)v \\ 0 & 1 \end{pmatrix}$
+- $\oplus/\ominus$ operators (right version): $X \oplus \tau = X \cdot \exp(\tau)$, $X \ominus Y = \log(Y^{-1} X)$
+- Adjoint map: $\text{Ad}_X(\tau) = X \tau X^{-1}$ (for matrix groups), satisfying $X \oplus \tau = (\text{Ad}_X \tau) \oplus X$
+
+## Applicable Problems
+
+- Predictions/regression targets carry geometric constraints: rotations, poses, unit quaternions -- forcing them into a Euclidean MLP violates constraints
+- Equivariance/invariance is required: when the input undergoes a rigid-body transformation, the output should covary or remain invariant (point clouds, molecules, multi-view geometry)
+- State evolves on a Lie group: inertial preintegration, motion models, differentiable physics/control
+- Orthogonal/unitary constraints on weight matrices: optimization on the Stiefel manifold
+
+## AI Design Translation
+
+- **Manifold-parameterized output head**: The network makes unconstrained predictions in the tangent space $\mathbb{R}^n$, then projects back to SO(3)/SE(3) via $\exp$, replacing the ad-hoc orthogonalization of 6D/9D rotation representations
+- **Manifold loss function**: Use $\ominus$ (geodesic error) as the loss, $L = \|X_{\text{pred}} \ominus X_{\text{gt}}\|^2$, naturally handling manifold topology
+- **Lie group RNN/ODE**: Hidden state $h_t \in G$, updated as $h_{t+1} = h_t \oplus f_\theta(h_t, x_t) = h_t \cdot \exp(f_\theta(h_t, x_t))$
+- **Orthogonal weight constraints**: $W = \exp(A)$ where $A$ is skew-symmetric, guaranteeing $W^T W = I$; alternatively, the Cayley transform $W = (I-A)(I+A)^{-1}$
+
+## Engineering Feasibility
+
+GPU friendliness: the key factor is whether exp/log has a closed form.
+- **SO(3)/SE(3) closed forms**: The Rodrigues formula is a finite algebraic expression on 3x3/4x4 small matrices, batchable as $[B,3,3]$/ $[B,4,4]$ tensors, per-sample independent, GPU-friendly
+- **General matrix exponential**: Taylor series + scaling-and-squaring iteration, with data-dependent step counts and divergent control flow, difficult to tensorize, GPU-unfriendly
+- **Small-matrix GEMM**: 3x3/4x4 matrices are too small to saturate Tensor Cores; value lies in "batchable and fusible" rather than "maximizing compute throughput"
+- **Low-precision critical issue**: $\sin\theta/\theta$ produces division by zero as $\theta \to 0$; $\log$ is singular as $\theta \to \pi$; fp16/bf16 yields NaN directly -- **must** implement Taylor expansion switching for small angles
+- **Kinematic chain seriality**: Discrete integration via successive $\exp$ multiplication is serial; rewriting as a parallel scan is required for parallelism
+
+## Risks and Failure Conditions
+
+- **Low-precision singularities**: Without Taylor expansion fallbacks at $\theta \to 0/\pi$, fp16 training produces NaN directly; fallback branches introduce warp divergence
+- **Mixing left and right perturbations**: Mixing $\oplus_R$ (right version / local frame) with $\oplus_L$ (left version / global frame) causes misalignment of covariances and gradients
+- **Treating SE(3) as T(3)xSO(3)**: Their tangent-space parameterizations differ; whether translation and rotation are coupled affects the Jacobian, and misuse introduces systematic bias
+- **Treating general matrix exp as an $O(1)$ operator**: Closed forms exist only for a few groups such as SO(3)/SE(3)/SE(2)/S1/S3; general groups require iterative series
+- **Over-application**: Imposing Lie group parameterization on tasks that do not need geometric constraints merely adds complexity and singularity risk
+
+## Further References
+
+- Distillation notes: references/books/micro-lie-theory.md (Section II-A Lie Group Definition, Section II-D Exponential Map, Section II-E Plus/Minus Operators)
+- Distillation notes: references/books/differential-geometry.md (Ch 5 Lie Groups)
+- Distillation notes: references/books/smooth-manifolds.md (Ch 7 Lie Groups)
+- Original text: Joan Sola et al., *A micro Lie theory*, Section II-A through Section II-F (complete Lie group toolkit)

package/knowledge-base/lie-theory/lie-group.md

@@ -0,0 +1,51 @@
+# 李群 (Lie Group)
+
+## 最小定义
+
+李群 $G$ 是同时具有光滑流形结构和群结构的空间，使得群乘法 $\mu: G \times G \to G$ 和求逆 $\iota: G \to G$ 都是光滑映射。它是"连续对称变换群"的精确数学对象——既是可微的空间，又是可乘的代数。
+
+## 核心公式
+
+- 指数映射：$\exp: \mathfrak{g} \to G$，$\exp(\xi) = \gamma_\xi(1)$，其中 $\gamma_\xi$ 是单参数子群
+- SO(3) Rodrigues 公式：$\exp([\theta]_\times) = I + \frac{\sin\|\theta\|}{\|\theta\|}[\theta]_\times + \frac{1-\cos\|\theta\|}{\|\theta\|^2}[\theta]_\times^2$
+- SE(3)：$\exp\begin{pmatrix} [\omega]_\times & v \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} \exp([\omega]_\times) & V(\theta)v \\ 0 & 1 \end{pmatrix}$
+- $\oplus/\ominus$ 算子（右版本）：$X \oplus \tau = X \cdot \exp(\tau)$，$X \ominus Y = \log(Y^{-1} X)$
+- 伴随映射：$\text{Ad}_X(\tau) = X \tau X^{-1}$（矩阵群），满足 $X \oplus \tau = (\text{Ad}_X \tau) \oplus X$
+
+## 适用问题
+
+- 预测/回归量带几何约束：旋转、位姿、单位四元数——硬塞进欧氏 MLP 会破坏约束
+- 需要等变性/不变性：输入做刚体变换，输出应协变或不变（点云、分子、多视几何）
+- 状态在李群上演化：惯性预积分、运动模型、可微物理/控制
+- 正交/酉约束的权重矩阵：Stiefel 流形上的优化
+
+## AI 设计翻译
+
+- **流形参数化输出头**：网络在切空间 $\mathbb{R}^n$ 自由预测，经 $\exp$ 投回 SO(3)/SE(3)，替代 6D/9D 旋转表示的 ad-hoc 正交化
+- **流形损失函数**：用 $\ominus$（测地误差）当 loss，$L = \|X_{\text{pred}} \ominus X_{\text{gt}}\|^2$，天然处理流形拓扑
+- **李群 RNN/ODE**：隐状态 $h_t \in G$，更新 $h_{t+1} = h_t \oplus f_\theta(h_t, x_t) = h_t \cdot \exp(f_\theta(h_t, x_t))$
+- **正交权重约束**：$W = \exp(A)$ 其中 $A$ 反对称，保证 $W^T W = I$；或用 Cayley 变换 $W = (I-A)(I+A)^{-1}$
+
+## 工程可行性
+
+GPU 友好度：关键看 exp/log 是否有闭式。
+- **SO(3)/SE(3) 闭式**：Rodrigues 公式是有限项代数表达式，3x3/4x4 小矩阵，可 batched 为 $[B,3,3]$/ $[B,4,4]$ 张量，逐样本独立，GPU 友好
+- **通用矩阵指数**：Taylor 级数 + scaling-and-squaring 迭代，步数数据相关、控制流发散、难张量化，GPU 不友好
+- **小矩阵 GEMM**：3x3/4x4 太小，吃不满 Tensor Core，价值在"可批量、可融合"而非"打满算力"
+- **低精度致命点**：$\sin\theta/\theta$ 在 $\theta \to 0$ 除零，$\theta \to \pi$ 时 log 奇异；fp16/bf16 直接 NaN，**必须** 对小角做 Taylor 展开切换
+- **运动链串行**：连乘 $\exp$ 的离散积分是串行递推，需改写成 parallel scan 才能并行
+
+## 风险与失效条件
+
+- **低精度奇异点**：$\theta \to 0/\pi$ 处不做 Taylor 兜底，fp16 训练直接 NaN；兜底分支又引入 warp divergence
+- **左右扰动混用**：$\oplus_R$（右版本/局部坐标系）和 $\oplus_L$（左版本/全局坐标系）混用导致协方差与梯度错位
+- **把 SE(3) 当成 T(3)xSO(3)**：二者切空间参数化不同，平移-旋转是否耦合改变雅可比，误用导致系统性偏差
+- **通用 matrix exp 当 O(1) 算子**：闭式只属于 SO(3)/SE(3)/SE(2)/S1/S3 等少数群，一般群需迭代级数
+- **过度套用**：不需要几何约束的任务硬上李群参数化，徒增复杂度与奇异点风险
+
+## 深入参考
+
+- 蒸馏稿：references/books/micro-lie-theory.md（§II-A 李群定义, §II-D 指数映射, §II-E 加减算子）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 5 Lie Groups）
+- 蒸馏稿：references/books/smooth-manifolds.md（Ch 7 Lie Groups）
+- 原书：Joan Sola et al., *A micro Lie theory*, §II-A ~ §II-F（完整李群工具链）

package/knowledge-base/lie-theory/representation.en.md

@@ -0,0 +1,52 @@
+# Representation
+
+## Minimal Definition
+
+A representation of a group $G$ on a vector space $V$ is a group homomorphism $\rho: G \to GL(V)$, realizing abstract group elements as computable linear transformations (matrices). The core problem of representation theory is decomposing a complex group action into a direct sum of irreducible representations (irreps), analogous to factoring integers into primes.
+
+## Core Formulas
+
+- Representation: $\rho(g_1 g_2) = \rho(g_1)\rho(g_2)$, $\rho(e) = I$
+- Character: $\chi_\rho(g) = \text{tr}(\rho(g))$, a class function encoding the essential information of the representation
+- Peter-Weyl theorem (compact groups): $L^2(G) \cong \bigoplus_{\pi \in \hat{G}} \dim(\pi) \cdot \pi$
+- Schur's lemma: $\rho_1, \rho_2$ irreducible and inequivalent $\Rightarrow$ $\text{Hom}_G(\rho_1, \rho_2) = 0$
+- Irreducible representations of SO(3): dimension $2l+1$, $l = 0,1,2,\ldots$, with spherical harmonics $Y_l^m$ as basis functions
+- Clebsch-Gordan decomposition: $\rho_1 \otimes \rho_2 \cong \bigoplus_k m_k \rho_k$
+
+## Applicable Problems
+
+- Signals defined on groups/spheres: require spherical harmonic expansion for frequency-domain analysis
+- Constructing equivariant feature spaces: each feature channel corresponds to an irreducible representation
+- Accelerating group convolutions: using the Fourier transform (irreducible representations) to convert convolution into pointwise multiplication in the frequency domain
+- Molecular/crystal symmetries: representations of point groups/space groups determine orbital symmetries and selection rules
+
+## AI Design Translation
+
+- **Spherical harmonic feature layer**: Expand 3D point cloud/molecular features into a spherical harmonic basis $Y_l^m$; each $(l,m)$ channel transforms according to an SO(3) irreducible representation, achieving strict rotational equivariance
+- **Group Fourier transform layer**: Transform a finite group signal $f: G \to \mathbb{R}$ to the frequency domain via $\hat{f}(\pi) = \sum_g f(g)\pi(g)$, converting convolution into per-irrep matrix multiplication
+- **Schur-constrained weight matrices**: Equivariant inter-layer maps $W: V_1 \to V_2$ must commute with the group action $W\rho_1(g) = \rho_2(g)W$; Schur's lemma forces $W$ to be block-diagonal/scalar, drastically reducing parameters
+- **Character pooling**: Extract invariants using $\chi_\rho(g) = \text{tr}(\rho(g))$ as input features for classification/regression
+
+## Engineering Feasibility
+
+GPU friendliness depends on the group size and representation dimension:
+- **Finite group representations**: Each irrep is a small matrix ($d \times d$); the group Fourier transform equals a batch of small GEMMs, $O(|G| \cdot d^2)$, batchable
+- **SO(3) spherical harmonic transform**: Fast algorithms exist ($O(L^2 \log L)$), but implementation is complex; the real-space to frequency-domain transform can be expressed as sparse matmul
+- **Clebsch-Gordan coefficients**: Once precomputed, they are fixed sparse tensors; contraction with features can be expressed as GEMM
+- **Sparsity from Schur constraints**: Weight matrices of equivariant layers are constrained to be block-diagonal/scalar, greatly reducing parameters but requiring sparse/block GEMM
+- Key bottleneck: the Clebsch-Gordan tensor size grows rapidly for high-dimensional irreps (large $l$ spherical harmonics)
+
+## Risks and Failure Conditions
+
+- **High-frequency spherical harmonic numerical instability**: $Y_l^m$ for large $l$ oscillates violently near the poles, causing severe precision loss under fp16
+- **Irrep completeness truncation**: Keeping only up to $l_{\max}$ spherical harmonics loses high-frequency information; truncation error must be determined experimentally
+- **Infinite-dimensional representations of non-compact groups**: Irreducible representations of groups such as the Lorentz group are infinite-dimensional and must be truncated in practice
+- **Freedom in representation selection**: Which irreps participate and to what order are hyperparameters with no automatic selection method
+- **Clebsch-Gordan tensor storage**: The number of CG coefficients grows as $O(l^3)$ with increasing $l$, increasing precomputation and storage costs
+
+## Further References
+
+- Distillation notes: references/books/micro-lie-theory.md (Section II-F Adjoint $\text{Ad}_X$ and Adjoint Matrix)
+- Distillation notes: references/books/differential-geometry.md (Ch 5 Lie Groups, adjoint representation section)
+- Original text: Joan Sola et al., *A micro Lie theory*, Section II-F (adjoint representation, equations 30--35)
+- Original text: Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 5 (representations of Lie groups)

package/knowledge-base/lie-theory/representation.md

@@ -0,0 +1,52 @@
+# 表示 (Representation)
+
+## 最小定义
+
+群 $G$ 在向量空间 $V$ 上的表示是群同态 $\rho: G \to GL(V)$，将抽象群元素实现为可计算的线性变换（矩阵）。表示论的核心问题是：把复杂的群作用分解为不可约表示（irrep）的直和，如同整数分解为素数。
+
+## 核心公式
+
+- 表示：$\rho(g_1 g_2) = \rho(g_1)\rho(g_2)$，$\rho(e) = I$
+- 特征标：$\chi_\rho(g) = \text{tr}(\rho(g))$，是类函数，编码表示的核心信息
+- Peter-Weyl 定理（紧群）：$L^2(G) \cong \bigoplus_{\pi \in \hat{G}} \dim(\pi) \cdot \pi$
+- Schur 引理：$\rho_1, \rho_2$ 不可约且不等价 $\Rightarrow$ $\text{Hom}_G(\rho_1, \rho_2) = 0$
+- SO(3) 的不可约表示：维数 $2l+1$，$l = 0,1,2,\ldots$，基函数为球谐函数 $Y_l^m$
+- Clebsch-Gordan 分解：$\rho_1 \otimes \rho_2 \cong \bigoplus_k m_k \rho_k$
+
+## 适用问题
+
+- 信号定义在群/球面上：需要用球谐展开做频域分析
+- 构建等变特征空间：每个特征通道对应一个不可约表示
+- 群卷积的加速：利用 Fourier 变换（不可约表示）将卷积变为频域逐点乘法
+- 分子/晶体对称性：点群/空间群的表示决定轨道对称性和选择定则
+
+## AI 设计翻译
+
+- **球谐特征层**：将 3D 点云/分子特征展开为球谐基 $Y_l^m$，每个 $(l,m)$ 通道按 SO(3) 不可约表示变换，实现严格旋转等变
+- **群 Fourier 变换层**：有限群信号 $f: G \to \mathbb{R}$ 经 $\hat{f}(\pi) = \sum_g f(g)\pi(g)$ 变换到频域，卷积变为逐 irrep 矩阵乘
+- **Schur 约束的权重矩阵**：等变层间映射 $W: V_1 \to V_2$ 必须与群作用交换 $W\rho_1(g) = \rho_2(g)W$，Schur 引理强制 $W$ 为块对角/标量，大幅减少参数
+- **特征标池化**：用 $\chi_\rho(g) = \text{tr}(\rho(g))$ 提取不变量作为分类/回归的输入特征
+
+## 工程可行性
+
+GPU 友好度取决于群的规模和表示维数：
+- **有限群表示**：每个 irrep 是小矩阵（$d \times d$），群 Fourier 变换 = 一批小 GEMM，$O(|G| \cdot d^2)$，可 batched
+- **SO(3) 球谐变换**：有快速算法（$O(L^2 \log L)$），但实现复杂；实空间→频域的变换可表达为稀疏 matmul
+- **Clebsch-Gordan 系数**：预计算后是固定的稀疏张量，与特征的缩并可 GEMM 化
+- **Schur 约束的稀疏性**：等变层的权重矩阵被约束为块对角/标量，参数大幅减少，但需要稀疏/分块 GEMM
+- 关键瓶颈：高维 irrep（大 $l$ 的球谐）的 Clebsch-Gordan 张量规模增长快
+
+## 风险与失效条件
+
+- **高频球谐数值不稳定**：大 $l$ 的 $Y_l^m$ 在极点附近振荡剧烈，fp16 下精度损失严重
+- **irrep 完备性截断**：只取到 $l_{\max}$ 阶球谐丢失高频信息，截断误差需要实验标定
+- **非紧群的表示无穷维**：Lorentz 群等的不可约表示是无穷维的，工程上必须截断
+- **表示选择的自由度**：哪些 irrep 参与、阶数多少是超参数，缺乏自动选择方法
+- **Clebsch-Gordan 张量的存储**：$l$ 增大时 CG 系数数量 $O(l^3)$ 增长，预计算和存储开销上升
+
+## 深入参考
+
+- 蒸馏稿：references/books/micro-lie-theory.md（§II-F 伴随 Ad_X 与伴随矩阵）
+- 蒸馏稿：references/books/differential-geometry.md（Ch 5 Lie Groups, 伴随表示部分）
+- 原书：Joan Sola et al., *A micro Lie theory*, §II-F（伴随表示，式 30-35）
+- 原书：Jeffrey M. Lee, *Manifolds and Differential Geometry*, Ch 5（李群的表示）

package/knowledge-base/matrix-analysis/low-rank-approximation.en.md

@@ -0,0 +1,49 @@
+# Low-Rank Approximation
+
+## Minimal Definition
+
+Given a matrix $A \in \mathbb{R}^{m \times n}$, find a matrix $B$ with rank at most $k$ that minimizes $\|A - B\|$. The Eckart-Young-Mirsky theorem guarantees that the truncated SVD provides the unique optimal solution under both the Frobenius norm and the spectral norm: $B_k = \sum_{i=1}^k \sigma_i u_i v_i^H$.
+
+## Core Formulas
+
+- SVD: $A = U\Sigma V^H = \sum_{i=1}^r \sigma_i u_i v_i^H$, $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_r > 0$
+- Truncated SVD (optimal rank-$k$ approximation): $A_k = U_k \Sigma_k V_k^H$
+- Eckart-Young error: $\|A - A_k\|_F = \sqrt{\sum_{i=k+1}^r \sigma_i^2}$, $\|A - A_k\|_2 = \sigma_{k+1}$
+- Randomized SVD: $A \approx Q(Q^HA)$, where $Q$ is the $Q$-factor from the QR decomposition of $A\Omega$ ($\Omega$ random Gaussian)
+- Effective rank: $r_{\text{eff}}(A) = \|A\|_F^2 / \|A\|_2^2 = \sum \sigma_i^2 / \sigma_1^2$
+- Nuclear norm (convex relaxation of rank): $\|A\|_* = \sum \sigma_i$, the dual of the spectral norm
+
+## Applicable Problems
+
+- LoRA weight compression: $W \approx W_0 + BA$, $B \in \mathbb{R}^{d \times r}, A \in \mathbb{R}^{r \times d}$, $r \ll d$
+- KV-Cache compression: projecting Key/Value caches into a low-dimensional subspace, reducing memory from $O(n)$ to $O(k)$
+- PCA / whitening: the top $k$ principal components of the data covariance matrix correspond to the truncated SVD
+- Gradient compression: the effective rank of gradient matrices in large models is often much lower than the nominal rank, allowing safe truncation
+- Recommender systems / matrix completion: low-rank factorization $R \approx UV^H$
+
+## AI Design Translation
+
+- **LoRA (Low-Rank Adaptation)**: Freeze $W_0$, train $\Delta W = BA$ ($r \ll d$), and merge $W = W_0 + BA$ at inference time. Forward pass = two matmul operations ($x \to Ax \to BAx$), reducing trainable parameters from $O(d^2)$ to $O(dr)$. Implemented via `torch.mm(B, torch.mm(A, x))` or merged into a single matmul.
+- **Randomized SVD operator**: For a large matrix $A \in \mathbb{R}^{m \times n}$, first sample $Y = A\Omega$ ($\Omega \in \mathbb{R}^{n \times (k+p)}$ random Gaussian), compute QR decomposition $Y = QR$, then form $B = Q^HA$ (small matrix $O(k \times n)$), and perform SVD on $B$. Total complexity $O(mnk)$ instead of $O(mn^2)$; all core operations are matmul.
+- **KV-Cache low-rank reduction**: Maintain $K_k = K P_k$ ($P_k$ being the projection onto the top $k$ principal components), performing incremental PCA or streaming SVD updates for each new token. Attention computation $\text{softmax}(Q K_k^H / \sqrt{d}) V_k$ involves three matmul operations, reducing the sequence dimension from $L$ to $k$.
+- **Nuclear norm regularization**: $\mathcal{L} = \mathcal{L}_{\text{task}} + \lambda \|W\|_*$ promotes low-rank solutions. However, nuclear norm computation requires full SVD ($O(n^3)$). Alternatives: (1) approximate with truncated SVD; (2) factorize $\|W\|_* = \min_{W=UV^H} \frac{1}{2}(\|U\|_F^2 + \|V\|_F^2)$, converting to Frobenius regularization on $U, V$.
+- **Gradient low-rank compression (distributed training)**: Truncate gradient $G$ to $G_k$ (top-$k$ SVD) before all-reduce, reducing communication from $O(d)$ to $O(kd)$. Use randomized SVD locally on each device, then merge.
+
+## Engineering Feasibility
+
+- **Primary operations**: matmul + small-matrix SVD. LoRA forward = two matmul operations; randomized SVD = three matmul operations + one small QR; truncated SVD = $O(k/n)$ fraction of full SVD (using Lanczos).
+- **GPU friendliness**: Extremely high. LoRA forward/backward are all tensor core matmul operations; the dominant cost of randomized SVD is also matmul. Small-matrix SVD has cuSOLVER batched implementations.
+- **Complexity**: LoRA forward $O(dk)$ per sample vs. $O(d^2)$ full rank; randomized SVD $O(mnk)$; full SVD $O(\min(m^2n, mn^2))$.
+- **Memory**: LoRA storage $O(dr)$ vs. $O(d^2)$; KV-Cache low-rank reduction $O(Lk)$ vs. $O(Ld)$.
+
+## Risks and Failure Conditions
+
+- **Incorrect rank selection**: $k$ too small causes information loss ($\sigma_{k+1}$ is non-negligible); $k$ too large negates the compression benefit. Solution: monitor the singular value decay curve and select the elbow point where $\sum_{i>k}\sigma_i^2 / \sum\sigma_i^2 < \epsilon$.
+- **Insufficient randomized SVD accuracy**: When oversampling $p$ is too small (typically $p = 5 \sim 10$) or the number of power iteration steps is insufficient, low-order singular value estimates can be significantly biased. Solution: add $q = 1 \sim 2$ power iteration steps $Y = (AA^H)^q A\Omega$, at the cost of additional matmul operations.
+- **LoRA not applicable to all layers**: Q/K/V in attention is typically effectively low-rank, but the effective rank of FFN layers and embeddings may be close to full rank, and applying LoRA forcefully degrades accuracy. Layer-by-layer effective rank diagnosis is required.
+- **SVD overhead of nuclear norm proximal**: Soft-thresholding $\text{prox}_{\lambda\|\cdot\|_*}(A) = U(\Sigma - \lambda I)_+ V^H$ requires SVD, which is prohibitively expensive for large matrices at every step. Solution: use factorization alternatives or randomized approximations.
+
+## Further References
+
+- Distilled notes: references/books/matrix-analysis.md (Section 2.6 SVD, Section 7.4 Polar Decomposition and SVD, nuclear norm-spectral norm duality Section 5.5)
+- Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 2 Section 2.6 (SVD) + Chapter 7 Section 7.3-7.4 (Polar Decomposition & SVD)

package/knowledge-base/matrix-analysis/low-rank-approximation.md

@@ -0,0 +1,49 @@
+# 低秩逼近 (Low-Rank Approximation)
+
+## 最小定义
+
+给定矩阵 $A \in \mathbb{R}^{m \times n}$，寻找秩不超过 $k$ 的矩阵 $B$ 使得 $\|A - B\|$ 最小。Eckart-Young-Mirsky 定理保证截断 SVD 给出 Frobenius 范数和谱范数下的唯一最优解：$B_k = \sum_{i=1}^k \sigma_i u_i v_i^H$。
+
+## 核心公式
+
+- SVD：$A = U\Sigma V^H = \sum_{i=1}^r \sigma_i u_i v_i^H$，$\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_r > 0$
+- 截断 SVD（最优秩-$k$ 逼近）：$A_k = U_k \Sigma_k V_k^H$
+- Eckart-Young 误差：$\|A - A_k\|_F = \sqrt{\sum_{i=k+1}^r \sigma_i^2}$，$\|A - A_k\|_2 = \sigma_{k+1}$
+- 随机化 SVD：$A \approx Q(Q^HA)$，$Q$ 为 $A\Omega$（$\Omega$ 随机高斯）的 QR 分解之 $Q$ 因子
+- 有效秩：$r_{\text{eff}}(A) = \|A\|_F^2 / \|A\|_2^2 = \sum \sigma_i^2 / \sigma_1^2$
+- 核范数（秩的凸松弛）：$\|A\|_* = \sum \sigma_i$，是谱范数的对偶
+
+## 适用问题
+
+- LoRA 权重压缩：$W \approx W_0 + BA$，$B \in \mathbb{R}^{d \times r}, A \in \mathbb{R}^{r \times d}$，$r \ll d$
+- KV-Cache 压缩：将 Key/Value 缓存投影到低维子空间，显存 $O(n) \to O(k)$
+- PCA / 白化：数据协方差的前 $k$ 个主成分即截断 SVD
+- 梯度压缩：大模型梯度矩阵的有效秩通常远低于名义秩，可安全截断
+- 推荐系统 / 矩阵补全：低秩因子分解 $R \approx UV^H$
+
+## AI 设计翻译
+
+- **LoRA (Low-Rank Adaptation)**：冻结 $W_0$，训练 $\Delta W = BA$（$r \ll d$），推理时合并 $W = W_0 + BA$。前向传播 = 两次 matmul（$x \to Ax \to BAx$），训练参数量从 $O(d^2)$ 降到 $O(dr)$。用 `torch.mm(B, torch.mm(A, x))` 或合并为单次 matmul。
+- **随机化 SVD 算子**：对大矩阵 $A \in \mathbb{R}^{m \times n}$，先采样 $Y = A\Omega$（$\Omega \in \mathbb{R}^{n \times (k+p)}$ 随机高斯），QR 分解 $Y = QR$，再算 $B = Q^HA$（小矩阵 $O(k \times n)$），对 $B$ 做 SVD。总复杂度 $O(mnk)$ 而非 $O(mn^2)$，核心操作全是 matmul。
+- **KV-Cache 低秩化**：维护 $K_k = K P_k$（$P_k$ 为投影到前 $k$ 主成分），每新到 token 做增量 PCA 或 streaming SVD 更新。Attention 计算 $\text{softmax}(Q K_k^H / \sqrt{d}) V_k$，三次 matmul，序列维度从 $L$ 降到 $k$。
+- **核范数正则化**：$\mathcal{L} = \mathcal{L}_{\text{task}} + \lambda \|W\|_*$ 促进低秩解。但核范数计算需完整 SVD（$O(n^3)$），替代方案：(1) 用截断 SVD 近似；(2) 因子化 $\|W\|_* = \min_{W=UV^H} \frac{1}{2}(\|U\|_F^2 + \|V\|_F^2)$ 转为对 $U, V$ 的 Frobenius 正则。
+- **梯度低秩压缩 (分布式训练)**：将梯度 $G$ 截断为 $G_k$（top-$k$ SVD）后 all-reduce 通信量从 $O(d)$ 降到 $O(kd)$。用随机化 SVD 在每卡本地算，再合并。
+
+## 工程可行性
+
+- **主要操作**：matmul + 小矩阵 SVD。LoRA 前向 = 两次 matmul；随机化 SVD = 三次 matmul + 一次小 QR；截断 SVD = 完整 SVD 的 $O(k/n)$ 倍（用 Lanczos）。
+- **GPU 友好度**：极高。LoRA 前向/反向全是 tensor core matmul；随机化 SVD 的主要开销也是 matmul。小矩阵 SVD 有 cuSOLVER 的 batched 实现。
+- **复杂度**：LoRA 前向 $O(dk)$ per sample vs. $O(d^2)$ 全秩；随机化 SVD $O(mnk)$；完整 SVD $O(\min(m^2n, mn^2))$。
+- **显存**：LoRA 存储 $O(dr)$ vs. $O(d^2)$；KV-Cache 低秩化 $O(Lk)$ vs. $O(Ld)$。
+
+## 风险与失效条件
+
+- **秩选择错误**：$k$ 过小导致信息丢失（$\sigma_{k+1}$ 不可忽略），$k$ 过大失去压缩意义。解决：监控奇异值衰减曲线，选 $\sum_{i>k}\sigma_i^2 / \sum\sigma_i^2 < \epsilon$ 的拐点。
+- **随机化 SVD 精度不足**：oversampling $p$ 太小（通常 $p = 5 \sim 10$）或 power iteration 次数不足时，低阶奇异值估计偏差大。解决：增加 $q = 1 \sim 2$ 步 power iteration $Y = (AA^H)^q A\Omega$，但增加 matmul 次数。
+- **LoRA 不适用于所有层**：Attention 的 Q/K/V 通常低秩有效，但 FFN 层和 embedding 的有效秩可能接近满秩，强行 LoRA 会损失精度。需逐层诊断有效秩。
+- **核范数 proximal 的 SVD 开销**：soft-thresholding $\text{prox}_{\lambda\|\cdot\|_*}(A) = U(\Sigma - \lambda I)_+ V^H$ 需要 SVD，大矩阵每步算不起。解决：用因子化替代或随机化近似。
+
+## 深入参考
+
+- 蒸馏稿：references/books/matrix-analysis.md（§2.6 SVD、§7.4 极分解与 SVD、核范数-谱范数对偶 §5.5）
+- 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 2 §2.6 (SVD) + Chapter 7 §7.3-7.4 (Polar Decomposition & SVD)

package/knowledge-base/matrix-analysis/matrix-perturbation.en.md

@@ -0,0 +1,49 @@
+# Matrix Perturbation Theory
+
+## Minimal Definition
+
+Studies how the eigenvalues, singular values, and eigenspaces of a matrix $A$ change under a small perturbation $E$. Core results: eigenvalues of Hermitian matrices are Lipschitz continuous with respect to perturbation (Weyl bound), while the stability of eigenvectors/eigenspaces is governed by the eigengap.
+
+## Core Formulas
+
+- **Weyl eigenvalue perturbation bound** (Hermitian): $|\lambda_i(A+E) - \lambda_i(A)| \leq \|E\|_2$
+- **Bauer-Fike theorem** (diagonalizable matrices): $\min_j |\lambda_i(A+E) - \lambda_j(A)| \leq \kappa(V) \|E\|_2$, where $V$ is the eigenvector matrix
+- **Davis-Kahan $\sin\Theta$ theorem**: $\|\sin\Theta(\hat{U}, U)\|_2 \leq \frac{\|E\|_2}{\delta}$, where $\delta$ is the gap between the subspace and the rest of the spectrum
+- **Singular value perturbation (Mirsky)**: $|\sigma_i(A+E) - \sigma_i(A)| \leq \|E\|_2$
+- **Geršgorin discs**: $\lambda_i(A) \in \bigcup_j \{z : |z - a_{jj}| \leq \sum_{k \neq j} |a_{jk}|\}$
+- **Condition number and relative error**: $\frac{|\delta x|}{|x|} \leq \kappa(A) \frac{\|\delta A\|}{\|A\|}$
+
+## Applicable Problems
+
+- Spectral drift analysis under quantization/low-precision training: how much do singular values of weight matrices shift under bf16/fp8?
+- Error bounds for model pruning/distillation: how large is the spectral change after removing $k$ parameters?
+- LoRA approximation error: how does the perturbation $\|W - W_0 - BA\|_2$ affect downstream outputs?
+- Training stability certificates: when gradient noise $\|E\|_2 \leq \epsilon$, eigenvalue drift is bounded by $\epsilon$
+- Numerical diagnostics: cheaply estimating spectral location via Geršgorin discs without running full EVD
+
+## AI Design Translation
+
+- **Geršgorin cheap spectral radius monitoring**: Every $N$ steps in the training loop, compute $\rho_{\text{est}} = \max_j (|a_{jj}| + \sum_{k\neq j}|a_{jk}|)$ as an upper bound on the spectral radius, requiring only $O(n^2)$ row-wise absolute-value summation. Implemented as `torch.sum(torch.abs(A), dim=1)`, an elementwise + reduce operation, extremely cheap, and embeddable in the training loop as a stability gate.
+- **Spectral drift bound for quantization error**: $W_{\text{quant}} = W + E$, $\|E\|_2 \leq \epsilon$; by the Weyl theorem, $\sigma_i$ shifts by $\leq \epsilon$. For an $L$-layer network, output perturbation $\leq \prod_i (\sigma_1(W_i) + \epsilon) - \prod_i \sigma_1(W_i)$. This guides quantization precision selection: if $\sigma_{\min}(W)$ is close to $\epsilon$, that layer requires higher precision.
+- **Davis-Kahan subspace stability**: The reliability of subspaces in PCA/LoRA is determined by the eigengap $\delta = \lambda_k - \lambda_{k+1}$. Larger $\delta$ yields a more stable truncated subspace; as $\delta \to 0$, the subspace becomes extremely sensitive to noise. This serves as a diagnostic tool for selecting the LoRA rank $r$: choose $r$ that maximizes $\delta_r$.
+- **Perturbation modeling for pruning**: Unstructured pruning = sparse perturbation $E$, $\|E\|_2 \leq \|E\|_F = \sqrt{\sum e_{ij}^2}$. The Weyl bound provides an upper bound on spectral drift, guiding the pruning ratio: maintain $\|E\|_2 / \sigma_1(W) < \tau$ (e.g., $\tau = 0.05$).
+- **Spectral regularization as a robustness certificate**: $\mathcal{L}_{\text{robust}} = \mathcal{L}_{\text{task}} + \lambda \max(0, \|E\|_2 - \epsilon)^2$, constraining spectral drift under perturbation. Combined with power iteration to estimate $\|E\|_2$, implemented as an additional loss term.
+
+## Engineering Feasibility
+
+- **Primary operations**: Geršgorin = elementwise abs + row-sum ($O(n^2)$); Weyl bound only requires $\|E\|_2$ (power iteration $O(n^2)$/step); Davis-Kahan requires eigengap (partial EVD, $O(n^2 k)$).
+- **GPU friendliness**: High. Geršgorin is pure elementwise + reduce; $\|E\|_2$ estimation is a matvec chain; eigengap uses the Lanczos algorithm (matmul + tridiagonal EVD). All operations support batching.
+- **Complexity**: Geršgorin $O(n^2)$; single power iteration $O(n^2)$; Lanczos $k$ steps $O(kn^2)$; full EVD $O(n^3)$ (should be avoided).
+- **Low precision**: The Weyl bound is itself Lipschitz; the estimation of $\|E\|_2$ under low precision has $\sim \sqrt{n} \cdot \text{eps}$ floating-point noise, which is typically negligible.
+
+## Risks and Failure Conditions
+
+- **Bauer-Fike amplification for non-normal matrices**: $\kappa(V)$ can be extremely large (high condition number of the eigenvector matrix for non-normal matrices), the Weyl bound no longer applies, and perturbations are amplified by a factor of $\kappa(V)$. Solution: use SVD singular values instead (the Mirsky bound does not depend on normality), or employ pseudospectral analysis.
+- **Overly loose Geršgorin bounds**: The union of discs may be much larger than the actual spectral range (especially for sparse matrices), yielding overly conservative bounds. Solution: apply a diagonal similarity transformation $D^{-1}AD$ to shrink the discs (Osborne balancing), or incorporate sparse structure corrections.
+- **Davis-Kahan gap assumption**: As $\delta \to 0$, the bound degenerates to $\infty$ (subspace becomes unidentifiable), and the low-rank approximation itself is no longer unique. The eigengap must be checked first to confirm that the subspace is well-defined.
+- **Weyl bound not directly applicable to non-Hermitian components**: The perturbation bound for $A + A^H$ does not directly yield eigenvalue perturbation bounds for $A$ itself. For non-symmetric matrices, one must revert to Bauer-Fike or pseudospectral analysis.
+
+## Further References
+
+- Distilled notes: references/books/matrix-analysis.md (Section 4.3 Eigenvalue Inequalities Weyl/Interlacing, Section 6.1-6.3 Geršgorin Discs and Perturbation Theorems, Section 5.8 Condition Numbers)
+- Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 4 Section 4.3 (Eigenvalue Inequalities) + Chapter 6 (Location and Perturbation of Eigenvalues Section 6.1-6.3)

package/knowledge-base/matrix-analysis/matrix-perturbation.md

@@ -0,0 +1,49 @@
+# 矩阵扰动 (Matrix Perturbation Theory)
+
+## 最小定义
+
+研究矩阵 $A$ 在受到小扰动 $E$ 后，其特征值、奇异值、特征子空间如何变化。核心结论：Hermitian 矩阵的特征值对扰动是 Lipschitz 连续的（Weyl 界），而特征向量/子空间的稳定性由特征值间隙（eigengap）决定。
+
+## 核心公式
+
+- **Weyl 特征值扰动界**（Hermitian）：$|\lambda_i(A+E) - \lambda_i(A)| \leq \|E\|_2$
+- **Bauer-Fike 定理**（可对角化矩阵）：$\min_j |\lambda_i(A+E) - \lambda_j(A)| \leq \kappa(V) \|E\|_2$，$V$ 为特征向量矩阵
+- **Davis-Kahan $\sin\Theta$ 定理**：$\|\sin\Theta(\hat{U}, U)\|_2 \leq \frac{\|E\|_2}{\delta}$，$\delta$ 为子空间与其余谱的间隙
+- **奇异值扰动 (Mirsky)**：$|\sigma_i(A+E) - \sigma_i(A)| \leq \|E\|_2$
+- **Ger\v{s}gorin 圆盘**：$\lambda_i(A) \in \bigcup_j \{z : |z - a_{jj}| \leq \sum_{k \neq j} |a_{jk}|\}$
+- **条件数与相对误差**：$\frac{|\delta x|}{|x|} \leq \kappa(A) \frac{\|\delta A\|}{\|A\|}$
+
+## 适用问题
+
+- 量化/低精度训练的谱漂移分析：bf16/fp8 下权重矩阵的奇异值偏移多少？
+- 模型剪枝/蒸馏的误差界：删掉 $k$ 个参数后谱变化有多大？
+- LoRA 近似误差：$\|W - W_0 - BA\|_2$ 的扰动如何影响下游输出？
+- 训练稳定性证书：梯度噪声 $\|E\|_2 \leq \epsilon$ 时特征值漂移不超过 $\epsilon$
+- 数值诊断：不跑完整 EVD，用 Ger\v{s}gorin 圆盘廉价估计谱位置
+
+## AI 设计翻译
+
+- **Ger\v{s}gorin 廉价谱半径监控**：训练循环中每 $N$ 步计算 $\rho_{\text{est}} = \max_j (|a_{jj}| + \sum_{k\neq j}|a_{jk}|)$ 作为谱半径上界，仅 $O(n^2)$ 逐行求绝对值和。实现为 `torch.sum(torch.abs(A), dim=1)`，是 elementwise + reduce，极廉价，可嵌入训练 loop 做稳定性 gate。
+- **量化误差的谱漂移界**：$W_{\text{quant}} = W + E$，$\|E\|_2 \leq \epsilon$，由 Weyl 定理 $\sigma_i$ 偏移 $\leq \epsilon$。对 $L$ 层网络，输出扰动 $\leq \prod_i (\sigma_1(W_i) + \epsilon) - \prod_i \sigma_1(W_i)$。指导量化精度选择：若 $\sigma_{\min}(W)$ 接近 $\epsilon$，则该层需更高精度。
+- **Davis-Kahan 子空间稳定性**：PCA/LoRA 中子空间的可靠性由 eigengap $\delta = \lambda_k - \lambda_{k+1}$ 决定。$\delta$ 越大，截断子空间越稳定；$\delta \to 0$ 时子空间对噪声极度敏感。可作为选择 LoRA rank $r$ 的诊断工具：选 $\delta_r$ 最大的 $r$。
+- **剪枝的扰动建模**：非结构化剪枝 = 稀疏扰动 $E$，$\|E\|_2 \leq \|E\|_F = \sqrt{\sum e_{ij}^2}$。Weyl 界给出谱漂移上界，指导剪枝比例：保持 $\|E\|_2 / \sigma_1(W) < \tau$（如 $\tau = 0.05$）。
+- **谱正则化作为鲁棒性证书**：$\mathcal{L}_{\text{robust}} = \mathcal{L}_{\text{task}} + \lambda \max(0, \|E\|_2 - \epsilon)^2$，约束扰动下的谱漂移。结合 power iteration 估计 $\|E\|_2$，实现为附加 loss 项。
+
+## 工程可行性
+
+- **主要操作**：Ger\v{s}gorin = elementwise abs + row-sum（$O(n^2)$）；Weyl 界只需 $\|E\|_2$（power iteration $O(n^2)$/step）；Davis-Kahan 需 eigengap（需部分 EVD，$O(n^2 k)$）。
+- **GPU 友好度**：高。Ger\v{s}gorin 是纯 elementwise + reduce；$\|E\|_2$ 估计是 matvec 链；eigengap 用 Lanczos 算法（matmul + tridiag EVD）。所有操作均可 batch。
+- **复杂度**：Ger\v{s}gorin $O(n^2)$；单次 power iteration $O(n^2)$；Lanczos $k$ 步 $O(kn^2)$；完整 EVD $O(n^3)$（应避免）。
+- **低精度**：Weyl 界本身是 Lipschitz 的，低精度下扰动 $\|E\|_2$ 的估计有 $\sim \sqrt{n} \cdot \text{eps}$ 的浮点噪声，通常可忽略。
+
+## 风险与失效条件
+
+- **非正规矩阵的 Bauer-Fike 放大**：$\kappa(V)$ 可能极大（非正规矩阵的特征向量矩阵条件数高），Weyl 界不再适用，扰动被放大 $\kappa(V)$ 倍。解决：改用 SVD 奇异值（Mirsky 界不依赖正规性），或用 pseudospectra 分析。
+- **Ger\v{s}gorin 过松**：圆盘并集可能远大于实际谱范围（尤其对稀疏矩阵），给出过于保守的界。解决：先做对角相似变换 $D^{-1}AD$ 压缩圆盘（Osborne 平衡），或结合稀疏结构修正。
+- **Davis-Kahan 的间隙假设**：$\delta \to 0$ 时界退化为 $\infty$（子空间不可辨识），此时低秩逼近本身不唯一。需先检测 eigengap，确认子空间良定义。
+- **Weyl 界对非 Hermitian 部分不直接适用**：$A + A^H$ 的扰动界不直接给出 $A$ 本身的特征值扰动。对非对称矩阵需回到 Bauer-Fike 或 pseudospectra。
+
+## 深入参考
+
+- 蒸馏稿：references/books/matrix-analysis.md（§4.3 特征值不等式 Weyl/交错、§6.1-6.3 Ger\v{s}gorin 圆盘与扰动定理、§5.8 条件数）
+- 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 4 §4.3 (Eigenvalue Inequalities) + Chapter 6 (Location and Perturbation of Eigenvalues §6.1-6.3)

package/knowledge-base/matrix-analysis/positive-semidefinite.en.md

@@ -0,0 +1,50 @@
+# Positive Semidefinite Matrices
+
+## Minimal Definition
+
+A Hermitian matrix $A$ is positive semidefinite (PSD) if $x^HAx \geq 0$ for all nonzero vectors $x$, denoted $A \succeq 0$. Equivalent conditions: all eigenvalues $\geq 0$; there exists $B$ such that $A = B^HB$ (Gram representation); all leading principal minors $\geq 0$. Positive definite (PD) requires strict $> 0$, denoted $A \succ 0$.
+
+## Core Formulas
+
+- PSD equivalence: $A \succeq 0 \iff \lambda_i(A) \geq 0 \ \forall i \iff A = B^HB$
+- PD equivalence: $A \succ 0 \iff \lambda_i(A) > 0 \ \forall i \iff$ all leading principal minors $> 0$ (Sylvester's criterion)
+- Cholesky decomposition: $A \succ 0 \implies A = LL^H$, $L$ lower triangular
+- Loewner partial order: $A \succeq B \iff A - B \succeq 0$
+- Schur product theorem: $A \succeq 0, B \succeq 0 \implies A \circ B \succeq 0$ (Hadamard product preserves PSD)
+- Simultaneous diagonalization: $A, B \succ 0 \implies \exists C$ such that $C^HAC = I, C^HBC = \Lambda$
+- Polar decomposition: $A = UP$, $P = (A^HA)^{1/2} \succeq 0$
+
+## Applicable Problems
+
+- Kernel methods: The Gram matrix $K_{ij} = k(x_i, x_j)$ must be PSD to guarantee the existence of an RKHS
+- Covariance matrices: $\Sigma = \mathbb{E}[xx^H] \succeq 0$; PCA/whitening relies on positive definiteness
+- Second-order optimization preconditioning: The PSD structure of the Hessian/Fisher information matrix guarantees descent directions
+- Semidefinite programming (SDP): optimizing a linear objective subject to $X \succeq 0$ constraints
+- Attention matrix analysis: Gram structure of the row-stochastic matrix output by softmax
+
+## AI Design Translation
+
+- **PSD kernel engineering (learnable kernels)**: Use the Schur product theorem to combine multiple PSD kernels: $K = K_1 \circ K_2 \circ \cdots$ (Hadamard product), guaranteeing the result remains PSD. Implemented as elementwise tensor multiplication `K = K1 * K2`, $O(n^2)$ elementwise, extremely GPU-friendly. Can parameterize $K_\theta(x,y) = \exp(-\|f_\theta(x)-f_\theta(y)\|^2)$ to guarantee PSD.
+- **Covariance whitening**: $\hat{x} = \Sigma^{-1/2}x$, where $\Sigma^{-1/2}$ is approximated via Newton-Schulz iteration (pure matmul). Newton-Schulz: $X_{k+1} = \frac{1}{2}X_k(3I - AX_k)$, two matmul operations per step, converging in 5-6 steps. BatchNorm can be viewed as an approximation to diagonal whitening.
+- **Cholesky preconditioner**: For a PSD Hessian $H$, use the $H = LL^H$ decomposition and solve $L^{-1}L^{-H}g$ instead of $H^{-1}g$. cuSOLVER provides batched Cholesky `potrf`. In K-FAC, the inversion of each Kronecker factor proceeds via Cholesky.
+- **Nearest PSD approximation (Higham)**: Given a symmetric matrix $A$, find the nearest PSD matrix $A_+ = \arg\min_{X \succeq 0} \|A - X\|_F$. Solution: EVD $A = U\Lambda U^H$, clamp negative values in $\Lambda$ to zero, $A_+ = U\Lambda_+ U^H$. Used to correct loss of positive definiteness in covariance matrices due to floating-point errors.
+- **Jitter / diagonal loading**: $A_{\text{stable}} = A + \epsilon I$ ($\epsilon \sim 10^{-6}$), ensuring numerical positive definiteness. Standard practice in Gaussian processes, kernel methods, and Cholesky decomposition. Implemented as `A + eps * torch.eye(n)`, a zero-cost operation.
+
+## Engineering Feasibility
+
+- **Primary operations**: Cholesky decomposition $O(n^3/3)$ (cuSOLVER batched available); Gram matrix construction $O(n^2d)$ (matmul); Newton-Schulz iteration $O(n^3)$/step (pure matmul); Hadamard product $O(n^2)$ (elementwise).
+- **GPU friendliness**: High. Gram matrix = matmul; Hadamard product = elementwise; Newton-Schulz = pure matmul chain; Cholesky has cuSOLVER batched versions for parallel execution across multiple groups.
+- **Complexity**: Gram construction $O(n^2d)$; Cholesky $O(n^3/3)$; Newton-Schulz 5 steps $O(5n^3)$; jitter $O(n)$.
+- **Low precision**: Cholesky may fail under bf16 (diagonal entries becoming negative); jitter must be added or the decomposition performed in fp32. Newton-Schulz is stable under bf16 (since it involves pure matmul without division).
+
+## Risks and Failure Conditions
+
+- **Loss of positive definiteness in floating point**: Covariance/Gram matrices may lose PSD property under bf16 ($\lambda_{\min} < 0$), causing Cholesky to fail outright. Solution: add jitter $\epsilon I$, compute the decomposition in fp32, or use Newton-Schulz (which does not involve square roots or division).
+- **Near-singularity**: As $\lambda_{\min} \to 0$, the condition number $\kappa \to \infty$, and the entries of $A^{-1}$ blow up in magnitude. Solution: truncate small eigenvalues (spectral cutoff) or apply ridge regularization $A + \lambda I$.
+- **Misuse of the Schur product theorem**: $A \circ B \succeq 0$ requires **both** $A$ and $B$ to be PSD; if either is not PSD, the result is not guaranteed. Each factor must be verified for PSD property in learnable kernel design.
+- **Non-differentiable SDP solvers**: Interior-point SDP solvers (e.g., SCS, MOSEK) cannot be integrated into the gradient graph and do not support end-to-end training. Solution: use a differentiable PSD projection layer (EVD + truncation + reconstruction) as a replacement.
+
+## Further References
+
+- Distilled notes: references/books/matrix-analysis.md (Ch 7 Positive Definite and Semidefinite Matrices, Section 7.5 Schur Product Theorem, Section 7.7 Loewner Partial Order)
+- Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 7 (Positive Definite and Semidefinite Matrices Section 7.1-7.8)

package/knowledge-base/matrix-analysis/positive-semidefinite.md

@@ -0,0 +1,50 @@
+# 半正定矩阵 (Positive Semidefinite Matrices)
+
+## 最小定义
+
+Hermitian 矩阵 $A$ 若对所有非零向量 $x$ 满足 $x^HAx \geq 0$，则称半正定（PSD），记 $A \succeq 0$。等价条件：所有特征值 $\geq 0$；存在 $B$ 使 $A = B^HB$（Gram 表示）；所有顺序主子式 $\geq 0$。正定（PD）要求严格 $> 0$，记 $A \succ 0$。
+
+## 核心公式
+
+- PSD 等价条件：$A \succeq 0 \iff \lambda_i(A) \geq 0 \ \forall i \iff A = B^HB$
+- 正定等价：$A \succ 0 \iff \lambda_i(A) > 0 \ \forall i \iff$ 所有顺序主子式 $> 0$（Sylvester 准则）
+- Cholesky 分解：$A \succ 0 \implies A = LL^H$，$L$ 下三角
+- Loewner 偏序：$A \succeq B \iff A - B \succeq 0$
+- Schur 积定理：$A \succeq 0, B \succeq 0 \implies A \circ B \succeq 0$（Hadamard 积保 PSD）
+- 同时对角化：$A, B \succ 0 \implies \exists C$ 使 $C^HAC = I, C^HBC = \Lambda$
+- 极分解：$A = UP$，$P = (A^HA)^{1/2} \succeq 0$
+
+## 适用问题
+
+- 核方法：Gram 矩阵 $K_{ij} = k(x_i, x_j)$ 必须 PSD 才能保证 RKHS 存在
+- 协方差矩阵：$\Sigma = \mathbb{E}[xx^H] \succeq 0$，PCA/白化依赖其正定性
+- 二阶优化预条件：Hessian/Fisher 信息矩阵的 PSD 结构保证下降方向
+- 半定规划 (SDP)：约束 $X \succeq 0$ 下的线性目标优化
+- Attention 矩阵分析：softmax 输出的行随机矩阵的 Gram 结构
+
+## AI 设计翻译
+
+- **PSD 核工程 (可学习核)**：用 Schur 积定理组合多个 PSD 核：$K = K_1 \circ K_2 \circ \cdots$（Hadamard 积），保证结果始终 PSD。实现为 elementwise 张量乘 `K = K1 * K2`，$O(n^2)$ elementwise，极 GPU 友好。可参数化 $K_\theta(x,y) = \exp(-\|f_\theta(x)-f_\theta(y)\|^2)$ 保证 PSD。
+- **协方差白化 (Whitening)**：$\hat{x} = \Sigma^{-1/2}x$，其中 $\Sigma^{-1/2}$ 通过 Newton-Schulz 迭代近似（纯 matmul）。Newton-Schulz：$X_{k+1} = \frac{1}{2}X_k(3I - AX_k)$，每步两次 matmul，5-6 步收敛。BatchNorm 可视为对角白化的近似。
+- **Cholesky 预条件子**：对 PSD Hessian $H$，用 $H = LL^H$ 分解后解 $L^{-1}L^{-H}g$ 代替 $H^{-1}g$。cuSOLVER 有 batched Cholesky `potrf`。K-FAC 中每个 Kronecker 因子的求逆即走 Cholesky。
+- **最近 PSD 近似 (Higham)**：给定对称矩阵 $A$，求最近 PSD 矩阵 $A_+ = \arg\min_{X \succeq 0} \|A - X\|_F$。解法：EVD $A = U\Lambda U^H$，将 $\Lambda$ 中负值截零，$A_+ = U\Lambda_+ U^H$。用于修正浮点误差导致的协方差矩阵失去正定性。
+- **Jitter / 对角加载**：$A_{\text{stable}} = A + \epsilon I$（$\epsilon \sim 10^{-6}$），保证数值正定性。高斯过程、核方法、Cholesky 分解中标准做法。实现为 `A + eps * torch.eye(n)`，零成本操作。
+
+## 工程可行性
+
+- **主要操作**：Cholesky 分解 $O(n^3/3)$（有 cuSOLVER batched）；Gram 矩阵构造 $O(n^2d)$（matmul）；Newton-Schulz 迭代 $O(n^3)$/step（纯 matmul）；Hadamard 积 $O(n^2)$（elementwise）。
+- **GPU 友好度**：高。Gram 矩阵 = matmul；Hadamard 积 = elementwise；Newton-Schulz = 纯 matmul 链；Cholesky 有 cuSOLVER batched 版本，可并行多组。
+- **复杂度**：构造 Gram $O(n^2d)$；Cholesky $O(n^3/3)$；Newton-Schulz 5 步 $O(5n^3)$；Jitter $O(n)$。
+- **低精度**：bf16 下 Cholesky 可能失败（对角元变负），必须加 jitter 或用 fp32 做分解。Newton-Schulz 在 bf16 下稳定（因为纯 matmul 不涉除法）。
+
+## 风险与失效条件
+
+- **浮点丢正定性**：协方差/Gram 矩阵在 bf16 下可能失去 PSD 性（$\lambda_{\min} < 0$），Cholesky 直接崩溃。解决：加 jitter $\epsilon I$，或用 fp32 计算分解，或用 Newton-Schulz（不涉及开方/除法）。
+- **近奇异性**：$\lambda_{\min} \to 0$ 时条件数 $\kappa \to \infty$，$A^{-1}$ 的元素量级爆炸。解决：截断小特征值（spectral cutoff）或用岭正则 $A + \lambda I$。
+- **Schur 积定理的滥用**：$A \circ B \succeq 0$ 要求 $A, B$ **都** PSD；若其中一个非 PSD，结果不保证。可学习核设计中需逐因子验证 PSD 性。
+- **SDP 求解器不可微**：半定规划的内点法求解器（如 SCS, MOSEK）不可嵌入梯度图，不能直接做端到端训练。解决：用可微的 PSD 投影层（EVD + 截断 + 重构）替代。
+
+## 深入参考
+
+- 蒸馏稿：references/books/matrix-analysis.md（Ch 7 Positive Definite and Semidefinite Matrices、§7.5 Schur 积定理、§7.7 Loewner 偏序）
+- 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 7 (Positive Definite and Semidefinite Matrices §7.1-7.8)