math-skill 3.0.0 → 3.1.0

package/design-patterns/compression/topology-preserving-compression.en.md

@@ -1,12 +1,12 @@
 # Topology-Preserving Compression
-> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
 
 ## Target Problem
-Use when compressing representations while homologically preserving the intrinsic topological structure of the data (connected components, loops, cavities): latent-space compression (a toroidal manifold must not collapse into a line segment), knowledge distillation (student--teacher homological equivalence), 3D mesh simplification (genus invariance), KV-Cache semantic preservation (cluster structure must not collapse). Core objective: **compress dimensions or counts while ensuring that changes in the persistence diagram of persistent homology remain controlled**.
+Use when compressing representations while homologically preserving the intrinsic topological structure of the data (connected components, loops, cavities): latent-space compression (a toroidal manifold must not collapse into a line segment), knowledge distillation (student--teacher homological equivalence), 3D mesh simplification (genus invariance), KV-Cache semantic preservation (cluster structure must not collapse). Core objective: **compress dimensions or counts while constraining and measuring changes in the persistence diagram of persistent homology**.
 
 ## Mathematical Foundations
-- Lenses: lenses/topological.md (topological invariants -- connected components and hole counts are invariant under continuous deformation), lenses/spectral.md (Gauss--Bonnet linking curvature and Euler characteristic), lenses/variational.md (constrained variational of compression ratio vs. topological fidelity)
-- Knowledge: knowledge-base/topology/persistent-homology.md (persistent homology, Vietoris--Rips filtration, bottleneck distance), knowledge-base/topology/euler-characteristic.md (Euler characteristic for rapid topological diagnostics), knowledge-base/matrix-analysis/matrix-perturbation.md (Davis--Kahan subspace perturbation bound)
+- Lenses: ../../lenses/topological.en.md (topological invariants -- connected components and hole counts are invariant under continuous deformation), ../../lenses/spectral.en.md (Gauss--Bonnet linking curvature and Euler characteristic), ../../lenses/variational.en.md (constrained variational of compression ratio vs. topological fidelity)
+- Knowledge: ../../knowledge-base/topology/persistent-homology.en.md (persistent homology, Vietoris--Rips filtration, bottleneck distance), ../../knowledge-base/topology/euler-characteristic.en.md (Euler characteristic for rapid topological diagnostics), ../../knowledge-base/matrix-analysis/matrix-perturbation.en.md (Davis--Kahan subspace perturbation bound)
 
 ## Required Mathematical Background
 - **Stability Theorem of Persistent Homology**: $d_B(D(X), D(Y)) \leq d_{GH}(X, Y)$; the Hausdorff distance bounds the change in persistence diagrams
@@ -46,15 +46,15 @@ Method 3 - Topology monitoring + adaptive compression ratio (at inference time):
 - **Topological diagnostics dashboard**: real-time plotting of $\beta_0(\epsilon), \beta_1(\epsilon)$ curves during training
 
 ## GPU Feasibility
-- Tensorization / GEMM: the distance matrix `cdist` is GEMM-dominated; $\chi(\epsilon)$ involves threshold counting + cumulative sum
-- Complexity: full persistent homology $O(N^3)$ is infeasible; Euler curve $O(N^2 |\epsilon|)$ is feasible; Landmark $O(m^3)$
-- Memory: the $N \times N$ distance matrix requires chunking or landmark reduction for $N > 8K$
-- Low precision: distance computation is stable in bf16 (positive-number addition); $\chi$ involves integer arithmetic with no precision concerns
-- Parallelism: each threshold in the $\epsilon$ sweep is independently parallel; landmark selection can be batch-parallelized
-- Operator fusion: cdist + threshold + count can be fused to avoid materializing the large distance matrix
+- **D1[v]/D2[v]**: the distance matrix `cdist` is GEMM-dominated; $\chi(\epsilon)$ involves threshold counting + cumulative sum
+- **D3[~]**: full persistent homology $O(N^3)$ is infeasible; Euler curve $O(N^2 |\epsilon|)$ is feasible; Landmark $O(m^3)$
+- **D4[~]**: the $N \times N$ distance matrix requires chunking or landmark reduction for $N > 8K$
+- **D5[v]**: distance computation is stable in bf16 (positive-number addition); $\chi$ involves integer arithmetic with no precision concerns
+- **D6[v]**: each threshold in the $\epsilon$ sweep is independently parallel; landmark selection can be batch-parallelized
+- **D8[v]**: cdist + threshold + count can be fused to avoid materializing the large distance matrix
 
 ## Paper-Worthy Formulation
-"Grounded in the Bottleneck stability theorem of persistent homology, we achieve $O(N^2)$-complexity topology-preserving regularization via Euler characteristic curve matching, ensuring that post-compression deviations in Betti numbers over persistence intervals are controlled by the Hausdorff distance."
+"Grounded in the Bottleneck stability theorem of persistent homology, we build a computable topology-preserving regularizer via Euler-characteristic curves or landmark approximations. Bottleneck distance can bound persistence-diagram changes when the filtration-function perturbation is controlled; Euler curves are incomplete proxies and do not by themselves guarantee per-degree Betti-number deviations, so persistence / Betti-curve deviations should be measured."
 
 ## Risks
 - **Persistent homology computational bottleneck**: exact boundary matrix reduction is highly serial ($O(N^3)$), necessitating reliance on Euler curve or landmark approximations

package/design-patterns/compression/topology-preserving-compression.md

@@ -1,12 +1,12 @@
 # Topology-Preserving Compression（拓扑保持压缩）
-> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行，需工程验证。
 
 ## 适用问题
-当压缩表示时需要同构保持数据的本质拓扑结构（连通性、环、空腔）时使用：隐空间压缩（环形流形不能压成线段）、知识蒸馏（学生-教师同调等价）、3D 网格简化（genus 不变）、KV-Cache 语义保持（聚类结构不坍缩）。核心诉求：**压缩维度或数量，保证持续同调的 persistence diagram 变化可控**。
+当压缩表示时需要同构保持数据的本质拓扑结构（连通性、环、空腔）时使用：隐空间压缩（环形流形不能压成线段）、知识蒸馏（学生-教师同调等价）、3D 网格简化（genus 不变）、KV-Cache 语义保持（聚类结构不坍缩）。核心诉求：**压缩维度或数量，同时约束并实测持续同调的 persistence diagram 变化**。
 
 ## 数学思想来源
-- 透镜：lenses/topological.md（拓扑不变量——连通性、洞数在连续变形下不变）、lenses/spectral.md（Gauss-Bonnet 连接曲率与欧拉示性数）、lenses/variational.md（压缩率 vs. 拓扑保真度的约束优化）
-- 知识：knowledge-base/topology/persistent-homology.md（持续同调、Vietoris-Rips 滤流、Bottleneck 距离）、knowledge-base/topology/euler-characteristic.md（欧拉示性数快速拓扑诊断）、knowledge-base/matrix-analysis/matrix-perturbation.md（Davis-Kahan 子空间扰动界）
+- 透镜：../../lenses/topological.md（拓扑不变量——连通性、洞数在连续变形下不变）、../../lenses/spectral.md（Gauss-Bonnet 连接曲率与欧拉示性数）、../../lenses/variational.md（压缩率 vs. 拓扑保真度的约束优化）
+- 知识：../../knowledge-base/topology/persistent-homology.md（持续同调、Vietoris-Rips 滤流、Bottleneck 距离）、../../knowledge-base/topology/euler-characteristic.md（欧拉示性数快速拓扑诊断）、../../knowledge-base/matrix-analysis/matrix-perturbation.md（Davis-Kahan 子空间扰动界）
 
 ## 需要的数学知识
 - **持续同调稳定性定理**：$d_B(D(X), D(Y)) \leq d_{GH}(X, Y)$，Hausdorff 距离界定 persistence diagram 变化
@@ -46,15 +46,15 @@
 - **拓扑诊断仪表板**：训练中实时绘制 $\beta_0(\epsilon), \beta_1(\epsilon)$ 曲线
 
 ## GPU 可行性
-- 张量化/GEMM：距离矩阵 `cdist` 核心为 GEMM；$\chi(\epsilon)$ 为阈值计数 + 累积和
-- 复杂度：完整持续同调 $O(N^3)$ 不可行；Euler curve $O(N^2 |\epsilon|)$ 可行；Landmark $O(m^3)$
-- 显存：$N \times N$ 距离矩阵在 $N > 8K$ 时需分块或 Landmark 降维
-- 低精度：距离计算在 bf16 下稳定（正数加法）；$\chi$ 为整数运算无精度问题
-- 并行：$\epsilon$ 扫描的每个阈值独立并行；Landmark 选择可批并行
-- 算子融合：cdist + threshold + count 可融合避免物化大距离矩阵
+- **D1[v]/D2[v]**：距离矩阵 `cdist` 核心为 GEMM；$\chi(\epsilon)$ 为阈值计数 + 累积和
+- **D3[~]**：完整持续同调 $O(N^3)$ 不可行；Euler curve $O(N^2 |\epsilon|)$ 可行；Landmark $O(m^3)$
+- **D4[~]**：$N \times N$ 距离矩阵在 $N > 8K$ 时需分块或 Landmark 降维
+- **D5[v]**：距离计算在 bf16 下稳定（正数加法）；$\chi$ 为整数运算无精度问题
+- **D6[v]**：$\epsilon$ 扫描的每个阈值独立并行；Landmark 选择可批并行
+- **D8[v]**：cdist + threshold + count 可融合避免物化大距离矩阵
 
 ## 论文表述方式
-"以持续同调的 Bottleneck 稳定性定理为理论基础，通过 Euler characteristic curve 匹配实现 $O(N^2)$ 复杂度的拓扑保真正则化，保证压缩后 Betti 数在持续区间上的偏差受 Hausdorff 距离控制。"
+"以持续同调的 Bottleneck 稳定性定理为理论基础，通过 Euler characteristic curve 或 landmark 近似构造可计算的拓扑正则。Bottleneck 距离可在滤流函数扰动受控时界定 persistence diagram 的变化；Euler curve 只是不完整代理，不能单独保证每阶 Betti 数偏差受控，需报告 persistence/Betti 曲线的实测偏差。"
 
 ## 风险
 - **持续同调计算瓶颈**：精确边界矩阵约化高度串行（$O(N^3)$），必须依赖 Euler curve 或 Landmark 近似

package/design-patterns/loss/constraint-penalty.en.md

@@ -1,12 +1,12 @@
 # Constraint Penalty
-> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
 
 ## Applicable Problems
 When the design involves hard constraints (e.g., probability simplex, orthogonality, capacity limits, load balancing) but end-to-end training is required. Typical scenarios: (1) MoE routing probabilities must lie on the $K$-simplex with load balancing; (2) Expert activation count is constrained (top-k); (3) Subspace projection matrices must satisfy orthogonality $W^T W = I$; (4) Feature norms are bounded $\|z\| \leq R$. Core objective: **transform mathematical constraints into differentiable penalty terms integrated into gradient-based variational**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/variational.md (constrained variational, Lagrangian duality, KKT conditions), lenses/geometric.md (manifold projection)
-- Knowledge: knowledge-base/optimization/lagrangian-duality.md (augmented Lagrangian method, penalty function method), knowledge-base/matrix-analysis/projection.md (projection operators, constraint sets)
+- Lenses: ../../lenses/variational.en.md (constrained variational, Lagrangian duality, KKT conditions), ../../lenses/geometric.en.md (manifold projection)
+- Knowledge: ../../knowledge-base/optimization/lagrangian-duality.en.md (augmented Lagrangian method, penalty function method), ../../knowledge-base/matrix-analysis/projection.en.md (projection operators, constraint sets)
 
 ## Required Mathematical Knowledge
 - **Penalty Function Method**: $\min f(x)$ s.t. $g(x)=0 \to \min f(x) + \rho/2 \cdot \|g(x)\|^2$ -- $\rho$ is gradually increased (exterior point method), driving constraint violation $\|g(x)\| \to 0$
@@ -29,6 +29,10 @@ Method 2 - Augmented Lagrangian Method (ALM):
   // lambda is a learnable parameter (nn.Parameter), updated via gradient ascent:
   lambda.data += rho * g(x).detach()   // dual ascent step
   // Converges faster than pure penalty, avoids rho -> infinity
+  For inequality constraints h(x) <= 0:
+  L_ALM = Sum_j 1/(2*rho) * [max(0, lambda_j + rho*h_j(x))^2 - lambda_j^2]
+  // lambda update: lambda_j <- max(0, lambda_j + rho*h_j(x))
+  // Strictly feasible constraints (lambda_j + rho*h_j(x) < 0) are not penalized
 
 Method 3 - Softmax Projection onto Simplex (load balancing special case):
   p = softmax(logits / tau)           // project onto Delta^{K-1}
@@ -43,7 +47,7 @@ Method 4 - Orthogonal Constraint Projection:
 
 ## Implementable Architectures
 - **Loss Wrapper**: ConstraintLoss(base_loss, constraints, rho_schedule) -- forward computes base_loss + Sum constraint.penalty()
-- **Dual Variable Management**: nn.Parameter stores lambda; negative learning rate set in the optimizer to achieve gradient ascent
+- **Dual Variable Management**: equality multipliers can use nn.Parameter with a negative learning rate for gradient ascent; inequality multipliers must satisfy $\lambda \geq 0$, so use explicit projected updates `lambda <- max(0, lambda + rho*h(x))` or clamp after optimizer steps
 - **Warm-up Strategy**: Optimize only base_loss for the first $N$ steps, then progressively activate constraint penalties
 - **Constraint Monitoring**: Record $\|g(x)\|$ at each step for visualization and adaptive rho adjustment
 
@@ -58,7 +62,7 @@ Method 4 - Orthogonal Constraint Projection:
 - **Operator Fusion**: Constraint computation + weighted summation + merging with base_loss can be fused into a single kernel
 
 ## Paper Phrasing
-"We employ the augmented Lagrangian method to convert hard constraints $g(x)=0$ into differentiable penalty terms $\lambda^T g(x) + \rho/2 \|g(x)\|^2$. Through alternating dual ascent updates of $\lambda$, constraint satisfaction is guaranteed to converge at rate $O(1/\rho)$ without requiring $\rho \to \infty$, reducing constraint violations by 3x compared to pure penalty methods in experiments."
+"We employ the augmented Lagrangian method to convert hard constraints $g(x)=0$ into differentiable penalty terms $\lambda^T g(x) + \rho/2 \|g(x)\|^2$, using alternating dual ascent updates of $\lambda$ rather than relying only on $\rho \to \infty$. Under convexity, constraint qualifications such as LICQ/MFCQ, and sufficiently accurate inner solves, ALM can converge to KKT solutions; in non-convex training, report the measured constraint-violation curve instead of claiming a universal $O(1/\rho)$ rate."
 
 ## Risks
 - Excessively fast rho growth leads to ill-conditioned variational landscapes (deteriorating condition numbers), causing gradient vanishing or explosion

package/design-patterns/loss/constraint-penalty.md

@@ -1,5 +1,5 @@
 # Constraint Penalty（约束惩罚损失）
-> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行，需工程验证。
 
 ## 适用问题
 当设计中有硬约束（如概率单纯形、正交性、容量限制、负载均衡）但需要端到端训练时使用。
@@ -8,9 +8,9 @@
 核心诉求：**将数学约束转化为可微惩罚项，融入梯度优化流程**。
 
 ## 数学思想来源
-- 透镜：lenses/variational.md（约束优化、拉格朗日对偶、KKT 条件）、lenses/geometric.md（流形投影）
-- 知识：knowledge-base/optimization/lagrangian-duality.md（增广拉格朗日法、惩罚函数法）、
-  knowledge-base/matrix-analysis/projection.md（投影算子、约束集）
+- 透镜：../../lenses/variational.md（约束优化、拉格朗日对偶、KKT 条件）、../../lenses/geometric.md（流形投影）
+- 知识：../../knowledge-base/optimization/lagrangian-duality.md（增广拉格朗日法、惩罚函数法）、
+  ../../knowledge-base/matrix-analysis/projection.md（投影算子、约束集）
 
 ## 需要的数学知识
 - **罚函数法**：min f(x) s.t. g(x)=0 → min f(x) + ρ/2 · ‖g(x)‖²
@@ -37,6 +37,10 @@
   // λ 为可学习参数（nn.Parameter），通过梯度上升更新：
   λ.data += ρ * g(x).detach()   // 对偶上升步
   // 比纯罚函数收敛快，避免 ρ → ∞
+  对不等式约束 h(x) ≤ 0：
+  L_ALM = Σ_j 1/(2ρ) · [max(0, λ_j + ρ·h_j(x))² - λ_j²]
+  // λ 更新：λ_j ← max(0, λ_j + ρ·h_j(x))
+  // 严格可行（λ_j + ρ·h_j(x) < 0）的约束不被惩罚
 
 方法3 - Softmax 投影到单纯形（负载均衡特例）：
   p = softmax(logits / τ)           // 投影到 Δ^{K-1}
@@ -52,7 +56,7 @@
 ## 可实现结构
 - **Loss Wrapper**：ConstraintLoss(base_loss, constraints, ρ_schedule)
   forward 时计算 base_loss + Σ constraint.penalty()
-- **对偶变量管理**：nn.Parameter 存储 λ，optimizer 中为 λ 设置负学习率实现梯度上升
+- **对偶变量管理**：等式乘子 λ 可用 nn.Parameter + 负学习率做梯度上升；不等式乘子必须保持 λ ≥ 0，建议用显式投影更新 `λ ← max(0, λ + ρ·h(x))` 或在 optimizer step 后 clamp
 - **warm-up 策略**：前 N 步只优化 base_loss，之后逐步激活约束惩罚
 - **约束监控**：每 step 记录 ‖g(x)‖ 用于可视化和自适应 ρ 调整
 
@@ -68,8 +72,7 @@
 
 ## 论文表述方式
 "采用增广拉格朗日法将硬约束 g(x)=0 转化为可微惩罚项 λ^Tg(x) + ρ/2‖g(x)‖²，
-通过对偶变量 λ 的交替上升更新，在无需 ρ→∞ 的条件下保证约束满足度以 O(1/ρ) 收敛，
-相比纯罚函数法在实验中减少 3× 的约束违反。"
+并通过对偶变量 λ 的交替上升更新避免单纯依赖 ρ→∞。在凸性、约束规格（如 LICQ/MFCQ）和内层子问题求解精度满足条件时，ALM 可收敛到 KKT 解；非凸训练中应报告实际约束违反曲线，而不能宣称通用 O(1/ρ) 速率。"
 
 ## 风险
 - ρ 增大过快导致优化 landscape 病态（条件数恶化），梯度消失或爆炸

package/design-patterns/loss/contrastive-loss.en.md

@@ -1,12 +1,12 @@
 # Contrastive Loss
-> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
 
 ## Applicable Problems
 When the model needs to learn "what is similar to what and what is different from what." Typical scenarios: (1) Different augmented views of the same input should be pulled closer (positive pairs), while different inputs should be pushed apart (negative pairs); (2) Shared representations should capture cross-task commonalities, while Private representations should distinguish task-specific features; (3) In the expert embedding space, similar inputs should be routed to the same expert. Core objective: **learn relative relationships rather than absolute values**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/geometric.md (metric spaces and distance functions), lenses/probabilistic.md (mutual information maximization)
-- Knowledge: knowledge-base/probability/entropy.md (conditional distributions and likelihood), knowledge-base/differential-geometry/manifold.md (geodesics and curvature)
+- Lenses: ../../lenses/geometric.en.md (metric spaces and distance functions), ../../lenses/probabilistic.en.md (mutual information maximization)
+- Knowledge: ../../knowledge-base/probability/entropy.en.md (conditional distributions and likelihood), ../../knowledge-base/differential-geometry/manifold.en.md (geodesics and curvature)
 
 ## Required Mathematical Knowledge
 - **InfoNCE Loss**: $L = -\log[\exp(\text{sim}(q,k^+)/\tau) / \sum_j \exp(\text{sim}(q,k_j)/\tau)]$ -- essentially a lower-bound estimate of mutual information in contrastive learning; $\tau$ is the temperature parameter controlling distribution sharpness
@@ -14,6 +14,32 @@ When the model needs to learn "what is similar to what and what is different fro
 - **NT-Xent (Normalized Temperature-scaled Cross Entropy)**: Softmax contrastive loss on the unit sphere $S^{d-1}$, with normalization eliminating scale effects
 - **Debiased Contrastive Learning**: Corrects for false negatives in negative samples, using prior $\tau^+$ to estimate the true negative sample distribution
 
+## Alignment-Uniformity Framework
+
+Representation quality in contrastive learning decomposes into two independent objectives (Wang & Isola, 2020):
+
+- **Alignment**: Positive pairs should have similar representations
+  $L_{\text{align}} = \mathbb{E}_{(x, x^+)}[\|f(x) - f(x^+)\|^2]$
+- **Uniformity**: Representations should be uniformly distributed on the unit hypersphere $S^{d-1}$
+  $L_{\text{uniform}} = \log \mathbb{E}_{x, x'}[\exp(-2\|f(x) - f(x')\|^2)]$
+
+**InfoNCE and alignment-uniformity**: As $N \to \infty$ with appropriate temperature $\tau$, the InfoNCE loss asymptotically decomposes into alignment + uniformity. For finite $N$, InfoNCE provides a lower bound on $I(X;Z)$, with tightness increasing in $N$.
+
+**Conditions for uniformity to hold**:
+- Sufficiently large negative sample count $N$ (theory requires $N \to \infty$; in practice $N \geq 1024$ is typically sufficient)
+- Temperature $\tau$ not too large ($\tau \to \infty$ causes loss to degenerate to a constant, eliminating the uniformity-driving force)
+- Representation dimension $d$ sufficient to support the intrinsic dimension of the data
+
+**Conditions for uniformity to fail**:
+- Insufficient negatives $\rightarrow$ weak uniformity pressure; representations may cluster on a local region of the sphere
+- Representation collapse: all inputs map to the same (or few) points, trivially minimizing alignment but completely destroying uniformity
+- Temperature $\tau$ too large $\rightarrow$ softmax degenerates to uniform distribution, gradients vanish, no uniformity guarantee
+- Severe positive/negative imbalance within the batch without queue compensation
+
+**What can be guaranteed at most**: Under ideal conditions (sufficiently large $N$, appropriate $\tau$, no collapse), minimizing contrastive loss is equivalent to jointly maximizing positive-pair alignment and representation uniformity.
+
+**What cannot be guaranteed**: Optimality of learned representations for downstream tasks (uniformity $\neq$ task relevance); semantic-level alignment (only geometric-level positive-pair proximity is guaranteed).
+
 ## AI Module Form
 ```
 Module: ContrastiveLoss
@@ -40,17 +66,17 @@ Hard Negative Mining:
 - **Multi-Granularity Contrast**: Apply contrastive objectives simultaneously at token-level, sequence-level, and expert-level
 
 ## GPU Feasibility
-- **Tensorization**: Similarity computation is $z_a @ z_n^T$ -- standard GEMM $(B \times d) @ (d \times N) = B \times N$
-- **GEMM-mappability**: Core computation is 1-2 matrix multiplications, perfectly mapped to cuBLAS
-- **Complexity**: $O(B \cdot N \cdot d)$ computation + $O(B \cdot N)$ storage for the logits matrix; approximately 64MB when B=256, N=65536
-- **Memory & KV-Cache**: Negative sample queue occupies $N \cdot d \cdot 4$ bytes, approximately 65536 * 256 * 4 = 64MB, fixed overhead
-- **Low Precision Stability**: Cosine similarity + softmax under fp16 requires caution for exp overflow; use log-sum-exp trick
-- **Parallelism & Communication**: On multiple GPUs, use all-gather to collect negatives from other GPUs to enlarge $N$ (MoCo v3 strategy)
-- **Sparse Structure**: After hard negative mining, only $k \ll N$ negatives are retained, effectively sparsifying the logits
-- **Operator Fusion**: L2-norm -> matmul -> scale -> log-softmax -> nll_loss can be fused
+- **D1[v]**: Similarity computation is $z_a @ z_n^T$ -- standard GEMM $(B \times d) @ (d \times N) = B \times N$
+- **D2[v]**: Core computation is 1-2 matrix multiplications, perfectly mapped to cuBLAS
+- **D3[v]**: $O(B \cdot N \cdot d)$ computation + $O(B \cdot N)$ storage for the logits matrix; approximately 64MB when B=256, N=65536
+- **D4[v]**: Negative sample queue occupies $N \cdot d \cdot 4$ bytes, approximately 65536 * 256 * 4 = 64MB, fixed overhead
+- **D5[v]**: Cosine similarity + softmax under fp16 requires caution for exp overflow; use log-sum-exp trick
+- **D6[v]**: On multiple GPUs, use all-gather to collect negatives from other GPUs to enlarge $N$ (MoCo v3 strategy)
+- **D7[v]**: After hard negative mining, only $k \ll N$ negatives are retained, effectively sparsifying the logits
+- **D8[v]**: L2-norm -> matmul -> scale -> log-softmax -> nll_loss can be fused
 
 ## Paper Phrasing
-"We employ temperature-scaled InfoNCE contrastive loss with a momentum encoder maintaining a negative sample queue of N=65536, maximizing the lower bound of positive-pair mutual information on the unit sphere. Theoretical analysis shows this lower bound converges at rate $O(1/\sqrt{N})$."
+"We employ temperature-scaled InfoNCE contrastive loss with a momentum encoder maintaining a negative-sample queue, optimizing proxies for positive-pair alignment and representation uniformity on the unit sphere. Mutual-information lower bounds and sampling-error rates depend on the negative-sample distribution, independence assumptions, and queue staleness; report ablations over queue size, temperature, negative-mining strategy, and downstream metrics."
 
 ## Risks
 - Too small $\tau$ causes training instability (excessively large gradients); too large $\tau$ makes all samples indistinguishable (degenerates to uniform distribution)

package/design-patterns/loss/contrastive-loss.md

@@ -1,5 +1,5 @@
 # Contrastive Loss（对比损失）
-> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行，需工程验证。
 
 ## 适用问题
 当需要让模型学会"什么和什么相似、什么和什么不同"时使用。典型场景：
@@ -8,9 +8,9 @@
 (3) 专家嵌入空间中相似输入应路由到同一专家。核心诉求：**学习相对关系而非绝对值**。
 
 ## 数学思想来源
-- 透镜：lenses/geometric.md（度量空间与距离函数）、lenses/probabilistic.md（互信息最大化）
-- 知识：knowledge-base/probability/entropy.md（条件分布与似然）、
-  knowledge-base/differential-geometry/manifold.md（测地线与曲率）
+- 透镜：../../lenses/geometric.md（度量空间与距离函数）、../../lenses/probabilistic.md（互信息最大化）
+- 知识：../../knowledge-base/probability/entropy.md（条件分布与似然）、
+  ../../knowledge-base/differential-geometry/manifold.md（测地线与曲率）
 
 ## 需要的数学知识
 - **InfoNCE 损失**：L = -log[exp(sim(q,k⁺)/τ) / Σ_j exp(sim(q,k_j)/τ)]
@@ -22,6 +22,32 @@
 - **去偏对比学习 (Debiased Contrastive)**：
   修正负样本中的假阴性问题，使用先验 τ⁺ 估计真实负样本分布
 
+## 对齐-均匀性理论 (Alignment-Uniformity Framework)
+
+对比学习的表示质量可分解为两个独立目标（Wang & Isola, 2020）：
+
+- **对齐 (Alignment)**：正对的表示应接近
+  $L_{\text{align}} = \mathbb{E}_{(x, x^+)}[\|f(x) - f(x^+)\|^2]$
+- **均匀性 (Uniformity)**：表示应在单位超球面 $S^{d-1}$ 上均匀分布
+  $L_{\text{uniform}} = \log \mathbb{E}_{x, x'}[\exp(-2\|f(x) - f(x')\|^2)]$
+
+**InfoNCE 与对齐-均匀性的关系**：当 $N \to \infty$ 且温度 $\tau$ 适当时，InfoNCE 损失渐近分解为 alignment + uniformity 之和。有限 $N$ 下，InfoNCE 给出 $I(X;Z)$ 的下界，下界紧度随 $N$ 增大。
+
+**均匀性成立的条件**：
+- 负样本数 $N$ 充分大（理论要求 $N \to \infty$，实践中 $N \geq 1024$ 通常足够）
+- 温度 $\tau$ 不过大（$\tau \to \infty$ 时 loss 退化为常数，丧失均匀性驱动力）
+- 表示维度 $d$ 足够支撑数据的本征维度
+
+**均匀性不成立的条件**：
+- 负样本不足 → 均匀性驱动力弱，表示可能聚集在球面局部
+- 表示坍塌 (representation collapse)：所有输入映射到同一/少数点，trivially 最小化 alignment 但完全丧失 uniformity
+- 温度 $\tau$ 过大 → softmax 趋于均匀分布，梯度消失，无均匀性保证
+- batch 内正负对比例严重失衡且未通过队列补偿
+
+**最多能保证什么**：在理想条件下（$N$ 充分大、$\tau$ 合适、无坍塌），对比损失最小化等价于同时最大化正对对齐度和表示均匀性。
+
+**不能保证什么**：不能保证学到的表示对下游任务最优（均匀性 ≠ 任务相关性）；不能保证语义层级的对齐（仅保证几何层面的正对接近）。
+
 ## AI 模块形式
 ```
 模块：ContrastiveLoss
@@ -48,18 +74,18 @@ Hard Negative Mining:
 - **多粒度对比**：同时在 token-level、sequence-level、expert-level 施加对比
 
 ## GPU 可行性
-- **张量化**：sim 计算为 z_a @ z_n^T → 标准 GEMM (B×d) @ (d×N) = B×N
-- **GEMM 可映射**：核心就是 1-2 次矩阵乘法，完美映射 cuBLAS
-- **复杂度**：O(B·N·d) 计算 + O(B·N) 存储 logits 矩阵，B=256,N=65536 时约 64MB
-- **显存与 KV-Cache**：负样本队列占 N·d·4 bytes ≈ 65536·256·4 = 64MB，固定开销
-- **低精度稳定**：cosine 相似度 + softmax 在 fp16 下需注意 exp 溢出，用 log-sum-exp trick
-- **并行与通信**：多 GPU 时用 all-gather 收集其他 GPU 的负样本扩大 N（MoCo v3 策略）
-- **稀疏结构**：hard negative mining 后只保留 k<<N 个负样本，有效稀疏化 logits
-- **算子融合**：L2-norm → matmul → scale → log-softmax → nll_loss 可融合
+- **D1[v]**：sim 计算为 z_a @ z_n^T → 标准 GEMM (B×d) @ (d×N) = B×N
+- **D2[v]**：核心就是 1-2 次矩阵乘法，完美映射 cuBLAS
+- **D3[v]**：O(B·N·d) 计算 + O(B·N) 存储 logits 矩阵，B=256,N=65536 时约 64MB
+- **D4[v]**：负样本队列占 N·d·4 bytes ≈ 65536·256·4 = 64MB，固定开销
+- **D5[v]**：cosine 相似度 + softmax 在 fp16 下需注意 exp 溢出，用 log-sum-exp trick
+- **D6[v]**：多 GPU 时用 all-gather 收集其他 GPU 的负样本扩大 N（MoCo v3 策略）
+- **D7[v]**：hard negative mining 后只保留 k<<N 个负样本，有效稀疏化 logits
+- **D8[v]**：L2-norm → matmul → scale → log-softmax → nll_loss 可融合
 
 ## 论文表述方式
 "采用温度缩放的 InfoNCE 对比损失，通过动量编码器维护 N=65536 的负样本队列，
-在单位球面上最大化正对互信息的下界，理论分析表明该下界以 O(1/√N) 收敛。"
+在单位球面上优化正对齐与表示均匀性代理。互信息下界和采样误差界依赖负样本分布、独立性与队列陈旧度假设，论文中应报告队列大小、温度、负样本策略和下游指标的消融。"
 
 ## 风险
 - τ 过小导致训练不稳定（梯度过大），τ 过大导致所有样本不分难易（退化均匀分布）

package/design-patterns/loss/information-bottleneck-loss.en.md

@@ -1,12 +1,12 @@
 # Information Bottleneck Loss
-> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
 
 ## Applicable Problems
 When a representation $Z$ must achieve an optimal balance between "retaining task-relevant information" and "compressing input redundancy." Typical scenarios: (1) Shared representations should retain only cross-task common information, discarding task-specific noise; (2) Private representations should retain only single-task unique information; (3) Routing features should maximize expert-task matching information. Core objective: **optimal information compression -- nothing more, nothing less, retaining only what is useful**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/probabilistic.md (information bottleneck principle, mutual information variational), lenses/variational.md (Lagrangian duality)
-- Knowledge: knowledge-base/probability/kl-divergence.md (IB theory, rate-distortion function), knowledge-base/probability/entropy.md (mutual information and conditional entropy)
+- Lenses: ../../lenses/probabilistic.en.md (information bottleneck principle, mutual information variational), ../../lenses/variational.en.md (Lagrangian duality)
+- Knowledge: ../../knowledge-base/probability/kl-divergence.en.md (IB theory, rate-distortion function), ../../knowledge-base/probability/entropy.en.md (mutual information and conditional entropy)
 
 ## Required Mathematical Knowledge
 - **Information Bottleneck Objective**: $\min I(X;Z) - \beta \cdot I(Z;Y)$, compressing $X \to Z$ while preserving the predictive power of $Z$ for $Y$
@@ -15,7 +15,7 @@ When a representation $Z$ must achieve an optimal balance between "retaining tas
   $I(Z;Y) \geq \mathbb{E}_{p(z,y)}[\log q(y|z)] + H(Y)$ (lower bound for prediction term)
 - **CPC (Contrastive Predictive Coding)**: InfoNCE lower bound on $I(Z_t; Z_{t+k})$
 - **MINE (Mutual Information Neural Estimation)**:
-  $I(X;Z) = \sup_\theta \{ \mathbb{E}[\log T_\theta(x,z)] - \log \mathbb{E}[T_\theta(x,z')] \}$
+  $I(X;Z) = \sup_\phi \{ \mathbb{E}_{p(x,z)}[T_\phi(x,z)] - \log \mathbb{E}_{p(x)p(z)}[\exp(T_\phi(x,z))] \}$ (Donsker-Varadhan representation, $T_\phi$ is a neural network critic)
 
 ## AI Module Form
 ```
@@ -32,7 +32,7 @@ Method 1 - VIB (Variational Information Bottleneck):
 
 Method 2 - Contrastive Mutual Information Estimation (no distributional assumptions):
   // NWJ estimator instead of KL
-  I_nwj(x;z) = E[f(x,z)] - exp(E[f(x,z')] - 1)  // f is a discriminator network
+  I_nwj(x;z) = E_{p(x,z)}[f(x,z)] - e^{-1} * E_{p(x)p(z)}[exp(f(x,z))]  // f is a discriminator network, expectations over joint and product of marginals respectively
   L_IB_contrast = I_nwj(x;z) - beta * InfoNCE(z, y)  // both terms differentiable
 
 Method 3 - Shared/Private IB Decomposition:
@@ -60,7 +60,7 @@ Method 3 - Shared/Private IB Decomposition:
 - **Operator Fusion**: Encoder forward + KL computation + discriminator forward can be partially fused
 
 ## Paper Phrasing
-"Based on information bottleneck theory, we formalize Shared/Private decomposition as $\min I(X;Z_s) + I(X;Z_p) - \beta_1 I(Z_s;Y_c) - \beta_2 I(Z_p;Y_s)$, replacing mutual information terms with variational upper and lower bounds for end-to-end variational, theoretically guaranteeing beta-optimality on the compression-prediction Pareto frontier."
+"Based on information bottleneck theory, we formalize Shared/Private decomposition as $\min I(X;Z_s) + I(X;Z_p) - \beta_1 I(Z_s;Y_c) - \beta_2 I(Z_p;Y_s)$, replacing mutual information terms with variational upper and lower bounds for end-to-end optimization. IB theory provides upper bounds on generalization error, but actual generalization also depends on optimization dynamics, data distribution, and model capacity."
 
 ## Risks
 - Mutual information estimators (MINE/NWJ) have high variance, causing training instability; large batches or moving averages are needed

package/design-patterns/loss/information-bottleneck-loss.md

@@ -1,5 +1,5 @@
 # Information Bottleneck Loss（信息瓶颈损失）
-> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行，需工程验证。
 
 ## 适用问题
 当需要让表示 Z 在"保留任务相关信息"与"压缩输入冗余信息"之间取得最优平衡时使用。
@@ -8,9 +8,9 @@
 核心诉求：**信息最优压缩——不多不少，只保留有用的**。
 
 ## 数学思想来源
-- 透镜：lenses/probabilistic.md（信息瓶颈原理、互信息优化）、lenses/variational.md（拉格朗日对偶）
-- 知识：knowledge-base/probability/kl-divergence.md（IB 理论、率失真函数）、
-  knowledge-base/probability/entropy.md（互信息与条件熵）
+- 透镜：../../lenses/probabilistic.md（信息瓶颈原理、互信息优化）、../../lenses/variational.md（拉格朗日对偶）
+- 知识：../../knowledge-base/probability/kl-divergence.md（IB 理论、率失真函数）、
+  ../../knowledge-base/probability/entropy.md（互信息与条件熵）
 
 ## 需要的数学知识
 - **信息瓶颈目标**：min I(X;Z) - β·I(Z;Y)，压缩 X→Z 同时保留 Z 对 Y 的预测力
@@ -19,7 +19,7 @@
   I(Z;Y) ≥ E_{p(z,y)}[log q(y|z)] + H(Y)  (用下界估计预测项)
 - **CPC (Contrastive Predictive Coding)**：I(Z_t; Z_{t+k}) 的 InfoNCE 下界
 - **MINE (Mutual Information Neural Estimation)**：
-  I(X;Z) = sup_θ { E[log T_θ(x,z)] - log E[T_θ(x,z')] }
+  I(X;Z) = sup_φ { E_{p(x,z)}[T_φ(x,z)] - log E_{p(x)p(z)}[exp(T_φ(x,z))] }  （Donsker-Varadhan 表示，T_φ 为神经网络 critic）
 
 ## AI 模块形式
 ```
@@ -36,7 +36,7 @@
 
 方法2 - 对比式互信息估计（无需分布假设）：
   // 用 NWJ 估计器替代 KL
-  I_nwj(x;z) = E[f(x,z)] - exp(E[f(x,z')] - 1)  // f 为判别网络
+  I_nwj(x;z) = E_{p(x,z)}[f(x,z)] - e^{-1} · E_{p(x)p(z)}[exp(f(x,z))]  // f 为判别网络，期望分别取 joint 和 product of marginals
   L_IB_contrast = I_nwj(x;z) - β · InfoNCE(z, y)  // 两项均可微
 
 方法3 - Shared/Private IB 分解：
@@ -65,7 +65,7 @@
 
 ## 论文表述方式
 "基于信息瓶颈理论，将 Shared/Private 分解形式化为 min I(X;Z_s)+I(X;Z_p)-β₁I(Z_s;Y_c)-β₂I(Z_p;Y_s)，
-通过变分上下界替代互信息项实现端到端优化，理论上保证压缩-预测 Pareto 前沿的 β-最优性。"
+通过变分上下界替代互信息项实现端到端优化。IB 理论给出泛化误差的上界，但实际泛化还依赖于优化动态、数据分布、模型容量等因素。"
 
 ## 风险
 - 互信息估计（MINE/NWJ）方差大，训练不稳定，需要大 batch 或 moving average

package/design-patterns/loss/orthogonality-loss.en.md

@@ -1,12 +1,12 @@
 # Orthogonality Loss
-> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as ✅ verified / ⚠️ retrofittable (needs validation) / ❌ infeasible. Unmarked claims are theoretically possible but require engineering validation.
+> **Rigor disclaimer**: Claims about complexity, memory, FlashAttention fusion, Tensor Core, and KV-Cache compression are marked as [v] verified / [~] retrofittable (needs validation) / [x] infeasible. Unmarked claims are theoretically possible but require engineering validation.
 
 ## Applicable Problems
 In multi-expert / multi-task settings, representations learned by submodules are highly overlapping and redundant, leading to poor parameter utilization. This loss is used when the $d$-dimensional feature space needs to be partitioned into $K$ non-interfering subspaces -- such as Shared-Private separation, MoE expert differentiation, and multi-task head decorrelation. Core objective: **ensure different modules see different things**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/projection.md (orthogonal projection and subspace decomposition), lenses/variational.md (regularization and saddle points)
-- Knowledge: knowledge-base/matrix-analysis/projection.md (spectral theorem, SVD, Schur decomposition), knowledge-base/probability/kl-divergence.md (redundancy and mutual information)
+- Lenses: ../../lenses/projection.en.md (orthogonal projection and subspace decomposition), ../../lenses/variational.en.md (regularization and saddle points)
+- Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (spectral theorem, SVD, Schur decomposition), ../../knowledge-base/probability/kl-divergence.en.md (redundancy and mutual information)
 
 ## Required Mathematical Knowledge
 - **Frobenius Inner Product and Orthogonality**: $\langle A, B \rangle_F = \text{tr}(A^T B)$; when $\langle A, B \rangle_F = 0$, $A \perp B$
@@ -18,17 +18,24 @@ In multi-expert / multi-task settings, representations learned by submodules are
 ```
 Module: OrthogonalDiversityLoss
 Input: K feature matrices {W_k in R^{d x r}}_{k=1}^K (weights or features of K submodules)
+  // Note: Grassmann distance requires QR decomposition of W_i first to obtain orthonormal bases Q_i
 
 Method 1 - Frobenius Orthogonal Regularization:
   L_orth = Sum_{i<j} ||W_i^T W_j||_F^2
   // Computation: O(K^2 * d * r^2), K typically <16 so cost is manageable
 
-Method 2 - Grassmann Distance (based on principal angles):
-  sigma_k = singular values of SVD(W_i^T W_j)
-  L_grass = Sum_{i<j} Sum_k sigma_k^2 * (1 - sigma_k^2)  // penalizes singular values that are neither 0 nor 1
-
-Method 3 - Efficient Cosine Decorrelation:
-  G = concat([W_1,...,W_K])^T * concat([W_1,...,W_K])  // single GEMM
+Method 2 - Subspace-overlap log barrier (based on principal angles):
+  sigma_k = singular values of SVD(Q_i^T Q_j) (= cos(theta_k), theta_k are principal angles)
+  // ⚠ Must first orthonormalize W_i, W_j: Q_i = qr(W_i).Q, Q_j = qr(W_j).Q
+  // Otherwise singular values may exceed 1, making -log(1-sigma^2+eps) undefined
+  // ⚠ Original formula sigma^2*(1-sigma^2) is wrong: penalty is 0 at BOTH sigma=0 (orthogonal) AND sigma=1 (complete overlap)!
+  // Fully overlapping subspaces receive zero penalty, defeating the orthogonality objective.
+  // Correct formula: log-barrier, 0 at sigma=0 and → +∞ as sigma→1
+  L_grass = Sum_{i<j} Sum_k -log(1 - sigma_k^2 + eps)  // = -Sum log(sin^2(theta_k)), 0 when orthogonal (theta=pi/2), →∞ when overlapping (theta→0)
+
+Method 3 - Efficient Normalized-Gram Decorrelation:
+  W_norm = column_normalize(concat([W_1,...,W_K]))       // without normalization this also penalizes norms, not only angles
+  G = W_norm^T * W_norm                                  // single GEMM
   L_corr = ||G * (1 - I)||_F^2   // mask out diagonal, penalize off-diagonal elements
 ```
 
@@ -38,17 +45,17 @@ Method 3 - Efficient Cosine Decorrelation:
 - **Block Computation**: When $K$ is large, perform mini-batch sampling over $(i,j)$ pairs, computing only $B$ out of $\binom{K}{2}$ pairs per step
 
 ## GPU Feasibility
-- **Tensorization**: Core operation is matmul ($W^T W$) -- standard GEMM, perfectly mapped to Tensor Cores
-- **GEMM-mappability**: Method 3 requires only 1 GEMM + 1 element-wise mask + Frobenius norm
-- **Complexity**: $O(K \cdot d \cdot r)$ storage + $O(d \cdot r^2 \cdot K)$ or $O(K^2 \cdot d \cdot r^2)$ computation; negligible when $K < 16$
-- **Memory & KV-Cache**: Intermediate matrices are on the order of $d \times r \cdot K$, adding no KV-Cache overhead
-- **Low Precision Stability**: Frobenius norm is a sum of squares, safe under fp16; Grassmann SVD is recommended in fp32
-- **Parallelism & Communication**: The $K$ pairs are embarrassingly parallel; can be distributed across GPUs with all-reduce
-- **Sparse Structure**: If $W_k$ is itself sparse (e.g., MoE gate), masking further increases sparsity
-- **Operator Fusion**: matmul -> mask -> square -> sum can be fused into a single CUDA kernel
+- **D1[v]**: Core operation is matmul ($W^T W$) -- standard GEMM, perfectly mapped to Tensor Cores
+- **D2[v]**: Method 3 requires only 1 GEMM + 1 element-wise mask + Frobenius norm
+- **D3[v]**: $O(K \cdot d \cdot r)$ storage + $O(d \cdot r^2 \cdot K)$ or $O(K^2 \cdot d \cdot r^2)$ computation; negligible when $K < 16$
+- **D4[v]**: Intermediate matrices are on the order of $d \times r \cdot K$, adding no KV-Cache overhead
+- **D5[v]**: Frobenius norm is a sum of squares, safe under fp16; Grassmann SVD is recommended in fp32
+- **D6[v]**: The $K$ pairs are embarrassingly parallel; can be distributed across GPUs with all-reduce
+- **D7[v]**: If $W_k$ is itself sparse (e.g., MoE gate), masking further increases sparsity
+- **D8[v]**: matmul -> mask -> square -> sum can be fused into a single CUDA kernel
 
 ## Paper Phrasing
-"We introduce an orthogonality regularizer L_orth = Sum_{i<j} ||W_i^T W_j||_F^2 that constrains the feature spaces of submodules to the approximately orthogonal Grassmann sub-manifold, theoretically guaranteeing that redundancy across $K$ subspaces decays as $O(1/\sqrt{d})$."
+"We introduce an orthogonality regularizer L_orth = Sum_{i<j} ||Q_i^T Q_j||_F^2, where Q_i is an orthonormal basis for W_i, to penalize overlap between feature subspaces of different submodules. This can reduce linear redundancy, but any redundancy-decay rate requires random-subspace or data-distribution assumptions and should be reported through principal angles, mutual-information estimates, or downstream ablations."
 
 ## Risks
 - Excessively large lambda causes variational difficulties (orthogonal constraint conflicts with task objective); requires careful tuning or adaptive lambda

package/design-patterns/loss/orthogonality-loss.md

@@ -1,5 +1,5 @@
 # Orthogonality Loss（正交性损失）
-> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「✅ 已验证 / ⚠️ 可改造需验证 / ❌ 不可行」。未标注的视为理论可行，需工程验证。
+> **严谨性声明**：本文件中涉及复杂度、显存、FlashAttention 融合、Tensor Core、KV-Cache 压缩的结论均标注为「[v] 已验证 / [~] 可改造需验证 / [x] 不可行」。未标注的视为理论可行，需工程验证。
 
 ## 适用问题
 多专家/多任务场景中，各子模块学习到的表示高度重叠、冗余，导致参数利用率低下。
@@ -7,9 +7,9 @@
 MoE 专家差异化、多任务 head 去相关。核心诉求：**让不同模块看到不同的东西**。
 
 ## 数学思想来源
-- 透镜：lenses/projection.md（正交投影与子空间分解）、lenses/variational.md（正则化与鞍点）
-- 知识：knowledge-base/matrix-analysis/projection.md（谱定理、SVD、Schur 分解）、
-  knowledge-base/probability/kl-divergence.md（冗余度与互信息）
+- 透镜：../../lenses/projection.md（正交投影与子空间分解）、../../lenses/variational.md（正则化与鞍点）
+- 知识：../../knowledge-base/matrix-analysis/projection.md（谱定理、SVD、Schur 分解）、
+  ../../knowledge-base/probability/kl-divergence.md（冗余度与互信息）
 
 ## 需要的数学知识
 - **Frobenius 内积与正交性**：⟨A, B⟩_F = tr(A^T B)，当 ⟨A, B⟩_F = 0 时 A⊥B
@@ -21,17 +21,24 @@ MoE 专家差异化、多任务 head 去相关。核心诉求：**让不同模
 ```
 模块：OrthogonalDiversityLoss
 输入：K 个特征矩阵 {W_k ∈ R^{d×r}}_{k=1}^K（K 个子模块的权重或特征）
+  // 注意：Grassmann 距离需先对 W_i 做 QR 分解取正交基 Q_i
 
 方法1 - Frobenius 正交正则：
   L_orth = Σ_{i<j} ‖W_i^T W_j‖_F²
   // 计算量：O(K² · d · r²)，K 一般 <16 所以可控
 
-方法2 - Grassmann 距离（基于主角度）：
-  σ_k = SVD(W_i^T W_j) 的奇异值
-  L_grass = Σ_{i<j} Σ_k σ_k²(1 - σ_k²)  // 惩罚非 0 非 1 的奇异值
+方法2 - 子空间重叠对数障碍（基于主角度）：
+  σ_k = SVD(Q_i^T Q_j) 的奇异值（= cos(θ_k)，θ_k 为主角度）
+  // ⚠ 必须先将 W_i, W_j 正交归一化：Q_i = qr(W_i).Q, Q_j = qr(W_j).Q
+  // 否则奇异值可能 > 1，导致 -log(1-σ²+ε) 未定义
+  // ⚠ 原公式 σ²(1-σ²) 错误：σ=0（正交）和 σ=1（完全重叠）时惩罚均为 0！
+  // 完全重叠的子空间获得零惩罚，违背正交性目标。
+  // 正确公式：对数障碍，σ=0 时为 0，σ→1 时 → +∞
+  L_grass = Σ_{i<j} Σ_k -log(1 - σ_k² + ε)  // = -Σ log(sin²θ_k)，正交时 θ=π/2 → 0，重叠时 θ→0 → ∞
 
-方法3 - 高效 cosine 去相关：
-  G = concat([W_1,...,W_K])^T · concat([W_1,...,W_K])  // 一次 GEMM
+方法3 - 高效归一化 Gram 去相关：
+  W_norm = column_normalize(concat([W_1,...,W_K]))       // 若不归一化，会同时惩罚范数而非只惩罚角度
+  G = W_norm^T · W_norm                                  // 一次 GEMM
   L_corr = ‖G ⊙ (1 - I)‖_F²   // mask 掉对角线，惩罚非对角元素
 ```
 
@@ -41,18 +48,17 @@ MoE 专家差异化、多任务 head 去相关。核心诉求：**让不同模
 - **分块计算**：当 K 很大时，对 (i,j) 对做 mini-batch 采样，每步只算 C(K,2) 中的 B 对
 
 ## GPU 可行性
-- **张量化**：核心操作为 matmul（W^T W）→ 标准 GEMM，完美映射 Tensor Core
-- **GEMM 可映射**：方法3 只需 1 次 GEMM + 1 次 element-wise mask + Frobenius 范数
-- **复杂度**：O(K·d·r) 存储 + O(d·r²·K) 或 O(K²·d·r²) 计算，K<16 时 negligible
-- **显存与 KV-Cache**：中间矩阵 d×r·K 量级，不增加 KV-Cache 负担
-- **低精度稳定**：Frobenius 范数为平方和，fp16 下 OK；Grassmann SVD 建议 fp32
-- **并行与通信**：K 对之间 embarrassingly parallel，可分 GPU 计算后 all-reduce
-- **稀疏结构**：若 W_k 本身稀疏（如 MoE gate），mask 后稀疏度进一步提升
-- **算子融合**：matmul → mask → square → sum 可融合为单个 CUDA kernel
+- **D1[v]**：核心操作为 matmul（W^T W）→ 标准 GEMM，完美映射 Tensor Core
+- **D2[v]**：方法3 只需 1 次 GEMM + 1 次 element-wise mask + Frobenius 范数
+- **D3[v]**：O(K·d·r) 存储 + O(d·r²·K) 或 O(K²·d·r²) 计算，K<16 时 negligible
+- **D4[v]**：中间矩阵 d×r·K 量级，不增加 KV-Cache 负担
+- **D5[v]**：Frobenius 范数为平方和，fp16 下 OK；Grassmann SVD 建议 fp32
+- **D6[v]**：K 对之间 embarrassingly parallel，可分 GPU 计算后 all-reduce
+- **D7[v]**：若 W_k 本身稀疏（如 MoE gate），mask 后稀疏度进一步提升
+- **D8[v]**：matmul → mask → square → sum 可融合为单个 CUDA kernel
 
 ## 论文表述方式
-"我们引入正交性正则项 L_orth = Σ_{i<j}‖W_i^T W_j‖_F²，将各子模块的特征空间约束
-到近似正交的 Grassmann 子流形上，理论上保证 K 个子空间的冗余度以 O(1/√d) 衰减。"
+"我们引入正交性正则项 L_orth = Σ_{i<j}‖Q_i^T Q_j‖_F²（其中 Q_i 为 W_i 的正交基），惩罚不同子模块特征子空间的重叠。该项可降低线性相关冗余，但冗余衰减速率需要随机子空间或数据分布假设支撑，应通过主角度、互信息估计或下游消融实测报告。"
 
 ## 风险
 - λ 过大导致优化困难（正交约束与任务目标冲突），需 careful tuning 或 adaptive λ