math-skill 3.0.1 → 3.1.1

package/design-patterns/loss/information-bottleneck-loss.en.md

@@ -5,8 +5,8 @@
 When a representation $Z$ must achieve an optimal balance between "retaining task-relevant information" and "compressing input redundancy." Typical scenarios: (1) Shared representations should retain only cross-task common information, discarding task-specific noise; (2) Private representations should retain only single-task unique information; (3) Routing features should maximize expert-task matching information. Core objective: **optimal information compression -- nothing more, nothing less, retaining only what is useful**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/probabilistic.md (information bottleneck principle, mutual information variational), lenses/variational.md (Lagrangian duality)
-- Knowledge: knowledge-base/probability/kl-divergence.md (IB theory, rate-distortion function), knowledge-base/probability/entropy.md (mutual information and conditional entropy)
+- Lenses: ../../lenses/probabilistic.en.md (information bottleneck principle, mutual information variational), ../../lenses/variational.en.md (Lagrangian duality)
+- Knowledge: ../../knowledge-base/probability/kl-divergence.en.md (IB theory, rate-distortion function), ../../knowledge-base/probability/entropy.en.md (mutual information and conditional entropy)
 
 ## Required Mathematical Knowledge
 - **Information Bottleneck Objective**: $\min I(X;Z) - \beta \cdot I(Z;Y)$, compressing $X \to Z$ while preserving the predictive power of $Z$ for $Y$
@@ -15,7 +15,7 @@ When a representation $Z$ must achieve an optimal balance between "retaining tas
   $I(Z;Y) \geq \mathbb{E}_{p(z,y)}[\log q(y|z)] + H(Y)$ (lower bound for prediction term)
 - **CPC (Contrastive Predictive Coding)**: InfoNCE lower bound on $I(Z_t; Z_{t+k})$
 - **MINE (Mutual Information Neural Estimation)**:
-  $I(X;Z) = \sup_\theta \{ \mathbb{E}[\log T_\theta(x,z)] - \log \mathbb{E}[T_\theta(x,z')] \}$
+  $I(X;Z) = \sup_\phi \{ \mathbb{E}_{p(x,z)}[T_\phi(x,z)] - \log \mathbb{E}_{p(x)p(z)}[\exp(T_\phi(x,z))] \}$ (Donsker-Varadhan representation, $T_\phi$ is a neural network critic)
 
 ## AI Module Form
 ```
@@ -32,7 +32,7 @@ Method 1 - VIB (Variational Information Bottleneck):
 
 Method 2 - Contrastive Mutual Information Estimation (no distributional assumptions):
   // NWJ estimator instead of KL
-  I_nwj(x;z) = E[f(x,z)] - exp(E[f(x,z')] - 1)  // f is a discriminator network
+  I_nwj(x;z) = E_{p(x,z)}[f(x,z)] - e^{-1} * E_{p(x)p(z)}[exp(f(x,z))]  // f is a discriminator network, expectations over joint and product of marginals respectively
   L_IB_contrast = I_nwj(x;z) - beta * InfoNCE(z, y)  // both terms differentiable
 
 Method 3 - Shared/Private IB Decomposition:
@@ -60,7 +60,7 @@ Method 3 - Shared/Private IB Decomposition:
 - **Operator Fusion**: Encoder forward + KL computation + discriminator forward can be partially fused
 
 ## Paper Phrasing
-"Based on information bottleneck theory, we formalize Shared/Private decomposition as $\min I(X;Z_s) + I(X;Z_p) - \beta_1 I(Z_s;Y_c) - \beta_2 I(Z_p;Y_s)$, replacing mutual information terms with variational upper and lower bounds for end-to-end variational, theoretically guaranteeing beta-optimality on the compression-prediction Pareto frontier."
+"Based on information bottleneck theory, we formalize Shared/Private decomposition as $\min I(X;Z_s) + I(X;Z_p) - \beta_1 I(Z_s;Y_c) - \beta_2 I(Z_p;Y_s)$, replacing mutual information terms with variational upper and lower bounds for end-to-end optimization. IB theory provides upper bounds on generalization error, but actual generalization also depends on optimization dynamics, data distribution, and model capacity."
 
 ## Risks
 - Mutual information estimators (MINE/NWJ) have high variance, causing training instability; large batches or moving averages are needed

package/design-patterns/loss/information-bottleneck-loss.md

@@ -8,9 +8,9 @@
 核心诉求：**信息最优压缩——不多不少，只保留有用的**。
 
 ## 数学思想来源
-- 透镜：lenses/probabilistic.md（信息瓶颈原理、互信息优化）、lenses/variational.md（拉格朗日对偶）
-- 知识：knowledge-base/probability/kl-divergence.md（IB 理论、率失真函数）、
-  knowledge-base/probability/entropy.md（互信息与条件熵）
+- 透镜：../../lenses/probabilistic.md（信息瓶颈原理、互信息优化）、../../lenses/variational.md（拉格朗日对偶）
+- 知识：../../knowledge-base/probability/kl-divergence.md（IB 理论、率失真函数）、
+  ../../knowledge-base/probability/entropy.md（互信息与条件熵）
 
 ## 需要的数学知识
 - **信息瓶颈目标**：min I(X;Z) - β·I(Z;Y)，压缩 X→Z 同时保留 Z 对 Y 的预测力
@@ -19,7 +19,7 @@
   I(Z;Y) ≥ E_{p(z,y)}[log q(y|z)] + H(Y)  (用下界估计预测项)
 - **CPC (Contrastive Predictive Coding)**：I(Z_t; Z_{t+k}) 的 InfoNCE 下界
 - **MINE (Mutual Information Neural Estimation)**：
-  I(X;Z) = sup_θ { E[log T_θ(x,z)] - log E[T_θ(x,z')] }
+  I(X;Z) = sup_φ { E_{p(x,z)}[T_φ(x,z)] - log E_{p(x)p(z)}[exp(T_φ(x,z))] }  （Donsker-Varadhan 表示，T_φ 为神经网络 critic）
 
 ## AI 模块形式
 ```
@@ -36,7 +36,7 @@
 
 方法2 - 对比式互信息估计（无需分布假设）：
   // 用 NWJ 估计器替代 KL
-  I_nwj(x;z) = E[f(x,z)] - exp(E[f(x,z')] - 1)  // f 为判别网络
+  I_nwj(x;z) = E_{p(x,z)}[f(x,z)] - e^{-1} · E_{p(x)p(z)}[exp(f(x,z))]  // f 为判别网络，期望分别取 joint 和 product of marginals
   L_IB_contrast = I_nwj(x;z) - β · InfoNCE(z, y)  // 两项均可微
 
 方法3 - Shared/Private IB 分解：
@@ -65,7 +65,7 @@
 
 ## 论文表述方式
 "基于信息瓶颈理论，将 Shared/Private 分解形式化为 min I(X;Z_s)+I(X;Z_p)-β₁I(Z_s;Y_c)-β₂I(Z_p;Y_s)，
-通过变分上下界替代互信息项实现端到端优化，理论上保证压缩-预测 Pareto 前沿的 β-最优性。"
+通过变分上下界替代互信息项实现端到端优化。IB 理论给出泛化误差的上界，但实际泛化还依赖于优化动态、数据分布、模型容量等因素。"
 
 ## 风险
 - 互信息估计（MINE/NWJ）方差大，训练不稳定，需要大 batch 或 moving average

package/design-patterns/loss/orthogonality-loss.en.md

@@ -5,8 +5,8 @@
 In multi-expert / multi-task settings, representations learned by submodules are highly overlapping and redundant, leading to poor parameter utilization. This loss is used when the $d$-dimensional feature space needs to be partitioned into $K$ non-interfering subspaces -- such as Shared-Private separation, MoE expert differentiation, and multi-task head decorrelation. Core objective: **ensure different modules see different things**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/projection.md (orthogonal projection and subspace decomposition), lenses/variational.md (regularization and saddle points)
-- Knowledge: knowledge-base/matrix-analysis/projection.md (spectral theorem, SVD, Schur decomposition), knowledge-base/probability/kl-divergence.md (redundancy and mutual information)
+- Lenses: ../../lenses/projection.en.md (orthogonal projection and subspace decomposition), ../../lenses/variational.en.md (regularization and saddle points)
+- Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (spectral theorem, SVD, Schur decomposition), ../../knowledge-base/probability/kl-divergence.en.md (redundancy and mutual information)
 
 ## Required Mathematical Knowledge
 - **Frobenius Inner Product and Orthogonality**: $\langle A, B \rangle_F = \text{tr}(A^T B)$; when $\langle A, B \rangle_F = 0$, $A \perp B$
@@ -18,17 +18,24 @@ In multi-expert / multi-task settings, representations learned by submodules are
 ```
 Module: OrthogonalDiversityLoss
 Input: K feature matrices {W_k in R^{d x r}}_{k=1}^K (weights or features of K submodules)
+  // Note: Grassmann distance requires QR decomposition of W_i first to obtain orthonormal bases Q_i
 
 Method 1 - Frobenius Orthogonal Regularization:
   L_orth = Sum_{i<j} ||W_i^T W_j||_F^2
   // Computation: O(K^2 * d * r^2), K typically <16 so cost is manageable
 
-Method 2 - Grassmann Distance (based on principal angles):
-  sigma_k = singular values of SVD(W_i^T W_j)
-  L_grass = Sum_{i<j} Sum_k sigma_k^2 * (1 - sigma_k^2)  // penalizes singular values that are neither 0 nor 1
-
-Method 3 - Efficient Cosine Decorrelation:
-  G = concat([W_1,...,W_K])^T * concat([W_1,...,W_K])  // single GEMM
+Method 2 - Subspace-overlap log barrier (based on principal angles):
+  sigma_k = singular values of SVD(Q_i^T Q_j) (= cos(theta_k), theta_k are principal angles)
+  // ⚠ Must first orthonormalize W_i, W_j: Q_i = qr(W_i).Q, Q_j = qr(W_j).Q
+  // Otherwise singular values may exceed 1, making -log(1-sigma^2+eps) undefined
+  // ⚠ Original formula sigma^2*(1-sigma^2) is wrong: penalty is 0 at BOTH sigma=0 (orthogonal) AND sigma=1 (complete overlap)!
+  // Fully overlapping subspaces receive zero penalty, defeating the orthogonality objective.
+  // Correct formula: log-barrier, 0 at sigma=0 and → +∞ as sigma→1
+  L_grass = Sum_{i<j} Sum_k -log(1 - sigma_k^2 + eps)  // = -Sum log(sin^2(theta_k)), 0 when orthogonal (theta=pi/2), →∞ when overlapping (theta→0)
+
+Method 3 - Efficient Normalized-Gram Decorrelation:
+  W_norm = column_normalize(concat([W_1,...,W_K]))       // without normalization this also penalizes norms, not only angles
+  G = W_norm^T * W_norm                                  // single GEMM
   L_corr = ||G * (1 - I)||_F^2   // mask out diagonal, penalize off-diagonal elements
 ```
 
@@ -38,17 +45,17 @@ Method 3 - Efficient Cosine Decorrelation:
 - **Block Computation**: When $K$ is large, perform mini-batch sampling over $(i,j)$ pairs, computing only $B$ out of $\binom{K}{2}$ pairs per step
 
 ## GPU Feasibility
-- **Tensorization**: Core operation is matmul ($W^T W$) -- standard GEMM, perfectly mapped to Tensor Cores
-- **GEMM-mappability**: Method 3 requires only 1 GEMM + 1 element-wise mask + Frobenius norm
-- **Complexity**: $O(K \cdot d \cdot r)$ storage + $O(d \cdot r^2 \cdot K)$ or $O(K^2 \cdot d \cdot r^2)$ computation; negligible when $K < 16$
-- **Memory & KV-Cache**: Intermediate matrices are on the order of $d \times r \cdot K$, adding no KV-Cache overhead
-- **Low Precision Stability**: Frobenius norm is a sum of squares, safe under fp16; Grassmann SVD is recommended in fp32
-- **Parallelism & Communication**: The $K$ pairs are embarrassingly parallel; can be distributed across GPUs with all-reduce
-- **Sparse Structure**: If $W_k$ is itself sparse (e.g., MoE gate), masking further increases sparsity
-- **Operator Fusion**: matmul -> mask -> square -> sum can be fused into a single CUDA kernel
+- **D1[v]**: Core operation is matmul ($W^T W$) -- standard GEMM, perfectly mapped to Tensor Cores
+- **D2[v]**: Method 3 requires only 1 GEMM + 1 element-wise mask + Frobenius norm
+- **D3[v]**: $O(K \cdot d \cdot r)$ storage + $O(d \cdot r^2 \cdot K)$ or $O(K^2 \cdot d \cdot r^2)$ computation; negligible when $K < 16$
+- **D4[v]**: Intermediate matrices are on the order of $d \times r \cdot K$, adding no KV-Cache overhead
+- **D5[v]**: Frobenius norm is a sum of squares, safe under fp16; Grassmann SVD is recommended in fp32
+- **D6[v]**: The $K$ pairs are embarrassingly parallel; can be distributed across GPUs with all-reduce
+- **D7[v]**: If $W_k$ is itself sparse (e.g., MoE gate), masking further increases sparsity
+- **D8[v]**: matmul -> mask -> square -> sum can be fused into a single CUDA kernel
 
 ## Paper Phrasing
-"We introduce an orthogonality regularizer L_orth = Sum_{i<j} ||W_i^T W_j||_F^2 that constrains the feature spaces of submodules to the approximately orthogonal Grassmann sub-manifold, theoretically guaranteeing that redundancy across $K$ subspaces decays as $O(1/\sqrt{d})$."
+"We introduce an orthogonality regularizer L_orth = Sum_{i<j} ||Q_i^T Q_j||_F^2, where Q_i is an orthonormal basis for W_i, to penalize overlap between feature subspaces of different submodules. This can reduce linear redundancy, but any redundancy-decay rate requires random-subspace or data-distribution assumptions and should be reported through principal angles, mutual-information estimates, or downstream ablations."
 
 ## Risks
 - Excessively large lambda causes variational difficulties (orthogonal constraint conflicts with task objective); requires careful tuning or adaptive lambda

package/design-patterns/loss/orthogonality-loss.md

@@ -7,9 +7,9 @@
 MoE 专家差异化、多任务 head 去相关。核心诉求：**让不同模块看到不同的东西**。
 
 ## 数学思想来源
-- 透镜：lenses/projection.md（正交投影与子空间分解）、lenses/variational.md（正则化与鞍点）
-- 知识：knowledge-base/matrix-analysis/projection.md（谱定理、SVD、Schur 分解）、
-  knowledge-base/probability/kl-divergence.md（冗余度与互信息）
+- 透镜：../../lenses/projection.md（正交投影与子空间分解）、../../lenses/variational.md（正则化与鞍点）
+- 知识：../../knowledge-base/matrix-analysis/projection.md（谱定理、SVD、Schur 分解）、
+  ../../knowledge-base/probability/kl-divergence.md（冗余度与互信息）
 
 ## 需要的数学知识
 - **Frobenius 内积与正交性**：⟨A, B⟩_F = tr(A^T B)，当 ⟨A, B⟩_F = 0 时 A⊥B
@@ -21,17 +21,24 @@ MoE 专家差异化、多任务 head 去相关。核心诉求：**让不同模
 ```
 模块：OrthogonalDiversityLoss
 输入：K 个特征矩阵 {W_k ∈ R^{d×r}}_{k=1}^K（K 个子模块的权重或特征）
+  // 注意：Grassmann 距离需先对 W_i 做 QR 分解取正交基 Q_i
 
 方法1 - Frobenius 正交正则：
   L_orth = Σ_{i<j} ‖W_i^T W_j‖_F²
   // 计算量：O(K² · d · r²)，K 一般 <16 所以可控
 
-方法2 - Grassmann 距离（基于主角度）：
-  σ_k = SVD(W_i^T W_j) 的奇异值
-  L_grass = Σ_{i<j} Σ_k σ_k²(1 - σ_k²)  // 惩罚非 0 非 1 的奇异值
+方法2 - 子空间重叠对数障碍（基于主角度）：
+  σ_k = SVD(Q_i^T Q_j) 的奇异值（= cos(θ_k)，θ_k 为主角度）
+  // ⚠ 必须先将 W_i, W_j 正交归一化：Q_i = qr(W_i).Q, Q_j = qr(W_j).Q
+  // 否则奇异值可能 > 1，导致 -log(1-σ²+ε) 未定义
+  // ⚠ 原公式 σ²(1-σ²) 错误：σ=0（正交）和 σ=1（完全重叠）时惩罚均为 0！
+  // 完全重叠的子空间获得零惩罚，违背正交性目标。
+  // 正确公式：对数障碍，σ=0 时为 0，σ→1 时 → +∞
+  L_grass = Σ_{i<j} Σ_k -log(1 - σ_k² + ε)  // = -Σ log(sin²θ_k)，正交时 θ=π/2 → 0，重叠时 θ→0 → ∞
 
-方法3 - 高效 cosine 去相关：
-  G = concat([W_1,...,W_K])^T · concat([W_1,...,W_K])  // 一次 GEMM
+方法3 - 高效归一化 Gram 去相关：
+  W_norm = column_normalize(concat([W_1,...,W_K]))       // 若不归一化，会同时惩罚范数而非只惩罚角度
+  G = W_norm^T · W_norm                                  // 一次 GEMM
   L_corr = ‖G ⊙ (1 - I)‖_F²   // mask 掉对角线，惩罚非对角元素
 ```
 
@@ -41,18 +48,17 @@ MoE 专家差异化、多任务 head 去相关。核心诉求：**让不同模
 - **分块计算**：当 K 很大时，对 (i,j) 对做 mini-batch 采样，每步只算 C(K,2) 中的 B 对
 
 ## GPU 可行性
-- **张量化**：核心操作为 matmul（W^T W）→ 标准 GEMM，完美映射 Tensor Core
-- **GEMM 可映射**：方法3 只需 1 次 GEMM + 1 次 element-wise mask + Frobenius 范数
-- **复杂度**：O(K·d·r) 存储 + O(d·r²·K) 或 O(K²·d·r²) 计算，K<16 时 negligible
-- **显存与 KV-Cache**：中间矩阵 d×r·K 量级，不增加 KV-Cache 负担
-- **低精度稳定**：Frobenius 范数为平方和，fp16 下 OK；Grassmann SVD 建议 fp32
-- **并行与通信**：K 对之间 embarrassingly parallel，可分 GPU 计算后 all-reduce
-- **稀疏结构**：若 W_k 本身稀疏（如 MoE gate），mask 后稀疏度进一步提升
-- **算子融合**：matmul → mask → square → sum 可融合为单个 CUDA kernel
+- **D1[v]**：核心操作为 matmul（W^T W）→ 标准 GEMM，完美映射 Tensor Core
+- **D2[v]**：方法3 只需 1 次 GEMM + 1 次 element-wise mask + Frobenius 范数
+- **D3[v]**：O(K·d·r) 存储 + O(d·r²·K) 或 O(K²·d·r²) 计算，K<16 时 negligible
+- **D4[v]**：中间矩阵 d×r·K 量级，不增加 KV-Cache 负担
+- **D5[v]**：Frobenius 范数为平方和，fp16 下 OK；Grassmann SVD 建议 fp32
+- **D6[v]**：K 对之间 embarrassingly parallel，可分 GPU 计算后 all-reduce
+- **D7[v]**：若 W_k 本身稀疏（如 MoE gate），mask 后稀疏度进一步提升
+- **D8[v]**：matmul → mask → square → sum 可融合为单个 CUDA kernel
 
 ## 论文表述方式
-"我们引入正交性正则项 L_orth = Σ_{i<j}‖W_i^T W_j‖_F²，将各子模块的特征空间约束
-到近似正交的 Grassmann 子流形上，理论上保证 K 个子空间的冗余度以 O(1/√d) 衰减。"
+"我们引入正交性正则项 L_orth = Σ_{i<j}‖Q_i^T Q_j‖_F²（其中 Q_i 为 W_i 的正交基），惩罚不同子模块特征子空间的重叠。该项可降低线性相关冗余，但冗余衰减速率需要随机子空间或数据分布假设支撑，应通过主角度、互信息估计或下游消融实测报告。"
 
 ## 风险
 - λ 过大导致优化困难（正交约束与任务目标冲突），需 careful tuning 或 adaptive λ

package/design-patterns/loss/variational-loss.en.md

@@ -5,8 +5,8 @@
 When sampling from latent variable distributions is required to generate diverse outputs. Typical scenarios: (1) Expert selection introduces discrete latent variables $z$ that need end-to-end variational; (2) Representation spaces need to model uncertainty; (3) Generative routing requires sampling from posterior distributions $p(z|x)$; (4) Bayesian mixture of experts. Core objective: **model distributions rather than point estimates in latent space, enabling uncertainty awareness and diversity**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/probabilistic.md (variational inference and ELBO), lenses/probabilistic.md (posterior and prior)
-- Knowledge: knowledge-base/probability/entropy.md (KL divergence, variational families), knowledge-base/probability/kl-divergence.md (ELBO derivation)
+- Lenses: ../../lenses/probabilistic.en.md (variational inference and ELBO), ../../lenses/probabilistic.en.md (posterior and prior)
+- Knowledge: ../../knowledge-base/probability/entropy.en.md (KL divergence, variational families), ../../knowledge-base/probability/kl-divergence.en.md (ELBO derivation)
 
 ## Required Mathematical Knowledge
 - **ELBO (Evidence Lower Bound)**: $\log p(x) \geq \mathbb{E}_{q(z|x)}[\log p(x|z)] - \text{KL}(q(z|x) \| p(z))$ -- the first term is reconstruction likelihood, the second regularizes the posterior toward the prior
@@ -31,8 +31,8 @@ Method 2 - Gumbel-Softmax (discrete expert selection):
   L_gumbel = CE(task_head(z_soft * features), y)  // straight-through estimator backprop
 
 Method 3 - Variational Information Bottleneck:
-  L_IB = -I(X; Z) + beta * I(Z; Y)  // maximize MI between Z and X, minimize redundancy between Z and Y
-  ~ E[-log q(x|z)] + beta * KL(q(z|x) || p(z))  // variational approximation
+  L_IB = I(X; Z) - beta * I(Z; Y)  // minimize information redundancy of Z about X, maximize predictive power of Z about Y
+  ~ E[-log q(y|z)] + beta * KL(q(z|x) || p(z))  // variational approximation (first term is prediction loss, second is compression penalty)
 ```
 
 ## Implementable Architectures

package/design-patterns/loss/variational-loss.md

@@ -8,9 +8,9 @@
 核心诉求：**在潜空间中建模分布而非点估计，获得不确定性感知与多样性**。
 
 ## 数学思想来源
-- 透镜：lenses/probabilistic.md（变分推断与 ELBO）、lenses/probabilistic.md（后验与先验）
-- 知识：knowledge-base/probability/entropy.md（KL 散度、变分族）、
-  knowledge-base/probability/kl-divergence.md（ELBO 推导）
+- 透镜：../../lenses/probabilistic.md（变分推断与 ELBO）、../../lenses/probabilistic.md（后验与先验）
+- 知识：../../knowledge-base/probability/entropy.md（KL 散度、变分族）、
+  ../../knowledge-base/probability/kl-divergence.md（ELBO 推导）
 
 ## 需要的数学知识
 - **ELBO（证据下界）**：log p(x) ≥ E_{q(z|x)}[log p(x|z)] - KL(q(z|x) ‖ p(z))
@@ -39,8 +39,8 @@
   L_gumbel = CE(task_head(z_soft ⊙ features), y)  // 直通估计器反向传播
 
 方法3 - 信息瓶颈变分：
-  L_IB = -I(X; Z) + β · I(Z; Y)  // 最大化 Z 与 X 的互信息，最小化 Z 与 Y 的冗余
-  ≈ E[-log q(x|z)] + β · KL(q(z|x) ‖ p(z))  // 变分近似
+  L_IB = I(X; Z) - β · I(Z; Y)  // 最小化 Z 对 X 的信息冗余，最大化 Z 对 Y 的预测能力
+  ≈ E[-log q(y|z)] + β · KL(q(z|x) ‖ p(z))  // 变分近似（第一项为预测损失，第二项为压缩惩罚）
 ```
 
 ## 可实现结构

package/design-patterns/overview.en.md

@@ -0,0 +1,36 @@
+# Design Translation Pattern Library
+
+> The design pattern library is not a complete model repository but a collection of "math → AI design" translation prototypes.
+> When existing patterns are insufficient, generate a temporary design candidate from the mathematical source and label it as a temporary design pattern, rather than refusing or force-fitting existing patterns.
+> Rigor convention: claims about complexity, memory, FlashAttention / Tensor Core / KV-Cache, and other engineering properties should be marked [v]/[~]/[x]; unmarked claims are theoretical possibilities requiring engineering validation.
+
+## Translation Paradigms
+
+| Mathematical Structure | AI Design Direction |
+|----------------------|-------------------|
+| Projection / Decomposition | subspace split / conflict removal / low-rank attention |
+| Spectral structure | token pruning / stability monitor / spectral filter |
+| Information theory | bottleneck loss / entropy gate / uncertainty routing |
+| Geometry / Metric | manifold representation / metric-aware update |
+| Topology | topology-preserving compression / obstruction loss |
+| Duality | constrained optimization / primal-dual training |
+| Symmetry / Group | equivariant features / weight sharing / orbit aggregation |
+| Variational | energy minimization / variational regularization |
+
+## Relationship to Activation Anchors
+
+```
+Activation anchors provide math tools → Design patterns translate into AI modules
+```
+
+When an activation anchor triggers the Knowledge Gap Protocol to generate a temporary knowledge card, a corresponding temporary design candidate should also be generated here.
+
+## By Component Type
+
+| Component | Directory | Count |
+|-----------|-----------|-------|
+| Attention | `attention/` | 5 |
+| Loss Functions | `loss/` | 5 |
+| Routing | `routing/` | 4 |
+| Representation | `representation/` | 4 |
+| Compression | `compression/` | 4 |

package/design-patterns/overview.md

@@ -0,0 +1,36 @@
+# 设计翻译原型库 / Design Translation Pattern Library
+
+> 设计模式库不是完整模型仓库，而是"数学 → AI 设计"的翻译原型集合。
+> 当现有模式不足时，应根据数学来源生成临时设计草案，并标记为 temporary design pattern，而不是拒绝或强行套用已有模式。
+> 严谨性约定：复杂度、显存、FlashAttention/Tensor Core/KV-Cache 等工程结论需用 [v]/[~]/[x] 标注；未标注结论视为理论可行但需工程验证。
+
+## 翻译范式
+
+| 数学结构 | AI 设计方向 |
+|---------|-----------|
+| 投影/分解 | subspace split / conflict removal / low-rank attention |
+| 谱结构 | token pruning / stability monitor / spectral filter |
+| 信息论 | bottleneck loss / entropy gate / uncertainty routing |
+| 几何/度量 | manifold representation / metric-aware update |
+| 拓扑 | topology-preserving compression / obstruction loss |
+| 对偶 | constrained optimization / primal-dual training |
+| 对称/群 | equivariant features / weight sharing / orbit aggregation |
+| 变分 | energy minimization / variational regularization |
+
+## 与知识锚点的关系
+
+```
+知识锚点提供数学工具 → 设计模式翻译成 AI 模块
+```
+
+当知识锚点触发 Knowledge Gap Protocol 生成临时知识卡时，此处也应生成对应的临时设计草案。
+
+## 按组件类型
+
+| 组件 | 目录 | 模式数 |
+|------|------|--------|
+| 注意力 | `attention/` | 5 |
+| 损失函数 | `loss/` | 5 |
+| 路由 | `routing/` | 4 |
+| 表示 | `representation/` | 4 |
+| 压缩 | `compression/` | 4 |

package/design-patterns/representation/equivariant-split.en.md

@@ -10,9 +10,9 @@ Typical scenarios: (1) Token permutation equivariance -- representation should c
 Core requirement: **encode symmetry priors into network architecture to reduce learning burden and improve generalization**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/geometric.md (group actions, invariant/equivariant maps), lenses/probabilistic.md (symmetry and information redundancy)
-- Knowledge: knowledge-base/matrix-analysis/projection.md (group representation theory, irreducible representations),
-  knowledge-base/differential-geometry/manifold.md (Lie groups, homogeneous spaces)
+- Lenses: ../../lenses/geometric.en.md (group actions, invariant/equivariant maps), ../../lenses/probabilistic.en.md (symmetry and information redundancy)
+- Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (group representation theory, irreducible representations),
+  ../../knowledge-base/differential-geometry/manifold.en.md (Lie groups, homogeneous spaces)
 
 ## Required Mathematical Background
 - **Group Action and Equivariance**: A map f is equivariant with respect to group G if and only if f(g * x) = g * f(x) for all g in G
@@ -78,7 +78,7 @@ Method 3 - Group convolution / group pooling:
 - **Operator fusion**: Split -> batched matmul -> concat can be fused; group pooling scatter + reduce can be fused
 
 ## Paper-Worthy Formulation
-"Leveraging Schur's lemma from group representation theory, we decompose the d-dimensional feature space into a direct sum of irreducible representations of symmetry group G, with each component independently processed by equivariance-constrained linear layers. This achieves a |G| / sum(m_i^2) improvement in parameter efficiency while guaranteeing network equivariance under group actions, reducing generalization error by O(1/sqrt(|G|)) on symmetric data."
+"Leveraging Schur's lemma from group representation theory, we decompose the d-dimensional feature space into a direct sum of irreducible representations of the symmetry group G, with each component processed by equivariance-constrained linear layers. Under the assumed group action this enforces equivariance and reduces learnable degrees of freedom; the exact parameter savings depend on the representation decomposition and channel multiplicities. Generalization gains must be measured on tasks with the corresponding symmetry and should not be stated as an unconditional O(1/sqrt(|G|)) rate."
 
 ## Risks
 - Improper group selection (too large constrains cause underfitting, too small fails to capture symmetries)

package/design-patterns/representation/equivariant-split.md

@@ -10,9 +10,9 @@
 核心诉求：**让网络结构编码对称性先验，减少学习负担，提升泛化**。
 
 ## 数学思想来源
-- 透镜：lenses/geometric.md（群作用、不变/等变映射）、lenses/probabilistic.md（对称性与信息冗余）
-- 知识：knowledge-base/matrix-analysis/projection.md（群表示论、不可约表示）、
-  knowledge-base/differential-geometry/manifold.md（李群、齐性空间）
+- 透镜：../../lenses/geometric.md（群作用、不变/等变映射）、../../lenses/probabilistic.md（对称性与信息冗余）
+- 知识：../../knowledge-base/matrix-analysis/projection.md（群表示论、不可约表示）、
+  ../../knowledge-base/differential-geometry/manifold.md（李群、齐性空间）
 
 ## 需要的数学知识
 - **群作用与等变性**：映射 f 对群 G 等变 ⟺ f(g·x) = g·f(x), ∀g∈G
@@ -79,8 +79,7 @@
 
 ## 论文表述方式
 "利用群表示论的 Schur 引理，将 d 维特征空间按对称群 G 的不可约表示分解为直和 ⊕mᵢVᵢ，
-每个分量使用受等变约束的线性层独立处理，参数效率提升 |G|/Σmᵢ² 倍，
-同时保证网络对群作用的等变性，在对称数据上泛化误差降低 O(1/√|G|)。"
+每个分量使用受等变约束的线性层独立处理，从而在满足群作用假设时严格保持等变性并减少可学习自由度。参数节省量取决于具体表示分解与通道重数；泛化收益需在具有相应对称性的任务上实测，不能无条件写成 O(1/√|G|)。"
 
 ## 风险
 - 群的选择不当（过大约束导致欠拟合，过小无法捕获对称性）

package/design-patterns/representation/manifold-representation.en.md

@@ -10,9 +10,9 @@ Use when input data resides in a high-dimensional space but is actually distribu
 Core requirement: **leverage the low-dimensional manifold structure of data to improve representation efficiency and generalization**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/geometric.md (Riemannian geometry, geodesics, curvature), lenses/probabilistic.md (intrinsic dimensionality)
-- Knowledge: knowledge-base/differential-geometry/manifold.md (manifolds, tangent spaces, exponential maps),
-  knowledge-base/matrix-analysis/projection.md (SVD, low-rank approximation, PCA)
+- Lenses: ../../lenses/geometric.en.md (Riemannian geometry, geodesics, curvature), ../../lenses/probabilistic.en.md (intrinsic dimensionality)
+- Knowledge: ../../knowledge-base/differential-geometry/manifold.en.md (manifolds, tangent spaces, exponential maps),
+  ../../knowledge-base/matrix-analysis/projection.en.md (SVD, low-rank approximation, PCA)
 
 ## Required Mathematical Background
 - **Manifold Hypothesis**: Data x in R^D actually lies on a smooth d-dimensional manifold M with d << D
@@ -66,17 +66,17 @@ Method 3 - Riemannian Optimization (optimize directly on the manifold):
 - **Adaptive d**: Local dimensionality varies across regions; estimate locally via PCA
 
 ## GPU Feasibility
-- **Tensorization**: Local linear projections are GEMM (d x D) @ (D x N); graph Laplacian regularization is SpMM
-- **GEMM-mappable**: K linear projections in Chart MoE form a batched GEMM (K x d x D) @ (D x N)
-- **Complexity**: k-NN construction O(N * D * log N) requires FAISS; manifold regularization O(N^2) requires sampling approximation
-- **Memory & KV-Cache**: K chart parameters K * d * D typically < 10 MB; k-NN graph N * k * 4 bytes
-- **Low-precision stability**: Distance computations and exp in fp16 require attention to numerical range; Riemannian retract recommended in fp32
-- **Parallelism & Communication**: K charts computed independently, perfectly parallel; k-NN search accelerated with FAISS GPU
-- **Sparse structure**: k-NN graph is naturally sparse; manifold regularization L is a sparse matrix, enabling SpMM acceleration
-- **Operator fusion**: Matmul + bias + activation within a chart can be fused; gating softmax + weighted-sum can be fused
+- **D1[v]**: Local linear projections are GEMM (d x D) @ (D x N); graph Laplacian regularization is SpMM
+- **D2[v]**: K linear projections in Chart MoE form a batched GEMM (K x d x D) @ (D x N)
+- **D3[~]**: k-NN construction O(N * D * log N) requires FAISS; manifold regularization O(N^2) requires sampling approximation
+- **D4[v]**: K chart parameters K * d * D typically < 10 MB; k-NN graph N * k * 4 bytes
+- **D5[~]**: Distance computations and exp in fp16 require attention to numerical range; Riemannian retract recommended in fp32
+- **D6[v]**: K charts computed independently, perfectly parallel; k-NN search accelerated with FAISS GPU
+- **D7[v]**: k-NN graph is naturally sparse; manifold regularization L is a sparse matrix, enabling SpMM acceleration
+- **D8[v]**: Matmul + bias + activation within a chart can be fused; gating softmax + weighted-sum can be fused
 
 ## Paper-Worthy Formulation
-"Based on the manifold hypothesis, we constrain D-dimensional token representations to a d*-dimensional (intrinsic dimension estimate) submanifold, achieving piecewise-linear approximation via K local coordinate charts (Chart MoE) and preserving global geodesic structure through graph Laplacian manifold regularization, with theoretical guarantees that embedding error converges at rate O(N^{-2/d*})."
+"Based on the manifold hypothesis, we model D-dimensional token representations as a low-intrinsic-dimensional structure, approximate it with K local coordinate charts (Chart MoE), and use graph-Laplacian regularization to encourage local-neighborhood consistency. Embedding-error or geodesic-preservation bounds require assumptions on sampling density, manifold smoothness, graph construction, and estimator choice; in practice report neighborhood preservation, reconstruction error, and downstream metrics."
 
 ## Risks
 - Inaccurate intrinsic dimension d* estimation leads to over-compression or dimension waste

package/design-patterns/representation/manifold-representation.md

@@ -10,9 +10,9 @@
 核心诉求：**利用数据的低维流形结构，提高表示效率和泛化能力**。
 
 ## 数学思想来源
-- 透镜：lenses/geometric.md（黎曼几何、测地线、曲率）、lenses/probabilistic.md（内在维度）
-- 知识：knowledge-base/differential-geometry/manifold.md（流形、切空间、指数映射）、
-  knowledge-base/matrix-analysis/projection.md（SVD、低秩近似、PCA）
+- 透镜：../../lenses/geometric.md（黎曼几何、测地线、曲率）、../../lenses/probabilistic.md（内在维度）
+- 知识：../../knowledge-base/differential-geometry/manifold.md（流形、切空间、指数映射）、
+  ../../knowledge-base/matrix-analysis/projection.md（SVD、低秩近似、PCA）
 
 ## 需要的数学知识
 - **流形假设**：数据 x ∈ R^D 实际分布在 d << D 维的光滑流形 M 上
@@ -66,19 +66,18 @@
 - **自适应 d**：不同区域的局部维度不同，用 PCA 局部估计
 
 ## GPU 可行性
-- **张量化**：局部线性投影为 GEMM (d×D)@(D×N)；图拉普拉斯正则为 SpMM
-- **GEMM 可映射**：Chart MoE 的 K 个线性投影为 batched GEMM (K×d×D)@(D×N)
-- **复杂度**：k-NN 构建 O(N·D·log N) 需 FAISS；流形正则 O(N²) 需采样近似
-- **显存与 KV-Cache**：K 个 chart 参数 K·d·D 通常 <10MB；k-NN 图 N·k·4 bytes
-- **低精度稳定**：距离计算和 exp 在 fp16 下需注意数值范围；黎曼 retract 建议 fp32
-- **并行与通信**：K 个 chart 独立计算，完美并行；k-NN 搜索用 FAISS GPU 加速
-- **稀疏结构**：k-NN 图天然稀疏，流形正则 L 为稀疏矩阵，SpMM 加速
-- **算子融合**：chart 内的 matmul+bias+activation 可融合；门控 softmax+weighted-sum 可融合
+- **D1[v]**：局部线性投影为 GEMM (d×D)@(D×N)；图拉普拉斯正则为 SpMM
+- **D2[v]**：Chart MoE 的 K 个线性投影为 batched GEMM (K×d×D)@(D×N)
+- **D3[~]**：k-NN 构建 O(N·D·log N) 需 FAISS；流形正则 O(N²) 需采样近似
+- **D4[v]**：K 个 chart 参数 K·d·D 通常 <10MB；k-NN 图 N·k·4 bytes
+- **D5[~]**：距离计算和 exp 在 fp16 下需注意数值范围；黎曼 retract 建议 fp32
+- **D6[v]**：K 个 chart 独立计算，完美并行；k-NN 搜索用 FAISS GPU 加速
+- **D7[v]**：k-NN 图天然稀疏，流形正则 L 为稀疏矩阵，SpMM 加速
+- **D8[v]**：chart 内的 matmul+bias+activation 可融合；门控 softmax+weighted-sum 可融合
 
 ## 论文表述方式
-"基于流形假设将 D 维 token 表示约束到 d*（内在维度估计）维子流形上，
-通过 K 个局部坐标卡（Chart MoE）实现分段线性近似，配合图拉普拉斯流形正则
-保持全局测地线结构，理论上保证嵌入误差以 O(N^{-2/d*}) 收敛。"
+"基于流形假设将 D 维 token 表示建模为低内在维度结构，
+通过 K 个局部坐标卡（Chart MoE）实现分段近似，并用图拉普拉斯正则鼓励局部邻域一致性。嵌入误差或测地线保持只在采样密度、流形光滑性、图构造和估计器假设满足时才有理论界，实际应报告邻域保持、重构误差和下游指标。"
 
 ## 风险
 - 内在维度 d* 估计不准确导致过度压缩或维度浪费

package/design-patterns/representation/shared-private-decomposition.en.md

@@ -9,16 +9,16 @@ Typical scenarios: (1) Multi-task MoE -- Shared experts handle linguistic common
 Core requirement: **explicitly separate commonality from individuality to prevent negative transfer and catastrophic forgetting**.
 
 ## Mathematical Inspiration
-- Lenses: lenses/projection.md (subspace decomposition, direct sum decomposition), lenses/probabilistic.md (information decomposition)
-- Knowledge: knowledge-base/matrix-analysis/projection.md (direct sum decomposition V = U + W, projection operators),
-  knowledge-base/probability/kl-divergence.md (information decomposition: shared/synergy/unique)
+- Lenses: ../../lenses/projection.en.md (subspace decomposition, direct sum decomposition), ../../lenses/probabilistic.en.md (information decomposition)
+- Knowledge: ../../knowledge-base/matrix-analysis/projection.en.md (direct sum decomposition V = U + W, projection operators),
+  ../../knowledge-base/probability/kl-divergence.en.md (information decomposition: shared/synergy/unique)
 
 ## Required Mathematical Background
 - **Direct Sum Decomposition**: R^d = S + P, where S intersect P = {0}, and every x = x_S + x_P is unique
   Projection matrices P_S + P_P = I, P_S * P_P = 0
 - **Information Decomposition (Williams & Beer PID)**:
   I(X; Y_1, Y_2) = Shared + Unique_1 + Unique_2 + Synergy
-  Shared = the redundant information component min(I(X; Y_1), I(X; Y_2))
+  Shared is the redundant-information term; min(I(X;Y_1), I(X;Y_2)) is only an early/simplified proxy, not the general Williams-Beer redundancy definition
 - **Low-Rank + Sparse Decomposition (RPCA)**: M = L + S, where L is low-rank (common) + S is sparse (specific)
   Solved via nuclear norm + L1 norm convex relaxation
 - **CCA (Canonical Correlation Analysis)**: max corr(W_1^T X, W_2^T Y), extracting shared variation between two sets of variables
@@ -42,12 +42,17 @@ Method 2 - Gated Decomposition (dynamic weighting):
   z = gate * z_shared + (1 - gate) * z_private
   // Gating allows per-dimension selection of shared/private contribution ratios
 
-Method 3 - Adversarial Decomposition (information-theoretic guarantee):
+Method 3 - Adversarial Decomposition (information-theoretic proxy):
   z_shared = E_shared(X)
   z_private = E_private[t](X)
   // Shared should be indistinguishable across tasks (adversarial gradient):
+  // ⚠ Sign correctness is critical! Original formula L_adv = -CE with flip_gradient creates a double sign reversal:
+  //   Discriminator minimizes -CE → gets worse at prediction; Encoder (sees +CE after reversal) → helps discriminator → shared becomes MORE task-specific!
+  // Correct approach: L_adv = +CE, combined with flip_gradient:
+  //   Discriminator minimizes +CE → learns to predict task from shared
+  //   Encoder (gradient reversed) sees -CE → maximizes discriminator loss → shared becomes task-independent
   task_pred = classifier(z_shared.flip_gradient())
-  L_adv = -CE(task_pred, t)        // Shared contains no task information
+  L_adv = CE(task_pred, t)        // Discriminator: minimize CE to predict task; Encoder: sees -CE after reversal, maximizes CE, making shared task-independent
   // Private should be discriminative across tasks:
   L_private = CE(classifier(z_private), t)
   L = L_task + lambda_adv * L_adv + lambda_priv * L_private
@@ -75,7 +80,7 @@ Dimension Allocation Principle:
 - **Operator fusion**: Additive fusion is trivial; gated fusion sigmoid -> multiply -> add can be fused
 
 ## Paper-Worthy Formulation
-"We decompose the multi-task representation space R^d into a direct sum S + P. The Shared subspace ensures task-invariance through adversarial training (H(T|Z_s) -> log T), while the Private subspace guarantees information complementarity with Shared via orthogonal regularization. Theoretical analysis shows that negative transfer decays as O(||P_S * P_P||_F) with increasing orthogonality."
+"We decompose the multi-task representation space R^d into a shared subspace S and task-private subspaces P: the shared branch reduces task identifiability through adversarial training, while the private branches use orthogonality / decorrelation regularization to reduce linear overlap with S. Information complementarity and reduced negative transfer must be validated with task-transfer matrices, mutual-information / PID proxies, and ablations; they are not automatically guaranteed by orthogonality alone."
 
 ## Risks
 - Min-max variational in adversarial training is unstable; the gradient reversal scale and lambda_adv require careful tuning

package/design-patterns/representation/shared-private-decomposition.md

@@ -9,16 +9,16 @@
 核心诉求：**显式分离共性与个性，避免负迁移和灾难性遗忘**。
 
 ## 数学思想来源
-- 透镜：lenses/projection.md（子空间分解、直和分解）、lenses/probabilistic.md（信息分解）
-- 知识：knowledge-base/matrix-analysis/projection.md（直和分解 V = U ⊕ W、投影算子）、
-  knowledge-base/probability/kl-divergence.md（信息分解：shared/synergy/unique）
+- 透镜：../../lenses/projection.md（子空间分解、直和分解）、../../lenses/probabilistic.md（信息分解）
+- 知识：../../knowledge-base/matrix-analysis/projection.md（直和分解 V = U ⊕ W、投影算子）、
+  ../../knowledge-base/probability/kl-divergence.md（信息分解：shared/synergy/unique）
 
 ## 需要的数学知识
 - **直和分解**：R^d = S ⊕ P，其中 S ∩ P = {0}，每个 x = x_S + x_P 唯一
   投影矩阵 P_S + P_P = I，P_S · P_P = 0
 - **信息分解 (Williams & Beer PID)**：
   I(X;Y₁,Y₂) = Shared + Unique₁ + Unique₂ + Synergy
-  Shared = min(I(X;Y₁), I(X;Y₂)) 的冗余信息部分
+  Shared 为冗余信息项；min(I(X;Y₁), I(X;Y₂)) 只是早期/简化代理，不是 Williams-Beer 冗余的一般定义
 - **低秩+稀疏分解 (RPCA)**：M = L + S，L 低秩（共性）+ S 稀疏（特异）
   通过核范数 + L1 范数凸松弛求解
 - **CCA (典型相关分析)**：max corr(W₁^T X, W₂^T Y)，提取两组变量的共享变异
@@ -42,12 +42,17 @@
   z = gate ⊙ z_shared + (1 - gate) ⊙ z_private
   // 门控允许逐维度选择 shared/private 的贡献比例
 
-方法3 - 对抗分解（信息论保证）：
+方法3 - 对抗分解（信息论代理）：
   z_shared = E_shared(X)
   z_private = E_private[t](X)
   // Shared 应无法区分任务（对抗梯度）：
+  // ⚠ 符号正确性至关重要！原公式 L_adv = -CE 与 flip_gradient 形成双重符号反转：
+  //   判别器最小化 -CE → 变差；编码器（梯度反转后看到 +CE）→ 帮助判别器 → shared 变得更任务相关！
+  // 正确做法：L_adv = +CE，配合 flip_gradient：
+  //   判别器最小化 +CE → 学会从 shared 预测任务
+  //   编码器（梯度反转）看到 -CE → 最大化判别器损失 → shared 不含任务信息
   task_pred = classifier(z_shared.flip_gradient())
-  L_adv = -CE(task_pred, t)       // Shared 不含任务信息
+  L_adv = CE(task_pred, t)       // 判别器：最小化 CE 预测任务；编码器：梯度反转后最大化 CE，使 shared 任务无关
   // Private 应能区分任务：
   L_private = CE(classifier(z_private), t)
   L = L_task + λ_adv·L_adv + λ_priv·L_private
@@ -75,9 +80,7 @@
 - **算子融合**：加法融合 trivial；门控融合的 sigmoid→multiply→add 可融合
 
 ## 论文表述方式
-"将多任务表示空间 R^d 分解为直和 S ⊕ P，Shared 子空间通过对抗训练确保任务无关性
-（H(T|Z_s)→log T），Private 子空间通过正交正则保证与 Shared 的信息互补，
-理论分析表明负迁移随正交性以 O(‖P_S·P_P‖_F) 衰减。"
+"将多任务表示空间 R^d 分解为共享子空间 S 与任务私有子空间 P：共享分支通过对抗训练降低任务可识别性，私有分支通过正交/去相关正则减少与共享分支的线性重叠。信息互补与负迁移降低需通过任务间迁移矩阵、互信息/PID 代理和消融实验验证，不能仅由正交性自动保证。"
 
 ## 风险
 - 对抗训练的 min-max 优化不稳定，梯度反转的 scale 和 λ_adv 需精心调节