math-skill 3.0.0 → 3.1.0

package/knowledge-base/matrix-analysis/matrix-perturbation.md

@@ -45,5 +45,17 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/matrix-analysis.md（§4.3 特征值不等式 Weyl/交错、§6.1-6.3 Ger\v{s}gorin 圆盘与扰动定理、§5.8 条件数）
+- 蒸馏稿：../../references/books/matrix-analysis.md（§4.3 特征值不等式 Weyl/交错、§6.1-6.3 Ger\v{s}gorin 圆盘与扰动定理、§5.8 条件数）
 - 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 4 §4.3 (Eigenvalue Inequalities) + Chapter 6 (Location and Perturbation of Eigenvalues §6.1-6.3)
+
+
+## 路由扩展
+- 若关注特征值灵敏度 → `spectral-decomposition.md`（谱分解的扰动分析）
+- 若涉及随机扰动界 → `../probability/concentration-inequality.md`（随机矩阵集中不等式）
+
+## 可扩展方向
+- 伪谱（pseudospectra）：非正规矩阵的谱灵敏度分析
+- 结构化扰动（structured perturbation）：保持矩阵结构的扰动分析
+- Davis-Kahan 定理变体：子空间扰动的多种界
+- 特征值交错（eigenvalue interlacing）：主子矩阵与特征值关系
+- Wilkinson 多项式：多项式根的条件数与扰动灵敏度

package/knowledge-base/matrix-analysis/positive-semidefinite.en.md

@@ -2,12 +2,12 @@
 
 ## Minimal Definition
 
-A Hermitian matrix $A$ is positive semidefinite (PSD) if $x^HAx \geq 0$ for all nonzero vectors $x$, denoted $A \succeq 0$. Equivalent conditions: all eigenvalues $\geq 0$; there exists $B$ such that $A = B^HB$ (Gram representation); all leading principal minors $\geq 0$. Positive definite (PD) requires strict $> 0$, denoted $A \succ 0$.
+A Hermitian matrix $A$ is positive semidefinite (PSD) if $x^HAx \geq 0$ for all nonzero vectors $x$, denoted $A \succeq 0$. Equivalent conditions: all eigenvalues $\geq 0$; there exists $B$ such that $A = B^HB$ (Gram representation); **all principal minors** $\geq 0$ (note: ALL principal minors, not just the leading ones). Positive definite (PD) requires strict $> 0$, denoted $A \succ 0$.
 
 ## Core Formulas
 
-- PSD equivalence: $A \succeq 0 \iff \lambda_i(A) \geq 0 \ \forall i \iff A = B^HB$
-- PD equivalence: $A \succ 0 \iff \lambda_i(A) > 0 \ \forall i \iff$ all leading principal minors $> 0$ (Sylvester's criterion)
+- PSD equivalence: $A \succeq 0 \iff \lambda_i(A) \geq 0 \ \forall i \iff A = B^HB \iff$ all principal minors $\geq 0$ (note: **all** principal minors, not just the leading ones)
+- PD equivalence (Sylvester's criterion): $A \succ 0 \iff \lambda_i(A) > 0 \ \forall i \iff$ all leading principal minors $> 0$ (checking only leading principal minors suffices; this is necessary and sufficient for PD)
 - Cholesky decomposition: $A \succ 0 \implies A = LL^H$, $L$ lower triangular
 - Loewner partial order: $A \succeq B \iff A - B \succeq 0$
 - Schur product theorem: $A \succeq 0, B \succeq 0 \implies A \circ B \succeq 0$ (Hadamard product preserves PSD)
@@ -20,12 +20,12 @@ A Hermitian matrix $A$ is positive semidefinite (PSD) if $x^HAx \geq 0$ for all
 - Covariance matrices: $\Sigma = \mathbb{E}[xx^H] \succeq 0$; PCA/whitening relies on positive definiteness
 - Second-order optimization preconditioning: The PSD structure of the Hessian/Fisher information matrix guarantees descent directions
 - Semidefinite programming (SDP): optimizing a linear objective subject to $X \succeq 0$ constraints
-- Attention matrix analysis: Gram structure of the row-stochastic matrix output by softmax
+- Attention matrix analysis: a row-stochastic softmax attention matrix is generally not a Gram/PSD matrix; PSD tools apply only after explicitly constructing a symmetric PSD kernel (e.g. $K_{ij}=k(x_i,x_j)$) or symmetrizing and projecting to the PSD cone.
 
 ## AI Design Translation
 
 - **PSD kernel engineering (learnable kernels)**: Use the Schur product theorem to combine multiple PSD kernels: $K = K_1 \circ K_2 \circ \cdots$ (Hadamard product), guaranteeing the result remains PSD. Implemented as elementwise tensor multiplication `K = K1 * K2`, $O(n^2)$ elementwise, extremely GPU-friendly. Can parameterize $K_\theta(x,y) = \exp(-\|f_\theta(x)-f_\theta(y)\|^2)$ to guarantee PSD.
-- **Covariance whitening**: $\hat{x} = \Sigma^{-1/2}x$, where $\Sigma^{-1/2}$ is approximated via Newton-Schulz iteration (pure matmul). Newton-Schulz: $X_{k+1} = \frac{1}{2}X_k(3I - AX_k)$, two matmul operations per step, converging in 5-6 steps. BatchNorm can be viewed as an approximation to diagonal whitening.
+- **Covariance whitening**: $\hat{x} = \Sigma^{-1/2}x$, where $\Sigma^{-1/2}$ can be approximated via scaled Newton-Schulz iteration (pure matmul). A common coupled form uses $Y_0=A/\alpha, Z_0=I$, $T_k=\frac{1}{2}(3I-Z_kY_k)$, $Y_{k+1}=Y_kT_k, Z_{k+1}=T_kZ_k$, with $A^{-1/2}\approx Z_k/\sqrt{\alpha}$; an equivalent single-variable form is $X_{k+1}=\frac{1}{2}X_k(3I-AX_k^2)$. Convergence requires spectral scaling and positive definiteness; in low precision, residual checks are needed. 5-6 steps is an engineering budget, not a universal guarantee. BatchNorm can be viewed as an approximation to diagonal whitening.
 - **Cholesky preconditioner**: For a PSD Hessian $H$, use the $H = LL^H$ decomposition and solve $L^{-1}L^{-H}g$ instead of $H^{-1}g$. cuSOLVER provides batched Cholesky `potrf`. In K-FAC, the inversion of each Kronecker factor proceeds via Cholesky.
 - **Nearest PSD approximation (Higham)**: Given a symmetric matrix $A$, find the nearest PSD matrix $A_+ = \arg\min_{X \succeq 0} \|A - X\|_F$. Solution: EVD $A = U\Lambda U^H$, clamp negative values in $\Lambda$ to zero, $A_+ = U\Lambda_+ U^H$. Used to correct loss of positive definiteness in covariance matrices due to floating-point errors.
 - **Jitter / diagonal loading**: $A_{\text{stable}} = A + \epsilon I$ ($\epsilon \sim 10^{-6}$), ensuring numerical positive definiteness. Standard practice in Gaussian processes, kernel methods, and Cholesky decomposition. Implemented as `A + eps * torch.eye(n)`, a zero-cost operation.
@@ -46,5 +46,19 @@ A Hermitian matrix $A$ is positive semidefinite (PSD) if $x^HAx \geq 0$ for all
 
 ## Further References
 
-- Distilled notes: references/books/matrix-analysis.md (Ch 7 Positive Definite and Semidefinite Matrices, Section 7.5 Schur Product Theorem, Section 7.7 Loewner Partial Order)
+- Distilled notes: ../../references/books/matrix-analysis.en.md (Ch 7 Positive Definite and Semidefinite Matrices, Section 7.5 Schur Product Theorem, Section 7.7 Loewner Partial Order)
 - Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 7 (Positive Definite and Semidefinite Matrices Section 7.1-7.8)
+
+
+## Routing Extensions
+- If solving SDP problems -> `../optimization/convex-optimization.md` (semidefinite programming as convex optimization)
+- If PSD matrix conditioning and perturbation are involved -> `matrix-perturbation.en.md` (eigenvalue perturbation bounds)
+- If used for Fisher information matrix -> `../probability/fisher-information.md` (PSD property of Fisher information)
+
+## Extensible Directions
+- Semidefinite programming (SDP): solution methods and applications
+- PSD completion: completing partially known PSD matrices
+- Matrix square root: unique square root of PSD matrices
+- Lowner order: partial order on the PSD cone
+- Operator monotone functions: Loewner-Heinz theorem
+- Completely positive matrices: CP decomposition and cone structure

package/knowledge-base/matrix-analysis/positive-semidefinite.md

@@ -2,12 +2,12 @@
 
 ## 最小定义
 
-Hermitian 矩阵 $A$ 若对所有非零向量 $x$ 满足 $x^HAx \geq 0$，则称半正定（PSD），记 $A \succeq 0$。等价条件：所有特征值 $\geq 0$；存在 $B$ 使 $A = B^HB$（Gram 表示）；所有顺序主子式 $\geq 0$。正定（PD）要求严格 $> 0$，记 $A \succ 0$。
+Hermitian 矩阵 $A$ 若对所有非零向量 $x$ 满足 $x^HAx \geq 0$，则称半正定（PSD），记 $A \succeq 0$。等价条件：所有特征值 $\geq 0$；存在 $B$ 使 $A = B^HB$（Gram 表示）；**所有主子式** $\geq 0$（注意：是全部主子式，不仅是顺序主子式）。正定（PD）要求严格 $> 0$，记 $A \succ 0$。
 
 ## 核心公式
 
-- PSD 等价条件：$A \succeq 0 \iff \lambda_i(A) \geq 0 \ \forall i \iff A = B^HB$
-- 正定等价：$A \succ 0 \iff \lambda_i(A) > 0 \ \forall i \iff$ 所有顺序主子式 $> 0$（Sylvester 准则）
+- PSD 等价条件：$A \succeq 0 \iff \lambda_i(A) \geq 0 \ \forall i \iff A = B^HB \iff$ 所有主子式 $\geq 0$（注意：是 **所有** 主子式，不仅是顺序主子式）
+- 正定等价（Sylvester 准则）：$A \succ 0 \iff \lambda_i(A) > 0 \ \forall i \iff$ 所有顺序主子式 $> 0$（仅需检验顺序主子式，这是 PD 的充分必要条件）
 - Cholesky 分解：$A \succ 0 \implies A = LL^H$，$L$ 下三角
 - Loewner 偏序：$A \succeq B \iff A - B \succeq 0$
 - Schur 积定理：$A \succeq 0, B \succeq 0 \implies A \circ B \succeq 0$（Hadamard 积保 PSD）
@@ -20,12 +20,12 @@ Hermitian 矩阵 $A$ 若对所有非零向量 $x$ 满足 $x^HAx \geq 0$，则称
 - 协方差矩阵：$\Sigma = \mathbb{E}[xx^H] \succeq 0$，PCA/白化依赖其正定性
 - 二阶优化预条件：Hessian/Fisher 信息矩阵的 PSD 结构保证下降方向
 - 半定规划 (SDP)：约束 $X \succeq 0$ 下的线性目标优化
-- Attention 矩阵分析：softmax 输出的行随机矩阵的 Gram 结构
+- Attention 矩阵分析：softmax 输出的行随机矩阵一般不是 Gram/PSD；只有显式构造对称 PSD kernel（如 $K_{ij}=k(x_i,x_j)$）或对称化并投影到 PSD cone 后，才能使用 PSD 工具。
 
 ## AI 设计翻译
 
 - **PSD 核工程 (可学习核)**：用 Schur 积定理组合多个 PSD 核：$K = K_1 \circ K_2 \circ \cdots$（Hadamard 积），保证结果始终 PSD。实现为 elementwise 张量乘 `K = K1 * K2`，$O(n^2)$ elementwise，极 GPU 友好。可参数化 $K_\theta(x,y) = \exp(-\|f_\theta(x)-f_\theta(y)\|^2)$ 保证 PSD。
-- **协方差白化 (Whitening)**：$\hat{x} = \Sigma^{-1/2}x$，其中 $\Sigma^{-1/2}$ 通过 Newton-Schulz 迭代近似（纯 matmul）。Newton-Schulz：$X_{k+1} = \frac{1}{2}X_k(3I - AX_k)$，每步两次 matmul，5-6 步收敛。BatchNorm 可视为对角白化的近似。
+- **协方差白化 (Whitening)**：$\hat{x} = \Sigma^{-1/2}x$，其中 $\Sigma^{-1/2}$ 可通过缩放后的 Newton-Schulz 迭代近似（纯 matmul）。常用 coupled 形式：$Y_0=A/\alpha, Z_0=I$，$T_k=\frac{1}{2}(3I-Z_kY_k)$，$Y_{k+1}=Y_kT_k, Z_{k+1}=T_kZ_k$，最后 $A^{-1/2}\approx Z_k/\sqrt{\alpha}$；或单变量形式 $X_{k+1}=\frac{1}{2}X_k(3I-AX_k^2)$。收敛需要谱缩放和正定条件，低精度下需残差监控；5-6 步只是常见工程预算。BatchNorm 可视为对角白化的近似。
 - **Cholesky 预条件子**：对 PSD Hessian $H$，用 $H = LL^H$ 分解后解 $L^{-1}L^{-H}g$ 代替 $H^{-1}g$。cuSOLVER 有 batched Cholesky `potrf`。K-FAC 中每个 Kronecker 因子的求逆即走 Cholesky。
 - **最近 PSD 近似 (Higham)**：给定对称矩阵 $A$，求最近 PSD 矩阵 $A_+ = \arg\min_{X \succeq 0} \|A - X\|_F$。解法：EVD $A = U\Lambda U^H$，将 $\Lambda$ 中负值截零，$A_+ = U\Lambda_+ U^H$。用于修正浮点误差导致的协方差矩阵失去正定性。
 - **Jitter / 对角加载**：$A_{\text{stable}} = A + \epsilon I$（$\epsilon \sim 10^{-6}$），保证数值正定性。高斯过程、核方法、Cholesky 分解中标准做法。实现为 `A + eps * torch.eye(n)`，零成本操作。
@@ -46,5 +46,19 @@ Hermitian 矩阵 $A$ 若对所有非零向量 $x$ 满足 $x^HAx \geq 0$，则称
 
 ## 深入参考
 
-- 蒸馏稿：references/books/matrix-analysis.md（Ch 7 Positive Definite and Semidefinite Matrices、§7.5 Schur 积定理、§7.7 Loewner 偏序）
+- 蒸馏稿：../../references/books/matrix-analysis.md（Ch 7 Positive Definite and Semidefinite Matrices、§7.5 Schur 积定理、§7.7 Loewner 偏序）
 - 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 7 (Positive Definite and Semidefinite Matrices §7.1-7.8)
+
+
+## 路由扩展
+- 若需要求解 SDP 问题 → `../optimization/convex-optimization.md`（半正定规划作为凸优化）
+- 若涉及 PSD 矩阵的条件与扰动 → `matrix-perturbation.md`（特征值扰动界）
+- 若用于 Fisher 信息矩阵 → `../probability/fisher-information.md`（Fisher 信息的 PSD 性质）
+
+## 可扩展方向
+- 半正定规划（semidefinite programming）：SDP 的求解方法与应用
+- PSD 补全（PSD completion）：部分已知 PSD 矩阵的补全
+- 矩阵平方根（matrix square root）：PSD 矩阵的唯一平方根
+- Lowner 序（Lowner order）：PSD 锥上的偏序关系
+- 算子单调函数（operator monotone functions）：Loewner-Heinz 定理
+- 完全正矩阵（completely positive matrices）：CP 分解与锥结构

package/knowledge-base/matrix-analysis/projection.en.md

@@ -43,5 +43,18 @@ Maps a vector $v$ onto a subspace $\mathcal{S}$ such that the residual $v - Pv$
 
 ## Further References
 
-- Distilled notes: references/books/matrix-analysis.md (Section 2.1 QR Decomposition, Section 4.2 Courant-Fischer Variational Characterization, Section 2.6 SVD)
+- Distilled notes: ../../references/books/matrix-analysis.en.md (Section 2.1 QR Decomposition, Section 4.2 Courant-Fischer Variational Characterization, Section 2.6 SVD)
 - Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 2 (Unitary Similarity) + Chapter 4 (Hermitian Matrices Section 4.2 Variational Characterizations)
+
+
+## Routing Extensions
+- If the goal is compression / dimensionality reduction -> `low-rank-approximation.en.md` (truncated SVD implementation)
+- If projection constraints are needed on a manifold -> `../optimization/riemannian-optimization.md` (constrained optimization on Riemannian manifolds)
+- If shared vs. private subspace separation is involved -> `shared-private-decomposition` (design pattern layer)
+
+## Extensible Directions
+- Oblique projection: non-orthogonal projection operators
+- Alternating projection: Von Neumann alternating projection convergence theorem
+- Projection onto convex sets (POCS): projecting onto intersections of convex sets
+- Randomized projection: Johnson-Lindenstrauss lemma and random projections
+- Subspace tracking: online subspace estimation methods

package/knowledge-base/matrix-analysis/projection.md

@@ -43,5 +43,18 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/matrix-analysis.md（§2.1 QR 分解、§4.2 Courant-Fischer 变分刻画、§2.6 SVD）
+- 蒸馏稿：../../references/books/matrix-analysis.md（§2.1 QR 分解、§4.2 Courant-Fischer 变分刻画、§2.6 SVD）
 - 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 2 (Unitary Similarity) + Chapter 4 (Hermitian Matrices §4.2 Variational Characterizations)
+
+
+## 路由扩展
+- 若目标是压缩/降维 → `low-rank-approximation.md`（截断 SVD 实现）
+- 若需要在流形上做投影约束 → `../optimization/riemannian-optimization.md`（黎曼流形上的约束优化）
+- 若涉及共享与私有子空间分离 → `shared-private-decomposition`（设计模式层）
+
+## 可扩展方向
+- 斜投影（oblique projection）：非正交投影算子
+- 交替投影（alternating projection）：Von Neumann 交替投影收敛定理
+- 凸集投影（POCS）：投影到多个凸集交集的方法
+- 随机投影（randomized projection）：Johnson-Lindenstrauss 引理与随机投影
+- 子空间跟踪（subspace tracking）：在线更新的子空间估计方法

package/knowledge-base/matrix-analysis/spectral-decomposition.en.md

@@ -44,5 +44,18 @@ Decomposes a matrix into a linear combination of eigenvalue-eigenvector pairs. F
 
 ## Further References
 
-- Distilled notes: references/books/matrix-analysis.md (Ch 1 Eigenvalues and Similarity, Section 2.4-2.5 Schur Triangularization and Normal Matrices, Section 4.2 Courant-Fischer)
+- Distilled notes: ../../references/books/matrix-analysis.en.md (Ch 1 Eigenvalues and Similarity, Section 2.4-2.5 Schur Triangularization and Normal Matrices, Section 4.2 Courant-Fischer)
 - Original text: Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 1 (Eigenvalues, Eigenvectors, Similarity) + Chapter 2 (Unitary Similarity Section 2.4-2.5)
+
+
+## Routing Extensions
+- If truncation approximation is needed -> `low-rank-approximation.en.md` (SVD-based low-rank approximation)
+- If used for attention mechanism design -> `spectral-attention` (design pattern layer)
+- If spectral concentration bounds are needed -> `../probability/concentration-inequality.md` (concentration inequalities for random matrix spectra)
+
+## Extensible Directions
+- SVD variants (truncated / randomized SVD): fast decomposition for large-scale matrices
+- CUR decomposition: interpretable matrix approximation via column/row sampling
+- Nystrom method: low-rank approximation of kernel matrices
+- Spectral graph theory (Laplacian eigenvalues): graph Laplacian eigenvalues and graph structure analysis
+- Random matrix theory: spectral distributions of large random matrices

package/knowledge-base/matrix-analysis/spectral-decomposition.md

@@ -44,5 +44,18 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/matrix-analysis.md（Ch 1 特征值与相似、§2.4-2.5 Schur 三角化与正规矩阵、§4.2 Courant-Fischer）
+- 蒸馏稿：../../references/books/matrix-analysis.md（Ch 1 特征值与相似、§2.4-2.5 Schur 三角化与正规矩阵、§4.2 Courant-Fischer）
 - 原书：Horn & Johnson, *Matrix Analysis* 2nd Ed., Chapter 1 (Eigenvalues, Eigenvectors, Similarity) + Chapter 2 (Unitary Similarity §2.4-2.5)
+
+
+## 路由扩展
+- 若需要截断近似 → `low-rank-approximation.md`（基于 SVD 的低秩近似）
+- 若用于注意力机制设计 → `spectral-attention`（设计模式层）
+- 若需要谱的集中界 → `../probability/concentration-inequality.md`（随机矩阵谱的集中不等式）
+
+## 可扩展方向
+- SVD 变体（truncated / randomized SVD）：大规模矩阵的快速分解
+- CUR 分解：基于列/行采样的可解释矩阵近似
+- Nystrom 方法：核矩阵的低秩近似
+- 谱图理论（Laplacian eigenvalues）：图拉普拉斯特征值与图结构分析
+- 随机矩阵理论：大维随机矩阵的谱分布

package/knowledge-base/optimization/constrained-optimization.en.md

@@ -13,7 +13,7 @@ Minimizes an objective $f(x)$ over a constraint set $\mathcal{C} = \{x : g_i(x)
   - $\lambda_i g_i(x) = 0$ (complementary slackness)
 - Projected gradient method: $x_{k+1} = \text{proj}_{\mathcal{C}}(x_k - \alpha \nabla f(x_k))$
 - Penalty function method: $\min_x f(x) + \frac{\rho}{2}\sum [\max(0, g_i(x))]^2 + \frac{\rho}{2}\sum h_j(x)^2$
-- Augmented Lagrangian: $\mathcal{L}_\rho(x,\lambda) = f(x) + \sum \lambda_i g_i(x) + \frac{\rho}{2}\sum g_i(x)^2$
+- Augmented Lagrangian (inequality constraints $g_i(x) \leq 0$): $\mathcal{L}_\rho(x,\lambda) = f(x) + \frac{\rho}{2}\sum_i \left[\max\!\left(0,\; \frac{\lambda_i}{\rho} + g_i(x)\right)^2 - \left(\frac{\lambda_i}{\rho}\right)^2\right]$ (strictly feasible constraints where $g_i(x) < 0$ and $\lambda_i/\rho + g_i(x) \leq 0$ are not penalized; for equality constraints this reduces to $\mathcal{L}_\rho = f(x) + \sum \nu_j h_j(x) + \frac{\rho}{2}\sum h_j(x)^2$)
 - Armijo line search (constrained version): $\alpha$ satisfies $f(\text{proj}_\mathcal{C}(x - \alpha \nabla f)) \leq f(x) - \sigma \alpha \|\nabla f\|^2$
 
 ## Applicable Problems
@@ -27,7 +27,7 @@ Minimizes an objective $f(x)$ over a constraint set $\mathcal{C} = \{x : g_i(x)
 ## AI Design Translation
 
 - **Weight clipping (WGAN)**: $W \leftarrow \text{clamp}(W, -c, c)$ is projection onto an $\ell_\infty$-box constraint. Implemented as `torch.clamp(W, -c, c)`, an elementwise operation with zero additional computation. Simple but coarse, less refined than spectral normalization.
-- **Spectral normalization**: Constrains $\sigma_{\max}(W) \leq 1$, i.e., projects onto the operator norm ball. Uses power iteration to estimate $u \leftarrow W^T W u / \|W^T W u\|$, then normalizes $W \leftarrow W / \sigma_{\max}$. Two matvec + norm operations per step; built into PyTorch as `torch.nn.utils.spectral_norm`.
+- **Spectral normalization**: A common engineering way to enforce $\sigma_{\max}(W) \leq 1$ is the scaling reparameterization $W \leftarrow W / \max(1,\sigma_{\max})$, with $\sigma_{\max}$ estimated by power iteration. This is not the Frobenius-nearest projection onto the operator-norm ball; the exact projection clips singular values individually. About two matvec + norm operations per step; built into PyTorch as `torch.nn.utils.spectral_norm`.
 - **Projected gradient for $\ell_2$-ball constraints**: $\text{proj}(w) = w \cdot \min(1, R/\|w\|_2)$, implemented as `w * min(1, R / w.norm())`, one norm + elementwise operation, $O(d)$. Used in trust regions and adversarial robustness $\epsilon$-ball constraints.
 - **Augmented Lagrangian for RLHF/PPO**: $\mathcal{L} = -\mathbb{E}[r] + \lambda(\text{KL}(\pi\|\pi_{\text{ref}}) - \epsilon) + \frac{\rho}{2}(\text{KL} - \epsilon)^2$. The inner loop optimizes $\pi$ via PPO; the outer loop updates $\lambda \leftarrow \lambda + \rho(\text{KL} - \epsilon)$. KL computation involves softmax + elementwise log-ratio, GPU-friendly.
 - **Penalty method for sparsity/low-rank constraints**: $\mathcal{L}_{\text{penalty}} = \mathcal{L}_{\text{task}} + \rho \sum_i \max(0, \|w_i\|_1 - \tau)^2$ constrains per-layer sparsity to not exceed $\tau$. The penalty term is elementwise + reduce, differentiable and GPU-friendly. $\rho$ increasing schedule: $\rho \leftarrow \beta \rho$ ($\beta > 1$), doubling every several steps.
@@ -49,5 +49,19 @@ Minimizes an objective $f(x)$ over a constraint set $\mathcal{C} = \{x : g_i(x)
 
 ## Further References
 
-- Distilled notes: references/books/optimization-ml.md (Ch 20-21 Equality/Inequality Constraints and KKT, Ch 24 Constrained Algorithms Section 24.3 Projected Gradient, Section 24.5 Augmented Lagrangian, Section 24.6 Penalty Methods)
+- Distilled notes: ../../references/books/optimization-ml.en.md (Ch 20-21 Equality/Inequality Constraints and KKT, Ch 24 Constrained Algorithms Section 24.3 Projected Gradient, Section 24.5 Augmented Lagrangian, Section 24.6 Penalty Methods)
 - Original text: Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 20-21 (Equality & Inequality Constraints) + Chapter 24 (Algorithms for Constrained Optimization Section 24.2-24.6)
+
+
+## Routing Extensions
+- If a dual perspective is needed -> `lagrangian-duality.en.md` (Lagrangian duality theory)
+- If constraints are on a manifold -> `riemannian-optimization.en.md` (Riemannian optimization for manifold constraints)
+- If penalty term design is needed -> `constraint-penalty` (design pattern layer for constraint penalties)
+
+## Extensible Directions
+- Penalty / barrier methods: converting constraints to penalty terms
+- Sequential quadratic programming (SQP): second-order methods for constrained optimization
+- Active set methods: identifying the active constraint set
+- Constraint qualification: LICQ, MFCQ and other regularity conditions
+- Exact penalty: exact solutions with finite penalty parameters
+- Augmented Lagrangian methods: combining duality and penalty functions

package/knowledge-base/optimization/constrained-optimization.md

@@ -13,7 +13,7 @@
   - $\lambda_i g_i(x) = 0$（互补松弛）
 - 投影梯度法：$x_{k+1} = \text{proj}_{\mathcal{C}}(x_k - \alpha \nabla f(x_k))$
 - 惩罚函数法：$\min_x f(x) + \frac{\rho}{2}\sum [\max(0, g_i(x))]^2 + \frac{\rho}{2}\sum h_j(x)^2$
-- 增广 Lagrangian：$\mathcal{L}_\rho(x,\lambda) = f(x) + \sum \lambda_i g_i(x) + \frac{\rho}{2}\sum g_i(x)^2$
+- 增广 Lagrangian（不等式约束 $g_i(x) \leq 0$）：$\mathcal{L}_\rho(x,\lambda) = f(x) + \frac{\rho}{2}\sum_i \left[\max\!\left(0,\; \frac{\lambda_i}{\rho} + g_i(x)\right)^2 - \left(\frac{\lambda_i}{\rho}\right)^2\right]$（严格可行约束 $g_i(x) < 0$ 且 $\lambda_i/\rho + g_i(x) \leq 0$ 时不受惩罚；等式约束退化为 $\mathcal{L}_\rho = f(x) + \sum \nu_j h_j(x) + \frac{\rho}{2}\sum h_j(x)^2$）
 - Armijo 线搜索（约束版）：$\alpha$ 满足 $f(\text{proj}_\mathcal{C}(x - \alpha \nabla f)) \leq f(x) - \sigma \alpha \|\nabla f\|^2$
 
 ## 适用问题
@@ -27,7 +27,7 @@
 ## AI 设计翻译
 
 - **Weight Clipping (WGAN)**：$W \leftarrow \text{clamp}(W, -c, c)$ 即投影到 $\ell_\infty$-box 约束。实现为 `torch.clamp(W, -c, c)`，elementwise 操作，零额外计算。简单但粗糙，不如谱归一化精细。
-- **谱归一化 (Spectral Normalization)**：约束 $\sigma_{\max}(W) \leq 1$，即投影到算子范数球。用 power iteration 估 $u \leftarrow W^T W u / \|W^T W u\|$，归一化 $W \leftarrow W / \sigma_{\max}$。每步 2 次 matvec + norm，PyTorch 内置 `torch.nn.utils.spectral_norm`。
+- **谱归一化 (Spectral Normalization)**：约束 $\sigma_{\max}(W) \leq 1$ 的常用工程做法是缩放重参数化 $W \leftarrow W / \max(1,\sigma_{\max})$，其中 $\sigma_{\max}$ 用 power iteration 估计。注意这不是 Frobenius 范数下到算子范数球的最近投影；真正最近投影需对奇异值逐个裁剪。每步约 2 次 matvec + norm，PyTorch 内置 `torch.nn.utils.spectral_norm`。
 - **投影梯度做 $\ell_2$-ball 约束**：$\text{proj}(w) = w \cdot \min(1, R/\|w\|_2)$，实现为 `w * min(1, R / w.norm())`，一次 norm + elementwise，$O(d)$。用于 trust region、对抗鲁棒的 $\epsilon$-ball 约束。
 - **增广 Lagrangian 做 RLHF/PPO**：$\mathcal{L} = -\mathbb{E}[r] + \lambda(\text{KL}(\pi\|\pi_{\text{ref}}) - \epsilon) + \frac{\rho}{2}(\text{KL} - \epsilon)^2$。内层对 $\pi$ 用 PPO 优化，外层 $\lambda \leftarrow \lambda + \rho(\text{KL} - \epsilon)$。KL 计算是 softmax + elementwise log-ratio，GPU 友好。
 - **惩罚法做稀疏/低秩约束**：$\mathcal{L}_{\text{penalty}} = \mathcal{L}_{\text{task}} + \rho \sum_i \max(0, \|w_i\|_1 - \tau)^2$ 约束每层稀疏度不超 $\tau$。惩罚项是 elementwise + reduce，可微且 GPU 友好。$\rho$ 递增策略：$\rho \leftarrow \beta \rho$（$\beta > 1$），每若干步加倍。
@@ -49,5 +49,19 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/optimization-ml.md（Ch 20-21 等式/不等式约束与 KKT、Ch 24 约束算法 §24.3 投影梯度 §24.5 增广 Lagrange §24.6 惩罚法）
+- 蒸馏稿：../../references/books/optimization-ml.md（Ch 20-21 等式/不等式约束与 KKT、Ch 24 约束算法 §24.3 投影梯度 §24.5 增广 Lagrange §24.6 惩罚法）
 - 原书：Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 20-21 (Equality & Inequality Constraints) + Chapter 24 (Algorithms for Constrained Optimization §24.2-24.6)
+
+
+## 路由扩展
+- 若需要从对偶视角分析 → `lagrangian-duality.md`（Lagrange 对偶理论）
+- 若约束在流形上 → `riemannian-optimization.md`（流形约束的黎曼优化）
+- 若需要惩罚项设计 → `constraint-penalty`（设计模式层的约束惩罚）
+
+## 可扩展方向
+- 罚函数/障碍函数（penalty / barrier methods）：将约束转化为惩罚项
+- 序列二次规划（SQP）：约束优化的二阶方法
+- 有效集方法（active set methods）：识别活跃约束集
+- 约束规范（constraint qualification）：LICQ, MFCQ 等正则条件
+- 精确罚函数（exact penalty）：有限罚参数下的精确解
+- 增广 Lagrangian 方法（augmented Lagrangian methods）：结合对偶与罚函数

package/knowledge-base/optimization/convex-optimization.en.md

@@ -2,7 +2,7 @@
 
 ## Minimal Definition
 
-Optimization problems that minimize a convex function over a convex set. Convexity guarantees that a local optimum is also a global optimum, and the first-order condition ($\nabla f = 0$) is both necessary and sufficient. Convex problems are the only class in optimization theory that guarantees global optimality with efficient solvability.
+Optimization problems that minimize a convex function over a convex set. Convexity guarantees that a local optimum is also a global optimum. For **unconstrained, differentiable** convex optimization, the first-order condition ($\nabla f(x^*) = 0$) is both necessary and sufficient; for **constrained** convex optimization ($\min f(x)$ s.t. $x \in C$), the KKT conditions are necessary and sufficient under Slater's condition; for **non-smooth** convex optimization, the optimality condition is $0 \in \partial f(x^*) + N_C(x^*)$, where $N_C$ is the normal cone of the constraint set. Convex problems are the only class in optimization theory that guarantees global optimality with efficient solvability.
 
 ## Core Formulas
 
@@ -46,5 +46,18 @@ Optimization problems that minimize a convex function over a convex set. Convexi
 
 ## Further References
 
-- Distilled notes: references/books/optimization-ml.md (Ch 22 Convex Optimization, Section 22.2 Convex Functions, Section 22.3 Convex Problems, Section 22.4 SDP/LMI)
+- Distilled notes: ../../references/books/optimization-ml.en.md (Ch 22 Convex Optimization, Section 22.2 Convex Functions, Section 22.3 Convex Problems, Section 22.4 SDP/LMI)
 - Original text: Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 22 (Convex Optimization Problems Section 22.1-22.4)
+
+
+## Routing Extensions
+- If the problem has constraints -> `constrained-optimization.en.md` (constrained convex optimization methods)
+- If the objective is non-smooth -> `proximal-method.en.md` (proximal methods for non-smooth parts)
+
+## Extensible Directions
+- Self-concordant functions: convergence guarantees for Newton's method
+- Interior point methods: polynomial-time methods for large-scale convex optimization
+- First-order method convergence rates: optimal convergence rates for gradient descent
+- Accelerated methods (Nesterov acceleration): Nesterov momentum and optimal first-order methods
+- Online convex optimization: regret analysis for sequential decision-making
+- Bandit convex optimization: convex optimization with zeroth-order information

package/knowledge-base/optimization/convex-optimization.md

@@ -2,7 +2,7 @@
 
 ## 最小定义
 
-在凸集上最小化凸函数的优化问题。凸性保证局部最优即全局最优，且一阶条件（$\nabla f = 0$）是充分必要的。凸问题是优化理论中唯一能保证全局最优且可高效求解的类别。
+在凸集上最小化凸函数的优化问题。凸性保证局部最优即全局最优。对于**无约束可微**凸优化，一阶条件（$\nabla f(x^*) = 0$）是充分必要的；对于**约束**凸优化（$\min f(x)$ s.t. $x \in C$），KKT 条件在 Slater 条件满足时是充分必要的；对于**非光滑**凸优化，最优性条件为 $0 \in \partial f(x^*) + N_C(x^*)$，其中 $N_C$ 为约束集的法锥。凸问题是优化理论中唯一能保证全局最优且可高效求解的类别。
 
 ## 核心公式
 
@@ -46,5 +46,18 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/optimization-ml.md（Ch 22 Convex Optimization、§22.2 凸函数、§22.3 凸问题、§22.4 SDP/LMI）
+- 蒸馏稿：../../references/books/optimization-ml.md（Ch 22 Convex Optimization、§22.2 凸函数、§22.3 凸问题、§22.4 SDP/LMI）
 - 原书：Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 22 (Convex Optimization Problems §22.1-22.4)
+
+
+## 路由扩展
+- 若问题有约束 → `constrained-optimization.md`（约束凸优化方法）
+- 若目标不可微 → `proximal-method.md`（近端方法处理非光滑部分）
+
+## 可扩展方向
+- 自协调函数（self-concordant functions）：Newton 法的收敛保证
+- 内点法（interior point methods）：大规模凸优化的多项式时间方法
+- 一阶方法收敛率（first-order convergence rates）：梯度下降的最优收敛速率
+- 加速方法（Nesterov acceleration）：Nesterov 动量加速与最优一阶方法
+- 在线凸优化（online convex optimization）：序列决策的 regret 分析
+- Bandit 凸优化（bandit convex optimization）：零阶信息下的凸优化

package/knowledge-base/optimization/index.en.md

@@ -0,0 +1,50 @@
+# Optimization Activation Index
+
+## Domain Signals
+Activate this domain direction when the problem involves:
+- Constrained optimization: objective function must satisfy equality or inequality constraints
+- Duality gap: need to analyze the relationship between primal and dual problems
+- Saddle point problems: min-max optimization or game-theoretic structure
+- Non-convex landscape: geometric structure analysis of loss functions
+- Manifold constraints: parameters need to be optimized on a manifold
+- Splitting operators: objective decomposes into multiple components for combined optimization
+
+## Core Anchors
+- `lagrangian-duality.en.md` — Lagrangian duality theory
+- `convex-optimization.en.md` — Convex optimization fundamentals
+- `constrained-optimization.en.md` — Constrained optimization methods
+- `riemannian-optimization.en.md` — Riemannian optimization
+- `proximal-method.en.md` — Proximal methods
+
+## Extended Concepts
+When core anchors are insufficient, the following concepts may need temporary activation:
+- ADMM: alternating direction method of multipliers, suitable for splitting structures
+- mirror descent: mirror descent, suitable for non-Euclidean geometry
+- Frank-Wolfe: conditional gradient method, suitable for sparsity constraints
+- stochastic optimization (SGD / SAdam convergence): convergence theory for stochastic optimization
+- second-order methods (Newton / quasi-Newton): second-order and quasi-Newton methods
+- trust region: trust region methods
+- line search: line search strategies and convergence guarantees
+- gradient clipping theory: theoretical analysis of gradient clipping
+- sharpness-aware minimization (SAM): sharpness-aware minimization
+- neural tangent kernel optimization landscape: optimization landscape under NTK
+- implicit regularization: implicit regularization effects
+- bilevel optimization: bilevel optimization and hyperparameter optimization
+- minimax optimization: minimax optimization theory
+- meta-learning optimization: optimization framework for meta-learning
+
+## Reference Book Directions
+- `../../references/books/optimization-ml.en.md`: comprehensive coverage of ML optimization, including convex optimization, stochastic methods, and second-order methods
+
+## AI Translation Directions
+- lagrangian duality → primal-dual training / adversarial loss / constrained generation
+- convex optimization → convex regularizers / proximal updates / mirror descent optimizer
+- constrained optimization → projected gradient / penalty loss / barrier methods in training
+- riemannian optimization → manifold-constrained parameters / natural gradient / geodesic update
+- proximal method → sparse regularization / ISTA/FISTA layers / proximal neural networks
+
+## Temporary Activation Rules
+When the problem requires mathematics not in the core anchors:
+1. First check whether extended concepts contain a match
+2. If yes, generate a temporary knowledge card based on the lens
+3. If no, enter the Knowledge Gap Protocol

package/knowledge-base/optimization/index.md

@@ -0,0 +1,50 @@
+# 最优化 激活索引 / Optimization Activation Index
+
+## 领域信号
+当问题涉及以下信号时，激活本领域方向：
+- 约束优化：目标函数需要满足等式或不等式约束
+- 对偶间隙：需要分析原始问题与对偶问题的关系
+- 鞍点问题：min-max 优化或博弈论结构
+- 非凸 landscape：损失函数的几何结构分析
+- 流形约束：参数需要在流形上优化
+- 分裂算子：目标函数可分解为多个部分的组合优化
+
+## 核心锚点
+- `lagrangian-duality.md` — Lagrange 对偶理论
+- `convex-optimization.md` — 凸优化基础
+- `constrained-optimization.md` — 约束优化方法
+- `riemannian-optimization.md` — 黎曼优化
+- `proximal-method.md` — 近端方法
+
+## 扩展概念
+当核心锚点不够时，以下概念可能需要临时激活：
+- ADMM：交替方向乘子法，适用于分裂结构
+- mirror descent：镜像下降，适用于非欧几何
+- Frank-Wolfe：条件梯度法，适用于稀疏约束
+- stochastic optimization（SGD / SAdam convergence）：随机优化的收敛理论
+- second-order methods（Newton / quasi-Newton）：二阶与拟牛顿方法
+- trust region：信赖域方法
+- line search：线搜索策略与收敛保证
+- gradient clipping theory：梯度裁剪的理论分析
+- sharpness-aware minimization (SAM)：尖锐度感知最小化
+- neural tangent kernel optimization landscape：NTK 下的优化景观
+- implicit regularization：隐式正则化效应
+- bilevel optimization：双层优化与超参数优化
+- minimax optimization：极小极大优化理论
+- meta-learning optimization：元学习的优化框架
+
+## 参考书方向
+- `../../references/books/optimization-ml.md`：机器学习优化的全面覆盖，包括凸优化、随机方法和二阶方法
+
+## AI 翻译方向
+- lagrangian duality → primal-dual training / adversarial loss / constrained generation
+- convex optimization → convex regularizers / proximal updates / mirror descent optimizer
+- constrained optimization → projected gradient / penalty loss / barrier methods in training
+- riemannian optimization → manifold-constrained parameters / natural gradient / geodesic update
+- proximal method → sparse regularization / ISTA/FISTA layers / proximal neural networks
+
+## 临时激活规则
+当问题需要的数学不在核心锚点中时：
+1. 先检查扩展概念中是否有匹配
+2. 若有，根据透镜生成临时知识卡
+3. 若无，进入 Knowledge Gap Protocol

package/knowledge-base/optimization/lagrangian-duality.en.md

@@ -24,9 +24,9 @@ Transforms a constrained optimization problem (primal) into a maximization probl
 
 ## AI Design Translation
 
-- **Minimax framework for adversarial training**: $L(\theta, \phi) = \mathbb{E}[\log D_\phi(x)] + \mathbb{E}[\log(1 - D_\phi(G_\theta(z)))]$. $G$ and $D$ alternate between gradient ascent and descent; the core operations are forward + backward passes (matmul chains). Gradient penalty / spectral norm constrains the Lipschitz constant of $D$ to ensure the existence of a saddle point.
+- **Minimax framework for adversarial training**: $L(\theta, \phi) = \mathbb{E}[\log D_\phi(x)] + \mathbb{E}[\log(1 - D_\phi(G_\theta(z)))]$. $G$ and $D$ alternate between gradient ascent and descent; the core operations are forward + backward passes (matmul chains). Gradient penalty / spectral norm can control the Lipschitz constant of $D$ and improve training stability, but it does not guarantee that a non-convex minimax problem has, or converges to, a benign saddle point.
 - **SVM dual + kernel trick**: Primal $\min_w \frac{1}{2}\|w\|^2 + C\sum\xi_i$ converted to dual $\max_\alpha \sum\alpha_i - \frac{1}{2}\alpha^T(K \circ yy^T)\alpha$. The kernel Gram matrix $K_{ij} = k(x_i, x_j)$ is computed via matmul (linear kernel) or elementwise operations (RBF kernel). Dual variables $\alpha \in \mathbb{R}^n$ are solved using SMO or gradient projection.
-- **Augmented Lagrangian constrained training**: $\mathcal{L}_{\text{AL}} = f(x) + \sum \lambda_i g_i(x) + \frac{\rho}{2}\sum g_i(x)^2$. The inner loop optimizes $x$ via SGD; the outer loop updates $\lambda$ via gradient ascent: $\lambda \leftarrow [\lambda + \rho g(x)]_+$. The $\rho$ term ensures inner-loop convexity, enabling convergence even for non-convex original problems.
+- **Augmented Lagrangian constrained training**: $\mathcal{L}_{\text{AL}} = f(x) + \sum \lambda_i g_i(x) + \frac{\rho}{2}\sum g_i(x)^2$. The inner loop optimizes $x$ via SGD; the outer loop updates $\lambda$ via gradient ascent: $\lambda \leftarrow [\lambda + \rho g(x)]_+$. The $\rho$ term improves the concavity of the dual function (making the dual problem easier to solve), but does NOT guarantee global convergence for non-convex original problems. Convergence depends on: (a) constraint qualifications (e.g., LICQ/MFCQ), (b) second-order sufficient conditions, (c) $\rho$ being large enough to satisfy local convexity conditions.
 - **Dual decomposition**: $\min \sum_k f_k(x_k)$ s.t. $\sum x_k \leq b$ decomposes into independent subproblems $\min_{x_k} f_k(x_k) + \lambda^T x_k$, with the master problem $\max_\lambda g(\lambda)$ solved via subgradient ascent. Naturally parallelizable, suitable for distributed training.
 - **Dual perspective on the information bottleneck**: $\min I(Z;X) - \beta I(Z;Y)$ can be formulated as constrained optimization, where $\beta$ is the dual variable. The variational IB relaxes this via the ELBO, reducing to standard VAE training (reparameterization trick + SGD).
 
@@ -39,12 +39,26 @@ Transforms a constrained optimization problem (primal) into a maximization probl
 
 ## Risks and Failure Conditions
 
-- **Duality gap for non-convex problems**: Strong duality is only guaranteed for convex problems + Slater's condition. In non-convex neural network training, $d^* < p^*$ is common, and the dual solution does not yield a primal-feasible solution. Solution: augmented Lagrangian (eliminates the gap when $\rho$ is sufficiently large) or SQP.
+- **Duality gap for non-convex problems**: Strong duality is only guaranteed for convex problems + Slater's condition. In non-convex neural network training, $d^* < p^*$ is common, and the dual solution does not yield a primal-feasible solution. Solution: for convex problems, augmented Lagrangian (with the exact penalty property when $\rho$ is sufficiently large, which can eliminate the duality gap); for non-convex problems, the duality gap may persist regardless of $\rho$, and the augmented Lagrangian only guarantees local convergence to KKT points -- alternatively, use SQP.
 - **Dual variable oscillation**: Improper step sizes for gradient ascent on $\lambda$ can cause dual variable oscillation and primal infeasibility. Solution: use adaptive step sizes (Adam updates for $\lambda$) or an increasing $\rho$ schedule in the augmented Lagrangian.
 - **Numerical determination of complementary slackness**: $\lambda_i g_i(x) = 0$ can only be satisfied to $\sim 10^{-6}$ in floating point; strict complementary slackness is unattainable. This affects SVM support vector identification; a threshold must be set.
 - **Mode collapse in minimax training**: The non-convex-non-concave game in GAN $\min\max$ leads to mode collapse or training instability. Additional regularization such as gradient penalty (WGAN-GP) or spectral normalization is required.
 
 ## Further References
 
-- Distilled notes: references/books/optimization-ml.md (Ch 23 Lagrangian Duality, Section 23.5 Strong Duality, Section 23.6.3 Slater's Condition)
+- Distilled notes: ../../references/books/optimization-ml.en.md (Ch 23 Lagrangian Duality, Section 23.5 Strong Duality, Section 23.6.3 Slater's Condition)
 - Original text: Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 23 (Lagrangian Duality Section 23.1-23.6) + Chapter 17 (LP Duality)
+
+
+## Routing Extensions
+- If starting from the primal problem -> `constrained-optimization.en.md` (primal constrained optimization)
+- If strong duality conditions are needed -> `convex-optimization.en.md` (strong duality theorem for convex problems)
+- If the dual form of IB objective is involved -> `../probability/information-bottleneck.md` (variational dual of information bottleneck)
+
+## Extensible Directions
+- Augmented Lagrangian: penalty-enhanced Lagrangian methods
+- Saddle point theory: existence and computation of Lagrangian saddle points
+- KKT condition regularity: constraint qualifications and KKT necessary conditions
+- Conic duality: duality theory for conic programming
+- Fenchel duality: convex conjugates and Fenchel-Rockafellar duality
+- Minimax duality: min-max theorems and duality gaps

package/knowledge-base/optimization/lagrangian-duality.md

@@ -24,9 +24,9 @@
 
 ## AI 设计翻译
 
-- **对抗训练的 minimax 框架**：$L(\theta, \phi) = \mathbb{E}[\log D_\phi(x)] + \mathbb{E}[\log(1 - D_\phi(G_\theta(z)))]$。$G$ 和 $D$ 交替梯度上升/下降，核心操作是前向 + 反向传播（matmul 链）。梯度惩罚 / spectral norm 约束 $D$ 的 Lipschitz 保证 minimax 有鞍点。
+- **对抗训练的 minimax 框架**：$L(\theta, \phi) = \mathbb{E}[\log D_\phi(x)] + \mathbb{E}[\log(1 - D_\phi(G_\theta(z)))]$。$G$ 和 $D$ 交替梯度上升/下降，核心操作是前向 + 反向传播（matmul 链）。梯度惩罚 / spectral norm 可控制 $D$ 的 Lipschitz 常数并改善训练稳定性，但不保证非凸 minimax 一定存在或收敛到良性鞍点。
 - **SVM 对偶 + 核技巧**：原始 $\min_w \frac{1}{2}\|w\|^2 + C\sum\xi_i$ 转对偶 $\max_\alpha \sum\alpha_i - \frac{1}{2}\alpha^T(K \circ yy^T)\alpha$。核 Gram $K_{ij} = k(x_i, x_j)$ 是 matmul（线性核）或 elementwise（RBF 核）。对偶变量 $\alpha \in \mathbb{R}^n$，用 SMO 或梯度投影求解。
-- **增广 Lagrangian 约束训练**：$\mathcal{L}_{\text{AL}} = f(x) + \sum \lambda_i g_i(x) + \frac{\rho}{2}\sum g_i(x)^2$。内层对 $x$ 用 SGD 优化，外层对 $\lambda$ 梯度上升更新：$\lambda \leftarrow [\lambda + \rho g(x)]_+$。$\rho$ 项保证内层凸性，即使原问题非凸也收敛。
+- **增广 Lagrangian 约束训练**：$\mathcal{L}_{\text{AL}} = f(x) + \sum \lambda_i g_i(x) + \frac{\rho}{2}\sum g_i(x)^2$。内层对 $x$ 用 SGD 优化，外层对 $\lambda$ 梯度上升更新：$\lambda \leftarrow [\lambda + \rho g(x)]_+$。$\rho$ 项改善对偶函数的凹性（使对偶问题更容易求解），但对非凸原问题不保证全局收敛。收敛性依赖于：(a) 约束规格（如 LICQ/MFCQ），(b) 二阶充分条件，(c) $\rho$ 足够大以满足局部凸性条件。
 - **对偶分解 (Dual Decomposition)**：$\min \sum_k f_k(x_k)$ s.t. $\sum x_k \leq b$ 分解为各子问题 $\min_{x_k} f_k(x_k) + \lambda^T x_k$ 独立求解，主问题 $\max_\lambda g(\lambda)$ 用次梯度上升。天然可并行，适合分布式训练。
 - **信息瓶颈的对偶视角**：$\min I(Z;X) - \beta I(Z;Y)$ 可写为带约束优化，$\beta$ 是对偶变量。Variational IB 用 ELBO 松弛后变成标准 VAE 训练（reparameterization trick + SGD）。
 
@@ -39,12 +39,26 @@
 
 ## 风险与失效条件
 
-- **非凸问题的对偶间隙**：强对偶仅对凸问题 + Slater 条件保证。非凸神经网络训练中 $d^* < p^*$ 常见，对偶解不给出原始可行解。解决：增广 Lagrangian（$\rho$ 足够大时消除间隙）或 SQP。
+- **非凸问题的对偶间隙**：强对偶仅对凸问题 + Slater 条件保证。非凸神经网络训练中 $d^* < p^*$ 常见，对偶解不给出原始可行解。解决：对凸问题，增广 Lagrangian（$\rho$ 足够大时具有 exact penalty 性质，可消除对偶间隙）；对非凸问题，对偶间隙可能持续存在，增广 Lagrangian 只能保证局部收敛到 KKT 点，或改用 SQP。
 - **对偶变量振荡**：$\lambda$ 的梯度上升步长不当会导致对偶变量振荡、原始不可行。解决：使用自适应步长（Adam 更新 $\lambda$）或增广 Lagrangian 的 $\rho$ 递增策略。
 - **互补松弛的数值判定**：$\lambda_i g_i(x) = 0$ 在浮点下只能达到 $\sim 10^{-6}$，严格互补松弛不可得。影响 SVM 支持向量识别等，需设阈值。
 - **Minimax 训练的 mode collapse**：GAN 中 $\min\max$ 的非凸-非凹博弈导致模式坍缩或训练不稳定。需梯度惩罚（WGAN-GP）、谱归一化等额外正则化。
 
 ## 深入参考
 
-- 蒸馏稿：references/books/optimization-ml.md（Ch 23 Lagrangian Duality、§23.5 强对偶、§23.6.3 Slater 条件）
+- 蒸馏稿：../../references/books/optimization-ml.md（Ch 23 Lagrangian Duality、§23.5 强对偶、§23.6.3 Slater 条件）
 - 原书：Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 23 (Lagrangian Duality §23.1-23.6) + Chapter 17 (LP Duality)
+
+
+## 路由扩展
+- 若需要从原始问题出发 → `constrained-optimization.md`（原始约束优化）
+- 若需要强对偶条件 → `convex-optimization.md`（凸问题的强对偶定理）
+- 若涉及 IB 目标函数的对偶形式 → `../probability/information-bottleneck.md`（信息瓶颈的变分对偶）
+
+## 可扩展方向
+- 增广 Lagrangian（augmented Lagrangian）：罚项增强的 Lagrangian 方法
+- 鞍点理论（saddle point theory）：Lagrangian 鞍点的存在性与求解
+- KKT 条件正则性（KKT regularity）：约束规范与 KKT 必要条件
+- 锥对偶（conic duality）：锥规划的对偶理论
+- Fenchel 对偶（Fenchel duality）：凸共轭与 Fenchel-Rockafellar 对偶
+- 极小极大对偶（minimax duality）：min-max 定理与对偶间隙

package/knowledge-base/optimization/proximal-method.en.md

@@ -48,5 +48,19 @@ For non-smooth or non-differentiable objective functions $f(x) = g(x) + h(x)$ ($
 
 ## Further References
 
-- Distilled notes: references/books/optimization-ml.md (Ch 8 Gradient Methods Section 8.3 Convergence Analysis, Ch 11 Quasi-Newton Section 11.5 BFGS, Ch 24 Constrained Algorithms Section 24.5 Augmented Lagrangian)
+- Distilled notes: ../../references/books/optimization-ml.en.md (Ch 8 Gradient Methods Section 8.3 Convergence Analysis, Ch 11 Quasi-Newton Section 11.5 BFGS, Ch 24 Constrained Algorithms Section 24.5 Augmented Lagrangian)
 - Original text: Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 24 (Constrained Algorithms Section 24.5 Augmented Lagrangian) + Parikh & Boyd, *Proximal Algorithms*, Foundations and Trends in Optimization, 2014
+
+
+## Routing Extensions
+- If the smooth part dominates -> `convex-optimization.en.md` (convex optimization for smooth parts)
+- If the proximal operator corresponds to a constraint -> `constrained-optimization.en.md` (equivalence of indicator functions and constraints)
+- If used for variational loss regularization -> `variational-loss` (design pattern layer for proximal regularization)
+
+## Extensible Directions
+- Proximal gradient (ISTA / FISTA): fast proximal methods for L1 regularization
+- Douglas-Rachford splitting: optimizing sums of two non-smooth functions
+- Primal-dual hybrid gradient (PDHG): first-order methods for saddle point problems
+- Alternating minimization: block-wise optimization for block-separable objectives
+- Block coordinate descent: block update strategies for high-dimensional problems
+- Proximal neural networks: embedding proximal operators into network architecture

package/knowledge-base/optimization/proximal-method.md

@@ -48,5 +48,19 @@
 
 ## 深入参考
 
-- 蒸馏稿：references/books/optimization-ml.md（Ch 8 梯度法 §8.3 收敛分析、Ch 11 拟牛顿 §11.5 BFGS、Ch 24 约束算法 §24.5 增广 Lagrange）
+- 蒸馏稿：../../references/books/optimization-ml.md（Ch 8 梯度法 §8.3 收敛分析、Ch 11 拟牛顿 §11.5 BFGS、Ch 24 约束算法 §24.5 增广 Lagrange）
 - 原书：Chong, Lu, Zak, *An Introduction to Optimization* 5th Ed., Chapter 24 (Constrained Algorithms §24.5 Augmented Lagrangian) + Parikh & Boyd, *Proximal Algorithms*, Foundations and Trends in Optimization, 2014
+
+
+## 路由扩展
+- 若可微部分主导 → `convex-optimization.md`（光滑部分的凸优化方法）
+- 若近端算子对应约束 → `constrained-optimization.md`（指示函数与约束的等价）
+- 若用于变分损失的正则化 → `variational-loss`（设计模式层的近端正则化）
+
+## 可扩展方向
+- 近端梯度（ISTA / FISTA）：L1 正则化的快速近端方法
+- Douglas-Rachford 分裂：两个非光滑函数之和的优化
+- 原始-对偶混合梯度（PDHG）：鞍点问题的一阶方法
+- 交替最小化（alternating minimization）：块可分目标的分块优化
+- 块坐标下降（block coordinate descent）：高维问题的分块更新策略
+- 近端神经网络（proximal neural networks）：将近端算子嵌入网络结构