math-skill 2.1.0 → 3.0.0

package/skills/optimization/original-texts.en.md

@@ -1,167 +0,0 @@
-# Mathematical Sources and Classic Texts
-
-## Euler-Lagrange Variational Equation & Brachistochrone (1696)
-
-> In 1696, Johann Bernoulli posed the brachistochrone problem: under gravity alone, along which curve does a particle descend from A to B in the shortest time? The answer is the cycloid.
-
-**Core equation**: For the functional J[y] = ∫_{a}^{b} F(x, y, y') dx, the extremizing function satisfies
-
-> ∂F/∂y - d/dx(∂F/∂y') = 0 (Euler-Lagrange equation)
-
-For the brachistochrone, F = √(1 + y'²) / √(2g(y₀ - y)). Substituting into the Euler-Lagrange equation yields the parametric equations of the cycloid: x = a(θ - sin θ), y = a(1 - cos θ).
-
-**Key idea**: The calculus of variations reduces "optimization over an entire class of functions" to solving a differential equation — this is the origin of infinite-dimensional optimization. Euler (1744) systematized the calculus of variations; Lagrange (1788) introduced the δ notation and the method of multipliers, both in the same lineage. The calculus of variations later evolved into optimal control theory (Pontryagin's Maximum Principle, 1961), with wide applications in spacecraft trajectory design, robotic path planning, and many other fields.
-
-## Fermat & Weierstrass Conditions for Unconstrained Optimization
-
-> Fermat (1636): A necessary condition for a differentiable function f to attain a local extremum at x* is ∇f(x*) = 0.
-> Weierstrass (19th c.): If f is twice differentiable and ∇f(x*) = 0, then the Hessian H(x*) is positive definite ⇔ x* is a strict local minimum; H(x*) is negative definite ⇔ x* is a strict local maximum.
-
-For convex functions, ∇f(x*) = 0 is not only necessary but also sufficient — and x* is the global minimum.
-
-**Key idea**: First-order conditions locate stationary points; second-order conditions determine the nature of the extremum — the most fundamental theorem of unconstrained optimization. The condition ∇f = 0 is the convergence target of virtually all iterative optimization algorithms (gradient descent, Newton's method).
-
-## Lagrange Multipliers (1788)
-
-> Finding the extremum of f(x) subject to g(x) = 0 is equivalent to finding the unconstrained extremum of L(x,λ) = f(x) - λg(x).
-
-**Key idea**: Transform a constrained optimization problem into an unconstrained one. At the optimal solution, the gradient of the objective function is parallel to the gradient of the constraint function: ∇f(x*) = λ∇g(x*).
-
-## Pareto Optimality (Vilfredo Pareto, ~1906)
-
-> In multi-objective optimization, a solution x is Pareto optimal if there does not exist y such that f_i(y) ≤ f_i(x) for all i with at least one f_j(y) < f_j(x) (strict improvement).
-
-In *Manuale di economia politica* (1906), Pareto introduced this concept into economics: the efficiency frontier of resource allocation. The Pareto frontier {f(x) : x is Pareto optimal} characterizes all trade-off solutions that cannot be simultaneously improved. Mathematically, for objectives f₁,...,fₘ, the Pareto frontier is {y ∈ ℝᵐ : y = (f₁(x),...,fₘ(x)), x Pareto optimal} — it is a (generally non-convex) surface in objective space.
-
-> Weighted scalarization: Convert multi-objective min f₁,...,fₘ into single-objective min Σw_i f_i (w_i > 0); each weight combination corresponds to a point on the Pareto frontier. This is a standard method for computing the Pareto frontier, but it cannot enumerate all Pareto-optimal solutions on non-convex frontiers.
-
-**Key idea**: Multi-objective optimization has no single "optimal solution" but rather a set of Pareto-optimal solutions among which trade-offs must be made. This is ubiquitous in economics (efficiency vs. equity), engineering (performance vs. cost), and machine learning (accuracy vs. complexity).
-
-## Game Theory & Optimization (von Neumann, 1928; Nash, 1950)
-
-> Nash equilibrium: A strategy profile (s₁*, ..., sₙ*) such that for each player i, s_i* is the best response given the other players' strategies — i.e., no player has an incentive to unilaterally deviate.
-
-Von Neumann (1928) proved the minimax theorem for zero-sum games: max_{x} min_{y} f(x,y) = min_{y} max_{x} f(x,y), which is essentially a precursor of linear programming duality. Nash (1950) generalized the equilibrium concept to non-zero-sum games; a Nash equilibrium is an " mutually optimal" optimization — each player's optimization problem takes the other players' strategies as constraints.
-
-> Existence of Nash equilibrium: Nash (1950) used the Brouwer fixed-point theorem to prove that every finite game has at least one mixed-strategy Nash equilibrium. This reveals a deep connection between game theory and topology.
-
-**Key idea**: Game theory extends optimization from "single-agent decision-making" to "multi-agent interactive decision-making" — optimality is no longer with respect to nature, but the best response to other rational agents.
-
-## Karush-Kuhn-Tucker Conditions (1939/1951)
-
-Necessary conditions for inequality-constrained optimization:
-
-> For min f(x) s.t. g_i(x) ≤ 0, h_j(x) = 0, the optimal solution x* satisfies:
-> - Stationarity: ∇f(x*) + Σμ_i∇g_i(x*) + Σλ_j∇h_j(x*) = 0
-> - Primal feasibility: g_i(x*) ≤ 0, h_j(x*) = 0
-> - Complementary slackness: μ_i · g_i(x*) = 0, μ_i ≥ 0
-
-Karush (1939, master's thesis) first derived these conditions; Kuhn & Tucker (1951) independently rediscovered and widely disseminated them.
-
-**Key idea**: At the optimal solution, either a constraint is inactive (μ_i = 0) or it is tight (g_i(x*) = 0). Complementary slackness is the cornerstone of duality theory — it ties the KKT conditions to a zero duality gap. When a constraint qualification holds (e.g., LICQ: the gradients of active constraints are linearly independent), the KKT conditions are necessary for optimality.
-
-## Linear Programming Duality (Farkas, 1902; Dantzig-von Neumann, 1947)
-
-> Farkas' Lemma (1902): Ax = b, x ≥ 0 has a solution ⟺ Aᵀy ≥ 0, bᵀy < 0 has no solution — this is the geometric foundation of LP duality.
-
-> Strong duality theorem: If the primal LP min cᵀx s.t. Ax = b, x ≥ 0 has a feasible solution, then the optimal values of the primal and the dual max bᵀy s.t. Aᵀy ≤ c are equal. The components of the dual variable y are called shadow prices — they measure the "marginal value" of each constraint.
-
-**Key idea**: Every LP has a "mirror" problem — the primal views the problem in terms of cost, the dual in terms of value. Zero duality gap is a profound fact: the minimum cost of resources equals the maximum revenue valued at shadow prices.
-
-## Simplex Method (Dantzig, 1947)
-
-The fundamental algorithm for linear programming. It searches for the optimal solution by moving along vertices of the feasible region.
-
-> Klee-Minty cube (1973): There exist LP instances for which the simplex method visits all 2ⁿ vertices — exponential worst-case complexity.
-
-> Interior-Point Methods: Karmarkar (1984) proposed a polynomial-time LP algorithm that converges along the central path through the interior of the feasible region, avoiding vertex enumeration. Modern interior-point methods (e.g., Mehrotra predictor-corrector) perform excellently in both practice and theory.
-
-**Key idea**: The optimal solution of a linear program always lies at a vertex of the feasible region. The simplex method exploits this structure by moving along edges; interior-point methods approach from the inside — the two are complementary.
-
-## Duality Theory
-
-> Weak duality: For any optimization problem, the dual optimal value d* ≤ primal optimal value p* (duality gap = p* - d* ≥ 0).
-
-> Strong duality (Slater's condition): If a convex optimization problem has a strictly feasible point (Slater point: g_i(x) < 0 strictly), then d* = p*, and the duality gap is zero.
-
-> Duality provides lower bounds (the primal optimal value ≥ the dual value), which is a critical tool in branch-and-bound, cutting-plane, and related algorithms.
-
-**Key idea**: Duality connects "minimizing cost" with "maximizing value" — two perspectives on the same problem. Slater's condition guarantees "no information loss" in convex optimization.
-
-## Gradient Descent (Cauchy, 1847; Robbins-Monro, 1951)
-
-> Cauchy (1847) proposed the method of steepest descent: x_{k+1} = x_k - α_k ∇f(x_k), iterating along the negative gradient direction with step size α_k. For L-smooth convex functions (Lipschitz-continuous gradient), with step size α = 1/L, f(x_k) - f(x*) ≤ L‖x₀ - x*‖² / (2k) — a convergence rate of O(1/k).
-
-> Newton's Method: x_{k+1} = x_k - H(x_k)⁻¹ ∇f(x_k), using second-order information, achieves quadratic convergence O(‖x_k - x*‖²) near the optimum.
-
-> Stochastic Gradient Descent (SGD): Robbins & Monro (1951) proposed stochastic approximation, replacing the true gradient with a noisy estimate g_k ≈ ∇f(x_k). The conditions Σα_k = ∞, Σα_k² < ∞ guarantee convergence. SGD converges in expectation at the cost of variance — minibatching is a trade-off between variance and efficiency.
-
-**Key idea**: Gradient descent is the most naive form of optimization — "go where it is steepest." SGD extends this idea to large-data settings: one need not examine all the data, only a random sample suffices to approximate the direction. The training of deep neural networks relies almost entirely on SGD and its variants (Adam, AdaGrad, etc.).
-
-## Integer Programming & NP-Hardness (Cook, 1971; Karp, 1972)
-
-> Integer Linear Programming (ILP): min cᵀx s.t. Ax ≤ b, x ∈ ℤⁿ — variables take integer values. Even 0-1 ILP (x ∈ {0,1}ⁿ) is NP-hard.
-
-> Cook (1971) proved that SAT is NP-complete (the Cook-Levin theorem); Karp (1972) listed 21 NP-complete problems, including integer programming, the knapsack problem, and the traveling salesman problem. ILP is NP-hard — no polynomial-time exact algorithm exists (under the assumption P ≠ NP).
-
-> Convex relaxation: Relaxing x ∈ ℤⁿ to x ∈ ℝⁿ yields an LP whose optimal value provides a lower bound for the ILP — this is precisely the core mechanism of branch-and-bound.
-
-**Key idea**: Discrete optimization is fundamentally harder than continuous optimization. The seemingly minor constraint of "integrality" pushes a problem from polynomial-time solvability to NP-hardness. Branch-and-bound and cutting-plane methods are the principal exact methods; heuristics are more commonly used in practice.
-
-## Convex Optimization (Boyd & Vandenberghe, 2004)
-
-> Convex optimization: min f(x) s.t. g_i(x) ≤ 0 (convex), h_j(x) = 0 (affine), x ∈ C (convex set). When f is convex, a local minimum is a global minimum.
-
-> Key properties: (1) Local minimum = global minimum; (2) The feasible region is a convex set; (3) Strong duality holds under Slater's condition; (4) Interior-point methods can solve it in polynomial time.
-
-Important subclasses encompassed by convex optimization:
-- LP (Linear Programming): f linear, constraints linear
-- QP (Quadratic Programming): f quadratic, constraints linear
-- SOCP (Second-Order Cone Programming): constraints include second-order cones ‖Ax + b‖ ≤ cᵀx + d
-- SDP (Semidefinite Programming): constraints include positive semidefinite matrix conditions X ≥ 0
-- GP (Geometric Programming): convex after logarithmic transformation
-
-Boyd & Vandenberghe's *Convex Optimization* (2004) unified all of the above problem classes and became the standard textbook and engineering reference for modern optimization. Convex optimization marks the boundary of "efficiently solvable optimization" — convexity guarantees algorithmic convergence and global optimality of solutions.
-
-**Key idea**: Convex optimization is the "sweet spot" of optimization theory — rich enough to encompass a vast number of practical problems, yet structured enough to guarantee tractability. Non-convex problems are often approximated via convex relaxation.
-
-## Bellman's Principle of Optimality (1957)
-
-> "An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision."
-
-> Recursive equation: V(s) = max_a { R(s,a) + γ · V(s') }, where s' is the result of the state transition.
-
-**Key idea**: The core of dynamic programming — substructures of an optimal solution are themselves optimal. The Bellman equation decomposes multi-stage decision-making into single-stage subproblems and is the theoretical foundation of reinforcement learning.
-
-## The Philosophical Significance of Optimization
-
-The core of optimization thinking is **making the best choice under constraints** — this is not only a mathematical question but also a question about life. Most decisions in life can be formulated as optimization problems:
-
-- **Objective function**: What do you want to maximize? (Happiness? Achievement? Freedom?)
-- **Constraints**: What limitations do you face? (Time? Money? Ability?)
-- **Feasible region**: What choices are available to you under the constraints?
-- **Optimal solution**: Among all choices, which one maximizes the objective?
-- **Dual perspective**: Viewing the problem in terms of "cost" or "value" yields the same answer — this is the philosophy of strong duality.
-- **Convex vs. non-convex**: Convex optimization has a unique global optimum; non-convex optimization has multiple local optima — the predicament of life lies precisely in the fact that the world is non-convex.
-
-## Timeline of Optimization
-
-| Year | Event |
-|------|-------|
-| 1636 | Fermat proposes the necessary condition ∇f = 0 for unconstrained extrema |
-| 1696 | Johann Bernoulli poses the brachistochrone problem; the seeds of the calculus of variations |
-| 1744 | Euler systematizes the calculus of variations |
-| 1788 | Lagrange publishes the method of multipliers in *Mécanique analytique* |
-| 1902 | Farkas' Lemma — the geometric foundation of LP duality |
-| ~1906 | Pareto introduces the efficiency frontier concept into economics |
-| 1928 | Von Neumann proves the minimax theorem (zero-sum games) |
-| 1939 | Karush derives the KKT conditions (master's thesis) |
-| 1947 | Dantzig invents the simplex method; von Neumann establishes LP duality |
-| 1950 | Nash proves the existence of equilibria in non-zero-sum games |
-| 1951 | Kuhn & Tucker publish the KKT conditions; Robbins-Monro propose stochastic approximation |
-| 1957 | Bellman publishes dynamic programming and the principle of optimality |
-| 1971 | Cook proves NP-completeness (the Cook-Levin theorem) |
-| 1972 | Karp lists 21 NP-complete problems |
-| 1973 | Klee-Minty prove exponential worst-case complexity of the simplex method |
-| 1984 | Karmarkar proposes the polynomial-time interior-point method |
-| 2004 | Boyd & Vandenberghe publish *Convex Optimization* |

package/skills/optimization/original-texts.md

@@ -1,168 +0,0 @@
-# 数学出处与经典文献 / Mathematical Sources and Classic Texts
-
-## Euler-Lagrange 变分方程与最速降线 / Euler-Lagrange Variational Equation & Brachistochrone (1696)
-
-> Johann Bernoulli 于 1696 年提出最速降线问题：在重力作用下，质点从 A 到 B 沿哪条曲线下降最快？答案是旋轮线（cycloid）。
-
-**核心方程**：对泛函 J[y] = ∫_{a}^{b} F(x, y, y') dx，极值函数满足
-
-> ∂F/∂y - d/dx(∂F/∂y') = 0 （Euler-Lagrange 方程）
-
-最速降线问题的 F = √(1 + y'²) / √(2g(y₀ - y))，代入 Euler-Lagrange 方程后解出旋轮线参数方程：x = a(θ - sin θ), y = a(1 - cos θ)。
-
-**思想**：变分法将"在整个函数类中寻优"化为微分方程求解——这是无穷维优化的起源。Euler（1744）系统化了变分法，Lagrange（1788）引入了 δ 符号与乘数法，两者一脉相承。变分法后来发展为最优控制理论（Pontryagin 最大值原理，1961），在航天轨道设计、机器人路径规划等领域广泛应用。
-
-## Fermat 与 Weierstrass 极值条件 / Fermat & Weierstrass Conditions for Unconstrained Optimization
-
-> Fermat（1636）：可微函数 f 在 x* 处取得局部极值的必要条件是 ∇f(x*) = 0。
-> Weierstrass（19th c.）：若 f 二阶可微且 ∇f(x*) = 0，则 Hessian H(x*) 正定 ⇔ x* 是严格局部极小；H(x*) 负定 ⇔ x* 是严格局部极大。
-
-对凸函数，∇f(x*) = 0 不仅是必要条件，也是充分条件——且 x* 是全局极小。
-
-**思想**：一阶条件找驻点，二阶条件判极值性质——无约束优化的最基本定理。∇f = 0 是几乎所有迭代优化算法（梯度下降、牛顿法）的收敛目标。
-
-## Lagrange 乘数法 / Lagrange Multipliers (1788)
-
-> 在约束 g(x) = 0 下求 f(x) 的极值，等价于求 L(x,λ) = f(x) - λg(x) 的无条件极值。
-
-**思想**：将约束优化问题转化为无约束问题。在最优解处，目标函数的梯度与约束函数的梯度平行：∇f(x*) = λ∇g(x*)。
-
-## Pareto 最优性 / Pareto Optimality (Vilfredo Pareto, ~1906)
-
-> 在多目标优化中，解 x 是 Pareto 最优的，若不存在 y 使得 f_i(y) ≤ f_i(x) ∀i 且至少一个 f_j(y) < f_j(x)（严格改善）。
-
-Pareto 在《Manuale di economia politica》(1906) 中将此概念引入经济学：资源配置的效率前沿。Pareto 前沿 Pareto frontier {f(x) : x is Pareto optimal} 刻画了所有不可同时改善的权衡解。数学上，对目标 f₁,...,fₘ 的 Pareto 前沿是 {y ∈ ℝᵐ : y = (f₁(x),...,fₘ(x)), x Pareto optimal}——它是目标空间中的一个（通常非凸的）曲面。
-
-> 加权标量化：将多目标 min f₁,...,fₘ 转化为单目标 min Σw_i f_i（w_i > 0），每个权重组合对应 Pareto 前沿上的一个点。这是求解 Pareto 前沿的常用方法，但对非凸前沿不能遍历所有 Pareto 最优解。
-
-**思想**：多目标优化没有唯一的"最优解"，而有一组 Pareto 最优解，需要在它们之间做权衡。这在经济学（效率与公平）、工程（性能与成本）、机器学习（精度与复杂度）中无处不在。
-
-## 博弈论与优化 / Game Theory & Optimization (von Neumann, 1928; Nash, 1950)
-
-> Nash 均衡：策略组 (s₁*, ..., sₙ*) 使得每个玩家 i 在其他玩家策略固定时，s_i* 是其最优响应——即无人有单方面偏离的动机。
-
-von Neumann（1928）证明了零和博弈的 minimax 定理：max_{x} min_{y} f(x,y) = min_{y} max_{x} f(x,y)，这本质上是线性规划对偶性的先声。Nash（1950）将均衡推广到非零和博弈，Nash 均衡是"互为最优"的优化——每个玩家的优化问题以他人策略为约束。
-
-> Nash 均衡的存在性：Nash（1950）用 Brouwer 不动点定理证明了每个有限博弈至少存在一个混合策略 Nash 均衡。这揭示了博弈论与拓扑学的深层联系。
-
-**思想**：博弈论将优化从"单人决策"推广到"多人交互决策"——最优不再是对自然的最优，而是对其他理性主体的最优响应。
-
-## KKT 条件 / Karush-Kuhn-Tucker Conditions (1939/1951)
-
-不等式约束优化的必要条件：
-
-> 对于 min f(x) s.t. g_i(x) ≤ 0, h_j(x) = 0，最优解 x* 满足：
-> - 平稳性：∇f(x*) + Σμ_i∇g_i(x*) + Σλ_j∇h_j(x*) = 0
-> - 可行性：g_i(x*) ≤ 0, h_j(x*) = 0
-> - 互补松弛性：μ_i · g_i(x*) = 0, μ_i ≥ 0
-
-Karush（1939，硕士论文）最早给出此条件；Kuhn & Tucker（1951）独立重新发现并广泛传播。
-
-**思想**：最优解处，要么约束不起作用（μ_i = 0），要么约束是紧的（g_i(x*) = 0）。互补松弛性是对偶理论的基石——它将 KKT 条件与对偶间隙为零紧密联系。当约束限定条件 constraint qualification 成立时（如 LICQ：活跃约束梯度线性无关），KKT 条件是最优性的必要条件。
-
-## 线性规划对偶 / Linear Programming Duality (Farkas, 1902; Dantzig-von Neumann, 1947)
-
-> Farkas 引理（1902）：Ax = b, x ≥ 0 有解 ⟺ Aᵀy ≥ 0, bᵀy < 0 无解——这是 LP 对偶的几何根基。
-
-> 强对偶定理：若原 LP min cᵀx s.t. Ax = b, x ≥ 0 有可行解，则 max bᵀy s.t. Aᵀy ≤ c 的最优值相等。对偶变量 y 的分量称为影子价格 shadow price——它衡量每个约束的"边际价值"。
-
-**思想**：每个 LP 都有一个"镜像"问题——原问题看成本，对偶问题看价值。对偶间隙为零是深刻的事实：资源的最小成本等于影子价格的最大收益。
-
-## Dantzig 单纯形法 / Simplex Method (1947)
-
-线性规划的基本算法。在可行域的顶点上搜索最优解。
-
-> Klee-Minty 立方体（1973）：存在 LP 实例使得单纯形法遍历所有 2ⁿ 个顶点——最坏情况指数复杂度。
-
-> 内点法 / Interior-Point Methods：Karmarkar（1984）提出多项式时间 LP 算法，在可行域内部沿中心路径收敛，避免顶点遍历。现代内点法（如 Mehrotra predictor-corrector）在实践和理论上均表现优异。
-
-**思想**：线性规划的最优解一定在可行域的顶点上。单纯形法利用此结构沿边移动；内点法从内部逼近，两者互补。
-
-## 对偶理论 / Duality Theory
-
-> 弱对偶：对任何优化问题，对偶问题的最优值 d* ≤ 原问题最优值 p*（对偶间隙 gap = p* - d* ≥ 0）。
-
-> 强对偶（Slater 条件）：若凸优化问题存在严格可行点（Slater point：g_i(x) < 0 严格成立），则 d* = p*，对偶间隙为零。
-
-> 对偶提供了下界（原问题的最优值 ≥ 对偶值），这在分支定界、 Cutting-plane 等算法中是关键工具。
-
-**思想**：对偶将"最小化成本"与"最大化价值"联系起来——两个视角看同一问题。Slater 条件保证了凸优化的"无信息损失"。
-
-## 梯度下降 / Gradient Descent (Cauchy, 1847; Robbins-Monro, 1951)
-
-> Cauchy（1847）提出最速下降法：x_{k+1} = x_k - α_k ∇f(x_k)，沿负梯度方向迭代，α_k 为步长。对 L-平滑凸函数（Lipschitz 连续梯度），步长 α = 1/L 时 f(x_k) - f(x*) ≤ L‖x₀ - x*‖² / (2k)——线性收敛率 O(1/k)。
-
-> 牛顿法 Newton's Method：x_{k+1} = x_k - H(x_k)⁻¹ ∇f(x_k)，利用二阶信息，在最优解附近达到二次收敛 O(‖x_k - x*‖²)。
-
-> 随机梯度下降 SGD：Robbins & Monro（1951）提出随机逼近，用噪声梯度 g_k ≈ ∇f(x_k) 替代真梯度。条件 Σα_k = ∞, Σα_k² < ∞ 保证收敛。SGD 在期望意义上收敛，代价是方差——minibatch 是方差与效率的折中。
-
-**思想**：梯度下降是最朴素的优化——"哪里最陡就往哪里走"。SGD 将此思想扩展到大数据场景：不需要看全部数据，只需随机采样一小部分即可逼近方向。深度学习的训练几乎全部依赖 SGD 及其变体（Adam, AdaGrad 等）。
-
-## 整数规划与 NP-困难 / Integer Programming & NP-Hardness (Cook, 1971; Karp, 1972)
-
-> 整数线性规划 ILP：min cᵀx s.t. Ax ≤ b, x ∈ ℤⁿ——变量取整数值。即使 0-1 ILP（x ∈ {0,1}ⁿ）也是 NP困难的。
-
-> Cook（1971）证明 SAT 是 NP 完全的（Cook-Levin 定理）；Karp（1972）列出 21 个 NP 完全问题，包含整数规划、背包问题、旅行商问题等。ILP 是 NP困难的——不存在（在 P ≠ NP 假设下）多项式时间精确算法。
-
-> 凸松弛：将 x ∈ ℤⁿ 松弛为 x ∈ ℝⁿ 得到 LP，其最优值给出 ILP 的下界——这正是分支定界法 branch-and-bound 的核心机制。
-
-**思想**：离散优化比连续优化本质上更难。"整数"这一看似微小的约束，将问题从多项式可解推向 NP 困难。分支定界 branch-and-bound、割平面 cutting-plane 是主要精确方法；启发法在实践中更常用。
-
-## 凸优化 / Convex Optimization (Boyd & Vandenberghe, 2004)
-
-> 凸优化：min f(x) s.t. g_i(x) ≤ 0 (凸), h_j(x) = 0 (线性), x ∈ C (凸集)。f 为凸函数时局部极小即全局极小。
-
-> 关键性质：(1) 局部极小 = 全局极小；(2) 可行域是凸集；(3) Slater 条件下强对偶成立；(4) 内点法在多项式时间内可解。
-
-凸优化涵盖的重要子类：
-- LP（线性规划）：f 线性，约束线性
-- QP（二次规划）：f 二次，约束线性
-- SOCP（二阶锥规划）：约束含二阶锥 ‖Ax + b‖ ≤ cᵀx + d
-- SDP（半定规划）：约束含矩阵半正定条件 X ≥ 0
-- GP（几何规划）：对数空间下化为凸优化
-
-Boyd & Vandenberghe《Convex Optimization》(2004) 统一了上述各类问题，成为现代优化的标准教材与工程参考。凸优化是"可高效求解的优化"的边界——凸性保证了算法的收敛性与解的全局性。
-
-**思想**：凸优化是优化理论的"甜蜜区"——足够丰富以涵盖大量实际问题，又足够结构化以保证可解性。非凸问题往往通过凸松弛 convex relaxation 来近似。
-
-## Bellman 最优性原理 / Bellman's Principle of Optimality (1957)
-
-> "一个最优策略具有这样的性质：无论初始状态和初始决策如何，剩余决策必须构成关于由初始决策产生的状态的最优策略。"
-> "An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision."
-
-> 递推方程：V(s) = max_a { R(s,a) + γ · V(s') }，其中 s' 为状态转移结果。
-
-**思想**：动态规划的核心——最优解的子结构也是最优的。Bellman 方程将多阶段决策分解为单阶段子问题，是强化学习的理论根基。
-
-## 优化的哲学意义
-
-优化思想的核心是**在约束下做最好的选择**——这不仅是数学问题，也是人生问题。生活中大多数决策都可以表述为优化问题：
-
-- **目标函数**：你想最大化什么？（幸福？成就？自由？）
-- **约束条件**：你面临什么限制？（时间？金钱？能力？）
-- **可行域**：在约束下你有什么选择？
-- **最优解**：在所有选择中，哪个使目标最大化？
-- **对偶视角**：从"成本"看和从"价值"看，答案是一致的——这正是强对偶的哲学。
-- **凸与非凸**：凸优化有唯一全局最优；非凸优化有多个局部最优——人生的困境恰在于世界是非凸的。
-
-## 优化理论历史年表 / Timeline of Optimization
-
-| 年份 | 事件 |
-|------|------|
-| 1636 | Fermat 提出无约束极值的必要条件 ∇f = 0 |
-| 1696 | Johann Bernoulli 提出最速降线问题，变分法萌芽 |
-| 1744 | Euler 系统化变分法 |
-| 1788 | Lagrange 发表乘数法，融入《分析力学》 |
-| 1902 | Farkas 引理——LP 对偶的几何基础 |
-| ~1906 | Pareto 将效率前沿概念引入经济学 |
-| 1928 | von Neumann 证明 minimax 定理（零和博弈） |
-| 1939 | Karush 给出 KKT 条件（硕士论文） |
-| 1947 | Dantzig 发明单纯形法；von Neumann 建立 LP 对偶 |
-| 1950 | Nash 证明非零和博弈均衡存在性 |
-| 1951 | Kuhn & Tucker 发表 KKT 条件；Robbins-Monro 提出随机逼近 |
-| 1957 | Bellman 发表动态规划与最优性原理 |
-| 1971 | Cook 证明 NP 完全性（Cook-Levin 定理） |
-| 1972 | Karp 列出 21 个 NP 完全问题 |
-| 1973 | Klee-Minty 证明单纯形法最坏情况指数复杂度 |
-| 1984 | Karmarkar 提出多项式时间内点法 |
-| 2004 | Boyd & Vandenberghe 出版《Convex Optimization》

package/skills/probability-statistics/SKILL.en.md

@@ -1,146 +0,0 @@
----
-name: probability-statistics
-description: |
-  Trigger when a problem involves quantifying uncertainty, probability distributions, Bayesian inference, hypothesis testing, regression modeling, experimental design, causal effect estimation; or designing probabilistic mechanisms for random algorithms/sampling/quantization/training dynamics.
----
-
-# Probability & Statistics
-
-> "Uncertainty is quantifiable -- probability is not just a mathematical formula, but a way to understand the random world. Quantifying uncertainty, extracting patterns from data, and making rational inferences and decisions."
->
-> -- Probability Theory, Mathematical Statistics
-
-## Core Principle
-
-**The world is full of uncertainty. Rational decision-making is not the pursuit of certainty, but the pursuit of expected-value-optimal choices under uncertainty -- probability is quantified belief, and data is quantified evidence.**
-
-> **Mathematical Formalization**
->
-> A probability space $(\Omega, \mathcal{F}, P)$ is defined by the Kolmogorov axioms: (1) **Non-negativity** $P(A) \ge 0$; (2) **Normalization** $P(\Omega) = 1$; (3) **Countable additivity**: for a countable family of mutually disjoint events $A_i$, $P(\cup_i A_i) = \sum_i P(A_i)$. From these follow the complement rule $P(A^c)=1-P(A)$ and the law of total probability $P(B)=\sum_i P(B|A_i)P(A_i)$.
->
-> **Bayesian updating**: $P(\theta|D) = P(D|\theta)\cdot P(\theta)/P(D)$; prior -> likelihood -> posterior; the posterior becomes the new prior upon encountering new data.
->
-> **Expectation and variance**: $E[X] = \sum x\cdot P(x)$ (discrete) or $\int x\cdot f(x)\,dx$ (continuous); $\text{Var}(X) = E[(X-\mu)^2] = E[X^2] - (E[X])^2$.
->
-> **Law of Large Numbers / CLT**: As $n \to \infty$, the sample frequency $f/n \to P(A)$ and the sample mean $\bar{X} \to \mu$ (convergence in probability); $\sqrt{n}\cdot(\bar{X} - \mu)/\sigma \xrightarrow{d} N(0,1)$, so the superposition of many independent small factors tends toward normality.
->
-> **Bias-variance trade-off**: Total error $= \text{Bias}^2 + \text{Variance} + \sigma^2$ (irreducible noise).
-
-## GPU-Friendliness (Cross-Cutting Check)
-
-When probabilistic methods are used for **random algorithm / sampling / training dynamics design**, they must pass the eight-dimensional gate in `../../references/gpu-friendly-math.md`:
-
-- **Batch sampling / Monte Carlo**: i.i.d. sampling is naturally parallelizable and GEMM-friendly (reparameterization trick, Gumbel-Softmax); viable in low precision, but RNG state and cross-device consistency must be managed.
-- **MLE / log-likelihood**: Element-wise likelihood is fusible and batchable (friendly); softmax cross-entropy should use logsumexp for numerical stability.
-- **Exact Bayesian integration / marginalization**: In high dimensions, the normalization constant $Z=P(D)$ and marginalization of $P(\theta|D)\propto P(D|\theta)P(\theta)$ admit no closed form -- adapt to **MCMC (HMC/NUTS) / variational inference ELBO** (see `../../references/books/optimization-ml.md`).
-- **Large posterior tables / covariances**: Incomputable -- use mean-field / low-rank factorized approximations (see `../../references/books/matrix-analysis.md`).
-- **Anti-patterns**: Sample-sequential MCMC, exact inference requiring global normalization, "beautiful but incomputable" high-dimensional integrals.
-
-Eight-dimensional minimum assessment (formal terms): **Tensorization** -- whether samples, particles, and distribution parameters can be batched; **GEMM-mappability** -- whether likelihood, regression, and covariance computations fall into matrix operations; **Complexity** -- number of sampling steps, integration dimension, posterior table size; **Memory & KV-Cache** -- whether particles, covariances, and sufficient statistics can be low-rank / streaming; **Low-precision stability** -- logsumexp, probability normalization, tail probabilities; **Parallelism & communication** -- RNG and cross-device reductions; **Sparse structure** -- whether the graphical model / covariance is structured; **Operator fusion** -- whether sampling, scoring, and normalization can be fused.
-
-> Cross-reference `../../references/books/optimization-ml.md` (variational / SAA / stochastic optimization) and `../../references/books/matrix-analysis.md` (low-rank covariance).
-
-## When NOT to Use
-
-- **Deterministic problems** (the answer is known or obtainable by deductive reasoning) -- probabilistic tools are unnecessary.
-- **Extremely small sample sizes ($n < 5$) that cannot be increased** -- one should report the data itself rather than draw inferences.
-- **Purely qualitative descriptive needs** -- data analysis is not required.
-- **Causal mechanisms are already well-established and require no probabilistic modeling** -- use deterministic causal chains directly.
-- **Data is entirely missing or severely corrupted** -- no meaningful statistical operations can be performed.
-
-## When to Use
-
-- Reviewing whether the statistical methods in a paper / experiment are correct and whether conclusions are significant.
-- Experimental data analysis, significance testing, effect-size assessment, regression modeling, and prediction.
-- Experimental design and power analysis (randomization, blocking, factorial designs).
-- Causal effect estimation (DAG + back-door criterion / do-calculus).
-- Bayesian inference and prior-posterior updating, goodness-of-fit tests ($\chi^2$/F).
-- **Designing probabilistic mechanisms for random algorithms / sampling / quantization / training dynamics** (Monte Carlo, reparameterization, probabilistic quantization) and evaluating their GPU viability.
-
-## Method
-
-### Step 1: Define the Random Phenomenon
-Identify the **random variable $X$** (the quantity to be measured), the **sample space $\Omega$** (all possible outcomes), the **event space $\mathcal{F}$** ($\sigma$-algebra on $\Omega$: contains $\Omega$, closed under complementation, closed under countable union), and the **probability measure $P$**: $\Omega\to[0,1]$ (satisfying the Kolmogorov axioms). Distinguish between discrete and continuous random variables.
-
-### Step 2: Choose the Probability Model
-Select a distribution based on data type, sample size, and physical mechanism:
-
-| Distribution | Use Case | Key Parameters |
-|---|---|---|
-| Binomial | Number of successes in $N$ independent Bernoulli trials | $n, p$ |
-| Poisson | Count of rare events per unit time/space | $\lambda$ (mean rate) |
-| Normal | Superposition of many independent small factors (CLT) | $\mu, \sigma^2$ |
-| Exponential | Waiting times; memoryless property $P(X>t+s\|X>s)=P(X>t)$ | $\lambda$ (rate) |
-| Bayesian | Prior + likelihood -> posterior; incremental belief updating | Prior parameters, likelihood function |
-| Student's $t$ | Small-sample mean inference; heavier tails than normal | $df = n - 1$ |
-| $\chi^2$ | Goodness-of-fit tests, contingency tables, variance estimation | $df$ |
-| $F$ | Comparing two variances, ANOVA | $df_1, df_2$ |
-| Uniform | Equally likely events, non-informative default | $a, b$ |
-
-Selection principles: discrete vs. continuous; large samples tend toward normal (use $t$ for small samples); physical mechanism (Poisson = rare independent events, Exponential = waiting times).
-
-### Step 3: Collect Data
-Verify that the **sample size** is sufficient (power analysis determines the minimum $n$), check for **sample bias** (selection / survivorship / voluntary response bias), assess **data quality** (measurement error / missing values / outliers), examine the **collection method** (random sampling vs. convenience sampling), and identify **block structure**. Representativeness is more important than volume -- a biased large sample is more dangerous than an unbiased small one.
-
-### Step 4: Statistical Inference
-**Point estimation / MLE**: $\hat{\theta} = \arg\max_\theta L(\theta;x)$, where $L(\theta;x)=f(x|\theta)$; asymptotically normal and asymptotically efficient for large samples. **Confidence intervals** provide an interval estimate at a given confidence level (e.g., 95%); the frequentist interpretation is that 95% of such intervals from repeated sampling contain the true value. **Hypothesis testing**: $H_0$ vs. $H_1$; p-value $= P(\text{data this extreme or more} \mid H_0)$; reject $H_0$ if $p < \alpha$. **Bayesian updating**: $P(\theta|D) = P(D|\theta)\cdot P(\theta)/P(D)$.
-
-### Step 5: Regression Modeling
-**Linear regression**: $Y = \beta_0 + \beta_1 X + \varepsilon$, $\varepsilon \sim N(0,\sigma^2)$; least-squares estimate $\hat{\beta} = (X^TX)^{-1}X^TY$; assumptions include linearity, independence, homoscedasticity, and normality of residuals. **Multiple / logistic regression**: $P(Y=1|X) = 1/(1+e^{-(\beta_0+\sum\beta_i X_i)})$; MLE estimation yields output probabilities. **Model selection**: AIC $= -2\ln L + 2k$ (favors predictive fit), BIC $= -2\ln L + k\cdot\ln n$ (favors parsimony), adjusted $R^2 = 1 - (1-R^2)(n-1)/(n-k-1)$. **Bias-variance trade-off**: Total error $= \text{Bias}^2 + \text{Variance} + \sigma^2$; use cross-validation to select complexity.
-
-### Step 6: Experimental Design
-**Randomization** ensures that treatment and control groups are equal in expectation on all variables (known and unknown). **Blocking** groups similar units together and compares within blocks to reduce confounding variation. **Power analysis**: $\text{Power} = 1 - \beta = P(\text{reject } H_0 \mid H_1 \text{ is true})$; given effect size $\delta$ and $\alpha$, compute the minimum required $n$; low power implies high false-negative rate. **Factorial designs**: $2^k$ designs study $k$ factors and their interactions simultaneously.
-
-### Step 7: Causal Inference
-**Confounding variables**: $Z$ affects both $X$ and $Y$, so $P(Y|X) \neq P(Y|do(X))$; without adjustment, the estimated effect is biased. **DAGs**: Nodes represent variables, arrows represent direct causal effects; identify back-door and mediating paths. **Do-calculus / back-door criterion**: If $S$ blocks all back-door paths from $X$ to $Y$ and contains no descendants of $X$, then $P(Y|do(X)) = \sum_s P(Y|X,S=s)\cdot P(S=s)$. **Randomization** directly blocks all back-door paths and is the gold standard for causal effect estimation.
-
-### Step 8: Interpret Results
-Distinguish **statistical significance** from **practical significance** ($p<0.05$ does not mean the effect is large; with large samples, even tiny differences become "significant"). Report **effect sizes**: Cohen's $d = (\mu_1-\mu_2)/\sigma$, odds ratio $= (P_1/(1-P_1))/(P_2/(1-P_2))$. Remember that **correlation does not imply causation** ($P(Y|X) \neq P(Y|do(X))$) and beware of **multiple comparisons** (for $m$ tests, the expected number of false positives is $m\cdot\alpha$; apply Bonferroni $\alpha_{\text{adj}}=\alpha/m$ or BH-FDR correction).
-
-### Step 9: Quantify Uncertainty
-Provide **confidence intervals** (frequentist: repeated-sampling coverage) or **credible intervals** (Bayesian: the probability that the parameter lies in this interval); state the reliability and statistical power of conclusions; identify sources of residual uncertainty (model assumptions, measurement error, unobserved confounders); report effect-size intervals rather than point estimates alone.
-
-## Common Errors
-
-| Error | Critique | Correct Approach |
-|---|---|---|
-| Treating the p-value as truth | $p=0.05$ does not mean "95% certainty"; it is the probability of observing this data or more extreme data under $H_0$ | Understand $p = P(\text{data or more extreme} \mid H_0)$ |
-| Confusing probability with frequency | Probability is a theoretical quantity $P(A)$; frequency is an empirical quantity $f(A)/n$ | Connect them via the law of large numbers: $f/n \to P$ as $n\to\infty$ |
-| Neglecting prior probabilities | Base-rate neglect: focusing only on new evidence while ignoring background probability; especially dangerous for rare-disease screening | Bayesian reasoning: $P(H\|E) = P(E\|H)\cdot P(H)/P(E)$ |
-| Survivorship bias | Looking only at survivors truncates the sample space | Consider the full sample space $\Omega$, including failures |
-| Small-sample trap | "Patterns" from $n=3$ are unreliable; variance estimates are unstable | Check sample size, compute power, use the $t$-distribution |
-| Confusing correlation with causation | Correlation between two variables does not mean one causes the other | Search for confounders; use DAGs + back-door criterion |
-| Ignoring regression to the mean | After extreme performance, values tend to regress toward $E[X]$ | Use regression analysis rather than intuitive trend judgment |
-| Using the normal distribution instead of $t$ for small samples | For $n<30$, the normal approximation underestimates uncertainty | Use the $t$-distribution with $df=n-1$ for $n<30$ |
-| Confusing AIC with BIC | AIC favors predictive fit; BIC favors parsimony | Use AIC for prediction, BIC for explanation |
-| Inferring causation from observational data | Without randomization, confounding cannot be eliminated; $P(Y\|X) \neq P(Y\|do(X))$ | Apply DAG + back-door criterion adjustment, or restrict claims to association only |
-| Ignoring multiple-comparison corrections | For $m$ tests, the expected number of false positives is $m\cdot\alpha$ | Apply Bonferroni $\alpha/m$ or BH-FDR |
-| Ignoring power and effect size | Low power may fail to detect a real effect; reporting only $p$ without effect size is incomplete | Compute required sample size before the study; report Cohen's $d$ / odds ratio |
-| Exact integration / large posterior tables are incomputable | High-dimensional normalization $Z=P(D)$ and marginalization have no closed form; sample-sequential MCMC is not parallelizable | Use MCMC / variational ELBO / low-rank approximations; pass the GPU eight-dimensional gate |
-
-## Operating Procedure
-
-When this skill is triggered, the output must include:
-
-1. **Random variable definitions**: `[Variable]: [Meaning] [Type: Discrete / Continuous]`, specifying the sample space $\Omega$ and event space $\mathcal{F}$
-2. **Probability model**: `[Distribution]: [Choice] because [Reason]`, noting $df$ or key parameters
-3. **Data assessment**: `[Sample size]: N, [Bias check]: [Result], [Collection method]: [Random / Convenience]`
-4. **Statistical inference**: `[Estimate]: [\hat{\theta}], [Confidence interval]: [a,b], [p-value]: [value], [Method]: [MLE / Bayesian]`
-5. **Regression / experimental design (if applicable)**: `[Model]: [Linear / Logistic] [Coefficients]: [\hat{\beta}] [Criterion]: [AIC / BIC]`; `[Design]: [Randomization / Block / Factorial] [Power]: [Power] [Minimum n]: [n]`
-6. **Causal assessment (if applicable)**: `[DAG]: [Paths], [Confounders]: [List], [Adjustment set]: [S], [Causal effect]: [P(Y|do(X))]`
-7. **Effect size and uncertainty**: Cohen's $d$ / odds ratio with confidence intervals, statistical power, and residual uncertainty
-8. **Bias check**: Systematically review the common-errors table, marking each as "None found" or "Warning: [issue found]"
-9. **[GPU viability]** (if used for random algorithms / sampling / training): Whether inference requires sampling / variational approximation; pass the eight-dimensional gate; label as friendly / amenable to adaptation / anti-pattern, with adaptation recommendations
-10. **Action recommendations**: Explicitly state "Next, I will..."
-
-**Output must not consist of analysis alone without conclusions.**
-
-## Relations to Other Skills
-
-- **Modeling**: Probabilistic models are essential modeling tools -- uncertainty requires a probabilistic framework.
-- **Optimization**: Optimization under uncertainty requires expected-value optimization or robust optimization; stochastic optimization / SAA converts expected objectives to deterministic approximations.
-- **Induction and analogy**: Discovering patterns in data is the probabilistic version of inductive reasoning.
-- **Logic and deduction**: Classical logic is deterministic; probabilistic logic is its generalization to uncertainty.
-- **Causal inference**: $P(Y|X)$ is a statistical association; $P(Y|do(X))$ is a causal effect; causal claims require mechanistic assumptions (DAGs) and cannot be derived from data alone.
-- **Information theory**: Entropy $H(X)=-\sum P(x)\ln P(x)$ quantifies uncertainty; information gain $I(X;Y)=H(X)-H(X|Y)$; the maximum-entropy principle selects the distribution that maximizes entropy subject to constraints.
-- **Modern mathematics activation**: `../../references/books/optimization-ml.md` (stochastic optimization / SAA, variational inference ELBO), `../../references/books/matrix-analysis.md` (low-rank covariance, factorized approximations).