math-skill 2.1.0 → 3.0.1

package/skills/optimization/SKILL.md

@@ -1,135 +0,0 @@
----
-name: optimization
-description: |
-  触发：问题涉及资源分配、取舍、最大化/最小化目标、约束下决策；或需判断凸性、使用拉格朗日/KKT、分析对偶结构；或为算法/算子/训练设计选择优化方法时调用。
-  English: Trigger when a problem involves resource allocation, trade-offs, maximizing/minimizing objectives, decisions under constraints; or needs convexity analysis, Lagrangian/KKT methods, duality structure; or choosing optimization methods for algorithm/operator/training design.
----
-
-> **语言路由**：若用户消息为英文，请读取并遵循同目录下的 `SKILL.en.md`，按其操作规程以英文输出；中文消息则继续使用本文件。
-
-# ⚖️ 优化思想 / Optimization
-
-> "在最一般的约束条件下，寻找目标函数的极值——凸性决定难度，KKT 给出必要条件，对偶揭示结构。"
-> "Under the most general constraints, find extrema of the objective — convexity determines difficulty, KKT gives necessity, duality reveals structure."
->
-> —— 最优化理论与运筹学 / Optimization Theory & Operations Research
-
-## 核心原则 / Core Principle
-
-**任何决策问题都可以表述为优化问题：在约束条件下最大化（或最小化）某个目标。优化的本质不是追求'最好'，而是在约束下追求'可行中的最好'。**
-
-**Any decision problem can be formulated as optimization: maximizing (or minimizing) an objective subject to constraints. The essence is not 'the best' in the abstract, but 'the best among the feasible'.**
-
-优化的三个核心要素：**目标（Objective）**、**约束（Constraints）**、**可行域（Feasible set）**。
-
-> **数学形式化 / Mathematical Formalization**
->
-> 一般优化问题：$\min_{x \in \mathbb{R}^n} f(x) \quad \text{s.t.} \quad g_i(x) \leq 0,\; i=1,\dots,m; \quad h_j(x) = 0,\; j=1,\dots,p$
->
-> 拉格朗日函数：$L(x, \lambda, \mu) = f(x) + \sum_i \lambda_i g_i(x) + \sum_j \mu_j h_j(x)$
->
-> KKT 条件（核心必要条件，Slater 等正则性下）：① 驻点 $\nabla_x L = 0$；② 原始可行 $g_i \le 0, h_j = 0$；③ 对偶可行 $\lambda_i \ge 0$；④ 互补松弛 $\lambda_i g_i = 0$。
->
-> 凸性：若 $f$ 与各 $g_i$ 凸、各 $h_j$ 线性，则问题凸；此时 **KKT 充分**，局部最优 = 全局最优。
-
-## GPU 友好性 / GPU-Friendliness（横切检查）
-
-当优化用于**算法/算子/训练设计**时，求解方法本身必须过 `../../references/gpu-friendly-math.md` 八维门：
-
-- **一阶法（SGD/Adam）**：GEMM 友好、可并行、低精度可行；注意优化器状态精度与分布式通信开销。
-- **二阶/牛顿法**：Hessian 求逆 $O(n^3)$、显存爆炸——典型"美但不可算"→ 改造为 **K-FAC / 低秩 / 对角近似**（见 `../../references/books/optimization-ml.md`、`matrix-analysis.md`）。
-- **约束投影**：投影是否有闭式、可张量化？迭代投影警惕串行依赖。
-- **分布式**：计算/通信能否 overlap；是否需要梯度压缩。
-
-八维最低判定（正式术语）：**张量化**看目标/约束/梯度能否批量；**GEMM 可映射**看主计算是否为矩阵乘、HVP、K-FAC 小矩阵；**复杂度**明确一阶/二阶/组合求解阶；**显存与 KV-Cache**检查优化器状态、Hessian、激活保存；**低精度稳定**检查条件数、阻尼、loss scaling；**并行与通信**检查梯度同步和通信 overlap；**稀疏结构**看预条件/约束是否块结构化；**算子融合**看更新、裁剪、正则能否融合。
-
-> 配合 `../../references/books/optimization-ml.md`（Chong/Lu/Żak）与 `../../references/books/matrix-analysis.md`。
-
-## 不适用场景 / When NOT to Use
-
-- **没有明确评价标准**（不知道什么是"好"）——先定义目标再优化。
-- **纯执行性任务**（如格式化代码）——没有优化空间。
-- **用户已确定方案**——优化已完成。
-- **本质是定性判断而非定量极值**——应先建模再优化。
-
-## 何时使用 / When to Use
-
-- 需要判断问题是否为凸优化以决定求解难度。
-- 为算法/算子/训练设计选择优化方法，并评估其 GPU 可行性。
-- 在约束条件下做可量化目标的理性决策。
-- 实验设计、资源分配、超参/结构搜索的系统优化。
-- 不确定当前策略是否最优，想系统分析（凸性、对偶、灵敏度）。
-
-## 方法流程 / Method
-
-### 第一步：定义目标 / Define the Objective
-明确最大化/最小化什么。关键：单目标还是多目标？是否可量化（否则找代理变量）？静态还是动态？**$f$ 是否凸**（凸则局部=全局，非凸需警惕局部极值）？方向错了，走得越远越偏。
-
-### 第二步：列出约束 / List the Constraints
-区分**硬约束**（物理/预算/deadline）与**软约束**（偏好/质量下限）；数学上分类：不等式 $g_i(x)\le 0$（定义可行域边界）、等式 $h_j(x)=0$（降维）、线性（可行域为凸多面体）vs 非线性（可能非凸）。
-
-### 第三步：类型分类 / Classify the Problem Type
-
-| 类型 | 目标 | 约束 | 核心性质 | 典型方法 |
-|------|------|------|----------|----------|
-| LP 线性规划 | 线性 | 线性不等式 | 最优在顶点 | 单纯形法 |
-| QP 二次规划 | 二次 | 线性 | 正定 QP 为凸 | 内点法 |
-| 凸优化 | 凸 | 凸不等式+线性等式 | 局部=全局 | 梯度下降、内点法 |
-| 非凸优化 | 非凸 | 任意 | 多局部极值 | 全局搜索、模拟退火 |
-| 组合优化 | 离散域 | 任意 | NP-hard 常见 | 分支定界、启发式 |
-| 随机优化 | 含随机项 | 可能含随机 | 期望最优 vs 随机可行 | SAA、鲁棒优化 |
-
-### 第四步：寻找最优解 / Find the Optimal Solution
-- **LP/QP/凸**：利用凸性，梯度类或内点法保证收敛到全局最优。
-- **非凸**：多起点、全局搜索，或松弛为凸近似。
-- **组合**：精确解常 NP-hard——小规模分支定界，大规模启发式/近似。
-- **随机**：样本平均近似（SAA）转化为确定性近似。
-- **信息不足**：满意解（satisficing）即可。
-
-### 第五步：灵敏度分析 / Sensitivity Analysis
-拉格朗日乘子 $\lambda_i^*$ 为第 $i$ 个约束的**影子价格**——约束放松一单位，目标改善约 $\lambda_i^*$。互补松弛：$\lambda_i^*=0$ 为非活跃约束（对最优解无影响）；$\lambda_i^*>0$ 为活跃约束（最优解恰在其边界）。关注：约束/目标微小变化时最优解如何变、哪些约束活跃。
-
-### 第六步：多目标与帕累托 / Multi-Objective & Pareto
-多目标 $f_1,\dots,f_k$ 通常无单一最优解。帕累托最优：不存在使所有目标同时改善的可行解。方法：**加权求和** $\min\sum w_i f_i$（不同权重对应前沿不同点）；**$\epsilon$-约束法** $\min f_1$ s.t. $f_i\le\epsilon_i$（遍历 $\epsilon_i$ 覆盖前沿）。
-
-### 第七步：监控约束变化 / Monitor Constraint Changes
-最优解依赖约束——约束变了就要重新优化。活跃约束的变化影响最大（影子价格高），非活跃约束的微小变化通常不影响最优解。
-
-## 常见错误 / Common Errors
-
-| 错误 / Error | 批评 / Critique | 正确做法 / Correct Approach |
-|-------------|----------------|---------------------------|
-| 没有明确目标就优化 | 方向不确定 | 先精确定义目标 |
-| 忽略隐式约束 | "最优解"实则不可行 | 穷尽检查所有约束 |
-| 陷入局部最优 | 非凸贪心不保证全局 | 验证凸性；非凸用多起点/全局法 |
-| 把最优当唯一 | 最优解可能不唯一 | 检查是否存在多个等价最优解 |
-| 多目标用单目标方法 | 不同目标需 trade-off | 使用帕累托分析 |
-| 未验证凸性 | 非凸误用凸方法 | 先判断凸性再选方法 |
-| 忽略对偶理论 | 对偶问题可能更易解 | 构造对偶，利用强对偶性 |
-| 混淆可行与最优 | 可行不一定最优 | 先验证可行性再验证最优性 |
-| 忽略计算/GPU 复杂度 | 二阶法/组合优化可能不可算 | 评估复杂度，过 GPU 八维门，必要时近似 |
-| 忘记重新优化 | 约束变了未更新 | 定期检查约束变化 |
-
-## 操作规程 / Operating Procedure
-
-当本 skill 被触发时，输出必须包含：
-
-1. **目标函数**：`[目标]: [描述]` + `[凸性]: [凸/非凸/未知]`
-2. **约束清单**：每条标 `[硬/软]` 与 `[不等式/等式]` `[线性/非线性]`
-3. **类型分类**：`[类型]: [LP/QP/凸/非凸/组合/随机]`
-4. **可行域分析**：哪些选项可行？活跃约束是哪些？
-5. **最优解/满意解**：`[策略]: [梯度法/内点法/全局搜索/satisficing/帕累托]`
-6. **灵敏度分析**：关键约束影子价格？变化 X% 结论如何变？
-7. **GPU 可行性**（若用于算法/算子/训练）：求解方法过八维门，标注友好/可改造/不友好 + 改造建议。
-8. **行动建议**：明确写出"接下来我将……"
-
-**输出不得只给分析而无结论。**
-
-## 与其他 skill 的关系 / Relations to Other Skills
-
-- **建模思想**：优化前需先建模——定义目标与约束本身就是建模。
-- **概率与统计**：不确定性下的优化需随机/鲁棒优化。
-- **变换思想**：变换到对偶问题常更易求解；对偶是优化中最深刻的变换。
-- **博弈思想**：多决策者同时优化即博弈，纳什均衡为多人优化的稳定点。
-- **算法思想**：求解依赖算法设计——凸优化用梯度法，组合优化需分支定界/启发式。
-- **现代数学激活**：`../../references/books/optimization-ml.md`（GPU 友好优化器、二阶法可行性）、`matrix-analysis.md`（条件数、低秩、预条件）。

package/skills/optimization/original-texts.en.md

@@ -1,167 +0,0 @@
-# Mathematical Sources and Classic Texts
-
-## Euler-Lagrange Variational Equation & Brachistochrone (1696)
-
-> In 1696, Johann Bernoulli posed the brachistochrone problem: under gravity alone, along which curve does a particle descend from A to B in the shortest time? The answer is the cycloid.
-
-**Core equation**: For the functional J[y] = ∫_{a}^{b} F(x, y, y') dx, the extremizing function satisfies
-
-> ∂F/∂y - d/dx(∂F/∂y') = 0 (Euler-Lagrange equation)
-
-For the brachistochrone, F = √(1 + y'²) / √(2g(y₀ - y)). Substituting into the Euler-Lagrange equation yields the parametric equations of the cycloid: x = a(θ - sin θ), y = a(1 - cos θ).
-
-**Key idea**: The calculus of variations reduces "optimization over an entire class of functions" to solving a differential equation — this is the origin of infinite-dimensional optimization. Euler (1744) systematized the calculus of variations; Lagrange (1788) introduced the δ notation and the method of multipliers, both in the same lineage. The calculus of variations later evolved into optimal control theory (Pontryagin's Maximum Principle, 1961), with wide applications in spacecraft trajectory design, robotic path planning, and many other fields.
-
-## Fermat & Weierstrass Conditions for Unconstrained Optimization
-
-> Fermat (1636): A necessary condition for a differentiable function f to attain a local extremum at x* is ∇f(x*) = 0.
-> Weierstrass (19th c.): If f is twice differentiable and ∇f(x*) = 0, then the Hessian H(x*) is positive definite ⇔ x* is a strict local minimum; H(x*) is negative definite ⇔ x* is a strict local maximum.
-
-For convex functions, ∇f(x*) = 0 is not only necessary but also sufficient — and x* is the global minimum.
-
-**Key idea**: First-order conditions locate stationary points; second-order conditions determine the nature of the extremum — the most fundamental theorem of unconstrained optimization. The condition ∇f = 0 is the convergence target of virtually all iterative optimization algorithms (gradient descent, Newton's method).
-
-## Lagrange Multipliers (1788)
-
-> Finding the extremum of f(x) subject to g(x) = 0 is equivalent to finding the unconstrained extremum of L(x,λ) = f(x) - λg(x).
-
-**Key idea**: Transform a constrained optimization problem into an unconstrained one. At the optimal solution, the gradient of the objective function is parallel to the gradient of the constraint function: ∇f(x*) = λ∇g(x*).
-
-## Pareto Optimality (Vilfredo Pareto, ~1906)
-
-> In multi-objective optimization, a solution x is Pareto optimal if there does not exist y such that f_i(y) ≤ f_i(x) for all i with at least one f_j(y) < f_j(x) (strict improvement).
-
-In *Manuale di economia politica* (1906), Pareto introduced this concept into economics: the efficiency frontier of resource allocation. The Pareto frontier {f(x) : x is Pareto optimal} characterizes all trade-off solutions that cannot be simultaneously improved. Mathematically, for objectives f₁,...,fₘ, the Pareto frontier is {y ∈ ℝᵐ : y = (f₁(x),...,fₘ(x)), x Pareto optimal} — it is a (generally non-convex) surface in objective space.
-
-> Weighted scalarization: Convert multi-objective min f₁,...,fₘ into single-objective min Σw_i f_i (w_i > 0); each weight combination corresponds to a point on the Pareto frontier. This is a standard method for computing the Pareto frontier, but it cannot enumerate all Pareto-optimal solutions on non-convex frontiers.
-
-**Key idea**: Multi-objective optimization has no single "optimal solution" but rather a set of Pareto-optimal solutions among which trade-offs must be made. This is ubiquitous in economics (efficiency vs. equity), engineering (performance vs. cost), and machine learning (accuracy vs. complexity).
-
-## Game Theory & Optimization (von Neumann, 1928; Nash, 1950)
-
-> Nash equilibrium: A strategy profile (s₁*, ..., sₙ*) such that for each player i, s_i* is the best response given the other players' strategies — i.e., no player has an incentive to unilaterally deviate.
-
-Von Neumann (1928) proved the minimax theorem for zero-sum games: max_{x} min_{y} f(x,y) = min_{y} max_{x} f(x,y), which is essentially a precursor of linear programming duality. Nash (1950) generalized the equilibrium concept to non-zero-sum games; a Nash equilibrium is an " mutually optimal" optimization — each player's optimization problem takes the other players' strategies as constraints.
-
-> Existence of Nash equilibrium: Nash (1950) used the Brouwer fixed-point theorem to prove that every finite game has at least one mixed-strategy Nash equilibrium. This reveals a deep connection between game theory and topology.
-
-**Key idea**: Game theory extends optimization from "single-agent decision-making" to "multi-agent interactive decision-making" — optimality is no longer with respect to nature, but the best response to other rational agents.
-
-## Karush-Kuhn-Tucker Conditions (1939/1951)
-
-Necessary conditions for inequality-constrained optimization:
-
-> For min f(x) s.t. g_i(x) ≤ 0, h_j(x) = 0, the optimal solution x* satisfies:
-> - Stationarity: ∇f(x*) + Σμ_i∇g_i(x*) + Σλ_j∇h_j(x*) = 0
-> - Primal feasibility: g_i(x*) ≤ 0, h_j(x*) = 0
-> - Complementary slackness: μ_i · g_i(x*) = 0, μ_i ≥ 0
-
-Karush (1939, master's thesis) first derived these conditions; Kuhn & Tucker (1951) independently rediscovered and widely disseminated them.
-
-**Key idea**: At the optimal solution, either a constraint is inactive (μ_i = 0) or it is tight (g_i(x*) = 0). Complementary slackness is the cornerstone of duality theory — it ties the KKT conditions to a zero duality gap. When a constraint qualification holds (e.g., LICQ: the gradients of active constraints are linearly independent), the KKT conditions are necessary for optimality.
-
-## Linear Programming Duality (Farkas, 1902; Dantzig-von Neumann, 1947)
-
-> Farkas' Lemma (1902): Ax = b, x ≥ 0 has a solution ⟺ Aᵀy ≥ 0, bᵀy < 0 has no solution — this is the geometric foundation of LP duality.
-
-> Strong duality theorem: If the primal LP min cᵀx s.t. Ax = b, x ≥ 0 has a feasible solution, then the optimal values of the primal and the dual max bᵀy s.t. Aᵀy ≤ c are equal. The components of the dual variable y are called shadow prices — they measure the "marginal value" of each constraint.
-
-**Key idea**: Every LP has a "mirror" problem — the primal views the problem in terms of cost, the dual in terms of value. Zero duality gap is a profound fact: the minimum cost of resources equals the maximum revenue valued at shadow prices.
-
-## Simplex Method (Dantzig, 1947)
-
-The fundamental algorithm for linear programming. It searches for the optimal solution by moving along vertices of the feasible region.
-
-> Klee-Minty cube (1973): There exist LP instances for which the simplex method visits all 2ⁿ vertices — exponential worst-case complexity.
-
-> Interior-Point Methods: Karmarkar (1984) proposed a polynomial-time LP algorithm that converges along the central path through the interior of the feasible region, avoiding vertex enumeration. Modern interior-point methods (e.g., Mehrotra predictor-corrector) perform excellently in both practice and theory.
-
-**Key idea**: The optimal solution of a linear program always lies at a vertex of the feasible region. The simplex method exploits this structure by moving along edges; interior-point methods approach from the inside — the two are complementary.
-
-## Duality Theory
-
-> Weak duality: For any optimization problem, the dual optimal value d* ≤ primal optimal value p* (duality gap = p* - d* ≥ 0).
-
-> Strong duality (Slater's condition): If a convex optimization problem has a strictly feasible point (Slater point: g_i(x) < 0 strictly), then d* = p*, and the duality gap is zero.
-
-> Duality provides lower bounds (the primal optimal value ≥ the dual value), which is a critical tool in branch-and-bound, cutting-plane, and related algorithms.
-
-**Key idea**: Duality connects "minimizing cost" with "maximizing value" — two perspectives on the same problem. Slater's condition guarantees "no information loss" in convex optimization.
-
-## Gradient Descent (Cauchy, 1847; Robbins-Monro, 1951)
-
-> Cauchy (1847) proposed the method of steepest descent: x_{k+1} = x_k - α_k ∇f(x_k), iterating along the negative gradient direction with step size α_k. For L-smooth convex functions (Lipschitz-continuous gradient), with step size α = 1/L, f(x_k) - f(x*) ≤ L‖x₀ - x*‖² / (2k) — a convergence rate of O(1/k).
-
-> Newton's Method: x_{k+1} = x_k - H(x_k)⁻¹ ∇f(x_k), using second-order information, achieves quadratic convergence O(‖x_k - x*‖²) near the optimum.
-
-> Stochastic Gradient Descent (SGD): Robbins & Monro (1951) proposed stochastic approximation, replacing the true gradient with a noisy estimate g_k ≈ ∇f(x_k). The conditions Σα_k = ∞, Σα_k² < ∞ guarantee convergence. SGD converges in expectation at the cost of variance — minibatching is a trade-off between variance and efficiency.
-
-**Key idea**: Gradient descent is the most naive form of optimization — "go where it is steepest." SGD extends this idea to large-data settings: one need not examine all the data, only a random sample suffices to approximate the direction. The training of deep neural networks relies almost entirely on SGD and its variants (Adam, AdaGrad, etc.).
-
-## Integer Programming & NP-Hardness (Cook, 1971; Karp, 1972)
-
-> Integer Linear Programming (ILP): min cᵀx s.t. Ax ≤ b, x ∈ ℤⁿ — variables take integer values. Even 0-1 ILP (x ∈ {0,1}ⁿ) is NP-hard.
-
-> Cook (1971) proved that SAT is NP-complete (the Cook-Levin theorem); Karp (1972) listed 21 NP-complete problems, including integer programming, the knapsack problem, and the traveling salesman problem. ILP is NP-hard — no polynomial-time exact algorithm exists (under the assumption P ≠ NP).
-
-> Convex relaxation: Relaxing x ∈ ℤⁿ to x ∈ ℝⁿ yields an LP whose optimal value provides a lower bound for the ILP — this is precisely the core mechanism of branch-and-bound.
-
-**Key idea**: Discrete optimization is fundamentally harder than continuous optimization. The seemingly minor constraint of "integrality" pushes a problem from polynomial-time solvability to NP-hardness. Branch-and-bound and cutting-plane methods are the principal exact methods; heuristics are more commonly used in practice.
-
-## Convex Optimization (Boyd & Vandenberghe, 2004)
-
-> Convex optimization: min f(x) s.t. g_i(x) ≤ 0 (convex), h_j(x) = 0 (affine), x ∈ C (convex set). When f is convex, a local minimum is a global minimum.
-
-> Key properties: (1) Local minimum = global minimum; (2) The feasible region is a convex set; (3) Strong duality holds under Slater's condition; (4) Interior-point methods can solve it in polynomial time.
-
-Important subclasses encompassed by convex optimization:
-- LP (Linear Programming): f linear, constraints linear
-- QP (Quadratic Programming): f quadratic, constraints linear
-- SOCP (Second-Order Cone Programming): constraints include second-order cones ‖Ax + b‖ ≤ cᵀx + d
-- SDP (Semidefinite Programming): constraints include positive semidefinite matrix conditions X ≥ 0
-- GP (Geometric Programming): convex after logarithmic transformation
-
-Boyd & Vandenberghe's *Convex Optimization* (2004) unified all of the above problem classes and became the standard textbook and engineering reference for modern optimization. Convex optimization marks the boundary of "efficiently solvable optimization" — convexity guarantees algorithmic convergence and global optimality of solutions.
-
-**Key idea**: Convex optimization is the "sweet spot" of optimization theory — rich enough to encompass a vast number of practical problems, yet structured enough to guarantee tractability. Non-convex problems are often approximated via convex relaxation.
-
-## Bellman's Principle of Optimality (1957)
-
-> "An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision."
-
-> Recursive equation: V(s) = max_a { R(s,a) + γ · V(s') }, where s' is the result of the state transition.
-
-**Key idea**: The core of dynamic programming — substructures of an optimal solution are themselves optimal. The Bellman equation decomposes multi-stage decision-making into single-stage subproblems and is the theoretical foundation of reinforcement learning.
-
-## The Philosophical Significance of Optimization
-
-The core of optimization thinking is **making the best choice under constraints** — this is not only a mathematical question but also a question about life. Most decisions in life can be formulated as optimization problems:
-
-- **Objective function**: What do you want to maximize? (Happiness? Achievement? Freedom?)
-- **Constraints**: What limitations do you face? (Time? Money? Ability?)
-- **Feasible region**: What choices are available to you under the constraints?
-- **Optimal solution**: Among all choices, which one maximizes the objective?
-- **Dual perspective**: Viewing the problem in terms of "cost" or "value" yields the same answer — this is the philosophy of strong duality.
-- **Convex vs. non-convex**: Convex optimization has a unique global optimum; non-convex optimization has multiple local optima — the predicament of life lies precisely in the fact that the world is non-convex.
-
-## Timeline of Optimization
-
-| Year | Event |
-|------|-------|
-| 1636 | Fermat proposes the necessary condition ∇f = 0 for unconstrained extrema |
-| 1696 | Johann Bernoulli poses the brachistochrone problem; the seeds of the calculus of variations |
-| 1744 | Euler systematizes the calculus of variations |
-| 1788 | Lagrange publishes the method of multipliers in *Mécanique analytique* |
-| 1902 | Farkas' Lemma — the geometric foundation of LP duality |
-| ~1906 | Pareto introduces the efficiency frontier concept into economics |
-| 1928 | Von Neumann proves the minimax theorem (zero-sum games) |
-| 1939 | Karush derives the KKT conditions (master's thesis) |
-| 1947 | Dantzig invents the simplex method; von Neumann establishes LP duality |
-| 1950 | Nash proves the existence of equilibria in non-zero-sum games |
-| 1951 | Kuhn & Tucker publish the KKT conditions; Robbins-Monro propose stochastic approximation |
-| 1957 | Bellman publishes dynamic programming and the principle of optimality |
-| 1971 | Cook proves NP-completeness (the Cook-Levin theorem) |
-| 1972 | Karp lists 21 NP-complete problems |
-| 1973 | Klee-Minty prove exponential worst-case complexity of the simplex method |
-| 1984 | Karmarkar proposes the polynomial-time interior-point method |
-| 2004 | Boyd & Vandenberghe publish *Convex Optimization* |

package/skills/optimization/original-texts.md

@@ -1,168 +0,0 @@
-# 数学出处与经典文献 / Mathematical Sources and Classic Texts
-
-## Euler-Lagrange 变分方程与最速降线 / Euler-Lagrange Variational Equation & Brachistochrone (1696)
-
-> Johann Bernoulli 于 1696 年提出最速降线问题：在重力作用下，质点从 A 到 B 沿哪条曲线下降最快？答案是旋轮线（cycloid）。
-
-**核心方程**：对泛函 J[y] = ∫_{a}^{b} F(x, y, y') dx，极值函数满足
-
-> ∂F/∂y - d/dx(∂F/∂y') = 0 （Euler-Lagrange 方程）
-
-最速降线问题的 F = √(1 + y'²) / √(2g(y₀ - y))，代入 Euler-Lagrange 方程后解出旋轮线参数方程：x = a(θ - sin θ), y = a(1 - cos θ)。
-
-**思想**：变分法将"在整个函数类中寻优"化为微分方程求解——这是无穷维优化的起源。Euler（1744）系统化了变分法，Lagrange（1788）引入了 δ 符号与乘数法，两者一脉相承。变分法后来发展为最优控制理论（Pontryagin 最大值原理，1961），在航天轨道设计、机器人路径规划等领域广泛应用。
-
-## Fermat 与 Weierstrass 极值条件 / Fermat & Weierstrass Conditions for Unconstrained Optimization
-
-> Fermat（1636）：可微函数 f 在 x* 处取得局部极值的必要条件是 ∇f(x*) = 0。
-> Weierstrass（19th c.）：若 f 二阶可微且 ∇f(x*) = 0，则 Hessian H(x*) 正定 ⇔ x* 是严格局部极小；H(x*) 负定 ⇔ x* 是严格局部极大。
-
-对凸函数，∇f(x*) = 0 不仅是必要条件，也是充分条件——且 x* 是全局极小。
-
-**思想**：一阶条件找驻点，二阶条件判极值性质——无约束优化的最基本定理。∇f = 0 是几乎所有迭代优化算法（梯度下降、牛顿法）的收敛目标。
-
-## Lagrange 乘数法 / Lagrange Multipliers (1788)
-
-> 在约束 g(x) = 0 下求 f(x) 的极值，等价于求 L(x,λ) = f(x) - λg(x) 的无条件极值。
-
-**思想**：将约束优化问题转化为无约束问题。在最优解处，目标函数的梯度与约束函数的梯度平行：∇f(x*) = λ∇g(x*)。
-
-## Pareto 最优性 / Pareto Optimality (Vilfredo Pareto, ~1906)
-
-> 在多目标优化中，解 x 是 Pareto 最优的，若不存在 y 使得 f_i(y) ≤ f_i(x) ∀i 且至少一个 f_j(y) < f_j(x)（严格改善）。
-
-Pareto 在《Manuale di economia politica》(1906) 中将此概念引入经济学：资源配置的效率前沿。Pareto 前沿 Pareto frontier {f(x) : x is Pareto optimal} 刻画了所有不可同时改善的权衡解。数学上，对目标 f₁,...,fₘ 的 Pareto 前沿是 {y ∈ ℝᵐ : y = (f₁(x),...,fₘ(x)), x Pareto optimal}——它是目标空间中的一个（通常非凸的）曲面。
-
-> 加权标量化：将多目标 min f₁,...,fₘ 转化为单目标 min Σw_i f_i（w_i > 0），每个权重组合对应 Pareto 前沿上的一个点。这是求解 Pareto 前沿的常用方法，但对非凸前沿不能遍历所有 Pareto 最优解。
-
-**思想**：多目标优化没有唯一的"最优解"，而有一组 Pareto 最优解，需要在它们之间做权衡。这在经济学（效率与公平）、工程（性能与成本）、机器学习（精度与复杂度）中无处不在。
-
-## 博弈论与优化 / Game Theory & Optimization (von Neumann, 1928; Nash, 1950)
-
-> Nash 均衡：策略组 (s₁*, ..., sₙ*) 使得每个玩家 i 在其他玩家策略固定时，s_i* 是其最优响应——即无人有单方面偏离的动机。
-
-von Neumann（1928）证明了零和博弈的 minimax 定理：max_{x} min_{y} f(x,y) = min_{y} max_{x} f(x,y)，这本质上是线性规划对偶性的先声。Nash（1950）将均衡推广到非零和博弈，Nash 均衡是"互为最优"的优化——每个玩家的优化问题以他人策略为约束。
-
-> Nash 均衡的存在性：Nash（1950）用 Brouwer 不动点定理证明了每个有限博弈至少存在一个混合策略 Nash 均衡。这揭示了博弈论与拓扑学的深层联系。
-
-**思想**：博弈论将优化从"单人决策"推广到"多人交互决策"——最优不再是对自然的最优，而是对其他理性主体的最优响应。
-
-## KKT 条件 / Karush-Kuhn-Tucker Conditions (1939/1951)
-
-不等式约束优化的必要条件：
-
-> 对于 min f(x) s.t. g_i(x) ≤ 0, h_j(x) = 0，最优解 x* 满足：
-> - 平稳性：∇f(x*) + Σμ_i∇g_i(x*) + Σλ_j∇h_j(x*) = 0
-> - 可行性：g_i(x*) ≤ 0, h_j(x*) = 0
-> - 互补松弛性：μ_i · g_i(x*) = 0, μ_i ≥ 0
-
-Karush（1939，硕士论文）最早给出此条件；Kuhn & Tucker（1951）独立重新发现并广泛传播。
-
-**思想**：最优解处，要么约束不起作用（μ_i = 0），要么约束是紧的（g_i(x*) = 0）。互补松弛性是对偶理论的基石——它将 KKT 条件与对偶间隙为零紧密联系。当约束限定条件 constraint qualification 成立时（如 LICQ：活跃约束梯度线性无关），KKT 条件是最优性的必要条件。
-
-## 线性规划对偶 / Linear Programming Duality (Farkas, 1902; Dantzig-von Neumann, 1947)
-
-> Farkas 引理（1902）：Ax = b, x ≥ 0 有解 ⟺ Aᵀy ≥ 0, bᵀy < 0 无解——这是 LP 对偶的几何根基。
-
-> 强对偶定理：若原 LP min cᵀx s.t. Ax = b, x ≥ 0 有可行解，则 max bᵀy s.t. Aᵀy ≤ c 的最优值相等。对偶变量 y 的分量称为影子价格 shadow price——它衡量每个约束的"边际价值"。
-
-**思想**：每个 LP 都有一个"镜像"问题——原问题看成本，对偶问题看价值。对偶间隙为零是深刻的事实：资源的最小成本等于影子价格的最大收益。
-
-## Dantzig 单纯形法 / Simplex Method (1947)
-
-线性规划的基本算法。在可行域的顶点上搜索最优解。
-
-> Klee-Minty 立方体（1973）：存在 LP 实例使得单纯形法遍历所有 2ⁿ 个顶点——最坏情况指数复杂度。
-
-> 内点法 / Interior-Point Methods：Karmarkar（1984）提出多项式时间 LP 算法，在可行域内部沿中心路径收敛，避免顶点遍历。现代内点法（如 Mehrotra predictor-corrector）在实践和理论上均表现优异。
-
-**思想**：线性规划的最优解一定在可行域的顶点上。单纯形法利用此结构沿边移动；内点法从内部逼近，两者互补。
-
-## 对偶理论 / Duality Theory
-
-> 弱对偶：对任何优化问题，对偶问题的最优值 d* ≤ 原问题最优值 p*（对偶间隙 gap = p* - d* ≥ 0）。
-
-> 强对偶（Slater 条件）：若凸优化问题存在严格可行点（Slater point：g_i(x) < 0 严格成立），则 d* = p*，对偶间隙为零。
-
-> 对偶提供了下界（原问题的最优值 ≥ 对偶值），这在分支定界、 Cutting-plane 等算法中是关键工具。
-
-**思想**：对偶将"最小化成本"与"最大化价值"联系起来——两个视角看同一问题。Slater 条件保证了凸优化的"无信息损失"。
-
-## 梯度下降 / Gradient Descent (Cauchy, 1847; Robbins-Monro, 1951)
-
-> Cauchy（1847）提出最速下降法：x_{k+1} = x_k - α_k ∇f(x_k)，沿负梯度方向迭代，α_k 为步长。对 L-平滑凸函数（Lipschitz 连续梯度），步长 α = 1/L 时 f(x_k) - f(x*) ≤ L‖x₀ - x*‖² / (2k)——线性收敛率 O(1/k)。
-
-> 牛顿法 Newton's Method：x_{k+1} = x_k - H(x_k)⁻¹ ∇f(x_k)，利用二阶信息，在最优解附近达到二次收敛 O(‖x_k - x*‖²)。
-
-> 随机梯度下降 SGD：Robbins & Monro（1951）提出随机逼近，用噪声梯度 g_k ≈ ∇f(x_k) 替代真梯度。条件 Σα_k = ∞, Σα_k² < ∞ 保证收敛。SGD 在期望意义上收敛，代价是方差——minibatch 是方差与效率的折中。
-
-**思想**：梯度下降是最朴素的优化——"哪里最陡就往哪里走"。SGD 将此思想扩展到大数据场景：不需要看全部数据，只需随机采样一小部分即可逼近方向。深度学习的训练几乎全部依赖 SGD 及其变体（Adam, AdaGrad 等）。
-
-## 整数规划与 NP-困难 / Integer Programming & NP-Hardness (Cook, 1971; Karp, 1972)
-
-> 整数线性规划 ILP：min cᵀx s.t. Ax ≤ b, x ∈ ℤⁿ——变量取整数值。即使 0-1 ILP（x ∈ {0,1}ⁿ）也是 NP困难的。
-
-> Cook（1971）证明 SAT 是 NP 完全的（Cook-Levin 定理）；Karp（1972）列出 21 个 NP 完全问题，包含整数规划、背包问题、旅行商问题等。ILP 是 NP困难的——不存在（在 P ≠ NP 假设下）多项式时间精确算法。
-
-> 凸松弛：将 x ∈ ℤⁿ 松弛为 x ∈ ℝⁿ 得到 LP，其最优值给出 ILP 的下界——这正是分支定界法 branch-and-bound 的核心机制。
-
-**思想**：离散优化比连续优化本质上更难。"整数"这一看似微小的约束，将问题从多项式可解推向 NP 困难。分支定界 branch-and-bound、割平面 cutting-plane 是主要精确方法；启发法在实践中更常用。
-
-## 凸优化 / Convex Optimization (Boyd & Vandenberghe, 2004)
-
-> 凸优化：min f(x) s.t. g_i(x) ≤ 0 (凸), h_j(x) = 0 (线性), x ∈ C (凸集)。f 为凸函数时局部极小即全局极小。
-
-> 关键性质：(1) 局部极小 = 全局极小；(2) 可行域是凸集；(3) Slater 条件下强对偶成立；(4) 内点法在多项式时间内可解。
-
-凸优化涵盖的重要子类：
-- LP（线性规划）：f 线性，约束线性
-- QP（二次规划）：f 二次，约束线性
-- SOCP（二阶锥规划）：约束含二阶锥 ‖Ax + b‖ ≤ cᵀx + d
-- SDP（半定规划）：约束含矩阵半正定条件 X ≥ 0
-- GP（几何规划）：对数空间下化为凸优化
-
-Boyd & Vandenberghe《Convex Optimization》(2004) 统一了上述各类问题，成为现代优化的标准教材与工程参考。凸优化是"可高效求解的优化"的边界——凸性保证了算法的收敛性与解的全局性。
-
-**思想**：凸优化是优化理论的"甜蜜区"——足够丰富以涵盖大量实际问题，又足够结构化以保证可解性。非凸问题往往通过凸松弛 convex relaxation 来近似。
-
-## Bellman 最优性原理 / Bellman's Principle of Optimality (1957)
-
-> "一个最优策略具有这样的性质：无论初始状态和初始决策如何，剩余决策必须构成关于由初始决策产生的状态的最优策略。"
-> "An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision."
-
-> 递推方程：V(s) = max_a { R(s,a) + γ · V(s') }，其中 s' 为状态转移结果。
-
-**思想**：动态规划的核心——最优解的子结构也是最优的。Bellman 方程将多阶段决策分解为单阶段子问题，是强化学习的理论根基。
-
-## 优化的哲学意义
-
-优化思想的核心是**在约束下做最好的选择**——这不仅是数学问题，也是人生问题。生活中大多数决策都可以表述为优化问题：
-
-- **目标函数**：你想最大化什么？（幸福？成就？自由？）
-- **约束条件**：你面临什么限制？（时间？金钱？能力？）
-- **可行域**：在约束下你有什么选择？
-- **最优解**：在所有选择中，哪个使目标最大化？
-- **对偶视角**：从"成本"看和从"价值"看，答案是一致的——这正是强对偶的哲学。
-- **凸与非凸**：凸优化有唯一全局最优；非凸优化有多个局部最优——人生的困境恰在于世界是非凸的。
-
-## 优化理论历史年表 / Timeline of Optimization
-
-| 年份 | 事件 |
-|------|------|
-| 1636 | Fermat 提出无约束极值的必要条件 ∇f = 0 |
-| 1696 | Johann Bernoulli 提出最速降线问题，变分法萌芽 |
-| 1744 | Euler 系统化变分法 |
-| 1788 | Lagrange 发表乘数法，融入《分析力学》 |
-| 1902 | Farkas 引理——LP 对偶的几何基础 |
-| ~1906 | Pareto 将效率前沿概念引入经济学 |
-| 1928 | von Neumann 证明 minimax 定理（零和博弈） |
-| 1939 | Karush 给出 KKT 条件（硕士论文） |
-| 1947 | Dantzig 发明单纯形法；von Neumann 建立 LP 对偶 |
-| 1950 | Nash 证明非零和博弈均衡存在性 |
-| 1951 | Kuhn & Tucker 发表 KKT 条件；Robbins-Monro 提出随机逼近 |
-| 1957 | Bellman 发表动态规划与最优性原理 |
-| 1971 | Cook 证明 NP 完全性（Cook-Levin 定理） |
-| 1972 | Karp 列出 21 个 NP 完全问题 |
-| 1973 | Klee-Minty 证明单纯形法最坏情况指数复杂度 |
-| 1984 | Karmarkar 提出多项式时间内点法 |
-| 2004 | Boyd & Vandenberghe 出版《Convex Optimization》