math-skill 1.0.0 → 2.1.0

package/skills/modeling/original-texts.en.md

@@ -0,0 +1,162 @@
+# Mathematical Sources and Classic Texts
+
+## Newton's *Principia Mathematica* (1687)
+
+> "The same effects of nature must always be assigned to the same causes."
+
+**The most important achievement in the history of mathematical modeling**: Newton unified celestial and terrestrial motion within a single mathematical framework — the law of universal gravitation F = GMm/r² and the three laws of motion. This was humanity's first systematic use of mathematical models to precisely describe the physical world, marking the birth of scientific modeling. The *Principia* established the methodological paradigm of "mathematical model → physical prediction → experimental verification," which remains the foundation of all modeling work to this day.
+
+---
+
+## Fourier's Heat Equation (1822)
+
+> ∂u/∂t = κ ∇²u
+
+In *Théorie analytique de la chaleur*, Fourier proposed a mathematical model for heat diffusion and, in doing so, invented Fourier analysis — the method of decomposing arbitrary functions into trigonometric series. This modeling achievement has a twofold significance: it provided a precise mathematical description of heat conduction, and it gave rise to the entire field of harmonic analysis. The Fourier transform remains a core tool in signal processing, quantum mechanics, image compression, and many other domains.
+
+**Modeling insight**: The new mathematics invented to solve a modeling problem often has a more profound and lasting impact than the original model itself.
+
+---
+
+## Maxwell's Equations (1865)
+
+> ∇·E = ρ/ε₀, ∇×E = -∂B/∂t
+> ∇·B = 0, ∇×B = μ₀J + μ₀ε₀∂E/∂t
+
+Maxwell unified electricity and magnetism into four partial differential equations, and from the mathematical model alone predicted the existence of electromagnetic waves — a prediction later confirmed experimentally by Hertz. This is one of the most brilliant examples in the history of modeling: the discovery of an entirely new physical phenomenon through pure mathematical deduction. The equations also imply the speed of light c = 1/√(μ₀ε₀), subsuming optics into electromagnetic theory — arguably the pinnacle of unification in modeling.
+
+---
+
+## Lotka-Volterra Predator-Prey Model (1925-1926)
+
+The classical population dynamics model:
+
+> dx/dt = αx - βxy (prey growth - predation)
+> dy/dt = δxy - γy (predator growth - natural death)
+
+**Modeling insight**: Through two simple differential equations, this model successfully explains the periodic oscillations observed in predator and prey populations in nature. It is the epitome of a "good model" — simple yet useful.
+
+---
+
+## Kermack-McKendrick SIR Epidemic Model (1927)
+
+> dS/dt = -βSI (decrease in susceptibles)
+> dI/dt = βSI - γI (change in infected)
+> dR/dt = γI (increase in recovered)
+
+**Modeling insight**: The basic reproduction number R₀ = β/γ determines whether an epidemic will take off. Kermack and McKendrick first proposed this model in their 1927 paper "A Contribution to the Mathematical Theory of Epidemics," laying the theoretical foundation for mathematical modeling of infectious diseases. This simple model was widely deployed during the COVID-19 pandemic, demonstrating the enduring value of classical models.
+
+---
+
+## Buckingham Pi Theorem (1914)
+
+> If a physical relationship involves n variables with k independent dimensions, the relationship can be reduced to one among (n-k) dimensionless Π groups.
+
+**Modeling insight**: The Buckingham Pi theorem formalizes dimensional analysis, an extremely important simplification tool in modeling. It tells us that any physical model can be cast in dimensionless form, thereby reducing the number of parameters and revealing essential structure. For example, the Reynolds number Re = ρvL/μ in fluid mechanics is a single Π quantity that unifies countless seemingly different flow regimes.
+
+---
+
+## Turing's Reaction-Diffusion Model (1952)
+
+> ∂u/∂t = D_u ∇²u + f(u,v)
+> ∂v/∂t = D_v ∇²v + g(u,v)
+
+In his paper "The Chemical Basis of Morphogenesis," Turing proved that two chemical substances diffusing at different rates and interacting can generate stable spatial patterns — spots, stripes, spirals — from an initially uniform state. This is a mathematical model of morphogenesis, explaining a wide range of phenomena from leopard spots to seashell patterns.
+
+**Modeling insight**: Mathematical models can explain "how order emerges from disorder" — no pre-existing pattern is required; patterns arise spontaneously from the dynamics of the equations alone. Turing thereby founded the theory of pattern formation.
+
+---
+
+## Lorenz System (1963)
+
+> dx/dt = σ(y - x)
+> dy/dt = x(ρ - y) - xz
+> dz/dt = xy - βz
+
+While numerically simulating an atmospheric convection model, Lorenz discovered that tiny differences in initial conditions lead to completely different long-term behavior — the "butterfly effect." Three seemingly simple equations revealed the essential feature of chaos: the long-term unpredictability of deterministic systems.
+
+**Modeling insight**: Chaos theory profoundly changed the philosophy of modeling — even if a model is perfectly correct, long-term prediction may be fundamentally impossible. Modelers must distinguish between "predictable timescales" and "unpredictable chaotic regimes."
+
+---
+
+## Kalman Filtering (1960)
+
+> x̂_{k|k} = x̂_{k|k-1} + K_k(z_k - H x̂_{k|k-1})
+
+The Kalman filter is a recursive estimation method based on state-space models: it uses the system dynamics model to predict the state and then corrects the prediction with observational data. It unifies "model prediction" and "data updating" within a single mathematical framework and is a cornerstone of modern control theory and signal processing.
+
+**Modeling insight**: Good modeling is not only about "building a model" but also about "making optimal estimates based on the model." The Kalman filter demonstrates how models and data work together — the model provides the prior, the data provide the correction, and neither is dispensable.
+
+---
+
+## Pólya's *How to Solve It* (1945)
+
+> "The first step in mathematical modeling is understanding the problem, the second is devising a plan, the third is carrying out the plan, and the fourth is looking back."
+
+Pólya's problem-solving framework is a precursor to modeling thinking: transforming an unfamiliar problem into a known mathematical problem.
+
+---
+
+## Akaike Information Criterion (1974)
+
+> AIC = -2 ln(L_max) + 2k
+
+In 1974, Akaike proposed AIC, formalizing the model selection problem for the first time: striking an optimal balance between goodness of fit (-2 ln L) and model complexity (2k), where k is the number of parameters and L_max is the maximum likelihood.
+
+**Modeling insight**: AIC established a mathematical standard for the "principle of parsimony" — neither the most complex nor the simplest model is best; rather, the optimal trade-off lies in minimizing information loss while keeping parameters parsimonious. This is the quantitative version of Box's dictum that "some are useful."
+
+---
+
+## Black-Scholes Model (1973)
+
+> C = S N(d₁) - K e^{-rT} N(d₂)
+> d₁ = [ln(S/K) + (r + σ²/2)T] / (σ√T)
+> d₂ = d₁ - σ√T
+
+Black, Scholes, and Merton developed a mathematical model for option pricing. Based on geometric Brownian motion dS = μS dt + σS dW and the no-arbitrage principle, they derived the partial differential equation ∂C/∂t + ½σ²S²∂²C/∂S² + rS∂C/∂S - rC = 0. This model is the most celebrated result in financial mathematics; Scholes and Merton were awarded the 1997 Nobel Prize in Economics for this work.
+
+**Modeling insight**: The Black-Scholes model is the best illustration of Box's dictum — its assumptions (constant volatility, continuous trading, frictionless markets) are all "wrong" in reality, yet it provides the core framework for pricing and risk management, and is therefore "useful."
+
+---
+
+## George Box's Famous Quote
+
+> "All models are wrong, but some are useful."
+
+**Modeling philosophy**:
+- A model is not reality — it is necessarily a simplification of reality
+- The value of a model lies not in its "truthfulness" but in its predictive and explanatory power
+- Criteria for a good model: parsimonious, testable, predictive
+
+---
+
+## General Principles of Modeling
+
+1. **Start simple**: Begin with the simplest model, then incrementally add complexity (Lorenz revealed chaos with three equations; Lotka-Volterra explained oscillations with two)
+2. **State assumptions explicitly**: Record and test every assumption (Black-Scholes assumptions, though wrong, are explicit — and so the model remains usable)
+3. **Validate and falsify**: Test models with independent data, not just by fitting (AIC quantifies the risk of overfitting)
+4. **Know the scope**: Every model has a domain of validity; beyond it, the model fails (Newtonian mechanics fails at high velocities and requires Einstein's correction)
+5. **Iterate**: Modeling is a cyclical process, not a one-shot endeavor (Pólya's "looking back" step)
+6. **Unify dimensions**: Use the Buckingham Pi theorem to cast models in dimensionless form, reducing parameters and revealing structure
+7. **Synergize models and data**: The Kalman filter demonstrates how model prediction and data correction work in a feedback loop
+
+---
+
+## Timeline of Mathematical Modeling
+
+| Year | Achievement | Field |
+|------|-------------|-------|
+| 1687 | Newton *Principia* | Mechanics |
+| 1822 | Fourier heat equation | Heat diffusion |
+| 1865 | Maxwell's equations | Electromagnetism |
+| 1914 | Buckingham Pi theorem | Dimensional analysis |
+| 1925-26 | Lotka-Volterra | Population dynamics |
+| 1927 | Kermack-McKendrick SIR | Epidemiology |
+| 1945 | Pólya *How to Solve It* | Methodology |
+| 1952 | Turing reaction-diffusion | Morphogenesis |
+| 1960 | Kalman filter | Estimation & control |
+| 1963 | Lorenz system | Chaos theory |
+| 1973 | Black-Scholes | Finance |
+| 1974 | Akaike AIC | Model selection |
+
+This timeline reveals a central pattern: great modeling achievements often transcend disciplinary boundaries. Newton unified the heavens and the earth, Maxwell unified electricity and magnetism, Turing unified chemistry and biology — the power of mathematical models lies precisely in their cross-domain universality.

package/skills/optimization/SKILL.en.md

@@ -0,0 +1,129 @@
+---
+name: optimization
+description: |
+  Trigger when a problem involves resource allocation, trade-offs, maximizing/minimizing objectives, decisions under constraints; or needs convexity analysis, Lagrangian/KKT methods, duality structure; or choosing optimization methods for algorithm/operator/training design.
+---
+
+# Optimization
+
+> "Under the most general constraints, find extrema of the objective -- convexity determines difficulty, KKT gives necessity, duality reveals structure."
+>
+> -- Optimization Theory & Operations Research
+
+## Core Principle
+
+**Any decision problem can be formulated as an optimization problem: maximizing (or minimizing) an objective subject to constraints. The essence of optimization is not the pursuit of "the best" in the abstract, but "the best among the feasible."**
+
+The three core elements of optimization: **Objective**, **Constraints**, **Feasible set**.
+
+> **Mathematical Formalization**
+>
+> General optimization problem: $\min_{x \in \mathbb{R}^n} f(x) \quad \text{s.t.} \quad g_i(x) \leq 0,\; i=1,\dots,m; \quad h_j(x) = 0,\; j=1,\dots,p$
+>
+> Lagrangian: $L(x, \lambda, \mu) = f(x) + \sum_i \lambda_i g_i(x) + \sum_j \mu_j h_j(x)$
+>
+> KKT conditions (essential necessary conditions, under Slater-type constraint qualifications): (1) Stationarity $\nabla_x L = 0$; (2) Primal feasibility $g_i \le 0, h_j = 0$; (3) Dual feasibility $\lambda_i \ge 0$; (4) Complementary slackness $\lambda_i g_i = 0$.
+>
+> Convexity: If $f$ and each $g_i$ are convex and each $h_j$ is linear, the problem is convex; in this case **KKT is sufficient**, and local optimality implies global optimality.
+
+## GPU-Friendliness (Cross-Cutting Check)
+
+When optimization is used for **algorithm/operator/training design**, the solution method itself must pass the eight-dimensional gate in `../../references/gpu-friendly-math.md`:
+
+- **First-order methods (SGD/Adam)**: GEMM-friendly, parallelizable, viable in low precision; pay attention to optimizer state precision and distributed communication overhead.
+- **Second-order / Newton methods**: Hessian inversion $O(n^3)$, memory explosion -- the classic "beautiful but incomputable" case -- adapt to **K-FAC / low-rank / diagonal approximations** (see `../../references/books/optimization-ml.md`, `matrix-analysis.md`).
+- **Constraint projection**: Does the projection admit a closed form and is it tensorizable? Iterative projection requires caution regarding serial dependencies.
+- **Distribution**: Can computation and communication overlap? Is gradient compression needed?
+
+Eight-dimensional minimum assessment (formal terms): **Tensorization** -- whether objectives / constraints / gradients can be batched; **GEMM-mappability** -- whether the primary computation is matrix multiplication, HVP, or small K-FAC matrices; **Complexity** -- order of first-order / second-order / combinatorial solvers; **Memory & KV-Cache** -- optimizer states, Hessian storage, activation retention; **Low-precision stability** -- condition numbers, damping, loss scaling; **Parallelism & communication** -- gradient synchronization and communication overlap; **Sparse structure** -- whether preconditioners / constraints are block-structured; **Operator fusion** -- whether updates, clipping, and regularization can be fused.
+
+> Cross-reference `../../references/books/optimization-ml.md` (Chong/Lu/Żak) and `../../references/books/matrix-analysis.md`.
+
+## When NOT to Use
+
+- **No clear evaluation criterion** (one does not know what "good" means) -- define the objective before optimizing.
+- **Purely execution-oriented tasks** (e.g., formatting code) -- there is no optimization space.
+- **The user has already decided on a plan** -- optimization is already complete.
+- **The problem is essentially qualitative judgment rather than quantitative extremum** -- model first, then optimize.
+
+## When to Use
+
+- When one needs to determine whether a problem is convex in order to assess solution difficulty.
+- Choosing optimization methods for algorithm/operator/training design and evaluating their GPU viability.
+- Making rational decisions with quantifiable objectives under constraints.
+- Systematic optimization of experimental design, resource allocation, and hyperparameter / architecture search.
+- When one is uncertain whether the current strategy is optimal and wishes to perform a systematic analysis (convexity, duality, sensitivity).
+
+## Method
+
+### Step 1: Define the Objective
+Clarify what is to be maximized / minimized. Key questions: Single-objective or multi-objective? Is it quantifiable (otherwise, find proxy variables)? Static or dynamic? **Is $f$ convex** (convex implies local = global; non-convex requires vigilance against local extrema)? A wrong objective leads further astray the further one proceeds.
+
+### Step 2: List the Constraints
+Distinguish **hard constraints** (physical / budget / deadline) from **soft constraints** (preferences / quality floors); mathematically classify: inequality $g_i(x)\le 0$ (defines the boundary of the feasible set), equality $h_j(x)=0$ (reduces dimensionality), linear (feasible set is a convex polyhedron) vs. nonlinear (may be non-convex).
+
+### Step 3: Classify the Problem Type
+
+| Type | Objective | Constraints | Key Property | Typical Method |
+|------|-----------|-------------|--------------|----------------|
+| LP (Linear Programming) | Linear | Linear inequalities | Optimum at a vertex | Simplex method |
+| QP (Quadratic Programming) | Quadratic | Linear | Positive-definite QP is convex | Interior-point method |
+| Convex optimization | Convex | Convex inequalities + linear equalities | Local = global | Gradient descent, interior-point methods |
+| Non-convex optimization | Non-convex | Arbitrary | Multiple local extrema | Global search, simulated annealing |
+| Combinatorial optimization | Discrete domain | Arbitrary | Frequently NP-hard | Branch-and-bound, heuristics |
+| Stochastic optimization | Contains random terms | May include stochastic constraints | Expected optimum vs. stochastic feasibility | SAA, robust optimization |
+
+### Step 4: Find the Optimal Solution
+- **LP/QP/Convex**: Exploit convexity; gradient-based or interior-point methods guarantee convergence to the global optimum.
+- **Non-convex**: Multi-start strategies, global search, or relaxation to convex approximations.
+- **Combinatorial**: Exact solutions are often NP-hard -- branch-and-bound for small scale, heuristics / approximations for large scale.
+- **Stochastic**: Sample Average Approximation (SAA) converts to a deterministic approximation.
+- **Insufficient information**: A satisficing solution suffices.
+
+### Step 5: Sensitivity Analysis
+The Lagrange multiplier $\lambda_i^*$ is the **shadow price** of the $i$-th constraint -- relaxing the constraint by one unit improves the objective by approximately $\lambda_i^*$. Complementary slackness: $\lambda_i^*=0$ indicates an inactive constraint (no effect on the optimal solution); $\lambda_i^*>0$ indicates an active constraint (the optimal solution lies precisely at its boundary). Focus on: how the optimal solution changes under small perturbations of constraints / objective, and which constraints are active.
+
+### Step 6: Multi-Objective & Pareto
+Multi-objective problems $f_1,\dots,f_k$ generally have no single optimal solution. Pareto optimality: no feasible solution exists that improves all objectives simultaneously. Methods: **Weighted sum** $\min\sum w_i f_i$ (different weights trace different points on the Pareto front); **$\epsilon$-constraint method** $\min f_1$ s.t. $f_i\le\epsilon_i$ (sweep $\epsilon_i$ to cover the front).
+
+### Step 7: Monitor Constraint Changes
+The optimal solution depends on the constraints -- when constraints change, re-optimization is required. Changes in active constraints have the greatest impact (high shadow prices); small changes in inactive constraints typically do not affect the optimal solution.
+
+## Common Errors
+
+| Error | Critique | Correct Approach |
+|-------|----------|------------------|
+| Optimizing without a clear objective | Direction is undefined | Precisely define the objective first |
+| Ignoring implicit constraints | The "optimal solution" is actually infeasible | Exhaustively verify all constraints |
+| Getting trapped in local optima | Greedy methods on non-convex problems do not guarantee global optimality | Verify convexity; use multi-start / global methods for non-convex problems |
+| Treating the optimum as unique | The optimal solution may not be unique | Check for the existence of multiple equivalent optima |
+| Using single-objective methods for multi-objective problems | Different objectives require trade-offs | Employ Pareto analysis |
+| Failing to verify convexity | Applying convex methods to non-convex problems | Determine convexity before selecting a method |
+| Ignoring duality theory | The dual problem may be easier to solve | Construct the dual and exploit strong duality |
+| Confusing feasibility with optimality | Feasible does not imply optimal | Verify feasibility first, then verify optimality |
+| Ignoring computational / GPU complexity | Second-order methods / combinatorial optimization may be incomputable | Assess complexity, pass the GPU eight-dimensional gate, approximate when necessary |
+| Forgetting to re-optimize | Failing to update when constraints change | Periodically check for constraint changes |
+
+## Operating Procedure
+
+When this skill is triggered, the output must include:
+
+1. **Objective function**: `[Objective]: [Description]` + `[Convexity]: [Convex / Non-convex / Unknown]`
+2. **Constraint list**: Each labeled `[Hard / Soft]` and `[Inequality / Equality]` `[Linear / Nonlinear]`
+3. **Problem type classification**: `[Type]: [LP / QP / Convex / Non-convex / Combinatorial / Stochastic]`
+4. **Feasible set analysis**: Which options are feasible? Which constraints are active?
+5. **Optimal / satisficing solution**: `[Strategy]: [Gradient method / Interior-point / Global search / Satisficing / Pareto]`
+6. **Sensitivity analysis**: Shadow prices of key constraints? How do conclusions change under X% variation?
+7. **GPU viability** (if used for algorithm/operator/training): Does the solution method pass the eight-dimensional gate? Label as friendly / retrofittable / unfriendly, with adaptation recommendations.
+8. **Action recommendations**: Explicitly state "Next, I will..."
+
+**Output must not consist of analysis alone without conclusions.**
+
+## Relations to Other Skills
+
+- **Modeling**: Optimization requires prior modeling -- defining objectives and constraints is itself an act of modeling.
+- **Probability and statistics**: Optimization under uncertainty requires stochastic / robust optimization.
+- **Transformation**: Transforming to the dual problem often makes optimization easier; duality is the most profound transformation in optimization.
+- **Game-theoretic thinking**: Multiple decision-makers simultaneously optimizing constitutes a game; the Nash equilibrium is the stable point of multi-player optimization.
+- **Algorithmic thinking**: Solution methods depend on algorithm design -- convex optimization uses gradient methods, combinatorial optimization requires branch-and-bound / heuristics.
+- **Modern mathematics activation**: `../../references/books/optimization-ml.md` (GPU-friendly optimizers, feasibility of second-order methods), `matrix-analysis.md` (condition numbers, low-rank, preconditioning).