math-skill 2.1.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en-US.md +159 -200
- package/README.md +159 -200
- package/agents/math-critic.en.md +2 -2
- package/agents/math-critic.md +8 -8
- package/commands/ask.md +3 -10
- package/design-patterns/attention/equivariant-attention.en.md +72 -0
- package/design-patterns/attention/equivariant-attention.md +72 -0
- package/design-patterns/attention/geometry-aware-attention.en.md +73 -0
- package/design-patterns/attention/geometry-aware-attention.md +72 -0
- package/design-patterns/attention/information-bottleneck-attention.en.md +72 -0
- package/design-patterns/attention/information-bottleneck-attention.md +72 -0
- package/design-patterns/attention/projection-attention.en.md +68 -0
- package/design-patterns/attention/projection-attention.md +68 -0
- package/design-patterns/attention/spectral-attention.en.md +71 -0
- package/design-patterns/attention/spectral-attention.md +71 -0
- package/design-patterns/compression/leverage-score-selection.en.md +71 -0
- package/design-patterns/compression/leverage-score-selection.md +71 -0
- package/design-patterns/compression/low-rank-kv-cache.en.md +60 -0
- package/design-patterns/compression/low-rank-kv-cache.md +60 -0
- package/design-patterns/compression/spectral-token-pruning.en.md +61 -0
- package/design-patterns/compression/spectral-token-pruning.md +61 -0
- package/design-patterns/compression/topology-preserving-compression.en.md +64 -0
- package/design-patterns/compression/topology-preserving-compression.md +64 -0
- package/design-patterns/loss/constraint-penalty.en.md +67 -0
- package/design-patterns/loss/constraint-penalty.md +78 -0
- package/design-patterns/loss/contrastive-loss.en.md +59 -0
- package/design-patterns/loss/contrastive-loss.md +68 -0
- package/design-patterns/loss/information-bottleneck-loss.en.md +69 -0
- package/design-patterns/loss/information-bottleneck-loss.md +74 -0
- package/design-patterns/loss/orthogonality-loss.en.md +56 -0
- package/design-patterns/loss/orthogonality-loss.md +60 -0
- package/design-patterns/loss/variational-loss.en.md +61 -0
- package/design-patterns/loss/variational-loss.md +71 -0
- package/design-patterns/representation/equivariant-split.en.md +87 -0
- package/design-patterns/representation/equivariant-split.md +89 -0
- package/design-patterns/representation/manifold-representation.en.md +85 -0
- package/design-patterns/representation/manifold-representation.md +87 -0
- package/design-patterns/representation/shared-private-decomposition.en.md +84 -0
- package/design-patterns/representation/shared-private-decomposition.md +86 -0
- package/design-patterns/representation/subspace-alignment.en.md +90 -0
- package/design-patterns/representation/subspace-alignment.md +92 -0
- package/design-patterns/routing/graph-routing.en.md +79 -0
- package/design-patterns/routing/graph-routing.md +81 -0
- package/design-patterns/routing/moe-routing.en.md +85 -0
- package/design-patterns/routing/moe-routing.md +87 -0
- package/design-patterns/routing/optimal-transport-routing.en.md +68 -0
- package/design-patterns/routing/optimal-transport-routing.md +72 -0
- package/design-patterns/routing/spectral-clustering-routing.en.md +88 -0
- package/design-patterns/routing/spectral-clustering-routing.md +90 -0
- package/knowledge-base/differential-geometry/connection.en.md +51 -0
- package/knowledge-base/differential-geometry/connection.md +51 -0
- package/knowledge-base/differential-geometry/curvature.en.md +49 -0
- package/knowledge-base/differential-geometry/curvature.md +49 -0
- package/knowledge-base/differential-geometry/geodesic.en.md +51 -0
- package/knowledge-base/differential-geometry/geodesic.md +51 -0
- package/knowledge-base/differential-geometry/manifold.en.md +47 -0
- package/knowledge-base/differential-geometry/manifold.md +47 -0
- package/knowledge-base/differential-geometry/metric-tensor.en.md +50 -0
- package/knowledge-base/differential-geometry/metric-tensor.md +50 -0
- package/knowledge-base/differential-geometry/tangent-space.en.md +48 -0
- package/knowledge-base/differential-geometry/tangent-space.md +48 -0
- package/knowledge-base/information-geometry/fisher-metric.en.md +56 -0
- package/knowledge-base/information-geometry/fisher-metric.md +56 -0
- package/knowledge-base/information-geometry/natural-gradient.en.md +54 -0
- package/knowledge-base/information-geometry/natural-gradient.md +54 -0
- package/knowledge-base/lie-theory/equivariance.en.md +54 -0
- package/knowledge-base/lie-theory/equivariance.md +54 -0
- package/knowledge-base/lie-theory/group-action.en.md +51 -0
- package/knowledge-base/lie-theory/group-action.md +51 -0
- package/knowledge-base/lie-theory/lie-algebra.en.md +51 -0
- package/knowledge-base/lie-theory/lie-algebra.md +51 -0
- package/knowledge-base/lie-theory/lie-group.en.md +51 -0
- package/knowledge-base/lie-theory/lie-group.md +51 -0
- package/knowledge-base/lie-theory/representation.en.md +52 -0
- package/knowledge-base/lie-theory/representation.md +52 -0
- package/knowledge-base/matrix-analysis/low-rank-approximation.en.md +49 -0
- package/knowledge-base/matrix-analysis/low-rank-approximation.md +49 -0
- package/knowledge-base/matrix-analysis/matrix-perturbation.en.md +49 -0
- package/knowledge-base/matrix-analysis/matrix-perturbation.md +49 -0
- package/knowledge-base/matrix-analysis/positive-semidefinite.en.md +50 -0
- package/knowledge-base/matrix-analysis/positive-semidefinite.md +50 -0
- package/knowledge-base/matrix-analysis/projection.en.md +47 -0
- package/knowledge-base/matrix-analysis/projection.md +47 -0
- package/knowledge-base/matrix-analysis/spectral-decomposition.en.md +48 -0
- package/knowledge-base/matrix-analysis/spectral-decomposition.md +48 -0
- package/knowledge-base/optimization/constrained-optimization.en.md +53 -0
- package/knowledge-base/optimization/constrained-optimization.md +53 -0
- package/knowledge-base/optimization/convex-optimization.en.md +50 -0
- package/knowledge-base/optimization/convex-optimization.md +50 -0
- package/knowledge-base/optimization/lagrangian-duality.en.md +50 -0
- package/knowledge-base/optimization/lagrangian-duality.md +50 -0
- package/knowledge-base/optimization/proximal-method.en.md +52 -0
- package/knowledge-base/optimization/proximal-method.md +52 -0
- package/knowledge-base/optimization/riemannian-optimization.en.md +51 -0
- package/knowledge-base/optimization/riemannian-optimization.md +51 -0
- package/knowledge-base/overview.en.md +29 -218
- package/knowledge-base/overview.md +29 -220
- package/knowledge-base/probability/concentration-inequality.en.md +45 -0
- package/knowledge-base/probability/concentration-inequality.md +45 -0
- package/knowledge-base/probability/entropy.en.md +47 -0
- package/knowledge-base/probability/entropy.md +47 -0
- package/knowledge-base/probability/fisher-information.en.md +48 -0
- package/knowledge-base/probability/fisher-information.md +48 -0
- package/knowledge-base/probability/information-bottleneck.en.md +49 -0
- package/knowledge-base/probability/information-bottleneck.md +49 -0
- package/knowledge-base/probability/kl-divergence.en.md +51 -0
- package/knowledge-base/probability/kl-divergence.md +51 -0
- package/knowledge-base/topology/euler-characteristic.en.md +54 -0
- package/knowledge-base/topology/euler-characteristic.md +54 -0
- package/knowledge-base/topology/fundamental-group.en.md +52 -0
- package/knowledge-base/topology/fundamental-group.md +52 -0
- package/knowledge-base/topology/persistent-homology.en.md +54 -0
- package/knowledge-base/topology/persistent-homology.md +54 -0
- package/lenses/algorithmic.en.md +49 -0
- package/lenses/algorithmic.md +49 -0
- package/lenses/axiomatization.en.md +48 -0
- package/lenses/axiomatization.md +48 -0
- package/lenses/categorical.en.md +48 -0
- package/lenses/categorical.md +48 -0
- package/lenses/causal.en.md +49 -0
- package/lenses/causal.md +49 -0
- package/lenses/duality.en.md +47 -0
- package/lenses/duality.md +47 -0
- package/lenses/game.en.md +50 -0
- package/lenses/game.md +50 -0
- package/lenses/geometric.en.md +53 -0
- package/lenses/geometric.md +53 -0
- package/lenses/local-to-global.en.md +52 -0
- package/lenses/local-to-global.md +52 -0
- package/lenses/perturbation.en.md +51 -0
- package/lenses/perturbation.md +51 -0
- package/lenses/probabilistic.en.md +48 -0
- package/lenses/probabilistic.md +48 -0
- package/lenses/projection.en.md +50 -0
- package/lenses/projection.md +50 -0
- package/lenses/spectral.en.md +52 -0
- package/lenses/spectral.md +52 -0
- package/lenses/symmetry.en.md +49 -0
- package/lenses/symmetry.md +49 -0
- package/lenses/topological.en.md +48 -0
- package/lenses/topological.md +48 -0
- package/lenses/variational.en.md +48 -0
- package/lenses/variational.md +48 -0
- package/package.json +9 -3
- package/references/books/abstract-algebra.md +5 -5
- package/references/books/algebraic-geometry-rising-sea.md +5 -5
- package/references/books/differential-geometry.md +3 -3
- package/references/books/matrix-analysis.md +4 -4
- package/references/books/micro-lie-theory.md +3 -3
- package/references/books/optimization-ml.md +1 -1
- package/references/books/smooth-manifolds.md +3 -3
- package/references/gpu-friendly-math.en.md +8 -8
- package/references/gpu-friendly-math.md +2 -2
- package/references/inspiration.en.md +1 -1
- package/references/inspiration.md +2 -2
- package/skills/math-research-activator/SKILL.en.md +161 -106
- package/skills/math-research-activator/SKILL.md +162 -110
- package/skills/math-research-activator/original-texts.en.md +9 -9
- package/skills/math-research-activator/original-texts.md +11 -11
- package/commands/abstraction.md +0 -11
- package/commands/algorithmic-thinking.md +0 -11
- package/commands/axiomatization.md +0 -11
- package/commands/causal-inference.md +0 -11
- package/commands/discrete-combinatorial.md +0 -11
- package/commands/game-theory.md +0 -11
- package/commands/induction-analogy.md +0 -11
- package/commands/information-theory.md +0 -11
- package/commands/logic-deduction.md +0 -11
- package/commands/modeling.md +0 -11
- package/commands/optimization.md +0 -11
- package/commands/probability-statistics.md +0 -11
- package/commands/symmetry-invariance.md +0 -11
- package/commands/topological-thinking.md +0 -11
- package/commands/transformation.md +0 -11
- package/skills/abstraction/SKILL.en.md +0 -117
- package/skills/abstraction/SKILL.md +0 -121
- package/skills/abstraction/original-texts.en.md +0 -163
- package/skills/abstraction/original-texts.md +0 -175
- package/skills/algorithmic-thinking/SKILL.en.md +0 -132
- package/skills/algorithmic-thinking/SKILL.md +0 -138
- package/skills/algorithmic-thinking/original-texts.en.md +0 -253
- package/skills/algorithmic-thinking/original-texts.md +0 -256
- package/skills/axiomatization/SKILL.en.md +0 -144
- package/skills/axiomatization/SKILL.md +0 -151
- package/skills/axiomatization/original-texts.en.md +0 -154
- package/skills/axiomatization/original-texts.md +0 -160
- package/skills/causal-inference/SKILL.en.md +0 -147
- package/skills/causal-inference/SKILL.md +0 -151
- package/skills/causal-inference/original-texts.en.md +0 -136
- package/skills/causal-inference/original-texts.md +0 -167
- package/skills/discrete-combinatorial/SKILL.en.md +0 -124
- package/skills/discrete-combinatorial/SKILL.md +0 -131
- package/skills/discrete-combinatorial/original-texts.en.md +0 -184
- package/skills/discrete-combinatorial/original-texts.md +0 -185
- package/skills/game-theory/SKILL.en.md +0 -117
- package/skills/game-theory/SKILL.md +0 -123
- package/skills/game-theory/original-texts.en.md +0 -131
- package/skills/game-theory/original-texts.md +0 -131
- package/skills/induction-analogy/SKILL.en.md +0 -145
- package/skills/induction-analogy/SKILL.md +0 -152
- package/skills/induction-analogy/original-texts.en.md +0 -140
- package/skills/induction-analogy/original-texts.md +0 -154
- package/skills/information-theory/SKILL.en.md +0 -134
- package/skills/information-theory/SKILL.md +0 -140
- package/skills/information-theory/original-texts.en.md +0 -127
- package/skills/information-theory/original-texts.md +0 -127
- package/skills/logic-deduction/SKILL.en.md +0 -130
- package/skills/logic-deduction/SKILL.md +0 -135
- package/skills/logic-deduction/original-texts.en.md +0 -160
- package/skills/logic-deduction/original-texts.md +0 -173
- package/skills/modeling/SKILL.en.md +0 -135
- package/skills/modeling/SKILL.md +0 -139
- package/skills/modeling/original-texts.en.md +0 -162
- package/skills/modeling/original-texts.md +0 -165
- package/skills/optimization/SKILL.en.md +0 -129
- package/skills/optimization/SKILL.md +0 -135
- package/skills/optimization/original-texts.en.md +0 -167
- package/skills/optimization/original-texts.md +0 -168
- package/skills/probability-statistics/SKILL.en.md +0 -146
- package/skills/probability-statistics/SKILL.md +0 -151
- package/skills/probability-statistics/original-texts.en.md +0 -191
- package/skills/probability-statistics/original-texts.md +0 -193
- package/skills/symmetry-invariance/SKILL.en.md +0 -135
- package/skills/symmetry-invariance/SKILL.md +0 -139
- package/skills/symmetry-invariance/original-texts.en.md +0 -206
- package/skills/symmetry-invariance/original-texts.md +0 -221
- package/skills/topological-thinking/SKILL.en.md +0 -124
- package/skills/topological-thinking/SKILL.md +0 -128
- package/skills/topological-thinking/original-texts.en.md +0 -134
- package/skills/topological-thinking/original-texts.md +0 -164
- package/skills/transformation/SKILL.en.md +0 -120
- package/skills/transformation/SKILL.md +0 -124
- package/skills/transformation/original-texts.en.md +0 -204
- package/skills/transformation/original-texts.md +0 -216
|
@@ -1,132 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: algorithmic-thinking
|
|
3
|
-
description: |
|
|
4
|
-
Trigger when a problem needs finite-step solving, algorithm design, complexity analysis (time/space), tractability & parallelism assessment (P/NP-hard/undecidable); or termination/correctness proofs; or designing executable algorithms/operators for GPU (assessing complexity order, parallelism, memory access & fusion).
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
# 🖥️ Algorithmic Thinking
|
|
8
|
-
|
|
9
|
-
> "An algorithm is the automation of thought — turning insight into precisely repeatable steps."
|
|
10
|
-
>
|
|
11
|
-
> — Computability Theory, Algorithm Analysis, Complexity Theory
|
|
12
|
-
|
|
13
|
-
## Core Principle
|
|
14
|
-
|
|
15
|
-
**Break problems into executable steps, assess cost, judge feasibility. The first principle is not "how to solve" but "whether it is solvable and at what cost" — some problems are inherently hard (NP-complete), some inherently unsolvable (undecidable); knowing these limits is as important as finding solutions.**
|
|
16
|
-
|
|
17
|
-
> **Mathematical Formalization**
|
|
18
|
-
>
|
|
19
|
-
> A Turing machine M: Σ* → Σ* halts within a finite number of steps. Time complexity T(n) = the maximum number of execution steps for an input of length n; space complexity S(n) = the maximum number of cells used. Church–Turing thesis: anything computable is computable by a Turing machine.
|
|
20
|
-
>
|
|
21
|
-
> Three-tier hierarchy: computable (an algorithm exists) → tractable (a polynomial-time algorithm exists) → intractable (only exponential-time or no algorithm). Computable but intractable is practically equivalent to uncomputable — brute force 2^n is worthless for n=100. P vs NP is precisely the boundary between tractable and intractable.
|
|
22
|
-
>
|
|
23
|
-
> Complexity notation: O(f(n)) upper bound, Ω(f(n)) lower bound, Θ(f(n)) tight bound. Common recurrences: T(n)=2T(n/2)+O(n) → O(n log n); T(n)=T(n-1)+O(n) → O(n²); T(n)=T(n/2)+O(1) → O(log n). Complexity classes: O(1) ⊂ O(log n) ⊂ O(n) ⊂ O(n log n) ⊂ O(n^k) ⊂ O(2^n) ⊂ O(n!).
|
|
24
|
-
>
|
|
25
|
-
> Complexity class definitions: P = polynomial-time decidable; NP-hard = at least as hard as every problem in NP; undecidable = no algorithm can decide it. NP-hard response: approximation (with error bounds) or heuristics (with probabilistic guarantees); undecidable response: find a decidable restricted version.
|
|
26
|
-
>
|
|
27
|
-
> Loop invariant: a property that holds before and after every iteration, which together with the termination condition implies output correctness. Hoare logic: {P} S {Q} — precondition P, after statement S, yields postcondition Q.
|
|
28
|
-
|
|
29
|
-
## GPU-Friendliness (Cross-Cutting Check)
|
|
30
|
-
|
|
31
|
-
Algorithmic thinking is the weapon most directly relevant to GPU — complexity order, parallelism, memory access/communication, and operator fusion directly determine whether an algorithm can run efficiently on GPU. Every algorithm must pass the `../../references/gpu-friendly-math.md` eight-dimension gate:
|
|
32
|
-
|
|
33
|
-
- **Complexity order**: Sub-quadratic O(n log n)/O(n) is friendly (fully utilizes parallel cores); O(n²) is borderline (GEMM is an exception, salvageable by Tensor Cores); O(2^n)/O(n!) is an anti-pattern.
|
|
34
|
-
- **Parallelism and communication**: Divide-and-conquer / data parallelism is friendly (sub-problems are independent, batchable); dependency chains in dynamic programming and serial search in backtracking require adaptation (chunking / skew scanning).
|
|
35
|
-
- **Memory and KV-Cache / Parallelism and communication**: Coalesced access is friendly; butterfly reductions and prefix scans have dependency patterns that require attention to bandwidth; random scatter/gather is unfriendly.
|
|
36
|
-
- **Operator fusion**: Multiple elementwise steps can be fused into a single kernel; algorithms that repeatedly read/write to memory — "beautiful but intractable" — should be rearranged into a single pass.
|
|
37
|
-
|
|
38
|
-
Eight-dimension minimum criteria (formal terms): **Tensorization** requires the main loop to be batchable; **GEMM-mappability** prioritizes expressing the core computation as GEMM/conv/scan; **complexity** must specify both time and space order; **memory and KV-cache** checks whether intermediate tables, state, and cache can be chunked; **low-precision stability** checks reduction, sorting, and iteration errors; **parallelism and communication** checks dependency graphs and cross-device synchronization; **sparse structure** accepts only structured sparsity; **operator fusion** requires reducing small kernels and memory round-trips.
|
|
39
|
-
|
|
40
|
-
| Algorithm characteristic | GPU classification | Notes |
|
|
41
|
-
|---|---|---|
|
|
42
|
-
| Divide-and-conquer / Merge / FFT | Friendly | Sub-quadratic, batchable, regular structure |
|
|
43
|
-
| Dynamic programming (dependency chains) | Adaptable | Chunking / skew scanning breaks serial dependencies |
|
|
44
|
-
| Backtracking / DFS serial search | Anti-pattern | Unpredictable branching, cannot fill SIMD lanes |
|
|
45
|
-
| Brute force O(2^n) | Anti-pattern | Complexity order itself is intractable |
|
|
46
|
-
|
|
47
|
-
> Used together with `../../references/books/matrix-analysis.md` (GEMM / low-rank decomposition for order reduction), `../../references/books/optimization-ml.md` (first-order / second-order method feasibility), `../../references/books/abstract-algebra.md` (parallel algorithms over semirings / finite fields).
|
|
48
|
-
|
|
49
|
-
## When NOT to Use
|
|
50
|
-
|
|
51
|
-
- **The problem has a closed-form solution and does not require step-by-step computation** — e.g., the quadratic formula directly gives the answer; algorithmic treatment only adds complexity.
|
|
52
|
-
- **The problem is qualitative rather than procedural** — e.g., "Is this structure elegant?" or "Is this proof insightful?" cannot be reduced to finite steps.
|
|
53
|
-
- **The input is unstructured and cannot be discretized** — e.g., a continuous perceptual data stream cannot directly serve as algorithm input; when preprocessing is more complex than the core problem, algorithmic treatment may not be optimal.
|
|
54
|
-
|
|
55
|
-
## When to Use
|
|
56
|
-
|
|
57
|
-
- Automating a repetitive process — the same operation must be performed on large volumes of data; manual execution does not scale.
|
|
58
|
-
- Verifying program termination — proving the algorithm halts in a finite number of steps is the cornerstone of reliability.
|
|
59
|
-
- Estimating computational cost — evaluating time/space consumption before large-scale runs to avoid resource exhaustion.
|
|
60
|
-
- Facing combinatorial explosion — the search space grows exponentially with input size, requiring pruning strategies.
|
|
61
|
-
- Judging problem feasibility — determining P / NP-hard / undecidable to decide on a solution strategy.
|
|
62
|
-
- Designing efficient solution procedures — from brute force to elegant algorithms, efficiency differences can span several orders of magnitude.
|
|
63
|
-
- **Designing executable algorithms/operators for GPU** — assessing complexity order, parallelism, memory access, and fusion so the algorithm is truly tractable on hardware.
|
|
64
|
-
|
|
65
|
-
## Method
|
|
66
|
-
|
|
67
|
-
### Step 1: Formalize I/O Spec
|
|
68
|
-
Define the input domain, output domain, and constraints. The input specification states data types (integer/floating-point/string/graph), scale range n∈[1,10^6], and validity conditions (preconditions); the output specification states the return type, precision, and boundary handling; constraints cover time/space limits and correctness guarantees (postconditions). Test: if you cannot write down the precondition/postcondition predicates, the specification is not yet formal enough.
|
|
69
|
-
|
|
70
|
-
### Step 2: Decompose into Sub-problems
|
|
71
|
-
Break the large problem into independently solvable pieces with clear boundaries and well-defined interfaces. Decomposition strategies: by data (processing different subsets), by function (completing different subtasks), by level (coarse-to-fine refinement). Good decomposition makes each sub-problem independently verifiable and the whole recursively verifiable; dependencies between sub-problems must be explicitly annotated to avoid implicit coupling.
|
|
72
|
-
|
|
73
|
-
### Step 3: Design the Procedure
|
|
74
|
-
Choose an algorithmic paradigm — this is the core decision; choosing wrong wastes all subsequent effort:
|
|
75
|
-
|
|
76
|
-
- **Divide-and-Conquer**: Partition, solve sub-parts, merge. Sub-problems are independent; merge cost is lower than solving the whole. Classics: merge sort, quicksort, Strassen matrix multiplication. Recurrence T(n)=aT(n/b)+f(n), solved by the Master Theorem.
|
|
77
|
-
- **Dynamic Programming (DP)**: Overlapping sub-problems, memoization to avoid repetition. Optimal substructure holds. Classics: shortest paths, knapsack, sequence alignment.
|
|
78
|
-
- **Greedy**: Locally optimal at each step, hoping for global optimality. Greedy-choice property holds. Classics: Dijkstra, Huffman, MST.
|
|
79
|
-
- **Backtracking**: Systematic search, pruning and retreating upon hitting dead ends. Search space has structure. Classics: N-queens, exact TSP, CSP.
|
|
80
|
-
- **Randomized**: Probabilistic shortcuts, trading randomness for efficiency. Deterministic cost is too high. Classics: randomized quicksort, Monte Carlo, Las Vegas.
|
|
81
|
-
|
|
82
|
-
Selection principle: independent → divide-and-conquer; overlapping → DP; greedy property → greedy; structured search → backtracking; deterministic cost too high → randomized. Practical problems often mix multiple paradigms.
|
|
83
|
-
|
|
84
|
-
### Step 4: Analyze Complexity
|
|
85
|
-
Time O(f(n)), space O(g(n)), distinguishing worst case from average case. Identify basic operation counts, solve recurrences (Master Theorem), and attend to how input distributions affect average-case complexity; beyond O (upper bound), attend to Ω (lower bound) and Θ (tight bound). Complexity analysis is an ongoing assessment during design, not post-hoc verification. Practical significance: for n=10^6, O(n) ≈ 10^6 steps (seconds), O(n²) ≈ 10^12 steps (infeasible), O(2^n) far exceeds the age of the universe.
|
|
86
|
-
|
|
87
|
-
### Step 5: Prove Correctness
|
|
88
|
-
Use loop invariants, structural induction, termination proofs, and pre/postconditions. An unverified algorithm is a guess. Loop invariant: initialization (holds before the loop), maintenance (preserved by each iteration), termination (invariant plus termination condition yields the postcondition); structural induction: assume correctness of sub-calls and prove the current call; termination: construct a decreasing measure and prove it strictly decreases; Hoare logic: {P} S {Q}.
|
|
89
|
-
|
|
90
|
-
### Step 6: Check Tractability
|
|
91
|
-
Is the problem in P? NP-hard? Undecidable? P → pursue the optimal exact algorithm; NP-hard → approximation/heuristics with explicit error bounds or probabilistic guarantees; undecidable → find a decidable restricted version. Common NP-hard problems: TSP, graph coloring, subset sum, SAT; common undecidable problems: the halting problem, Hilbert's tenth problem.
|
|
92
|
-
|
|
93
|
-
### Step 7: Optimize and Improve
|
|
94
|
-
Reduce constant factors, leverage data structures, parallelize — based on bottleneck analysis, not blind tuning. Optimization levels: (1) Algorithm level — switch to a superior paradigm (O(n²) → O(n log n) yields the largest gain); (2) Data structure level — switch to better storage/query structures; (3) Implementation level — caching, parallelism, SIMD, operator fusion; (4) Problem level — reformulate for easier solving (may yield qualitative leaps). Priority: algorithm level > data structure level > implementation level; problem level may enable leaps. Re-verify correctness after every optimization.
|
|
95
|
-
|
|
96
|
-
## Common Errors
|
|
97
|
-
|
|
98
|
-
| Error | Critique | Correct Approach |
|
|
99
|
-
|-------|----------|-----------------|
|
|
100
|
-
| Assuming exponential algorithms are acceptable | NP-hard problems require heuristics for large n, not exact solutions; 2^50 steps at 1ns each takes 35 years | Use approximation/heuristics for large n with explicit error bounds |
|
|
101
|
-
| Failing to prove termination | Infinite loops are a real risk; hanging is far more dangerous than slow results | Prove termination via a decreasing measure; construct a decreasing function on a well-ordered set |
|
|
102
|
-
| Confusing worst case with average case | Quicksort is worst-case O(n²) but average O(n log n); the worst case can occur in practice | Clearly state the analysis scenario and note assumptions |
|
|
103
|
-
| Confusing existence with constructibility | A solution may exist but be uncomputable; existence ≠ constructibility | Distinguish existence proofs from algorithmic constructions |
|
|
104
|
-
| Underestimating constant factors | O(n log n) with a huge constant may be slower than O(n²) with a small constant | Benchmark empirically; attend to constants and cache effects |
|
|
105
|
-
| Ignoring space complexity | A time-efficient algorithm may exhaust memory; DFS needs O(n) but BFS needs O(2^n) | Analyze both time and space |
|
|
106
|
-
| Premature optimization | Optimizing before proving correctness; the direction may be entirely wrong | Prove correctness first → analyze complexity → then optimize |
|
|
107
|
-
| Ignoring input distribution | Average-case complexity depends on distributional assumptions; random input ≠ real input | State distributional assumptions explicitly; analyze real-world scenarios |
|
|
108
|
-
| Serial / unfusable algorithms on GPU | Unpredictable branching, serial dependency chains, repeated memory reads/writes — "beautiful but intractable" | Adapt to parallel / fusible / sub-quadratic form; pass the GPU eight-dimension gate |
|
|
109
|
-
|
|
110
|
-
## Operating Procedure
|
|
111
|
-
|
|
112
|
-
When this skill is triggered, the output must include:
|
|
113
|
-
|
|
114
|
-
1. **Input specification**: `[Input]: [description]` — e.g., `[Input]: unsorted integer array A[1..n], n∈[1,10^6]`
|
|
115
|
-
2. **Output specification**: `[Output]: [description]` — e.g., `[Output]: A sorted in ascending order`
|
|
116
|
-
3. **Algorithm paradigm**: `[Paradigm]: [divide-and-conquer/DP/greedy/backtracking/randomized] because [reason]`
|
|
117
|
-
4. **Complexity**: `[Time]: O([f(n)]) [Space]: O([g(n)]) [Scenario]: [worst/average]`
|
|
118
|
-
5. **Correctness**: `[Loop invariant/induction strategy]: [content] ✅ proved / ❌ not proved`
|
|
119
|
-
6. **Tractability**: `[Class]: [P/NP-hard/undecidable] [Response]: [exact/approximate/heuristic/reformulate]`
|
|
120
|
-
7. **Improvement suggestions**: `[Optimization direction]: [specific suggestions]`
|
|
121
|
-
8. **[GPU feasibility]** (if used for algorithm/operator design) — complexity order / parallelism / memory access / fusion assessment, annotate as friendly / retrofittable / anti-pattern + adaptation suggestions
|
|
122
|
-
|
|
123
|
-
**The output must not present analysis alone without a conclusion.**
|
|
124
|
-
|
|
125
|
-
## Relations to Other Skills
|
|
126
|
-
|
|
127
|
-
- **Optimization thinking**: Gradient descent, branch-and-bound, and other optimization algorithms are instances of algorithmic thinking — optimization focuses on the optimal value, while algorithms focus on the steps and cost to reach it.
|
|
128
|
-
- **Transformation thinking**: FFT is a paradigmatic application of algorithmic thinking to transformations — O(n²) → O(n log n); transformations map problems to easily solvable spaces, and algorithms solve them efficiently in the transformed space.
|
|
129
|
-
- **Logical deduction**: Correctness proofs use deductive reasoning (loop invariants, induction) — an algorithm correctness proof is essentially a chain of deductive steps.
|
|
130
|
-
- **Induction and analogy**: Divide-and-conquer uses structural analogy — sub-problems are isomorphic to the original problem — which is precisely the algorithmic embodiment of inductive thinking.
|
|
131
|
-
- **Discrete / Combinatorial thinking**: Algorithms process discrete objects, and combinatorial counting underpins complexity analysis — combinatorial explosion is the core obstacle algorithmic thinking must overcome.
|
|
132
|
-
- **Modern mathematics activation**: `../../references/books/matrix-analysis.md` (GEMM / low-rank decomposition for order reduction), `../../references/books/optimization-ml.md` (GPU feasibility of first-order / second-order methods), `../../references/books/abstract-algebra.md` (parallel algorithms over semirings / finite fields).
|
|
@@ -1,138 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: algorithmic-thinking
|
|
3
|
-
description: |
|
|
4
|
-
触发:问题需要有限步骤求解、算法设计、复杂度分析(时间/空间)、可行性与并行性评估(P/NP-hard/不可判定);或需证明终止性/正确性;或为 GPU 设计可执行算法/算子(评估复杂度阶、并行度、访存与融合)时调用。
|
|
5
|
-
English: Trigger when a problem needs finite-step solving, algorithm design, complexity analysis (time/space), tractability & parallelism assessment (P/NP-hard/undecidable); or termination/correctness proofs; or designing executable algorithms/operators for GPU (assessing complexity order, parallelism, memory access & fusion).
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
> **语言路由**:若用户消息为英文,请读取并遵循同目录下的 `SKILL.en.md`,按其操作规程以英文输出;中文消息则继续使用本文件。
|
|
9
|
-
|
|
10
|
-
# 🖥️ 算法与计算思想 / Algorithmic Thinking
|
|
11
|
-
|
|
12
|
-
> "算法是思想的自动化——将洞察转化为可重复执行的精确步骤。"
|
|
13
|
-
> "An algorithm is the automation of thought — turning insight into precisely repeatable steps."
|
|
14
|
-
>
|
|
15
|
-
> —— 计算理论、算法分析、复杂度理论 / Computability Theory, Algorithm Analysis, Complexity Theory
|
|
16
|
-
|
|
17
|
-
## 核心原则 / Core Principle
|
|
18
|
-
|
|
19
|
-
**把问题拆解为可执行步骤,评估代价,判断可行性。算法思想的第一原则不是"如何求解",而是"是否可解、代价几何"——某些问题本质困难(NP 完全),某些本质不可解(不可判定),了解这些极限与找到解同样重要。**
|
|
20
|
-
|
|
21
|
-
**Break problems into executable steps, assess cost, judge feasibility. The first principle is not "how to solve" but "whether it is solvable and at what cost" — some problems are inherently hard (NP-complete), some inherently unsolvable (undecidable); knowing these limits is as important as finding solutions.**
|
|
22
|
-
|
|
23
|
-
> **数学形式化 / Mathematical Formalization**
|
|
24
|
-
>
|
|
25
|
-
> 图灵机 M: Σ* → Σ* 在有限步骤内停机。时间复杂度 T(n) = 长度为 n 的输入的最大执行步数;空间复杂度 S(n) = 最大使用单元数。丘奇-图灵论题:任何可计算的都可由图灵机计算。
|
|
26
|
-
>
|
|
27
|
-
> 三级层次:可计算(算法存在)→ 可行(多项式时间算法存在)→ 难解(仅指数时间或无算法)。可计算但不可行在实践中等同不可计算——蛮力 2^n 对 n=100 毫无价值。P vs NP 正是可行与难解的分界。
|
|
28
|
-
>
|
|
29
|
-
> 复杂度符号:O(f(n)) 上界,Ω(f(n)) 下界,Θ(f(n)) 精确界。常见递推:T(n)=2T(n/2)+O(n)→O(n log n);T(n)=T(n-1)+O(n)→O(n²);T(n)=T(n/2)+O(1)→O(log n)。复杂度类:O(1)⊂O(log n)⊂O(n)⊂O(n log n)⊂O(n^k)⊂O(2^n)⊂O(n!)。
|
|
30
|
-
>
|
|
31
|
-
> 复杂度类定义:P = 多项式时间可判定;NP-hard = 至少与所有 NP 问题同等困难;不可判定 = 无任何算法可判定。NP-hard 应对:近似(误差界限)或启发式(概率保证);不可判定应对:寻找可判定的受限版本。
|
|
32
|
-
>
|
|
33
|
-
> 循环不变量:每次迭代前后成立的性质,加终止条件蕴含输出正确。Hoare 逻辑:{P} S {Q}——前置条件 P 经语句 S 产生后置条件 Q。
|
|
34
|
-
|
|
35
|
-
## GPU 友好性 / GPU-Friendliness(横切检查)
|
|
36
|
-
|
|
37
|
-
算法思想是与 GPU 最直接相关的武器——复杂度阶、并行度、访存/通信、算子融合直接决定一个算法能否在 GPU 上高效跑起来。每个算法须过 `../../references/gpu-friendly-math.md` 八维门:
|
|
38
|
-
|
|
39
|
-
- **复杂度阶**:亚二次 O(n log n)/O(n) 友好(吃满并行核);O(n²) 边界(GEMM 例外,Tensor Core 可救);O(2^n)/O(n!) 反模式。
|
|
40
|
-
- **并行与通信**:分治/数据并行友好(子问题独立、可批量);动规的依赖链、回溯的串行搜索需改造(分块/斜扫描)。
|
|
41
|
-
- **显存与 KV-Cache / 并行与通信**:coalesced 访问友好;butterfly 归约、前缀扫描的依赖需注意带宽;随机 scatter/gather 不友好。
|
|
42
|
-
- **算子融合**:多步 elementwise 可融合为单 kernel;反复读写显存的"美丽但不可算"算法应重排为一次过。
|
|
43
|
-
|
|
44
|
-
八维最低判定(正式术语):**张量化**要求主循环可批量化;**GEMM 可映射**优先把核心计算写成 GEMM/conv/scan;**复杂度**必须给时间与空间阶;**显存与 KV-Cache**检查中间表、状态、缓存是否可分块;**低精度稳定**检查归约、排序、迭代误差;**并行与通信**检查依赖图与跨设备同步;**稀疏结构**只接受结构化稀疏;**算子融合**要求减少小 kernel 与往返显存。
|
|
45
|
-
|
|
46
|
-
| 算法特征 | GPU 分类 | 说明 |
|
|
47
|
-
|---|---|---|
|
|
48
|
-
| 分治/归并/FFT | 友好 | 亚二次、可批量、结构规则 |
|
|
49
|
-
| 动规(依赖链) | 可改造 | 分块/斜扫描打破串行依赖 |
|
|
50
|
-
| 回溯/DFS 串行搜索 | 反模式 | 分支不可预测、无法填满 SIMD |
|
|
51
|
-
| 蛮力 O(2^n) | 反模式 | 复杂度阶本身不可算 |
|
|
52
|
-
|
|
53
|
-
> 配合 `../../references/books/matrix-analysis.md`(GEMM/低秩分解降阶)、`../../references/books/optimization-ml.md`(一阶/二阶法可行性)、`../../references/books/abstract-algebra.md`(半环/有限域上的并行算法)。
|
|
54
|
-
|
|
55
|
-
## 不适用场景 / When NOT to Use
|
|
56
|
-
|
|
57
|
-
- **问题有闭式解,不需要逐步计算**——如二次方程求根公式直接给答案,算法化反增复杂度。
|
|
58
|
-
- **问题本质是定性的而非程序性的**——如"这个结构是否优美""这个证明是否有洞察力"无法化为有限步骤。
|
|
59
|
-
- **输入无结构、无法离散化**——如连续感知数据流无法直接作为算法输入;预处理比核心问题更复杂时算法化未必最优。
|
|
60
|
-
|
|
61
|
-
## 何时使用 / When to Use
|
|
62
|
-
|
|
63
|
-
- 需要自动化重复过程——同一操作对大量数据反复执行,手动不可扩展。
|
|
64
|
-
- 需要验证程序终止——证明算法在有限步内停机,是可靠性的基石。
|
|
65
|
-
- 需要估算计算代价——大规模运行前评估时间/空间消耗,避免资源耗尽。
|
|
66
|
-
- 面对组合爆炸——搜索空间随输入指数增长,需剪枝策略。
|
|
67
|
-
- 需要判断问题可行性——确定 P / NP-hard / 不可判定,决定求解策略。
|
|
68
|
-
- 需要设计高效求解步骤——从蛮力到精巧算法,效率差异可达数个量级。
|
|
69
|
-
- **为 GPU 设计可执行算法/算子**——评估复杂度阶、并行度、访存与融合,使算法在硬件上真正可算。
|
|
70
|
-
|
|
71
|
-
## 方法流程 / Method
|
|
72
|
-
|
|
73
|
-
### 第一步:形式化输入输出规格 / Formalize I/O Spec
|
|
74
|
-
定义输入域、输出域、约束条件。输入规格明确数据类型(整数/浮点/字符串/图)、规模范围 n∈[1,10^6]、合法性条件(前置条件);输出规格明确返回类型、精度、边界处理;约束覆盖时间/空间限制与正确性保证(后置条件)。检验:若写不出前置/后置条件谓词,则规格尚未足够形式化。
|
|
75
|
-
|
|
76
|
-
### 第二步:分解子问题 / Decompose into Sub-problems
|
|
77
|
-
将大问题拆为可独立求解的小块,子问题边界清晰、接口明确。分解策略:按数据(处理不同子集)、按功能(完成不同子任务)、按层次(先粗后细逐层精化)。好的分解使每个子问题可独立验证,整体可递归验证;子问题间依赖须显式标注,避免隐式耦合。
|
|
78
|
-
|
|
79
|
-
### 第三步:设计程序 / Design the Procedure
|
|
80
|
-
选择算法范式——核心决策,选错则后续努力白费:
|
|
81
|
-
|
|
82
|
-
- **分治 (Divide-and-Conquer)**:分割、求解子部分、合并。子问题独立、合并代价低于整体。经典:归并/快速排序、Strassen 矩阵乘。递推 T(n)=aT(n/b)+f(n) 由主定理求解。
|
|
83
|
-
- **动态规划 (DP)**:重叠子问题、备忘录化避免重复。最优子结构成立。经典:最短路、背包、序列对齐。
|
|
84
|
-
- **贪心 (Greedy)**:每步局部最优,期望全局。贪心选择性质成立。经典:Dijkstra、Huffman、MST。
|
|
85
|
-
- **回溯 (Backtracking)**:系统性搜索、遇死路剪枝回退。搜索空间有结构。经典:N 皇后、TSP 精确解、CSP。
|
|
86
|
-
- **随机化 (Randomized)**:概率捷径,以随机换效率。确定性代价过高。经典:随机快排、Monte Carlo、Las Vegas。
|
|
87
|
-
|
|
88
|
-
选择原则:独立→分治、重叠→动规、贪心性质→贪心、结构化搜索→回溯、确定性代价过高→随机化。实际问题常混合多种范式。
|
|
89
|
-
|
|
90
|
-
### 第四步:分析复杂度 / Analyze Complexity
|
|
91
|
-
时间 O(f(n))、空间 O(g(n)),区分最坏与平均。识别基本操作计数、求解递推(主定理)、关注输入分布对平均复杂度的影响;除 O(上界)外关注 Ω(下界)与 Θ(精确界)。复杂度分析是设计中的持续评估,非事后验证。实际意义:n=10^6 时 O(n)≈10^6 步(秒级),O(n²)≈10^12 步(不可行),O(2^n) 远超宇宙年龄。
|
|
92
|
-
|
|
93
|
-
### 第五步:证明正确性 / Prove Correctness
|
|
94
|
-
使用循环不变量、结构归纳、终止证明、前置/后置条件。未经证明的算法是猜测。循环不变量:初始化(循环前成立)、维护(每次迭代保持)、终止(不变量加终止条件给后置条件);结构归纳:假设子调用正确证当前调用;终止:构造递减度量证其严格递减;Hoare 逻辑:{P} S {Q}。
|
|
95
|
-
|
|
96
|
-
### 第六步:检查可行性 / Check Tractability
|
|
97
|
-
问题属 P?NP-hard?不可判定?P → 追求最优精确算法;NP-hard → 近似/启发式,明确误差界限或概率保证;不可判定 → 寻找可判定的受限版本。常见 NP-hard:TSP、图着色、子集和、SAT;常见不可判定:停机问题、希尔伯特第十问题。
|
|
98
|
-
|
|
99
|
-
### 第七步:优化与改进 / Optimize and Improve
|
|
100
|
-
减少常数因子、利用数据结构、并行化,基于瓶颈分析而非盲目调参。优化层次:(1) 算法级——换更优范式(O(n²)→O(n log n) 收益最大);(2) 数据结构级——换更优存储/查询;(3) 实现级——缓存、并行、SIMD、算子融合;(4) 问题级——换表述使更易求解(可能质的飞跃)。优先级:算法级 > 数据结构级 > 实现级,问题级可能跃迁。每次优化后重新验证正确性。
|
|
101
|
-
|
|
102
|
-
## 常见错误 / Common Errors
|
|
103
|
-
|
|
104
|
-
| 错误 / Error | 批评 / Critique | 正确做法 / Correct Approach |
|
|
105
|
-
|---|---|---|
|
|
106
|
-
| 假设指数算法可接受 | NP-hard 对大 n 需启发式而非精确;2^50 步即使每步 1ns 需 35 年 | 对大 n 用近似/启发式,明确误差界限 |
|
|
107
|
-
| 未证明终止性 | 无限循环是真实风险,挂起远比慢结果危险 | 用递减度量证终止,构造良序集上递减函数 |
|
|
108
|
-
| 混淆最坏与平均情况 | 快排最坏 O(n²) 但平均 O(n log n),最坏可能在实中出现 | 明确分析场景,注明假设 |
|
|
109
|
-
| 混淆存在性与可构造性 | 解可能存在但不可计算;存在性≠可构造性 | 区分存在性证明与算法构造 |
|
|
110
|
-
| 低估常数因子 | 巨大常数的 O(n log n) 可能比小常数 O(n²) 慢 | 实测比较,关注常数与缓存效应 |
|
|
111
|
-
| 忽略空间复杂度 | 时间高效的算法可能爆内存;DFS 需 O(n) 但 BFS 需 O(2^n) | 同时分析时间与空间 |
|
|
112
|
-
| 过早优化 | 未证正确就优化,方向可能全错 | 先证正确→析复杂度→再优化 |
|
|
113
|
-
| 忽视输入分布 | 平均复杂度依赖分布假设,随机输入≠实际输入 | 标明分布假设,分析实际场景 |
|
|
114
|
-
| 串行/不可融合算法上 GPU | 分支不可预测、依赖链串行、反复读写显存——"美但不可算" | 改造为并行/可融合/亚二次,过 GPU 八维门 |
|
|
115
|
-
|
|
116
|
-
## 操作规程 / Operating Procedure
|
|
117
|
-
|
|
118
|
-
当本 skill 被触发时,输出必须包含:
|
|
119
|
-
|
|
120
|
-
1. **输入规格**:`[输入]: [描述]` — 例如:`[输入]: 无序整数数组 A[1..n],n∈[1,10^6]`
|
|
121
|
-
2. **输出规格**:`[输出]: [描述]` — 例如:`[输出]: A 的升序排列`
|
|
122
|
-
3. **算法范式**:`[范式]: [分治/动规/贪心/回溯/随机化] 因为 [理由]`
|
|
123
|
-
4. **复杂度**:`[时间]: O([f(n)]) [空间]: O([g(n)]) [场景]: [最坏/平均]`
|
|
124
|
-
5. **正确性**:`[循环不变量/归纳策略]: [内容] ✅ 已证 / ❌ 未证`
|
|
125
|
-
6. **可行性**:`[类别]: [P/NP-hard/不可判定] [应对]: [精确/近似/启发式/重新表述]`
|
|
126
|
-
7. **改进建议**:`[优化方向]: [具体建议]`
|
|
127
|
-
8. **[GPU 可行性]**(若用于算法/算子设计)— 复杂度阶/并行度/访存/融合评估,标注友好/可改造/反模式 + 改造建议
|
|
128
|
-
|
|
129
|
-
**输出不得只给分析而无结论。**
|
|
130
|
-
|
|
131
|
-
## 与其他 skill 的关系 / Relations to Other Skills
|
|
132
|
-
|
|
133
|
-
- **优化思想**:梯度下降、分支定界等优化算法是算法思想的实例——优化关注最优值,算法关注达到最优值的步骤与代价。
|
|
134
|
-
- **变换思想**:FFT 是算法思想应用于变换的典范——O(n²)→O(n log n);变换把问题映到易解空间,算法在变换空间高效求解。
|
|
135
|
-
- **逻辑演绎**:正确性证明使用演绎推理(循环不变量、归纳)——算法正确性证明本质是一串演绎步骤。
|
|
136
|
-
- **归纳与类比**:分治用结构类比——子问题与原问题同构——正是归纳思想的算法化。
|
|
137
|
-
- **离散/组合思想**:算法处理离散对象,组合计数支撑复杂度分析——组合爆炸是算法思维要克服的核心障碍。
|
|
138
|
-
- **现代数学激活**:`../../references/books/matrix-analysis.md`(GEMM/低秩分解降阶)、`../../references/books/optimization-ml.md`(一阶/二阶法 GPU 可行性)、`../../references/books/abstract-algebra.md`(半环/有限域上的并行算法)。
|
|
@@ -1,253 +0,0 @@
|
|
|
1
|
-
# Algorithmic Thinking -- Original Texts & Historical Context
|
|
2
|
-
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
## 1. Turing Machine (1936)
|
|
6
|
-
|
|
7
|
-
Alan Turing's paper **"On Computable Numbers, with an Application to the Entscheidungsproblem"** (1936) defined a precise mathematical model of "computability" -- the Turing machine.
|
|
8
|
-
|
|
9
|
-
> "The 'computable' numbers may be described briefly as the real numbers whose expressions as a decimal are calculable by finite means."
|
|
10
|
-
> -- Alan Turing, 1936
|
|
11
|
-
|
|
12
|
-
**Core definition:**
|
|
13
|
-
- A Turing machine $M = (Q, \Sigma, \Gamma, \delta, q_0, q_{\text{accept}}, q_{\text{reject}})$, where $\delta: Q \times \Gamma \to Q \times \Gamma \times \{L, R\}$ is the transition function.
|
|
14
|
-
- Universal Turing machine: can simulate the execution of any Turing machine $M$ on input $w$, i.e., $U(\langle M \rangle, w) = M(w)$.
|
|
15
|
-
|
|
16
|
-
**Halting problem:**
|
|
17
|
-
- The halting problem $H = \{\langle M, w \rangle : M \text{ halts on } w\}$ is undecidable.
|
|
18
|
-
- Proof: Suppose $H$ is decidable. Construct $D(\langle M \rangle) = \text{run } H(\langle M, \langle M \rangle \rangle)$; if $M$ halts on $\langle M \rangle$ then $D$ loops forever, otherwise $D$ halts. Applying $D(\langle D \rangle)$ produces a contradiction.
|
|
19
|
-
|
|
20
|
-
**Entscheidungsproblem (Decision Problem):**
|
|
21
|
-
- Proposed by Hilbert in 1928: Does there exist an algorithm to decide the validity of first-order logic statements?
|
|
22
|
-
- Turing and Church independently proved: No such algorithm exists. The validity of first-order logic is undecidable.
|
|
23
|
-
|
|
24
|
-
---
|
|
25
|
-
|
|
26
|
-
## 2. Church-Turing Thesis
|
|
27
|
-
|
|
28
|
-
**Thesis (not a theorem):**
|
|
29
|
-
> Every "effectively calculable" function is Turing-computable.
|
|
30
|
-
|
|
31
|
-
**Equivalent formulations:**
|
|
32
|
-
- Church: $\lambda$-definable functions (lambda calculus, 1936)
|
|
33
|
-
- Godel-Herbrand-Kleene: general recursive functions
|
|
34
|
-
- Post: Post canonical systems
|
|
35
|
-
- The sets of functions computable by all these models are identical, supporting the plausibility of the thesis.
|
|
36
|
-
|
|
37
|
-
**Strong Church-Turing thesis:**
|
|
38
|
-
- Any problem solvable on a physical device in time $T$ can be solved on a Turing machine in time $O(T^k)$ ($k$ a constant).
|
|
39
|
-
- That is: all reasonable computational models are equivalent up to polynomial time.
|
|
40
|
-
- Quantum computing challenges this -- does there exist a polynomial speedup with $k > 1$?
|
|
41
|
-
|
|
42
|
-
---
|
|
43
|
-
|
|
44
|
-
## 3. Euclid's Algorithm (~300 BC)
|
|
45
|
-
|
|
46
|
-
**The oldest algorithm still in use.**
|
|
47
|
-
|
|
48
|
-
**GCD computation:**
|
|
49
|
-
$$\gcd(a, b) = \gcd(b, a \bmod b), \quad a \bmod b \neq 0$$
|
|
50
|
-
$$\gcd(a, 0) = a$$
|
|
51
|
-
|
|
52
|
-
**Termination proof:**
|
|
53
|
-
- At each step $b' = a \bmod b < b$, so the sequence $b_0 > b_1 > b_2 > \cdots \geq 0$ is strictly decreasing.
|
|
54
|
-
- Within finitely many steps $b_k = 0$ and the algorithm terminates.
|
|
55
|
-
|
|
56
|
-
**Extended Euclidean algorithm:**
|
|
57
|
-
- Computes $\gcd(a, b)$ while simultaneously finding $x, y$ such that $ax + by = \gcd(a, b)$.
|
|
58
|
-
- Application: modular inverse $a^{-1} \bmod m$ -- when $\gcd(a, m) = 1$, $ax \equiv 1 \pmod{m}$ and $x = a^{-1}$.
|
|
59
|
-
|
|
60
|
-
**Time complexity:**
|
|
61
|
-
- Lame's theorem (1844): The number of steps in Euclid's algorithm is at most $5 \times$ the number of decimal digits of the smaller number, i.e., $O(\log \min(a, b))$.
|
|
62
|
-
|
|
63
|
-
---
|
|
64
|
-
|
|
65
|
-
## 4. Knuth -- The Art of Computer Programming (1968-)
|
|
66
|
-
|
|
67
|
-
Donald Knuth's multi-volume magnum opus **TAOCP** systematized the field of algorithm analysis.
|
|
68
|
-
|
|
69
|
-
> "The process of preparing programs for a digital computer is especially attractive, not only because it can be economically and scientifically rewarding, but also because it can be an aesthetic experience much like composing poetry or music."
|
|
70
|
-
> -- Donald Knuth, TAOCP Vol. 1, Preface
|
|
71
|
-
|
|
72
|
-
**Contributions:**
|
|
73
|
-
- Introduced and popularized big-O notation from number theory into computer science. $O(f(n))$: there exist constants $c, n_0$ such that $T(n) \leq c \cdot f(n)$ for all $n \geq n_0$.
|
|
74
|
-
- A systematic taxonomy of algorithms (sorting, searching, arithmetic, combinatorial, etc.).
|
|
75
|
-
- MMIX/MIX mixed pseudo-machines for precise models of algorithm analysis.
|
|
76
|
-
- Volumes: Vol. 1 (Fundamental Algorithms, 1968), Vol. 2 (Seminumerical Algorithms, 1969), Vol. 3 (Sorting and Searching, 1973), Vol. 4A (Combinatorial Algorithms, 2011).
|
|
77
|
-
|
|
78
|
-
---
|
|
79
|
-
|
|
80
|
-
## 5. Cook-Levin Theorem (1971)
|
|
81
|
-
|
|
82
|
-
**NP-completeness:**
|
|
83
|
-
|
|
84
|
-
> "The question 'is P = NP?' is one of the most important open questions in theoretical computer science."
|
|
85
|
-
> -- Stephen Cook, 1971
|
|
86
|
-
|
|
87
|
-
**The Cook-Levin theorem:**
|
|
88
|
-
- SAT (the Boolean satisfiability problem) is NP-complete.
|
|
89
|
-
- Proof: The accepting computation of a nondeterministic Turing machine $M$ for any NP language $L$ on input $w$ can be encoded as a SAT formula $\varphi_{M,w}$; $w \in L \iff \varphi_{M,w}$ is satisfiable.
|
|
90
|
-
- Reduction: $L \leq_p \text{SAT}$ holds for all $L \in \text{NP}$.
|
|
91
|
-
|
|
92
|
-
**Karp's 21 NP-complete problems (1972):**
|
|
93
|
-
- Richard Karp, in "Reducibility among Combinatorial Problems," proved 21 classic problems to be NP-complete, including:
|
|
94
|
-
- 3SAT, CLIQUE, VERTEX-COVER, HAM-CYCLE, TSP, SUBSET-SUM, PARTITION, etc.
|
|
95
|
-
|
|
96
|
-
**P vs NP:**
|
|
97
|
-
- $\text{P} = \text{NP}$? If true, all NP problems would have polynomial-time algorithms. The prevailing conjecture is $\text{P} \neq \text{NP}$. One of the Clay Mathematics Institute Millennium Prize Problems.
|
|
98
|
-
|
|
99
|
-
---
|
|
100
|
-
|
|
101
|
-
## 6. Divide-and-Conquer
|
|
102
|
-
|
|
103
|
-
**Key idea:** Decompose the problem into subproblems, solve recursively, and combine results.
|
|
104
|
-
|
|
105
|
-
**Canonical examples:**
|
|
106
|
-
- **Merge sort:** $T(n) = 2T(n/2) + O(n) \Rightarrow T(n) = O(n \log n)$. A stable sort.
|
|
107
|
-
- **Quicksort:** Average $O(n \log n)$, worst case $O(n^2)$. Hoare (1962).
|
|
108
|
-
- **Binary search:** $T(n) = T(n/2) + O(1) \Rightarrow T(n) = O(\log n)$. Requires a sorted array.
|
|
109
|
-
|
|
110
|
-
**Master theorem:**
|
|
111
|
-
$$T(n) = a\,T(n/b) + f(n)$$
|
|
112
|
-
- Case 1: If $f(n) = O(n^{\log_b a - \varepsilon})$, then $T(n) = \Theta(n^{\log_b a})$.
|
|
113
|
-
- Case 2: If $f(n) = \Theta(n^{\log_b a} \log^k n)$, then $T(n) = \Theta(n^{\log_b a} \log^{k+1} n)$.
|
|
114
|
-
- Case 3: If $f(n) = \Omega(n^{\log_b a + \varepsilon})$ and $af(n/b) \leq cf(n)$, then $T(n) = \Theta(f(n))$.
|
|
115
|
-
|
|
116
|
-
---
|
|
117
|
-
|
|
118
|
-
## 7. Dynamic Programming
|
|
119
|
-
|
|
120
|
-
> "I decided to use the word 'dynamic' ... it was something not even a Congressman could object to."
|
|
121
|
-
> -- Richard Bellman, on naming "dynamic programming" (1957)
|
|
122
|
-
|
|
123
|
-
**Core principles:**
|
|
124
|
-
1. **Optimal substructure:** An optimal solution contains optimal solutions to subproblems.
|
|
125
|
-
2. **Overlapping subproblems:** Different subproblems share smaller sub-subproblems, so naive recursion redundantly recomputes them.
|
|
126
|
-
|
|
127
|
-
**Classic algorithms:**
|
|
128
|
-
- **Shortest paths:** Dijkstra (1959) -- single-source shortest paths, $O((V+E)\log V)$.
|
|
129
|
-
- **Floyd-Warshall (1962):** All-pairs shortest paths, $O(V^3)$. Recurrence: $d_{ij}^{(k)} = \min(d_{ij}^{(k-1)}, d_{ik}^{(k-1)} + d_{kj}^{(k-1)})$.
|
|
130
|
-
- **Knapsack problem:** $O(nW)$ pseudo-polynomial time.
|
|
131
|
-
|
|
132
|
-
**Bottom-up vs Top-down (memoization):**
|
|
133
|
-
- Bottom-up: Fill in a table from the smallest subproblems to the largest.
|
|
134
|
-
- Top-down + memoization: Recurse but cache the results of already-solved subproblems.
|
|
135
|
-
|
|
136
|
-
---
|
|
137
|
-
|
|
138
|
-
## 8. Greedy Algorithms
|
|
139
|
-
|
|
140
|
-
**Key idea:** At each step, choose the locally optimal option, hoping to reach a globally optimal solution.
|
|
141
|
-
|
|
142
|
-
**Classic algorithms:**
|
|
143
|
-
- **Kruskal's minimum spanning tree (1956):** Add edges in ascending order of weight, skipping those that form a cycle. $O(E \log E)$. Greedy is optimal for MST problems.
|
|
144
|
-
- **Huffman coding (1952):** Repeatedly merge the two nodes with the lowest frequencies. Produces an optimal prefix code.
|
|
145
|
-
|
|
146
|
-
**When greedy works:**
|
|
147
|
-
- **Matroid structure:** If the set of feasible solutions forms a matroid $(E, \mathcal{I})$ -- satisfying the hereditary property and the exchange property -- then the greedy algorithm yields an optimal solution for any weight function.
|
|
148
|
-
- Exchange property: If $A, B \in \mathcal{I}$ and $|A| < |B|$, then there exists $e \in B \setminus A$ such that $A \cup \{e\} \in \mathcal{I}$.
|
|
149
|
-
|
|
150
|
-
---
|
|
151
|
-
|
|
152
|
-
## 9. Backtracking and Search
|
|
153
|
-
|
|
154
|
-
**Depth-First Search (DFS) / Breadth-First Search (BFS):**
|
|
155
|
-
- DFS: Uses a stack (or recursion), explores as deep as possible first; used for topological sorting and connected components.
|
|
156
|
-
- BFS: Uses a queue, expands level by level; used for shortest paths in unweighted graphs.
|
|
157
|
-
|
|
158
|
-
**Backtracking:**
|
|
159
|
-
- Systematically enumerates candidate solutions, retreating as soon as a constraint is violated.
|
|
160
|
-
- Examples: N-Queens problem, subset enumeration, permutation enumeration.
|
|
161
|
-
|
|
162
|
-
**Pruning and branch-and-bound:**
|
|
163
|
-
- Pruning: Uses constraints to eliminate impossible branches early, reducing the search space.
|
|
164
|
-
- Branch-and-bound: For optimization problems, maintains a bound on the current best solution and prunes branches that cannot improve it. Commonly used for TSP and integer programming.
|
|
165
|
-
|
|
166
|
-
**Constraint Satisfaction Problems (CSP):**
|
|
167
|
-
- A set of variables + domains + constraints. Backtracking search + constraint propagation (AC-3, etc.).
|
|
168
|
-
|
|
169
|
-
---
|
|
170
|
-
|
|
171
|
-
## 10. Randomized Algorithms
|
|
172
|
-
|
|
173
|
-
**Two types:**
|
|
174
|
-
- **Las Vegas algorithms:** Always produce correct results; running time is random. Example: randomized quicksort.
|
|
175
|
-
- **Monte Carlo algorithms:** Running time is deterministic; result may be incorrect (but error probability can be made arbitrarily small). Example: Miller-Rabin primality test.
|
|
176
|
-
|
|
177
|
-
**Miller-Rabin primality test (1976/1980):**
|
|
178
|
-
- For an odd number $n$, choose a random $a \in \{2, \ldots, n-1\}$ and check whether the sequence $a^{d} \bmod n$ satisfies the strengthened conditions of Fermat's little theorem.
|
|
179
|
-
- Error probability per test $\leq 1/4$; after $k$ independent tests, error probability $\leq 4^{-k}$.
|
|
180
|
-
- Time complexity $O(k \log^3 n)$.
|
|
181
|
-
|
|
182
|
-
**Randomized quicksort:**
|
|
183
|
-
- Randomly select a pivot; expected number of comparisons $2n \ln n \approx 1.39 n \log_2 n$; expected running time $O(n \log n)$.
|
|
184
|
-
- The probability of the worst case $O(n^2)$ can be shown to be extremely low.
|
|
185
|
-
|
|
186
|
-
---
|
|
187
|
-
|
|
188
|
-
## 11. Decidability and Undecidability
|
|
189
|
-
|
|
190
|
-
**The halting problem is undecidable (see Section 1).**
|
|
191
|
-
|
|
192
|
-
**Rice's theorem (1953):**
|
|
193
|
-
> Every non-trivial semantic property of Turing machines is undecidable.
|
|
194
|
-
|
|
195
|
-
- "Semantic property": depends only on the function computed by $M$, not on the structure of $M$.
|
|
196
|
-
- "Non-trivial": holds for some Turing machines but not for others.
|
|
197
|
-
- Examples: "Does $M$ compute a constant function?", "Does $M$ halt on some input?", "Does $M$ compute a decidable language?" -- all undecidable.
|
|
198
|
-
|
|
199
|
-
**Post Correspondence Problem (PCP, 1946):**
|
|
200
|
-
- Given a set of dominos $\{(t_1, b_1), \ldots, (t_k, b_k)\}$ (where $t_i, b_i$ are strings), does there exist a sequence $i_1, i_2, \ldots, i_m$ such that $t_{i_1} t_{i_2} \cdots t_{i_m} = b_{i_1} b_{i_2} \cdots b_{i_m}$?
|
|
201
|
-
- PCP is undecidable. Commonly used to prove the undecidability of other problems.
|
|
202
|
-
|
|
203
|
-
---
|
|
204
|
-
|
|
205
|
-
## 12. Complexity Classes
|
|
206
|
-
|
|
207
|
-
| Class | Definition | Description |
|
|
208
|
-
|-------|-----------|-------------|
|
|
209
|
-
| **P** | $\{L : L \text{ is decidable in } O(n^k) \text{ time}\}$ | Decidable in polynomial time |
|
|
210
|
-
| **NP** | $\{L : L \text{ has a polynomial-length certificate}\}$ | Decidable in nondeterministic polynomial time |
|
|
211
|
-
| **co-NP** | $\{L : \bar{L} \in \text{NP}\}$ | Complement of NP |
|
|
212
|
-
| **EXP** | $\{L : L \text{ is decidable in } O(2^{n^k}) \text{ time}\}$ | Decidable in exponential time |
|
|
213
|
-
| **R** | $\{L : L \text{ is decidable}\}$ | Recursive / decidable languages |
|
|
214
|
-
| **RE** | $\{L : L \text{ is recognizable by a Turing machine}\}$ | Recursively enumerable languages |
|
|
215
|
-
|
|
216
|
-
**Hierarchies:**
|
|
217
|
-
$$\text{P} \subseteq \text{NP} \subseteq \text{PSPACE} \subseteq \text{EXP} \subseteq \text{R} \subseteq \text{RE}$$
|
|
218
|
-
$$\text{P} \neq \text{EXP} \quad (\text{time hierarchy theorem, Hartmanis-Stearns 1965})$$
|
|
219
|
-
$$\text{R} \neq \text{RE} \quad (\text{halting problem} \in \text{RE} \setminus \text{R})$$
|
|
220
|
-
|
|
221
|
-
---
|
|
222
|
-
|
|
223
|
-
## 13. Sorting Lower Bound
|
|
224
|
-
|
|
225
|
-
**Lower bound for comparison-based sorting:**
|
|
226
|
-
|
|
227
|
-
$$T(n) \geq \Omega(n \log n)$$
|
|
228
|
-
|
|
229
|
-
**Information-theoretic argument:**
|
|
230
|
-
- The number of permutations of $n$ elements $= n!$; each comparison halves the search space at most.
|
|
231
|
-
- The decision tree height $h$ must satisfy $2^h \geq n!$, so $h \geq \log_2(n!) = \Omega(n \log n)$.
|
|
232
|
-
- By Stirling's formula: $\log_2(n!) \approx n \log_2 n - n \log_2 e + O(\log n)$.
|
|
233
|
-
|
|
234
|
-
**Breaking the bound:**
|
|
235
|
-
- Non-comparison sorts can surpass $\Omega(n \log n)$: counting sort $O(n+k)$, radix sort $O(d(n+k))$ -- but require special structure in the input.
|
|
236
|
-
|
|
237
|
-
---
|
|
238
|
-
|
|
239
|
-
## 14. Algorithm Design Paradigms -- Summary Table
|
|
240
|
-
|
|
241
|
-
| Paradigm | Key Idea | When to Use | Canonical Example |
|
|
242
|
-
|----------|---------|-------------|-------------------|
|
|
243
|
-
| Divide-and-Conquer | Decompose -> recurse -> combine | Problem naturally decomposes into independent subproblems | Merge sort, Binary search |
|
|
244
|
-
| Dynamic Programming | Overlapping subproblems + optimal substructure; fill a table to avoid recomputation | Severe subproblem overlap with optimal substructure | Floyd-Warshall, Knapsack |
|
|
245
|
-
| Greedy | Choose locally optimal at each step | Matroid structure or provable greedy correctness | Kruskal MST, Huffman coding |
|
|
246
|
-
| Backtracking | Systematic search + pruning | Search space is finite but large | N-Queens, CSP |
|
|
247
|
-
| Branch-and-Bound | Search + maintain upper/lower bounds for pruning | Optimization problems where bounds on solutions can be computed | TSP (exact), Integer programming |
|
|
248
|
-
| Randomized | Introduce randomness to reduce expected cost or error probability | Probabilistic guarantees or expected efficiency needed | Miller-Rabin, Randomized quicksort |
|
|
249
|
-
| Reduction | Transform a problem into a known problem | An efficient algorithm already exists for the target problem | SAT -> 3SAT (Cook-Levin) |
|
|
250
|
-
|
|
251
|
-
---
|
|
252
|
-
|
|
253
|
-
*This file provides mathematical references and historical context for the algorithmic-thinking skill, covering core results from ancient Greece to modern computational theory.*
|