numopt-js 0.3.0 → 0.4.0

package/README.md

@@ -11,6 +11,10 @@ A flexible numerical optimization library for JavaScript/TypeScript that works s
 
 - **Gradient Descent**: Simple, robust optimization algorithm with line search support
 - **Line Search**: Backtracking line search with Armijo condition for optimal step sizes (following Nocedal & Wright, *Numerical Optimization* (2nd ed.), Algorithm 3.1)
+- **Strong Wolfe Line Search**: Robust line search satisfying Strong Wolfe conditions (recommended for quasi-Newton methods)
+- **BFGS**: Quasi-Newton method that updates a dense inverse Hessian approximation (unconstrained smooth optimization)
+- **L-BFGS**: Memory-efficient quasi-Newton method (two-loop recursion) for larger parameter counts
+- **CMA-ES**: Derivative-free black-box optimization using a covariance-adapting search distribution (unconstrained)
 - **Gauss-Newton Method**: Efficient method for nonlinear least squares problems
 - **Levenberg-Marquardt Algorithm**: Robust algorithm combining Gauss-Newton with damping
 - **Constrained Gauss-Newton**: Efficient constrained nonlinear least squares using effective Jacobian
@@ -32,6 +36,35 @@ A flexible numerical optimization library for JavaScript/TypeScript that works s
 npm install numopt-js
 ```
 
+## Start Here (Pick Your Path)
+
+- **Minimize a scalar cost** (smooth unconstrained optimization): use **Gradient Descent** (`cost: (p) => number`, `grad: (p) => Float64Array`). Start at [Gradient Descent](#gradient-descent).
+- **Minimize a scalar cost (faster than GD for many problems)**: use **BFGS** or **L-BFGS** (`cost: (p) => number`, `grad: (p) => Float64Array`). Start at [BFGS / L-BFGS](#bfgs--l-bfgs).
+- **Minimize a scalar cost (black-box, no gradient)**: use **CMA-ES** (`cost: (p) => number`). Start at [CMA-ES](#cma-es-black-box-optimization).
+- **Fit a model with residuals** (nonlinear least squares): use **Levenberg–Marquardt** or **Gauss–Newton** (`residual: (p) => Float64Array`, optional `jacobian: (p) => Matrix`). Start at [Levenberg-Marquardt](#levenberg-marquardt-nonlinear-least-squares).
+- **Equality-constrained problems** \(c(p, x) = 0\): use **Adjoint** / **Constrained GN/LM** (`constraint: (p, x) => Float64Array`). Start at [Adjoint Method](#adjoint-method-constrained-optimization).
+- **Browser usage**: start at [Browser Usage](#browser-usage).
+
+**Why `Float64Array`?** This library uses `Float64Array` for predictable numeric performance. You can always convert from normal arrays with `new Float64Array([1, 2, 3])` (more details below).
+
+## Cost Function vs Residual Function (Important)
+
+- **Cost function**: `cost(p) -> number` (used by `gradientDescent`)
+- **Residual function**: `residual(p) -> Float64Array` (used by `gaussNewton` / `levenbergMarquardt`), where the library minimizes \(f(p) = 1/2 \|r(p)\|^2\)
+
+## Result Object (What to Look At)
+
+Most algorithms return a `result` with these fields:
+
+- **Common (all algorithms)**: `finalParameters`, `converged`, `iterations`, `finalCost`
+- **Gradient Descent**: `finalGradientNorm`, `usedLineSearch`
+- **BFGS / L-BFGS**: `finalGradientNorm`
+- **CMA-ES**: `functionEvaluations`, `finalStepSize`, `finalMaxStdDev`
+- **Gauss-Newton / Levenberg–Marquardt**: `finalResidualNorm` (and LM also has `finalLambda`)
+- **Constrained algorithms / Adjoint**: `finalStates`, `finalConstraintNorm`
+
+Note: `result.parameters` is a deprecated alias of `result.finalParameters` and will be removed in a future release.
+
 ## 60-second Quick Start (Node)
 
 Pick **one** of the following and run it.
@@ -55,6 +88,30 @@ const result = gradientDescent(new Float64Array([5, -3]), cost, grad, {
 console.log(result.finalParameters);
 ```
 
+## BFGS / L-BFGS
+
+Use these for smooth unconstrained problems when you can provide a gradient.
+
+```js
+import { bfgs, lbfgs } from 'numopt-js';
+
+const cost = (params) => (params[0] - 1) ** 2 + (params[1] + 2) ** 2;
+const grad = (params) => new Float64Array([2 * (params[0] - 1), 2 * (params[1] + 2)]);
+
+const bfgsResult = bfgs(new Float64Array([10, 10]), cost, grad, {
+  maxIterations: 200,
+  tolerance: 1e-8
+});
+
+const lbfgsResult = lbfgs(new Float64Array([10, 10]), cost, grad, {
+  maxIterations: 200,
+  tolerance: 1e-8,
+  historySize: 10
+});
+
+console.log(bfgsResult.finalParameters, lbfgsResult.finalParameters);
+```
+
 Run:
 
 ```bash
@@ -86,6 +143,48 @@ Run:
 node quick.cjs
 ```
 
+## CMA-ES (Black-box Optimization)
+
+Use CMA-ES when your cost function is a **black box** and you **don’t have a gradient**.
+
+Two important inputs:
+- `initialStepSize` (sigma0): your initial error guess / search radius
+- `randomSeed`: set this for reproducible runs (recommended for testing and debugging)
+- `restartStrategy`: use `"ipop"` for multi-modal problems (λ doubles per restart)
+- `profiling`: enable lightweight timing breakdown in the result
+
+```js
+import { cmaEs } from 'numopt-js';
+
+const sphere = (params) => params.reduce((sum, v) => sum + v * v, 0);
+
+const result = cmaEs(new Float64Array([10, -7, 3, 5]), sphere, {
+  maxIterations: 200,
+  populationSize: 20,
+  initialStepSize: 2.0,
+  randomSeed: 123456,
+  targetCost: 1e-10,
+  restartStrategy: "none",
+  profiling: true,
+});
+
+console.log(result.finalParameters, result.finalCost, result.stopReason, result.profiling);
+```
+
+### IPOP (Restart Strategy)
+
+For multi-modal problems (e.g., Rastrigin), use IPOP restarts:
+
+```js
+const result = cmaEs(new Float64Array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5]), sphere, {
+  maxIterations: 1200,
+  populationSize: 20,
+  initialStepSize: 2.5,
+  randomSeed: 123456,
+  restartStrategy: "ipop",
+});
+```
+
 ## Node Usage (ESM + CommonJS)
 
 numopt-js supports both ESM (`import`) and CommonJS (`require`) in Node.js.
@@ -113,6 +212,10 @@ const { gradientDescent } = require('numopt-js');
 
 numopt-js is designed to work seamlessly in browser environments. The library automatically provides a browser-optimized bundle that includes all dependencies.
 
+**Important**:
+- **Don’t use `file://`** for the import-maps / direct-import examples. Serve your files via a local static server (for example: `npx serve`, `python -m http.server`, or Vite) so ES modules load correctly.
+- **SSR frameworks (Next.js, etc.)**: run numopt-js on the client side. If you hit SSR errors, move the code into a client component (`"use client"`) or dynamically import it with SSR disabled.
+
 ### Option 1: Bundler (Recommended)
 
 If you're using a bundler (Vite/Webpack/Rollup), just import from the package and the bundler will resolve the browser build via `package.json` exports.
@@ -177,6 +280,15 @@ After installing dependencies with `npm install`, you can run the example script
 - `npm run example:constrained-gauss-newton` — constrained least squares via effective Jacobian
 - `npm run example:constrained-lm` — constrained Levenberg–Marquardt
 
+**Start with these (recommended reading order):**
+
+- `npm run example:gradient` — smallest end-to-end example (scalar cost + gradient)
+- `npm run example:rosenbrock` — shows why line search matters on a classic non-convex problem
+- `npm run example:lm` — first least-squares example (residuals, optional numeric Jacobian)
+- `npm run example:constrained-gauss-newton` — first constrained least-squares example
+- `npm run example:constrained-lm` — robust constrained least-squares (damping)
+- `npm run example:adjoint` — constrained optimization with states \(x\) and parameters \(p\)
+
 **Pick an algorithm:**
 
 - Gradient Descent — stable first choice for smooth problems (see below)
@@ -350,6 +462,10 @@ const gradientFn = createFiniteDiffGradient(costFn, { stepSize: 1e-8 });
 const gradient = finiteDiffGradient(params, costFn, { stepSize: 1e-8 });
 ```
 
+**Practical tips (finite differences)**:
+- **Scale your parameters** so typical values are around \(O(1)\). If one parameter is \(10^{-6}\) and another is \(10^{6}\), a single global `stepSize` will often fail.
+- If you work in physical units, consider **normalizing inputs/parameters** first, then convert back after optimization.
+
 ### Adjoint Method (Constrained Optimization)
 
 The adjoint method efficiently solves constrained optimization problems by solving for an adjoint variable λ instead of explicitly inverting matrices. This requires solving only one linear system per iteration, making it much more efficient than naive approaches.
@@ -636,7 +752,7 @@ Extends `ConstrainedGaussNewtonOptions` with:
 - `tolStep?: number` - Tolerance for step size convergence (default: 1e-6)
 - `tolResidual?: number` - Tolerance for residual norm convergence (default: 1e-6)
 
-**Note**: The constraint function `c(p, x)` does not need to return a vector with the same length as the state vector `x`. The adjoint method supports both square and non-square constraint Jacobians (overdetermined and underdetermined systems). For non-square matrices, the method uses QR decomposition or pseudo-inverse to solve the adjoint equation.
+**Note**: The constraint function `c(p, x)` does not need to return a vector with the same length as the state vector `x`. The constrained solvers support both square and non-square constraint Jacobians (overdetermined and underdetermined systems) by solving the relevant linear systems in a least-squares sense (with regularization when needed). If you see instability, try scaling/normalizing your states/constraints.
 
 #### Numerical Differentiation Options
 

package/dist/core/bfgs.d.ts

@@ -0,0 +1,16 @@
+/**
+ * This file implements the (dense) BFGS algorithm for unconstrained smooth optimization.
+ *
+ * Role in system:
+ * - Quasi-Newton optimizer for scalar cost functions with user-provided gradients
+ * - Uses Strong Wolfe line search to encourage curvature conditions needed for stable updates
+ * - Dense method: stores a full inverse Hessian approximation (O(n^2) memory)
+ *
+ * For first-time readers:
+ * - Start with `bfgs` (main entry point)
+ * - Then read `updateInverseHessianApproximation` (core BFGS update)
+ * - Finally, check safeguard helpers (descent direction / curvature checks)
+ */
+import type { BfgsOptions, CostFn, GradientFn, OptimizationResult } from './types.js';
+export declare function bfgs(initialParameters: Float64Array, costFunction: CostFn, gradientFunction: GradientFn, options?: BfgsOptions): OptimizationResult;
+//# sourceMappingURL=bfgs.d.ts.map

package/dist/core/bfgs.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"bfgs.d.ts","sourceRoot":"","sources":["../../src/core/bfgs.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAGH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,EAAE,UAAU,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAgItF,wBAAgB,IAAI,CAClB,iBAAiB,EAAE,YAAY,EAC/B,YAAY,EAAE,MAAM,EACpB,gBAAgB,EAAE,UAAU,EAC5B,OAAO,GAAE,WAAgB,GACxB,kBAAkB,CA2FpB"}

package/dist/core/bfgs.js

@@ -0,0 +1,167 @@
+/**
+ * This file implements the (dense) BFGS algorithm for unconstrained smooth optimization.
+ *
+ * Role in system:
+ * - Quasi-Newton optimizer for scalar cost functions with user-provided gradients
+ * - Uses Strong Wolfe line search to encourage curvature conditions needed for stable updates
+ * - Dense method: stores a full inverse Hessian approximation (O(n^2) memory)
+ *
+ * For first-time readers:
+ * - Start with `bfgs` (main entry point)
+ * - Then read `updateInverseHessianApproximation` (core BFGS update)
+ * - Finally, check safeguard helpers (descent direction / curvature checks)
+ */
+import { Matrix } from 'ml-matrix';
+import { strongWolfeLineSearch } from './lineSearch.js';
+import { Logger } from './logger.js';
+import { checkGradientConvergence, createConvergenceResult } from './convergence.js';
+import { addVectors, dotProduct, scaleVector, subtractVectors, vectorNorm } from '../utils/matrix.js';
+const DEFAULT_MAX_ITERATIONS = 1000;
+const DEFAULT_TOLERANCE = 1e-6;
+const DEFAULT_USE_LINE_SEARCH = true;
+const DEFAULT_FIXED_STEP_SIZE = 1.0;
+const INVALID_STEP_SIZE = 0.0;
+const NEGATIVE_GRADIENT_DIRECTION = -1.0;
+const MINIMUM_CURVATURE_THRESHOLD = 1e-10;
+function createIdentityMatrix(dimension) {
+    // NOTE: Provide both dimensions to stay compatible with our (older) local typing history
+    // and with ml-matrix's API where `columns` is optional.
+    return Matrix.eye(dimension, dimension);
+}
+function multiplyMatrixVector(matrix, vector) {
+    const result = new Float64Array(vector.length);
+    for (let rowIndex = 0; rowIndex < matrix.rows; rowIndex++) {
+        let sum = 0.0;
+        for (let columnIndex = 0; columnIndex < matrix.columns; columnIndex++) {
+            sum += matrix.get(rowIndex, columnIndex) * vector[columnIndex];
+        }
+        result[rowIndex] = sum;
+    }
+    return result;
+}
+function computeBfgsSearchDirection(inverseHessianApproximation, currentGradient) {
+    const approximateNewtonDirection = multiplyMatrixVector(inverseHessianApproximation, currentGradient);
+    return scaleVector(approximateNewtonDirection, NEGATIVE_GRADIENT_DIRECTION);
+}
+function ensureDescentDirectionOrFallback(currentGradient, proposedSearchDirection, currentInverseHessianApproximation, logger, iteration, currentCost) {
+    const directionalDerivative = dotProduct(currentGradient, proposedSearchDirection);
+    const isDescentDirection = directionalDerivative < 0.0;
+    if (isDescentDirection) {
+        return { searchDirection: proposedSearchDirection, inverseHessianApproximation: currentInverseHessianApproximation };
+    }
+    // WHY: If numerical issues yield a non-descent direction, reset H to identity and fall back to -g.
+    logger.warn('bfgs', iteration, 'Non-descent direction detected; resetting inverse Hessian and using negative gradient.', [
+        { key: 'Cost:', value: currentCost },
+        { key: 'Directional derivative:', value: directionalDerivative }
+    ]);
+    return {
+        searchDirection: scaleVector(currentGradient, NEGATIVE_GRADIENT_DIRECTION),
+        inverseHessianApproximation: createIdentityMatrix(currentGradient.length)
+    };
+}
+function updateInverseHessianApproximation(inverseHessianApproximation, stepVector, gradientChangeVector, logger, iteration, currentCost) {
+    const stepDotGradientChange = dotProduct(stepVector, gradientChangeVector);
+    const curvatureIsTooWeak = stepDotGradientChange <= MINIMUM_CURVATURE_THRESHOLD;
+    if (curvatureIsTooWeak) {
+        // WHY: If curvature is weak/negative, the BFGS update can break positive definiteness.
+        logger.warn('bfgs', iteration, 'Curvature condition too weak; resetting inverse Hessian approximation.', [
+            { key: 'Cost:', value: currentCost },
+            { key: 'stepDotGradientChange:', value: stepDotGradientChange }
+        ]);
+        return createIdentityMatrix(stepVector.length);
+    }
+    const curvatureScaling = 1.0 / stepDotGradientChange;
+    const stepMatrix = Matrix.columnVector(Array.from(stepVector));
+    const gradientChangeMatrix = Matrix.columnVector(Array.from(gradientChangeVector));
+    const identityMatrix = createIdentityMatrix(stepVector.length);
+    const stepGradientOuterProduct = stepMatrix.mmul(gradientChangeMatrix.transpose()).mul(curvatureScaling);
+    const gradientStepOuterProduct = gradientChangeMatrix.mmul(stepMatrix.transpose()).mul(curvatureScaling);
+    const leftFactor = identityMatrix.sub(stepGradientOuterProduct);
+    const rightFactor = identityMatrix.sub(gradientStepOuterProduct);
+    const rankTwoPart = leftFactor.mmul(inverseHessianApproximation).mmul(rightFactor);
+    const rankOnePart = stepMatrix.mmul(stepMatrix.transpose()).mul(curvatureScaling);
+    return rankTwoPart.add(rankOnePart);
+}
+function computeNextParameters(currentParameters, searchDirection, stepSize) {
+    const stepVector = scaleVector(searchDirection, stepSize);
+    return addVectors(currentParameters, stepVector);
+}
+function handleLineSearchFailure(currentParameters, iteration, currentCost, gradientNorm, logger) {
+    logger.warn('bfgs', iteration, 'Line search failed (non-descent direction).', [
+        { key: 'Cost:', value: currentCost },
+        { key: 'Gradient norm:', value: gradientNorm }
+    ]);
+    return {
+        finalParameters: currentParameters,
+        parameters: currentParameters,
+        iterations: iteration + 1,
+        converged: false,
+        finalCost: currentCost,
+        finalGradientNorm: gradientNorm
+    };
+}
+export function bfgs(initialParameters, costFunction, gradientFunction, options = {}) {
+    const maxIterations = options.maxIterations ?? DEFAULT_MAX_ITERATIONS;
+    const tolerance = options.tolerance ?? DEFAULT_TOLERANCE;
+    const useLineSearch = options.useLineSearch ?? DEFAULT_USE_LINE_SEARCH;
+    const fixedStepSize = options.stepSize ?? DEFAULT_FIXED_STEP_SIZE;
+    const onIteration = options.onIteration;
+    const logger = new Logger(options.logLevel, options.verbose);
+    let currentParameters = new Float64Array(initialParameters);
+    let currentCost = costFunction(currentParameters);
+    let inverseHessianApproximation = createIdentityMatrix(currentParameters.length);
+    for (let iteration = 0; iteration < maxIterations; iteration++) {
+        const currentGradient = gradientFunction(currentParameters);
+        const gradientNorm = vectorNorm(currentGradient);
+        if (onIteration)
+            onIteration(iteration, currentCost, currentParameters);
+        if (checkGradientConvergence(gradientNorm, tolerance, iteration)) {
+            logger.info('bfgs', iteration, 'Converged', [
+                { key: 'Cost:', value: currentCost },
+                { key: 'Gradient norm:', value: gradientNorm }
+            ]);
+            return createConvergenceResult(currentParameters, iteration, true, currentCost, gradientNorm);
+        }
+        const proposedSearchDirection = computeBfgsSearchDirection(inverseHessianApproximation, currentGradient);
+        const descentResult = ensureDescentDirectionOrFallback(currentGradient, proposedSearchDirection, inverseHessianApproximation, logger, iteration, currentCost);
+        const searchDirection = descentResult.searchDirection;
+        inverseHessianApproximation = descentResult.inverseHessianApproximation;
+        const stepSize = useLineSearch
+            ? strongWolfeLineSearch(costFunction, gradientFunction, currentParameters, searchDirection, options.lineSearchOptions)
+            : fixedStepSize;
+        if (stepSize === INVALID_STEP_SIZE) {
+            return handleLineSearchFailure(currentParameters, iteration, currentCost, gradientNorm, logger);
+        }
+        const newParameters = computeNextParameters(currentParameters, searchDirection, stepSize);
+        const stepVector = subtractVectors(newParameters, currentParameters);
+        const stepNorm = vectorNorm(stepVector);
+        const newCost = costFunction(newParameters);
+        const newGradient = gradientFunction(newParameters);
+        const gradientChangeVector = subtractVectors(newGradient, currentGradient);
+        inverseHessianApproximation = updateInverseHessianApproximation(inverseHessianApproximation, stepVector, gradientChangeVector, logger, iteration, newCost);
+        logger.debug('bfgs', iteration, 'Progress', [
+            { key: 'Cost:', value: currentCost },
+            { key: 'Gradient norm:', value: gradientNorm },
+            { key: 'Step size:', value: stepSize },
+            { key: 'Step norm:', value: stepNorm }
+        ]);
+        currentParameters = new Float64Array(newParameters);
+        currentCost = newCost;
+    }
+    const finalGradient = gradientFunction(currentParameters);
+    const finalGradientNorm = vectorNorm(finalGradient);
+    logger.warn('bfgs', undefined, 'Maximum iterations reached', [
+        { key: 'Iterations:', value: maxIterations },
+        { key: 'Final cost:', value: currentCost },
+        { key: 'Final gradient norm:', value: finalGradientNorm }
+    ]);
+    return {
+        finalParameters: currentParameters,
+        parameters: currentParameters,
+        iterations: maxIterations,
+        converged: false,
+        finalCost: currentCost,
+        finalGradientNorm: finalGradientNorm
+    };
+}
+//# sourceMappingURL=bfgs.js.map

package/dist/core/bfgs.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"bfgs.js","sourceRoot":"","sources":["../../src/core/bfgs.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,OAAO,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;AAEnC,OAAO,EAAE,qBAAqB,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AACrC,OAAO,EAAE,wBAAwB,EAAE,uBAAuB,EAAE,MAAM,kBAAkB,CAAC;AACrF,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,WAAW,EAAE,eAAe,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAEtG,MAAM,sBAAsB,GAAG,IAAI,CAAC;AACpC,MAAM,iBAAiB,GAAG,IAAI,CAAC;AAC/B,MAAM,uBAAuB,GAAG,IAAI,CAAC;AACrC,MAAM,uBAAuB,GAAG,GAAG,CAAC;AACpC,MAAM,iBAAiB,GAAG,GAAG,CAAC;AAC9B,MAAM,2BAA2B,GAAG,CAAC,GAAG,CAAC;AACzC,MAAM,2BAA2B,GAAG,KAAK,CAAC;AAE1C,SAAS,oBAAoB,CAAC,SAAiB;IAC7C,yFAAyF;IACzF,wDAAwD;IACxD,OAAO,MAAM,CAAC,GAAG,CAAC,SAAS,EAAE,SAAS,CAAC,CAAC;AAC1C,CAAC;AAED,SAAS,oBAAoB,CAAC,MAAc,EAAE,MAAoB;IAChE,MAAM,MAAM,GAAG,IAAI,YAAY,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IAC/C,KAAK,IAAI,QAAQ,GAAG,CAAC,EAAE,QAAQ,GAAG,MAAM,CAAC,IAAI,EAAE,QAAQ,EAAE,EAAE,CAAC;QAC1D,IAAI,GAAG,GAAG,GAAG,CAAC;QACd,KAAK,IAAI,WAAW,GAAG,CAAC,EAAE,WAAW,GAAG,MAAM,CAAC,OAAO,EAAE,WAAW,EAAE,EAAE,CAAC;YACtE,GAAG,IAAI,MAAM,CAAC,GAAG,CAAC,QAAQ,EAAE,WAAW,CAAC,GAAG,MAAM,CAAC,WAAW,CAAC,CAAC;QACjE,CAAC;QACD,MAAM,CAAC,QAAQ,CAAC,GAAG,GAAG,CAAC;IACzB,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,SAAS,0BAA0B,CAAC,2BAAmC,EAAE,eAA6B;IACpG,MAAM,0BAA0B,GAAG,oBAAoB,CAAC,2BAA2B,EAAE,eAAe,CAAC,CAAC;IACtG,OAAO,WAAW,CAAC,0BAA0B,EAAE,2BAA2B,CAAC,CAAC;AAC9E,CAAC;AAED,SAAS,gCAAgC,CACvC,eAA6B,EAC7B,uBAAqC,EACrC,kCAA0C,EAC1C,MAAc,EACd,SAAiB,EACjB,WAAmB;IAEnB,MAAM,qBAAqB,GAAG,UAAU,CAAC,eAAe,EAAE,uBAAuB,CAAC,CAAC;IACnF,MAAM,kBAAkB,GAAG,qBAAqB,GAAG,GAAG,CAAC;IACvD,IAAI,kBAAkB,EAAE,CAAC;QACvB,OAAO,EAAE,eAAe,EAAE,uBAAuB,EAAE,2BAA2B,EAAE,kCAAkC,EAAE,CAAC;IACvH,CAAC;IAED,mGAAmG;IACnG,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,wFAAwF,EAAE;QACvH,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,WAAW,EAAE;QACpC,EAAE,GAAG,EAAE,yBAAyB,EAAE,KAAK,EAAE,qBAAqB,EAAE;KACjE,CAAC,CAAC;IACH,OAAO;QACL,eAAe,EAAE,WAAW,CAAC,eAAe,EAAE,2BAA2B,CAAC;QAC1E,2BAA2B,EAAE,oBAAoB,CAAC,eAAe,CAAC,MAAM,CAAC;KAC1E,CAAC;AACJ,CAAC;AAED,SAAS,iCAAiC,CACxC,2BAAmC,EACnC,UAAwB,EACxB,oBAAkC,EAClC,MAAc,EACd,SAAiB,EACjB,WAAmB;IAEnB,MAAM,qBAAqB,GAAG,UAAU,CAAC,UAAU,EAAE,oBAAoB,CAAC,CAAC;IAC3E,MAAM,kBAAkB,GAAG,qBAAqB,IAAI,2BAA2B,CAAC;IAChF,IAAI,kBAAkB,EAAE,CAAC;QACvB,uFAAuF;QACvF,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,wEAAwE,EAAE;YACvG,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,WAAW,EAAE;YACpC,EAAE,GAAG,EAAE,wBAAwB,EAAE,KAAK,EAAE,qBAAqB,EAAE;SAChE,CAAC,CAAC;QACH,OAAO,oBAAoB,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC;IACjD,CAAC;IAED,MAAM,gBAAgB,GAAG,GAAG,GAAG,qBAAqB,CAAC;IACrD,MAAM,UAAU,GAAG,MAAM,CAAC,YAAY,CAAC,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC;IAC/D,MAAM,oBAAoB,GAAG,MAAM,CAAC,YAAY,CAAC,KAAK,CAAC,IAAI,CAAC,oBAAoB,CAAC,CAAC,CAAC;IAEnF,MAAM,cAAc,GAAG,oBAAoB,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC;IAC/D,MAAM,wBAAwB,GAAG,UAAU,CAAC,IAAI,CAAC,oBAAoB,CAAC,SAAS,EAAE,CAAC,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAC;IACzG,MAAM,wBAAwB,GAAG,oBAAoB,CAAC,IAAI,CAAC,UAAU,CAAC,SAAS,EAAE,CAAC,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAC;IAEzG,MAAM,UAAU,GAAG,cAAc,CAAC,GAAG,CAAC,wBAAwB,CAAC,CAAC;IAChE,MAAM,WAAW,GAAG,cAAc,CAAC,GAAG,CAAC,wBAAwB,CAAC,CAAC;IAEjE,MAAM,WAAW,GAAG,UAAU,CAAC,IAAI,CAAC,2BAA2B,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;IACnF,MAAM,WAAW,GAAG,UAAU,CAAC,IAAI,CAAC,UAAU,CAAC,SAAS,EAAE,CAAC,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAC;IAElF,OAAO,WAAW,CAAC,GAAG,CAAC,WAAW,CAAC,CAAC;AACtC,CAAC;AAED,SAAS,qBAAqB,CAC5B,iBAA+B,EAC/B,eAA6B,EAC7B,QAAgB;IAEhB,MAAM,UAAU,GAAG,WAAW,CAAC,eAAe,EAAE,QAAQ,CAAC,CAAC;IAC1D,OAAO,UAAU,CAAC,iBAAiB,EAAE,UAAU,CAAC,CAAC;AACnD,CAAC;AAED,SAAS,uBAAuB,CAC9B,iBAA+B,EAC/B,SAAiB,EACjB,WAAmB,EACnB,YAAoB,EACpB,MAAc;IAEd,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,6CAA6C,EAAE;QAC5E,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,WAAW,EAAE;QACpC,EAAE,GAAG,EAAE,gBAAgB,EAAE,KAAK,EAAE,YAAY,EAAE;KAC/C,CAAC,CAAC;IACH,OAAO;QACL,eAAe,EAAE,iBAAiB;QAClC,UAAU,EAAE,iBAAiB;QAC7B,UAAU,EAAE,SAAS,GAAG,CAAC;QACzB,SAAS,EAAE,KAAK;QAChB,SAAS,EAAE,WAAW;QACtB,iBAAiB,EAAE,YAAY;KAChC,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,IAAI,CAClB,iBAA+B,EAC/B,YAAoB,EACpB,gBAA4B,EAC5B,UAAuB,EAAE;IAEzB,MAAM,aAAa,GAAG,OAAO,CAAC,aAAa,IAAI,sBAAsB,CAAC;IACtE,MAAM,SAAS,GAAG,OAAO,CAAC,SAAS,IAAI,iBAAiB,CAAC;IACzD,MAAM,aAAa,GAAG,OAAO,CAAC,aAAa,IAAI,uBAAuB,CAAC;IACvE,MAAM,aAAa,GAAG,OAAO,CAAC,QAAQ,IAAI,uBAAuB,CAAC;IAClE,MAAM,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC;IACxC,MAAM,MAAM,GAAG,IAAI,MAAM,CAAC,OAAO,CAAC,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,CAAC;IAE7D,IAAI,iBAAiB,GAAG,IAAI,YAAY,CAAC,iBAAiB,CAAC,CAAC;IAC5D,IAAI,WAAW,GAAG,YAAY,CAAC,iBAAiB,CAAC,CAAC;IAClD,IAAI,2BAA2B,GAAG,oBAAoB,CAAC,iBAAiB,CAAC,MAAM,CAAC,CAAC;IAEjF,KAAK,IAAI,SAAS,GAAG,CAAC,EAAE,SAAS,GAAG,aAAa,EAAE,SAAS,EAAE,EAAE,CAAC;QAC/D,MAAM,eAAe,GAAG,gBAAgB,CAAC,iBAAiB,CAAC,CAAC;QAC5D,MAAM,YAAY,GAAG,UAAU,CAAC,eAAe,CAAC,CAAC;QAEjD,IAAI,WAAW;YAAE,WAAW,CAAC,SAAS,EAAE,WAAW,EAAE,iBAAiB,CAAC,CAAC;QAExE,IAAI,wBAAwB,CAAC,YAAY,EAAE,SAAS,EAAE,SAAS,CAAC,EAAE,CAAC;YACjE,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,WAAW,EAAE;gBAC1C,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,WAAW,EAAE;gBACpC,EAAE,GAAG,EAAE,gBAAgB,EAAE,KAAK,EAAE,YAAY,EAAE;aAC/C,CAAC,CAAC;YACH,OAAO,uBAAuB,CAAC,iBAAiB,EAAE,SAAS,EAAE,IAAI,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC;QAChG,CAAC;QAED,MAAM,uBAAuB,GAAG,0BAA0B,CAAC,2BAA2B,EAAE,eAAe,CAAC,CAAC;QACzG,MAAM,aAAa,GAAG,gCAAgC,CACpD,eAAe,EACf,uBAAuB,EACvB,2BAA2B,EAC3B,MAAM,EACN,SAAS,EACT,WAAW,CACZ,CAAC;QACF,MAAM,eAAe,GAAG,aAAa,CAAC,eAAe,CAAC;QACtD,2BAA2B,GAAG,aAAa,CAAC,2BAA2B,CAAC;QAExE,MAAM,QAAQ,GAAG,aAAa;YAC5B,CAAC,CAAC,qBAAqB,CAAC,YAAY,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,eAAe,EAAE,OAAO,CAAC,iBAAiB,CAAC;YACtH,CAAC,CAAC,aAAa,CAAC;QAElB,IAAI,QAAQ,KAAK,iBAAiB,EAAE,CAAC;YACnC,OAAO,uBAAuB,CAAC,iBAAiB,EAAE,SAAS,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,CAAC,CAAC;QAClG,CAAC;QAED,MAAM,aAAa,GAAG,qBAAqB,CAAC,iBAAiB,EAAE,eAAe,EAAE,QAAQ,CAAC,CAAC;QAC1F,MAAM,UAAU,GAAG,eAAe,CAAC,aAAa,EAAE,iBAAiB,CAAC,CAAC;QACrE,MAAM,QAAQ,GAAG,UAAU,CAAC,UAAU,CAAC,CAAC;QAExC,MAAM,OAAO,GAAG,YAAY,CAAC,aAAa,CAAC,CAAC;QAC5C,MAAM,WAAW,GAAG,gBAAgB,CAAC,aAAa,CAAC,CAAC;QACpD,MAAM,oBAAoB,GAAG,eAAe,CAAC,WAAW,EAAE,eAAe,CAAC,CAAC;QAE3E,2BAA2B,GAAG,iCAAiC,CAC7D,2BAA2B,EAC3B,UAAU,EACV,oBAAoB,EACpB,MAAM,EACN,SAAS,EACT,OAAO,CACR,CAAC;QAEF,MAAM,CAAC,KAAK,CAAC,MAAM,EAAE,SAAS,EAAE,UAAU,EAAE;YAC1C,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,WAAW,EAAE;YACpC,EAAE,GAAG,EAAE,gBAAgB,EAAE,KAAK,EAAE,YAAY,EAAE;YAC9C,EAAE,GAAG,EAAE,YAAY,EAAE,KAAK,EAAE,QAAQ,EAAE;YACtC,EAAE,GAAG,EAAE,YAAY,EAAE,KAAK,EAAE,QAAQ,EAAE;SACvC,CAAC,CAAC;QAEH,iBAAiB,GAAG,IAAI,YAAY,CAAC,aAAa,CAAC,CAAC;QACpD,WAAW,GAAG,OAAO,CAAC;IACxB,CAAC;IAED,MAAM,aAAa,GAAG,gBAAgB,CAAC,iBAAiB,CAAC,CAAC;IAC1D,MAAM,iBAAiB,GAAG,UAAU,CAAC,aAAa,CAAC,CAAC;IAEpD,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,4BAA4B,EAAE;QAC3D,EAAE,GAAG,EAAE,aAAa,EAAE,KAAK,EAAE,aAAa,EAAE;QAC5C,EAAE,GAAG,EAAE,aAAa,EAAE,KAAK,EAAE,WAAW,EAAE;QAC1C,EAAE,GAAG,EAAE,sBAAsB,EAAE,KAAK,EAAE,iBAAiB,EAAE;KAC1D,CAAC,CAAC;IAEH,OAAO;QACL,eAAe,EAAE,iBAAiB;QAClC,UAAU,EAAE,iBAAiB;QAC7B,UAAU,EAAE,aAAa;QACzB,SAAS,EAAE,KAAK;QAChB,SAAS,EAAE,WAAW;QACtB,iBAAiB,EAAE,iBAAiB;KACrC,CAAC;AACJ,CAAC"}

package/dist/core/cmaEs.d.ts

@@ -0,0 +1,17 @@
+/**
+ * This file implements vanilla CMA-ES and IPOP-CMA-ES restart strategy
+ * for unconstrained black-box optimization (no gradients required).
+ *
+ * Role in system:
+ * - Provides a derivative-free optimizer for scalar cost functions
+ * - Adds IPOP restarts (λ doubles per restart) while preserving libcmaes semantics
+ * - Mirrors libcmaes default parameter formulas and core stop criteria
+ *
+ * For first-time readers:
+ * - Start with `cmaEs()` (public entry point)
+ * - `runSingleCmaEs()` executes one CMA-ES run (no restarts)
+ * - Restart logic wraps `runSingleCmaEs()` when `restartStrategy: "ipop"`
+ */
+import type { CostFn, CmaEsOptions, CmaEsResult } from './types.js';
+export declare function cmaEs(initialParameters: Float64Array, costFunction: CostFn, options?: CmaEsOptions): CmaEsResult;
+//# sourceMappingURL=cmaEs.d.ts.map

package/dist/core/cmaEs.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cmaEs.d.ts","sourceRoot":"","sources":["../../src/core/cmaEs.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AAGH,OAAO,KAAK,EAAE,MAAM,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAyvBpE,wBAAgB,KAAK,CACnB,iBAAiB,EAAE,YAAY,EAC/B,YAAY,EAAE,MAAM,EACpB,OAAO,GAAE,YAAiB,GACzB,WAAW,CAsIb"}