@reicek/neataptic-ts 0.1.25 → 0.1.26

package/src/methods/activation/README.md

@@ -156,304 +156,314 @@ forward-value question and the local-slope question, which is why the file is
 organized around reusable transfer-curve families instead of separate forward
 and derivative tables.
 
-### logisticActivation
+### absoluteActivation
 
 ```ts
-logisticActivation(
+absoluteActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Logistic (sigmoid) activation implementation.
+Absolute activation implementation.
 
-This is the bounded baseline most readers already know: useful when a node
-should behave like a probability-like squashing unit, but also the easiest
-activation to saturate if pre-activation values become too large in
-magnitude.
+Absolute value discards sign and keeps only magnitude. It is unusual as a
+default hidden-layer choice, but it can be useful in experiments where the
+intensity of a signal matters more than whether it was positive or negative.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Logistic output or derivative.
+Returns: Absolute output or derivative.
 
-### tanhActivation
+### ActivationFunction
 
 ```ts
-tanhActivation(
+ActivationFunction(
   inputValue: number,
-  shouldComputeDerivative: boolean,
+  shouldComputeDerivative: boolean | undefined,
 ): number
 ```
 
-Hyperbolic tangent activation implementation.
-
-Compared with {@link logisticActivation}, tanh stays bounded while remaining
-zero-centered, which often makes hidden activations easier to interpret when
-positive and negative evidence should balance around zero.
+Shared contract for activation implementations used by the runtime registry.
 
-Parameters:
-- `inputValue` - - Input to evaluate.
-- `shouldComputeDerivative` - - Whether to compute the derivative.
+This utility layer is where the math-facing side of the activation chapter
+lives. Each exported function keeps the same two-mode signature so the
+higher-level {@link Activation} registry can switch between forward
+evaluation and derivative lookup without wrapping every implementation in a
+different adapter.
 
-Returns: Tanh output or derivative.
+Read the implementations in four practical groups:
 
-### identityActivation
+- bounded classics such as {@link logisticActivation},
+  {@link sigmoidActivation}, and {@link tanhActivation},
+- sparse or piecewise-linear gates such as {@link reluActivation},
+  {@link stepActivation}, and {@link hardTanhActivation},
+- shape-specialized functions such as {@link gaussianActivation},
+  {@link sinusoidActivation}, and {@link bentIdentityActivation},
+- smoother modern hidden-layer candidates such as {@link softplusActivation},
+  {@link swishActivation}, {@link geluActivation}, and
+  {@link mishActivation}.
 
-```ts
-identityActivation(
-  inputValue: number,
-  shouldComputeDerivative: boolean,
-): number
-```
+A useful reading order is:
 
-Identity activation implementation.
+1. compare {@link logisticActivation}, {@link tanhActivation}, and
+   {@link reluActivation} to anchor the classic bounded-versus-sparse trade,
+2. scan {@link softplusActivation}, {@link swishActivation},
+   {@link geluActivation}, and {@link mishActivation} when you want smoother
+   hidden-layer behavior,
+3. finish with the shape-specialized helpers when your experiment needs a
+   periodic, radial, or sign-like response rather than a general default.
 
-Use this when you explicitly want a linear pass-through unit. It is most
-common in regression-style output layers or in experiments where the upstream
-topology already provides the non-linearity and you only need a value relay.
+Keep the derivative flag in mind while reading: every helper answers both the
+forward-value question and the local-slope question, which is why the file is
+organized around reusable transfer-curve families instead of separate forward
+and derivative tables.
 
 Parameters:
-- `inputValue` - - Input to evaluate.
-- `shouldComputeDerivative` - - Whether to compute the derivative.
+- `inputValue` - - Input to the activation function.
+- `shouldComputeDerivative` - - Whether to compute the derivative instead of the value.
 
-Returns: Identity output or derivative.
+Returns: Activation output or derivative at the input.
 
-### stepActivation
+### bentIdentityActivation
 
 ```ts
-stepActivation(
+bentIdentityActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Step activation implementation.
+Bent identity activation implementation.
 
-This is the hard-threshold ancestor of smoother gates: it cleanly separates
-negative from positive evidence, but its zero derivative almost everywhere
-makes it a poor default for gradient-based training.
+Bent identity stays close to a linear pass-through while adding a gentle
+non-linearity near the origin. It is useful when pure identity feels too weak
+but a strongly saturating activation would distort the signal too early.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Step output or derivative.
+Returns: Bent identity output or derivative.
 
-### reluActivation
+### bipolarActivation
 
 ```ts
-reluActivation(
+bipolarActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Rectified Linear Unit (ReLU) activation implementation.
+Bipolar activation implementation.
 
-ReLU is the practical default for many hidden layers because it is cheap,
-sparse, and easy to optimize. This implementation follows the common library
-convention of returning `0` for the derivative at exactly `0`, even though
-the mathematical derivative is not uniquely defined there.
+This is the sign-function version of a hard classifier: values collapse to
+`-1` or `1` with no middle ground. It is mostly useful as a historical or
+diagnostic contrast against smoother bounded activations.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: ReLU output or derivative.
+Returns: Bipolar output or derivative.
 
-### softsignActivation
+### bipolarSigmoidActivation
 
 ```ts
-softsignActivation(
+bipolarSigmoidActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Softsign activation implementation.
+Bipolar sigmoid activation implementation.
 
-Softsign behaves like a gentler, cheaper-to-reason-about cousin of tanh. It
-still compresses values into a bounded range, but the tails decay more
-gradually, which can make it a useful comparison point when tanh feels too
-eager to saturate.
+This is the `[-1, 1]` counterpart to {@link logisticActivation}. In practice
+it is another route to a tanh-like curve, but the explicit bipolar naming is
+helpful when comparing older NEAT-era literature or porting legacy settings.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Softsign output or derivative.
+Returns: Bipolar sigmoid output or derivative.
 
-### sinusoidActivation
+### gaussianActivation
 
 ```ts
-sinusoidActivation(
+gaussianActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Sinusoid activation implementation.
+Gaussian activation implementation.
 
-Reach for this when periodic structure matters more than monotonicity. A
-sinusoidal response can encode cycles and phase relationships that ordinary
-squashing activations tend to smooth away.
+Gaussian responses peak at the origin and shrink toward zero on both sides,
+which makes them useful when you want a node to behave more like a localized
+detector than a broad monotonic amplifier.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Sinusoid output or derivative.
+Returns: Gaussian output or derivative.
 
-### gaussianActivation
+### geluActivation
 
 ```ts
-gaussianActivation(
+geluActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Gaussian activation implementation.
+Gaussian Error Linear Unit (GELU) activation implementation.
 
-Gaussian responses peak at the origin and shrink toward zero on both sides,
-which makes them useful when you want a node to behave more like a localized
-detector than a broad monotonic amplifier.
+GELU keeps the soft gating idea of Swish but uses a Gaussian-CDF-shaped
+weighting curve. This implementation uses the common tanh-based
+approximation, which is fast enough for ordinary training code while staying
+close to the exact GELU shape used in many transformer-era models.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Gaussian output or derivative.
+Returns: GELU output or derivative.
 
-### bentIdentityActivation
+### hardTanhActivation
 
 ```ts
-bentIdentityActivation(
+hardTanhActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Bent identity activation implementation.
+Hard tanh activation implementation.
 
-Bent identity stays close to a linear pass-through while adding a gentle
-non-linearity near the origin. It is useful when pure identity feels too weak
-but a strongly saturating activation would distort the signal too early.
+Hard tanh keeps tanh's familiar `[-1, 1]` output range while replacing the
+curved middle section with a cheap piecewise-linear clamp. That makes it a
+practical compromise when you want bounded outputs without paying for a full
+smooth tanh evaluation.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Bent identity output or derivative.
+Returns: Hard tanh output or derivative.
 
-### bipolarActivation
+### identityActivation
 
 ```ts
-bipolarActivation(
+identityActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Bipolar activation implementation.
+Identity activation implementation.
 
-This is the sign-function version of a hard classifier: values collapse to
-`-1` or `1` with no middle ground. It is mostly useful as a historical or
-diagnostic contrast against smoother bounded activations.
+Use this when you explicitly want a linear pass-through unit. It is most
+common in regression-style output layers or in experiments where the upstream
+topology already provides the non-linearity and you only need a value relay.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Bipolar output or derivative.
+Returns: Identity output or derivative.
 
-### bipolarSigmoidActivation
+### inverseActivation
 
 ```ts
-bipolarSigmoidActivation(
+inverseActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Bipolar sigmoid activation implementation.
+Inverse activation implementation.
 
-This is the `[-1, 1]` counterpart to {@link logisticActivation}. In practice
-it is another route to a tanh-like curve, but the explicit bipolar naming is
-helpful when comparing older NEAT-era literature or porting legacy settings.
+This helper mirrors a value around `1`, which makes it more of a niche
+transformation than a general-purpose hidden activation. It is best read as
+part of the method vocabulary's legacy and experimentation shelf rather than
+as a recommended first-choice default.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Bipolar sigmoid output or derivative.
+Returns: Inverse output or derivative.
 
-### hardTanhActivation
+### logisticActivation
 
 ```ts
-hardTanhActivation(
+logisticActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Hard tanh activation implementation.
+Logistic (sigmoid) activation implementation.
 
-Hard tanh keeps tanh's familiar `[-1, 1]` output range while replacing the
-curved middle section with a cheap piecewise-linear clamp. That makes it a
-practical compromise when you want bounded outputs without paying for a full
-smooth tanh evaluation.
+This is the bounded baseline most readers already know: useful when a node
+should behave like a probability-like squashing unit, but also the easiest
+activation to saturate if pre-activation values become too large in
+magnitude.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Hard tanh output or derivative.
+Returns: Logistic output or derivative.
 
-### absoluteActivation
+### mishActivation
 
 ```ts
-absoluteActivation(
+mishActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Absolute activation implementation.
+Mish activation implementation.
 
-Absolute value discards sign and keeps only magnitude. It is unusual as a
-default hidden-layer choice, but it can be useful in experiments where the
-intensity of a signal matters more than whether it was positive or negative.
+Mish is another smooth self-gated option, built from `x * tanh(softplus(x))`.
+It keeps more negative-side nuance than ReLU while still rewarding large
+positive evidence. The implementation reuses the same softplus stability
+strategy so the helper behaves sensibly in the far positive and negative
+tails.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Absolute output or derivative.
+Returns: Mish output or derivative.
 
-### inverseActivation
+### reluActivation
 
 ```ts
-inverseActivation(
+reluActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Inverse activation implementation.
+Rectified Linear Unit (ReLU) activation implementation.
 
-This helper mirrors a value around `1`, which makes it more of a niche
-transformation than a general-purpose hidden activation. It is best read as
-part of the method vocabulary's legacy and experimentation shelf rather than
-as a recommended first-choice default.
+ReLU is the practical default for many hidden layers because it is cheap,
+sparse, and easy to optimize. This implementation follows the common library
+convention of returning `0` for the derivative at exactly `0`, even though
+the mathematical derivative is not uniquely defined there.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Inverse output or derivative.
+Returns: ReLU output or derivative.
 
 ### seluActivation
 
@@ -478,162 +488,152 @@ Parameters:
 
 Returns: SELU output or derivative.
 
-### softplusActivation
+### sigmoidActivation
 
 ```ts
-softplusActivation(
+sigmoidActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Softplus activation implementation with stability guards.
+Sigmoid alias activation implementation.
 
-Softplus is the smooth sibling of {@link reluActivation}: for large positive
-values it behaves almost linearly, for large negative values it fades toward
-zero, and around the origin it avoids the hard corner that makes ReLU
-piecewise. The threshold checks keep the implementation numerically stable in
-the extreme tails.
+This intentionally delegates to {@link logisticActivation} so callers can use
+either the mathematically explicit `logistic` name or the more common
+deep-learning alias `sigmoid` without creating two separate implementations.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Softplus output or derivative.
+Returns: Sigmoid output or derivative.
 
-### swishActivation
+### sinusoidActivation
 
 ```ts
-swishActivation(
+sinusoidActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Swish activation implementation.
+Sinusoid activation implementation.
 
-Swish multiplies the input by a sigmoid gate, so the unit can dampen itself
-smoothly instead of snapping to zero as ReLU does. That makes it a useful
-comparison point when experimenting with smoother hidden-layer behavior.
+Reach for this when periodic structure matters more than monotonicity. A
+sinusoidal response can encode cycles and phase relationships that ordinary
+squashing activations tend to smooth away.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Swish output or derivative.
+Returns: Sinusoid output or derivative.
 
-### geluActivation
+### softplusActivation
 
 ```ts
-geluActivation(
+softplusActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Gaussian Error Linear Unit (GELU) activation implementation.
+Softplus activation implementation with stability guards.
 
-GELU keeps the soft gating idea of Swish but uses a Gaussian-CDF-shaped
-weighting curve. This implementation uses the common tanh-based
-approximation, which is fast enough for ordinary training code while staying
-close to the exact GELU shape used in many transformer-era models.
+Softplus is the smooth sibling of {@link reluActivation}: for large positive
+values it behaves almost linearly, for large negative values it fades toward
+zero, and around the origin it avoids the hard corner that makes ReLU
+piecewise. The threshold checks keep the implementation numerically stable in
+the extreme tails.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: GELU output or derivative.
+Returns: Softplus output or derivative.
 
-### mishActivation
+### softsignActivation
 
 ```ts
-mishActivation(
+softsignActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Mish activation implementation.
+Softsign activation implementation.
 
-Mish is another smooth self-gated option, built from `x * tanh(softplus(x))`.
-It keeps more negative-side nuance than ReLU while still rewarding large
-positive evidence. The implementation reuses the same softplus stability
-strategy so the helper behaves sensibly in the far positive and negative
-tails.
+Softsign behaves like a gentler, cheaper-to-reason-about cousin of tanh. It
+still compresses values into a bounded range, but the tails decay more
+gradually, which can make it a useful comparison point when tanh feels too
+eager to saturate.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Mish output or derivative.
+Returns: Softsign output or derivative.
 
-### sigmoidActivation
+### stepActivation
 
 ```ts
-sigmoidActivation(
+stepActivation(
   inputValue: number,
   shouldComputeDerivative: boolean,
 ): number
 ```
 
-Sigmoid alias activation implementation.
+Step activation implementation.
 
-This intentionally delegates to {@link logisticActivation} so callers can use
-either the mathematically explicit `logistic` name or the more common
-deep-learning alias `sigmoid` without creating two separate implementations.
+This is the hard-threshold ancestor of smoother gates: it cleanly separates
+negative from positive evidence, but its zero derivative almost everywhere
+makes it a poor default for gradient-based training.
 
 Parameters:
 - `inputValue` - - Input to evaluate.
 - `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Sigmoid output or derivative.
+Returns: Step output or derivative.
 
-### ActivationFunction
+### swishActivation
 
 ```ts
-ActivationFunction(
+swishActivation(
   inputValue: number,
-  shouldComputeDerivative: boolean | undefined,
+  shouldComputeDerivative: boolean,
 ): number
 ```
 
-Shared contract for activation implementations used by the runtime registry.
+Swish activation implementation.
 
-This utility layer is where the math-facing side of the activation chapter
-lives. Each exported function keeps the same two-mode signature so the
-higher-level {@link Activation} registry can switch between forward
-evaluation and derivative lookup without wrapping every implementation in a
-different adapter.
+Swish multiplies the input by a sigmoid gate, so the unit can dampen itself
+smoothly instead of snapping to zero as ReLU does. That makes it a useful
+comparison point when experimenting with smoother hidden-layer behavior.
 
-Read the implementations in four practical groups:
+Parameters:
+- `inputValue` - - Input to evaluate.
+- `shouldComputeDerivative` - - Whether to compute the derivative.
 
-- bounded classics such as {@link logisticActivation},
-  {@link sigmoidActivation}, and {@link tanhActivation},
-- sparse or piecewise-linear gates such as {@link reluActivation},
-  {@link stepActivation}, and {@link hardTanhActivation},
-- shape-specialized functions such as {@link gaussianActivation},
-  {@link sinusoidActivation}, and {@link bentIdentityActivation},
-- smoother modern hidden-layer candidates such as {@link softplusActivation},
-  {@link swishActivation}, {@link geluActivation}, and
-  {@link mishActivation}.
+Returns: Swish output or derivative.
 
-A useful reading order is:
+### tanhActivation
 
-1. compare {@link logisticActivation}, {@link tanhActivation}, and
-   {@link reluActivation} to anchor the classic bounded-versus-sparse trade,
-2. scan {@link softplusActivation}, {@link swishActivation},
-   {@link geluActivation}, and {@link mishActivation} when you want smoother
-   hidden-layer behavior,
-3. finish with the shape-specialized helpers when your experiment needs a
-   periodic, radial, or sign-like response rather than a general default.
+```ts
+tanhActivation(
+  inputValue: number,
+  shouldComputeDerivative: boolean,
+): number
+```
 
-Keep the derivative flag in mind while reading: every helper answers both the
-forward-value question and the local-slope question, which is why the file is
-organized around reusable transfer-curve families instead of separate forward
-and derivative tables.
+Hyperbolic tangent activation implementation.
+
+Compared with {@link logisticActivation}, tanh stays bounded while remaining
+zero-centered, which often makes hidden activations easier to interpret when
+positive and negative evidence should balance around zero.
 
 Parameters:
-- `inputValue` - - Input to the activation function.
-- `shouldComputeDerivative` - - Whether to compute the derivative instead of the value.
+- `inputValue` - - Input to evaluate.
+- `shouldComputeDerivative` - - Whether to compute the derivative.
 
-Returns: Activation output or derivative at the input.
+Returns: Tanh output or derivative.