@reicek/neataptic-ts 0.1.26 → 0.1.27

package/src/multithreading/workers/browser/testworker.ts

@@ -1,3 +1,43 @@
+/**
+ * Browser-side evaluation worker wrapper for serialized networks.
+ *
+ * This chapter is the Web Worker half of the multithreading boundary. It does
+ * not teach activation math again. Its job is to take the flat dataset and
+ * network contract from `multithreading/`, package it into browser-friendly
+ * transfer buffers, and return one scalar error without blocking the main
+ * thread.
+ *
+ * The lifecycle is intentionally coarse grained. The constructor transfers the
+ * dataset once and creates a blob-backed worker program. Each `evaluate()` call
+ * then sends only the activation, state, and connection buffers for one
+ * candidate network. That keeps the wrapper focused on whole-network scoring
+ * jobs rather than chatty per-layer messaging.
+ *
+ * Read the file in runtime order: constructor first, `evaluate()` second,
+ * `_createBlobString()` last. That matches the actual browser story: bootstrap
+ * a worker, run repeated evaluations, then inspect how the inline worker code
+ * is assembled.
+ *
+ * ```mermaid
+ * flowchart LR
+ *   Host[Browser host] --> Dataset[Transfer dataset once]
+ *   Dataset --> Blob[Create blob worker]
+ *   Blob --> Evaluate[Transfer network buffers]
+ *   Evaluate --> Error[Return scalar error]
+ * ```
+ *
+ * For background on the browser primitive behind this wrapper, see MDN Web
+ * Docs,
+ * [Using Web Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers).
+ *
+ * Example: create one browser worker and reuse it across repeated evaluations.
+ *
+ * ```ts
+ * const worker = new TestWorker(serializedSet, { name: 'mse' });
+ * const score = await worker.evaluate(network);
+ * worker.terminate();
+ * ```
+ */
 import Multi from '../../multi';
 
 /**

package/src/multithreading/workers/node/README.md

@@ -1,16 +1,42 @@
 # multithreading/workers/node
 
-Handles messages sent to the worker process.
-
-This function listens for messages sent to the worker process and performs one of two actions:
-1. If the message contains serialized activations, states, and connections, it evaluates the network using the dataset.
-2. If the message contains a dataset and cost function, it initializes the worker with the provided data.
+Node-side evaluation worker wrapper for serialized networks.
+
+This chapter is the server-runtime twin of the browser worker wrapper. It
+keeps the same public evaluation contract - serialized dataset in,
+serialized network in, scalar score out - but implements it with a forked
+helper process instead of a browser `Worker` instance.
+
+That split matters because Node and the browser expose different isolation
+primitives even when the computation is the same. This file owns process
+startup, one-time dataset handoff, repeated `evaluate()` calls, and cleanup.
+The neighboring `worker.ts` file owns the child-process message handler that
+turns those payloads back into a local evaluation run.
+
+Read the folder as two layers: `testworker.ts` is the host-side facade and
+`worker.ts` is the child-process entrypoint. Together they keep evaluation
+parallelism explicit without leaking process lifecycle details into the rest
+of the library.
+
+```mermaid
+flowchart LR
+  Host[Node host] --> Fork[Fork helper process]
+  Fork --> Init[Send dataset and cost]
+  Init --> Evaluate[Send serialized network]
+  Evaluate --> Score[Receive scalar score]
+```
 
-## multithreading/workers/node/worker.ts
+For background on the Node primitive used here, see Node.js Documentation,
+[child_process.fork](https://nodejs.org/api/child_process.html#child_processforkmodulepath-args-options).
 
-### WorkerMessage
+Example: keep one worker process around while scoring several candidate
+networks.
 
-Interface for messages sent to the worker process.
+```ts
+const worker = new TestWorker(serializedSet, { name: 'mse' });
+const score = await worker.evaluate(network);
+worker.terminate();
+```
 
 ## multithreading/workers/node/testworker.ts
 
@@ -24,11 +50,12 @@ Interface for serializable network used in worker evaluation.
 
 ### TestWorker
 
-TestWorker class for handling network evaluations in a Node.js environment using Worker Threads.
+TestWorker class for handling network evaluations in a Node.js environment
+through a forked helper process.
 
 This implementation aligns with the Instinct algorithm's emphasis on efficient evaluation of
-neural networks in parallel environments. The use of Worker Threads allows for offloading
-computationally expensive tasks, such as network evaluation, to separate threads.
+neural networks in parallel environments. The use of a forked process allows for offloading
+computationally expensive tasks, such as network evaluation, to a separate process.
 
 Example:
 
@@ -86,3 +113,15 @@ Example:
 const worker = new TestWorker([0, 1, 2], { name: 'mse' });
 // ...use worker.evaluate(...) as needed
 worker.terminate();
+
+## multithreading/workers/node/worker.ts
+
+Handles messages sent to the worker process.
+
+This function listens for messages sent to the worker process and performs one of two actions:
+1. If the message contains serialized activations, states, and connections, it evaluates the network using the dataset.
+2. If the message contains a dataset and cost function, it initializes the worker with the provided data.
+
+### WorkerMessage
+
+Interface for messages sent to the worker process.

package/src/multithreading/workers/node/docs.order.json

@@ -0,0 +1,4 @@
+{
+  "introFile": "testworker.ts",
+  "fileOrder": ["testworker.ts", "worker.ts"]
+}

package/src/multithreading/workers/node/testworker.ts

@@ -1,3 +1,42 @@
+/**
+ * Node-side evaluation worker wrapper for serialized networks.
+ *
+ * This chapter is the server-runtime twin of the browser worker wrapper. It
+ * keeps the same public evaluation contract - serialized dataset in,
+ * serialized network in, scalar score out - but implements it with a forked
+ * helper process instead of a browser `Worker` instance.
+ *
+ * That split matters because Node and the browser expose different isolation
+ * primitives even when the computation is the same. This file owns process
+ * startup, one-time dataset handoff, repeated `evaluate()` calls, and cleanup.
+ * The neighboring `worker.ts` file owns the child-process message handler that
+ * turns those payloads back into a local evaluation run.
+ *
+ * Read the folder as two layers: `testworker.ts` is the host-side facade and
+ * `worker.ts` is the child-process entrypoint. Together they keep evaluation
+ * parallelism explicit without leaking process lifecycle details into the rest
+ * of the library.
+ *
+ * ```mermaid
+ * flowchart LR
+ *   Host[Node host] --> Fork[Fork helper process]
+ *   Fork --> Init[Send dataset and cost]
+ *   Init --> Evaluate[Send serialized network]
+ *   Evaluate --> Score[Receive scalar score]
+ * ```
+ *
+ * For background on the Node primitive used here, see Node.js Documentation,
+ * [child_process.fork](https://nodejs.org/api/child_process.html#child_processforkmodulepath-args-options).
+ *
+ * Example: keep one worker process around while scoring several candidate
+ * networks.
+ *
+ * ```ts
+ * const worker = new TestWorker(serializedSet, { name: 'mse' });
+ * const score = await worker.evaluate(network);
+ * worker.terminate();
+ * ```
+ */
 import { fork, ChildProcess } from 'child_process';
 import * as path from 'path';
 
@@ -16,11 +55,12 @@ interface CostFunction {
 }
 
 /**
- * TestWorker class for handling network evaluations in a Node.js environment using Worker Threads.
+ * TestWorker class for handling network evaluations in a Node.js environment
+ * through a forked helper process.
  *
  * This implementation aligns with the Instinct algorithm's emphasis on efficient evaluation of
- * neural networks in parallel environments. The use of Worker Threads allows for offloading
- * computationally expensive tasks, such as network evaluation, to separate threads.
+ * neural networks in parallel environments. The use of a forked process allows for offloading
+ * computationally expensive tasks, such as network evaluation, to a separate process.
  *
  * @see {@link https://medium.com/data-science/neuro-evolution-on-steroids-82bd14ddc2f6#4-constraints Instinct Algorithm - Section 4 Constraints}
  *

package/src/multithreading/workers/workers.ts

@@ -1,5 +1,40 @@
 /**
- * Utility class for managing workers in both Node.js and browser environments.
+ * Runtime loader shelf for browser and Node evaluation workers.
+ *
+ * The multithreading root explains the serialization contract. This file
+ * answers a narrower question: once a dataset and a network have already been
+ * flattened into portable arrays, which runtime-specific worker wrapper should
+ * own the evaluation run?
+ *
+ * Keeping that choice behind one tiny loader shelf has two benefits. The host
+ * runtime can defer importing browser-only or Node-only code until it actually
+ * needs it, and the rest of the library can talk about one worker boundary
+ * without pretending that Web Workers and forked Node helpers expose the same
+ * constructor surface.
+ *
+ * `getBrowserTestWorker()` loads the wrapper that creates a blob-backed worker
+ * in the browser. `getNodeTestWorker()` loads the wrapper that forks a helper
+ * process in Node. Both wrappers consume the same serialized payload shape and
+ * return the same scalar evaluation contract.
+ *
+ * ```mermaid
+ * flowchart LR
+ *   Host[Host runtime] --> Choose[Choose worker wrapper]
+ *   Choose --> Browser[Browser worker facade]
+ *   Choose --> Node[Node worker facade]
+ *   Browser --> Score[Scalar evaluation]
+ *   Node --> Score
+ * ```
+ *
+ * Example: choose the runtime-specific worker facade without importing both
+ * implementations up front.
+ *
+ * ```ts
+ * const WorkerClass =
+ *   typeof window === 'undefined'
+ *     ? await Workers.getNodeTestWorker()
+ *     : await Workers.getBrowserTestWorker();
+ * ```
  */
 export class Workers {
   /**

package/src/neat/README.md

@@ -1,5 +1,235 @@
 # neat
 
+Root chapter map and public default knobs for the internal `src/neat` controller surface.
+
+`src/neat.ts` is the public control desk. `src/neat/` is the machine room
+behind it. This folder owns the controller chapters that make a run real:
+initialization, evaluation, evolution, mutation, speciation, telemetry,
+persistence, and the shared bookkeeping that keeps experiments deterministic
+instead of magical. Promoting the defaults file to the chapter opening is
+intentional because default knobs are the quickest way to make the
+controller's personality legible before a reader dives into implementation
+detail.
+
+The most helpful reading move is to split the folder into four working
+lanes. `init/`, `evaluate/`, and `evolve/` explain how a population is
+created, scored, and replaced. `mutation/`, `selection/`, and `speciation/`
+explain how search pressure and diversity are managed. `telemetry/`,
+`lineage/`, `diversity/`, and `multiobjective/` explain how the run becomes
+inspectable. `export/`, `rng/`, `cache/`, `maintenance/`, and `pruning/`
+explain how the controller stays reproducible and tractable as experiments
+get larger.
+
+The flat root files are small bridges rather than the whole story.
+`neat.defaults.constants.ts` and `neat.types.ts` keep the public constructor
+and option bag readable. `neat.lineage.ts` and `neat.constants.ts` keep
+small shared logic close to the root when multiple chapters need it. The
+deeper folders own the heavier policy and runtime details.
+
+These defaults matter because they are the baseline promises the controller
+makes when a caller says "give me an ordinary NEAT run." Population size,
+mutation tempo, compatibility pressure, structural ceilings, and
+observability sampling are not random numbers. They are the quiet assumptions
+that decide whether the controller behaves like a conservative search, an
+exploratory search, or an unstable one.
+
+That design follows the original NEAT intuition: protect structural
+innovation long enough for it to compete, rather than forcing every new
+topology to beat established species immediately. See Stanley and
+Miikkulainen,
+[Evolving Neural Networks through Augmenting Topologies](https://nn.cs.utexas.edu/?stanley:ec02),
+for the background behind the compatibility and growth vocabulary that keeps
+surfacing across this folder.
+
+Read this root chapter in three passes. Start with this defaults file and
+`neat.types.ts` for the public knobs and broad contracts. Continue into
+`evaluate/`, `evolve/`, and `speciation/` for the live search loop. Finish
+with `telemetry/`, `lineage/`, `multiobjective/`, `export/`, and `rng/` when
+you want to inspect, replay, or compare runs rather than only advance them.
+
+```mermaid
+flowchart TD
+  classDef base fill:#08131f,stroke:#1ea7ff,color:#dff6ff,stroke-width:1px;
+  classDef accent fill:#0f2233,stroke:#ffd166,color:#fff4cc,stroke-width:1.5px;
+
+  Root["src/neat root"]:::accent --> Loop["init / evaluate / evolve"]:::base
+  Root --> Pressure["mutation / selection / speciation"]:::base
+  Root --> Observe["telemetry / lineage / diversity / multiobjective"]:::base
+  Root --> Replay["export / rng / cache / maintenance / pruning"]:::base
+  Root --> Bridges["root bridges<br/>defaults / types / constants"]:::base
+```
+
+```mermaid
+flowchart LR
+  classDef base fill:#08131f,stroke:#1ea7ff,color:#dff6ff,stroke-width:1px;
+  classDef accent fill:#0f2233,stroke:#ffd166,color:#fff4cc,stroke-width:1.5px;
+
+  Defaults[Root defaults]:::accent --> Population[popsize elitism provenance]:::base
+  Defaults --> Variation[mutationRate mutationAmount]:::base
+  Defaults --> Species[compatibility and weight coefficients]:::base
+  Defaults --> Observation[diversity and novelty samples]:::base
+  Population --> Run[Controller behavior]:::base
+  Variation --> Run
+  Species --> Run
+  Observation --> Run
+```
+
+Example: build one explicit baseline options bag from the documented root
+defaults.
+
+```ts
+const baselineOptions = {
+  popsize: DEFAULT_POPULATION_SIZE,
+  mutationRate: DEFAULT_MUTATION_RATE,
+  compatibilityThreshold: DEFAULT_COMPATIBILITY_THRESHOLD,
+};
+```
+
+Example: keep the observability defaults visible when teaching or
+benchmarking runs.
+
+```ts
+const observabilityDefaults = {
+  diversityPairSample: DEFAULT_DIVERSITY_PAIR_SAMPLE,
+  diversityGraphletSample: DEFAULT_DIVERSITY_GRAPHLET_SAMPLE,
+  noveltyK: DEFAULT_NOVELTY_K,
+};
+```
+
+## neat/neat.defaults.constants.ts
+
+### DEFAULT_COMPATIBILITY_THRESHOLD
+
+Default compatibility threshold controlling speciation distance.
+
+This starts the speciation-pressure family of defaults. It is the neutral
+boundary the controller uses before adaptive tuning or custom settings make
+species splits stricter or more permissive.
+
+### DEFAULT_DISJOINT_COEFF
+
+Default disjoint coefficient for NEAT compatibility distance.
+
+Matching the excess coefficient by default gives the root controller a
+balanced structural view: excess and disjoint innovation gaps both count as
+first-class evidence during compatibility comparisons.
+
+### DEFAULT_DIVERSITY_GRAPHLET_SAMPLE
+
+Default graphlet sample size used by diversity metrics in fast mode.
+
+Read this beside {@link DEFAULT_DIVERSITY_PAIR_SAMPLE}: pair samples give the
+controller quick distance evidence, while graphlet samples provide a small
+structural texture read without forcing whole-population analysis.
+
+### DEFAULT_DIVERSITY_PAIR_SAMPLE
+
+Default pair-sample size used by diversity metrics in fast mode.
+
+This starts the observability-sampling family. The root controller uses a
+bounded sample instead of exhaustive pair checks so diversity reads stay
+cheap enough for ordinary runs.
+
+### DEFAULT_ELITISM
+
+Default elitism count applied when unspecified.
+
+Read this beside {@link DEFAULT_POPULATION_SIZE} and
+{@link DEFAULT_PROVENANCE}: the trio defines how much of each generation is
+reserved for carry-over, how much is freshly injected, and how much capacity
+remains for ordinary offspring.
+
+### DEFAULT_EXCESS_COEFF
+
+Default excess coefficient for NEAT compatibility distance.
+
+This begins the root compatibility-weight family. These coefficients explain
+which kinds of genome disagreement matter most when the controller decides
+whether two genomes still belong in the same species neighborhood.
+
+### DEFAULT_MAX_CONNS
+
+Default maximum allowed connections where `Infinity` means unbounded growth.
+
+This preserves the same baseline policy as {@link DEFAULT_MAX_NODES}: the
+controller does not impose a fixed connection ceiling unless the caller wants
+one.
+
+### DEFAULT_MAX_GATES
+
+Default maximum allowed gates where `Infinity` means unbounded growth.
+
+Gate limits stay in the same family as node and connection limits so the
+whole structural-cap story remains consistent at the root surface.
+
+### DEFAULT_MAX_NODES
+
+Default maximum allowed nodes where `Infinity` means unbounded growth.
+
+Read the three `DEFAULT_MAX_*` exports as one structural-ceiling family.
+Leaving them unbounded by default tells the root controller to rely on
+mutation policy, pruning, and adaptive limits instead of an immediate hard
+cap.
+
+### DEFAULT_MUTATION_AMOUNT
+
+Default number of mutation operations applied per genome.
+
+The default keeps the baseline search policy conservative: most runs mutate
+often enough to keep topology moving, but each genome usually pays for only
+one structural or parametric change per mutation pass.
+
+### DEFAULT_MUTATION_RATE
+
+Default mutation rate used by the root controller when no explicit rate is supplied.
+
+This belongs to the same search-tempo family as
+{@link DEFAULT_MUTATION_AMOUNT}. Together they define how often mutation is
+attempted and how many mutation steps a genome can receive once mutation is
+active.
+
+### DEFAULT_NEAT_CONSTRUCTOR_DEFAULTS
+
+Shared defaults packet consumed by the constructor bootstrap chapter.
+
+The root public surface still exports the individual constants for callers
+and docs, but the constructor now hands one named packet to `init/` instead
+of rebuilding the same object inline inside `src/neat.ts`.
+
+### DEFAULT_NOVELTY_K
+
+Default neighbor count for novelty search when `k` is unspecified.
+
+This closes the root observability-and-exploration shelf. It controls how
+many nearby behaviors contribute to novelty before the caller tunes novelty
+search more explicitly.
+
+### DEFAULT_POPULATION_SIZE
+
+Default population size when caller does not specify `popsize`.
+
+This opens the root defaults shelf's search-volume family. It controls how
+many genomes compete in each generation before elitism, provenance, or
+mutation pressure begin to reshape the population.
+
+### DEFAULT_PROVENANCE
+
+Default provenance count applied when unspecified.
+
+Provenance is the root controller's small "fresh seed" policy. A value of
+`0` means the default run does not spend population budget on extra
+generation-zero style injections unless the caller asks for them.
+
+### DEFAULT_WEIGHT_DIFF_COEFF
+
+Default average weight difference coefficient for compatibility distance.
+
+This keeps parameter drift relevant without letting weight deltas dominate
+the whole speciation read. In the default family, topology disagreement still
+carries more weight than modest edge-weight differences.
+
+## neat/neat.types.ts
+
 Root compatibility types for the public `Neat` controller boundary.
 
 The chapter exists to keep `src/neat.ts` orchestration-first while still
@@ -15,8 +245,6 @@ Read the file in three passes:
 - finish with the restore and mutation aliases when you are tracing how the
   root facade forwards work into the `init/`, `rng/`, and `export/` chapters.
 
-## neat/neat.types.ts
-
 ### NeatExportFitnessFunction
 
 ```ts
@@ -434,153 +662,3 @@ Parameters:
 - `graph` - - Network to summarize structurally.
 
 Returns: Shannon-style entropy of the out-degree distribution.
-
-## neat/neat.defaults.constants.ts
-
-Public default knobs for the root `Neat` controller.
-
-These constants describe the controller personality a caller gets before a
-custom option bag starts bending the run toward a different search style.
-Keeping them in their own root chapter lets `src/neat.ts` stay focused on
-orchestration while the `init/` chapter consumes one shared defaults packet.
-
-The constants fall into four small families:
-
-- search volume and tempo,
-- speciation pressure,
-- structural ceilings,
-- observability sampling.
-
-Read them as the public defaults shelf, not as hidden implementation trivia.
-These values are the baseline promises the root controller makes when a user
-says, "give me an ordinary NEAT run," without specifying every knob.
-
-### DEFAULT_COMPATIBILITY_THRESHOLD
-
-Default compatibility threshold controlling speciation distance.
-
-This starts the speciation-pressure family of defaults. It is the neutral
-boundary the controller uses before adaptive tuning or custom settings make
-species splits stricter or more permissive.
-
-### DEFAULT_DISJOINT_COEFF
-
-Default disjoint coefficient for NEAT compatibility distance.
-
-Matching the excess coefficient by default gives the root controller a
-balanced structural view: excess and disjoint innovation gaps both count as
-first-class evidence during compatibility comparisons.
-
-### DEFAULT_DIVERSITY_GRAPHLET_SAMPLE
-
-Default graphlet sample size used by diversity metrics in fast mode.
-
-Read this beside {@link DEFAULT_DIVERSITY_PAIR_SAMPLE}: pair samples give the
-controller quick distance evidence, while graphlet samples provide a small
-structural texture read without forcing whole-population analysis.
-
-### DEFAULT_DIVERSITY_PAIR_SAMPLE
-
-Default pair-sample size used by diversity metrics in fast mode.
-
-This starts the observability-sampling family. The root controller uses a
-bounded sample instead of exhaustive pair checks so diversity reads stay
-cheap enough for ordinary runs.
-
-### DEFAULT_ELITISM
-
-Default elitism count applied when unspecified.
-
-Read this beside {@link DEFAULT_POPULATION_SIZE} and
-{@link DEFAULT_PROVENANCE}: the trio defines how much of each generation is
-reserved for carry-over, how much is freshly injected, and how much capacity
-remains for ordinary offspring.
-
-### DEFAULT_EXCESS_COEFF
-
-Default excess coefficient for NEAT compatibility distance.
-
-This begins the root compatibility-weight family. These coefficients explain
-which kinds of genome disagreement matter most when the controller decides
-whether two genomes still belong in the same species neighborhood.
-
-### DEFAULT_MAX_CONNS
-
-Default maximum allowed connections where `Infinity` means unbounded growth.
-
-This preserves the same baseline policy as {@link DEFAULT_MAX_NODES}: the
-controller does not impose a fixed connection ceiling unless the caller wants
-one.
-
-### DEFAULT_MAX_GATES
-
-Default maximum allowed gates where `Infinity` means unbounded growth.
-
-Gate limits stay in the same family as node and connection limits so the
-whole structural-cap story remains consistent at the root surface.
-
-### DEFAULT_MAX_NODES
-
-Default maximum allowed nodes where `Infinity` means unbounded growth.
-
-Read the three `DEFAULT_MAX_*` exports as one structural-ceiling family.
-Leaving them unbounded by default tells the root controller to rely on
-mutation policy, pruning, and adaptive limits instead of an immediate hard
-cap.
-
-### DEFAULT_MUTATION_AMOUNT
-
-Default number of mutation operations applied per genome.
-
-The default keeps the baseline search policy conservative: most runs mutate
-often enough to keep topology moving, but each genome usually pays for only
-one structural or parametric change per mutation pass.
-
-### DEFAULT_MUTATION_RATE
-
-Default mutation rate used by the root controller when no explicit rate is supplied.
-
-This belongs to the same search-tempo family as
-{@link DEFAULT_MUTATION_AMOUNT}. Together they define how often mutation is
-attempted and how many mutation steps a genome can receive once mutation is
-active.
-
-### DEFAULT_NEAT_CONSTRUCTOR_DEFAULTS
-
-Shared defaults packet consumed by the constructor bootstrap chapter.
-
-The root public surface still exports the individual constants for callers
-and docs, but the constructor now hands one named packet to `init/` instead
-of rebuilding the same object inline inside `src/neat.ts`.
-
-### DEFAULT_NOVELTY_K
-
-Default neighbor count for novelty search when `k` is unspecified.
-
-This closes the root observability-and-exploration shelf. It controls how
-many nearby behaviors contribute to novelty before the caller tunes novelty
-search more explicitly.
-
-### DEFAULT_POPULATION_SIZE
-
-Default population size when caller does not specify `popsize`.
-
-This opens the root defaults shelf's search-volume family. It controls how
-many genomes compete in each generation before elitism, provenance, or
-mutation pressure begin to reshape the population.
-
-### DEFAULT_PROVENANCE
-
-Default provenance count applied when unspecified.
-
-Provenance is the root controller's small "fresh seed" policy. A value of
-`0` means the default run does not spend population budget on extra
-generation-zero style injections unless the caller asks for them.
-
-### DEFAULT_WEIGHT_DIFF_COEFF
-
-Default average weight difference coefficient for compatibility distance.
-
-This keeps parameter drift relevant without letting weight deltas dominate
-the whole speciation read. In the default family, topology disagreement still
-carries more weight than modest edge-weight differences.