@daneren2005/shared-memory-ecs 1.8.1 → 1.9.0

package/README.md

@@ -193,6 +193,195 @@ template.
 
 Classless factories keep the original behavior: they create `BaseEntity` and scan the complete registry.
 
+## System scheduling
+
+`world.update(elapsedTime)` uses `DefaultScheduler` to dispatch systems in registration order.
+Synchronous systems run immediately, workers can overlap after dispatch, and iterable systems can
+continue across updates. Systems keep their existing `shouldRun()`, `deltaBetweenRuns`, and `firstRun`
+behavior. The world advances clocks, applies `timeScale`, skips new updates while paused, prepares shared
+memory, and manages deferred component frees.
+
+Select a scheduler at construction; `world.scheduler` is readonly:
+
+```ts
+import { BaseWorld, DefaultScheduler } from '@daneren2005/shared-memory-ecs';
+
+const world = new BaseWorld(registry, {
+	scheduler: new DefaultScheduler<Components>()
+});
+world.update(16); // Synchronous result: { lastSystemError: null } when no dispatch error occurred.
+```
+
+Custom policies implement `Scheduler<C>`. Its synchronous `update(context, elapsedTime)` receives the
+scaled delta and a `SchedulerContext<C>` exposing the live `systems` array, the world's `emit` method,
+and optional `gameTime` (supplied by `BaseWorld`).
+It returns `SchedulerUpdateResult` (`{ lastSystemError?: Error | null }`). `DefaultScheduler` preserves
+the existing dispatch timing events, error reporting, and continuation after a system throws.
+
+Optional `systemAdded(context, system)` and `systemRemoved(context, system)` hooks receive notifications
+after the array changes and before the corresponding world event. Duplicate names remain supported;
+`addSystemIfNotExists` checks names and `removeSystem` removes the first matching name. Direct assignment
+or mutation of `world.systems` bypasses membership hooks, so policies must read the live array when needed.
+The default policy preserves native `forEach` mutation behavior during dispatch.
+
+Optional `reset(context)` runs after systems are cleared during reload or non-pristine `clear()`;
+pristine worlds skip that teardown. `destroy(context)` runs once after system destruction.
+`runCompleted(context, system)` receives the existing completion boundary: a synchronous run returned,
+an iterable queue drained, or a current-generation worker reply's effects were applied. Fallback
+completion can arrive inside scheduler dispatch. The world processes eligible deferred frees before
+calling this hook. Dispatch return values and `system-<name>-finished` events do not imply completion.
+These hooks expose lifecycle notifications; they do not establish deterministic ticks or safe edit barriers.
+
+Use `ConflictScheduler` to prevent systems from overlapping when one writes a component the other reads
+or writes. Conflicts follow registration order; read-only systems and systems accessing independent
+components can overlap:
+
+```ts
+import { BaseWorld, ConflictScheduler, EntityWorkerSystem } from '@daneren2005/shared-memory-ecs';
+
+const world = new BaseWorld(registry, {
+	scheduler: new ConflictScheduler<Components>()
+});
+
+class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }> {
+	constructor() {
+		super(world, {
+			name: 'Damage',
+			required: ['health'],
+			writes: ['health'],
+			updateFunction: damageUpdate,
+			getWorker: () => new Worker(new URL('./damage-worker.ts', import.meta.url), { type: 'module' })
+		});
+	}
+}
+world.addSystem(new DamageSystem());
+```
+
+The scheduler compiles its schedule when systems are added or removed. Inspect it before the first
+update using the scheduler instance:
+
+```ts
+const scheduler = new ConflictScheduler<Components>();
+const world = new BaseWorld(registry, { scheduler });
+// Register your systems here.
+console.log(scheduler.schedule.order.map(system => system.name));
+console.table(scheduler.schedule.batches.map((batch, index) => ({
+	batch: index,
+	systems: batch.map(system => system.name).join(', ')
+})));
+```
+
+`schedule` is a frozen `ConflictSchedule<C>` containing system references. `order` flattens the
+compiled dependency levels, and `batches` group systems that can run concurrently within one update.
+Every conflict points from the earlier registered system to the later one. Each system goes into the
+earliest level after its conflicting predecessors. These groups describe dependencies, not runtime
+barriers: a system starts when its own predecessors finish, even if unrelated systems in an earlier
+group are still running. Actual dispatch can also overlap independent work from later host updates.
+
+`writes` lists every component a system changes, including components reached outside its main query.
+Use `writes: []` for a read-only system. Reads are inferred from `EntitySystem.components` and from worker
+`required`, `optional`, and all named `queries`. The built-in `entity` component is also a read. Add
+`reads: ['otherComponent']` for access outside those queries, including filters and hooks. A plain
+`System` or `IterableSystem` supplies both `reads` and `writes`; an `EntitySystem` without a component
+query supplies `reads` explicitly. Component names are checked against the system's component type.
+
+Omitting `writes`, or leaving reads unknown, makes a system exclusive. `exclusive: true` also reserves
+exclusive execution for a system whose callbacks, structural changes, or shared resources cannot be
+described through component access. Workers with `createsEntities: true` or `addsComponents: true` remain
+exclusive. Declare writes to `entity` when changing its flags; arbitrary callback effects still need
+explicit declarations or exclusive execution.
+
+For parallel worker creation, `EntityWorkerSystem` and `WorkerSystem` accept a component list in the
+existing capability option, for example `createsEntities: ['health', 'movement']`. List every game
+component the system can create, including components supplied by factory templates. The built-in
+`entity` component is included automatically. `writes` still declares changes to existing components;
+creation declarations do not authorize undeclared callback or attachment effects on existing data.
+Both boolean and list forms enable the same worker allocator, factory templates, and stable atomic eids.
+The default scheduler treats both forms identically and adopts results immediately on completion.
+
+Under `ConflictScheduler`, compatible creators can allocate and initialize blocks concurrently, even
+in the same component pool. Publication stays in scheduling order and waits for earlier readers and
+writers of the affected components; later readers and writers wait until publication is applied.
+A creator can therefore compute beside an earlier reader without changing that reader's query snapshot.
+A later query waits for publication when the creator's component list can supply every required
+component of that query. For example, creating `['transform', 'item']` does not block a query requiring
+`['transform', 'tree']`, even when it optionally reads `item` or excludes it with `not`. Main and named
+queries are checked separately; a match with any one retains the dependency. An empty or entity-only
+query still observes every creation. Explicit `reads` outside queries and all `writes` retain their
+component conflicts, so accessing `world.entities` directly must still be declared.
+
+Creation lists describe all components a system can create, not one exact entity shape. Optional
+components and exclusions therefore cannot prove a query unrelated when its required components could
+all be created. Custom query filters need explicit `reads` for accesses outside their declared
+components. Custom membership overrides retain conservative component-level creation access unless
+they provide accurate `creationReadQueries` metadata. This optional system getter exposes separate
+required-component lists to the conflict scheduler; `undefined` uses `creationReadComponents` as before.
+Queued updates keep their original query and creation snapshots across schedule rebuilds.
+
+Worker completion effects are kept together, preserving component-change, event, death, and adoption
+order within each run. Attachment hooks and callbacks affecting other shared data still need truthful
+`writes` or `exclusive: true`. Completion promises and memory-free notifications settle only after
+application; cancellation discards unpublished blocks. These rules apply only to the conflict scheduler.
+
+Each host update queues one update per registered system with its scaled delta and logical game time.
+Updates of the same system stay sequential. Read/write and write/write conflicts preserve host-update
+order, then registration order within an update, including dependencies on work that has not started
+yet. Read/read access across different systems can overlap. Independent systems can advance into later
+updates while a slow worker is unfinished; there is no whole-update or whole-batch completion barrier.
+Exclusive work remains a barrier to all earlier and later work, including workers using the boolean
+creation capability.
+
+When asynchronous systems finish applying effects, a microtask dispatches newly ready work after
+their completion promises settle, without waiting for another animation frame. Deltas stay in each
+system's order without merging or dropping them; `deltaBetweenRuns`, remainders and `firstRun` retain
+their existing behavior. While unpaused, sliced systems continue at most once per host update with
+zero extra delta, and release their dependents when the slice queue drains. Conflicting work can still
+build a backlog.
+
+During scheduled `shouldRun()` and `update()` calls, `System.gameTime` reports the logical time of the
+queued update. Worker `world.gameTime` uses that captured value, on real workers and the synchronous
+fallback alike. Delayed dispatch therefore does not borrow a later host update's clock. Outside that
+scope, `System.gameTime` returns the live world clock. Custom schedulers can use
+`system.withGameTime(time, () => system.update(delta))` to establish the same scope. Plain systems and
+`addDataToWorld()` implementations should use `this.gameTime` when they need scheduled time; direct
+reads of `this.world.gameTime` still see the host clock. The default scheduler continues using its
+existing `update(delta)` path and live clock.
+
+The compiled schedule is reused until membership changes or `scheduler.rebuild(world)` is called.
+Call `rebuild` after changing access declarations, or to inspect direct edits to `world.systems` before
+the next update; direct membership edits are otherwise checked before each dispatch. Queued updates
+retain their original access reservations, even if metadata is rebuilt while a worker is running.
+Additions join future host updates; removed pending updates are skipped when their predecessors finish,
+and removed running systems retain their reservations until completion. With `ConflictScheduler`,
+`world.pause()` or `world.paused = true` stops enqueueing new updates but finishes draining existing
+work without further `world.update()` calls. Queued systems retain their captured deltas and logical
+times; sliced runs continue through timer tasks with zero extra delta, yielding between slices.
+Worker effects publish as their dependencies finish. Draining does not advance either world clock;
+calling `world.update()` while paused still advances only `playerTime`. `world.resume()` allows new
+work to be queued on the next host update. Reload/clear and destroy discard queued updates and
+cancel scheduled microtasks and slice timers. The world prepares shared memory before deferred dispatch and prevents
+new dispatch while clearing or destroyed. Errors from automatic dispatch emit `system-error`; they
+cannot change a result already returned by `world.update()`.
+
+Custom asynchronous systems must report `isCurrentlyRunning()` and notify completion through
+`onRunFinished()` when their work and effects finish. Schedulers may implement the optional
+`prepareEffects(context, system)` hook: it captures the dispatched run and returns a handler receiving
+`SchedulerEffects` with `apply()` and `discard()`. The handler must eventually call one of them.
+`beforeClear(context)` lets such schedulers discard pending effects before the world awaits workers.
+The optional `pause(context)` hook is called when the world's paused flag becomes true, including
+direct assignment. `context.paused` indicates whether new host updates are suspended;
+`context.dispatchAllowed` and `prepareForSystemDispatch()` gate deferred dispatch during teardown.
+Schedulers without these hooks retain immediate application and the existing clear path.
+
+This scheduler provides component conflict ordering with the existing synchronous update API. It does
+not implement fixed simulation ticks or impose a stable order on unrelated workers' callbacks.
+World clocks continue advancing with host updates. Iterable slicing uses wall-clock budgets;
+entity workers can opt into deterministic work batches instead.
+For a fixed input-update sequence, declared conflicting operations and each system's deltas/logical
+times remain ordered independently of worker completion order. Independent callbacks and timing events
+can arrive in different orders; shared effects must be declared as conflicts or marked exclusive.
+Repeatable simulations also need deterministic inputs and system code, and safe external world edits.
+
 ## Iterating entities
 
 `world.entities` is a `Map` keyed by `eid`, not an array, and so is `entities` on `EntitySystem` and
@@ -273,7 +462,37 @@ class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }
 }
 ```
 
-### Update-function hooks: `init`, `queryChanged` and `preRun`
+### Incremental entity work
+
+Set `workBatchSize: 1024` in an `EntityWorkerSystem` configuration to process at most
+1,024 snapshot positions per scheduled update. The size must be a positive safe integer.
+Batch boundaries depend on entity count and insertion order, never elapsed wall time.
+The last batch can be smaller, and removed entities consume their original positions without
+being updated. New members wait for the next pass; current component bundles and named queries
+are refreshed before every batch.
+
+`deltaBetweenRuns` starts each full pass. All its batches share the original `elapsedTime`,
+logical `gameTime`, and data from `addDataToWorld()`. Delta accumulated during the pass is
+carried toward the next pass. `preRun` receives the full current entity list once at the
+start, while optional `preBatch` receives only the selected entities before each batch.
+Worker-local fields assigned to `world` survive until the pass ends. `queryChanged` and
+`entityRemoved` still receive each batch's membership changes.
+
+Each batch publishes callbacks, creations and component changes independently. Under
+`ConflictScheduler`, its applied completion releases conflicting systems, allowing physics
+or other frequent work between batches. Access declarations must cover both hooks and
+entity updates. This opt-in permits other systems to change shared state between batches;
+use `preBatch` to refresh live indexes and avoid retaining component block references across
+batch boundaries. A failed preparation hook abandons the remaining pass. Clear/reload drops
+the pending snapshot and worker-local pass state.
+
+This also works with `DefaultScheduler` and the main-thread fallback. It does not apply to
+`WorkerSystem`, whose function runs once over named queries. Full-query preparation and
+query-delta handling remain unbounded, so a count limit alone cannot guarantee a frame deadline.
+Paused conflict scheduling drains queued batches; unqueued pass work resumes with future
+host updates. `waitForRunToComplete()` waits for the active batch and its publication.
+
+### Update-function hooks: `init`, `queryChanged`, `preRun` and `preBatch`
 
 `addDataToWorld` runs on the main thread and re-sends its data every run. When the data instead needs to
 *live in the worker* — computed once, or too big to ship each frame — attach hooks to the update function
@@ -282,7 +501,11 @@ itself. Both are optional properties on the `EntityUpdateFunction`.
 **`init`** runs on `system.finishLoading()` — once at startup (`world.init()` calls it), and again on each
 `world.load()` so a reused world re-seeds its workers. It receives whatever the system's `getInitData()` returned
 (structured-cloned across the boundary) and returns a `Partial<W>` that is merged onto `world` on every
-subsequent run. Use it for state that must be seeded from the main thread but then persist inside the worker
+subsequent run. The same `load` message also carries the system's current query membership, so the worker has
+every loaded entity cached before `finishLoading()` resolves. `init(world, data, entities, queries)` receives those
+populated lists plus a base `world` with `gameTime`, `getString` and `heap` (no `elapsedTime` or callbacks: init
+seeds worker state and must not modify entities); that initial membership never reaches `queryChanged`, so build any index over the loaded entities
+in `init`. Use it for state that must be seeded from the main thread but then persist inside the worker
 — a seeded RNG, a lookup table, a config object — without paying to re-send it each frame. Type the init
 data with the `D` type parameter (the fourth on `EntityUpdateFunction` / `EntityWorkerSystem`) so `getInitData`
 and the `init` hook agree on its shape; it defaults to `unknown` when a system has no init data:
@@ -300,7 +523,7 @@ const damageUpdate: EntityUpdateFunction<Components, { health: Int32Array }, Dam
 
 // Runs once in the worker; its return is merged onto `world` before every run. `data` is typed as
 // `DamageInitData | undefined` (undefined when the system supplies no getInitData).
-damageUpdate.init = (data) => ({ damage: data?.baseDamage ?? 1 });
+damageUpdate.init = (_world, data) => ({ damage: data?.baseDamage ?? 1 });
 
 class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }, DamageWorld, DamageInitData> {
 	constructor(world: BaseWorld<Components>) {
@@ -335,11 +558,13 @@ damageUpdate.preRun = (world, entities, queries, callbacks) => {
 with that query's name and its `{ added, removed }` delta. Use it to keep a worker-local structure in sync with a
 set of entities that rarely changes - a navigation grid of static walls, a lookup of static targets - instead of
 rescanning the whole query every run. `added` can repeat an entity already in the query (its components changed
-and its blocks were re-sent), so treat it as an upsert. Keep that structure on the `init`-returned state so a
-reloaded world rebuilds it from scratch; the reload re-sends every member as added:
+and its blocks were re-sent), so treat it as an upsert. Seed that structure in `init` from the loaded `queries`,
+since entities present at load are never reported to `queryChanged`; a reload re-runs `init`, rebuilding it:
 
 ```ts
-wallUpdate.init = () => ({ walls: new Map<number, Rect>() });
+wallUpdate.init = (_world, _data, _entities, queries) => ({
+	walls: new Map((queries.walls ?? []).map(({ entityId, components }) => [entityId, rectOf(components.transform)])),
+});
 wallUpdate.queryChanged = (world, queryName, delta) => {
 	delta.removed.forEach(entityId => world.walls.delete(entityId));
 	delta.added.forEach(({ entityId, components }) => world.walls.set(entityId, rectOf(components.transform)));
@@ -390,9 +615,15 @@ import { createSystemWorker } from '@daneren2005/shared-memory-ecs/worker';
 createSystemWorker(self, spawnerUpdate, registry);
 ```
 
-The run function may carry the same `init` / `queryChanged` / `entityRemoved` hooks as an `updateFunction`. Everything else —
+The run function may carry the same `init` / `queryChanged` / `entityRemoved` hooks as an `updateFunction`; its
+`init(world, data, queries)` omits the entity list, since there is no main query. Everything else —
 callbacks, worker fallback, off-thread creation — behaves exactly as an `EntityWorkerSystem`.
 
+The worker helpers report unhandled promise rejections on the main thread with the system name,
+preserving the error message and stack. Rejections before the startup handshake completes reject
+`system.init()`; later rejections are logged. Custom worker scopes can provide
+`addEventListener('unhandledrejection', listener)` to enable this reporting.
+
 ### Importing in workers
 
 Each worker entry file is bundled on its own, and a single-entry bundle cannot tree-shake this package's
@@ -527,15 +758,15 @@ createEntityWorker(world, { type: 'ship', x: 100, y: 150 }, callbacks);
 counter (so it never collides with one the main thread or another worker hands out), and for each component
 the merged config triggers, pushes a block into its pool and writes the values - all off-thread. With entity
 classes registered, the worker also resolves the template's class and considers only its allowed components. It reports
-back an id-plus-block-indexes descriptor; when the run completes the main thread *adopts* it, building the
+back an id-plus-block-indexes descriptor; when completion effects are applied the main thread *adopts* it, building the
 always-present `entity` component there (interning `type` and `class` is a main-thread job), asking the factory
-for the registered wrapper, and attaching each block the worker wrote - no block is copied or re-allocated. Like
-every other worker report-back, the new entity first
-exists on the following frame, so the system picks it up next run.
+for the registered wrapper, and attaching each block the worker wrote - no block is copied or re-allocated. The new entity joins queries at adoption and is available to subsequent dispatches. The default scheduler
+adopts immediately on completion; the conflict scheduler can defer adoption until publication is safe.
 
 Two things make this work, both opt-in so only the workers that create entities pay for them:
 
-- Register the system with `createsEntities: true`. That ships the factory templates to its worker on load.
+- Register the system with `createsEntities: true`, or a component list for parallel conflict scheduling.
+  Both forms ship the factory templates to its worker on load.
 - In that system's worker entry, pass your component registry to `createEntitySystemWorker`, so the worker has
   each component's block builder:
 
@@ -683,7 +914,9 @@ from `entity-removed`, which fires for every removal, reloads included).
 
 A world is meant to be reused rather than rebuilt. `world.load(config)` swaps in a fresh scenario — it removes
 the old entities, clears every system, and loads the new batch (deferred frees from the old contents drain over
-the following updates, as above). `await world.clear()` instead tears the world all the way back down to an
+the following updates, as above). Loading itself is synchronous, but it returns a promise that resolves once every
+system's `finishLoading()` (e.g. a worker re-seeding its `init` data) completes — `await world.load(config)` before
+starting the game loop. `await world.clear()` instead tears the world all the way back down to an
 empty, reusable state: it waits for every system's in-flight worker run to finish so nothing is still reading
 the memory, frees the held blocks immediately, and resolves once the world is ready to load into again.
 
@@ -714,13 +947,21 @@ Every entry is an `{ avg, min, max, samples }` over the window just closed:
 - `stats.update` - one whole `world.update` call on the thread the world lives on.
 - `stats.systems[]` - per system, in run order: `run` is the run itself on its worker, as the worker measured
   it, and `events` is what handling that run's results (the events it reported onto entities, the entities it
-  asked to be created) cost back on the calling thread. A system running on the main-thread fallback never
-  reports either, so it sits at zero with `samples: 0` - which is what tells it apart from one that genuinely
-  cost nothing.
+  asked to be created) cost back on the calling thread. Both worker and main-thread fallback runs report
+  these timings. For `workBatchSize`, one sample sums all batches in a complete pass: batches taking 3, 5,
+  and 4 ms report a 12 ms run with `samples: 1`. Event handling is summed across the pass too; gaps between
+  batches are excluded. Failed preparation closes the attempt with the time spent so far.
 - `stats.events` - every system's event handling added together. Workers finish on their own schedule rather
   than on a frame boundary, so there is no per-frame combined sample to take; these are the per-system figures
   summed at the end of the window, giving what a run of every system costs the main thread between them.
 
+To coordinate reporting with an external clock, construct with `{ autoUpdate: false }` and call
+`timing.flush()` at each reporting boundary. It publishes and returns a snapshot of all collected update,
+worker, and event samples, then starts a fresh window. This also works while paused: windows without work
+report zeroes. A batched pass retains its partial totals across reporting boundaries and contributes
+one sample when its final batch arrives (event timing completes when its results are handled). Clear/reload
+drops unfinished passes, and the next pass starts fresh totals. Automatic reporting remains enabled by default.
+
 `getSystemStats(name)` pulls one system out of the latest snapshot, `reset()` throws away everything collected
 so far (worth doing after loading a new scene, when the samples either side are not comparable), and
 `destroy()` unhooks it from the world.