@adobe/data-ai 0.10.21 → 0.10.23

package/.claude/rules/features/ecs/conformance.md

@@ -0,0 +1,85 @@
+---
+paths:
+  - '**/features/*/ecs/conformance/**/*.ts'
+---
+
+# ecs/conformance/ — proving the ECS matches the spec
+
+Test tier. The spec's cases are replayed against the ECS: every spec action against
+the `MainService` action of the same name, and every derivation against its
+computed. The folder holds:
+
+```
+conformance/
+  projection.ts         the store ⇄ State mapping
+  implementation.ts     the spec paired with its ECS build
+  frame.cases.ts        whole-frame cases (real-time features)
+  conformance.test.ts   Conformance.checkFeature(implementation)
+```
+
+## `projection.ts`
+
+```ts
+export const projection = { fromState, toState, toData };
+```
+
+- `fromState(store, state)` clears the store, inserts each entity (supplying tags and
+  implementation-only columns), sets the resources, and returns the
+  `spec id → entity` map it built. The runner resolves case ids through it.
+- `toState(store)` reads the whole store back as a `State`.
+- `toData(store, entity)` reads one entity as its value. It is needed only when a
+  computed emits entity ids (`hydrate`).
+
+## `implementation.ts`
+
+```ts
+export const implementation = Conformance.implementation(spec, {
+  plugin: MainService.plugin,
+  computedPlugin: ComputedDatabase.plugin,   // required when the spec has derivations
+  projection,
+  hydrate: ["visibleTodos"],                 // exactly the computeds that emit entity ids
+  services: { game: GameService.createFake },   // base fakes for app-injected services
+  frame: { args: { dt: (db, dt) => { db.store.resources.frameDelta = dt; } },
+           unmodelled: { physics: "wasm rigid-body step" },
+           cases: frameCases },                          // real-time features
+});
+```
+
+- **`computedPlugin`** is the `ComputedDatabase` layer, built on its own, so a
+  `withCache` value from a layer above can't go stale across the seed.
+- **`services`** are fakes injected into every database the runner builds. They are
+  needed for a service whose `create` throws. Each case's recording doubles override
+  them.
+- **`frame`** is required when the spec has `systems`. `args` writes each frame data
+  arg into the store; `unmodelled` names, with a reason, each system the spec doesn't
+  model; `cases` are whole-frame cases (`frame.cases.ts` here, typed
+  `Conformance.FrameCases<State, typeof systems>`). Each spec system conforms by
+  running just that system; each frame case runs every modelled system once in
+  `db.system.order` (`systems.md`). Systems keep their in-place column writes.
+- `hydrate` exactness and `computedPlugin` requiredness are checked at compile time.
+
+## `conformance.test.ts`
+
+```ts
+Conformance.checkFeature(implementation);
+```
+
+This single call conforms every spec action and derivation and round-trips every
+`State.samples` entry through the projection. A spec op with no same-named action or
+computed fails by name, as does an ECS system that is neither modelled nor unmodelled. So when the UI's real op is richer (`dragTodo`),
+add a thin same-named action (`reorderTodo`) for the spec op to pair with. The UI
+calls actions, so conforming actions covers the path the app runs.
+
+- **Comparison.** `ReadonlyArray` is positional; `ReadonlySet` and `ReadonlyMap` are
+  order-independent. Ids compare up to a bijection. Floats use
+  `match: { tolerance }`, set on the spec (default `0.01`).
+- **Extra ECS tests** (collision selection, a refill after the field clears) live here
+  too, since they need the spec or the projection.
+
+## Escape hatch: per-case ambient context
+
+A feature whose action reads ambient per-peer context (a `userId` seeded under a
+custom concurrency, as in p2p presence) can't use `checkFeature`. It drives
+`Conformance.runActions` directly, feeding it
+`Conformance.adaptCases(spec.fns, spec.cases, spec.services, true)` with its own
+`makeDb` and `seedContext`. This is the only sanctioned use of the lower-level runners.

package/.claude/rules/features/ecs/core.md

@@ -0,0 +1,43 @@
+---
+paths:
+  - '**/features/*/ecs/core/**/*.ts'
+---
+
+# ecs/core/ — the schema
+
+The core database builds the feature's `data/` components and resources into a plugin
+and chooses how entities are packed into archetypes.
+
+```ts
+// core/archetypes.ts — the spec entity plus implementation-only columns
+export const archetypes = Database.archetypes(components, {
+  Todo: ["todo", ...Todo],
+});
+
+// core/core-database.ts
+const coreDatabasePlugin = Database.Plugin.create({ components, resources, archetypes });
+
+export type CoreDatabase = Database.Plugin.ToDatabase<typeof coreDatabasePlugin>;
+type CoreComponents = Store.Components<Database.Plugin.ToStore<typeof coreDatabasePlugin>>;
+
+export namespace CoreDatabase {
+  export const plugin = coreDatabasePlugin;
+  export type Store = Database.Plugin.ToStore<typeof coreDatabasePlugin>;
+  export type Index = Database.Index<CoreComponents>;
+}
+```
+
+- **Import `components` and `resources` from `data/`.** Never author schemas here.
+- **Spread the entity tuple** from `data/entities/` and add implementation-only
+  columns (a tag). `UpperCase` archetype names.
+- **Never pack a `nonPersistent` column into a persistent entity's archetype** unless
+  it has a real, non-null default. On load such a column is stripped, and every
+  reloaded entity then drops out of that archetype and out of every query on it. Add
+  the column on demand with `t.update` instead. A persisting feature keeps a save/load
+  round-trip test (`data-lit-todo`'s `app/persistence.test.ts`).
+- **A feature built on another** `extends` the lower feature's core plugin and adds its
+  own `components` barrel (which already spreads the lower one) and archetypes.
+- **`Component` / `Archetype` key types**, when needed, derive from the `components`
+  barrel (`Extract<keyof typeof components, string>`), never from the plugin: the
+  plugin is built from `archetypes`, so a plugin-derived key type is circular when used
+  to check an archetype.

package/.claude/rules/features/ecs/index.md

@@ -0,0 +1,72 @@
+---
+paths:
+  - '**/features/*/ecs/main-service.ts'
+  - '**/features/*/ecs/*/*-database.ts'
+---
+
+# ecs/ — the implementation
+
+The feature implemented as a reactive Entity-Component-System, conformed to
+`spec/`. It is a chain of `Database.Plugin` layers, each extending the one before and
+adding one facet:
+
+```
+ecs/
+  core/          core-database.ts, archetypes.ts        the schema (core.md)
+  indexes/       index-database.ts + one file per index + index.ts
+  transactions/  transaction-database.ts + one file per transaction + index.ts
+  services/      service-database.ts + database-bound service factories
+  computed/      computed-database.ts + one file per computed + index.ts
+  actions/       action-database.ts + one file per action + index.ts
+  systems/       system-database.ts (systems declared inline)
+  main-service.ts
+  conformance/   test tier (conformance.md)
+```
+
+The order is fixed: `core → indexes → transactions → services → computed → actions
+→ systems`. Services sit below computed because computeds may derive from service
+observables; a service factory therefore never reads a computed. A feature creates
+only the layers it uses. Each layer's file exports a namespace mirroring its plugin:
+
+```ts
+const indexDatabasePlugin = Database.Plugin.create({ extends: CoreDatabase.plugin, indexes });
+export type IndexDatabase = Database.Plugin.ToDatabase<typeof indexDatabasePlugin>;
+export namespace IndexDatabase {
+  export const plugin = indexDatabasePlugin;
+  export type Store = Database.Plugin.ToStore<typeof indexDatabasePlugin>;
+}
+```
+
+Runtime code here imports `data/` and `services/`, never `spec/`.
+
+## `MainService` — the assembled database
+
+`main-service.ts` aliases the top layer, so consumers never name it:
+
+```ts
+export { ActionDatabase as MainService } from "./actions/action-database.js";
+```
+
+`ui/`, `app/` and `conformance/` reference `MainService.plugin` / `MainService.Store`.
+Adding or dropping a layer changes this one line. When another feature or package
+imports a feature's databases, it aliases them feature-qualified
+(`import { CoreDatabase as MainCoreDatabase }`).
+
+## What each op takes
+
+- **Transactions** take the store: `CoreDatabase.Store`, or `IndexDatabase.Store` once
+  they read an index. A store is the transaction context (it carries index handles and
+  `t.userId`).
+- **Computed, service factories and actions** take the whole database, typed on the
+  lowest layer that exposes what they read or call — never their own layer.
+
+## Folders are flat until they aren't
+
+A behavioural folder holds one file per op plus an `index.ts` barrel that feeds the
+facet. When a cohesive cluster grows past about half a dozen files, give it a
+subfolder with its own `index.ts` (e.g. `transactions/order/`). A helper several ops
+share lives beside them but stays **out of the barrel**, or it would be registered
+as an op.
+
+The per-layer rules are in this folder. How a plugin's authored and derived surfaces
+are modelled is in the rules-root `plugin-modelling.md`.

package/.claude/rules/features/{services/main-service → ecs}/indexes.md

@@ -1,24 +1,23 @@
 ---
 paths:
-  - '**/features/*/services/main-service/index-database/indexes/**/*.ts'
+  - '**/features/*/ecs/indexes/**/*.ts'
 ---
 
-# database/indexes/ — ECS indexes
+# ecs/indexes/ — ECS indexes
 
 One index descriptor per file: an object `satisfies CoreDatabase.Index` that
 tells the store to maintain a lookup keyed on a component, so queries and
 computed values can find entities without scanning.
 
 ```ts
-import type { CoreDatabase } from "../../core-database/core-database.js";
+import type { CoreDatabase } from "../core/core-database.js";
 
 export const byComplete = {
     key: "complete",
 } as const satisfies CoreDatabase.Index;
 ```
 
-`CoreDatabase.Index` is bound to every component (all scopes live in the one
-`core-database`). Archetype-scoped indexes carry an extra `archetype` field the
+`CoreDatabase.Index` is bound to every component in the core database. Archetype-scoped indexes carry an extra `archetype` field the
 `Index` helper can't express, so declare those with a bare `as const` (no
 `satisfies`) and let `index-database.ts`'s `create()` validate them.
 
@@ -37,7 +36,7 @@ Derive a discrete **bucket key** — but prefer a **computed-key extractor**
 recomputes the key from its source columns, so the index stays correct with no
 maintenance; a stored bucket column must instead be recomputed at every
 insertion/update site (or by a system) and silently drifts if one is missed. The
-derivation itself is a pure `data/` helper.
+derivation itself is a pure `data/values/` helper.
 
 Bucketed broad-phase only pays off at scale (hundreds+ candidates per query);
 below that a linear scan over an archetype wins. Index when the scan is a
@@ -58,6 +57,6 @@ below that a linear scan over an archetype wins. Index when the scan is a
   multi-value index — one bucket entry per element — and `find({ col: element })`
   takes the element, not the array.
 - **Model a many-to-many as a denormalized array + two indexes.** A
-  `todo.assignees: string[]` (owned/displayed by one feature) plus a unique
+  `todo.assignees: string[]` (owned by one feature) plus a unique
   `usersByName` (name → user) and a multi-value `todosByAssignee` (assignee →
   todos) gives efficient navigation in both directions from one join key.

package/.claude/rules/features/ecs/services.md

@@ -0,0 +1,36 @@
+---
+paths:
+  - '**/features/*/ecs/services/**/*.ts'
+---
+
+# ecs/services/ — registering services on the database
+
+`service-database.ts` registers the feature's services under the `services` facet,
+keyed as the spec's `Services` map, and pins the two together:
+
+```ts
+const serviceDatabasePlugin = Database.Plugin.create({
+  extends: TransactionDatabase.plugin,
+  services: {
+    nameGenerator: NameGeneratorService.create,
+    analytics: AnalyticsService.create,
+  },
+});
+export type ServiceDatabase = Database.Plugin.ToDatabase<typeof serviceDatabasePlugin>;
+type _ServicesPin = Assert<Equal<ServiceDatabase["services"], Services>>;
+```
+
+- **A `services/` service** registers its `create`. If `create` throws (the service
+  needs runtime input), the app injects the real one (`../../app.md`), and conformance
+  injects a fake (`conformance.md`).
+- **A database-bound service** (it reads observables or calls transactions, such as
+  an agent) is a factory here, one per file: `createAgentService(db, …)`, with `db`
+  typed on the lowest layer it needs. It sits below computed, so it derives what it
+  observes from the store (`db.derive`) and `data/values/` helpers, sharing a read
+  helper with the transactions where useful (tictactoe's `readBoard`). The `index.ts`
+  barrel re-exports the factories.
+- **A service built from another service** needs that service registered first, so
+  register it in a second `Plugin.create` extending the first: a factory only sees
+  the services of the plugins it extends.
+- Register each service once, in the feature that owns it, under one key. Combined
+  plugins require the same factory per key.

package/.claude/rules/features/{services/main-service → ecs}/systems.md

@@ -1,9 +1,9 @@
 ---
 paths:
-  - '**/features/*/services/main-service/system-database/**/*.ts'
+  - '**/features/*/ecs/systems/**/*.ts'
 ---
 
-# database/systems/ — the real-time tick loop
+# ecs/systems/ — the real-time tick loop
 
 Systems are the framework's real-time backbone: functions the scheduler runs
 **every frame**, in dependency order, to advance a simulation. A feature only
@@ -21,19 +21,19 @@ arrow must be written inline in `Plugin.create` for two reasons: only there is `
 strongly typed (to the assembled plugin's database, with a **writable `store`**), and
 a pre-typed standalone `create` reference breaks the scheduler's system-name inference
 (the name union collapses to `never`, which then rejects every `schedule` entry). Keep
-each `create` thin by delegating the real per-frame work to a pure `data/` step
-function; if an ecs-specific body is substantial, extract it to a `systems/<name>.ts`
-helper `(db) => () => void` (annotate its `db`) and call it from the inline `create`.
+each `create` thin by delegating math to pure `data/values/` helpers (never `spec/`);
+if an ecs-specific body is substantial, extract it to a `systems/<name>.ts` helper
+`(db) => () => void` (annotate its `db`) and call it from the inline `create`.
 
 ```ts
 // in system-database.ts
 import { Database, scheduler } from "@adobe/data/ecs";
-import { Motion } from "../../data/motion/motion.js";
+import { Motion } from "../../data/values/motion/motion.js";
 
 const plugin = Database.Plugin.create({
-    // combine scheduler with the CURRENT TOP layer — ComputedDatabase here, but
-    // ActionDatabase / ServiceDatabase if the feature built those (systems come last).
-    extends: Database.Plugin.combine(ComputedDatabase.plugin, scheduler),
+    // combine scheduler with the current top layer — ActionDatabase, since every
+    // spec action has an action (systems come last).
+    extends: Database.Plugin.combine(ActionDatabase.plugin, scheduler),
     systems: {
         control: { create: (db) => () => { /* input → ship; db.transactions.fireBullet() */ } },
         movement: {
@@ -59,18 +59,15 @@ const plugin = Database.Plugin.create({
   `db.store.update / insert / delete` for lifecycle (spawning, dying) that changes
   archetype membership. Resources: `db.store.resources.x = …`.
 - **Dispatch a transaction for a discrete atomic event** — when the effect is a
-  self-contained event that already has a conformant transaction (score a hit,
-  lose a life, spawn a wave, fire a bullet), call `db.transactions.*` instead of
-  re-implementing it: you reuse the `data/`-verified logic and observers (a reactive
-  HUD) are notified. Reserve direct `db.store` writes for the hot per-row paths
+  self-contained event (score a hit, lose a life, spawn a wave, fire a bullet), call
+  `db.transactions.*`: observers (a reactive HUD) are notified. Reserve direct `db.store` writes for the hot per-row paths
   (movement, aging) that run on every entity every frame and shouldn't pay
   per-entity transaction overhead.
 - **Optimize hot paths for speed.** With many entities, query with efficient
   `queryArchetypes` and read/write minimal columns directly; avoid intermediate
-  allocations. On a hot path treat `data/` step functions as **spec only** — use
-  them when the path isn't hot or you can't beat them by writing the store
-  directly; off hot paths, reuse them freely. (Only transactions are observable —
-  direct store writes fire no observers.)
+  allocations. Never turn per-row work into per-entity transactions to make it
+  conformable: `frame` conformance checks in-place writes as they are. (Only
+  transactions are observable — direct store writes fire no observers.)
 - **Iterate archetypes per `archetypes.md`.** Express selection with
   `queryArchetypes(include, { exclude })`; when a system destroys/migrates rows
   (bullets expiring, entities dying) iterate **tail → head** so hole-fills don't
@@ -79,25 +76,22 @@ const plugin = Database.Plugin.create({
   a transaction that inserts/deletes invalidates any live archetype cursor, so
   snapshot the ids you'll act on (or reverse-iterate) *before* dispatching
   transactions inside a per-frame loop.
-- **Replicate the oracle's top-level guard in every system.** If the `data/`
-  `step` opens with `if (isGameOver(state)) return state`, each split system must
-  early-return on the same condition — a frozen frame is only a true no-op if
-  *every* system honors the guard.
+- **Every system guards itself** (e.g. early-return once the game is over), as its
+  spec function does — a frozen frame is only a true no-op if *every* system honors
+  the guard.
 - **Index reads go through `db.indexes`** (plural) — the writable `db.store` a
   system receives is typed *without* its index handles, so a broad-phase query is
   `db.indexes.byCell.find({ cell })`, not `db.store.indexes.…`.
 - **Ordering is declared under `schedule`, not implied.** `schedule.before` /
   `schedule.after` are hard constraints (name sibling systems); `schedule.during`
   is a soft same-tier hint. Systems with no ordering relation share a tier
-  (conceptually parallel) — never rely on declaration order. When you split a
-  monolithic `data/` `step` into several ordered systems, the schedule **must
-  mirror the step's internal sequence** (advance → fire → age → collide → refill)
-  — otherwise the tick diverges from the oracle even though each system is
-  individually correct. Watch the subtle case: if a spawn step (fire) is followed
-  by a step that also processes the just-spawned entity (the new bullet is aged
-  and advanced this same tick), you can't advance *all* bodies in one system
-  before the spawn — split that entity's advance out so it runs *after* the spawn,
-  alongside the step that owns it.
+  (conceptually parallel) — never rely on declaration order; systems sharing a tier
+  must commute (conformance checks it). The schedule is the only statement of order:
+  the spec never re-sequences systems, and frame conformance composes them in the
+  order the schedule derives. Watch the subtle case: if a spawn (fire) is followed by
+  a step that also processes the just-spawned entity (the new bullet is aged and
+  advanced this same tick), you can't advance *all* bodies in one system before the
+  spawn — split that entity's advance out so it runs *after* the spawn.
 - Immediate-mode rendering: store the canvas in a session resource and render to
   it inside a system (see @adobe/data-gpu for patterns).
 
@@ -109,32 +103,52 @@ Systems don't run themselves. Combine the built-in scheduler so they run on
 ```ts
 import { Database, scheduler } from "@adobe/data/ecs";
 // in system-database.ts:
-Database.Plugin.create({ extends: Database.Plugin.combine(ComputedDatabase.plugin, scheduler), systems });
+Database.Plugin.create({ extends: Database.Plugin.combine(ActionDatabase.plugin, scheduler), systems });
 ```
 
 The scheduler adds a `schedulerState` resource (`"running" | "paused" |
 "disposed"`) — gate start/pause/resume through it (start `"paused"` if the game
-begins on a user action). A headless host (tests, server sim) omits the
-scheduler and drives frames itself by calling `db.system.functions[name]()` for
-each `name` in `db.system.order` — so a simulation is fully runnable and testable
-with no rAF and no rendering attached.
+begins on a user action). A headless host (tests, server sim) drives frames itself: for each tier in
+`db.system.order` (an array of tiers), call `db.system.functions[name]()` for each
+name, skipping `schedulerSystem` and init-only systems (no function). So a simulation
+is fully runnable and testable with no rAF and no rendering attached.
 
 ## Layer
 
-`system-database.ts` extends the feature's **current top** main-service layer (`ActionDatabase` /
-`ServiceDatabase` / `ComputedDatabase` — whichever it built; systems come last in the pipeline, so
-they must sit atop any service/action layers, not a hardcoded `ComputedDatabase`), combined with
+`system-database.ts` extends the feature's top ecs layer (`ActionDatabase`, since every spec
+action has an action; systems come last in the pipeline), combined with
 `scheduler`, and declares the `systems` map **inline** (see above —
 this is the one facet not split one-per-file, because `create`'s `db` is only typed
-inline and standalone declarations break name inference). A `systems/` folder is
-optional: it holds extracted per-frame body helpers (`(db) => () => void`) only when
-an inline body grows too large — the schedule and registration still live inline in
-`system-database.ts`. The headless conformance test drives one frame via
-`db.system.order` / `db.system.functions` and compares to the `data/` `step` oracle,
-**reusing the feature projection** (`services/main-service/conformance.md`); it lives at
-`system-database/tick-loop.test.ts`. Test **selection/detection** logic (which
-entities collide, which pair resolves) separately, with seeded edge-case geometries
-— it's a system concern, not a transaction, and where subtle bugs hide.
+inline and standalone declarations break name inference). Extracted per-frame body
+helpers (`(db) => () => void`) sit beside it only when an inline body grows too large.
+
+## Conformance
+
+Each system has a same-named pure function in the spec's `systems` group
+(`../spec/index.md`), and the implementation declares how frame args reach the store
+and which systems the spec does not model (`conformance.md`):
+
+```ts
+frame: {
+  args: { dt: (db, dt) => { db.store.resources.frameDelta = dt; },
+          input: (db, input) => { db.store.resources.input = input; } },
+  unmodelled: { physics: "wasm rigid-body step", seedLevel: "init-only" },
+  cases: frameCases,   // ecs/conformance/frame.cases.ts
+},
+```
+
+- **Per system:** each case seeds the store, writes the case's args, runs **just that
+  system**, and compares with its spec function. Selection and detection logic (which
+  entities collide) is tested here, with seeded edge-case geometries.
+- **Per frame:** each frame case (`ecs/conformance/frame.cases.ts`) folds the spec
+  system functions in `db.system.order` and also runs one real frame; both must give
+  `after`. The order comes only from the `schedule` declarations.
+- **Shared guards:** the spec's `systems.noOp` states (game over) are checked against
+  every system, on both sides.
+- **Unmodelled systems** (wasm, init-only) are skipped on both sides and reported as
+  skipped tests. Every system is modelled or unmodelled — a compile error otherwise.
+  To test an unmodelled system's glue, put the wasm call behind a service and model it
+  with a fake.
 
 ## Iterating archetype rows
 

package/.claude/rules/features/ecs/transactions.md

@@ -0,0 +1,32 @@
+---
+paths:
+  - '**/features/*/ecs/transactions/**/*.ts'
+---
+
+# ecs/transactions/ — atomic mutations
+
+One mutation per file: a function taking the transaction store and Data args,
+returning `void`. All its writes commit or roll back together.
+
+```ts
+export const toggleComplete = (t: CoreDatabase.Store, { id }: { readonly id: Entity }) => {
+  const row = t.read(id, t.archetypes.Todo);
+  if (row === null) return;               // not a todo: no-op, as the spec says
+  t.update(id, { complete: !row.complete });
+};
+```
+
+- **Type the store** `CoreDatabase.Store`, or `IndexDatabase.Store` once it reads an
+  index.
+- **Validate and silently return** on illegal input rather than throwing, so replays
+  under sync stay idempotent.
+- **Guard by entity kind.** `t.read(id)` returns any entity. Check the entity is the
+  kind the op expects (`t.read(id, t.archetypes.Todo)`, or its tag) before writing, or
+  an id naming another kind gets written to.
+- **Decisions come from `data/values/` helpers**; the transaction reads the touched
+  slice, applies the helper, and writes the result.
+- **Transactions are an implementation detail.** Conformance checks the action that
+  calls them (`actions.md`), so a transaction needs no spec op of its own. Give one a
+  unit test only when it has logic no action covers.
+- **The barrel re-exports only mutations.** A read helper shared by several
+  transactions (`readBoard`) lives beside them, outside the barrel.