microecs 0.3.0__tar.gz → 0.3.2__tar.gz

{microecs-0.3.0 → microecs-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: microecs
-Version: 0.3.0
+Version: 0.3.2
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -24,8 +24,8 @@ There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_field_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
 Few relevant concepts:
 
@@ -43,7 +43,8 @@ from microecs import World, Component
 
 # components
 class HasPosition(Component):
-    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"}) # 'shape' + 'dtype' are always required
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
+    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasVelocity(Component):
     velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasColor(Component):
@@ -68,7 +69,7 @@ def main():
     render_system: list[Callable] = RenderSystem()
     update_systems: list[Callable] = [MotionSystem()]
 
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_field_metadata=None)
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
     for _ in range(n_objects):
         # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
         world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
@@ -87,3 +88,67 @@ def main():
         render_system(world=world)
         rl.EndDrawing()
 ```
+
+## The field (`_Field`) numpy contract
+
+`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
+result back. So the contract is exactly:
+
+> Any op that treats rows independently behaves **identically to numpy** on the concatenated
+> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+
+That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
+raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
+**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
+write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
+`test/unit/test_field_numpy_parity.py`.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.
+
+## Benchmark: ECS vs OOP
+
+The same physics step -- `pos += vel*dt` over N=100k entities split across 2 pools -- computed 8
+ways, all verified to produce the identical result. Reproduce with
+`python test/manual/perf/bench_ecs_vs_oop.py` (it prints `{mode: avg_seconds_per_step}`).
+
+| pattern | ns/entity | vs OOP-scalar |
+|---|---:|---:|
+| `micro-ecs-pool-vectorized` — `for pool: pool.f[:] = pool.f + …` | 0.9 | **52× faster** |
+| `micro-ecs-vectorized` — `qr.f[:] = qr.f + …` (the `_Field`) | 1.8 | **27× faster** |
+| **`oop-scalar`** — `for o: o.x += o.vx*dt` (python floats) | 48 | 1× (baseline) |
+| `oop-numpy` — objects holding `(2,)` numpy arrays | 605 | 13× slower |
+| `micro-ecs-zip-rows` — `for p, v in zip(qr.pos, qr.vel)` | 744 | 15× slower |
+| `micro-ecs-pool-loop` — `for pool: for i: pool.f[i]` | 870 | 18× slower |
+| `micro-ecs-get-entity` — `world.get_entity(eid)` per entity | 1674 | 35× slower |
+| `micro-ecs-index` — `qr.f[i]` per entity (an `np.searchsorted` each) | 4573 | 95× slower |
+
+*Intel Core Ultra 7 165H, single-threaded, python 3.12 / numpy 2.2. ns/entity = seconds/step ÷ N.*
+
+Three things to take from it:
+
+1. **Vectorized wins big.** Batched ops (`_Field` or per-pool) run at 1–2 ns/entity — **27–52×
+   faster** than the *fastest* OOP loop. Same for data-parallel branches: an `np.where` clamp or
+   bounce is ~34× faster than a per-entity `if`.
+2. **Per-entity loops are a cliff, not a tie.** Every per-entity microecs path is **15–95× slower**
+   than idiomatic float-based OOP — because microecs is numpy-backed, so a per-entity step pays
+   numpy's tiny-array overhead (`oop-numpy` shows the same ~13× tax). One unavoidable per-entity
+   pass (~750 ns/entity) costs ~500× a vectorized op (~1.5 ns) and will dominate the frame.
+3. **If you must loop, loop right.** `zip`-rows (15×) < pool-loop (18×) < `get_entity` (35×) <
+   `qr.f[i]` (95× — never in a hot loop; `qr.f[i] += …` also raises, you must bind the row first).
+
+**Rule of thumb:** keep systems vectorized and push branches into `np.where` / `np.clip`. If a
+workload is *irreducibly* per-entity (data-dependent control flow), plain python objects beat
+microecs ~15× — use them there. microecs is the right tool for **vectorizable** simulation.

microecs-0.3.2/README.md

@@ -0,0 +1,142 @@
+# MicroECS
+
+Minimal (~300 LoC) Entity Component System in python and numpy. Examples also use raylib for rendering.
+
+Usage:
+
+- `pip install -r requirements.txt`
+- Sandbox: `python main.py`
+- Tests: `pytest test/`
+
+There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `World`:
+
+- `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
+- `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+
+Few relevant concepts:
+
+- `Pool` operates on array indices, while `World` operates on entity IDs (also integers). This allows seamless movement between pools while the high-level systems still working as intended.
+- All mutable operations on `World` are lazy. These are: `add_entity`, `remove_entity`, `add_component`, `remove_component`. They are added to a command buffer which is only executed when calling `world.update()`.
+- `Systems` are a convention, they are not part of this library. They can be defined at application level and act as hooks or callbacks. The `World` object doesn't need to know more than entities and components.
+
+Super simplified main loop structure:
+
+```python
+from typing import Callable
+import numpy as np
+import raylib as rl
+from microecs import World, Component
+
+# components
+class HasPosition(Component):
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
+    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
+class HasVelocity(Component):
+    velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
+class HasColor(Component):
+    color: np.ndarray = field(metadata={"shape": (4, ), "dtype": "int32"})
+
+# systems: Note they are a convention!
+class RenderSystem:
+    def __call__(self, world: World):
+        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
+        for position, color in zip(query_result.position, query_result.color): # draw each entity
+            DrawEntity(position, color)
+
+class MotionSystem:
+    def __call__(self, world: World):
+        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
+        qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
+        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
+        for pool in qr.pool_list:
+            pool.position[:] = pool.position + pool.velocity * DT
+
+def main():
+    render_system: list[Callable] = RenderSystem()
+    update_systems: list[Callable] = [MotionSystem()]
+
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
+    for _ in range(n_objects):
+        # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
+        world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
+                         position=           np.array((x, y), "float32"), # data as kwargs
+                         color=         np.array("black", dtype="int32"),
+                         velocity=         np.array((vx, vy), "float32"))
+
+    while not rl.WindowShouldClose():
+        world.update() # must be called at each tick so the lazy methods are processed and entities are updated
+        # update stuff...
+        _ = [system(world=world) for system in update_systems]
+        # draw stuff, e.g. using raylib
+        rl.BeginDrawing()
+        rl.ClearBackground(rl.RAYWHITE)
+        rl.DrawFPS(rl.GetScreenWidth() - 100, 0)
+        render_system(world=world)
+        rl.EndDrawing()
+```
+
+## The field (`_Field`) numpy contract
+
+`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
+result back. So the contract is exactly:
+
+> Any op that treats rows independently behaves **identically to numpy** on the concatenated
+> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+
+That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
+raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
+**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
+write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
+`test/unit/test_field_numpy_parity.py`.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.
+
+## Benchmark: ECS vs OOP
+
+The same physics step -- `pos += vel*dt` over N=100k entities split across 2 pools -- computed 8
+ways, all verified to produce the identical result. Reproduce with
+`python test/manual/perf/bench_ecs_vs_oop.py` (it prints `{mode: avg_seconds_per_step}`).
+
+| pattern | ns/entity | vs OOP-scalar |
+|---|---:|---:|
+| `micro-ecs-pool-vectorized` — `for pool: pool.f[:] = pool.f + …` | 0.9 | **52× faster** |
+| `micro-ecs-vectorized` — `qr.f[:] = qr.f + …` (the `_Field`) | 1.8 | **27× faster** |
+| **`oop-scalar`** — `for o: o.x += o.vx*dt` (python floats) | 48 | 1× (baseline) |
+| `oop-numpy` — objects holding `(2,)` numpy arrays | 605 | 13× slower |
+| `micro-ecs-zip-rows` — `for p, v in zip(qr.pos, qr.vel)` | 744 | 15× slower |
+| `micro-ecs-pool-loop` — `for pool: for i: pool.f[i]` | 870 | 18× slower |
+| `micro-ecs-get-entity` — `world.get_entity(eid)` per entity | 1674 | 35× slower |
+| `micro-ecs-index` — `qr.f[i]` per entity (an `np.searchsorted` each) | 4573 | 95× slower |
+
+*Intel Core Ultra 7 165H, single-threaded, python 3.12 / numpy 2.2. ns/entity = seconds/step ÷ N.*
+
+Three things to take from it:
+
+1. **Vectorized wins big.** Batched ops (`_Field` or per-pool) run at 1–2 ns/entity — **27–52×
+   faster** than the *fastest* OOP loop. Same for data-parallel branches: an `np.where` clamp or
+   bounce is ~34× faster than a per-entity `if`.
+2. **Per-entity loops are a cliff, not a tie.** Every per-entity microecs path is **15–95× slower**
+   than idiomatic float-based OOP — because microecs is numpy-backed, so a per-entity step pays
+   numpy's tiny-array overhead (`oop-numpy` shows the same ~13× tax). One unavoidable per-entity
+   pass (~750 ns/entity) costs ~500× a vectorized op (~1.5 ns) and will dominate the frame.
+3. **If you must loop, loop right.** `zip`-rows (15×) < pool-loop (18×) < `get_entity` (35×) <
+   `qr.f[i]` (95× — never in a hot loop; `qr.f[i] += …` also raises, you must bind the row first).
+
+**Rule of thumb:** keep systems vectorized and push branches into `np.where` / `np.clip`. If a
+workload is *irreducibly* per-entity (data-dependent control flow), plain python objects beat
+microecs ~15× — use them there. microecs is the right tool for **vectorizable** simulation.

{microecs-0.3.0 → microecs-0.3.2}/microecs/pool.py

@@ -31,6 +31,7 @@ class Pool:
 
         for _field, field_shape, field_dtype in zip(self.fields, self.shapes, self.dtypes):
             new_item: np.ndarray = entity_fields[_field] # checked in World._get_entity_pool(entity).
+            assert isinstance(new_item, np.ndarray), f"{_field=} {new_item=} {type(new_item)}"
             assert new_item.shape == field_shape, f"{_field=} {new_item=}, {new_item.shape=}, {field_shape=}"
             assert np.issubdtype(new_item.dtype, field_dtype), f"{_field=} {new_item=} {new_item.dtype=} {field_dtype=}"
             self.data[_field][self.size] = new_item

{microecs-0.3.0 → microecs-0.3.2}/microecs/query_result.py

@@ -6,8 +6,6 @@ from .utils import Shape
 from .pool import Pool
 
 class _Field(np.lib.mixins.NDArrayOperatorsMixin):
-    _PER_POOL_OK = {np.where, np.clip}
-
     def __init__(self, parts: list[np.ndarray]):
         self.parts = parts
         self._lens = [len(p) for p in self.parts]
@@ -27,14 +25,20 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
         return x
 
     def _apply_fn_on_parts(self, fn: Callable, op_args: list, **kwargs):
+        # op args can be 1 element (-qr.velocity), 2 elements (qr.position * 0.1), 3 elements (np.where(a, b, c)), etc.
+        # all of them must be chunked based on how many we have in this _Field so each subpart is called independently.
         results = []
-        for i in range(len(self.parts)):
+        for i, part in enumerate(self.parts):
             pool_args = [self._chunk(x, i) for x in op_args]
-            results.append(fn(*pool_args, **kwargs))
+            part_result: _Field = fn(*pool_args, **kwargs)
+            # we expect f(arr(N, ...)) -> arr(N, ...) where N = number of items in the pool
+            # for e.g. np.linalg.norm(velocity, axis=1) should do (N, 2) -> (N, 1) so the first axis is preserved
+            assert len(part_result) == part.shape[0], f"Result: {part_result.shape} vs {part.shape}"
+            results.append(part_result)
         return _Field(results)
 
     def __array_ufunc__(self, ufunc, method, *inputs, out=None, **kwargs):
-        """wrapper forelementwise (python) primitives, e.g. qr.position[:] += 1"""
+        """wrapper for elementwise (python) primitives, e.g. qr.position[:] += 1"""
         if method != "__call__":
             return NotImplemented
         if out is None:
@@ -48,8 +52,6 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
 
     def __array_function__(self, func: Callable, _types, args: list, kwargs: dict):
         """wrapper for elementwise numpy functions, e.g. qr.velocity[:] = np.where(mask, -qr.velocity, qr.velocity)"""
-        if func not in self._PER_POOL_OK:
-            return NotImplemented # add them manually
         return self._apply_fn_on_parts(func, args, **kwargs)
 
     # qr.position[:] = <field | scalar | per-entity broadcast>   -> scatter through the views

{microecs-0.3.0 → microecs-0.3.2}/microecs/world.py

@@ -1,6 +1,7 @@
 """world.py - The world container for ECS. It manages all the pools (one per archetype). Entities are id-based."""
 from typing import Callable, get_type_hints
 from functools import partial
+from dataclasses import fields
 import numpy as np
 
 from .pool import Pool
@@ -13,27 +14,29 @@ class World:
     Generic container for pools of components. Newly added components are assigned a unique id and go in a pool
     Parameters
     - components The list of components that the world accepts
-    - extra_field_metadata The list of required extra metadata for each field besides shape and dtype.
+    - extra_metadata The list of required extra metadata for each field besides shape and dtype.
     """
-    def __init__(self, components: list[ComponentType], extra_field_metadata: list[str] | None = None):
-        self.extra_field_metadata = extra_field_metadata or []
+    def __init__(self, components: list[ComponentType], extra_metadata: list[str] | None = None):
+        self.extra_metadata = extra_metadata or []
+        assert isinstance(self.extra_metadata, list), type(self.extra_metadata)
         self._check_components(components)
         self.pools: dict[PoolKey, Pool] = {}
         self.pool_to_components: dict[Pool, list[ComponentType]] = {}
         # components management
         self.component_names = [x.__name__ for x in components]
         self.component_types = list(components)
-        self.component_to_bit: dict[type, int] = {t: 2**i for i, t in enumerate(components)} # unique bit for querying
-        self.component_to_shapes: dict[type, list[Shape]] = {
-            t: [f.metadata["shape"] for f in t.__dataclass_fields__.values()] for t in components}
-        self.component_to_dtypes: dict[type, list[str]] = {
-            t: [f.metadata["dtype"] for f in t.__dataclass_fields__.values()] for t in components}
-        self.component_to_field_names: dict[type, list[str]] = {t: list(t.__dataclass_fields__) for t in components}
+        self.component_to_bit: dict[ComponentType, int] = {t: 2**i for i, t in enumerate(components)} # bit for querying
+        self.component_to_shapes: dict[ComponentType, list[Shape]] = {
+            c: [f.metadata["shape"] for f in fields(c)] for c in components}
+        self.component_to_dtypes: dict[ComponentType, list[str]] = {
+            c: [f.metadata["dtype"] for f in fields(c)] for c in components}
+        self.component_to_field_names: dict[ComponentType, list[str]] = {
+            c: [f.name for f in fields(c)] for c in components}
         # entity id management
         self._eid_to_pool_ix: dict[EntityId, tuple[Pool, int]] = {}
         self._pool_ids: dict[Pool, list[EntityId]] = {}
         self._last_id: EntityId = -1
-        self._live_ids: set[EntityId] = set() # 'eager' mode ids so e.g. calling remove_entity twice before update fails
+        self.live_ids: set[EntityId] = set() # 'eager' mode ids so e.g. calling remove_entity twice before update fails
         # command buffer management. {add/remove}_{entity/component} are lazy. Taken into account after update().
         self._command_buffer: list[Callable] = []
         self._cache: dict[tuple[PoolKey, PoolKey], QueryResult] = {} # include+exclude key
@@ -54,15 +57,16 @@ class World:
         """Adds an entity to the world based on components (data->kwargs). Returns an entity id. Lazy; call update()"""
         self._check_components_against_pool(components, **kwargs)
         self._last_id += 1
-        self._live_ids.add(self._last_id)
+        self.live_ids.add(self._last_id)
         self._command_buffer.append(partial(self._add_to_pool, entity_id=self._last_id,
                                             components=components, **kwargs))
+        logger.debug(f"Created entity. ID: {self._last_id}. Components: {[c.__name__ for c in components]}")
         return self._last_id
 
     def remove_entity(self, entity_id: EntityId):
         """Removes an entity based on its unique entity id. Lazy; call update()"""
-        assert entity_id in self._live_ids, f"Entity id: {entity_id} not in {self._live_ids=}"
-        self._live_ids.remove(entity_id)
+        assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"
+        self.live_ids.remove(entity_id)
         self._command_buffer.append(partial(self._pop_from_pool, entity_id=entity_id))
 
     def get_entity(self, entity_id: EntityId) -> tuple[EntityData, list[ComponentType]]:
@@ -75,13 +79,13 @@ class World:
 
     def add_component(self, entity_id: EntityId, component: ComponentType, **kwargs):
         """Adds a component to an entity given its id. Component data is sent to kwargs. Lazy; call update()"""
-        assert entity_id in self._live_ids, f"Entity id: {entity_id} not in {self._live_ids=}"
+        assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"
         assert component in self.component_types, f"Component '{component}' not in {self.component_types}"
         self._command_buffer.append(partial(self._do_add_component, entity_id=entity_id, component=component, **kwargs))
 
     def remove_component(self, entity_id: EntityId, component: ComponentType):
         """Removes a component from an entity given its id. Lazy; call update()"""
-        assert entity_id in self._live_ids, f"Entity id: {entity_id} not in {self._live_ids=}"
+        assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"
         assert component in self.component_types, f"Component '{component}' not in {self.component_types}"
         self._command_buffer.append(partial(self._do_remove_component, entity_id=entity_id, component=component))
 
@@ -170,15 +174,15 @@ class World:
         for component in components:
             for _field in self.component_to_field_names[component]:
                 expected_fields.add(_field)
-                assert _field in entity_fields, f"Entity doesn't have '{component}/{_field}'"
+                assert _field in entity_fields, f"Entity doesn't have '{component.__name__}/{_field}'"
         assert (extra := set(entity_fields) - expected_fields) == set(), f"Extra fields: {extra}; {expected_fields=}"
 
     def _get_entity_pool(self, components: list[ComponentType]) -> Pool:
         if (key := self._make_key(components)) not in self.pools:
-            fields = sum([self.component_to_field_names[component] for component in components], []) # merge fields
+            _fields = sum([self.component_to_field_names[component] for component in components], []) # merge fields
             shapes = sum([self.component_to_shapes[component] for component in components], []) # merge shapes
             dtypes = sum([self.component_to_dtypes[component] for component in components], []) # merge dtypes
-            self.pools[key] = Pool(fields, shapes, dtypes)
+            self.pools[key] = Pool(_fields, shapes, dtypes)
             self.pool_to_components[self.pools[key]] = components
         return self.pools[key]
 
@@ -190,21 +194,23 @@ class World:
         return key
 
     def _check_components(self, components: list[ComponentType]):
-        _query_result_reserved_names = _qres = sorted(vars(QueryResult([], {}, {}, [])))
-        _dtypes = {"float32", "int32", "bool", "str", "object"}
+        qr_reserved_names = _qres = sorted(vars(QueryResult([], {}, {}, [])))
+        dtypes = {"float32", "int32", "bool", "str", "object"}
+        expected_meta = {"shape", "dtype", *self.extra_metadata}
 
         for component in components:
-            assert hasattr(component, "__dataclass_fields__"), f"Component '{component}' is not a dataclass"
+            cn = component.__name__
+            assert hasattr(component, "__dataclass_fields__"), f"Component '{cn}' is not a dataclass"
             hints = get_type_hints(component) # make it work with from __future__ import annotations
-            for field_name, _field in component.__dataclass_fields__.items():
-                assert hints[field_name] is np.ndarray, f"Field '{field_name}' of '{component=}' not an array: {_field}"
-                assert _field.metadata.keys() == {"shape", "dtype", *self.extra_field_metadata}, _field.metadata
-                assert isinstance(_field.metadata["shape"], tuple), _field.metadata["shape"]
-                assert isinstance(fmd := _field.metadata["dtype"], str) and fmd in _dtypes, f"{fmd} not in {_dtypes}"
-                assert field_name not in _query_result_reserved_names, f"Field '{field_name}' in {_qres}"
+            for f in fields(component):
+                assert hints[f.name] is np.ndarray, f"Field '{cn}/{f.name}' not an array: {f}"
+                assert f.name not in qr_reserved_names, f"Field '{cn}/{f.name}' in {qr_reserved_names}"
+                assert f.metadata.keys() == expected_meta, f"Field '{cn}/{f.name}':\n{f.metadata} vs\n{expected_meta}"
+                assert isinstance(f.metadata["shape"], tuple), f.metadata["shape"]
+                assert isinstance(fmd := f.metadata["dtype"], str) and fmd in dtypes, f"{fmd} not in {dtypes}"
 
     def __len__(self):
-        return len(self._live_ids)
+        return len(self.live_ids)
 
     def __repr__(self):
         return (f"[World]\n- Entities: {len(self)} (last id: {self._last_id})"

{microecs-0.3.0 → microecs-0.3.2}/microecs.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: microecs
-Version: 0.3.0
+Version: 0.3.2
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -24,8 +24,8 @@ There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_field_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
 Few relevant concepts:
 
@@ -43,7 +43,8 @@ from microecs import World, Component
 
 # components
 class HasPosition(Component):
-    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"}) # 'shape' + 'dtype' are always required
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
+    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasVelocity(Component):
     velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasColor(Component):
@@ -68,7 +69,7 @@ def main():
     render_system: list[Callable] = RenderSystem()
     update_systems: list[Callable] = [MotionSystem()]
 
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_field_metadata=None)
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
     for _ in range(n_objects):
         # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
         world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
@@ -87,3 +88,67 @@ def main():
         render_system(world=world)
         rl.EndDrawing()
 ```
+
+## The field (`_Field`) numpy contract
+
+`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
+result back. So the contract is exactly:
+
+> Any op that treats rows independently behaves **identically to numpy** on the concatenated
+> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+
+That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
+raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
+**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
+write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
+`test/unit/test_field_numpy_parity.py`.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.
+
+## Benchmark: ECS vs OOP
+
+The same physics step -- `pos += vel*dt` over N=100k entities split across 2 pools -- computed 8
+ways, all verified to produce the identical result. Reproduce with
+`python test/manual/perf/bench_ecs_vs_oop.py` (it prints `{mode: avg_seconds_per_step}`).
+
+| pattern | ns/entity | vs OOP-scalar |
+|---|---:|---:|
+| `micro-ecs-pool-vectorized` — `for pool: pool.f[:] = pool.f + …` | 0.9 | **52× faster** |
+| `micro-ecs-vectorized` — `qr.f[:] = qr.f + …` (the `_Field`) | 1.8 | **27× faster** |
+| **`oop-scalar`** — `for o: o.x += o.vx*dt` (python floats) | 48 | 1× (baseline) |
+| `oop-numpy` — objects holding `(2,)` numpy arrays | 605 | 13× slower |
+| `micro-ecs-zip-rows` — `for p, v in zip(qr.pos, qr.vel)` | 744 | 15× slower |
+| `micro-ecs-pool-loop` — `for pool: for i: pool.f[i]` | 870 | 18× slower |
+| `micro-ecs-get-entity` — `world.get_entity(eid)` per entity | 1674 | 35× slower |
+| `micro-ecs-index` — `qr.f[i]` per entity (an `np.searchsorted` each) | 4573 | 95× slower |
+
+*Intel Core Ultra 7 165H, single-threaded, python 3.12 / numpy 2.2. ns/entity = seconds/step ÷ N.*
+
+Three things to take from it:
+
+1. **Vectorized wins big.** Batched ops (`_Field` or per-pool) run at 1–2 ns/entity — **27–52×
+   faster** than the *fastest* OOP loop. Same for data-parallel branches: an `np.where` clamp or
+   bounce is ~34× faster than a per-entity `if`.
+2. **Per-entity loops are a cliff, not a tie.** Every per-entity microecs path is **15–95× slower**
+   than idiomatic float-based OOP — because microecs is numpy-backed, so a per-entity step pays
+   numpy's tiny-array overhead (`oop-numpy` shows the same ~13× tax). One unavoidable per-entity
+   pass (~750 ns/entity) costs ~500× a vectorized op (~1.5 ns) and will dominate the frame.
+3. **If you must loop, loop right.** `zip`-rows (15×) < pool-loop (18×) < `get_entity` (35×) <
+   `qr.f[i]` (95× — never in a hot loop; `qr.f[i] += …` also raises, you must bind the row first).
+
+**Rule of thumb:** keep systems vectorized and push branches into `np.where` / `np.clip`. If a
+workload is *irreducibly* per-entity (data-dependent control flow), plain python objects beat
+microecs ~15× — use them there. microecs is the right tool for **vectorizable** simulation.

{microecs-0.3.0 → microecs-0.3.2}/setup.py

@@ -3,7 +3,7 @@ from pathlib import Path
 from setuptools import setup, find_packages
 
 NAME = "microecs"
-VERSION = "0.3.0"
+VERSION = "0.3.2"
 DESCRIPTION = "MicroECS: Minimal Entity Component System (ECS) in python and numpy"
 URL = "https://gitlab.com/meehai/microecs"
 

microecs-0.3.0/README.md

@@ -1,77 +0,0 @@
-# MicroECS
-
-Minimal (~300 LoC) Entity Component System in python and numpy. Examples also use raylib for rendering.
-
-Usage:
-
-- `pip install -r requirements.txt`
-- Sandbox: `python main.py`
-- Tests: `pytest test/`
-
-There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `World`:
-
-- `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
-- `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_field_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
-
-Few relevant concepts:
-
-- `Pool` operates on array indices, while `World` operates on entity IDs (also integers). This allows seamless movement between pools while the high-level systems still working as intended.
-- All mutable operations on `World` are lazy. These are: `add_entity`, `remove_entity`, `add_component`, `remove_component`. They are added to a command buffer which is only executed when calling `world.update()`.
-- `Systems` are a convention, they are not part of this library. They can be defined at application level and act as hooks or callbacks. The `World` object doesn't need to know more than entities and components.
-
-Super simplified main loop structure:
-
-```python
-from typing import Callable
-import numpy as np
-import raylib as rl
-from microecs import World, Component
-
-# components
-class HasPosition(Component):
-    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"}) # 'shape' + 'dtype' are always required
-class HasVelocity(Component):
-    velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
-class HasColor(Component):
-    color: np.ndarray = field(metadata={"shape": (4, ), "dtype": "int32"})
-
-# systems: Note they are a convention!
-class RenderSystem:
-    def __call__(self, world: World):
-        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
-        for position, color in zip(query_result.position, query_result.color): # draw each entity
-            DrawEntity(position, color)
-
-class MotionSystem:
-    def __call__(self, world: World):
-        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
-        qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
-        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
-        for pool in qr.pool_list:
-            pool.position[:] = pool.position + pool.velocity * DT
-
-def main():
-    render_system: list[Callable] = RenderSystem()
-    update_systems: list[Callable] = [MotionSystem()]
-
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_field_metadata=None)
-    for _ in range(n_objects):
-        # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
-        world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
-                         position=           np.array((x, y), "float32"), # data as kwargs
-                         color=         np.array("black", dtype="int32"),
-                         velocity=         np.array((vx, vy), "float32"))
-
-    while not rl.WindowShouldClose():
-        world.update() # must be called at each tick so the lazy methods are processed and entities are updated
-        # update stuff...
-        _ = [system(world=world) for system in update_systems]
-        # draw stuff, e.g. using raylib
-        rl.BeginDrawing()
-        rl.ClearBackground(rl.RAYWHITE)
-        rl.DrawFPS(rl.GetScreenWidth() - 100, 0)
-        render_system(world=world)
-        rl.EndDrawing()
-```