microecs 0.2.4__tar.gz → 0.3.1__tar.gz

{microecs-0.2.4 → microecs-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: microecs
-Version: 0.2.4
+Version: 0.3.1
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -24,8 +24,8 @@ There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_field_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
 Few relevant concepts:
 
@@ -43,6 +43,7 @@ from microecs import World, Component
 
 # components
 class HasPosition(Component):
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
     position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasVelocity(Component):
     velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
@@ -51,14 +52,14 @@ class HasColor(Component):
 
 # systems: Note they are a convention!
 class RenderSystem:
-    def __call__(self, world: World): # must override
-        query_result = world.query_and((HasPosition, HasColor)) # contiguous-like view of all entities matching
+    def __call__(self, world: World):
+        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
         for position, color in zip(query_result.position, query_result.color): # draw each entity
             DrawEntity(position, color)
 
 class MotionSystem:
-    def __call__(self, world: World): # must override
-        qr = world.query_and((HasPosition, HasVelocity))
+    def __call__(self, world: World):
+        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
         qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
         # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
         for pool in qr.pool_list:
@@ -68,7 +69,7 @@ def main():
     render_system: list[Callable] = RenderSystem()
     update_systems: list[Callable] = [MotionSystem()]
 
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_field_metadata=None)
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
     for _ in range(n_objects):
         # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
         world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
@@ -87,3 +88,32 @@ def main():
         render_system(world=world)
         rl.EndDrawing()
 ```
+
+## The field (`_Field`) numpy contract
+
+`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
+result back. So the contract is exactly:
+
+> Any op that treats rows independently behaves **identically to numpy** on the concatenated
+> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+
+That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
+raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
+**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
+write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
+`test/unit/test_field_numpy_parity.py`.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.

{microecs-0.2.4 → microecs-0.3.1}/README.md

@@ -12,8 +12,8 @@ There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_field_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
 Few relevant concepts:
 
@@ -31,6 +31,7 @@ from microecs import World, Component
 
 # components
 class HasPosition(Component):
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
     position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasVelocity(Component):
     velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
@@ -39,14 +40,14 @@ class HasColor(Component):
 
 # systems: Note they are a convention!
 class RenderSystem:
-    def __call__(self, world: World): # must override
-        query_result = world.query_and((HasPosition, HasColor)) # contiguous-like view of all entities matching
+    def __call__(self, world: World):
+        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
         for position, color in zip(query_result.position, query_result.color): # draw each entity
             DrawEntity(position, color)
 
 class MotionSystem:
-    def __call__(self, world: World): # must override
-        qr = world.query_and((HasPosition, HasVelocity))
+    def __call__(self, world: World):
+        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
         qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
         # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
         for pool in qr.pool_list:
@@ -56,7 +57,7 @@ def main():
     render_system: list[Callable] = RenderSystem()
     update_systems: list[Callable] = [MotionSystem()]
 
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_field_metadata=None)
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
     for _ in range(n_objects):
         # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
         world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
@@ -75,3 +76,32 @@ def main():
         render_system(world=world)
         rl.EndDrawing()
 ```
+
+## The field (`_Field`) numpy contract
+
+`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
+result back. So the contract is exactly:
+
+> Any op that treats rows independently behaves **identically to numpy** on the concatenated
+> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+
+That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
+raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
+**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
+write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
+`test/unit/test_field_numpy_parity.py`.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.

{microecs-0.2.4 → microecs-0.3.1}/microecs/query_result.py

@@ -6,8 +6,6 @@ from .utils import Shape
 from .pool import Pool
 
 class _Field(np.lib.mixins.NDArrayOperatorsMixin):
-    _PER_POOL_OK = {np.where, np.clip}
-
     def __init__(self, parts: list[np.ndarray]):
         self.parts = parts
         self._lens = [len(p) for p in self.parts]
@@ -27,14 +25,20 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
         return x
 
     def _apply_fn_on_parts(self, fn: Callable, op_args: list, **kwargs):
+        # op args can be 1 element (-qr.velocity), 2 elements (qr.position * 0.1), 3 elements (np.where(a, b, c)), etc.
+        # all of them must be chunked based on how many we have in this _Field so each subpart is called independently.
         results = []
-        for i in range(len(self.parts)):
+        for i, part in enumerate(self.parts):
             pool_args = [self._chunk(x, i) for x in op_args]
-            results.append(fn(*pool_args, **kwargs))
+            part_result: _Field = fn(*pool_args, **kwargs)
+            # we expect f(arr(N, ...)) -> arr(N, ...) where N = number of items in the pool
+            # for e.g. np.linalg.norm(velocity, axis=1) should do (N, 2) -> (N, 1) so the first axis is preserved
+            assert len(part_result) == part.shape[0], f"Result: {part_result.shape} vs {part.shape}"
+            results.append(part_result)
         return _Field(results)
 
     def __array_ufunc__(self, ufunc, method, *inputs, out=None, **kwargs):
-        """wrapper forelementwise (python) primitives, e.g. qr.position[:] += 1"""
+        """wrapper for elementwise (python) primitives, e.g. qr.position[:] += 1"""
         if method != "__call__":
             return NotImplemented
         if out is None:
@@ -48,8 +52,6 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
 
     def __array_function__(self, func: Callable, _types, args: list, kwargs: dict):
         """wrapper for elementwise numpy functions, e.g. qr.velocity[:] = np.where(mask, -qr.velocity, qr.velocity)"""
-        if func not in self._PER_POOL_OK:
-            return NotImplemented # add them manually
         return self._apply_fn_on_parts(func, args, **kwargs)
 
     # qr.position[:] = <field | scalar | per-entity broadcast>   -> scatter through the views

{microecs-0.2.4 → microecs-0.3.1}/microecs/world.py

@@ -1,6 +1,7 @@
 """world.py - The world container for ECS. It manages all the pools (one per archetype). Entities are id-based."""
 from typing import Callable, get_type_hints
 from functools import partial
+from dataclasses import fields
 import numpy as np
 
 from .pool import Pool
@@ -13,30 +14,32 @@ class World:
     Generic container for pools of components. Newly added components are assigned a unique id and go in a pool
     Parameters
     - components The list of components that the world accepts
-    - extra_field_metadata The list of required extra metadata for each field besides shape and dtype.
+    - extra_metadata The list of required extra metadata for each field besides shape and dtype.
     """
-    def __init__(self, components: list[ComponentType], extra_field_metadata: list[str] | None = None):
-        self.extra_field_metadata = extra_field_metadata or []
+    def __init__(self, components: list[ComponentType], extra_metadata: list[str] | None = None):
+        self.extra_metadata = extra_metadata or []
+        assert isinstance(self.extra_metadata, list), type(self.extra_metadata)
         self._check_components(components)
         self.pools: dict[PoolKey, Pool] = {}
         self.pool_to_components: dict[Pool, list[ComponentType]] = {}
         # components management
         self.component_names = [x.__name__ for x in components]
         self.component_types = list(components)
-        self.component_to_bit: dict[type, int] = {t: 2**i for i, t in enumerate(components)} # unique bit for querying
-        self.component_to_shapes: dict[type, list[Shape]] = {
-            t: [f.metadata["shape"] for f in t.__dataclass_fields__.values()] for t in components}
-        self.component_to_dtypes: dict[type, list[str]] = {
-            t: [f.metadata["dtype"] for f in t.__dataclass_fields__.values()] for t in components}
-        self.component_to_field_names: dict[type, list[str]] = {t: list(t.__dataclass_fields__) for t in components}
+        self.component_to_bit: dict[ComponentType, int] = {t: 2**i for i, t in enumerate(components)} # bit for querying
+        self.component_to_shapes: dict[ComponentType, list[Shape]] = {
+            c: [f.metadata["shape"] for f in fields(c)] for c in components}
+        self.component_to_dtypes: dict[ComponentType, list[str]] = {
+            c: [f.metadata["dtype"] for f in fields(c)] for c in components}
+        self.component_to_field_names: dict[ComponentType, list[str]] = {
+            c: [f.name for f in fields(c)] for c in components}
         # entity id management
         self._eid_to_pool_ix: dict[EntityId, tuple[Pool, int]] = {}
         self._pool_ids: dict[Pool, list[EntityId]] = {}
         self._last_id: EntityId = -1
-        self._live_ids: set[EntityId] = set() # 'eager' mode ids so e.g. calling remove_entity twice before update fails
+        self.live_ids: set[EntityId] = set() # 'eager' mode ids so e.g. calling remove_entity twice before update fails
         # command buffer management. {add/remove}_{entity/component} are lazy. Taken into account after update().
         self._command_buffer: list[Callable] = []
-        self._cache: dict[PoolKey, QueryResult] = {}
+        self._cache: dict[tuple[PoolKey, PoolKey], QueryResult] = {} # include+exclude key
         logger.debug(f"Created scene with components: {self.component_names}")
 
     # public api
@@ -54,15 +57,16 @@ class World:
         """Adds an entity to the world based on components (data->kwargs). Returns an entity id. Lazy; call update()"""
         self._check_components_against_pool(components, **kwargs)
         self._last_id += 1
-        self._live_ids.add(self._last_id)
+        self.live_ids.add(self._last_id)
         self._command_buffer.append(partial(self._add_to_pool, entity_id=self._last_id,
                                             components=components, **kwargs))
+        logger.debug(f"Created entity. ID: {self._last_id}. Components: {[c.__name__ for c in components]}")
         return self._last_id
 
     def remove_entity(self, entity_id: EntityId):
         """Removes an entity based on its unique entity id. Lazy; call update()"""
-        assert entity_id in self._live_ids, f"Entity id: {entity_id} not in {self._live_ids=}"
-        self._live_ids.remove(entity_id)
+        assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"
+        self.live_ids.remove(entity_id)
         self._command_buffer.append(partial(self._pop_from_pool, entity_id=entity_id))
 
     def get_entity(self, entity_id: EntityId) -> tuple[EntityData, list[ComponentType]]:
@@ -75,30 +79,44 @@ class World:
 
     def add_component(self, entity_id: EntityId, component: ComponentType, **kwargs):
         """Adds a component to an entity given its id. Component data is sent to kwargs. Lazy; call update()"""
-        assert entity_id in self._live_ids, f"Entity id: {entity_id} not in {self._live_ids=}"
+        assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"
         assert component in self.component_types, f"Component '{component}' not in {self.component_types}"
         self._command_buffer.append(partial(self._do_add_component, entity_id=entity_id, component=component, **kwargs))
 
     def remove_component(self, entity_id: EntityId, component: ComponentType):
         """Removes a component from an entity given its id. Lazy; call update()"""
-        assert entity_id in self._live_ids, f"Entity id: {entity_id} not in {self._live_ids=}"
+        assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"
         assert component in self.component_types, f"Component '{component}' not in {self.component_types}"
         self._command_buffer.append(partial(self._do_remove_component, entity_id=entity_id, component=component))
 
-    def query_and(self, component_types: list[ComponentType]) -> QueryResult:
-        """returns A QueryResult object with the entities that have all the requested components (entity ids too)."""
+    def query(self, *include: ComponentType, exclude: list[ComponentType] | None = None) -> QueryResult:
+        """
+        Queries the world for entities that match the include set of components.
+        Syntax: `world.query(A, B, exclude=[C, D])` is, in logic form, a chain of 'ands': `A & B & ~C & ~D`.
+        Return: A `QueryResult` object with the entities that have all the requested components. Has `EntityIds` too.
+        """
+
         # Note: we can cache the queries. The only time it can get invalidated (via public API) is at update().
-        if (key := self._make_key(component_types)) in self._cache:
+        include_key = self._make_key(include)
+        exclude_key = self._make_key(exclude or [])
+        if (key := (include_key, exclude_key)) in self._cache:
             return self._cache[key]
 
+        # archetype_key = (1 0 0 1 1) &
+        #           key = (1 0 0 0 1)
+        #              -> (1 0 0 0 1) OK
+        # but
+        # archetype_key = (1 0 0 1 1) &
+        #           key = (1 0 1 0 0)
+        #              -> (1 0 0 0 0) NOT OK
         res = []
         for archetype_key, archetype_pool in self.pools.items():
-            if (archetype_key & key) == key: # key is subset of archetype_key
+            if (archetype_key & include_key) == include_key and (archetype_key & exclude_key) == 0:
                 res.append(archetype_pool)
 
-        field_names  = sum([self.component_to_field_names[c] for c in component_types], [])
-        field_shapes = dict(zip(field_names, sum([self.component_to_shapes[c] for c in component_types], [])))
-        field_dtypes = dict(zip(field_names, sum([self.component_to_dtypes[c] for c in component_types], [])))
+        field_names = sum([self.component_to_field_names[c] for c in include], [])
+        field_shapes = dict(zip(field_names, sum([self.component_to_shapes[c] for c in include], [])))
+        field_dtypes = dict(zip(field_names, sum([self.component_to_dtypes[c] for c in include], [])))
         entity_ids = np.array(sum((self._pool_ids[p] for p in res), []), dtype="int64")
 
         self._cache[key] = QueryResult(res, field_shapes=field_shapes, field_dtypes=field_dtypes, entity_ids=entity_ids)
@@ -156,15 +174,15 @@ class World:
         for component in components:
             for _field in self.component_to_field_names[component]:
                 expected_fields.add(_field)
-                assert _field in entity_fields, f"Entity doesn't have '{component}/{_field}'"
+                assert _field in entity_fields, f"Entity doesn't have '{component.__name__}/{_field}'"
         assert (extra := set(entity_fields) - expected_fields) == set(), f"Extra fields: {extra}; {expected_fields=}"
 
     def _get_entity_pool(self, components: list[ComponentType]) -> Pool:
         if (key := self._make_key(components)) not in self.pools:
-            fields = sum([self.component_to_field_names[component] for component in components], []) # merge fields
+            _fields = sum([self.component_to_field_names[component] for component in components], []) # merge fields
             shapes = sum([self.component_to_shapes[component] for component in components], []) # merge shapes
             dtypes = sum([self.component_to_dtypes[component] for component in components], []) # merge dtypes
-            self.pools[key] = Pool(fields, shapes, dtypes)
+            self.pools[key] = Pool(_fields, shapes, dtypes)
             self.pool_to_components[self.pools[key]] = components
         return self.pools[key]
 
@@ -176,21 +194,23 @@ class World:
         return key
 
     def _check_components(self, components: list[ComponentType]):
-        _query_result_reserved_names = _qres = sorted(vars(QueryResult([], {}, {}, [])))
-        _dtypes = {"float32", "int32", "bool", "str", "object"}
+        qr_reserved_names = _qres = sorted(vars(QueryResult([], {}, {}, [])))
+        dtypes = {"float32", "int32", "bool", "str", "object"}
+        expected_meta = {"shape", "dtype", *self.extra_metadata}
 
         for component in components:
-            assert hasattr(component, "__dataclass_fields__"), f"Component '{component}' is not a dataclass"
+            cn = component.__name__
+            assert hasattr(component, "__dataclass_fields__"), f"Component '{cn}' is not a dataclass"
             hints = get_type_hints(component) # make it work with from __future__ import annotations
-            for field_name, _field in component.__dataclass_fields__.items():
-                assert hints[field_name] is np.ndarray, f"Field '{field_name}' of '{component=}' not an array: {_field}"
-                assert _field.metadata.keys() == {"shape", "dtype", *self.extra_field_metadata}, _field.metadata
-                assert isinstance(_field.metadata["shape"], tuple), _field.metadata["shape"]
-                assert isinstance(fmd := _field.metadata["dtype"], str) and fmd in _dtypes, f"{fmd} not in {_dtypes}"
-                assert field_name not in _query_result_reserved_names, f"Field '{field_name}' in {_qres}"
+            for f in fields(component):
+                assert hints[f.name] is np.ndarray, f"Field '{cn}/{f.name}' not an array: {f}"
+                assert f.name not in qr_reserved_names, f"Field '{cn}/{f.name}' in {qr_reserved_names}"
+                assert f.metadata.keys() == expected_meta, f"Field '{cn}/{f.name}':\n{f.metadata} vs\n{expected_meta}"
+                assert isinstance(f.metadata["shape"], tuple), f.metadata["shape"]
+                assert isinstance(fmd := f.metadata["dtype"], str) and fmd in dtypes, f"{fmd} not in {dtypes}"
 
     def __len__(self):
-        return len(self._live_ids)
+        return len(self.live_ids)
 
     def __repr__(self):
         return (f"[World]\n- Entities: {len(self)} (last id: {self._last_id})"

{microecs-0.2.4 → microecs-0.3.1}/microecs.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: microecs
-Version: 0.2.4
+Version: 0.3.1
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -24,8 +24,8 @@ There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_field_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
 Few relevant concepts:
 
@@ -43,6 +43,7 @@ from microecs import World, Component
 
 # components
 class HasPosition(Component):
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
     position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
 class HasVelocity(Component):
     velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
@@ -51,14 +52,14 @@ class HasColor(Component):
 
 # systems: Note they are a convention!
 class RenderSystem:
-    def __call__(self, world: World): # must override
-        query_result = world.query_and((HasPosition, HasColor)) # contiguous-like view of all entities matching
+    def __call__(self, world: World):
+        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
         for position, color in zip(query_result.position, query_result.color): # draw each entity
             DrawEntity(position, color)
 
 class MotionSystem:
-    def __call__(self, world: World): # must override
-        qr = world.query_and((HasPosition, HasVelocity))
+    def __call__(self, world: World):
+        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
         qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
         # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
         for pool in qr.pool_list:
@@ -68,7 +69,7 @@ def main():
     render_system: list[Callable] = RenderSystem()
     update_systems: list[Callable] = [MotionSystem()]
 
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_field_metadata=None)
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
     for _ in range(n_objects):
         # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
         world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
@@ -87,3 +88,32 @@ def main():
         render_system(world=world)
         rl.EndDrawing()
 ```
+
+## The field (`_Field`) numpy contract
+
+`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
+result back. So the contract is exactly:
+
+> Any op that treats rows independently behaves **identically to numpy** on the concatenated
+> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+
+That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
+raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
+**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
+write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
+`test/unit/test_field_numpy_parity.py`.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.

{microecs-0.2.4 → microecs-0.3.1}/setup.py

@@ -3,7 +3,7 @@ from pathlib import Path
 from setuptools import setup, find_packages
 
 NAME = "microecs"
-VERSION = "0.2.4"
+VERSION = "0.3.1"
 DESCRIPTION = "MicroECS: Minimal Entity Component System (ECS) in python and numpy"
 URL = "https://gitlab.com/meehai/microecs"