microecs 0.2.0__tar.gz → 0.2.2__tar.gz

{microecs-0.2.0 → microecs-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: microecs
-Version: 0.2.0
+Version: 0.2.2
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -24,7 +24,7 @@ There are only 5 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`.
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
 - `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other.
 - `System` is an abstract class that queries the `World` for a subset of `Pools` matching some components. It updates the entities in these pools given some logic (e.g. collisions, motion physics or simply calls the drawing functions). They are merely a convention, not tied to `World` per se.
 
@@ -59,7 +59,7 @@ class MotionSystem(TickSystem)
     def on_tick(self, world: World): # must override
         qr = world.query_and((HasPosition, HasVelocity))
         qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
-        # alternative, if you prefer per-pool update. May be faster in the extreme cases as it avoid the _Field creation
+        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
         for pool in qr.pool_list:
             pool.position[:] = pool.position + pool.velocity * DT
 

{microecs-0.2.0 → microecs-0.2.2}/README.md

@@ -12,7 +12,7 @@ There are only 5 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`.
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
 - `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other.
 - `System` is an abstract class that queries the `World` for a subset of `Pools` matching some components. It updates the entities in these pools given some logic (e.g. collisions, motion physics or simply calls the drawing functions). They are merely a convention, not tied to `World` per se.
 
@@ -47,7 +47,7 @@ class MotionSystem(TickSystem)
     def on_tick(self, world: World): # must override
         qr = world.query_and((HasPosition, HasVelocity))
         qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
-        # alternative, if you prefer per-pool update. May be faster in the extreme cases as it avoid the _Field creation
+        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
         for pool in qr.pool_list:
             pool.position[:] = pool.position + pool.velocity * DT
 

{microecs-0.2.0 → microecs-0.2.2}/microecs/query_result.py

@@ -19,6 +19,20 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
         """Creates a numpy array from the underlying pool parts"""
         return np.concatenate(self.parts)
 
+    def _chunk(self, x: T, i: int) -> T:
+        if isinstance(x, _Field):
+            return x.parts[i]
+        if isinstance(x, np.ndarray) and x.ndim == len(self.shape) and x.shape[0] == self.len:
+            return x[self._bounds[i]:self._bounds[i + 1]]
+        return x
+
+    def _apply_fn_on_parts(self, fn: Callable, op_args: list, **kwargs):
+        results = []
+        for i in range(len(self.parts)):
+            pool_args = [self._chunk(x, i) for x in op_args]
+            results.append(fn(*pool_args, **kwargs))
+        return _Field(results)
+
     def __array_ufunc__(self, ufunc, method, *inputs, out=None, **kwargs):
         """wrapper forelementwise (python) primitives, e.g. qr.position[:] += 1"""
         if method != "__call__":
@@ -38,20 +52,6 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
             return NotImplemented # add them manually
         return self._apply_fn_on_parts(func, args, **kwargs)
 
-    def _chunk(self, x: T, i: int) -> T:
-        if isinstance(x, _Field):
-            return x.parts[i]
-        if isinstance(x, np.ndarray) and x.ndim >= 1 and x.shape[0] == self.len:   # full-N raw -> slice per pool
-            return x[self._bounds[i]:self._bounds[i + 1]]
-        return x
-
-    def _apply_fn_on_parts(self, fn: Callable, op_args: list, **kwargs):
-        results = []
-        for i in range(len(self.parts)):
-            pool_args = [self._chunk(x, i) for x in op_args]
-            results.append(fn(*pool_args, **kwargs))
-        return _Field(results)
-
     # qr.position[:] = <field | scalar | per-entity broadcast>   -> scatter through the views
     def __setitem__(self, key, value):
         if (not (isinstance(key, slice) and key == slice(None)) and
@@ -85,23 +85,32 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
 
 class QueryResult:
     """A list of pools seen as a contiguous view. Fields (qr.position) implement array interface to look like numpy"""
-    def __init__(self, pool_list: list[Pool], field_shapes: dict[str, Shape], field_dtypes: dict[str, np.dtype]):
+    def __init__(self, pool_list: list[Pool], field_shapes: dict[str, Shape], field_dtypes: dict[str, np.dtype],
+                 entity_ids: np.ndarray):
         self.pool_list = pool_list
-        self.field_shapes = field_shapes
-        self.field_dtypes = field_dtypes
-        self.fields = list(field_shapes)
-        self.data: dict[str, np.ndarray] = {f: [p.data[f][0:len(p)] for p in pool_list] for f in field_shapes.keys()}
-        self.len = sum(len(pool) for pool in self.pool_list)
+        self.entity_ids = entity_ids
+        self._field_shapes = field_shapes
+        self._field_dtypes = field_dtypes
+        self._fields = list(field_shapes)
+        self._data: dict[str, np.ndarray] = {f: [p.data[f][0:len(p)] for p in pool_list] for f in field_shapes.keys()}
+        self._len = sum(len(pool) for pool in self.pool_list)
 
     def __getattr__(self, name):
-        if (data := self.__dict__.get("data")) is not None and name in data:
+        if (data := self.__dict__.get("_data")) is not None and name in data:
             # the 'or' part is in case no pools match the query and we want qr.position[:] += 1 still to work (noop)
-            return _Field(data[name] or [np.empty((0, *self.field_shapes[name]), self.field_dtypes[name])])
+            return _Field(data[name] or [np.empty((0, *self._field_shapes[name]), self._field_dtypes[name])])
         raise AttributeError(name)
 
+    def __setattr__(self, name, value):
+        if (data := self.__dict__.get("_data")) is not None and name in data:
+            getattr(self, name)[:] = value   # recarray semantics: assigning a field scatters into it
+            return
+        super().__setattr__(name, value)
+
     def __len__(self):
-        return self.len
+        return self._len
 
     def __repr__(self):
-        return (f"[QueryResult]\n- Fields: {self.fields}\n- Pools: {len(self.pool_list)}\n- Len: {self.len}"
-                f"\n- Shapes: {list(self.field_shapes.values())}\n- Dtypes: {list(self.field_dtypes.values())}")
+        return (f"[QueryResult]\n- Entities: {len(self.entity_ids)}\n- Fields: {self._fields}"
+                f"\n- Pools: {len(self.pool_list)}\n- Len: {self._len}\n- Shapes: {list(self._field_shapes.values())}"
+                f"\n- Dtypes: {list(self._field_dtypes.values())}")

{microecs-0.2.0 → microecs-0.2.2}/microecs/world.py

@@ -25,11 +25,12 @@ class World:
         self.component_to_field_names: dict[type, list[str]] = {t: list(t.__dataclass_fields__) for t in components}
         # entity id management
         self._eid_to_pool_ix: dict[EntityId, tuple[Pool, int]] = {}
-        self._pool_ix_to_eid: dict[tuple[Pool, int], EntityId] = {}
+        self._pool_ids: dict[Pool, list[EntityId]] = {}
         self._last_id: EntityId = -1
         self._live_ids: set[EntityId] = set() # 'eager' mode ids so e.g. calling remove_entity twice before update fails
         # command buffer management. {add/remove}_{entity/component} are lazy. Taken into account after update().
         self._command_buffer: list[Callable] = []
+        self._cache: dict[PoolKey, QueryResult] = {}
         logger.debug(f"Created scene with components: {self.component_names}")
 
     # public api
@@ -38,7 +39,10 @@ class World:
         """commits the underlying pool changes from the systems between two updates. Should be called in main loop."""
         for fn in self._command_buffer:
             fn()
-        self._command_buffer.clear()
+
+        if len(self._command_buffer) > 0:
+            self._command_buffer.clear()
+            self._cache.clear()
 
     def add_entity(self, components: list[ComponentType], **kwargs) -> EntityId:
         """Adds an entity to the world based on components (data->kwargs). Returns an entity id. Lazy; call update()"""
@@ -77,16 +81,22 @@ class World:
 
     def query_and(self, component_types: list[ComponentType]) -> QueryResult:
         """returns A QueryResult object with the entities that have all the requested components (entity ids too)."""
-        key = self._make_key(component_types)
+        # Note: we can cache the queries. The only time it can get invalidated (via public API) is at update().
+        if (key := self._make_key(component_types)) in self._cache:
+            return self._cache[key]
+
         res = []
         for archetype_key, archetype_pool in self.pools.items():
             if (archetype_key & key) == key: # key is subset of archetype_key
                 res.append(archetype_pool)
 
         field_names  = sum([self.component_to_field_names[c] for c in component_types], [])
-        field_shapes = sum([self.component_to_shapes[c]      for c in component_types], [])
-        field_dtypes = sum([self.component_to_dtypes[c]      for c in component_types], [])
-        return QueryResult(res, dict(zip(field_names, field_shapes)), dict(zip(field_names, field_dtypes)))
+        field_shapes = dict(zip(field_names, sum([self.component_to_shapes[c] for c in component_types], [])))
+        field_dtypes = dict(zip(field_names, sum([self.component_to_dtypes[c] for c in component_types], [])))
+        entity_ids = np.array(sum((self._pool_ids[p] for p in res), []), dtype="int64")
+
+        self._cache[key] = QueryResult(res, field_shapes=field_shapes, field_dtypes=field_dtypes, entity_ids=entity_ids)
+        return self._cache[key]
 
     # private stuff
 
@@ -97,20 +107,22 @@ class World:
         pool = self._get_entity_pool(components)
         pool_index = pool.add_entity(**kwargs)
         self._eid_to_pool_ix[entity_id] = (pool, pool_index)
-        self._pool_ix_to_eid[(pool, pool_index)] = entity_id
+        self._pool_ids.setdefault(pool, []).append(entity_id)
+        assert len(self._pool_ids[pool]) == len(pool), (pool, len(self._pool_ids[pool]), len(pool))
 
     def _pop_from_pool(self, entity_id: EntityId) -> tuple[EntityData, list[ComponentType]]:
         """common function that updates the entities inside a pool (after popswap) and removes them if they get empty"""
         old_pool, pool_ix = self._eid_to_pool_ix.pop(entity_id)
         entity = old_pool.pop_entity(pool_ix)
         components = self.pool_to_components[old_pool]
-        id_which_was_last_in_pool = self._pool_ix_to_eid.pop((old_pool, len(old_pool)))
+        id_which_was_last_in_pool = self._pool_ids[old_pool].pop()
         if entity_id != id_which_was_last_in_pool:
             self._eid_to_pool_ix[id_which_was_last_in_pool] = (old_pool, pool_ix) # we re-use the popped id (swapped)
-            self._pool_ix_to_eid[(old_pool, pool_ix)] = id_which_was_last_in_pool
+            self._pool_ids[old_pool][pool_ix] = id_which_was_last_in_pool
         if len(old_pool) == 0:
             del self.pools[self._make_key(components)]
             del self.pool_to_components[old_pool]
+            del self._pool_ids[old_pool]
         return entity, components
 
     def _do_add_component(self, entity_id: EntityId, component: ComponentType, **kwargs):
@@ -158,7 +170,9 @@ class World:
         return key
 
     def _check_components(self, components: list[ComponentType]):
+        _query_result_reserved_names = _qres = sorted(vars(QueryResult([], {}, {}, [])))
         _dtypes = {"float32", "int32", "bool", "str", "object"}
+
         for component in components:
             assert hasattr(component, "__dataclass_fields__"), f"Component '{component}' is not a dataclass"
             hints = get_type_hints(component) # make it work with from __future__ import annotations
@@ -167,6 +181,7 @@ class World:
                 assert _field.metadata.keys() == {"shape", "dtype"}, _field.metadata
                 assert isinstance(_field.metadata["shape"], tuple), _field.metadata["shape"]
                 assert isinstance(fmd := _field.metadata["dtype"], str) and fmd in _dtypes, f"{fmd} not in {_dtypes}"
+                assert field_name not in _query_result_reserved_names, f"Field '{field_name}' in {_qres}"
 
     def __len__(self):
         return len(self._live_ids)

{microecs-0.2.0 → microecs-0.2.2}/microecs.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: microecs
-Version: 0.2.0
+Version: 0.2.2
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -24,7 +24,7 @@ There are only 5 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `Wo
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`.
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()`. It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
 - `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other.
 - `System` is an abstract class that queries the `World` for a subset of `Pools` matching some components. It updates the entities in these pools given some logic (e.g. collisions, motion physics or simply calls the drawing functions). They are merely a convention, not tied to `World` per se.
 
@@ -59,7 +59,7 @@ class MotionSystem(TickSystem)
     def on_tick(self, world: World): # must override
         qr = world.query_and((HasPosition, HasVelocity))
         qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
-        # alternative, if you prefer per-pool update. May be faster in the extreme cases as it avoid the _Field creation
+        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
         for pool in qr.pool_list:
             pool.position[:] = pool.position + pool.velocity * DT
 

{microecs-0.2.0 → microecs-0.2.2}/setup.py

@@ -3,7 +3,7 @@ from pathlib import Path
 from setuptools import setup, find_packages
 
 NAME = "microecs"
-VERSION = "0.2.0"
+VERSION = "0.2.2"
 DESCRIPTION = "MicroECS: Minimal Entity Component System (ECS) in python and numpy"
 URL = "https://gitlab.com/meehai/microecs"