microecs 0.3.1__tar.gz → 0.3.3__tar.gz

{microecs-0.3.1 → microecs-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: microecs
-Version: 0.3.1
+Version: 0.3.3
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -9,6 +9,14 @@ Description-Content-Type: text/markdown
 License-File: LICENSE.TXT
 Requires-Dist: numpy>=2.2.0
 Requires-Dist: loggez>=0.8
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
 
 # MicroECS
 
@@ -16,24 +24,35 @@ Minimal (~300 LoC) Entity Component System in python and numpy. Examples also us
 
 Usage:
 
-- `pip install -r requirements.txt`
-- Sandbox: `python main.py`
-- Tests: `pytest test/`
+- Via pip: `pip install microecs`
+- From source code:
+```bash
+git clone https://gitlab.com/meehai/microecs      # clone the source code
+cd microecs                                       # go in the cloned directory
+python -m venv .venv && source .venv/bin/activate # make a virtual env, optional but useful
+python -m pip install -e .                        # install micro ecs in this virtual env
+python -m pytest test/                            # run the unit & integration tests to verify installation
+python examples/01-hello-world.py                 # run the basic hello world example (others in that dir)
+```
+
+Docs: [meehai.gitlab.io/microecs](https://meehai.gitlab.io/microecs/)
 
-There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `World`:
+## Relevant primitives: `Component`, `Pool`, `QueryResult`, `World`
+
+There are only 4 primitives (bottom up):
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's interface. For all intents and purposes it should feel like a `(N, ...)` view over all selected entities. To get a proper numpy array, use `qr.numpy()`. To iterate over each entity in a query result (e.g. rendering), use `for eid, position in zip(qr.entity_ids, qr.position): ...`.
 - `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
-Few relevant concepts:
+### Few relevant concepts:
 
 - `Pool` operates on array indices, while `World` operates on entity IDs (also integers). This allows seamless movement between pools while the high-level systems still working as intended.
 - All mutable operations on `World` are lazy. These are: `add_entity`, `remove_entity`, `add_component`, `remove_component`. They are added to a command buffer which is only executed when calling `world.update()`.
 - `Systems` are a convention, they are not part of this library. They can be defined at application level and act as hooks or callbacks. The `World` object doesn't need to know more than entities and components.
 
-Super simplified main loop structure:
+## Super simplified main loop structure
 
 ```python
 from typing import Callable
@@ -89,29 +108,67 @@ def main():
         rl.EndDrawing()
 ```
 
-## The field (`_Field`) numpy contract
+## Per-entity systems (e.g. rendering, foreign APIs)
+
+Not every system vectorizes. A renderer calls a draw function per primitive; same for any per-entity foreign API. That's a per-entity system inside an otherwise-vectorized app. The cheapest way to iterate through all the entities is by using `zip()` on the `QueryResult` object on the fields you need, e.g:
+
+```python
+qr = world.query(HasPosition, HasColor, HasRadius)
+for pos, color, radius in zip(qr.position, qr.color, qr.radius):     # one entity per step, fields aligned
+    rl.DrawCircle(int(pos[0]), int(pos[1]), float(radius[0]), color) # per-entity by necessity (no "draw all")
+```
+
+There are other ways to extract a single entity e.g. `qr.some_field[i]` or `e = world.get_entity(eid)`, but these are slower in the general case and should be avoided in expensive loops, like rendering, even if they are a bit more ergonomic. See benchmark below.
+
+<details>
+<summary> Microbenchmark: ECS vs OOP on a simple physics step </summary>
+
+Benchmark: we run the same physics step `pos += vel*dt` over N=100k entities split across 2 pools in various ways (ECS or OOP). All methods are verified to produce the identical result. Reproduce with `python test/manual/perf/bench_ecs_vs_oop.py` (it prints `{mode: avg_seconds_per_step}`).
+
+| pattern | ns/entity | vs OOP-scalar |
+|---|---:|---:|
+| `micro-ecs-pool-vectorized` — `for pool: pool.f[:] = pool.f + …` | 0.9 | **52× faster** |
+| `micro-ecs-vectorized` — `qr.f[:] = qr.f + …` (the `_Field`) | 1.8 | **27× faster** |
+| **`oop-scalar`** — `for o: o.x += o.vx*dt` (python floats) | 48 | 1× (baseline) |
+| `oop-numpy` — objects holding `(2,)` numpy arrays | 605 | 13× slower |
+| `micro-ecs-zip-rows` — `for p, v in zip(qr.pos, qr.vel)` | 744 | 15× slower |
+| `micro-ecs-pool-loop` — `for pool: for i: pool.f[i]` | 870 | 18× slower |
+| `micro-ecs-get-entity` — `world.get_entity(eid)` per entity | 1674 | 35× slower |
+| `micro-ecs-index` — `qr.f[i]` per entity (an `np.searchsorted` each) | 4573 | 95× slower |
+
+Three things to take from it:
+
+1. **Vectorized wins big.** Batched ops (`_Field` or per-pool) run at 1–2 ns/entity — **27–52×
+   faster** than the *fastest* OOP loop. Same for data-parallel branches: an `np.where` clamp or
+   bounce is ~34× faster than a per-entity `if`.
+2. **Per-entity loops are a cliff, not a tie.** Every per-entity microecs path is **15–95× slower**
+   than idiomatic float-based OOP — because microecs is numpy-backed, so a per-entity step pays
+   numpy's tiny-array overhead (`oop-numpy` shows the same ~13× tax). One unavoidable per-entity
+   pass (~750 ns/entity) costs ~500× a vectorized op (~1.5 ns) and will dominate the frame.
+3. **If you must loop, loop right.** `zip`-rows (15×) < pool-loop (18×) < `get_entity` (35×) <
+   `qr.f[i]` (95× — never in a hot loop; `qr.f[i] += …` also raises, you must bind the row first).
+
+**Rule of thumb:** keep systems vectorized and push branches into `np.where` / `np.clip`. If a
+workload is *irreducibly* per-entity (data-dependent control flow), plain python objects beat
+microecs ~15× — use them there. microecs is the right tool for **vectorizable** simulation.
+
+</details>
 
-`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
-numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
-result back. So the contract is exactly:
+## How much are `Pool` and `QueryResult` numpy-like and corner cases
 
-> Any op that treats rows independently behaves **identically to numpy** on the concatenated
-> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+Given `qr=world.query(A, B)`, then `qr.position` returns a `_Field`: a view over the matching pools that behaves like one contiguous-like `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches result back.
 
-That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
-raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
-**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
-write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
-`test/unit/test_field_numpy_parity.py`.
+That covers elementwise math and ufuncs (e.g. `np.where`, `np.linalg.norm(..., axis=1)` etc.), broadcasting (every operand shape numpy accepts, and it raises on the ones numpy rejects). See `test/unit/test_field_numpy_parity.py` for a whole set of operations comparing both.
 
 Edge cases worth knowing:
 
-- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+- **Not a full ndarray — these raise, never lie.** Entity-axis indexing beyond a single
   `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
   methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
   first with `qr.f.numpy()`.
 - **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
-  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  `axis=0` run within each pool and reset at pool boundaries — they do **not** see all entities
   at once, so they differ from numpy. They're allowed, but if you want a global result, do
   `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
   longer matches the pool's row count.

microecs-0.3.3/README.md

@@ -0,0 +1,156 @@
+# MicroECS
+
+Minimal (~300 LoC) Entity Component System in python and numpy. Examples also use raylib for rendering.
+
+Usage:
+
+- Via pip: `pip install microecs`
+- From source code:
+```bash
+git clone https://gitlab.com/meehai/microecs      # clone the source code
+cd microecs                                       # go in the cloned directory
+python -m venv .venv && source .venv/bin/activate # make a virtual env, optional but useful
+python -m pip install -e .                        # install micro ecs in this virtual env
+python -m pytest test/                            # run the unit & integration tests to verify installation
+python examples/01-hello-world.py                 # run the basic hello world example (others in that dir)
+```
+
+Docs: [meehai.gitlab.io/microecs](https://meehai.gitlab.io/microecs/)
+
+## Relevant primitives: `Component`, `Pool`, `QueryResult`, `World`
+
+There are only 4 primitives (bottom up):
+
+- `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
+- `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's interface. For all intents and purposes it should feel like a `(N, ...)` view over all selected entities. To get a proper numpy array, use `qr.numpy()`. To iterate over each entity in a query result (e.g. rendering), use `for eid, position in zip(qr.entity_ids, qr.position): ...`.
+- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
+
+### Few relevant concepts:
+
+- `Pool` operates on array indices, while `World` operates on entity IDs (also integers). This allows seamless movement between pools while the high-level systems still working as intended.
+- All mutable operations on `World` are lazy. These are: `add_entity`, `remove_entity`, `add_component`, `remove_component`. They are added to a command buffer which is only executed when calling `world.update()`.
+- `Systems` are a convention, they are not part of this library. They can be defined at application level and act as hooks or callbacks. The `World` object doesn't need to know more than entities and components.
+
+## Super simplified main loop structure
+
+```python
+from typing import Callable
+import numpy as np
+import raylib as rl
+from microecs import World, Component
+
+# components
+class HasPosition(Component):
+    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
+    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
+class HasVelocity(Component):
+    velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
+class HasColor(Component):
+    color: np.ndarray = field(metadata={"shape": (4, ), "dtype": "int32"})
+
+# systems: Note they are a convention!
+class RenderSystem:
+    def __call__(self, world: World):
+        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
+        for position, color in zip(query_result.position, query_result.color): # draw each entity
+            DrawEntity(position, color)
+
+class MotionSystem:
+    def __call__(self, world: World):
+        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
+        qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
+        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
+        for pool in qr.pool_list:
+            pool.position[:] = pool.position + pool.velocity * DT
+
+def main():
+    render_system: list[Callable] = RenderSystem()
+    update_systems: list[Callable] = [MotionSystem()]
+
+    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
+    for _ in range(n_objects):
+        # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
+        world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
+                         position=           np.array((x, y), "float32"), # data as kwargs
+                         color=         np.array("black", dtype="int32"),
+                         velocity=         np.array((vx, vy), "float32"))
+
+    while not rl.WindowShouldClose():
+        world.update() # must be called at each tick so the lazy methods are processed and entities are updated
+        # update stuff...
+        _ = [system(world=world) for system in update_systems]
+        # draw stuff, e.g. using raylib
+        rl.BeginDrawing()
+        rl.ClearBackground(rl.RAYWHITE)
+        rl.DrawFPS(rl.GetScreenWidth() - 100, 0)
+        render_system(world=world)
+        rl.EndDrawing()
+```
+
+## Per-entity systems (e.g. rendering, foreign APIs)
+
+Not every system vectorizes. A renderer calls a draw function per primitive; same for any per-entity foreign API. That's a per-entity system inside an otherwise-vectorized app. The cheapest way to iterate through all the entities is by using `zip()` on the `QueryResult` object on the fields you need, e.g:
+
+```python
+qr = world.query(HasPosition, HasColor, HasRadius)
+for pos, color, radius in zip(qr.position, qr.color, qr.radius):     # one entity per step, fields aligned
+    rl.DrawCircle(int(pos[0]), int(pos[1]), float(radius[0]), color) # per-entity by necessity (no "draw all")
+```
+
+There are other ways to extract a single entity e.g. `qr.some_field[i]` or `e = world.get_entity(eid)`, but these are slower in the general case and should be avoided in expensive loops, like rendering, even if they are a bit more ergonomic. See benchmark below.
+
+<details>
+<summary> Microbenchmark: ECS vs OOP on a simple physics step </summary>
+
+Benchmark: we run the same physics step `pos += vel*dt` over N=100k entities split across 2 pools in various ways (ECS or OOP). All methods are verified to produce the identical result. Reproduce with `python test/manual/perf/bench_ecs_vs_oop.py` (it prints `{mode: avg_seconds_per_step}`).
+
+| pattern | ns/entity | vs OOP-scalar |
+|---|---:|---:|
+| `micro-ecs-pool-vectorized` — `for pool: pool.f[:] = pool.f + …` | 0.9 | **52× faster** |
+| `micro-ecs-vectorized` — `qr.f[:] = qr.f + …` (the `_Field`) | 1.8 | **27× faster** |
+| **`oop-scalar`** — `for o: o.x += o.vx*dt` (python floats) | 48 | 1× (baseline) |
+| `oop-numpy` — objects holding `(2,)` numpy arrays | 605 | 13× slower |
+| `micro-ecs-zip-rows` — `for p, v in zip(qr.pos, qr.vel)` | 744 | 15× slower |
+| `micro-ecs-pool-loop` — `for pool: for i: pool.f[i]` | 870 | 18× slower |
+| `micro-ecs-get-entity` — `world.get_entity(eid)` per entity | 1674 | 35× slower |
+| `micro-ecs-index` — `qr.f[i]` per entity (an `np.searchsorted` each) | 4573 | 95× slower |
+
+Three things to take from it:
+
+1. **Vectorized wins big.** Batched ops (`_Field` or per-pool) run at 1–2 ns/entity — **27–52×
+   faster** than the *fastest* OOP loop. Same for data-parallel branches: an `np.where` clamp or
+   bounce is ~34× faster than a per-entity `if`.
+2. **Per-entity loops are a cliff, not a tie.** Every per-entity microecs path is **15–95× slower**
+   than idiomatic float-based OOP — because microecs is numpy-backed, so a per-entity step pays
+   numpy's tiny-array overhead (`oop-numpy` shows the same ~13× tax). One unavoidable per-entity
+   pass (~750 ns/entity) costs ~500× a vectorized op (~1.5 ns) and will dominate the frame.
+3. **If you must loop, loop right.** `zip`-rows (15×) < pool-loop (18×) < `get_entity` (35×) <
+   `qr.f[i]` (95× — never in a hot loop; `qr.f[i] += …` also raises, you must bind the row first).
+
+**Rule of thumb:** keep systems vectorized and push branches into `np.where` / `np.clip`. If a
+workload is *irreducibly* per-entity (data-dependent control flow), plain python objects beat
+microecs ~15× — use them there. microecs is the right tool for **vectorizable** simulation.
+
+</details>
+
+## How much are `Pool` and `QueryResult` numpy-like and corner cases
+
+Given `qr=world.query(A, B)`, then `qr.position` returns a `_Field`: a view over the matching pools that behaves like one contiguous-like `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches result back.
+
+That covers elementwise math and ufuncs (e.g. `np.where`, `np.linalg.norm(..., axis=1)` etc.), broadcasting (every operand shape numpy accepts, and it raises on the ones numpy rejects). See `test/unit/test_field_numpy_parity.py` for a whole set of operations comparing both.
+
+Edge cases worth knowing:
+
+- **Not a full ndarray — these raise, never lie.** Entity-axis indexing beyond a single
+  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
+  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
+  first with `qr.f.numpy()`.
+- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
+  `axis=0` run within each pool and reset at pool boundaries — they do **not** see all entities
+  at once, so they differ from numpy. They're allowed, but if you want a global result, do
+  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
+  longer matches the pool's row count.
+- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
+  mix a `_Field` from one `world.query(...)` into an op on another.

{microecs-0.3.1 → microecs-0.3.3}/microecs/pool.py

@@ -17,7 +17,7 @@ class Pool:
         self.shapes = shapes
         self.dtypes = dtypes
 
-        self.data: dict[str, np.ndarray] = {} # the actual data is stored in a dynamic arrray, one per field
+        self.data: dict[str, np.ndarray] = {} # the actual data is stored in dict of dynamic arrrays, one per field
         self.size = 0
         self.capacity = Pool.INITIAL_CAPACITY
         for _field, shape, dtype in zip(fields, shapes, dtypes):
@@ -31,6 +31,7 @@ class Pool:
 
         for _field, field_shape, field_dtype in zip(self.fields, self.shapes, self.dtypes):
             new_item: np.ndarray = entity_fields[_field] # checked in World._get_entity_pool(entity).
+            assert isinstance(new_item, np.ndarray), f"{_field=} {new_item=} {type(new_item)}"
             assert new_item.shape == field_shape, f"{_field=} {new_item=}, {new_item.shape=}, {field_shape=}"
             assert np.issubdtype(new_item.dtype, field_dtype), f"{_field=} {new_item=} {new_item.dtype=} {field_dtype=}"
             self.data[_field][self.size] = new_item

{microecs-0.3.1 → microecs-0.3.3}/microecs/query_result.py

@@ -91,6 +91,9 @@ class _Field(np.lib.mixins.NDArrayOperatorsMixin):
     def __len__(self):
         return self.len
 
+    def __repr__(self):
+        return f"[Field] Shape: {self.shape} (across {len(self.parts)} pools)"
+
 class QueryResult:
     """A list of pools seen as a contiguous view. Fields (qr.position) implement array interface to look like numpy"""
     def __init__(self, pool_list: list[Pool], field_shapes: dict[str, Shape], field_dtypes: dict[str, np.dtype],

{microecs-0.3.1 → microecs-0.3.3}/microecs/world.py

@@ -77,6 +77,12 @@ class World:
         components = self.pool_to_components[pool]
         return entity, components
 
+    def set_entity_data(self, entity_id: EntityId, field_name: str, value: np.ndarray):
+        """Sets the value of one specific entity's field given an enttiy id. Non-vectorized operation, use rarely"""
+        assert entity_id in self._eid_to_pool_ix, f"Entity id {entity_id} not found. Perhaps you didn't world.update()"
+        pool, pool_ix = self._eid_to_pool_ix[entity_id]
+        pool.data[field_name][pool_ix] = value
+
     def add_component(self, entity_id: EntityId, component: ComponentType, **kwargs):
         """Adds a component to an entity given its id. Component data is sent to kwargs. Lazy; call update()"""
         assert entity_id in self.live_ids, f"Entity id: {entity_id} not in {self.live_ids=}"

{microecs-0.3.1 → microecs-0.3.3}/microecs.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: microecs
-Version: 0.3.1
+Version: 0.3.3
 Summary: MicroECS: Minimal Entity Component System (ECS) in python and numpy
 Home-page: https://gitlab.com/meehai/microecs
 License: MIT
@@ -9,6 +9,14 @@ Description-Content-Type: text/markdown
 License-File: LICENSE.TXT
 Requires-Dist: numpy>=2.2.0
 Requires-Dist: loggez>=0.8
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
 
 # MicroECS
 
@@ -16,24 +24,35 @@ Minimal (~300 LoC) Entity Component System in python and numpy. Examples also us
 
 Usage:
 
-- `pip install -r requirements.txt`
-- Sandbox: `python main.py`
-- Tests: `pytest test/`
+- Via pip: `pip install microecs`
+- From source code:
+```bash
+git clone https://gitlab.com/meehai/microecs      # clone the source code
+cd microecs                                       # go in the cloned directory
+python -m venv .venv && source .venv/bin/activate # make a virtual env, optional but useful
+python -m pip install -e .                        # install micro ecs in this virtual env
+python -m pytest test/                            # run the unit & integration tests to verify installation
+python examples/01-hello-world.py                 # run the basic hello world example (others in that dir)
+```
+
+Docs: [meehai.gitlab.io/microecs](https://meehai.gitlab.io/microecs/)
 
-There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `World`:
+## Relevant primitives: `Component`, `Pool`, `QueryResult`, `World`
+
+There are only 4 primitives (bottom up):
 
 - `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
 - `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
+- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's interface. For all intents and purposes it should feel like a `(N, ...)` view over all selected entities. To get a proper numpy array, use `qr.numpy()`. To iterate over each entity in a query result (e.g. rendering), use `for eid, position in zip(qr.entity_ids, qr.position): ...`.
 - `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
 
-Few relevant concepts:
+### Few relevant concepts:
 
 - `Pool` operates on array indices, while `World` operates on entity IDs (also integers). This allows seamless movement between pools while the high-level systems still working as intended.
 - All mutable operations on `World` are lazy. These are: `add_entity`, `remove_entity`, `add_component`, `remove_component`. They are added to a command buffer which is only executed when calling `world.update()`.
 - `Systems` are a convention, they are not part of this library. They can be defined at application level and act as hooks or callbacks. The `World` object doesn't need to know more than entities and components.
 
-Super simplified main loop structure:
+## Super simplified main loop structure
 
 ```python
 from typing import Callable
@@ -89,29 +108,67 @@ def main():
         rl.EndDrawing()
 ```
 
-## The field (`_Field`) numpy contract
+## Per-entity systems (e.g. rendering, foreign APIs)
+
+Not every system vectorizes. A renderer calls a draw function per primitive; same for any per-entity foreign API. That's a per-entity system inside an otherwise-vectorized app. The cheapest way to iterate through all the entities is by using `zip()` on the `QueryResult` object on the fields you need, e.g:
+
+```python
+qr = world.query(HasPosition, HasColor, HasRadius)
+for pos, color, radius in zip(qr.position, qr.color, qr.radius):     # one entity per step, fields aligned
+    rl.DrawCircle(int(pos[0]), int(pos[1]), float(radius[0]), color) # per-entity by necessity (no "draw all")
+```
+
+There are other ways to extract a single entity e.g. `qr.some_field[i]` or `e = world.get_entity(eid)`, but these are slower in the general case and should be avoided in expensive loops, like rendering, even if they are a bit more ergonomic. See benchmark below.
+
+<details>
+<summary> Microbenchmark: ECS vs OOP on a simple physics step </summary>
+
+Benchmark: we run the same physics step `pos += vel*dt` over N=100k entities split across 2 pools in various ways (ECS or OOP). All methods are verified to produce the identical result. Reproduce with `python test/manual/perf/bench_ecs_vs_oop.py` (it prints `{mode: avg_seconds_per_step}`).
+
+| pattern | ns/entity | vs OOP-scalar |
+|---|---:|---:|
+| `micro-ecs-pool-vectorized` — `for pool: pool.f[:] = pool.f + …` | 0.9 | **52× faster** |
+| `micro-ecs-vectorized` — `qr.f[:] = qr.f + …` (the `_Field`) | 1.8 | **27× faster** |
+| **`oop-scalar`** — `for o: o.x += o.vx*dt` (python floats) | 48 | 1× (baseline) |
+| `oop-numpy` — objects holding `(2,)` numpy arrays | 605 | 13× slower |
+| `micro-ecs-zip-rows` — `for p, v in zip(qr.pos, qr.vel)` | 744 | 15× slower |
+| `micro-ecs-pool-loop` — `for pool: for i: pool.f[i]` | 870 | 18× slower |
+| `micro-ecs-get-entity` — `world.get_entity(eid)` per entity | 1674 | 35× slower |
+| `micro-ecs-index` — `qr.f[i]` per entity (an `np.searchsorted` each) | 4573 | 95× slower |
+
+Three things to take from it:
+
+1. **Vectorized wins big.** Batched ops (`_Field` or per-pool) run at 1–2 ns/entity — **27–52×
+   faster** than the *fastest* OOP loop. Same for data-parallel branches: an `np.where` clamp or
+   bounce is ~34× faster than a per-entity `if`.
+2. **Per-entity loops are a cliff, not a tie.** Every per-entity microecs path is **15–95× slower**
+   than idiomatic float-based OOP — because microecs is numpy-backed, so a per-entity step pays
+   numpy's tiny-array overhead (`oop-numpy` shows the same ~13× tax). One unavoidable per-entity
+   pass (~750 ns/entity) costs ~500× a vectorized op (~1.5 ns) and will dominate the frame.
+3. **If you must loop, loop right.** `zip`-rows (15×) < pool-loop (18×) < `get_entity` (35×) <
+   `qr.f[i]` (95× — never in a hot loop; `qr.f[i] += …` also raises, you must bind the row first).
+
+**Rule of thumb:** keep systems vectorized and push branches into `np.where` / `np.clip`. If a
+workload is *irreducibly* per-entity (data-dependent control flow), plain python objects beat
+microecs ~15× — use them there. microecs is the right tool for **vectorizable** simulation.
+
+</details>
 
-`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
-numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
-result back. So the contract is exactly:
+## How much are `Pool` and `QueryResult` numpy-like and corner cases
 
-> Any op that treats rows independently behaves **identically to numpy** on the concatenated
-> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
+Given `qr=world.query(A, B)`, then `qr.position` returns a `_Field`: a view over the matching pools that behaves like one contiguous-like `(N, *e)`
+numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches result back.
 
-That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
-raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
-**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
-write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
-`test/unit/test_field_numpy_parity.py`.
+That covers elementwise math and ufuncs (e.g. `np.where`, `np.linalg.norm(..., axis=1)` etc.), broadcasting (every operand shape numpy accepts, and it raises on the ones numpy rejects). See `test/unit/test_field_numpy_parity.py` for a whole set of operations comparing both.
 
 Edge cases worth knowing:
 
-- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
+- **Not a full ndarray — these raise, never lie.** Entity-axis indexing beyond a single
   `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
   methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
   first with `qr.f.numpy()`.
 - **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
-  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
+  `axis=0` run within each pool and reset at pool boundaries — they do **not** see all entities
   at once, so they differ from numpy. They're allowed, but if you want a global result, do
   `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
   longer matches the pool's row count.

{microecs-0.3.1 → microecs-0.3.3}/setup.py

@@ -3,7 +3,7 @@ from pathlib import Path
 from setuptools import setup, find_packages
 
 NAME = "microecs"
-VERSION = "0.3.1"
+VERSION = "0.3.3"
 DESCRIPTION = "MicroECS: Minimal Entity Component System (ECS) in python and numpy"
 URL = "https://gitlab.com/meehai/microecs"
 

microecs-0.3.1/README.md

@@ -1,107 +0,0 @@
-# MicroECS
-
-Minimal (~300 LoC) Entity Component System in python and numpy. Examples also use raylib for rendering.
-
-Usage:
-
-- `pip install -r requirements.txt`
-- Sandbox: `python main.py`
-- Tests: `pytest test/`
-
-There are only 4 primitives (bottom up): `Component`, `Pool`, `QueryResult`, `World`:
-
-- `Component` is a simple python dataclass holding only data. All entries must be numpy arrays with metadata fields: shape and dtype. We support 5 dtypes only: `int32`, `float32`, `bool`, `str` and `object`. A component with no fields is a valid **tag** for querying (e.g. `class Frozen(Component): pass`).
-- `Pool` is a simple 'archetype' dynamic array, holding entities of the same type (same set of components). Usses `Components` metadata to construct contiguous arrays for all entities of the same type.
-- `QueryResult` is a list of pools that match some query on all the entities of the `World`. It acts as a contiguous numpy-like container that implements numpy's `__array_function__` and `__array_ufunc__`. For all intents and purposes it should feel like a `(N, ...)` view over all the entities. If you need a numpy array (not all ops are supported, for e.g. indexing on the first axis), use `QueryResult.numpy()` (see [the field numpy contract](#the-field-_field-numpy-contract) below). It also exposes `entity_ids`: a flat `(N,)` array of the matched entities' ids, in the same pool-by-pool order as the fields, so you can `zip(qr.entity_ids, qr.position)` or feed an id back to `world.get_entity` / `world.remove_entity`.
-- `World` is a manager of `Pools` and has an overview of all the entities in the scene. It also manages the migration of entities from one pool to the other. A `World` can also require extra metadata keys on every field via `World(extra_metadata=["serializable"])`, to enforce component-level behavior such as field serialization.
-
-Few relevant concepts:
-
-- `Pool` operates on array indices, while `World` operates on entity IDs (also integers). This allows seamless movement between pools while the high-level systems still working as intended.
-- All mutable operations on `World` are lazy. These are: `add_entity`, `remove_entity`, `add_component`, `remove_component`. They are added to a command buffer which is only executed when calling `world.update()`.
-- `Systems` are a convention, they are not part of this library. They can be defined at application level and act as hooks or callbacks. The `World` object doesn't need to know more than entities and components.
-
-Super simplified main loop structure:
-
-```python
-from typing import Callable
-import numpy as np
-import raylib as rl
-from microecs import World, Component
-
-# components
-class HasPosition(Component):
-    # 'shape' + 'dtype' are always required. For additional metadata (e.g. examples/03-serialization) use extra_metadata
-    position: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
-class HasVelocity(Component):
-    velocity: np.ndarray = field(metadata={"shape": (2, ), "dtype": "float32"})
-class HasColor(Component):
-    color: np.ndarray = field(metadata={"shape": (4, ), "dtype": "int32"})
-
-# systems: Note they are a convention!
-class RenderSystem:
-    def __call__(self, world: World):
-        query_result = world.query(HasPosition, HasColor, exclude=[]) # contiguous-like view of all entities matching
-        for position, color in zip(query_result.position, query_result.color): # draw each entity
-            DrawEntity(position, color)
-
-class MotionSystem:
-    def __call__(self, world: World):
-        qr = world.query(HasPosition, HasVelocity) # 'exclude' is optional
-        qr.position[:] = qr.position + qr.velocity * DT # writes back to all the underlying pools using numpy's rules
-        # Alternative for per-pool update. Less ergonomic, but maybe faster in extreme cases as it avoids the _Field obj
-        for pool in qr.pool_list:
-            pool.position[:] = pool.position + pool.velocity * DT
-
-def main():
-    render_system: list[Callable] = RenderSystem()
-    update_systems: list[Callable] = [MotionSystem()]
-
-    world = World(components=[HasPosition, HasColor, HasVelocity], extra_metadata=None) # extra_metadata is optional
-    for _ in range(n_objects):
-        # NOTE: world.{add/remove}_{entity/component} are lazy. They take effect after the first world.update() call.
-        world.add_entity(components=(HasPosition, HasVelocity, HasColor), # tuple of components (types)
-                         position=           np.array((x, y), "float32"), # data as kwargs
-                         color=         np.array("black", dtype="int32"),
-                         velocity=         np.array((vx, vy), "float32"))
-
-    while not rl.WindowShouldClose():
-        world.update() # must be called at each tick so the lazy methods are processed and entities are updated
-        # update stuff...
-        _ = [system(world=world) for system in update_systems]
-        # draw stuff, e.g. using raylib
-        rl.BeginDrawing()
-        rl.ClearBackground(rl.RAYWHITE)
-        rl.DrawFPS(rl.GetScreenWidth() - 100, 0)
-        render_system(world=world)
-        rl.EndDrawing()
-```
-
-## The field (`_Field`) numpy contract
-
-`qr.position` returns a `_Field`: a view over the matching pools that behaves like one `(N, *e)`
-numpy array (e.g. `(N, 2)` for a `(2,)` field). It applies each op **per pool** and stitches the
-result back. So the contract is exactly:
-
-> Any op that treats rows independently behaves **identically to numpy** on the concatenated
-> `(N, *e)` array — same values, same shape, and the same error on bad shapes.
-
-That covers elementwise math and ufuncs, broadcasting (every operand shape numpy accepts, and it
-raises on the ones numpy rejects), comparisons, `np.where` / `np.clip`, batched `np.matmul`,
-**feature-axis** reductions/indexing (`np.linalg.norm(qr.velocity, axis=1)`, `qr.pose[:, 0]`), and
-write-back broadcasting (`qr.position[:] = <scalar | (*e,) | (N, *e)>`). Pinned in
-`test/unit/test_field_numpy_parity.py`.
-
-Edge cases worth knowing:
-
-- **Not a full ndarray — these *raise*, never lie.** Entity-axis indexing beyond a single
-  `qr.f[i]` (`qr.f[:]`, `qr.f[2:4]`, `qr.f[mask]`, fancy), partial entity writes, and ndarray
-  methods/attrs (`.sum()`, `.mean()`, `.dtype`, `.ndim`, `.T`). Need any of these? Materialize
-  first with `qr.f.numpy()`.
-- **Axis-0 ops are per-pool, not global (footgun).** `np.sort` / `np.cumsum` / `np.sum` over
-  `axis=0` run *within each pool* and reset at pool boundaries — they do **not** see all entities
-  at once, so they differ from numpy. They're allowed, but if you want a global result, do
-  `qr.f.numpy()` first. A reduction that collapses the entity axis is rejected when its length no
-  longer matches the pool's row count.
-- **Operands must come from the same query.** Alignment is per-pool, not by flat index, so don't
-  mix a `_Field` from one `world.query(...)` into an op on another.