hyperdrone 0.1.4__tar.gz → 0.1.6__tar.gz

hyperdrone-0.1.6/PKG-INFO

@@ -0,0 +1,318 @@
+Metadata-Version: 2.5
+Name: hyperdrone
+Version: 0.1.6
+Summary: The fastest drone simulator and renderer (raytracer).
+Author-email: Jonas Eschmann <jonas.eschmann@gmail.com>
+License: MIT
+Requires-Python: >=3.9
+Requires-Dist: nanobind>=2.0
+Requires-Dist: numpy
+Provides-Extra: examples
+Requires-Dist: imageio; extra == 'examples'
+Requires-Dist: imageio-ffmpeg; extra == 'examples'
+Requires-Dist: matplotlib; extra == 'examples'
+Requires-Dist: scipy; extra == 'examples'
+Provides-Extra: gym
+Requires-Dist: gymnasium>=1.0; extra == 'gym'
+Description-Content-Type: text/markdown
+
+# hyperdrone
+
+Drone simulation stack for RLtools: raytracing renderer, L2F multirotor dynamics, and
+environment setup under one Python package.
+
+| Subpackage | What it is |
+|---|---|
+| `hyperdrone.render` | Raytracing renderer (OptiX / Metal / Vulkan / generic CPU). Self-contained — no dependency on drone dynamics. |
+| `hyperdrone.dynamics` | Vectorized, stateful L2F multirotor simulator (`Sim`); cpu and cuda variants. |
+| `hyperdrone.env` | The RL environment: the C++ `MultiEnvironment<hyperdrone::World>` driven through the exact rl_tools batch verbs. |
+| `hyperdrone.gym` | Optional Gymnasium `VectorEnv` adapter over `hyperdrone.env` (`pip install "hyperdrone[gym]"`). |
+| `hyperdrone.jit` | Shared compile-and-cache infrastructure both domain packages build on. |
+| `hyperdrone.cuda` | CUDA staging helpers (`upload` → DLPack tensor sets). |
+| `hyperdrone.conta` | Client for the sha1 content-addressed asset store — protocol twin of the C++ `include/conta/conta.h`, sharing one cache across both languages. |
+
+Native components are JIT-compiled per set of compile-time constants (resolution, camera
+count, drone count, ...) into a persistent cache; later uses load the cached library
+directly. `render` and `dynamics` are peers that never import each other — the coupling
+surface is a plain DLPack tensor of packed camera bases, device-resident on OptiX + CUDA.
+
+## Notebooks
+
+| Notebook | Open in Colab |
+|---|---|
+| [HyperDrone](hyperdrone/examples/HyperDrone.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/HyperDrone.ipynb#scrollTo=g-cDv8bdN7j9) |
+| [Benchmark](hyperdrone/examples/Benchmark.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/Benchmark.ipynb) |
+| [Custom](hyperdrone/examples/Custom.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/Custom.ipynb) |
+| [Orbiter](hyperdrone/examples/Orbiter.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/Orbiter.ipynb) |
+
+## Install
+
+From an rl-tools checkout (development):
+
+```bash
+.venv/bin/pip install -e interface/python/hyperdrone
+```
+
+From an sdist (self-contained — bundles the rl-tools headers and raytracing sources, no
+checkout needed):
+
+```bash
+pip install hyperdrone-<version>.tar.gz
+```
+
+Install the dependencies used by the packaged examples with:
+
+```bash
+pip install "hyperdrone[examples]"
+```
+
+Requirements: CMake >= 3.24, a C++17 compiler, and git for dependency downloads.
+GLB loading uses the Assimp 6.0.4 release archive with a pinned SHA-256, shared by render and env builds. Cached archives are reused offline and verified by CMake before extraction.
+CUDA + OptiX driver is required for the OptiX render backend and cuda dynamics,
+Vulkan dev + glslang for the VULKAN backend. The WEBGPU backend fetches a hash-pinned
+wgpu-native prebuilt at first build (no extra system packages). Nothing beyond the
+build toolchain for GENERIC + cpu.
+
+## Environment
+
+| Variable | Meaning |
+|---|---|
+| `HYPERDRONE_RENDER_BACKEND` | `OPTIX` \| `METAL` \| `VULKAN` \| `WEBGPU` \| `GENERIC` \| `AUTO` (default: Metal on macOS, OptiX elsewhere) |
+| `HYPERDRONE_DYNAMICS_DEVICE` | `CPU` \| `CUDA` \| `AUTO` (default: cuda when available) |
+| `HYPERDRONE_CACHE_DIR` | root for CMake build trees and downloaded build dependencies (default `~/.cache/hyperdrone`) |
+| `HYPERDRONE_RLTOOLS_ROOT` | rl-tools source root override (default: enclosing checkout, else the vendored tree) |
+| `HYPERDRONE_PROCTHOR_PATH` | local ProcTHOR example scene override (otherwise downloaded and cached) |
+| `HYPERDRONE_SKIP_BUILD` | skip the CMake staleness check when the artifacts already exist |
+| `HYPERDRONE_OFFLINE` | forbid network during builds (requires seeded/vendored dependencies) |
+| `HYPERDRONE_BUILD_JOBS` | parallel build jobs (default 5) |
+
+Each component and variant gets its own build tree (`render-optix-*`, `dynamics-cuda-*`,
+...); switching never invalidates another's cache. All configure/build steps run under a
+per-tree file lock, so many worker processes can share one cache safely. CMake
+FetchContent sources and native build state live in `<cache>/.dependencies`; the native
+build does not write generated files into an editable checkout or installed package.
+
+Whenever a renderer is successfully allocated, RLtools reports the selected backend on
+stderr, for example `#rl_tools::rendering::raytracing: backend=metal`. The
+`Renderer.backend` property provides the same lowercase name programmatically.
+
+## Rendering
+
+```python
+import math, numpy as np
+from hyperdrone import render
+from hyperdrone.examples.data import procthor_scene_path
+
+scene = render.load_scene(procthor_scene_path(), fidelity="high")
+
+renderer = render.Renderer(width=320, height=240, num_cameras=4, output="rgbd", fidelity="high")
+renderer.init(scene)
+
+camera = renderer.camera(position=(0, 0, 1.5), look_at=(1, 0, 1.5), fov=math.radians(80))
+renderer.set_cameras(np.repeat(camera[None], 4, axis=0))
+renderer.render()
+
+rgb   = renderer.frame()    # (4, 240, 320, 4) uint8
+depth = renderer.depth()    # (4, 240, 320) float32
+```
+
+The renderer operates in the L2F FLU frame: +X forward, +Y left, +Z up; GLB assets are
+swizzled at load time. Procedural scenes (`render.Scene()` + `render.Object` /
+`render.Mesh` / `render.SceneLight`), segmentation output, overlays (per-camera dynamic
+content via `AssetPool` + `spawn`/`attach`), motion blur, and anti-aliasing follow the
+same API as before under `hyperdrone.render.*`.
+
+Converted ProcTHOR scenes carry ambient `(0.6, 0.6, 0.6)` and a black background in
+`scenes[].extras.rl_tools.environment` (`mode: "solid"`, `ambient` and `background`
+are nonnegative linear RGB triples). Scene loading imports these settings; importing
+objects or assemblies leaves the parent environment alone. Ambient affects `high`
+and `veryhigh`, approximating indirect illumination without light bounces. Override
+imported settings with `scene.set_environment(...)` before `renderer.init(scene)`.
+
+`num_probes` sizes the collision-probe pass (`generate_probe_directions()`, `render("collision")`, `collisions()`; one probe per camera by default). `num_probes=0` compiles that pass out entirely, the counterpart of the probe-ray macro the C++ benchmark targets set: no probe buffers are allocated, the default `render()` target runs only the image channels, and the probe methods raise.
+
+### Zero-copy I/O (DLPack)
+
+```python
+renderer.frame()                  # snapshot copy (safe to keep)
+renderer.frame(copy=False)        # numpy view of the staging buffer (refreshed in place)
+rgba = torch.from_dlpack(renderer.frame_dlpack())
+# live uint8 (num_cameras, height, width, 4), zero-copy GPU tensor on OptiX
+packed = torch.from_dlpack(renderer.frame_raw_dlpack())
+# same memory as packed uint32 (num_cameras, height, width)
+```
+
+Inputs accept any DLPack producer. CUDA-resident camera input (OptiX):
+`set_cameras` dispatches on `__dlpack_device__`, so a CUDA tensor — a torch GPU tensor,
+a set pre-uploaded with `hyperdrone.cuda.upload`, or `Sim.camera_bases()` — is handed
+over device-to-device on the render stream, fully async; `stream=` takes the producer's
+cudaStream_t handle for event-ordered hand-off.
+
+## Dynamics
+
+```python
+from hyperdrone import dynamics
+
+sim = dynamics.Sim(num_drones=4096, model="x500_sim", device="cuda")
+sim.reset(seed=0)                     # deterministic; host-sampled, identical on cpu/cuda
+sim.step(actions)                     # (N, 4) float32 in [-1, 1], host or CUDA tensor
+sim.state["position"]                 # zero-copy DLPack views, CUDA-resident on cuda
+sim.state.numpy("position")           # host copy
+sim.observe()                         # (N, 18): position, rotation matrix, velocities
+sim.parameters["mass"] = masses       # per-drone runtime parameters (until the next reset)
+
+cameras = sim.camera_bases(fov=math.radians(100), aspect=renderer.aspect)
+renderer.set_cameras(cameras, stream=sim.stream)   # device-resident hand-off
+```
+
+`num_drones` and domain randomization are compile-time (the JIT key); the model preset
+(`crazyflie`, `x500_real`, `x500_sim`, `mrs`, ...), integration `dt`, and all physical
+parameters are runtime. Reward and termination live in the
+environment (`hyperdrone.env`), not the simulator. One process can use one dynamics
+variant (cpu or cuda).
+
+## RL environment
+
+```python
+import numpy as np
+from hyperdrone.env import EnvConfig, MultiEnvironment
+
+config = EnvConfig(num_environments=1, instances=1024, cam_width=32, cam_height=32, task="target_frame")
+env = MultiEnvironment("scenes/", config=config, seed=0)
+
+mask = np.ones(env.total_instances, dtype=np.uint8)
+env.reset(mask)                    # resample parameters + states where mask is set
+env.render(mask)                   # render the FPV cameras (mask marks fresh episodes)
+observations = env.observe()       # (total, observation_dim) float32
+env.step(actions)                  # (total, action_dim) float32 in [-1, 1]
+rewards, terminated = env.rewards(), env.terminated()
+critic_input = env.observe_privileged()
+env.rotate_scene()                 # deterministic scene rotation; reset all instances after
+env.observation_layout             # named blocks: which channels/values mean what
+```
+
+`reset(mask)` samples parameters and states immediately. `observe()` renders as needed,
+so explicit `render(mask)` calls are optional. The training loop owns episode counters and
+chooses when to reset; the Gymnasium adapter below supplies same-step autoreset and time limits.
+
+All environment semantics — reset, reward, termination, scene scheduling, observation
+composition — live on the C++ side (`rl_tools::rl::environments::hyperdrone::MultiEnvironment<World>`);
+the binding marshals tensors and nothing else, and a seeded rollout is pinned bit-exact
+against the C++ verbs by a golden test. The scenes argument is a directory of `.glb`
+scenes (sorted corpus) or a list of references — `.glb` paths, `conta:HASH` strings, or
+conta store manifest entries `{"description": ..., "hash": ...}` — used as the corpus in
+list order and partitioned across environments; conta references (scenes, `drone_asset=`,
+`gate_asset=`) resolve through `hyperdrone.conta` into the shared content-addressed
+cache, so a config pins the exact corpus on any machine. Configuration follows the C++
+extension ladder:
+`preset=` names the platform (`"crazyflie"`, `"x500_fpv"` — SELF_VISIBLE, pass a
+body/prop_* GLB via `drone_asset=` — and `"x500_fpv_imu"`, the same platform stepping at
+IMU rate), `task=` names the wrapper (`"target_frame"`, `"moving_gate"`,
+`"visual_inertial_localization"`), `n_agents=` enables multi-agent, and `EnvConfig(spec_header=...)` pins
+an arbitrary C++ specification (a header defining `hyperdrone_env_user::WORLD`, hashed
+into the JIT key). A spec header can go beyond constants to a full user-authored task
+wrapper — verb overloads registering and moving entities, compiled reward, termination:
+`hyperdrone/examples/Orbiter.ipynb` walks through one, and its contract is pinned by
+`tests/env/user_task_header.h`. The render backend for the environment follows
+`HYPERDRONE_ENV_BACKEND` (default: the render backend selection).
+
+Manual composition of `Sim` and `Renderer` (no MDP — rendering research, data
+generation) remains first-class: see `python -m hyperdrone.examples.drone_flythrough`
+for the wiring. `FreeSpaceSampler` lives in `hyperdrone.render` — it rides the
+renderer's collision probes and is deterministic given a seed.
+
+### Visual-inertial localization
+
+`task="visual_inertial_localization"` on `preset="x500_fpv_imu"` is the localization
+benchmark: fixed-length synchronized episodes stepping at IMU rate (`env.dt`), a camera
+frame every `env.frame_stride` steps, and the IMU sample of each step from
+`env.observe_imu()` (named by `env.observation_layout_imu`: accelerometer, gyroscope,
+frame age, new-frame flag). The task is autonomous: the RAPTOR autopilot built into it
+flies a waypoint route (its checkpoint is pinned by content hash and fetched through
+conta on first use), so `action_dim` is 0 and `step()` takes no actions. The privileged
+observation carries the current `waypoint` next to the dynamics state; the drone's own
+body is not rendered, matching the C++ benchmark harness. Resets must be all-or-none per
+environment.
+
+```python
+config = EnvConfig(instances=2, cam_width=160, cam_height=120, preset="x500_fpv_imu", task="visual_inertial_localization")
+env = MultiEnvironment(scenes, config=config, seed=0)
+env.reset(); env.render(np.ones(env.total_instances, dtype=np.uint8))
+for step in range(env.episode_step_limit):
+    if step > 0: env.render()
+    if step % env.frame_stride == 0:
+        frames = env.frames()          # the estimator's camera frame
+    env.step()
+    imu = env.observe_imu()            # the estimator's IMU sample for this step
+    truth = env.observe_privileged()   # position / rotation matrix / velocities / waypoint
+```
+
+`python -m hyperdrone.examples.visual_inertial_localization` runs an episode and scores
+IMU dead reckoning against the ground truth, the same protocol as the C++ demo.
+
+### Gymnasium
+
+```python
+from hyperdrone.gym import VectorEnv   # pip install "hyperdrone[gym]"
+env = VectorEnv("scenes/", config=EnvConfig(instances=1024), seed=0)
+observations, infos = env.reset()
+observations, rewards, terminations, truncations, infos = env.step(actions)
+```
+
+The adapter resets instances that terminate or reach `episode_step_limit` within the same
+step. Returned observations start the new episode for those instances; no final observation
+is exposed. `truncations` flags time limits that are not terminations. The core packages
+never import gymnasium.
+
+End-to-end example: `python -m hyperdrone.examples.drone_flythrough`; renderer benchmark:
+`python -m hyperdrone.examples.benchmark` (flag-compatible with the C++ benchmark
+counterpart).
+
+## Publishing
+
+From the repository root, build and upload a release with:
+
+```bash
+interface/python/hyperdrone/scripts/publish.sh
+```
+
+Set the release version in `pyproject.toml` and `hyperdrone/__init__.py` first. The
+script uses the repository's `.venv` (creating it if needed), installs the build and
+upload tools, builds a bundled source archive and a `py3-none-any` wheel from that
+archive, checks both with Twine, and uploads both to PyPI. Authenticate through your
+existing Twine configuration (`~/.pypirc`, keyring, or `TWINE_PASSWORD`), or enter a
+PyPI API token when prompted.
+
+Use `interface/python/hyperdrone/scripts/publish.sh --build-only` to build and check
+without uploading. Each run keeps its two artifacts in a fresh
+`interface/python/hyperdrone/dist/release.*` directory, so older builds are never
+included in an upload. Native components remain JIT-compiled on first use.
+
+## Tests
+
+```bash
+HYPERDRONE_RENDER_BACKEND=GENERIC .venv/bin/python -m pytest interface/python/hyperdrone/tests -v
+```
+
+`tests/jit` exercises the build infrastructure against a toy component in seconds;
+`tests/test_architecture.py` enforces the import DAG (render and dynamics never see each
+other); `scripts/test_sdist.sh` is the self-containment rot guard (build sdist → clean
+venv → vendored-tree render).
+
+For labelled ProcTHOR rendering, preserve GLB root instances and select the output meaning:
+
+```python
+scene = render.load_scene(procthor_scene_path(), rgb=False, preserve_instances=True)
+renderer = render.Renderer(width=512, height=512, output="segmentation",
+                           semantic_segmentation=True)  # False selects instance IDs
+renderer.init(scene)
+# Set cameras and render as usual.
+labels = renderer.segmentation()  # uint32; renderer.segmentation_mode identifies the mode
+classes = scene.segmentation_classes  # {class_id: name}, taxonomy: scene.segmentation_taxonomy
+```
+
+Semantic IDs come from AI2THOR-Hab's taxonomy: unknown is `0` (including Ceiling, which has no dataset category); background is `0xFFFFFFFF` in both modes. Instance IDs identify scene instances: use `scene.instance_object(id)` then `scene.object_name(object_index)` for static hits. Overlay instance IDs use the renderer's global slot layout. Semantic IDs must never be used as object indices. Loading remains welded by default for RGB; semantic rendering rejects a welded object containing multiple classes. Assemblies and asset-pool copies retain labels and the taxonomy. Composing conflicting taxonomies requires an explicit remapping before semantic rendering.
+
+Assign labels before `Renderer.init()`: `object.segmentation_class = id`, `scene.set_object_segmentation_class(index, id)`, `assembly.set_segmentation_class(index, id)`, or `pool.set_object_segmentation_class(asset, object_index, id)` retains the current taxonomy and explicitly makes that object's class valid, including welded objects. Assignment rejects IDs absent from that taxonomy. `object.assign_segmentation_class(id, taxonomy="custom", classes={0: "Unknown", 7: "Furniture"})` replaces the taxonomy; omitting taxonomy/classes selects unnamed numeric IDs.
+
+Objects, scenes, assemblies and pools expose `remap_segmentation_classes(mapping, taxonomy="custom", classes={...})`; pools also expose `remap_asset_segmentation_classes(asset, mapping, ...)`. Omitting taxonomy/classes selects unnamed numeric IDs. Mappings must cover every used class, and validation failures leave labels unchanged. Remapping preserves valid class boundaries and rejects mixed-class welded objects; explicitly assign one class first if collapsing their labels is intentional. Remap conflicting sources separately before composition. The C++ equivalents are device-first `assign_segmentation_class` and `remap_segmentation_classes` in `rendering/segmentation_cpu.h`, with a shared immutable target taxonomy (or `nullptr` for unnamed IDs).

{hyperdrone-0.1.4 → hyperdrone-0.1.6}/README.md

@@ -18,6 +18,15 @@ count, drone count, ...) into a persistent cache; later uses load the cached lib
 directly. `render` and `dynamics` are peers that never import each other — the coupling
 surface is a plain DLPack tensor of packed camera bases, device-resident on OptiX + CUDA.
 
+## Notebooks
+
+| Notebook | Open in Colab |
+|---|---|
+| [HyperDrone](hyperdrone/examples/HyperDrone.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/HyperDrone.ipynb#scrollTo=g-cDv8bdN7j9) |
+| [Benchmark](hyperdrone/examples/Benchmark.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/Benchmark.ipynb) |
+| [Custom](hyperdrone/examples/Custom.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/Custom.ipynb) |
+| [Orbiter](hyperdrone/examples/Orbiter.ipynb) | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rl-tools/rl-tools-internal/blob/render/interface/python/hyperdrone/hyperdrone/examples/Orbiter.ipynb) |
+
 ## Install
 
 From an rl-tools checkout (development):
@@ -39,11 +48,12 @@ Install the dependencies used by the packaged examples with:
 pip install "hyperdrone[examples]"
 ```
 
-Requirements: CMake >= 3.24, a C++17 compiler, and per feature: assimp (system package)
-for GLB loading, CUDA + OptiX driver for the OptiX render backend and cuda dynamics,
+Requirements: CMake >= 3.24, a C++17 compiler, and git for dependency downloads.
+GLB loading uses the Assimp 6.0.4 release archive with a pinned SHA-256, shared by render and env builds. Cached archives are reused offline and verified by CMake before extraction.
+CUDA + OptiX driver is required for the OptiX render backend and cuda dynamics,
 Vulkan dev + glslang for the VULKAN backend. The WEBGPU backend fetches a hash-pinned
 wgpu-native prebuilt at first build (no extra system packages). Nothing beyond the
-compiler for GENERIC + cpu.
+build toolchain for GENERIC + cpu.
 
 ## Environment
 
@@ -94,6 +104,13 @@ swizzled at load time. Procedural scenes (`render.Scene()` + `render.Object` /
 content via `AssetPool` + `spawn`/`attach`), motion blur, and anti-aliasing follow the
 same API as before under `hyperdrone.render.*`.
 
+Converted ProcTHOR scenes carry ambient `(0.6, 0.6, 0.6)` and a black background in
+`scenes[].extras.rl_tools.environment` (`mode: "solid"`, `ambient` and `background`
+are nonnegative linear RGB triples). Scene loading imports these settings; importing
+objects or assemblies leaves the parent environment alone. Ambient affects `high`
+and `veryhigh`, approximating indirect illumination without light bounces. Override
+imported settings with `scene.set_environment(...)` before `renderer.init(scene)`.
+
 `num_probes` sizes the collision-probe pass (`generate_probe_directions()`, `render("collision")`, `collisions()`; one probe per camera by default). `num_probes=0` compiles that pass out entirely, the counterpart of the probe-ray macro the C++ benchmark targets set: no probe buffers are allocated, the default `render()` target runs only the image channels, and the probe methods raise.
 
 ### Zero-copy I/O (DLPack)
@@ -263,3 +280,21 @@ HYPERDRONE_RENDER_BACKEND=GENERIC .venv/bin/python -m pytest interface/python/hy
 `tests/test_architecture.py` enforces the import DAG (render and dynamics never see each
 other); `scripts/test_sdist.sh` is the self-containment rot guard (build sdist → clean
 venv → vendored-tree render).
+
+For labelled ProcTHOR rendering, preserve GLB root instances and select the output meaning:
+
+```python
+scene = render.load_scene(procthor_scene_path(), rgb=False, preserve_instances=True)
+renderer = render.Renderer(width=512, height=512, output="segmentation",
+                           semantic_segmentation=True)  # False selects instance IDs
+renderer.init(scene)
+# Set cameras and render as usual.
+labels = renderer.segmentation()  # uint32; renderer.segmentation_mode identifies the mode
+classes = scene.segmentation_classes  # {class_id: name}, taxonomy: scene.segmentation_taxonomy
+```
+
+Semantic IDs come from AI2THOR-Hab's taxonomy: unknown is `0` (including Ceiling, which has no dataset category); background is `0xFFFFFFFF` in both modes. Instance IDs identify scene instances: use `scene.instance_object(id)` then `scene.object_name(object_index)` for static hits. Overlay instance IDs use the renderer's global slot layout. Semantic IDs must never be used as object indices. Loading remains welded by default for RGB; semantic rendering rejects a welded object containing multiple classes. Assemblies and asset-pool copies retain labels and the taxonomy. Composing conflicting taxonomies requires an explicit remapping before semantic rendering.
+
+Assign labels before `Renderer.init()`: `object.segmentation_class = id`, `scene.set_object_segmentation_class(index, id)`, `assembly.set_segmentation_class(index, id)`, or `pool.set_object_segmentation_class(asset, object_index, id)` retains the current taxonomy and explicitly makes that object's class valid, including welded objects. Assignment rejects IDs absent from that taxonomy. `object.assign_segmentation_class(id, taxonomy="custom", classes={0: "Unknown", 7: "Furniture"})` replaces the taxonomy; omitting taxonomy/classes selects unnamed numeric IDs.
+
+Objects, scenes, assemblies and pools expose `remap_segmentation_classes(mapping, taxonomy="custom", classes={...})`; pools also expose `remap_asset_segmentation_classes(asset, mapping, ...)`. Omitting taxonomy/classes selects unnamed numeric IDs. Mappings must cover every used class, and validation failures leave labels unchanged. Remapping preserves valid class boundaries and rejects mixed-class welded objects; explicitly assign one class first if collapsing their labels is intentional. Remap conflicting sources separately before composition. The C++ equivalents are device-first `assign_segmentation_class` and `remap_segmentation_classes` in `rendering/segmentation_cpu.h`, with a shared immutable target taxonomy (or `nullptr` for unnamed IDs).

{hyperdrone-0.1.4 → hyperdrone-0.1.6}/hyperdrone/__init__.py

@@ -16,7 +16,7 @@ hyperdrone (or any subpackage) is cheap; the first build happens on first use.
 """
 import importlib
 
-__version__ = "0.1.4"
+__version__ = "0.1.5"
 
 _SUBPACKAGES = ("render", "dynamics", "env", "gym", "jit", "cuda")
 

{hyperdrone-0.1.4 → hyperdrone-0.1.6}/hyperdrone/_native/env/CMakeLists.txt

@@ -28,7 +28,7 @@ if(HYPERDRONE_VARIANT STREQUAL "VULKAN")
     find_package(Vulkan REQUIRED COMPONENTS glslangValidator)
 endif()
 
-find_package(assimp REQUIRED)
+unset(FETCHCONTENT_SOURCE_DIR_ASSIMP CACHE)
 add_subdirectory(${RLTOOLS_ROOT} rl_tools EXCLUDE_FROM_ALL)
 set(RL_TOOLS_RENDERING_RAYTRACING_BACKEND_RESOLVED ${HYPERDRONE_VARIANT})
 add_subdirectory(${RLTOOLS_ROOT}/src/rendering/raytracing/backends rl_tools_rt_backends EXCLUDE_FROM_ALL)

{hyperdrone-0.1.4 → hyperdrone-0.1.6}/hyperdrone/_native/render/CMakeLists.txt

@@ -35,9 +35,7 @@ if(HYPERDRONE_VARIANT STREQUAL "VULKAN")
     find_package(Vulkan REQUIRED COMPONENTS glslangValidator)
 endif()
 
-# imported targets are directory-scoped; the backends subtree below needs assimp visible
-# from this scope, not just inside the rl_tools subtree
-find_package(assimp REQUIRED)
+unset(FETCHCONTENT_SOURCE_DIR_ASSIMP CACHE)
 add_subdirectory(${RLTOOLS_ROOT} rl_tools EXCLUDE_FROM_ALL)
 # the resolved-backend variable is directory-scoped inside the rl_tools subtree, so the
 # backends dispatch needs it set again here

{hyperdrone-0.1.4 → hyperdrone-0.1.6}/hyperdrone/_native/render/scene_bindings.cpp

@@ -3,6 +3,7 @@
 #include <nanobind/stl/string.h>
 #include <nanobind/stl/array.h>
 #include <nanobind/stl/vector.h>
+#include <nanobind/stl/map.h>
 
 #include <rl_tools/operations/cpu_mux.h>
 #include <rl_tools/rendering/raytracing/operations_cpu_common.h>
@@ -11,6 +12,8 @@
 #include "bindings.h"
 
 #include <cmath>
+#include <cstdint>
+#include <cstring>
 #include <stdexcept>
 #include <string>
 
@@ -27,22 +30,50 @@ using hyperdrone::make_owned_array;
 
 using DEVICE = rlt::devices::DEVICE_FACTORY<>;
 static DEVICE g_device;
+using rlt::segmentation_taxonomy;
+using rlt::assign_segmentation_class;
+using rlt::remap_segmentation_classes;
 
 // the Python-facing "Scene" is a Bundle: the composed scene plus its loader/computed
 // metadata (bounds, max ray length), which the renderer consumes at init
 using SceneBundle = rlt::rendering::Bundle<float>;
 
-template <typename TARGET>
-static bool load_dispatch(TARGET& target, const std::string& path, int fidelity, bool rgb){
+template <typename TARGET, typename... MODES>
+static bool load_dispatch(TARGET& target, const std::string& path, int fidelity, bool rgb, const MODES&... modes){
+    using rlt::load;
     switch(fidelity){
-        case 0: return rgb ? rlt::load<rrt::Low, true>(g_device, target, path) : rlt::load<rrt::Low, false>(g_device, target, path);
-        case 1: return rgb ? rlt::load<rrt::Medium, true>(g_device, target, path) : rlt::load<rrt::Medium, false>(g_device, target, path);
-        case 2: return rgb ? rlt::load<rrt::High, true>(g_device, target, path) : rlt::load<rrt::High, false>(g_device, target, path);
-        case 3: return rgb ? rlt::load<rrt::VeryHigh, true>(g_device, target, path) : rlt::load<rrt::VeryHigh, false>(g_device, target, path);
+        case 0: return rgb ? load<rrt::Low, true>(g_device, target, path, modes...) : load<rrt::Low, false>(g_device, target, path, modes...);
+        case 1: return rgb ? load<rrt::Medium, true>(g_device, target, path, modes...) : load<rrt::Medium, false>(g_device, target, path, modes...);
+        case 2: return rgb ? load<rrt::High, true>(g_device, target, path, modes...) : load<rrt::High, false>(g_device, target, path, modes...);
+        case 3: return rgb ? load<rrt::VeryHigh, true>(g_device, target, path, modes...) : load<rrt::VeryHigh, false>(g_device, target, path, modes...);
     }
     throw std::invalid_argument("hyperdrone: fidelity must be in [0, 3]");
 }
 
+template <typename TARGET>
+static std::string taxonomy_name(const TARGET& target){
+    const auto taxonomy = segmentation_taxonomy(g_device, target);
+    return taxonomy ? taxonomy->name : std::string{};
+}
+
+template <typename TARGET>
+static std::map<uint32_t, std::string> class_names(const TARGET& target){
+    const auto taxonomy = segmentation_taxonomy(g_device, target);
+    return taxonomy ? taxonomy->classes : std::map<uint32_t, std::string>{};
+}
+
+static std::shared_ptr<const rlt::rendering::SegmentationTaxonomy> make_taxonomy(const std::string& name, const std::map<uint32_t, std::string>& classes){
+    if(name.empty() && classes.empty()){
+        return nullptr;
+    }
+    return std::make_shared<const rlt::rendering::SegmentationTaxonomy>(rlt::rendering::SegmentationTaxonomy{name, classes});
+}
+
+template <typename TARGET>
+static void remap_labels(TARGET& target, const std::map<uint32_t, uint32_t>& mapping, const std::string& taxonomy, const std::map<uint32_t, std::string>& classes){
+    remap_segmentation_classes(g_device, target, mapping, make_taxonomy(taxonomy, classes));
+}
+
 static rrt::Mesh make_mesh(FloatArray vertices, nb::ndarray<const int32_t, nb::c_contig, nb::device::cpu> indices,
                            Vec3 color, nb::object normals, nb::object tex_coords,
                            float metallic, float roughness, Vec3 emissive){
@@ -149,12 +180,22 @@ void register_scene_bindings(nb::module_& m){
 
     nb::class_<rrt::Object>(m, "Object")
         .def("__init__", [](rrt::Object* object, const std::string& name, uint32_t segmentation_class){
-            new (object) rrt::Object();
-            object->name = name;
-            object->segmentation_class = segmentation_class;
+            rrt::Object value;
+            value.name = name;
+            assign_segmentation_class(g_device, value, segmentation_class);
+            new (object) rrt::Object(std::move(value));
         }, nb::arg("name") = "", nb::arg("segmentation_class") = 0)
         .def_rw("name", &rrt::Object::name)
-        .def_rw("segmentation_class", &rrt::Object::segmentation_class)
+        .def_prop_rw("segmentation_class", [](const rrt::Object& object){ return object.segmentation_class; },
+                     [](rrt::Object& object, uint32_t id){ assign_segmentation_class(g_device, object, id); })
+        .def("assign_segmentation_class", [](rrt::Object& object, uint32_t id, const std::string& taxonomy, const std::map<uint32_t, std::string>& classes){
+            assign_segmentation_class(g_device, object, id, make_taxonomy(taxonomy, classes));
+        }, nb::arg("id"), nb::arg("taxonomy") = "", nb::arg("classes") = std::map<uint32_t, std::string>{},
+           "Assign one class and its taxonomy before init; omitted taxonomy selects unnamed numeric classes.")
+        .def("remap_segmentation_classes", &remap_labels<rrt::Object>, nb::arg("mapping"), nb::arg("taxonomy") = "", nb::arg("classes") = std::map<uint32_t, std::string>{})
+        .def_prop_ro("segmentation_class_valid", [](const rrt::Object& object){ return object.segmentation_class_valid; })
+        .def_prop_ro("segmentation_taxonomy", [](const rrt::Object& object){ return object.segmentation_taxonomy ? object.segmentation_taxonomy->name : std::string{}; })
+        .def_prop_ro("segmentation_classes", [](const rrt::Object& object){ return object.segmentation_taxonomy ? object.segmentation_taxonomy->classes : std::map<uint32_t, std::string>{}; })
         .def("add_mesh", [](rrt::Object& object, const rrt::Mesh& mesh){ object.meshes.push_back(mesh); })
         .def("add_light", [](rrt::Object& object, const rrt::SceneLight& light){ object.lights.push_back(light); })
         .def_prop_ro("num_meshes", [](const rrt::Object& object){ return object.meshes.size(); });
@@ -162,6 +203,8 @@ void register_scene_bindings(nb::module_& m){
     nb::class_<rrt::ObjectAssembly>(m, "ObjectAssembly")
         .def(nb::init<>())
         .def_prop_ro("num_objects", [](const rrt::ObjectAssembly& assembly){ return assembly.objects.size(); })
+        .def_prop_ro("segmentation_taxonomy", &taxonomy_name<rrt::ObjectAssembly>)
+        .def_prop_ro("segmentation_classes", &class_names<rrt::ObjectAssembly>)
         .def_prop_ro("num_parts", [](const rrt::ObjectAssembly& assembly){ return assembly.parts.size(); })
         .def("object_name", [](const rrt::ObjectAssembly& assembly, size_t index){ return assembly.objects.at(index).name; })
         .def("object_names", [](const rrt::ObjectAssembly& assembly){
@@ -171,8 +214,9 @@ void register_scene_bindings(nb::module_& m){
         })
         .def("segmentation_class", [](const rrt::ObjectAssembly& assembly, size_t index){ return assembly.objects.at(index).segmentation_class; })
         .def("set_segmentation_class", [](rrt::ObjectAssembly& assembly, size_t index, uint32_t segmentation_class){
-            assembly.objects.at(index).segmentation_class = segmentation_class;
+            assign_segmentation_class(g_device, assembly, index, segmentation_class);
         })
+        .def("remap_segmentation_classes", &remap_labels<rrt::ObjectAssembly>, nb::arg("mapping"), nb::arg("taxonomy") = "", nb::arg("classes") = std::map<uint32_t, std::string>{})
         .def("part", [](const rrt::ObjectAssembly& assembly, size_t index){
             const auto& part = assembly.parts.at(index);
             return nb::make_tuple(part.object, make_owned_array(part.transform, {3, 4}));
@@ -183,20 +227,34 @@ void register_scene_bindings(nb::module_& m){
         .def("add_assembly", [](rrt::AssetPool& pool, const rrt::ObjectAssembly& assembly){ return rlt::add(g_device, pool, assembly).index; })
         .def("add_object", [](rrt::AssetPool& pool, const rrt::Object& object){ return rlt::add(g_device, pool, object).index; })
         .def("add_mesh", [](rrt::AssetPool& pool, const rrt::Mesh& mesh){ return rlt::add(g_device, pool, mesh).index; })
-        .def_prop_ro("num_assets", [](const rrt::AssetPool& pool){ return pool.assemblies.size(); });
+        .def_prop_ro("num_assets", [](const rrt::AssetPool& pool){ return pool.assemblies.size(); })
+        .def_prop_ro("segmentation_taxonomy", &taxonomy_name<rrt::AssetPool>)
+        .def_prop_ro("segmentation_classes", &class_names<rrt::AssetPool>)
+        .def("object_segmentation_class", [](const rrt::AssetPool& pool, size_t asset, size_t object){
+            return pool.assemblies.at(asset).objects.at(object).segmentation_class;
+        })
+        .def("set_object_segmentation_class", [](rrt::AssetPool& pool, size_t asset, size_t object, uint32_t id){
+            assign_segmentation_class(g_device, pool, asset, object, id);
+        })
+        .def("remap_segmentation_classes", &remap_labels<rrt::AssetPool>, nb::arg("mapping"), nb::arg("taxonomy") = "", nb::arg("classes") = std::map<uint32_t, std::string>{})
+        .def("remap_asset_segmentation_classes", [](rrt::AssetPool& pool, size_t asset, const std::map<uint32_t, uint32_t>& mapping, const std::string& taxonomy, const std::map<uint32_t, std::string>& classes){
+            remap_labels(pool.assemblies.at(asset), mapping, taxonomy, classes);
+        }, nb::arg("asset"), nb::arg("mapping"), nb::arg("taxonomy") = "", nb::arg("classes") = std::map<uint32_t, std::string>{});
 
     nb::class_<SceneBundle>(m, "Scene")
         .def(nb::init<>())
-        .def("load", [](SceneBundle& bundle, const std::string& path, int fidelity, bool rgb){
+        .def("load", [](SceneBundle& bundle, const std::string& path, int fidelity, bool rgb, bool preserve_instances){
             bool success;
             {
                 nb::gil_scoped_release release;
-                success = load_dispatch(bundle, path, fidelity, rgb);
+                success = preserve_instances
+                    ? load_dispatch(bundle, path, fidelity, rgb, rlt::Mode<rlt::mode::rendering::SceneRoots<>>{})
+                    : load_dispatch(bundle, path, fidelity, rgb);
             }
             if(!success){
                 throw std::runtime_error("hyperdrone: failed to load scene from " + path);
             }
-        }, nb::arg("path"), nb::arg("fidelity") = 2, nb::arg("rgb") = true)
+        }, nb::arg("path"), nb::arg("fidelity") = 2, nb::arg("rgb") = true, nb::arg("preserve_instances") = false)
         .def("add_object", [](SceneBundle& bundle, const rrt::Object& object, nb::object transform){
             if(transform.is_none()){
                 return rlt::add(g_device, bundle.scene, object);
@@ -219,14 +277,41 @@ void register_scene_bindings(nb::module_& m){
             return nb::make_tuple(placement.first_instance, placement.num_instances);
         }, nb::arg("assembly"), nb::arg("transform") = nb::none())
         .def("add_light", [](SceneBundle& bundle, const rrt::SceneLight& light){ bundle.scene.lights.push_back(light); })
+        .def("set_environment", [](SceneBundle& bundle, Vec3 ambient, Vec3 background){
+            const auto valid_component = [](float value){
+                // Bit inspection keeps NaN/Inf validation effective under -ffast-math.
+                uint32_t bits;
+                static_assert(sizeof(bits) == sizeof(value));
+                std::memcpy(&bits, &value, sizeof(bits));
+                volatile uint32_t representation = bits;
+                return (representation & 0x7f800000u) != 0x7f800000u && value >= 0.0f;
+            };
+            for(int component = 0; component < 3; component++){
+                if(!valid_component(ambient[component]) || !valid_component(background[component])){
+                    throw std::invalid_argument("hyperdrone: environment colors must be finite and nonnegative");
+                }
+            }
+            auto& environment = bundle.scene.environment;
+            environment.mode = rrt::Scene::Environment::Mode::SOLID;
+            for(int component = 0; component < 3; component++){
+                environment.ambient[component] = ambient[component];
+                environment.horizon[component] = background[component];
+            }
+        }, nb::arg("ambient") = Vec3{0.1f, 0.1f, 0.1f}, nb::arg("background") = Vec3{0.0f, 0.0f, 0.0f},
+           "Set linear RGB ambient illumination (high/veryhigh) and a solid background before Renderer.init().")
         .def_prop_ro("num_objects", [](const SceneBundle& bundle){ return bundle.scene.objects.size(); })
         .def_prop_ro("num_instances", [](const SceneBundle& bundle){ return bundle.scene.instances.size(); })
         .def_prop_ro("num_lights", [](const SceneBundle& bundle){ return bundle.scene.lights.size(); })
+        .def_prop_ro("segmentation_taxonomy", [](const SceneBundle& bundle){ return taxonomy_name(bundle.scene); })
+        .def_prop_ro("segmentation_classes", [](const SceneBundle& bundle){ return class_names(bundle.scene); })
         .def("object_name", [](const SceneBundle& bundle, size_t index){ return bundle.scene.objects.at(index).name; })
         .def("object_segmentation_class", [](const SceneBundle& bundle, size_t index){ return bundle.scene.objects.at(index).segmentation_class; })
         .def("set_object_segmentation_class", [](SceneBundle& bundle, size_t index, uint32_t segmentation_class){
-            bundle.scene.objects.at(index).segmentation_class = segmentation_class;
+            assign_segmentation_class(g_device, bundle.scene, index, segmentation_class);
         })
+        .def("remap_segmentation_classes", [](SceneBundle& bundle, const std::map<uint32_t, uint32_t>& mapping, const std::string& taxonomy, const std::map<uint32_t, std::string>& classes){
+            remap_labels(bundle.scene, mapping, taxonomy, classes);
+        }, nb::arg("mapping"), nb::arg("taxonomy") = "", nb::arg("classes") = std::map<uint32_t, std::string>{})
         .def("instance_object", [](const SceneBundle& bundle, size_t index){ return bundle.scene.instances.at(index).object; })
         .def("instance_objects", [](const SceneBundle& bundle){
             std::vector<size_t> objects;