euler-preprocess 3.15.0__tar.gz → 3.17.0__tar.gz

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-preprocess
-Version: 3.15.0
+Version: 3.17.0
 Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
 Author-email: Daniel Rothenpieler <rothenpielerdaniel@gmail.com>
 License-Expression: MIT
@@ -9,7 +9,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy
 Requires-Dist: Pillow
-Requires-Dist: euler-loading>=2.24.0
+Requires-Dist: euler-loading>=2.29.0
 Requires-Dist: ds-crawler>=2.11.0
 Requires-Dist: euler-dataset-contract>=0.5.0
 Provides-Extra: gpu
@@ -94,9 +94,7 @@ loader.
   "modalities": {
     "rgb": {"path": "/path/to/rgb", "split": "train"},
     "depth": "/path/to/depth",
-    "semantic_segmentation": "/path/to/classSegmentation"
-  },
-  "hierarchical_modalities": {
+    "semantic_segmentation": "/path/to/classSegmentation",
     "intrinsics": {"path": "/path/to/intrinsics"}
   }
 }
@@ -109,17 +107,36 @@ loader.
 | `output_slot` | Optional slot selector when `pipeline.output_targets` has several entries. Defaults to `rgb` for `fog` and `depth` for `sky-depth` / `radial`. |
 | `sample` | Optional 0-based dataset index. Only `dataset[sample]` is transformed — useful for small benchmark slices of large datasets. |
 | `samples` | Optional multi-sample selector: a list of indices (`[0, 10, 20]`) or a slice object (`{"start": 0, "stop": 1000, "step": 2, "count": 100}`). `stop` is exclusive, `count` caps the result. Mutually exclusive with `sample`. |
-| `modalities` | Modalities that participate in sample-ID intersection. Each value is a path string or an object with `path` and optional `split`. |
-| `hierarchical_modalities` | Per-scene data (e.g. intrinsics), same format. Loaded once per scene and cached. |
+| `modalities` | All input modalities. Each value is a path string or an object with `path` and optional `split`. Whether a modality is per-sample or hierarchical is inferred (see below). |
+| `hierarchical_modalities` | Legacy, optional. Same format; entries are pooled with `modalities` and resolved the same way. |
 | `pipeline` | Optional runtime routing block compatible with `euler-inference`. |
 
 **Required modalities per transform:**
 
-| Transform | `modalities` | `hierarchical_modalities` |
+| Transform | Required | Optional |
 |---|---|---|
-| `fog` | `rgb`, `depth`, `semantic_segmentation` | `intrinsics` when available; used for radial depth conversion and camera-profile optics |
+| `fog` | `rgb`, `depth`, `semantic_segmentation` | `intrinsics`; used for radial depth conversion and camera-profile optics |
 | `sky-depth` | `depth`, `semantic_segmentation` | — |
-| `radial` | `depth` | `intrinsics` |
+| `radial` | `depth`, `intrinsics` | — |
+
+`fog` and `sky-depth` need `semantic_segmentation` as a boolean sky mask. They load
+it with the `sky_mask` function of the euler-loading module that the index's
+dataset head names, so a `class_segmentation` index works as is: the sky is the
+head's `skyclass` (Synscapes `[23, 23, 23]`, Real Drive Sim `[29, 0, 0]`, VKITTI2
+`[90, 200, 255]`), and the log names the loader and the sky definition in use.
+Set `"function"` in the modality entry to use another loader function, or
+`"function": null` for the one the head declares. A non-binary mask raises an
+error rather than marking every non-zero class as sky.
+
+**Per-sample vs. hierarchical modalities.** You don't declare a modality's role:
+it is inferred from the ds-crawler indexes. A modality whose qualified IDs
+(hierarchy path plus file ID) match the transform's per-sample inputs is joined per
+sample, e.g. Synscapes' per-image `meta/<id>.json` intrinsics. A modality whose
+files share no sample IDs but sit at or above the samples' hierarchy level is
+matched hierarchically and shared by every sample beneath it, e.g. one intrinsics
+file per scene or camera. A modality that is neither raises an error, and the
+inferred roles are logged. Listing a per-sample modality under
+`hierarchical_modalities` logs a warning and loads it per sample.
 
 When a modality directory contains ds-crawler split files
 (`.ds_crawler/split_<name>.json`), set `split` on that modality to select a subset.
@@ -659,7 +676,8 @@ where sky depth is encoded as zero or infinity and needs a large finite value.
 }
 ```
 
-`sky_depth_value` defaults to `1000.0`.
+`sky_depth_value` defaults to `1000.0`. The sky mask comes from the dataset head as
+described under [Dataset Config](#dataset-config).
 
 CLI runs write a source-backed depth dataset mirroring the input depth modality's paths,
 filenames, extensions, and metadata. Standalone `SkyDepthTransform(...)` usage keeps the
@@ -677,7 +695,7 @@ d_radial(u, v) = d_planar(u, v) * sqrt(((u - cx)/fx)^2 + ((v - cy)/fy)^2 + 1)
 ```
 
 The transform config takes no parameters (`{}`); intrinsics are read from the
-`intrinsics` hierarchical modality.
+`intrinsics` modality, stored either per sample or per scene/camera.
 
 CLI runs write a source-backed depth dataset mirroring the input depth modality's layout
 and writer metadata, with `meta.radial_depth` set to `true` in the emitted `index.json`.

euler_preprocess-3.15.0/euler_preprocess.egg-info/PKG-INFO → euler_preprocess-3.17.0/README.md

@@ -1,23 +1,3 @@
-Metadata-Version: 2.4
-Name: euler-preprocess
-Version: 3.15.0
-Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
-Author-email: Daniel Rothenpieler <rothenpielerdaniel@gmail.com>
-License-Expression: MIT
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy
-Requires-Dist: Pillow
-Requires-Dist: euler-loading>=2.24.0
-Requires-Dist: ds-crawler>=2.11.0
-Requires-Dist: euler-dataset-contract>=0.5.0
-Provides-Extra: gpu
-Requires-Dist: torch; extra == "gpu"
-Provides-Extra: progress
-Requires-Dist: tqdm; extra == "progress"
-Dynamic: license-file
-
 <!-- euler header — shared across the euler packages.
      Per package, change only: the <h1>, the tagline, and the badge URLs. -->
 <p align="center">
@@ -94,9 +74,7 @@ loader.
   "modalities": {
     "rgb": {"path": "/path/to/rgb", "split": "train"},
     "depth": "/path/to/depth",
-    "semantic_segmentation": "/path/to/classSegmentation"
-  },
-  "hierarchical_modalities": {
+    "semantic_segmentation": "/path/to/classSegmentation",
     "intrinsics": {"path": "/path/to/intrinsics"}
   }
 }
@@ -109,17 +87,36 @@ loader.
 | `output_slot` | Optional slot selector when `pipeline.output_targets` has several entries. Defaults to `rgb` for `fog` and `depth` for `sky-depth` / `radial`. |
 | `sample` | Optional 0-based dataset index. Only `dataset[sample]` is transformed — useful for small benchmark slices of large datasets. |
 | `samples` | Optional multi-sample selector: a list of indices (`[0, 10, 20]`) or a slice object (`{"start": 0, "stop": 1000, "step": 2, "count": 100}`). `stop` is exclusive, `count` caps the result. Mutually exclusive with `sample`. |
-| `modalities` | Modalities that participate in sample-ID intersection. Each value is a path string or an object with `path` and optional `split`. |
-| `hierarchical_modalities` | Per-scene data (e.g. intrinsics), same format. Loaded once per scene and cached. |
+| `modalities` | All input modalities. Each value is a path string or an object with `path` and optional `split`. Whether a modality is per-sample or hierarchical is inferred (see below). |
+| `hierarchical_modalities` | Legacy, optional. Same format; entries are pooled with `modalities` and resolved the same way. |
 | `pipeline` | Optional runtime routing block compatible with `euler-inference`. |
 
 **Required modalities per transform:**
 
-| Transform | `modalities` | `hierarchical_modalities` |
+| Transform | Required | Optional |
 |---|---|---|
-| `fog` | `rgb`, `depth`, `semantic_segmentation` | `intrinsics` when available; used for radial depth conversion and camera-profile optics |
+| `fog` | `rgb`, `depth`, `semantic_segmentation` | `intrinsics`; used for radial depth conversion and camera-profile optics |
 | `sky-depth` | `depth`, `semantic_segmentation` | — |
-| `radial` | `depth` | `intrinsics` |
+| `radial` | `depth`, `intrinsics` | — |
+
+`fog` and `sky-depth` need `semantic_segmentation` as a boolean sky mask. They load
+it with the `sky_mask` function of the euler-loading module that the index's
+dataset head names, so a `class_segmentation` index works as is: the sky is the
+head's `skyclass` (Synscapes `[23, 23, 23]`, Real Drive Sim `[29, 0, 0]`, VKITTI2
+`[90, 200, 255]`), and the log names the loader and the sky definition in use.
+Set `"function"` in the modality entry to use another loader function, or
+`"function": null` for the one the head declares. A non-binary mask raises an
+error rather than marking every non-zero class as sky.
+
+**Per-sample vs. hierarchical modalities.** You don't declare a modality's role:
+it is inferred from the ds-crawler indexes. A modality whose qualified IDs
+(hierarchy path plus file ID) match the transform's per-sample inputs is joined per
+sample, e.g. Synscapes' per-image `meta/<id>.json` intrinsics. A modality whose
+files share no sample IDs but sit at or above the samples' hierarchy level is
+matched hierarchically and shared by every sample beneath it, e.g. one intrinsics
+file per scene or camera. A modality that is neither raises an error, and the
+inferred roles are logged. Listing a per-sample modality under
+`hierarchical_modalities` logs a warning and loads it per sample.
 
 When a modality directory contains ds-crawler split files
 (`.ds_crawler/split_<name>.json`), set `split` on that modality to select a subset.
@@ -659,7 +656,8 @@ where sky depth is encoded as zero or infinity and needs a large finite value.
 }
 ```
 
-`sky_depth_value` defaults to `1000.0`.
+`sky_depth_value` defaults to `1000.0`. The sky mask comes from the dataset head as
+described under [Dataset Config](#dataset-config).
 
 CLI runs write a source-backed depth dataset mirroring the input depth modality's paths,
 filenames, extensions, and metadata. Standalone `SkyDepthTransform(...)` usage keeps the
@@ -677,7 +675,7 @@ d_radial(u, v) = d_planar(u, v) * sqrt(((u - cx)/fx)^2 + ((v - cy)/fy)^2 + 1)
 ```
 
 The transform config takes no parameters (`{}`); intrinsics are read from the
-`intrinsics` hierarchical modality.
+`intrinsics` modality, stored either per sample or per scene/camera.
 
 CLI runs write a source-backed depth dataset mirroring the input depth modality's layout
 and writer metadata, with `meta.radial_depth` set to `true` in the emitted `index.json`.

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/cli.py

@@ -178,7 +178,14 @@ def _run_transform(args: argparse.Namespace, transform_class: type) -> int:
 
     required_modalities = transform_class.REQUIRED_MODALITIES
     required_hierarchical = transform_class.REQUIRED_HIERARCHICAL_MODALITIES or None
-    dataset = build_dataset(config, required_modalities, required_hierarchical)
+    # euler-loading logs which loader (and sky definition) each modality uses.
+    get_logger("euler_loading")
+    dataset = build_dataset(
+        config,
+        required_modalities,
+        required_hierarchical,
+        transform_class.MODALITY_FUNCTIONS,
+    )
     output_backends = prepare_output_backends(config, dataset, transform_class)
     primary_backend = next(iter(output_backends.values()))
     samples = _select_configured_samples(config, dataset, logger)

euler_preprocess-3.17.0/euler_preprocess/common/dataset.py

@@ -0,0 +1,224 @@
+from __future__ import annotations
+
+from collections import Counter
+
+from euler_preprocess.common.logging import get_logger
+
+
+def _make_modality(entry: str | dict, function: str | None = None):
+    """Build a ``Modality``; an entry's own ``function`` replaces *function*."""
+    from euler_loading import Modality
+
+    if isinstance(entry, str):
+        return Modality(entry, function=function)
+    options = {
+        key: entry[key]
+        for key in (
+            "split",
+            "metadata_scope",
+            "collapse_single",
+            "cache",
+            "applies_to",
+            "hierarchy_scope",
+            "function",
+        )
+        if key in entry
+    }
+    options.setdefault("function", function)
+    return Modality(entry["path"], **options)
+
+
+def _load_index_tree(name: str, modality) -> dict:
+    """Load a modality's ds-crawler tree the same way ``MultiModalDataset`` does."""
+    from ds_crawler import index_dataset_from_path
+    from euler_loading._ds_crawler_utils import load_index_output
+    from euler_loading.dataset import _get_index_tree, _resolve_metadata_scope
+
+    modality = _resolve_metadata_scope(name, modality)
+    index = load_index_output(
+        modality.path,
+        split=modality.split,
+        metadata_scope=modality.metadata_scope,
+        index_dataset_from_path_fn=index_dataset_from_path,
+    )
+    return _get_index_tree(index)
+
+
+def _check_attachment(
+    name: str, seed: str, tree: dict, samples_per_level: Counter
+) -> None:
+    """Check that hierarchical modality *name* applies to the samples by tree position."""
+    from euler_loading.indexing import collect_hierarchical_files, match_hierarchical_files
+
+    files_by_level = collect_hierarchical_files(tree)
+    attached = {
+        level: len(match_hierarchical_files(level, files_by_level))
+        for level in samples_per_level
+    }
+    covered = sum(count for level, count in samples_per_level.items() if attached[level])
+    if not covered:
+        raise ValueError(
+            f"Modality '{name}' shares no sample IDs with '{seed}' and none of its "
+            "files sit at or above the hierarchy level of any sample, so it can be "
+            "neither joined per sample nor matched hierarchically. Check that its "
+            f"ds-crawler IDs and hierarchy follow the same conventions as '{seed}'."
+        )
+    logger = get_logger()
+    total = sum(samples_per_level.values())
+    log = logger.info if covered == total else logger.warning
+    log(
+        "Modality '%s' is hierarchical (no sample IDs shared with '%s'); it applies "
+        "to %d/%d samples by hierarchy position.",
+        name,
+        seed,
+        covered,
+        total,
+    )
+    most = max(attached.values())
+    if most > 1:
+        logger.warning(
+            "Hierarchical modality '%s' attaches up to %d files to one sample. If it "
+            "holds per-sample data, its ds-crawler IDs do not match '%s'.",
+            name,
+            most,
+            seed,
+        )
+
+
+def _resolve_roles(
+    modalities: dict, per_sample: set[str]
+) -> tuple[list[str], list[str]]:
+    """Split modality names into regular and hierarchical ones.
+
+    The role follows from the ds-crawler indexes.  A modality is regular
+    (joined per sample) when its qualified file IDs -- hierarchy path plus
+    file ID -- overlap those of the seed modality.  Otherwise it is
+    hierarchical (matched by tree position, like per-scene intrinsics) and
+    must sit at or above the hierarchy level of at least one sample.
+
+    *per_sample* modalities are regular by contract and the first one is the
+    seed.  Without one, the seed is the modality aligned with the most others,
+    then the one with the most files, then the first listed.
+    """
+    from euler_loading.indexing import collect_files
+
+    names = list(modalities)
+    if len(names) < 2 or per_sample.issuperset(names):
+        return names, []
+
+    trees = {name: _load_index_tree(name, modalities[name]) for name in names}
+    # Qualified sample ID -> hierarchy path, keyed like euler-loading's lookups.
+    sample_ids = {}
+    for name, tree in trees.items():
+        sample_ids[name] = {
+            "/".join(r.hierarchy_path + (r.file_entry["id"],)): r.hierarchy_path
+            for r in collect_files(tree)
+        }
+
+    def aligned(name: str, other: str) -> bool:
+        return not sample_ids[name].keys().isdisjoint(sample_ids[other].keys())
+
+    anchors = [name for name in names if name in per_sample]
+    seed = anchors[0] if anchors else max(
+        names,
+        key=lambda name: (
+            sum(aligned(other, name) for other in names),
+            len(sample_ids[name]),
+        ),
+    )
+    regular = [
+        name
+        for name in names
+        if name in per_sample or name == seed or aligned(name, seed)
+    ]
+    hierarchical = [name for name in names if name not in regular]
+    get_logger().info(
+        "Modality roles inferred from ds-crawler indexes: per-sample %s; "
+        "hierarchical %s",
+        ", ".join(regular),
+        ", ".join(hierarchical) or "none",
+    )
+
+    # Without common samples MultiModalDataset raises its own, clearer error.
+    common = set(sample_ids[seed]).intersection(*(sample_ids[name] for name in regular))
+    if common:
+        samples_per_level = Counter(sample_ids[seed][sample_id] for sample_id in common)
+        for name in hierarchical:
+            _check_attachment(name, seed, trees[name], samples_per_level)
+    return regular, hierarchical
+
+
+def build_dataset(
+    config: dict,
+    required_modalities: set[str],
+    required_hierarchical: set[str] | None = None,
+    modality_functions: dict[str, str] | None = None,
+):
+    """Build a ``MultiModalDataset`` from a config dict.
+
+    Each modality entry is either a plain path string or a dict with
+    ``path`` and optional ``split``.  Loader resolution (which function to
+    call, which module to use) is handled by euler-loading via the
+    ds-crawler index at each path — point the config at a path whose
+    index declares the function you want (e.g. a ``sky_mask`` index for
+    boolean sky masks vs. a ``class_segmentation`` index for raw class
+    id maps).
+
+    Every modality can be listed under ``modalities``: whether it is joined
+    per sample or matched hierarchically (e.g. per-scene intrinsics) is
+    inferred from the ds-crawler indexes.  Entries under the legacy
+    ``hierarchical_modalities`` key are pooled with them and resolved the
+    same way.  *required_modalities* are per-sample inputs and anchor that
+    inference; *required_hierarchical* names further required modalities,
+    whose role is inferred like any other.
+
+    *modality_functions* maps a modality to the loader function to use from
+    the module its dataset head names (a transform's ``MODALITY_FUNCTIONS``),
+    e.g. ``sky_mask`` for a ``class_segmentation`` index; the sky then comes
+    from the head's ``skyclass``.  A modality entry's own ``function`` key
+    overrides it, and ``"function": null`` keeps the head's function.
+    """
+    from euler_loading import MultiModalDataset, SamplePreprocessor
+
+    raw_modalities = config.get("modalities", {})
+    raw_hierarchical = config.get("hierarchical_modalities", {})
+
+    duplicated = raw_modalities.keys() & raw_hierarchical.keys()
+    if duplicated:
+        raise ValueError(
+            "Modalities listed under both 'modalities' and "
+            f"'hierarchical_modalities': {', '.join(sorted(duplicated))}"
+        )
+    configured = {**raw_modalities, **raw_hierarchical}
+
+    required = set(required_modalities) | set(required_hierarchical or ())
+    missing = required - configured.keys()
+    if missing:
+        raise ValueError(
+            f"Missing required modalities in config: {', '.join(sorted(missing))}. "
+            f"'modalities' must contain at least: {', '.join(sorted(required))}"
+        )
+
+    functions = modality_functions or {}
+    modalities = {
+        name: _make_modality(entry, functions.get(name))
+        for name, entry in configured.items()
+    }
+    regular, hierarchical = _resolve_roles(modalities, set(required_modalities))
+    logger = get_logger()
+    for name in regular:
+        if name in raw_hierarchical:
+            logger.warning(
+                "Modality '%s' is listed under 'hierarchical_modalities' but holds "
+                "per-sample data; loading it per sample.",
+                name,
+            )
+
+    return MultiModalDataset(
+        modalities={name: modalities[name] for name in regular},
+        hierarchical_modalities={name: modalities[name] for name in hierarchical}
+        or None,
+        transforms=[SamplePreprocessor.from_config(config["preprocessing"])]
+        if config.get("preprocessing")
+        else None,
+    )

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/common/intrinsics.py

@@ -13,8 +13,9 @@ except ImportError:
 def extract_intrinsics(sample: dict) -> np.ndarray | None:
     """Extract the 3x3 intrinsics matrix from a sample dict.
 
-    Hierarchical modalities are stored as ``sample[name][file_id]``.
-    For intrinsics the convention is ``sample["intrinsics"]["intrinsics"]``.
+    Per-sample intrinsics are stored directly as ``sample["intrinsics"]``.
+    Hierarchical modalities are stored as ``sample[name][file_id]``; for
+    intrinsics the convention is ``sample["intrinsics"]["intrinsics"]``.
     """
     intr_dict = sample.get("intrinsics")
     if intr_dict is None:

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/common/normalize.py

@@ -98,7 +98,9 @@ def normalize_sky_mask(sky_mask) -> np.ndarray:
     """Normalise a sky mask to a 2-D boolean numpy array ``(H, W)``.
 
     Handles torch tensors (GPU loaders) and the ``(1, H, W)`` channel-first
-    layout produced by euler-loading's GPU loaders.
+    layout produced by euler-loading's GPU loaders. A non-boolean input may
+    only hold 0/1: casting class IDs to bool would mark every non-zero class
+    as sky.
     """
     mask = _to_numpy(sky_mask)
     # CHW → take first channel: (C, H, W) → (H, W)
@@ -109,4 +111,15 @@ def normalize_sky_mask(sky_mask) -> np.ndarray:
         mask = mask[0]
     if mask.ndim == 3 and mask.shape[-1] == 1:
         mask = mask[..., 0]
+    if mask.dtype != np.bool_ and np.any((mask != 0) & (mask != 1)):
+        values = np.unique(mask)
+        shown = ", ".join(str(value) for value in values[:8])
+        if values.size > 8:
+            shown += ", ..."
+        raise ValueError(
+            "semantic_segmentation must be a boolean sky mask, got a "
+            f"{mask.dtype} array with values [{shown}]. Load it with the "
+            "dataset's euler-loading sky_mask loader (the CLI selects it "
+            "automatically) or pass a boolean mask."
+        )
     return mask.astype(bool)

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/common/transform.py

@@ -12,6 +12,19 @@ class Transform(ABC):
     Subclasses declare the modalities they need via class variables and
     implement :meth:`run` to process samples.
 
+    Required modalities:
+        :attr:`REQUIRED_MODALITIES` are per-sample inputs (RGB, depth, masks);
+        they are always joined per sample and anchor the role inference of
+        every other modality.  :attr:`REQUIRED_HIERARCHICAL_MODALITIES` are
+        further required inputs, such as camera intrinsics, that a dataset may
+        store per sample or once per hierarchy level; their role is inferred
+        from the ds-crawler indexes (see
+        :func:`~euler_preprocess.common.dataset.build_dataset`).
+        :attr:`MODALITY_FUNCTIONS` maps a modality to the euler-loading loader
+        function the transform needs, taken from the loader module its
+        dataset head names (``{"semantic_segmentation": "sky_mask"}`` reads a
+        class-segmentation index as sky masks).
+
     Output slots:
         Most transforms produce a single output (the *primary* slot, declared
         via :attr:`OUTPUT_SLOT`).  Transforms that produce additional auxiliary
@@ -23,6 +36,7 @@ class Transform(ABC):
 
     REQUIRED_MODALITIES: ClassVar[set[str]] = set()
     REQUIRED_HIERARCHICAL_MODALITIES: ClassVar[set[str]] = set()
+    MODALITY_FUNCTIONS: ClassVar[dict[str, str]] = {}
     SOURCE_MODALITY: ClassVar[str | None] = None
     OUTPUT_SLOT: ClassVar[str | None] = None
     OUTPUT_SLOTS: ClassVar[tuple[str, ...]] = ()

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/fog/atmospheric_light.py

@@ -124,7 +124,7 @@ class AtmosphericLightResolver:
         airlight_is_estimated = uses_estimated_airlight(al_spec)
         if airlight_is_estimated:
             raw_ls_base = normalize_atmospheric_light_torch(
-                estimated_airlight_t
+                estimated_airlight_t, allow_8bit=False
             ).squeeze(0)
         else:
             sampled_al = sample_value(al_spec, rng)
@@ -156,7 +156,7 @@ class AtmosphericLightResolver:
         airlight_is_estimated = uses_estimated_airlight(al_spec)
         if airlight_is_estimated:
             ls_base = self._estimate_batch_torch(rgb_batch, items, device, method)
-            ls_base = normalize_atmospheric_light_torch(ls_base)
+            ls_base = normalize_atmospheric_light_torch(ls_base, allow_8bit=False)
         else:
             ls_values = []
             for item in items:

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/fog/inference.py

@@ -58,7 +58,7 @@ def render_fog_image(
     mode: FogInferenceMode = "cpu",
     sample_id: str | None = None,
     sample_index: int = 0,
-    sky_class: Sequence[int | float] | None = (29, 0, 0),
+    sky_class: Sequence[int | float] | None = None,
 ) -> np.ndarray:
     """Render and return only the RGB image for one in-memory sample."""
 
@@ -89,14 +89,15 @@ def render_fog_sample(
     mode: FogInferenceMode = "cpu",
     sample_id: str | None = None,
     sample_index: int = 0,
-    sky_class: Sequence[int | float] | None = (29, 0, 0),
+    sky_class: Sequence[int | float] | None = None,
 ) -> FogInferenceResult:
     """Render fog/camera artifacts for a single in-memory sample.
 
     The call accepts the four modalities used by :class:`FogTransform` without
     requiring a dataset reader or output backend. ``semantic_segmentation`` may
     be either a boolean sky mask or an RGB semantic label map; RGB maps are
-    converted using ``sky_class`` which defaults to ``[29, 0, 0]``.
+    converted using ``sky_class``, the map's sky colour as its dataset head
+    declares it in ``skyclass`` (e.g. ``[29, 0, 0]`` for Real Drive Sim).
     """
 
     if mode not in ("cpu", "gpu"):

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/fog/logging.py

@@ -1,6 +1,8 @@
 from __future__ import annotations
 
 import logging
+from collections.abc import Sequence
+from typing import Any
 
 
 def log_config(
@@ -18,7 +20,14 @@ def log_config(
     resize_depth: bool,
     seed,
     contrast_threshold_default: float,
+    scenarios: Sequence[tuple[str, Any, Any]] = (),
+    scenario_scope: str = "sample",
 ) -> None:
+    """Log the resolved run settings.
+
+    *scenarios* holds ``(name, weight, model)`` per scenario profile, with
+    ``model`` ``None`` when the profile defers to ``selection``.
+    """
     logger.info("Loaded config from %s -> output %s", config_path, out_path)
     logger.info(
         "Device: requested=%s active=%s use_gpu=%s torch=%s",
@@ -36,6 +45,22 @@ def log_config(
         seed,
         contrast_threshold_default,
     )
+    if scenarios:
+        logger.info(
+            "Model selection: %d scenario_profiles (scope=%s) choose the model; "
+            "'selection' only applies to profiles without one",
+            len(scenarios),
+            scenario_scope,
+        )
+        for name, weight, model in scenarios:
+            logger.info(
+                "Scenario %s: weight=%s model=%s",
+                name,
+                weight,
+                "<selection>" if model is None else model,
+            )
+        if all(model is not None for _name, _weight, model in scenarios):
+            return
     selection = config.get("selection")
     if selection:
         mode = selection.get("mode", "fixed")

{euler_preprocess-3.15.0 → euler_preprocess-3.17.0}/euler_preprocess/fog/models.py

@@ -160,18 +160,34 @@ def resolve_scattering_coefficient(
     )
 
 
-def normalize_atmospheric_light(value: np.ndarray) -> np.ndarray:
+def normalize_atmospheric_light(
+    value: np.ndarray, *, allow_8bit: bool = True
+) -> np.ndarray:
+    """Return ``value`` as an RGB triple clipped to ``[0, 1]``.
+
+    With *allow_8bit*, values above 1 are read as an 8-bit colour and divided
+    by 255. That only suits configured colours: estimated airlight comes from
+    ``[0, 1]`` RGB and may overshoot 1 slightly after the estimators' luminance
+    rescaling, so estimate callers pass ``allow_8bit=False`` to clip instead.
+    """
     value = np.asarray(value, dtype=np.float32)
     if value.ndim == 0:
         value = np.array([value, value, value], dtype=np.float32)
     if value.ndim != 1 or value.shape[0] != 3:
         raise ValueError("atmospheric_light must be scalar or length-3 list")
-    if value.max() > 1.0:
+    if allow_8bit and value.max() > 1.0:
         value = value / 255.0
     return np.clip(value, 0.0, 1.0)
 
 
-def normalize_atmospheric_light_torch(value: "torch.Tensor") -> "torch.Tensor":
+def normalize_atmospheric_light_torch(
+    value: "torch.Tensor", *, allow_8bit: bool = True
+) -> "torch.Tensor":
+    """Torch counterpart of :func:`normalize_atmospheric_light`.
+
+    The 8-bit check uses the maximum over every row, so stacked estimates must
+    pass ``allow_8bit=False`` or one overshooting sample rescales the batch.
+    """
     value_t = value.to(torch.float32)
     if value_t.ndim == 0:
         value_t = value_t.repeat(3)
@@ -180,7 +196,7 @@ def normalize_atmospheric_light_torch(value: "torch.Tensor") -> "torch.Tensor":
     if value_t.ndim != 2 or value_t.shape[-1] != 3:
         raise ValueError("atmospheric_light must be scalar or length-3 list")
     max_val = float(value_t.max().item()) if value_t.numel() else 0.0
-    if max_val > 1.0:
+    if allow_8bit and max_val > 1.0:
         value_t = value_t / 255.0
     return torch.clamp(value_t, 0.0, 1.0)
 
@@ -1653,7 +1669,9 @@ def apply_model(
     al_spec = model_cfg.get("atmospheric_light", "from_sky")
     airlight_is_estimated = uses_estimated_airlight(al_spec)
     if airlight_is_estimated:
-        raw_ls_base = normalize_atmospheric_light(estimated_airlight)
+        raw_ls_base = normalize_atmospheric_light(
+            estimated_airlight, allow_8bit=False
+        )
     else:
         sampled_al = sample_value(al_spec, rng)
         raw_ls_base = normalize_atmospheric_light(np.asarray(sampled_al))