euler-preprocess 3.0.0__tar.gz → 3.2.0__tar.gz

{euler_preprocess-3.0.0 → euler_preprocess-3.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-preprocess
-Version: 3.0.0
+Version: 3.2.0
 Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
 Requires-Python: >=3.9
 Description-Content-Type: text/markdown
@@ -170,6 +170,7 @@ Controls the fog simulation.
 | `capture` / `capture_artifacts` | Optional post-fog camera artifact pipeline. Omit it or set `{"stages": []}` for the legacy no-op path. Set `true`, `{"preset": "camera"}`, or a custom `stages` list to enable optics, raw sensor, ISP, and compression artifacts. |
 | `camera_profile` | Optional named or inline camera profile whose stage defaults are merged into the capture stack before per-stage overrides. Built-ins are `"default"`, `"generic"`, `"dashcam"`, and `"low_light_fog"`. |
 | `camera_profiles` | Optional map of project-specific named profiles. Use this for calibrated lens, sensor, ISP, and transport settings. |
+| `scenario_profiles` | Optional top-level correlated condition sampler. Each sampled scenario can choose the fog model, override model parameters, select the airlight method, switch camera profile/settings, and force named capture-stage condition profiles. |
 | `augmentations` | Optional stepped augmentation set. When present, every input sample produces every configured augmentation and uses the file-id hierarchy output layout described below. |
 
 ### Processing Pipeline
@@ -214,8 +215,32 @@ For tighter control, provide explicit stages in camera order:
       "bayer_pattern": "RGGB",
       "iso": {"dist": "choice", "values": [200, 400, 800]},
       "base_iso": 100,
+      "auto_exposure": {
+        "enabled": true,
+        "metering": "center_weighted",
+        "target_luminance": {"dist": "uniform", "min": 0.16, "max": 0.24},
+        "highlight_protection": 0.7,
+        "resolve_iso": true,
+        "max_iso": 1600
+      },
       "full_well_electrons": [14000, 12000, 13000],
       "read_noise_electrons": {"dist": "uniform", "min": 2.0, "max": 6.0},
+      "shadow_recovery_noise": {
+        "enabled": true,
+        "luminance_threshold": {"dist": "uniform", "min": 0.16, "max": 0.24},
+        "luma_sigma": {"dist": "uniform", "min": 0.0, "max": 0.001},
+        "chroma_sigma": {"dist": "uniform", "min": 0.012, "max": 0.028},
+        "chroma_mode": "balanced",
+        "red_chroma_gain": {"dist": "uniform", "min": 0.7, "max": 1.0},
+        "blue_chroma_gain": {"dist": "uniform", "min": 1.7, "max": 2.7},
+        "chroma_axis_correlation": {"dist": "uniform", "min": 0.05, "max": 0.25},
+        "chroma_spatial_sigma": 0.0,
+        "chroma_fine_fraction": 1.0,
+        "chroma_luminance_preservation": 1.0,
+        "black_noise_floor": 0.25,
+        "black_suppression_luminance": 0.03,
+        "black_suppression_softness": 0.08
+      },
       "black_level": [0.003, 0.0035, 0.003],
       "white_level": [1.0, 0.995, 1.0],
       "adc_bit_depth": 12,
@@ -272,11 +297,32 @@ Supported stage types:
 | Stage | Main effects |
 |---|---|
 | `optics` | Defocus/MTF blur, motion blur, bloom, veiling glare, vignetting, chromatic aberration, lens distortion, windshield haze, optional droplets. |
-| `sensor` | Exposure, white balance, camera matrix, Bayer mosaic, shot/read noise, fixed-pattern noise, row/column banding, hot/dead pixels, bilinear demosaic. |
+| `sensor` | Image-driven or sampled exposure, white balance, camera matrix, Bayer mosaic, shot/read noise, fixed-pattern noise, row/column banding, shadow-local recovery noise, hot/dead pixels, bilinear demosaic. |
 | `isp` | Denoising, color correction, tone mapping, sRGB/gamma, local contrast, sharpening halos, saturation shifts. |
 | `transport` | Crop/resize, bit-depth quantization, JPEG round-trip compression. |
 | `exposure` | Lightweight standalone exposure and white-balance stage for simple custom chains. |
 
+Set `sensor.auto_exposure.enabled` to meter the rendered image before raw sensor
+sampling. `target_luminance`, `metering`, `highlight_*`, and gain bounds choose
+the exposure; `resolve_iso` can raise ISO from the metering pressure, dark pixel
+fraction, and fog opacity. When auto exposure is enabled, `exposure_gain` still
+applies as scenario-specific exposure compensation.
+
+Set `sensor.shadow_recovery_noise.enabled` to add extra post-demosaic luma and
+chroma corruption only where the pre-exposure rendered luminance was low. This
+is useful for reducing broad global grain while keeping lifted shadows visibly
+noisy. For dark high-ISO scenes, keep `luma_sigma` much lower than
+`chroma_sigma`, use `chroma_mode: "balanced"`, and leave
+`chroma_luminance_preservation` near `1.0`; this makes the local corruption read
+as color noise instead of black luma speckles. `red_chroma_gain`,
+`blue_chroma_gain`, and `chroma_axis_correlation` can match camera-specific
+color noise, including blue/purple-biased high-ISO noise. Keep
+`chroma_spatial_sigma` near `0` and avoid chroma subsampling when the target
+noise is fine-grained rather than blocky. `black_noise_floor` with
+`black_suppression_luminance`/`black_suppression_softness` reduces the extra
+noise in near-clipped black regions, so the strongest visible noise sits in
+dim-but-readable shadows.
+
 Any stage can define `condition_profiles` to sample coherent per-image settings
 before the stage runs. This is useful for exposure states where ISO, exposure
 gain, read noise, banding, and dark/fog noise modulation should move together:
@@ -291,6 +337,51 @@ gain, read noise, banding, and dark/fog noise modulation should move together:
 }
 ```
 
+Top-level `scenario_profiles` sample one latent scene/camera condition before
+rendering. The selected scenario is merged over the root config, so it can drive
+fog density, atmospheric light, camera profile, capture-stage overrides, ISP, and
+compression together:
+
+```json
+"scenario_profiles": [
+  {
+    "name": "clean_low_noise_haze",
+    "weight": 0.22,
+    "model": "heterogeneous_k_ls",
+    "airlight_method": "dcp_heuristic",
+    "models": {
+      "heterogeneous_k_ls": {
+        "visibility_m": {"dist": "uniform", "min": 60.0, "max": 130.0}
+      }
+    },
+    "capture_overrides": {
+      "sensor": {"condition_profile": "clean_daylight"},
+      "isp": {"denoise_sigma": {"dist": "uniform", "min": 0.08, "max": 0.32}}
+    }
+  },
+  {
+    "name": "underexposed_dense_gloom",
+    "weight": 0.25,
+    "model": "heterogeneous_k_ls",
+    "airlight_method": "dcp_heuristic",
+    "models": {
+      "heterogeneous_k_ls": {
+        "visibility_m": {"dist": "uniform", "min": 18.0, "max": 55.0}
+      }
+    },
+    "capture_overrides": {
+      "sensor": {"condition_profile": "underexposed_noisy"},
+      "transport": {"jpeg": {"quality": {"dist": "uniform", "min": 54, "max": 78}}}
+    }
+  }
+]
+```
+
+`capture_overrides` is merged after camera-profile and stage settings. Use
+`condition_profile` to force one named profile from a stage's
+`condition_profiles`; if omitted, the stage continues sampling its own profile
+weights locally.
+
 ### Fog Model
 
 The core equation is the **Koschmieder model** (atmospheric scattering):

{euler_preprocess-3.0.0 → euler_preprocess-3.2.0}/README.md

@@ -156,6 +156,7 @@ Controls the fog simulation.
 | `capture` / `capture_artifacts` | Optional post-fog camera artifact pipeline. Omit it or set `{"stages": []}` for the legacy no-op path. Set `true`, `{"preset": "camera"}`, or a custom `stages` list to enable optics, raw sensor, ISP, and compression artifacts. |
 | `camera_profile` | Optional named or inline camera profile whose stage defaults are merged into the capture stack before per-stage overrides. Built-ins are `"default"`, `"generic"`, `"dashcam"`, and `"low_light_fog"`. |
 | `camera_profiles` | Optional map of project-specific named profiles. Use this for calibrated lens, sensor, ISP, and transport settings. |
+| `scenario_profiles` | Optional top-level correlated condition sampler. Each sampled scenario can choose the fog model, override model parameters, select the airlight method, switch camera profile/settings, and force named capture-stage condition profiles. |
 | `augmentations` | Optional stepped augmentation set. When present, every input sample produces every configured augmentation and uses the file-id hierarchy output layout described below. |
 
 ### Processing Pipeline
@@ -200,8 +201,32 @@ For tighter control, provide explicit stages in camera order:
       "bayer_pattern": "RGGB",
       "iso": {"dist": "choice", "values": [200, 400, 800]},
       "base_iso": 100,
+      "auto_exposure": {
+        "enabled": true,
+        "metering": "center_weighted",
+        "target_luminance": {"dist": "uniform", "min": 0.16, "max": 0.24},
+        "highlight_protection": 0.7,
+        "resolve_iso": true,
+        "max_iso": 1600
+      },
       "full_well_electrons": [14000, 12000, 13000],
       "read_noise_electrons": {"dist": "uniform", "min": 2.0, "max": 6.0},
+      "shadow_recovery_noise": {
+        "enabled": true,
+        "luminance_threshold": {"dist": "uniform", "min": 0.16, "max": 0.24},
+        "luma_sigma": {"dist": "uniform", "min": 0.0, "max": 0.001},
+        "chroma_sigma": {"dist": "uniform", "min": 0.012, "max": 0.028},
+        "chroma_mode": "balanced",
+        "red_chroma_gain": {"dist": "uniform", "min": 0.7, "max": 1.0},
+        "blue_chroma_gain": {"dist": "uniform", "min": 1.7, "max": 2.7},
+        "chroma_axis_correlation": {"dist": "uniform", "min": 0.05, "max": 0.25},
+        "chroma_spatial_sigma": 0.0,
+        "chroma_fine_fraction": 1.0,
+        "chroma_luminance_preservation": 1.0,
+        "black_noise_floor": 0.25,
+        "black_suppression_luminance": 0.03,
+        "black_suppression_softness": 0.08
+      },
       "black_level": [0.003, 0.0035, 0.003],
       "white_level": [1.0, 0.995, 1.0],
       "adc_bit_depth": 12,
@@ -258,11 +283,32 @@ Supported stage types:
 | Stage | Main effects |
 |---|---|
 | `optics` | Defocus/MTF blur, motion blur, bloom, veiling glare, vignetting, chromatic aberration, lens distortion, windshield haze, optional droplets. |
-| `sensor` | Exposure, white balance, camera matrix, Bayer mosaic, shot/read noise, fixed-pattern noise, row/column banding, hot/dead pixels, bilinear demosaic. |
+| `sensor` | Image-driven or sampled exposure, white balance, camera matrix, Bayer mosaic, shot/read noise, fixed-pattern noise, row/column banding, shadow-local recovery noise, hot/dead pixels, bilinear demosaic. |
 | `isp` | Denoising, color correction, tone mapping, sRGB/gamma, local contrast, sharpening halos, saturation shifts. |
 | `transport` | Crop/resize, bit-depth quantization, JPEG round-trip compression. |
 | `exposure` | Lightweight standalone exposure and white-balance stage for simple custom chains. |
 
+Set `sensor.auto_exposure.enabled` to meter the rendered image before raw sensor
+sampling. `target_luminance`, `metering`, `highlight_*`, and gain bounds choose
+the exposure; `resolve_iso` can raise ISO from the metering pressure, dark pixel
+fraction, and fog opacity. When auto exposure is enabled, `exposure_gain` still
+applies as scenario-specific exposure compensation.
+
+Set `sensor.shadow_recovery_noise.enabled` to add extra post-demosaic luma and
+chroma corruption only where the pre-exposure rendered luminance was low. This
+is useful for reducing broad global grain while keeping lifted shadows visibly
+noisy. For dark high-ISO scenes, keep `luma_sigma` much lower than
+`chroma_sigma`, use `chroma_mode: "balanced"`, and leave
+`chroma_luminance_preservation` near `1.0`; this makes the local corruption read
+as color noise instead of black luma speckles. `red_chroma_gain`,
+`blue_chroma_gain`, and `chroma_axis_correlation` can match camera-specific
+color noise, including blue/purple-biased high-ISO noise. Keep
+`chroma_spatial_sigma` near `0` and avoid chroma subsampling when the target
+noise is fine-grained rather than blocky. `black_noise_floor` with
+`black_suppression_luminance`/`black_suppression_softness` reduces the extra
+noise in near-clipped black regions, so the strongest visible noise sits in
+dim-but-readable shadows.
+
 Any stage can define `condition_profiles` to sample coherent per-image settings
 before the stage runs. This is useful for exposure states where ISO, exposure
 gain, read noise, banding, and dark/fog noise modulation should move together:
@@ -277,6 +323,51 @@ gain, read noise, banding, and dark/fog noise modulation should move together:
 }
 ```
 
+Top-level `scenario_profiles` sample one latent scene/camera condition before
+rendering. The selected scenario is merged over the root config, so it can drive
+fog density, atmospheric light, camera profile, capture-stage overrides, ISP, and
+compression together:
+
+```json
+"scenario_profiles": [
+  {
+    "name": "clean_low_noise_haze",
+    "weight": 0.22,
+    "model": "heterogeneous_k_ls",
+    "airlight_method": "dcp_heuristic",
+    "models": {
+      "heterogeneous_k_ls": {
+        "visibility_m": {"dist": "uniform", "min": 60.0, "max": 130.0}
+      }
+    },
+    "capture_overrides": {
+      "sensor": {"condition_profile": "clean_daylight"},
+      "isp": {"denoise_sigma": {"dist": "uniform", "min": 0.08, "max": 0.32}}
+    }
+  },
+  {
+    "name": "underexposed_dense_gloom",
+    "weight": 0.25,
+    "model": "heterogeneous_k_ls",
+    "airlight_method": "dcp_heuristic",
+    "models": {
+      "heterogeneous_k_ls": {
+        "visibility_m": {"dist": "uniform", "min": 18.0, "max": 55.0}
+      }
+    },
+    "capture_overrides": {
+      "sensor": {"condition_profile": "underexposed_noisy"},
+      "transport": {"jpeg": {"quality": {"dist": "uniform", "min": 54, "max": 78}}}
+    }
+  }
+]
+```
+
+`capture_overrides` is merged after camera-profile and stage settings. Use
+`condition_profile` to force one named profile from a stage's
+`condition_profiles`; if omitted, the stage continues sampling its own profile
+weights locally.
+
 ### Fog Model
 
 The core equation is the **Koschmieder model** (atmospheric scattering):

euler_preprocess-3.2.0/euler_preprocess/fog/__init__.py

@@ -0,0 +1,11 @@
+from euler_preprocess.fog.inference import (
+    FogInferenceResult,
+    render_fog_image,
+    render_fog_sample,
+)
+
+__all__ = [
+    "FogInferenceResult",
+    "render_fog_image",
+    "render_fog_sample",
+]

{euler_preprocess-3.0.0 → euler_preprocess-3.2.0}/euler_preprocess/fog/atmospheric_light.py

@@ -155,12 +155,13 @@ class AtmosphericLightResolver:
         items: list[dict],
         device: Any,
         contrast_threshold_default: float,
+        method: str | None = None,
     ) -> tuple[list[float], "torch.Tensor"]:
         """Resolve beta and dampened base L_s for a uniform-model torch batch."""
         al_spec = items[0]["model_cfg"].get("atmospheric_light", "from_sky")
         airlight_is_estimated = uses_estimated_airlight(al_spec)
         if airlight_is_estimated:
-            ls_base = self._estimate_batch_torch(rgb_batch, items, device)
+            ls_base = self._estimate_batch_torch(rgb_batch, items, device, method)
             ls_base = normalize_atmospheric_light_torch(ls_base)
         else:
             ls_values = []
@@ -209,17 +210,19 @@ class AtmosphericLightResolver:
         rgb_batch: "torch.Tensor",
         items: list[dict],
         device: Any,
+        method: str | None = None,
     ) -> "torch.Tensor":
-        if self.method == "from_sky":
+        resolved_method = method or self.method
+        if resolved_method == "from_sky":
             return self._estimate_from_sky_batch_torch(rgb_batch, items, device)
-        estimator = self._get_estimator_torch(self.method)
+        estimator = self._get_estimator_torch(resolved_method)
         if estimator is None:
             raise RuntimeError(
-                f"Torch airlight estimator unavailable for method '{self.method}'."
+                f"Torch airlight estimator unavailable for method '{resolved_method}'."
             )
         al_list = []
         for idx, item in enumerate(items):
-            if self.method == "dcp":
+            if resolved_method == "dcp":
                 al_list.append(estimator.compute(rgb_batch[idx]))
             else:
                 sky_mask_t = torch.from_numpy(item["sky_mask"]).to(