euler-preprocess 2.2.0__tar.gz → 3.0.0__tar.gz

{euler_preprocess-2.2.0 → euler_preprocess-3.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-preprocess
-Version: 2.2.0
+Version: 3.0.0
 Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
 Requires-Python: >=3.9
 Description-Content-Type: text/markdown
@@ -98,7 +98,7 @@ starting at index 10 with stride 5.
 
 | Transform | `modalities` | `hierarchical_modalities` |
 |---|---|---|
-| `fog` | `rgb`, `depth`, `semantic_segmentation` | — (intrinsics optional) |
+| `fog` | `rgb`, `depth`, `semantic_segmentation` | `intrinsics` when available; used for radial depth conversion and camera-profile optics |
 | `sky-depth` | `depth`, `semantic_segmentation` | — |
 | `radial` | `depth` | `intrinsics` |
 
@@ -150,6 +150,8 @@ Controls the fog simulation.
   "contrast_threshold": 0.05,
   "device": "cpu",
   "gpu_batch_size": 4,
+  "capture": { "preset": "camera" },
+  "camera_profile": "dashcam",
   "augmentations": { ... },
   "selection": { ... },
   "models": { ... }
@@ -165,8 +167,130 @@ Controls the fog simulation.
 | `contrast_threshold` | Threshold *C_t* used in the visibility-to-attenuation conversion (default `0.05`). |
 | `device` | `"cpu"`, `"cuda"`, `"mps"`, or `"gpu"` (alias for cuda). |
 | `gpu_batch_size` | Batch size when running on GPU. Uniform-model samples are batched; heterogeneous samples are processed individually. |
+| `capture` / `capture_artifacts` | Optional post-fog camera artifact pipeline. Omit it or set `{"stages": []}` for the legacy no-op path. Set `true`, `{"preset": "camera"}`, or a custom `stages` list to enable optics, raw sensor, ISP, and compression artifacts. |
+| `camera_profile` | Optional named or inline camera profile whose stage defaults are merged into the capture stack before per-stage overrides. Built-ins are `"default"`, `"generic"`, `"dashcam"`, and `"low_light_fog"`. |
+| `camera_profiles` | Optional map of project-specific named profiles. Use this for calibrated lens, sensor, ISP, and transport settings. |
 | `augmentations` | Optional stepped augmentation set. When present, every input sample produces every configured augmentation and uses the file-id hierarchy output layout described below. |
 
+### Processing Pipeline
+
+Fog generation is split into two phases:
+
+1. Ideal scene rendering: physics-based fog and auxiliary `scattering_coefficient`
+   / `atmospheric_light` maps are computed.
+2. Capture artifacts: camera-specific effects are applied to the rendered RGB
+   only. Physical fog maps stay stable while the RGB output can receive
+   exposure shifts, lens blur, vignetting, raw sensor noise, Bayer/demosaicing
+   artifacts, ISP tone/gamma/sharpening, and JPEG/resize/quantization effects.
+
+This keeps physical fog maps stable while making the RGB output extensible for
+real-camera simulation.
+
+### Capture Artifact Stack
+
+Enable the recommended camera stack with:
+
+```json
+"capture": { "preset": "camera" }
+```
+
+or equivalently:
+
+```json
+"capture": true
+```
+
+For tighter control, provide explicit stages in camera order:
+
+```json
+"camera_profiles": {
+  "real_drive_front": {
+    "optics": {
+      "lens_distortion": -0.012,
+      "vignetting_strength": 0.18,
+      "windshield_haze": {"enabled": true, "probability": 0.55}
+    },
+    "sensor": {
+      "bayer_pattern": "RGGB",
+      "iso": {"dist": "choice", "values": [200, 400, 800]},
+      "base_iso": 100,
+      "full_well_electrons": [14000, 12000, 13000],
+      "read_noise_electrons": {"dist": "uniform", "min": 2.0, "max": 6.0},
+      "black_level": [0.003, 0.0035, 0.003],
+      "white_level": [1.0, 0.995, 1.0],
+      "adc_bit_depth": 12,
+      "post_demosaic_bit_depth": 12
+    },
+    "isp": {
+      "tone_map": "reinhard",
+      "gamma": "srgb",
+      "denoise_sigma": 0.2,
+      "sharpen_amount": 0.2,
+      "saturation": 0.9
+    },
+    "transport": {
+      "jpeg": {"enabled": true, "quality": {"dist": "uniform", "min": 65, "max": 92}},
+      "bit_depth": 8
+    }
+  }
+},
+"camera_profile": "real_drive_front",
+"capture": {
+  "stages": [
+    {
+      "type": "optics",
+      "blur_sigma": {"dist": "uniform", "min": 0.2, "max": 0.8},
+      "vignetting_strength": 0.15,
+      "windshield_haze": {"enabled": true, "probability": 0.4},
+      "droplets": {"enabled": false}
+    },
+    {
+      "type": "sensor",
+      "input_space": "srgb",
+      "exposure_gain": {"dist": "uniform", "min": 0.85, "max": 1.2},
+      "row_noise_sigma": 0.003
+    },
+    {
+      "type": "isp",
+      "tone_map": "reinhard",
+      "gamma": "srgb",
+      "denoise_sigma": 0.2,
+      "sharpen_amount": 0.2,
+      "saturation": 0.9
+    },
+    {
+      "type": "transport",
+      "jpeg": {"enabled": true, "quality": {"dist": "uniform", "min": 65, "max": 92}},
+      "bit_depth": 8
+    }
+  ]
+}
+```
+
+Supported stage types:
+
+| Stage | Main effects |
+|---|---|
+| `optics` | Defocus/MTF blur, motion blur, bloom, veiling glare, vignetting, chromatic aberration, lens distortion, windshield haze, optional droplets. |
+| `sensor` | Exposure, white balance, camera matrix, Bayer mosaic, shot/read noise, fixed-pattern noise, row/column banding, hot/dead pixels, bilinear demosaic. |
+| `isp` | Denoising, color correction, tone mapping, sRGB/gamma, local contrast, sharpening halos, saturation shifts. |
+| `transport` | Crop/resize, bit-depth quantization, JPEG round-trip compression. |
+| `exposure` | Lightweight standalone exposure and white-balance stage for simple custom chains. |
+
+Any stage can define `condition_profiles` to sample coherent per-image settings
+before the stage runs. This is useful for exposure states where ISO, exposure
+gain, read noise, banding, and dark/fog noise modulation should move together:
+
+```json
+{
+  "type": "sensor",
+  "condition_profiles": [
+    {"name": "clean_daylight", "weight": 0.25, "exposure_gain": 1.0, "iso": 100},
+    {"name": "underexposed_noisy", "weight": 0.25, "exposure_gain": 0.65, "iso": 1600}
+  ]
+}
+```
+
 ### Fog Model
 
 The core equation is the **Koschmieder model** (atmospheric scattering):
@@ -224,6 +348,39 @@ When `airlight` is `"dcp_heuristic"`, you can optionally add:
 - `white_bias + cool_bias` must be `<= 1`.
 - The tint bias preserves the estimated airlight luminance, so it shifts colour without silently changing fog density.
 
+### Airlight Intensity Dampening
+
+Estimated airlight is dampened by default as fog density increases. This keeps
+strong fog closer to the low, grey lighting seen in real in-car fog footage
+instead of letting DCP-style estimates wash dense fog toward white.
+
+Each fog model can override the dampening curve:
+
+```json
+"airlight_dampening": {
+  "enabled": true,
+  "apply_to": "estimated",
+  "reference_visibility_m": 80.0,
+  "min_factor": 0.45,
+  "max_factor": 1.0,
+  "strength": 1.0
+}
+```
+
+The factor is:
+`min_factor + (max_factor - min_factor) / (1 + strength * beta / reference_beta)`.
+`reference_beta` is either `reference_scattering_coefficient` /
+`reference_beta`, or it is derived from `reference_visibility_m` using the
+model's contrast threshold. The default applies only when `atmospheric_light`
+uses an estimated airlight method (`"from_sky"`, `"dcp"`, or
+`"dcp_heuristic"`); literal RGB `atmospheric_light` values stay exact unless
+`apply_to` is set to `"all"`. Set `"enabled": false` or `apply_to: "none"` to
+preserve the previous undampened behavior.
+
+For `heterogeneous_ls` and `heterogeneous_k_ls`, the Perlin atmospheric-light
+field is sampled around the dampened base airlight, so the spatial variation
+does not reintroduce the old over-illuminated look.
+
 ### Model Selection
 
 Each image is assigned a fog model via the `selection` block:
@@ -232,10 +389,10 @@ Each image is assigned a fog model via the `selection` block:
 "selection": {
   "mode": "weighted",
   "weights": {
-    "uniform": 1.0,
-    "heterogeneous_k": 0.0,
-    "heterogeneous_ls": 0.0,
-    "heterogeneous_k_ls": 0.0
+    "uniform": 0.25,
+    "heterogeneous_k": 0.35,
+    "heterogeneous_ls": 0.25,
+    "heterogeneous_k_ls": 0.15
   }
 }
 ```
@@ -264,7 +421,10 @@ Each model specifies a `visibility_m` distribution from which a visibility dista
 | `lognormal` | `mean`, `sigma`, optional `min`/`max` | Log-normal. |
 | `choice` | `values`, optional `weights` | Discrete weighted choice. |
 
-The sampled visibility *V* is converted to the attenuation coefficient: **k = -ln(C_t) / V**.
+The sampled visibility *V* is converted to the attenuation coefficient:
+**k = -ln(C_t) / V**. This happens once per output image. For
+`heterogeneous_k` and `heterogeneous_k_ls`, that sampled value is the base
+coefficient that is then spatially modulated by the noise field.
 
 ### Stepped Augmentations
 
@@ -309,9 +469,12 @@ variants:
       "scattering_coefficient": 0.15,
       "atmospheric_light": [1.0, 1.0, 1.0],
       "k_hetero": {
-        "scales": "auto",
-        "min_factor": 0.5,
-        "max_factor": 1.5,
+        "scales": "smooth_auto",
+        "correlation_length_fraction": 0.25,
+        "octaves": 3,
+        "min_factor": 0.65,
+        "max_factor": 1.45,
+        "contrast": 0.65,
         "normalize_to_mean": true
       }
     }
@@ -327,36 +490,79 @@ MOR/beta descriptors when available. euler-loading exposes these as
 
 ### Heterogeneous Noise Fields
 
-Both `k_hetero` and `ls_hetero` use Perlin FBM (fractional Brownian motion) to generate spatially-varying factor fields:
+Both `k_hetero` and `ls_hetero` use Perlin FBM (fractional Brownian
+motion) to generate spatially-varying factor fields. For realistic fog,
+prefer the smooth mode: it keeps Perlin wavelengths tied to the image size,
+then optionally reduces noise contrast and applies a final blur before mapping
+the noise to physical factors.
 
 ```json
 "k_hetero": {
-  "scales": "auto",
-  "min_scale": 2,
+  "scales": "smooth_auto",
+  "correlation_length_fraction": 0.25,
+  "octaves": 3,
   "max_scale": null,
-  "min_factor": 0.0,
-  "max_factor": 1.0,
+  "min_factor": 0.65,
+  "max_factor": 1.45,
+  "contrast": 0.65,
+  "smooth_sigma_fraction": 0.0,
   "normalize_to_mean": true
 }
 ```
 
-The noise field (values in [0, 1]) is mapped to a factor field: `factor(x) = min_factor + (max_factor - min_factor) * noise(x)`. When `normalize_to_mean` is `true`, the factor field is rescaled so its spatial mean equals 1.0, preserving the overall fog density while introducing spatial variation.
+The noise field (values in [0, 1]) is mapped to a factor field:
+`factor(x) = min_factor + (max_factor - min_factor) * noise(x)`.
+`contrast < 1` compresses the noise around 0.5 before this mapping, avoiding
+extreme local fog density. When `normalize_to_mean` is `true`, the factor field
+is rescaled so its spatial mean equals 1.0, preserving the overall fog density
+while introducing spatial variation. In other words, with heterogeneous `k`:
+`k(x) = k_sampled * factor(x)`. If `visibility_m` / MOR was sampled from a
+distribution, `k_sampled` is the coefficient derived from that one sampled MOR.
+With `normalize_to_mean: true`, the arithmetic mean of the per-pixel `k` map
+equals `k_sampled`; the median is not forced to match. With
+`normalize_to_mean: false`, the map mean shifts by the mean of the factor field.
+
+For `heterogeneous_ls` / `heterogeneous_k_ls`, `ls_hetero` can also include a
+weak view-direction illumination prior. This modulates the atmospheric-light
+field itself, so the rendered effect is still gated by fog transmittance:
+
+```json
+"ls_hetero": {
+  "ls_gradient": {
+    "enabled": true,
+    "probability": 0.65,
+    "axis": "vertical",
+    "top_factor": {"dist": "uniform", "min": 1.03, "max": 1.14},
+    "bottom_factor": {"dist": "uniform", "min": 0.88, "max": 0.99},
+    "gamma": {"dist": "uniform", "min": 0.85, "max": 1.6},
+    "normalize_to_mean": true,
+    "fog_opacity_weight": 0.65
+  }
+}
+```
 
 | Parameter | Effect |
 |---|---|
 | `min_factor` / `max_factor` | Range of the multiplicative factor. |
 | `normalize_to_mean` | Rescale factors so the image-wide mean equals the base value. Recommended for `k_hetero`. |
-| `scales` / `min_scale` / `max_scale` | Control spatial frequency content. |
+| `scales: "smooth_auto"` | Build low-frequency Perlin scales from the image size. |
+| `correlation_length_fraction` | Approximate smallest fog feature size as a fraction of the shorter image side. Larger values create smoother gradients. |
+| `octaves` / `lacunarity` / `max_scale` | Control how many increasingly broad Perlin components are mixed. |
+| `contrast` | Compress or expand the Perlin range before mapping to factors. Values below 1 are recommended. |
+| `smooth_sigma` / `smooth_sigma_fraction` | Optional final Gaussian blur in pixels or as a fraction of the shorter image side. |
+| `ls_gradient` | Optional `L_s` top-to-bottom or left-to-right factor field. Keep it weak and probabilistic to avoid a deterministic image-position shortcut. |
 
 ### Fog Output
 
 CLI runs write a source-backed RGB dataset. The output keeps the source RGB
-dataset's relative paths, basenames, extensions, and `output.json` metadata so
-the result stays loadable by `euler-loading`:
+dataset's relative paths, basenames, extensions, and ds-crawler metadata so the
+result stays loadable by `euler-loading`:
 
 ```
 <output_path>/
-  .ds_crawler/output.json
+  .ds_crawler/dataset-head.json
+  .ds_crawler/ds-crawler.json
+  .ds_crawler/index.json
   Scene01/
     Camera_0/
       00000.png
@@ -371,7 +577,9 @@ the source file id instead:
 
 ```
 <output_path>/
-  .ds_crawler/output.json
+  .ds_crawler/dataset-head.json
+  .ds_crawler/ds-crawler.json
+  .ds_crawler/index.json
   Scene01/
     Camera_0/
       00000/
@@ -427,6 +635,6 @@ No special parameters are required. The transform reads intrinsics from the `int
 ### Radial Output
 
 CLI runs write a source-backed depth dataset mirroring the input depth
-modality's layout and writer metadata. The emitted `output.json` also flips
+modality's layout and writer metadata. The emitted `index.json` also flips
 `meta.radial_depth` to `true`. Standalone/direct `RadialTransform(...)` usage
 keeps the legacy `.npy` output behavior.

{euler_preprocess-2.2.0 → euler_preprocess-3.0.0}/README.md

@@ -84,7 +84,7 @@ starting at index 10 with stride 5.
 
 | Transform | `modalities` | `hierarchical_modalities` |
 |---|---|---|
-| `fog` | `rgb`, `depth`, `semantic_segmentation` | — (intrinsics optional) |
+| `fog` | `rgb`, `depth`, `semantic_segmentation` | `intrinsics` when available; used for radial depth conversion and camera-profile optics |
 | `sky-depth` | `depth`, `semantic_segmentation` | — |
 | `radial` | `depth` | `intrinsics` |
 
@@ -136,6 +136,8 @@ Controls the fog simulation.
   "contrast_threshold": 0.05,
   "device": "cpu",
   "gpu_batch_size": 4,
+  "capture": { "preset": "camera" },
+  "camera_profile": "dashcam",
   "augmentations": { ... },
   "selection": { ... },
   "models": { ... }
@@ -151,8 +153,130 @@ Controls the fog simulation.
 | `contrast_threshold` | Threshold *C_t* used in the visibility-to-attenuation conversion (default `0.05`). |
 | `device` | `"cpu"`, `"cuda"`, `"mps"`, or `"gpu"` (alias for cuda). |
 | `gpu_batch_size` | Batch size when running on GPU. Uniform-model samples are batched; heterogeneous samples are processed individually. |
+| `capture` / `capture_artifacts` | Optional post-fog camera artifact pipeline. Omit it or set `{"stages": []}` for the legacy no-op path. Set `true`, `{"preset": "camera"}`, or a custom `stages` list to enable optics, raw sensor, ISP, and compression artifacts. |
+| `camera_profile` | Optional named or inline camera profile whose stage defaults are merged into the capture stack before per-stage overrides. Built-ins are `"default"`, `"generic"`, `"dashcam"`, and `"low_light_fog"`. |
+| `camera_profiles` | Optional map of project-specific named profiles. Use this for calibrated lens, sensor, ISP, and transport settings. |
 | `augmentations` | Optional stepped augmentation set. When present, every input sample produces every configured augmentation and uses the file-id hierarchy output layout described below. |
 
+### Processing Pipeline
+
+Fog generation is split into two phases:
+
+1. Ideal scene rendering: physics-based fog and auxiliary `scattering_coefficient`
+   / `atmospheric_light` maps are computed.
+2. Capture artifacts: camera-specific effects are applied to the rendered RGB
+   only. Physical fog maps stay stable while the RGB output can receive
+   exposure shifts, lens blur, vignetting, raw sensor noise, Bayer/demosaicing
+   artifacts, ISP tone/gamma/sharpening, and JPEG/resize/quantization effects.
+
+This keeps physical fog maps stable while making the RGB output extensible for
+real-camera simulation.
+
+### Capture Artifact Stack
+
+Enable the recommended camera stack with:
+
+```json
+"capture": { "preset": "camera" }
+```
+
+or equivalently:
+
+```json
+"capture": true
+```
+
+For tighter control, provide explicit stages in camera order:
+
+```json
+"camera_profiles": {
+  "real_drive_front": {
+    "optics": {
+      "lens_distortion": -0.012,
+      "vignetting_strength": 0.18,
+      "windshield_haze": {"enabled": true, "probability": 0.55}
+    },
+    "sensor": {
+      "bayer_pattern": "RGGB",
+      "iso": {"dist": "choice", "values": [200, 400, 800]},
+      "base_iso": 100,
+      "full_well_electrons": [14000, 12000, 13000],
+      "read_noise_electrons": {"dist": "uniform", "min": 2.0, "max": 6.0},
+      "black_level": [0.003, 0.0035, 0.003],
+      "white_level": [1.0, 0.995, 1.0],
+      "adc_bit_depth": 12,
+      "post_demosaic_bit_depth": 12
+    },
+    "isp": {
+      "tone_map": "reinhard",
+      "gamma": "srgb",
+      "denoise_sigma": 0.2,
+      "sharpen_amount": 0.2,
+      "saturation": 0.9
+    },
+    "transport": {
+      "jpeg": {"enabled": true, "quality": {"dist": "uniform", "min": 65, "max": 92}},
+      "bit_depth": 8
+    }
+  }
+},
+"camera_profile": "real_drive_front",
+"capture": {
+  "stages": [
+    {
+      "type": "optics",
+      "blur_sigma": {"dist": "uniform", "min": 0.2, "max": 0.8},
+      "vignetting_strength": 0.15,
+      "windshield_haze": {"enabled": true, "probability": 0.4},
+      "droplets": {"enabled": false}
+    },
+    {
+      "type": "sensor",
+      "input_space": "srgb",
+      "exposure_gain": {"dist": "uniform", "min": 0.85, "max": 1.2},
+      "row_noise_sigma": 0.003
+    },
+    {
+      "type": "isp",
+      "tone_map": "reinhard",
+      "gamma": "srgb",
+      "denoise_sigma": 0.2,
+      "sharpen_amount": 0.2,
+      "saturation": 0.9
+    },
+    {
+      "type": "transport",
+      "jpeg": {"enabled": true, "quality": {"dist": "uniform", "min": 65, "max": 92}},
+      "bit_depth": 8
+    }
+  ]
+}
+```
+
+Supported stage types:
+
+| Stage | Main effects |
+|---|---|
+| `optics` | Defocus/MTF blur, motion blur, bloom, veiling glare, vignetting, chromatic aberration, lens distortion, windshield haze, optional droplets. |
+| `sensor` | Exposure, white balance, camera matrix, Bayer mosaic, shot/read noise, fixed-pattern noise, row/column banding, hot/dead pixels, bilinear demosaic. |
+| `isp` | Denoising, color correction, tone mapping, sRGB/gamma, local contrast, sharpening halos, saturation shifts. |
+| `transport` | Crop/resize, bit-depth quantization, JPEG round-trip compression. |
+| `exposure` | Lightweight standalone exposure and white-balance stage for simple custom chains. |
+
+Any stage can define `condition_profiles` to sample coherent per-image settings
+before the stage runs. This is useful for exposure states where ISO, exposure
+gain, read noise, banding, and dark/fog noise modulation should move together:
+
+```json
+{
+  "type": "sensor",
+  "condition_profiles": [
+    {"name": "clean_daylight", "weight": 0.25, "exposure_gain": 1.0, "iso": 100},
+    {"name": "underexposed_noisy", "weight": 0.25, "exposure_gain": 0.65, "iso": 1600}
+  ]
+}
+```
+
 ### Fog Model
 
 The core equation is the **Koschmieder model** (atmospheric scattering):
@@ -210,6 +334,39 @@ When `airlight` is `"dcp_heuristic"`, you can optionally add:
 - `white_bias + cool_bias` must be `<= 1`.
 - The tint bias preserves the estimated airlight luminance, so it shifts colour without silently changing fog density.
 
+### Airlight Intensity Dampening
+
+Estimated airlight is dampened by default as fog density increases. This keeps
+strong fog closer to the low, grey lighting seen in real in-car fog footage
+instead of letting DCP-style estimates wash dense fog toward white.
+
+Each fog model can override the dampening curve:
+
+```json
+"airlight_dampening": {
+  "enabled": true,
+  "apply_to": "estimated",
+  "reference_visibility_m": 80.0,
+  "min_factor": 0.45,
+  "max_factor": 1.0,
+  "strength": 1.0
+}
+```
+
+The factor is:
+`min_factor + (max_factor - min_factor) / (1 + strength * beta / reference_beta)`.
+`reference_beta` is either `reference_scattering_coefficient` /
+`reference_beta`, or it is derived from `reference_visibility_m` using the
+model's contrast threshold. The default applies only when `atmospheric_light`
+uses an estimated airlight method (`"from_sky"`, `"dcp"`, or
+`"dcp_heuristic"`); literal RGB `atmospheric_light` values stay exact unless
+`apply_to` is set to `"all"`. Set `"enabled": false` or `apply_to: "none"` to
+preserve the previous undampened behavior.
+
+For `heterogeneous_ls` and `heterogeneous_k_ls`, the Perlin atmospheric-light
+field is sampled around the dampened base airlight, so the spatial variation
+does not reintroduce the old over-illuminated look.
+
 ### Model Selection
 
 Each image is assigned a fog model via the `selection` block:
@@ -218,10 +375,10 @@ Each image is assigned a fog model via the `selection` block:
 "selection": {
   "mode": "weighted",
   "weights": {
-    "uniform": 1.0,
-    "heterogeneous_k": 0.0,
-    "heterogeneous_ls": 0.0,
-    "heterogeneous_k_ls": 0.0
+    "uniform": 0.25,
+    "heterogeneous_k": 0.35,
+    "heterogeneous_ls": 0.25,
+    "heterogeneous_k_ls": 0.15
   }
 }
 ```
@@ -250,7 +407,10 @@ Each model specifies a `visibility_m` distribution from which a visibility dista
 | `lognormal` | `mean`, `sigma`, optional `min`/`max` | Log-normal. |
 | `choice` | `values`, optional `weights` | Discrete weighted choice. |
 
-The sampled visibility *V* is converted to the attenuation coefficient: **k = -ln(C_t) / V**.
+The sampled visibility *V* is converted to the attenuation coefficient:
+**k = -ln(C_t) / V**. This happens once per output image. For
+`heterogeneous_k` and `heterogeneous_k_ls`, that sampled value is the base
+coefficient that is then spatially modulated by the noise field.
 
 ### Stepped Augmentations
 
@@ -295,9 +455,12 @@ variants:
       "scattering_coefficient": 0.15,
       "atmospheric_light": [1.0, 1.0, 1.0],
       "k_hetero": {
-        "scales": "auto",
-        "min_factor": 0.5,
-        "max_factor": 1.5,
+        "scales": "smooth_auto",
+        "correlation_length_fraction": 0.25,
+        "octaves": 3,
+        "min_factor": 0.65,
+        "max_factor": 1.45,
+        "contrast": 0.65,
         "normalize_to_mean": true
       }
     }
@@ -313,36 +476,79 @@ MOR/beta descriptors when available. euler-loading exposes these as
 
 ### Heterogeneous Noise Fields
 
-Both `k_hetero` and `ls_hetero` use Perlin FBM (fractional Brownian motion) to generate spatially-varying factor fields:
+Both `k_hetero` and `ls_hetero` use Perlin FBM (fractional Brownian
+motion) to generate spatially-varying factor fields. For realistic fog,
+prefer the smooth mode: it keeps Perlin wavelengths tied to the image size,
+then optionally reduces noise contrast and applies a final blur before mapping
+the noise to physical factors.
 
 ```json
 "k_hetero": {
-  "scales": "auto",
-  "min_scale": 2,
+  "scales": "smooth_auto",
+  "correlation_length_fraction": 0.25,
+  "octaves": 3,
   "max_scale": null,
-  "min_factor": 0.0,
-  "max_factor": 1.0,
+  "min_factor": 0.65,
+  "max_factor": 1.45,
+  "contrast": 0.65,
+  "smooth_sigma_fraction": 0.0,
   "normalize_to_mean": true
 }
 ```
 
-The noise field (values in [0, 1]) is mapped to a factor field: `factor(x) = min_factor + (max_factor - min_factor) * noise(x)`. When `normalize_to_mean` is `true`, the factor field is rescaled so its spatial mean equals 1.0, preserving the overall fog density while introducing spatial variation.
+The noise field (values in [0, 1]) is mapped to a factor field:
+`factor(x) = min_factor + (max_factor - min_factor) * noise(x)`.
+`contrast < 1` compresses the noise around 0.5 before this mapping, avoiding
+extreme local fog density. When `normalize_to_mean` is `true`, the factor field
+is rescaled so its spatial mean equals 1.0, preserving the overall fog density
+while introducing spatial variation. In other words, with heterogeneous `k`:
+`k(x) = k_sampled * factor(x)`. If `visibility_m` / MOR was sampled from a
+distribution, `k_sampled` is the coefficient derived from that one sampled MOR.
+With `normalize_to_mean: true`, the arithmetic mean of the per-pixel `k` map
+equals `k_sampled`; the median is not forced to match. With
+`normalize_to_mean: false`, the map mean shifts by the mean of the factor field.
+
+For `heterogeneous_ls` / `heterogeneous_k_ls`, `ls_hetero` can also include a
+weak view-direction illumination prior. This modulates the atmospheric-light
+field itself, so the rendered effect is still gated by fog transmittance:
+
+```json
+"ls_hetero": {
+  "ls_gradient": {
+    "enabled": true,
+    "probability": 0.65,
+    "axis": "vertical",
+    "top_factor": {"dist": "uniform", "min": 1.03, "max": 1.14},
+    "bottom_factor": {"dist": "uniform", "min": 0.88, "max": 0.99},
+    "gamma": {"dist": "uniform", "min": 0.85, "max": 1.6},
+    "normalize_to_mean": true,
+    "fog_opacity_weight": 0.65
+  }
+}
+```
 
 | Parameter | Effect |
 |---|---|
 | `min_factor` / `max_factor` | Range of the multiplicative factor. |
 | `normalize_to_mean` | Rescale factors so the image-wide mean equals the base value. Recommended for `k_hetero`. |
-| `scales` / `min_scale` / `max_scale` | Control spatial frequency content. |
+| `scales: "smooth_auto"` | Build low-frequency Perlin scales from the image size. |
+| `correlation_length_fraction` | Approximate smallest fog feature size as a fraction of the shorter image side. Larger values create smoother gradients. |
+| `octaves` / `lacunarity` / `max_scale` | Control how many increasingly broad Perlin components are mixed. |
+| `contrast` | Compress or expand the Perlin range before mapping to factors. Values below 1 are recommended. |
+| `smooth_sigma` / `smooth_sigma_fraction` | Optional final Gaussian blur in pixels or as a fraction of the shorter image side. |
+| `ls_gradient` | Optional `L_s` top-to-bottom or left-to-right factor field. Keep it weak and probabilistic to avoid a deterministic image-position shortcut. |
 
 ### Fog Output
 
 CLI runs write a source-backed RGB dataset. The output keeps the source RGB
-dataset's relative paths, basenames, extensions, and `output.json` metadata so
-the result stays loadable by `euler-loading`:
+dataset's relative paths, basenames, extensions, and ds-crawler metadata so the
+result stays loadable by `euler-loading`:
 
 ```
 <output_path>/
-  .ds_crawler/output.json
+  .ds_crawler/dataset-head.json
+  .ds_crawler/ds-crawler.json
+  .ds_crawler/index.json
   Scene01/
     Camera_0/
       00000.png
@@ -357,7 +563,9 @@ the source file id instead:
 
 ```
 <output_path>/
-  .ds_crawler/output.json
+  .ds_crawler/dataset-head.json
+  .ds_crawler/ds-crawler.json
+  .ds_crawler/index.json
   Scene01/
     Camera_0/
       00000/
@@ -413,6 +621,6 @@ No special parameters are required. The transform reads intrinsics from the `int
 ### Radial Output
 
 CLI runs write a source-backed depth dataset mirroring the input depth
-modality's layout and writer metadata. The emitted `output.json` also flips
+modality's layout and writer metadata. The emitted `index.json` also flips
 `meta.radial_depth` to `true`. Standalone/direct `RadialTransform(...)` usage
 keeps the legacy `.npy` output behavior.