spectro-kernel 0.2.0__tar.gz → 0.2.1__tar.gz

{spectro_kernel-0.2.0 → spectro_kernel-0.2.1}/CHANGELOG.md

@@ -6,6 +6,86 @@ Until `1.0.0` the public API may change between minor versions.
 
 ## [Unreleased]
 
+## [0.2.1] — 2026-06-09
+
+End-to-end aurora-parity patch. Diagnosed and validated bit-exactly on
+real M42 long-slit data: with these two changes a downstream aurora
+reduction pipeline running entirely on spectro-kernel matches its
+science-core baseline at **Pearson 0.99996** (median |Δ| 0.82 %),
+where v0.2.0 alone scored 0.107 (output silently shifted by +32768 ADU
+on every step).
+
+### Fixed — uint16 ``BZERO``/``BSCALE`` leak in the easyspec staging helpers (BUG)
+
+The shared helper ``stage_target`` in
+[`reduction/_easyspec_apply.py`](src/spectro_kernel/algorithms/reduction/_easyspec_apply.py)
+wrote ``ctx.image.data`` as float64 on a new ``PrimaryHDU`` and copied
+the source header verbatim. When the source frame originated from a
+**uint16 raw file** (``BITPIX=16``, ``BZERO=32768``, ``BSCALE=1`` —
+astropy's convention), the scaling keywords landed on the float64 HDU
+and astropy then re-applied them on read-back, **silently adding 32768
+ADU to every pixel**. The companion ``build_corrected_imageframe``
+re-attached the same scaling block to the corrected ``ImageFrame``, so
+the next pipeline step that re-staged it inherited the leak.
+
+The fix strips a small set of reserved / scaling keywords —
+``SIMPLE, BITPIX, EXTEND, BZERO, BSCALE, BLANK, END, NAXIS*, PCOUNT,
+GCOUNT, XTENSION`` — before they reach either the staged HDU or the
+header re-attached downstream. astropy re-derives the structural keys
+from the data itself; the non-structural metadata (``EXPTIME``,
+``OBSERVER``, etc.) survives.
+
+Affects all four easyspec apply wrappers in one shot:
+``subtract_bias_easyspec``, ``subtract_dark_easyspec``,
+``flat_normalize_easyspec``, ``extract_spectrum_easyspec``. Native-numpy
+algorithms (``dark_subtract``, ``flat_normalize``, ``subtract_sky_2d``,
+the v0.2.0 geometry / denoising suite) were never affected — they
+operate directly on ``ctx.image.data`` without round-tripping through a
+FITS file.
+
+5 regression tests added in
+[`tests/unit/test_easyspec_apply_staging.py`](tests/unit/test_easyspec_apply_staging.py).
+
+### Added — ``extract_spectrum_boxcar`` (literature aperture extraction)
+
+New algorithm in
+[`algorithms/extraction/boxcar.py`](src/spectro_kernel/algorithms/extraction/boxcar.py).
+Same easyspec tracing step as ``extract_spectrum_easyspec``, but
+extraction is a **pure-numpy aperture sum** (clamped to the detector
+bounds on every column).
+
+Why a new brick: ``extract_spectrum_easyspec`` delegates the extraction
+to ``easyspec.extracting``, whose per-column internal sky window
+crashes with ``ValueError: broadcast (W,) vs (S,)`` when the aperture
+half-width is wide (≥ a few hundred px) and the trace sits near a
+detector edge — the typical aurora / extended-source geometry. The new
+brick keeps easyspec's argmax tracing (the trace itself is fine) but
+performs the aperture sum in numpy, with ``max(0, …)`` / ``min(rows,
+…)`` clipping on every column so it is **edge-safe by construction**.
+
+Supports two weightings:
+
+- ``"tophat"`` — unweighted box sum (standard IRAF ``apall``).
+- ``"gaussian"`` — profile-weighted Horne 1986 estimator (the average
+  column profile is fit with ``astropy.modeling`` to derive the
+  weights).
+
+``shift_y_pixels = 0`` disables the background subtraction (pure
+column sum); a positive value samples sky from two symmetric flanking
+windows. References: **Horne 1986, PASP 98, 609** ; **Tody 1986, Proc.
+SPIE 627, 733** (IRAF ``apall`` / ``aptrace``). easyspec is imported
+lazily inside ``run()`` (v0.1.5+ pattern).
+
+6 tests in
+[`tests/unit/test_extract_boxcar.py`](tests/unit/test_extract_boxcar.py),
+including the wide-aperture edge case that motivated the brick.
+
+### Numbers
+
+**98 algorithms** registered (was 97). 254 tests pass (11 new), 3
+skipped. ``ruff check`` clean, ``mkdocs build --strict`` clean,
+``twine check --strict`` clean.
+
 ## [0.2.0] — 2026-06-09
 
 A library expansion focused on long-slit reduction (aurora, stellar,

{spectro_kernel-0.2.0 → spectro_kernel-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: spectro-kernel
-Version: 0.2.0
+Version: 0.2.1
 Summary: A shared catalogue of astronomical spectroscopy algorithms, composable into reproducible pipelines, usable as a Python library, a CLI, or an MCP server.
 Project-URL: Homepage, https://github.com/matthieulel/spectro-kernel
 Project-URL: Documentation, https://matthieulel.github.io/spectro-kernel/

{spectro_kernel-0.2.0 → spectro_kernel-0.2.1}/docs/tutorials/long-slit-reduction.md

@@ -63,8 +63,11 @@ steps:
       band_below_hi: 160
   - algorithm: subtract_sky_2d
     params: {trace_row: 120, trace_half_width: 6, sky_offset: 4, sky_half_width: 10}
-  - algorithm: extract_spectrum_sum
-    params: {trace_row: 120, half_width: 6}
+  # extract_spectrum_boxcar is the edge-safe boxcar for extended sources
+  # (aurora, nebulae). For a stellar point source, extract_spectrum_sum
+  # or extract_spectrum_easyspec are also valid choices.
+  - algorithm: extract_spectrum_boxcar
+    params: {trace_method: "argmax", trace_half_width: 6, extraction_weights: "tophat", shift_y_pixels: 0}
 
   # ── 4. In-situ wavelength calibration from the simultaneously-acquired
   #     sky reference (no arc lamp needed).

spectro_kernel-0.2.1/src/spectro_kernel/algorithms/extraction/boxcar.py

@@ -0,0 +1,335 @@
+"""Algorithm: aperture (boxcar / optimal) extraction of a 1D spectrum.
+
+Pairs with :class:`ExtractSpectrumEasyspec` — same tracing step, different
+**extraction**. Where ``extract_spectrum_easyspec`` delegates the
+extraction to ``easyspec.extracting`` (whose internal per-column sky
+window crashes with ``ValueError: broadcast (W,) vs (S,)`` for wide
+apertures near a detector edge), this brick keeps easyspec's argmax
+tracing but extracts with a pure-numpy aperture sum that is **edge-safe
+by construction** (``max(0, …)`` / ``min(rows, …)`` clipping on every
+column).
+
+The right tool for **extended sources filling the slit** — aurora,
+nebulae, the Sun's photospheric profile in a daylight slit shot —
+where the aperture half-width is large (≥ 100 px) and the trace sits
+within a few hundred rows of either edge.
+
+The wavelength axis is the pixel index (calibrate downstream with
+``wavelength_calibrate_polynomial`` /
+``wavelength_calibration_in_situ`` / ``wavelength_calibration_solar``).
+"""
+
+from __future__ import annotations
+
+import contextlib
+import io
+from typing import Any
+
+import numpy as np
+
+from ...base import AlgorithmOutput, BaseAlgorithm
+from ...errors import InvalidParameterError
+from ...registry import register_algorithm
+from ...types import AlgorithmCategory, Spectrum1D, WorkContext
+
+
+def _boxcar_extract(
+    image: np.ndarray,
+    trace_coefficients: np.ndarray,
+    trace_half_width: int,
+    extraction_weights: str,
+    shift_y_pixels: int,
+) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
+    """Aperture (boxcar / Horne) extraction around a per-column polynomial trace.
+
+    Pure-numpy and edge-safe: every per-column slice is clamped to
+    ``[0, image.shape[0]]``, so apertures wider than the available row
+    span do not raise — they just integrate fewer rows there.
+
+    ``shift_y_pixels == 0`` disables the background subtraction (the
+    flanking windows collapse to length 0 and contribute zero).
+
+    Parameters
+    ----------
+    image
+        2D detector frame, rows × columns.
+    trace_coefficients
+        Polynomial coefficients of the trace y(x), highest order first
+        (``numpy.poly1d`` convention).
+    trace_half_width
+        Aperture half-width in rows.
+    extraction_weights
+        ``"tophat"`` for the unweighted box sum (the standard IRAF
+        ``apall`` recipe) or ``"gaussian"`` for the profile-weighted
+        Horne 1986 estimator.
+    shift_y_pixels
+        Total height of the sky window flanking the aperture (above and
+        below). ``0`` disables sky subtraction.
+
+    Returns
+    -------
+    (flux, flux_error, sky)
+        Three 1D arrays of length ``image.shape[1]``.
+
+    References
+    ----------
+    - Horne 1986, PASP 98, 609 — optimal extraction (the tophat is the
+      standard unweighted variant).
+    - IRAF ``apall`` / ``aptrace`` (Tody 1986, SPIE 627, 733).
+    """
+    from astropy.modeling.fitting import LevMarLSQFitter  # noqa: PLC0415
+    from astropy.modeling.models import Gaussian1D  # noqa: PLC0415
+
+    nx = image.shape[1]
+    nrows = image.shape[0]
+    xvals = np.arange(nx)
+    trace_center = np.polyval(trace_coefficients, xvals)
+    median_trace = np.median(trace_center)
+    ymin = int(median_trace - shift_y_pixels)
+    ymax = int(median_trace + shift_y_pixels)
+
+    # Per-column background from the two windows flanking the aperture.
+    background1 = np.zeros(nx, dtype=np.float64)
+    background2 = np.zeros(nx, dtype=np.float64)
+    for i in range(nx):
+        y_trace = int(trace_center[i])
+        # Edge-safe slices: clamp to detector bounds.
+        upper_lo = max(0, min(nrows, y_trace + trace_half_width))
+        upper_hi = max(0, min(nrows, ymax))
+        lower_lo = max(0, min(nrows, ymin))
+        lower_hi = max(0, min(nrows, y_trace - trace_half_width))
+        bg1 = image[upper_lo:upper_hi, i]
+        bg2 = image[lower_lo:lower_hi, i]
+        background1[i] = float(np.median(bg1)) if bg1.size > 0 else 0.0
+        background2[i] = float(np.median(bg2)) if bg2.size > 0 else 0.0
+    background = 0.5 * (background1 + background2)
+
+    full_height = 2 * trace_half_width
+    if extraction_weights == "gaussian":
+        # Build the average column profile from cutouts that span the
+        # full ±trace_half_width window (drop edge cuts).
+        full_cutouts: list[np.ndarray] = []
+        for i, y_trace_f in enumerate(trace_center):
+            y_trace = int(y_trace_f)
+            y_lo = y_trace - trace_half_width
+            y_hi = y_trace + trace_half_width
+            if 0 <= y_lo and y_hi <= nrows:
+                full_cutouts.append(image[y_lo:y_hi, i] - background[i])
+        if not full_cutouts:
+            raise InvalidParameterError(
+                "extraction_weights='gaussian' needs at least one column where "
+                "the full ±trace_half_width window fits inside the detector."
+            )
+        mean_profile = np.mean(np.asarray(full_cutouts), axis=0)
+        profile_x = np.arange(-trace_half_width, trace_half_width, dtype=np.float64)
+        fitter = LevMarLSQFitter(calc_uncertainties=True)
+        guess = Gaussian1D(
+            amplitude=float(np.max(mean_profile)),
+            mean=0.0,
+            stddev=float(trace_half_width) / 2.0,
+        )
+        with contextlib.suppress(Exception):
+            guess = fitter(guess, profile_x, mean_profile)
+        weights = np.asarray(guess(profile_x), dtype=np.float64)
+        weights_sum = float(np.sum(weights))
+        if weights_sum > 0.0:
+            weights = weights / weights_sum
+        else:
+            # Degenerate fit → fall back to tophat for robustness.
+            weights = np.ones(full_height, dtype=np.float64) / float(full_height)
+    elif extraction_weights == "tophat":
+        weights = np.ones(full_height, dtype=np.float64) / float(full_height)
+    else:
+        raise InvalidParameterError(
+            f"extraction_weights must be 'tophat' or 'gaussian', got "
+            f"{extraction_weights!r}."
+        )
+
+    flux = np.zeros(nx, dtype=np.float64)
+    flux_error = np.zeros(nx, dtype=np.float64)
+    weights_sq_sum = float(np.sum(weights ** 2)) or 1.0
+    for i, y_trace_f in enumerate(trace_center):
+        y_trace = int(y_trace_f)
+        y_lo = max(0, y_trace - trace_half_width)
+        y_hi = min(nrows, y_trace + trace_half_width)
+        cutout = image[y_lo:y_hi, i]
+        cutout_nobg = cutout - background[i]
+        if cutout_nobg.size == weights.size:
+            flux[i] = float(np.sum(cutout_nobg * weights) / weights_sq_sum)
+            flux_error[i] = float(
+                np.sqrt(np.sum(np.abs(cutout) * weights ** 2)) / weights_sq_sum
+            )
+        else:
+            # Edge column: fall back to an unweighted box sum on the
+            # available rows. Reports plain Poisson-like error.
+            flux[i] = float(np.sum(cutout_nobg))
+            flux_error[i] = float(np.sqrt(np.sum(np.abs(cutout))))
+    return flux, flux_error, background
+
+
+@register_algorithm(
+    "extract_spectrum_boxcar",
+    category=AlgorithmCategory.EXTRACTION,
+    version="1.0.0",
+)
+class ExtractSpectrumBoxcar(BaseAlgorithm):
+    """Trace (easyspec) + pure-numpy aperture extraction on ``ctx.image``.
+
+    Tracing reuses ``easyspec.extraction.extraction().tracing`` (argmax
+    or moments per column, polynomial fit across columns) — the trace is
+    not the issue. **Extraction** is an edge-safe numpy aperture sum
+    around ``±trace_half_width`` of the per-column trace position, with
+    optional background subtraction from two flanking windows.
+
+    Choose this over ``extract_spectrum_easyspec`` when the source fills
+    much of the slit (aurora, nebulae) — easyspec's internal sky window
+    crashes for wide apertures near a detector edge. With
+    ``shift_y_pixels = 0`` the background is zero and the algorithm
+    reduces to a pure column sum (the standard IRAF ``apall`` recipe).
+
+    The wavelength axis is the pixel index; calibrate downstream.
+    """
+
+    backend = "easyspec"
+    references = [
+        "Horne 1986, PASP 98, 609 — optimal aperture extraction "
+        "(tophat is the standard unweighted variant).",
+        "Tody 1986, Proc. SPIE 627, 733 — IRAF apall / aptrace heritage.",
+        "easyspec.extraction.extraction.tracing — argmax / moments trace fit.",
+    ]
+    long_description = (
+        "Tracing uses easyspec.tracing (argmax → polynomial). Extraction "
+        "is a numpy aperture sum over ±trace_half_width around the "
+        "per-column trace position, fully edge-safe (any partial slice "
+        "outside the detector contributes zero). tophat ⇒ standard "
+        "boxcar; gaussian ⇒ profile-weighted Horne 1986. easyspec is "
+        "imported lazily inside run()."
+    )
+    default_params: dict[str, Any] = {
+        "trace_method": "argmax",
+        "trace_poly_order": 2,
+        "trace_y_pixel_range": 15,
+        "trace_peak_height": 100.0,
+        "trace_peak_distance": 50,
+        "trace_half_width": 7,
+        "extraction_weights": "tophat",
+        "shift_y_pixels": 0,
+    }
+    param_descriptions = {
+        "trace_method": "easyspec trace method: 'argmax', 'moments' or 'multi'.",
+        "trace_poly_order": "Polynomial order of the trace fit.",
+        "trace_y_pixel_range": "Half-window (rows) for the trace search.",
+        "trace_peak_height": "Minimum peak height when locating the trace.",
+        "trace_peak_distance": "Minimum separation between traces (multi mode).",
+        "trace_half_width": "Aperture half-width (rows) for both trace and extraction.",
+        "extraction_weights": "'tophat' (boxcar sum) or 'gaussian' (Horne-weighted).",
+        "shift_y_pixels": (
+            "Half-height of the sky window flanking the aperture (rows). "
+            "0 disables background subtraction."
+        ),
+    }
+    input_requirements = ["image"]
+    output_produces = ["spectrum"]
+
+    def run(self, ctx: WorkContext, params: dict[str, Any]) -> AlgorithmOutput:
+        if ctx.image is None:
+            raise InvalidParameterError("extract_spectrum_boxcar needs ctx.image.")
+
+        try:
+            import matplotlib  # noqa: PLC0415
+
+            matplotlib.use("Agg")
+        except ImportError:  # pragma: no cover
+            pass
+
+        try:
+            from easyspec.extraction import extraction as _EasyExtraction  # noqa: PLC0415
+        except ImportError:
+            return AlgorithmOutput.fail(
+                "extract_spectrum_boxcar needs easyspec for the tracing step. "
+                "Install with: pip install 'spectro-kernel[reduction]'."
+            )
+
+        image = np.asarray(ctx.image.data, dtype=np.float64)
+        if image.ndim != 2:
+            raise InvalidParameterError(
+                "extract_spectrum_boxcar requires a 2D detector frame."
+            )
+
+        thw = int(params["trace_half_width"])
+        if thw <= 0:
+            raise InvalidParameterError("trace_half_width must be a positive integer.")
+        poly_order = int(params["trace_poly_order"])
+        if poly_order < 1:
+            raise InvalidParameterError("trace_poly_order must be ≥ 1.")
+        ew = str(params["extraction_weights"])
+        if ew not in ("tophat", "gaussian"):
+            raise InvalidParameterError("extraction_weights must be 'tophat' or 'gaussian'.")
+
+        # ── Tracing (easyspec) ──────────────────────────────────────────
+        extractor = _EasyExtraction()
+        extractor.aspect_ratio = image.shape[0] / image.shape[1]
+        with contextlib.redirect_stdout(io.StringIO()):
+            polymodels = extractor.tracing(
+                target_spec_data=image,
+                method=str(params["trace_method"]),
+                y_pixel_range=int(params["trace_y_pixel_range"]),
+                xlims=None,
+                poly_order=poly_order,
+                trace_half_width=thw,
+                peak_height=float(params["trace_peak_height"]),
+                distance=int(params["trace_peak_distance"]),
+                main_plot=False,
+                plot_residuals=False,
+            )
+        if not polymodels:
+            return AlgorithmOutput.fail(
+                "No spectral trace detected — lower trace_peak_height or "
+                "check the image is properly oriented (spectral axis along columns)."
+            )
+        # easyspec returns coefficients lowest-order first as
+        # polymodel.cN attributes; numpy.poly1d wants highest first.
+        coeffs = np.asarray(
+            [getattr(polymodels[0], f"c{i}").value for i in range(poly_order + 1)][::-1],
+            dtype=np.float64,
+        )
+
+        # ── Extraction (numpy, edge-safe) ──────────────────────────────
+        flux, flux_error, sky = _boxcar_extract(
+            image,
+            coeffs,
+            thw,
+            ew,
+            int(params["shift_y_pixels"]),
+        )
+
+        ctx.spectrum = Spectrum1D(
+            wavelength=np.arange(flux.size, dtype=np.float64),
+            flux=flux,
+            uncertainty=flux_error,
+            wavelength_unit="pixel",
+            flux_unit="counts",
+            meta={
+                **ctx.image.meta,
+                "extraction": "boxcar",
+                "extraction_weights": ew,
+                "aperture_half_width": thw,
+                "sky_median": float(np.median(sky)),
+                "needs_wavelength_calibration": True,
+            },
+            header=dict(ctx.image.header),
+        )
+        return AlgorithmOutput.ok(
+            metrics={
+                "npix": float(flux.size),
+                "flux_median": float(np.median(flux)),
+                "sky_median": float(np.median(sky)),
+            },
+            message=(
+                f"Boxcar-extracted {flux.size}-pixel spectrum "
+                f"({ew} weights, aperture ±{thw} px)."
+            ),
+        )
+
+
+__all__ = ["ExtractSpectrumBoxcar", "_boxcar_extract"]

{spectro_kernel-0.2.0 → spectro_kernel-0.2.1}/src/spectro_kernel/algorithms/reduction/_easyspec_apply.py

@@ -19,6 +19,30 @@ from astropy.io import fits
 from ...errors import InvalidParameterError
 from ...types import ImageFrame, WorkContext
 
+# Reserved / structural FITS keywords that astropy derives from the data
+# itself when serialising a PrimaryHDU. Copying them off an integer source
+# header onto a float64 HDU corrupts the round-trip: in particular,
+# ``BZERO=32768, BSCALE=1`` from a uint16 raw frame triggers astropy to
+# re-apply the +32768 offset on read-back, which silently shifts every
+# pixel by 32768 ADU. The same applies to any header re-attached to the
+# corrected ImageFrame downstream — strip there too so the next step's
+# stage_target does not inherit the leak.
+_STRUCTURAL_KEYWORDS: frozenset[str] = frozenset(
+    {
+        "SIMPLE", "BITPIX", "EXTEND", "BZERO", "BSCALE", "BLANK", "END",
+        "NAXIS", "NAXIS1", "NAXIS2", "NAXIS3", "PCOUNT", "GCOUNT", "XTENSION",
+    }
+)
+
+
+def _filter_structural(header: dict | fits.Header) -> dict:
+    """Return a copy of *header* with structural / scaling keywords stripped."""
+    return {
+        str(key): value
+        for key, value in dict(header).items()
+        if str(key).upper() not in _STRUCTURAL_KEYWORDS
+    }
+
 
 def stage_target(stage: str, target_path: str | None, ctx_image: ImageFrame | None) -> str:
     """Return a directory containing the single target FITS to process.
@@ -27,6 +51,11 @@ def stage_target(stage: str, target_path: str | None, ctx_image: ImageFrame | No
     no copy). Otherwise the in-memory ``ctx.image`` is written to a temporary
     FITS in the same directory. Either way the returned directory is what
     EasySpec's ``data_paths(targets=...)`` expects.
+
+    Scaling/structural keywords (``BZERO``, ``BSCALE``, ``BITPIX``, …) from
+    the source header are stripped before being attached to the staged
+    float64 HDU — otherwise astropy re-applies the uint16 convention on
+    read-back and silently adds 32768 to every pixel.
     """
     target_dir = Path(stage) / "targets"
     target_dir.mkdir(parents=True, exist_ok=True)
@@ -40,7 +69,7 @@ def stage_target(stage: str, target_path: str | None, ctx_image: ImageFrame | No
         raise InvalidParameterError("Provide either target_path or have ctx.image loaded.")
     hdu = fits.PrimaryHDU(np.asarray(ctx_image.data, dtype=np.float64))
     if ctx_image.header:
-        for key, value in ctx_image.header.items():
+        for key, value in _filter_structural(ctx_image.header).items():
             with contextlib.suppress(Exception):
                 hdu.header[str(key)[:8]] = value
     hdu.writeto(target_dir / "target.fits", overwrite=True)
@@ -82,12 +111,19 @@ def first_value(result_dict: dict) -> np.ndarray:
 def build_corrected_imageframe(
     array: np.ndarray, header_source: str, step: str, extra_meta: dict | None = None
 ) -> ImageFrame:
-    """Wrap an easyspec-returned array back into an :class:`ImageFrame`."""
+    """Wrap an easyspec-returned array back into an :class:`ImageFrame`.
+
+    The header read back from disk is stripped of structural/scaling
+    keywords (``BZERO``, ``BSCALE``, ``BITPIX``, …) before being attached
+    to the new ImageFrame. Otherwise the next pipeline step that calls
+    ``stage_target`` on this frame would inherit the leak and re-apply
+    the uint16 offset.
+    """
     meta = {"step": step, "backend": "easyspec"}
     if extra_meta:
         meta.update(extra_meta)
     try:
-        header = dict(fits.getheader(header_source))
+        header = _filter_structural(fits.getheader(header_source))
     except Exception:  # noqa: BLE001 - missing header isn't fatal
         header = {}
     return ImageFrame(data=array, header=header, frame_type="science", meta=meta)

{spectro_kernel-0.2.0 → spectro_kernel-0.2.1}/src/spectro_kernel/version.py

@@ -1,3 +1,3 @@
 """Single source of truth for the package version."""
 
-__version__ = "0.2.0"
+__version__ = "0.2.1"

spectro_kernel-0.2.1/tests/unit/test_easyspec_apply_staging.py

@@ -0,0 +1,156 @@
+"""Regression tests for the easyspec staging helpers.
+
+Guards against the v0.2.0 uint16 BZERO/BSCALE leak (fix #1 of the
+2026-06-09 aurora-parity audit). The bug:
+
+  stage_target writes ctx.image.data as float64 on a PrimaryHDU but
+  copies the source header verbatim — including BZERO=32768/BSCALE=1
+  from a uint16 raw frame. astropy then re-applies that scaling on the
+  next read, silently adding 32768 ADU to every pixel.
+
+The fix strips structural/scaling keywords before they touch the HDU.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import numpy as np
+from astropy.io import fits
+
+from spectro_kernel.algorithms.reduction._easyspec_apply import (
+    _STRUCTURAL_KEYWORDS,
+    _filter_structural,
+    build_corrected_imageframe,
+    stage_target,
+)
+from spectro_kernel.types import ImageFrame
+
+
+def _staged_fits(tmp_path: Path) -> Path:
+    """First .fits file under <tmp_path>/targets — what stage_target produces."""
+    target_dir = Path(tmp_path) / "targets"
+    files = list(target_dir.glob("*.fits"))
+    assert files, f"stage_target produced no FITS in {target_dir}"
+    return files[0]
+
+
+def test_filter_structural_drops_scaling_and_structural_keys():
+    raw = {
+        "BZERO": 32768,
+        "BSCALE": 1,
+        "BITPIX": 16,
+        "NAXIS": 2,
+        "NAXIS1": 100,
+        "NAXIS2": 50,
+        "EXPTIME": 60.0,
+        "OBSERVER": "Skibotn",
+    }
+    cleaned = _filter_structural(raw)
+    assert "EXPTIME" in cleaned and cleaned["EXPTIME"] == 60.0
+    assert "OBSERVER" in cleaned
+    for key in _STRUCTURAL_KEYWORDS & raw.keys():
+        assert key not in cleaned
+
+
+def test_stage_target_strips_uint16_bzero(tmp_path):
+    """The headline regression: a uint16-style header must not leak +32768."""
+    truth = np.full((8, 16), 170.0, dtype=np.float64)
+    img = ImageFrame(
+        data=truth,
+        header={
+            # Exactly what astropy attaches when reading a uint16 raw frame.
+            "BZERO": 32768,
+            "BSCALE": 1,
+            "BITPIX": 16,
+            "NAXIS": 2,
+            "EXPTIME": 60.0,
+            "OBSERVER": "test",
+        },
+    )
+    stage_target(str(tmp_path), None, img)
+    staged = _staged_fits(tmp_path)
+    with fits.open(staged, memmap=False) as hdul:
+        back = np.asarray(hdul[0].data, dtype=np.float64)
+        staged_hdr = dict(hdul[0].header)
+
+    # Bit-exact round-trip — was truth + 32768 before the fix.
+    np.testing.assert_array_equal(back, truth)
+    # The staged HDU is float64; astropy does not attach BZERO/BSCALE
+    # for float arrays. Either the keys are absent, or they take the
+    # neutral defaults — never the leaked +32768/1 combo.
+    assert staged_hdr.get("BZERO", 0.0) == 0.0
+    assert staged_hdr.get("BSCALE", 1.0) == 1.0
+    assert staged_hdr.get("BITPIX") == -64  # float64
+    # Non-structural keys survive the filter.
+    assert staged_hdr.get("EXPTIME") == 60.0
+
+
+def test_stage_target_with_explicit_path_symlinks_unchanged(tmp_path):
+    """When target_path is supplied, the file is symlinked unchanged."""
+    src = tmp_path / "raw.fits"
+    truth = np.full((4, 4), 7.5, dtype=np.float32)
+    fits.PrimaryHDU(truth).writeto(src)
+    stage_target(str(tmp_path), str(src), ctx_image=None)
+    target_dir = tmp_path / "targets"
+    linked = target_dir / "raw.fits"
+    assert linked.is_symlink()
+    np.testing.assert_array_equal(
+        np.asarray(fits.getdata(linked)), truth.astype(np.float64),
+    )
+
+
+def test_build_corrected_imageframe_strips_structural(tmp_path):
+    """The ImageFrame re-attached after an easyspec step must not carry BZERO."""
+    src = tmp_path / "after_easyspec.fits"
+    data = np.full((4, 4), 100.0, dtype=np.float64)
+    hdu = fits.PrimaryHDU(data)
+    # Manually inject a uint16-style scaling block + a benign keyword.
+    hdu.header["BZERO"] = 32768
+    hdu.header["BSCALE"] = 1
+    hdu.header["OBSERVER"] = "Skibotn"
+    hdu.writeto(src)
+    frame = build_corrected_imageframe(
+        np.full((4, 4), 170.0, dtype=np.float64),
+        header_source=str(src),
+        step="subtract_bias",
+        extra_meta={"backend_extra": "ok"},
+    )
+    # Header on the returned frame is filtered.
+    assert "BZERO" not in frame.header
+    assert "BSCALE" not in frame.header
+    assert frame.header.get("OBSERVER") == "Skibotn"
+    # Meta carries the step + extra info.
+    assert frame.meta["step"] == "subtract_bias"
+    assert frame.meta["backend"] == "easyspec"
+    assert frame.meta["backend_extra"] == "ok"
+
+
+def test_stage_then_corrected_chain_is_leak_free(tmp_path):
+    """End-to-end: a raw uint16-headered frame goes through stage→correct→stage
+    and the second staging must NOT re-introduce the +32768 offset."""
+    truth = np.full((6, 6), 170.0, dtype=np.float64)
+    raw_header = {"BZERO": 32768, "BSCALE": 1, "BITPIX": 16, "OBSERVER": "loop"}
+    img = ImageFrame(data=truth, header=raw_header)
+
+    # Step 1: stage the raw frame.
+    stage_target(str(tmp_path), None, img)
+    staged1 = _staged_fits(tmp_path)
+
+    # Step 2: build a corrected ImageFrame from the staged file (this is
+    # what build_corrected_imageframe does after an easyspec call).
+    corrected = build_corrected_imageframe(
+        np.asarray(fits.getdata(staged1), dtype=np.float64),
+        header_source=str(staged1),
+        step="round-trip",
+    )
+
+    # Step 3: stage the corrected frame again — simulates the next pipeline
+    # step. Header must NOT carry BZERO/BSCALE so the next round-trip stays
+    # leak-free.
+    stage_root_2 = tmp_path / "second"
+    stage_root_2.mkdir()
+    stage_target(str(stage_root_2), None, corrected)
+    staged2 = _staged_fits(stage_root_2)
+    back = np.asarray(fits.getdata(staged2), dtype=np.float64)
+    np.testing.assert_array_equal(back, truth)