pyreduce-astro 0.7a6__tar.gz → 0.7b1__tar.gz

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/CHANGELOG.md

@@ -1,6 +1,37 @@
 # Changelog
 
 
+## [0.7b1] - 2026-01-06
+
+### Changed
+- **Curvature algorithm rewrite**: Replace 2D model fitting with row-tracking method for better robustness
+- Rename `make_shear.py` to `slit_curve.py`
+- Rename curvature coefficients: `tilt`/`shear` → `p1`/`p2` throughout codebase
+- Rename `arc_extraction` to `simple_extraction`
+- Split curvature `extraction_height` into separate `extraction_height` and `curve_height` parameters
+- Save file renamed from `.shear.npz` to `.curve.npz`
+
+### Added
+- `discover_channels()` for automatic channel detection from data files
+- CLI: `--target` is now optional; loops over all targets if not specified
+- CLI: Uses `$REDUCE_DATA` for base_dir, reads default input_dir from config
+- Comprehensive CLI test coverage
+
+### Fixed
+- Curvature step index error when traces are removed during processing
+- Validate base_dir and input_dir exist with clear error messages
+- File sorting to correctly loop over all nights when not specified
+- CLI dynamic dependency loading for step commands
+
+## [0.7a7] - 2026-01-04
+
+### Added
+- `--settings` option for CLI to override reduction parameters from JSON file
+
+### Changed
+- Rename `extraction_width` to `extraction_height` in settings (clarifies coordinate system)
+- Rename `orders` to `traces` in extract.py internal API
+
 ## [0.7a6] - 2026-01-03
 
 ### Added

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/CLAUDE.md

@@ -54,6 +54,16 @@ pyreduce/
     └── slit_func_2d_xi_zeta_bd.c
 ```
 
+## Image Coordinate Convention
+
+PyReduce uses the convention that **dispersion runs horizontally (along x-axis)** and **cross-dispersion runs vertically (along y-axis)**. The `clipnflip()` function in `instruments/common.py` rotates and flips raw images to ensure this orientation.
+
+This means:
+- **Columns (x)** = wavelength/dispersion direction
+- **Rows (y)** = spatial/cross-dispersion direction
+- **Traces** are polynomial functions of x, giving y-position
+- **`extraction_height`** refers to pixels above/below each trace (in y)
+
 ## Pipeline Steps
 
 The reduction pipeline consists of these steps (in typical order):
@@ -64,7 +74,7 @@ The reduction pipeline consists of these steps (in typical order):
 | `bias` | `Bias` | Combine bias frames into master bias |
 | `flat` | `Flat` | Combine flat frames, subtract bias |
 | `trace` | `OrderTracing` | Trace echelle order positions on flat |
-| `curvature` | `SlitCurvatureDetermination` | Measure slit tilt/shear from arc lamp |
+| `curvature` | `SlitCurvatureDetermination` | Measure slit curvature from arc lamp |
 | `scatter` | `BackgroundScatter` | Model inter-order scattered light |
 | `norm_flat` | `NormalizeFlatField` | Normalize flat, extract blaze function |
 | `wavecal_master` | `WavelengthCalibrationMaster` | Extract wavelength calibration spectrum |
@@ -145,7 +155,7 @@ Defines HOW to reduce - algorithm parameters per step:
     "filter_size": 120
   },
   "norm_flat": {
-    "extraction_width": 0.5,
+    "extraction_height": 0.5,
     "smooth_slitfunction": 1,
     "oversampling": 10
   },
@@ -156,7 +166,7 @@ Defines HOW to reduce - algorithm parameters per step:
   },
   "science": {
     "extraction_method": "optimal",
-    "extraction_width": 0.5,
+    "extraction_height": 0.5,
     "oversampling": 10
   }
 }
@@ -250,12 +260,16 @@ uv run reduce list-steps
 
 ```bash
 uv sync                              # Install dependencies
+uv run reduce-build                  # Compile C extensions
+uv run reduce-clean                  # Remove compiled extensions
 uv run pre-commit install            # Setup hooks (once)
 uv run pytest -m unit                # Fast unit tests
 uv run pytest --instrument=UVES      # Test single instrument
 uv run ruff check --fix .            # Lint and fix
 ```
 
+After a fresh clone or `rm -rf .venv`, run `uv sync && uv run reduce-build` to set up.
+
 ### Adding Instruments
 
 1. Create `pyreduce/instruments/name.yaml` with detector/header config
@@ -283,3 +297,43 @@ uv run ruff check --fix .            # Lint and fix
 | `pyreduce/instruments/models.py` | Pydantic config models |
 | `pyreduce/clib/*.c` | C code for slit function decomposition |
 | `hatch_build.py` | CFFI extension build hook |
+
+## Release Process
+
+To release a new version (e.g., `0.7a6`):
+
+1. **Update documentation** for any renamed steps, new CLI options, etc:
+   - `README.md` - Quick start examples
+   - `docs/cli.md` - CLI reference
+   - `docs/index.md`, `docs/howto.md`, `docs/examples.md` - Usage examples
+   - `docs/configuration_file.md` - Config key names
+
+2. **Update CHANGELOG.md** with release date and changes
+
+3. **Update version** in `pyproject.toml`
+
+3a. **sync** - run `uv sync` to get the new version into uv.lock
+
+4. **Update example headers** - all `examples/*.py` have PEP 723 metadata:
+   ```python
+   # /// script
+   # dependencies = ["pyreduce-astro>=0.7a6"]
+   # ///
+   ```
+
+5. **Commit, tag, and push**:
+   ```bash
+   git add -A && git commit -m "Release v0.7a6"
+   git tag v0.7a6
+   git push && git push --tags
+   ```
+
+6. **Monitor GitHub Actions** - the tag push triggers:
+   - Tests on Python 3.13
+   - Wheel builds (Linux, Windows, macOS)
+   - PyPI upload
+   - GitHub Release creation
+
+   ```bash
+   gh run watch  # watch the triggered workflow
+   ```

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyreduce-astro
-Version: 0.7a6
+Version: 0.7b1
 Summary: A data reduction package for echelle spectrographs
 Project-URL: Homepage, https://github.com/ivh/PyReduce
 Project-URL: Documentation, https://pyreduce-astro.readthedocs.io
@@ -37,7 +37,6 @@ Requires-Dist: scikit-image>=0.22.0
 Requires-Dist: scipy>=1.11.0
 Requires-Dist: spectres>=2.2.0
 Requires-Dist: tqdm>=4.66.0
-Requires-Dist: wget>=3.2
 Description-Content-Type: text/markdown
 
 [![CI](https://github.com/ivh/PyReduce/actions/workflows/python-publish.yml/badge.svg)](https://github.com/ivh/PyReduce/actions/workflows/python-publish.yml)
@@ -101,4 +100,4 @@ Full documentation at [ReadTheDocs](https://pyreduce-astro.readthedocs.io/).
 
 ## Output
 
-PyReduce creates `.ech` files (standard FITS with binary table extension). Headers include original keywords plus PyReduce-specific ones prefixed with `e_`.
+PyReduce creates `.fits` files (standard FITS with binary table extension). Headers include original keywords plus PyReduce-specific ones prefixed with `e_`.

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/README.md

@@ -59,4 +59,4 @@ Full documentation at [ReadTheDocs](https://pyreduce-astro.readthedocs.io/).
 
 ## Output
 
-PyReduce creates `.ech` files (standard FITS with binary table extension). Headers include original keywords plus PyReduce-specific ones prefixed with `e_`.
+PyReduce creates `.fits` files (standard FITS with binary table extension). Headers include original keywords plus PyReduce-specific ones prefixed with `e_`.

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/aj_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 AJ instrument example: Fiber bundle tracing with direct function calls.
@@ -26,6 +26,7 @@ import os
 
 import numpy as np
 
+from pyreduce.extract import extract
 from pyreduce.instruments.instrument_info import load_instrument
 from pyreduce.trace import group_and_refit, merge_traces, trace
 
@@ -41,6 +42,7 @@ orders_file = os.path.join(raw_dir, "ANDES_75fibre_J_orders.npz")
 # Output
 output_dir = os.path.expanduser("~/REDUCE_DATA/AJ/reduced")
 output_file = os.path.join(output_dir, "fiber_traces.npz")
+output_blaze = os.path.join(output_dir, "fiber_blaze.npz")
 
 # Load order centers from npz file
 orders_data = np.load(orders_file)
@@ -65,7 +67,7 @@ trace_params = {
     "border_width": 0,
     "manual": False,
     "auto_merge_threshold": 1.0,
-    "merge_min_threshold": 0.1,
+    "merge_min_threshold": 1.0,  # 1.0 disables merging entirely
     "sigma": 0,
     "plot": 1,
 }
@@ -90,6 +92,10 @@ print(f"  Shape: {img_even.shape}, dtype: {img_even.dtype}")
 print(f"Loading {file_odd}...")
 img_odd, head_odd = instrument.load_fits(file_odd, channel=channel, extension=0)
 
+# Combine even and odd flats for extraction
+img_combined = img_even.astype(np.float64) + img_odd.astype(np.float64)
+print(f"Combined flat shape: {img_combined.shape}")
+
 # --- Step 2: Trace each flat independently ---
 print("\nTracing even-illuminated fibers...")
 traces_even, cr_even = trace(
@@ -166,4 +172,38 @@ else:
     save_dict["column_range"] = all_cr
 
 np.savez(output_file, **save_dict)
-print(f"\nSaved to: {output_file}")
+print(f"\nSaved traces to: {output_file}")
+
+# --- Step 6: Extract spectra from all fiber traces ---
+# Flatten all traces into a single orders array for extraction
+all_traces = np.vstack([traces_by_order[k] for k in sorted(traces_by_order.keys())])
+all_cr = np.vstack([cr_by_order[k] for k in sorted(cr_by_order.keys())])
+print(f"\nExtracting {len(all_traces)} fiber traces...")
+
+# Extraction parameters
+extract_params = {
+    "extraction_height": 0.5,
+    "lambda_sf": 0.1,  # smooth_slitfunction
+    "lambda_sp": 0,  # smooth_spectrum
+    "osample": 10,  # oversampling
+    "swath_width": 200,
+    "plot": 1,
+}
+
+norm, spec, blaze, unc = extract(
+    img_combined,
+    all_traces,
+    column_range=all_cr,
+    extraction_type="normalize",
+    gain=1.0,
+    readnoise=0.0,
+    dark=0.0,
+    **extract_params,
+)
+
+print(f"  Extracted blaze shape: {blaze.shape}")
+print(f"  Normalized flat shape: {norm.shape}")
+
+# Save extraction results
+np.savez(output_blaze, blaze=blaze, norm=norm, spec=spec, unc=unc)
+print(f"Saved blaze to: {output_blaze}")

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/crires_plus_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce
@@ -21,14 +21,14 @@ steps = (
     # "bias",
     # "flat",
     # "trace",
-    # "curvature",
+    "curvature",
     # "scatter",
     # "norm_flat",
     # "wavecal_master",
     # "wavecal_init",
     # "wavecal",
     # "freq_comb_master",
-    "freq_comb",
+    # "freq_comb",
     # "science",
     # "continuum",
     # "finalize",
@@ -48,6 +48,5 @@ Pipeline.from_instrument(
     base_dir=base_dir,
     input_dir=input_dir,
     output_dir=output_dir,
-    allow_calibration_only=True,
     # order_range=(0, 4),
 ).run()

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/custom_instrument_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/harpn_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/harps_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/harps_gridsearch.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 import os.path
 
@@ -71,7 +71,7 @@ kwargs_comb["nstep"] = 0
 kwargs["plot"] = False
 
 
-for key in ["extraction_method", "extraction_width", "extraction_cutoff"]:
+for key in ["extraction_method", "extraction_height", "extraction_cutoff"]:
     del kwargs_comb[key]
     del kwargs[key]
 

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/just_one_swath.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 import matplotlib.pyplot as plt
 import numpy as np

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/jwst_miri_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/jwst_niriss_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/lick_apf_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce
@@ -17,9 +17,9 @@ night = None
 channel = ""
 steps = (
     "bias",
-    # "flat",
-    # "trace",
-    # "norm_flat",
+    "flat",
+    "trace",
+    "norm_flat",
     # "wavecal",
     # "curvature",
     "science",
@@ -42,5 +42,5 @@ Pipeline.from_instrument(
     input_dir=input_dir,
     output_dir=output_dir,
     # order_range=(0, 25),
-    plot=0,
+    plot=1,
 ).run()

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/mcdonald_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/metis_ifu_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/metis_lss_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/micado_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce
@@ -48,11 +48,11 @@ config = pyreduce.configuration.get_configuration_for_instrument(instrument)
 
 
 # config["trace"]["noise"] = 100
-# config["curvature"]["extraction_width"] = 350 # curvature can still be improved with this and the following parameters
+# config["curvature"]["extraction_height"] = 350 # curvature can still be improved with this and the following parameters
 # config["curvature"]["peak_threshold"] =10
 # config["curvature"]["peak_width"] =2 #CHECK 6 also works and detects one less line
 # config["curvature"]["window_width"] = 5
-# config["wavecal"]["extraction_width"] = 350
+# config["wavecal"]["extraction_height"] = 350
 
 # NOTE: micado.thar_master.fits (created and controlled by wavecal_master) is NOT overwritten if any parameter in the steps in or before it are changed. Thus it has to be deleted before running PyReduce again.
 

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/neid_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 NEID reduction example

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/nirspec_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/toes_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 import numpy as np
 
@@ -90,7 +90,7 @@ config["trace"]["min_width"] = 200
 config["trace"]["manual"] = True
 config["norm_flat"]["oversampling"] = 8  # Subpixel scale for slit function modelling
 config["norm_flat"]["swath_width"] = 400  # Extraction swath width (columns)
-config["wavecal_master"]["extraction_width"] = 2
+config["wavecal_master"]["extraction_height"] = 2
 config["wavecal_master"]["collapse_function"] = "sum"
 config["wavecal_master"]["bias_scaling"] = "number_of_files"
 config["wavecal"]["medium"] = "vac"
@@ -102,7 +102,7 @@ config["science"]["oversampling"] = 8  # Subpixel scale for slit function modell
 config["science"]["swath_width"] = 400  # Extraction swath width (columns)
 config["science"]["smooth_slitfunction"] = 1.0  # Smoothing of the slit function
 config["science"]["smooth_spectrum"] = 1.0e-6  # Smoothing in spectral direction
-config["science"]["extraction_width"] = [5, 5]  # Extraction slit height (rows)
+config["science"]["extraction_height"] = [5, 5]  # Extraction slit height (rows)
 config["science"]["bias_scaling"] = "number_of_files"
 
 # Since we can't find the files ourselves (at least not without defining the criteria we are looking for)

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/uves_callfunc.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Example showing direct function calls for each step.

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/uves_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/examples/xshooter_example.py

@@ -1,6 +1,6 @@
 # /// script
 # requires-python = ">=3.13"
-# dependencies = ["pyreduce-astro>=0.7a6"]
+# dependencies = ["pyreduce-astro>=0.7b1"]
 # ///
 """
 Simple usage example for PyReduce
@@ -41,7 +41,7 @@ input_dir = "raw"
 output_dir = "reduced"
 
 config = get_configuration_for_instrument(instrument)
-# config["science"]["extraction_method"] = "arc"
+# config["science"]["extraction_method"] = "simple"
 # config["science"]["extraction_cutoff"] = 0
 
 Pipeline.from_instrument(

{pyreduce_astro-0.7a6 → pyreduce_astro-0.7b1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pyreduce-astro"
-version = "0.7a6"
+version = "0.7b1"
 requires-python = ">=3.13"
 description = "A data reduction package for echelle spectrographs"
 readme = "README.md"
@@ -42,7 +42,6 @@ dependencies = [
     "python-dateutil>=2.8.0",
     "scikit-image>=0.22.0",
     "tqdm>=4.66.0",
-    "wget>=3.2",
     "Pillow>=10.0.0",
 ]
 
@@ -55,6 +54,7 @@ dev = [
     "sphinx>=7.2.0",
     "ipython>=9.8.0",
     "myst-parser",
+    "setuptools",
 ]
 
 [project.urls]
@@ -65,6 +65,8 @@ Issues = "https://github.com/ivh/PyReduce/issues"
 
 [project.scripts]
 reduce = "pyreduce.__main__:main"
+reduce-build = "pyreduce.clib.build_extract:build"
+reduce-clean = "pyreduce.clib.build_extract:clean"
 
 [build-system]
 requires = ["hatchling>=1.25.0", "cffi>=1.17.1", "setuptools"]