svc-processing 0.1.2__tar.gz → 0.1.4__tar.gz

{svc_processing-0.1.2 → svc_processing-0.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: svc-processing
-Version: 0.1.2
+Version: 0.1.4
 Summary: Pure-Python SVC HR-1024i hyperspectral .sig processing pipeline
 Author: Gold Lab (GrapeSPEC project), Cornell University
 Author-email: Cole Regnier <nr466@cornell.edu>
@@ -70,21 +70,35 @@ Generated outputs (gitignored): `pipeline_outputs/sig_processed/<run>/`, `pipeli
 ```bash
 python3.11 -m venv .venv
 source .venv/bin/activate
-python -m pip install svc-processing
+python -m pip install svc-processing   # package name
 
-svc-pipeline --init-config   # writes a starter config/config.json here
+svc-pipeline --init-config   # command name — different from the package name above
 # Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
 svc-pipeline
 ```
 
-**Working from a clone of this repo** (config/config.json already exists):
+**Working from a clone of this repo** (config/config.json already exists) — run the
+setup script once, then activate the environment it created:
+
+```bash
+./setup.sh            # creates .venv, installs everything, stages the demo data
+# ./setup.sh --dev    # add the test & lint tools for development
+
+source .venv/bin/activate          # macOS / Linux
+# .venv\Scripts\activate           # Windows PowerShell
+
+# Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
+svc-pipeline config.json
+```
+
+The script always installs into *this* repo's `.venv` no matter which folder you run
+it from, so it avoids the "Directory '.' is not installable" and "No module named …"
+errors that come from being in the wrong directory. Prefer to do it by hand?
 
 ```bash
 python3.11 -m venv .venv
 source .venv/bin/activate
 python -m pip install -e ".[dev,demo]"
-
-# Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
 svc-pipeline config.json
 ```
 
@@ -126,8 +140,10 @@ Outputs land under `pipeline_outputs/` by default. See [`config/README.md`](conf
    └────────────────────────────────────┘
 ```
 
-The `svc-pipeline` console script glues Stages 1 and 2 together through
-`pipeline.cli`; Stage 3 is invoked from notebooks against the Stage 2 output.
+The `svc-pipeline` console script runs Stages 1 and 2 through `pipeline.cli`,
+and Stage 3 too once a `groups_csv` is configured (or passed via
+`--groups-csv`) — see [`config/README.md`](config/README.md#grouping--stage-3-optional).
+Stage 3 is also available directly from notebooks against the Stage 2 output.
 The pipeline is **single-pass and idempotent per input directory** — previous
 processed `.sig` files in the target directory are deleted at the start of each
 run.
@@ -146,7 +162,9 @@ svc-pipeline [config] [options]
 |---|---|
 | `config` | Run-config JSON (positional, optional; default `config/config.json`). Bare names resolve under `config/`, so `config.json`, `config`, and `config/config.json` are equivalent. See [`config/README.md`](config/README.md) for schema. |
 | `--input-dir <path>` | Override `sig_input_dir` and process only this directory. |
-| `--step {1,2,all}` | `1` = process + summary CSV only; `2` = resample only (requires Stage 1 to have been run); `all` = both (default). |
+| `--step {1,2,3,all}` | `1` = process + summary CSV only; `2` = resample only (requires Stage 1); `3` = group + average only (requires Stage 2 and `groups_csv`); `all` = every stage that's configured (default). |
+| `--groups-csv <path>` | Override the config's `groups_csv` to group repeat scans and average them. |
+| `--group-method {mean,median,sum,min,max}` | Override the config's `group_agg_method` for Stage 3 (default `mean`). |
 | `--verbose` | Print INFO/DEBUG logs before and after each stage. |
 
 ### Expected output layout
@@ -157,10 +175,11 @@ pipeline_outputs/
 │   ├── <truncated *.sig files>
 │   └── <input_dir_name>_processed_sig_summary.csv
 └── sig_resampled/<input_dir_name>/
-    └── <input_dir_name>_merged_spectra.csv     # 2101 columns (400–2500 nm)
+    ├── <input_dir_name>_merged_spectra.csv     # 2101 columns (400–2500 nm)
+    └── <input_dir_name>_grouped_spectra.csv    # only if groups_csv is configured
 ```
 
-`summary_csv_name` and `merged_csv_name` in the run config are suffixes; the real filenames are prefixed with the input directory name. Verify your config in [`config/README.md`](config/README.md).
+`summary_csv_name`, `merged_csv_name`, and `grouped_csv_name` in the run config are suffixes; the real filenames are prefixed with the input directory name. Verify your config in [`config/README.md`](config/README.md).
 
 ---
 

{svc_processing-0.1.2 → svc_processing-0.1.4}/README.md

@@ -37,21 +37,35 @@ Generated outputs (gitignored): `pipeline_outputs/sig_processed/<run>/`, `pipeli
 ```bash
 python3.11 -m venv .venv
 source .venv/bin/activate
-python -m pip install svc-processing
+python -m pip install svc-processing   # package name
 
-svc-pipeline --init-config   # writes a starter config/config.json here
+svc-pipeline --init-config   # command name — different from the package name above
 # Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
 svc-pipeline
 ```
 
-**Working from a clone of this repo** (config/config.json already exists):
+**Working from a clone of this repo** (config/config.json already exists) — run the
+setup script once, then activate the environment it created:
+
+```bash
+./setup.sh            # creates .venv, installs everything, stages the demo data
+# ./setup.sh --dev    # add the test & lint tools for development
+
+source .venv/bin/activate          # macOS / Linux
+# .venv\Scripts\activate           # Windows PowerShell
+
+# Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
+svc-pipeline config.json
+```
+
+The script always installs into *this* repo's `.venv` no matter which folder you run
+it from, so it avoids the "Directory '.' is not installable" and "No module named …"
+errors that come from being in the wrong directory. Prefer to do it by hand?
 
 ```bash
 python3.11 -m venv .venv
 source .venv/bin/activate
 python -m pip install -e ".[dev,demo]"
-
-# Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
 svc-pipeline config.json
 ```
 
@@ -93,8 +107,10 @@ Outputs land under `pipeline_outputs/` by default. See [`config/README.md`](conf
    └────────────────────────────────────┘
 ```
 
-The `svc-pipeline` console script glues Stages 1 and 2 together through
-`pipeline.cli`; Stage 3 is invoked from notebooks against the Stage 2 output.
+The `svc-pipeline` console script runs Stages 1 and 2 through `pipeline.cli`,
+and Stage 3 too once a `groups_csv` is configured (or passed via
+`--groups-csv`) — see [`config/README.md`](config/README.md#grouping--stage-3-optional).
+Stage 3 is also available directly from notebooks against the Stage 2 output.
 The pipeline is **single-pass and idempotent per input directory** — previous
 processed `.sig` files in the target directory are deleted at the start of each
 run.
@@ -113,7 +129,9 @@ svc-pipeline [config] [options]
 |---|---|
 | `config` | Run-config JSON (positional, optional; default `config/config.json`). Bare names resolve under `config/`, so `config.json`, `config`, and `config/config.json` are equivalent. See [`config/README.md`](config/README.md) for schema. |
 | `--input-dir <path>` | Override `sig_input_dir` and process only this directory. |
-| `--step {1,2,all}` | `1` = process + summary CSV only; `2` = resample only (requires Stage 1 to have been run); `all` = both (default). |
+| `--step {1,2,3,all}` | `1` = process + summary CSV only; `2` = resample only (requires Stage 1); `3` = group + average only (requires Stage 2 and `groups_csv`); `all` = every stage that's configured (default). |
+| `--groups-csv <path>` | Override the config's `groups_csv` to group repeat scans and average them. |
+| `--group-method {mean,median,sum,min,max}` | Override the config's `group_agg_method` for Stage 3 (default `mean`). |
 | `--verbose` | Print INFO/DEBUG logs before and after each stage. |
 
 ### Expected output layout
@@ -124,10 +142,11 @@ pipeline_outputs/
 │   ├── <truncated *.sig files>
 │   └── <input_dir_name>_processed_sig_summary.csv
 └── sig_resampled/<input_dir_name>/
-    └── <input_dir_name>_merged_spectra.csv     # 2101 columns (400–2500 nm)
+    ├── <input_dir_name>_merged_spectra.csv     # 2101 columns (400–2500 nm)
+    └── <input_dir_name>_grouped_spectra.csv    # only if groups_csv is configured
 ```
 
-`summary_csv_name` and `merged_csv_name` in the run config are suffixes; the real filenames are prefixed with the input directory name. Verify your config in [`config/README.md`](config/README.md).
+`summary_csv_name`, `merged_csv_name`, and `grouped_csv_name` in the run config are suffixes; the real filenames are prefixed with the input directory name. Verify your config in [`config/README.md`](config/README.md).
 
 ---
 

{svc_processing-0.1.2 → svc_processing-0.1.4}/pipeline/cli.py

@@ -28,7 +28,13 @@ def _configure_logging(verbose: bool) -> logging.Logger:
 
 
 def _parse_args() -> argparse.Namespace:
-    parser = argparse.ArgumentParser(description="Run the SIG processing pipeline")
+    parser = argparse.ArgumentParser(
+        description="Run the SIG processing pipeline",
+        epilog=(
+            "Full README, TUTORIAL, and config reference: "
+            "https://github.com/regs08/SVCProcessingPipeline"
+        ),
+    )
     parser.add_argument(
         "config",
         nargs="?",
@@ -41,9 +47,13 @@ def _parse_args() -> argparse.Namespace:
     )
     parser.add_argument(
         "--step",
-        choices=["1", "2", "all"],
+        choices=["1", "2", "3", "all"],
         default="all",
-        help="Which step to run: 1=process+summary CSV, 2=Python resampling only, all=run both steps.",
+        help=(
+            "Which step to run: 1=process+summary CSV, 2=Python resampling only, "
+            "3=group+average only (requires groups_csv; step 2 must have already run), "
+            "all=every step that's configured (step 3 only if groups_csv is set)."
+        ),
     )
     parser.add_argument(
         "--verbose",
@@ -55,6 +65,18 @@ def _parse_args() -> argparse.Namespace:
         action="store_true",
         help="Write a starter config/config.json in the current directory and exit.",
     )
+    parser.add_argument(
+        "--groups-csv",
+        help=(
+            "Override the config's groups_csv: group repeat scans of the same sample "
+            "and average them (a CSV with scans/scan_id + name columns)."
+        ),
+    )
+    parser.add_argument(
+        "--group-method",
+        choices=["mean", "median", "sum", "min", "max"],
+        help="Override the config's group_agg_method for step 3 (default: mean).",
+    )
     return parser.parse_args()
 
 
@@ -85,9 +107,16 @@ def main() -> None:
     # warning for any non-default value before the per-directory work begins.
     run_config.processing_params()
 
+    groups_csv_override = Path(args.groups_csv).expanduser() if args.groups_csv else None
+
     overall_results: list[tuple[Path, dict[str, Path | None]]] = []
     for input_dir in input_dirs:
-        settings = run_config.settings_for(input_dir, verbose=args.verbose)
+        settings = run_config.settings_for(
+            input_dir,
+            verbose=args.verbose,
+            groups_csv_override=groups_csv_override,
+            group_method_override=args.group_method,
+        )
         results = Pipeline(settings, logger).run(args.step)
         overall_results.append((input_dir, results))
 

{svc_processing-0.1.2 → svc_processing-0.1.4}/pipeline/run_config.py

@@ -76,6 +76,9 @@ class PipelineSettings:
     processing_params: dict
     correction_types: dict[str, str]
     instrument_numbers: dict[str, str]
+    groups_csv: Path | None
+    group_agg_method: str
+    grouped_csv_name: str
 
 
 class RunConfig:
@@ -299,7 +302,14 @@ class RunConfig:
             self._logger.info("Instrument serials from config: %s", serials)
         return correction_types, instrument_numbers
 
-    def settings_for(self, input_dir: Path, *, verbose: bool) -> PipelineSettings:
+    def settings_for(
+        self,
+        input_dir: Path,
+        *,
+        verbose: bool,
+        groups_csv_override: str | Path | None = None,
+        group_method_override: str | None = None,
+    ) -> PipelineSettings:
         """Build the fully-resolved :class:`PipelineSettings` for one input directory."""
         input_dir = Path(input_dir).expanduser()
         source_name = input_dir.name or "sig_input"
@@ -325,6 +335,11 @@ class RunConfig:
             if key is not None
         }
 
+        groups_csv_value = groups_csv_override or self.data.get("groups_csv")
+        groups_csv = _resolve_under(self.base_dir, str(groups_csv_value)) if groups_csv_value else None
+        group_agg_method = str(group_method_override or self.data.get("group_agg_method", "mean"))
+        grouped_csv_name = f"{source_name}_{self.data.get('grouped_csv_name', 'grouped_spectra.csv')}"
+
         return PipelineSettings(
             source_name=source_name,
             input_dir=input_dir,
@@ -337,4 +352,7 @@ class RunConfig:
             processing_params=self.processing_params(),
             correction_types=correction_types,
             instrument_numbers=instrument_numbers,
+            groups_csv=groups_csv,
+            group_agg_method=group_agg_method,
+            grouped_csv_name=grouped_csv_name,
         )

{svc_processing-0.1.2 → svc_processing-0.1.4}/pipeline/runner.py

@@ -1,4 +1,4 @@
-"""Execution of the two SIG processing stages.
+"""Execution of the three SIG processing stages.
 
 :class:`Pipeline` runs, for a single resolved
 :class:`~pipeline.run_config.PipelineSettings`:
@@ -7,6 +7,9 @@
   the calibration end-line and write a per-run summary CSV.
 * Stage 2 (:meth:`Pipeline.resample`) — run the pure-Python resampler over the
   processed files.
+* Stage 3 (:meth:`Pipeline.group_and_average`) — optional; only runs when the
+  config supplies a ``groups_csv``. Groups repeat scans of the same sample and
+  averages them into one spectrum per group.
 
 Config loading that produces the ``PipelineSettings`` lives in
 ``pipeline/run_config.py``; the CLI that wires them together lives in
@@ -20,6 +23,7 @@ import logging
 from pathlib import Path
 from typing import Iterable
 
+from .processor import GroupSpec, SigSpectraAverager
 from .resampler import resample_spectra
 from .run_config import PipelineSettings
 from .sig_processor import SigFileProcessor
@@ -38,12 +42,14 @@ class Pipeline:
         """Run the requested step(s) and report the produced artifacts.
 
         ``step``: ``"1"`` = Stage 1 only, ``"2"`` = Stage 2 only (Stage 1 must
-        have run previously), ``"all"`` = both.
+        have run previously), ``"3"`` = Stage 3 only (Stage 2 must have run
+        previously; requires ``groups_csv`` to be configured), ``"all"`` = every
+        stage that's configured to run (Stage 3 only if ``groups_csv`` is set).
         """
-        summary_csv: Path | None
+        summary_csv: Path | None = None
         if step in {"1", "all"}:
             summary_csv = self.process_sig_files()
-        else:
+        elif step == "2":
             summary_csv = self.settings.summary_csv if self.settings.summary_csv.exists() else None
             if summary_csv is None:
                 self.logger.error(
@@ -54,8 +60,17 @@ class Pipeline:
         merged_csv: Path | None = None
         if step in {"2", "all"}:
             merged_csv = self.resample(summary_csv)
+        elif step == "3":
+            merged_path = self.settings.resampled_dir / self.settings.merged_csv_name
+            merged_csv = merged_path if merged_path.exists() else None
+            if merged_csv is None:
+                self.logger.error("Merged CSV not found at %s (run --step 2 first).", merged_path.resolve())
 
-        return {"summary_csv": summary_csv, "merged_csv": merged_csv}
+        grouped_csv: Path | None = None
+        if step in {"3", "all"} and self.settings.groups_csv is not None:
+            grouped_csv = self.group_and_average(merged_csv)
+
+        return {"summary_csv": summary_csv, "merged_csv": merged_csv, "grouped_csv": grouped_csv}
 
     # ── Stage 1: process raw .sig files ──────────────────────────────────────
 
@@ -194,6 +209,44 @@ class Pipeline:
         logger.warning("Merged spectra file was not created at %s", merged_path)
         return None
 
+    # ── Stage 3: group & average (optional) ──────────────────────────────────
+
+    def group_and_average(self, merged_csv: Path | None) -> Path | None:
+        """Average repeat scans of the same sample into one spectrum per group.
+
+        Requires ``settings.groups_csv`` (checked by the caller) and a merged
+        CSV from Stage 2. Groups are read via :meth:`GroupSpec.from_csv`
+        (``scans``/``scan_id`` + ``name`` columns) and applied with
+        :class:`SigSpectraAverager`.
+        """
+        settings = self.settings
+        logger = self.logger
+        if merged_csv is None:
+            logger.error("Skipping grouping because no merged CSV is available.")
+            return None
+
+        groups_csv = settings.groups_csv
+        if groups_csv is None or not groups_csv.exists():
+            logger.error("Groups CSV not found: %s", groups_csv.resolve() if groups_csv else groups_csv)
+            return None
+
+        groups = GroupSpec.from_csv(groups_csv)
+        if not groups:
+            logger.warning("No groups found in %s; nothing to average.", groups_csv.resolve())
+            return None
+
+        if settings.verbose:
+            logger.info("Grouping %s using %s (%s)", merged_csv, groups_csv.resolve(), settings.group_agg_method)
+
+        averager = SigSpectraAverager.from_csv(merged_csv)
+        grouped_df = averager.aggregate(groups, method=settings.group_agg_method)
+
+        output_path = settings.resampled_dir / settings.grouped_csv_name
+        output_path.parent.mkdir(parents=True, exist_ok=True)
+        grouped_df.to_csv(output_path, index=False)
+        logger.info("Grouped/averaged spectra written to %s", output_path.resolve())
+        return output_path
+
     # ── helpers ──────────────────────────────────────────────────────────────
 
     @staticmethod

{svc_processing-0.1.2 → svc_processing-0.1.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "svc-processing"
-version = "0.1.2"
+version = "0.1.4"
 description = "Pure-Python SVC HR-1024i hyperspectral .sig processing pipeline"
 readme = "README.md"
 requires-python = ">=3.11"

{svc_processing-0.1.2 → svc_processing-0.1.4}/svc_processing.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: svc-processing
-Version: 0.1.2
+Version: 0.1.4
 Summary: Pure-Python SVC HR-1024i hyperspectral .sig processing pipeline
 Author: Gold Lab (GrapeSPEC project), Cornell University
 Author-email: Cole Regnier <nr466@cornell.edu>
@@ -70,21 +70,35 @@ Generated outputs (gitignored): `pipeline_outputs/sig_processed/<run>/`, `pipeli
 ```bash
 python3.11 -m venv .venv
 source .venv/bin/activate
-python -m pip install svc-processing
+python -m pip install svc-processing   # package name
 
-svc-pipeline --init-config   # writes a starter config/config.json here
+svc-pipeline --init-config   # command name — different from the package name above
 # Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
 svc-pipeline
 ```
 
-**Working from a clone of this repo** (config/config.json already exists):
+**Working from a clone of this repo** (config/config.json already exists) — run the
+setup script once, then activate the environment it created:
+
+```bash
+./setup.sh            # creates .venv, installs everything, stages the demo data
+# ./setup.sh --dev    # add the test & lint tools for development
+
+source .venv/bin/activate          # macOS / Linux
+# .venv\Scripts\activate           # Windows PowerShell
+
+# Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
+svc-pipeline config.json
+```
+
+The script always installs into *this* repo's `.venv` no matter which folder you run
+it from, so it avoids the "Directory '.' is not installable" and "No module named …"
+errors that come from being in the wrong directory. Prefer to do it by hand?
 
 ```bash
 python3.11 -m venv .venv
 source .venv/bin/activate
 python -m pip install -e ".[dev,demo]"
-
-# Edit config/config.json — replace "<PATH_TO_SIG_INPUT_ROOT>" with your data path.
 svc-pipeline config.json
 ```
 
@@ -126,8 +140,10 @@ Outputs land under `pipeline_outputs/` by default. See [`config/README.md`](conf
    └────────────────────────────────────┘
 ```
 
-The `svc-pipeline` console script glues Stages 1 and 2 together through
-`pipeline.cli`; Stage 3 is invoked from notebooks against the Stage 2 output.
+The `svc-pipeline` console script runs Stages 1 and 2 through `pipeline.cli`,
+and Stage 3 too once a `groups_csv` is configured (or passed via
+`--groups-csv`) — see [`config/README.md`](config/README.md#grouping--stage-3-optional).
+Stage 3 is also available directly from notebooks against the Stage 2 output.
 The pipeline is **single-pass and idempotent per input directory** — previous
 processed `.sig` files in the target directory are deleted at the start of each
 run.
@@ -146,7 +162,9 @@ svc-pipeline [config] [options]
 |---|---|
 | `config` | Run-config JSON (positional, optional; default `config/config.json`). Bare names resolve under `config/`, so `config.json`, `config`, and `config/config.json` are equivalent. See [`config/README.md`](config/README.md) for schema. |
 | `--input-dir <path>` | Override `sig_input_dir` and process only this directory. |
-| `--step {1,2,all}` | `1` = process + summary CSV only; `2` = resample only (requires Stage 1 to have been run); `all` = both (default). |
+| `--step {1,2,3,all}` | `1` = process + summary CSV only; `2` = resample only (requires Stage 1); `3` = group + average only (requires Stage 2 and `groups_csv`); `all` = every stage that's configured (default). |
+| `--groups-csv <path>` | Override the config's `groups_csv` to group repeat scans and average them. |
+| `--group-method {mean,median,sum,min,max}` | Override the config's `group_agg_method` for Stage 3 (default `mean`). |
 | `--verbose` | Print INFO/DEBUG logs before and after each stage. |
 
 ### Expected output layout
@@ -157,10 +175,11 @@ pipeline_outputs/
 │   ├── <truncated *.sig files>
 │   └── <input_dir_name>_processed_sig_summary.csv
 └── sig_resampled/<input_dir_name>/
-    └── <input_dir_name>_merged_spectra.csv     # 2101 columns (400–2500 nm)
+    ├── <input_dir_name>_merged_spectra.csv     # 2101 columns (400–2500 nm)
+    └── <input_dir_name>_grouped_spectra.csv    # only if groups_csv is configured
 ```
 
-`summary_csv_name` and `merged_csv_name` in the run config are suffixes; the real filenames are prefixed with the input directory name. Verify your config in [`config/README.md`](config/README.md).
+`summary_csv_name`, `merged_csv_name`, and `grouped_csv_name` in the run config are suffixes; the real filenames are prefixed with the input directory name. Verify your config in [`config/README.md`](config/README.md).
 
 ---
 

{svc_processing-0.1.2 → svc_processing-0.1.4}/tests/test_cli.py

@@ -14,6 +14,18 @@ def _write_config(path: Path, data: dict) -> Path:
     return path
 
 
+def test_help_links_to_github_repo(
+    monkeypatch: pytest.MonkeyPatch,
+    capsys: pytest.CaptureFixture[str],
+) -> None:
+    monkeypatch.setattr(sys, "argv", ["svc-pipeline", "--help"])
+
+    with pytest.raises(SystemExit):
+        _parse_args()
+
+    assert "github.com/regs08/SVCProcessingPipeline" in capsys.readouterr().out
+
+
 def test_parse_args_step_and_input_dir_default_to_all_and_none(monkeypatch: pytest.MonkeyPatch) -> None:
     monkeypatch.setattr(sys, "argv", ["svc-pipeline", "config.json"])
 
@@ -39,6 +51,20 @@ def test_parse_args_accepts_step_input_dir_and_verbose(monkeypatch: pytest.Monke
     assert args.verbose is True
 
 
+def test_parse_args_accepts_step_three_and_group_overrides(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.setattr(
+        sys,
+        "argv",
+        ["svc-pipeline", "--step", "3", "--groups-csv", "naming_ids/groups.csv", "--group-method", "median"],
+    )
+
+    args = _parse_args()
+
+    assert args.step == "3"
+    assert args.groups_csv == "naming_ids/groups.csv"
+    assert args.group_method == "median"
+
+
 def test_main_input_dir_override_bypasses_config_input_and_placeholder(
     tmp_path: Path,
     monkeypatch: pytest.MonkeyPatch,
@@ -73,6 +99,49 @@ def test_main_input_dir_override_bypasses_config_input_and_placeholder(
     assert "not produced" in captured.out
 
 
+def test_main_step_three_groups_existing_merged_csv(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+    capsys: pytest.CaptureFixture[str],
+) -> None:
+    groups_csv = tmp_path / "groups.csv"
+    groups_csv.write_text("scan_id,name\n1,leaf_a\n2,leaf_a\n")
+
+    config_path = _write_config(
+        tmp_path / "config.json",
+        {
+            "sig_input_dir": "<PATH_TO_SIG_INPUT_ROOT>",
+            "processed_dir": "sig_processed",
+            "resampled_dir": "sig_resampled",
+            "output_dir": str(tmp_path / "out"),
+            "summary_csv_name": "summary.csv",
+            "merged_csv_name": "merged.csv",
+            "groups_csv": str(groups_csv),
+        },
+    )
+    input_dir = tmp_path / "myinput"
+    input_dir.mkdir()
+
+    resampled_dir = tmp_path / "out" / "sig_resampled" / "myinput"
+    resampled_dir.mkdir(parents=True)
+    merged_csv = resampled_dir / "myinput_merged.csv"
+    merged_csv.write_text("sample_name,400\nleaf.0001,1.0\nleaf.0002,3.0\n")
+
+    monkeypatch.setattr(
+        sys,
+        "argv",
+        ["svc-pipeline", str(config_path), "--input-dir", str(input_dir), "--step", "3"],
+    )
+
+    main()
+
+    grouped_csv = resampled_dir / "myinput_grouped_spectra.csv"
+    assert grouped_csv.exists()
+    captured = capsys.readouterr()
+    assert "grouped_csv" in captured.out
+    assert str(grouped_csv.resolve()) in captured.out
+
+
 def test_main_init_config_writes_starter_and_exits(
     tmp_path: Path,
     monkeypatch: pytest.MonkeyPatch,

{svc_processing-0.1.2 → svc_processing-0.1.4}/tests/test_run_config.py

@@ -124,3 +124,58 @@ def test_settings_for_resolves_output_paths(tmp_path: Path) -> None:
     assert settings.merged_csv_name == "example_input_merged_spectra.csv"
     assert settings.correction_types["silver"] == "2517.9"
     assert settings.instrument_numbers["silver"] == "1202103"
+
+
+def test_settings_for_grouping_defaults_when_unconfigured(tmp_path: Path) -> None:
+    data = {
+        "processed_dir": "sig_processed",
+        "resampled_dir": "sig_resampled",
+        "summary_csv_name": "summary.csv",
+        "merged_csv_name": "merged.csv",
+    }
+    config = RunConfig(data, path=tmp_path / "config.json", base_dir=tmp_path, logger=_logger())
+
+    settings = config.settings_for(Path("example_input"), verbose=False)
+
+    assert settings.groups_csv is None
+    assert settings.group_agg_method == "mean"
+    assert settings.grouped_csv_name == "example_input_grouped_spectra.csv"
+
+
+def test_settings_for_resolves_groups_csv_from_config(tmp_path: Path) -> None:
+    data = {
+        "processed_dir": "sig_processed",
+        "resampled_dir": "sig_resampled",
+        "summary_csv_name": "summary.csv",
+        "merged_csv_name": "merged.csv",
+        "groups_csv": "naming_ids/groups.csv",
+        "group_agg_method": "median",
+    }
+    config = RunConfig(data, path=tmp_path / "config.json", base_dir=tmp_path, logger=_logger())
+
+    settings = config.settings_for(Path("example_input"), verbose=False)
+
+    assert settings.groups_csv == tmp_path / "naming_ids/groups.csv"
+    assert settings.group_agg_method == "median"
+
+
+def test_settings_for_cli_overrides_take_priority_over_config(tmp_path: Path) -> None:
+    data = {
+        "processed_dir": "sig_processed",
+        "resampled_dir": "sig_resampled",
+        "summary_csv_name": "summary.csv",
+        "merged_csv_name": "merged.csv",
+        "groups_csv": "naming_ids/groups.csv",
+        "group_agg_method": "mean",
+    }
+    config = RunConfig(data, path=tmp_path / "config.json", base_dir=tmp_path, logger=_logger())
+
+    settings = config.settings_for(
+        Path("example_input"),
+        verbose=False,
+        groups_csv_override=tmp_path / "override_groups.csv",
+        group_method_override="max",
+    )
+
+    assert settings.groups_csv == tmp_path / "override_groups.csv"
+    assert settings.group_agg_method == "max"

svc_processing-0.1.4/tests/test_runner.py

@@ -0,0 +1,136 @@
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+from pipeline.run_config import PipelineSettings
+from pipeline.runner import Pipeline
+
+
+def _settings(
+    tmp_path: Path,
+    input_dir: Path,
+    *,
+    groups_csv: Path | None = None,
+    group_agg_method: str = "mean",
+) -> PipelineSettings:
+    return PipelineSettings(
+        source_name=input_dir.name or "input",
+        input_dir=input_dir,
+        processed_dir=tmp_path / "processed",
+        resampled_dir=tmp_path / "resampled",
+        summary_csv=tmp_path / "processed" / "summary.csv",
+        merged_csv_name="merged.csv",
+        end_line_overrides={},
+        verbose=False,
+        processing_params={
+            "band_min": 400,
+            "band_max": 410,
+            "resample_fwhm_nm": 10.0,
+            "splice_interp_wvl": (5.0, 2.0),
+            "fixed_sensor": 2,
+        },
+        correction_types={"bronze": "2520.4", "silver": "2517.9"},
+        instrument_numbers={"bronze": "2212118", "silver": "1202103"},
+        groups_csv=groups_csv,
+        group_agg_method=group_agg_method,
+        grouped_csv_name="grouped.csv",
+    )
+
+
+def test_process_sig_files_returns_none_for_missing_input(tmp_path: Path) -> None:
+    pipeline = Pipeline(_settings(tmp_path, tmp_path / "missing"), logging.getLogger("tests.runner"))
+
+    assert pipeline.process_sig_files() is None
+
+
+def test_process_sig_files_returns_none_for_empty_input(tmp_path: Path) -> None:
+    input_dir = tmp_path / "empty"
+    input_dir.mkdir()
+    pipeline = Pipeline(_settings(tmp_path, input_dir), logging.getLogger("tests.runner"))
+
+    assert pipeline.process_sig_files() is None
+
+
+def test_run_step_two_without_summary_does_not_resample(tmp_path: Path) -> None:
+    input_dir = tmp_path / "input"
+    input_dir.mkdir()
+    pipeline = Pipeline(_settings(tmp_path, input_dir), logging.getLogger("tests.runner"))
+
+    result = pipeline.run("2")
+
+    assert result == {"summary_csv": None, "merged_csv": None, "grouped_csv": None}
+
+
+def test_run_all_skips_grouping_when_groups_csv_not_configured(tmp_path: Path) -> None:
+    input_dir = tmp_path / "input"
+    input_dir.mkdir()
+    pipeline = Pipeline(_settings(tmp_path, input_dir), logging.getLogger("tests.runner"))
+
+    result = pipeline.run("all")
+
+    assert result["grouped_csv"] is None
+
+
+def test_group_and_average_writes_grouped_csv(tmp_path: Path) -> None:
+    resampled_dir = tmp_path / "resampled"
+    resampled_dir.mkdir(parents=True)
+    merged_csv = resampled_dir / "merged.csv"
+    merged_csv.write_text("sample_name,400,401\nleaf.0001,1.0,2.0\nleaf.0002,3.0,4.0\n")
+
+    groups_csv = tmp_path / "groups.csv"
+    groups_csv.write_text("scan_id,name\n1,leaf_a\n2,leaf_a\n")
+
+    settings = _settings(tmp_path, tmp_path / "input", groups_csv=groups_csv)
+    pipeline = Pipeline(settings, logging.getLogger("tests.runner"))
+
+    output = pipeline.group_and_average(merged_csv)
+
+    assert output == resampled_dir / "grouped.csv"
+    written = output.read_text()
+    assert "leaf_a" in written
+    assert "2.0" in written  # mean(1.0, 3.0)
+    assert "3.0" in written  # mean(2.0, 4.0)
+
+
+def test_group_and_average_missing_groups_file_returns_none(tmp_path: Path) -> None:
+    resampled_dir = tmp_path / "resampled"
+    resampled_dir.mkdir(parents=True)
+    merged_csv = resampled_dir / "merged.csv"
+    merged_csv.write_text("sample_name,400\nleaf.0001,1.0\n")
+
+    settings = _settings(tmp_path, tmp_path / "input", groups_csv=tmp_path / "missing_groups.csv")
+    pipeline = Pipeline(settings, logging.getLogger("tests.runner"))
+
+    assert pipeline.group_and_average(merged_csv) is None
+
+
+def test_run_step_three_without_merged_csv_logs_error(tmp_path: Path) -> None:
+    input_dir = tmp_path / "input"
+    input_dir.mkdir()
+    groups_csv = tmp_path / "groups.csv"
+    groups_csv.write_text("scan_id,name\n1,leaf_a\n")
+    pipeline = Pipeline(_settings(tmp_path, input_dir, groups_csv=groups_csv), logging.getLogger("tests.runner"))
+
+    result = pipeline.run("3")
+
+    assert result == {"summary_csv": None, "merged_csv": None, "grouped_csv": None}
+
+
+def test_run_step_three_groups_existing_merged_csv(tmp_path: Path) -> None:
+    input_dir = tmp_path / "input"
+    input_dir.mkdir()
+    resampled_dir = tmp_path / "resampled"
+    resampled_dir.mkdir(parents=True)
+    merged_csv = resampled_dir / "merged.csv"
+    merged_csv.write_text("sample_name,400\nleaf.0001,1.0\nleaf.0002,3.0\n")
+    groups_csv = tmp_path / "groups.csv"
+    groups_csv.write_text("scan_id,name\n1,leaf_a\n2,leaf_a\n")
+
+    pipeline = Pipeline(_settings(tmp_path, input_dir, groups_csv=groups_csv), logging.getLogger("tests.runner"))
+
+    result = pipeline.run("3")
+
+    assert result["merged_csv"] == merged_csv
+    assert result["grouped_csv"] == resampled_dir / "grouped.csv"
+    assert result["grouped_csv"].exists()

svc_processing-0.1.2/tests/test_runner.py

@@ -1,53 +0,0 @@
-from __future__ import annotations
-
-import logging
-from pathlib import Path
-
-from pipeline.run_config import PipelineSettings
-from pipeline.runner import Pipeline
-
-
-def _settings(tmp_path: Path, input_dir: Path) -> PipelineSettings:
-    return PipelineSettings(
-        source_name=input_dir.name or "input",
-        input_dir=input_dir,
-        processed_dir=tmp_path / "processed",
-        resampled_dir=tmp_path / "resampled",
-        summary_csv=tmp_path / "processed" / "summary.csv",
-        merged_csv_name="merged.csv",
-        end_line_overrides={},
-        verbose=False,
-        processing_params={
-            "band_min": 400,
-            "band_max": 410,
-            "resample_fwhm_nm": 10.0,
-            "splice_interp_wvl": (5.0, 2.0),
-            "fixed_sensor": 2,
-        },
-        correction_types={"bronze": "2520.4", "silver": "2517.9"},
-        instrument_numbers={"bronze": "2212118", "silver": "1202103"},
-    )
-
-
-def test_process_sig_files_returns_none_for_missing_input(tmp_path: Path) -> None:
-    pipeline = Pipeline(_settings(tmp_path, tmp_path / "missing"), logging.getLogger("tests.runner"))
-
-    assert pipeline.process_sig_files() is None
-
-
-def test_process_sig_files_returns_none_for_empty_input(tmp_path: Path) -> None:
-    input_dir = tmp_path / "empty"
-    input_dir.mkdir()
-    pipeline = Pipeline(_settings(tmp_path, input_dir), logging.getLogger("tests.runner"))
-
-    assert pipeline.process_sig_files() is None
-
-
-def test_run_step_two_without_summary_does_not_resample(tmp_path: Path) -> None:
-    input_dir = tmp_path / "input"
-    input_dir.mkdir()
-    pipeline = Pipeline(_settings(tmp_path, input_dir), logging.getLogger("tests.runner"))
-
-    result = pipeline.run("2")
-
-    assert result == {"summary_csv": None, "merged_csv": None}