cosmic-metrics 0.1.1__tar.gz

cosmic_metrics-0.1.1/PKG-INFO

@@ -0,0 +1,178 @@
+Metadata-Version: 2.4
+Name: cosmic-metrics
+Version: 0.1.1
+Summary: Per-block quality metrics of NenuFAR Cosmic Dawn data, for block selection and combination weights
+Author: Eric Le Saux
+Author-email: eric.ar.saos@gmail.com
+Requires-Python: >=3.10
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Provides-Extra: docs
+Provides-Extra: iono
+Provides-Extra: report
+Provides-Extra: test
+Requires-Dist: aoquality (>=0.6)
+Requires-Dist: astropy
+Requires-Dist: bokeh (>=3.8) ; extra == "report"
+Requires-Dist: click (>=8.0)
+Requires-Dist: h5py
+Requires-Dist: libpipe (>=0.9)
+Requires-Dist: matplotlib
+Requires-Dist: myst-parser (>=3.0) ; extra == "docs"
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: ps_eor (>=1.0)
+Requires-Dist: pspipe (>=0.14)
+Requires-Dist: pydata-sphinx-theme (>=0.16) ; extra == "docs"
+Requires-Dist: pytest ; extra == "test"
+Requires-Dist: ruff ; extra == "test"
+Requires-Dist: sphinx (>=7.0) ; extra == "docs"
+Requires-Dist: spinifex (>=2.0) ; extra == "iono"
+Requires-Dist: threadpoolctl (>=3.1)
+Requires-Dist: tomli ; python_version < "3.11"
+Project-URL: Documentation, https://cosmic-metrics.readthedocs.io
+Project-URL: Homepage, https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics
+Project-URL: Issues, https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/issues
+Project-URL: Repository, https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics
+Description-Content-Type: text/markdown
+
+# cosmic-metrics
+
+Per-observation quality metrics for NenuFAR Cosmic Dawn data. Read QUALITY
+statistics, calibration solutions, power spectra, horizon-filtered variance and
+optional GNSS ionosphere maps; inspect the resulting table, reject poor blocks
+and compute pspipe weights.
+
+**Documentation:** [Read the Docs](https://cosmic-metrics.readthedocs.io/en/latest/)
+
+![Cosmic Metrics object and data flow](https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/raw/main/docs/_static/workflow.svg)
+
+[How the objects work together](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/overview.html)
+explains the diagram.
+
+## Install
+
+Requires Python 3.10+ and the NenuFAR processing stack (`aoquality`, `ps_eor`,
+`pspipe`), installed with it from PyPI:
+
+```console
+python -m pip install cosmic-metrics
+python -m pip install 'cosmic-metrics[report]'  # Optional interactive HTML report
+python -m pip install 'cosmic-metrics[iono]'  # Optional spinifex map reader
+```
+
+From a checkout, use `python -m pip install -e .` with the same extras.
+
+## Python workflow
+
+A `Dataset` contains observation blocks, named source readers and a managed raw
+cache. Metric objects define calculations. `compute_metrics()` returns an
+independent `MetricTable` containing scalars and observation metadata.
+
+```python
+import cosmic_metrics as cm
+from cosmic_metrics import metrics as m
+
+dataset = cm.Dataset.from_toml('metrics.toml')
+noise = m.RegionPower('dt_V', **m.REG_B)
+phase = m.PhaseChange(120, source='solutions_l2a')
+table = dataset.compute_metrics([noise, phase], n_workers=1)
+print(table.df)
+print(table.missing_reasons)
+
+# Example threshold only; choose science cuts for your dataset.
+selection = table.select_blocks([
+    cm.Rule(phase.name, '>', 3, scale='mad'),
+], missing='reject')
+weights = table.compute_weights(noise.name, selection=selection)
+table.save('metrics.csv')
+selection.save('selection.json', weights=weights)
+
+print(dataset.cache.inspect())
+preview = dataset.cache.clear(stale_only=True, dry_run=True)
+```
+
+Adapt [examples/nt04_l3.toml](https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/blob/main/examples/nt04_l3.toml) to your input paths and save it
+as `metrics.toml`. QUALITY and calibration templates are resolved per block;
+source names choose which reader a metric uses. All block times are UTC MJD
+seconds. Cache validity uses exact time bounds and dependency signatures.
+For scripts with `n_workers > 1`, use a main guard.
+
+Prepare the expensive raw arrays using only the dataset TOML, without defining
+metrics:
+
+```console
+cosmic-metrics cache populate metrics.toml --dry-run
+cosmic-metrics cache populate metrics.toml
+cosmic-metrics cache status metrics.toml
+```
+
+The example uses Slurm/libpipe; set `[worker].nodes="localhost"` for local
+population, or use SSH host names/ranges. The CLI accepts the same host and worker
+overrides as nenucal, for example `--hosts localhost --max-concurrent 4`. It runs
+in the foreground; use a persistent terminal session for unattended work.
+Use `--slurm-nodes 4` to override the Slurm allocation size for one run while
+retaining the account, partition and other Slurm options from the TOML.
+Valid cache entries are skipped; use `--force` to rebuild them explicitly.
+`[cache].require=true` makes later notebook computations raise on
+missing/stale PS or variance entries. QUALITY and calibration remain uncached.
+Run population before the Python example when using that configuration. See
+[cache preparation](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache_population.html)
+for worker settings, progress logs and resuming interrupted jobs.
+
+## Command-line workflow
+
+The dataset TOML can also define the metrics (`[[metrics]]`), selection stages
+(`[[selection]]`) and an output directory, so that no Python is needed:
+
+```console
+cosmic-metrics compute metrics.toml --dry-run   # list blocks and metric columns
+cosmic-metrics compute metrics.toml             # metrics.csv
+cosmic-metrics select metrics.toml              # selection.json
+cosmic-metrics report metrics.toml              # interactive report.html
+```
+
+The report shows every metric against LST or date, with linked selection across
+metric panels, the profile of a selected block in all metrics and a night × LST
+grid. See the
+[command-line guide](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/command_line.html).
+
+For RFI checks in a specific frequency band, configure a `variance` source and
+compute `m.BandVariance((67, 68))` or `m.BandVarianceRatio((67, 68))`. These reduce
+the horizon-filtered channel variance around 67–68 MHz; the ratio uses `dt_V`
+as its noise reference. See the
+[band-variance guide](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/metrics.html#rfi-diagnostics-in-a-frequency-band)
+for units, filtering and interpretation.
+
+## Documentation
+
+Start with the
+[complete synthetic example](https://cosmic-metrics.readthedocs.io/en/latest/getting_started.html),
+which needs no input data. The guide and generated API reference cover the current
+implementation:
+
+- [Dataset configuration, blocks and paths](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/datasets.html)
+- [Metric definitions, array shapes and physical units](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/metrics.html)
+- [Selection, persistence and pspipe weights](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/selection_weights.html)
+- [Cache inspection, invalidation and cleanup](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache.html)
+- [Cache preparation locally and on Slurm](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache_population.html)
+- [Command-line analysis and report](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/command_line.html)
+- [Custom readers and metrics](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/extending.html)
+- [Python API reference](https://cosmic-metrics.readthedocs.io/en/latest/api/index.html)
+- [Development and documentation builds](https://cosmic-metrics.readthedocs.io/en/latest/development.html)
+
+The earlier metrics prototype is retained on the `eric-prototype` branch.
+
+## Validate and build docs
+
+```console
+python -m pip install -e '.[test,docs,report]'
+python -m pytest
+python -m ruff check src tests docs/conf.py
+python -m sphinx -W --keep-going -b html docs docs/_build/html
+```
+

cosmic_metrics-0.1.1/README.md

@@ -0,0 +1,135 @@
+# cosmic-metrics
+
+Per-observation quality metrics for NenuFAR Cosmic Dawn data. Read QUALITY
+statistics, calibration solutions, power spectra, horizon-filtered variance and
+optional GNSS ionosphere maps; inspect the resulting table, reject poor blocks
+and compute pspipe weights.
+
+**Documentation:** [Read the Docs](https://cosmic-metrics.readthedocs.io/en/latest/)
+
+![Cosmic Metrics object and data flow](https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/raw/main/docs/_static/workflow.svg)
+
+[How the objects work together](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/overview.html)
+explains the diagram.
+
+## Install
+
+Requires Python 3.10+ and the NenuFAR processing stack (`aoquality`, `ps_eor`,
+`pspipe`), installed with it from PyPI:
+
+```console
+python -m pip install cosmic-metrics
+python -m pip install 'cosmic-metrics[report]'  # Optional interactive HTML report
+python -m pip install 'cosmic-metrics[iono]'  # Optional spinifex map reader
+```
+
+From a checkout, use `python -m pip install -e .` with the same extras.
+
+## Python workflow
+
+A `Dataset` contains observation blocks, named source readers and a managed raw
+cache. Metric objects define calculations. `compute_metrics()` returns an
+independent `MetricTable` containing scalars and observation metadata.
+
+```python
+import cosmic_metrics as cm
+from cosmic_metrics import metrics as m
+
+dataset = cm.Dataset.from_toml('metrics.toml')
+noise = m.RegionPower('dt_V', **m.REG_B)
+phase = m.PhaseChange(120, source='solutions_l2a')
+table = dataset.compute_metrics([noise, phase], n_workers=1)
+print(table.df)
+print(table.missing_reasons)
+
+# Example threshold only; choose science cuts for your dataset.
+selection = table.select_blocks([
+    cm.Rule(phase.name, '>', 3, scale='mad'),
+], missing='reject')
+weights = table.compute_weights(noise.name, selection=selection)
+table.save('metrics.csv')
+selection.save('selection.json', weights=weights)
+
+print(dataset.cache.inspect())
+preview = dataset.cache.clear(stale_only=True, dry_run=True)
+```
+
+Adapt [examples/nt04_l3.toml](https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/blob/main/examples/nt04_l3.toml) to your input paths and save it
+as `metrics.toml`. QUALITY and calibration templates are resolved per block;
+source names choose which reader a metric uses. All block times are UTC MJD
+seconds. Cache validity uses exact time bounds and dependency signatures.
+For scripts with `n_workers > 1`, use a main guard.
+
+Prepare the expensive raw arrays using only the dataset TOML, without defining
+metrics:
+
+```console
+cosmic-metrics cache populate metrics.toml --dry-run
+cosmic-metrics cache populate metrics.toml
+cosmic-metrics cache status metrics.toml
+```
+
+The example uses Slurm/libpipe; set `[worker].nodes="localhost"` for local
+population, or use SSH host names/ranges. The CLI accepts the same host and worker
+overrides as nenucal, for example `--hosts localhost --max-concurrent 4`. It runs
+in the foreground; use a persistent terminal session for unattended work.
+Use `--slurm-nodes 4` to override the Slurm allocation size for one run while
+retaining the account, partition and other Slurm options from the TOML.
+Valid cache entries are skipped; use `--force` to rebuild them explicitly.
+`[cache].require=true` makes later notebook computations raise on
+missing/stale PS or variance entries. QUALITY and calibration remain uncached.
+Run population before the Python example when using that configuration. See
+[cache preparation](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache_population.html)
+for worker settings, progress logs and resuming interrupted jobs.
+
+## Command-line workflow
+
+The dataset TOML can also define the metrics (`[[metrics]]`), selection stages
+(`[[selection]]`) and an output directory, so that no Python is needed:
+
+```console
+cosmic-metrics compute metrics.toml --dry-run   # list blocks and metric columns
+cosmic-metrics compute metrics.toml             # metrics.csv
+cosmic-metrics select metrics.toml              # selection.json
+cosmic-metrics report metrics.toml              # interactive report.html
+```
+
+The report shows every metric against LST or date, with linked selection across
+metric panels, the profile of a selected block in all metrics and a night × LST
+grid. See the
+[command-line guide](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/command_line.html).
+
+For RFI checks in a specific frequency band, configure a `variance` source and
+compute `m.BandVariance((67, 68))` or `m.BandVarianceRatio((67, 68))`. These reduce
+the horizon-filtered channel variance around 67–68 MHz; the ratio uses `dt_V`
+as its noise reference. See the
+[band-variance guide](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/metrics.html#rfi-diagnostics-in-a-frequency-band)
+for units, filtering and interpretation.
+
+## Documentation
+
+Start with the
+[complete synthetic example](https://cosmic-metrics.readthedocs.io/en/latest/getting_started.html),
+which needs no input data. The guide and generated API reference cover the current
+implementation:
+
+- [Dataset configuration, blocks and paths](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/datasets.html)
+- [Metric definitions, array shapes and physical units](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/metrics.html)
+- [Selection, persistence and pspipe weights](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/selection_weights.html)
+- [Cache inspection, invalidation and cleanup](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache.html)
+- [Cache preparation locally and on Slurm](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache_population.html)
+- [Command-line analysis and report](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/command_line.html)
+- [Custom readers and metrics](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/extending.html)
+- [Python API reference](https://cosmic-metrics.readthedocs.io/en/latest/api/index.html)
+- [Development and documentation builds](https://cosmic-metrics.readthedocs.io/en/latest/development.html)
+
+The earlier metrics prototype is retained on the `eric-prototype` branch.
+
+## Validate and build docs
+
+```console
+python -m pip install -e '.[test,docs,report]'
+python -m pytest
+python -m ruff check src tests docs/conf.py
+python -m sphinx -W --keep-going -b html docs docs/_build/html
+```

cosmic_metrics-0.1.1/pyproject.toml

@@ -0,0 +1,72 @@
+[project]
+name = "cosmic-metrics"
+version = "0.1.1"
+description = "Per-block quality metrics of NenuFAR Cosmic Dawn data, for block selection and combination weights"
+readme = "README.md"
+authors = [
+    { name = "Eric Le Saux", email = "eric.ar.saos@gmail.com" },
+    { name = "Florent Mertens", email = "florent.mertens@gmail.com" },
+]
+requires-python = ">=3.10"
+dependencies = [
+    "numpy",
+    "pandas",
+    "h5py",
+    "astropy",
+    "matplotlib",
+    "threadpoolctl >= 3.1",
+    "click >= 8.0",
+    "libpipe >= 0.9",
+    "tomli; python_version < '3.11'",
+    "aoquality >= 0.6",
+    "ps_eor >= 1.0",
+    "pspipe >= 0.14",
+]
+
+[project.optional-dependencies]
+iono = ["spinifex >= 2.0"]
+report = ["bokeh >= 3.8"]
+test = ["pytest", "ruff"]
+docs = ["sphinx >= 7.0", "myst-parser >= 3.0", "pydata-sphinx-theme >= 0.16"]
+
+[project.urls]
+Homepage = "https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics"
+Repository = "https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics"
+Documentation = "https://cosmic-metrics.readthedocs.io"
+Issues = "https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/issues"
+
+[project.scripts]
+cosmic-metrics = "cosmic_metrics.cli:main"
+
+[build-system]
+requires = ["poetry-core>=2.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.poetry]
+packages = [{ include = "cosmic_metrics", from = "src" }]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.ruff.lint]
+select = [
+    "E",
+    "W",
+    "F",
+    "I",
+    "B",
+    "UP",
+    "C4",
+    "SIM",
+    "PIE",
+    "NPY",
+    "RUF",
+]
+
+[tool.ruff.lint.per-file-ignores]
+"**/__init__.py" = ["F401"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]

cosmic_metrics-0.1.1/src/cosmic_metrics/__init__.py

@@ -0,0 +1,12 @@
+"""Per-block quality metrics of NenuFAR Cosmic Dawn data, for block selection and weights."""
+
+from . import metrics, sources
+from .analysis import Analysis
+from .blocks import Block, BlockSet
+from .cache import Cache, CacheMissError
+from .dataset import Dataset
+from .report import write_report
+from .selection import Rule, SelectionResult, Stage
+from .table import MetricTable
+
+__version__ = '0.1.1'

cosmic_metrics-0.1.1/src/cosmic_metrics/analysis.py

@@ -0,0 +1,266 @@
+"""Analysis: the metrics, detrending, selection stages and outputs of a dataset TOML."""
+
+import difflib
+import inspect
+import os
+from pathlib import Path
+
+import pandas as pd
+
+from .dataset import _resolve, tomllib
+from .metrics import METRICS
+from .selection import Rule, SelectionResult, Stage
+from .table import INFO, MetricTable
+
+TABLE, SELECTION, REPORT = 'metrics.csv', 'selection.json', 'report.html'
+N_PANELS = 6
+
+
+def _check_args(func, where, *args, **kwargs):
+    signature = inspect.signature(func)
+    try:
+        signature.bind(*args, **kwargs)
+    except TypeError as exc:
+        names = ', '.join(list(signature.parameters)[len(args):])
+        raise ValueError(f'{where}: {exc} (arguments: {names})') from None
+
+
+def _unknown(name, names, what):
+    close = difflib.get_close_matches(name, names, n=1)
+    hint = f'; did you mean {close[0]}?' if close else ''
+    return ValueError(f'{what} {name!r} is not a defined metric{hint}')
+
+
+def _tables(cfg, key, path):
+    entries = cfg.get(key, [])
+    if not isinstance(entries, list) or not all(isinstance(e, dict) for e in entries):
+        raise ValueError(f'{path}: write each entry of {key} as a [[{key}]] table')
+    return entries
+
+
+def _section(cfg, key, allowed, path):
+    section = cfg.get(key, {})
+    unknown = set(section) - set(allowed)
+    if unknown:
+        raise ValueError(f'{path}: unknown [{key}] options: {", ".join(sorted(unknown))}')
+    return section
+
+
+class Analysis:
+    """Metric definitions, detrending, selection stages and output paths.
+
+    Construct with Python objects or :meth:`from_toml`. The ``cosmic-metrics
+    compute``, ``select`` and ``report`` commands run the same methods.
+
+    Args:
+        metrics (iterable of Metric): Definitions with unique output names.
+        detrend (iterable of (str, dict)): Metric name and :meth:`MetricTable.detrend`
+            arguments; each adds one column after the metrics.
+        stages (iterable of Stage): Selection stages applied in order.
+        output_dir (str or pathlib.Path or None): Directory of the written files.
+        panels (list of str or None): Report panels shown at load; None shows the
+            metrics used by the rules, then the first metrics, up to six.
+        title (str or None): Report title.
+
+    Attributes:
+        names (list of str): Columns of the computed table, metrics then detrended.
+        table_path (pathlib.Path): metrics.csv in output_dir.
+        selection_path (pathlib.Path): selection.json in output_dir.
+        report_path (pathlib.Path): report.html in output_dir.
+
+    Raises:
+        ValueError: Duplicate columns, or detrend, rules or panels naming an
+            undefined metric.
+    """
+
+    def __init__(self, metrics, detrend=(), stages=(), output_dir=None, panels=None, title=None):
+        self.metrics = list(metrics)
+        self.detrend = [(metric, dict(opts)) for metric, opts in detrend]
+        self.stages = list(stages)
+        self.output_dir = None if output_dir is None else Path(output_dir)
+        self.panels = None if panels is None else list(panels)
+        self.title = title or 'Block quality metrics'
+        names = self.names
+        duplicate = sorted({n for n in names if names.count(n) > 1})
+        if duplicate:
+            raise ValueError(f'Duplicate metric columns: {", ".join(duplicate)}')
+        base = [m.name for m in self.metrics]
+        for metric, opts in self.detrend:
+            if metric not in base:
+                raise _unknown(metric, base, 'Detrended metric')
+            if opts.get('mode', 'difference') not in ('difference', 'ratio'):
+                raise ValueError(f'detrend mode must be difference or ratio, not '
+                                 f'{opts["mode"]!r}')
+            if opts.get('against', 'lst') not in INFO + base:
+                raise ValueError(f'detrend against must be a table column, not '
+                                 f'{opts["against"]!r}')
+        stage_names = [s.name for s in self.stages]
+        if len(set(stage_names)) != len(stage_names):
+            raise ValueError('Duplicate selection stage names')
+        for name in [r.metric for s in self.stages for r in s.rules] + (self.panels or []):
+            if name not in names:
+                raise _unknown(name, names, 'Column')
+
+    @classmethod
+    def from_toml(cls, path):
+        """Read ``[[metrics]]``, ``[[selection]]``, ``[output]`` and ``[report]``.
+
+        Args:
+            path (str or pathlib.Path): Dataset TOML. Metric entries hold ``type``
+                (a class of :mod:`cosmic_metrics.metrics`), its constructor arguments
+                and an optional ``detrend`` table. Selection entries hold ``name``,
+                ``rules`` (:class:`Rule` fields), ``missing`` and ``exclude``.
+
+        Returns:
+            Analysis: ``[output].dir`` is relative to the TOML directory. Blocks and
+            sources are read by :meth:`Dataset.from_toml`, not here.
+
+        Raises:
+            ValueError: Unknown metric types, arguments or options.
+        """
+        path = os.fspath(path)
+        with open(path, 'rb') as f:
+            cfg = tomllib.load(f)
+        metrics, detrend = [], []
+        for i, entry in enumerate(_tables(cfg, 'metrics', path), 1):
+            where = f'{path}: [[metrics]] {i}'
+            opts = dict(entry)
+            kind = opts.pop('type', None)
+            if kind not in METRICS:
+                raise ValueError(f'{where}: type must be one of {", ".join(METRICS)}, '
+                                 f'not {kind!r}')
+            trend = opts.pop('detrend', None)
+            _check_args(METRICS[kind], f'{where} ({kind})', **opts)
+            metrics.append(METRICS[kind](**opts))
+            if trend is not None:
+                if not isinstance(trend, dict):
+                    raise ValueError(f'{where}: detrend must be a table of MetricTable.detrend '
+                                     f'arguments')
+                _check_args(MetricTable.detrend, f'{where} detrend', None, metrics[-1].name,
+                            **trend)
+                detrend.append((metrics[-1].name, trend))
+        stages = []
+        for i, entry in enumerate(_tables(cfg, 'selection', path), 1):
+            where = f'{path}: [[selection]] {i}'
+            _check_args(Stage, where, **{'name': '', **entry})
+            opts = dict(entry)
+            rules = []
+            for j, rule in enumerate(opts.pop('rules', []), 1):
+                _check_args(Rule, f'{where} rule {j}', **rule)
+                rules.append(Rule(**rule))
+            stages.append(Stage(rules=rules, **{'name': f'stage{i}', **opts}))
+        root = os.path.dirname(os.path.abspath(path))
+        output = _resolve(_section(cfg, 'output', ['dir'], path), root)
+        report = _section(cfg, 'report', ['panels', 'title'], path)
+        return cls(metrics, detrend, stages, output.get('dir'), report.get('panels'),
+                   report.get('title'))
+
+    @property
+    def names(self):
+        against = inspect.signature(MetricTable.detrend).parameters['against'].default
+        return ([m.name for m in self.metrics]
+                + [opts.get('name') or f'{metric}_detrend_{opts.get("against", against)}'
+                   for metric, opts in self.detrend])
+
+    def _output(self, name):
+        if self.output_dir is None:
+            raise ValueError('No output directory; set [output].dir in the dataset TOML')
+        return self.output_dir / name
+
+    @property
+    def table_path(self):
+        return self._output(TABLE)
+
+    @property
+    def selection_path(self):
+        return self._output(SELECTION)
+
+    @property
+    def report_path(self):
+        return self._output(REPORT)
+
+    def compute_metrics(self, dataset, n_workers=1, verbose=True):
+        """Compute the metrics on a dataset, then add the detrended columns.
+
+        Args:
+            dataset (Dataset): Blocks and sources, e.g. from the same TOML.
+            n_workers (int): Positive process count.
+            verbose (bool): Print the progress of :meth:`Dataset.compute_metrics`.
+
+        Returns:
+            MetricTable: Columns in the order of :attr:`names`.
+        """
+        table = dataset.compute_metrics(self.metrics, n_workers=n_workers, verbose=verbose)
+        for metric, opts in self.detrend:
+            table = table.detrend(metric, **opts)
+        return table
+
+    def load_table(self):
+        """Read the table written by ``cosmic-metrics compute``.
+
+        Raises:
+            ValueError: Its metrics differ from :attr:`names`.
+        """
+        table = MetricTable.load(self.table_path)
+        missing = [n for n in self.names if n not in table.metric_names]
+        extra = [n for n in table.metric_names if n not in self.names]
+        if missing or extra:
+            raise ValueError(f'{self.table_path} does not hold the metrics of the TOML '
+                             f'(missing: {", ".join(missing) or "none"}; other: '
+                             f'{", ".join(extra) or "none"}); run cosmic-metrics compute')
+        return table
+
+    def select_blocks(self, table, verbose=False):
+        """Apply the stages in order; each sees only the blocks kept so far.
+
+        Args:
+            table (MetricTable): Table with the rule columns.
+            verbose (bool): Print the rejected count of each stage.
+
+        Returns:
+            SelectionResult: Every block of the table. Reasons are prefixed with the
+            stage name, e.g. ``early: Std_mean_XX > 4 MAD (5.2)``. Without stages,
+            all blocks are kept.
+        """
+        keep = pd.Series(True, index=table.df.index, name='keep')
+        reasons = {obs_id: [] for obs_id in table.df.index}
+        for stage in self.stages:
+            ids = list(keep.index[keep])
+            excluded = set(stage.exclude) & set(ids)
+            ids = [obs_id for obs_id in ids if obs_id not in excluded]
+            rejected = sorted(excluded)
+            for obs_id in rejected:
+                reasons[obs_id].append(f'{stage.name}: excluded')
+            if stage.rules and ids:
+                sub = MetricTable(table.df.loc[ids], table.metric_names, table.missing_reasons)
+                result = sub.select_blocks(stage.rules, stage.missing)
+                for obs_id in result.rejected_ids():
+                    reasons[obs_id].extend(f'{stage.name}: {r}'
+                                           for r in result.rejection_reasons[obs_id])
+                    rejected.append(obs_id)
+            keep[rejected] = False
+            if verbose:
+                print(f'{stage.name}: {len(rejected)} / {len(ids) + len(excluded)} blocks '
+                      f'rejected', flush=True)
+        return SelectionResult(keep, reasons, table.df['night'])
+
+    def write_report(self, table, selection=None, path=None):
+        """Write the interactive HTML report (needs Bokeh, the ``report`` extra).
+
+        Args:
+            table (MetricTable): Blocks and metrics to show.
+            selection (SelectionResult or None): Kept/rejected status and reasons.
+            path (str or pathlib.Path or None): Destination; default report_path.
+
+        Returns:
+            pathlib.Path: Written file.
+        """
+        from .report import write_report
+
+        panels = self.panels
+        if panels is None:
+            used = [r.metric for s in self.stages for r in s.rules]
+            panels = list(dict.fromkeys(used + table.metric_names))[:N_PANELS]
+        path = Path(self.report_path if path is None else path)
+        write_report(table, path, selection, panels=panels, title=self.title)
+        return path