metis-test-runner 0.2.0a0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,46 @@
1
+ # External repositories (cloned separately via metis-meta-package bootstrap)
2
+ METIS_Pipeline/
3
+ METIS_Simulations/
4
+ METIS_DRLD/
5
+ metis-meta-package/
6
+ MetisWISE/
7
+ inst_pkgs/
8
+
9
+ # Test/scratch YAML files (root-level only; examples/ are tracked)
10
+ /*.yaml
11
+
12
+ # Local test data and pipeline outputs
13
+ data/
14
+ output/
15
+ logs/
16
+ uv-cache/
17
+ EDPS_data/
18
+
19
+ # Claude Code internal files
20
+ .claude/
21
+
22
+ # Pipeline environment (generated by Install tab; contains local paths)
23
+ .env
24
+
25
+ # Python
26
+ __pycache__/
27
+ *.py[cod]
28
+ *.egg-info/
29
+ .venv/
30
+ dist/
31
+ build/
32
+ *.egg
33
+ uv.lock
34
+ *.log
35
+
36
+ # FITS files (large binary data)
37
+ *.fits
38
+ *.fits.gz
39
+
40
+ # OS / editor
41
+ .DS_Store
42
+ *.swp
43
+ *.swo
44
+ *~
45
+ .idea/
46
+ .vscode/
@@ -0,0 +1,53 @@
1
+ # Changelog
2
+
3
+ ## 0.2.0
4
+
5
+ First PyPI release as `metis-test-runner`.
6
+
7
+ ### Changed
8
+ - Restructured the project as a proper Python package under
9
+ `src/metis_test_runner/`. Modules are now imported as
10
+ `from metis_test_runner import gui, run_metis, archive`.
11
+ - Switched the build backend from `uv` to `hatchling`. The `[tool.uv]` and
12
+ `[dependency-groups]` sections are gone. Pipeline dependencies that live on
13
+ ESO mirrors (`pycpl`, `edps`, `pyesorex`, `adari_core`) are still installed
14
+ at runtime by the GUI's Install tab — they are not in `[project.dependencies]`
15
+ because they are not on PyPI.
16
+ - The GUI's Install tab and the Archive tab's MetisWISE installer now shell
17
+ out to `pip` (via `sys.executable -m pip`) with `--extra-index-url` for the
18
+ ESO and pycpl mirrors. `uv` is no longer required to use MTR.
19
+ - Runtime data (the `METIS_Pipeline` and `METIS_Simulations` clones, `.env`,
20
+ instrument packages, DRLD) now lives in a user data directory resolved via
21
+ `platformdirs` — `~/.local/share/metis-test-runner/` on Linux. Override the
22
+ location with the `METIS_DATA_DIR` environment variable. The per-asset env
23
+ vars (`METIS_META_PKG`, `METIS_SIMULATIONS_DIR`, `METIS_INST_PKGS`) still
24
+ work for back-compat.
25
+ - The CLI is now invoked as `mtr-cli` (or `python -m metis_test_runner.run_metis`)
26
+ instead of `python src/run_metis.py`.
27
+
28
+ ### Added
29
+ - `mtr` and `mtr-cli` console-script entry points. After installation, both
30
+ are on `PATH` and can be invoked from any directory.
31
+ - `metis_test_runner.paths` — a single seam for every filesystem location MTR
32
+ reads or writes outside the package itself.
33
+ - `metis_test_runner.indexes` — ESO mirror URL constants.
34
+ - BSD-3-Clause `LICENSE` file.
35
+ - Trusted-publishing GitHub workflow (`.github/workflows/publish.yml`)
36
+ publishes to PyPI on `v*.*.*` tag pushes.
37
+
38
+ ### Removed
39
+ - `launch.sh` — superseded by the `mtr` console script. Developers now use
40
+ `pip install -e .[dev]` to set up a working copy.
41
+
42
+ ### Notes / known limitations
43
+ - The `--runner metapkg` mode of `mtr-cli` still shells out to `uv run --project
44
+ <metis-meta-package>`. This is the only remaining `uv` dependency and is
45
+ guarded with a clear error message if `uv` is not installed. It will be
46
+ removed once `metis-meta-package` is published to PyPI.
47
+ - `gui.REPO_ROOT` is kept as a back-compat alias pointing at `paths.data_dir()`
48
+ so existing test monkeypatches continue to work. To be removed in a future
49
+ release.
50
+
51
+ ## 0.1.0
52
+
53
+ Internal pre-PyPI release. Distributed via `git clone` + `./launch.sh`.
@@ -0,0 +1,28 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2025, AstarVienna
4
+
5
+ Redistribution and use in source and binary forms, with or without
6
+ modification, are permitted provided that the following conditions are met:
7
+
8
+ 1. Redistributions of source code must retain the above copyright notice, this
9
+ list of conditions and the following disclaimer.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright notice,
12
+ this list of conditions and the following disclaimer in the documentation
13
+ and/or other materials provided with the distribution.
14
+
15
+ 3. Neither the name of the copyright holder nor the names of its
16
+ contributors may be used to endorse or promote products derived from
17
+ this software without specific prior written permission.
18
+
19
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,379 @@
1
+ Metadata-Version: 2.4
2
+ Name: metis-test-runner
3
+ Version: 0.2.0a0
4
+ Summary: GUI and CLI wrapper for end-to-end ESO METIS pipeline testing
5
+ Project-URL: Homepage, https://github.com/AstarVienna/MTR
6
+ Project-URL: Issues, https://github.com/AstarVienna/MTR/issues
7
+ Project-URL: Changelog, https://github.com/AstarVienna/MTR/blob/main/CHANGELOG.md
8
+ Author: AstarVienna
9
+ License: BSD-3-Clause
10
+ License-File: LICENSE
11
+ Keywords: ELT,ESO,METIS,astronomy,pipeline,scopesim
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Environment :: X11 Applications :: Qt
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: License :: OSI Approved :: BSD License
16
+ Classifier: Operating System :: MacOS
17
+ Classifier: Operating System :: POSIX :: Linux
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Scientific/Engineering :: Astronomy
21
+ Requires-Python: <3.14,>=3.12
22
+ Requires-Dist: platformdirs>=4.0
23
+ Requires-Dist: pyqt6-qt6==6.6.0
24
+ Requires-Dist: pyqt6==6.6.0
25
+ Requires-Dist: python-dotenv>=1.0
26
+ Requires-Dist: pyyaml
27
+ Provides-Extra: dev
28
+ Requires-Dist: build; extra == 'dev'
29
+ Requires-Dist: pytest; extra == 'dev'
30
+ Requires-Dist: twine; extra == 'dev'
31
+ Description-Content-Type: text/markdown
32
+
33
+ # METIS Test Runner
34
+
35
+ <p align="center">
36
+ <a href="https://github.com/eiseleb47/MTR/actions/workflows/unit_tests.yaml"><img src="https://img.shields.io/github/actions/workflow/status/eiseleb47/MTR/unit_tests.yaml?branch=main&label=unit%20tests&style=for-the-badge&labelColor=1e1e2e&color=a6e3a1&logo=github&logoColor=cdd6f4" alt="Unit Tests"></a>
37
+ <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3-89b4fa?style=for-the-badge&labelColor=1e1e2e&logo=python&logoColor=cdd6f4" alt="Python 3"></a>
38
+ <a href="https://github.com/eiseleb47/MTR/commits/main"><img src="https://img.shields.io/github/last-commit/eiseleb47/MTR?style=for-the-badge&labelColor=1e1e2e&color=cba6f7&logo=git&logoColor=cdd6f4" alt="Last Commit"></a>
39
+ <a href="https://github.com/eiseleb47/MTR"><img src="https://img.shields.io/badge/platform-linux-fab387?style=for-the-badge&labelColor=1e1e2e&logo=linux&logoColor=cdd6f4" alt="Platform"></a>
40
+ </p>
41
+
42
+ A graphical front-end for end-to-end testing of the [METIS instrument pipeline](https://github.com/AstarVienna/METIS_Pipeline). It generates synthetic FITS observations via [ScopeSim](https://scopesim.readthedocs.io/) and then runs the matching [EDPS](https://www.eso.org/sci/software/edps/) reduction workflow — all from a single, self-contained GUI. A command-line interface (`mtr-cli`) is also shipped as a fallback for scripted or headless use.
43
+
44
+ ## Install
45
+
46
+ MTR is distributed on PyPI. The recommended installer is **pipx**, which creates an isolated venv per application and never touches your system Python.
47
+
48
+ **Step 1 — install pipx** (one-time, skip if you already have it):
49
+
50
+ ```bash
51
+ # Debian / Ubuntu
52
+ sudo apt install pipx && pipx ensurepath
53
+
54
+ # Fedora
55
+ sudo dnf install pipx && pipx ensurepath
56
+
57
+ # macOS (Homebrew)
58
+ brew install pipx && pipx ensurepath
59
+
60
+ # Any platform with Python ≥ 3.12 (fallback)
61
+ python -m pip install --user pipx && pipx ensurepath
62
+ ```
63
+
64
+ **Step 2 — install MTR**:
65
+
66
+ ```bash
67
+ pipx install metis-test-runner
68
+ mtr # launches the GUI from anywhere
69
+ mtr-cli examples/LMS_RAD_06.yaml # CLI equivalent
70
+ ```
71
+
72
+ If you'd rather not install pipx, use Python's built-in venv module:
73
+
74
+ ```bash
75
+ python -m venv ~/.venvs/mtr
76
+ ~/.venvs/mtr/bin/pip install metis-test-runner
77
+ ~/.venvs/mtr/bin/mtr # or: source ~/.venvs/mtr/bin/activate && mtr
78
+ ```
79
+
80
+ On first launch, open the **Install** tab and click *Install / Update* to fetch the METIS_Pipeline, METIS_Simulations, and ESO pipeline dependencies into `~/.local/share/metis-test-runner/` (override with `METIS_DATA_DIR=/path`). All pipeline dependencies install into the same isolated venv that MTR itself lives in — they never leak into your system Python.
81
+
82
+ ## System Dependencies
83
+
84
+ The GUI requires a handful of system libraries for Qt6 / OpenGL rendering. Python, Git, and basic CLI tools (`curl`, `tar`, …) are assumed to already be present.
85
+
86
+ **Debian / Ubuntu**
87
+
88
+ ```bash
89
+ sudo apt install \
90
+ libgl1 libegl1 libglib2.0-0 libxkbcommon0 \
91
+ libfontconfig1 libfreetype6 libdbus-1-3 \
92
+ libxkbcommon-x11-0 libxcb-cursor0 libxcb-keysyms1 libxcb-icccm4 libxcb-shape0
93
+ ```
94
+
95
+ **Fedora**
96
+
97
+ ```bash
98
+ sudo dnf install \
99
+ mesa-libGL mesa-libEGL glib2 libxkbcommon \
100
+ fontconfig freetype dbus-libs \
101
+ libxkbcommon-x11 xcb-util-cursor xcb-util-keysyms xcb-util-wm
102
+ ```
103
+
104
+ **macOS (Homebrew)**
105
+
106
+ On macOS, Qt uses the native Cocoa backend so X11/xcb packages are not needed:
107
+
108
+ ```bash
109
+ brew install freetype fontconfig glib
110
+ ```
111
+
112
+ > **Tip:** if the GUI crashes at startup with *"could not find or load the Qt platform plugin"*, set `QT_DEBUG_PLUGINS=1` to get detailed diagnostics:
113
+ > ```bash
114
+ > QT_DEBUG_PLUGINS=1 mtr
115
+ > ```
116
+ > The output will list exactly which shared library failed to load.
117
+
118
+ ## The GUI
119
+
120
+ The GUI is the recommended way to drive the test runner. It exposes every CLI flag through labelled controls, remembers your settings between sessions, and streams colour-coded live output from the pipeline.
121
+
122
+ Launch it with:
123
+
124
+ ```bash
125
+ mtr
126
+ ```
127
+
128
+ A **Light / Dark theme** button lives in the toolbar and toggles on the fly.
129
+
130
+ ### Install tab
131
+
132
+ The Install tab performs the full pipeline bootstrap non-interactively. Use it if you do **not** already have the pipeline installed. Clicking **Install / Update** will:
133
+
134
+ 1. Clone (or update, if already present) `METIS_Pipeline` and `METIS_Simulations` into the user data directory (`~/.local/share/metis-test-runner/` by default)
135
+ 2. `pip install` all ESO pipeline Python dependencies — `pycpl`, `edps`, `pyesorex`, `adari_core`, `scopesim`, `scopesim_templates` — into the same isolated venv that hosts MTR (via `--extra-index-url` against the ESO mirrors). Out-of-band installs such as MetisWISE from the Archive tab are preserved.
136
+ 3. Write `.env` into the user data directory (environment variables for PYTHONPATH, plugin directories, etc.)
137
+ 4. Initialise and configure EDPS on port 4444
138
+
139
+ Re-running is safe — existing repositories are updated in place rather than re-cloned.
140
+
141
+ **Skip this tab** if you already have the pipeline installed via one of these paths — jump straight to the Run tab instead:
142
+
143
+ - **metis-meta-package** — choose runner `metapkg` and set *Meta-package dir* to your `metis-meta-package` folder
144
+ - **Bare-metal / ESO docs install** — choose runner `native`
145
+ - **Pipeline container** (Docker / Podman) — choose runner `docker` or `podman` and supply the container name
146
+
147
+ ### Archive tab
148
+
149
+ The Archive tab connects to the remote METIS AIT archive via the
150
+ [MetisWISE](https://github.com/AstarVienna/MetisWISE) client. It has two pages:
151
+
152
+ 1. **Install & Configure** — paste the OmegaCEN credentials from the
153
+ [METIS wiki](https://metis.strw.leidenuniv.nl/wiki/doku.php?id=ait:archive)
154
+ and click **Install MetisWISE** to pip-install the package into the same
155
+ isolated venv that hosts MTR. Then fill in the five `[global]` fields (`database_user`,
156
+ `database_password`, `project`, `database_tablespacename`, `database_name`)
157
+ and click **Save & Test Connection**. The values are written to
158
+ `~/.awe/Environment.cfg`; `data_server`, port and protocol are inherited
159
+ from the MetisWISE-packaged default (`metis-ds.hpc.rug.nl:8013`, https).
160
+
161
+ 2. **Query & Download** — filter by raw classification tag or master
162
+ `PRO.CATG`, click **Search**, select files and download them to a local
163
+ directory.
164
+
165
+ To auto-pull missing master calibrations during a pipeline run, add
166
+ `--auto-fetch-calibrations` to the Run tab's options (or the CLI).
167
+
168
+ ### Run tab
169
+
170
+ The Run tab wraps `mtr-cli` in a file-picker UI. All CLI options are exposed as form controls; runner-specific fields (container name, meta-package path) show and hide based on the selected runner.
171
+
172
+ Workflow:
173
+
174
+ 1. **Add input files** via the file browser — YAML observation blocks (`*.yaml`, `*.yml`) and AIT-format CSV test sequences (`*.csv`) may be added freely and mixed in a single run. A live tally (e.g. `2 YAML · 1 CSV`) appears below the list.
175
+ 2. **Tune options** — output directory, CPU cores, auto-calibration, runner mode, pipeline mode (simulate + run, simulate only, pipeline only), simulations directory, instrument packages directory
176
+ 3. **Pick a workflow** (only for CSV-only runs that include the pipeline) — workflow auto-detection reads YAML content, so when the list contains only CSV files and you intend to run the pipeline, the *Workflow* dropdown appears and must be set; otherwise it stays hidden
177
+ 4. **Click Run** — the Run button becomes Stop, and pipeline output streams into the log view with ANSI colouring stripped and stderr highlighted
178
+ 5. **Inspect output** — the pane below the option form shows exactly where simulation frames and pipeline products will be written, updating live as you edit the output path
179
+
180
+ Settings are persisted via `QSettings` and restored on next launch, so you can re-run the last configuration with two clicks.
181
+
182
+ ## Prerequisites (runner modes)
183
+
184
+ Regardless of whether you drive the runner from the GUI or the CLI, the underlying pipeline tools have to live *somewhere*. Three layouts are supported — pick the one that matches your install:
185
+
186
+ **Option A — consolidated install** (runner `metapkg`, default)
187
+
188
+ The Install tab takes care of this automatically. It pip-installs all pipeline
189
+ dependencies (`pycpl`, `edps`, `pyesorex`, `adari_core`, `scopesim`,
190
+ `scopesim_templates`) into the same isolated venv that hosts MTR, clones
191
+ `METIS_Pipeline` and `METIS_Simulations` into the user data directory
192
+ (`~/.local/share/metis-test-runner/` by default), and writes a `.env` file
193
+ there. The runner reads that `.env` automatically.
194
+
195
+ Use the GUI's *Meta-package dir* field (or `--meta-pkg` / `--simulations-dir`)
196
+ to point at an external installation if needed.
197
+
198
+ **Option B — Docker or Podman container** (runner `docker` / `podman`)
199
+
200
+ Build and start the pipeline container from [METIS_Pipeline/toolbox/](https://github.com/AstarVienna/METIS_Pipeline/tree/main/toolbox):
201
+
202
+ ```bash
203
+ cd METIS_Pipeline/toolbox
204
+ docker build -t metispipeline .
205
+ docker run -d --name metis-pipeline --net=host \
206
+ --mount type=bind,source=/path/to/output,target=/output \
207
+ metispipeline
208
+ ```
209
+
210
+ Then in the GUI, set runner to `docker` (or `podman`) and enter the container name. The output directory must be bind-mounted into the container so EDPS can write products back to the host.
211
+
212
+ **Option C — bare-metal or inside a container** (runner `native`)
213
+
214
+ If the pipeline tools (`edps`, `python`, ScopeSim) are already on your PATH — either because you are running *inside* a container, or have installed everything directly — no extra setup is needed. Select runner `native`.
215
+
216
+ ScopeSim instrument packages (Armazones, ELT, METIS) will be downloaded into `./inst_pkgs/` in your current working directory on first use. Set the GUI's *Instrument packages* field (or `--inst-pkgs PATH`) to download or reuse packages from a fixed location instead.
217
+
218
+ > **Tip:** always launch the GUI (or invoke `mtr-cli`) from the same directory — otherwise ScopeSim will download a fresh copy of the instrument packages into every new directory, cluttering your filesystem.
219
+
220
+ ## Input Formats
221
+
222
+ Two input formats are supported and may be mixed in a single run:
223
+
224
+ - **YAML observation blocks** (`*.yaml`, `*.yml`) — the primary, human-authored format. Workflow auto-detection works on YAML content.
225
+ - **AIT-format CSV test sequences** (`*.csv`) — the AIT performance-test sheet exported as CSV. Parsed by `metis_simulations.csvParser` at simulation time. Because MTR does **not** inspect CSV content itself, CSV-only runs that execute the pipeline require an explicit workflow choice (the GUI's *Workflow* dropdown, or `--workflow NAME` on the CLI).
226
+
227
+ See [`examples/small_test.csv`](examples/small_test.csv) for a minimal CSV; the rest of this section covers YAML.
228
+
229
+ ### YAML Format
230
+
231
+ Each top-level key in the YAML is one *observation block*. The workflow (`lm_img`, `n_img`, `ifu`, `lm_lss`, `n_lss`, …) and the deepest pipeline target task are inferred automatically from the YAML content — primarily from `properties.tech`, falling back to `mode`.
232
+
233
+ Required fields per block:
234
+
235
+ ```yaml
236
+ BLOCK_NAME:
237
+ do.catg: <EDPS classification tag> # e.g. DETLIN_IFU_RAW, IFU_SCI_RAW
238
+ mode: <scopesim mode> # e.g. wcu_lms, lms
239
+ source:
240
+ name: <scopesim source name> # e.g. empty_sky, star
241
+ kwargs: {}
242
+ properties:
243
+ dit: <float> # detector integration time (s)
244
+ ndit: <int> # number of integrations
245
+ catg: <CALIB|SCIENCE>
246
+ tech: <LMS|IMAGE,LM|LSS,LM|…>
247
+ type: <DETLIN|DARK|FLAT|…>
248
+ tplname: <ESO template name>
249
+ nObs: <int> # number of exposures to simulate
250
+ ```
251
+
252
+ See `examples/LMS_RAD_06.yaml` for a complete IFU example covering the full calibration + science chain.
253
+
254
+ ## Supported Workflows
255
+
256
+ | EDPS Workflow | `tech` values |
257
+ |---|---|
258
+ | `metis_lm_img_wkf` | `IMAGE,LM` |
259
+ | `metis_n_img_wkf` | `IMAGE,N` |
260
+ | `metis_ifu_wkf` | `LMS`, `IFU`, `RAVC,IFU` |
261
+ | `metis_lm_lss_wkf` | `LSS,LM` |
262
+ | `metis_n_lss_wkf` | `LSS,N` |
263
+ | `metis_lm_ravc_wkf` | `RAVC,LM` |
264
+ | `metis_lm_app_wkf` | `APP,LM` |
265
+ | `metis_pupil_imaging_wkf` | `PUP,LM`, `PUP,N` |
266
+
267
+ ## Output Layout
268
+
269
+ Output is written under the chosen output directory (default: `./output/<timestamp>/`):
270
+
271
+ - `<output-dir>/sim/` — synthetic FITS frames from ScopeSim
272
+ - `<output-dir>/pipeline/` — reduced data products from EDPS
273
+
274
+ The GUI displays the resolved paths live under the *Output directory* field so you can see exactly where products will land before you hit Run.
275
+
276
+ ## Command-Line Fallback
277
+
278
+ `mtr-cli` is the headless interface that the GUI drives under the hood. It is useful for scripting, CI jobs, and SSH sessions without a display. It accepts the same options as the GUI.
279
+
280
+ ```bash
281
+ mtr-cli [OPTIONS] input1.yaml [input2.csv ...]
282
+ ```
283
+
284
+ YAML and CSV inputs may be mixed in any combination.
285
+
286
+ ### Options
287
+
288
+ | Flag | Default | Description |
289
+ |---|---|---|
290
+ | `-o / --output` | `./output/<timestamp>` | Root directory for all outputs (env: `METIS_OUTPUT_DIR`) |
291
+ | `--runner {metapkg,native,docker,podman}` | `metapkg` | Execution mode (see below; env: `METIS_RUNNER`) |
292
+ | `--container NAME` | — | Container name/ID for `docker` / `podman` runners (env: `METIS_CONTAINER`) |
293
+ | `--calib [N]` | `1` | Auto-generate N calibration frames (dark/flat) per unique config, inferred from input content. Pass `--calib 0` to disable. |
294
+ | `--workflow NAME` | auto-detect | Force EDPS workflow name (e.g. `metis.metis_lm_img_wkf`). Required when all inputs are `.csv` and the pipeline will run, because workflow auto-detection is YAML-only. Overrides auto-detection when YAML is also present. |
295
+ | `--cores N` | `4` | CPU cores used for parallel simulations |
296
+ | `--no-sim` | off | Skip simulation; run pipeline on existing FITS data (source defaults to `<output>/sim/` — override with `--pipeline-input`) |
297
+ | `--pipeline-input DIR` | `<output>/sim/` | Directory containing FITS files to feed the pipeline (only with `--no-sim`; env: `METIS_PIPELINE_INPUT`) |
298
+ | `--no-pipeline` | off | Run simulation only; skip EDPS pipeline |
299
+ | `--meta-pkg PATH` | `.` (repo root) | Path to the pipeline environment directory (`metapkg` runner only). Falls back to `./pipeline/` then `./metis-meta-package/`. (env: `METIS_META_PKG`) |
300
+ | `--simulations-dir PATH` | `./METIS_Simulations` (host) or `/home/metis/METIS_Simulations` (container) | Path to ScopeSim scripts (env: `METIS_SIMULATIONS_DIR`) |
301
+ | `--inst-pkgs PATH` | see below | Path to ScopeSim instrument packages (Armazones, ELT, METIS). Defaults to `./inst_pkgs` for `metapkg`/`native`, and container-resolved `./inst_pkgs` for `docker`/`podman` (env: `METIS_INST_PKGS`) |
302
+ | `--auto-fetch-calibrations` | off | Before running the pipeline, query the remote METIS archive (via MetisWISE) for any master calibrations the input set is missing and download them into the pipeline input directory. Requires MetisWISE to be installed and `~/.awe/Environment.cfg` to hold valid credentials — see the Archive tab. |
303
+ | `--prefer-masters` | off | Set EDPS `association_preference` to `master_per_quality_level` for this run, preferring master calibrations over reduced raw data. |
304
+
305
+ ### Runner modes
306
+
307
+ | Mode | When to use |
308
+ |---|---|
309
+ | `metapkg` (default) | You used the GUI's Install tab (or an external `metis-meta-package` install). Pipeline tools are pip-installed alongside MTR in the same isolated venv. For external `metis-meta-package` checkouts, `uv` must be on PATH (`pipx install uv`). |
310
+ | `native` | Tools (`edps`, `python`, ScopeSim) are installed directly on PATH — e.g. you are running **inside** a Docker/Podman container, or have a bare-metal install. |
311
+ | `docker` / `podman` | Tools live inside a container and you are running the script **outside** it. The runner wraps every command with `docker exec` / `podman exec`. |
312
+
313
+ > **Note for `docker` / `podman` runners:** the output directory (`-o`) must be bind-mounted into the container so EDPS can write pipeline products to it. The `--simulations-dir` flag should point to the path of `METIS_Simulations/Simulations` *inside* the container (default: `/home/metis/METIS_Simulations`).
314
+
315
+ ### Examples
316
+
317
+ ```bash
318
+ # Full run with metis-meta-package (default)
319
+ mtr-cli examples/LMS_RAD_06.yaml
320
+
321
+ # Inside a container or bare-metal install (tools on PATH)
322
+ mtr-cli --runner native examples/LMS_RAD_06.yaml
323
+
324
+ # Exec into a running Docker container from the host
325
+ mtr-cli --runner docker --container metis-pipeline examples/LMS_RAD_06.yaml
326
+
327
+ # Exec into a running Podman container; set runner via env var
328
+ METIS_RUNNER=podman METIS_CONTAINER=metis-pipeline mtr-cli examples/LMS_RAD_06.yaml
329
+
330
+ # Multiple YAML files, custom output dir, with auto-calibration frames
331
+ mtr-cli -o /tmp/myrun --calib obs1.yaml obs2.yaml
332
+
333
+ # Crank up parallelism for big simulation batches
334
+ mtr-cli --cores 12 examples/LMS_RAD_06.yaml
335
+
336
+ # Only simulate, inspect the FITS files manually
337
+ mtr-cli --no-pipeline examples/LMS_RAD_06.yaml
338
+
339
+ # Only run the pipeline on previously simulated data
340
+ mtr-cli --no-sim -o /tmp/myrun examples/LMS_RAD_06.yaml
341
+
342
+ # Pipeline-only with FITS files from a custom location
343
+ mtr-cli --no-sim --pipeline-input /data/sim_fits -o /tmp/myrun
344
+
345
+ # CSV-only input, simulate only (workflow auto-detection not needed)
346
+ mtr-cli --no-pipeline examples/small_test.csv
347
+
348
+ # CSV-only input, full simulate + pipeline run (workflow must be explicit)
349
+ mtr-cli --workflow metis.metis_lm_img_wkf examples/small_test.csv
350
+
351
+ # Mixed YAML + CSV in a single run
352
+ mtr-cli examples/small_test.yaml examples/small_test.csv
353
+ ```
354
+
355
+ ## Repository Layout
356
+
357
+ ```
358
+ MTR/
359
+ ├── src/metis_test_runner/
360
+ │ ├── gui.py # Graphical front-end (PyQt6) — primary entry point
361
+ │ ├── run_metis.py # Headless CLI (used directly or wrapped by the GUI)
362
+ │ ├── archive.py # MetisWISE archive integration
363
+ │ ├── paths.py # User data directory resolution (platformdirs)
364
+ │ ├── indexes.py # ESO mirror pip-index URL constants
365
+ │ └── examples/ # Bundled YAML / CSV example inputs
366
+ ├── container/
367
+ │ ├── Dockerfile # Ubuntu 24.04 GUI container (Qt6 / Wayland)
368
+ │ └── compose.yml # Podman / Docker Compose for the GUI service
369
+ ├── tests/ # Unit tests (pytest)
370
+ └── pyproject.toml # Project metadata (hatchling build backend)
371
+ ```
372
+
373
+ ## Related Repositories
374
+
375
+ This runner is designed to work alongside the following repos, which are installed via the GUI's Install tab:
376
+
377
+ - **[METIS_Pipeline](https://github.com/AstarVienna/METIS_Pipeline)** — the core Python/C pipeline, EDPS workflows, and PyEsoRex recipes
378
+ - **[METIS_Simulations](https://github.com/AstarVienna/METIS_Simulations)** — ScopeSim scripts that generate synthetic FITS observations for each observing mode
379
+ - **[metis-meta-package](https://github.com/eiseleb47/metis-meta-package)** — legacy standalone meta-installer (still supported as a fallback; the Install tab now installs pipeline dependencies directly into the main virtual environment)