oceanarray 0.5.0__tar.gz → 0.5.1__tar.gz

{oceanarray-0.5.0 → oceanarray-0.5.1}/.github/workflows/tests.yml

@@ -43,9 +43,17 @@ jobs:
       - name: Run unit tests
         # Coverage gate runs once, on the canonical ubuntu + 3.11 leg; the extra
         # version legs run --no-cov so they only verify install + test pass.
+        env:
+          # Windows: the netCDF4 wheel points HDF5_PLUGIN_PATH at its bundled plugins
+          # folder, and HDF5 rescans it on every filter query -- xarray calls
+          # var.filters() once per variable, so each NETCDF4 open costs ~36 ms/variable
+          # on the hosted runner. "::" preloads no plugins. oceanarray writes zlib
+          # (built into HDF5), so only zstd/bzip2/blosc files would need them -- none
+          # here. No effect on Linux/macOS.
+          HDF5_PLUGIN_PRELOAD: "::"
         run: >
           pytest tests/unit -m "not slow and not local and not needs_seasenselib" -q
-          ${{ (matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11') && '--cov=oceanarray --cov-report=term-missing --cov-fail-under=38' || '--no-cov' }}
+          ${{ (matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11') && '--cov=oceanarray --cov-report=term-missing --cov-fail-under=58' || '--no-cov' }}
 
   integration:
     runs-on: ubuntu-latest

{oceanarray-0.5.0 → oceanarray-0.5.1}/CHANGELOG.md

@@ -6,6 +6,48 @@ All notable changes to oceanarray are documented here.
 
 ## [Unreleased]
 
+## [0.5.1] — 2026-10-23
+
+> The units-display entries — *Units tables come from the shared report layer*, *N² keeps the spelling it arrives with*, *Legacy vocabulary files moved and renamed*, and the two *Deprecated* items — landed as direct-to-main commits (`cdcffc9`/`8982831`/`735adaa`, the units display sweep), not a pull request; [#107](https://github.com/ocean-uhh/oceanarray/pull/107) documented the sweep and added the derived-quantity guard. See `.claude/done/ROADMAP.md`.
+
+### Changed
+
+- **Post-release tidy** ([#105](https://github.com/ocean-uhh/oceanarray/pull/105)): `find_best_dtype` passes `timedelta64` variables through with their native kind instead of miscasting them; the dead `proc_machine` report-context key is removed; the CLI reference page is generated from the argument parser (`sphinx-argparse`); CONTRIBUTING and the README scope line refreshed.
+
+- **Vendored report-design layer adopted** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): the report presentation layer moves to `oceanarray/reports/_vendored/`, a directory copied byte-identical from ctdcast `15fa38b` that carries its own `MANIFEST.txt` and self-check (`check.py`). The six scattered vendored files (`config/report_tokens.py`, `config/report.mplstyle`, `reports/_encode.py`, `_css.py`, `_manifest.py`, `templates/_nav.html`), the root `VENDORED.txt`, and `scripts/check_vendored.py` are removed; a new owned `reports/design.py` holds the oceanarray-specific configuration (package accent, QC/sensor predicates, `ATTR_GROUP_EXTRAS`, the per-figure source toggle). `reports/__init__.py` resolves its public names lazily so importing the vendored layer does not pull the page pipeline. No change to the NetCDF outputs or the public API.
+- **NetCDF-inventory tables rendered by the vendored builders** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): the coordinates/variables/scalars/globals tables now come from the shared `nc_tables` builders and `_nc_tables.html` macros. Type labels read `string` rather than `str`, each table carries a `source:` caption, the coordinates table gains a Dims column, and the `history` global attribute is shown once (in its dedicated processing-history panel, excluded from the globals table via the builder's `exclude=`). An all-empty Standard/long-name column drops along with units/min-max/attributes (`optional=` includes `stdname`). The QC-companion marker shows the file's real suffix — `(_qc)` for oceanarray's lowercase QC variables, not a hard-coded `(_QC)` — and is coloured with the shared `--qc-mark` token. The local inventory templates and row builders are removed.
+- **Processing-history panel is now tables** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): the instrument/grid/stack reports render the processing history as a key/value `properties_table` for the header items plus a sortable Time·Event table for the dated entries (a `|`-joined entry splits into same-time rows), replacing the `dl`-style list that wrapped long entries into orphan rows. The Time·Event table carries a leading `#` column — the row's 1-based position in the `history` attribute — so sorting any column never loses the original sequence (events sharing one timestamp would otherwise be unorderable).
+- **Array page map** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): the mooring-position map is the vendored Leaflet `track_map` (a marker per mooring, each carrying a permanent name label in the marker's colour, hover-linked to the mooring list via `data-track-id`, offline bundled assets) behind the `design.ARRAY_MAP_LEAFLET` flag — the interactive map on screen, its matplotlib PNG fallback when printing (never both). The static PNG gains a `source:` caption, a Mercator-square framed extent (a line of moorings no longer renders as a sliver), and family marker colours; the mooring name links to its report and the jump-nav uses the family markup.
+- **Array masthead metadata** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): the array summary masthead gains deployment/recovery and duration, position and mooring extent, water-depth range, an instrument by-type breakdown, deployment status, and processed-to counts — all read or derived from the array/mooring YAMLs at render (a missing value is an em-dash).
+- **base.html de-duplicated against the shared stylesheet** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): report chrome rules that `emit_css()` already provides are removed from the per-page `<style>` block (only the oceanarray-specific additions remain), guarded by `test_no_base_rule_duplicates_vendored_css`.
+- **Absent figure panels say why** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)): a figure whose draw raises now renders a short stub with the reason instead of vanishing (`as_panel`, ported from ctdcast).
+- **Coverage floor raised** ([#106](https://github.com/ocean-uhh/oceanarray/pull/106)) from 38% to 58%: `main` and this branch both measure 59% unit coverage, so the CI `--cov-fail-under` gate is ratcheted to one point below.
+- **Units tables come from the shared report layer**: the vendored layer is re-copied from ctdcast `f2af1d8`, where `units_display.py` became `units.py` + `units_tables.py` (canonical spellings, accepted aliases with factors, display labels, keyed by CF `standard_name`). `oceanarray.config.unit_tables` holds oceanarray's differences from those tables (N² stored as `s-2`; the `ml l-1`, `mL/L` and `s⁻²` spellings) and `unit_label()`, the display entry point the plotters now call. The storage pass is `processors.units.to_canonical_units`. It accepts the same spellings as before for every registry quantity (N² aside, below), still takes each variable's canonical spelling from `parameters.VARIABLES`, still raises on a missing units attribute, and still records `units_converted_from` / `units_conversion_factor` on a conversion; the dune2 goldens are byte-identical.
+- **Plot-axis units derived from the registry through the pipeline** ([#107](https://github.com/ocean-uhh/oceanarray/pull/107)): `parameters.vunit`/`vlabel` now format each variable's canonical CF `units` through `unit_label` (`degree_Celsius` → `°C`), so every axis/colorbar label is produced by the one display rule; the duplicate `label_units` column is removed from `VARIABLES`. The attitude/engineering variables `pitch`, `roll`, `heading`, `tilt`, `current_direction`, `bin_pressure` are registered (canonical CF `degree`/`dbar`), and the reader spellings `degrees`/`volts` are accepted as aliases of `degree`/`V`; the storage pass rewrites a stored `degrees` to `degree` at factor 1. Rendered labels are unchanged.
+- **Derived-quantity unit guard** ([#107](https://github.com/ocean-uhh/oceanarray/pull/107)): a plot whose output unit is computed from a source variable (displacement in km from velocity, height-above-seabed in m from pressure, instrument spacing in dbar, temperature PSD) warns via `unit_tables.warn_if_not_canonical` when the source variable does not carry the canonical unit the derivation assumes. Silent on every oceanarray-written file (the storage pass guarantees canonical units); it catches a foreign or hand-edited file the derivation would otherwise mislabel.
+- **N² keeps the spelling it arrives with**: `s-2` and `rad2 s-2` are both accepted and neither is rewritten into the other (`rad2 s-2` used to be refused), so N² is the one quantity whose stored spelling can differ between files.
+- **Legacy vocabulary files moved and renamed**: `config/OS1_var_names.yaml`, `config/OS1_sensor_attrs.yaml` and `legacy/OS1_vocab_attrs.yaml` are now `legacy/OS1_variable_names.yaml`, `legacy/OS1_sensor_attributes.yaml` and `legacy/OS1_variable_attributes.yaml`, beside the legacy `convertOS.py` that takes them as inputs. No other oceanarray code reads them; the wheel no longer ships them.
+- **Report figure layout** ([#110](https://github.com/ocean-uhh/oceanarray/pull/110)): a single-tier current rose (one QC tier, no magnetic panel) renders as a third-slot circle centred in whitespace instead of a full-slot-width square; the knockdown-displacement figure reserves left margin for its pressure y-labels so a deep mooring's four-digit dbar labels no longer clip. The unreachable `GRID_COMBINED_HYDRO` grid profile is removed (T-S stays its own section).
+- **`°` accepted as a degree spelling** ([#110](https://github.com/ocean-uhh/oceanarray/pull/110)): the storage pass accepts the `°` symbol as the canonical `degree` (factor 1) for attitude variables. `deg` is not accepted here — it is scoped to latitude/longitude in the vendored unit tables and arrives, scoped, at the next vendored sync.
+- **Report layer re-vendored from ctdcast v0.4.0** ([#111](https://github.com/ocean-uhh/oceanarray/pull/111)): the `degree`/`degrees`/`°` alias moves into the vendored family tables (the duplicate package-side copy is removed; unit resolution is unchanged). The array-map graticule gains edge labels drawn inside the west/south edges, a bottom-right scale bar, and antimeridian handling for basin-scale tracks.
+- **ADCP bin pressures are tilt-corrected for knock-down** ([#112](https://github.com/ocean-uhh/oceanarray/pull/112)): `compute_adcp_bin_pressure` reduces each bin's vertical range by `cos(θ)` (θ = instrument tilt from pitch/roll, `arccos(cos P_true·cos R)`) before converting to pressure, so a knocked-down profiler's bins are placed at their true depth; `bin_pressure` gains `tilt_correction` / `tilt_definition` attributes. When the firmware recorded earth coordinates with bin mapping on (RDI `EX` command), it has already tilt-placed the bins, so the correction is **not** applied (`tilt_correction = "none (firmware bin mapping)"`). Every current UHH deployment is earth + bin-mapping, so stored outputs are unchanged; the correction is exercised only by beam-coordinate input. Stage 1 surfaces `firmware_coordinate_system` / `firmware_bin_mapping` from the raw metadata for the guard.
+
+### Added
+
+- **Readers for files already processed by their originator** ([#108](https://github.com/ocean-uhh/oceanarray/pull/108)): `file_type: oceansites` (OceanSITES 1.3 netCDF, and the OSNAP variant with variables on `TIME`) and `file_type: rodb` (RAPID/BODC `.microcat`, `.edt`, `.lowedt`), in `oceanarray.tools.readers` (`read_oceansites`, `read_rodb`). Variables are renamed to oceanarray names; the originator's QC flags are kept as `<name>_qc_originator`; the originator's metadata as `originator_*` attributes. Velocities are geographic, so Stage 3 applies no rotation and no second magnetic declination (`originator_magnetic_declination`). New instrument class `currentmeter` for a processed point current meter. Tested on trimmed real OSNAP files (WHOI, GEOMAR, Scripps, RODB) in `tests/fixtures/raw/osnap/`.
+- **`units_override`** ([#108](https://github.com/ocean-uhh/oceanarray/pull/108)) per instrument in the mooring YAML corrects a file whose units attribute is wrong; Stage 1 refuses an imported (`oceansites` / `rodb`) instrument whose values fall outside the registry's valid range for the stated units, naming this key.
+- **`stack: false`** ([#108](https://github.com/ocean-uhh/oceanarray/pull/108)) per instrument in the mooring YAML keeps an instrument out of the stack and grid while it is still processed through stages 1-3 and reported (reference data); the stack names it in an `instruments_excluded` attribute. A low-passed file (`originator_low_passed`, e.g. RODB `.lowedt`) reaching the stack without it is omitted with an error naming the key.
+- Unit spellings accepted: `decibar`, `meter/second`, `meter`, `psu` (practical salinity only), and RODB `cm s-1` (converted ×0.01 to `m s-1`).
+
+### Deprecated
+
+- `oceanarray.processors.units.canonicalise_units` → `to_canonical_units`, and `oceanarray.processors.units.format_units` → `oceanarray.config.unit_tables.unit_label`. The old names warn and are removed in 0.6.0.
+- `oceanarray.parameters.UNIT_ALIASES` and `VARIABLE_UNIT_ALIASES`: the spellings now live in the shared units tables (`oceanarray.config.unit_tables.TABLES`, with `PSU` / `NTU` scoped by CF `standard_name`). Both names still resolve, rebuilt from those tables, with a warning; removed in 0.6.0.
+
+### Breaking changes
+
+- None. The NetCDF outputs and the public API (`oceanarray` CLI, `from oceanarray.reports import MooringReport`) are unchanged. The report HTML changes (masthead nav, inventory-table layout and type labels, `source:` captions); the committed report goldens are rebaselined to match.
+
 ## [0.5.0] — 2026-10-12
 
 ### Added

{oceanarray-0.5.0 → oceanarray-0.5.1}/CITATION.cff

@@ -7,8 +7,8 @@ authors:
   affiliation: University of Hamburg, Bundesstraße 53, 20146 Hamburg, Germany
 title: oceanarray
 doi: 10.5281/zenodo.21890564
-version: 0.5.0
-date-released: 2026-10-12
+version: 0.5.1
+date-released: 2026-10-23
 url: https://github.com/ocean-uhh/oceanarray
 repository-code: https://github.com/ocean-uhh/oceanarray
 license: MIT

{oceanarray-0.5.0 → oceanarray-0.5.1}/CONTRIBUTING.md

@@ -79,7 +79,21 @@ For all other bugs:
 - Why it would be useful to other users (not just your specific cruise).
 - Whether you are willing to implement it yourself.
 
-Keep in mind that oceanarray is designed for reproducible processing of moored array data. Features that require cruise-specific raw data, add heavy new dependencies, or break the CF-NetCDF output contract are unlikely to be merged.
+### Scope
+
+oceanarray **processes moored-array observations**: raw instrument files from a mooring (MicroCATs, RBRs, Nortek and RDI current meters and ADCPs, SBE 16/56, Seapoint, …) → CF-NetCDF at each stage — stage 1 (raw → NetCDF), stage 2 (deployment window, clock), stage 3 (QC, velocity rotation, calibration-dip offsets applied) — then a stack on a common time axis and a grid on pressure levels, each step documented in an HTML report. The OceanSITES export of the stacked and gridded products is the next in-scope step.
+
+**Fixed — not open to variation:** the output contract. Variable names, CF standard names and the canonical unit spellings come from one registry (`config/parameters.py`) and are the same in every file the package writes; QC flags follow the QARTOD/OceanSITES vocabulary; every processing parameter is recorded in the output's attributes so a file can be reprocessed from itself. Contributions that change these change every user's files, and are discussed first, as an issue.
+
+**Open — alternative methods welcome, as selectable options with the default unchanged:**
+
+- **Gridding.** The grid step is where the method choices are made — today linear interpolation in pressure onto a fixed `dp` grid, with σ₀ targets for isopycnal products. Alternatives: a **PCHIP spline**; the **RAPID climatological-gradient interpolation** (`tools/rapid_interp.py`, written but **not yet integrated** into `grid.py`); and bounds handling above the shallowest instrument — **extrapolate to the surface**, clip, or mask. All welcome as named methods the user picks in the config, each recording its name and parameters in the output.
+- **Corrections to individual instruments.** How calibration-dip offsets and other per-instrument corrections (drift, pressure-sensor offset) are applied in stage 3 — constant, linear in time, pre/post weighted — is a method choice; the *derivation* of caldip offsets stays in `caldip`, oceanarray applies its output.
+- **QC tests.** New QARTOD-style tests or thresholds, as long as the flags land in the existing `_qc` companions with their thresholds recorded.
+- **Diagnostic figures and report panels.** New figures are welcome, and the report is meant to let a user choose their own selection of panels per page (a config-driven profile); contributions that make that selection easier are in scope.
+- **Readers.** Instrument readers live in `seasenselib`; oceanarray's stage 1 dispatches on the file type (`config/parameters.INSTRUMENT_FILE_TYPES`) and keeps only a small set of its own legacy readers (`tools/readers.py`: RODB and the old Nortek CSV). A reader for an instrument `seasenselib` does not cover is welcome — but write it in `seasenselib` first and add the file type here; a reader that only oceanarray would ever use can live in `tools/readers.py`. (Where exactly that line is drawn is not written down yet — ask.)
+
+**Out of scope, because a sibling owns it:** shipboard CTD processing (`ctdcast`); deriving calibration-dip offsets (`caldip`); glider data (`glidertest`, `seagliderOG1`); transport and AMOC estimates from the gridded products (`AMOCatlas`); general-purpose plotting. Also out: features that need cruise-specific raw data in the repository, heavy new dependencies, or that break the CF-NetCDF contract. If you are unsure whether an idea fits, open an issue describing it rather than deciding alone.
 
 ---
 
@@ -89,10 +103,10 @@ Keep in mind that oceanarray is designed for reproducible processing of moored a
 
 | Layer | Modules | Role |
 |-------|---------|------|
-| Tier 1 | `tools/`, `utilities.py` | Pure functions, no side effects |
+| Tier 1 | `tools/`, `writers/`, `utilities.py`, `inspect.py` | Low-level I/O and pure functions — raw-file readers, the single netCDF writer, dataset inspection |
 | Tier 2 | `processors/stage1.py` … `processors/grid.py` | Processing orchestrators — read, transform, write CF-NetCDF |
 | Tier 3 | `cli.py` | Entry point — argument parsing, subcommand dispatch |
-| — | `plotters/`, `report/` | Visualisation and HTML report generation |
+| — | `plotters/`, `reports/` | Visualisation and HTML report generation |
 | — | `analysis/` | Science utilities (filtering, isopycnals, QC) |
 | — | `config/` | Parameters, variable registry, YAML validation |
 
@@ -117,7 +131,7 @@ PDF report output (WeasyPrint, which needs native pango/cairo libraries), use
 
 1. Fork the repository and create a feature branch from `main`.
 2. Make your changes, following the conventions below.
-3. Run `ruff check . --fix` and then `pytest` — all tests must pass.
+3. Run `ruff check . --fix` and `ruff format .`, then `pytest` — all tests must pass. (`ruff format` owns line wrapping, which is why `E501` is ignored in `pyproject.toml`.)
 4. Open a pull request against `main`. Prefix the title with a bracket tag: `[FEAT]`, `[FIX]`, `[REFACTOR]`, `[DOC]`, `[TEST]`, `[CI]`, or `[CLEANUP]`. (PR titles use bracket tags; individual **commit messages** use conventional-commit prefixes — see below.)
 
 Keep each pull request to one logical change. A rename PR that also fixes a bug is a PR nobody can review — split them. (This matters most once more than one person is working on the code.)
@@ -135,8 +149,8 @@ Co-authored-by: Name <email@example.com>
 - **Docstrings**: NumPy style on every public function and class. Include `Parameters`, `Returns`, and `Raises` sections. Period at the end.
 - **Type annotations**: all public function arguments and return values.
 - **Lint**: `ruff check .` must be clean. No bare `except:`. No suppressed warnings without a comment explaining why.
-- **Units**: never strip or assume units. Use `gsw` for all seawater property calculations.
-- **Plot style**: discrete colorbars (`BoundaryNorm`, ≤ 20 levels); call `plt.style.use(str(P.MPLSTYLE))` at the start of every plot function.
+- **Units**: every registered variable carries the registry's canonical spelling (`config/parameters.VARIABLES[var]["units"]`), and that spelling is identical in every file the package writes — except N², kept as `s-2` or `rad2 s-2` as it arrives (`config/unit_tables.UNIT_SPELLINGS_KEPT`). The pipeline rewrites known spellings through the shared units tables (`config/unit_tables.TABLES`) and refuses unknown ones. A new spelling is added **with evidence** — a reader that emits it — never from memory: to `config/unit_tables.UNIT_ALIAS_EXTRAS` if only oceanarray's readers emit it, or upstream in ctdcast's vendored `units_tables.py` if it belongs to the family. A mass↔volume conversion is never a scalar alias. Use `gsw` for all seawater property calculations.
+- **Plot style**: discrete colorbars (`BoundaryNorm`, ≤ 20 levels). Report figures are styled at one choke point — `reports/_encode.render_b64` applies the vendored `report.mplstyle` and `reports/_slots.render` draws each figure at its slot width — so a report plotter does not call `plt.style.use` itself.
 - **Data provenance**: store all processing parameters (thresholds, coefficients) in NC output attributes so any file can be reprocessed exactly from itself.
 - **No silent defaults**: never substitute a default, guess, or approximation when the correct value cannot be determined. Raise, or warn loudly and record what was assumed in the output's metadata. A plausible wrong number is worse than an error, because nothing downstream can detect it.
 
@@ -150,7 +164,9 @@ pytest tests/unit -m "not slow and not needs_seasenselib"  # without seasenselib
 pytest --cov=oceanarray --cov-report=term-missing -q  # with coverage
 ```
 
-New or changed code must have a test. For anything numerical, assert a **value** you can justify independently of the code — `pytest.approx` or `numpy.testing.assert_allclose` against an analytic case, an invariant that must hold for any input, or a cross-check against `gsw` or another implementation — rather than a value produced by running the code (which proves only that the code has not changed). Do not add tests that pass when nothing happened: `assert result is not None` on a function that returns `None` on failure is the shape to avoid. Integration tests use committed NetCDF fixtures in `tests/fixtures/`.
+New or changed code must have a test. For anything numerical, assert a **value** you can justify independently of the code — `pytest.approx` or `numpy.testing.assert_allclose` against an analytic case, an invariant that must hold for any input, or a cross-check against `gsw` or another implementation — rather than a value produced by running the code (which proves only that the code has not changed). Do not add tests that pass when nothing happened: `assert result is not None` on a function that returns `None` on failure is the shape to avoid.
+
+Fixtures are **real instrument data** — the committed `dune2` raw/proc set in `tests/fixtures/`, trimmed down — not synthetic mocks; a new reader path gets a small real file. Report HTML is **golden-tested** (`tests/fixtures/golden/`, rendered and compared by `tests/unit/test_report_golden.py`): a change to the rendered output is re-baselined (`REBASELINE_GOLDEN=1 pytest`) only after a before/after look, never silently, so the intended change *is* the reviewable diff.
 
 ---
 

{oceanarray-0.5.0/oceanarray.egg-info → oceanarray-0.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: oceanarray
-Version: 0.5.0
+Version: 0.5.1
 Summary: Multi-stage processing pipeline for moored oceanographic observations: raw instrument files to QC-flagged CF-compliant NetCDF with automated HTML reports.
 Maintainer-email: Eleanor Frajka-Williams <eleanorfrajka@gmail.com>
 License: MIT License
@@ -63,6 +63,7 @@ Requires-Dist: pytest-cov>=4.1; extra == "test"
 Provides-Extra: docs
 Requires-Dist: sphinx>=7.2; extra == "docs"
 Requires-Dist: sphinx-rtd-theme>=2.0; extra == "docs"
+Requires-Dist: sphinx-argparse>=0.4; extra == "docs"
 Requires-Dist: myst-nb>=1.0; extra == "docs"
 Requires-Dist: nbsphinx>=0.9; extra == "docs"
 Requires-Dist: pypandoc>=1.11; extra == "docs"
@@ -86,6 +87,8 @@ Dynamic: license-file
 
 Python tools for processing moored oceanographic array observations from raw instrument files to quality-controlled, CF-compliant NetCDF.
 
+It covers a single mooring's instruments — stages 1–3, stacking, and pressure gridding, each documented in an HTML report. Deriving calibration-dip offsets is handled by `caldip`, and shipboard CTD processing by `ctdcast`.
+
 **Documentation**: https://ocean-uhh.github.io/oceanarray/
 
 ## Installation

{oceanarray-0.5.0 → oceanarray-0.5.1}/README.md

@@ -9,6 +9,8 @@
 
 Python tools for processing moored oceanographic array observations from raw instrument files to quality-controlled, CF-compliant NetCDF.
 
+It covers a single mooring's instruments — stages 1–3, stacking, and pressure gridding, each documented in an HTML report. Deriving calibration-dip offsets is handled by `caldip`, and shipboard CTD processing by `ctdcast`.
+
 **Documentation**: https://ocean-uhh.github.io/oceanarray/
 
 ## Installation

oceanarray-0.5.1/docs/source/cli_reference.rst

@@ -0,0 +1,320 @@
+.. _cli_reference:
+
+===================
+CLI Reference
+===================
+
+The typical workflow is: validate the YAML, run ``oceanarray process`` stage by
+stage (including the ``stack`` and ``grid`` stages), inspect with
+``oceanarray report``, then use ``oceanarray run`` to reproduce the full pipeline
+in one command.
+
+All processing subcommands accept a mooring name as the first positional
+argument.  The mooring name is used to locate the YAML file at
+``{proc_dir}/{mooring}/{mooring}.mooring.yaml``.
+
+Without ``--force``, every subcommand skips any output file that already
+exists.  Add ``--force`` to overwrite.
+
+The flag lists below are generated from the argument parser
+(:func:`oceanarray.cli.build_parser`), so they cannot drift from the code.
+
+----
+
+``oceanarray validate``
+------------------------
+
+Check one or more YAML configuration files for missing required fields,
+unrecognised values, and structural errors.  Does not require ``--raw-dir``
+or ``--proc-dir``.  Run this before any processing to catch typos in
+``file_type``, missing ``hab``, or malformed serial numbers.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: validate
+   :nodescription:
+
+**Example**
+
+.. code-block:: bash
+
+   oceanarray validate /data/proc/dsG3_1_2026/dsG3_1_2026.mooring.yaml
+
+Prints errors and warnings to the console.  Exits with a non-zero status
+if any errors are found.
+
+----
+
+``oceanarray init``
+-------------------
+
+Create a skeleton mooring YAML file with commented template fields for all
+mandatory metadata and one example entry per instrument type.  Edit the file
+to fill in real values and delete unused instrument blocks.  Typically run
+once, before ``oceanarray validate``.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: init
+   :nodescription:
+
+**Example**
+
+.. code-block:: bash
+
+   oceanarray init dsG3_1_2026 --proc-dir /data/proc
+
+Creates ``/data/proc/dsG3_1_2026/dsG3_1_2026.mooring.yaml`` (the directory
+is created if it does not exist).
+
+----
+
+``oceanarray process``
+-----------------------
+
+Run one or more per-instrument processing stages.  Stage 2 requires that
+stage 1 has already run; stage 3 requires stage 2.
+
+The recommended sequence for a new mooring is:
+
+1. ``--stage 1``: verify that all raw files are reachable.
+2. ``--stage 2``: check that trimming looks right; adjust
+   ``deployment_time`` / ``recovery_time`` in the YAML if needed.
+3. ``--stage 3``: apply QC and velocity rotation once stages 1 and 2 are
+   confirmed.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: process
+   :nodescription:
+
+**Stage descriptions**
+
+- **Stage 1**: reads raw instrument files via seasenselib and writes
+  ``{mooring}_{serial}_stage1.nc``.  Data are stored as-is — no QC, no
+  trimming.
+- **Stage 2**: trims to ``deployment_time`` / ``recovery_time`` from the
+  YAML and applies clock drift correction.  Writes
+  ``{mooring}_{serial}_stage2.nc``.
+- **Stage 3**: applies QARTOD gross-range QC flags, derives salinity,
+  interpolates pressure for instruments without a pressure port, rotates
+  Aquadopp velocities to earth coordinates, and applies magnetic declination
+  correction.  Writes ``{mooring}_{serial}_stage3.nc``.
+- **stack**: interpolates all instruments onto a common time axis and writes
+  ``{mooring}_stack.nc``.  Instruments sampling faster than ``--dt`` are
+  subsampled by nearest-neighbour; slower instruments are interpolated linearly.
+  The 60-second default suits most mooring data; for high-frequency analysis
+  (e.g. internal waves from fast thermistors) work from the ``_stage3.nc`` files
+  or build a finer stack with ``--dt``.
+- **grid**: interpolates the stack onto a regular pressure grid and writes
+  ``{mooring}_grid.nc``.  Values outside the instrument depth range at each time
+  step are set to NaN.  QC flags from stage 3 are not consulted — data flagged
+  suspect or bad are treated the same as good unless already NaN.
+
+**Output created**
+
+Per-instrument: ``{mooring}_{serial}_stage{N}.nc`` in ``{proc_dir}/{mooring}/{instrument}/``.
+Mooring-level: ``{mooring}_stack.nc`` and ``{mooring}_grid.nc`` in ``{proc_dir}/{mooring}/``.
+
+**Examples**
+
+Run stage 1 for all instruments:
+
+.. code-block:: bash
+
+   oceanarray process dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --stage 1
+
+Rerun stage 2 for a single instrument after updating the YAML:
+
+.. code-block:: bash
+
+   oceanarray process dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --stage 2 --serial 26261 --force
+
+Full pipeline — all instrument stages plus stack and grid:
+
+.. code-block:: bash
+
+   oceanarray process dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --stage 1 2 3 stack grid
+
+Stack and grid only (after stages 1–3 already ran), non-default grid spacing:
+
+.. code-block:: bash
+
+   oceanarray process dsG3_1_2026 --proc-dir /data/proc --stage stack grid --dp 10 --pmin 100 --pmax 2000
+
+----
+
+``oceanarray report``
+----------------------
+
+Generate HTML reports for a mooring.  Without any report-type flags,
+generates only the mooring summary page.  See :doc:`reports` for a
+description of what each report page contains.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: report
+   :nodescription:
+
+**Output created**
+
+``{mooring}_report.html`` (always), plus optional
+``{mooring}_stack_report.html``, ``{mooring}_grid_report.html``, and
+``report/instrument/{mooring}_{serial}_report.html``.
+
+**Examples**
+
+Summary and per-instrument pages:
+
+.. code-block:: bash
+
+   oceanarray report dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --instruments
+
+All report types (equivalent to ``--stack --grid --instruments --pdf``):
+
+.. code-block:: bash
+
+   oceanarray report dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --all
+
+Array-level index across all moorings:
+
+.. code-block:: bash
+
+   oceanarray report cruise.array.yaml --array --report-dir /data/reports
+
+Cruise recovery table:
+
+.. code-block:: bash
+
+   oceanarray report dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --cruise-table
+
+----
+
+``oceanarray run``
+------------------
+
+Run the complete processing pipeline in one command: stages 1, 2, and 3,
+followed by stack, grid, and all report pages.  Continues past individual
+instrument failures — check the processing logs in
+``{proc_dir}/{mooring}/processing_logs/`` for details.
+
+Use ``oceanarray run`` only after confirming that stage 1 can read all raw
+files and that the ``deployment_time`` / ``recovery_time`` values in the
+YAML produce the correct trim (verified with ``oceanarray process --stage 2``
+and a quick report inspection).
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: run
+   :nodescription:
+
+**Output created**
+
+All stage NC files, ``{mooring}_stack.nc``, ``{mooring}_grid.nc``,
+mooring summary report, stack report, grid report, and per-instrument
+reports.
+
+**Example**
+
+.. code-block:: bash
+
+   oceanarray run dsG3_1_2026 --raw-dir /data/raw --proc-dir /data/proc --force
+
+----
+
+``oceanarray list``
+--------------------
+
+Print a reference table of allowed ``instrument:`` and ``file_type:`` values
+for mooring YAML configuration.  Use this to look up the correct reader name
+for a given instrument type, or to check which ``instrument:`` values are
+recognised.
+
+Does not require a mooring name, ``--raw-dir``, or ``--proc-dir``.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: list
+   :nodescription:
+
+**Example**
+
+.. code-block:: bash
+
+   oceanarray list
+
+----
+
+Preliminary visualisation commands
+------------------------------------
+
+The following commands produce standalone plots and animations.  They are
+available for quick inspection but are considered preliminary — the output
+style, variable names, and flag set may change in a future release.
+
+``oceanarray plot``
+~~~~~~~~~~~~~~~~~~~~
+
+Show an interactive overview plot of processed data, or save it to a file.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: plot
+   :nodescription:
+
+**Example**
+
+.. code-block:: bash
+
+   oceanarray plot dsG3_1_2026 --proc-dir /data/proc --var_y pressure --var_color temperature --output overview.png
+
+----
+
+``oceanarray animate``
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Generate an animated hodograph (tip of the velocity vector tracing a path
+over time) for Aquadopp or ADCP instruments.  Requires ``east_velocity``
+and ``north_velocity`` in the stage 3 output.  The ``Pillow`` library must
+be installed for GIF output.
+
+.. argparse::
+   :module: oceanarray.cli
+   :func: build_parser
+   :prog: oceanarray
+   :path: animate
+   :nodescription:
+
+**Example**
+
+.. code-block:: bash
+
+   oceanarray animate dsG3_1_2026 --proc-dir /data/proc --serial 400115 -o dsG3_400115_hodograph.gif
+
+----
+
+Deprecated flags
+----------------
+
+``--basedir DIR`` (removed)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``--basedir`` flag has been **removed**.  Passing it prints a migration
+message naming the replacement and exits without processing.  Use
+``--raw-dir`` and ``--proc-dir`` with the mooring-first layout instead.
+
+See :doc:`migration` for step-by-step migration instructions.

{oceanarray-0.5.0 → oceanarray-0.5.1}/docs/source/conf.py

@@ -30,6 +30,7 @@ extensions = [
     "sphinx.ext.viewcode",
     "sphinx.ext.napoleon",
     "sphinx.ext.mathjax",
+    "sphinxarg.ext",
     "nbsphinx",
     "myst_parser",
 ]

{oceanarray-0.5.0 → oceanarray-0.5.1}/docs/source/legacy.rst

@@ -38,6 +38,8 @@ active processing code:
 
 ``convertOS.py``
    OceanSites format conversion from RODB — superseded by the Stage 1–3 pipeline.
+   Its YAML inputs live beside it: ``OS1_variable_names.yaml``,
+   ``OS1_variable_attributes.yaml`` and ``OS1_sensor_attributes.yaml``.
 
 ``rapid_interp.py`` (top-level)
    RAPID-specific interpolation helpers — not part of the active pipeline.

{oceanarray-0.5.0 → oceanarray-0.5.1}/docs/source/project_structure.md

@@ -71,8 +71,7 @@ oceanarray/
 │   ├── config/                    # Configuration
 │   │   ├── parameters.py          # Package-level defaults (VARIABLES, QC ranges, style)
 │   │   ├── validation.py          # Mooring YAML validation
-│   │   ├── OS1_var_names.yaml     # OceanSITES variable name mappings
-│   │   ├── OS1_sensor_attrs.yaml  # OceanSITES sensor attributes
+│   │   ├── unit_tables.py         # Units: oceanarray's overrides over the shared tables
 │   │   ├── logging.yaml           # Logging configuration
 │   │   └── legacy/                # Legacy RAPID/RODB configuration files
 │   │

{oceanarray-0.5.0 → oceanarray-0.5.1}/docs/source/yaml_configuration.rst

@@ -297,6 +297,23 @@ Optional fields
      - string
      - Special sentinel value indicating no data file exists yet.
        Processing is skipped; the instrument appears in reports as pending.
+   * - ``stack``
+     - boolean
+     - Set to ``false`` to keep this instrument out of the stack and grid
+       while still processing it through Stages 1–3 and its instrument
+       report (reference data, e.g. a low-passed RODB ``.lowedt`` file).
+       The stack records it in its ``instruments_excluded`` attribute.
+       Default ``true``.  A low-passed file reaching the stack without it is
+       omitted with an error naming this key.
+   * - ``units_override``
+     - mapping
+     - Corrects a file whose ``units`` attribute is wrong, as
+       ``{variable: units}``, e.g. ``{conductivity: mS/cm}`` for a file that
+       labels mS/cm values ``S/m``.  Applied in Stage 1 before the units pass;
+       the stated value is kept in the variable's ``units_override_from``
+       attribute.  For ``oceansites`` and ``rodb`` files Stage 1 checks each
+       variable's median against the registry's valid range and refuses the
+       instrument, naming this key, when the stated units look wrong.
 
 Additional fields
 ~~~~~~~~~~~~~~~~~
@@ -349,7 +366,7 @@ Valid values and their descriptions:
      - Reader families accepted by ``file_type``
    * - ``microcat``
      - SeaBird SBE37 CTD
-     - ``sbe-cnv``, ``sbe-ascii``, ``sbe-hex``
+     - ``sbe-cnv``, ``sbe-ascii``, ``sbe-hex``, ``oceansites``, ``rodb``
    * - ``sbe56``
      - SeaBird SBE56 temperature logger
      - ``sbe-cnv``
@@ -365,6 +382,10 @@ Valid values and their descriptions:
    * - ``aquadopp``
      - Nortek Aquadopp current meter
      - ``nortek-raw``, ``nortek-ascii``, ``nortek-csv``
+   * - ``currentmeter``
+     - Point current meter delivered already processed (e.g. an Aanderaa RCM
+       from an OceanSITES or RODB archive)
+     - ``oceansites``, ``rodb``
    * - ``ADCP``
      - RDI WorkHorse broadband ADCP
      - ``rdi-raw``
@@ -417,6 +438,22 @@ Valid ``file_type`` values
      - Nortek CSV string-data export
      - Aquadopp (400### series)
      -
+   * - ``oceansites``
+     - OceanSITES netCDF (oceanarray reader)
+     - MicroCAT, current meter
+     - A single-instrument time series already processed by its originator:
+       OceanSITES 1.3 (variables on ``TIME, DEPTH``) or the OSNAP variant
+       (variables on ``TIME``).  ``TEMP``, ``PSAL``, ``CNDC``, ``PRES``,
+       ``UCUR``, ``VCUR``, ``WCUR`` are read; a ``<CODE>_QC`` variable is kept
+       as ``<name>_qc_originator``.  Velocities are geographic, so Stage 3
+       applies no rotation and no magnetic declination.
+   * - ``rodb``
+     - RODB text (oceanarray reader)
+     - MicroCAT, current meter
+     - RAPID/BODC ``.microcat``, ``.edt`` and ``.lowedt`` files.  Units are
+       those documented for each column code (``U``/``V``/``W`` in cm/s,
+       converted to m s-1).  An ``.edt`` header's ``MagDeviation`` is recorded
+       as ``originator_magnetic_declination`` and not applied again.
    * - ``rbr-rsk``
      - RBR Ruskin SQLite
      - RBR Solo, Duet, etc.

{oceanarray-0.5.0 → oceanarray-0.5.1}/notebooks/legacy/demo_batch_instrument.ipynb

@@ -46,9 +46,9 @@
     "ds_list = readers.load_dataset(files)\n",
     "\n",
     "config_dir = Path(\"..\") / \"oceanarray\" / \"config\"\n",
-    "var_map_yaml = config_dir / \"OS1_var_names.yaml\"\n",
-    "vocab_yaml = config_dir / \"OS1_vocab_attrs.yaml\"\n",
-    "sensor_yaml = config_dir / \"OS1_sensor_attrs.yaml\"\n",
+    "var_map_yaml = Path(\"..\") / \"oceanarray\" / \"legacy\" / \"OS1_variable_names.yaml\"\n",
+    "vocab_yaml = Path(\"..\") / \"oceanarray\" / \"legacy\" / \"OS1_variable_attributes.yaml\"\n",
+    "sensor_yaml = Path(\"..\") / \"oceanarray\" / \"legacy\" / \"OS1_sensor_attributes.yaml\"\n",
     "project_yaml = config_dir / \"project_RAPID.yaml\"\n",
     "\n",
     "ds_list_OS = []\n",

{oceanarray-0.5.0 → oceanarray-0.5.1}/notebooks/legacy/demo_instrument.ipynb

@@ -323,9 +323,9 @@
     "if 0:\n",
     "    metadata_txt = data_dir / \"wb1_12_2015_6123.use\"\n",
     "    config_dir = Path(\"..\") / \"oceanarray\" / \"config\"\n",
-    "    var_map_yaml = config_dir / \"OS1_var_names.yaml\"\n",
-    "    vocab_yaml = config_dir / \"OS1_vocab_attrs.yaml\"\n",
-    "    sensor_yaml = config_dir / \"OS1_sensor_attrs.yaml\"\n",
+    "    var_map_yaml = Path(\"..\") / \"oceanarray\" / \"legacy\" / \"OS1_variable_names.yaml\"\n",
+    "    vocab_yaml = Path(\"..\") / \"oceanarray\" / \"legacy\" / \"OS1_variable_attributes.yaml\"\n",
+    "    sensor_yaml = Path(\"..\") / \"oceanarray\" / \"legacy\" / \"OS1_sensor_attributes.yaml\"\n",
     "    project_yaml = config_dir / \"project_RAPID.yaml\"\n",
     "    ds_OS = convertOS.convert_rodb_to_oceansites(\n",
     "        ds_cal,\n",