pycontrails 0.53.0__tar.gz → 0.54.0__tar.gz

{pycontrails-0.53.0 → pycontrails-0.54.0}/.github/workflows/release.yaml

@@ -40,7 +40,7 @@ jobs:
 
       # https://cibuildwheel.readthedocs.io/en/stable/options/#testing
       - name: Build wheels
-        uses: pypa/cibuildwheel@v2.20
+        uses: pypa/cibuildwheel@v2.21
         env:
           CIBW_BUILD: cp310-* cp311-* cp312-* cp313-*
           CIBW_SKIP: '*-win32 *-manylinux_i686 *-musllinux*'

{pycontrails-0.53.0 → pycontrails-0.54.0}/.github/workflows/test.yaml

@@ -123,7 +123,6 @@ jobs:
         shell: bash
         run: |
           pip check
-          make black-check
-          make ruff
+          make lint
           make mypy
           make pytest

{pycontrails-0.53.0 → pycontrails-0.54.0}/.pre-commit-config.yaml

@@ -1,13 +1,21 @@
 repos:
   - repo: local
     hooks:
-      - id: ruff
-        name: ruff
+      - id: ruff-format
+        name: ruff format
+        language_version: python3
+        language: python
+        entry: ruff format
+        types: [file]
+        files: \.(py|ipynb)$
+      - id: ruff-lint
+        name: ruff lint
         language_version: python3
         language: python
         entry: ruff check
         args: [--fix, --exit-non-zero-on-fix]
-        types: [python]
+        types: [file]
+        files: \.(py|ipynb)$
       - id: doc8
         name: doc8
         language_version: python3
@@ -15,13 +23,6 @@ repos:
         entry: doc8
         types: [file]
         files: \.(rst)$
-      - id: black
-        name: black
-        language_version: python3
-        language: python
-        entry: black
-        types: [file]
-        files: \.(py|ipynb)$
       - id: nb-clean
         name: nb-clean
         language_version: python3

{pycontrails-0.53.0 → pycontrails-0.54.0}/CHANGELOG.md

@@ -1,5 +1,40 @@
 # Changelog
 
+## v0.54.0
+
+### Features
+
+- Perform model-level to pressure-level conversion in-house instead of relying on `metview`. This update has several advantages:
+  - The `ARCOERA5` and `ERA5ModelLevel` interfaces no longer require `metview` to be installed. Similarly, grib files and the associated tooling can also largely be avoided.
+  - The computation is performed using `xarray` and `dask` tooling, which means the result can be computed lazily if desired.
+  - The computation is defined using `numpy` operations (some of which release the GIL) and can be parallelized using threading through `dask` (this is the default behavior).
+  - The computation is generally a bit faster than the `metview` implementation (this depends on the exact chunking of the model level meteorology data). This chunking can be tuned by the user to optimize runtime performance or memory usage.
+
+  See the `ml_to_pl` function for low-level details.
+
+- Update the `ARCOERA5` and `ERA5ModelLevel` interfaces to use the new model-level to pressure-level conversion. The ERA5 model level data is now downloaded as the netcdf format instead of grib. This format change decreases the download size.
+
+### Breaking changes
+
+- Rename `levels` -> `model_levels` in the `ARCOERA5` and `ERA5ModelLevel` constructors. Rename `cache_grib` -> `cache_download`.
+- Rename `pressure_levels_at_model_levels` -> `model_level_reference_pressure`. Add a new `model_level_pressure` method that requires surface pressure data.
+
+### Fixes
+
+- Update `ERA5ModelLevel` for the new CDS-Beta server.
+
+### Internals
+
+- Use `ruff` to format the codebase in place of `black`.
+- Run `ruff` linting and formatting over the notebook examples in the documentation.
+- Update development documentation with new links.
+
+## v0.53.1
+
+### Features
+
+- Support `ERA5` downloads from [CDS-Beta](https://cds-beta.climate.copernicus.eu/). The updated interface is backwards compatible with the legacy CDS server. The choice of CDS server is governed by the `url` parameter in the `ERA5` constructor.
+
 ## v0.53.0
 
 ### Breaking changes
@@ -88,13 +123,13 @@
 ### Features
 
 - Add tools for running [APCEMM](https://github.com/MIT-LAE/APCEMM) from within pycontrails. This includes:
-    - utilities for generating APCEMM input files and running APCEMM (`pycontrails.models.apcemm.utils`)
-    - an interface (`pycontrails.models.apcemm.apcemm`) that allows users to run APCEMM as a pycontrails `Model`.
+  - utilities for generating APCEMM input files and running APCEMM (`pycontrails.models.apcemm.utils`)
+  - an interface (`pycontrails.models.apcemm.apcemm`) that allows users to run APCEMM as a pycontrails `Model`.
 - Add [APCEMM tutorial notebook](https://py.contrails.org/integrations/APCEMM.html).
 - Add prescribed sedimentation rate to `DryAdvection` model.
 - Add `Landsat` and `Sentinel` datalibs for searching, retrieving, and visualizing Landsat 8-9 and Sentinel-2 imagery. The datalibs include:
-    - Tools for querying Landsat and Sentinel-2 imagery for intersections with user-defined regions (`landsat.query`, `sentinel.query`) or flights (`landsat.intersect`, `sentinel.intersect`). These tools use BigQuery tables and require a Google Cloud Platform account with access to the BigQuery API.
-    - Tools for downloading and visualizing imagery from Landsat (`Landsat`) and Sentinel-2 (`Sentinel`). These tools retrieve data anonymously from Google Cloud Platform storage buckets and can be used without a Google Cloud Platform account.
+  - Tools for querying Landsat and Sentinel-2 imagery for intersections with user-defined regions (`landsat.query`, `sentinel.query`) or flights (`landsat.intersect`, `sentinel.intersect`). These tools use BigQuery tables and require a Google Cloud Platform account with access to the BigQuery API.
+  - Tools for downloading and visualizing imagery from Landsat (`Landsat`) and Sentinel-2 (`Sentinel`). These tools retrieve data anonymously from Google Cloud Platform storage buckets and can be used without a Google Cloud Platform account.
 - Add tutorial notebooks demonstrating how to use `Landsat` and `Sentinel` datalibs to find flights in high-resolution satellite imagery.
 - Modify `Flight.to_geojson_multilinestring` to make grouping key optional and to support multiple antimeridian crossings.
 - Update the `pycontrails` build system to require `numpy 2.0` per the [official numpy guidelines](https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-0-specific-advice). Note that the runtime requirement for `pycontrails` remains `numpy>=1.22`.

{pycontrails-0.53.0 → pycontrails-0.54.0}/Makefile

@@ -71,14 +71,12 @@ dev-pycontrails-bada:
 # QC, Test
 # -----------
 
-ruff: black-check
-	ruff check pycontrails tests
+lint:
+	ruff check pycontrails tests docs
+	ruff format --check pycontrails tests docs
 
-black:
-	black pycontrails tests
-
-black-check:
-	black pycontrails tests --check
+format:
+	ruff format pycontrails tests docs
 
 # https://taplo.tamasfe.dev/
 taplo:
@@ -107,7 +105,7 @@ pytest-cov:
 		--ignore=tests/unit/test_zarr.py \
 		tests/unit
 
-test: ruff mypy black-check nb-black-check pytest doctest nb-test
+test: lint mypy pytest doctest nb-test
 
 profile:
 	python -m cProfile -o $(script).prof $(script)
@@ -178,11 +176,8 @@ doctest: ensure-numpy-2 ensure-era5-cached ensure-gcp-credentials
 doc8:
 	doc8 docs
 
-nb-black:
-	black docs/**/*.ipynb
-
-nb-black-check:
-	black docs/**/*.ipynb --check
+nb-format-check:
+	ruff format --check docs/**/*.ipynb
 
 # Note must be kept in sync with 
 # `.pre-commit-config.yaml` and `make nb-clean-check`
@@ -206,7 +201,7 @@ nb-clean-check:
 # TODO: 
 #   - Install eccodes on Github Action so GFS notebook can run
 #   - Provide meteorology data for `run-cocip-on-flight` tutorial
-nb-test: ensure-era5-cached nb-clean-check nb-black-check nb-check-links
+nb-test: ensure-era5-cached nb-clean-check nb-format-check nb-check-links
 	python -m pytest --nbval-lax \
 		--ignore=docs/integrations/ACCF.ipynb \
 		--ignore=docs/integrations/APCEMM.ipynb \
@@ -231,7 +226,7 @@ nb-check-links:
 # NOTE notebooks from docs/integrations/ manually
 # Add `skip-execution` cell tag if you want to skip a cell
 # Add `raises-exception` cell tag if you know the cell raises exception
-nb-execute: ensure-era5-cached nb-black-check nb-check-links
+nb-execute: ensure-era5-cached nb-format-check nb-check-links
 	jupyter nbconvert --inplace \
 		--to notebook \
 		--execute \

{pycontrails-0.53.0/pycontrails.egg-info → pycontrails-0.54.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pycontrails
-Version: 0.53.0
+Version: 0.54.0
 Summary: Python library for modeling aviation climate impacts
 Author-email: Breakthrough Energy <py@contrails.org>
 License: Apache-2.0
@@ -36,7 +36,6 @@ Requires-Dist: xarray>=2022.3
 Provides-Extra: complete
 Requires-Dist: pycontrails[ecmwf,gcp,gfs,jupyter,pyproj,sat,vis,zarr]; extra == "complete"
 Provides-Extra: dev
-Requires-Dist: black[jupyter]==24.8.0; extra == "dev"
 Requires-Dist: dep_license; extra == "dev"
 Requires-Dist: fastparquet>=0.8; extra == "dev"
 Requires-Dist: ipdb>=0.13; extra == "dev"
@@ -50,7 +49,7 @@ Requires-Dist: pyarrow>=5.0; extra == "dev"
 Requires-Dist: pytest>=8.2; extra == "dev"
 Requires-Dist: pytest-cov>=2.11; extra == "dev"
 Requires-Dist: requests>=2.25; extra == "dev"
-Requires-Dist: ruff==0.5.7; extra == "dev"
+Requires-Dist: ruff==0.6.2; extra == "dev"
 Requires-Dist: setuptools; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: doc8>=1.1; extra == "docs"
@@ -73,7 +72,6 @@ Requires-Dist: ecmwf-api-client>=1.6; extra == "ecmwf"
 Requires-Dist: netcdf4>=1.6.1; extra == "ecmwf"
 Requires-Dist: platformdirs>=3.0; extra == "ecmwf"
 Requires-Dist: requests>=2.25; extra == "ecmwf"
-Requires-Dist: lxml>=5.1.0; extra == "ecmwf"
 Provides-Extra: gcp
 Requires-Dist: google-cloud-storage>=2.1; extra == "gcp"
 Requires-Dist: platformdirs>=3.0; extra == "gcp"

{pycontrails-0.53.0 → pycontrails-0.54.0}/docs/develop.rst

@@ -91,17 +91,17 @@ but should also be run locally before submitting PRs:
 
     $ make test
 
-Lint the repository with `ruff <https://beta.ruff.rs/docs/>`__:
+Lint the repository with `ruff <https://docs.astral.sh/ruff/>`__:
 
 .. code-block:: bash
 
-    $ make ruff
+    $ make lint
 
-Autoformat the repository with `black <https://black.readthedocs.io/en/stable/>`__:
+Autoformat the repository with `ruff <https://docs.astral.sh/ruff/formatter/>`__:
 
 .. code-block:: bash
 
-    $ make black
+    $ make format
 
 Run type checking with `mypy <https://www.mypy-lang.org/>`__:
 
@@ -109,7 +109,7 @@ Run type checking with `mypy <https://www.mypy-lang.org/>`__:
 
     $ make mypy
 
-Run unit tests with `pytest <https://docs.pytest.org/en/7.2.x/>`__:
+Run unit tests with `pytest <https://docs.pytest.org/en/latest/>`__:
 
 .. code-block:: bash
 
@@ -121,7 +121,7 @@ Run notebook validation with `nbval <https://github.com/computationalmodelling/n
 
     $ make nb-test
 
-Run doctests with `pytest <https://docs.pytest.org/en/7.2.x/>`__:
+Run doctests with `pytest <https://docs.pytest.org/en/latest/>`__:
 
 .. code-block:: bash
 
@@ -294,7 +294,7 @@ The project uses `Black <https://black.readthedocs.io/en/stable/>`__,
 standardize code-formatting. These tools are run automatically in a
 pre-commit hook.
 
-The project uses `pytest <https://docs.pytest.org/en/7.2.x/>`__ to run
+The project uses `pytest <https://docs.pytest.org/en/latest/>`__ to run
 unit tests. New code should include clear unit tests for implementation
 and output values. New test files should be included in the
 `/tests/unit/ directory <https://github.com/contrailcirrus/pycontrails/tree/main/tests/unit>`__.

{pycontrails-0.53.0 → pycontrails-0.54.0}/docs/integrations/APCEMM.ipynb

@@ -26,17 +26,16 @@
     "import os\n",
     "\n",
     "import cartopy.crs as ccrs\n",
+    "import matplotlib.pyplot as plt\n",
     "import numpy as np\n",
     "import pandas as pd\n",
-    "import matplotlib.colors as colors\n",
-    "import matplotlib.pyplot as plt\n",
     "import xarray as xr\n",
+    "from matplotlib import colors\n",
     "\n",
-    "from pycontrails import Flight, MetDataset, MetVariable\n",
+    "from pycontrails import Flight, MetVariable\n",
     "from pycontrails.core.met_var import Geopotential\n",
     "from pycontrails.datalib.ecmwf import ARCOERA5\n",
     "from pycontrails.models.apcemm.apcemm import APCEMM\n",
-    "from pycontrails.models.apcemm.inputs import APCEMMInput\n",
     "from pycontrails.models.humidity_scaling import HistogramMatching\n",
     "from pycontrails.models.issr import ISSR\n",
     "from pycontrails.models.ps_model import PSFlight\n",
@@ -54,14 +53,7 @@
     "\n",
     "This demo uses model-level [ERA5](https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5) data from the [ARCO ERA5 dataset](https://github.com/google-research/arco-era5) for met data. \n",
     "\n",
-    ">  Note this will download ~3 GB of meteorology data to your computer\n",
-    "\n",
-    "Processing ARCO ERA5 data requires [metview](https://metview.readthedocs.io/en/latest/) binaries and python bindings. If pycontrails is installed in a conda environment, metview can be installed by running\n",
-    "```bash\n",
-    "conda install -c conda-forge metview\n",
-    "conda install -c conda-forge metview-python\n",
-    "python -m metview selfcheck\n",
-    "```"
+    ">  Note this will download ~3 GB of meteorology data to your computer"
    ]
   },
   {