pycontrails 0.54.4__tar.gz → 0.54.6__tar.gz

{pycontrails-0.54.4 → pycontrails-0.54.6}/CHANGELOG.md

@@ -1,5 +1,52 @@
 # Changelog
 
+## v0.54.6
+
+### Features
+
+- Add support for generic (model-agnostic) meteorology data to `Cocip` and `CocipGrid`.
+
+- Add two new parameters to the `DryAdvection` model.
+  - If the `verbose_outputs` parameter is enabled, additional wind-shear data is included in the output.
+  - If the `include_source_in_output` parameter is enabled, the source data with any of the intermediate artifacts (e.g., interpolated met data, wind-shear data, etc.) is included in the output.
+
+  Both parameters are disabled by default.
+
+### Fixes
+
+- Update the CDS URL referenced throughout pycontrails from ``https://cds-beta.climate.copernicus.eu`` to ``https://cds.climate.copernicus.eu``.
+
+### Internals
+
+- Suppress mypy `return-value` errors for functions in `geo.py` where mypy fails to correctly infer return types of numpy ufuncs applied to xarray objects.
+- Change `AircraftPerformance` and downstream implementations for better support in running over `Fleet` sources. The runtime of `PSFlight` remains the same.
+
+## v0.54.5
+
+### Features
+
+- This release brings a number of very minor performance improvements to the low-level pycontrails data structures (`VectorDataset` and `MetDataset`). Cumulatively, these changes should bring in a small but nontrivial speedup (~5%) when running a model such as `Cocip` or `DryAdvection` on a single `Flight` source.
+  - Core `Flight` methods such as `copy`, `filter`, and `downselect_met` are now ~10x faster for typical use cases.
+  - Converting between `Fleet` and `Flight` instances via `Fleet.from_seq` and `Fleet.to_flight_list` are also ~5x faster.
+- Implement low-memory met-downselection logic in `DryAdvection`. This is the same logic used in `CocipGrid` to reduce memory consumption by only loading the necessary time slices of the `met` data into memory. If `met` is already loaded into memory, this change will have no effect.
+
+### Breaking Changes
+
+- Remove the `copy` parameter from `GeovectorDataset.downselect_met`. This method always returns a view of the original dataset.
+- Remove the `validate` parameter in  the `MetDataArray` constructor. Input data is now always validated.
+
+### Fixes
+
+- Make slightly more explicit when data is copied in the `VectorDataset` constructor: data is now always shallow-copied, and the `copy` parameter governs whether to copy the underlying arrays.
+- Call `downselect_met` in `DryAdvection.eval`. (This was previously forgotten.)
+- Fix minor bug in `CocipGrid` downselect met logic introduced in v0.54.4. This bug may have caused some met slices to be reloaded when running `CocipGrid.eval` with lazily-loaded `met` and `rad` data.
+
+### Internals
+
+- Add internal `VectorDataset._from_fastpath` and `MetDataset._from_fastpath` class methods to skip data validation.
+- Define `__slots__` on `MetBase`, `MetDataset`, `MetDataArray`, and `AttrDict`.
+- When `MetDataset` and `MetDataArray` shared a common implementation, move the implementation to `MetBase`. This was the case for the `copy`, `downselect`, and `wrap_longitude` methods.
+
 ## v0.54.4
 
 ### Features
@@ -13,7 +60,7 @@
   1. If there is a shallow climb (ROCD < 500 ft/min), then always assume that the flight will climb at the next time step.
   1. If there is a shallow descent (-250 < ROCD < 0 ft/min), then always assume that the flight will descend at the final time step.
 
-  Conditions (3) to (6) is based on the logic that the aircraft will generally prefer to climb to a higher altitude as early as possible, and descend to a lower altitude as late as possible, because a higher altitude can reduce drag and fuel consumption.
+  Conditions (3) to (6) are based on the logic that the aircraft will generally prefer to climb to a higher altitude as early as possible, and descend to a lower altitude as late as possible, because a higher altitude can reduce drag and fuel consumption.
 
 ### Breaking changes
 

{pycontrails-0.54.4/pycontrails.egg-info → pycontrails-0.54.6}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: pycontrails
-Version: 0.54.4
+Version: 0.54.6
 Summary: Python library for modeling aviation climate impacts
 Author-email: Breakthrough Energy <py@contrails.org>
 License: Apache-2.0
@@ -49,8 +49,7 @@ Requires-Dist: pyarrow>=5.0; extra == "dev"
 Requires-Dist: pytest>=8.2; extra == "dev"
 Requires-Dist: pytest-cov>=2.11; extra == "dev"
 Requires-Dist: requests>=2.25; extra == "dev"
-Requires-Dist: ruff==0.8.0; extra == "dev"
-Requires-Dist: setuptools; extra == "dev"
+Requires-Dist: ruff>=0.9.0; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: doc8>=1.1; extra == "docs"
 Requires-Dist: furo>=2023.3; extra == "docs"

{pycontrails-0.54.4 → pycontrails-0.54.6}/docs/notebooks/ECMWF.ipynb

@@ -16,7 +16,7 @@
     "\n",
     "Support provided for:\n",
     "\n",
-    "- [ERA5](https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5) via the [Copernicus Data Store (CDS-Beta)](https://cds-beta.climate.copernicus.eu/) using [cdsapi](https://github.com/ecmwf/cdsapi) or user provided files\n",
+    "- [ERA5](https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5) via the [Copernicus Data Store](https://cds.climate.copernicus.eu/) using [cdsapi](https://github.com/ecmwf/cdsapi) or user provided files\n",
     "- [HRES](https://confluence.ecmwf.int/display/FUG/Section+2.1.1.2+Rationale+for+High+Resolution) and [ENS](https://confluence.ecmwf.int/display/FUG/Section+2.1.2.1+Medium+Range+Ensemble+forecasts) via [MARS](https://confluence.ecmwf.int/display/UDOC/MARS+user+documentation) using [ecmwf-api-client](https://github.com/ecmwf/ecmwf-api-client) or user provided files.\n",
     "\n",
     "For both ERA5 and HRES, we provide interfaces for accessing \"pressure-level data\" (fields pre-interpolated to a fixed set of pressure levels) or \"model-level data\" (fields retrieved on the native vertical grid and [interpolated after retrieval to an arbitrary set of pressure levels](model-levels.ipynb)). We recommend using model-level data when possible, as the resolution of pressure-level data is coarse relative to the vertical scale of ice-supersaturated regions.\n",
@@ -32,8 +32,8 @@
     "\n",
     "### Access\n",
     "\n",
-    "- Requires account with [Copernicus Data Portal](https://cds-beta.climate.copernicus.eu/)\n",
-    "- Provide `url` and `key` credentials on input, or refer to the [CDS API Documentation](https://cds-beta.climate.copernicus.eu/how-to-api) for how to create `~/.cdsapirc` file to configure access.\n",
+    "- Requires account with [Copernicus Data Portal](https://cds.climate.copernicus.eu/)\n",
+    "- Provide `url` and `key` credentials on input, or refer to the [CDS API Documentation](https://cds.climate.copernicus.eu/how-to-api) for how to create `~/.cdsapirc` file to configure access.\n",
     "\n",
     "### Reference\n",
     "\n",
@@ -87,7 +87,7 @@
     "    time=\"2022-03-01 00:00:00\",\n",
     "    variables=[\"t\", \"q\", \"u\", \"v\", \"w\", \"ciwc\", \"z\", \"cc\"],  # supports CF name or short names\n",
     "    pressure_levels=[200, 250, 300],\n",
-    "    # url=\"https://cds-beta.climate.copernicus.eu/api\",\n",
+    "    # url=\"https://cds.climate.copernicus.eu/api\",\n",
     "    # key=\"<key>\"\n",
     ")\n",
     "era5"
@@ -132,7 +132,7 @@
     "        \"cc\",\n",
     "    ],  # supports CF name or short names\n",
     "    pressure_levels=[300, 250, 200],\n",
-    "    # url=\"https://cds-beta.climate.copernicus.eu/api\",\n",
+    "    # url=\"https://cds.climate.copernicus.eu/api\",\n",
     "    # key=\"<key>\"\n",
     ")\n",
     "era5"
@@ -1429,7 +1429,7 @@
     "era5 = ERA5(\n",
     "    time=(\"2022-03-01 00:00:00\", \"2022-03-01 03:00:00\"),\n",
     "    variables=[\"tsr\", \"ttr\"],\n",
-    "    # url=\"https://cds-beta.climate.copernicus.eu/api\",\n",
+    "    # url=\"https://cds.climate.copernicus.eu/api\",\n",
     "    # key=\"<key>\"\n",
     ")\n",
     "era5"
@@ -3242,8 +3242,7 @@
     "\n",
     "### Access\n",
     "\n",
-    "Users within ECMWF Member and Co-operating States may contact their Computing Representative to obtain access to MARS. \n",
-    "All other users may [request a username and password](https://accounts.ecmwf.int/auth/realms/ecmwf/protocol/openid-connect/registrations?client_id=apps&response_type=code&scope=openid%20email&redirect_uri=https://www.ecmwf.int) and then [get an api key](https://api.ecmwf.int/v1/key/).\n",
+    "Users within ECMWF Member and Co-operating States may contact their Computing Representative to obtain access to MARS. All other users may [request a username and password](https://confluence.ecmwf.int/display/WEBAPI/Access+MARS) and then [get an api key](https://api.ecmwf.int/v1/key/).\n",
     "\n",
     "Provide `url`, `key`, and `email` credentials on input, or see [ECMWF API Client documentation](https://github.com/ecmwf/ecmwf-api-client#configure) to configure  local `~/.ecmwfapirc` file:\n",
     "\n",
@@ -6072,7 +6071,7 @@
     "    variables=variables,\n",
     "    pressure_levels=[300, 250, 150],\n",
     "    cachestore=gcp,\n",
-    "    # url=\"https://cds-beta.climate.copernicus.eu/api\",\n",
+    "    # url=\"https://cds.climate.copernicus.eu/api\",\n",
     "    # key=\"<key>\"\n",
     ")"
    ]

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/_version.py

@@ -12,5 +12,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.54.4'
-__version_tuple__ = version_tuple = (0, 54, 4)
+__version__ = version = '0.54.6'
+__version_tuple__ = version_tuple = (0, 54, 6)

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/core/aircraft_performance.py

@@ -96,22 +96,52 @@ class AircraftPerformance(Model):
 
     source: Flight
 
-    @abc.abstractmethod
     @overload
     def eval(self, source: Fleet, **params: Any) -> Fleet: ...
 
-    @abc.abstractmethod
     @overload
     def eval(self, source: Flight, **params: Any) -> Flight: ...
 
-    @abc.abstractmethod
     @overload
     def eval(self, source: None = ..., **params: Any) -> NoReturn: ...
 
-    @abc.abstractmethod
     def eval(self, source: Flight | None = None, **params: Any) -> Flight:
         """Evaluate the aircraft performance model.
 
+        Parameters
+        ----------
+        source : Flight
+            Flight trajectory to evaluate. Can be a :class:`Flight` or :class:`Fleet`.
+        params : Any
+            Override :attr:`params` with keyword arguments.
+
+        Returns
+        -------
+        Flight
+            Flight trajectory with aircraft performance data.
+        """
+        self.update_params(params)
+        self.set_source(source)
+        self.source = self.require_source_type(Flight)
+        self.downselect_met()
+        self.set_source_met()
+        self._cleanup_indices()
+
+        # Calculate true airspeed if not included on source
+        self.ensure_true_airspeed_on_source()
+
+        if isinstance(self.source, Fleet):
+            fls = [self.eval_flight(fl) for fl in self.source.to_flight_list()]
+            self.source = Fleet.from_seq(fls, attrs=self.source.attrs, broadcast_numeric=False)
+            return self.source
+
+        self.source = self.eval_flight(self.source)
+        return self.source
+
+    @abc.abstractmethod
+    def eval_flight(self, fl: Flight) -> Flight:
+        """Evaluate the aircraft performance model on a single flight trajectory.
+
         The implementing model adds the following fields to the source flight:
 
         - ``aircraft_mass``: aircraft mass at each waypoint, [:math:`kg`]
@@ -128,18 +158,6 @@ class AircraftPerformance(Model):
         - ``max_mach``: maximum Mach number
         - ``max_altitude``: maximum altitude, [:math:`m`]
         - ``total_fuel_burn``: total fuel burn, [:math:`kg`]
-
-        Parameters
-        ----------
-        source : Flight
-            Flight trajectory to evaluate.
-        params : Any
-            Override :attr:`params` with keyword arguments.
-
-        Returns
-        -------
-        Flight
-            Flight trajectory with aircraft performance data.
         """
 
     @override

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/core/airports.py

@@ -162,7 +162,7 @@ def find_nearest_airport(
     ) & airports["latitude"].between((latitude - bbox), (latitude + bbox))
 
     # Find the nearest airport from largest to smallest airport type
-    search_priority = ["large_airport", "medium_airport", "small_airport"]
+    search_priority = ("large_airport", "medium_airport", "small_airport")
 
     for airport_type in search_priority:
         is_airport_type = airports["type"] == airport_type
@@ -171,7 +171,7 @@ def find_nearest_airport(
         if len(nearest_airports) == 1:
             return nearest_airports["icao_code"].values[0]
 
-        elif len(nearest_airports) > 1:
+        if len(nearest_airports) > 1:
             distance = distance_to_airports(
                 nearest_airports,
                 longitude,
@@ -181,8 +181,7 @@ def find_nearest_airport(
             i_nearest = np.argmin(distance)
             return nearest_airports["icao_code"].values[i_nearest]
 
-        else:
-            continue
+        continue
 
     return None
 

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/core/fleet.py

@@ -133,18 +133,19 @@ class Fleet(Flight):
 
     @override
     def copy(self, **kwargs: Any) -> Self:
-        kwargs.setdefault("fuel", self.fuel)
         kwargs.setdefault("fl_attrs", self.fl_attrs)
+        kwargs.setdefault("final_waypoints", self.final_waypoints)
         return super().copy(**kwargs)
 
     @override
     def filter(self, mask: npt.NDArray[np.bool_], copy: bool = True, **kwargs: Any) -> Self:
-        kwargs.setdefault("fuel", self.fuel)
-
         flight_ids = set(np.unique(self["flight_id"][mask]))
         fl_attrs = {k: v for k, v in self.fl_attrs.items() if k in flight_ids}
         kwargs.setdefault("fl_attrs", fl_attrs)
 
+        final_waypoints = np.array(self.final_waypoints[mask], copy=copy)
+        kwargs.setdefault("final_waypoints", final_waypoints)
+
         return super().filter(mask, copy=copy, **kwargs)
 
     @override
@@ -187,8 +188,9 @@ class Fleet(Flight):
             in ``seq``.
         """
 
+        # Create a shallow copy because we add additional keys in _validate_fl
         def _shallow_copy(fl: Flight) -> Flight:
-            return Flight(VectorDataDict(fl.data), attrs=fl.attrs, copy=False, fuel=fl.fuel)
+            return Flight._from_fastpath(fl.data, fl.attrs, fuel=fl.fuel)
 
         def _maybe_warn(fl: Flight) -> Flight:
             if not fl:
@@ -217,7 +219,18 @@ class Fleet(Flight):
             )
 
         data = {var: np.concatenate([fl[var] for fl in seq]) for var in seq[0]}
-        return cls(data=data, attrs=attrs, copy=False, fuel=fuel, fl_attrs=fl_attrs)
+
+        final_waypoints = np.zeros(data["time"].size, dtype=bool)
+        final_waypoint_indices = np.cumsum([fl.size for fl in seq]) - 1
+        final_waypoints[final_waypoint_indices] = True
+
+        return cls._from_fastpath(
+            data,
+            attrs,
+            fuel=fuel,
+            fl_attrs=fl_attrs,
+            final_waypoints=final_waypoints,
+        )
 
     @property
     def n_flights(self) -> int:
@@ -246,11 +259,19 @@ class Fleet(Flight):
             List of Flights in the same order as was passed into the ``Fleet`` instance.
         """
         indices = self.dataframe.groupby("flight_id", sort=False).indices
+        if copy:
+            return [
+                Flight._from_fastpath(
+                    {k: v[idx] for k, v in self.data.items()},
+                    self.fl_attrs[flight_id],
+                    fuel=self.fuel,
+                ).copy()
+                for flight_id, idx in indices.items()
+            ]
         return [
-            Flight(
-                data=VectorDataDict({k: v[idx] for k, v in self.data.items()}),
-                attrs=self.fl_attrs[flight_id],
-                copy=copy,
+            Flight._from_fastpath(
+                {k: v[idx] for k, v in self.data.items()},
+                self.fl_attrs[flight_id],
                 fuel=self.fuel,
             )
             for flight_id, idx in indices.items()

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/core/flight.py

@@ -891,7 +891,7 @@ class Flight(GeoVectorDataset):
         """
         methods = "geodesic", "linear"
         if fill_method not in methods:
-            raise ValueError(f'Unknown `fill_method`. Supported  methods: {", ".join(methods)}')
+            raise ValueError(f"Unknown `fill_method`. Supported  methods: {', '.join(methods)}")
 
         # STEP 0: If self is empty, return an empty flight
         if not self:
@@ -957,7 +957,12 @@ class Flight(GeoVectorDataset):
                 msg = f"{msg} Pass 'keep_original_index=True' to keep the original index."
             warnings.warn(msg)
 
-        return type(self)(data=df, attrs=self.attrs, fuel=self.fuel)
+        # Reorder columns (this is unimportant but makes the output more canonical)
+        coord_names = ("longitude", "latitude", "altitude", "time")
+        df = df[[*coord_names, *[c for c in df.columns if c not in set(coord_names)]]]
+
+        data = {k: v.to_numpy() for k, v in df.items()}
+        return type(self)._from_fastpath(data, attrs=self.attrs, fuel=self.fuel)
 
     def clean_and_resample(
         self,
@@ -1977,9 +1982,7 @@ def filter_altitude(
             result[i0:i1] = altitude_filt[i0:i1]
 
     # reapply Savitzky-Golay filter to smooth climb and descent
-    result = _sg_filter(result, window_length=kernel_size)
-
-    return result
+    return _sg_filter(result, window_length=kernel_size)
 
 
 def segment_duration(

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/core/flightplan.py

@@ -21,24 +21,24 @@ def to_atc_plan(plan: dict[str, Any]) -> str:
     --------
     :func:`parse_atc_plan`
     """
-    ret = f'(FPL-{plan["callsign"]}-{plan["flight_rules"]}'
-    ret += f'{plan["type_of_flight"]}\n'
+    ret = f"(FPL-{plan['callsign']}-{plan['flight_rules']}"
+    ret += f"{plan['type_of_flight']}\n"
     ret += "-"
     if "number_aircraft" in plan and plan["number_aircraft"] <= 10:
         ret += plan["number_aircraft"]
-    ret += f'{plan["type_of_aircraft"]}/{plan["wake_category"]}-'
-    ret += f'{plan["equipment"]}/{plan["transponder"]}\n'
-    ret += f'-{plan["departure_icao"]}{plan["time"]}\n'
-    ret += f'-{plan["speed_type"]}{plan["speed"]}{plan["level_type"]}'
-    ret += f'{plan["level"]} {plan["route"]}\n'
+    ret += f"{plan['type_of_aircraft']}/{plan['wake_category']}-"
+    ret += f"{plan['equipment']}/{plan['transponder']}\n"
+    ret += f"-{plan['departure_icao']}{plan['time']}\n"
+    ret += f"-{plan['speed_type']}{plan['speed']}{plan['level_type']}"
+    ret += f"{plan['level']} {plan['route']}\n"
     if "destination_icao" in plan and "duration" in plan:
-        ret += f'-{plan["destination_icao"]}{plan["duration"]}'
+        ret += f"-{plan['destination_icao']}{plan['duration']}"
     if "alt_icao" in plan:
-        ret += f' {plan["alt_icao"]}'
+        ret += f" {plan['alt_icao']}"
     if "second_alt_icao" in plan:
-        ret += f' {plan["second_alt_icao"]}'
+        ret += f" {plan['second_alt_icao']}"
     ret += "\n"
-    ret += f'-{plan["other_info"]})\n'
+    ret += f"-{plan['other_info']})\n"
     if "supplementary_info" in plan:
         ret += " ".join([f"{i[0]}/{i[1]}" for i in plan["supplementary_info"].items()])
 

{pycontrails-0.54.4 → pycontrails-0.54.6}/pycontrails/core/interpolation.py

@@ -245,6 +245,9 @@ def _pick_method(scipy_version: str, method: str) -> str:
     str
         Interpolation method adjusted for compatibility with this class.
     """
+    if method == "linear":
+        return method
+
     try:
         version = scipy_version.split(".")
         major = int(version[0])
@@ -486,15 +489,15 @@ def interp(
         da = _localize(da, coords)
 
     indexes = da._indexes
-    x = indexes["longitude"].index.to_numpy()  # type: ignore[attr-defined]
-    y = indexes["latitude"].index.to_numpy()  # type: ignore[attr-defined]
-    z = indexes["level"].index.to_numpy()  # type: ignore[attr-defined]
+    x = indexes["longitude"].index.values  # type: ignore[attr-defined]
+    y = indexes["latitude"].index.values  # type: ignore[attr-defined]
+    z = indexes["level"].index.values  # type: ignore[attr-defined]
     if any(v.dtype != np.float64 for v in (x, y, z)):
         msg = "da must have float64 dtype for longitude, latitude, and level coordinates"
         raise ValueError(msg)
 
     # Convert t and time to float64
-    t = indexes["time"].index.to_numpy()  # type: ignore[attr-defined]
+    t = indexes["time"].index.values  # type: ignore[attr-defined]
     offset = t[0]
     t = _floatize_time(t, offset)