h5netcdf 1.6.1__tar.gz → 1.6.3__tar.gz

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/CHANGELOG.rst

@@ -1,50 +1,70 @@
 Change Log
 ----------
 
+Version 1.6.3 (June 30th, 2025):
+
+- fix invalid string format specifier, match raises/warns with messages in test suite,
+  remove tests for h5py < 3.7, fix sphinx issue and pr roles in CHANGELOG.rst (:issue:`269`, :pull:`270`).
+  By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
+
+Version 1.6.2 (June 26th, 2025):
+
+- Codespell fixes (:pull:`261`).
+  By `Kurt Schwehr <https://github.com/schwehr>`_
+- Fix hsds/h5pyd test fixture spinup issues (:pull:`265`).
+  By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
+- Fix and add circular referrer tests for Python 3.14 and update CI matrix (:pull:`264`).
+  By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
+- Avoid opening h5pyd file to check if there is a preexisting file,
+  instead remap mode "a" -> "r+", resort to "w" if file doesn't exist (:issue:`262`, :pull:`266`).
+  By `Jonas Grönberg <https://github.com/JonasGronberg>`_ and `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
+- Reduce CI time by installing available scientific-python-nightly-wheels and using pip cache (:pull:`267`).
+  By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
+
 Version 1.6.1 (March 7th, 2025):
 
 - Let Variable.chunks return None for scalar variables, independent of what the underlying
-  h5ds object returns ({pull}`259`).
+  h5ds object returns (:pull:`259`).
   By `Rickard Holmberg <https://github.com/rho-novatron>`_
 
 Version 1.6.0 (March 7th, 2025):
 
-- Allow specifying `h5netcdf.File(driver="h5pyd")` to force the use of h5pyd ({issue}`255`, {pull}`256`).
+- Allow specifying `h5netcdf.File(driver="h5pyd")` to force the use of h5pyd (:issue:`255`, :pull:`256`).
   By `Rickard Holmberg <https://github.com/rho-novatron>`_
-- Add pytest-mypy-plugins for xarray nightly test ({pull}`257`).
+- Add pytest-mypy-plugins for xarray nightly test (:pull:`257`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
 
 Version 1.5.0 (January 26th, 2025):
 
-- Update CI to new versions (Python 3.13, 3.14 alpha), remove numpy 1 from h5pyd runs ({pull}`250`).
+- Update CI to new versions (Python 3.13, 3.14 alpha), remove numpy 1 from h5pyd runs (:pull:`250`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
-- Update CI and reinstate h5pyd/hsds test runs ({pull}`247`).
+- Update CI and reinstate h5pyd/hsds test runs (:pull:`247`).
   By `John Readey  <https://github.com/jreadey>`_
 - Allow ``zlib`` to be used as an alias for ``gzip`` for enhanced compatibility with h5netcdf's API and xarray.
   By `Mark Harfouche <https://github.com/hmaarrfk>`_
 
 Version 1.4.1 (November 13th, 2024):
 
-- Add CI run for hdf5 1.10.6, fix complex tests, fix enum/user type tests ({pull}`244`).
+- Add CI run for hdf5 1.10.6, fix complex tests, fix enum/user type tests (:pull:`244`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
 
 
 Version 1.4.0 (October 7th, 2024):
 
-- Add UserType class, add EnumType ({pull}`229`).
+- Add UserType class, add EnumType (:pull:`229`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
-- Refactor fillvalue and dtype handling for user types, enhance sanity checks and tests ({pull}`230`).
+- Refactor fillvalue and dtype handling for user types, enhance sanity checks and tests (:pull:`230`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
-- Add VLType and CompoundType, commit complex compound type to file. Align with nc-complex ({pull}`227`).
+- Add VLType and CompoundType, commit complex compound type to file. Align with nc-complex (:pull:`227`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
 - Update h5pyd testing.
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
-- CI and lint maintenance ({pull}`235`).
+- CI and lint maintenance (:pull:`235`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
 - Support wrapping an h5py ``File`` object. Closing the h5netcdf file object
-  does not close the h5py file ({pull}`238`).
+  does not close the h5py file (:pull:`238`).
   By `Thomas Kluyver <https://github.com/takluyver>`_
-- CI and lint maintenance (format README.rst, use more f-strings, change Python 3.9 to 3.10 in CI) ({pull}`239`).
+- CI and lint maintenance (format README.rst, use more f-strings, change Python 3.9 to 3.10 in CI) (:pull:`239`).
   By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
 
 Version 1.3.0 (November 7th, 2023):

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: h5netcdf
-Version: 1.6.1
+Version: 1.6.3
 Summary: netCDF4 via h5py
 Author-email: Stephan Hoyer <shoyer@gmail.com>, Kai Mühlbauer <kmuehlbauer@wradlib.org>
 Maintainer-email: h5netcdf developers <devteam@h5netcdf.org>
@@ -56,6 +56,7 @@ Requires-Dist: packaging
 Provides-Extra: test
 Requires-Dist: netCDF4; extra == "test"
 Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
 
 h5netcdf
 ========
@@ -318,7 +319,7 @@ The following describes the behavior of h5netcdf with respect to order tracking
 for a few key versions:
 
 - Version 0.12.0 and earlier, the ``track_order`` parameter`order was missing
-  and thus order tracking was implicitely set to ``False``.
+  and thus order tracking was implicitly set to ``False``.
 - Version 0.13.0 enabled order tracking by setting the parameter
   ``track_order`` to ``True`` by default without deprecation.
 - Versions 0.13.1 to 1.0.2 set ``track_order`` to ``False`` due to a bug in a

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/README.rst

@@ -259,7 +259,7 @@ The following describes the behavior of h5netcdf with respect to order tracking
 for a few key versions:
 
 - Version 0.12.0 and earlier, the ``track_order`` parameter`order was missing
-  and thus order tracking was implicitely set to ``False``.
+  and thus order tracking was implicitly set to ``False``.
 - Version 0.13.0 enabled order tracking by setting the parameter
   ``track_order`` to ``True`` by default without deprecation.
 - Versions 0.13.1 to 1.0.2 set ``track_order`` to ``False`` due to a bug in a

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/doc/conf.py

@@ -117,7 +117,7 @@ napoleon_type_aliases = {
     "Path": "~~pathlib.Path",
 }
 
-# handle release substition
+# handle release substitution
 url = "https://github.com/h5netcdf"
 
 # get version

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/doc/devguide.rst

@@ -19,6 +19,8 @@ Contributors
 - `Frédéric Laliberté <https://github.com/laliberte>`_
 - `Ghislain Vaillant <https://github.com/ghisvail>`_
 - `John Readey <https://github.com/jreadey>`_
+- `Jonas Grönberg <https://github.com/JonasGronberg>`_
+- `Kurt Schwehr <https://github.com/schwehr>`_
 - `Lion Krischer <https://github.com/krischer>`_
 - `Mark Harfouche <https://github.com/hmaarrfk>`_
 - `Martin Raspaud <https://github.com/mraspaud>`_

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/doc/index.rst

@@ -50,7 +50,7 @@ by Stephan Hoyer. The first `official` ``h5netcdf`` announcement was made by Ste
 `xarray issue tracker`_ only one day later.
 
 The library evolved constantly over the years (fixing bugs and adding enhancements)
-and gained contributions from 19 other :ref:`contributors` so far. The library is widely used,
+and gained contributions from 21 other :ref:`contributors` so far. The library is widely used,
 especially as backend within `xarray`_.
 
 Early 2020 Kai Mühlbauer started to add contributions and after some time he volunteered

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/h5netcdf/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '1.6.1'
-__version_tuple__ = version_tuple = (1, 6, 1)
+__version__ = version = '1.6.3'
+__version_tuple__ = version_tuple = (1, 6, 3)

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/h5netcdf/core.py

@@ -982,16 +982,16 @@ class Group(Mapping):
         for k, v in self._all_dimensions.maps[0].items():
             if k in value:
                 if v != value[k]:
-                    raise ValueError(f"cannot modify existing dimension {k:!r}")
+                    raise ValueError(f"cannot modify existing dimension {k!r}")
             else:
                 raise ValueError(
-                    f"new dimensions do not include existing dimension {k:!r}"
+                    f"new dimensions do not include existing dimension {k!r}"
                 )
         self._dimensions.update(value)
 
     def _create_child_group(self, name):
         if name in self:
-            raise ValueError(f"unable to create group {name:!r} (name already exists)")
+            raise ValueError(f"unable to create group {name!r} (name already exists)")
         kwargs = {}
         kwargs.update(track_order=self._track_order)
 
@@ -1035,7 +1035,7 @@ class Group(Mapping):
     ):
         if name in self:
             raise ValueError(
-                f"unable to create variable {name:!r} (name already exists)"
+                f"unable to create variable {name!r} (name already exists)"
             )
         if data is not None:
             data = np.asarray(data)
@@ -1519,16 +1519,35 @@ class File(Group):
                             "No module named 'h5pyd'. h5pyd is required for "
                             f"opening urls: {path}"
                         )
+                    self._preexisting_file = mode in {"r", "r+", "a"}
+                    # remap "a" -> "r+" to check file existence
+                    # fallback to "w" if not
+                    _mode = mode
+                    if mode == "a":
+                        mode = "r+"
+                    self._h5py = h5pyd
                     try:
-                        with h5pyd.File(path, "r", **kwargs) as f:  # noqa
-                            pass
-                        self._preexisting_file = True
+                        self._h5file = self._h5py.File(
+                            path, mode, track_order=track_order, **kwargs
+                        )
+                        self._preexisting_file = mode != "w"
                     except OSError:
-                        self._preexisting_file = False
-                    self._h5py = h5pyd
-                    self._h5file = self._h5py.File(
-                        path, mode, track_order=track_order, **kwargs
-                    )
+                        # if file does not exist, create it
+                        if _mode == "a":
+                            mode = "w"
+                            self._h5file = self._h5py.File(
+                                path, mode, track_order=track_order, **kwargs
+                            )
+                            self._preexisting_file = False
+                            msg = (
+                                "Append mode for h5pyd now probes with 'r+' first and "
+                                "only falls back to 'w' if the file is missing.\n"
+                                "To silence this warning use 'r+' (open-existing) or 'w' "
+                                "(create-new) directly."
+                            )
+                            warnings.warn(msg, UserWarning, stacklevel=2)
+                        else:
+                            raise
                 else:
                     self._preexisting_file = os.path.exists(path) and mode != "w"
                     self._h5py = h5py

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/h5netcdf/dimensions.py

@@ -22,7 +22,7 @@ class Dimensions(MutableMapping):
         if not self._group._root._writable:
             raise RuntimeError("H5NetCDF: Write to read only")
         if name in self._objects:
-            raise ValueError(f"dimension {name:!r} already exists")
+            raise ValueError(f"dimension {name!r} already exists")
 
         self._objects[name] = Dimension(self._group, name, size, create_h5ds=True)
 

h5netcdf-1.6.3/h5netcdf/tests/conftest.py

@@ -0,0 +1,83 @@
+import os
+import tempfile
+import time
+from pathlib import Path
+from shutil import rmtree
+
+import pytest
+
+try:
+    from h5pyd import Folder
+    from hsds.hsds_app import HsdsApp
+
+    with_reqd_pkgs = True
+except ImportError:
+    with_reqd_pkgs = False
+
+
+@pytest.fixture(scope="session")
+def hsds_up():
+    """Provide HDF Highly Scalable Data Service (HSDS) for h5pyd testing."""
+    if not with_reqd_pkgs:
+        pytest.skip("Required packages h5pyd and hsds not available")
+
+    root_dir = Path(tempfile.mkdtemp(prefix="tmp-hsds-root-"))
+    bucket_name = "pytest"
+    os.environ["BUCKET_NAME"] = bucket_name
+    # need to create a directory for our bucket
+    (root_dir / bucket_name).mkdir()
+
+    kwargs = {
+        "username": "h5netcdf-pytest",
+        "password": "TestEarlyTestEverything",
+        "root_dir": str(root_dir),
+        "logfile": str(root_dir / "hsds.log"),
+        "log_level": "DEBUG",
+        "host": "localhost",
+        "sn_port": 5101,
+    }
+
+    os.environ.update(
+        {
+            "BUCKET_NAME": bucket_name,
+            "HS_USERNAME": kwargs["username"],
+            "HS_PASSWORD": kwargs["password"],
+            "HS_USE_HTTPS": "False",
+        }
+    )
+
+    hsds = HsdsApp(**kwargs)
+
+    try:
+        hsds.run()
+        timeout = time.time() + 60
+        while not hsds.ready:
+            if time.time() > timeout:
+                raise TimeoutError("HSDS server did not become ready in time")
+            time.sleep(1)
+
+        os.environ["HS_ENDPOINT"] = hsds.endpoint
+        # make folders expected by pytest
+        Folder("/home/", mode="w")
+        Folder("/home/h5netcdf-pytest/", mode="w")
+
+        yield True
+
+    except Exception as err:
+        log_path = kwargs["logfile"]
+        if os.path.exists(log_path):
+            with open(log_path) as f:
+                print("\n=== HSDS Log ===")
+                print(f.read())
+        else:
+            print(f"HSDS log not found at: {log_path}")
+        raise err
+
+    finally:
+        try:
+            hsds.check_processes()
+            hsds.stop()
+        except Exception:
+            pass
+
+    rmtree(root_dir, ignore_errors=True)

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/h5netcdf/tests/test_h5netcdf.py

@@ -3,7 +3,9 @@ import io
 import random
 import re
 import string
+import sys
 import tempfile
+import weakref
 from os import environ as env
 
 import h5py
@@ -11,7 +13,7 @@ import netCDF4
 import numpy as np
 import pytest
 from packaging import version
-from pytest import raises
+from pytest import raises, warns
 
 import h5netcdf
 from h5netcdf import legacyapi
@@ -30,6 +32,7 @@ except ImportError:
 
 
 remote_h5 = ("http:", "hdf5:")
+python_version = version.parse(".".join(map(str, sys.version_info[:3])))
 
 
 @pytest.fixture
@@ -37,26 +40,30 @@ def tmp_local_netcdf(tmpdir):
     return str(tmpdir.join("testfile.nc"))
 
 
+@pytest.fixture()
+def setup_h5pyd_config(hsds_up):
+    env["HS_ENDPOINT"] = "http://127.0.0.1:5101"
+    env["HS_USERNAME"] = "h5netcdf-pytest"
+    env["HS_PASSWORD"] = "TestEarlyTestEverything"
+    env["HS_USE_HTTPS"] = "False"
+
+
 @pytest.fixture(params=["testfile.nc", "hdf5://testfile"])
-def tmp_local_or_remote_netcdf(request, tmpdir, hsds_up):
-    if request.param.startswith(remote_h5):
-        if without_h5pyd:
-            pytest.skip("h5pyd package not available")
-        elif not hsds_up:
-            pytest.skip("HSDS service not running")
-        rnd = "".join(random.choice(string.ascii_uppercase) for _ in range(5))
-        return (
-            "hdf5://"
-            + "home"
-            + "/"
-            + env["HS_USERNAME"]
-            + "/"
-            + "testfile"
-            + rnd
-            + ".nc"
-        )
+def tmp_local_or_remote_netcdf(request, tmpdir):
+    param = request.param
+    if param.startswith(remote_h5):
+        try:
+            hsds_up = request.getfixturevalue("hsds_up")
+        except pytest.skip.Exception:
+            pytest.skip("HSDS not available")
+
+        if not hsds_up:
+            pytest.skip("HSDS fixture returned False (not running)")
+
+        rnd = "".join(random.choices(string.ascii_uppercase, k=5))
+        return f"hdf5://home/{env['HS_USERNAME']}/testfile{rnd}.nc"
     else:
-        return str(tmpdir.join(request.param))
+        return str(tmpdir.join(param))
 
 
 @pytest.fixture(params=[True, False])
@@ -157,7 +164,10 @@ def write_legacy_netcdf(tmp_netcdf, write_module):
     v = ds.createVariable("foo_unlimited", float, ("x", "unlimited"))
     v[...] = 1
 
-    with raises((h5netcdf.CompatibilityError, TypeError)):
+    with raises(
+        (h5netcdf.CompatibilityError, TypeError),
+        match=r"(?i)(boolean dtypes are not a supported NetCDF feature|illegal primitive data type)",
+    ):
         ds.createVariable("boolean", np.bool_, ("x"))
 
     g = ds.createGroup("subgroup")
@@ -250,7 +260,7 @@ def read_legacy_netcdf(tmp_netcdf, read_module, write_module):
     if write_module is not netCDF4:
         # skip for now: https://github.com/Unidata/netcdf4-python/issues/388
         assert ds.other_attr == "yes"
-    with pytest.raises(AttributeError):
+    with raises(AttributeError, match="not found"):
         ds.does_not_exist
     assert set(ds.dimensions) == set(
         ["x", "y", "z", "empty", "string3", "mismatched_dim", "unlimited"]
@@ -646,25 +656,27 @@ def test_optional_netcdf4_attrs(tmp_local_or_remote_netcdf):
 def test_error_handling(tmp_local_or_remote_netcdf):
     with h5netcdf.File(tmp_local_or_remote_netcdf, "w") as ds:
         ds.dimensions["x"] = 1
-        with raises(ValueError):
+        with raises(ValueError, match="already exists"):
             ds.dimensions["x"] = 2
-        with raises(ValueError):
+        with raises(ValueError, match="cannot modify existing dimension"):
             ds.dimensions = {"x": 2}
-        with raises(ValueError):
+        with raises(
+            ValueError, match="new dimensions do not include existing dimension"
+        ):
             ds.dimensions = {"y": 3}
         ds.create_variable("x", ("x",), dtype=float)
-        with raises(ValueError):
+        with raises(ValueError, match="unable to create variable"):
             ds.create_variable("x", ("x",), dtype=float)
-        with raises(ValueError):
+        with raises(ValueError, match="name parameter cannot be an empty string"):
             ds.create_variable("y/", ("x",), dtype=float)
         ds.create_group("subgroup")
-        with raises(ValueError):
+        with raises(ValueError, match="unable to create group"):
             ds.create_group("subgroup")
 
 
 def test_decode_string_error(tmp_local_or_remote_netcdf):
     write_h5netcdf(tmp_local_or_remote_netcdf)
-    with pytest.raises(TypeError):
+    with raises(TypeError, match="keyword argument is not allowed"):
         with h5netcdf.legacyapi.Dataset(
             tmp_local_or_remote_netcdf, "r", decode_vlen_strings=True
         ) as ds:
@@ -731,10 +743,10 @@ def test_invalid_netcdf4(tmp_local_or_remote_netcdf):
                 check_invalid_netcdf4(var, i)
 
     with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
-        with raises(ValueError):
+        with raises(ValueError, match="has no dimension scale associated"):
             ds["bar"].variables["foo1"].dimensions
 
-    with raises(ValueError):
+    with raises(ValueError, match="unknown value"):
         with h5netcdf.File(tmp_local_or_remote_netcdf, "r", phony_dims="srt") as ds:
             pass
 
@@ -799,7 +811,7 @@ def test_invalid_netcdf4_mixed(tmp_local_or_remote_netcdf):
             check_invalid_netcdf4_mixed(var, 3)
 
     with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
-        with raises(ValueError):
+        with raises(ValueError, match="has no dimension scale associated with"):
             ds.variables["foo1"].dimensions
 
 
@@ -817,12 +829,12 @@ def test_invalid_netcdf_malformed_dimension_scales(tmp_local_or_remote_netcdf):
         f["z"].make_scale()
         f["foo1"].dims[0].attach_scale(f["x"])
 
-    with raises(ValueError):
+    with raises(ValueError, match="has mixing of labeled and unlabeled dimensions"):
         with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
             assert ds
             print(ds)
 
-    with raises(ValueError):
+    with raises(ValueError, match="has mixing of labeled and unlabeled dimensions"):
         with h5netcdf.File(tmp_local_or_remote_netcdf, "r", phony_dims="sort") as ds:
             assert ds
             print(ds)
@@ -936,14 +948,17 @@ def test_invalid_netcdf_error(tmp_local_or_remote_netcdf):
         f.create_variable(
             "lzf_compressed", data=[1], dimensions=("x"), compression="lzf"
         )
-        with pytest.raises(h5netcdf.CompatibilityError):
+        with raises(
+            h5netcdf.CompatibilityError,
+            match="scale-offset filters are not a supported NetCDF feature",
+        ):
             f.create_variable("scaleoffset", data=[1], dimensions=("x",), scaleoffset=0)
 
 
 def test_invalid_netcdf_okay(tmp_local_or_remote_netcdf):
     if tmp_local_or_remote_netcdf.startswith(remote_h5):
         pytest.skip("h5pyd does not support NumPy complex dtype yet")
-    with pytest.warns(UserWarning, match="invalid netcdf features"):
+    with warns(UserWarning, match="invalid netcdf features"):
         with h5netcdf.File(tmp_local_or_remote_netcdf, "w", invalid_netcdf=True) as f:
             f.create_variable(
                 "lzf_compressed", data=[1], dimensions=("x"), compression="lzf"
@@ -965,7 +980,7 @@ def test_invalid_netcdf_overwrite_valid(tmp_local_netcdf):
     # https://github.com/h5netcdf/h5netcdf/issues/165
     with netCDF4.Dataset(tmp_local_netcdf, mode="w"):
         pass
-    with pytest.warns(UserWarning):
+    with warns(UserWarning, match="You are writing invalid netcdf features"):
         with h5netcdf.File(tmp_local_netcdf, "a", invalid_netcdf=True) as f:
             f.create_variable(
                 "lzf_compressed", data=[1], dimensions=("x"), compression="lzf"
@@ -994,7 +1009,7 @@ def test_reopen_file_different_dimension_sizes(tmp_local_netcdf):
 
 
 def test_invalid_then_valid_no_ncproperties(tmp_local_or_remote_netcdf):
-    with pytest.warns(UserWarning, match="invalid netcdf features"):
+    with warns(UserWarning, match="invalid netcdf features"):
         with h5netcdf.File(tmp_local_or_remote_netcdf, "w", invalid_netcdf=True):
             pass
     with h5netcdf.File(tmp_local_or_remote_netcdf, "a"):
@@ -1012,11 +1027,8 @@ def test_creating_and_resizing_unlimited_dimensions(tmp_local_or_remote_netcdf):
         f.dimensions["z"] = None
         f.resize_dimension("z", 20)
 
-        with pytest.raises(ValueError) as e:
+        with raises(ValueError, match="is not unlimited and thus cannot be resized"):
             f.resize_dimension("y", 20)
-        assert e.value.args[0] == (
-            "Dimension 'y' is not unlimited and thus cannot be resized."
-        )
 
     h5 = get_hdf5_module(tmp_local_or_remote_netcdf)
     # Assert some behavior observed by using the C netCDF bindings.
@@ -1042,11 +1054,10 @@ def test_creating_variables_with_unlimited_dimensions(tmp_local_or_remote_netcdf
 
         # Trying to create a variable while the current size of the dimension
         # is still zero will fail.
-        with pytest.raises(ValueError) as e:
+        with raises(ValueError, match="Shape tuple is incompatible with data"):
             f.create_variable(
                 "dummy2", data=np.array([[1, 2], [3, 4]]), dimensions=("x", "y")
             )
-        assert e.value.args[0] == "Shape tuple is incompatible with data"
 
         # Creating a coordinate variable
         f.create_variable("x", dimensions=("x",), dtype=np.int64)
@@ -1071,7 +1082,7 @@ def test_creating_variables_with_unlimited_dimensions(tmp_local_or_remote_netcdf
             # We don't expect any errors. This is effectively a void context manager
             expected_errors = memoryview(b"")
         else:
-            expected_errors = pytest.raises(TypeError)
+            expected_errors = raises(TypeError, match="Can't broadcast")
         with expected_errors as e:
             f.variables["dummy3"][:] = np.ones((5, 2))
         if not tmp_local_or_remote_netcdf.startswith(remote_h5):
@@ -1108,11 +1119,10 @@ def test_writing_to_an_unlimited_dimension(tmp_local_or_remote_netcdf):
         f.dimensions["z"] = None
 
         # Cannot create it without first resizing it.
-        with pytest.raises(ValueError) as e:
+        with raises(ValueError, match="Shape tuple is incompatible with data"):
             f.create_variable(
                 "dummy1", data=np.array([[1, 2, 3]]), dimensions=("x", "y")
             )
-            assert e.value.args[0] == "Shape tuple is incompatible with data"
 
         # Without data.
         f.create_variable("dummy1", dimensions=("x", "y"), dtype=np.int64)
@@ -1141,7 +1151,9 @@ def test_writing_to_an_unlimited_dimension(tmp_local_or_remote_netcdf):
 
         # broadcast writing
         if tmp_local_or_remote_netcdf.startswith(remote_h5):
-            expected_errors = pytest.raises(OSError)
+            expected_errors = raises(
+                OSError, match="Got asyncio.IncompleteReadError during binary read"
+            )
         else:
             # We don't expect any errors. This is effectively a void context manager
             expected_errors = memoryview(b"")
@@ -1300,6 +1312,24 @@ def test_overwrite_existing_file(tmp_local_netcdf):
         assert ds.attrs._h5attrs.get("_NCProperties", False)
 
 
+def test_overwrite_existing_remote_file(tmp_local_or_remote_netcdf):
+    # create file with legacyapi
+    with legacyapi.Dataset(tmp_local_or_remote_netcdf, "w") as ds:
+        ds.createDimension("x", 10)
+
+    # check attribute
+    with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
+        assert ds.attrs._h5attrs.get("_NCProperties", False)
+
+    # overwrite file with new api
+    with h5netcdf.File(tmp_local_or_remote_netcdf, "w") as ds:
+        ds.dimensions["x"] = 10
+
+    # check attribute
+    with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
+        assert ds.attrs._h5attrs.get("_NCProperties", False)
+
+
 def test_scales_on_append(tmp_local_netcdf):
     # create file with _NCProperties attribute
     with netCDF4.Dataset(tmp_local_netcdf, "w") as ds:
@@ -1555,7 +1585,38 @@ def test_no_circular_references(tmp_local_or_remote_netcdf):
         refs = gc.get_referrers(ds)
         for ref in refs:
             print(ref)
-        assert len(refs) == 1
+        if python_version >= version.parse("3.14"):
+            assert len(refs) == 0
+        else:
+            assert len(refs) == 1
+
+
+def test_no_circular_references_py314(tmp_local_or_remote_netcdf):
+    # https://github.com/h5py/h5py/issues/2019
+    with h5netcdf.File(tmp_local_or_remote_netcdf, "w") as ds:
+        ds.dimensions["x"] = 2
+        ds.dimensions["y"] = 2
+
+    # clean up everything
+    gc.collect()
+    gc.garbage.clear()
+
+    # use weakref to hold on object
+    file_ref = None
+    with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
+        file_ref = weakref.ref(ds)
+
+    # clean up
+    gc.collect()
+
+    # check garbage list
+    if file_ref() is not None:
+        print("Uncollectable object:", file_ref())
+        print("Potential GC garbage:")
+        for obj in gc.garbage:
+            print(repr(obj))
+
+    assert file_ref() is None or "<Closed h5netcdf.File>"
 
 
 def test_expanded_variables_netcdf4(tmp_local_netcdf, netcdf_write_module):
@@ -1693,7 +1754,7 @@ def test_track_order_specification(tmp_local_netcdf):
     # While netcdf4-c has historically only allowed track_order to be True
     # There doesn't seem to be a good reason for this
     # https://github.com/Unidata/netcdf-c/issues/2054 historically, h5netcdf
-    # has not specified this parameter (leaving it implicitely as False)
+    # has not specified this parameter (leaving it implicitly as False)
     # We want to make sure we allow both here
     with h5netcdf.File(tmp_local_netcdf, "w", track_order=False):
         pass
@@ -1717,18 +1778,10 @@ def test_more_than_7_attr_creation(tmp_local_netcdf):
 # https://github.com/h5netcdf/h5netcdf/issues/136#issuecomment-1017457067
 @pytest.mark.parametrize("track_order", [False, True])
 def test_more_than_7_attr_creation_track_order(tmp_local_netcdf, track_order):
-    h5py_version = version.parse(h5py.__version__)
-    if track_order and h5py_version < version.parse("3.7.0"):
-        expected_errors = pytest.raises(KeyError)
-    else:
-        # We don't expect any errors. This is effectively a void context manager
-        expected_errors = memoryview(b"")
-
     with h5netcdf.File(tmp_local_netcdf, "w", track_order=track_order) as h5file:
-        with expected_errors:
-            for i in range(100):
-                h5file.attrs[f"key{i}"] = i
-                h5file.attrs[f"key{i}"] = 0
+        for i in range(100):
+            h5file.attrs[f"key{i}"] = i
+            h5file.attrs[f"key{i}"] = 0
 
 
 def test_group_names(tmp_local_netcdf):
@@ -1815,18 +1868,11 @@ def test_bool_slicing_length_one_dim(tmp_local_netcdf):
         data = ds["hello"][bool_slice, :]
         np.testing.assert_equal(data, np.zeros((1, 2)))
 
-    # should raise for h5py >= 3.0.0 and h5py < 3.7.0
+    # regression test
     # https://github.com/h5py/h5py/pull/2079
     # https://github.com/h5netcdf/h5netcdf/pull/125/
     with h5netcdf.File(tmp_local_netcdf, "r") as ds:
-        h5py_version = version.parse(h5py.__version__)
-        if version.parse("3.0.0") <= h5py_version < version.parse("3.7.0"):
-            error = "Indexing arrays must have integer dtypes"
-            with pytest.raises(TypeError) as e:
-                ds["hello"][bool_slice, :]
-            assert error == str(e.value)
-        else:
-            ds["hello"][bool_slice, :]
+        ds["hello"][bool_slice, :]
 
 
 def test_fancy_indexing(tmp_local_or_remote_netcdf):
@@ -2247,38 +2293,36 @@ def test_user_type_errors_new_api(tmp_local_or_remote_netcdf):
             enum_type = ds.create_enumtype(np.uint8, "enum_t", enum_dict1)
 
             if tmp_local_or_remote_netcdf.startswith(remote_h5):
-                testcontext = pytest.raises(RuntimeError, match="Conflict")
+                testcontext = raises(RuntimeError, match="Conflict")
             else:
-                testcontext = pytest.raises(
-                    (KeyError, TypeError), match="name already exists"
-                )
+                testcontext = raises((KeyError, TypeError), match="name already exists")
             with testcontext:
                 ds.create_enumtype(np.uint8, "enum_t", enum_dict2)
 
             enum_type2 = g.create_enumtype(np.uint8, "enum_t2", enum_dict2)
             g.create_enumtype(np.uint8, "enum_t", enum_dict2)
-            with pytest.raises(TypeError, match="Please provide h5netcdf user type"):
+            with raises(TypeError, match="Please provide h5netcdf user type"):
                 ds.create_variable(
                     "enum_var1",
                     ("enum_dim",),
                     dtype=enum_type._h5ds,
                     fillvalue=enum_dict1["missing"],
                 )
-            with pytest.raises(TypeError, match="is not committed into current file"):
+            with raises(TypeError, match="is not committed into current file"):
                 ds.create_variable(
                     "enum_var2",
                     ("enum_dim",),
                     dtype=enum_type_ext,
                     fillvalue=enum_dict1["missing"],
                 )
-            with pytest.raises(TypeError, match="is not accessible in current group"):
+            with raises(TypeError, match="is not accessible in current group"):
                 ds.create_variable(
                     "enum_var3",
                     ("enum_dim",),
                     dtype=enum_type2,
                     fillvalue=enum_dict2["missing"],
                 )
-            with pytest.raises(TypeError, match="Another dtype with same name"):
+            with raises(TypeError, match="Another dtype with same name"):
                 g.create_variable(
                     "enum_var4",
                     ("enum_dim",),
@@ -2297,38 +2341,36 @@ def test_user_type_errors_legacyapi(tmp_local_or_remote_netcdf):
             g = ds.createGroup("subgroup")
             enum_type = ds.createEnumType(np.uint8, "enum_t", enum_dict1)
             if tmp_local_or_remote_netcdf.startswith(remote_h5):
-                testcontext = pytest.raises(RuntimeError, match="Conflict")
+                testcontext = raises(RuntimeError, match="Conflict")
             else:
-                testcontext = pytest.raises(
-                    (KeyError, TypeError), match="name already exists"
-                )
+                testcontext = raises((KeyError, TypeError), match="name already exists")
             with testcontext:
                 ds.createEnumType(np.uint8, "enum_t", enum_dict1)
 
             enum_type2 = g.createEnumType(np.uint8, "enum_t2", enum_dict2)
             g.create_enumtype(np.uint8, "enum_t", enum_dict2)
-            with pytest.raises(TypeError, match="Please provide h5netcdf user type"):
+            with raises(TypeError, match="Please provide h5netcdf user type"):
                 ds.createVariable(
                     "enum_var1",
                     enum_type._h5ds,
                     ("enum_dim",),
                     fill_value=enum_dict1["missing"],
                 )
-            with pytest.raises(TypeError, match="is not committed into current file"):
+            with raises(TypeError, match="is not committed into current file"):
                 ds.createVariable(
                     "enum_var2",
                     enum_type_ext,
                     ("enum_dim",),
                     fill_value=enum_dict1["missing"],
                 )
-            with pytest.raises(TypeError, match="is not accessible in current group"):
+            with raises(TypeError, match="is not accessible in current group"):
                 ds.createVariable(
                     "enum_var3",
                     enum_type2,
                     ("enum_dim",),
                     fill_value=enum_dict2["missing"],
                 )
-            with pytest.raises(TypeError, match="Another dtype with same name"):
+            with raises(TypeError, match="Another dtype with same name"):
                 g.createVariable(
                     "enum_var4",
                     enum_type,
@@ -2346,7 +2388,7 @@ def test_enum_type_errors_new_api(tmp_local_or_remote_netcdf):
         enum_type2 = ds.create_enumtype(np.uint8, "enum_t2", enum_dict2)
 
         # 1.
-        with pytest.warns(UserWarning, match="default fill_value 0 which IS defined"):
+        with warns(UserWarning, match="default fill_value 0 which IS defined"):
             ds.create_variable(
                 "enum_var1",
                 ("enum_dim",),
@@ -2354,18 +2396,14 @@ def test_enum_type_errors_new_api(tmp_local_or_remote_netcdf):
             )
         # 2. is for legacyapi only
         # 3.
-        with pytest.warns(
-            UserWarning, match="default fill_value 0 which IS NOT defined"
-        ):
+        with warns(UserWarning, match="default fill_value 0 which IS NOT defined"):
             ds.create_variable(
                 "enum_var2",
                 ("enum_dim",),
                 dtype=enum_type,
             )
         # 4.
-        with pytest.warns(
-            UserWarning, match="with specified fill_value 0 which IS NOT"
-        ):
+        with warns(UserWarning, match="with specified fill_value 0 which IS NOT"):
             ds.create_variable(
                 "enum_var3",
                 ("enum_dim",),
@@ -2373,9 +2411,7 @@ def test_enum_type_errors_new_api(tmp_local_or_remote_netcdf):
                 fillvalue=0,
             )
         # 5.
-        with pytest.raises(
-            ValueError, match="with specified fill_value 100 which IS NOT"
-        ):
+        with raises(ValueError, match="with specified fill_value 100 which IS NOT"):
             ds.create_variable(
                 "enum_var4",
                 ("enum_dim",),
@@ -2393,14 +2429,14 @@ def test_enum_type_errors_legacyapi(tmp_local_or_remote_netcdf):
         enum_type2 = ds.createEnumType(np.uint8, "enum_t2", enum_dict2)
 
         # 1.
-        with pytest.warns(UserWarning, match="default fill_value 255 which IS defined"):
+        with warns(UserWarning, match="default fill_value 255 which IS defined"):
             ds.createVariable(
                 "enum_var1",
                 enum_type2,
                 ("enum_dim",),
             )
         # 2.
-        with pytest.raises(ValueError, match="default fill_value 255 which IS NOT"):
+        with raises(ValueError, match="default fill_value 255 which IS NOT"):
             ds.createVariable(
                 "enum_var2",
                 enum_type,
@@ -2408,9 +2444,7 @@ def test_enum_type_errors_legacyapi(tmp_local_or_remote_netcdf):
             )
         # 3. is only for new api
         # 4.
-        with pytest.warns(
-            UserWarning, match="interpreted as '_UNDEFINED' by netcdf-c."
-        ):
+        with warns(UserWarning, match="interpreted as '_UNDEFINED' by netcdf-c."):
             ds.createVariable(
                 "enum_var3",
                 enum_type,
@@ -2418,9 +2452,7 @@ def test_enum_type_errors_legacyapi(tmp_local_or_remote_netcdf):
                 fill_value=0,
             )
         # 5.
-        with pytest.raises(
-            ValueError, match="with specified fill_value 100 which IS NOT"
-        ):
+        with raises(ValueError, match="with specified fill_value 100 which IS NOT"):
             ds.createVariable("enum_var4", enum_type, ("enum_dim",), fill_value=100)
 
 
@@ -2438,9 +2470,8 @@ def test_enum_type(tmp_local_or_remote_netcdf):
             "enum_var", ("enum_dim",), dtype=enum_type, fillvalue=enum_dict["missing"]
         )
         v[0:3] = [1, 2, 3]
-        with pytest.raises(ValueError) as e:
+        with raises(ValueError, match="assign illegal value"):
             v[3] = 5
-        assert "assign illegal value(s)" in e.value.args[0]
 
     # check, if new API can read them
     with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
@@ -2481,9 +2512,8 @@ def test_enum_type(tmp_local_or_remote_netcdf):
             "enum_var", enum_type, ("enum_dim",), fill_value=enum_dict["missing"]
         )
         v[0:3] = [1, 2, 3]
-        with pytest.raises(ValueError) as e:
+        with raises(ValueError, match="assign illegal value"):
             v[3] = 5
-        assert "assign illegal value(s)" in e.value.args[0]
 
     # check, if new API can read them
     with h5netcdf.File(tmp_local_or_remote_netcdf, "r") as ds:
@@ -2525,9 +2555,7 @@ def test_enum_type(tmp_local_or_remote_netcdf):
                 "enum_var", enum_type, ("enum_dim",), fill_value=enum_dict["missing"]
             )
             v[0:3] = [1, 2, 3]
-            with pytest.raises(
-                ValueError, match="assign illegal value to Enum variable"
-            ):
+            with raises(ValueError, match="assign illegal value to Enum variable"):
                 v[3] = 5
 
         # check, if new API can read them
@@ -2714,14 +2742,14 @@ def test_complex_type_creation_errors(tmp_local_netcdf):
 
     with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
         ds.createDimension("x", size=len(complex_array))
-        with pytest.raises(TypeError, match="data type 'c4' not understood"):
+        with raises(TypeError, match="data type 'c4' not understood"):
             ds.createVariable("data", "c4", ("x",))
 
     if "complex256" not in np.sctypeDict:
         pytest.skip("numpy 'complex256' dtype not available")
     with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
         ds.createDimension("x", size=len(complex_array))
-        with pytest.raises(
+        with raises(
             TypeError,
             match="Currently only 'complex64' and 'complex128' dtypes are allowed.",
         ):
@@ -2773,3 +2801,19 @@ def test_h5pyd_nonchunked_scalars(hsds_up):
         assert ds["foo"]._h5ds.chunks == (1,)
         # However, since it is a scalar dataset, we should not expose the chunking
         assert ds["foo"].chunks is None
+
+
+def test_h5pyd_append(hsds_up):
+    if without_h5pyd:
+        pytest.skip("h5pyd package not available")
+    elif not hsds_up:
+        pytest.skip("HSDS service not running")
+    rnd = "".join(random.choice(string.ascii_uppercase) for _ in range(5))
+    fname = f"hdf5://testfile{rnd}.nc"
+
+    with warns(UserWarning, match="Append mode for h5pyd"):
+        with h5netcdf.File(fname, "a", driver="h5pyd") as ds:
+            assert not ds._preexisting_file
+
+    with h5netcdf.File(fname, "a", driver="h5pyd") as ds:
+        assert ds._preexisting_file

{h5netcdf-1.6.1 → h5netcdf-1.6.3}/h5netcdf.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: h5netcdf
-Version: 1.6.1
+Version: 1.6.3
 Summary: netCDF4 via h5py
 Author-email: Stephan Hoyer <shoyer@gmail.com>, Kai Mühlbauer <kmuehlbauer@wradlib.org>
 Maintainer-email: h5netcdf developers <devteam@h5netcdf.org>
@@ -56,6 +56,7 @@ Requires-Dist: packaging
 Provides-Extra: test
 Requires-Dist: netCDF4; extra == "test"
 Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
 
 h5netcdf
 ========
@@ -318,7 +319,7 @@ The following describes the behavior of h5netcdf with respect to order tracking
 for a few key versions:
 
 - Version 0.12.0 and earlier, the ``track_order`` parameter`order was missing
-  and thus order tracking was implicitely set to ``False``.
+  and thus order tracking was implicitly set to ``False``.
 - Version 0.13.0 enabled order tracking by setting the parameter
   ``track_order`` to ``True`` by default without deprecation.
 - Versions 0.13.1 to 1.0.2 set ``track_order`` to ``False`` due to a bug in a

h5netcdf-1.6.1/h5netcdf/tests/conftest.py

@@ -1,65 +0,0 @@
-import os
-import tempfile
-from pathlib import Path
-from shutil import rmtree
-
-import pytest
-
-try:
-    from h5pyd import Folder
-    from hsds.hsds_app import HsdsApp
-
-    with_reqd_pkgs = True
-except ImportError:
-    with_reqd_pkgs = False
-
-
-@pytest.fixture(scope="session")
-def hsds_up():
-    """Provide HDF Highly Scalabale Data Service (HSDS) for h5pyd testing."""
-    if with_reqd_pkgs:
-        root_dir = Path(tempfile.mkdtemp(prefix="tmp-hsds-root-"))
-        bucket_name = "pytest"
-        os.environ["BUCKET_NAME"] = bucket_name
-        os.mkdir(
-            f"{root_dir}/{bucket_name}"
-        )  # need to create a directory for our bucket
-
-        hs_username = "h5netcdf-pytest"
-        hs_password = "TestEarlyTestEverything"
-
-        kwargs = {}
-        kwargs["username"] = hs_username
-        kwargs["password"] = hs_password
-        kwargs["root_dir"] = str(root_dir)
-        kwargs["logfile"] = f"{root_dir}/hsds.log"
-        kwargs["log_level"] = "DEBUG"
-        kwargs["host"] = "localhost"
-        kwargs["sn_port"] = 5101
-
-        try:
-            hsds = HsdsApp(**kwargs)
-
-            hsds.run()
-            is_up = hsds.ready
-
-            if is_up:
-                os.environ["HS_ENDPOINT"] = hsds.endpoint
-                os.environ["HS_USERNAME"] = hs_username
-                os.environ["HS_PASSWORD"] = hs_password
-                # make folders expected by pytest
-                # pytest/home/h5netcdf-pytest
-                # Folder("/pytest/", mode='w')
-                Folder("/home/", mode="w")
-                Folder("/home/h5netcdf-pytest/", mode="w")
-        except Exception:
-            is_up = False
-
-        yield is_up
-        hsds.check_processes()  # this will capture hsds log output
-        hsds.stop()
-
-        rmtree(root_dir, ignore_errors=True)
-
-    else:
-        yield False