atlas-python 0.16.1__tar.gz → 0.16.2__tar.gz

{atlas_python-0.16.1 → atlas_python-0.16.2}/Cargo.lock

@@ -71,7 +71,7 @@ dependencies = [
 
 [[package]]
 name = "atlas-python"
-version = "0.16.1"
+version = "0.16.2"
 dependencies = [
  "atlas-rust",
  "ndarray",
@@ -86,7 +86,7 @@ dependencies = [
 
 [[package]]
 name = "atlas-rust"
-version = "0.16.1"
+version = "0.16.2"
 dependencies = [
  "array-format",
  "async-trait",

{atlas_python-0.16.1 → atlas_python-0.16.2}/Cargo.toml

@@ -5,7 +5,7 @@ members = ["atlas-python"]
 # Registry name on crates.io (`atlas` is taken). The library is still
 # imported as `atlas` — see `[lib]` below.
 name = "atlas-rust"
-version = "0.16.1"
+version = "0.16.2"
 edition = "2024"
 description = "Single-file immutable store for thousands of N-dimensional datasets, local or on object storage."
 license = "Apache-2.0"

{atlas_python-0.16.1 → atlas_python-0.16.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: atlas-python
-Version: 0.16.1
+Version: 0.16.2
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Science/Research
 Classifier: Programming Language :: Python :: 3
@@ -44,6 +44,9 @@ atlas info   /data/collection
 atlas rm     /data/collection 2024-02.nc 2024-03.nc
 ```
 
+`python -m atlas` runs the same command without a PATH lookup, for a shell that
+cannot find `atlas`.
+
 | Extra | Install | Adds |
 |---|---|---|
 | cloud | `pip install "atlas-python[cloud]"` | S3 / GCS / Azure / HTTP via [obstore](https://github.com/developmentseed/obstore) |
@@ -120,8 +123,9 @@ command gives the same content as a structure.
 
 ## Ingest
 
-`create` scans a directory for `.nc`, `.nc4`, `.cdf`, and `.netcdf`. It sorts
-them, and writes one dataset per file, named after the file. Each coordinate
+`create` scans a directory for `.nc`, `.nc4`, `.cdf`, and `.netcdf`, and
+descends into every subdirectory. It sorts them, and writes one dataset per
+file, named after the file. Each coordinate
 and data variable becomes an array. Each variable attribute becomes a per-array
 attribute. `_FillValue` becomes the fill of the array.
 

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "atlas-python"
-version = "0.16.1"
+version = "0.16.2"
 edition = "2021"
 readme = "README.md"
 

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/README.md

@@ -16,6 +16,9 @@ atlas info   /data/collection
 atlas rm     /data/collection 2024-02.nc 2024-03.nc
 ```
 
+`python -m atlas` runs the same command without a PATH lookup, for a shell that
+cannot find `atlas`.
+
 | Extra | Install | Adds |
 |---|---|---|
 | cloud | `pip install "atlas-python[cloud]"` | S3 / GCS / Azure / HTTP via [obstore](https://github.com/developmentseed/obstore) |
@@ -92,8 +95,9 @@ command gives the same content as a structure.
 
 ## Ingest
 
-`create` scans a directory for `.nc`, `.nc4`, `.cdf`, and `.netcdf`. It sorts
-them, and writes one dataset per file, named after the file. Each coordinate
+`create` scans a directory for `.nc`, `.nc4`, `.cdf`, and `.netcdf`, and
+descends into every subdirectory. It sorts them, and writes one dataset per
+file, named after the file. Each coordinate
 and data variable becomes an array. Each variable attribute becomes a per-array
 attribute. `_FillValue` becomes the fill of the array.
 

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/docs/cli.md

@@ -3,6 +3,10 @@
 `pip install atlas-python` puts `atlas` on your PATH. Five subcommands, one per
 operation.
 
+`python -m atlas` runs the same command without a PATH lookup. Use it in a
+container, in a CI job, or when the shell cannot find `atlas`. See
+[Installation](installation.md#the-atlas-command-is-not-found).
+
 ```text
 atlas create <netcdf-dir> <collection>   build a collection
 atlas rm     <collection> <name>...      remove datasets
@@ -26,7 +30,7 @@ Every subcommand takes `--json` and `--log-file PATH`, plus the remote flags
 ```bash
 $ atlas create /data/nc /data/collection --skip-unsupported --log-file ingest.log
 $ cat ingest.log
-2026-09-01 14:30:41 INFO    atlas.cli: atlas 0.16.1: create /data/nc ...
+2026-09-01 14:30:41 INFO    atlas.cli: atlas 0.16.2: create /data/nc ...
 2026-09-01 14:30:41 INFO    atlas.ops: ingesting 2 file(s) into /data/collection
 2026-09-01 14:30:41 WARNING atlas.ops: /data/nc/buoy.nc: skipped array 'flag' of dtype bool: numpy dtype dtype('bool') is not supported by atlas (supported: ...)
 2026-09-01 14:30:41 INFO    atlas.ops: wrote 1 dataset(s); skipped 0 file(s) and 1 array(s)
@@ -44,16 +48,19 @@ stderr.
 atlas create /data/nc /data/collection
 ```
 
-Each NetCDF file becomes one dataset, named after the file. `2024-01.nc`
-becomes `2024-01.nc`, suffix and all. The files land in sorted order, which
-makes the ordinals of a collection reproducible.
+The scan descends into every subdirectory. Each NetCDF file becomes one
+dataset, named after the file. `2024-01.nc` becomes `2024-01.nc`, suffix and
+all. A name carries no directory, so two files of one name in two
+subdirectories collide. The files land in sorted order, which makes the
+ordinals of a collection reproducible.
 
 Nothing at the destination is readable until every file lands, with the footer.
 A failure part-way leaves no collection, and not a partial one.
 
 | Flag | Effect |
 |---|---|
-| `-r`, `--recursive` | Descend into subdirectories |
+| `--no-recursive` | Scan the top directory alone. The scan descends by default |
+| `-r`, `--recursive` | Accepted for compatibility. The scan already descends |
 | `--codec {zstd,lz4,none}` | Block compression. Default `zstd` |
 | `--chunk-size SIZE` | Block size to aim for. Default `128MiB` |
 | `--open-chunks MODE` | How files are read: `auto`, `native`, `none`, or a JSON dict |
@@ -121,7 +128,7 @@ reads. Each misaligned block then costs a read-modify-write. Use
 
 ```bash
 # One collection from a tree of monthly directories, tolerating bad files
-atlas create /data/nc /data/collection --recursive --skip-errors
+atlas create /data/nc /data/collection --skip-errors
 
 # A big grid, chunked for selective reads, straight to a bucket
 atlas create /data/nc s3://bucket/2024 --chunk-size 64MiB --region eu-west-1

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/docs/guides/creating.md

@@ -15,9 +15,20 @@ atlas create /data/nc /data/collection
 ## What happens
 
 `create` collects every file that matches `.nc`, `.nc4`, `.cdf`, or `.netcdf`.
-It **sorts** them, and writes one dataset per file, named after the file.
-`2024-01.nc` becomes the dataset `2024-01.nc`. The suffix is part of the name,
-so `jan.nc` and `jan.nc4` are two datasets.
+The scan **descends into every subdirectory**. It **sorts** the result, and
+writes one dataset per file, named after the file. `2024-01.nc` becomes the
+dataset `2024-01.nc`. The suffix is part of the name, so `jan.nc` and
+`jan.nc4` are two datasets.
+
+Pass `recursive=False`, or `--no-recursive`, to scan the top directory alone:
+
+```python
+atlas.create("/data/nc", dest, recursive=False)
+```
+
+```bash
+atlas create /data/nc /data/collection --no-recursive
+```
 
 The sort matters. An ordinal comes from the write order, so a sorted ingest
 makes it reproducible. Rebuild the same directory, and every dataset lands at
@@ -26,8 +37,8 @@ the same position.
 Check what the call picks up before you run it:
 
 ```python
-atlas.find_netcdf_files("/data/nc")             # sorted list of paths
-atlas.find_netcdf_files("/data/nc", recursive=True)
+atlas.find_netcdf_files("/data/nc")                  # sorted, and recursive
+atlas.find_netcdf_files("/data/nc", recursive=False)  # the top level alone
 ```
 
 ## All or nothing
@@ -236,7 +247,13 @@ turns it off.
 | Destination URL cannot be resolved | `SourceError` |
 
 Two files with one name surprise people. `a/jan.nc` and `b/jan.nc` both want
-the name `jan.nc`. Rename one, or ingest them into two collections.
+the name `jan.nc`. The scan descends by default, so a tree of monthly
+directories hits this often. A dataset name carries no directory, because a
+name may hold no `/`.
+
+Three ways out. Rename the files. Ingest each subdirectory into its own
+collection. Or pass `on_error="skip"`, which keeps the first file and reports
+the second.
 
 For the dtype rules, see [Supported dtypes](dtypes.md).
 

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/docs/installation.md

@@ -47,6 +47,47 @@ To run the test suite:
 pytest atlas-python/tests/ -v
 ```
 
+## The `atlas` command is not found
+
+`pip install` puts the `atlas` script in the script directory of the
+environment it installed into. The shell finds it only when that directory is
+on `PATH`.
+
+Run the command through Python instead. This form needs no `PATH` entry, so it
+works everywhere the package imports:
+
+```bash
+python -m atlas --version
+python -m atlas ls /data/collection
+```
+
+To fix the `PATH` itself, first find where the script landed:
+
+```bash
+pip show -f atlas-python | grep -i bin
+python -c "import sysconfig; print(sysconfig.get_path('scripts'))"
+```
+
+Three causes cover almost every case:
+
+- **`pip install --user`.** The script goes to `~/.local/bin`. Many
+  distributions leave that off `PATH`. Add it:
+  `export PATH="$HOME/.local/bin:$PATH"`.
+- **A virtual environment nobody activated.** Run `source .venv/bin/activate`,
+  or call `.venv/bin/atlas` directly.
+- **Two Pythons.** `pip` and `python` resolve to different environments. Use
+  `python -m pip install atlas-python` to install into the interpreter you
+  actually run.
+
+Confirm the package itself is sound:
+
+```bash
+python -c "import atlas; print(atlas.__version__, atlas.__file__)"
+```
+
+A version and a path mean the install worked, and only the script lookup
+failed.
+
 ## Optional: cloud storage (S3, GCS, Azure)
 
 Install the `cloud` extra to open or create an atlas store on S3, GCS, Azure

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/tests/test_cli.py

@@ -165,6 +165,27 @@ def test_log_file_records_a_failure(capsys, tmp_path):
     assert "ERROR" in log.read_text()
 
 
+def test_create_descends_by_default_and_no_recursive_opts_out(capsys, tmp_path):
+    from conftest import make_dataset
+
+    src = tmp_path / "nc"
+    (src / "sub").mkdir(parents=True)
+    make_dataset(1).to_netcdf(src / "sub" / "deep.nc")
+    make_dataset(2).to_netcdf(src / "top.nc")
+
+    run(capsys, "create", str(src), str(tmp_path / "all"), "-q")
+    code, out, _ = run(capsys, "ls", str(tmp_path / "all"))
+    assert out.split() == ["deep.nc", "top.nc"]
+
+    run(capsys, "create", str(src), str(tmp_path / "flat"), "-q", "--no-recursive")
+    code, out, _ = run(capsys, "ls", str(tmp_path / "flat"))
+    assert out.split() == ["top.nc"]
+
+    # -r still parses, so an existing script keeps working.
+    code, _, err = run(capsys, "create", str(src), str(tmp_path / "r"), "-q", "-r")
+    assert code == 0, err
+
+
 # ── ls ───────────────────────────────────────────────────────────────
 
 
@@ -307,6 +328,41 @@ def test_info_json(capsys, collection):
     assert i["format_version"] == 1
 
 
+# ── entry points ─────────────────────────────────────────────────────
+
+
+def test_python_dash_m_atlas_runs_the_cli(netcdf_dir, tmp_path):
+    """`python -m atlas` needs no directory on PATH, so it always works."""
+    import subprocess
+    import sys
+
+    dest = tmp_path / "c"
+    done = subprocess.run(
+        [sys.executable, "-m", "atlas", "create", str(netcdf_dir), str(dest), "-q"],
+        capture_output=True,
+        text=True,
+    )
+    assert done.returncode == 0, done.stderr
+
+    done = subprocess.run(
+        [sys.executable, "-m", "atlas", "ls", str(dest)],
+        capture_output=True,
+        text=True,
+    )
+    assert done.returncode == 0, done.stderr
+    assert done.stdout.split() == ["2024-01.nc", "2024-02.nc", "2024-03.nc"]
+
+
+def test_the_console_script_is_declared():
+    """The wheel must carry the `atlas` console script."""
+    from importlib.metadata import entry_points
+
+    scripts = {
+        e.name: e.value for e in entry_points(group="console_scripts")
+    }
+    assert scripts.get("atlas") == "atlas._cli:main"
+
+
 # ── parsing ──────────────────────────────────────────────────────────
 
 

{atlas_python-0.16.1 → atlas_python-0.16.2}/atlas-python/tests/test_ops.py

@@ -48,33 +48,34 @@ def test_create_is_ordered_by_filename(netcdf_dir, tmp_path):
         )
 
 
-def test_create_finds_files_recursively_when_asked(tmp_path):
+def test_create_descends_into_subdirectories_by_default(tmp_path):
     nested = tmp_path / "nc" / "2024" / "q1"
     nested.mkdir(parents=True)
     make_dataset(1).to_netcdf(nested / "jan.nc")
+    make_dataset(2).to_netcdf(tmp_path / "nc" / "top.nc")
 
-    flat = atlas.find_netcdf_files(tmp_path / "nc")
-    assert flat == []
+    found = atlas.find_netcdf_files(tmp_path / "nc")
+    assert [p.name for p in found] == ["jan.nc", "top.nc"]
 
-    found = atlas.find_netcdf_files(tmp_path / "nc", recursive=True)
-    assert [p.name for p in found] == ["jan.nc"]
+    atlas.create(tmp_path / "nc", str(tmp_path / "c"))
+    assert atlas.list_datasets(str(tmp_path / "c")) == ["jan.nc", "top.nc"]
 
-    atlas.create(tmp_path / "nc", str(tmp_path / "c"), recursive=True)
-    assert atlas.list_datasets(str(tmp_path / "c")) == ["jan.nc"]
 
+def test_recursive_false_scans_the_top_directory_alone(tmp_path):
+    nested = tmp_path / "nc" / "2024" / "q1"
+    nested.mkdir(parents=True)
+    make_dataset(1).to_netcdf(nested / "jan.nc")
+    make_dataset(2).to_netcdf(tmp_path / "nc" / "top.nc")
 
-def test_an_empty_directory_is_an_error(tmp_path):
-    (tmp_path / "empty").mkdir()
-    with pytest.raises(atlas.AtlasError, match="no NetCDF files"):
-        atlas.create(tmp_path / "empty", str(tmp_path / "c"))
-
+    found = atlas.find_netcdf_files(tmp_path / "nc", recursive=False)
+    assert [p.name for p in found] == ["top.nc"]
 
-def test_a_missing_directory_is_an_error(tmp_path):
-    with pytest.raises(atlas.AtlasError, match="not a directory"):
-        atlas.create(tmp_path / "nope", str(tmp_path / "c"))
+    atlas.create(tmp_path / "nc", str(tmp_path / "c"), recursive=False)
+    assert atlas.list_datasets(str(tmp_path / "c")) == ["top.nc"]
 
 
-def test_two_files_with_the_same_name_collide(tmp_path):
+def test_a_name_repeated_across_subdirectories_collides(tmp_path):
+    """A dataset name carries no directory, so two `jan.nc` files clash."""
     src = tmp_path / "nc"
     (src / "a").mkdir(parents=True)
     (src / "b").mkdir()
@@ -82,7 +83,24 @@ def test_two_files_with_the_same_name_collide(tmp_path):
     make_dataset(2).to_netcdf(src / "b" / "jan.nc")
 
     with pytest.raises(atlas.AtlasError, match="duplicate dataset name"):
-        atlas.create(src, str(tmp_path / "c"), recursive=True)
+        atlas.create(src, str(tmp_path / "c"))
+
+    # on_error="skip" keeps the first and reports the second.
+    result = atlas.create(src, str(tmp_path / "c2"), on_error="skip")
+    assert result["written"] == ["jan.nc"]
+    assert len(result["skipped"]) == 1
+    assert "duplicate dataset name" in result["skipped"][0]["error"]
+
+
+def test_an_empty_directory_is_an_error(tmp_path):
+    (tmp_path / "empty").mkdir()
+    with pytest.raises(atlas.AtlasError, match="no NetCDF files"):
+        atlas.create(tmp_path / "empty", str(tmp_path / "c"))
+
+
+def test_a_missing_directory_is_an_error(tmp_path):
+    with pytest.raises(atlas.AtlasError, match="not a directory"):
+        atlas.create(tmp_path / "nope", str(tmp_path / "c"))
 
 
 def test_a_bad_file_abandons_the_collection_by_default(tmp_path):

{atlas_python-0.16.1 → atlas_python-0.16.2}/docs/python.md

@@ -46,10 +46,15 @@ for `ATLS`, and you get nothing.
 
 ## Ingest
 
-`create` scans a directory for `.nc`, `.nc4`, `.cdf`, and `.netcdf` files. It
-sorts them, and writes one dataset per file, named after the file. `2024-01.nc`
-becomes `2024-01.nc`, suffix and all. The sort makes the ordinals reproducible.
-Build the same directory twice, and every dataset lands at the same position.
+`create` scans a directory for `.nc`, `.nc4`, `.cdf`, and `.netcdf` files, and
+descends into every subdirectory. It sorts them, and writes one dataset per
+file, named after the file. `2024-01.nc` becomes `2024-01.nc`, suffix and all.
+The sort makes the ordinals reproducible. Build the same directory twice, and
+every dataset lands at the same position.
+
+A dataset name carries no directory, because a name may hold no `/`. Two files
+of one name in two subdirectories therefore collide. `recursive=False` scans
+the top directory alone.
 
 One writer does all of it. Nothing at the destination is readable until the
 last file lands, with the footer. A failure part-way leaves no collection. That

{atlas_python-0.16.1 → atlas_python-0.16.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "atlas-python"
-version = "0.16.1"
+version = "0.16.2"
 description = "Python bindings for the ATLAS array store"
 readme = "README.md"
 requires-python = ">=3.10"

{atlas_python-0.16.1 → atlas_python-0.16.2}/python/atlas/__init__.pyi

@@ -57,7 +57,7 @@ def create(
     directory: Union[str, os.PathLike[str]],
     destination: Source,
     *,
-    recursive: bool = False,
+    recursive: bool = True,
     codec: str = "zstd",
     chunks: Optional[dict[str, Sequence[int]]] = None,
     open_chunks: Union[str, dict[str, int], None] = "auto",
@@ -69,10 +69,14 @@ def create(
 ) -> dict[str, Any]:
     """Builds a collection at ``destination`` from the NetCDF files in ``directory``.
 
+    The scan descends into every subdirectory. Pass ``recursive=False`` for the
+    top directory alone.
+
     Each file becomes one dataset, named after the file. ``jan_2024.nc``
     becomes ``jan_2024.nc``, suffix and all. The suffix tells two files apart,
-    so ``jan.nc`` and ``jan.nc4`` are two datasets. The files land in sorted
-    order, which fixes the ordinals of the collection.
+    so ``jan.nc`` and ``jan.nc4`` are two datasets. A name carries no
+    directory, so two files of one name in two subdirectories collide. The
+    files land in sorted order, which fixes the ordinals of the collection.
 
     Nothing at ``destination`` is readable until every file lands, with the
     footer. A failure part-way leaves no collection, not a partial one.
@@ -85,7 +89,8 @@ def create(
         directory: Where the NetCDF files are.
         destination: Where to write the collection. A local path, or a URL for
             object storage.
-        recursive: Descend into the subdirectories. Off by default.
+        recursive: Descend into the subdirectories. On by default. Set it
+            false for the top directory alone.
         codec: Block compression. ``"zstd"`` is the default. ``"lz4"`` and
             ``"none"`` are the others. Each block records its own codec, so a
             reader needs no argument.
@@ -225,10 +230,12 @@ def info(source: Source, **store_options: Any) -> dict[str, Any]:
     ...
 
 def find_netcdf_files(
-    directory: Union[str, os.PathLike[str]], recursive: bool = False
+    directory: Union[str, os.PathLike[str]], recursive: bool = True
 ) -> list[pathlib.Path]:
     """NetCDF files in ``directory``, sorted. :func:`create` ingests these.
 
+    The walk descends into every subdirectory, as :func:`create` does.
+
     Call it to see what a ``create`` call picks up, before you run that call.
 
     Raises:

atlas_python-0.16.2/python/atlas/__main__.py

@@ -0,0 +1,14 @@
+"""Runs the command as ``python -m atlas``.
+
+The ``atlas`` console script needs its directory on ``PATH``. This entry point
+does not. It works in any environment that can import the package, which makes
+it the reliable form inside a container, a CI job, or a virtual environment
+nobody activated.
+"""
+
+import sys
+
+from ._cli import main
+
+if __name__ == "__main__":
+    sys.exit(main())

{atlas_python-0.16.1 → atlas_python-0.16.2}/python/atlas/_cli.py

@@ -156,7 +156,7 @@ def cmd_create(args: argparse.Namespace) -> int:
     result = _ops.create(
         args.directory,
         args.destination,
-        recursive=args.recursive,
+        recursive=not args.no_recursive,
         codec=args.codec,
         chunks=chunks,
         open_chunks=open_chunks,
@@ -312,7 +312,17 @@ def build_parser() -> argparse.ArgumentParser:
     )
     p.add_argument("directory", help="directory holding the NetCDF files")
     p.add_argument("destination", help="where to write the collection")
-    p.add_argument("-r", "--recursive", action="store_true", help="descend into subdirectories")
+    p.add_argument(
+        "--no-recursive",
+        action="store_true",
+        help="scan the top directory alone. The scan descends by default",
+    )
+    p.add_argument(
+        "-r",
+        "--recursive",
+        action="store_true",
+        help="accepted for compatibility. The scan already descends",
+    )
     p.add_argument(
         "--codec",
         default="zstd",

{atlas_python-0.16.1 → atlas_python-0.16.2}/python/atlas/_ops.py

@@ -56,9 +56,13 @@ def dataset_name(path: "pathlib.Path | str") -> str:
 
 
 def find_netcdf_files(
-    directory: "pathlib.Path | str", recursive: bool = False
+    directory: "pathlib.Path | str", recursive: bool = True
 ) -> list[pathlib.Path]:
-    """NetCDF files in `directory`, sorted. The sort fixes a collection's order."""
+    """NetCDF files in `directory`, sorted. The sort fixes a collection's order.
+
+    The walk descends into every subdirectory. Pass `recursive=False` for the
+    top directory alone.
+    """
     root = pathlib.Path(directory)
     if not root.is_dir():
         raise AtlasError(f"not a directory: {root}")
@@ -96,7 +100,7 @@ def create(
     directory: "pathlib.Path | str",
     destination: Any,
     *,
-    recursive: bool = False,
+    recursive: bool = True,
     codec: str = "zstd",
     chunks: Optional[dict[str, Sequence[int]]] = None,
     open_chunks: Any = "auto",
@@ -108,10 +112,16 @@ def create(
 ) -> dict[str, Any]:
     """Builds a collection at `destination` from the NetCDF files in `directory`.
 
+    The scan descends into every subdirectory. Pass `recursive=False` for the
+    top directory alone.
+
     Each file becomes one dataset, named after the file. `2024-01.nc` becomes
-    `2024-01.nc`, suffix and all. Nothing at `destination` is readable until
-    every file lands, with the footer. A failure therefore leaves no half-built
-    collection.
+    `2024-01.nc`, suffix and all. A name carries no directory, so two files of
+    one name in two subdirectories collide. `on_error="skip"` keeps the first
+    and reports the second.
+
+    Nothing at `destination` is readable until every file lands, with the
+    footer. A failure therefore leaves no half-built collection.
 
     Each file opens with dask chunking, under `open_chunks="auto"` by default.
     A file far larger than memory then streams block by block. Those blocks