kaparoo-python 0.7.0__tar.gz → 0.8.0__tar.gz

kaparoo_python-0.8.0/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: kaparoo-python
+Version: 0.8.0
+Summary: Personally common and useful Python features
+Keywords: filesystem,pathlib,paths,glob,filters,pattern-matching,dataset,sequence,batching,timer,aggregation,metrics,utilities,typed
+Author: Jaewoo Park
+Author-email: Jaewoo Park <kaparoo2001@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Filesystems
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Utilities
+Classifier: Typing :: Typed
+Requires-Python: >=3.14
+Project-URL: GitHub, https://www.github.com/kaparoo/kaparoo-python
+Project-URL: Issues, https://www.github.com/kaparoo/kaparoo-python/issues
+Description-Content-Type: text/markdown
+
+# kaparoo-python
+
+[![PyPI version](https://img.shields.io/pypi/v/kaparoo-python.svg)](https://pypi.org/project/kaparoo-python/)
+[![Downloads](https://pepy.tech/badge/kaparoo-python)](https://pypi.org/project/kaparoo-python/)
+[![Python](https://img.shields.io/badge/python-3.14+-blue.svg)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![ty](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ty/main/assets/badge/v0.json)](https://github.com/astral-sh/ty)
+[![Copier](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/copier-org/copier/master/img/badge/badge-grayscale-inverted-border-orange.json)](https://github.com/copier-org/copier)
+
+*Personally common and useful Python features.*
+
+## 📦 Installation
+
+Requires Python 3.14+.
+
+```bash
+# With uv (recommended)
+uv add kaparoo-python
+
+# With pip
+pip install kaparoo-python
+```
+
+## 🧩 Modules
+
+Each submodule ships its own README with focused examples.
+
+### [`kaparoo.filesystem`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem)
+
+`pathlib`-based filesystem helpers: existence checks (`*_exists`),
+`ensure_*` validators, `make_dir(s)` (with a destructive `clean` reset
+option), `dir_empty(s)`, `reserve_path(s)` guards for not-yet-existing
+destinations, `StagedFile` / `StagedDirectory` for safe (atomic) writes,
+path stringification, and a small exception hierarchy.
+
+### [`kaparoo.filesystem.search`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem/search)
+
+Filesystem traversal with composable filters: `search_paths` /
+`search_files` / `search_dirs`, wired to the `kaparoo.filters` DSL via
+`part_filter` / `name_filter` / `predicate`, with `min_depth` / `max_depth`
+control and a subtree-pruning `exclude`.
+
+### [`kaparoo.filesystem.hierarchy`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem/hierarchy)
+
+A declarative description of a filesystem tree: `File` / `Directory`
+nodes whose names are drawn from the `kaparoo.filters` DSL (so one node
+can stand for many regularly-named siblings), plus `Exclusive` / `Together`
+constraints and per-node attribute `condition`s. It drives four disk
+operations — `locate` (map on-disk paths to spec nodes), `validate` (check
+a directory against the spec), `conformer` (build a `search` predicate from
+a spec), and `scaffold` (create the tree on disk).
+
+### [`kaparoo.filters`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filters)
+
+A declarative, composable string-matching DSL: a `Filter` family
+(pattern, multi-pattern, logical, and enumerable `Literal` / `OneOf` /
+`Template`) that round-trips through JSON-friendly dicts, plus an
+extension hook for custom filter kinds. Used by
+`kaparoo.filesystem.search` for path matching and
+`kaparoo.filesystem.hierarchy` for declaring trees.
+
+### [`kaparoo.utils`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/utils)
+
+`Timer` / `SpanTimer` context-manager-and-decorator timers (with
+`lap`-split and `measure`-block timings); `Aggregator` for nested,
+pluggable metric aggregation (the batch → epoch → run pattern);
+`ensure_one_of` / `ensure_in_range` validation guards; plus helpers
+for `Optional[T]` values (`replace_if_none`, `unwrap_or_default`, ...).
+
+### [`kaparoo.data`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/data)
+
+Building blocks for dataset code: `DataSequence[T, M]` ABC (item +
+metadata), composers (`SlicedSequence`, `ConcatSequence`,
+`TransformedSequence`, `WindowedSequence`, `ZippedSequence`), file-backed
+templates (`FileFolderSequence`, `FileListSequence`, `SingleFileSequence`),
+and `generate_batches`.
+
+## 🎯 Quick example
+
+Search a tree with composable filters:
+
+```python
+from kaparoo.filesystem import search_files
+from kaparoo.filters import And, EndsWith, Equals, Not
+
+# All .py files except __init__.py
+py_files = search_files(
+    "src",
+    name_filter=And((EndsWith(".py"), Not(Equals("__init__.py")))),
+)
+```
+
+…or describe a tree declaratively and check a directory against it:
+
+```python
+from kaparoo.filesystem.hierarchy import Directory, File, validate
+from kaparoo.filters import Glob
+
+spec = Directory("dataset", [
+    File("metadata.json"),
+    Directory("images", [File(Glob("*.png"))]),
+])
+report = validate(spec, "data/dataset", root_as_top=True)
+assert report.ok  # required entries present, nothing unexpected
+```
+
+See each submodule's README for more.
+
+## 📋 TODO
+
+See [TODO.md](./TODO.md) for tracked open items.
+
+## 📜 Changelog
+
+See [CHANGELOG.md](./CHANGELOG.md) for the version history.
+
+## ⚖️ License
+
+This project is distributed under the terms of the [MIT](./LICENSE) license.

kaparoo_python-0.7.0/PKG-INFO → kaparoo_python-0.8.0/README.md

@@ -1,24 +1,3 @@
-Metadata-Version: 2.4
-Name: kaparoo-python
-Version: 0.7.0
-Summary: Personally common and useful Python features
-Keywords: filesystem,pathlib,paths,utilities
-Author: Jaewoo Park
-Author-email: Jaewoo Park <kaparoo2001@gmail.com>
-License-Expression: MIT
-License-File: LICENSE
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.14
-Classifier: Programming Language :: Python :: Implementation :: CPython
-Classifier: Typing :: Typed
-Requires-Python: >=3.14
-Project-URL: GitHub, https://www.github.com/kaparoo/kaparoo-python
-Project-URL: Issues, https://www.github.com/kaparoo/kaparoo-python/issues
-Description-Content-Type: text/markdown
-
 # kaparoo-python
 
 [![PyPI version](https://img.shields.io/pypi/v/kaparoo-python.svg)](https://pypi.org/project/kaparoo-python/)
@@ -58,18 +37,37 @@ path stringification, and a small exception hierarchy.
 
 ### [`kaparoo.filesystem.search`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem/search)
 
-Filesystem traversal with composable filters. Includes `search_paths` /
-`search_files` / `search_dirs`, a `Filter` family (pattern, multi-pattern,
-logical) that round-trips through JSON-friendly dicts, and an extension
-hook for custom filter kinds.
+Filesystem traversal with composable filters: `search_paths` /
+`search_files` / `search_dirs`, wired to the `kaparoo.filters` DSL via
+`part_filter` / `name_filter` / `predicate`, with `min_depth` / `max_depth`
+control and a subtree-pruning `exclude`.
+
+### [`kaparoo.filesystem.hierarchy`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem/hierarchy)
+
+A declarative description of a filesystem tree: `File` / `Directory`
+nodes whose names are drawn from the `kaparoo.filters` DSL (so one node
+can stand for many regularly-named siblings), plus `Exclusive` / `Together`
+constraints and per-node attribute `condition`s. It drives four disk
+operations — `locate` (map on-disk paths to spec nodes), `validate` (check
+a directory against the spec), `conformer` (build a `search` predicate from
+a spec), and `scaffold` (create the tree on disk).
+
+### [`kaparoo.filters`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filters)
+
+A declarative, composable string-matching DSL: a `Filter` family
+(pattern, multi-pattern, logical, and enumerable `Literal` / `OneOf` /
+`Template`) that round-trips through JSON-friendly dicts, plus an
+extension hook for custom filter kinds. Used by
+`kaparoo.filesystem.search` for path matching and
+`kaparoo.filesystem.hierarchy` for declaring trees.
 
 ### [`kaparoo.utils`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/utils)
 
 `Timer` / `SpanTimer` context-manager-and-decorator timers (with
 `lap`-split and `measure`-block timings); `Aggregator` for nested,
-pluggable metric aggregation (the batch → epoch → run pattern;
-experimental); plus a small family of helpers for working with
-`Optional[T]` values (`replace_if_none`, `unwrap_or_default`, ...).
+pluggable metric aggregation (the batch → epoch → run pattern);
+`ensure_one_of` / `ensure_in_range` validation guards; plus helpers
+for `Optional[T]` values (`replace_if_none`, `unwrap_or_default`, ...).
 
 ### [`kaparoo.data`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/data)
 
@@ -81,9 +79,11 @@ and `generate_batches`.
 
 ## 🎯 Quick example
 
+Search a tree with composable filters:
+
 ```python
 from kaparoo.filesystem import search_files
-from kaparoo.filesystem.search.filters import And, EndsWith, Equals, Not
+from kaparoo.filters import And, EndsWith, Equals, Not
 
 # All .py files except __init__.py
 py_files = search_files(
@@ -92,6 +92,20 @@ py_files = search_files(
 )
 ```
 
+…or describe a tree declaratively and check a directory against it:
+
+```python
+from kaparoo.filesystem.hierarchy import Directory, File, validate
+from kaparoo.filters import Glob
+
+spec = Directory("dataset", [
+    File("metadata.json"),
+    Directory("images", [File(Glob("*.png"))]),
+])
+report = validate(spec, "data/dataset", root_as_top=True)
+assert report.ok  # required entries present, nothing unexpected
+```
+
 See each submodule's README for more.
 
 ## 📋 TODO

{kaparoo_python-0.7.0 → kaparoo_python-0.8.0}/kaparoo/data/README.md

@@ -3,6 +3,23 @@
 Building blocks for dataset code: a `Sequence`-based abstract base, a
 small set of composers, and ready-to-subclass file-backed templates.
 
+## Contents
+
+- [Modules](#modules)
+- [DataSequence](#datasequence)
+- [Composers](#composers)
+  - [`SlicedSequence`](#slicedsequence)
+  - [`ConcatSequence`](#concatsequence)
+  - [`TransformedSequence`](#transformedsequence)
+  - [`WindowedSequence`](#windowedsequence)
+  - [`ZippedSequence`](#zippedsequence)
+- [Templates](#templates)
+  - [`FileFolderSequence`](#filefoldersequence)
+  - [`FileListSequence`](#filelistsequence)
+  - [`SingleFileSequence`](#singlefilesequence)
+- [generate_batches](#generate_batches)
+- [See also](#see-also)
+
 ## Modules
 
 - [`sequences/base`](./sequences/base.py) — `DataSequence[T, M]` abstract base
@@ -27,10 +44,11 @@ metadata channel. Subclasses implement two abstract methods:
 | `get_meta(index) -> M` | Produce the i-th item's metadata. |
 
 The base derives `get_items` / `get_metas` (bulk) and `get_pair` /
-`get_pairs` (item + metadata together). `__getitem__` returns the item
-only — slicing yields a `SlicedSequence`. The `M` type parameter
-defaults to `None`; set it explicitly when items carry meaningful
-metadata (paths, labels, line numbers, ...).
+`get_pairs` (item + metadata together). `__getitem__` returns one item by
+index, or a plain list of items for a slice (`get_items` over the slice's
+range) — not a `SlicedSequence`. The `M` type parameter defaults to `None`;
+set it explicitly when items carry meaningful metadata (paths, labels, line
+numbers, ...).
 
 ```python
 from kaparoo.data.sequences import DataSequence
@@ -90,7 +108,11 @@ A lazy view that applies a `transform` callable to each item of
 `source`. The transform is called on demand in `get_item` -- nothing
 is computed at construction. `get_meta` passes through `source.get_meta`
 unchanged by default; override it in a subclass when `M_out` differs
-from `M_in`.
+from `M_in`. **That override is required, not optional**: if you declare a
+different `M_out` but forget it, the default silently returns the source's
+`M_in` metadata typed as `M_out` (a `cast` hides the mismatch, and Python
+erases generics at runtime, so nothing catches it until the wrong value is
+used).
 
 ```python
 from kaparoo.data.sequences import TransformedSequence
@@ -130,7 +152,7 @@ class FirstFrameMeta(WindowedSequence[bytes, Path]):
     def get_meta(self, index):
         # window's metadata is its first frame's metadata
         index = self._normalize_index(index)
-        return self._source.get_meta(index * self._step)
+        return self.source.get_meta(index * self.step)
 
 # 3-frame windows, hop 1, no intra-window skip
 windows = FirstFrameMeta(frames, size=3)
@@ -178,8 +200,9 @@ implement three methods:
   source path, `M` defaults to `Path` and `get_meta` can be
   `return self.get_file(index)`.
 
-The base exposes `root: Path`, `files: tuple[Path, ...]` (fresh snapshot),
-and `get_file(index) -> Path`. Paths are stored root-relative
+The base exposes `root: Path`, `files: tuple[Path, ...]` (an immutable
+snapshot, built once and cached), and `get_file(index) -> Path`. Paths are
+stored root-relative
 internally, so the sequence stays compact and survives a relocated root.
 
 **Parameterized subclasses**: when `list_files` needs instance options
@@ -289,5 +312,7 @@ list(generate_batches(range(7), size=3, step=3, drop_last=False))
 
 ## See also
 
-- [`kaparoo.filesystem`](../filesystem/) for path helpers and search
-- [`kaparoo.utils`](../utils/) for `Timer` and Optional helpers
+- [`kaparoo.filesystem`](../filesystem/) — path helpers for file-backed
+  sequences
+- [`kaparoo.filesystem.search`](../filesystem/search/) — discover the files a
+  `FileFolderSequence` wraps

{kaparoo_python-0.7.0 → kaparoo_python-0.8.0}/kaparoo/data/sequences/base.py

@@ -1,3 +1,5 @@
+"""The `DataSequence[T, M]` abstract base: indexable items with metadata."""
+
 from __future__ import annotations
 
 __all__ = ("DataSequence",)
@@ -32,7 +34,7 @@ class DataSequence[T, M = None](Sequence[T]):
 
     @abstractmethod
     def __len__(self) -> int:
-        raise NotImplementedError
+        """Return the number of items in the sequence."""
 
     # --- item access -------------------------------------------------------
 
@@ -50,24 +52,36 @@ class DataSequence[T, M = None](Sequence[T]):
 
     @abstractmethod
     def get_item(self, index: int) -> T:
-        raise NotImplementedError
+        """Fetch and return the item at `index`."""
 
     def get_items(self, indices: Sequence[int]) -> Sequence[T]:
+        """Fetch many items at once, in `indices` order.
+
+        Defaults to one `get_item` per index; override to use a backing
+        store's native batch read.
+        """
         return [self.get_item(index) for index in indices]
 
     # --- metadata access ---------------------------------------------------
 
     @abstractmethod
     def get_meta(self, index: int) -> M:
-        raise NotImplementedError
+        """Return the metadata for the item at `index` (`None` when `M` is `None`)."""
 
     def get_metas(self, indices: Sequence[int]) -> Sequence[M]:
+        """Fetch many metadata values at once, in `indices` order.
+
+        Defaults to one `get_meta` per index; override alongside
+        `get_items` when a batch read is cheaper.
+        """
         return [self.get_meta(index) for index in indices]
 
     # --- combined item + metadata ------------------------------------------
 
     def get_pair(self, index: int) -> tuple[T, M]:
+        """Return the `(item, metadata)` pair at `index`."""
         return self.get_item(index), self.get_meta(index)
 
     def get_pairs(self, indices: Sequence[int]) -> Sequence[tuple[T, M]]:
+        """Fetch many `(item, metadata)` pairs at once, in `indices` order."""
         return [self.get_pair(index) for index in indices]