kaparoo-python 0.2.0__tar.gz → 0.3.0__tar.gz

{kaparoo_python-0.2.0 → kaparoo_python-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kaparoo-python
-Version: 0.2.0
+Version: 0.3.0
 Summary: Personally common and useful Python features
 Keywords: filesystem,pathlib,paths,utilities
 Author: Jaewoo Park
@@ -46,32 +46,48 @@ pip install kaparoo-python
 
 ## 🧩 Modules
 
-### `kaparoo.filesystem`
+Each submodule ships its own README with focused examples.
 
-`pathlib`-based filesystem helpers.
+### [`kaparoo.filesystem`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem)
 
-- **`existence`** — existence checks (`*_exists`) and `ensure_*` validators.
-- **`directory`** — `make_dir(s)`, `dir_empty(s)` (with `_unsafe` variants).
-- **`utils`** — `stringify_path(s)`, `wrap_path(s)`.
-- **`exceptions`** — `DirectoryNotFoundError`, `NotAFileError`.
-- **`types`** — `StrPath`, `StrPaths`.
+`pathlib`-based filesystem helpers: existence checks (`*_exists`),
+`ensure_*` validators, `make_dir(s)`, `dir_empty(s)`, path
+stringification, and a small exception hierarchy.
 
-### `kaparoo.filesystem.search`
+### [`kaparoo.filesystem.search`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem/search)
 
-Filesystem traversal with composable filters.
+Filesystem traversal with composable filters. Includes `search_paths` /
+`search_files` / `search_dirs`, a `Filter` family (pattern, multi-pattern,
+logical) that round-trips through JSON-friendly dicts, and an extension
+hook for custom filter kinds.
 
-- **Entry points** — `search_paths`, `search_files`, `search_dirs`.
-- **Pattern filters** — `Equals`, `StartsWith`, `EndsWith`, `Contains`,
-  `Regex`, `Glob`.
-- **Multi-pattern filters** — `EqualsAny`, `StartsWithAny`, `EndsWithAny`,
-  `ContainsAny`.
-- **Logical filters** — `And`, `Or`, `Not`.
-- **Deprecated** — `get_paths`, `get_files`, `get_dirs` (use `search_*`).
+### [`kaparoo.utils`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/utils)
 
-### `kaparoo.utils`
+`Timer` / `LapTimer` context-manager-and-decorator timers, plus a small
+family of helpers for working with `Optional[T]` values
+(`replace_if_none`, `unwrap_or_default`, ...).
 
-- **`timer`** — `Timer` and `LapTimer` context-manager / decorator timers.
-- **`optional`** — `replace_if_none`, `factory_if_none`, `unwrap_or_*`.
+### [`kaparoo.data`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/data)
+
+Building blocks for dataset code: `DataSequence[T, M]` ABC (item +
+metadata), composers (`SlicedSequence`, `ConcatSequence`,
+`WindowedSequence`), file-backed templates (`FileFolderSequence`,
+`SingleFileSequence`), and `generate_batches`.
+
+## 🎯 Quick example
+
+```python
+from kaparoo.filesystem import search_files
+from kaparoo.filesystem.search.filters import And, EndsWith, Equals, Not
+
+# All .py files except __init__.py
+py_files = search_files(
+    "src",
+    name_filter=And((EndsWith(".py"), Not(Equals("__init__.py")))),
+)
+```
+
+See each submodule's README for more.
 
 ## 📋 TODO
 

kaparoo_python-0.3.0/README.md

@@ -0,0 +1,81 @@
+# kaparoo-python
+
+[![PyPI version](https://img.shields.io/pypi/v/kaparoo-python.svg)](https://pypi.org/project/kaparoo-python/)
+[![Downloads](https://pepy.tech/badge/kaparoo-python)](https://pypi.org/project/kaparoo-python/)
+[![Python](https://img.shields.io/badge/python-3.14+-blue.svg)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![ty](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ty/main/assets/badge/v0.json)](https://github.com/astral-sh/ty)
+[![Copier](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/copier-org/copier/master/img/badge/badge-grayscale-inverted-border-orange.json)](https://github.com/copier-org/copier)
+
+*Personally common and useful Python features.*
+
+## 📦 Installation
+
+Requires Python 3.14+.
+
+```bash
+# With uv (recommended)
+uv add kaparoo-python
+
+# With pip
+pip install kaparoo-python
+```
+
+## 🧩 Modules
+
+Each submodule ships its own README with focused examples.
+
+### [`kaparoo.filesystem`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem)
+
+`pathlib`-based filesystem helpers: existence checks (`*_exists`),
+`ensure_*` validators, `make_dir(s)`, `dir_empty(s)`, path
+stringification, and a small exception hierarchy.
+
+### [`kaparoo.filesystem.search`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/filesystem/search)
+
+Filesystem traversal with composable filters. Includes `search_paths` /
+`search_files` / `search_dirs`, a `Filter` family (pattern, multi-pattern,
+logical) that round-trips through JSON-friendly dicts, and an extension
+hook for custom filter kinds.
+
+### [`kaparoo.utils`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/utils)
+
+`Timer` / `LapTimer` context-manager-and-decorator timers, plus a small
+family of helpers for working with `Optional[T]` values
+(`replace_if_none`, `unwrap_or_default`, ...).
+
+### [`kaparoo.data`](https://github.com/kaparoo/kaparoo-python/tree/main/kaparoo/data)
+
+Building blocks for dataset code: `DataSequence[T, M]` ABC (item +
+metadata), composers (`SlicedSequence`, `ConcatSequence`,
+`WindowedSequence`), file-backed templates (`FileFolderSequence`,
+`SingleFileSequence`), and `generate_batches`.
+
+## 🎯 Quick example
+
+```python
+from kaparoo.filesystem import search_files
+from kaparoo.filesystem.search.filters import And, EndsWith, Equals, Not
+
+# All .py files except __init__.py
+py_files = search_files(
+    "src",
+    name_filter=And((EndsWith(".py"), Not(Equals("__init__.py")))),
+)
+```
+
+See each submodule's README for more.
+
+## 📋 TODO
+
+See [TODO.md](./TODO.md) for tracked open items.
+
+## 📜 Changelog
+
+See [CHANGELOG.md](./CHANGELOG.md) for the version history.
+
+## ⚖️ License
+
+This project is distributed under the terms of the [MIT](./LICENSE) license.

kaparoo_python-0.3.0/kaparoo/data/README.md

@@ -0,0 +1,216 @@
+# `kaparoo.data`
+
+Building blocks for dataset code: a `Sequence`-based abstract base, a
+small set of composers, and ready-to-subclass file-backed templates.
+
+## Modules
+
+- [`sequences/base`](./sequences/base.py) — `DataSequence[T, M]` abstract base
+- [`sequences/composers`](./sequences/composers.py) — `SlicedSequence`,
+  `ConcatSequence`, `WindowedSequence`
+- [`sequences/templates`](./sequences/templates.py) — `FileFolderSequence`,
+  `SingleFileSequence`
+- [`sequences/utils`](./sequences/utils.py) — `generate_batches`
+
+All public symbols are re-exported from both `kaparoo.data` and
+`kaparoo.data.sequences`.
+
+## DataSequence
+
+`DataSequence[T, M]` is a `Sequence[T]` ABC that adds a parallel
+metadata channel. Subclasses implement two abstract methods:
+
+| Method | Purpose |
+| --- | --- |
+| `get_item(index) -> T` | Decode the i-th item. |
+| `get_meta(index) -> M` | Produce the i-th item's metadata. |
+
+The base derives `get_items` / `get_metas` (bulk) and `get_pair` /
+`get_pairs` (item + metadata together). `__getitem__` returns the item
+only — slicing yields a `SlicedSequence`. The `M` type parameter
+defaults to `None`; set it explicitly when items carry meaningful
+metadata (paths, labels, line numbers, ...).
+
+```python
+from kaparoo.data.sequences import DataSequence
+
+class Labeled(DataSequence[bytes, str]):
+    def __init__(self, items, labels):
+        self._items = items
+        self._labels = labels
+
+    def __len__(self):
+        return len(self._items)
+
+    def get_item(self, index):
+        return self._items[index]
+
+    def get_meta(self, index):
+        return self._labels[index]
+
+ds = Labeled([b"a", b"b"], ["cat", "dog"])
+ds[0]                # b"a"           (item only)
+ds.get_pair(0)       # (b"a", "cat")  (item + metadata)
+list(ds.get_metas([0, 1]))  # ["cat", "dog"]
+```
+
+## Composers
+
+### `SlicedSequence`
+
+A stable-length view over `source` exposing only items at the given
+`indices`. `indices` is materialized as a tuple, so `len()` is O(1) and
+random access is O(1) into the index table. **Duplicates are allowed,
+order is preserved.**
+
+```python
+from kaparoo.data.sequences import SlicedSequence
+
+view = SlicedSequence(dataset, [3, 7, 11])
+view[0]   # == dataset[3]
+view[1]   # == dataset[7]
+```
+
+### `ConcatSequence`
+
+End-to-end concatenation of zero or more sources. Lookup is O(log N) in
+the number of sources via cumulative-length `bisect_right`.
+
+```python
+from kaparoo.data.sequences import ConcatSequence
+
+combined = ConcatSequence(train_a, train_b, train_c)
+len(combined)  # == len(train_a) + len(train_b) + len(train_c)
+```
+
+### `WindowedSequence`
+
+An abstract sliding-window view: each item is a `tuple[T, ...]` of
+`size` frames from `source`. Per-frame `M_in` and window-level
+`M_out` are independent type parameters, so subclasses decide how
+metadata aggregates.
+
+```python
+from pathlib import Path
+from kaparoo.data.sequences import WindowedSequence
+
+class FirstFrameMeta(WindowedSequence[bytes, Path, Path]):
+    def get_meta(self, index):
+        # window's metadata is its first frame's metadata
+        index = self._normalize_index(index)
+        return self._source.get_meta(index * self._step)
+
+# 3-frame windows, hop 1, no intra-window skip
+windows = FirstFrameMeta(frames, size=3)
+windows[0]            # (frames[0], frames[1], frames[2])
+windows.get_meta(0)   # frames.get_meta(0)
+```
+
+`size`, `step`, `skip` follow the same semantics as
+[`generate_batches`](#generate_batches).
+
+## Templates
+
+### `FileFolderSequence`
+
+Folder-rooted base for "one file per item" datasets. Subclasses
+implement three methods:
+
+- `list_files(self, root)` — return the full `Path` of every file to
+  expose, in order. Called once from the base's `__init__`. Every
+  returned path must be under `root`.
+- `load_file(self, path)` — decode a single file. Called lazily on each
+  `get_item`.
+- `get_meta(self, index)` — per-item metadata. When metadata is the
+  source path, `M` defaults to `Path` and `get_meta` can be
+  `return self.get_file(index)`.
+
+The base exposes `root: Path`, `files: tuple[Path, ...]` (fresh snapshot),
+and `get_file(index) -> Path`. Paths are stored root-relative
+internally, so the sequence stays compact and survives a relocated root.
+
+**Parameterized subclasses**: when `list_files` needs instance options
+(patterns, recursive flags, ...), set them on `self` **before** calling
+`super().__init__(root)` — the base invokes `list_files` from its own
+`__init__`.
+
+```python
+from pathlib import Path
+from kaparoo.data.sequences import FileFolderSequence
+
+class GlobFolder(FileFolderSequence[bytes]):
+    def __init__(self, root, *, pattern="*", recursive=False):
+        # Set state BEFORE super().__init__() so list_files can read it.
+        self._pattern = pattern
+        self._recursive = recursive
+        super().__init__(root)
+
+    def list_files(self, root):
+        glob_fn = root.rglob if self._recursive else root.glob
+        return sorted(p for p in glob_fn(self._pattern) if p.is_file())
+
+    def get_meta(self, index):
+        return self.get_file(index)
+
+    def load_file(self, path):
+        return path.read_bytes()
+
+folder = GlobFolder("data", pattern="*.png", recursive=True)
+```
+
+### `SingleFileSequence`
+
+Thin ABC for the "one file, many records" pattern (a video with many
+frames, a CSV with many rows, ...). The base validates that `path`
+exists and is a regular file and exposes it via the `path` property.
+Indexing strategies vary too widely across formats to abstract here —
+subclasses own opening, indexing, and decoding.
+
+```python
+from kaparoo.data.sequences import SingleFileSequence
+
+class LinesFile(SingleFileSequence[str, int]):
+    def __init__(self, path):
+        super().__init__(path)
+        self._lines = tuple(self.path.read_text().splitlines())
+
+    def __len__(self):
+        return len(self._lines)
+
+    def get_item(self, index):
+        return self._lines[index]
+
+    def get_meta(self, index):
+        return index + 1  # 1-based line number
+```
+
+## generate_batches
+
+A windowing iterator over any `Sequence`. `size` is the only positional
+parameter; `step` / `skip` / `start` / `stop` / `drop_last` are
+keyword-only.
+
+| Parameter | Effect |
+| --- | --- |
+| `size` | items per window |
+| `step` *(default 1)* | distance between consecutive windows |
+| `skip` *(default 1)* | intra-window stride |
+| `start`, `stop` | restrict the source range; `start == stop` yields nothing |
+| `drop_last` *(default `True`)* | drop a trailing partial window if any |
+
+```python
+from kaparoo.data.sequences import generate_batches
+
+# Overlapping 3-windows (default step=1)
+list(generate_batches(range(6), size=3))
+# [[0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5]]
+
+# Non-overlapping batches
+list(generate_batches(range(7), size=3, step=3, drop_last=False))
+# [[0, 1, 2], [3, 4, 5], [6]]
+```
+
+## See also
+
+- [`kaparoo.filesystem`](../filesystem/) for path helpers and search
+- [`kaparoo.utils`](../utils/) for `Timer` and Optional helpers

kaparoo_python-0.3.0/kaparoo/data/__init__.py

@@ -0,0 +1,19 @@
+__all__ = (
+    "ConcatSequence",
+    "DataSequence",
+    "FileFolderSequence",
+    "SingleFileSequence",
+    "SlicedSequence",
+    "WindowedSequence",
+    "generate_batches",
+)
+
+from kaparoo.data.sequences import (
+    ConcatSequence,
+    DataSequence,
+    FileFolderSequence,
+    SingleFileSequence,
+    SlicedSequence,
+    WindowedSequence,
+    generate_batches,
+)

kaparoo_python-0.3.0/kaparoo/data/sequences/__init__.py

@@ -0,0 +1,23 @@
+from __future__ import annotations
+
+__all__ = (
+    "ConcatSequence",
+    "DataSequence",
+    "FileFolderSequence",
+    "SingleFileSequence",
+    "SlicedSequence",
+    "WindowedSequence",
+    "generate_batches",
+)
+
+from kaparoo.data.sequences.base import DataSequence
+from kaparoo.data.sequences.composers import (
+    ConcatSequence,
+    SlicedSequence,
+    WindowedSequence,
+)
+from kaparoo.data.sequences.templates import (
+    FileFolderSequence,
+    SingleFileSequence,
+)
+from kaparoo.data.sequences.utils import generate_batches

kaparoo_python-0.3.0/kaparoo/data/sequences/base.py

@@ -0,0 +1,73 @@
+from __future__ import annotations
+
+__all__ = ("DataSequence",)
+
+from abc import abstractmethod
+from collections.abc import Sequence
+from typing import overload
+
+
+class DataSequence[T, M = None](Sequence[T]):
+    """An ordered, lazily-loaded, read-only sequence with per-item metadata.
+
+    Subclasses implement `get_item` (and `__len__`) to fetch a single
+    item by index. Sequence operations (`ds[i]`, `ds[i:j]`, `for x in
+    ds`, `x in ds`, `reversed(ds)`, ...) come from the inherited
+    `collections.abc.Sequence` protocol; only `get_item` need be
+    overridden, and `get_items` may be overridden for batch-fetch
+    optimization.
+
+    The second type parameter `M` carries per-item metadata (labels,
+    source paths, timestamps, ...). Subclasses implement `get_meta`.
+    When the data has no metadata, parameterize as `DataSequence[T]`
+    (so `M` defaults to `None`) and let `get_meta` simply return
+    `None`.
+
+    Type Parameters:
+        T: Element type. `ds[i]` and `get_item(i)` return `T`.
+        M: Per-item metadata type. `get_meta(i)` returns `M`. Defaults
+            to `None` -- meaning "no metadata", in which case
+            subclasses still implement `get_meta` but as a no-op.
+    """
+
+    @abstractmethod
+    def __len__(self) -> int:
+        raise NotImplementedError
+
+    # --- item access -------------------------------------------------------
+
+    @overload
+    def __getitem__(self, index: int, /) -> T: ...
+
+    @overload
+    def __getitem__(self, index: slice, /) -> Sequence[T]: ...
+
+    def __getitem__(self, index: int | slice, /) -> T | Sequence[T]:
+        if isinstance(index, slice):
+            start, stop, step = index.indices(len(self))
+            return self.get_items(range(start, stop, step))
+        return self.get_item(index)
+
+    @abstractmethod
+    def get_item(self, index: int) -> T:
+        raise NotImplementedError
+
+    def get_items(self, indices: Sequence[int]) -> Sequence[T]:
+        return [self.get_item(index) for index in indices]
+
+    # --- metadata access ---------------------------------------------------
+
+    @abstractmethod
+    def get_meta(self, index: int) -> M:
+        raise NotImplementedError
+
+    def get_metas(self, indices: Sequence[int]) -> Sequence[M]:
+        return [self.get_meta(index) for index in indices]
+
+    # --- combined item + metadata ------------------------------------------
+
+    def get_pair(self, index: int) -> tuple[T, M]:
+        return self.get_item(index), self.get_meta(index)
+
+    def get_pairs(self, indices: Sequence[int]) -> Sequence[tuple[T, M]]:
+        return [self.get_pair(index) for index in indices]