pyrill 0.2.0__tar.gz

pyrill-0.2.0/PKG-INFO

@@ -0,0 +1,99 @@
+Metadata-Version: 2.3
+Name: pyrill
+Version: 0.2.0
+Summary: Asynchronous streams for Python
+Keywords: stream,flow,dataflow,reactive
+Author: Alfred
+Author-email: Alfred <alfred82santa@gmail.com>
+License: LGPL-2.1-or-later
+Classifier: Development Status :: 4 - Beta
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Typing :: Typed
+Requires-Python: >=3.14
+Project-URL: Repository, https://github.com/pyrill-framework/pyrill
+Project-URL: Homepage, https://github.com/pyrill-framework/pyrill
+Project-URL: Documentation, https://pyrill-framework.readthedocs.io/stable/
+Project-URL: Issues, https://github.com/pyrill-framework/pyrill/issues
+Description-Content-Type: text/markdown
+
+# PyRill
+
+Asynchronous streams for Python.
+
+PyRill builds asynchronous data pipelines in plain Python. You connect small
+elements (sources, stages and sinks) with `>>`, and the data flows through them one
+frame at a time, on top of `asyncio` and with no dependencies.
+
+```python
+import asyncio
+
+from pyrill.accumulators import ListAcc
+from pyrill.mappers import Map
+from pyrill.sinks import Last
+from pyrill.sources import IterableSource
+
+
+async def main() -> None:
+    pipeline = (
+        IterableSource(source=["alpha", "beta", "gamma"])
+        >> Map(map_func=str.upper)
+        >> ListAcc()
+        >> Last()
+    )
+    print(await pipeline.get_frame())
+
+
+asyncio.run(main())
+```
+
+```text
+['ALPHA', 'BETA', 'GAMMA']
+```
+
+## Why PyRill
+
+Programs that handle streams (files, sockets, API responses, queues) tend to grow
+hand-written loops that mix reading, parsing, filtering, timing and error handling.
+PyRill turns each of those concerns into a small, typed, reusable element that you
+can test alone and connect with the others.
+
+- **Pull-based.** Work happens only when a frame is needed, so a slow consumer slows
+  the whole pipeline, and frames do not pile up in memory unless you add a buffer.
+- **Streaming.** A pipeline works on one frame at a time, so memory use depends on
+  the size of the frames, not on the length of the stream.
+- **Batteries included.** Elements for files and asyncio streams, byte and text
+  re-chunking, safe UTF-8 decoding, CSV, JSON, branching (`Tee`), pairing (`Zip`),
+  merging (`Union`), timeouts and deadlines, pausing (`Gate`), buffering (`Queue`)
+  and more.
+- **Extensible.** Write your own sources, stages and sinks, or package a chain of
+  stages as a single reusable stage.
+- **Typed.** Elements are generic classes, so type checkers follow the frames through
+  a pipeline.
+
+## Installation
+
+PyRill needs Python 3.14 or later and has no other dependencies.
+
+```console
+$ pip install pyrill
+```
+
+To work on PyRill itself, clone the repository, run `uv sync`, and run the tests with
+`uv run pytest`.
+
+## Documentation
+
+Full documentation, including a quickstart, the core concepts, task-oriented guides
+and a catalog of every element, is at
+<https://pyrill-framework.readthedocs.io/stable/>. The same pages are in the
+[`docs/`](docs/source) folder.
+
+The [messages and monitoring guide](docs/source/guides/messages-and-monitoring.rst)
+explains how to observe lifecycle changes, rejected frames and file activity, and
+publish custom progress events through the pipeline's message bus.
+
+## Project
+
+- Repository and issue tracker: <https://github.com/pyrill-framework/pyrill>
+- License: LGPL-2.1-or-later

pyrill-0.2.0/README.md

@@ -0,0 +1,80 @@
+# PyRill
+
+Asynchronous streams for Python.
+
+PyRill builds asynchronous data pipelines in plain Python. You connect small
+elements (sources, stages and sinks) with `>>`, and the data flows through them one
+frame at a time, on top of `asyncio` and with no dependencies.
+
+```python
+import asyncio
+
+from pyrill.accumulators import ListAcc
+from pyrill.mappers import Map
+from pyrill.sinks import Last
+from pyrill.sources import IterableSource
+
+
+async def main() -> None:
+    pipeline = (
+        IterableSource(source=["alpha", "beta", "gamma"])
+        >> Map(map_func=str.upper)
+        >> ListAcc()
+        >> Last()
+    )
+    print(await pipeline.get_frame())
+
+
+asyncio.run(main())
+```
+
+```text
+['ALPHA', 'BETA', 'GAMMA']
+```
+
+## Why PyRill
+
+Programs that handle streams (files, sockets, API responses, queues) tend to grow
+hand-written loops that mix reading, parsing, filtering, timing and error handling.
+PyRill turns each of those concerns into a small, typed, reusable element that you
+can test alone and connect with the others.
+
+- **Pull-based.** Work happens only when a frame is needed, so a slow consumer slows
+  the whole pipeline, and frames do not pile up in memory unless you add a buffer.
+- **Streaming.** A pipeline works on one frame at a time, so memory use depends on
+  the size of the frames, not on the length of the stream.
+- **Batteries included.** Elements for files and asyncio streams, byte and text
+  re-chunking, safe UTF-8 decoding, CSV, JSON, branching (`Tee`), pairing (`Zip`),
+  merging (`Union`), timeouts and deadlines, pausing (`Gate`), buffering (`Queue`)
+  and more.
+- **Extensible.** Write your own sources, stages and sinks, or package a chain of
+  stages as a single reusable stage.
+- **Typed.** Elements are generic classes, so type checkers follow the frames through
+  a pipeline.
+
+## Installation
+
+PyRill needs Python 3.14 or later and has no other dependencies.
+
+```console
+$ pip install pyrill
+```
+
+To work on PyRill itself, clone the repository, run `uv sync`, and run the tests with
+`uv run pytest`.
+
+## Documentation
+
+Full documentation, including a quickstart, the core concepts, task-oriented guides
+and a catalog of every element, is at
+<https://pyrill-framework.readthedocs.io/stable/>. The same pages are in the
+[`docs/`](docs/source) folder.
+
+The [messages and monitoring guide](docs/source/guides/messages-and-monitoring.rst)
+explains how to observe lifecycle changes, rejected frames and file activity, and
+publish custom progress events through the pipeline's message bus.
+
+## Project
+
+- Repository and issue tracker: <https://github.com/pyrill-framework/pyrill>
+- License: LGPL-2.1-or-later

pyrill-0.2.0/pyproject.toml

@@ -0,0 +1,146 @@
+[project]
+name = "pyrill"
+version = "0.2.0"
+description = "Asynchronous streams for Python"
+readme = "README.md"
+requires-python = ">=3.14"
+dependencies = []
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.14",
+    "Typing :: Typed",
+]
+keywords = [
+    "stream",
+    "flow",
+    "dataflow",
+    "reactive",
+]
+
+[[project.authors]]
+name = "Alfred"
+email = "alfred82santa@gmail.com"
+
+[project.license]
+text = "LGPL-2.1-or-later"
+
+[project.urls]
+Repository = "https://github.com/pyrill-framework/pyrill"
+Homepage = "https://github.com/pyrill-framework/pyrill"
+Documentation = "https://pyrill-framework.readthedocs.io/stable/"
+Issues = "https://github.com/pyrill-framework/pyrill/issues"
+
+[build-system]
+requires = ["uv_build>=0.11.3,<0.12.0"]
+build-backend = "uv_build"
+
+[tool.pytest]
+asyncio_mode = "auto"
+
+[tool.coverage.run]
+omit = [".venv/*"]
+branch = true
+relative_files = false
+
+[tool.coverage.report]
+exclude_also = [
+    "def __repr__",
+    'if self\.debug',
+    "raise AssertionError",
+    "raise NotImplementedError",
+    "if 0:",
+    "if __name__ == .__main__.:",
+    '@(abc\.)?abstractmethod',
+    "if TYPE_CHECKING",
+    "@overload",
+]
+
+[tool.coverage.paths]
+source = ["src/"]
+
+[tool.poe]
+envfile = [
+    ".env",
+    ".env.local",
+]
+executor = "uv"
+
+[tool.poe.tasks]
+build = "uv build"
+test = "pytest -v -s --junitxml=pytest.xml --cov src --cov-report term-missing --cov-report xml:coverage.xml --cov-fail-under=85 --exitfirst"
+typecheck = "mypy src"
+format = "ruff format src/**/*.py tests/**/*.py"
+"format:check" = "ruff format src/**/*.py tests/**/*.py --check"
+lint = "ruff check src/**/*.py tests/**/*.py"
+"lint:fix" = "ruff check src/**/*.py tests/**/*.py --fix"
+
+[tool.poe.tasks.clean]
+cmd = """
+rm -rf ./**/src/**/*.pyc
+       ./**/src/**/*.pyo
+       ./**/src/**/__pycache__
+       ./**/tests/**/*.pyc
+       ./**/tests/**/*.pyo
+       ./**/tests/**/__pycache__
+       .coverage
+       coverage.xml
+       pytest.xml
+       .*_cache
+"""
+empty_glob = "null"
+
+[tool.poe.tasks.validate]
+help = "Execute all validations: typecheck, lint, and test"
+sequence = [
+    "typecheck",
+    "format:check",
+    "lint",
+    "test",
+]
+
+[tool.poe.tasks.prebuild]
+help = "Pre-build checks: typecheck and lint"
+sequence = [
+    "typecheck",
+    "lint",
+]
+
+[tool.poe.tasks."clean:all"]
+help = "Complete cleanup: cache, coverage, and temporary files"
+sequence = [
+    "clean",
+    { cmd = """
+rm -rf dist
+                      build
+                      .coverage
+                      coverage.xml
+                      pytest.xml
+                      **/.*_cache
+            """ },
+]
+
+[tool.poe.tasks.fix]
+help = "Fix linting and formatting issues, then run tests"
+sequence = [
+    "format",
+    "lint:fix",
+]
+
+[tool.poe.tasks.docs]
+help = "Build the documentation"
+cmd = "sphinx-build -b html docs/source docs/build"
+
+[tool.poe.tasks."docs:clean"]
+help = "Clean the documentation build"
+cmd = "rm -rf docs/build"
+
+[dependency-groups]
+dev = [
+    "mypy>=2.4.0",
+    "pytest>=9.1.1",
+    "pytest-asyncio>=1.4.0",
+    "pytest-cov>=7.1.0",
+    "ruff>=0.17.3",
+]
+docs = ["sphinx>=9.2.0"]

pyrill-0.2.0/pyproject.toml.orig

@@ -0,0 +1,132 @@
+[project]
+name = "pyrill"
+version = "0.2.0"
+description = "Asynchronous streams for Python"
+readme = "README.md"
+authors = [{ name = "Alfred", email = "alfred82santa@gmail.com" }]
+requires-python = ">=3.14"
+dependencies = []
+license = { text = "LGPL-2.1-or-later" }
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.14",
+    "Typing :: Typed",
+]
+keywords = ["stream", "flow", "dataflow", "reactive"]
+
+[project.urls]
+Repository = "https://github.com/pyrill-framework/pyrill"
+Homepage = "https://github.com/pyrill-framework/pyrill"
+Documentation = "https://pyrill-framework.readthedocs.io/stable/"
+Issues = "https://github.com/pyrill-framework/pyrill/issues"
+
+
+[build-system]
+requires = ["uv_build>=0.11.3,<0.12.0"]
+build-backend = "uv_build"
+
+
+[tool.pytest]
+asyncio_mode = "auto"
+
+[dependency-groups]
+dev = [
+    "mypy>=2.4.0",
+    "pytest>=9.1.1",
+    "pytest-asyncio>=1.4.0",
+    "pytest-cov>=7.1.0",
+    "ruff>=0.17.3",
+]
+docs = ["sphinx>=9.2.0"]
+
+[tool.coverage.run]
+omit = [".venv/*"]
+branch = true
+relative_files = false
+
+[tool.coverage.report]
+# Regexes for lines to exclude from consideration
+exclude_also = [
+    # Dont complain about missing debug-only code:
+    "def __repr__",
+    "if self\\.debug",
+
+    # Don't complain if tests don't hit defensive assertion code:
+    "raise AssertionError",
+    "raise NotImplementedError",
+
+    # Don't complain if non-runnable code isn't run:
+    "if 0:",
+    "if __name__ == .__main__.:",
+
+    # Don't complain about abstract methods, they aren't run:
+    "@(abc\\.)?abstractmethod",
+
+    # Don't complain type checking imports, they aren't run:
+    "if TYPE_CHECKING",
+
+    # Don't complain overloads, they aren't run:
+    "@overload",
+]
+
+[tool.coverage.paths]
+source = ["src/"]
+
+[tool.poe]
+envfile = [".env", ".env.local"]
+executor = "uv"
+
+[tool.poe.tasks]
+build = "uv build"
+test = "pytest -v -s --junitxml=pytest.xml --cov src --cov-report term-missing --cov-report xml:coverage.xml --cov-fail-under=85 --exitfirst"
+typecheck = "mypy src"
+format = "ruff format src/**/*.py tests/**/*.py"
+"format:check" = "ruff format src/**/*.py tests/**/*.py --check"
+lint = "ruff check src/**/*.py tests/**/*.py"
+"lint:fix" = "ruff check src/**/*.py tests/**/*.py --fix"
+
+[tool.poe.tasks.clean]
+cmd = """
+rm -rf ./**/src/**/*.pyc
+       ./**/src/**/*.pyo
+       ./**/src/**/__pycache__
+       ./**/tests/**/*.pyc
+       ./**/tests/**/*.pyo
+       ./**/tests/**/__pycache__
+       .coverage
+       coverage.xml
+       pytest.xml
+       .*_cache
+"""
+empty_glob = "null"
+
+[tool.poe.tasks.validate]
+help = "Execute all validations: typecheck, lint, and test"
+sequence = ["typecheck", "format:check", "lint", "test"]
+
+[tool.poe.tasks.prebuild]
+help = "Pre-build checks: typecheck and lint"
+sequence = ["typecheck", "lint"]
+
+[tool.poe.tasks."clean:all"]
+help = "Complete cleanup: cache, coverage, and temporary files"
+sequence = ["clean", { cmd = """rm -rf dist
+                      build
+                      .coverage
+                      coverage.xml
+                      pytest.xml
+                      **/.*_cache
+            """ }]
+
+[tool.poe.tasks.fix]
+help = "Fix linting and formatting issues, then run tests"
+sequence = ["format", "lint:fix"]
+
+[tool.poe.tasks.docs]
+help = "Build the documentation"
+cmd = "sphinx-build -b html docs/source docs/build"
+
+[tool.poe.tasks."docs:clean"]
+help = "Clean the documentation build"
+cmd = "rm -rf docs/build"

pyrill-0.2.0/src/pyrill/accumulators.py

@@ -0,0 +1,195 @@
+from collections.abc import Sized
+from typing import Any
+
+from .base import BaseStage
+
+__all__ = [
+    "Count",
+    "ListAcc",
+    "Size",
+    "SumAcc",
+]
+
+
+class SumAcc[AccData: (str, bytes, int, complex, float)](BaseStage[AccData, AccData]):
+    """
+    Stage that emits the running sum of the frames received so far.
+
+    After every frame the stage emits the accumulated value, so a source of ``n``
+    frames produces ``n`` outputs. The sum uses ``+``, so it also concatenates ``str``
+    and ``bytes`` frames. The accumulator is reset to ``initial_value`` on every mount
+    and cleared on unmount.
+
+    :param initial_value: Value the sum starts from. When ``None`` (default), the first
+        frame becomes the initial value.
+
+    Input and output::
+
+        input:  1, 2, 3
+        output: 1, 3, 6
+
+    Usage:
+
+    .. code-block:: python
+
+        from pyrill.accumulators import SumAcc
+        from pyrill.sources import IterableSource
+
+        source = IterableSource(source=[1, 2, 3])
+        stage = source >> SumAcc(initial_value=10)
+
+        await stage.mount()
+        frames = [frame async for frame in stage]
+        # [11, 13, 16]
+    """
+
+    def __init__(self, *args: Any, initial_value: AccData | None = None, **kwargs: Any):
+        super().__init__(*args, **kwargs)
+
+        self.initial_value: AccData | None = initial_value
+
+        self._accum: AccData | None = None
+
+    async def _mount(self) -> None:
+        self._accum = self.initial_value
+        await super()._mount()
+
+    async def _unmount(self) -> None:
+        self._accum = None
+        await super()._unmount()
+
+    async def _process_frame(self, frame: AccData) -> AccData:
+        if self._accum is None:
+            self._accum = frame
+        else:
+            self._accum += frame
+        return self._accum
+
+
+class ListAcc[Source_co](BaseStage[Source_co, list[Source_co]]):
+    """
+    Stage that emits the list of all the frames received so far.
+
+    After every frame the stage emits a new list (a copy, so earlier outputs are not
+    modified later) holding every frame seen since the stage was mounted. The list is
+    reset on every mount.
+
+    Input and output::
+
+        input:  "a", "b", "c"
+        output: ["a"], ["a", "b"], ["a", "b", "c"]
+
+    Usage:
+
+    .. code-block:: python
+
+        from pyrill.accumulators import ListAcc
+        from pyrill.sources import IterableSource
+
+        source = IterableSource(source=["a", "b", "c"])
+        stage = source >> ListAcc()
+
+        await stage.mount()
+        frames = [frame async for frame in stage]
+        # [["a"], ["a", "b"], ["a", "b", "c"]]
+    """
+
+    def __init__(self, *args: Any, **kwargs: Any):
+        super().__init__(*args, **kwargs)
+
+        self._accum: list[Source_co] = []
+
+    async def _mount(self) -> None:
+        self._accum = []
+        await super()._mount()
+
+    async def _process_frame(self, frame: Source_co) -> list[Source_co]:
+        self._accum.append(frame)
+
+        return self._accum.copy()
+
+
+class Count(BaseStage[Any, int]):
+    """
+    Stage that emits how many frames it has received so far.
+
+    The frame content is ignored. After every frame the stage emits the number of frames
+    seen since it was mounted, starting at ``1``. The counter is reset on mount and on
+    unmount.
+
+    Input and output::
+
+        input:  "a", "b", "c"
+        output: 1, 2, 3
+
+    Usage:
+
+    .. code-block:: python
+
+        from pyrill.accumulators import Count
+        from pyrill.sources import IterableSource
+
+        source = IterableSource(source=["a", "b", "c"])
+        stage = source >> Count()
+
+        await stage.mount()
+        frames = [frame async for frame in stage]
+        # [1, 2, 3]
+    """
+
+    _counter: int = 0
+
+    async def _mount(self) -> None:
+        self._counter = 0
+        await super()._mount()
+
+    async def _unmount(self) -> None:
+        self._counter = 0
+        await super()._unmount()
+
+    async def _process_frame(self, frame: Any) -> int:
+        self._counter += 1
+        return self._counter
+
+
+class Size(BaseStage[Sized, int]):
+    """
+    Stage that emits the total ``len`` of the frames received so far.
+
+    Frames can be any sized object (``str``, ``bytes``, ``list``...). After every frame
+    the stage emits the sum of the lengths seen since it was mounted. The total is reset
+    on mount and on unmount.
+
+    Input and output::
+
+        input:  "ab", "cde", "f"
+        output: 2, 5, 6
+
+    Usage:
+
+    .. code-block:: python
+
+        from pyrill.accumulators import Size
+        from pyrill.sources import IterableSource
+
+        source = IterableSource(source=["ab", "cde", "f"])
+        stage = source >> Size()
+
+        await stage.mount()
+        frames = [frame async for frame in stage]
+        # [2, 5, 6]
+    """
+
+    _acc: int = 0
+
+    async def _mount(self) -> None:
+        self._acc = 0
+        await super()._mount()
+
+    async def _unmount(self) -> None:
+        self._acc = 0
+        await super()._unmount()
+
+    async def _process_frame(self, frame: Sized) -> int:
+        self._acc += len(frame)
+        return self._acc