async-graph-data-flow 1.5.1__tar.gz → 2.0.0__tar.gz

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/CHANGELOG.md

@@ -8,16 +8,43 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 ## [Unreleased] - YYYY-MM-DD
 
 ### Added
-
 ### Changed
-
 ### Deprecated
+### Removed
+### Fixed
+### Security
+
+## [2.0.0] - 2026-04-21
+
+### Added
+- Python 3.14 is officially supported and tested on CI. (#16)
+- `AsyncGraph.add_node` now accepts `functools.partial` objects and functions
+  decorated with `functools.wraps` as node functions. The default node name and
+  the async-generator function check resolve through the embedded function, so
+  neither `name=` nor `check_async_gen=False` is required in these cases. (#16)
+- Added GitHub Actions for PyPI releases. (#16)
 
 ### Removed
+- Removed the `queue_size` argument at `add_node`, deprecated in v1.6.0.
+  To configure queue size, pass a pre-sized `asyncio.Queue` via the `queue`
+  argument instead. (#16)
 
 ### Fixed
+- Fixed a bug where tail items could be dropped when nodes were not added in
+  topological order. Queues are now drained in topological order. (#16)
+
+## [1.6.0] - 2025-03-19
+
+### Added
+- Added support for custom edge behaviors, through the new `queue` argument at `add_node`
+  for a custom queue object. (#14)
+
+### Deprecated
+- Deprecated the `queue_size` argument at `add_node` -- use the new `queue` instead. (#14)
 
 ### Security
+- Resolved CVE-2024-56326, CVE-2024-56201, and CVE-2025-27516
+  by updating jinja2 to v3.1.6 in requirements for Sphinx docs. (#14)
 
 ## [1.5.1] - 2024-11-07
 

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2022-2024, Civis Analytics
+Copyright (c) 2022-2026, Civis Analytics
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

{async_graph_data_flow-1.5.1/src/async_graph_data_flow.egg-info → async_graph_data_flow-2.0.0}/PKG-INFO

@@ -1,9 +1,9 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: async-graph-data-flow
-Version: 1.5.1
-Summary: Asynchronous functions that pass data along a directed acyclic graph
+Version: 2.0.0
+Summary: Data flow along a directed acyclic graph of asynchronous generator functions
 Author-email: Civis Analytics <opensource@civisanalytics.com>
-License: BSD 3-Clause
+License-Expression: BSD-3-Clause
 Project-URL: Documentation, https://async-graph-data-flow.readthedocs.io
 Project-URL: Source, https://github.com/civisanalytics/async-graph-data-flow
 Keywords: async,asynchronous,asyncio,data flow,graph
@@ -11,7 +11,6 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Information Technology
 Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: BSD License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
@@ -19,23 +18,25 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Framework :: AsyncIO
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Provides-Extra: dev
-Requires-Dist: black==24.10.0; extra == "dev"
-Requires-Dist: flake8==7.1.1; extra == "dev"
-Requires-Dist: pytest==8.3.3; extra == "dev"
-Requires-Dist: mypy==1.13.0; extra == "dev"
-Requires-Dist: build==1.2.2; extra == "dev"
-Requires-Dist: twine==5.1.1; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: flake8; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: mypy; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
 Provides-Extra: docs
-Requires-Dist: Sphinx==8.1.3; extra == "docs"
-Requires-Dist: furo==2024.8.6; extra == "docs"
-Requires-Dist: numpydoc==1.8.0; extra == "docs"
-Requires-Dist: sphinx-copybutton==0.5.2; extra == "docs"
-Requires-Dist: sphinxcontrib-mermaid==1.0.0; extra == "docs"
+Requires-Dist: Sphinx; extra == "docs"
+Requires-Dist: furo; extra == "docs"
+Requires-Dist: numpydoc; extra == "docs"
+Requires-Dist: sphinx-copybutton; extra == "docs"
+Requires-Dist: sphinxcontrib-mermaid; extra == "docs"
+Dynamic: license-file
 
 # async-graph-data-flow
 
@@ -59,25 +60,26 @@ BSD 3-Clause License. Please see `LICENSE.txt` in the GitHub source code for det
 
 The latest code under development is available on GitHub at
 https://github.com/civisanalytics/async-graph-data-flow.
-To obtain this version for experimental features or for development:
+To obtain this version for experimental features or for development,
+we recommend using [uv](https://docs.astral.sh/uv/) to manage the development environment:
 
 ```bash
 git clone https://github.com/civisanalytics/async-graph-data-flow.git
 cd async-graph-data-flow
-pip install -e ".[dev]"
+uv pip install -e ".[dev]"
 ```
 
 To run tests and styling checks:
 
 ```bash
-pytest
-flake8 src tests examples
-black --check src tests examples
+uv run pytest
+uv run flake8 src tests examples
+uv run black --check src tests examples
 ```
 
 ## Building Documentation
 
-We use the Sphinx framework. The documentation source files are in `docs/source/`.
+We use the Sphinx framework. The documentation source files are in `docs/`.
 These files can be updated as necessary.
 
 The public documentation is accessible at https://async-graph-data-flow.readthedocs.io.
@@ -88,5 +90,5 @@ The builds by the Read The Docs site generate the necessary files (the HTML page
 for the public documentation. All these auto-generated files are explicitly not versioned (see `.gitignore`).
 
 To build the documentation locally (for testing and development),
-install the full doc-related dependencies: `pip install -r docs/requirements.txt`,
-then run `sphinx-build -b html docs/ docs/build/`.
+install the full doc-related dependencies: `uv pip install -r docs/requirements.txt`,
+then run `uv run sphinx-build -b html docs/ docs/build/`.

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/README.md

@@ -20,25 +20,26 @@ BSD 3-Clause License. Please see `LICENSE.txt` in the GitHub source code for det
 
 The latest code under development is available on GitHub at
 https://github.com/civisanalytics/async-graph-data-flow.
-To obtain this version for experimental features or for development:
+To obtain this version for experimental features or for development,
+we recommend using [uv](https://docs.astral.sh/uv/) to manage the development environment:
 
 ```bash
 git clone https://github.com/civisanalytics/async-graph-data-flow.git
 cd async-graph-data-flow
-pip install -e ".[dev]"
+uv pip install -e ".[dev]"
 ```
 
 To run tests and styling checks:
 
 ```bash
-pytest
-flake8 src tests examples
-black --check src tests examples
+uv run pytest
+uv run flake8 src tests examples
+uv run black --check src tests examples
 ```
 
 ## Building Documentation
 
-We use the Sphinx framework. The documentation source files are in `docs/source/`.
+We use the Sphinx framework. The documentation source files are in `docs/`.
 These files can be updated as necessary.
 
 The public documentation is accessible at https://async-graph-data-flow.readthedocs.io.
@@ -49,5 +50,5 @@ The builds by the Read The Docs site generate the necessary files (the HTML page
 for the public documentation. All these auto-generated files are explicitly not versioned (see `.gitignore`).
 
 To build the documentation locally (for testing and development),
-install the full doc-related dependencies: `pip install -r docs/requirements.txt`,
-then run `sphinx-build -b html docs/ docs/build/`.
+install the full doc-related dependencies: `uv pip install -r docs/requirements.txt`,
+then run `uv run sphinx-build -b html docs/ docs/build/`.

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/pyproject.toml

@@ -1,14 +1,14 @@
 [build-system]
-requires = ["setuptools >= 69.0.0", "wheel"]
+requires = ["setuptools >= 77.0.0", "wheel"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "async-graph-data-flow"
-version = "1.5.1"
-description = "Asynchronous functions that pass data along a directed acyclic graph"
+version = "2.0.0"
+description = "Data flow along a directed acyclic graph of asynchronous generator functions"
 readme = "README.md"
 requires-python = ">= 3.10"
-license = { text = "BSD 3-Clause" }
+license = "BSD-3-Clause"
 authors = [ { name = "Civis Analytics", email = "opensource@civisanalytics.com" } ]
 keywords = [
     "async",
@@ -22,7 +22,6 @@ classifiers = [
     "Intended Audience :: Developers",
     "Intended Audience :: Information Technology",
     "Intended Audience :: Science/Research",
-    "License :: OSI Approved :: BSD License",
     "Operating System :: OS Independent",
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3 :: Only",
@@ -30,6 +29,7 @@ classifiers = [
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
     "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
     "Framework :: AsyncIO",
 ]
 
@@ -40,21 +40,21 @@ Source = "https://github.com/civisanalytics/async-graph-data-flow"
 [project.optional-dependencies]
 dev = [
     # Running tests and linters
-    "black == 24.10.0",
-    "flake8 == 7.1.1",
-    "pytest == 8.3.3",
-    "mypy == 1.13.0",
+    "black",
+    "flake8",
+    "pytest",
+    "mypy",
 
     # Managing source distributions
-    "build == 1.2.2",
-    "twine == 5.1.1",
+    "build",
+    "twine",
 ]
 docs = [
-    "Sphinx == 8.1.3",
-    "furo == 2024.8.6",
-    "numpydoc == 1.8.0",
-    "sphinx-copybutton == 0.5.2",
-    "sphinxcontrib-mermaid == 1.0.0",
+    "Sphinx",
+    "furo",
+    "numpydoc",
+    "sphinx-copybutton",
+    "sphinxcontrib-mermaid",
 ]
 
 [tool.setuptools.packages.find]

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/src/async_graph_data_flow/__init__.py

@@ -3,6 +3,5 @@ from importlib.metadata import version
 from .executor import AsyncExecutor
 from .graph import AsyncGraph
 
-
 __version__ = version("async-graph-data-flow")
 __all__ = ["AsyncGraph", "AsyncExecutor"]

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/src/async_graph_data_flow/executor.py

@@ -4,11 +4,11 @@ import logging
 import time
 import traceback
 from collections import deque
-from typing import Any, Iterable
+from collections.abc import Iterable
+from typing import Any
 
 from .graph import AsyncGraph, InvalidAsyncGraphError
 
-
 _LOG = logging.getLogger(__name__)
 
 _DEFAULT_DATA_FLOW_LOGGING_NODE_FORMAT = " {node} - in={in}, out={out}, err={err}"
@@ -132,20 +132,16 @@ class AsyncExecutor:
         """
         self._data_flow_logging = True
 
-        if node_format and isinstance(node_format, str):
+        if node_format:
             self._data_flow_logging_node_format = node_format
 
-        if (
-            node_filter
-            and not isinstance(node_filter, str)
-            and all(map(lambda x: isinstance(x, str), node_filter))
-        ):
+        if node_filter:
             self._data_flow_logging_node_filter = set(node_filter)
 
-        if time_interval and isinstance(time_interval, int):
+        if time_interval:
             self._data_flow_logging_time_interval = time_interval
 
-        if logger and isinstance(logger, logging.Logger):
+        if logger:
             self._logger = logger
 
     def turn_off_data_flow_logging(self) -> None:
@@ -284,7 +280,10 @@ class AsyncExecutor:
         self._exceptions = {}
 
         for node_name, node in self._graph._nodes.items():
-            queue = asyncio.Queue(maxsize=node.queue_size)
+            if node.queue is None:
+                queue = asyncio.Queue()
+            else:
+                queue = node.queue
             self._node_queues[node_name] = queue
             self._data_flow_stats[node_name] = {"in": 0, "out": 0, "err": 0}
             self._exceptions[node_name] = deque(maxlen=self._max_exceptions)
@@ -296,8 +295,8 @@ class AsyncExecutor:
 
         await self._producer()
 
-        for queue in self._node_queues.values():
-            await queue.join()
+        for node_name in self._graph._topological_sort():
+            await self._node_queues[node_name].join()
 
         for task in self._consumer_tasks.values():
             task.cancel()
@@ -329,7 +328,7 @@ class AsyncExecutor:
             Each key in this dictionary is the name (str) of the node function,
             and its corresponding value is the args (tuple)
             (in which case the node function will be called as ``func(*args)``
-            -- provide ``None`` if you want ``func()`` with no args).
+            -- provide ``()`` (an empty tuple) if you want ``func()`` with no args).
             If ``start_nodes`` is ``None`` or isn't provided,
             nodes that have no incoming edges are treated as start nodes.
         """

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/src/async_graph_data_flow/graph.py

@@ -1,6 +1,9 @@
+import asyncio
+import functools
 import inspect
 from collections import OrderedDict
-from typing import Any, Callable, NamedTuple
+from collections.abc import AsyncGenerator, Callable
+from typing import Any, NamedTuple
 
 
 class InvalidAsyncGraphError(Exception):
@@ -8,9 +11,9 @@ class InvalidAsyncGraphError(Exception):
 
 
 class _Node(NamedTuple):
-    func: Callable
+    func: Callable[..., AsyncGenerator]
     name: str
-    queue_size: int
+    queue: asyncio.Queue | None
     max_tasks: int
     halt_on_exception: bool
     unpack_input: bool
@@ -32,25 +35,32 @@ class AsyncGraph:
 
     def add_node(
         self,
-        func: Callable,
+        func: Callable[..., AsyncGenerator],
         *,
         name: str | None = None,
         halt_on_exception: bool = False,
         unpack_input: bool = True,
         max_tasks: int = 1,
-        queue_size: int = 10_000,
+        queue: asyncio.Queue | None = None,
         check_async_gen: bool = True,
     ) -> None:
         """Add a node by providing its function and optional configurations.
 
         Parameters
         ----------
-        func : Callable
-            The function that this node runs.
+        func : Callable[..., AsyncGenerator]
+            The asynchronous generator function that this node runs.
             See notes below for the function's requirements.
         name : str, optional
-            The name of this node. If not provided, the ``__name__`` attribute
-            ``func`` is used.
+            The name of this node. If not provided, the name is taken from
+            the node function's ``__name__``, resolved as follows:
+
+            * If ``func`` is a :func:`functools.partial` object, use
+              ``func.func.__name__``.
+            * If ``func`` has a ``__wrapped__`` attribute (because it was
+              decorated via :func:`functools.wraps`), use
+              ``func.__wrapped__.__name__``.
+            * Otherwise, use ``func.__name__``.
         halt_on_exception : bool, optional
             To halt graph execution when this node has an unhandled exception,
             set this argument to ``True``. Defaults to ``False``.
@@ -62,16 +72,21 @@ class AsyncGraph:
             See notes below for more details.
         max_tasks : int, optional
             The number of tasks that this node runs concurrently.
-        queue_size : int, optional
-            The maximum number of data items allowed to be
-            in the :class:`~asyncio.Queue` object between this node as a source
-            and its destination node(s).
+        queue : asyncio.Queue, optional
+            The queue object that collects items from this node's source nodes,
+            via ``await queue.put(item)``, and then feed into this node
+            with items retrieved by ``await queue.get()``.
+            This queue object must be an instance of either :class:`~asyncio.Queue` or
+            a subclass of :class:`~asyncio.Queue`.
+            If ``None`` or not given, it defaults to an unbounded ``asyncio.Queue()``.
         check_async_gen : bool, optional
             If ``True`` (the default), the callable ``func`` is verified to be an async
-            generator function by :func:`inspect.isasyncgenfunction`.
-            Pass in ``False`` to disable this check if ``func`` would fail the check
-            while the callable under the hood is still an async generator function
-            (e.g., your function is wrapped by a decorator).
+            generator function by :func:`inspect.isasyncgenfunction`. If ``func`` is a
+            :func:`functools.partial` object, or if it has a ``__wrapped__`` attribute
+            set by :func:`functools.wraps`, the embedded function is checked instead.
+            Pass in ``False`` to disable this check if ``func`` would still fail the
+            check after that unwrapping (e.g., your decorator doesn't use
+            :func:`functools.wraps`).
 
         Notes
         -----
@@ -124,38 +139,64 @@ class AsyncGraph:
             own default values.
         """  # noqa: E501
 
-        name = name or func.__name__
-        if check_async_gen and not inspect.isasyncgenfunction(func):
+        embedded_func = self._get_embedded_func(func)
+        name = name or embedded_func.__name__
+        if check_async_gen and not inspect.isasyncgenfunction(embedded_func):
             raise TypeError(f"node '{name}' isn't an async generator function")
         if name in self._nodes:
             raise ValueError(f"node '{name}' already exists in the graph")
+        if queue is not None and not isinstance(queue, asyncio.Queue):
+            raise TypeError(f"queue must be an instance of asyncio.Queue: {queue}")
         self._nodes[name] = _Node(
             func=func,
             name=name,
-            queue_size=queue_size,
+            queue=queue,
             max_tasks=max_tasks,
             halt_on_exception=halt_on_exception,
             unpack_input=unpack_input,
         )
         self._nodes_to_edges[name] = set()
 
-    def add_edge(self, src_node: str | Callable, dst_node: str | Callable) -> None:
+    @staticmethod
+    def _get_embedded_func(
+        func: Callable[..., AsyncGenerator],
+    ) -> Callable[..., AsyncGenerator]:
+        """Resolve the underlying function for name and async-gen detection.
+
+        Repeatedly unwraps :func:`functools.partial` objects and callables with
+        a ``__wrapped__`` attribute (set by :func:`functools.wraps`), so that
+        stacked decorators and ``partial(wrapped_func)`` combinations resolve
+        to the innermost function.
+        """
+        while True:
+            if isinstance(func, functools.partial):
+                func = func.func
+            elif hasattr(func, "__wrapped__"):
+                func = func.__wrapped__
+            else:
+                return func
+
+    def add_edge(
+        self,
+        src_node: str | Callable[..., AsyncGenerator],
+        dst_node: str | Callable[..., AsyncGenerator],
+    ) -> None:
         """Add an edge.
 
         Parameters
         ----------
-        src_node : str | Callable
+        src_node : str | Callable[..., AsyncGenerator]
             The source node, either the function name or the function itself.
-        dst_node : str | Callable
+        dst_node : str | Callable[..., AsyncGenerator]
             The destination node, either the function name or the function itself.
         """
         if not isinstance(src_node, str):
-            src_node = src_node.__name__
+            src_node = self._get_embedded_func(src_node).__name__
         if src_node not in self._nodes:
             raise ValueError(f"src_node '{src_node}' not registered in the graph")
 
         if not isinstance(dst_node, str):
-            dst_node = dst_node.__name__
+            dst_node = self._get_embedded_func(dst_node).__name__
         if dst_node not in self._nodes:
             raise ValueError(f"dst_node '{dst_node}' not registered in the graph")
 
@@ -188,29 +229,16 @@ class AsyncGraph:
         i: int,
         is_checked: list[bool],
         iter_stack: list[bool],
-        graph: dict[str, set[str]],
+        nodes: list[str],
     ) -> bool:
         """Code based on: https://www.geeksforgeeks.org/detect-cycle-in-a-graph/"""
-        # Use the OrderedDict self._nodes_to_edges for its ordering.
-        nodes = list(self._nodes_to_edges.keys())
         is_checked[i] = True
         iter_stack[i] = True
 
-        try:
-            dst_nodes = graph[nodes[i]]
-        except KeyError:
-            iter_stack[i] = False
-            return False
-
-        for dst_node in dst_nodes:
-            try:
-                edge_pos = nodes.index(dst_node)
-            except ValueError:
-                iter_stack[i] = False
-                return False
-
+        for dst_node in self._nodes_to_edges[nodes[i]]:
+            edge_pos = nodes.index(dst_node)
             if not is_checked[edge_pos] and self._graph_validator(
-                edge_pos, is_checked, iter_stack, graph
+                edge_pos, is_checked, iter_stack, nodes
             ):
                 return True
             elif iter_stack[edge_pos]:
@@ -220,24 +248,42 @@ class AsyncGraph:
         return False
 
     def _is_graph_cyclic(self) -> bool:
-        """Code based on: https://www.geeksforgeeks.org/detect-cycle-in-a-graph/"""
-        iter_stack = is_checked = [False] * (len(self._nodes) + 1)
-        for i in range(len(self._nodes)):
+        nodes = list(self._nodes_to_edges.keys())
+        is_checked = [False] * len(nodes)
+        iter_stack = [False] * len(nodes)
+        for i in range(len(nodes)):
             if not is_checked[i] and self._graph_validator(
-                i, is_checked, iter_stack, self._nodes_to_edges
+                i, is_checked, iter_stack, nodes
             ):
                 return True
         return False
 
+    def _topological_sort(self) -> list[str]:
+        """Return node names in a topological order (sources before sinks).
+
+        Sibling order among nodes at the same depth is unspecified (the
+        underlying edge storage is a ``set``). Callers must not rely on it.
+        The graph is guaranteed acyclic by :meth:`add_edge`, so this always succeeds.
+        """
+        indegree: dict[str, int] = {name: 0 for name in self._nodes}
+        for dst_nodes in self._nodes_to_edges.values():
+            for dst in dst_nodes:
+                indegree[dst] += 1
+
+        ready = [name for name in self._nodes if indegree[name] == 0]
+        ordered: list[str] = []
+        while ready:
+            name = ready.pop(0)
+            ordered.append(name)
+            for dst in self._nodes_to_edges[name]:
+                indegree[dst] -= 1
+                if indegree[dst] == 0:
+                    ready.append(dst)
+        return ordered
+
     def _get_start_nodes(self) -> set[str]:
-        root_nodes = set()
-        for src_node in self._nodes_to_edges.keys():
-            root_node = True
-            for dst_nodes in self._nodes_to_edges.values():
-                if src_node in dst_nodes:
-                    root_node = False
-                    break
-
-            if root_node:
-                root_nodes.add(src_node)
-        return root_nodes
+        indegree: dict[str, int] = {name: 0 for name in self._nodes}
+        for dst_nodes in self._nodes_to_edges.values():
+            for dst in dst_nodes:
+                indegree[dst] += 1
+        return {name for name, d in indegree.items() if d == 0}

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0/src/async_graph_data_flow.egg-info}/PKG-INFO

@@ -1,9 +1,9 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: async-graph-data-flow
-Version: 1.5.1
-Summary: Asynchronous functions that pass data along a directed acyclic graph
+Version: 2.0.0
+Summary: Data flow along a directed acyclic graph of asynchronous generator functions
 Author-email: Civis Analytics <opensource@civisanalytics.com>
-License: BSD 3-Clause
+License-Expression: BSD-3-Clause
 Project-URL: Documentation, https://async-graph-data-flow.readthedocs.io
 Project-URL: Source, https://github.com/civisanalytics/async-graph-data-flow
 Keywords: async,asynchronous,asyncio,data flow,graph
@@ -11,7 +11,6 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Information Technology
 Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: BSD License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
@@ -19,23 +18,25 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Framework :: AsyncIO
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Provides-Extra: dev
-Requires-Dist: black==24.10.0; extra == "dev"
-Requires-Dist: flake8==7.1.1; extra == "dev"
-Requires-Dist: pytest==8.3.3; extra == "dev"
-Requires-Dist: mypy==1.13.0; extra == "dev"
-Requires-Dist: build==1.2.2; extra == "dev"
-Requires-Dist: twine==5.1.1; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: flake8; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: mypy; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
 Provides-Extra: docs
-Requires-Dist: Sphinx==8.1.3; extra == "docs"
-Requires-Dist: furo==2024.8.6; extra == "docs"
-Requires-Dist: numpydoc==1.8.0; extra == "docs"
-Requires-Dist: sphinx-copybutton==0.5.2; extra == "docs"
-Requires-Dist: sphinxcontrib-mermaid==1.0.0; extra == "docs"
+Requires-Dist: Sphinx; extra == "docs"
+Requires-Dist: furo; extra == "docs"
+Requires-Dist: numpydoc; extra == "docs"
+Requires-Dist: sphinx-copybutton; extra == "docs"
+Requires-Dist: sphinxcontrib-mermaid; extra == "docs"
+Dynamic: license-file
 
 # async-graph-data-flow
 
@@ -59,25 +60,26 @@ BSD 3-Clause License. Please see `LICENSE.txt` in the GitHub source code for det
 
 The latest code under development is available on GitHub at
 https://github.com/civisanalytics/async-graph-data-flow.
-To obtain this version for experimental features or for development:
+To obtain this version for experimental features or for development,
+we recommend using [uv](https://docs.astral.sh/uv/) to manage the development environment:
 
 ```bash
 git clone https://github.com/civisanalytics/async-graph-data-flow.git
 cd async-graph-data-flow
-pip install -e ".[dev]"
+uv pip install -e ".[dev]"
 ```
 
 To run tests and styling checks:
 
 ```bash
-pytest
-flake8 src tests examples
-black --check src tests examples
+uv run pytest
+uv run flake8 src tests examples
+uv run black --check src tests examples
 ```
 
 ## Building Documentation
 
-We use the Sphinx framework. The documentation source files are in `docs/source/`.
+We use the Sphinx framework. The documentation source files are in `docs/`.
 These files can be updated as necessary.
 
 The public documentation is accessible at https://async-graph-data-flow.readthedocs.io.
@@ -88,5 +90,5 @@ The builds by the Read The Docs site generate the necessary files (the HTML page
 for the public documentation. All these auto-generated files are explicitly not versioned (see `.gitignore`).
 
 To build the documentation locally (for testing and development),
-install the full doc-related dependencies: `pip install -r docs/requirements.txt`,
-then run `sphinx-build -b html docs/ docs/build/`.
+install the full doc-related dependencies: `uv pip install -r docs/requirements.txt`,
+then run `uv run sphinx-build -b html docs/ docs/build/`.

async_graph_data_flow-2.0.0/src/async_graph_data_flow.egg-info/requires.txt

@@ -0,0 +1,15 @@
+
+[dev]
+black
+flake8
+pytest
+mypy
+build
+twine
+
+[docs]
+Sphinx
+furo
+numpydoc
+sphinx-copybutton
+sphinxcontrib-mermaid

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/tests/test_executor.py

@@ -1,4 +1,5 @@
 import asyncio
+import functools
 import time
 
 import pytest
@@ -75,6 +76,34 @@ def test_async_executor_linear():
     assert etl_executor.data_flow_stats["load"].get("err") == 0
 
 
+def test_async_executor_with_functools_partial_node():
+    async def transform(prefix, data):
+        yield f"{prefix}:{data}"
+
+    async def extract():
+        yield "hello"
+        yield "world"
+
+    collected: list[str] = []
+
+    async def load(data):
+        collected.append(data)
+        yield
+
+    partial_transform = functools.partial(transform, "greet")
+
+    etl_graph = AsyncGraph()
+    etl_graph.add_node(extract)
+    etl_graph.add_node(partial_transform)
+    etl_graph.add_node(load)
+    etl_graph.add_edge("extract", "transform")
+    etl_graph.add_edge("transform", "load")
+
+    AsyncExecutor(etl_graph).execute()
+
+    assert collected == ["greet:hello", "greet:world"]
+
+
 def test_async_executor_linear_with_exception_with_halt():
     async def extract():
         yield "data1"
@@ -469,3 +498,72 @@ def test_graph_with_no_edges():
 
     executor.execute(start_nodes={"node1": ("foo",)})
     assert executor.start_nodes == {"node1": ("foo",)}
+
+
+def test_diamond_dag_reverse_insertion_order():
+    """Regression test for issue #15: tail items must be processed even when
+    nodes are added in reverse topological order (sink before sources)."""
+    sink_items: list[int] = []
+
+    async def src():
+        for i in range(4):
+            yield i
+
+    async def branch_a(x):
+        yield x
+
+    async def branch_b(x):
+        yield x + 100
+
+    async def sink(x):
+        sink_items.append(x)
+        yield
+
+    graph = AsyncGraph()
+    graph.add_node(sink, unpack_input=False)
+    graph.add_node(branch_b, unpack_input=False)
+    graph.add_node(branch_a, unpack_input=False)
+    graph.add_node(src, unpack_input=False)
+
+    graph.add_edge(src, branch_a)
+    graph.add_edge(src, branch_b)
+    graph.add_edge(branch_a, sink)
+    graph.add_edge(branch_b, sink)
+
+    AsyncExecutor(graph).execute()
+
+    assert sorted(sink_items) == [0, 1, 2, 3, 100, 101, 102, 103]
+
+
+def test_diamond_dag_forward_insertion_order():
+    """Symmetric sanity check: adding nodes in forward topological order still works."""
+    sink_items: list[int] = []
+
+    async def src():
+        for i in range(4):
+            yield i
+
+    async def branch_a(x):
+        yield x
+
+    async def branch_b(x):
+        yield x + 100
+
+    async def sink(x):
+        sink_items.append(x)
+        yield
+
+    graph = AsyncGraph()
+    graph.add_node(src, unpack_input=False)
+    graph.add_node(branch_a, unpack_input=False)
+    graph.add_node(branch_b, unpack_input=False)
+    graph.add_node(sink, unpack_input=False)
+
+    graph.add_edge(src, branch_a)
+    graph.add_edge(src, branch_b)
+    graph.add_edge(branch_a, sink)
+    graph.add_edge(branch_b, sink)
+
+    AsyncExecutor(graph).execute()
+
+    assert sorted(sink_items) == [0, 1, 2, 3, 100, 101, 102, 103]

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/tests/test_graph.py

@@ -1,3 +1,4 @@
+import functools
 import inspect
 from unittest import mock
 
@@ -64,6 +65,66 @@ class TestAsyncGraphAddNode:
         )
         AsyncGraph().add_node(some_func, check_async_gen=False)
 
+    def test_add_node_with_functools_partial(self):
+        async def producer(prefix, message):
+            yield f"{prefix}:{message}"
+
+        partial_producer = functools.partial(producer, "hello")
+
+        graph = AsyncGraph()
+        graph.add_node(partial_producer)
+
+        assert "producer" in graph._nodes
+        assert graph._nodes["producer"].name == "producer"
+        assert graph._nodes["producer"].func is partial_producer
+
+    def test_add_node_with_functools_partial_explicit_name(self):
+        async def producer(prefix, message):
+            yield f"{prefix}:{message}"
+
+        partial_producer = functools.partial(producer, "hello")
+
+        graph = AsyncGraph()
+        graph.add_node(partial_producer, name="custom")
+
+        assert "custom" in graph._nodes
+        assert "producer" not in graph._nodes
+        assert graph._nodes["custom"].func is partial_producer
+
+    def test_add_node_with_functools_partial_non_async_gen(self):
+        def not_async_gen(prefix, message):
+            return f"{prefix}:{message}"
+
+        partial_fn = functools.partial(not_async_gen, "hello")
+
+        with pytest.raises(TypeError) as excinfo:
+            AsyncGraph().add_node(partial_fn)
+        assert str(excinfo.value) == (
+            "node 'not_async_gen' isn't an async generator function"
+        )
+
+    def test_add_node_with_functools_wraps_decorator(self):
+        async def some_func():
+            yield "foo"
+
+        def some_decorator(func):
+            @functools.wraps(func)
+            def inner_func(*args, **kwargs):
+                return func(*args, **kwargs)
+
+            return inner_func
+
+        wrapped = some_decorator(some_func)
+
+        # The wrapper itself is not an async generator function, but because
+        # functools.wraps sets __wrapped__, add_node unwraps to the inner
+        # async generator and the check passes without check_async_gen=False.
+        assert not inspect.isasyncgenfunction(wrapped)
+        graph = AsyncGraph()
+        graph.add_node(wrapped)
+        assert "some_func" in graph._nodes
+        assert graph._nodes["some_func"].func is wrapped
+
     def test_add_node_with_valid_node_args(self):
         etl_graph = async_graph_without_nodes_mock()
         assert len(etl_graph._nodes.keys()) == 0
@@ -84,7 +145,7 @@ class TestAsyncGraphAddNode:
                 "halt_on_exception": False,
                 "max_tasks": 1,
                 "name": "extract_node",
-                "queue_size": 10_000,
+                "queue": None,
                 "unpack_input": True,
             },
             {
@@ -92,7 +153,7 @@ class TestAsyncGraphAddNode:
                 "halt_on_exception": False,
                 "max_tasks": 1,
                 "name": "transform_node",
-                "queue_size": 10_000,
+                "queue": None,
                 "unpack_input": True,
             },
             {
@@ -100,7 +161,7 @@ class TestAsyncGraphAddNode:
                 "halt_on_exception": False,
                 "max_tasks": 1,
                 "name": "load_node",
-                "queue_size": 10_000,
+                "queue": None,
                 "unpack_input": True,
             },
         ]
@@ -135,6 +196,40 @@ class TestAsyncGraphAddEdge:
         assert len(etl_graph.edges) == 1
         assert etl_graph.edges == {("extract_node", "transform_node")}
 
+    def test_add_edge_with_functools_partial_callable(self):
+        async def producer(prefix, message):
+            yield f"{prefix}:{message}"
+
+        async def consumer(message):
+            yield message
+
+        partial_producer = functools.partial(producer, "hello")
+
+        graph = AsyncGraph()
+        graph.add_node(partial_producer)
+        graph.add_node(consumer)
+        graph.add_edge(src_node=partial_producer, dst_node=consumer)
+
+        assert graph.edges == {("producer", "consumer")}
+
+    def test_add_node_unwraps_stacked_wrappers(self):
+        async def some_func():
+            yield "foo"
+
+        def outer(func):
+            @functools.wraps(func)
+            def inner(*args, **kwargs):
+                return func(*args, **kwargs)
+
+            return inner
+
+        doubly_wrapped = outer(outer(some_func))
+        partial_over_wrapped = functools.partial(doubly_wrapped)
+
+        graph = AsyncGraph()
+        graph.add_node(partial_over_wrapped)
+        assert "some_func" in graph._nodes
+
     def test_add_edge_graph_acyclic(self):
         etl_graph = async_graph_with_nodes_mock()
 

{async_graph_data_flow-1.5.1 → async_graph_data_flow-2.0.0}/tests/test_version_number.py

@@ -3,7 +3,6 @@ import re
 
 import async_graph_data_flow
 
-
 REPO_DIR = os.path.dirname(os.path.dirname(__file__))
 
 

async_graph_data_flow-1.5.1/src/async_graph_data_flow.egg-info/requires.txt

@@ -1,15 +0,0 @@
-
-[dev]
-black==24.10.0
-flake8==7.1.1
-pytest==8.3.3
-mypy==1.13.0
-build==1.2.2
-twine==5.1.1
-
-[docs]
-Sphinx==8.1.3
-furo==2024.8.6
-numpydoc==1.8.0
-sphinx-copybutton==0.5.2
-sphinxcontrib-mermaid==1.0.0