lightweight-dag 0.1.1__tar.gz

lightweight_dag-0.1.1/PKG-INFO

@@ -0,0 +1,393 @@
+Metadata-Version: 2.3
+Name: lightweight-dag
+Version: 0.1.1
+Summary: A lightweight Python library for defining and executing DAG workflows
+Author: Lorenzo Gardini
+Author-email: Lorenzo Gardini <l.gardo98@gmail.com>
+Requires-Dist: pydantic>=2.13.5
+Requires-Dist: pyyaml>=6.0.3
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# lightweight-dag
+
+Python library for defining, validating, and executing directed acyclic graphs
+(DAGs). Operations are registered Python functions. DAGs can be loaded from
+YAML or dictionaries, grouped into named suites, and executed for only the
+requested outputs.
+
+Requires Python 3.11 or later.
+
+## Installation
+
+Install the published package from PyPI with uv:
+
+```bash
+uv add lightweight-dag
+```
+
+To install the development and test dependencies when working from a clone of
+the repository:
+
+```bash
+uv sync --group dev
+```
+
+The public API is available from the package root:
+
+```python
+from lightweight_dag import Dag, DagExecutionResult, DagSuite, Node, dag_node
+```
+
+## Registering operations
+
+Operations must be synchronous functions defined at module level. Passing one
+argument to the decorator registers a global operation:
+
+```python
+from lightweight_dag import dag_node
+
+
+@dag_node("normalize")
+def normalize(values):
+    return [value.strip().lower() for value in values]
+```
+
+Passing two arguments registers an operation for a specific DAG:
+
+```python
+from lightweight_dag import dag_node
+
+
+@dag_node("example_report", "filter_orders")
+def filter_monthly_orders(orders, minimum_total):
+    return [order for order in orders if order["total"] >= minimum_total]
+```
+
+A DAG without `name` looks up only global operations. A named DAG looks up only
+operations registered for that name; there is no fallback to the global scope.
+The same operation can therefore have different implementations in different
+scopes. Registering an operation twice in the same scope is an error.
+
+## Defining and running a suite
+
+A YAML suite maps DAG names to DAG definitions. Each `Node.inputs` entry must
+refer to another node in the same DAG. Source nodes have no inputs and produce
+their values through their registered operation. The block-style suite example
+in [examples/dags.yaml](examples/dags.yaml) defines both `monthly_report` and
+`sales_audit`; the following code loads and executes both DAGs.
+
+Register each operation in the scope of the DAG that uses it. The same
+operation ID can have different implementations in different DAG scopes:
+
+```python
+from pathlib import Path
+
+from lightweight_dag import DagSuite, dag_node
+
+
+@dag_node("monthly_report", "load_orders")
+def load_monthly_orders():
+        return [{"total": 75}, {"total": 35}]
+
+
+@dag_node("monthly_report", "filter_orders")
+def filter_monthly_orders(orders, minimum_total):
+        return [order for order in orders if order["total"] >= minimum_total]
+
+
+@dag_node("monthly_report", "summarize_orders")
+def summarize_monthly_orders(eligible_orders):
+        return {"count": len(eligible_orders)}
+
+
+@dag_node("sales_audit", "load_orders")
+def load_audit_orders():
+        return [{"total": 75}, {"total": 35}]
+
+
+@dag_node("sales_audit", "calculate_total")
+def calculate_total(orders):
+        return sum(order["total"] for order in orders)
+
+
+@dag_node("sales_audit", "classify_total")
+def classify_total(total, review_threshold):
+        return "review" if total >= review_threshold else "ok"
+
+
+suite = DagSuite.from_yaml(Path("examples/dags.yaml").read_text())
+monthly_report = suite["monthly_report"]
+sales_audit = suite["sales_audit"]
+
+report_result = monthly_report.run(outputs=["summary"])
+audit_result = sales_audit.run(outputs=["audit_result"])
+
+assert report_result.results["summary"] == {"count": 1}
+assert audit_result.results["audit_result"] == "review"
+```
+
+The operation registry is scoped: `monthly_report` cannot use operations
+registered only for `sales_audit`, and named DAGs do not fall back to global
+operations. The YAML key supplies the DAG name; an explicit `name` field is
+optional, but if present it must match the key. `suite[name]` raises `KeyError`
+for an unknown name.
+
+`DagSuite.from_dict(...)` accepts the same name-to-definition mapping. For a
+single DAG, use `Dag.from_yaml(...)` or `Dag.from_dict(...)`. This standalone
+example builds a DAG from a dictionary and executes its output:
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("make_number")
+def make_number():
+        return 21
+
+
+@dag_node("double_number")
+def double_number(number):
+        return number * 2
+
+
+dag = Dag.from_dict(
+        {
+                "nodes": [
+                        {"node_id": "number", "operation": "make_number"},
+                        {
+                                "node_id": "answer",
+                                "operation": "double_number",
+                                "inputs": ["number"],
+                        },
+                ]
+        }
+)
+result = dag.run(outputs=["answer"])
+assert result.results["answer"] == 42
+```
+
+To load a single DAG from YAML, register its global operations and pass the
+YAML document to `Dag.from_yaml(...)`:
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("yaml_source")
+def yaml_source(): return 21
+
+
+@dag_node("yaml_double")
+def yaml_double(number): return number * 2
+
+
+dag = Dag.from_yaml("""{nodes: [
+{node_id: number, operation: yaml_source},
+{node_id: answer, operation: yaml_double, inputs: [number]}
+]}""")
+assert dag.run(outputs=["answer"]).results["answer"] == 42
+```
+
+## Inputs and execution
+
+Call `Dag.run()` with no arguments to execute every node. To execute only
+selected outputs and their dependencies, pass their IDs in `outputs`. Each
+operation receives its predecessor node results and the YAML `params` as
+keyword arguments. Argument names correspond to predecessor IDs or parameter
+keys.
+
+```python
+all_results = monthly_report.run()
+assert set(all_results.results) == {"orders", "eligible_orders", "summary"}
+
+summary_result = monthly_report.run(outputs=["summary"])
+assert summary_result.results["summary"] == {"count": 1}
+
+if summary_result.status:
+        print(summary_result.results["summary"])
+```
+
+An unknown output raises `ValueError`. The result contains `status` and
+`results`; `results` contains the computed node values. With `outputs=[]`, no
+nodes are executed and `results` is empty. Omitting `outputs` is different:
+all nodes are selected, including independent branches.
+
+Shared predecessors are computed only once, and nodes outside the requested
+paths are not executed. In this graph, requesting only `d` computes `a`, `b`,
+and `d`; `c` is not executed:
+
+```mermaid
+flowchart TD
+        a --> b --> c
+        b --> d
+```
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("get_a")
+def get_a():
+        return "a"
+
+
+@dag_node("make_b")
+def make_b(a):
+        return f"{a}->b"
+
+
+@dag_node("make_c")
+def make_c(b):
+        return f"{b}->c"
+
+
+@dag_node("make_d")
+def make_d(b):
+        return f"{b}->d"
+
+
+dag = Dag.from_dict({"nodes": [
+        {"node_id": "a", "operation": "get_a"},
+        {"node_id": "b", "operation": "make_b", "inputs": ["a"]},
+        {"node_id": "c", "operation": "make_c", "inputs": ["b"]},
+        {"node_id": "d", "operation": "make_d", "inputs": ["b"]},
+]})
+result = dag.run(outputs=["d"])
+
+assert result.results == {
+        "a": "a",
+        "b": "a->b",
+        "d": "a->b->d",
+}
+assert "c" not in result.results
+```
+
+### Using an LLM with LangChain
+
+This example simulates reading a customer record from a database, then sends
+that context and a user prompt to an LLM through LangChain. Install the optional
+integration and set an API key before running it:
+
+```bash
+pip install langchain-openai
+export OPENAI_API_KEY="your-api-key"
+```
+
+```python
+from lightweight_dag import Dag, dag_node
+from langchain_openai import ChatOpenAI
+from langchain_core.prompts import ChatPromptTemplate
+
+
+@dag_node("load_customer")
+def load_customer():
+        # Replace this in-memory record with a database query.
+        return {"name": "Ada", "plan": "Pro", "last_order": "Notebook"}
+
+
+@dag_node("answer_user")
+def answer_user(customer_data, user_prompt):
+        prompt = ChatPromptTemplate.from_messages([
+                ("system", "Answer using the customer data provided: {customer_data}"),
+                ("human", "{user_prompt}"),
+        ])
+        chain = prompt | ChatOpenAI(model="gpt-4o-mini")
+        response = chain.invoke({
+                "customer_data": customer_data,
+                "user_prompt": user_prompt,
+        })
+        return response.content
+
+
+support_dag = Dag.from_dict({"nodes": [
+        {"node_id": "customer_data", "operation": "load_customer"},
+        {
+                "node_id": "answer",
+                "operation": "answer_user",
+                "inputs": ["customer_data"],
+                "params": {"user_prompt": "Qual è il mio ultimo ordine?"},
+        },
+]})
+result = support_dag.run(outputs=["answer"])
+print(result.results["answer"])
+```
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("branching_demo", "label")
+def label_node(label, **predecessors): return label
+
+
+branching_dag = Dag.from_dict({"name": "branching_demo", "nodes": [
+{"node_id": "A", "operation": "label", "params": {"label": "A"}},
+{"node_id": "B", "operation": "label", "inputs": ["A"], "params": {"label": "B"}},
+{"node_id": "E", "operation": "label", "inputs": ["B"], "params": {"label": "E"}},
+{"node_id": "G", "operation": "label", "inputs": ["E"], "params": {"label": "G"}},
+{"node_id": "H", "operation": "label", "inputs": ["E"], "params": {"label": "H"}},
+{"node_id": "F", "operation": "label", "inputs": ["B"], "params": {"label": "F"}},
+{"node_id": "I", "operation": "label", "inputs": ["F"], "params": {"label": "I"}},
+{"node_id": "J", "operation": "label", "inputs": ["F"], "params": {"label": "J"}},
+{"node_id": "K", "operation": "label", "inputs": ["G", "H"], "params": {"label": "K"}},
+{"node_id": "N", "operation": "label", "inputs": ["I", "J", "G"], "params": {"label": "N"}},
+]})
+result = branching_dag.run(outputs=["G", "J", "A"])
+assert set(result.results) == {"A", "B", "E", "F", "G", "J"}
+```
+
+If an operation raises an exception, its result and the results of its
+dependent nodes become `None`, and `status` becomes `False`. Requested,
+independent branches continue to execute. This example demonstrates the
+failure result:
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("always_fail")
+def always_fail(): raise RuntimeError("operation failed")
+
+
+@dag_node("consume_result")
+def consume_result(value): return value
+
+
+failure_dag = Dag.from_dict({"nodes": [
+{"node_id": "failed", "operation": "always_fail"},
+{"node_id": "dependent", "operation": "consume_result", "inputs": ["failed"]},
+]})
+failure_result = failure_dag.run(outputs=["dependent"])
+assert failure_result.status is False
+assert failure_result.results["failed"] is None
+assert failure_result.results["dependent"] is None
+```
+
+## Validation and security
+
+Loading validates that node IDs are unique, every dependency refers to an
+existing node, the graph is acyclic, and operations are registered in the
+correct scope. It also rejects unknown fields, collisions between node inputs
+and params, and duplicate YAML keys. `params` values must be strings or
+numbers. `Node`, `Dag`, and `DagSuite` definitions, including their
+collections, are immutable after validation; `DagExecutionResult` contains
+the results of an individual execution.
+
+Topological sorting uses Kahn's algorithm with $O(V+E)$ complexity, where $V$
+is the number of nodes and $E$ is the number of dependencies. Execution sorts
+only the subgraph needed for the requested outputs.
+
+YAML definitions can invoke Python functions registered by the application.
+Only load and execute configurations from trusted sources.
+
+## Testing and building
+
+```bash
+uv run pytest
+uv build
+```
+
+`pytest` automatically measures line and branch coverage for the entire
+package. To run an individual test without coverage, use
+`uv run pytest --no-cov ...`.

lightweight_dag-0.1.1/README.md

@@ -0,0 +1,382 @@
+# lightweight-dag
+
+Python library for defining, validating, and executing directed acyclic graphs
+(DAGs). Operations are registered Python functions. DAGs can be loaded from
+YAML or dictionaries, grouped into named suites, and executed for only the
+requested outputs.
+
+Requires Python 3.11 or later.
+
+## Installation
+
+Install the published package from PyPI with uv:
+
+```bash
+uv add lightweight-dag
+```
+
+To install the development and test dependencies when working from a clone of
+the repository:
+
+```bash
+uv sync --group dev
+```
+
+The public API is available from the package root:
+
+```python
+from lightweight_dag import Dag, DagExecutionResult, DagSuite, Node, dag_node
+```
+
+## Registering operations
+
+Operations must be synchronous functions defined at module level. Passing one
+argument to the decorator registers a global operation:
+
+```python
+from lightweight_dag import dag_node
+
+
+@dag_node("normalize")
+def normalize(values):
+    return [value.strip().lower() for value in values]
+```
+
+Passing two arguments registers an operation for a specific DAG:
+
+```python
+from lightweight_dag import dag_node
+
+
+@dag_node("example_report", "filter_orders")
+def filter_monthly_orders(orders, minimum_total):
+    return [order for order in orders if order["total"] >= minimum_total]
+```
+
+A DAG without `name` looks up only global operations. A named DAG looks up only
+operations registered for that name; there is no fallback to the global scope.
+The same operation can therefore have different implementations in different
+scopes. Registering an operation twice in the same scope is an error.
+
+## Defining and running a suite
+
+A YAML suite maps DAG names to DAG definitions. Each `Node.inputs` entry must
+refer to another node in the same DAG. Source nodes have no inputs and produce
+their values through their registered operation. The block-style suite example
+in [examples/dags.yaml](examples/dags.yaml) defines both `monthly_report` and
+`sales_audit`; the following code loads and executes both DAGs.
+
+Register each operation in the scope of the DAG that uses it. The same
+operation ID can have different implementations in different DAG scopes:
+
+```python
+from pathlib import Path
+
+from lightweight_dag import DagSuite, dag_node
+
+
+@dag_node("monthly_report", "load_orders")
+def load_monthly_orders():
+        return [{"total": 75}, {"total": 35}]
+
+
+@dag_node("monthly_report", "filter_orders")
+def filter_monthly_orders(orders, minimum_total):
+        return [order for order in orders if order["total"] >= minimum_total]
+
+
+@dag_node("monthly_report", "summarize_orders")
+def summarize_monthly_orders(eligible_orders):
+        return {"count": len(eligible_orders)}
+
+
+@dag_node("sales_audit", "load_orders")
+def load_audit_orders():
+        return [{"total": 75}, {"total": 35}]
+
+
+@dag_node("sales_audit", "calculate_total")
+def calculate_total(orders):
+        return sum(order["total"] for order in orders)
+
+
+@dag_node("sales_audit", "classify_total")
+def classify_total(total, review_threshold):
+        return "review" if total >= review_threshold else "ok"
+
+
+suite = DagSuite.from_yaml(Path("examples/dags.yaml").read_text())
+monthly_report = suite["monthly_report"]
+sales_audit = suite["sales_audit"]
+
+report_result = monthly_report.run(outputs=["summary"])
+audit_result = sales_audit.run(outputs=["audit_result"])
+
+assert report_result.results["summary"] == {"count": 1}
+assert audit_result.results["audit_result"] == "review"
+```
+
+The operation registry is scoped: `monthly_report` cannot use operations
+registered only for `sales_audit`, and named DAGs do not fall back to global
+operations. The YAML key supplies the DAG name; an explicit `name` field is
+optional, but if present it must match the key. `suite[name]` raises `KeyError`
+for an unknown name.
+
+`DagSuite.from_dict(...)` accepts the same name-to-definition mapping. For a
+single DAG, use `Dag.from_yaml(...)` or `Dag.from_dict(...)`. This standalone
+example builds a DAG from a dictionary and executes its output:
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("make_number")
+def make_number():
+        return 21
+
+
+@dag_node("double_number")
+def double_number(number):
+        return number * 2
+
+
+dag = Dag.from_dict(
+        {
+                "nodes": [
+                        {"node_id": "number", "operation": "make_number"},
+                        {
+                                "node_id": "answer",
+                                "operation": "double_number",
+                                "inputs": ["number"],
+                        },
+                ]
+        }
+)
+result = dag.run(outputs=["answer"])
+assert result.results["answer"] == 42
+```
+
+To load a single DAG from YAML, register its global operations and pass the
+YAML document to `Dag.from_yaml(...)`:
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("yaml_source")
+def yaml_source(): return 21
+
+
+@dag_node("yaml_double")
+def yaml_double(number): return number * 2
+
+
+dag = Dag.from_yaml("""{nodes: [
+{node_id: number, operation: yaml_source},
+{node_id: answer, operation: yaml_double, inputs: [number]}
+]}""")
+assert dag.run(outputs=["answer"]).results["answer"] == 42
+```
+
+## Inputs and execution
+
+Call `Dag.run()` with no arguments to execute every node. To execute only
+selected outputs and their dependencies, pass their IDs in `outputs`. Each
+operation receives its predecessor node results and the YAML `params` as
+keyword arguments. Argument names correspond to predecessor IDs or parameter
+keys.
+
+```python
+all_results = monthly_report.run()
+assert set(all_results.results) == {"orders", "eligible_orders", "summary"}
+
+summary_result = monthly_report.run(outputs=["summary"])
+assert summary_result.results["summary"] == {"count": 1}
+
+if summary_result.status:
+        print(summary_result.results["summary"])
+```
+
+An unknown output raises `ValueError`. The result contains `status` and
+`results`; `results` contains the computed node values. With `outputs=[]`, no
+nodes are executed and `results` is empty. Omitting `outputs` is different:
+all nodes are selected, including independent branches.
+
+Shared predecessors are computed only once, and nodes outside the requested
+paths are not executed. In this graph, requesting only `d` computes `a`, `b`,
+and `d`; `c` is not executed:
+
+```mermaid
+flowchart TD
+        a --> b --> c
+        b --> d
+```
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("get_a")
+def get_a():
+        return "a"
+
+
+@dag_node("make_b")
+def make_b(a):
+        return f"{a}->b"
+
+
+@dag_node("make_c")
+def make_c(b):
+        return f"{b}->c"
+
+
+@dag_node("make_d")
+def make_d(b):
+        return f"{b}->d"
+
+
+dag = Dag.from_dict({"nodes": [
+        {"node_id": "a", "operation": "get_a"},
+        {"node_id": "b", "operation": "make_b", "inputs": ["a"]},
+        {"node_id": "c", "operation": "make_c", "inputs": ["b"]},
+        {"node_id": "d", "operation": "make_d", "inputs": ["b"]},
+]})
+result = dag.run(outputs=["d"])
+
+assert result.results == {
+        "a": "a",
+        "b": "a->b",
+        "d": "a->b->d",
+}
+assert "c" not in result.results
+```
+
+### Using an LLM with LangChain
+
+This example simulates reading a customer record from a database, then sends
+that context and a user prompt to an LLM through LangChain. Install the optional
+integration and set an API key before running it:
+
+```bash
+pip install langchain-openai
+export OPENAI_API_KEY="your-api-key"
+```
+
+```python
+from lightweight_dag import Dag, dag_node
+from langchain_openai import ChatOpenAI
+from langchain_core.prompts import ChatPromptTemplate
+
+
+@dag_node("load_customer")
+def load_customer():
+        # Replace this in-memory record with a database query.
+        return {"name": "Ada", "plan": "Pro", "last_order": "Notebook"}
+
+
+@dag_node("answer_user")
+def answer_user(customer_data, user_prompt):
+        prompt = ChatPromptTemplate.from_messages([
+                ("system", "Answer using the customer data provided: {customer_data}"),
+                ("human", "{user_prompt}"),
+        ])
+        chain = prompt | ChatOpenAI(model="gpt-4o-mini")
+        response = chain.invoke({
+                "customer_data": customer_data,
+                "user_prompt": user_prompt,
+        })
+        return response.content
+
+
+support_dag = Dag.from_dict({"nodes": [
+        {"node_id": "customer_data", "operation": "load_customer"},
+        {
+                "node_id": "answer",
+                "operation": "answer_user",
+                "inputs": ["customer_data"],
+                "params": {"user_prompt": "Qual è il mio ultimo ordine?"},
+        },
+]})
+result = support_dag.run(outputs=["answer"])
+print(result.results["answer"])
+```
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("branching_demo", "label")
+def label_node(label, **predecessors): return label
+
+
+branching_dag = Dag.from_dict({"name": "branching_demo", "nodes": [
+{"node_id": "A", "operation": "label", "params": {"label": "A"}},
+{"node_id": "B", "operation": "label", "inputs": ["A"], "params": {"label": "B"}},
+{"node_id": "E", "operation": "label", "inputs": ["B"], "params": {"label": "E"}},
+{"node_id": "G", "operation": "label", "inputs": ["E"], "params": {"label": "G"}},
+{"node_id": "H", "operation": "label", "inputs": ["E"], "params": {"label": "H"}},
+{"node_id": "F", "operation": "label", "inputs": ["B"], "params": {"label": "F"}},
+{"node_id": "I", "operation": "label", "inputs": ["F"], "params": {"label": "I"}},
+{"node_id": "J", "operation": "label", "inputs": ["F"], "params": {"label": "J"}},
+{"node_id": "K", "operation": "label", "inputs": ["G", "H"], "params": {"label": "K"}},
+{"node_id": "N", "operation": "label", "inputs": ["I", "J", "G"], "params": {"label": "N"}},
+]})
+result = branching_dag.run(outputs=["G", "J", "A"])
+assert set(result.results) == {"A", "B", "E", "F", "G", "J"}
+```
+
+If an operation raises an exception, its result and the results of its
+dependent nodes become `None`, and `status` becomes `False`. Requested,
+independent branches continue to execute. This example demonstrates the
+failure result:
+
+```python
+from lightweight_dag import Dag, dag_node
+
+
+@dag_node("always_fail")
+def always_fail(): raise RuntimeError("operation failed")
+
+
+@dag_node("consume_result")
+def consume_result(value): return value
+
+
+failure_dag = Dag.from_dict({"nodes": [
+{"node_id": "failed", "operation": "always_fail"},
+{"node_id": "dependent", "operation": "consume_result", "inputs": ["failed"]},
+]})
+failure_result = failure_dag.run(outputs=["dependent"])
+assert failure_result.status is False
+assert failure_result.results["failed"] is None
+assert failure_result.results["dependent"] is None
+```
+
+## Validation and security
+
+Loading validates that node IDs are unique, every dependency refers to an
+existing node, the graph is acyclic, and operations are registered in the
+correct scope. It also rejects unknown fields, collisions between node inputs
+and params, and duplicate YAML keys. `params` values must be strings or
+numbers. `Node`, `Dag`, and `DagSuite` definitions, including their
+collections, are immutable after validation; `DagExecutionResult` contains
+the results of an individual execution.
+
+Topological sorting uses Kahn's algorithm with $O(V+E)$ complexity, where $V$
+is the number of nodes and $E$ is the number of dependencies. Execution sorts
+only the subgraph needed for the requested outputs.
+
+YAML definitions can invoke Python functions registered by the application.
+Only load and execute configurations from trusted sources.
+
+## Testing and building
+
+```bash
+uv run pytest
+uv build
+```
+
+`pytest` automatically measures line and branch coverage for the entire
+package. To run an individual test without coverage, use
+`uv run pytest --no-cov ...`.

lightweight_dag-0.1.1/pyproject.toml

@@ -0,0 +1,36 @@
+[project]
+name = "lightweight-dag"
+version = "0.1.1"
+description = "A lightweight Python library for defining and executing DAG workflows"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "pydantic>=2.13.5",
+    "pyyaml>=6.0.3",
+]
+
+[[project.authors]]
+name = "Lorenzo Gardini"
+email = "l.gardo98@gmail.com"
+
+[build-system]
+requires = ["uv_build>=0.10.0,<0.11.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "pytest>=9.1.1",
+    "pytest-cov>=7.1.0",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = [
+    "--cov=lightweight_dag",
+    "--cov-branch",
+    "--cov-report=term-missing",
+]
+
+[tool.coverage.run]
+branch = true
+source = ["lightweight_dag"]

lightweight_dag-0.1.1/pyproject.toml.orig

@@ -0,0 +1,35 @@
+[project]
+name = "lightweight-dag"
+version = "0.1.1"
+description = "A lightweight Python library for defining and executing DAG workflows"
+readme = "README.md"
+authors = [
+    { name = "Lorenzo Gardini", email = "l.gardo98@gmail.com" }
+]
+requires-python = ">=3.11"
+dependencies = [
+    "pydantic>=2.13.5",
+    "pyyaml>=6.0.3",
+]
+
+[build-system]
+requires = ["uv_build>=0.10.0,<0.11.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "pytest>=9.1.1",
+    "pytest-cov>=7.1.0",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = [
+    "--cov=lightweight_dag",
+    "--cov-branch",
+    "--cov-report=term-missing",
+]
+
+[tool.coverage.run]
+branch = true
+source = ["lightweight_dag"]

lightweight_dag-0.1.1/src/lightweight_dag/__init__.py

@@ -0,0 +1,13 @@
+"""Public API for building and running lightweight DAG workflows."""
+
+from .dag import Dag, DagExecutionResult, Node
+from .node_registry import dag_node
+from .suite import DagSuite
+
+__all__ = [
+    "Dag",
+    "DagExecutionResult",
+    "DagSuite",
+    "Node",
+    "dag_node",
+]

lightweight_dag-0.1.1/src/lightweight_dag/dag.py

@@ -0,0 +1,306 @@
+"""Typed DAG models, graph validation, execution, and named DAG suites."""
+
+from collections import deque
+from functools import cached_property
+from typing import Any
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+from lightweight_dag.node_registry import get_registered_nodes
+
+
+class _FrozenList(list):
+    """A list-compatible container that rejects in-place mutation."""
+
+    @staticmethod
+    def _reject_mutation(*args, **kwargs):
+        raise TypeError("DAG model collections are immutable")
+
+    __delitem__ = __iadd__ = __imul__ = __setitem__ = _reject_mutation
+    append = clear = extend = insert = pop = remove = reverse = sort = _reject_mutation
+
+
+class _FrozenDict(dict):
+    """A dict-compatible container that rejects in-place mutation."""
+
+    @staticmethod
+    def _reject_mutation(*args, **kwargs):
+        raise TypeError("DAG model collections are immutable")
+
+    __delitem__ = __ior__ = __setitem__ = _reject_mutation
+    clear = pop = popitem = setdefault = update = _reject_mutation
+
+
+def _load_yaml(yaml_str: str) -> Any:
+    """Load safe YAML while rejecting duplicate mapping keys."""
+    import yaml
+    from yaml.constructor import ConstructorError
+
+    class UniqueKeyLoader(yaml.SafeLoader):
+        pass
+
+    def construct_unique_mapping(loader, node, deep=False):
+        loader.flatten_mapping(node)
+        mapping = {}
+        for key_node, value_node in node.value:
+            key = loader.construct_object(key_node, deep=deep)
+            if key in mapping:
+                raise ConstructorError(
+                    "while constructing a mapping",
+                    node.start_mark,
+                    f"found duplicate key {key!r}",
+                    key_node.start_mark,
+                )
+            mapping[key] = loader.construct_object(value_node, deep=deep)
+        return mapping
+
+    UniqueKeyLoader.add_constructor(
+        yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
+        construct_unique_mapping,
+    )
+    return yaml.load(yaml_str, Loader=UniqueKeyLoader)
+
+
+class Node(BaseModel):
+    """One operation in a DAG and the IDs of its predecessor nodes."""
+
+    node_id: str = Field(..., description="The unique identifier for this node")
+    operation: str = Field(..., description="The operation performed by this node")
+    inputs: list[str] = Field(
+        default_factory=list, description="List of input node IDs"
+    )
+    params: dict[str, int | float | str] = Field(
+        default_factory=dict, description="Dictionary of parameters for this node"
+    )
+
+    model_config = ConfigDict(
+        extra="forbid",
+        frozen=True,
+    )
+
+    @field_validator("inputs")
+    @classmethod
+    def freeze_inputs(cls, value: list[str]) -> list[str]:
+        return _FrozenList(value)
+
+    @field_validator("params")
+    @classmethod
+    def freeze_params(
+        cls, value: dict[str, int | float | str]
+    ) -> dict[str, int | float | str]:
+        return _FrozenDict(value)
+
+
+class DagExecutionResult(BaseModel):
+    """Execution status and values produced or retained during a DAG run."""
+
+    status: bool
+    results: dict[str, Any] = Field(
+        default_factory=dict, description="Dictionary of results for this DAG execution"
+    )
+
+
+class Dag(BaseModel):
+    """A validated directed acyclic graph of registered operations.
+
+    Unnamed DAGs resolve global operations. Named DAGs resolve operations from
+    their matching registry scope only.
+    """
+
+    nodes: list[Node]
+    name: str | None = Field(
+        default=None, description="Optional registry scope for this DAG"
+    )
+
+    model_config = ConfigDict(
+        extra="forbid",
+        frozen=True,
+    )
+
+    @field_validator("name")
+    @classmethod
+    def check_name(cls, value: str | None) -> str | None:
+        """Reject blank names while allowing unnamed DAGs."""
+        if value is not None and not value.strip():
+            raise ValueError("DAG name must be non-empty")
+        return value
+
+    @field_validator("nodes")
+    @classmethod
+    def freeze_collections(cls, value: list) -> list:
+        return _FrozenList(value)
+
+    @cached_property
+    def _nodes_by_id(self) -> dict[str, Node]:
+        return _FrozenDict({node.node_id: node for node in self.nodes})
+
+    @model_validator(mode="after")
+    def check_duplicate_node_ids(self) -> "Dag":
+        """Reject repeated node IDs."""
+        node_ids = [node.node_id for node in self.nodes]
+        if len(node_ids) != len(set(node_ids)):
+            raise ValueError("Node IDs must be unique")
+        return self
+
+    @model_validator(mode="after")
+    def check_inputs_exist(self) -> "Dag":
+        """Ensure node inputs refer to existing nodes."""
+        node_ids = {node.node_id for node in self.nodes}
+        missing_inputs = {
+            input_id
+            for node in self.nodes
+            for input_id in node.inputs
+            if input_id not in node_ids
+        }
+        if missing_inputs:
+            raise ValueError(
+                f"Inputs reference non-existent nodes: {sorted(missing_inputs)}"
+            )
+        return self
+
+    @model_validator(mode="after")
+    def check_input_parameter_conflicts(self) -> "Dag":
+        """Reject keyword names that collide between inputs and params."""
+        conflicts = {
+            node.node_id: sorted(set(node.inputs) & node.params.keys())
+            for node in self.nodes
+            if set(node.inputs) & node.params.keys()
+        }
+        if conflicts:
+            raise ValueError(f"Node inputs conflict with parameters: {conflicts}")
+        return self
+
+    @model_validator(mode="after")
+    def check_acyclic(self) -> "Dag":
+        """Reject dependency cycles."""
+        self.topological_sort(self.nodes)
+        return self
+
+    @model_validator(mode="after")
+    def check_operations_are_registered(self) -> "Dag":
+        """Ensure every operation exists in this DAG's registry scope."""
+        registered_functions = (
+            get_registered_nodes()
+            if self.name is None
+            else get_registered_nodes(self.name)
+        )
+        function_not_found = [
+            node.node_id
+            for node in self.nodes
+            if node.operation not in registered_functions
+        ]
+        if function_not_found:
+            raise ValueError(
+                f"Operations reference non-existent functions: {sorted(function_not_found)}"
+            )
+        return self
+
+    @classmethod
+    def from_yaml(cls, yaml_str: str) -> "Dag":
+        """Parse and validate one DAG from a YAML document."""
+        data = _load_yaml(yaml_str)
+        return cls.model_validate(data)
+
+    @classmethod
+    def from_dict(cls, data: dict) -> "Dag":
+        """Validate one DAG from a dictionary."""
+        return cls.model_validate(data)
+
+    @classmethod
+    def topological_sort(cls, nodes: list[Node]) -> list[str]:
+        """Return node IDs in dependency order using Kahn's linear-time algorithm."""
+        node_ids = [node.node_id for node in nodes]
+        if len(node_ids) != len(set(node_ids)):
+            raise ValueError("Node IDs must be unique")
+
+        known_ids = set(node_ids)
+        missing_inputs = {
+            input_id
+            for node in nodes
+            for input_id in node.inputs
+            if input_id not in known_ids
+        }
+        if missing_inputs:
+            raise ValueError(
+                f"Inputs reference non-existent nodes: {sorted(missing_inputs)}"
+            )
+
+        in_degree = {}
+        dependents = {node_id: [] for node_id in node_ids}
+        for node in nodes:
+            dependencies = set(node.inputs) & known_ids
+            in_degree[node.node_id] = len(dependencies)
+            for dependency in dependencies:
+                dependents[dependency].append(node.node_id)
+
+        ready = deque(node.node_id for node in nodes if in_degree[node.node_id] == 0)
+        result = []
+        while ready:
+            node_id = ready.popleft()
+            result.append(node_id)
+            for dependent in dependents[node_id]:
+                in_degree[dependent] -= 1
+                if in_degree[dependent] == 0:
+                    ready.append(dependent)
+
+        if len(result) != len(nodes):
+            raise ValueError("DAG contains a cycle")
+
+        return result
+
+    def run(self, outputs: list[str] | None = None) -> DagExecutionResult:
+        """Execute requested outputs and their transitive dependencies.
+
+        Each operation receives predecessor results and node params as keyword
+        arguments. Failed nodes are skipped along with their descendants, while
+        independent requested branches continue to run. If outputs is omitted,
+        every node is selected; an empty list selects no nodes.
+        """
+        nodes_by_id = self._nodes_by_id
+        selected_outputs = list(nodes_by_id) if outputs is None else outputs
+        unknown_outputs = set(selected_outputs) - nodes_by_id.keys()
+        if unknown_outputs:
+            raise ValueError(
+                f"Outputs reference non-existent nodes: {sorted(unknown_outputs)}"
+            )
+
+        required_node_ids = set()
+        required_nodes = []
+        pending_node_ids = list(selected_outputs)
+        while pending_node_ids:
+            node_id = pending_node_ids.pop()
+            if node_id not in required_node_ids:
+                required_node_ids.add(node_id)
+                node = nodes_by_id[node_id]
+                required_nodes.append(node)
+                pending_node_ids.extend(node.inputs)
+        sorted_node_ids = [node_id for node_id in self.topological_sort(required_nodes)]
+        context = {}
+        is_run_successful = True
+        failed_node_ids = set()
+        registered_functions = (
+            get_registered_nodes()
+            if self.name is None
+            else get_registered_nodes(self.name)
+        )
+
+        for node_id in sorted_node_ids:
+            node = nodes_by_id[node_id]
+            if any(input_id in failed_node_ids for input_id in node.inputs):
+                context[node_id] = None
+                failed_node_ids.add(node_id)
+                is_run_successful = False
+                continue
+
+            node_inputs = {input_id: context[input_id] for input_id in node.inputs}
+            node_inputs.update(node.params)
+            try:
+                result = registered_functions[node.operation](**node_inputs)
+            except Exception:
+                context[node_id] = None
+                failed_node_ids.add(node_id)
+                is_run_successful = False
+            else:
+                context[node_id] = result
+
+        return DagExecutionResult(status=is_run_successful, results=context)

lightweight_dag-0.1.1/src/lightweight_dag/node_registry.py

@@ -0,0 +1,83 @@
+"""Registry for synchronous operations used by DAGs."""
+
+import inspect
+from collections.abc import Callable
+from threading import Lock
+from typing import Any, TypeVar, overload
+
+F = TypeVar("F", bound=Callable[..., Any])
+
+_nodes: dict[tuple[str | None, str], Callable[..., Any]] = {}
+_nodes_lock = Lock()
+_MISSING = object()
+
+
+@overload
+def dag_node(node_id: str) -> Callable[[F], F]: ...
+
+
+@overload
+def dag_node(dag_name: str, node_id: str) -> Callable[[F], F]: ...
+
+
+def dag_node(
+    dag_name_or_node_id: str,
+    node_id: str | None | object = _MISSING,
+) -> Callable[[F], F]:
+    """Register a global operation or one scoped to a named DAG.
+
+    Use ``@dag_node("operation")`` for a global operation or
+    ``@dag_node("dag_name", "operation")`` for a DAG-specific version.
+    Named DAG scopes do not fall back to global registrations.
+    """
+    if node_id is _MISSING:
+        dag_name = None
+        operation_id = dag_name_or_node_id
+    else:
+        dag_name = dag_name_or_node_id
+        operation_id = node_id
+
+    if not isinstance(operation_id, str) or not operation_id:
+        raise ValueError("node_id must be a non-empty string")
+    if dag_name is not None and (not isinstance(dag_name, str) or not dag_name):
+        raise ValueError("dag_name must be a non-empty string")
+
+    def register(func: F) -> F:
+        if inspect.iscoroutinefunction(func):
+            raise TypeError("@dag_node does not support async functions")
+        if not inspect.isfunction(func):
+            raise TypeError("@dag_node can only decorate functions")
+        if func.__qualname__ != func.__name__:
+            raise TypeError("@dag_node can only decorate module-level functions")
+        key = (dag_name, operation_id)
+        with _nodes_lock:
+            if key in _nodes:
+                raise ValueError(f"Node ID '{operation_id}' is already registered")
+            _nodes[key] = func
+        return func
+
+    return register
+
+
+def get_registered_nodes(
+    dag_name: str | None = None,
+) -> dict[str, Callable[..., Any]]:
+    """Return a copy of the global registry or exactly one named-DAG scope."""
+    with _nodes_lock:
+        return {
+            node_id: func
+            for (scope, node_id), func in _nodes.items()
+            if scope == dag_name
+        }
+
+
+def get_node_ids(dag_name: str | None = None) -> list[str]:
+    """Return operation IDs in the global registry or named-DAG scope."""
+    with _nodes_lock:
+        return [node_id for scope, node_id in _nodes if scope == dag_name]
+
+
+def get_node(node_id: str, dag_name: str | None = None) -> Callable[..., Any] | None:
+    """Look up an operation by ID in the requested registry scope."""
+    with _nodes_lock:
+        return _nodes.get((dag_name, node_id))

lightweight_dag-0.1.1/src/lightweight_dag/suite.py

@@ -0,0 +1,80 @@
+"""Named DAG suite loading and validation."""
+
+from collections.abc import Mapping
+from typing import Any
+
+from pydantic import BaseModel, ConfigDict, field_validator, model_validator
+
+from lightweight_dag.dag import Dag, _FrozenDict, _load_yaml
+
+
+class DagSuite(BaseModel):
+    """A name-indexed collection of DAGs, accessible with ``suite[name]``."""
+
+    dags: dict[str, Dag]
+
+    model_config = ConfigDict(
+        extra="forbid",
+        frozen=True,
+    )
+
+    @field_validator("dags")
+    @classmethod
+    def freeze_dags(cls, value: dict[str, Dag]) -> dict[str, Dag]:
+        return _FrozenDict(value)
+
+    @model_validator(mode="after")
+    def check_dag_names_match_keys(self) -> "DagSuite":
+        """Ensure every suite key matches the corresponding DAG name."""
+        mismatched_names = {
+            suite_name: dag.name
+            for suite_name, dag in self.dags.items()
+            if suite_name != dag.name
+        }
+        if mismatched_names:
+            raise ValueError(f"Suite keys must match DAG names: {mismatched_names}")
+        return self
+
+    def __getitem__(self, name: str) -> Dag:
+        """Return the DAG registered under ``name``."""
+        return self.dags[name]
+
+    @classmethod
+    def from_dict(cls, data: Mapping[str, Mapping[str, Any] | Dag]) -> "DagSuite":
+        """Build a suite from a mapping of names to DAG definitions or models."""
+        if not isinstance(data, Mapping):
+            raise TypeError("DAG suite must be a mapping of names to DAG definitions")
+
+        dags = {}
+        for name, dag_data in data.items():
+            if not isinstance(name, str):
+                raise TypeError("DAG suite names must be strings")
+            if not name:
+                raise ValueError("DAG suite names must be non-empty strings")
+
+            if isinstance(dag_data, Dag):
+                if dag_data.name != name:
+                    raise ValueError(
+                        f"DAG name '{dag_data.name}' does not match suite key '{name}'"
+                    )
+                dags[name] = dag_data
+                continue
+
+            if not isinstance(dag_data, Mapping):
+                raise TypeError(
+                    f"DAG definition for '{name}' must be a mapping or a Dag"
+                )
+            configured_name = dag_data.get("name", name)
+            if configured_name != name:
+                raise ValueError(
+                    f"DAG name '{configured_name}' does not match suite key '{name}'"
+                )
+            dags[name] = Dag.model_validate({**dag_data, "name": name})
+
+        return cls(dags=dags)
+
+    @classmethod
+    def from_yaml(cls, yaml_str: str) -> "DagSuite":
+        """Parse a name-to-DAG mapping from YAML and validate each DAG."""
+        data = _load_yaml(yaml_str)
+        return cls.from_dict(data)