flowrep 0.2.0__tar.gz → 0.4.0__tar.gz

{flowrep-0.2.0 → flowrep-0.4.0}/.gitignore

@@ -11,3 +11,4 @@ apidoc/
 .ipynb_checkpoints/
 test_times.dat
 core.*
+src/flowrep/_version.py

flowrep-0.4.0/PKG-INFO

@@ -0,0 +1,379 @@
+Metadata-Version: 2.4
+Name: flowrep
+Version: 0.4.0
+Summary: flowrep - Your premier tool for workflow representations
+Project-URL: Homepage, https://pyiron.org/
+Project-URL: Documentation, https://flowrep.readthedocs.io
+Project-URL: Repository, https://github.com/pyiron/flowrep
+Author-email: Sam Waseda <waseda@mpie.de>, Liam Huber <liamhuber@greyhavensolutions.com>
+License: BSD 3-Clause License
+        
+        Copyright (c) 2024, Max-Planck-Institut für Nachhaltige Materialien GmbH - Computational Materials Design (CM) Department
+        All rights reserved.
+        
+        Redistribution and use in source and binary forms, with or without
+        modification, are permitted provided that the following conditions are met:
+        
+        * Redistributions of source code must retain the above copyright notice, this
+          list of conditions and the following disclaimer.
+        
+        * Redistributions in binary form must reproduce the above copyright notice,
+          this list of conditions and the following disclaimer in the documentation
+          and/or other materials provided with the distribution.
+        
+        * Neither the name of the copyright holder nor the names of its
+          contributors may be used to endorse or promote products derived from
+          this software without specific prior written permission.
+        
+        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+License-File: LICENSE
+Keywords: pyiron
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: <3.15,>=3.11
+Requires-Dist: pydantic<2.13.0,>=2.12.0
+Requires-Dist: pyiron-snippets<2.0.0,>=1.2.1
+Provides-Extra: notebooks
+Requires-Dist: numpy==2.4.3; extra == 'notebooks'
+Provides-Extra: pwd
+Requires-Dist: python-workflow-definition==0.1.4; extra == 'pwd'
+Provides-Extra: storage
+Requires-Dist: bagofholding==0.1.10; extra == 'storage'
+Provides-Extra: storage-widget
+Requires-Dist: bagofholding==0.1.10; extra == 'storage-widget'
+Requires-Dist: ipytree==0.2.2; extra == 'storage-widget'
+Description-Content-Type: text/markdown
+
+# flowrep
+
+[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb)
+[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
+[![Coverage](https://codecov.io/gh/pyiron/flowrep/graph/badge.svg)](https://codecov.io/gh/pyiron/flowrep)
+[![Documentation](https://readthedocs.org/projects/flowrep/badge/?version=latest)](https://flowrep.readthedocs.io/en/latest/?badge=latest)
+
+[![Anaconda](https://anaconda.org/conda-forge/flowrep/badges/version.svg)](https://anaconda.org/conda-forge/flowrep)
+[![Last Updated](https://anaconda.org/conda-forge/flowrep/badges/latest_release_date.svg)](https://anaconda.org/conda-forge/flowrep)
+[![Platform](https://anaconda.org/conda-forge/flowrep/badges/platforms.svg)](https://anaconda.org/conda-forge/flowrep)
+[![Downloads](https://anaconda.org/conda-forge/flowrep/badges/downloads.svg)](https://anaconda.org/conda-forge/flowrep)
+
+
+## flowrep — Workflow Recipes from Python
+
+**flowrep** turns plain Python functions into shareable, versionable *workflow
+recipes* — JSON-serialisable graphs that describe *what* to compute (which
+functions, how they connect) without doing the computation or holding any data.
+Recipes are prospective blueprints that a Workflow Management System (WfMS) can
+digest, visualise, and execute.
+
+Flowchart-style representations are already the lingua franca for describing
+processes in science and engineering. flowrep gives you a way to author them in
+Python and pass them around as simple JSON, validated by
+[Pydantic](https://docs.pydantic.dev/) and enriched with version provenance so
+they stay robust through time.
+
+
+## Installation
+
+```bash
+conda install -c conda-forge flowrep   # recommended
+pip install flowrep                     # also available from PyPI
+```
+
+
+## Quick Start
+
+Define a couple of simple functions — it will be helpful to make our return statements 
+nicely named variables, but it's not a necessity:
+
+```python
+>>> import flowrep as fr
+
+>>> def add(a, b):
+...     return a + b
+
+>>> def multiply(x, y):
+...     product = x * y
+...     return product
+
+```
+
+Compose them into a workflow with the `@workflow` decorator.  The body reads like 
+normal Python — assign function calls to variables, wire outputs into inputs, 
+return results:
+
+```python
+>>> @fr.workflow
+... def linear(x, slope, intercept):
+...     """y = slope * x + intercept"""
+...     scaled = multiply(x, slope)
+...     result = add(scaled, intercept)
+...     return result
+
+```
+
+The decorated function is still just its usual callable self:
+
+```python
+>>> linear(3, 2, 1)
+7
+
+```
+
+But it now carries a recipe on its `flowrep_recipe` attribute — a pure-data
+Pydantic model whose structure you can inspect directly
+(`model_dump(mode="json")` gives a Python dict, `model_dump_json()` gives a
+JSON string):
+
+```python
+>>> recipe = linear.flowrep_recipe.model_dump(mode="json")
+>>> recipe["type"]
+'workflow'
+>>> recipe["inputs"]
+['x', 'slope', 'intercept']
+>>> recipe["outputs"]
+['result']
+>>> sorted(recipe["nodes"])
+['add_0', 'multiply_0']
+>>> recipe["nodes"]["multiply_0"]["outputs"]
+['product']
+>>> recipe["edges"]
+{'add_0.a': 'multiply_0.product'}
+>>> recipe["output_edges"]
+{'result': 'add_0.output_0'}
+>>> recipe["nodes"]["add_0"]["reference"]["info"]["qualname"]
+'add'
+
+```
+
+A key difference between a workflow graph and typical python is that for a graph we 
+need named handles for all of our output values.
+We can see in the `edges` that the recipe nicely used our return variable to give 
+the "multiply" a nice output label.
+For our "add" node, what we returned couldn't be parsed nicely as a label, so in 
+`output_edges` we see it got assigned a default name `"output_0"`.
+
+Every `"atomic"` node for running a Python function and every `"workflow"` not created 
+by parsing a function defition carries a `reference` recording for the Python function
+to which it maps (module, qualname, and package version when available), so a WfMS can resolve
+and execute them later. The recipe also captures exactly how data flows: 
+`input_edges` wire workflow inputs to child node ports, `edges` connect sibling
+outputs to inputs, and `output_edges` name which child port produces each 
+workflow output.
+
+The recipe *is* the shareable artifact. It could equally have been authored in a
+GUI editor or generated by another tool — flowrep doesn't care where it came
+from, only that it validates. (The availability of referenced functions on your
+machine, and their actual behaviour, lie outside flowrep's responsibility. We
+simply check that the recipe is internally consistent.)
+
+
+## Features
+
+**Pure Python foundation.** Knowing basic Python is enough to get started. The
+decorators `@atomic` and `@workflow` (plus their functional counterparts
+`parse_atomic` and `parse_workflow`) handle the heavy lifting; the recipe format
+is plain JSON.
+
+**Pydantic validation.** Recipes are full Pydantic models — they validate on
+construction, serialise with `model_dump_json()`, and deserialise with
+`model_validate_json()`.
+
+**Version provenance.** Every node reference carries module, qualname, and
+(optionally) package version metadata. Constraints like `forbid_main`,
+`forbid_locals`, and `require_version` can be enforced at parse time to ensure
+recipes only reference published, versioned code.
+
+**Composability.** Workflows nest: a `@workflow`-decorated function can call
+another `@workflow`-decorated function, and the child's recipe is embedded as a
+sub-graph. Build complex pipelines from small, testable pieces.
+
+**Flow control.** Recipes go beyond simple DAGs. The `@workflow` parser
+recognises `for`, `while`, `if`/`elif`/`else`, and `try`/`except` — the same
+control structures you use in real code. Flow control nodes are inherently
+dynamic: their exact execution path depends on data and cannot be known until
+run-time, but their IO signature is always fully known a priori.
+
+### Example: flow control and nesting
+
+So far we've seen `"workflow"` nodes, and alluded to `"atomic"` nodes.
+You can also explicitly attach an atomic recipe to a function definition, but most of 
+the time you won't need to since the workflow parser auto-parses undecorated function 
+calls as atomic nodes.
+
+On the other hand, we can parse `@workflow`-decorated functions _as workflows_ to 
+compose complex workflows with nested subgraphs.
+
+Beyond simple `"atomic"` and DAG `"workflow"` nodes, there are also recipe formats 
+for flow control structures like `while`. 
+Just like the other nodes, these can be written directly as JSON, or parsed inside a 
+workflow from a python function definition -- with some syntax restrictions.
+This is covered in more detail in the [user guide](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb), but here let's see both of these features 
+in use at once by nesting a while loop inside a sub-workflow.
+Let's look at a `while` loop, where our main syntax restriction is that the condition 
+must be a function call:
+
+```python
+>>> def is_less_than_target(value, target):
+...     result = value < target
+...     return result
+
+>>> @fr.atomic  # This is optional, but doesn't hurt us
+... def double(x):
+...     doubled = x * 2
+...     return doubled
+
+>>> @fr.workflow
+... def double_until(x, target):
+...     """Repeatedly double `x` until it reaches `target`."""
+...     while is_less_than_target(x, target):
+...         x = double(x)
+...     return x
+
+>>> double_until(3, 40)
+48
+
+```
+
+And then we can compose workflows by nesting:
+
+```python
+>>> @fr.workflow
+... def double_and_add(a, b, target):
+...     big_a = double_until(a, target)
+...     result = add(big_a, b)
+...     return result
+
+>>> double_and_add(3, 100, 40)
+148
+
+```
+
+The resulting recipe captures the full structure — the while-loop, the nested
+workflow, and all the edges between them — as a single pydantic model that can be dumped 
+to a JSON document.
+
+We can see the layers of nested subgraphs:
+
+```python
+>>> recipe_json = double_and_add.flowrep_recipe.model_dump(mode="json")
+>>> [child for child in recipe_json["nodes"]]
+['double_until_0', 'add_0']
+
+>>> [nested_child for nested_child in recipe_json["nodes"]["double_until_0"]["nodes"]]
+['while_0']
+
+```
+
+Although the while-node is dynamic -- and therefore we _can't_ know its exact nodes 
+until run-time, it must and does still have well-defined IO signature.
+We can look at the template it will follow, e.g., by peeking at the part of the recipe 
+used for the "while" condition:
+
+```python
+>>> recipe_json["nodes"]["double_until_0"]["nodes"]["while_0"]["case"]["condition"]["node"]["type"]
+'atomic'
+
+>>> recipe_json["nodes"]["double_until_0"]["nodes"]["while_0"]["case"]["condition"]["node"]["reference"]["info"]["qualname"]
+'is_less_than_target'
+
+```
+
+And to see how it will forward it's inputs down into its prospective subgraph:
+
+```python
+>>> recipe_json["nodes"]["double_until_0"]["nodes"]["while_0"]["input_edges"]
+{'condition.value': 'x', 'condition.target': 'target', 'body.x': 'x'}
+
+```
+
+We run these in the examples above to show two things: first, that even when nested, 
+the decorated functions are still just python functions; second, to show in the following 
+section that the recipe we parse from this are intended to give the same result as 
+these underlying functions when we run the recipe with a WfMS.
+
+## Beyond Recipes: Live Data and Execution
+
+Recipes are *prospective* — they describe a computation template without holding
+data. For retrospective analysis (inspecting what actually happened during a
+run), flowrep provides two additional layers accessible through the API:
+
+```python
+>>> from flowrep.api import tools as frt
+
+```
+
+**`flowrep.api.tools.recipe2live`** converts a recipe into a *live* object — a mutable
+data structure whose input and output ports can hold actual Python values. Live
+objects mirror the recipe graph but trade JSON-serializability for the ability to
+carry arbitrary data:
+
+```python
+>>> live_wf = frt.recipe2live(double_and_add.flowrep_recipe)
+
+```
+
+**`flowrep.api.tools.run_recipe`** goes one step further: it executes the recipe with
+the provided inputs and returns a fully populated live object. This is powered
+by a minimal, built-in WfMS intended as a reference implementation and for use
+in tests and documentation (like this!):
+
+```python
+>>> retrospective = frt.run_recipe(
+...     double_and_add.flowrep_recipe, a=3, b=100, target=40
+... )
+>>> retrospective.output_ports["result"].value
+148
+
+```
+
+Because every child node's ports are populated too, the live graph gives you
+full data provenance — you can walk the tree and inspect exactly what each node
+received and produced.
+For flow control nodes, which are _prospectively_ "black boxes", we find that 
+_retrospectively_ they are simple DAGs.
+For our while-loop, that means that we can retrospectively inspect the provenance of 
+each loop iteration:
+
+```python
+>>> while_loop = retrospective.nodes["double_until_0"].nodes["while_0"]
+>>> while_loop.nodes["body_0"].output_ports["x"].value
+6
+>>> while_loop.nodes["body_1"].output_ports["x"].value
+12
+
+```
+
+For a deeper look at all available node types, edge semantics, version
+provenance, and the live/WfMS layer, see the
+[user guide](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb).
+
+
+## Documentation
+
+- The user guide notebook comprehensively covers all node types, edge models, flow control, versioning, 
+  live/retrospective data formats, a demo WfMS implementation, and recipe format converters. 
+  Launch it interactively on
+  [mybinder](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb).
+- [readthedocs](https://flowrep.readthedocs.io/en/latest/)
+
+
+## Contributing
+
+Contributions are welcome! Please see
+[`CODE_OF_CONDUCT.md`](../CODE_OF_CONDUCT.md) for community guidelines.