nextpipe 0.2.2.dev0__tar.gz → 0.3.0__tar.gz

{nextpipe-0.2.2.dev0 → nextpipe-0.3.0}/.gitignore

@@ -163,3 +163,6 @@ cython_debug/
 # Don't include key files
 key*.txt
 key*.json
+
+# Documentation stuff
+site/

nextpipe-0.3.0/.markdownlint.jsonc

@@ -0,0 +1,51 @@
+// Reference: https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint
+// Rules: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#rules
+//
+// Run markdown linting:
+// - Install [markdownlint-cli2](https://github.com/DavidAnson/markdownlint-cli2):
+// npm install markdownlint-cli2 --global
+// - Lint. Stand at the root of the repo and run:
+// markdownlint-cli2 "**/*.md"
+// - Ignore a certain rule for some lines:
+// <!-- markdownlint-disable MD013 -->
+// Your line here
+// <!-- markdownlint-enable MD013 -->
+{
+  // Use the default configurations as a starting point
+  "default": true,
+
+  // Max line length configurations
+  "MD013": {
+    "code_blocks": false,
+    "headings": false,
+    "line_length": 100,
+    "stern": false,
+    "strict": false,
+    "tables": false
+  },
+
+  // GitBook compatibility configurations
+  // Unordered lists use asterisks
+  "MD004": { "style": "asterisk" },
+
+  // First line in a file should be a top-level heading specifying a
+  // description, as such:
+  // ---
+  // description: You will learn the basics of getting started on cloud.
+  // ---
+  "MD041": { "level": 1, "front_matter_title": "^\\s*description\\s*[:=]" },
+
+  // There can only be a single top-level heading, which should be the YAML
+  // front matter containing the "title" property.
+  "MD025": { "level": 1, "front_matter_title": "^\\s*title\\s*[:=]" },
+
+  // Don't enforce heading level continuity.
+  "MD001": false,
+
+  // Use spaces as opposed to hard-tabs for consistency.
+  "MD010": { "spaces_per_tab": 4 },
+
+  // Material for mkdocs has admonitions which are indented by 4 spaces and not
+  // fenced.
+  "MD046": false
+}

nextpipe-0.3.0/.readthedocs.yaml

@@ -0,0 +1,19 @@
+# Read the Docs configuration file for MkDocs projects
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+mkdocs:
+  configuration: mkdocs.yml
+
+# Optionally declare the Python requirements required to build your docs
+python:
+  install:
+    - requirements: docs/requirements.txt

{nextpipe-0.2.2.dev0 → nextpipe-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nextpipe
-Version: 0.2.2.dev0
+Version: 0.3.0
 Summary: Framework for Decision Pipeline modeling and execution
 Project-URL: Homepage, https://www.nextmv.io
 Project-URL: Documentation, https://www.nextmv.io/docs
@@ -105,26 +105,65 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.9
 Requires-Dist: dataclasses-json>=0.6.7
-Requires-Dist: nextmv>=0.25.0
+Requires-Dist: nextmv>=0.28.0
 Requires-Dist: requests>=2.31.0
 Provides-Extra: dev
 Requires-Dist: goldie>=0.1.7; extra == 'dev'
 Requires-Dist: ruff>=0.11.6; extra == 'dev'
 Description-Content-Type: text/markdown
 
-# nextpipe
-
-Framework for Decision Pipeline modeling and execution.
+# Nextpipe
+
+<!-- markdownlint-disable MD033 MD013 -->
+
+<p align="center">
+  <a href="https://nextmv.io"><img src="https://cdn.prod.website-files.com/60dee0fad10d14c8ab66dd74/65c66addcd07eed09be35114_blog-banner-what-is-cicd-for-decision-science-p-2000.jpeg" alt="Nextmv" width="45%"></a>
+</p>
+<p align="center">
+    <em>Nextmv: The home for all your optimization work</em>
+</p>
+<p align="center">
+<a href="https://github.com/nextmv-io/nextpipe/actions/workflows/test.yml" target="_blank">
+    <img src="https://github.com/nextmv-io/nextpipe/actions/workflows/test.yml/badge.svg?event=push&branch=develop" alt="Test">
+</a>
+<a href="https://github.com/nextmv-io/nextpipe/actions/workflows/lint.yml" target="_blank">
+    <img src="https://github.com/nextmv-io/nextpipe/actions/workflows/lint.yml/badge.svg?event=push&branch=develop" alt="Lint">
+</a>
+<a href="https://pypi.org/project/nextpipe" target="_blank">
+    <img src="https://img.shields.io/pypi/v/nextpipe?color=%2334D058&label=nextpipe" alt="Package version">
+</a>
+<a href="https://pypi.org/project/nextpipe" target="_blank">
+    <img src="https://img.shields.io/pypi/pyversions/nextpipe.svg?color=%2334D058" alt="Supported Python versions">
+</a>
+</p>
+
+<!-- markdownlint-enable MD033 MD013 -->
+
+Nextpipe is a Python package that provides a framework for Decision Workflows
+modeling and execution. It provides first-class support for Workflows in the
+[Nextmv Platform][nextmv].
+
+> [!IMPORTANT]  
+> Please note that `nextpipe` is provided as _source-available_ software
+> (not _open-source_). For further information, please refer to the
+> [LICENSE](./LICENSE.md) file.
+
+📖 To learn more about the `nextpipe`, visit the [docs][docs].
 
 ## Installation
 
+The package is hosted on [PyPI][nextpipe-pypi]. Python `>=3.9` is required.
+
+Install via `pip`:
+
 ```bash
 pip install nextpipe
 ```
 
 ## Preview
 
-Example of a pipeline utilizing multiple routing solvers, and picking the best result.
+Example of a pipeline utilizing multiple routing solvers, and picking the best
+result.
 
 ```mermaid
 graph LR
@@ -151,54 +190,6 @@ graph LR
   pick_best(pick_best)
 ```
 
-## Modeling
-
-Pipeline modeling is done by defining a flow as steps and decorating them.
-The following example demonstrates a 3 step pipeline that makes use of dynamic fanout and joining of results.
-
-```python
-from nextpipe import AppOption, AppRunConfig, FlowSpec, app, foreach, join, needs, step
-
-
-class Flow(FlowSpec):
-    @foreach()  # Run the successor step for each item in the result list of this step
-    @step
-    def prepare(data: dict):
-        """
-        Creates 3 copies of the input and configures them for 3 different app parameters.
-        """
-        inputs = [copy.deepcopy(data) for _ in range(3)]
-        run_configs = [AppRunConfig(input, [AppOption("param", i)]) for i, input in enumerate(inputs)]
-        return run_configs
-
-    @app(app_id="echo")
-    @needs(predecessors=[prepare])
-    @step
-    def solve():
-        """
-        Imitates a solver app.
-        """
-        pass
-
-    @needs(predecessors=[solve])
-    @join()  # Collect the results from the previous 'foreach' step and combine them into a list passed as the arg
-    @step
-    def merge(results: list[dict]):
-        """Merges the results."""
-        return results
-```
-
-The pipeline can simply be executed from the main func of the app.
-
-```python
-# Run workflow
-flow = Flow("DecisionFlow", input.data)
-flow.run()
-
-# Write out the result
-print(json.dumps(flow.get_result(flow.merge)))
-```
-
-## Examples
-
-You can find further examples of how to use `nextpipe` in the [examples](./examples) directory.
+[nextpipe-pypi]: https://pypi.org/project/nextpipe/
+[nextmv]: https://nextmv.io
+[docs]: https://nextpipe.docs.nextmv.io/en/latest/

nextpipe-0.3.0/README.md

@@ -0,0 +1,81 @@
+# Nextpipe
+
+<!-- markdownlint-disable MD033 MD013 -->
+
+<p align="center">
+  <a href="https://nextmv.io"><img src="https://cdn.prod.website-files.com/60dee0fad10d14c8ab66dd74/65c66addcd07eed09be35114_blog-banner-what-is-cicd-for-decision-science-p-2000.jpeg" alt="Nextmv" width="45%"></a>
+</p>
+<p align="center">
+    <em>Nextmv: The home for all your optimization work</em>
+</p>
+<p align="center">
+<a href="https://github.com/nextmv-io/nextpipe/actions/workflows/test.yml" target="_blank">
+    <img src="https://github.com/nextmv-io/nextpipe/actions/workflows/test.yml/badge.svg?event=push&branch=develop" alt="Test">
+</a>
+<a href="https://github.com/nextmv-io/nextpipe/actions/workflows/lint.yml" target="_blank">
+    <img src="https://github.com/nextmv-io/nextpipe/actions/workflows/lint.yml/badge.svg?event=push&branch=develop" alt="Lint">
+</a>
+<a href="https://pypi.org/project/nextpipe" target="_blank">
+    <img src="https://img.shields.io/pypi/v/nextpipe?color=%2334D058&label=nextpipe" alt="Package version">
+</a>
+<a href="https://pypi.org/project/nextpipe" target="_blank">
+    <img src="https://img.shields.io/pypi/pyversions/nextpipe.svg?color=%2334D058" alt="Supported Python versions">
+</a>
+</p>
+
+<!-- markdownlint-enable MD033 MD013 -->
+
+Nextpipe is a Python package that provides a framework for Decision Workflows
+modeling and execution. It provides first-class support for Workflows in the
+[Nextmv Platform][nextmv].
+
+> [!IMPORTANT]  
+> Please note that `nextpipe` is provided as _source-available_ software
+> (not _open-source_). For further information, please refer to the
+> [LICENSE](./LICENSE.md) file.
+
+📖 To learn more about the `nextpipe`, visit the [docs][docs].
+
+## Installation
+
+The package is hosted on [PyPI][nextpipe-pypi]. Python `>=3.9` is required.
+
+Install via `pip`:
+
+```bash
+pip install nextpipe
+```
+
+## Preview
+
+Example of a pipeline utilizing multiple routing solvers, and picking the best
+result.
+
+```mermaid
+graph LR
+  fetch_data(prepare_data)
+  fetch_data --> run_nextroute
+  fetch_data --> run_ortools
+  fetch_data --> run_pyvroom
+  run_nextroute{ }
+  run_nextroute_join{ }
+  run_nextroute_0(run_nextroute_0)
+  run_nextroute --> run_nextroute_0
+  run_nextroute_0 --> run_nextroute_join
+  run_nextroute_1(run_nextroute_1)
+  run_nextroute --> run_nextroute_1
+  run_nextroute_1 --> run_nextroute_join
+  run_nextroute_2(run_nextroute_2)
+  run_nextroute --> run_nextroute_2
+  run_nextroute_2 --> run_nextroute_join
+  run_nextroute_join --> pick_best
+  run_ortools(run_ortools)
+  run_ortools --> pick_best
+  run_pyvroom(run_pyvroom)
+  run_pyvroom --> pick_best
+  pick_best(pick_best)
+```
+
+[nextpipe-pypi]: https://pypi.org/project/nextpipe/
+[nextmv]: https://nextmv.io
+[docs]: https://nextpipe.docs.nextmv.io/en/latest/

nextpipe-0.3.0/docs/examples/basic-chained-workflow.md

@@ -0,0 +1,123 @@
+# Basic Chained Workflow Example
+
+!!! tip
+
+    This example uses the [`echo` app](../tutorials/echo.md), make sure to complete
+    that tutorial first.
+
+This is a basic example of a three-step, sequential workflow. It demonstrates
+the basic structure of a workflow, which consists of steps being chained
+together. Each step can depend on the output of the previous step.
+
+```python
+from typing import Any
+
+import nextmv
+
+from nextpipe import FlowSpec, app, needs, step
+
+
+class Workflow(FlowSpec):
+    @step
+    def prepare(input: dict[str, Any]) -> dict[str, Any]:
+        """Prepares the data."""
+
+        input["prepared"] = True
+
+        return input
+
+    @app(app_id="echo")
+    @needs(predecessors=[prepare])
+    @step
+    def solve() -> dict[str, Any]:
+        """Runs the model."""
+        pass
+
+    @needs(predecessors=[solve])
+    @step
+    def enhance(result: dict[str, Any]) -> dict[str, Any]:
+        """Enhances the result."""
+
+        output = result["solution"]
+        output["enhanced"] = True
+
+        return output
+
+
+def main():
+    """Runs the workflow."""
+
+    # Load input data
+    input = nextmv.load()
+
+    # Run workflow
+    workflow = Workflow("DecisionWorkflow", input.data)
+    workflow.run()
+
+    # Write the result
+    result = workflow.get_result(workflow.enhance)
+    nextmv.write(result)
+
+
+if __name__ == "__main__":
+    main()
+```
+
+Run the example:
+
+```bash
+$ echo '{"hello": "world!"}' | python main.py
+[nextpipe] No application ID or run ID found, uplink is inactive.
+[nextpipe] Flow: Workflow
+[nextpipe] nextpipe: v0.2.2.dev0
+[nextpipe] nextmv: 0.28.0
+[nextpipe] Flow graph steps:
+[nextpipe] Step:
+[nextpipe]   Definition: Step(prepare)
+[nextpipe]   Docstring: Prepares the data.
+[nextpipe] Step:
+[nextpipe]   Definition: Step(solve, StepNeeds(prepare), StepRun(echo, devint, {}, InputType.JSON, False))
+[nextpipe]   Docstring: Runs the model.
+[nextpipe] Step:
+[nextpipe]   Definition: Step(enhance, StepNeeds(solve))
+[nextpipe]   Docstring: Enhances the result.
+[nextpipe] Mermaid diagram:
+[nextpipe] graph LR
+  prepare(prepare)
+  prepare --> solve
+  solve(solve)
+  solve --> enhance
+  enhance(enhance)
+
+[nextpipe] Mermaid URL: https://mermaid.ink/svg/Z3JhcGggTFIKICBwcmVwYXJlKHByZXBhcmUpCiAgcHJlcGFyZSAtLT4gc29sdmUKICBzb2x2ZShzb2x2ZSkKICBzb2x2ZSAtLT4gZW5oYW5jZQogIGVuaGFuY2UoZW5oYW5jZSkK?theme=dark
+[nextpipe] Running node prepare_0
+[nextpipe] Running node solve_0
+[nextpipe] Running node enhance_0
+{
+  "echo": {
+    "data": {
+      "hello": "world!",
+      "prepared": true
+    },
+    "args": []
+  },
+  "enhanced": true
+}
+```
+
+As you can observe from the output, `nextpipe` automatically generates a
+Mermaid diagram to visualize the flow structure.
+
+```mermaid
+graph LR
+  prepare(prepare)
+  prepare --> solve
+  solve(solve)
+  solve --> enhance
+  enhance(enhance)
+```
+
+The diagram can be viewed in a browser by following the [link][mermaid-link]
+provided in the output. The diagram shows the flow of data between the steps.
+
+[mermaid-link]: https://mermaid.ink/svg/Z3JhcGggTFIKICBwcmVwYXJlKHByZXBhcmUpCiAgcHJlcGFyZSAtLT4gc29sdmUKICBzb2x2ZShzb2x2ZSkKICBzb2x2ZSAtLT4gZW5oYW5jZQogIGVuaGFuY2UoZW5oYW5jZSkK?theme=dark