kpipeliner 1.12.1__tar.gz

kpipeliner-1.12.1/.gitignore

@@ -0,0 +1,11 @@
+.venv/
+*.pyc
+.claude/
+__pycache__/
+/dist/
+*.egg-info/
+graphify-out/
+milestones.md
+decisions.md
+CLAUDE.md
+Readme

kpipeliner-1.12.1/PKG-INFO

@@ -0,0 +1,121 @@
+Metadata-Version: 2.5
+Name: kpipeliner
+Version: 1.12.1
+Summary: kpipeliner (kpl) is a node-based pipeline manager.
+Project-URL: Repository, https://github.com/Kaptura-GmbH/kpipeliner
+Keywords: automation,nodes,pipeline,workflow
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Requires-Dist: fastapi>=0.111.0
+Requires-Dist: psutil>=5.9
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: tqdm>=4.65
+Requires-Dist: uvicorn[standard]>=0.30.0
+Requires-Dist: websockets>=12.0
+Provides-Extra: all
+Requires-Dist: kwebui; (python_version >= '3.12') and extra == 'all'
+Requires-Dist: mcp>=2.0.0; extra == 'all'
+Provides-Extra: kwui
+Requires-Dist: kwebui; (python_version >= '3.12') and extra == 'kwui'
+Provides-Extra: mcp
+Requires-Dist: mcp>=2.0.0; extra == 'mcp'
+Description-Content-Type: text/markdown
+
+# kpipeliner
+
+A node-based pipeline manager. A pipeline is a graph of nodes; each node is a
+standalone Python program (its own subprocess, optionally its own
+venv/conda/pixi environment) with typed inputs and outputs declared in YAML.
+Build and run pipelines from the CLI (`kpl`), a visual web GUI, or hand them
+over to an LLM through MCP.
+
+![kpipeliner GUI](docs/wiki/images/gui-overview.png)
+
+## Highlights
+
+- **Graph = YAML.** Connections are just strings (`nodes.<name>.<output>`) —
+  no hidden edge list, easy to read, easy to script.
+- **Subprocess isolation.** Every node runs on its own, with its own Python
+  environment if it needs one; a crash in one node never takes down another.
+- **Caching by default.** Node outputs are cached per run-UUID; re-running a
+  pipeline only recomputes what actually changed.
+- **CLI + GUI + MCP.** Script it, click through it in the browser, or connect
+  an MCP client and let an LLM design/inspect/run it for you.
+- **Groups & reusable components.** Bundle nodes in the GUI and export them as
+  drag-and-drop components for other pipelines.
+
+## Install
+
+kpipeliner is a normal Python package — installing it is the whole installation
+(dependencies, the prebuilt web GUI and the `kpl` command; no Node.js, no installer
+script). With [uv](https://docs.astral.sh/uv/):
+
+```bash
+uv tool install kpipeliner          # `kpl` ends up on your PATH
+kpl --help
+```
+
+or into an environment of your own: `uv pip install kpipeliner` (or `pip install kpipeliner`).
+
+Optional extras — nothing imports them unless you use the feature, so leaving them out is fine:
+
+```bash
+uv tool install "kpipeliner[mcp]"    # `kpl mcp` / `kpl gui --mcp` (LLM client access)
+uv tool install "kpipeliner[kwui]"   # `kpl run --kwui` (a web UI per node)
+uv tool install "kpipeliner[all]"    # both
+```
+
+Configuration lives in a `config.yaml` that ships with the package; the first run copies it
+to `~/.config/kpipeliner/config.yaml` (or `$KPL_HOME/config.yaml`) so your edits survive
+upgrades. `kpl config` prints its location.
+
+**Developing kpipeliner itself:** clone the repository and run `uv pip install -e .` (or
+`uv sync --all-extras`) — an editable install; the checkout's own `config.yaml` is used.
+To build the package: `uv build` (rebuild the GUI first with `npm run build` in
+`kpipeline_gui/frontend` if the frontend changed — the built `dist/` is what ships).
+
+## Quick start
+
+```bash
+mkdir my_pipeline && cd my_pipeline
+kpl init                 # creates graph_config.yaml, data/, cache/
+kpl list                 # see available node types
+kpl add <node_type>      # add a node instance
+kpl tree                 # inspect the graph
+kpl run                  # run it (or: kpl gui to do all of this visually)
+```
+
+## Commands
+
+| Command | Description |
+|---|---|
+| `kpl init` | Initialize a pipeline in the current directory |
+| `kpl gui` | Launch the web GUI (port 8601+) |
+| `kpl gui --mcp` | GUI + an MCP endpoint at `/mcp`, same port |
+| `kpl mcp` | Headless MCP server only — no GUI/REST API |
+| `kpl list` | List available node types |
+| `kpl add <type> [name]` | Add a node instance to the graph |
+| `kpl run [node]` | Run the declared outputs (or one node) + dependencies |
+| `kpl run_debug [node]` | Same, with full log output |
+| `kpl run --kwui` | Run behind a per-node web UI (kwebui) |
+| `kpl tree` | Print the graph with dependencies and I/O |
+| `kpl status` | Show cached/frozen state per node |
+| `kpl clean [node]` | Clear cache (all nodes, or one) |
+| `kpl kwui init` | Create `workflow.py` for kwui mode in this pipeline |
+| `kpl base_node create` | Scaffold a new node type in the current directory |
+| `kpl base_node install [--node_name N]` | Install the current directory as a node type |
+| `kpl base_node uninstall <name>` | Remove an installed node type |
+| `kpl node_env install [--node_type T]` | Install a node's Python environment |
+| `kpl uninstall` | Remove kpl from your PATH |
+
+Run any command with `--help` for its full option list.
+
+## Documentation
+
+- **[User Guide](docs/wiki/User-Guide.md)** — the full walkthrough: CLI and
+  GUI workflows, groups, custom nodes, MCP, kwui mode, and the
+  `graph_config.yaml`/`node_config.yaml`/`config.yaml` reference.
+- **[kpipeliner.md](kpipeliner.md)** — a self-contained agent guide for
+  building pipelines/nodes (also installed as a Claude Code skill).
+- **[Readme](Readme)** — version history / changelog.

kpipeliner-1.12.1/README.md

@@ -0,0 +1,97 @@
+# kpipeliner
+
+A node-based pipeline manager. A pipeline is a graph of nodes; each node is a
+standalone Python program (its own subprocess, optionally its own
+venv/conda/pixi environment) with typed inputs and outputs declared in YAML.
+Build and run pipelines from the CLI (`kpl`), a visual web GUI, or hand them
+over to an LLM through MCP.
+
+![kpipeliner GUI](docs/wiki/images/gui-overview.png)
+
+## Highlights
+
+- **Graph = YAML.** Connections are just strings (`nodes.<name>.<output>`) —
+  no hidden edge list, easy to read, easy to script.
+- **Subprocess isolation.** Every node runs on its own, with its own Python
+  environment if it needs one; a crash in one node never takes down another.
+- **Caching by default.** Node outputs are cached per run-UUID; re-running a
+  pipeline only recomputes what actually changed.
+- **CLI + GUI + MCP.** Script it, click through it in the browser, or connect
+  an MCP client and let an LLM design/inspect/run it for you.
+- **Groups & reusable components.** Bundle nodes in the GUI and export them as
+  drag-and-drop components for other pipelines.
+
+## Install
+
+kpipeliner is a normal Python package — installing it is the whole installation
+(dependencies, the prebuilt web GUI and the `kpl` command; no Node.js, no installer
+script). With [uv](https://docs.astral.sh/uv/):
+
+```bash
+uv tool install kpipeliner          # `kpl` ends up on your PATH
+kpl --help
+```
+
+or into an environment of your own: `uv pip install kpipeliner` (or `pip install kpipeliner`).
+
+Optional extras — nothing imports them unless you use the feature, so leaving them out is fine:
+
+```bash
+uv tool install "kpipeliner[mcp]"    # `kpl mcp` / `kpl gui --mcp` (LLM client access)
+uv tool install "kpipeliner[kwui]"   # `kpl run --kwui` (a web UI per node)
+uv tool install "kpipeliner[all]"    # both
+```
+
+Configuration lives in a `config.yaml` that ships with the package; the first run copies it
+to `~/.config/kpipeliner/config.yaml` (or `$KPL_HOME/config.yaml`) so your edits survive
+upgrades. `kpl config` prints its location.
+
+**Developing kpipeliner itself:** clone the repository and run `uv pip install -e .` (or
+`uv sync --all-extras`) — an editable install; the checkout's own `config.yaml` is used.
+To build the package: `uv build` (rebuild the GUI first with `npm run build` in
+`kpipeline_gui/frontend` if the frontend changed — the built `dist/` is what ships).
+
+## Quick start
+
+```bash
+mkdir my_pipeline && cd my_pipeline
+kpl init                 # creates graph_config.yaml, data/, cache/
+kpl list                 # see available node types
+kpl add <node_type>      # add a node instance
+kpl tree                 # inspect the graph
+kpl run                  # run it (or: kpl gui to do all of this visually)
+```
+
+## Commands
+
+| Command | Description |
+|---|---|
+| `kpl init` | Initialize a pipeline in the current directory |
+| `kpl gui` | Launch the web GUI (port 8601+) |
+| `kpl gui --mcp` | GUI + an MCP endpoint at `/mcp`, same port |
+| `kpl mcp` | Headless MCP server only — no GUI/REST API |
+| `kpl list` | List available node types |
+| `kpl add <type> [name]` | Add a node instance to the graph |
+| `kpl run [node]` | Run the declared outputs (or one node) + dependencies |
+| `kpl run_debug [node]` | Same, with full log output |
+| `kpl run --kwui` | Run behind a per-node web UI (kwebui) |
+| `kpl tree` | Print the graph with dependencies and I/O |
+| `kpl status` | Show cached/frozen state per node |
+| `kpl clean [node]` | Clear cache (all nodes, or one) |
+| `kpl kwui init` | Create `workflow.py` for kwui mode in this pipeline |
+| `kpl base_node create` | Scaffold a new node type in the current directory |
+| `kpl base_node install [--node_name N]` | Install the current directory as a node type |
+| `kpl base_node uninstall <name>` | Remove an installed node type |
+| `kpl node_env install [--node_type T]` | Install a node's Python environment |
+| `kpl uninstall` | Remove kpl from your PATH |
+
+Run any command with `--help` for its full option list.
+
+## Documentation
+
+- **[User Guide](docs/wiki/User-Guide.md)** — the full walkthrough: CLI and
+  GUI workflows, groups, custom nodes, MCP, kwui mode, and the
+  `graph_config.yaml`/`node_config.yaml`/`config.yaml` reference.
+- **[kpipeliner.md](kpipeliner.md)** — a self-contained agent guide for
+  building pipelines/nodes (also installed as a Claude Code skill).
+- **[Readme](Readme)** — version history / changelog.

kpipeliner-1.12.1/kpipeline/__init__.py

@@ -0,0 +1,12 @@
+from kpipeline.src.kparameter import KParameter
+from kpipeline.src.knode import KNode, NodeState
+from kpipeline.src.kgraph import KGraph
+from kpipeline.src.executor import PipelineExecutor
+
+__all__ = ['KParameter', 'KNode', 'KGraph', 'NodeState', 'PipelineExecutor']
+
+try:
+    from importlib.metadata import PackageNotFoundError, version as _version
+    __version__ = _version('kpipeliner')
+except Exception:  # not installed (plain checkout on sys.path)
+    __version__ = 'unknown'

kpipeliner-1.12.1/kpipeline/config.yaml

@@ -0,0 +1,49 @@
+# kpipeliner installation-wide configuration.
+#
+# This is the packaged DEFAULT. The first time kpl runs, it copies this file to your
+# own config directory so your edits survive upgrades of the package:
+#     $KPL_HOME/config.yaml   if the KPL_HOME environment variable is set, else
+#     ~/.config/kpipeliner/config.yaml
+# Run `kpl config` to see the exact path. Relative paths below are relative to that
+# directory; absolute paths and ~ are fine too. Delete a key to get its default back.
+
+# Directory containing the base node types (the node library your pipelines use).
+# Nodes are grouped in category subdirectories; a category starting with '_' is
+# ignored; nodes directly in the root get the category 'no category'. A pipeline can
+# opt out of / extend this via load_default_base_nodes_dir and custom_base_nodes_dirs
+# in its graph_config.yaml. (Legacy key name: base_nodes)
+default_base_nodes_dir: base_nodes
+
+# Directory for groups exported from the GUI as reusable components. Components are
+# stored as <group_base_nodes>/<category>/<name>.yaml and appear in the components
+# panel under 'groups/<category>'.
+group_base_nodes: group_base_nodes
+
+# Whether a `nodes.X.Y` input reference is resolved lazily (the node's own process
+# reads the cached value the first time its code touches self.inputs[name].value) or
+# eagerly (kpl reads it up front and passes it on the command line). Note: either way
+# the producing node is run first — lazy only postpones READING its result.
+lazy_input_resolution: true
+
+# Where `kpl run --kwui` / `kpl run_debug --kwui` serve workflow.py (needs the optional
+# `kwui` extra: pip install "kpipeliner[kwui]").
+# host: 0.0.0.0 listens on all interfaces, so the page is reachable from other machines
+#   on the network (kwebui has no authentication — use 127.0.0.1 to keep it on this
+#   machine).
+# port: starting port; the first free one at or above it is used.
+# open_browser: open a browser on this machine as soon as the server is up.
+# Per-run overrides: --kwui-host / --kwui-port / --kwui-no-browser.
+kwui:
+  host: 0.0.0.0
+  port: 8701
+  open_browser: true
+
+# Appearance of the web GUI (`kpl gui`); the GUI itself also edits these.
+gui:
+  background:
+    pattern: dots  # dots, lines, cross, none
+    gap: 10
+    size: 1
+    color: '#2d3748'
+  snap:
+    enabled: true