uv-task-runner 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- uv_task_runner-0.1.1/PKG-INFO +353 -0
- uv_task_runner-0.1.1/README.md +331 -0
- uv_task_runner-0.1.1/pyproject.toml +115 -0
- uv_task_runner-0.1.1/src/uv_task_runner/__init__.py +29 -0
- uv_task_runner-0.1.1/src/uv_task_runner/__main__.py +92 -0
- uv_task_runner-0.1.1/src/uv_task_runner/errors.py +6 -0
- uv_task_runner-0.1.1/src/uv_task_runner/pipeline.py +152 -0
- uv_task_runner-0.1.1/src/uv_task_runner/py.typed +0 -0
- uv_task_runner-0.1.1/src/uv_task_runner/settings.py +85 -0
- uv_task_runner-0.1.1/src/uv_task_runner/task.py +209 -0
- uv_task_runner-0.1.1/src/uv_task_runner/template_config.toml +50 -0
- uv_task_runner-0.1.1/src/uv_task_runner/utils.py +28 -0
|
@@ -0,0 +1,353 @@
|
|
|
1
|
+
Metadata-Version: 2.3
|
|
2
|
+
Name: uv-task-runner
|
|
3
|
+
Version: 0.1.1
|
|
4
|
+
Summary: A simple utility to run multiple Python scripts sequentially or in parallel, with isolated environments, monitoring and error handling.
|
|
5
|
+
Author: bjhardcastle
|
|
6
|
+
Author-email: bjhardcastle <ben.hardcastle@alleninstitute.org>
|
|
7
|
+
License: MIT
|
|
8
|
+
Classifier: Development Status :: 4 - Beta
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Operating System :: Microsoft :: Windows
|
|
15
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
16
|
+
Requires-Dist: pydantic-settings>=2
|
|
17
|
+
Requires-Dist: eval-type-backport ; python_full_version < '3.10'
|
|
18
|
+
Requires-Python: >=3.8
|
|
19
|
+
Project-URL: Issues, https://github.com/AllenNeuralDynamics/uv-task-runner/issues
|
|
20
|
+
Project-URL: Repository, https://github.com/AllenNeuralDynamics/uv-task-runner
|
|
21
|
+
Description-Content-Type: text/markdown
|
|
22
|
+
|
|
23
|
+
# uv-task-runner
|
|
24
|
+
|
|
25
|
+
Run multiple Python scripts in parallel or in sequence, with per-script dependency and Python version isolation via [uv](https://docs.astral.sh/uv/).
|
|
26
|
+
|
|
27
|
+
Each script is invoked as `uv run <script>`, so scripts can declare their own dependencies and Python version using [PEP 723 inline metadata](https://peps.python.org/pep-0723/). No more shared mega-environments.
|
|
28
|
+
|
|
29
|
+
[](https://pypi.org/project/uv-task-runner/)
|
|
30
|
+
[](https://pypi.org/project/uv-task-runner/)
|
|
31
|
+
|
|
32
|
+
[](https://github.com/astral-sh/ty)
|
|
33
|
+
[](https://app.codecov.io/github/AllenNeuralDynamics/uv-task-runner)
|
|
34
|
+
[](https://github.com/AllenNeuralDynamics/uv-task-runner/actions/workflows/publish.yaml)
|
|
35
|
+
[](https://github.com/AllenNeuralDynamics/uv-task-runner/issues)
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Requirements
|
|
41
|
+
|
|
42
|
+
- Python 3.8+
|
|
43
|
+
- `uv` on PATH. See https://docs.astral.sh/uv/getting-started/installation/
|
|
44
|
+
|
|
45
|
+
## Installation
|
|
46
|
+
|
|
47
|
+
Make available globally:
|
|
48
|
+
```bash
|
|
49
|
+
uv install uv-task-runner
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
Or run CLI tool in temporary environment:
|
|
53
|
+
```bash
|
|
54
|
+
uv run uv-task-runner
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Or add library to Python project:
|
|
58
|
+
```bash
|
|
59
|
+
uv add uv-task-runner
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Usage
|
|
65
|
+
|
|
66
|
+
### CLI
|
|
67
|
+
|
|
68
|
+
Generate an annotated config file in the current directory:
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
uv run uv-task-runner --init # writes uv_task_runner.toml
|
|
72
|
+
uv run uv-task-runner --init my_tasks.toml # custom path
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Or write it by hand. Minimal `uv_task_runner.toml`:
|
|
76
|
+
|
|
77
|
+
```toml
|
|
78
|
+
[[tasks]]
|
|
79
|
+
task_path = "scripts/preprocess.py"
|
|
80
|
+
|
|
81
|
+
[[tasks]]
|
|
82
|
+
task_path = "scripts/analyze.py"
|
|
83
|
+
task_args = ["--output", "results/"]
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Then run:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
uv run uv-task-runner
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Use a different config file:
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
uv run uv-task-runner --config path/to/config.toml
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Override settings at the command line (CLI args take precedence over TOML):
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
uv run uv-task-runner --parallel --fail-fast --log-level DEBUG
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Tasks can also be passed directly via `--tasks` as a JSON array (the TOML config is recommended for anything beyond a quick one-off, as shell escaping is error-prone):
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
# Single task
|
|
108
|
+
uv run uv-task-runner --tasks "[{\"task_path\":\"scripts/my_script.py\"}]"
|
|
109
|
+
|
|
110
|
+
# Multiple tasks with args
|
|
111
|
+
uv run uv-task-runner --tasks "[{\"task_path\":\"scripts/a.py\"},{\"task_path\":\"scripts/b.py\",\"task_args\":[\"--verbose\"]}]"
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
Note: double quotes inside the JSON must be escaped with `\"`. All `TaskConfig` fields are supported.
|
|
115
|
+
|
|
116
|
+
### Example output
|
|
117
|
+
|
|
118
|
+
Given a `uv_task_runner.toml`:
|
|
119
|
+
|
|
120
|
+
```toml
|
|
121
|
+
# Tasks are executed in order below if parallel=false (default):
|
|
122
|
+
[[tasks]]
|
|
123
|
+
task_path = "examples/script_a.py"
|
|
124
|
+
task_args = ["--param1", "updated_value"]
|
|
125
|
+
wait = false # don't wait for script_a.py to finish before starting the next task
|
|
126
|
+
|
|
127
|
+
[[tasks]]
|
|
128
|
+
task_path = "https://gist.githubusercontent.com/TAJD/1d389deba4221343caef5155090674eb/raw/13984206c008fdb35d2d574fa76b682991f00a08/error_handling.py"
|
|
129
|
+
|
|
130
|
+
[[tasks]]
|
|
131
|
+
task_path = "examples/script_b.py"
|
|
132
|
+
# if script does not declare dependencies with PEP 723 metadata it's possible to customize uv run args:
|
|
133
|
+
uv_run_args = ["--python", "3.14", "--verbose", "--script", "--no-project"]
|
|
134
|
+
|
|
135
|
+
[[tasks]]
|
|
136
|
+
task_path = "examples/script_c.py"
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Running `uv run uv-task-runner` produces:
|
|
140
|
+
|
|
141
|
+
```
|
|
142
|
+
2026-03-02 13:32:27 | INFO | Running 4 task(s).
|
|
143
|
+
2026-03-02 13:32:27 | INFO | Running command: uv run --quiet --script examples/script_a.py --param1 updated_value
|
|
144
|
+
2026-03-02 13:32:27 | INFO | examples/script_a.py is running: not waiting for it to finish.
|
|
145
|
+
2026-03-02 13:32:27 | INFO | Running command: uv run --quiet --script https://gist.githubusercontent.com/TAJD/1d389deba4221343caef5155090674eb/raw/13984206c008fdb35d2d574fa76b682991f00a08/error_handling.py
|
|
146
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] Error: The divisor 'b' cannot be zero.
|
|
147
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] Error: The divisor 'b' cannot be zero.
|
|
148
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] Stack trace:
|
|
149
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] File "C:\Users\BEN~1.HAR\AppData\Local\Temp\error_handlingjKocFl.py", line 52, in <module>
|
|
150
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] simple_example()
|
|
151
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] File "C:\Users\BEN~1.HAR\AppData\Local\Temp\error_handlingjKocFl.py", line 47, in simple_example
|
|
152
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] result = divide_numbers_stacktrace(10, 0)
|
|
153
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] File "C:\Users\BEN~1.HAR\AppData\Local\Temp\error_handlingjKocFl.py", line 37, in divide_numbers_stacktrace
|
|
154
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] return nested_division()
|
|
155
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] File "C:\Users\BEN~1.HAR\AppData\Local\Temp\error_handlingjKocFl.py", line 34, in nested_division
|
|
156
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824] stack_trace = ''.join(traceback.format_stack())
|
|
157
|
+
2026-03-02 13:32:27 | INFO | [error_handling.py:164824]
|
|
158
|
+
2026-03-02 13:32:27 | INFO | https://gist.githubusercontent.com/TAJD/1d389deba4221343caef5155090674eb/raw/13984206c008fdb35d2d574fa76b682991f00a08/error_handling.py completed successfully.
|
|
159
|
+
2026-03-02 13:32:27 | INFO | Running command: uv run --python 3.14 --verbose --script --no-project examples/script_b.py
|
|
160
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG uv 0.10.7 (08ab1a344 2026-02-27)
|
|
161
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Found project root: `C:\Users\ben.hardcastle\github\uv-plugin-architecture`
|
|
162
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG No workspace root found, using project root
|
|
163
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Ignoring discovered project due to `--no-project`
|
|
164
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG No project found; searching for Python interpreter
|
|
165
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Using request connect timeout of 10s and read timeout of 30s
|
|
166
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Searching for Python 3.14 in virtual environments, managed installations, search path, or registry
|
|
167
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Found `cpython-3.13.1-windows-x86_64-none` at `C:\Users\ben.hardcastle\github\uv-plugin-architecture\.venv\Scripts\python.exe` (active virtual environment)
|
|
168
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Skipping interpreter at `.venv\Scripts\python.exe` from active virtual environment: does not satisfy request `3.14`
|
|
169
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Found `cpython-3.13.1-windows-x86_64-none` at `C:\Users\ben.hardcastle\github\uv-plugin-architecture\.venv\Scripts\python.exe` (virtual environment)
|
|
170
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Skipping interpreter at `.venv\Scripts\python.exe` from virtual environment: does not satisfy request `3.14`
|
|
171
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Searching for managed installations at `C:\Users\ben.hardcastle\cache\uv\python`
|
|
172
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Skipping managed installation `cpython-3.13.1-windows-x86_64-none`: does not satisfy `3.14`
|
|
173
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Found `cpython-3.13.1-windows-x86_64-none` at `C:\Users\ben.hardcastle\github\uv-plugin-architecture\.venv\Scripts\python.exe` (first executable in the search path)
|
|
174
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Skipping interpreter at `.venv\Scripts\python.exe` from first executable in the search path: does not satisfy request `3.14`
|
|
175
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] INFO Fetching requested Python...
|
|
176
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Downloading https://github.com/astral-sh/python-build-standalone/releases/download/20260211/cpython-3.14.3%2B20260211-x86_64-pc-windows-msvc-install_only_stripped.tar.gz
|
|
177
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] DEBUG Extracting cpython-3.14.3-20260211-x86_64-pc-windows-msvc-install_only_stripped.tar.gz to temporary location: C:\Users\ben.hardcastle\cache\uv\python\.temp\.tmpajp9EC
|
|
178
|
+
2026-03-02 13:32:27 | INFO | [script_b.py:145032] Downloading cpython-3.14.3-windows-x86_64-none (download) (21.3MiB)
|
|
179
|
+
2026-03-02 13:32:35 | INFO | [script_a.py:162304] script_a.py loaded polars version 1.38.1
|
|
180
|
+
2026-03-02 13:32:35 | INFO | [script_a.py:162304] script_a.py running on Python 3.11.9
|
|
181
|
+
2026-03-02 13:32:35 | INFO | [script_a.py:162304] script_a.py successfully received param1 from command line: updated_value
|
|
182
|
+
2026-03-02 13:32:35 | INFO | [script_a.py:162304] script_a.py finished
|
|
183
|
+
2026-03-02 13:32:38 | INFO | [script_b.py:145032] Downloaded cpython-3.14.3-windows-x86_64-none (download)
|
|
184
|
+
2026-03-02 13:32:38 | INFO | [script_b.py:145032] DEBUG Moving C:\Users\ben.hardcastle\cache\uv\python\.temp\.tmpajp9EC\python to C:\Users\ben.hardcastle\cache\uv\python\cpython-3.14.3-windows-x86_64-none
|
|
185
|
+
2026-03-02 13:32:38 | INFO | [script_b.py:145032] DEBUG Created link C:\Users\ben.hardcastle\cache\uv\python\cpython-3.14-windows-x86_64-none -> C:\Users\ben.hardcastle\cache\uv\python\cpython-3.14.3-windows-x86_64-none
|
|
186
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] DEBUG Using Python 3.14.3 interpreter at: C:\Users\ben.hardcastle\cache\uv\python\cpython-3.14.3-windows-x86_64-none\python.exe
|
|
187
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] DEBUG Running `python examples/script_b.py`
|
|
188
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] script_b.py loaded on Python 3.14.3
|
|
189
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] Traceback (most recent call last):
|
|
190
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] File "C:\Users\ben.hardcastle\github\uv-plugin-architecture\scripts\script_b.py", line 5, in <module>
|
|
191
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] raise ValueError(f"Simulated error in {Path(__file__).name}")
|
|
192
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] ValueError: Simulated error in script_b.py
|
|
193
|
+
2026-03-02 13:32:39 | INFO | [script_b.py:145032] DEBUG Command exited with code: 1
|
|
194
|
+
2026-03-02 13:32:39 | ERROR | examples/script_b.py failed with exit code 1
|
|
195
|
+
2026-03-02 13:32:39 | INFO | Running command: uv run --quiet --script examples/script_c.py
|
|
196
|
+
2026-03-02 13:32:44 | INFO | [script_c.py:36208] script_c.py loaded on Python 3.13.1
|
|
197
|
+
2026-03-02 13:32:44 | INFO | [script_c.py:36208] script_c.py finished
|
|
198
|
+
2026-03-02 13:32:44 | INFO | examples/script_c.py completed successfully.
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Key things to note:
|
|
202
|
+
- `script_a.py` has `wait=false`: it starts immediately and execution continues without waiting for it. With `log_multiline=true`, its output is buffered until exit — but since the parent exits first, **no output is captured** and a warning is emitted.
|
|
203
|
+
- `error_handling.py` is fetched from a URL. Its multiline stderr (a stack trace) is emitted as a single log block because `log_multiline=true`.
|
|
204
|
+
- `script_b.py` exits non-zero, logged at `ERROR` level.
|
|
205
|
+
- `script_c.py` mixes stdout lines directly into the log stream (lines without the `[name:pid]` prefix come from the script's own `print()` calls).
|
|
206
|
+
|
|
207
|
+
### Python API
|
|
208
|
+
|
|
209
|
+
```python
|
|
210
|
+
from uv_task_runner import run_tasks, TaskConfig
|
|
211
|
+
|
|
212
|
+
results = run_tasks([
|
|
213
|
+
TaskConfig(task_path="scripts/preprocess.py"),
|
|
214
|
+
TaskConfig(task_path="scripts/analyze.py", task_args=["--output", "results/"]),
|
|
215
|
+
])
|
|
216
|
+
|
|
217
|
+
for r in results.task_results:
|
|
218
|
+
print(r.task_path, r.exit_code, r.duration_seconds)
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
For more control, use `Pipeline` directly:
|
|
222
|
+
|
|
223
|
+
```python
|
|
224
|
+
from uv_task_runner import Pipeline, Settings, TaskConfig
|
|
225
|
+
|
|
226
|
+
pipeline = Pipeline(
|
|
227
|
+
tasks=[
|
|
228
|
+
TaskConfig(task_path="scripts/a.py"),
|
|
229
|
+
TaskConfig(task_path="scripts/b.py"),
|
|
230
|
+
],
|
|
231
|
+
parallel=True,
|
|
232
|
+
fail_fast=True,
|
|
233
|
+
)
|
|
234
|
+
result = pipeline.run()
|
|
235
|
+
print(result.aborted, result.aborted_by)
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
---
|
|
239
|
+
|
|
240
|
+
## Configuration reference
|
|
241
|
+
|
|
242
|
+
### Global settings applied to `Pipeline`
|
|
243
|
+
|
|
244
|
+
| Key | Type | Default | Description |
|
|
245
|
+
|-----|------|---------|-------------|
|
|
246
|
+
| `parallel` | bool | `false` | Run all tasks concurrently. `false` runs them one at a time in listed order. |
|
|
247
|
+
| `fail_fast` | bool | `false` | Terminate remaining tasks on the first failure. |
|
|
248
|
+
| `dry_run` | bool | `false` | Print what would run without executing any tasks. |
|
|
249
|
+
| `log_level` | string or int | `"INFO"` | Standard logging level names: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`, case-insensitive. |
|
|
250
|
+
| `log_multiline` | bool | `false` | Buffer each task's stdout/stderr and emit as a single log message per stream. Default `false` logs lines as they arrive. With `parallel=true`, interleaved output from concurrent tasks can make multiline output (e.g. stack traces) hard to read: set `log_multiline=true` to keep them together at the cost of buffering until process exit. Has no readability effect when `parallel=false`. |
|
|
251
|
+
|
|
252
|
+
### Per-task settings applied to `TaskConfig`
|
|
253
|
+
|
|
254
|
+
| Key | Type | Default | Description |
|
|
255
|
+
|-----|------|---------|-------------|
|
|
256
|
+
| `task_path` | string | required | Path to the script, relative to the config file. Can also be a URL (e.g. a GitHub raw file). |
|
|
257
|
+
| `task_args` | list[string] | `[]` | Arguments passed to the script (`sys.argv`). |
|
|
258
|
+
| `uv_run_args` | list[string] | `["--quiet", "--script"]` | Arguments passed to `uv run` before the script path. |
|
|
259
|
+
| `wait` | bool | `true` | Wait for the task to finish before proceeding. `false` spawns the process and continues immediately. |
|
|
260
|
+
|
|
261
|
+
### Callback hooks (Python API only)
|
|
262
|
+
|
|
263
|
+
`TaskConfig` accepts Python callables for `on_task_start` and `on_task_end`. These are not settable via TOML.
|
|
264
|
+
|
|
265
|
+
```python
|
|
266
|
+
def on_start(task_path: str, pid: int) -> None:
|
|
267
|
+
print(f"Started {task_path} (PID {pid})")
|
|
268
|
+
|
|
269
|
+
def on_end(task_path: str, result: TaskResult) -> None:
|
|
270
|
+
print(f"{task_path} exited {result.exit_code} after {result.duration_seconds:.1f}s")
|
|
271
|
+
|
|
272
|
+
TaskConfig(
|
|
273
|
+
task_path="scripts/a.py",
|
|
274
|
+
on_task_start=on_start,
|
|
275
|
+
on_task_end=on_end,
|
|
276
|
+
)
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
`Pipeline` accepts `on_pipeline_start` and `on_pipeline_end` in the same way.
|
|
280
|
+
|
|
281
|
+
Hooks run synchronously in the parent process. Keep them fast; for slow operations, open a background thread inside the hook.
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## How scripts are run
|
|
286
|
+
|
|
287
|
+
Each task is executed as:
|
|
288
|
+
|
|
289
|
+
```
|
|
290
|
+
uv run [uv_run_args] [task_path] [task_args]
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
Scripts can declare their own Python version and dependencies using PEP 723 metadata:
|
|
294
|
+
|
|
295
|
+
```python
|
|
296
|
+
# /// script
|
|
297
|
+
# requires-python = ">=3.11"
|
|
298
|
+
# dependencies = ["polars>=0.20", "requests"]
|
|
299
|
+
# ///
|
|
300
|
+
|
|
301
|
+
import polars as pl
|
|
302
|
+
# ...
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
`uv` resolves and installs dependencies for each script independently. Scripts with different Python versions or incompatible dependency sets run without conflict.
|
|
306
|
+
|
|
307
|
+
---
|
|
308
|
+
|
|
309
|
+
## Return values
|
|
310
|
+
|
|
311
|
+
`Pipeline.run()` and `run_tasks()` return a `PipelineResult`:
|
|
312
|
+
|
|
313
|
+
```python
|
|
314
|
+
@dataclass
|
|
315
|
+
class PipelineResult:
|
|
316
|
+
task_results: list[TaskResult]
|
|
317
|
+
aborted: bool # True if fail_fast triggered early termination
|
|
318
|
+
aborted_by: str | None # task_path that caused the abort, or None
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
Each `TaskResult`:
|
|
322
|
+
|
|
323
|
+
```python
|
|
324
|
+
@dataclass
|
|
325
|
+
class TaskResult:
|
|
326
|
+
task_path: str
|
|
327
|
+
exit_code: int | None # None if wait=False
|
|
328
|
+
success: bool
|
|
329
|
+
duration_seconds: float
|
|
330
|
+
stdout: str # Empty string if wait=False
|
|
331
|
+
stderr: str # Empty string if wait=False
|
|
332
|
+
pid: int
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
The CLI entry point always exits with code 0. Inspect `PipelineResult` when using the Python API.
|
|
336
|
+
|
|
337
|
+
---
|
|
338
|
+
|
|
339
|
+
## Limitations
|
|
340
|
+
|
|
341
|
+
**No DAG-style task dependencies.** Sequential pipelines with `fail_fast=True` naturally express
|
|
342
|
+
linear chains ("run B only after A succeeds"). What is not supported is graph-style dependencies,
|
|
343
|
+
e.g. "run C after both A and B succeed" when A and B run in parallel. To implement phased parallel
|
|
344
|
+
execution, call `run_tasks()` or `Pipeline.run()` multiple times in sequence, or consider Snakemake,
|
|
345
|
+
Airflow, Prefect, or similar tools.
|
|
346
|
+
|
|
347
|
+
**`log_multiline=true` always buffers until process exit.** Output is held in a `stream.read()` call that blocks until the subprocess closes stdout. For normal `wait=true` tasks this means output appears as a single block at the end rather than in real-time. For `wait=false` (fire-and-forget) tasks it is worse: if the parent exits before the subprocess finishes, the daemon thread is killed and **no output is logged at all**. The default (`log_multiline=false`) logs lines as they arrive, which avoids both problems at the cost of interleaved output from concurrent tasks.
|
|
348
|
+
|
|
349
|
+
`TaskResult.stdout`/`stderr` are always empty for `wait=false` tasks regardless of buffering mode, because the capture threads are not joined before the result is collected. The subprocess will be reported as still running on pipeline exit.
|
|
350
|
+
|
|
351
|
+
**No per-task timeouts.** A hung task will block indefinitely. As a workaround, wrap the script invocation with `timeout` (Unix) or a similar mechanism.
|
|
352
|
+
|
|
353
|
+
**No task naming.** Tasks are identified by `task_path` in results and log output. Long paths or URLs can make logs harder to read.
|