processes 2.0.1__tar.gz → 3.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (89) hide show
  1. processes-3.1.0/.github/PULL_REQUEST_TEMPLATE.md +10 -0
  2. {processes-2.0.1 → processes-3.1.0}/.github/workflows/docs.yml +2 -2
  3. {processes-2.0.1 → processes-3.1.0}/.github/workflows/lint.yml +2 -2
  4. {processes-2.0.1 → processes-3.1.0}/.github/workflows/mypy.yml +2 -2
  5. {processes-2.0.1 → processes-3.1.0}/.github/workflows/publish.yml +4 -4
  6. {processes-2.0.1 → processes-3.1.0}/.github/workflows/tags.yml +2 -2
  7. {processes-2.0.1 → processes-3.1.0}/.github/workflows/tests.yml +2 -2
  8. {processes-2.0.1 → processes-3.1.0}/CHANGELOG.md +33 -0
  9. processes-3.1.0/CONTRIBUTING.md +135 -0
  10. processes-3.1.0/PKG-INFO +408 -0
  11. processes-3.1.0/README.md +383 -0
  12. processes-3.1.0/RELEASE_NOTES.md +36 -0
  13. {processes-2.0.1 → processes-3.1.0}/docs/examples/advanced.md +53 -11
  14. {processes-2.0.1 → processes-3.1.0}/docs/index.md +75 -5
  15. {processes-2.0.1 → processes-3.1.0}/docs/reference.md +2 -1
  16. {processes-2.0.1 → processes-3.1.0}/mkdocs.yml +10 -0
  17. {processes-2.0.1 → processes-3.1.0}/pyproject.toml +3 -3
  18. {processes-2.0.1 → processes-3.1.0}/src/processes/__init__.py +6 -7
  19. processes-3.1.0/src/processes/_email_internals.py +168 -0
  20. processes-3.1.0/src/processes/_error_data.py +38 -0
  21. processes-3.1.0/src/processes/_log_formatting.py +44 -0
  22. processes-3.1.0/src/processes/_tb_utils.py +79 -0
  23. processes-3.1.0/src/processes/email_config.py +91 -0
  24. processes-3.1.0/src/processes/exceptions.py +36 -0
  25. {processes-2.0.1 → processes-3.1.0}/src/processes/process.py +71 -51
  26. processes-3.1.0/src/processes/task.py +421 -0
  27. processes-3.1.0/tests/base_test.py +62 -0
  28. processes-3.1.0/tests/manual_tests/__init__.py +0 -0
  29. {processes-2.0.1 → processes-3.1.0}/tests/manual_tests/manual_pipeline_inspect.py +8 -8
  30. {processes-2.0.1 → processes-3.1.0}/tests/manual_tests/manual_themed_tracebacks.py +15 -13
  31. processes-3.1.0/tests/test_args_kwargs.py +125 -0
  32. {processes-2.0.1 → processes-3.1.0}/tests/test_complex_dag_failures.py +29 -98
  33. processes-3.1.0/tests/test_dependencies.py +232 -0
  34. processes-3.1.0/tests/test_email_themes.py +531 -0
  35. processes-3.1.0/tests/test_logfiles.py +84 -0
  36. processes-3.1.0/tests/test_normal_run_no_errors.py +137 -0
  37. {processes-2.0.1 → processes-3.1.0}/tests/test_parallel_race_conditions.py +21 -33
  38. processes-3.1.0/tests/test_timeout_retry.py +338 -0
  39. processes-3.1.0/tests/test_unique_name.py +40 -0
  40. processes-2.0.1/PKG-INFO +0 -234
  41. processes-2.0.1/README.md +0 -210
  42. processes-2.0.1/src/processes/exception_html_formatter.py +0 -351
  43. processes-2.0.1/src/processes/html_logging.py +0 -160
  44. processes-2.0.1/src/processes/task.py +0 -307
  45. processes-2.0.1/tests/conftest.py +0 -9
  46. processes-2.0.1/tests/log_cleaner.py +0 -8
  47. processes-2.0.1/tests/test_args_kwargs.py +0 -166
  48. processes-2.0.1/tests/test_dependencies.py +0 -266
  49. processes-2.0.1/tests/test_email_themes.py +0 -489
  50. processes-2.0.1/tests/test_examples.py +0 -48
  51. processes-2.0.1/tests/test_logfiles.py +0 -130
  52. processes-2.0.1/tests/test_normal_run_no_errors.py +0 -174
  53. processes-2.0.1/tests/test_unique_name.py +0 -45
  54. {processes-2.0.1 → processes-3.1.0}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
  55. {processes-2.0.1 → processes-3.1.0}/.github/ISSUE_TEMPLATE/custom.md +0 -0
  56. {processes-2.0.1 → processes-3.1.0}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
  57. {processes-2.0.1 → processes-3.1.0}/.github/workflows/lint-pr.yml +0 -0
  58. {processes-2.0.1 → processes-3.1.0}/.gitignore +0 -0
  59. {processes-2.0.1 → processes-3.1.0}/LICENSE +0 -0
  60. {processes-2.0.1 → processes-3.1.0}/assets/banner.svg +0 -0
  61. {processes-2.0.1 → processes-3.1.0}/assets/logo.png +0 -0
  62. {processes-2.0.1 → processes-3.1.0}/assets/logo.svg +0 -0
  63. {processes-2.0.1 → processes-3.1.0}/assets/social_banner.png +0 -0
  64. {processes-2.0.1 → processes-3.1.0}/assets/social_banner.svg +0 -0
  65. {processes-2.0.1 → processes-3.1.0}/docs/assets/favicon.svg +0 -0
  66. {processes-2.0.1 → processes-3.1.0}/docs/examples/basic.md +0 -0
  67. {processes-2.0.1 → processes-3.1.0}/docs/examples/intro.md +0 -0
  68. {processes-2.0.1 → processes-3.1.0}/examples/01_basic_tasks_and_dependencies/README.md +0 -0
  69. {processes-2.0.1 → processes-3.1.0}/examples/01_basic_tasks_and_dependencies/example1.py +0 -0
  70. {processes-2.0.1 → processes-3.1.0}/examples/02_task_dependencies_result_passing/README.md +0 -0
  71. {processes-2.0.1 → processes-3.1.0}/examples/02_task_dependencies_result_passing/example2.py +0 -0
  72. {processes-2.0.1 → processes-3.1.0}/examples/README.md +0 -0
  73. {processes-2.0.1 → processes-3.1.0}/pytest.ini +0 -0
  74. /processes-2.0.1/tests/__init__.py → /processes-3.1.0/src/processes/py.typed +0 -0
  75. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/languages/de.json +0 -0
  76. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/languages/en.json +0 -0
  77. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/languages/es.json +0 -0
  78. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/languages/fr.json +0 -0
  79. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/languages/it.json +0 -0
  80. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/languages/pt.json +0 -0
  81. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/palettes/catppuccin.css +0 -0
  82. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/palettes/neobones.css +0 -0
  83. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/palettes/neutral.css +0 -0
  84. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/palettes/slate.css +0 -0
  85. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/styles/classic.html +0 -0
  86. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/styles/compact.html +0 -0
  87. {processes-2.0.1 → processes-3.1.0}/src/processes/themes/styles/modern.html +0 -0
  88. {processes-2.0.1/tests/manual_tests → processes-3.1.0/tests}/__init__.py +0 -0
  89. {processes-2.0.1 → processes-3.1.0}/uv.lock +0 -0
@@ -0,0 +1,10 @@
1
+ ## What and why
2
+ <!-- Describe the change and the motivation behind it -->
3
+
4
+ ## Type of change
5
+ - [ ] `feat` — new feature
6
+ - [ ] `fix` — bug fix
7
+ - [ ] `refactor` — no behavior change
8
+ - [ ] `docs` — documentation only
9
+ - [ ] `test` — tests only
10
+ - [ ] `chore` / `ci` / `build`
@@ -20,8 +20,8 @@ jobs:
20
20
  deploy:
21
21
  runs-on: ubuntu-latest
22
22
  steps:
23
- - uses: actions/checkout@v4
24
- - uses: astral-sh/setup-uv@v5
23
+ - uses: actions/checkout@v6
24
+ - uses: astral-sh/setup-uv@v6
25
25
  - name: Install dependencies
26
26
  run: uv sync --all-groups
27
27
  - name: Build and Deploy
@@ -14,10 +14,10 @@ jobs:
14
14
  lint:
15
15
  runs-on: ubuntu-latest
16
16
  steps:
17
- - uses: actions/checkout@v4
17
+ - uses: actions/checkout@v6
18
18
 
19
19
  - name: Install uv
20
- uses: astral-sh/setup-uv@v5
20
+ uses: astral-sh/setup-uv@v6
21
21
  with:
22
22
  enable-cache: true # Speeds up future runs
23
23
 
@@ -14,10 +14,10 @@ jobs:
14
14
  lint:
15
15
  runs-on: ubuntu-latest
16
16
  steps:
17
- - uses: actions/checkout@v4
17
+ - uses: actions/checkout@v6
18
18
 
19
19
  - name: Install uv
20
- uses: astral-sh/setup-uv@v5
20
+ uses: astral-sh/setup-uv@v6
21
21
  with:
22
22
  enable-cache: true # Speeds up future runs
23
23
 
@@ -8,9 +8,9 @@ jobs:
8
8
  quality-gate:
9
9
  runs-on: ubuntu-latest
10
10
  steps:
11
- - uses: actions/checkout@v4
11
+ - uses: actions/checkout@v6
12
12
  - name: Install uv
13
- uses: astral-sh/setup-uv@v5
13
+ uses: astral-sh/setup-uv@v6
14
14
 
15
15
  - name: Install dependencies
16
16
  run: uv sync --all-extras --dev
@@ -33,9 +33,9 @@ jobs:
33
33
  contents: read
34
34
 
35
35
  steps:
36
- - uses: actions/checkout@v4
36
+ - uses: actions/checkout@v6
37
37
  - name: Install uv
38
- uses: astral-sh/setup-uv@v5
38
+ uses: astral-sh/setup-uv@v6
39
39
 
40
40
  - name: Build package
41
41
  run: uv build
@@ -15,10 +15,10 @@ jobs:
15
15
 
16
16
  steps:
17
17
  - name: Checkout Code
18
- uses: actions/checkout@v4
18
+ uses: actions/checkout@v6
19
19
 
20
20
  - name: Install uv
21
- uses: astral-sh/setup-uv@v5
21
+ uses: astral-sh/setup-uv@v6
22
22
  with:
23
23
  enable-cache: true
24
24
 
@@ -16,10 +16,10 @@ jobs:
16
16
 
17
17
  steps:
18
18
  - name: Checkout code
19
- uses: actions/checkout@v4
19
+ uses: actions/checkout@v6
20
20
 
21
21
  - name: Install uv
22
- uses: astral-sh/setup-uv@v5
22
+ uses: astral-sh/setup-uv@v6
23
23
  with:
24
24
  version: "latest"
25
25
  enable-cache: true
@@ -1,3 +1,36 @@
1
+ ## v3.1.0 (2026-06-13)
2
+
3
+ ### Feat
4
+
5
+ - **logging**: include failure context in task logfiles
6
+
7
+ ### Refactor
8
+
9
+ - extract typed _ErrorData from task_context via shared base formatter
10
+ - inline single-use _task_context_lines helper
11
+ - move task logfile formatting out of _email_internals
12
+
13
+ ## v3.0.1 (2026-06-13)
14
+
15
+ ### Refactor
16
+
17
+ - **process**: use bare raise and extract _is_done helper
18
+
19
+ ## v3.0.0 (2026-06-13)
20
+
21
+ ### BREAKING CHANGE
22
+
23
+ - html_mail_handler parameter removed from Task; use
24
+ smtp_config and email_style instead. HTMLSMTPHandler removed from public API.
25
+
26
+ ### Feat
27
+
28
+ - **html_logging**: expose last_path_traced_vars on HTMLSMTPHandler
29
+
30
+ ### Refactor
31
+
32
+ - **email**: replace HTMLSMTPHandler with SMTPConfig/HTMLEmailStyle
33
+
1
34
  ## v2.0.1 (2026-06-12)
2
35
 
3
36
  ### Fix
@@ -0,0 +1,135 @@
1
+ # Contributing to Processes
2
+
3
+ Thanks for your interest in contributing! This document covers the two
4
+ contributor paths, project conventions, and how to submit a change.
5
+
6
+ ## Contributor paths
7
+
8
+ **Maintainer** (direct push access): clone the repository directly.
9
+
10
+ ```bash
11
+ git clone https://github.com/oliverm91/processes.git
12
+ cd processes
13
+ uv sync
14
+ ```
15
+
16
+ **External contributor** (everyone else): fork first, then clone your fork.
17
+
18
+ ```bash
19
+ # 1. Fork on GitHub (button top-right of the repo page)
20
+ # 2. Clone your fork
21
+ git clone https://github.com/<your-username>/processes.git
22
+ cd processes
23
+ uv sync
24
+ ```
25
+
26
+ Both paths use [uv](https://docs.astral.sh/uv/) for dependency management.
27
+ `uv sync` installs all dev dependencies (pytest, mypy, ruff, mkdocs, commitizen).
28
+
29
+ ## Project layout
30
+
31
+ ```
32
+ src/processes/ # library source (the public package)
33
+ tests/ # pytest test suite
34
+ examples/ # runnable usage examples
35
+ docs/ # mkdocs documentation source
36
+ ```
37
+
38
+ Keep new public symbols in `src/processes/` and re-export them from
39
+ `src/processes/__init__.py` if they are part of the stable API.
40
+
41
+ ## Running checks locally
42
+
43
+ Before opening a pull request, make sure the following pass:
44
+
45
+ ```bash
46
+ uv run pytest # test suite
47
+ uv run mypy # strict type checking
48
+ uv run ruff check . # linting
49
+ uv run ruff format . # formatting
50
+ ```
51
+
52
+ CI runs the same checks on every push and pull request.
53
+
54
+ ## Style guide
55
+
56
+ - Python >= 3.10 syntax.
57
+ - Type hints on all new code. `mypy` is configured with `strict = true` and
58
+ `disallow_untyped_defs = true`.
59
+ - Ruff rules: `F`, `E`, `W`, `I`, `B`, `UP`. Line length is 100.
60
+ - Prefer the standard library — the library has zero runtime dependencies.
61
+ - No comments that just restate the code. A comment is welcome only when it
62
+ captures a non-obvious *why*.
63
+
64
+ ## Tests
65
+
66
+ - Add or update tests in `tests/` for any behavior change.
67
+ - Keep tests focused and deterministic; avoid sleeps where possible.
68
+ - If you fix a bug, add a regression test that fails before the fix.
69
+
70
+ ## Commit messages
71
+
72
+ This project follows [Conventional Commits](https://www.conventionalcommits.org/),
73
+ enforced by [commitizen](https://commitizen-tools.github.io/commitizen/). Use
74
+ `uv run cz commit` to be prompted through an interactive commit, or write the
75
+ message by hand:
76
+
77
+ ```
78
+ <type>(<scope>): <short description>
79
+
80
+ <optional body>
81
+
82
+ <optional footer>
83
+ ```
84
+
85
+ Common types: `feat`, `fix`, `docs`, `style`, `refactor`, `test`, `chore`,
86
+ `build`, `ci`.
87
+
88
+ ## Submitting a pull request
89
+
90
+ 1. Create a feature branch: `git checkout -b feat/my-change`.
91
+ 2. Make your changes, add tests, and confirm all checks pass.
92
+ 3. Push the branch to your fork (or directly, if you're a maintainer) and open
93
+ a PR against `main` on the upstream repository.
94
+ 4. Fill in a clear description of the *what* and the *why*.
95
+ 5. Make sure CI is green and the changelog preview looks reasonable for
96
+ user-visible changes.
97
+ 6. Be ready to revise — code review is a normal part of the process.
98
+
99
+ Prefer several small, focused PRs over one large one.
100
+
101
+ ## Documentation
102
+
103
+ User-facing changes should be reflected in the docs under `docs/`. To preview
104
+ locally:
105
+
106
+ ```bash
107
+ uv run mkdocs serve
108
+ ```
109
+
110
+ ## Versioning and releases
111
+
112
+ Versions are derived from git tags via `hatch-vcs` (`v$version`, e.g. `v2.0.1`).
113
+ Do not edit the version in `pyproject.toml` manually. To cut a release, the
114
+ maintainer runs:
115
+
116
+ ```bash
117
+ uv run cz bump
118
+ ```
119
+
120
+ This updates the version, regenerates the changelog, creates a tag, and
121
+ triggers the `publish.yml` workflow to push the release to PyPI.
122
+
123
+ ## Reporting issues
124
+
125
+ Use the GitHub issue tracker. Please include:
126
+
127
+ - A minimal, self-contained reproduction.
128
+ - The Python version, library version, and operating system.
129
+ - The full traceback for crashes, and the smallest possible task graph that
130
+ triggers the problem.
131
+
132
+ ## License
133
+
134
+ By contributing, you agree that your contributions will be licensed under the
135
+ MIT License.
@@ -0,0 +1,408 @@
1
+ Metadata-Version: 2.4
2
+ Name: processes
3
+ Version: 3.1.0
4
+ Summary: Orchestrate graphs of callables in Python with automatic dependency resolution, parallel execution, retries, timeouts, and HTML email alerts on failure — zero dependencies
5
+ Project-URL: Homepage, https://github.com/oliverm91/processes
6
+ Project-URL: Documentation, https://oliverm91.github.io/processes/
7
+ Project-URL: Repository, https://github.com/oliverm91/processes
8
+ Project-URL: Issues, https://github.com/oliverm91/processes/issues
9
+ Project-URL: Changelog, https://github.com/oliverm91/processes/blob/main/CHANGELOG.md
10
+ Author-email: Oliver Mohr Bonometti <oliver.mohr.b@gmail.com>
11
+ License-Expression: MIT
12
+ License-File: LICENSE
13
+ Keywords: dag,dependencies,etl,parallel,process,tasks,topological-sort,workflow
14
+ Classifier: Development Status :: 4 - Beta
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.10
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Programming Language :: Python :: 3.14
22
+ Classifier: Typing :: Typed
23
+ Requires-Python: >=3.10
24
+ Description-Content-Type: text/markdown
25
+
26
+ <div align="center">
27
+ <img src="https://raw.githubusercontent.com/oliverm91/processes/refs/heads/main/assets/banner.svg" width="100%" alt="Processes - Smart Task Orchestration">
28
+ </div>
29
+
30
+ # Processes: Smart Task Orchestration
31
+
32
+ [![Python Version](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/)
33
+ ![Fast & Lightweight](https://img.shields.io/badge/Library-Pure%20Python-green.svg)
34
+ [![Documentation](https://img.shields.io/badge/docs-GitHub%20Pages-blue.svg)](https://oliverm91.github.io/processes/)
35
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
36
+ [![PyPI version](https://img.shields.io/pypi/v/processes.svg)](https://pypi.org/project/processes/)
37
+
38
+ [![Python Tests Status](https://github.com/oliverm91/processes/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/oliverm91/processes/actions/workflows/tests.yml)
39
+ [![Ruff Lint Status](https://github.com/oliverm91/processes/actions/workflows/lint.yml/badge.svg?branch=main)](https://github.com/oliverm91/processes/actions/workflows/lint.yml)
40
+ [![mypy-check](https://github.com/oliverm91/processes/actions/workflows/mypy.yml/badge.svg)](https://github.com/oliverm91/processes/actions/workflows/mypy.yml)
41
+
42
+ ---
43
+
44
+ **Run a list of Python callables that depend on each other — in parallel when possible, with per-task log files and optional HTML email notification on failure. Zero dependencies. Pure Python 3.10+.**
45
+
46
+ ---
47
+
48
+ ## ✨ Why Processes?
49
+
50
+ - 🔗 **Declare what depends on what** — write your tasks in any order; the runtime sorts them so every dependency runs first.
51
+ - ⚡ **Run in parallel when you can** — independent tasks run together on a thread pool; the runtime switches on automatically for jobs with 10+ tasks.
52
+ - 🛡️ **One failure doesn't stop the rest** — a failed task skips only the jobs that depend on it, and **every other part of the workflow keeps running**.
53
+ - 📝 **One log file per task** — share a single log across the whole run, or keep them separate for easier debugging.
54
+ - 📧 **Email alerts when something breaks** — pass an `SMTPConfig` to a task and get a styled HTML email (with traceback, task context, and the list of jobs that were skipped) the instant it raises.
55
+ - 🧰 **Modern, strictly-typed Python 3.10+** — `from __future__ import annotations`, full `mypy --strict` clean, `dict[str, TaskResult]`, `set[str]`, `|` unions.
56
+
57
+ ---
58
+
59
+ ## ⚙️ How it works
60
+
61
+ A `Process` holds a list of `Task`s. At construction it validates names, types, dependency references, and detects cycles — raising before anything runs.
62
+
63
+ When you call `process.run()`, tasks are topologically sorted and scheduled: dependencies first, independent tasks in parallel.
64
+
65
+ A `TaskDependency` can forward an upstream result directly into a downstream function, as a positional or keyword argument. The result is a `ProcessResult` with `passed_tasks_results` and `failed_tasks` for inspection.
66
+
67
+ ---
68
+
69
+ ## 🚀 Quick start
70
+
71
+ A 15-line "hello pipeline" — one upstream task feeding a downstream one, run in parallel.
72
+
73
+ ```python
74
+ from processes import Process, Task, TaskDependency
75
+
76
+
77
+ def load_users() -> list[dict]:
78
+ return [{"id": 1}, {"id": 2}, {"id": 3}]
79
+
80
+
81
+ def enrich(users: list[dict]) -> list[dict]:
82
+ return [{**u, "name": f"user-{u['id']}"} for u in users]
83
+
84
+
85
+ tasks = [
86
+ Task("load", "run.log", load_users),
87
+ Task(
88
+ "enrich",
89
+ "run.log",
90
+ enrich,
91
+ dependencies=[TaskDependency("load", use_result_as_additional_args=True)],
92
+ ),
93
+ ]
94
+
95
+ with Process(tasks) as p:
96
+ result = p.run(parallel=True)
97
+
98
+ print(result.passed_tasks_results["enrich"].result)
99
+ # [{'id': 1, 'name': 'user-1'}, {'id': 2, 'name': 'user-2'}, {'id': 3, 'name': 'user-3'}]
100
+ ```
101
+
102
+ ---
103
+
104
+ ## 🧪 End-to-end example
105
+
106
+ A realistic mini-pipeline: fetch two sources **in parallel**, transform them, aggregate, and notify — with per-task log files, result piping, and one task deliberately failing to show fault isolation.
107
+
108
+ <details>
109
+ <summary>Show the full end-to-end example</summary>
110
+
111
+ ```python
112
+ import logging
113
+ from pathlib import Path
114
+
115
+ from processes import HTMLEmailStyle, Process, SMTPConfig, Task, TaskDependency
116
+
117
+ LOG_DIR = Path("logs")
118
+ LOG_DIR.mkdir(exist_ok=True)
119
+
120
+
121
+ # --- 1. Two independent "fetch" tasks that run in parallel -----------------
122
+ def fetch_orders() -> list[dict]:
123
+ logging.info("querying orders API")
124
+ return [{"order_id": 1, "amount": 42.0}, {"order_id": 2, "amount": 17.5}]
125
+
126
+
127
+ def fetch_inventory() -> list[dict]:
128
+ logging.info("querying inventory API")
129
+ return [{"sku": "A-1", "qty": 12}, {"sku": "B-2", "qty": 3}]
130
+
131
+
132
+ # --- 2. Two transforms that consume the upstream results -------------------
133
+ def total_revenue(orders: list[dict]) -> float:
134
+ total = sum(o["amount"] for o in orders)
135
+ logging.info("revenue computed: %s", total)
136
+ return total
137
+
138
+
139
+ def stock_value(inventory: list[dict], *, price_per_unit: float = 10.0) -> float:
140
+ value = sum(i["qty"] for i in inventory) * price_per_unit
141
+ logging.info("stock value: %s", value)
142
+ return value
143
+
144
+
145
+ # --- 3. An aggregator that joins the two branches --------------------------
146
+ def build_report(*, revenue: float, stock: float) -> str:
147
+ return f"daily-report | revenue={revenue:.2f} stock={stock:.2f}"
148
+
149
+
150
+ # --- 4. A flaky notifier that ALWAYS fails — to show fault isolation -------
151
+ def notify_slack(report: str) -> None:
152
+ raise RuntimeError("slack webhook returned 503")
153
+
154
+
155
+ # --- 5. A sibling task that does NOT depend on notify and still runs -------
156
+ def archive_report(report: str) -> str:
157
+ out = LOG_DIR / "report.txt"
158
+ out.write_text(report)
159
+ return str(out)
160
+
161
+
162
+ # --- 6. Optional: SMTP config so failures page on-call --------------------
163
+ smtp = SMTPConfig(
164
+ mailhost=("smtp.example.com", 587),
165
+ fromaddr="alerts@example.com",
166
+ toaddrs=["oncall@example.com"],
167
+ credentials=("user", "pass"),
168
+ secure=(),
169
+ )
170
+
171
+ tasks = [
172
+ Task("fetch_orders", LOG_DIR / "fetch_orders.log", fetch_orders),
173
+ Task("fetch_inventory", LOG_DIR / "fetch_inventory.log", fetch_inventory),
174
+
175
+ Task(
176
+ "compute_revenue",
177
+ LOG_DIR / "compute_revenue.log",
178
+ total_revenue,
179
+ dependencies=[TaskDependency("fetch_orders", use_result_as_additional_args=True)],
180
+ ),
181
+ Task(
182
+ "compute_stock",
183
+ LOG_DIR / "compute_stock.log",
184
+ stock_value,
185
+ kwargs={"price_per_unit": 7.25},
186
+ dependencies=[
187
+ TaskDependency(
188
+ "fetch_inventory",
189
+ use_result_as_additional_kwargs=True,
190
+ additional_kwarg_name="inventory",
191
+ )
192
+ ],
193
+ ),
194
+
195
+ Task(
196
+ "build_report",
197
+ LOG_DIR / "build_report.log",
198
+ build_report,
199
+ dependencies=[
200
+ TaskDependency("compute_revenue", use_result_as_additional_kwargs=True,
201
+ additional_kwarg_name="revenue"),
202
+ TaskDependency("compute_stock", use_result_as_additional_kwargs=True,
203
+ additional_kwarg_name="stock"),
204
+ ],
205
+ ),
206
+
207
+ # notify_slack fails on purpose. archive_report is a *sibling*
208
+ # of notify_slack (both depend on build_report), so it has no
209
+ # dependency on the failed task and runs normally — the rest of
210
+ # the workflow is not blackholed by one broken step.
211
+ Task(
212
+ "notify_slack",
213
+ LOG_DIR / "notify_slack.log",
214
+ notify_slack,
215
+ dependencies=[TaskDependency("build_report", use_result_as_additional_args=True)],
216
+ smtp_config=smtp,
217
+ ),
218
+ Task(
219
+ "archive_report",
220
+ LOG_DIR / "archive_report.log",
221
+ archive_report,
222
+ dependencies=[TaskDependency("build_report", use_result_as_additional_args=True)],
223
+ ),
224
+ ]
225
+
226
+ with Process(tasks) as process:
227
+ result = process.run(parallel=True)
228
+
229
+ print("passed:", sorted(result.passed_tasks_results))
230
+ # archive_report, build_report, compute_revenue, compute_stock, fetch_inventory, fetch_orders
231
+ print("failed:", sorted(result.failed_tasks))
232
+ # notify_slack
233
+ print("report:", result.passed_tasks_results["build_report"].result)
234
+ # daily-report | revenue=59.50 stock=262.50
235
+ ```
236
+
237
+ The failing `notify_slack` task does **not** abort the run. `archive_report` is a sibling of the failed task (both depend on the successful `build_report`), so it runs unaffected — the rest of the workflow is not blackholed by one broken step. The HTML email handler also fires on the `notify_slack` task, paging on-call with the full traceback and the list of downstream tasks that were skipped because of it.
238
+
239
+ </details>
240
+
241
+ ---
242
+
243
+ ## 📚 API Reference
244
+
245
+ <details>
246
+ <summary>Show API reference</summary>
247
+
248
+ ### `Task`
249
+
250
+ ```python
251
+ Task(
252
+ name: str,
253
+ log_path: str | os.PathLike,
254
+ func: Callable[..., Any],
255
+ args: tuple = (),
256
+ kwargs: dict | None = None,
257
+ dependencies: list[TaskDependency] | None = None,
258
+ smtp_config: SMTPConfig | None = None,
259
+ email_style: HTMLEmailStyle | None = None,
260
+ timeout: float | None = None,
261
+ retries: int | None = 0,
262
+ retry_on: tuple[type[Exception], ...] | None = None,
263
+ )
264
+ ```
265
+
266
+ - `name` — unique within the `Process`; no spaces.
267
+ - `log_path` — the file this task logs to (INFO level, format `%(asctime)s - %(name)s - %(levelname)s - %(message)s`).
268
+ - `func` — the callable; receives `func(*args, **kwargs)` after result-injection.
269
+ - `smtp_config` — when set, fires an HTML email on `logging.ERROR`; body includes `task_name`, `function`, `args`, `kwargs`, and `downstream_impact`.
270
+ - `email_style` — optional presentation override; defaults to `HTMLEmailStyle()` (modern, neutral, English) when `smtp_config` is set.
271
+ - `timeout` — seconds allowed per attempt; `None` means no limit. When the timeout fires the underlying thread is detached (Python threading limitation).
272
+ - `retries` — additional attempts after the first failure; `0` or `None` means a single attempt. Defaults to `0`.
273
+ - `retry_on` — tuple of exception types that trigger a retry. When `retries >= 1` and `retry_on` is `None`, defaults to `(ConnectionError, TimeoutError)` at call time.
274
+
275
+ ### `TaskDependency`
276
+
277
+ ```python
278
+ TaskDependency(
279
+ task_name: str,
280
+ use_result_as_additional_args: bool = False,
281
+ use_result_as_additional_kwargs: bool = False,
282
+ additional_kwarg_name: str = "",
283
+ )
284
+ ```
285
+
286
+ - `use_result_as_additional_args=True` — upstream result appended as the next **positional** arg.
287
+ - `use_result_as_additional_kwargs=True` with a non-empty `additional_kwarg_name` — upstream result injected as a **keyword** arg.
288
+ - Both flags can be combined (positional first, then kwarg).
289
+
290
+ ### `Process`
291
+
292
+ ```python
293
+ Process(tasks: list[Task]) # validates types, names, deps, cycles
294
+
295
+ process.run(parallel: bool | None = None, max_workers: int = 4) -> ProcessResult
296
+ ```
297
+
298
+ - Raises `DependencyNotFoundError`, `CircularDependencyError`, `TypeError`, `ValueError` on construction if the workflow is malformed.
299
+ - `parallel=None` auto-parallelises when `len(tasks) >= 10`; `max_workers=1` is always sequential.
300
+ - Use as a context manager — it cleans up `FileHandler`s on exit.
301
+
302
+ ### `ProcessResult`
303
+
304
+ ```python
305
+ result.passed_tasks_results # dict[str, TaskResult] — name → TaskResult for every task that succeeded
306
+ result.failed_tasks # set[str] — all tasks that did not produce a result (errored + skipped)
307
+ result.errored_tasks # set[str] — tasks whose function actually raised
308
+ result.skipped_tasks # set[str] — tasks skipped because an upstream dependency failed
309
+
310
+ TaskResult(worked: bool, result: Any, exception: Exception | None)
311
+ ```
312
+
313
+ ### `SMTPConfig`
314
+
315
+ ```python
316
+ SMTPConfig(
317
+ mailhost, # (host, port)
318
+ fromaddr,
319
+ toaddrs, # list[str]
320
+ credentials=None, # (username, password) | None
321
+ secure=None, # () = STARTTLS; omit for no encryption
322
+ timeout=5,
323
+ )
324
+ ```
325
+
326
+ ### `HTMLEmailStyle`
327
+
328
+ ```python
329
+ HTMLEmailStyle(
330
+ style="modern", # classic | modern | compact
331
+ palette="neutral", # neutral | catppuccin | neobones | slate
332
+ language="en", # en | es | pt | fr | de | it
333
+ traced_vars_frame_filter=None, # substring to pick the traced frame | None
334
+ )
335
+ ```
336
+
337
+ All fields are optional — omit `HTMLEmailStyle` entirely to use the defaults.
338
+
339
+ </details>
340
+
341
+ <details>
342
+ <summary>Show fault-tolerance rules in detail</summary>
343
+
344
+ When a task raises:
345
+
346
+ 1. The exception is caught and stored in `TaskResult.exception`; the task name goes into `failed_tasks` and `errored_tasks`.
347
+ 2. **Every task that depends on it (directly or indirectly) is skipped** — added to `failed_tasks` and `skipped_tasks` without running.
348
+ 3. **Every other independent part of the workflow keeps running.** With `parallel=True` they keep running concurrently on the worker pool.
349
+ 4. After `run()` returns, `ProcessResult.errored_tasks` and `ProcessResult.skipped_tasks` let you distinguish root failures from cascade skips for triage or alerting.
350
+
351
+ When a task has `retries >= 1`, a failure matching `retry_on` triggers another attempt before the task is declared failed and its dependants are skipped. This gives transient errors (network blips, connection resets) a chance to resolve without aborting downstream work.
352
+
353
+ This makes the library a good fit for fan-out / fan-in pipelines, "best-effort" notifications, and any workflow where one broken step should not blackhole the rest.
354
+
355
+ </details>
356
+
357
+ <details>
358
+ <summary>Show comparison with other libraries</summary>
359
+
360
+ | | **Processes** | Airflow | Celery | Luigi |
361
+ |---|---|---|---|---|
362
+ | External dependencies | **None** | many | broker (Redis/RabbitMQ) | few |
363
+ | Setup cost | `pip install` | cluster | broker + workers | task + config |
364
+ | Parallelism | built-in | via executors | via workers | via workers |
365
+ | Per-task file logs | **yes (built-in)** | via handlers | via signals | partial |
366
+ | HTML email on failure | **yes (built-in)** | via callbacks | via signals | manual |
367
+ | DAG validation at construction | **yes** | yes (DAG file) | n/a | partial |
368
+ | Strict typing (`mypy --strict`) | **yes** | partial | partial | no |
369
+
370
+ `Processes` is **not** a distributed scheduler — there are no workers on remote machines, no SLA monitoring, no web UI. If you need any of those, you need Airflow or a similar orchestrator. If you want a small, fast, dependency-aware pipeline that *just runs* in a single process, this is it.
371
+
372
+ </details>
373
+
374
+ <details>
375
+ <summary>Show advanced configuration</summary>
376
+
377
+ - **Shared log file** — pass the same `log_path` to every `Task` for a single combined run.log; pass distinct paths for per-task isolation.
378
+ - **Auto-parallel** — `Process.run()` with no argument runs sequentially for small workflows and switches to parallel for `len(tasks) >= 10`. Pass `parallel=True` or `parallel=False` to force the mode.
379
+ - **Result inspection** — iterate `result.passed_tasks_results.items()` to log or post-process every successful task; iterate `result.failed_tasks` for triage.
380
+ - **Re-raising** — wrap `process.run()` in `try/except` if you need a non-zero exit code on any failure; the library itself does not raise on partial failure.
381
+
382
+ </details>
383
+
384
+ ---
385
+
386
+ ## 📦 Installation
387
+
388
+ From PyPI:
389
+
390
+ ```bash
391
+ pip install processes
392
+ ```
393
+
394
+ Or straight from the repository (pure Python, no build step):
395
+
396
+ ```bash
397
+ pip install git+https://github.com/oliverm91/processes.git
398
+ ```
399
+
400
+ Requires **Python 3.10+**.
401
+
402
+ ---
403
+
404
+ ## 📄 License & contributing
405
+
406
+ Released under the **MIT License** — see [docs](https://oliverm91.github.io/processes/) for full API details.
407
+
408
+ Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for the workflow, style, and commit-message conventions used by this project.