tinyorch 0.1.1__tar.gz → 0.1.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: tinyorch
3
- Version: 0.1.1
3
+ Version: 0.1.2
4
4
  Summary: A tiny, zero-dependency orchestrator for single-node, item-level pipelines.
5
5
  Author: Murilegus
6
6
  Author-email: Murilegus <murilegus@outlook.com>
@@ -10,6 +10,7 @@ Classifier: Development Status :: 3 - Alpha
10
10
  Classifier: Programming Language :: Python :: 3
11
11
  Classifier: Programming Language :: Python :: 3.12
12
12
  Classifier: Programming Language :: Python :: 3.13
13
+ Classifier: Programming Language :: Python :: 3.14
13
14
  Classifier: Typing :: Typed
14
15
  Requires-Python: >=3.12
15
16
  Project-URL: Homepage, https://github.com/murilegus/tinyorch
@@ -22,12 +23,18 @@ Description-Content-Type: text/markdown
22
23
  [![ci](https://github.com/murilegus/tinyorch/actions/workflows/ci.yml/badge.svg)](https://github.com/murilegus/tinyorch/actions/workflows/ci.yml)
23
24
  [![PyPI](https://img.shields.io/pypi/v/tinyorch)](https://pypi.org/project/tinyorch/)
24
25
  [![Python](https://img.shields.io/pypi/pyversions/tinyorch)](https://pypi.org/project/tinyorch/)
25
- [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
26
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/murilegus/tinyorch/blob/main/LICENSE)
26
27
 
27
28
  A tiny, zero-dependency orchestrator for single-node, item-level pipelines.
28
29
 
29
30
  🚧 **Status**: alpha — phase 1 released (0.1), phase 2 in progress. See [Roadmap](#-roadmap).
30
31
 
32
+ > ⚠️ **What exists today (0.1):** `@task` / `@flow` decorators and in-memory run
33
+ > tracking — nothing more. Pipelining, bounded buffers, per-item resume and
34
+ > resources are **planned, not implemented**. The comparisons and performance
35
+ > claims below describe the design target; **none of them has been benchmarked
36
+ > yet**.
37
+
31
38
  ## 🚀 Quickstart
32
39
 
33
40
  ```bash
@@ -70,33 +77,42 @@ type checkers.
70
77
 
71
78
  ## 🤔 Why tinyorch?
72
79
 
73
- tinyorch is for the job that is **too small for Prefect, too finite for Flink,
74
- and too painful as a bare script**: a few thousand to a few million items
75
- (URLs, documents, records) flowing through I/O-heavy stages on one machine,
76
- where any stage can fail halfway.
80
+ tinyorch is aimed at the job that is **too small for Prefect, too finite for
81
+ Flink, and too painful as a bare script**: a few thousand to a few million
82
+ items (URLs, documents, records) flowing through I/O-heavy stages on one
83
+ machine, where any stage can fail halfway.
84
+
85
+ Three families of tools already exist for pipelines. The table below is how
86
+ tinyorch positions itself against them.
77
87
 
78
- Three families of tools already exist for pipelines. Each is right for its
79
- sweet spot and awkward outside it:
88
+ > ⚠️ **Unverified.** The other tools' columns are generalizations from their
89
+ > documentation, not from measurements or side-by-side use, and some cells are
90
+ > coarser than the tools deserve. The **tinyorch** column is the design target
91
+ > for phases 2–4; cells marked *(planned)* are not implemented in 0.1.
80
92
 
81
- | | Workflow orchestrators<br>Airflow · Prefect · Dagster | Stream processors<br>Flink · Bytewax · Faust | Bare script<br>`asyncio.gather` + `tenacity` | **tinyorch** |
93
+ | | Workflow orchestrators<br>Airflow · Prefect · Dagster | Stream processors<br>Flink · Bytewax · Faust | Bare script<br>`asyncio.gather` + `tenacity` | **tinyorch** (target) |
82
94
  |---|---|---|---|---|
83
95
  | Input | bounded, per task | **unbounded**, per event | bounded | bounded, per item |
84
- | Execution | one stage finishes before the next starts | continuous | ad hoc | pipelined, bounded buffers |
85
- | Resume after crash | per task / per run | cluster checkpoints | hand-rolled or none | per item |
86
- | Shared resources (browser, LLM client) | per process, awkward | n/a | manual | first-class |
96
+ | Execution | one stage finishes before the next starts | continuous | ad hoc | pipelined, bounded buffers *(planned, phase 2)* |
97
+ | Resume after crash | per task / per run | cluster checkpoints | hand-rolled or none | per item *(planned, phase 3)* |
98
+ | Shared resources (browser, LLM client) | per process, awkward | n/a | manual | first-class *(planned, phase 4)* |
87
99
  | Infra | server + database | cluster | none | none |
88
- | Sweet spot | multi-team, multi-machine DAGs | 24/7 event streams | one-off | one node, 10³–10⁶ items, crash-prone I/O |
100
+ | Sweet spot | multi-team, multi-machine DAGs | 24/7 event streams | one-off | one node, 10³–10⁶ items, crash-prone I/O *(unmeasured)* |
89
101
 
90
102
  The architectural difference tinyorch is built around: in a stage-at-a-time
91
103
  orchestrator, data crossing a task boundary is materialized (in RAM or an
92
- object store) before the next stage starts. tinyorch instead pipelines items
93
- through stages with bounded buffers and checkpoints each item, so a rerun only
94
- does what is left and memory stays flat regardless of input size.
104
+ object store) before the next stage starts. tinyorch is designed to instead
105
+ pipeline items through stages with bounded buffers and checkpoint each item,
106
+ so that a rerun only does what is left and memory stays flat regardless of
107
+ input size. **This is a design goal, not a measured result**: none of it is
108
+ implemented in 0.1, and no benchmark backs the memory or resume claims yet.
95
109
 
96
110
  ![Stage-at-a-time orchestrators vs tinyorch pipelining](https://raw.githubusercontent.com/murilegus/tinyorch/main/docs/assets/stage-vs-pipelined.svg)
97
111
 
98
- **tinyorch is not a stream processor.** It takes a *bounded* input and
99
- executes it with streaming semantics. There are no windows, no event time, no
112
+ *Conceptual diagram of the planned execution model, not a benchmark.*
113
+
114
+ **tinyorch is not a stream processor.** It is meant to take a *bounded* input
115
+ and execute it with streaming semantics. There are no windows, no event time, no
100
116
  unbounded sources, no distributed state. If you need those, use Flink.
101
117
 
102
118
  **It is not a replacement for Prefect or Airflow either.** When the pipeline
@@ -106,9 +122,11 @@ aims to make that boundary obvious rather than blurry.
106
122
 
107
123
  ## 🏗️ Architecture
108
124
 
109
- The core is stdlib-only and defines Protocols; everything that touches the
110
- outside world (state stores, browsers, LLM clients, CLI) is an adapter shipped
111
- as an optional extra.
125
+ **Planned architecture** — today only the `@task` / `@flow` core exists; no
126
+ Protocols or adapters have been written yet. The intent: the core is
127
+ stdlib-only and defines Protocols; everything that touches the outside world
128
+ (state stores, browsers, LLM clients, CLI) is an adapter shipped as an
129
+ optional extra.
112
130
 
113
131
  ![tinyorch architecture](https://raw.githubusercontent.com/murilegus/tinyorch/main/docs/assets/architecture.svg)
114
132
 
@@ -140,9 +158,7 @@ Progress is tracked in [GitHub milestones](https://github.com/murilegus/tinyorch
140
158
 
141
159
  ## 📄 Documentation
142
160
 
143
- - [`docs/design/`](docs/design/) — design decisions are recorded as ADRs.
144
-
145
-
161
+ - [`docs/design/`](https://github.com/murilegus/tinyorch/tree/main/docs/design) — design decisions are recorded as ADRs.
146
162
  ## 🎫 License
147
163
 
148
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
164
+ This project is licensed under the MIT License - see the [LICENSE](https://github.com/murilegus/tinyorch/blob/main/LICENSE) file for details.
@@ -3,12 +3,18 @@
3
3
  [![ci](https://github.com/murilegus/tinyorch/actions/workflows/ci.yml/badge.svg)](https://github.com/murilegus/tinyorch/actions/workflows/ci.yml)
4
4
  [![PyPI](https://img.shields.io/pypi/v/tinyorch)](https://pypi.org/project/tinyorch/)
5
5
  [![Python](https://img.shields.io/pypi/pyversions/tinyorch)](https://pypi.org/project/tinyorch/)
6
- [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
6
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/murilegus/tinyorch/blob/main/LICENSE)
7
7
 
8
8
  A tiny, zero-dependency orchestrator for single-node, item-level pipelines.
9
9
 
10
10
  🚧 **Status**: alpha — phase 1 released (0.1), phase 2 in progress. See [Roadmap](#-roadmap).
11
11
 
12
+ > ⚠️ **What exists today (0.1):** `@task` / `@flow` decorators and in-memory run
13
+ > tracking — nothing more. Pipelining, bounded buffers, per-item resume and
14
+ > resources are **planned, not implemented**. The comparisons and performance
15
+ > claims below describe the design target; **none of them has been benchmarked
16
+ > yet**.
17
+
12
18
  ## 🚀 Quickstart
13
19
 
14
20
  ```bash
@@ -51,33 +57,42 @@ type checkers.
51
57
 
52
58
  ## 🤔 Why tinyorch?
53
59
 
54
- tinyorch is for the job that is **too small for Prefect, too finite for Flink,
55
- and too painful as a bare script**: a few thousand to a few million items
56
- (URLs, documents, records) flowing through I/O-heavy stages on one machine,
57
- where any stage can fail halfway.
60
+ tinyorch is aimed at the job that is **too small for Prefect, too finite for
61
+ Flink, and too painful as a bare script**: a few thousand to a few million
62
+ items (URLs, documents, records) flowing through I/O-heavy stages on one
63
+ machine, where any stage can fail halfway.
64
+
65
+ Three families of tools already exist for pipelines. The table below is how
66
+ tinyorch positions itself against them.
58
67
 
59
- Three families of tools already exist for pipelines. Each is right for its
60
- sweet spot and awkward outside it:
68
+ > ⚠️ **Unverified.** The other tools' columns are generalizations from their
69
+ > documentation, not from measurements or side-by-side use, and some cells are
70
+ > coarser than the tools deserve. The **tinyorch** column is the design target
71
+ > for phases 2–4; cells marked *(planned)* are not implemented in 0.1.
61
72
 
62
- | | Workflow orchestrators<br>Airflow · Prefect · Dagster | Stream processors<br>Flink · Bytewax · Faust | Bare script<br>`asyncio.gather` + `tenacity` | **tinyorch** |
73
+ | | Workflow orchestrators<br>Airflow · Prefect · Dagster | Stream processors<br>Flink · Bytewax · Faust | Bare script<br>`asyncio.gather` + `tenacity` | **tinyorch** (target) |
63
74
  |---|---|---|---|---|
64
75
  | Input | bounded, per task | **unbounded**, per event | bounded | bounded, per item |
65
- | Execution | one stage finishes before the next starts | continuous | ad hoc | pipelined, bounded buffers |
66
- | Resume after crash | per task / per run | cluster checkpoints | hand-rolled or none | per item |
67
- | Shared resources (browser, LLM client) | per process, awkward | n/a | manual | first-class |
76
+ | Execution | one stage finishes before the next starts | continuous | ad hoc | pipelined, bounded buffers *(planned, phase 2)* |
77
+ | Resume after crash | per task / per run | cluster checkpoints | hand-rolled or none | per item *(planned, phase 3)* |
78
+ | Shared resources (browser, LLM client) | per process, awkward | n/a | manual | first-class *(planned, phase 4)* |
68
79
  | Infra | server + database | cluster | none | none |
69
- | Sweet spot | multi-team, multi-machine DAGs | 24/7 event streams | one-off | one node, 10³–10⁶ items, crash-prone I/O |
80
+ | Sweet spot | multi-team, multi-machine DAGs | 24/7 event streams | one-off | one node, 10³–10⁶ items, crash-prone I/O *(unmeasured)* |
70
81
 
71
82
  The architectural difference tinyorch is built around: in a stage-at-a-time
72
83
  orchestrator, data crossing a task boundary is materialized (in RAM or an
73
- object store) before the next stage starts. tinyorch instead pipelines items
74
- through stages with bounded buffers and checkpoints each item, so a rerun only
75
- does what is left and memory stays flat regardless of input size.
84
+ object store) before the next stage starts. tinyorch is designed to instead
85
+ pipeline items through stages with bounded buffers and checkpoint each item,
86
+ so that a rerun only does what is left and memory stays flat regardless of
87
+ input size. **This is a design goal, not a measured result**: none of it is
88
+ implemented in 0.1, and no benchmark backs the memory or resume claims yet.
76
89
 
77
90
  ![Stage-at-a-time orchestrators vs tinyorch pipelining](https://raw.githubusercontent.com/murilegus/tinyorch/main/docs/assets/stage-vs-pipelined.svg)
78
91
 
79
- **tinyorch is not a stream processor.** It takes a *bounded* input and
80
- executes it with streaming semantics. There are no windows, no event time, no
92
+ *Conceptual diagram of the planned execution model, not a benchmark.*
93
+
94
+ **tinyorch is not a stream processor.** It is meant to take a *bounded* input
95
+ and execute it with streaming semantics. There are no windows, no event time, no
81
96
  unbounded sources, no distributed state. If you need those, use Flink.
82
97
 
83
98
  **It is not a replacement for Prefect or Airflow either.** When the pipeline
@@ -87,9 +102,11 @@ aims to make that boundary obvious rather than blurry.
87
102
 
88
103
  ## 🏗️ Architecture
89
104
 
90
- The core is stdlib-only and defines Protocols; everything that touches the
91
- outside world (state stores, browsers, LLM clients, CLI) is an adapter shipped
92
- as an optional extra.
105
+ **Planned architecture** — today only the `@task` / `@flow` core exists; no
106
+ Protocols or adapters have been written yet. The intent: the core is
107
+ stdlib-only and defines Protocols; everything that touches the outside world
108
+ (state stores, browsers, LLM clients, CLI) is an adapter shipped as an
109
+ optional extra.
93
110
 
94
111
  ![tinyorch architecture](https://raw.githubusercontent.com/murilegus/tinyorch/main/docs/assets/architecture.svg)
95
112
 
@@ -121,9 +138,7 @@ Progress is tracked in [GitHub milestones](https://github.com/murilegus/tinyorch
121
138
 
122
139
  ## 📄 Documentation
123
140
 
124
- - [`docs/design/`](docs/design/) — design decisions are recorded as ADRs.
125
-
126
-
141
+ - [`docs/design/`](https://github.com/murilegus/tinyorch/tree/main/docs/design) — design decisions are recorded as ADRs.
127
142
  ## 🎫 License
128
143
 
129
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
144
+ This project is licensed under the MIT License - see the [LICENSE](https://github.com/murilegus/tinyorch/blob/main/LICENSE) file for details.
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "tinyorch"
3
- version = "0.1.1"
3
+ version = "0.1.2"
4
4
  description = "A tiny, zero-dependency orchestrator for single-node, item-level pipelines."
5
5
  readme = "README.md"
6
6
  license = "MIT"
@@ -11,6 +11,7 @@ classifiers = [
11
11
  "Programming Language :: Python :: 3",
12
12
  "Programming Language :: Python :: 3.12",
13
13
  "Programming Language :: Python :: 3.13",
14
+ "Programming Language :: Python :: 3.14",
14
15
  "Typing :: Typed",
15
16
  ]
16
17
  dependencies = []
@@ -44,6 +45,7 @@ include = [
44
45
  ]
45
46
  typeCheckingMode = "strict"
46
47
  failOnWarnings = true
48
+ reportUnnecessaryTypeIgnoreComment = "error"
47
49
 
48
50
  [[tool.basedpyright.executionEnvironments]]
49
51
  root = "tests"
@@ -51,7 +53,7 @@ root = "tests"
51
53
  [tool.pytest.ini_options]
52
54
  asyncio_mode = "auto"
53
55
  testpaths = ["tests"]
54
- addopts = "--cov=tinyorch --cov-report=term-missing --cov-fail-under=95"
56
+ addopts = "--cov=tinyorch --cov-report=term-missing"
55
57
 
56
58
  [tool.coverage.run]
57
59
  branch = true
@@ -61,7 +63,7 @@ source = ["tinyorch"]
61
63
  exclude_also = [
62
64
  "if TYPE_CHECKING:",
63
65
  "@overload",
64
- '\.\.\.',
66
+ '^\s*\.\.\.$',
65
67
  ]
66
68
 
67
69
  [tool.ruff]
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "tinyorch"
3
- version = "0.1.1"
3
+ version = "0.1.2"
4
4
  description = "A tiny, zero-dependency orchestrator for single-node, item-level pipelines."
5
5
  readme = "README.md"
6
6
  authors = [{ name = "Murilegus", email = "murilegus@outlook.com" }]
@@ -12,6 +12,7 @@ classifiers = [
12
12
  "Programming Language :: Python :: 3",
13
13
  "Programming Language :: Python :: 3.12",
14
14
  "Programming Language :: Python :: 3.13",
15
+ "Programming Language :: Python :: 3.14",
15
16
  "Typing :: Typed",
16
17
  ]
17
18
  dependencies = []
@@ -38,6 +39,9 @@ build-backend = "uv_build"
38
39
  include = ["src", "tests"]
39
40
  typeCheckingMode = "strict"
40
41
  failOnWarnings = true
42
+ # Off by default even in strict; without it a stale `# pyright: ignore[...]`
43
+ # passes silently and the negative checks in tests/test_typing.py guard nothing.
44
+ reportUnnecessaryTypeIgnoreComment = "error"
41
45
 
42
46
  [[tool.basedpyright.executionEnvironments]]
43
47
  root = "tests"
@@ -45,7 +49,9 @@ root = "tests"
45
49
  [tool.pytest.ini_options]
46
50
  asyncio_mode = "auto"
47
51
  testpaths = ["tests"]
48
- addopts = "--cov=tinyorch --cov-report=term-missing --cov-fail-under=95"
52
+ # The coverage floor is enforced in CI (`pytest --cov-fail-under=95`), not here,
53
+ # so running a single test file locally does not fail on partial coverage.
54
+ addopts = "--cov=tinyorch --cov-report=term-missing"
49
55
 
50
56
  [tool.coverage.run]
51
57
  branch = true
@@ -55,7 +61,7 @@ source = ["tinyorch"]
55
61
  exclude_also = [
56
62
  "if TYPE_CHECKING:",
57
63
  "@overload",
58
- "\\.\\.\\.",
64
+ "^\\s*\\.\\.\\.$",
59
65
  ]
60
66
 
61
67
  [tool.ruff]
@@ -7,6 +7,7 @@ import inspect
7
7
  from typing import TYPE_CHECKING, cast, override
8
8
 
9
9
  from tinyorch._context import enter_run
10
+ from tinyorch._validate import check_wrappable
10
11
 
11
12
  if TYPE_CHECKING:
12
13
  from collections.abc import Awaitable, Callable
@@ -14,6 +15,7 @@ if TYPE_CHECKING:
14
15
 
15
16
  class Flow[**P, R]:
16
17
  def __init__(self, fn: Callable[P, R], *, name: str | None = None) -> None:
18
+ check_wrappable(fn, "flow")
17
19
  self.fn: Callable[P, R] = fn
18
20
  self.name: str = name or fn.__name__
19
21
  self.is_async: bool = inspect.iscoroutinefunction(fn)
@@ -8,6 +8,7 @@ import time
8
8
  from typing import TYPE_CHECKING, cast, override
9
9
 
10
10
  from tinyorch._context import RunContext, TaskCall, current_run
11
+ from tinyorch._validate import check_wrappable
11
12
 
12
13
  if TYPE_CHECKING:
13
14
  from collections.abc import Awaitable, Callable
@@ -15,6 +16,7 @@ if TYPE_CHECKING:
15
16
 
16
17
  class Task[**P, R]:
17
18
  def __init__(self, fn: Callable[P, R], *, name: str | None = None) -> None:
19
+ check_wrappable(fn, "task")
18
20
  self.fn: Callable[P, R] = fn
19
21
  self.name: str = name or fn.__name__
20
22
  self.is_async: bool = inspect.iscoroutinefunction(fn)
@@ -0,0 +1,32 @@
1
+ # SPDX-FileCopyrightText: © 2026 Murilegus <murilegus@outlook.com>
2
+ # SPDX-License-Identifier: MIT
3
+
4
+ from __future__ import annotations
5
+
6
+ import inspect
7
+ from typing import TYPE_CHECKING
8
+
9
+ if TYPE_CHECKING:
10
+ from collections.abc import Callable
11
+
12
+
13
+ def check_wrappable(fn: Callable[..., object], kind: str) -> None:
14
+ """Reject callables whose body would not run inside the bookkeeping.
15
+
16
+ Only plain ``def`` / ``async def`` functions and bound methods are
17
+ accepted. See ADR-002.
18
+ """
19
+ if not (inspect.isfunction(fn) or inspect.ismethod(fn)):
20
+ msg = (
21
+ f"@{kind} expects a function defined with 'def' or 'async def', "
22
+ f"got {type(fn).__name__}; wrap it in a function first"
23
+ )
24
+ raise TypeError(msg)
25
+
26
+ if inspect.isgeneratorfunction(fn) or inspect.isasyncgenfunction(fn):
27
+ msg = (
28
+ f"@{kind} does not support generator functions ({fn.__qualname__}): "
29
+ f"the body would run after the {kind} call has already returned. "
30
+ "Return a value from a plain 'def' or 'async def' instead"
31
+ )
32
+ raise TypeError(msg)
File without changes
File without changes
File without changes