pyflowstep 0.5.0__tar.gz → 0.7.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/PKG-INFO +915 -840
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/README.md +911 -837
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/pyproject.toml +4 -3
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/pyproject.toml.orig +96 -93
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/__init__.py +22 -7
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/compilers.py +18 -3
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/dependencies.py +327 -326
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/exceptions.py +139 -137
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/flow.py +163 -129
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/inputs.py +58 -7
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/json_schema.py +5 -6
- pyflowstep-0.7.0/src/pyflowstep/loaders.py +80 -0
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/steps.py +15 -11
- pyflowstep-0.5.0/src/pyflowstep/annotations.py +0 -90
- pyflowstep-0.5.0/src/pyflowstep/parsers.py +0 -181
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/py.typed +0 -0
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/registry.py +0 -0
- {pyflowstep-0.5.0 → pyflowstep-0.7.0}/src/pyflowstep/validators.py +0 -0
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pyflowstep
|
|
3
|
-
Version: 0.
|
|
4
|
-
Summary: A lightweight, typed Python library for composing functions into readable, reusable flows that can be defined as JSON.
|
|
5
|
-
Keywords: flow,pipeline,composition,functional,fluent-interface,json,json-schema,workflow,dsl
|
|
3
|
+
Version: 0.7.0
|
|
4
|
+
Summary: A lightweight, typed Python library for composing functions into readable, reusable flows that can be defined as JSON or TOML.
|
|
5
|
+
Keywords: flow,pipeline,composition,functional,fluent-interface,json,json-schema,toml,workflow,dsl
|
|
6
6
|
Author: aaltatan
|
|
7
7
|
Author-email: aaltatan <a.altatan@gmail.com>
|
|
8
8
|
License-Expression: GPL-3.0-only
|
|
@@ -15,846 +15,921 @@ Classifier: Programming Language :: Python :: 3.12
|
|
|
15
15
|
Classifier: Programming Language :: Python :: 3.13
|
|
16
16
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
17
17
|
Classifier: Typing :: Typed
|
|
18
|
+
Requires-Dist: pyargprocessors>=0.1.0
|
|
18
19
|
Requires-Python: >=3.12
|
|
19
20
|
Project-URL: Homepage, https://github.com/aaltatan/pyflowstep
|
|
20
21
|
Project-URL: Repository, https://github.com/aaltatan/pyflowstep
|
|
21
22
|
Project-URL: Issues, https://github.com/aaltatan/pyflowstep/issues
|
|
22
23
|
Description-Content-Type: text/markdown
|
|
23
24
|
|
|
24
|
-
# pyflowstep
|
|
25
|
-
|
|
26
|
-
A lightweight, typed Python library for composing functions into readable, reusable **flows** — a functional replacement for the fluent-interface (method-chaining) pattern, with flows that can be defined, validated and stored as **JSON**.
|
|
27
|
-
|
|
28
|
-
`pyflowstep` focuses on a functional style:
|
|
29
|
-
|
|
30
|
-
- steps are plain functions: `(subject, *args, **kwargs) -> subject`
|
|
31
|
-
- flows are immutable values that compose with `>>`
|
|
32
|
-
- step arguments are validated and
|
|
33
|
-
- raw JSON values become typed arguments with a `
|
|
34
|
-
- steps get external objects (a mailer, a database session) through FastAPI-style `Depends`
|
|
35
|
-
- the caller hands per-run values to a flow by name: `flow(page, user=user)` fills every `user: User = Input()`
|
|
36
|
-
- registry-based registration keeps steps organized and discoverable
|
|
37
|
-
- flow definitions can be compiled from dictionaries or JSON
|
|
38
|
-
- every registry can describe its flow language as a JSON Schema
|
|
39
|
-
|
|
40
|
-
It pairs naturally with its siblings [pyspecification](https://github.com/aaltatan/pyspecification) (predicates) and [pyformula](https://github.com/aaltatan/pyformula) (formulas),
|
|
41
|
-
|
|
42
|
-
---
|
|
43
|
-
|
|
44
|
-
## Why use pyflowstep?
|
|
45
|
-
|
|
46
|
-
A fluent API makes you put every operation on the class and `return self` from each method:
|
|
47
|
-
|
|
48
|
-
```python
|
|
49
|
-
page.navigate("https://www.google.com").fill("input[name=q]", "pyflowstep").click("#go").wait("#result")
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
That couples *what can be done* to *one class*, you cannot store the chain, reuse half of it, build it from data, or add an operation without editing the class.
|
|
53
|
-
|
|
54
|
-
With `pyflowstep`, operations are small functions and the chain is a value:
|
|
55
|
-
|
|
56
|
-
```python
|
|
57
|
-
search = navigate("https://www.google.com") >> fill("input[name=q]", "pyflowstep") >> click("#go")
|
|
58
|
-
|
|
59
|
-
search(page) # run it
|
|
60
|
-
search >> wait("#result") # extend it (a new flow, `search` is unchanged)
|
|
61
|
-
login >> search >> logout # combine flows
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
…and the same flow can come from JSON:
|
|
65
|
-
|
|
66
|
-
```json
|
|
67
|
-
[
|
|
68
|
-
{"name": "navigate", "args": ["https://www.google.com"]},
|
|
69
|
-
{"name": "fill", "args": ["input[name=q]", "pyflowstep"]},
|
|
70
|
-
{"name": "click", "kwargs": {"selector": "#go"}}
|
|
71
|
-
]
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
---
|
|
75
|
-
|
|
76
|
-
## Installation
|
|
77
|
-
|
|
78
|
-
```bash
|
|
79
|
-
pip install pyflowstep
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
or with uv:
|
|
83
|
-
|
|
84
|
-
```bash
|
|
85
|
-
uv add pyflowstep
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
Requires Python 3.12+.
|
|
89
|
-
|
|
90
|
-
---
|
|
91
|
-
|
|
92
|
-
## Quick start
|
|
93
|
-
|
|
94
|
-
```python
|
|
95
|
-
from typing import Any
|
|
96
|
-
|
|
97
|
-
from pyflowstep import step
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
class Page:
|
|
101
|
-
def navigate(self, url: str) -> None:
|
|
102
|
-
print(f"Navigating to {url}")
|
|
103
|
-
|
|
104
|
-
def click(self, selector: str) -> None:
|
|
105
|
-
print(f"Clicking on {selector}")
|
|
106
|
-
|
|
107
|
-
def fill(self, selector: str, value: Any) -> None:
|
|
108
|
-
print(f"Filling {selector} with {value}")
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
@step
|
|
112
|
-
def navigate(page: Page, url: str) -> Page:
|
|
113
|
-
page.navigate(url)
|
|
114
|
-
return page
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
@step
|
|
118
|
-
def fill(page: Page, selector: str, value: Any) -> Page:
|
|
119
|
-
page.fill(selector, value)
|
|
120
|
-
return page
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
@step
|
|
124
|
-
def click(page: Page, selector: str) -> Page:
|
|
125
|
-
page.click(selector)
|
|
126
|
-
return page
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
search = navigate("https://www.google.com") >> fill("input[name=q]", 1111) >> click("#go")
|
|
130
|
-
|
|
131
|
-
print(search) # Flow(navigate >> fill >> click)
|
|
132
|
-
search(Page())
|
|
133
|
-
# Navigating to https://www.google.com
|
|
134
|
-
# Filling input[name=q] with 1111
|
|
135
|
-
# Clicking on #go
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
---
|
|
139
|
-
|
|
140
|
-
## Core concepts
|
|
141
|
-
|
|
142
|
-
### Flow
|
|
143
|
-
|
|
144
|
-
A `Flow[T]` is an immutable sequence of `T -> T` actions. Calling it threads the subject through every action in order.
|
|
145
|
-
|
|
146
|
-
```python
|
|
147
|
-
from pyflowstep import Flow, compose
|
|
148
|
-
|
|
149
|
-
increment = Flow[int](lambda n: n + 1)
|
|
150
|
-
double = Flow[int](lambda n: n * 2)
|
|
151
|
-
|
|
152
|
-
(increment >> double)(3) # 8
|
|
153
|
-
(double >> increment)(3) # 7
|
|
154
|
-
compose(increment, double)(3) # 8, same as increment >> double
|
|
155
|
-
Flow[int]()(3) # 3, an empty flow is the identity
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
- `>>` accepts flows **and** plain callables, on either side: `flow >> fn`, `fn >> flow`.
|
|
159
|
-
- Composition flattens: `(a >> b) >> c` and `a >> (b >> c)` have the same three actions.
|
|
160
|
-
- Flows support `len()`, iteration, `.actions` and a readable `repr`.
|
|
161
|
-
|
|
162
|
-
### step
|
|
163
|
-
|
|
164
|
-
`@step` turns a function whose **first positional parameter is the subject** into a *step factory*. Calling the factory with the remaining arguments returns a single-step flow.
|
|
165
|
-
|
|
166
|
-
```python
|
|
167
|
-
from pyflowstep import step
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
@step
|
|
171
|
-
def add(total: int, amount: int) -> int:
|
|
172
|
-
return total + amount
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
@step
|
|
176
|
-
def multiply(total: int, factor: int) -> int:
|
|
177
|
-
return total * factor
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
pipeline = add(2) >> multiply(10) >> add(amount=1)
|
|
181
|
-
pipeline # Flow(add >> multiply >> add)
|
|
182
|
-
pipeline(1) # 31
|
|
183
|
-
```
|
|
184
|
-
|
|
185
|
-
`step` and `tap` do one thing: turn a function into a flow factory. Naming a step belongs to the [registry](#registry). Three markers can sit on a parameter: [`
|
|
186
|
-
|
|
187
|
-
Arguments are bound against the function signature **immediately**, so mistakes fail fast:
|
|
188
|
-
|
|
189
|
-
```python
|
|
190
|
-
add() # MissingArgumentError: missing a required argument: 'amount' for step 'add'
|
|
191
|
-
add(1, 2) # TooManyArgumentsError: too many positional arguments for step 'add'
|
|
192
|
-
add(1, x=2) # UnexpectedKeywordArgumentError: got an unexpected keyword argument 'x' ...
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
### tap
|
|
196
|
-
|
|
197
|
-
For side-effect steps on a mutable subject, `@tap` ignores the return value and passes the subject on unchanged — no more `return page` boilerplate:
|
|
198
|
-
|
|
199
|
-
```python
|
|
200
|
-
from pyflowstep import tap
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
@tap
|
|
204
|
-
def click(page: Page, selector: str) -> None:
|
|
205
|
-
page.click(selector)
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
---
|
|
209
|
-
|
|
210
|
-
## Registry
|
|
211
|
-
|
|
212
|
-
`StepsRegistry[T]` collects named steps for one subject type. It is the vocabulary of your flow language: the compiler and the JSON schema only know about the steps registered in it.
|
|
213
|
-
|
|
214
|
-
```python
|
|
215
|
-
from pyflowstep import StepsRegistry
|
|
216
|
-
|
|
217
|
-
page_steps = StepsRegistry[Page]()
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
@page_steps.tap()
|
|
221
|
-
def navigate(page: Page, url: str) -> None:
|
|
222
|
-
"""Open a url in the page."""
|
|
223
|
-
page.navigate(url)
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
@page_steps.tap()
|
|
227
|
-
def click(page: Page, selector: str) -> None:
|
|
228
|
-
"""Click the element matching a CSS selector."""
|
|
229
|
-
page.click(selector)
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
@page_steps.tap(name="type")
|
|
233
|
-
def fill(page: Page, selector: str, value: str) -> None:
|
|
234
|
-
"""Type a value into a field."""
|
|
235
|
-
page.fill(selector, value)
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
@page_steps.tap(hidden=True)
|
|
239
|
-
def debug_dump(page: Page) -> None:
|
|
240
|
-
"""Python-only helper, never exposed to JSON."""
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
page_steps["click"]("#go") # look up a step factory by name
|
|
244
|
-
"type" in page_steps # True
|
|
245
|
-
list(page_steps.steps) # ['navigate', 'click', 'type'] (hidden steps are excluded)
|
|
246
|
-
|
|
247
|
-
page_steps.register(lambda page: page, name="noop") # register without a decorator
|
|
248
|
-
```
|
|
249
|
-
|
|
250
|
-
| Method / attribute | Description |
|
|
251
|
-
| ---------------------------------------------------------------- | ----------------------------------------------------------- |
|
|
252
|
-
| `step(name=, description=, hidden=)` | Decorator for `(subject, ...) -> subject` functions |
|
|
253
|
-
| `tap(name=, description=, hidden=)` | Decorator for side-effect functions (return value ignored) |
|
|
254
|
-
| `register(fn, *, name=, description=, hidden=, passthrough=)` | Register without decorator syntax |
|
|
255
|
-
| `steps` | Read-only mapping of visible steps |
|
|
256
|
-
| `registry[name]`, `name in registry`, `len()`, iteration | Lookup over visible steps |
|
|
257
|
-
|
|
258
|
-
- `name` is the step's public name: the compiler looks it up, and `repr` and argument errors show it (`Flow(type)` above, not `Flow(fill)`).
|
|
259
|
-
- `description` overrides the docstring, which becomes the step description in the JSON schema.
|
|
260
|
-
|
|
261
|
-
---
|
|
262
|
-
|
|
263
|
-
##
|
|
264
|
-
|
|
265
|
-
Flows often come from JSON or a web form, where every value arrives as a string, number, boolean, list, object or null, while your steps want dates, decimals, enums, clean text or loaded data. Mark the parameter with `
|
|
266
|
-
|
|
267
|
-
```python
|
|
268
|
-
from typing import Annotated
|
|
269
|
-
|
|
270
|
-
from
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
@page_steps.tap()
|
|
274
|
-
def wait(page: Page, selector: str, timeout: Annotated[float,
|
|
275
|
-
page.wait(selector, timeout)
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
wait("#result", "10") # timeout is 10.0
|
|
279
|
-
```
|
|
280
|
-
|
|
281
|
-
```json
|
|
282
|
-
{"name": "wait", "args": ["#result", "10"]}
|
|
283
|
-
```
|
|
284
|
-
|
|
285
|
-
The marker sits next to the parameter it changes, and it works with the plain `@step` and `@tap` decorators too; no registry is required.
|
|
286
|
-
|
|
287
|
-
### Name it once, reuse it
|
|
288
|
-
|
|
289
|
-
A `type` alias gives a
|
|
290
|
-
|
|
291
|
-
```python
|
|
292
|
-
from decimal import Decimal
|
|
293
|
-
|
|
294
|
-
type Money = Annotated[Decimal,
|
|
295
|
-
type Text = Annotated[str,
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
@catalog_steps.step()
|
|
299
|
-
def price_between(products: Catalog, low: Money, high: Money) -> Catalog: ...
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
@catalog_steps.step()
|
|
303
|
-
def search(products: Catalog, text: Text, min_stars: Annotated[float,
|
|
304
|
-
```
|
|
305
|
-
|
|
306
|
-
### Loading data from a value in the JSON
|
|
307
|
-
|
|
308
|
-
A
|
|
309
|
-
|
|
310
|
-
```python
|
|
311
|
-
def load_names(path: str) -> frozenset[str]:
|
|
312
|
-
return frozenset(Path(path).read_text().splitlines())
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
type Names = Annotated[frozenset[str],
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
@catalog_steps.step()
|
|
319
|
-
def exclude(products: Catalog, names: Names) -> Catalog:
|
|
320
|
-
return tuple(product for product in products if product.name not in names)
|
|
321
|
-
```
|
|
322
|
-
|
|
323
|
-
```json
|
|
324
|
-
{"name": "exclude", "args": ["discontinued.txt"]}
|
|
325
|
-
```
|
|
326
|
-
|
|
327
|
-
The file is read **once, when the flow is built**. Every run of that flow reuses the loaded names. For an object that must be fresh on every run, or that the JSON must not choose at all, use a [dependency](#dependencies) instead.
|
|
328
|
-
|
|
329
|
-
### `*args` and `**kwargs`
|
|
330
|
-
|
|
331
|
-
On `*args` the
|
|
332
|
-
|
|
333
|
-
```python
|
|
334
|
-
@barista.step()
|
|
335
|
-
def top_with(drink: Drink, *toppings: Text) -> Drink: ...
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
@catalog_steps.step()
|
|
339
|
-
def where(products: Catalog, *, in_stock: Annotated[bool,
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
where(in_stock="yes", brand=" SONIC ") # in_stock=True, brand="sonic"
|
|
343
|
-
```
|
|
344
|
-
|
|
345
|
-
###
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
A
|
|
378
|
-
|
|
379
|
-
|
|
|
380
|
-
|
|
|
381
|
-
|
|
|
382
|
-
| `
|
|
383
|
-
|
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
def
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
```
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
[
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
#
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
)
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
```
|
|
654
|
-
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
720
|
-
|
|
721
|
-
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
763
|
-
|
|
764
|
-
|
|
765
|
-
|
|
766
|
-
|
|
767
|
-
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
|
|
778
|
-
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
|
|
782
|
-
|
|
783
|
-
|
|
784
|
-
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
797
|
-
|
|
798
|
-
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
|
|
802
|
-
|
|
803
|
-
|
|
804
|
-
|
|
805
|
-
|
|
806
|
-
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
811
|
-
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
|
|
25
|
+
# pyflowstep
|
|
26
|
+
|
|
27
|
+
A lightweight, typed Python library for composing functions into readable, reusable **flows** — a functional replacement for the fluent-interface (method-chaining) pattern, with flows that can be defined, validated and stored as **JSON**.
|
|
28
|
+
|
|
29
|
+
`pyflowstep` focuses on a functional style:
|
|
30
|
+
|
|
31
|
+
- steps are plain functions: `(subject, *args, **kwargs) -> subject`
|
|
32
|
+
- flows are immutable values that compose with `>>`
|
|
33
|
+
- step arguments are validated and processed when a flow is **built**, not halfway through running it
|
|
34
|
+
- raw JSON values become typed arguments with a [pyargprocessors](https://github.com/aaltatan/pyargprocessors) `Process` marker next to the parameter
|
|
35
|
+
- steps get external objects (a mailer, a database session) through FastAPI-style `Depends`
|
|
36
|
+
- the caller hands per-run values to a flow by name: `flow(page, user=user)` fills every `user: User = Input()`
|
|
37
|
+
- registry-based registration keeps steps organized and discoverable
|
|
38
|
+
- flow definitions can be compiled from dictionaries or JSON
|
|
39
|
+
- every registry can describe its flow language as a JSON Schema
|
|
40
|
+
|
|
41
|
+
It pairs naturally with its siblings [pyspecification](https://github.com/aaltatan/pyspecification) (predicates) and [pyformula](https://github.com/aaltatan/pyformula) (formulas). Its only dependency is [pyargprocessors](https://github.com/aaltatan/pyargprocessors) (argument processing), which itself has none.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Why use pyflowstep?
|
|
46
|
+
|
|
47
|
+
A fluent API makes you put every operation on the class and `return self` from each method:
|
|
48
|
+
|
|
49
|
+
```python
|
|
50
|
+
page.navigate("https://www.google.com").fill("input[name=q]", "pyflowstep").click("#go").wait("#result")
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
That couples *what can be done* to *one class*, you cannot store the chain, reuse half of it, build it from data, or add an operation without editing the class.
|
|
54
|
+
|
|
55
|
+
With `pyflowstep`, operations are small functions and the chain is a value:
|
|
56
|
+
|
|
57
|
+
```python
|
|
58
|
+
search = navigate("https://www.google.com") >> fill("input[name=q]", "pyflowstep") >> click("#go")
|
|
59
|
+
|
|
60
|
+
search(page) # run it
|
|
61
|
+
search >> wait("#result") # extend it (a new flow, `search` is unchanged)
|
|
62
|
+
login >> search >> logout # combine flows
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
…and the same flow can come from JSON:
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
[
|
|
69
|
+
{"name": "navigate", "args": ["https://www.google.com"]},
|
|
70
|
+
{"name": "fill", "args": ["input[name=q]", "pyflowstep"]},
|
|
71
|
+
{"name": "click", "kwargs": {"selector": "#go"}}
|
|
72
|
+
]
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Installation
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
pip install pyflowstep
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
or with uv:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
uv add pyflowstep
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Requires Python 3.12+.
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## Quick start
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
from typing import Any
|
|
97
|
+
|
|
98
|
+
from pyflowstep import step
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
class Page:
|
|
102
|
+
def navigate(self, url: str) -> None:
|
|
103
|
+
print(f"Navigating to {url}")
|
|
104
|
+
|
|
105
|
+
def click(self, selector: str) -> None:
|
|
106
|
+
print(f"Clicking on {selector}")
|
|
107
|
+
|
|
108
|
+
def fill(self, selector: str, value: Any) -> None:
|
|
109
|
+
print(f"Filling {selector} with {value}")
|
|
110
|
+
|
|
111
|
+
|
|
112
|
+
@step
|
|
113
|
+
def navigate(page: Page, url: str) -> Page:
|
|
114
|
+
page.navigate(url)
|
|
115
|
+
return page
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
@step
|
|
119
|
+
def fill(page: Page, selector: str, value: Any) -> Page:
|
|
120
|
+
page.fill(selector, value)
|
|
121
|
+
return page
|
|
122
|
+
|
|
123
|
+
|
|
124
|
+
@step
|
|
125
|
+
def click(page: Page, selector: str) -> Page:
|
|
126
|
+
page.click(selector)
|
|
127
|
+
return page
|
|
128
|
+
|
|
129
|
+
|
|
130
|
+
search = navigate("https://www.google.com") >> fill("input[name=q]", 1111) >> click("#go")
|
|
131
|
+
|
|
132
|
+
print(search) # Flow(navigate >> fill >> click)
|
|
133
|
+
search(Page())
|
|
134
|
+
# Navigating to https://www.google.com
|
|
135
|
+
# Filling input[name=q] with 1111
|
|
136
|
+
# Clicking on #go
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Core concepts
|
|
142
|
+
|
|
143
|
+
### Flow
|
|
144
|
+
|
|
145
|
+
A `Flow[T]` is an immutable sequence of `T -> T` actions. Calling it threads the subject through every action in order.
|
|
146
|
+
|
|
147
|
+
```python
|
|
148
|
+
from pyflowstep import Flow, compose
|
|
149
|
+
|
|
150
|
+
increment = Flow[int](lambda n: n + 1)
|
|
151
|
+
double = Flow[int](lambda n: n * 2)
|
|
152
|
+
|
|
153
|
+
(increment >> double)(3) # 8
|
|
154
|
+
(double >> increment)(3) # 7
|
|
155
|
+
compose(increment, double)(3) # 8, same as increment >> double
|
|
156
|
+
Flow[int]()(3) # 3, an empty flow is the identity
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
- `>>` accepts flows **and** plain callables, on either side: `flow >> fn`, `fn >> flow`.
|
|
160
|
+
- Composition flattens: `(a >> b) >> c` and `a >> (b >> c)` have the same three actions.
|
|
161
|
+
- Flows support `len()`, iteration, `.actions` and a readable `repr`.
|
|
162
|
+
|
|
163
|
+
### step
|
|
164
|
+
|
|
165
|
+
`@step` turns a function whose **first positional parameter is the subject** into a *step factory*. Calling the factory with the remaining arguments returns a single-step flow.
|
|
166
|
+
|
|
167
|
+
```python
|
|
168
|
+
from pyflowstep import step
|
|
169
|
+
|
|
170
|
+
|
|
171
|
+
@step
|
|
172
|
+
def add(total: int, amount: int) -> int:
|
|
173
|
+
return total + amount
|
|
174
|
+
|
|
175
|
+
|
|
176
|
+
@step
|
|
177
|
+
def multiply(total: int, factor: int) -> int:
|
|
178
|
+
return total * factor
|
|
179
|
+
|
|
180
|
+
|
|
181
|
+
pipeline = add(2) >> multiply(10) >> add(amount=1)
|
|
182
|
+
pipeline # Flow(add >> multiply >> add)
|
|
183
|
+
pipeline(1) # 31
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
`step` and `tap` do one thing: turn a function into a flow factory. Naming a step belongs to the [registry](#registry). Three markers can sit on a parameter: [`Process`](#processing-arguments) to convert the value passed for it, [`Depends`](#dependencies) to inject an object nobody passes, and [`Input`](#run-inputs) to receive a value from whoever runs the flow.
|
|
187
|
+
|
|
188
|
+
Arguments are bound against the function signature **immediately**, so mistakes fail fast:
|
|
189
|
+
|
|
190
|
+
```python
|
|
191
|
+
add() # MissingArgumentError: missing a required argument: 'amount' for step 'add'
|
|
192
|
+
add(1, 2) # TooManyArgumentsError: too many positional arguments for step 'add'
|
|
193
|
+
add(1, x=2) # UnexpectedKeywordArgumentError: got an unexpected keyword argument 'x' ...
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
### tap
|
|
197
|
+
|
|
198
|
+
For side-effect steps on a mutable subject, `@tap` ignores the return value and passes the subject on unchanged — no more `return page` boilerplate:
|
|
199
|
+
|
|
200
|
+
```python
|
|
201
|
+
from pyflowstep import tap
|
|
202
|
+
|
|
203
|
+
|
|
204
|
+
@tap
|
|
205
|
+
def click(page: Page, selector: str) -> None:
|
|
206
|
+
page.click(selector)
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
---
|
|
210
|
+
|
|
211
|
+
## Registry
|
|
212
|
+
|
|
213
|
+
`StepsRegistry[T]` collects named steps for one subject type. It is the vocabulary of your flow language: the compiler and the JSON schema only know about the steps registered in it.
|
|
214
|
+
|
|
215
|
+
```python
|
|
216
|
+
from pyflowstep import StepsRegistry
|
|
217
|
+
|
|
218
|
+
page_steps = StepsRegistry[Page]()
|
|
219
|
+
|
|
220
|
+
|
|
221
|
+
@page_steps.tap()
|
|
222
|
+
def navigate(page: Page, url: str) -> None:
|
|
223
|
+
"""Open a url in the page."""
|
|
224
|
+
page.navigate(url)
|
|
225
|
+
|
|
226
|
+
|
|
227
|
+
@page_steps.tap()
|
|
228
|
+
def click(page: Page, selector: str) -> None:
|
|
229
|
+
"""Click the element matching a CSS selector."""
|
|
230
|
+
page.click(selector)
|
|
231
|
+
|
|
232
|
+
|
|
233
|
+
@page_steps.tap(name="type")
|
|
234
|
+
def fill(page: Page, selector: str, value: str) -> None:
|
|
235
|
+
"""Type a value into a field."""
|
|
236
|
+
page.fill(selector, value)
|
|
237
|
+
|
|
238
|
+
|
|
239
|
+
@page_steps.tap(hidden=True)
|
|
240
|
+
def debug_dump(page: Page) -> None:
|
|
241
|
+
"""Python-only helper, never exposed to JSON."""
|
|
242
|
+
|
|
243
|
+
|
|
244
|
+
page_steps["click"]("#go") # look up a step factory by name
|
|
245
|
+
"type" in page_steps # True
|
|
246
|
+
list(page_steps.steps) # ['navigate', 'click', 'type'] (hidden steps are excluded)
|
|
247
|
+
|
|
248
|
+
page_steps.register(lambda page: page, name="noop") # register without a decorator
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
| Method / attribute | Description |
|
|
252
|
+
| ---------------------------------------------------------------- | ----------------------------------------------------------- |
|
|
253
|
+
| `step(name=, description=, hidden=)` | Decorator for `(subject, ...) -> subject` functions |
|
|
254
|
+
| `tap(name=, description=, hidden=)` | Decorator for side-effect functions (return value ignored) |
|
|
255
|
+
| `register(fn, *, name=, description=, hidden=, passthrough=)` | Register without decorator syntax |
|
|
256
|
+
| `steps` | Read-only mapping of visible steps |
|
|
257
|
+
| `registry[name]`, `name in registry`, `len()`, iteration | Lookup over visible steps |
|
|
258
|
+
|
|
259
|
+
- `name` is the step's public name: the compiler looks it up, and `repr` and argument errors show it (`Flow(type)` above, not `Flow(fill)`).
|
|
260
|
+
- `description` overrides the docstring, which becomes the step description in the JSON schema.
|
|
261
|
+
|
|
262
|
+
---
|
|
263
|
+
|
|
264
|
+
## Processing arguments
|
|
265
|
+
|
|
266
|
+
Flows often come from JSON or a web form, where every value arrives as a string, number, boolean, list, object or null, while your steps want dates, decimals, enums, clean text or loaded data. Mark the parameter with the `Process(fn)` marker of [pyargprocessors](https://github.com/aaltatan/pyargprocessors) inside `Annotated`, and `fn` is applied to the value that is passed for it:
|
|
267
|
+
|
|
268
|
+
```python
|
|
269
|
+
from typing import Annotated
|
|
270
|
+
|
|
271
|
+
from pyargprocessors import Process
|
|
272
|
+
|
|
273
|
+
|
|
274
|
+
@page_steps.tap()
|
|
275
|
+
def wait(page: Page, selector: str, timeout: Annotated[float, Process(float)] = 5.0) -> None:
|
|
276
|
+
page.wait(selector, timeout)
|
|
277
|
+
|
|
278
|
+
|
|
279
|
+
wait("#result", "10") # timeout is 10.0
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
```json
|
|
283
|
+
{"name": "wait", "args": ["#result", "10"]}
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
The marker sits next to the parameter it changes, and it works with the plain `@step` and `@tap` decorators too; no registry is required. `pyargprocessors` is installed together with `pyflowstep`.
|
|
287
|
+
|
|
288
|
+
### Name it once, reuse it
|
|
289
|
+
|
|
290
|
+
A `type` alias gives a processed type a name, so many steps can share it:
|
|
291
|
+
|
|
292
|
+
```python
|
|
293
|
+
from decimal import Decimal
|
|
294
|
+
|
|
295
|
+
type Money = Annotated[Decimal, Process(Decimal)]
|
|
296
|
+
type Text = Annotated[str, Process(str.strip), Process(str.lower)] # several processors run left to right
|
|
297
|
+
|
|
298
|
+
|
|
299
|
+
@catalog_steps.step()
|
|
300
|
+
def price_between(products: Catalog, low: Money, high: Money) -> Catalog: ...
|
|
301
|
+
|
|
302
|
+
|
|
303
|
+
@catalog_steps.step()
|
|
304
|
+
def search(products: Catalog, text: Text, min_stars: Annotated[float, Process(float)] = 0) -> Catalog: ...
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
### Loading data from a value in the JSON
|
|
308
|
+
|
|
309
|
+
A processor can be any function of one value, so the JSON can send a path and the step can receive what was loaded from it:
|
|
310
|
+
|
|
311
|
+
```python
|
|
312
|
+
def load_names(path: str) -> frozenset[str]:
|
|
313
|
+
return frozenset(Path(path).read_text().splitlines())
|
|
314
|
+
|
|
315
|
+
|
|
316
|
+
type Names = Annotated[frozenset[str], Process(load_names)]
|
|
317
|
+
|
|
318
|
+
|
|
319
|
+
@catalog_steps.step()
|
|
320
|
+
def exclude(products: Catalog, names: Names) -> Catalog:
|
|
321
|
+
return tuple(product for product in products if product.name not in names)
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
```json
|
|
325
|
+
{"name": "exclude", "args": ["discontinued.txt"]}
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
The file is read **once, when the flow is built**. Every run of that flow reuses the loaded names. For an object that must be fresh on every run, or that the JSON must not choose at all, use a [dependency](#dependencies) instead.
|
|
329
|
+
|
|
330
|
+
### `*args` and `**kwargs`
|
|
331
|
+
|
|
332
|
+
On `*args` the processor applies to each item, and on `**kwargs` to each value:
|
|
333
|
+
|
|
334
|
+
```python
|
|
335
|
+
@barista.step()
|
|
336
|
+
def top_with(drink: Drink, *toppings: Text) -> Drink: ...
|
|
337
|
+
|
|
338
|
+
|
|
339
|
+
@catalog_steps.step()
|
|
340
|
+
def where(products: Catalog, *, in_stock: Annotated[bool, Process(to_bool)] = False, **fields: Text) -> Catalog: ...
|
|
341
|
+
|
|
342
|
+
|
|
343
|
+
where(in_stock="yes", brand=" SONIC ") # in_stock=True, brand="sonic"
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
### Lists and nulls
|
|
347
|
+
|
|
348
|
+
A JSON list or a `null` is one value, so `Process(int)` would be applied to the whole of it. The `each` and `optional` helpers of `pyargprocessors` build the processor you need:
|
|
349
|
+
|
|
350
|
+
```python
|
|
351
|
+
from pyargprocessors import each, optional
|
|
352
|
+
|
|
353
|
+
type Prices = Annotated[list[Decimal], Process(each(Decimal))] # every item of a list
|
|
354
|
+
type Limit = Annotated[int | None, Process(optional(int))] # null is kept as None
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
### Which values are processed, and when
|
|
358
|
+
|
|
359
|
+
- Only values that are **actually passed**, positionally or by keyword. Default values are never processed.
|
|
360
|
+
- Processing runs **once per factory call**, while the flow is being built, not every time the flow runs.
|
|
361
|
+
- Arguments are checked against the step's signature first, so a missing or extra argument is reported as an `ArgumentError` before any processor runs.
|
|
362
|
+
|
|
363
|
+
```python
|
|
364
|
+
flow = add_syrup(" Vanilla ", "2") # processed here: flavor="vanilla", pumps=2
|
|
365
|
+
flow(drink) # the step runs with the processed values
|
|
366
|
+
flow(another_drink) # nothing is processed again
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
### Errors
|
|
370
|
+
|
|
371
|
+
A processor that fails on a value raises `ProcessArgumentError`, chained to the original exception. It has the name of the argument (`error.argument`), the value as it was passed (`error.value`) and the original exception (`error.cause`). Inside a compiled flow it also carries the JSON path of the step, so a bad value is found before anything runs:
|
|
372
|
+
|
|
373
|
+
```text
|
|
374
|
+
pyargprocessors.exceptions.ProcessArgumentError: Argument 'limit' with value 'two' failed to process, invalid literal for int() with base 10: 'two'
|
|
375
|
+
at $[1]
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
A marker that cannot work raises `InvalidProcessorError` when the step is created:
|
|
379
|
+
|
|
380
|
+
| Problem | Example |
|
|
381
|
+
| ---------------------------------------------------- | ---------------------------------------------------- |
|
|
382
|
+
| The processor is not callable | `Process("int")` |
|
|
383
|
+
| The marker is used as a default value | `amount: int = Process(int)`, write `Annotated[int, Process(int)]` |
|
|
384
|
+
| The marker is on the subject (the first parameter) | `def step(page: Annotated[Page, Process(...)], ...)` |
|
|
385
|
+
| One parameter has both `Process` and `Depends` or `Input` | nothing is passed for it, so there is nothing to process |
|
|
386
|
+
|
|
387
|
+
Both exceptions come from `pyargprocessors`. They are `TypeError`s and derive from `PyargprocessorsError`, **not** from `PyflowstepError`:
|
|
388
|
+
|
|
389
|
+
```python
|
|
390
|
+
from pyargprocessors import ProcessArgumentError
|
|
391
|
+
|
|
392
|
+
try:
|
|
393
|
+
flow = compiler.compile(definition)
|
|
394
|
+
except ProcessArgumentError as error: # a value in the definition is wrong
|
|
395
|
+
...
|
|
396
|
+
except PyflowstepError as error: # the definition itself is wrong
|
|
397
|
+
...
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
### In the JSON schema
|
|
401
|
+
|
|
402
|
+
A processed parameter is described by **what the JSON must send**. The schema takes it from the type hint of the processor's own parameter (`load_names(path: str)` means `string`). If the processor has no type hint, as with `int`, `Decimal` or an enum class, the schema falls back to the parameter's type.
|
|
403
|
+
|
|
404
|
+
| Parameter | Schema |
|
|
405
|
+
| -------------------------------------------- | ------------------------------- |
|
|
406
|
+
| `names: Annotated[frozenset[str], Process(load_names)]` | `{"type": "string"}` |
|
|
407
|
+
| `limit: Annotated[int, Process(int)]` | `{"type": "integer"}` |
|
|
408
|
+
| `kind: Annotated[Milk, Process(Milk)]` | `{"enum": ["whole", "oat", "almond"], "type": "string"}` |
|
|
409
|
+
|
|
410
|
+
### Upgrading
|
|
411
|
+
|
|
412
|
+
Argument processing moved to `pyargprocessors`, and `Parse` became its `Process`:
|
|
413
|
+
|
|
414
|
+
| Before | Now |
|
|
415
|
+
| ------------------------------------------------- | ------------------------------------------------------- |
|
|
416
|
+
| `from pyflowstep import Parse` | `from pyargprocessors import Process` |
|
|
417
|
+
| `Annotated[float, Parse(float)]` | `Annotated[float, Process(float)]` |
|
|
418
|
+
| `pyflowstep.ParseArgumentError` | `pyargprocessors.ProcessArgumentError`, no longer a `PyflowstepError` |
|
|
419
|
+
| `pyflowstep.InvalidParserError` | `pyargprocessors.InvalidProcessorError`, no longer a `PyflowstepError` |
|
|
420
|
+
| `... failed to parse, ...` in the message | `... failed to process, ...` |
|
|
421
|
+
|
|
422
|
+
Coming from 0.1, the `processors=` option of the registry is gone as well. Move each entry onto its parameter:
|
|
423
|
+
|
|
424
|
+
| 0.1 | Now |
|
|
425
|
+
| ------------------------------------------------- | ------------------------------------------------------- |
|
|
426
|
+
| `processors={"timeout": float}` | `timeout: Annotated[float, Process(float)]` |
|
|
427
|
+
| `processors=normalize` (every argument) | annotate each parameter, usually with a shared alias such as `Text` |
|
|
428
|
+
| `processors={"pumps": int, ...: normalize}` | `pumps: Annotated[int, Process(int)]`, the others `Text` |
|
|
429
|
+
| `...` covering `**kwargs` | `**fields: Text` |
|
|
430
|
+
| `InvalidProcessorsError` | removed; a key can no longer be misspelled |
|
|
431
|
+
|
|
432
|
+
See [`examples/processing.py`](examples/processing.py) for every form working together on data from a web form.
|
|
433
|
+
|
|
434
|
+
---
|
|
435
|
+
|
|
436
|
+
## Dependencies
|
|
437
|
+
|
|
438
|
+
Some steps need objects that cannot be written in JSON: a mailer, a database session, an API client. Mark the parameter with `Depends(provider)` and it stops being an argument. Nobody passes it, neither Python nor JSON; `provider` is called to produce it when the flow runs.
|
|
439
|
+
|
|
440
|
+
```python
|
|
441
|
+
from pyflowstep import Depends, StepsRegistry
|
|
442
|
+
|
|
443
|
+
steps = StepsRegistry[Order]()
|
|
444
|
+
|
|
445
|
+
|
|
446
|
+
def get_mailer() -> Mailer:
|
|
447
|
+
return Mailer("smtp.example.com")
|
|
448
|
+
|
|
449
|
+
|
|
450
|
+
@steps.tap()
|
|
451
|
+
def send_email(order: Order, template: str, mailer: Mailer = Depends(get_mailer)) -> None:
|
|
452
|
+
mailer.send(order.customer, template)
|
|
453
|
+
|
|
454
|
+
|
|
455
|
+
send_email("receipt") # only `template` is an argument
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
```json
|
|
459
|
+
[{"name": "send_email", "args": ["receipt"]}]
|
|
460
|
+
```
|
|
461
|
+
|
|
462
|
+
It works the same with the plain `@step` and `@tap` decorators; no registry is required.
|
|
463
|
+
|
|
464
|
+
### Always the default value
|
|
465
|
+
|
|
466
|
+
`Depends(...)` is written as the default value of the parameter, in steps and in providers. To name a dependency once and reuse it in many steps, keep the marker in a constant:
|
|
467
|
+
|
|
468
|
+
```python
|
|
469
|
+
# inline: quick, for a one-off
|
|
470
|
+
def send_email(order: Order, template: str, mailer: Mailer = Depends(get_mailer)) -> None: ...
|
|
471
|
+
|
|
472
|
+
|
|
473
|
+
# a constant: name the dependency once, reuse it
|
|
474
|
+
MAILER = Depends(get_mailer)
|
|
475
|
+
|
|
476
|
+
def send_email(order: Order, template: str, mailer: Mailer = MAILER) -> None: ...
|
|
477
|
+
def send_invoice(order: Order, mailer: Mailer = MAILER) -> None: ...
|
|
478
|
+
```
|
|
479
|
+
|
|
480
|
+
It is never written inside `Annotated`. Only a default value tells a type checker that the parameter is not passed, so `send_email("receipt")` type-checks; `Annotated[Mailer, Depends(get_mailer)]` raises `InvalidDependencyError` when the step is created.
|
|
481
|
+
|
|
482
|
+
### One object per flow run
|
|
483
|
+
|
|
484
|
+
A flow run is one scope, like one request in a web framework. A provider is called **at most once per run**, and every step of that run receives the same object. The next run starts fresh.
|
|
485
|
+
|
|
486
|
+
A provider written as a generator is cleaned up when the run ends. If a step fails, the error is raised inside the provider at its `yield`, so it can roll back:
|
|
487
|
+
|
|
488
|
+
```python
|
|
489
|
+
from collections.abc import Iterator
|
|
490
|
+
|
|
491
|
+
|
|
492
|
+
def get_session() -> Iterator[Session]:
|
|
493
|
+
session = Session()
|
|
494
|
+
try:
|
|
495
|
+
yield session # shared by every step of this run
|
|
496
|
+
except Exception:
|
|
497
|
+
session.rollback() # a step failed
|
|
498
|
+
raise
|
|
499
|
+
else:
|
|
500
|
+
session.commit() # the whole flow succeeded
|
|
501
|
+
finally:
|
|
502
|
+
session.close()
|
|
503
|
+
|
|
504
|
+
|
|
505
|
+
SESSION = Depends(get_session)
|
|
506
|
+
|
|
507
|
+
|
|
508
|
+
@steps.tap()
|
|
509
|
+
def save(order: Order, session: Session = SESSION) -> None:
|
|
510
|
+
session.add(order)
|
|
511
|
+
|
|
512
|
+
|
|
513
|
+
@steps.tap()
|
|
514
|
+
def audit(order: Order, action: str, session: Session = SESSION) -> None:
|
|
515
|
+
session.add(AuditRow(order.id, action)) # the same session `save` used
|
|
516
|
+
|
|
517
|
+
|
|
518
|
+
flow = save() >> audit("fulfilled")
|
|
519
|
+
|
|
520
|
+
flow(order_1) # session A: opened, used twice, committed, closed
|
|
521
|
+
flow(order_2) # session B
|
|
522
|
+
```
|
|
523
|
+
|
|
524
|
+
Cleanups run in reverse order, and a provider cannot swallow a step's error: it is always re-raised. Flows nested inside a flow, or called from inside a step, join the run that is already open.
|
|
525
|
+
|
|
526
|
+
### Sub-dependencies
|
|
527
|
+
|
|
528
|
+
A provider's own parameters can use `Depends` too:
|
|
529
|
+
|
|
530
|
+
```python
|
|
531
|
+
def get_settings() -> Settings:
|
|
532
|
+
return Settings()
|
|
533
|
+
|
|
534
|
+
|
|
535
|
+
def get_mailer(settings: Settings = Depends(get_settings)) -> Mailer:
|
|
536
|
+
return Mailer(settings.smtp_host)
|
|
537
|
+
```
|
|
538
|
+
|
|
539
|
+
Any other provider parameter must have a default. A provider can be any callable: a function, a class, a `functools.partial`, or an object with `__call__`.
|
|
540
|
+
|
|
541
|
+
### Replacing a dependency in tests
|
|
542
|
+
|
|
543
|
+
```python
|
|
544
|
+
from pyflowstep import override_dependencies
|
|
545
|
+
|
|
546
|
+
with override_dependencies({get_mailer: FakeMailer}):
|
|
547
|
+
flow(order) # every step that asked for get_mailer gets a FakeMailer
|
|
548
|
+
|
|
549
|
+
flow(order) # the real mailer again
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
Keys are the original providers and values the providers to call instead. Nested blocks add up, and everything is restored when the block exits, even after an error.
|
|
553
|
+
|
|
554
|
+
### Rules
|
|
555
|
+
|
|
556
|
+
- **Invisible to JSON.** A dependency is absent from the JSON schema, cannot be processed, and passing one raises `UnexpectedKeywordArgumentError` (or `TooManyArgumentsError`) when the flow is built.
|
|
557
|
+
- **Declarations are checked early**, when the step is created, with `InvalidDependencyError`: a provider that is not callable, is circular, or has a required parameter that is not a dependency; a dependency on the subject or on a positional-only parameter; `Depends` written inside `Annotated`.
|
|
558
|
+
- **Providers run late**, when the flow runs. An error inside a provider surfaces then, not at compile time.
|
|
559
|
+
- A dependency is for an object the JSON must **not** choose. When the JSON should pick one by name (`"via": "email"`), use [`Process`](#processing-arguments) with a function that turns the name into the object.
|
|
560
|
+
|
|
561
|
+
Using ruff? Its `B008` rule flags function calls in argument defaults. Tell it `Depends` is a marker:
|
|
562
|
+
|
|
563
|
+
```toml
|
|
564
|
+
[tool.ruff.lint.flake8-bugbear]
|
|
565
|
+
extend-immutable-calls = ["pyflowstep.Depends"]
|
|
566
|
+
```
|
|
567
|
+
|
|
568
|
+
See [`examples/dependencies.py`](examples/dependencies.py) for a complete flow with a mailer, a per-run session and a test override.
|
|
569
|
+
|
|
570
|
+
### Upgrading from 0.4
|
|
571
|
+
|
|
572
|
+
`Annotated[T, Depends(provider)]` is gone, for the same reason `Input[T]` went in 0.4: a type checker reported `send_email("receipt")` as missing its `mailer` argument. Move the marker to the default value, in steps and in providers:
|
|
573
|
+
|
|
574
|
+
| 0.4 | 0.5 |
|
|
575
|
+
| --------------------------------------------------------- | -------------------------------------------- |
|
|
576
|
+
| `mailer: Annotated[Mailer, Depends(get_mailer)]` | `mailer: Mailer = Depends(get_mailer)` |
|
|
577
|
+
| `type MailerDep = Annotated[Mailer, Depends(get_mailer)]` | `MAILER = Depends(get_mailer)` |
|
|
578
|
+
| `mailer: MailerDep` | `mailer: Mailer = MAILER` |
|
|
579
|
+
|
|
580
|
+
---
|
|
581
|
+
|
|
582
|
+
## Run inputs
|
|
583
|
+
|
|
584
|
+
Some values exist only where the flow is run: the logged-in user, the record being processed, credentials read from a prompt. No provider can build them and JSON must not hold them. Give the parameter the default `Input()` and the caller supplies it, by name, when it runs the flow:
|
|
585
|
+
|
|
586
|
+
```python
|
|
587
|
+
from pyflowstep import Input, StepsRegistry
|
|
588
|
+
|
|
589
|
+
steps = StepsRegistry[Page]()
|
|
590
|
+
|
|
591
|
+
|
|
592
|
+
@steps.tap()
|
|
593
|
+
def login(page: Page, url: str, credentials: Credentials = Input()) -> None:
|
|
594
|
+
page.navigate(url)
|
|
595
|
+
page.fill("#user", credentials.username)
|
|
596
|
+
page.fill("#password", credentials.password)
|
|
597
|
+
|
|
598
|
+
|
|
599
|
+
@steps.tap()
|
|
600
|
+
def fill_year(page: Page, selector: str, voucher: Voucher = Input()) -> None:
|
|
601
|
+
page.fill(selector, voucher.year)
|
|
602
|
+
|
|
603
|
+
|
|
604
|
+
flow = login("https://example.com/login") >> fill_year("#year") # inputs are not arguments
|
|
605
|
+
|
|
606
|
+
flow(page, credentials=credentials, voucher=voucher) # they are given to the run
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
```json
|
|
610
|
+
[
|
|
611
|
+
{"name": "login", "args": ["https://example.com/login"]},
|
|
612
|
+
{"name": "fill_year", "args": ["#year"]}
|
|
613
|
+
]
|
|
614
|
+
```
|
|
615
|
+
|
|
616
|
+
The name of the input is the name of the parameter, and every step that declares it receives the same value.
|
|
617
|
+
|
|
618
|
+
`Input()` is written as the default value, never inside `Annotated`, so that type checkers see the parameter as optional and accept `fill_year("#year")`.
|
|
619
|
+
|
|
620
|
+
### Checked before anything runs
|
|
621
|
+
|
|
622
|
+
A flow knows the inputs its steps require, and checks them before the first step runs. A forgotten input never fails halfway through:
|
|
623
|
+
|
|
624
|
+
```python
|
|
625
|
+
flow.inputs # frozenset({'credentials', 'voucher'})
|
|
626
|
+
|
|
627
|
+
flow(page, credentials=credentials)
|
|
628
|
+
# MissingInputError: missing run input 'voucher' for step 'fill_year'; run the flow as flow(subject, voucher=...)
|
|
629
|
+
```
|
|
630
|
+
|
|
631
|
+
### Typed calls
|
|
632
|
+
|
|
633
|
+
A flow is typed as `flow(obj, **inputs: Any)`, because Python's typing cannot merge the inputs of several steps into one signature. To have a type checker check the inputs, write the signature once as a stub and decorate it with `flow.signature`:
|
|
634
|
+
|
|
635
|
+
```python
|
|
636
|
+
@flow.signature
|
|
637
|
+
def log_in_and_fill(page: Page, /, *, credentials: Credentials, voucher: Voucher) -> Page: ...
|
|
638
|
+
|
|
639
|
+
log_in_and_fill(page, credentials=credentials) # type checker: missing argument "voucher"
|
|
640
|
+
```
|
|
641
|
+
|
|
642
|
+
The subject is positional-only and every input keyword-only. Required inputs have no default; optional ones need one (`= ...` is fine), but the step's own default is what gets used. The stub is checked against the steps when it is decorated, and `InvalidInputError` names any input that is missing, unknown, or required when it should be optional (or the reverse). Types are not compared, only names. mypy reports a `...` body as `empty-body`; write `raise NotImplementedError` instead if that bothers you.
|
|
643
|
+
|
|
644
|
+
### Rules
|
|
645
|
+
|
|
646
|
+
- **Invisible to JSON**, like a dependency: an input is absent from the JSON schema, cannot be processed, and passing one while building the flow raises `UnexpectedKeywordArgumentError` (or `TooManyArgumentsError`).
|
|
647
|
+
- **`Input(default=...)` makes the input optional**: with `note: str = Input(default="")`, `""` is used when the caller passes no `note`. Optional inputs are not listed in `flow.inputs`.
|
|
648
|
+
- **Extra inputs are ignored**, so one caller can run different flows, each using the inputs it needs.
|
|
649
|
+
- **Nested flows see the inputs of the run they join.** A flow called from inside a step can be given inputs of its own, `inner(page, voucher=other)`; they are laid over the outer ones for that call only.
|
|
650
|
+
- **Declarations are checked early**, when the step is created, with `InvalidInputError`: an input on the subject, on a positional-only parameter, or written inside `Annotated`. A provider cannot take an input.
|
|
651
|
+
|
|
652
|
+
Using ruff? Add `Input` next to `Depends` for its `B008` rule:
|
|
653
|
+
|
|
654
|
+
```toml
|
|
655
|
+
[tool.ruff.lint.flake8-bugbear]
|
|
656
|
+
extend-immutable-calls = ["pyflowstep.Depends", "pyflowstep.Input"]
|
|
657
|
+
```
|
|
658
|
+
|
|
659
|
+
### Upgrading from 0.3
|
|
660
|
+
|
|
661
|
+
`Input[T]` is gone, because a type checker reported `fill_year("#year")` as missing its `voucher` argument. Move the marker to the default value:
|
|
662
|
+
|
|
663
|
+
| 0.3 | 0.4 |
|
|
664
|
+
| ---------------------------------- | ---------------------------------------- |
|
|
665
|
+
| `voucher: Input[Voucher]` | `voucher: Voucher = Input()` |
|
|
666
|
+
| `note: Input[str] = ""` | `note: str = Input(default="")` |
|
|
667
|
+
|
|
668
|
+
### `Input` or `Depends`?
|
|
669
|
+
|
|
670
|
+
| The object… | Use |
|
|
671
|
+
| ------------------------------------------------------------ | ------------------------------------- |
|
|
672
|
+
| is written in the flow definition (a selector, a limit) | a plain argument, with `Process` if needed |
|
|
673
|
+
| can be built by a function, the same way for every caller (a mailer, a session) | `Depends(provider)` |
|
|
674
|
+
| is only known to whoever runs the flow (a user, a record, credentials) | `Input()` |
|
|
675
|
+
|
|
676
|
+
---
|
|
677
|
+
|
|
678
|
+
## Compiling flows from JSON
|
|
679
|
+
|
|
680
|
+
`FlowCompiler` turns a flow definition — a list of step dictionaries — into a `Flow`.
|
|
681
|
+
|
|
682
|
+
```python
|
|
683
|
+
from pyflowstep import FlowCompiler
|
|
684
|
+
|
|
685
|
+
compiler = FlowCompiler(page_steps.steps)
|
|
686
|
+
|
|
687
|
+
login = compiler.compile(
|
|
688
|
+
[
|
|
689
|
+
{"name": "navigate", "args": ["https://example.com/login"]},
|
|
690
|
+
{"name": "type", "args": ["#email", "ada@example.com"]},
|
|
691
|
+
{"name": "click", "kwargs": {"selector": "button[type=submit]"}},
|
|
692
|
+
],
|
|
693
|
+
)
|
|
694
|
+
|
|
695
|
+
login(Page())
|
|
696
|
+
```
|
|
697
|
+
|
|
698
|
+
`compile` takes already-parsed data, so a database row or an API payload works too.
|
|
699
|
+
|
|
700
|
+
### Flow files
|
|
701
|
+
|
|
702
|
+
`compile_file` picks a parser by the file suffix. JSON and TOML are built in:
|
|
703
|
+
|
|
704
|
+
```python
|
|
705
|
+
login = compiler.compile_file("login.json") # a list of steps, as above
|
|
706
|
+
login = compiler.compile_file("login.toml")
|
|
707
|
+
```
|
|
708
|
+
|
|
709
|
+
A TOML flow lists its steps as `[[steps]]` tables:
|
|
710
|
+
|
|
711
|
+
```toml
|
|
712
|
+
[[steps]]
|
|
713
|
+
name = "navigate"
|
|
714
|
+
args = ["https://example.com/login"]
|
|
715
|
+
|
|
716
|
+
[[steps]]
|
|
717
|
+
name = "click"
|
|
718
|
+
kwargs = { selector = "button[type=submit]" }
|
|
719
|
+
```
|
|
720
|
+
|
|
721
|
+
Add any other format by registering a parser that turns the file text into a list of steps:
|
|
722
|
+
|
|
723
|
+
```python
|
|
724
|
+
import yaml
|
|
725
|
+
|
|
726
|
+
from pyflowstep import register_flow_format
|
|
727
|
+
|
|
728
|
+
register_flow_format(".yaml", yaml.safe_load)
|
|
729
|
+
register_flow_format(".yml", yaml.safe_load)
|
|
730
|
+
```
|
|
731
|
+
|
|
732
|
+
An unregistered suffix raises `UnsupportedFlowFormatError`; a file that does not parse raises `InvalidFlowDefinitionError`. Errors from a file carry an extra `in <path>` note. `load_flow_definition(path)` returns the parsed data without compiling it.
|
|
733
|
+
|
|
734
|
+
Each step dictionary has this shape — only `name` is required:
|
|
735
|
+
|
|
736
|
+
```json
|
|
737
|
+
{"name": "fill", "args": ["#email"], "kwargs": {"value": "ada@example.com"}}
|
|
738
|
+
```
|
|
739
|
+
|
|
740
|
+
Everything is validated while compiling, before any step runs. Errors carry a note with their JSON path:
|
|
741
|
+
|
|
742
|
+
```text
|
|
743
|
+
pyargprocessors.exceptions.ProcessArgumentError: Argument 'url' with value 'http://insecure.example.com' failed to process, only https urls are allowed
|
|
744
|
+
at $[1]
|
|
745
|
+
```
|
|
746
|
+
|
|
747
|
+
| Problem | Exception |
|
|
748
|
+
| ------------------------------------------------ | ------------------------------------------------------ |
|
|
749
|
+
| Not a list / malformed step dict / bad file | `InvalidFlowDefinitionError` |
|
|
750
|
+
| Unknown or hidden step name | `StepDoesNotExistError` (lists the available steps) |
|
|
751
|
+
| Missing / extra / duplicated arguments | `MissingArgumentError`, `TooManyArgumentsError`, ... |
|
|
752
|
+
| A processor rejects a value | `ProcessArgumentError` (from `pyargprocessors`) |
|
|
753
|
+
|
|
754
|
+
A compiled flow is an ordinary `Flow`, so it composes with Python steps: `login >> click("#profile")`.
|
|
755
|
+
|
|
756
|
+
---
|
|
757
|
+
|
|
758
|
+
## JSON Schema
|
|
759
|
+
|
|
760
|
+
Describe your flow language so a UI, a validator, or an LLM can produce valid flows:
|
|
761
|
+
|
|
762
|
+
```python
|
|
763
|
+
import json
|
|
764
|
+
|
|
765
|
+
from pyflowstep import get_flow_json_schema, get_step_json_schema
|
|
766
|
+
|
|
767
|
+
print(json.dumps(get_flow_json_schema(page_steps.steps), indent=2))
|
|
768
|
+
```
|
|
769
|
+
|
|
770
|
+
Output, trimmed to the `click` step:
|
|
771
|
+
|
|
772
|
+
```json
|
|
773
|
+
{
|
|
774
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
775
|
+
"type": "array",
|
|
776
|
+
"items": {
|
|
777
|
+
"oneOf": [
|
|
778
|
+
{
|
|
779
|
+
"type": "object",
|
|
780
|
+
"properties": {
|
|
781
|
+
"name": {"const": "click"},
|
|
782
|
+
"args": {"type": "array", "prefixItems": [{"type": "string"}], "items": false},
|
|
783
|
+
"kwargs": {
|
|
784
|
+
"type": "object",
|
|
785
|
+
"properties": {"selector": {"type": "string"}},
|
|
786
|
+
"required": [],
|
|
787
|
+
"additionalProperties": false
|
|
788
|
+
}
|
|
789
|
+
},
|
|
790
|
+
"required": ["name"],
|
|
791
|
+
"additionalProperties": false,
|
|
792
|
+
"description": "Click the element matching a CSS selector."
|
|
793
|
+
}
|
|
794
|
+
]
|
|
795
|
+
}
|
|
796
|
+
}
|
|
797
|
+
```
|
|
798
|
+
|
|
799
|
+
Supported annotations: `str`, `int`, `float`, `bool`, `None`, `Decimal`, `datetime`, `date`, `time`, `UUID`, `list`/`tuple`/`set`/`frozenset`, `dict`, `Literal`, `Enum`, unions, `Annotated`, `TypedDict`, and `type` aliases. Anything else maps to `{}` (any value). JSON-compatible default values are included as `default`.
|
|
800
|
+
|
|
801
|
+
---
|
|
802
|
+
|
|
803
|
+
## Examples
|
|
804
|
+
|
|
805
|
+
Runnable examples live in [`examples/`](examples):
|
|
806
|
+
|
|
807
|
+
- [`examples/browser.py`](examples/browser.py) — the `Page` automation above: `tap` steps, an https-only processor, reusable sub-flows and a JSON login scenario.
|
|
808
|
+
|
|
809
|
+
```bash
|
|
810
|
+
uv run python -m examples.browser
|
|
811
|
+
```
|
|
812
|
+
|
|
813
|
+
- [`examples/coffee.py`](examples/coffee.py) — a coffee shop where every step returns a **new** immutable `Drink`. The menu is JSON, orders are customized with Python steps, and a hidden staff-only `discount` step is invisible to the menu.
|
|
814
|
+
|
|
815
|
+
```bash
|
|
816
|
+
uv run python -m examples.coffee
|
|
817
|
+
```
|
|
818
|
+
|
|
819
|
+
```python
|
|
820
|
+
order("latte", size("L"), add_syrup("vanilla")).describe()
|
|
821
|
+
# 'L espresso, 1 shot(s), 200ml whole milk, 1x vanilla syrup -> $3.65'
|
|
822
|
+
```
|
|
823
|
+
|
|
824
|
+
- [`examples/processing.py`](examples/processing.py) — every way to use `Process`, side by side. Shop filters arrive from a web form as raw strings (`"4"`, `"yes"`, `" SONIC "`, a file path) and are turned into typed, clean arguments, including a set of names loaded from that path:
|
|
825
|
+
|
|
826
|
+
```bash
|
|
827
|
+
uv run python -m examples.processing
|
|
828
|
+
```
|
|
829
|
+
|
|
830
|
+
```python
|
|
831
|
+
type Text = Annotated[str, Process(str.strip), Process(str.lower)]
|
|
832
|
+
type Names = Annotated[frozenset[str], Process(load_names)]
|
|
833
|
+
|
|
834
|
+
@catalog_steps.step()
|
|
835
|
+
def where(products: Catalog, *, in_stock: Annotated[bool, Process(to_bool)] = False, **fields: Text) -> Catalog: ...
|
|
836
|
+
|
|
837
|
+
@catalog_steps.step()
|
|
838
|
+
def exclude(products: Catalog, names: Names) -> Catalog: ...
|
|
839
|
+
```
|
|
840
|
+
|
|
841
|
+
- [`examples/dependencies.py`](examples/dependencies.py) — steps that need objects JSON cannot describe. A fulfilment flow stored as JSON uses a mailer (with a sub-dependency) and a database session that is opened once per run, shared by two steps, then committed or rolled back. Also shows swapping the mailer for a fake in a test:
|
|
842
|
+
|
|
843
|
+
```bash
|
|
844
|
+
uv run python -m examples.dependencies
|
|
845
|
+
```
|
|
846
|
+
|
|
847
|
+
```python
|
|
848
|
+
@fulfilment_steps.tap()
|
|
849
|
+
def send_email(order: Order, template: str, mailer: Mailer = Depends(get_mailer)) -> None: ...
|
|
850
|
+
|
|
851
|
+
@fulfilment_steps.tap()
|
|
852
|
+
def save(order: Order, session: Session = SESSION) -> None: ...
|
|
853
|
+
```
|
|
854
|
+
|
|
855
|
+
### Working with pyspecification and pyformula
|
|
856
|
+
|
|
857
|
+
Predicates and formulas are just callables, so they plug straight into steps:
|
|
858
|
+
|
|
859
|
+
```python
|
|
860
|
+
from dataclasses import dataclass, replace
|
|
861
|
+
from decimal import Decimal
|
|
862
|
+
|
|
863
|
+
from pyformula import variable
|
|
864
|
+
from pyspecification import object_rule
|
|
865
|
+
from pyflowstep import step
|
|
866
|
+
|
|
867
|
+
|
|
868
|
+
@dataclass(frozen=True)
|
|
869
|
+
class Invoice:
|
|
870
|
+
subtotal: Decimal
|
|
871
|
+
tax: Decimal = Decimal(0)
|
|
872
|
+
notes: tuple[str, ...] = ()
|
|
873
|
+
|
|
874
|
+
|
|
875
|
+
@object_rule()
|
|
876
|
+
def is_large(invoice: Invoice, limit: Decimal) -> bool:
|
|
877
|
+
return invoice.subtotal >= limit
|
|
878
|
+
|
|
879
|
+
|
|
880
|
+
@variable()
|
|
881
|
+
def subtotal(invoice: Invoice) -> Decimal:
|
|
882
|
+
return invoice.subtotal
|
|
883
|
+
|
|
884
|
+
|
|
885
|
+
@step
|
|
886
|
+
def note_if(invoice: Invoice, rule, note: str) -> Invoice:
|
|
887
|
+
return replace(invoice, notes=(*invoice.notes, note)) if rule(invoice) else invoice
|
|
888
|
+
|
|
889
|
+
|
|
890
|
+
@step
|
|
891
|
+
def apply_tax(invoice: Invoice, formula) -> Invoice:
|
|
892
|
+
return replace(invoice, tax=Decimal(str(formula(invoice))))
|
|
893
|
+
|
|
894
|
+
|
|
895
|
+
checkout = note_if(is_large(Decimal(100)), "needs approval") >> apply_tax(round(subtotal * 0.15, 2))
|
|
896
|
+
```
|
|
897
|
+
|
|
898
|
+
---
|
|
899
|
+
|
|
900
|
+
## API reference
|
|
901
|
+
|
|
902
|
+
| Name | Kind | Description |
|
|
903
|
+
| ------------------------------------------------------------ | --------- | --------------------------------------------------------------- |
|
|
904
|
+
| `Flow[T]` | class | Immutable, callable sequence of actions; composes with `>>` |
|
|
905
|
+
| `compose(*actions)` | function | Combine actions and flows into one flat flow |
|
|
906
|
+
| `step` / `tap` | decorator | Turn a function into a step factory |
|
|
907
|
+
| `StepsRegistry[T]` | class | Named collection of steps |
|
|
908
|
+
| `pyargprocessors.Process(fn)` | marker | Apply `fn` to the value passed for a parameter, see [Processing arguments](#processing-arguments) |
|
|
909
|
+
| `Depends(provider)` | marker | Inject a parameter by calling `provider`, see [Dependencies](#dependencies) |
|
|
910
|
+
| `override_dependencies(mapping)` | context manager | Replace providers inside a `with` block, for tests |
|
|
911
|
+
| `Input(default=...)` | marker | Receive a parameter from the caller of the flow, `flow(subject, name=value)`, see [Run inputs](#run-inputs) |
|
|
912
|
+
| `Flow.inputs` | property | The names of the inputs a flow requires |
|
|
913
|
+
| `Flow.signature(stub)` | decorator | Give a flow a typed call signature, checked against its inputs, see [Typed calls](#typed-calls) |
|
|
914
|
+
| `FlowCompiler[T](steps)` | class | `compile(definition)` turns parsed step dicts into a flow |
|
|
915
|
+
| `validate_step_dict(item, path)` | function | Validate one step dictionary |
|
|
916
|
+
| `get_json_schema(annotation)` | function | JSON Schema of a type annotation |
|
|
917
|
+
| `get_step_json_schema(name, step)` | function | JSON Schema of one step dictionary |
|
|
918
|
+
| `get_flow_json_schema(steps)` | function | JSON Schema of a whole flow definition |
|
|
919
|
+
|
|
920
|
+
All exceptions derive from `PyflowstepError`; argument errors, `InvalidDependencyError`, `InvalidInputError` and `MissingInputError` also derive from `TypeError`, definition errors from `ValueError`, and `StepDoesNotExistError` from `LookupError`. The exceptions of argument processing, `ProcessArgumentError` and `InvalidProcessorError`, belong to `pyargprocessors`: they are `TypeError`s too, but derive from `PyargprocessorsError`.
|
|
921
|
+
|
|
922
|
+
---
|
|
923
|
+
|
|
924
|
+
## Development
|
|
925
|
+
|
|
926
|
+
```bash
|
|
927
|
+
uv sync
|
|
928
|
+
uv run pytest --cov --cov-report term-missing
|
|
929
|
+
```
|
|
930
|
+
|
|
931
|
+
The test suite also runs every docstring example (`--doctest-modules`).
|
|
932
|
+
|
|
933
|
+
## License
|
|
934
|
+
|
|
935
|
+
GPL-3.0
|