pyflowstep 0.4.0__tar.gz → 0.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/PKG-INFO +130 -72
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/README.md +128 -71
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/pyproject.toml +2 -2
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/pyproject.toml.orig +95 -93
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/__init__.py +98 -90
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/compilers.py +149 -147
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/dependencies.py +327 -326
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/exceptions.py +129 -137
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/flow.py +163 -129
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/inputs.py +58 -7
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/json_schema.py +5 -6
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/steps.py +15 -15
- pyflowstep-0.4.0/src/pyflowstep/annotations.py +0 -90
- pyflowstep-0.4.0/src/pyflowstep/parsers.py +0 -181
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/py.typed +0 -0
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/registry.py +0 -0
- {pyflowstep-0.4.0 → pyflowstep-0.6.0}/src/pyflowstep/validators.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pyflowstep
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.6.0
|
|
4
4
|
Summary: A lightweight, typed Python library for composing functions into readable, reusable flows that can be defined as JSON.
|
|
5
5
|
Keywords: flow,pipeline,composition,functional,fluent-interface,json,json-schema,workflow,dsl
|
|
6
6
|
Author: aaltatan
|
|
@@ -15,6 +15,7 @@ Classifier: Programming Language :: Python :: 3.12
|
|
|
15
15
|
Classifier: Programming Language :: Python :: 3.13
|
|
16
16
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
17
17
|
Classifier: Typing :: Typed
|
|
18
|
+
Requires-Dist: pyargprocessors>=0.1.0
|
|
18
19
|
Requires-Python: >=3.12
|
|
19
20
|
Project-URL: Homepage, https://github.com/aaltatan/pyflowstep
|
|
20
21
|
Project-URL: Repository, https://github.com/aaltatan/pyflowstep
|
|
@@ -29,15 +30,15 @@ A lightweight, typed Python library for composing functions into readable, reusa
|
|
|
29
30
|
|
|
30
31
|
- steps are plain functions: `(subject, *args, **kwargs) -> subject`
|
|
31
32
|
- flows are immutable values that compose with `>>`
|
|
32
|
-
- step arguments are validated and
|
|
33
|
-
- raw JSON values become typed arguments with a `
|
|
33
|
+
- step arguments are validated and processed when a flow is **built**, not halfway through running it
|
|
34
|
+
- raw JSON values become typed arguments with a [pyargprocessors](https://github.com/aaltatan/pyargprocessors) `Process` marker next to the parameter
|
|
34
35
|
- steps get external objects (a mailer, a database session) through FastAPI-style `Depends`
|
|
35
36
|
- the caller hands per-run values to a flow by name: `flow(page, user=user)` fills every `user: User = Input()`
|
|
36
37
|
- registry-based registration keeps steps organized and discoverable
|
|
37
38
|
- flow definitions can be compiled from dictionaries or JSON
|
|
38
39
|
- every registry can describe its flow language as a JSON Schema
|
|
39
40
|
|
|
40
|
-
It pairs naturally with its siblings [pyspecification](https://github.com/aaltatan/pyspecification) (predicates) and [pyformula](https://github.com/aaltatan/pyformula) (formulas),
|
|
41
|
+
It pairs naturally with its siblings [pyspecification](https://github.com/aaltatan/pyspecification) (predicates) and [pyformula](https://github.com/aaltatan/pyformula) (formulas). Its only dependency is [pyargprocessors](https://github.com/aaltatan/pyargprocessors) (argument processing), which itself has none.
|
|
41
42
|
|
|
42
43
|
---
|
|
43
44
|
|
|
@@ -182,7 +183,7 @@ pipeline # Flow(add >> multiply >> add)
|
|
|
182
183
|
pipeline(1) # 31
|
|
183
184
|
```
|
|
184
185
|
|
|
185
|
-
`step` and `tap` do one thing: turn a function into a flow factory. Naming a step belongs to the [registry](#registry). Three markers can sit on a parameter: [`
|
|
186
|
+
`step` and `tap` do one thing: turn a function into a flow factory. Naming a step belongs to the [registry](#registry). Three markers can sit on a parameter: [`Process`](#processing-arguments) to convert the value passed for it, [`Depends`](#dependencies) to inject an object nobody passes, and [`Input`](#run-inputs) to receive a value from whoever runs the flow.
|
|
186
187
|
|
|
187
188
|
Arguments are bound against the function signature **immediately**, so mistakes fail fast:
|
|
188
189
|
|
|
@@ -260,18 +261,18 @@ page_steps.register(lambda page: page, name="noop") # register without a decora
|
|
|
260
261
|
|
|
261
262
|
---
|
|
262
263
|
|
|
263
|
-
##
|
|
264
|
+
## Processing arguments
|
|
264
265
|
|
|
265
|
-
Flows often come from JSON or a web form, where every value arrives as a string, number, boolean, list, object or null, while your steps want dates, decimals, enums, clean text or loaded data. Mark the parameter with `
|
|
266
|
+
Flows often come from JSON or a web form, where every value arrives as a string, number, boolean, list, object or null, while your steps want dates, decimals, enums, clean text or loaded data. Mark the parameter with the `Process(fn)` marker of [pyargprocessors](https://github.com/aaltatan/pyargprocessors) inside `Annotated`, and `fn` is applied to the value that is passed for it:
|
|
266
267
|
|
|
267
268
|
```python
|
|
268
269
|
from typing import Annotated
|
|
269
270
|
|
|
270
|
-
from
|
|
271
|
+
from pyargprocessors import Process
|
|
271
272
|
|
|
272
273
|
|
|
273
274
|
@page_steps.tap()
|
|
274
|
-
def wait(page: Page, selector: str, timeout: Annotated[float,
|
|
275
|
+
def wait(page: Page, selector: str, timeout: Annotated[float, Process(float)] = 5.0) -> None:
|
|
275
276
|
page.wait(selector, timeout)
|
|
276
277
|
|
|
277
278
|
|
|
@@ -282,17 +283,17 @@ wait("#result", "10") # timeout is 10.0
|
|
|
282
283
|
{"name": "wait", "args": ["#result", "10"]}
|
|
283
284
|
```
|
|
284
285
|
|
|
285
|
-
The marker sits next to the parameter it changes, and it works with the plain `@step` and `@tap` decorators too; no registry is required.
|
|
286
|
+
The marker sits next to the parameter it changes, and it works with the plain `@step` and `@tap` decorators too; no registry is required. `pyargprocessors` is installed together with `pyflowstep`.
|
|
286
287
|
|
|
287
288
|
### Name it once, reuse it
|
|
288
289
|
|
|
289
|
-
A `type` alias gives a
|
|
290
|
+
A `type` alias gives a processed type a name, so many steps can share it:
|
|
290
291
|
|
|
291
292
|
```python
|
|
292
293
|
from decimal import Decimal
|
|
293
294
|
|
|
294
|
-
type Money = Annotated[Decimal,
|
|
295
|
-
type Text = Annotated[str,
|
|
295
|
+
type Money = Annotated[Decimal, Process(Decimal)]
|
|
296
|
+
type Text = Annotated[str, Process(str.strip), Process(str.lower)] # several processors run left to right
|
|
296
297
|
|
|
297
298
|
|
|
298
299
|
@catalog_steps.step()
|
|
@@ -300,19 +301,19 @@ def price_between(products: Catalog, low: Money, high: Money) -> Catalog: ...
|
|
|
300
301
|
|
|
301
302
|
|
|
302
303
|
@catalog_steps.step()
|
|
303
|
-
def search(products: Catalog, text: Text, min_stars: Annotated[float,
|
|
304
|
+
def search(products: Catalog, text: Text, min_stars: Annotated[float, Process(float)] = 0) -> Catalog: ...
|
|
304
305
|
```
|
|
305
306
|
|
|
306
307
|
### Loading data from a value in the JSON
|
|
307
308
|
|
|
308
|
-
A
|
|
309
|
+
A processor can be any function of one value, so the JSON can send a path and the step can receive what was loaded from it:
|
|
309
310
|
|
|
310
311
|
```python
|
|
311
312
|
def load_names(path: str) -> frozenset[str]:
|
|
312
313
|
return frozenset(Path(path).read_text().splitlines())
|
|
313
314
|
|
|
314
315
|
|
|
315
|
-
type Names = Annotated[frozenset[str],
|
|
316
|
+
type Names = Annotated[frozenset[str], Process(load_names)]
|
|
316
317
|
|
|
317
318
|
|
|
318
319
|
@catalog_steps.step()
|
|
@@ -328,7 +329,7 @@ The file is read **once, when the flow is built**. Every run of that flow reuses
|
|
|
328
329
|
|
|
329
330
|
### `*args` and `**kwargs`
|
|
330
331
|
|
|
331
|
-
On `*args` the
|
|
332
|
+
On `*args` the processor applies to each item, and on `**kwargs` to each value:
|
|
332
333
|
|
|
333
334
|
```python
|
|
334
335
|
@barista.step()
|
|
@@ -336,66 +337,99 @@ def top_with(drink: Drink, *toppings: Text) -> Drink: ...
|
|
|
336
337
|
|
|
337
338
|
|
|
338
339
|
@catalog_steps.step()
|
|
339
|
-
def where(products: Catalog, *, in_stock: Annotated[bool,
|
|
340
|
+
def where(products: Catalog, *, in_stock: Annotated[bool, Process(to_bool)] = False, **fields: Text) -> Catalog: ...
|
|
340
341
|
|
|
341
342
|
|
|
342
343
|
where(in_stock="yes", brand=" SONIC ") # in_stock=True, brand="sonic"
|
|
343
344
|
```
|
|
344
345
|
|
|
345
|
-
###
|
|
346
|
+
### Lists and nulls
|
|
346
347
|
|
|
347
|
-
|
|
348
|
-
- Parsing runs **once per factory call**, while the flow is being built, not every time the flow runs.
|
|
349
|
-
- Arguments are checked against the step's signature first, so a missing or extra argument is reported as an `ArgumentError` before any parser runs.
|
|
348
|
+
A JSON list or a `null` is one value, so `Process(int)` would be applied to the whole of it. The `each` and `optional` helpers of `pyargprocessors` build the processor you need:
|
|
350
349
|
|
|
351
350
|
```python
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
351
|
+
from pyargprocessors import each, optional
|
|
352
|
+
|
|
353
|
+
type Prices = Annotated[list[Decimal], Process(each(Decimal))] # every item of a list
|
|
354
|
+
type Limit = Annotated[int | None, Process(optional(int))] # null is kept as None
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
### Which values are processed, and when
|
|
358
|
+
|
|
359
|
+
- Only values that are **actually passed**, positionally or by keyword. Default values are never processed.
|
|
360
|
+
- Processing runs **once per factory call**, while the flow is being built, not every time the flow runs.
|
|
361
|
+
- Arguments are checked against the step's signature first, so a missing or extra argument is reported as an `ArgumentError` before any processor runs.
|
|
362
|
+
|
|
363
|
+
```python
|
|
364
|
+
flow = add_syrup(" Vanilla ", "2") # processed here: flavor="vanilla", pumps=2
|
|
365
|
+
flow(drink) # the step runs with the processed values
|
|
366
|
+
flow(another_drink) # nothing is processed again
|
|
355
367
|
```
|
|
356
368
|
|
|
357
369
|
### Errors
|
|
358
370
|
|
|
359
|
-
A
|
|
371
|
+
A processor that fails on a value raises `ProcessArgumentError`, chained to the original exception. It has the name of the argument (`error.argument`), the value as it was passed (`error.value`) and the original exception (`error.cause`). Inside a compiled flow it also carries the JSON path of the step, so a bad value is found before anything runs:
|
|
360
372
|
|
|
361
373
|
```text
|
|
362
|
-
|
|
374
|
+
pyargprocessors.exceptions.ProcessArgumentError: Argument 'limit' with value 'two' failed to process, invalid literal for int() with base 10: 'two'
|
|
363
375
|
at $[1]
|
|
364
376
|
```
|
|
365
377
|
|
|
366
|
-
A marker that cannot work raises `
|
|
378
|
+
A marker that cannot work raises `InvalidProcessorError` when the step is created:
|
|
367
379
|
|
|
368
380
|
| Problem | Example |
|
|
369
381
|
| ---------------------------------------------------- | ---------------------------------------------------- |
|
|
370
|
-
| The
|
|
371
|
-
| The marker is used as a default value | `amount: int =
|
|
372
|
-
| The marker is on the subject (the first parameter) | `def step(page: Annotated[Page,
|
|
373
|
-
| One parameter has both `
|
|
382
|
+
| The processor is not callable | `Process("int")` |
|
|
383
|
+
| The marker is used as a default value | `amount: int = Process(int)`, write `Annotated[int, Process(int)]` |
|
|
384
|
+
| The marker is on the subject (the first parameter) | `def step(page: Annotated[Page, Process(...)], ...)` |
|
|
385
|
+
| One parameter has both `Process` and `Depends` or `Input` | nothing is passed for it, so there is nothing to process |
|
|
386
|
+
|
|
387
|
+
Both exceptions come from `pyargprocessors`. They are `TypeError`s and derive from `PyargprocessorsError`, **not** from `PyflowstepError`:
|
|
388
|
+
|
|
389
|
+
```python
|
|
390
|
+
from pyargprocessors import ProcessArgumentError
|
|
391
|
+
|
|
392
|
+
try:
|
|
393
|
+
flow = compiler.compile(definition)
|
|
394
|
+
except ProcessArgumentError as error: # a value in the definition is wrong
|
|
395
|
+
...
|
|
396
|
+
except PyflowstepError as error: # the definition itself is wrong
|
|
397
|
+
...
|
|
398
|
+
```
|
|
374
399
|
|
|
375
400
|
### In the JSON schema
|
|
376
401
|
|
|
377
|
-
A
|
|
402
|
+
A processed parameter is described by **what the JSON must send**. The schema takes it from the type hint of the processor's own parameter (`load_names(path: str)` means `string`). If the processor has no type hint, as with `int`, `Decimal` or an enum class, the schema falls back to the parameter's type.
|
|
378
403
|
|
|
379
404
|
| Parameter | Schema |
|
|
380
405
|
| -------------------------------------------- | ------------------------------- |
|
|
381
|
-
| `names: Annotated[frozenset[str],
|
|
382
|
-
| `limit: Annotated[int,
|
|
383
|
-
| `kind: Annotated[Milk,
|
|
406
|
+
| `names: Annotated[frozenset[str], Process(load_names)]` | `{"type": "string"}` |
|
|
407
|
+
| `limit: Annotated[int, Process(int)]` | `{"type": "integer"}` |
|
|
408
|
+
| `kind: Annotated[Milk, Process(Milk)]` | `{"enum": ["whole", "oat", "almond"], "type": "string"}` |
|
|
384
409
|
|
|
385
|
-
### Upgrading
|
|
410
|
+
### Upgrading
|
|
411
|
+
|
|
412
|
+
Argument processing moved to `pyargprocessors`, and `Parse` became its `Process`:
|
|
413
|
+
|
|
414
|
+
| Before | Now |
|
|
415
|
+
| ------------------------------------------------- | ------------------------------------------------------- |
|
|
416
|
+
| `from pyflowstep import Parse` | `from pyargprocessors import Process` |
|
|
417
|
+
| `Annotated[float, Parse(float)]` | `Annotated[float, Process(float)]` |
|
|
418
|
+
| `pyflowstep.ParseArgumentError` | `pyargprocessors.ProcessArgumentError`, no longer a `PyflowstepError` |
|
|
419
|
+
| `pyflowstep.InvalidParserError` | `pyargprocessors.InvalidProcessorError`, no longer a `PyflowstepError` |
|
|
420
|
+
| `... failed to parse, ...` in the message | `... failed to process, ...` |
|
|
386
421
|
|
|
387
|
-
|
|
422
|
+
Coming from 0.1, the `processors=` option of the registry is gone as well. Move each entry onto its parameter:
|
|
388
423
|
|
|
389
|
-
| 0.1 |
|
|
424
|
+
| 0.1 | Now |
|
|
390
425
|
| ------------------------------------------------- | ------------------------------------------------------- |
|
|
391
|
-
| `processors={"timeout": float}` | `timeout: Annotated[float,
|
|
426
|
+
| `processors={"timeout": float}` | `timeout: Annotated[float, Process(float)]` |
|
|
392
427
|
| `processors=normalize` (every argument) | annotate each parameter, usually with a shared alias such as `Text` |
|
|
393
|
-
| `processors={"pumps": int, ...: normalize}` | `pumps: Annotated[int,
|
|
428
|
+
| `processors={"pumps": int, ...: normalize}` | `pumps: Annotated[int, Process(int)]`, the others `Text` |
|
|
394
429
|
| `...` covering `**kwargs` | `**fields: Text` |
|
|
395
|
-
| `ProcessArgumentError` | `ParseArgumentError` |
|
|
396
430
|
| `InvalidProcessorsError` | removed; a key can no longer be misspelled |
|
|
397
431
|
|
|
398
|
-
See [`examples/
|
|
432
|
+
See [`examples/processing.py`](examples/processing.py) for every form working together on data from a web form.
|
|
399
433
|
|
|
400
434
|
---
|
|
401
435
|
|
|
@@ -427,23 +461,23 @@ send_email("receipt") # only `template` is an argument
|
|
|
427
461
|
|
|
428
462
|
It works the same with the plain `@step` and `@tap` decorators; no registry is required.
|
|
429
463
|
|
|
430
|
-
###
|
|
464
|
+
### Always the default value
|
|
431
465
|
|
|
432
|
-
|
|
433
|
-
from typing import Annotated
|
|
466
|
+
`Depends(...)` is written as the default value of the parameter, in steps and in providers. To name a dependency once and reuse it in many steps, keep the marker in a constant:
|
|
434
467
|
|
|
435
|
-
|
|
468
|
+
```python
|
|
469
|
+
# inline: quick, for a one-off
|
|
436
470
|
def send_email(order: Order, template: str, mailer: Mailer = Depends(get_mailer)) -> None: ...
|
|
437
471
|
|
|
438
472
|
|
|
439
|
-
#
|
|
440
|
-
|
|
473
|
+
# a constant: name the dependency once, reuse it
|
|
474
|
+
MAILER = Depends(get_mailer)
|
|
441
475
|
|
|
442
|
-
def send_email(order: Order, template: str, mailer:
|
|
443
|
-
def send_invoice(order: Order, mailer:
|
|
476
|
+
def send_email(order: Order, template: str, mailer: Mailer = MAILER) -> None: ...
|
|
477
|
+
def send_invoice(order: Order, mailer: Mailer = MAILER) -> None: ...
|
|
444
478
|
```
|
|
445
479
|
|
|
446
|
-
|
|
480
|
+
It is never written inside `Annotated`. Only a default value tells a type checker that the parameter is not passed, so `send_email("receipt")` type-checks; `Annotated[Mailer, Depends(get_mailer)]` raises `InvalidDependencyError` when the step is created.
|
|
447
481
|
|
|
448
482
|
### One object per flow run
|
|
449
483
|
|
|
@@ -468,16 +502,16 @@ def get_session() -> Iterator[Session]:
|
|
|
468
502
|
session.close()
|
|
469
503
|
|
|
470
504
|
|
|
471
|
-
|
|
505
|
+
SESSION = Depends(get_session)
|
|
472
506
|
|
|
473
507
|
|
|
474
508
|
@steps.tap()
|
|
475
|
-
def save(order: Order, session:
|
|
509
|
+
def save(order: Order, session: Session = SESSION) -> None:
|
|
476
510
|
session.add(order)
|
|
477
511
|
|
|
478
512
|
|
|
479
513
|
@steps.tap()
|
|
480
|
-
def audit(order: Order, action: str, session:
|
|
514
|
+
def audit(order: Order, action: str, session: Session = SESSION) -> None:
|
|
481
515
|
session.add(AuditRow(order.id, action)) # the same session `save` used
|
|
482
516
|
|
|
483
517
|
|
|
@@ -519,10 +553,10 @@ Keys are the original providers and values the providers to call instead. Nested
|
|
|
519
553
|
|
|
520
554
|
### Rules
|
|
521
555
|
|
|
522
|
-
- **Invisible to JSON.** A dependency is absent from the JSON schema, cannot be
|
|
523
|
-
- **Declarations are checked early**, when the step is created, with `InvalidDependencyError`: a provider that is not callable, is circular, or has a required parameter that is not a dependency; a dependency on the subject
|
|
556
|
+
- **Invisible to JSON.** A dependency is absent from the JSON schema, cannot be processed, and passing one raises `UnexpectedKeywordArgumentError` (or `TooManyArgumentsError`) when the flow is built.
|
|
557
|
+
- **Declarations are checked early**, when the step is created, with `InvalidDependencyError`: a provider that is not callable, is circular, or has a required parameter that is not a dependency; a dependency on the subject or on a positional-only parameter; `Depends` written inside `Annotated`.
|
|
524
558
|
- **Providers run late**, when the flow runs. An error inside a provider surfaces then, not at compile time.
|
|
525
|
-
- A dependency is for an object the JSON must **not** choose. When the JSON should pick one by name (`"via": "email"`), use [`
|
|
559
|
+
- A dependency is for an object the JSON must **not** choose. When the JSON should pick one by name (`"via": "email"`), use [`Process`](#processing-arguments) with a function that turns the name into the object.
|
|
526
560
|
|
|
527
561
|
Using ruff? Its `B008` rule flags function calls in argument defaults. Tell it `Depends` is a marker:
|
|
528
562
|
|
|
@@ -533,6 +567,16 @@ extend-immutable-calls = ["pyflowstep.Depends"]
|
|
|
533
567
|
|
|
534
568
|
See [`examples/dependencies.py`](examples/dependencies.py) for a complete flow with a mailer, a per-run session and a test override.
|
|
535
569
|
|
|
570
|
+
### Upgrading from 0.4
|
|
571
|
+
|
|
572
|
+
`Annotated[T, Depends(provider)]` is gone, for the same reason `Input[T]` went in 0.4: a type checker reported `send_email("receipt")` as missing its `mailer` argument. Move the marker to the default value, in steps and in providers:
|
|
573
|
+
|
|
574
|
+
| 0.4 | 0.5 |
|
|
575
|
+
| --------------------------------------------------------- | -------------------------------------------- |
|
|
576
|
+
| `mailer: Annotated[Mailer, Depends(get_mailer)]` | `mailer: Mailer = Depends(get_mailer)` |
|
|
577
|
+
| `type MailerDep = Annotated[Mailer, Depends(get_mailer)]` | `MAILER = Depends(get_mailer)` |
|
|
578
|
+
| `mailer: MailerDep` | `mailer: Mailer = MAILER` |
|
|
579
|
+
|
|
536
580
|
---
|
|
537
581
|
|
|
538
582
|
## Run inputs
|
|
@@ -584,13 +628,26 @@ flow(page, credentials=credentials)
|
|
|
584
628
|
# MissingInputError: missing run input 'voucher' for step 'fill_year'; run the flow as flow(subject, voucher=...)
|
|
585
629
|
```
|
|
586
630
|
|
|
631
|
+
### Typed calls
|
|
632
|
+
|
|
633
|
+
A flow is typed as `flow(obj, **inputs: Any)`, because Python's typing cannot merge the inputs of several steps into one signature. To have a type checker check the inputs, write the signature once as a stub and decorate it with `flow.signature`:
|
|
634
|
+
|
|
635
|
+
```python
|
|
636
|
+
@flow.signature
|
|
637
|
+
def log_in_and_fill(page: Page, /, *, credentials: Credentials, voucher: Voucher) -> Page: ...
|
|
638
|
+
|
|
639
|
+
log_in_and_fill(page, credentials=credentials) # type checker: missing argument "voucher"
|
|
640
|
+
```
|
|
641
|
+
|
|
642
|
+
The subject is positional-only and every input keyword-only. Required inputs have no default; optional ones need one (`= ...` is fine), but the step's own default is what gets used. The stub is checked against the steps when it is decorated, and `InvalidInputError` names any input that is missing, unknown, or required when it should be optional (or the reverse). Types are not compared, only names. mypy reports a `...` body as `empty-body`; write `raise NotImplementedError` instead if that bothers you.
|
|
643
|
+
|
|
587
644
|
### Rules
|
|
588
645
|
|
|
589
|
-
- **Invisible to JSON**, like a dependency: an input is absent from the JSON schema, cannot be
|
|
646
|
+
- **Invisible to JSON**, like a dependency: an input is absent from the JSON schema, cannot be processed, and passing one while building the flow raises `UnexpectedKeywordArgumentError` (or `TooManyArgumentsError`).
|
|
590
647
|
- **`Input(default=...)` makes the input optional**: with `note: str = Input(default="")`, `""` is used when the caller passes no `note`. Optional inputs are not listed in `flow.inputs`.
|
|
591
648
|
- **Extra inputs are ignored**, so one caller can run different flows, each using the inputs it needs.
|
|
592
649
|
- **Nested flows see the inputs of the run they join.** A flow called from inside a step can be given inputs of its own, `inner(page, voucher=other)`; they are laid over the outer ones for that call only.
|
|
593
|
-
- **Declarations are checked early**, when the step is created, with `InvalidInputError`: an input on the subject, on a positional-only parameter,
|
|
650
|
+
- **Declarations are checked early**, when the step is created, with `InvalidInputError`: an input on the subject, on a positional-only parameter, or written inside `Annotated`. A provider cannot take an input.
|
|
594
651
|
|
|
595
652
|
Using ruff? Add `Input` next to `Depends` for its `B008` rule:
|
|
596
653
|
|
|
@@ -612,7 +669,7 @@ extend-immutable-calls = ["pyflowstep.Depends", "pyflowstep.Input"]
|
|
|
612
669
|
|
|
613
670
|
| The object… | Use |
|
|
614
671
|
| ------------------------------------------------------------ | ------------------------------------- |
|
|
615
|
-
| is written in the flow definition (a selector, a limit) | a plain argument, with `
|
|
672
|
+
| is written in the flow definition (a selector, a limit) | a plain argument, with `Process` if needed |
|
|
616
673
|
| can be built by a function, the same way for every caller (a mailer, a session) | `Depends(provider)` |
|
|
617
674
|
| is only known to whoever runs the flow (a user, a record, credentials) | `Input()` |
|
|
618
675
|
|
|
@@ -656,7 +713,7 @@ Each step dictionary has this shape — only `name` is required:
|
|
|
656
713
|
Everything is validated while compiling, before any step runs. Errors carry a note with their JSON path:
|
|
657
714
|
|
|
658
715
|
```text
|
|
659
|
-
|
|
716
|
+
pyargprocessors.exceptions.ProcessArgumentError: Argument 'url' with value 'http://insecure.example.com' failed to process, only https urls are allowed
|
|
660
717
|
at $[1]
|
|
661
718
|
```
|
|
662
719
|
|
|
@@ -665,7 +722,7 @@ at $[1]
|
|
|
665
722
|
| Not a list / malformed step dict | `InvalidFlowDefinitionError` |
|
|
666
723
|
| Unknown or hidden step name | `StepDoesNotExistError` (lists the available steps) |
|
|
667
724
|
| Missing / extra / duplicated arguments | `MissingArgumentError`, `TooManyArgumentsError`, ... |
|
|
668
|
-
| A
|
|
725
|
+
| A processor rejects a value | `ProcessArgumentError` (from `pyargprocessors`) |
|
|
669
726
|
|
|
670
727
|
A compiled flow is an ordinary `Flow`, so it composes with Python steps: `login >> click("#profile")`.
|
|
671
728
|
|
|
@@ -720,7 +777,7 @@ Supported annotations: `str`, `int`, `float`, `bool`, `None`, `Decimal`, `dateti
|
|
|
720
777
|
|
|
721
778
|
Runnable examples live in [`examples/`](examples):
|
|
722
779
|
|
|
723
|
-
- [`examples/browser.py`](examples/browser.py) — the `Page` automation above: `tap` steps, an https-only
|
|
780
|
+
- [`examples/browser.py`](examples/browser.py) — the `Page` automation above: `tap` steps, an https-only processor, reusable sub-flows and a JSON login scenario.
|
|
724
781
|
|
|
725
782
|
```bash
|
|
726
783
|
uv run python -m examples.browser
|
|
@@ -737,18 +794,18 @@ Runnable examples live in [`examples/`](examples):
|
|
|
737
794
|
# 'L espresso, 1 shot(s), 200ml whole milk, 1x vanilla syrup -> $3.65'
|
|
738
795
|
```
|
|
739
796
|
|
|
740
|
-
- [`examples/
|
|
797
|
+
- [`examples/processing.py`](examples/processing.py) — every way to use `Process`, side by side. Shop filters arrive from a web form as raw strings (`"4"`, `"yes"`, `" SONIC "`, a file path) and are turned into typed, clean arguments, including a set of names loaded from that path:
|
|
741
798
|
|
|
742
799
|
```bash
|
|
743
|
-
uv run python -m examples.
|
|
800
|
+
uv run python -m examples.processing
|
|
744
801
|
```
|
|
745
802
|
|
|
746
803
|
```python
|
|
747
|
-
type Text = Annotated[str,
|
|
748
|
-
type Names = Annotated[frozenset[str],
|
|
804
|
+
type Text = Annotated[str, Process(str.strip), Process(str.lower)]
|
|
805
|
+
type Names = Annotated[frozenset[str], Process(load_names)]
|
|
749
806
|
|
|
750
807
|
@catalog_steps.step()
|
|
751
|
-
def where(products: Catalog, *, in_stock: Annotated[bool,
|
|
808
|
+
def where(products: Catalog, *, in_stock: Annotated[bool, Process(to_bool)] = False, **fields: Text) -> Catalog: ...
|
|
752
809
|
|
|
753
810
|
@catalog_steps.step()
|
|
754
811
|
def exclude(products: Catalog, names: Names) -> Catalog: ...
|
|
@@ -765,7 +822,7 @@ Runnable examples live in [`examples/`](examples):
|
|
|
765
822
|
def send_email(order: Order, template: str, mailer: Mailer = Depends(get_mailer)) -> None: ...
|
|
766
823
|
|
|
767
824
|
@fulfilment_steps.tap()
|
|
768
|
-
def save(order: Order, session:
|
|
825
|
+
def save(order: Order, session: Session = SESSION) -> None: ...
|
|
769
826
|
```
|
|
770
827
|
|
|
771
828
|
### Working with pyspecification and pyformula
|
|
@@ -821,18 +878,19 @@ checkout = note_if(is_large(Decimal(100)), "needs approval") >> apply_tax(round(
|
|
|
821
878
|
| `compose(*actions)` | function | Combine actions and flows into one flat flow |
|
|
822
879
|
| `step` / `tap` | decorator | Turn a function into a step factory |
|
|
823
880
|
| `StepsRegistry[T]` | class | Named collection of steps |
|
|
824
|
-
| `
|
|
881
|
+
| `pyargprocessors.Process(fn)` | marker | Apply `fn` to the value passed for a parameter, see [Processing arguments](#processing-arguments) |
|
|
825
882
|
| `Depends(provider)` | marker | Inject a parameter by calling `provider`, see [Dependencies](#dependencies) |
|
|
826
883
|
| `override_dependencies(mapping)` | context manager | Replace providers inside a `with` block, for tests |
|
|
827
884
|
| `Input(default=...)` | marker | Receive a parameter from the caller of the flow, `flow(subject, name=value)`, see [Run inputs](#run-inputs) |
|
|
828
885
|
| `Flow.inputs` | property | The names of the inputs a flow requires |
|
|
886
|
+
| `Flow.signature(stub)` | decorator | Give a flow a typed call signature, checked against its inputs, see [Typed calls](#typed-calls) |
|
|
829
887
|
| `FlowCompiler[T](steps)` | class | `compile(definition)` turns parsed step dicts into a flow |
|
|
830
888
|
| `validate_step_dict(item, path)` | function | Validate one step dictionary |
|
|
831
889
|
| `get_json_schema(annotation)` | function | JSON Schema of a type annotation |
|
|
832
890
|
| `get_step_json_schema(name, step)` | function | JSON Schema of one step dictionary |
|
|
833
891
|
| `get_flow_json_schema(steps)` | function | JSON Schema of a whole flow definition |
|
|
834
892
|
|
|
835
|
-
All exceptions derive from `PyflowstepError`; argument errors, `
|
|
893
|
+
All exceptions derive from `PyflowstepError`; argument errors, `InvalidDependencyError`, `InvalidInputError` and `MissingInputError` also derive from `TypeError`, definition errors from `ValueError`, and `StepDoesNotExistError` from `LookupError`. The exceptions of argument processing, `ProcessArgumentError` and `InvalidProcessorError`, belong to `pyargprocessors`: they are `TypeError`s too, but derive from `PyargprocessorsError`.
|
|
836
894
|
|
|
837
895
|
---
|
|
838
896
|
|