@db-lyon/flowkit 0.17.2 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +53 -1
- package/dist/.tsbuildinfo +1 -1
- package/dist/config/index.d.ts +4 -2
- package/dist/config/index.d.ts.map +1 -1
- package/dist/config/index.js +2 -1
- package/dist/config/index.js.map +1 -1
- package/dist/config/loader.d.ts +11 -0
- package/dist/config/loader.d.ts.map +1 -1
- package/dist/config/loader.js +4 -0
- package/dist/config/loader.js.map +1 -1
- package/dist/config/schema.d.ts +1486 -79
- package/dist/config/schema.d.ts.map +1 -1
- package/dist/config/schema.js +125 -8
- package/dist/config/schema.js.map +1 -1
- package/dist/config/strict.d.ts +40 -0
- package/dist/config/strict.d.ts.map +1 -0
- package/dist/config/strict.js +146 -0
- package/dist/config/strict.js.map +1 -0
- package/dist/flow/index.d.ts +2 -2
- package/dist/flow/index.d.ts.map +1 -1
- package/dist/flow/index.js +1 -1
- package/dist/flow/index.js.map +1 -1
- package/dist/flow/runner.d.ts +279 -6
- package/dist/flow/runner.d.ts.map +1 -1
- package/dist/flow/runner.js +845 -40
- package/dist/flow/runner.js.map +1 -1
- package/dist/index.d.ts +13 -5
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +6 -2
- package/dist/index.js.map +1 -1
- package/dist/task/base-task.d.ts +55 -1
- package/dist/task/base-task.d.ts.map +1 -1
- package/dist/task/base-task.js +39 -0
- package/dist/task/base-task.js.map +1 -1
- package/dist/task/composite.d.ts +66 -0
- package/dist/task/composite.d.ts.map +1 -0
- package/dist/task/composite.js +21 -0
- package/dist/task/composite.js.map +1 -0
- package/dist/task/index.d.ts +7 -1
- package/dist/task/index.d.ts.map +1 -1
- package/dist/task/index.js +3 -0
- package/dist/task/index.js.map +1 -1
- package/dist/task/options-schema.d.ts +51 -0
- package/dist/task/options-schema.d.ts.map +1 -0
- package/dist/task/options-schema.js +102 -0
- package/dist/task/options-schema.js.map +1 -0
- package/dist/task/registry.d.ts +39 -1
- package/dist/task/registry.d.ts.map +1 -1
- package/dist/task/registry.js +44 -2
- package/dist/task/registry.js.map +1 -1
- package/dist/task/warnings.d.ts +25 -0
- package/dist/task/warnings.d.ts.map +1 -0
- package/dist/task/warnings.js +30 -0
- package/dist/task/warnings.js.map +1 -0
- package/docs/api-reference.md +273 -0
- package/docs/configuration.md +262 -1
- package/docs/custom-tasks.md +155 -0
- package/docs/releases.md +47 -0
- package/package.json +1 -1
package/docs/configuration.md
CHANGED
|
@@ -33,6 +33,10 @@ tasks:
|
|
|
33
33
|
| `description` | `string` | no | Human-readable description |
|
|
34
34
|
| `group` | `string` | no | Logical grouping label |
|
|
35
35
|
| `options` | `object` | no | Default options (merged with step-level overrides) |
|
|
36
|
+
| `options_schema` | `object` | no | Declared options, refining the class's `optionsSchema`. See [Declaring options](custom-tasks.md#declaring-options) |
|
|
37
|
+
| `outputs` | `object` | no | Declared `data` outputs (`type`, `description`), for `describe`. Not enforced |
|
|
38
|
+
| `deprecated` | `boolean \| string` | no | Mark the task deprecated. See [Deprecation](#deprecation) |
|
|
39
|
+
| `replaced_by` | `string` | no | The task to use instead, named in the deprecation warning |
|
|
36
40
|
|
|
37
41
|
### Flow definition
|
|
38
42
|
|
|
@@ -49,22 +53,32 @@ flows:
|
|
|
49
53
|
flow: other_flow # reference another flow (nesting)
|
|
50
54
|
3:
|
|
51
55
|
task: None # skip sentinel — step is always skipped
|
|
56
|
+
4:
|
|
57
|
+
flow: None # same, for a step that referenced a flow
|
|
52
58
|
```
|
|
53
59
|
|
|
54
60
|
| Field | Type | Required | Description |
|
|
55
61
|
|-------|------|----------|-------------|
|
|
56
62
|
| `description` | `string` | yes | Human-readable flow description |
|
|
57
63
|
| `steps` | `object` | yes | Steps keyed by number (execution order) |
|
|
64
|
+
| `checks` | `array` | no | Flow-level preflight checks. See [Preflight checks](#preflight-checks) |
|
|
65
|
+
| `options_scope` | `'flat' \| 'step'` | no | How runtime `params` reach steps. See [Step-scoped runtime options](#step-scoped-runtime-options) |
|
|
66
|
+
| `deprecated` | `boolean \| string` | no | Mark the flow deprecated. See [Deprecation](#deprecation) |
|
|
67
|
+
| `replaced_by` | `string` | no | The flow to use instead, named in the deprecation warning |
|
|
58
68
|
|
|
59
69
|
### Flow step
|
|
60
70
|
|
|
61
|
-
Each step must have exactly one of `task` or `flow` (mutually exclusive), unless `task: None` is used to mark a skipped step.
|
|
71
|
+
Each step must have exactly one of `task` or `flow` (mutually exclusive), unless `task: None` or `flow: None` is used to mark a skipped step. `None` in either slot wins, so an overlay can switch off an inherited step by number whichever key the base used.
|
|
62
72
|
|
|
63
73
|
| Field | Type | Required | Description |
|
|
64
74
|
|-------|------|----------|-------------|
|
|
65
75
|
| `task` | `string` | one of task/flow | Task name to execute |
|
|
66
76
|
| `flow` | `string` | one of task/flow | Nested flow name to execute |
|
|
67
77
|
| `options` | `object` | no | Override options for this step |
|
|
78
|
+
| `when` | `string \| boolean` | no | Run the step only when truthy |
|
|
79
|
+
| `ignore_failure` | `boolean` | no | Record a failure and continue |
|
|
80
|
+
| `retries` / `retryDelay` / `retryOn` | | no | See [Per-step retry](#per-step-retry) |
|
|
81
|
+
| `checks` | `array` | no | Preflight checks. See [Preflight checks](#preflight-checks) |
|
|
68
82
|
|
|
69
83
|
Step numbers are sorted numerically at execution time, so `1, 2, 10` runs in that order (not lexicographic `1, 10, 2`).
|
|
70
84
|
|
|
@@ -93,6 +107,56 @@ flows:
|
|
|
93
107
|
|
|
94
108
|
Runtime parameters passed to `FlowRunner.run({ params })` merge on top with the highest priority (**task defaults < step overrides < runtime params**).
|
|
95
109
|
|
|
110
|
+
### Step-scoped runtime options
|
|
111
|
+
|
|
112
|
+
By default runtime `params` are **flat**: every key is merged into every step's
|
|
113
|
+
options, so `params: { environment: 'prod' }` reaches the build, the test and
|
|
114
|
+
the deploy alike. Opt in to the **step** scope to address each step instead:
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
flows:
|
|
118
|
+
release:
|
|
119
|
+
options_scope: step # or FlowRunnerConfig.optionsScope / run({ optionsScope })
|
|
120
|
+
steps:
|
|
121
|
+
1: { task: build }
|
|
122
|
+
2: { flow: ci } # ci: 1: lint, 2: test
|
|
123
|
+
3: { task: deploy }
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
```typescript
|
|
127
|
+
await runner.run({
|
|
128
|
+
flowName: 'release',
|
|
129
|
+
params: {
|
|
130
|
+
deploy: { environment: 'prod' }, // every step running the `deploy` task
|
|
131
|
+
'2/2': { coverage: 90 }, // step 2 of the flow run by step 2 (ci's `test`)
|
|
132
|
+
'1': { target: 'release' }, // main step 1
|
|
133
|
+
},
|
|
134
|
+
});
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Under the step scope each `params` key is a **selector** and its value an
|
|
138
|
+
options object:
|
|
139
|
+
|
|
140
|
+
| Selector | Matches |
|
|
141
|
+
|----------|---------|
|
|
142
|
+
| a task name, e.g. `deploy` or `asset.list` (dots are part of the name) | every step running that task, anywhere in the run: main steps, nested flows and hook steps |
|
|
143
|
+
| a step path, e.g. `3` or `2/1` | one main step; `/` descends into the flow a step runs, as in an expanded plan's `path` |
|
|
144
|
+
|
|
145
|
+
- A path is more specific than a name: when both address a step, the path's
|
|
146
|
+
value wins on a shared key.
|
|
147
|
+
- The scoped value takes the runtime slot in the precedence order, so it
|
|
148
|
+
still overrides task defaults, enclosing-flow overrides and step options.
|
|
149
|
+
- A selector that matches no task name or step path, or a value that is not an
|
|
150
|
+
object, fails the run (and a plan) before anything starts, naming each bad key.
|
|
151
|
+
- Hook steps are addressed by task name only.
|
|
152
|
+
- The scope is fixed by the flow the run starts on. Nested flows follow it and
|
|
153
|
+
their own `options_scope` is ignored.
|
|
154
|
+
- `when:` expressions and `conditionEvaluator` still receive `params` as passed.
|
|
155
|
+
|
|
156
|
+
Resolution order for the scope: `run({ optionsScope })`, then the flow's
|
|
157
|
+
`options_scope`, then `FlowRunnerConfig.optionsScope`, then `flat`. Nothing
|
|
158
|
+
changes unless one of them says `step`.
|
|
159
|
+
|
|
96
160
|
### Step references
|
|
97
161
|
|
|
98
162
|
Option values may reference the output of earlier steps in the same flow using `${steps.<id>.<path>}`:
|
|
@@ -121,6 +185,43 @@ flows:
|
|
|
121
185
|
|
|
122
186
|
References resolve just before the step runs, against the results of already-completed steps in the current flow. Nested flows have their own reference scope — they don't see their parent flow's steps.
|
|
123
187
|
|
|
188
|
+
#### Binding by identity, and the uniqueness rule
|
|
189
|
+
|
|
190
|
+
A reference names a step by **identity**, its number or its task name, never by
|
|
191
|
+
position relative to the referencing step. That is why step numbers are worth
|
|
192
|
+
keeping stable (and gapped: `10`, `20`, `30`): an overlay that inserts step `15`
|
|
193
|
+
changes no reference.
|
|
194
|
+
|
|
195
|
+
- A number binds to exactly one step. Prefer it whenever a task runs more than
|
|
196
|
+
once in a flow.
|
|
197
|
+
- A name binds to the step running that task. When several main steps run the
|
|
198
|
+
same task, the runtime rule is the one above: the most recently completed
|
|
199
|
+
one wins. That is well defined but fragile, because which one is "most
|
|
200
|
+
recent" depends on where the referencing step sits.
|
|
201
|
+
- The same resolution applies in `when:` and in check `when`s. Hook steps see
|
|
202
|
+
every main step that ran. A flow step's `options` are overrides for the
|
|
203
|
+
tasks inside the nested flow, and references in them resolve against the
|
|
204
|
+
nested flow's steps.
|
|
205
|
+
|
|
206
|
+
Set `strictStepReferences: true` on the `FlowRunner` to make the uniqueness
|
|
207
|
+
rule an error instead: a run or plan refuses to start, naming each reference,
|
|
208
|
+
when a `${steps.<id>}` in a step's options, `when` or checks is **ambiguous** (a
|
|
209
|
+
name used by more than one main step), **unknown** (no such number or name) or
|
|
210
|
+
**forward** (the step has not run yet at that point). The check covers the flow
|
|
211
|
+
a run starts on and every flow nested under it. It is opt-in because a config
|
|
212
|
+
that relies on "most recent wins" would otherwise stop running.
|
|
213
|
+
`runner.checkStepReferences(flowName)` returns the same findings without
|
|
214
|
+
enforcing them, for a linter or a `describe` command:
|
|
215
|
+
|
|
216
|
+
```typescript
|
|
217
|
+
runner.checkStepReferences('release');
|
|
218
|
+
// [{ flowName: 'release', stepNumber: 3, reference: '${steps.deploy.url}', kind: 'ambiguous',
|
|
219
|
+
// message: '${steps.deploy.url}: "deploy" is the name of steps 1, 2; reference one by number' }]
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
Task definition defaults are not scanned: they resolve in whichever step runs
|
|
223
|
+
the task.
|
|
224
|
+
|
|
124
225
|
### Flow-level hooks
|
|
125
226
|
|
|
126
227
|
A flow can attach steps that run around the main step sequence, keyed by flow outcome:
|
|
@@ -227,6 +328,108 @@ steps:
|
|
|
227
328
|
|
|
228
329
|
Returns `{ text, parsed?, usage? }`. Provider failures become step failures; missing provider is a clear error.
|
|
229
330
|
|
|
331
|
+
### Preflight checks
|
|
332
|
+
|
|
333
|
+
A flow or a step can declare `checks`: conditions that gate it, evaluated by the
|
|
334
|
+
same `conditionEvaluator` as `when:` (or the built-in `${...}` truthiness
|
|
335
|
+
fallback). A check **fires** when its `when` is truthy, and its `action` applies:
|
|
336
|
+
|
|
337
|
+
| `action` | At run time |
|
|
338
|
+
|----------|-------------|
|
|
339
|
+
| `error` | The step fails with a `CheckFailedError` carrying `message`, before it starts. The flow aborts whatever `ignore_failure` says: a gate is not a failure to tolerate. |
|
|
340
|
+
| `skip` | The step is skipped (`skipReason: 'check'`). |
|
|
341
|
+
| `warn` | A `check` warning is added to `FlowRunResult.warnings` and the step runs. |
|
|
342
|
+
|
|
343
|
+
```yaml
|
|
344
|
+
flows:
|
|
345
|
+
import_assets:
|
|
346
|
+
checks: # gate the whole flow
|
|
347
|
+
- when: "not editor.connected"
|
|
348
|
+
action: error
|
|
349
|
+
message: No editor is connected.
|
|
350
|
+
steps:
|
|
351
|
+
1:
|
|
352
|
+
task: import
|
|
353
|
+
checks:
|
|
354
|
+
- when: "editor.has_modal_dialog"
|
|
355
|
+
action: error
|
|
356
|
+
message: A modal dialog is open in the editor.
|
|
357
|
+
- when: "not project.python_enabled"
|
|
358
|
+
action: skip
|
|
359
|
+
2:
|
|
360
|
+
task: validate
|
|
361
|
+
checks:
|
|
362
|
+
- { when: "${project.dirty}", action: warn, message: Unsaved changes. }
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
- Step checks run in order, just before the step would run and after `when:`
|
|
366
|
+
has let it. Error beats skip beats warn.
|
|
367
|
+
- Flow checks run before anything in the flow, `on_start` included. An `error`
|
|
368
|
+
fails the flow with no steps; a `skip` returns success with every main step
|
|
369
|
+
skipped (`skipReason: 'check'`). For a nested flow the verdict becomes the
|
|
370
|
+
flow step's.
|
|
371
|
+
- Checks on hook steps are enforced too; a fired `error` is reported in
|
|
372
|
+
`hookErrors`.
|
|
373
|
+
- A check whose expression throws (for example one that reads a step result
|
|
374
|
+
that does not exist) fails the step the way a throwing `when:` does, and
|
|
375
|
+
`ignore_failure` applies.
|
|
376
|
+
- The evaluator's `ConditionContext` carries `check`, `step`, `flowName` and
|
|
377
|
+
`references` (the runner's host namespaces) alongside the usual `steps`,
|
|
378
|
+
`params`, `context` and `error`, so a check can be written against host state.
|
|
379
|
+
- Fired checks are listed on the step (`FlowStepResult.checks`) and for the
|
|
380
|
+
whole run, nested flows included (`FlowRunResult.checks`).
|
|
381
|
+
|
|
382
|
+
`FlowRunner.preflight(flowName, params?, { skip? })` evaluates every check in a
|
|
383
|
+
flow without running anything and reports what would happen to each step:
|
|
384
|
+
|
|
385
|
+
```typescript
|
|
386
|
+
const pf = await runner.preflight('import_assets', { path: '/Game/X' });
|
|
387
|
+
pf.ok; // false when any `error` check fired
|
|
388
|
+
pf.checks; // the flow's own checks, evaluated
|
|
389
|
+
pf.steps; // [{ path: '1', status: 'error', checks: [...] }, { path: '2', status: 'run', ... }]
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
Each row has a `status` of `run`, `skip` (statically, or a `skip` check fired),
|
|
393
|
+
`error` or `unknown`. Nothing has run at preflight, so checks see no step
|
|
394
|
+
results; one that reads a step result cannot be evaluated, its outcome carries
|
|
395
|
+
the `error`, and the row is `unknown` rather than guessed. Nested flows are
|
|
396
|
+
expanded with the same paths as an expanded plan, and hook steps are listed as
|
|
397
|
+
`<phase>/<n>`. Plan mode (`run({ plan: true })`) lists each step's declared
|
|
398
|
+
`checks` unevaluated.
|
|
399
|
+
|
|
400
|
+
### Deprecation
|
|
401
|
+
|
|
402
|
+
Tasks and flows can be retired without breaking the configs that still use
|
|
403
|
+
them. A deprecated task or flow still runs; the run and the plan say so:
|
|
404
|
+
|
|
405
|
+
```yaml
|
|
406
|
+
tasks:
|
|
407
|
+
deploy_legacy:
|
|
408
|
+
class_path: tasks.Deploy
|
|
409
|
+
deprecated: "Targets the old cluster." # or just `true`
|
|
410
|
+
replaced_by: deploy
|
|
411
|
+
|
|
412
|
+
flows:
|
|
413
|
+
release_v1:
|
|
414
|
+
deprecated: true
|
|
415
|
+
replaced_by: release
|
|
416
|
+
steps:
|
|
417
|
+
1: { task: deploy_legacy }
|
|
418
|
+
```
|
|
419
|
+
|
|
420
|
+
- A run collects a structured warning for every deprecated task or flow it
|
|
421
|
+
ran, on the step's `result.warnings` and, without repeats, on
|
|
422
|
+
`FlowRunResult.warnings`:
|
|
423
|
+
`{ code: 'deprecated', kind: 'task', name: 'deploy_legacy', replacedBy: 'deploy', message: 'Task "deploy_legacy" is deprecated: Targets the old cluster. Use "deploy" instead.' }`.
|
|
424
|
+
`runTask` puts it on the returned `TaskResult.warnings`. The runner's logger
|
|
425
|
+
gets it at `warn` too.
|
|
426
|
+
- Plan mode marks each deprecated row with `deprecated` and `replaced_by` and
|
|
427
|
+
returns the same warnings on the plan's `warnings`. Skipped steps are not
|
|
428
|
+
reported.
|
|
429
|
+
- A task class can declare `static deprecated` and `static replacedBy`; a
|
|
430
|
+
definition's `deprecated` (including `false`) overrides the class.
|
|
431
|
+
- `FlowRunner.describeTask(name)` and `describeFlow(name)` expose both fields.
|
|
432
|
+
|
|
230
433
|
## Config layering
|
|
231
434
|
|
|
232
435
|
`loadConfig()` merges up to four layers, left to right:
|
|
@@ -315,6 +518,46 @@ Result: `['eslint', 'prettier', 'my-custom-plugin']`
|
|
|
315
518
|
|
|
316
519
|
The `__merge` annotation is stripped from the final array.
|
|
317
520
|
|
|
521
|
+
## Strict validation
|
|
522
|
+
|
|
523
|
+
Zod drops keys a schema does not declare, so a typo such as `retires: 3` or
|
|
524
|
+
`ignore_failur: true` loads cleanly and then does nothing. Pass `strict` to
|
|
525
|
+
make the loader reject them instead:
|
|
526
|
+
|
|
527
|
+
```typescript
|
|
528
|
+
const { config } = loadConfig({
|
|
529
|
+
filename: 'pipeline.yml',
|
|
530
|
+
schema: EngineConfigSchema,
|
|
531
|
+
strict: true,
|
|
532
|
+
});
|
|
533
|
+
// UnknownConfigKeyError: Unknown config key:
|
|
534
|
+
// flows.ci.steps.2.retires (did you mean "retries"?)
|
|
535
|
+
```
|
|
536
|
+
|
|
537
|
+
The check runs on the merged layers, before the schema parses them, and walks
|
|
538
|
+
the schema you pass: tasks, flows, steps, hook steps, agents, agent tools and
|
|
539
|
+
budgets, plus any section a host adds with `EngineConfigSchema.extend(...)`.
|
|
540
|
+
Free-form maps (`options`, an agent's `schema`, a tool's `parameters`) are not
|
|
541
|
+
checked, and neither is any object schema declared `.passthrough()` or with a
|
|
542
|
+
`.catchall()`.
|
|
543
|
+
|
|
544
|
+
A host that keeps sections in the same file but does not declare them in the
|
|
545
|
+
schema it passes lists them as `passthroughKeys`. Only top-level keys can be
|
|
546
|
+
exempted this way:
|
|
547
|
+
|
|
548
|
+
```typescript
|
|
549
|
+
loadConfig({
|
|
550
|
+
filename: 'ue-mcp.yml',
|
|
551
|
+
schema: HostConfigSchema,
|
|
552
|
+
strict: { passthroughKeys: ['bridge', 'editor'] },
|
|
553
|
+
});
|
|
554
|
+
```
|
|
555
|
+
|
|
556
|
+
Strict validation is opt-in. Without `strict`, unknown keys are dropped
|
|
557
|
+
exactly as before. `findUnknownKeys(schema, value)` and
|
|
558
|
+
`assertKnownKeys(schema, value)` run the same check on config that does not
|
|
559
|
+
come through `loadConfig`.
|
|
560
|
+
|
|
318
561
|
## Finding config files
|
|
319
562
|
|
|
320
563
|
`findConfigFile()` walks up parent directories to locate a file:
|
|
@@ -361,3 +604,21 @@ const { config } = loadConfig({
|
|
|
361
604
|
|
|
362
605
|
// config.tasks, config.flows, config.database, config.features
|
|
363
606
|
```
|
|
607
|
+
|
|
608
|
+
### Reusing the step and flow schemas
|
|
609
|
+
|
|
610
|
+
A host that declares flows outside the main config file (a plugin manifest, a
|
|
611
|
+
flow built in code) should validate them with flowkit's own schemas rather than
|
|
612
|
+
a copy, so every step field (`when`, `ignore_failure`, `retries`, `None` skips,
|
|
613
|
+
and whatever is added later) keeps working there too.
|
|
614
|
+
|
|
615
|
+
```typescript
|
|
616
|
+
import { FlowDefinitionSchema, FlowStepObjectSchema, refineFlowStep } from '@db-lyon/flowkit';
|
|
617
|
+
|
|
618
|
+
// A whole flow, with a host-only field.
|
|
619
|
+
const ManifestFlowSchema = FlowDefinitionSchema.extend({ group: z.string().optional() });
|
|
620
|
+
|
|
621
|
+
// A step with a host-only field. `FlowStepSchema` is refined and cannot be
|
|
622
|
+
// extended, so extend the object form and refine it again.
|
|
623
|
+
const ManifestStepSchema = refineFlowStep(FlowStepObjectSchema.extend({ label: z.string().optional() }));
|
|
624
|
+
```
|
package/docs/custom-tasks.md
CHANGED
|
@@ -55,6 +55,81 @@ export default class FetchData extends BaseTask<MyOptions> {
|
|
|
55
55
|
| Member | Description |
|
|
56
56
|
|--------|-------------|
|
|
57
57
|
| `validate()` | Called before `execute()`. Throw to abort with a validation error. |
|
|
58
|
+
| `static optionsSchema` | Declared options, checked before the task runs. See below. |
|
|
59
|
+
| `static outputs` | Declared `data` keys, for `describe` and docs. Not enforced. |
|
|
60
|
+
| `static description` | Used when the task definition has no `description`. |
|
|
61
|
+
|
|
62
|
+
### Declaring options
|
|
63
|
+
|
|
64
|
+
A task class can declare its options statically, the way a CumulusCI task
|
|
65
|
+
declares `task_options`. `FlowRunner` checks a step's final options against the
|
|
66
|
+
declaration before it constructs the task, so a bad option fails with a message
|
|
67
|
+
naming the task and the option, and the task never runs:
|
|
68
|
+
|
|
69
|
+
```typescript
|
|
70
|
+
import { BaseTask, type OptionSpecs, type TaskResult } from '@db-lyon/flowkit';
|
|
71
|
+
|
|
72
|
+
export default class Deploy extends BaseTask<{ environment: string; replicas: number }> {
|
|
73
|
+
static description = 'Deploy the build';
|
|
74
|
+
static optionsSchema: OptionSpecs = {
|
|
75
|
+
environment: { type: 'string', enum: ['staging', 'prod'], required: true, description: 'Target' },
|
|
76
|
+
replicas: { type: 'integer', minimum: 1, maximum: 5, default: 2 },
|
|
77
|
+
};
|
|
78
|
+
static outputs = { url: { type: 'string', description: 'Where it landed' } };
|
|
79
|
+
|
|
80
|
+
get taskName() { return 'deploy'; }
|
|
81
|
+
async execute(): Promise<TaskResult> { /* ... */ return { success: true }; }
|
|
82
|
+
}
|
|
83
|
+
// Task "deploy": option "environment" must be one of ["staging","prod"]
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Each option spec takes `type` (one type or a list), `description`,
|
|
87
|
+
`required`, `default`, and the JSON Schema constraint keywords `enum`, `const`,
|
|
88
|
+
`minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `minLength`,
|
|
89
|
+
`maxLength`, `pattern`, `minItems`, `maxItems`, `items`, `properties`,
|
|
90
|
+
`additionalProperties` and `nullable`.
|
|
91
|
+
|
|
92
|
+
- `default` fills an option no layer supplied. It is the lowest precedence of
|
|
93
|
+
all, below the task definition's `options`.
|
|
94
|
+
- `required` is checked after every layer (defaults, definition options, step
|
|
95
|
+
options, runtime params) has been merged.
|
|
96
|
+
- Options the schema does not declare are allowed through.
|
|
97
|
+
- A failed check is not retried, whatever the step's `retries`.
|
|
98
|
+
|
|
99
|
+
A task definition refines the class declaration with `options_schema`, option
|
|
100
|
+
by option and field by field, so one configured variant can narrow an enum,
|
|
101
|
+
change a default or add an option without restating the rest:
|
|
102
|
+
|
|
103
|
+
```yaml
|
|
104
|
+
tasks:
|
|
105
|
+
deploy_staging:
|
|
106
|
+
class_path: tasks.Deploy
|
|
107
|
+
options_schema:
|
|
108
|
+
environment: { enum: [staging], default: staging }
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
The check runs on every `FlowRunner` path: flow steps, hook steps, `runTask`,
|
|
112
|
+
rollback and composite child steps. A task built directly (`new Deploy(...)`)
|
|
113
|
+
or through `this.call()` from another task is not checked.
|
|
114
|
+
`validateTaskOptions(specs, options)` and `assertTaskOptions(name, specs,
|
|
115
|
+
options)` run the same check for a host that constructs tasks itself.
|
|
116
|
+
|
|
117
|
+
### Describing a task
|
|
118
|
+
|
|
119
|
+
`registry.describe(name, taskDefinitions?)` folds a task's class metadata and
|
|
120
|
+
its configured definition into one object, for a host's `describe` command or
|
|
121
|
+
generated docs. `FlowRunner.describeTask(name)` does the same with the
|
|
122
|
+
runner's own definitions:
|
|
123
|
+
|
|
124
|
+
```typescript
|
|
125
|
+
const d = await runner.describeTask('deploy');
|
|
126
|
+
// {
|
|
127
|
+
// name: 'deploy', class_path: 'tasks.Deploy', description: 'Deploy the build',
|
|
128
|
+
// options: { replicas: 2 }, // schema defaults, then definition options
|
|
129
|
+
// options_schema: { environment: {...}, replicas: {...} },
|
|
130
|
+
// outputs: { url: {...} },
|
|
131
|
+
// }
|
|
132
|
+
```
|
|
58
133
|
|
|
59
134
|
### Available on `this`
|
|
60
135
|
|
|
@@ -87,6 +162,86 @@ The `${ns.path}` references in those configured defaults are interpolated for yo
|
|
|
87
162
|
|
|
88
163
|
Calling a task requires a registry on the context, which `FlowRunner` supplies. A task constructed by hand without one throws.
|
|
89
164
|
|
|
165
|
+
## Composite tasks
|
|
166
|
+
|
|
167
|
+
A flow is a fixed sequence. When the children depend on the input (one child
|
|
168
|
+
per item in a list, or pages until a cap), write a **composite**: a task that
|
|
169
|
+
runs each child with `this.step()`. Unlike `this.call()`, a child step goes
|
|
170
|
+
through the runner's bookkeeping, exactly like a flow step:
|
|
171
|
+
|
|
172
|
+
- it is recorded under the parent, as `TaskResult.children` (a
|
|
173
|
+
`FlowStepResult[]` whose entries carry a `path` such as `2/1`), and is
|
|
174
|
+
therefore visible in the run result tree;
|
|
175
|
+
- the runner's `beforeStep` / `afterStep` hooks fire for it, with that path;
|
|
176
|
+
- the step retry policy applies (`{ retries, retryDelay, retryOn }` as the third
|
|
177
|
+
argument), as do option schemas and deprecation warnings;
|
|
178
|
+
- its rollback record is harvested with the parent's: inside a flow with
|
|
179
|
+
`rollback_on_failure`, children unwind in reverse after the parent's own
|
|
180
|
+
record.
|
|
181
|
+
|
|
182
|
+
```typescript
|
|
183
|
+
class ImportBatch extends BaseTask<{ files: string[]; failFast?: boolean }> {
|
|
184
|
+
get taskName() { return 'import_batch'; }
|
|
185
|
+
|
|
186
|
+
async execute(): Promise<TaskResult> {
|
|
187
|
+
const results = [];
|
|
188
|
+
for (const file of this.options.files) {
|
|
189
|
+
const r = await this.step('asset.import', { file }, { retries: 2 });
|
|
190
|
+
results.push({ file, ok: r.success });
|
|
191
|
+
if (!r.success && this.options.failFast) {
|
|
192
|
+
return { success: false, error: r.error, data: { results, stoppedAt: file } };
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
return { success: true, data: { results } };
|
|
196
|
+
}
|
|
197
|
+
}
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
`this.step(target, options?, spec?)` takes a configured task name, `{ task:
|
|
201
|
+
name }` or `{ flow: name }`, and returns the child's `TaskResult`. It never
|
|
202
|
+
throws for a failed child: the composite decides whether to go on. The
|
|
203
|
+
`options` are the composite's runtime data. They are layered over the child
|
|
204
|
+
task's configured defaults verbatim, not interpolated, and runtime `params` and
|
|
205
|
+
enclosing-flow overrides do not reach children. For a flow child they are the
|
|
206
|
+
flow's `params`.
|
|
207
|
+
|
|
208
|
+
The runner supplies this for every task it runs, whether as a flow step, a hook,
|
|
209
|
+
or on its own through `runner.runTask(name, options)`, so a composite behaves
|
|
210
|
+
the same with or without an enclosing flow. Under `runTask`, the children are on
|
|
211
|
+
the returned `TaskResult.children`, and `collectRollbackRecords(result)` returns
|
|
212
|
+
every record in the tree (children first, invoke in reverse) for a host that
|
|
213
|
+
does its own rollback. A task constructed by hand has no runner: a task child
|
|
214
|
+
falls back to `this.call()` without bookkeeping, and a flow child fails.
|
|
215
|
+
|
|
216
|
+
### `expand`: the child plan without running it
|
|
217
|
+
|
|
218
|
+
A composite can declare a static `expand(options, ctx)` that returns the
|
|
219
|
+
children it would run, as `{ task | flow, options? }` entries, without running
|
|
220
|
+
anything. Return `null` when the children depend on runtime results (a pager
|
|
221
|
+
cannot know how many pages there are):
|
|
222
|
+
|
|
223
|
+
```typescript
|
|
224
|
+
class ImportBatch extends BaseTask<{ files: string[] }> {
|
|
225
|
+
static expand(options: Record<string, unknown>): ChildPlanEntry[] | null {
|
|
226
|
+
const files = options.files;
|
|
227
|
+
return Array.isArray(files) ? files.map((file) => ({ task: 'asset.import', options: { file } })) : null;
|
|
228
|
+
}
|
|
229
|
+
// ...
|
|
230
|
+
}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
- `run({ plan: true, expandComposites: true })` follows each composite row with
|
|
234
|
+
its children, one level deeper and with paths (`1/1`, `1/2`), and marks the
|
|
235
|
+
row `composite: 'expanded'`. A composite whose `expand` returns `null` (or
|
|
236
|
+
throws) is marked `composite: 'opaque'`. Without `expandComposites` the plan is
|
|
237
|
+
unchanged. It composes with `expandNestedFlows`.
|
|
238
|
+
- `runner.expandTask(name, options?)` returns the child plan for a `describe`
|
|
239
|
+
command, or `null`.
|
|
240
|
+
- `expand` receives the options as far as they can be resolved before a run:
|
|
241
|
+
configured defaults, the step's options and its runtime params, with host
|
|
242
|
+
references interpolated and step references left as written. `ctx` carries
|
|
243
|
+
`taskName`, `taskDefinitions`, `flows` and `references`.
|
|
244
|
+
|
|
90
245
|
## The task lifecycle
|
|
91
246
|
|
|
92
247
|
When `task.run()` is called (by the flow runner):
|
package/docs/releases.md
CHANGED
|
@@ -1,5 +1,52 @@
|
|
|
1
1
|
# Release notes
|
|
2
2
|
|
|
3
|
+
## 0.18.0
|
|
4
|
+
|
|
5
|
+
Task and flow primitives for hosts that model every action as a task, a flow
|
|
6
|
+
or a composite. Everything here is additive or opt-in: a config and a host
|
|
7
|
+
that use none of it run exactly as before.
|
|
8
|
+
|
|
9
|
+
- **`flow: None`** skips a step the way `task: None` does, so an overlay can
|
|
10
|
+
switch off an inherited step whichever key the base used.
|
|
11
|
+
- **Shared step schema.** `FlowStepObjectSchema` (unrefined, extendable),
|
|
12
|
+
`refineFlowStep(schema)` and `FlowStepsSchema` let a host validate its own
|
|
13
|
+
manifests with the runner's step fields instead of a copy.
|
|
14
|
+
- **Strict config (opt-in).** `loadConfig({ strict: true })` fails on keys the
|
|
15
|
+
schema does not declare, naming each by path with a did-you-mean;
|
|
16
|
+
`{ passthroughKeys }` exempts host top-level sections.
|
|
17
|
+
`findUnknownKeys` / `assertKnownKeys` / `UnknownConfigKeyError` run the same
|
|
18
|
+
check elsewhere.
|
|
19
|
+
- **Option schemas.** Task classes declare `static optionsSchema` and
|
|
20
|
+
`static outputs`; a definition refines them with `options_schema` and
|
|
21
|
+
`outputs`. `FlowRunner` fills schema defaults and validates before
|
|
22
|
+
constructing the task, on every runner path, with a `TaskOptionsError` naming
|
|
23
|
+
the task and option (never retried). `registry.describe(name, defs)` and
|
|
24
|
+
`runner.describeTask(name)` fold class metadata with the definition.
|
|
25
|
+
- **Deprecation.** `deprecated` / `replaced_by` on task and flow definitions,
|
|
26
|
+
`static deprecated` / `static replacedBy` on classes. Running or planning one
|
|
27
|
+
adds a structured `RunWarning` to the step, `TaskResult.warnings` and
|
|
28
|
+
`FlowRunResult.warnings`; `describeTask` and the new `describeFlow` expose it.
|
|
29
|
+
- **Step-scoped runtime options (opt-in).** `options_scope: step` (flow),
|
|
30
|
+
`FlowRunnerConfig.optionsScope` or `run({ optionsScope })` make each `params`
|
|
31
|
+
key a selector (task name, or step path such as `2/1`) addressing only the
|
|
32
|
+
matching steps. Unknown selectors fail before anything runs. Default `flat`.
|
|
33
|
+
- **Preflight checks.** Flows and steps declare
|
|
34
|
+
`checks: [{ when, action: error | warn | skip, message }]`, evaluated by the
|
|
35
|
+
`conditionEvaluator`, whose `ConditionContext` now also carries `references`,
|
|
36
|
+
`step`, `check` and `flowName`. The runner enforces them;
|
|
37
|
+
`runner.preflight(flowName, params)` reports every step's outcome without
|
|
38
|
+
running anything.
|
|
39
|
+
- **Composites.** `ctx.step(target, options, spec)` / `BaseTask.step` run a
|
|
40
|
+
child task or flow through the runner: recorded on `TaskResult.children`,
|
|
41
|
+
step hooks, retries, option checks, deprecation and rollback capture, under a
|
|
42
|
+
flow or a bare `runTask`. An optional static `expand(options, ctx)` lists the
|
|
43
|
+
children for `run({ plan: true, expandComposites: true })` and
|
|
44
|
+
`runner.expandTask(name)`. `collectRollbackRecords(result)` walks the tree.
|
|
45
|
+
- **Step references.** The identity rule and the most-recent-wins tie-break
|
|
46
|
+
are documented. `runner.checkStepReferences(flowName)` reports ambiguous,
|
|
47
|
+
unknown and forward `${steps.x}` references; `strictStepReferences: true`
|
|
48
|
+
refuses to run or plan such a flow (opt-in).
|
|
49
|
+
|
|
3
50
|
## 0.17.0
|
|
4
51
|
|
|
5
52
|
`AgentTaskOptions` and `AgentPromptOptions` now accept an optional programmatic
|