flowsh-cli 0.8.3__tar.gz → 0.10.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/PKG-INFO +25 -17
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/README.md +24 -16
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/pyproject.toml +3 -2
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/src/flowsh_cli/__init__.py +2 -2
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/src/flowsh_cli/cli.py +26 -4
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/src/flowsh_cli/models.py +68 -3
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/src/flowsh_cli/render.py +98 -23
- flowsh_cli-0.10.0/src/flowsh_cli/skill.py +244 -0
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/src/flowsh_cli/__main__.py +0 -0
- {flowsh_cli-0.8.3 → flowsh_cli-0.10.0}/src/flowsh_cli/examples.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: flowsh-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.10.0
|
|
4
4
|
Summary: Generate Bash harness scripts from workflow YAML files.
|
|
5
5
|
License-Expression: MIT
|
|
6
6
|
Classifier: Programming Language :: Python :: 3.11
|
|
@@ -16,7 +16,7 @@ Description-Content-Type: text/markdown
|
|
|
16
16
|
|
|
17
17
|
# flowsh-cli
|
|
18
18
|
|
|
19
|
-
`flowsh-cli` turns workflow YAML into executable
|
|
19
|
+
`flowsh-cli` turns workflow YAML into executable shell scripts.
|
|
20
20
|
|
|
21
21
|
Use it when you want a small, reproducible way to encode a workflow once and rerun it locally or in CI.
|
|
22
22
|
|
|
@@ -28,7 +28,7 @@ uvx flowsh-cli path/to/workflows.yml
|
|
|
28
28
|
|
|
29
29
|
That reads the workflow YAML at the path you provide and writes harness scripts to the current working directory.
|
|
30
30
|
|
|
31
|
-
In this repository, the example workflow file lives at
|
|
31
|
+
In this repository, the example workflow file lives at `workflows.yml`.
|
|
32
32
|
|
|
33
33
|
Useful flags:
|
|
34
34
|
|
|
@@ -68,18 +68,23 @@ workflows:
|
|
|
68
68
|
expandPrompt: true
|
|
69
69
|
prompt: |
|
|
70
70
|
Review issue ${ISSUE_NUMBER} and summarize the repository state.
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
71
|
+
- type: parallel
|
|
72
|
+
steps:
|
|
73
|
+
- type: bash
|
|
74
|
+
run: echo "worker A"
|
|
75
|
+
- type: bash
|
|
76
|
+
run: echo "worker B"
|
|
77
|
+
- type: for
|
|
78
|
+
in: ITEMS
|
|
79
|
+
item: ITEM
|
|
80
|
+
steps:
|
|
81
|
+
- type: bash
|
|
82
|
+
run: echo "$ITEM"
|
|
83
|
+
- type: while
|
|
84
|
+
condition: '[ -n "$(ls doc/plan/steps/planned/ 2>/dev/null)" ]'
|
|
85
|
+
steps:
|
|
86
|
+
- type: bash
|
|
87
|
+
run: echo "keep looping until the queue is empty"
|
|
83
88
|
```
|
|
84
89
|
|
|
85
90
|
Harness paths are derived from workflow ids. `wf_example` becomes `example.sh` in the current working directory.
|
|
@@ -90,8 +95,9 @@ Harness paths are derived from workflow ids. `wf_example` becomes `example.sh` i
|
|
|
90
95
|
|---|---|---|
|
|
91
96
|
| `vars` | Execute shell commands and export their stdout into shell variables | Variable names must be uppercase shell identifiers. |
|
|
92
97
|
| `bash` | Run shell commands | Runs with `bash -euo pipefail`. |
|
|
93
|
-
| `agent` | Call
|
|
98
|
+
| `agent` | Call the agent runtime | Supports `agent`, `model`, `command`, `capture`, `expandPrompt`, and `dangerouslySkipPermissions`. |
|
|
94
99
|
| `for` | Iterate over newline-delimited values from a previous `vars` step | Flat iteration only; nested `for` steps are not supported. |
|
|
100
|
+
| `while` | Re-evaluate a Bash condition before each iteration | Use for dynamic queues or other stateful loops that must keep discovering new work. |
|
|
95
101
|
| `parallel` | Run child steps concurrently | Children run as separate branches and the parent waits for all of them. |
|
|
96
102
|
|
|
97
103
|
## Agent Behavior
|
|
@@ -100,7 +106,9 @@ Harness paths are derived from workflow ids. `wf_example` becomes `example.sh` i
|
|
|
100
106
|
|
|
101
107
|
`expandPrompt: true` does plain text replacement only. It does not evaluate shell expressions like `$(...)`, backticks, or globs.
|
|
102
108
|
|
|
103
|
-
Set `
|
|
109
|
+
Set `capture: VARIABLE_NAME` on an `agent` step when you want the full agent output stored in a shell variable for later `vars` or `bash` steps.
|
|
110
|
+
|
|
111
|
+
Set `dangerouslySkipPermissions: true` only when you want the generated harness to pass `--dangerously-skip-permissions` to the agent runtime. The YAML alias `dangerously-skip-permissions` is also accepted.
|
|
104
112
|
|
|
105
113
|
## Validation And Safety
|
|
106
114
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# flowsh-cli
|
|
2
2
|
|
|
3
|
-
`flowsh-cli` turns workflow YAML into executable
|
|
3
|
+
`flowsh-cli` turns workflow YAML into executable shell scripts.
|
|
4
4
|
|
|
5
5
|
Use it when you want a small, reproducible way to encode a workflow once and rerun it locally or in CI.
|
|
6
6
|
|
|
@@ -12,7 +12,7 @@ uvx flowsh-cli path/to/workflows.yml
|
|
|
12
12
|
|
|
13
13
|
That reads the workflow YAML at the path you provide and writes harness scripts to the current working directory.
|
|
14
14
|
|
|
15
|
-
In this repository, the example workflow file lives at
|
|
15
|
+
In this repository, the example workflow file lives at `workflows.yml`.
|
|
16
16
|
|
|
17
17
|
Useful flags:
|
|
18
18
|
|
|
@@ -52,18 +52,23 @@ workflows:
|
|
|
52
52
|
expandPrompt: true
|
|
53
53
|
prompt: |
|
|
54
54
|
Review issue ${ISSUE_NUMBER} and summarize the repository state.
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
55
|
+
- type: parallel
|
|
56
|
+
steps:
|
|
57
|
+
- type: bash
|
|
58
|
+
run: echo "worker A"
|
|
59
|
+
- type: bash
|
|
60
|
+
run: echo "worker B"
|
|
61
|
+
- type: for
|
|
62
|
+
in: ITEMS
|
|
63
|
+
item: ITEM
|
|
64
|
+
steps:
|
|
65
|
+
- type: bash
|
|
66
|
+
run: echo "$ITEM"
|
|
67
|
+
- type: while
|
|
68
|
+
condition: '[ -n "$(ls doc/plan/steps/planned/ 2>/dev/null)" ]'
|
|
69
|
+
steps:
|
|
70
|
+
- type: bash
|
|
71
|
+
run: echo "keep looping until the queue is empty"
|
|
67
72
|
```
|
|
68
73
|
|
|
69
74
|
Harness paths are derived from workflow ids. `wf_example` becomes `example.sh` in the current working directory.
|
|
@@ -74,8 +79,9 @@ Harness paths are derived from workflow ids. `wf_example` becomes `example.sh` i
|
|
|
74
79
|
|---|---|---|
|
|
75
80
|
| `vars` | Execute shell commands and export their stdout into shell variables | Variable names must be uppercase shell identifiers. |
|
|
76
81
|
| `bash` | Run shell commands | Runs with `bash -euo pipefail`. |
|
|
77
|
-
| `agent` | Call
|
|
82
|
+
| `agent` | Call the agent runtime | Supports `agent`, `model`, `command`, `capture`, `expandPrompt`, and `dangerouslySkipPermissions`. |
|
|
78
83
|
| `for` | Iterate over newline-delimited values from a previous `vars` step | Flat iteration only; nested `for` steps are not supported. |
|
|
84
|
+
| `while` | Re-evaluate a Bash condition before each iteration | Use for dynamic queues or other stateful loops that must keep discovering new work. |
|
|
79
85
|
| `parallel` | Run child steps concurrently | Children run as separate branches and the parent waits for all of them. |
|
|
80
86
|
|
|
81
87
|
## Agent Behavior
|
|
@@ -84,7 +90,9 @@ Harness paths are derived from workflow ids. `wf_example` becomes `example.sh` i
|
|
|
84
90
|
|
|
85
91
|
`expandPrompt: true` does plain text replacement only. It does not evaluate shell expressions like `$(...)`, backticks, or globs.
|
|
86
92
|
|
|
87
|
-
Set `
|
|
93
|
+
Set `capture: VARIABLE_NAME` on an `agent` step when you want the full agent output stored in a shell variable for later `vars` or `bash` steps.
|
|
94
|
+
|
|
95
|
+
Set `dangerouslySkipPermissions: true` only when you want the generated harness to pass `--dangerously-skip-permissions` to the agent runtime. The YAML alias `dangerously-skip-permissions` is also accepted.
|
|
88
96
|
|
|
89
97
|
## Validation And Safety
|
|
90
98
|
|
|
@@ -4,7 +4,7 @@ build-backend = "uv_build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "flowsh-cli"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.10.0"
|
|
8
8
|
description = "Generate Bash harness scripts from workflow YAML files."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.11"
|
|
@@ -29,7 +29,7 @@ flowsh-cli = "flowsh_cli.cli:main"
|
|
|
29
29
|
|
|
30
30
|
[dependency-groups]
|
|
31
31
|
dev = [
|
|
32
|
-
"pytest>=
|
|
32
|
+
"pytest>=9.0.3,<10",
|
|
33
33
|
"ruff>=0.14,<0.15",
|
|
34
34
|
]
|
|
35
35
|
|
|
@@ -45,3 +45,4 @@ select = ["E", "F", "I", "UP", "B", "SIM"]
|
|
|
45
45
|
|
|
46
46
|
[tool.ruff.lint.per-file-ignores]
|
|
47
47
|
"tests/test_workflow_to_harness.py" = ["E501"]
|
|
48
|
+
"src/flowsh_cli/skill.py" = ["E501"]
|
|
@@ -1,9 +1,9 @@
|
|
|
1
|
-
"""flowsh-cli: generate
|
|
1
|
+
"""flowsh-cli: generate shell scripts from workflow YAML files."""
|
|
2
2
|
|
|
3
3
|
from flowsh_cli.models import Workflow, WorkflowParseError, parse_workflows
|
|
4
4
|
from flowsh_cli.render import harness_path, render_harness
|
|
5
5
|
|
|
6
|
-
__version__ = "0.
|
|
6
|
+
__version__ = "0.10.0"
|
|
7
7
|
|
|
8
8
|
__all__ = [
|
|
9
9
|
"Workflow",
|
|
@@ -14,11 +14,15 @@ from flowsh_cli import __version__
|
|
|
14
14
|
from flowsh_cli.examples import example_yaml, examples_index
|
|
15
15
|
from flowsh_cli.models import Workflow, WorkflowParseError, parse_workflows, workflow_schema_yaml
|
|
16
16
|
from flowsh_cli.render import harness_path, render_harness
|
|
17
|
+
from flowsh_cli.skill import skill_text
|
|
18
|
+
|
|
19
|
+
DEFAULT_WORKFLOW_YAML = Path("workflows.yml")
|
|
20
|
+
WORKFLOW_YAML_ARGUMENT = typer.Argument(DEFAULT_WORKFLOW_YAML, help="Path to workflow.yml")
|
|
17
21
|
|
|
18
22
|
app = typer.Typer(
|
|
19
23
|
add_completion=False,
|
|
20
24
|
context_settings={"terminal_width": 120, "max_content_width": 120},
|
|
21
|
-
help="Generate reproducible
|
|
25
|
+
help="Generate reproducible shell scripts from workflow YAML files.",
|
|
22
26
|
pretty_exceptions_enable=False,
|
|
23
27
|
)
|
|
24
28
|
|
|
@@ -32,9 +36,9 @@ def main(argv: Sequence[str] | None = None) -> int:
|
|
|
32
36
|
return 0
|
|
33
37
|
|
|
34
38
|
|
|
35
|
-
@app.command(help="Generate reproducible
|
|
39
|
+
@app.command(help="Generate reproducible shell scripts from workflow YAML files.")
|
|
36
40
|
def generate(
|
|
37
|
-
workflow_yaml:
|
|
41
|
+
workflow_yaml: Path = WORKFLOW_YAML_ARGUMENT,
|
|
38
42
|
workflow: Annotated[
|
|
39
43
|
str | None,
|
|
40
44
|
typer.Option(
|
|
@@ -90,6 +94,15 @@ def generate(
|
|
|
90
94
|
is_eager=True,
|
|
91
95
|
),
|
|
92
96
|
] = None,
|
|
97
|
+
skill: Annotated[
|
|
98
|
+
bool,
|
|
99
|
+
typer.Option(
|
|
100
|
+
"--skill",
|
|
101
|
+
callback=lambda value: print_skill(value),
|
|
102
|
+
help="Show the flowsh-cli skill description and exit.",
|
|
103
|
+
is_eager=True,
|
|
104
|
+
),
|
|
105
|
+
] = False,
|
|
93
106
|
output: Annotated[
|
|
94
107
|
Path | None,
|
|
95
108
|
typer.Option(
|
|
@@ -99,12 +112,13 @@ def generate(
|
|
|
99
112
|
),
|
|
100
113
|
] = None,
|
|
101
114
|
) -> None:
|
|
102
|
-
"""Generate
|
|
115
|
+
"""Generate shell scripts from workflow YAML."""
|
|
103
116
|
|
|
104
117
|
_ = version
|
|
105
118
|
_ = schema
|
|
106
119
|
_ = examples
|
|
107
120
|
_ = example
|
|
121
|
+
_ = skill
|
|
108
122
|
|
|
109
123
|
try:
|
|
110
124
|
workflows = parse_workflows(workflow_yaml)
|
|
@@ -173,6 +187,14 @@ def print_example(value: str | None) -> None:
|
|
|
173
187
|
raise typer.Exit
|
|
174
188
|
|
|
175
189
|
|
|
190
|
+
def print_skill(value: bool) -> None:
|
|
191
|
+
if not value:
|
|
192
|
+
return
|
|
193
|
+
|
|
194
|
+
print(skill_text(), end="")
|
|
195
|
+
raise typer.Exit
|
|
196
|
+
|
|
197
|
+
|
|
176
198
|
def write_harnesses(
|
|
177
199
|
workflows: list[Workflow], *, dry_run: bool, force: bool, output_path: Path | None = None
|
|
178
200
|
) -> None:
|
|
@@ -19,6 +19,12 @@ from pydantic import (
|
|
|
19
19
|
|
|
20
20
|
MAX_WORKFLOW_YAML_BYTES = 1_048_576
|
|
21
21
|
|
|
22
|
+
# Execution model: `when:` conditions are evaluated directly in the parent harness
|
|
23
|
+
# process, so any exported variable is visible there immediately. Step bodies for
|
|
24
|
+
# `bash`, `vars`, `for`, and `while` each execute in their own subprocess
|
|
25
|
+
# (`bash -euo pipefail <<'EOF' ... EOF`), so only exported variables (e.g. `vars`
|
|
26
|
+
# step values or a `capture`d agent output) are visible inside them.
|
|
27
|
+
|
|
22
28
|
|
|
23
29
|
class WorkflowParseError(ValueError):
|
|
24
30
|
"""Raised when the input YAML cannot be parsed or validated."""
|
|
@@ -94,6 +100,16 @@ class AgentStep(BaseStep):
|
|
|
94
100
|
agent: str | None = None
|
|
95
101
|
model: str | None = None
|
|
96
102
|
command: str | None = None
|
|
103
|
+
capture: str | None = Field(
|
|
104
|
+
default=None,
|
|
105
|
+
description=(
|
|
106
|
+
"Name of a shell variable that receives the full agent output. "
|
|
107
|
+
"The value is exported, so it is visible in this step's siblings' `when:` "
|
|
108
|
+
"conditions (evaluated in-process) and in later steps' bodies — including "
|
|
109
|
+
"inside `bash`, `vars`, `for`, and `while` steps, each of which runs its own "
|
|
110
|
+
"body in a separate subprocess."
|
|
111
|
+
),
|
|
112
|
+
)
|
|
97
113
|
dangerouslySkipPermissions: bool = Field(
|
|
98
114
|
default=False,
|
|
99
115
|
validation_alias=AliasChoices(
|
|
@@ -112,6 +128,16 @@ class AgentStep(BaseStep):
|
|
|
112
128
|
"are replaced. All other shell syntax in the prompt is safe to use."
|
|
113
129
|
),
|
|
114
130
|
)
|
|
131
|
+
expandFields: bool = Field(
|
|
132
|
+
default=False,
|
|
133
|
+
description=(
|
|
134
|
+
"When true, substitutes ${VAR} and $VAR tokens from vars steps into the model, "
|
|
135
|
+
"agent, and command fields at runtime, using the same safe plain-text replacement "
|
|
136
|
+
"as expandPrompt. Also implies prompt expansion (equivalent to expandPrompt: true) "
|
|
137
|
+
"so that expandFields: true alone expands model, agent, command, and prompt. "
|
|
138
|
+
"Setting both expandPrompt and expandFields is allowed; both simply apply."
|
|
139
|
+
),
|
|
140
|
+
)
|
|
115
141
|
|
|
116
142
|
@model_validator(mode="before")
|
|
117
143
|
@classmethod
|
|
@@ -138,8 +164,15 @@ class AgentStep(BaseStep):
|
|
|
138
164
|
@field_validator("agent")
|
|
139
165
|
@classmethod
|
|
140
166
|
def validate_agent(cls, value: str | None) -> str | None:
|
|
141
|
-
if value is not None and not re.fullmatch(r"[A-Za-z0-9_-]+", value):
|
|
142
|
-
raise ValueError("must match ^[A-Za-z0-9_-]+$")
|
|
167
|
+
if value is not None and not re.fullmatch(r"[A-Za-z0-9_${}-]+", value):
|
|
168
|
+
raise ValueError("must match ^[A-Za-z0-9_${}-]+$")
|
|
169
|
+
return value
|
|
170
|
+
|
|
171
|
+
@field_validator("capture")
|
|
172
|
+
@classmethod
|
|
173
|
+
def validate_capture(cls, value: str | None) -> str | None:
|
|
174
|
+
if value is not None and not re.fullmatch(r"[A-Z_][A-Z0-9_]*", value):
|
|
175
|
+
raise ValueError("must match ^[A-Z_][A-Z0-9_]*$")
|
|
143
176
|
return value
|
|
144
177
|
|
|
145
178
|
|
|
@@ -195,6 +228,37 @@ class ForStep(BaseStep):
|
|
|
195
228
|
raise ValueError("must match ^[A-Z_][A-Z0-9_]*$")
|
|
196
229
|
return value
|
|
197
230
|
|
|
231
|
+
@field_validator("steps")
|
|
232
|
+
@classmethod
|
|
233
|
+
def reject_nested_for_steps(cls, value: list[Step]) -> list[Step]:
|
|
234
|
+
for step in value:
|
|
235
|
+
if isinstance(step, ForStep):
|
|
236
|
+
raise ValueError("nested for steps are not supported")
|
|
237
|
+
return value
|
|
238
|
+
|
|
239
|
+
|
|
240
|
+
class WhileStep(BaseStep):
|
|
241
|
+
type: Literal["while"]
|
|
242
|
+
condition: str
|
|
243
|
+
steps: list[Step] = Field(min_length=1)
|
|
244
|
+
|
|
245
|
+
@field_validator("condition")
|
|
246
|
+
@classmethod
|
|
247
|
+
def validate_condition(cls, value: str) -> str:
|
|
248
|
+
if value.strip() == "":
|
|
249
|
+
raise ValueError("must not be empty")
|
|
250
|
+
if has_unsafe_control_characters(value):
|
|
251
|
+
raise ValueError("must not contain unsafe control characters")
|
|
252
|
+
return value
|
|
253
|
+
|
|
254
|
+
@field_validator("steps")
|
|
255
|
+
@classmethod
|
|
256
|
+
def reject_nested_while_steps(cls, value: list[Step]) -> list[Step]:
|
|
257
|
+
for step in value:
|
|
258
|
+
if isinstance(step, WhileStep):
|
|
259
|
+
raise ValueError("nested while steps are not supported")
|
|
260
|
+
return value
|
|
261
|
+
|
|
198
262
|
|
|
199
263
|
class ParallelStep(BaseStep):
|
|
200
264
|
type: Literal["parallel"]
|
|
@@ -202,12 +266,13 @@ class ParallelStep(BaseStep):
|
|
|
202
266
|
|
|
203
267
|
|
|
204
268
|
Step = Annotated[
|
|
205
|
-
VarsStep | BashStep | AgentStep | ForStep | ParallelStep,
|
|
269
|
+
VarsStep | BashStep | AgentStep | ForStep | WhileStep | ParallelStep,
|
|
206
270
|
Field(discriminator="type"),
|
|
207
271
|
]
|
|
208
272
|
|
|
209
273
|
|
|
210
274
|
ForStep.model_rebuild()
|
|
275
|
+
WhileStep.model_rebuild()
|
|
211
276
|
ParallelStep.model_rebuild()
|
|
212
277
|
|
|
213
278
|
|
|
@@ -10,6 +10,7 @@ from flowsh_cli.models import (
|
|
|
10
10
|
ParallelStep,
|
|
11
11
|
Step,
|
|
12
12
|
VarsStep,
|
|
13
|
+
WhileStep,
|
|
13
14
|
Workflow,
|
|
14
15
|
WorkflowParam,
|
|
15
16
|
)
|
|
@@ -121,19 +122,26 @@ def render_harness(workflow: Workflow) -> str:
|
|
|
121
122
|
"catch() {",
|
|
122
123
|
' local step_name="$1"',
|
|
123
124
|
' local exit_code="$2"',
|
|
124
|
-
'
|
|
125
|
+
' local step_type="${3:-}"',
|
|
126
|
+
' local hint=""',
|
|
127
|
+
' if [[ "$step_type" == "vars" && "$exit_code" == "127" ]]; then',
|
|
128
|
+
' hint=" — vars values are shell commands, not literal strings;'
|
|
129
|
+
' check workflow definition"',
|
|
130
|
+
" fi",
|
|
131
|
+
' log ERROR "Step failed: ${step_name} [${step_type}] (exit=${exit_code})${hint}"',
|
|
125
132
|
"}",
|
|
126
133
|
"",
|
|
127
134
|
section("run_step() - dry-run and failure handling; streams output via tee"),
|
|
128
135
|
"run_step() {",
|
|
129
136
|
' local step_name="$1"',
|
|
137
|
+
' local step_type="${2:-}"',
|
|
130
138
|
"",
|
|
131
139
|
' if [[ "$DRY_RUN" == true ]]; then',
|
|
132
|
-
' log INFO "[DRY-RUN] would run: ${step_name}"',
|
|
140
|
+
' log INFO "[DRY-RUN] would run: ${step_name} [${step_type}]"',
|
|
133
141
|
" return 0",
|
|
134
142
|
" fi",
|
|
135
143
|
"",
|
|
136
|
-
' log INFO "Running step: ${step_name}"',
|
|
144
|
+
' log INFO "Running step: ${step_name} [${step_type}]"',
|
|
137
145
|
"",
|
|
138
146
|
" set +e",
|
|
139
147
|
' if ( : >> "$LOG_FILE" ) 2>/dev/null; then',
|
|
@@ -146,7 +154,7 @@ def render_harness(workflow: Workflow) -> str:
|
|
|
146
154
|
" set -e",
|
|
147
155
|
"",
|
|
148
156
|
" if [[ $status -ne 0 ]]; then",
|
|
149
|
-
' catch "$step_name" "$status"',
|
|
157
|
+
' catch "$step_name" "$status" "$step_type"',
|
|
150
158
|
" fi",
|
|
151
159
|
' return "$status"',
|
|
152
160
|
"}",
|
|
@@ -154,13 +162,14 @@ def render_harness(workflow: Workflow) -> str:
|
|
|
154
162
|
section("run_stateful_step() - dry-run and failure handling without subshells"),
|
|
155
163
|
"run_stateful_step() {",
|
|
156
164
|
' local step_name="$1"',
|
|
165
|
+
' local step_type="${2:-}"',
|
|
157
166
|
"",
|
|
158
167
|
' if [[ "$DRY_RUN" == true ]]; then',
|
|
159
|
-
' log INFO "[DRY-RUN] would run: ${step_name}"',
|
|
168
|
+
' log INFO "[DRY-RUN] would run: ${step_name} [${step_type}]"',
|
|
160
169
|
" return 0",
|
|
161
170
|
" fi",
|
|
162
171
|
"",
|
|
163
|
-
' log INFO "Running step: ${step_name}"',
|
|
172
|
+
' log INFO "Running step: ${step_name} [${step_type}]"',
|
|
164
173
|
"",
|
|
165
174
|
" set +e",
|
|
166
175
|
' "$step_name"',
|
|
@@ -168,18 +177,19 @@ def render_harness(workflow: Workflow) -> str:
|
|
|
168
177
|
" set -e",
|
|
169
178
|
"",
|
|
170
179
|
" if [[ $status -ne 0 ]]; then",
|
|
171
|
-
' catch "$step_name" "$status"',
|
|
180
|
+
' catch "$step_name" "$status" "$step_type"',
|
|
172
181
|
" fi",
|
|
173
182
|
' return "$status"',
|
|
174
183
|
"}",
|
|
175
184
|
"",
|
|
176
|
-
section("run_agent() - prompt handling and
|
|
185
|
+
section("run_agent() - prompt handling and CLI invocation"),
|
|
177
186
|
"run_agent() {",
|
|
178
187
|
' local prompt="$1"',
|
|
179
188
|
' local agent="${2:-}"',
|
|
180
189
|
' local model="${3:-}"',
|
|
181
190
|
' local command="${4:-}"',
|
|
182
|
-
' local
|
|
191
|
+
' local capture="${5:-}"',
|
|
192
|
+
' local dangerously_skip_permissions="${6:-false}"',
|
|
183
193
|
"",
|
|
184
194
|
" local cmd=(opencode run --format json)",
|
|
185
195
|
' if [[ -n "$agent" ]]; then',
|
|
@@ -205,7 +215,18 @@ def render_harness(workflow: Workflow) -> str:
|
|
|
205
215
|
" return 127",
|
|
206
216
|
" fi",
|
|
207
217
|
"",
|
|
208
|
-
'
|
|
218
|
+
' if [[ -n "$capture" ]]; then',
|
|
219
|
+
" local output",
|
|
220
|
+
" local status=0",
|
|
221
|
+
' output="$("${cmd[@]}" -- "$prompt")"',
|
|
222
|
+
" status=$?",
|
|
223
|
+
" printf '%s\\n' \"$output\"",
|
|
224
|
+
' printf -v "$capture" \'%s\' "$output"',
|
|
225
|
+
' export "$capture"',
|
|
226
|
+
' return "$status"',
|
|
227
|
+
" else",
|
|
228
|
+
' "${cmd[@]}" -- "$prompt"',
|
|
229
|
+
" fi",
|
|
209
230
|
"}",
|
|
210
231
|
"",
|
|
211
232
|
section(f"Starting workflow: {workflow.name}"),
|
|
@@ -248,9 +269,36 @@ def render_step(index: int, step: Step, used_function_names: set[str] | None = N
|
|
|
248
269
|
body_lines = [
|
|
249
270
|
f" while IFS= read -r {step.item}; do",
|
|
250
271
|
f" export {step.item}",
|
|
251
|
-
*[
|
|
272
|
+
*[
|
|
273
|
+
f" run_step {fn} {inner_step.type}"
|
|
274
|
+
for fn, inner_step in zip(inner_fns, step.steps, strict=True)
|
|
275
|
+
],
|
|
252
276
|
f' done <<< "${{{step.in_}}}"',
|
|
253
277
|
]
|
|
278
|
+
elif isinstance(step, WhileStep):
|
|
279
|
+
child_fns: list[str] = []
|
|
280
|
+
for i, child in enumerate(step.steps, start=1):
|
|
281
|
+
child_title = child.name or default_step_title(i, child)
|
|
282
|
+
child_fn = step_function_name(i, child.name, used_function_names)
|
|
283
|
+
child_fns.append(child_fn)
|
|
284
|
+
prefix_lines.append(section(f"While child {i} ({child.type}): {child_title}"))
|
|
285
|
+
prefix_lines.append(f"{child_fn}() {{")
|
|
286
|
+
prefix_lines.extend(_render_step_body(child, child_title))
|
|
287
|
+
prefix_lines.append("}")
|
|
288
|
+
prefix_lines.append("")
|
|
289
|
+
|
|
290
|
+
body_lines = []
|
|
291
|
+
if step.when is not None:
|
|
292
|
+
body_lines.append(f" if ! ({step.when}); then")
|
|
293
|
+
body_lines.append(f" log INFO {bash_quote(f'Step skipped (when): {title}')}")
|
|
294
|
+
body_lines.append(" return 0")
|
|
295
|
+
body_lines.append(" fi")
|
|
296
|
+
body_lines.append("")
|
|
297
|
+
|
|
298
|
+
body_lines.append(f" while ({step.condition}); do")
|
|
299
|
+
for child_fn in child_fns:
|
|
300
|
+
body_lines.append(f" run_step {child_fn} || return $?")
|
|
301
|
+
body_lines.append(" done")
|
|
254
302
|
elif isinstance(step, ParallelStep):
|
|
255
303
|
child_fns: list[str] = []
|
|
256
304
|
for i, child in enumerate(step.steps, start=1):
|
|
@@ -273,7 +321,7 @@ def render_step(index: int, step: Step, used_function_names: set[str] | None = N
|
|
|
273
321
|
else:
|
|
274
322
|
body_lines = _render_step_body(step, title)
|
|
275
323
|
|
|
276
|
-
runner = "run_stateful_step" if isinstance(step, (VarsStep, ForStep)) else "run_step"
|
|
324
|
+
runner = "run_stateful_step" if isinstance(step, (VarsStep, ForStep, WhileStep)) else "run_step"
|
|
277
325
|
outer_lines = [
|
|
278
326
|
section(f"Step {index} ({step.type}): {title}"),
|
|
279
327
|
f"{function_name}() {{",
|
|
@@ -283,13 +331,32 @@ def render_step(index: int, step: Step, used_function_names: set[str] | None = N
|
|
|
283
331
|
outer_lines.extend(
|
|
284
332
|
[
|
|
285
333
|
"}",
|
|
286
|
-
f"{runner} {function_name}",
|
|
334
|
+
f"{runner} {function_name} {step.type}",
|
|
287
335
|
"",
|
|
288
336
|
]
|
|
289
337
|
)
|
|
290
338
|
return prefix_lines + outer_lines
|
|
291
339
|
|
|
292
340
|
|
|
341
|
+
def _find_var_tokens(text: str) -> list[str]:
|
|
342
|
+
"""Return unique ${VAR}/$VAR tokens (matching [A-Z_][A-Z0-9_]*) in order of first appearance."""
|
|
343
|
+
braced = re.findall(r"\$\{([A-Z_][A-Z0-9_]*)\}", text)
|
|
344
|
+
bare = re.findall(r"\$([A-Z_][A-Z0-9_]*)(?!\w)", text)
|
|
345
|
+
seen: dict[str, None] = {}
|
|
346
|
+
for var in braced + bare:
|
|
347
|
+
seen[var] = None
|
|
348
|
+
return list(seen)
|
|
349
|
+
|
|
350
|
+
|
|
351
|
+
def _expand_var_lines(local_name: str, tokens: list[str]) -> list[str]:
|
|
352
|
+
"""Return bash lines that expand ${VAR}/$VAR literal tokens in `local_name` at runtime."""
|
|
353
|
+
lines: list[str] = []
|
|
354
|
+
for var in tokens:
|
|
355
|
+
lines.append(f' _p=\'${{{var}}}\'; {local_name}="${{{local_name}//"$_p"/"${var}"}}"')
|
|
356
|
+
lines.append(f' _p=\'${var}\'; {local_name}="${{{local_name}//"$_p"/"${var}"}}"')
|
|
357
|
+
return lines
|
|
358
|
+
|
|
359
|
+
|
|
293
360
|
def _render_step_body(step: Step, title: str) -> list[str]:
|
|
294
361
|
"""Return indented body lines for a step function (no wrapper, no run_step call)."""
|
|
295
362
|
lines: list[str] = []
|
|
@@ -338,23 +405,24 @@ def _render_step_body(step: Step, title: str) -> list[str]:
|
|
|
338
405
|
" )",
|
|
339
406
|
]
|
|
340
407
|
)
|
|
341
|
-
if step.expandPrompt:
|
|
408
|
+
if step.expandPrompt or step.expandFields:
|
|
342
409
|
_prompt_no_code = re.sub(r"```.*?```", "", step.prompt, flags=re.DOTALL)
|
|
343
|
-
|
|
344
|
-
bare = re.findall(r"\$([A-Z_][A-Z0-9_]*)(?!\w)", _prompt_no_code)
|
|
345
|
-
seen: dict[str, None] = {}
|
|
346
|
-
for var in braced + bare:
|
|
347
|
-
seen[var] = None
|
|
348
|
-
for var in seen:
|
|
349
|
-
lines.append(f' _p=\'${{{var}}}\'; prompt="${{prompt//"$_p"/"${var}"}}"')
|
|
350
|
-
lines.append(f' _p=\'${var}\'; prompt="${{prompt//"$_p"/"${var}"}}"')
|
|
410
|
+
lines.extend(_expand_var_lines("prompt", _find_var_tokens(_prompt_no_code)))
|
|
351
411
|
lines.append(f" local agent={bash_quote(step.agent or '')}")
|
|
412
|
+
if step.expandFields and step.agent:
|
|
413
|
+
lines.extend(_expand_var_lines("agent", _find_var_tokens(step.agent)))
|
|
352
414
|
lines.append(f" local model={bash_quote(step.model or '')}")
|
|
415
|
+
if step.expandFields and step.model:
|
|
416
|
+
lines.extend(_expand_var_lines("model", _find_var_tokens(step.model)))
|
|
353
417
|
lines.append(f" local command={bash_quote(step.command or '')}")
|
|
418
|
+
if step.expandFields and step.command:
|
|
419
|
+
lines.extend(_expand_var_lines("command", _find_var_tokens(step.command)))
|
|
420
|
+
lines.append(f" local capture={bash_quote(step.capture or '')}")
|
|
354
421
|
dangerous_skip_permissions = "true" if step.dangerouslySkipPermissions else "false"
|
|
355
422
|
lines.append(f" local dangerously_skip_permissions={dangerous_skip_permissions}")
|
|
356
423
|
lines.append(
|
|
357
|
-
' run_agent "$prompt" "$agent" "$model"
|
|
424
|
+
' run_agent "$prompt" "$agent" "$model" '
|
|
425
|
+
'"$command" "$capture" "$dangerously_skip_permissions"'
|
|
358
426
|
)
|
|
359
427
|
elif isinstance(step, ForStep):
|
|
360
428
|
raise AssertionError("nested for steps are not supported")
|
|
@@ -393,6 +461,8 @@ def default_step_title(index: int, step: Step) -> str:
|
|
|
393
461
|
return truncate_one_line(step.prompt)
|
|
394
462
|
if isinstance(step, ForStep):
|
|
395
463
|
return f"for {step.item} in {step.in_}"
|
|
464
|
+
if isinstance(step, WhileStep):
|
|
465
|
+
return f"while {truncate_one_line(step.condition)}"
|
|
396
466
|
if isinstance(step, ParallelStep):
|
|
397
467
|
return f"parallel ({len(step.steps)} steps)"
|
|
398
468
|
return f"step {index}"
|
|
@@ -468,6 +538,11 @@ def _render_arg_block(params: list[WorkflowParam]) -> list[str]:
|
|
|
468
538
|
|
|
469
539
|
lines: list[str] = [
|
|
470
540
|
"DRY_RUN=false",
|
|
541
|
+
]
|
|
542
|
+
for param in params:
|
|
543
|
+
lines.append(f'{param.name}="${{{param.name}:-}}"')
|
|
544
|
+
lines.append(f"export {param.name}")
|
|
545
|
+
lines += [
|
|
471
546
|
"POSITIONAL_ARGS=()",
|
|
472
547
|
"",
|
|
473
548
|
"while [[ $# -gt 0 ]]; do",
|
|
@@ -0,0 +1,244 @@
|
|
|
1
|
+
from __future__ import annotations
|
|
2
|
+
|
|
3
|
+
SKILL_MARKDOWN = """\
|
|
4
|
+
---
|
|
5
|
+
name: flowsh-cli
|
|
6
|
+
description: Generate executable shell harness scripts from workflow definitions. Use when creating, modifying, validating, reviewing, troubleshooting, or generating workflow-to-shell automation, especially for orchestrating bash and agent calls.
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Workflow-to-Shell Generation with flowsh-cli
|
|
10
|
+
|
|
11
|
+
Use this skill whenever working with workflow definitions that produce executable Bash or shell scripts.
|
|
12
|
+
|
|
13
|
+
## Purpose
|
|
14
|
+
|
|
15
|
+
flowsh-cli converts workflow definitions into executable shell harness scripts.
|
|
16
|
+
|
|
17
|
+
Workflow definitions are the source of truth.
|
|
18
|
+
|
|
19
|
+
Generated shell scripts are build artifacts derived from those workflow definitions.
|
|
20
|
+
|
|
21
|
+
The CLI parses, validates, and generates in a single operation. Validation is therefore an essential part of workflow generation.
|
|
22
|
+
|
|
23
|
+
A workflow is considered correct when:
|
|
24
|
+
|
|
25
|
+
* It conforms to the current schema.
|
|
26
|
+
* Validation succeeds.
|
|
27
|
+
* Generated shell output matches the intended behavior.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
# Mandatory Operating Procedure
|
|
32
|
+
|
|
33
|
+
## Rule 1: Always Consult the Current Schema First
|
|
34
|
+
|
|
35
|
+
Before creating, modifying, reviewing, troubleshooting, or generating workflows:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
uvx flowsh-cli --schema
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
The schema is the authoritative specification.
|
|
42
|
+
|
|
43
|
+
Do not rely on:
|
|
44
|
+
|
|
45
|
+
* Memory
|
|
46
|
+
* Existing workflow files
|
|
47
|
+
* Previous examples
|
|
48
|
+
* Assumptions about supported fields
|
|
49
|
+
|
|
50
|
+
If a workflow conflicts with the schema, the schema is authoritative.
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Rule 2: Validate Through Generation
|
|
55
|
+
|
|
56
|
+
flowsh-cli validates workflows as part of generation.
|
|
57
|
+
|
|
58
|
+
Use:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
uvx flowsh-cli [workflow-name].yml
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Validation failures must be resolved before reasoning about generated shell output.
|
|
65
|
+
|
|
66
|
+
Do not assume a workflow is valid simply because it appears similar to previous examples.
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## Rule 3: Always Verify After Changes
|
|
71
|
+
|
|
72
|
+
After modifying a workflow:
|
|
73
|
+
|
|
74
|
+
1. Retrieve the current schema.
|
|
75
|
+
2. Run a dry-run generation.
|
|
76
|
+
3. Review any validation errors or warnings.
|
|
77
|
+
4. Review the generated output.
|
|
78
|
+
5. Confirm generated shell behavior matches intent.
|
|
79
|
+
|
|
80
|
+
Example:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
uvx flowsh-cli [workflow-name].yml --dry-run
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
A workflow modification is not complete until a successful dry-run has been reviewed.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Rule 4: Make Minimal Changes
|
|
91
|
+
|
|
92
|
+
When modifying workflows:
|
|
93
|
+
|
|
94
|
+
* Prefer the smallest possible change.
|
|
95
|
+
* Avoid unrelated refactoring.
|
|
96
|
+
* Preserve existing behavior unless explicitly requested.
|
|
97
|
+
* Keep workflow structure stable.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
# Workflow Review Priorities
|
|
102
|
+
|
|
103
|
+
When reviewing workflows:
|
|
104
|
+
|
|
105
|
+
1. Schema compliance.
|
|
106
|
+
2. Successful validation.
|
|
107
|
+
3. Workflow correctness.
|
|
108
|
+
4. Generated shell correctness.
|
|
109
|
+
5. Shell safety.
|
|
110
|
+
6. Maintainability.
|
|
111
|
+
|
|
112
|
+
Generated shell behavior should only be reviewed after validation succeeds.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
# Common Commands
|
|
117
|
+
|
|
118
|
+
Display help:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
uvx flowsh-cli --help
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Display the current schema:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
uvx flowsh-cli --schema
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Generate all workflows:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
uvx flowsh-cli [workflow-name].yml
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Generate a specific workflow:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
uvx flowsh-cli [workflow-name].yml --workflow wf_example
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Preview generation without writing files:
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
uvx flowsh-cli [workflow-name].yml --dry-run
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Overwrite existing generated files:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
uvx flowsh-cli [workflow-name].yml --force
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
# Folder Structure & File Management
|
|
157
|
+
|
|
158
|
+
By convention, organize workflow files and generated harness scripts under a `.harness/` folder:
|
|
159
|
+
|
|
160
|
+
```
|
|
161
|
+
project/
|
|
162
|
+
└── .harness/
|
|
163
|
+
├── [workflow-name].yml # Workflow file
|
|
164
|
+
└── [harness-name].sh # Generated harness script
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
**Convention:**
|
|
168
|
+
- Store workflow definitions under `[workflow-name].yml` files
|
|
169
|
+
- Generated harness scripts are produced in `.harness/` directory with names matching `[harness-name].sh`
|
|
170
|
+
- Generated artifacts are build products—regenerate them as needed from source workflows
|
|
171
|
+
- Use `.gitignore` to exclude generated `.sh` files if desired, or commit them for reproducibility
|
|
172
|
+
|
|
173
|
+
**Generation command:**
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
uvx flowsh-cli .harness/[workflow-name].yml
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
This generates harnesses in `.harness/` alongside the source workflow definitions.
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
# Example Workflow
|
|
184
|
+
|
|
185
|
+
Always verify against the current schema before creating production workflows.
|
|
186
|
+
|
|
187
|
+
```yaml
|
|
188
|
+
workflows:
|
|
189
|
+
- id: wf_example
|
|
190
|
+
name: Example Workflow
|
|
191
|
+
|
|
192
|
+
steps:
|
|
193
|
+
- type: vars
|
|
194
|
+
values:
|
|
195
|
+
TODAY: date -u +%F
|
|
196
|
+
|
|
197
|
+
- type: bash
|
|
198
|
+
run: |
|
|
199
|
+
echo "$TODAY"
|
|
200
|
+
|
|
201
|
+
- type: agent
|
|
202
|
+
prompt: |
|
|
203
|
+
Summarize the output.
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
---
|
|
207
|
+
|
|
208
|
+
# Troubleshooting Procedure
|
|
209
|
+
|
|
210
|
+
If generation fails:
|
|
211
|
+
|
|
212
|
+
1. Retrieve the latest schema.
|
|
213
|
+
2. Review validation errors.
|
|
214
|
+
3. Correct the workflow definition.
|
|
215
|
+
4. Run a dry-run.
|
|
216
|
+
5. Review generated output.
|
|
217
|
+
6. Apply the smallest possible fix.
|
|
218
|
+
7. Repeat until validation and generation succeed.
|
|
219
|
+
|
|
220
|
+
---
|
|
221
|
+
|
|
222
|
+
# Expected Behavior
|
|
223
|
+
|
|
224
|
+
Always:
|
|
225
|
+
|
|
226
|
+
* Consult the latest schema first.
|
|
227
|
+
* Treat the schema as authoritative.
|
|
228
|
+
* Use flowsh-cli validation before evaluating generated output.
|
|
229
|
+
* Run a dry-run after workflow modifications.
|
|
230
|
+
* Review generated shell behavior before declaring success.
|
|
231
|
+
* Prefer minimal changes.
|
|
232
|
+
|
|
233
|
+
Never:
|
|
234
|
+
|
|
235
|
+
* Assume schema details from memory.
|
|
236
|
+
* Skip schema inspection.
|
|
237
|
+
* Ignore validation errors.
|
|
238
|
+
* Skip dry-run verification after changes.
|
|
239
|
+
* Declare success without reviewing generated output.
|
|
240
|
+
"""
|
|
241
|
+
|
|
242
|
+
|
|
243
|
+
def skill_text() -> str:
|
|
244
|
+
return SKILL_MARKDOWN
|
|
File without changes
|
|
File without changes
|