dyngle 0.5.0__tar.gz → 0.7.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of dyngle might be problematic. Click here for more details.

@@ -0,0 +1,260 @@
1
+ # Dyngle
2
+
3
+ Use cases
4
+
5
+ - A task runner
6
+ - A lightweight workflow engine
7
+ - A replacement for Make in Python projects
8
+ - A replacement for short functions in RC files
9
+ - Freedom from quirky Bash syntax
10
+
11
+ Technical foundations
12
+
13
+ - Configuration, task definition, and flow control in YAML
14
+ - Operations as system commands using a familiar shell-like syntax
15
+ - Expressions and logic in pure Python
16
+
17
+ ## Quick installation (MacOS)
18
+
19
+ ```bash
20
+ brew install python@3.11
21
+ python3.11 -m pip install pipx
22
+ pipx install dyngle
23
+ ```
24
+
25
+ ## Getting started
26
+
27
+ Create a file `.dyngle.yml`:
28
+
29
+ ```yaml
30
+ dyngle:
31
+ operations:
32
+ hello:
33
+ - echo "Hello world"
34
+ ```
35
+
36
+ Run an operation:
37
+
38
+ ```bash
39
+ dyngle run hello
40
+ ```
41
+
42
+ ## Configuration
43
+
44
+ Dyngle reads configuration from YAML files. You can specify the config file location using:
45
+
46
+ - `--config` command line option
47
+ - `DYNGLE_CONFIG` environment variable
48
+ - `.dyngle.yml` in current directory
49
+ - `~/.dyngle.yml` in home directory
50
+
51
+ ## Operations
52
+
53
+ Operations are defined under `dyngle:` in the configuration. In its simplest form, an Operation takes the form of a YAML array defining the steps, as a system command with space-separated arguments. In that sense, a Dyngle operation looks something akin to a "PHONY" Make target, a short Bash script, or a CI/CD job. As a serious example, consider the `init` operation from the Dyngle configuration delivered with the project's source code.
54
+
55
+ ```yaml
56
+ dyngle:
57
+ operations:
58
+ init:
59
+ - rm -rf .venv
60
+ - python3.11 -m venv .venv
61
+ - .venv/bin/pip install --upgrade pip poetry
62
+ ```
63
+
64
+ The elements of the YAML array _look_ like lines of Bash, but Dyngle replaces the shell with YAML-based flow control and Python-based expressions (described below). So shell-specific operators such as `|`, `>`, and `$VARIABLES` won't work.
65
+
66
+ ## Data and Templates
67
+
68
+ Dyngle maintains a block of Data throughout an operation, which is a set of named values (Python `dict`, YAML "mapping"). The values are usually strings but can also be other data types that are valid in both YAML and Python.
69
+
70
+ The `dyngle run` command feeds the contents of stdin to the Operation as Data, by converting a YAML mapping to named Python values. The values may be substituted into commands or arguments in Steps using double-curly-bracket syntax (`{{` and `}}`) similar to Jinja2.
71
+
72
+ For example, consider the following configuration:
73
+
74
+ ``` yaml
75
+ dyngle:
76
+ operations:
77
+ hello:
78
+ - echo "Hello {{name}}!"
79
+ ```
80
+
81
+ Cram some YAML into stdin to try it in your shell:
82
+
83
+ ```bash
84
+ echo "name: Francis" | dyngle run hello
85
+ ```
86
+
87
+ The output will say:
88
+
89
+ ```text
90
+ Hello Francis!
91
+ ```
92
+
93
+ ## Expressions
94
+
95
+ Operations may contain Expressions, written in Python, that can be referenced in operation steps using the same syntax as for Data. In the case of a naming conflict, an Expression takes precedence over Data with the same name. Expressions can reference names in the Data directly.
96
+
97
+ Expressions may be defined in either of two ways in the configuration:
98
+
99
+ 1. Global Expressions, under the `dyngle:` mapping, using the `expressions:` key.
100
+ 2. Local Expressions, within a single Operation, in which case the Steps of the operation require a `steps:` key.
101
+
102
+ Here's an example of a global Expression
103
+
104
+ ```yaml
105
+ dyngle:
106
+ expressions:
107
+ count: len(name)
108
+ operations:
109
+ say-hello:
110
+ - echo "Hello {{name}}! Your name has {{count}} characters."
111
+ ```
112
+
113
+ For completeness, consider the following example using a local Expression for the same purpose.
114
+
115
+ ```yaml
116
+ dyngle:
117
+ operations:
118
+ say-hello:
119
+ expressions:
120
+ count: len(name)
121
+ steps:
122
+ - echo "Hello {{name}}! Your name has {{count}} characters."
123
+ ```
124
+
125
+ Expressions can use a controlled subset of the Python standard library, including:
126
+
127
+ - Built-in data types such as `str()`
128
+ - Essential built-in functions such as `len()`
129
+ - The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
130
+ - A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
131
+ - A restricted version of `Path()` that only operates within the current working directory
132
+ - Various other useful utilities, mostly read-only, such as the `math` module
133
+ - A special function called `resolve` which resolves data expressions using the same logic as in templates
134
+ - An array `args` containing arguments passed to the `dyngle run` command after the Operation name
135
+
136
+ **NOTE** Some capabilities of the Expression namespace might be limited in the future. The goal is support purely read-only operations within Expressions.
137
+
138
+ Expressions behave like functions that take no arguments, using the Data as a namespace. So Expressions reference Data directly as local names in Python.
139
+
140
+ YAML keys can contain hyphens, which are fully supported in Dyngle. To reference a hyphenated key in an Expression, choose:
141
+
142
+ - Reference the name using underscores instead of hyphens (they are automatically replaced), OR
143
+ - Use the built-in special-purpose `resolve()` function (which can also be used to reference other expressions)
144
+
145
+
146
+
147
+ ```yaml
148
+ dyngle:
149
+ expressions:
150
+ say-hello: >-
151
+ 'Hello ' + full_name + '!'
152
+ ```
153
+
154
+ ... or using the `resolve()` function, which also allows expressions to essentially call other expressions, using the same underlying data set.
155
+
156
+ ```yaml
157
+ dyngle:
158
+ expressions:
159
+ hello: >-
160
+ 'Hello ' + resolve('formal-name') + '!'
161
+ formal-name: >-
162
+ 'Ms. ' + full_name
163
+ ```
164
+
165
+ Note it's also _possible_ to call other expressions by name as functions, if they only return hard-coded values (i.e. constants).
166
+
167
+ ```yaml
168
+ dyngle:
169
+ expressions:
170
+ author-name: Francis Potter
171
+ author-hello: >-
172
+ 'Hello ' + author_name()
173
+ ```
174
+
175
+ Here are some slightly more sophisticated exercises using Expression reference syntax:
176
+
177
+ ```yaml
178
+ dyngle:
179
+ operations:
180
+ reference-hyphenated-data-key:
181
+ expressions:
182
+ spaced-name: "' '.join([x for x in first_name])"
183
+ count-name: len(resolve('first-name'))
184
+ x-name: "'X' * int(resolve('count-name'))"
185
+ steps:
186
+ - echo "Your name is {{first-name}} with {{count-name}} characters, but I will call you '{{spaced-name}}' or maybe '{{x-name}}'"
187
+ reference-expression-using-function-syntax:
188
+ expressions:
189
+ name: "'George'"
190
+ works: "name()"
191
+ double: "name * 2"
192
+ fails: double()
193
+ steps:
194
+ - echo "It works to call you {{works}}"
195
+ # - echo "I have trouble calling you {{fails}}"
196
+ ```
197
+
198
+ Finally, here's an example using args:
199
+
200
+ ```yaml
201
+ dyngle:
202
+ operations:
203
+ name-from-arg:
204
+ expressions:
205
+ name: "args[0]"
206
+ steps:
207
+ - echo "Hello {{name}}"
208
+ ```
209
+
210
+ ## Data assignment operator
211
+
212
+ The Steps parser supports one special operator which assigns the output (stdout) of its command to a Data field for use in subsequent steps.
213
+
214
+ The operator is `=>` and must go after the command and its arguments. Follow the operator with the name of the Data key to assign.
215
+
216
+ Example:
217
+
218
+ ```yaml
219
+ dyngle:
220
+ operations:
221
+ today:
222
+ expressions:
223
+ just-the-date: resolve('full-date')[0:10]
224
+ steps:
225
+ - date => full-date
226
+ - echo "Today is {{just-the-date}}"
227
+ ```
228
+
229
+ (Note Dyngle does provide Python date operations in the Expression namespace, which might provide a better way to perform the same operation as this example, but it suffices to demonstrate assignment)
230
+
231
+ ## Lifecycle
232
+
233
+ The lifecycle of an operation is:
234
+
235
+ 1. Load Data if it exists from YAML on stdin (if no tty)
236
+ 2. Find the named Operation in the configuration
237
+ 2. Perform template rendering on the first Step, using Data and Expressions
238
+ 3. Execute the Step in a subprocess
239
+ 4. Continue with the next Step
240
+
241
+ Note that operations in the config are _not_ full shell lines. They are passed directly to the system.
242
+
243
+ ## Imports
244
+
245
+ Configuration files can import other configuration files, by providing an entry `imports:` with an array of filepaths. The most obvious example is a Dyngle config in a local directory which imports the user-level configuration.
246
+
247
+ ```yaml
248
+ dyngle:
249
+ imports:
250
+ - ~/.dyngle.yml
251
+ expressions:
252
+ operations:
253
+ ```
254
+
255
+ In the event of item name conflicts, expressions and operations are loaded from imports in the order specified, so imports lower in the array will override those higher up. The expressions and operations defined in the main file override the imports. Imports are not recursive.
256
+
257
+ ## Security
258
+
259
+ Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. Use with caution.
260
+
dyngle-0.7.0/PKG-INFO ADDED
@@ -0,0 +1,276 @@
1
+ Metadata-Version: 2.3
2
+ Name: dyngle
3
+ Version: 0.7.0
4
+ Summary: Run lightweight local workflows
5
+ License: MIT
6
+ Author: Steampunk Wizard
7
+ Author-email: dyngle@steamwiz.io
8
+ Requires-Python: >=3.11,<3.12
9
+ Classifier: License :: OSI Approved :: MIT License
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Programming Language :: Python :: 3.11
12
+ Requires-Dist: requests (>=2.32.3,<3.0.0)
13
+ Requires-Dist: wizlib (>=3.1.4,<4.0.0)
14
+ Description-Content-Type: text/markdown
15
+
16
+ # Dyngle
17
+
18
+ Use cases
19
+
20
+ - A task runner
21
+ - A lightweight workflow engine
22
+ - A replacement for Make in Python projects
23
+ - A replacement for short functions in RC files
24
+ - Freedom from quirky Bash syntax
25
+
26
+ Technical foundations
27
+
28
+ - Configuration, task definition, and flow control in YAML
29
+ - Operations as system commands using a familiar shell-like syntax
30
+ - Expressions and logic in pure Python
31
+
32
+ ## Quick installation (MacOS)
33
+
34
+ ```bash
35
+ brew install python@3.11
36
+ python3.11 -m pip install pipx
37
+ pipx install dyngle
38
+ ```
39
+
40
+ ## Getting started
41
+
42
+ Create a file `.dyngle.yml`:
43
+
44
+ ```yaml
45
+ dyngle:
46
+ operations:
47
+ hello:
48
+ - echo "Hello world"
49
+ ```
50
+
51
+ Run an operation:
52
+
53
+ ```bash
54
+ dyngle run hello
55
+ ```
56
+
57
+ ## Configuration
58
+
59
+ Dyngle reads configuration from YAML files. You can specify the config file location using:
60
+
61
+ - `--config` command line option
62
+ - `DYNGLE_CONFIG` environment variable
63
+ - `.dyngle.yml` in current directory
64
+ - `~/.dyngle.yml` in home directory
65
+
66
+ ## Operations
67
+
68
+ Operations are defined under `dyngle:` in the configuration. In its simplest form, an Operation takes the form of a YAML array defining the steps, as a system command with space-separated arguments. In that sense, a Dyngle operation looks something akin to a "PHONY" Make target, a short Bash script, or a CI/CD job. As a serious example, consider the `init` operation from the Dyngle configuration delivered with the project's source code.
69
+
70
+ ```yaml
71
+ dyngle:
72
+ operations:
73
+ init:
74
+ - rm -rf .venv
75
+ - python3.11 -m venv .venv
76
+ - .venv/bin/pip install --upgrade pip poetry
77
+ ```
78
+
79
+ The elements of the YAML array _look_ like lines of Bash, but Dyngle replaces the shell with YAML-based flow control and Python-based expressions (described below). So shell-specific operators such as `|`, `>`, and `$VARIABLES` won't work.
80
+
81
+ ## Data and Templates
82
+
83
+ Dyngle maintains a block of Data throughout an operation, which is a set of named values (Python `dict`, YAML "mapping"). The values are usually strings but can also be other data types that are valid in both YAML and Python.
84
+
85
+ The `dyngle run` command feeds the contents of stdin to the Operation as Data, by converting a YAML mapping to named Python values. The values may be substituted into commands or arguments in Steps using double-curly-bracket syntax (`{{` and `}}`) similar to Jinja2.
86
+
87
+ For example, consider the following configuration:
88
+
89
+ ``` yaml
90
+ dyngle:
91
+ operations:
92
+ hello:
93
+ - echo "Hello {{name}}!"
94
+ ```
95
+
96
+ Cram some YAML into stdin to try it in your shell:
97
+
98
+ ```bash
99
+ echo "name: Francis" | dyngle run hello
100
+ ```
101
+
102
+ The output will say:
103
+
104
+ ```text
105
+ Hello Francis!
106
+ ```
107
+
108
+ ## Expressions
109
+
110
+ Operations may contain Expressions, written in Python, that can be referenced in operation steps using the same syntax as for Data. In the case of a naming conflict, an Expression takes precedence over Data with the same name. Expressions can reference names in the Data directly.
111
+
112
+ Expressions may be defined in either of two ways in the configuration:
113
+
114
+ 1. Global Expressions, under the `dyngle:` mapping, using the `expressions:` key.
115
+ 2. Local Expressions, within a single Operation, in which case the Steps of the operation require a `steps:` key.
116
+
117
+ Here's an example of a global Expression
118
+
119
+ ```yaml
120
+ dyngle:
121
+ expressions:
122
+ count: len(name)
123
+ operations:
124
+ say-hello:
125
+ - echo "Hello {{name}}! Your name has {{count}} characters."
126
+ ```
127
+
128
+ For completeness, consider the following example using a local Expression for the same purpose.
129
+
130
+ ```yaml
131
+ dyngle:
132
+ operations:
133
+ say-hello:
134
+ expressions:
135
+ count: len(name)
136
+ steps:
137
+ - echo "Hello {{name}}! Your name has {{count}} characters."
138
+ ```
139
+
140
+ Expressions can use a controlled subset of the Python standard library, including:
141
+
142
+ - Built-in data types such as `str()`
143
+ - Essential built-in functions such as `len()`
144
+ - The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
145
+ - A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
146
+ - A restricted version of `Path()` that only operates within the current working directory
147
+ - Various other useful utilities, mostly read-only, such as the `math` module
148
+ - A special function called `resolve` which resolves data expressions using the same logic as in templates
149
+ - An array `args` containing arguments passed to the `dyngle run` command after the Operation name
150
+
151
+ **NOTE** Some capabilities of the Expression namespace might be limited in the future. The goal is support purely read-only operations within Expressions.
152
+
153
+ Expressions behave like functions that take no arguments, using the Data as a namespace. So Expressions reference Data directly as local names in Python.
154
+
155
+ YAML keys can contain hyphens, which are fully supported in Dyngle. To reference a hyphenated key in an Expression, choose:
156
+
157
+ - Reference the name using underscores instead of hyphens (they are automatically replaced), OR
158
+ - Use the built-in special-purpose `resolve()` function (which can also be used to reference other expressions)
159
+
160
+
161
+
162
+ ```yaml
163
+ dyngle:
164
+ expressions:
165
+ say-hello: >-
166
+ 'Hello ' + full_name + '!'
167
+ ```
168
+
169
+ ... or using the `resolve()` function, which also allows expressions to essentially call other expressions, using the same underlying data set.
170
+
171
+ ```yaml
172
+ dyngle:
173
+ expressions:
174
+ hello: >-
175
+ 'Hello ' + resolve('formal-name') + '!'
176
+ formal-name: >-
177
+ 'Ms. ' + full_name
178
+ ```
179
+
180
+ Note it's also _possible_ to call other expressions by name as functions, if they only return hard-coded values (i.e. constants).
181
+
182
+ ```yaml
183
+ dyngle:
184
+ expressions:
185
+ author-name: Francis Potter
186
+ author-hello: >-
187
+ 'Hello ' + author_name()
188
+ ```
189
+
190
+ Here are some slightly more sophisticated exercises using Expression reference syntax:
191
+
192
+ ```yaml
193
+ dyngle:
194
+ operations:
195
+ reference-hyphenated-data-key:
196
+ expressions:
197
+ spaced-name: "' '.join([x for x in first_name])"
198
+ count-name: len(resolve('first-name'))
199
+ x-name: "'X' * int(resolve('count-name'))"
200
+ steps:
201
+ - echo "Your name is {{first-name}} with {{count-name}} characters, but I will call you '{{spaced-name}}' or maybe '{{x-name}}'"
202
+ reference-expression-using-function-syntax:
203
+ expressions:
204
+ name: "'George'"
205
+ works: "name()"
206
+ double: "name * 2"
207
+ fails: double()
208
+ steps:
209
+ - echo "It works to call you {{works}}"
210
+ # - echo "I have trouble calling you {{fails}}"
211
+ ```
212
+
213
+ Finally, here's an example using args:
214
+
215
+ ```yaml
216
+ dyngle:
217
+ operations:
218
+ name-from-arg:
219
+ expressions:
220
+ name: "args[0]"
221
+ steps:
222
+ - echo "Hello {{name}}"
223
+ ```
224
+
225
+ ## Data assignment operator
226
+
227
+ The Steps parser supports one special operator which assigns the output (stdout) of its command to a Data field for use in subsequent steps.
228
+
229
+ The operator is `=>` and must go after the command and its arguments. Follow the operator with the name of the Data key to assign.
230
+
231
+ Example:
232
+
233
+ ```yaml
234
+ dyngle:
235
+ operations:
236
+ today:
237
+ expressions:
238
+ just-the-date: resolve('full-date')[0:10]
239
+ steps:
240
+ - date => full-date
241
+ - echo "Today is {{just-the-date}}"
242
+ ```
243
+
244
+ (Note Dyngle does provide Python date operations in the Expression namespace, which might provide a better way to perform the same operation as this example, but it suffices to demonstrate assignment)
245
+
246
+ ## Lifecycle
247
+
248
+ The lifecycle of an operation is:
249
+
250
+ 1. Load Data if it exists from YAML on stdin (if no tty)
251
+ 2. Find the named Operation in the configuration
252
+ 2. Perform template rendering on the first Step, using Data and Expressions
253
+ 3. Execute the Step in a subprocess
254
+ 4. Continue with the next Step
255
+
256
+ Note that operations in the config are _not_ full shell lines. They are passed directly to the system.
257
+
258
+ ## Imports
259
+
260
+ Configuration files can import other configuration files, by providing an entry `imports:` with an array of filepaths. The most obvious example is a Dyngle config in a local directory which imports the user-level configuration.
261
+
262
+ ```yaml
263
+ dyngle:
264
+ imports:
265
+ - ~/.dyngle.yml
266
+ expressions:
267
+ operations:
268
+ ```
269
+
270
+ In the event of item name conflicts, expressions and operations are loaded from imports in the order specified, so imports lower in the array will override those higher up. The expressions and operations defined in the main file override the imports. Imports are not recursive.
271
+
272
+ ## Security
273
+
274
+ Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. Use with caution.
275
+
276
+
@@ -8,6 +8,7 @@ from wizlib.ui_handler import UIHandler
8
8
  from dyngle.command import DyngleCommand
9
9
  from dyngle.error import DyngleError
10
10
  from dyngle.expression import expression
11
+ from dyngle.operation import Operation
11
12
 
12
13
 
13
14
  class DyngleApp(WizApp):
@@ -44,15 +45,31 @@ class DyngleApp(WizApp):
44
45
 
45
46
  @cached_property
46
47
  def operations(self):
47
- operations = self._get_configuration_details('operations')
48
- if not operations:
49
- raise DyngleError("No operations defined")
48
+ operations_configs = self._get_configuration_details('operations')
49
+ if not operations_configs:
50
+ raise DyngleError("No operations defined in configuration")
51
+ operations = {}
52
+ for key, config in operations_configs.items():
53
+ if isinstance(config, list):
54
+ operation = Operation({}, config)
55
+ elif isinstance(config, dict):
56
+ local_expr_texts = config.get('expressions') or {}
57
+ local_expressions = _expressions_from_texts(local_expr_texts)
58
+ steps = config.get('steps') or []
59
+ operation = Operation(local_expressions, steps)
60
+ else:
61
+ raise DyngleError(f"Invalid operation configuration for {key}")
62
+ operations[key] = operation
50
63
  return operations
51
64
 
52
65
  @cached_property
53
- def expressions(self):
66
+ def global_expressions(self):
54
67
  expr_texts = self._get_configuration_details('expressions')
55
- if expr_texts:
56
- return {k: expression(t) for k, t in expr_texts.items()}
57
- else:
58
- return {}
68
+ return _expressions_from_texts(expr_texts)
69
+
70
+
71
+ def _expressions_from_texts(expr_texts):
72
+ if expr_texts:
73
+ return {k: expression(t) for k, t in expr_texts.items()}
74
+ else:
75
+ return {}
@@ -34,19 +34,14 @@ class RunCommand(DyngleCommand):
34
34
 
35
35
  @DyngleCommand.wrap
36
36
  def execute(self):
37
- expressions = self.app.expressions
38
- operations = self.app.operations
39
- self._validate_operation_exists(operations)
40
- steps = operations[self.operation]
41
37
  data_string = self.app.stream.text
42
38
  data = safe_load(data_string) or {}
43
39
  data['args'] = self.args
44
- for step_template in steps:
45
- step = Template(step_template).render(data, expressions)
46
- parts = shlex.split(step)
47
- result = subprocess.run(parts)
48
- if result.returncode != 0:
49
- raise DyngleError(
50
- f'Task failed with code {result.returncode}: {step}')
40
+
41
+ operations = self.app.operations
42
+ self._validate_operation_exists(operations)
43
+ operation = operations[self.operation]
44
+
45
+ operation.run(data, self.app.global_expressions)
51
46
 
52
47
  return f'Operation "{self.operation}" completed successfully'
@@ -0,0 +1,60 @@
1
+ from dataclasses import dataclass
2
+ from functools import cached_property
3
+ import re
4
+ import shlex
5
+ import subprocess
6
+
7
+ from dyngle.error import DyngleError
8
+ from dyngle.template import Template
9
+
10
+
11
+ @dataclass
12
+ class Operation:
13
+
14
+ local_expressions: dict
15
+
16
+ steps: list
17
+
18
+ def run(self, data: dict, global_expressions: dict):
19
+ # The data dict is mutable
20
+ steps = self.steps
21
+ expressions = global_expressions | self.local_expressions
22
+ for markup in steps:
23
+ step = Step(markup)
24
+ step.run(data, expressions)
25
+
26
+
27
+ STEP_PATTERN = re.compile(
28
+ r'^\s*(?:([\w.-]+)\s+->\s+)?(.+?)(?:\s+=>\s+([\w.-]+))?\s*$')
29
+
30
+
31
+ def parse_step(markup):
32
+ if match := STEP_PATTERN.match(markup):
33
+ input, command_text, output = match.groups()
34
+ command_template = shlex.split(command_text.strip())
35
+ return input, command_template, output
36
+ else:
37
+ raise DyngleError(f"Invalid step markup {{markup}}")
38
+
39
+
40
+ @dataclass
41
+ class Step:
42
+
43
+ markup: str
44
+
45
+ def __post_init__(self):
46
+ self.input, self.command_template, self.output = \
47
+ parse_step(self.markup)
48
+
49
+ def run(self, data, expressions):
50
+ command = [Template(word).render(data, expressions)
51
+ for word in self.command_template]
52
+ pipes = {}
53
+ if self.output:
54
+ pipes['stdout'] = subprocess.PIPE
55
+ result = subprocess.run(command, text=True, **pipes)
56
+ if result.returncode != 0:
57
+ raise DyngleError(
58
+ f'Step failed with code {result.returncode}: {self.markup}')
59
+ if self.output:
60
+ data[self.output] = result.stdout
@@ -2,6 +2,8 @@ from dataclasses import dataclass
2
2
  from functools import partial
3
3
  import re
4
4
 
5
+ from dyngle.error import DyngleError
6
+
5
7
 
6
8
  PATTERN = re.compile(r'\{\{\s*([^}]+)\s*\}\}')
7
9
 
@@ -44,6 +46,9 @@ class Template:
44
46
  parts = key.split('.')
45
47
  current = live_data
46
48
  for part in parts:
49
+ if part not in current:
50
+ raise DyngleError(
51
+ f"Invalid expression or data reference '{key}'")
47
52
  current = current[part]
48
53
  if callable(current):
49
54
  return current(live_data)
@@ -8,7 +8,7 @@ description = "Run lightweight local workflows"
8
8
  authors = ["Steampunk Wizard <dyngle@steamwiz.io>"]
9
9
  license = "MIT"
10
10
  readme = "PACKAGE.md"
11
- version = "0.5.0"
11
+ version = "0.7.0"
12
12
 
13
13
  [tool.poetry.dependencies]
14
14
  python = "~3.11"
dyngle-0.5.0/PACKAGE.md DELETED
@@ -1,141 +0,0 @@
1
- # Dyngle
2
-
3
- ## Run lightweight local workflows
4
-
5
- Dyngle is a simple workflow runner that executes sequences of commands defined in configuration files. It's like a lightweight combination of Make and a task runner, designed for automating common development and operational tasks.
6
-
7
- ## Basic usage
8
-
9
- Create a configuration file (e.g., `.dyngle.yml`) with your workflows:
10
-
11
- ```yaml
12
- dyngle:
13
- operations:
14
- build:
15
- - python -m pip install -e .
16
- - python -m pytest
17
- deploy:
18
- - docker build -t myapp .
19
- - docker push myapp
20
- clean:
21
- - rm -rf __pycache__
22
- - rm -rf .pytest_cache
23
- ```
24
-
25
- Run an operation:
26
-
27
- ```bash
28
- dyngle run build
29
- ```
30
-
31
- ## Configuration
32
-
33
- Dyngle reads configuration from YAML files. You can specify the config file location using:
34
-
35
- - `--config` command line option
36
- - `DYNGLE_CONFIG` environment variable
37
- - `.dyngle.yml` in current directory
38
- - `~/.dyngle.yml` in home directory
39
-
40
- The configuration has 2 parts: `operations:` and `expressions`.
41
-
42
- ## Data
43
-
44
- Dyngle maintains a block of data throughout operations, which is parsed from YAML in stdin.
45
-
46
- ## Operations
47
-
48
- Operations contain steps as a YAML array. The lifecycle of an operation is:
49
-
50
- 1. Load input data if it exists from YAML on stdin (if no tty)
51
- 2. Perform template rendering on a step, using data and expressions (see below)
52
- 3. Execute the step in a subprocess
53
- 4. Continue with the next step
54
-
55
- Note that operations in the config are _not_ full shell lines. They are passed directly to the system.
56
-
57
- ## Templates
58
-
59
- Prior to running commands, the line containing that command is processed as a template. Entries from the data set can be substituted into the command line using Jinja-like expressions in double-curly brackets (`{{` and `}}`).
60
-
61
- For example, if stdin contains the following data:
62
-
63
- ```yaml
64
- name: Francis
65
- ```
66
-
67
- And the command looks like:
68
-
69
- ``` yaml
70
- - echo "Hello {{name}}!"
71
- ```
72
-
73
- Then the command will output "Hello Francis!".
74
-
75
-
76
- ## Expressions
77
-
78
- Configs can also contain expressions, written in Python, that can also be referenced in operation steps.
79
-
80
- ```yaml
81
- dyngle:
82
- expressions:
83
- say-hello: >-
84
- 'Hello ' + name + '!'
85
- operations:
86
- say-hello: echo {{say-hello}}
87
- ```
88
-
89
- Expressions can use a controlled subset of the Python standard library, including:
90
-
91
- - Built-in data types such as `str()`
92
- - Essential built-in functions such as `len()`
93
- - The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
94
- - A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
95
- - A restricted version of `Path()` that only operates within the current working directory
96
- - Various other useful utilities, mostly read-only, such as the `math` module
97
- - A special function called `resolve` which resolves data expressions using the same logic as in templates
98
-
99
- Data keys containing hyphens are converted to valid Python names by replacing hyphens with underscores.
100
-
101
- Expressions can reference data directly as local names in Python (using the underscore replacements)...
102
-
103
- ```yaml
104
- dyngle:
105
- expressions:
106
- say-hello: >-
107
- 'Hello ' + full_name + '!'
108
- ```
109
-
110
- ... or using the `resolve()` function, which also allows expressions to essentially call other expressions, using the same underlying data set.
111
-
112
- ```yaml
113
- dyngle:
114
- expressions:
115
- hello: >-
116
- 'Hello ' + resolve('formal-name') + '!'
117
- formal-name: >-
118
- 'Ms. ' + full_name
119
- ```
120
-
121
- Note it's also _possible_ to call other expressions by name as functions, if they only return hard-coded values (i.e. constants).
122
-
123
- ```yaml
124
- dyngle:
125
- expressions:
126
- author-name: Francis Potter
127
- author-hello: >-
128
- 'Hello ' + author_name()
129
- ```
130
-
131
- ## Security
132
-
133
- Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. Use with caution.
134
-
135
- ## Quick installation (MacOS)
136
-
137
- ```bash
138
- brew install python@3.11
139
- python3.11 -m pip install pipx
140
- pipx install dyngle
141
- ```
dyngle-0.5.0/PKG-INFO DELETED
@@ -1,157 +0,0 @@
1
- Metadata-Version: 2.3
2
- Name: dyngle
3
- Version: 0.5.0
4
- Summary: Run lightweight local workflows
5
- License: MIT
6
- Author: Steampunk Wizard
7
- Author-email: dyngle@steamwiz.io
8
- Requires-Python: >=3.11,<3.12
9
- Classifier: License :: OSI Approved :: MIT License
10
- Classifier: Programming Language :: Python :: 3
11
- Classifier: Programming Language :: Python :: 3.11
12
- Requires-Dist: requests (>=2.32.3,<3.0.0)
13
- Requires-Dist: wizlib (>=3.1.4,<4.0.0)
14
- Description-Content-Type: text/markdown
15
-
16
- # Dyngle
17
-
18
- ## Run lightweight local workflows
19
-
20
- Dyngle is a simple workflow runner that executes sequences of commands defined in configuration files. It's like a lightweight combination of Make and a task runner, designed for automating common development and operational tasks.
21
-
22
- ## Basic usage
23
-
24
- Create a configuration file (e.g., `.dyngle.yml`) with your workflows:
25
-
26
- ```yaml
27
- dyngle:
28
- operations:
29
- build:
30
- - python -m pip install -e .
31
- - python -m pytest
32
- deploy:
33
- - docker build -t myapp .
34
- - docker push myapp
35
- clean:
36
- - rm -rf __pycache__
37
- - rm -rf .pytest_cache
38
- ```
39
-
40
- Run an operation:
41
-
42
- ```bash
43
- dyngle run build
44
- ```
45
-
46
- ## Configuration
47
-
48
- Dyngle reads configuration from YAML files. You can specify the config file location using:
49
-
50
- - `--config` command line option
51
- - `DYNGLE_CONFIG` environment variable
52
- - `.dyngle.yml` in current directory
53
- - `~/.dyngle.yml` in home directory
54
-
55
- The configuration has 2 parts: `operations:` and `expressions`.
56
-
57
- ## Data
58
-
59
- Dyngle maintains a block of data throughout operations, which is parsed from YAML in stdin.
60
-
61
- ## Operations
62
-
63
- Operations contain steps as a YAML array. The lifecycle of an operation is:
64
-
65
- 1. Load input data if it exists from YAML on stdin (if no tty)
66
- 2. Perform template rendering on a step, using data and expressions (see below)
67
- 3. Execute the step in a subprocess
68
- 4. Continue with the next step
69
-
70
- Note that operations in the config are _not_ full shell lines. They are passed directly to the system.
71
-
72
- ## Templates
73
-
74
- Prior to running commands, the line containing that command is processed as a template. Entries from the data set can be substituted into the command line using Jinja-like expressions in double-curly brackets (`{{` and `}}`).
75
-
76
- For example, if stdin contains the following data:
77
-
78
- ```yaml
79
- name: Francis
80
- ```
81
-
82
- And the command looks like:
83
-
84
- ``` yaml
85
- - echo "Hello {{name}}!"
86
- ```
87
-
88
- Then the command will output "Hello Francis!".
89
-
90
-
91
- ## Expressions
92
-
93
- Configs can also contain expressions, written in Python, that can also be referenced in operation steps.
94
-
95
- ```yaml
96
- dyngle:
97
- expressions:
98
- say-hello: >-
99
- 'Hello ' + name + '!'
100
- operations:
101
- say-hello: echo {{say-hello}}
102
- ```
103
-
104
- Expressions can use a controlled subset of the Python standard library, including:
105
-
106
- - Built-in data types such as `str()`
107
- - Essential built-in functions such as `len()`
108
- - The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
109
- - A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
110
- - A restricted version of `Path()` that only operates within the current working directory
111
- - Various other useful utilities, mostly read-only, such as the `math` module
112
- - A special function called `resolve` which resolves data expressions using the same logic as in templates
113
-
114
- Data keys containing hyphens are converted to valid Python names by replacing hyphens with underscores.
115
-
116
- Expressions can reference data directly as local names in Python (using the underscore replacements)...
117
-
118
- ```yaml
119
- dyngle:
120
- expressions:
121
- say-hello: >-
122
- 'Hello ' + full_name + '!'
123
- ```
124
-
125
- ... or using the `resolve()` function, which also allows expressions to essentially call other expressions, using the same underlying data set.
126
-
127
- ```yaml
128
- dyngle:
129
- expressions:
130
- hello: >-
131
- 'Hello ' + resolve('formal-name') + '!'
132
- formal-name: >-
133
- 'Ms. ' + full_name
134
- ```
135
-
136
- Note it's also _possible_ to call other expressions by name as functions, if they only return hard-coded values (i.e. constants).
137
-
138
- ```yaml
139
- dyngle:
140
- expressions:
141
- author-name: Francis Potter
142
- author-hello: >-
143
- 'Hello ' + author_name()
144
- ```
145
-
146
- ## Security
147
-
148
- Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. Use with caution.
149
-
150
- ## Quick installation (MacOS)
151
-
152
- ```bash
153
- brew install python@3.11
154
- python3.11 -m pip install pipx
155
- pipx install dyngle
156
- ```
157
-
File without changes
File without changes
File without changes
File without changes