dyngle 0.4.1__tar.gz → 0.4.3__tar.gz

dyngle-0.4.3/PACKAGE.md

@@ -0,0 +1,141 @@
+# Dyngle
+
+## Run lightweight local workflows
+
+Dyngle is a simple workflow runner that executes sequences of commands defined in configuration files. It's like a lightweight combination of Make and a task runner, designed for automating common development and operational tasks.
+
+## Basic usage
+
+Create a configuration file (e.g., `.dyngle.yml`) with your workflows:
+
+```yaml
+dyngle:
+  operations:
+    build:
+      - python -m pip install -e .
+      - python -m pytest
+    deploy:
+      - docker build -t myapp .
+      - docker push myapp
+    clean:
+      - rm -rf __pycache__
+      - rm -rf .pytest_cache
+```
+
+Run an operation:
+
+```bash
+dyngle run build
+```
+
+## Configuration
+
+Dyngle reads configuration from YAML files. You can specify the config file location using:
+
+- `--config` command line option
+- `DYNGLE_CONFIG` environment variable  
+- `.dyngle.yml` in current directory
+- `~/.dyngle.yml` in home directory
+
+The configuration has 2 parts: `operations:` and `expressions`.
+
+## Data
+
+Dyngle maintains a block of data throughout operations, which is parsed from YAML in stdin.
+
+## Operations
+
+Operations contain steps as a YAML array. The lifecycle of an operation is:
+
+1. Load input data if it exists from YAML on stdin (if no tty)
+2. Perform template rendering on a step, using data and expressions (see below)
+3. Execute the step in a subprocess
+4. Continue with the next step
+
+Note that operations in the config are _not_ full shell lines. They are passed directly to the system.
+
+## Templates
+
+Prior to running commands, the line containing that command is processed as a template. Entries from the data set can be substituted into the command line using Jinja-like expressions in double-curly brackets (`{{` and `}}`).
+
+For example, if stdin contains the following data:
+
+```yaml
+name: Francis
+```
+
+And the command looks like:
+
+``` yaml
+- echo "Hello {{name}}!"
+```
+
+Then the command will output "Hello Francis!".
+
+
+## Expressions
+
+Configs can also contain expressions, written in Python, that can also be referenced in operation steps.
+
+```yaml
+dyngle:
+  expressions:
+    say-hello: >-
+        'Hello ' + name + '!'
+  operations:
+    say-hello: echo {{say-hello}}
+```
+
+Expressions can use a controlled subset of the Python standard library, including:
+
+- Built-in data types such as `str()`
+- Essential built-in functions such as `len()`
+- The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
+- A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
+- A restricted version of `Path()` that only operates within the current working directory
+- Various other useful utilities, mostly read-only, such as the `math` module
+- A special function called `resolve` which resolves data expressions using the same logic as in templates
+
+Data keys containing hyphens are converted to valid Python names by replacing hyphens with underscores.
+
+Expressions can reference data directly as local names in Python (using the underscore replacements)...
+
+```yaml
+dyngle:
+  expressions:
+    say-hello: >-
+        'Hello ' + full_name + '!'
+```
+
+... or using the `resolve()` function, which also allows expressions to essentially call other expressions, using the same underlying data set.
+
+```yaml
+dyngle:
+  expressions:
+    hello: >-
+        'Hello ' + resolve('formal-name') + '!'
+    formal-name: >-
+        'Ms. ' + full_name
+```
+
+Note it's also _possible_ to call other expressions by name as functions, if they only return hard-coded values (i.e. constants).
+
+```yaml
+dyngle:
+  expressions:
+    author-name: Francis Potter
+    author-hello: >-
+        'Hello ' + author_name()
+``` 
+
+## Security
+
+Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. Use with caution.
+
+## Quick installation (MacOS)
+
+```bash
+brew install python@3.11
+python3.11 -m pip install pipx
+pipx install dyngle
+```

{dyngle-0.4.1 → dyngle-0.4.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: dyngle
-Version: 0.4.1
+Version: 0.4.3
 Summary: Run lightweight local workflows
 License: MIT
 Author: Steampunk Wizard
@@ -52,30 +52,45 @@ Dyngle reads configuration from YAML files. You can specify the config file loca
 - `.dyngle.yml` in current directory
 - `~/.dyngle.yml` in home directory
 
-## Workflow structure
+The configuration has 2 parts: `operations:` and `expressions`.
 
-Each operation is defined as a list of tasks under `dyngle.operations`. Tasks are executed sequentially using Python's subprocess module for security.
+## Data
 
-Example with multiple operations:
+Dyngle maintains a block of data throughout operations, which is parsed from YAML in stdin.
+
+## Operations
+
+Operations contain steps as a YAML array. The lifecycle of an operation is:
+
+1. Load input data if it exists from YAML on stdin (if no tty)
+2. Perform template rendering on a step, using data and expressions (see below)
+3. Execute the step in a subprocess
+4. Continue with the next step
+
+Note that operations in the config are _not_ full shell lines. They are passed directly to the system.
+
+## Templates
+
+Prior to running commands, the line containing that command is processed as a template. Entries from the data set can be substituted into the command line using Jinja-like expressions in double-curly brackets (`{{` and `}}`).
+
+For example, if stdin contains the following data:
 
 ```yaml
-dyngle:
-  operations:
-    test:
-      - python -m unittest discover
-      - python -m coverage report
-    docs:
-      - sphinx-build docs docs/_build
-      - open docs/_build/index.html
-    setup:
-      - python -m venv venv
-      - source venv/bin/activate
-      - pip install -r requirements.txt
+name: Francis
 ```
 
+And the command looks like:
+
+``` yaml
+- echo "Hello {{name}}!"
+```
+
+Then the command will output "Hello Francis!".
+
+
 ## Expressions
 
-Configs can also contain expressions.
+Configs can also contain expressions, written in Python, that can also be referenced in operation steps.
 
 ```yaml
 dyngle:
@@ -85,6 +100,7 @@ dyngle:
   operations:
     say-hello: echo {{say-hello}}
 ```
+
 Expressions can use a controlled subset of the Python standard library, including:
 
 - Built-in data types such as `str()`
@@ -92,10 +108,44 @@ Expressions can use a controlled subset of the Python standard library, includin
 - The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
 - A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
 - A restricted version of `Path()` that only operates within the current working directory
+- Various other useful utilities, mostly read-only, such as the `math` module
+- A special function called `resolve` which resolves data expressions using the same logic as in templates
+
+Data keys containing hyphens are converted to valid Python names by replacing hyphens with underscores.
+
+Expressions can reference data directly as local names in Python (using the underscore replacements)...
+
+```yaml
+dyngle:
+  expressions:
+    say-hello: >-
+        'Hello ' + full_name + '!'
+```
+
+... or using the `resolve()` function, which also allows expressions to essentially call other expressions, using the same underlying data set.
+
+```yaml
+dyngle:
+  expressions:
+    hello: >-
+        'Hello ' + resolve('formal-name') + '!'
+    formal-name: >-
+        'Ms. ' + full_name
+```
+
+Note it's also _possible_ to call other expressions by name as functions, if they only return hard-coded values (i.e. constants).
+
+```yaml
+dyngle:
+  expressions:
+    author-name: Francis Potter
+    author-hello: >-
+        'Hello ' + author_name()
+``` 
 
 ## Security
 
-Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. 
+Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. Use with caution.
 
 ## Quick installation (MacOS)
 

{dyngle-0.4.1 → dyngle-0.4.3}/dyngle/command/run_command.py

@@ -18,6 +18,8 @@ class RunCommand(DyngleCommand):
     def add_args(cls, parser: WizParser):
         super().add_args(parser)
         parser.add_argument('operation', help='Operation name to run')
+        parser.add_argument(
+            'args', nargs='*', help='Optional operation arguments')
 
     def handle_vals(self):
         super().handle_vals()
@@ -37,7 +39,8 @@ class RunCommand(DyngleCommand):
         self._validate_operation_exists(operations)
         steps = operations[self.operation]
         data_string = self.app.stream.text
-        data = safe_load(data_string)
+        data = safe_load(data_string) or {}
+        data['args'] = self.args
         for step_template in steps:
             step = Template(step_template).render(data, expressions)
             parts = shlex.split(step)

{dyngle-0.4.1 → dyngle-0.4.3}/dyngle/expression.py

@@ -9,6 +9,8 @@ import json
 import re
 import yaml
 
+from dyngle.template import Template
+
 
 def formatted_datetime(dt: datetime, format_string=None) -> str:
     """Safe datetime formatting using string operations"""
@@ -72,9 +74,38 @@ GLOBALS = {
 }
 
 
-def _evaluate(expression: str, data: dict) -> str:
+def _evaluate(expression: str, locals: dict) -> str:
+    """Evaluate a Python expression with safe globals and user data context.
+
+    Safely evaluates a Python expression string using a restricted set of
+    global functions and modules, combined with user-provided data. The
+    expression is evaluated in a sandboxed environment that includes basic
+    Python built-ins, mathematical operations, date/time handling, and data
+    manipulation utilities.
+
+    Parameters
+    ----------
+    expression : str
+        A valid Python expression string to be evaluated.
+    data : dict
+        Dictionary containing variables and values to be made available during
+        expression evaluation. Note that hyphens in keys will be replaced by
+        underscores to create valid Python names.
+
+    Returns
+    -------
+    str
+        String representation of the evaluated expression result. If the result
+        is a tuple, returns the string representation of the last element.
+
+    Raises
+    ------
+    DyngleError
+        If the expression contains invalid variable names that are not found in
+        the provided data dictionary or global context.
+    """
     try:
-        result = eval(expression, GLOBALS, data)
+        result = eval(expression, GLOBALS, locals)
     except KeyError:
         raise DyngleError(f"The following expression contains " +
                           f"at least one invalid name: {expression}")
@@ -83,6 +114,9 @@ def _evaluate(expression: str, data: dict) -> str:
 
 
 def expression(text: str) -> Callable[[dict], str]:
-    def evaluate(data: dict) -> str:
-        return _evaluate(text, data)
+    def evaluate(data: dict = None) -> str:
+        items = data.items() if data else ()
+        locals = {k.replace('-', '_'): v for k, v in items}
+        def resolve(s): return Template.resolve(s, data)
+        return _evaluate(text, locals | {'resolve': resolve})
     return evaluate

dyngle-0.4.3/dyngle/template.py

@@ -0,0 +1,51 @@
+from dataclasses import dataclass
+from functools import partial
+import re
+
+
+PATTERN = re.compile(r'\{\{\s*([^}]+)\s*\}\}')
+
+
+@dataclass
+class Template:
+
+    template: str
+
+    def render(self, data: dict = None, expressions: dict = None) -> str:
+        """Render the template with the provided data and expressions.
+
+        Parameters
+        ----------
+        data : dict
+            String data to insert
+        expressions : dict
+            Functions to call with data
+
+        Returns
+        -------
+        str
+            Template rendered with expression resolution and values inserted.
+        """
+
+        data = data if data else {}
+        expressions = expressions if expressions else {}
+        resolver = partial(self._resolve, live_data=data | expressions)
+        return PATTERN.sub(resolver, self.template)
+
+    def _resolve(self, match, *, live_data: dict):
+        """Resolve a single name/path from the template. The argument is a
+        merge of the raw data and the expressions, either of which are valid
+        substitutions."""
+        key = match.group(1).strip()
+        return self.resolve(key, live_data)
+
+    @staticmethod
+    def resolve(key: str, live_data: dict):
+        parts = key.split('.')
+        current = live_data
+        for part in parts:
+            current = current[part]
+        if callable(current):
+            return current(live_data)
+        else:
+            return current

{dyngle-0.4.1 → dyngle-0.4.3}/pyproject.toml

@@ -8,7 +8,7 @@ description = "Run lightweight local workflows"
 authors = ["Steampunk Wizard <dyngle@steamwiz.io>"]
 license = "MIT"
 readme = "PACKAGE.md"
-version = "0.4.1"
+version = "0.4.3"
 
 [tool.poetry.dependencies]
 python = "~3.11"

dyngle-0.4.1/PACKAGE.md

@@ -1,91 +0,0 @@
-# Dyngle
-
-## Run lightweight local workflows
-
-Dyngle is a simple workflow runner that executes sequences of commands defined in configuration files. It's like a lightweight combination of Make and a task runner, designed for automating common development and operational tasks.
-
-## Basic usage
-
-Create a configuration file (e.g., `.dyngle.yml`) with your workflows:
-
-```yaml
-dyngle:
-  operations:
-    build:
-      - python -m pip install -e .
-      - python -m pytest
-    deploy:
-      - docker build -t myapp .
-      - docker push myapp
-    clean:
-      - rm -rf __pycache__
-      - rm -rf .pytest_cache
-```
-
-Run an operation:
-
-```bash
-dyngle run build
-```
-
-## Configuration
-
-Dyngle reads configuration from YAML files. You can specify the config file location using:
-
-- `--config` command line option
-- `DYNGLE_CONFIG` environment variable  
-- `.dyngle.yml` in current directory
-- `~/.dyngle.yml` in home directory
-
-## Workflow structure
-
-Each operation is defined as a list of tasks under `dyngle.operations`. Tasks are executed sequentially using Python's subprocess module for security.
-
-Example with multiple operations:
-
-```yaml
-dyngle:
-  operations:
-    test:
-      - python -m unittest discover
-      - python -m coverage report
-    docs:
-      - sphinx-build docs docs/_build
-      - open docs/_build/index.html
-    setup:
-      - python -m venv venv
-      - source venv/bin/activate
-      - pip install -r requirements.txt
-```
-
-## Expressions
-
-Configs can also contain expressions.
-
-```yaml
-dyngle:
-  expressions:
-    say-hello: >-
-        'Hello ' + name + '!'
-  operations:
-    say-hello: echo {{say-hello}}
-```
-Expressions can use a controlled subset of the Python standard library, including:
-
-- Built-in data types such as `str()`
-- Essential built-in functions such as `len()`
-- The core modules from the `datetime` package (but some methods such as `strftime()` will fail)
-- A specialized function called `formatted()` to perform string formatting operations on a `datetime` object
-- A restricted version of `Path()` that only operates within the current working directory
-
-## Security
-
-Commands are executed using Python's `subprocess.run()` with arguments split in a shell-like fashion. The shell is not used, which reduces the likelihood of shell injection attacks. However, note that Dyngle is not robust to malicious configuration. 
-
-## Quick installation (MacOS)
-
-```bash
-brew install python@3.11
-python3.11 -m pip install pipx
-pipx install dyngle
-```

dyngle-0.4.1/dyngle/template.py

@@ -1,31 +0,0 @@
-from dataclasses import dataclass
-from functools import partial
-import re
-
-
-PATTERN = re.compile(r'\{\{\s*([^}]+)\s*\}\}')
-
-
-@dataclass
-class Template:
-
-    template: str
-
-    def render(self, data: dict = None, expressions: dict = None) -> str:
-        """Render the template with the provided data."""
-        data = data if data else {}
-        expressions = expressions if expressions else {}
-        resolver = partial(self._resolve, data=data, expressions=expressions)
-        return PATTERN.sub(resolver, self.template)
-
-    def _resolve(self, match, *, data: dict, expressions: dict):
-        """Resolve a single name/path from the template."""
-        key = match.group(1).strip()
-        if key in expressions:
-            return expressions[key](data)
-        else:
-            parts = key.split('.')
-            current = data
-            for part in parts:
-                current = current[part]
-            return current