pycli-dsl 0.1.0__tar.gz

pycli_dsl-0.1.0/PKG-INFO

@@ -0,0 +1,601 @@
+Metadata-Version: 2.3
+Name: pycli-dsl
+Version: 0.1.0
+Summary: A Python-compatible DevOps DSL transpiler extending Python with first-class shell command execution
+Keywords: devops,dsl,transpiler,shell,bash,cli
+Author: Vito Vessia
+Author-email: Vito Vessia <vito.vessia@wolterskluwer.com>
+License: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Systems Administration
+Classifier: Topic :: Software Development :: Compilers
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/vitovex/pycli
+Project-URL: Repository, https://github.com/vitovex/pycli
+Project-URL: Issues, https://github.com/vitovex/pycli/issues
+Project-URL: Documentation, https://github.com/vitovex/pycli#readme
+Description-Content-Type: text/markdown
+
+# pycli
+
+[![CI Pipeline](https://github.com/vitovex/pycli/actions/workflows/ci.yml/badge.svg)](https://github.com/vitovex/pycli/actions/workflows/ci.yml)
+[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue.svg)](https://www.python.org/)
+[![Platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey.svg)](https://github.com/vitovex/pycli/actions)
+
+A lightweight Python-compatible DevOps DSL that extends Python with first-class shell command execution.
+
+## Overview
+
+`pycli` transpiles `.spy` files into pure standard Python (`.py`), providing:
+- **Python Readability**: Natural Python syntax and standard ecosystem compatibility.
+- **PowerShell-like Command Invocation**: Run shell commands directly with `$(...)`.
+- **Bash-like Command Composition**: Pipelines (`|`), redirections (`>`, `>>`, `<`), and subcommands.
+- **Native Object Handling**: Seamless integration with Python objects, string interpolation `{var}`, and list expansion (`{*files}`).
+- **Cross-Platform Compatibility**: Tested and verified across Linux (`ubuntu-latest`), macOS (`macos-latest`), and Windows (`windows-latest`).
+- **Zero Custom VM**: Transpiled directly to standard Python and executed on standard CPython.
+
+See the full specification in [docs/pycli-grammar.md](docs/pycli-grammar.md).
+
+## Example
+
+### Source DSL (`deploy.spy`)
+
+```python
+target_env = "production"
+branch = $(git branch --show-current).text
+
+# 1. Pipelines (|) and line streaming
+live_pods = $(kubectl get pods -n {target_env} | grep -E 'Running|Pending').lines
+for pod in live_pods:
+    print(f"Active pod: {pod}")
+
+# 2. Context managers: temporary directory (cd) and environment variables (env)
+with cd("frontend"), env(NODE_ENV=target_env):
+    $(npm ci)!
+    build_log = $(npm run build).tee
+
+# 3. List expansion (splat), structured JSON output, and safe probing (?)
+artifacts = ["dist/app.js", "dist/app.css"]
+$(gzip -k {*artifacts})
+
+cluster = $(az aks show --name prod-cluster --resource-group {target_env}).json
+print(f"Cluster FQDN: {cluster.fqdn}")
+
+status = $(curl -sSf http://localhost:8080/health)?
+if status:
+    print("Health check passed successfully!")
+```
+
+### Transpiled Python (`deploy.py`)
+
+```python
+from pycli.runtime import cd, env, run, run_expanded
+
+target_env = "production"
+branch = run("git branch --show-current").text
+
+# 1. Pipelines (|) and line streaming
+live_pods = run(f"""kubectl get pods -n {target_env} | grep -E 'Running|Pending'""").lines
+for pod in live_pods:
+    print(f"Active pod: {pod}")
+
+# 2. Context managers: temporary directory (cd) and environment variables (env)
+with cd("frontend"), env(NODE_ENV=target_env):
+    run("npm ci", capture=False, check=True)
+    build_log = run("npm run build", tee=True)
+
+# 3. List expansion (splat), structured JSON output, and safe probing (?)
+artifacts = ["dist/app.js", "dist/app.css"]
+run_expanded("gzip", "-k", *artifacts, capture=False)
+
+cluster = run(f"az aks show --name prod-cluster --resource-group {target_env}").json
+print(f"Cluster FQDN: {cluster.fqdn}")
+
+status = run("curl -sSf http://localhost:8080/health", suppress_errors=True)
+if status:
+    print("Health check passed successfully!")
+```
+
+## CLI Usage
+
+### Global Installation (Centralized Command)
+
+You can install `pycli` globally into your system `PATH` using `uv tool`:
+
+```powershell
+uv tool install --editable . --force
+```
+
+This registers two commands globally on your machine:
+- **`spy`**: Ultra-concise runner for `.spy` scripts.
+- **`pycli`**: The full CLI tool with subcommands.
+
+Once installed, you can run `.spy` scripts from **any folder or terminal**:
+```powershell
+spy script.spy
+# or
+pycli script.spy
+```
+
+### Direct Script Execution without Global Install
+
+If working inside this repository with `uv`:
+```powershell
+uv run pycli script.spy
+```
+
+### Transpile `.spy` to `.py`
+
+```powershell
+# Output to stdout with syntax coloring
+spy transpile script.spy
+
+# Output to a file (clean Python without ANSI codes)
+spy transpile script.spy -o script.py
+
+# Force / disable color
+spy transpile script.spy --color
+spy transpile script.spy --no-color
+
+# Validate generated Python code with ast.parse
+spy transpile script.spy --validate
+
+# Disable automatic shell_quote() sanitization on interpolations
+spy transpile script.spy --unsafe-interpolation
+```
+
+### Run `.spy` Scripts
+
+```powershell
+# Run a script directly
+spy run script.spy
+# or simply
+spy script.spy
+
+# Run with generated Python syntax validation
+spy run --validate script.spy
+
+# Run with unsafe interpolation (disables shell_quote)
+spy run --unsafe-interpolation script.spy
+
+# Run with warning on untrusted external scripts
+spy run --warn-external script.spy
+```
+
+---
+
+## Security & Robustness
+
+### Automatic Shell Interpolation Sanitization
+By default, all variable interpolations `{var}` and dynamic redirection targets are wrapped with `shell_quote(var)` (`shlex.quote`) during transpilation. This protects against shell injection attacks if variables contain metacharacters (`;`, `&&`, `|`, etc.).
+If raw, unquoted shell syntax expansion is explicitly needed, pass `--unsafe-interpolation` or use `transpile(..., unsafe_interpolation=True)`.
+
+### Execution Privilege Model (Not a Sandbox)
+`pycli` executes `.spy` files on standard CPython runtimes with the full privileges and environment of the user running the process. It is **not** a sandbox. When executing `.spy` scripts from external or untrusted sources, use `--warn-external` and verify the script contents.
+
+### Command Execution Controls
+The runtime functions `run()`, `run_expanded()`, and `async_run()` support robust controls:
+- **`timeout`**: Terminate hanging processes and raise `CommandTimeoutError`.
+- **`encoding`**: Custom text decoding (default `"utf-8"`, configurable to CP1252, Latin-1, etc.).
+- **`max_output_bytes`**: Cap memory consumption by truncating stdout/stderr beyond a threshold (`res.truncated = True`).
+
+
+
+
+## Supported Language Features
+
+| Feature | Syntax Example | Target Python Equivalent |
+|---|---|---|
+| **Statement Form** | `$(git status)` | `run("git status", capture=False)` (streams output to console) |
+| **Expression Form** | `res = $(git status)` | `res = run("git status")` (captures stdout/stderr) |
+| **Strict Mode** | `$(git status)!` | `run("git status", capture=False, check=True)` |
+| **Safe Mode** | `$(curl http://...) ?` | `run(..., suppress_errors=True)` (no exception on non-zero exit code) |
+| **Background / Async** | `job = $(docker build .) &` | `run_bg(...)` returning `BackgroundJob` (`.wait()`, `.poll()`, `.kill()`) |
+| **Async / Await** | `res = await $(git pull)` | `await async_run(...)` for `asyncio` workflows |
+| **Live Stream & Capture** | `res = $(npm test).tee` | `run(..., tee=True)` (live streaming to console + captured in `res`) |
+| **Stdin Piping** | `$(kubectl apply -f -).input(yaml)` | `run(..., input=yaml)` |
+| **Output Line Iteration** | `for line in $(git log): ...` | Direct iteration over `res`, or `res.lines` and `res.text` |
+| **Context Managers** | `with cd(dir):`, `with env(K="V"):` | Temporary directory and environment variable scoping |
+| **Quote Interpolation** | `$(echo "{name}" '{raw}')` | Double quotes interpolate `{expr}`; single quotes stay strictly literal |
+| **List Expansion (Splat)** | `$(rm {*files})` | `run_expanded("rm", *files, capture=False)` |
+| **Pipelines** | `$(kubectl get pods \| grep api)` | `run("kubectl get pods \| grep api", capture=False)` |
+| **Redirection** | `$(git status > status.txt)` | `run("git status > status.txt", capture=False)` |
+| **Subcommands** | `$(echo $(git branch --show-current))` | `run("echo $(git branch --show-current)", capture=False)` |
+| **Truthiness** | `if $(git diff --quiet): ...` | `if run("git diff --quiet"): ...` (truthy if `exit_code == 0`) |
+| **Structured Output (JSON)** | `vms = $(az vm list).json` | Navigable `DynamicObj` via `vm.name` or `vm["name"]` |
+| **Interactive REPL** | `spy repl` or `spy` | Interactive shell with on-the-fly transpilation |
+| **CommandResult Properties** | `res = $(git status)` | `res.stdout`, `res.stderr`, `res.exit_code`, `res.lines`, `res.text` |
+| **Modular .spy Imports** | `import devops_utils` | Seamlessly import `.spy` files and packages via Python `importlib` hook |
+
+---
+
+## Complete Language Syntax Reference
+
+`spy` is a superset of standard Python. Everything that is valid in Python 3.12+ is fully valid in `.spy`. `spy` introduces the **Command Expression** `$(...)` for seamless command-line execution and shell orchestration.
+
+### 1. Command Invocation Forms
+
+#### Statement Form (Unassigned)
+When a command expression appears as a standalone line or single-line statement:
+```python
+$(terraform init)
+if should_apply: $(terraform apply -auto-approve)
+```
+- **Execution**: The command is executed and its output (`stdout` and `stderr`) is streamed live to the console in real-time.
+- **Return value**: Discarded (`capture=False`).
+
+#### Expression Form (Assigned / Inline)
+When a command expression is assigned to a variable, passed as a function argument, or used in an expression:
+```python
+current_branch = $(git branch --show-current)
+log_output = $(git log -n 10).text
+```
+- **Execution**: The output is captured silently and returned as a `CommandResult` instance.
+
+---
+
+### 2. Execution Modifiers
+
+Modifiers are placed immediately after the closing parenthesis `)` of a command:
+
+| Modifier | Syntax | Behavior | Python Equivalent |
+|---|---|---|---|
+| **Strict (`!`)** | `$(cmd)!` | Raises `CommandError` if exit code != 0 | `run(..., check=True)` |
+| **Safe (`?`)** | `$(cmd)?` | Suppresses errors; never raises exceptions on failure | `run(..., suppress_errors=True)` |
+| **Background (`&`)** | `job = $(cmd) &` | Spawns in background non-blockingly; returns `BackgroundJob` | `run_bg(...)` |
+
+#### Examples:
+```python
+# 1. Strict mode: abort pipeline if build fails
+try:
+    $(docker build -t app:latest .)!
+except Exception as err:
+    print(f"Build failed with exit code: {err.result.exit_code}")
+
+# 2. Safe mode: probe an endpoint or optional service without try/except
+probe = $(curl -sSf http://localhost:8080/health)?
+if probe:
+    print("Service is healthy!")
+else:
+    print(f"Service offline (exit code: {probe.exit_code})")
+
+# 3. Background jobs: run long-running tasks concurrently (&)
+job = $(mvn clean package) &
+print("Maven build started in background...")
+result = job.wait()
+print(f"Build finished with code: {result.exit_code}")
+
+# 3.1 Waiting for multiple parallel background jobs with wait_all(...)
+j_api = $(deploy-service api) &
+j_web = $(deploy-service web) &
+j_db  = $(deploy-service db) &
+
+# Accepts variable arguments wait_all(j1, j2, ...) or a list wait_all([j1, j2, ...]):
+all_results = wait_all(j_api, j_web, j_db)
+for res in all_results:
+    print(f"Completed: {res.command} (exit code: {res.exit_code})")
+
+# 4. Async / Await inside coroutines
+async def pull_repo(name: str):
+    res = await $(git -C {name} pull origin main)
+    return res.stdout
+
+# 4.1 Running multiple async commands in parallel with asyncio.gather
+async def update_all_microservices():
+    repos = ["frontend", "backend", "worker"]
+    # All pull commands execute concurrently:
+    results = await asyncio.gather(*(pull_repo(r) for r in repos))
+    print(f"Updated {len(results)} repositories simultaneously.")
+```
+
+---
+
+### 3. Chaining Methods & Modifiers
+
+You can chain properties and helper methods directly onto command expressions:
+
+#### `.json` — Structured JSON Output
+Automatically parses JSON standard output into a navigable `DynamicObj`:
+```python
+pods = $(kubectl get pods -o json).json
+for item in pods.items:
+    print(f"Pod: {item.metadata.name} | Status: {item.status.phase}")
+    # Supports both dot access and dict access:
+    print(f"Namespace: {item['metadata']['namespace']}")
+```
+
+#### `.tee` — Live Console Streaming + Output Capture
+Streams stdout/stderr in real-time to the console while simultaneously capturing the complete result in the variable:
+```python
+# Output is displayed immediately on screen AND stored in 'test_run'
+test_run = $(pytest tests/ -v).tee
+if test_run.exit_code != 0:
+    print("Failed test log:", test_run.stderr)
+```
+
+#### `.input(...)` — Feeding Stdin Data
+Pipes string or binary data directly into the standard input of the subprocess:
+```python
+manifest = """
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: app-config
+"""
+$(kubectl apply -f -).input(manifest)
+```
+
+#### `.lines` & `.text`
+- `.lines`: Returns a `list[str]` of non-empty lines from `stdout` (stripped of trailing newlines).
+- `.text`: Returns the trimmed `stdout` string (`stdout.strip()`).
+```python
+branches = $(git branch --list).lines
+first_line = $(head -n 1 file.txt).text
+```
+
+---
+
+### 4. Interpolation & Quoting Semantics
+
+`spy` provides precise rules for parameter interpolation to keep shell scripts intuitive:
+
+#### Unquoted Variable / Expression Interpolation
+Any Python expression enclosed in `{...}` is evaluated and inserted into the command string:
+```python
+target_cluster = "prod-us-east-1"
+$(kubectl config use-context {target_cluster})
+$(az vm list --resource-group {config.resource_group})
+```
+
+#### Double Quotes (`"..."`) — Interpolation Enabled
+Double-quoted command strings expand `{expression}`:
+```python
+name = "World"
+$(echo "Hello, {name}!")
+# Evaluates to: echo "Hello, World!"
+```
+
+#### Single Quotes (`'...'`) — Strictly Literal
+Single-quoted command strings preserve braces literally. No interpolation occurs inside single quotes. This is critical for shell tools like `awk`, regex patterns, or inline sub-scripts:
+```python
+# Braces remain literal {print $1}:
+$(awk '{print $1}' access.log)
+
+# Regex stays literal:
+$(grep -E '^[0-9]{4}-[0-9]{2}' server.log)
+```
+
+#### List Expansion / Splat (`{*iterable}`)
+Expands a Python list, tuple, or iterable into space-separated command-line arguments:
+```python
+files = ["service.py", "models.py", "utils.py"]
+$(ruff check {*files})
+# Transpiles to: run_expanded("ruff", "check", *files, capture=False)
+```
+
+---
+
+### 5. Preserved Shell Semantics
+
+`spy` passes command strings to the underlying shell without interfering with native shell operators.
+
+#### Pipelines (`|`)
+Connect the standard output of one command directly to the standard input of the next:
+```python
+# 1. Pipeline in statement form (streaming output directly to terminal)
+$(kubectl get pods -n prod | grep -v Completed | sort)
+
+# 2. Pipeline in expression form (captured and iterated)
+failed_jobs = $(docker ps -a | grep "Exited (" | awk '{print $1}').lines
+for container_id in failed_jobs:
+    print(f"Removing dead container: {container_id}")
+    $(docker rm {container_id})
+
+# 3. Chaining with Python processing
+build_errors = $(cargo check 2>&1 | grep "error\[E").lines
+if build_errors:
+    print(f"Found {len(build_errors)} compile errors:")
+    for err in build_errors:
+        print("  -", err)
+```
+
+#### Redirections (`>`, `>>`, `<`)
+Direct process outputs or inputs to and from filesystem files:
+```python
+# Overwrite file with stdout (>)
+$(terraform output -json > tf_outputs.json)
+
+# Append to log file (>>)
+$(date >> deployment.log)
+$(echo "Deployed by {user} on {branch}" >> deployment.log)
+
+# Read input from file (<)
+$(mysql -u root -p{db_pass} my_database < migration.sql)!
+```
+
+#### Subcommands (`$(...)`)
+Inner shell command substitutions are handled directly by the shell runtime:
+```python
+# Create timestamped tarball using subshell date command:
+$(tar -czf backup-$(date +%Y%m%d).tar.gz /var/data)
+
+# Create git release tag from file content:
+$(git tag release-$(cat VERSION))
+```
+
+---
+
+### 6. The `CommandResult` Object
+
+Captured command expressions return a `CommandResult` instance with rich inspection capabilities:
+
+| Attribute / Method | Type | Description |
+|---|---|---|
+| `res.stdout` | `str` | Full standard output |
+| `res.stderr` | `str` | Full standard error |
+| `res.exit_code` | `int` | Process exit status code (`0` = success) |
+| `res.duration` | `float` | Command execution time in seconds |
+| `res.command` | `str` | Exact command string executed |
+| `res.lines` | `list[str]` | List of non-empty stdout lines |
+| `res.text` | `str` | Trimmed standard output (`stdout.strip()`) |
+| `res.json` | `DynamicObj` | Parsed JSON object / list |
+| `for line in res:` | `Iterator[str]` | Iterate directly over lines in stdout |
+| `res[index]` | `str` | Access a specific line by index |
+| `bool(res)` | `bool` | **Truthiness**: `True` if `exit_code == 0`, else `False` |
+
+#### Truthiness Example:
+```python
+if $(git diff --quiet):
+    print("Working tree clean")
+else:
+    print("Uncommitted changes detected")
+```
+
+---
+
+### 7. Built-in Context Managers
+
+Every `.spy` script and module automatically has access to `cd()` and `env()` as first-class primitives without requiring any manual `import` statement.
+
+#### `with cd(path)` — Directory Navigation
+Changes current working directory for the duration of the `with` block and **guarantees** restoration to the previous directory upon exiting, even if an exception occurs:
+```python
+# 1. Work in a specific subproject directory
+with cd("services/billing"):
+    $(cargo build --release)!
+    $(cargo test)
+
+# 2. Nested directory navigation
+with cd("packages"):
+    with cd("frontend"):
+        $(npm test)
+    # Automatically back in "packages"
+# Automatically back in the root directory
+```
+
+#### `with env(**kwargs)` — Temporary Environment Variables
+Sets or overrides environment variables for the duration of the `with` block and safely restores the original environment afterwards:
+```python
+with env(AWS_DEFAULT_REGION="eu-west-1", STAGE="staging"):
+    $(aws s3 ls)
+    $(serverless deploy)
+# AWS_DEFAULT_REGION and STAGE are restored to their original values
+```
+
+#### Combining `cd()` and `env()`
+You can combine multiple context managers cleanly on a single line:
+```python
+with cd("apps/backend"), env(DATABASE_URL="postgres://test:5432/db", LOG_LEVEL="DEBUG"):
+    $(alembic upgrade head)!
+    $(pytest -v)
+```
+
+---
+
+### 8. Modular Architecture (`.spy` Imports)
+
+You can structure large DevOps and infrastructure projects into modular files. `.spy` scripts can import other `.spy` scripts or packages natively:
+
+```python
+# main.spy
+import devops_utils
+from infrastructure.cloud import deploy_cluster
+
+status = devops_utils.get_git_status()
+deploy_cluster("production")
+```
+
+The underlying import hook compiles `.spy` files into standard Python bytecode on the fly with zero disk pollution.
+
+---
+
+### 9. Interactive REPL
+
+`spy` includes a dedicated interactive read-eval-print loop with instant transpilation:
+
+```powershell
+# Launch interactive shell
+spy repl
+# or simply
+spy
+```
+
+```text
+>>> branch = $(git branch --show-current).text
+>>> branch
+'main'
+>>> for file in $(git ls-files):
+...     if file.endswith(".spy"):
+...         print("Found spy script:", file)
+... 
+```
+
+---
+
+## Examples
+
+The repository includes runnable `.spy` examples in the `examples/` directory:
+
+- [examples/demo.spy](examples/demo.spy): Basic overview demonstrating variable interpolation, list expansion, and status checking.
+- [examples/syntax_reference.spy](examples/syntax_reference.spy): Comprehensive, executable reference covering every syntax construct and execution mode.
+- [examples/advanced_features.spy](examples/advanced_features.spy): Practical demonstration of the 8 advanced productivity features (streaming, `.tee`, `.input(...)`, `cd()`/`env()`, background jobs `&`, safe mode `?`, quote semantics).
+- [examples/parallel_async_jobs.spy](examples/parallel_async_jobs.spy): Concurrent process orchestration showing how to launch multiple background jobs with `&`, await them all with `wait_all(...)`, and coordinate async coroutines with `asyncio.gather(...)`.
+- [examples/complex_devops.spy](examples/complex_devops.spy): Advanced pipeline orchestrator demonstrating Python dataclasses, object-oriented design, dynamic JSON parsing, strict mode error handling (`try/except CommandError`), and splat expansion.
+- [examples/modular_demo.spy](examples/modular_demo.spy) & [examples/devops_utils.spy](examples/devops_utils.spy): Modular multi-file architecture demonstrating how a `.spy` file can seamlessly import reusable functions, classes, and shell workflows from another `.spy` file or package.
+
+Run them directly with `spy`:
+```powershell
+spy examples/demo.spy
+spy examples/syntax_reference.spy
+spy examples/advanced_features.spy
+spy examples/parallel_async_jobs.spy
+spy examples/complex_devops.spy
+spy examples/modular_demo.spy
+```
+
+
+## Getting Started
+
+### Prerequisites
+
+- Python `>= 3.12`
+- `uv` package manager
+- Supported Platforms: Linux, macOS, and Windows (all tested in CI)
+
+### Development Setup
+
+```powershell
+# Install dependencies
+uv sync
+
+# Run tests
+uv run pytest
+
+# Run CLI help
+uv run pycli --help
+```
+
+## Editor Support
+
+Language extensions and syntax highlighting definitions are provided in the `editors/` directory:
+
+- **Visual Studio Code & Antigravity IDE** ([editors/vscode](editors/vscode)): Full Python + embedded shell grammar, command delimiter highlighting, interpolation scoping, and snippets (`cmd`, `cmdvar`, `cmdjson`, `cmdstrict`, `cmdsplat`, `cmdif`).
+  - To install in VSCode:
+    ```powershell
+    Copy-Item -Recurse -Force "editors/vscode" "$env:USERPROFILE\.vscode\extensions\pycli-vscode"
+    ```
+  - To install in Antigravity IDE:
+    ```powershell
+    Copy-Item -Recurse -Force "editors/vscode" "$env:USERPROFILE\.antigravity-ide\extensions\pycli-vscode"
+    ```
+- **Notepad++** ([editors/notepadplusplus](editors/notepadplusplus)): User Defined Language (UDL) definition for `.spy` files.
+  - To install locally:
+    ```powershell
+    Copy-Item -Force "editors/notepadplusplus/pycli.xml" "$env:APPDATA\Notepad++\userDefineLangs\"
+    ```
+