etlplus 0.4.5__tar.gz → 0.6.1__tar.gz

{etlplus-0.4.5 → etlplus-0.6.1}/.pre-commit-config.yaml

@@ -159,7 +159,11 @@ repos:
     rev: v1.19.0
     hooks:
       - id: mypy
-        args: [--ignore-missing-imports, --install-types, --non-interactive]
+        args:
+          - --cache-dir=.mypy_cache/pre-commit
+          - --ignore-missing-imports
+          - --install-types
+          - --non-interactive
 
   - repo: https://github.com/pycqa/flake8
     rev: 7.3.0

etlplus-0.6.1/MANIFEST.in

@@ -0,0 +1,12 @@
+# MANIFEST.in
+# ETLPlus
+#
+# Copyright © 2026 Dagitali LLC. All rights reserved.
+#
+# Contains commands that allow lists of files to be discovered and manipulated.
+#
+# See:
+# 1. https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html
+
+# Include Jinja template files in the etlplus package
+recursive-include etlplus/templates *.j2

{etlplus-0.4.5/etlplus.egg-info → etlplus-0.6.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: etlplus
-Version: 0.4.5
+Version: 0.6.1
 Summary: A Swiss Army knife for simple ETL operations
 Home-page: https://github.com/Dagitali/ETLPlus
 Author: ETLPlus Team
@@ -61,6 +61,8 @@ package and command-line interface for data extraction, validation, transformati
   - [Quickstart](#quickstart)
   - [Usage](#usage)
     - [Command Line Interface](#command-line-interface)
+      - [Inspect Pipelines](#inspect-pipelines)
+      - [Render SQL DDL](#render-sql-ddl)
       - [Extract Data](#extract-data)
       - [Validate Data](#validate-data)
       - [Transform Data](#transform-data)
@@ -88,6 +90,14 @@ package and command-line interface for data extraction, validation, transformati
 
 ## Features
 
+- **Check** data pipeline definitions before running them:
+  - Summarize jobs, sources, targets, and transforms
+  - Confirm configuration changes by printing focused sections on demand
+
+- **Render** SQL DDL from shared table specs:
+  - Generate CREATE TABLE or view statements
+  - Swap templates or direct output to files for database migrations
+
 - **Extract** data from multiple sources:
   - Files (CSV, JSON, XML, YAML)
   - Databases (connection string support)
@@ -169,6 +179,44 @@ etlplus --help
 etlplus --version
 ```
 
+#### Check Pipelines
+
+Use `etlplus check` to explore pipeline YAML definitions without running them. The command can print
+job names, summarize configured sources and targets, or drill into specific sections.
+
+List jobs and show a pipeline summary:
+```bash
+etlplus check --config examples/configs/pipeline.yml --jobs
+etlplus check --config examples/configs/pipeline.yml --summary
+```
+
+Show sources or transforms for troubleshooting:
+```bash
+etlplus check --config examples/configs/pipeline.yml --sources
+etlplus check --config examples/configs/pipeline.yml --transforms
+```
+
+#### Render SQL DDL
+
+Use `etlplus render` to turn table schema specs into ready-to-run SQL. Render from a pipeline config
+or from a standalone schema file, and choose the built-in `ddl` or `view` templates (or provide your
+own).
+
+Render all tables defined in a pipeline:
+```bash
+etlplus render --config examples/configs/pipeline.yml --template ddl
+```
+
+Render a single table in that pipeline:
+```bash
+etlplus render --config examples/configs/pipeline.yml --table customers --template view
+```
+
+Render from a standalone table spec to a file:
+```bash
+etlplus render --spec schemas/customer.yml --template view -o temp/customer_view.sql
+```
+
 #### Extract Data
 
 Note: For file sources, the format is normally inferred from the filename extension. Use
@@ -306,6 +354,19 @@ For YAML-driven pipelines executed end-to-end (extract → validate → transfor
 - Authoring: [`docs/pipeline-guide.md`](docs/pipeline-guide.md)
 - Runner API and internals: [`docs/run-module.md`](docs/run-module.md)
 
+CLI quick reference for pipelines:
+
+```bash
+# List jobs or show a pipeline summary
+etlplus check --config examples/configs/pipeline.yml --jobs
+etlplus check --config examples/configs/pipeline.yml --summary
+
+# Run a job
+etlplus run --config examples/configs/pipeline.yml --job file_to_file_customers
+
+# Deprecated shim (will be removed): etlplus pipeline
+```
+
 ### Complete ETL Pipeline Example
 
 ```bash

{etlplus-0.4.5 → etlplus-0.6.1}/README.md

@@ -19,6 +19,8 @@ package and command-line interface for data extraction, validation, transformati
   - [Quickstart](#quickstart)
   - [Usage](#usage)
     - [Command Line Interface](#command-line-interface)
+      - [Inspect Pipelines](#inspect-pipelines)
+      - [Render SQL DDL](#render-sql-ddl)
       - [Extract Data](#extract-data)
       - [Validate Data](#validate-data)
       - [Transform Data](#transform-data)
@@ -46,6 +48,14 @@ package and command-line interface for data extraction, validation, transformati
 
 ## Features
 
+- **Check** data pipeline definitions before running them:
+  - Summarize jobs, sources, targets, and transforms
+  - Confirm configuration changes by printing focused sections on demand
+
+- **Render** SQL DDL from shared table specs:
+  - Generate CREATE TABLE or view statements
+  - Swap templates or direct output to files for database migrations
+
 - **Extract** data from multiple sources:
   - Files (CSV, JSON, XML, YAML)
   - Databases (connection string support)
@@ -127,6 +137,44 @@ etlplus --help
 etlplus --version
 ```
 
+#### Check Pipelines
+
+Use `etlplus check` to explore pipeline YAML definitions without running them. The command can print
+job names, summarize configured sources and targets, or drill into specific sections.
+
+List jobs and show a pipeline summary:
+```bash
+etlplus check --config examples/configs/pipeline.yml --jobs
+etlplus check --config examples/configs/pipeline.yml --summary
+```
+
+Show sources or transforms for troubleshooting:
+```bash
+etlplus check --config examples/configs/pipeline.yml --sources
+etlplus check --config examples/configs/pipeline.yml --transforms
+```
+
+#### Render SQL DDL
+
+Use `etlplus render` to turn table schema specs into ready-to-run SQL. Render from a pipeline config
+or from a standalone schema file, and choose the built-in `ddl` or `view` templates (or provide your
+own).
+
+Render all tables defined in a pipeline:
+```bash
+etlplus render --config examples/configs/pipeline.yml --template ddl
+```
+
+Render a single table in that pipeline:
+```bash
+etlplus render --config examples/configs/pipeline.yml --table customers --template view
+```
+
+Render from a standalone table spec to a file:
+```bash
+etlplus render --spec schemas/customer.yml --template view -o temp/customer_view.sql
+```
+
 #### Extract Data
 
 Note: For file sources, the format is normally inferred from the filename extension. Use
@@ -264,6 +312,19 @@ For YAML-driven pipelines executed end-to-end (extract → validate → transfor
 - Authoring: [`docs/pipeline-guide.md`](docs/pipeline-guide.md)
 - Runner API and internals: [`docs/run-module.md`](docs/run-module.md)
 
+CLI quick reference for pipelines:
+
+```bash
+# List jobs or show a pipeline summary
+etlplus check --config examples/configs/pipeline.yml --jobs
+etlplus check --config examples/configs/pipeline.yml --summary
+
+# Run a job
+etlplus run --config examples/configs/pipeline.yml --job file_to_file_customers
+
+# Deprecated shim (will be removed): etlplus pipeline
+```
+
 ### Complete ETL Pipeline Example
 
 ```bash

{etlplus-0.4.5 → etlplus-0.6.1}/docs/pipeline-guide.md

@@ -378,31 +378,32 @@ jobs:
 Once you have a pipeline YAML, you can run jobs either from the
 command line or directly from Python.
 
-### CLI: `etlplus pipeline` and `etlplus run`
+### CLI: `etlplus check` (inspect) and `etlplus run` (execute)
 
-List jobs defined in a pipeline file:
+List jobs or show a summary from a pipeline file:
 
 ```bash
-etlplus pipeline --config examples/configs/pipeline.yml --list
+etlplus check --config examples/configs/pipeline.yml --jobs
+etlplus check --config examples/configs/pipeline.yml --summary
 ```
 
 Run a specific job end-to-end (extract → validate → transform → load):
 
 ```bash
-etlplus pipeline --config examples/configs/pipeline.yml --run file_to_file_customers
-
-# Equivalent, using the dedicated run command
 etlplus run --config examples/configs/pipeline.yml --job file_to_file_customers
 ```
 
 Notes:
 
-- Both commands read the same YAML schema described in this guide.
+- These commands read the same YAML schema described in this guide.
 - Environment-variable substitution (e.g. `${GITHUB_TOKEN}`) is applied the same way as when loading
   configs via the Python API.
 - For more details on the orchestration implementation, see
   [Runner internals: etlplus.run](run-module.md).
 
+Deprecated: `etlplus pipeline` is still available as a shim but will be removed in a future release;
+prefer `check` and `run`.
+
 ### Python: `etlplus.run.run`
 
 To trigger a job programmatically, use the high-level runner function exposed by the package: