etlplus 0.16.7__tar.gz → 0.16.9__tar.gz

{etlplus-0.16.7 → etlplus-0.16.9}/CONTRIBUTING.md

@@ -14,7 +14,7 @@ email, or any other method with the owners of this repository before making a ch
   - [Type Checking](#type-checking)
   - [Typing Philosophy](#typing-philosophy)
   - [Testing](#testing)
-    - [Unit vs. Integration](#unit-vs-integration)
+    - [Unit vs. Smoke vs. Integration](#unit-vs-smoke-vs-integration)
     - [Where to put tests](#where-to-put-tests)
     - [Common patterns](#common-patterns)
 
@@ -104,9 +104,9 @@ class ExampleConfig:
 
 ## Testing
 
-### Unit vs. Integration
+### Unit vs. Smoke vs. Integration
 
-Use these guidelines to decide whether a test belongs in the unit or integration suite:
+Use these guidelines to decide whether a test belongs in the unit, smoke, or integration suite:
 
 - Unit tests (put under `tests/unit/`):
   - Exercise a single function or class in isolation (no orchestration across modules).
@@ -117,13 +117,19 @@ Use these guidelines to decide whether a test belongs in the unit or integration
 - Integration tests (put under `tests/integration/`):
   - Exercise end-to-end flows across modules and boundaries (CLI `main()`, `run()` pipeline orchestration, file connectors, API client pagination wiring).
   - Can use temporary files/directories, and stub network with fakes/mocks.
-  - Examples in this repo: CLI end-to-end, pipeline smoke tests, pagination strategy, runner defaults for pagination/rate limits, target URL composition.
+  - Examples in this repo: CLI end-to-end, pagination strategy, runner defaults for pagination/rate limits, target URL composition.
+
+- Smoke tests (put under `tests/smoke/`):
+  - Minimal end-to-end checks for core flows; very fast and stable.
+  - May touch temporary files but avoid external network calls.
+  - Examples in this repo: pipeline smoke tests.
 
 If a test calls `etlplus.cli.main()` or `etlplus.ops.run.run()`, it is integration by default.
 
 ### Where to put tests
 
 - Unit tests live in `tests/unit/` and should import and test specific functions/classes directly.
+- Smoke tests live in `tests/smoke/` and should focus on minimal end-to-end CLI/pipeline checks.
 - Integration tests live in `tests/integration/` and may simulate real usage using CLI argv, temporary files, and fake clients.
 - The `tests/integration/conftest.py` applies `@pytest.mark.integration` to all tests in that folder. You don’t need to add the marker per test.
 - Markers are declared in `pytest.ini`. Avoid introducing ad-hoc markers without adding them there.

{etlplus-0.16.7/etlplus.egg-info → etlplus-0.16.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: etlplus
-Version: 0.16.7
+Version: 0.16.9
 Summary: A Swiss Army knife for simple ETL operations
 Home-page: https://github.com/Dagitali/ETLPlus
 Author: ETLPlus Team
@@ -355,9 +355,8 @@ etlplus --help
 etlplus --version
 ```
 
-The CLI is implemented with Typer (Click-based). The legacy argparse parser has been removed
-(`create_parser` now raises), so rely on the documented commands/flags and run `etlplus <command>
---help` for current options.
+The CLI is implemented with Typer (Click-based). The legacy argparse parser has been removed, so
+rely on the documented commands/flags and run `etlplus <command> --help` for current options.
 
 **Example error messages:**
 
@@ -722,10 +721,12 @@ pytest tests/ -v
 
 #### Test Layers
 
-We split tests into two layers:
+We split tests into three layers:
 
 - **Unit (`tests/unit/`)**: single function or class, no real I/O, fast, uses stubs/monkeypatch
   (e.g. small helpers in `etlplus.utils`, transform + validate helpers).
+- **Smoke (`tests/smoke/`)**: minimal end-to-end checks for core flows; may touch temp files but
+  avoids external network calls.
 - **Integration (`tests/integration/`)**: end-to-end flows (CLI `main()`, pipeline `run()`,
   pagination + rate limit defaults, file/API connector interactions) may touch temp files and use
   fake clients.

{etlplus-0.16.7 → etlplus-0.16.9}/README.md

@@ -305,9 +305,8 @@ etlplus --help
 etlplus --version
 ```
 
-The CLI is implemented with Typer (Click-based). The legacy argparse parser has been removed
-(`create_parser` now raises), so rely on the documented commands/flags and run `etlplus <command>
---help` for current options.
+The CLI is implemented with Typer (Click-based). The legacy argparse parser has been removed, so
+rely on the documented commands/flags and run `etlplus <command> --help` for current options.
 
 **Example error messages:**
 
@@ -672,10 +671,12 @@ pytest tests/ -v
 
 #### Test Layers
 
-We split tests into two layers:
+We split tests into three layers:
 
 - **Unit (`tests/unit/`)**: single function or class, no real I/O, fast, uses stubs/monkeypatch
   (e.g. small helpers in `etlplus.utils`, transform + validate helpers).
+- **Smoke (`tests/smoke/`)**: minimal end-to-end checks for core flows; may touch temp files but
+  avoids external network calls.
 - **Integration (`tests/integration/`)**: end-to-end flows (CLI `main()`, pipeline `run()`,
   pagination + rate limit defaults, file/API connector interactions) may touch temp files and use
   fake clients.

{etlplus-0.16.7 → etlplus-0.16.9}/docs/README.md

@@ -1,8 +1,12 @@
 # Documentation Notes
 
+- [Documentation Notes](#documentation-notes)
+  - [CLI Parser Status](#cli-parser-status)
+  - [Migration Hints](#migration-hints)
+
 ## CLI Parser Status
-- The CLI is Typer/Click-only. The historical `argparse` parser has been removed; `create_parser`
-  now raises to force migration.
+
+- The CLI is Typer/Click-only. The historical `argparse` parser has been removed.
 - Downstream tools should invoke the Typer app exported at `etlplus.cli.commands.app` (e.g., `python
   -m etlplus` or `etlplus ...`).
 - Handler functions still accept keyword arguments; the legacy namespace shim is temporary and will
@@ -10,8 +14,9 @@
   handlers with explicit keyword arguments if you integrate programmatically.
 
 ## Migration Hints
-- Replace any imports of `etlplus.cli.main.create_parser` with Typer invocations (`etlplus` binary
-  or `app` directly).
+
+- Replace any `argparse`-based integrations with Typer invocations (`etlplus` binary or `app`
+  directly).
 - If you maintained custom subcommands around the old parser, port them to Typer by attaching to
   `app` or wrapping the `etlplus` executable.
 - Tests and examples now target the Typer surface; expect argparse-focused helpers (e.g., namespace

{etlplus-0.16.7 → etlplus-0.16.9}/docs/pipeline-guide.md

@@ -3,15 +3,39 @@
 This guide explains how to author an ETLPlus pipeline YAML, using the example at `in/pipeline.yml`
 as a reference.
 
+- [Pipeline Authoring Guide](#pipeline-authoring-guide)
+  - [Overview](#overview)
+  - [Running a Pipeline from YAML (CLI)](#running-a-pipeline-from-yaml-cli)
+  - [Top-Level Structure](#top-level-structure)
+  - [APIs](#apis)
+    - [Profiles, base\_path, and auth](#profiles-base_path-and-auth)
+    - [Runner behavior with `base_path` (sources and targets)](#runner-behavior-with-base_path-sources-and-targets)
+  - [Databases](#databases)
+  - [File Systems](#file-systems)
+  - [Sources](#sources)
+  - [Validations](#validations)
+  - [Transforms](#transforms)
+  - [Targets](#targets)
+  - [Connector Parsing and Extension](#connector-parsing-and-extension)
+  - [Jobs](#jobs)
+  - [Running Pipelines (CLI and Python)](#running-pipelines-cli-and-python)
+    - [CLI: `etlplus check` (Inspect) and `etlplus run` (Execute)](#cli-etlplus-check-inspect-and-etlplus-run-execute)
+    - [Python: `etlplus.ops.run.run`](#python-etlplusopsrunrun)
+  - [Tips](#tips)
+  - [Design notes: Mapping inputs, dict outputs](#design-notes-mapping-inputs-dict-outputs)
+    - [Merge Semantics (Python 3.13)](#merge-semantics-python-313)
+    - [Extending Config Shapes](#extending-config-shapes)
+
+## Overview
+
 ETLPlus focuses on simple, JSON-first ETL. The pipeline file is a declarative description that your
 runner (a script, Makefile, CI job) can parse and execute using ETLPlus primitives: `extract`,
 `validate`, `transform`, and `load`.
 
-CLI note: ETLPlus uses Typer for command parsing. The legacy argparse parser has been removed
-(`create_parser` now raises). Use the documented `etlplus` commands and flags (check `etlplus
---help`) when wiring your runner.
+CLI note: ETLPlus uses Typer for command parsing. The legacy argparse parser has been removed. Use
+the documented `etlplus` commands and flags (check `etlplus --help`) when wiring your runner.
 
-## Running a pipeline from YAML (CLI)
+## Running a Pipeline from YAML (CLI)
 
 Use the built-in `etlplus run` command to execute jobs defined in a pipeline YAML. The command reads
 your config, resolves vars and env placeholders, then runs the requested job:
@@ -30,7 +54,7 @@ etlplus run --config examples/configs/pipeline.yml --job api_to_file_github_repo
 For scripted usage inside a larger Python project, prefer importing the Python API directly (e.g.,
 `extract`, `transform`, `validate`, `load`) instead of invoking the CLI subprocess.
 
-## Top-level structure
+## Top-Level Structure
 
 A pipeline file typically includes:
 
@@ -205,7 +229,7 @@ databases:
 Note: Database extract/load in ETLPlus is minimal today; consider this a placeholder for
 orchestration that calls into DB clients.
 
-## File systems
+## File Systems
 
 Point to local/cloud locations and logical folders:
 
@@ -362,7 +386,7 @@ targets:
       Content-Type: application/json
 ```
 
-## Connector parsing and extension
+## Connector Parsing and Extension
 
 Under the hood, source and target entries are parsed via a single tolerant constructor that looks at
 the `type` field and builds a concrete connector dataclass:
@@ -405,12 +429,12 @@ Notes:
 - `depends_on` is optional and can be a string or list of job names.
 - Jobs without dependencies run first when ordered as a DAG.
 
-## Running pipelines (CLI and Python)
+## Running Pipelines (CLI and Python)
 
 Once you have a pipeline YAML, you can run jobs either from the
 command line or directly from Python.
 
-### CLI: `etlplus check` (inspect) and `etlplus run` (execute)
+### CLI: `etlplus check` (Inspect) and `etlplus run` (Execute)
 
 List jobs or show a summary from a pipeline file:
 
@@ -508,7 +532,7 @@ Any]` rather than `dict[str, Any]` for inputs. Why?
 
 Practically, you can pass a plain `dict` everywhere and it will work.
 
-### Merge semantics (Python 3.13)
+### Merge Semantics (Python 3.13)
 
 We use the dict union operator for clarity:
 
@@ -521,7 +545,7 @@ Header precedence (lowest → highest):
 2. `profiles.<name>.headers`
 3. API top-level `headers`
 
-### Extending config shapes
+### Extending Config Shapes
 
 When adding new config objects or fields:
 

{etlplus-0.16.7 → etlplus-0.16.9}/etlplus/cli/main.py

@@ -3,15 +3,13 @@
 
 Entry point helpers for the Typer-powered ``etlplus`` CLI.
 
-This module exposes :func:`main` for the console script as well as
-:func:`create_parser`, which is kept for compatibility but now raises.
+This module exposes :func:`main` for the console script.
 """
 
 from __future__ import annotations
 
 import contextlib
 import sys
-import warnings
 
 import click
 import typer
@@ -121,25 +119,6 @@ def _is_unknown_command_error(
 # SECTION: FUNCTIONS ======================================================== #
 
 
-def create_parser() -> object:
-    """
-    Deprecated legacy entrypoint.
-
-    The argparse-based parser has been removed. Use the Typer-powered
-    ``etlplus`` CLI instead (``etlplus.cli.commands.app``).
-    """
-    warnings.warn(
-        'create_parser is deprecated and no longer returns an argparse '
-        'parser. Use the Typer CLI entrypoint instead.',
-        DeprecationWarning,
-        stacklevel=2,
-    )
-    raise RuntimeError(
-        'The legacy argparse parser has been removed. Invoke the Typer-based '
-        'CLI via `etlplus` or import `etlplus.cli.commands.app`.',
-    )
-
-
 def main(
     argv: list[str] | None = None,
 ) -> int:

{etlplus-0.16.7 → etlplus-0.16.9/etlplus.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: etlplus
-Version: 0.16.7
+Version: 0.16.9
 Summary: A Swiss Army knife for simple ETL operations
 Home-page: https://github.com/Dagitali/ETLPlus
 Author: ETLPlus Team
@@ -355,9 +355,8 @@ etlplus --help
 etlplus --version
 ```
 
-The CLI is implemented with Typer (Click-based). The legacy argparse parser has been removed
-(`create_parser` now raises), so rely on the documented commands/flags and run `etlplus <command>
---help` for current options.
+The CLI is implemented with Typer (Click-based). The legacy argparse parser has been removed, so
+rely on the documented commands/flags and run `etlplus <command> --help` for current options.
 
 **Example error messages:**
 
@@ -722,10 +721,12 @@ pytest tests/ -v
 
 #### Test Layers
 
-We split tests into two layers:
+We split tests into three layers:
 
 - **Unit (`tests/unit/`)**: single function or class, no real I/O, fast, uses stubs/monkeypatch
   (e.g. small helpers in `etlplus.utils`, transform + validate helpers).
+- **Smoke (`tests/smoke/`)**: minimal end-to-end checks for core flows; may touch temp files but
+  avoids external network calls.
 - **Integration (`tests/integration/`)**: end-to-end flows (CLI `main()`, pipeline `run()`,
   pagination + rate limit defaults, file/API connector interactions) may touch temp files and use
   fake clients.

{etlplus-0.16.7 → etlplus-0.16.9}/etlplus.egg-info/SOURCES.txt

@@ -174,6 +174,7 @@ examples/data/sample.json
 examples/data/sample.xml
 examples/data/sample.xsd
 examples/data/sample.yaml
+tests/README.md
 tests/__init__.py
 tests/conftest.py
 tests/integration/conftest.py
@@ -181,10 +182,10 @@ tests/integration/test_i_cli.py
 tests/integration/test_i_config_load.py
 tests/integration/test_i_examples_data_parity.py
 tests/integration/test_i_pagination_strategy.py
-tests/integration/test_i_pipeline_smoke.py
 tests/integration/test_i_run.py
 tests/integration/test_i_run_profile_pagination_defaults.py
 tests/integration/test_i_run_profile_rate_limit_defaults.py
+tests/smoke/test_s_pipeline.py
 tests/unit/conftest.py
 tests/unit/test_u_config.py
 tests/unit/test_u_main.py

etlplus-0.16.9/pytest.ini

@@ -0,0 +1,13 @@
+# pytest.ini
+# ETLPlus
+#
+# Copyright © 2025 Dagitali LLC. All rights reserved.
+
+[pytest]
+testpaths = tests/unit tests/smoke tests/integration
+addopts = -ra
+markers =
+    property: property-based tests (Hypothesis) for URL composition and encoding
+    smoke: fast end-to-end tests covering core flows
+    integration: tests that exercise multiple modules or external I/O boundaries
+    unit: tests that isolate individual modules or functions

etlplus-0.16.9/tests/README.md

@@ -0,0 +1,38 @@
+# Tests Overview
+
+ETLPlus uses three categories of tests. Each category lives in its own folder and is tagged with a
+pytest marker to make selection easy.
+
+## Unit tests (`tests/unit/`)
+
+- Scope: single function or class in isolation
+- I/O: no real file system or network I/O (use stubs/monkeypatch)
+- Speed: fast, deterministic
+- Marker: `unit`
+
+## Smoke tests (`tests/smoke/`)
+
+- Scope: minimal end-to-end sanity checks for core flows
+- I/O: may touch temp files; no external network calls
+- Speed: very fast; intended to catch obvious regressions
+- Marker: `smoke`
+
+## Integration tests (`tests/integration/`)
+
+- Scope: cross-module workflows and CLI/pipeline paths
+- I/O: may touch temp files; may use fakes/mocks for HTTP clients
+- Speed: slower than unit/smoke
+- Marker: `integration`
+
+## Running tests
+
+```bash
+pytest tests/unit
+pytest tests/smoke
+pytest tests/integration
+
+# Marker-based selection
+pytest -m unit
+pytest -m smoke
+pytest -m integration
+```

etlplus-0.16.7/tests/integration/test_i_pipeline_smoke.py → etlplus-0.16.9/tests/smoke/test_s_pipeline.py

@@ -1,9 +1,8 @@
 """
-:mod:`tests.integration.test_i_pipeline_smoke` module.
+:mod:`tests.smoke.test_s_pipeline` module.
 
-Pipeline smoke integration test suite exercising a minimal file→file job via
-the CLI (using the deprecated-free path). Parametrized to verify both empty
-and non-empty inputs.
+Smoke test suite exercising a minimal file→file job via the CLI. Parametrized
+to verify both empty and non-empty inputs.
 
 Notes
 -----
@@ -28,14 +27,14 @@ if TYPE_CHECKING:  # pragma: no cover - typing helpers only
 # SECTION: HELPERS ========================================================== #
 
 
-pytestmark = pytest.mark.integration
+pytestmark = pytest.mark.smoke
 
 
 # SECTION: TESTS ============================================================ #
 
 
-class TestPipelineSmoke:
-    """Integration test suite for file→file job via CLI."""
+class TestPipeline:
+    """Smoke test suite for file→file job via CLI."""
 
     @pytest.mark.parametrize(
         'data_in',

etlplus-0.16.7/pytest.ini

@@ -1,12 +0,0 @@
-# pytest.ini
-# ETLPlus
-#
-# Copyright © 2025 Dagitali LLC. All rights reserved.
-
-[pytest]
-testpaths = tests/unit tests/integration
-addopts = -ra
-markers =
-	property: property-based tests (Hypothesis) for URL composition and encoding
-	integration: tests that exercise multiple modules or external I/O boundaries
-    unit: tests that isolate individual modules or functions