etlplus 0.14.1__tar.gz → 0.15.5__tar.gz

{etlplus-0.14.1/etlplus.egg-info → etlplus-0.15.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: etlplus
-Version: 0.14.1
+Version: 0.15.5
 Summary: A Swiss Army knife for simple ETL operations
 Home-page: https://github.com/Dagitali/ETLPlus
 Author: ETLPlus Team
@@ -805,12 +805,12 @@ Navigate to detailed documentation for each subpackage:
 
 - [etlplus.api](etlplus/api/README.md): Lightweight HTTP client and paginated REST helpers
 - [etlplus.file](etlplus/file/README.md): Unified file format support and helpers
-- [etlplus.config](etlplus/config/README.md): Configuration helpers for connectors, pipelines, jobs,
-  and profiles
-- [etlplus.cli](etlplus/cli/README.md): Command-line interface for ETLPlus workflows
+- [etlplus.cli](etlplus/cli/README.md): Command-line interface definitions for `etlplus`
 - [etlplus.database](etlplus/database/README.md): Database engine, schema, and ORM helpers
 - [etlplus.templates](etlplus/templates/README.md): SQL and DDL template helpers
 - [etlplus.validation](etlplus/validation/README.md): Data validation utilities and helpers
+- [etlplus.workflow](etlplus/workflow/README.md): Helpers for data connectors, pipelines, jobs, and
+  profiles
 
 ### Community Health
 

{etlplus-0.14.1 → etlplus-0.15.5}/README.md

@@ -755,12 +755,12 @@ Navigate to detailed documentation for each subpackage:
 
 - [etlplus.api](etlplus/api/README.md): Lightweight HTTP client and paginated REST helpers
 - [etlplus.file](etlplus/file/README.md): Unified file format support and helpers
-- [etlplus.config](etlplus/config/README.md): Configuration helpers for connectors, pipelines, jobs,
-  and profiles
-- [etlplus.cli](etlplus/cli/README.md): Command-line interface for ETLPlus workflows
+- [etlplus.cli](etlplus/cli/README.md): Command-line interface definitions for `etlplus`
 - [etlplus.database](etlplus/database/README.md): Database engine, schema, and ORM helpers
 - [etlplus.templates](etlplus/templates/README.md): SQL and DDL template helpers
 - [etlplus.validation](etlplus/validation/README.md): Data validation utilities and helpers
+- [etlplus.workflow](etlplus/workflow/README.md): Helpers for data connectors, pipelines, jobs, and
+  profiles
 
 ### Community Health
 

{etlplus-0.14.1 → etlplus-0.15.5}/docs/pipeline-guide.md

@@ -377,9 +377,9 @@ Details:
 - Unknown or malformed entries are skipped rather than failing the whole load (keeping pipeline
   authoring permissive).
 - The connector kind is also available as a type-safe literal in code as
-  `etlplus.config.ConnectorType` (values: `"file" | "database" | "api"`).
+  `etlplus.workflow.ConnectorType` (values: `"file" | "database" | "api"`).
 
-To add new connector kinds in the future, implement a new dataclass in `etlplus.config.connector`
+To add new connector kinds in the future, implement a new dataclass in `etlplus.workflow.connector`
 and extend the internal parser to handle its `type` value.
 
 ## Jobs

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/README.md

@@ -1,4 +1,4 @@
-# etlplus package
+# `etlplus` Package
 
 The `etlplus` package provides a unified Python API and CLI for ETL operations: extraction,
 validation, transformation, and loading of data from files, APIs, and databases.
@@ -13,12 +13,12 @@ Back to project overview: see the top-level [README](../README.md).
 
 - [etlplus.api](api/README.md): Lightweight HTTP client and paginated REST helpers
 - [etlplus.file](file/README.md): Unified file format support and helpers
-- [etlplus.config](config/README.md): Configuration helpers for connectors, pipelines, jobs, and
-  profiles
-- [etlplus.cli](cli/README.md): Command-line interface for ETLPlus workflows
+- [etlplus.cli](cli/README.md): Command-line interface definitions for `etlplus`
 - [etlplus.database](database/README.md): Database engine, schema, and ORM helpers
 - [etlplus.templates](templates/README.md): SQL and DDL template helpers
 - [etlplus.validation](validation/README.md): Data validation utilities and helpers
+- [etlplus.workflow](etlplus/workflow/README.md): Helpers for data connectors, pipelines, jobs, and
+  profiles
 
 ## Quickstart
 

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/README.md

@@ -1,4 +1,4 @@
-# etlplus.api subpackage
+# `etlplus.api` Subpackage
 
 Documentation for the `etlplus.api` subpackage: a lightweight HTTP client and helpers for paginated
 REST endpoints.
@@ -12,7 +12,7 @@ REST endpoints.
 
 Back to project overview: see the top-level [README](../../README.md).
 
-- [etlplus.api subpackage](#etlplusapi-subpackage)
+- [`etlplus.api` Subpackage](#etlplusapi-subpackage)
   - [Installation](#installation)
   - [Quickstart](#quickstart)
     - [Overriding Rate Limits Per Call](#overriding-rate-limits-per-call)
@@ -22,6 +22,7 @@ Back to project overview: see the top-level [README](../../README.md).
   - [Authentication](#authentication)
   - [Errors and Rate Limiting](#errors-and-rate-limiting)
   - [Types and Transport](#types-and-transport)
+  - [Config Schemas](#config-schemas)
   - [Supporting Modules](#supporting-modules)
   - [Minimal Contract](#minimal-contract)
   - [See also](#see-also)
@@ -225,6 +226,36 @@ providers can fall back to their own defaults. If you already possess a static t
   `etlplus/api/request_manager.py` wraps `requests` sessions plus retry orchestration. Advanced
   users may consult those modules to adapt behavior.
 
+## Config Schemas
+
+`etlplus.api.types` defines TypedDict-based configuration shapes for API profiles and endpoints.
+Runtime parsing remains permissive in `etlplus.api.config`, but these types improve IDE
+autocomplete and static analysis.
+
+Exported types:
+
+- `ApiConfigMap`: top-level API config shape
+- `ApiProfileConfigMap`: per-profile API config shape
+- `ApiProfileDefaultsMap`: defaults block within a profile
+- `EndpointMap`: endpoint config shape
+
+Example:
+
+```python
+from etlplus.api import ApiConfigMap
+
+api_cfg: ApiConfigMap = {
+    "base_url": "https://example.test",
+    "headers": {"Authorization": "Bearer token"},
+    "endpoints": {
+        "users": {
+            "path": "/users",
+            "method": "GET",
+        },
+    },
+}
+```
+
 ## Supporting Modules
 
 - `etlplus.api.types` collects friendly aliases such as `Headers`, `Params`, `Url`, and

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/__init__.py

@@ -78,6 +78,7 @@ from .config import ApiConfig
 from .config import ApiProfileConfig
 from .config import EndpointConfig
 from .endpoint_client import EndpointClient
+from .enums import HttpMethod
 from .pagination import CursorPaginationConfigMap
 from .pagination import PagePaginationConfigMap
 from .pagination import PaginationClient
@@ -123,6 +124,7 @@ __all__ = [
     'RequestOptions',
     'RetryStrategy',
     # Enums
+    'HttpMethod',
     'PaginationType',
     # Functions
     'build_http_adapter',

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/auth.py

@@ -136,7 +136,7 @@ def _truncate(
     limit: int = MAX_LOG_BODY,
 ) -> str:
     """
-    Return ``text`` shortened to ``limit`` characters for logging.
+    Return *text* shortened to *limit* characters for logging.
 
     Parameters
     ----------

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/config.py

@@ -3,11 +3,6 @@
 
 Configuration dataclasses for REST API services, profiles, and endpoints.
 
-These models used to live under :mod:`etlplus.config`, but they belong in the
-API layer because they compose runtime types such as
-:class:`etlplus.api.EndpointClient`, :class:`etlplus.api.PaginationConfig`, and
-:class:`etlplus.api.RateLimitConfig`.
-
 Notes
 -----
 - TypedDict references remain editor hints only; :meth:`from_obj` accepts
@@ -18,6 +13,7 @@ Notes
 
 from __future__ import annotations
 
+from collections.abc import Callable
 from collections.abc import Mapping
 from dataclasses import dataclass
 from dataclasses import field
@@ -29,20 +25,20 @@ from typing import overload
 from urllib.parse import urlsplit
 from urllib.parse import urlunsplit
 
-from ..enums import HttpMethod
 from ..types import StrAnyMap
 from ..types import StrStrMap
 from ..utils import cast_str_dict
 from ..utils import coerce_dict
 from ..utils import maybe_mapping
 from .endpoint_client import EndpointClient
+from .enums import HttpMethod
 from .pagination import PaginationConfig
 from .rate_limiting import RateLimitConfig
 
 if TYPE_CHECKING:
-    from ..config.types import ApiConfigMap
-    from ..config.types import ApiProfileConfigMap
-    from ..config.types import EndpointMap
+    from .types import ApiConfigMap
+    from .types import ApiProfileConfigMap
+    from .types import EndpointMap
 
 
 # SECTION: EXPORTS ========================================================== #
@@ -72,7 +68,7 @@ def _effective_service_defaults(
     fallback_headers: dict[str, str],
 ) -> tuple[str, dict[str, str]]:
     """
-    Return ``(base_url, headers)`` using ``profiles`` when present.
+    Return ``(base_url, headers)`` using *profiles* when present.
 
     Parameters
     ----------
@@ -91,7 +87,7 @@ def _effective_service_defaults(
     Raises
     ------
     TypeError
-        If no profiles are defined and ``fallback_base`` is not a string.
+        If no profiles are defined and *fallback_base* is not a string.
     """
     if profiles:
         name = 'default' if 'default' in profiles else next(iter(profiles))
@@ -106,6 +102,33 @@ def _effective_service_defaults(
     return fallback_base, fallback_headers
 
 
+def _freeze_mapping(
+    mapping: Mapping[Any, Any],
+    *,
+    key_cast: Callable[[Any], Any] | None = None,
+) -> MappingProxyType:
+    """
+    Return an immutable copy of a mapping, optionally normalizing keys.
+
+    Parameters
+    ----------
+    mapping : Mapping[Any, Any]
+        Source mapping to freeze.
+    key_cast : Callable[[Any], Any] | None, optional
+        Optional key coercion applied to each key.
+
+    Returns
+    -------
+    MappingProxyType
+        Read-only mapping proxy with normalized keys.
+    """
+    if key_cast is None:
+        data = dict(mapping)
+    else:
+        data = {key_cast(key): value for key, value in mapping.items()}
+    return MappingProxyType(data)
+
+
 def _normalize_method(
     value: Any,
 ) -> Any | None:
@@ -232,16 +255,8 @@ class ApiProfileConfig:
     # -- Magic Methods (Object Lifecycle) -- #
 
     def __post_init__(self) -> None:
-        object.__setattr__(
-            self,
-            'headers',
-            MappingProxyType(dict(self.headers)),
-        )
-        object.__setattr__(
-            self,
-            'auth',
-            MappingProxyType(dict(self.auth)),
-        )
+        object.__setattr__(self, 'headers', _freeze_mapping(self.headers))
+        object.__setattr__(self, 'auth', _freeze_mapping(self.auth))
 
     # -- Class Methods -- #
 
@@ -340,20 +355,16 @@ class ApiConfig:
     # -- Magic Methods (Object Lifecycle) -- #
 
     def __post_init__(self) -> None:
-        object.__setattr__(
-            self,
-            'headers',
-            MappingProxyType(dict(self.headers)),
-        )
+        object.__setattr__(self, 'headers', _freeze_mapping(self.headers))
         object.__setattr__(
             self,
             'endpoints',
-            MappingProxyType({str(k): v for k, v in self.endpoints.items()}),
+            _freeze_mapping(self.endpoints, key_cast=str),
         )
         object.__setattr__(
             self,
             'profiles',
-            MappingProxyType({str(k): v for k, v in self.profiles.items()}),
+            _freeze_mapping(self.profiles, key_cast=str),
         )
 
     # -- Internal Instance Methods -- #
@@ -545,12 +556,12 @@ class EndpointConfig:
         object.__setattr__(
             self,
             'path_params',
-            MappingProxyType(dict(self.path_params)),
+            _freeze_mapping(self.path_params),
         )
         object.__setattr__(
             self,
             'query_params',
-            MappingProxyType(dict(self.query_params)),
+            _freeze_mapping(self.query_params),
         )
 
     # -- Class Methods -- #

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/endpoint_client.py

@@ -759,8 +759,8 @@ class EndpointClient:
         Raises
         ------
         KeyError
-            If ``endpoint_key`` is unknown or a required placeholder in the
-            path has no corresponding entry in ``path_parameters``.
+            If *endpoint_key* is unknown or a required placeholder in the path
+            has no corresponding entry in *path_parameters*.
         ValueError
             If the path template is invalid.
 
@@ -836,7 +836,7 @@ class EndpointClient:
         """
         Sleep for the specified seconds if positive.
 
-        The optional ``sleeper`` is useful for tests (e.g., pass
+        The optional *sleeper* is useful for tests (e.g., pass
         ``lambda s: None``). Defaults to using time.sleep when not provided.
 
         Parameters
@@ -870,7 +870,7 @@ class EndpointClient:
         rate_limit : RateLimitConfigMap | None
             Client-wide rate limit configuration.
         overrides : RateLimitOverrides, optional
-            Per-call overrides that take precedence over ``rate_limit``.
+            Per-call overrides that take precedence over *rate_limit*.
 
         Returns
         -------

etlplus-0.15.5/etlplus/api/enums.py

@@ -0,0 +1,51 @@
+"""
+:mod:`etlplus.api.enums` module.
+
+File-specific REST API-aligned enums and helpers.
+"""
+
+from __future__ import annotations
+
+from ..enums import CoercibleStrEnum
+
+# SECTION: EXPORTS ========================================================= #
+
+
+__all__ = [
+    # Enums
+    'HttpMethod',
+]
+
+
+# SECTION: ENUMS ============================================================ #
+
+
+class HttpMethod(CoercibleStrEnum):
+    """Supported HTTP verbs that accept JSON payloads."""
+
+    # -- Constants -- #
+
+    CONNECT = 'connect'
+    DELETE = 'delete'
+    GET = 'get'
+    HEAD = 'head'
+    OPTIONS = 'options'
+    PATCH = 'patch'
+    POST = 'post'
+    PUT = 'put'
+    TRACE = 'trace'
+
+    # -- Getters -- #
+
+    @property
+    def allows_body(self) -> bool:
+        """
+        Whether the method typically allows a request body.
+
+        Notes
+        -----
+        - RFCs do not strictly forbid bodies on some other methods (e.g.,
+            ``DELETE``), but many servers/clients do not expect them. We mark
+            ``POST``, ``PUT``, and ``PATCH`` as True.
+        """
+        return self in {HttpMethod.POST, HttpMethod.PUT, HttpMethod.PATCH}

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/pagination/client.py

@@ -1,5 +1,5 @@
 """
-:mod:`etlplus.api.client` module.
+:mod:`etlplus.api.pagination.client` module.
 
 Client-facing pagination driver for REST API responses.
 

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/pagination/config.py

@@ -398,7 +398,7 @@ class PaginationConfig(BoundsWarningsMixin):
         Returns
         -------
         Self | None
-            Parsed pagination configuration, or ``None`` if ``obj`` isn't a
+            Parsed pagination configuration, or ``None`` if *obj* isn't a
             mapping.
 
         Notes

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/pagination/paginator.py

@@ -66,14 +66,14 @@ def _resolve_path(
     path: str | None,
 ) -> Any:
     """
-    Resolve dotted ``path`` within ``obj`` or return ``_MISSING``.
+    Resolve dotted *path* within *obj* or return ``_MISSING``.
 
     Parameters
     ----------
     obj : Any
         JSON payload from an API response.
     path : str | None
-        Dotted path to the target value within ``obj``.
+        Dotted path to the target value within *obj*.
 
     Returns
     -------
@@ -665,7 +665,7 @@ class Paginator:
         records_path : str | None
             Optional dotted path to the records within the payload.
         fallback_path : str | None
-            Secondary dotted path consulted when ``records_path`` resolves to
+            Secondary dotted path consulted when *records_path* resolves to
             ``None`` or an empty list.
 
         Returns
@@ -675,9 +675,8 @@ class Paginator:
 
         Notes
         -----
-        Supports dotted path extraction via ``records_path`` and handles
-        lists, mappings, and scalars by coercing non-dict items into
-        ``{"value": x}``.
+        Supports dotted path extraction via *records_path* and handles lists,
+        mappings, and scalars by coercing non-dict items into ``{"value": x}``.
         """
         resolver = partial(_resolve_path, x)
         data = resolver(records_path)
@@ -729,7 +728,7 @@ class Paginator:
         Returns
         -------
         PaginationType | None
-            Detected pagination type, or ``default`` if not found.
+            Detected pagination type, or *default* if not found.
         """
         if not config:
             return default

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/rate_limiting/config.py

@@ -1,5 +1,5 @@
 """
-:mod:`etlplus.api.rate_limiting.rate_limiter` module.
+:mod:`etlplus.api.rate_limiting.config` module.
 
 Rate limiting configuration primitives.
 
@@ -84,14 +84,14 @@ def _merge_rate_limit(
     overrides: RateLimitOverrides = None,
 ) -> dict[str, Any]:
     """
-    Merge ``rate_limit`` and ``overrides`` honoring override precedence.
+    Merge *rate_limit* and *overrides* honoring override precedence.
 
     Parameters
     ----------
     rate_limit : StrAnyMap | None
         Base rate-limit configuration.
     overrides : RateLimitOverrides, optional
-        Override configuration with precedence over ``rate_limit``.
+        Override configuration with precedence over *rate_limit*.
 
     Returns
     -------
@@ -268,6 +268,18 @@ class RateLimitConfig(BoundsWarningsMixin):
     ) -> Self:
         """
         Normalize rate-limit config and overrides into a single instance.
+
+        Parameters
+        ----------
+        rate_limit : StrAnyMap | RateLimitConfig | None, optional
+            Base rate-limit configuration to normalize.
+        overrides : RateLimitOverrides, optional
+            Override values that take precedence over *rate_limit*.
+
+        Returns
+        -------
+        Self
+            Normalized rate-limit configuration.
         """
         normalized = _coerce_rate_limit_map(rate_limit)
         cfg = _merge_rate_limit(normalized, overrides)
@@ -318,7 +330,7 @@ class RateLimitConfig(BoundsWarningsMixin):
         Returns
         -------
         Self | None
-            Parsed instance, or ``None`` if ``obj`` isn't a mapping.
+            Parsed instance, or ``None`` if *obj* isn't a mapping.
         """
         if obj is None:
             return None

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/rate_limiting/rate_limiter.py

@@ -20,6 +20,7 @@ from __future__ import annotations
 
 import time
 from dataclasses import dataclass
+from typing import Self
 
 from ...utils import to_float
 from ...utils import to_positive_float
@@ -143,13 +144,13 @@ class RateLimiter:
     # -- Class Methods -- #
 
     @classmethod
-    def disabled(cls) -> RateLimiter:
+    def disabled(cls) -> Self:
         """
         Create a limiter that never sleeps.
 
         Returns
         -------
-        RateLimiter
+        Self
             Instance with rate limiting disabled.
         """
         return cls(sleep_seconds=0.0)
@@ -158,7 +159,7 @@ class RateLimiter:
     def fixed(
         cls,
         seconds: float,
-    ) -> RateLimiter:
+    ) -> Self:
         """
         Create a limiter with a fixed non-negative delay.
 
@@ -170,7 +171,7 @@ class RateLimiter:
 
         Returns
         -------
-        RateLimiter
+        Self
             Instance with the specified delay.
         """
         value = to_float(seconds, 0.0, minimum=0.0) or 0.0
@@ -181,7 +182,7 @@ class RateLimiter:
     def from_config(
         cls,
         cfg: RateLimitInput,
-    ) -> RateLimiter:
+    ) -> Self:
         """
         Build a :class:`RateLimiter` from a configuration mapping.
 
@@ -201,12 +202,10 @@ class RateLimiter:
 
         Returns
         -------
-        RateLimiter
+        Self
             Instance with normalized ``sleep_seconds`` and ``max_per_sec``.
         """
         config = RateLimitConfig.from_inputs(rate_limit=cfg)
-        if config is None:
-            return cls.disabled()
 
         # RateLimiter.__post_init__ will normalize and enforce invariants.
         return cls(**config.as_mapping())
@@ -236,7 +235,7 @@ class RateLimiter:
             Base rate-limit configuration. May contain ``"sleep_seconds"`` or
             ``"max_per_sec"``.
         overrides : RateLimitOverrides, optional
-            Optional overrides with the same keys as ``rate_limit``.
+            Optional overrides with the same keys as *rate_limit*.
 
         Returns
         -------
@@ -261,6 +260,4 @@ class RateLimiter:
             rate_limit=rate_limit,
             overrides=overrides,
         )
-        if config is None or not config.sleep_seconds:
-            return 0.0
-        return float(config.sleep_seconds)
+        return float(config.sleep_seconds) if config.sleep_seconds else 0.0

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/request_manager.py

@@ -14,6 +14,7 @@ from collections.abc import Sequence
 from dataclasses import dataclass
 from dataclasses import field
 from functools import partial
+from types import TracebackType
 from typing import Any
 from typing import cast
 
@@ -137,7 +138,7 @@ class RequestManager:
         self,
         exc_type: type[BaseException] | None,
         exc: BaseException | None,
-        tb: Any,
+        tb: TracebackType | None,
     ) -> None:
         """
         Exit the runtime context and close owned sessions.
@@ -148,7 +149,7 @@ class RequestManager:
             Exception type if raised, else ``None``.
         exc : BaseException | None
             Exception instance if raised, else ``None``.
-        tb : Any
+        tb : TracebackType | None
             Traceback if an exception was raised, else ``None``.
         """
         if self._ctx_session is None:
@@ -275,7 +276,7 @@ class RequestManager:
 
         try:
             policy = self.retry
-            if not policy:
+            if policy is None:
                 try:
                     return fetch(url, **call_kwargs)
                 except requests.RequestException as exc:  # pragma: no cover
@@ -438,9 +439,13 @@ class RequestManager:
             if isinstance(payload, dict):
                 return cast(JSONDict, payload)
             if isinstance(payload, list):
-                if all(isinstance(item, dict) for item in payload):
-                    return cast(JSONData, payload)
-                return [{'value': item} for item in payload]
+                out: list[JSONDict] = []
+                for item in payload:
+                    if isinstance(item, dict):
+                        out.append(cast(JSONDict, item))
+                    else:
+                        out.append({'value': item})
+                return cast(JSONData, out)
             return {'value': payload}
         return {
             'content': response.text,

{etlplus-0.14.1 → etlplus-0.15.5}/etlplus/api/retry_manager.py

@@ -278,7 +278,7 @@ class RetryManager:
         **kwargs: Any,
     ) -> JSONData:
         """
-        Execute ``func`` with exponential-backoff retries.
+        Execute *func* with exponential-backoff retries.
 
         Parameters
         ----------
@@ -287,7 +287,7 @@ class RetryManager:
         url : str
             URL for the API request.
         **kwargs : Any
-            Additional keyword arguments to pass to ``func``
+            Additional keyword arguments to pass to *func*
 
         Returns
         -------