etlplus 0.14.1__tar.gz → 0.15.4__tar.gz

{etlplus-0.14.1/etlplus.egg-info → etlplus-0.15.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: etlplus
-Version: 0.14.1
+Version: 0.15.4
 Summary: A Swiss Army knife for simple ETL operations
 Home-page: https://github.com/Dagitali/ETLPlus
 Author: ETLPlus Team
@@ -805,12 +805,12 @@ Navigate to detailed documentation for each subpackage:
 
 - [etlplus.api](etlplus/api/README.md): Lightweight HTTP client and paginated REST helpers
 - [etlplus.file](etlplus/file/README.md): Unified file format support and helpers
-- [etlplus.config](etlplus/config/README.md): Configuration helpers for connectors, pipelines, jobs,
-  and profiles
-- [etlplus.cli](etlplus/cli/README.md): Command-line interface for ETLPlus workflows
+- [etlplus.cli](etlplus/cli/README.md): Command-line interface definitions for `etlplus`
 - [etlplus.database](etlplus/database/README.md): Database engine, schema, and ORM helpers
 - [etlplus.templates](etlplus/templates/README.md): SQL and DDL template helpers
 - [etlplus.validation](etlplus/validation/README.md): Data validation utilities and helpers
+- [etlplus.workflow](etlplus/workflow/README.md): Helpers for data connectors, pipelines, jobs, and
+  profiles
 
 ### Community Health
 

{etlplus-0.14.1 → etlplus-0.15.4}/README.md

@@ -755,12 +755,12 @@ Navigate to detailed documentation for each subpackage:
 
 - [etlplus.api](etlplus/api/README.md): Lightweight HTTP client and paginated REST helpers
 - [etlplus.file](etlplus/file/README.md): Unified file format support and helpers
-- [etlplus.config](etlplus/config/README.md): Configuration helpers for connectors, pipelines, jobs,
-  and profiles
-- [etlplus.cli](etlplus/cli/README.md): Command-line interface for ETLPlus workflows
+- [etlplus.cli](etlplus/cli/README.md): Command-line interface definitions for `etlplus`
 - [etlplus.database](etlplus/database/README.md): Database engine, schema, and ORM helpers
 - [etlplus.templates](etlplus/templates/README.md): SQL and DDL template helpers
 - [etlplus.validation](etlplus/validation/README.md): Data validation utilities and helpers
+- [etlplus.workflow](etlplus/workflow/README.md): Helpers for data connectors, pipelines, jobs, and
+  profiles
 
 ### Community Health
 

{etlplus-0.14.1 → etlplus-0.15.4}/docs/pipeline-guide.md

@@ -377,9 +377,9 @@ Details:
 - Unknown or malformed entries are skipped rather than failing the whole load (keeping pipeline
   authoring permissive).
 - The connector kind is also available as a type-safe literal in code as
-  `etlplus.config.ConnectorType` (values: `"file" | "database" | "api"`).
+  `etlplus.workflow.ConnectorType` (values: `"file" | "database" | "api"`).
 
-To add new connector kinds in the future, implement a new dataclass in `etlplus.config.connector`
+To add new connector kinds in the future, implement a new dataclass in `etlplus.workflow.connector`
 and extend the internal parser to handle its `type` value.
 
 ## Jobs

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/README.md

@@ -1,4 +1,4 @@
-# etlplus package
+# `etlplus` Package
 
 The `etlplus` package provides a unified Python API and CLI for ETL operations: extraction,
 validation, transformation, and loading of data from files, APIs, and databases.
@@ -13,12 +13,12 @@ Back to project overview: see the top-level [README](../README.md).
 
 - [etlplus.api](api/README.md): Lightweight HTTP client and paginated REST helpers
 - [etlplus.file](file/README.md): Unified file format support and helpers
-- [etlplus.config](config/README.md): Configuration helpers for connectors, pipelines, jobs, and
-  profiles
-- [etlplus.cli](cli/README.md): Command-line interface for ETLPlus workflows
+- [etlplus.cli](cli/README.md): Command-line interface definitions for `etlplus`
 - [etlplus.database](database/README.md): Database engine, schema, and ORM helpers
 - [etlplus.templates](templates/README.md): SQL and DDL template helpers
 - [etlplus.validation](validation/README.md): Data validation utilities and helpers
+- [etlplus.workflow](etlplus/workflow/README.md): Helpers for data connectors, pipelines, jobs, and
+  profiles
 
 ## Quickstart
 

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/README.md

@@ -1,4 +1,4 @@
-# etlplus.api subpackage
+# `etlplus.api` Subpackage
 
 Documentation for the `etlplus.api` subpackage: a lightweight HTTP client and helpers for paginated
 REST endpoints.
@@ -12,7 +12,7 @@ REST endpoints.
 
 Back to project overview: see the top-level [README](../../README.md).
 
-- [etlplus.api subpackage](#etlplusapi-subpackage)
+- [`etlplus.api` Subpackage](#etlplusapi-subpackage)
   - [Installation](#installation)
   - [Quickstart](#quickstart)
     - [Overriding Rate Limits Per Call](#overriding-rate-limits-per-call)
@@ -22,6 +22,7 @@ Back to project overview: see the top-level [README](../../README.md).
   - [Authentication](#authentication)
   - [Errors and Rate Limiting](#errors-and-rate-limiting)
   - [Types and Transport](#types-and-transport)
+  - [Config Schemas](#config-schemas)
   - [Supporting Modules](#supporting-modules)
   - [Minimal Contract](#minimal-contract)
   - [See also](#see-also)
@@ -225,6 +226,36 @@ providers can fall back to their own defaults. If you already possess a static t
   `etlplus/api/request_manager.py` wraps `requests` sessions plus retry orchestration. Advanced
   users may consult those modules to adapt behavior.
 
+## Config Schemas
+
+`etlplus.api.types` defines TypedDict-based configuration shapes for API profiles and endpoints.
+Runtime parsing remains permissive in `etlplus.api.config`, but these types improve IDE
+autocomplete and static analysis.
+
+Exported types:
+
+- `ApiConfigMap`: top-level API config shape
+- `ApiProfileConfigMap`: per-profile API config shape
+- `ApiProfileDefaultsMap`: defaults block within a profile
+- `EndpointMap`: endpoint config shape
+
+Example:
+
+```python
+from etlplus.api import ApiConfigMap
+
+api_cfg: ApiConfigMap = {
+    "base_url": "https://example.test",
+    "headers": {"Authorization": "Bearer token"},
+    "endpoints": {
+        "users": {
+            "path": "/users",
+            "method": "GET",
+        },
+    },
+}
+```
+
 ## Supporting Modules
 
 - `etlplus.api.types` collects friendly aliases such as `Headers`, `Params`, `Url`, and

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/__init__.py

@@ -78,6 +78,7 @@ from .config import ApiConfig
 from .config import ApiProfileConfig
 from .config import EndpointConfig
 from .endpoint_client import EndpointClient
+from .enums import HttpMethod
 from .pagination import CursorPaginationConfigMap
 from .pagination import PagePaginationConfigMap
 from .pagination import PaginationClient
@@ -123,6 +124,7 @@ __all__ = [
     'RequestOptions',
     'RetryStrategy',
     # Enums
+    'HttpMethod',
     'PaginationType',
     # Functions
     'build_http_adapter',

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/config.py

@@ -3,11 +3,6 @@
 
 Configuration dataclasses for REST API services, profiles, and endpoints.
 
-These models used to live under :mod:`etlplus.config`, but they belong in the
-API layer because they compose runtime types such as
-:class:`etlplus.api.EndpointClient`, :class:`etlplus.api.PaginationConfig`, and
-:class:`etlplus.api.RateLimitConfig`.
-
 Notes
 -----
 - TypedDict references remain editor hints only; :meth:`from_obj` accepts
@@ -18,6 +13,7 @@ Notes
 
 from __future__ import annotations
 
+from collections.abc import Callable
 from collections.abc import Mapping
 from dataclasses import dataclass
 from dataclasses import field
@@ -29,20 +25,20 @@ from typing import overload
 from urllib.parse import urlsplit
 from urllib.parse import urlunsplit
 
-from ..enums import HttpMethod
 from ..types import StrAnyMap
 from ..types import StrStrMap
 from ..utils import cast_str_dict
 from ..utils import coerce_dict
 from ..utils import maybe_mapping
 from .endpoint_client import EndpointClient
+from .enums import HttpMethod
 from .pagination import PaginationConfig
 from .rate_limiting import RateLimitConfig
 
 if TYPE_CHECKING:
-    from ..config.types import ApiConfigMap
-    from ..config.types import ApiProfileConfigMap
-    from ..config.types import EndpointMap
+    from .types import ApiConfigMap
+    from .types import ApiProfileConfigMap
+    from .types import EndpointMap
 
 
 # SECTION: EXPORTS ========================================================== #
@@ -106,6 +102,33 @@ def _effective_service_defaults(
     return fallback_base, fallback_headers
 
 
+def _freeze_mapping(
+    mapping: Mapping[Any, Any],
+    *,
+    key_cast: Callable[[Any], Any] | None = None,
+) -> MappingProxyType:
+    """
+    Return an immutable copy of a mapping, optionally normalizing keys.
+
+    Parameters
+    ----------
+    mapping : Mapping[Any, Any]
+        Source mapping to freeze.
+    key_cast : Callable[[Any], Any] | None, optional
+        Optional key coercion applied to each key.
+
+    Returns
+    -------
+    MappingProxyType
+        Read-only mapping proxy with normalized keys.
+    """
+    if key_cast is None:
+        data = dict(mapping)
+    else:
+        data = {key_cast(key): value for key, value in mapping.items()}
+    return MappingProxyType(data)
+
+
 def _normalize_method(
     value: Any,
 ) -> Any | None:
@@ -232,16 +255,8 @@ class ApiProfileConfig:
     # -- Magic Methods (Object Lifecycle) -- #
 
     def __post_init__(self) -> None:
-        object.__setattr__(
-            self,
-            'headers',
-            MappingProxyType(dict(self.headers)),
-        )
-        object.__setattr__(
-            self,
-            'auth',
-            MappingProxyType(dict(self.auth)),
-        )
+        object.__setattr__(self, 'headers', _freeze_mapping(self.headers))
+        object.__setattr__(self, 'auth', _freeze_mapping(self.auth))
 
     # -- Class Methods -- #
 
@@ -340,20 +355,16 @@ class ApiConfig:
     # -- Magic Methods (Object Lifecycle) -- #
 
     def __post_init__(self) -> None:
-        object.__setattr__(
-            self,
-            'headers',
-            MappingProxyType(dict(self.headers)),
-        )
+        object.__setattr__(self, 'headers', _freeze_mapping(self.headers))
         object.__setattr__(
             self,
             'endpoints',
-            MappingProxyType({str(k): v for k, v in self.endpoints.items()}),
+            _freeze_mapping(self.endpoints, key_cast=str),
         )
         object.__setattr__(
             self,
             'profiles',
-            MappingProxyType({str(k): v for k, v in self.profiles.items()}),
+            _freeze_mapping(self.profiles, key_cast=str),
         )
 
     # -- Internal Instance Methods -- #
@@ -545,12 +556,12 @@ class EndpointConfig:
         object.__setattr__(
             self,
             'path_params',
-            MappingProxyType(dict(self.path_params)),
+            _freeze_mapping(self.path_params),
         )
         object.__setattr__(
             self,
             'query_params',
-            MappingProxyType(dict(self.query_params)),
+            _freeze_mapping(self.query_params),
         )
 
     # -- Class Methods -- #

etlplus-0.15.4/etlplus/api/enums.py

@@ -0,0 +1,51 @@
+"""
+:mod:`etlplus.api.enums` module.
+
+File-specific REST API-aligned enums and helpers.
+"""
+
+from __future__ import annotations
+
+from ..enums import CoercibleStrEnum
+
+# SECTION: EXPORTS ========================================================= #
+
+
+__all__ = [
+    # Enums
+    'HttpMethod',
+]
+
+
+# SECTION: ENUMS ============================================================ #
+
+
+class HttpMethod(CoercibleStrEnum):
+    """Supported HTTP verbs that accept JSON payloads."""
+
+    # -- Constants -- #
+
+    CONNECT = 'connect'
+    DELETE = 'delete'
+    GET = 'get'
+    HEAD = 'head'
+    OPTIONS = 'options'
+    PATCH = 'patch'
+    POST = 'post'
+    PUT = 'put'
+    TRACE = 'trace'
+
+    # -- Getters -- #
+
+    @property
+    def allows_body(self) -> bool:
+        """
+        Whether the method typically allows a request body.
+
+        Notes
+        -----
+        - RFCs do not strictly forbid bodies on some other methods (e.g.,
+            ``DELETE``), but many servers/clients do not expect them. We mark
+            ``POST``, ``PUT``, and ``PATCH`` as True.
+        """
+        return self in {HttpMethod.POST, HttpMethod.PUT, HttpMethod.PATCH}

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/pagination/client.py

@@ -1,5 +1,5 @@
 """
-:mod:`etlplus.api.client` module.
+:mod:`etlplus.api.pagination.client` module.
 
 Client-facing pagination driver for REST API responses.
 

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/rate_limiting/config.py

@@ -1,5 +1,5 @@
 """
-:mod:`etlplus.api.rate_limiting.rate_limiter` module.
+:mod:`etlplus.api.rate_limiting.config` module.
 
 Rate limiting configuration primitives.
 
@@ -268,6 +268,18 @@ class RateLimitConfig(BoundsWarningsMixin):
     ) -> Self:
         """
         Normalize rate-limit config and overrides into a single instance.
+
+        Parameters
+        ----------
+        rate_limit : StrAnyMap | RateLimitConfig | None, optional
+            Base rate-limit configuration to normalize.
+        overrides : RateLimitOverrides, optional
+            Override values that take precedence over ``rate_limit``.
+
+        Returns
+        -------
+        Self
+            Normalized rate-limit configuration.
         """
         normalized = _coerce_rate_limit_map(rate_limit)
         cfg = _merge_rate_limit(normalized, overrides)

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/rate_limiting/rate_limiter.py

@@ -20,6 +20,7 @@ from __future__ import annotations
 
 import time
 from dataclasses import dataclass
+from typing import Self
 
 from ...utils import to_float
 from ...utils import to_positive_float
@@ -143,13 +144,13 @@ class RateLimiter:
     # -- Class Methods -- #
 
     @classmethod
-    def disabled(cls) -> RateLimiter:
+    def disabled(cls) -> Self:
         """
         Create a limiter that never sleeps.
 
         Returns
         -------
-        RateLimiter
+        Self
             Instance with rate limiting disabled.
         """
         return cls(sleep_seconds=0.0)
@@ -158,7 +159,7 @@ class RateLimiter:
     def fixed(
         cls,
         seconds: float,
-    ) -> RateLimiter:
+    ) -> Self:
         """
         Create a limiter with a fixed non-negative delay.
 
@@ -170,7 +171,7 @@ class RateLimiter:
 
         Returns
         -------
-        RateLimiter
+        Self
             Instance with the specified delay.
         """
         value = to_float(seconds, 0.0, minimum=0.0) or 0.0
@@ -181,7 +182,7 @@ class RateLimiter:
     def from_config(
         cls,
         cfg: RateLimitInput,
-    ) -> RateLimiter:
+    ) -> Self:
         """
         Build a :class:`RateLimiter` from a configuration mapping.
 
@@ -201,12 +202,10 @@ class RateLimiter:
 
         Returns
         -------
-        RateLimiter
+        Self
             Instance with normalized ``sleep_seconds`` and ``max_per_sec``.
         """
         config = RateLimitConfig.from_inputs(rate_limit=cfg)
-        if config is None:
-            return cls.disabled()
 
         # RateLimiter.__post_init__ will normalize and enforce invariants.
         return cls(**config.as_mapping())
@@ -261,6 +260,4 @@ class RateLimiter:
             rate_limit=rate_limit,
             overrides=overrides,
         )
-        if config is None or not config.sleep_seconds:
-            return 0.0
-        return float(config.sleep_seconds)
+        return float(config.sleep_seconds) if config.sleep_seconds else 0.0

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/request_manager.py

@@ -14,6 +14,7 @@ from collections.abc import Sequence
 from dataclasses import dataclass
 from dataclasses import field
 from functools import partial
+from types import TracebackType
 from typing import Any
 from typing import cast
 
@@ -137,7 +138,7 @@ class RequestManager:
         self,
         exc_type: type[BaseException] | None,
         exc: BaseException | None,
-        tb: Any,
+        tb: TracebackType | None,
     ) -> None:
         """
         Exit the runtime context and close owned sessions.
@@ -148,7 +149,7 @@ class RequestManager:
             Exception type if raised, else ``None``.
         exc : BaseException | None
             Exception instance if raised, else ``None``.
-        tb : Any
+        tb : TracebackType | None
             Traceback if an exception was raised, else ``None``.
         """
         if self._ctx_session is None:
@@ -275,7 +276,7 @@ class RequestManager:
 
         try:
             policy = self.retry
-            if not policy:
+            if policy is None:
                 try:
                     return fetch(url, **call_kwargs)
                 except requests.RequestException as exc:  # pragma: no cover
@@ -438,9 +439,13 @@ class RequestManager:
             if isinstance(payload, dict):
                 return cast(JSONDict, payload)
             if isinstance(payload, list):
-                if all(isinstance(item, dict) for item in payload):
-                    return cast(JSONData, payload)
-                return [{'value': item} for item in payload]
+                out: list[JSONDict] = []
+                for item in payload:
+                    if isinstance(item, dict):
+                        out.append(cast(JSONDict, item))
+                    else:
+                        out.append({'value': item})
+                return cast(JSONData, out)
             return {'value': payload}
         return {
             'content': response.text,

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/transport.py

@@ -191,7 +191,19 @@ def _build_retry_value(
 def _normalize_retry_kwargs(
     retries_cfg: Mapping[str, Any],
 ) -> dict[str, Any]:
-    """Filter and normalize urllib3 ``Retry`` kwargs from a mapping."""
+    """
+    Filter and normalize urllib3 ``Retry`` kwargs from a mapping.
+
+    Parameters
+    ----------
+    retries_cfg : Mapping[str, Any]
+        Raw retry configuration mapping.
+
+    Returns
+    -------
+    dict[str, Any]
+        Filtered and normalized keyword arguments for ``Retry``.
+    """
     allowed_keys = {
         'total',
         'connect',
@@ -239,7 +251,7 @@ def _resolve_max_retries(
     """
     match retries_cfg:
         case int():
-            return retries_cfg
+            return to_maximum_int(retries_cfg, 0)
         case Mapping():
             try:
                 return _build_retry_value(retries_cfg)

{etlplus-0.14.1 → etlplus-0.15.4}/etlplus/api/types.py

@@ -20,8 +20,11 @@ Examples
 from __future__ import annotations
 
 from collections.abc import Callable
+from collections.abc import Mapping
 from dataclasses import dataclass
 from typing import Any
+from typing import Self
+from typing import TypedDict
 from typing import cast
 
 from ..types import JSONData
@@ -39,6 +42,11 @@ __all__ = [
     'Headers',
     'Params',
     'Url',
+    # Typed Dicts
+    'ApiConfigMap',
+    'ApiProfileConfigMap',
+    'ApiProfileDefaultsMap',
+    'EndpointMap',
 ]
 
 
@@ -48,6 +56,88 @@ __all__ = [
 _UNSET = object()
 
 
+# SECTION: TYPED DICTS ====================================================== #
+
+
+class ApiConfigMap(TypedDict, total=False):
+    """
+    Top-level API config shape parsed by ApiConfig.from_obj.
+
+    Either provide a ``base_url`` with optional ``headers`` and ``endpoints``,
+    or provide ``profiles`` with at least one profile having a ``base_url``.
+
+    See Also
+    --------
+    - :class:`etlplus.api.config.ApiConfig`
+    """
+
+    base_url: str
+    headers: StrAnyMap
+    endpoints: Mapping[str, EndpointMap | str]
+    profiles: Mapping[str, ApiProfileConfigMap]
+
+
+class ApiProfileConfigMap(TypedDict, total=False):
+    """
+    Shape accepted for a profile entry under ApiConfigMap.profiles.
+
+    Notes
+    -----
+    ``base_url`` is required at runtime when profiles are provided.
+
+    See Also
+    --------
+    - :class:`etlplus.api.config.ApiProfileConfig`
+    """
+
+    base_url: str
+    headers: StrAnyMap
+    base_path: str
+    auth: StrAnyMap
+    defaults: ApiProfileDefaultsMap
+
+
+class ApiProfileDefaultsMap(TypedDict, total=False):
+    """
+    Defaults block available under a profile (all keys optional).
+
+    Notes
+    -----
+    Runtime expects header values to be str; typing remains permissive.
+
+    See Also
+    --------
+    - :class:`etlplus.api.config.ApiProfileConfig`
+    - :class:`etlplus.api.pagination.PaginationConfig`
+    - :class:`etlplus.api.rate_limiting.RateLimitConfig`
+    """
+
+    headers: StrAnyMap
+    pagination: Any
+    rate_limit: Any
+
+
+class EndpointMap(TypedDict, total=False):
+    """
+    Shape accepted by EndpointConfig.from_obj.
+
+    One of ``path`` or ``url`` should be provided.
+
+    See Also
+    --------
+    - :class:`etlplus.api.config.EndpointConfig`
+    """
+
+    path: str
+    url: str
+    method: str
+    path_params: StrAnyMap
+    query_params: StrAnyMap
+    body: Any
+    pagination: Any
+    rate_limit: Any
+
+
 # SECTION: DATA CLASSES ===================================================== #
 
 
@@ -75,9 +165,9 @@ class RequestOptions:
     # -- Magic Methods (Object Lifecycle) -- #
 
     def __post_init__(self) -> None:
-        if self.params:
+        if self.params is not None:
             object.__setattr__(self, 'params', dict(self.params))
-        if self.headers:
+        if self.headers is not None:
             object.__setattr__(self, 'headers', dict(self.headers))
 
     # -- Instance Methods -- #
@@ -92,9 +182,9 @@ class RequestOptions:
             Keyword arguments for ``requests`` methods.
         """
         kw: dict[str, Any] = {}
-        if self.params:
+        if self.params is not None:
             kw['params'] = dict(self.params)
-        if self.headers:
+        if self.headers is not None:
             kw['headers'] = dict(self.headers)
         if self.timeout is not None:
             kw['timeout'] = self.timeout
@@ -106,7 +196,7 @@ class RequestOptions:
         params: Params | None | object = _UNSET,
         headers: Headers | None | object = _UNSET,
         timeout: float | None | object = _UNSET,
-    ) -> RequestOptions:
+    ) -> Self:
         """
         Return a copy with the provided fields replaced.
 
@@ -146,7 +236,7 @@ class RequestOptions:
         else:
             next_timeout = cast(float | None, timeout)
 
-        return RequestOptions(
+        return self.__class__(
             params=next_params,
             headers=next_headers,
             timeout=next_timeout,