DataExcept 0.4.0__tar.gz → 0.4.1__tar.gz

{dataexcept-0.4.0 → dataexcept-0.4.1}/CHANGELOG.md

@@ -7,6 +7,124 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.4.1] - 2026-08-25
+
+### Added
+
+- Tests that derive the facts stated in more than one file, instead of trusting
+  them to be copied correctly: the citation, security policy, changelog and
+  checklist must all name the current version, checklist sections must be
+  numbered uniquely, no document may quote a test count, and the supported
+  Python range must agree across `requires-python`, the classifiers, the CI
+  matrix and the release gate.
+- A contract-coverage test. The whole-hierarchy suites called `pytest.skip()`
+  when they could not build a class, so a class could sit outside the pickling,
+  message and inheritance guarantees with CI still green — and two were doing
+  exactly that. Skipping now requires an explicit, reasoned entry in an
+  exclusion table, which is empty.
+
+- **Python 3.14 support.** `requires-python` capped at `<3.14`, so the package
+  refused to install on the current stable interpreter — 3.14 was released in
+  October 2025 and is no longer upcoming. The range is now `>=3.10,<3.15`, 3.14
+  is in the classifiers and the CI matrix, and `test (3.14)` joins the checks
+  the release workflow requires before it will build.
+
+### Changed
+
+- `SECURITY.md` states the boundary precisely, including what is **not**
+  covered: a bare non-URL secret written into free-form text cannot be
+  recognised. The previous wording claimed credentials "never appear ...
+  whatever you pass in", which was broader than the implementation.
+
+- The serialization and hierarchy guarantees are stated more precisely.
+  "Survives a process boundary intact" now says what happens to state that
+  cannot be serialized, and "catches anything this package raises" is now
+  "every operational exception" — constructors deliberately raise plain
+  `TypeError` for invalid arguments, which sits outside the hierarchy.
+
+### Fixed
+
+- **An exception pickled by 0.4.0 could not be loaded by 0.4.1.** Restoring the
+  chain added three parameters to the private `_rebuild`, and an older payload
+  passes only three arguments — so a queued exception that outlived an upgrade,
+  or one sent by a worker on the previous release, raised `TypeError` on
+  unpickling. The new parameters carry defaults.
+
+- The test probe fed a bare string to `Sequence[str]` parameters, so
+  `DataFormatError` and `DtypeMismatchError` correctly rejected it and were
+  silently skipped. The probe builds them properly now; nothing is skipped.
+- `is_number` accepted booleans, because `bool` subclasses `int` and so
+  satisfies `numbers.Real`. A boolean is never a meaningful metric, threshold
+  or ratio, and accepting one hid a caller passing the wrong variable.
+- `MergeKeyError("id", "cust_id")` silently became `['i', 'd']` — a bare string
+  is a sequence of strings. It now raises `TypeError`, matching
+  `DtypeMismatchError`, which already rejected this.
+- `SECURITY.md` named 0.3.x as supported at 0.4.0, `CHECKLIST.md` was dated to
+  the previous release and numbered two sections 12, and the README quoted a
+  test count that no longer matched. `scripts/bump_version.py` now writes every
+  file that states the version, so they cannot drift apart by hand again.
+- `black`'s target version is pinned rather than inferred from
+  `requires-python`. Adding 3.14 made inference pick `py314`, and black then
+  refused to verify its own output when run on an older interpreter — which is
+  what CI does.
+
+- **Exception chaining was lost on a pickle round trip.** `__reduce__` saved
+  `args` and `__dict__`, but `__cause__`, `__context__` and
+  `__suppress_context__` are special exception state rather than `__dict__`
+  entries — so a wrapped exception rebuilt in another process no longer showed
+  what actually failed. All three are now restored explicitly. The 0.4.0 tests
+  did not catch this because they compared `args`, rendered text and `__dict__`
+  and never the chain; they now check it.
+- **An exception carrying unpickleable state could not cross a process
+  boundary at all.** Several classes accept arbitrary caller state, so a
+  lambda, generator, open file or locally defined class made the whole
+  exception unserializable — replacing the real failure with a serialization
+  error about it. Such values are now replaced by an `UnpicklableValue`
+  describing what was there, and an unpickleable cause by an
+  `UnpicklableCause`, so the exception still arrives.
+
+### Security
+
+- **Redaction is no longer defeated by the traceback.** `log_exception` passed
+  `exc_info`, so logging rendered the whole exception chain — including a
+  wrapped third-party exception whose own message still quoted the
+  credential-bearing URL. Redacting what DataExcept renders did nothing about
+  that. When the chain contains a URL, `log_exception` now formats the
+  traceback and scrubs it; every other exception keeps the structured
+  `exc_info` path, so nothing changes for them.
+- `SECURITY.md` documents what remains outside that boundary: the wrapped
+  exception object is still reachable, so `traceback.print_exc()`,
+  `repr(exc.__dict__)` or `logger.error(..., exc_info=True)` will render its
+  text. A third-party exception's message is not ours to rewrite.
+
+- **Every GitHub Action is pinned to a full-length commit SHA.** All 34
+  references used mutable tags, including `pypa/gh-action-pypi-publish` in the
+  OIDC publishing job — the step that holds the credential which uploads to
+  PyPI. Whoever controls an action repository can move a tag to different code
+  at any time; a commit SHA is the only immutable reference. Each pin carries
+  the version in a trailing comment so it stays reviewable and Dependabot can
+  still update it, and a test fails if any mutable reference reappears.
+
+- **Credentials in a URL path are now redacted.** `redact_url` kept the whole
+  path, so `WebhookError` logged a Slack webhook URL — which Slack documents as
+  a secret in its entirety — unchanged. Where the path *is* the credential the
+  path is now dropped, keeping the host, which is what makes the error
+  actionable.
+- **Sensitive parameters are matched by substring, not an exact allowlist.**
+  `X-Amz-Signature`, `X-Amz-Credential`, `auth_token` and `refresh_token` all
+  passed through before. Fragments are covered too, so an OAuth
+  `#access_token=` no longer survives.
+- **A secret can no longer be reintroduced after redaction.** Redacting only
+  the structured argument left two open routes: a caller-supplied `message`,
+  and the text of a wrapped exception quoting the original URL. Every message
+  in the hierarchy now passes through one scrubbing boundary, so a URL is
+  redacted wherever it appears. Where the library was handed the secret
+  explicitly, that exact value is removed from the message as well.
+- **URL-bearing fields beyond the four patched classes are redacted.**
+  `DataLoadingError.source` documents itself as "file path, URL" and rendered
+  a presigned S3 URL verbatim. Nine such fields now use `redact_if_url`, which
+  leaves ordinary file paths untouched.
+
 ## [0.4.0] - 2026-08-24
 
 ### Added
@@ -236,7 +354,6 @@ project description, and its quick-start example did not run.
 - `examples/example_usage.py` raised `TimeoutError` with keyword arguments the
   builtin does not accept — a live instance of the shadowing hazard.
 
-
 ## [0.1.0] - 2026-08-24
 
 First public release.
@@ -268,7 +385,8 @@ First public release.
 - Published to PyPI via OIDC trusted publishing; no long-lived API token is
   involved in a release.
 
-[Unreleased]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.4.0...HEAD
+[Unreleased]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.4.1...HEAD
+[0.4.1]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.4.0...v0.4.1
 [0.4.0]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.2.0...v0.2.1

{dataexcept-0.4.0 → dataexcept-0.4.1}/CITATION.cff

@@ -1,7 +1,7 @@
 cff-version: 1.2.0
 message: "If you use this software, please cite it using the following metadata."
 title: "DataExcept"
-version: "0.4.0"
+version: "0.4.1"
 authors:
   - family-names: "Ribeiro"
     given-names: "Diogo"
@@ -11,4 +11,4 @@ authors:
 type: software
 url: "https://github.com/DiogoRibeiro7/DataExcept"
 repository-code: "https://github.com/DiogoRibeiro7/DataExcept"
-date-released: "2026-08-24"
+date-released: "2026-08-25"

{dataexcept-0.4.0 → dataexcept-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: DataExcept
-Version: 0.4.0
+Version: 0.4.1
 Summary: A Python package providing structured, easily-extendable custom exception types.
 License-Expression: MIT
 License-File: LICENSE
@@ -9,13 +9,14 @@ Author: Diogo Ribeiro
 Author-email: dfr@esmad.ipp.pt
 Maintainer: Diogo Ribeiro
 Maintainer-email: diogo.debastos.ribeiro@gmail.com
-Requires-Python: >=3.10,<3.14
+Requires-Python: >=3.10,<3.15
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Requires-Dist: tomli ; python_version < "3.11"
 Project-URL: Changelog, https://github.com/DiogoRibeiro7/DataExcept/releases
 Project-URL: Documentation, https://diogoribeiro7.github.io/DataExcept/
@@ -41,13 +42,13 @@ Description-Content-Type: text/markdown
 
 ## 🎯 Key Features
 
-- **🏗️ Hierarchical Structure**: Catch one specific error, a whole domain, or everything via `DataExceptError`
+- **🏗️ Hierarchical Structure**: Catch one specific error, a whole domain, or every operational error via `DataExceptError`
 - **📦 One Import**: Every exception is available from `dataexcept` directly, or from its domain module — same objects either way
-- **📊 Data Science Focused**: 99 exception classes covering ML pipelines, feature engineering, model training
-- **🔧 Production Ready**: Logging helpers, error context, and exceptions that survive a process boundary intact
+- **📊 Data Science Focused**: 100 exception classes covering ML pipelines, feature engineering, model training
+- **🔧 Production Ready**: Logging helpers, error context, and exceptions that pickle — so they cross a process boundary with their message, attributes and cause intact
 - **📚 Academic Quality**: Proper documentation, type hints, and citation support
-- **🐍 Python 3.10+**: Modern Python with full type safety
-- **🧪 Well Tested**: 690+ tests at 92% branch coverage of the package, gated in CI
+- **🐍 Python 3.10 – 3.14**: Every supported version tested in CI, with full type safety
+- **🧪 Well Tested**: Full branch coverage of the package gated in CI, with contract tests over every exception class
 
 ## 📦 Quick Installation
 
@@ -213,7 +214,7 @@ except Exception as exc:
 ### Command Line Interface
 
 ```bash
-# List every exception class the package exports (99 of them, alphabetically)
+# List every exception class the package exports (100 of them, alphabetically)
 $ dataexcept list
 ApiError
 AuthenticationError
@@ -224,7 +225,7 @@ BiasDetectionError
 
 # Check version
 $ dataexcept --version
-dataexcept 0.4.0
+dataexcept 0.4.1
 ```
 
 ## 🎯 Use Cases
@@ -374,7 +375,7 @@ If you use DataExcept in your research, please cite it:
   author = {Ribeiro, Diogo},
   title = {DataExcept: Structured Exception Handling for Data Science},
   url = {https://github.com/DiogoRibeiro7/DataExcept},
-  version = {0.4.0},
+  version = {0.4.1},
   year = {2026},
   publisher = {GitHub}
 }

{dataexcept-0.4.0 → dataexcept-0.4.1}/README.md

@@ -15,13 +15,13 @@
 
 ## 🎯 Key Features
 
-- **🏗️ Hierarchical Structure**: Catch one specific error, a whole domain, or everything via `DataExceptError`
+- **🏗️ Hierarchical Structure**: Catch one specific error, a whole domain, or every operational error via `DataExceptError`
 - **📦 One Import**: Every exception is available from `dataexcept` directly, or from its domain module — same objects either way
-- **📊 Data Science Focused**: 99 exception classes covering ML pipelines, feature engineering, model training
-- **🔧 Production Ready**: Logging helpers, error context, and exceptions that survive a process boundary intact
+- **📊 Data Science Focused**: 100 exception classes covering ML pipelines, feature engineering, model training
+- **🔧 Production Ready**: Logging helpers, error context, and exceptions that pickle — so they cross a process boundary with their message, attributes and cause intact
 - **📚 Academic Quality**: Proper documentation, type hints, and citation support
-- **🐍 Python 3.10+**: Modern Python with full type safety
-- **🧪 Well Tested**: 690+ tests at 92% branch coverage of the package, gated in CI
+- **🐍 Python 3.10 – 3.14**: Every supported version tested in CI, with full type safety
+- **🧪 Well Tested**: Full branch coverage of the package gated in CI, with contract tests over every exception class
 
 ## 📦 Quick Installation
 
@@ -187,7 +187,7 @@ except Exception as exc:
 ### Command Line Interface
 
 ```bash
-# List every exception class the package exports (99 of them, alphabetically)
+# List every exception class the package exports (100 of them, alphabetically)
 $ dataexcept list
 ApiError
 AuthenticationError
@@ -198,7 +198,7 @@ BiasDetectionError
 
 # Check version
 $ dataexcept --version
-dataexcept 0.4.0
+dataexcept 0.4.1
 ```
 
 ## 🎯 Use Cases
@@ -348,7 +348,7 @@ If you use DataExcept in your research, please cite it:
   author = {Ribeiro, Diogo},
   title = {DataExcept: Structured Exception Handling for Data Science},
   url = {https://github.com/DiogoRibeiro7/DataExcept},
-  version = {0.4.0},
+  version = {0.4.1},
   year = {2026},
   publisher = {GitHub}
 }

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/__init__.py

@@ -38,7 +38,7 @@ from . import (  # noqa: F401
     security_exceptions,
 )
 from ._deprecation import resolve_deprecated
-from .base import DataExceptError
+from .base import DataExceptError, UnpicklableCause, UnpicklableValue
 from .database_exceptions import (
     DatabaseConnectionError,
     DatabaseError,
@@ -163,8 +163,12 @@ from .security_exceptions import (
 )
 
 __all__ = [
-    # The root of the hierarchy: catches anything this package raises.
+    # The root of the hierarchy: catches every operational exception the
+    # package raises.
     "DataExceptError",
+    # Placeholders for state that could not survive serialization.
+    "UnpicklableCause",
+    "UnpicklableValue",
     # Every exception class the package defines.
     "ApiError",
     "AuthenticationError",

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/_validation.py

@@ -15,8 +15,14 @@ def is_number(value: object) -> bool:
     science. ``numbers.Real`` accepts them because NumPy registers its scalar
     types with the ABC, and it needs no dependency on NumPy to do so.
 
+    Booleans are excluded: ``bool`` subclasses ``int``, so ``numbers.Real``
+    would accept ``True`` as a metric.
+
     This is a function rather than an inline ``isinstance`` because narrowing a
     value to ``numbers.Real`` defeats mypy's inference for the rest of the
     enclosing class.
     """
-    return isinstance(value, numbers.Real)
+    # bool subclasses int, so numbers.Real accepts True and False. A boolean
+    # is never a meaningful metric, threshold or ratio, and accepting one hides
+    # a caller passing the wrong variable.
+    return not isinstance(value, bool) and isinstance(value, numbers.Real)

dataexcept-0.4.1/dataexcept/base.py

@@ -0,0 +1,190 @@
+"""The root of the DataExcept exception hierarchy.
+
+Every operational exception this package defines derives from
+:class:`DataExceptError`, so a caller can catch the whole library with one
+clause while still catching narrowly where it matters::
+
+    try:
+        run_pipeline()
+    except ValidationError:
+        ...              # exactly this failure
+    except DataExceptError:
+        ...              # anything else DataExcept raised
+
+(Constructors also raise plain ``TypeError`` when given invalid arguments.
+Those are programming errors, not operational ones, and are deliberately not
+part of this hierarchy.)
+
+The base also carries the serialization contract for the hierarchy. Two
+problems make that necessary:
+
+* Most constructors take several arguments while ``Exception.args`` holds only
+  the rendered message, so the default protocol -- which replays ``args``
+  through ``__init__`` -- cannot rebuild them.
+* Several exceptions accept arbitrary caller state (``DataValidationError``
+  takes any ``value``), and that state may not be pickleable at all.
+
+An exception that cannot cross a process boundary is useless exactly where a
+data pipeline needs it most, so rather than fail, unpickleable state is
+replaced by a description of what was there.
+"""
+
+from __future__ import annotations
+
+import pickle
+from typing import Any, Dict, Optional, Tuple, Type
+
+from .redaction import redact_urls_in_text
+
+__all__ = ["DataExceptError", "UnpicklableCause", "UnpicklableValue"]
+
+#: Attribute names used across the package to hold the exception that caused
+#: this one. Checked in order; the first that holds an exception wins.
+_CAUSE_ATTRIBUTES = ("original", "original_exception", "cause")
+
+
+class UnpicklableValue:
+    """Stands in for state that could not survive serialization.
+
+    An exception carrying a lambda, an open file or a lock would otherwise be
+    unraisable across a process boundary. Keeping a description preserves what
+    the value was for debugging, which is the reason it was attached.
+    """
+
+    __slots__ = ("description",)
+
+    def __init__(self, description: str) -> None:
+        self.description = description
+
+    def __repr__(self) -> str:
+        return f"<unpicklable: {self.description}>"
+
+    def __str__(self) -> str:
+        return self.__repr__()
+
+    def __eq__(self, other: object) -> bool:
+        return (
+            isinstance(other, UnpicklableValue)
+            and other.description == self.description
+        )
+
+    def __hash__(self) -> int:
+        return hash(self.description)
+
+
+def _safe(value: Any) -> Any:
+    """Return *value*, or a placeholder if it cannot be pickled."""
+    try:
+        pickle.dumps(value)
+    except Exception:
+        try:
+            description = f"{type(value).__name__}: {value!r}"
+        except Exception:  # pragma: no cover - a repr that itself raises
+            description = type(value).__name__
+        return UnpicklableValue(description[:200])
+    return value
+
+
+def _safe_exception(exc: Optional[BaseException]) -> Optional[BaseException]:
+    """Return *exc*, or an exception describing it if it cannot be pickled."""
+    if exc is None:
+        return None
+    try:
+        pickle.dumps(exc)
+    except Exception:
+        return UnpicklableCause(f"{type(exc).__name__}: {exc}")
+    return exc
+
+
+def _rebuild(
+    cls: Type["DataExceptError"],
+    args: Tuple[Any, ...],
+    state: Dict[str, Any],
+    cause: Optional[BaseException] = None,
+    context: Optional[BaseException] = None,
+    suppress_context: bool = False,
+) -> "DataExceptError":
+    """Recreate *cls* without replaying its ``__init__``.
+
+    Constructors validate and render a message from their arguments; replaying
+    them would need those arguments, which ``args`` does not carry. Restoring
+    ``args`` and ``__dict__`` directly reproduces the exception exactly.
+
+    The three chain arguments carry defaults so that a payload pickled by an
+    earlier version -- which passed only ``cls``, ``args`` and ``state`` --
+    still loads. An exception can outlive an upgrade: it may sit in a task
+    queue, or be sent by a worker running the previous release.
+
+    ``__cause__``, ``__context__`` and ``__suppress_context__`` live outside
+    ``__dict__`` -- they are special exception state -- so they are restored
+    explicitly. Without this the chain is silently lost, and a traceback
+    rebuilt in another process no longer shows what actually failed.
+    """
+    exc = cls.__new__(cls)
+    Exception.__init__(exc, *args)
+    exc.__dict__.update(state)
+    exc.__cause__ = cause
+    exc.__context__ = context
+    exc.__suppress_context__ = suppress_context
+    return exc
+
+
+class DataExceptError(Exception):
+    """Base class for every operational exception DataExcept raises."""
+
+    #: Passed to redact_urls_in_text when scrubbing this class's message.
+    #: WebhookError sets it False, because a webhook URL's path *is* the
+    #: credential.
+    _keep_url_path = True
+
+    def __init__(self, *args: Any) -> None:
+        # One boundary for the whole hierarchy: whatever built the message --
+        # a constructor, a caller-supplied `message`, or the text of a wrapped
+        # exception quoting the original URL -- it is scrubbed here. Redacting
+        # only the structured argument leaves all three of those routes open.
+        keep_path = type(self)._keep_url_path
+        if args and isinstance(args[0], str):
+            args = (redact_urls_in_text(args[0], keep_path=keep_path),) + args[1:]
+        # Many classes also store the message on self.message and render *that*
+        # in __str__, so scrubbing args alone would leave the rendered form
+        # untouched. Subclasses set it before calling up, so it is here to fix.
+        stored = self.__dict__.get("message")
+        if isinstance(stored, str):
+            # Written straight into __dict__, symmetric with the read above:
+            # this rewrites state a subclass already stored, rather than the
+            # base declaring an attribute of its own.
+            self.__dict__["message"] = redact_urls_in_text(stored, keep_path=keep_path)
+        super().__init__(*args)
+        # Constructors that wrap another exception record it on an attribute.
+        # Mirroring it into __cause__ is what makes a traceback print the
+        # underlying failure, exactly as `raise ... from exc` would; assigning
+        # __cause__ also sets __suppress_context__, as `raise from` does.
+        for attribute in _CAUSE_ATTRIBUTES:
+            candidate = getattr(self, attribute, None)
+            if isinstance(candidate, BaseException):
+                self.__cause__ = candidate
+                break
+
+    def __reduce__(self) -> Tuple[Any, Tuple[Any, ...]]:
+        args = tuple(_safe(arg) for arg in self.args)
+        state = {key: _safe(value) for key, value in self.__dict__.items()}
+        return (
+            _rebuild,
+            (
+                type(self),
+                args,
+                state,
+                _safe_exception(self.__cause__),
+                _safe_exception(self.__context__),
+                self.__suppress_context__,
+            ),
+        )
+
+
+class UnpicklableCause(DataExceptError):
+    """Stands in for a cause that could not be serialized.
+
+    ``__cause__`` and ``__context__`` must be exceptions, so the placeholder
+    used for ordinary attributes will not do here. Dropping the chain instead
+    would silently lose the reason for the failure.
+    """

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/dataengineering_exceptions.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 from typing import Optional
 
 from .base import DataExceptError
+from .redaction import redact_if_url
 
 
 class DataEngineeringError(DataExceptError):
@@ -111,7 +112,7 @@ class MissingPartitionError(DataEngineeringError):
             message: Optional custom error message.
         """
         self.partition = partition
-        self.location = location
+        self.location = redact_if_url(location)
         default = f"Partition '{partition}' not found at {location}"
         super().__init__(message or default)
 

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/datascience_exceptions/ingestion.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 from typing import Any, Optional, Sequence
 
 from .._validation import is_number
+from ..redaction import redact_if_url
 from .base import DataScienceError
 
 
@@ -26,7 +27,7 @@ class DataLoadingError(DataScienceError):
             )
 
         message = f"Failed to load data from {source!r}: {original}"
-        self.source = source
+        self.source = redact_if_url(source)
         self.original = original
         super().__init__(message)
 

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/datascience_exceptions/operations.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 from typing import Any, Optional
 
 from .._validation import is_number
+from ..redaction import redact_if_url
 from .base import DataScienceError
 
 
@@ -26,7 +27,7 @@ class ModelSerializationError(DataScienceError):
             )
 
         message = f"Failed to serialize to {path!r}: {original}"
-        self.path = path
+        self.path = redact_if_url(path)
         self.original = original
         super().__init__(message)
 

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/exceptions/notification.py

@@ -41,9 +41,14 @@ class EmailError(NotificationError):
 class WebhookError(NotificationError):
     """Raised when a webhook POST fails."""
 
+    # Slack, Discord and others put the secret in the webhook path, so keeping
+    # the path would defeat the redaction. This also applies to the scrubbing
+    # of the whole message, which is how a wrapped HTTP exception quoting the
+    # original URL gets its path dropped too.
+    _keep_url_path = False
+
     def __init__(self, url: str, original_exception: Exception | None = None):
-        # Webhook URLs commonly authenticate through a query parameter.
-        self.url = redact_url(url)
+        self.url = redact_url(url, keep_path=False)
         # original_exception is set by NotificationError.__init__ below.
         msg = f"Webhook to URL '{self.url}' failed"
         if original_exception:

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/io_exceptions.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 from .base import DataExceptError
+from .redaction import redact_if_url
 
 
 class CustomIOError(DataExceptError):
@@ -21,7 +22,7 @@ class FileReadError(CustomIOError):
             path: File path that could not be read.
             original: Optional underlying exception.
         """
-        self.path = path
+        self.path = redact_if_url(path)
         self.original = original
         msg = f"Failed to read file '{path}'"
         if original:
@@ -39,7 +40,7 @@ class FileWriteError(CustomIOError):
             path: File path that could not be written to.
             original: Optional underlying exception.
         """
-        self.path = path
+        self.path = redact_if_url(path)
         self.original = original
         msg = f"Failed to write file '{path}'"
         if original:
@@ -56,7 +57,7 @@ class FileLockError(CustomIOError):
         Args:
             path: Path of the lock file.
         """
-        self.path = path
+        self.path = redact_if_url(path)
         super().__init__(f"Unable to obtain lock for '{path}'")
 
 

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/logging_helpers.py

@@ -5,8 +5,11 @@ from __future__ import annotations
 import contextlib
 import json
 import logging
+import traceback
 from typing import Any, Iterator, Mapping, Optional
 
+from .redaction import redact_urls_in_text
+
 Context = Mapping[str, Any]
 
 __all__ = [
@@ -37,6 +40,25 @@ def _build_extra(context: Context | None) -> dict[str, Any] | None:
     return {"dataexcept_context": serialized}
 
 
+def _chain_mentions_a_url(exc: BaseException) -> bool:
+    """True if *exc* or anything it chains to renders a URL.
+
+    A cheap pre-check: walking the chain and testing for "://" avoids
+    formatting a traceback for every exception that is logged.
+    """
+    seen: set[int] = set()
+    current: BaseException | None = exc
+    while current is not None and id(current) not in seen:
+        seen.add(id(current))
+        try:
+            if "://" in str(current):
+                return True
+        except Exception:  # pragma: no cover - a __str__ that itself raises
+            return True
+        current = current.__cause__ or current.__context__
+    return False
+
+
 def log_exception(
     exc: Exception,
     logger: Optional[logging.Logger] = None,
@@ -46,11 +68,29 @@ def log_exception(
     """Log *exc* at the given log *level* using *logger*.
 
     If *logger* is ``None`` a module level logger is used.
+
+    DataExcept redacts what it renders, but a wrapped third-party exception
+    renders itself: an HTTP client's error may quote the credential-bearing URL
+    it was called with, and ``exc_info`` makes logging print that whole chain.
+    When the chain contains a URL the traceback is formatted and scrubbed here;
+    otherwise the structured ``exc_info`` path is used unchanged, so ordinary
+    exceptions keep the shape log aggregators expect.
     """
     if logger is None:
         logger = logging.getLogger(__name__)
+    extra = _build_extra(context)
+
+    if _chain_mentions_a_url(exc):
+        formatted = "".join(
+            traceback.format_exception(type(exc), exc, exc.__traceback__)
+        )
+        keep_path = getattr(type(exc), "_keep_url_path", True)
+        scrubbed = redact_urls_in_text(formatted, keep_path=keep_path).rstrip()
+        logger.log(level, "%s\n%s", exc, scrubbed, extra=extra)
+        return
+
     exc_info = (type(exc), exc, exc.__traceback__)
-    logger.log(level, "%s", exc, exc_info=exc_info, extra=_build_extra(context))
+    logger.log(level, "%s", exc, exc_info=exc_info, extra=extra)
 
 
 @contextlib.contextmanager

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/pandas_exceptions.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 from typing import Optional, Sequence
 
 from .base import DataExceptError
+from .redaction import redact_if_url
 
 
 class PandasError(DataExceptError):
@@ -98,10 +99,11 @@ class MergeKeyError(PandasError):
     """
 
     def __init__(self, left_keys: Sequence[str], right_keys: Sequence[str]) -> None:
-        if not all(isinstance(k, str) for k in left_keys):
-            raise TypeError("left_keys must be a sequence of strings")
-        if not all(isinstance(k, str) for k in right_keys):
-            raise TypeError("right_keys must be a sequence of strings")
+        # A bare string is a sequence of strings, so "id" would silently become
+        # ['i', 'd']. Reject it, as DtypeMismatchError already does.
+        for name, keys in (("left_keys", left_keys), ("right_keys", right_keys)):
+            if isinstance(keys, str) or not all(isinstance(k, str) for k in keys):
+                raise TypeError(f"{name} must be a sequence of strings, not a string")
         self.left_keys = list(left_keys)
         self.right_keys = list(right_keys)
         msg = f"Failed to merge on keys {self.left_keys} and {self.right_keys}"
@@ -126,7 +128,7 @@ class PandasIOError(PandasError):
             raise TypeError(
                 f"original must be Exception, got {type(original).__name__}"
             )
-        self.path = path
+        self.path = redact_if_url(path)
         self.original = original
         msg = f"Pandas I/O operation failed on {path!r}: {original}"
         super().__init__(msg)

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/pipeline_exceptions.py

@@ -6,7 +6,7 @@ from typing import Any, Optional
 
 from ._deprecation import resolve_deprecated
 from .base import DataExceptError
-from .redaction import redact_url
+from .redaction import redact_if_url, redact_url
 
 
 class PipelineError(DataExceptError):
@@ -45,7 +45,7 @@ class StorageError(PipelineError):
         message: Optional[str] = None,
     ) -> None:
         default = f"Storage {operation} failed at location: '{location}'."
-        self.location = location
+        self.location = redact_if_url(location)
         self.operation = operation
         super().__init__(message or default)
 
@@ -187,7 +187,7 @@ class DataFetchError(PipelineError):
         message: Optional[str] = None,
     ) -> None:
         default = f"Failed to fetch '{source}' data for cid={cid}"
-        self.source = source
+        self.source = redact_if_url(source)
         self.cid = cid
         super().__init__(message or default)
 

dataexcept-0.4.1/dataexcept/redaction.py

@@ -0,0 +1,201 @@
+"""Redaction helpers for values that must not reach a log.
+
+Several exceptions here are raised with credentials in hand: an authentication
+token, a database URL carrying a password, a webhook URL whose *path* is the
+secret. Those values end up in the exception message, and
+:func:`dataexcept.logging_helpers.log_exception` logs ``str(exc)``, so without
+redaction a failed delivery writes the credential to the log.
+
+The aim is to keep an error debuggable while giving up the secret. A redacted
+value carries a short, non-reversible fingerprint, so repeated failures of the
+*same* credential stay recognisable in a log without the credential appearing
+in it.
+
+What this can and cannot do is stated in ``SECURITY.md``. In short: values the
+library is *given* as credentials are redacted, and URLs are redacted wherever
+they appear -- including inside a message you supplied and inside the text of a
+wrapped exception. A bare, non-URL secret pasted into free-form text cannot be
+recognised and is not redacted.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import re
+from typing import Optional
+from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
+
+__all__ = [
+    "fingerprint",
+    "redact_if_url",
+    "redact_secret",
+    "redact_url",
+    "redact_urls_in_text",
+    "remove_secret",
+]
+
+PLACEHOLDER = "***"
+
+#: Below this length a "secret" is not removed from free text. Substring
+#: replacement of a short value corrupts ordinary words -- removing "tok" from
+#: "Invalid authentication token" mangles the message and tells a reader
+#: nothing. The structured field is redacted regardless of length.
+MIN_REMOVABLE_SECRET_LENGTH = 8
+
+#: Substrings that mark a query or fragment parameter as carrying a secret.
+#: Matched anywhere in the parameter name, case insensitively, so
+#: ``X-Amz-Signature``, ``auth_token`` and ``refresh_token`` are all covered.
+#: Over-matching here is harmless; under-matching leaks.
+SENSITIVE_PARAM_MARKERS = frozenset(
+    {
+        "auth",
+        "credential",
+        "key",
+        "passwd",
+        "password",
+        "pwd",
+        "secret",
+        "session",
+        "sig",
+        "token",
+    }
+)
+
+#: Finds URLs inside free-form text, so a credential cannot slip through in a
+#: caller-supplied message or in the text of a wrapped exception.
+_URL_IN_TEXT = re.compile(r"\b[a-zA-Z][a-zA-Z0-9+.\-]*://[^\s'\"<>,;)\]}]+")
+
+
+def _is_sensitive(name: str) -> bool:
+    lowered = name.lower()
+    return any(marker in lowered for marker in SENSITIVE_PARAM_MARKERS)
+
+
+def fingerprint(value: str) -> str:
+    """Return a short, one-way fingerprint of *value*.
+
+    Enough to tell "the same bad token again" from "a different bad token",
+    and not enough to recover the token.
+    """
+    digest = hashlib.sha256(value.encode("utf-8", "replace")).hexdigest()
+    return digest[:8]
+
+
+def redact_secret(value: Optional[str]) -> Optional[str]:
+    """Replace a secret with a placeholder and its fingerprint."""
+    if value is None:
+        return None
+    if not value:
+        return PLACEHOLDER
+    return f"{PLACEHOLDER}({fingerprint(value)})"
+
+
+def remove_secret(text: str, secret: Optional[str]) -> str:
+    """Replace every occurrence of a known *secret* in *text*.
+
+    Used where the library was handed the secret explicitly, so it can be
+    removed even from a message the caller wrote themselves.
+    """
+    if not secret or not text or len(secret) < MIN_REMOVABLE_SECRET_LENGTH:
+        return text
+    return text.replace(secret, f"{PLACEHOLDER}({fingerprint(secret)})")
+
+
+def _redact_params(query: str) -> tuple[str, bool]:
+    if not query:
+        return query, False
+    pairs = parse_qsl(query, keep_blank_values=True)
+    if not any(_is_sensitive(key) for key, _ in pairs):
+        return query, False
+    return (
+        urlencode(
+            [
+                (key, PLACEHOLDER if _is_sensitive(key) else value)
+                for key, value in pairs
+            ],
+            # Keep the placeholder legible rather than percent-encoded.
+            safe="*",
+        ),
+        True,
+    )
+
+
+def redact_url(url: Optional[str], *, keep_path: bool = True) -> Optional[str]:
+    """Strip credentials from *url*.
+
+    Scheme, host and port are always kept: those are what make an error
+    actionable. Userinfo, sensitive query parameters and sensitive fragment
+    parameters are always removed.
+
+    Pass ``keep_path=False`` where the path itself is the credential. An
+    incoming webhook URL is the common case -- Slack, Discord and others put
+    the secret in the path, so preserving it would defeat the point.
+    """
+    if not url:
+        return url
+
+    try:
+        parts = urlsplit(url)
+    except ValueError:  # pragma: no cover - urlsplit is extremely permissive
+        return PLACEHOLDER
+
+    if not parts.scheme or not parts.netloc:
+        # Not a URL with a host; a bare path or plain string is returned
+        # untouched rather than mangled.
+        return url
+
+    redacted = False
+
+    netloc = parts.netloc
+    if "@" in netloc:
+        _, _, host = netloc.rpartition("@")
+        netloc = f"{PLACEHOLDER}:{PLACEHOLDER}@{host}"
+        redacted = True
+
+    query, query_redacted = _redact_params(parts.query)
+    redacted = redacted or query_redacted
+
+    fragment = parts.fragment
+    if "=" in fragment:
+        # OAuth implicit flow returns the token in the fragment.
+        fragment, fragment_redacted = _redact_params(fragment)
+        redacted = redacted or fragment_redacted
+
+    path = parts.path
+    if not keep_path and path.strip("/"):
+        path = f"/{PLACEHOLDER}"
+        redacted = True
+
+    if not redacted:
+        # Hand back exactly what was passed in. Rebuilding would normalise it,
+        # and "sqlite://" loses its slashes on the way through.
+        return url
+
+    return urlunsplit((parts.scheme, netloc, path, query, fragment))
+
+
+def redact_if_url(value: Optional[str], *, keep_path: bool = True) -> Optional[str]:
+    """Redact *value* only if it is a URL, leaving file paths untouched.
+
+    Fields such as ``DataLoadingError.source`` document themselves as "file
+    path or URL", so they cannot be redacted unconditionally without mangling
+    ordinary paths.
+    """
+    if not isinstance(value, str) or "://" not in value:
+        return value
+    return redact_url(value, keep_path=keep_path)
+
+
+def redact_urls_in_text(text: str, *, keep_path: bool = True) -> str:
+    """Redact every URL found in free-form *text*.
+
+    This is the boundary that stops a secret being reintroduced after the
+    structured argument was redacted -- through a caller-supplied ``message``,
+    or through the text of a wrapped exception that quotes the original URL.
+    """
+    if not text or "://" not in text:
+        return text
+    return _URL_IN_TEXT.sub(
+        lambda match: redact_url(match.group(0), keep_path=keep_path) or "",
+        text,
+    )

{dataexcept-0.4.0 → dataexcept-0.4.1}/dataexcept/security_exceptions.py

@@ -3,7 +3,7 @@
 from __future__ import annotations
 
 from .base import DataExceptError
-from .redaction import redact_secret
+from .redaction import redact_secret, remove_secret
 
 
 class SecurityError(DataExceptError):
@@ -62,7 +62,9 @@ class InvalidTokenError(SecurityError):
         default = "Invalid authentication token"
         if token:
             default += f": {self.token}"
-        super().__init__(message or default)
+        # The library was handed the secret, so it can be removed even from a
+        # message the caller wrote themselves.
+        super().__init__(remove_secret(message or default, token))
 
 
 __all__ = [

{dataexcept-0.4.0 → dataexcept-0.4.1}/pyproject.toml

@@ -1,11 +1,11 @@
 [project]
 name = "DataExcept"
-version = "0.4.0"
+version = "0.4.1"
 description = "A Python package providing structured, easily-extendable custom exception types."
 readme = "README.md"
 license = "MIT"
 license-files = ["LICENSE"]
-requires-python = ">=3.10,<3.14"
+requires-python = ">=3.10,<3.15"
 authors = [{ name = "Diogo Ribeiro", email = "dfr@esmad.ipp.pt" }]
 maintainers = [{ name = "Diogo Ribeiro", email = "diogo.debastos.ribeiro@gmail.com" }]
 keywords = ["exceptions", "errors", "logging"]
@@ -16,6 +16,7 @@ classifiers = [
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
     "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
 ]
 dependencies = [
     "tomli ; python_version < '3.11'",
@@ -66,6 +67,10 @@ profile = "black"
 
 [tool.black]
 line-length = 88
+# Pinned rather than inferred from requires-python: inference picks the highest
+# supported version, and black then refuses to verify its own output when run
+# on an older interpreter -- which is what CI does.
+target-version = ["py310"]
 
 [tool.ruff]
 line-length = 88

dataexcept-0.4.0/dataexcept/base.py

@@ -1,67 +0,0 @@
-"""The root of the DataExcept exception hierarchy.
-
-Every exception this package raises derives from :class:`DataExceptError`, so a
-caller can catch everything the library can raise with a single clause while
-still catching narrowly where it matters::
-
-    try:
-        run_pipeline()
-    except ValidationError:
-        ...              # exactly this failure
-    except DataExceptError:
-        ...              # anything else DataExcept raised
-
-The base also gives the whole hierarchy a working serialization contract. Many
-of these exceptions take several constructor arguments while ``Exception.args``
-holds only the rendered message, so the default pickling protocol -- which
-replays ``args`` through ``__init__`` -- could not rebuild them. That made them
-unusable across a process boundary, which is where a data pipeline most needs
-them.
-"""
-
-from __future__ import annotations
-
-from typing import Any, Dict, Tuple, Type
-
-__all__ = ["DataExceptError"]
-
-#: Attribute names used across the package to hold the exception that caused
-#: this one. Checked in order; the first that holds an exception wins.
-_CAUSE_ATTRIBUTES = ("original", "original_exception", "cause")
-
-
-def _rebuild(
-    cls: Type["DataExceptError"], args: Tuple[Any, ...], state: Dict[str, Any]
-) -> "DataExceptError":
-    """Recreate *cls* without replaying its ``__init__``.
-
-    Constructors here validate and render a message from their arguments;
-    replaying them would need the original arguments, which ``args`` does not
-    carry. Restoring ``args`` and ``__dict__` directly reproduces the exception
-    exactly, including its message and every attribute it recorded.
-    """
-    exc = cls.__new__(cls)
-    Exception.__init__(exc, *args)
-    exc.__dict__.update(state)
-    return exc
-
-
-class DataExceptError(Exception):
-    """Base class for every exception DataExcept raises."""
-
-    def __init__(self, *args: Any) -> None:
-        super().__init__(*args)
-        # Constructors that wrap another exception record it on an attribute.
-        # Mirroring it into __cause__ is what makes a traceback print the
-        # underlying failure, exactly as `raise ... from exc` would; assigning
-        # __cause__ also sets __suppress_context__, as `raise from` does.
-        for attribute in _CAUSE_ATTRIBUTES:
-            candidate = getattr(self, attribute, None)
-            if isinstance(candidate, BaseException):
-                self.__cause__ = candidate
-                break
-
-    def __reduce__(
-        self,
-    ) -> Tuple[Any, Tuple[Type["DataExceptError"], Tuple[Any, ...], Dict[str, Any]]]:
-        return (_rebuild, (type(self), self.args, self.__dict__.copy()))

dataexcept-0.4.0/dataexcept/redaction.py

@@ -1,114 +0,0 @@
-"""Redaction helpers for values that must not reach a log.
-
-Several exceptions here are raised with credentials in hand: an authentication
-token, a database URL carrying a password, a webhook URL with a signing
-parameter. Those values end up in the exception message, and
-:func:`dataexcept.logging_helpers.log_exception` logs ``str(exc)``, so without
-redaction a failed connection writes the password to the log.
-
-The aim is to keep an error debuggable while giving up the secret. A redacted
-value carries a short, non-reversible fingerprint, so repeated failures of the
-*same* credential are still recognisable in a log without the credential
-appearing in it.
-"""
-
-from __future__ import annotations
-
-import hashlib
-from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
-
-__all__ = ["fingerprint", "redact_secret", "redact_url"]
-
-PLACEHOLDER = "***"
-
-#: Query parameters whose value is treated as a secret. Matched case
-#: insensitively against the whole parameter name.
-SENSITIVE_QUERY_PARAMS = frozenset(
-    {
-        "access_token",
-        "api_key",
-        "apikey",
-        "auth",
-        "credential",
-        "key",
-        "password",
-        "private_key",
-        "pwd",
-        "refresh_token",
-        "secret",
-        "session",
-        "sig",
-        "signature",
-        "token",
-    }
-)
-
-
-def fingerprint(value: str) -> str:
-    """Return a short, one-way fingerprint of *value*.
-
-    Enough to tell "the same bad token again" from "a different bad token",
-    and not enough to recover the token.
-    """
-    digest = hashlib.sha256(value.encode("utf-8", "replace")).hexdigest()
-    return digest[:8]
-
-
-def redact_secret(value: str | None) -> str | None:
-    """Replace a secret with a placeholder and its fingerprint."""
-    if value is None:
-        return None
-    if not value:
-        return PLACEHOLDER
-    return f"{PLACEHOLDER}({fingerprint(value)})"
-
-
-def redact_url(url: str | None) -> str | None:
-    """Strip credentials and sensitive query parameters from *url*.
-
-    The scheme, host, port and path are kept, because those are what make the
-    error actionable. Anything that authenticates is replaced.
-    """
-    if not url:
-        return url
-
-    try:
-        parts = urlsplit(url)
-    except ValueError:  # pragma: no cover - urlsplit is extremely permissive
-        return PLACEHOLDER
-
-    if not parts.scheme and not parts.netloc:
-        # Not a URL at all; treat the whole thing as sensitive rather than
-        # returning it unchanged.
-        return url
-
-    netloc = parts.netloc
-    redacted = False
-    if "@" in netloc:
-        _, _, host = netloc.rpartition("@")
-        netloc = f"{PLACEHOLDER}:{PLACEHOLDER}@{host}"
-        redacted = True
-
-    query = parts.query
-    if query:
-        pairs = parse_qsl(query, keep_blank_values=True)
-        if any(key.lower() in SENSITIVE_QUERY_PARAMS for key, _ in pairs):
-            query = urlencode(
-                [
-                    (
-                        key,
-                        PLACEHOLDER if key.lower() in SENSITIVE_QUERY_PARAMS else value,
-                    )
-                    for key, value in pairs
-                ],
-                # Keep the placeholder legible rather than percent-encoded.
-                safe="*",
-            )
-            redacted = True
-
-    if not redacted:
-        # Nothing sensitive, so hand back exactly what was passed in.
-        # Rebuilding would normalise it -- "sqlite://" loses its slashes.
-        return url
-
-    return urlunsplit((parts.scheme, netloc, parts.path, query, parts.fragment))