DataExcept 0.3.0__tar.gz → 0.4.0__tar.gz

{dataexcept-0.3.0 → dataexcept-0.4.0}/CHANGELOG.md

@@ -7,6 +7,106 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-08-24
+
+### Added
+
+- **`DataExceptError`, the root of the hierarchy.** Every exception the package
+  raises now derives from it, so one clause catches the whole library:
+  ```python
+  except DataExceptError:
+      ...
+  ```
+  The nine domain roots (`JobError`, `DataScienceError`, `PipelineError` and
+  the rest) sit beneath it and still catch only their own domain, so granular
+  handling is unchanged.
+
+### Changed
+
+- `SECURITY.md` claimed 0.1.x was the supported version, and `CHECKLIST.md`
+  still described an uncommitted lockfile, 30 of 96 top-level exports, 109 tests
+  and 85% coverage. Both now match the project.
+- **Coverage is now measured honestly, and gated.** `coverage run -m pytest`
+  had no `source` setting, so it counted the tests and examples in the
+  denominator — and test files are by definition fully executed. The reported
+  figure was inflated: 86% when the package alone was at 79%.
+  `[tool.coverage.run]` now restricts measurement to `dataexcept` and enables
+  branch coverage, and `fail_under = 91` stops it regressing. The honest number
+  is **92%**; the README, checklist and roadmap now quote that rather than the
+  inflated one.
+- The CLI is exercised in-process as well as through a subprocess. The
+  subprocess tests verify real invocation but coverage cannot see inside them,
+  which left `__main__.py` reporting 23% despite being tested. It now reports
+  93%.
+
+### Fixed
+
+- **NumPy scalars are accepted where a number is expected.** Eleven validations
+  used `isinstance(value, (int, float))`, which rejects `numpy.float32` and
+  `numpy.int64` — a poor answer from a library aimed at data science. They now
+  use `numbers.Real`, which NumPy registers its scalar types with, so this needs
+  no dependency on NumPy. Arrays and non-numbers are still rejected.
+- **A wrapped exception is now chained.** Constructors that take an underlying
+  exception recorded it on an attribute but never set `__cause__`, so a
+  traceback did not show what actually failed. `DataExceptError` now mirrors it,
+  and Python prints "The above exception was the direct cause of the following
+  exception" as if `raise ... from` had been used.
+- `from dataexcept import *` failed under `-W error::DeprecationWarning`,
+  because `__all__` listed the deprecated `job_exceptions` module. It is no
+  longer advertised there; it remains importable until 1.0.0.
+- A redundant `global` declaration in `examples/lambda_main.py` raised three
+  `F824` warnings. CI linted only `dataexcept` and `tests` with flake8 while the
+  formatters covered `examples` and `scripts`; flake8 now covers all four.
+- **Four exceptions discarded the reason for the failure.**
+  `DataLoadingError`, `MissingDataError`, `ModelSerializationError` and
+  `DeploymentError` rendered only their identifying attribute —
+  `[DataLoadingError] orders.csv` — while "invalid utf-8", "disk full" or
+  "permission denied" sat unseen in `args[0]`. Since logging uses `str(exc)`,
+  the part a reader needs never reached the log. All four now render the full
+  message, and a test asserts no class drops it.
+- **Exceptions can now cross a process boundary.** They could not be pickled:
+  most constructors take several arguments while `Exception.args` holds only
+  the rendered message, and the default protocol replays `args` through
+  `__init__`. Of 98 classes, 39 raised `TypeError` on unpickling and a further
+  47 came back with different state — only 10 round-tripped exactly. Raising
+  one inside a `ProcessPoolExecutor` killed the pool with `BrokenProcessPool`.
+  `DataExceptError.__reduce__` restores `args` and `__dict__` directly instead
+  of replaying `__init__`. All 97 constructible classes now round-trip with
+  identical type, message and attributes, covered by a test per class plus a
+  real process-pool test.
+- The stability policy and README both claimed `except JobError:` catches
+  "anything else this library raises". It did not — there were nine
+  disconnected trees under `Exception`, so `JobError` caught neither
+  `ModelTrainingError` nor `PipelineError` nor `DatabaseError`. The claim is
+  now true of `DataExceptError`, and the docs say which base covers what.
+
+### Security
+
+- **A release now has to prove where it came from.** The release workflow
+  checked only that the tag text matched `pyproject.toml`, so a tag pushed to
+  an unreviewed branch could reach the PyPI publishing job. It now refuses to
+  build unless the tagged commit is reachable from `main` and the same checks
+  branch protection requires are green on that exact commit.
+- The built wheel is tested before it is published. A new `verify-wheel` job
+  installs the artifact, deletes the source package so nothing can import it by
+  accident, and runs the whole suite against what will actually be uploaded.
+  `scripts/check_wheel.py` then asserts the distribution ships `py.typed` and a
+  complete `__all__` — a file can be present in the repository and missing from
+  the artifact.
+- **Credentials are no longer written into exception messages.** `log_exception`
+  logs `str(exc)`, so a failed connection put the database password in the log.
+  `InvalidTokenError` embedded the whole token; `DatabaseConnectionError` the
+  whole connection URL including username and password; `WebhookError` and
+  `ApiError` the URL including any signing or key parameter.
+  These are now redacted before being stored or rendered, so the raw value is
+  absent from the message, the attributes and a pickle of the exception. A
+  secret renders as `***(1a2b3c4d)` — a truncated SHA-256, so the same bad
+  credential failing repeatedly stays correlatable in a log without appearing
+  in it. Host, port and path survive, because those are what make the error
+  actionable.
+  `QueryExecutionError` still embeds the SQL it is given; SECURITY.md now says
+  so explicitly rather than leaving it to be discovered.
+
 ## [0.3.0] - 2026-08-24
 
 ### Added
@@ -168,7 +268,8 @@ First public release.
 - Published to PyPI via OIDC trusted publishing; no long-lived API token is
   involved in a release.
 
-[Unreleased]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.3.0...HEAD
+[Unreleased]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.4.0...HEAD
+[0.4.0]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.2.0...v0.2.1
 [0.2.0]: https://github.com/DiogoRibeiro7/DataExcept/compare/v0.1.0...v0.2.0

{dataexcept-0.3.0 → dataexcept-0.4.0}/CITATION.cff

@@ -1,7 +1,7 @@
 cff-version: 1.2.0
 message: "If you use this software, please cite it using the following metadata."
 title: "DataExcept"
-version: "0.3.0"
+version: "0.4.0"
 authors:
   - family-names: "Ribeiro"
     given-names: "Diogo"

{dataexcept-0.3.0 → dataexcept-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: DataExcept
-Version: 0.3.0
+Version: 0.4.0
 Summary: A Python package providing structured, easily-extendable custom exception types.
 License-Expression: MIT
 License-File: LICENSE
@@ -41,13 +41,13 @@ Description-Content-Type: text/markdown
 
 ## 🎯 Key Features
 
-- **🏗️ Hierarchical Structure**: Catch specific errors or broad categories
+- **🏗️ Hierarchical Structure**: Catch one specific error, a whole domain, or everything via `DataExceptError`
 - **📦 One Import**: Every exception is available from `dataexcept` directly, or from its domain module — same objects either way
-- **📊 Data Science Focused**: 98 exception classes covering ML pipelines, feature engineering, model training
-- **🔧 Production Ready**: Comprehensive logging helpers and error context
+- **📊 Data Science Focused**: 99 exception classes covering ML pipelines, feature engineering, model training
+- **🔧 Production Ready**: Logging helpers, error context, and exceptions that survive a process boundary intact
 - **📚 Academic Quality**: Proper documentation, type hints, and citation support
 - **🐍 Python 3.10+**: Modern Python with full type safety
-- **🧪 Well Tested**: Broad test suite with comprehensive edge case handling (see the coverage badge above)
+- **🧪 Well Tested**: 690+ tests at 92% branch coverage of the package, gated in CI
 
 ## 📦 Quick Installation
 
@@ -105,8 +105,7 @@ def load_dataset(file_path: str) -> pd.DataFrame:
 ### Exception Hierarchies
 
 ```python
-from dataexcept import JobError
-from dataexcept.datascience_exceptions import ModelTrainingError, ConvergenceError
+from dataexcept import ConvergenceError, DataExceptError, ModelTrainingError
 
 try:
     # Your ML pipeline
@@ -119,8 +118,8 @@ except ModelTrainingError:
     # Handle any training-related error
     logger.error("Training failed, falling back to simpler model")
     train_simple_model()
-except JobError:
-    # Handle any job-related error
+except DataExceptError:
+    # Handle anything else DataExcept raised
     logger.error("Job failed, notifying administrators")
     send_alert()
 ```
@@ -214,7 +213,7 @@ except Exception as exc:
 ### Command Line Interface
 
 ```bash
-# List every exception class the package exports (98 of them, alphabetically)
+# List every exception class the package exports (99 of them, alphabetically)
 $ dataexcept list
 ApiError
 AuthenticationError
@@ -225,7 +224,7 @@ BiasDetectionError
 
 # Check version
 $ dataexcept --version
-dataexcept 0.3.0
+dataexcept 0.4.0
 ```
 
 ## 🎯 Use Cases
@@ -375,7 +374,7 @@ If you use DataExcept in your research, please cite it:
   author = {Ribeiro, Diogo},
   title = {DataExcept: Structured Exception Handling for Data Science},
   url = {https://github.com/DiogoRibeiro7/DataExcept},
-  version = {0.3.0},
+  version = {0.4.0},
   year = {2026},
   publisher = {GitHub}
 }

{dataexcept-0.3.0 → dataexcept-0.4.0}/README.md

@@ -15,13 +15,13 @@
 
 ## 🎯 Key Features
 
-- **🏗️ Hierarchical Structure**: Catch specific errors or broad categories
+- **🏗️ Hierarchical Structure**: Catch one specific error, a whole domain, or everything via `DataExceptError`
 - **📦 One Import**: Every exception is available from `dataexcept` directly, or from its domain module — same objects either way
-- **📊 Data Science Focused**: 98 exception classes covering ML pipelines, feature engineering, model training
-- **🔧 Production Ready**: Comprehensive logging helpers and error context
+- **📊 Data Science Focused**: 99 exception classes covering ML pipelines, feature engineering, model training
+- **🔧 Production Ready**: Logging helpers, error context, and exceptions that survive a process boundary intact
 - **📚 Academic Quality**: Proper documentation, type hints, and citation support
 - **🐍 Python 3.10+**: Modern Python with full type safety
-- **🧪 Well Tested**: Broad test suite with comprehensive edge case handling (see the coverage badge above)
+- **🧪 Well Tested**: 690+ tests at 92% branch coverage of the package, gated in CI
 
 ## 📦 Quick Installation
 
@@ -79,8 +79,7 @@ def load_dataset(file_path: str) -> pd.DataFrame:
 ### Exception Hierarchies
 
 ```python
-from dataexcept import JobError
-from dataexcept.datascience_exceptions import ModelTrainingError, ConvergenceError
+from dataexcept import ConvergenceError, DataExceptError, ModelTrainingError
 
 try:
     # Your ML pipeline
@@ -93,8 +92,8 @@ except ModelTrainingError:
     # Handle any training-related error
     logger.error("Training failed, falling back to simpler model")
     train_simple_model()
-except JobError:
-    # Handle any job-related error
+except DataExceptError:
+    # Handle anything else DataExcept raised
     logger.error("Job failed, notifying administrators")
     send_alert()
 ```
@@ -188,7 +187,7 @@ except Exception as exc:
 ### Command Line Interface
 
 ```bash
-# List every exception class the package exports (98 of them, alphabetically)
+# List every exception class the package exports (99 of them, alphabetically)
 $ dataexcept list
 ApiError
 AuthenticationError
@@ -199,7 +198,7 @@ BiasDetectionError
 
 # Check version
 $ dataexcept --version
-dataexcept 0.3.0
+dataexcept 0.4.0
 ```
 
 ## 🎯 Use Cases
@@ -349,7 +348,7 @@ If you use DataExcept in your research, please cite it:
   author = {Ribeiro, Diogo},
   title = {DataExcept: Structured Exception Handling for Data Science},
   url = {https://github.com/DiogoRibeiro7/DataExcept},
-  version = {0.3.0},
+  version = {0.4.0},
   year = {2026},
   publisher = {GitHub}
 }

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/__init__.py

@@ -4,6 +4,12 @@ Every exception the package defines is importable straight from here::
 
     from dataexcept import ValidationError, ModelTrainingError
 
+They all derive from :class:`DataExceptError`, so one clause catches anything
+this package raises::
+
+    except DataExceptError:
+        ...
+
 The domain modules (``datascience_exceptions``, ``pipeline_exceptions`` and so
 on) remain importable and export the same objects, so both spellings work and
 refer to the same classes.
@@ -32,6 +38,7 @@ from . import (  # noqa: F401
     security_exceptions,
 )
 from ._deprecation import resolve_deprecated
+from .base import DataExceptError
 from .database_exceptions import (
     DatabaseConnectionError,
     DatabaseError,
@@ -156,6 +163,8 @@ from .security_exceptions import (
 )
 
 __all__ = [
+    # The root of the hierarchy: catches anything this package raises.
+    "DataExceptError",
     # Every exception class the package defines.
     "ApiError",
     "AuthenticationError",
@@ -261,12 +270,14 @@ __all__ = [
     "log_exception",
     "log_then_raise",
     # Domain modules, for callers who prefer a qualified import.
+    # job_exceptions is deliberately absent: it is deprecated, and listing it
+    # makes `from dataexcept import *` emit a DeprecationWarning, which turns
+    # into an error under -W error::DeprecationWarning. It stays importable.
     "database_exceptions",
     "dataengineering_exceptions",
     "datascience_exceptions",
     "exceptions",
     "io_exceptions",
-    "job_exceptions",
     "logging_helpers",
     "network_exceptions",
     "pandas_exceptions",

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/__main__.py

@@ -21,11 +21,17 @@ _DEPRECATED_MODULES = frozenset({"dataexcept.job_exceptions"})
 def _iter_exception_modules() -> Iterable[ModuleType]:
     """Yield every non-deprecated submodule that explicitly defines ``__all__``."""
     allowed_suffixes = ("exceptions", "_exceptions")
+    # dataexcept.base holds DataExceptError, the root of the hierarchy, and
+    # does not match the suffix rule.
+    always_include = {"dataexcept.base"}
 
     for module_info in pkgutil.walk_packages(
         _PKG_PATH, prefix="dataexcept.", onerror=lambda name: None
     ):
-        if not module_info.name.endswith(allowed_suffixes):
+        if (
+            not module_info.name.endswith(allowed_suffixes)
+            and module_info.name not in always_include
+        ):
             continue
         if module_info.name in _DEPRECATED_MODULES:
             continue

dataexcept-0.4.0/dataexcept/_validation.py

@@ -0,0 +1,22 @@
+"""Small runtime checks shared by the exception constructors."""
+
+from __future__ import annotations
+
+import numbers
+
+__all__ = ["is_number"]
+
+
+def is_number(value: object) -> bool:
+    """Return True for any real number, including NumPy scalars.
+
+    ``isinstance(value, (int, float))`` rejects ``numpy.float32`` and
+    ``numpy.int64``, which is a poor answer from a library aimed at data
+    science. ``numbers.Real`` accepts them because NumPy registers its scalar
+    types with the ABC, and it needs no dependency on NumPy to do so.
+
+    This is a function rather than an inline ``isinstance`` because narrowing a
+    value to ``numbers.Real`` defeats mypy's inference for the rest of the
+    enclosing class.
+    """
+    return isinstance(value, numbers.Real)

dataexcept-0.4.0/dataexcept/base.py

@@ -0,0 +1,67 @@
+"""The root of the DataExcept exception hierarchy.
+
+Every exception this package raises derives from :class:`DataExceptError`, so a
+caller can catch everything the library can raise with a single clause while
+still catching narrowly where it matters::
+
+    try:
+        run_pipeline()
+    except ValidationError:
+        ...              # exactly this failure
+    except DataExceptError:
+        ...              # anything else DataExcept raised
+
+The base also gives the whole hierarchy a working serialization contract. Many
+of these exceptions take several constructor arguments while ``Exception.args``
+holds only the rendered message, so the default pickling protocol -- which
+replays ``args`` through ``__init__`` -- could not rebuild them. That made them
+unusable across a process boundary, which is where a data pipeline most needs
+them.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Tuple, Type
+
+__all__ = ["DataExceptError"]
+
+#: Attribute names used across the package to hold the exception that caused
+#: this one. Checked in order; the first that holds an exception wins.
+_CAUSE_ATTRIBUTES = ("original", "original_exception", "cause")
+
+
+def _rebuild(
+    cls: Type["DataExceptError"], args: Tuple[Any, ...], state: Dict[str, Any]
+) -> "DataExceptError":
+    """Recreate *cls* without replaying its ``__init__``.
+
+    Constructors here validate and render a message from their arguments;
+    replaying them would need the original arguments, which ``args`` does not
+    carry. Restoring ``args`` and ``__dict__` directly reproduces the exception
+    exactly, including its message and every attribute it recorded.
+    """
+    exc = cls.__new__(cls)
+    Exception.__init__(exc, *args)
+    exc.__dict__.update(state)
+    return exc
+
+
+class DataExceptError(Exception):
+    """Base class for every exception DataExcept raises."""
+
+    def __init__(self, *args: Any) -> None:
+        super().__init__(*args)
+        # Constructors that wrap another exception record it on an attribute.
+        # Mirroring it into __cause__ is what makes a traceback print the
+        # underlying failure, exactly as `raise ... from exc` would; assigning
+        # __cause__ also sets __suppress_context__, as `raise from` does.
+        for attribute in _CAUSE_ATTRIBUTES:
+            candidate = getattr(self, attribute, None)
+            if isinstance(candidate, BaseException):
+                self.__cause__ = candidate
+                break
+
+    def __reduce__(
+        self,
+    ) -> Tuple[Any, Tuple[Type["DataExceptError"], Tuple[Any, ...], Dict[str, Any]]]:
+        return (_rebuild, (type(self), self.args, self.__dict__.copy()))

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/database_exceptions.py

@@ -2,8 +2,11 @@
 
 from __future__ import annotations
 
+from .base import DataExceptError
+from .redaction import redact_url
 
-class DatabaseError(Exception):
+
+class DatabaseError(DataExceptError):
     """Base exception for database-related errors."""
 
     pass
@@ -19,8 +22,9 @@ class DatabaseConnectionError(DatabaseError):
             db_url: Database connection URL.
             message: Optional custom error message.
         """
-        self.db_url = db_url
-        default = f"Failed to connect to database at '{db_url}'"
+        # A connection URL routinely carries a username and password.
+        self.db_url = redact_url(db_url)
+        default = f"Failed to connect to database at '{self.db_url}'"
         super().__init__(message or default)
 
 

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/dataengineering_exceptions.py

@@ -4,8 +4,10 @@ from __future__ import annotations
 
 from typing import Optional
 
+from .base import DataExceptError
 
-class DataEngineeringError(Exception):
+
+class DataEngineeringError(DataExceptError):
     """Base exception for data engineering errors."""
 
     pass

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/datascience_exceptions/base.py

@@ -2,8 +2,10 @@
 
 from __future__ import annotations
 
+from ..base import DataExceptError
 
-class DataScienceError(Exception):
+
+class DataScienceError(DataExceptError):
     """Base exception for data science errors."""
 
     def __init__(self, message: str) -> None:

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/datascience_exceptions/ingestion.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 from typing import Any, Optional, Sequence
 
+from .._validation import is_number
 from .base import DataScienceError
 
 
@@ -30,7 +31,7 @@ class DataLoadingError(DataScienceError):
         super().__init__(message)
 
     def __str__(self) -> str:
-        return f"[DataLoadingError] {self.source}"
+        return f"[DataLoadingError:{self.source}] {self.message}"
 
 
 class DataFormatError(DataScienceError):
@@ -105,7 +106,7 @@ class MissingDataError(DataScienceError):
         super().__init__(message)
 
     def __str__(self) -> str:
-        return f"[MissingDataError] {self.feature}"
+        return f"[MissingDataError:{self.feature}] {self.message}"
 
 
 class OutlierDetectionError(DataScienceError):
@@ -227,9 +228,9 @@ class DataImbalanceError(DataScienceError):
     def __init__(
         self, ratio: float, threshold: float, message: Optional[str] = None
     ) -> None:
-        if not isinstance(ratio, (int, float)):
+        if not is_number(ratio):
             raise TypeError(f"ratio must be numeric, got {type(ratio).__name__}")
-        if not isinstance(threshold, (int, float)):
+        if not is_number(threshold):
             raise TypeError(
                 f"threshold must be numeric, got {type(threshold).__name__}"
             )

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/datascience_exceptions/operations.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 from typing import Any, Optional
 
+from .._validation import is_number
 from .base import DataScienceError
 
 
@@ -30,7 +31,7 @@ class ModelSerializationError(DataScienceError):
         super().__init__(message)
 
     def __str__(self) -> str:
-        return f"[ModelSerializationError] {self.path}"
+        return f"[ModelSerializationError:{self.path}] {self.message}"
 
 
 class DeploymentError(DataScienceError):
@@ -57,7 +58,7 @@ class DeploymentError(DataScienceError):
         super().__init__(msg)
 
     def __str__(self) -> str:
-        return f"[DeploymentError] {self.target}"
+        return f"[DeploymentError:{self.target}] {self.message}"
 
 
 class DataDriftError(DataScienceError):
@@ -74,7 +75,7 @@ class DataDriftError(DataScienceError):
     ) -> None:
         if not isinstance(feature, str):
             raise TypeError(f"feature must be str, got {type(feature).__name__}")
-        if not isinstance(drift_score, (int, float)):
+        if not is_number(drift_score):
             raise TypeError(
                 f"drift_score must be number, got {type(drift_score).__name__}"
             )

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/datascience_exceptions/training.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 from typing import Any, Optional
 
+from .._validation import is_number
 from .base import DataScienceError
 
 
@@ -79,7 +80,7 @@ class TrainingTimeoutError(ModelTrainingError):
     """Raised when model training exceeds a time limit."""
 
     def __init__(self, model_type: str, timeout: float) -> None:
-        if not isinstance(timeout, (int, float)):
+        if not is_number(timeout):
             raise TypeError(f"timeout must be a number, got {type(timeout).__name__}")
         message = f"Training '{model_type}' exceeded timeout of {timeout} seconds"
         self.timeout = float(timeout)
@@ -127,7 +128,7 @@ class ModelEvaluationError(DataScienceError):
     ) -> None:
         if not isinstance(metric, str):
             raise TypeError(f"metric must be str, got {type(metric).__name__}")
-        if not isinstance(value, (int, float)):
+        if not is_number(value):
             raise TypeError(f"value must be number, got {type(value).__name__}")
 
         if message is None:
@@ -330,11 +331,11 @@ class OverfittingError(DataScienceError):
     """
 
     def __init__(self, train_metric: float, val_metric: float) -> None:
-        if not isinstance(train_metric, (int, float)):
+        if not is_number(train_metric):
             raise TypeError(
                 ("train_metric must be numeric, got " f"{type(train_metric).__name__}")
             )
-        if not isinstance(val_metric, (int, float)):
+        if not is_number(val_metric):
             raise TypeError(
                 f"val_metric must be numeric, got {type(val_metric).__name__}"
             )
@@ -360,11 +361,11 @@ class UnderfittingError(DataScienceError):
     """
 
     def __init__(self, train_metric: float, threshold: float) -> None:
-        if not isinstance(train_metric, (int, float)):
+        if not is_number(train_metric):
             raise TypeError(
                 ("train_metric must be numeric, got " f"{type(train_metric).__name__}")
             )
-        if not isinstance(threshold, (int, float)):
+        if not is_number(threshold):
             raise TypeError(
                 f"threshold must be numeric, got {type(threshold).__name__}"
             )
@@ -426,11 +427,11 @@ class BiasDetectionError(DataScienceError):
     ) -> None:
         if not isinstance(feature, str):
             raise TypeError(f"feature must be str, got {type(feature).__name__}")
-        if not isinstance(bias_score, (int, float)):
+        if not is_number(bias_score):
             raise TypeError(
                 f"bias_score must be numeric, got {type(bias_score).__name__}"
             )
-        if not isinstance(threshold, (int, float)):
+        if not is_number(threshold):
             raise TypeError(
                 f"threshold must be numeric, got {type(threshold).__name__}"
             )

dataexcept-0.4.0/dataexcept/exceptions/base.py

@@ -0,0 +1,7 @@
+from ..base import DataExceptError
+
+
+class JobError(DataExceptError):
+    """Base exception for all job-related errors."""
+
+    pass

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/exceptions/notification.py

@@ -1,4 +1,5 @@
 # notification.py
+from ..redaction import redact_url
 from .base import JobError
 
 
@@ -30,7 +31,7 @@ class EmailError(NotificationError):
     ):
         self.recipient = recipient
         self.subject = subject
-        self.original_exception = original_exception
+        # original_exception is set by NotificationError.__init__ below.
         msg = f"Email to '{recipient}' with subject '{subject}' failed"
         if original_exception:
             msg += f": {original_exception}"
@@ -41,9 +42,10 @@ class WebhookError(NotificationError):
     """Raised when a webhook POST fails."""
 
     def __init__(self, url: str, original_exception: Exception | None = None):
-        self.url = url
-        self.original_exception = original_exception
-        msg = f"Webhook to URL '{url}' failed"
+        # Webhook URLs commonly authenticate through a query parameter.
+        self.url = redact_url(url)
+        # original_exception is set by NotificationError.__init__ below.
+        msg = f"Webhook to URL '{self.url}' failed"
         if original_exception:
             msg += f": {original_exception}"
         super().__init__("webhook", original_exception, message=msg)

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/io_exceptions.py

@@ -2,8 +2,10 @@
 
 from __future__ import annotations
 
+from .base import DataExceptError
 
-class CustomIOError(Exception):
+
+class CustomIOError(DataExceptError):
     """Base exception for I/O errors."""
 
     pass

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/network_exceptions.py

@@ -2,8 +2,10 @@
 
 from __future__ import annotations
 
+from .base import DataExceptError
 
-class NetworkError(Exception):
+
+class NetworkError(DataExceptError):
     """Base exception for network-related errors.
 
     Example:

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/pandas_exceptions.py

@@ -4,8 +4,10 @@ from __future__ import annotations
 
 from typing import Optional, Sequence
 
+from .base import DataExceptError
 
-class PandasError(Exception):
+
+class PandasError(DataExceptError):
     """Base exception for pandas-related errors."""
 
 

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/pipeline_exceptions.py

@@ -5,9 +5,11 @@ from __future__ import annotations
 from typing import Any, Optional
 
 from ._deprecation import resolve_deprecated
+from .base import DataExceptError
+from .redaction import redact_url
 
 
-class PipelineError(Exception):
+class PipelineError(DataExceptError):
     """Base exception for pipeline errors."""
 
     pass
@@ -147,10 +149,11 @@ class ApiError(PipelineError):
         status_code: Optional[int] = None,
         message: Optional[str] = None,
     ) -> None:
-        default = f"API call failed: {endpoint}"
+        # An endpoint URL may authenticate through a query parameter.
+        self.endpoint = redact_url(endpoint)
+        default = f"API call failed: {self.endpoint}"
         if status_code is not None:
             default += f" (status {status_code})"
-        self.endpoint = endpoint
         self.status_code = status_code
         super().__init__(message or default)
 

dataexcept-0.4.0/dataexcept/redaction.py

@@ -0,0 +1,114 @@
+"""Redaction helpers for values that must not reach a log.
+
+Several exceptions here are raised with credentials in hand: an authentication
+token, a database URL carrying a password, a webhook URL with a signing
+parameter. Those values end up in the exception message, and
+:func:`dataexcept.logging_helpers.log_exception` logs ``str(exc)``, so without
+redaction a failed connection writes the password to the log.
+
+The aim is to keep an error debuggable while giving up the secret. A redacted
+value carries a short, non-reversible fingerprint, so repeated failures of the
+*same* credential are still recognisable in a log without the credential
+appearing in it.
+"""
+
+from __future__ import annotations
+
+import hashlib
+from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
+
+__all__ = ["fingerprint", "redact_secret", "redact_url"]
+
+PLACEHOLDER = "***"
+
+#: Query parameters whose value is treated as a secret. Matched case
+#: insensitively against the whole parameter name.
+SENSITIVE_QUERY_PARAMS = frozenset(
+    {
+        "access_token",
+        "api_key",
+        "apikey",
+        "auth",
+        "credential",
+        "key",
+        "password",
+        "private_key",
+        "pwd",
+        "refresh_token",
+        "secret",
+        "session",
+        "sig",
+        "signature",
+        "token",
+    }
+)
+
+
+def fingerprint(value: str) -> str:
+    """Return a short, one-way fingerprint of *value*.
+
+    Enough to tell "the same bad token again" from "a different bad token",
+    and not enough to recover the token.
+    """
+    digest = hashlib.sha256(value.encode("utf-8", "replace")).hexdigest()
+    return digest[:8]
+
+
+def redact_secret(value: str | None) -> str | None:
+    """Replace a secret with a placeholder and its fingerprint."""
+    if value is None:
+        return None
+    if not value:
+        return PLACEHOLDER
+    return f"{PLACEHOLDER}({fingerprint(value)})"
+
+
+def redact_url(url: str | None) -> str | None:
+    """Strip credentials and sensitive query parameters from *url*.
+
+    The scheme, host, port and path are kept, because those are what make the
+    error actionable. Anything that authenticates is replaced.
+    """
+    if not url:
+        return url
+
+    try:
+        parts = urlsplit(url)
+    except ValueError:  # pragma: no cover - urlsplit is extremely permissive
+        return PLACEHOLDER
+
+    if not parts.scheme and not parts.netloc:
+        # Not a URL at all; treat the whole thing as sensitive rather than
+        # returning it unchanged.
+        return url
+
+    netloc = parts.netloc
+    redacted = False
+    if "@" in netloc:
+        _, _, host = netloc.rpartition("@")
+        netloc = f"{PLACEHOLDER}:{PLACEHOLDER}@{host}"
+        redacted = True
+
+    query = parts.query
+    if query:
+        pairs = parse_qsl(query, keep_blank_values=True)
+        if any(key.lower() in SENSITIVE_QUERY_PARAMS for key, _ in pairs):
+            query = urlencode(
+                [
+                    (
+                        key,
+                        PLACEHOLDER if key.lower() in SENSITIVE_QUERY_PARAMS else value,
+                    )
+                    for key, value in pairs
+                ],
+                # Keep the placeholder legible rather than percent-encoded.
+                safe="*",
+            )
+            redacted = True
+
+    if not redacted:
+        # Nothing sensitive, so hand back exactly what was passed in.
+        # Rebuilding would normalise it -- "sqlite://" loses its slashes.
+        return url
+
+    return urlunsplit((parts.scheme, netloc, parts.path, query, parts.fragment))

{dataexcept-0.3.0 → dataexcept-0.4.0}/dataexcept/security_exceptions.py

@@ -2,8 +2,11 @@
 
 from __future__ import annotations
 
+from .base import DataExceptError
+from .redaction import redact_secret
 
-class SecurityError(Exception):
+
+class SecurityError(DataExceptError):
     """Base exception for security errors."""
 
     pass
@@ -53,10 +56,12 @@ class InvalidTokenError(SecurityError):
             token: The problematic token.
             message: Optional custom error message.
         """
-        self.token = token
+        # The raw token is never stored or rendered: this exception is often
+        # logged, and the caller already holds the value it passed in.
+        self.token = redact_secret(token)
         default = "Invalid authentication token"
         if token:
-            default += f": {token}"
+            default += f": {self.token}"
         super().__init__(message or default)
 
 

{dataexcept-0.3.0 → dataexcept-0.4.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "DataExcept"
-version = "0.3.0"
+version = "0.4.0"
 description = "A Python package providing structured, easily-extendable custom exception types."
 readme = "README.md"
 license = "MIT"
@@ -85,7 +85,9 @@ max-complexity = 8
 [tool.ruff.lint.per-file-ignores]
 # assert is how a test asserts, and "/tmp/data" there is a string literal used
 # as fixture data rather than a path that is ever opened.
-"tests/**" = ["S101", "S108", "S603"]
+# S301: the serialization tests unpickle data they created a line earlier.
+# S105/S106: redaction tests need credential-shaped literals to redact.
+"tests/**" = ["S101", "S105", "S106", "S108", "S301", "S603"]
 # The lambda example asserts to narrow Optionals for the type checker.
 "examples/**" = ["S101"]
 # bump_version.py deliberately shells out to poetry, by name, on a maintainer's
@@ -98,3 +100,20 @@ files = ["dataexcept"]
 no_implicit_optional = true
 warn_redundant_casts = true
 warn_unused_configs = true
+
+[tool.coverage.run]
+# Without this, `coverage run -m pytest` measures the tests and examples too,
+# which inflates the figure: the test files are by definition fully executed.
+source = ["dataexcept"]
+branch = true
+
+[tool.coverage.report]
+# The floor is a ratchet against regression, not a target. Raise it when the
+# real number rises; never lower it to make a build pass.
+fail_under = 91
+show_missing = true
+exclude_also = [
+    "if TYPE_CHECKING:",
+    "raise NotImplementedError",
+    "if __name__ == .__main__.:",
+]

dataexcept-0.3.0/dataexcept/exceptions/base.py

@@ -1,4 +0,0 @@
-class JobError(Exception):
-    """Base exception for all job-related errors."""
-
-    pass