edaprep 0.2.0__tar.gz → 0.2.2__tar.gz

{edaprep-0.2.0/src/edaprep.egg-info → edaprep-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: edaprep
-Version: 0.2.0
+Version: 0.2.2
 Summary: Transparent, leakage-safe EDA and ML preprocessing with an explainable planner.
 Author-email: bijay <bijaybeezoe@gmail.com>
 License: MIT
@@ -125,6 +125,8 @@ pip install "edaprep[all]"
 
 Python 3.9–3.13, tested on Linux, macOS and Windows.
 
+Would rather try it than install it? There is a [runnable notebook on Kaggle](https://www.kaggle.com/code/bijaybeezoe/every-preprocessing-decision-with-its-reason) that needs no setup — it measures two preprocessing leaks on Telco Churn, one worth `+0.000001` and one scoring a perfect `1.00000` AUC on pure noise, then shows what this library does about them.
+
 ---
 
 ## What it does
@@ -268,12 +270,13 @@ looked promising was slower than pandas.
 
 | | |
 |---|---|
-| [Workflow mining](docs/design-rationale.md) | what 13 repositories revealed, and the 9 defects found |
+| [Design rationale](docs/design-rationale.md) | the workflow tabular notebooks converge on, where it reliably goes wrong, and how each design decision follows |
 | [Architecture](docs/architecture.md) | package design, the planner, execution model |
 | [User guide](docs/guide.md) | installation to production, with the train/test workflow |
 | [Performance](docs/performance.md) | benchmarks, method, and what optimisation actually changed |
 | [Extending](docs/extending.md) | custom transformers, rules and backends |
 | [Example](examples/end_to_end.py) | raw dataset to ML-ready, end to end |
+| [Runnable notebook](https://www.kaggle.com/code/bijaybeezoe/every-preprocessing-decision-with-its-reason) | Kaggle, no install: the `select_dtypes` failure and two leaks, measured on Telco Churn |
 
 ---
 
@@ -296,15 +299,37 @@ CI actually enforces, and points at issues scoped so that each one names the fil
 change and the test to write.
 
 Issues labelled [`good first issue`](https://github.com/bijay-odyssey/edaprep/labels/good%20first%20issue)
-are a deliberate starting set. Comment to claim one.
+are a deliberate starting set. **Comment on one to claim it before you start** — two
+people once fixed the same issue eight hours apart, and one of them had to be turned
+away.
 
 ```bash
 pip install -e ".[dev]"
-pytest                       # 358 tests, ~15s
+pytest                       # 367 tests, ~15s
 ruff check src/ tests/ benchmarks/ examples/
 python benchmarks/bench.py
 ```
 
+## Contributors
+
+This library argues that a decision is worth little without the reasoning behind it,
+so it would be odd to credit only the diffs. Measurement, review and design that
+changed what shipped are listed here alongside the commits.
+
+| | |
+|---|---|
+| [@qiaobochi040726-source](https://github.com/qiaobochi040726-source) | Removed `Config.n_jobs`, a setting the library accepted and never read ([#11](https://github.com/bijay-odyssey/edaprep/pull/11)) |
+| [@zbs-ops](https://github.com/zbs-ops) | Benchmarked parallelising the per-column profiling loop, which is what decided [#9](https://github.com/bijay-odyssey/edaprep/issues/9) — no commit, and the reason the parameter went |
+| [@Jeferson681](https://github.com/Jeferson681) | Made the missing-indicator and high-missing rules see gaps introduced at the cast step ([#16](https://github.com/bijay-odyssey/edaprep/pull/16)) |
+| [@luziyi123448-gif](https://github.com/luziyi123448-gif) | Independent fix for the same issue whose design was the better one; that design shipped in [#18](https://github.com/bijay-odyssey/edaprep/issues/18) ([#17](https://github.com/bijay-odyssey/edaprep/pull/17)) |
+| [@LeonxLJX](https://github.com/LeonxLJX) | Formatted the tree and made `ruff format` a real CI gate rather than an advisory one ([#25](https://github.com/bijay-odyssey/edaprep/pull/25)) |
+| [@TrueFurina](https://github.com/TrueFurina) | First to run the Codecov upload path, which is how we learned it fails silently from a fork with no token — CI green, nothing reported; coverage moved to the job summary instead ([#21](https://github.com/bijay-odyssey/edaprep/pull/21)) |
+
+[GitHub's contributors graph](https://github.com/bijay-odyssey/edaprep/graphs/contributors)
+counts commits on `main`, so it cannot show a benchmark that settled an argument or a
+review that caught a bug. This table can. If you contributed something that changed the
+library and you are not on it, that is an oversight worth an issue — please open one.
+
 ## Licence
 
 MIT.

{edaprep-0.2.0 → edaprep-0.2.2}/README.md

@@ -79,6 +79,8 @@ pip install "edaprep[all]"
 
 Python 3.9–3.13, tested on Linux, macOS and Windows.
 
+Would rather try it than install it? There is a [runnable notebook on Kaggle](https://www.kaggle.com/code/bijaybeezoe/every-preprocessing-decision-with-its-reason) that needs no setup — it measures two preprocessing leaks on Telco Churn, one worth `+0.000001` and one scoring a perfect `1.00000` AUC on pure noise, then shows what this library does about them.
+
 ---
 
 ## What it does
@@ -222,12 +224,13 @@ looked promising was slower than pandas.
 
 | | |
 |---|---|
-| [Workflow mining](docs/design-rationale.md) | what 13 repositories revealed, and the 9 defects found |
+| [Design rationale](docs/design-rationale.md) | the workflow tabular notebooks converge on, where it reliably goes wrong, and how each design decision follows |
 | [Architecture](docs/architecture.md) | package design, the planner, execution model |
 | [User guide](docs/guide.md) | installation to production, with the train/test workflow |
 | [Performance](docs/performance.md) | benchmarks, method, and what optimisation actually changed |
 | [Extending](docs/extending.md) | custom transformers, rules and backends |
 | [Example](examples/end_to_end.py) | raw dataset to ML-ready, end to end |
+| [Runnable notebook](https://www.kaggle.com/code/bijaybeezoe/every-preprocessing-decision-with-its-reason) | Kaggle, no install: the `select_dtypes` failure and two leaks, measured on Telco Churn |
 
 ---
 
@@ -250,15 +253,37 @@ CI actually enforces, and points at issues scoped so that each one names the fil
 change and the test to write.
 
 Issues labelled [`good first issue`](https://github.com/bijay-odyssey/edaprep/labels/good%20first%20issue)
-are a deliberate starting set. Comment to claim one.
+are a deliberate starting set. **Comment on one to claim it before you start** — two
+people once fixed the same issue eight hours apart, and one of them had to be turned
+away.
 
 ```bash
 pip install -e ".[dev]"
-pytest                       # 358 tests, ~15s
+pytest                       # 367 tests, ~15s
 ruff check src/ tests/ benchmarks/ examples/
 python benchmarks/bench.py
 ```
 
+## Contributors
+
+This library argues that a decision is worth little without the reasoning behind it,
+so it would be odd to credit only the diffs. Measurement, review and design that
+changed what shipped are listed here alongside the commits.
+
+| | |
+|---|---|
+| [@qiaobochi040726-source](https://github.com/qiaobochi040726-source) | Removed `Config.n_jobs`, a setting the library accepted and never read ([#11](https://github.com/bijay-odyssey/edaprep/pull/11)) |
+| [@zbs-ops](https://github.com/zbs-ops) | Benchmarked parallelising the per-column profiling loop, which is what decided [#9](https://github.com/bijay-odyssey/edaprep/issues/9) — no commit, and the reason the parameter went |
+| [@Jeferson681](https://github.com/Jeferson681) | Made the missing-indicator and high-missing rules see gaps introduced at the cast step ([#16](https://github.com/bijay-odyssey/edaprep/pull/16)) |
+| [@luziyi123448-gif](https://github.com/luziyi123448-gif) | Independent fix for the same issue whose design was the better one; that design shipped in [#18](https://github.com/bijay-odyssey/edaprep/issues/18) ([#17](https://github.com/bijay-odyssey/edaprep/pull/17)) |
+| [@LeonxLJX](https://github.com/LeonxLJX) | Formatted the tree and made `ruff format` a real CI gate rather than an advisory one ([#25](https://github.com/bijay-odyssey/edaprep/pull/25)) |
+| [@TrueFurina](https://github.com/TrueFurina) | First to run the Codecov upload path, which is how we learned it fails silently from a fork with no token — CI green, nothing reported; coverage moved to the job summary instead ([#21](https://github.com/bijay-odyssey/edaprep/pull/21)) |
+
+[GitHub's contributors graph](https://github.com/bijay-odyssey/edaprep/graphs/contributors)
+counts commits on `main`, so it cannot show a benchmark that settled an argument or a
+review that caught a bug. This table can. If you contributed something that changed the
+library and you are not on it, that is an oversight worth an issue — please open one.
+
 ## Licence
 
 MIT.

edaprep-0.2.2/src/edaprep/_version.py

@@ -0,0 +1 @@
+__version__ = "0.2.2"

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/backends/base.py

@@ -118,15 +118,11 @@ class Backend(ABC):
         """Quantile matrix of shape ``(len(levels), len(columns))``."""
 
     @abstractmethod
-    def group_mean(
-        self, frame: Any, value_column: str, group_column: str
-    ) -> Dict[Any, float]:
+    def group_mean(self, frame: Any, value_column: str, group_column: str) -> Dict[Any, float]:
         """Mean of ``value_column`` per level of ``group_column``."""
 
     @abstractmethod
-    def duplicated_rows(
-        self, frame: Any, subset: Optional[Sequence[str]] = None
-    ) -> np.ndarray:
+    def duplicated_rows(self, frame: Any, subset: Optional[Sequence[str]] = None) -> np.ndarray:
         """Boolean array: True where the row repeats an earlier one."""
 
     # -- construction --------------------------------------------------------------------

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/backends/pandas_backend.py

@@ -109,9 +109,7 @@ class PandasBackend(Backend):
             return frame.duplicated(subset=list(subset) if subset else None).to_numpy()
         except TypeError:
             return (
-                frame.astype(str)
-                .duplicated(subset=list(subset) if subset else None)
-                .to_numpy()
+                frame.astype(str).duplicated(subset=list(subset) if subset else None).to_numpy()
             )
 
     # -- construction --------------------------------------------------------------------

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/core/base.py

@@ -227,8 +227,7 @@ class Transformer(ABC):
     ) -> FitContext:
         if not isinstance(X, pd.DataFrame):
             raise TypeError(
-                f"{type(self).__name__}.fit expects a pandas DataFrame, got "
-                f"{type(X).__name__}."
+                f"{type(self).__name__}.fit expects a pandas DataFrame, got {type(X).__name__}."
             )
         if self.uses_target and y is None:
             raise LeakageError.target_required(type(self).__name__)

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/core/pipeline.py

@@ -226,9 +226,7 @@ class Pipeline(Transformer):
         self._context_ = context
         return super().fit_transform(X, y, context)
 
-    def transform(
-        self, X: pd.DataFrame, context: Optional[FitContext] = None
-    ) -> pd.DataFrame:
+    def transform(self, X: pd.DataFrame, context: Optional[FitContext] = None) -> pd.DataFrame:
         check_is_fitted(self)
         context = context or getattr(self, "_context_", None) or FitContext(config=self.config)
         # Only the most recent transform is described in the report; otherwise calling
@@ -437,9 +435,7 @@ class AutoPipeline(Pipeline):
         self.report_ = self.report()
         return out
 
-    def transform(
-        self, X: pd.DataFrame, context: Optional[FitContext] = None
-    ) -> pd.DataFrame:
+    def transform(self, X: pd.DataFrame, context: Optional[FitContext] = None) -> pd.DataFrame:
         out = super().transform(X, context)
         self.report_ = self.report()
         return out
@@ -497,9 +493,7 @@ class AutoPipeline(Pipeline):
                     "rule": d.rule,
                     "source": d.source,
                 }
-                for d in sorted(
-                    self.plan_.decisions, key=lambda d: (d.column, d.stage.order)
-                )
+                for d in sorted(self.plan_.decisions, key=lambda d: (d.column, d.stage.order))
             ]
         )
 

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/eda/analyzer.py

@@ -137,9 +137,7 @@ class EDAReport:
         if self.issues:
             parts.append("<h2>Findings</h2>")
             for issue in sorted(self.issues, key=lambda i: -i.severity.rank):
-                parts.append(
-                    f"<div class='w {_e(issue.severity)}'>{_e(issue.message)}</div>"
-                )
+                parts.append(f"<div class='w {_e(issue.severity)}'>{_e(issue.message)}</div>")
         parts.append("</main>")
         html = "\n".join(parts)
         if path:
@@ -262,9 +260,7 @@ class EDA:
     def profile_(self) -> DatasetProfile:
         """The profile, computed once and reused."""
         if self._profile is None:
-            self._profile = profile_dataset(
-                self.data, target=self.target, config=self.config
-            )
+            self._profile = profile_dataset(self.data, target=self.target, config=self.config)
         return self._profile
 
     def analyze(

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/eda/categorical.py

@@ -72,9 +72,7 @@ def categorical_summary(
 def _note(cp, high_cardinality: int, n_rare: int) -> str:
     notes = []
     if cp.n_unique > high_cardinality:
-        notes.append(
-            f"high cardinality: one-hot would add {cp.n_unique} columns"
-        )
+        notes.append(f"high cardinality: one-hot would add {cp.n_unique} columns")
     if cp.is_near_constant:
         notes.append(f"near-constant ({cp.modal_frequency:.1%} one value)")
     if n_rare:

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/eda/correlation.py

@@ -133,9 +133,7 @@ def variance_inflation(
 
     # copy=True for the same reason as in preprocessing/selection.py: pandas 2.3+
     # returns a read-only view, and `fill_diagonal` writes in place.
-    corr = frame.corr(method="pearson", numeric_only=True).to_numpy(
-        dtype=np.float64, copy=True
-    )
+    corr = frame.corr(method="pearson", numeric_only=True).to_numpy(dtype=np.float64, copy=True)
     corr = np.nan_to_num(corr, nan=0.0)
     np.fill_diagonal(corr, 1.0)
 

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/eda/numerical.py

@@ -78,9 +78,7 @@ def numerical_summary(
         return frame
     # Most-skewed first: those are the columns a reader most needs to look at.
     frame["_sort"] = frame["skew"].abs().fillna(-1)
-    frame = frame.sort_values("_sort", ascending=False, ignore_index=True).drop(
-        columns="_sort"
-    )
+    frame = frame.sort_values("_sort", ascending=False, ignore_index=True).drop(columns="_sort")
     numeric_cols = frame.select_dtypes(include="number").columns
     frame[numeric_cols] = frame[numeric_cols].round(4)
     return frame

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/eda/target.py

@@ -30,9 +30,7 @@ from ..types import NUMERIC_LIKE, SemanticType
 __all__ = ["target_summary", "target_relationships", "benjamini_hochberg"]
 
 
-def target_summary(
-    data: pd.DataFrame, profile: DatasetProfile, target: str
-) -> Dict[str, Any]:
+def target_summary(data: pd.DataFrame, profile: DatasetProfile, target: str) -> Dict[str, Any]:
     """Distribution of the target, plus the imbalance measurement."""
     series = data[target]
     out: Dict[str, Any] = {
@@ -121,9 +119,7 @@ def target_relationships(
             "column": name,
             "semantic": str(cp.semantic),
             "association": (
-                round(cp.target_association, 4)
-                if cp.target_association is not None
-                else None
+                round(cp.target_association, 4) if cp.target_association is not None else None
             ),
             "measure": cp.target_association_kind,
         }
@@ -145,9 +141,7 @@ def target_relationships(
         frame["p_value"] = frame["p_value"].round(6)
 
     frame["_sort"] = frame["association"].fillna(-1)
-    return frame.sort_values("_sort", ascending=False, ignore_index=True).drop(
-        columns="_sort"
-    )
+    return frame.sort_values("_sort", ascending=False, ignore_index=True).drop(columns="_sort")
 
 
 def _test(

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/planning/decisions.py

@@ -169,9 +169,7 @@ class Plan:
     @property
     def decisions(self) -> List[Decision]:
         """Every decision, including the ones that resolved to doing nothing."""
-        return [d for step in self.steps for d in step.decisions] + list(
-            self.noop_decisions
-        )
+        return [d for step in self.steps for d in step.decisions] + list(self.noop_decisions)
 
     def for_column(self, column: str) -> List[Decision]:
         """Every decision affecting ``column``, in stage order."""
@@ -199,9 +197,7 @@ class Plan:
 
     @property
     def uses_target(self) -> bool:
-        return any(
-            d.action in ("encode_target",) for d in self.decisions
-        )
+        return any(d.action in ("encode_target",) for d in self.decisions)
 
     # -- editing --------------------------------------------------------------------
 
@@ -252,9 +248,7 @@ class Plan:
     def from_dict(cls, data: Dict[str, Any]) -> "Plan":
         return cls(
             steps=tuple(PlannedStep.from_dict(s) for s in data.get("steps", ())),
-            noop_decisions=tuple(
-                Decision.from_dict(d) for d in data.get("noop_decisions", ())
-            ),
+            noop_decisions=tuple(Decision.from_dict(d) for d in data.get("noop_decisions", ())),
             target=data.get("target"),
             model_family=data.get("model_family"),
             dropped_columns=dict(data.get("dropped_columns", {})),

{edaprep-0.2.0 → edaprep-0.2.2}/src/edaprep/planning/planner.py

@@ -92,9 +92,7 @@ class Planner:
         # DROP_COLUMNS first: a dropped column takes no further part in planning, which
         # keeps the plan free of steps that operate on columns that will not exist.
         for name in feature_columns:
-            decision = self.rules.decide(
-                Stage.DROP_COLUMNS, profile.columns[name], context
-            )
+            decision = self.rules.decide(Stage.DROP_COLUMNS, profile.columns[name], context)
             if decision is not None and decision.action == "drop":
                 dropped[name] = decision.rationale
                 decisions_by_stage.setdefault(Stage.DROP_COLUMNS, []).append(decision)
@@ -259,9 +257,7 @@ class Planner:
             columns = ()
 
         if stage is Stage.OUTLIERS:
-            params["per_column_method"] = {
-                d.column: d.params.get("method") for d in decisions
-            }
+            params["per_column_method"] = {d.column: d.params.get("method") for d in decisions}
             params["per_column_strategy"] = {
                 d.column: d.params.get("strategy") for d in decisions
             }
@@ -277,9 +273,7 @@ class Planner:
             params["threshold"] = decisions[0].params.get("threshold")
         elif stage is Stage.DATETIME:
             explicit = {
-                d.column: d.params["features"]
-                for d in decisions
-                if d.params.get("features")
+                d.column: d.params["features"] for d in decisions if d.params.get("features")
             }
             if explicit:
                 params["per_column_features"] = explicit