classifier-toolkit 0.3.2__tar.gz → 0.3.4__tar.gz

{classifier_toolkit-0.3.2 → classifier_toolkit-0.3.4}/.github/workflows/docs.yml

@@ -6,10 +6,12 @@ on:
     paths:
       - "docs/**"
       - "mkdocs.yml"
+  workflow_dispatch:
 
 jobs:
   deploy:
     runs-on: default
+    if: github.event_name != 'workflow_dispatch' || github.ref == 'refs/heads/master'
     permissions:
       contents: write
     steps:

{classifier_toolkit-0.3.2 → classifier_toolkit-0.3.4}/.gitignore

@@ -78,6 +78,7 @@ target/
 # Jupyter Notebook
 .ipynb_checkpoints
 *.ipynb
+!examples/*.ipynb
 
 # IPython
 profile_default/

{classifier_toolkit-0.3.2 → classifier_toolkit-0.3.4}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: classifier-toolkit
-Version: 0.3.2
-Project-URL: Documentation, http://classifier-toolkit.gh-pages.qonto.co/
+Version: 0.3.4
+Project-URL: Documentation, https://supreme-adventure-jg5qkyr.pages.github.io/
 Author-email: "senih.yilmaz" <senih.yilmaz@qonto.com>, "jeremy.fraoua" <jeremy.fraoua@qonto.com>, "gauthier.marquand" <gauthier.marquand@qonto.com>, "arnaud.alepee" <arnaud.alepee@qonto.com>
 License-File: LICENSE
 Requires-Python: <3.14,>=3.9

classifier_toolkit-0.3.4/docs/examples/feature_selection_advanced.md

@@ -0,0 +1,111 @@
+# Advanced Feature Selection Examples
+
+## Feature Stability Analysis
+
+```python
+from classifier_toolkit.feature_selection import FeatureStability
+
+stability = FeatureStability(
+    X=X,
+    y=y,
+    target_col="target",
+    model_params={"n_estimators": 100},
+    method_params={"d": 100, "n_sets": 10},
+    model="xgboost",
+    method="jackknife",
+)
+
+stability.fit()
+results = stability.calculate_stability()
+print(results)
+```
+
+## Boruta Feature Selection
+
+```python
+from classifier_toolkit.feature_selection import BorutaSelector
+
+selector = BorutaSelector(
+    estimator_name="xgboost",
+    n_estimators="auto",
+    perc=99,
+    alpha=0.001,
+)
+
+X_selected = selector.fit_transform(X, y)
+print(selector.selected_features_)
+```
+
+## Bayesian Feature Search
+
+```python
+from classifier_toolkit.feature_selection import BayesianFeatureSelector
+
+selector = BayesianFeatureSelector(
+    estimator_name="lightgbm",
+    n_features_to_select=10,
+    max_trials=100,
+    scoring="average_precision",
+)
+
+X_selected = selector.fit_transform(X, y)
+print(selector.selected_features_)
+```
+
+## RFE with CatBoost
+
+```python
+from classifier_toolkit.feature_selection import RFECatBoostSelector
+
+selector = RFECatBoostSelector(
+    model_params={"iterations": 200, "depth": 4, "verbose": False},
+    cat_features=["legal_sector"],
+    n_features_to_select=10,
+)
+
+selector.fit(X_train, y_train, cv_folds=cv_folds)
+print(selector.selected_features_)
+print(selector.rankings_)
+```
+
+## Elastic Net Selection
+
+```python
+from classifier_toolkit.feature_selection import ElasticNetLogisticSelector
+
+selector = ElasticNetLogisticSelector(
+    n_features_to_select=10,
+    scoring="average_precision",
+)
+
+X_selected = selector.fit_transform(X, y)
+print(selector.selected_features_)
+```
+
+## Combining Multiple Methods
+
+```python
+from classifier_toolkit.feature_selection import (
+    MetaSelector,
+    RFESelector,
+    BorutaSelector,
+)
+
+selector = MetaSelector(
+    model_params=[],
+    rfe_params={},
+    rfe_catboost_params={},
+    sequential_params={},
+    elastic_net_params={},
+    n_features_to_select=5,
+    methods=[
+        RFESelector(estimator_name="lightgbm", n_features_to_select=10),
+        BorutaSelector(estimator_name="xgboost", n_estimators="auto"),
+    ],
+    voting="majority",
+)
+
+X_selected = selector.fit_transform(X, y)
+method_results = selector.get_method_specific_results()
+print(selector.selected_features_)
+```

classifier_toolkit-0.3.4/docs/examples/feature_selection_example.md

@@ -0,0 +1,123 @@
+# Feature Selection Examples
+
+## RFE with LightGBM
+
+```python
+from classifier_toolkit.feature_selection import RFESelector
+
+selector = RFESelector(
+    estimator_name="lightgbm",
+    n_features_to_select=10,
+    scoring="average_precision",
+    cat_features=["legal_sector", "industry"],
+)
+
+selector.fit(X_train, y_train, cv_folds=cv_folds)
+
+print(selector.selected_features_)
+print(selector.prauc_scores_)
+```
+
+### Auto-select the optimal number of features
+
+```python
+selector = RFESelector(
+    estimator_name="lightgbm",
+    n_features_to_select=-1,  # auto — picks the peak PR-AUC
+    scoring="average_precision",
+)
+
+selector.fit(X_train, y_train, cv_folds=cv_folds)
+print(f"Selected {selector.n_features_to_select} features")
+print(selector.selected_features_)
+```
+
+## Recursive Feature Eliminator (large feature sets)
+
+```python
+from classifier_toolkit.feature_selection import RecursiveFeatureEliminator
+
+rfe = RecursiveFeatureEliminator(
+    estimator_name="lightgbm",
+    model_params={"random_state": 42},
+    n_features_to_select=10,
+    overfitting_strategy="rollback",
+)
+
+rfe.fit(X_train, y_train, X_val=X_val, y_val=y_val)
+X_selected = rfe.transform(X_train)
+
+print(rfe.selected_features_)
+```
+
+## Sequential Selection
+
+```python
+from classifier_toolkit.feature_selection import SequentialSelector
+
+selector = SequentialSelector(
+    estimator_name="lightgbm",
+    method="forward",
+    n_features_to_select=5,
+    scoring="roc_auc",
+    cv=5,
+)
+
+X_selected = selector.fit_transform(X, y)
+selector.print_feature_importances()
+```
+
+## Ensemble Selection with MetaSelector
+
+```python
+from classifier_toolkit.feature_selection import (
+    MetaSelector,
+    RFESelector,
+    SequentialSelector,
+)
+
+selector = MetaSelector(
+    model_params=[],
+    rfe_params={},
+    rfe_catboost_params={},
+    sequential_params={},
+    elastic_net_params={},
+    n_features_to_select=5,
+    methods=[
+        RFESelector(
+            estimator_name="lightgbm",
+            n_features_to_select=10,
+        ),
+        SequentialSelector(
+            estimator_params={},
+            method="forward",
+            estimator_name="lightgbm",
+            n_features_to_select=10,
+        ),
+    ],
+    voting="majority",
+)
+
+X_selected = selector.fit_transform(X, y)
+method_results = selector.get_method_specific_results()
+importances = selector.get_feature_importances()
+```
+
+## Feature Combination Searcher (exhaustive subset search)
+
+```python
+from classifier_toolkit.feature_selection import FeatureCombinationSearcher
+
+searcher = FeatureCombinationSearcher(
+    required_features=["feature_a", "feature_b"],
+    candidate_features=["feature_c", "feature_d", "feature_e"],
+    categorical_features=["feature_b"],
+    model_params={"random_state": 42, "n_estimators": 100},
+    estimator_name="lightgbm",
+    min_features=3,
+    max_features=4,
+)
+
+results = searcher.search(X_train, y_train, X_val, y_val, X_test, y_test)
+print(results.sort_values("val_auc", ascending=False).head())
+```

classifier_toolkit-0.3.4/docs/feature_selection/overview.md

@@ -0,0 +1,89 @@
+# Feature Selection Package Overview
+
+The Feature Selection package provides a comprehensive toolkit for selecting the most predictive features for your machine learning models. It includes wrapper methods, embedded methods, ensemble orchestration, and stability analysis.
+
+## Links
+
+- [GitHub Repository](https://github.com/qonto-private/classifier-toolkit)
+- [PyPI Package](https://pypi.org/project/classifier-toolkit/)
+
+## Core Components
+
+### [Meta Selector](meta_selector.md)
+
+The central orchestrator that combines multiple feature selection methods and aggregates their results via voting:
+
+- Majority, union, and intersection voting strategies
+- Per-method result tracking
+- Combined feature importance scores
+
+### Wrapper Methods
+
+#### RFE — Cross-validated elimination
+
+- [RFE](wrapper_methods/rfe.md) — `RFESelector` (XGBoost, LightGBM, Random Forest) and `RFECatBoostSelector` (CatBoost native). Eliminates features step by step, scored via cross-validation. Supports auto feature count (`n_features_to_select=-1`), SHAP-combined ranking, and re-ranking.
+
+#### Recursive Feature Eliminator — Adaptive two-phase elimination
+
+- [Recursive Feature Eliminator](wrapper_methods/recursive_feature_eliminator.md) — `RecursiveFeatureEliminator`. A more advanced eliminator designed for large feature sets: runs a fast batch phase first, then switches to one-by-one removal with overfitting detection and rollback. Use this when you have 50+ features and want early stopping if val performance drops.
+
+> **RFE vs RecursiveFeatureEliminator**: `RFESelector` is a cross-validated ranker — it repeatedly fits, scores all folds, and drops the lowest-ranked feature. `RecursiveFeatureEliminator` is an adaptive eliminator with a two-phase strategy and overfitting guard — better suited for large feature sets where early stopping matters.
+
+#### Other wrapper methods
+
+- [Feature Combination Searcher](wrapper_methods/combination_search.md) — `FeatureCombinationSearcher`. Exhaustive search over all subsets of a small candidate pool. Use when you want to evaluate every possible combination rather than relying on importance rankings.
+- [Sequential Selection](wrapper_methods/sequential_selection.md) — `SequentialSelector`. Forward, backward, and bidirectional greedy selection.
+- [Bayesian Search](wrapper_methods/bayesian_search.md) — `BayesianFeatureSelector`. Bayesian optimisation over feature subsets.
+- [Boruta](wrapper_methods/boruta.md) — `BorutaSelector`. Shadow-feature statistical testing.
+
+### Embedded Methods
+
+- [Elastic Net](embedded_methods/elastic_net.md) — `ElasticNetLogisticSelector`. Regularization-based selection via L1/L2 penalty.
+
+### Analysis Tools
+
+- [Feature Stability](feature_stability.md) — `FeatureStability`. Measures how consistently features are selected across resampling runs.
+
+### Utils
+
+- [Data Handling](utils/data_handling.md) — CV fold management and data utilities
+- [Scoring](utils/scoring.md) — Custom scoring functions (PR-AUC, TPR, FPR)
+
+## Getting Started
+
+```python
+from classifier_toolkit.feature_selection import (
+    MetaSelector,
+    RFESelector,
+    SequentialSelector,
+)
+
+selector = MetaSelector(
+    model_params=[],
+    rfe_params={},
+    rfe_catboost_params={},
+    sequential_params={},
+    elastic_net_params={},
+    n_features_to_select=5,
+    methods=[
+        RFESelector(estimator_name="lightgbm", n_features_to_select=10),
+        SequentialSelector(
+            estimator_params={},
+            method="forward",
+            estimator_name="lightgbm",
+            n_features_to_select=10,
+        ),
+    ],
+    voting="majority",
+)
+
+X_selected = selector.fit_transform(X, y)
+method_results = selector.get_method_specific_results()
+```
+
+## Best Practices
+
+1. **Run Feature Reduction first** — use the [Feature Reduction](../feature_reduction/overview.md) module to drop low-variance, high-correlation, and drifted features before running selection. Smaller input sets make selection faster and more stable.
+2. **Use cross-validation** — all wrapper methods accept `cv_folds` for robust scoring; avoid fitting on a single split.
+3. **Combine methods with MetaSelector** — consensus features selected by multiple methods are more stable than those selected by one.
+4. **Check stability** — after selection, use `FeatureStability` to verify that selected features remain consistent across resampling.

{classifier_toolkit-0.3.2 → classifier_toolkit-0.3.4}/docs/feature_selection/wrapper_methods/combination_search.md

@@ -32,7 +32,7 @@ print(results.sort_values("val_auc", ascending=False).head())
 
 ## Notes
 
-- Complexity grows as C(n_candidates, k) — keep the candidate pool small (< 15 features)
+- Complexity grows as C(n_candidates, k) — for best performance, keeping the candidate pool under 15 features is recommended, though the searcher works fine with larger pools (20+ features) when runtime is acceptable
 - `required_features` are always included in every combination
 - Supports CatBoost and LightGBM backends
 

classifier_toolkit-0.3.4/docs/index.md

@@ -0,0 +1,92 @@
+# Classifier Toolkit
+
+A comprehensive Python toolkit for the full machine learning workflow — from data preparation to model deployment.
+
+## Overview
+
+Classifier Toolkit is organized into six modules:
+
+- **EDA**: Data inspection, visualization, univariate/bivariate analysis, feature engineering, and automated warnings
+- **Feature Reduction**: Pre-selection filters to remove low-variance, high-correlation, drifted, or counter-intuitive features before feature selection
+- **Feature Selection**: Wrapper, embedded, and ensemble methods for selecting the most predictive features
+- **Model Training**: Hyperparameter tuning for CatBoost and LightGBM via grid search or Bayesian optimization
+- **Data Partition**: Train/test splitting with temporal awareness, preprocessing, and dtype optimization
+- **Explainability**: SHAP-based explanations, interaction analysis, and misclassification diagnostics
+
+## Installation
+
+```bash
+pip install classifier-toolkit
+```
+
+## Quick Start
+
+### EDA
+
+```python
+from classifier_toolkit.eda import EDAToolkit
+
+eda = EDAToolkit(
+    dataframe=df,
+    target_column="target",
+    numerical_columns=["amount", "age"],
+    categorical_columns=["category"],
+)
+
+eda.perform_checks()
+eda.plot_numerical_distributions()
+```
+
+### Feature Reduction
+
+```python
+from classifier_toolkit.feature_reduction import FeatureReducer
+
+reducer = FeatureReducer(df, target_col="target")
+reducer.run_all()
+features_to_drop = reducer.get_features_to_drop()
+```
+
+### Feature Selection
+
+```python
+from classifier_toolkit.feature_selection import RFESelector
+
+selector = RFESelector(
+    estimator_name="lightgbm",
+    n_features_to_select=10,
+    scoring="average_precision",
+)
+
+selector.fit(X_train, y_train, cv_folds=cv_folds)
+print(selector.selected_features_)
+```
+
+### Model Training
+
+```python
+from classifier_toolkit.model_training.hyper_parameter_tuning.tuner import Tuner
+
+tuner = Tuner(
+    X=X_train, y=y_train,
+    model_name="lightgbm",
+    X_val=X_val, y_val=y_val,
+    search_method="bayesian",
+    n_trials=50,
+    optimization_metric="prauc",
+)
+result = tuner.tune()
+best_model = result["best_model"]
+```
+
+## Documentation Structure
+
+- **Guides**: Conceptual explanations and usage examples for each module
+- **API Reference**: Auto-generated documentation for all classes and functions
+- **Examples**: End-to-end notebooks and worked examples
+
+## Links
+
+- [GitHub Repository](https://github.com/qonto-private/classifier-toolkit)
+- [PyPI Package](https://pypi.org/project/classifier-toolkit/)
+- [Issue Tracker](https://github.com/qonto-private/classifier-toolkit/issues)

classifier_toolkit-0.3.4/docs/model_training/overview.md

@@ -0,0 +1,51 @@
+# Model Training Overview
+
+The Model Training module provides hyperparameter tuning for **CatBoost** and **LightGBM** classifiers. It handles the full tuning loop — parameter search, cross-validation, early stopping, and final model refit — so you can focus on defining the search space rather than boilerplate.
+
+## Components
+
+### [Tuner](tuner.md)
+
+The `Tuner` class is the main entry point. It supports two search strategies:
+
+- **Grid search** — exhaustive evaluation of all parameter combinations, with automatic filtering of invalid conditional parameter combinations (e.g. CatBoost `boosting_type="Ordered"` constraints)
+- **Bayesian search** — Optuna-based intelligent sampling, with support for persistent studies and resumption across runs
+
+Both strategies share the same interface and return a unified result object.
+
+## Key Features
+
+- **Unified API** — same `Tuner` constructor and `.tune()` call for grid and Bayesian search, for both CatBoost and LightGBM
+- **Validation modes** — holdout split (`X_val`/`y_val`) or k-fold CV (`cv_folds`), mutually exclusive
+- **Automatic final refit** — the winning model is retrained on all data (train + val), with tree count pinned to the mean `best_iteration` across CV folds to avoid over-training
+- **Conditional parameter handling** — invalid combinations (e.g. `subsample` with `bootstrap_type="Bayesian"`) are filtered automatically; no trial is wasted
+- **Secondary test set** — pass `X_test_v2`/`y_test_v2` to evaluate on a sub-population (e.g. a product segment) alongside the primary test set
+- **Parallel trials** — `parallel_trials > 1` runs grid trials concurrently via `ThreadPoolExecutor`
+
+## Quick Example
+
+```python
+from classifier_toolkit.model_training.hyper_parameter_tuning.tuner import Tuner
+
+tuner = Tuner(
+    X=X_train, y=y_train,
+    model_name="lightgbm",
+    X_val=X_val, y_val=y_val,
+    X_test=X_test, y_test=y_test,
+    search_method="grid",
+    grid_param_values={
+        "learning_rate": [0.05, 0.1],
+        "num_leaves":    [31, 63],
+        "max_depth":     [6, 8],
+    },
+    optimization_metric="prauc",
+)
+result = tuner.tune()
+
+best_model   = result["best_model"]
+best_params  = result["best_params"]
+best_score   = result["best_score"]
+test_metrics = result.get("test_metrics")
+```
+
+See the [Tuner guide](tuner.md) for full parameter documentation, Bayesian search examples, and the complete result object schema.