rapidsegment 1.2.2__tar.gz → 1.2.2.post1__tar.gz

{rapidsegment-1.2.2 → rapidsegment-1.2.2.post1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rapidsegment
-Version: 1.2.2
+Version: 1.2.2.post1
 Summary: A fast, multi-segment population scoring and combinatorial heuristic rule extraction engine.
 Author: Bishwarup Biswas
 Author-email: Bishwarup Biswas <bishwarup1429@gmail.com>
@@ -41,7 +41,7 @@ Description-Content-Type: text/markdown
 > This open‑source library (`RapidSegment`) is an independent, community‑driven predictive analytics framework. It is **completely unaffiliated** with any commercial products, SaaS platforms, or enterprise solutions of the same or similar name. Any overlap in nomenclature is purely coincidental.
 
 <p align="center">
-  <img width="300" height="500" alt="adfd9cdf-251f-44e4-af79-20802d4a7a01" src="https://github.com/user-attachments/assets/803790d0-7ebe-4e79-a415-65f1daaea40b" />
+  <img width="1983" height="793" alt="f09fbbc6-17b6-4e86-96e7-5291a4175456" src="https://github.com/user-attachments/assets/f2584720-246b-4bda-b5a2-1b843bbec474" />
 </p>
 
 # 🚀 RapidSegment – Strategic Segmentation & Scorecard Engine
@@ -71,13 +71,14 @@ Description-Content-Type: text/markdown
 
 ## ✨ Features
 
-- **🔎 Automated Rule Discovery** – Uses Optimal Binning + Apriori pruning to find multi‑way (1‑, 2‑, 3‑way) conditions that maximise lift and volume.
+- **🔎 Automated Rule Discovery** – Uses Optimal Binning (or fast naive quantile binning) + Apriori pruning to find multi‑way (1‑, 2‑, 3‑way) conditions that maximise lift and volume.
 - **🧩 Hierarchical Segments** – Extracts mutually exclusive rules sequentially on a shrinking residual dataset, ensuring clean portfolio decomposition.
-- **⚡ Hyper‑Efficient** – Leverages **DuckDB** for vectorised SQL aggregations and **NumPy** (or DuckDB’s native quantiles) for blazing‑fast scoring.
+- **🔀 Adjacent-Bin Expansion** – Optionally merges neighbouring bins (`max_expansion_hops`) to recover higher-event rules that pure single-bin candidates miss.
+- **⚡ Hyper‑Efficient & Out-of-Core** – Leverages **DuckDB** (disk-backed by default) for vectorised SQL aggregations; spills to disk so large datasets fit in limited RAM.
 - **☁️ BigQuery Ready** – Optional feature screening runs natively inside Google BigQuery, downloading only the most predictive columns.
 - **📦 Production‑Ready Outputs** – Exports pure ANSI SQL filters and a JSON scorecard with decile thresholds, ready for deployment.
 - **📊 Transparent Weighting** – Uses the segment response rate to compute intuitive integer weights, while retaining lift, response rate, and capture rate for each segment.
-- **🔬 Audit Trail** – Built‑in diagnostic (`explain_feature_journey`) to trace any feature’s lifecycle through the extraction process.
+- **🔬 Full Audit Trail** – `explain_feature_journey`, `explain_no_segments`, and `generate_feature_health_report` for complete diagnostics.
 
 ---
 
@@ -134,14 +135,16 @@ builder = StrategicSegmentBuilder(
     top_n_vars=15,
     max_segments=5,
     max_feature_reuse=1,
-    param_grid={"min_sample_size": [1000,2500,5000], "min_lift": [2.0,3.5,5.0]},
+    param_grid={"min_sample_size": [1000, 2500, 5000], "min_lift": [1.5, 2.0, 3.0]},
     enable_diversity=True,
     feature_groups={
         "delinquency": ["max_dpd_12m", "risk_segment"],
         "utilization": ["utilization_avg_3m"]
     },
     ignore_features=["cust_id"],
-    sort_priority="lift_rate_count"
+    sort_priority="rate_lift_count",   # current default
+    binning_method="optimal",          # or "naive"
+    max_expansion_hops=1,              # enable adjacent-bin expansion
 )
 
 # 3. Extract hierarchical segments
@@ -202,7 +205,10 @@ flowchart LR
 ### 🔍 `StrategicSegmentBuilder`
 - **Purpose**: The core segmentation engine. It discovers high‑lift rules using Optimal Binning + Apriori pruning + grid search.
 - **Outputs**: A list of segments, each with a pure ANSI SQL `WHERE` clause, plus metrics (count, rate, lift).
-- **Unique Feature**: Built‑in diagnostic (`explain_feature_journey`) to audit feature usage across iterations.
+- **Diagnostics**
+  - `explain_feature_journey(feature)` – full audit trail of a feature across iterations.
+  - `explain_no_segments()` – human-readable report explaining why extraction stopped early or returned zero segments.
+  - `generate_feature_health_report(data, features)` – DuckDB-native bin-level health report (counts, events, response rate, missing flag).
 
 ### 📊 `StrategicSegmentScore`
 - **Purpose**: Converts binary segment flags into a weighted scorecard with decile thresholds.
@@ -304,6 +310,31 @@ flowchart TD
 ### 1. Feature Ranking & Binning
 Optimal Binning (via `optbinning`) computes the Information Value (IV) for each feature, automatically handling categorical and numerical types. Only the top `top_n_vars` features proceed.
 
+### Naive Binning (Fast Quantile Path)
+
+When `binning_method="naive"`, the engine skips OptBinning and builds bins directly inside DuckDB:
+
+**Numerical features**
+- Compute `naive_bins` quantiles with `QUANTILE_CONT`.
+- Force the outermost edges to `-∞` and `+∞`.
+- Assign every row to a half-open interval: `[lower, upper)`.
+- Nulls go into a dedicated `Missing` bin.
+
+**Categorical features**
+- Each distinct value becomes its own bin: `[value]`.
+- Null / empty / “None” / “nan” values are grouped into `Missing`.
+
+**Why it exists**
+- Extremely fast on large data (pure SQL, no Python loops).
+- Produces stable, equal-frequency bins that are easy to interpret.
+- Works seamlessly with **adjacent-bin expansion** (`max_expansion_hops > 0`), which can later merge neighbouring bins to recover higher-event rules.
+
+**Trade-off**
+- Optimal Binning usually finds slightly more predictive cut-points.
+- Naive binning is preferred when speed or simplicity matters more than maximal IV.
+
+Both paths feed the same downstream pipeline (IV ranking → Apriori → expansion → champion selection).
+
 ### 2. Apriori Pruning
 The engine evaluates combinations in a layered fashion:
 
@@ -376,18 +407,25 @@ Scores are computed as the sum of weights for all segments a customer triggers.
 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
 | `target` | `str` | **Required** | Binary target column name. |
-| `n_jobs` | `int` | `-1` | Number of parallel workers. |
-| `min_sample_size` | `int` | `1000` | Minimum rows for a rule. |
-| `min_lift` | `float` | `2.0` | Minimum lift ratio. |
-| `min_events` | `int` | `5` | Minimum positive events. |
-| `top_n_vars` | `int` | `20` | Number of top‑IV features to consider. |
+| `n_jobs` | `int` | `-1` | Parallel workers for IV/binning (`-1` = all but one core). |
+| `min_sample_size` | `int` | `1000` | Absolute minimum rows for a valid rule. |
+| `min_lift` | `float` | `1.5` | Absolute minimum lift (hard constraint). |
+| `min_events` | `int` | `100` | Minimum positive events for a valid rule. |
+| `top_n_vars` | `int` | `15` | Number of top features passed to the Apriori engine. |
 | `max_segments` | `int` | `10` | Maximum segments to extract. |
-| `max_feature_reuse` | `int` | `1` | How many times a feature can appear. |
-| `enable_diversity` | `bool` | `False` | Prevent same‑group feature combinations. |
-| `enable_1way/2way/3way` | `bool` | `True` | Toggle rule complexity levels. |
-| `feature_groups` | `dict` | `{}` | Business‑category mapping for diversity. |
-| `ignore_features` | `list` | `[]` | Columns to drop before processing. |
-| `sort_priority` | `STR` | `lift_count_rate` | Order in which rules will ranked and selected. Sample capture heavy/Response Rate Heavy/Lift Heavy. |
+| `max_feature_reuse` | `int` | `1` | Max times any single feature may appear across segments. |
+| `param_grid` | `dict` | `{}` | Optional grid of `{min_sample_size, min_lift}` to sweep. |
+| `enable_diversity` | `bool` | `False` | Block combinations that mix features from the same group. |
+| `enable_1way` / `enable_2way` / `enable_3way` | `bool` | `True` | Toggle 1-, 2-, and 3-way rules. |
+| `feature_groups` | `dict` | `{}` | Business-category → column list (used by diversity). |
+| `ignore_features` | `list` | `[]` | Columns to exclude before IV calculation. |
+| `sort_priority` | `str` | `"rate_lift_count"` | Ranking key for champion selection (many variants supported). |
+| `binning_method` | `str` | `"optimal"` | `"optimal"` (OptBinning) or `"naive"` (quantile bins). |
+| `naive_bins` | `int` | `5` | Number of quantile bins when `binning_method="naive"`. |
+| `max_expansion_hops` | `int` | `0` | Adjacent-bin merge distance (0 = disabled). |
+| `selection_metric` | `str` | `"iv"` | Rank features by `"iv"` or `"response_rate"`. |
+| `expand_log_mode` | `str` | `"none"` | Expansion logging: `"none"` \| `"summary"` \| `"champion"` \| `"full"`. |
+| `db_path` / `db_temp_dir` | `str` | `None` | Optional explicit DuckDB file + temp dir (auto-created otherwise). |
 
 **Output** – list of dicts with keys: `segment_id`, `rule_string`, `sql_filter`, `count`, `rate`, `lift`, `meta_applied_sample_size`, `meta_applied_min_lift`.
 
@@ -441,7 +479,7 @@ This project is licensed under the MIT License – see the [LICENSE](LICENSE) fi
 
 ---
 
-**Built with ❤️ by Bishwarup Biswas, Gemini & DeepSeek.**  
+**Built with ❤️ by Bishwarup Biswas**  
 Special Thanks to Mr. [Guillermo Navas Palencia](https://github.com/guillermo-navas-palencia)  for creating [Optbinning](https://github.com/guillermo-navas-palencia/optbinning) library.
 
 _Independent, open‑source, and ready for production._

{rapidsegment-1.2.2 → rapidsegment-1.2.2.post1}/README.md

@@ -4,7 +4,7 @@
 > This open‑source library (`RapidSegment`) is an independent, community‑driven predictive analytics framework. It is **completely unaffiliated** with any commercial products, SaaS platforms, or enterprise solutions of the same or similar name. Any overlap in nomenclature is purely coincidental.
 
 <p align="center">
-  <img width="300" height="500" alt="adfd9cdf-251f-44e4-af79-20802d4a7a01" src="https://github.com/user-attachments/assets/803790d0-7ebe-4e79-a415-65f1daaea40b" />
+  <img width="1983" height="793" alt="f09fbbc6-17b6-4e86-96e7-5291a4175456" src="https://github.com/user-attachments/assets/f2584720-246b-4bda-b5a2-1b843bbec474" />
 </p>
 
 # 🚀 RapidSegment – Strategic Segmentation & Scorecard Engine
@@ -34,13 +34,14 @@
 
 ## ✨ Features
 
-- **🔎 Automated Rule Discovery** – Uses Optimal Binning + Apriori pruning to find multi‑way (1‑, 2‑, 3‑way) conditions that maximise lift and volume.
+- **🔎 Automated Rule Discovery** – Uses Optimal Binning (or fast naive quantile binning) + Apriori pruning to find multi‑way (1‑, 2‑, 3‑way) conditions that maximise lift and volume.
 - **🧩 Hierarchical Segments** – Extracts mutually exclusive rules sequentially on a shrinking residual dataset, ensuring clean portfolio decomposition.
-- **⚡ Hyper‑Efficient** – Leverages **DuckDB** for vectorised SQL aggregations and **NumPy** (or DuckDB’s native quantiles) for blazing‑fast scoring.
+- **🔀 Adjacent-Bin Expansion** – Optionally merges neighbouring bins (`max_expansion_hops`) to recover higher-event rules that pure single-bin candidates miss.
+- **⚡ Hyper‑Efficient & Out-of-Core** – Leverages **DuckDB** (disk-backed by default) for vectorised SQL aggregations; spills to disk so large datasets fit in limited RAM.
 - **☁️ BigQuery Ready** – Optional feature screening runs natively inside Google BigQuery, downloading only the most predictive columns.
 - **📦 Production‑Ready Outputs** – Exports pure ANSI SQL filters and a JSON scorecard with decile thresholds, ready for deployment.
 - **📊 Transparent Weighting** – Uses the segment response rate to compute intuitive integer weights, while retaining lift, response rate, and capture rate for each segment.
-- **🔬 Audit Trail** – Built‑in diagnostic (`explain_feature_journey`) to trace any feature’s lifecycle through the extraction process.
+- **🔬 Full Audit Trail** – `explain_feature_journey`, `explain_no_segments`, and `generate_feature_health_report` for complete diagnostics.
 
 ---
 
@@ -97,14 +98,16 @@ builder = StrategicSegmentBuilder(
     top_n_vars=15,
     max_segments=5,
     max_feature_reuse=1,
-    param_grid={"min_sample_size": [1000,2500,5000], "min_lift": [2.0,3.5,5.0]},
+    param_grid={"min_sample_size": [1000, 2500, 5000], "min_lift": [1.5, 2.0, 3.0]},
     enable_diversity=True,
     feature_groups={
         "delinquency": ["max_dpd_12m", "risk_segment"],
         "utilization": ["utilization_avg_3m"]
     },
     ignore_features=["cust_id"],
-    sort_priority="lift_rate_count"
+    sort_priority="rate_lift_count",   # current default
+    binning_method="optimal",          # or "naive"
+    max_expansion_hops=1,              # enable adjacent-bin expansion
 )
 
 # 3. Extract hierarchical segments
@@ -165,7 +168,10 @@ flowchart LR
 ### 🔍 `StrategicSegmentBuilder`
 - **Purpose**: The core segmentation engine. It discovers high‑lift rules using Optimal Binning + Apriori pruning + grid search.
 - **Outputs**: A list of segments, each with a pure ANSI SQL `WHERE` clause, plus metrics (count, rate, lift).
-- **Unique Feature**: Built‑in diagnostic (`explain_feature_journey`) to audit feature usage across iterations.
+- **Diagnostics**
+  - `explain_feature_journey(feature)` – full audit trail of a feature across iterations.
+  - `explain_no_segments()` – human-readable report explaining why extraction stopped early or returned zero segments.
+  - `generate_feature_health_report(data, features)` – DuckDB-native bin-level health report (counts, events, response rate, missing flag).
 
 ### 📊 `StrategicSegmentScore`
 - **Purpose**: Converts binary segment flags into a weighted scorecard with decile thresholds.
@@ -267,6 +273,31 @@ flowchart TD
 ### 1. Feature Ranking & Binning
 Optimal Binning (via `optbinning`) computes the Information Value (IV) for each feature, automatically handling categorical and numerical types. Only the top `top_n_vars` features proceed.
 
+### Naive Binning (Fast Quantile Path)
+
+When `binning_method="naive"`, the engine skips OptBinning and builds bins directly inside DuckDB:
+
+**Numerical features**
+- Compute `naive_bins` quantiles with `QUANTILE_CONT`.
+- Force the outermost edges to `-∞` and `+∞`.
+- Assign every row to a half-open interval: `[lower, upper)`.
+- Nulls go into a dedicated `Missing` bin.
+
+**Categorical features**
+- Each distinct value becomes its own bin: `[value]`.
+- Null / empty / “None” / “nan” values are grouped into `Missing`.
+
+**Why it exists**
+- Extremely fast on large data (pure SQL, no Python loops).
+- Produces stable, equal-frequency bins that are easy to interpret.
+- Works seamlessly with **adjacent-bin expansion** (`max_expansion_hops > 0`), which can later merge neighbouring bins to recover higher-event rules.
+
+**Trade-off**
+- Optimal Binning usually finds slightly more predictive cut-points.
+- Naive binning is preferred when speed or simplicity matters more than maximal IV.
+
+Both paths feed the same downstream pipeline (IV ranking → Apriori → expansion → champion selection).
+
 ### 2. Apriori Pruning
 The engine evaluates combinations in a layered fashion:
 
@@ -339,18 +370,25 @@ Scores are computed as the sum of weights for all segments a customer triggers.
 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
 | `target` | `str` | **Required** | Binary target column name. |
-| `n_jobs` | `int` | `-1` | Number of parallel workers. |
-| `min_sample_size` | `int` | `1000` | Minimum rows for a rule. |
-| `min_lift` | `float` | `2.0` | Minimum lift ratio. |
-| `min_events` | `int` | `5` | Minimum positive events. |
-| `top_n_vars` | `int` | `20` | Number of top‑IV features to consider. |
+| `n_jobs` | `int` | `-1` | Parallel workers for IV/binning (`-1` = all but one core). |
+| `min_sample_size` | `int` | `1000` | Absolute minimum rows for a valid rule. |
+| `min_lift` | `float` | `1.5` | Absolute minimum lift (hard constraint). |
+| `min_events` | `int` | `100` | Minimum positive events for a valid rule. |
+| `top_n_vars` | `int` | `15` | Number of top features passed to the Apriori engine. |
 | `max_segments` | `int` | `10` | Maximum segments to extract. |
-| `max_feature_reuse` | `int` | `1` | How many times a feature can appear. |
-| `enable_diversity` | `bool` | `False` | Prevent same‑group feature combinations. |
-| `enable_1way/2way/3way` | `bool` | `True` | Toggle rule complexity levels. |
-| `feature_groups` | `dict` | `{}` | Business‑category mapping for diversity. |
-| `ignore_features` | `list` | `[]` | Columns to drop before processing. |
-| `sort_priority` | `STR` | `lift_count_rate` | Order in which rules will ranked and selected. Sample capture heavy/Response Rate Heavy/Lift Heavy. |
+| `max_feature_reuse` | `int` | `1` | Max times any single feature may appear across segments. |
+| `param_grid` | `dict` | `{}` | Optional grid of `{min_sample_size, min_lift}` to sweep. |
+| `enable_diversity` | `bool` | `False` | Block combinations that mix features from the same group. |
+| `enable_1way` / `enable_2way` / `enable_3way` | `bool` | `True` | Toggle 1-, 2-, and 3-way rules. |
+| `feature_groups` | `dict` | `{}` | Business-category → column list (used by diversity). |
+| `ignore_features` | `list` | `[]` | Columns to exclude before IV calculation. |
+| `sort_priority` | `str` | `"rate_lift_count"` | Ranking key for champion selection (many variants supported). |
+| `binning_method` | `str` | `"optimal"` | `"optimal"` (OptBinning) or `"naive"` (quantile bins). |
+| `naive_bins` | `int` | `5` | Number of quantile bins when `binning_method="naive"`. |
+| `max_expansion_hops` | `int` | `0` | Adjacent-bin merge distance (0 = disabled). |
+| `selection_metric` | `str` | `"iv"` | Rank features by `"iv"` or `"response_rate"`. |
+| `expand_log_mode` | `str` | `"none"` | Expansion logging: `"none"` \| `"summary"` \| `"champion"` \| `"full"`. |
+| `db_path` / `db_temp_dir` | `str` | `None` | Optional explicit DuckDB file + temp dir (auto-created otherwise). |
 
 **Output** – list of dicts with keys: `segment_id`, `rule_string`, `sql_filter`, `count`, `rate`, `lift`, `meta_applied_sample_size`, `meta_applied_min_lift`.
 
@@ -404,7 +442,7 @@ This project is licensed under the MIT License – see the [LICENSE](LICENSE) fi
 
 ---
 
-**Built with ❤️ by Bishwarup Biswas, Gemini & DeepSeek.**  
+**Built with ❤️ by Bishwarup Biswas**  
 Special Thanks to Mr. [Guillermo Navas Palencia](https://github.com/guillermo-navas-palencia)  for creating [Optbinning](https://github.com/guillermo-navas-palencia/optbinning) library.
 
 _Independent, open‑source, and ready for production._

{rapidsegment-1.2.2 → rapidsegment-1.2.2.post1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "rapidsegment"
-version = "1.2.2"
+version = "1.2.2.post1"
 description = "A fast, multi-segment population scoring and combinatorial heuristic rule extraction engine."
 readme = "README.md"
 license = "MIT"

{rapidsegment-1.2.2 → rapidsegment-1.2.2.post1}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "rapidsegment"
-version = "1.2.2"
+version = "1.2.2.post1"
 description = "A fast, multi-segment population scoring and combinatorial heuristic rule extraction engine."
 readme = "README.md"
 authors = [

{rapidsegment-1.2.2 → rapidsegment-1.2.2.post1}/src/rapidsegment/builder.py

@@ -95,7 +95,8 @@ class StrategicSegmentBuilder:
         feature_groups: Optional[Dict[str, List[str]]] = None,
         ignore_features: Optional[List[str]] = None,
         sort_priority: str = "rate_lift_count",
-        binning_method: str = "optimal_cart",  # "naive" | "optimal_cart" | "optimal_quantile" | "optimal" (alias of cart)        naive_bins: int = 5 ,
+        binning_method: str = "optimal_cart",  # "naive" | "optimal_cart" | "optimal_quantile" | "optimal" (alias of cart)        
+        naive_bins: int = 5 ,
         max_expansion_hops: int = 0,
         selection_metric: str = "iv",
         expand_log_mode: str = "none",
@@ -1964,4 +1965,4 @@ class StrategicSegmentBuilder:
                 })
     
         health_report_df = pd.DataFrame(rows)
-        return health_report_df
+        return health_report_df