gitlabds 2.4.1__tar.gz → 2.4.3__tar.gz

{gitlabds-2.4.1 → gitlabds-2.4.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitlabds
-Version: 2.4.1
+Version: 2.4.3
 Summary: GitLab Data Science Tools
 Author-email: Kevin Dietz <kdietz@gitlab.com>
 License-Expression: MIT
@@ -914,7 +914,7 @@ shap_df = gitlabds.per_record_shap(
 #### Description
 "Change X -> +Y" counterfactual search, direction chosen empirically. For each actionable lever SHAP surfaced for a record, evaluates candidate values drawn from peer-group quantiles, re-scores the model at each, and keeps the single move that maximizes `gain` — the improvement in the model score, signed by `score_goal` so that bigger is always better. No monotonicity assumption is made.
 
-`gitlabds.per_record_counterfactuals(model, df, shap_df, unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, score_goal="maximize", actionable_values=["Yes", "Partial"], min_gain=0.005, impact_medium=0.01, impact_high=0.03, levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5):`
+`gitlabds.per_record_counterfactuals(model, df, shap_df, unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, score_goal="maximize", actionable_values=["Yes", "Partial"], min_gain=0.005, impact_medium=0.01, impact_high=0.03, levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5, prefer_nearest_within=None):`
 
 #### Parameters:
 - _**model**_ : trained model (as in `per_record_shap`). How probabilities are scored is chosen from the model object itself — `.predict_proba()` where available, otherwise a raw `xgboost.Booster` or `lightgbm.Booster` is scored natively. An unsupported type raises a `TypeError` naming it
@@ -926,10 +926,11 @@ shap_df = gitlabds.per_record_shap(
 - _**direction**_ : optional per-feature constraint mapping feature -> `'increasing'` | `'decreasing'` | `'both'` | `'none'` (dict or Series). `'increasing'`/`'decreasing'` restrict candidates to moves in that direction (an achievability constraint); `'both'` allows either; `'none'` excludes the feature from lever evaluation entirely, for features with no meaningful good direction (categoricals, identifiers, context-only numerics). Features absent from the mapping, or `direction=None`, default to `'both'`. A blank/NaN raises — normalize it to `'none'` where the feature dictionary is read
 - _**score_goal**_ : `"maximize"` if a higher predicted probability of class 1 is the good outcome (e.g. class 1 = uptier), `"minimize"` if it is the bad one (e.g. class 1 = churn). Sets the sign of `gain`, which drives best-move selection, the `min_gain` filter, `impact_bucket`, and the output ordering. This is the outcome's polarity only — `direction` above is a separate, feature-level concept and is unaffected. Default is `"maximize"`
 - _**actionable_values**_ : values in `shap_df["actionable"]` treated as actionable levers
-- _**min_gain**_ : drop levers whose best achievable gain is below this. Note this is a threshold on `gain`, not on raw `delta_p`: under `score_goal="minimize"` a good move has a negative `delta_p` but a positive `gain`, so this stays positive for both polarities. Negative values (or `-inf`) are allowed, and keep moves that make the outcome worse — useful for inspecting the full candidate distribution while tuning thresholds
+- _**min_gain**_ : drop levers whose best achievable gain is below this. Note this is a threshold on `gain`, not on raw `delta_p`: under `score_goal="minimize"` a good move has a negative `delta_p` but a positive `gain`, so this stays positive for both polarities. Negative values (or `-inf`) are allowed, and keep moves that make the outcome worse — useful for inspecting the full candidate distribution while tuning thresholds. With `prefer_nearest_within` set, selection runs first, so this threshold applies to the gain of the chosen (nearer, lower-gain) candidate rather than the lever's best — a lever whose best gain clears `min_gain` can still be dropped on the nearer move it settled for
 - _**impact_medium, impact_high**_ : `gain` thresholds bucketing each lever into Low/Medium/High `impact_bucket`. Only meaningful for `gain > 0` — under a negative `min_gain` the retained harmful moves stay `"Low"` (the level-crossing floor doesn't apply to them), so read `gain`'s sign rather than the bucket alone
 - _**levels**_ : dict mapping label -> upper bound on P(class 1) for that level (e.g., default `{"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}` means `p <= 0.03` -> "poor fit", `0.03 < p <= 0.10` -> "good fit", `p > 0.10` -> "best fit"). Key order doesn't matter — levels are sorted by cutoff and labels follow that order — but the top level's cutoff must be `float("inf")`. Cutoffs are always in P(class 1) space regardless of `score_goal`, and `crosses_level` is a plain inequality, so name the labels for your own outcome: a churn model would pass something like `{"low risk": 0.03, "medium risk": 0.10, "high risk": float("inf")}`. Used to detect whether a move crosses a level boundary (any crossing move is floored at "Medium" impact)
 - _**n_bins**_ : quantile bins for numeric `peer_group` columns
+- _**prefer_nearest_within**_ : `None` (default) keeps the highest-gain candidate per (record, feature). A float in `(0, 1]` instead keeps the *nearest* candidate that still captures that fraction of the best gain: among candidates with `gain >= prefer_nearest_within * best_gain`, the one with the smallest `abs(intervention_value - current_value)` wins, ties broken on higher gain. 
 
 #### Returns
 - Long DataFrame, one row per (record, lever): `current_value`, `intervention_value`, `intervention_quantile`, `p_before`, `p_after`, `delta_p`, `gain`, `impact_bucket`, `level_before`, `level_after`, `crosses_level`
@@ -960,6 +961,16 @@ counterfactuals_df = gitlabds.per_record_counterfactuals(
     direction=dict(zip(feature_dict["field"], feature_dict["good_direction"])),
     levels={"low risk": 0.03, "medium risk": 0.10, "high risk": float("inf")},
 )
+
+# Prefer a nearer, more sellable move: the closest candidate that still captures
+# at least 70% of the best available gain.
+counterfactuals_df = gitlabds.per_record_counterfactuals(
+    model=trained_model,
+    df=model_features,
+    shap_df=shap_df,
+    peer_group=peer_group_df,
+    prefer_nearest_within=0.7,
+)
 ```
 
 </details>

{gitlabds-2.4.1 → gitlabds-2.4.3}/README.md

@@ -878,7 +878,7 @@ shap_df = gitlabds.per_record_shap(
 #### Description
 "Change X -> +Y" counterfactual search, direction chosen empirically. For each actionable lever SHAP surfaced for a record, evaluates candidate values drawn from peer-group quantiles, re-scores the model at each, and keeps the single move that maximizes `gain` — the improvement in the model score, signed by `score_goal` so that bigger is always better. No monotonicity assumption is made.
 
-`gitlabds.per_record_counterfactuals(model, df, shap_df, unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, score_goal="maximize", actionable_values=["Yes", "Partial"], min_gain=0.005, impact_medium=0.01, impact_high=0.03, levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5):`
+`gitlabds.per_record_counterfactuals(model, df, shap_df, unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, score_goal="maximize", actionable_values=["Yes", "Partial"], min_gain=0.005, impact_medium=0.01, impact_high=0.03, levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5, prefer_nearest_within=None):`
 
 #### Parameters:
 - _**model**_ : trained model (as in `per_record_shap`). How probabilities are scored is chosen from the model object itself — `.predict_proba()` where available, otherwise a raw `xgboost.Booster` or `lightgbm.Booster` is scored natively. An unsupported type raises a `TypeError` naming it
@@ -890,10 +890,11 @@ shap_df = gitlabds.per_record_shap(
 - _**direction**_ : optional per-feature constraint mapping feature -> `'increasing'` | `'decreasing'` | `'both'` | `'none'` (dict or Series). `'increasing'`/`'decreasing'` restrict candidates to moves in that direction (an achievability constraint); `'both'` allows either; `'none'` excludes the feature from lever evaluation entirely, for features with no meaningful good direction (categoricals, identifiers, context-only numerics). Features absent from the mapping, or `direction=None`, default to `'both'`. A blank/NaN raises — normalize it to `'none'` where the feature dictionary is read
 - _**score_goal**_ : `"maximize"` if a higher predicted probability of class 1 is the good outcome (e.g. class 1 = uptier), `"minimize"` if it is the bad one (e.g. class 1 = churn). Sets the sign of `gain`, which drives best-move selection, the `min_gain` filter, `impact_bucket`, and the output ordering. This is the outcome's polarity only — `direction` above is a separate, feature-level concept and is unaffected. Default is `"maximize"`
 - _**actionable_values**_ : values in `shap_df["actionable"]` treated as actionable levers
-- _**min_gain**_ : drop levers whose best achievable gain is below this. Note this is a threshold on `gain`, not on raw `delta_p`: under `score_goal="minimize"` a good move has a negative `delta_p` but a positive `gain`, so this stays positive for both polarities. Negative values (or `-inf`) are allowed, and keep moves that make the outcome worse — useful for inspecting the full candidate distribution while tuning thresholds
+- _**min_gain**_ : drop levers whose best achievable gain is below this. Note this is a threshold on `gain`, not on raw `delta_p`: under `score_goal="minimize"` a good move has a negative `delta_p` but a positive `gain`, so this stays positive for both polarities. Negative values (or `-inf`) are allowed, and keep moves that make the outcome worse — useful for inspecting the full candidate distribution while tuning thresholds. With `prefer_nearest_within` set, selection runs first, so this threshold applies to the gain of the chosen (nearer, lower-gain) candidate rather than the lever's best — a lever whose best gain clears `min_gain` can still be dropped on the nearer move it settled for
 - _**impact_medium, impact_high**_ : `gain` thresholds bucketing each lever into Low/Medium/High `impact_bucket`. Only meaningful for `gain > 0` — under a negative `min_gain` the retained harmful moves stay `"Low"` (the level-crossing floor doesn't apply to them), so read `gain`'s sign rather than the bucket alone
 - _**levels**_ : dict mapping label -> upper bound on P(class 1) for that level (e.g., default `{"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}` means `p <= 0.03` -> "poor fit", `0.03 < p <= 0.10` -> "good fit", `p > 0.10` -> "best fit"). Key order doesn't matter — levels are sorted by cutoff and labels follow that order — but the top level's cutoff must be `float("inf")`. Cutoffs are always in P(class 1) space regardless of `score_goal`, and `crosses_level` is a plain inequality, so name the labels for your own outcome: a churn model would pass something like `{"low risk": 0.03, "medium risk": 0.10, "high risk": float("inf")}`. Used to detect whether a move crosses a level boundary (any crossing move is floored at "Medium" impact)
 - _**n_bins**_ : quantile bins for numeric `peer_group` columns
+- _**prefer_nearest_within**_ : `None` (default) keeps the highest-gain candidate per (record, feature). A float in `(0, 1]` instead keeps the *nearest* candidate that still captures that fraction of the best gain: among candidates with `gain >= prefer_nearest_within * best_gain`, the one with the smallest `abs(intervention_value - current_value)` wins, ties broken on higher gain. 
 
 #### Returns
 - Long DataFrame, one row per (record, lever): `current_value`, `intervention_value`, `intervention_quantile`, `p_before`, `p_after`, `delta_p`, `gain`, `impact_bucket`, `level_before`, `level_after`, `crosses_level`
@@ -924,6 +925,16 @@ counterfactuals_df = gitlabds.per_record_counterfactuals(
     direction=dict(zip(feature_dict["field"], feature_dict["good_direction"])),
     levels={"low risk": 0.03, "medium risk": 0.10, "high risk": float("inf")},
 )
+
+# Prefer a nearer, more sellable move: the closest candidate that still captures
+# at least 70% of the best available gain.
+counterfactuals_df = gitlabds.per_record_counterfactuals(
+    model=trained_model,
+    df=model_features,
+    shap_df=shap_df,
+    peer_group=peer_group_df,
+    prefer_nearest_within=0.7,
+)
 ```
 
 </details>

{gitlabds-2.4.1 → gitlabds-2.4.3}/gitlabds/record_benchmarking.py

@@ -297,6 +297,7 @@ def per_record_counterfactuals(
     impact_high=0.03,
     levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")},
     n_bins=5,
+    prefer_nearest_within=None,
 ):
     """
     "change X -> +Y", direction chosen empirically.
@@ -335,6 +336,10 @@ def per_record_counterfactuals(
         a negative delta_p but a positive gain, so this stays positive for both polarities.
         Negative values (or -inf) are allowed, and keep moves that make the outcome worse --
         useful for inspecting the full candidate distribution while tuning thresholds.
+        With prefer_nearest_within set, selection runs first, so this threshold applies to
+        the gain of the chosen (nearer, lower-gain) candidate rather than the lever's best:
+        a lever whose best gain clears min_gain can still be dropped on the nearer move it
+        settled for.
     impact_medium, impact_high : gain thresholds bucketing each lever into
         Low/Medium/High impact_bucket. gain shrinks mechanically near the tails of
         the probability distribution, so a fixed pp threshold alone under-weights
@@ -360,6 +365,11 @@ def per_record_counterfactuals(
         "high risk": float("inf")}. Used only to detect level crossings for impact_bucket,
         not to reproduce your production leveling logic.
     n_bins : quantile bins for numeric peer_group columns.
+    prefer_nearest_within : None (default) keeps the highest-gain candidate per (record,
+        feature) -- the historical behavior. A float in (0, 1] instead keeps the *nearest*
+        candidate that still captures that fraction of the best gain: among candidates with
+        gain >= prefer_nearest_within * best_gain, the one with the smallest
+        abs(intervention_value - current_value) wins, ties broken on higher gain. 
 
     Returns a long DataFrame: one row per (record, lever) with the best move found
     (current_value, intervention_value, intervention_quantile, p_before/p_after/delta_p,
@@ -368,6 +378,13 @@ def per_record_counterfactuals(
     """
     if impact_medium >= impact_high:
         raise ValueError(f"impact_medium ({impact_medium}) must be less than impact_high ({impact_high})")
+    if prefer_nearest_within is not None:
+        prefer_nearest_within = float(prefer_nearest_within)
+        if not 0 < prefer_nearest_within <= 1:
+            raise ValueError(
+                "prefer_nearest_within must be greater than 0 and at most 1 (the fraction of "
+                f"the best gain a nearer candidate must still capture), got {prefer_nearest_within}"
+            )
     _score_goal = str(score_goal).strip().lower()
     if _score_goal not in ("maximize", "minimize"):
         raise ValueError(f"score_goal must be 'maximize' or 'minimize', got {score_goal!r}")
@@ -442,9 +459,23 @@ def per_record_counterfactuals(
         return pd.DataFrame()
     out = pd.concat(parts, ignore_index=True)
     out["gain"] = sign * out["delta_p"]
-    # Keep the single best-scoring candidate per (record, feature).
-    out = (out.sort_values("gain", ascending=False)
-              .drop_duplicates([unique_id, "feature"], keep="first"))
+    # Keep one candidate per (record, feature): the best-scoring one, or -- with
+    # prefer_nearest_within -- the nearest one that still captures that share of the best
+    # gain. Done before the min_gain filter and the impact/level logic below, so all of
+    # those describe the move actually chosen.
+    if prefer_nearest_within is None:
+        out = (out.sort_values("gain", ascending=False)
+                  .drop_duplicates([unique_id, "feature"], keep="first"))
+    else:
+        best = out.groupby([unique_id, "feature"], observed=True)["gain"].transform("max")
+        # Scaling only makes sense for a positive best gain: at best_gain <= 0, multiplying
+        # by a fraction raises the bar above the best candidate and would empty the group.
+        out = out[out["gain"] >= np.where(best > 0, prefer_nearest_within * best, best)]
+        distance = (out["intervention_value"] - out["current_value"]).abs()
+        out = (out.assign(_distance=distance)
+                  .sort_values(["_distance", "gain"], ascending=[True, False])
+                  .drop_duplicates([unique_id, "feature"], keep="first")
+                  .drop(columns="_distance"))
     out = out[out["gain"] >= min_gain]
     out["impact_bucket"] = pd.cut(out["gain"],
                                    bins=[-np.inf, impact_medium, impact_high, np.inf],

{gitlabds-2.4.1 → gitlabds-2.4.3}/gitlabds/split_data.py

@@ -140,7 +140,6 @@ def split_data(
         train_df, test_df = train_test_split(
             df_copy,
             train_size=train_pct,
-            test_size=1-train_pct,
             random_state=random_state,
             shuffle=shuffle,
             stratify=stratify_param
@@ -241,7 +240,6 @@ def split_data(
         train_groups, test_groups = train_test_split(
             group_target.index.values,
             train_size=train_pct,
-            test_size=1 - train_pct,
             random_state=random_state,
             shuffle=shuffle,
             stratify=group_stratify,
@@ -263,7 +261,6 @@ def split_data(
         X_train, X_test, y_train, y_test = train_test_split(
             X, y,
             train_size=train_pct,
-            test_size=1-train_pct,
             random_state=random_state,
             shuffle=shuffle,
             stratify=stratify_param

{gitlabds-2.4.1 → gitlabds-2.4.3}/gitlabds.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitlabds
-Version: 2.4.1
+Version: 2.4.3
 Summary: GitLab Data Science Tools
 Author-email: Kevin Dietz <kdietz@gitlab.com>
 License-Expression: MIT
@@ -914,7 +914,7 @@ shap_df = gitlabds.per_record_shap(
 #### Description
 "Change X -> +Y" counterfactual search, direction chosen empirically. For each actionable lever SHAP surfaced for a record, evaluates candidate values drawn from peer-group quantiles, re-scores the model at each, and keeps the single move that maximizes `gain` — the improvement in the model score, signed by `score_goal` so that bigger is always better. No monotonicity assumption is made.
 
-`gitlabds.per_record_counterfactuals(model, df, shap_df, unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, score_goal="maximize", actionable_values=["Yes", "Partial"], min_gain=0.005, impact_medium=0.01, impact_high=0.03, levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5):`
+`gitlabds.per_record_counterfactuals(model, df, shap_df, unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, score_goal="maximize", actionable_values=["Yes", "Partial"], min_gain=0.005, impact_medium=0.01, impact_high=0.03, levels={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5, prefer_nearest_within=None):`
 
 #### Parameters:
 - _**model**_ : trained model (as in `per_record_shap`). How probabilities are scored is chosen from the model object itself — `.predict_proba()` where available, otherwise a raw `xgboost.Booster` or `lightgbm.Booster` is scored natively. An unsupported type raises a `TypeError` naming it
@@ -926,10 +926,11 @@ shap_df = gitlabds.per_record_shap(
 - _**direction**_ : optional per-feature constraint mapping feature -> `'increasing'` | `'decreasing'` | `'both'` | `'none'` (dict or Series). `'increasing'`/`'decreasing'` restrict candidates to moves in that direction (an achievability constraint); `'both'` allows either; `'none'` excludes the feature from lever evaluation entirely, for features with no meaningful good direction (categoricals, identifiers, context-only numerics). Features absent from the mapping, or `direction=None`, default to `'both'`. A blank/NaN raises — normalize it to `'none'` where the feature dictionary is read
 - _**score_goal**_ : `"maximize"` if a higher predicted probability of class 1 is the good outcome (e.g. class 1 = uptier), `"minimize"` if it is the bad one (e.g. class 1 = churn). Sets the sign of `gain`, which drives best-move selection, the `min_gain` filter, `impact_bucket`, and the output ordering. This is the outcome's polarity only — `direction` above is a separate, feature-level concept and is unaffected. Default is `"maximize"`
 - _**actionable_values**_ : values in `shap_df["actionable"]` treated as actionable levers
-- _**min_gain**_ : drop levers whose best achievable gain is below this. Note this is a threshold on `gain`, not on raw `delta_p`: under `score_goal="minimize"` a good move has a negative `delta_p` but a positive `gain`, so this stays positive for both polarities. Negative values (or `-inf`) are allowed, and keep moves that make the outcome worse — useful for inspecting the full candidate distribution while tuning thresholds
+- _**min_gain**_ : drop levers whose best achievable gain is below this. Note this is a threshold on `gain`, not on raw `delta_p`: under `score_goal="minimize"` a good move has a negative `delta_p` but a positive `gain`, so this stays positive for both polarities. Negative values (or `-inf`) are allowed, and keep moves that make the outcome worse — useful for inspecting the full candidate distribution while tuning thresholds. With `prefer_nearest_within` set, selection runs first, so this threshold applies to the gain of the chosen (nearer, lower-gain) candidate rather than the lever's best — a lever whose best gain clears `min_gain` can still be dropped on the nearer move it settled for
 - _**impact_medium, impact_high**_ : `gain` thresholds bucketing each lever into Low/Medium/High `impact_bucket`. Only meaningful for `gain > 0` — under a negative `min_gain` the retained harmful moves stay `"Low"` (the level-crossing floor doesn't apply to them), so read `gain`'s sign rather than the bucket alone
 - _**levels**_ : dict mapping label -> upper bound on P(class 1) for that level (e.g., default `{"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}` means `p <= 0.03` -> "poor fit", `0.03 < p <= 0.10` -> "good fit", `p > 0.10` -> "best fit"). Key order doesn't matter — levels are sorted by cutoff and labels follow that order — but the top level's cutoff must be `float("inf")`. Cutoffs are always in P(class 1) space regardless of `score_goal`, and `crosses_level` is a plain inequality, so name the labels for your own outcome: a churn model would pass something like `{"low risk": 0.03, "medium risk": 0.10, "high risk": float("inf")}`. Used to detect whether a move crosses a level boundary (any crossing move is floored at "Medium" impact)
 - _**n_bins**_ : quantile bins for numeric `peer_group` columns
+- _**prefer_nearest_within**_ : `None` (default) keeps the highest-gain candidate per (record, feature). A float in `(0, 1]` instead keeps the *nearest* candidate that still captures that fraction of the best gain: among candidates with `gain >= prefer_nearest_within * best_gain`, the one with the smallest `abs(intervention_value - current_value)` wins, ties broken on higher gain. 
 
 #### Returns
 - Long DataFrame, one row per (record, lever): `current_value`, `intervention_value`, `intervention_quantile`, `p_before`, `p_after`, `delta_p`, `gain`, `impact_bucket`, `level_before`, `level_after`, `crosses_level`
@@ -960,6 +961,16 @@ counterfactuals_df = gitlabds.per_record_counterfactuals(
     direction=dict(zip(feature_dict["field"], feature_dict["good_direction"])),
     levels={"low risk": 0.03, "medium risk": 0.10, "high risk": float("inf")},
 )
+
+# Prefer a nearer, more sellable move: the closest candidate that still captures
+# at least 70% of the best available gain.
+counterfactuals_df = gitlabds.per_record_counterfactuals(
+    model=trained_model,
+    df=model_features,
+    shap_df=shap_df,
+    peer_group=peer_group_df,
+    prefer_nearest_within=0.7,
+)
 ```
 
 </details>

{gitlabds-2.4.1 → gitlabds-2.4.3}/gitlabds.egg-info/SOURCES.txt

@@ -25,4 +25,6 @@ gitlabds.egg-info/dependency_links.txt
 gitlabds.egg-info/requires.txt
 gitlabds.egg-info/top_level.txt
 tests/test_feature_reduction.py
-tests/test_snowflake_connectors.py
+tests/test_record_benchmarking.py
+tests/test_snowflake_connectors.py
+tests/test_split_data.py

gitlabds-2.4.3/tests/test_record_benchmarking.py

@@ -0,0 +1,182 @@
+"""
+Unit tests for gitlabds.record_benchmarking. No model is trained and `shap` is never
+called: `per_record_counterfactuals` only needs an object exposing .predict_proba() and a
+shap_df-shaped frame naming the (record, feature) levers to test, so both are faked here
+with a closed-form probability that makes each candidate's gain predictable.
+
+Run with:  pytest tests/test_record_benchmarking.py
+"""
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from gitlabds.record_benchmarking import per_record_counterfactuals
+
+
+# ---------------------------------------------------------------------------
+# Fixtures and fakes
+# ---------------------------------------------------------------------------
+
+
+class RampModel:
+    """
+    P(class 1) = base + slope * f, i.e. monotonic in the single feature "f" with
+    diminishing *relative* returns per unit moved. Lets each candidate quantile's gain be
+    computed by hand, so a test can assert which move the selection logic picked.
+    """
+
+    def __init__(self, base=0.0, slope=0.05):
+        self.base = base
+        self.slope = slope
+
+    def predict_proba(self, X):
+        p1 = np.clip(self.base + self.slope * np.asarray(X["f"], dtype=float), 0.0, 1.0)
+        return np.column_stack([1 - p1, p1])
+
+
+@pytest.fixture
+def df():
+    """f = 0..9, so the global peer quantiles are p25=2.25, p50=4.5, p90=8.1."""
+    return pd.DataFrame(
+        {"f": np.arange(10, dtype=float)},
+        index=pd.Index([f"r{i}" for i in range(10)], name="record_id"),
+    )
+
+
+def make_shap_df(records, feature="f"):
+    """Minimal stand-in for per_record_shap output: enough to scope the levers."""
+    return pd.DataFrame({
+        "record_id": list(records),
+        "feature": feature,
+        "actionable": "Yes",
+    })
+
+
+def run(df, records, **kwargs):
+    return per_record_counterfactuals(
+        model=RampModel(),
+        df=df,
+        shap_df=make_shap_df(records),
+        candidate_quantiles=[0.25, 0.5, 0.9],
+        **kwargs,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Default behavior
+# ---------------------------------------------------------------------------
+
+
+class TestDefaultSelection:
+    def test_keeps_the_highest_gain_candidate(self, df):
+        # r0 (f=0) can move to 2.25, 4.5 or 8.1; the ramp makes the farthest the best.
+        out = run(df, ["r0"])
+        assert len(out) == 1
+        row = out.iloc[0]
+        assert row["intervention_quantile"] == 0.9
+        assert row["intervention_value"] == pytest.approx(8.1)
+        assert row["gain"] == pytest.approx(0.405)
+
+    def test_prefer_nearest_within_none_matches_the_default(self, df):
+        records = ["r0", "r1", "r2"]
+        pd.testing.assert_frame_equal(
+            run(df, records),
+            run(df, records, prefer_nearest_within=None),
+        )
+
+
+# ---------------------------------------------------------------------------
+# prefer_nearest_within
+# ---------------------------------------------------------------------------
+
+
+class TestPreferNearestWithin:
+    def test_nearer_candidate_within_the_threshold_wins(self, df):
+        # Gains for r0 are 0.1125 (2.25), 0.225 (4.5) and 0.405 (8.1). At 0.5, only the
+        # last two clear 0.2025, so the nearer of those -- the median -- is chosen.
+        out = run(df, ["r0"], prefer_nearest_within=0.5)
+        assert len(out) == 1
+        row = out.iloc[0]
+        assert row["intervention_quantile"] == 0.5
+        assert row["intervention_value"] == pytest.approx(4.5)
+        assert row["gain"] == pytest.approx(0.225)
+
+    def test_threshold_too_high_leaves_the_best_gain_move(self, df):
+        # 0.9 * 0.405 = 0.3645, which only the best candidate clears.
+        out = run(df, ["r0"], prefer_nearest_within=0.9)
+        assert out.iloc[0]["intervention_value"] == pytest.approx(8.1)
+
+    def test_lower_threshold_reaches_a_nearer_candidate(self, df):
+        # 0.25 * 0.405 = 0.101, low enough to admit p25 as well.
+        out = run(df, ["r0"], prefer_nearest_within=0.25)
+        assert out.iloc[0]["intervention_value"] == pytest.approx(2.25)
+
+    def test_one_row_per_record_feature_like_the_default(self, df):
+        records = ["r0", "r1", "r2", "r3"]
+        default = run(df, records)
+        nearest = run(df, records, prefer_nearest_within=0.5)
+        assert list(nearest.columns) == list(default.columns)
+        assert set(nearest["record_id"]) == set(default["record_id"])
+        assert not nearest.duplicated(["record_id", "feature"]).any()
+
+    def test_impact_bucket_describes_the_chosen_move(self, df):
+        # The bucket is computed after selection, so it reflects the nearer move's gain --
+        # here a full bucket below what the default pick would have earned. A single level
+        # keeps the level-crossing floor from promoting it, so the bucket is the gain's.
+        kwargs = dict(impact_medium=0.25, impact_high=0.35, levels={"same level": float("inf")})
+        out = run(df, ["r0"], prefer_nearest_within=0.5, **kwargs)
+        assert out.iloc[0]["gain"] == pytest.approx(0.225)
+        assert not out.iloc[0]["crosses_level"]
+        assert out.iloc[0]["impact_bucket"] == "Low"
+        assert run(df, ["r0"], **kwargs).iloc[0]["impact_bucket"] == "High"
+
+    def test_single_candidate_record_is_unaffected(self, df):
+        # r5 (f=5) constrained upward has only p90 (8.1) available.
+        kwargs = dict(direction={"f": "increasing"})
+        default = run(df, ["r5"], **kwargs)
+        nearest = run(df, ["r5"], prefer_nearest_within=0.5, **kwargs)
+        assert len(default) == 1
+        assert default.iloc[0]["intervention_value"] == pytest.approx(8.1)
+        pd.testing.assert_frame_equal(default, nearest)
+
+    def test_minimize_goal_picks_the_nearer_candidate(self, df):
+        # Class 1 is the bad outcome here: the ramp runs downward, so every move lowers
+        # p and gain = -delta_p. Same gains as the maximize case, same expected pick.
+        model = RampModel(base=0.5, slope=-0.05)
+        kwargs = dict(
+            model=model, df=df, shap_df=make_shap_df(["r0"]),
+            candidate_quantiles=[0.25, 0.5, 0.9], score_goal="minimize",
+        )
+        default = per_record_counterfactuals(**kwargs)
+        nearest = per_record_counterfactuals(**kwargs, prefer_nearest_within=0.5)
+        assert default.iloc[0]["intervention_value"] == pytest.approx(8.1)
+        assert nearest.iloc[0]["intervention_value"] == pytest.approx(4.5)
+        assert nearest.iloc[0]["delta_p"] == pytest.approx(-0.225)
+        assert nearest.iloc[0]["gain"] == pytest.approx(0.225)
+
+    def test_harmful_moves_keep_their_best_candidate(self, df):
+        # Under a negative min_gain every candidate for r9 (f=9) makes the outcome worse,
+        # so best_gain is negative; scaling it would drop the lever entirely.
+        kwargs = dict(min_gain=-np.inf, direction={"f": "decreasing"})
+        default = run(df, ["r9"], **kwargs)
+        nearest = run(df, ["r9"], prefer_nearest_within=0.5, **kwargs)
+        assert len(nearest) == 1
+        assert nearest.iloc[0]["gain"] < 0
+        pd.testing.assert_frame_equal(default, nearest)
+
+
+class TestPreferNearestWithinValidation:
+    @pytest.mark.parametrize("value", [0, -0.1, 1.5, 2])
+    def test_out_of_range_raises(self, df, value):
+        with pytest.raises(ValueError, match="prefer_nearest_within"):
+            run(df, ["r0"], prefer_nearest_within=value)
+
+    def test_non_numeric_raises(self, df):
+        with pytest.raises(ValueError):
+            run(df, ["r0"], prefer_nearest_within="most")
+
+    def test_one_is_allowed(self, df):
+        # 1.0 means "only candidates tied with the best gain", i.e. the default pick.
+        out = run(df, ["r0"], prefer_nearest_within=1.0)
+        assert out.iloc[0]["intervention_value"] == pytest.approx(8.1)

gitlabds-2.4.3/tests/test_split_data.py

@@ -0,0 +1,59 @@
+import numpy as np
+import pandas as pd
+import pytest
+
+from gitlabds import split_data
+
+
+def _frame(n: int, groups: int = 0) -> pd.DataFrame:
+    rng = np.random.default_rng(0)
+    data = {
+        "x1": rng.random(n),
+        "x2": rng.random(n),
+        "y": (np.arange(n) % 2).astype(int),
+    }
+    if groups:
+        data["grp"] = np.repeat(np.arange(groups), n // groups)
+    return pd.DataFrame(data)
+
+
+@pytest.mark.parametrize("train_pct", [0.7, 0.85])
+@pytest.mark.parametrize("n", [100, 1000])
+def test_no_dv_split_on_exact_multiples(n, train_pct):
+    """Row counts where (1 - train_pct) * n is a whole number must not raise."""
+    df = _frame(n)
+    train_df, y_train, test_df, y_test, _ = split_data(
+        df, train_pct=train_pct, stratify=False
+    )
+    assert len(train_df) + len(test_df) == n
+    assert y_train.empty and y_test.empty
+
+
+@pytest.mark.parametrize("train_pct", [0.7, 0.85])
+@pytest.mark.parametrize("n", [100, 1000])
+def test_row_level_split_on_exact_multiples(n, train_pct):
+    df = _frame(n)
+    X_train, y_train, X_test, y_test, _ = split_data(
+        df, dv="y", train_pct=train_pct, stratify=False
+    )
+    assert len(X_train) + len(X_test) == n
+    assert len(y_train) == len(X_train)
+    assert len(y_test) == len(X_test)
+
+
+@pytest.mark.parametrize("train_pct", [0.7, 0.85])
+def test_group_split_on_exact_multiple_group_count(train_pct):
+    """The group count, not the row count, drives the group-aware split size."""
+    n, groups = 2000, 100
+    df = _frame(n, groups=groups)
+    X_train, y_train, X_test, y_test, _ = split_data(
+        df,
+        dv="y",
+        groupby="grp",
+        train_pct=train_pct,
+        stratify=False,
+    )
+    assert len(X_train) + len(X_test) == n
+    train_groups = set(X_train["grp"])
+    test_groups = set(X_test["grp"])
+    assert not (train_groups & test_groups)