gitlabds 2.1.11__tar.gz → 2.2.0__tar.gz

{gitlabds-2.1.11 → gitlabds-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitlabds
-Version: 2.1.11
+Version: 2.2.0
 Summary: GitLab Data Science Tools
 Author-email: Kevin Dietz <kdietz@gitlab.com>
 License-Expression: MIT
@@ -63,7 +63,7 @@ pip install gitlabds
 
 #### Snowflake Feature Store
 
-The `serve_features` and `clear_feature_serving_locks` functions require additional Snowflake dependencies.
+The `serve_features`, `clear_feature_serving_locks`, and `get_feature_descriptions` functions require additional Snowflake dependencies.
 
 To use these functions, install with:
 
@@ -815,6 +815,151 @@ results = gitlabds.prescriptions(
 ```
 </details>
 
+### Record Benchmarking
+
+<details><summary> Value vs. Peer Status </summary>
+
+#### Description
+Status label for a value vs. its peer median (+/- tolerance), not percentile rank.
+
+`gitlabds.value_vs_peer_status(value, peer_median, good_direction=None, tolerance=0.15, no_activity_value=None, peer_percentile=None, ceiling_pct=0.95, ceiling_label="near ceiling", labels=("below average", "average", "above average", "No activity")):`
+
+#### Parameters:
+- _**value**_ : value(s) to evaluate, scalar or array-like
+- _**peer_median**_ : peer group median(s) to compare against
+- _**good_direction**_ : `"decreasing"` flips which side of the median counts as "above" (e.g., for a metric where lower is better). Anything else is treated as increasing/default.
+- _**tolerance**_ : relative distance from the peer median (as a fraction of the median) required to be classified above/below average rather than "average". Default is 0.15 (a value must be >15% above/below its peer median to leave the "average" band)
+- _**no_activity_value**_ : value that should always map to `labels[3]` ("No activity") regardless of tolerance
+- _**peer_percentile**_ : optional 0-1 percentile rank within the peer group. When provided, an "above average" reading that also sits in the top `ceiling_pct` toward `good_direction` is upgraded to `ceiling_label`
+- _**ceiling_pct**_ : percentile threshold used with `peer_percentile` to detect a "tapped out" strength. Default is 0.95
+- _**ceiling_label**_ : label applied when the ceiling condition is met. Default is "near ceiling"
+- _**labels**_ : tuple of (below average, average, above average, no activity) labels
+
+#### Returns
+- Series of status labels, one per input value
+
+#### Examples:
+```python
+import gitlabds
+
+status = gitlabds.value_vs_peer_status(
+    value=df["monthly_spend"],
+    peer_median=df["peer_median_spend"],
+    peer_percentile=df["peer_percentile_spend"],
+)
+```
+
+</details>
+
+<details><summary> Per-Record SHAP </summary>
+
+#### Description
+Per-record SHAP attribution (log-odds / margin space) for a tree model. Uses tree_path_dependent TreeSHAP, which natively handles the model's categorical splits. Returns the top_n features by |contribution| per record, guaranteeing at least min_actionable actionable ones.
+
+`gitlabds.per_record_shap(model, df, feature_dict, unique_id="record_id", top_n=25, min_actionable=3, actionable_values=["Yes", "Partial"]):`
+
+#### Parameters:
+- _**model**_ : trained tree-based model (e.g., LightGBM, XGBoost)
+- _**df**_ : model-ready feature matrix, indexed by `unique_id`
+- _**feature_dict**_ : DataFrame or path to CSV with `field` and `actionable` columns
+- _**unique_id**_ : name to use for the record identifier column in the output. Default is "record_id"
+- _**top_n**_ : number of features to return per record, ranked by |contribution|. Default is 25
+- _**min_actionable**_ : minimum number of actionable features guaranteed per record, even if outside the top_n. Default is 3
+- _**actionable_values**_ : values in `feature_dict["actionable"]` treated as actionable
+
+#### Returns
+- Long DataFrame, one row per (record, feature): `unique_id`, `rank`, `feature`, `contribution`, `direction`, `feature_value`, `actionable`
+
+#### Requirements
+This function requires the `shap` package (included in core dependencies).
+
+#### Examples:
+```python
+import gitlabds
+
+shap_df = gitlabds.per_record_shap(
+    model=trained_model,
+    df=model_features,
+    feature_dict="feature_dict.csv",
+)
+```
+
+</details>
+
+<details><summary> Per-Record Counterfactuals </summary>
+
+#### Description
+"Change X -> +Y" counterfactual search, direction chosen empirically. For each actionable lever SHAP surfaced for a record, evaluates candidate values drawn from peer-group quantiles, re-scores the model at each, and keeps the single move that maximizes the gain in predicted probability. No monotonicity assumption is made.
+
+`gitlabds.per_record_counterfactuals(model, df, shap_df, model_algo="lgb", unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, actionable_values=["Yes", "Partial"], min_delta_p=0.005, impact_medium=0.01, impact_high=0.03, bands={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5):`
+
+#### Parameters:
+- _**model**_ : trained model (as in `per_record_shap`)
+- _**df**_ : model-ready feature matrix, indexed by `unique_id`
+- _**shap_df**_ : output of `per_record_shap`; scopes which (record, feature) levers to test
+- _**model_algo**_ : `"lgb"` or `"xgb"`/`"xgboost"`; controls how probabilities are scored
+- _**unique_id**_ : name of the record identifier column. Default is "record_id"
+- _**peer_group**_ : Series/DataFrame indexed by `unique_id`; numeric columns are auto-binned into `n_bins` quantile bands. `None` uses one global group
+- _**candidate_quantiles**_ : peer-group quantiles evaluated as candidate values
+- _**direction**_ : optional per-feature achievability constraint mapping feature -> `'increasing'` | `'decreasing'` | `'both'` (dict or Series)
+- _**actionable_values**_ : values in `shap_df["actionable"]` treated as actionable levers
+- _**min_delta_p**_ : drop levers whose best achievable gain is below this
+- _**impact_medium, impact_high**_ : delta_p thresholds bucketing each lever into Low/Medium/High `impact_bucket`
+- _**bands**_ : dict mapping label -> upper bound on `p_before`/`p_after` for that band, ordered worst to best (e.g., default `{"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}` means `p <= 0.03` -> "poor fit", `0.03 < p <= 0.10` -> "good fit", `p > 0.10` -> "best fit"). Key order doesn't matter (sorted by cutoff), but the top band's cutoff must be `float("inf")`. Used to detect whether a move crosses a band boundary (any crossing move is floored at "Medium" impact). Pass your own dict (e.g., `{"low": 0.03, "medium": 0.10, "high": float("inf")}`) to match your own pipeline
+- _**n_bins**_ : quantile bins for numeric `peer_group` columns
+
+#### Returns
+- Long DataFrame, one row per (record, lever): `current_value`, `intervention_value`, `intervention_quantile`, `p_before`, `p_after`, `delta_p`, `impact_bucket`, `band_before`, `band_after`, `crosses_band`
+
+#### Requirements
+For XGBoost models without `predict_proba`, requires the `xgboost` package (included in core dependencies).
+
+#### Examples:
+```python
+import gitlabds
+
+counterfactuals_df = gitlabds.per_record_counterfactuals(
+    model=trained_model,
+    df=model_features,
+    shap_df=shap_df,
+    peer_group=peer_group_df,
+)
+```
+
+</details>
+
+<details><summary> Per-Record Peer Percentiles </summary>
+
+#### Description
+Peer benchmarking. Each record's percentile for each feature within its peer group, plus the group's p25/p50/p75 for context. Pure pandas -- no model required.
+
+`gitlabds.per_record_peer_percentiles(df, feature_dict, peer_group=None, unique_id="record_id", actionable_values=["Yes", "Partial"], features=None, n_bins=5):`
+
+#### Parameters:
+- _**df**_ : model features DataFrame, indexed by `unique_id`
+- _**feature_dict**_ : DataFrame or path to CSV with `field` and `actionable` columns
+- _**peer_group**_ : Series/DataFrame indexed by `unique_id`; numeric columns are auto-binned into `n_bins` quantile bands. `None` uses one global group
+- _**unique_id**_ : name of the record identifier column. Default is "record_id"
+- _**actionable_values**_ : values in `feature_dict["actionable"]` treated as actionable
+- _**features**_ : optional explicit list of features to benchmark (used as-is, no actionable filter). If `None` (default), computed for every actionable feature in `df`
+- _**n_bins**_ : quantile bins for numeric `peer_group` columns
+
+#### Returns
+- Long DataFrame, one row per (record, feature): `unique_id`, `feature`, `record_value`, `peer_group`, `percentile`, `peer_p25`, `peer_p50`, `peer_p75`
+
+#### Examples:
+```python
+import gitlabds
+
+percentiles_df = gitlabds.per_record_peer_percentiles(
+    df=model_features,
+    feature_dict="feature_dict.csv",
+    peer_group=peer_group_df,
+)
+```
+
+</details>
+
 ### Model Monitoring
 
 <details><summary> Generate Baseline Features </summary>
@@ -1072,6 +1217,45 @@ print(f"Cleared {locks_cleared} locks")
 
 </details>
 
+<details><summary> Get Feature Descriptions </summary>
+
+#### Description
+Look up each feature's source feature view and description from the feature store's registered feature views (column comments set via `FeatureView.attach_feature_desc(...)`). Useful for documenting a model's fields or auditing everything registered in the store.
+
+`gitlabds.get_feature_descriptions(session, feature_store, features=None):`
+
+#### Parameters:
+- **_session_** : Active Snowpark session (e.g., `fs_engine._session`)
+- **_feature_store_** : Initialized FeatureStore instance (e.g., `fs_engine`); used to enumerate the registered feature views and locate their db/schema
+- **_features_** : Feature names to look up. If None, returns every feature in the store's registered feature views.
+
+#### Returns
+- DataFrame with columns: `feature`, `feature_view`, `data_type`, `description`. Features found in more than one feature view have their views/descriptions joined (`" | "` / `" || "`) so nothing is dropped. When `features` is provided, unmatched features (e.g., transformed or dummy-coded features) come back with blank `feature_view` / `data_type` / `description` for the user to populate manually, and the caller's order/list is preserved.
+
+#### Requirements
+This function requires Snowflake packages. Install with:
+```bash
+pip install snowflake-snowpark-python snowflake-ml-python
+```
+
+#### Examples:
+
+```python
+import gitlabds
+
+# Look up a specific list (e.g. the model fields from a config)
+desc = gitlabds.get_feature_descriptions(
+    session=fs_engine._session,
+    feature_store=fs_engine,
+    features=scoring_config["model"]["fields"],
+)
+
+# Or pull every feature registered in the store
+all_desc = gitlabds.get_feature_descriptions(session=fs_engine._session, feature_store=fs_engine)
+```
+
+</details>
+
 ### SQL and Trend Analysis
 
 <details><summary> SQL Trend Query Generator </summary>

{gitlabds-2.1.11 → gitlabds-2.2.0}/README.md

@@ -31,7 +31,7 @@ pip install gitlabds
 
 #### Snowflake Feature Store
 
-The `serve_features` and `clear_feature_serving_locks` functions require additional Snowflake dependencies.
+The `serve_features`, `clear_feature_serving_locks`, and `get_feature_descriptions` functions require additional Snowflake dependencies.
 
 To use these functions, install with:
 
@@ -783,6 +783,151 @@ results = gitlabds.prescriptions(
 ```
 </details>
 
+### Record Benchmarking
+
+<details><summary> Value vs. Peer Status </summary>
+
+#### Description
+Status label for a value vs. its peer median (+/- tolerance), not percentile rank.
+
+`gitlabds.value_vs_peer_status(value, peer_median, good_direction=None, tolerance=0.15, no_activity_value=None, peer_percentile=None, ceiling_pct=0.95, ceiling_label="near ceiling", labels=("below average", "average", "above average", "No activity")):`
+
+#### Parameters:
+- _**value**_ : value(s) to evaluate, scalar or array-like
+- _**peer_median**_ : peer group median(s) to compare against
+- _**good_direction**_ : `"decreasing"` flips which side of the median counts as "above" (e.g., for a metric where lower is better). Anything else is treated as increasing/default.
+- _**tolerance**_ : relative distance from the peer median (as a fraction of the median) required to be classified above/below average rather than "average". Default is 0.15 (a value must be >15% above/below its peer median to leave the "average" band)
+- _**no_activity_value**_ : value that should always map to `labels[3]` ("No activity") regardless of tolerance
+- _**peer_percentile**_ : optional 0-1 percentile rank within the peer group. When provided, an "above average" reading that also sits in the top `ceiling_pct` toward `good_direction` is upgraded to `ceiling_label`
+- _**ceiling_pct**_ : percentile threshold used with `peer_percentile` to detect a "tapped out" strength. Default is 0.95
+- _**ceiling_label**_ : label applied when the ceiling condition is met. Default is "near ceiling"
+- _**labels**_ : tuple of (below average, average, above average, no activity) labels
+
+#### Returns
+- Series of status labels, one per input value
+
+#### Examples:
+```python
+import gitlabds
+
+status = gitlabds.value_vs_peer_status(
+    value=df["monthly_spend"],
+    peer_median=df["peer_median_spend"],
+    peer_percentile=df["peer_percentile_spend"],
+)
+```
+
+</details>
+
+<details><summary> Per-Record SHAP </summary>
+
+#### Description
+Per-record SHAP attribution (log-odds / margin space) for a tree model. Uses tree_path_dependent TreeSHAP, which natively handles the model's categorical splits. Returns the top_n features by |contribution| per record, guaranteeing at least min_actionable actionable ones.
+
+`gitlabds.per_record_shap(model, df, feature_dict, unique_id="record_id", top_n=25, min_actionable=3, actionable_values=["Yes", "Partial"]):`
+
+#### Parameters:
+- _**model**_ : trained tree-based model (e.g., LightGBM, XGBoost)
+- _**df**_ : model-ready feature matrix, indexed by `unique_id`
+- _**feature_dict**_ : DataFrame or path to CSV with `field` and `actionable` columns
+- _**unique_id**_ : name to use for the record identifier column in the output. Default is "record_id"
+- _**top_n**_ : number of features to return per record, ranked by |contribution|. Default is 25
+- _**min_actionable**_ : minimum number of actionable features guaranteed per record, even if outside the top_n. Default is 3
+- _**actionable_values**_ : values in `feature_dict["actionable"]` treated as actionable
+
+#### Returns
+- Long DataFrame, one row per (record, feature): `unique_id`, `rank`, `feature`, `contribution`, `direction`, `feature_value`, `actionable`
+
+#### Requirements
+This function requires the `shap` package (included in core dependencies).
+
+#### Examples:
+```python
+import gitlabds
+
+shap_df = gitlabds.per_record_shap(
+    model=trained_model,
+    df=model_features,
+    feature_dict="feature_dict.csv",
+)
+```
+
+</details>
+
+<details><summary> Per-Record Counterfactuals </summary>
+
+#### Description
+"Change X -> +Y" counterfactual search, direction chosen empirically. For each actionable lever SHAP surfaced for a record, evaluates candidate values drawn from peer-group quantiles, re-scores the model at each, and keeps the single move that maximizes the gain in predicted probability. No monotonicity assumption is made.
+
+`gitlabds.per_record_counterfactuals(model, df, shap_df, model_algo="lgb", unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, actionable_values=["Yes", "Partial"], min_delta_p=0.005, impact_medium=0.01, impact_high=0.03, bands={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5):`
+
+#### Parameters:
+- _**model**_ : trained model (as in `per_record_shap`)
+- _**df**_ : model-ready feature matrix, indexed by `unique_id`
+- _**shap_df**_ : output of `per_record_shap`; scopes which (record, feature) levers to test
+- _**model_algo**_ : `"lgb"` or `"xgb"`/`"xgboost"`; controls how probabilities are scored
+- _**unique_id**_ : name of the record identifier column. Default is "record_id"
+- _**peer_group**_ : Series/DataFrame indexed by `unique_id`; numeric columns are auto-binned into `n_bins` quantile bands. `None` uses one global group
+- _**candidate_quantiles**_ : peer-group quantiles evaluated as candidate values
+- _**direction**_ : optional per-feature achievability constraint mapping feature -> `'increasing'` | `'decreasing'` | `'both'` (dict or Series)
+- _**actionable_values**_ : values in `shap_df["actionable"]` treated as actionable levers
+- _**min_delta_p**_ : drop levers whose best achievable gain is below this
+- _**impact_medium, impact_high**_ : delta_p thresholds bucketing each lever into Low/Medium/High `impact_bucket`
+- _**bands**_ : dict mapping label -> upper bound on `p_before`/`p_after` for that band, ordered worst to best (e.g., default `{"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}` means `p <= 0.03` -> "poor fit", `0.03 < p <= 0.10` -> "good fit", `p > 0.10` -> "best fit"). Key order doesn't matter (sorted by cutoff), but the top band's cutoff must be `float("inf")`. Used to detect whether a move crosses a band boundary (any crossing move is floored at "Medium" impact). Pass your own dict (e.g., `{"low": 0.03, "medium": 0.10, "high": float("inf")}`) to match your own pipeline
+- _**n_bins**_ : quantile bins for numeric `peer_group` columns
+
+#### Returns
+- Long DataFrame, one row per (record, lever): `current_value`, `intervention_value`, `intervention_quantile`, `p_before`, `p_after`, `delta_p`, `impact_bucket`, `band_before`, `band_after`, `crosses_band`
+
+#### Requirements
+For XGBoost models without `predict_proba`, requires the `xgboost` package (included in core dependencies).
+
+#### Examples:
+```python
+import gitlabds
+
+counterfactuals_df = gitlabds.per_record_counterfactuals(
+    model=trained_model,
+    df=model_features,
+    shap_df=shap_df,
+    peer_group=peer_group_df,
+)
+```
+
+</details>
+
+<details><summary> Per-Record Peer Percentiles </summary>
+
+#### Description
+Peer benchmarking. Each record's percentile for each feature within its peer group, plus the group's p25/p50/p75 for context. Pure pandas -- no model required.
+
+`gitlabds.per_record_peer_percentiles(df, feature_dict, peer_group=None, unique_id="record_id", actionable_values=["Yes", "Partial"], features=None, n_bins=5):`
+
+#### Parameters:
+- _**df**_ : model features DataFrame, indexed by `unique_id`
+- _**feature_dict**_ : DataFrame or path to CSV with `field` and `actionable` columns
+- _**peer_group**_ : Series/DataFrame indexed by `unique_id`; numeric columns are auto-binned into `n_bins` quantile bands. `None` uses one global group
+- _**unique_id**_ : name of the record identifier column. Default is "record_id"
+- _**actionable_values**_ : values in `feature_dict["actionable"]` treated as actionable
+- _**features**_ : optional explicit list of features to benchmark (used as-is, no actionable filter). If `None` (default), computed for every actionable feature in `df`
+- _**n_bins**_ : quantile bins for numeric `peer_group` columns
+
+#### Returns
+- Long DataFrame, one row per (record, feature): `unique_id`, `feature`, `record_value`, `peer_group`, `percentile`, `peer_p25`, `peer_p50`, `peer_p75`
+
+#### Examples:
+```python
+import gitlabds
+
+percentiles_df = gitlabds.per_record_peer_percentiles(
+    df=model_features,
+    feature_dict="feature_dict.csv",
+    peer_group=peer_group_df,
+)
+```
+
+</details>
+
 ### Model Monitoring
 
 <details><summary> Generate Baseline Features </summary>
@@ -1040,6 +1185,45 @@ print(f"Cleared {locks_cleared} locks")
 
 </details>
 
+<details><summary> Get Feature Descriptions </summary>
+
+#### Description
+Look up each feature's source feature view and description from the feature store's registered feature views (column comments set via `FeatureView.attach_feature_desc(...)`). Useful for documenting a model's fields or auditing everything registered in the store.
+
+`gitlabds.get_feature_descriptions(session, feature_store, features=None):`
+
+#### Parameters:
+- **_session_** : Active Snowpark session (e.g., `fs_engine._session`)
+- **_feature_store_** : Initialized FeatureStore instance (e.g., `fs_engine`); used to enumerate the registered feature views and locate their db/schema
+- **_features_** : Feature names to look up. If None, returns every feature in the store's registered feature views.
+
+#### Returns
+- DataFrame with columns: `feature`, `feature_view`, `data_type`, `description`. Features found in more than one feature view have their views/descriptions joined (`" | "` / `" || "`) so nothing is dropped. When `features` is provided, unmatched features (e.g., transformed or dummy-coded features) come back with blank `feature_view` / `data_type` / `description` for the user to populate manually, and the caller's order/list is preserved.
+
+#### Requirements
+This function requires Snowflake packages. Install with:
+```bash
+pip install snowflake-snowpark-python snowflake-ml-python
+```
+
+#### Examples:
+
+```python
+import gitlabds
+
+# Look up a specific list (e.g. the model fields from a config)
+desc = gitlabds.get_feature_descriptions(
+    session=fs_engine._session,
+    feature_store=fs_engine,
+    features=scoring_config["model"]["fields"],
+)
+
+# Or pull every feature registered in the store
+all_desc = gitlabds.get_feature_descriptions(session=fs_engine._session, feature_store=fs_engine)
+```
+
+</details>
+
 ### SQL and Trend Analysis
 
 <details><summary> SQL Trend Query Generator </summary>

{gitlabds-2.1.11 → gitlabds-2.2.0}/gitlabds/__init__.py

@@ -3,6 +3,7 @@ from .config_generator import ConfigGenerator
 from .dummy import dummy_code, dummy_top, apply_dummy
 from .feature_reduction import drop_categorical, remove_low_variation, remove_outcome_proxies, correlation_reduction
 from .insights import marginal_effects, prescriptions
+from .record_benchmarking import value_vs_peer_status, per_record_shap, per_record_counterfactuals, per_record_peer_percentiles
 from .memory_optimization import reduce_memory_usage, sparse_encode_df, identify_categorical_candidates, compact_categorical, memory_profile_df, memory_optimization
 from .missing import missing_values, apply_missing_values
 from .model_evaluator import ModelEvaluator
@@ -14,6 +15,7 @@ from .trends import generate_sql_trend_query, trend_analysis
 # Optional imports for Snowflake feature serving
 try:
     from .serving_features import clear_feature_serving_locks, serve_features
+    from .feature_descriptions import get_feature_descriptions
 except ImportError as e:
     # Snowflake packages not installed - create placeholder functions with helpful error messages
     def _snowflake_not_installed(*args, **kwargs):
@@ -21,7 +23,8 @@ except ImportError as e:
             "Snowflake feature serving functions require additional dependencies. "
             "Install them with: pip install snowflake-snowpark-python snowflake-ml-python\n\n"
         )
-    
+
     clear_feature_serving_locks = _snowflake_not_installed
     serve_features = _snowflake_not_installed
+    get_feature_descriptions = _snowflake_not_installed
 

gitlabds-2.2.0/gitlabds/feature_descriptions.py

@@ -0,0 +1,106 @@
+"""
+Feature description lookup for Snowflake Feature Store integration.
+
+Given a list of feature names, return each feature's source feature view and its
+description (pulled from the registered feature views' column comments, which is
+where FeatureView.attach_feature_desc(...) stores them). Follows the same call
+pattern as gitlabds.serve_features: pass the feature-store session and object.
+
+Note: This module requires snowflake-snowpark-python and snowflake-ml-python packages.
+"""
+
+from typing import Iterable, Optional
+import pandas as pd
+
+# Optional imports with graceful fallback
+try:
+    from snowflake.snowpark import Session
+    from snowflake.ml.feature_store import FeatureStore
+    SNOWFLAKE_AVAILABLE = True
+except ImportError:
+    SNOWFLAKE_AVAILABLE = False
+    Session = None
+    FeatureStore = None
+
+
+def _check_snowflake_available():
+    """Check if Snowflake packages are available."""
+    if not SNOWFLAKE_AVAILABLE:
+        raise ImportError(
+            "Snowflake packages are required for feature descriptions. "
+            "Install with: pip install snowflake-snowpark-python snowflake-ml-python"
+        )
+
+
+def get_feature_descriptions(
+    session: 'Session',
+    feature_store: 'FeatureStore',
+    features: Optional[Iterable[str]] = None,
+) -> pd.DataFrame:
+    """
+    Look up each feature's source feature view and description from the feature
+    store's registered feature views (column comments set via
+    FeatureView.attach_feature_desc(...)).
+
+    Parameters
+    ----------
+    session : snowflake.snowpark.Session
+        Active Snowpark session (e.g., fs_engine._session).
+    feature_store : snowflake.ml.feature_store.FeatureStore
+        Initialized FeatureStore instance (e.g., fs_engine); used to enumerate the
+        registered feature views and to locate their db/schema.
+    features : Iterable[str], optional
+        Feature names to look up. If None, returns every feature in the store's
+        registered feature views.
+
+    Returns
+    -------
+    pd.DataFrame
+        Columns: [feature, feature_view, data_type, description]. Features found
+        in more than one feature view have their views/descriptions joined
+        (" | " / " || ") so nothing is dropped. When `features` is provided,
+        unmatched features (e.g., transformed or dummy-coded features) come back
+        with blank feature_view / data_type / description for the user to
+        populate manually, and order/list is preserved.
+    """
+    _check_snowflake_available()
+
+    # Registered feature views -> exact published table names (e.g. ACCOUNT_CI_USAGE$1.0).
+    fvs = feature_store.list_feature_views().select("NAME", "VERSION").to_pandas()
+    tables = [f"{n}${v}" for n, v in zip(fvs["NAME"], fvs["VERSION"])]
+
+    # Database/schema come from the live feature-store session, not hard-coded.
+    # Pulled directly rather than parsed out of get_fully_qualified_current_schema(),
+    # which would require quote-aware splitting to handle identifiers containing ".".
+    database = session.get_current_database().strip('"')
+    schema = session.get_current_schema().strip('"')
+
+    def sql_list(values):
+        return ",".join("'" + str(v).replace("'", "''") + "'" for v in values)
+
+    meta = session.sql(
+        "SELECT TABLE_NAME, COLUMN_NAME, DATA_TYPE, COMMENT "
+        f"FROM {database}.INFORMATION_SCHEMA.COLUMNS "
+        f"WHERE TABLE_SCHEMA = '{schema}' AND TABLE_NAME IN ({sql_list(tables)})"
+    ).to_pandas()
+    meta.columns = [c.upper() for c in meta.columns]
+
+    meta["feature_view"] = meta["TABLE_NAME"].str.split("$").str[0]
+    meta["feature"] = meta["COLUMN_NAME"].str.lower()
+
+    catalog = meta.groupby("feature", as_index=False).agg(
+        feature_view=("feature_view", lambda s: " | ".join(sorted(set(s)))),
+        data_type=("DATA_TYPE", lambda s: " | ".join(sorted(set(s.dropna().astype(str))))),
+        description=("COMMENT", lambda s: " || ".join(dict.fromkeys(x for x in s if x))),
+    )
+
+    if features is None:
+        return catalog.sort_values("feature").reset_index(drop=True)
+
+    # Preserve the caller's feature list/order; blanks where unmatched.
+    requested = pd.DataFrame({"feature": list(features)})
+    requested["_key"] = requested["feature"].str.lower()
+    out = requested.merge(
+        catalog.rename(columns={"feature": "_key"}), on="_key", how="left"
+    ).drop(columns="_key")
+    return out[["feature", "feature_view", "data_type", "description"]].fillna("")

gitlabds-2.2.0/gitlabds/record_benchmarking.py

@@ -0,0 +1,360 @@
+"""
+Per-record model insight utilities.
+
+These functions benchmark, explain, and prescribe changes for any indexed set of
+scored records (accounts, users, deals, transactions, etc.) -- `unique_id` is just
+whichever column identifies a row in your data. Includes:
+
+- value_vs_peer_status : status label for a value vs. its peer median (+/- band)
+- per_record_shap : per-record SHAP attribution (log-odds/margin space) for a tree model
+- per_record_counterfactuals : per-record "change X -> +Y" counterfactual search
+- per_record_peer_percentiles : per-record percentile standing within a peer group
+"""
+
+import pandas as pd
+import numpy as np
+
+
+def value_vs_peer_status(value, peer_median, good_direction=None, tolerance=0.15,
+                          no_activity_value=None, peer_percentile=None, ceiling_pct=0.95,
+                          ceiling_label="near ceiling",
+                          labels=("below average", "average", "above average", "No activity")):
+    """
+    Status from value vs. its peer median (+/- tolerance), not percentile rank.
+
+    value : value(s) to evaluate, scalar or array-like.
+    peer_median : peer group median(s) to compare against; broadcasts against value.
+    good_direction : "decreasing" flips which side of the median counts as "above"
+        (e.g., for a metric where lower is better). Anything else (including None,
+        the default) is treated as increasing.
+    tolerance : relative distance from the peer median (as a fraction of the median)
+        required to be classified above/below average rather than "average". E.g.
+        the default 0.15 means a value must be >15% above (or below) its peer median
+        to leave the "average" band.
+    no_activity_value : value that should always map to labels[3] ("No activity")
+        regardless of tolerance, in addition to actual missing values.
+    peer_percentile : optional 0-1 percentile rank within the peer group. When given,
+        an "above average" reading that also sits in the top ceiling_pct toward
+        good_direction is upgraded to ceiling_label -- a tapped-out strength, not a
+        lever, rather than just "above average".
+    ceiling_pct : percentile threshold (toward good_direction) used with
+        peer_percentile to detect the ceiling condition above.
+    ceiling_label : label applied when the ceiling condition is met.
+    labels : (below average, average, above average, no activity) labels to use
+        in place of the defaults.
+
+    Returns a Series of status labels, one per input value.
+    """
+    value = pd.Series(value)
+    # index=value.index (not .reindex()) so a bare scalar broadcasts correctly.
+    peer_median = pd.Series(peer_median, index=value.index)
+    decreasing = pd.Series(good_direction, index=value.index).astype(str).str.lower().eq("decreasing") \
+        if good_direction is not None else pd.Series(False, index=value.index)
+    eff_value = value.where(~decreasing, -value)
+    eff_median = peer_median.where(~decreasing, -peer_median)
+    denom = eff_median.abs()
+    with np.errstate(divide="ignore", invalid="ignore"):
+        rel_diff = (eff_value - eff_median) / denom
+    rel_diff = rel_diff.where(denom > 0, np.sign(eff_value - eff_median))
+    status = pd.Series(np.select([rel_diff > tolerance, rel_diff < -tolerance],
+                                  [labels[2], labels[0]], default=labels[1]),
+                        index=value.index)
+    # "near ceiling": already above the peer median AND in the top ceiling_pct of the peer
+    # group toward good_direction. Gating on labels[2] (above average) means a compressed
+    # peer group whose top decile is still ~median is left "average", not falsely capped.
+    # Applied before the null / no-activity masks below so those still take precedence.
+    if peer_percentile is not None:
+        pct = pd.Series(peer_percentile, index=value.index).astype(float)
+        eff_pct = pct.where(~decreasing, 1 - pct)   # percentile toward the good direction
+        status = status.mask((eff_pct >= ceiling_pct) & (status == labels[2]), ceiling_label)
+    status = status.mask(value.isna(), labels[3])
+    if no_activity_value is not None:
+        no_activity_value = pd.Series(no_activity_value, index=value.index)
+        status = status.where(value != no_activity_value, labels[3])
+    return status
+
+
+def per_record_shap(
+    model,
+    df,
+    feature_dict,
+    unique_id="record_id",
+    top_n=25,
+    min_actionable=3,
+    actionable_values=["Yes", "Partial"],
+):
+    """
+    Per-record SHAP attribution (log-odds / margin space) for a tree model.
+    Uses tree_path_dependent TreeSHAP, which natively handles the model's categorical
+    splits (probability-space SHAP requires a numeric interventional background and is not
+    available for LightGBM models with native categorical features). Contributions are in
+    log-odds: `+` pushes the record's score above the base rate, `-` below; magnitude ranks
+    the drivers.
+
+    model : trained tree-based model (e.g., LightGBM, XGBoost). Its underlying booster
+        is used directly if the model exposes one (e.g. `model.booster_`).
+    df : model-ready feature matrix, indexed by unique_id.
+    feature_dict : DataFrame, or path to a CSV, with 'field' and 'actionable' columns
+        mapping each feature to whether/how it can be acted on.
+    unique_id : name to use for the record identifier column in the output.
+    top_n : number of features to return per record, ranked by |contribution|.
+    min_actionable : minimum number of actionable features guaranteed per record,
+        pulled in from outside the top_n if needed.
+    actionable_values : values in feature_dict["actionable"] treated as actionable.
+
+    Returns a long DataFrame, one row per (record, feature): unique_id, rank, feature,
+    contribution, direction, feature_value, actionable.
+    """
+    import shap, warnings
+    if isinstance(feature_dict, str):
+        feature_dict = pd.read_csv(feature_dict)
+    feats = list(df.columns)
+    ids = df.index
+    booster = getattr(model, "booster_", model)
+    with warnings.catch_warnings():           # silence shap's informational list-output notice
+        warnings.simplefilter("ignore")
+        sv = shap.TreeExplainer(booster, feature_perturbation="tree_path_dependent").shap_values(df)
+    if isinstance(sv, list):          # binary classifiers return [class0, class1]
+        sv = sv[-1]
+    sv = np.asarray(sv)
+    if sv.ndim == 3:
+        sv = sv[:, :, -1]
+    contrib = pd.DataFrame(sv, columns=feats)
+    c = contrib.assign(**{unique_id: ids}).melt(unique_id, var_name="feature", value_name="contribution")
+    v = df.reset_index(drop=True).assign(**{unique_id: ids}).melt(unique_id, var_name="feature", value_name="feature_value")
+    d = c.merge(v, on=[unique_id, "feature"])
+    d["abs_contribution"] = d["contribution"].abs()
+    act = dict(zip(feature_dict["field"], feature_dict["actionable"]))
+    d["actionable"] = d["feature"].map(act)
+    d["is_actionable"] = d["actionable"].isin(actionable_values)
+    # Top_n per record by |contribution|, guaranteeing >= min_actionable actionable.
+    d["pool_rank"] = d.groupby(unique_id, observed=True)["abs_contribution"].rank(method="first", ascending=False)
+    in_top = d["pool_rank"] <= top_n
+    act_in_top = (d[in_top & d["is_actionable"]].groupby(unique_id, observed=True).size()
+                  .reindex(pd.unique(ids), fill_value=0))
+    need = (min_actionable - act_in_top).clip(lower=0)
+    outside = d[(~in_top) & d["is_actionable"]].copy()
+    outside["rec_rank"] = outside.groupby(unique_id, observed=True)["abs_contribution"].rank(method="first", ascending=False)
+    outside = outside.merge(need.rename("need"), left_on=unique_id, right_index=True)
+    keep = d.loc[d.index[in_top].union(outside.index[outside["rec_rank"] <= outside["need"]])].copy()
+    keep["direction"] = np.where(keep["contribution"] >= 0, "increases", "decreases")
+    keep["rank"] = keep.groupby(unique_id, observed=True)["abs_contribution"].rank(method="first", ascending=False).astype(int)
+    # abs_contribution stays internal (used above for pool_rank/rec_rank/rank) — not in the
+    # final output, since nothing downstream reads it: rank already gives the ordering, and
+    # contribution itself already carries the magnitude.
+    cols = [unique_id, "rank", "feature", "contribution",
+            "direction", "feature_value", "actionable"]
+    return keep.sort_values([unique_id, "rank"]).reset_index(drop=True)[cols]
+
+
+def per_record_counterfactuals(
+    model,
+    df,
+    shap_df,
+    model_algo="lgb",
+    unique_id="record_id",
+    peer_group=None,
+    candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9],
+    direction=None,
+    actionable_values=["Yes", "Partial"],
+    min_delta_p=0.005,
+    impact_medium=0.01,
+    impact_high=0.03,
+    bands={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")},
+    n_bins=5,
+):
+    """
+    "change X -> +Y", direction chosen empirically.
+    For each actionable lever SHAP surfaced for a record, evaluate a set of candidate
+    values (peer-group quantiles), re-score the model at each, and keep the single move
+    that maximizes the gain in predicted probability. The model decides whether that
+    means increasing or decreasing the feature, so non-monotonic and negatively related
+    features are handled correctly. No monotonicity assumption is made.
+
+    model, df, model_algo, unique_id : as in per_record_shap (df indexed by unique_id).
+    shap_df : per_record_shap output; scopes which (record, feature) levers to test.
+    peer_group : Series/DataFrame indexed by unique_id; numeric columns auto-binned into
+        n_bins quantile bands. None -> one global group. Candidate values are drawn
+        from each feature's distribution within the record's peer group.
+    candidate_quantiles : peer-group quantiles evaluated as candidate values.
+    direction : optional per-feature achievability constraint mapping feature ->
+        'increasing' | 'decreasing' | 'both' (dict or Series). Missing/None -> 'both' (unconstrained).
+    min_delta_p : drop levers whose best achievable gain is below this.
+    impact_medium, impact_high : delta_p thresholds bucketing each lever into
+        Low/Medium/High impact_bucket. delta_p shrinks mechanically near the tails of
+        the probability distribution, so a fixed pp threshold alone under-weights
+        moves for low-scoring/mid-band records relative to those near a band edge.
+        bands (below) corrects for that: any move that crosses a band boundary is
+        floored at "Medium" regardless of its raw delta_p, since crossing bands is
+        typically the thing that actually changes which downstream action gets taken.
+        Tune impact_medium/high against the real delta_p distribution for your model;
+        bands should mirror whatever thresholds are live in your own scoring/banding
+        logic (if that logic considers factors beyond the score itself, bands are
+        only meant to detect crossings for impact_bucket purposes, not to reproduce
+        it exactly).
+    bands : dict mapping label -> upper bound (on p_before/p_after) for that band,
+        ordered worst to best (e.g. {"poor fit": 0.03, "good fit": 0.10, "best fit":
+        float("inf")} means p <= 0.03 -> "poor fit", 0.03 < p <= 0.10 -> "good fit",
+        p > 0.10 -> "best fit"). The dict's key order doesn't matter -- bands are
+        sorted by their cutoff value -- but the highest-scoring band's cutoff must be
+        float("inf") since it has no upper bound. Used only to detect band crossings
+        for impact_bucket, not to reproduce your production banding logic. Pass your
+        own dict (e.g. {"low": 0.03, "medium": 0.10, "high": float("inf")}) to match
+        your own pipeline.
+    n_bins : quantile bins for numeric peer_group columns.
+
+    Returns a long DataFrame: one row per (record, lever) with the best move found
+    (current_value, intervention_value, intervention_quantile, p_before/p_after/delta_p,
+    impact_bucket, band_before, band_after, crosses_band).
+    """
+    if impact_medium >= impact_high:
+        raise ValueError(f"impact_medium ({impact_medium}) must be less than impact_high ({impact_high})")
+    ids = df.index
+    # Peer-group key: numeric peer columns auto-binned into n_bins quantile bands.
+    if peer_group is None:
+        grp = pd.Series("all", index=ids)
+    else:
+        pk = peer_group.reindex(ids)
+        pk = pk.to_frame() if isinstance(pk, pd.Series) else pk
+        key = None
+        for c in pk.columns:
+            band = (pd.qcut(pk[c], q=n_bins, duplicates="drop").astype(str)
+                    if pd.api.types.is_numeric_dtype(pk[c]) else pk[c].astype(str))
+            key = band if key is None else key.str.cat(band, sep=" | ")
+        grp = pd.Series(np.asarray(key), index=ids)
+    # Accept up/down/both or increasing/decreasing/both (from the dictionary
+    # good_direction column); anything else -> "both".
+    _dirmap = {"increasing": "up", "increase": "up", "up": "up",
+               "decreasing": "down", "decrease": "down", "down": "down",
+               "both": "both"}
+    direction = {k: _dirmap.get(str(v).lower(), "both")
+                 for k, v in (dict(direction).items() if direction is not None else [])}
+    def proba(X):
+        if model_algo.lower() in ("xgb", "xgboost") and not hasattr(model, "predict_proba"):
+            import xgboost as xgb
+            return np.asarray(model.predict(xgb.DMatrix(X)))
+        return model.predict_proba(X)[:, 1]
+    p0 = pd.Series(proba(df), index=ids)  # unclipped baseline
+    levers = shap_df[shap_df["actionable"].isin(actionable_values)]
+    records_by_feat = levers.groupby("feature")[unique_id].apply(set)
+    Xw = df.copy()  # single reusable working matrix
+    parts = []
+    for f, recs in records_by_feat.items():
+        if f not in df.columns:
+            continue
+        cur = df[f]
+        in_scope = pd.Series(ids.isin(recs), index=ids)
+        allowed = direction.get(f, "both")
+        orig = df[f].values
+        for q in candidate_quantiles:
+            cand = df.groupby(grp)[f].transform(lambda s: s.quantile(q))
+            move = in_scope & cand.notna() & (cand != cur)
+            if allowed == "up":
+                move &= cand > cur
+            elif allowed == "down":
+                move &= cand < cur
+            if not move.any():
+                continue
+            Xw[f] = np.where(move.values, cand.values, orig)
+            p1 = pd.Series(proba(Xw), index=ids)
+            m = move.values
+            parts.append(pd.DataFrame({
+                unique_id: ids[m],
+                "feature": f,
+                "current_value": cur[m].values,
+                "intervention_value": cand[m].values,
+                "intervention_quantile": q,
+                "peer_group": grp[m].values,
+                "p_before": p0[m].values,
+                "p_after": p1[m].values,
+                "delta_p": (p1 - p0)[m].values,
+            }))
+        Xw[f] = orig  # restore before next feature
+    if not parts:
+        return pd.DataFrame()
+    out = pd.concat(parts, ignore_index=True)
+    # Keep the single best-scoring candidate per (record, feature).
+    out = (out.sort_values("delta_p", ascending=False)
+              .drop_duplicates([unique_id, "feature"], keep="first"))
+    out = out[out["delta_p"] >= min_delta_p]
+    if impact_medium >= impact_high:
+        raise ValueError(f"impact_medium ({impact_medium}) must be less than impact_high ({impact_high})")
+    out["impact_bucket"] = pd.cut(out["delta_p"],
+                                   bins=[-np.inf, impact_medium, impact_high, np.inf],
+                                   labels=["Low", "Medium", "High"]).astype(str)
+    # Band-crossing floor: a move that pushes the record into a better band is at
+    # least "Medium" impact even if its raw delta_p is small (see docstring).
+    _sorted_bands = sorted(bands.items(), key=lambda kv: kv[1])
+    _edges = [-np.inf] + [cutoff for _, cutoff in _sorted_bands]
+    _labels = [label for label, _ in _sorted_bands]
+    def _band(p):
+        return pd.cut(p, bins=_edges, labels=_labels).astype(str)
+    out["band_before"] = _band(out["p_before"])
+    out["band_after"] = _band(out["p_after"])
+    out["crosses_band"] = out["band_before"] != out["band_after"]
+    out.loc[out["crosses_band"] & (out["impact_bucket"] == "Low"), "impact_bucket"] = "Medium"
+    return out.sort_values([unique_id, "delta_p"], ascending=[True, False]).reset_index(drop=True)
+
+
+def per_record_peer_percentiles(
+    df,
+    feature_dict,
+    peer_group=None,
+    unique_id="record_id",
+    actionable_values=["Yes", "Partial"],
+    features=None,
+    n_bins=5,
+):
+    """
+    Peer benchmarking. Each record's percentile for each feature within its
+    peer group, plus the group's p25/p50/p75 for context. Pure pandas (no model, no LLM).
+
+    df : model features df indexed by unique_id.
+    feature_dict : DataFrame/path; uses 'field' + 'actionable'.
+    peer_group : Series/DataFrame indexed by unique_id; numeric columns auto-binned into
+        n_bins quantile bands. None -> one global group.
+    features : optional explicit list. If given, used as-is (just needs to be a column
+        in df) -- no actionable filter, since an explicit request means you know what
+        you want, including non-actionable descriptive fields.
+        If None (default), computed for every actionable model feature in df.
+
+    Returns a long DataFrame: one row per (record, feature).
+    """
+    if isinstance(feature_dict, str):
+        feature_dict = pd.read_csv(feature_dict)
+    if features is not None:
+        cols = [f for f in features if f in df.columns]
+    else:
+        act_fields = set(feature_dict.loc[feature_dict["actionable"].isin(actionable_values), "field"])
+        cols = [f for f in df.columns if f in act_fields]
+    ids = df.index
+    # Peer-group key: numeric peer columns auto-binned into n_bins quantile bands.
+    if peer_group is None:
+        grp = pd.Series("all", index=ids)
+    else:
+        pk = peer_group.reindex(ids)
+        pk = pk.to_frame() if isinstance(pk, pd.Series) else pk
+        key = None
+        for c in pk.columns:
+            band = (pd.qcut(pk[c], q=n_bins, duplicates="drop").astype(str)
+                    if pd.api.types.is_numeric_dtype(pk[c]) else pk[c].astype(str))
+            key = band if key is None else key.str.cat(band, sep=" | ")
+        grp = pd.Series(np.asarray(key), index=ids)
+    parts = []
+    for f in cols:
+        g = df.groupby(grp)[f]
+        parts.append(pd.DataFrame({
+            unique_id: ids,
+            "feature": f,
+            "record_value": df[f].values,
+            "peer_group": grp.values,
+            "percentile": g.rank(pct=True).values,
+            "peer_p25": g.transform(lambda s: s.quantile(0.25)).values,
+            "peer_p50": g.transform(lambda s: s.quantile(0.50)).values,
+            "peer_p75": g.transform(lambda s: s.quantile(0.75)).values,
+        }))
+    if not parts:
+        return pd.DataFrame(columns=[unique_id, "feature", "record_value", "peer_group",
+                                     "percentile", "peer_p25", "peer_p50", "peer_p75"])
+    out = pd.concat(parts, ignore_index=True)
+    return out.sort_values([unique_id, "feature"]).reset_index(drop=True)

{gitlabds-2.1.11 → gitlabds-2.2.0}/gitlabds.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitlabds
-Version: 2.1.11
+Version: 2.2.0
 Summary: GitLab Data Science Tools
 Author-email: Kevin Dietz <kdietz@gitlab.com>
 License-Expression: MIT
@@ -63,7 +63,7 @@ pip install gitlabds
 
 #### Snowflake Feature Store
 
-The `serve_features` and `clear_feature_serving_locks` functions require additional Snowflake dependencies.
+The `serve_features`, `clear_feature_serving_locks`, and `get_feature_descriptions` functions require additional Snowflake dependencies.
 
 To use these functions, install with:
 
@@ -815,6 +815,151 @@ results = gitlabds.prescriptions(
 ```
 </details>
 
+### Record Benchmarking
+
+<details><summary> Value vs. Peer Status </summary>
+
+#### Description
+Status label for a value vs. its peer median (+/- tolerance), not percentile rank.
+
+`gitlabds.value_vs_peer_status(value, peer_median, good_direction=None, tolerance=0.15, no_activity_value=None, peer_percentile=None, ceiling_pct=0.95, ceiling_label="near ceiling", labels=("below average", "average", "above average", "No activity")):`
+
+#### Parameters:
+- _**value**_ : value(s) to evaluate, scalar or array-like
+- _**peer_median**_ : peer group median(s) to compare against
+- _**good_direction**_ : `"decreasing"` flips which side of the median counts as "above" (e.g., for a metric where lower is better). Anything else is treated as increasing/default.
+- _**tolerance**_ : relative distance from the peer median (as a fraction of the median) required to be classified above/below average rather than "average". Default is 0.15 (a value must be >15% above/below its peer median to leave the "average" band)
+- _**no_activity_value**_ : value that should always map to `labels[3]` ("No activity") regardless of tolerance
+- _**peer_percentile**_ : optional 0-1 percentile rank within the peer group. When provided, an "above average" reading that also sits in the top `ceiling_pct` toward `good_direction` is upgraded to `ceiling_label`
+- _**ceiling_pct**_ : percentile threshold used with `peer_percentile` to detect a "tapped out" strength. Default is 0.95
+- _**ceiling_label**_ : label applied when the ceiling condition is met. Default is "near ceiling"
+- _**labels**_ : tuple of (below average, average, above average, no activity) labels
+
+#### Returns
+- Series of status labels, one per input value
+
+#### Examples:
+```python
+import gitlabds
+
+status = gitlabds.value_vs_peer_status(
+    value=df["monthly_spend"],
+    peer_median=df["peer_median_spend"],
+    peer_percentile=df["peer_percentile_spend"],
+)
+```
+
+</details>
+
+<details><summary> Per-Record SHAP </summary>
+
+#### Description
+Per-record SHAP attribution (log-odds / margin space) for a tree model. Uses tree_path_dependent TreeSHAP, which natively handles the model's categorical splits. Returns the top_n features by |contribution| per record, guaranteeing at least min_actionable actionable ones.
+
+`gitlabds.per_record_shap(model, df, feature_dict, unique_id="record_id", top_n=25, min_actionable=3, actionable_values=["Yes", "Partial"]):`
+
+#### Parameters:
+- _**model**_ : trained tree-based model (e.g., LightGBM, XGBoost)
+- _**df**_ : model-ready feature matrix, indexed by `unique_id`
+- _**feature_dict**_ : DataFrame or path to CSV with `field` and `actionable` columns
+- _**unique_id**_ : name to use for the record identifier column in the output. Default is "record_id"
+- _**top_n**_ : number of features to return per record, ranked by |contribution|. Default is 25
+- _**min_actionable**_ : minimum number of actionable features guaranteed per record, even if outside the top_n. Default is 3
+- _**actionable_values**_ : values in `feature_dict["actionable"]` treated as actionable
+
+#### Returns
+- Long DataFrame, one row per (record, feature): `unique_id`, `rank`, `feature`, `contribution`, `direction`, `feature_value`, `actionable`
+
+#### Requirements
+This function requires the `shap` package (included in core dependencies).
+
+#### Examples:
+```python
+import gitlabds
+
+shap_df = gitlabds.per_record_shap(
+    model=trained_model,
+    df=model_features,
+    feature_dict="feature_dict.csv",
+)
+```
+
+</details>
+
+<details><summary> Per-Record Counterfactuals </summary>
+
+#### Description
+"Change X -> +Y" counterfactual search, direction chosen empirically. For each actionable lever SHAP surfaced for a record, evaluates candidate values drawn from peer-group quantiles, re-scores the model at each, and keeps the single move that maximizes the gain in predicted probability. No monotonicity assumption is made.
+
+`gitlabds.per_record_counterfactuals(model, df, shap_df, model_algo="lgb", unique_id="record_id", peer_group=None, candidate_quantiles=[0.1, 0.25, 0.5, 0.75, 0.9], direction=None, actionable_values=["Yes", "Partial"], min_delta_p=0.005, impact_medium=0.01, impact_high=0.03, bands={"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}, n_bins=5):`
+
+#### Parameters:
+- _**model**_ : trained model (as in `per_record_shap`)
+- _**df**_ : model-ready feature matrix, indexed by `unique_id`
+- _**shap_df**_ : output of `per_record_shap`; scopes which (record, feature) levers to test
+- _**model_algo**_ : `"lgb"` or `"xgb"`/`"xgboost"`; controls how probabilities are scored
+- _**unique_id**_ : name of the record identifier column. Default is "record_id"
+- _**peer_group**_ : Series/DataFrame indexed by `unique_id`; numeric columns are auto-binned into `n_bins` quantile bands. `None` uses one global group
+- _**candidate_quantiles**_ : peer-group quantiles evaluated as candidate values
+- _**direction**_ : optional per-feature achievability constraint mapping feature -> `'increasing'` | `'decreasing'` | `'both'` (dict or Series)
+- _**actionable_values**_ : values in `shap_df["actionable"]` treated as actionable levers
+- _**min_delta_p**_ : drop levers whose best achievable gain is below this
+- _**impact_medium, impact_high**_ : delta_p thresholds bucketing each lever into Low/Medium/High `impact_bucket`
+- _**bands**_ : dict mapping label -> upper bound on `p_before`/`p_after` for that band, ordered worst to best (e.g., default `{"poor fit": 0.03, "good fit": 0.10, "best fit": float("inf")}` means `p <= 0.03` -> "poor fit", `0.03 < p <= 0.10` -> "good fit", `p > 0.10` -> "best fit"). Key order doesn't matter (sorted by cutoff), but the top band's cutoff must be `float("inf")`. Used to detect whether a move crosses a band boundary (any crossing move is floored at "Medium" impact). Pass your own dict (e.g., `{"low": 0.03, "medium": 0.10, "high": float("inf")}`) to match your own pipeline
+- _**n_bins**_ : quantile bins for numeric `peer_group` columns
+
+#### Returns
+- Long DataFrame, one row per (record, lever): `current_value`, `intervention_value`, `intervention_quantile`, `p_before`, `p_after`, `delta_p`, `impact_bucket`, `band_before`, `band_after`, `crosses_band`
+
+#### Requirements
+For XGBoost models without `predict_proba`, requires the `xgboost` package (included in core dependencies).
+
+#### Examples:
+```python
+import gitlabds
+
+counterfactuals_df = gitlabds.per_record_counterfactuals(
+    model=trained_model,
+    df=model_features,
+    shap_df=shap_df,
+    peer_group=peer_group_df,
+)
+```
+
+</details>
+
+<details><summary> Per-Record Peer Percentiles </summary>
+
+#### Description
+Peer benchmarking. Each record's percentile for each feature within its peer group, plus the group's p25/p50/p75 for context. Pure pandas -- no model required.
+
+`gitlabds.per_record_peer_percentiles(df, feature_dict, peer_group=None, unique_id="record_id", actionable_values=["Yes", "Partial"], features=None, n_bins=5):`
+
+#### Parameters:
+- _**df**_ : model features DataFrame, indexed by `unique_id`
+- _**feature_dict**_ : DataFrame or path to CSV with `field` and `actionable` columns
+- _**peer_group**_ : Series/DataFrame indexed by `unique_id`; numeric columns are auto-binned into `n_bins` quantile bands. `None` uses one global group
+- _**unique_id**_ : name of the record identifier column. Default is "record_id"
+- _**actionable_values**_ : values in `feature_dict["actionable"]` treated as actionable
+- _**features**_ : optional explicit list of features to benchmark (used as-is, no actionable filter). If `None` (default), computed for every actionable feature in `df`
+- _**n_bins**_ : quantile bins for numeric `peer_group` columns
+
+#### Returns
+- Long DataFrame, one row per (record, feature): `unique_id`, `feature`, `record_value`, `peer_group`, `percentile`, `peer_p25`, `peer_p50`, `peer_p75`
+
+#### Examples:
+```python
+import gitlabds
+
+percentiles_df = gitlabds.per_record_peer_percentiles(
+    df=model_features,
+    feature_dict="feature_dict.csv",
+    peer_group=peer_group_df,
+)
+```
+
+</details>
+
 ### Model Monitoring
 
 <details><summary> Generate Baseline Features </summary>
@@ -1072,6 +1217,45 @@ print(f"Cleared {locks_cleared} locks")
 
 </details>
 
+<details><summary> Get Feature Descriptions </summary>
+
+#### Description
+Look up each feature's source feature view and description from the feature store's registered feature views (column comments set via `FeatureView.attach_feature_desc(...)`). Useful for documenting a model's fields or auditing everything registered in the store.
+
+`gitlabds.get_feature_descriptions(session, feature_store, features=None):`
+
+#### Parameters:
+- **_session_** : Active Snowpark session (e.g., `fs_engine._session`)
+- **_feature_store_** : Initialized FeatureStore instance (e.g., `fs_engine`); used to enumerate the registered feature views and locate their db/schema
+- **_features_** : Feature names to look up. If None, returns every feature in the store's registered feature views.
+
+#### Returns
+- DataFrame with columns: `feature`, `feature_view`, `data_type`, `description`. Features found in more than one feature view have their views/descriptions joined (`" | "` / `" || "`) so nothing is dropped. When `features` is provided, unmatched features (e.g., transformed or dummy-coded features) come back with blank `feature_view` / `data_type` / `description` for the user to populate manually, and the caller's order/list is preserved.
+
+#### Requirements
+This function requires Snowflake packages. Install with:
+```bash
+pip install snowflake-snowpark-python snowflake-ml-python
+```
+
+#### Examples:
+
+```python
+import gitlabds
+
+# Look up a specific list (e.g. the model fields from a config)
+desc = gitlabds.get_feature_descriptions(
+    session=fs_engine._session,
+    feature_store=fs_engine,
+    features=scoring_config["model"]["fields"],
+)
+
+# Or pull every feature registered in the store
+all_desc = gitlabds.get_feature_descriptions(session=fs_engine._session, feature_store=fs_engine)
+```
+
+</details>
+
 ### SQL and Trend Analysis
 
 <details><summary> SQL Trend Query Generator </summary>

{gitlabds-2.1.11 → gitlabds-2.2.0}/gitlabds.egg-info/SOURCES.txt

@@ -6,6 +6,7 @@ gitlabds/__init__.py
 gitlabds/baselines.py
 gitlabds/config_generator.py
 gitlabds/dummy.py
+gitlabds/feature_descriptions.py
 gitlabds/feature_reduction.py
 gitlabds/insights.py
 gitlabds/memory_optimization.py
@@ -13,6 +14,7 @@ gitlabds/missing.py
 gitlabds/model_evaluator.py
 gitlabds/monitoring_metrics.py
 gitlabds/outliers.py
+gitlabds/record_benchmarking.py
 gitlabds/serving_features.py
 gitlabds/split_data.py
 gitlabds/trends.py