comparative-edge 0.1.0__tar.gz

comparative_edge-0.1.0/PKG-INFO

@@ -0,0 +1,499 @@
+Metadata-Version: 2.4
+Name: comparative-edge
+Version: 0.1.0
+Summary: Discover where outcomes behave reliably, validate the behavior, and compare it against an external expectation.
+Author: Henry
+Author-email: osas2henry@gmail.com
+License: MIT
+Keywords: validation,binary-outcome,edge-detection,candidate-ranges,pandas
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.23
+Requires-Dist: pandas>=1.5
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: keywords
+Dynamic: license
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# Comparative Edge
+
+**Discover where outcomes behave reliably. Establish what your model can realistically expect. Compare that expectation with the outside world.**
+
+Comparative Edge is a lightweight Python toolkit for discovering **validated performance ranges** from historical binary outcomes.
+
+It is designed for situations where you have:
+
+* a measurable value or condition,
+* a binary outcome represented by `0` or `1`,
+* historical observations,
+* and an external expectation, benchmark, target, or opportunity that you eventually want to compare against your own model.
+
+The central philosophy:
+
+> **Model first. Comparison second. Decision last.**
+
+Comparative Edge does not allow an external expectation to define what your model should believe. It first uses your own historical data to discover and validate behavioral ranges. Only after an expectation has been established should it be compared with an external reference.
+
+---
+
+# The Problem
+
+Many decisions involve two sources of information: what **your own data says should happen**, and what **the outside world says should happen**.
+
+The outside expectation may be attractive. It may also be wrong for the conditions you're dealing with. The danger is letting that external expectation influence the model itself.
+
+For example, imagine a company running paid advertising. Every day it records:
+
+```text
+ad_spend    outcome
+───────────────────
+50             0
+75             1
+100            1
+125            0
+150            1
+175            1
+200            0
+```
+
+Here `1` means the desired outcome occurred and `0` means it didn't. The business wants to know whether the amount it spends is associated with a reliable outcome.
+
+Comparative Edge can search the historical values and discover that certain spending regions behave differently:
+
+```text
+ad spend range      observed outcome
+────────────────────────────────────
+<= $80                    48%
+$80 – $140                61%
+$140 – $200               54%
+>= $200                   43%
+```
+
+The important discovery isn't simply that one range has the highest percentage. The question is:
+
+> **Which behavior is sufficiently consistent across historical periods to establish a realistic expectation?**
+
+Once that expectation is established, it can be compared with an external one. Suppose an external campaign target assumes:
+
+```text
+Expected outcome = 75%
+```
+
+while your validated historical expectation is:
+
+```text
+Your model = 61%
+```
+
+Comparative Edge exposes the disagreement:
+
+```text
+Your model              External expectation
+    61%                         75%
+     │                           │
+     └───────────┬───────────────┘
+                 ↓
+             -14% gap
+```
+
+The package doesn't tell the business what to do. It makes the **risk/reward difference visible before commitment**.
+
+---
+
+# Core Philosophy
+
+## 1. Let your data establish the expectation
+
+The model should establish its own expectation from historical evidence. An external benchmark should not decide what the model ought to produce.
+
+```text
+Historical data → Model behavior → Validated expectation
+```
+
+## 2. Validate the behavior
+
+A range that looks good across the entire dataset may not actually be reliable. Comparative Edge tests candidate ranges across multiple validation folds and eliminates ranges whose behavior falls outside the required performance band.
+
+```text
+Candidate range
+      ↓
+Fold 1 → Pass
+      ↓
+Fold 2 → Pass
+      ↓
+Fold 3 → Pass
+      ↓
+Fold 4 → Pass
+      ↓
+Fold 5 → Pass
+      ↓
+Validated range
+```
+
+The objective isn't an impressive historical result. It's **repeatable behavior**.
+
+## 3. Compare only after establishing your expectation
+
+```text
+Your model → Your expectation → COMPARE ← External expectation
+```
+
+This preserves the independence of the model.
+
+## 4. Use the difference to understand risk and reward
+
+The comparison doesn't automatically determine whether an opportunity is good or bad. It tells you how far the external expectation is from what your own evidence supports. That difference can be used as a decision boundary.
+
+---
+
+# Binary Outcomes
+
+```text
+1 = desired outcome occurred
+0 = desired outcome did not occur
+```
+
+Applicable to many problems: sale/no sale, conversion/no conversion, success/failure, approved/rejected, retained/churned, delivery/no delivery, event/no event.
+
+The other column represents the **value or condition** whose relationship with the outcome you want to investigate, e.g. `customer_score → conversion`, `ad_spend → sale`, `processing_time → success`, `temperature → failure`.
+
+---
+
+# What the Algorithm Does
+
+```text
+DataFrame
+   ↓
+Select value + outcome
+   ↓
+Generate percentile edges
+   ↓
+Create candidate boundaries
+   ↓
+Test candidates on validation folds
+   ↓
+Remove weak or unstable candidates
+   ↓
+Score surviving candidates
+   ↓
+Return the winning validated boundary
+```
+
+---
+
+# Candidate Directions
+
+Comparative Edge doesn't assume higher values are better. It can investigate three shapes of behavior.
+
+## Left
+
+Find behavior below a threshold: `value <= edge`. Example: `ad_spend <= $100`.
+
+## Right
+
+Find behavior above a threshold: `value >= edge`. Example: `customer_score >= 75`.
+
+## Range
+
+Find behavior between two boundaries: `low <= value <= high`. Example: `$80 <= ad_spend <= $140`.
+
+## All
+
+Test all three shapes together and let them compete within the same candidate pool. Useful when you don't want to assume the relationship beforehand; the data determines whether the useful behavior occurs below a threshold, above a threshold, or inside a range.
+
+---
+
+# Percentile-Based Candidate Generation
+
+Rather than testing every possible numerical value, Comparative Edge creates candidate boundaries from percentile points, e.g. `0% - 7% - 14% - 21% - ... - 100%`. These percentile boundaries are converted into actual dataset values, and candidate ranges are constructed from those values. Duplicate percentile values are removed automatically.
+
+---
+
+# Validation
+
+Suppose `n_folds = 5`. The dataset is divided into five sequential folds. Each candidate must satisfy the configured performance requirements in every fold it's evaluated in; candidates that fail are eliminated.
+
+```text
+                    Candidates
+                        │
+          ┌─────────────┼─────────────┐
+          ↓             ↓             ↓
+        Fold 1        Fold 1        Fold 1
+          │             │             │
+         Pass          Fail          Pass
+          │             X             │
+        Fold 2                       Fold 2
+          │                            │
+         Pass                         Pass
+          │                            │
+        Fold 3                       Fold 3
+          │                            │
+         Pass                         Fail
+          │                            X
+        Survives
+```
+
+This makes the resulting expectation less dependent on a single portion of the dataset.
+
+---
+
+# Target Performance
+
+```python
+target_pct=50
+target_band_width=10
+```
+
+creates an acceptable performance band of `50% to 60%`. A candidate must fall inside this band during validation to remain a survivor. For a surviving candidate:
+
+```text
+observed = 57%, target = 50%  ->  difference = +7%
+observed = 46%, target = 50%  ->  difference = -4%
+```
+
+The sign preserves direction: negative means below expectation, zero means matches, positive means above.
+
+---
+
+# Minimum Sample Protection
+
+```python
+min_count=20
+```
+
+A candidate with fewer than the required observations in a validation fold is eliminated, preventing small samples from becoming apparently strong ranges.
+
+---
+
+# Installation
+
+```bash
+pip install comparative-edge
+```
+
+---
+
+# Basic Usage
+
+The DataFrame should contain exactly one binary outcome column and one numeric value column. `min_count` requires that many rows to fall inside a candidate range **within every fold**, so make sure your dataset is large enough for your chosen `n_folds` and `min_count` (roughly `n_folds * min_count` rows at minimum):
+
+```python
+import numpy as np
+import pandas as pd
+
+rng = np.random.default_rng(7)
+n = 500
+ad_spend = rng.uniform(50, 300, n).round(2)
+# outcome is more likely when ad_spend falls between 100 and 200
+prob = np.where((ad_spend > 100) & (ad_spend < 200), 0.58, 0.30)
+outcome = (rng.random(n) < prob).astype(int)
+
+df = pd.DataFrame({"ad_spend": ad_spend, "outcome": outcome})
+```
+
+```python
+from comparative_edge import search
+
+winner = search(
+    df,
+    outcome_col="outcome",
+    target_pct=50,
+    range_bins=15,
+    target_band_width=10,
+    n_folds=5,
+    min_count=20,
+)
+# winner -> (126.63, 231.62)
+```
+
+---
+
+# Direction
+
+Default is `direction="all"`. You can restrict the search:
+
+```python
+search(df, outcome_col="outcome", target_pct=50, direction="left")   # below a threshold
+search(df, outcome_col="outcome", target_pct=50, direction="right")  # above a threshold
+search(df, outcome_col="outcome", target_pct=50, direction="range")  # bounded ranges only
+search(df, outcome_col="outcome", target_pct=50, direction="all")    # everything
+```
+
+---
+
+# Parameters
+
+| Parameter           | Default | Description                                                        |
+| -------------------- | ------: | ------------------------------------------------------------------ |
+| `df`                 |     n/a | Input DataFrame: exactly one value column plus `outcome_col`       |
+| `outcome_col`        |     n/a | Binary `0/1` outcome column                                        |
+| `target_pct`         |     n/a | **Required.** Target/reference outcome percentage, `0` to `100`    |
+| `range_bins`         |    `15` | Number of percentile boundaries used to build candidates (`>= 2`)  |
+| `target_band_width`  |    `10` | Width of the acceptable target band above `target_pct` (`>= 0`, and `target_pct + target_band_width <= 100`) |
+| `n_folds`            |     `5` | Number of validation folds, `1` to `9`                             |
+| `min_count`          |    `20` | Minimum observations required per candidate per fold (`>= 1`)      |
+| `direction`          | `"all"` | `"left"`, `"right"`, `"range"`, or `"all"`                         |
+| `display`            | `False` | Print the analysis output                                          |
+| `verbose`            | `False` | Print detailed fold-level candidate keep/drop information          |
+
+Invalid values for any parameter raise `ValueError` or `TypeError` up front, before any search runs.
+
+---
+
+# Display Mode
+
+```python
+display=True
+```
+
+Example output:
+
+```text
+Comparative Edge
+ad_spend vs outcome
+
+  target       50-60%
+  direction    all
+  range_bins   15
+  n_folds      5-fold
+  min_count    20
+
+Results   (total: 10)
+  Range                General_pct   Vs_target Coverage_pct  Coverage_n
+  <=80.000                   51.20        1.20        20.00           2
+  (80.000, 140.000)          58.43        8.43        40.00           4
+  (140.000, 200.000)         54.91        4.91        30.00           3
+  >=200.000                  47.10       -2.90        10.00           1
+
+Winner   ad_spend (80.000, 140.000)
+hit_pct = 58.43%  ·  target 50-60%  ·  +8.43 over floor  ·  coverage 40.0% (4 / 10)
+```
+
+`Coverage_pct` and `Coverage_n` show what share (and raw count) of the full dataset each surviving range covers, out of the `total` printed above the table. The winner line always shows a signed `vs_target` value: `+8.43` when the range beats the target floor, `-2.90` if it ends up below it.
+
+```python
+verbose=True
+```
+
+also shows how candidates were kept or eliminated in each fold.
+
+---
+
+# Understanding the Result
+
+If `search` returns `(80.0, 140.0)`, the winning validated region is approximately `80 <= value <= 140`:
+
+```text
+Range:       80 - 140
+Hit rate:    58.43%
+Target:      50%
+Difference:  +8.43%
+Coverage:    40.0% of all observations (4 / 10)
+```
+
+This region demonstrated an observed outcome rate above the target while surviving the configured validation process, along with how much of your data that region represents.
+
+If no candidate survives validation, `search` returns `None`. Unbounded sides come back as `None` rather than `-inf`/`inf`: a left-direction winner looks like `(None, 45.2)`, a right-direction winner like `(12.0, None)`.
+
+---
+
+# Comparative Edge Is Not a Prediction Engine
+
+It is a **validation and comparison layer**, not a replacement for your predictive model. Its role is to help answer:
+
+> **Where does my model demonstrate reliable behavior, and how does that expectation compare with an external reference?**
+
+It does not determine whether an external benchmark is correct, whether an opportunity should be accepted, whether a model is causally correct, or whether historical behavior will continue indefinitely. It provides evidence for those decisions.
+
+---
+
+# Model First, External Expectation Second
+
+Consider a model that expects 60% performance, and an external source claiming 75%: a 15-point disagreement. Comparative Edge doesn't respond by changing the model to 75%. Instead, the difference becomes information. Maybe the external expectation is optimistic. Maybe the model is missing a variable. Maybe the historical sample doesn't represent current conditions. Maybe the opportunity genuinely reflects a change in conditions. The package doesn't assume which explanation is correct; it makes the disagreement visible so it can be investigated.
+
+---
+
+# Risk / Reward
+
+The comparison becomes:
+
+```text
+What my evidence supports   vs   What the opportunity requires
+```
+
+A large disagreement can indicate a poor fit between the opportunity and the model's demonstrated behavior. A close agreement can indicate the opportunity is more consistent with the model's expectation. This doesn't automatically make an opportunity good or bad; it provides a **risk/reward reference point** for the decision-maker.
+
+---
+
+# Applications
+
+Comparative Edge is industry-neutral. The same framework applies anywhere an outcome can be `0` or `1`:
+
+* **Advertising**: `ad_spend → conversion / no conversion` - does spend correspond with expected outcome?
+* **Sales**: `lead_value → sale / no sale` - which value ranges consistently produce a sale?
+* **Operations**: `processing_time → success / failure` - which operating ranges see more consistent success?
+* **Customer Analytics**: `customer_score → retention / churn` - which score ranges show reliable retention?
+* **Quality Control**: `measurement → pass / fail` - which measurement ranges correspond with acceptable outcomes?
+* **Forecasting**: `forecast_value → event / no event` - where does the forecast show consistent historical behavior?
+
+The domain changes; the underlying question stays the same:
+
+> **What does my own evidence realistically support, and how does that compare with what is expected externally?**
+
+---
+
+# Limitations
+
+The current design expects exactly one numeric value column besides the outcome column, a binary `0`/`1` outcome, sufficient observations for validation, and numerical values that can be ordered into meaningful boundaries.
+
+The percentile edges used to build candidates are computed from the **full** dataset before folding begins, and the winner's reported hit rate, `vs_target`, and coverage are also scored against the full dataset. Fold elimination genuinely filters out unstable candidates, but the final displayed numbers for the winner are in-sample, not held-out: treat them as a validated *screen*, not an out-of-sample performance guarantee.
+
+Validation folds are contiguous, sequential slices of the DataFrame in its existing row order, not shuffled or randomly sampled. If your data is ordered by date or by the value column itself, folds may not be independent; shuffle beforehand if that matters for your use case.
+
+It does not attempt to solve causal inference, multivariate feature interactions, time-series forecasting, probability calibration, model training, concept drift detection, or statistical significance testing. Those problems can complement Comparative Edge, but they're outside its core purpose.
+
+---
+
+# The Philosophy Behind the Package
+
+Comparative Edge was created around a practical problem: **how do you know whether an opportunity's expected reward is consistent with what your own model says is realistically achievable?**
+
+The answer shouldn't begin with the opportunity. It should begin with your evidence. First establish what your data supports. Then validate whether that behavior is repeatable. Then compare it with the outside expectation. Only then decide whether the difference represents an opportunity, a warning, or a reason to investigate further.
+
+```text
+        YOUR DATA
+            ↓
+       YOUR MODEL
+            ↓
+   VALIDATED BEHAVIOR
+            ↓
+     YOUR EXPECTATION
+            │
+        COMPARISON
+            │
+            ↑
+ EXTERNAL EXPECTATION
+            ↓
+    RISK / REWARD VIEW
+            ↓
+         DECISION
+```
+
+Comparative Edge isn't about making the model agree with the world. It's about **finding out where the model agrees with the world, where it disagrees, and how large that difference is.**
+
+> **Model first. Opportunity second. Comparison before commitment.**