edgepoint 2.0.2__tar.gz → 3.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- edgepoint-3.0.0/PKG-INFO +319 -0
- edgepoint-3.0.0/README.md +291 -0
- edgepoint-3.0.0/edgepoint/aggregate_kfold_combos.py +274 -0
- edgepoint-3.0.0/edgepoint/aggregate_kfold_edges.py +144 -0
- edgepoint-3.0.0/edgepoint/main.py +545 -0
- edgepoint-3.0.0/edgepoint/run_kfold_find.py +161 -0
- edgepoint-3.0.0/edgepoint/run_kfold_search.py +251 -0
- edgepoint-3.0.0/edgepoint/sanity_check.py +372 -0
- edgepoint-3.0.0/edgepoint/split_router.py +54 -0
- edgepoint-3.0.0/edgepoint/stratified_folds.py +81 -0
- edgepoint-3.0.0/edgepoint/test_and_combo_val.py +318 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint/train_combo.py +3 -37
- edgepoint-3.0.0/edgepoint/utils.py +93 -0
- edgepoint-3.0.0/edgepoint/validations.py +62 -0
- edgepoint-3.0.0/edgepoint.egg-info/PKG-INFO +319 -0
- edgepoint-3.0.0/edgepoint.egg-info/SOURCES.txt +25 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/pyproject.toml +1 -1
- {edgepoint-2.0.2 → edgepoint-3.0.0}/setup.py +1 -1
- edgepoint-2.0.2/PKG-INFO +0 -562
- edgepoint-2.0.2/README.md +0 -534
- edgepoint-2.0.2/edgepoint/validations.py +0 -122
- edgepoint-2.0.2/edgepoint.egg-info/PKG-INFO +0 -562
- edgepoint-2.0.2/edgepoint.egg-info/SOURCES.txt +0 -15
- {edgepoint-2.0.2 → edgepoint-3.0.0}/LICENSE +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint/__init__.py +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint/core.py +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint/core_v1.py +0 -0
- /edgepoint-2.0.2/edgepoint/main.py → /edgepoint-3.0.0/edgepoint/main_old.py +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint.egg-info/dependency_links.txt +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint.egg-info/requires.txt +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/edgepoint.egg-info/top_level.txt +0 -0
- {edgepoint-2.0.2 → edgepoint-3.0.0}/setup.cfg +0 -0
edgepoint-3.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,319 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: edgepoint
|
|
3
|
+
Version: 3.0.0
|
|
4
|
+
Summary: Find the point in a numeric column above which a binary outcome becomes meaningfully and reliably better.
|
|
5
|
+
Author: Henry
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: threshold,decision-threshold,binary-classification,feature-engineering,data-analysis,analytics,statistics,edge,threshold-detection
|
|
8
|
+
Classifier: Development Status :: 4 - Beta
|
|
9
|
+
Classifier: Intended Audience :: Developers
|
|
10
|
+
Classifier: Intended Audience :: Science/Research
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Topic :: Scientific/Engineering
|
|
18
|
+
Classifier: Topic :: Scientific/Engineering :: Information Analysis
|
|
19
|
+
Requires-Python: >=3.9
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE
|
|
22
|
+
Requires-Dist: numpy>=1.21
|
|
23
|
+
Requires-Dist: pandas>=1.5
|
|
24
|
+
Provides-Extra: test
|
|
25
|
+
Requires-Dist: pytest>=7.0; extra == "test"
|
|
26
|
+
Dynamic: license-file
|
|
27
|
+
Dynamic: requires-python
|
|
28
|
+
|
|
29
|
+
# edgepoint
|
|
30
|
+
|
|
31
|
+
> Find the point in a numeric column above which a binary outcome becomes meaningfully and reliably better, then check whether that finding actually holds up on data it never saw.
|
|
32
|
+
|
|
33
|
+
> Built for data analysts who aren't deep into data science and don't
|
|
34
|
+
> want to be, who just want a quick, trustworthy answer to a data
|
|
35
|
+
> question without writing a model, tuning hyperparameters, or learning
|
|
36
|
+
> a new toolbox to get there.
|
|
37
|
+
|
|
38
|
+
> **A note on terms:** throughout this README, "threshold", "edge
|
|
39
|
+
> point", "metric", "variable", and "column" are all used
|
|
40
|
+
> interchangeably. They all just mean a numeric column in your
|
|
41
|
+
> DataFrame.
|
|
42
|
+
|
|
43
|
+
`edgepoint` splits your data into train and test, searches train for the
|
|
44
|
+
best single-column threshold(s) *and* the best AND-combination(s) of
|
|
45
|
+
those thresholds, then replays every one of those exact picks against
|
|
46
|
+
test with no re-optimization. What you get back isn't just "the best
|
|
47
|
+
split we could find", it's that, plus an honest readout of how it held
|
|
48
|
+
up on data it never touched.
|
|
49
|
+
|
|
50
|
+
```python
|
|
51
|
+
from edgepoint import search
|
|
52
|
+
|
|
53
|
+
edgepoints_df, top_combos = search(df, outcome_col="hit")
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Correlation is not causation
|
|
59
|
+
|
|
60
|
+
`search()` finds correlations, not causes. A great-looking threshold
|
|
61
|
+
doesn't mean crossing it *causes* the outcome to improve, it may be
|
|
62
|
+
riding along with something else entirely, or just noise. This risk is
|
|
63
|
+
worse on small data: an 80% hit rate on 5 rows is far more likely to be
|
|
64
|
+
a fluke than the same 80% on 500 rows.
|
|
65
|
+
|
|
66
|
+
So supervise what you pass in. Only include columns with a plausible
|
|
67
|
+
reason to relate to the outcome, treat thin `n` results as leads to
|
|
68
|
+
investigate rather than decisions to act on, and re-run as more data
|
|
69
|
+
arrives rather than locking in a threshold found on a thin slice.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Where this applies
|
|
74
|
+
|
|
75
|
+
Any continuous metric (or several) plus a binary outcome pair, for example:
|
|
76
|
+
|
|
77
|
+
- **Marketing**: at what engagement score does churn risk drop off
|
|
78
|
+
- **Credit**: at what score does default rate become acceptable
|
|
79
|
+
- **Healthcare**: at what biomarker level does an outcome rate jump
|
|
80
|
+
- **Product**: at what usage count does upgrade to paid rate spike
|
|
81
|
+
- **Manufacturing**: at what sensor reading does defect rate climb
|
|
82
|
+
- **Trading / betting**: at what signal strength value (or combination of
|
|
83
|
+
signals) does a pick's hit rate become reliable
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## `outcome_col`: telling edgepoint what "good" means
|
|
88
|
+
|
|
89
|
+
`outcome_col` is your `0`/`1` (or `True`/`False`) judgment of what counts
|
|
90
|
+
as a win on each row. `edgepoint` has no idea what a good sale, patient
|
|
91
|
+
outcome, or bet looks like in your business. Everything else it computes
|
|
92
|
+
(`hit_rate`, `gap`, `coverage`, `score`) is just counting, over and over,
|
|
93
|
+
against the column you point it at.
|
|
94
|
+
|
|
95
|
+
- It must already exist. Derive it (e.g. "closed within 7 days") before
|
|
96
|
+
calling `search()`.
|
|
97
|
+
- One outcome per run. Two definitions of success means two calls.
|
|
98
|
+
- Rare outcomes (almost all `0`s or almost all `1`s) leave little contrast
|
|
99
|
+
to find an edge across, and columns are likely to get skipped for
|
|
100
|
+
failing `min_coverage`.
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## Quick start
|
|
105
|
+
|
|
106
|
+
```python
|
|
107
|
+
import pandas as pd
|
|
108
|
+
from edgepoint import search
|
|
109
|
+
|
|
110
|
+
df = pd.DataFrame({
|
|
111
|
+
"score": [
|
|
112
|
+
72.68, 58.92, 62.15, 8.13, 33.3, 66.19, 73.79, 35.43,
|
|
113
|
+
65.06, 63.13, 61.47, 45.13, 39.16, 38.15, 29.41, 59.91,
|
|
114
|
+
54.12, 86.35, 76.02, 63.13, 51.63, 0, 77.4, 81.66,
|
|
115
|
+
50.13, 52.4, 52.82, 86.12, 57.58, 13.66, 37.29, 32.91,
|
|
116
|
+
73.66, 96.2, 36.3, 22.74, 65.54, 23.98, 61.59, 32.27
|
|
117
|
+
],
|
|
118
|
+
"age": [
|
|
119
|
+
33.9, 41.9, 30.8, 56.5, 45.8, 45.1, 43.3, 54.3,
|
|
120
|
+
52.5, 54.2, 43.9, 18, 31.3, 38, 48.3, 55.8,
|
|
121
|
+
35.4, 56, 20.2, 27.7, 28.1, 39.4, 44.3, 39.9,
|
|
122
|
+
53.6, 26.1, 63.1, 42.3, 46.3, 20.2, 39.2, 35.3,
|
|
123
|
+
27.7, 31.8, 26.5, 32.2, 35.1, 43.5, 45.8, 43.8
|
|
124
|
+
],
|
|
125
|
+
"hit": [
|
|
126
|
+
1, 1, 0, 0, 0, 1, 1, 0,
|
|
127
|
+
0, 1, 1, 0, 0, 0, 0, 1,
|
|
128
|
+
1, 1, 1, 1, 0, 0, 0, 1,
|
|
129
|
+
0, 1, 1, 1, 0, 0, 0, 0,
|
|
130
|
+
1, 1, 0, 0, 1, 0, 1, 0
|
|
131
|
+
]
|
|
132
|
+
})
|
|
133
|
+
|
|
134
|
+
edgepoints_df, top_combos = search(
|
|
135
|
+
df,
|
|
136
|
+
outcome_col="hit",
|
|
137
|
+
mode="quick",
|
|
138
|
+
min_coverage=10,
|
|
139
|
+
show_progress=True,
|
|
140
|
+
top_combo_n=3,
|
|
141
|
+
)
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
With `show_progress=True`, `search()` prints a 6 stage log (data
|
|
145
|
+
sanity/splitting, train search, test validation, merged edges, merged
|
|
146
|
+
combos, top combo summary) as it runs. With `show_progress=False` it runs
|
|
147
|
+
silently and just returns the two values below.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Auto split & auto fold
|
|
152
|
+
|
|
153
|
+
There's no `train_split` or `k` parameter to set. Both are worked out
|
|
154
|
+
for you from the data itself:
|
|
155
|
+
|
|
156
|
+
- **Split**: `search()` takes `df` exactly as ordered and treats the
|
|
157
|
+
**oldest rows as train, the newest rows as test** (a walk forward
|
|
158
|
+
split; train never sees the future relative to test). The train/test
|
|
159
|
+
ratio is picked automatically from row count. If your rows aren't
|
|
160
|
+
already in chronological order, or you don't want a chronological
|
|
161
|
+
split at all, **shuffle `df` yourself before passing it in**.
|
|
162
|
+
`search()` never reorders or randomizes rows on its own.
|
|
163
|
+
- **Folds**: within train and again within test, the number of k-folds
|
|
164
|
+
used for validation is auto-sized off row count and `min_coverage`, so
|
|
165
|
+
each fold still has enough rows to mean something. Folds themselves are
|
|
166
|
+
**randomly stratified**, rows are shuffled and split so each fold keeps
|
|
167
|
+
roughly the same positive rate as the whole set, rather than plain
|
|
168
|
+
random slicing that could hand one fold almost all `0`s or almost all
|
|
169
|
+
`1`s.
|
|
170
|
+
|
|
171
|
+
---
|
|
172
|
+
|
|
173
|
+
## How the pipeline works
|
|
174
|
+
|
|
175
|
+
1. **Data sanity / splitting**: `df` is cleaned first. Invalid outcome
|
|
176
|
+
rows dropped, `True`/`False` normalized to `1`/`0`, non-numeric,
|
|
177
|
+
constant, or near-unique predictor columns dropped, remaining
|
|
178
|
+
`NaN`/`inf` filled. The cleaned data is then split into train/test
|
|
179
|
+
automatically, chronologically, based on row count, with no split
|
|
180
|
+
ratio to configure.
|
|
181
|
+
2. **Train section**: every numeric column is searched for its best
|
|
182
|
+
single-column threshold (percentile-based candidate edges, scored on
|
|
183
|
+
a shrinkage-discounted gap/coverage blend), then qualifying columns
|
|
184
|
+
are combined into AND-combinations (size capped by `mode`) and
|
|
185
|
+
k-fold validated on train itself.
|
|
186
|
+
3. **Test section**: every train-side edge and surviving combo is
|
|
187
|
+
replayed against test, unchanged, with its own k-fold validation.
|
|
188
|
+
4. **Merge edges**: train and test side stats for each column are
|
|
189
|
+
merged side by side (`_tr`/`_te` suffixes) into `edgepoints_df`,
|
|
190
|
+
ranked by a blended `final_score`.
|
|
191
|
+
5. **Merge combos**: same idea, but only combos that qualified on
|
|
192
|
+
*both* train and test survive.
|
|
193
|
+
6. **Top combos**: the best `top_combo_n` combos are printed and
|
|
194
|
+
returned as `top_combos`.
|
|
195
|
+
|
|
196
|
+
A combo (or column) only counts as a real edge if its hit rate clears the
|
|
197
|
+
**global baseline**, the dataset's own overall hit rate ("doing
|
|
198
|
+
nothing"), computed once on the full df and reused, unchanged, at every
|
|
199
|
+
stage. Nothing is ever re-tuned on test.
|
|
200
|
+
|
|
201
|
+
---
|
|
202
|
+
|
|
203
|
+
## Modes
|
|
204
|
+
|
|
205
|
+
`mode` controls how large a column combination can get during combo
|
|
206
|
+
search:
|
|
207
|
+
|
|
208
|
+
| Mode | Combo size |
|
|
209
|
+
| ------------ | -------------------------------------------------------------------- |
|
|
210
|
+
| `"quick"` | up to 2 columns per combo |
|
|
211
|
+
| `"standard"` | up to 3 columns per combo |
|
|
212
|
+
| `"deep"` | auto-fit, the largest size that keeps total combos under a fixed safety ceiling |
|
|
213
|
+
|
|
214
|
+
`"quick"`/`"standard"` are refused up front if they'd blow past that
|
|
215
|
+
ceiling given how many columns qualified.
|
|
216
|
+
|
|
217
|
+
---
|
|
218
|
+
|
|
219
|
+
## Parameters
|
|
220
|
+
|
|
221
|
+
| Parameter | Default | Description |
|
|
222
|
+
| -------------- | ------- | ----------------------------------------------------------------------------- |
|
|
223
|
+
| `df` | required | Input DataFrame |
|
|
224
|
+
| `outcome_col` | `"hit"` | Binary outcome column (`0`/`1`, `True`/`False`, or a mix) |
|
|
225
|
+
| `mode` | `"quick"` | `"quick"`, `"standard"`, or `"deep"`, controls combo size, see above |
|
|
226
|
+
| `min_coverage` | `10` | Minimum % of rows a threshold/combo must cover to qualify (train and test) |
|
|
227
|
+
| `show_progress`| `True` | Print the full stage-by-stage log |
|
|
228
|
+
| `top_combo_n` | `3` | How many top combos to return. `1` returns a single dict; more returns a list |
|
|
229
|
+
|
|
230
|
+
---
|
|
231
|
+
|
|
232
|
+
## Returns
|
|
233
|
+
|
|
234
|
+
`search()` returns a 2-tuple: `(edgepoints_df, top_combos)`.
|
|
235
|
+
|
|
236
|
+
**`edgepoints_df`**: one row per column that appeared on either side,
|
|
237
|
+
with train (`_tr`) and test (`_te`) versions of `hit_rate`, `miss_rate`,
|
|
238
|
+
`gap`, `n`, `coverage`, plus a blended `final_score`, sorted best first.
|
|
239
|
+
|
|
240
|
+
**`top_combos`**: a single dict if `top_combo_n=1`, else a list of up to
|
|
241
|
+
`top_combo_n` dicts, each shaped like:
|
|
242
|
+
|
|
243
|
+
```python
|
|
244
|
+
{
|
|
245
|
+
"combo_num": 3,
|
|
246
|
+
"combo": "score, age",
|
|
247
|
+
"edges": {"score": 49.46, "age": 39.5},
|
|
248
|
+
"hit_rate_tr": 80.0, "hit_rate_te": 60.0,
|
|
249
|
+
"coverage_tr": 41.67, "coverage_te": 31.25,
|
|
250
|
+
"n_tr": 10, "n_te": 5,
|
|
251
|
+
"score_tr": 0.62, "score_te": 0.41,
|
|
252
|
+
"final_score": 21.49,
|
|
253
|
+
}
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
If nothing qualified, you get a single dict with `combo_num: None` and
|
|
257
|
+
empty `edges`.
|
|
258
|
+
|
|
259
|
+
### What a combo actually tells you
|
|
260
|
+
|
|
261
|
+
Think of `top_combos` as the answer to *"which traits, combined,
|
|
262
|
+
actually separate my winners from everyone else, and by how much?"*
|
|
263
|
+
A single-column result might tell you customers 34+ buy at 58% vs 22%
|
|
264
|
+
for everyone else, useful, but crude, since plenty of 22-year-olds buy
|
|
265
|
+
too. A combo sharpens that: `age >= 34 AND minutes_on_site >= 6.5` might
|
|
266
|
+
come back at a 71% hit rate on train and 68% on test, covering 18% of
|
|
267
|
+
visitors against a 24% baseline. That's a different, more useful claim.
|
|
268
|
+
It's not age alone, it's that age *stacked with* engagement time is
|
|
269
|
+
where the real signal was hiding, and it held up on data the search
|
|
270
|
+
never saw. `top_combos` typically hands you a few of these to compare,
|
|
271
|
+
so you can weigh, say, "age + time on site" against "past purchases +
|
|
272
|
+
email opens," and pick whichever combo is both strong *and* backed by
|
|
273
|
+
enough rows (`n_tr`/`n_te`) to actually trust.
|
|
274
|
+
|
|
275
|
+
---
|
|
276
|
+
|
|
277
|
+
## Using the result correctly
|
|
278
|
+
|
|
279
|
+
The thresholds `search()` returns aren't meant to become permanent. They're
|
|
280
|
+
the best candidates given the data available right now, checked once
|
|
281
|
+
against a held-out slice of that same data. Re-run `search()` as new rows
|
|
282
|
+
arrive. Each run rechecks the spike against a fresh split and tells you
|
|
283
|
+
plainly if it moved, held, or disappeared. Treat it as proposing and
|
|
284
|
+
stress-testing a threshold once, not certifying one forever.
|
|
285
|
+
|
|
286
|
+
---
|
|
287
|
+
|
|
288
|
+
## Supported outcome values
|
|
289
|
+
|
|
290
|
+
`outcome_col` may contain `0`/`1`, `True`/`False`, `0.0`/`1.0`, or any mix
|
|
291
|
+
of the above. Any other value (strings, `2`, `-1`, etc.) raises a
|
|
292
|
+
`ValueError` before any column is searched. A full stop, not a
|
|
293
|
+
per-column skip.
|
|
294
|
+
|
|
295
|
+
---
|
|
296
|
+
|
|
297
|
+
## Design notes
|
|
298
|
+
|
|
299
|
+
- **Percentile-based candidate edges**: candidate edges follow the
|
|
300
|
+
column's own quantiles, so dense clusters get finer cuts and sparse
|
|
301
|
+
tails get wider ones.
|
|
302
|
+
- **Weighted score, not hard cutoffs**: `min_coverage` is a hard gate;
|
|
303
|
+
among what clears it, gap and coverage are blended into one score
|
|
304
|
+
rather than requiring both to clear fixed minimums independently.
|
|
305
|
+
- **Shrinkage-discounted gaps**: a dramatic gap on a handful of rows is
|
|
306
|
+
discounted toward zero before scoring, so it can't outrank a smaller,
|
|
307
|
+
better-supported gap. Raw hit rate/gap are always what's reported.
|
|
308
|
+
- **Sign-flip penalty**: if a combo's gap direction flips between train
|
|
309
|
+
and test, that's a concrete overfitting signal, so `final_score` is
|
|
310
|
+
halved for that row.
|
|
311
|
+
- **One-directional search**: `edgepoint` only asks "is there a point
|
|
312
|
+
above which the rate is *better*?" It doesn't independently verify the
|
|
313
|
+
below side against its own bar.
|
|
314
|
+
|
|
315
|
+
---
|
|
316
|
+
|
|
317
|
+
## License
|
|
318
|
+
|
|
319
|
+
MIT
|
|
@@ -0,0 +1,291 @@
|
|
|
1
|
+
# edgepoint
|
|
2
|
+
|
|
3
|
+
> Find the point in a numeric column above which a binary outcome becomes meaningfully and reliably better, then check whether that finding actually holds up on data it never saw.
|
|
4
|
+
|
|
5
|
+
> Built for data analysts who aren't deep into data science and don't
|
|
6
|
+
> want to be, who just want a quick, trustworthy answer to a data
|
|
7
|
+
> question without writing a model, tuning hyperparameters, or learning
|
|
8
|
+
> a new toolbox to get there.
|
|
9
|
+
|
|
10
|
+
> **A note on terms:** throughout this README, "threshold", "edge
|
|
11
|
+
> point", "metric", "variable", and "column" are all used
|
|
12
|
+
> interchangeably. They all just mean a numeric column in your
|
|
13
|
+
> DataFrame.
|
|
14
|
+
|
|
15
|
+
`edgepoint` splits your data into train and test, searches train for the
|
|
16
|
+
best single-column threshold(s) *and* the best AND-combination(s) of
|
|
17
|
+
those thresholds, then replays every one of those exact picks against
|
|
18
|
+
test with no re-optimization. What you get back isn't just "the best
|
|
19
|
+
split we could find", it's that, plus an honest readout of how it held
|
|
20
|
+
up on data it never touched.
|
|
21
|
+
|
|
22
|
+
```python
|
|
23
|
+
from edgepoint import search
|
|
24
|
+
|
|
25
|
+
edgepoints_df, top_combos = search(df, outcome_col="hit")
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Correlation is not causation
|
|
31
|
+
|
|
32
|
+
`search()` finds correlations, not causes. A great-looking threshold
|
|
33
|
+
doesn't mean crossing it *causes* the outcome to improve, it may be
|
|
34
|
+
riding along with something else entirely, or just noise. This risk is
|
|
35
|
+
worse on small data: an 80% hit rate on 5 rows is far more likely to be
|
|
36
|
+
a fluke than the same 80% on 500 rows.
|
|
37
|
+
|
|
38
|
+
So supervise what you pass in. Only include columns with a plausible
|
|
39
|
+
reason to relate to the outcome, treat thin `n` results as leads to
|
|
40
|
+
investigate rather than decisions to act on, and re-run as more data
|
|
41
|
+
arrives rather than locking in a threshold found on a thin slice.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Where this applies
|
|
46
|
+
|
|
47
|
+
Any continuous metric (or several) plus a binary outcome pair, for example:
|
|
48
|
+
|
|
49
|
+
- **Marketing**: at what engagement score does churn risk drop off
|
|
50
|
+
- **Credit**: at what score does default rate become acceptable
|
|
51
|
+
- **Healthcare**: at what biomarker level does an outcome rate jump
|
|
52
|
+
- **Product**: at what usage count does upgrade to paid rate spike
|
|
53
|
+
- **Manufacturing**: at what sensor reading does defect rate climb
|
|
54
|
+
- **Trading / betting**: at what signal strength value (or combination of
|
|
55
|
+
signals) does a pick's hit rate become reliable
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## `outcome_col`: telling edgepoint what "good" means
|
|
60
|
+
|
|
61
|
+
`outcome_col` is your `0`/`1` (or `True`/`False`) judgment of what counts
|
|
62
|
+
as a win on each row. `edgepoint` has no idea what a good sale, patient
|
|
63
|
+
outcome, or bet looks like in your business. Everything else it computes
|
|
64
|
+
(`hit_rate`, `gap`, `coverage`, `score`) is just counting, over and over,
|
|
65
|
+
against the column you point it at.
|
|
66
|
+
|
|
67
|
+
- It must already exist. Derive it (e.g. "closed within 7 days") before
|
|
68
|
+
calling `search()`.
|
|
69
|
+
- One outcome per run. Two definitions of success means two calls.
|
|
70
|
+
- Rare outcomes (almost all `0`s or almost all `1`s) leave little contrast
|
|
71
|
+
to find an edge across, and columns are likely to get skipped for
|
|
72
|
+
failing `min_coverage`.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Quick start
|
|
77
|
+
|
|
78
|
+
```python
|
|
79
|
+
import pandas as pd
|
|
80
|
+
from edgepoint import search
|
|
81
|
+
|
|
82
|
+
df = pd.DataFrame({
|
|
83
|
+
"score": [
|
|
84
|
+
72.68, 58.92, 62.15, 8.13, 33.3, 66.19, 73.79, 35.43,
|
|
85
|
+
65.06, 63.13, 61.47, 45.13, 39.16, 38.15, 29.41, 59.91,
|
|
86
|
+
54.12, 86.35, 76.02, 63.13, 51.63, 0, 77.4, 81.66,
|
|
87
|
+
50.13, 52.4, 52.82, 86.12, 57.58, 13.66, 37.29, 32.91,
|
|
88
|
+
73.66, 96.2, 36.3, 22.74, 65.54, 23.98, 61.59, 32.27
|
|
89
|
+
],
|
|
90
|
+
"age": [
|
|
91
|
+
33.9, 41.9, 30.8, 56.5, 45.8, 45.1, 43.3, 54.3,
|
|
92
|
+
52.5, 54.2, 43.9, 18, 31.3, 38, 48.3, 55.8,
|
|
93
|
+
35.4, 56, 20.2, 27.7, 28.1, 39.4, 44.3, 39.9,
|
|
94
|
+
53.6, 26.1, 63.1, 42.3, 46.3, 20.2, 39.2, 35.3,
|
|
95
|
+
27.7, 31.8, 26.5, 32.2, 35.1, 43.5, 45.8, 43.8
|
|
96
|
+
],
|
|
97
|
+
"hit": [
|
|
98
|
+
1, 1, 0, 0, 0, 1, 1, 0,
|
|
99
|
+
0, 1, 1, 0, 0, 0, 0, 1,
|
|
100
|
+
1, 1, 1, 1, 0, 0, 0, 1,
|
|
101
|
+
0, 1, 1, 1, 0, 0, 0, 0,
|
|
102
|
+
1, 1, 0, 0, 1, 0, 1, 0
|
|
103
|
+
]
|
|
104
|
+
})
|
|
105
|
+
|
|
106
|
+
edgepoints_df, top_combos = search(
|
|
107
|
+
df,
|
|
108
|
+
outcome_col="hit",
|
|
109
|
+
mode="quick",
|
|
110
|
+
min_coverage=10,
|
|
111
|
+
show_progress=True,
|
|
112
|
+
top_combo_n=3,
|
|
113
|
+
)
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
With `show_progress=True`, `search()` prints a 6 stage log (data
|
|
117
|
+
sanity/splitting, train search, test validation, merged edges, merged
|
|
118
|
+
combos, top combo summary) as it runs. With `show_progress=False` it runs
|
|
119
|
+
silently and just returns the two values below.
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
## Auto split & auto fold
|
|
124
|
+
|
|
125
|
+
There's no `train_split` or `k` parameter to set. Both are worked out
|
|
126
|
+
for you from the data itself:
|
|
127
|
+
|
|
128
|
+
- **Split**: `search()` takes `df` exactly as ordered and treats the
|
|
129
|
+
**oldest rows as train, the newest rows as test** (a walk forward
|
|
130
|
+
split; train never sees the future relative to test). The train/test
|
|
131
|
+
ratio is picked automatically from row count. If your rows aren't
|
|
132
|
+
already in chronological order, or you don't want a chronological
|
|
133
|
+
split at all, **shuffle `df` yourself before passing it in**.
|
|
134
|
+
`search()` never reorders or randomizes rows on its own.
|
|
135
|
+
- **Folds**: within train and again within test, the number of k-folds
|
|
136
|
+
used for validation is auto-sized off row count and `min_coverage`, so
|
|
137
|
+
each fold still has enough rows to mean something. Folds themselves are
|
|
138
|
+
**randomly stratified**, rows are shuffled and split so each fold keeps
|
|
139
|
+
roughly the same positive rate as the whole set, rather than plain
|
|
140
|
+
random slicing that could hand one fold almost all `0`s or almost all
|
|
141
|
+
`1`s.
|
|
142
|
+
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
## How the pipeline works
|
|
146
|
+
|
|
147
|
+
1. **Data sanity / splitting**: `df` is cleaned first. Invalid outcome
|
|
148
|
+
rows dropped, `True`/`False` normalized to `1`/`0`, non-numeric,
|
|
149
|
+
constant, or near-unique predictor columns dropped, remaining
|
|
150
|
+
`NaN`/`inf` filled. The cleaned data is then split into train/test
|
|
151
|
+
automatically, chronologically, based on row count, with no split
|
|
152
|
+
ratio to configure.
|
|
153
|
+
2. **Train section**: every numeric column is searched for its best
|
|
154
|
+
single-column threshold (percentile-based candidate edges, scored on
|
|
155
|
+
a shrinkage-discounted gap/coverage blend), then qualifying columns
|
|
156
|
+
are combined into AND-combinations (size capped by `mode`) and
|
|
157
|
+
k-fold validated on train itself.
|
|
158
|
+
3. **Test section**: every train-side edge and surviving combo is
|
|
159
|
+
replayed against test, unchanged, with its own k-fold validation.
|
|
160
|
+
4. **Merge edges**: train and test side stats for each column are
|
|
161
|
+
merged side by side (`_tr`/`_te` suffixes) into `edgepoints_df`,
|
|
162
|
+
ranked by a blended `final_score`.
|
|
163
|
+
5. **Merge combos**: same idea, but only combos that qualified on
|
|
164
|
+
*both* train and test survive.
|
|
165
|
+
6. **Top combos**: the best `top_combo_n` combos are printed and
|
|
166
|
+
returned as `top_combos`.
|
|
167
|
+
|
|
168
|
+
A combo (or column) only counts as a real edge if its hit rate clears the
|
|
169
|
+
**global baseline**, the dataset's own overall hit rate ("doing
|
|
170
|
+
nothing"), computed once on the full df and reused, unchanged, at every
|
|
171
|
+
stage. Nothing is ever re-tuned on test.
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Modes
|
|
176
|
+
|
|
177
|
+
`mode` controls how large a column combination can get during combo
|
|
178
|
+
search:
|
|
179
|
+
|
|
180
|
+
| Mode | Combo size |
|
|
181
|
+
| ------------ | -------------------------------------------------------------------- |
|
|
182
|
+
| `"quick"` | up to 2 columns per combo |
|
|
183
|
+
| `"standard"` | up to 3 columns per combo |
|
|
184
|
+
| `"deep"` | auto-fit, the largest size that keeps total combos under a fixed safety ceiling |
|
|
185
|
+
|
|
186
|
+
`"quick"`/`"standard"` are refused up front if they'd blow past that
|
|
187
|
+
ceiling given how many columns qualified.
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
## Parameters
|
|
192
|
+
|
|
193
|
+
| Parameter | Default | Description |
|
|
194
|
+
| -------------- | ------- | ----------------------------------------------------------------------------- |
|
|
195
|
+
| `df` | required | Input DataFrame |
|
|
196
|
+
| `outcome_col` | `"hit"` | Binary outcome column (`0`/`1`, `True`/`False`, or a mix) |
|
|
197
|
+
| `mode` | `"quick"` | `"quick"`, `"standard"`, or `"deep"`, controls combo size, see above |
|
|
198
|
+
| `min_coverage` | `10` | Minimum % of rows a threshold/combo must cover to qualify (train and test) |
|
|
199
|
+
| `show_progress`| `True` | Print the full stage-by-stage log |
|
|
200
|
+
| `top_combo_n` | `3` | How many top combos to return. `1` returns a single dict; more returns a list |
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Returns
|
|
205
|
+
|
|
206
|
+
`search()` returns a 2-tuple: `(edgepoints_df, top_combos)`.
|
|
207
|
+
|
|
208
|
+
**`edgepoints_df`**: one row per column that appeared on either side,
|
|
209
|
+
with train (`_tr`) and test (`_te`) versions of `hit_rate`, `miss_rate`,
|
|
210
|
+
`gap`, `n`, `coverage`, plus a blended `final_score`, sorted best first.
|
|
211
|
+
|
|
212
|
+
**`top_combos`**: a single dict if `top_combo_n=1`, else a list of up to
|
|
213
|
+
`top_combo_n` dicts, each shaped like:
|
|
214
|
+
|
|
215
|
+
```python
|
|
216
|
+
{
|
|
217
|
+
"combo_num": 3,
|
|
218
|
+
"combo": "score, age",
|
|
219
|
+
"edges": {"score": 49.46, "age": 39.5},
|
|
220
|
+
"hit_rate_tr": 80.0, "hit_rate_te": 60.0,
|
|
221
|
+
"coverage_tr": 41.67, "coverage_te": 31.25,
|
|
222
|
+
"n_tr": 10, "n_te": 5,
|
|
223
|
+
"score_tr": 0.62, "score_te": 0.41,
|
|
224
|
+
"final_score": 21.49,
|
|
225
|
+
}
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
If nothing qualified, you get a single dict with `combo_num: None` and
|
|
229
|
+
empty `edges`.
|
|
230
|
+
|
|
231
|
+
### What a combo actually tells you
|
|
232
|
+
|
|
233
|
+
Think of `top_combos` as the answer to *"which traits, combined,
|
|
234
|
+
actually separate my winners from everyone else, and by how much?"*
|
|
235
|
+
A single-column result might tell you customers 34+ buy at 58% vs 22%
|
|
236
|
+
for everyone else, useful, but crude, since plenty of 22-year-olds buy
|
|
237
|
+
too. A combo sharpens that: `age >= 34 AND minutes_on_site >= 6.5` might
|
|
238
|
+
come back at a 71% hit rate on train and 68% on test, covering 18% of
|
|
239
|
+
visitors against a 24% baseline. That's a different, more useful claim.
|
|
240
|
+
It's not age alone, it's that age *stacked with* engagement time is
|
|
241
|
+
where the real signal was hiding, and it held up on data the search
|
|
242
|
+
never saw. `top_combos` typically hands you a few of these to compare,
|
|
243
|
+
so you can weigh, say, "age + time on site" against "past purchases +
|
|
244
|
+
email opens," and pick whichever combo is both strong *and* backed by
|
|
245
|
+
enough rows (`n_tr`/`n_te`) to actually trust.
|
|
246
|
+
|
|
247
|
+
---
|
|
248
|
+
|
|
249
|
+
## Using the result correctly
|
|
250
|
+
|
|
251
|
+
The thresholds `search()` returns aren't meant to become permanent. They're
|
|
252
|
+
the best candidates given the data available right now, checked once
|
|
253
|
+
against a held-out slice of that same data. Re-run `search()` as new rows
|
|
254
|
+
arrive. Each run rechecks the spike against a fresh split and tells you
|
|
255
|
+
plainly if it moved, held, or disappeared. Treat it as proposing and
|
|
256
|
+
stress-testing a threshold once, not certifying one forever.
|
|
257
|
+
|
|
258
|
+
---
|
|
259
|
+
|
|
260
|
+
## Supported outcome values
|
|
261
|
+
|
|
262
|
+
`outcome_col` may contain `0`/`1`, `True`/`False`, `0.0`/`1.0`, or any mix
|
|
263
|
+
of the above. Any other value (strings, `2`, `-1`, etc.) raises a
|
|
264
|
+
`ValueError` before any column is searched. A full stop, not a
|
|
265
|
+
per-column skip.
|
|
266
|
+
|
|
267
|
+
---
|
|
268
|
+
|
|
269
|
+
## Design notes
|
|
270
|
+
|
|
271
|
+
- **Percentile-based candidate edges**: candidate edges follow the
|
|
272
|
+
column's own quantiles, so dense clusters get finer cuts and sparse
|
|
273
|
+
tails get wider ones.
|
|
274
|
+
- **Weighted score, not hard cutoffs**: `min_coverage` is a hard gate;
|
|
275
|
+
among what clears it, gap and coverage are blended into one score
|
|
276
|
+
rather than requiring both to clear fixed minimums independently.
|
|
277
|
+
- **Shrinkage-discounted gaps**: a dramatic gap on a handful of rows is
|
|
278
|
+
discounted toward zero before scoring, so it can't outrank a smaller,
|
|
279
|
+
better-supported gap. Raw hit rate/gap are always what's reported.
|
|
280
|
+
- **Sign-flip penalty**: if a combo's gap direction flips between train
|
|
281
|
+
and test, that's a concrete overfitting signal, so `final_score` is
|
|
282
|
+
halved for that row.
|
|
283
|
+
- **One-directional search**: `edgepoint` only asks "is there a point
|
|
284
|
+
above which the rate is *better*?" It doesn't independently verify the
|
|
285
|
+
below side against its own bar.
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## License
|
|
290
|
+
|
|
291
|
+
MIT
|