sparpartner 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- sparpartner-0.1.0/LICENSE +4 -0
- sparpartner-0.1.0/PKG-INFO +227 -0
- sparpartner-0.1.0/README.md +209 -0
- sparpartner-0.1.0/pyproject.toml +25 -0
- sparpartner-0.1.0/setup.cfg +4 -0
- sparpartner-0.1.0/setup.py +22 -0
- sparpartner-0.1.0/sparpartner/__init__.py +22 -0
- sparpartner-0.1.0/sparpartner/main.py +474 -0
- sparpartner-0.1.0/sparpartner.egg-info/PKG-INFO +227 -0
- sparpartner-0.1.0/sparpartner.egg-info/SOURCES.txt +11 -0
- sparpartner-0.1.0/sparpartner.egg-info/dependency_links.txt +1 -0
- sparpartner-0.1.0/sparpartner.egg-info/requires.txt +1 -0
- sparpartner-0.1.0/sparpartner.egg-info/top_level.txt +2 -0
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: sparpartner
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Deterministic, benchmark-driven stratified sampler for train/test prep.
|
|
5
|
+
Author: Henry
|
|
6
|
+
Author-email: Henry <osas2henry@gmail.com>
|
|
7
|
+
License: All Rights Reserved
|
|
8
|
+
Classifier: License :: Other/Proprietary License
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.8
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
License-File: LICENSE
|
|
14
|
+
Requires-Dist: pandas>=1.3
|
|
15
|
+
Dynamic: author
|
|
16
|
+
Dynamic: license-file
|
|
17
|
+
Dynamic: requires-python
|
|
18
|
+
|
|
19
|
+
# sparpartner
|
|
20
|
+
|
|
21
|
+
A deterministic stratified sampler for train/test prep. It doesn't
|
|
22
|
+
split your data randomly, it deliberately finds the rows that most
|
|
23
|
+
resemble a benchmark case, so you can hold those out as a genuine
|
|
24
|
+
test set and train on everything else.
|
|
25
|
+
|
|
26
|
+
## Why this exists
|
|
27
|
+
|
|
28
|
+
A random train/test split assumes the test set should look
|
|
29
|
+
statistically like the train set. That's the wrong question if
|
|
30
|
+
what you actually want to know is: **does the model generalize past
|
|
31
|
+
one specific profile, or did it just memorize the neighborhood
|
|
32
|
+
around it?**
|
|
33
|
+
|
|
34
|
+
`sparpartner` answers that by ranking every row in your data by how
|
|
35
|
+
closely it resembles a benchmark ("bench_marks") you define, then
|
|
36
|
+
sorting on that resemblance. You then slice the sorted frame
|
|
37
|
+
yourself:
|
|
38
|
+
|
|
39
|
+
- **Lookalikes as test, the rest as train**: train on rows *unlike*
|
|
40
|
+
the benchmark, test on the rows that *are* like it. This is the
|
|
41
|
+
harder, more honest check: it tells you whether the model actually
|
|
42
|
+
learned something general, or only performs well near cases it's
|
|
43
|
+
already seen a lot of.
|
|
44
|
+
- **Lookalikes as train, the rest as test**: the reverse, if you
|
|
45
|
+
want to check the opposite direction.
|
|
46
|
+
|
|
47
|
+
`sparpartner` only produces the ranking. The actual train/test cut
|
|
48
|
+
is a plain slice on your side (see [Usage](#usage) below).
|
|
49
|
+
|
|
50
|
+
## Where the idea comes from
|
|
51
|
+
|
|
52
|
+
Two unrelated places, both about comparing something against a
|
|
53
|
+
reference on purpose rather than at random:
|
|
54
|
+
|
|
55
|
+
**A boxer preparing for a match.** A fighter in camp doesn't spar
|
|
56
|
+
with whoever's free in the gym, they specifically look for a
|
|
57
|
+
sparring partner who moves, reaches, and hits like the opponent
|
|
58
|
+
they're about to face. Training against a random partner tells you
|
|
59
|
+
nothing about how you'll actually do; training against someone who
|
|
60
|
+
resembles the real threat does. `sparpartner` applies that same
|
|
61
|
+
logic to a model: instead of a random holdout, it finds the rows
|
|
62
|
+
that resemble the toughest, most relevant "opponent" profile and
|
|
63
|
+
holds those back as the real test, so what's left to train on is
|
|
64
|
+
everything *unlike* that opponent, and the test genuinely checks
|
|
65
|
+
whether the model can handle the match it's actually walking into.
|
|
66
|
+
|
|
67
|
+
**Astrology's approach to comparing a chart to a reference.** Reading
|
|
68
|
+
a chart against a benchmark isn't a single yes/no match, it's
|
|
69
|
+
several weighted dimensions (sun sign, moon sign, rising, houses,
|
|
70
|
+
and so on) each compared individually, then combined into one
|
|
71
|
+
overall resemblance reading. No single dimension decides the
|
|
72
|
+
outcome alone; the composite is what matters. That's the same shape
|
|
73
|
+
`sparpartner` uses: several weighted signals, each scored
|
|
74
|
+
independently against a benchmark, summed into one composite
|
|
75
|
+
`_score` rather than a single all-or-nothing match.
|
|
76
|
+
|
|
77
|
+
## How the scoring works
|
|
78
|
+
|
|
79
|
+
You give it:
|
|
80
|
+
|
|
81
|
+
- `df`: your data
|
|
82
|
+
- `bench_marks`: a dict of `{column_name: benchmark_value}`, one
|
|
83
|
+
entry per signal you care about
|
|
84
|
+
- `custom_weights`: a list of `[column_name, weight]` pairs, saying
|
|
85
|
+
which columns matter and how much
|
|
86
|
+
|
|
87
|
+
For each weighted column, `sparpartner` auto-detects the column's
|
|
88
|
+
type and scores every row's distance to the benchmark on a 0–1
|
|
89
|
+
scale (1.0 = exact match, 0.0 = as far as possible):
|
|
90
|
+
|
|
91
|
+
| Detected type | How distance is measured |
|
|
92
|
+
|---|---|
|
|
93
|
+
| **numeric** | `abs(value - bench)`, capped by the column's own max observed distance from bench |
|
|
94
|
+
| **date** | both sides converted to "age in days" relative to the benchmark date, capped by the column's own max observed age |
|
|
95
|
+
| **string** | exact match = 1, anything else = 0 |
|
|
96
|
+
|
|
97
|
+
Date detection is automatic, it only treats a column as a date if
|
|
98
|
+
its values look date-shaped (contain a separator like `-`, `/`, `.`
|
|
99
|
+
or a recognizable month name) **and** parse successfully at least
|
|
100
|
+
98% of the time. Bare numeric-looking strings (e.g. `"12345"`) are
|
|
101
|
+
never mistaken for dates, and object-dtype columns of digit strings
|
|
102
|
+
are treated as exact-match strings, not numbers. Only a real
|
|
103
|
+
numeric dtype gets the numeric path.
|
|
104
|
+
|
|
105
|
+
Each column's 0–1 score is multiplied by its weight and summed into
|
|
106
|
+
one raw score per row, then divided by the total weight so the
|
|
107
|
+
final `_score` always lands in the 0–1 range, however many signals
|
|
108
|
+
or weights you used.
|
|
109
|
+
|
|
110
|
+
### Tie-breaking
|
|
111
|
+
|
|
112
|
+
Rows that land on the exact same `_score` aren't left to random or
|
|
113
|
+
arbitrary order. Ties are broken by the per-signal score of the
|
|
114
|
+
**highest-weight** signal first, then the next-highest, cascading
|
|
115
|
+
down the weighted signal list until the tie resolves. Signals that
|
|
116
|
+
share the same weight are compared in the order they appear in
|
|
117
|
+
`custom_weights`. The tie-break always sorts in the same direction
|
|
118
|
+
as the main score (see `look_alike` below). Only if every signal is
|
|
119
|
+
exhausted and rows are still tied does it fall back to original row
|
|
120
|
+
order.
|
|
121
|
+
|
|
122
|
+
## `look_alike`
|
|
123
|
+
|
|
124
|
+
Controls sort direction:
|
|
125
|
+
|
|
126
|
+
- `look_alike=True` (default): highest `_score` (closest to
|
|
127
|
+
benchmark) sorted to the **top**.
|
|
128
|
+
- `look_alike=False`: highest `_score` sorted to the **bottom**,
|
|
129
|
+
farthest-from-benchmark rows come first.
|
|
130
|
+
|
|
131
|
+
## Usage
|
|
132
|
+
|
|
133
|
+
### Sample usage
|
|
134
|
+
|
|
135
|
+
```python
|
|
136
|
+
import pandas as pd
|
|
137
|
+
from sparpartner import sample
|
|
138
|
+
|
|
139
|
+
df = pd.DataFrame({
|
|
140
|
+
"id": [1, 2, 3, 4, 5],
|
|
141
|
+
"age": [25, 30, 47, 52, 33],
|
|
142
|
+
"signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
|
|
143
|
+
"country": ["US", "US", "CA", "US", "MX"],
|
|
144
|
+
})
|
|
145
|
+
|
|
146
|
+
bench_marks = {
|
|
147
|
+
"age": 30,
|
|
148
|
+
"signup_date": "2023-01-01",
|
|
149
|
+
"country": "US",
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
custom_weights = [
|
|
153
|
+
["age", 2],
|
|
154
|
+
["signup_date", 1],
|
|
155
|
+
["country", 1],
|
|
156
|
+
]
|
|
157
|
+
|
|
158
|
+
result = sample(
|
|
159
|
+
df,
|
|
160
|
+
bench_marks,
|
|
161
|
+
custom_weights,
|
|
162
|
+
look_alike=True, # True = most similar rows first
|
|
163
|
+
show_progress=True, # prints the full scoring breakdown
|
|
164
|
+
return_score=True, # keep _score columns in the output
|
|
165
|
+
drop_nan=True,
|
|
166
|
+
)
|
|
167
|
+
|
|
168
|
+
print(result)
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
`age=30`, `signup_date="2023-01-01"`, and `country="US"` closely
|
|
172
|
+
match row `id=1` (age 25, close date, US), so with
|
|
173
|
+
`look_alike=True` that row lands at or near the top of the sorted
|
|
174
|
+
output. Flip `look_alike=False` to instead surface the rows that
|
|
175
|
+
look *least* like the benchmark (e.g. the Mexico row with the
|
|
176
|
+
furthest signup date).
|
|
177
|
+
|
|
178
|
+
```python
|
|
179
|
+
# --- post-sample: turn the ranking into an actual train/test split ---
|
|
180
|
+
# Re-run with look_alike=False so lookalikes sort to the BOTTOM of `result`.
|
|
181
|
+
ranked = sample(df, bench_marks, custom_weights, look_alike=False)
|
|
182
|
+
|
|
183
|
+
# Slice however large you want the test set to be, e.g. the bottom 30%:
|
|
184
|
+
cut = int(len(ranked) * 0.7)
|
|
185
|
+
train = ranked.iloc[:cut] # non-lookalikes
|
|
186
|
+
test = ranked.iloc[cut:] # lookalikes, the harder, honest test set
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### Parameters
|
|
190
|
+
|
|
191
|
+
| Name | Type | Default | What it does |
|
|
192
|
+
|---|---|---|---|
|
|
193
|
+
| `df` | DataFrame | required | Must contain a column for every name in `custom_weights` |
|
|
194
|
+
| `bench_marks` | dict | required | `{column_name: benchmark_value}` |
|
|
195
|
+
| `custom_weights` | list of `[name, weight]` | required | Which columns to score and how much each contributes |
|
|
196
|
+
| `look_alike` | bool | `True` | `True` = lookalikes sorted to top; `False` = sorted to bottom |
|
|
197
|
+
| `show_progress` | bool | `False` | Prints a full readout: header, per-column type/cap/sample scores, sort direction, top/bottom ranked rows with weighted contributions, and a health check flagging any signal that isn't adding useful separation |
|
|
198
|
+
| `return_score` | bool | `False` | If `True`, keeps `_score` and `_score_<name>` columns in the result instead of dropping them |
|
|
199
|
+
| `drop_nan` | bool | `True` | If `True`, drops any row with a NaN component score (e.g. missing value or unparseable date) after printing a sanity check of what was dropped and why |
|
|
200
|
+
|
|
201
|
+
### Validation
|
|
202
|
+
|
|
203
|
+
`custom_weights` is validated upfront, before any scoring starts, and
|
|
204
|
+
raises `ValueError` if:
|
|
205
|
+
|
|
206
|
+
- a column name isn't a string, or doesn't exist in `df`
|
|
207
|
+
- a column name has no matching entry in `bench_marks`
|
|
208
|
+
- the same column name appears more than once
|
|
209
|
+
- a weight isn't numeric (bools are rejected too, a `bool` is
|
|
210
|
+
technically an `int` in Python but was never meant as a weight)
|
|
211
|
+
- a weight is `NaN`
|
|
212
|
+
|
|
213
|
+
## A couple of things worth knowing
|
|
214
|
+
|
|
215
|
+
- **Zero-variance columns**: if every row (including the benchmark)
|
|
216
|
+
has the same value for a signal, there's nothing to measure
|
|
217
|
+
distance against, that signal scores every row 1.0 rather than
|
|
218
|
+
dividing by zero.
|
|
219
|
+
- **Negative total weight**: if your weights sum to ≤ 0, the
|
|
220
|
+
normalization step is skipped and `_score` is left as the raw
|
|
221
|
+
weighted sum instead of being silently divided by a non-positive
|
|
222
|
+
number.
|
|
223
|
+
- **Object-dtype numeric strings**: a column of strings like
|
|
224
|
+
`"100"`, `"200"` (object dtype, no separator) is scored as an
|
|
225
|
+
exact-match string column, *not* auto-converted to numeric. Only
|
|
226
|
+
genuine numeric dtypes (`int`, `float`) get the numeric distance
|
|
227
|
+
path.
|
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
# sparpartner
|
|
2
|
+
|
|
3
|
+
A deterministic stratified sampler for train/test prep. It doesn't
|
|
4
|
+
split your data randomly, it deliberately finds the rows that most
|
|
5
|
+
resemble a benchmark case, so you can hold those out as a genuine
|
|
6
|
+
test set and train on everything else.
|
|
7
|
+
|
|
8
|
+
## Why this exists
|
|
9
|
+
|
|
10
|
+
A random train/test split assumes the test set should look
|
|
11
|
+
statistically like the train set. That's the wrong question if
|
|
12
|
+
what you actually want to know is: **does the model generalize past
|
|
13
|
+
one specific profile, or did it just memorize the neighborhood
|
|
14
|
+
around it?**
|
|
15
|
+
|
|
16
|
+
`sparpartner` answers that by ranking every row in your data by how
|
|
17
|
+
closely it resembles a benchmark ("bench_marks") you define, then
|
|
18
|
+
sorting on that resemblance. You then slice the sorted frame
|
|
19
|
+
yourself:
|
|
20
|
+
|
|
21
|
+
- **Lookalikes as test, the rest as train**: train on rows *unlike*
|
|
22
|
+
the benchmark, test on the rows that *are* like it. This is the
|
|
23
|
+
harder, more honest check: it tells you whether the model actually
|
|
24
|
+
learned something general, or only performs well near cases it's
|
|
25
|
+
already seen a lot of.
|
|
26
|
+
- **Lookalikes as train, the rest as test**: the reverse, if you
|
|
27
|
+
want to check the opposite direction.
|
|
28
|
+
|
|
29
|
+
`sparpartner` only produces the ranking. The actual train/test cut
|
|
30
|
+
is a plain slice on your side (see [Usage](#usage) below).
|
|
31
|
+
|
|
32
|
+
## Where the idea comes from
|
|
33
|
+
|
|
34
|
+
Two unrelated places, both about comparing something against a
|
|
35
|
+
reference on purpose rather than at random:
|
|
36
|
+
|
|
37
|
+
**A boxer preparing for a match.** A fighter in camp doesn't spar
|
|
38
|
+
with whoever's free in the gym, they specifically look for a
|
|
39
|
+
sparring partner who moves, reaches, and hits like the opponent
|
|
40
|
+
they're about to face. Training against a random partner tells you
|
|
41
|
+
nothing about how you'll actually do; training against someone who
|
|
42
|
+
resembles the real threat does. `sparpartner` applies that same
|
|
43
|
+
logic to a model: instead of a random holdout, it finds the rows
|
|
44
|
+
that resemble the toughest, most relevant "opponent" profile and
|
|
45
|
+
holds those back as the real test, so what's left to train on is
|
|
46
|
+
everything *unlike* that opponent, and the test genuinely checks
|
|
47
|
+
whether the model can handle the match it's actually walking into.
|
|
48
|
+
|
|
49
|
+
**Astrology's approach to comparing a chart to a reference.** Reading
|
|
50
|
+
a chart against a benchmark isn't a single yes/no match, it's
|
|
51
|
+
several weighted dimensions (sun sign, moon sign, rising, houses,
|
|
52
|
+
and so on) each compared individually, then combined into one
|
|
53
|
+
overall resemblance reading. No single dimension decides the
|
|
54
|
+
outcome alone; the composite is what matters. That's the same shape
|
|
55
|
+
`sparpartner` uses: several weighted signals, each scored
|
|
56
|
+
independently against a benchmark, summed into one composite
|
|
57
|
+
`_score` rather than a single all-or-nothing match.
|
|
58
|
+
|
|
59
|
+
## How the scoring works
|
|
60
|
+
|
|
61
|
+
You give it:
|
|
62
|
+
|
|
63
|
+
- `df`: your data
|
|
64
|
+
- `bench_marks`: a dict of `{column_name: benchmark_value}`, one
|
|
65
|
+
entry per signal you care about
|
|
66
|
+
- `custom_weights`: a list of `[column_name, weight]` pairs, saying
|
|
67
|
+
which columns matter and how much
|
|
68
|
+
|
|
69
|
+
For each weighted column, `sparpartner` auto-detects the column's
|
|
70
|
+
type and scores every row's distance to the benchmark on a 0–1
|
|
71
|
+
scale (1.0 = exact match, 0.0 = as far as possible):
|
|
72
|
+
|
|
73
|
+
| Detected type | How distance is measured |
|
|
74
|
+
|---|---|
|
|
75
|
+
| **numeric** | `abs(value - bench)`, capped by the column's own max observed distance from bench |
|
|
76
|
+
| **date** | both sides converted to "age in days" relative to the benchmark date, capped by the column's own max observed age |
|
|
77
|
+
| **string** | exact match = 1, anything else = 0 |
|
|
78
|
+
|
|
79
|
+
Date detection is automatic, it only treats a column as a date if
|
|
80
|
+
its values look date-shaped (contain a separator like `-`, `/`, `.`
|
|
81
|
+
or a recognizable month name) **and** parse successfully at least
|
|
82
|
+
98% of the time. Bare numeric-looking strings (e.g. `"12345"`) are
|
|
83
|
+
never mistaken for dates, and object-dtype columns of digit strings
|
|
84
|
+
are treated as exact-match strings, not numbers. Only a real
|
|
85
|
+
numeric dtype gets the numeric path.
|
|
86
|
+
|
|
87
|
+
Each column's 0–1 score is multiplied by its weight and summed into
|
|
88
|
+
one raw score per row, then divided by the total weight so the
|
|
89
|
+
final `_score` always lands in the 0–1 range, however many signals
|
|
90
|
+
or weights you used.
|
|
91
|
+
|
|
92
|
+
### Tie-breaking
|
|
93
|
+
|
|
94
|
+
Rows that land on the exact same `_score` aren't left to random or
|
|
95
|
+
arbitrary order. Ties are broken by the per-signal score of the
|
|
96
|
+
**highest-weight** signal first, then the next-highest, cascading
|
|
97
|
+
down the weighted signal list until the tie resolves. Signals that
|
|
98
|
+
share the same weight are compared in the order they appear in
|
|
99
|
+
`custom_weights`. The tie-break always sorts in the same direction
|
|
100
|
+
as the main score (see `look_alike` below). Only if every signal is
|
|
101
|
+
exhausted and rows are still tied does it fall back to original row
|
|
102
|
+
order.
|
|
103
|
+
|
|
104
|
+
## `look_alike`
|
|
105
|
+
|
|
106
|
+
Controls sort direction:
|
|
107
|
+
|
|
108
|
+
- `look_alike=True` (default): highest `_score` (closest to
|
|
109
|
+
benchmark) sorted to the **top**.
|
|
110
|
+
- `look_alike=False`: highest `_score` sorted to the **bottom**,
|
|
111
|
+
farthest-from-benchmark rows come first.
|
|
112
|
+
|
|
113
|
+
## Usage
|
|
114
|
+
|
|
115
|
+
### Sample usage
|
|
116
|
+
|
|
117
|
+
```python
|
|
118
|
+
import pandas as pd
|
|
119
|
+
from sparpartner import sample
|
|
120
|
+
|
|
121
|
+
df = pd.DataFrame({
|
|
122
|
+
"id": [1, 2, 3, 4, 5],
|
|
123
|
+
"age": [25, 30, 47, 52, 33],
|
|
124
|
+
"signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
|
|
125
|
+
"country": ["US", "US", "CA", "US", "MX"],
|
|
126
|
+
})
|
|
127
|
+
|
|
128
|
+
bench_marks = {
|
|
129
|
+
"age": 30,
|
|
130
|
+
"signup_date": "2023-01-01",
|
|
131
|
+
"country": "US",
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
custom_weights = [
|
|
135
|
+
["age", 2],
|
|
136
|
+
["signup_date", 1],
|
|
137
|
+
["country", 1],
|
|
138
|
+
]
|
|
139
|
+
|
|
140
|
+
result = sample(
|
|
141
|
+
df,
|
|
142
|
+
bench_marks,
|
|
143
|
+
custom_weights,
|
|
144
|
+
look_alike=True, # True = most similar rows first
|
|
145
|
+
show_progress=True, # prints the full scoring breakdown
|
|
146
|
+
return_score=True, # keep _score columns in the output
|
|
147
|
+
drop_nan=True,
|
|
148
|
+
)
|
|
149
|
+
|
|
150
|
+
print(result)
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
`age=30`, `signup_date="2023-01-01"`, and `country="US"` closely
|
|
154
|
+
match row `id=1` (age 25, close date, US), so with
|
|
155
|
+
`look_alike=True` that row lands at or near the top of the sorted
|
|
156
|
+
output. Flip `look_alike=False` to instead surface the rows that
|
|
157
|
+
look *least* like the benchmark (e.g. the Mexico row with the
|
|
158
|
+
furthest signup date).
|
|
159
|
+
|
|
160
|
+
```python
|
|
161
|
+
# --- post-sample: turn the ranking into an actual train/test split ---
|
|
162
|
+
# Re-run with look_alike=False so lookalikes sort to the BOTTOM of `result`.
|
|
163
|
+
ranked = sample(df, bench_marks, custom_weights, look_alike=False)
|
|
164
|
+
|
|
165
|
+
# Slice however large you want the test set to be, e.g. the bottom 30%:
|
|
166
|
+
cut = int(len(ranked) * 0.7)
|
|
167
|
+
train = ranked.iloc[:cut] # non-lookalikes
|
|
168
|
+
test = ranked.iloc[cut:] # lookalikes, the harder, honest test set
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
### Parameters
|
|
172
|
+
|
|
173
|
+
| Name | Type | Default | What it does |
|
|
174
|
+
|---|---|---|---|
|
|
175
|
+
| `df` | DataFrame | required | Must contain a column for every name in `custom_weights` |
|
|
176
|
+
| `bench_marks` | dict | required | `{column_name: benchmark_value}` |
|
|
177
|
+
| `custom_weights` | list of `[name, weight]` | required | Which columns to score and how much each contributes |
|
|
178
|
+
| `look_alike` | bool | `True` | `True` = lookalikes sorted to top; `False` = sorted to bottom |
|
|
179
|
+
| `show_progress` | bool | `False` | Prints a full readout: header, per-column type/cap/sample scores, sort direction, top/bottom ranked rows with weighted contributions, and a health check flagging any signal that isn't adding useful separation |
|
|
180
|
+
| `return_score` | bool | `False` | If `True`, keeps `_score` and `_score_<name>` columns in the result instead of dropping them |
|
|
181
|
+
| `drop_nan` | bool | `True` | If `True`, drops any row with a NaN component score (e.g. missing value or unparseable date) after printing a sanity check of what was dropped and why |
|
|
182
|
+
|
|
183
|
+
### Validation
|
|
184
|
+
|
|
185
|
+
`custom_weights` is validated upfront, before any scoring starts, and
|
|
186
|
+
raises `ValueError` if:
|
|
187
|
+
|
|
188
|
+
- a column name isn't a string, or doesn't exist in `df`
|
|
189
|
+
- a column name has no matching entry in `bench_marks`
|
|
190
|
+
- the same column name appears more than once
|
|
191
|
+
- a weight isn't numeric (bools are rejected too, a `bool` is
|
|
192
|
+
technically an `int` in Python but was never meant as a weight)
|
|
193
|
+
- a weight is `NaN`
|
|
194
|
+
|
|
195
|
+
## A couple of things worth knowing
|
|
196
|
+
|
|
197
|
+
- **Zero-variance columns**: if every row (including the benchmark)
|
|
198
|
+
has the same value for a signal, there's nothing to measure
|
|
199
|
+
distance against, that signal scores every row 1.0 rather than
|
|
200
|
+
dividing by zero.
|
|
201
|
+
- **Negative total weight**: if your weights sum to ≤ 0, the
|
|
202
|
+
normalization step is skipped and `_score` is left as the raw
|
|
203
|
+
weighted sum instead of being silently divided by a non-positive
|
|
204
|
+
number.
|
|
205
|
+
- **Object-dtype numeric strings**: a column of strings like
|
|
206
|
+
`"100"`, `"200"` (object dtype, no separator) is scored as an
|
|
207
|
+
exact-match string column, *not* auto-converted to numeric. Only
|
|
208
|
+
genuine numeric dtypes (`int`, `float`) get the numeric distance
|
|
209
|
+
path.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=61.0"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "sparpartner"
|
|
7
|
+
version = "0.1.0"
|
|
8
|
+
description = "Deterministic, benchmark-driven stratified sampler for train/test prep."
|
|
9
|
+
readme = "README.md"
|
|
10
|
+
requires-python = ">=3.8"
|
|
11
|
+
license = { text = "All Rights Reserved" }
|
|
12
|
+
authors = [
|
|
13
|
+
{ name = "Henry", email = "osas2henry@gmail.com" }
|
|
14
|
+
]
|
|
15
|
+
dependencies = [
|
|
16
|
+
"pandas>=1.3"
|
|
17
|
+
]
|
|
18
|
+
classifiers = [
|
|
19
|
+
"License :: Other/Proprietary License",
|
|
20
|
+
"Programming Language :: Python :: 3",
|
|
21
|
+
"Operating System :: OS Independent",
|
|
22
|
+
]
|
|
23
|
+
|
|
24
|
+
[tool.setuptools.packages.find]
|
|
25
|
+
where = ["."]
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
from setuptools import setup, find_packages
|
|
2
|
+
|
|
3
|
+
setup(
|
|
4
|
+
name="sparpartner",
|
|
5
|
+
version="0.1.0",
|
|
6
|
+
description="Deterministic, benchmark-driven stratified sampler for train/test prep.",
|
|
7
|
+
long_description=open("README.md", encoding="utf-8").read(),
|
|
8
|
+
long_description_content_type="text/markdown",
|
|
9
|
+
author="Henry",
|
|
10
|
+
author_email="osas2henry@gmail.com",
|
|
11
|
+
license="Proprietary",
|
|
12
|
+
packages=find_packages(),
|
|
13
|
+
install_requires=[
|
|
14
|
+
"pandas>=1.3",
|
|
15
|
+
],
|
|
16
|
+
python_requires=">=3.8",
|
|
17
|
+
classifiers=[
|
|
18
|
+
"License :: Other/Proprietary License",
|
|
19
|
+
"Programming Language :: Python :: 3",
|
|
20
|
+
"Operating System :: OS Independent",
|
|
21
|
+
],
|
|
22
|
+
)
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
"""
|
|
2
|
+
sparpartner
|
|
3
|
+
============
|
|
4
|
+
|
|
5
|
+
Type-aware benchmark scoring / stratified sampler for train-test prep.
|
|
6
|
+
|
|
7
|
+
Guards against overfitting by letting you deliberately hold out the
|
|
8
|
+
rows that most resemble a benchmark case (rather than a random split),
|
|
9
|
+
so a model trained on the rest can be tested against its true
|
|
10
|
+
"opponent" instead of an easy, self-similar sample.
|
|
11
|
+
|
|
12
|
+
Usage
|
|
13
|
+
-----
|
|
14
|
+
from sparpartner import sample
|
|
15
|
+
|
|
16
|
+
result = sample(df, bench_marks, custom_weights, look_alike=True)
|
|
17
|
+
"""
|
|
18
|
+
|
|
19
|
+
from .main import sample
|
|
20
|
+
|
|
21
|
+
__all__ = ["sample"]
|
|
22
|
+
__version__ = "0.1.0"
|
|
@@ -0,0 +1,474 @@
|
|
|
1
|
+
import re
|
|
2
|
+
import warnings
|
|
3
|
+
import pandas as pd
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
# Values must look date-shaped (contain a date separator or month name)
|
|
7
|
+
# before we even attempt pd.to_datetime — this stops bare digit strings
|
|
8
|
+
# like "2023", "12345" (zip codes), or plain IDs from being mistaken
|
|
9
|
+
# for dates just because to_datetime happens to be able to parse them.
|
|
10
|
+
_MONTH_NAMES = (
|
|
11
|
+
r"jan(?:uary)?|feb(?:ruary)?|mar(?:ch)?|apr(?:il)?|may|jun(?:e)?|"
|
|
12
|
+
r"jul(?:y)?|aug(?:ust)?|sep(?:t(?:ember)?)?|oct(?:ober)?|"
|
|
13
|
+
r"nov(?:ember)?|dec(?:ember)?"
|
|
14
|
+
)
|
|
15
|
+
_DATE_LIKE_PATTERN = re.compile(
|
|
16
|
+
rf"[-/.]|\b(?:{_MONTH_NAMES})\b", re.IGNORECASE
|
|
17
|
+
)
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
def _looks_date_shaped(series, min_fraction=0.9):
|
|
21
|
+
"""
|
|
22
|
+
True only if at least `min_fraction` of the column's non-null
|
|
23
|
+
values contain a date separator (-, /, .) or a recognizable month
|
|
24
|
+
name/abbreviation. Bare digit strings ("2023", "12345", plain IDs)
|
|
25
|
+
don't match this and are never even sent to pd.to_datetime, so
|
|
26
|
+
they can't be misdetected as dates just because to_datetime is
|
|
27
|
+
lenient enough to parse them.
|
|
28
|
+
"""
|
|
29
|
+
non_null = series.dropna().astype(str)
|
|
30
|
+
if len(non_null) == 0:
|
|
31
|
+
return False
|
|
32
|
+
return non_null.str.contains(_DATE_LIKE_PATTERN).mean() >= min_fraction
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
# =============================================================================
|
|
36
|
+
# INPUT VALIDATION
|
|
37
|
+
# =============================================================================
|
|
38
|
+
|
|
39
|
+
def _validate_custom_weights(df, bench_marks, custom_weights):
|
|
40
|
+
"""
|
|
41
|
+
Validates custom_weights, a list of [name, weight] pairs, against
|
|
42
|
+
df and bench_marks — run once, upfront, before any scoring loop
|
|
43
|
+
starts, so bad input fails fast instead of partway through.
|
|
44
|
+
|
|
45
|
+
Raises ValueError if:
|
|
46
|
+
- a name (index 0 of the pair) is not a string
|
|
47
|
+
- a name doesn't match a column in df
|
|
48
|
+
- a name has no matching entry in bench_marks
|
|
49
|
+
- the same name appears more than once (duplicate signal —
|
|
50
|
+
ambiguous which weight/bench should apply, and it would
|
|
51
|
+
silently double-count that column's contribution to _score)
|
|
52
|
+
- a weight (index 1 of the pair) is not numeric — bools are
|
|
53
|
+
rejected too (bool is technically an int subclass in Python
|
|
54
|
+
but was never intended as a weight), and NaN floats are
|
|
55
|
+
rejected since they'd silently poison every row's _score
|
|
56
|
+
"""
|
|
57
|
+
seen = set()
|
|
58
|
+
duplicates = set()
|
|
59
|
+
for pair in custom_weights:
|
|
60
|
+
name, weight = pair[0], pair[1]
|
|
61
|
+
|
|
62
|
+
if not isinstance(name, str):
|
|
63
|
+
raise ValueError(
|
|
64
|
+
f"sparpartner: signal name must be a string, got {type(name).__name__}: {name!r}"
|
|
65
|
+
)
|
|
66
|
+
|
|
67
|
+
if name not in df.columns:
|
|
68
|
+
raise ValueError(f"sparpartner: column {name!r} not found in df")
|
|
69
|
+
|
|
70
|
+
if name not in bench_marks:
|
|
71
|
+
raise ValueError(f"sparpartner: no bench_marks entry for {name!r}")
|
|
72
|
+
|
|
73
|
+
if name in seen:
|
|
74
|
+
duplicates.add(name)
|
|
75
|
+
seen.add(name)
|
|
76
|
+
|
|
77
|
+
is_numeric = isinstance(weight, (int, float)) and not isinstance(weight, bool)
|
|
78
|
+
if not is_numeric:
|
|
79
|
+
raise ValueError(
|
|
80
|
+
f"sparpartner: weight for {name!r} must be numeric (int or float), "
|
|
81
|
+
f"got {type(weight).__name__}: {weight!r}"
|
|
82
|
+
)
|
|
83
|
+
if isinstance(weight, float) and pd.isna(weight):
|
|
84
|
+
raise ValueError(f"sparpartner: weight for {name!r} is NaN — must be a real number")
|
|
85
|
+
|
|
86
|
+
if duplicates:
|
|
87
|
+
raise ValueError(
|
|
88
|
+
f"sparpartner: duplicate signal name(s) in custom_weights: {sorted(duplicates)}"
|
|
89
|
+
)
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
# =============================================================================
|
|
93
|
+
# TYPE DETECTION + NUMERIC CONVERSION
|
|
94
|
+
# =============================================================================
|
|
95
|
+
|
|
96
|
+
def _to_numeric_series(series, bench_value):
|
|
97
|
+
"""
|
|
98
|
+
Given a column and its benchmark value, returns (col_numeric,
|
|
99
|
+
bench_numeric, kind) — both values reduced to plain numbers so the
|
|
100
|
+
same distance formula works regardless of dtype, plus `kind` so
|
|
101
|
+
the caller knows exactly which path was taken (needed to pick the
|
|
102
|
+
right cap — re-guessing dtype after this point is unreliable for
|
|
103
|
+
date-strings stored as object dtype).
|
|
104
|
+
|
|
105
|
+
- numeric dtype -> used as-is; bench_numeric = float(bench_value)
|
|
106
|
+
kind = "numeric"
|
|
107
|
+
- date/datetime -> both column and bench converted to "age in
|
|
108
|
+
days" relative to the benchmark date, so the
|
|
109
|
+
benchmark itself always lands on 0. col dtype
|
|
110
|
+
is auto-detected either from actual dtype, or
|
|
111
|
+
by first requiring the column's string values
|
|
112
|
+
look date-shaped (contain a separator like
|
|
113
|
+
-/. or a month name — bare digit strings like
|
|
114
|
+
"2023" or a zip code never qualify) and then
|
|
115
|
+
successfully parsing with pd.to_datetime for
|
|
116
|
+
essentially every value (>=98%).
|
|
117
|
+
kind = "date"
|
|
118
|
+
- string/object -> 1 if the cell equals bench_value exactly,
|
|
119
|
+
else 0; bench_numeric = 1 (a perfect match
|
|
120
|
+
scores as touching the benchmark exactly).
|
|
121
|
+
kind = "string"
|
|
122
|
+
"""
|
|
123
|
+
# --- date/datetime path ---
|
|
124
|
+
is_datetime_dtype = pd.api.types.is_datetime64_any_dtype(series)
|
|
125
|
+
parsed_as_date = None
|
|
126
|
+
if not is_datetime_dtype and not pd.api.types.is_numeric_dtype(series):
|
|
127
|
+
if _looks_date_shaped(series):
|
|
128
|
+
with warnings.catch_warnings():
|
|
129
|
+
warnings.simplefilter("ignore", UserWarning)
|
|
130
|
+
parsed_as_date = pd.to_datetime(series, errors="coerce")
|
|
131
|
+
# only treat as a date column if parsing worked for
|
|
132
|
+
# essentially every value — otherwise this is a real
|
|
133
|
+
# string column that happened to partially parse
|
|
134
|
+
if parsed_as_date.notna().mean() < 0.98:
|
|
135
|
+
parsed_as_date = None
|
|
136
|
+
|
|
137
|
+
if is_datetime_dtype or parsed_as_date is not None:
|
|
138
|
+
col_dt = series if is_datetime_dtype else parsed_as_date
|
|
139
|
+
col_dt = pd.to_datetime(col_dt, errors="coerce")
|
|
140
|
+
bench_dt = pd.to_datetime(bench_value)
|
|
141
|
+
|
|
142
|
+
col_numeric = (col_dt - bench_dt).dt.days.abs().astype(float)
|
|
143
|
+
bench_numeric = 0.0
|
|
144
|
+
return col_numeric, bench_numeric, "date"
|
|
145
|
+
|
|
146
|
+
# --- numeric path ---
|
|
147
|
+
if pd.api.types.is_numeric_dtype(series):
|
|
148
|
+
return series.astype(float), float(bench_value), "numeric"
|
|
149
|
+
|
|
150
|
+
# --- string/exact-match path ---
|
|
151
|
+
col_numeric = (series == bench_value).astype(float)
|
|
152
|
+
bench_numeric = 1.0
|
|
153
|
+
return col_numeric, bench_numeric, "string"
|
|
154
|
+
|
|
155
|
+
|
|
156
|
+
# =============================================================================
|
|
157
|
+
# PROGRESS DISPLAY (pure print — no logic, no return values)
|
|
158
|
+
# =============================================================================
|
|
159
|
+
|
|
160
|
+
def _progress_header(bench_marks, custom_weights, look_alike, row_count):
|
|
161
|
+
total_weight = sum(w for _, w in custom_weights)
|
|
162
|
+
title = "SPARPARTNER, type-aware benchmark scoring"
|
|
163
|
+
print(f"\n{title}")
|
|
164
|
+
print("-" * len(title))
|
|
165
|
+
look_alike_note = ("highest score sorted UP, lookalikes on top" if look_alike
|
|
166
|
+
else "highest score sorted DOWN, lookalikes pushed to bottom")
|
|
167
|
+
print(f" {'rows in pool':<16} = {row_count}")
|
|
168
|
+
print(f" {'look_alike':<16} = {look_alike} ({look_alike_note})")
|
|
169
|
+
|
|
170
|
+
signals_title = "SIGNALS"
|
|
171
|
+
print(f"\n {signals_title}")
|
|
172
|
+
print(f" {'-' * len(signals_title)}")
|
|
173
|
+
for name, weight in custom_weights:
|
|
174
|
+
bench = bench_marks.get(name, "<!! MISSING !!>")
|
|
175
|
+
print(f" {name:<20} weight={weight:<6} bench={bench!r}")
|
|
176
|
+
print(f"\n {'total weight':<16} = {total_weight}")
|
|
177
|
+
|
|
178
|
+
|
|
179
|
+
def _progress_column(name, weight, kind, bench_value, cap, distance, score):
|
|
180
|
+
title = f"[{name}] detected type = {kind}"
|
|
181
|
+
print(f"\n {title}")
|
|
182
|
+
print(f" {'-' * len(title)}")
|
|
183
|
+
print(f" {'bench':<20} = {bench_value!r}")
|
|
184
|
+
print(f" {'weight':<20} = {weight}")
|
|
185
|
+
print(f" {'cap':<20} = {cap}"
|
|
186
|
+
+ (" [!] cap <= 0, every row will score 1.0 for this signal (no variance to score against)"
|
|
187
|
+
if cap is not None and not pd.isna(cap) and cap <= 0 else ""))
|
|
188
|
+
n_perfect = int((distance == 0).sum())
|
|
189
|
+
print(f" {'exact matches':<20} = {n_perfect}/{len(distance)} rows")
|
|
190
|
+
print(f" {'sample distances':<20} : {distance.head(3).round(4).tolist()}")
|
|
191
|
+
print(f" {'sample scores':<20} : {score.head(3).round(4).tolist()}")
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
def _progress_sort_apply(df_before, look_alike, sort_cols, n=5):
|
|
195
|
+
ascending = not look_alike
|
|
196
|
+
direction = ("descending, closest-to-benchmark scores first"
|
|
197
|
+
if look_alike else
|
|
198
|
+
"ascending, farthest-from-benchmark scores first")
|
|
199
|
+
title = f"APPLYING look_alike SORT (look_alike={look_alike})"
|
|
200
|
+
print(f"\n {title}")
|
|
201
|
+
print(f" {'-' * len(title)}")
|
|
202
|
+
print(f" {'sort call':<20} : df.sort_values(_score + {len(sort_cols) - 1} tie-break signal(s), ascending={ascending})")
|
|
203
|
+
print(f" {'direction':<20} : {direction}")
|
|
204
|
+
tie_break_names = sort_cols[1:]
|
|
205
|
+
if not tie_break_names:
|
|
206
|
+
tie_break_display = "(none, single signal)"
|
|
207
|
+
else:
|
|
208
|
+
shown = tie_break_names[:3]
|
|
209
|
+
remaining = len(tie_break_names) - len(shown)
|
|
210
|
+
tie_break_display = ", ".join(shown) + (f", ... (+{remaining} more)" if remaining > 0 else "")
|
|
211
|
+
print(f" {'tie-break order':<20} : {tie_break_display}")
|
|
212
|
+
before = df_before["_score"].round(4).tolist()
|
|
213
|
+
print(f" {'before sort':<20} : {before[:n]}")
|
|
214
|
+
|
|
215
|
+
|
|
216
|
+
def _progress_sort_glimpse(df, look_alike, n=5):
|
|
217
|
+
scores = df["_score"].round(4).tolist()
|
|
218
|
+
head = scores[:n]
|
|
219
|
+
tail = scores[-n:]
|
|
220
|
+
print(f" {'after sort (head)':<20} : {head}")
|
|
221
|
+
print(f" {'after sort (tail)':<20} : {tail}")
|
|
222
|
+
|
|
223
|
+
|
|
224
|
+
def _progress_top_rows(df, custom_weights, look_alike, n=5):
|
|
225
|
+
label = "TOP" if look_alike else "BOTTOM (farthest from benchmark)"
|
|
226
|
+
title = f"{label} {n} ROWS AFTER SCORING"
|
|
227
|
+
print(f"\n {title}")
|
|
228
|
+
print(f" {'-' * len(title)}")
|
|
229
|
+
display_cols = [c for c in df.columns if not c.startswith("_score")]
|
|
230
|
+
id_col = "id" if "id" in df.columns else display_cols[0]
|
|
231
|
+
for i in range(min(n, len(df))):
|
|
232
|
+
row = df.iloc[i]
|
|
233
|
+
print(f"\n #{i+1} [{row.get(id_col, '?')}] total _score = {row['_score']:.4f}")
|
|
234
|
+
for name, weight in custom_weights:
|
|
235
|
+
comp_key = f"_score_{name}"
|
|
236
|
+
if comp_key in df.columns:
|
|
237
|
+
comp_score = row[comp_key]
|
|
238
|
+
contribution = comp_score * weight
|
|
239
|
+
print(f" {name:<20} score={comp_score:.4f} x weight={weight:<6} = {contribution:.4f}")
|
|
240
|
+
|
|
241
|
+
|
|
242
|
+
def _progress_health_check(df, custom_weights, bench_marks):
|
|
243
|
+
title = "HEALTH CHECK"
|
|
244
|
+
print(f"\n {title}")
|
|
245
|
+
print(f" {'-' * len(title)}")
|
|
246
|
+
for name, weight in custom_weights:
|
|
247
|
+
comp_key = f"_score_{name}"
|
|
248
|
+
if comp_key not in df.columns:
|
|
249
|
+
continue
|
|
250
|
+
avg_score = df[comp_key].mean()
|
|
251
|
+
quality = "high quality" if avg_score > 0.7 else "moderate" if avg_score > 0.4 else "low quality"
|
|
252
|
+
print(f" {name:<20} avg score={avg_score:.4f} ({quality})")
|
|
253
|
+
if avg_score == 0.0:
|
|
254
|
+
print(f" [!] ZERO rows are close to bench={bench_marks[name]!r} for {name!r}, "
|
|
255
|
+
f"this signal is contributing nothing useful to ranking.")
|
|
256
|
+
|
|
257
|
+
dist_title = "OVERALL SCORE DISTRIBUTION"
|
|
258
|
+
print(f"\n {dist_title}")
|
|
259
|
+
print(f" {'-' * len(dist_title)}")
|
|
260
|
+
print(f" {'min _score':<20} = {df['_score'].min():.4f}")
|
|
261
|
+
print(f" {'max _score':<20} = {df['_score'].max():.4f}")
|
|
262
|
+
print(f" {'mean _score':<20} = {df['_score'].mean():.4f}")
|
|
263
|
+
|
|
264
|
+
|
|
265
|
+
def _progress_drop_nan(df, score_cols, nan_mask):
|
|
266
|
+
title = "DROP_NAN CHECK"
|
|
267
|
+
n_dropped = int(nan_mask.sum())
|
|
268
|
+
print(f"\n {title}")
|
|
269
|
+
print(f" {'-' * len(title)}")
|
|
270
|
+
print(f" {'rows scanned':<20} = {len(df)}")
|
|
271
|
+
print(f" {'rows with NaN':<20} = {n_dropped}")
|
|
272
|
+
print(f" {'rows kept':<20} = {len(df) - n_dropped}")
|
|
273
|
+
if n_dropped:
|
|
274
|
+
offending = [c for c in score_cols if df[c].isna().any()]
|
|
275
|
+
print(f" {'NaN found in':<20} = {offending}")
|
|
276
|
+
|
|
277
|
+
|
|
278
|
+
def _progress_normalize(raw_scores, normalized_scores, total_weight):
|
|
279
|
+
title = "NORMALIZE CHECK"
|
|
280
|
+
print(f"\n {title}")
|
|
281
|
+
print(f" {'-' * len(title)}")
|
|
282
|
+
print(f" {'total weight':<20} = {total_weight}")
|
|
283
|
+
print(f" {'formula':<20} : _score = raw_score / total_weight")
|
|
284
|
+
n = min(5, len(raw_scores))
|
|
285
|
+
raw_label = f"raw (first {n})"
|
|
286
|
+
norm_label = f"normalized (first {n})"
|
|
287
|
+
print(f" {raw_label:<20} = {raw_scores.head(n).round(4).tolist()}")
|
|
288
|
+
print(f" {norm_label:<20} = {normalized_scores.head(n).round(4).tolist()}")
|
|
289
|
+
print(f" {'min normalized':<20} = {normalized_scores.min():.4f}")
|
|
290
|
+
print(f" {'max normalized':<20} = {normalized_scores.max():.4f}")
|
|
291
|
+
|
|
292
|
+
|
|
293
|
+
def _progress_normalize_skip(total_weight):
|
|
294
|
+
title = "NORMALIZE CHECK"
|
|
295
|
+
print(f"\n {title}")
|
|
296
|
+
print(f" {'-' * len(title)}")
|
|
297
|
+
print(f" {'total weight':<20} = {total_weight}")
|
|
298
|
+
print(f" [!] total weight <= 0. skipping normalization, _score left as raw sum")
|
|
299
|
+
|
|
300
|
+
|
|
301
|
+
# =============================================================================
|
|
302
|
+
# SPARPARTNER
|
|
303
|
+
# =============================================================================
|
|
304
|
+
|
|
305
|
+
def sample(df, bench_marks, custom_weights, look_alike=True, show_progress=False, return_score=False, drop_nan=True):
|
|
306
|
+
"""
|
|
307
|
+
Stratified sampler for train/test prep — guards against overfitting
|
|
308
|
+
by letting you deliberately hold out the rows that most resemble a
|
|
309
|
+
benchmark case (rather than a random split), so a model trained on
|
|
310
|
+
the rest can be tested against its true "opponent" instead of an
|
|
311
|
+
easy, self-similar sample.
|
|
312
|
+
|
|
313
|
+
Type-aware scoring: mirrors the reference file's
|
|
314
|
+
fixed_distance_score, closest to the benchmark scores highest.
|
|
315
|
+
|
|
316
|
+
For each [name, weight] in custom_weights, looks up bench_marks[name]
|
|
317
|
+
and detects the column's type automatically:
|
|
318
|
+
- numeric column -> distance = abs(value - bench), cap = the
|
|
319
|
+
column's own max observed distance from
|
|
320
|
+
bench (so a bench of 0 or a negative
|
|
321
|
+
number still scores correctly — the
|
|
322
|
+
farthest row from bench scores 0, bench
|
|
323
|
+
itself scores 1, regardless of bench's
|
|
324
|
+
own magnitude)
|
|
325
|
+
- date column -> value and bench both converted to "age in
|
|
326
|
+
days" relative to the benchmark date
|
|
327
|
+
(benchmark itself = 0), cap = the column's
|
|
328
|
+
own max observed age (so scores stay in
|
|
329
|
+
a sane 0-1 range regardless of the actual
|
|
330
|
+
day-count magnitude)
|
|
331
|
+
- string column -> 1 if exact match to bench else 0, cap = 1
|
|
332
|
+
|
|
333
|
+
score = clip(1 - distance / cap, 0, 1) — same formula as the
|
|
334
|
+
reference file's fixed_distance_score.
|
|
335
|
+
|
|
336
|
+
Each column's score is multiplied by its weight and summed into
|
|
337
|
+
one raw score per row, then divided by the total weight so the
|
|
338
|
+
final "_score" always lands in 0-1 regardless of how many signals
|
|
339
|
+
or what weights were used (1.0 = perfect lookalike across every
|
|
340
|
+
signal, 0.0 = as far as possible on every signal). A "NORMALIZE
|
|
341
|
+
CHECK" print always runs — regardless of show_progress — showing
|
|
342
|
+
the total weight and a few raw-vs-normalized score examples so
|
|
343
|
+
this step is easy to sanity-check.
|
|
344
|
+
|
|
345
|
+
look_alike : bool, default True.
|
|
346
|
+
True -> highest _score sorted to the TOP — rows that look most
|
|
347
|
+
like the benchmark (lookalikes) come first.
|
|
348
|
+
False -> highest _score sorted to the BOTTOM instead — rows
|
|
349
|
+
farthest from the benchmark come first.
|
|
350
|
+
|
|
351
|
+
Tie-breaking: rows that land on the exact same "_score" are not
|
|
352
|
+
left to pandas' stable sort alone. They're broken next by the
|
|
353
|
+
per-signal score (_score_<name>, the 0-1 value before weighting)
|
|
354
|
+
of the highest-weight signal in custom_weights; if still tied,
|
|
355
|
+
the next-highest-weight signal is checked, cascading down the
|
|
356
|
+
weight-sorted signal list until the tie breaks. Signals sharing
|
|
357
|
+
the same weight are compared in the order they appear in
|
|
358
|
+
custom_weights. Every tie-break column sorts in the same
|
|
359
|
+
direction as "_score" itself — a higher per-signal score wins
|
|
360
|
+
under look_alike=True, a lower one wins under look_alike=False.
|
|
361
|
+
Only if every signal is exhausted and rows are still tied does
|
|
362
|
+
the order fall back to pandas' stable sort (original row order).
|
|
363
|
+
|
|
364
|
+
Parameters
|
|
365
|
+
----------
|
|
366
|
+
df : DataFrame
|
|
367
|
+
Must contain a column for every name in custom_weights.
|
|
368
|
+
bench_marks : dict
|
|
369
|
+
{column_name: benchmark_value} — one entry per weighted signal.
|
|
370
|
+
custom_weights : list[[name, weight]]
|
|
371
|
+
Which columns to score and how much each contributes.
|
|
372
|
+
look_alike : bool, default True
|
|
373
|
+
See above.
|
|
374
|
+
show_progress : bool, default False
|
|
375
|
+
If True, prints a client-centric readout: a header naming every
|
|
376
|
+
signal/weight/benchmark, a per-column breakdown of detected
|
|
377
|
+
type + cap + sample distances/scores as it scores, a sort-apply
|
|
378
|
+
readout showing the sort_values call being made plus scores
|
|
379
|
+
before and after, so the look_alike direction is visible as
|
|
380
|
+
it's applied, the top (or bottom) N ranked
|
|
381
|
+
rows with each signal's value x weight contribution spelled
|
|
382
|
+
out, and a final health check flagging any signal that isn't
|
|
383
|
+
contributing useful separation.
|
|
384
|
+
return_score : bool, default False
|
|
385
|
+
If True, keeps the "_score" and per-signal "_score_<name>"
|
|
386
|
+
columns in the returned DataFrame. If False (default), those
|
|
387
|
+
columns are dropped before returning — they're still used
|
|
388
|
+
internally for sorting and (if show_progress) for the
|
|
389
|
+
progress readout, just not included in the result.
|
|
390
|
+
drop_nan : bool, default True
|
|
391
|
+
If True, after scoring, any row with a NaN in any per-signal
|
|
392
|
+
"_score_<name>" column is dropped (e.g. from a missing value
|
|
393
|
+
in a signal column, or a date that failed to parse). A sanity
|
|
394
|
+
print always runs first — regardless of show_progress —
|
|
395
|
+
reporting rows scanned, rows with NaN, rows kept, and which
|
|
396
|
+
signal(s) had the NaN, before the drop happens. If False, rows
|
|
397
|
+
with NaN scores are kept and will carry a NaN "_score".
|
|
398
|
+
|
|
399
|
+
Returns
|
|
400
|
+
-------
|
|
401
|
+
df — a copy of the input, sorted by _score (direction depends on
|
|
402
|
+
look_alike), with the "_score" (and per-signal "_score_<name>")
|
|
403
|
+
columns included only if return_score is True.
|
|
404
|
+
"""
|
|
405
|
+
_validate_custom_weights(df, bench_marks, custom_weights)
|
|
406
|
+
|
|
407
|
+
df = df.copy()
|
|
408
|
+
df["_score"] = 0.0
|
|
409
|
+
|
|
410
|
+
if show_progress:
|
|
411
|
+
_progress_header(bench_marks, custom_weights, look_alike, len(df))
|
|
412
|
+
|
|
413
|
+
for name, weight in custom_weights:
|
|
414
|
+
bench_value = bench_marks[name]
|
|
415
|
+
col_numeric, bench_numeric, kind = _to_numeric_series(df[name], bench_value)
|
|
416
|
+
|
|
417
|
+
distance = (col_numeric - bench_numeric).abs()
|
|
418
|
+
cap = distance.max() if kind in ("date", "numeric") else bench_numeric
|
|
419
|
+
|
|
420
|
+
if cap is None or pd.isna(cap) or cap <= 0:
|
|
421
|
+
score = pd.Series(1.0, index=df.index)
|
|
422
|
+
else:
|
|
423
|
+
score = (1.0 - (distance / cap)).clip(0.0, 1.0)
|
|
424
|
+
|
|
425
|
+
df[f"_score_{name}"] = score
|
|
426
|
+
df["_score"] += score * weight
|
|
427
|
+
|
|
428
|
+
if show_progress:
|
|
429
|
+
_progress_column(name, weight, kind, bench_value, cap, distance, score)
|
|
430
|
+
|
|
431
|
+
if drop_nan:
|
|
432
|
+
score_cols = [f"_score_{name}" for name, _ in custom_weights]
|
|
433
|
+
nan_mask = df[score_cols].isna().any(axis=1)
|
|
434
|
+
_progress_drop_nan(df, score_cols, nan_mask)
|
|
435
|
+
if nan_mask.any():
|
|
436
|
+
df = df[~nan_mask].reset_index(drop=True)
|
|
437
|
+
|
|
438
|
+
total_weight = sum(w for _, w in custom_weights)
|
|
439
|
+
if total_weight > 0:
|
|
440
|
+
raw_scores = df["_score"].copy()
|
|
441
|
+
df["_score"] = df["_score"] / total_weight
|
|
442
|
+
_progress_normalize(raw_scores, df["_score"], total_weight)
|
|
443
|
+
else:
|
|
444
|
+
_progress_normalize_skip(total_weight)
|
|
445
|
+
|
|
446
|
+
# Tie-break: rows with an identical "_score" are ordered next by the
|
|
447
|
+
# per-signal score (_score_<name>, the 0-1 value before weighting) of
|
|
448
|
+
# the highest-weight signal, then the next-highest weight, and so on,
|
|
449
|
+
# cascading through custom_weights sorted by descending weight until
|
|
450
|
+
# the tie breaks or the signals run out (any remaining tie then falls
|
|
451
|
+
# back to pandas' stable sort, i.e. original row order). Signals that
|
|
452
|
+
# share the same weight are compared in the order they appear in
|
|
453
|
+
# custom_weights. Every tie-break column sorts in the same direction
|
|
454
|
+
# as "_score" itself, so a higher per-signal score wins under
|
|
455
|
+
# look_alike=True and a lower one wins under look_alike=False.
|
|
456
|
+
tie_break_signals = sorted(custom_weights, key=lambda pair: -pair[1])
|
|
457
|
+
sort_cols = ["_score"] + [f"_score_{name}" for name, _ in tie_break_signals]
|
|
458
|
+
ascending = not look_alike
|
|
459
|
+
|
|
460
|
+
if show_progress:
|
|
461
|
+
_progress_sort_apply(df, look_alike, sort_cols)
|
|
462
|
+
|
|
463
|
+
df = df.sort_values(sort_cols, ascending=ascending).reset_index(drop=True)
|
|
464
|
+
|
|
465
|
+
if show_progress:
|
|
466
|
+
_progress_sort_glimpse(df, look_alike)
|
|
467
|
+
_progress_top_rows(df, custom_weights, look_alike)
|
|
468
|
+
_progress_health_check(df, custom_weights, bench_marks)
|
|
469
|
+
|
|
470
|
+
if not return_score:
|
|
471
|
+
score_cols = [c for c in df.columns if c == "_score" or c.startswith("_score_")]
|
|
472
|
+
df = df.drop(columns=score_cols)
|
|
473
|
+
|
|
474
|
+
return df
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: sparpartner
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Deterministic, benchmark-driven stratified sampler for train/test prep.
|
|
5
|
+
Author: Henry
|
|
6
|
+
Author-email: Henry <osas2henry@gmail.com>
|
|
7
|
+
License: All Rights Reserved
|
|
8
|
+
Classifier: License :: Other/Proprietary License
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.8
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
License-File: LICENSE
|
|
14
|
+
Requires-Dist: pandas>=1.3
|
|
15
|
+
Dynamic: author
|
|
16
|
+
Dynamic: license-file
|
|
17
|
+
Dynamic: requires-python
|
|
18
|
+
|
|
19
|
+
# sparpartner
|
|
20
|
+
|
|
21
|
+
A deterministic stratified sampler for train/test prep. It doesn't
|
|
22
|
+
split your data randomly, it deliberately finds the rows that most
|
|
23
|
+
resemble a benchmark case, so you can hold those out as a genuine
|
|
24
|
+
test set and train on everything else.
|
|
25
|
+
|
|
26
|
+
## Why this exists
|
|
27
|
+
|
|
28
|
+
A random train/test split assumes the test set should look
|
|
29
|
+
statistically like the train set. That's the wrong question if
|
|
30
|
+
what you actually want to know is: **does the model generalize past
|
|
31
|
+
one specific profile, or did it just memorize the neighborhood
|
|
32
|
+
around it?**
|
|
33
|
+
|
|
34
|
+
`sparpartner` answers that by ranking every row in your data by how
|
|
35
|
+
closely it resembles a benchmark ("bench_marks") you define, then
|
|
36
|
+
sorting on that resemblance. You then slice the sorted frame
|
|
37
|
+
yourself:
|
|
38
|
+
|
|
39
|
+
- **Lookalikes as test, the rest as train**: train on rows *unlike*
|
|
40
|
+
the benchmark, test on the rows that *are* like it. This is the
|
|
41
|
+
harder, more honest check: it tells you whether the model actually
|
|
42
|
+
learned something general, or only performs well near cases it's
|
|
43
|
+
already seen a lot of.
|
|
44
|
+
- **Lookalikes as train, the rest as test**: the reverse, if you
|
|
45
|
+
want to check the opposite direction.
|
|
46
|
+
|
|
47
|
+
`sparpartner` only produces the ranking. The actual train/test cut
|
|
48
|
+
is a plain slice on your side (see [Usage](#usage) below).
|
|
49
|
+
|
|
50
|
+
## Where the idea comes from
|
|
51
|
+
|
|
52
|
+
Two unrelated places, both about comparing something against a
|
|
53
|
+
reference on purpose rather than at random:
|
|
54
|
+
|
|
55
|
+
**A boxer preparing for a match.** A fighter in camp doesn't spar
|
|
56
|
+
with whoever's free in the gym, they specifically look for a
|
|
57
|
+
sparring partner who moves, reaches, and hits like the opponent
|
|
58
|
+
they're about to face. Training against a random partner tells you
|
|
59
|
+
nothing about how you'll actually do; training against someone who
|
|
60
|
+
resembles the real threat does. `sparpartner` applies that same
|
|
61
|
+
logic to a model: instead of a random holdout, it finds the rows
|
|
62
|
+
that resemble the toughest, most relevant "opponent" profile and
|
|
63
|
+
holds those back as the real test, so what's left to train on is
|
|
64
|
+
everything *unlike* that opponent, and the test genuinely checks
|
|
65
|
+
whether the model can handle the match it's actually walking into.
|
|
66
|
+
|
|
67
|
+
**Astrology's approach to comparing a chart to a reference.** Reading
|
|
68
|
+
a chart against a benchmark isn't a single yes/no match, it's
|
|
69
|
+
several weighted dimensions (sun sign, moon sign, rising, houses,
|
|
70
|
+
and so on) each compared individually, then combined into one
|
|
71
|
+
overall resemblance reading. No single dimension decides the
|
|
72
|
+
outcome alone; the composite is what matters. That's the same shape
|
|
73
|
+
`sparpartner` uses: several weighted signals, each scored
|
|
74
|
+
independently against a benchmark, summed into one composite
|
|
75
|
+
`_score` rather than a single all-or-nothing match.
|
|
76
|
+
|
|
77
|
+
## How the scoring works
|
|
78
|
+
|
|
79
|
+
You give it:
|
|
80
|
+
|
|
81
|
+
- `df`: your data
|
|
82
|
+
- `bench_marks`: a dict of `{column_name: benchmark_value}`, one
|
|
83
|
+
entry per signal you care about
|
|
84
|
+
- `custom_weights`: a list of `[column_name, weight]` pairs, saying
|
|
85
|
+
which columns matter and how much
|
|
86
|
+
|
|
87
|
+
For each weighted column, `sparpartner` auto-detects the column's
|
|
88
|
+
type and scores every row's distance to the benchmark on a 0–1
|
|
89
|
+
scale (1.0 = exact match, 0.0 = as far as possible):
|
|
90
|
+
|
|
91
|
+
| Detected type | How distance is measured |
|
|
92
|
+
|---|---|
|
|
93
|
+
| **numeric** | `abs(value - bench)`, capped by the column's own max observed distance from bench |
|
|
94
|
+
| **date** | both sides converted to "age in days" relative to the benchmark date, capped by the column's own max observed age |
|
|
95
|
+
| **string** | exact match = 1, anything else = 0 |
|
|
96
|
+
|
|
97
|
+
Date detection is automatic, it only treats a column as a date if
|
|
98
|
+
its values look date-shaped (contain a separator like `-`, `/`, `.`
|
|
99
|
+
or a recognizable month name) **and** parse successfully at least
|
|
100
|
+
98% of the time. Bare numeric-looking strings (e.g. `"12345"`) are
|
|
101
|
+
never mistaken for dates, and object-dtype columns of digit strings
|
|
102
|
+
are treated as exact-match strings, not numbers. Only a real
|
|
103
|
+
numeric dtype gets the numeric path.
|
|
104
|
+
|
|
105
|
+
Each column's 0–1 score is multiplied by its weight and summed into
|
|
106
|
+
one raw score per row, then divided by the total weight so the
|
|
107
|
+
final `_score` always lands in the 0–1 range, however many signals
|
|
108
|
+
or weights you used.
|
|
109
|
+
|
|
110
|
+
### Tie-breaking
|
|
111
|
+
|
|
112
|
+
Rows that land on the exact same `_score` aren't left to random or
|
|
113
|
+
arbitrary order. Ties are broken by the per-signal score of the
|
|
114
|
+
**highest-weight** signal first, then the next-highest, cascading
|
|
115
|
+
down the weighted signal list until the tie resolves. Signals that
|
|
116
|
+
share the same weight are compared in the order they appear in
|
|
117
|
+
`custom_weights`. The tie-break always sorts in the same direction
|
|
118
|
+
as the main score (see `look_alike` below). Only if every signal is
|
|
119
|
+
exhausted and rows are still tied does it fall back to original row
|
|
120
|
+
order.
|
|
121
|
+
|
|
122
|
+
## `look_alike`
|
|
123
|
+
|
|
124
|
+
Controls sort direction:
|
|
125
|
+
|
|
126
|
+
- `look_alike=True` (default): highest `_score` (closest to
|
|
127
|
+
benchmark) sorted to the **top**.
|
|
128
|
+
- `look_alike=False`: highest `_score` sorted to the **bottom**,
|
|
129
|
+
farthest-from-benchmark rows come first.
|
|
130
|
+
|
|
131
|
+
## Usage
|
|
132
|
+
|
|
133
|
+
### Sample usage
|
|
134
|
+
|
|
135
|
+
```python
|
|
136
|
+
import pandas as pd
|
|
137
|
+
from sparpartner import sample
|
|
138
|
+
|
|
139
|
+
df = pd.DataFrame({
|
|
140
|
+
"id": [1, 2, 3, 4, 5],
|
|
141
|
+
"age": [25, 30, 47, 52, 33],
|
|
142
|
+
"signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
|
|
143
|
+
"country": ["US", "US", "CA", "US", "MX"],
|
|
144
|
+
})
|
|
145
|
+
|
|
146
|
+
bench_marks = {
|
|
147
|
+
"age": 30,
|
|
148
|
+
"signup_date": "2023-01-01",
|
|
149
|
+
"country": "US",
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
custom_weights = [
|
|
153
|
+
["age", 2],
|
|
154
|
+
["signup_date", 1],
|
|
155
|
+
["country", 1],
|
|
156
|
+
]
|
|
157
|
+
|
|
158
|
+
result = sample(
|
|
159
|
+
df,
|
|
160
|
+
bench_marks,
|
|
161
|
+
custom_weights,
|
|
162
|
+
look_alike=True, # True = most similar rows first
|
|
163
|
+
show_progress=True, # prints the full scoring breakdown
|
|
164
|
+
return_score=True, # keep _score columns in the output
|
|
165
|
+
drop_nan=True,
|
|
166
|
+
)
|
|
167
|
+
|
|
168
|
+
print(result)
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
`age=30`, `signup_date="2023-01-01"`, and `country="US"` closely
|
|
172
|
+
match row `id=1` (age 25, close date, US), so with
|
|
173
|
+
`look_alike=True` that row lands at or near the top of the sorted
|
|
174
|
+
output. Flip `look_alike=False` to instead surface the rows that
|
|
175
|
+
look *least* like the benchmark (e.g. the Mexico row with the
|
|
176
|
+
furthest signup date).
|
|
177
|
+
|
|
178
|
+
```python
|
|
179
|
+
# --- post-sample: turn the ranking into an actual train/test split ---
|
|
180
|
+
# Re-run with look_alike=False so lookalikes sort to the BOTTOM of `result`.
|
|
181
|
+
ranked = sample(df, bench_marks, custom_weights, look_alike=False)
|
|
182
|
+
|
|
183
|
+
# Slice however large you want the test set to be, e.g. the bottom 30%:
|
|
184
|
+
cut = int(len(ranked) * 0.7)
|
|
185
|
+
train = ranked.iloc[:cut] # non-lookalikes
|
|
186
|
+
test = ranked.iloc[cut:] # lookalikes, the harder, honest test set
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### Parameters
|
|
190
|
+
|
|
191
|
+
| Name | Type | Default | What it does |
|
|
192
|
+
|---|---|---|---|
|
|
193
|
+
| `df` | DataFrame | required | Must contain a column for every name in `custom_weights` |
|
|
194
|
+
| `bench_marks` | dict | required | `{column_name: benchmark_value}` |
|
|
195
|
+
| `custom_weights` | list of `[name, weight]` | required | Which columns to score and how much each contributes |
|
|
196
|
+
| `look_alike` | bool | `True` | `True` = lookalikes sorted to top; `False` = sorted to bottom |
|
|
197
|
+
| `show_progress` | bool | `False` | Prints a full readout: header, per-column type/cap/sample scores, sort direction, top/bottom ranked rows with weighted contributions, and a health check flagging any signal that isn't adding useful separation |
|
|
198
|
+
| `return_score` | bool | `False` | If `True`, keeps `_score` and `_score_<name>` columns in the result instead of dropping them |
|
|
199
|
+
| `drop_nan` | bool | `True` | If `True`, drops any row with a NaN component score (e.g. missing value or unparseable date) after printing a sanity check of what was dropped and why |
|
|
200
|
+
|
|
201
|
+
### Validation
|
|
202
|
+
|
|
203
|
+
`custom_weights` is validated upfront, before any scoring starts, and
|
|
204
|
+
raises `ValueError` if:
|
|
205
|
+
|
|
206
|
+
- a column name isn't a string, or doesn't exist in `df`
|
|
207
|
+
- a column name has no matching entry in `bench_marks`
|
|
208
|
+
- the same column name appears more than once
|
|
209
|
+
- a weight isn't numeric (bools are rejected too, a `bool` is
|
|
210
|
+
technically an `int` in Python but was never meant as a weight)
|
|
211
|
+
- a weight is `NaN`
|
|
212
|
+
|
|
213
|
+
## A couple of things worth knowing
|
|
214
|
+
|
|
215
|
+
- **Zero-variance columns**: if every row (including the benchmark)
|
|
216
|
+
has the same value for a signal, there's nothing to measure
|
|
217
|
+
distance against, that signal scores every row 1.0 rather than
|
|
218
|
+
dividing by zero.
|
|
219
|
+
- **Negative total weight**: if your weights sum to ≤ 0, the
|
|
220
|
+
normalization step is skipped and `_score` is left as the raw
|
|
221
|
+
weighted sum instead of being silently divided by a non-positive
|
|
222
|
+
number.
|
|
223
|
+
- **Object-dtype numeric strings**: a column of strings like
|
|
224
|
+
`"100"`, `"200"` (object dtype, no separator) is scored as an
|
|
225
|
+
exact-match string column, *not* auto-converted to numeric. Only
|
|
226
|
+
genuine numeric dtypes (`int`, `float`) get the numeric distance
|
|
227
|
+
path.
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
setup.py
|
|
5
|
+
sparpartner/__init__.py
|
|
6
|
+
sparpartner/main.py
|
|
7
|
+
sparpartner.egg-info/PKG-INFO
|
|
8
|
+
sparpartner.egg-info/SOURCES.txt
|
|
9
|
+
sparpartner.egg-info/dependency_links.txt
|
|
10
|
+
sparpartner.egg-info/requires.txt
|
|
11
|
+
sparpartner.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
pandas>=1.3
|