robustsignalmaker 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- robustsignalmaker-0.2.0/LICENSE +21 -0
- robustsignalmaker-0.2.0/PKG-INFO +356 -0
- robustsignalmaker-0.2.0/README.md +307 -0
- robustsignalmaker-0.2.0/pyproject.toml +45 -0
- robustsignalmaker-0.2.0/robustsignalmaker/__init__.py +188 -0
- robustsignalmaker-0.2.0/robustsignalmaker/config.py +91 -0
- robustsignalmaker-0.2.0/robustsignalmaker/masking.py +518 -0
- robustsignalmaker-0.2.0/robustsignalmaker/metrics.py +193 -0
- robustsignalmaker-0.2.0/robustsignalmaker/models.py +309 -0
- robustsignalmaker-0.2.0/robustsignalmaker/nested_cv.py +527 -0
- robustsignalmaker-0.2.0/robustsignalmaker/py.typed +0 -0
- robustsignalmaker-0.2.0/robustsignalmaker/representations.py +226 -0
- robustsignalmaker-0.2.0/robustsignalmaker/reproducibility.py +96 -0
- robustsignalmaker-0.2.0/robustsignalmaker/results.py +177 -0
- robustsignalmaker-0.2.0/robustsignalmaker/segments.py +116 -0
- robustsignalmaker-0.2.0/robustsignalmaker/selection.py +559 -0
- robustsignalmaker-0.2.0/robustsignalmaker/synthetic.py +244 -0
- robustsignalmaker-0.2.0/robustsignalmaker/tuning.py +142 -0
- robustsignalmaker-0.2.0/robustsignalmaker/validity.py +625 -0
- robustsignalmaker-0.2.0/robustsignalmaker.egg-info/PKG-INFO +356 -0
- robustsignalmaker-0.2.0/robustsignalmaker.egg-info/SOURCES.txt +23 -0
- robustsignalmaker-0.2.0/robustsignalmaker.egg-info/dependency_links.txt +1 -0
- robustsignalmaker-0.2.0/robustsignalmaker.egg-info/requires.txt +6 -0
- robustsignalmaker-0.2.0/robustsignalmaker.egg-info/top_level.txt +1 -0
- robustsignalmaker-0.2.0/setup.cfg +4 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Amanda S Barnard
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,356 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: robustsignalmaker
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: NaN-aware, leakage-free stability selection for scientific signals and spectra
|
|
5
|
+
Author: Amanda S Barnard
|
|
6
|
+
License: MIT License
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026 Amanda S Barnard
|
|
9
|
+
|
|
10
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
11
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
12
|
+
in the Software without restriction, including without limitation the rights
|
|
13
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
14
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
15
|
+
furnished to do so, subject to the following conditions:
|
|
16
|
+
|
|
17
|
+
The above copyright notice and this permission notice shall be included in all
|
|
18
|
+
copies or substantial portions of the Software.
|
|
19
|
+
|
|
20
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
21
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
22
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
23
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
24
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
25
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
26
|
+
SOFTWARE.
|
|
27
|
+
|
|
28
|
+
Project-URL: Homepage, https://github.com/amaxiom/RobustSignalMaker
|
|
29
|
+
Project-URL: Source, https://github.com/amaxiom/RobustSignalMaker
|
|
30
|
+
Project-URL: Changelog, https://github.com/amaxiom/RobustSignalMaker/blob/main/CHANGELOG.md
|
|
31
|
+
Classifier: Development Status :: 4 - Beta
|
|
32
|
+
Classifier: Intended Audience :: Science/Research
|
|
33
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
34
|
+
Classifier: Programming Language :: Python :: 3
|
|
35
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
36
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
37
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
38
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
39
|
+
Classifier: Topic :: Scientific/Engineering
|
|
40
|
+
Requires-Python: >=3.9
|
|
41
|
+
Description-Content-Type: text/markdown
|
|
42
|
+
License-File: LICENSE
|
|
43
|
+
Requires-Dist: numpy>=1.24
|
|
44
|
+
Requires-Dist: scipy>=1.10
|
|
45
|
+
Requires-Dist: scikit-learn>=1.3
|
|
46
|
+
Provides-Extra: tables
|
|
47
|
+
Requires-Dist: pandas>=1.5; extra == "tables"
|
|
48
|
+
Dynamic: license-file
|
|
49
|
+
|
|
50
|
+
# RobustSignalMaker
|
|
51
|
+
|
|
52
|
+
NaN-aware, leakage-free stability selection for scientific signals and
|
|
53
|
+
spectra. RSM identifies the important parts of a series (time series,
|
|
54
|
+
spectra, diffraction patterns, mass spectra, sensor recordings) to retain
|
|
55
|
+
and removes the rest, returning a reproducible region of the sampling axis
|
|
56
|
+
and an honest estimate of what that region can predict.
|
|
57
|
+
|
|
58
|
+
## Install
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
pip install robustsignalmaker
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Python 3.9 or newer. Three required dependencies, all of which you almost
|
|
65
|
+
certainly already have: `numpy>=1.24`, `scipy>=1.10`, `scikit-learn>=1.3`.
|
|
66
|
+
There is no deep-learning dependency; the gated engine is numpy with
|
|
67
|
+
hand-derived gradients.
|
|
68
|
+
|
|
69
|
+
One optional extra, for writing result tables through pandas rather than
|
|
70
|
+
the built-in csv writer:
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
pip install "robustsignalmaker[tables]"
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Verify the install:
|
|
77
|
+
|
|
78
|
+
```python
|
|
79
|
+
import robustsignalmaker as rsm
|
|
80
|
+
print(rsm.__version__)
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The package ships a `py.typed` marker, so type checkers will use its
|
|
84
|
+
annotations without a stub package.
|
|
85
|
+
|
|
86
|
+
## Why RSM
|
|
87
|
+
|
|
88
|
+
Most band-selection tools report a single selection and leave you to guess
|
|
89
|
+
how much of it would survive a different sample. RSM's deliverable is the
|
|
90
|
+
reproducible band set and an honest account of what discarding the rest
|
|
91
|
+
costs:
|
|
92
|
+
|
|
93
|
+
- **Leakage-safe stability selection is the product.** Selection is
|
|
94
|
+
aggregated over resamples into stability frequencies
|
|
95
|
+
(Meinshausen-Buhlmann style, with Shah-Samworth complementary pairs
|
|
96
|
+
available), wrapped in nested cross-validation where every fitted
|
|
97
|
+
statistic lives inside the training fold, and compared against a matched
|
|
98
|
+
full-signal baseline with a preserved / significantly-better /
|
|
99
|
+
significantly-worse verdict. On six real datasets every selection
|
|
100
|
+
compresses hard and returns `preserved`: tecator 7 of 20 segments, corn 4
|
|
101
|
+
of 35, ovarian SELDI-TOF 6 of 64, RRUFF Raman 8.5 of 64 per fold,
|
|
102
|
+
BasicMotions 2.75 of 10, and four of eighteen bond-angle bands on a gold
|
|
103
|
+
nanoparticle dataset.
|
|
104
|
+
|
|
105
|
+
Real instrument data is where that gets hard, because a selection frequency
|
|
106
|
+
is not well defined when different samples observe different parts of the
|
|
107
|
+
axis. The rest of the machinery exists to make it well defined:
|
|
108
|
+
|
|
109
|
+
- **No value is ever fabricated.** Every pooling, lens, and model operation
|
|
110
|
+
is renormalised over the evidence that actually exists (validity masks
|
|
111
|
+
travel with the data end to end). Detector dropout, saturation,
|
|
112
|
+
instrument-range gaps and not-detected zeros are absence, not zeros.
|
|
113
|
+
- **Three-valued verdicts.** Every candidate band ends as selected,
|
|
114
|
+
rejected, or UNASSESSABLE. A region nobody measured can never read as
|
|
115
|
+
"stably rejected", and every selection frequency is reported beside its
|
|
116
|
+
observation support.
|
|
117
|
+
- **Honesty guards.** Samples with too little evidence are refused, not
|
|
118
|
+
guessed. If the missingness pattern alone predicts the target (a
|
|
119
|
+
non-ignorable, instrument-linked confound), RSM warns and stamps the
|
|
120
|
+
result; imputation-first pipelines destroy exactly that alarm.
|
|
121
|
+
- **Your choice of predictor.** Selection is done by the gated head, but the
|
|
122
|
+
final model on the chosen bands is yours: `refit_model=` takes `lin`,
|
|
123
|
+
`rdg`, `las`, `eln`, `log`, `svm`, `rf`, `xgb`, `mlp`, or `pls` for PLS
|
|
124
|
+
regression and PLS-DA. Swapping it never changes the selection, and the
|
|
125
|
+
refusal rule survives: a random forest refuses the same thin samples the
|
|
126
|
+
head refuses rather than predicting from imputed values.
|
|
127
|
+
- **Lens marginalisation.** Band importance depends on how a signal is
|
|
128
|
+
represented (raw intensity, derivatives, smoothed scales). RSM can
|
|
129
|
+
marginalise the selection over a lens ensemble and reports how much the
|
|
130
|
+
lenses agree.
|
|
131
|
+
|
|
132
|
+
## Input contract
|
|
133
|
+
|
|
134
|
+
```
|
|
135
|
+
X : float array (n_samples, n_channels, n_points)
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
All samples share one sampling axis of length `n_points`. A single-channel
|
|
139
|
+
spectrum is `n_channels=1`, so reshape a plain `(n_samples, n_points)`
|
|
140
|
+
matrix with `X[:, None, :]`.
|
|
141
|
+
|
|
142
|
+
Missingness is declared in one of two ways, and they are equivalent:
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
import numpy as np
|
|
146
|
+
|
|
147
|
+
# 1. NaN in X means "not measured"
|
|
148
|
+
X = np.random.default_rng(0).normal(size=(20, 1, 64))
|
|
149
|
+
X[3, 0, 10:14] = np.nan
|
|
150
|
+
|
|
151
|
+
# 2. or pass an explicit boolean validity mask, True = observed
|
|
152
|
+
V = np.isfinite(X)
|
|
153
|
+
V[5, 0, 20:24] = False # censored, though a value is present
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Pass the mask as the third argument to `fit`, or leave it out and let the
|
|
157
|
+
NaNs speak: `sel.fit(X, y)` and `sel.fit(X, y, V)` are the same call.
|
|
158
|
+
|
|
159
|
+
Use the explicit mask when a missing value is not representable as NaN, for
|
|
160
|
+
example a mass-spectrometry zero that means "not detected" rather than
|
|
161
|
+
"intensity zero", or a saturated detector reading you want treated as
|
|
162
|
+
censored.
|
|
163
|
+
|
|
164
|
+
`y` may be regression, binary or multiclass, and may contain NaN for
|
|
165
|
+
unlabelled samples: they are excluded from every split, loss, resample and
|
|
166
|
+
score, and counted in the result rather than silently absorbed.
|
|
167
|
+
|
|
168
|
+
Multi-channel series get one gate per band shared across channels, with
|
|
169
|
+
per-channel missingness handled underneath the gate.
|
|
170
|
+
|
|
171
|
+
## Quick start
|
|
172
|
+
|
|
173
|
+
This example is self-contained and runs as written.
|
|
174
|
+
|
|
175
|
+
```python
|
|
176
|
+
import numpy as np
|
|
177
|
+
from robustsignalmaker import BootstrapMaskSelector, make_signal_control
|
|
178
|
+
|
|
179
|
+
# a control with a known informative band, so the answer is checkable
|
|
180
|
+
c = make_signal_control(n=200, n_points=256, task="binary", seed=0)
|
|
181
|
+
X, y = c.X, c.y # X is (200, 1, 256)
|
|
182
|
+
|
|
183
|
+
X = X.copy()
|
|
184
|
+
X[:80, 0, 180:210] = np.nan # 80 samples never saw this stretch
|
|
185
|
+
|
|
186
|
+
sel = BootstrapMaskSelector(task="binary", segment=16,
|
|
187
|
+
n_bootstrap=30, seed=0).fit(X, y)
|
|
188
|
+
|
|
189
|
+
print("kept bands :", sel.selected_segments_)
|
|
190
|
+
print("point indices :", sel.selected_points()[:12], "...")
|
|
191
|
+
print("coverage :", round(sel.coverage(), 3))
|
|
192
|
+
truth = sorted(set(np.flatnonzero(c.informative) // 16))
|
|
193
|
+
print("truth was :", truth) # the planted band, for comparison
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
## Reading the output
|
|
197
|
+
|
|
198
|
+
This is the part worth five minutes, because a selection frequency without
|
|
199
|
+
its support is the thing RSM exists to stop you reporting.
|
|
200
|
+
|
|
201
|
+
```python
|
|
202
|
+
import numpy as np
|
|
203
|
+
|
|
204
|
+
for k in range(sel.grid.n_segments):
|
|
205
|
+
print(f"band {k:3d} pi={sel.pi_[k]!s:>6} "
|
|
206
|
+
f"support={sel.support_[k]:.2f} {sel.verdicts_[k]}")
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
- `pi_[k]` is the selection frequency for band `k`, computed only over the
|
|
210
|
+
resamples in which that band was ASSESSABLE. It is `nan`, never `0`, when
|
|
211
|
+
too few resamples could assess it.
|
|
212
|
+
- `support_[k]` is that denominator: the fraction of resamples that could
|
|
213
|
+
assess the band. **Read `pi_` and `support_` together.** A `pi_` of 0.9 at
|
|
214
|
+
support 0.2 is a much weaker statement than the same value at support 1.0.
|
|
215
|
+
- `verdicts_[k]` is `"selected"`, `"rejected"` or `"unassessable"`. The
|
|
216
|
+
third value is the point of the library: absence of evidence is reported
|
|
217
|
+
as such, not as evidence of absence.
|
|
218
|
+
- `sel.stability_report()` gives the run-level diagnostics in one dict:
|
|
219
|
+
`coverage`, `assessable_universe`, `unassessable_fraction`,
|
|
220
|
+
`ambiguous_fraction` (bands stuck between 0.2 and 0.8, so no threshold
|
|
221
|
+
helps), `n_failed_fits`, `strata` and `degenerate`.
|
|
222
|
+
|
|
223
|
+
`ambiguous_fraction` is the go/no-go number. If it is high, `pi_` is diffuse
|
|
224
|
+
rather than bimodal, and no choice of `tau` will produce a trustworthy set;
|
|
225
|
+
the honest move is to report that rather than to pick a threshold.
|
|
226
|
+
|
|
227
|
+
## The honest verdict: nested cross-validation
|
|
228
|
+
|
|
229
|
+
`BootstrapMaskSelector` tells you which bands are stable. It does not tell
|
|
230
|
+
you what the selection is worth, because it saw all the data. For that, wrap
|
|
231
|
+
it:
|
|
232
|
+
|
|
233
|
+
```python
|
|
234
|
+
from robustsignalmaker import NestedCV
|
|
235
|
+
|
|
236
|
+
result = NestedCV(k_outer=5, segment=16, random_state=0).run(X, y)
|
|
237
|
+
s = result.summary()
|
|
238
|
+
print(s["score_name"], round(s["score_mean"], 4),
|
|
239
|
+
"vs full signal", round(s["baseline_score_mean"], 4))
|
|
240
|
+
print("verdict:", s["verdict"],
|
|
241
|
+
" bands kept per fold:", round(s["mean_n_selected"], 2))
|
|
242
|
+
print("missingness informative:", s["missingness_informative"])
|
|
243
|
+
|
|
244
|
+
result.save("rsm_results") # CSV tables + JSON summary + pickle
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Every statistic, including the selection itself, is refitted inside each
|
|
248
|
+
training fold. Nothing crosses a fold boundary.
|
|
249
|
+
|
|
250
|
+
**Read `preserved` correctly.** It means nothing SIGNIFICANT was lost, not
|
|
251
|
+
that nothing was lost. On several real datasets the full signal is ahead on
|
|
252
|
+
the point estimate and the paired test cannot resolve the difference at five
|
|
253
|
+
folds, which is why both means are printed beside the verdict. The verdict
|
|
254
|
+
values are `preserved`, `sig.better`, `sig.worse` and `undecidable`.
|
|
255
|
+
|
|
256
|
+
If your samples are not independent (repeated measurements of the same
|
|
257
|
+
specimen, several instruments per physical sample), pass groups so a
|
|
258
|
+
specimen is never split across folds:
|
|
259
|
+
|
|
260
|
+
```python
|
|
261
|
+
result = NestedCV(k_outer=5, segment=16, random_state=0).run(X, y, groups=specimen_id)
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
Failing to do this is the most common way to get an optimistic number out of
|
|
265
|
+
spectral data.
|
|
266
|
+
|
|
267
|
+
## Recipes
|
|
268
|
+
|
|
269
|
+
**Choosing the penalty.** Sparsity strength is per-dataset, so it is a
|
|
270
|
+
library feature rather than something to guess:
|
|
271
|
+
|
|
272
|
+
```python
|
|
273
|
+
from robustsignalmaker import lam_frontier, lam_for_coverage
|
|
274
|
+
|
|
275
|
+
def make(lam):
|
|
276
|
+
return BootstrapMaskSelector(task="binary", segment=16, lam=lam,
|
|
277
|
+
n_bootstrap=20, seed=0).fit(X, y)
|
|
278
|
+
|
|
279
|
+
for row in lam_frontier(make, [0.02, 0.05, 0.1, 0.2]):
|
|
280
|
+
print(row) # coverage, stability, ambiguity, support
|
|
281
|
+
|
|
282
|
+
hit = lam_for_coverage(make, target=0.25)
|
|
283
|
+
print(hit["lam"], hit["coverage"], hit["target_reached"])
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
`target_reached` is False when the bracket cannot deliver the coverage you
|
|
287
|
+
asked for. The best point found is still returned, so a miss is visible
|
|
288
|
+
rather than renamed a hit.
|
|
289
|
+
|
|
290
|
+
**Marginalising over lenses.** Which bands look important depends on whether
|
|
291
|
+
you look at raw intensity, a derivative, or a smoothed scale. Treat that
|
|
292
|
+
choice as a nuisance parameter:
|
|
293
|
+
|
|
294
|
+
```python
|
|
295
|
+
from robustsignalmaker import RepresentationEnsembleSelector, default_ensemble
|
|
296
|
+
|
|
297
|
+
ens = RepresentationEnsembleSelector(
|
|
298
|
+
representations=default_ensemble(3, seed=0),
|
|
299
|
+
task="binary", segment=16, n_bootstrap=20, seed=0).fit(X, y)
|
|
300
|
+
|
|
301
|
+
print(ens.selected_segments_)
|
|
302
|
+
print("lens agreement:", round(float(ens.representation_agreement()), 3))
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
Low agreement is informative in itself: it means the answer you would have
|
|
306
|
+
reported depends on a preprocessing choice you might not have thought of as
|
|
307
|
+
a choice.
|
|
308
|
+
|
|
309
|
+
**Point-wise instead of banded.** Set `segment=1`; it is the same code path.
|
|
310
|
+
|
|
311
|
+
## Troubleshooting
|
|
312
|
+
|
|
313
|
+
- **`InsufficientEvidenceError`** means RSM refused rather than guessed.
|
|
314
|
+
Usual causes: too few samples for the effective-sample-size floor
|
|
315
|
+
(`n_min_hard=8`), an evidence threshold set above what the data carries
|
|
316
|
+
(`rho_min`, `o_min`), or sparse data whose natural scale differs from the
|
|
317
|
+
defaults. For sparse spectra such as mass spectrometry, declare the scale:
|
|
318
|
+
`min_valid_frac=0.02, rho_min=0.01`.
|
|
319
|
+
- **All `pi_` are nan.** Nothing was assessable. Check that your validity
|
|
320
|
+
mask has the polarity right (True = observed) and that `o_min` is not
|
|
321
|
+
above your observation rate.
|
|
322
|
+
- **`MissingnessInformativeWarning`.** The missingness pattern alone
|
|
323
|
+
predicts the target, so the observation process is confounded with what
|
|
324
|
+
you are trying to measure. RSM reports this and does not attempt to fix
|
|
325
|
+
it. Do not silence it: it usually means an instrument or batch effect is
|
|
326
|
+
standing in for the label.
|
|
327
|
+
- **A verdict of `undecidable`.** Fewer than two folds were comparable,
|
|
328
|
+
usually because predictions were refused for lack of evidence. The
|
|
329
|
+
refusal count is in the result summary.
|
|
330
|
+
- **Everything looks stable and nothing predicts.** Stability alone never
|
|
331
|
+
ranks methods. A perfectly reproducible wrong answer scores 1.0; always
|
|
332
|
+
read the score beside it.
|
|
333
|
+
|
|
334
|
+
## The RobustMaker family
|
|
335
|
+
|
|
336
|
+
| Package | Selects | Data |
|
|
337
|
+
|---|---|---|
|
|
338
|
+
| RobustModelMaker | columns | tabular features |
|
|
339
|
+
| RobustPixelMaker | patches | scientific images |
|
|
340
|
+
| RobustSignalMaker | points and bands | signals and spectra |
|
|
341
|
+
|
|
342
|
+
## Documentation and examples
|
|
343
|
+
|
|
344
|
+
The repository carries four guides (user, API reference, interpretation,
|
|
345
|
+
implementation),
|
|
346
|
+
six executed example notebooks (NIR spectroscopy, multi-instrument
|
|
347
|
+
calibration, SELDI-TOF mass spectrometry, Raman mineral identification,
|
|
348
|
+
wearable sensors, and gold nanoparticle structure functions), and a
|
|
349
|
+
benchmark suite whose findings, including the negative ones and the
|
|
350
|
+
predictions that were falsified, are recorded in FINDINGS.md.
|
|
351
|
+
|
|
352
|
+
Project home: https://github.com/amaxiom/RobustSignalMaker
|
|
353
|
+
|
|
354
|
+
## Licence
|
|
355
|
+
|
|
356
|
+
MIT. Copyright (c) 2026 Amanda S Barnard.
|
|
@@ -0,0 +1,307 @@
|
|
|
1
|
+
# RobustSignalMaker
|
|
2
|
+
|
|
3
|
+
NaN-aware, leakage-free stability selection for scientific signals and
|
|
4
|
+
spectra. RSM identifies the important parts of a series (time series,
|
|
5
|
+
spectra, diffraction patterns, mass spectra, sensor recordings) to retain
|
|
6
|
+
and removes the rest, returning a reproducible region of the sampling axis
|
|
7
|
+
and an honest estimate of what that region can predict.
|
|
8
|
+
|
|
9
|
+
## Install
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
pip install robustsignalmaker
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Python 3.9 or newer. Three required dependencies, all of which you almost
|
|
16
|
+
certainly already have: `numpy>=1.24`, `scipy>=1.10`, `scikit-learn>=1.3`.
|
|
17
|
+
There is no deep-learning dependency; the gated engine is numpy with
|
|
18
|
+
hand-derived gradients.
|
|
19
|
+
|
|
20
|
+
One optional extra, for writing result tables through pandas rather than
|
|
21
|
+
the built-in csv writer:
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
pip install "robustsignalmaker[tables]"
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Verify the install:
|
|
28
|
+
|
|
29
|
+
```python
|
|
30
|
+
import robustsignalmaker as rsm
|
|
31
|
+
print(rsm.__version__)
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
The package ships a `py.typed` marker, so type checkers will use its
|
|
35
|
+
annotations without a stub package.
|
|
36
|
+
|
|
37
|
+
## Why RSM
|
|
38
|
+
|
|
39
|
+
Most band-selection tools report a single selection and leave you to guess
|
|
40
|
+
how much of it would survive a different sample. RSM's deliverable is the
|
|
41
|
+
reproducible band set and an honest account of what discarding the rest
|
|
42
|
+
costs:
|
|
43
|
+
|
|
44
|
+
- **Leakage-safe stability selection is the product.** Selection is
|
|
45
|
+
aggregated over resamples into stability frequencies
|
|
46
|
+
(Meinshausen-Buhlmann style, with Shah-Samworth complementary pairs
|
|
47
|
+
available), wrapped in nested cross-validation where every fitted
|
|
48
|
+
statistic lives inside the training fold, and compared against a matched
|
|
49
|
+
full-signal baseline with a preserved / significantly-better /
|
|
50
|
+
significantly-worse verdict. On six real datasets every selection
|
|
51
|
+
compresses hard and returns `preserved`: tecator 7 of 20 segments, corn 4
|
|
52
|
+
of 35, ovarian SELDI-TOF 6 of 64, RRUFF Raman 8.5 of 64 per fold,
|
|
53
|
+
BasicMotions 2.75 of 10, and four of eighteen bond-angle bands on a gold
|
|
54
|
+
nanoparticle dataset.
|
|
55
|
+
|
|
56
|
+
Real instrument data is where that gets hard, because a selection frequency
|
|
57
|
+
is not well defined when different samples observe different parts of the
|
|
58
|
+
axis. The rest of the machinery exists to make it well defined:
|
|
59
|
+
|
|
60
|
+
- **No value is ever fabricated.** Every pooling, lens, and model operation
|
|
61
|
+
is renormalised over the evidence that actually exists (validity masks
|
|
62
|
+
travel with the data end to end). Detector dropout, saturation,
|
|
63
|
+
instrument-range gaps and not-detected zeros are absence, not zeros.
|
|
64
|
+
- **Three-valued verdicts.** Every candidate band ends as selected,
|
|
65
|
+
rejected, or UNASSESSABLE. A region nobody measured can never read as
|
|
66
|
+
"stably rejected", and every selection frequency is reported beside its
|
|
67
|
+
observation support.
|
|
68
|
+
- **Honesty guards.** Samples with too little evidence are refused, not
|
|
69
|
+
guessed. If the missingness pattern alone predicts the target (a
|
|
70
|
+
non-ignorable, instrument-linked confound), RSM warns and stamps the
|
|
71
|
+
result; imputation-first pipelines destroy exactly that alarm.
|
|
72
|
+
- **Your choice of predictor.** Selection is done by the gated head, but the
|
|
73
|
+
final model on the chosen bands is yours: `refit_model=` takes `lin`,
|
|
74
|
+
`rdg`, `las`, `eln`, `log`, `svm`, `rf`, `xgb`, `mlp`, or `pls` for PLS
|
|
75
|
+
regression and PLS-DA. Swapping it never changes the selection, and the
|
|
76
|
+
refusal rule survives: a random forest refuses the same thin samples the
|
|
77
|
+
head refuses rather than predicting from imputed values.
|
|
78
|
+
- **Lens marginalisation.** Band importance depends on how a signal is
|
|
79
|
+
represented (raw intensity, derivatives, smoothed scales). RSM can
|
|
80
|
+
marginalise the selection over a lens ensemble and reports how much the
|
|
81
|
+
lenses agree.
|
|
82
|
+
|
|
83
|
+
## Input contract
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
X : float array (n_samples, n_channels, n_points)
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
All samples share one sampling axis of length `n_points`. A single-channel
|
|
90
|
+
spectrum is `n_channels=1`, so reshape a plain `(n_samples, n_points)`
|
|
91
|
+
matrix with `X[:, None, :]`.
|
|
92
|
+
|
|
93
|
+
Missingness is declared in one of two ways, and they are equivalent:
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
import numpy as np
|
|
97
|
+
|
|
98
|
+
# 1. NaN in X means "not measured"
|
|
99
|
+
X = np.random.default_rng(0).normal(size=(20, 1, 64))
|
|
100
|
+
X[3, 0, 10:14] = np.nan
|
|
101
|
+
|
|
102
|
+
# 2. or pass an explicit boolean validity mask, True = observed
|
|
103
|
+
V = np.isfinite(X)
|
|
104
|
+
V[5, 0, 20:24] = False # censored, though a value is present
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Pass the mask as the third argument to `fit`, or leave it out and let the
|
|
108
|
+
NaNs speak: `sel.fit(X, y)` and `sel.fit(X, y, V)` are the same call.
|
|
109
|
+
|
|
110
|
+
Use the explicit mask when a missing value is not representable as NaN, for
|
|
111
|
+
example a mass-spectrometry zero that means "not detected" rather than
|
|
112
|
+
"intensity zero", or a saturated detector reading you want treated as
|
|
113
|
+
censored.
|
|
114
|
+
|
|
115
|
+
`y` may be regression, binary or multiclass, and may contain NaN for
|
|
116
|
+
unlabelled samples: they are excluded from every split, loss, resample and
|
|
117
|
+
score, and counted in the result rather than silently absorbed.
|
|
118
|
+
|
|
119
|
+
Multi-channel series get one gate per band shared across channels, with
|
|
120
|
+
per-channel missingness handled underneath the gate.
|
|
121
|
+
|
|
122
|
+
## Quick start
|
|
123
|
+
|
|
124
|
+
This example is self-contained and runs as written.
|
|
125
|
+
|
|
126
|
+
```python
|
|
127
|
+
import numpy as np
|
|
128
|
+
from robustsignalmaker import BootstrapMaskSelector, make_signal_control
|
|
129
|
+
|
|
130
|
+
# a control with a known informative band, so the answer is checkable
|
|
131
|
+
c = make_signal_control(n=200, n_points=256, task="binary", seed=0)
|
|
132
|
+
X, y = c.X, c.y # X is (200, 1, 256)
|
|
133
|
+
|
|
134
|
+
X = X.copy()
|
|
135
|
+
X[:80, 0, 180:210] = np.nan # 80 samples never saw this stretch
|
|
136
|
+
|
|
137
|
+
sel = BootstrapMaskSelector(task="binary", segment=16,
|
|
138
|
+
n_bootstrap=30, seed=0).fit(X, y)
|
|
139
|
+
|
|
140
|
+
print("kept bands :", sel.selected_segments_)
|
|
141
|
+
print("point indices :", sel.selected_points()[:12], "...")
|
|
142
|
+
print("coverage :", round(sel.coverage(), 3))
|
|
143
|
+
truth = sorted(set(np.flatnonzero(c.informative) // 16))
|
|
144
|
+
print("truth was :", truth) # the planted band, for comparison
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
## Reading the output
|
|
148
|
+
|
|
149
|
+
This is the part worth five minutes, because a selection frequency without
|
|
150
|
+
its support is the thing RSM exists to stop you reporting.
|
|
151
|
+
|
|
152
|
+
```python
|
|
153
|
+
import numpy as np
|
|
154
|
+
|
|
155
|
+
for k in range(sel.grid.n_segments):
|
|
156
|
+
print(f"band {k:3d} pi={sel.pi_[k]!s:>6} "
|
|
157
|
+
f"support={sel.support_[k]:.2f} {sel.verdicts_[k]}")
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
- `pi_[k]` is the selection frequency for band `k`, computed only over the
|
|
161
|
+
resamples in which that band was ASSESSABLE. It is `nan`, never `0`, when
|
|
162
|
+
too few resamples could assess it.
|
|
163
|
+
- `support_[k]` is that denominator: the fraction of resamples that could
|
|
164
|
+
assess the band. **Read `pi_` and `support_` together.** A `pi_` of 0.9 at
|
|
165
|
+
support 0.2 is a much weaker statement than the same value at support 1.0.
|
|
166
|
+
- `verdicts_[k]` is `"selected"`, `"rejected"` or `"unassessable"`. The
|
|
167
|
+
third value is the point of the library: absence of evidence is reported
|
|
168
|
+
as such, not as evidence of absence.
|
|
169
|
+
- `sel.stability_report()` gives the run-level diagnostics in one dict:
|
|
170
|
+
`coverage`, `assessable_universe`, `unassessable_fraction`,
|
|
171
|
+
`ambiguous_fraction` (bands stuck between 0.2 and 0.8, so no threshold
|
|
172
|
+
helps), `n_failed_fits`, `strata` and `degenerate`.
|
|
173
|
+
|
|
174
|
+
`ambiguous_fraction` is the go/no-go number. If it is high, `pi_` is diffuse
|
|
175
|
+
rather than bimodal, and no choice of `tau` will produce a trustworthy set;
|
|
176
|
+
the honest move is to report that rather than to pick a threshold.
|
|
177
|
+
|
|
178
|
+
## The honest verdict: nested cross-validation
|
|
179
|
+
|
|
180
|
+
`BootstrapMaskSelector` tells you which bands are stable. It does not tell
|
|
181
|
+
you what the selection is worth, because it saw all the data. For that, wrap
|
|
182
|
+
it:
|
|
183
|
+
|
|
184
|
+
```python
|
|
185
|
+
from robustsignalmaker import NestedCV
|
|
186
|
+
|
|
187
|
+
result = NestedCV(k_outer=5, segment=16, random_state=0).run(X, y)
|
|
188
|
+
s = result.summary()
|
|
189
|
+
print(s["score_name"], round(s["score_mean"], 4),
|
|
190
|
+
"vs full signal", round(s["baseline_score_mean"], 4))
|
|
191
|
+
print("verdict:", s["verdict"],
|
|
192
|
+
" bands kept per fold:", round(s["mean_n_selected"], 2))
|
|
193
|
+
print("missingness informative:", s["missingness_informative"])
|
|
194
|
+
|
|
195
|
+
result.save("rsm_results") # CSV tables + JSON summary + pickle
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Every statistic, including the selection itself, is refitted inside each
|
|
199
|
+
training fold. Nothing crosses a fold boundary.
|
|
200
|
+
|
|
201
|
+
**Read `preserved` correctly.** It means nothing SIGNIFICANT was lost, not
|
|
202
|
+
that nothing was lost. On several real datasets the full signal is ahead on
|
|
203
|
+
the point estimate and the paired test cannot resolve the difference at five
|
|
204
|
+
folds, which is why both means are printed beside the verdict. The verdict
|
|
205
|
+
values are `preserved`, `sig.better`, `sig.worse` and `undecidable`.
|
|
206
|
+
|
|
207
|
+
If your samples are not independent (repeated measurements of the same
|
|
208
|
+
specimen, several instruments per physical sample), pass groups so a
|
|
209
|
+
specimen is never split across folds:
|
|
210
|
+
|
|
211
|
+
```python
|
|
212
|
+
result = NestedCV(k_outer=5, segment=16, random_state=0).run(X, y, groups=specimen_id)
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Failing to do this is the most common way to get an optimistic number out of
|
|
216
|
+
spectral data.
|
|
217
|
+
|
|
218
|
+
## Recipes
|
|
219
|
+
|
|
220
|
+
**Choosing the penalty.** Sparsity strength is per-dataset, so it is a
|
|
221
|
+
library feature rather than something to guess:
|
|
222
|
+
|
|
223
|
+
```python
|
|
224
|
+
from robustsignalmaker import lam_frontier, lam_for_coverage
|
|
225
|
+
|
|
226
|
+
def make(lam):
|
|
227
|
+
return BootstrapMaskSelector(task="binary", segment=16, lam=lam,
|
|
228
|
+
n_bootstrap=20, seed=0).fit(X, y)
|
|
229
|
+
|
|
230
|
+
for row in lam_frontier(make, [0.02, 0.05, 0.1, 0.2]):
|
|
231
|
+
print(row) # coverage, stability, ambiguity, support
|
|
232
|
+
|
|
233
|
+
hit = lam_for_coverage(make, target=0.25)
|
|
234
|
+
print(hit["lam"], hit["coverage"], hit["target_reached"])
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
`target_reached` is False when the bracket cannot deliver the coverage you
|
|
238
|
+
asked for. The best point found is still returned, so a miss is visible
|
|
239
|
+
rather than renamed a hit.
|
|
240
|
+
|
|
241
|
+
**Marginalising over lenses.** Which bands look important depends on whether
|
|
242
|
+
you look at raw intensity, a derivative, or a smoothed scale. Treat that
|
|
243
|
+
choice as a nuisance parameter:
|
|
244
|
+
|
|
245
|
+
```python
|
|
246
|
+
from robustsignalmaker import RepresentationEnsembleSelector, default_ensemble
|
|
247
|
+
|
|
248
|
+
ens = RepresentationEnsembleSelector(
|
|
249
|
+
representations=default_ensemble(3, seed=0),
|
|
250
|
+
task="binary", segment=16, n_bootstrap=20, seed=0).fit(X, y)
|
|
251
|
+
|
|
252
|
+
print(ens.selected_segments_)
|
|
253
|
+
print("lens agreement:", round(float(ens.representation_agreement()), 3))
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Low agreement is informative in itself: it means the answer you would have
|
|
257
|
+
reported depends on a preprocessing choice you might not have thought of as
|
|
258
|
+
a choice.
|
|
259
|
+
|
|
260
|
+
**Point-wise instead of banded.** Set `segment=1`; it is the same code path.
|
|
261
|
+
|
|
262
|
+
## Troubleshooting
|
|
263
|
+
|
|
264
|
+
- **`InsufficientEvidenceError`** means RSM refused rather than guessed.
|
|
265
|
+
Usual causes: too few samples for the effective-sample-size floor
|
|
266
|
+
(`n_min_hard=8`), an evidence threshold set above what the data carries
|
|
267
|
+
(`rho_min`, `o_min`), or sparse data whose natural scale differs from the
|
|
268
|
+
defaults. For sparse spectra such as mass spectrometry, declare the scale:
|
|
269
|
+
`min_valid_frac=0.02, rho_min=0.01`.
|
|
270
|
+
- **All `pi_` are nan.** Nothing was assessable. Check that your validity
|
|
271
|
+
mask has the polarity right (True = observed) and that `o_min` is not
|
|
272
|
+
above your observation rate.
|
|
273
|
+
- **`MissingnessInformativeWarning`.** The missingness pattern alone
|
|
274
|
+
predicts the target, so the observation process is confounded with what
|
|
275
|
+
you are trying to measure. RSM reports this and does not attempt to fix
|
|
276
|
+
it. Do not silence it: it usually means an instrument or batch effect is
|
|
277
|
+
standing in for the label.
|
|
278
|
+
- **A verdict of `undecidable`.** Fewer than two folds were comparable,
|
|
279
|
+
usually because predictions were refused for lack of evidence. The
|
|
280
|
+
refusal count is in the result summary.
|
|
281
|
+
- **Everything looks stable and nothing predicts.** Stability alone never
|
|
282
|
+
ranks methods. A perfectly reproducible wrong answer scores 1.0; always
|
|
283
|
+
read the score beside it.
|
|
284
|
+
|
|
285
|
+
## The RobustMaker family
|
|
286
|
+
|
|
287
|
+
| Package | Selects | Data |
|
|
288
|
+
|---|---|---|
|
|
289
|
+
| RobustModelMaker | columns | tabular features |
|
|
290
|
+
| RobustPixelMaker | patches | scientific images |
|
|
291
|
+
| RobustSignalMaker | points and bands | signals and spectra |
|
|
292
|
+
|
|
293
|
+
## Documentation and examples
|
|
294
|
+
|
|
295
|
+
The repository carries four guides (user, API reference, interpretation,
|
|
296
|
+
implementation),
|
|
297
|
+
six executed example notebooks (NIR spectroscopy, multi-instrument
|
|
298
|
+
calibration, SELDI-TOF mass spectrometry, Raman mineral identification,
|
|
299
|
+
wearable sensors, and gold nanoparticle structure functions), and a
|
|
300
|
+
benchmark suite whose findings, including the negative ones and the
|
|
301
|
+
predictions that were falsified, are recorded in FINDINGS.md.
|
|
302
|
+
|
|
303
|
+
Project home: https://github.com/amaxiom/RobustSignalMaker
|
|
304
|
+
|
|
305
|
+
## Licence
|
|
306
|
+
|
|
307
|
+
MIT. Copyright (c) 2026 Amanda S Barnard.
|