ipss 1.1.12__tar.gz → 1.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ipss-1.1.12/ipss.egg-info → ipss-1.2.0}/PKG-INFO +9 -10
- {ipss-1.1.12 → ipss-1.2.0}/README.md +8 -9
- {ipss-1.1.12 → ipss-1.2.0}/ipss/base_selectors/__init__.py +2 -0
- {ipss-1.1.12 → ipss-1.2.0}/ipss/base_selectors/gb.py +3 -1
- {ipss-1.1.12 → ipss-1.2.0}/ipss/base_selectors/linear.py +18 -19
- {ipss-1.1.12 → ipss-1.2.0}/ipss/base_selectors/rf.py +9 -5
- {ipss-1.1.12 → ipss-1.2.0}/ipss/helpers.py +45 -12
- {ipss-1.1.12 → ipss-1.2.0}/ipss/main.py +40 -22
- {ipss-1.1.12 → ipss-1.2.0}/ipss/preselection.py +8 -6
- {ipss-1.1.12 → ipss-1.2.0}/ipss/processing.py +33 -20
- {ipss-1.1.12 → ipss-1.2.0/ipss.egg-info}/PKG-INFO +9 -10
- {ipss-1.1.12 → ipss-1.2.0}/ipss.egg-info/SOURCES.txt +0 -1
- {ipss-1.1.12 → ipss-1.2.0}/pyproject.toml +1 -1
- ipss-1.1.12/MANIFEST.in +0 -1
- {ipss-1.1.12 → ipss-1.2.0}/LICENSE +0 -0
- {ipss-1.1.12 → ipss-1.2.0}/ipss/__init__.py +0 -0
- {ipss-1.1.12 → ipss-1.2.0}/ipss/base_selectors/ufi.py +0 -0
- {ipss-1.1.12 → ipss-1.2.0}/ipss.egg-info/dependency_links.txt +0 -0
- {ipss-1.1.12 → ipss-1.2.0}/ipss.egg-info/requires.txt +0 -0
- {ipss-1.1.12 → ipss-1.2.0}/ipss.egg-info/top_level.txt +0 -0
- {ipss-1.1.12 → ipss-1.2.0}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: ipss
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.2.0
|
|
4
4
|
Summary: Python implementation of integrated path stability selection (IPSS)
|
|
5
5
|
Author-email: Omar Melikechi <omar.melikechi@gmail.com>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -23,16 +23,14 @@ Dynamic: license-file
|
|
|
23
23
|
|
|
24
24
|
> Fast, flexible feature selection with false discovery control
|
|
25
25
|
|
|
26
|
-
##
|
|
26
|
+
## Papers
|
|
27
27
|
|
|
28
28
|
- **Integrated path stability selection**
|
|
29
29
|
[*Journal of the American Statistical Association*](https://doi.org/10.1080/01621459.2025.2525589) • [arXiv](https://arxiv.org/abs/2403.15877)
|
|
30
30
|
|
|
31
|
-
- **Nonparametric IPSS
|
|
31
|
+
- **Nonparametric IPSS**
|
|
32
32
|
[*Bioinformatics*](https://doi.org/10.1093/bioinformatics/btaf299) • [arXiv](https://arxiv.org/abs/2410.02208)
|
|
33
33
|
|
|
34
|
-
> "*Integrated path stability selection*" introduces IPSS and applies it to regularized models such as lasso and adaptive lasso. "*Nonparametric IPSS: Fast, flexible feature selection with false discovery control*" extends IPSS to arbitrary feature importance scores, such as those from gradient boosting and random forests.
|
|
35
|
-
|
|
36
34
|
## Installation
|
|
37
35
|
```
|
|
38
36
|
pip install ipss
|
|
@@ -51,6 +49,7 @@ ipss_output = ipss(X,y)
|
|
|
51
49
|
target_fdr = 0.1
|
|
52
50
|
q_values = ipss_output['q_values']
|
|
53
51
|
selected_features = [idx for idx, q_value in q_values.items() if q_value <= target_fdr]
|
|
52
|
+
|
|
54
53
|
print(f'Selected features (target FDR = {target_fdr}): {selected_features}')
|
|
55
54
|
```
|
|
56
55
|
### Outputs
|
|
@@ -59,7 +58,7 @@ print(f'Selected features (target FDR = {target_fdr}): {selected_features}')
|
|
|
59
58
|
- `q_values`: Dictionary whose keys are feature indices and values are their q-values (dict of length `p`).
|
|
60
59
|
- `runtime`: Runtime of the algorithm in seconds (float).
|
|
61
60
|
- `selected_features`: Indices of features selected by IPSS; empty list if `target_fp` and `target_fdr` are not specified (list of ints).
|
|
62
|
-
- `stability_paths`: Estimated selection probabilities at each parameter
|
|
61
|
+
- `stability_paths`: Estimated selection probabilities at each parameter (array of shape `(n_alphas, p)`).
|
|
63
62
|
|
|
64
63
|
### Selecting features
|
|
65
64
|
Each feature (column of `X`) is assigned:
|
|
@@ -125,7 +124,7 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
125
124
|
- Users can provide their own feature importance function (see example above).
|
|
126
125
|
- `selector_args`: Arguments for the base algorithm (dict; default `None`).
|
|
127
126
|
- `preselect`: Preselect/filter features prior to subsampling (bool; default `True`).
|
|
128
|
-
- `preselector`: Method used to preselect features (default `None`, which resolves to `'dcor'` for every `selector` except `'l1'`/`'adaptive_lasso'`, which use their own linear preselection
|
|
127
|
+
- `preselector`: Method used to preselect features (default `None`, which resolves to `'dcor'` for every `selector` except `'l1'`/`'adaptive_lasso'`, which use their own linear preselection). Options:
|
|
129
128
|
- `'gb'`: Gradient boosting (XGBoost) feature importances.
|
|
130
129
|
- `'rf'`: Random forest (scikit-learn) feature importances.
|
|
131
130
|
- `'l1'` or `'adaptive_lasso'`: (Adaptive) lasso/logistic regression coefficients.
|
|
@@ -146,9 +145,10 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
146
145
|
- `'h3'`: Cubic function, ```h3(x) = (2x - 1)**3 if x >= 0.5 else 0```.
|
|
147
146
|
- `cutoff`: Maximum value of the theoretical integral bound `I(Lambda)` (positive float; default `0.05`).
|
|
148
147
|
- `delta`: Defines probability measure; see `Associated papers` (float; defaults depend on `selector`).
|
|
149
|
-
- `standardize_X`: Scale features to have mean 0, standard deviation 1 (bool; default `None`).
|
|
150
|
-
- `center_y`: Center response to have mean 0 (bool; default `None`).
|
|
148
|
+
- `standardize_X`: Scale features to have mean 0, standard deviation 1 (bool; default `None`, which standardizes for `'l1'` and `'adaptive_lasso'` only).
|
|
149
|
+
- `center_y`: Center response to have mean 0 (bool; default `None`, which centers for `'l1'` and `'adaptive_lasso'` only). Binary responses are never centered.
|
|
151
150
|
- `n_jobs`: Number of jobs to run in parallel (int; default `1`).
|
|
151
|
+
- `random_state`: Seed for preselection, subsampling, and the `'gb'`, `'rf'`, and `'ufi'` selectors (int or `None`; default `None`, which uses NumPy's global random state). Each selector fit gets its own seed; `selector_args['random_state']` is ignored.
|
|
152
152
|
|
|
153
153
|
### General observations/recommendations:
|
|
154
154
|
- `selector = 'gb'` often best for capturing nonlinear relationships.
|
|
@@ -164,4 +164,3 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
164
164
|
- Results are robust to `cutoff` provided it is between `0.025` and `0.1`.
|
|
165
165
|
- Features are automatically standardized for the penalized regression methods.
|
|
166
166
|
- The response is automatically centered for the regularized regression methods.
|
|
167
|
-
|
|
@@ -2,16 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
> Fast, flexible feature selection with false discovery control
|
|
4
4
|
|
|
5
|
-
##
|
|
5
|
+
## Papers
|
|
6
6
|
|
|
7
7
|
- **Integrated path stability selection**
|
|
8
8
|
[*Journal of the American Statistical Association*](https://doi.org/10.1080/01621459.2025.2525589) • [arXiv](https://arxiv.org/abs/2403.15877)
|
|
9
9
|
|
|
10
|
-
- **Nonparametric IPSS
|
|
10
|
+
- **Nonparametric IPSS**
|
|
11
11
|
[*Bioinformatics*](https://doi.org/10.1093/bioinformatics/btaf299) • [arXiv](https://arxiv.org/abs/2410.02208)
|
|
12
12
|
|
|
13
|
-
> "*Integrated path stability selection*" introduces IPSS and applies it to regularized models such as lasso and adaptive lasso. "*Nonparametric IPSS: Fast, flexible feature selection with false discovery control*" extends IPSS to arbitrary feature importance scores, such as those from gradient boosting and random forests.
|
|
14
|
-
|
|
15
13
|
## Installation
|
|
16
14
|
```
|
|
17
15
|
pip install ipss
|
|
@@ -30,6 +28,7 @@ ipss_output = ipss(X,y)
|
|
|
30
28
|
target_fdr = 0.1
|
|
31
29
|
q_values = ipss_output['q_values']
|
|
32
30
|
selected_features = [idx for idx, q_value in q_values.items() if q_value <= target_fdr]
|
|
31
|
+
|
|
33
32
|
print(f'Selected features (target FDR = {target_fdr}): {selected_features}')
|
|
34
33
|
```
|
|
35
34
|
### Outputs
|
|
@@ -38,7 +37,7 @@ print(f'Selected features (target FDR = {target_fdr}): {selected_features}')
|
|
|
38
37
|
- `q_values`: Dictionary whose keys are feature indices and values are their q-values (dict of length `p`).
|
|
39
38
|
- `runtime`: Runtime of the algorithm in seconds (float).
|
|
40
39
|
- `selected_features`: Indices of features selected by IPSS; empty list if `target_fp` and `target_fdr` are not specified (list of ints).
|
|
41
|
-
- `stability_paths`: Estimated selection probabilities at each parameter
|
|
40
|
+
- `stability_paths`: Estimated selection probabilities at each parameter (array of shape `(n_alphas, p)`).
|
|
42
41
|
|
|
43
42
|
### Selecting features
|
|
44
43
|
Each feature (column of `X`) is assigned:
|
|
@@ -104,7 +103,7 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
104
103
|
- Users can provide their own feature importance function (see example above).
|
|
105
104
|
- `selector_args`: Arguments for the base algorithm (dict; default `None`).
|
|
106
105
|
- `preselect`: Preselect/filter features prior to subsampling (bool; default `True`).
|
|
107
|
-
- `preselector`: Method used to preselect features (default `None`, which resolves to `'dcor'` for every `selector` except `'l1'`/`'adaptive_lasso'`, which use their own linear preselection
|
|
106
|
+
- `preselector`: Method used to preselect features (default `None`, which resolves to `'dcor'` for every `selector` except `'l1'`/`'adaptive_lasso'`, which use their own linear preselection). Options:
|
|
108
107
|
- `'gb'`: Gradient boosting (XGBoost) feature importances.
|
|
109
108
|
- `'rf'`: Random forest (scikit-learn) feature importances.
|
|
110
109
|
- `'l1'` or `'adaptive_lasso'`: (Adaptive) lasso/logistic regression coefficients.
|
|
@@ -125,9 +124,10 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
125
124
|
- `'h3'`: Cubic function, ```h3(x) = (2x - 1)**3 if x >= 0.5 else 0```.
|
|
126
125
|
- `cutoff`: Maximum value of the theoretical integral bound `I(Lambda)` (positive float; default `0.05`).
|
|
127
126
|
- `delta`: Defines probability measure; see `Associated papers` (float; defaults depend on `selector`).
|
|
128
|
-
- `standardize_X`: Scale features to have mean 0, standard deviation 1 (bool; default `None`).
|
|
129
|
-
- `center_y`: Center response to have mean 0 (bool; default `None`).
|
|
127
|
+
- `standardize_X`: Scale features to have mean 0, standard deviation 1 (bool; default `None`, which standardizes for `'l1'` and `'adaptive_lasso'` only).
|
|
128
|
+
- `center_y`: Center response to have mean 0 (bool; default `None`, which centers for `'l1'` and `'adaptive_lasso'` only). Binary responses are never centered.
|
|
130
129
|
- `n_jobs`: Number of jobs to run in parallel (int; default `1`).
|
|
130
|
+
- `random_state`: Seed for preselection, subsampling, and the `'gb'`, `'rf'`, and `'ufi'` selectors (int or `None`; default `None`, which uses NumPy's global random state). Each selector fit gets its own seed; `selector_args['random_state']` is ignored.
|
|
131
131
|
|
|
132
132
|
### General observations/recommendations:
|
|
133
133
|
- `selector = 'gb'` often best for capturing nonlinear relationships.
|
|
@@ -143,4 +143,3 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
143
143
|
- Results are robust to `cutoff` provided it is between `0.025` and `0.1`.
|
|
144
144
|
- Features are automatically standardized for the penalized regression methods.
|
|
145
145
|
- The response is automatically centered for the regularized regression methods.
|
|
146
|
-
|
|
@@ -5,6 +5,7 @@ Update (version 1.1.7): SCAD and MCP temporarily disabled due to skglm dependenc
|
|
|
5
5
|
"""
|
|
6
6
|
|
|
7
7
|
from .linear import (
|
|
8
|
+
adaptive_weights,
|
|
8
9
|
fit_l1_regressor,
|
|
9
10
|
fit_l1_classifier,
|
|
10
11
|
fit_adaptive_lasso_classifier,
|
|
@@ -28,6 +29,7 @@ from .ufi import (
|
|
|
28
29
|
|
|
29
30
|
__all__ = [
|
|
30
31
|
# linear
|
|
32
|
+
"adaptive_weights",
|
|
31
33
|
"fit_l1_regressor",
|
|
32
34
|
"fit_l1_classifier",
|
|
33
35
|
"fit_adaptive_lasso_classifier",
|
|
@@ -5,6 +5,7 @@ import xgboost as xgb
|
|
|
5
5
|
|
|
6
6
|
# gradient boosting classifier
|
|
7
7
|
def fit_gb_classifier(X, y, **kwargs):
|
|
8
|
+
kwargs.setdefault('random_state', np.random.randint(np.iinfo(np.int32).max))
|
|
8
9
|
model = xgb.XGBClassifier(**kwargs)
|
|
9
10
|
model.fit(X,y)
|
|
10
11
|
feature_importances = model.feature_importances_
|
|
@@ -12,7 +13,8 @@ def fit_gb_classifier(X, y, **kwargs):
|
|
|
12
13
|
|
|
13
14
|
# gradient boosting regressor
|
|
14
15
|
def fit_gb_regressor(X, y, **kwargs):
|
|
16
|
+
kwargs.setdefault('random_state', np.random.randint(np.iinfo(np.int32).max))
|
|
15
17
|
model = xgb.XGBRegressor(**kwargs)
|
|
16
18
|
model.fit(X,y)
|
|
17
19
|
feature_importances = model.feature_importances_
|
|
18
|
-
return feature_importances
|
|
20
|
+
return feature_importances
|
|
@@ -10,23 +10,29 @@ import numpy as np
|
|
|
10
10
|
# from skglm.penalties import SCAD
|
|
11
11
|
from sklearn.linear_model import Lasso, lasso_path, LogisticRegression, Ridge, RidgeClassifier
|
|
12
12
|
|
|
13
|
+
# adaptive lasso weights from an initial l2-penalized fit (logistic regression if binary, ridge otherwise)
|
|
14
|
+
def adaptive_weights(X, y, binary_response, epsilon=1e-6):
|
|
15
|
+
with warnings.catch_warnings():
|
|
16
|
+
warnings.simplefilter('ignore')
|
|
17
|
+
if binary_response:
|
|
18
|
+
init_model = LogisticRegression(penalty='l2', solver='liblinear', max_iter=1000)
|
|
19
|
+
else:
|
|
20
|
+
init_model = Ridge()
|
|
21
|
+
init_model.fit(X, y)
|
|
22
|
+
init_coef = init_model.coef_.flatten()
|
|
23
|
+
return 1 / (np.abs(init_coef) + epsilon)
|
|
24
|
+
|
|
13
25
|
# adaptive lasso classifier
|
|
14
26
|
def fit_adaptive_lasso_classifier(X, y, alphas, epsilon=1e-6):
|
|
15
27
|
n_alphas = len(alphas)
|
|
16
28
|
n_features = X.shape[1]
|
|
17
29
|
coefficients = np.zeros((n_alphas, n_features))
|
|
18
30
|
|
|
19
|
-
#
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
init_model = LogisticRegression(penalty='l2', solver='liblinear', max_iter=1000)
|
|
23
|
-
init_model.fit(X, y)
|
|
24
|
-
init_coef = init_model.coef_.flatten()
|
|
25
|
-
|
|
26
|
-
weights = 1 / (np.abs(init_coef) + epsilon)
|
|
31
|
+
# compute adaptive weights and reweight features
|
|
32
|
+
weights = adaptive_weights(X, y, binary_response=True, epsilon=epsilon)
|
|
33
|
+
X_weighted = X / weights[np.newaxis, :]
|
|
27
34
|
|
|
28
35
|
for i, alpha in enumerate(alphas):
|
|
29
|
-
X_weighted = X / weights[np.newaxis, :]
|
|
30
36
|
model = LogisticRegression(penalty='l1', C=1/alpha, solver='liblinear', max_iter=1000)
|
|
31
37
|
with warnings.catch_warnings():
|
|
32
38
|
warnings.simplefilter('ignore')
|
|
@@ -41,19 +47,12 @@ def fit_adaptive_lasso_regressor(X, y, alphas, epsilon=1e-6):
|
|
|
41
47
|
n_features = X.shape[1]
|
|
42
48
|
coefficients = np.zeros((n_alphas, n_features))
|
|
43
49
|
|
|
44
|
-
#
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
init_ridge = Ridge()
|
|
48
|
-
init_ridge.fit(X,y)
|
|
49
|
-
init_coef = init_ridge.coef_
|
|
50
|
-
|
|
51
|
-
# compute adaptive weights
|
|
52
|
-
weights = 1 / (np.abs(init_coef) + epsilon)
|
|
50
|
+
# compute adaptive weights and reweight features
|
|
51
|
+
weights = adaptive_weights(X, y, binary_response=False, epsilon=epsilon)
|
|
52
|
+
X_weighted = X / weights[np.newaxis, :]
|
|
53
53
|
|
|
54
54
|
# solve weighted lasso for each alpha
|
|
55
55
|
for i, alpha in enumerate(alphas):
|
|
56
|
-
X_weighted = X / weights[np.newaxis, :]
|
|
57
56
|
lasso = Lasso(alpha=alpha)
|
|
58
57
|
with warnings.catch_warnings():
|
|
59
58
|
warnings.simplefilter('ignore')
|
|
@@ -4,6 +4,7 @@ import numpy as np
|
|
|
4
4
|
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
|
|
5
5
|
from sklearn.inspection import permutation_importance
|
|
6
6
|
from sklearn.model_selection import train_test_split
|
|
7
|
+
from sklearn.utils import check_random_state
|
|
7
8
|
|
|
8
9
|
# random forest classifier
|
|
9
10
|
def fit_rf_classifier(X, y, **kwargs):
|
|
@@ -13,8 +14,9 @@ def fit_rf_classifier(X, y, **kwargs):
|
|
|
13
14
|
if importance_type == 'mdi':
|
|
14
15
|
feature_importances = model.feature_importances_
|
|
15
16
|
elif importance_type == 'permutation':
|
|
16
|
-
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)
|
|
17
|
-
|
|
17
|
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=kwargs.get('random_state'))
|
|
18
|
+
model.fit(X_train, y_train)
|
|
19
|
+
perm_importance = permutation_importance(model, X_test, y_test, n_repeats=1, random_state=kwargs.get('random_state'))
|
|
18
20
|
feature_importances = perm_importance.importances_mean
|
|
19
21
|
else:
|
|
20
22
|
raise ValueError("importance_type must be either 'mdi' or 'permutation'")
|
|
@@ -28,14 +30,16 @@ def fit_rf_regressor(X, y, **kwargs):
|
|
|
28
30
|
if importance_type == 'mdi':
|
|
29
31
|
feature_importances = model.feature_importances_
|
|
30
32
|
elif importance_type == 'permutation':
|
|
31
|
-
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
|
|
32
|
-
|
|
33
|
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=kwargs.get('random_state'))
|
|
34
|
+
model.fit(X_train, y_train)
|
|
35
|
+
perm_importance = permutation_importance(model, X_test, y_test, n_repeats=1, random_state=kwargs.get('random_state'))
|
|
33
36
|
feature_importances = perm_importance.importances_mean
|
|
34
37
|
elif importance_type == 'shadow':
|
|
35
38
|
n, p = X.shape
|
|
36
39
|
X_shadow = X.copy()
|
|
40
|
+
rng = check_random_state(kwargs.get('random_state'))
|
|
37
41
|
for i in range(p):
|
|
38
|
-
|
|
42
|
+
rng.shuffle(X_shadow[:,i])
|
|
39
43
|
X_combined = np.hstack((X, X_shadow))
|
|
40
44
|
model.fit(X_combined, y)
|
|
41
45
|
importances_combined = model.feature_importances_
|
|
@@ -58,13 +58,12 @@ def resolve_selector(selector, binary_response):
|
|
|
58
58
|
return selector
|
|
59
59
|
|
|
60
60
|
def resolve_preselector(preselector, selector, binary_response):
|
|
61
|
-
# default: dcor for every selector except (adaptive) lasso, which keeps its own linear
|
|
62
|
-
# preselection; custom (callable) selectors default to reusing themselves as the preselector
|
|
61
|
+
# default: dcor for every selector except (adaptive) lasso, which keeps its own linear preselection
|
|
63
62
|
if preselector is None:
|
|
64
63
|
regularization_selectors = ('lasso', 'logistic_regression', 'adaptive_lasso_classifier', 'adaptive_lasso_regressor')
|
|
65
|
-
if
|
|
66
|
-
return
|
|
67
|
-
return
|
|
64
|
+
if selector in regularization_selectors:
|
|
65
|
+
return selector
|
|
66
|
+
return 'dcor'
|
|
68
67
|
if not isinstance(preselector, str):
|
|
69
68
|
return preselector
|
|
70
69
|
if preselector == 'adaptive_lasso':
|
|
@@ -80,11 +79,7 @@ def resolve_preselector(preselector, selector, binary_response):
|
|
|
80
79
|
return preselector
|
|
81
80
|
|
|
82
81
|
def check_response_type(y, selector):
|
|
83
|
-
|
|
84
|
-
if len(unique_values) == 1:
|
|
85
|
-
print(f"Error: The response variable `y` has only one unique value: {unique_values[0]}.")
|
|
86
|
-
return None, None
|
|
87
|
-
binary_response = len(unique_values) == 2
|
|
82
|
+
binary_response = len(np.unique(y)) == 2
|
|
88
83
|
return binary_response, resolve_selector(selector, binary_response)
|
|
89
84
|
|
|
90
85
|
def compute_alphas(X, y, n_alphas, max_features, binary_response=False):
|
|
@@ -94,8 +89,9 @@ def compute_alphas(X, y, n_alphas, max_features, binary_response=False):
|
|
|
94
89
|
scaled_residuals = y - y_mean * (1 - y_mean)
|
|
95
90
|
alpha_max = 5 / np.max(np.abs(np.dot(X.T, scaled_residuals) / n))
|
|
96
91
|
selector = LogisticRegression(penalty='l1', solver='liblinear', tol=1e-3, warm_start=True, class_weight='balanced')
|
|
97
|
-
if np.
|
|
92
|
+
if not np.isfinite(alpha_max):
|
|
98
93
|
alpha_max = 100
|
|
94
|
+
alpha_max = compute_alpha_max_logistic(selector, X, y, alpha_max)
|
|
99
95
|
alpha_min = alpha_max * 1e-10
|
|
100
96
|
test_alphas = np.logspace(np.log10(alpha_max/2), np.log10(alpha_min), 100)
|
|
101
97
|
else:
|
|
@@ -120,6 +116,44 @@ def compute_alphas(X, y, n_alphas, max_features, binary_response=False):
|
|
|
120
116
|
alphas = np.logspace(np.log10(alpha_max), np.log10(alpha_min), n_alphas)
|
|
121
117
|
return alphas
|
|
122
118
|
|
|
119
|
+
# smallest alpha (to within a factor of tol) at which l1 logistic regression selects no features
|
|
120
|
+
def compute_alpha_max_logistic(selector, X, y, alpha_start, tol=1.1, max_steps=50):
|
|
121
|
+
def n_selected(alpha):
|
|
122
|
+
selector.set_params(C=1/alpha)
|
|
123
|
+
with warnings.catch_warnings():
|
|
124
|
+
warnings.simplefilter('ignore')
|
|
125
|
+
selector.fit(X, y)
|
|
126
|
+
return np.sum(selector.coef_ != 0)
|
|
127
|
+
|
|
128
|
+
# bracket the threshold: features are selected at lower, none at upper
|
|
129
|
+
if n_selected(alpha_start) > 0:
|
|
130
|
+
upper = alpha_start
|
|
131
|
+
for _ in range(max_steps):
|
|
132
|
+
upper *= 2
|
|
133
|
+
if n_selected(upper) == 0:
|
|
134
|
+
break
|
|
135
|
+
else:
|
|
136
|
+
return alpha_start
|
|
137
|
+
lower = upper / 2
|
|
138
|
+
else:
|
|
139
|
+
lower = alpha_start
|
|
140
|
+
for _ in range(max_steps):
|
|
141
|
+
lower /= 2
|
|
142
|
+
if n_selected(lower) > 0:
|
|
143
|
+
break
|
|
144
|
+
else:
|
|
145
|
+
return alpha_start
|
|
146
|
+
upper = lower * 2
|
|
147
|
+
|
|
148
|
+
# bisect on a log scale
|
|
149
|
+
while upper / lower > tol:
|
|
150
|
+
mid = np.sqrt(lower * upper)
|
|
151
|
+
if n_selected(mid) > 0:
|
|
152
|
+
lower = mid
|
|
153
|
+
else:
|
|
154
|
+
upper = mid
|
|
155
|
+
return upper
|
|
156
|
+
|
|
123
157
|
def compute_correlation(X):
|
|
124
158
|
corr_matrix = np.corrcoef(X, rowvar=False)
|
|
125
159
|
abs_corr_matrix = np.abs(corr_matrix)
|
|
@@ -250,4 +284,3 @@ def selector_and_args(selector, selector_args):
|
|
|
250
284
|
selector_function = selector
|
|
251
285
|
return selector_function, selector_args
|
|
252
286
|
|
|
253
|
-
|
|
@@ -1,13 +1,21 @@
|
|
|
1
1
|
# Integrated path stability selection
|
|
2
2
|
|
|
3
3
|
import time
|
|
4
|
+
import warnings
|
|
4
5
|
|
|
5
6
|
from joblib import Parallel, delayed
|
|
6
7
|
import numpy as np
|
|
8
|
+
from sklearn.utils import check_random_state
|
|
7
9
|
|
|
10
|
+
from .base_selectors import (fit_gb_classifier, fit_gb_regressor, fit_rf_classifier, fit_rf_regressor,
|
|
11
|
+
fit_ufi_classifier, fit_ufi_regressor)
|
|
8
12
|
from .helpers import return_null_result
|
|
9
13
|
from .processing import postprocess_ipss, preprocess_ipss
|
|
10
14
|
|
|
15
|
+
# built-in selectors that take a random_state; each of their fits gets its own seed
|
|
16
|
+
seeded_selectors = (fit_gb_classifier, fit_gb_regressor, fit_rf_classifier, fit_rf_regressor,
|
|
17
|
+
fit_ufi_classifier, fit_ufi_regressor)
|
|
18
|
+
|
|
11
19
|
#----------------------------------------------------------------
|
|
12
20
|
# IPSS
|
|
13
21
|
#----------------------------------------------------------------
|
|
@@ -20,7 +28,7 @@ Inputs:
|
|
|
20
28
|
|
|
21
29
|
Optional
|
|
22
30
|
----------------
|
|
23
|
-
selector: gradient boosting ('gb'), l1 regularization ('l1'), random forest ('rf'
|
|
31
|
+
selector: gradient boosting ('gb'), l1 regularization ('l1'), random forest ('rf'), or a custom selector
|
|
24
32
|
selector_args: arguments for selector
|
|
25
33
|
target_fp: target number of false positives
|
|
26
34
|
target_fdr: target false discovery rate
|
|
@@ -35,10 +43,12 @@ Inputs:
|
|
|
35
43
|
cutoff: max value of theoretical integral bound I(Lambda)
|
|
36
44
|
delta: determines probability measure mu_delta(dlambda) = z_delta^{-1}lambda^{-delta}dlambda
|
|
37
45
|
subsample_size: size of the subsampled datasets
|
|
38
|
-
standardize_X: whether to standardize features to have mean 0, standard deviation 1
|
|
39
|
-
center_y: whether to center the response to have mean 0
|
|
46
|
+
standardize_X: whether to standardize features to have mean 0, standard deviation 1 (None: only for 'l1' and 'adaptive_lasso')
|
|
47
|
+
center_y: whether to center the response to have mean 0 (None: only for 'l1' and 'adaptive_lasso'; binary responses are never centered)
|
|
40
48
|
n_jobs: number of jobs to run in parallel
|
|
41
49
|
force_regression: force ipss to use regression (if False, classification is used if response is binary)
|
|
50
|
+
random_state: seed (int, RandomState, or None) for preselection, subsampling, and the 'gb', 'rf', and 'ufi'
|
|
51
|
+
selectors, each of whose fits gets its own seed. None uses numpy's global random state
|
|
42
52
|
|
|
43
53
|
Outputs:
|
|
44
54
|
efp_scores: efp (expected false positive) score for each feature
|
|
@@ -50,18 +60,20 @@ Outputs:
|
|
|
50
60
|
def ipss(X, y, selector='gb', selector_args=None, preselect=True, preselector=None, preselector_args=None,
|
|
51
61
|
B=None, n_alphas=None, ipss_function=None, cutoff=0.05, delta=None, subsample_size=None,
|
|
52
62
|
target_fp=None, target_fdr=None, standardize_X=None, center_y=None, n_jobs=1,
|
|
53
|
-
force_regression=False, _return_details=False):
|
|
63
|
+
force_regression=False, random_state=None, _return_details=False):
|
|
54
64
|
|
|
55
65
|
# start timer
|
|
56
66
|
start = time.time()
|
|
57
67
|
|
|
68
|
+
rng = check_random_state(random_state)
|
|
69
|
+
|
|
58
70
|
# prepare ipss arguments and data prior to estimating selection probabilities
|
|
59
71
|
prep = preprocess_ipss(X, y, selector, selector_args, preselect, preselector, preselector_args,
|
|
60
|
-
B, n_alphas, ipss_function, delta, standardize_X, center_y, force_regression)
|
|
72
|
+
B, n_alphas, ipss_function, delta, standardize_X, center_y, force_regression, rng)
|
|
61
73
|
|
|
62
74
|
if prep is None:
|
|
63
75
|
runtime = time.time() - start
|
|
64
|
-
return return_null_result(
|
|
76
|
+
return return_null_result(np.shape(X)[1], runtime)
|
|
65
77
|
|
|
66
78
|
X, y = prep['X'], prep['y']
|
|
67
79
|
selector_function, selector_args = prep['selector_function'], prep['selector_args']
|
|
@@ -69,15 +81,22 @@ def ipss(X, y, selector='gb', selector_args=None, preselect=True, preselector=No
|
|
|
69
81
|
p_full, preselect_indices = prep['p_full'], prep['preselect_indices']
|
|
70
82
|
ipss_function = prep['ipss_function']
|
|
71
83
|
|
|
84
|
+
if selector_function in seeded_selectors and 'random_state' in selector_args:
|
|
85
|
+
warnings.warn("selector_args['random_state'] is ignored; use random_state instead.", UserWarning)
|
|
86
|
+
selector_args = {key: value for key, value in selector_args.items() if key != 'random_state'}
|
|
87
|
+
|
|
72
88
|
n, p = X.shape
|
|
73
89
|
|
|
74
90
|
# subsample size
|
|
75
91
|
if subsample_size is None or subsample_size > n // 2:
|
|
76
92
|
subsample_size = n // 2
|
|
77
93
|
|
|
94
|
+
# one seed per subsampling step
|
|
95
|
+
seeds = rng.randint(np.iinfo(np.int32).max, size=B)
|
|
96
|
+
|
|
78
97
|
# estimate selection probabilities
|
|
79
98
|
results = np.array(Parallel(n_jobs=n_jobs)(delayed(selection)(X, y, alphas, subsample_size,
|
|
80
|
-
selector_function, **selector_args) for
|
|
99
|
+
selector_function, seeds[b], **selector_args) for b in range(B)))
|
|
81
100
|
|
|
82
101
|
# process the estimated selection probabilities and compute final results (efp scores, q-values, etc.)
|
|
83
102
|
ipss_output = postprocess_ipss(results, alphas, n_alphas, p, p_full, preselect_indices, B, cutoff, delta,
|
|
@@ -89,25 +108,24 @@ def ipss(X, y, selector='gb', selector_args=None, preselect=True, preselector=No
|
|
|
89
108
|
|
|
90
109
|
return ipss_output
|
|
91
110
|
|
|
111
|
+
|
|
92
112
|
# subsampler for estimating selection probabilities
|
|
93
|
-
def selection(X, y, alphas, subsample_size, selector_function, **kwargs):
|
|
113
|
+
def selection(X, y, alphas, subsample_size, selector_function, seed=None, **kwargs):
|
|
94
114
|
n, p = X.shape
|
|
95
|
-
|
|
96
|
-
|
|
115
|
+
rng = check_random_state(seed)
|
|
116
|
+
indices = rng.permutation(n)
|
|
97
117
|
|
|
98
118
|
# take two disjoint subsamples
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
indicators =
|
|
109
|
-
indicators[:, 0, :] = np.array(selector_function(X[idx1, :], y[idx1], alphas, **kwargs))
|
|
110
|
-
indicators[:, 1, :] = np.array(selector_function(X[idx2, :], y[idx2], alphas, **kwargs))
|
|
119
|
+
halves = [indices[:subsample_size], indices[subsample_size:2*subsample_size]]
|
|
120
|
+
alpha_args = () if alphas is None else (alphas,)
|
|
121
|
+
|
|
122
|
+
# built-in tree selectors get a new seed for each fit unless one is given
|
|
123
|
+
seed_fits = selector_function in seeded_selectors and 'random_state' not in kwargs
|
|
124
|
+
|
|
125
|
+
indicators = np.empty((2, p)) if alphas is None else np.empty((len(alphas), 2, p))
|
|
126
|
+
for i, idx in enumerate(halves):
|
|
127
|
+
fit_args = {**kwargs, 'random_state': rng.randint(np.iinfo(np.int32).max)} if seed_fits else kwargs
|
|
128
|
+
indicators[..., i, :] = selector_function(X[idx, :], y[idx], *alpha_args, **fit_args)
|
|
111
129
|
|
|
112
130
|
return indicators
|
|
113
131
|
|
|
@@ -5,6 +5,7 @@ import warnings
|
|
|
5
5
|
import numpy as np
|
|
6
6
|
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
|
|
7
7
|
from sklearn.linear_model import Lasso, LogisticRegression
|
|
8
|
+
from sklearn.utils import check_random_state
|
|
8
9
|
import xgboost as xgb
|
|
9
10
|
|
|
10
11
|
# squared distance correlation between response and each feature
|
|
@@ -27,8 +28,9 @@ def compute_dcor_scores(X, y):
|
|
|
27
28
|
scores[j] = dcov_sqr / np.sqrt(denom) if denom > 0 else 0.0
|
|
28
29
|
return scores
|
|
29
30
|
|
|
30
|
-
def preselection(X, y, preselector, preselector_args=None):
|
|
31
|
+
def preselection(X, y, preselector, preselector_args=None, random_state=None):
|
|
31
32
|
n, p = X.shape
|
|
33
|
+
rng = check_random_state(random_state)
|
|
32
34
|
|
|
33
35
|
if preselector_args is None:
|
|
34
36
|
preselector_args = {}
|
|
@@ -56,14 +58,14 @@ def preselection(X, y, preselector, preselector_args=None):
|
|
|
56
58
|
preselector_args_local.setdefault('C', 1 / alpha)
|
|
57
59
|
model = LogisticRegression(**preselector_args_local)
|
|
58
60
|
else:
|
|
59
|
-
preselector_args_local.setdefault('alpha', alpha)
|
|
61
|
+
preselector_args_local.setdefault('alpha', alpha * np.std(y))
|
|
60
62
|
model = Lasso(**preselector_args_local)
|
|
61
63
|
|
|
62
64
|
feature_importances = np.zeros(p)
|
|
63
65
|
with warnings.catch_warnings():
|
|
64
66
|
warnings.simplefilter('ignore')
|
|
65
67
|
for _ in range(n_runs):
|
|
66
|
-
indices =
|
|
68
|
+
indices = rng.choice(n, size=n, replace=True)
|
|
67
69
|
X_sampled, y_sampled = X[indices], y[indices]
|
|
68
70
|
model.fit(X_sampled, y_sampled)
|
|
69
71
|
feature_importances += np.abs(model.coef_).ravel()
|
|
@@ -77,7 +79,7 @@ def preselection(X, y, preselector, preselector_args=None):
|
|
|
77
79
|
model = model_class(**preselector_args_local)
|
|
78
80
|
feature_importances = np.zeros(p)
|
|
79
81
|
for _ in range(n_runs):
|
|
80
|
-
model.set_params(random_state=
|
|
82
|
+
model.set_params(random_state=rng.randint(1e5))
|
|
81
83
|
model.fit(X,y)
|
|
82
84
|
feature_importances += model.feature_importances_
|
|
83
85
|
preselect_indices = np.argsort(feature_importances)[::-1][:n_keep]
|
|
@@ -91,7 +93,7 @@ def preselection(X, y, preselector, preselector_args=None):
|
|
|
91
93
|
model = model_class(**preselector_args_local)
|
|
92
94
|
feature_importances = np.zeros(p)
|
|
93
95
|
for i in range(n_runs):
|
|
94
|
-
model.set_params(random_state=
|
|
96
|
+
model.set_params(random_state=rng.randint(1e5))
|
|
95
97
|
model.fit(X, y)
|
|
96
98
|
feature_importances += model.feature_importances_
|
|
97
99
|
nonzero_mask = feature_importances > 0
|
|
@@ -104,7 +106,7 @@ def preselection(X, y, preselector, preselector_args=None):
|
|
|
104
106
|
else:
|
|
105
107
|
n_extra = n_keep - len(nonzero_indices)
|
|
106
108
|
zero_indices = np.where(~nonzero_mask)[0]
|
|
107
|
-
extra_indices =
|
|
109
|
+
extra_indices = rng.choice(zero_indices, size=n_extra, replace=False)
|
|
108
110
|
preselect_indices = np.concatenate([nonzero_indices, extra_indices])
|
|
109
111
|
|
|
110
112
|
elif preselector == 'dcor':
|
|
@@ -3,15 +3,23 @@
|
|
|
3
3
|
import warnings
|
|
4
4
|
|
|
5
5
|
import numpy as np
|
|
6
|
+
from scipy.sparse import issparse
|
|
6
7
|
from sklearn.preprocessing import StandardScaler
|
|
7
8
|
|
|
9
|
+
from .base_selectors import adaptive_weights
|
|
8
10
|
from .helpers import (check_response_type, compute_alphas, compute_delta, compute_qvalues, integrate,
|
|
9
11
|
resolve_preselector, resolve_selector, score_based_selection, selector_and_args)
|
|
10
12
|
from .preselection import preselection
|
|
11
13
|
|
|
12
14
|
# prepare ipss arguments and data
|
|
13
15
|
def preprocess_ipss(X, y, selector, selector_args, preselect, preselector, preselector_args,
|
|
14
|
-
B, n_alphas, ipss_function, delta, standardize_X, center_y, force_regression=False):
|
|
16
|
+
B, n_alphas, ipss_function, delta, standardize_X, center_y, force_regression=False, random_state=None):
|
|
17
|
+
|
|
18
|
+
# cast to float64 (float32 alpha grids underflow/overflow in integrate)
|
|
19
|
+
X = X.astype(np.float64) if issparse(X) else np.asarray(X, dtype=np.float64)
|
|
20
|
+
y = np.asarray(y)
|
|
21
|
+
if y.dtype.kind in 'biuf':
|
|
22
|
+
y = y.astype(np.float64)
|
|
15
23
|
|
|
16
24
|
# specify whether base estimator is a regularization or variable importance method
|
|
17
25
|
selector_type = 'importance'
|
|
@@ -21,15 +29,6 @@ def preprocess_ipss(X, y, selector, selector_args, preselect, preselector, prese
|
|
|
21
29
|
# empty set for selector args if none specified
|
|
22
30
|
selector_args = selector_args or {}
|
|
23
31
|
|
|
24
|
-
# if no preselector is specified and the selector is a custom callable, the preselector
|
|
25
|
-
# defaults to reusing that callable, so it shares selector_args unless overridden
|
|
26
|
-
if preselector is None and not isinstance(selector, str):
|
|
27
|
-
if preselector_args is None:
|
|
28
|
-
preselector_args = {}
|
|
29
|
-
for key in selector_args:
|
|
30
|
-
if key not in preselector_args:
|
|
31
|
-
preselector_args[key] = selector_args[key]
|
|
32
|
-
|
|
33
32
|
# number of subsamples
|
|
34
33
|
B = B if B is not None else 100 if selector == 'gb' else 50
|
|
35
34
|
|
|
@@ -38,12 +37,18 @@ def preprocess_ipss(X, y, selector, selector_args, preselect, preselector, prese
|
|
|
38
37
|
y = y.ravel()
|
|
39
38
|
|
|
40
39
|
# check response type
|
|
40
|
+
if np.unique(y).size == 1:
|
|
41
|
+
raise ValueError(f'The response y has only one unique value: {y[0]}.')
|
|
41
42
|
if force_regression:
|
|
42
43
|
binary_response = False
|
|
43
44
|
selector = resolve_selector(selector, binary_response)
|
|
44
45
|
else:
|
|
45
46
|
binary_response, selector = check_response_type(y, selector)
|
|
46
47
|
|
|
48
|
+
# code binary responses as 0/1 (required by XGBoost)
|
|
49
|
+
if binary_response:
|
|
50
|
+
y = (y == np.unique(y)[1]).astype(np.float64)
|
|
51
|
+
|
|
47
52
|
# ipss function
|
|
48
53
|
if ipss_function is None:
|
|
49
54
|
ipss_function = 'h2' if selector in ['lasso', 'logistic_regression'] else 'h3'
|
|
@@ -52,19 +57,21 @@ def preprocess_ipss(X, y, selector, selector_args, preselect, preselector, prese
|
|
|
52
57
|
if delta is None:
|
|
53
58
|
delta = compute_delta(X, selector)
|
|
54
59
|
|
|
55
|
-
# standardize and center data
|
|
56
|
-
if
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
60
|
+
# standardize and center data (by default only for l1 selectors); never center a binary response
|
|
61
|
+
if standardize_X is None:
|
|
62
|
+
standardize_X = selector_type == 'regularization'
|
|
63
|
+
if center_y is None:
|
|
64
|
+
center_y = selector_type == 'regularization'
|
|
65
|
+
if standardize_X:
|
|
66
|
+
X = StandardScaler().fit_transform(X)
|
|
67
|
+
if center_y and not binary_response:
|
|
68
|
+
y = y - np.mean(y)
|
|
62
69
|
|
|
63
70
|
# preselect features to reduce dimension
|
|
64
71
|
p_full = X.shape[1]
|
|
65
72
|
if preselect:
|
|
66
73
|
preselector = resolve_preselector(preselector, selector, binary_response)
|
|
67
|
-
X, preselect_indices = preselection(X, y, preselector, preselector_args)
|
|
74
|
+
X, preselect_indices = preselection(X, y, preselector, preselector_args, random_state)
|
|
68
75
|
if preselect_indices.size == 0:
|
|
69
76
|
warnings.warn('Preselection step removed all features. Returning null result.', UserWarning)
|
|
70
77
|
return None
|
|
@@ -80,7 +87,14 @@ def preprocess_ipss(X, y, selector, selector_args, preselect, preselector, prese
|
|
|
80
87
|
# alphas
|
|
81
88
|
if n_alphas is None:
|
|
82
89
|
n_alphas = 25 if selector_type == 'regularization' else 100
|
|
83
|
-
alphas =
|
|
90
|
+
alphas = None
|
|
91
|
+
if selector_type == 'regularization':
|
|
92
|
+
# adaptive lasso fits a lasso on X / weights, so calibrate the grid on that reweighted design
|
|
93
|
+
X_alphas = X
|
|
94
|
+
if selector in ['adaptive_lasso_classifier', 'adaptive_lasso_regressor']:
|
|
95
|
+
epsilon = selector_args.get('epsilon', 1e-6)
|
|
96
|
+
X_alphas = X / adaptive_weights(X, y, binary_response, epsilon)
|
|
97
|
+
alphas = compute_alphas(X_alphas, y, n_alphas, max_features, binary_response)
|
|
84
98
|
|
|
85
99
|
# selector function and args
|
|
86
100
|
selector_function, selector_args = selector_and_args(selector, selector_args)
|
|
@@ -216,4 +230,3 @@ def ipss_scores(stability_paths, B, alphas, average_selected, ipss_function, del
|
|
|
216
230
|
|
|
217
231
|
return scores, integral, alphas, stop_index
|
|
218
232
|
|
|
219
|
-
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: ipss
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.2.0
|
|
4
4
|
Summary: Python implementation of integrated path stability selection (IPSS)
|
|
5
5
|
Author-email: Omar Melikechi <omar.melikechi@gmail.com>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -23,16 +23,14 @@ Dynamic: license-file
|
|
|
23
23
|
|
|
24
24
|
> Fast, flexible feature selection with false discovery control
|
|
25
25
|
|
|
26
|
-
##
|
|
26
|
+
## Papers
|
|
27
27
|
|
|
28
28
|
- **Integrated path stability selection**
|
|
29
29
|
[*Journal of the American Statistical Association*](https://doi.org/10.1080/01621459.2025.2525589) • [arXiv](https://arxiv.org/abs/2403.15877)
|
|
30
30
|
|
|
31
|
-
- **Nonparametric IPSS
|
|
31
|
+
- **Nonparametric IPSS**
|
|
32
32
|
[*Bioinformatics*](https://doi.org/10.1093/bioinformatics/btaf299) • [arXiv](https://arxiv.org/abs/2410.02208)
|
|
33
33
|
|
|
34
|
-
> "*Integrated path stability selection*" introduces IPSS and applies it to regularized models such as lasso and adaptive lasso. "*Nonparametric IPSS: Fast, flexible feature selection with false discovery control*" extends IPSS to arbitrary feature importance scores, such as those from gradient boosting and random forests.
|
|
35
|
-
|
|
36
34
|
## Installation
|
|
37
35
|
```
|
|
38
36
|
pip install ipss
|
|
@@ -51,6 +49,7 @@ ipss_output = ipss(X,y)
|
|
|
51
49
|
target_fdr = 0.1
|
|
52
50
|
q_values = ipss_output['q_values']
|
|
53
51
|
selected_features = [idx for idx, q_value in q_values.items() if q_value <= target_fdr]
|
|
52
|
+
|
|
54
53
|
print(f'Selected features (target FDR = {target_fdr}): {selected_features}')
|
|
55
54
|
```
|
|
56
55
|
### Outputs
|
|
@@ -59,7 +58,7 @@ print(f'Selected features (target FDR = {target_fdr}): {selected_features}')
|
|
|
59
58
|
- `q_values`: Dictionary whose keys are feature indices and values are their q-values (dict of length `p`).
|
|
60
59
|
- `runtime`: Runtime of the algorithm in seconds (float).
|
|
61
60
|
- `selected_features`: Indices of features selected by IPSS; empty list if `target_fp` and `target_fdr` are not specified (list of ints).
|
|
62
|
-
- `stability_paths`: Estimated selection probabilities at each parameter
|
|
61
|
+
- `stability_paths`: Estimated selection probabilities at each parameter (array of shape `(n_alphas, p)`).
|
|
63
62
|
|
|
64
63
|
### Selecting features
|
|
65
64
|
Each feature (column of `X`) is assigned:
|
|
@@ -125,7 +124,7 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
125
124
|
- Users can provide their own feature importance function (see example above).
|
|
126
125
|
- `selector_args`: Arguments for the base algorithm (dict; default `None`).
|
|
127
126
|
- `preselect`: Preselect/filter features prior to subsampling (bool; default `True`).
|
|
128
|
-
- `preselector`: Method used to preselect features (default `None`, which resolves to `'dcor'` for every `selector` except `'l1'`/`'adaptive_lasso'`, which use their own linear preselection
|
|
127
|
+
- `preselector`: Method used to preselect features (default `None`, which resolves to `'dcor'` for every `selector` except `'l1'`/`'adaptive_lasso'`, which use their own linear preselection). Options:
|
|
129
128
|
- `'gb'`: Gradient boosting (XGBoost) feature importances.
|
|
130
129
|
- `'rf'`: Random forest (scikit-learn) feature importances.
|
|
131
130
|
- `'l1'` or `'adaptive_lasso'`: (Adaptive) lasso/logistic regression coefficients.
|
|
@@ -146,9 +145,10 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
146
145
|
- `'h3'`: Cubic function, ```h3(x) = (2x - 1)**3 if x >= 0.5 else 0```.
|
|
147
146
|
- `cutoff`: Maximum value of the theoretical integral bound `I(Lambda)` (positive float; default `0.05`).
|
|
148
147
|
- `delta`: Defines probability measure; see `Associated papers` (float; defaults depend on `selector`).
|
|
149
|
-
- `standardize_X`: Scale features to have mean 0, standard deviation 1 (bool; default `None`).
|
|
150
|
-
- `center_y`: Center response to have mean 0 (bool; default `None`).
|
|
148
|
+
- `standardize_X`: Scale features to have mean 0, standard deviation 1 (bool; default `None`, which standardizes for `'l1'` and `'adaptive_lasso'` only).
|
|
149
|
+
- `center_y`: Center response to have mean 0 (bool; default `None`, which centers for `'l1'` and `'adaptive_lasso'` only). Binary responses are never centered.
|
|
151
150
|
- `n_jobs`: Number of jobs to run in parallel (int; default `1`).
|
|
151
|
+
- `random_state`: Seed for preselection, subsampling, and the `'gb'`, `'rf'`, and `'ufi'` selectors (int or `None`; default `None`, which uses NumPy's global random state). Each selector fit gets its own seed; `selector_args['random_state']` is ignored.
|
|
152
152
|
|
|
153
153
|
### General observations/recommendations:
|
|
154
154
|
- `selector = 'gb'` often best for capturing nonlinear relationships.
|
|
@@ -164,4 +164,3 @@ The [examples](https://github.com/omelikechi/ipss/tree/main/examples) folder inc
|
|
|
164
164
|
- Results are robust to `cutoff` provided it is between `0.025` and `0.1`.
|
|
165
165
|
- Features are automatically standardized for the penalized regression methods.
|
|
166
166
|
- The response is automatically centered for the regularized regression methods.
|
|
167
|
-
|
ipss-1.1.12/MANIFEST.in
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|