utilsds-models 0.0.1__tar.gz → 0.0.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- utilsds_models-0.0.2/PKG-INFO +97 -0
- utilsds_models-0.0.2/docs/ALTERNATIVE_README.md +79 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/pyproject.toml +1 -1
- utilsds_models-0.0.2/utilsds_models.egg-info/PKG-INFO +97 -0
- utilsds_models-0.0.1/PKG-INFO +0 -106
- utilsds_models-0.0.1/docs/ALTERNATIVE_README.md +0 -88
- utilsds_models-0.0.1/utilsds_models.egg-info/PKG-INFO +0 -106
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/README.md +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/setup.cfg +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds-models/__init__.py +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds-models/custom_metrics.py +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds-models/data_processing.py +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds-models/evip_dynamic.py +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds-models/metrics.py +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds-models/visualization.py +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds_models.egg-info/SOURCES.txt +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds_models.egg-info/dependency_links.txt +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds_models.egg-info/requires.txt +0 -0
- {utilsds_models-0.0.1 → utilsds_models-0.0.2}/utilsds_models.egg-info/top_level.txt +0 -0
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: utilsds-models
|
|
3
|
+
Version: 0.0.2
|
|
4
|
+
Summary: Solution for specific models
|
|
5
|
+
Author-email: DS Team <ds@sts.pl>
|
|
6
|
+
License: MIT License
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
9
|
+
Classifier: Operating System :: OS Independent
|
|
10
|
+
Requires-Python: >=3.12
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
Requires-Dist: pandas>=2.2.2
|
|
13
|
+
Requires-Dist: numpy>=1.26.0
|
|
14
|
+
Requires-Dist: scikit-learn>=1.5.0
|
|
15
|
+
Requires-Dist: matplotlib>=3.9.0
|
|
16
|
+
Provides-Extra: dev
|
|
17
|
+
Requires-Dist: pre-commit>=3.5.0; extra == "dev"
|
|
18
|
+
|
|
19
|
+
# utilsds-models
|
|
20
|
+
|
|
21
|
+
A library of classes and functions used in DS Team modeling projects.
|
|
22
|
+
It extends the [utilsds](https://pypi.org/project/utilsds/) package with components specific to selected models
|
|
23
|
+
(data processing, NGR metrics, EVIP metrics, and a custom LightGBM objective).
|
|
24
|
+
|
|
25
|
+
Requires Python >= 3.12.
|
|
26
|
+
|
|
27
|
+
## Installation
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
uv sync
|
|
31
|
+
source .venv/bin/activate
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Or from PyPI (after publication):
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
pip install utilsds-models
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Modules
|
|
41
|
+
|
|
42
|
+
### `data_processing`
|
|
43
|
+
|
|
44
|
+
Scikit-learn-compatible classes and helper functions for combining test results.
|
|
45
|
+
|
|
46
|
+
- **`ColumnCopyImputer`**: fills missing values by copying from other columns.
|
|
47
|
+
- **`NullImputerWithFlags`**: imputes nulls with optional flags (`_isnull_flag`) and strategies `mean`, `median`, `mode`.
|
|
48
|
+
- **`MaxMultiplierImputer`**: imputes using `max * multiplier` from training data, with optional flags.
|
|
49
|
+
- **`SportsHybridEncoder`**: encodes sports as binary features for the top N disciplines plus aggregations for the rest.
|
|
50
|
+
- **`LabelEncoderTransformer`**: label encoding for categorical columns (e.g. for LightGBM).
|
|
51
|
+
- **`DerivedFeatureCreator`**: creates derived features (e.g. division by 7 or 30).
|
|
52
|
+
- **`combine_test_data`**: combines test features, target, predictions, and metadata into a single DataFrame.
|
|
53
|
+
|
|
54
|
+
### `custom_metrics`
|
|
55
|
+
|
|
56
|
+
Evaluation metrics with time-based weights (`days_since_ftd`) and a custom LightGBM objective.
|
|
57
|
+
|
|
58
|
+
- **`EvalMetric`**: cohort-level (aggregated by `days_since_ftd`) and sample-level metrics with time-decay weighting:
|
|
59
|
+
- `cohort_weighted_mae`, `cohort_weighted_mse`, `cohort_weighted_mape`
|
|
60
|
+
- `sample_weighted_mae`, `sample_weighted_mse`, `sample_weighted_mape`
|
|
61
|
+
- `create_lgb_metric`: factory for LightGBM metrics with time weights
|
|
62
|
+
- **`DaysWeightedObjective`**: custom LightGBM objective with time weights (modes: `mae`, `mse`, `mape`).
|
|
63
|
+
|
|
64
|
+
### `metrics`
|
|
65
|
+
|
|
66
|
+
- **`calculate_ngr_metrics`**: computes NGR error metrics (MAE, MAPE, ME, MPE) in both standard
|
|
67
|
+
and business-optimal variants with weights based on `days_since_ftd`. Optionally applies late-stage
|
|
68
|
+
prediction correction over the customer lifecycle.
|
|
69
|
+
|
|
70
|
+
### `visualization`
|
|
71
|
+
|
|
72
|
+
- **`calculate_ngr_metrics`**: NGR evaluation function (equivalent to the `metrics` module).
|
|
73
|
+
|
|
74
|
+
### `evip_dynamic`
|
|
75
|
+
|
|
76
|
+
Classification metrics with a false-positive (FP) budget constraint.
|
|
77
|
+
|
|
78
|
+
- **`recall_with_fp_cap`**: recall with a penalty for exceeding the FP budget in binary classification.
|
|
79
|
+
- **`weighted_premium_recall_with_fp_cap`**: weighted recall for premium classes (1 and 2) with a penalty
|
|
80
|
+
for exceeding the FPR budget among class 0 samples.
|
|
81
|
+
|
|
82
|
+
## Dependencies
|
|
83
|
+
|
|
84
|
+
- `pandas>=2.2.2`
|
|
85
|
+
- `numpy>=1.26.0`
|
|
86
|
+
- `scikit-learn>=1.5.0`
|
|
87
|
+
- `matplotlib>=3.9.0`
|
|
88
|
+
|
|
89
|
+
## Publishing to PyPI
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
uv pip install build twine
|
|
93
|
+
uv run python -m build
|
|
94
|
+
twine upload --skip-existing dist/*
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Before publishing, bump the version in `pyproject.toml` (section `[project]`).
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# utilsds-models
|
|
2
|
+
|
|
3
|
+
A library of classes and functions used in DS Team modeling projects.
|
|
4
|
+
It extends the [utilsds](https://pypi.org/project/utilsds/) package with components specific to selected models
|
|
5
|
+
(data processing, NGR metrics, EVIP metrics, and a custom LightGBM objective).
|
|
6
|
+
|
|
7
|
+
Requires Python >= 3.12.
|
|
8
|
+
|
|
9
|
+
## Installation
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
uv sync
|
|
13
|
+
source .venv/bin/activate
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Or from PyPI (after publication):
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
pip install utilsds-models
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
## Modules
|
|
23
|
+
|
|
24
|
+
### `data_processing`
|
|
25
|
+
|
|
26
|
+
Scikit-learn-compatible classes and helper functions for combining test results.
|
|
27
|
+
|
|
28
|
+
- **`ColumnCopyImputer`**: fills missing values by copying from other columns.
|
|
29
|
+
- **`NullImputerWithFlags`**: imputes nulls with optional flags (`_isnull_flag`) and strategies `mean`, `median`, `mode`.
|
|
30
|
+
- **`MaxMultiplierImputer`**: imputes using `max * multiplier` from training data, with optional flags.
|
|
31
|
+
- **`SportsHybridEncoder`**: encodes sports as binary features for the top N disciplines plus aggregations for the rest.
|
|
32
|
+
- **`LabelEncoderTransformer`**: label encoding for categorical columns (e.g. for LightGBM).
|
|
33
|
+
- **`DerivedFeatureCreator`**: creates derived features (e.g. division by 7 or 30).
|
|
34
|
+
- **`combine_test_data`**: combines test features, target, predictions, and metadata into a single DataFrame.
|
|
35
|
+
|
|
36
|
+
### `custom_metrics`
|
|
37
|
+
|
|
38
|
+
Evaluation metrics with time-based weights (`days_since_ftd`) and a custom LightGBM objective.
|
|
39
|
+
|
|
40
|
+
- **`EvalMetric`**: cohort-level (aggregated by `days_since_ftd`) and sample-level metrics with time-decay weighting:
|
|
41
|
+
- `cohort_weighted_mae`, `cohort_weighted_mse`, `cohort_weighted_mape`
|
|
42
|
+
- `sample_weighted_mae`, `sample_weighted_mse`, `sample_weighted_mape`
|
|
43
|
+
- `create_lgb_metric`: factory for LightGBM metrics with time weights
|
|
44
|
+
- **`DaysWeightedObjective`**: custom LightGBM objective with time weights (modes: `mae`, `mse`, `mape`).
|
|
45
|
+
|
|
46
|
+
### `metrics`
|
|
47
|
+
|
|
48
|
+
- **`calculate_ngr_metrics`**: computes NGR error metrics (MAE, MAPE, ME, MPE) in both standard
|
|
49
|
+
and business-optimal variants with weights based on `days_since_ftd`. Optionally applies late-stage
|
|
50
|
+
prediction correction over the customer lifecycle.
|
|
51
|
+
|
|
52
|
+
### `visualization`
|
|
53
|
+
|
|
54
|
+
- **`calculate_ngr_metrics`**: NGR evaluation function (equivalent to the `metrics` module).
|
|
55
|
+
|
|
56
|
+
### `evip_dynamic`
|
|
57
|
+
|
|
58
|
+
Classification metrics with a false-positive (FP) budget constraint.
|
|
59
|
+
|
|
60
|
+
- **`recall_with_fp_cap`**: recall with a penalty for exceeding the FP budget in binary classification.
|
|
61
|
+
- **`weighted_premium_recall_with_fp_cap`**: weighted recall for premium classes (1 and 2) with a penalty
|
|
62
|
+
for exceeding the FPR budget among class 0 samples.
|
|
63
|
+
|
|
64
|
+
## Dependencies
|
|
65
|
+
|
|
66
|
+
- `pandas>=2.2.2`
|
|
67
|
+
- `numpy>=1.26.0`
|
|
68
|
+
- `scikit-learn>=1.5.0`
|
|
69
|
+
- `matplotlib>=3.9.0`
|
|
70
|
+
|
|
71
|
+
## Publishing to PyPI
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
uv pip install build twine
|
|
75
|
+
uv run python -m build
|
|
76
|
+
twine upload --skip-existing dist/*
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Before publishing, bump the version in `pyproject.toml` (section `[project]`).
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "utilsds-models"
|
|
7
|
-
version = "0.0.
|
|
7
|
+
version = "0.0.2"
|
|
8
8
|
description = "Solution for specific models"
|
|
9
9
|
readme = {file = "docs/ALTERNATIVE_README.md", content-type = "text/markdown"}
|
|
10
10
|
requires-python = ">=3.12"
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: utilsds-models
|
|
3
|
+
Version: 0.0.2
|
|
4
|
+
Summary: Solution for specific models
|
|
5
|
+
Author-email: DS Team <ds@sts.pl>
|
|
6
|
+
License: MIT License
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
9
|
+
Classifier: Operating System :: OS Independent
|
|
10
|
+
Requires-Python: >=3.12
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
Requires-Dist: pandas>=2.2.2
|
|
13
|
+
Requires-Dist: numpy>=1.26.0
|
|
14
|
+
Requires-Dist: scikit-learn>=1.5.0
|
|
15
|
+
Requires-Dist: matplotlib>=3.9.0
|
|
16
|
+
Provides-Extra: dev
|
|
17
|
+
Requires-Dist: pre-commit>=3.5.0; extra == "dev"
|
|
18
|
+
|
|
19
|
+
# utilsds-models
|
|
20
|
+
|
|
21
|
+
A library of classes and functions used in DS Team modeling projects.
|
|
22
|
+
It extends the [utilsds](https://pypi.org/project/utilsds/) package with components specific to selected models
|
|
23
|
+
(data processing, NGR metrics, EVIP metrics, and a custom LightGBM objective).
|
|
24
|
+
|
|
25
|
+
Requires Python >= 3.12.
|
|
26
|
+
|
|
27
|
+
## Installation
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
uv sync
|
|
31
|
+
source .venv/bin/activate
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Or from PyPI (after publication):
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
pip install utilsds-models
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Modules
|
|
41
|
+
|
|
42
|
+
### `data_processing`
|
|
43
|
+
|
|
44
|
+
Scikit-learn-compatible classes and helper functions for combining test results.
|
|
45
|
+
|
|
46
|
+
- **`ColumnCopyImputer`**: fills missing values by copying from other columns.
|
|
47
|
+
- **`NullImputerWithFlags`**: imputes nulls with optional flags (`_isnull_flag`) and strategies `mean`, `median`, `mode`.
|
|
48
|
+
- **`MaxMultiplierImputer`**: imputes using `max * multiplier` from training data, with optional flags.
|
|
49
|
+
- **`SportsHybridEncoder`**: encodes sports as binary features for the top N disciplines plus aggregations for the rest.
|
|
50
|
+
- **`LabelEncoderTransformer`**: label encoding for categorical columns (e.g. for LightGBM).
|
|
51
|
+
- **`DerivedFeatureCreator`**: creates derived features (e.g. division by 7 or 30).
|
|
52
|
+
- **`combine_test_data`**: combines test features, target, predictions, and metadata into a single DataFrame.
|
|
53
|
+
|
|
54
|
+
### `custom_metrics`
|
|
55
|
+
|
|
56
|
+
Evaluation metrics with time-based weights (`days_since_ftd`) and a custom LightGBM objective.
|
|
57
|
+
|
|
58
|
+
- **`EvalMetric`**: cohort-level (aggregated by `days_since_ftd`) and sample-level metrics with time-decay weighting:
|
|
59
|
+
- `cohort_weighted_mae`, `cohort_weighted_mse`, `cohort_weighted_mape`
|
|
60
|
+
- `sample_weighted_mae`, `sample_weighted_mse`, `sample_weighted_mape`
|
|
61
|
+
- `create_lgb_metric`: factory for LightGBM metrics with time weights
|
|
62
|
+
- **`DaysWeightedObjective`**: custom LightGBM objective with time weights (modes: `mae`, `mse`, `mape`).
|
|
63
|
+
|
|
64
|
+
### `metrics`
|
|
65
|
+
|
|
66
|
+
- **`calculate_ngr_metrics`**: computes NGR error metrics (MAE, MAPE, ME, MPE) in both standard
|
|
67
|
+
and business-optimal variants with weights based on `days_since_ftd`. Optionally applies late-stage
|
|
68
|
+
prediction correction over the customer lifecycle.
|
|
69
|
+
|
|
70
|
+
### `visualization`
|
|
71
|
+
|
|
72
|
+
- **`calculate_ngr_metrics`**: NGR evaluation function (equivalent to the `metrics` module).
|
|
73
|
+
|
|
74
|
+
### `evip_dynamic`
|
|
75
|
+
|
|
76
|
+
Classification metrics with a false-positive (FP) budget constraint.
|
|
77
|
+
|
|
78
|
+
- **`recall_with_fp_cap`**: recall with a penalty for exceeding the FP budget in binary classification.
|
|
79
|
+
- **`weighted_premium_recall_with_fp_cap`**: weighted recall for premium classes (1 and 2) with a penalty
|
|
80
|
+
for exceeding the FPR budget among class 0 samples.
|
|
81
|
+
|
|
82
|
+
## Dependencies
|
|
83
|
+
|
|
84
|
+
- `pandas>=2.2.2`
|
|
85
|
+
- `numpy>=1.26.0`
|
|
86
|
+
- `scikit-learn>=1.5.0`
|
|
87
|
+
- `matplotlib>=3.9.0`
|
|
88
|
+
|
|
89
|
+
## Publishing to PyPI
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
uv pip install build twine
|
|
93
|
+
uv run python -m build
|
|
94
|
+
twine upload --skip-existing dist/*
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Before publishing, bump the version in `pyproject.toml` (section `[project]`).
|
utilsds_models-0.0.1/PKG-INFO
DELETED
|
@@ -1,106 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: utilsds-models
|
|
3
|
-
Version: 0.0.1
|
|
4
|
-
Summary: Solution for specific models
|
|
5
|
-
Author-email: DS Team <ds@sts.pl>
|
|
6
|
-
License: MIT License
|
|
7
|
-
Classifier: Programming Language :: Python :: 3
|
|
8
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
9
|
-
Classifier: Operating System :: OS Independent
|
|
10
|
-
Requires-Python: >=3.12
|
|
11
|
-
Description-Content-Type: text/markdown
|
|
12
|
-
Requires-Dist: pandas>=2.2.2
|
|
13
|
-
Requires-Dist: numpy>=1.26.0
|
|
14
|
-
Requires-Dist: scikit-learn>=1.5.0
|
|
15
|
-
Requires-Dist: matplotlib>=3.9.0
|
|
16
|
-
Provides-Extra: dev
|
|
17
|
-
Requires-Dist: pre-commit>=3.5.0; extra == "dev"
|
|
18
|
-
|
|
19
|
-
# utilsds
|
|
20
|
-
|
|
21
|
-
Utilsds is a library that includes classes and functions used in data science projects such as:
|
|
22
|
-
|
|
23
|
-
- **algorithm**:
|
|
24
|
-
- `Algorithm`: Base class for fitting, training, and getting hyperparameters of machine learning models.
|
|
25
|
-
|
|
26
|
-
- **data_ops**:
|
|
27
|
-
- `DataOperations`: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
|
|
28
|
-
- BigQuery operations:
|
|
29
|
-
- `load_bq_data`: Load data from tables, views, and SQL files.
|
|
30
|
-
- `save_bq_view`, `save_bq_table`: Save views and tables.
|
|
31
|
-
- `load_bq_procedure`: Execute stored procedures.
|
|
32
|
-
- `load_bq_details`: Get table/view details and schema.
|
|
33
|
-
- `delete_bq_data`: Delete data with safety confirmations.
|
|
34
|
-
- `dry_run`: Perform dry runs to estimate query costs.
|
|
35
|
-
- Cloud Storage operations:
|
|
36
|
-
- `save_gcs_bucket`: Create buckets.
|
|
37
|
-
- `save_gcs_file`, `load_gcs_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
|
|
38
|
-
- Local file operations:
|
|
39
|
-
- `save_local_file`, `load_local_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
|
|
40
|
-
|
|
41
|
-
- **data_processing**:
|
|
42
|
-
- `SkewnessTransformer`: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
|
|
43
|
-
- `NullReplacer`: Replace null values in specified columns with configurable strategies.
|
|
44
|
-
- `ColumnDropper`: Drop specified columns from a DataFrame.
|
|
45
|
-
- `OutliersCleaner`: Clean outliers by clipping values outside specified percentile ranges.
|
|
46
|
-
- `CategoricalMapper`: Map values in categorical columns according to a specified mapping scheme.
|
|
47
|
-
- `NumericalMapper`: Convert numerical columns to categorical by binning.
|
|
48
|
-
- `Encoder`: One-hot encode categorical columns in the data.
|
|
49
|
-
- `Normalizer`: Normalize numerical columns using a provided scaler.
|
|
50
|
-
|
|
51
|
-
- **data_split**:
|
|
52
|
-
- `train_test_validation_split`: Split data into training, testing, and validation sets.
|
|
53
|
-
- `resample_X_y`: resample train data and target column.
|
|
54
|
-
|
|
55
|
-
- **ds_statistics**:
|
|
56
|
-
- `test_kruskal_wallis`: Perform the Kruskal-Wallis statistical test.
|
|
57
|
-
- `test_agosto_pearsona`: Test for normality using D'Agostino-Pearson test.
|
|
58
|
-
|
|
59
|
-
- **evaluate**:
|
|
60
|
-
- `ModelEvaluator`: Evaluate models and generate plots for diagnostics.
|
|
61
|
-
- `ShapExplainer`: Explain model predictions using SHAP values.
|
|
62
|
-
|
|
63
|
-
- **experiments**:
|
|
64
|
-
- `VertexExperiment`: Manage experiments with Vertex AI.
|
|
65
|
-
|
|
66
|
-
- **optuna**:
|
|
67
|
-
- `Optuna`: Optimize hyperparameters using Optuna.
|
|
68
|
-
|
|
69
|
-
- **metrics**:
|
|
70
|
-
- `Metrics`: Calculate metrics for both classification and regression models.
|
|
71
|
-
|
|
72
|
-
- **modeling**:
|
|
73
|
-
- `Modeling`: Manage modeling, metrics, and logging with Vertex AI.
|
|
74
|
-
|
|
75
|
-
- **Supervised**:
|
|
76
|
-
- `LazyClassifier`: A classifier that automatically trains and evaluates multiple models.
|
|
77
|
-
- `LazyRegressor`: A regressor that automatically trains and evaluates multiple models.
|
|
78
|
-
- `get_card_split`: Function to split data into card-like groups.
|
|
79
|
-
- `adjusted_rsquared`: Calculate adjusted R-squared for regression models.
|
|
80
|
-
|
|
81
|
-
- **visualization**:
|
|
82
|
-
- `MetricsPlot`: Compare metrics for different parameter values.
|
|
83
|
-
- `Radar`: Create radar plots for visualizing data.
|
|
84
|
-
- `cluster_characteristics`: Analyze cluster characteristics.
|
|
85
|
-
- `comparison_density`: Compare density distributions.
|
|
86
|
-
- `elbow_visualisation`: Visualize the elbow method for clustering.
|
|
87
|
-
- `describe_clusters_metrics`: Describe metrics for clusters.
|
|
88
|
-
- `category_null_variables`: Visualize null variables in categorical data.
|
|
89
|
-
- `normal_distr_plots`: Visualize normal distribution plots.
|
|
90
|
-
- `distplot_limitations`: Visualize limitations of distplot.
|
|
91
|
-
- `boxplot_limitations`: Visualize limitations of boxplot.
|
|
92
|
-
- `violinplot_limitations`: Visualize limitations of violinplot.
|
|
93
|
-
- `countplot_limitations`: Visualize limitations of countplot.
|
|
94
|
-
- `categorical_variable_perc`: Visualize percentage of categorical variables.
|
|
95
|
-
- `spearman_correlation`: Visualize spearman correlation.
|
|
96
|
-
- `calculate_crammers_v`: Calculate Crammer's V.
|
|
97
|
-
|
|
98
|
-
- **what_if_streamlit**:
|
|
99
|
-
- `ShapSaver`: Save SHAP explainer components for lazy loading in what-if analysis.
|
|
100
|
-
- `ColumnMetadataGenerator`: Generate column metadata from a DataFrame or CSV file.
|
|
101
|
-
|
|
102
|
-
- **monitoring**:
|
|
103
|
-
- `mapping`: Create column mapping from configuration file for Evidently.
|
|
104
|
-
- `test_data`: Test data for issues using Evidently test suites.
|
|
105
|
-
- `check_data_drift`: Check data for drift using Evidently metrics.
|
|
106
|
-
- `send_email_with_table`: Send email notifications with HTML tables for monitoring alerts.
|
|
@@ -1,88 +0,0 @@
|
|
|
1
|
-
# utilsds
|
|
2
|
-
|
|
3
|
-
Utilsds is a library that includes classes and functions used in data science projects such as:
|
|
4
|
-
|
|
5
|
-
- **algorithm**:
|
|
6
|
-
- `Algorithm`: Base class for fitting, training, and getting hyperparameters of machine learning models.
|
|
7
|
-
|
|
8
|
-
- **data_ops**:
|
|
9
|
-
- `DataOperations`: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
|
|
10
|
-
- BigQuery operations:
|
|
11
|
-
- `load_bq_data`: Load data from tables, views, and SQL files.
|
|
12
|
-
- `save_bq_view`, `save_bq_table`: Save views and tables.
|
|
13
|
-
- `load_bq_procedure`: Execute stored procedures.
|
|
14
|
-
- `load_bq_details`: Get table/view details and schema.
|
|
15
|
-
- `delete_bq_data`: Delete data with safety confirmations.
|
|
16
|
-
- `dry_run`: Perform dry runs to estimate query costs.
|
|
17
|
-
- Cloud Storage operations:
|
|
18
|
-
- `save_gcs_bucket`: Create buckets.
|
|
19
|
-
- `save_gcs_file`, `load_gcs_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
|
|
20
|
-
- Local file operations:
|
|
21
|
-
- `save_local_file`, `load_local_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
|
|
22
|
-
|
|
23
|
-
- **data_processing**:
|
|
24
|
-
- `SkewnessTransformer`: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
|
|
25
|
-
- `NullReplacer`: Replace null values in specified columns with configurable strategies.
|
|
26
|
-
- `ColumnDropper`: Drop specified columns from a DataFrame.
|
|
27
|
-
- `OutliersCleaner`: Clean outliers by clipping values outside specified percentile ranges.
|
|
28
|
-
- `CategoricalMapper`: Map values in categorical columns according to a specified mapping scheme.
|
|
29
|
-
- `NumericalMapper`: Convert numerical columns to categorical by binning.
|
|
30
|
-
- `Encoder`: One-hot encode categorical columns in the data.
|
|
31
|
-
- `Normalizer`: Normalize numerical columns using a provided scaler.
|
|
32
|
-
|
|
33
|
-
- **data_split**:
|
|
34
|
-
- `train_test_validation_split`: Split data into training, testing, and validation sets.
|
|
35
|
-
- `resample_X_y`: resample train data and target column.
|
|
36
|
-
|
|
37
|
-
- **ds_statistics**:
|
|
38
|
-
- `test_kruskal_wallis`: Perform the Kruskal-Wallis statistical test.
|
|
39
|
-
- `test_agosto_pearsona`: Test for normality using D'Agostino-Pearson test.
|
|
40
|
-
|
|
41
|
-
- **evaluate**:
|
|
42
|
-
- `ModelEvaluator`: Evaluate models and generate plots for diagnostics.
|
|
43
|
-
- `ShapExplainer`: Explain model predictions using SHAP values.
|
|
44
|
-
|
|
45
|
-
- **experiments**:
|
|
46
|
-
- `VertexExperiment`: Manage experiments with Vertex AI.
|
|
47
|
-
|
|
48
|
-
- **optuna**:
|
|
49
|
-
- `Optuna`: Optimize hyperparameters using Optuna.
|
|
50
|
-
|
|
51
|
-
- **metrics**:
|
|
52
|
-
- `Metrics`: Calculate metrics for both classification and regression models.
|
|
53
|
-
|
|
54
|
-
- **modeling**:
|
|
55
|
-
- `Modeling`: Manage modeling, metrics, and logging with Vertex AI.
|
|
56
|
-
|
|
57
|
-
- **Supervised**:
|
|
58
|
-
- `LazyClassifier`: A classifier that automatically trains and evaluates multiple models.
|
|
59
|
-
- `LazyRegressor`: A regressor that automatically trains and evaluates multiple models.
|
|
60
|
-
- `get_card_split`: Function to split data into card-like groups.
|
|
61
|
-
- `adjusted_rsquared`: Calculate adjusted R-squared for regression models.
|
|
62
|
-
|
|
63
|
-
- **visualization**:
|
|
64
|
-
- `MetricsPlot`: Compare metrics for different parameter values.
|
|
65
|
-
- `Radar`: Create radar plots for visualizing data.
|
|
66
|
-
- `cluster_characteristics`: Analyze cluster characteristics.
|
|
67
|
-
- `comparison_density`: Compare density distributions.
|
|
68
|
-
- `elbow_visualisation`: Visualize the elbow method for clustering.
|
|
69
|
-
- `describe_clusters_metrics`: Describe metrics for clusters.
|
|
70
|
-
- `category_null_variables`: Visualize null variables in categorical data.
|
|
71
|
-
- `normal_distr_plots`: Visualize normal distribution plots.
|
|
72
|
-
- `distplot_limitations`: Visualize limitations of distplot.
|
|
73
|
-
- `boxplot_limitations`: Visualize limitations of boxplot.
|
|
74
|
-
- `violinplot_limitations`: Visualize limitations of violinplot.
|
|
75
|
-
- `countplot_limitations`: Visualize limitations of countplot.
|
|
76
|
-
- `categorical_variable_perc`: Visualize percentage of categorical variables.
|
|
77
|
-
- `spearman_correlation`: Visualize spearman correlation.
|
|
78
|
-
- `calculate_crammers_v`: Calculate Crammer's V.
|
|
79
|
-
|
|
80
|
-
- **what_if_streamlit**:
|
|
81
|
-
- `ShapSaver`: Save SHAP explainer components for lazy loading in what-if analysis.
|
|
82
|
-
- `ColumnMetadataGenerator`: Generate column metadata from a DataFrame or CSV file.
|
|
83
|
-
|
|
84
|
-
- **monitoring**:
|
|
85
|
-
- `mapping`: Create column mapping from configuration file for Evidently.
|
|
86
|
-
- `test_data`: Test data for issues using Evidently test suites.
|
|
87
|
-
- `check_data_drift`: Check data for drift using Evidently metrics.
|
|
88
|
-
- `send_email_with_table`: Send email notifications with HTML tables for monitoring alerts.
|
|
@@ -1,106 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: utilsds-models
|
|
3
|
-
Version: 0.0.1
|
|
4
|
-
Summary: Solution for specific models
|
|
5
|
-
Author-email: DS Team <ds@sts.pl>
|
|
6
|
-
License: MIT License
|
|
7
|
-
Classifier: Programming Language :: Python :: 3
|
|
8
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
9
|
-
Classifier: Operating System :: OS Independent
|
|
10
|
-
Requires-Python: >=3.12
|
|
11
|
-
Description-Content-Type: text/markdown
|
|
12
|
-
Requires-Dist: pandas>=2.2.2
|
|
13
|
-
Requires-Dist: numpy>=1.26.0
|
|
14
|
-
Requires-Dist: scikit-learn>=1.5.0
|
|
15
|
-
Requires-Dist: matplotlib>=3.9.0
|
|
16
|
-
Provides-Extra: dev
|
|
17
|
-
Requires-Dist: pre-commit>=3.5.0; extra == "dev"
|
|
18
|
-
|
|
19
|
-
# utilsds
|
|
20
|
-
|
|
21
|
-
Utilsds is a library that includes classes and functions used in data science projects such as:
|
|
22
|
-
|
|
23
|
-
- **algorithm**:
|
|
24
|
-
- `Algorithm`: Base class for fitting, training, and getting hyperparameters of machine learning models.
|
|
25
|
-
|
|
26
|
-
- **data_ops**:
|
|
27
|
-
- `DataOperations`: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
|
|
28
|
-
- BigQuery operations:
|
|
29
|
-
- `load_bq_data`: Load data from tables, views, and SQL files.
|
|
30
|
-
- `save_bq_view`, `save_bq_table`: Save views and tables.
|
|
31
|
-
- `load_bq_procedure`: Execute stored procedures.
|
|
32
|
-
- `load_bq_details`: Get table/view details and schema.
|
|
33
|
-
- `delete_bq_data`: Delete data with safety confirmations.
|
|
34
|
-
- `dry_run`: Perform dry runs to estimate query costs.
|
|
35
|
-
- Cloud Storage operations:
|
|
36
|
-
- `save_gcs_bucket`: Create buckets.
|
|
37
|
-
- `save_gcs_file`, `load_gcs_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
|
|
38
|
-
- Local file operations:
|
|
39
|
-
- `save_local_file`, `load_local_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
|
|
40
|
-
|
|
41
|
-
- **data_processing**:
|
|
42
|
-
- `SkewnessTransformer`: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
|
|
43
|
-
- `NullReplacer`: Replace null values in specified columns with configurable strategies.
|
|
44
|
-
- `ColumnDropper`: Drop specified columns from a DataFrame.
|
|
45
|
-
- `OutliersCleaner`: Clean outliers by clipping values outside specified percentile ranges.
|
|
46
|
-
- `CategoricalMapper`: Map values in categorical columns according to a specified mapping scheme.
|
|
47
|
-
- `NumericalMapper`: Convert numerical columns to categorical by binning.
|
|
48
|
-
- `Encoder`: One-hot encode categorical columns in the data.
|
|
49
|
-
- `Normalizer`: Normalize numerical columns using a provided scaler.
|
|
50
|
-
|
|
51
|
-
- **data_split**:
|
|
52
|
-
- `train_test_validation_split`: Split data into training, testing, and validation sets.
|
|
53
|
-
- `resample_X_y`: resample train data and target column.
|
|
54
|
-
|
|
55
|
-
- **ds_statistics**:
|
|
56
|
-
- `test_kruskal_wallis`: Perform the Kruskal-Wallis statistical test.
|
|
57
|
-
- `test_agosto_pearsona`: Test for normality using D'Agostino-Pearson test.
|
|
58
|
-
|
|
59
|
-
- **evaluate**:
|
|
60
|
-
- `ModelEvaluator`: Evaluate models and generate plots for diagnostics.
|
|
61
|
-
- `ShapExplainer`: Explain model predictions using SHAP values.
|
|
62
|
-
|
|
63
|
-
- **experiments**:
|
|
64
|
-
- `VertexExperiment`: Manage experiments with Vertex AI.
|
|
65
|
-
|
|
66
|
-
- **optuna**:
|
|
67
|
-
- `Optuna`: Optimize hyperparameters using Optuna.
|
|
68
|
-
|
|
69
|
-
- **metrics**:
|
|
70
|
-
- `Metrics`: Calculate metrics for both classification and regression models.
|
|
71
|
-
|
|
72
|
-
- **modeling**:
|
|
73
|
-
- `Modeling`: Manage modeling, metrics, and logging with Vertex AI.
|
|
74
|
-
|
|
75
|
-
- **Supervised**:
|
|
76
|
-
- `LazyClassifier`: A classifier that automatically trains and evaluates multiple models.
|
|
77
|
-
- `LazyRegressor`: A regressor that automatically trains and evaluates multiple models.
|
|
78
|
-
- `get_card_split`: Function to split data into card-like groups.
|
|
79
|
-
- `adjusted_rsquared`: Calculate adjusted R-squared for regression models.
|
|
80
|
-
|
|
81
|
-
- **visualization**:
|
|
82
|
-
- `MetricsPlot`: Compare metrics for different parameter values.
|
|
83
|
-
- `Radar`: Create radar plots for visualizing data.
|
|
84
|
-
- `cluster_characteristics`: Analyze cluster characteristics.
|
|
85
|
-
- `comparison_density`: Compare density distributions.
|
|
86
|
-
- `elbow_visualisation`: Visualize the elbow method for clustering.
|
|
87
|
-
- `describe_clusters_metrics`: Describe metrics for clusters.
|
|
88
|
-
- `category_null_variables`: Visualize null variables in categorical data.
|
|
89
|
-
- `normal_distr_plots`: Visualize normal distribution plots.
|
|
90
|
-
- `distplot_limitations`: Visualize limitations of distplot.
|
|
91
|
-
- `boxplot_limitations`: Visualize limitations of boxplot.
|
|
92
|
-
- `violinplot_limitations`: Visualize limitations of violinplot.
|
|
93
|
-
- `countplot_limitations`: Visualize limitations of countplot.
|
|
94
|
-
- `categorical_variable_perc`: Visualize percentage of categorical variables.
|
|
95
|
-
- `spearman_correlation`: Visualize spearman correlation.
|
|
96
|
-
- `calculate_crammers_v`: Calculate Crammer's V.
|
|
97
|
-
|
|
98
|
-
- **what_if_streamlit**:
|
|
99
|
-
- `ShapSaver`: Save SHAP explainer components for lazy loading in what-if analysis.
|
|
100
|
-
- `ColumnMetadataGenerator`: Generate column metadata from a DataFrame or CSV file.
|
|
101
|
-
|
|
102
|
-
- **monitoring**:
|
|
103
|
-
- `mapping`: Create column mapping from configuration file for Evidently.
|
|
104
|
-
- `test_data`: Test data for issues using Evidently test suites.
|
|
105
|
-
- `check_data_drift`: Check data for drift using Evidently metrics.
|
|
106
|
-
- `send_email_with_table`: Send email notifications with HTML tables for monitoring alerts.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|