utilsds-models 0.0.1__tar.gz

utilsds_models-0.0.1/PKG-INFO

@@ -0,0 +1,106 @@
+Metadata-Version: 2.4
+Name: utilsds-models
+Version: 0.0.1
+Summary: Solution for specific models
+Author-email: DS Team <ds@sts.pl>
+License: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: pandas>=2.2.2
+Requires-Dist: numpy>=1.26.0
+Requires-Dist: scikit-learn>=1.5.0
+Requires-Dist: matplotlib>=3.9.0
+Provides-Extra: dev
+Requires-Dist: pre-commit>=3.5.0; extra == "dev"
+
+# utilsds
+
+Utilsds is a library that includes classes and functions used in data science projects such as:
+
+- **algorithm**:
+  - `Algorithm`: Base class for fitting, training, and getting hyperparameters of machine learning models.
+
+- **data_ops**:
+  - `DataOperations`: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
+  - BigQuery operations:
+    - `load_bq_data`: Load data from tables, views, and SQL files.
+    - `save_bq_view`, `save_bq_table`: Save views and tables.
+    - `load_bq_procedure`: Execute stored procedures.
+    - `load_bq_details`: Get table/view details and schema.
+    - `delete_bq_data`: Delete data with safety confirmations.
+    - `dry_run`: Perform dry runs to estimate query costs.
+  - Cloud Storage operations:
+    - `save_gcs_bucket`: Create buckets.
+    - `save_gcs_file`, `load_gcs_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
+  - Local file operations:
+    - `save_local_file`, `load_local_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
+
+- **data_processing**:
+  - `SkewnessTransformer`: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
+  - `NullReplacer`: Replace null values in specified columns with configurable strategies.
+  - `ColumnDropper`: Drop specified columns from a DataFrame.
+  - `OutliersCleaner`: Clean outliers by clipping values outside specified percentile ranges.
+  - `CategoricalMapper`: Map values in categorical columns according to a specified mapping scheme.
+  - `NumericalMapper`: Convert numerical columns to categorical by binning.
+  - `Encoder`: One-hot encode categorical columns in the data.
+  - `Normalizer`: Normalize numerical columns using a provided scaler.
+
+- **data_split**:
+  - `train_test_validation_split`: Split data into training, testing, and validation sets.
+  - `resample_X_y`: resample train data and target column.
+
+- **ds_statistics**:
+  - `test_kruskal_wallis`: Perform the Kruskal-Wallis statistical test.
+  - `test_agosto_pearsona`: Test for normality using D'Agostino-Pearson test.
+
+- **evaluate**:
+  - `ModelEvaluator`: Evaluate models and generate plots for diagnostics.
+  - `ShapExplainer`: Explain model predictions using SHAP values.
+
+- **experiments**:
+  - `VertexExperiment`: Manage experiments with Vertex AI.
+
+- **optuna**:
+  - `Optuna`: Optimize hyperparameters using Optuna.
+
+- **metrics**:
+  - `Metrics`: Calculate metrics for both classification and regression models.
+
+- **modeling**:
+  - `Modeling`: Manage modeling, metrics, and logging with Vertex AI.
+
+- **Supervised**:
+  - `LazyClassifier`: A classifier that automatically trains and evaluates multiple models.
+  - `LazyRegressor`: A regressor that automatically trains and evaluates multiple models.
+  - `get_card_split`: Function to split data into card-like groups.
+  - `adjusted_rsquared`: Calculate adjusted R-squared for regression models.
+
+- **visualization**:
+  - `MetricsPlot`: Compare metrics for different parameter values.
+  - `Radar`: Create radar plots for visualizing data.
+  - `cluster_characteristics`: Analyze cluster characteristics.
+  - `comparison_density`: Compare density distributions.
+  - `elbow_visualisation`: Visualize the elbow method for clustering.
+  - `describe_clusters_metrics`: Describe metrics for clusters.
+  - `category_null_variables`: Visualize null variables in categorical data.
+  - `normal_distr_plots`: Visualize normal distribution plots.
+  - `distplot_limitations`: Visualize limitations of distplot.
+  - `boxplot_limitations`: Visualize limitations of boxplot.
+  - `violinplot_limitations`: Visualize limitations of violinplot.
+  - `countplot_limitations`: Visualize limitations of countplot.
+  - `categorical_variable_perc`: Visualize percentage of categorical variables.
+  - `spearman_correlation`: Visualize spearman correlation.
+  - `calculate_crammers_v`: Calculate Crammer's V.
+
+- **what_if_streamlit**:
+  - `ShapSaver`: Save SHAP explainer components for lazy loading in what-if analysis.
+  - `ColumnMetadataGenerator`: Generate column metadata from a DataFrame or CSV file.
+
+- **monitoring**:
+  - `mapping`: Create column mapping from configuration file for Evidently.
+  - `test_data`: Test data for issues using Evidently test suites.
+  - `check_data_drift`: Check data for drift using Evidently metrics.
+  - `send_email_with_table`: Send email notifications with HTML tables for monitoring alerts.

utilsds_models-0.0.1/README.md

@@ -0,0 +1,18 @@
+# utils
+
+Rozwiązanie zawierające klasy i funkcje wykorzystywane podczas projektów.
+Struktura repozytorium stworzono tak, aby udostępnić ją jako [utilsds](https://pypi.org/project/utilsds/).
+
+Biblioteka została stworzona na podstawie [tej instrukcji](https://www.turing.com/kb/how-to-create-pypi-packages).
+
+Aby zaktualizować klasy/funkcje pierwsze na stronie https://pypi.org/ należy:
+- stworzyć konto
+- dodać 2FA z wykorzystaniem aplikacji Google Authenticator
+- stworzyć plik home/.pypirc (najlepiej zarządzać tym plikiem z poziomu konsoli, ponieważ pozostaje on ukryty)
+- wygenerować w ustawieniach `API tokens` a następnie wkleić wygenerowany kod do wcześniej stworzonego pliku .pypirc (zgodnie z instrukacjami podanymi w ustawieniach).
+- wprowadzić aktualizacje
+- dokonać modyfikacji w pliku `pyproject.toml` (szczególnie zwrócić uwagę na inkrementację numeracji wersji w sekcji `[project]`)
+- jeżeli jest potrzebne, zainstalować narzędzia build: `pip install --upgrade build twine`
+- wykonać komendę `python -m build`
+- wykonać komendę `twine upload --skip-existing dist/*`
+- spushować zmiany na gitlaba

utilsds_models-0.0.1/docs/ALTERNATIVE_README.md

@@ -0,0 +1,88 @@
+# utilsds
+
+Utilsds is a library that includes classes and functions used in data science projects such as:
+
+- **algorithm**:
+  - `Algorithm`: Base class for fitting, training, and getting hyperparameters of machine learning models.
+
+- **data_ops**:
+  - `DataOperations`: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
+  - BigQuery operations:
+    - `load_bq_data`: Load data from tables, views, and SQL files.
+    - `save_bq_view`, `save_bq_table`: Save views and tables.
+    - `load_bq_procedure`: Execute stored procedures.
+    - `load_bq_details`: Get table/view details and schema.
+    - `delete_bq_data`: Delete data with safety confirmations.
+    - `dry_run`: Perform dry runs to estimate query costs.
+  - Cloud Storage operations:
+    - `save_gcs_bucket`: Create buckets.
+    - `save_gcs_file`, `load_gcs_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
+  - Local file operations:
+    - `save_local_file`, `load_local_file`: Save and load files (.pkl, .json, .csv, .html, .sql).
+
+- **data_processing**:
+  - `SkewnessTransformer`: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
+  - `NullReplacer`: Replace null values in specified columns with configurable strategies.
+  - `ColumnDropper`: Drop specified columns from a DataFrame.
+  - `OutliersCleaner`: Clean outliers by clipping values outside specified percentile ranges.
+  - `CategoricalMapper`: Map values in categorical columns according to a specified mapping scheme.
+  - `NumericalMapper`: Convert numerical columns to categorical by binning.
+  - `Encoder`: One-hot encode categorical columns in the data.
+  - `Normalizer`: Normalize numerical columns using a provided scaler.
+
+- **data_split**:
+  - `train_test_validation_split`: Split data into training, testing, and validation sets.
+  - `resample_X_y`: resample train data and target column.
+
+- **ds_statistics**:
+  - `test_kruskal_wallis`: Perform the Kruskal-Wallis statistical test.
+  - `test_agosto_pearsona`: Test for normality using D'Agostino-Pearson test.
+
+- **evaluate**:
+  - `ModelEvaluator`: Evaluate models and generate plots for diagnostics.
+  - `ShapExplainer`: Explain model predictions using SHAP values.
+
+- **experiments**:
+  - `VertexExperiment`: Manage experiments with Vertex AI.
+
+- **optuna**:
+  - `Optuna`: Optimize hyperparameters using Optuna.
+
+- **metrics**:
+  - `Metrics`: Calculate metrics for both classification and regression models.
+
+- **modeling**:
+  - `Modeling`: Manage modeling, metrics, and logging with Vertex AI.
+
+- **Supervised**:
+  - `LazyClassifier`: A classifier that automatically trains and evaluates multiple models.
+  - `LazyRegressor`: A regressor that automatically trains and evaluates multiple models.
+  - `get_card_split`: Function to split data into card-like groups.
+  - `adjusted_rsquared`: Calculate adjusted R-squared for regression models.
+
+- **visualization**:
+  - `MetricsPlot`: Compare metrics for different parameter values.
+  - `Radar`: Create radar plots for visualizing data.
+  - `cluster_characteristics`: Analyze cluster characteristics.
+  - `comparison_density`: Compare density distributions.
+  - `elbow_visualisation`: Visualize the elbow method for clustering.
+  - `describe_clusters_metrics`: Describe metrics for clusters.
+  - `category_null_variables`: Visualize null variables in categorical data.
+  - `normal_distr_plots`: Visualize normal distribution plots.
+  - `distplot_limitations`: Visualize limitations of distplot.
+  - `boxplot_limitations`: Visualize limitations of boxplot.
+  - `violinplot_limitations`: Visualize limitations of violinplot.
+  - `countplot_limitations`: Visualize limitations of countplot.
+  - `categorical_variable_perc`: Visualize percentage of categorical variables.
+  - `spearman_correlation`: Visualize spearman correlation.
+  - `calculate_crammers_v`: Calculate Crammer's V.
+
+- **what_if_streamlit**:
+  - `ShapSaver`: Save SHAP explainer components for lazy loading in what-if analysis.
+  - `ColumnMetadataGenerator`: Generate column metadata from a DataFrame or CSV file.
+
+- **monitoring**:
+  - `mapping`: Create column mapping from configuration file for Evidently.
+  - `test_data`: Test data for issues using Evidently test suites.
+  - `check_data_drift`: Check data for drift using Evidently metrics.
+  - `send_email_with_table`: Send email notifications with HTML tables for monitoring alerts.

utilsds_models-0.0.1/pyproject.toml

@@ -0,0 +1,35 @@
+[build-system]
+requires = ["setuptools>=80.0.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "utilsds-models"
+version = "0.0.1"
+description = "Solution for specific models"
+readme = {file = "docs/ALTERNATIVE_README.md", content-type = "text/markdown"}
+requires-python = ">=3.12"
+license = {text = "MIT License"}
+authors = [
+    {name = "DS Team", email = "ds@sts.pl"}
+]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+]
+dependencies = [
+    "pandas>=2.2.2",
+    "numpy>=1.26.0",
+    "scikit-learn>=1.5.0",
+    "matplotlib>=3.9.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pre-commit>=3.5.0",
+]
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["utilsds-models*"]
+exclude = ["tests*", "docs*"]

utilsds_models-0.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+