pypreml 0.1.0__tar.gz

pypreml-0.1.0/PKG-INFO

@@ -0,0 +1,223 @@
+Metadata-Version: 2.4
+Name: pypreml
+Version: 0.1.0
+Summary: Production‑ready machine learning toolkit for EDA, preprocessing, and modelling.
+Author-email: Ali Nazer <alinazer30@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/alinazer30/preml
+Project-URL: Repository, https://github.com/alinazer30/preml
+Project-URL: Issues, https://github.com/alinazer30/preml/issues
+Keywords: machine learning,eda,preprocessing,feature engineering,scikit-learn
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: pandas>=1.3.0
+Requires-Dist: numpy>=1.21.0
+Requires-Dist: scipy>=1.7.0
+Requires-Dist: matplotlib>=3.4.0
+Requires-Dist: seaborn>=0.11.0
+Requires-Dist: scikit-learn>=1.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov>=3.0; extra == "dev"
+
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+# PreML
+
+PreML is a modular Python library for exploratory data analysis, statistical recommendations, preprocessing pipeline generation, and feature engineering guidance for tabular datasets.
+
+The source code now lives inside the `preml/` package directory, which keeps the project organized and matches the installed namespace.
+
+## What This Project Does
+
+- Computes statistical facts from tabular data.
+- Turns those facts into evidence-based recommendations.
+- Builds scikit-learn compatible preprocessing pipelines.
+- Suggests feature engineering ideas from measurable patterns.
+- Keeps analysis, recommendations, and preprocessing separated into focused modules.
+
+## File Guide
+
+Use this map to understand how the current files fit together:
+
+| File | Purpose |
+| --- | --- |
+| [preml/__init__.py](preml/__init__.py) | Package exports and top-level public API. |
+| [preml/config.py](preml/config.py) | Central thresholds and defaults via `MLToolkitConfig`. |
+| [preml/statistics_engine.py](preml/statistics_engine.py) | Computes dataset facts such as profiles, missingness, outliers, and correlations. |
+| [preml/recommendation_engine.py](preml/recommendation_engine.py) | Converts statistics into recommendations and model guidance. |
+| [preml/eda.py](preml/eda.py) | Orchestrates the full EDA flow and produces summaries. |
+| [preml/preprocessing.py](preml/preprocessing.py) | Builds scikit-learn preprocessing pipelines from EDA results. |
+| [preml/feature_engineering.py](preml/feature_engineering.py) | Proposes new features from statistical evidence. |
+| [preml/visualization.py](preml/visualization.py) | Plotting and visual analysis helpers. |
+| [preml/schema.py](preml/schema.py) | Shared dataclasses used across the library. |
+| [preml/exceptions.py](preml/exceptions.py) | Custom exception hierarchy. |
+| [requirements.txt](requirements.txt) | Runtime dependencies for local installs. |
+| [pyproject.toml](pyproject.toml) | Build metadata and packaging configuration. |
+| [.gitignore](.gitignore) | Ignores caches, build artifacts, notebooks, and local environments. |
+| [tests/](tests/) | Test suite. |
+
+## Highlights
+
+- Automatic EDA for missing values, outliers, correlation, and feature profiling.
+- Evidence-based recommendations for imputation, encoding, scaling, transformation, and feature selection.
+- Scikit-learn compatible preprocessing pipelines tailored to feature types and data quality signals.
+- Feature engineering suggestions grounded in statistical evidence rather than column names.
+- Clean module boundaries that make the package easier to test, extend, and maintain.
+
+## Installation
+
+### From source
+
+```bash
+git clone https://github.com/alinazer30/preml.git
+cd preml
+python -m pip install -e .
+```
+
+### Dependencies
+
+The project targets Python 3.9+ and uses:
+
+- pandas
+- numpy
+- scipy
+- matplotlib
+- seaborn
+- scikit-learn
+
+## Quick Start
+
+```python
+import pandas as pd
+
+from preml.eda import EDAAnalyzer, quick_eda
+from preml.preprocessing import PreprocessingBuilder
+
+df = pd.read_csv("your_dataset.csv")
+
+# Run the full analysis
+analysis = quick_eda(df, target="target_column")
+
+# Or use the orchestrator directly
+analyzer = EDAAnalyzer(df, target="target_column")
+analysis = analyzer.run()
+print(analyzer.summary())
+
+# Build a preprocessing pipeline from the analysis output
+builder = PreprocessingBuilder(analysis)
+pipeline = builder.build_pipeline()
+X = builder.fit_transform(df)
+```
+
+## Typical Workflow
+
+1. Load your tabular dataset into a pandas DataFrame.
+2. Run `EDAAnalyzer` or `quick_eda` to compute facts and recommendations.
+3. Build a preprocessing pipeline with `PreprocessingBuilder`.
+4. Inspect the returned schema objects and recommendations to guide modelling decisions.
+
+## Public API
+
+The package exposes the most common shared types at the package root:
+
+```python
+from preml import MLToolkitConfig, default_config
+from preml import DataValidationError, RecommendationError
+from preml import FeatureProfile, Recommendation, TargetProfile
+```
+
+Useful module entry points:
+
+- `preml.eda.EDAAnalyzer`
+- `preml.eda.quick_eda`
+- `preml.preprocessing.PreprocessingBuilder`
+- `preml.recommendation_engine.RecommendationEngine`
+- `preml.statistics_engine.StatisticsEngine`
+
+## Configuration
+
+All thresholds and defaults are defined in `MLToolkitConfig`.
+
+```python
+from preml.config import MLToolkitConfig
+from preml.eda import EDAAnalyzer
+
+config = MLToolkitConfig(
+    missing_threshold=0.2,
+    correlation_threshold=0.85,
+    random_state=42,
+)
+
+analyzer = EDAAnalyzer(df, target="target_column", config=config)
+analysis = analyzer.run()
+```
+
+Key configuration values include:
+
+- `missing_threshold`
+- `high_cardinality_threshold`
+- `max_unique_for_categorical_like`
+- `correlation_threshold`
+- `skewness_threshold`
+- `outlier_method`
+- `iqr_multiplier`
+- `random_state`
+
+## GitHub Workflow
+
+Recommended commands to publish the project cleanly:
+
+```bash
+git init
+git add README.md pyproject.toml requirements.txt .gitignore preml/ tests/
+git status
+git commit -m "Prepare packaged ML toolkit"
+git branch -M main
+git remote add origin https://github.com/<your-username>/preml.git
+git push -u origin main
+```
+
+Best practices when pushing:
+
+- Commit only the files you intended to change.
+- Run the test suite before pushing.
+- Keep the README synchronized with the real repository layout.
+- Avoid committing virtual environments, caches, notebooks, and build artifacts.
+- Prefer small, descriptive commits such as `docs: reorganize README` or `fix: align packaging layout`.
+
+## Development
+
+Run the test suite with:
+
+```bash
+pytest tests/
+```
+
+If you want to sanity-check the edited modules locally, you can also run:
+
+```bash
+python -m py_compile preml/preprocessing.py preml/eda.py preml/recommendation_engine.py
+```
+
+## Contributing
+
+Contributions are welcome. Keep changes aligned with the existing architecture, preserve the separation between facts and recommendations, and add tests when you change behaviour.
+
+## License
+
+MIT License.
+
+## Author
+
+Ali Nazer – alinazer30@gmail.com

pypreml-0.1.0/README.md

@@ -0,0 +1,190 @@
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+# PreML
+
+PreML is a modular Python library for exploratory data analysis, statistical recommendations, preprocessing pipeline generation, and feature engineering guidance for tabular datasets.
+
+The source code now lives inside the `preml/` package directory, which keeps the project organized and matches the installed namespace.
+
+## What This Project Does
+
+- Computes statistical facts from tabular data.
+- Turns those facts into evidence-based recommendations.
+- Builds scikit-learn compatible preprocessing pipelines.
+- Suggests feature engineering ideas from measurable patterns.
+- Keeps analysis, recommendations, and preprocessing separated into focused modules.
+
+## File Guide
+
+Use this map to understand how the current files fit together:
+
+| File | Purpose |
+| --- | --- |
+| [preml/__init__.py](preml/__init__.py) | Package exports and top-level public API. |
+| [preml/config.py](preml/config.py) | Central thresholds and defaults via `MLToolkitConfig`. |
+| [preml/statistics_engine.py](preml/statistics_engine.py) | Computes dataset facts such as profiles, missingness, outliers, and correlations. |
+| [preml/recommendation_engine.py](preml/recommendation_engine.py) | Converts statistics into recommendations and model guidance. |
+| [preml/eda.py](preml/eda.py) | Orchestrates the full EDA flow and produces summaries. |
+| [preml/preprocessing.py](preml/preprocessing.py) | Builds scikit-learn preprocessing pipelines from EDA results. |
+| [preml/feature_engineering.py](preml/feature_engineering.py) | Proposes new features from statistical evidence. |
+| [preml/visualization.py](preml/visualization.py) | Plotting and visual analysis helpers. |
+| [preml/schema.py](preml/schema.py) | Shared dataclasses used across the library. |
+| [preml/exceptions.py](preml/exceptions.py) | Custom exception hierarchy. |
+| [requirements.txt](requirements.txt) | Runtime dependencies for local installs. |
+| [pyproject.toml](pyproject.toml) | Build metadata and packaging configuration. |
+| [.gitignore](.gitignore) | Ignores caches, build artifacts, notebooks, and local environments. |
+| [tests/](tests/) | Test suite. |
+
+## Highlights
+
+- Automatic EDA for missing values, outliers, correlation, and feature profiling.
+- Evidence-based recommendations for imputation, encoding, scaling, transformation, and feature selection.
+- Scikit-learn compatible preprocessing pipelines tailored to feature types and data quality signals.
+- Feature engineering suggestions grounded in statistical evidence rather than column names.
+- Clean module boundaries that make the package easier to test, extend, and maintain.
+
+## Installation
+
+### From source
+
+```bash
+git clone https://github.com/alinazer30/preml.git
+cd preml
+python -m pip install -e .
+```
+
+### Dependencies
+
+The project targets Python 3.9+ and uses:
+
+- pandas
+- numpy
+- scipy
+- matplotlib
+- seaborn
+- scikit-learn
+
+## Quick Start
+
+```python
+import pandas as pd
+
+from preml.eda import EDAAnalyzer, quick_eda
+from preml.preprocessing import PreprocessingBuilder
+
+df = pd.read_csv("your_dataset.csv")
+
+# Run the full analysis
+analysis = quick_eda(df, target="target_column")
+
+# Or use the orchestrator directly
+analyzer = EDAAnalyzer(df, target="target_column")
+analysis = analyzer.run()
+print(analyzer.summary())
+
+# Build a preprocessing pipeline from the analysis output
+builder = PreprocessingBuilder(analysis)
+pipeline = builder.build_pipeline()
+X = builder.fit_transform(df)
+```
+
+## Typical Workflow
+
+1. Load your tabular dataset into a pandas DataFrame.
+2. Run `EDAAnalyzer` or `quick_eda` to compute facts and recommendations.
+3. Build a preprocessing pipeline with `PreprocessingBuilder`.
+4. Inspect the returned schema objects and recommendations to guide modelling decisions.
+
+## Public API
+
+The package exposes the most common shared types at the package root:
+
+```python
+from preml import MLToolkitConfig, default_config
+from preml import DataValidationError, RecommendationError
+from preml import FeatureProfile, Recommendation, TargetProfile
+```
+
+Useful module entry points:
+
+- `preml.eda.EDAAnalyzer`
+- `preml.eda.quick_eda`
+- `preml.preprocessing.PreprocessingBuilder`
+- `preml.recommendation_engine.RecommendationEngine`
+- `preml.statistics_engine.StatisticsEngine`
+
+## Configuration
+
+All thresholds and defaults are defined in `MLToolkitConfig`.
+
+```python
+from preml.config import MLToolkitConfig
+from preml.eda import EDAAnalyzer
+
+config = MLToolkitConfig(
+    missing_threshold=0.2,
+    correlation_threshold=0.85,
+    random_state=42,
+)
+
+analyzer = EDAAnalyzer(df, target="target_column", config=config)
+analysis = analyzer.run()
+```
+
+Key configuration values include:
+
+- `missing_threshold`
+- `high_cardinality_threshold`
+- `max_unique_for_categorical_like`
+- `correlation_threshold`
+- `skewness_threshold`
+- `outlier_method`
+- `iqr_multiplier`
+- `random_state`
+
+## GitHub Workflow
+
+Recommended commands to publish the project cleanly:
+
+```bash
+git init
+git add README.md pyproject.toml requirements.txt .gitignore preml/ tests/
+git status
+git commit -m "Prepare packaged ML toolkit"
+git branch -M main
+git remote add origin https://github.com/<your-username>/preml.git
+git push -u origin main
+```
+
+Best practices when pushing:
+
+- Commit only the files you intended to change.
+- Run the test suite before pushing.
+- Keep the README synchronized with the real repository layout.
+- Avoid committing virtual environments, caches, notebooks, and build artifacts.
+- Prefer small, descriptive commits such as `docs: reorganize README` or `fix: align packaging layout`.
+
+## Development
+
+Run the test suite with:
+
+```bash
+pytest tests/
+```
+
+If you want to sanity-check the edited modules locally, you can also run:
+
+```bash
+python -m py_compile preml/preprocessing.py preml/eda.py preml/recommendation_engine.py
+```
+
+## Contributing
+
+Contributions are welcome. Keep changes aligned with the existing architecture, preserve the separation between facts and recommendations, and add tests when you change behaviour.
+
+## License
+
+MIT License.
+
+## Author
+
+Ali Nazer – alinazer30@gmail.com

pypreml-0.1.0/preml/__init__.py

@@ -0,0 +1,87 @@
+# Copyright (c) 2026 Ali Nazer
+# Licensed under the MIT License.
+# See the LICENSE file in the project root for license information.
+"""
+preml — A professional machine learning toolkit for EDA,
+preprocessing, feature engineering, and modeling.
+
+The package is organized into clearly separated modules, each
+responsible for a single domain concern:
+
+- config          : Central configuration and thresholds.
+- exceptions      : Custom exception hierarchy.
+- schema          : Strongly‑typed data models (dataclasses).
+- statistics_engine: Extracts statistical facts from data.
+- recommendation_engine: Generates evidence‑based recommendations.
+- eda             : Orchestrates analysis and produces insights.
+- visualization   : Creates plots (statistics‑free).
+- preprocessing   : Builds sklearn‑compatible pipelines.
+- feature_engineering: Suggests and creates new features.
+- model_utils     : Baseline models, cross‑validation, metrics.
+- report          : Generates reports in various formats.
+"""
+
+__version__ = "0.1.0"
+__author__ = "Your Name <your.email@example.com>"
+
+# Expose the most commonly used classes at package level
+from preml.config import MLToolkitConfig, default_config
+from preml.exceptions import (
+    MLToolkitError,
+    DataValidationError,
+    StatisticsError,
+    RecommendationError,
+    PreprocessingError,
+    FeatureEngineeringError,
+    ModelError,
+    ReportError,
+    VisualizationError,
+)
+from preml.schema import (
+    DatasetMetadata,
+    DuplicateReport,
+    InfiniteReport,
+    MissingColumnReport,
+    MissingReport,
+    OutlierReport,
+    NumericDistributionProfile,
+    CategoricalProfile,
+    FeatureProfile,
+    CorrelationPair,
+    TargetProfile,
+    Evidence,
+    Recommendation,
+    PipelineSuggestion,
+    ModelRecommendation,
+)
+
+__all__ = [
+    "MLToolkitConfig",
+    "default_config",
+    # Exceptions
+    "MLToolkitError",
+    "DataValidationError",
+    "StatisticsError",
+    "RecommendationError",
+    "PreprocessingError",
+    "FeatureEngineeringError",
+    "ModelError",
+    "ReportError",
+    "VisualizationError",
+    # Schema
+    "DatasetMetadata",
+    "DuplicateReport",
+    "InfiniteReport",
+    "MissingColumnReport",
+    "MissingReport",
+    "OutlierReport",
+    "NumericDistributionProfile",
+    "CategoricalProfile",
+    "FeatureProfile",
+    "CorrelationPair",
+    "TargetProfile",
+    "Evidence",
+    "Recommendation",
+    "PipelineSuggestion",
+    "ModelRecommendation",
+]

pypreml-0.1.0/preml/config.py

@@ -0,0 +1,115 @@
+"""Central configuration module for preml.
+
+This module defines all tunable parameters, thresholds, and defaults
+used throughout the library. It is designed to be imported anywhere
+without creating circular dependencies and does not rely on any
+heavy external packages (only the standard library).
+"""
+
+from dataclasses import dataclass, field
+from typing import Literal, Optional
+
+
+@dataclass
+class MLToolkitConfig:
+    """Immutable configuration container for preml.
+
+    All parameters have sensible defaults, but can be overridden
+    by creating a new instance or modifying attributes directly.
+
+    Attributes:
+        missing_threshold: Ratio above which a column is flagged as
+            having high missingness (0.0 to 1.0).
+        high_cardinality_threshold: Number of unique values above which
+            a categorical column is considered high cardinality.
+        max_unique_for_categorical_like: Maximum unique values for a
+            numeric column to be considered as categorical-like.
+        correlation_threshold: Absolute correlation coefficient above
+            which a pair of features is flagged as highly correlated.
+        skewness_threshold: Absolute skewness above which a distribution
+            is considered heavily skewed.
+        kurtosis_threshold: Absolute kurtosis above which a distribution
+            is considered heavily tailed (not currently used).
+        cv_threshold: Coefficient of Variation above which a feature
+            is marked as highly dispersed.
+        zero_percent_threshold: Percentage of zeros in a numeric column
+            that triggers a "consider binary flag" suggestion.
+        negative_percent_threshold: Similar for negative values.
+        outlier_method: Method used for outlier detection.
+            'iqr': Interquartile Range (robust).
+            'zscore': Z‑score (sensitive to skew, generally discouraged).
+        iqr_multiplier: Multiplier for IQR bounds (default 1.5).
+        zscore_threshold: Threshold for Z‑score outlier detection.
+        constant_variance_threshold: Variance below this value is
+            considered constant/quasi‑constant.
+        enable_feature_engineering: If True, the recommendation engine
+            may suggest feature engineering techniques (ratios, interactions,
+            etc.). They are always based on statistical evidence, never on
+            column names.
+        max_plot_cols: Maximum number of individual feature plots to
+            generate in one go (avoid overwhelming the user).
+        plot_style: Seaborn style to use for all visualizations.
+        color_palette: Seaborn color palette.
+        figure_size: Default (width, height) for main figures.
+        random_state: Random seed for reproducibility in imputation,
+            encoding, etc.
+    """
+
+    # ----------------------------------------------------------------
+    # Missingness & Data Quality
+    # ----------------------------------------------------------------
+    missing_threshold: float = 0.4
+    high_cardinality_threshold: int = 50
+    max_unique_for_categorical_like: int = 15
+
+    # ----------------------------------------------------------------
+    # Correlation & Distribution
+    # ----------------------------------------------------------------
+    correlation_threshold: float = 0.9
+    skewness_threshold: float = 1.5
+    kurtosis_threshold: float = 4.0        # Not actively used
+    cv_threshold: float = 2.0
+    zero_percent_threshold: float = 50.0
+    negative_percent_threshold: float = 5.0
+
+    # ----------------------------------------------------------------
+    # Outlier Detection
+    # ----------------------------------------------------------------
+    outlier_method: Literal["iqr", "zscore"] = "iqr"
+    iqr_multiplier: float = 1.5
+    zscore_threshold: float = 3.0
+
+    # ----------------------------------------------------------------
+    # Variance & Constants
+    # ----------------------------------------------------------------
+    constant_variance_threshold: float = 1e-6
+
+    # ----------------------------------------------------------------
+    # Feature Engineering Toggle
+    # ----------------------------------------------------------------
+    enable_feature_engineering: bool = True
+
+    # ----------------------------------------------------------------
+    # Visualization
+    # ----------------------------------------------------------------
+    max_plot_cols: int = 20
+    plot_style: str = "whitegrid"
+    color_palette: str = "muted"
+    figure_size: tuple = (12, 6)
+
+    # ----------------------------------------------------------------
+    # Reproducibility
+    # ----------------------------------------------------------------
+    random_state: int = 42
+
+    # ----------------------------------------------------------------
+    # Additional settings may be added in the future.
+    # ----------------------------------------------------------------
+
+
+# ----------------------------------------------------------------------
+# Default configuration instance
+# ----------------------------------------------------------------------
+# Users can import this singleton and modify attributes at runtime,
+# or pass their own MLToolkitConfig instance to functions that accept one.
+default_config = MLToolkitConfig()