imperfekt 0.2.3__tar.gz

imperfekt-0.2.3/.gitignore

@@ -0,0 +1,25 @@
+# Data
+data/
+*.parquet
+*.csv
+*.sas7bdat
+
+# Python stuff
+/**/__pycache__/
+*$py.class
+**.egg-info
+dist/
+build/
+.eggs/
+
+# Virtual environments
+.venv/
+venv/
+
+# etc
+.vscode
+.idea/
+.ruff_cache
+.ipynb_checkpoints/
+.DS_Store
+Thumbs.db

imperfekt-0.2.3/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Tamara Krafft
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

imperfekt-0.2.3/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.4
+Name: imperfekt
+Version: 0.2.3
+Summary: A framework to analyze imperfections (missingness, noise) in time-series datasets.
+Project-URL: Homepage, https://github.com/krafftta/imperfekt
+Project-URL: Repository, https://github.com/krafftta/imperfekt
+Project-URL: Issues, https://github.com/krafftta/imperfekt/issues
+Author: Tamara Krafft
+License-Expression: MIT
+License-File: LICENSE
+Keywords: data-quality,healthcare,missing-data,missingness-analysis,polars,time-series
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Healthcare Industry
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: plotly>=5.18.0
+Requires-Dist: polars>=0.20.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: scipy>=1.13.0
+Provides-Extra: dev
+Requires-Dist: kaleido>=0.2.1; extra == 'dev'
+Requires-Dist: matplotlib>=3.8.0; extra == 'dev'
+Requires-Dist: missingno>=0.5.2; extra == 'dev'
+Requires-Dist: mypy>=1.7.0; extra == 'dev'
+Requires-Dist: pre-commit>=3.5.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
+Requires-Dist: pytest>=7.4.0; extra == 'dev'
+Requires-Dist: ruff>=0.1.0; extra == 'dev'
+Requires-Dist: scikit-posthocs>=0.9.0; extra == 'dev'
+Requires-Dist: statsmodels>=0.14.0; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
+Requires-Dist: mkdocs>=1.5.0; extra == 'docs'
+Requires-Dist: mkdocstrings[python]>=0.24.0; extra == 'docs'
+Provides-Extra: full
+Requires-Dist: kaleido>=0.2.1; extra == 'full'
+Requires-Dist: matplotlib>=3.8.0; extra == 'full'
+Requires-Dist: missingno>=0.5.2; extra == 'full'
+Requires-Dist: scikit-posthocs>=0.9.0; extra == 'full'
+Requires-Dist: statsmodels>=0.14.0; extra == 'full'
+Provides-Extra: stats
+Requires-Dist: scikit-posthocs>=0.9.0; extra == 'stats'
+Requires-Dist: statsmodels>=0.14.0; extra == 'stats'
+Provides-Extra: viz
+Requires-Dist: kaleido>=0.2.1; extra == 'viz'
+Requires-Dist: matplotlib>=3.8.0; extra == 'viz'
+Description-Content-Type: text/markdown
+
+# Imperfekt - Understanding Data Imperfections in Time-Series
+
+[![PyPI version](https://badge.fury.io/py/imperfekt.svg)](https://badge.fury.io/py/imperfekt)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+
+A comprehensive analysis toolkit for studying "imperfect" data patterns in time-series datasets.
+Imperfection refers to missingness, noise, and other data quality issues that can be indicated using a binary mask.
+
+## Overview
+
+This library provides tools to analyze data quality issues in time-series data, including:
+- **Intravariable analysis** of imperfection patterns for individual variables
+- **Intervariable analysis** of co-occurring imperfections across multiple parameters
+- **Feature generation** based on missingness patterns for downstream ML tasks
+
+## Installation
+
+Install the library using `pip`:
+
+```bash
+pip install imperfekt
+```
+
+For additional features:
+```bash
+# With visualization export support
+pip install imperfekt[viz]
+
+# With statistical tests
+pip install imperfekt[stats]
+
+# Full installation
+pip install imperfekt[full]
+```
+
+## Quick Start
+
+```python
+import polars as pl
+from imperfekt import Imperfekt, FeatureGenerator
+
+# Load your time-series data
+df = pl.read_parquet("your_data.parquet")
+
+# Run full imperfection analysis
+analyzer = Imperfekt(
+    df=df,
+    id_col="id",           # Unique identifier column
+    clock_col="clock",     # Timestamp column
+    cols=["var1", "var2"], # Variables to analyze
+    save_path="./results"
+)
+results = analyzer.run()
+
+# Or generate missingness-aware features for ML
+fg = FeatureGenerator(
+    df=df,
+    id_col="id",
+    clock_col="clock",
+    variable_cols=["var1", "var2"]
+)
+features_df = fg.add_binary_masks().add_temporal_features().df
+```
+
+## Library Structure
+
+```
+imperfekt/
+├── analysis/
+│   ├── preliminary/     # Basic data exploration
+│   ├── intravariable/      # Single variable analysis
+│   ├── intervariable/    # Multi-variable patterns
+│   └── utils/           # Shared utilities
+├── features/            # Feature engineering
+│   ├── core.py          # FeatureGenerator class
+│   ├── temporal.py      # Time-based features
+│   └── interaction.py   # Variable interactions
+└── config/              # Default settings
+```
+
+## Data Format
+
+The library expects time-series data with the following structure:
+
+| Column | Description |
+|--------|-------------|
+| `id` | Unique identifier for each time-series (e.g., patient, sensor) |
+| `clock` | Timestamp for each observation |
+| `var1`, `var2`, ... | Variables to analyze |
+
+## Key Dependencies
+
+- **polars**: High-performance data processing
+- **plotly**: Interactive visualizations
+- **scipy**: Statistical computations
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+

imperfekt-0.2.3/README.md

@@ -0,0 +1,105 @@
+# Imperfekt - Understanding Data Imperfections in Time-Series
+
+[![PyPI version](https://badge.fury.io/py/imperfekt.svg)](https://badge.fury.io/py/imperfekt)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+
+A comprehensive analysis toolkit for studying "imperfect" data patterns in time-series datasets.
+Imperfection refers to missingness, noise, and other data quality issues that can be indicated using a binary mask.
+
+## Overview
+
+This library provides tools to analyze data quality issues in time-series data, including:
+- **Intravariable analysis** of imperfection patterns for individual variables
+- **Intervariable analysis** of co-occurring imperfections across multiple parameters
+- **Feature generation** based on missingness patterns for downstream ML tasks
+
+## Installation
+
+Install the library using `pip`:
+
+```bash
+pip install imperfekt
+```
+
+For additional features:
+```bash
+# With visualization export support
+pip install imperfekt[viz]
+
+# With statistical tests
+pip install imperfekt[stats]
+
+# Full installation
+pip install imperfekt[full]
+```
+
+## Quick Start
+
+```python
+import polars as pl
+from imperfekt import Imperfekt, FeatureGenerator
+
+# Load your time-series data
+df = pl.read_parquet("your_data.parquet")
+
+# Run full imperfection analysis
+analyzer = Imperfekt(
+    df=df,
+    id_col="id",           # Unique identifier column
+    clock_col="clock",     # Timestamp column
+    cols=["var1", "var2"], # Variables to analyze
+    save_path="./results"
+)
+results = analyzer.run()
+
+# Or generate missingness-aware features for ML
+fg = FeatureGenerator(
+    df=df,
+    id_col="id",
+    clock_col="clock",
+    variable_cols=["var1", "var2"]
+)
+features_df = fg.add_binary_masks().add_temporal_features().df
+```
+
+## Library Structure
+
+```
+imperfekt/
+├── analysis/
+│   ├── preliminary/     # Basic data exploration
+│   ├── intravariable/      # Single variable analysis
+│   ├── intervariable/    # Multi-variable patterns
+│   └── utils/           # Shared utilities
+├── features/            # Feature engineering
+│   ├── core.py          # FeatureGenerator class
+│   ├── temporal.py      # Time-based features
+│   └── interaction.py   # Variable interactions
+└── config/              # Default settings
+```
+
+## Data Format
+
+The library expects time-series data with the following structure:
+
+| Column | Description |
+|--------|-------------|
+| `id` | Unique identifier for each time-series (e.g., patient, sensor) |
+| `clock` | Timestamp for each observation |
+| `var1`, `var2`, ... | Variables to analyze |
+
+## Key Dependencies
+
+- **polars**: High-performance data processing
+- **plotly**: Interactive visualizations
+- **scipy**: Statistical computations
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+

imperfekt-0.2.3/imperfekt/__init__.py

@@ -0,0 +1,22 @@
+"""
+imperfekt: A framework to analyze imperfections in time-series datasets.
+
+This library provides tools for:
+- Intravariable analysis of missing data patterns
+- Intervariable analysis of co-occurring missingness
+- Feature generation based on missingness patterns
+"""
+
+from imperfekt.analysis.imperfekt import Imperfekt
+from imperfekt.analysis.intervariable.intervariable import IntervariableImperfection
+from imperfekt.analysis.intravariable.intravariable import IntravariableImperfection
+from imperfekt.features.core import FeatureGenerator
+
+__version__ = "0.2.0"
+__all__ = [
+    "Imperfekt",
+    "IntravariableImperfection",
+    "IntervariableImperfection",
+    "FeatureGenerator",
+    "__version__",
+]

imperfekt-0.2.3/imperfekt/analysis/README.md

@@ -0,0 +1,59 @@
+# Analysis Module
+
+This module provides statistical analysis tools for characterizing imperfection (missingness and noise) in time-series data.
+
+## Structure
+
+| Submodule | Purpose |
+|-----------|---------|
+| `preliminary/` | Descriptive statistics, normality tests, correlation, and autocorrelation |
+| `intravariable/` | Within-column analysis: gap patterns, Markov chains, windowed significance |
+| `intervariable/` | Between-column analysis: MCAR tests, MAR/MNAR detection, symmetric/asymmetric correlation |
+| `utils/` | Shared utilities for statistics, visualization, and HTML reporting |
+
+## Usage
+
+```python
+from imperfekt.analysis import Imperfekt
+
+df = pl.DataFrame({
+    "patient": ["a", "a", "a", "a", "b", "b", "b"],
+    "time": [
+        "2023-01-01 08:00", "2023-01-01 08:05", "2023-01-01 08:10", "2023-01-01 08:15",
+        "2023-01-02 12:00", "2023-01-02 12:05", "2023-01-02 12:10"
+    ],
+    "heartrate": [60, None, 70, None, 80, 85, None],
+    "blood_pressure": [120, 125, None, None, 130, None, 140],
+    "resprate": [12, 14, None, 16, 18, None, 20],
+}).with_columns(
+    pl.col("time").str.strptime(pl.Datetime, format="%Y-%m-%d %H:%M")
+)
+
+analyzer = Imperfekt(
+    df=df,
+    id_col="patient",
+    clock_col="time",
+    clock_no_col="time_no",
+    save_path="/path",
+    plot_library="matplotlib",
+    renderer="notebook_connected",
+)
+# Run all analyses
+analyzer.run()
+
+# Run preliminary analysis only
+analyzer.preliminary.run()
+```
+
+## Detailed Documentation
+
+Each submodule contains its own README with further details:
+
+- [Preliminary Analysis](preliminary/README.md)
+- [Intravariable Analysis](intravariable/README.md)
+- [Intervariable Analysis](intervariable/README.md)
+
+### Overview Figure
+![Imperfekt Analysis Matrix](imperfekt.png "Imperfekt Analysis Matrix")
+
+