datapruning 2.1.2__tar.gz → 2.1.3__tar.gz

datapruning-2.1.3/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: datapruning
+Version: 2.1.3
+Summary: Intelligent dataset optimization for cleaner, smaller training data
+License: Proprietary
+Project-URL: Homepage, https://www.datapruning.com
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: scikit-learn>=1.3.0
+Provides-Extra: torch
+Requires-Dist: torch>=2.0.0; extra == "torch"
+Provides-Extra: clip
+Requires-Dist: torch>=2.0.0; extra == "clip"
+Requires-Dist: transformers>=4.30.0; extra == "clip"
+Requires-Dist: pillow>=9.0.0; extra == "clip"
+Dynamic: license-file
+
+# DataPruning
+
+Intelligent dataset optimization — reduces dataset size by selecting the most informative rows while preserving model quality.
+
+## Installation
+
+```bash
+pip install datapruning
+```
+
+For image/text multimodal support (CLIP encoding):
+
+```bash
+pip install datapruning[clip]
+```
+
+## Quick Start
+
+### CSV / Tabular Data
+
+```python
+import pandas as pd
+from datapruning.sdk import DatasetEngine
+
+df = pd.read_csv("dataset.csv")
+engine = DatasetEngine(df, target_col="label")
+
+# Analyze the dataset
+analysis = engine.analyze()
+
+# Get a recommendation
+rec = engine.recommend()
+print(f"Best method: {rec.display_name} ({rec.confidence}% confidence)")
+
+# Optimize — keep 50% of most informative rows
+result = engine.optimize(keep_ratio=0.5)
+print(f"Kept {result['runtime_seconds']:.1f}s — {result['explainability']['rows_removed']} rows removed")
+
+# Download the optimized dataset
+result["optimized_df"].to_csv("optimized.csv", index=False)
+```
+
+### Image Datasets
+
+```python
+from datapruning.sdk import DatasetEngine
+
+# Create engine from a folder of images
+engine = DatasetEngine.from_images(
+    folder_path="./photos",
+    labels=[0, 1, 0, 1, ...],  # one label per image
+    keep_ratio=0.5,
+)
+
+# Analyze and optimize
+analysis = engine.analyze()
+result = engine.optimize(keep_ratio=0.5)
+result["optimized_df"].to_csv("optimized_images.csv", index=False)
+```
+
+### Text Datasets
+
+```python
+engine = DatasetEngine.from_texts(
+    texts=["label A sample", "label B sample", ...],
+    labels=[0, 1, 0, 1, ...],
+)
+result = engine.optimize(keep_ratio=0.5)
+```
+
+## Algorithms
+
+DataPruning selects the best algorithm automatically based on dataset characteristics:
+
+| Method | What it does | Best for |
+|--------|-------------|----------|
+| **Margin Optimization** | Analyzes classification confidence, keeps boundary samples | General tabular data |
+| **Importance Scoring** | Measures sample difficulty via feature distances | Quick baseline, any dataset |
+| **Diversity Selection** | Picks the most representative subset (K-Center) | High-dimensional data |
+| **Noise Filtering** | Drops redundant and potentially mislabeled samples | Noisy/messy data |
+| **Full Pipeline** | Combines all signals for maximum quality | Complex datasets |
+
+## Limits
+
+- Minimum: 1,000 rows (tabular) or 10 images
+- Maximum: 300,000 rows (tabular) or 1,000 images
+- Supported image formats: JPG, PNG, BMP, GIF, TIFF, WebP
+
+## Requirements
+
+- Python >= 3.10
+- Pandas 2.0+
+- NumPy 1.24+
+- scikit-learn 1.3+
+
+Optional (for image/text support):
+- PyTorch 2.0+
+- transformers 4.30+
+- Pillow 9.0+
+
+## Features
+
+- Runs locally — no data uploaded anywhere
+- Auto-detects task type (classification/regression) and target column
+- Auto-encodes categorical/string columns
+- 5 optimization algorithms with automatic selection
+- Preserves minority class balance
+- CSV and image dataset support
+- Pure Python — works on any platform
+
+## Links
+
+- Website: [datapruning.com](https://www.datapruning.com)
+- PyPI: [pypi.org/project/datapruning](https://pypi.org/project/datapruning/)
+
+## License
+
+Proprietary. See LICENSE file.

datapruning-2.1.3/README.md

@@ -0,0 +1,118 @@
+# DataPruning
+
+Intelligent dataset optimization — reduces dataset size by selecting the most informative rows while preserving model quality.
+
+## Installation
+
+```bash
+pip install datapruning
+```
+
+For image/text multimodal support (CLIP encoding):
+
+```bash
+pip install datapruning[clip]
+```
+
+## Quick Start
+
+### CSV / Tabular Data
+
+```python
+import pandas as pd
+from datapruning.sdk import DatasetEngine
+
+df = pd.read_csv("dataset.csv")
+engine = DatasetEngine(df, target_col="label")
+
+# Analyze the dataset
+analysis = engine.analyze()
+
+# Get a recommendation
+rec = engine.recommend()
+print(f"Best method: {rec.display_name} ({rec.confidence}% confidence)")
+
+# Optimize — keep 50% of most informative rows
+result = engine.optimize(keep_ratio=0.5)
+print(f"Kept {result['runtime_seconds']:.1f}s — {result['explainability']['rows_removed']} rows removed")
+
+# Download the optimized dataset
+result["optimized_df"].to_csv("optimized.csv", index=False)
+```
+
+### Image Datasets
+
+```python
+from datapruning.sdk import DatasetEngine
+
+# Create engine from a folder of images
+engine = DatasetEngine.from_images(
+    folder_path="./photos",
+    labels=[0, 1, 0, 1, ...],  # one label per image
+    keep_ratio=0.5,
+)
+
+# Analyze and optimize
+analysis = engine.analyze()
+result = engine.optimize(keep_ratio=0.5)
+result["optimized_df"].to_csv("optimized_images.csv", index=False)
+```
+
+### Text Datasets
+
+```python
+engine = DatasetEngine.from_texts(
+    texts=["label A sample", "label B sample", ...],
+    labels=[0, 1, 0, 1, ...],
+)
+result = engine.optimize(keep_ratio=0.5)
+```
+
+## Algorithms
+
+DataPruning selects the best algorithm automatically based on dataset characteristics:
+
+| Method | What it does | Best for |
+|--------|-------------|----------|
+| **Margin Optimization** | Analyzes classification confidence, keeps boundary samples | General tabular data |
+| **Importance Scoring** | Measures sample difficulty via feature distances | Quick baseline, any dataset |
+| **Diversity Selection** | Picks the most representative subset (K-Center) | High-dimensional data |
+| **Noise Filtering** | Drops redundant and potentially mislabeled samples | Noisy/messy data |
+| **Full Pipeline** | Combines all signals for maximum quality | Complex datasets |
+
+## Limits
+
+- Minimum: 1,000 rows (tabular) or 10 images
+- Maximum: 300,000 rows (tabular) or 1,000 images
+- Supported image formats: JPG, PNG, BMP, GIF, TIFF, WebP
+
+## Requirements
+
+- Python >= 3.10
+- Pandas 2.0+
+- NumPy 1.24+
+- scikit-learn 1.3+
+
+Optional (for image/text support):
+- PyTorch 2.0+
+- transformers 4.30+
+- Pillow 9.0+
+
+## Features
+
+- Runs locally — no data uploaded anywhere
+- Auto-detects task type (classification/regression) and target column
+- Auto-encodes categorical/string columns
+- 5 optimization algorithms with automatic selection
+- Preserves minority class balance
+- CSV and image dataset support
+- Pure Python — works on any platform
+
+## Links
+
+- Website: [datapruning.com](https://www.datapruning.com)
+- PyPI: [pypi.org/project/datapruning](https://pypi.org/project/datapruning/)
+
+## License
+
+Proprietary. See LICENSE file.

{datapruning-2.1.2 → datapruning-2.1.3}/datapruning/__init__.py

@@ -5,6 +5,8 @@ import pandas as pd
 from . import pipeline
 from .sdk import DatasetEngine
 
+__all__ = ["DatasetEngine", "Pruner", "prune_dataframe"]
+
 logger = logging.getLogger(__name__)
 
 MIN_ROWS = 1000

{datapruning-2.1.2 → datapruning-2.1.3}/datapruning/algorithms/el2n_scorer.py

@@ -16,31 +16,33 @@ class EL2NScorerOptimizer(BaseOptimizer):
     def fit(self, X: pd.DataFrame, y: pd.Series) -> "EL2NScorerOptimizer":
         X_arr = X.values.astype(np.float64)
         y_arr = y.values
-        feat_norms = np.linalg.norm(X_arr, axis=1)
-        feat_norms_normed = (feat_norms - feat_norms.min()) / max(feat_norms.max() - feat_norms.min(), 1e-8)
         classes = np.unique(y_arr)
-        centroid_scores = np.zeros(len(y_arr), dtype=np.float64)
+        scores = np.zeros(len(y_arr), dtype=np.float64)
         for cls in classes:
             cls_mask = y_arr == cls
             centroid = X_arr[cls_mask].mean(axis=0)
             dists = np.linalg.norm(X_arr - centroid, axis=1)
-            dists_normed = (dists - dists.min()) / max(dists.max() - dists.min(), 1e-8)
-            centroid_scores = np.maximum(centroid_scores, dists_normed)
-        combined = 0.5 * feat_norms_normed + 0.5 * centroid_scores
-        self._scores = pd.Series(combined, index=y.index)
+            scores = np.maximum(scores, dists)
+        self._scores = pd.Series(scores, index=y.index)
         return self
 
     def _select_rows(self, X: pd.DataFrame, y: pd.Series, keep_ratio: float) -> pd.Index:
         if self._scores is None:
             self.fit(X, y)
+        classes = y.unique()
+        if len(classes) < 2:
+            n_target = max(1, int(len(y) * keep_ratio))
+            return y.sample(n=n_target, random_state=42).index
         minority_class = y.value_counts().idxmin()
         minority_idx = y[y == minority_class].index
         majority_idx = y[y != minority_class].index
-        n_total_target = int(len(y) * keep_ratio)
-        n_majority_target = max(n_total_target - len(minority_idx), 0)
+        n_total_target = max(1, int(len(y) * keep_ratio))
+        n_minority = min(len(minority_idx), int(n_total_target * 0.8))
+        n_majority_target = max(n_total_target - n_minority, 0)
         majority_scores = self._scores.loc[majority_idx].sort_values(ascending=False)
         kept_majority_idx = majority_scores.index[:n_majority_target]
-        return minority_idx.append(kept_majority_idx)
+        kept_minority_idx = minority_idx[:n_minority]
+        return kept_minority_idx.append(kept_majority_idx)
 
     def why_selected(self) -> str:
         return ("Measures how difficult each sample is for the model to classify by analyzing "
@@ -62,5 +64,5 @@ class EL2NScorerOptimizer(BaseOptimizer):
         if task_type == "regression":
             return 0.0
         if n_rows < 200:
-            return 0.1
-        return 0.5
+            return 0.2
+        return 0.4

{datapruning-2.1.2 → datapruning-2.1.3}/datapruning/algorithms/kcenter_select.py

@@ -48,9 +48,11 @@ class KCenterSelectOptimizer(BaseOptimizer):
         n_cols = scan.get("columns", 0)
 
         if task_type == "regression":
-            return 0.2
+            return 0.3
+        if n_rows < 500:
+            return 0.6
         if n_cols > n_rows:
-            return 0.8
+            return 0.9
         if n_cols > 50:
-            return 0.65
-        return 0.4
+            return 0.85
+        return 0.8

{datapruning-2.1.2 → datapruning-2.1.3}/datapruning/algorithms/margin_scorer.py

@@ -37,16 +37,18 @@ class MarginScorerOptimizer(BaseOptimizer):
             self.fit(X, y)
         classes = y.unique()
         if len(classes) < 2:
-            n_target = int(len(y) * keep_ratio)
-            return y.nlargest(n_target).index
+            n_target = max(1, int(len(y) * keep_ratio))
+            return y.sample(n=n_target, random_state=42).index
         minority_class = y.value_counts().idxmin()
         minority_idx = y[y == minority_class].index
         majority_idx = y[y != minority_class].index
-        n_total_target = int(len(y) * keep_ratio)
-        n_majority_target = max(n_total_target - len(minority_idx), 0)
-        majority_scores = self._scores.loc[majority_idx].sort_values(ascending=False)
+        n_total_target = max(1, int(len(y) * keep_ratio))
+        n_minority = min(len(minority_idx), int(n_total_target * 0.8))
+        n_majority_target = max(n_total_target - n_minority, 0)
+        majority_scores = self._scores.loc[majority_idx].sort_values(ascending=True)
         kept_majority_idx = majority_scores.index[:n_majority_target]
-        return minority_idx.append(kept_majority_idx)
+        kept_minority_idx = minority_idx[:n_minority]
+        return kept_minority_idx.append(kept_majority_idx)
 
     def why_selected(self) -> str:
         return ("Analyzes how confidently each data point is classified, keeping the most "
@@ -64,14 +66,9 @@ class MarginScorerOptimizer(BaseOptimizer):
         scan = analysis.get("scan", analysis)
         task_type = scan.get("task_type", "unknown")
         n_rows = scan.get("rows", 0)
-        n_cols = scan.get("columns", 0)
 
         if task_type == "regression":
             return 0.0
         if n_rows < 500:
-            return 0.1
-        if n_rows > 200_000:
-            return 0.6
-        if n_cols > 100:
-            return 0.7
-        return 0.85
+            return 0.2
+        return 0.3

{datapruning-2.1.2 → datapruning-2.1.3}/datapruning/algorithms/sliding_window.py

@@ -40,8 +40,8 @@ class SlidingWindowOptimizer(BaseOptimizer):
         low_pct = max(0, (1 - keep_ratio) * 100 / 2)
         high_pct = min(100, 100 - low_pct)
         if len(classes) < 2:
-            n_target = int(len(y) * keep_ratio)
-            return y.nlargest(n_target).index
+            n_target = max(1, int(len(y) * keep_ratio))
+            return y.sample(n=n_target, random_state=42).index
         selected = sliding_window_select(
             self._scores,
             low_percentile=low_pct,
@@ -53,10 +53,10 @@ class SlidingWindowOptimizer(BaseOptimizer):
         minority_idx = y[y == minority_class].index
         selected_idx = X.index[selected]
         all_idx = selected_idx.union(minority_idx).unique()
-        n_target = int(len(y) * keep_ratio)
+        n_target = max(1, int(len(y) * keep_ratio))
         if len(all_idx) > n_target:
             scores_at_idx = pd.Series(self._scores, index=X.index).loc[all_idx]
-            all_idx = scores_at_idx.sort_values(ascending=False).index[:n_target]
+            all_idx = scores_at_idx.sort_values(ascending=True).index[:n_target]
         return all_idx
 
     def why_selected(self) -> str:

datapruning-2.1.3/datapruning.egg-info/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: datapruning
+Version: 2.1.3
+Summary: Intelligent dataset optimization for cleaner, smaller training data
+License: Proprietary
+Project-URL: Homepage, https://www.datapruning.com
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: scikit-learn>=1.3.0
+Provides-Extra: torch
+Requires-Dist: torch>=2.0.0; extra == "torch"
+Provides-Extra: clip
+Requires-Dist: torch>=2.0.0; extra == "clip"
+Requires-Dist: transformers>=4.30.0; extra == "clip"
+Requires-Dist: pillow>=9.0.0; extra == "clip"
+Dynamic: license-file
+
+# DataPruning
+
+Intelligent dataset optimization — reduces dataset size by selecting the most informative rows while preserving model quality.
+
+## Installation
+
+```bash
+pip install datapruning
+```
+
+For image/text multimodal support (CLIP encoding):
+
+```bash
+pip install datapruning[clip]
+```
+
+## Quick Start
+
+### CSV / Tabular Data
+
+```python
+import pandas as pd
+from datapruning.sdk import DatasetEngine
+
+df = pd.read_csv("dataset.csv")
+engine = DatasetEngine(df, target_col="label")
+
+# Analyze the dataset
+analysis = engine.analyze()
+
+# Get a recommendation
+rec = engine.recommend()
+print(f"Best method: {rec.display_name} ({rec.confidence}% confidence)")
+
+# Optimize — keep 50% of most informative rows
+result = engine.optimize(keep_ratio=0.5)
+print(f"Kept {result['runtime_seconds']:.1f}s — {result['explainability']['rows_removed']} rows removed")
+
+# Download the optimized dataset
+result["optimized_df"].to_csv("optimized.csv", index=False)
+```
+
+### Image Datasets
+
+```python
+from datapruning.sdk import DatasetEngine
+
+# Create engine from a folder of images
+engine = DatasetEngine.from_images(
+    folder_path="./photos",
+    labels=[0, 1, 0, 1, ...],  # one label per image
+    keep_ratio=0.5,
+)
+
+# Analyze and optimize
+analysis = engine.analyze()
+result = engine.optimize(keep_ratio=0.5)
+result["optimized_df"].to_csv("optimized_images.csv", index=False)
+```
+
+### Text Datasets
+
+```python
+engine = DatasetEngine.from_texts(
+    texts=["label A sample", "label B sample", ...],
+    labels=[0, 1, 0, 1, ...],
+)
+result = engine.optimize(keep_ratio=0.5)
+```
+
+## Algorithms
+
+DataPruning selects the best algorithm automatically based on dataset characteristics:
+
+| Method | What it does | Best for |
+|--------|-------------|----------|
+| **Margin Optimization** | Analyzes classification confidence, keeps boundary samples | General tabular data |
+| **Importance Scoring** | Measures sample difficulty via feature distances | Quick baseline, any dataset |
+| **Diversity Selection** | Picks the most representative subset (K-Center) | High-dimensional data |
+| **Noise Filtering** | Drops redundant and potentially mislabeled samples | Noisy/messy data |
+| **Full Pipeline** | Combines all signals for maximum quality | Complex datasets |
+
+## Limits
+
+- Minimum: 1,000 rows (tabular) or 10 images
+- Maximum: 300,000 rows (tabular) or 1,000 images
+- Supported image formats: JPG, PNG, BMP, GIF, TIFF, WebP
+
+## Requirements
+
+- Python >= 3.10
+- Pandas 2.0+
+- NumPy 1.24+
+- scikit-learn 1.3+
+
+Optional (for image/text support):
+- PyTorch 2.0+
+- transformers 4.30+
+- Pillow 9.0+
+
+## Features
+
+- Runs locally — no data uploaded anywhere
+- Auto-detects task type (classification/regression) and target column
+- Auto-encodes categorical/string columns
+- 5 optimization algorithms with automatic selection
+- Preserves minority class balance
+- CSV and image dataset support
+- Pure Python — works on any platform
+
+## Links
+
+- Website: [datapruning.com](https://www.datapruning.com)
+- PyPI: [pypi.org/project/datapruning](https://pypi.org/project/datapruning/)
+
+## License
+
+Proprietary. See LICENSE file.

{datapruning-2.1.2 → datapruning-2.1.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "datapruning"
-version = "2.1.2"
+version = "2.1.3"
 description = "Intelligent dataset optimization for cleaner, smaller training data"
 readme = "README.md"
 requires-python = ">=3.10"

datapruning-2.1.2/PKG-INFO

@@ -1,81 +0,0 @@
-Metadata-Version: 2.4
-Name: datapruning
-Version: 2.1.2
-Summary: Intelligent dataset optimization for cleaner, smaller training data
-License: Proprietary
-Project-URL: Homepage, https://www.datapruning.com
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Programming Language :: Python :: 3.14
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy>=1.24.0
-Requires-Dist: pandas>=2.0.0
-Requires-Dist: scikit-learn>=1.3.0
-Provides-Extra: torch
-Requires-Dist: torch>=2.0.0; extra == "torch"
-Provides-Extra: clip
-Requires-Dist: torch>=2.0.0; extra == "clip"
-Requires-Dist: transformers>=4.30.0; extra == "clip"
-Requires-Dist: pillow>=9.0.0; extra == "clip"
-Dynamic: license-file
-
-# DataPruning
-
-Intelligent dataset pruning for ML — reduces dataset size by selecting the most informative rows.
-
-## Installation
-
-```bash
-pip install datapruning
-```
-
-**Note:** PyTorch (~2 GB) is a required dependency.
-
-## Requirements
-
-- Python >= 3.10
-- PyTorch 2.0+
-- Pandas 2.0+
-- NumPy 1.24+
-
-## Limits
-
-- Minimum: 1,000 rows per dataset
-- Maximum: 300,000 rows per dataset
-
-## Usage
-
-```python
-import pandas as pd
-from datapruning import Pruner
-
-df = pd.read_csv("dataset.csv")
-p = Pruner(df)
-filtered = p.prune(target_col="label", keep_ratio=0.5)
-
-print(f"Kept {len(filtered)} / {len(df)} rows")
-```
-
-## Features
-
-- Runs locally, no data upload
-- Pre-training dataset optimization
-- Pandas DataFrame input / output
-- Compiled processing module
-
-## Links
-
-- Website: [datapruning.com](https://www.datapruning.com)
-
-## License
-
-Proprietary. See LICENSE file.

datapruning-2.1.2/README.md

@@ -1,51 +0,0 @@
-# DataPruning
-
-Intelligent dataset pruning for ML — reduces dataset size by selecting the most informative rows.
-
-## Installation
-
-```bash
-pip install datapruning
-```
-
-**Note:** PyTorch (~2 GB) is a required dependency.
-
-## Requirements
-
-- Python >= 3.10
-- PyTorch 2.0+
-- Pandas 2.0+
-- NumPy 1.24+
-
-## Limits
-
-- Minimum: 1,000 rows per dataset
-- Maximum: 300,000 rows per dataset
-
-## Usage
-
-```python
-import pandas as pd
-from datapruning import Pruner
-
-df = pd.read_csv("dataset.csv")
-p = Pruner(df)
-filtered = p.prune(target_col="label", keep_ratio=0.5)
-
-print(f"Kept {len(filtered)} / {len(df)} rows")
-```
-
-## Features
-
-- Runs locally, no data upload
-- Pre-training dataset optimization
-- Pandas DataFrame input / output
-- Compiled processing module
-
-## Links
-
-- Website: [datapruning.com](https://www.datapruning.com)
-
-## License
-
-Proprietary. See LICENSE file.

datapruning-2.1.2/datapruning.egg-info/PKG-INFO

@@ -1,81 +0,0 @@
-Metadata-Version: 2.4
-Name: datapruning
-Version: 2.1.2
-Summary: Intelligent dataset optimization for cleaner, smaller training data
-License: Proprietary
-Project-URL: Homepage, https://www.datapruning.com
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Programming Language :: Python :: 3.14
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy>=1.24.0
-Requires-Dist: pandas>=2.0.0
-Requires-Dist: scikit-learn>=1.3.0
-Provides-Extra: torch
-Requires-Dist: torch>=2.0.0; extra == "torch"
-Provides-Extra: clip
-Requires-Dist: torch>=2.0.0; extra == "clip"
-Requires-Dist: transformers>=4.30.0; extra == "clip"
-Requires-Dist: pillow>=9.0.0; extra == "clip"
-Dynamic: license-file
-
-# DataPruning
-
-Intelligent dataset pruning for ML — reduces dataset size by selecting the most informative rows.
-
-## Installation
-
-```bash
-pip install datapruning
-```
-
-**Note:** PyTorch (~2 GB) is a required dependency.
-
-## Requirements
-
-- Python >= 3.10
-- PyTorch 2.0+
-- Pandas 2.0+
-- NumPy 1.24+
-
-## Limits
-
-- Minimum: 1,000 rows per dataset
-- Maximum: 300,000 rows per dataset
-
-## Usage
-
-```python
-import pandas as pd
-from datapruning import Pruner
-
-df = pd.read_csv("dataset.csv")
-p = Pruner(df)
-filtered = p.prune(target_col="label", keep_ratio=0.5)
-
-print(f"Kept {len(filtered)} / {len(df)} rows")
-```
-
-## Features
-
-- Runs locally, no data upload
-- Pre-training dataset optimization
-- Pandas DataFrame input / output
-- Compiled processing module
-
-## Links
-
-- Website: [datapruning.com](https://www.datapruning.com)
-
-## License
-
-Proprietary. See LICENSE file.