datapruning 1.0.2__tar.gz → 1.0.5__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,2 @@
1
+ include build_src/_core.pyx
2
+ include build_src/_core.c
@@ -1,79 +1,73 @@
1
- Metadata-Version: 2.4
2
- Name: datapruning
3
- Version: 1.0.2
4
- Summary: Intelligent data pruning for ML datasets
5
- License: Proprietary
6
- Project-URL: Homepage, https://www.datapruning.com
7
- Classifier: Development Status :: 4 - Beta
8
- Classifier: Intended Audience :: Developers
9
- Classifier: Intended Audience :: Science/Research
10
- Classifier: Programming Language :: Python :: 3
11
- Classifier: Programming Language :: Python :: 3.10
12
- Classifier: Programming Language :: Python :: 3.11
13
- Classifier: Programming Language :: Python :: 3.12
14
- Classifier: Programming Language :: Python :: 3.13
15
- Classifier: Programming Language :: Python :: 3.14
16
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
17
- Requires-Python: >=3.10
18
- Description-Content-Type: text/markdown
19
- License-File: LICENSE
20
- Requires-Dist: numpy>=1.24.0
21
- Requires-Dist: pandas>=2.0.0
22
- Requires-Dist: torch>=2.0.0
23
- Dynamic: license-file
24
-
25
- # DataPruning Lite
26
-
27
- Local-first dataset pruning. Free for datasets up to 100K rows.
28
-
29
- ## Overview
30
-
31
- DataPruning reduces dataset size before training by selecting a smaller,
32
- more informative subset of data. Runs entirely on your machine through a
33
- compiled SDK no data upload, no external processing.
34
-
35
- ## Installation
36
-
37
- ```bash
38
- pip install datapruning
39
- ```
40
-
41
- **Note:** PyTorch (~2 GB) is a required dependency and will be installed
42
- automatically. See [pytorch.org](https://pytorch.org) for GPU/CPU options.
43
-
44
- ## Requirements
45
-
46
- - Python 3.9–3.13
47
- - PyTorch 2.0+
48
- - Pandas 2.0+
49
- - NumPy 1.24+
50
-
51
- ## Usage
52
-
53
- ```python
54
- import pandas as pd
55
- from datapruning import Pruner
56
-
57
- df = pd.read_csv("dataset.csv")
58
- p = Pruner(df)
59
- filtered = p.prune(target_col="label", keep_ratio=0.5)
60
-
61
- print(f"Kept {len(filtered)} / {len(df)} rows")
62
- ```
63
-
64
- ## Features
65
-
66
- - Local execution, no data upload
67
- - Pre-training dataset optimization
68
- - Pandas DataFrame input
69
- - Compiled processing module
70
- - Free tier: up to 100K rows per dataset
71
-
72
- ## Limits
73
-
74
- Datasets exceeding 100,000 rows require an enterprise license.
75
- Visit https://www.datapruning.com for details.
76
-
77
- ## License
78
-
79
- Proprietary software. See LICENSE file.
1
+ Metadata-Version: 2.4
2
+ Name: datapruning
3
+ Version: 1.0.5
4
+ Summary: Intelligent data pruning for ML datasets
5
+ License: Proprietary
6
+ Project-URL: Homepage, https://www.datapruning.com
7
+ Classifier: Development Status :: 4 - Beta
8
+ Classifier: Intended Audience :: Developers
9
+ Classifier: Intended Audience :: Science/Research
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Programming Language :: Python :: 3.10
12
+ Classifier: Programming Language :: Python :: 3.11
13
+ Classifier: Programming Language :: Python :: 3.12
14
+ Classifier: Programming Language :: Python :: 3.13
15
+ Classifier: Programming Language :: Python :: 3.14
16
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
17
+ Requires-Python: >=3.10
18
+ Description-Content-Type: text/markdown
19
+ Requires-Dist: numpy>=1.24.0
20
+ Requires-Dist: pandas>=2.0.0
21
+ Requires-Dist: torch>=2.0.0
22
+
23
+ # DataPruning
24
+
25
+ Intelligent dataset pruning for ML — reduces dataset size by selecting the most informative rows.
26
+
27
+ ## Installation
28
+
29
+ ```bash
30
+ pip install datapruning
31
+ ```
32
+
33
+ **Note:** PyTorch (~2 GB) is a required dependency.
34
+
35
+ ## Requirements
36
+
37
+ - Python >= 3.10
38
+ - PyTorch 2.0+
39
+ - Pandas 2.0+
40
+ - NumPy 1.24+
41
+
42
+ ## Limits
43
+
44
+ - Minimum: 1,000 rows per dataset
45
+ - Maximum: 300,000 rows per dataset
46
+
47
+ ## Usage
48
+
49
+ ```python
50
+ import pandas as pd
51
+ from datapruning import Pruner
52
+
53
+ df = pd.read_csv("dataset.csv")
54
+ p = Pruner(df)
55
+ filtered = p.prune(target_col="label", keep_ratio=0.5)
56
+
57
+ print(f"Kept {len(filtered)} / {len(df)} rows")
58
+ ```
59
+
60
+ ## Features
61
+
62
+ - Runs locally, no data upload
63
+ - Pre-training dataset optimization
64
+ - Pandas DataFrame input / output
65
+ - Compiled processing module
66
+
67
+ ## Links
68
+
69
+ - Website: [datapruning.com](https://www.datapruning.com)
70
+
71
+ ## License
72
+
73
+ Proprietary. See LICENSE file.
@@ -0,0 +1,51 @@
1
+ # DataPruning
2
+
3
+ Intelligent dataset pruning for ML — reduces dataset size by selecting the most informative rows.
4
+
5
+ ## Installation
6
+
7
+ ```bash
8
+ pip install datapruning
9
+ ```
10
+
11
+ **Note:** PyTorch (~2 GB) is a required dependency.
12
+
13
+ ## Requirements
14
+
15
+ - Python >= 3.10
16
+ - PyTorch 2.0+
17
+ - Pandas 2.0+
18
+ - NumPy 1.24+
19
+
20
+ ## Limits
21
+
22
+ - Minimum: 1,000 rows per dataset
23
+ - Maximum: 300,000 rows per dataset
24
+
25
+ ## Usage
26
+
27
+ ```python
28
+ import pandas as pd
29
+ from datapruning import Pruner
30
+
31
+ df = pd.read_csv("dataset.csv")
32
+ p = Pruner(df)
33
+ filtered = p.prune(target_col="label", keep_ratio=0.5)
34
+
35
+ print(f"Kept {len(filtered)} / {len(df)} rows")
36
+ ```
37
+
38
+ ## Features
39
+
40
+ - Runs locally, no data upload
41
+ - Pre-training dataset optimization
42
+ - Pandas DataFrame input / output
43
+ - Compiled processing module
44
+
45
+ ## Links
46
+
47
+ - Website: [datapruning.com](https://www.datapruning.com)
48
+
49
+ ## License
50
+
51
+ Proprietary. See LICENSE file.