diffai-python 0.3.5__tar.gz → 0.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. diffai_python-0.5.0/PKG-INFO +37 -0
  2. diffai_python-0.5.0/README.md +24 -0
  3. diffai_python-0.5.0/diffai_python/__init__.py +28 -0
  4. diffai_python-0.5.0/diffai_python.egg-info/PKG-INFO +37 -0
  5. diffai_python-0.5.0/diffai_python.egg-info/SOURCES.txt +8 -0
  6. diffai_python-0.5.0/diffai_python.egg-info/dependency_links.txt +1 -0
  7. diffai_python-0.5.0/diffai_python.egg-info/requires.txt +1 -0
  8. diffai_python-0.5.0/diffai_python.egg-info/top_level.txt +1 -0
  9. diffai_python-0.5.0/pyproject.toml +22 -0
  10. diffai_python-0.5.0/setup.cfg +4 -0
  11. diffai_python-0.3.5/Cargo.toml +0 -63
  12. diffai_python-0.3.5/PKG-INFO +0 -374
  13. diffai_python-0.3.5/README.md +0 -336
  14. diffai_python-0.3.5/diffai-core/Cargo.toml +0 -50
  15. diffai_python-0.3.5/diffai-core/README.md +0 -380
  16. diffai_python-0.3.5/diffai-core/benches/diff_benchmark.rs +0 -59
  17. diffai_python-0.3.5/diffai-core/benches/ml_performance.rs +0 -255
  18. diffai_python-0.3.5/diffai-core/src/analysis_results_diff.rs +0 -303
  19. diffai_python-0.3.5/diffai-core/src/lib.rs +0 -4040
  20. diffai_python-0.3.5/diffai-python/.gitignore +0 -65
  21. diffai_python-0.3.5/diffai-python/Cargo.lock +0 -2436
  22. diffai_python-0.3.5/diffai-python/Cargo.toml +0 -31
  23. diffai_python-0.3.5/diffai-python/README.md +0 -336
  24. diffai_python-0.3.5/diffai-python/diffai +0 -0
  25. diffai_python-0.3.5/diffai-python/src/diffai/__init__.py +0 -303
  26. diffai_python-0.3.5/diffai-python/src/diffai/__main__.py +0 -8
  27. diffai_python-0.3.5/diffai-python/src/diffai/installer.py +0 -206
  28. diffai_python-0.3.5/diffai-python/src/main.rs +0 -22
  29. diffai_python-0.3.5/diffai-python/test_integration.py +0 -314
  30. diffai_python-0.3.5/pyproject.toml +0 -108
  31. diffai_python-0.3.5/src/diffai/__init__.py +0 -303
  32. diffai_python-0.3.5/src/diffai/__main__.py +0 -8
  33. diffai_python-0.3.5/src/diffai/installer.py +0 -206
@@ -0,0 +1,37 @@
1
+ Metadata-Version: 2.4
2
+ Name: diffai-python
3
+ Version: 0.5.0
4
+ Summary: DEPRECATED: Use 'diffai' instead. This package is a compatibility shim.
5
+ Author: kako-jun
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/kako-jun/diffai-python
8
+ Classifier: Development Status :: 7 - Inactive
9
+ Classifier: Programming Language :: Python :: 3
10
+ Requires-Python: >=3.8
11
+ Description-Content-Type: text/markdown
12
+ Requires-Dist: diffai>=0.5.0
13
+
14
+ # diffai-python (DEPRECATED)
15
+
16
+ **This package has been renamed to [`diffai`](https://pypi.org/project/diffai/).**
17
+
18
+ ## Migration
19
+
20
+ ```bash
21
+ pip uninstall diffai-python
22
+ pip install diffai
23
+ ```
24
+
25
+ Update your imports:
26
+
27
+ ```python
28
+ # Old (deprecated)
29
+ import diffai_python
30
+
31
+ # New
32
+ import diffai
33
+ ```
34
+
35
+ ## Why the change?
36
+
37
+ The package has been renamed to provide a cleaner import experience that matches the package name.
@@ -0,0 +1,24 @@
1
+ # diffai-python (DEPRECATED)
2
+
3
+ **This package has been renamed to [`diffai`](https://pypi.org/project/diffai/).**
4
+
5
+ ## Migration
6
+
7
+ ```bash
8
+ pip uninstall diffai-python
9
+ pip install diffai
10
+ ```
11
+
12
+ Update your imports:
13
+
14
+ ```python
15
+ # Old (deprecated)
16
+ import diffai_python
17
+
18
+ # New
19
+ import diffai
20
+ ```
21
+
22
+ ## Why the change?
23
+
24
+ The package has been renamed to provide a cleaner import experience that matches the package name.
@@ -0,0 +1,28 @@
1
+ """
2
+ DEPRECATED: This package has been renamed to 'diffai'.
3
+
4
+ Please update your dependencies:
5
+ pip install diffai
6
+
7
+ And update your imports:
8
+ # Old (deprecated)
9
+ import diffai_python
10
+
11
+ # New
12
+ import diffai
13
+ """
14
+
15
+ import warnings
16
+
17
+ warnings.warn(
18
+ "The 'diffai-python' package is deprecated and will be removed in a future release. "
19
+ "Please use 'diffai' instead:\n"
20
+ " pip install diffai\n"
21
+ " import diffai",
22
+ DeprecationWarning,
23
+ stacklevel=2
24
+ )
25
+
26
+ # Re-export everything from the new package
27
+ from diffai import *
28
+ from diffai import __version__
@@ -0,0 +1,37 @@
1
+ Metadata-Version: 2.4
2
+ Name: diffai-python
3
+ Version: 0.5.0
4
+ Summary: DEPRECATED: Use 'diffai' instead. This package is a compatibility shim.
5
+ Author: kako-jun
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/kako-jun/diffai-python
8
+ Classifier: Development Status :: 7 - Inactive
9
+ Classifier: Programming Language :: Python :: 3
10
+ Requires-Python: >=3.8
11
+ Description-Content-Type: text/markdown
12
+ Requires-Dist: diffai>=0.5.0
13
+
14
+ # diffai-python (DEPRECATED)
15
+
16
+ **This package has been renamed to [`diffai`](https://pypi.org/project/diffai/).**
17
+
18
+ ## Migration
19
+
20
+ ```bash
21
+ pip uninstall diffai-python
22
+ pip install diffai
23
+ ```
24
+
25
+ Update your imports:
26
+
27
+ ```python
28
+ # Old (deprecated)
29
+ import diffai_python
30
+
31
+ # New
32
+ import diffai
33
+ ```
34
+
35
+ ## Why the change?
36
+
37
+ The package has been renamed to provide a cleaner import experience that matches the package name.
@@ -0,0 +1,8 @@
1
+ README.md
2
+ pyproject.toml
3
+ diffai_python/__init__.py
4
+ diffai_python.egg-info/PKG-INFO
5
+ diffai_python.egg-info/SOURCES.txt
6
+ diffai_python.egg-info/dependency_links.txt
7
+ diffai_python.egg-info/requires.txt
8
+ diffai_python.egg-info/top_level.txt
@@ -0,0 +1 @@
1
+ diffai>=0.5.0
@@ -0,0 +1 @@
1
+ diffai_python
@@ -0,0 +1,22 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "diffai-python"
7
+ version = "0.5.0"
8
+ description = "DEPRECATED: Use 'diffai' instead. This package is a compatibility shim."
9
+ readme = "README.md"
10
+ license = "MIT"
11
+ authors = [
12
+ { name = "kako-jun" }
13
+ ]
14
+ classifiers = [
15
+ "Development Status :: 7 - Inactive",
16
+ "Programming Language :: Python :: 3",
17
+ ]
18
+ requires-python = ">=3.8"
19
+ dependencies = ["diffai>=0.5.0"]
20
+
21
+ [project.urls]
22
+ Homepage = "https://github.com/kako-jun/diffai-python"
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -1,63 +0,0 @@
1
- [workspace]
2
- resolver = "2"
3
- members = [
4
- "diffai-core",
5
- "diffai-cli"
6
- ]
7
-
8
- [workspace.package]
9
- version = "0.3.5"
10
- edition = "2021"
11
- authors = ["kako-jun"]
12
- license = "MIT"
13
- description = "🤖 AI/ML specialized diff tool for deep learning models - PyTorch, Safetensors, tensor analysis, model comparison, experiment tracking"
14
- homepage = "https://github.com/kako-jun/diffai"
15
- repository = "https://github.com/kako-jun/diffai"
16
- documentation = "https://github.com/kako-jun/diffai/tree/main/docs"
17
- readme = "README.md"
18
- keywords = ["diff", "ai", "ml", "pytorch", "safetensors"]
19
- categories = ["command-line-utilities", "development-tools", "algorithms", "science"]
20
- exclude = [
21
- ".github/",
22
- "target/"
23
- ]
24
- rust-version = "1.75"
25
-
26
- [workspace.dependencies]
27
- anyhow = "1.0"
28
- clap = { version = "4.0", features = ["derive", "cargo"] }
29
- colored = "3.0"
30
- csv = "1.3"
31
- configparser = "3.0"
32
- quick-xml = { version = "0.31", features = ["serialize"] }
33
- serde = { version = "1.0", features = ["derive"] }
34
- serde_json = "1.0"
35
- serde_yml = "0.0.12"
36
- toml = "0.8"
37
- walkdir = "2.5"
38
- dirs = "5.0"
39
- regex = "1.0"
40
- similar = "2.7"
41
- tempfile = "3.0"
42
- assert_cmd = "2.0"
43
- predicates = "3.1"
44
- criterion = { version = "0.5", features = ["html_reports"] }
45
- # AI/ML dependencies
46
- candle-core = "0.8"
47
- candle-nn = "0.8"
48
- safetensors = "0.4"
49
- bytemuck = "1.14"
50
- # Scientific data dependencies
51
- ndarray = "0.16"
52
- # numpy = "0.21" # Removed: only needed for Python bindings, causes PyO3 issues
53
- # hdf5 = "0.8" # Requires system HDF5 libraries
54
- zip = "2.2"
55
- matfile = "0.1" # MATLAB .mat file support
56
- # Python bindings (removed - now using binary distribution)
57
- # Cross-project integration
58
- diffx-core = "0.1"
59
-
60
- [profile.release]
61
- lto = true
62
- codegen-units = 1
63
- panic = "abort"
@@ -1,374 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: diffai-python
3
- Version: 0.3.5
4
- Classifier: Development Status :: 4 - Beta
5
- Classifier: Environment :: Console
6
- Classifier: Intended Audience :: Developers
7
- Classifier: Intended Audience :: Science/Research
8
- Classifier: License :: OSI Approved :: MIT License
9
- Classifier: Operating System :: OS Independent
10
- Classifier: Programming Language :: Python :: 3
11
- Classifier: Programming Language :: Python :: 3.8
12
- Classifier: Programming Language :: Python :: 3.9
13
- Classifier: Programming Language :: Python :: 3.10
14
- Classifier: Programming Language :: Python :: 3.11
15
- Classifier: Programming Language :: Python :: 3.12
16
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
17
- Classifier: Topic :: Software Development :: Libraries :: Python Modules
18
- Classifier: Topic :: Text Processing :: General
19
- Classifier: Topic :: Utilities
20
- Requires-Dist: pytest>=6.0 ; extra == 'dev'
21
- Requires-Dist: pytest-cov ; extra == 'dev'
22
- Requires-Dist: black ; extra == 'dev'
23
- Requires-Dist: isort ; extra == 'dev'
24
- Requires-Dist: mypy ; extra == 'dev'
25
- Requires-Dist: ruff ; extra == 'dev'
26
- Provides-Extra: dev
27
- Summary: AI/ML specialized diff tool for deep tensor comparison and analysis
28
- Keywords: ai,ml,machine-learning,diff,tensor,pytorch,safetensors,numpy,matlab,model-comparison,diffai,artificial-intelligence
29
- Author: kako-jun
30
- License: MIT
31
- Requires-Python: >=3.8
32
- Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
33
- Project-URL: Homepage, https://github.com/kako-jun/diffai
34
- Project-URL: Documentation, https://github.com/kako-jun/diffai/blob/main/docs/
35
- Project-URL: Repository, https://github.com/kako-jun/diffai.git
36
- Project-URL: Issues, https://github.com/kako-jun/diffai/issues
37
- Project-URL: Changelog, https://github.com/kako-jun/diffai/blob/main/CHANGELOG.md
38
-
39
- # diffai-python
40
-
41
- AI/ML specialized diff tool for deep tensor comparison and analysis - Python Package
42
-
43
- [![PyPI version](https://badge.fury.io/py/diffai-python.svg)](https://badge.fury.io/py/diffai-python)
44
- [![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
45
- [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
46
-
47
- ## Overview
48
-
49
- **diffai-python** provides Python bindings for [diffai](https://github.com/kako-jun/diffai), an AI/ML specialized diff tool. This package bundles the high-performance Rust binary and provides a clean Python API for integration into ML workflows, notebooks, and automation scripts.
50
-
51
- Following the same distribution pattern as [ruff](https://github.com/astral-sh/ruff), this package distributes a pre-compiled binary for maximum performance while providing a convenient Python interface.
52
-
53
- ## Features
54
-
55
- - **High Performance**: Uses the native diffai Rust binary for maximum speed
56
- - **Zero Dependencies**: Self-contained package with bundled binary
57
- - **ML-Focused**: Specialized analysis for PyTorch, Safetensors, NumPy, and MATLAB files
58
- - **Scientific Computing**: Full support for NumPy arrays and MATLAB .mat files
59
- - **Multiple Output Formats**: CLI, JSON, and YAML outputs for different use cases
60
- - **Python Integration**: Clean API for programmatic use in ML pipelines
61
-
62
- ## Installation
63
-
64
- ```bash
65
- pip install diffai-python
66
- ```
67
-
68
- ## Quick Start
69
-
70
- ### Command Line Usage
71
-
72
- After installation, the `diffai` command is available:
73
-
74
- ```bash
75
- # Compare ML models (30+ analysis features automatic)
76
- diffai model_v1.safetensors model_v2.safetensors
77
-
78
- # Compare NumPy arrays
79
- diffai data_v1.npy data_v2.npy
80
-
81
- # JSON output for automation (all ML features included)
82
- diffai model_v1.pt model_v2.pt --output json
83
- ```
84
-
85
- ### Python API Usage
86
-
87
- ```python
88
- import diffai
89
-
90
- # Basic comparison
91
- result = diffai.diff("model_v1.safetensors", "model_v2.safetensors")
92
- print(result.raw_output)
93
-
94
- # With options
95
- options = diffai.DiffOptions(
96
- output_format=diffai.OutputFormat.JSON
97
- )
98
- result = diffai.diff("model_v1.pt", "model_v2.pt", options)
99
-
100
- # Access structured data
101
- if result.is_json:
102
- data = result.data
103
- print(f"Found {len(data)} differences")
104
- ```
105
-
106
- ### Advanced ML Analysis
107
-
108
- ```python
109
- # Comprehensive ML model analysis (automatic for ML models)
110
- result = diffai.diff(
111
- "baseline.safetensors",
112
- "improved.safetensors",
113
- stats=True # Enable statistical analysis
114
- )
115
-
116
- print(result.raw_output)
117
-
118
- # ML-specific analysis features (automatic for ML models)
119
- # - architecture_comparison: Model architecture and structural changes
120
- # - memory_analysis: Memory usage and optimization opportunities
121
- # - anomaly_detection: Numerical anomalies and training issues
122
- # - convergence_analysis: Training convergence patterns
123
- # - gradient_analysis: Gradient flow health assessment
124
- # - quantization_analysis: Quantization effect analysis
125
- ```
126
-
127
- ## Supported Formats
128
-
129
- ### Input Formats
130
- - **ML Models**: `.safetensors`, `.pt`, `.pth`, `.bin` (PyTorch)
131
- - **Scientific Data**: `.npy`, `.npz` (NumPy), `.mat` (MATLAB)
132
- - **Structured Data**: `.json`, `.yaml`, `.toml`, `.xml`, `.ini`, `.csv`
133
-
134
- ### Output Formats
135
- - **CLI**: Colored terminal output (default)
136
- - **JSON**: Machine-readable format for automation
137
- - **YAML**: Human-readable structured format
138
-
139
- ## ML Analysis Features (Automatic)
140
-
141
- The package provides 30+ specialized ML analysis features that run automatically for PyTorch and Safetensors files:
142
-
143
- - **Detailed tensor statistics**: Mean, std, min, max, shape, dtype
144
- - **Model structure comparison**: Architecture and structural changes
145
- - **Memory usage analysis**: Memory optimization opportunities
146
- - **Numerical anomaly detection**: Training issues and anomalies
147
- - **Training convergence analysis**: Convergence patterns
148
- - **Gradient information analysis**: Gradient flow health
149
- - **Layer similarity comparison**: Inter-layer analysis
150
- - **Detailed change summary**: Comprehensive change patterns
151
- - **Quantization impact analysis**: Quantization effects
152
- - **Change magnitude sorting**: Priority-sorted differences
153
- - **Plus 20+ additional specialized features**
154
-
155
- ## API Reference
156
-
157
- ### Main Functions
158
-
159
- ```python
160
- # Compare two files
161
- def diff(input1: str, input2: str, options: Optional[DiffOptions] = None, **kwargs) -> DiffResult
162
-
163
- # Main CLI entry point
164
- def main() -> None
165
- ```
166
-
167
- ### Configuration
168
-
169
- ```python
170
- @dataclass
171
- class DiffOptions:
172
- # Basic options
173
- input_format: Optional[str] = None
174
- output_format: Optional[OutputFormat] = None
175
- recursive: bool = False
176
- verbose: bool = False
177
-
178
- # For scientific data (NumPy/MATLAB)
179
- stats: bool = False # Only used for NumPy/MATLAB files
180
- # Note: ML analysis runs automatically for PyTorch/Safetensors
181
- ```
182
-
183
- ### Results
184
-
185
- ```python
186
- class DiffResult:
187
- raw_output: str # Raw output from diffai
188
- format_type: str # Output format used
189
- return_code: int # Process return code
190
-
191
- @property
192
- def data(self) -> Any # Parsed data (JSON when applicable)
193
-
194
- @property
195
- def is_json(self) -> bool # True if JSON format
196
- ```
197
-
198
- ## Use Cases
199
-
200
- ### Research & Development
201
- ```python
202
- # Compare fine-tuning results
203
- before = "model_baseline.safetensors"
204
- after = "model_finetuned.safetensors"
205
-
206
- result = diffai.diff(before, after)
207
- ```
208
-
209
- ### MLOps Integration
210
- ```python
211
- # Automated model validation in CI/CD
212
- def validate_model_changes(old_model, new_model):
213
- result = diffai.diff(old_model, new_model,
214
- output_format=diffai.OutputFormat.JSON)
215
-
216
- if result.is_json:
217
- # Check for critical issues
218
- for item in result.data:
219
- if 'AnomalyDetection' in item and 'critical' in str(item):
220
- raise ValueError("Critical model anomaly detected")
221
-
222
- return result
223
-
224
- ### MLflow Integration
225
- ```python
226
- import mlflow
227
- import diffai
228
-
229
- def log_model_comparison(run_id1, run_id2):
230
- """Compare models between MLflow runs"""
231
-
232
- # Download models from MLflow
233
- model1_path = mlflow.artifacts.download_artifacts(
234
- run_id=run_id1, artifact_path="model/model.pt"
235
- )
236
- model2_path = mlflow.artifacts.download_artifacts(
237
- run_id=run_id2, artifact_path="model/model.pt"
238
- )
239
-
240
- # Compare with diffai
241
- result = diffai.diff(model1_path, model2_path,
242
- output_format=diffai.OutputFormat.JSON)
243
-
244
- # Log results to MLflow
245
- with mlflow.start_run():
246
- mlflow.log_dict(result.data, "model_comparison.json")
247
- if result.is_json:
248
- # Extract metrics for logging
249
- for item in result.data:
250
- if 'TensorStatsChanged' in item:
251
- mlflow.log_metric("tensor_changes", len(result.data))
252
- break
253
-
254
- return result
255
-
256
- ### Weights & Biases Integration
257
- ```python
258
- import wandb
259
- import diffai
260
-
261
- def log_model_comparison_wandb(model1_path, model2_path):
262
- """Log model comparison to Weights & Biases"""
263
-
264
- result = diffai.diff(model1_path, model2_path,
265
- output_format=diffai.OutputFormat.JSON)
266
-
267
- # Log to wandb
268
- wandb.log({"model_comparison": result.data})
269
-
270
- if result.is_json:
271
- # Log specific metrics
272
- memory_changes = [item for item in result.data if 'MemoryAnalysis' in item]
273
- if memory_changes:
274
- wandb.log({"memory_impact_detected": len(memory_changes)})
275
-
276
- return result
277
- ```
278
-
279
- ### Jupyter Notebooks
280
- ```python
281
- # Interactive analysis in notebooks
282
- result = diffai.diff("checkpoint_100.pt", "checkpoint_200.pt")
283
-
284
- # Display results
285
- if result.is_json:
286
- from IPython.display import display, JSON
287
- display(JSON(result.data))
288
- else:
289
- print(result.raw_output)
290
- ```
291
-
292
- ## Binary Distribution
293
-
294
- This package follows the same pattern as [ruff](https://github.com/astral-sh/ruff):
295
-
296
- - Pre-compiled `diffai` binary is bundled with the Python package
297
- - No external dependencies or system requirements
298
- - Cross-platform compatibility (Windows, macOS, Linux)
299
- - Maximum performance through native Rust implementation
300
-
301
- ## Testing
302
-
303
- Run the integration tests:
304
-
305
- ```bash
306
- cd diffai-python
307
- python test_integration.py
308
- ```
309
-
310
- The test suite includes:
311
- - Binary availability verification
312
- - Basic diff functionality
313
- - JSON output parsing
314
- - ML analysis options
315
- - Error handling
316
-
317
- ## Contributing
318
-
319
- This package is part of the [diffai](https://github.com/kako-jun/diffai) project. Please see the main repository for contribution guidelines.
320
-
321
- ## License
322
-
323
- MIT License - see [LICENSE](../LICENSE) for details.
324
-
325
- ## Related Projects
326
-
327
- - **[diffai](https://github.com/kako-jun/diffai)**: Main Rust CLI tool
328
- - **[diffx](https://github.com/kako-jun/diffx)**: Generic structured data diff tool
329
- - **[ruff](https://github.com/astral-sh/ruff)**: Inspiration for Python packaging approach
330
-
331
- ## Error Handling
332
-
333
- The diffai-python package provides comprehensive error handling for various failure scenarios:
334
-
335
- ### DiffaiError
336
- The base exception class for all diffai-related errors:
337
-
338
- ```python
339
- import diffai
340
-
341
- try:
342
- result = diffai.diff("model1.pt", "model2.pt")
343
- except diffai.DiffaiError as e:
344
- print(f"Diffai error: {e}")
345
- ```
346
-
347
- ### BinaryNotFoundError
348
- Raised when the diffai binary cannot be found:
349
-
350
- ```python
351
- import diffai
352
-
353
- try:
354
- result = diffai.diff("model1.pt", "model2.pt")
355
- except diffai.DiffaiError as e:
356
- if "binary not found" in str(e):
357
- print("Please install diffai binary or ensure it's in PATH")
358
- # Fallback or installation logic here
359
- ```
360
-
361
- ### Binary Installation
362
- If the binary is not found, you can install it manually:
363
-
364
- ```bash
365
- # Install via pip (includes binary)
366
- pip install diffai-python
367
-
368
- # Or install Rust version globally
369
- cargo install diffai-cli
370
- ```
371
-
372
- ## License
373
-
374
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.