diffai-python 0.3.4__tar.gz → 0.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. diffai_python-0.5.0/PKG-INFO +37 -0
  2. diffai_python-0.5.0/README.md +24 -0
  3. diffai_python-0.5.0/diffai_python/__init__.py +28 -0
  4. diffai_python-0.5.0/diffai_python.egg-info/PKG-INFO +37 -0
  5. diffai_python-0.5.0/diffai_python.egg-info/SOURCES.txt +8 -0
  6. diffai_python-0.5.0/diffai_python.egg-info/dependency_links.txt +1 -0
  7. diffai_python-0.5.0/diffai_python.egg-info/requires.txt +1 -0
  8. diffai_python-0.5.0/diffai_python.egg-info/top_level.txt +1 -0
  9. diffai_python-0.5.0/pyproject.toml +22 -0
  10. diffai_python-0.5.0/setup.cfg +4 -0
  11. diffai_python-0.3.4/Cargo.toml +0 -63
  12. diffai_python-0.3.4/PKG-INFO +0 -385
  13. diffai_python-0.3.4/README.md +0 -347
  14. diffai_python-0.3.4/diffai-core/Cargo.toml +0 -50
  15. diffai_python-0.3.4/diffai-core/README.md +0 -355
  16. diffai_python-0.3.4/diffai-core/benches/diff_benchmark.rs +0 -59
  17. diffai_python-0.3.4/diffai-core/benches/ml_performance.rs +0 -255
  18. diffai_python-0.3.4/diffai-core/src/analysis_results_diff.rs +0 -303
  19. diffai_python-0.3.4/diffai-core/src/lib.rs +0 -4040
  20. diffai_python-0.3.4/diffai-python/.gitignore +0 -65
  21. diffai_python-0.3.4/diffai-python/Cargo.lock +0 -2436
  22. diffai_python-0.3.4/diffai-python/Cargo.toml +0 -31
  23. diffai_python-0.3.4/diffai-python/README.md +0 -347
  24. diffai_python-0.3.4/diffai-python/diffai +0 -0
  25. diffai_python-0.3.4/diffai-python/src/diffai/__init__.py +0 -303
  26. diffai_python-0.3.4/diffai-python/src/diffai/__main__.py +0 -8
  27. diffai_python-0.3.4/diffai-python/src/diffai/installer.py +0 -206
  28. diffai_python-0.3.4/diffai-python/src/main.rs +0 -22
  29. diffai_python-0.3.4/diffai-python/test_integration.py +0 -314
  30. diffai_python-0.3.4/pyproject.toml +0 -108
  31. diffai_python-0.3.4/src/diffai/__init__.py +0 -303
  32. diffai_python-0.3.4/src/diffai/__main__.py +0 -8
  33. diffai_python-0.3.4/src/diffai/installer.py +0 -206
@@ -0,0 +1,37 @@
1
+ Metadata-Version: 2.4
2
+ Name: diffai-python
3
+ Version: 0.5.0
4
+ Summary: DEPRECATED: Use 'diffai' instead. This package is a compatibility shim.
5
+ Author: kako-jun
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/kako-jun/diffai-python
8
+ Classifier: Development Status :: 7 - Inactive
9
+ Classifier: Programming Language :: Python :: 3
10
+ Requires-Python: >=3.8
11
+ Description-Content-Type: text/markdown
12
+ Requires-Dist: diffai>=0.5.0
13
+
14
+ # diffai-python (DEPRECATED)
15
+
16
+ **This package has been renamed to [`diffai`](https://pypi.org/project/diffai/).**
17
+
18
+ ## Migration
19
+
20
+ ```bash
21
+ pip uninstall diffai-python
22
+ pip install diffai
23
+ ```
24
+
25
+ Update your imports:
26
+
27
+ ```python
28
+ # Old (deprecated)
29
+ import diffai_python
30
+
31
+ # New
32
+ import diffai
33
+ ```
34
+
35
+ ## Why the change?
36
+
37
+ The package has been renamed to provide a cleaner import experience that matches the package name.
@@ -0,0 +1,24 @@
1
+ # diffai-python (DEPRECATED)
2
+
3
+ **This package has been renamed to [`diffai`](https://pypi.org/project/diffai/).**
4
+
5
+ ## Migration
6
+
7
+ ```bash
8
+ pip uninstall diffai-python
9
+ pip install diffai
10
+ ```
11
+
12
+ Update your imports:
13
+
14
+ ```python
15
+ # Old (deprecated)
16
+ import diffai_python
17
+
18
+ # New
19
+ import diffai
20
+ ```
21
+
22
+ ## Why the change?
23
+
24
+ The package has been renamed to provide a cleaner import experience that matches the package name.
@@ -0,0 +1,28 @@
1
+ """
2
+ DEPRECATED: This package has been renamed to 'diffai'.
3
+
4
+ Please update your dependencies:
5
+ pip install diffai
6
+
7
+ And update your imports:
8
+ # Old (deprecated)
9
+ import diffai_python
10
+
11
+ # New
12
+ import diffai
13
+ """
14
+
15
+ import warnings
16
+
17
+ warnings.warn(
18
+ "The 'diffai-python' package is deprecated and will be removed in a future release. "
19
+ "Please use 'diffai' instead:\n"
20
+ " pip install diffai\n"
21
+ " import diffai",
22
+ DeprecationWarning,
23
+ stacklevel=2
24
+ )
25
+
26
+ # Re-export everything from the new package
27
+ from diffai import *
28
+ from diffai import __version__
@@ -0,0 +1,37 @@
1
+ Metadata-Version: 2.4
2
+ Name: diffai-python
3
+ Version: 0.5.0
4
+ Summary: DEPRECATED: Use 'diffai' instead. This package is a compatibility shim.
5
+ Author: kako-jun
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/kako-jun/diffai-python
8
+ Classifier: Development Status :: 7 - Inactive
9
+ Classifier: Programming Language :: Python :: 3
10
+ Requires-Python: >=3.8
11
+ Description-Content-Type: text/markdown
12
+ Requires-Dist: diffai>=0.5.0
13
+
14
+ # diffai-python (DEPRECATED)
15
+
16
+ **This package has been renamed to [`diffai`](https://pypi.org/project/diffai/).**
17
+
18
+ ## Migration
19
+
20
+ ```bash
21
+ pip uninstall diffai-python
22
+ pip install diffai
23
+ ```
24
+
25
+ Update your imports:
26
+
27
+ ```python
28
+ # Old (deprecated)
29
+ import diffai_python
30
+
31
+ # New
32
+ import diffai
33
+ ```
34
+
35
+ ## Why the change?
36
+
37
+ The package has been renamed to provide a cleaner import experience that matches the package name.
@@ -0,0 +1,8 @@
1
+ README.md
2
+ pyproject.toml
3
+ diffai_python/__init__.py
4
+ diffai_python.egg-info/PKG-INFO
5
+ diffai_python.egg-info/SOURCES.txt
6
+ diffai_python.egg-info/dependency_links.txt
7
+ diffai_python.egg-info/requires.txt
8
+ diffai_python.egg-info/top_level.txt
@@ -0,0 +1 @@
1
+ diffai>=0.5.0
@@ -0,0 +1 @@
1
+ diffai_python
@@ -0,0 +1,22 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "diffai-python"
7
+ version = "0.5.0"
8
+ description = "DEPRECATED: Use 'diffai' instead. This package is a compatibility shim."
9
+ readme = "README.md"
10
+ license = "MIT"
11
+ authors = [
12
+ { name = "kako-jun" }
13
+ ]
14
+ classifiers = [
15
+ "Development Status :: 7 - Inactive",
16
+ "Programming Language :: Python :: 3",
17
+ ]
18
+ requires-python = ">=3.8"
19
+ dependencies = ["diffai>=0.5.0"]
20
+
21
+ [project.urls]
22
+ Homepage = "https://github.com/kako-jun/diffai-python"
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -1,63 +0,0 @@
1
- [workspace]
2
- resolver = "2"
3
- members = [
4
- "diffai-core",
5
- "diffai-cli"
6
- ]
7
-
8
- [workspace.package]
9
- version = "0.3.4"
10
- edition = "2021"
11
- authors = ["kako-jun"]
12
- license = "MIT"
13
- description = "🤖 AI/ML specialized diff tool for deep learning models - PyTorch, Safetensors, tensor analysis, model comparison, experiment tracking"
14
- homepage = "https://github.com/kako-jun/diffai"
15
- repository = "https://github.com/kako-jun/diffai"
16
- documentation = "https://github.com/kako-jun/diffai/tree/main/docs"
17
- readme = "README.md"
18
- keywords = ["diff", "ai", "ml", "pytorch", "safetensors"]
19
- categories = ["command-line-utilities", "development-tools", "algorithms", "science"]
20
- exclude = [
21
- ".github/",
22
- "target/"
23
- ]
24
- rust-version = "1.75"
25
-
26
- [workspace.dependencies]
27
- anyhow = "1.0"
28
- clap = { version = "4.0", features = ["derive", "cargo"] }
29
- colored = "3.0"
30
- csv = "1.3"
31
- configparser = "3.0"
32
- quick-xml = { version = "0.31", features = ["serialize"] }
33
- serde = { version = "1.0", features = ["derive"] }
34
- serde_json = "1.0"
35
- serde_yml = "0.0.12"
36
- toml = "0.8"
37
- walkdir = "2.5"
38
- dirs = "5.0"
39
- regex = "1.0"
40
- similar = "2.7"
41
- tempfile = "3.0"
42
- assert_cmd = "2.0"
43
- predicates = "3.1"
44
- criterion = { version = "0.5", features = ["html_reports"] }
45
- # AI/ML dependencies
46
- candle-core = "0.8"
47
- candle-nn = "0.8"
48
- safetensors = "0.4"
49
- bytemuck = "1.14"
50
- # Scientific data dependencies
51
- ndarray = "0.16"
52
- # numpy = "0.21" # Removed: only needed for Python bindings, causes PyO3 issues
53
- # hdf5 = "0.8" # Requires system HDF5 libraries
54
- zip = "2.2"
55
- matfile = "0.1" # MATLAB .mat file support
56
- # Python bindings (removed - now using binary distribution)
57
- # Cross-project integration
58
- diffx-core = "0.1"
59
-
60
- [profile.release]
61
- lto = true
62
- codegen-units = 1
63
- panic = "abort"
@@ -1,385 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: diffai-python
3
- Version: 0.3.4
4
- Classifier: Development Status :: 4 - Beta
5
- Classifier: Environment :: Console
6
- Classifier: Intended Audience :: Developers
7
- Classifier: Intended Audience :: Science/Research
8
- Classifier: License :: OSI Approved :: MIT License
9
- Classifier: Operating System :: OS Independent
10
- Classifier: Programming Language :: Python :: 3
11
- Classifier: Programming Language :: Python :: 3.8
12
- Classifier: Programming Language :: Python :: 3.9
13
- Classifier: Programming Language :: Python :: 3.10
14
- Classifier: Programming Language :: Python :: 3.11
15
- Classifier: Programming Language :: Python :: 3.12
16
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
17
- Classifier: Topic :: Software Development :: Libraries :: Python Modules
18
- Classifier: Topic :: Text Processing :: General
19
- Classifier: Topic :: Utilities
20
- Requires-Dist: pytest>=6.0 ; extra == 'dev'
21
- Requires-Dist: pytest-cov ; extra == 'dev'
22
- Requires-Dist: black ; extra == 'dev'
23
- Requires-Dist: isort ; extra == 'dev'
24
- Requires-Dist: mypy ; extra == 'dev'
25
- Requires-Dist: ruff ; extra == 'dev'
26
- Provides-Extra: dev
27
- Summary: AI/ML specialized diff tool for deep tensor comparison and analysis
28
- Keywords: ai,ml,machine-learning,diff,tensor,pytorch,safetensors,numpy,matlab,model-comparison,diffai,artificial-intelligence
29
- Author: kako-jun
30
- License: MIT
31
- Requires-Python: >=3.8
32
- Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
33
- Project-URL: Homepage, https://github.com/kako-jun/diffai
34
- Project-URL: Documentation, https://github.com/kako-jun/diffai/blob/main/docs/
35
- Project-URL: Repository, https://github.com/kako-jun/diffai.git
36
- Project-URL: Issues, https://github.com/kako-jun/diffai/issues
37
- Project-URL: Changelog, https://github.com/kako-jun/diffai/blob/main/CHANGELOG.md
38
-
39
- # diffai-python
40
-
41
- AI/ML specialized diff tool for deep tensor comparison and analysis - Python Package
42
-
43
- [![PyPI version](https://badge.fury.io/py/diffai-python.svg)](https://badge.fury.io/py/diffai-python)
44
- [![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
45
- [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
46
-
47
- ## Overview
48
-
49
- **diffai-python** provides Python bindings for [diffai](https://github.com/kako-jun/diffai), an AI/ML specialized diff tool. This package bundles the high-performance Rust binary and provides a clean Python API for integration into ML workflows, notebooks, and automation scripts.
50
-
51
- Following the same distribution pattern as [ruff](https://github.com/astral-sh/ruff), this package distributes a pre-compiled binary for maximum performance while providing a convenient Python interface.
52
-
53
- ## Features
54
-
55
- - **High Performance**: Uses the native diffai Rust binary for maximum speed
56
- - **Zero Dependencies**: Self-contained package with bundled binary
57
- - **ML-Focused**: Specialized analysis for PyTorch, Safetensors, NumPy, and MATLAB files
58
- - **Scientific Computing**: Full support for NumPy arrays and MATLAB .mat files
59
- - **Multiple Output Formats**: CLI, JSON, and YAML outputs for different use cases
60
- - **Python Integration**: Clean API for programmatic use in ML pipelines
61
-
62
- ## Installation
63
-
64
- ```bash
65
- pip install diffai-python
66
- ```
67
-
68
- ## Quick Start
69
-
70
- ### Command Line Usage
71
-
72
- After installation, the `diffai` command is available:
73
-
74
- ```bash
75
- # Compare ML models
76
- diffai model_v1.safetensors model_v2.safetensors --stats
77
-
78
- # Compare NumPy arrays
79
- diffai data_v1.npy data_v2.npy --stats
80
-
81
- # JSON output for automation
82
- diffai model_v1.pt model_v2.pt --output json
83
- ```
84
-
85
- ### Python API Usage
86
-
87
- ```python
88
- import diffai
89
-
90
- # Basic comparison
91
- result = diffai.diff("model_v1.safetensors", "model_v2.safetensors")
92
- print(result.raw_output)
93
-
94
- # With options
95
- options = diffai.DiffOptions(
96
- stats=True,
97
- architecture_comparison=True,
98
- output_format=diffai.OutputFormat.JSON
99
- )
100
- result = diffai.diff("model_v1.pt", "model_v2.pt", options)
101
-
102
- # Access structured data
103
- if result.is_json:
104
- data = result.data
105
- print(f"Found {len(data)} differences")
106
- ```
107
-
108
- ### Advanced ML Analysis
109
-
110
- ```python
111
- # Comprehensive ML model analysis
112
- result = diffai.diff(
113
- "baseline.safetensors",
114
- "improved.safetensors",
115
- stats=True,
116
- architecture_comparison=True,
117
- memory_analysis=True,
118
- anomaly_detection=True,
119
- convergence_analysis=True
120
- )
121
-
122
- print(result.raw_output)
123
- ```
124
-
125
- ## Supported Formats
126
-
127
- ### Input Formats
128
- - **ML Models**: `.safetensors`, `.pt`, `.pth`, `.bin` (PyTorch)
129
- - **Scientific Data**: `.npy`, `.npz` (NumPy), `.mat` (MATLAB)
130
- - **Structured Data**: `.json`, `.yaml`, `.toml`, `.xml`, `.ini`, `.csv`
131
-
132
- ### Output Formats
133
- - **CLI**: Colored terminal output (default)
134
- - **JSON**: Machine-readable format for automation
135
- - **YAML**: Human-readable structured format
136
-
137
- ## ML Analysis Features
138
-
139
- The package provides 11 specialized ML analysis features:
140
-
141
- - `--stats`: Detailed tensor statistics
142
- - `--architecture-comparison`: Model structure comparison
143
- - `--memory-analysis`: Memory usage analysis
144
- - `--anomaly-detection`: Numerical anomaly detection
145
- - `--convergence-analysis`: Training convergence analysis
146
- - `--gradient-analysis`: Gradient information analysis
147
- - `--similarity-matrix`: Layer similarity comparison
148
- - `--change-summary`: Detailed change summary
149
- - `--quantization-analysis`: Quantization impact analysis
150
- - `--sort-by-change-magnitude`: Sort by change magnitude
151
- - `--show-layer-impact`: Layer-specific impact analysis
152
-
153
- ## API Reference
154
-
155
- ### Main Functions
156
-
157
- ```python
158
- # Compare two files
159
- def diff(input1: str, input2: str, options: Optional[DiffOptions] = None, **kwargs) -> DiffResult
160
-
161
- # Main CLI entry point
162
- def main() -> None
163
- ```
164
-
165
- ### Configuration
166
-
167
- ```python
168
- @dataclass
169
- class DiffOptions:
170
- # Basic options
171
- input_format: Optional[str] = None
172
- output_format: Optional[OutputFormat] = None
173
- recursive: bool = False
174
- verbose: bool = False
175
-
176
- # ML analysis options
177
- stats: bool = False
178
- architecture_comparison: bool = False
179
- memory_analysis: bool = False
180
- anomaly_detection: bool = False
181
- # ... and more
182
- ```
183
-
184
- ### Results
185
-
186
- ```python
187
- class DiffResult:
188
- raw_output: str # Raw output from diffai
189
- format_type: str # Output format used
190
- return_code: int # Process return code
191
-
192
- @property
193
- def data(self) -> Any # Parsed data (JSON when applicable)
194
-
195
- @property
196
- def is_json(self) -> bool # True if JSON format
197
- ```
198
-
199
- ## Use Cases
200
-
201
- ### Research & Development
202
- ```python
203
- # Compare fine-tuning results
204
- before = "model_baseline.safetensors"
205
- after = "model_finetuned.safetensors"
206
-
207
- result = diffai.diff(before, after,
208
- stats=True,
209
- convergence_analysis=True)
210
- ```
211
-
212
- ### MLOps Integration
213
- ```python
214
- # Automated model validation in CI/CD
215
- def validate_model_changes(old_model, new_model):
216
- result = diffai.diff(old_model, new_model,
217
- output_format=diffai.OutputFormat.JSON,
218
- anomaly_detection=True,
219
- memory_analysis=True)
220
-
221
- if result.is_json:
222
- # Check for critical issues
223
- for item in result.data:
224
- if 'AnomalyDetection' in item and 'critical' in str(item):
225
- raise ValueError("Critical model anomaly detected")
226
-
227
- return result
228
-
229
- ### MLflow Integration
230
- ```python
231
- import mlflow
232
- import diffai
233
-
234
- def log_model_comparison(run_id1, run_id2):
235
- """Compare models between MLflow runs"""
236
-
237
- # Download models from MLflow
238
- model1_path = mlflow.artifacts.download_artifacts(
239
- run_id=run_id1, artifact_path="model/model.pt"
240
- )
241
- model2_path = mlflow.artifacts.download_artifacts(
242
- run_id=run_id2, artifact_path="model/model.pt"
243
- )
244
-
245
- # Compare with diffai
246
- result = diffai.diff(model1_path, model2_path,
247
- output_format=diffai.OutputFormat.JSON,
248
- stats=True,
249
- architecture_comparison=True)
250
-
251
- # Log results to MLflow
252
- with mlflow.start_run():
253
- mlflow.log_dict(result.data, "model_comparison.json")
254
- if result.is_json:
255
- # Extract metrics for logging
256
- for item in result.data:
257
- if 'TensorStatsChanged' in item:
258
- mlflow.log_metric("tensor_changes", len(result.data))
259
- break
260
-
261
- return result
262
-
263
- ### Weights & Biases Integration
264
- ```python
265
- import wandb
266
- import diffai
267
-
268
- def log_model_comparison_wandb(model1_path, model2_path):
269
- """Log model comparison to Weights & Biases"""
270
-
271
- result = diffai.diff(model1_path, model2_path,
272
- output_format=diffai.OutputFormat.JSON,
273
- stats=True,
274
- memory_analysis=True,
275
- convergence_analysis=True)
276
-
277
- # Log to wandb
278
- wandb.log({"model_comparison": result.data})
279
-
280
- if result.is_json:
281
- # Log specific metrics
282
- memory_changes = [item for item in result.data if 'MemoryAnalysis' in item]
283
- if memory_changes:
284
- wandb.log({"memory_impact_detected": len(memory_changes)})
285
-
286
- return result
287
- ```
288
-
289
- ### Jupyter Notebooks
290
- ```python
291
- # Interactive analysis in notebooks
292
- result = diffai.diff("checkpoint_100.pt", "checkpoint_200.pt",
293
- stats=True, memory_analysis=True)
294
-
295
- # Display results
296
- if result.is_json:
297
- from IPython.display import display, JSON
298
- display(JSON(result.data))
299
- else:
300
- print(result.raw_output)
301
- ```
302
-
303
- ## Binary Distribution
304
-
305
- This package follows the same pattern as [ruff](https://github.com/astral-sh/ruff):
306
-
307
- - Pre-compiled `diffai` binary is bundled with the Python package
308
- - No external dependencies or system requirements
309
- - Cross-platform compatibility (Windows, macOS, Linux)
310
- - Maximum performance through native Rust implementation
311
-
312
- ## Testing
313
-
314
- Run the integration tests:
315
-
316
- ```bash
317
- cd diffai-python
318
- python test_integration.py
319
- ```
320
-
321
- The test suite includes:
322
- - Binary availability verification
323
- - Basic diff functionality
324
- - JSON output parsing
325
- - ML analysis options
326
- - Error handling
327
-
328
- ## Contributing
329
-
330
- This package is part of the [diffai](https://github.com/kako-jun/diffai) project. Please see the main repository for contribution guidelines.
331
-
332
- ## License
333
-
334
- MIT License - see [LICENSE](../LICENSE) for details.
335
-
336
- ## Related Projects
337
-
338
- - **[diffai](https://github.com/kako-jun/diffai)**: Main Rust CLI tool
339
- - **[diffx](https://github.com/kako-jun/diffx)**: Generic structured data diff tool
340
- - **[ruff](https://github.com/astral-sh/ruff)**: Inspiration for Python packaging approach
341
-
342
- ## Error Handling
343
-
344
- The diffai-python package provides comprehensive error handling for various failure scenarios:
345
-
346
- ### DiffaiError
347
- The base exception class for all diffai-related errors:
348
-
349
- ```python
350
- import diffai
351
-
352
- try:
353
- result = diffai.diff("model1.pt", "model2.pt")
354
- except diffai.DiffaiError as e:
355
- print(f"Diffai error: {e}")
356
- ```
357
-
358
- ### BinaryNotFoundError
359
- Raised when the diffai binary cannot be found:
360
-
361
- ```python
362
- import diffai
363
-
364
- try:
365
- result = diffai.diff("model1.pt", "model2.pt")
366
- except diffai.DiffaiError as e:
367
- if "binary not found" in str(e):
368
- print("Please install diffai binary or ensure it's in PATH")
369
- # Fallback or installation logic here
370
- ```
371
-
372
- ### Binary Installation
373
- If the binary is not found, you can install it manually:
374
-
375
- ```bash
376
- # Install via pip (includes binary)
377
- pip install diffai-python
378
-
379
- # Or install Rust version globally
380
- cargo install diffai-cli
381
- ```
382
-
383
- ## License
384
-
385
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.