pyTorchAutoForge 0.2.2.dev0__tar.gz → 0.3.1.dev0__tar.gz

{pytorchautoforge-0.2.2.dev0 → pytorchautoforge-0.3.1.dev0}/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2025 PeterC
+Copyright (c) 2025 Pietro Califano
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

{pytorchautoforge-0.2.2.dev0 → pytorchautoforge-0.3.1.dev0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: pyTorchAutoForge
-Version: 0.2.2.dev0
-Summary: PyTorchAutoForge library is based on raw PyTorch and designed to automate DNN development, model tracking and deployment, tightly integrated with MLflow and Optuna. It supports Spiking networks libraries (WIP). Deployment can be performed using ONNx, pyTorch facilities or TensorRT (WIP). The library is designed to be compatible with Jetson Orin Nano Jetpack rev6.1, with bash script to automatically configure virtualenv.
+Version: 0.3.1.dev0
+Summary: PyTorchAutoForge library is based on raw PyTorch and designed to automate DNN development, model tracking and deployment, tightly integrated with MLflow and Optuna. It supports Spiking networks libraries (WIP). Deployment can be performed using ONNX, pyTorch facilities or TensorRT (WIP). The library is designed to be compatible with Jetson Orin Nano Jetpack rev6.1, with bash script to automatically configure virtualenv.
 Author-email: "Pietro Califano (PC)" <petercalifano.gs@gmail.com>
 License: MIT
 License-File: LICENSE
@@ -18,14 +18,13 @@ Requires-Dist: dacite
 Requires-Dist: expelliarmus; platform_machine == 'x86_64'
 Requires-Dist: inputimeout
 Requires-Dist: kornia
-Requires-Dist: lietorch; platform_machine == 'x86_64'
 Requires-Dist: matplotlib<=3.10.0
-Requires-Dist: mlflow<=3.2
+Requires-Dist: mlflow<=3.4
 Requires-Dist: msgpack
 Requires-Dist: mypy
 Requires-Dist: norse; platform_machine == 'x86_64'
 Requires-Dist: numpy
-Requires-Dist: onnx<=1.18.0
+Requires-Dist: onnx
 Requires-Dist: onnxruntime
 Requires-Dist: onnxscript
 Requires-Dist: optuna
@@ -36,7 +35,7 @@ Requires-Dist: seaborn
 Requires-Dist: statsmodels
 Requires-Dist: tonic; platform_machine == 'x86_64'
 Requires-Dist: torch-tb-profiler
-Requires-Dist: torch<=2.8.0; platform_machine == 'x86_64'
+Requires-Dist: torch; platform_machine == 'x86_64'
 Requires-Dist: torchinfo
 Requires-Dist: torchvision; platform_machine == 'x86_64'
 Provides-Extra: cuda-all
@@ -45,35 +44,25 @@ Requires-Dist: pynvml; (platform_machine == 'x86_64') and extra == 'cuda-all'
 Description-Content-Type: text/markdown
 
 # pyTorchAutoForge
-_Warning: Work in progress :)_
+**_WARNING: work in progress. Do not hesitate to open issues for improvements or problems!_**
 
 A library based on PyTorch (<https://pytorch.org/>) and designed to automate ML models development, tracking and deployment, integrated with MLflow and Optuna (<https://mlflow.org/>, <https://optuna.org/>). It also supports spiking networks libraries (WIP). Model optimization and deployment can be performed using ONNx, pyTorch facilities or TensorRT (WIP). The library aims to be compatible with Jetson Orin Nano Jetpack rev6.1. Several other functionalities and utilities for sklearn and pySR (<https://github.com/MilesCranmer/PySR>) are included (see README and documentation).
 
 ## Installation using pip
 
-This is the suggested installation method, the others are mostly intended for development and may not be completely up-to-date with the newest release versions. 
-Run in a conda or virtual environment:
+The suggested installation method is through pip as the others are mostly intended for development and may not be completely up-to-date with the newest release versions. 
+In whatever conda or virtual environment you like (preferably with a sufficiently new torch release, to install from pypi:
 
 ```bash
 pip install pyTorchAutoForge
 ```
 
-Dependencies for the core modules should be installed automatically using pip.
+Or from a local copy of the repository (requires `hatch` module for the build):
 
-## Manual installation (venv)
-
-1) Clone the repository
-2) Create a virtual environment using python >= 3.10 (tested with 3.11), using `python -m venv <your_venv_name>`
-3) Activate the virtual environment using `source <your_venv_name>/bin/activate`
-4) Install the requirements using `pip install -r requirements.txt`
-5) Install the package using `pip install .` in the root folder of the repository
-
-## Manual installation (conda)
-
-### Option A:
-  1) Clone the repository
-  2) Create a new conda environment (python >=3.10) using the provided `enrivonment.yml` file
+```bash
+cd pyTorchAutoforge
+pip install .
+```
 
-### Option B;
-  1) Clone the repository
-  2) Use the automatic installation script `conda_install.sh`. There are several options, use those you need. It will automatically create a new environment named **autoforge**.
+An automatic installation script `conda_install.sh` is provided and should work in most cases. Note that it will automatically create a new environment named **autoforge** and makes several assumptions about your environment.
+Dependencies for the core modules should be installed automatically using pip. However, this is currently not fully tested. Please open related issues.

pytorchautoforge-0.3.1.dev0/README.md

@@ -0,0 +1,23 @@
+# pyTorchAutoForge
+**_WARNING: work in progress. Do not hesitate to open issues for improvements or problems!_**
+
+A library based on PyTorch (<https://pytorch.org/>) and designed to automate ML models development, tracking and deployment, integrated with MLflow and Optuna (<https://mlflow.org/>, <https://optuna.org/>). It also supports spiking networks libraries (WIP). Model optimization and deployment can be performed using ONNx, pyTorch facilities or TensorRT (WIP). The library aims to be compatible with Jetson Orin Nano Jetpack rev6.1. Several other functionalities and utilities for sklearn and pySR (<https://github.com/MilesCranmer/PySR>) are included (see README and documentation).
+
+## Installation using pip
+
+The suggested installation method is through pip as the others are mostly intended for development and may not be completely up-to-date with the newest release versions. 
+In whatever conda or virtual environment you like (preferably with a sufficiently new torch release, to install from pypi:
+
+```bash
+pip install pyTorchAutoForge
+```
+
+Or from a local copy of the repository (requires `hatch` module for the build):
+
+```bash
+cd pyTorchAutoforge
+pip install .
+```
+
+An automatic installation script `conda_install.sh` is provided and should work in most cases. Note that it will automatically create a new environment named **autoforge** and makes several assumptions about your environment.
+Dependencies for the core modules should be installed automatically using pip. However, this is currently not fully tested. Please open related issues.

{pytorchautoforge-0.2.2.dev0 → pytorchautoforge-0.3.1.dev0}/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '0.2.2.dev0'
-__version_tuple__ = version_tuple = (0, 2, 2, 'dev0')
+__version__ = version = '0.3.1.dev0'
+__version_tuple__ = version_tuple = (0, 3, 1, 'dev0')
 
 __commit_id__ = commit_id = None

{pytorchautoforge-0.2.2.dev0 → pytorchautoforge-0.3.1.dev0}/pyTorchAutoForge/api/onnx/ModelHandlerONNx.py

@@ -5,9 +5,7 @@ from pyTorchAutoForge.utils import AddZerosPadding, torch_to_numpy, timeit_avera
 from numpy.testing import assert_allclose
 from pyTorchAutoForge.utils import numpy_to_torch
 
-# TODO: add support to use onnx simplify 
-#simplified_model, check_status = onnxsim.simplify(onnx_model)
-
+# TODO update torch exporter API
 class ModelHandlerONNx:
     """
      _summary_
@@ -64,12 +62,13 @@ class ModelHandlerONNx:
             print('No name provided for the ONNx model. Assign default value.')
             onnx_model_name = 'onnx_export'
         
-        os.makedirs(os.path.dirname(self.onnx_export_path), exist_ok=True)
+        # Create export folder if it does not exist
+        os.makedirs(self.onnx_export_path, exist_ok=True)
 
         # Check if any model is already exported in the export path and append ID to the filename if any
         nameID = 0
         onnx_model_name_tmp = onnx_model_name + "_" + str(nameID)
-        while os.path.isfile(os.path.join(os.path.dirname(self.onnx_export_path), onnx_model_name_tmp + ".onnx")):
+        while os.path.isfile(os.path.join(self.onnx_export_path, onnx_model_name_tmp + ".onnx")):
             onnx_model_name_tmp = onnx_model_name + "_" + str(nameID)
             nameID += 1
 
@@ -112,6 +111,7 @@ class ModelHandlerONNx:
 
         # Assign input tensor from init if not provided
         if input_tensor is None and self.dummy_input_sample is not None:
+            assert torch.is_tensor(self.dummy_input_sample), "Dummy input sample must be a torch tensor."
             input_tensor = self.dummy_input_sample
         else:
             raise ValueError("Input tensor must be provided or dummy input sample must be provided when constructing this class.")
@@ -150,7 +150,7 @@ class ModelHandlerONNx:
             # Reload the model from disk
             self.onnx_model = self.onnx_load(onnx_filepath=self.onnx_filepath)
             
-            self.onnx_validate(self.onnx_model,
+            self.onnx_validate(onnx_model=self.onnx_model,
                                test_sample=numpy_to_torch(self.dummy_input_sample))
 
         if self.run_onnx_simplify:
@@ -165,7 +165,7 @@ class ModelHandlerONNx:
                             onnx_model_name: str = 'onnx_dynamo_export', 
                             dynamic_axes: dict | None = None,
                             IO_names: dict | None = None,
-                            enable_verbose: bool = False) -> None:
+                            enable_verbose: bool = False) -> str:
         """Export the model to ONNx format using TorchDynamo."""
 
         # Prepare export folder and compose name
@@ -174,6 +174,7 @@ class ModelHandlerONNx:
 
         # Assign input tensor from init if not provided
         if input_tensor is None and self.dummy_input_sample is not None:
+            assert torch.is_tensor(self.dummy_input_sample), "Dummy input sample must be a torch tensor."
             input_tensor = self.dummy_input_sample
         else:
             raise ValueError("Input tensor must be provided or dummy input sample must be provided when constructing this class.")
@@ -206,6 +207,9 @@ class ModelHandlerONNx:
                                         dynamo=True, report=self.generate_report,
                                          verbose=enable_verbose)
 
+        if onnx_program is None:
+            raise RuntimeError("TorchDynamo ONNX export failed, no model generated.")
+        
         # Call model optimization
         onnx_program.optimize()
 
@@ -254,8 +258,8 @@ class ModelHandlerONNx:
     # %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     def onnx_validate(self, 
                       onnx_model: onnx.ModelProto | str, 
-                      test_sample : torch.Tensor | numpy.ndarray = None, 
-                      output_sample : torch.Tensor | numpy.ndarray = None) -> None:
+                      test_sample : torch.Tensor | numpy.ndarray | None = None, 
+                      output_sample : torch.Tensor | numpy.ndarray | None = None) -> None:
         """Validate the ONNx model using onnx.checker.check_model."""
 
         # If onnx_model is a string, load the model from the file

{pytorchautoforge-0.2.2.dev0 → pytorchautoforge-0.3.1.dev0}/pyTorchAutoForge/datasets/DatasetClasses.py

@@ -872,7 +872,7 @@ class ImagesLabelsDatasetBase(Dataset):
 
         return cls(image_paths, camera_path, transform, image_backend)
 
-    def get_all_labels_container(self):
+    def get_all_labels_container(self) -> ImagesLabelsContainer:
         """
         Get the labels container for this dataset.
 
@@ -899,7 +899,7 @@ class ImagesLabelsDatasetBase(Dataset):
 
         container = ImagesLabelsContainer(images=np.empty_like((0, 0)),
                                           labels=lbl_array,
-                                          labels_datakeys=self.dset_cfg.lbl_dtype,
+                                          labels_datakeys=self.dset_cfg.lbl_vector_data_keys,
                                           labels_sizes=lbl_size_dict)
         return container