nnodely 1.3.0__tar.gz → 1.4.0__tar.gz

{nnodely-1.3.0/nnodely.egg-info → nnodely-1.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: nnodely
-Version: 1.3.0
+Version: 1.4.0
 Summary: Model-structured neural network framework for the modeling and control of physical systems
 Author-email: Gastone Pietro Rosati Papini <tonegas@gmail.com>
 License: MIT License
@@ -32,15 +32,16 @@ Classifier: Operating System :: OS Independent
 Requires-Python: <3.13,>=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: numpy==1.26.4; platform_machine == "x86_64"
-Requires-Dist: torch==2.2.2; platform_machine == "x86_64"
-Requires-Dist: numpy; platform_machine != "x86_64"
-Requires-Dist: torch; platform_machine != "x86_64"
+Requires-Dist: numpy==1.26.4; platform_machine == "x86_64" and python_version == "3.10"
+Requires-Dist: torch==2.2.2; platform_machine == "x86_64" and python_version == "3.10"
+Requires-Dist: numpy
+Requires-Dist: torch
 Requires-Dist: onnx
 Requires-Dist: pandas
 Requires-Dist: reportlab
 Requires-Dist: matplotlib
 Requires-Dist: onnxruntime
+Dynamic: license-file
 
 <p align="center">
 <img src="https://raw.githubusercontent.com/tonegas/nnodely/main/imgs/logo_white_info.png" alt="logo" >
@@ -51,7 +52,7 @@ Requires-Dist: onnxruntime
 [![Coverage Status](https://readthedocs.org/projects/nnodely/badge/?version=latest&style=default)](https://nnodely.readthedocs.io/en/latest/)
 
 <a name="readme-top"></a>
-## Model-structured neural network framework for the modeling and control of physical systems
+# Model-structured neural network framework for the modeling and control of physical systems
 
 Modeling, control, and estimation of physical systems are fundamental to a wide range of engineering applications. 
 However, integrating data-driven methods like neural networks into these domains presents significant challenges, 
@@ -62,7 +63,7 @@ MSNNs merge the flexibility of neural networks with structures grounded in physi
 providing a powerful tool for representing and managing complex physical systems. Moreover, a MSNN thanks to the reduced
 number of parameters needs fewer data for training and can be used in real-time applications.
 
-### Why use nnodely
+## Why use nnodely
 
 The framework's goal is to allow the users fast modeling and control of a any mechanical systems using 
 a hybrind approach between neural networks and physical models.
@@ -92,13 +93,13 @@ a Pytorch (nnodely independent) and ONNX export.
       <a href="#settingstarted">Getting Started</a>
     </li>
     <li>
-      <a href="#basicfunctionalities">Basic Functionalities</a>
+      <a href="#basicfunctionalities">Basic Example</a>
       <ul>
-        <li><a href="#structuredneuralmodel">Build the structured neural model</a></li>
-        <li><a href="#neuralizemodel">Neuralize the structured neural model</a></li>
+        <li><a href="#structuredneuralmodel">Build the neural model</a></li>
+        <li><a href="#neuralizemodel">Neuralize the neural model</a></li>
         <li><a href="#loaddataset">Load the dataset</a></li>
-        <li><a href="#trainmodel">Train the structured neural network</a></li>
-        <li><a href="#testmodel">Test the structured neural model</a></li>
+        <li><a href="#trainmodel">Train the neural model</a></li>
+        <li><a href="#testmodel">Test the neural model</a></li>
       </ul>
     </li>
     <li>
@@ -109,15 +110,14 @@ a Pytorch (nnodely independent) and ONNX export.
         <li><a href="#examplesfolder">examples folder</a></li>
       </ul>
     </li>
-    <li>
-      <a href="#license">License</a>
-    </li>
+    <li><a href="#license">License</a></li>
+    <li><a href="#references">References</a></li>
   </ol>
 </details>
 
 <!-- GETTING STARTED -->
 <a name="settingstarted"></a>
-### Getting Started
+## Getting Started
 You can install the nnodely framework from PyPI via:
   ```sh
   pip install nnodely
@@ -191,7 +191,7 @@ Our formulation is more general and can take into account the noise of the measu
 The estimator can also be seen as the composition of the force contributions due to the position and velocity of the mass plus the contribution of external forces.
 
 <a name="neuralizemodel"></a>
-### Neuralize the structured neural model
+### Neuralize the neural model
 Let's now try to train our observer using the data we have.
 We perform:
 ```python
@@ -233,7 +233,7 @@ mass_spring_damper.loadData(name='mass_spring_dataset', source=data_folder, form
 Finally, the dataset is loaded. **nnodely** loads all the files that are in a source folder.
 
 <a name="trainmodel"></a>
-### Train the structured neural network
+### Train the neural network
 Using the dataset created the training is performed on the model.
 
 ```python
@@ -241,7 +241,7 @@ mass_spring_damper.trainModel()
 ```
 
 <a name="testmodel"></a>
-### Test the structured neural model
+### Test the neural model
 In order to test the results we need to create a input, in this case is defined by:
 1. `x` with 5 sample because the sample time is 0.2 and the window of `x`is 1 second.
 2. `F` is one sample because only the last sample is needed.
@@ -309,7 +309,7 @@ This operation is presented in [[1]](#1).
 3. __localmodel.py__ this file contains the logic for build a local model. This operation is presented in [[1]](#1), [[3]](#3), [[4]](#4) and [[5]](#5).
 4. __parametricfunction.py__ are the user custom function. The function can use the pytorch syntax. A parametric function is presented in [[3]](#3), [[4]](#4), [[5]](#5).
 5. __equationlearner.py__ contains the logic for the equation learner. The equation learner is used for learn a relation input outpur following a list of activation functions. The first implementation is presented in [[6]](#6).
-6. __timeoperation.py__ contains the time operation functions. The time operation are used for extract a time window from a signal.
+6. __timeoperation.py__ contains the time operation functions. The time operation are used for extract a time window from a signal. The derivative operation can be used to implement Physics-informed neural network [[7]](#7) Sobolev learning [[8]](#8).
 
 <a name="testsfolder"></a>
 ### Tests Folder
@@ -329,6 +329,7 @@ This project is released under the license [License: MIT](https://opensource.org
 
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 
+<a name="references"></a>
 ## References
 <a id="1">[1]</a> 
 Mauro Da Lio, Daniele Bortoluzzi, Gastone Pietro Rosati Papini. (2019). 
@@ -359,4 +360,17 @@ and [[code extended]](https://github.com/tonegas/nnodely-applications/blob/main/
 <a id="6">[6]</a> 
 Hector Perez-Villeda, Justus Piater, Matteo Saveriano. (2023).
 Learning and extrapolation of robotic skills using task-parameterized equation learner networks.
-Robotics and Autonomous Systems. https://doi.org/10.1016/j.robot.2022.104309
+Robotics and Autonomous Systems. https://doi.org/10.1016/j.robot.2022.104309 (look the [[code]](https://github.com/tonegas/nnodely-applications/blob/main/equation_learner/equation_learner.ipynb))
+
+<a id="7">[7]</a> 
+M. Raissi. P. Perdikaris b, G.E. Karniadakis a. (2019).
+Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
+Journal of Computational Physics. https://doi.org/10.1016/j.jcp.2018.10.045 (look the [[example Burger's equation]](https://github.com/tonegas/nnodely-applications/blob/main/pinn/pinn_Burgers_equation.ipynb))
+
+<a id="8">[8]</a> 
+Wojciech Marian Czarnecki, Simon Osindero, Max Jaderberg, Grzegorz Świrszcz, Razvan Pascanu. (2017).
+Sobolev Training for Neural Networks.
+arXiv. https://doi.org/10.48550/arXiv.1706.04859 (look the [[code]](https://github.com/tonegas/nnodely-applications/blob/main/sobolev/Sobolev_learning.ipynb))
+
+
+<p align="right">(<a href="#readme-top">back to top</a>)</p>

{nnodely-1.3.0 → nnodely-1.4.0}/README.md

@@ -7,7 +7,7 @@
 [![Coverage Status](https://readthedocs.org/projects/nnodely/badge/?version=latest&style=default)](https://nnodely.readthedocs.io/en/latest/)
 
 <a name="readme-top"></a>
-## Model-structured neural network framework for the modeling and control of physical systems
+# Model-structured neural network framework for the modeling and control of physical systems
 
 Modeling, control, and estimation of physical systems are fundamental to a wide range of engineering applications. 
 However, integrating data-driven methods like neural networks into these domains presents significant challenges, 
@@ -18,7 +18,7 @@ MSNNs merge the flexibility of neural networks with structures grounded in physi
 providing a powerful tool for representing and managing complex physical systems. Moreover, a MSNN thanks to the reduced
 number of parameters needs fewer data for training and can be used in real-time applications.
 
-### Why use nnodely
+## Why use nnodely
 
 The framework's goal is to allow the users fast modeling and control of a any mechanical systems using 
 a hybrind approach between neural networks and physical models.
@@ -48,13 +48,13 @@ a Pytorch (nnodely independent) and ONNX export.
       <a href="#settingstarted">Getting Started</a>
     </li>
     <li>
-      <a href="#basicfunctionalities">Basic Functionalities</a>
+      <a href="#basicfunctionalities">Basic Example</a>
       <ul>
-        <li><a href="#structuredneuralmodel">Build the structured neural model</a></li>
-        <li><a href="#neuralizemodel">Neuralize the structured neural model</a></li>
+        <li><a href="#structuredneuralmodel">Build the neural model</a></li>
+        <li><a href="#neuralizemodel">Neuralize the neural model</a></li>
         <li><a href="#loaddataset">Load the dataset</a></li>
-        <li><a href="#trainmodel">Train the structured neural network</a></li>
-        <li><a href="#testmodel">Test the structured neural model</a></li>
+        <li><a href="#trainmodel">Train the neural model</a></li>
+        <li><a href="#testmodel">Test the neural model</a></li>
       </ul>
     </li>
     <li>
@@ -65,15 +65,14 @@ a Pytorch (nnodely independent) and ONNX export.
         <li><a href="#examplesfolder">examples folder</a></li>
       </ul>
     </li>
-    <li>
-      <a href="#license">License</a>
-    </li>
+    <li><a href="#license">License</a></li>
+    <li><a href="#references">References</a></li>
   </ol>
 </details>
 
 <!-- GETTING STARTED -->
 <a name="settingstarted"></a>
-### Getting Started
+## Getting Started
 You can install the nnodely framework from PyPI via:
   ```sh
   pip install nnodely
@@ -147,7 +146,7 @@ Our formulation is more general and can take into account the noise of the measu
 The estimator can also be seen as the composition of the force contributions due to the position and velocity of the mass plus the contribution of external forces.
 
 <a name="neuralizemodel"></a>
-### Neuralize the structured neural model
+### Neuralize the neural model
 Let's now try to train our observer using the data we have.
 We perform:
 ```python
@@ -189,7 +188,7 @@ mass_spring_damper.loadData(name='mass_spring_dataset', source=data_folder, form
 Finally, the dataset is loaded. **nnodely** loads all the files that are in a source folder.
 
 <a name="trainmodel"></a>
-### Train the structured neural network
+### Train the neural network
 Using the dataset created the training is performed on the model.
 
 ```python
@@ -197,7 +196,7 @@ mass_spring_damper.trainModel()
 ```
 
 <a name="testmodel"></a>
-### Test the structured neural model
+### Test the neural model
 In order to test the results we need to create a input, in this case is defined by:
 1. `x` with 5 sample because the sample time is 0.2 and the window of `x`is 1 second.
 2. `F` is one sample because only the last sample is needed.
@@ -265,7 +264,7 @@ This operation is presented in [[1]](#1).
 3. __localmodel.py__ this file contains the logic for build a local model. This operation is presented in [[1]](#1), [[3]](#3), [[4]](#4) and [[5]](#5).
 4. __parametricfunction.py__ are the user custom function. The function can use the pytorch syntax. A parametric function is presented in [[3]](#3), [[4]](#4), [[5]](#5).
 5. __equationlearner.py__ contains the logic for the equation learner. The equation learner is used for learn a relation input outpur following a list of activation functions. The first implementation is presented in [[6]](#6).
-6. __timeoperation.py__ contains the time operation functions. The time operation are used for extract a time window from a signal.
+6. __timeoperation.py__ contains the time operation functions. The time operation are used for extract a time window from a signal. The derivative operation can be used to implement Physics-informed neural network [[7]](#7) Sobolev learning [[8]](#8).
 
 <a name="testsfolder"></a>
 ### Tests Folder
@@ -285,6 +284,7 @@ This project is released under the license [License: MIT](https://opensource.org
 
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 
+<a name="references"></a>
 ## References
 <a id="1">[1]</a> 
 Mauro Da Lio, Daniele Bortoluzzi, Gastone Pietro Rosati Papini. (2019). 
@@ -315,4 +315,17 @@ and [[code extended]](https://github.com/tonegas/nnodely-applications/blob/main/
 <a id="6">[6]</a> 
 Hector Perez-Villeda, Justus Piater, Matteo Saveriano. (2023).
 Learning and extrapolation of robotic skills using task-parameterized equation learner networks.
-Robotics and Autonomous Systems. https://doi.org/10.1016/j.robot.2022.104309
+Robotics and Autonomous Systems. https://doi.org/10.1016/j.robot.2022.104309 (look the [[code]](https://github.com/tonegas/nnodely-applications/blob/main/equation_learner/equation_learner.ipynb))
+
+<a id="7">[7]</a> 
+M. Raissi. P. Perdikaris b, G.E. Karniadakis a. (2019).
+Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
+Journal of Computational Physics. https://doi.org/10.1016/j.jcp.2018.10.045 (look the [[example Burger's equation]](https://github.com/tonegas/nnodely-applications/blob/main/pinn/pinn_Burgers_equation.ipynb))
+
+<a id="8">[8]</a> 
+Wojciech Marian Czarnecki, Simon Osindero, Max Jaderberg, Grzegorz Świrszcz, Razvan Pascanu. (2017).
+Sobolev Training for Neural Networks.
+arXiv. https://doi.org/10.48550/arXiv.1706.04859 (look the [[code]](https://github.com/tonegas/nnodely-applications/blob/main/sobolev/Sobolev_learning.ipynb))
+
+
+<p align="right">(<a href="#readme-top">back to top</a>)</p>

nnodely-1.4.0/nnodely/__init__.py

@@ -0,0 +1,47 @@
+
+__version__ = '1.4.0'
+
+import sys
+major, minor = sys.version_info.major, sys.version_info.minor
+
+import logging
+LOG_LEVEL = logging.INFO
+
+if major < 3:
+    sys.exit("Sorry, Python 2 is not supported. You need Python >= 3.10 for "+__package__+".")
+elif minor < 9:
+    sys.exit("Sorry, You need Python >= 3.10 for "+__package__+".")
+else:
+    print(f'>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>'+
+          f' {__package__}_v{__version__} '.center(20, '-')+
+          f'<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<')
+
+# Network input, outputs and parameters
+from nnodely.layers.input import Input, State, Connect, ClosedLoop
+from nnodely.layers.parameter import Parameter, Constant, SampleTime
+from nnodely.layers.output import Output
+
+# Network elements
+from nnodely.layers.activation import Relu, ELU, Softmax, Sigmoid, Identity
+from nnodely.layers.fir import Fir
+from nnodely.layers.linear import Linear
+from nnodely.layers.arithmetic import Add, Sum, Sub, Mul, Div, Pow, Neg
+from nnodely.layers.trigonometric import Sin, Cos, Tan, Cosh, Tanh, Sech
+from nnodely.layers.parametricfunction import ParamFun
+from nnodely.layers.fuzzify import Fuzzify
+from nnodely.layers.part import  Part, Select, Concatenate, SamplePart, SampleSelect, TimePart, TimeConcatenate
+from nnodely.layers.localmodel import LocalModel
+from nnodely.layers.equationlearner import EquationLearner
+from nnodely.layers.timeoperation import Integrate, Derivate
+from nnodely.layers.interpolation import Interpolation
+
+# Main nnodely classes
+from nnodely.nnodely import nnodely, Modely, clearNames
+from nnodely.visualizer import Visualizer, TextVisualizer, MPLVisualizer, MPLNotebookVisualizer
+from nnodely.exporter import StandardExporter
+
+# Basic nnodely
+from nnodely.basic.optimizer import Optimizer, SGD, Adam
+
+# Support functions
+from nnodely.support.initializer import init_negexp, init_lin, init_constant, init_exp

{nnodely-1.3.0/nnodely → nnodely-1.4.0/nnodely/basic}/loss.py

@@ -1,8 +1,8 @@
 import torch.nn as nn
 import torch
-from nnodely.utils import check
+from nnodely.support.utils import check
 
-available_losses = ['mse', 'rmse', 'mae']
+available_losses = ['mse', 'rmse', 'mae', 'cross_entropy']
 
 # class CustomRMSE(nn.Module):
 #     def __init__(self):
@@ -15,15 +15,22 @@ available_losses = ['mse', 'rmse', 'mae']
 #         return self.mse(inA, inB)
 
 class CustomLoss(nn.Module):
-    def __init__(self, loss_type='mse'):
+    def __init__(self, loss_type='mse', **kwargs):
         super(CustomLoss, self).__init__()
         check(loss_type in available_losses, TypeError, f'The \"{loss_type}\" loss is not available. Possible losses are: {available_losses}.')
         self.loss_type = loss_type
-        self.loss = nn.MSELoss()
-        if self.loss_type == 'mae':
-            self.loss = nn.L1Loss()
+        self.loss = nn.MSELoss(**kwargs)
+        if callable(loss_type):
+            self.loss = loss_type
+        elif self.loss_type == 'mae':
+            self.loss = nn.L1Loss(**kwargs)
+        elif self.loss_type == 'cross_entropy':
+            self.loss = nn.CrossEntropyLoss(**kwargs)
         
     def forward(self, inA, inB):
+        if self.loss_type == 'cross_entropy':
+            inB = inB.squeeze().float() if inA.shape == inB.shape else inB.squeeze().long()
+            inA = inA.squeeze()
         res = self.loss(inA,inB)
         if self.loss_type == 'rmse':
             res = torch.sqrt(res)

{nnodely-1.3.0/nnodely → nnodely-1.4.0/nnodely/basic}/model.py

@@ -1,6 +1,6 @@
 from itertools import product
-from nnodely.utils import TORCH_DTYPE
-from nnodely import initializer
+from nnodely.support.utils import TORCH_DTYPE
+from nnodely.support import initializer
 import numpy as np
 
 import torch.nn as nn
@@ -34,14 +34,6 @@ class Model(nn.Module):
         self.input_n_samples = {key:value['ntot'] for key, value in (model_def['Inputs']|model_def['States']).items()}
         self.minimizers_keys = [self.minimizers[key]['A'] for key in self.minimizers] + [self.minimizers[key]['B'] for key in self.minimizers]
 
-        #print('inputs: ',self.inputs)
-        #print('outputs: ',self.outputs)
-        #print('relations: ',self.relations)
-        #print('params: ',self.params)
-        #print('constants: ',self.constants)
-        #print('sample_time: ',self.sample_time)
-        #print('states: ',self.states)
-
         ## Build the network
         self.all_parameters = {}
         self.all_constants = {}
@@ -149,7 +141,7 @@ class Model(nn.Module):
         self.network_output_predictions = set(self.outputs.values())
 
         ## list of network minimization outputs
-        self.network_output_minimizers = [] 
+        self.network_output_minimizers = []
         for _,value in self.minimizers.items():
             self.network_output_minimizers.append(self.outputs[value['A']]) if value['A'] in self.outputs.keys() else self.network_output_minimizers.append(value['A'])
             self.network_output_minimizers.append(self.outputs[value['B']]) if value['B'] in self.outputs.keys() else self.network_output_minimizers.append(value['B'])
@@ -178,7 +170,7 @@ class Model(nn.Module):
                             layer_inputs.append(self.all_constants[key])
                         elif key in available_states: ## relation that takes a state
                             layer_inputs.append(kwargs[key])
-                        elif key in available_inputs:  ## relation that takes inputs 
+                        elif key in available_inputs:  ## relation that takes inputs
                             layer_inputs.append(kwargs[key])
                         elif key in self.all_parameters.keys(): ## relation that takes parameters
                             layer_inputs.append(self.all_parameters[key])
@@ -205,7 +197,7 @@ class Model(nn.Module):
         minimize_dict = {}
         for key in self.minimizers_keys:
             minimize_dict[key] = result_dict[self.outputs[key]] if key in self.outputs.keys() else result_dict[key]
-                
+
         return output_dict, minimize_dict, closed_loop_update_dict, connect_update_dict