readerbotda 1.3.2__tar.gz

readerbotda-1.3.2/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.3
+Name: readerbotda
+Version: 1.3.2
+Summary: Pacchetto per lettura e visualizzazione file di log sensore BOTDA Cohaerentia
+Author: Marco Brunero
+Author-email: marco.brunero@cohaerentia.com
+Requires-Python: >3.10
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: bokeh (>=3.7.2,<4.0.0)
+Requires-Dist: dacite (>=1.9.2,<2.0.0)
+Requires-Dist: numpy (>=2.2.4,<3.0.0)
+Requires-Dist: plotly (>=6.0.1,<7.0.0)
+Requires-Dist: progress (>=1.6,<2.0)
+Requires-Dist: typer (>=0.15.2,<0.16.0)
+Description-Content-Type: text/markdown
+
+# Reader BOTDA
+
+Pacchetto per lettura e visualizzazione delle misure salvate da software per sensore BOTDA.
+
+## Installazione
+
+Da cartella locale dove è stato scaricato il pacchetto è possibile installare tramite:
+
+```bash
+pip install .
+```
+
+In questo modo i files dovrebbero essere ancora editabili e la versione installata dovrebbe automaticamente aggiornarsi. Nota bene: se si sta usando un notebook, è necessario riavviare il kernel, non è sufficiente re importare il pacchetto.
+
+Dovrebbe anche essere possibile installare direttamente tramite link al repository privato ma devo ancora guardare come.
+
+## Utilizzo
+
+Per prima cosa è necessario caricare il necessario dai due moduli di lettura e visualizzazione:
+
+```python
+from ReaderBOTDA.reader import Profile, multipleProfile, Raw
+from ReaderBOTDA.plotter.plotly import Plotly
+```
+
+Per il resto del codice utilizzeremo sempre lo stesso oggetto `plotter`, che va inizializzato e poi fornito quando si leggono i files.
+
+```python
+plotter = Plotly(theme='plotly')
+```
+
+Si può scegliere un tema che verrà usato per tutte le misure generate dal Plotter. [doc link](https://plotly.com/python/templates/)
+
+> Available templates: ['ggplot2', 'seaborn', 'simple_white', 'plotly', 'plotly_white', 'plotly_dark', 'presentation', 'xgridoff', 'ygridoff', 'gridon', 'none']
+
+L'idea è che chiunque potrebbe creare una nuova classe parente della classe `Plotter` in modo che non venga utilizzato il pacchetto `plotly` ma quello che si preferirà (bokeh, matplotlib...). La nuova classe dovrà avere una definizione di tutti i metodi astratti della classe `Plotter`.
+
+### Misura Singola
+
+Lettura di singola misura da un singolo file "profilo".
+
+```python
+misura = Profile('data/profiles/2021-11-08_16-51-16.652_rawarray.json',plotter=plotter)
+
+fig = misura.plot() #misura.plot(title='Titolo custom')
+plotter.show(fig)
+fig.write_html("prova.html", full_html=False, include_plotlyjs='cdn')
+```
+
+### Misure multiple in una cartella
+
+Lettura di tutti i file di tipo profilo in una cartella:
+
+```python
+from datetime import datetime
+folder = 'data/profiles/'
+misure = multipleProfile(folder, plotter = plotter)
+misure = multipleProfile(folder, plotter = plotter, n_measure=10)
+misure = multipleProfile(folder, plotter = plotter, n_measure=10, start_measure=5)
+misure.plot()
+misure.plot(startTime=datetime(2021,11,8,16,51,18))
+misure.plot(startTime=datetime(2021,11,8,16,51,18),stopTime=datetime(2021,11,8,16,51,21))
+```
+
+E' possibile limitare la lettura a `n_measure` file consecutivi; in questo caso è anche possibile utilizzare il parametro `start_measure` per leggere `n_measure` a partire da `start_measure`.
+Nei plot è quindi possibile indicare un range temporale utilizzando i parametri opzionali `startTime` e/o `stopTime`.
+
+Infine è possibile calcolare e visualizzare media e deviazione standard dei profili misurati:
+
+```python
+misure.calcStatistics(plot=True)
+```
+
+#### Leggere solo un subset
+
+Sono disponibili i parametri opzionali:
+
+- `start_measure`, che può essere un intero o un `datetime.datetime`
+- `stop_measure`, che può essere un `datetime.datetime`
+- `n_measure`, che può essere un intero
+
+```python
+misure = multipleProfile(folder, plotter = plotter, n_measure=2, start_measure=1)
+
+from datetime import datetime
+start_datetime = datetime(2023,2,21,13,56,0)
+stop_datetime = datetime(2023,2,21,13,57,0)
+
+misure = multipleProfile(folder, plotter = plotter,start_measure=start_datetime,stop_measure=stop_datetime)
+misure = multipleProfile(folder, plotter = plotter,start_measure=start_datetime,n_measure=2)
+misure = multipleProfile(folder, plotter = plotter,stop_measure=stop_datetime)
+misure = multipleProfile(folder, plotter = plotter,start_measure=2,stop_measure=stop_datetime)
+```
+
+#### Calcolo correlazione
+
+```python
+correlazioni_max = misure.calcCorrelations(type='max',reference='previous',range=(20,200))
+correlazioni_bfs = misure.calcCorrelations(type='bfs',reference='previous')
+correlazioni_bfs = misure.calcCorrelations(type='bfs',reference='first')
+```
+
+Si utilizza la funzione [`np.corrcoef`](https://numpy.org/doc/stable/reference/generated/numpy.corrcoef.html) per calcolo correlazione tra varie misure, dell'array dei massimi o dell'array dei bfs. Il riferimento su cui è calcolata la correlazione può essere la misura precedente o la primissima misura del set. E' inoltre possibile indicare un range, espresso in campioni, su cui limitare il calcolo. La funzione `calcCorrelations` ritorna un `np.array`.
+
+### Misura Raw
+
+Lettura di file di debug che contiene intera matrice BGS e test dei metodi disponibili.
+
+```python
+raw = Raw(filename='data/raw/2021-11-08_16-51-16.652_rawmatrix.json', plotter=plotter)
+raw.plot2d(title='prova titolo')
+raw.plotMax()
+raw.plot3d()
+```
+
+E' possibile plottare tutti i profili BGS e indicare uno specifico indice da mettere in primo piano:
+
+```python
+fig = raw.plotBGS(index=125)
+plotter.show(fig)
+```
+
+## Plotter Bokeh
+
+Oltre al plotter Plotly è disponibile anche una versione che utilizza il pacchetto `bokeh`. Non sono al momento disponibili i metodi `Bokeh.plot2d()` e `Bokeh.plot3d()`.
+
+```python
+from ReaderBOTDA.plotter.bokeh import Bokeh
+from bokeh.io import output_notebook
+output_notebook()
+plotter = Bokeh(theme='night_sky')
+```
+
+I temi a disposizione di default sono:
+>caliber, dark_minimal, light_minimal, night_sky, contrast
+
+A questo punto è possibile chiamare i vari metodi come già mostrato per plotter Plotly:
+
+```python
+misura = Profile('data/profiles/2021-11-08_16-51-16.652_rawarray.json',plotter=plotter)
+fig = misura.plot()
+plotter.show(fig)
+```
+

readerbotda-1.3.2/README.md

@@ -0,0 +1,143 @@
+# Reader BOTDA
+
+Pacchetto per lettura e visualizzazione delle misure salvate da software per sensore BOTDA.
+
+## Installazione
+
+Da cartella locale dove è stato scaricato il pacchetto è possibile installare tramite:
+
+```bash
+pip install .
+```
+
+In questo modo i files dovrebbero essere ancora editabili e la versione installata dovrebbe automaticamente aggiornarsi. Nota bene: se si sta usando un notebook, è necessario riavviare il kernel, non è sufficiente re importare il pacchetto.
+
+Dovrebbe anche essere possibile installare direttamente tramite link al repository privato ma devo ancora guardare come.
+
+## Utilizzo
+
+Per prima cosa è necessario caricare il necessario dai due moduli di lettura e visualizzazione:
+
+```python
+from ReaderBOTDA.reader import Profile, multipleProfile, Raw
+from ReaderBOTDA.plotter.plotly import Plotly
+```
+
+Per il resto del codice utilizzeremo sempre lo stesso oggetto `plotter`, che va inizializzato e poi fornito quando si leggono i files.
+
+```python
+plotter = Plotly(theme='plotly')
+```
+
+Si può scegliere un tema che verrà usato per tutte le misure generate dal Plotter. [doc link](https://plotly.com/python/templates/)
+
+> Available templates: ['ggplot2', 'seaborn', 'simple_white', 'plotly', 'plotly_white', 'plotly_dark', 'presentation', 'xgridoff', 'ygridoff', 'gridon', 'none']
+
+L'idea è che chiunque potrebbe creare una nuova classe parente della classe `Plotter` in modo che non venga utilizzato il pacchetto `plotly` ma quello che si preferirà (bokeh, matplotlib...). La nuova classe dovrà avere una definizione di tutti i metodi astratti della classe `Plotter`.
+
+### Misura Singola
+
+Lettura di singola misura da un singolo file "profilo".
+
+```python
+misura = Profile('data/profiles/2021-11-08_16-51-16.652_rawarray.json',plotter=plotter)
+
+fig = misura.plot() #misura.plot(title='Titolo custom')
+plotter.show(fig)
+fig.write_html("prova.html", full_html=False, include_plotlyjs='cdn')
+```
+
+### Misure multiple in una cartella
+
+Lettura di tutti i file di tipo profilo in una cartella:
+
+```python
+from datetime import datetime
+folder = 'data/profiles/'
+misure = multipleProfile(folder, plotter = plotter)
+misure = multipleProfile(folder, plotter = plotter, n_measure=10)
+misure = multipleProfile(folder, plotter = plotter, n_measure=10, start_measure=5)
+misure.plot()
+misure.plot(startTime=datetime(2021,11,8,16,51,18))
+misure.plot(startTime=datetime(2021,11,8,16,51,18),stopTime=datetime(2021,11,8,16,51,21))
+```
+
+E' possibile limitare la lettura a `n_measure` file consecutivi; in questo caso è anche possibile utilizzare il parametro `start_measure` per leggere `n_measure` a partire da `start_measure`.
+Nei plot è quindi possibile indicare un range temporale utilizzando i parametri opzionali `startTime` e/o `stopTime`.
+
+Infine è possibile calcolare e visualizzare media e deviazione standard dei profili misurati:
+
+```python
+misure.calcStatistics(plot=True)
+```
+
+#### Leggere solo un subset
+
+Sono disponibili i parametri opzionali:
+
+- `start_measure`, che può essere un intero o un `datetime.datetime`
+- `stop_measure`, che può essere un `datetime.datetime`
+- `n_measure`, che può essere un intero
+
+```python
+misure = multipleProfile(folder, plotter = plotter, n_measure=2, start_measure=1)
+
+from datetime import datetime
+start_datetime = datetime(2023,2,21,13,56,0)
+stop_datetime = datetime(2023,2,21,13,57,0)
+
+misure = multipleProfile(folder, plotter = plotter,start_measure=start_datetime,stop_measure=stop_datetime)
+misure = multipleProfile(folder, plotter = plotter,start_measure=start_datetime,n_measure=2)
+misure = multipleProfile(folder, plotter = plotter,stop_measure=stop_datetime)
+misure = multipleProfile(folder, plotter = plotter,start_measure=2,stop_measure=stop_datetime)
+```
+
+#### Calcolo correlazione
+
+```python
+correlazioni_max = misure.calcCorrelations(type='max',reference='previous',range=(20,200))
+correlazioni_bfs = misure.calcCorrelations(type='bfs',reference='previous')
+correlazioni_bfs = misure.calcCorrelations(type='bfs',reference='first')
+```
+
+Si utilizza la funzione [`np.corrcoef`](https://numpy.org/doc/stable/reference/generated/numpy.corrcoef.html) per calcolo correlazione tra varie misure, dell'array dei massimi o dell'array dei bfs. Il riferimento su cui è calcolata la correlazione può essere la misura precedente o la primissima misura del set. E' inoltre possibile indicare un range, espresso in campioni, su cui limitare il calcolo. La funzione `calcCorrelations` ritorna un `np.array`.
+
+### Misura Raw
+
+Lettura di file di debug che contiene intera matrice BGS e test dei metodi disponibili.
+
+```python
+raw = Raw(filename='data/raw/2021-11-08_16-51-16.652_rawmatrix.json', plotter=plotter)
+raw.plot2d(title='prova titolo')
+raw.plotMax()
+raw.plot3d()
+```
+
+E' possibile plottare tutti i profili BGS e indicare uno specifico indice da mettere in primo piano:
+
+```python
+fig = raw.plotBGS(index=125)
+plotter.show(fig)
+```
+
+## Plotter Bokeh
+
+Oltre al plotter Plotly è disponibile anche una versione che utilizza il pacchetto `bokeh`. Non sono al momento disponibili i metodi `Bokeh.plot2d()` e `Bokeh.plot3d()`.
+
+```python
+from ReaderBOTDA.plotter.bokeh import Bokeh
+from bokeh.io import output_notebook
+output_notebook()
+plotter = Bokeh(theme='night_sky')
+```
+
+I temi a disposizione di default sono:
+>caliber, dark_minimal, light_minimal, night_sky, contrast
+
+A questo punto è possibile chiamare i vari metodi come già mostrato per plotter Plotly:
+
+```python
+misura = Profile('data/profiles/2021-11-08_16-51-16.652_rawarray.json',plotter=plotter)
+fig = misura.plot()
+plotter.show(fig)
+```

readerbotda-1.3.2/ReaderBOTDA/converter.py

@@ -0,0 +1,16 @@
+from ReaderBOTDA.settings import parseSettingsDict, Settings
+from ReaderBOTDA.reader import Profile
+import h5py
+from os import path
+from glob import glob
+
+def convert_to_h5(folder: str, putput_filename: str):
+
+    filelist = glob(path.join(folder,'*.json'))
+
+    if filelist:
+        with h5py.File("mytestfile.hdf5", "w") as f:
+            f.create_group("bfs")
+            for file in filelist:
+                temp = Profile(filename=file)
+                

readerbotda-1.3.2/ReaderBOTDA/plotter/abc.py

@@ -0,0 +1,35 @@
+from abc import ABC, abstractmethod
+
+class Plotter(ABC):
+
+    @abstractmethod
+    def single_plot(self,position, profile, title: str = None):
+        pass
+
+    @abstractmethod
+    def multiple_plot(self,position, profiles, timestamps, title: str = ''):
+        pass
+
+    @abstractmethod
+    def statistics(self,position,mean,std,title:str = ''):
+        pass
+
+    @abstractmethod
+    def raw2d_plot(self,position, frequency, BGS, title: str = None):
+        pass
+    
+    @abstractmethod
+    def rawBGS_plot(self,frequency, BGS, index: int = None, title: str = None):
+        pass
+
+    @abstractmethod
+    def raw3d_plot(self,position, frequency, BGS, title: str = None):
+        pass
+
+    @abstractmethod
+    def max_plot(self,position,max,title: str = None):
+        pass
+
+    @abstractmethod
+    def max_stat_plot(self,max_mean,max_std, title:str = ''):
+        pass

readerbotda-1.3.2/ReaderBOTDA/plotter/bokeh.py

@@ -0,0 +1,74 @@
+from .abc import Plotter
+
+from bokeh.plotting import figure
+from bokeh.models.layouts import Column
+from bokeh.plotting.figure import Figure as bokehFigure
+from bokeh.plotting import show as bokehShow
+from bokeh.models import Legend
+from bokeh.layouts import gridplot
+from bokeh.palettes import Bokeh8
+from bokeh.io import curdoc
+
+import numpy as np
+
+class Bokeh(Plotter):
+    def __init__(self,theme:str='caliber'):
+        curdoc().theme = theme
+        #TODO: capire come prendere la palette del tema scelto, invece che usare sempre Bokeh8. 
+        self.palette = Bokeh8
+    
+    def single_plot(self, position, profile, title: str = None) -> bokehFigure:
+        fig = figure(title=title, x_axis_label='Position (m)', y_axis_label='BFS (MHz)',sizing_mode='stretch_width')
+        fig.line(position, profile) 
+        return fig
+
+    def multiple_plot(self, position, profiles, timestamps, title: str = '') -> bokehFigure:
+        fig = figure(title=title, x_axis_label='Position (m)', y_axis_label='BFS (MHz)',sizing_mode='stretch_width')
+        fig.add_layout(Legend(), 'right')
+        for i,(time,color) in enumerate(zip(timestamps, self.palette)):
+            fig.line(x=position,y=profiles[:,i],
+                color=color,
+                legend_label=time.strftime("%m/%d/%Y, %H:%M:%S"))
+        return fig
+
+    def statistics(self, position, mean: np.ndarray,
+                   std: np.ndarray, title:str = '') -> Column:
+        f1 = figure(title=title,y_axis_label='Mean BFS (MHz)')
+        f1.line(x=position, y=mean)
+        f1.varea(x=position, y1=mean+std, y2=mean-std,fill_alpha=0.3)
+        f2 = figure(x_axis_label='Position (m)', y_axis_label='Std Dev (MHz)', y_range=(0,10))
+        f2.line(x=position, y=std)
+        return gridplot([f1,f2],ncols=1,height=150,sizing_mode='stretch_width')
+
+    def max_plot(self, position, max, title: str = None) -> bokehFigure:
+        fig = figure(title=title, x_axis_label='Position (m)', y_axis_label='Max BGS Amplitude',sizing_mode='stretch_width')
+        fig.line(x=position,y=max)
+        return fig
+    
+    def max_stat_plot(self,max_mean: np.ndarray, max_std: np.ndarray, title:str = '')-> bokehFigure:
+        fig = figure(title=title,y_axis_label='Max BGS (Volts)',sizing_mode='stretch_width')
+        fig.line(y=max_mean)
+        fig.varea(y1=max_mean+max_std, y2=max_mean-max_std,fill_alpha=0.3)
+        return fig
+    
+    def rawBGS_plot(self, frequency, BGS, index: int = None, title: str = None) -> bokehFigure:
+        fig = figure(title=title,x_axis_label='Frequency (GHz)', y_axis_label='Amplitude (V)',sizing_mode='stretch_width')
+        if not index:
+            for single in np.transpose(BGS):
+                fig.line(x=frequency,y=single)
+        else:
+            for single in np.transpose(BGS):
+                fig.line(x=frequency,y=single,color='rgba(175,175,175,0.15)')
+            fig.line(x=frequency,y=BGS[:,index])        
+        return fig
+
+    def raw3d_plot(self, position, frequency, BGS, title: str = None):
+        raise NameError('Function raw3d_plot not yet defined for Bokeh plotter.')
+
+    def raw2d_plot(self, position, frequency, BGS, title: str = None):
+        raise NameError('Function raw2d_plot not yet defined for Bokeh plotter.')
+    
+    @staticmethod
+    def show(fig) -> None:
+        bokehShow(fig)
+        pass

readerbotda-1.3.2/ReaderBOTDA/plotter/plotly.py

@@ -0,0 +1,255 @@
+from .abc import Plotter
+
+from typing import List
+
+import plotly.graph_objects as go
+import plotly.io as pio
+from plotly.subplots import make_subplots
+
+from datetime import datetime
+import numpy as np
+
+
+class Plotly(Plotter):
+
+    def __init__(self, theme="ggplot2", colorscale: str = "YlGnBu") -> None:
+        self.theme = theme
+        self.colorscale = colorscale
+        self.firstColor = pio.templates[self.theme].layout.colorway[0]
+        super().__init__()
+        pass
+
+    def _applyTheme(self, fig) -> go.Figure:
+        return fig.update_layout(
+            template=self.theme,
+            scene_xaxis=dict(separatethousands=False),
+            scene_yaxis=dict(separatethousands=False),
+        )
+
+    def single_plot(self, position, profile, title: str = None) -> go.Figure:
+        fig = go.Figure(
+            data=go.Scatter(x=position, y=profile, name=title),
+            layout=dict(
+                title=title,
+                xaxis=dict(title=dict(text="Position (m)")),
+                yaxis=dict(title=dict(text="BFS (MHz)")),
+            ),
+        )
+        return self._applyTheme(fig)
+
+    def multiple_plot(
+        self,
+        position: np.ndarray,
+        profiles: np.ndarray,
+        timestamps: List[datetime],
+        title: str = "",
+    ) -> go.Figure:
+        fig = go.Figure(
+            layout=dict(
+                title=title,
+                xaxis=dict(title=dict(text="Position (m)")),
+                yaxis=dict(title=dict(text="BFS (MHz)")),
+            )
+        )
+        for i, time in enumerate(timestamps):
+            fig.add_trace(
+                go.Scatter(
+                    x=position,
+                    y=profiles[:, i],
+                    mode="lines",
+                    name=time.strftime("%m/%d/%Y, %H:%M:%S"),
+                )
+            )
+        return self._applyTheme(fig)
+
+    def statistics(
+        self, position: np.ndarray, mean: np.ndarray, std: np.ndarray, title: str = ""
+    ) -> go.Figure:
+
+        fig = make_subplots(rows=2, cols=1, column_titles=[title])
+        fig.add_trace(
+            go.Scatter(
+                x=position,
+                y=mean,
+                line=dict(color=self.firstColor),
+                mode="lines",
+                showlegend=False,
+            ),
+            row=1,
+            col=1,
+        )
+        fig.add_trace(
+            go.Scatter(
+                name="Upper Bound",
+                x=position,
+                y=mean + std,
+                mode="lines",
+                marker=dict(color="#444"),
+                line=dict(width=0),
+                showlegend=False,
+            ),
+            row=1,
+            col=1,
+        )
+        fig.add_trace(
+            go.Scatter(
+                name="Lower Bound",
+                x=position,
+                y=mean - std,
+                marker=dict(color=self.firstColor),
+                line=dict(width=0, color=self.firstColor),
+                mode="lines",
+                # fillcolor='rgba(68, 68, 68, 0.3)',
+                fill="tonexty",
+                showlegend=False,
+            ),
+            row=1,
+            col=1,
+        )
+
+        fig.update_yaxes(title_text="BFS (MHz)", row=1, col=1)
+        fig.update_layout(hovermode="x")
+
+        fig.add_trace(
+            go.Scatter(
+                name="std",
+                x=position,
+                y=std,
+                line=dict(color=self.firstColor),
+                showlegend=False,
+            ),
+            row=2,
+            col=1,
+        )
+        fig.update_yaxes(title_text="Std Dev (MHz)", range=[0, 10], row=2, col=1)
+        fig.update_xaxes(title_text="Position (m)", matches="x", row=2, col=1)
+        return self._applyTheme(fig)
+
+    def max_stat_plot(
+        self, max_mean: np.ndarray, max_std: np.ndarray, title: str = ""
+    ) -> go.Figure:
+
+        fig = go.Figure(layout=dict(title=title))
+        fig.add_trace(
+            go.Scatter(
+                y=max_mean,
+                line=dict(color=self.firstColor),
+                mode="lines",
+                showlegend=False,
+            )
+        )
+        fig.add_trace(
+            go.Scatter(
+                name="Upper Bound",
+                y=max_mean + max_std,
+                mode="lines",
+                marker=dict(color="#444"),
+                line=dict(width=0),
+                showlegend=False,
+            )
+        )
+        fig.add_trace(
+            go.Scatter(
+                name="Lower Bound",
+                y=max_mean - max_std,
+                marker=dict(color=self.firstColor),
+                line=dict(width=0, color=self.firstColor),
+                mode="lines",
+                # fillcolor='rgba(68, 68, 68, 0.3)',
+                fill="tonexty",
+                showlegend=False,
+            )
+        )
+
+        fig.update_yaxes(title_text="Max BGS (Volts)")
+
+        return self._applyTheme(fig)
+
+    def raw2d_plot(self, position, frequency, BGS, title: str = None) -> go.Figure:
+        fig = go.Figure(
+            data=go.Contour(
+                z=BGS,
+                x=position,
+                y=frequency,
+                showscale=False,
+                colorscale=self.colorscale,
+                line_width=0,
+            ),
+            layout=dict(
+                title=title,
+                xaxis=dict(title=dict(text="Position (m)")),
+                yaxis=dict(title=dict(text="Frequency (GHz)")),
+            ),
+        )
+        return self._applyTheme(fig)
+
+    def raw3d_plot(self, position, frequency, BGS, title: str = None):
+        hovertemplate = "Pos: %{x:.2f} m, Freq: %{y:.3f} GHz, Ampl: %{z:.3f} V"
+        fig = go.Figure(
+            data=go.Surface(
+                z=BGS, x=position, y=frequency, hovertemplate=hovertemplate
+            ),
+            layout=dict(
+                title=title,
+                scene=dict(
+                    xaxis_title="Position (m)",
+                    yaxis_title="Frequency (GHz)",
+                    zaxis_title="Amplitude (V)",
+                ),
+            ),
+        )
+        return self._applyTheme(fig)
+
+    def rawBGS_plot(
+        self, frequency, BGS, index: int = None, title: str = None
+    ) -> go.Figure:
+        fig = go.Figure(
+            layout=dict(
+                title=title,
+                hovermode=False,
+                xaxis=dict(title=dict(text="Frequency (GHz)")),
+                yaxis=dict(title=dict(text="Amplitude (V)")),
+            )
+        )
+        if not index:
+            for single in np.transpose(BGS):
+                fig.add_trace(
+                    go.Scatter(x=frequency, y=single, mode="lines", showlegend=False)
+                )
+        else:
+            for single in np.transpose(BGS):
+                fig.add_trace(
+                    go.Scatter(
+                        x=frequency,
+                        y=single,
+                        line=dict(color="rgba(175,175,175,0.15)"),
+                        mode="lines",
+                        showlegend=False,
+                    )
+                )
+            fig.add_trace(
+                go.Scatter(
+                    x=frequency,
+                    y=BGS[:, index],
+                    line=dict(color=self.firstColor),
+                    mode="lines",
+                    showlegend=False,
+                )
+            )
+        return self._applyTheme(fig)
+
+    def max_plot(self, position, max, title: str = None) -> go.Figure:
+        fig = go.Figure(
+            data=go.Scatter(x=position, y=max, name=title),
+            layout=dict(
+                title=title,
+                xaxis=dict(title=dict(text="Position (m)")),
+                yaxis=dict(title=dict(text="Max BGS Amplitude")),
+            ),
+        )
+        return self._applyTheme(fig)
+
+    @staticmethod
+    def show(fig: go.Figure) -> None:
+        fig.show()
+        pass

readerbotda-1.3.2/ReaderBOTDA/reader.py

@@ -0,0 +1,261 @@
+from ReaderBOTDA.settings import parseSettingsDict, Settings
+from ReaderBOTDA.plotter.abc import Plotter
+from ReaderBOTDA.plotter.plotly import Plotly
+
+from dataclasses import dataclass
+from pathlib import Path
+from json import load
+import numpy as np
+from datetime import datetime # TODO usare numpy anche per questo?
+from os import path
+from glob import glob
+from typing import Tuple, Union, Literal
+import warnings
+from progress.bar import Bar
+
+class NoFilesSelected(Exception):
+    def __init__(self, folder, message='No file found in folder'):
+        self.message = f'{message}: {folder}'
+        super().__init__(self.message)
+
+
+
+#TODO da implementare nei metodi. E aggiungere anche i campi MaxMean e MaxStd
+@dataclass
+class Statistics:
+    BFSmean: np.array([])
+    BFSstd: np.array([])
+    BFSstd_mean: float
+
+class Profile:
+
+    filename: str = ''
+    plotter: Plotter
+    settings: Settings
+    sw_version: str = '<1.2.0.0'
+    BFS: np.ndarray = np.array([])
+    MaxGain: np.ndarray = np.array([])
+    timestamp: datetime = datetime.now
+    
+
+    def __init__(self, filename: Union[str,Path], plotter: Plotter = Plotly(), name: str = None) -> None:
+        self.filename = filename
+        self.plotter = plotter
+        with open(self.filename) as file:
+            text = load(file)
+        self.settings = parseSettingsDict(text['Settings'])
+        if 'sw_version' in text:
+            self.sw_version = text['sw_version']
+        if 'sw_version' in text and text['sw_version'] == '':
+            self.sw_version = 'ambiente sviluppo'
+        self.timestamp = datetime.strptime(
+            text['Time Stamp'], '%Y-%m-%d'+'T'+'%H:%M:%S.%f'+'Z')  # "2021-11-08T16:51:16.652Z"
+        if not name:
+            self.name = self.timestamp.strftime("%m/%d/%Y, %H:%M:%S")
+        else:
+            self.name = name
+
+        self.BFS = np.array(text['Profile'])
+        self._createPositionArray()
+        if 'arrayMaxGain' in text:
+            self.MaxGain = np.array(text['arrayMaxGain'])
+
+    def _createPositionArray(self):
+        self.position = np.linspace(
+            0, self.settings.Cable.Length, num=len(self.BFS), endpoint=True)
+        self.spatialResolution = self.position[1]
+
+    def plot(self,title:str = None):
+        if not title:
+            title = self.name
+        # TODO usare xrange espresso in metri per affettare array e poi passarli a single_plot. xrange:Tuple[float,float]=None
+        return self.plotter.single_plot(self.position,self.BFS,title=title)
+    
+    def plotMax(self,title: str = None):
+        if self.MaxGain.size == 0:
+            warnings.warn("Max array in not available. If possibile, load the raw file of the same measure.")
+            return None
+        if not title:
+            title = self.name
+        return self.plotter.max_plot(self.position, self.MaxGain, title)
+
+
+class multipleProfile():
+
+    statistics: Statistics
+    plotter: Plotter
+
+    def __init__(self, folder: Union[str,Path], plotter: Plotter = Plotly(),
+                 n_measure: int = None,
+                 start_measure: Union[int,datetime] = 0,
+                 stop_measure: datetime = None) -> None:
+        
+        self.folder = folder
+        self.plotter = plotter
+        
+        filelist = glob(path.join(folder,'*.json'))
+        if start_measure and isinstance(start_measure, datetime):
+            timestamps_files = [ datetime.strptime('_'.join(path.basename(file).split('_')[:2]), '%Y-%m-%d_%H-%M-%S.%f' ) for file in filelist]
+            primo = next((x for x,value in enumerate(timestamps_files) if value >= start_measure),0)
+            if stop_measure:
+                ultimo = next((x for x,value in enumerate(timestamps_files) if value > stop_measure),len(filelist))
+            elif n_measure:
+                ultimo = primo + n_measure
+            else:
+                ultimo = len(filelist)
+            filelist = filelist[primo:ultimo]
+
+        if n_measure and (start_measure==0 or isinstance(start_measure,int)):
+            filelist = filelist[start_measure:start_measure+n_measure]
+
+        if isinstance(start_measure,int) and not n_measure and stop_measure:
+            timestamps_files = [ datetime.strptime('_'.join(path.basename(file).split('_')[:2]), '%Y-%m-%d_%H-%M-%S.%f' ) for file in filelist]
+            ultimo = next((x for x,value in enumerate(timestamps_files) if value > stop_measure),len(filelist))
+            filelist = filelist[start_measure:ultimo]
+
+        if len(filelist) == 0:
+            raise NoFilesSelected(folder=folder)
+        
+            
+        timestamps = list()
+        with Bar('Redaing files',max=len(filelist)) as bar:
+            for file in filelist:
+                temp = Profile(filename=file)
+                timestamps.append(temp.timestamp)
+                try:
+                    self.BFS = np.column_stack((self.BFS, temp.BFS))
+                    self.MaxGain = np.column_stack((self.MaxGain, temp.MaxGain))
+                except AttributeError:
+                    self.BFS = temp.BFS
+                    self.MaxGain = temp.MaxGain
+                bar.next()
+
+        self.timestamps = np.array(timestamps)
+        self.settings = temp.settings
+        self.sw_version = temp.sw_version
+        self.position = temp.position
+        self.calcStatistics()
+        self.MaxGainMean = self.MaxGain.mean(axis=1)
+        self.MaxGainStd = self.MaxGain.std(axis=1)
+
+    def calcCorrelations(self, type:Literal['max','bfs'],
+                         reference:Literal['first','previous']='previous',
+                         range:Tuple[float,float]=None) -> np.array:
+        '''Ritorna correlazione tra prima misura e misura n-esima.
+        Si può scegliere se effettuarla su matrice dei BFS o matrice dei massimi'''
+        
+        if type == 'max':
+            correlations = np.corrcoef(self.MaxGain[range[0]:range[1]] if range else self.MaxGain,rowvar=False)
+        else:
+            correlations = np.corrcoef(self.BFS[range[0]:range[1]] if range else self.BFS,rowvar=False)
+
+        if reference == 'first':
+            return correlations[0,:]
+
+        indici = np.arange(1,np.shape(correlations)[0])
+        return np.insert(correlations[indici,indici-1],0,1)
+
+
+    def calcStatistics(self, plot:bool=False, range:Tuple[float,float]=None, title:str=None)->Statistics:
+        # TODO gestione input range
+        self.statistics = Statistics(BFSmean=self.BFS.mean(axis=1),
+                                     BFSstd=self.BFS.std(axis=1),
+                                     BFSstd_mean=self.BFS.std(axis=1).mean())
+
+        if plot:
+            if not title:
+                title=self.folder
+            return self.plotter.statistics(self.position,
+                                           self.statistics.BFSmean,
+                                           self.statistics.BFSstd, title=title)
+        return self.statistics
+
+    def plotStatistics(self, title:str=None):
+        if not title:
+            title=self.folder
+        # TODO se sono state calcolate con range allora position è sbagliato
+        return self.plotter.statistics(self.position, self.statistics, title=title)
+
+    def plot(self, startTime: datetime = datetime.min, stopTime: datetime = datetime.now(), title: str = None):
+        if not title:
+            title=self.folder
+        match = [i for i,date in enumerate(self.timestamps) if date >= startTime and date <= stopTime]
+        return self.plotter.multiple_plot(self.position,self.BFS[:,match],self.timestamps[match],title=title)
+    
+    def plotMax(self,title: str = None):
+        if self.MaxGain.size == 0:
+            warnings.warn("Max array in not available. If possibile, load the raw file of the same measure.")
+            return None
+        if not title:
+            title = self.folder
+        return self.plotter.max_stat_plot(self.MaxGainMean, self.MaxGainStd, title)
+
+
+class Raw():
+
+    filename: Union[str,Path] = Path('')
+    plotter: Plotter
+    settings: Settings
+    sw_version: str = '<1.2.0.0'
+    BGS: np.ndarray = np.array([])
+    residuo: np.ndarray = np.array([])
+    timestamp: datetime = datetime.now
+
+    def __init__(self, filename: Union[str,Path], plotter: Plotter = Plotly()) -> None:
+        self.filename = filename
+        self.plotter = plotter
+        with open(self.filename) as file:
+            text = load(file)
+        self.settings = parseSettingsDict(text['Settings'])
+        if 'sw_version' in text:
+            self.sw_version = text['sw_version']
+        if 'sw_version' in text and text['sw_version'] == '':
+            self.sw_version = 'ambiente sviluppo'
+        self.BGS = np.transpose(np.array(text['Raw']))
+        self.timestamp = datetime.strptime(
+            text['Time Stamp'], '%Y-%m-%d'+'T'+'%H:%M:%S.%f'+'Z')  # "2021-11-08T16:51:16.652Z"
+        self._createPositionArray()
+        self._createFrequencyArray()
+        try:
+            self.residuo = np.array(text['residuo'])
+        except KeyError:
+            self.residuo = np.zeros(np.shape(self.frequency))
+
+    def _createPositionArray(self) -> None:
+        '''Crea array numpy delle posizioni e la risoluzione spaziale'''
+        self.position = np.linspace(
+            0, self.settings.Cable.Length, num=self.BGS.shape[1], endpoint=True)
+        self.spatialResolution = self.position[1]
+
+    def _createFrequencyArray(self) -> None:
+        '''Crea array numpy delle frequenze, espresse in GHz'''     
+        self.frequency = np.linspace( self.settings.Clock.StartMHz,
+                                     self.settings.Clock.StartMHz + self.settings.Clock.StepMHz * self.BGS.shape[0], num= self.BGS.shape[0], endpoint=True) / 1000
+
+    def plot2d(self, title: str = None):
+        if not title:
+            title=self.filename
+        return self.plotter.raw2d_plot(self.position, self.frequency, self.BGS, title=title)
+
+    def plot3d(self, title: str = None):
+        if not title:
+            title=self.filename
+        return self.plotter.raw3d_plot(self.position, self.frequency, self.BGS, title=title)
+
+    def plotBGS(self, index: int = None, title: str = None):
+        '''Plot 2D di tutti gli spettri BGS'''
+        if not title:
+            title=self.filename
+        return self.plotter.rawBGS_plot(self.frequency, self.BGS, index=index, title=title)
+
+    def plotMax(self, title: str = None):
+        if not title:
+            title=self.filename
+        return self.plotter.max_plot(self.position, self.BGS.max(axis=0), title)
+
+if __name__ == '__main__':
+    '''Non va se si lancia direttamente questo script a meno di non cambiare i primi due import da modules.settings a settings; uguale per plotter.'''
+    #a = Profile(filename='data/profiles/2021-11-08_16-51-16.652_rawarray.json')
+    #b = multipleProfile(folder='data/profiles/')
+    c = Raw(filename='data/raw/2021-11-08_16-51-16.652_rawmatrix.json')
+    # print(b.BFS.shape)

readerbotda-1.3.2/ReaderBOTDA/settings.py

@@ -0,0 +1,126 @@
+from dataclasses import dataclass
+from json import load
+from typing import Dict, Union, Optional
+from dacite import from_dict
+from enum import Enum
+
+class EstimationMode(str, Enum):
+    WEIGHTED = 'Weighted Correlation'
+    CORRELATION = 'Correlation'
+    LORFIT = 'Lorentzian fitting'
+    POLYFIT = 'Polyfit'
+
+class DynamicRange(int, Enum):
+    '''Dynamic Range della picoscope espresso in mV o V.'''
+    mV10 = 0
+    mV20 = 1
+    mV50 = 2
+    mV100 = 3
+    mV200 = 4
+    mV500 = 5
+    V1 = 6
+    V2 = 7
+    V5 = 8
+    V10 = 9
+    V20 = 10
+
+@dataclass
+class Pulse:
+    Width_ns: float
+    RepRate_kHz: float
+    Amplitude_V: float
+    Offset_V: float
+    EDFAoutput: float
+
+    def __str__(self):
+       return f"Width: {self.Width_ns} ns,\tRep Rate: {self.RepRate_kHz} kHz,\tAmplitude: {self.Amplitude_V} V,\tOffset: {self.Offset_V} V,\tEDFA Pump: {self.EDFAoutput} mA."
+
+@dataclass
+class Clock:
+    Power: float
+    StartMHz: float
+    StopMHz: float
+    StepMHz: int
+
+    def __str__(self):
+        return f"Power: {self.Power} dBm,\tStart Freq: {self.StartMHz} MHz,\tStop Freq: {self.StopMHz} MHz,\tStep Freq: {self.StepMHz} MHz."
+
+@dataclass
+class Scope:
+    Offset: float
+    Average: int
+    DynamicRange: Union[int,DynamicRange]
+    TriggerDelay_m: float
+    TriggerLevel_V: float
+
+    def __post_init__(self):
+        self.DynamicRange = DynamicRange(self.DynamicRange)
+
+    def __str__(self):
+        return f"Averages: {self.Average},\tRange: {self.DynamicRange.name},\tOffset: {self.Offset} V,\tTrigger Delay: {self.TriggerDelay_m} m,\tTrigger Level: {self.TriggerLevel_V} V."
+
+@dataclass
+class Cable:
+    Length: float
+    RefractiveIndex: float
+    CoeffStrainGHz: float
+    CoeffTemperatureGHz: float
+    ReferenceBFS_GHz: float
+
+    def __str__(self):
+        return f"Length: {self.Length} m,\tRefractive Index: {self.RefractiveIndex},\tStrain Coeff: {self.CoeffStrainGHz} GHz/microstrain,\tTemperature Coeff: {self.CoeffTemperatureGHz} GHz/°C,\tReference BFS: {self.ReferenceBFS_GHz} GHz."
+
+@dataclass
+class Brillouin:
+    EstimationMode: Union[str,int,EstimationMode]
+    SplineRes: float
+    Subset: float
+    RemoveBackground: bool
+    RemovePulseGhost: bool
+    CalibrationLength: float
+    peak_2nd_thr: Optional[float]
+    polyfit_lorFWHM: Optional[float]
+    polyfit_prefilter_bw: Optional[float]
+    polyfit_prefilter_taps: Optional[int]
+
+    def __post_init__(self):
+        if isinstance(self.EstimationMode, EstimationMode):
+            pass
+        self.EstimationMode = EstimationMode(self.EstimationMode)
+
+    def __str__(self):
+        if self.EstimationMode == EstimationMode.POLYFIT:
+            additional_settings = f"Lorentzian FWHM: {self.polyfit_lorFWHM} MHz,\tPrefilter BW: {self.polyfit_prefilter_bw},\tPrefilter taps: {self.polyfit_prefilter_taps}."
+        elif self.EstimationMode == EstimationMode.WEIGHTED:
+            additional_settings = f"Spline Resolution: {self.SplineRes} MHz,\tSubset: {self.Subset} MHz,\t2nd peak thr: {self.peak_2nd_thr}."
+        else:
+            additional_settings = f"Spline Resolution: {self.SplineRes} MHz,\tSubset: {self.Subset} MHz,\t2nd peak thr: {self.peak_2nd_thr}."
+        return f"Mode: {self.EstimationMode},\t{additional_settings}"
+
+@dataclass
+class Settings:
+    Pulse: Pulse
+    Clock: Clock
+    Scope: Scope
+    Cable: Cable
+    Brillouin: Brillouin
+
+    def __str__(self):
+        return f"PULSE\t- {self.Pulse}\nCLOCK\t- {self.Clock}\nSCOPE\t- {self.Scope}\nCABLE\t- {self.Cable}\nBRILL\t- {self.Brillouin}"
+
+def parseSettingsDict(dict: Dict) -> Settings:
+    dict['Brillouin']['EstimationMode'] = dict['Brillouin'].pop('Estimation mode')
+    dict['Brillouin']['CalibrationLength'] = dict['Brillouin'].pop('CalibrationLength(S)')
+    if '2nd_peak_thr' in dict['Brillouin']:
+        dict['Brillouin']['peak_2nd_thr'] = dict['Brillouin'].pop('2nd_peak_thr')
+    return from_dict(data_class=Settings, data=dict)
+
+if __name__ == '__main__':
+    #filename='data/2021/profiles/2021-11-08_16-51-16.652_rawarray.json'
+    filename='data/2023_09/rawarray/2023-09-26_10-33-20.210_rawarray.json'
+    #filename='C:/Users/marbr/OneDrive/Cohaerentia/00 - Sensori/Brillouin/BOTDA/Misure/2023_12 - polyfit/gialla_15MHz/rawarray/2023-12-11_10-30-23.093_rawarray.json'
+    with open(filename) as file:
+        text = load(file)
+    setting_test = text['Settings']
+    settings = parseSettingsDict(setting_test)
+    print(settings)

readerbotda-1.3.2/pyproject.toml

@@ -0,0 +1,24 @@
+[project]
+name = "readerbotda"
+version = "1.3.2"
+description = "Pacchetto per lettura e visualizzazione file di log sensore BOTDA Cohaerentia"
+authors = [
+    {name = "Marco Brunero",email = "marco.brunero@cohaerentia.com"}
+]
+readme = "README.md"
+requires-python = ">3.10"
+dependencies = [
+    "dacite (>=1.9.2,<2.0.0)",
+    "plotly (>=6.0.1,<7.0.0)",
+    "numpy (>=2.2.4,<3.0.0)",
+    "bokeh (>=3.7.2,<4.0.0)",
+    "typer (>=0.15.2,<0.16.0)",
+    "progress (>=1.6,<2.0)"
+]
+
+[tool.poetry.scripts]
+botda-converter = "readBOTDA:app"
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"