evefile 0.1.0rc2__tar.gz → 0.2.0__tar.gz

{evefile-0.1.0rc2/evefile.egg-info → evefile-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: evefile
-Version: 0.1.0rc2
+Version: 0.2.0
 Summary: Transitional package to read eveH5 files containing synchrotron radiometry data recorded at BESSY/MLS in Berlin
 Home-page: https://www.ptb.de/cms/en/ptb/fachabteilungen/abt7/ptb-sr.html
 Author: Till Biskup
@@ -64,18 +64,32 @@ evefile
 
 *Transitional package to read eveH5 files containing synchrotron radiometry data recorded at BESSY/MLS in Berlin.*
 
-Welcome! This is evefile, a Python package for **importing (synchrotron) radiometry data** obtained at one of the beamlines at **BESSY-II or MLS in Berlin**, mostly operated by the German National Metrology Institute, the `Physikalisch-Technische Bundesanstalt (PTB) <https://www.ptb.de/>`_. This package acts as transitional interface between the (eveH5) data files and the processing and analysis code. For related packages for importing, viewing, and analysing those data, have a look at the "related projects" section below.
+Welcome! This is evefile, a Python package for **importing (synchrotron) radiometry data** obtained at one of the beamlines at **BESSY-II or MLS in Berlin**, mostly operated by the German National Metrology Institute, the `Physikalisch-Technische Bundesanstalt (PTB) <https://www.ptb.de/>`_. This package acts as *transitional* interface between the (eveH5) data files and the processing and analysis code. For related packages for importing, viewing, and analysing those data, have a look at the "related projects" section below.
+
+Loading the contents of a data file of a measurement is as simple as::
+
+    import evefile
+
+    file = evefile.EveFile(filename="my_measurement_file.h5")
+
+Here, ``file`` contains all the information contained in the data file as a hierarchy of Python objects.
 
 
 Features
 ========
 
-A list of (planned) features:
+A list of features:
 
 * Importer for eve HDF5 files (used at PTB in Berlin, Germany)
 
 * Fully backwards-compatible to older eveH5 versions
 
+* Complete information available that is contained in an eveH5 file
+
+* Data are (only) loaded on demand, not when loading the file
+
+* Powerful and intuitive abstractions, allowing for associative access to data and information – beyond a purely tabular view of the data
+
 
 And to make it even more convenient for users and future-proof:
 

{evefile-0.1.0rc2 → evefile-0.2.0}/README.rst

@@ -9,18 +9,32 @@ evefile
 
 *Transitional package to read eveH5 files containing synchrotron radiometry data recorded at BESSY/MLS in Berlin.*
 
-Welcome! This is evefile, a Python package for **importing (synchrotron) radiometry data** obtained at one of the beamlines at **BESSY-II or MLS in Berlin**, mostly operated by the German National Metrology Institute, the `Physikalisch-Technische Bundesanstalt (PTB) <https://www.ptb.de/>`_. This package acts as transitional interface between the (eveH5) data files and the processing and analysis code. For related packages for importing, viewing, and analysing those data, have a look at the "related projects" section below.
+Welcome! This is evefile, a Python package for **importing (synchrotron) radiometry data** obtained at one of the beamlines at **BESSY-II or MLS in Berlin**, mostly operated by the German National Metrology Institute, the `Physikalisch-Technische Bundesanstalt (PTB) <https://www.ptb.de/>`_. This package acts as *transitional* interface between the (eveH5) data files and the processing and analysis code. For related packages for importing, viewing, and analysing those data, have a look at the "related projects" section below.
+
+Loading the contents of a data file of a measurement is as simple as::
+
+    import evefile
+
+    file = evefile.EveFile(filename="my_measurement_file.h5")
+
+Here, ``file`` contains all the information contained in the data file as a hierarchy of Python objects.
 
 
 Features
 ========
 
-A list of (planned) features:
+A list of features:
 
 * Importer for eve HDF5 files (used at PTB in Berlin, Germany)
 
 * Fully backwards-compatible to older eveH5 versions
 
+* Complete information available that is contained in an eveH5 file
+
+* Data are (only) loaded on demand, not when loading the file
+
+* Powerful and intuitive abstractions, allowing for associative access to data and information – beyond a purely tabular view of the data
+
 
 And to make it even more convenient for users and future-proof:
 

evefile-0.2.0/VERSION

@@ -0,0 +1 @@
+0.2.0

{evefile-0.1.0rc2 → evefile-0.2.0}/evefile/boundaries/evefile.py

@@ -146,9 +146,10 @@ import os
 
 import pandas as pd
 
+import evefile.entities.data
 from evefile.entities.file import File
 from evefile.boundaries.eveh5 import HDF5File
-from evefile.controllers import version_mapping, joining
+from evefile.controllers import version_mapping, joining, timestamp_mapping
 
 
 logger = logging.getLogger(__name__)
@@ -221,6 +222,11 @@ class EveFile(File):
         The keys of the dictionary are the (guaranteed to be unique) HDF
         dataset names, not the "given" names usually familiar to the users.
 
+        **Please note:** For monitors, in contrast to data stored in the
+        :attr:`data` attribute, names are *not unique*. Hence, the only way to
+        address an individual monitor unequivocally is by its ID that is
+        identical to the HDF dataset name.
+
         Each item is an instance of
         :class:`evefile.entities.data.MonitorData`.
 
@@ -261,7 +267,8 @@ class EveFile(File):
     def __init__(self, filename="", load=True):
         super().__init__()
         self.filename = filename
-        self._join_factory = joining.JoinFactory(evefile=self)
+        self._join_factory = joining.JoinFactory(file=self)
+        self._monitor_mapper = timestamp_mapping.Mapper(file=self)
         if load:
             if not filename:
                 raise ValueError("No filename given")
@@ -387,14 +394,24 @@ class EveFile(File):
         if self.metadata.preferred_axis:
             output[0] = self.data[self.metadata.preferred_axis]
         if self.metadata.preferred_channel:
-            output[1] = self.data[self.metadata.preferred_channel]
+            if self.metadata.preferred_channel in self.data:
+                output[1] = self.data[self.metadata.preferred_channel]
+            else:
+                logger.warning(
+                    "Dataset {} not found, possibly an "
+                    "attribute or option?".format(
+                        self.metadata.preferred_channel
+                    )
+                )
         if self.metadata.preferred_normalisation_channel:
             output[2] = self.data[
                 self.metadata.preferred_normalisation_channel
             ]
         return output
 
-    def get_joined_data(self, data=None, mode="AxisOrChannelPositions"):
+    def get_joined_data(
+        self, data=None, mode="AxisOrChannelPositions", include_monitors=False
+    ):
         """
         Retrieve data objects with commensurate dimensions.
 
@@ -420,6 +437,15 @@ class EveFile(File):
 
             Default: "AxisOrChannelPositions"
 
+        include_monitors : :class:`bool`
+            Whether to include (all) monitors in joined data.
+
+            If set to :obj:`True`, all monitors available will automatically
+            be mapped to :obj:`DeviceData <evefile.entities.data.DeviceData>`
+            objects and included in the list of joined data.
+
+            Default: :obj:`False`
+
         Returns
         -------
         data : :class:`list`
@@ -431,10 +457,37 @@ class EveFile(File):
         """
         if not data:
             data = list(self.data.values())
+        data = [
+            (
+                self._convert_str_to_data_object(item)
+                if isinstance(item, str)
+                else item
+            )
+            for item in data
+        ]
+        if include_monitors:
+            monitors = self.get_monitors()
+            if isinstance(monitors, list):
+                data.extend(monitors)
+            else:
+                data.append(monitors)
         joiner = self._join_factory.get_join(mode=mode)
         return joiner.join(data)
 
-    def get_dataframe(self, data=None, mode="AxisOrChannelPositions"):
+    def _convert_str_to_data_object(self, name_or_id=""):
+        try:
+            result = self.data[name_or_id]
+        except KeyError:
+            try:
+                result = self.get_data(name_or_id)
+            except KeyError:
+                # Valid situation: monitor
+                result = self.get_monitors(name_or_id)
+        return result
+
+    def get_dataframe(
+        self, data=None, mode="AxisOrChannelPositions", include_monitors=False
+    ):
         """
         Retrieve Pandas DataFrame with given data objects as columns.
 
@@ -445,6 +498,18 @@ class EveFile(File):
         The names of the columns of the returned DataFrame are the names (not
         IDs) of the respective datasets.
 
+        .. note::
+
+            At least for the time being, for each data object involved only
+            the ``data`` attribute will be contained in the returned
+            DataFrame as a column. This is of particular importance for more
+            complicated data types, such as :class:`NormalizedChannelData
+            <evefile.entities.data.NormalizedChannelData>`,
+            :class:`AverageChannelData
+            <evefile.entities.data.AverageChannelData>`,
+            and :class:`IntervalChannelData
+            <evefile.entities.data.IntervalChannelData>`.
+
         .. important::
 
             While working with a Pandas DataFrame may seem convenient,
@@ -457,12 +522,12 @@ class EveFile(File):
         Parameters
         ----------
         data : :class:`list`
-            (Names/IDs of) data objects whose data should be joined.
+            (Names/IDs of) data objects whose data should be included.
 
             You can provide either names or IDs or the actual data objects.
 
             If no data are given, by default all data available will be
-            joined.
+            included.
 
             Default: :obj:`None`
 
@@ -473,6 +538,19 @@ class EveFile(File):
 
             Default: "AxisOrChannelPositions"
 
+        include_monitors : :class:`bool`
+            Whether to include (all) monitors in the dataframe.
+
+            If set to :obj:`True`, all monitors available will automatically
+            be mapped to :obj:`DeviceData <evefile.entities.data.DeviceData>`
+            objects and included in the dataframe.
+
+            Note that for mapped monitors, their IDs are used as column
+            names rather than the "given" names, as for monitors, names are
+            *not unique*.
+
+            Default: :obj:`False`
+
         Returns
         -------
         dataframe : :class:`pandas.DataFrame`
@@ -484,10 +562,17 @@ class EveFile(File):
         """
         if not data:
             data = list(self.data.values())
-        joined_data = self.get_joined_data(data=data, mode=mode)
+        joined_data = self.get_joined_data(
+            data=data, mode=mode, include_monitors=include_monitors
+        )
+        for item in joined_data:
+            # Ensure IDs are used for monitors, as names are not unique
+            if isinstance(item, evefile.entities.data.DeviceData):
+                item.metadata.name = item.metadata.id
         dataframe = pd.DataFrame(
             {item.metadata.name: item.data for item in joined_data}
         )
+        dataframe.index = joined_data[0].position_counts
         dataframe.index.name = "position"
         return dataframe
 
@@ -552,3 +637,93 @@ class EveFile(File):
         print("\nMONITORS")
         for item in self.monitors.values():
             print(item)
+
+    def get_monitors(self, monitors=None):
+        """
+        Retrieve monitors as mapped device data objects by ID.
+
+        While you can get the original monitor data objects by accessing the
+        :attr:`monitors <evefile.entities.file.File.monitors>` attribute
+        directly, there, they are stored as
+        :obj:`MonitorData <evefile.entities.data.MonitorData>` objects with
+        timestamps instead of position counts. To relate monitors to the
+        measured data, the former need to be mapped to position counts.
+
+        .. note::
+
+            Monitors, in contrast to data objects stored in the
+            :attr:`data <evefile.entities.file.File.data>` attribute,
+            have *no unique names*. Hence, the only way to unequivocally
+            access a given monitor (or the respective, mapped
+            :obj:`DeviceData <evefile.entities.data.DeviceData>` object) is
+            by means of its unique ID.
+
+
+        Parameters
+        ----------
+        monitors : :class:`str` | :class:`list`
+            Name or list of names of monitors to retrieve
+
+        Returns
+        -------
+        device_data : :class:`evefile.entities.data.Data` | :class:`list`
+            Device data object(s) corresponding to the ID(s).
+
+            In case of a list of data objects, each object is of type
+            :class:`evefile.entities.data.DeviceData`.
+
+        """
+        device_data = []
+        if not monitors:
+            monitors = list(self.monitors)
+        if isinstance(monitors, (list, tuple)):
+            for item in monitors:
+                device_data.append(self._monitor_mapper.map(item))
+        else:
+            device_data.append(self._monitor_mapper.map(monitors))
+        if len(device_data) == 1:
+            device_data = device_data[0]
+        return device_data
+
+    def get_snapshots(self):
+        """
+        Retrieve Pandas DataFrame with snapshots as rows.
+
+        Snapshots serve generally two functions:
+
+        #. Provide base values for axes.
+
+           In case of joining data using :meth:`get_joined_data`, for axes,
+           typically the previous values are used for positions no axes
+           values have been recorded. Snapshots are used if available.
+
+        #. Provide telemetry data for the setup the data were recorded with.
+
+           Snapshots regularly contain many more parameters than motor axes
+           used and detector channels recorded. Generally, this provides a
+           lot of telemetry data regarding the setup used for recording the
+           data.
+
+        The first function is served by the :meth:`get_joined_data` method
+        automatically. The second function can be served by having a look at
+        a summary containing all snapshot data. This is the aim of this
+        method: returning a Pandas DataFrame containing all snapshots as
+        rows and the position counts as columns.
+
+
+        Returns
+        -------
+        snapshot_dataframe : :class:`pandas.DataFrame`
+            Pandas DataFrame containing all snapshots as rows.
+
+            The indices (names of the rows) are the names (not IDs) of the
+            respective snapshot datasets.
+
+        """
+        snapshot_dataframes = []
+        for snapshot in self.snapshots.values():
+            data = dict(zip(snapshot.position_counts, snapshot.data))
+            snapshot_dataframes.append(
+                pd.DataFrame(data=data, index=[snapshot.metadata.name])
+            )
+        return pd.concat(snapshot_dataframes)