h5yaml 0.2.1__tar.gz → 0.3.0__tar.gz

{h5yaml-0.2.1 → h5yaml-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: h5yaml
-Version: 0.2.1
+Version: 0.3.0
 Summary: Use YAML configuration file to generate HDF5/netCDF4 formated files.
 Project-URL: Homepage, https://github.com/rmvanhees/h5_yaml
 Project-URL: Source, https://github.com/rmvanhees/h5_yaml
@@ -79,17 +79,21 @@ And reinstall `h5py` and `netCDF4` using the commands:
 
 The YAML file should be structured as follows:
 
- * The top level are: 'groups', 'dimensions', 'compounds' and 'variables'
+ * The top level are: 'groups', 'dimensions', 'compounds', 'variables', 'attrs\_global' and 'attrs\_groups'.
+ * > 'attrs\_global' and 'attrs\_groups' are added in version 0.3.0
+ * The names of the attributes, groups, dimensions, compounds and viariable should be specified as PosixPaths, however:
+   * The names of groups should never start with a slash (always erlative to root);
+   * All other elements which are stored in root should also not start with a slash;
+   * But these elements require a starting slash (absolute paths) when they are stored not the root. 
  * The section 'groups' are optional, but you should provide each group you want to use
    in your file. The 'groups' section in the YAML file may look like this:
-
    ```
    groups:
      - engineering_data
      - image_attributes
      - navigation_data
-     - processing_control
      - science_data
+     - processing_control/input_data
    ```
 
  * The section 'dimensions' is obligatory, you should define the dimensions for each
@@ -145,14 +149,6 @@ The YAML file should be structured as follows:
        dark_offs: [f4, '1', dark-offset]
    ```
 
-   Alternatively, provide a list with names of YAML files which contain the definitions
-   of the compounds.
-
-   ```
-   compounds:
-     - h5_nomhk_tm.yaml
-     - h5_science_hk.yaml
-   ```
  * The 'variables' are defined by their data-type ('_dtype') and dimensions ('_dims'),
    and optionally chunk sizes ('_chunks'), compression ('_compression'), variable length
    ('_vlen'). In addition, each variable can have as many attributes as you like,
@@ -160,6 +156,16 @@ The YAML file should be structured as follows:
 
    ```
    variables:
+     /science_data/detector_images:
+       _dtype: u2
+       _dims: [number_of_images, samples_per_image]
+	   _compression: 3
+       _FillValue: 65535
+       long_name: Detector pixel values
+       coverage_content_type: image
+       units: '1'
+       valid_min: 0
+       valid_max: 65534
      /image_attributes/nr_coadditions:
        _dtype: u2
        _dims: [number_of_images]
@@ -175,16 +181,14 @@ The YAML file should be structured as follows:
        units: seconds
      stats_163:
        _dtype: stats_dtype
-       _vlen: True
        _dims: [days]
+       _vlen: True
        comment: detector map statistics (MPS=163)
    ```
 
-### Notes and ToDo:
+### Notes and ToDo
 
- * The usage of older versions of h5py may result in broken netCDF4 files
- * Explain usage of parameter '_chunks', which is currently not correctly implemented.
- * Explain that the usage of variable length data-sets may break netCDF4 compatibility
+* The layout of a HDF5 or netCDF4 file can be complex. From version 0.3.0, you can split the file definition over several YAML files and provide a list with the names of YAML files as input to H5Yaml and NcYaml.  
 
 ## Support [TBW]
 

{h5yaml-0.2.1 → h5yaml-0.3.0}/README.md

@@ -47,17 +47,21 @@ And reinstall `h5py` and `netCDF4` using the commands:
 
 The YAML file should be structured as follows:
 
- * The top level are: 'groups', 'dimensions', 'compounds' and 'variables'
+ * The top level are: 'groups', 'dimensions', 'compounds', 'variables', 'attrs\_global' and 'attrs\_groups'.
+ * > 'attrs\_global' and 'attrs\_groups' are added in version 0.3.0
+ * The names of the attributes, groups, dimensions, compounds and viariable should be specified as PosixPaths, however:
+   * The names of groups should never start with a slash (always erlative to root);
+   * All other elements which are stored in root should also not start with a slash;
+   * But these elements require a starting slash (absolute paths) when they are stored not the root. 
  * The section 'groups' are optional, but you should provide each group you want to use
    in your file. The 'groups' section in the YAML file may look like this:
-
    ```
    groups:
      - engineering_data
      - image_attributes
      - navigation_data
-     - processing_control
      - science_data
+     - processing_control/input_data
    ```
 
  * The section 'dimensions' is obligatory, you should define the dimensions for each
@@ -113,14 +117,6 @@ The YAML file should be structured as follows:
        dark_offs: [f4, '1', dark-offset]
    ```
 
-   Alternatively, provide a list with names of YAML files which contain the definitions
-   of the compounds.
-
-   ```
-   compounds:
-     - h5_nomhk_tm.yaml
-     - h5_science_hk.yaml
-   ```
  * The 'variables' are defined by their data-type ('_dtype') and dimensions ('_dims'),
    and optionally chunk sizes ('_chunks'), compression ('_compression'), variable length
    ('_vlen'). In addition, each variable can have as many attributes as you like,
@@ -128,6 +124,16 @@ The YAML file should be structured as follows:
 
    ```
    variables:
+     /science_data/detector_images:
+       _dtype: u2
+       _dims: [number_of_images, samples_per_image]
+	   _compression: 3
+       _FillValue: 65535
+       long_name: Detector pixel values
+       coverage_content_type: image
+       units: '1'
+       valid_min: 0
+       valid_max: 65534
      /image_attributes/nr_coadditions:
        _dtype: u2
        _dims: [number_of_images]
@@ -143,16 +149,14 @@ The YAML file should be structured as follows:
        units: seconds
      stats_163:
        _dtype: stats_dtype
-       _vlen: True
        _dims: [days]
+       _vlen: True
        comment: detector map statistics (MPS=163)
    ```
 
-### Notes and ToDo:
+### Notes and ToDo
 
- * The usage of older versions of h5py may result in broken netCDF4 files
- * Explain usage of parameter '_chunks', which is currently not correctly implemented.
- * Explain that the usage of variable length data-sets may break netCDF4 compatibility
+* The layout of a HDF5 or netCDF4 file can be complex. From version 0.3.0, you can split the file definition over several YAML files and provide a list with the names of YAML files as input to H5Yaml and NcYaml.  
 
 ## Support [TBW]