pynxtools 0.3.2__tar.gz → 0.4.0__tar.gz

{pynxtools-0.3.2 → pynxtools-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pynxtools
-Version: 0.3.2
+Version: 0.4.0
 Summary: Extend NeXus for experiments and characterization in Materials Science and Materials Engineering and serve as a NOMAD parser implementation for NeXus.
 Author: The NOMAD Authors
 License: 
@@ -223,7 +223,7 @@ Requires-Dist: click_default_group
 Requires-Dist: h5py>=3.6.0
 Requires-Dist: xarray>=0.20.2
 Requires-Dist: PyYAML>=6.0
-Requires-Dist: numpy>=1.21.2
+Requires-Dist: numpy<2.0.0,>=1.21.2
 Requires-Dist: pandas>=1.3.2
 Requires-Dist: ase>=3.19.0
 Requires-Dist: mergedeep
@@ -235,6 +235,7 @@ Requires-Dist: mkdocs; extra == "docs"
 Requires-Dist: mkdocs-material; extra == "docs"
 Requires-Dist: mkdocs-material-extensions; extra == "docs"
 Requires-Dist: mkdocs-macros-plugin; extra == "docs"
+Requires-Dist: mkdocs-click; extra == "docs"
 Provides-Extra: dev
 Requires-Dist: mypy; extra == "dev"
 Requires-Dist: ruff==0.3.4; extra == "dev"
@@ -245,7 +246,7 @@ Requires-Dist: structlog; extra == "dev"
 Requires-Dist: types-pyyaml; extra == "dev"
 Requires-Dist: types-pytz; extra == "dev"
 Requires-Dist: types-requests; extra == "dev"
-Requires-Dist: pip-tools; extra == "dev"
+Requires-Dist: uv; extra == "dev"
 Requires-Dist: pre-commit; extra == "dev"
 Provides-Extra: convert
 Requires-Dist: pynxtools[apm,ellips,em,mpes,stm,xps,xrd]; extra == "convert"
@@ -298,12 +299,12 @@ pip install git+https://github.com/FAIRmat-NFDI/pynxtools.git
 
 `pynxtools` (previously called `nexusutils`) is intended as a parser for combining various instrument output formats and electronic lab notebook (ELN) formats to an hdf5 file according to NeXus application definitions.
 
-Additionally, the software is used in the research data management system NOMAD for
+Additionally, the software can be used as a plugin in the research data management system NOMAD for
 making experimental data searchable and publishable.
 NOMAD is developed by the FAIRMAT consortium, as a part of the German National Research Data Infrastructure
 (NFDI).
 
-The software tools are located inside [`pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/pynxtools) and they are
+The software tools are located inside [`pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/src/pynxtools) and they are
 shipped with unit tests located in [`tests`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/tests).
 Some examples with real datasets are provided in [`examples`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/examples).
 It guides you through the process of converting instrument raw
@@ -311,9 +312,16 @@ data into the NeXus standard and visualising the files content.
 
 # Command line tools
 
-- [**dataconverter**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/pynxtools/dataconverter/README.md): Creates compliant instances of NeXus/HDF5 files to [NeXus schemas](https://nexusformat.org).
-- [**read_nexus**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/pynxtools/nexus/README.md): Outputs a debug log for a given NeXus file.
-- [**generate_eln**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/pynxtools/eln_mapper/README.md): Outputs ELN files that can be used to add metadata to the dataconverter routine.
+- [**dataconverter**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/dataconverter/README.md): Creates compliant instances of NeXus/HDF5 files to [NeXus schemas](https://nexusformat.org).
+- [**read_nexus**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/nexus/README.md): Outputs a debug log for a given NeXus file.
+- [**generate_eln**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/eln_mapper/README.md): Outputs ELN files that can be used to add metadata to the dataconverter routine.
+
+# NOMAD integration
+
+To use pynxtools with NOMAD, simply install it in the same environment as the `nomad-lab` package.
+NOMAD will recognize pynxtools as a plugin automatically and offer automatic parsing of `.nxs` files
+and a schema for NeXus application definitions.
+pynxtools is already included in the NOMAD main deployment and NOMAD NeXus distribution images.
 
 # Documentation
 Documentation for the different tools can be found [here](https://fairmat-nfdi.github.io/pynxtools/).
@@ -377,7 +385,7 @@ on how to build on this work, or to get your parser included into NOMAD, you can
 
 ### Does this software require NOMAD or NOMAD OASIS ?
 
-No. The data files produced here can be uploaded to Nomad. Therefore, this acts like the framework to design schemas and instances of data within the NeXus universe.
+No. The data files produced here can be uploaded to Nomad. Therefore, this acts like the framework to design schemas and instances of data within the NeXus universe. It can, however, be used as a NOMAD plugin to parse nexus files, please see the section above for details.
 
 # Troubleshooting
 

{pynxtools-0.3.2 → pynxtools-0.4.0}/README.md

@@ -32,12 +32,12 @@ pip install git+https://github.com/FAIRmat-NFDI/pynxtools.git
 
 `pynxtools` (previously called `nexusutils`) is intended as a parser for combining various instrument output formats and electronic lab notebook (ELN) formats to an hdf5 file according to NeXus application definitions.
 
-Additionally, the software is used in the research data management system NOMAD for
+Additionally, the software can be used as a plugin in the research data management system NOMAD for
 making experimental data searchable and publishable.
 NOMAD is developed by the FAIRMAT consortium, as a part of the German National Research Data Infrastructure
 (NFDI).
 
-The software tools are located inside [`pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/pynxtools) and they are
+The software tools are located inside [`pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/src/pynxtools) and they are
 shipped with unit tests located in [`tests`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/tests).
 Some examples with real datasets are provided in [`examples`](https://github.com/FAIRmat-NFDI/pynxtools/tree/master/examples).
 It guides you through the process of converting instrument raw
@@ -45,9 +45,16 @@ data into the NeXus standard and visualising the files content.
 
 # Command line tools
 
-- [**dataconverter**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/pynxtools/dataconverter/README.md): Creates compliant instances of NeXus/HDF5 files to [NeXus schemas](https://nexusformat.org).
-- [**read_nexus**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/pynxtools/nexus/README.md): Outputs a debug log for a given NeXus file.
-- [**generate_eln**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/pynxtools/eln_mapper/README.md): Outputs ELN files that can be used to add metadata to the dataconverter routine.
+- [**dataconverter**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/dataconverter/README.md): Creates compliant instances of NeXus/HDF5 files to [NeXus schemas](https://nexusformat.org).
+- [**read_nexus**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/nexus/README.md): Outputs a debug log for a given NeXus file.
+- [**generate_eln**](https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/eln_mapper/README.md): Outputs ELN files that can be used to add metadata to the dataconverter routine.
+
+# NOMAD integration
+
+To use pynxtools with NOMAD, simply install it in the same environment as the `nomad-lab` package.
+NOMAD will recognize pynxtools as a plugin automatically and offer automatic parsing of `.nxs` files
+and a schema for NeXus application definitions.
+pynxtools is already included in the NOMAD main deployment and NOMAD NeXus distribution images.
 
 # Documentation
 Documentation for the different tools can be found [here](https://fairmat-nfdi.github.io/pynxtools/).
@@ -111,7 +118,7 @@ on how to build on this work, or to get your parser included into NOMAD, you can
 
 ### Does this software require NOMAD or NOMAD OASIS ?
 
-No. The data files produced here can be uploaded to Nomad. Therefore, this acts like the framework to design schemas and instances of data within the NeXus universe.
+No. The data files produced here can be uploaded to Nomad. Therefore, this acts like the framework to design schemas and instances of data within the NeXus universe. It can, however, be used as a NOMAD plugin to parse nexus files, please see the section above for details.
 
 # Troubleshooting
 

{pynxtools-0.3.2 → pynxtools-0.4.0}/dev-requirements.txt

@@ -1,17 +1,11 @@
-#
-# This file is autogenerated by pip-compile with Python 3.10
-# by the following command:
-#
-#    pip-compile --extra=dev --extra=docs --output-file=dev-requirements.txt pyproject.toml
-#
+# This file was autogenerated by uv via the following command:
+#    uv pip compile --extra=dev --extra=docs --output-file=dev-requirements.txt pyproject.toml
 anytree==2.12.1
     # via pynxtools (pyproject.toml)
 ase==3.22.1
     # via pynxtools (pyproject.toml)
 babel==2.14.0
     # via mkdocs-material
-build==1.1.1
-    # via pip-tools
 certifi==2024.2.2
     # via requests
 cfgv==3.4.0
@@ -20,17 +14,17 @@ charset-normalizer==3.3.2
     # via requests
 click==8.1.7
     # via
+    #   pynxtools (pyproject.toml)
     #   click-default-group
     #   mkdocs
-    #   pip-tools
-    #   pynxtools (pyproject.toml)
+    #   mkdocs-click
 click-default-group==1.2.4
     # via pynxtools (pyproject.toml)
 colorama==0.4.6
     # via mkdocs-material
 contourpy==1.2.0
     # via matplotlib
-coverage[toml]==7.4.4
+coverage==7.4.4
     # via pytest-cov
 cycler==0.12.1
     # via matplotlib
@@ -61,11 +55,12 @@ jinja2==3.1.3
     #   mkdocs-material
 kiwisolver==1.4.5
     # via matplotlib
-lxml==5.1.0
+lxml==5.2.2
     # via pynxtools (pyproject.toml)
 markdown==3.6
     # via
     #   mkdocs
+    #   mkdocs-click
     #   mkdocs-material
     #   pymdown-extensions
 markupsafe==2.1.5
@@ -76,47 +71,48 @@ matplotlib==3.8.3
     # via ase
 mergedeep==1.3.4
     # via
-    #   mkdocs
     #   pynxtools (pyproject.toml)
+    #   mkdocs
 mkdocs==1.5.3
     # via
+    #   pynxtools (pyproject.toml)
     #   mkdocs-macros-plugin
     #   mkdocs-material
-    #   pynxtools (pyproject.toml)
+mkdocs-click==0.8.1
+    # via pynxtools (pyproject.toml)
 mkdocs-macros-plugin==1.0.5
     # via pynxtools (pyproject.toml)
 mkdocs-material==9.5.15
     # via pynxtools (pyproject.toml)
 mkdocs-material-extensions==1.3.1
     # via
-    #   mkdocs-material
     #   pynxtools (pyproject.toml)
+    #   mkdocs-material
 mypy==1.9.0
     # via pynxtools (pyproject.toml)
 mypy-extensions==1.0.0
     # via mypy
 nodeenv==1.8.0
     # via pre-commit
-numpy==1.26.4
+numpy==1.22.4
     # via
+    #   pynxtools (pyproject.toml)
     #   ase
     #   contourpy
     #   h5py
     #   matplotlib
     #   pandas
-    #   pynxtools (pyproject.toml)
     #   scipy
     #   xarray
 packaging==24.0
     # via
-    #   build
     #   matplotlib
     #   mkdocs
     #   pytest
     #   xarray
 paginate==0.5.6
     # via mkdocs-material
-pandas==2.2.1
+pandas==1.5.3
     # via
     #   pynxtools (pyproject.toml)
     #   xarray
@@ -124,8 +120,6 @@ pathspec==0.12.1
     # via mkdocs
 pillow==10.2.0
     # via matplotlib
-pip-tools==7.4.1
-    # via pynxtools (pyproject.toml)
 platformdirs==4.2.0
     # via
     #   mkdocs
@@ -140,10 +134,6 @@ pymdown-extensions==10.7.1
     # via mkdocs-material
 pyparsing==3.1.2
     # via matplotlib
-pyproject-hooks==1.0.0
-    # via
-    #   build
-    #   pip-tools
 pytest==8.1.1
     # via
     #   pynxtools (pyproject.toml)
@@ -163,11 +153,11 @@ pytz==2024.1
     # via pandas
 pyyaml==6.0.1
     # via
+    #   pynxtools (pyproject.toml)
     #   mkdocs
     #   mkdocs-macros-plugin
     #   pre-commit
     #   pymdown-extensions
-    #   pynxtools (pyproject.toml)
     #   pyyaml-env-tag
 pyyaml-env-tag==0.1
     # via mkdocs
@@ -179,6 +169,10 @@ ruff==0.3.4
     # via pynxtools (pyproject.toml)
 scipy==1.12.0
     # via ase
+setuptools==70.0.0
+    # via
+    #   pynxtools (pyproject.toml)
+    #   nodeenv
 six==1.16.0
     # via
     #   anytree
@@ -189,11 +183,8 @@ termcolor==2.4.0
     # via mkdocs-macros-plugin
 tomli==2.0.1
     # via
-    #   build
     #   coverage
     #   mypy
-    #   pip-tools
-    #   pyproject-hooks
     #   pytest
 types-pytz==2024.1.0.20240203
     # via pynxtools (pyproject.toml)
@@ -203,23 +194,17 @@ types-requests==2.31.0.20240311
     # via pynxtools (pyproject.toml)
 typing-extensions==4.10.0
     # via mypy
-tzdata==2024.1
-    # via pandas
 urllib3==2.2.1
     # via
     #   requests
     #   types-requests
+uv==0.2.9
+    # via pynxtools (pyproject.toml)
 virtualenv==20.25.1
     # via pre-commit
 watchdog==4.0.0
     # via mkdocs
-wheel==0.43.0
-    # via pip-tools
-xarray==2024.2.0
+xarray==2023.12.0
     # via pynxtools (pyproject.toml)
 zipp==3.18.1
     # via importlib-metadata
-
-# The following packages are considered to be unsafe in a requirements file:
-# pip
-# setuptools

{pynxtools-0.3.2 → pynxtools-0.4.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [build-system]
 requires = ["setuptools>=64.0.1", "setuptools-scm[toml]>=6.2"]
-backend-path = ["pynxtools"]
+backend-path = ["src/pynxtools"]
 build-backend = "_build_wrapper"
 
 [project]
@@ -28,7 +28,7 @@ dependencies = [
     "h5py>=3.6.0",
     "xarray>=0.20.2",
     "PyYAML>=6.0",
-    "numpy>=1.21.2",
+    "numpy>=1.21.2,<2.0.0",
     "pandas>=1.3.2",
     "ase>=3.19.0",
     "mergedeep",
@@ -47,6 +47,7 @@ docs = [
     "mkdocs-material",
     "mkdocs-material-extensions",
     "mkdocs-macros-plugin",
+    "mkdocs-click",
 ]
 dev = [
     "mypy",
@@ -58,7 +59,7 @@ dev = [
     "types-pyyaml",
     "types-pytz",
     "types-requests",
-    "pip-tools",
+    "uv",
     "pre-commit",
 ]
 convert = [
@@ -86,6 +87,11 @@ ellips = [
     "pynxtools-ellips",
 ]
 
+[project.entry-points.'nomad.plugin']
+nexus_parser = "pynxtools.nomad.entrypoints:nexus_parser"
+nexus_schema = "pynxtools.nomad.entrypoints:nexus_schema"
+nexus_data_converter = "pynxtools.nomad.entrypoints:nexus_data_converter"
+
 [project.scripts]
 read_nexus = "pynxtools.nexus.nexus:main"
 dataconverter = "pynxtools.dataconverter.convert:main_cli"
@@ -93,20 +99,22 @@ generate_eln = "pynxtools.eln_mapper.eln_mapper:get_eln"
 
 [tool.setuptools.package-data]
 pynxtools = ["definitions/**/*.xml", "definitions/**/*.xsd"]
-"pynxtools.dataconverter.readers.xps" = ["*.json"]
 "pynxtools.dataconverter.readers.hall" = ["enum_map.json"]
 "pynxtools.dataconverter.readers.rii_database.formula_parser" = ["dispersion_function_grammar.lark"]
 
 [tool.setuptools.packages.find]
-exclude = ["pynxtools/definitions*"]
+where = [
+    "src",
+]
+exclude = ["src/pynxtools/definitions*"]
 
 [tool.setuptools_scm]
 version_scheme = "no-guess-dev"
 local_scheme = "node-and-date"
 
 [tool.ruff]
-include = ["pynxtools/*.py", "tests/*.py"]
-exclude = ["pynxtools/definitions"]
+include = ["src/*.py", "tests/*.py"]
+exclude = ["src/pynxtools/definitions"]
 line-length = 88
 indent-width = 4
 
@@ -145,4 +153,4 @@ ignore_missing_imports = true
 follow_imports = "silent"
 no_strict_optional = true
 disable_error_code = "import, annotation-unchecked"
-exclude = ["pynxtools/definitions/*"]
+exclude = ["src/pynxtools/definitions/*"]

{pynxtools-0.3.2 → pynxtools-0.4.0/src}/pynxtools/_build_wrapper.py

@@ -7,8 +7,11 @@ import os
 from subprocess import CalledProcessError, run
 from typing import Optional
 
-from setuptools import build_meta as _orig
-from setuptools.build_meta import *  # pylint: disable=wildcard-import,unused-wildcard-import
+try:
+    from setuptools import build_meta as _orig
+    from setuptools.build_meta import *  # pylint: disable=wildcard-import,unused-wildcard-import
+except ImportError:
+    pass
 
 
 def get_vcs_version(tag_match="*[0-9]*") -> Optional[str]:

pynxtools-0.4.0/src/pynxtools/data/NXtest.nxdl.xml

@@ -0,0 +1,76 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<?xml-stylesheet type="text/xsl" href="nxdlformat.xsl" ?>
+<definition xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd"
+    xmlns="http://definition.nexusformat.org/nxdl/3.1"
+ name="NXtest"
+ extends="NXobject"
+ type="group"
+ category="application"
+>
+    <doc>This is a dummy NXDL to test out the dataconverter.</doc>
+    <group type="NXentry">
+        <field name="program_name"/>
+        <field name="definition">
+            <doc>This is a dummy NXDL to test out the dataconverter.</doc>
+            <attribute name="version"/>
+            <enumeration>
+                <item value="NXTEST"/>
+                <item value="NXtest"/>
+            </enumeration>
+        </field>
+        <group type="NXdata" name="OPTIONAL_group" optional="true">
+            <field name="required_field" required="true" type="NX_INT">
+                <doc>A dummy entry to test optional parent check for a required child.</doc>
+            </field>
+            <field name="optional_field" optional="true" type="NX_INT">
+                <doc>A dummy entry to test optional parent check for an optional child.</doc>
+            </field>
+        </group>
+        <group type="NXdata" name="NXODD_name">
+            <field name="float_value" type="NX_FLOAT" optional="true" units="NX_ENERGY">
+                <doc>A dummy entry for a float value.</doc>
+            </field>
+            <field name="bool_value" type="NX_BOOLEAN" required="true" units="NX_UNITLESS">
+                <doc>A dummy entry for a bool value.</doc>
+            </field>
+            <field name="int_value" type="NX_INT" units="NX_LENGTH">
+                <doc>A dummy entry for an int value.</doc>
+            </field>
+            <field name="posint_value" type="NX_POSINT" units="NX_LENGTH">
+                <doc>A dummy entry for a positive int value.</doc>
+            </field>
+            <field name="char_value" type="NX_CHAR" units="NX_UNITLESS">
+                <doc>A dummy entry for a char value.</doc>
+            </field>
+            <field name="date_value" type="NX_DATE_TIME" units="NX_UNITLESS">
+                <doc>A dummy entry for a date value.</doc>
+            </field>
+            <field name="type">
+                <enumeration>
+                    <item value="1st type" />
+                    <item value="2nd type" />
+                    <item value="3rd type" />
+                    <item value="4th type" />
+                </enumeration>
+            </field>
+        </group>
+        <group type="NXnote" name="required_group">
+            <doc>This is a required yet empty group.</doc>
+        </group>
+        <group type="NXnote" name="required_group2">
+            <doc>This is a second required yet empty group.</doc>
+        </group>
+        <group type="NXdata" name="optional_parent" optional="true">
+            <field name="required_child" required="true" type="NX_INT">
+                <doc>A dummy entry to test optional parent check for required child.</doc>
+            </field>
+            <field name="optional_child" optional="true" type="NX_INT">
+                <doc>A dummy entry to test optional parent check for required child.</doc>
+            </field>
+            <group type="NXdata" name="req_group_in_opt_group">
+                <doc>This is a required group in an optional group.</doc>
+            </group>
+        </group>
+    </group>
+</definition>

pynxtools-0.4.0/src/pynxtools/data/NXtest_extended.nxdl.xml

@@ -0,0 +1,23 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<?xml-stylesheet type="text/xsl" href="nxdlformat.xsl" ?>
+<definition xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd"
+    xmlns="http://definition.nexusformat.org/nxdl/3.1"
+ name="NXtest_extended"
+ extends="NXtest"
+ type="group"
+ category="application"
+>
+    <doc>This is a dummy NXDL to test an extended application definition.</doc>
+    <group type="NXentry">
+        <field name="definition">
+            <doc>This is a dummy NXDL to test out the dataconverter.</doc>
+            <enumeration>
+                <item value="NXtest_extended"/>
+            </enumeration>
+        </field>
+        <field name="extended_field" type="NX_FLOAT" units="NX_ENERGY">
+            <doc>A dummy entry for an extended field.</doc>
+        </field>
+    </group>
+</definition>

{pynxtools-0.3.2 → pynxtools-0.4.0/src}/pynxtools/dataconverter/convert.py

@@ -39,7 +39,6 @@ from pynxtools.dataconverter.readers.base.reader import BaseReader
 from pynxtools.dataconverter.template import Template
 from pynxtools.dataconverter.validation import validate_dict_against
 from pynxtools.dataconverter.writer import Writer
-from pynxtools.nexus import nexus
 
 logger = logging.getLogger(__name__)
 logger.setLevel(logging.INFO)