fmu-manipulation-toolbox 1.8.4.3b0__tar.gz → 1.9__tar.gz

{fmu_manipulation_toolbox-1.8.4.3b0 → fmu_manipulation_toolbox-1.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fmu_manipulation_toolbox
-Version: 1.8.4.3b0
+Version: 1.9
 Summary: FMU Manipulation Toolbox is a python application for modifying Functional Mock-up Units (FMUs) without recompilation or bundling them into FMU Containers
 Home-page: https://github.com/grouperenault/fmu_manipulation_toolbox/
 Author: Nicolas.LAURENT@Renault.com

{fmu_manipulation_toolbox-1.8.4.3b0 → fmu_manipulation_toolbox-1.9}/README.md

@@ -1,12 +1,23 @@
-![Title](fmu_manipulation_toolbox/resources/fmu_manipulation_toolbox.png)
+![](fmu_manipulation_toolbox/resources/fmu_manipulation_toolbox.png)
 
-FMU Manipulation Toolbox is a python package which help to analyse and modify a [Functional Mock-up Units (FMUs)](http://fmi-standard.org/)
-without recompilation. It is highly customizable. It can even modify the `modelDescription.xml` file.
+FMU Manipulation Toolbox is a python package which help to analyse, modify or combine
+[Functional Mock-up Units (FMUs)](http://fmi-standard.org/) without recompilation. It is highly customizable as it comes with
+a Python API.
 
-Manipulating the `modelDescription.xml` can be a dangerous thing! Communicating with the FMU-developer and adapting
-the way the FMU is generated, is preferable when possible.
+FMU Manipulation Toolbox can be used in different ways :
+- Using a Graphical User Interface: suitable for end user
+- Using a Command Line Interface: useful in 
+- Using a python API: the most efficient for automation (CI/CD, transformation scripts, ...)
 
-FMU Manipulation Toolbox also allows to group FMU's inside FMU Containers. (see [doc/container.md](doc/container.md))
+Major features:
+- Analyse FMU content: list ports and their attributes, check compliance of `ModelDescription.xml` with XSD, ...
+- Alter FMU by modifying its `modelDescription.xml` file. NOTE: manipulating this file can be a dangerous
+  thing! Communicating with the FMU-developer and adapting the way the FMU is generated is preferable when
+  possible.
+- Add binary interface. Typical use case is porting a 32bits FMUs to 64bits (or vice et versa). 
+- Nest FMUs in a [FMU Container](doc/container.md)
+
+FMI versions 2.0 and 3.0 are supported.
 
 ## Installation
 
@@ -17,29 +28,39 @@ Two options available to install FMU Manipulation Toolbox:
 - Compile and install from [github repository](https://github.com/grouperenault/fmu_manipulation_toolbox). You will need 
   - Python required packages. See [`requirements.txt`](requirements.txt).
   - C compiler (C99 or later)
-  - CMake (>= 3.16)
+  - CMake (>= 3.20)
+
 
 ### Supported platforms
-FMU Manipulation Toolbox is tested on:
-- Windows 10/11
+
+FMU Manipulation Toolbox is packaged for:
+- Windows 10/11 (primary platform)
 - Linux (Ubuntu 22.04)
+- Darwin
 
-Compilation is reported to work on:
-- MacOS (Apple Silicon and Intel)
 
 ## Graphical User Interface
 
-FMU Manipulation Toolbox is released with a GUI. You can launch it with the following command `fmutool`
-(without any option)
+FMU Manipulation Toolbox is released with a GUI. You can launch it with the following command `fmutool-gui`
 
 ![GUI](doc/gui.png "GUI")
 
+Button colors descriptions:
+- red: remove information from the `modelDescription.xml`
+- orange: alter `modelDescription.xml`
+- green: add component into the FMU or check it
+- violet: extract and save
+- blue: filter actions scope or exit
+
+**Original FMU is never modified**. Use `Save` button to get modified copy of the original FMU.
+
 
 ## Command Line Interface
 
 FMU Manipulation Toolbox comes with 2 commands:
 - `fmutool`: a versatile analysis and manipulation tool for FMU.
-- `fmucontainer`: group FMU's inside FMU Containers. (see [container/README.md](container/README.md))
+- `fmucontainer`: group FMUs inside FMU Containers. (see [container/README.md](container/README.md))
+- `fmusplit: to extract FMUs from a FMU Container.
 
 
 ### Analysis and Manipulation tool:
@@ -131,24 +152,29 @@ optional arguments:
 ### FMU Containers
 
 ```
-usage: fmucontainer [-h] [-fmu-directory FMU_DIRECTORY] -container filename.{csv|json|ssp},[:step_size] [-debug]
-                    [-no-auto-input] [-no-auto-output] [-auto-parameter] [-no-auto-link] [-mt] [-profile] [-dump-json]
+usage: fmucontainer [-h] [-fmu-directory FMU_DIRECTORY] [-fmi FMI_VERSION]
+                    -container filename.{csv|json|ssp},[:step_size] [-debug] [-no-auto-input] [-no-auto-output]
+                    [-auto-parameter] [-auto-local] [-no-auto-link] [-mt] [-profile] [-sequential] [-dump-json]
 
 Generate FMU from FMU's
 
-optional arguments:
+options:
   -h, -help
   -fmu-directory FMU_DIRECTORY      Directory containing initial FMU’s and used to generate containers. If not defined,
                                     current directory is used. (default: .)
+  -fmi FMI_VERSION                  Define version of FMI to be used for container interface.Only '2' or '3' is
+                                    supported. (default: 2)
   -container filename.{csv|json|ssp},[:step_size]
                                     Description of the container to create. (default: [])
   -debug                            Add lot of useful log during the process. (default: False)
   -no-auto-input                    Create ONLY explicit input. (default: True)
   -no-auto-output                   Create ONLY explicit output. (default: True)
   -auto-parameter                   Expose parameters of the embedded fmu's. (default: False)
+  -auto-local                       Expose local variables of the embedded fmu's. (default: False)
   -no-auto-link                     Create ONLY explicit links. (default: True)
   -mt                               Enable Multi-Threaded mode for the generated container. (default: False)
   -profile                          Enable Profiling mode for the generated container. (default: False)
+  -sequential                       Use sequential mode to schedule embedded fmu's. (default: False)
   -dump-json                        Dump a JSON file for each container. (default: False)
 ```
 
@@ -158,14 +184,13 @@ You can write your own FMU Manipulation scripts. Once you downloaded fmutool mod
 adding the `import` statement lets you access the API :
 
 ```python
-from fmu_manipulation_toolbox.operations import FMU, OperationExtractNames, OperationStripTopLevel,
-
-OperationRenameFromCSV
+from fmu_manipulation_toolbox.operations import ...
 ```
 
+
 ### remove toplevel bus (if any)
 
-Give a FMU with the following I/O structure
+Given a FMU with the following I/O structure
 ```
 ├── Parameters
 │   ├── Foo
@@ -189,6 +214,8 @@ The following transformation will lead into:
 
 The following code will do this transformation: 
 ```python
+from fmu_manipulation_toolbox.operations import FMU, OperationStripTopLevel
+
 fmu = FMU(r"bouncing_ball.fmu")
 operation = OperationStripTopLevel()
 fmu.apply_operation(operation)
@@ -200,8 +227,10 @@ fmu.repack(r"bouncing_ball-modified.fmu")
 The following code will dump all FMU's Scalars names into a CSV:
 
 ```python
+from fmu_manipulation_toolbox.operations import FMU, OperationSaveNamesToCSV
+
 fmu = FMU(r"bouncing_ball.fmu")
-operation = OperationExtractNames()
+operation = OperationSaveNamesToCSV()
 fmu.apply_operation(operation)
 operation.write_csv(r"bouncing_ball.csv")
 ```
@@ -219,15 +248,20 @@ g;g;4;parameter;fixed
 e;e;5;parameter;tunable
 ```
 
+
 ### Read CSV and rename FMU ports
 
-CSV file should contain- 2 columns:
+CSV file should contain 2 columns:
 1. the current name
 2. the new name
 
 ```python
+from fmu_manipulation_toolbox.operations import FMU, OperationRenameFromCSV
+
 fmu = FMU(r"bouncing_ball.fmu")
 operation = OperationRenameFromCSV(r"bouncing_ball-modified.csv")
 fmu.apply_operation(operation)
 fmu.repack(r"bouncing_ball-renamed.fmu")
 ```
+
+More operations exist in [`Operation.py`](fmu_manipulation_toolbox/operations.py)

{fmu_manipulation_toolbox-1.8.4.3b0 → fmu_manipulation_toolbox-1.9}/fmu_manipulation_toolbox/__main__.py

@@ -10,7 +10,7 @@ def gui():
 
 
 def cli():
-    from .cli import fmutool
+    from .cli.fmutool import fmutool
     fmutool()
 
 

fmu_manipulation_toolbox-1.9/fmu_manipulation_toolbox/__version__.py

@@ -0,0 +1 @@
+'V1.9'

{fmu_manipulation_toolbox-1.8.4.3b0 → fmu_manipulation_toolbox-1.9}/fmu_manipulation_toolbox/assembly.py

@@ -37,7 +37,7 @@ class Connection:
 
 
 class AssemblyNode:
-    def __init__(self, name: Optional[str], step_size: float = None, mt=False, profiling=False,
+    def __init__(self, name: Optional[str], step_size: float = None, mt=False, profiling=False, sequential=False,
                  auto_link=True, auto_input=True, auto_output=True, auto_parameter=False, auto_local=False):
         self.name = name
         if step_size:
@@ -50,6 +50,7 @@ class AssemblyNode:
             self.step_size = None
         self.mt = mt
         self.profiling = profiling
+        self.sequential = sequential
         self.auto_link = auto_link
         self.auto_input = auto_input
         self.auto_output = auto_output
@@ -100,12 +101,13 @@ class AssemblyNode:
     def add_start_value(self, fmu_filename: str, port_name: str, value: str):
         self.start_values[Port(fmu_filename, port_name)] = value
 
-    def make_fmu(self, fmu_directory: Path, debug=False, description_pathname=None):
+    def make_fmu(self, fmu_directory: Path, debug=False, description_pathname=None, fmi_version=2):
         for node in self.children.values():
-            node.make_fmu(fmu_directory, debug=debug)
+            node.make_fmu(fmu_directory, debug=debug, fmi_version=fmi_version)
 
         identifier = str(Path(self.name).stem)
-        container = FMUContainer(identifier, fmu_directory, description_pathname=description_pathname)
+        container = FMUContainer(identifier, fmu_directory, description_pathname=description_pathname,
+                                 fmi_version=fmi_version)
 
         for fmu_name in self.fmu_names_list:
             container.get_fmu(fmu_name)
@@ -138,7 +140,8 @@ class AssemblyNode:
         for link_rule in wired.rule_link:
             self.add_link(link_rule[0], link_rule[1], link_rule[2], link_rule[3])
 
-        container.make_fmu(self.name, self.step_size, mt=self.mt, profiling=self.profiling, debug=debug)
+        container.make_fmu(self.name, self.step_size, mt=self.mt, profiling=self.profiling, sequential=self.sequential,
+                           debug=debug)
 
         for node in self.children.values():
             logger.info(f"Deleting transient FMU Container '{node.name}'")
@@ -216,7 +219,7 @@ class AssemblyError(Exception):
 
 
 class Assembly:
-    def __init__(self, filename: str, step_size=None, auto_link=True,  auto_input=True, debug=False,
+    def __init__(self, filename: str, step_size=None, auto_link=True,  auto_input=True, debug=False, sequential=False,
                  auto_output=True, mt=False, profiling=False, fmu_directory: Path = Path("."), auto_parameter=False,
                  auto_local=False):
         self.filename = Path(filename)
@@ -228,6 +231,7 @@ class Assembly:
         self.default_auto_parameter = auto_parameter
         self.default_auto_local = auto_local
         self.default_mt = mt
+        self.default_sequential = sequential
         self.default_profiling = profiling
         self.fmu_directory = fmu_directory
         self.transient_filenames: List[Path] = []
@@ -238,7 +242,7 @@ class Assembly:
 
         self.input_pathname = fmu_directory / self.filename
         self.description_pathname = self.input_pathname   # For inclusion in FMU
-        self.root = None
+        self.root: Optional[AssemblyNode] = None
         self.read()
 
     def add_transient_file(self, filename: str):
@@ -283,8 +287,9 @@ class Assembly:
         name = str(self.filename.with_suffix(".fmu"))
         self.root = AssemblyNode(name, step_size=self.default_step_size, auto_link=self.default_auto_link,
                                  mt=self.default_mt, profiling=self.default_profiling,
-                                 auto_input=self.default_auto_input, auto_output=self.default_auto_output,
-                                 auto_parameter=self.default_auto_parameter, auto_local=self.default_auto_local)
+                                 sequential=self.default_sequential, auto_input=self.default_auto_input,
+                                 auto_output=self.default_auto_output, auto_parameter=self.default_auto_parameter,
+                                 auto_local=self.default_auto_local)
 
         with open(self.input_pathname) as file:
             reader = csv.reader(file, delimiter=';')
@@ -385,6 +390,7 @@ class Assembly:
         name = data.get("name", None)                                                       # 1
         mt = data.get("mt", self.default_mt)                                                # 2
         profiling = data.get("profiling", self.default_profiling)                           # 3
+        sequential = data.get("sequential", self.default_sequential)                        # 3b
         auto_link = data.get("auto_link", self.default_auto_link)                           # 4
         auto_input = data.get("auto_input", self.default_auto_input)                        # 5
         auto_output = data.get("auto_output", self.default_auto_output)                     # 6
@@ -393,11 +399,12 @@ class Assembly:
         step_size = data.get("step_size", self.default_step_size)                           # 7
 
         node = AssemblyNode(name, step_size=step_size, auto_link=auto_link, mt=mt, profiling=profiling,
+                            sequential=sequential,
                             auto_input=auto_input, auto_output=auto_output, auto_parameter=auto_parameter,
                             auto_local=auto_local)
 
         for key, value in data.items():
-            if key in ('name', 'step_size', 'auto_link', 'auto_input', 'auto_output', 'mt', 'profiling',
+            if key in ('name', 'step_size', 'auto_link', 'auto_input', 'auto_output', 'mt', 'profiling', 'sequential',
                        'auto_parameter', 'auto_local'):
                 continue  # Already read
 
@@ -455,6 +462,7 @@ class Assembly:
         json_node["name"] = node.name                      # 1
         json_node["mt"] = node.mt                          # 2
         json_node["profiling"] = node.profiling            # 3
+        json_node["sequential"] = node.sequential          # 3b
         json_node["auto_link"] = node.auto_link            # 4
         json_node["auto_input"] = node.auto_input          # 5
         json_node["auto_output"] = node.auto_output        # 6
@@ -510,8 +518,9 @@ class Assembly:
             self.root = sdd.parse(self.description_pathname)
             self.root.name = str(self.filename.with_suffix(".fmu"))
 
-    def make_fmu(self, dump_json=False):
-        self.root.make_fmu(self.fmu_directory, debug=self.debug, description_pathname=self.description_pathname)
+    def make_fmu(self, dump_json=False, fmi_version=2):
+        self.root.make_fmu(self.fmu_directory, debug=self.debug, description_pathname=self.description_pathname,
+                           fmi_version=fmi_version)
         if dump_json:
             dump_file = Path(self.input_pathname.stem + "-dump").with_suffix(".json")
             logger.info(f"Dump Json '{dump_file}'")

{fmu_manipulation_toolbox-1.8.4.3b0 → fmu_manipulation_toolbox-1.9}/fmu_manipulation_toolbox/checker.py

@@ -1,10 +1,13 @@
 import importlib.util
 import inspect
+import logging
 import os
 import xmlschema
 from xmlschema.validators.exceptions import XMLSchemaValidationError
 from .operations import OperationAbstract
 
+logger = logging.getLogger("fmu_manipulation_toolbox")
+
 
 class OperationGenericCheck(OperationAbstract):
     SUPPORTED_FMI_VERSIONS = ('2.0', '3.0')
@@ -17,7 +20,7 @@ class OperationGenericCheck(OperationAbstract):
 
     def fmi_attrs(self, attrs):
         if attrs['fmiVersion'] not in self.SUPPORTED_FMI_VERSIONS:
-            print(f"ERROR: Expected FMI {','.join(self.SUPPORTED_FMI_VERSIONS)} versions.")
+            logger.error(f"Expected FMI {','.join(self.SUPPORTED_FMI_VERSIONS)} versions.")
             return
 
         fmi_name = f"fmi{attrs['fmiVersion'][0]}"
@@ -27,15 +30,15 @@ class OperationGenericCheck(OperationAbstract):
         try:
             xmlschema.validate(self.fmu.descriptor_filename, schema=xsd_filename)
         except XMLSchemaValidationError as error:
-            print(error.reason, error.msg)
+            logger.error(error.reason, error.msg)
         else:
             self.compliant_with_version = attrs['fmiVersion']
 
     def closure(self):
         if self.compliant_with_version:
-            print(f"INFO: This FMU seems to be compliant with FMI-{self.compliant_with_version}.")
+            logger.info(f"This FMU seems to be compliant with FMI-{self.compliant_with_version}.")
         else:
-            print(f"ERROR: This FMU does not validate with FMI standard.")
+            logger.error(f"This FMU does not validate with FMI standard.")
 
 
 checker_list = [OperationGenericCheck]
@@ -44,20 +47,20 @@ checker_list = [OperationGenericCheck]
 def add_from_file(checker_filename: str):
     spec = importlib.util.spec_from_file_location(checker_filename, checker_filename)
     if not spec:
-        print(f"ERROR: Cannot load {checker_filename}. Is this a python file?")
+        logger.error(f"Cannot load '{checker_filename}'. Is this a python file?")
         return
     try:
         checker_module = importlib.util.module_from_spec(spec)
         try:
             spec.loader.exec_module(checker_module)
         except (ModuleNotFoundError, SyntaxError) as error:
-            print(f"ERROR: Cannot load {checker_filename}: {error})")
+            logger.error(f"Cannot load '{checker_filename}': {error})")
             return
 
         for checker_name, checker_class in inspect.getmembers(checker_module, inspect.isclass):
             if OperationAbstract in checker_class.__bases__:
                 checker_list.append(checker_class)
-                print(f"Adding checker: {checker_filename}|{checker_name}")
+                logger.info(f"Adding checker: {checker_filename}|{checker_name}")
 
     except AttributeError:
-        print(f"ERROR: {checker_filename} should implement class 'OperationCheck'")
+        logger.error(f"'{checker_filename}' should implement class 'OperationCheck'")

fmu_manipulation_toolbox-1.9/fmu_manipulation_toolbox/cli/fmucontainer.py

@@ -0,0 +1,105 @@
+import argparse
+import logging
+import sys
+
+from typing import *
+from pathlib import Path
+
+from .utils import setup_logger, make_wide
+from ..assembly import Assembly, AssemblyError
+from ..container import FMUContainerError
+from ..version import __version__ as version
+
+
+def fmucontainer(command_options: Sequence[str]):
+    logger = setup_logger()
+
+    logger.info(f"FMUContainer version {version}")
+    parser = argparse.ArgumentParser(prog="fmucontainer", description="Generate FMU from FMU's",
+                                     formatter_class=make_wide(argparse.ArgumentDefaultsHelpFormatter),
+                                     add_help=False,
+                                     epilog="see: https://github.com/grouperenault/fmu_manipulation_toolbox/blob/main/"
+                                            "container/README.md")
+
+    parser.add_argument('-h', '-help', action="help")
+
+    parser.add_argument("-fmu-directory", action="store", dest="fmu_directory", required=False, default=".",
+                        help="Directory containing initial FMU’s and used to generate containers. "
+                             "If not defined, current directory is used.")
+
+    parser.add_argument("-fmi", action="store", dest="fmi_version", required=False, default="2",
+                        help="Define version of FMI to be used for container interface."
+                             "Only '2' or '3' is supported.")
+
+    parser.add_argument("-container", action="append", dest="container_descriptions_list", default=[],
+                        metavar="filename.{csv|json|ssp},[:step_size]", required=True,
+                        help="Description of the container to create.")
+
+    parser.add_argument("-debug", action="store_true", dest="debug",
+                        help="Add lot of useful log during the process.")
+
+    parser.add_argument("-no-auto-input", action="store_false", dest="auto_input", default=True,
+                        help="Create ONLY explicit input.")
+
+    parser.add_argument("-no-auto-output", action="store_false", dest="auto_output", default=True,
+                        help="Create ONLY explicit output.")
+
+    parser.add_argument("-auto-parameter", action="store_true", dest="auto_parameter", default=False,
+                        help="Expose parameters of the embedded fmu's.")
+
+    parser.add_argument("-auto-local", action="store_true", dest="auto_local", default=False,
+                        help="Expose local variables of the embedded fmu's.")
+
+    parser.add_argument("-no-auto-link", action="store_false", dest="auto_link", default=True,
+                        help="Create ONLY explicit links.")
+
+    parser.add_argument("-mt", action="store_true", dest="mt", default=False,
+                        help="Enable Multi-Threaded mode for the generated container.")
+
+    parser.add_argument("-profile", action="store_true", dest="profiling", default=False,
+                        help="Enable Profiling mode for the generated container.")
+
+    parser.add_argument("-sequential", action="store_true", dest="sequential", default=False,
+                        help="Use sequential mode to schedule embedded fmu's.")
+
+    parser.add_argument("-dump-json",  action="store_true", dest="dump", default=False,
+                        help="Dump a JSON file for each container.")
+
+    config = parser.parse_args(command_options)
+
+    if config.debug:
+        logger.setLevel(logging.DEBUG)
+
+    fmu_directory = Path(config.fmu_directory)
+    logger.info(f"FMU directory: '{fmu_directory}'")
+
+    for description in config.container_descriptions_list:
+        try:
+            tokens = description.split(":")
+            filename = ":".join(tokens[:-1])
+            step_size = float(tokens[-1])
+        except ValueError:
+            step_size = None
+            filename = description
+
+        try:
+            assembly = Assembly(filename, step_size=step_size, auto_link=config.auto_link,
+                                auto_input=config.auto_input, auto_output=config.auto_output,
+                                auto_local=config.auto_local, mt=config.mt, sequential=config.sequential,
+                                profiling=config.profiling, fmu_directory=fmu_directory, debug=config.debug,
+                                auto_parameter=config.auto_parameter)
+        except FileNotFoundError as e:
+            logger.fatal(f"Cannot read file: {e}")
+            sys.exit(-1)
+        except (FMUContainerError, AssemblyError) as e:
+            logger.fatal(f"{filename}: {e}")
+            sys.exit(-2)
+
+        try:
+            assembly.make_fmu(dump_json=config.dump, fmi_version=int(config.fmi_version))
+        except FMUContainerError as e:
+            logger.fatal(f"{filename}: {e}")
+            sys.exit(-3)
+
+if __name__ == "__main__":
+    fmucontainer(sys.argv[1:])

fmu_manipulation_toolbox-1.9/fmu_manipulation_toolbox/cli/fmusplit.py

@@ -0,0 +1,48 @@
+import argparse
+import logging
+import sys
+
+from typing import *
+
+from .utils import setup_logger, make_wide
+from ..split import FMUSplitter, FMUSplitterError
+from ..version import __version__ as version
+
+
+def fmusplit(command_options: Sequence[str]):
+    logger = setup_logger()
+
+    logger.info(f"FMUSplit version {version}")
+    parser = argparse.ArgumentParser(prog="fmusplit", description="Split FMU Container into FMU's",
+                                     formatter_class=make_wide(argparse.ArgumentDefaultsHelpFormatter),
+                                     add_help=False,
+                                     epilog="see: https://github.com/grouperenault/fmu_manipulation_toolbox/blob/main/"
+                                            "container/README.md")
+    parser.add_argument('-h', '-help', action="help")
+
+    parser.add_argument("-debug", action="store_true", dest="debug",
+                        help="Add lot of useful log during the process.")
+
+    parser.add_argument("-fmu", action="append", dest="fmu_filename_list", default=[],
+                        metavar="filename.fmu", required=True,
+                        help="Description of the FMU container to split.")
+
+    config = parser.parse_args(command_options)
+
+    if config.debug:
+        logger.setLevel(logging.DEBUG)
+
+    for fmu_filename in config.fmu_filename_list:
+        try:
+            splitter = FMUSplitter(fmu_filename)
+            splitter.split_fmu()
+        except FMUSplitterError as e:
+            logger.fatal(f"{fmu_filename}: {e}")
+            sys.exit(-1)
+        except FileNotFoundError as e:
+            logger.fatal(f"Cannot read file: {e}")
+            sys.exit(-2)
+
+
+if __name__ == "__main__":
+    fmusplit(sys.argv[1:])