ioiocore 0.0.1__tar.gz

ioiocore-0.0.1/CHANGELOG.md

@@ -0,0 +1,11 @@
+# Change Log
+
+## 0.0.1
+### Added
+- Initial release
+
+### Changed
+- Nothing 
+
+### Removed
+- Nothing 

ioiocore-0.0.1/HOW_TO_BUILD.md

@@ -0,0 +1,50 @@
+# Buld instructions
+## Prerequisites
+- Visual Studio Code<br>
+&nbsp;&nbsp;&nbsp; - Python (Microsoft) v2024.16.1 Extension<br>
+&nbsp;&nbsp;&nbsp; - Python Debugger (Microsoft) v2024.10.00<br>
+&nbsp;&nbsp;&nbsp; - Pylance (Microsoft) v2024.10.01 Extension<br>
+&nbsp;&nbsp;&nbsp; - Markdown Preview Github Styling (Matt Bierner) v2.1.0 Extension<br>
+- Python >= 3.11<br>
+&nbsp;&nbsp;&nbsp; - pip (must be accessible from commandline)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**Note**: ensure that the pip package is installed ```python -m ensurepip --upgrade```<br>
+&nbsp;&nbsp;&nbsp; - pipreqs (must be accessible from commandline)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**Note**: ensure that the pipreqs package is installed ```pip install pipreqs```<br>
+&nbsp;&nbsp;&nbsp; - build (must be accessible from commandline)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**Note**: ensure that the build package is installed ```pip install build```<br>
+&nbsp;&nbsp;&nbsp; - setuptools<br>
+&nbsp;&nbsp;&nbsp; - numpy (for tests only)<br>
+
+## Create requirements file
+
+Create or update requirements.txt<br>
+```pipreqs ./ioiocore --force```
+
+## Build
+Update version number:<br>
+```./ioiocore/___version__.py```
+
+build wheel:<br>
+```python -m build```<br>
+distributables are built to the ```./dist``` folder
+
+### Upload to pypi
+Upload to pypi using the following command: <br>
+```python -m twine upload --repository pypi dist/*```
+
+## Installation
+### Manual installation
+Copy ```.whl``` file from  ```./dist``` folder and install from command line: <br>
+```pip install ./dist/ioiocore-0.0.1-py3-none-any.whl```
+
+### Installation using PyPi
+Package can be downloaded, installed and updated via pypi if uploaded ([Upload to pypi](#upload-to-pypi)).<br>
+```pip install ioiocore```<br>
+```pip install ioiocore --upgrade```
+
+### Uninstall
+```pip uninstall ioiocore```
+
+## Run tests
+Go to project base folder and execute:<br>
+```python ./test/run_all.py```

ioiocore-0.0.1/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2024 g.tec medical engineering GmbH
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

ioiocore-0.0.1/MANIFEST.in

@@ -0,0 +1 @@
+include *.md *.pdf LICENSE .\\ioiocore\\requirements.txt CHANGELOG

ioiocore-0.0.1/PKG-INFO

@@ -0,0 +1,37 @@
+Metadata-Version: 2.1
+Name: ioiocore
+Version: 0.0.1
+Summary: A realtime processing framework for python
+Home-page: https://www.gtec.at
+Author: g.tec medical engineering GmbH
+Author-email: support@gtec.at
+License: MIT
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# IOIOCore
+
+IOIOCore is general purpose data propagation framework for realtime applications written in python.
+
+# Node
+A node is a individual unit or component that performs a specific task in a larger system. Each node typically has inputs, outputs, and processing logic that transforms data as it moves through the system. Nodes are interconnected to form a pipeline where the output of one node becomes the input of another, enabling complex workflows or data transformations.
+
+## Input and Output Ports
+Nodes receive data through input ports and send processed data out through output ports. The number and type of ports may vary depending on the node's purpose.
+
+## Processing Logic
+Each node contains specific logic or algorithms to process the data it receives. This could range from basic transformations (like filtering or mapping) to more complex operations (like aggregation or machine learning inference).
+
+Create specific nodes by deriving from the Node parent class. Define input and output ports and implement the processing logic for child objects.
+
+# Sources
+
+TBD
+
+# Pipeline
+
+TBD

ioiocore-0.0.1/README.md

@@ -0,0 +1,22 @@
+# IOIOCore
+
+IOIOCore is general purpose data propagation framework for realtime applications written in python.
+
+# Node
+A node is a individual unit or component that performs a specific task in a larger system. Each node typically has inputs, outputs, and processing logic that transforms data as it moves through the system. Nodes are interconnected to form a pipeline where the output of one node becomes the input of another, enabling complex workflows or data transformations.
+
+## Input and Output Ports
+Nodes receive data through input ports and send processed data out through output ports. The number and type of ports may vary depending on the node's purpose.
+
+## Processing Logic
+Each node contains specific logic or algorithms to process the data it receives. This could range from basic transformations (like filtering or mapping) to more complex operations (like aggregation or machine learning inference).
+
+Create specific nodes by deriving from the Node parent class. Define input and output ports and implement the processing logic for child objects.
+
+# Sources
+
+TBD
+
+# Pipeline
+
+TBD

ioiocore-0.0.1/ioiocore/__init__.py

@@ -0,0 +1,22 @@
+# -*- coding: utf-8 -*-
+"""
+ioiocore
+-------
+
+A realtime processing framework for python
+
+:copyright: (c) 2024 g.tec medical engineering GmbH
+
+"""
+
+# compatibility
+from __future__ import absolute_import, division, print_function
+
+# get version
+from .__version__ import __version__
+
+# allow lazy loading
+#pattern
+from .pattern.node import Node
+from .pattern.sources import ManualSource, FixedFrequencySource, DeviceSource, FrequencyGenerator
+from .pattern.pipeline import Pipeline

ioiocore-0.0.1/ioiocore/__version__.py

@@ -0,0 +1,12 @@
+# -*- coding: utf-8 -*-
+"""
+ioiocore.version
+---------------
+
+Version tracker.
+
+:copyright: (c) 2024 g.tec medical engineering
+"""
+
+VERSION = (0, 0, 1)
+__version__ = ".".join(map(str, VERSION))

ioiocore-0.0.1/ioiocore/pattern/frequency_generator.py

@@ -0,0 +1,44 @@
+import time
+import threading
+from abc import ABC, abstractmethod
+
+class FrequencyGenerator(ABC):
+    def __init__(self, frequencyHz : float):
+        self.__realtimeClockRunning = False
+        self.__realtimeClockThread = None
+        self.__targetDt = 1000/frequencyHz
+        self.__timestampStart = None
+        self.__timestamp = 0
+        self.__samplesSent = 0
+
+    def __del__(self):
+        self.stop()
+
+    def start(self):
+        if not self.__realtimeClockRunning:
+            self.__realtimeClockRunning = True
+            self.__realtimeClockThread = threading.Thread(target=self.__updateThread_DoWork, daemon=True)
+            self.__realtimeClockThread.start()
+           
+    def stop(self):
+        if self.__realtimeClockRunning:
+            self.__realtimeClockRunning = False
+            self.__realtimeClockThread.join(500)
+
+    def __updateThread_DoWork(self):
+        if self.__timestampStart is None:
+            self.__timestampStart = time.time()*1000
+        while self.__realtimeClockRunning:
+            self.__timestamp = time.time()*1000 - self.__timestampStart
+
+            if self.__samplesSent >= round(self.__timestamp / self.__targetDt):
+                time.sleep(self.__targetDt /1000)
+            else:
+                samplesToSend = round(self.__timestamp / self.__targetDt) - self.__samplesSent
+                for i in range(0, samplesToSend):
+                    self.update()
+                    self.__samplesSent += 1
+
+    @abstractmethod
+    def update(self):
+        pass

ioiocore-0.0.1/ioiocore/pattern/node.py

@@ -0,0 +1,165 @@
+from .src.i_node import INode
+from .src.o_node import ONode
+from abc import abstractmethod
+import json
+
+class Node():
+    class __INode(INode):
+        def __init__(self, updateFcn, inputPorts = None ):
+            super().__init__(inputPorts)
+            self.__updateFcn = updateFcn
+
+        def __del__(self):
+            self.__updateFcn = None
+            super().__del__()
+
+        def update(self):
+            data = []
+            for i in range(0,len(self.InputPorts)):
+                if(self.InputPorts[i].DataCount > 0):
+                    data.append(self.InputPorts[i].read())
+                else:
+                    data.append(None)
+            self.__updateFcn(data)
+
+    class __ONode(ONode):
+        def __init__(self, outputPorts = None ):
+            super().__init__(outputPorts)
+
+        def __del__(self):
+            super().__del__()
+
+        def write_internal(self, data):
+            if len(data) is not len(self.OutputPorts):
+                raise ValueError('Number of OutputPorts do not match received data.')
+            
+            for i in range(0,len(data)):
+                self.write(i, data[i])
+
+    def __init__(self, inputPorts : list[str] = None , outputPorts : list[str] = None):
+        if inputPorts is None and outputPorts is None:
+            raise ValueError('Node without inputs and outputs is not allowed. \'inputPorts\' and/or \'outputPorts\' must be defined.')
+        if inputPorts is not None:
+            duplicates = self.__checkForDuplicates(inputPorts)
+            if duplicates:
+                raise ValueError(f'Duplicate input port names found {duplicates}. Every port must feature a unique name')
+        if outputPorts is not None:
+            duplicates = self.__checkForDuplicates(outputPorts)
+            if duplicates:
+                raise ValueError(f'Duplicate input port names found {duplicates}. Every port must feature a unique name')
+        if inputPorts is not None and outputPorts is not None:
+            duplicates = self.__checkForDuplicates(outputPorts + inputPorts)
+            if len(duplicates) > 0:
+                raise ValueError(f'Duplicate port names found {duplicates}. Every port must feature a unique name')
+        self.__inode = None
+        self.__onode = None
+        if inputPorts is not None:
+            self.__inode = self.__INode(self.__update, inputPorts)
+        if outputPorts is not None:
+            self.__onode = self.__ONode(outputPorts)
+
+    def __checkForDuplicates(self, l):
+        seen = set()
+        duplicates = set()
+        for item in l:
+            if item in seen:
+                duplicates.add(item)
+            else:
+                seen.add(item)
+        return duplicates
+    
+    def __del__(self):
+        pass
+
+    def __dict__(self):
+        oPorts = {}
+        iPorts = {}
+        if self.__onode is not None:
+            cnt = 0
+            for op in self.__onode.OutputPorts:
+                oPorts[cnt] = op.PortName
+                cnt+=1
+        if self.__inode is not None:
+            cnt = 0
+            for ip in self.__inode.InputPorts:
+                iPorts[cnt] = ip.PortName
+                cnt+=1
+
+        return {
+            "name" : self.__class__.__name__,
+            "outputs" : oPorts,
+            "inputs" : iPorts
+        }
+
+    def __repr__(self):
+        return json.dumps(self.__dict__(), indent=4)
+
+    def __get_output_port_index_by_name(self, node, outputPortName):
+        port = None
+        for p in node.__onode.OutputPorts:
+            if p.PortName == outputPortName:
+                port = p
+                break
+        if port is None:
+            raise ValueError(f'Could not find input port named \'{outputPortName}\'.')
+        
+        return node.__onode.OutputPorts.index(port)
+
+    def __get_input_port_index_by_name(self, node, inputPortName):
+        port = None
+        for p in node.__inode.InputPorts:
+            if p.PortName == inputPortName:
+                port = p
+                break
+            
+        if port is None:
+            raise ValueError(f'Could not find output port named\'{inputPortName}\'.')
+        
+        return node.__inode.InputPorts.index(port)
+
+    def __update(self, data):
+        if self.__inode is None:
+            raise TypeError('No input ports defined. Update is only available if input ports are defined.')
+        
+        dataOut = self.update(data)
+
+        if self.__onode is not None:
+            self.write(dataOut)
+
+    def write(self, data):
+        if self.__onode is None:
+            raise TypeError('No output ports defined. Write is only available if output ports are defined.')
+        
+        self.__onode.write_internal(data)
+    
+    def connect_by_name(self, outputPortName, node, inputPortName):
+        if self.__onode is None:
+            raise ValueError('Connect is only available if outputs are defined.')
+        io = self.__get_output_port_index_by_name(self, outputPortName)
+        ii = self.__get_input_port_index_by_name(node, inputPortName)
+        self.connect_by_id(io, node, ii)
+
+    def disconnect_by_name(self, outputPortName, node, inputPortName):
+        if self.__onode is None:
+            raise ValueError('Connect is only available if outputs are defined.')
+        io = self.__get_output_port_index_by_name(self, outputPortName)
+        ii = self.__get_input_port_index_by_name(node, inputPortName)
+        self.disconnect_by_id(io, node, ii)
+
+    def connect_by_id(self, outputPortId, node, inputPortId):
+        if self.__onode is None:
+            raise ValueError('Connect is only available if outputs are defined.')
+        if len(self.__onode.OutputPorts) <= outputPortId:
+            raise ValueError('Port {outputPortId} is out of bounds')
+        self.__onode.connect(outputPortId, node.__inode.InputPorts[inputPortId])
+
+    def disconnect_by_id(self, outputPortId, node, inputPortId):
+        if self.__onode is None:
+            raise ValueError('Disconnect is only available if outputs are defined.')
+        if len(self.__onode.OutputPorts) <= outputPortId:
+            raise ValueError('Port {outputPortId} is out of bounds')
+        self.__onode.disconnect(outputPortId, node.__inode.InputPorts[inputPortId])
+
+    @abstractmethod
+    def update(self, data):
+        pass

ioiocore-0.0.1/ioiocore/pattern/pipeline.py

@@ -0,0 +1,96 @@
+from .node import Node
+from .sources import *
+
+class Pipeline:
+    def __init__(self):
+        self.__nodes = {} 
+
+    def add_node(self, name: str, node: Node):
+        if name in self.__nodes.keys():
+            raise ValueError(f"A node with the name '{name}' already exists.")
+        self.__nodes[name] = node
+
+    def remove_node(self, name: str):
+        if name not in self.__nodes.keys():
+            raise KeyError(f"No node found with the name '{name}'.")
+        del self.__nodes[name]
+    
+    def __get_ports(self, n1 : Node):
+        return [n1.__dict__()['inputs'], n1.__dict__()['outputs']]
+
+    def __n1_is_output(self, n1 : Node, port_name1 : str, n2 : Node, port_name2 : str ):
+        n1p = self.__get_ports(n1)
+        n2p = self.__get_ports(n2)
+        if len(n1p[1]) == 0 and len(n2p[1]) == 0 :
+            raise ValueError(f"No outputs found for {n1.__class__.__name__} or {n2.__class__.__name__}.")
+        if len(n1p[0]) == 0 and len(n2p[0])== 0 :
+            raise ValueError(f"No inputs found for {n1.__class__.__name__} or {n2.__class__.__name__}.")
+        
+        n1_isOutput = False
+        if port_name1 in n1p[0].values():
+            n1_isOutput = False
+        if port_name1 in n1p[1].values():
+            n1_isOutput = True
+        if port_name1 not in n1p[0].values() and port_name1 not in n1p[1].values():
+            raise ValueError(f'Port {port_name1} not found in {n1.__class__.__name__}.')
+        if port_name2 not in n2p[0].values() and port_name2 not in n2p[1].values():
+            raise ValueError(f'Port {port_name2} not found in {n2.__class__.__name__}.')
+        return n1_isOutput
+
+    def __check_nodes(self, node_name1 : str, node_name2 : str):
+        n1 : Node = None
+        try:
+            n1 : Node = self.__nodes[node_name1] 
+        except:
+            raise ValueError(f'Pipeline does not contain \'{node_name1}\'.')
+        try:
+            n2 : Node = self.__nodes[node_name2]
+        except:
+            raise ValueError(f'Pipeline does not contain \'{node_name2}\'.')
+        return [n1, n2]
+
+    def connect(self, node_name1 : str, port_name1 : str, node_name2 : str, port_name2 : str):
+        nodes = self.__check_nodes(node_name1, node_name2)
+        ports = [port_name1, port_name2]
+        if self.__n1_is_output(nodes[0], port_name1, nodes[1], port_name2):
+            nodes[0].connect_by_name(ports[0], nodes[1], ports[1])
+        else:
+            nodes[1].connect_by_name(ports[1], nodes[0], ports[0])
+
+    def disconnect(self, node_name1 : str, port_name1 : str, node_name2 : str, port_name2 : str):
+        nodes = self.__check_nodes(node_name1, node_name2)
+        ports = [port_name1, port_name2]
+        if self.__n1_is_output(nodes[0], port_name1, nodes[1], port_name2):
+            nodes[0].disconnect_by_name(ports[0], nodes[1], ports[1])
+        else:
+            nodes[1].disconnect_by_name(ports[1], nodes[0], ports[0])
+
+    def get_source_nodes(self):
+        sources : list[Node] = []
+        for node in self.__nodes.values():
+            np = self.__get_ports(node)
+            if len(np[0]) == 0 and len(np[1]) > 0:
+                sources.append(node)    
+        if len(sources) is None:
+            raise ValueError(f'No source node was found.')
+        return sources
+        
+    def start(self):
+        found = False
+        for node in self.__nodes.values():
+            if hasattr(node, 'start'):
+                found = True
+                node.start()
+
+        if found == False:
+            raise ValueError(f'Could not find a source providing a \'start\' method.')
+
+    def stop(self):
+        found = False
+        for node in self.__nodes.values():
+            if hasattr(node, 'stop'):
+                found = True
+                node.stop()
+
+        if found == False:
+            raise ValueError(f'Could not find a source providing a \'stop\' method.')

ioiocore-0.0.1/ioiocore/pattern/sources.py

@@ -0,0 +1,64 @@
+from .node import Node
+from .frequency_generator import FrequencyGenerator
+from abc import abstractmethod
+
+class ManualSource(Node):
+    def __init__(self, outputPorts):
+        super().__init__(inputPorts=None, outputPorts=outputPorts)
+
+    def __del__(self):
+        super().__del__()   
+
+class FixedFrequencySource(Node, FrequencyGenerator):
+    def __init__(self, outputPorts, samplingRateHz):
+        super().__init__(inputPorts=None, outputPorts=outputPorts)
+        super(Node, self).__init__(samplingRateHz)
+
+    def __del__(self):
+        super().__del__()
+        super(Node, self).__del__()  
+
+    def update(self):
+        self.write(self.generate_sample())
+
+    @abstractmethod
+    def generate_sample(self):
+        pass
+
+class DeviceSource(Node):
+    def __init__(self, outputPorts, serial):
+        super().__init__(inputPorts=None, outputPorts=outputPorts)
+
+    def __del__(self):
+        super().__del__()
+
+    @staticmethod
+    @abstractmethod
+    def start_scanning():
+        pass
+
+    @staticmethod
+    @abstractmethod
+    def stop_scanning():
+        pass
+
+    @staticmethod
+    @abstractmethod
+    def get_available_devices():
+        pass
+
+    @abstractmethod
+    def open(self, serial):
+        pass
+
+    @abstractmethod
+    def close(self):
+        pass
+
+    @abstractmethod
+    def start(self):
+        pass
+    
+    @abstractmethod
+    def stop(self):
+        pass

ioiocore-0.0.1/ioiocore/pattern/src/i_node.py

@@ -0,0 +1,82 @@
+from .i_port import IPort
+import threading
+from abc import ABC, abstractmethod
+from enum import Enum
+import time
+
+class INode(ABC):
+    class UpdateMode(Enum):
+        Synchronized = 1 #update when all input ports delivered data
+        Asynchron = 2    #update when any input ports delivered data
+
+    def __init__(self, inputPort : list[str]):
+        self.InputPorts : list[IPort] = []
+        for p in inputPort:
+            port = IPort(p)
+            port.set_data_available_eventhandler(self.__on_data_available)
+            self.InputPorts.append(port)
+        self.NodeUpdateMode = self.UpdateMode.Synchronized
+
+        self.__event : threading.Event = threading.Event()
+        self.__updateThreadRunning = False
+        self.__updateThread = None
+        self.__updateCnt = 0
+        self.__totalTimeMs = 0
+        self.__updateTimeMs = 0
+        self.__start()
+
+    def __del__(self):
+        self.__stop()
+
+    def __start(self):
+        if not self.__updateThreadRunning:
+            self.__updateThreadRunning = True
+            self.__updateThread = threading.Thread(target=self.__updateThread_DoWork, daemon=True)
+            self.__updateThread.start()
+           
+    def __stop(self):
+        if self.__updateThreadRunning:
+            self.__updateThreadRunning = False
+            self.__event.set()
+            self.__updateThread.join()
+            self.__updateThread = None
+
+    def __on_data_available(self):
+        if self.NodeUpdateMode is self.UpdateMode.Asynchron:
+            self.__event.set()
+        else:
+            allPortsAcquired = True
+            for port in self.InputPorts:
+                if port.DataCount <= 0:
+                    allPortsAcquired = False
+                    break
+            if allPortsAcquired:
+                self.__event.set()
+
+    def __updateThread_DoWork(self):
+        try:
+            self.__event.wait()
+            self.__event.clear()
+            while self.__updateThreadRunning:
+                if self.__updateThreadRunning:
+                    updateCnt = -1
+                    for port in self.InputPorts:
+                        if updateCnt == -1 or port.DataCount <= updateCnt:
+                            updateCnt = port.DataCount
+                    for i in range(updateCnt):
+                        start = time.time()
+                        self.update()
+                        end = time.time()
+                        self.__updateCnt += 1
+                        self.__totalTimeMs += (end - start)*1000
+                        self.__updateTimeMs = self.__totalTimeMs / self.__updateCnt
+
+                self.__event.wait()
+                self.__event.clear()
+                    
+        except Exception as e:
+            self.__stop()
+
+    @abstractmethod
+    def update(self):
+        pass

ioiocore-0.0.1/ioiocore/pattern/src/i_port.py

@@ -0,0 +1,32 @@
+import queue
+
+class IPort:
+    def __init__(self, portName : str):
+        self.__queue : queue.Queue = queue.Queue()
+        self.DataCount : int = 0
+        self.PortName = portName
+        self.IsConnected = False
+        self.__eventHandler : function
+        self.__readCnt = 0
+    
+    def write(self, data):
+        self.__queue.put(data, block=True, timeout=None)
+        self.DataCount = self.__queue.qsize()
+        if self.__eventHandler is not None:
+            self.__eventHandler()
+
+    def read(self):
+        if self.DataCount > 0:
+            try:
+                self.__readCnt += 1
+                return self.__queue.get(block=True, timeout=None)
+            except:
+                return None
+        else:
+            return None 
+
+    def set_data_available_eventhandler(self, handler):
+        self.__eventHandler = handler
+
+    def remove_data_available_eventhandler(self, handler):
+        self.__eventHandler = None