learning-loop-node 0.16.1__tar.gz → 0.17.0__tar.gz

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: learning-loop-node
-Version: 0.16.1
+Version: 0.17.0
 Summary: Python Library for Nodes which connect to the Zauberzeug Learning Loop
 Home-page: https://github.com/zauberzeug/learning_loop_node
 License: MIT
@@ -85,34 +85,48 @@ from learning_loop_node/learning_loop_node
 
 Detector Nodes are normally deployed on edge devices like robots or machinery but can also run in the cloud to provide backend services for an app or similar. These nodes register themself at the Learning Loop. They provide REST and Socket.io APIs to run inference on images. The processed images can automatically be used for active learning: e.g. uncertain predictions will be send to the Learning Loop.
 
-### Running Inference
+### Inference API
 
 Images can be send to the detector node via socketio or rest.
-The later approach can be used via curl,
+Via **REST** you may provide the following parameters:
+
+- `autoupload`: configures auto-submission to the learning loop; `filtered` (default), `all`, `disabled`
+- `camera_id`: a camera identifier (string) used to improve the autoupload filtering
+- `tags`: comma separated list of tags to add to the image in the learning loop
+- `source`: optional source identifier (str) for the image (e.g. a robot id)
+- `autoupload`: configures auto-submission to the learning loop; `filtered` (default), `all`, `disabled`
+- `creation_date`: optional creation date (str) for the image in isoformat (e.g. `2023-01-30T12:34:56`)
 
 Example usage:
 
-`curl --request POST -F 'file=@test.jpg' localhost:8004/detect`
+`curl --request POST -F 'file=@test.jpg' -H 'autoupload: all' -H 'camera_id: front_cam' localhost:8004/detect`
+
+To use the **SocketIO** inference EPs, the caller needs to connect to the detector node's SocketIO server and emit the `detect` or `batch_detect` event with the image data and image metadata.
+Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
+
+### Upload API
 
-Where 8804 is the specified port in this example.
-You can additionally provide the following camera parameters:
+The detector has a **REST** endpoint to upload images (and detections) to the Learning Loop. The endpoint takes a POST request with one or multiple images. The images are expected to be in jpg format. The following optional parameters may be set via headers:
 
-- `autoupload`: configures auto-submission to the learning loop; `filtered` (default), `all`, `disabled` (example curl parameter `-H 'autoupload: all'`)
-- `camera-id`: a string which groups images for submission together (example curl parameter `-H 'camera-id: front_cam'`)
+- `source`: optional source identifier (str) for the image (e.g. a robot id)
+- `creation_date`: optional creation date (str) for the image in isoformat (e.g. `2023-01-30T12:34:56`)
+- `upload_priority`: A boolean flag to prioritize the upload (defaults to False)
 
-To use the socketio interface, the caller needs to connect to the detector node's socketio server and emit the `detect` or `batch_detect` event with the image data and image metadata. Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
+Example:
 
-The detector also has a sio **upload endpoint** that can be used to upload images and detections to the learning loop. The function receives a json dictionary, with the following entries:
+`curl -X POST -F 'files=@test.jpg' "http://localhost:/upload"`
+
+The detector also has a **SocketIO** upload endpoint that can be used to upload images and detections to the learning loop. The function receives a json dictionary, with the following entries:
 
 - `image`: the image data in jpg format
-- `tags`: a list of strings. If not provided the tag is `picked_by_system`
-- `detections`: a dictionary representing the detections. UUIDs for the classes are automatically determined based on the category names. This field is optional. If not provided, no detections are uploaded.
-- `source`: optional source identifier for the image
-- `creation_date`: optional creation date for the image
+
+- `metadata`: a dictionary representing the image metadata. If metadata contains detections and/or annotations, UUIDs for the classes are automatically determined based on the category names. Metadata should follow the schema of the `ImageMetadata` data class.
 - `upload_priority`: boolean flag to prioritize the upload (defaults to False)
 
 The endpoint returns None if the upload was successful and an error message otherwise.
 
+For both ways to upload an image, the tag `picked_by_system` is automatically added to the image metadata.
+
 ### Changing the model versioning mode
 
 The detector can be configured to one of the following behaviors:
@@ -164,12 +178,6 @@ The outbox mode can also be queried via:
 - HTTP: `curl http://localhost/outbox_mode`
 - SocketIO: `sio.emit('get_outbox_mode')`
 
-### Explicit upload
-
-The detector has a REST endpoint to upload images (and detections) to the Learning Loop. The endpoint takes a POST request with the image and optionally the detections. The image is expected to be in jpg format. The detections are expected to be a json dictionary. Example:
-
-`curl -X POST -F 'files=@test.jpg' "http://localhost:/upload"`
-
 ## Trainer Node
 
 Trainers fetch the images and anntoations from the Learning Loop to train new models.
@@ -184,7 +192,7 @@ A Conveter Node converts models from one format into another.
 
 ...
 
-#### Test operability
+### Test operability
 
 Assumend there is a Converter Node which converts models of format 'format_a' into 'format_b'.
 Upload a model with

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/README.md

@@ -45,34 +45,48 @@ from learning_loop_node/learning_loop_node
 
 Detector Nodes are normally deployed on edge devices like robots or machinery but can also run in the cloud to provide backend services for an app or similar. These nodes register themself at the Learning Loop. They provide REST and Socket.io APIs to run inference on images. The processed images can automatically be used for active learning: e.g. uncertain predictions will be send to the Learning Loop.
 
-### Running Inference
+### Inference API
 
 Images can be send to the detector node via socketio or rest.
-The later approach can be used via curl,
+Via **REST** you may provide the following parameters:
+
+- `autoupload`: configures auto-submission to the learning loop; `filtered` (default), `all`, `disabled`
+- `camera_id`: a camera identifier (string) used to improve the autoupload filtering
+- `tags`: comma separated list of tags to add to the image in the learning loop
+- `source`: optional source identifier (str) for the image (e.g. a robot id)
+- `autoupload`: configures auto-submission to the learning loop; `filtered` (default), `all`, `disabled`
+- `creation_date`: optional creation date (str) for the image in isoformat (e.g. `2023-01-30T12:34:56`)
 
 Example usage:
 
-`curl --request POST -F 'file=@test.jpg' localhost:8004/detect`
+`curl --request POST -F 'file=@test.jpg' -H 'autoupload: all' -H 'camera_id: front_cam' localhost:8004/detect`
+
+To use the **SocketIO** inference EPs, the caller needs to connect to the detector node's SocketIO server and emit the `detect` or `batch_detect` event with the image data and image metadata.
+Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
+
+### Upload API
 
-Where 8804 is the specified port in this example.
-You can additionally provide the following camera parameters:
+The detector has a **REST** endpoint to upload images (and detections) to the Learning Loop. The endpoint takes a POST request with one or multiple images. The images are expected to be in jpg format. The following optional parameters may be set via headers:
 
-- `autoupload`: configures auto-submission to the learning loop; `filtered` (default), `all`, `disabled` (example curl parameter `-H 'autoupload: all'`)
-- `camera-id`: a string which groups images for submission together (example curl parameter `-H 'camera-id: front_cam'`)
+- `source`: optional source identifier (str) for the image (e.g. a robot id)
+- `creation_date`: optional creation date (str) for the image in isoformat (e.g. `2023-01-30T12:34:56`)
+- `upload_priority`: A boolean flag to prioritize the upload (defaults to False)
 
-To use the socketio interface, the caller needs to connect to the detector node's socketio server and emit the `detect` or `batch_detect` event with the image data and image metadata. Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
+Example:
 
-The detector also has a sio **upload endpoint** that can be used to upload images and detections to the learning loop. The function receives a json dictionary, with the following entries:
+`curl -X POST -F 'files=@test.jpg' "http://localhost:/upload"`
+
+The detector also has a **SocketIO** upload endpoint that can be used to upload images and detections to the learning loop. The function receives a json dictionary, with the following entries:
 
 - `image`: the image data in jpg format
-- `tags`: a list of strings. If not provided the tag is `picked_by_system`
-- `detections`: a dictionary representing the detections. UUIDs for the classes are automatically determined based on the category names. This field is optional. If not provided, no detections are uploaded.
-- `source`: optional source identifier for the image
-- `creation_date`: optional creation date for the image
+
+- `metadata`: a dictionary representing the image metadata. If metadata contains detections and/or annotations, UUIDs for the classes are automatically determined based on the category names. Metadata should follow the schema of the `ImageMetadata` data class.
 - `upload_priority`: boolean flag to prioritize the upload (defaults to False)
 
 The endpoint returns None if the upload was successful and an error message otherwise.
 
+For both ways to upload an image, the tag `picked_by_system` is automatically added to the image metadata.
+
 ### Changing the model versioning mode
 
 The detector can be configured to one of the following behaviors:
@@ -124,12 +138,6 @@ The outbox mode can also be queried via:
 - HTTP: `curl http://localhost/outbox_mode`
 - SocketIO: `sio.emit('get_outbox_mode')`
 
-### Explicit upload
-
-The detector has a REST endpoint to upload images (and detections) to the Learning Loop. The endpoint takes a POST request with the image and optionally the detections. The image is expected to be in jpg format. The detections are expected to be a json dictionary. Example:
-
-`curl -X POST -F 'files=@test.jpg' "http://localhost:/upload"`
-
 ## Trainer Node
 
 Trainers fetch the images and anntoations from the Learning Loop to train new models.
@@ -144,7 +152,7 @@ A Conveter Node converts models from one format into another.
 
 ...
 
-#### Test operability
+### Test operability
 
 Assumend there is a Converter Node which converts models of format 'format_a' into 'format_b'.
 Upload a model with

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/data_classes/__init__.py

@@ -1,4 +1,9 @@
-from .annotations import AnnotationData, SegmentationAnnotation, ToolOutput, UserInput
+from .annotation_data import (
+    AnnotationData,
+    SegmentationAnnotation,
+    ToolOutput,
+    UserInput,
+)
 from .detections import (
     BoxDetection,
     ClassificationDetection,

learning_loop_node-0.17.0/learning_loop_node/data_classes/annotations.py

@@ -0,0 +1,44 @@
+
+import sys
+from dataclasses import dataclass, field
+from typing import Optional
+
+KWONLY_SLOTS = {'kw_only': True, 'slots': True} if sys.version_info >= (3, 10) else {}
+
+
+@dataclass(**KWONLY_SLOTS)
+class BoxAnnotation():
+    """Coordinates according to COCO format. x,y is the top left corner of the box.
+    x increases to the right, y increases downwards.
+    """
+    category_name: str = field(metadata={'description': 'Category name'})
+    x: int = field(metadata={'description': 'X coordinate (left to right)'})
+    y: int = field(metadata={'description': 'Y coordinate (top to bottom)'})
+    width: int = field(metadata={'description': 'Width'})
+    height: int = field(metadata={'description': 'Height'})
+    category_id: Optional[str] = field(default=None, metadata={'description': 'Category UUID'})
+
+    def __str__(self):
+        return f'x:{int(self.x)} y: {int(self.y)}, w: {int(self.width)} h: {int(self.height)} -> {self.category_name}'
+
+
+@dataclass(**KWONLY_SLOTS)
+class PointAnnotation():
+    """Coordinates according to COCO format. x,y is the center of the point.
+    x increases to the right, y increases downwards."""
+    category_name: str = field(metadata={'description': 'Category name'})
+    x: float = field(metadata={'description': 'X coordinate (right)'})
+    y: float = field(metadata={'description': 'Y coordinate (down)'})
+    category_id: Optional[str] = field(default=None, metadata={'description': 'Category UUID'})
+
+    def __str__(self):
+        return f'x:{int(self.x)} y: {int(self.y)}, -> {self.category_name}'
+
+
+@dataclass(**KWONLY_SLOTS)
+class ClassificationAnnotation():
+    category_name: str = field(metadata={'description': 'Category name'})
+    category_id: Optional[str] = field(default=None, metadata={'description': 'Category UUID'})
+
+    def __str__(self):
+        return f'-> {self.category_name}'

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/data_classes/detections.py

@@ -9,10 +9,6 @@ import numpy as np
 KWONLY_SLOTS = {'kw_only': True, 'slots': True} if sys.version_info >= (3, 10) else {}
 
 
-def current_datetime():
-    return datetime.now().isoformat(sep='_', timespec='milliseconds')
-
-
 @dataclass(**KWONLY_SLOTS)
 class BoxDetection():
     """Coordinates according to COCO format. x,y is the top left corner of the box.
@@ -25,7 +21,7 @@ class BoxDetection():
     height: int = field(metadata={'description': 'Height'})
     model_name: str = field(metadata={'description': 'Model name'})
     confidence: float = field(metadata={'description': 'Confidence'})
-    category_id: Optional[str] = field(default=None, metadata={'description': 'Category ID'})
+    category_id: Optional[str] = field(default=None, metadata={'description': 'Category UUID'})
 
     def intersection_over_union(self, other_detection: 'BoxDetection') -> float:
         # https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/
@@ -59,7 +55,7 @@ class PointDetection():
     y: float = field(metadata={'description': 'Y coordinate (down)'})
     model_name: str = field(metadata={'description': 'Model name'})
     confidence: float = field(metadata={'description': 'Confidence'})
-    category_id: Optional[str] = field(default=None, metadata={'description': 'Category ID'})
+    category_id: Optional[str] = field(default=None, metadata={'description': 'Category UUID'})
 
     def distance(self, other: 'PointDetection') -> float:
         return np.sqrt((other.x - self.x)**2 + (other.y - self.y)**2)
@@ -73,7 +69,7 @@ class ClassificationDetection():
     category_name: str = field(metadata={'description': 'Category name'})
     model_name: str = field(metadata={'description': 'Model name'})
     confidence: float = field(metadata={'description': 'Confidence'})
-    category_id: Optional[str] = field(default=None, metadata={'description': 'Category ID'})
+    category_id: Optional[str] = field(default=None, metadata={'description': 'Category UUID'})
 
     def __str__(self):
         return f'c: {self.confidence:.2f} -> {self.category_name}'

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/data_classes/image_metadata.py

@@ -4,7 +4,13 @@ from dataclasses import dataclass, field
 from datetime import datetime
 from typing import List, Optional
 
-from .detections import BoxDetection, ClassificationDetection, PointDetection, SegmentationDetection
+from .annotations import BoxAnnotation, ClassificationAnnotation, PointAnnotation
+from .detections import (
+    BoxDetection,
+    ClassificationDetection,
+    PointDetection,
+    SegmentationDetection,
+)
 
 # pylint: disable=too-many-instance-attributes
 
@@ -25,6 +31,14 @@ class ImageMetadata():
         'description': 'List of segmentation detections'})
     classification_detections: List[ClassificationDetection] = field(default_factory=list, metadata={
         'description': 'List of classification detections'})
+
+    box_annotations: List[BoxAnnotation] = field(default_factory=list, metadata={
+        'description': 'List of box annotations'})
+    point_annotations: List[PointAnnotation] = field(default_factory=list, metadata={
+        'description': 'List of point annotations'})
+    classification_annotation: Optional[ClassificationAnnotation] = field(default=None, metadata={
+        'description': 'Classification annotation'})
+
     tags: List[str] = field(default_factory=list, metadata={
         'description': 'List of tags'})
 

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/detector/detector_logic.py

@@ -42,18 +42,18 @@ class DetectorLogic():
     def init(self):
         """Called when a (new) model was loaded. Initialize the model. Model information available via `self.model_info`"""
 
-    def evaluate_with_all_info(self, image: bytes, tags: List[str], source: Optional[str] = None, creation_date: Optional[str] = None) -> ImageMetadata:  # pylint: disable=unused-argument
-        """Called by the detector node when an image should be evaluated (REST or SocketIO).
-        Tags, source come from the caller and may be used in this function. 
-        By default, this function simply calls `evaluate`"""
-        return self.evaluate(image)
-
     @abstractmethod
-    def evaluate(self, image: bytes) -> ImageMetadata:
+    def evaluate(self, image: bytes) -> ImageMetadata:  # pylint: disable=unused-argument
         """Evaluate the image and return the detections.
-        The object should return empty detections if it is not initialized"""
+
+        Called by the detector node when an image should be evaluated (REST or SocketIO).
+        The resulting detections should be stored in the ImageMetadata.
+        Tags stored in the ImageMetadata will be uploaded to the learning loop.
+        The function should return empty metadata if the detector is not initialized."""
 
     @abstractmethod
     def batch_evaluate(self, images: List[bytes]) -> ImagesMetadata:
         """Evaluate a batch of images and return the detections.
-        The object should return empty detections if it is not initialized"""
+        The resulting detections per image should be stored in the ImagesMetadata.
+        Tags stored in the ImagesMetadata will be uploaded to the learning loop.
+        The function should return empty metadata if the detector is not initialized."""

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/detector/detector_node.py

@@ -8,6 +8,11 @@ from dataclasses import asdict
 from datetime import datetime
 from typing import Dict, List, Optional
 
+try:
+    from typing import Literal
+except ImportError:  # Python <= 3.8
+    from typing_extensions import Literal  # type: ignore
+
 import socketio
 from dacite import from_dict
 from fastapi.encoders import jsonable_encoder
@@ -223,10 +228,10 @@ class DetectorNode(Node):
             try:
                 det = await self.get_detections(
                     raw_image=data['image'],
-                    camera_id=data.get('camera-id', None) or data.get('mac', None),
+                    camera_id=data.get('camera_id', None),
                     tags=data.get('tags', []),
                     source=data.get('source', None),
-                    autoupload=data.get('autoupload', None),
+                    autoupload=data.get('autoupload', 'filtered'),
                     creation_date=data.get('creation_date', None)
                 )
                 if det is None:
@@ -245,9 +250,9 @@ class DetectorNode(Node):
                 det = await self.get_batch_detections(
                     raw_images=data['images'],
                     tags=data.get('tags', []),
-                    camera_id=data.get('camera-id', None) or data.get('mac', None),
+                    camera_id=data.get('camera_id', None),
                     source=data.get('source', None),
-                    autoupload=data.get('autoupload', None),
+                    autoupload=data.get('autoupload', 'filtered'),
                     creation_date=data.get('creation_date', None)
                 )
                 if det is None:
@@ -296,27 +301,30 @@ class DetectorNode(Node):
 
         @self.sio.event
         async def upload(sid, data: Dict) -> Dict:
-            """Upload an image with detections"""
+            """Upload a single image with metadata to the learning loop.
 
+            The data dict must contain:
+            - image: The image bytes to upload
+            - metadata: The metadata for the image (optional)
+            """
             self.log.debug('Processing upload via socketio.')
-            detection_data = data.get('detections', {})
-            if detection_data and self.detector_logic.model_info is not None:
+
+            metadata = data.get('metadata', None)
+            if metadata:
                 try:
-                    image_metadata = from_dict(data_class=ImageMetadata, data=detection_data)
+                    image_metadata = from_dict(data_class=ImageMetadata, data=metadata)
                 except Exception as e:
                     self.log.exception('could not parse detections')
                     return {'error': str(e)}
-                image_metadata = self.add_category_id_to_detections(self.detector_logic.model_info, image_metadata)
+                if self.detector_logic.model_info is not None:
+                    image_metadata = self.add_category_id_to_detections(self.detector_logic.model_info, image_metadata)
             else:
                 image_metadata = ImageMetadata()
 
             try:
                 await self.upload_images(
                     images=[data['image']],
-                    image_metadata=image_metadata,
-                    tags=data.get('tags', []),
-                    source=data.get('source', None),
-                    creation_date=data.get('creation_date', None),
+                    images_metadata=ImagesMetadata(items=[image_metadata]) if metadata else None,
                     upload_priority=data.get('upload_priority', False)
                 )
             except Exception as e:
@@ -506,34 +514,34 @@ class DetectorNode(Node):
                              *,
                              camera_id: Optional[str] = None,
                              source: Optional[str] = None,
-                             autoupload: Optional[str] = None,
+                             autoupload: Literal['filtered', 'all', 'disabled'],
                              creation_date: Optional[str] = None) -> ImageMetadata:
         """ Main processing function for the detector node when an image is received via REST or SocketIO.
         This function infers the detections from the image, cares about uploading to the loop and returns the detections as ImageMetadata object.
         Note: raw_image is a numpy array of type uint8, but not in the correct shape!
-        It can be converted e.g. using cv2.imdecode(raw_image, cv2.IMREAD_COLOR)"""
+        It can be converted e.g. using cv2.imdecode(np.frombuffer(image, np.uint8), cv2.IMREAD_COLOR)"""
 
         await self.detection_lock.acquire()
-        detections = await run.io_bound(self.detector_logic.evaluate_with_all_info, raw_image, tags, source, creation_date)
+        metadata = await run.io_bound(self.detector_logic.evaluate, raw_image)
+        metadata.tags.extend(tags)
+        metadata.source = source
+        metadata.created = creation_date
         self.detection_lock.release()
 
-        fix_shape_detections(detections)
-        n_bo, n_cl = len(detections.box_detections), len(detections.classification_detections)
-        n_po, n_se = len(detections.point_detections), len(detections.segmentation_detections)
+        fix_shape_detections(metadata)
+        n_bo, n_cl = len(metadata.box_detections), len(metadata.classification_detections)
+        n_po, n_se = len(metadata.point_detections), len(metadata.segmentation_detections)
         self.log.debug('Detected: %d boxes, %d points, %d segs, %d classes', n_bo, n_po, n_se, n_cl)
 
-        autoupload = autoupload or 'filtered'
-        if autoupload == 'filtered' and camera_id is not None:
-            background_tasks.create(self.relevance_filter.may_upload_detections(
-                detections, camera_id, raw_image, tags, source, creation_date
-            ))
+        if autoupload == 'filtered':
+            background_tasks.create(self.relevance_filter.may_upload_detections(metadata, camera_id, raw_image))
         elif autoupload == 'all':
-            background_tasks.create(self.outbox.save(raw_image, detections, tags, source, creation_date))
+            background_tasks.create(self.outbox.save(raw_image, metadata))
         elif autoupload == 'disabled':
             pass
         else:
             self.log.error('unknown autoupload value %s', autoupload)
-        return detections
+        return metadata
 
     async def get_batch_detections(self,
                                    raw_images: List[bytes],
@@ -541,7 +549,7 @@ class DetectorNode(Node):
                                    *,
                                    camera_id: Optional[str] = None,
                                    source: Optional[str] = None,
-                                   autoupload: Optional[str] = None,
+                                   autoupload: str = 'filtered',
                                    creation_date: Optional[str] = None) -> ImagesMetadata:
         """ Processing function for the detector node when a a batch inference is requested via SocketIO.
         This function infers the detections from all images, cares about uploading to the loop and returns the detections as a list of ImageMetadata."""
@@ -556,13 +564,10 @@ class DetectorNode(Node):
             n_po, n_se = len(detections.point_detections), len(detections.segmentation_detections)
             self.log.debug('Detected: %d boxes, %d points, %d segs, %d classes', n_bo, n_po, n_se, n_cl)
 
-            autoupload = autoupload or 'filtered'
-            if autoupload == 'filtered' and camera_id is not None:
-                background_tasks.create(self.relevance_filter.may_upload_detections(
-                    detections, camera_id, raw_image, tags, source, creation_date
-                ))
+            if autoupload == 'filtered':
+                background_tasks.create(self.relevance_filter.may_upload_detections(detections, camera_id, raw_image))
             elif autoupload == 'all':
-                background_tasks.create(self.outbox.save(raw_image, detections, tags, source, creation_date))
+                background_tasks.create(self.outbox.save(raw_image, detections))
             elif autoupload == 'disabled':
                 pass
             else:
@@ -572,24 +577,25 @@ class DetectorNode(Node):
     async def upload_images(
             self, *,
             images: List[bytes],
-            image_metadata: Optional[ImageMetadata] = None,
-            tags: Optional[List[str]] = None,
-            source: Optional[str],
-            creation_date: Optional[str],
+            images_metadata: Optional[ImagesMetadata] = None,
             upload_priority: bool = False
     ) -> None:
         """Save images to the outbox using an asyncio executor.
-        Used by SIO and REST upload endpoints."""
+        Used by SIO and REST upload endpoints.
 
-        if image_metadata is None:
-            image_metadata = ImageMetadata()
-        if tags is None:
-            tags = []
+        :param images: List of images to upload
+        :param images_metadata: Optional metadata for all images
+        :param upload_priority: Whether to upload the images with priority
+        :raises ValueError: If the number of images and number of metadata items do not match
+        """
 
-        tags.append('picked_by_system')
+        if images_metadata and len(images_metadata.items) != len(images):
+            raise ValueError('Number of images and number of metadata items do not match')
 
-        for image in images:
-            await self.outbox.save(image, image_metadata, tags, source, creation_date, upload_priority)
+        for i, image in enumerate(images):
+            image_metadata = images_metadata.items[i] if images_metadata else ImageMetadata()
+            image_metadata.tags.append('picked_by_system')
+            await self.outbox.save(image, image_metadata, upload_priority)
 
     def add_category_id_to_detections(self, model_info: ModelInformation, image_metadata: ImageMetadata):
         def find_category_id_by_name(categories: List[Category], category_name: str):
@@ -628,9 +634,9 @@ def step_into(new_dir):
         os.chdir(previous_dir)
 
 
-def fix_shape_detections(detections: ImageMetadata):
+def fix_shape_detections(metadata: ImageMetadata):
     # TODO This is a quick fix.. check how loop upload detections deals with this
-    for seg_detection in detections.segmentation_detections:
+    for seg_detection in metadata.segmentation_detections:
         if isinstance(seg_detection.shape, Shape):
             points = ','.join([str(value) for p in seg_detection.shape.points for _,
                                value in asdict(p).items()])

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/detector/inbox_filter/relevance_filter.py

@@ -9,28 +9,30 @@ class RelevanceFilter():
 
     def __init__(self, outbox: Outbox) -> None:
         self.cam_histories: Dict[str, CamObservationHistory] = {}
+        self.unknown_cam_history: CamObservationHistory = CamObservationHistory()
         self.outbox: Outbox = outbox
 
     async def may_upload_detections(self,
                                     image_metadata: ImageMetadata,
-                                    cam_id: str,
-                                    raw_image: bytes,
-                                    tags: List[str],
-                                    source: Optional[str] = None,
-                                    creation_date: Optional[str] = None) -> List[str]:
+                                    cam_id: Optional[str],
+                                    raw_image: bytes) -> List[str]:
         """Check if the detection should be uploaded to the outbox.
         If so, upload it and return the list of causes for the upload.
         """
         for group in self.cam_histories.values():
             group.forget_old_detections()
 
-        if cam_id not in self.cam_histories:
-            self.cam_histories[cam_id] = CamObservationHistory()
-        causes = self.cam_histories[cam_id].get_causes_to_upload(image_metadata)
+        if cam_id is None:
+            history = self.unknown_cam_history
+        else:
+            if cam_id not in self.cam_histories:
+                self.cam_histories[cam_id] = CamObservationHistory()
+            history = self.cam_histories[cam_id]
+
+        causes = history.get_causes_to_upload(image_metadata)
         if len(image_metadata) >= 80:
             causes.append('unexpected_observations_count')
         if len(causes) > 0:
-            tags = tags if tags is not None else []
-            tags.extend(causes)
-            await self.outbox.save(raw_image, image_metadata, tags, source, creation_date)
+            image_metadata.tags.extend(causes)
+            await self.outbox.save(raw_image, image_metadata)
         return causes

{learning_loop_node-0.16.1 → learning_loop_node-0.17.0}/learning_loop_node/detector/outbox.py

@@ -78,9 +78,6 @@ class Outbox():
     async def save(self,
                    image: bytes,
                    image_metadata: Optional[ImageMetadata] = None,
-                   tags: Optional[List[str]] = None,
-                   source: Optional[str] = None,
-                   creation_date: Optional[str] = None,
                    upload_priority: bool = False) -> None:
 
         if not await run.io_bound(self._is_valid_jpg, image):
@@ -89,12 +86,11 @@ class Outbox():
 
         if image_metadata is None:
             image_metadata = ImageMetadata()
-        if not tags:
-            tags = []
+
         identifier = datetime.now().isoformat(sep='_', timespec='microseconds')
 
         try:
-            await run.io_bound(self._save_files_to_disk, identifier, image, image_metadata, tags, source, creation_date, upload_priority)
+            await run.io_bound(self._save_files_to_disk, identifier, image, image_metadata, upload_priority)
         except Exception as e:
             self.log.error('Failed to save files for image %s: %s', identifier, e)
             return
@@ -110,9 +106,6 @@ class Outbox():
                             identifier: str,
                             image: bytes,
                             image_metadata: ImageMetadata,
-                            tags: List[str],
-                            source: Optional[str],
-                            creation_date: Optional[str],
                             upload_priority: bool) -> None:
         subpath = 'priority' if upload_priority else 'normal'
         full_path = f'{self.path}/{subpath}/{identifier}'
@@ -120,14 +113,6 @@ class Outbox():
             raise FileExistsError(f'Directory with identifier {identifier} already exists')
 
         tmp = f'{GLOBALS.data_folder}/tmp/{identifier}'
-        image_metadata.tags = tags
-        if self._is_valid_isoformat(creation_date):
-            image_metadata.created = creation_date
-        else:
-            image_metadata.created = identifier
-
-        image_metadata.source = source or 'unknown'
-
         os.makedirs(tmp, exist_ok=True)
 
         with open(tmp + f'/image_{identifier}.json', 'w') as f:
@@ -139,6 +124,7 @@ class Outbox():
         if not os.path.exists(tmp):
             self.log.error('Could not rename %s to %s', tmp, full_path)
             raise FileNotFoundError(f'Could not rename {tmp} to {full_path}')
+
         os.rename(tmp, full_path)
 
     async def _trim_upload_queue(self) -> None: