learning-loop-node 0.25.0__tar.gz → 0.26.0__tar.gz

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: learning_loop_node
-Version: 0.25.0
+Version: 0.26.0
 Summary: Python Library for Nodes which connect to the Zauberzeug Learning Loop
 Author-email: Zauberzeug GmbH <info@zauberzeug.com>
 License: MIT
@@ -114,9 +114,10 @@ Example usage:
 To use the **SocketIO** inference EPs, the caller needs to connect to the detector node's SocketIO server and emit the `detect` or `batch_detect` event with the image data and image metadata. The `detect` endpoint receives a dictionary, with the following entries:
 
 - `image`: The image data as dictionary with the following keys:
-  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+  - `format`: `raw` (default) or `jpeg`
+  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`), or the JPEG-encoded image for format `jpeg`
+  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
 - `camera_id`: optional camera identifier (string) used to improve the autoupload filtering
 - `tags`: optional list of tags to add to the image in the learning loop
 - `source`: optional source string
@@ -127,6 +128,8 @@ The `batch_detect` endpoint receives a dictionary, with the same entries as the
 
 - `images`: List of image data dictionaries, each with the same structure as the `image` entry in the `detect` endpoint
 
+Sending a camera's JPEG as it is (`format: 'jpeg'`) saves the bandwidth of a raw pixel array and spares the detector a re-encode for the upload.
+
 Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
 
 ### Upload API
@@ -144,9 +147,10 @@ Example:
 The detector also has a **SocketIO** upload endpoint that can be used to upload images and detections to the learning loop. The function receives a dictionary, with the following entries:
 
 - `image`: the image data as dictionary with the following keys:
-  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+  - `format`: `raw` (default) or `jpeg`
+  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`), or the JPEG-encoded image for format `jpeg`
+  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
 - `metadata`: a dictionary representing the image metadata. If metadata contains detections and/or annotations, UUIDs for the classes are automatically determined based on the category names. Metadata should follow the schema of the `ImageMetadata` data class.
 - `upload_priority`: Optional boolean flag to prioritize the upload (defaults to False)
 

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/README.md

@@ -81,9 +81,10 @@ Example usage:
 To use the **SocketIO** inference EPs, the caller needs to connect to the detector node's SocketIO server and emit the `detect` or `batch_detect` event with the image data and image metadata. The `detect` endpoint receives a dictionary, with the following entries:
 
 - `image`: The image data as dictionary with the following keys:
-  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+  - `format`: `raw` (default) or `jpeg`
+  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`), or the JPEG-encoded image for format `jpeg`
+  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
 - `camera_id`: optional camera identifier (string) used to improve the autoupload filtering
 - `tags`: optional list of tags to add to the image in the learning loop
 - `source`: optional source string
@@ -94,6 +95,8 @@ The `batch_detect` endpoint receives a dictionary, with the same entries as the
 
 - `images`: List of image data dictionaries, each with the same structure as the `image` entry in the `detect` endpoint
 
+Sending a camera's JPEG as it is (`format: 'jpeg'`) saves the bandwidth of a raw pixel array and spares the detector a re-encode for the upload.
+
 Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
 
 ### Upload API
@@ -111,9 +114,10 @@ Example:
 The detector also has a **SocketIO** upload endpoint that can be used to upload images and detections to the learning loop. The function receives a dictionary, with the following entries:
 
 - `image`: the image data as dictionary with the following keys:
-  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+  - `format`: `raw` (default) or `jpeg`
+  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`), or the JPEG-encoded image for format `jpeg`
+  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
 - `metadata`: a dictionary representing the image metadata. If metadata contains detections and/or annotations, UUIDs for the classes are automatically determined based on the category names. Metadata should follow the schema of the `ImageMetadata` data class.
 - `upload_priority`: Optional boolean flag to prioritize the upload (defaults to False)
 

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/data_exchanger.py

@@ -12,9 +12,11 @@ from typing import Any, Dict, List, Optional
 import aiofiles  # type: ignore
 
 from .data_classes import Context
-from .helpers.misc import create_resource_paths, create_task, is_valid_image
+from .helpers.misc import create_resource_paths, create_task, is_valid_image, is_valid_uuid4
 from .loop_communication import LoopCommunicator
-from .trainer.exceptions import CriticalError
+from .trainer.exceptions import CriticalError, ModelUploadFailedError
+
+logger = logging.getLogger(__name__)
 
 
 class DownloadError(Exception):
@@ -171,20 +173,41 @@ class DataExchanger():
         logging.info('Downloaded model %s(%s) to %s.', model_uuid, model_format, target_folder)
         return created_files
 
-    async def upload_model_get_uuid(self, context: Context, files: List[str], training_number: Optional[int], mformat: str) -> str:
+    async def upload_model_get_uuid(self, context: Context, files: List[str], training_number: Optional[int], mformat: str) -> str | None:
         """Used by the trainers. Function returns the new model uuid to use for detection.
+        A loop that already stores the model in this format answers 409 and should name the model's uuid there.
 
-        :return: The new model uuid.
-        :raise CriticalError: If the upload does not return status code 200.
+        :return: The new model uuid, or None if the loop already stores the model in this format but names no valid uuid.
+        :raise CriticalError: If the loop rejects the upload with a client error (other than 408 and 429).
+        :raise ModelUploadFailedError: If the loop could not store the model, e.g. on a server error.
         """
-        response = await self.loop_communicator.put(f'/{context.organization}/projects/{context.project}/trainings/{training_number}/models/latest/{mformat}/file', files=files)
-        if response.status_code != 200:
-            logging.error('Could not upload model for training %s, format %s: %s',
-                          training_number, mformat, response.text)
-            raise CriticalError(
-                f'Could not upload model for training {training_number}, format {mformat}: {response.text}')
+        path = f'/{context.organization}/projects/{context.project}/trainings/{training_number}/models/latest/{mformat}/file'
+        response = await self.loop_communicator.put(path, files=files)
+        if response.status_code == HTTPStatus.CONFLICT:
+            try:
+                stored_model_id = response.json()['detail']['id']
+            except (ValueError, KeyError, TypeError):  # older loops name no id
+                stored_model_id = None
+            if not is_valid_uuid4(stored_model_id):
+                logger.warning('The loop already stores the model of training %s in format %s, but names no id',
+                               training_number, mformat)
+                return None
+            logger.info('The loop already stores model %s of training %s in format %s',
+                        stored_model_id, training_number, mformat)
+            return stored_model_id
+        if response.status_code != HTTPStatus.OK:
+            logger.error('Could not upload model for training %s, format %s: %s %s',
+                         training_number, mformat, response.status_code, response.text)
+            error = CriticalError if _rejects_upload(response.status_code) else ModelUploadFailedError
+            raise error(
+                f'Could not upload model for training {training_number}, format {mformat}: {response.status_code} {response.text}')
 
         uploaded_model = response.json()
-        logging.info('Uploaded model for training %s, format %s. Response is: %s',
-                     training_number, mformat, uploaded_model)
+        logger.info('Uploaded model for training %s, format %s. Response is: %s',
+                    training_number, mformat, uploaded_model)
         return uploaded_model['id']
+
+
+def _rejects_upload(status_code: int) -> bool:
+    """A client error other than 408 and 429 answers the same however often the upload is sent."""
+    return 400 <= status_code < 500 and status_code not in (HTTPStatus.REQUEST_TIMEOUT, HTTPStatus.TOO_MANY_REQUESTS)

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/detector/detector_node.py

@@ -6,6 +6,7 @@ import os
 import shutil
 import subprocess
 import sys
+from collections.abc import Sequence
 from dataclasses import asdict, dataclass
 from datetime import datetime
 from typing import Dict, List, Literal, Optional, Union
@@ -30,7 +31,7 @@ from ..data_exchanger import DataExchanger, DownloadError
 from ..enums import OperationMode, VersionMode
 from ..globals import GLOBALS
 from ..helpers import background_tasks, environment_reader, run
-from ..helpers.misc import numpy_image_from_dict
+from ..helpers.misc import ImageData, image_from_dict, jpg_bytes_to_numpy_array
 from ..node import Node
 from .categories import category_by_name
 from .detector_logic import DetectorLogic, DetectorLogicFactory
@@ -230,9 +231,10 @@ class DetectorNode(Node):
 
             The data dict has the following schema:
             - image: The image data as dictionary:
-              - bytes: bytes of the ndarray
-              - dtype: data type of the ndarray
-              - shape: shape of the ndarray
+              - format: 'raw' (default) or 'jpeg' (optional)
+              - bytes: bytes of the ndarray, or the JPEG-encoded image for format 'jpeg'
+              - dtype: data type of the ndarray (raw only)
+              - shape: shape of the ndarray (raw only)
             - camera_id: Optional camera ID
             - tags: Optional list of tags
             - source: Optional source string
@@ -240,7 +242,7 @@ class DetectorNode(Node):
             - creation_date: Optional creation date in isoformat string
             """
             try:
-                image = numpy_image_from_dict(data['image'])
+                image = image_from_dict(data['image'])
             except Exception:
                 self.log.exception('could not parse image from socketio')
                 return {'error': 'could not parse image from data'}
@@ -252,7 +254,7 @@ class DetectorNode(Node):
                     tags=data.get('tags', []),
                     source=data.get('source', None),
                     autoupload=data.get('autoupload', 'filtered'),
-                    creation_date=data.get('creation_date', None)
+                    creation_date=data.get('creation_date', None),
                 )
                 return jsonable_encoder(asdict(det))
             except DetectorUnavailableError as e:
@@ -271,7 +273,7 @@ class DetectorNode(Node):
             """
             try:
                 images_data = data['images']
-                images = [numpy_image_from_dict(image) for image in images_data]
+                images = [image_from_dict(image) for image in images_data]
             except Exception:
                 self.log.exception('could not parse images from socketio')
                 return {'error': 'could not parse images from data'}
@@ -283,7 +285,7 @@ class DetectorNode(Node):
                     camera_id=data.get('camera_id', None),
                     source=data.get('source', None),
                     autoupload=data.get('autoupload', 'filtered'),
-                    creation_date=data.get('creation_date', None)
+                    creation_date=data.get('creation_date', None),
                 )
                 return jsonable_encoder(asdict(det))
             except DetectorUnavailableError as e:
@@ -336,9 +338,11 @@ class DetectorNode(Node):
 
             The data dict must contain:
             - image: The image data as dictionary with the following keys:
-                - bytes: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-                - dtype: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-                - shape: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+                - format: 'raw' (default) or 'jpeg' (optional)
+                - bytes: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`),
+                  or the JPEG-encoded image for format 'jpeg'
+                - dtype: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+                - shape: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
             - metadata: The metadata for the image (optional). Besides detections and annotations it
               may carry `tags`, `source`, `created` and `state`. A `state` of e.g. "trash" makes the
               loop file the image into that state instead of its default one.
@@ -362,7 +366,7 @@ class DetectorNode(Node):
                 image_metadata = ImageMetadata()
 
             try:
-                image = numpy_image_from_dict(data['image'])
+                image = image_from_dict(data['image'])
             except Exception:
                 self.log.exception('could not parse image from socketio')
                 return {'error': 'could not parse image from data'}
@@ -607,7 +611,7 @@ class DetectorNode(Node):
             self.log.error('could not reload app')
 
     async def get_detections(self,
-                             image: np.ndarray,
+                             image: ImageData,
                              tags: List[str],
                              *,
                              camera_id: Optional[str] = None,
@@ -620,11 +624,15 @@ class DetectorNode(Node):
         Used when an image is received via REST or SocketIO.
         This function infers the detections from the image,
         cares about uploading to the loop and returns the detections as ImageMetadata object.
+        The image is a pixel array or JPEG bytes; JPEG bytes are decoded for the inference
+        and stored as they are if the image is uploaded.
         Raises exception if no model is loaded.
         """
+        _unwrap_detector(self._detector)  # fails fast, before the decode
+        array = await _pixel_array(image)
         async with self.detection_lock:
             detector = _unwrap_detector(self._detector)
-            metadata = await run.io_bound(detector.logic.evaluate, image)
+            metadata = await run.io_bound(detector.logic.evaluate, array)
 
         metadata.tags.extend(tags)
         metadata.source = source
@@ -646,7 +654,7 @@ class DetectorNode(Node):
         return metadata
 
     async def get_batch_detections(self,
-                                   images: List[np.ndarray],
+                                   images: Sequence[ImageData],
                                    tags: List[str],
                                    *,
                                    camera_id: Optional[str] = None,
@@ -658,11 +666,15 @@ class DetectorNode(Node):
 
         This function infers the detections from all images,
         cares about uploading to the loop and returns the detections as a list of ImageMetadata.
+        Each image is a pixel array or JPEG bytes; JPEG bytes are decoded for the inference
+        and stored as they are if the image is uploaded.
         Raises exception if no model is loaded.
         """
+        _unwrap_detector(self._detector)  # fails fast, before the decode
+        arrays = await asyncio.gather(*(_pixel_array(image) for image in images))
         async with self.detection_lock:
             detector = _unwrap_detector(self._detector)
-            all_detections = await run.io_bound(detector.logic.batch_evaluate, images)
+            all_detections = await run.io_bound(detector.logic.batch_evaluate, arrays)
 
         for metadata in all_detections.items:
             metadata.tags.extend(tags)
@@ -687,14 +699,14 @@ class DetectorNode(Node):
 
     async def upload_images(
             self, *,
-            images: List[np.ndarray],
+            images: Sequence[ImageData],
             images_metadata: Optional[ImagesMetadata] = None,
             upload_priority: bool = False
     ) -> None:
         """Save images to the outbox using an asyncio executor.
         Used by SIO and REST upload endpoints.
 
-        :param images: List of images to upload
+        :param images: List of images to upload, pixel arrays or JPEG bytes
         :param images_metadata: Optional metadata for all images
         :param upload_priority: Whether to upload the images with priority
         :raises ValueError: If the number of images and number of metadata items do not match
@@ -760,6 +772,13 @@ class DetectorUnavailableError(Exception):
     pass
 
 
+async def _pixel_array(image: ImageData) -> np.ndarray:
+    """The pixels of an image; JPEG bytes are decoded in a thread, as that takes a while for a camera image."""
+    if isinstance(image, np.ndarray):
+        return image
+    return await run.io_bound(jpg_bytes_to_numpy_array, image)
+
+
 def _unwrap_detector(state: '_DetectorState') -> _ActiveDetector:
     """Return the active detector or raise DetectorUnavailableError."""
     match state:

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/detector/inbox_filter/relevance_filter.py

@@ -1,8 +1,7 @@
 from typing import Dict, List, Optional
 
-import numpy as np
-
 from ...data_classes.image_metadata import ImageMetadata
+from ...helpers.misc import ImageData
 from ..outbox import Outbox
 from .cam_observation_history import CamObservationHistory
 
@@ -17,7 +16,7 @@ class RelevanceFilter():
     async def may_upload_detections(self,
                                     image_metadata: ImageMetadata,
                                     cam_id: Optional[str],
-                                    image: np.ndarray) -> List[str]:
+                                    image: ImageData) -> List[str]:
         """Check if the detection should be uploaded to the outbox.
         If so, upload it and return the list of causes for the upload.
         """

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/detector/outbox.py

@@ -1,5 +1,4 @@
 import asyncio
-import io
 import json
 import logging
 import os
@@ -17,15 +16,13 @@ from typing import List, Optional, Tuple, TypeVar, Union
 
 import aiohttp
 import numpy as np
-import PIL
-import PIL.Image  # type: ignore
 from fastapi.encoders import jsonable_encoder
 
 from ..data_classes import ImageMetadata
 from ..enums import OutboxMode
 from ..globals import GLOBALS
 from ..helpers import environment_reader, run
-from ..helpers.misc import numpy_array_to_jpg_bytes
+from ..helpers.misc import ImageData, numpy_array_to_jpg_bytes
 
 T = TypeVar('T')
 
@@ -80,16 +77,17 @@ class Outbox():
             self.upload_folders.append(file)
 
     async def save(self,
-                   image: np.ndarray,
+                   image: ImageData,
                    image_metadata: Optional[ImageMetadata] = None,
                    upload_priority: bool = False) -> None:
         """
-        Save an image and its metadata to disk. 
+        Save an image and its metadata to disk.
 
+        A pixel array is encoded as JPEG here, image bytes are stored unchanged for the loop to validate.
         The data will be picked up by the continuous upload process.
         """
 
-        jpg_bytes = numpy_array_to_jpg_bytes(image)
+        jpg_bytes = numpy_array_to_jpg_bytes(image) if isinstance(image, np.ndarray) else image
 
         if image_metadata is None:
             image_metadata = ImageMetadata()
@@ -262,14 +260,6 @@ class Outbox():
         else:
             self.log.error('Could not upload images: %s', response.content)
 
-    def _is_valid_jpg(self, image: bytes) -> bool:
-        try:
-            _ = PIL.Image.open(io.BytesIO(image), formats=['JPEG'])
-            return True
-        except Exception:
-            self.log.exception('Invalid jpg image')
-            return False
-
     async def ensure_continuous_upload_stopped(self) -> bool:
         self.log.debug('Outbox: Ensuring continuous upload')
         if not self._upload_process_alive():

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/detector/rest/detect.py

@@ -4,7 +4,6 @@ from typing import TYPE_CHECKING, Literal, Optional
 from fastapi import APIRouter, File, Header, Request, UploadFile
 
 from ...data_classes.image_metadata import ImageMetadata
-from ...helpers.misc import jpg_bytes_to_numpy_array
 
 if TYPE_CHECKING:
     from ..detector_node import DetectorNode
@@ -42,7 +41,7 @@ async def http_detect(
         raise Exception(f'Uploaded file {file.filename} is no image file.') from exc
 
     try:
-        detections = await node.get_detections(image=jpg_bytes_to_numpy_array(file_bytes),
+        detections = await node.get_detections(image=file_bytes,
                                                camera_id=camera_id or None,
                                                tags=tags.split(',') if tags else [],
                                                source=source,

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/detector/rest/upload.py

@@ -3,7 +3,6 @@ from typing import TYPE_CHECKING, List, Optional
 from fastapi import APIRouter, File, Query, Request, UploadFile
 
 from ...data_classes.image_metadata import ImageMetadata, ImagesMetadata
-from ...helpers.misc import jpg_bytes_to_numpy_array
 
 if TYPE_CHECKING:
     from ..detector_node import DetectorNode
@@ -29,14 +28,8 @@ async def upload_image(request: Request,
     """
     node: 'DetectorNode' = request.app
 
-    files_bytes = [await file.read() for file in files]
-    image_metadatas = []
-    images = []
-    for file_bytes in files_bytes:
-        images.append(jpg_bytes_to_numpy_array(file_bytes))
-        image_metadatas.append(ImageMetadata(source=source, created=creation_date))
-
-    images_metadata = ImagesMetadata(items=image_metadatas)
+    images = [await file.read() for file in files]
+    images_metadata = ImagesMetadata(items=[ImageMetadata(source=source, created=creation_date) for _ in images])
 
     await node.upload_images(images=images,
                              images_metadata=images_metadata,

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/helpers/misc.py

@@ -10,12 +10,12 @@ import sys
 from dataclasses import asdict
 from glob import glob
 from time import perf_counter
-from typing import Any, Coroutine, Dict, List, Optional, Tuple, TypeVar
+from typing import Any, Coroutine, Dict, List, Literal, Optional, Tuple, TypeVar, Union, get_args
 from uuid import UUID, uuid4
 
 import numpy as np
 import pynvml
-from PIL import Image
+from PIL import Image, ImageOps
 
 from ..data_classes.general import Context
 from ..data_classes.socket_response import SocketResponse
@@ -220,31 +220,56 @@ def create_training_folder(project_folder: str, trainings_id: str) -> str:
     return training_folder
 
 
+ImageData = Union[np.ndarray, bytes]
+"""An image as it was received: either a pixel array or a JPEG-encoded image."""
+
+ImageFormat = Literal['raw', 'jpeg']
+"""How the `bytes` of an image dict are encoded: a pixel array in C order described by `dtype` and `shape`, or a JPEG."""
+
+
 def jpg_bytes_to_numpy_array(jpg_bytes: bytes) -> np.ndarray:
-    """Convert jpg bytes to numpy array."""
-    image = Image.open(io.BytesIO(jpg_bytes))
+    """Decode image bytes into a pixel array, rotated as the EXIF orientation tag says."""
+    image = ImageOps.exif_transpose(Image.open(io.BytesIO(jpg_bytes)))
     return np.array(image)
 
 
 def numpy_array_to_jpg_bytes(image_array: np.ndarray) -> bytes:
-    """Convert jpg bytes to numpy array."""
+    """Encode a pixel array as JPEG."""
     buffer = io.BytesIO()
     Image.fromarray(image_array).save(buffer, format="JPEG")
     jpg_bytes = buffer.getvalue()
     return jpg_bytes
 
 
+def image_format_of(image_data: Dict) -> ImageFormat:
+    """The format an image dict declares, 'raw' if it declares none."""
+    image_format = image_data.get('format', 'raw')
+    if image_format not in get_args(ImageFormat):
+        raise ValueError(f'unknown image format: {image_format}')
+    return image_format
+
+
 def numpy_image_from_dict(image_data: Dict) -> np.ndarray:
     """
     Convert image dict to numpy array.
 
     The image_data dict should have the following keys:
-    - bytes: bytes of the image in C order (row-major)
-    - dtype: data type of the pixel values
-    - shape: shape of the image (height, width, channels)
+    - format: 'raw' (default) or 'jpeg' (optional)
+    - bytes: bytes of the image in C order (row-major), or the JPEG-encoded image for format 'jpeg'
+    - dtype: data type of the pixel values (raw only)
+    - shape: shape of the image (height, width, channels) (raw only)
     """
+    if image_format_of(image_data) == 'jpeg':
+        return jpg_bytes_to_numpy_array(image_data['bytes'])
 
     image_bytes = image_data['bytes']
     image_dtype = image_data['dtype']
     image_shape = image_data['shape']
     return np.frombuffer(image_bytes, dtype=image_dtype).reshape(image_shape, order='C')
+
+
+def image_from_dict(image_data: Dict) -> ImageData:
+    """The image of an image dict as received: the JPEG bytes for format 'jpeg', the pixel array otherwise."""
+    if image_format_of(image_data) == 'jpeg':
+        return image_data['bytes']
+    return numpy_image_from_dict(image_data)

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/trainer/exceptions.py

@@ -5,6 +5,12 @@ class CriticalError(Exception):
     """
 
 
+class ModelUploadFailedError(Exception):
+    """Raised when the loop could not store an uploaded model, e.g. because it answered with a server error.
+    Unlike a CriticalError it keeps the training, so the upload is tried again later.
+    """
+
+
 class TrainingRefusedError(Exception):
     """Raised when a trainer cannot take on a training it was asked to begin; nothing was set up."""
 

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/trainer/io_helpers.py

@@ -55,6 +55,7 @@ class ActiveTrainingIO:
         self.context = context
 
         self.mup_path = f'{training_folder}/model_uploading_progress.txt'
+        self.muuid_path = f'{training_folder}/uploaded_model_uuid.txt'
         # string with placeholder gor index
         self.det_path = f'{training_folder}' + '/detections_{0}.json'
         self.dufi_path = f'{training_folder}/detection_uploading_json_index.txt'
@@ -73,6 +74,17 @@ class ActiveTrainingIO:
         with open(self.mup_path, 'r') as f:
             return f.read().split(',')
 
+    def save_uploaded_model_uuid(self, model_uuid: str) -> None:
+        """Replaces the file in one step, so a crash while writing cannot leave it empty."""
+        with open(f'{self.muuid_path}.tmp', 'w') as f:
+            f.write(model_uuid)
+        os.replace(f'{self.muuid_path}.tmp', self.muuid_path)
+
+    def load_uploaded_model_uuid(self) -> str:
+        """:raise FileNotFoundError: If no upload has named the model uuid yet."""
+        with open(self.muuid_path) as f:
+            return f.read()
+
     # detections
 
     def _get_detection_file_names(self) -> List[Path]:

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node/trainer/trainer_logic_generic.py

@@ -33,9 +33,17 @@ from .training_slot import TrainingSlot, training_slot_from_env
 if TYPE_CHECKING:
     from .trainer_node import TrainerNode
 
+BASE_RETRY_DELAY = 5  # seconds before the first retry of a failed state
+MAX_RETRY_DELAY = 300  # seconds
+
 logger = logging.getLogger('learning_loop_node.trainer_logic_generic')
 
 
+def _retry_delay(failures: int) -> float:
+    """The wait before the next attempt of a state that has failed `failures` times in a row."""
+    return min(BASE_RETRY_DELAY * 2 ** min(failures - 1, 16), MAX_RETRY_DELAY)
+
+
 class TrainerLogicGeneric(ABC):
 
     def __init__(self, model_format: str, training_slot: TrainingSlot | None = None) -> None:
@@ -45,6 +53,7 @@ class TrainerLogicGeneric(ABC):
         # '/{context.organization}/projects/{context.project}/models/{model_id}/{model_format}/file'
         self.model_format: str = model_format
         self.errors = Errors()
+        self._state_failures: Dict[str, int] = {}  # failed attempts in a row, by error key
 
         self.training_task: Optional[asyncio.Task] = None
         self.shutdown_event: asyncio.Event = asyncio.Event()
@@ -264,6 +273,7 @@ class TrainerLogicGeneric(ABC):
         Either via `begin_training` or `try_continue_run_if_incomplete`.
         """
         self.errors.reset_all()
+        self._state_failures.clear()
         try:
             await self.training_slot.wait_for(self._slot_claim(self.training.context, self.training.training_number))
             await self._training_loop()
@@ -321,8 +331,11 @@ class TrainerLogicGeneric(ABC):
         - If the loop sends a StopTraining event, this will raise a CancelledError.
         - States can raise a CriticalError indicating that there is no point in retrying the state.
         - If any other error occurs, the error is stored in the errors object and the state is reset to the previous state.
+          The state is retried after a wait until the training is aborted.
         '''
 
+        if failures := self._state_failures.get(error_key):
+            await asyncio.sleep(_retry_delay(failures))
         await asyncio.sleep(0.1)
         logger.info('Performing state: %s', state_during)
         previous_state = self.training.training_state
@@ -351,6 +364,7 @@ class TrainerLogicGeneric(ABC):
             print('Error in %s - Exception: %s', state_during, e, flush=True)
             self.errors.set(error_key, str(e))
             logger.exception('Error in %s - Exception: %s', state_during, e)
+            self._state_failures[error_key] = self._state_failures.get(error_key, 0) + 1
             self.training.training_state = previous_state
             return
         else:
@@ -359,6 +373,7 @@ class TrainerLogicGeneric(ABC):
                 self.errors.reset(error_key)
             self.training.training_state = state_after
 
+        self._state_failures.pop(error_key, None)
         self.last_training_io.save(self.training)
 
     async def _prepare(self) -> None:
@@ -440,10 +455,13 @@ class TrainerLogicGeneric(ABC):
         Note that with the latest trainers the conversion to (.wts) is done by the trainer.
         The conversion from .wts to .engine is done by the detector (needs to be done on target hardware).
         Note that trainer may train with different classes, which is why we send an initial model.json file.
+        A format the loop already stores counts as uploaded.
+        The model UUID is kept beside the upload progress, so a retry after a restart still knows it.
 
         :return: The new model UUID.
-        :raise CriticalError: If the latest model files cannot be obtained.
-        :raise Exception: If the response for the model upload attempt is invalid.
+        :raise CriticalError: If the latest model files cannot be obtained,
+            or the model is uploaded but its UUID is unknown.
+        :raise ModelUploadFailedError: If the loop could not store a model file; the state is retried.
         """
 
         files = await self._get_latest_model_files()
@@ -456,20 +474,21 @@ class TrainerLogicGeneric(ABC):
 
         already_uploaded_formats = self.active_training_io.load_model_upload_progress()
 
-        model_uuid = None
         for file_format in [f for f in files if f not in already_uploaded_formats]:
             _files = files[file_format] + [self._dump_categories_to_json()]
             assert len([f for f in _files if 'model.json' in f]) == 1, "model.json must be included exactly once"
 
             model_uuid = await self.node.data_exchanger.upload_model_get_uuid(context, _files, self.training.training_number, file_format)
+            if model_uuid is not None:
+                self.active_training_io.save_uploaded_model_uuid(model_uuid)
 
             already_uploaded_formats.append(file_format)
             self.active_training_io.save_model_upload_progress(already_uploaded_formats)
 
-        if not model_uuid:
-            raise Exception('Invalid response for model upload attempt')
-
-        return model_uuid
+        try:
+            return self.active_training_io.load_uploaded_model_uuid()
+        except FileNotFoundError as e:
+            raise CriticalError('The model is uploaded, but the loop never answered with its id') from e
 
     def _dump_categories_to_json(self) -> str:
         """Dumps the categories to a json file and returns the path to the file.

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/learning_loop_node.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: learning_loop_node
-Version: 0.25.0
+Version: 0.26.0
 Summary: Python Library for Nodes which connect to the Zauberzeug Learning Loop
 Author-email: Zauberzeug GmbH <info@zauberzeug.com>
 License: MIT
@@ -114,9 +114,10 @@ Example usage:
 To use the **SocketIO** inference EPs, the caller needs to connect to the detector node's SocketIO server and emit the `detect` or `batch_detect` event with the image data and image metadata. The `detect` endpoint receives a dictionary, with the following entries:
 
 - `image`: The image data as dictionary with the following keys:
-  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+  - `format`: `raw` (default) or `jpeg`
+  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`), or the JPEG-encoded image for format `jpeg`
+  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
 - `camera_id`: optional camera identifier (string) used to improve the autoupload filtering
 - `tags`: optional list of tags to add to the image in the learning loop
 - `source`: optional source string
@@ -127,6 +128,8 @@ The `batch_detect` endpoint receives a dictionary, with the same entries as the
 
 - `images`: List of image data dictionaries, each with the same structure as the `image` entry in the `detect` endpoint
 
+Sending a camera's JPEG as it is (`format: 'jpeg'`) saves the bandwidth of a raw pixel array and spares the detector a re-encode for the upload.
+
 Example code can be found [in the rosys implementation](https://github.com/zauberzeug/rosys/blob/main/rosys/vision/detector_hardware.py).
 
 ### Upload API
@@ -144,9 +147,10 @@ Example:
 The detector also has a **SocketIO** upload endpoint that can be used to upload images and detections to the learning loop. The function receives a dictionary, with the following entries:
 
 - `image`: the image data as dictionary with the following keys:
-  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`)
-  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.)
-  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`)
+  - `format`: `raw` (default) or `jpeg`
+  - `bytes`: bytes of the ndarray (retrieved via `ndarray.tobytes(order='C')`), or the JPEG-encoded image for format `jpeg`
+  - `dtype`: data type of the ndarray as string (e.g. `uint8`, `float32`, etc.) (raw only)
+  - `shape`: shape of the ndarray as tuple of ints (e.g. `(480, 640, 3)`) (raw only)
 - `metadata`: a dictionary representing the image metadata. If metadata contains detections and/or annotations, UUIDs for the classes are automatically determined based on the category names. Metadata should follow the schema of the `ImageMetadata` data class.
 - `upload_priority`: Optional boolean flag to prioritize the upload (defaults to False)
 

{learning_loop_node-0.25.0 → learning_loop_node-0.26.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "learning_loop_node"
-version = "0.25.0"
+version = "0.26.0"
 description = "Python Library for Nodes which connect to the Zauberzeug Learning Loop"
 authors = [{ name = "Zauberzeug GmbH", email = "info@zauberzeug.com" }]
 license = { text = "MIT" }