learning-loop-node 0.13.6__tar.gz → 0.14.0__tar.gz

{learning_loop_node-0.13.6 → learning_loop_node-0.14.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: learning-loop-node
-Version: 0.13.6
+Version: 0.14.0
 Summary: Python Library for Nodes which connect to the Zauberzeug Learning Loop
 Home-page: https://github.com/zauberzeug/learning_loop_node
 License: MIT
@@ -63,8 +63,8 @@ You can configure connection to our Learning Loop by specifying the following en
 | LOOP_USERNAME            | USERNAME     | Learning Loop user name                                      | all besides Detector      |
 | LOOP_PASSWORD            | PASSWORD     | Learning Loop password                                       | all besides Detector      |
 | LOOP_SSL_CERT_PATH       | -            | Path to the SSL certificate                                  | all (opt.)                |
-| LOOP_ORGANIZATION        | ORGANIZATION | Organization name                                            | Detector                  |
-| LOOP_PROJECT             | PROJECT      | Project name                                                 | Detector (opt.)           |
+| LOOP_ORGANIZATION        | ORGANIZATION | Organization ID                                              | Detector                  |
+| LOOP_PROJECT             | PROJECT      | Project ID                                                   | Detector (opt.)           |
 | MIN_UNCERTAIN_THRESHOLD  | -            | smallest confidence (float) at which auto-upload will happen | Detector (opt.)           |
 | MAX_UNCERTAIN_THRESHOLD  | -            | largest confidence (float) at which auto-upload will happen  | Detector (opt.)           |
 | INFERENCE_BATCH_SIZE     | -            | Batch size of trainer when calculating detections            | Trainer (opt.)            |
@@ -73,6 +73,8 @@ You can configure connection to our Learning Loop by specifying the following en
 | TRAINER_IDLE_TIMEOUT_SEC | -            | Automatically shutdown trainer after timeout (in seconds)    | Trainer (opt.)            |
 | USE_BACKDOOR_CONTROLS    | -            | Always enable backdoor controls (set to 1)                   | Trainer / Detector (opt.) |
 
+Note that organization and project IDs are always lower case and may differ from the names in the Learning Loop which can have uppercase letters.
+
 #### Testing
 
 We use github actions for CI. Tests can also be executed locally by running
@@ -103,6 +105,9 @@ The detector also has a sio **upload endpoint** that can be used to upload image
 - `image`: the image data in jpg format
 - `tags`: a list of strings. If not provided the tag is `picked_by_system`
 - `detections`: a dictionary representing the detections. UUIDs for the classes are automatically determined based on the category names. This field is optional. If not provided, no detections are uploaded.
+- `source`: optional source identifier for the image
+- `creation_date`: optional creation date for the image
+- `upload_priority`: boolean flag to prioritize the upload (defaults to False)
 
 The endpoint returns None if the upload was successful and an error message otherwise.
 
@@ -185,58 +190,52 @@ Upload a model with
 The model should now be available for the format 'format_a'
 `curl "https://learning-loop.ai/api/zauberzeug/projects/demo/models?format=format_a"`
 
-````
-
+```json
 {
-"models": [
-{
-"id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
-"version": "4.0",
-"formats": [
-"format_a"
-],
-"created": "2021-06-01T06:28:21.289092",
-"comment": "uploaded at 2021-06-01 06:28:21.288442",
-...
+  "models": [
+    {
+    "id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
+    "version": "4.0",
+    "formats": [
+      "format_a"
+    ],
+    "created": "2021-06-01T06:28:21.289092",
+    "comment": "uploaded at 2021-06-01 06:28:21.288442",
+    ...
+    }
+  ]
 }
-]
-}
-
 ```
 
 but not in the format_b
 `curl "https://learning-loop.ai/api/zauberzeug/projects/demo/models?format=format_b"`
 
-```
-
+```json
 {
-"models": []
+  "models": []
 }
-
 ```
 
 Connect the Node to the Learning Loop by simply starting the container.
 After a short time the converted model should be available as well.
 `curl https://learning-loop.ai/api/zauberzeug/projects/demo/models?format=format_b`
 
-```
-
-{
-"models": [
+```json
 {
-"id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
-"version": "4.0",
-"formats": [
-"format_a",
-"format_b",
-],
-"created": "2021-06-01T06:28:21.289092",
-"comment": "uploaded at 2021-06-01 06:28:21.288442",
-...
-}
-]
+  "models": [
+  {
+  "id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
+    "version": "4.0",
+    "formats": [
+      "format_a",
+      "format_b",
+    ],
+    "created": "2021-06-01T06:28:21.289092",
+    "comment": "uploaded at 2021-06-01 06:28:21.288442",
+    ...
+  }
+  ]
 }
-
 ```
 
 ## About Models (the currency between Nodes)
@@ -255,6 +254,4 @@ After a short time the converted model should be available as well.
 - Nodes add properties to `model.json`, which contains all the information which are needed by subsequent nodes. These are typically the properties:
   - `resolution`: resolution in which the model expects images (as `int`, since the resolution is mostly square - later, ` resolution_x`` resolution_y ` would also be conceivable or `resolutions` to give a list of possible resolutions)
   - `categories`: list of categories with name, id, (later also type), in the order in which they are used by the model -- this is neccessary to be robust about renamings
-```
-````
 

{learning_loop_node-0.13.6 → learning_loop_node-0.14.0}/README.md

@@ -23,8 +23,8 @@ You can configure connection to our Learning Loop by specifying the following en
 | LOOP_USERNAME            | USERNAME     | Learning Loop user name                                      | all besides Detector      |
 | LOOP_PASSWORD            | PASSWORD     | Learning Loop password                                       | all besides Detector      |
 | LOOP_SSL_CERT_PATH       | -            | Path to the SSL certificate                                  | all (opt.)                |
-| LOOP_ORGANIZATION        | ORGANIZATION | Organization name                                            | Detector                  |
-| LOOP_PROJECT             | PROJECT      | Project name                                                 | Detector (opt.)           |
+| LOOP_ORGANIZATION        | ORGANIZATION | Organization ID                                              | Detector                  |
+| LOOP_PROJECT             | PROJECT      | Project ID                                                   | Detector (opt.)           |
 | MIN_UNCERTAIN_THRESHOLD  | -            | smallest confidence (float) at which auto-upload will happen | Detector (opt.)           |
 | MAX_UNCERTAIN_THRESHOLD  | -            | largest confidence (float) at which auto-upload will happen  | Detector (opt.)           |
 | INFERENCE_BATCH_SIZE     | -            | Batch size of trainer when calculating detections            | Trainer (opt.)            |
@@ -33,6 +33,8 @@ You can configure connection to our Learning Loop by specifying the following en
 | TRAINER_IDLE_TIMEOUT_SEC | -            | Automatically shutdown trainer after timeout (in seconds)    | Trainer (opt.)            |
 | USE_BACKDOOR_CONTROLS    | -            | Always enable backdoor controls (set to 1)                   | Trainer / Detector (opt.) |
 
+Note that organization and project IDs are always lower case and may differ from the names in the Learning Loop which can have uppercase letters.
+
 #### Testing
 
 We use github actions for CI. Tests can also be executed locally by running
@@ -63,6 +65,9 @@ The detector also has a sio **upload endpoint** that can be used to upload image
 - `image`: the image data in jpg format
 - `tags`: a list of strings. If not provided the tag is `picked_by_system`
 - `detections`: a dictionary representing the detections. UUIDs for the classes are automatically determined based on the category names. This field is optional. If not provided, no detections are uploaded.
+- `source`: optional source identifier for the image
+- `creation_date`: optional creation date for the image
+- `upload_priority`: boolean flag to prioritize the upload (defaults to False)
 
 The endpoint returns None if the upload was successful and an error message otherwise.
 
@@ -145,58 +150,52 @@ Upload a model with
 The model should now be available for the format 'format_a'
 `curl "https://learning-loop.ai/api/zauberzeug/projects/demo/models?format=format_a"`
 
-````
-
+```json
 {
-"models": [
-{
-"id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
-"version": "4.0",
-"formats": [
-"format_a"
-],
-"created": "2021-06-01T06:28:21.289092",
-"comment": "uploaded at 2021-06-01 06:28:21.288442",
-...
+  "models": [
+    {
+    "id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
+    "version": "4.0",
+    "formats": [
+      "format_a"
+    ],
+    "created": "2021-06-01T06:28:21.289092",
+    "comment": "uploaded at 2021-06-01 06:28:21.288442",
+    ...
+    }
+  ]
 }
-]
-}
-
 ```
 
 but not in the format_b
 `curl "https://learning-loop.ai/api/zauberzeug/projects/demo/models?format=format_b"`
 
-```
-
+```json
 {
-"models": []
+  "models": []
 }
-
 ```
 
 Connect the Node to the Learning Loop by simply starting the container.
 After a short time the converted model should be available as well.
 `curl https://learning-loop.ai/api/zauberzeug/projects/demo/models?format=format_b`
 
-```
-
-{
-"models": [
+```json
 {
-"id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
-"version": "4.0",
-"formats": [
-"format_a",
-"format_b",
-],
-"created": "2021-06-01T06:28:21.289092",
-"comment": "uploaded at 2021-06-01 06:28:21.288442",
-...
-}
-]
+  "models": [
+  {
+  "id": "3c20d807-f71c-40dc-a996-8a8968aa5431",
+    "version": "4.0",
+    "formats": [
+      "format_a",
+      "format_b",
+    ],
+    "created": "2021-06-01T06:28:21.289092",
+    "comment": "uploaded at 2021-06-01 06:28:21.288442",
+    ...
+  }
+  ]
 }
-
 ```
 
 ## About Models (the currency between Nodes)
@@ -215,5 +214,3 @@ After a short time the converted model should be available as well.
 - Nodes add properties to `model.json`, which contains all the information which are needed by subsequent nodes. These are typically the properties:
   - `resolution`: resolution in which the model expects images (as `int`, since the resolution is mostly square - later, ` resolution_x`` resolution_y ` would also be conceivable or `resolutions` to give a list of possible resolutions)
   - `categories`: list of categories with name, id, (later also type), in the order in which they are used by the model -- this is neccessary to be robust about renamings
-```
-````

{learning_loop_node-0.13.6 → learning_loop_node-0.14.0}/learning_loop_node/data_exchanger.py

@@ -143,7 +143,7 @@ class DataExchanger():
         logging.info('Downloading model data for uuid %s from the loop to %s..', model_uuid, target_folder)
 
         path = f'/{context.organization}/projects/{context.project}/models/{model_uuid}/{model_format}/file'
-        response = await self.loop_communicator.get(path, requires_login=False)
+        response = await self.loop_communicator.get(path, requires_login=False, timeout=60*10)
         if response.status_code != 200:
             decoded_content = response.content.decode('utf-8')
             logging.error('could not download loop/%s: %s, content: %s', path,

{learning_loop_node-0.13.6 → learning_loop_node-0.14.0}/learning_loop_node/detector/detector_logic.py

@@ -2,8 +2,6 @@ import logging
 from abc import abstractmethod
 from typing import List, Optional
 
-import numpy as np
-
 from ..data_classes import ImageMetadata, ModelInformation
 from ..globals import GLOBALS
 from .exceptions import NodeNeedsRestartError
@@ -44,13 +42,13 @@ class DetectorLogic():
     def init(self):
         """Called when a (new) model was loaded. Initialize the model. Model information available via `self.model_info`"""
 
-    def evaluate_with_all_info(self, image: np.ndarray, tags: List[str], source: Optional[str] = None, creation_date: Optional[str] = None) -> ImageMetadata:  # pylint: disable=unused-argument
+    def evaluate_with_all_info(self, image: bytes, tags: List[str], source: Optional[str] = None, creation_date: Optional[str] = None) -> ImageMetadata:  # pylint: disable=unused-argument
         """Called by the detector node when an image should be evaluated (REST or SocketIO).
         Tags, source come from the caller and may be used in this function. 
         By default, this function simply calls `evaluate`"""
         return self.evaluate(image)
 
     @abstractmethod
-    def evaluate(self, image: np.ndarray) -> ImageMetadata:
+    def evaluate(self, image: bytes) -> ImageMetadata:
         """Evaluate the image and return the detections.
         The object should return empty detections if it is not initialized"""

{learning_loop_node-0.13.6 → learning_loop_node-0.14.0}/learning_loop_node/detector/detector_node.py

@@ -1,14 +1,12 @@
 import asyncio
 import contextlib
-import math
 import os
 import shutil
 import subprocess
 import sys
 from dataclasses import asdict
 from datetime import datetime
-from threading import Thread
-from typing import Dict, List, Optional
+from typing import Dict, List, Optional, cast
 
 import numpy as np
 import socketio
@@ -30,7 +28,7 @@ from ..data_classes.socket_response import SocketResponse
 from ..data_exchanger import DataExchanger, DownloadError
 from ..enums import OperationMode, VersionMode
 from ..globals import GLOBALS
-from ..helpers import environment_reader
+from ..helpers import background_tasks, environment_reader, run
 from ..node import Node
 from .detector_logic import DetectorLogic
 from .exceptions import NodeNeedsRestartError
@@ -227,7 +225,7 @@ class DetectorNode(Node):
         async def detect(sid, data: Dict) -> Dict:
             try:
                 det = await self.get_detections(
-                    raw_image=np.frombuffer(data['image'], np.uint8),
+                    raw_image=data['image'],
                     camera_id=data.get('camera-id', None) or data.get('mac', None),
                     tags=data.get('tags', []),
                     source=data.get('source', None),
@@ -279,9 +277,10 @@ class DetectorNode(Node):
                 return {'error': str(e)}
 
         @self.sio.event
-        async def upload(sid, data: Dict) -> Optional[Dict]:
-            '''upload an image with detections'''
+        async def upload(sid, data: Dict) -> Dict:
+            """Upload an image with detections"""
 
+            self.log.debug('Processing upload via socketio.')
             detection_data = data.get('detections', {})
             if detection_data and self.detector_logic.model_info is not None:
                 try:
@@ -293,22 +292,19 @@ class DetectorNode(Node):
             else:
                 image_metadata = ImageMetadata()
 
-            tags = data.get('tags', [])
-            tags.append('picked_by_system')
-
-            source = data.get('source', None)
-            creation_date = data.get('creation_date', None)
-
-            self.log.debug('running upload via socketio. tags: %s, source: %s, creation_date: %s',
-                           tags, source, creation_date)
-
-            loop = asyncio.get_event_loop()
             try:
-                await loop.run_in_executor(None, self.outbox.save, data['image'], image_metadata, tags, source, creation_date)
+                await self.upload_images(
+                    images=[data['image']],
+                    image_metadata=image_metadata,
+                    tags=data.get('tags', []),
+                    source=data.get('source', None),
+                    creation_date=data.get('creation_date', None),
+                    upload_priority=data.get('upload_priority', False)
+                )
             except Exception as e:
                 self.log.exception('could not upload via socketio')
                 return {'error': str(e)}
-            return None
+            return {'status': 'OK'}
 
         @self.sio.event
         def connect(sid, environ, auth) -> None:
@@ -344,16 +340,21 @@ class DetectorNode(Node):
                 with step_into(GLOBALS.data_folder):
                     model_symlink = 'model'
                     target_model_folder = f'models/{self.target_model.version}'
-                    if not os.path.exists(target_model_folder):
-                        os.makedirs(target_model_folder)
-                        await self.data_exchanger.download_model(target_model_folder,
-                                                                 Context(organization=self.organization,
-                                                                         project=self.project),
-                                                                 self.target_model.id,
-                                                                 self.detector_logic.model_format)
-                        self.log.info('Downloaded model %s', self.target_model.version)
-                    else:
+                    if os.path.exists(target_model_folder) and len(os.listdir(target_model_folder)) > 0:
                         self.log.info('No need to download model %s (already exists)', self.target_model.version)
+                    else:
+                        os.makedirs(target_model_folder, exist_ok=True)
+                        try:
+                            await self.data_exchanger.download_model(target_model_folder,
+                                                                     Context(organization=self.organization,
+                                                                             project=self.project),
+                                                                     self.target_model.id,
+                                                                     self.detector_logic.model_format)
+                            self.log.info('Downloaded model %s', self.target_model.version)
+                        except Exception:
+                            self.log.exception('Could not download model %s', self.target_model.version)
+                            shutil.rmtree(target_model_folder, ignore_errors=True)
+                            return
                     try:
                         os.unlink(model_symlink)
                         os.remove(model_symlink)
@@ -422,16 +423,23 @@ class DetectorNode(Node):
         self.log_status_on_change(status.state or 'None', status)
 
         # NOTE: sending organization and project is no longer required!
-        response = await self.sio_client.call('update_detector', (self.organization, self.project, jsonable_encoder(asdict(status))))
+        try:
+            response = await self.sio_client.call('update_detector', (self.organization, self.project, jsonable_encoder(asdict(status))))
+        except TimeoutError:
+            self.socket_connection_broken = True
+            self.log.exception('TimeoutError for sending status update (will try to reconnect):')
+            raise Exception('Status update failed due to timeout') from None
+
         if not response:
             self.socket_connection_broken = True
-            return
+            self.log.error('Status update failed (will try to reconnect): %s', response)
+            raise Exception('Status update failed: Did not receive a response from the learning loop')
 
         socket_response = from_dict(data_class=SocketResponse, data=response)
         if not socket_response.success:
             self.socket_connection_broken = True
-            self.log.error('Statusupdate failed: %s', response)
-            raise Exception(f'Statusupdate failed: {response}')
+            self.log.error('Status update failed (will try to reconnect): %s', response)
+            raise Exception(f'Status update failed. Response from learning loop: {response}')
 
         assert socket_response.payload is not None
 
@@ -457,7 +465,7 @@ class DetectorNode(Node):
             self.log.warning('Operation mode set to %s, but sync failed: %s', mode, e)
 
     def reload(self, reason: str):
-        '''provide a cause for the reload'''
+        """provide a cause for the reload"""
 
         self.log.info('########## reloading app because %s', reason)
         if os.path.isfile('/app/app_code/restart/restart.py'):
@@ -470,7 +478,7 @@ class DetectorNode(Node):
             self.log.error('could not reload app')
 
     async def get_detections(self,
-                             raw_image: np.ndarray,
+                             raw_image: bytes,
                              camera_id: Optional[str],
                              tags: List[str],
                              source: Optional[str] = None,
@@ -482,8 +490,7 @@ class DetectorNode(Node):
         It can be converted e.g. using cv2.imdecode(raw_image, cv2.IMREAD_COLOR)"""
 
         await self.detection_lock.acquire()
-        loop = asyncio.get_event_loop()
-        detections = await loop.run_in_executor(None, self.detector_logic.evaluate_with_all_info, raw_image, tags, source, creation_date)
+        detections = await run.io_bound(self.detector_logic.evaluate_with_all_info, raw_image, tags, source, creation_date)
         self.detection_lock.release()
 
         fix_shape_detections(detections)
@@ -491,21 +498,40 @@ class DetectorNode(Node):
         n_po, n_se = len(detections.point_detections), len(detections.segmentation_detections)
         self.log.debug('Detected: %d boxes, %d points, %d segs, %d classes', n_bo, n_po, n_se, n_cl)
 
-        if autoupload is None or autoupload == 'filtered':  # NOTE default is filtered
-            Thread(target=self.relevance_filter.may_upload_detections,
-                   args=(detections, camera_id, raw_image, tags, source, creation_date)).start()
+        autoupload = autoupload or 'filtered'
+        if autoupload == 'filtered' and camera_id is not None:
+            background_tasks.create(self.relevance_filter.may_upload_detections(
+                detections, camera_id, raw_image, tags, source, creation_date
+            ))
         elif autoupload == 'all':
-            Thread(target=self.outbox.save, args=(raw_image, detections, tags, source, creation_date)).start()
+            background_tasks.create(self.outbox.save(raw_image, detections, tags, source, creation_date))
         elif autoupload == 'disabled':
             pass
         else:
             self.log.error('unknown autoupload value %s', autoupload)
         return detections
 
-    async def upload_images(self, images: List[bytes], source: Optional[str], creation_date: Optional[str]):
-        loop = asyncio.get_event_loop()
+    async def upload_images(
+            self, *,
+            images: List[bytes],
+            image_metadata: Optional[ImageMetadata] = None,
+            tags: Optional[List[str]] = None,
+            source: Optional[str],
+            creation_date: Optional[str],
+            upload_priority: bool = False
+    ) -> None:
+        """Save images to the outbox using an asyncio executor.
+        Used by SIO and REST upload endpoints."""
+
+        if image_metadata is None:
+            image_metadata = ImageMetadata()
+        if tags is None:
+            tags = []
+
+        tags.append('picked_by_system')
+
         for image in images:
-            await loop.run_in_executor(None, self.outbox.save, image, ImageMetadata(), ['picked_by_system'], source, creation_date)
+            await self.outbox.save(image, image_metadata, tags, source, creation_date, upload_priority)
 
     def add_category_id_to_detections(self, model_info: ModelInformation, image_metadata: ImageMetadata):
         def find_category_id_by_name(categories: List[Category], category_name: str):

{learning_loop_node-0.13.6 → learning_loop_node-0.14.0}/learning_loop_node/detector/inbox_filter/relevance_filter.py

@@ -11,14 +11,16 @@ class RelevanceFilter():
         self.cam_histories: Dict[str, CamObservationHistory] = {}
         self.outbox: Outbox = outbox
 
-    def may_upload_detections(self,
-                              image_metadata: ImageMetadata,
-                              cam_id: str,
-                              raw_image: bytes,
-                              tags: List[str],
-                              source: Optional[str] = None,
-                              creation_date: Optional[str] = None
-                              ) -> List[str]:
+    async def may_upload_detections(self,
+                                    image_metadata: ImageMetadata,
+                                    cam_id: str,
+                                    raw_image: bytes,
+                                    tags: List[str],
+                                    source: Optional[str] = None,
+                                    creation_date: Optional[str] = None) -> List[str]:
+        """Check if the detection should be uploaded to the outbox.
+        If so, upload it and return the list of causes for the upload.
+        """
         for group in self.cam_histories.values():
             group.forget_old_detections()
 
@@ -30,5 +32,5 @@ class RelevanceFilter():
         if len(causes) > 0:
             tags = tags if tags is not None else []
             tags.extend(causes)
-            self.outbox.save(raw_image, image_metadata, tags, source, creation_date)
+            await self.outbox.save(raw_image, image_metadata, tags, source, creation_date)
         return causes