omdnotificationforwarder 4.6__tar.gz → 4.7__tar.gz

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/.gitignore

@@ -6,3 +6,4 @@ sdist
 examples/
 .opencode/
 notificationforwarder/tests/var/
+/openspec/

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: omdnotificationforwarder
-Version: 4.6
+Version: 4.7
 Summary: A framework for notification scripts for OMD
 Project-URL: Homepage, https://github.com/lausser/noteventificationforhandlerwarder
 Project-URL: Bug Tracker, https://github.com/lausser/noteventificationforhandlerwarder/issues
@@ -490,6 +490,46 @@ For broader confidence, run the full suite:
 pytest
 ```
 
+### Focused lint gate
+
+`notificationforwarder/.flake8` selects `E9,E722,F63,F7,F82` and sets
+`max-line-length = 120`. The check fails for undefined names, syntax errors,
+other selected structural errors, and bare `except` handlers.
+Undefined-name detection (F821) stays enabled: it is not suppressed globally,
+per file, or with inline `noqa`. Forwarders whose `logger` is injected by the
+component loader bind `logger = None`; they do not suppress F821.
+
+CI runs this gate on Python 3.12 with flake8 7.5.0. The `test` extra lists
+`flake8` without a pin; Python 3.9 and 3.10 cannot install flake8 7.5.0
+(that release requires Python >=3.11), which is why the gate job is pinned
+separately.
+
+Run the gate:
+
+```bash
+# from notificationforwarder/
+flake8 --config .flake8 src/notificationforwarder --count --statistics
+
+# from the repository root
+flake8 --config notificationforwarder/.flake8 notificationforwarder/src/notificationforwarder --count --statistics
+```
+
+Expect zero findings on a clean tree.
+
+Full diagnostic report, from `notificationforwarder/`. This is not the gate:
+codes it reports that the gate does not select are not certified fixed or
+harmless; reviewers read this report for those findings (examples on the
+current tree: E501, F401, E402):
+
+```bash
+flake8 --isolated src/notificationforwarder/ --max-line-length=120 --statistics --count
+```
+
+`tests/test_lint_gate.py` is the committed rejection probe: it proves the gate
+rejects an undefined name, fails closed on a missing config, and that the
+`logger = None` bindings do not hide other F821 findings. It runs flake8 from
+the same interpreter that has the `test` extra installed.
+
 ## Reporters
 
 Like *forwarder* and *formatter*, a *reporter* is an instance of a *NotificationReporter* class defined in a file named *reporter.py*. There is one class coming with notificationforwarder, the *NaemonlogReporter*. It's purpose it to write a message to the Naemon logfile. When notificationforwarder is run as a standalone script (and not triggered as a notificationhandler by Naemon), the *NaemonlogReporter* can nevertheless leave a line in the Naemon log.
@@ -527,6 +567,20 @@ If the receiver rejects a spooled event for good (for example because of an inva
 
 This means that in a busy monitoring environment where notifications flow regularly, spooled events get drained naturally — each new notification triggers a flush attempt on its way through.
 
+To monitor the size of the spool, use the line "spooling queue length". It is logged with the whole number of spooled events every time an event is spooled (level WARNING) and at the end of every flush (level INFO), so the value also goes down again while the spool drains. With the JSON logger, the number can be read with the regular expression `"message":"spooling queue length","queue_length":(\d+)`. The value is only logged when notificationforwarder runs, so it can be outdated when no notifications arrive. A check that does not depend on notifications can count the rows in the spool database directly:
+
+```
+sqlite3 $OMD_ROOT/var/tmp/notificationforwarder_<forwarder>_notifications.db "SELECT COUNT(*) FROM events_<forwarder>"
+```
+
+`<forwarder>` is the value of `--forwarder`, followed by `_` and the value of `--forwardertag` if one is set (for example `remedy_TEST`).
+
+### Correlating Spool Log Entries
+
+When a failed delivery is spooled, its `forward failed` line identifies the spool row and includes the stored raw event and its summary, if available. During replay, the delete, stays-in-spool, discard, could-not-format and delete-trash lines likewise include the row id and raw event, plus a non-empty summary when one exists. Failed replays also include any submit error. Every expired row gets its own id and raw-event line alongside the aggregate expired count; an undecodable expired row is logged with its id and payload text.
+
+The text logger includes these details in each message, for example `delete spooled event 1 summary <sum: disk full> raw event <{'description': 'disk full'}>`. The JSON logger exposes them in the `msg` object as `event_id`, `raw_event` (the full structured stored event), and `summary` when available. The move-to-end line links the old `event_id` to `new_event_id`; follow that link if an event receives a new spool id before deletion.
+
 ### Delivery Order
 
 Spooled events are stored and re-submitted in the order in which they failed. As long as every delivery succeeds, the receiver gets the events in the order in which they were created. After delivery failures, this is not guaranteed. A newer event can reach the receiver before an older one, for example a RECOVERY before its PROBLEM:

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/README.md

@@ -469,6 +469,46 @@ For broader confidence, run the full suite:
 pytest
 ```
 
+### Focused lint gate
+
+`notificationforwarder/.flake8` selects `E9,E722,F63,F7,F82` and sets
+`max-line-length = 120`. The check fails for undefined names, syntax errors,
+other selected structural errors, and bare `except` handlers.
+Undefined-name detection (F821) stays enabled: it is not suppressed globally,
+per file, or with inline `noqa`. Forwarders whose `logger` is injected by the
+component loader bind `logger = None`; they do not suppress F821.
+
+CI runs this gate on Python 3.12 with flake8 7.5.0. The `test` extra lists
+`flake8` without a pin; Python 3.9 and 3.10 cannot install flake8 7.5.0
+(that release requires Python >=3.11), which is why the gate job is pinned
+separately.
+
+Run the gate:
+
+```bash
+# from notificationforwarder/
+flake8 --config .flake8 src/notificationforwarder --count --statistics
+
+# from the repository root
+flake8 --config notificationforwarder/.flake8 notificationforwarder/src/notificationforwarder --count --statistics
+```
+
+Expect zero findings on a clean tree.
+
+Full diagnostic report, from `notificationforwarder/`. This is not the gate:
+codes it reports that the gate does not select are not certified fixed or
+harmless; reviewers read this report for those findings (examples on the
+current tree: E501, F401, E402):
+
+```bash
+flake8 --isolated src/notificationforwarder/ --max-line-length=120 --statistics --count
+```
+
+`tests/test_lint_gate.py` is the committed rejection probe: it proves the gate
+rejects an undefined name, fails closed on a missing config, and that the
+`logger = None` bindings do not hide other F821 findings. It runs flake8 from
+the same interpreter that has the `test` extra installed.
+
 ## Reporters
 
 Like *forwarder* and *formatter*, a *reporter* is an instance of a *NotificationReporter* class defined in a file named *reporter.py*. There is one class coming with notificationforwarder, the *NaemonlogReporter*. It's purpose it to write a message to the Naemon logfile. When notificationforwarder is run as a standalone script (and not triggered as a notificationhandler by Naemon), the *NaemonlogReporter* can nevertheless leave a line in the Naemon log.
@@ -506,6 +546,20 @@ If the receiver rejects a spooled event for good (for example because of an inva
 
 This means that in a busy monitoring environment where notifications flow regularly, spooled events get drained naturally — each new notification triggers a flush attempt on its way through.
 
+To monitor the size of the spool, use the line "spooling queue length". It is logged with the whole number of spooled events every time an event is spooled (level WARNING) and at the end of every flush (level INFO), so the value also goes down again while the spool drains. With the JSON logger, the number can be read with the regular expression `"message":"spooling queue length","queue_length":(\d+)`. The value is only logged when notificationforwarder runs, so it can be outdated when no notifications arrive. A check that does not depend on notifications can count the rows in the spool database directly:
+
+```
+sqlite3 $OMD_ROOT/var/tmp/notificationforwarder_<forwarder>_notifications.db "SELECT COUNT(*) FROM events_<forwarder>"
+```
+
+`<forwarder>` is the value of `--forwarder`, followed by `_` and the value of `--forwardertag` if one is set (for example `remedy_TEST`).
+
+### Correlating Spool Log Entries
+
+When a failed delivery is spooled, its `forward failed` line identifies the spool row and includes the stored raw event and its summary, if available. During replay, the delete, stays-in-spool, discard, could-not-format and delete-trash lines likewise include the row id and raw event, plus a non-empty summary when one exists. Failed replays also include any submit error. Every expired row gets its own id and raw-event line alongside the aggregate expired count; an undecodable expired row is logged with its id and payload text.
+
+The text logger includes these details in each message, for example `delete spooled event 1 summary <sum: disk full> raw event <{'description': 'disk full'}>`. The JSON logger exposes them in the `msg` object as `event_id`, `raw_event` (the full structured stored event), and `summary` when available. The move-to-end line links the old `event_id` to `new_event_id`; follow that link if an event receives a new spool id before deletion.
+
 ### Delivery Order
 
 Spooled events are stored and re-submitted in the order in which they failed. As long as every delivery succeeds, the receiver gets the events in the order in which they were created. After delivery failures, this is not guaranteed. A newer event can reach the receiver before an older one, for example a RECOVERY before its PROBLEM:

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/bin/notificationforwarder

@@ -78,7 +78,7 @@ Example for an HTTP-based reporter:
                       help='Logger type: text (default) or json, or custom logger module',
                       default='text')
     parser.add_argument('--version', action='version',
-                      version=f'%(prog)s 4.6')
+                      version=f'%(prog)s 4.7')
 
     args = parser.parse_args()
     if not hasattr(args, 'formatter'):

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/pyproject.toml

@@ -21,7 +21,7 @@ packages = ["src/notificationforwarder"]
 
 [project]
 name = "omdnotificationforwarder"
-version = "4.6"
+version = "4.7"
 authors = [
   { name="Gerhard Lausser", email="lausser@yahoo.com" },
 ]

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/baseclass.py

@@ -269,13 +269,11 @@ class NotificationForwarder(object):
             result = self.forward_formatted(formatted_event)
             success, report_payload, error_message = apply_forward_result(result)
             if not success and not formatted_event.is_heartbeat:
-                spooled = self.spool(raw_event)
-                if spooled:
-                    failure_context = {
-                        'formatted_event': formatted_event,
-                        'spooled': True,
-                        'status': 'failed',
-                    }
+                spool_success, event_id = self.spool(raw_event)
+                if spool_success:
+                    failure_context = self.spool_event_context(event_id, formatted_event, raw_event)
+                    failure_context['spooled'] = True
+                    failure_context['status'] = 'failed'
                     if error_message:
                         failure_context['exception'] = error_message
                     logger.warning("forward failed", failure_context)
@@ -400,19 +398,38 @@ class NotificationForwarder(object):
 
     def spool(self, raw_event):
         try:
-            self.spool_store.enqueue(raw_event)
+            event_id = self.spool_store.enqueue(raw_event)
             spooled_events = self.num_spooled_events()
             logger.warning("spooling queue length", {
                 'queue_length': spooled_events
             })
-            return True
+            return (True, event_id)
         except Exception as e:
             logger.critical("database error", {
                 'database_error': True,
                 'exception': e
             })
             logger.info("raw event details", {'raw_event': raw_event})
-            return False
+            return (False, None)
+
+    def spool_event_context(self, event_id, formatted_event, raw_event):
+        """Build the log context for a line which names a spool entry.
+
+        The id and the decoded raw event are always included, so every line
+        answers by itself which event is behind it. The summary is only
+        included when a formatted event exists and has a non-empty summary,
+        otherwise the key is omitted. The raw event carries the content even
+        when no formatted event exists (formatter failure).
+        """
+        context = {
+            'event_id': event_id,
+            'raw_event': raw_event,
+        }
+        if formatted_event is not None:
+            context['formatted_event'] = formatted_event
+            if getattr(formatted_event, 'summary', None):
+                context['summary'] = formatted_event.summary
+        return context
 
     def acquire_lock_with_retry(self, lock_file, max_attempts=3, base_delay=0.1):
         return acquire_lock_with_retry(lock_file, logger, max_attempts=max_attempts, base_delay=base_delay)
@@ -454,7 +471,8 @@ class NotificationForwarder(object):
             if locked:
                 self.baseclass_logs_summary = True
                 try:
-                    dropped = self.spool_store.prune_expired(self.max_spool_minutes)
+                    dropped_rows = self.spool_store.prune_expired(self.max_spool_minutes)
+                    dropped = len(dropped_rows)
                     replay_attempted = 0
                     replayed = 0
                     stayed_in_spool = 0
@@ -464,6 +482,24 @@ class NotificationForwarder(object):
                             'spooled_count': dropped,
                             'action': 'dropped'
                         })
+                        # one line per expired event, so the operator can see
+                        # which events were thrown away. decoding happens
+                        # outside the database write lock, wrapped per row,
+                        # so an undecodable payload cannot abort the flush
+                        for expired_id, expired_text in dropped_rows:
+                            try:
+                                expired_event = self.spool_store.decode(expired_text)
+                            except Exception as e:
+                                logger.critical("expired spool entry could not be decoded", {
+                                    'event_id': expired_id,
+                                    'payload_text': expired_text,
+                                    'action': 'expired_undecodable',
+                                    'exception': e,
+                                })
+                                continue
+                            context = self.spool_event_context(expired_id, None, expired_event)
+                            context['action'] = 'expired'
+                            logger.info("expired spooled event", context)
                     # walk through the rows which existed when the flush started.
                     # rows spooled meanwhile by concurrent processes are left for the
                     # next flush and each row is tried at most once. the first failed
@@ -484,13 +520,13 @@ class NotificationForwarder(object):
                         for id, text in id_events:
                             last_id = id
                             raw_event = self.spool_store.decode(text)
-                            formatted_event = self.format_event(raw_event)
+                            # Keep the stored event intact for logging even if plugins mutate their input.
+                            formatted_event = self.format_event(self.spool_store.decode(text))
                             if formatted_event and formatted_event.is_discarded:
                                 deleted_trash += 1
-                                logger.info("discard spooled event during replay", {
-                                    'event_id': id,
-                                    'action': 'discard_during_replay'
-                                })
+                                context = self.spool_event_context(id, formatted_event, raw_event)
+                                context['action'] = 'discard_during_replay'
+                                logger.info("discard spooled event during replay", context)
                                 self.spool_store.delete(id)
                             elif formatted_event:
                                 replay_attempted += 1
@@ -503,35 +539,29 @@ class NotificationForwarder(object):
                                 if success:
                                     self.spool_store.delete(id)
                                     replayed += 1
-                                    logger.info("delete spooled event", {
-                                        'spooled_count': 1,
-                                        'event_id': id,
-                                        'action': 'delete'
-                                    })
+                                    context = self.spool_event_context(id, formatted_event, raw_event)
+                                    context['spooled_count'] = 1
+                                    context['action'] = 'delete'
+                                    logger.info("delete spooled event", context)
                                 else:
                                     stayed_in_spool += 1
-                                    context = {
-                                        'event_id': id,
-                                        'action': 'stays_in_spool'
-                                    }
+                                    context = self.spool_event_context(id, formatted_event, raw_event)
                                     if error_message:
                                         context['exception'] = error_message
+                                    context['action'] = 'stays_in_spool'
                                     logger.critical("event stays in spool", context)
                                     self.flush_stopped_at = id
                                     break
                             else:
                                 deleted_trash += 1
-                                logger.critical("could not format spooled event", {
-                                    'raw_event': raw_event,
-                                    'event_id': id,
-                                    'spooled_count': 1,
-                                    'action': 'could_not_format'
-                                })
+                                context = self.spool_event_context(id, None, raw_event)
+                                context['spooled_count'] = 1
+                                context['action'] = 'could_not_format'
+                                logger.critical("could not format spooled event", context)
                                 self.spool_store.delete(id)
-                                logger.info("delete trash event", {
-                                    'event_id': id,
-                                    'action': 'delete_trash'
-                                })
+                                context = self.spool_event_context(id, None, raw_event)
+                                context['action'] = 'delete_trash'
+                                logger.info("delete trash event", context)
                         if self.flush_stopped_at is not None:
                             # only the rows this flush was responsible for, not
                             # the ones spooled concurrently after the snapshot
@@ -548,6 +578,11 @@ class NotificationForwarder(object):
                         'dropped_count': dropped,
                     })
                     self.spool_store.commit()
+                    # same line as in spool(), so a monitored queue length
+                    # also goes down again while the spool drains
+                    logger.info("spooling queue length", {
+                        'queue_length': self.spool_store.count()
+                    })
                 except Exception as e:
                     logger.critical("database flush+resubmit failed", {
                         'database_error': True,

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/example/forwarder.py

@@ -3,6 +3,8 @@ import time
 import logging
 from notificationforwarder.baseclass import NotificationForwarder, NotificationFormatter, timeout
 
+logger = None  # Set by the component loader.
+
 
 class ExampleForwarder(NotificationForwarder):
     def __init__(self, opts):

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/json/logger.py

@@ -89,7 +89,7 @@ class JsonLogger(NotificationLogger):
                         msg_data["exception"]["trace"] = ''.join(
                             traceback.format_exception(*context['exc_info'])
                         )
-                    except:
+                    except Exception:
                         pass
             else:
                 msg_data["exception"] = str(exc)
@@ -139,6 +139,18 @@ class JsonLogger(NotificationLogger):
             msg_data["reporter_class"] = rpt.__class__.__name__
             msg_data["reporter_module"] = rpt.__module_file__
 
+        # Add the full decoded raw event as its own structured field, even
+        # when a formatted event is also present. This is the payload as it
+        # was stored in the spool and is not reduced to Naemon keys.
+        if 'raw_event' in context and context['raw_event'] is not None:
+            msg_data["raw_event"] = context['raw_event']
+        # The summary of the spooled event, only present when the formatted
+        # event exists and has a non-empty summary
+        if 'summary' in context and context['summary']:
+            msg_data["summary"] = context['summary']
+        if 'payload_text' in context:
+            msg_data["payload_text"] = context['payload_text']
+
         # Add event data for errors
         if 'event_data' in context:
             msg_data["event_data"] = context['event_data']

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/nostr/forwarder.py

@@ -7,6 +7,8 @@ except ImportError as exc:
 
 from notificationforwarder.baseclass import NotificationForwarder, timeout
 
+logger = None  # Set by the component loader.
+
 
 class NostrForwarder(NotificationForwarder):
     def __init__(self, opts):

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/rabbitmq/forwarder.py

@@ -3,6 +3,8 @@ import json
 import logging
 from notificationforwarder.baseclass import NotificationForwarder, NotificationFormatter, timeout
 
+logger = None  # Set by the component loader.
+
 
 class RabbitmqForwarder(NotificationForwarder):
     def __init__(self, opts):
@@ -50,7 +52,7 @@ class RabbitmqForwarder(NotificationForwarder):
             logger.critical("rabbitmq connect failed with error {}".format(e))
             return False
 
-    def disconnect(iself):
+    def disconnect(self):
         try:
             self.connection.close()
         except Exception as e:

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/spool.py

@@ -45,18 +45,28 @@ class SpoolStore:
         sql_insert = "INSERT INTO " + self.table_name + "(payload) VALUES (?)"
         with self._lock:
             self.cursor.execute(sql_insert, (json.dumps(raw_event),))
+            # capture the lastrowid inside the lock, a later INSERT on the
+            # shared cursor by another thread would overwrite it
+            event_id = self.cursor.lastrowid
             self.connection.commit()
+            return event_id
 
     def prune_expired(self, max_spool_minutes):
         outdated = int(time.time() - 60 * max_spool_minutes)
-        sql_delete = "DELETE FROM " + self.table_name + " WHERE CAST(STRFTIME('%s', timestamp) AS INTEGER) < ?"
+        sql_expire = "WHERE CAST(STRFTIME('%s', timestamp) AS INTEGER) < ?"
         with self._lock:
-            self.cursor.execute(sql_delete, (outdated,))
-            dropped = self.cursor.rowcount
-            # commit immediately, otherwise the implicit transaction keeps
-            # the database write-locked for other processes during a flush
-            self.connection.commit()
-            return dropped
+            self.cursor.execute("BEGIN IMMEDIATE")
+            try:
+                # The transaction prevents another connection from changing
+                # the rows between the selection and deletion.
+                self.cursor.execute("SELECT id, payload FROM " + self.table_name + " " + sql_expire, (outdated,))
+                expired = self.cursor.fetchall()
+                self.cursor.execute("DELETE FROM " + self.table_name + " " + sql_expire, (outdated,))
+                self.connection.commit()
+                return expired
+            except Exception:
+                self.connection.rollback()
+                raise
 
     def max_id(self):
         with self._lock:

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/text/logger.py

@@ -36,6 +36,24 @@ class TextLogger(NotificationLogger):
         else:
             log_func(full_message)
 
+    def _spool_event_line(self, lead, context):
+        """Render a line which names a spool entry.
+
+        The lead already contains the event id. The summary is appended when
+        the context carries one, the raw event is the payload stored in the
+        spool, and the submit error ends the line when it exists."""
+        description = [lead]
+        if context.get('summary'):
+            description.append(f"summary <{context['summary']}>")
+        if 'raw_event' in context:
+            description.append(f"raw event <{context['raw_event']}>")
+        exception = context.get('exception')
+        if exception:
+            description.append(f"with exception <{exception}>")
+        if 'payload_text' in context:
+            description.append(f"payload <{context['payload_text']}>")
+        return " ".join(description)
+
     def _build_message(self, message, context):
         """
         Build text log message from structured context
@@ -70,6 +88,12 @@ class TextLogger(NotificationLogger):
         if 'raw_event' in context and message.startswith("raw event"):
             parts.append(f"caused error {context.get('exception', 'unknown')}")
 
+        # The spool-time line for a failed delivery which was spooled:
+        # names the spool row id, the stored raw event and the summary
+        if message == "forward failed" and 'event_id' in context and 'raw_event' in context:
+            return self._spool_event_line(
+                f"forward failed, spool id {context['event_id']}", context)
+
         # Add formatter info for formatting errors
         if 'event_data' in context and 'formatter_instance' in context:
             fmt = context['formatter_instance']
@@ -98,10 +122,13 @@ class TextLogger(NotificationLogger):
             return (f"event {context.get('event_id', '')} was rejected while the receiver accepts "
                     f"other events and moved to the end of the spool as {context.get('new_event_id', '')}")
 
-        # Handle spooled count messages
-        if 'spooled_count' in context and 'action' in context:
+        # Handle action-typed spool operations. The counting operations carry
+        # a spooled_count, the per-event lines describe one spooled row each
+        # with id, summary (when one exists) and raw event.
+        if 'action' in context:
             action = context['action']
-            count = context['spooled_count']
+            count = context.get('spooled_count')
+            event_id = context.get('event_id', '')
             if action == 'resend':
                 return f"there are {count} spooled events to be re-sent"
             elif action == 'dropped':
@@ -109,13 +136,23 @@ class TextLogger(NotificationLogger):
             elif action == 'could_not_submit':
                 return f"{count} spooled events could not be submitted"
             elif action == 'delete':
-                return f"delete spooled event {context.get('event_id', '')}"
+                return self._spool_event_line(f"delete spooled event {event_id}", context)
             elif action == 'stays_in_spool':
-                return f"event {context.get('event_id', '')} stays in spool"
+                return self._spool_event_line(f"event {event_id} stays in spool", context)
             elif action == 'delete_trash':
-                return f"delete trash event {context.get('event_id', '')}"
+                return self._spool_event_line(f"delete trash event {event_id}", context)
+            elif action == 'discard_during_replay':
+                return self._spool_event_line(
+                    f"discard spooled event during replay {event_id}", context)
+            elif action == 'expired':
+                return self._spool_event_line(
+                    f"expired spooled event {event_id}", context)
+            elif action == 'expired_undecodable':
+                return (f"expired spool entry {event_id} could not be decoded, "
+                        f"payload <{context.get('payload_text', '')}> with exception <{context.get('exception', '')}>")
             elif action == 'could_not_format':
-                return f"could not format spooled {context.get('raw_event', '')}. sorry, but i will delete this garbage with id {context.get('event_id', '')}"
+                return (f"could not format spooled {context.get('raw_event', '')}. "
+                        f"sorry, but i will delete this garbage with id {event_id}")
 
         # Add queue length warnings
         if 'queue_length' in context:

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/src/notificationforwarder/webhook/forwarder.py

@@ -4,6 +4,8 @@ requests.packages.urllib3.disable_warnings(requests.packages.urllib3.exceptions.
 import os
 from notificationforwarder.baseclass import NotificationForwarder, timeout
 
+logger = None  # Set by the component loader.
+
 class WebhookForwarder(NotificationForwarder):
     def __init__(self, opts):
         super(self.__class__, self).__init__(opts)

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/tests/TESTS.md

@@ -36,6 +36,7 @@ This document describes every test file, test case, and the code paths they cove
 - [test_oauth2webhook.py](#test_oauth2webhookpy)
 - [test_reporter.py](#test_reporterpy)
 - [test_logger.py](#test_loggerpy)
+- [test_lint_gate.py](#test_lint_gatepy)
 
 ---
 
@@ -193,6 +194,16 @@ Spooled events 0, 1, 2 (ids 1, 2, 3) are replayed through a mocked `submit()`; t
 | `test_no_move_when_event_is_already_last` | The rejected event is the only one → stays with id 1, no move log | Moving the last event would change nothing. |
 | `test_moved_event_still_expires` | Event 0 spooled 4 minutes ago is moved, then `prune_expired(3)` drops it | Retention is measured from the original spool time. |
 
+### class TestQueueLengthAfterFlush
+
+| Test | What it verifies | Intention |
+|------|------------------|-----------|
+| `test_queue_length_zero_after_drain` | A flush that delivers all three spooled events logs info "spooling queue length" with `queue_length` 0 | A monitored queue length goes back down once the spool drained. |
+| `test_queue_length_after_stopped_flush` | A flush that stops at the first failure logs `queue_length` 3 | The value is the remaining spool, also after a stopped flush. |
+| `test_queue_length_includes_concurrently_spooled_events` | Another connection spools an event while the flush delivers all three → `queue_length` 1 | The value is the whole spool, not only the flush's snapshot. |
+| `test_queue_length_not_logged_without_lock` | Flush lock not acquired → no "spooling queue length" line | Only a flush that ran reports a value. |
+| `test_queue_length_json_format` | With the JSON logger, the regex `"message":"spooling queue length","queue_length":(\d+)` reads 0 after the drain | The documented monitoring regex works on real log output. |
+
 ### class TestSpoolStore
 
 Low-level `SpoolStore` operations against a real SQLite database.

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/tests/test_builtin_plugins_focus.py

@@ -1,9 +1,11 @@
 import json
+import importlib.util
 import logging
 import os
 import shutil
 import sys
 from types import SimpleNamespace
+from unittest.mock import Mock
 
 import pytest
 
@@ -58,6 +60,22 @@ def _formatted_event(eventopts):
     return FormattedEvent(dict(eventopts))
 
 
+def test_rabbitmq_disconnect_closes_connection_without_broker(monkeypatch):
+    monkeypatch.setitem(sys.modules, "pika", SimpleNamespace())
+    path = os.path.join(OMD_ROOT, "..", "src", "notificationforwarder", "rabbitmq", "forwarder.py")
+    spec = importlib.util.spec_from_file_location("rabbitmq_disconnect_test", path)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    connection = Mock()
+    runner = SimpleNamespace(connection=connection)
+
+    module.RabbitmqForwarder.disconnect(runner)
+    connection.close.assert_called_once_with()
+    connection.close.side_effect = RuntimeError("close failed")
+    module.RabbitmqForwarder.disconnect(runner)
+    assert connection.close.call_count == 2
+
+
 def test_builtin_formatters_render_expected_payloads(setup):
     cases = [
         (
@@ -186,7 +204,7 @@ def test_webhook_forward_failure_spools_and_handles_spool_errors(setup, monkeypa
     )
 
     spooled = []
-    monkeypatch.setattr(forwarder, "spool", lambda raw_event: spooled.append(raw_event) or True)
+    monkeypatch.setattr(forwarder, "spool", lambda raw_event: spooled.append(raw_event) or (True, 1))
     forwarder.forward({"description": "needs retry"})
     assert spooled and spooled[0]["description"] == "needs retry"
 
@@ -200,7 +218,7 @@ def test_webhook_forward_failure_spools_and_handles_spool_errors(setup, monkeypa
         "post",
         lambda *args, **kwargs: SimpleNamespace(status_code=500, text="nope", reason="bad"),
     )
-    monkeypatch.setattr(forwarder2, "spool", lambda raw_event: False)
+    monkeypatch.setattr(forwarder2, "spool", lambda raw_event: (False, None))
     forwarder2.forward({"description": "cannot persist"})
 
 

{omdnotificationforwarder-4.6 → omdnotificationforwarder-4.7}/tests/test_classes.py

@@ -150,7 +150,7 @@ def test_example_forwarder_forward_timeout(setup):
     log = open(get_logfile(example)).read()
     # this is the global log, written by the baseclass
     assert "submit ran into a timeout" in log
-    assert "spooled <sum: halo i bims 1 alarm vong naemon her>" in log
+    assert "forward failed, spool id 1 summary <sum: halo i bims 1 alarm vong naemon her>" in log
     assert "WARNING - spooling queue length is 1" in log
     eventopts = {
         "description": "halo i bim au 1 alarm vong naemon her",
@@ -159,7 +159,7 @@ def test_example_forwarder_forward_timeout(setup):
     example = notificationforwarder.baseclass.new("example", None, "example", True, True,  forwarderopts)
     example.forward(eventopts)
     log = open(get_logfile(example)).read()
-    assert "spooled <sum: halo i bim au 1 alarm vong naemon her>" in log
+    assert "forward failed, spool id 2 summary <sum: halo i bim au 1 alarm vong naemon her>" in log
     assert "WARNING - spooling queue length is 2" in log
     # now the last two events were spooled and are in the database