@trillboards/edge-sdk 0.2.81 → 0.2.84

package/deploy/detector/README.md

@@ -1,8 +1,9 @@
 # `trillboards-detector` — the on-device object-detector service
 
-RF-DETR Medium, fp32 ONNX, served over loopback on **127.0.0.1:9380**. It is the
-cascade gate that sits in front of the on-device VLM: at the ANALYZE cadence it
-removes ~84.7% of 5,645 ms VLM calls for ~2.1% of events.
+A LIBRARY of detectors, served over loopback on **127.0.0.1:9380**. One is
+shipped today — RF-DETR Medium, fp32 ONNX — and it is the cascade gate that sits
+in front of the on-device VLM: at the ANALYZE cadence it removes ~84.7% of
+5,645 ms VLM calls for ~2.1% of events.
 
 The decision record — which model, why not the alternatives, and every number
 above — is [`docs/runbooks/cctv-detector-tier.md`](../../../../../docs/runbooks/cctv-detector-tier.md).
@@ -21,37 +22,116 @@ POST /detect   { image_b64, classes?, min_confidence }
      200       { detections: [{ label, confidence, box: {x,y,width,height} }],
                  inference_ms, model }
      400       the request cannot be acted on (bad base64, empty body,
-                 min_confidence outside [0,1], every requested class unknown)
+                 min_confidence outside [0,1], ANY requested class unknown)
      413       body over 32 MiB
      500       inference failed, or a detection could not be put on the wire
      503       no usable model — NEVER a 200 with an empty array
 
-GET  /health   200 when a load completed AND a probe inference succeeded
-               503 otherwise, with the reason. Runs no inference.
+GET  /health   200 when the PINNED model's load completed AND its probe
+                 inference succeeded; 503 otherwise, with the reason. Also
+                 reports which models are resident. Runs no inference.
 ```
 
 * `label` is the class **name**, never an index.
 * `box` is in **pixels of the submitted frame**, not normalised.
 * `classes` filters at the **service**, so the wire stays small. Absent means
   every class; an empty array is a 400, never a synonym for "all".
+* `model` names the **set that answered** — `rfdetr_medium`, or
+  `rfdetr_medium+ppe` when a request spans two. The client types it
+  `string | null` and does not branch on it, so a joined string is contract-safe.
 
-Three properties are load-bearing rather than stylistic:
+## The model library
 
-**Down is never empty.** A missing model is 503 and a failed inference is 500.
-Neither may become `200 []`. A cascade that reads "detector broken" as "venue
-quiet" suppresses every alert while every health surface stays green.
+[`docs/runbooks/sensing-model-library.md`](../../../../../docs/runbooks/sensing-model-library.md)
+is the architecture; §3 is what this service implements. **The wire contract
+carries no model selector and it stays that way** — the connector must not hold
+the catalogue, and `classes` already expresses the request completely, so the
+service owns class → model routing because it is the only component that knows
+what is resident on that box right now.
 
-**We refuse what the client would refuse.** `build_detect_response` re-checks
-every detection against the client's own fifteen rules and answers 500 rather
-than emitting a payload the client will reject — because a rejected payload
-costs the whole frame *and* is indistinguishable, from the client's side, from
-the service being down.
+```
+labels.py     one committed ModelVocabulary per model (slots -> class names,
+              logit width, measured RAM). ClassIndex is the UNION over them.
+registry.py   name -> session map, the class index, lazy load, LRU eviction
+              under a RAM budget, and the merge. STDLIB ONLY.
+loader.py     the poll loop + the real ORT session factory and probe frame.
+```
+
+* **A class routes to the FIRST catalogue model that names it.** Adding a
+  specialist can never move an existing class off the model it was scored on —
+  `person` stays on `rfdetr_medium` even after a PPE model that also emits it.
+* **A committed vocabulary advertises nothing until its artifact is on disk.**
+  `ClassIndex` is built over models whose file is present, so a vocabulary added
+  before its ONNX ships contributes no requestable class. That is the
+  `yolov8_nano` failure made unreachable rather than merely forbidden.
+* **The pinned model is eager and never evicted; specialists are lazy.**
+  `/health` has always meant "a load completed and a probe inference succeeded",
+  and `LocalDetectorClient.isHealthy()` turns anything else into "send the frame
+  to the VLM" — so the general gate has to be resident for the service to claim
+  it is up. One 632 MiB model fits any budget that can hold anything.
+* **Every model runs a probe inference before it may answer**, on its first
+  request and on every reload. A session that constructs is not a session that
+  infers.
+* **A request spanning two models runs both and merges**, in catalogue order.
+  `inference_ms` is the SUM, because they run serially on one box.
+
+### The RAM ceiling is INJECTED — this service owns no RAM policy
+
+`--ram-budget-mb`, or `TRILLBOARDS_DETECTOR_RAM_BUDGET_MB`. **There is no
+default and there must not be one.**
+
+There is already an allocation authority and it is server-side:
+`sensingProfileService.selectModelForDeviceTier` picks a model from
+`deviceRamMb` + `deviceTier`, and `registerInternalControlSurface` gates each
+model on `ramMb >= model.min_ram_mb`. A ceiling chosen *here* would be a second
+authority that cannot see the first — the server deciding what a device gets
+while the service independently decides what it keeps. That is the
+`yolov8_nano` shape: advertised by one component, unloadable by another, for six
+months, with every surface green.
+
+So the ceiling is a parameter, and **absent is a state, not a value to fill in**:
+
+| ceiling | what happens |
+| --- | --- |
+| injected | the resident set must fit it; over it, a request is **refused with what did not fit**, never answered with the half that did |
+| **not injected** | the pinned model still loads and answers — the server already chose it, and holding what we were told to hold is the mechanism doing its job — and **any second model is refused**, naming who supplies the ceiling |
+
+`/health` carries `ram_ceiling_injected` so a reader can see which of those two
+states the box is in. The service does not exit when nothing was injected: all
+three supervisors restart on a non-zero exit, and this file's own rule for a
+required input that has not arrived is *start, and report the reason*.
+
+**Eviction is a last-resort safety valve, not the allocator.** It logs a WARNING
+naming the box as oversubscribed against what it was allocated, and `/health`
+carries a top-level `evictions` count — because a service quietly thrashing
+under a ceiling looks exactly like one working fine.
+
+### The footprint column the server does not have
+
+`min_ram_mb` is a **gate**. `model_size_mb` is a **file size**. `gemma_4_e2b`
+carries 8000 and 2580 against a *measured working set* of 3.31 GB — three
+different quantities, and the one portfolio admission needs, **resident
+footprint**, is in none of them. Summing `min_ram_mb` over-rejects; summing
+`model_size_mb` under-rejects.
+
+So each load is measured, after the probe (a session that has not run a forward
+pass has not allocated its working buffers), and published per model:
+
+```
+rss_delta_bytes   bytes the process grew across this model's load
+rss_source        proc-statm       Linux, CURRENT rss, exact
+                  psapi-workingset Windows, CURRENT working set, exact
+                  rusage-maxrss    macOS/fallback, a HIGH-WATER MARK — the
+                                   delta is a LOWER BOUND and 0 is legitimate
+```
 
-**Health reports a load that actually happened.** `GET /v1/models` on litert-lm
-answers 200 without ever building an engine, which is how a UTF-8 BOM in its
-config 500'd every frame on every Windows connector while the installer's health
-check *and* its CPU-fallback re-check both passed. So this loader builds the
-session, runs a real probe inference, and `/health` reports that outcome.
+The instrument is named with the number because the three platforms do not
+answer the same question, and a reader that cannot tell them apart will average
+a high-water mark into a current reading.
+
+**It is reported and never used for admission.** Admission follows the
+*committed* `approx_ram_bytes`; admitting on numbers only this service can see
+would be the second authority again, by a different route.
 
 ## Where it runs
 
@@ -87,6 +167,11 @@ reclaims the old bytes. There is no GC for abandoned model dirs anywhere on this
 path and the existing OTA already orphans 752 MiB per update; a version-named
 directory would add to that.
 
+**One directory per model NAME is also what makes the library work**: each
+catalogue entry resolves its own `models/<name>/model.onnx`, a specialist's
+artifact arriving later widens the requestable vocabulary within one poll
+interval with no restart, and its disappearing narrows it again.
+
 ## Tests
 
 ```bash
@@ -94,7 +179,9 @@ directory would add to that.
 python3 -m unittest discover -s . -p "test_*.py" -v
 ```
 
-Runs in `guards-fast`. **Everything except the post-processing cases needs
+Runs in `guards-fast`. `test_registry.py` covers the whole residency policy —
+routing, lazy load, eviction, and every refusal — with FAKE sessions, so it runs
+there too. **Everything except the post-processing cases needs
 nothing but the standard library** — those are the only ones that touch numpy,
 and they skip themselves with a stated reason when it is absent. That invariant
 is the durable claim; the counts (45 of 60 on 2026-08-24) are a snapshot and go

package/deploy/detector/bench/verify_against_reference.py

@@ -33,6 +33,7 @@ from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 
+from trillboards_detector.labels import RFDETR_MEDIUM  # noqa: E402
 from trillboards_detector.session import OnnxDetector  # noqa: E402
 
 
@@ -75,7 +76,11 @@ def main() -> int:
     )
 
     # ── this service ─────────────────────────────────────────────────────────
-    detector = OnnxDetector(Path(args.onnx), "rfdetr_medium")
+    # The VOCABULARY, not just the name: a session labels its slots from the
+    # committed vocabulary it was built with, so the control has to hand over
+    # the same one the service would — otherwise it would compare the reference
+    # against a differently-labelled run and still report agreement.
+    detector = OnnxDetector(Path(args.onnx), RFDETR_MEDIUM)
     batch = detector.detect(image_path.read_bytes(), args.threshold, None)
     ours_rows = sorted(
         (d.label, round(d.confidence, 4), (round(d.x, 2), round(d.y, 2), round(d.x + d.width, 2), round(d.y + d.height, 2)))

package/deploy/detector/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "trillboards-detector"
-version = "0.1.0"
+version = "0.2.0"
 description = "On-device object-detector service for the Trillboards BYO-CCTV connector (loopback HTTP, ONNX Runtime)."
 requires-python = ">=3.10"
 

package/deploy/detector/tests/test_loader.py

@@ -0,0 +1,188 @@
+"""The half of the loader that needs a real runtime to even import.
+
+`loader.py` imports `session.py`, which imports onnxruntime, numpy and Pillow —
+so unlike `test_registry.py` these cases cannot run on a bare interpreter and
+skip themselves with a stated reason, exactly as `test_session.py` does. The
+residency policy itself is deliberately NOT here: it lives in `registry.py`,
+which imports none of that, so every rule about routing, eviction and refusal is
+exercised where the artifact is absent.
+
+What is left for this file is the catalogue construction and the ONE-SNAPSHOT
+property of `/health` — both of which are about `DetectorLoader` specifically.
+"""
+
+from __future__ import annotations
+
+import unittest
+
+try:
+    from trillboards_detector.labels import RFDETR_MEDIUM, ModelVocabulary
+    from trillboards_detector.loader import DetectorLoader, build_catalogue
+
+    HAVE_RUNTIME = True
+except ImportError:  # pragma: no cover - exercised only on a bare interpreter
+    HAVE_RUNTIME = False
+
+PPE = None
+if HAVE_RUNTIME:
+    PPE = ModelVocabulary(
+        model_name="ppe",
+        slot_names={0: "glove", 1: "hairnet"},
+        logit_slots=2,
+        approx_ram_bytes=400 * 1024 * 1024,
+    )
+
+
+@unittest.skipUnless(HAVE_RUNTIME, "onnxruntime/numpy/Pillow are not installed")
+class TestBuildCatalogue(unittest.TestCase):
+    def test_the_pinned_model_comes_first_so_it_wins_every_class_tie(self):
+        """Catalogue order IS the disambiguation rule. `person` must keep coming
+        from the model the 98.8/98.8 was measured on, whatever a later
+        specialist also emits."""
+        entries = build_catalogue("rfdetr_medium", None, (PPE, RFDETR_MEDIUM), {"HOME": "/h"})
+        self.assertEqual([e.name for e in entries], ["rfdetr_medium", "ppe"])
+        self.assertTrue(entries[0].pinned)
+        self.assertFalse(entries[1].pinned)
+
+    def test_every_entry_resolves_its_own_per_model_directory(self):
+        entries = build_catalogue("rfdetr_medium", None, (RFDETR_MEDIUM, PPE), {"HOME": "/h"})
+        self.assertEqual(
+            [str(e.path) for e in entries],
+            ["/h/.trillboards/models/rfdetr_medium/model.onnx",
+             "/h/.trillboards/models/ppe/model.onnx"],
+        )
+
+    def test_an_explicit_path_overrides_only_the_pinned_entry(self):
+        """`--model` / `TRILLBOARDS_DETECTOR_MODEL` is the bench escape hatch and
+        it names ONE artifact, so it must not relocate every specialist onto the
+        same file."""
+        from pathlib import Path
+
+        entries = build_catalogue(
+            "rfdetr_medium", Path("/bench/rfdetr.onnx"), (RFDETR_MEDIUM, PPE), {"HOME": "/h"}
+        )
+        self.assertEqual(str(entries[0].path), "/bench/rfdetr.onnx")
+        self.assertEqual(str(entries[1].path), "/h/.trillboards/models/ppe/model.onnx")
+
+    def test_a_default_model_with_no_committed_vocabulary_is_refused(self):
+        """A name with no class list is a model nothing can label — `yolov8_nano`
+        exactly, and it must fail at start rather than in a venue."""
+        with self.assertRaisesRegex(ValueError, "no committed vocabulary"):
+            build_catalogue("yolov8_nano", None, (RFDETR_MEDIUM,), {"HOME": "/h"})
+
+
+@unittest.skipUnless(HAVE_RUNTIME, "onnxruntime/numpy/Pillow are not installed")
+class TestHealthIsOneRead(unittest.TestCase):
+    def test_info_takes_exactly_one_snapshot(self):
+        """`/health` must describe ONE moment.
+
+        Reading the top-level `state` from a second snapshot lets the watcher
+        thread move between the two, so the payload could answer
+        `state: "ready"` next to a `models` map showing the pinned model still
+        loading. Neither half would be wrong and the pair would describe no
+        moment — the same shape `postgres-tls-uri-override.md` records for two
+        scans a minute apart.
+        """
+        loader = DetectorLoader(
+            model_path="/nonexistent/model.onnx",
+            model_name="rfdetr_medium",
+            vocabularies=(RFDETR_MEDIUM, PPE),
+            log=lambda _m: None,
+        )
+        real = loader.registry.snapshot
+        calls = []
+
+        def counting():
+            calls.append(1)
+            return real()
+
+        loader.registry.snapshot = counting
+        payload = loader.info()
+        self.assertEqual(len(calls), 1, "`/health` read the registry twice; it must describe one moment")
+        # And it is still the payload every existing reader parses.
+        self.assertEqual(payload["model"], "rfdetr_medium")
+        self.assertEqual(sorted(payload["models"]), ["ppe", "rfdetr_medium"])
+        self.assertEqual(payload["resident_models"], [])
+
+    def test_before_the_first_stat_the_answer_is_loading_and_names_no_reason(self):
+        """`absent` is a CLAIM about the disk, and at construction nobody has
+        looked. The service starts before its watcher thread does, so a probe
+        can land in that window — and the single-model loader this replaced
+        correctly said `loading` there. Reporting `absent` for a file we have not
+        yet stat'd is a fact asserted without evidence, which is the failure
+        shape this whole service is written against."""
+        loader = DetectorLoader(
+            model_path="/nonexistent/model.onnx",
+            model_name="rfdetr_medium",
+            vocabularies=(RFDETR_MEDIUM, PPE),
+            log=lambda _m: None,
+        )
+        payload = loader.info()
+        self.assertEqual(payload["state"], "loading")
+        self.assertNotIn("reason", payload)
+
+    def test_after_the_first_stat_a_missing_artifact_is_unavailable_WITH_a_reason(self):
+        """And once we HAVE looked, the answer must carry why — the OTA has not
+        landed, which is a different problem from a corrupt artifact and the one
+        the installers' post-install probes print."""
+        loader = DetectorLoader(
+            model_path="/nonexistent/model.onnx",
+            model_name="rfdetr_medium",
+            vocabularies=(RFDETR_MEDIUM, PPE),
+            log=lambda _m: None,
+        )
+        loader.registry.refresh()
+        payload = loader.info()
+        self.assertEqual(payload["state"], "unavailable")
+        self.assertIn("arrives over the air", payload["reason"])
+        self.assertEqual(payload["models"]["rfdetr_medium"]["state"], "absent")
+
+    def test_the_ceiling_state_and_the_eviction_count_reach_the_payload(self):
+        """The registry knowing a fact is not the same as `/health` carrying it.
+
+        Both of these were computed in `snapshot()` and dropped on the floor by
+        `info()` for a commit — the registry was honest and the wire was silent,
+        which is the exact shape of a consumer with no producer. A reader has to
+        be able to see that NO authority has set a ceiling (distinct from a
+        ceiling of some number), and that a box is evicting at all.
+        """
+        loader = DetectorLoader(
+            model_path="/nonexistent/model.onnx",
+            model_name="rfdetr_medium",
+            vocabularies=(RFDETR_MEDIUM, PPE),
+            log=lambda _m: None,
+        )
+        payload = loader.info()
+        self.assertIn("ram_ceiling_injected", payload)
+        self.assertFalse(payload["ram_ceiling_injected"])
+        self.assertIsNone(payload["ram_budget_bytes"])
+        self.assertEqual(payload["evictions"], 0)
+
+        told = DetectorLoader(
+            model_path="/nonexistent/model.onnx",
+            model_name="rfdetr_medium",
+            ram_budget_bytes=1024 * 1024 * 1024,
+            vocabularies=(RFDETR_MEDIUM, PPE),
+            log=lambda _m: None,
+        )
+        self.assertTrue(told.info()["ram_ceiling_injected"])
+
+    def test_the_top_level_state_stays_the_three_words_that_shipped(self):
+        """`idle` and `absent` have no top-level equivalent: every existing
+        reader — three installers' post-install probes, the integration test —
+        was written against ready/loading/unavailable, and
+        `LocalDetectorClient.isHealthy()` turns anything but 2xx into "send the
+        frame to the VLM"."""
+        for pinned, expected in [
+            ("ready", "ready"),
+            ("loading", "loading"),
+            ("idle", "unavailable"),
+            ("absent", "unavailable"),
+            ("unavailable", "unavailable"),
+        ]:
+            with self.subTest(pinned=pinned):
+                self.assertEqual(DetectorLoader._top_level_state(pinned), expected)
+
+
+if __name__ == "__main__":
+    unittest.main()

package/deploy/detector/tests/test_modelpath.py

@@ -16,16 +16,25 @@ import unittest
 from pathlib import Path
 
 from trillboards_detector.labels import (
+    BUILTIN_VOCABULARIES,
     COCO91_CLASS_NAMES,
     EXPECTED_LOGIT_SLOTS,
-    resolve_requested_classes,
+    RFDETR_MEDIUM,
+    ClassIndex,
 )
 from trillboards_detector.modelpath import (
     DEFAULT_MODEL_NAME,
     resolve_data_dir,
     resolve_model_path,
+    resolve_ram_budget_mb,
 )
 
+#: The single-model index this file's vocabulary assertions are about. It used
+#: to be a module-level constant (`KNOWN_CLASS_NAMES`), which is exactly what
+#: made a specialist inexpressible; the resolution rules it tests did not
+#: change, only where the vocabulary comes from.
+COCO_INDEX = ClassIndex([RFDETR_MEDIUM])
+
 
 class TestModelPath(unittest.TestCase):
     def test_home_wins(self):
@@ -94,15 +103,51 @@ class TestLabelMap(unittest.TestCase):
         self.assertEqual(len(names), len(set(names)))
 
     def test_requested_classes_fold_case_underscores_and_hyphens(self):
-        wanted, unknown = resolve_requested_classes(["Person", "CELL_PHONE", "dining-table"])
+        wanted, unknown = COCO_INDEX.resolve(["Person", "CELL_PHONE", "dining-table"])
         self.assertEqual(wanted, {"person", "cell phone", "dining table"})
         self.assertEqual(unknown, [])
 
     def test_an_unknown_class_is_returned_never_silently_dropped(self):
-        wanted, unknown = resolve_requested_classes(["person", "shoplifter"])
+        wanted, unknown = COCO_INDEX.resolve(["person", "shoplifter"])
         self.assertEqual(wanted, {"person"})
         self.assertEqual(unknown, ["shoplifter"])
 
+    def test_every_committed_class_name_is_already_canonical(self):
+        """`_to_wire` puts the DECLARED name on the wire and the filter matches
+        the CANONICAL one, so a vocabulary declaring `cell_phone` would emit a
+        label the caller's own request had just been folded away from — a class
+        that silently matches nothing on a box where everything looks loaded.
+        `ModelVocabulary.__post_init__` refuses it at import; this pins that the
+        shipped vocabularies satisfy it rather than merely that the check
+        exists."""
+        for vocabulary in BUILTIN_VOCABULARIES:
+            index = ClassIndex([vocabulary])
+            for name in vocabulary.class_names:
+                with self.subTest(model=vocabulary.model_name, name=name):
+                    self.assertEqual(index.resolve([name]), ({name}, []))
+
+
+class TestRamBudgetEnv(unittest.TestCase):
+    """The portfolio budget's env override. A bad value must not be ignored.
+
+    Silently falling back to the default would leave an operator convinced they
+    had raised a limit that never moved — an operation reporting success while
+    doing nothing, which is the failure shape this tree keeps catching.
+    """
+
+    def test_absent_means_take_the_committed_default(self):
+        self.assertIsNone(resolve_ram_budget_mb({}))
+        self.assertIsNone(resolve_ram_budget_mb({"TRILLBOARDS_DETECTOR_RAM_BUDGET_MB": "  "}))
+
+    def test_an_integer_is_read_as_mib(self):
+        self.assertEqual(resolve_ram_budget_mb({"TRILLBOARDS_DETECTOR_RAM_BUDGET_MB": "2048"}), 2048)
+
+    def test_a_malformed_value_raises_rather_than_defaulting(self):
+        for bad in ("1.5gb", "2048MB", "-1", "0", "lots"):
+            with self.subTest(bad=bad):
+                with self.assertRaises(ValueError):
+                    resolve_ram_budget_mb({"TRILLBOARDS_DETECTOR_RAM_BUDGET_MB": bad})
+
 
 if __name__ == "__main__":
     unittest.main()