@trillboards/edge-sdk 0.2.73 → 0.2.76

package/deploy/detector/README.md

@@ -0,0 +1,133 @@
+# `trillboards-detector` — the on-device object-detector service
+
+RF-DETR Medium, fp32 ONNX, served over loopback on **127.0.0.1:9380**. It is the
+cascade gate that sits in front of the on-device VLM: at the ANALYZE cadence it
+removes ~84.7% of 5,645 ms VLM calls for ~2.1% of events.
+
+The decision record — which model, why not the alternatives, and every number
+above — is [`docs/runbooks/cctv-detector-tier.md`](../../../../../docs/runbooks/cctv-detector-tier.md).
+Read it before changing anything about the model. **Do not quantise**: INT8 loses
+31.7 points of recall *silently* and is slower than fp32.
+
+## The wire
+
+The consumer is
+[`src/LocalDetectorClient.ts`](../../src/LocalDetectorClient.ts). It shipped
+first and **its contract is frozen** — this service accommodates it, never the
+other way round.
+
+```
+POST /detect   { image_b64, classes?, min_confidence }
+     200       { detections: [{ label, confidence, box: {x,y,width,height} }],
+                 inference_ms, model }
+     400       the request cannot be acted on (bad base64, empty body,
+                 min_confidence outside [0,1], every requested class unknown)
+     413       body over 32 MiB
+     500       inference failed, or a detection could not be put on the wire
+     503       no usable model — NEVER a 200 with an empty array
+
+GET  /health   200 when a load completed AND a probe inference succeeded
+               503 otherwise, with the reason. Runs no inference.
+```
+
+* `label` is the class **name**, never an index.
+* `box` is in **pixels of the submitted frame**, not normalised.
+* `classes` filters at the **service**, so the wire stays small. Absent means
+  every class; an empty array is a 400, never a synonym for "all".
+
+Three properties are load-bearing rather than stylistic:
+
+**Down is never empty.** A missing model is 503 and a failed inference is 500.
+Neither may become `200 []`. A cascade that reads "detector broken" as "venue
+quiet" suppresses every alert while every health surface stays green.
+
+**We refuse what the client would refuse.** `build_detect_response` re-checks
+every detection against the client's own fifteen rules and answers 500 rather
+than emitting a payload the client will reject — because a rejected payload
+costs the whole frame *and* is indistinguishable, from the client's side, from
+the service being down.
+
+**Health reports a load that actually happened.** `GET /v1/models` on litert-lm
+answers 200 without ever building an engine, which is how a UTF-8 BOM in its
+config 500'd every frame on every Windows connector while the installer's health
+check *and* its CPU-fallback re-check both passed. So this loader builds the
+session, runs a real probe inference, and `/health` reports that outcome.
+
+## Where it runs
+
+Its own `uv tool`, so its own venv; its own OS-supervised service; Node never
+spawns it.
+
+| platform | supervisor | provisioned by |
+| --- | --- | --- |
+| macOS | `deploy/macos/com.trillboards.detector.plist` (LaunchAgent) | `deploy/macos/scripts/setup-detector.sh` |
+| Linux | `deploy/systemd/trillboards-detector.service` | `deploy/systemd/setup-detector.sh` |
+| Windows | Scheduled Task `TrillboardsDetector` | `deploy/windows/setup-detector.ps1` |
+
+All three are **best effort**. Without Gemma the connector cannot sense at all,
+so that provisioner may abort an install; this one only makes sensing cheaper,
+and a cost optimisation must never block a customer pairing a camera.
+
+## The model is OTA'd, never bundled
+
+`~/.trillboards/models/rfdetr_medium/model.onnx`, or wherever
+`${TRILLBOARDS_DATA_DIR:-$HOME/.trillboards}` resolves — the same derivation
+`config.ts`'s `DEFAULTS.dataDir` uses, because a service that resolves a
+different directory from the process that downloads the model is permanently
+unhealthy on a box whose artifact is sitting right there.
+
+The macOS code-only build is capped at `MAX_SLIM_PKG_BYTES` = 250 MiB and the
+code slice is already ~207 MB, so a ~128 MiB ONNX would make the build `die`.
+It arrives through edge-core's `ModelManager`, the same channel Gemma's weights
+use. The service **starts without it**, reports 503 with the reason, and picks
+the artifact up within one poll interval of it landing — no restart.
+
+One directory per model name and a fixed file name inside it, so an update
+reclaims the old bytes. There is no GC for abandoned model dirs anywhere on this
+path and the existing OTA already orphans 752 MiB per update; a version-named
+directory would add to that.
+
+## Tests
+
+```bash
+# The wire contract. Stdlib only — no model, no onnxruntime, no pip install.
+python3 -m unittest discover -s . -p "test_*.py" -v
+```
+
+Runs in `guards-fast`. **Everything except the post-processing cases needs
+nothing but the standard library** — those are the only ones that touch numpy,
+and they skip themselves with a stated reason when it is absent. That invariant
+is the durable claim; the counts (45 of 60 on 2026-08-24) are a snapshot and go
+stale the next time a case is added, so derive them by running it rather than
+reading them here.
+
+```bash
+# The real client against the real service, over loopback.
+cd ../.. && TRILLBOARDS_DETECTOR_MODEL=/path/to/rfdetr-medium.onnx \
+  npx jest src/__tests__/localDetectorService.integration.test.ts
+```
+
+Skips — loudly, with the reason — when the Python tool or the artifact is
+absent. It is the only thing that can catch a two-language disagreement, and it
+earned its place immediately: it found that the client sends every request
+`Transfer-Encoding: chunked` (it sets no `Content-Length`) while `http.server`
+does not decode chunked bodies, so the service was answering 400 to every real
+call.
+
+## The transcription control
+
+`session.py` transcribes four steps from the `rfdetr` package — the
+antialias-free square resize, ImageNet normalisation, `_select_topk`, and
+`_gather_and_scale_boxes` — because that package pulls ~3 GB of torch and is not
+going on a shop PC. A transcription can drift, so:
+
+```bash
+uv venv /tmp/rfdetr-ref --python 3.12
+uv pip install --python /tmp/rfdetr-ref/bin/python rfdetr onnx onnxruntime pillow
+/tmp/rfdetr-ref/bin/python bench/verify_against_reference.py \
+  --onnx /path/to/rfdetr-medium.onnx --image ../windows/e2e/fixtures/person.jpg
+```
+
+Runs both pipelines on one image and fails on any disagreement. A documented
+manual run, not a CI step — it needs a checkpoint download and ~3 GB of
+dependencies.

package/deploy/detector/bench/verify_against_reference.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+"""The control: this service's pipeline vs. the `rfdetr` package, same image.
+
+WHY THIS EXISTS. `session.py` TRANSCRIBES four steps from `rfdetr` — the
+antialias-free square resize, the ImageNet normalisation, `_select_topk`, and
+`_gather_and_scale_boxes` — because shipping torch + torchvision +
+transformers (~3 GB) to a shop PC to run a 128 MiB graph is not an option. A
+transcription can drift from its source silently: every one of those four steps
+has a plausible wrong version that still returns boxes, and boxes that are
+plausible and wrong are exactly what `docs/runbooks/cctv-detector-tier.md` §5
+was written about.
+
+So this runs both and fails on any disagreement. It is the same instinct as §5
+rule 5 — *validate the rig before believing a surprise* — applied to the rig
+itself.
+
+NOT A CI STEP, deliberately: it needs the `rfdetr` package and its ~3 GB of
+dependencies plus a checkpoint download. It is a documented manual run, the same
+shape as the live half of `scripts/proposals/endpoint-matrix.js`.
+
+    uv venv /tmp/rfdetr-ref --python 3.12
+    uv pip install --python /tmp/rfdetr-ref/bin/python rfdetr onnx onnxruntime pillow
+    /tmp/rfdetr-ref/bin/python -m pip install -e <this dir>
+    /tmp/rfdetr-ref/bin/python bench/verify_against_reference.py \\
+        --onnx <exported>.onnx --image ../windows/e2e/fixtures/person.jpg
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from trillboards_detector.session import OnnxDetector  # noqa: E402
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--onnx", required=True)
+    ap.add_argument("--image", required=True)
+    ap.add_argument("--threshold", type=float, default=0.25)
+    # Tolerances. The exported fp32 graph is numerically identical to the `.pth`
+    # to ~1e-6 per the runbook, so the only real slack is the resize kernel:
+    # Pillow's BILINEAR and torchvision's antialias=False agree closely but not
+    # bit-for-bit, which moves a confidence in the third decimal and a box edge
+    # by well under a pixel.
+    ap.add_argument("--conf-tol", type=float, default=0.02)
+    ap.add_argument("--box-tol-px", type=float, default=2.0)
+    args = ap.parse_args()
+
+    from PIL import Image
+    from rfdetr import RFDETRMedium
+
+    image_path = Path(args.image)
+    with Image.open(image_path) as im:
+        width, height = im.size
+    print(f"image: {image_path} ({width}x{height})")
+
+    # ── the reference ────────────────────────────────────────────────────────
+    reference = RFDETRMedium()
+    ref = reference.predict(str(image_path), threshold=args.threshold)
+    ref_rows = sorted(
+        (
+            str(name),
+            round(float(conf), 4),
+            tuple(round(float(v), 2) for v in box),
+        )
+        for name, conf, box in zip(
+            ref.data.get("class_name", [""] * len(ref.confidence)),
+            ref.confidence,
+            ref.xyxy,
+        )
+    )
+
+    # ── this service ─────────────────────────────────────────────────────────
+    detector = OnnxDetector(Path(args.onnx), "rfdetr_medium")
+    batch = detector.detect(image_path.read_bytes(), args.threshold, None)
+    ours_rows = sorted(
+        (d.label, round(d.confidence, 4), (round(d.x, 2), round(d.y, 2), round(d.x + d.width, 2), round(d.y + d.height, 2)))
+        for d in batch.detections
+    )
+
+    print(f"\nreference ({len(ref_rows)}):")
+    for r in ref_rows:
+        print("  ", r)
+    print(f"\nthis service ({len(ours_rows)}):  [inference {batch.inference_ms:.1f} ms]")
+    for r in ours_rows:
+        print("  ", r)
+
+    problems: list[str] = []
+    if len(ref_rows) != len(ours_rows):
+        problems.append(f"detection COUNT differs: reference {len(ref_rows)} vs ours {len(ours_rows)}")
+    else:
+        for i, (a, b) in enumerate(zip(ref_rows, ours_rows)):
+            if a[0] != b[0]:
+                problems.append(f"row {i}: label {a[0]!r} vs {b[0]!r}")
+                continue
+            if abs(a[1] - b[1]) > args.conf_tol:
+                problems.append(f"row {i} ({a[0]}): confidence {a[1]} vs {b[1]}")
+            for axis, (u, v) in enumerate(zip(a[2], b[2])):
+                if abs(u - v) > args.box_tol_px:
+                    problems.append(f"row {i} ({a[0]}): box[{axis}] {u} vs {v}")
+
+    if problems:
+        print("\nDISAGREEMENT — the transcription in session.py has drifted:")
+        for p in problems:
+            print("  -", p)
+        return 1
+
+    print(
+        f"\nAGREE: {len(ref_rows)} detection(s), labels identical, "
+        f"confidence within {args.conf_tol}, boxes within {args.box_tol_px}px."
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/deploy/detector/pyproject.toml

@@ -0,0 +1,49 @@
+[project]
+name = "trillboards-detector"
+version = "0.1.0"
+description = "On-device object-detector service for the Trillboards BYO-CCTV connector (loopback HTTP, ONNX Runtime)."
+requires-python = ">=3.10"
+
+# PINNED, for the same reason litert-lm is pinned, and the reason is written out
+# in `deploy/macos/scripts/setup-litertlm.sh`: an UNPINNED `uv tool install`
+# installs whatever is current on the day a customer happens to reinstall, and
+# the thing that then changes is not a changelog entry — it is the number this
+# whole placement decision was made on. `docs/runbooks/cctv-detector-tier.md`
+# picks RF-DETR Medium over the on-device VLM on a measured ~63 ms, and ORT's
+# CPU execution provider is where that ms lives: kernel fusion, thread
+# heuristics and SIMD dispatch all move between minor releases. litert-lm's
+# accelerator behaviour moved twice with no release of ours and reproduced on no
+# dev box; the pin is what stops that shape repeating here.
+#
+# These are the versions the service was measured against — see
+# `docs/runbooks/cctv-detector-tier.md` §Runtime. A bump is a re-measurement,
+# not a version-number edit.
+dependencies = [
+  "onnxruntime==1.29.0",
+  "numpy==2.5.2",
+  "pillow==12.3.0",
+]
+
+[project.scripts]
+trillboards-detector = "trillboards_detector.cli:main"
+
+# HATCHLING, NOT SETUPTOOLS, AND THAT IS A CORRECTNESS CHOICE.
+#
+# setuptools stages the package into a `build/lib/` directory and copies from
+# THERE on later builds. A stale `build/` therefore silently shadows the real
+# source: measured here on 2026-08-23, `uv tool install --force` reported
+# "Installed 1 executable" and installed the PREVIOUS revision of server.py,
+# with no warning anywhere. On a customer machine that is the whole failure
+# shape this codebase keeps hitting — an operation that reports success while
+# doing nothing — and it would ship an old detector behind a green install.
+#
+# Hatchling builds from the source tree with no intermediate staging directory
+# and writes no `.egg-info`, so the staleness has nowhere to live. `.gitignore`
+# and the two installer staging excludes cover the leftovers of the old
+# backend; this makes new ones impossible.
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["trillboards_detector"]

package/deploy/detector/tests/test_modelpath.py

@@ -0,0 +1,108 @@
+"""The artifact's home, and the label map's invariants. Stdlib only.
+
+`resolve_data_dir` has to agree with `config.ts`'s `DEFAULTS.dataDir` exactly:
+
+    path.join(process.env.HOME || process.env.USERPROFILE || '/tmp', '.trillboards')
+
+If the Python service and the Node process that downloads the model disagree
+about that directory, the detector is permanently unhealthy on a machine whose
+artifact is sitting right there — and `/health` would report the truth about a
+path nobody else uses.
+"""
+
+from __future__ import annotations
+
+import unittest
+from pathlib import Path
+
+from trillboards_detector.labels import (
+    COCO91_CLASS_NAMES,
+    EXPECTED_LOGIT_SLOTS,
+    resolve_requested_classes,
+)
+from trillboards_detector.modelpath import (
+    DEFAULT_MODEL_NAME,
+    resolve_data_dir,
+    resolve_model_path,
+)
+
+
+class TestModelPath(unittest.TestCase):
+    def test_home_wins(self):
+        env = {"HOME": "/Users/x", "USERPROFILE": r"C:\Users\y"}
+        self.assertEqual(resolve_data_dir(env), Path("/Users/x/.trillboards"))
+
+    def test_userprofile_is_the_windows_fallback(self):
+        self.assertEqual(
+            resolve_data_dir({"USERPROFILE": r"C:\Users\y"}), Path(r"C:\Users\y") / ".trillboards"
+        )
+
+    def test_tmp_is_the_last_resort_exactly_as_in_config_ts(self):
+        self.assertEqual(resolve_data_dir({}), Path("/tmp/.trillboards"))
+
+    def test_an_empty_home_falls_through_rather_than_resolving_to_dot_trillboards(self):
+        """`''` is falsy in JS too, so `config.ts` skips it. A Python `.get`
+        returning `''` would otherwise produce a RELATIVE `.trillboards`, i.e. a
+        different directory per working directory."""
+        self.assertEqual(resolve_data_dir({"HOME": "", "USERPROFILE": ""}), Path("/tmp/.trillboards"))
+
+    def test_the_default_layout_is_one_directory_per_model_name(self):
+        p = resolve_model_path(DEFAULT_MODEL_NAME, {"HOME": "/h"})
+        self.assertEqual(p, Path("/h/.trillboards/models/rfdetr_medium/model.onnx"))
+
+    def test_a_fixed_file_name_means_an_update_reclaims_the_old_bytes(self):
+        """No version in the path, deliberately. The model OTA already orphans
+        752 MiB per update and nothing GCs abandoned model dirs; a
+        version-named directory would add 128 MiB per release on a box with no
+        disk-space preflight anywhere on this path."""
+        a = resolve_model_path("rfdetr_medium", {"HOME": "/h"})
+        b = resolve_model_path("rfdetr_medium", {"HOME": "/h"})
+        self.assertEqual(a, b)
+
+    def test_the_explicit_override_wins_over_everything(self):
+        env = {"HOME": "/h", "TRILLBOARDS_DETECTOR_MODEL": "/bench/rfdetr.onnx"}
+        self.assertEqual(resolve_model_path(env=env), Path("/bench/rfdetr.onnx"))
+
+    def test_the_data_dir_override_relocates_the_whole_store(self):
+        env = {"HOME": "/h", "TRILLBOARDS_DATA_DIR": "/var/lib/trillboards"}
+        self.assertEqual(
+            resolve_model_path("rfdetr_medium", env),
+            Path("/var/lib/trillboards/models/rfdetr_medium/model.onnx"),
+        )
+
+
+class TestLabelMap(unittest.TestCase):
+    def test_person_is_slot_1_which_is_what_the_eval_scored(self):
+        """`detector-detections/rfdetr-medium.json` records
+        `class_selected: person, class_index: 1, class_confirmed_live: true`.
+        Every accuracy number in the runbook is a claim about THAT index."""
+        self.assertEqual(COCO91_CLASS_NAMES[1], "person")
+
+    def test_the_map_fits_inside_the_declared_logit_width(self):
+        self.assertLess(max(COCO91_CLASS_NAMES), EXPECTED_LOGIT_SLOTS)
+        self.assertEqual(EXPECTED_LOGIT_SLOTS, 91)
+
+    def test_it_is_the_80_named_coco_categories_with_the_gaps_intact(self):
+        """80 names across a 91-wide axis. The eleven gaps are real — they are
+        why an unnamed slot has to be dropped rather than guessed."""
+        self.assertEqual(len(COCO91_CLASS_NAMES), 80)
+        for gap in (0, 12, 26, 29, 30, 45, 66, 68, 69, 71, 83):
+            self.assertNotIn(gap, COCO91_CLASS_NAMES)
+
+    def test_names_are_unique_so_a_filter_cannot_be_ambiguous(self):
+        names = list(COCO91_CLASS_NAMES.values())
+        self.assertEqual(len(names), len(set(names)))
+
+    def test_requested_classes_fold_case_underscores_and_hyphens(self):
+        wanted, unknown = resolve_requested_classes(["Person", "CELL_PHONE", "dining-table"])
+        self.assertEqual(wanted, {"person", "cell phone", "dining table"})
+        self.assertEqual(unknown, [])
+
+    def test_an_unknown_class_is_returned_never_silently_dropped(self):
+        wanted, unknown = resolve_requested_classes(["person", "shoplifter"])
+        self.assertEqual(wanted, {"person"})
+        self.assertEqual(unknown, ["shoplifter"])
+
+
+if __name__ == "__main__":
+    unittest.main()