@apocaliss92/scrypted-advanced-notifier 4.8.39 → 5.0.1

package/CHANGELOG.md

@@ -1,6 +1,10 @@
 <details>
 <summary>Changelog</summary>
 
+### 5.0.0
+Finally An app is completed. It's called CamStack and it's available on testflight and as pwa on the scrypted local istance. Also available on https://camstack.zentik.app
+New docs is up on https://advanced-notifier-docs.zentik.app/docs/advanced-notifier
+
 ### 4.8.39
 
 Added section on every battery camera to setup the battery management based on customizable thresholds

package/README.md

@@ -1,267 +1,6 @@
-# Scrypted advanced notifier
+# Scrypted Advanced Notifier
 
 ☕️ If this extension works well for you, consider buying me a coffee. Thanks!
 [Buy me a coffee!](https://buymeacoffee.com/apocaliss92)
 
-[For requests and bugs](https://github.com/apocaliss92/scrypted-advanced-notifier)
-
-# Getting started
-
-## MQTT
-
-To enable MQTT exporting:
-
-- enable `MQTT enabled` in the general -> general tab
-- setup the authentication parameters in the tab general -> MQTT, check the `Use MQTT plugin credentials` to use the credentials set on the MQTT plugin
-- Check `Use NVR detections` if you want the images stored on MQTT to be the clipped ones from NVR.
-- Check `Audio pressure (dB) detection` if you want a continuous reporting of the audio kept by the camera (dBs)
-- Check `Check objects occupancy regularly` if you want regular occupancy data checks on the camera
-- Cameras enabled to the plugin will be automatically enabled to MQTT. Can be disabled in the camera's section Advanced notifier -> Report to MQTT
-
-The plugin will export to MQTT the following entities:
-
-- PTZ controls
-- Restart control
-- Notifications enabled
-- Basic detection information (motion, animal, person, vehicle, face, plate, generic object). Same information will be available for every rule associated to a camera
-  - Latest image
-  - Triggered
-  - Last trigger time (disabled by default)
-  - Amount of objects (if enabled)
-- Online status
-- Sleeping status
-- Battery status
-- Recording switch (NVR privacy mode)
-- Current dBs (if enabled)
-
-IMPORTANT
-If you edit a device name, to force the correct re-creation of the ha entities you will need to delete manually the entities on HA and restart the plugin
-
-## Notifications
-
-The plugin provides customized way to deliver notifications. It is based on rules. Each rule can be activated based on several factors, i.e. active sensors, time ranges, security system status. Any notifier can be used but the only fully supported currently are (can be extended for any other):
-
-- Zentik - Full round notifier application. Perfect for apple devices (iOS + watchOS, iPad, MacOS), developed by me with an eye to Advanced notifier and support whatever was possible, available for beta testing:
-  - iOS under https://testflight.apple.com/join/dFqETQEm
-  - PWA (android, web) under https://notifier.zentik.app
-  - Scrypted plugin available https://www.npmjs.com/package/@apocaliss92/scrypted-zentik
-- Native Scrypted notifiers (i.e. Scrypted iPhone app...)
-- Ntfy
-- Homeassistant push notifications
-- Pushover
-- Telegram
-
-  Highly suggested to utilize Zentik, since it will give a nice experience for both push notifications and an history of the past one. Otherwise combinations of other notifiers can be used, Pushover or NTFY as notifiers storage, in combination with a homeassistant or NVR one, setting its priority to the lowest. This will allow to have a rich notification and also to store it on another notifier. This because notifiers such as pushover, ntfy or telegram do not have a nice support to actions. Following parameters are required to successfully send notifications
-
-- `Scrypted token`: Token stored on the scrypted entity on homeassistant
-- `NVR url`: Url pointing to the NVR instance, should be accessible from outside
-
-Each notifier will be fully configurable on every rule, with possibility to set: actions, addSnoozeActions or priority.
-Default actions can be set on every camera, will be added to each notification
-
-All notifiers currently support critical notifications.
-
-Notifications can be disabled for a specific camera on the camera page, Advanced notifier => Notifier => `Notifications enabled` (available on MQTT as well)
-Notifications can be disabled globally on the general tab of the plugin
-
-### Scrypted NVR notifiers
-
-Plugins supports scripting of the NVR buitin notifiers, following features are available:
-
-- discover to MQTT
-- Notifier notifications disabled: completely disable notifications for a specific notifier
-- Camera notifications disabled: disable notifications for a specific camera
-- Schedule notifications on both cameras and notifiers
-- Translate notifications with the plugin `Texts` section (enabled by default)
-- Enable AI to generate descriptions. To make this to work, each camera and notifier should be extended with the Advanced notifier pluign and activate the AI flag on both. Reason is that ai calls can be expensive and needs to be explicitely enabled on both entities
-
-**NVR notifiers can be used both as plugin notifiers, then with rules and everything, or just to enhance the NVR notifications.**
-
-- If you want to use it as plugin notifier, you should keep the notifier enabled (at the very bottom of the page) BUT disable all the detection classes (on the device page of the device, i.e. `Scrypted iPhone App (user)`)
-- If you want the plugin to just enhance the NVR notifications, there is nothing to change to make it work with the plugin. Just extend the notifier with this plugin and use the features you like to use
-
-## Rules
-
-Rules can be of following types: Detection, Occupancy, Audio, Timelapse. These properties are in common with all, some are hidden until the `Show more configurations` gets activated
-
-- `Activation type`: when the rule shoul be active
-  - Always
-  - Schedule, defined in a time range during the day
-  - OnActive, will be active only if the camera will be listed in the `"OnActive" devices` selector (plugin => rules => general). This selector can be driven by MQTT with a topic specified in `Active entities topic` under General => MQTT. The message to this topic can contain either a list of device IDs, names or homeassistant entityId (check homeassistant section)
-- `Notifiers`: notifiers to notify, additional properties will be applied depending on the selected ones
-  - `Pushover priority` priority to use on pushover
-  - `Homeassistant Actions` actions to show on the homessistant push notifications, of type `{"action":"open_door","title":"Open door","icon":"sfsymbols:door"}`, check homeassistant documentation for further info
-- `Open sensors` which sensors should be open to enable the rule
-- `Closed sensors` which sensors should be closed to enable the rule
-- `Alarm modes` which alarm states should enable the rule. The alarm system device can be defined in the plugin page under Rules => `Security system`
-- `Notify with a clip` available only for detection and occupancy rules, the plugin will activate a decoder to save the last frames of the camera. On the trigger of a rule, a short clip will be generated and sent instead of a simple snapshot. It supports 2 types:
-  - MP4: supported only by homeassistant and partially the others
-  - GIF: supported by homeassistant, pushover
-
-### Detection
-
-These rules can be created for multiple cameras (on the plugin page) or per single camera. They allow to specify which object detections should trigger a notification:
-
-- Create a new rule adding a new text in the `Detection rules` selector and hit save. A new tab will appear
-- Set the activation type
-- Set the notifiers to notify on the detection
-- Check `Use NVR detections` to trigger the rule only as effect of detections from NVR plugin. This will include cropped images stored on MQTT and will be in sync with the NVR app events reel
-- Set the detection classes and the minimum score to trigger the notification
-- Set `Minimum notification delay` to debounce further notifications (overrides the camera settings)
-- Set `Minimum MQTT publish delay` to debounce the image update on MQTT for this rule
-- Set `Whitelisted zones` to use only detections on these zones
-- Set `Blacklisted zones` to ignore detections coming from these zones
-- Set `Disable recording in seconds` to enable NVR recording for some seconds and disable it afterwords
-- Set a `Custom text` if a specific text should be applied. By default detection rules will use the texts defined in the plugin tab `Texts`, many placeholder are available to enrich the content
-- Check `Enable AI to generate descriptions` if you want to let AI generate a description text out of the image. AI settings are available on the plugin page under the AI, currently supported: GoogleAi, OpenAi, Claude, Groq
-- Set `CLIP Description` to use semantic search and filter out even more detections. It will be applied at the very end of the chain, when all the filters already had effect. Set `CLIP confidence level` to finetune the confidence level of the search
-- Set `AI filter` to send the image to the choosen AI tool to confirm the input prompt
-- Set `Image post processing` to process notification images:
-  - MarkBoundaries will drawn a coloured rectangle around the detected object
-  - Crop will crop the image around the detected object
-
-#### Audio classification
-
-Detection rules can also detect audio labels, i.e. crying, scream, speech etc. This will be possible adding in the detection calsses setting the "Audio" label. A new setting will appear to specify which labels to consider. The classifier to be used can be set in the Audio analysis section of the camera device:
-
-- YAMNET ('YAMNet Audio Classification' plugin), this will be used by the plugin onboarded audio analyser
-- DISABLED, no classifier will be run, external plugins will be able to still forward events, such as Frigate Bridge
-
-Impact of the onboard classifier is relatively small and it can replace completely the smart audio sensor
-
-### Recording (only on camera)
-
-These rules let the cameras to record configurable videoclips. Mostly use these on camera where NVR might be overkill to be used, the app will still show the clips in a nice way
-
-They are based on some criteria:
-
-- detection classes, what should initially trigger the recording
-- score threshold, the minimum score to trigger the recording, leave empty for any detection
-- Minimum delay between clips, how many seconds to wait, at minimum, to record the following clip
-- Post event seconds, how many seconds to record, at minimum, after the recording starts. The default camera prebuffer will be extra pre-event seconds on top
-- Max clip length, seconds cap of the clip, following detections will prolong the original clip length
-- Prolong clip on motion, prolong the clip also for simple motion events
-
-### Occupancy (only on camera)
-
-These rules will monitor a specific area to mark it as occupied or not
-
-- Make sure to set an object detector on the plugin page under Rules => `Object Detector`
-- Create a new rule adding a new text in the `Occupancy rules` selector and hit save. A new tab will appear
-- Set the activation type
-- Set the notifiers to notify on the occupancy change
-- Set the detection class to monitor
-- Set the camera zone to monitor, must be an `Observe` type zone defined in the `Object detection` section of the camera
-- (Optional) set a capture zone to reduce the frame used for the detection, may increase success rate
-- Set `Zone type`
-  - `Intersect` if the objects can be considered detected if falling in any portion of the zone
-  - `Contain` if the objects should be completely included in the detection zone
-- Set a `Score threshold`, in case of static detections should be pretty low (default 0.3)
-- Set `Occupancy confirmation`, it's a confirmation period in seconds to avoid false results. Set it depending on your specific case
-- Set `Force update in seconds` to force an occupancy check if no detection happens. Any detection running on the camera will anyways check all the occupancy rules
-- Set the `Max objects` the zone can contain. The zone will be marked as occupied if the detected objects are >= of the number set here
-- Set a text in both `Zone occupied text` and `Zone not occupied text` for the notification texts
-- Activate `Confirm occupancy with AI` to confirm occupancy results to reduce even more false positives. Under the plugin AI section is possible to customize the prompt. Results may vary depending on the model used
-
-### Timelapse (only on camera)
-
-Define a timeframe, the plugin will collect frames from the camera and generate a clip out of it at the end of the defined range. All the generated timelapses will be available as videoclip on the NVR app, only if the `Enable Camera` on the plugin page will be enabled.
-
-- Create a new rule adding a new text in the `Timelapse rules` selector and hit save. A new tab will appear
-- Define the week days and the start/end times. i.e. start at 11pm and end at 8am
-- Set the notifiers to notify the generated clip
-  - If an homeassistant notifier is used and the final clip will be <50bm, the clip will be shown as preview of the push notification!
-- Set a `Notification text` for the notification message
-- Set a `Frames acquisition delay`, a frame will be generated according to this. Each non-motion detection will always add a frame
-  - In future will be possible to add frames based on specific detection classes and even small clips
-- Set a `Timelapse framerate`, this will depend on the timespan you will chose and how long you want the final clip to be
-- Use the `Generate now` button to reuse the frames collected the previous session. They will be stored until the following session starts
-
-### Audio (only on camera)
-
-**Audio rules will activate only if a source of audio measurement is active. The plugin provides an onboarded audio analyser, which will activate when any audio rule is running.**
-Audio rules will monitor the audio received by the camera
-
-- Create a new rule adding a new text in the `Audio rules` selector and hit save. A new tab will appear
-- Set the notifiers to notify the event
-- Set a `Notification text` for the notification message
-- Set a `Decibel threshold` for the audio level to alert
-- Set `Duration in seconds` if the audio should last at least these seconds to trigger a notification. Leave blank to notify right away
-
-## Sequences
-
-In the sequences section, on the plugin page, you will be able to define a custom sequence with various steps. Each sequence can contain actions such as: Wait, Script, PTZ preset, Switch on/off, Lock, Entry open/close.
-
-Sequences can be attached to rules at different moments:
-
-- **On-activation sequences** (rules with activation type other than Always): run when the rule becomes active.
-- **On-deactivation sequences**: run when the rule becomes inactive.
-- **On-trigger sequences**: run when the rule is triggered (e.g. detection matched, occupancy change).
-- **On-reset sequences**: run when the rule is reset (e.g. after the trigger window ends).
-- **On-generated sequences**: run when all artifacts for that rule have been generated (video clip, gif, image). Available for Detection, Occupancy and Timelapse rules.
-
-When a sequence is run with a **payload** (e.g. On-generated), Script actions receive it as run variables. In Scrypted scripts, access it via `variables.payload`.
-
-### Payload for On-generated sequences
-
-For **On-generated sequences**, the payload passed to scripts has the following shape:
-
-- **Detection and Occupancy rules**: `{ rule, videoUrl?, gifUrl?, imageUrl }`
-  - `rule`: the rule object (name, notifiers, settings, etc.).
-  - `videoUrl`: URL of the generated video clip (if clip type is MP4).
-  - `gifUrl`: URL of the generated GIF (if clip type is GIF).
-  - `imageUrl`: URL of the stored snapshot used for the notification.
-
-- **Timelapse rules**: `{ rule, videoUrl, imageUrl, videoPath?, imagePath? }`
-  - `rule`: the timelapse rule object.
-  - `videoUrl`: URL to the generated timelapse video.
-  - `imageUrl`: URL to the generated thumbnail image.
-  - `videoPath`, `imagePath`: filesystem paths where the video and image were saved (when available).
-
-(Initially only a few action types will be available, ask for more if you wish something more specific)
-
-## Stored images
-
-The plugin will store on filesystem, if configured, images for every basic detection and rule. Set the following configurations on the plugin page under the Storage tab
-
-- `Storage path`: If set, the images used to populate MQTT topic will be also stored on the drive path
-
-## Additional camera settings
-
-- `Minimum snapshot acquisition delay`, minimum seconds to wait until a new snapshot can be taken from a camera, keep it around 5 seconds for cameras with weak hardware
-- `Off motion duration`, amount of seconds to consider motion as ended for rules/detections affecting the camera. It will override the motion off events
-- `Snapshot from Decoder`, take snapshots from the camera decoded stream. If set to `Always` it will be active only if any detection rule with videoclips, timelapse or occupancy rule is running. If set `OnMotion` it will run only during motion sessions, usefull if your camera gives many snapshot timeout errors. `Auto` will be the default and regulate it when required
-- Set `Minimum notification delay` to debounce further notifications
-- Set `Minimum MQTT publish delay` to debounce the image update on MQTT for this basic detections
-
-## Webhooks
-
-Some basic webhooks are available
-
-### Latest snapshot
-
-Will provide the latest registered image for each type, on the camera settings will be provided the basic url, {IMAGE_NAME} should be replaced with one of the following:
-
-- `object-detection__{ motion | any_object | animal | person | vehicle }`
-- `object-detection__{ motion | any_object | animal | person | vehicle }__{ NVR | Frigate }`
-- `object-detection__face-{ known person label }`
-- `object-detection__face-{ known person label }_{ NVR | Frigate }`
-- `ruleImage__{ ruleName }`
-- `ruleClip__{ ruleName }`
-- `ruleGif__{ ruleName }`
-
-The base path is available under each camera in the section Advanced Notifier -> Webhooks.
-
-### POST detection images
-
-Provide multiple urls, for each detection, POST a b64 image with some additional metadata. Filter
-on some classes and define a minimum delay.
-
-## Adanced Alarm System
-
-The plugin provides a security system hooked into the plugin detection rules. To use it this will be required:
-
-- Create 1 or more detection rule on the plugin page with activation type `AdvancedSecuritySystem` and set 1 or more modes to activate the rule
-- Setup the provided `Advanced security system` device with preferred preferences, such as texts or devices that can be bypassed during the activation
-
-The device is discovered on MQTT and completely compatible with Homekit.
+**Documentation:** [https://advanced-notifier-docs.zentik.app/](https://advanced-notifier-docs.zentik.app/)

package/package.json

@@ -5,8 +5,12 @@
     "type": "git",
     "url": "https://github.com/apocaliss92/scrypted-advanced-notifier"
   },
-  "version": "4.8.39",
+  "version": "5.0.1",
   "scripts": {
+    "docs:dev": "cd docs && npm run dev",
+    "docs:build": "cd docs && npm run build",
+    "docs:start": "cd docs && npm run start",
+    "docs:install": "cd docs && npm install",
     "scrypted-setup-project": "scrypted-setup-project",
     "prescrypted-setup-project": "scrypted-package-json",
     "build": "scrypted-webpack",

package/docs/INTERFACE_DESCRIPTORS_MIGRATION.md

@@ -1,184 +0,0 @@
-# Migrazione da Webhook HTTP a Socket SDK con interfaceDescriptors
-
-Piano per esporre i metodi Events App via **interfaceDescriptors** (come [@scrypted/llm](https://github.com/scryptedapp/llm)), eliminando le chiamate REST e usando solo la socket SDK.
-
----
-
-## 1. Come funziona interfaceDescriptors (LLM plugin)
-
-Dal [package.json dell'LLM](https://github.com/scryptedapp/llm/blob/main/package.json):
-
-```json
-{
-  "scrypted": {
-    "interfaces": ["DeviceProvider", "UserDatabase", ...],
-    "interfaceDescriptors": {
-      "UserDatabase": {
-        "name": "UserDatabase",
-        "methods": ["openDatabase"],
-        "properties": []
-      }
-    }
-  }
-}
-```
-
-- **interfaceDescriptors** dichiara interfacce custom con metodi e proprietà
-- Il server Scrypted usa questi descrittori per esporre i metodi via RPC sulla socket
-- Il client può chiamare `device.openDatabase()` invece di fare HTTP
-
----
-
-## 2. Metodi Events App da esporre (handleEventsAppRequest)
-
-| apimethod | payload | Note |
-|-----------|---------|------|
-| GetConfigs | — | |
-| GetCamerasStatus | — | |
-| GetEvents | fromDate, tillDate, limit, offset, sources, cameras, detectionClasses, eventSource, filter, groupingRange | |
-| GetVideoclips | fromDate, tillDate, limit, offset, cameras, detectionClasses | |
-| GetCameraDayData | deviceId, day | |
-| GetClusteredDayData | deviceId, days, bucketMs, enabledClasses, classFilter | |
-| GetClusterEvents | clusterId, deviceId, startMs, endMs | |
-| GetArtifacts | deviceId, day | |
-| GetLatestRuleArtifacts | deviceId, limit | |
-| RemoteLog | level, message | |
-
----
-
-## 3. Modifiche al plugin advanced-notifier
-
-### 3.1 package.json — aggiungere interfaceDescriptors
-
-```json
-{
-  "scrypted": {
-    "interfaces": ["Settings", "DeviceProvider", "MixinProvider", "HttpRequestHandler", "Videoclips", "LauncherApplication", "PushHandler"],
-    "interfaceDescriptors": {
-      "EventsAppApi": {
-        "name": "EventsAppApi",
-        "methods": [
-          "getConfigs",
-          "getCamerasStatus",
-          "getEvents",
-          "getVideoclips",
-          "getCameraDayData",
-          "getClusteredDayData",
-          "getClusterEvents",
-          "getArtifacts",
-          "getLatestRuleArtifacts",
-          "remoteLog"
-        ],
-        "properties": []
-      }
-    }
-  }
-}
-```
-
-### 3.2 utils.ts — costante interfaccia
-
-```ts
-export const EVENTS_APP_API_INTERFACE = "EventsAppApi";
-```
-
-### 3.3 main.ts — aggiungere interfaccia al data fetcher
-
-In `onDeviceDiscovered` per DATA_FETCHER_NATIVE_ID:
-
-```ts
-interfaces: [
-  ScryptedInterface.VideoClips,
-  ScryptedInterface.EventRecorder,
-  ScryptedInterface.Settings,
-  EVENTS_APP_API_INTERFACE,  // <-- aggiungere
-],
-```
-
-### 3.4 dataFetcher.ts — implementare EventsAppApi
-
-La classe `AdvancedNotifierDataFetcher` deve implementare i metodi pubblici che mappano 1:1 con gli apimethod. Esempio:
-
-```ts
-// EventsAppApi interface
-async getConfigs(): Promise<{ cameras: ...; enabledDetectionSources: string[] }> {
-  const { statusCode, body } = await this.handleEventsAppRequest('GetConfigs', {});
-  if (statusCode !== 200) throw new Error(JSON.stringify(body));
-  return body as any;
-}
-async getCamerasStatus(): Promise<CamerasStatusResponse> { ... }
-async getEvents(payload: GetEventsPayload): Promise<GetEventsResponse> { ... }
-// ... etc
-```
-
-Oppure, più pulito: estrarre la logica da `handleEventsAppRequest` in metodi dedicati e far sì che `handleEventsAppRequest` li chiami, così si evita duplicazione.
-
----
-
-## 4. Come il server Scrypted gestisce interfaceDescriptors
-
-Il server Scrypted (koush/scrypted) legge `interfaceDescriptors` dal `package.json` del plugin. Quando un device dichiara un'interfaccia in `interfaces`, il server:
-
-1. Verifica che l'interfaccia sia in `interfaceDescriptors` (per interfacce custom)
-2. Espone i metodi via RPC sulla socket Engine.IO
-3. Il client `@scrypted/client` può chiamare `device.getConfigs()` e la chiamata viene serializzata e inviata via socket
-
-Non serve modificare il server: il supporto è già presente. Il client deve solo usare `client.systemManager.getDeviceById(deviceId)` e chiamare i metodi sull'oggetto restituito.
-
----
-
-## 5. Modifiche al client (camstack / scrypted-an-frontend)
-
-### 5.1 Trovare il device Events App
-
-Il device "Advanced notifier data fetcher" ha tipo `API` e implementa `EventsAppApi`. Per ottenere il suo ID:
-
-```ts
-const state = client.systemManager.getSystemState();
-const eventsAppDeviceId = Object.entries(state).find(
-  ([_, d]) => (d as any)?.interfaces?.includes?.('EventsAppApi')
-)?.[0];
-```
-
-Oppure cercare per nome/tipo se lo stato lo espone.
-
-### 5.2 Sostituire fetch con chiamate SDK
-
-**Prima (HTTP):**
-```ts
-const res = await fetch(`${baseUrl}/eventsApp`, {
-  method: 'POST',
-  body: JSON.stringify({ apimethod: 'GetClusteredDayData', payload: { deviceId, days, bucketMs } }),
-  headers: { 'Content-Type': 'application/json', Authorization: getAuthHeader(auth) },
-});
-const data = await res.json();
-```
-
-**Dopo (Socket):**
-```ts
-const client = await getScryptedClient(auth);
-const device = client.systemManager.getDeviceById(eventsAppDeviceId) as EventsAppApi;
-const data = await device.getClusteredDayData({ deviceId, days, bucketMs });
-```
-
-### 5.3 Cosa resta su HTTP
-
-- **URL di immagini/thumbnail/video**: usati in `<Image src={url} />` e `<Video source={{ uri }} />` — devono restare URL HTTP. Il plugin continua a servire `/eventThumbnail/...`, `/eventImage/...`, `/eventVideoclip/...` via HttpRequestHandler.
-- **Autenticazione**: la socket SDK usa già le credenziali del client (login con username/password). Non serve più Basic auth per le chiamate dati.
-
----
-
-## 6. Ordine di implementazione
-
-1. **Plugin**: aggiungere `interfaceDescriptors` e `EVENTS_APP_API_INTERFACE`, implementare i metodi su `AdvancedNotifierDataFetcher`
-2. **Mantenere HttpRequestHandler**: per `apimethod` POST a `/eventsApp` — opzionale durante la transizione (fallback)
-3. **Client**: creare `eventsAppSdk.ts` che usa la socket; `eventsAppApi.ts` può passare a usare l'SDK quando il client è connesso
-4. **Rimuovere** le chiamate fetch a `/eventsApp` dal client una volta validato l'SDK
-
----
-
-## 7. Riferimenti
-
-- [LLM plugin package.json](https://github.com/scryptedapp/llm/blob/main/package.json) — esempio interfaceDescriptors
-- [Scrypted Developer Docs](https://developer.scrypted.app/) — interfacce e plugin
-- [@scrypted/client](https://www.npmjs.com/package/@scrypted/client) — SDK client con socket

package/docs/PLAN-RECORDER-MIXIN-AND-UNIFIED-PIPELINE.md

@@ -1,203 +0,0 @@
-# Piano: Advanced Notifier Recorder mixin e pipeline unificata
-
-Piano per un’estensiva modifica al plugin Advanced Notifier: **sostituire** decoder, ffmpeg audio e recorder attuali con **una sola pipeline** che supporti clip on-demand, recording con retention (motion, detection, ecc.) e spostare tutta la logica eventi/recording in un nuovo mixin **Advanced Notifier Recorder**.
-
----
-
-## 1. Situazione attuale (da sostituire)
-
-### 1.1 Componenti separati
-
-| Componente | File | Ruolo | Input | Output |
-|------------|------|--------|-------|--------|
-| **Decoder** | `cameraMixin.ts` | Loop frame per motion/detection | `getVideoStream(decoderStream)` (solo video, no audio) | JPEG in `lastFrame` + `storeDecoderFrame()` |
-| **Audio** | `audioAnalyzerUtils.ts` | Analisi volume/classificazione | RTSP → ffmpeg `-vn -dn -sn` → PCM 16kHz mono | Eventi `audio` → `processAudioDetection` → `addMotionEvent` |
-| **Recording** | `videoRecorderUtils.ts` | Clip su trigger (recording rules) | RTSP → ffmpeg `-c:v copy|libx264` (no audio in pratica) | `.mp4` in `recordedEvents/` |
-
-Problemi:
-- **Tre consumi separati** dello stream (decoder, audio ffmpeg, recording ffmpeg) = più connessioni RTSP e più carico.
-- **Nessuna pipeline unica** video+audio: decoder senza audio, recorder senza audio reale, audio da secondo ffmpeg.
-- **Eventi e recording** sono in `cameraMixin` + `main.ts`; nessun modulo dedicato “recorder/events”.
-
-### 1.2 Dove vivono eventi e clip oggi
-
-- **Scrittura eventi:** `main.ts` → `storeEventImage()`, `addMotionEvent()` → `enqueueDbWrite` → `writeEventsAndMotionBatch()` in `db.ts` (path `storagePath/{deviceId}/events/dbs/{YYYYMMDD}.json`).
-- **Trigger recording:** `cameraMixin.processAccumulatedDetections()` → `startRecording({ triggerTime, rules, candidates })`; prolungamento da `ensureRecordingMotionCheckInterval()`.
-- **Clip:** `cameraMixin.getVideoClipsInternal()` legge da `recordedEventsPath` e da rule-generated path; `VideoRtspFfmpegRecorder` scrive in `recordedEvents/`.
-
----
-
-## 2. Obiettivi
-
-1. **Una sola pipeline** per dispositivo camera:
-   - Un input (stream video, con o senza audio).
-   - Da lì: **frame per analysis** (motion/detection), **audio** (se presente), **segmenti di recording** (buffer/scratch + clip finali).
-2. **Clip on-demand:** generazione clip a partire da un intervallo temporale (es. “ultimi 30 s”, o “dalle 12:00:00 per 60 s”) usando la pipeline, senza avviare un secondo ffmpeg ad hoc.
-3. **Recording con retention rules:** continuare a supportare “record on motion / on detection” con regole configurabili; retention (es. “tieni 7 giorni”, “solo eventi con persona”) gestita nel nuovo mixin.
-4. **Nuovo mixin “Advanced Notifier Recorder”:** contiene tutta la logica eventi + recording + clip; il camera mixin resta “analysis + notifiche”, il plugin orchestra e espone API.
-
----
-
-## 3. Architettura target
-
-### 3.1 Pipeline unica (per camera)
-
-```
-                    ┌─────────────────────────────────────────────────────────┐
-                    │                  UNIFIED RECORDER PIPELINE                │
-  Stream (RTSP/     │  ┌─────────────┐    ┌──────────────┐    ┌─────────────┐  │
-  getVideoStream)   │  │   Ingest    │───▶│   Circular   │───▶│   Outputs   │  │
-  ─────────────────▶│  │ (demux +   │    │   Buffer     │    │ - Analysis   │  │
-                    │  │  optional   │    │ (e.g. 60s)   │    │   frames     │  │
-                    │  │  audio)     │    │              │    │ - Audio      │  │
-                    │  └─────────────┘    └──────┬───────┘    │   chunks     │  │
-                    │         │                   │           │ - Clips      │  │
-                    │         │                   │           │   (on-demand │  │
-                    │         │                   │           │   or rule)  │  │
-                    │         │                   ▼           └─────────────┘  │
-                    │         │            ┌──────────────┐                     │
-                    │         └───────────│  Retention   │                     │
-                    │                      │  & clip      │                     │
-                    │                      │  writer      │                     │
-                    └─────────────────────┴──────────────┴─────────────────────┘
-```
-
-- **Ingest:** un processo ffmpeg (o un solo consumer Scrypted) che legge **un** stream (video + audio se disponibile): demux, decode video (e opzionale audio), scrive in un **buffer circolare** (segmenti in memoria o su disco, es. anelli da 60 s).
-- **Outputs dalla pipeline:**
-  - **Analysis:** copia frame (o callback) verso il decoder esistente / motion / detection (così il camera mixin continua a ricevere frame senza aprire un secondo stream).
-  - **Audio:** stessi chunk PCM usati per analisi (soglie, YAMNET) e, se serve, per mux nei clip.
-  - **Clips:**
-    - **On-demand:** da buffer circolare + eventuale “tail live” → un segmento [start, end] → file .mp4 (o altro) generato dalla pipeline (es. segmenti già in formato adatto, o un secondo pass ffmpeg breve).
-    - **Retention rules:** quando una regola dice “record”, la pipeline scrive da buffer + live in un file in `recordedEvents/` (o path configurato), con possibile post-processing (thumbnail, metadati).
-
-### 3.2 Nuovo mixin: Advanced Notifier Recorder
-
-- **Nome proposto:** `AdvancedNotifierRecorderMixin` (file es. `src/recorderMixin.ts`).
-- **Interfacce Scrypted da considerare:** `EventRecorder`, `VideoClips` (se già usate), più l’eventuale nuova interfaccia per “clip on-demand” (es. `getClipForTimeRange(deviceId, startTime, endTime)`).
-- **Responsabilità:**
-  - **Eventi:** ricevere “eventi” e “motion” dal camera mixin (o dalla pipeline) e scriverli nel DB (delega a `main.ts` per `enqueueDbWrite` o incorpora la logica se si sposta anche la queue nel recorder).
-  - **Recording:** gestione regole di recording (motion, detection, retention); avvio/arresto segmenti di recording tramite la **pipeline unica** (non più `VideoRtspFfmpegRecorder` separato).
-  - **Clip on-demand:** esporre API per “genera clip per [deviceId, start, end]” usando il buffer + writer della pipeline.
-  - **Retention:** pulizia clip/segmenti secondo retention rules (giorni, tipo evento, spazio disco); possibile integrazione con “rimuovi clip più vecchi di X” già presente in `main.ts`.
-
-### 3.3 Ruolo del camera mixin dopo il refactor
-
-- **Conservare:** regole detection, notifiche, occupancy, timelapse, UI settings (decoder type, stream destination, ecc.).
-- **Cambiare:**
-  - **Decoder:** non più un loop che chiama `getVideoStream()` da solo; invece **riceve frame dalla pipeline** del recorder (o legge da un’API del recorder “get next frame for analysis”). In questo modo c’è un solo consumer dello stream.
-  - **Audio:** non più `AudioRtspFfmpegStream` nel camera mixin; il recorder espone chunk audio (o callback) e il camera mixin continua a chiamare `processAudioDetection` con quei chunk.
-  - **Recording:** nessuna chiamata a `startRecording` / `VideoRtspFfmpegRecorder` dal camera mixin; il camera mixin segnala al recorder “c’è un evento/motion che matcha una recording rule” e il **recorder** avvia/ prolunga il segmento tramite la pipeline.
-- **Eventi:** il camera mixin può continuare a chiamare `plugin.storeEventImage()` e `plugin.addMotionEvent()`; l’implementazione di queste può essere spostata nel recorder mixin (e il plugin le delega al recorder), così tutta la “scrittura eventi” è in un posto.
-
-### 3.4 Plugin (main.ts)
-
-- **Composizione:** oltre a `AdvancedNotifierCameraMixin` e `AdvancedNotifierNotifierMixin`, introdurre `AdvancedNotifierRecorderMixin`.
-  - Opzione A: il **recorder è un mixin sulla stessa camera** (stesso device, tre mixin: notifier, camera, recorder). La pipeline unica è di proprietà del recorder; camera e notifier la “usano” tramite il recorder.
-  - Opzione B: il recorder è un **device separato** “Recorder” per camera (uno-a-uno). La pipeline vive nel device Recorder; la camera mixin comunica con esso via plugin.
-- **Percorsi e storage:** `getRecordedEventPath`, `getEventPaths`, `storeEventImage`, `addMotionEvent` possono restare in `main.ts` come facade che delega al recorder mixin (per device camera), così l’API pubblica del plugin non cambia.
-- **DB queue:** `dbWriteQueue` / `enqueueDbWrite` / `runDbWriteProcess` possono restare in `main.ts` o essere spostati nel recorder; il recorder in ogni caso deve poter scrivere eventi/motion nel DB.
-
----
-
-## 4. Pipeline unica: dettaglio tecnico
-
-### 4.1 Scelta implementativa
-
-- **Opzione 1 – FFmpeg unico (demux + buffer + tee):** un processo ffmpeg che:
-  - Legge RTSP (o riceve stream da Scrypted).
-  - Demux video + audio.
-  - Scrive in un **segment file** circolare (es. `segment_%03d.m4s` o simile) o in un **named pipe / shared memory** letto da Node.
-  - Opzionale: `tee` per inviare copia a un secondo output (es. analisi).
-  - Pro: un solo processo, meno connessioni. Contro: complessità buffer/segmenti e sincronizzazione con “clip da intervallo”.
-- **Opzione 2 – Consumer Scrypted + buffer in Node:** un solo `getVideoStream()` (con audio se il backend lo supporta); in Node un consumer che:
-  - Legge frame (e eventuale audio) e li mette in un **buffer circolare** (es. anello di segmenti in memoria o file).
-  - Espone “slice del buffer” per clip on-demand e per “scrivi da start a end” per recording.
-  - Pro: massimo controllo in JS. Contro: possibile overhead e complessità (codec, mux) se i frame arrivano già codificati.
-- **Opzione 3 – Ibrido:** ffmpeg per ingest e buffer su disco (segmenti brevi, es. 5–10 s); un servizio in Node che tiene un indice (startTime → file) e per clip on-demand concatena/rimux con ffmpeg. Recording “su regola” = copia di segmenti già scritti + append live fino a fine evento.
-
-Raccomandazione: partire da **Opzione 3** per avere un buffer su disco ben definito e clip on-demand affidabili; unificare comunque in **un solo ffmpeg di ingest** (video+audio) che produce segmenti, e un “RecorderPipeline” in Node che gestisce indice, retention e generazione clip.
-
-### 4.2 Buffer circolare / segmenti
-
-- **Formato:** segmenti brevi (es. 5–15 s) in formato adatto al concatenamento (es. fMP4 o segmenti MPEG-TS).
-- **Indice:** struttura (in memoria o file) che mappa `[startTime, endTime]` → lista file segmenti.
-- **Retention:** job periodico che rimuove segmenti oltre la retention (o oltre lo spazio massimo); i clip “recorded” (salvati in `recordedEvents/`) sono copie permanenti fino a quando non scatta la loro retention.
-
-### 4.3 Clip on-demand
-
-- **Input:** `deviceId`, `startTime`, `endTime` (timestamp Unix o ms).
-- **Logica:** dalla pipeline (indice segmenti) individuare i segmenti che coprono [startTime, endTime]; concatenare (concat demuxer ffmpeg o copy) e scrivere un file .mp4; opzionale: estrarre thumbnail a metà clip.
-- **Output:** path del file clip (e thumbnail) da esporre via API (es. `VideoClips.getVideoClip` o nuova `getClipForTimeRange`).
-
-### 4.4 Recording con retention rules
-
-- **Regole:** come oggi (motion, classi detection, ecc.) ma interpretate dal **recorder mixin**.
-- **Trigger:** il camera mixin (o la pipeline) segnala “motion on” / “detection X”; il recorder confronta con le regole e decide “start recording” / “prolong”.
-- **Scrittura:** invece di avviare un `VideoRtspFfmpegRecorder` separato, il recorder dice alla pipeline “da adesso scrivi in un file in `recordedEvents/` fino a fine evento (o max duration)”. La pipeline può:
-  - copiare dal buffer (segmenti già scritti) per la parte “pre-trigger” (es. 30 s prima),
-  - poi appendere live fino a “motion off” + post-buffer.
-- **Retention:** regole tipo “conserva 7 giorni”, “solo eventi con persona”; il recorder applica la pulizia sui file in `recordedEvents/` (e eventualmente sui segmenti del buffer).
-
----
-
-## 5. Piano di implementazione (fasi)
-
-### Fase 1 – Fondamenta recorder e spostamento eventi
-
-1. **Creare `recorderMixin.ts`** (Advanced Notifier Recorder mixin).
-   - Interfacce: almeno ciò che serve per “eventi” e “clip” (EventRecorder / VideoClips se già usate).
-   - Implementare **delega** di `storeEventImage` e `addMotionEvent`: il plugin, quando è un device camera con recorder mixin, inoltra al recorder; il recorder chiama la stessa logica di scrittura DB (o sposta `enqueueDbWrite` nel recorder).
-2. **Registrare il mixin in `main.ts`:** per le camera, creare anche il recorder mixin (stesso device o device figlio); mantenere l’API `storeEventImage` / `addMotionEvent` sul plugin che delega al recorder.
-3. **Test:** verificare che eventi e motion continuino a essere scritti e letti come oggi (Events App, Data Fetcher).
-
-### Fase 2 – Pipeline unica (ingest + buffer)
-
-1. **Modulo “RecorderPipeline”** (es. `src/recorderPipeline.ts` o sotto `src/recorder/`):
-   - Ingest: un processo ffmpeg che legge **un** stream (RTSP o URL da `getVideoStream` se possibile) con **video + audio**, output in segmenti (fMP4 o TS).
-   - Parametri: cameraId, stream URL, path directory segmenti, lunghezza segmento, lunghezza buffer (es. 60 s = 12 segmenti da 5 s).
-   - Scrittura segmenti e indice (startTime/endTime per segmento).
-2. **Integrazione nel recorder mixin:** all’avvio della camera (o on-demand quando serve recording/clip), avviare la pipeline per quella camera; fermarla quando la camera viene rilasciata.
-3. **Sostituire l’audio analyzer:** invece di avviare `AudioRtspFfmpegStream`, il recorder legge l’audio dalla pipeline (dai segmenti o da un output ffmpeg dedicato “solo audio” tee). Fornire i chunk al camera mixin per `processAudioDetection` (stessa API).
-4. **Sostituire il decoder:** il decoder non chiama più `getVideoStream()` direttamente; la pipeline espone “frame per analysis” (es. estrazione frame dai segmenti con ffmpeg, o tee video verso un output che il camera mixin consuma). Il camera mixin continua a fare motion/detection sui frame così forniti.
-
-### Fase 3 – Recording e clip dalla pipeline
-
-1. **Recording:** rimuovere `VideoRtspFfmpegRecorder` e `startRecording` dal camera mixin. Nel recorder:
-   - Alla notifica “start recording” (da camera mixin o da regole interne), chiedere alla pipeline di “salvare da buffer[start] a live fino a stop”.
-   - Implementare “prolong on motion” leggendo lo stato motion dalla pipeline/camera mixin.
-2. **Clip on-demand:** implementare `getClipForTimeRange(deviceId, startTime, endTime)` (o equivalente) usando l’indice segmenti; concatenare e scrivere .mp4; restituire path o URL.
-3. **Retention:** job nel recorder che applica retention rules su `recordedEvents/` e sui segmenti del buffer; integrare con la logica di rimozione clip già presente in `main.ts` (es. spostarla nel recorder).
-
-### Fase 4 – Pulizia e opzionali
-
-1. **Rimuovere** da `cameraMixin.ts`: `startRecording`, `stopRecording`, `ensureRecordingMotionCheckInterval`, uso di `VideoRtspFfmpegRecorder`, `AudioRtspFfmpegStream` (sostituito dalla pipeline), e il loop decoder “standalone” (sostituito da frame dalla pipeline).
-2. **Deprecare** (o rimuovere) `videoRecorderUtils.ts` e `audioAnalyzerUtils.ts` nella forma attuale; eventualmente tenere helper riutilizzabili (es. estrazione thumbnail) dentro il modulo pipeline/recorder.
-3. **Documentazione:** aggiornare README e doc per “single pipeline”, “recorder mixin”, “retention rules”.
-4. **Settings:** spostare le impostazioni “recording rules”, “retention”, “buffer length”, “decoder source” (pipeline vs legacy, se si mantiene fallback) nel recorder mixin o in una sezione “Recording” condivisa.
-
----
-
-## 6. Riepilogo file toccati / nuovi
-
-| Azione | File |
-|--------|------|
-| **Nuovo** | `src/recorderMixin.ts` – Advanced Notifier Recorder mixin (eventi, recording, clip, retention). |
-| **Nuovo** | `src/recorderPipeline.ts` (o `src/recorder/`) – Ingest ffmpeg, buffer segmenti, indice, export clip. |
-| **Modifica** | `src/main.ts` – Registrazione recorder mixin, delega `storeEventImage`/`addMotionEvent` al recorder, eventuale spostamento DB queue. |
-| **Modifica** | `src/cameraMixin.ts` – Rimuovere decoder standalone, audio analyzer, startRecording/VideoRtspFfmpegRecorder; ricevere frame e audio dalla pipeline/recorder; mantenere detection, notifiche, regole. |
-| **Modifica** | `src/db.ts` – Solo se la scrittura eventi viene spostata nel recorder (stesso schema, diverso chiamante). |
-| **Deprecare/rimuovere** | `src/videoRecorderUtils.ts` – Sostituito dalla pipeline. |
-| **Deprecare/rimuovere** | `src/audioAnalyzerUtils.ts` – Sostituito da audio dalla pipeline. |
-
----
-
-## 7. Rischi e mitigazioni
-
-- **Compatibilità:** mantenere l’API pubblica del plugin (EventRecorder, VideoClips, getVideoClips, getRecordedEventPath, storeEventImage, addMotionEvent) così che camstack e altri client non cambino.
-- **Performance:** un solo ffmpeg per camera può essere un single point of failure; prevedere restart automatico e backoff come in `VideoRtspFfmpegRecorder`/`AudioRtspFfmpegStream`.
-- **Disco:** il buffer circolare su disco consuma spazio; configurare lunghezza massima e retention chiara.
-- **Migrazione:** per rollout graduale, si può mantenere un “legacy mode” (decoder + audio ffmpeg + VideoRtspFfmpegRecorder) disattivabile da setting “Use unified recorder pipeline”, e abilitare la nuova pipeline solo quando il setting è on.
-
----
-
-Questo piano può essere usato come base per issue, task e PR incrementali (una fase per volta).