@blueharford/scrypted-spatial-awareness 0.4.7 → 0.5.0-beta

package/README.md

@@ -51,6 +51,7 @@ Done! Your camera topology is configured.
 ### Visual Editor
 - **Floor Plan** - Upload image or draw with built-in tools
 - **Drag & Drop** - Place cameras, draw connections
+- **Polygon Zone Drawing** - Draw custom zones (yards, driveways, patios, etc.)
 - **Live Tracking** - Watch objects move in real-time
 
 ### AI Features (optional)
@@ -58,11 +59,74 @@ Done! Your camera topology is configured.
 - **Auto-Learning** - Transit times adjust based on observations
 - **Connection Suggestions** - System suggests new camera paths
 - **Landmark Discovery** - AI identifies landmarks from footage
+- **Auto-Topology Discovery** - Vision LLM analyzes camera views to build topology
 
 ### Integrations
 - **MQTT** - Home Assistant integration
 - **REST API** - Query tracked objects programmatically
 
+## Topology Configuration
+
+The plugin uses topology data (landmarks, zones, connections) to generate meaningful alerts. Camera names are **not** used for location descriptions - only topology landmarks matter.
+
+### Setting Up Landmarks
+
+For best alert quality, configure landmarks in the topology editor:
+
+1. **Entry/Exit Points** - Mark where people enter/exit your property
+   - Examples: `Driveway`, `Front Gate`, `Side Gate`, `Street`
+   - Set `isEntryPoint: true` or `isExitPoint: true`
+
+2. **Access Points** - Paths and walkways
+   - Examples: `Front Walkway`, `Back Path`, `Garage Door`
+   - Type: `access`
+
+3. **Zones** - Areas of your property
+   - Examples: `Front Yard`, `Back Yard`, `Side Yard`, `Patio`
+   - Type: `zone`
+
+4. **Structures** - Buildings and fixed features
+   - Examples: `Garage`, `Shed`, `Front Porch`, `Deck`
+   - Type: `structure`
+
+5. **Features** - Other notable landmarks
+   - Examples: `Mailbox`, `Pool`, `Garden`, `Trash Cans`
+   - Type: `feature`
+
+### Linking Landmarks to Cameras
+
+Each camera should have `visibleLandmarks` configured - the landmarks visible in that camera's view:
+
+```json
+{
+  "cameras": [{
+    "deviceId": "abc123",
+    "name": "Front Camera",
+    "context": {
+      "visibleLandmarks": ["front-door", "driveway", "mailbox"]
+    }
+  }]
+}
+```
+
+### Example Topology
+
+```json
+{
+  "landmarks": [
+    { "id": "driveway", "name": "Driveway", "type": "access", "isEntryPoint": true },
+    { "id": "front-door", "name": "Front Door", "type": "access" },
+    { "id": "backyard", "name": "Back Yard", "type": "zone" },
+    { "id": "garage", "name": "Garage", "type": "structure" }
+  ]
+}
+```
+
+With proper landmarks, alerts become rich and contextual:
+- "Person arrived at the Driveway from Main Street"
+- "Person moved from the Front Porch heading towards the Back Yard"
+- "Person left the Garage towards Driveway after 2m on property - visited Driveway > Front Door > Garage"
+
 ## Configuration
 
 ### Settings
@@ -108,6 +172,98 @@ Base URL: `/endpoint/@blueharford/scrypted-spatial-awareness`
 | `/api/training/apply` | POST | Apply results to topology |
 | `/api/training/status` | GET | Current training status |
 
+### Discovery API
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/discovery/scan` | POST | Run full discovery scan |
+| `/api/discovery/status` | GET | Current discovery status |
+| `/api/discovery/suggestions` | GET | Pending suggestions |
+| `/api/discovery/camera/{id}` | GET | Analyze single camera |
+
+## Auto-Topology Discovery
+
+The plugin can automatically analyze camera views using a vision-capable LLM to discover landmarks, zones, and camera connections.
+
+### How It Works
+
+1. **Capture Snapshots** - System takes a picture from each camera
+2. **Scene Analysis** - Vision LLM identifies landmarks, zones, and edges in each view
+3. **Cross-Camera Correlation** - LLM correlates findings across cameras to identify shared landmarks and connections
+4. **Suggestions** - Discoveries are presented as suggestions you can accept or reject
+
+### Using Discovery
+
+**Manual Scan:**
+1. Open the topology editor (`/ui/editor`)
+2. Find the "Auto-Discovery" section in the sidebar
+3. Click "Scan Now"
+4. Review and accept/reject suggestions
+
+**Automatic Scan:**
+- Set `Auto-Discovery Interval (hours)` in plugin settings
+- System will periodically scan and generate suggestions
+- Set to 0 to disable automatic scanning
+
+### Discovery Settings
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| Auto-Discovery Interval | 0 (disabled) | Hours between automatic scans (0 = disabled) |
+| Min Landmark Confidence | 0.6 | Minimum confidence for landmark suggestions |
+| Min Connection Confidence | 0.5 | Minimum confidence for connection suggestions |
+| Auto-Accept Threshold | 0.85 | Auto-accept suggestions above this confidence |
+
+> **Rate Limiting Note:** If you set the interval to less than 1 hour, a warning will appear in the discovery status. Frequent scans can consume significant LLM API quota and may be rate-limited by your provider.
+
+### Requirements
+
+- **Vision-capable LLM** - Install @scrypted/llm with a vision model (OpenAI GPT-4V, Claude, etc.)
+- **Camera access** - Plugin needs camera.takePicture() capability
+
+### What Gets Discovered
+
+- **Landmarks**: Doors, gates, mailbox, garage, structures, fences
+- **Zones**: Front yard, driveway, patio, street, walkways
+- **Connections**: Suggested camera paths with transit time estimates
+- **Edges**: What's visible at frame boundaries (for correlation)
+
+## Zone Drawing
+
+The visual editor includes a polygon zone drawing tool for marking areas on your floor plan.
+
+### How to Draw Zones
+
+1. Click the **Draw Zone** button in the toolbar (green)
+2. Enter a zone name and select the type (yard, driveway, patio, etc.)
+3. Click **Start Drawing**
+4. Click on the canvas to add polygon points
+5. **Double-click** or press **Enter** to finish the zone
+6. Press **Escape** to cancel, **Backspace** to undo last point
+
+### Zone Types
+
+| Type | Color | Description |
+|------|-------|-------------|
+| Yard | Green | Front yard, backyard, side yard |
+| Driveway | Gray | Driveway, parking area |
+| Street | Dark Gray | Street, sidewalk |
+| Patio | Orange | Patio, deck |
+| Walkway | Brown | Walkways, paths |
+| Parking | Light Gray | Parking lot, parking space |
+| Garden | Light Green | Garden, landscaped area |
+| Pool | Blue | Pool area |
+| Garage | Medium Gray | Garage area |
+| Entrance | Pink | Entry areas |
+| Custom | Purple | Custom zone type |
+
+### Using Zones
+
+- Click on a zone to select it and edit its properties
+- Zones are color-coded by type for easy identification
+- Zones help provide context for object movement descriptions
+- Auto-Discovery can suggest zones based on camera analysis
+
 ## MQTT Topics
 
 Base: `scrypted/spatial-awareness`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@blueharford/scrypted-spatial-awareness",
-  "version": "0.4.7",
+  "version": "0.5.0-beta",
   "description": "Cross-camera object tracking for Scrypted NVR with spatial awareness",
   "author": "Joshua Seidel <blueharford>",
   "license": "Apache-2.0",

package/src/core/object-correlator.ts

@@ -46,7 +46,24 @@ export class ObjectCorrelator {
       }
     }
 
-    if (candidates.length === 0) return null;
+    if (candidates.length === 0) {
+      // No candidates above threshold - try to find best match with relaxed criteria
+      // This helps when there's only one object of this class active
+      const sameClassObjects = activeObjects.filter(
+        o => o.className === sighting.detection.className
+      );
+
+      if (sameClassObjects.length === 1) {
+        // Only one object of this class - likely the same one
+        const candidate = await this.evaluateCandidate(sameClassObjects[0], sighting);
+        // Accept with lower threshold if timing is reasonable
+        if (candidate.confidence >= 0.3 && candidate.factors.timing > 0) {
+          return candidate;
+        }
+      }
+
+      return null;
+    }
 
     // Sort by confidence (highest first)
     candidates.sort((a, b) => b.confidence - a.confidence);
@@ -137,15 +154,23 @@ export class ObjectCorrelator {
 
     if (!connection) {
       // No defined connection - still allow correlation based on reasonable timing
-      // Allow up to 2 minutes transit between any cameras
-      const MAX_UNCHARTED_TRANSIT = 120000; // 2 minutes
+      // Allow up to 5 minutes transit between any cameras (property could be large)
+      const MAX_UNCHARTED_TRANSIT = 300000; // 5 minutes
       if (transitTime > 0 && transitTime < MAX_UNCHARTED_TRANSIT) {
         // Score based on how reasonable the timing is
-        // Shorter transits are more likely to be the same object
-        const timingScore = Math.max(0.3, 1 - (transitTime / MAX_UNCHARTED_TRANSIT));
-        return timingScore;
+        // Give higher base score for reasonable transits (encourages matching)
+        if (transitTime < 60000) {
+          // Under 1 minute - very likely same object
+          return 0.9;
+        } else if (transitTime < 120000) {
+          // Under 2 minutes - probably same object
+          return 0.7;
+        } else {
+          // 2-5 minutes - possible but less certain
+          return Math.max(0.4, 0.7 - (transitTime - 120000) / 180000 * 0.3);
+        }
       }
-      return 0.2;
+      return 0.3; // Even long transits get some credit
     }
     const { min, typical, max } = connection.transitTime;