@alteriom/painlessmesh 1.6.1 → 1.7.2

package/docs/MQTT_SCHEMA_REVIEW.md

@@ -0,0 +1,690 @@
+# MQTT Command Schema Review & Mesh Reporting Analysis
+
+**Date:** October 12, 2025  
+**Reviewer:** Alteriom Development Team  
+**Document:** MQTT_COMMAND_SCHEMA_PROPOSAL.md Analysis  
+**Status:** ✅ APPROVED with RECOMMENDED ADDITIONS
+
+---
+
+## Executive Summary
+
+### Overall Assessment
+
+The MQTT command schema proposal is **comprehensive and production-ready** for device control operations. However, analysis reveals a **critical gap in mesh network topology reporting** that should be addressed in the same release (v0.5.0).
+
+### Key Findings
+
+| Area | Status | Details |
+|------|--------|---------|
+| Command Schema | ✅ **Excellent** | Complete, well-documented, ready to implement |
+| Response Tracking | ✅ **Excellent** | Correlation IDs, latency metrics, error codes |
+| Migration Path | ✅ **Excellent** | Clear upgrade from v0.4.0 control_response |
+| Standard Commands | ✅ **Excellent** | 30+ documented commands across 6 categories |
+| **Mesh Topology** | ⚠️ **MISSING** | No schema for network structure reporting |
+| **Mesh Events** | ⚠️ **MISSING** | No schema for real-time mesh notifications |
+
+---
+
+## Part 1: Command Schema Review
+
+### ✅ Strengths
+
+#### 1. Complete Command Lifecycle
+```json
+// Command → Response with correlation
+{
+  "command": "read_sensors",
+  "correlation_id": "cmd-1728745800-001",
+  "parameters": {"immediate": true}
+}
+// ↓
+{
+  "success": true,
+  "correlation_id": "cmd-1728745800-001",
+  "latency_ms": 1250
+}
+```
+
+#### 2. Comprehensive Error Handling
+- 12 standard error codes (TIMEOUT, INVALID_PARAMS, SENSOR_NOT_AVAILABLE, etc.)
+- Human-readable messages
+- Machine-readable error_code field
+- Gateway-generated errors for timeouts
+
+#### 3. Well-Organized Command Categories
+- **Device Control (1-99):** RESET, SLEEP, LED_CONTROL, RELAY_SWITCH, PWM_SET
+- **Configuration (100-199):** GET_CONFIG, SET_CONFIG, SAVE_CONFIG, SET_SAMPLE_RATE
+- **Status (200-255):** GET_STATUS, GET_METRICS, GET_DIAGNOSTICS, START_MONITORING
+
+#### 4. Production-Ready Features
+- Priority queuing (low, normal, high, urgent)
+- Configurable timeouts (1000-300000ms)
+- Command parameter validation
+- Custom command support with `custom_` prefix
+
+#### 5. Excellent Documentation
+- 4 complete examples (success, error, timeout)
+- JavaScript and Python client code
+- Migration path from v0.4.0
+- Implementation checklist
+
+### 📋 Minor Suggestions
+
+1. **Add Batch Command Support**
+   ```json
+   {
+     "event": "command_batch",
+     "commands": [
+       {"command": "led_control", "parameters": {...}},
+       {"command": "set_interval", "parameters": {...}}
+     ],
+     "correlation_id": "batch-001"
+   }
+   ```
+
+2. **Add Scheduled Command Support**
+   ```json
+   {
+     "command": "read_sensors",
+     "schedule": {
+       "execute_at": "2025-10-12T16:00:00Z",
+       "repeat": "hourly"
+     }
+   }
+   ```
+
+---
+
+## Part 2: Mesh Reporting Gap Analysis
+
+### ⚠️ Critical Missing Feature: Topology Reporting
+
+#### The Problem
+
+**Current State:**
+- MQTT topic exists: `mesh/topology` (in implementation docs)
+- Gateway publishes topology updates
+- **NO STANDARDIZED SCHEMA** ❌
+
+**Impact:**
+```
+❌ Web dashboards can't reliably parse topology
+❌ DevOps lacks standardized monitoring format
+❌ Third-party tools can't visualize mesh
+❌ Historical topology analysis impossible
+```
+
+#### Use Cases Not Addressed
+
+**1. Network Visualization**
+```
+Dashboard needs to display:
+├── Which nodes are online?
+├── How are they connected?
+├── What's the signal quality?
+├── Where are the bottlenecks?
+└── Which paths are redundant?
+```
+
+**2. Performance Monitoring**
+```
+Monitoring system needs:
+├── Average hop count to each node
+├── Connection quality metrics
+├── Network diameter changes
+├── Node churn rate (joins/leaves per hour)
+└── Message routing efficiency
+```
+
+**3. Debugging & Troubleshooting**
+```
+When "Node X is unreachable":
+├── What path should messages take?
+├── Which intermediate nodes?
+├── Are there alternative routes?
+├── Is the network partitioned?
+└── What's the RSSI at each hop?
+```
+
+---
+
+## Part 3: Proposed Mesh Topology Schema
+
+### New Schema: mesh_topology.schema.json
+
+#### High-Level Structure
+```json
+{
+  "event": "mesh_topology",
+  "mesh_id": "MESH-001",
+  "gateway_node_id": "ALT-6825DD341CA4",
+  "nodes": [...],           // Array of node objects
+  "connections": [...],     // Array of connection/edge objects
+  "metrics": {...},         // Network-wide aggregates
+  "update_type": "full"     // "full" or "incremental"
+}
+```
+
+#### Node Object Schema
+```json
+{
+  "node_id": "ALT-441D64F804A0",
+  "role": "sensor",                    // gateway|sensor|repeater|bridge
+  "status": "online",                  // online|offline|unknown
+  "last_seen": "2025-10-12T14:59:58Z",
+  "firmware_version": "SN 2.3.4",
+  "uptime_seconds": 72000,
+  "free_memory_kb": 42,
+  "connection_count": 2
+}
+```
+
+#### Connection Object Schema
+```json
+{
+  "from_node": "ALT-6825DD341CA4",
+  "to_node": "ALT-441D64F804A0",
+  "quality": 95,              // 0-100 percentage
+  "latency_ms": 12,           // Round-trip time
+  "rssi": -42,                // WiFi signal strength (dBm)
+  "hop_count": 1              // Hops from gateway
+}
+```
+
+#### Network Metrics Object
+```json
+{
+  "total_nodes": 4,
+  "online_nodes": 4,
+  "network_diameter": 2,           // Max hop count
+  "avg_connection_quality": 85,
+  "messages_per_second": 12.4
+}
+```
+
+### Complete Example: 4-Node Mesh
+
+**MQTT Topic:** `alteriom/mesh/MESH-001/topology`
+
+```json
+{
+  "schema_version": 1,
+  "device_id": "ALT-6825DD341CA4",
+  "device_type": "gateway",
+  "timestamp": "2025-10-12T15:00:00Z",
+  "firmware_version": "GW 2.3.4",
+  "event": "mesh_topology",
+  "mesh_id": "MESH-001",
+  "gateway_node_id": "ALT-6825DD341CA4",
+  "nodes": [
+    {
+      "node_id": "ALT-6825DD341CA4",
+      "role": "gateway",
+      "status": "online",
+      "last_seen": "2025-10-12T15:00:00Z",
+      "firmware_version": "GW 2.3.4",
+      "uptime_seconds": 86400,
+      "free_memory_kb": 128,
+      "connection_count": 3
+    },
+    {
+      "node_id": "ALT-441D64F804A0",
+      "role": "sensor",
+      "status": "online",
+      "last_seen": "2025-10-12T14:59:58Z",
+      "firmware_version": "SN 2.3.4",
+      "uptime_seconds": 72000,
+      "free_memory_kb": 42,
+      "connection_count": 2
+    },
+    {
+      "node_id": "ALT-9A3B2C1D0E5F",
+      "role": "sensor",
+      "status": "online",
+      "last_seen": "2025-10-12T14:59:55Z",
+      "firmware_version": "SN 2.3.4",
+      "uptime_seconds": 64800,
+      "free_memory_kb": 38,
+      "connection_count": 1
+    },
+    {
+      "node_id": "ALT-7F8E9D0A1B2C",
+      "role": "repeater",
+      "status": "online",
+      "last_seen": "2025-10-12T14:59:59Z",
+      "firmware_version": "RP 2.3.4",
+      "uptime_seconds": 43200,
+      "free_memory_kb": 96,
+      "connection_count": 3
+    }
+  ],
+  "connections": [
+    {
+      "from_node": "ALT-6825DD341CA4",
+      "to_node": "ALT-441D64F804A0",
+      "quality": 95,
+      "latency_ms": 12,
+      "rssi": -42,
+      "hop_count": 1
+    },
+    {
+      "from_node": "ALT-6825DD341CA4",
+      "to_node": "ALT-7F8E9D0A1B2C",
+      "quality": 88,
+      "latency_ms": 18,
+      "rssi": -55,
+      "hop_count": 1
+    },
+    {
+      "from_node": "ALT-441D64F804A0",
+      "to_node": "ALT-7F8E9D0A1B2C",
+      "quality": 82,
+      "latency_ms": 24,
+      "rssi": -62,
+      "hop_count": 2
+    },
+    {
+      "from_node": "ALT-7F8E9D0A1B2C",
+      "to_node": "ALT-9A3B2C1D0E5F",
+      "quality": 75,
+      "latency_ms": 32,
+      "rssi": -68,
+      "hop_count": 2
+    }
+  ],
+  "metrics": {
+    "total_nodes": 4,
+    "online_nodes": 4,
+    "network_diameter": 2,
+    "avg_connection_quality": 85,
+    "messages_per_second": 12.4
+  },
+  "update_type": "full"
+}
+```
+
+### Incremental Updates
+
+**When a node joins:**
+```json
+{
+  "event": "mesh_topology",
+  "mesh_id": "MESH-001",
+  "nodes": [
+    {
+      "node_id": "ALT-NEW12345678",
+      "role": "sensor",
+      "status": "online"
+    }
+  ],
+  "connections": [
+    {
+      "from_node": "ALT-7F8E9D0A1B2C",
+      "to_node": "ALT-NEW12345678",
+      "quality": 78
+    }
+  ],
+  "update_type": "incremental"
+}
+```
+
+---
+
+## Part 4: Additional Mesh Schemas
+
+### Schema 2: mesh_event.schema.json
+
+**Purpose:** Real-time notifications of mesh state changes
+
+```json
+{
+  "event": "mesh_event",
+  "event_type": "node_leave",
+  "affected_nodes": ["ALT-441D64F804A0"],
+  "timestamp": "2025-10-12T15:10:00Z",
+  "details": {
+    "reason": "timeout",
+    "last_seen": "2025-10-12T15:08:45Z",
+    "connections_lost": 2
+  }
+}
+```
+
+**Event Types:**
+- `node_join` - New node entered mesh
+- `node_leave` - Node left mesh (clean disconnect)
+- `node_timeout` - Node lost due to timeout
+- `connection_lost` - Direct connection failed
+- `connection_restored` - Connection recovered
+- `network_split` - Mesh partitioned
+- `network_merged` - Partitions rejoined
+- `route_changed` - Routing table updated
+
+### Schema 3: mesh_diagnostics.schema.json
+
+**Purpose:** Detailed mesh health information
+
+```json
+{
+  "event": "mesh_diagnostics",
+  "diagnostic_type": "full_report",
+  "routing_table": [
+    {
+      "destination": "ALT-441D64F804A0",
+      "next_hop": "ALT-441D64F804A0",
+      "hop_count": 1,
+      "path_quality": 95
+    }
+  ],
+  "message_statistics": {
+    "total_sent": 15432,
+    "total_received": 14987,
+    "total_dropped": 45,
+    "retransmissions": 123
+  },
+  "connection_history": [
+    {
+      "node_id": "ALT-441D64F804A0",
+      "connects": 1,
+      "disconnects": 0,
+      "avg_uptime_seconds": 72000
+    }
+  ]
+}
+```
+
+---
+
+## Part 5: Mesh Command Extensions
+
+### Add Mesh-Specific Commands (300-399)
+
+Extend the standard command list with mesh operations:
+
+```json
+// Topology management
+"get_topology"      // Request current mesh topology
+"scan_neighbors"    // Scan for nearby mesh nodes
+"force_reconnect"   // Force reconnection to mesh
+"optimize_routes"   // Trigger routing optimization
+
+// Diagnostics
+"mesh_diagnostics"  // Run mesh health check
+"connection_test"   // Test connection to specific node
+"trace_route"       // Trace message path to node
+
+// Network management
+"set_tx_power"      // Adjust WiFi transmit power
+"change_channel"    // Switch WiFi channel
+"isolate_node"      // Temporarily isolate node for testing
+```
+
+**Example: Get Topology Command**
+```json
+{
+  "event": "command",
+  "command": "get_topology",
+  "correlation_id": "cmd-topology-001",
+  "parameters": {
+    "format": "full",          // "full" or "summary"
+    "include_metrics": true,
+    "include_history": false
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "event": "command_response",
+  "command": "get_topology",
+  "correlation_id": "cmd-topology-001",
+  "success": true,
+  "result": {
+    // Full topology object as per mesh_topology.schema.json
+  },
+  "latency_ms": 850
+}
+```
+
+---
+
+## Part 6: Implementation Roadmap
+
+### v0.5.0 Scope (RECOMMENDED)
+
+#### High Priority - Include Now ✅
+
+1. **command.schema.json** (from proposal) ✅
+2. **command_response.schema.json** (from proposal) ✅
+3. **mesh_topology.schema.json** (NEW) ⚠️
+4. **mesh_event.schema.json** (NEW) ⚠️
+
+**Rationale:** Topology reporting is essential for mesh monitoring. Without it, users can't visualize or debug their networks effectively.
+
+#### Medium Priority - Consider for v0.5.0 📋
+
+5. **mesh_diagnostics.schema.json** (NEW)
+6. **Mesh command extensions** (300-399 command IDs)
+
+**Rationale:** Nice-to-have for advanced debugging, but not blocking.
+
+#### Low Priority - Defer to v0.6.0 ⏳
+
+7. **node_discovery.schema.json** (auto-configuration)
+8. **route_metrics.schema.json** (per-message tracking)
+9. **Batch command schema** (send multiple commands)
+10. **Scheduled command schema** (future execution)
+
+---
+
+## Part 7: Schema Comparison Matrix
+
+| Feature | Command Proposal | Mesh Addition | Current painlessMesh | Gap |
+|---------|-----------------|---------------|---------------------|-----|
+| Device control | ✅ Covered | N/A | ✅ Implemented | ✅ None |
+| Command tracking | ✅ correlation_id | N/A | ✅ commandId field | ✅ None |
+| Error handling | ✅ 12 error codes | N/A | ✅ StatusPackage | ✅ None |
+| Network topology | ❌ Not covered | ✅ Proposed | ⚠️ Ad-hoc format | ⚠️ **Critical** |
+| Node list | ❌ Not covered | ✅ Proposed | ✅ getNodeList() | ⚠️ Schema needed |
+| Connection graph | ❌ Not covered | ✅ Proposed | ⚠️ Not exposed | ⚠️ **Critical** |
+| Signal quality | ❌ Not covered | ✅ Proposed (RSSI) | ⚠️ Not exposed | ⚠️ Important |
+| Network events | ❌ Not covered | ✅ Proposed | ⚠️ Callbacks only | ⚠️ Important |
+| Routing table | ❌ Not covered | ✅ Proposed | ⚠️ Internal only | 📋 Future |
+| Hop counts | ❌ Not covered | ✅ Proposed | ⚠️ Not tracked | 📋 Future |
+
+---
+
+## Part 8: Benefits Analysis
+
+### With Command Schema Only (Proposal)
+
+✅ Control devices remotely  
+✅ Track command execution  
+✅ Handle errors gracefully  
+❌ Can't visualize network  
+❌ Can't debug connectivity issues  
+❌ Can't monitor mesh health
+
+### With Command + Topology Schemas (Recommended)
+
+✅ Control devices remotely  
+✅ Track command execution  
+✅ Handle errors gracefully  
+✅ **Visualize network graph** (D3.js/Cytoscape.js)  
+✅ **Debug connectivity issues** (trace paths)  
+✅ **Monitor mesh health** (quality metrics)  
+✅ **Detect network problems** (partitions, bottlenecks)  
+✅ **Historical analysis** (topology over time)  
+✅ **Automated alerts** (node offline, poor quality)
+
+---
+
+## Part 9: Example Web Dashboard Integration
+
+### Topology Visualization with D3.js
+
+```javascript
+import mqtt from 'mqtt';
+import * as d3 from 'd3';
+
+const client = mqtt.connect('mqtt://broker.local:1883');
+client.subscribe('alteriom/mesh/+/topology');
+
+client.on('message', (topic, message) => {
+  const topology = JSON.parse(message.toString());
+  
+  if (topology.event === 'mesh_topology') {
+    renderMeshGraph(topology);
+  }
+});
+
+function renderMeshGraph(topology) {
+  const nodes = topology.nodes.map(n => ({
+    id: n.node_id,
+    role: n.role,
+    status: n.status,
+    memory: n.free_memory_kb
+  }));
+  
+  const links = topology.connections.map(c => ({
+    source: c.from_node,
+    target: c.to_node,
+    quality: c.quality,
+    latency: c.latency_ms
+  }));
+  
+  // D3.js force-directed graph
+  const simulation = d3.forceSimulation(nodes)
+    .force('link', d3.forceLink(links).id(d => d.id))
+    .force('charge', d3.forceManyBody())
+    .force('center', d3.forceCenter(width / 2, height / 2));
+  
+  // Render nodes with color based on status
+  svg.selectAll('circle')
+    .data(nodes)
+    .enter().append('circle')
+    .attr('r', 20)
+    .attr('fill', d => d.status === 'online' ? 'green' : 'red');
+  
+  // Render links with thickness based on quality
+  svg.selectAll('line')
+    .data(links)
+    .enter().append('line')
+    .attr('stroke-width', d => d.quality / 10)
+    .attr('stroke', d => d.quality > 80 ? 'green' : d.quality > 50 ? 'orange' : 'red');
+}
+```
+
+### Real-Time Event Monitoring
+
+```javascript
+client.subscribe('alteriom/mesh/+/events');
+
+client.on('message', (topic, message) => {
+  const event = JSON.parse(message.toString());
+  
+  if (event.event === 'mesh_event') {
+    switch (event.event_type) {
+      case 'node_leave':
+        showNotification('⚠️ Node Offline', 
+          `Node ${event.affected_nodes[0]} disconnected`);
+        updateTopology(); // Refresh graph
+        break;
+      
+      case 'node_join':
+        showNotification('✅ New Node', 
+          `Node ${event.affected_nodes[0]} joined mesh`);
+        updateTopology();
+        break;
+      
+      case 'network_split':
+        showAlert('🚨 Network Partition Detected!', 
+          'Mesh has split into multiple segments');
+        break;
+    }
+  }
+});
+```
+
+---
+
+## Part 10: Recommendations
+
+### For @alteriom/mqtt-schema Maintainers
+
+#### Immediate Actions (v0.5.0)
+
+1. ✅ **Approve command schemas** from proposal (ready as-is)
+2. ⚠️ **Add mesh_topology.schema.json** (critical for monitoring)
+3. ⚠️ **Add mesh_event.schema.json** (important for alerts)
+4. 📝 **Update TypeScript types** to include mesh schemas
+5. 🧪 **Add validation tests** for topology messages
+
+#### Future Considerations (v0.6.0)
+
+6. 📋 **Add mesh_diagnostics.schema.json** (advanced debugging)
+7. 📋 **Add node_discovery.schema.json** (auto-configuration)
+8. ⏳ **Add batch_command.schema.json** (efficiency)
+9. ⏳ **Add scheduled_command.schema.json** (automation)
+
+### For alteriom-firmware Team
+
+#### Immediate Actions
+
+1. ✅ **Implement command bridge** (already done!)
+2. ⚠️ **Add topology reporter** in gateway
+   - Export `mesh.getNodeList()` as topology message
+   - Publish full topology every 60 seconds
+   - Publish incremental updates on node join/leave
+3. ⚠️ **Add mesh event publisher**
+   - Hook into painlessMesh callbacks
+   - Publish `mesh_event` on topology changes
+4. 🧪 **Test with web dashboard** (D3.js visualization)
+
+#### Future Work
+
+5. 📋 **Add `get_topology` command handler**
+6. 📋 **Implement mesh diagnostics command**
+7. 📋 **Add connection quality tracking** (RSSI, latency)
+
+---
+
+## Conclusion
+
+### Summary
+
+The MQTT command schema proposal is **excellent and ready for implementation**. However, to provide a **complete mesh management solution**, we strongly recommend adding **mesh topology and event schemas** in the same release (v0.5.0).
+
+### Final Verdict
+
+| Component | Status | Action |
+|-----------|--------|--------|
+| Command Schema | ✅ **APPROVED** | Implement as proposed |
+| Response Schema | ✅ **APPROVED** | Implement as proposed |
+| Topology Schema | ⚠️ **MISSING** | **Add to v0.5.0** |
+| Event Schema | ⚠️ **MISSING** | **Add to v0.5.0** |
+
+### Proposed v0.5.0 Release Scope
+
+**Include:**
+1. command.schema.json ✅
+2. command_response.schema.json ✅
+3. mesh_topology.schema.json ⚠️ **NEW**
+4. mesh_event.schema.json ⚠️ **NEW**
+
+**Benefits:**
+- Complete bidirectional control (commands + responses)
+- Complete mesh visibility (topology + events)
+- Production-ready monitoring solution
+- Enables web dashboard visualization
+- Supports automated alerting
+
+---
+
+**Reviewed By:** Alteriom Development Team  
+**Date:** October 12, 2025  
+**Status:** ✅ APPROVED WITH ADDITIONS RECOMMENDED  
+**Next Step:** Submit mesh schemas to @alteriom/mqtt-schema maintainers
+