@alteriom/painlessmesh 1.8.14 → 1.9.0

package/docs/MQTT_SCHEMA_REVIEW.md

@@ -1,690 +0,0 @@
-# MQTT Command Schema Review & Mesh Reporting Analysis
-
-**Date:** October 12, 2025  
-**Reviewer:** Alteriom Development Team  
-**Document:** MQTT_COMMAND_SCHEMA_PROPOSAL.md Analysis  
-**Status:** ✅ APPROVED with RECOMMENDED ADDITIONS
-
----
-
-## Executive Summary
-
-### Overall Assessment
-
-The MQTT command schema proposal is **comprehensive and production-ready** for device control operations. However, analysis reveals a **critical gap in mesh network topology reporting** that should be addressed in the same release (v0.5.0).
-
-### Key Findings
-
-| Area | Status | Details |
-|------|--------|---------|
-| Command Schema | ✅ **Excellent** | Complete, well-documented, ready to implement |
-| Response Tracking | ✅ **Excellent** | Correlation IDs, latency metrics, error codes |
-| Migration Path | ✅ **Excellent** | Clear upgrade from v0.4.0 control_response |
-| Standard Commands | ✅ **Excellent** | 30+ documented commands across 6 categories |
-| **Mesh Topology** | ⚠️ **MISSING** | No schema for network structure reporting |
-| **Mesh Events** | ⚠️ **MISSING** | No schema for real-time mesh notifications |
-
----
-
-## Part 1: Command Schema Review
-
-### ✅ Strengths
-
-#### 1. Complete Command Lifecycle
-```json
-// Command → Response with correlation
-{
-  "command": "read_sensors",
-  "correlation_id": "cmd-1728745800-001",
-  "parameters": {"immediate": true}
-}
-// ↓
-{
-  "success": true,
-  "correlation_id": "cmd-1728745800-001",
-  "latency_ms": 1250
-}
-```
-
-#### 2. Comprehensive Error Handling
-- 12 standard error codes (TIMEOUT, INVALID_PARAMS, SENSOR_NOT_AVAILABLE, etc.)
-- Human-readable messages
-- Machine-readable error_code field
-- Gateway-generated errors for timeouts
-
-#### 3. Well-Organized Command Categories
-- **Device Control (1-99):** RESET, SLEEP, LED_CONTROL, RELAY_SWITCH, PWM_SET
-- **Configuration (100-199):** GET_CONFIG, SET_CONFIG, SAVE_CONFIG, SET_SAMPLE_RATE
-- **Status (200-255):** GET_STATUS, GET_METRICS, GET_DIAGNOSTICS, START_MONITORING
-
-#### 4. Production-Ready Features
-- Priority queuing (low, normal, high, urgent)
-- Configurable timeouts (1000-300000ms)
-- Command parameter validation
-- Custom command support with `custom_` prefix
-
-#### 5. Excellent Documentation
-- 4 complete examples (success, error, timeout)
-- JavaScript and Python client code
-- Migration path from v0.4.0
-- Implementation checklist
-
-### 📋 Minor Suggestions
-
-1. **Add Batch Command Support**
-   ```json
-   {
-     "event": "command_batch",
-     "commands": [
-       {"command": "led_control", "parameters": {...}},
-       {"command": "set_interval", "parameters": {...}}
-     ],
-     "correlation_id": "batch-001"
-   }
-   ```
-
-2. **Add Scheduled Command Support**
-   ```json
-   {
-     "command": "read_sensors",
-     "schedule": {
-       "execute_at": "2025-10-12T16:00:00Z",
-       "repeat": "hourly"
-     }
-   }
-   ```
-
----
-
-## Part 2: Mesh Reporting Gap Analysis
-
-### ⚠️ Critical Missing Feature: Topology Reporting
-
-#### The Problem
-
-**Current State:**
-- MQTT topic exists: `mesh/topology` (in implementation docs)
-- Gateway publishes topology updates
-- **NO STANDARDIZED SCHEMA** ❌
-
-**Impact:**
-```
-❌ Web dashboards can't reliably parse topology
-❌ DevOps lacks standardized monitoring format
-❌ Third-party tools can't visualize mesh
-❌ Historical topology analysis impossible
-```
-
-#### Use Cases Not Addressed
-
-**1. Network Visualization**
-```
-Dashboard needs to display:
-├── Which nodes are online?
-├── How are they connected?
-├── What's the signal quality?
-├── Where are the bottlenecks?
-└── Which paths are redundant?
-```
-
-**2. Performance Monitoring**
-```
-Monitoring system needs:
-├── Average hop count to each node
-├── Connection quality metrics
-├── Network diameter changes
-├── Node churn rate (joins/leaves per hour)
-└── Message routing efficiency
-```
-
-**3. Debugging & Troubleshooting**
-```
-When "Node X is unreachable":
-├── What path should messages take?
-├── Which intermediate nodes?
-├── Are there alternative routes?
-├── Is the network partitioned?
-└── What's the RSSI at each hop?
-```
-
----
-
-## Part 3: Proposed Mesh Topology Schema
-
-### New Schema: mesh_topology.schema.json
-
-#### High-Level Structure
-```json
-{
-  "event": "mesh_topology",
-  "mesh_id": "MESH-001",
-  "gateway_node_id": "ALT-6825DD341CA4",
-  "nodes": [...],           // Array of node objects
-  "connections": [...],     // Array of connection/edge objects
-  "metrics": {...},         // Network-wide aggregates
-  "update_type": "full"     // "full" or "incremental"
-}
-```
-
-#### Node Object Schema
-```json
-{
-  "node_id": "ALT-441D64F804A0",
-  "role": "sensor",                    // gateway|sensor|repeater|bridge
-  "status": "online",                  // online|offline|unknown
-  "last_seen": "2025-10-12T14:59:58Z",
-  "firmware_version": "SN 2.3.4",
-  "uptime_seconds": 72000,
-  "free_memory_kb": 42,
-  "connection_count": 2
-}
-```
-
-#### Connection Object Schema
-```json
-{
-  "from_node": "ALT-6825DD341CA4",
-  "to_node": "ALT-441D64F804A0",
-  "quality": 95,              // 0-100 percentage
-  "latency_ms": 12,           // Round-trip time
-  "rssi": -42,                // WiFi signal strength (dBm)
-  "hop_count": 1              // Hops from gateway
-}
-```
-
-#### Network Metrics Object
-```json
-{
-  "total_nodes": 4,
-  "online_nodes": 4,
-  "network_diameter": 2,           // Max hop count
-  "avg_connection_quality": 85,
-  "messages_per_second": 12.4
-}
-```
-
-### Complete Example: 4-Node Mesh
-
-**MQTT Topic:** `alteriom/mesh/MESH-001/topology`
-
-```json
-{
-  "schema_version": 1,
-  "device_id": "ALT-6825DD341CA4",
-  "device_type": "gateway",
-  "timestamp": "2025-10-12T15:00:00Z",
-  "firmware_version": "GW 2.3.4",
-  "event": "mesh_topology",
-  "mesh_id": "MESH-001",
-  "gateway_node_id": "ALT-6825DD341CA4",
-  "nodes": [
-    {
-      "node_id": "ALT-6825DD341CA4",
-      "role": "gateway",
-      "status": "online",
-      "last_seen": "2025-10-12T15:00:00Z",
-      "firmware_version": "GW 2.3.4",
-      "uptime_seconds": 86400,
-      "free_memory_kb": 128,
-      "connection_count": 3
-    },
-    {
-      "node_id": "ALT-441D64F804A0",
-      "role": "sensor",
-      "status": "online",
-      "last_seen": "2025-10-12T14:59:58Z",
-      "firmware_version": "SN 2.3.4",
-      "uptime_seconds": 72000,
-      "free_memory_kb": 42,
-      "connection_count": 2
-    },
-    {
-      "node_id": "ALT-9A3B2C1D0E5F",
-      "role": "sensor",
-      "status": "online",
-      "last_seen": "2025-10-12T14:59:55Z",
-      "firmware_version": "SN 2.3.4",
-      "uptime_seconds": 64800,
-      "free_memory_kb": 38,
-      "connection_count": 1
-    },
-    {
-      "node_id": "ALT-7F8E9D0A1B2C",
-      "role": "repeater",
-      "status": "online",
-      "last_seen": "2025-10-12T14:59:59Z",
-      "firmware_version": "RP 2.3.4",
-      "uptime_seconds": 43200,
-      "free_memory_kb": 96,
-      "connection_count": 3
-    }
-  ],
-  "connections": [
-    {
-      "from_node": "ALT-6825DD341CA4",
-      "to_node": "ALT-441D64F804A0",
-      "quality": 95,
-      "latency_ms": 12,
-      "rssi": -42,
-      "hop_count": 1
-    },
-    {
-      "from_node": "ALT-6825DD341CA4",
-      "to_node": "ALT-7F8E9D0A1B2C",
-      "quality": 88,
-      "latency_ms": 18,
-      "rssi": -55,
-      "hop_count": 1
-    },
-    {
-      "from_node": "ALT-441D64F804A0",
-      "to_node": "ALT-7F8E9D0A1B2C",
-      "quality": 82,
-      "latency_ms": 24,
-      "rssi": -62,
-      "hop_count": 2
-    },
-    {
-      "from_node": "ALT-7F8E9D0A1B2C",
-      "to_node": "ALT-9A3B2C1D0E5F",
-      "quality": 75,
-      "latency_ms": 32,
-      "rssi": -68,
-      "hop_count": 2
-    }
-  ],
-  "metrics": {
-    "total_nodes": 4,
-    "online_nodes": 4,
-    "network_diameter": 2,
-    "avg_connection_quality": 85,
-    "messages_per_second": 12.4
-  },
-  "update_type": "full"
-}
-```
-
-### Incremental Updates
-
-**When a node joins:**
-```json
-{
-  "event": "mesh_topology",
-  "mesh_id": "MESH-001",
-  "nodes": [
-    {
-      "node_id": "ALT-NEW12345678",
-      "role": "sensor",
-      "status": "online"
-    }
-  ],
-  "connections": [
-    {
-      "from_node": "ALT-7F8E9D0A1B2C",
-      "to_node": "ALT-NEW12345678",
-      "quality": 78
-    }
-  ],
-  "update_type": "incremental"
-}
-```
-
----
-
-## Part 4: Additional Mesh Schemas
-
-### Schema 2: mesh_event.schema.json
-
-**Purpose:** Real-time notifications of mesh state changes
-
-```json
-{
-  "event": "mesh_event",
-  "event_type": "node_leave",
-  "affected_nodes": ["ALT-441D64F804A0"],
-  "timestamp": "2025-10-12T15:10:00Z",
-  "details": {
-    "reason": "timeout",
-    "last_seen": "2025-10-12T15:08:45Z",
-    "connections_lost": 2
-  }
-}
-```
-
-**Event Types:**
-- `node_join` - New node entered mesh
-- `node_leave` - Node left mesh (clean disconnect)
-- `node_timeout` - Node lost due to timeout
-- `connection_lost` - Direct connection failed
-- `connection_restored` - Connection recovered
-- `network_split` - Mesh partitioned
-- `network_merged` - Partitions rejoined
-- `route_changed` - Routing table updated
-
-### Schema 3: mesh_diagnostics.schema.json
-
-**Purpose:** Detailed mesh health information
-
-```json
-{
-  "event": "mesh_diagnostics",
-  "diagnostic_type": "full_report",
-  "routing_table": [
-    {
-      "destination": "ALT-441D64F804A0",
-      "next_hop": "ALT-441D64F804A0",
-      "hop_count": 1,
-      "path_quality": 95
-    }
-  ],
-  "message_statistics": {
-    "total_sent": 15432,
-    "total_received": 14987,
-    "total_dropped": 45,
-    "retransmissions": 123
-  },
-  "connection_history": [
-    {
-      "node_id": "ALT-441D64F804A0",
-      "connects": 1,
-      "disconnects": 0,
-      "avg_uptime_seconds": 72000
-    }
-  ]
-}
-```
-
----
-
-## Part 5: Mesh Command Extensions
-
-### Add Mesh-Specific Commands (300-399)
-
-Extend the standard command list with mesh operations:
-
-```json
-// Topology management
-"get_topology"      // Request current mesh topology
-"scan_neighbors"    // Scan for nearby mesh nodes
-"force_reconnect"   // Force reconnection to mesh
-"optimize_routes"   // Trigger routing optimization
-
-// Diagnostics
-"mesh_diagnostics"  // Run mesh health check
-"connection_test"   // Test connection to specific node
-"trace_route"       // Trace message path to node
-
-// Network management
-"set_tx_power"      // Adjust WiFi transmit power
-"change_channel"    // Switch WiFi channel
-"isolate_node"      // Temporarily isolate node for testing
-```
-
-**Example: Get Topology Command**
-```json
-{
-  "event": "command",
-  "command": "get_topology",
-  "correlation_id": "cmd-topology-001",
-  "parameters": {
-    "format": "full",          // "full" or "summary"
-    "include_metrics": true,
-    "include_history": false
-  }
-}
-```
-
-**Response:**
-```json
-{
-  "event": "command_response",
-  "command": "get_topology",
-  "correlation_id": "cmd-topology-001",
-  "success": true,
-  "result": {
-    // Full topology object as per mesh_topology.schema.json
-  },
-  "latency_ms": 850
-}
-```
-
----
-
-## Part 6: Implementation Roadmap
-
-### v0.5.0 Scope (RECOMMENDED)
-
-#### High Priority - Include Now ✅
-
-1. **command.schema.json** (from proposal) ✅
-2. **command_response.schema.json** (from proposal) ✅
-3. **mesh_topology.schema.json** (NEW) ⚠️
-4. **mesh_event.schema.json** (NEW) ⚠️
-
-**Rationale:** Topology reporting is essential for mesh monitoring. Without it, users can't visualize or debug their networks effectively.
-
-#### Medium Priority - Consider for v0.5.0 📋
-
-5. **mesh_diagnostics.schema.json** (NEW)
-6. **Mesh command extensions** (300-399 command IDs)
-
-**Rationale:** Nice-to-have for advanced debugging, but not blocking.
-
-#### Low Priority - Defer to v0.6.0 ⏳
-
-7. **node_discovery.schema.json** (auto-configuration)
-8. **route_metrics.schema.json** (per-message tracking)
-9. **Batch command schema** (send multiple commands)
-10. **Scheduled command schema** (future execution)
-
----
-
-## Part 7: Schema Comparison Matrix
-
-| Feature | Command Proposal | Mesh Addition | Current painlessMesh | Gap |
-|---------|-----------------|---------------|---------------------|-----|
-| Device control | ✅ Covered | N/A | ✅ Implemented | ✅ None |
-| Command tracking | ✅ correlation_id | N/A | ✅ commandId field | ✅ None |
-| Error handling | ✅ 12 error codes | N/A | ✅ StatusPackage | ✅ None |
-| Network topology | ❌ Not covered | ✅ Proposed | ⚠️ Ad-hoc format | ⚠️ **Critical** |
-| Node list | ❌ Not covered | ✅ Proposed | ✅ getNodeList() | ⚠️ Schema needed |
-| Connection graph | ❌ Not covered | ✅ Proposed | ⚠️ Not exposed | ⚠️ **Critical** |
-| Signal quality | ❌ Not covered | ✅ Proposed (RSSI) | ⚠️ Not exposed | ⚠️ Important |
-| Network events | ❌ Not covered | ✅ Proposed | ⚠️ Callbacks only | ⚠️ Important |
-| Routing table | ❌ Not covered | ✅ Proposed | ⚠️ Internal only | 📋 Future |
-| Hop counts | ❌ Not covered | ✅ Proposed | ⚠️ Not tracked | 📋 Future |
-
----
-
-## Part 8: Benefits Analysis
-
-### With Command Schema Only (Proposal)
-
-✅ Control devices remotely  
-✅ Track command execution  
-✅ Handle errors gracefully  
-❌ Can't visualize network  
-❌ Can't debug connectivity issues  
-❌ Can't monitor mesh health
-
-### With Command + Topology Schemas (Recommended)
-
-✅ Control devices remotely  
-✅ Track command execution  
-✅ Handle errors gracefully  
-✅ **Visualize network graph** (D3.js/Cytoscape.js)  
-✅ **Debug connectivity issues** (trace paths)  
-✅ **Monitor mesh health** (quality metrics)  
-✅ **Detect network problems** (partitions, bottlenecks)  
-✅ **Historical analysis** (topology over time)  
-✅ **Automated alerts** (node offline, poor quality)
-
----
-
-## Part 9: Example Web Dashboard Integration
-
-### Topology Visualization with D3.js
-
-```javascript
-import mqtt from 'mqtt';
-import * as d3 from 'd3';
-
-const client = mqtt.connect('mqtt://broker.local:1883');
-client.subscribe('alteriom/mesh/+/topology');
-
-client.on('message', (topic, message) => {
-  const topology = JSON.parse(message.toString());
-  
-  if (topology.event === 'mesh_topology') {
-    renderMeshGraph(topology);
-  }
-});
-
-function renderMeshGraph(topology) {
-  const nodes = topology.nodes.map(n => ({
-    id: n.node_id,
-    role: n.role,
-    status: n.status,
-    memory: n.free_memory_kb
-  }));
-  
-  const links = topology.connections.map(c => ({
-    source: c.from_node,
-    target: c.to_node,
-    quality: c.quality,
-    latency: c.latency_ms
-  }));
-  
-  // D3.js force-directed graph
-  const simulation = d3.forceSimulation(nodes)
-    .force('link', d3.forceLink(links).id(d => d.id))
-    .force('charge', d3.forceManyBody())
-    .force('center', d3.forceCenter(width / 2, height / 2));
-  
-  // Render nodes with color based on status
-  svg.selectAll('circle')
-    .data(nodes)
-    .enter().append('circle')
-    .attr('r', 20)
-    .attr('fill', d => d.status === 'online' ? 'green' : 'red');
-  
-  // Render links with thickness based on quality
-  svg.selectAll('line')
-    .data(links)
-    .enter().append('line')
-    .attr('stroke-width', d => d.quality / 10)
-    .attr('stroke', d => d.quality > 80 ? 'green' : d.quality > 50 ? 'orange' : 'red');
-}
-```
-
-### Real-Time Event Monitoring
-
-```javascript
-client.subscribe('alteriom/mesh/+/events');
-
-client.on('message', (topic, message) => {
-  const event = JSON.parse(message.toString());
-  
-  if (event.event === 'mesh_event') {
-    switch (event.event_type) {
-      case 'node_leave':
-        showNotification('⚠️ Node Offline', 
-          `Node ${event.affected_nodes[0]} disconnected`);
-        updateTopology(); // Refresh graph
-        break;
-      
-      case 'node_join':
-        showNotification('✅ New Node', 
-          `Node ${event.affected_nodes[0]} joined mesh`);
-        updateTopology();
-        break;
-      
-      case 'network_split':
-        showAlert('🚨 Network Partition Detected!', 
-          'Mesh has split into multiple segments');
-        break;
-    }
-  }
-});
-```
-
----
-
-## Part 10: Recommendations
-
-### For @alteriom/mqtt-schema Maintainers
-
-#### Immediate Actions (v0.5.0)
-
-1. ✅ **Approve command schemas** from proposal (ready as-is)
-2. ⚠️ **Add mesh_topology.schema.json** (critical for monitoring)
-3. ⚠️ **Add mesh_event.schema.json** (important for alerts)
-4. 📝 **Update TypeScript types** to include mesh schemas
-5. 🧪 **Add validation tests** for topology messages
-
-#### Future Considerations (v0.6.0)
-
-6. 📋 **Add mesh_diagnostics.schema.json** (advanced debugging)
-7. 📋 **Add node_discovery.schema.json** (auto-configuration)
-8. ⏳ **Add batch_command.schema.json** (efficiency)
-9. ⏳ **Add scheduled_command.schema.json** (automation)
-
-### For alteriom-firmware Team
-
-#### Immediate Actions
-
-1. ✅ **Implement command bridge** (already done!)
-2. ⚠️ **Add topology reporter** in gateway
-   - Export `mesh.getNodeList()` as topology message
-   - Publish full topology every 60 seconds
-   - Publish incremental updates on node join/leave
-3. ⚠️ **Add mesh event publisher**
-   - Hook into painlessMesh callbacks
-   - Publish `mesh_event` on topology changes
-4. 🧪 **Test with web dashboard** (D3.js visualization)
-
-#### Future Work
-
-5. 📋 **Add `get_topology` command handler**
-6. 📋 **Implement mesh diagnostics command**
-7. 📋 **Add connection quality tracking** (RSSI, latency)
-
----
-
-## Conclusion
-
-### Summary
-
-The MQTT command schema proposal is **excellent and ready for implementation**. However, to provide a **complete mesh management solution**, we strongly recommend adding **mesh topology and event schemas** in the same release (v0.5.0).
-
-### Final Verdict
-
-| Component | Status | Action |
-|-----------|--------|--------|
-| Command Schema | ✅ **APPROVED** | Implement as proposed |
-| Response Schema | ✅ **APPROVED** | Implement as proposed |
-| Topology Schema | ⚠️ **MISSING** | **Add to v0.5.0** |
-| Event Schema | ⚠️ **MISSING** | **Add to v0.5.0** |
-
-### Proposed v0.5.0 Release Scope
-
-**Include:**
-1. command.schema.json ✅
-2. command_response.schema.json ✅
-3. mesh_topology.schema.json ⚠️ **NEW**
-4. mesh_event.schema.json ⚠️ **NEW**
-
-**Benefits:**
-- Complete bidirectional control (commands + responses)
-- Complete mesh visibility (topology + events)
-- Production-ready monitoring solution
-- Enables web dashboard visualization
-- Supports automated alerting
-
----
-
-**Reviewed By:** Alteriom Development Team  
-**Date:** October 12, 2025  
-**Status:** ✅ APPROVED WITH ADDITIONS RECOMMENDED  
-**Next Step:** Submit mesh schemas to @alteriom/mqtt-schema maintainers
-