@alteriom/painlessmesh 1.9.12 → 1.9.14

package/CHANGELOG.md

@@ -19,6 +19,67 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - TBD
 
+## [1.9.14] - 2025-12-19
+
+### Fixed
+
+- **Gateway Connection Timeout During Long HTTP Requests** - Fixed `sendToInternet()` timeouts with slow APIs like CallmeBot WhatsApp
+  - **Issue**: Mesh connections closed after 10s while HTTP requests could take 30s, causing "Request timed out" errors even when messages were successfully delivered
+  - **Fix**: Bridge now disables connection timeout during HTTP request processing, ensuring ACK delivery regardless of request duration
+  - **Impact**: WhatsApp/CallmeBot users and other slow API integrations now receive proper acknowledgments
+  - **Compatibility**: Fully backward compatible, no API changes required
+  - See GATEWAY_CONNECTION_TIMEOUT_FIX.md for detailed analysis
+- **Hard Reset on ESP32-C6 - Insufficient AsyncClient Deletion Spacing** - Fixed ESP32-C6 heap corruption crashes caused by insufficient spacing between AsyncClient deletions during network disruptions
+  - **Root Cause**: The 250ms spacing between AsyncClient deletions was marginally insufficient for ESP32-C6 hardware, which uses AsyncTCP v3.3.0+ and has different timing characteristics due to RISC-V architecture and enhanced cleanup validation
+  - **Symptom**: Device crashes with "CORRUPT HEAP: Bad head at 0x40838a24. Expected 0xabba1234 got 0x4081fae4" even with deletions spaced 264ms apart (only 14ms above minimum), particularly during TCP retries, channel changes, and sendToInternet() operations
+  - **Hardware Specific**: ESP32-C6 requires more cleanup time due to:
+    - AsyncTCP v3.3.0+ with additional validation steps
+    - RISC-V architecture vs. Xtensa (different instruction timing)
+    - Enhanced heap allocator with more aggressive corruption detection
+    - Different WiFi stack and memory management implementation
+  - **Solution**: Increased `TCP_CLIENT_DELETION_SPACING_MS` from 250ms to 500ms
+    - Provides 2x safety margin for universal ESP32 compatibility
+    - Accommodates ESP32-C6, ESP32-S2/S3, ESP32-C3, ESP32-H2, and future variants
+    - Maintains backward compatibility with ESP32/ESP8266
+    - Minimal performance impact (250ms additional delay in worst-case scenarios)
+  - **Analysis**: Crash occurred at 264ms spacing, demonstrating 250ms was too close to minimum requirement for ESP32-C6; 500ms provides adequate safety margin (89% buffer vs. observed crash)
+  - **Testing**: All test suites pass (68+ assertions in TCP/connection tests, 1000+ total assertions)
+  - **Documentation**: Added ISSUE_HARD_RESET_ESP32C6_SPACING_FIX.md with detailed ESP32-C6 analysis, architecture comparison, and AsyncTCP v3.3.0+ timing requirements
+  - **Impact**: Eliminates critical heap corruption on ESP32-C6 and provides enhanced safety for all ESP32 variants
+
+## [1.9.13] - 2025-12-19
+
+### Fixed
+
+- **HTTP 203 "Permanent Response" Issue with sendToInternet()** - Fixed issue where HTTP 203 (Non-Authoritative Information) responses from APIs like Callmebot WhatsApp appeared "permanent" without automatic recovery
+  - **Root Cause**: HTTP 203 responses (indicating cached/proxied responses) were correctly identified as failures but treated as terminal - requests were immediately removed from the pending queue without retry
+  - **Symptom**: User sees repeated "❌ Failed to send WhatsApp: Ambiguous response - HTTP 203..." messages with no automatic recovery, requiring manual intervention
+  - **Solution**: Modified `handleGatewayAck()` to implement intelligent retry logic for retryable failure types:
+    - HTTP 203 (Non-Authoritative Information) - cached/proxied responses, often temporary
+    - HTTP 5xx (Server Errors) - transient server issues (500, 502, 503, 504, etc.)
+    - HTTP 429 (Too Many Requests) - rate limiting with exponential backoff
+    - HTTP 0 (Network Errors) - connection failures, timeouts
+    - Non-retryable: HTTP 4xx client errors (except 429), HTTP 3xx redirects
+  - **Retry Behavior**: Uses exponential backoff (2s, 4s, 8s, 16s...) with configurable max retries (default: 3)
+  - **Testing**: Added comprehensive test coverage (50 assertions in 7 test cases) for retry classification and behavior
+  - **Documentation**: Added ISSUE_HTTP_203_RETRY_FIX.md with detailed analysis, examples, and HTTP 203 explanation
+  - **Impact**: Automatic recovery from temporary API caching issues, eliminating the "permanent" failure problem
+  - **API Compatibility**: Fully backward compatible, no breaking changes, existing code works unchanged
+
+- **Hard Reset on Node - AsyncClient Deletion Race Condition** - Fixed ESP32/ESP8266 heap corruption crashes caused by race condition in deletion spacing logic during network disruptions
+  - **Root Cause**: The `scheduleAsyncClientDeletion()` function was updating `lastScheduledDeletionTime` at BOTH scheduling time (line 111) and execution time (line 130), creating a race condition where scheduler jitter could cause deletions to execute with insufficient spacing
+  - **Symptom**: Device crashes with "CORRUPT HEAP: Bad head at 0x40831d54. Expected 0xabba1234 got 0x4081fae4" even with 250ms spacing constant, particularly during network disruptions, TCP retries, or WiFi reconnection cycles
+  - **Race Condition Scenario**: When Task A scheduled for time T1 executes late at T1+jitter, it updates `lastScheduledDeletionTime` to T1+jitter, potentially AFTER Task B's scheduled time (which was calculated based on T1), causing Task B to execute with less than 250ms spacing
+  - **Solution**: Removed the execution-time update of `lastScheduledDeletionTime` in the task callback, relying solely on the scheduling-time update
+    - Ensures consistent, predictable spacing based on planned execution times
+    - Eliminates race condition where execution-time updates could "rewind" the timestamp
+    - Makes spacing calculation immune to scheduler jitter
+    - Guarantees minimum 250ms spacing between AsyncClient deletions in all scenarios
+  - **Why This Works**: By only updating at scheduling time, subsequent deletions are always spaced from the PREVIOUS deletion's planned time, not its actual execution time, providing conservative spacing guarantees even with scheduler jitter
+  - **Testing**: All test suites pass (1000+ assertions), including TCP retry (52), connection (3), and timing tests (7)
+  - **Documentation**: Added ISSUE_HARD_RESET_DELETION_RACE_FIX.md with detailed mathematical proof and race condition analysis
+  - **Impact**: Eliminates heap corruption crashes during network disruptions, enables stable operation through TCP retries and WiFi reconnection cycles
+
 ## [1.9.12] - 2025-12-18
 
 ### Fixed

package/README.md

@@ -4,7 +4,7 @@
 
 <div align="center">
 
-**Version 1.9.12** - Latest release with AsyncClient abort() timing fix for heap corruption prevention
+**Version 1.9.14** - Latest release with ESP32-C6 spacing fix and gateway timeout improvements
 
 [![CI/CD Pipeline](https://github.com/Alteriom/painlessMesh/actions/workflows/ci.yml/badge.svg)](https://github.com/Alteriom/painlessMesh/actions/workflows/ci.yml)
 [![Documentation](https://github.com/Alteriom/painlessMesh/actions/workflows/docs.yml/badge.svg)](https://github.com/Alteriom/painlessMesh/actions/workflows/docs.yml)

package/library.json

@@ -6,7 +6,7 @@
     "type": "git",
     "url": "https://github.com/Alteriom/painlessMesh"
   },
-  "version": "1.9.12",
+  "version": "1.9.14",
   "frameworks": [
     "arduino"
   ],

package/library.properties

@@ -1,5 +1,5 @@
 name=Alteriom PainlessMesh
-version=1.9.12
+version=1.9.14
 author=Coopdis,Scotty Franzyshen,Edwin van Leeuwen,Germán Martín,Maximilian Schwarz,Doanh Doanh,Alteriom
 maintainer=Alteriom
 sentence=A painless way to setup a mesh with ESP8266 and ESP32 devices with Alteriom extensions

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@alteriom/painlessmesh",
-  "version": "1.9.12",
+  "version": "1.9.14",
   "description": "painlessMesh is a user-friendly library for creating mesh networks with ESP8266 and ESP32 devices. This Alteriom fork includes additional packages for sensor data (SensorPackage), device commands (CommandPackage), and status monitoring (StatusPackage). It handles routing and network management automatically, so you can focus on your application. The library uses JSON-based messaging and syncs time across all nodes, making it ideal for coordinated behaviour like synchronized light displays or sensor networks reporting to a central node.",
   "keywords": [
     "arduino",

package/src/AlteriomPainlessMesh.h

@@ -29,10 +29,10 @@
 /**
  * @brief AlteriomPainlessMesh library version information
  */
-#define ALTERIOM_PAINLESS_MESH_VERSION "1.9.12"
+#define ALTERIOM_PAINLESS_MESH_VERSION "1.9.14"
 #define ALTERIOM_PAINLESS_MESH_VERSION_MAJOR 1
 #define ALTERIOM_PAINLESS_MESH_VERSION_MINOR 9
-#define ALTERIOM_PAINLESS_MESH_VERSION_PATCH 12
+#define ALTERIOM_PAINLESS_MESH_VERSION_PATCH 14
 
 /**
  * @brief Library description and usage information

package/src/arduino/wifi.hpp

@@ -1246,9 +1246,16 @@ class Mesh : public painlessmesh::Mesh<Connection> {
     Log(STARTUP,
         "initBridgeStatusBroadcast(): Setting up bridge status broadcast\n");
 
+    // CRITICAL FIX: Schedule tasks with a small delay to avoid crashes when
+    // called immediately after stop()/init cycle. The delay allows the scheduler
+    // and internal task structures to stabilize before adding new tasks.
+    // This fixes the "Load access fault" Guru Meditation error that occurred
+    // when promoting to bridge role.
+    const uint32_t INIT_DELAY_MS = 100;
+
     // Register ourselves as a bridge in the knownBridges list
     // This ensures the bridge knows about itself and reports correct status
-    this->addTask([this]() {
+    this->addTask(INIT_DELAY_MS, TASK_ONCE, [this]() {
       // Check Internet connectivity: WiFi connected AND valid IP address
       bool hasInternet = (WiFi.status() == WL_CONNECTED) &&
                          (WiFi.localIP() != IPAddress(0, 0, 0, 0));
@@ -1269,12 +1276,16 @@ class Mesh : public painlessmesh::Mesh<Connection> {
     });
 
     // Create periodic task to broadcast bridge status
-    bridgeStatusTask = this->addTask(this->bridgeStatusIntervalMs, TASK_FOREVER,
-                                     [this]() { this->sendBridgeStatus(); });
+    // Schedule with delay to avoid crashes during stop/init cycle
+    this->addTask(INIT_DELAY_MS, TASK_ONCE, [this]() {
+      bridgeStatusTask = this->addTask(this->bridgeStatusIntervalMs, TASK_FOREVER,
+                                       [this]() { this->sendBridgeStatus(); });
+    });
 
     // Send immediate broadcast so nodes can discover this bridge right away
     // This ensures bridge is discoverable before the first periodic broadcast
-    this->addTask([this]() {
+    // Use slightly larger delay to allow bridge status task to be set up first
+    this->addTask(INIT_DELAY_MS + 50, TASK_ONCE, [this]() {
       Log(STARTUP, "Sending initial bridge status broadcast\n");
       this->sendBridgeStatus();
     });
@@ -2101,13 +2112,26 @@ class Mesh : public painlessmesh::Mesh<Connection> {
 
     this->callbackList.onPackage(
         protocol::GATEWAY_DATA, [this](protocol::Variant& variant,
-                                       std::shared_ptr<Connection>, uint32_t) {
+                                       std::shared_ptr<Connection> connection, uint32_t) {
           auto pkg = variant.to<gateway::GatewayDataPackage>();
 
           Log(COMMUNICATION,
               "Gateway received Internet request: msgId=%u dest=%s\n",
               pkg.messageId, pkg.destination.c_str());
 
+          // Disable connection timeout during HTTP request processing
+          // HTTP requests can take up to 30 seconds (GATEWAY_HTTP_TIMEOUT_MS)
+          // but mesh connections timeout after 10 seconds (NODE_TIMEOUT).
+          // We disable the timeout here to prevent connection drop during
+          // long-running HTTP requests. The timeout will be automatically
+          // re-enabled when the next sync packet is received.
+          if (connection) {
+            connection->timeOutTask.disable();
+            Log(COMMUNICATION,
+                "Gateway disabled connection timeout for node %u during HTTP request\n",
+                connection->nodeId);
+          }
+
           // Check Internet connectivity
           if (WiFi.status() != WL_CONNECTED) {
             sendGatewayAck(pkg, false, 0, "Gateway not connected to Internet");

package/src/painlessMesh.h

@@ -5,8 +5,8 @@
  * @file painlessMesh.h
  * @brief Main header file for Alteriom painlessMesh library
  * 
- * @version 1.9.12
- * @date 2025-12-18
+ * @version 1.9.14
+ * @date 2025-12-19
  * 
  * painlessMesh is a user-friendly library for creating mesh networks with 
  * ESP8266 and ESP32 devices. This Alteriom fork includes additional packages 

package/src/painlessmesh/connection.hpp

@@ -26,7 +26,9 @@ static const uint32_t TCP_CLIENT_CLEANUP_DELAY_MS = 1000; // 1000ms delay before
 // When multiple AsyncClients are deleted in rapid succession, the AsyncTCP library's
 // internal cleanup routines can interfere with each other, causing heap corruption
 // This spacing ensures each deletion completes before the next one begins
-static const uint32_t TCP_CLIENT_DELETION_SPACING_MS = 250; // 250ms spacing between deletions
+// Increased from 250ms to 500ms to support ESP32-C6 and other ESP32 variants which
+// require more time for AsyncTCP internal cleanup operations
+static const uint32_t TCP_CLIENT_DELETION_SPACING_MS = 500; // 500ms spacing between deletions
 
 // Global state to track AsyncClient deletion scheduling and execution
 // This ensures deletions are spaced out even when multiple deletion requests arrive simultaneously
@@ -124,10 +126,10 @@ inline void scheduleAsyncClientDeletion(Scheduler* scheduler, AsyncClient* clien
     using namespace logger;
     Log(CONNECTION, "%s: Deferred cleanup of AsyncClient executing now\n", logPrefix);
     
-    // Update the last deletion time when the deletion actually executes
-    // This ensures subsequent deletions are spaced from the actual execution time,
-    // not just the scheduled time, preventing concurrent cleanup operations
-    lastScheduledDeletionTime = millis();
+    // Note: lastScheduledDeletionTime is updated at scheduling time (before this task runs), not here
+    // This ensures consistent spacing based on when deletions were scheduled, preventing
+    // the race condition where execution-time updates could "rewind" the timestamp
+    // and cause subsequent deletions to be scheduled too close together
     
     delete client;
   });

package/src/painlessmesh/mesh.hpp

@@ -1503,13 +1503,63 @@ class Mesh : public ntp::MeshTime, public plugin::PackageHandler<T> {
 
     PendingInternetRequest& request = it->second;
 
-    // Call user callback
-    if (request.callback) {
-      request.callback(ack.success, ack.httpStatus, ack.error);
+    // Check if this is a success response
+    if (ack.success) {
+      // Success - call callback and remove request
+      if (request.callback) {
+        request.callback(ack.success, ack.httpStatus, ack.error);
+      }
+      pendingInternetRequests.erase(it);
+      return;
+    }
+
+    // Failure response - determine if retryable
+    bool isRetryable = false;
+    
+    // HTTP 203 (Non-Authoritative Information) indicates cached/proxied response
+    // This is often temporary and retrying may succeed when cache expires
+    if (ack.httpStatus == 203) {
+      isRetryable = true;
+      Log(COMMUNICATION, "handleGatewayAck(): HTTP 203 detected, marking as retryable\n");
+    }
+    // HTTP 5xx server errors are typically transient
+    else if (ack.httpStatus >= 500 && ack.httpStatus < 600) {
+      isRetryable = true;
+      Log(COMMUNICATION, "handleGatewayAck(): HTTP 5xx server error, marking as retryable\n");
+    }
+    // HTTP 429 (Too Many Requests) should be retried with backoff
+    else if (ack.httpStatus == 429) {
+      isRetryable = true;
+      Log(COMMUNICATION, "handleGatewayAck(): HTTP 429 rate limit, marking as retryable\n");
     }
+    // Network errors (httpStatus == 0) are retryable
+    else if (ack.httpStatus == 0) {
+      isRetryable = true;
+      Log(COMMUNICATION, "handleGatewayAck(): Network error, marking as retryable\n");
+    }
+    // HTTP 4xx client errors (except 429) are NOT retryable
+    // HTTP 3xx redirects are NOT retryable (should be followed by HTTPClient)
+    // Other status codes are NOT retryable
 
-    // Remove from pending
-    pendingInternetRequests.erase(it);
+    // If retryable and have retries left, schedule retry
+    if (isRetryable && request.retryCount < request.maxRetries) {
+      Log(COMMUNICATION, "handleGatewayAck(): Scheduling retry for msgId=%u (attempt %u/%u)\n",
+          ack.messageId, request.retryCount + 1, request.maxRetries);
+      scheduleInternetRetry(ack.messageId);
+    } else {
+      // Not retryable or max retries reached - call callback and remove
+      if (request.retryCount >= request.maxRetries) {
+        Log(ERROR, "handleGatewayAck(): Max retries reached for msgId=%u\n", ack.messageId);
+      } else {
+        Log(COMMUNICATION, "handleGatewayAck(): Non-retryable failure for msgId=%u (HTTP %u)\n",
+            ack.messageId, ack.httpStatus);
+      }
+      
+      if (request.callback) {
+        request.callback(ack.success, ack.httpStatus, ack.error);
+      }
+      pendingInternetRequests.erase(it);
+    }
   }
 
   /**