@alteriom/painlessmesh 1.6.1 → 1.7.3

package/docs/improvements/README.md

@@ -1,69 +1,212 @@
-# painlessMesh Library Improvements
-
-This document outlines the comprehensive improvements made to the painlessMesh library to enhance performance, security, and maintainability.
-
-## Overview
-
-The improvements focus on four key areas:
-1. **Performance Optimization** - Memory management and processing efficiency
-2. **Security & Robustness** - Input validation and attack prevention  
-3. **Monitoring & Diagnostics** - Performance metrics and health monitoring
-4. **Code Quality** - Bug fixes and maintainability improvements
-
-## New Features
-
-### 1. Input Validation & Security (`validation.hpp`)
-
-Comprehensive security framework to protect against malicious or malformed messages.
-
-- **Message Validation**: JSON schema validation, field type checking, size limits
-- **Rate Limiting**: Per-node message rate limiting to prevent spam
-- **Secure Random**: Hardware-based random number generation
-- **Node ID Validation**: Verify node IDs are within valid ranges
-
-### 2. Performance Metrics & Monitoring (`metrics.hpp`)
-
-Advanced monitoring capabilities for performance optimization and diagnostics.
-
-- **Message Statistics**: Throughput, latency, error tracking, loss rate calculation
-- **Memory Monitoring**: Heap tracking, peak usage, critical alerts
-- **Network Topology**: Connection stability, node count tracking, hop analysis  
-- **JSON Reports**: Detailed status reports for integration with monitoring systems
-
-### 3. Memory Management Optimization (`memory.hpp`)
-
-Efficient memory management to reduce fragmentation and improve performance.
-
-- **Object Pooling**: Reuse objects to minimize allocation overhead
-- **String Buffers**: Pre-allocated buffers to avoid frequent reallocations
-- **Memory Statistics**: Track allocations and detect leaks
-
-### 4. Protocol Improvements
-
-Fixed critical issues and enhanced performance of core protocol handling.
-
-- **Issue #521 Resolution**: Fixed crashes in protocol::Variant copy operations
-- **Move Semantics**: Efficient move constructors and assignment operators
-- **Buffer Optimization**: Enhanced zero-copy operations in buffer handling
-
-## Performance Impact
-
-- **Memory Usage**: 10-20% reduction in memory fragmentation
-- **Message Processing**: 5-15% faster validation and processing
-- **Network Efficiency**: Reduced retransmissions due to better error handling
-- **CPU Usage**: More efficient algorithms reduce processing overhead
-
-## Testing & Quality
-
-- **100% Test Pass Rate**: All existing and new tests pass
-- **New Test Suites**: Comprehensive tests for validation and metrics
-- **Static Analysis**: Code passes all static analysis checks
-- **Memory Testing**: No memory leaks detected
-
-## Examples
-
-See `examples/alteriom/improved_sensor_node.ino` for a complete demonstration of the new features.
-
----
-
-For detailed API documentation and usage examples, see the individual header files.
+# painlessMesh Improvements Documentation
+
+This directory contains documentation for improvements made to painlessMesh, including completed features (Phases 1-2) and proposed enhancements (Phase 3+).
+
+---
+
+## Overview
+
+The improvements focus on four key areas:
+
+1. **Performance Optimization** - Memory management and processing efficiency
+2. **Security & Robustness** - Input validation and attack prevention  
+3. **Monitoring & Diagnostics** - Performance metrics and health monitoring
+4. **OTA & Status Enhancements** - Advanced firmware distribution and monitoring
+
+---
+
+## Documentation Structure
+
+### 📋 Current State
+
+**[OTA_STATUS_ENHANCEMENTS.md](OTA_STATUS_ENHANCEMENTS.md)** - Complete reference guide
+- ✅ **Phase 1 (v1.6.x):** Compressed OTA + Enhanced Status Package
+- ✅ **Phase 2 (v1.7.0):** Broadcast OTA + MQTT Status Bridge
+- 📋 **Phase 3 (Future):** Progressive rollout, P2P distribution, telemetry streams
+- Decision matrices, architecture diagrams, performance expectations
+- Quick reference for choosing implementation options
+
+### 🔧 Technical Details
+
+**[IMPLEMENTATION_HISTORY.md](IMPLEMENTATION_HISTORY.md)** - Implementation details for Phases 1-2
+- Technical specifications and code changes
+- Performance analysis and benchmarks
+- Testing documentation (80 assertions passing)
+- Files modified and API changes
+- Memory impact and scalability analysis
+
+### 🚀 Future Roadmap
+
+**[FUTURE_PROPOSALS.md](FUTURE_PROPOSALS.md)** - Proposed Phase 3+ features
+- Progressive Rollout OTA (Option 1B) - Zero-downtime updates
+- Peer-to-Peer Distribution (Option 1C) - Viral propagation for 100+ nodes
+- MQTT-Integrated OTA (Option 1D) - Cloud-based management
+- Mesh Status Service (Option 2B) - RESTful API for status queries
+- Telemetry Stream (Option 2C) - Real-time monitoring with delta encoding
+- Health Dashboard (Option 2D) - Web-based management interface
+
+---
+
+## Completed Features
+
+### Core Library Improvements
+
+**1. Input Validation & Security (`validation.hpp`)**
+- JSON schema validation and field type checking
+- Per-node rate limiting to prevent spam
+- Hardware-based secure random number generation
+- Node ID validation
+
+**2. Performance Metrics & Monitoring (`metrics.hpp`)**
+- Message statistics (throughput, latency, error tracking)
+- Memory monitoring (heap tracking, peak usage, alerts)
+- Network topology (connection stability, hop analysis)
+- JSON reports for integration with monitoring systems
+
+**3. Memory Management Optimization (`memory.hpp`)**
+- Object pooling to minimize allocation overhead
+- Pre-allocated string buffers
+- Memory statistics and leak detection
+
+**4. Protocol Improvements**
+- Issue #521 resolution (protocol::Variant copy operations)
+- Move semantics for efficient operations
+- Enhanced zero-copy buffer operations
+
+### OTA & Status Features (Phases 1-2)
+
+**Phase 1 (v1.6.x):**
+- ✅ Compressed OTA infrastructure (40-60% bandwidth reduction)
+- ✅ Enhanced StatusPackage (18 comprehensive fields)
+
+**Phase 2 (v1.7.0):**
+- ✅ Broadcast OTA (98% traffic reduction for 50-node mesh)
+- ✅ MQTT Status Bridge (Grafana/InfluxDB integration)
+
+---
+
+## Performance Impact
+
+**Core Improvements:**
+- Memory usage: 10-20% reduction in fragmentation
+- Message processing: 5-15% faster validation
+- Network efficiency: Reduced retransmissions
+- CPU usage: More efficient algorithms
+
+**OTA Improvements (Phase 1-2):**
+- Update speed: 75% faster (Phase 2 broadcast mode)
+- Network bandwidth: 50% reduction (Phase 1 compression) + 98% reduction (Phase 2 broadcast)
+- Scalability: Proven up to 50-100 nodes
+- Memory overhead: +7-13KB total
+
+---
+
+## Testing & Quality
+
+- ✅ **100% Test Pass Rate**: All existing and new tests pass
+- ✅ **80 Assertions**: Comprehensive Phase 1-2 test coverage
+- ✅ **No Regressions**: Backward compatibility maintained
+- ✅ **Static Analysis**: Code passes all checks
+- ✅ **Memory Testing**: No leaks detected
+
+---
+
+## Quick Start
+
+### "How do I use the new OTA features?"
+
+**Phase 1-2 are available now in v1.7.0:**
+
+```cpp
+// Enable compressed + broadcast OTA (both Phase 1 and 2 features)
+mesh.offerOTA("sensor", "ESP32", md5, parts, false, true, true);
+//                                            ^^^^^ ^^^^ ^^^^
+//                                            forced bcast compress
+```
+
+**See:** [OTA_STATUS_ENHANCEMENTS.md](OTA_STATUS_ENHANCEMENTS.md) for decision guide
+
+### "How do I use the new status monitoring?"
+
+**Enhanced StatusPackage (Phase 1):**
+
+```cpp
+#include "examples/alteriom/alteriom_sensor_package.hpp"
+
+alteriom::EnhancedStatusPackage status;
+status.uptime = millis() / 1000;
+status.freeMemory = ESP.getFreeHeap() / 1024;
+status.nodeCount = mesh.getNodeList().size();
+mesh.sendBroadcast(status.toJsonString());
+```
+
+**MQTT Bridge (Phase 2):**
+
+```cpp
+#include "examples/bridge/mqtt_status_bridge.hpp"
+
+MqttStatusBridge bridge(mesh, mqttClient);
+bridge.setPublishInterval(30000);  // 30 seconds
+bridge.begin();
+```
+
+**See:** [OTA_STATUS_ENHANCEMENTS.md](OTA_STATUS_ENHANCEMENTS.md) for detailed usage
+
+---
+
+## Examples
+
+**Core Improvements:**
+- `examples/alteriom/improved_sensor_node.ino` - Demonstrates validation and metrics
+
+**Phase 1-2 Features:**
+- `examples/alteriom/phase1_features.ino` - Compressed OTA + Enhanced Status
+- `examples/alteriom/phase2_features.ino` - Broadcast OTA + MQTT Bridge
+- `examples/bridge/mqtt_bridge_example.ino` - MQTT integration example
+- `examples/otaSender/otaSender.ino` - OTA sender implementation
+- `examples/otaReceiver/otaReceiver.ino` - OTA receiver implementation
+
+---
+
+## Related Documentation
+
+### User Documentation
+- [Feature History](../releases/FEATURE_HISTORY.md) - User-facing docs, migration guides, usage patterns
+- [Phase 1 Guide](../PHASE1_GUIDE.md) - Complete Phase 1 usage guide (if exists)
+- [Phase 2 Guide](../PHASE2_GUIDE.md) - Complete Phase 2 usage guide (if exists)
+
+### API Documentation
+- [Core API Reference](../api/core-api.md) - API documentation
+- [Metrics API](../../src/painlessmesh/metrics.hpp) - Performance metrics
+- [Validation API](../../src/painlessmesh/validation.hpp) - Input validation
+- [Alteriom Packages](../../examples/alteriom/alteriom_sensor_package.hpp) - Package definitions
+
+### Architecture
+- [Mesh Architecture](../architecture/mesh-architecture.md) - Core architecture
+- [Plugin System](../architecture/plugin-system.md) - Plugin architecture
+
+---
+
+## Contributing
+
+Interested in implementing Phase 3 features or improving existing ones?
+
+1. Review [FUTURE_PROPOSALS.md](FUTURE_PROPOSALS.md) for detailed specifications
+2. Check [GitHub Issues](https://github.com/Alteriom/painlessMesh/issues) for discussions
+3. Read [Contributing Guide](../development/contributing.md) for workflow
+4. Open an issue to discuss your implementation plan
+5. Submit a pull request with implementation and tests
+
+---
+
+## Questions & Support
+
+- **GitHub Issues:** <https://github.com/Alteriom/painlessMesh/issues>
+- **Discussions:** <https://github.com/Alteriom/painlessMesh/discussions>
+- **Documentation:** <https://alteriom.github.io/painlessMesh/>
+
+---
+
+**Last Updated:** October 2025  
+**Current Version:** v1.7.0  
+**Status:** Phases 1-2 Complete ✅ | Phase 3 Proposed 📋

package/docs/platformio-publishing.md

@@ -0,0 +1,255 @@
+# PlatformIO Library Publishing Guide
+
+This guide explains how to publish the AlteriomPainlessMesh library to the PlatformIO Library Registry.
+
+## Prerequisites
+
+### 1. PlatformIO Account Setup
+
+1. Visit [https://platformio.org/](https://platformio.org/) and create an account
+2. Verify your email address
+3. Log in to your PlatformIO account
+
+### 2. Authentication Token
+
+1. Go to [https://platformio.org/account/token](https://platformio.org/account/token)
+2. Generate a new Personal Access Token
+3. Copy the token and store it securely
+4. Set up the token in your local environment:
+
+   ```powershell
+   # Set environment variable (Windows PowerShell)
+   $env:PLATFORMIO_AUTH_TOKEN="YOUR_TOKEN_HERE"
+   
+   # Or for Command Prompt
+   set PLATFORMIO_AUTH_TOKEN=YOUR_TOKEN_HERE
+   
+   # Or use login command
+   pio account login
+   ```
+
+## Library Configuration
+4. Set up the token in your local environment:
+   ```powershell
+   pio account token --set YOUR_TOKEN_HERE
+   ```
+
+## Library Configuration
+
+### Required Files
+Ensure these files are properly configured:
+
+#### 1. `library.json` (Primary PlatformIO Configuration)
+```json
+{
+    "name": "AlteriomPainlessMesh",
+    "keywords": "ethernet, m2m, iot, mesh, alteriom, sensor, esp32, esp8266, json, time-sync, wireless, communication",
+    "description": "painlessMesh library with Alteriom extensions for sensor networks",
+    "repository": {
+        "type": "git", 
+        "url": "https://github.com/Alteriom/painlessMesh"
+    },
+    "version": "1.6.1",
+    "frameworks": ["arduino"],
+    "platforms": ["espressif8266", "espressif32"],
+    "dependencies": [...],
+    "authors": [...],
+    "license": "LGPL-3.0",
+    "homepage": "https://github.com/Alteriom/painlessMesh",
+    "headers": "painlessMesh.h",
+    "examples": ["examples/basic/basic.ino", "examples/alteriom/alteriom_sensor_node.ino"],
+    "export": {
+        "include": "src"
+    }
+}
+```
+
+#### 2. `library.properties` (Arduino Library Manager)
+Should remain compatible for dual publishing:
+```properties
+name=AlteriomPainlessMesh
+version=1.6.1
+author=Coopdis,Scotty Franzyshen,Edwin van Leeuwen,Germán Martín,Maximilian Schwarz,Doanh Doanh,Alteriom
+maintainer=Alteriom
+sentence=A painless way to setup a mesh with ESP8266 and ESP32 devices with Alteriom extensions
+paragraph=...
+category=Communication
+url=https://github.com/Alteriom/painlessMesh
+architectures=esp8266,esp32
+includes=AlteriomPainlessMesh.h
+depends=ArduinoJson, TaskScheduler
+```
+
+## Publishing Process
+
+### Method 1: Git Tag Publishing (Recommended)
+
+This method automatically publishes when you create a Git tag:
+
+1. **Ensure all files are committed and pushed:**
+   ```powershell
+   git add .
+   git commit -m "Prepare v1.6.1 for PlatformIO Library Registry"
+   git push origin main
+   ```
+
+2. **Create and push a Git tag:**
+   ```powershell
+   git tag v1.6.1
+   git push origin v1.6.1
+   ```
+
+3. **PlatformIO will automatically detect the new tag and import the library**
+   - Monitor at: [https://platformio.org/lib/show/LIBRARY_ID/AlteriomPainlessMesh](https://platformio.org/lib)
+   - It may take 5-15 minutes for the library to appear
+
+### Method 2: Manual Package Upload
+
+If automatic detection doesn't work:
+
+1. **Create a tarball of your library:**
+   ```powershell
+   # Create archive excluding unnecessary files
+   tar --exclude='.git' --exclude='test' --exclude='bin' --exclude='.vscode' --exclude='node_modules' -czf AlteriomPainlessMesh-1.6.1.tar.gz .
+   ```
+
+2. **Submit via PlatformIO Library Registry:**
+   - Visit [https://platformio.org/lib/register](https://platformio.org/lib/register)
+   - Upload the created tarball
+   - Fill in any additional metadata
+
+### Method 3: Using PlatformIO CLI
+
+```powershell
+# Ensure you're authenticated
+pio account token --set YOUR_TOKEN_HERE
+
+# Publish the library
+pio pkg publish .
+```
+
+## Verification
+
+### 1. Check Library Status
+```powershell
+# Search for your published library
+pio pkg search "AlteriomPainlessMesh"
+
+# View detailed information
+pio pkg show alteriom/AlteriomPainlessMesh
+```
+
+### 2. Test Installation
+Create a test project to verify the library can be installed:
+```powershell
+mkdir test_project
+cd test_project
+pio project init --board esp32dev
+
+# Add to platformio.ini:
+# lib_deps = alteriom/AlteriomPainlessMesh@^1.6.1
+
+pio pkg install
+```
+
+## Updating the Library
+
+For future releases:
+
+1. **Update version numbers:**
+   - `library.json` → `"version": "1.6.2"`
+   - `library.properties` → `version=1.6.2`
+   - `package.json` → `"version": "1.6.2"`
+
+2. **Commit changes:**
+   ```powershell
+   git add .
+   git commit -m "Bump version to 1.6.2"
+   git push origin main
+   ```
+
+3. **Create new tag:**
+   ```powershell
+   git tag v1.6.2
+   git push origin v1.6.2
+   ```
+
+4. **Verify update appears in registry**
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Library name conflicts:**
+   - Use a unique name like "AlteriomPainlessMesh" instead of "painlessMesh"
+   - Check existing libraries: `pio pkg search "painless"`
+
+2. **Authentication errors:**
+   - Verify token: `pio account show`
+   - Regenerate token if needed
+
+3. **Dependency resolution errors:**
+   - Ensure all dependencies exist in PlatformIO Registry
+   - Check version constraints (use `^` for flexible versions)
+
+4. **Git repository requirements:**
+   - Repository must be publicly accessible
+   - Tags must follow semantic versioning (v1.6.1)
+   - library.json must be in repository root
+
+### Useful Commands
+
+```powershell
+# Check authentication status
+pio account show
+
+# List all your published packages
+pio pkg search --owner="YOUR_USERNAME"
+
+# View package statistics
+pio pkg stats
+
+# Update package metadata
+pio pkg update
+```
+
+## Best Practices
+
+1. **Semantic Versioning:**
+   - Use format: MAJOR.MINOR.PATCH (e.g., 1.6.1)
+   - Increment MAJOR for breaking changes
+   - Increment MINOR for new features
+   - Increment PATCH for bug fixes
+
+2. **Documentation:**
+   - Include comprehensive README.md
+   - Provide working examples in examples/ directory
+   - Document all public APIs
+
+3. **Testing:**
+   - Test library installation in clean environments
+   - Verify examples compile successfully
+   - Test on both ESP8266 and ESP32 platforms
+
+4. **Dependency Management:**
+   - Specify minimum required versions
+   - Use version ranges (^1.6.0) for flexibility
+   - Platform-specific dependencies when needed
+
+## PlatformIO Library Registry URLs
+
+- **Library Registry:** [https://platformio.org/lib](https://platformio.org/lib)
+- **Account Management:** [https://platformio.org/account](https://platformio.org/account)
+- **Submit Library:** [https://platformio.org/lib/register](https://platformio.org/lib/register)
+- **Documentation:** [https://docs.platformio.org/en/latest/librarymanager/index.html](https://docs.platformio.org/en/latest/librarymanager/index.html)
+
+---
+
+**Note:** After successful publication, users will be able to install your library using:
+```ini
+[env:esp32dev]
+platform = espressif32
+board = esp32dev
+framework = arduino
+lib_deps = alteriom/AlteriomPainlessMesh@^1.6.1
+```

package/docs/platformio-setup-summary.md

@@ -0,0 +1,121 @@
+# PlatformIO Publishing Setup Summary
+
+## ✅ What's Been Implemented
+
+### 1. Dedicated PlatformIO Publishing Workflow
+- **File**: `.github/workflows/platformio-publish.yml`
+- **Triggers**: Automatic on releases, manual dispatch for testing
+- **Features**: Complete validation, authentication, and publishing pipeline
+
+### 2. Library Configuration Enhanced
+- **Updated `library.json`**: PlatformIO-specific optimizations
+- **Name**: Changed to "AlteriomPainlessMesh" for uniqueness
+- **Dependencies**: All verified available in PlatformIO Registry
+- **Metadata**: Enhanced with license, examples, export configuration
+
+### 3. Release Process Integration
+- **Main release workflow** now references PlatformIO publishing
+- **Documentation updated** in `RELEASE_GUIDE.md`
+- **Comprehensive instructions** in `docs/platformio-publishing.md`
+
+## 🚀 How It Works
+
+### Automatic Publishing
+1. **Release Created**: GitHub release triggers PlatformIO workflow
+2. **Validation**: Library.json format and dependencies checked
+3. **Authentication**: Uses `PLATFORMIO_AUTH_TOKEN` secret
+4. **Publication**: Direct publishing via PlatformIO CLI
+5. **Verification**: Registry confirmation and user notification
+
+### Manual Publishing (Alternative)
+1. Go to GitHub Actions → PlatformIO Library Publishing
+2. Click "Run workflow"
+3. Enter version number and optional force publish
+4. Workflow handles the rest
+
+## 🔧 Setup Required
+
+### One-Time Setup: PlatformIO Account
+
+1. **Create Account**: <https://platformio.org/account/register>
+2. **Generate Token**: <https://platformio.org/account/token>
+3. **Add to Secrets**: Repository Settings → Secrets → Actions
+   - Name: `PLATFORMIO_AUTH_TOKEN`
+   - Value: [your token from step 2]
+
+**Local Testing (Optional):**
+```powershell
+# Method 1: Environment variable
+$env:PLATFORMIO_AUTH_TOKEN="YOUR_TOKEN_HERE"
+pio account show  # Verify authentication
+
+# Method 2: Interactive login (if token doesn't work)
+pio account login
+
+# Test publishing
+pio pkg publish . --no-interactive
+```
+
+**Troubleshooting Authentication:**
+```powershell
+# If authentication fails:
+1. Go to https://platformio.org/account/token
+2. Copy your existing token OR generate a new one
+3. Set it as environment variable: $env:PLATFORMIO_AUTH_TOKEN="token"
+4. Test: pio account show
+```
+
+### Current Status for v1.6.1
+Since v1.6.1 is already released and published to other platforms:
+
+#### Option A: Test with Manual Workflow
+```
+1. Go to: https://github.com/Alteriom/painlessMesh/actions/workflows/platformio-publish.yml
+2. Click "Run workflow"
+3. Set version: 1.6.1
+4. Enable force_publish: true
+5. Click "Run workflow"
+```
+
+#### Option B: Wait for Next Release
+The PlatformIO workflow will automatically trigger on your next release (v1.6.2, etc.)
+
+## 📋 Future Releases
+
+For all future releases, PlatformIO publishing is now **fully automated**:
+
+```bash
+# Standard release process remains the same
+./scripts/bump-version.sh patch
+# Edit CHANGELOG.md
+git add . && git commit -m "release: v1.6.2" && git push
+```
+
+This will now automatically:
+- ✅ Create GitHub release
+- ✅ Publish to NPM 
+- ✅ **Publish to PlatformIO Registry**
+- ✅ Update GitHub Wiki
+- ✅ Prepare Arduino Library Manager package
+
+## 🔍 Verification
+
+After publishing, verify at:
+- **Registry**: <https://registry.platformio.org/libraries>
+- **Search**: Search for "AlteriomPainlessMesh"
+- **Installation**: `pio pkg install --library "alteriom/AlteriomPainlessMesh@^1.6.1"`
+
+## 📚 Documentation
+
+Complete documentation available:
+- **Publishing Guide**: `docs/platformio-publishing.md`
+- **Release Process**: `RELEASE_GUIDE.md` (updated)
+- **Workflow Details**: `.github/workflows/platformio-publish.yml`
+
+## 🎯 Next Steps
+
+1. **Add PlatformIO Token**: Set up the `PLATFORMIO_AUTH_TOKEN` secret
+2. **Test Workflow**: Run manual workflow for v1.6.1 (optional)
+3. **Next Release**: PlatformIO publishing will be automatic
+
+The PlatformIO publishing is now fully integrated into your release pipeline! 🎉