@winkjs/composer 0.8.0 → 0.8.1

package/README.md

@@ -38,7 +38,7 @@ The Quick Start below builds one such pipeline end to end.
 
 ## Quick Start
 
-Requires Node.js 22 or newer.
+Requires Node.js 22 or newer. Each release is certified on the newest long-term support line of Node.js, 26 today, and tested on 22 and 24.
 
 The Quick Start flow replays a CSV feed of pump motor temperatures. It cleans each reading, flags a hot motor, confirms the heat is not a blip, and prints an alert. One command scaffolds it as a runnable project:
 
@@ -106,15 +106,15 @@ The example above replays a CSV file. A live deployment reads from an MQTT broke
 
 The numbers below come from a pure compute benchmark — every step a live message takes through an 8-node flow, from arrival to final output. **Storage and MQTT I/O are excluded** — these are compute ceilings, not full-deployment numbers.
 
-| Configuration        | Throughput              |
-|----------------------|-------------------------|
-| Raspberry Pi 5       | ~100K messages/second   |
-| MacBook Pro (M4 Max) | ~1.1M messages/second   |
-| Tracking 10K assets  | ~500K messages/second   |
+| Configuration                 | Throughput              |
+|-------------------------------|-------------------------|
+| Revolution Pi Connect 5 (CM5) | 200K+ messages/second   |
+| MacBook Pro (M4 Max)          | ~1.1M messages/second   |
+| Tracking 10K assets           | ~500K messages/second   |
 
-Both non-Pi rows were measured on a MacBook Pro (M4 Max) under Node.js 22. Every figure is single-core.
+The first row was measured on a Revolution Pi Connect 5, an industrial computer built on the Raspberry Pi Compute Module 5. The other two rows were measured on a MacBook Pro (M4 Max). All three ran under Node.js 26. Every figure is single-core.
 
-Two configurations produced these numbers. The first two rows interleave 10 asset pipelines in random order — 4.5 million messages (10 pipelines × 900 data points × 500 iterations). The third row runs the same flow with 10,000 assets — 10,000 isolated states alive at once, 9 million messages, under 140 MB of heap. Pipelines are created dynamically as each asset first appears; timing uses `process.hrtime.bigint()`. The same pipeline [runs in your browser](https://composer.winkjs.org/docs/benchmark) — browser performance varies from native Node.js due to JIT differences.
+Two configurations produced these numbers. The first two rows interleave 10 asset pipelines in random order — 4.5 million messages (10 pipelines × 900 data points × 500 iterations). The third row runs the same flow with 10,000 assets — 10,000 isolated states alive at once, 9 million messages, about 150 MB of heap. Pipelines are created dynamically as each asset first appears; timing uses `process.hrtime.bigint()`. The same pipeline [runs in your browser](https://composer.winkjs.org/docs/benchmark) — browser performance varies from native Node.js due to JIT differences.
 
 Reproduce them from a repo clone. `node benchmark/compare.js 10 500` reports the median of three rounds. `node benchmark/run-benchmark.js static 10000 1` gives the 10K-asset row.
 

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@winkjs/composer",
-    "version": "0.8.0",
+    "version": "0.8.1",
     "description": "Composable Streaming Intelligence — turn streaming data into actionable insights, from edge to cloud",
     "keywords": [
         "streaming",
@@ -61,23 +61,25 @@
         "access": "public"
     },
     "devDependencies": {
-        "c8": "^10.1.3",
+        "c8": "^12.0.0",
         "chai": "^5.2.0",
         "classic-level": "^3.0.0",
         "eslint": "^9.21.0",
         "eslint-plugin-security": "^4.0.0",
-        "mocha": "^11.1.0",
+        "mocha": "^12.0.3",
         "sinon": "^19.0.2"
     },
     "dependencies": {
         "@questdb/nodejs-client": "~4.2.0",
-        "glob": "^13.0.6",
-        "mqtt": "^5.15.1",
+        "glob": "^14.0.3",
+        "mqtt": "^5.16.1",
         "msgpackr": "^1.11.10",
         "pg": "^8.20.0"
     },
     "overrides": {
         "serialize-javascript": "^7.1.0",
-        "diff": "^8.0.3"
+        "diff": "^8.0.3",
+        "minimatch": "^11.0.3",
+        "glob": "^14.0.3"
     }
 }

package/src/core/emitter-manager/mqtt/constants.js

@@ -61,11 +61,27 @@ export const QUEUE_CRITICAL_THRESHOLD = 0.8;
  *
  * The emitter uses the client's UniqueMessageIdProvider, which never
  * reissues an id that is in use. So the unacked window must never
- * hold more messages than the id space has ids. 60,000 leaves margin
- * for ids stuck on packets that are never acknowledged.
+ * hold more messages than the id space has ids. The id space alone
+ * would allow about 60,000.
+ *
+ * A second, tighter limit sets the value. libuv 1.52 and later (the
+ * version inside Node 24 and Node 26) refuses one socket write call
+ * that carries more than 1,048,576 buffer pieces. The call fails
+ * with `write EINVAL` and the connection drops. Source:
+ * `uv__check_before_write` in libuv `src/unix/stream.c`, added by
+ * libuv PR #5014. mqtt-packet corks the socket, so every publish
+ * accepted inside one event-loop turn leaves in ONE write call. One
+ * MQTT 5 publish with this emitter's properties is 29 pieces. The
+ * accepted window is 0.9 × maxQueueSize, and the whole window must
+ * fit one call: 33,000 × 0.9 × 29 = 861,300 pieces, 82% of the
+ * limit. Even the full cap fits: 33,000 × 29 = 957,000, 91%.
+ * mqtt.js resends a stored backlog one packet per acknowledgement,
+ * but it releases the publishes queued during that resend in one
+ * turn, so the window bounds that path too. The constant keeps its
+ * original name. The id space is still the first of its two reasons.
  * @type {number}
  */
-export const MQTT_INFLIGHT_ID_LIMIT = 60000;
+export const MQTT_INFLIGHT_ID_LIMIT = 33000;
 
 // Message expiry intervals (seconds), indexed by the caller's
 // `options.type`. A dictionary indexed by an external key has no

package/src/core/emitter-manager/mqtt/emitter.js

@@ -105,9 +105,10 @@
  * - Pre-flight reject fires at `pressure >= STORAGE_PRESSURE_LIMIT` (0.9),
  *   leaving 10% headroom for in-flight drain on shutdown and a clean band
  *   between the yellow-health threshold (0.66) and exhaustion (1.0).
- * - The cap is `maxQueueSize`, clamped to 60,000: every unacknowledged
- *   QoS-1 message holds a 16-bit packet id, so one connection can never
- *   carry more (see MQTT_INFLIGHT_ID_LIMIT).
+ * - The cap is `maxQueueSize`, clamped to MQTT_INFLIGHT_ID_LIMIT
+ *   (33,000). Two limits set that ceiling: the 16-bit packet-id space,
+ *   and the libuv cap on buffer pieces per socket write. constants.js
+ *   carries both with their sources.
  * - The old byte-axis limit (`maxQueueBytes`) died with the disk store;
  *   the count cap bounds memory at cap × payload size.
  *

package/src/core/emitter-manager/mqtt/index.js

@@ -65,7 +65,9 @@
  *
  *   `brokerUrl` falls back to `MQTT_BROKER_URL`. The reconnect period
  *   gains a random share of up to 20% per client (`utils/jitter`).
- *   The cap is clamped to 60,000, the packet-id ceiling.
+ *   The cap is clamped to 33,000, the in-flight ceiling. constants.js
+ *   carries its two reasons: the packet-id space and the libuv write
+ *   limit.
  *
  * Related decisions beyond ADR-018 and ADR-021. ADR-022: this adapter
  * writes the `winkDedupId` the source dedups on. ADR-023: `publishNow`

package/src/core/emitter-manager/mqtt/resolve-config.js

@@ -213,11 +213,11 @@ const resolveGrace = function ( config ) {
 }; // resolveGrace()
 
 /**
- * Resolves the unacked cap. Clamped to the 16-bit packet-id space:
- * every unacknowledged QoS-1 message holds a packet id, so one
- * connection can never carry more than 65,535; 60,000 leaves working
- * headroom (same clamp the LevelDB store applied, kept with the same
- * warning).
+ * Resolves the unacked cap, clamped to MQTT_INFLIGHT_ID_LIMIT. Two
+ * limits set that ceiling: the 16-bit packet-id space, and the libuv
+ * cap on buffer pieces per socket write. constants.js carries both
+ * with their sources. The clamp prints the same warning the LevelDB
+ * store applied.
  *
  * @param {Object} config - emitter config
  * @returns {number} the effective cap
@@ -227,7 +227,7 @@ const resolveQueueSize = function ( config ) {
     if ( requestedQueueSize > MQTT_INFLIGHT_ID_LIMIT ) {
         logger.warn(
             `winkComposer/mqttEmitter: maxQueueSize ${requestedQueueSize} exceeds the MQTT ` +
-            `packet-id ceiling — clamped to ${MQTT_INFLIGHT_ID_LIMIT}`
+            `in-flight ceiling — clamped to ${MQTT_INFLIGHT_ID_LIMIT}`
         );
     }
     return Math.min( requestedQueueSize, MQTT_INFLIGHT_ID_LIMIT );