@encharm/cws 4.11.0 → 4.11.2

package/CHANGELOG.md

@@ -1,3 +1,9 @@
+## Released 4.11.2
+* microdeflate, output unchanged: match symbols write the bit accumulator unconditionally (their sizes vary, so the flush branch mispredicted), literals keep the predictable conditional flush; the 32 KB distance-code table is replaced by a computed code. Profiled with hardware counters on the production EPYC 9454P: 1.84 -> 1.72 µs/KB on the RPC capture (1.58 -> 1.48 on 24 KB messages), 1.13 -> 1.00 on Apple M; incompressible input unchanged. Tried and rejected with measurements: a packed 32 KB hash table (gcc emits partial-register masks on Zen), a single-branch candidate check (an unconditional random load costs more than the mispredicts it saves), `__restrict`, and smaller or larger tables.
+
+## Released 4.11.1
+* microdeflate speed-ups, output unchanged: matches extend 8 bytes per step, the bit writer stores 8 bytes at a time, length/distance codes go out with their extra bits in one write, and each hash slot carries an 8-bit tag so a stale candidate is rejected without touching the input. Measured on the RPC capture: 1.74 -> 1.20 µs/KB (1.45x), byte-identical output. Incompressible input (random bytes) now falls back to stored blocks: 3x faster and 0.999 instead of 0.948 ratio.
+
 ## Released 4.11.0
 * Prepared messages for fan-out: `new PreparedMessage(bytes)` copies a payload into native memory once; `ws.send(prepared, { prefix })` sends `prefix + payload` as one frame without copying or compressing the payload per socket. Uncompressed, the payload goes out as a second gather buffer. Compressed (shared compressor mode), the prefix is emitted as a DEFLATE stored block ahead of the payload's deflate blocks, which are built once on the first compressed send and cached on the handle; standard inflaters accept the result unchanged. Sockets with context takeover and client sockets take the regular copying path. Measured with 300 subscribers and a 3 KB payload: 0.9 -> 0.2 µs of JS-thread time per socket, and no per-socket compression at all.
 * `send()` accepts any `ArrayBufferView` (typed arrays are sent from their own offset, no `Buffer.from` copy needed).

package/CLAUDE.md

@@ -25,7 +25,7 @@ Tests bind ports 3000 (ws) and 3001 (wss, certs in `tests/certs/`). The test fil
 
 1. Download official Node header tarballs for one pinned version per supported major into `targets/` (`VER_115`=Node 20, `VER_127`=Node 22, `VER_137`=Node 24, `VER_147`=Node 26; the number is the Node ABI / `process.versions.modules`).
 2. Compile `src/*.cpp` once per ABI with `g++`/`cl` directly, with `-I src/headers/$V` for the matching Node major.
-3. Build the vendored zlib-ng (`deps/zlib-ng`, native `zng_` API) once per OS/arch: CMake into `deps/zlib-ng/build-<OS>-<arch>/` on macOS/Linux, `nmake -f win32\Makefile.msc` on Windows. The bindings are compiled with `-DCWS_ZLIB_NG` and link it statically. `src/MicroDeflate.h` is a self-contained fixed-Huffman DEFLATE encoder used for independent messages (shared mode) via `zlib::deflateIndependent`; `src/Zlib.cpp` is the only file that includes a zlib header; without the define (node-gyp fallback) it uses Node's zlib.
+3. Build the vendored zlib-ng (`deps/zlib-ng`, native `zng_` API) once per OS/arch: CMake into `deps/zlib-ng/build-<OS>-<arch>/` on macOS/Linux, `nmake -f win32\Makefile.msc` on Windows. The bindings are compiled with `-DCWS_ZLIB_NG` and link it statically. `src/MicroDeflate.h` is a self-contained fixed-Huffman DEFLATE encoder used for independent messages (shared mode) via `zlib::deflateIndependent` (greedy LZ77 over a tagged 13-bit hash table, 8-byte match extension, computed distance codes, stored-block fallback when the input does not compress; output must stay byte-identical across speed work, and speed claims need the production EPYC, not only Apple silicon: gcc/Zen and clang/M-series disagreed on every layout change in 4.11.2, only the hybrid bit-writer flush won on both); Measured and rejected on the RPC capture, so do not retry without new data: dynamic Huffman trees per block (+9.4% ratio at 2.4x the time on EPYC), two candidates per hash slot (+2.5% at 1.5x), 3-byte minimum matches (worse ratio), packed 32 KB table with 16-bit tags (gcc partial-register stalls on Zen), single-branch candidate check (unconditional random load), `__restrict`, 12/14-bit tables. libdeflate level 1 reaches 3.28 vs our 2.86 at 2.6x the time; `src/Zlib.cpp` is the only file that includes a zlib header; without the define (node-gyp fallback) it uses Node's zlib.
 4. Emit `dist/bindings/cws_<platform>_<arch>_node<ABI>.node`.
 
 ```sh

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@encharm/cws",
-  "version": "4.11.0",
+  "version": "4.11.2",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
   "description": "cWS - fast C++ WebSocket implementation for Node.js",

package/src/MicroDeflate.h

@@ -15,11 +15,15 @@
 #include <cstdint>
 #include <cstring>
 #include <cstddef>
+#ifdef _MSC_VER
+#include <intrin.h>
+#endif
 
 namespace cWS {
 namespace microdeflate {
 
-// Worst case output for `length` input bytes (all literals at 9 bits + block overhead).
+// Worst case output for `length` input bytes (all literals at 9 bits + block overhead),
+// plus slack for the 8-byte stores of the bit writer.
 inline size_t bound(size_t length) {
     return length + length / 8 + 32;
 }
@@ -27,6 +31,9 @@ inline size_t bound(size_t length) {
 class Encoder {
     static const int HASH_BITS = 13;
     uint32_t table[1 << HASH_BITS];
+    // 8 more hash bits per slot: a candidate whose tag differs cannot match, and rejecting it
+    // here saves the dependent random load into the input that dominated the literal path.
+    uint8_t tags[1 << HASH_BITS];
 
     static const uint16_t *lenBase() { static const uint16_t t[29] = {3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43,51,59,67,83,99,115,131,163,195,227,258}; return t; }
     static const uint8_t *lenExtra() { static const uint8_t t[29] = {0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,5,0}; return t; }
@@ -36,7 +43,6 @@ class Encoder {
     struct Tables {
         uint16_t litCode[288]; uint8_t litBits[288];
         uint8_t lenCode[259];
-        uint8_t distCode[32769];
         uint8_t distCodeRev[30];
         static uint32_t rev(uint32_t v, int n) { uint32_t r = 0; for (int i = 0; i < n; i++) { r = (r << 1) | (v & 1); v >>= 1; } return r; }
         Tables() {
@@ -49,21 +55,85 @@ class Encoder {
                 litCode[i] = (uint16_t) rev(code, bits); litBits[i] = (uint8_t) bits;
             }
             for (int c = 0; c < 29; c++) for (int l = lenBase()[c]; l < (c < 28 ? lenBase()[c + 1] : 259); l++) lenCode[l] = (uint8_t) c;
-            for (int c = 0; c < 30; c++) { for (int d = distBase()[c]; d < (c < 29 ? distBase()[c + 1] : 32769); d++) distCode[d] = (uint8_t) c; distCodeRev[c] = (uint8_t) rev(c, 5); }
+            for (int c = 0; c < 30; c++) distCodeRev[c] = (uint8_t) rev(c, 5);
         }
     };
     static const Tables &tables() { static const Tables t; return t; }
 
+    // 64-bit accumulator flushed 8 bytes at a time with one unaligned store (the output
+    // buffer has slack for the over-write, see bound()). Literals (8 or 9 bits) drain the
+    // accumulator conditionally: their flush pattern repeats every few symbols and predicts
+    // well, and an unconditional store per literal made incompressible input store-bound on
+    // Zen 4 (+58%). Match symbols vary in size, so their flush branch mispredicted; they
+    // store unconditionally instead (-7% on the RPC capture on Zen 4, -10% on Apple M).
     struct BitWriter {
         uint8_t *out; size_t pos = 0; uint64_t acc = 0; int n = 0;
-        inline void put(uint32_t v, int bits) { acc |= (uint64_t) v << n; n += bits; while (n >= 8) { out[pos++] = (uint8_t) acc; acc >>= 8; n -= 8; } }
-        inline void flushByte() { if (n) { out[pos++] = (uint8_t) acc; acc = 0; n = 0; } }
+        inline void put(uint32_t v, int bits) {
+            acc |= (uint64_t) v << n; n += bits;
+            memcpy(out + pos, &acc, 8); pos += n >> 3; acc >>= n & ~7; n &= 7;
+        }
+        inline void putLit(uint32_t v, int bits) {
+            acc |= (uint64_t) v << n; n += bits;
+            if (n >= 32) { memcpy(out + pos, &acc, 8); pos += n >> 3; acc >>= n & ~7; n &= 7; }
+        }
+        inline void flushByte() { while (n > 0) { out[pos++] = (uint8_t) acc; acc >>= 8; n -= 8; } acc = 0; n = 0; }
     };
 
-    static inline uint32_t hash4(const uint8_t *p) { uint32_t v; memcpy(&v, p, 4); return (v * 2654435761u) >> (32 - HASH_BITS); }
+    static inline uint32_t load32(const uint8_t *p) { uint32_t v; memcpy(&v, p, 4); return v; }
+    static inline uint64_t load64(const uint8_t *p) { uint64_t v; memcpy(&v, p, 8); return v; }
+    static inline uint32_t hash32(const uint8_t *p) { return load32(p) * 2654435761u; }
+    static inline uint32_t slot(uint32_t h) { return h >> (32 - HASH_BITS); }
+    static inline uint8_t tag(uint32_t h) { return (uint8_t) (h >> (32 - HASH_BITS - 8)); }
+    // Distance code from the highest set bit of dist-1: codes 0-3 are exact, then two codes
+    // per power of two, the lower one chosen by the bit below the msb. Replaces a 32 KB table.
+    static inline int distCode(uint32_t dist) {
+        uint32_t d = dist - 1;
+        if (d < 4) return (int) d;
+#ifdef _MSC_VER
+        unsigned long msb; _BitScanReverse(&msb, d);
+#else
+        int msb = 31 - __builtin_clz(d);
+#endif
+        return ((int) msb << 1) | (int) ((d >> (msb - 1)) & 1);
+    }
+    static inline int ctz64(uint64_t v) {
+#ifdef _MSC_VER
+        unsigned long r; _BitScanForward64(&r, v); return (int) r;
+#else
+        return __builtin_ctzll(v);
+#endif
+    }
+
+    // Length of the common prefix of a and b, at most `max`: 8 bytes per step while
+    // both sides have 8 bytes left (little-endian: the first differing byte is the
+    // lowest set byte of the xor), then byte by byte.
+    static inline size_t matchLength(const uint8_t *a, const uint8_t *b, size_t max) {
+        size_t m = 0;
+        while (m + 8 <= max) {
+            uint64_t x = load64(a + m) ^ load64(b + m);
+            if (x) return m + (ctz64(x) >> 3);
+            m += 8;
+        }
+        while (m < max && a[m] == b[m]) m++;
+        return m;
+    }
 
 public:
-    Encoder() { memset(table, 0xff, sizeof(table)); tables(); }
+    Encoder() { memset(table, 0xff, sizeof(table)); memset(tags, 0, sizeof(tags)); tables(); }
+
+    // Stored blocks: what an incompressible message costs, 5 bytes per 65535 plus the tail.
+    static size_t storeBlocks(const uint8_t *in, size_t length, uint8_t *out) {
+        size_t pos = 0;
+        for (size_t i = 0; i < length; ) {
+            size_t n = length - i > 65535 ? 65535 : length - i;
+            out[pos++] = 0; // BFINAL=0, BTYPE=00
+            out[pos++] = (uint8_t) n; out[pos++] = (uint8_t) (n >> 8);
+            out[pos++] = (uint8_t) ~n; out[pos++] = (uint8_t) (~n >> 8);
+            memcpy(out + pos, in + i, n); pos += n; i += n;
+        }
+        out[pos++] = 0; // empty stored block header, byte aligned; the 00 00 ff ff tail is implied
+        return pos;
+    }
 
     // `out` must have room for bound(length) bytes. Returns the compressed length
     // (without the 4-byte sync-flush tail).
@@ -74,30 +144,34 @@ public:
         size_t i = 0;
         const size_t limit = length >= 4 ? length - 4 : 0;
         while (i < limit) {
-            uint32_t h = hash4(in + i);
-            uint32_t cand = table[h];
-            table[h] = (uint32_t) i;
-            if (cand < i && i - cand <= 32768 && memcmp(in + cand, in + i, 4) == 0) {
+            uint32_t h = hash32(in + i), s = slot(h);
+            uint8_t g = tag(h);
+            uint32_t cand = table[s];
+            bool tagged = tags[s] == g;
+            table[s] = (uint32_t) i;
+            tags[s] = g;
+            if (tagged && cand < i && i - cand <= 32768 && load32(in + cand) == load32(in + i)) {
                 size_t maxLen = length - i; if (maxLen > 258) maxLen = 258;
-                size_t m = 4;
-                while (m < maxLen && in[cand + m] == in[i + m]) m++;
+                size_t m = 4 + matchLength(in + cand + 4, in + i + 4, maxLen - 4);
                 uint32_t dist = (uint32_t) (i - cand);
                 int lc = t.lenCode[m];
-                w.put(t.litCode[257 + lc], t.litBits[257 + lc]);
-                if (lenExtra()[lc]) w.put((uint32_t) (m - lenBase()[lc]), lenExtra()[lc]);
-                int dc = t.distCode[dist];
-                w.put(t.distCodeRev[dc], 5);
-                if (distExtra()[dc]) w.put(dist - distBase()[dc], distExtra()[dc]);
-                if (i + 1 < limit) table[hash4(in + i + 1)] = (uint32_t) (i + 1);
+                // length code + its extra bits in one put (at most 8 + 5 bits), same for distance (5 + 13)
+                w.put(t.litCode[257 + lc] | ((uint32_t) (m - lenBase()[lc]) << t.litBits[257 + lc]), t.litBits[257 + lc] + lenExtra()[lc]);
+                int dc = distCode(dist);
+                w.put(t.distCodeRev[dc] | ((dist - distBase()[dc]) << 5), 5 + distExtra()[dc]);
+                if (i + 1 < limit) { uint32_t h1 = hash32(in + i + 1); table[slot(h1)] = (uint32_t) (i + 1); tags[slot(h1)] = tag(h1); }
                 i += m;
             } else {
-                w.put(t.litCode[in[i]], t.litBits[in[i]]);
+                w.putLit(t.litCode[in[i]], t.litBits[in[i]]);
                 i++;
             }
         }
-        for (; i < length; i++) w.put(t.litCode[in[i]], t.litBits[in[i]]);
-        w.put(t.litCode[256], t.litBits[256]);             // end of block
-        w.put(0, 1); w.put(0, 2); w.flushByte();           // empty stored block: BFINAL=0, BTYPE=00, then byte-align
+        for (; i < length; i++) w.putLit(t.litCode[in[i]], t.litBits[in[i]]);
+        w.putLit(t.litCode[256], t.litBits[256]);          // end of block
+        w.putLit(0, 1); w.putLit(0, 2); w.flushByte();     // empty stored block: BFINAL=0, BTYPE=00, then byte-align
+        if (w.pos > length + 5 * (length / 65535 + 1) + 1) {
+            return storeBlocks(in, length, out); // incompressible: stored blocks are smaller
+        }
         out[w.pos++] = 0; out[w.pos++] = 0; out[w.pos++] = 0xff; out[w.pos++] = 0xff;
         return w.pos - 4;
     }