@encharm/cws 4.8.4 → 4.10.0

package/CHANGELOG.md

@@ -1,3 +1,10 @@
+## Released 4.10.0
+* microdeflate: a built-in ~150-line raw-DEFLATE encoder (fixed Huffman, greedy LZ77, no stream state, no per-message hash reset) now compresses independent messages, i.e. the shared-compressor mode. Measured on a real BSON RPC stream: same ratio as zlib-ng level 1 (2.84 vs 2.85) at ~1.7x its speed; a compressed 2 KB message costs 4.6 µs of worker CPU instead of 6.0. Output is standard DEFLATE (round-trip tested against zlib's inflater). `CWS_MICRO_DEFLATE=0` falls back to zlib-ng; `zlibBackend` reports `+ microdeflate` when active. Dedicated windows (context takeover) and inflate stay on zlib-ng.
+* Compression moved to the send worker: `send()` of a compressed message queues the raw payload and the worker deflates + frames it. Main-thread cost of a compressed 2 KB RPC message drops from ~6 µs to ~0.7 µs (Linux, per-thread measurement); wire output is byte-identical. Main-thread write paths (same-tick terminate, full worker queue, drain loop after a short write) deflate pending messages themselves first.
+
+## Released 4.9.0
+* Send worker thread: the end-of-tick gathered writes run on a dedicated thread (lock-free SPSC hand-off via the vendored `readerwriterqueue`), taking the kernel's TCP work off the JavaScript thread. Measured on a loaded EPYC: 30-65% less main-thread CPU per message, 1.7-2.2x fan-out throughput. `CWS_SEND_THREAD=0` disables; `sendThread` export reports status. A socket closed with a send in flight keeps its fd open until that send completes (prevents fd reuse races).
+
 ## Released 4.8.4
 * `npm install` no longer rebuilds the binding from source when a prebuilt one matches the platform and Node ABI. Since the node-gyp fallback was repaired in 4.8.0, every install on a machine with a compiler was silently replacing the shipped zlib-ng binding with a Node-zlib build. `CWS_FORCE_BUILD=1` forces the source build.
 

package/CLAUDE.md

@@ -25,7 +25,7 @@ Tests bind ports 3000 (ws) and 3001 (wss, certs in `tests/certs/`). The test fil
 
 1. Download official Node header tarballs for one pinned version per supported major into `targets/` (`VER_115`=Node 20, `VER_127`=Node 22, `VER_137`=Node 24, `VER_147`=Node 26; the number is the Node ABI / `process.versions.modules`).
 2. Compile `src/*.cpp` once per ABI with `g++`/`cl` directly, with `-I src/headers/$V` for the matching Node major.
-3. Build the vendored zlib-ng (`deps/zlib-ng`, native `zng_` API) once per OS/arch: CMake into `deps/zlib-ng/build-<OS>-<arch>/` on macOS/Linux, `nmake -f win32\Makefile.msc` on Windows. The bindings are compiled with `-DCWS_ZLIB_NG` and link it statically. `src/Zlib.cpp` is the only file that includes a zlib header; without the define (node-gyp fallback) it uses Node's zlib.
+3. Build the vendored zlib-ng (`deps/zlib-ng`, native `zng_` API) once per OS/arch: CMake into `deps/zlib-ng/build-<OS>-<arch>/` on macOS/Linux, `nmake -f win32\Makefile.msc` on Windows. The bindings are compiled with `-DCWS_ZLIB_NG` and link it statically. `src/MicroDeflate.h` is a self-contained fixed-Huffman DEFLATE encoder used for independent messages (shared mode) via `zlib::deflateIndependent`; `src/Zlib.cpp` is the only file that includes a zlib header; without the define (node-gyp fallback) it uses Node's zlib.
 4. Emit `dist/bindings/cws_<platform>_<arch>_node<ABI>.node`.
 
 ```sh
@@ -72,6 +72,10 @@ The client side never touches Node sockets: `native.connect(clientGroup, url, ws
 
 `Socket::sendTransformed` does not write when the socket is corkable (WebSockets only, plain TCP only) and `NodeData::corkState->enabled` is set. It frames the message into the socket's `messageQueue` and registers the socket in the per-loop `corkState->pending` list. `Socket::flushCorked` drains that list with one `sendmsg`/`WSASend` gather per socket from two libuv hooks registered in `registerCheck` (`src/Addon.h`): a `uv_prepare` (before the loop blocks in poll) and the existing `uv_check` (after poll and after Node's immediates). Anything that cannot be written falls through to the normal `UV_WRITABLE` drain loop in `ioHandler`. `closeSocket`/`onEnd`/`transfer` call `flushCorkedOnClose`, which unregisters the socket and pushes what it can to the kernel without invoking callbacks. `CWS_CORK=0` disables it at addon load (`corkEnabledFromEnv`). `HttpSocket` is never corkable because `Hub::upgrade` deletes it synchronously.
 
+### Send worker thread
+
+`SendWorker` (`src/SendWorker.cpp`) starts one `std::thread` at addon load when corking is enabled. `Socket::uncork` moves up to 512 queued frames into a `Socket::SendOp` (ownership moves with them) and hands it over through `deps/readerwriterqueue` (blocking SPSC main→worker, plain SPSC worker→main plus a `uv_async`). The worker only calls `sendmsg`/`WSASend` and fills `result`/`error`; `Socket::sendComplete` on the main thread pops what was sent, requeues the rest at the head, resubmits if more queued, arms `UV_WRITABLE` for the classic drain loop on a short write, runs send callbacks, and calls `endCb` (= `STATE::onEnd`, set in `setState`) on a hard error. While an op is in flight `write()` appends to the queue and the drain loop stays out. `closeSocket` orphans an in-flight op (`socket = nullptr`, `closeFd = true`) and lets the completion close the fd, so a reused fd number can never receive the old socket's bytes. SSL sockets never use the worker. `CWS_SEND_THREAD=0` disables. Compressed sends: `WebSocket::send` queues the raw payload as a `compressPending` message (`enqueueCompressPending`); `performSend` deflates + frames it on the worker with the socket's window (`op->deflateWindow`) or the worker's own shared compressor, then builds the iovecs. Any main-thread write path calls `materializePending()` first (`WebSocket::materialize` via `materializeCb`, using the hub's compressor). A socket closed mid-flight hands its window to the op (`destroyWindow`, `workerOwnsWindow`) so `onEnd` does not free it under the worker.
+
 ### Things that are easy to get wrong
 
 - `dist/*.js` and `dist/bindings/*.node` are committed. Rebuild TS with `npm run build-ts` and commit the output; binaries must be rebuilt on each platform when C++ changes.

package/README.md

@@ -14,6 +14,8 @@ This table is true if you run ssl directly with `cws` (`Node.js`). In case if yo
 
 | cWS Version | Node 26 | Node 24  | Node 22 | Node 20 |
 |-------------|---------|----------|---------|----------
+| 4.10.0      |    X    |    X     |    X    |   X     |
+| 4.9.0       |    X    |    X     |    X    |   X     |
 | 4.8.4       |    X    |    X     |    X    |   X     |
 | 4.8.3       |    X    |    X     |    X    |   X     |
 | 4.8.2       |    X    |    X     |    X    |   X     |
@@ -185,6 +187,8 @@ const wsServer = new WebSocket.Server({
    *   sliding window (streaming compression; far better on small messages). windowBits 9..15 and
    *   memLevel 1..9 pick its memory tier: 15/8 = ~256 KB per socket (default), 12/5 = ~32 KB, 10/3 = ~8 KB.
    *   threshold = minimum message size in bytes to compress (default 0); send(..., { compress }) overrides.
+   *   Independent messages (the shared mode) are compressed by the built-in microdeflate encoder, ~1.7x faster
+   *   than zlib-ng level 1 at the same ratio; CWS_MICRO_DEFLATE=0 falls back to zlib-ng.
    *   level 1..9 (default 2): deflate level of the per-socket compressor. The prebuilt bindings use zlib-ng
    *   (see `zlibBackend` export): level 1 is its very fast "quick" strategy (~3-4x less CPU than zlib
    *   level 1 for ~8% worse ratio), level 2 matches zlib level 1's ratio.
@@ -282,6 +286,14 @@ All `send()` calls made to a socket during one event-loop iteration are framed i
 
 Set `CWS_CORK=0` in the environment to disable it and write every message immediately (previous behaviour). Corking applies to plain TCP sockets only; TLS sockets always write immediately.
 
+### Send worker thread (performance)
+
+The gathered write itself (the kernel's TCP work, which dominates a busy socket server's CPU) runs on a dedicated worker thread, not on the JavaScript thread. The end-of-tick flush hands each socket's frames to the worker through a lock-free queue; the worker performs the send and reports back, and completions, callbacks and `bufferedAmount` are handled on the main thread as before. Per-socket ordering is preserved, `send()` followed by `close()`/`terminate()` in the same tick still delivers, and a socket that closes while its send is in flight keeps its fd open until the send has finished. Measured on a busy 96-thread EPYC: 30-65% less main-thread CPU per message and 1.7-2.2x fan-out throughput, for 10-25% more total CPU on the worker.
+
+With permessage-deflate enabled, compression of outgoing messages also runs on the worker: `send()` queues the raw payload and the worker deflates and frames it before writing, so a compressed `send()` costs the JavaScript thread the same as an uncompressed one (measured: 6 µs → 0.7 µs per 2 KB message).
+
+Set `CWS_SEND_THREAD=0` to disable it (sends and compression then happen on the main thread at the end of the tick). The `sendThread` export reports `'active'` or the reason it is not. TLS sockets always send on the main thread.
+
 ### Secure WebSocket
 You can use `wss://` with `cws` by providing `https` server to `cws` and setting `secureProtocol` on https options:
 

package/binding.gyp

@@ -13,11 +13,13 @@
                 'src/WebSocket.cpp',
                 'src/HTTPSocket.cpp',
                 'src/Socket.cpp',
-                'src/Zlib.cpp'
+                'src/Zlib.cpp',
+                'src/SendWorker.cpp'
             ],
             # Node private headers vendored per major (see src/headers/); the Makefile passes the same -I.
             'include_dirs': [
-                'src/headers/<!(node -p "parseInt(process.versions.node)")'
+                'src/headers/<!(node -p "parseInt(process.versions.node)")',
+                'deps/readerwriterqueue'
             ],
             'defines': ['HAVE_OPENSSL=1', 'NODE_WANT_INTERNALS=1'],
             'conditions': [

package/deps/readerwriterqueue/LICENSE.md

@@ -0,0 +1,28 @@
+This license applies to all the code in this repository except that written by third
+parties, namely the files in benchmarks/ext, which have their own licenses, and Jeff
+Preshing's semaphore implementation (used in the blocking queues) which has a zlib
+license (embedded in atomicops.h).
+
+Simplified BSD License:
+
+Copyright (c) 2013-2021, Cameron Desrochers  
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+- Redistributions of source code must retain the above copyright notice, this list of
+conditions and the following disclaimer.
+- Redistributions in binary form must reproduce the above copyright notice, this list of
+conditions and the following disclaimer in the documentation and/or other materials
+provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
+OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
+TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

package/deps/readerwriterqueue/README.md

@@ -0,0 +1,186 @@
+
+# A single-producer, single-consumer lock-free queue for C++
+
+This mini-repository has my very own implementation of a lock-free queue (that I designed from scratch) for C++.
+
+It only supports a two-thread use case (one consuming, and one producing). The threads can't switch roles, though
+you could use this queue completely from a single thread if you wish (but that would sort of defeat the purpose!).
+
+Note: If you need a general-purpose multi-producer, multi-consumer lock free queue, I have [one of those too][mpmc].
+
+This repository also includes a [circular-buffer SPSC queue][circular] which supports blocking on enqueue as well as dequeue.
+
+
+## Features
+
+- [Blazing fast][benchmarks]
+- Compatible with C++11 (supports moving objects instead of making copies)
+- Fully generic (templated container of any type) -- just like `std::queue`, you never need to allocate memory for elements yourself
+  (which saves you the hassle of writing a lock-free memory manager to hold the elements you're queueing)
+- Allocates memory up front, in contiguous blocks
+- Provides a `try_enqueue` method which is guaranteed never to allocate memory (the queue starts with an initial capacity)
+- Also provides an `enqueue` method which can dynamically grow the size of the queue as needed
+- Also provides `try_emplace`/`emplace` convenience methods
+- Has a blocking version with `wait_dequeue`
+- Completely "wait-free" (no compare-and-swap loop). Enqueue and dequeue are always O(1) (not counting memory allocation)
+- On x86, the memory barriers compile down to no-ops, meaning enqueue and dequeue are just a simple series of loads and stores (and branches)
+
+
+## Use
+
+Simply drop the readerwriterqueue.h (or readerwritercircularbuffer.h) and atomicops.h files into your source code and include them :-)
+A modern compiler is required (MSVC2010+, GCC 4.7+, ICC 13+, or any C++11 compliant compiler should work).
+
+Note: If you're using GCC, you really do need GCC 4.7 or above -- [4.6 has a bug][gcc46bug] that prevents the atomic fence primitives
+from working correctly.
+
+Example:
+
+```cpp
+using namespace moodycamel;
+
+ReaderWriterQueue<int> q(100);       // Reserve space for at least 100 elements up front
+
+q.enqueue(17);                       // Will allocate memory if the queue is full
+bool succeeded = q.try_enqueue(18);  // Will only succeed if the queue has an empty slot (never allocates)
+assert(succeeded);
+
+int number;
+succeeded = q.try_dequeue(number);  // Returns false if the queue was empty
+
+assert(succeeded && number == 17);
+
+// You can also peek at the front item of the queue (consumer only)
+int* front = q.peek();
+assert(*front == 18);
+succeeded = q.try_dequeue(number);
+assert(succeeded && number == 18);
+front = q.peek(); 
+assert(front == nullptr);           // Returns nullptr if the queue was empty
+```
+
+The blocking version has the exact same API, with the addition of `wait_dequeue` and
+`wait_dequeue_timed` methods:
+
+```cpp
+BlockingReaderWriterQueue<int> q;
+
+std::thread reader([&]() {
+    int item;
+#if 1
+    for (int i = 0; i != 100; ++i) {
+        // Fully-blocking:
+        q.wait_dequeue(item);
+    }
+#else
+    for (int i = 0; i != 100; ) {
+        // Blocking with timeout
+        if (q.wait_dequeue_timed(item, std::chrono::milliseconds(5)))
+            ++i;
+    }
+#endif
+});
+std::thread writer([&]() {
+    for (int i = 0; i != 100; ++i) {
+        q.enqueue(i);
+        std::this_thread::sleep_for(std::chrono::milliseconds(10));
+    }
+});
+writer.join();
+reader.join();
+
+assert(q.size_approx() == 0);
+```
+    
+Note that `wait_dequeue` will block indefinitely while the queue is empty; this
+means care must be taken to only call `wait_dequeue` if you're sure another element
+will come along eventually, or if the queue has a static lifetime. This is because
+destroying the queue while a thread is waiting on it will invoke undefined behaviour.
+
+The blocking circular buffer has a fixed number of slots, but is otherwise quite similar to
+use:
+
+```cpp
+BlockingReaderWriterCircularBuffer<int> q(1024);  // pass initial capacity
+
+q.try_enqueue(1);
+int number;
+q.try_dequeue(number);
+assert(number == 1);
+
+q.wait_enqueue(123);
+q.wait_dequeue(number);
+assert(number == 123);
+
+q.wait_dequeue_timed(number, std::chrono::milliseconds(10));
+```
+
+
+## CMake
+### Using targets in your project
+Using this project as a part of an existing CMake project is easy.
+
+In your CMakeLists.txt:
+```
+include(FetchContent)
+
+FetchContent_Declare(
+  readerwriterqueue
+  GIT_REPOSITORY    https://github.com/cameron314/readerwriterqueue
+  GIT_TAG           master
+)
+
+FetchContent_MakeAvailable(readerwriterqueue)
+
+add_library(my_target main.cpp)
+target_link_libraries(my_target PUBLIC readerwriterqueue)
+```
+
+In main.cpp:
+```cpp
+#include <readerwriterqueue.h>
+
+int main()
+{
+    moodycamel::ReaderWriterQueue<int> q(100);
+}
+```
+
+### Installing into system directories
+As an alternative to including the source files in your project directly,
+you can use CMake to install the library in your system's include directory:
+
+```
+mkdir build
+cd build
+cmake ..
+make install
+```
+
+Then, you can include it from your source code:
+```
+#include <readerwriterqueue/readerwriterqueue.h>
+```
+
+## Disclaimers
+
+The queue should only be used on platforms where aligned integer and pointer access is atomic; fortunately, that
+includes all modern processors (e.g. x86/x86-64, ARM, and PowerPC). *Not* for use with a DEC Alpha processor (which has very weak memory ordering) :-)
+
+Note that it's only been tested on x86(-64); if someone has access to other processors I'd love to run some tests on
+anything that's not x86-based.
+
+## More info
+
+See the [LICENSE.md][license] file for the license (simplified BSD).
+
+My [blog post][blog] introduces the context that led to this code, and may be of interest if you're curious
+about lock-free programming.
+
+
+[blog]: http://moodycamel.com/blog/2013/a-fast-lock-free-queue-for-c++
+[license]: LICENSE.md
+[benchmarks]: http://moodycamel.com/blog/2013/a-fast-lock-free-queue-for-c++#benchmarks
+[gcc46bug]: http://stackoverflow.com/questions/16429669/stdatomic-thread-fence-has-undefined-reference
+[mpmc]: https://github.com/cameron314/concurrentqueue
+[circular]: readerwritercircularbuffer.h