iodine 0.7.42 → 0.7.43

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a3ae02e70ea4439584f432552b08cab2cb26ee33c70b2a7263fd56f75bbf7f40
-  data.tar.gz: 8fa79e4b09b1f7d9d889038a9e091da2632759683d6410c318c1d965f9768b4d
+  metadata.gz: 7a444a825ab9cf0e6bd11fe14fb20c6a777e9282c1bea56cf5dc869087ce7a44
+  data.tar.gz: 4c698a81975a588e8a5c185d57f9ca76ae1d37200d2a258f5d38631de386e9dd
 SHA512:
-  metadata.gz: 063baf66e22412189a923b4f8bb566ab9cfd14b267460bced8a6774abcc4ae9ae028a737a9e4c4963c84c3629e71bd147fb16d24d836b47eee09940a8a30f9a3
-  data.tar.gz: b16abe85cfebb592c05e9e87be4401fbaac436791489da5aab14199c30b3cfd9bd187ef4a382e2f6c6fc4024949b8b283d0879cf41d63123ca0845327b8b54c5
+  metadata.gz: 9659c57e4d0c9f884b52dc07cf7b4c1a707f78d27bf03d058db1c2bfe2c873141a7d128c56b933098fa61a1f8fc48cce1319fea298c15bc8c606bb665b6d7d88
+  data.tar.gz: f183fb08856e81963d90d03ba0319f2633a72634beb8b37b2a5cee6c66dfa61710de5532abb5c82dd059d0c8382b9968a824f2caefeee2a53213e2bab2bac0b3

data/CHANGELOG.md

@@ -6,9 +6,15 @@ Please notice that this change log contains changes for upcoming releases as wel
 
 ## Changes:
 
+#### Change log v.0.7.43 (2020-11-03)
+
+**Fix**: Fixes an issue where the GVL state in user-spawned threads is inaccurate. This issue only occurs if spawning a new thread and calling certain Iodine methods from a user thread.
+
+**Fix**: validate that data passed by the user to `write` is a String object and print warnings / raise exceptions if t isn't. Credit to Vamsi Ambati for asking a question that exposed this issue.
+
 #### Change log v.0.7.42 (2020-08-02)
 
-Fix: Implement fix suggested by @Shelvak (Néstor Coppi) in issue #98.
+**Fix**: Implement fix suggested by @Shelvak (Néstor Coppi) in issue #98.
 
 #### Change log v.0.7.41 (2020-07-24)
 

data/README.md

@@ -576,8 +576,8 @@ Iodine is written in C and allows some compile-time customizations, such as:
 These options can be used, for example, like so:
 
 ```bash
-$ CFLAGS="-DFIO_FORCE_MALLOC=1 -DHTTP_MAX_HEADER_COUNT=64" \
-  gem install iodine
+  gem install iodine -- \
+      --with-cflags=\"-DHTTP_MAX_HEADER_LENGTH=48000 -DFIO_FORCE_MALLOC=1 -DHTTP_MAX_HEADER_COUNT=64\"
 ```
 
 More possible compile time options can be found in the [facil.io documentation](http://facil.io).

data/SPEC-PubSub-Draft.md

@@ -1,66 +1,106 @@
-# Ruby Pub/Sub API Specification Draft
+# Ruby pub/sub API Specification Draft
+
+### Draft State
+
+This draft is under discussion and will be implemented by iodine starting with the 0.8.x versions.
+
+---
 
 ## Purpose
 
-The pub/sub design is idiomatic to WebSocket and EventSource approaches as well as other reactive programming techniques.
+This document details a Rack specification extension for publish/subscribe (pub/sub) modules that can extend WebSocket / EventSource Rack servers.
 
-The purpose of this specification is to offer a recommendation for pub/sub design that will allow applications to be implementation agnostic (not care which pub/sub extension is used)\*.
+The purpose of this specification is:
 
-Simply put, applications will not have to worry about the chosen pub/sub implementation or about inter-process communication.
+1. To keep separation of concerns by avoiding inter-process-communication logic (IPC) in the application code base.
 
-This should simplify the idiomatic `subscribe` / `publish` approach to real-time data pushing.
+   This is important since IPC is often implemented using pipes / sockets, which could introduce network and IO concerns into the application code.
 
-\* The pub/sub extension could be implemented by any external library as long as the API conforms to the specification. The extension will have to manage the fact that some servers `fork` and manage inter-process communication for pub/sub propagation (or limit it's support to specific servers). Also, servers that opt to implement the pub/sub layer, could perform optimizations related to connection handling and pub/sub lifetimes.
+2. To specify a common publish/subscribe (pub/sub) API for servers and pub/sub modules, allowing applications to easily switch between conforming implementations and servers.
 
-## Pub/Sub handling
+    Since pub/sub design is idiomatic to WebSocket and EventSource approaches, as well as other reactive programming techniques, it is better if applications aren't locked in to a specific server / implementation.
 
-Conforming Pub/Sub implementations **MUST** implement the following pub/sub related methods within the WebSocket/SSE `client` object (as defined in the Rack WebSockets / EventSource specification draft):
+3. To provide common considerations and guidelines for pub/sub implementors to consider when implementing their pub/sub modules.
 
-* `subscribe(to, opt = {}) { |from, message| optional_block }` where `to` is a named *channel* and `opt` is a Hash object that *SHOULD* support the following possible keys (unsupported keys *MUST* be ignored):
+    Some concerns are common for pub/sub implementors, such as integrating third party message brokers (Redis, RabbitMQ, Cassandra)
 
-    * `:match` indicates a matching algorithm should be applied to the `to` variable (`to` is a pattern).
-    
-        Possible (suggested) values should include [`:redis`](https://github.com/antirez/redis/blob/398b2084af067ae4d669e0ce5a63d3bc89c639d3/src/util.c#L46-L167), [`:nats`](https://nats.io/documentation/faq/#wildcards) or [`:rabbitmq`](https://www.rabbitmq.com/tutorials/tutorial-five-ruby.html). Pub/Sub implementations *MAY* support none, some or all of these common pattern resolution schemes.
-    
-    * `:handler` is an alternative to the optional block. It should accept Proc like objects (objects that answer to `.call(from, msg)`).
+## Pub/Sub Instance Methods
+
+Conforming pub/sub implementations **MUST** implement the following pub/sub instance methods:
+
+* `pubsub?` **MUST** return the pub/sub API version as an integer number. Currently, set to `0` (development version).
 
-    * If an optional `block` (or `:handler`) is provided, if will be called when a publication was received. Otherwise, the message alone (**not** the channel data) **MUST** be sent directly to the WebSocket / EventSource client.
+* `subscribe(to, is_pattern = false) { |from, message| optional_block }` where:
 
-    * `:as` accepts either `:text` or `:binary` Symbol objects.
+    * `to` is a named **channel** (or **pattern**).
 
-        This option is only valid if the optional `block` is missing and the connection is a WebSocket connection. Note that SSE connections are limited to text data by design.
+        The implementation **MAY** support pattern matching for named channels (`to`). The pattern matching algorithm used, if any, **SHOULD** be documented.
 
-        This will dictate the encoding for outgoing WebSocket message when publications are directly sent to the client (as a text message or a binary blob). `:text` will be the default value for a missing `:as` option.
+        If the implementation does **NOT** support pattern matching and `is_pattern` is truthful, the implementation **MUST** raise and exception.
 
-        Servers *MAY* ignore this value if they set the message type (text/binary) based on UTF-8 validation.
+        `to` **SHOULD** be a String, but implementations **MAY** support Symbols as aliases to Strings (in which case `:my_channel` is the same `'my_channel'`).
 
-    If a subscription to `to` already exists, it should be *replaced* by the new subscription (the old subscription should be canceled / unsubscribed).
+    * `block` is optional and accepts (if provided) two arguments (`from` which is equal to `to` and `message` which contains the data's content).
 
-    When the `subscribe` method is called within a WebSocket / SSE Callback object, the subscription must be closed automatically when the connection is closed.
+        `block` (if provided) **MUST** be called when a publication was received by the named channel.
+
+        If no `block` is provided:
+
+        * If the pub/sub instance is an extended WebSocket / EventSource (SSE) `client` object (see the WebSocket / EventSource extension draft) data should be directly sent to the `client`.
+
+        * If the pub/sub isn't linked with a `client` / connection, an exception **MUST** be raised.
+
+    If a subscription to `to` already exists for the same pub/sub instance, it should be *replaced* by the new subscription (the old subscription should be canceled / unsubscribed).
+
+    If the pub/sub instance is an extended WebSocket / EventSource (SSE) `client` object (see the WebSocket / EventSource extension draft), the subscription **MUST** be closed automatically when the connection is closed (when `on_close` is called).
+
+    Otherwise, the subscription **MUST** be closed automatically when the pub/sub object goes out of scope and is garbage collected (if this ever happens).
     
     The `subscribe` method **MUST** return `nil` on a known failure (i.e., when the connection is already closed), or any truthful value on success.
 
-    A global variation for this method (allowing global subscriptions to be created) **SHOULD** be made available as `Rack::PubSub.subscribe`.
+* `unsubscribe(from, is_pattern = false)` should cancel a subscription to the `from` named channel / pattern.
+
+* `publish(to, message, engine = nil)` where:
 
-    When the `subscribe` method isn't called from within a connection, it should be considered a global (non connection related) subscription and an exception should be raised if a `block` or `:handler` isn't provided by the user.
+    * `to` is a named channel, same as detailed in `subscribe`.
 
-* `unsubscribe(from)` should cancel a subscription to the `from` named channel / pattern.
+        Implementations **MAY** support pattern based publishing. This **SHOULD** be documented as well as how patterns are detected (as opposed to named channels). Note that pattern publishing isn't supported by common backends (such as Redis) and introduces complex privacy and security concerns.
 
-* `publish(to, message, engine = nil)` (preferably supporting named arguments) where:
+    * `message` a String containing the data to be published.
 
-    * `to` a String that identifies the channel / stream / subject for the publication ("channel" is the semantic used by Redis, it is similar to "subject" or "stream" in other pub/sub systems).
+        If `message` is NOT a String, the implementation **MAY** convert the data silently to JSON. Otherwise, the implementation **MUST** raise an exception.
 
-    * `message` a String with containing the data to be published.
+        `message` encoding (binary / UTZ-8) **MAY** be altered during publication, but any change **MUST** result in valid encoding.
 
-    * `engine` routes the publish method to the specified Pub/Sub Engine (see later on). If none is specified, the default engine should be used. If `false` is specified, the message should be forwarded to all subscribed clients.
+    * `engine` routes the publish method to the specified pub/sub Engine (see later on). If none is specified, the default engine should be used. If `false` is specified, the message **MUST** be forwarded to all subscribed clients on the **same process**.
 
-    The `publish` method must return `true` if a publication was scheduled (not necessarily performed). If it's already known that the publication would fail, the method should return `false`.
+    The `publish` method **MUST** return `true` if a publication was scheduled (not necessarily performed). If it's already known that the publication would fail, the method should return `false`.
 
     An implementation **MUST** call the relevant PubSubEngine's `publish` method after performing any internal book keeping logic. If `engine` is `nil`, the default PubSubEngine should be called. If `engine` is `false`, the implementation **MUST** forward the published message to the actual clients (if any).
 
     A global alias for this method (allowing it to be accessed from outside active connections) **MAY** be defined as `Rack::PubSub.publish`.
 
-Implementations **MUST** implement the following methods in one of their public classes / modules (iodine implements these under `Iodine::PubSub`):
+## Integrating a Pub/Sub module into a WebSocket / EventSource (SSE) `client` object
+
+A conforming pub/sub module **MUST** be designed so that it can be integrated into WebSocket / EventSource (SSE) `client` objects be `include`-ing their class.
+
+This **MUST** result in a behavior where subscriptions are destroyed / canceled once the `client` object state changes to "closed" - i.e., either when the `on_close` callback is called, or the first time the method `client.open?` returns `false`.
+
+The idiomatic way to add a pub/sub module to a `client`'s class is:
+
+```ruby
+client.class.prepend MyPubSubModule unless client.pubsub?
+```
+
+The pub/sub module **MUST** expect and support this behavior.
+
+**Note**: the use of `prepend` (rather than `include`) is chosen so that it's possible to override the `client`'s instance method of `pubsub?`.
+
+## Connecting to External Backends (pub/sub Engines)
+
+It is common for scaling applications to require an external message broker backend such as Redis, RabbitMQ, etc'. The following requirements set a common interface for such "engine" implementation and integration.
+
+Pub/sub implementations **MUST** implement the following module / class methods in one of their public classes / modules (iodine implements these under `Iodine::PubSub`):
 
 * `attach(engine)` where `engine` is a `PubSubEngine` object, as described in this specification.
 
@@ -72,9 +112,9 @@ Implementations **MUST** implement the following methods in one of their public
 
 * `detach(engine)` where `engine` is a PubSubEngine object as described in this specification.
 
-    Removes an engine from the pub/sub system. The opposite of `attach`.
+    The implementation **MUST** remove the engine from the attached engine list. The opposite of `attach`.
 
-* `default = engine` sets a default pub/sub engine, where `engine` is a PubSubEngine object as described in this specification.
+* `default=(engine)` sets a default pub/sub engine, where `engine` is a PubSubEngine object as described in this specification.
 
     Implementations **MUST** forward any `publish` method calls to the default pub/sub engine, unless an `engine` is specified in arguments passes to the `publish` method.
 
@@ -84,34 +124,36 @@ Implementations **MUST** implement the following methods in one of their public
 
     Implementations **MUST** behave as if the engine was newly registered and (re)inform the engine of any existing subscriptions by calling engine's `subscribe` callback for each existing subscription.
 
-Implementations **MAY** implement pub/sub internally (in which case the `default` engine is the server itself or a server's module).
+A `PubSubEngine` instance object **MUST** implement the following methods:
 
-However, servers **MUST** support external pub/sub "engines" as described above, using PubSubEngine objects.
+* `subscribe(to, is_pattern = false)` this method informs the engine that a subscription to the specified channel / pattern exists for the calling the process. It **MUST ONLY** be called **once** for all existing and future subscriptions to that channel within the process.
 
-`PubSubEngine` objects **MUST** implement the following methods:
+    The method **MUST** return `true` if a subscription was scheduled (or performed) or `false` if the subscription is known to fail.
 
-* `subscribe(channel, match=nil)` this method performs the subscription to the specified channel.
+    This method will be called by the pub/sub implementation (for each registered engine). The engine **MAY** assume that the method would never be called directly by an application / client.
 
-    If `match` is a Symbol that the engine recognizes (i.e., `:redis`, `:nats`, etc'), the engine should behave accordingly. i.e., the value `:redis` on a Redis engine will invoke the PSUBSCRIBE Redis command.
+    This method **MUST NOT** raise an exception.
 
-    The method must return `true` if a subscription was scheduled (or performed) or `false` if the subscription is known to fail.
+* `unsubscribe(from, is_pattern = false)` this method informs an engine that there are no more subscriptions to the named channel / pattern for the calling process.
 
-    This method will be called by the server (for each registered engine). The engine may assume that the method would never be called directly by an application.
+    The method's semantics are similar to `subscribe` only is performs the opposite action.
 
-* `unsubscribe(channel, match=nil)` this method performs closes the subscription to the specified channel.
+    This method **MUST NOT** raise an exception.
 
-    The method's semantics are similar to `subscribe`.
+    This method will be called by the pub/sub implementation (for each registered engine). The engine **MAY** assume that the method would never be called directly by an application / client.
 
-    This method will be called by the server (for each registered engine). The engine may assume that the method would never be called directly by an application.
-
-* `publish(channel, message)` where both `channel` and `message` are String objects.
+* `publish(channel, message)` where both `channel` is either a Symbol or a String (both being equivalent) and `message` **MUST** be a String.
 
     This method will be called by the server when a message is published using the engine.
 
-    The engine **MUST** assume that the method might get called directly by an application.
+    This method will be called by the pub/sub implementation (for each registered engine).
+
+    The engine **MUST** assume that the method **MAY** be called directly by an application / client.
+
+In order for a PubSubEngine instance object to publish messages to all subscribed clients on a particular process, it **SHOULD** call the implementation's global `publish` method with the engine set to `false`.
 
-When a PubSubEngine object receives a published message, it *should* call:
+i.e., if the implementation's global `publish` method is in a class called `Iodine`:
 
 ```ruby
-Foo::PubSub.publish channel, message, false
+Iodine.publish channel, message, false
 ```

data/SPEC-WebSocket-Draft.md

@@ -5,62 +5,72 @@ This draft is also implemented by [the Agoo server](https://github.com/ohler55/a
 ---
 ## Purpose
 
-This document details WebSocket / EventSource connection support for Rack servers.
+This document details a Rack specification extension for WebSocket / EventSource servers.
 
-The purpose of these specifications is:
+The purpose of this specification is:
 
-1. To allow separation of concerns between the transport layer and the application, thereby allowing the application to be server agnostic.
+1. To improve application safety by phasing out the use of `hijack` and replacing it with the use of application object callbacks.
 
-    Simply put, when choosing between conforming servers, the application doesn’t need to have any knowledge about the chosen server.
+    This should make it easer for applications to accept WebSocket and EventSource (SSE) connections without exposing themselves to risks and errors related to IO / network logic (such as slow client attacks, buffer flooding, etc').
 
-2. To support “native" (server-side) WebSocket and EventSource (SSE) connections and using application side callbacks.
+2. To improve separation of concerns between servers and applications, moving the IO / network logic related to WebSocket and EventSource (SSE) back to the server.
 
-    Simply put, to make it easy for applications to accept WebSocket and EventSource (SSE) connections from WebSocket and EventSource clients (commonly browsers) while abstracting away any transport layer details.
-
-3. Allow applications to use WebSocket and EventSource (SSE) on HTTP/2 servers. Note: current `hijack` practices will break network connections when attempting to implement EventSource (SSE).
+    Simply put, when using a server that support this extension, the application / framework doesn’t need to have any knowledge about networking, transport protocols, IO streams, polling, etc'.
 
 ## Rack WebSockets / EventSource
 
-Servers that publish WebSocket and/or EventSource (SSE) support using the `env['rack.upgrade?']` value MUST follow the requirements set in this document.
+Servers that publish WebSocket and/or EventSource (SSE) support using the `env['rack.upgrade?']` value **MUST** follow the requirements set in this document.
 
 This document reserves the Rack `env` Hash keys of `rack.upgrade?` and `rack.upgrade`.
 
-A conforming server MUST set `env['rack.upgrade?']` to `:websocket` for incoming WebSocket connections and `:sse` for incoming EventSource (SSE) connections.
+A conforming server **MUST** set `env['rack.upgrade?']` to `:websocket` for incoming WebSocket connections and `:sse` for incoming EventSource (SSE) connections. 
+
+If a connection is not "upgradeable", a conforming server **SHOULD** set `env['rack.upgrade?']` to either `nil` or `false`. Setting the `env['rack.upgrade?']` to either `false` or `nil` should make it easier for applications to test for server support during a normal HTTP request.
+
+If the connection is upgradeable and a client application set a value for `env['rack.upgrade']`:
+
+* the server **MUST** use that value as a WebSocket / EventSource Callback Object unless the response status code `>= 300` (redirection / error status code).
+
+* The response `body` **MUST NOT** be sent when switching to a Callback Object.
+
+If a connection is **NOT** upgradeable and a client application set a value for `env['rack.upgrade']`:
 
-If a connection is not "upgradable", a conforming server SHOULD set `env['rack.upgrade?']` to either `nil` or `false`. 
+* The server **SHOULD** ignore the Callback Object and process the response as if it did not exist.
 
-The historical use of `upgrade.websocket?` and `upgrade.websocket` (iodine 0.4.x) will be gradually deprecated.
+* A server **MAY** use the Callback Object to allow a client to "hijack" the data stream as raw data stream. Such behavior **MUST** be documented.
 
 ### The WebSocket / EventSource Callback Object
 
 WebSocket and EventSource connection upgrade and handling is performed using a Callback Object.
 
-The Callback Object could be a any object which implements any of the following callbacks:
+The Callback Object could be a any object which implements any (of none) of the following callbacks:
 
-* `on_open(client)` WILL be called once the connection had been established.
+* `on_open(client)` **MUST** be called once the connection had been established and/or the Callback Object had been linked to the `client` object.
 
-* `on_message(client, data)` WILL be called when incoming WebSocket data is received.
+* `on_message(client, data)` **MUST** be called when incoming WebSocket data is received.
 
     This callback is ignored for EventSource connections.
 
-    `data` will be a String with an encoding of UTF-8 for text messages and `binary` encoding for non-text messages (as specified by the WebSocket Protocol).
+    `data` **MUST** be a String with an encoding of UTF-8 for text messages and `binary` encoding for non-text messages (as specified by the WebSocket Protocol).
 
     The *callback object* **MUST** assume that the `data` String will be a **recyclable buffer** and that it's content will be corrupted the moment the `on_message` callback returns.
 
-    Servers **MAY**, optionally, implement a **recyclable buffer** for the `on_message` callback. However, this is optional and it is *not* required.
+    Servers **MAY**, optionally, implement a **recyclable buffer** for the `on_message` callback. However, this is optional, is *not* required and might result in issues in cases where the client code is less than pristine.
 
-* `on_drained(client)` **MAY** be called when the the `client.write` buffer becomes empty. **If** `client.pending` returns a non-zero value, the `on_drained` callback **MUST** be called once the write buffer becomes empty.
+* `on_drained(client)` **MAY** be called when the `client.write` buffer becomes empty. **If** `client.pending` ever returns a non-zero value (see later on), the `on_drained` callback **MUST** be called once the write buffer becomes empty.
 
 * `on_shutdown(client)` **MAY** be called during the server's graceful shutdown process, _before_ the connection is closed and in addition to the `on_close` function (which is called _after_ the connection is closed.
 
-* `on_close(client)` **MUST** be called _after_ the connection was closed for whatever reason (socket errors, parsing errors, timeouts, client disconnection, `client.close` being called, etc').
+* `on_close(client)` **MUST** be called _after_ the connection was closed for whatever reason (socket errors, parsing errors, timeouts, client disconnection, `client.close` being called, etc') or the Callback Object was replaced by another Callback Object.
 
 
 The server **MUST** provide the Callback Object with a `client` object, that supports the following methods (this approach promises applications could be server agnostic):
 
-* `write(data)` will schedule the data to be sent. `data` **MUST** be a String.
+* `env` **MUST** return the Rack `env` hash related to the originating HTTP request. Some changes to the `env` hash (such as removal of the IO hijacking support) **MAY** be implemented by the server.
 
-    A call to `write` only promises that the data is scheduled to be sent. Servers are encouraged to avoid blocking and return immediately, deferring the actual `write` operation for later.
+* `write(data)` **MUST** schedule **all** the data to be sent. `data` **MUST** be a String. Servers **MAY** silently convert non-String objects to JSON if an application attempts to `write` a non-String value, otherwise servers **SHOULD** throw an exception.
+
+    A call to `write` only promises that the data is scheduled to be sent. Implementation details may differ across servers.
 
     `write` shall return `true` on success and `false` if the connection is closed.
 
@@ -70,27 +80,52 @@ The server **MUST** provide the Callback Object with a `client` object, that sup
 
     * If `data` is binary encoded it will be sent as non-text (as specified by the WebSocket Protocol).
 
-    A server **SHOULD** document whether `write` will block. It is **RECOMMENDED** that servers implement buffered IO, allowing `write` to return immediately when resources allow and block (or, possibly, disconnect) when the IO buffer is full.
+    A server **SHOULD** document its concurrency model, allowing developers to know whether `write` will block or not, whether buffered IO is implemented, etc'.
+
+    For example, evented servers are encouraged to avoid blocking and return immediately, deferring the actual `write` operation for later. However, (process/thread/fiber) per-connection based servers **MAY** choose to return only after all the data was sent. Documenting these differences will allows applications to choose the model that best fits their needs and environments.
 
-* `close` closes the connection once all the data in the outgoing queue was sent. If `close` is called while there is still data to be sent, `close` will only take effect once the data was sent.
+* `close` closes the connection once all the data scheduled using `write` was sent. If `close` is called while there is still data to be sent, `close` **SHOULD** return immediately and only take effect once the data was sent.
 
     `close` shall always return `nil`.
 
-* `open?` returns `true` if the connection isn't known to have been closed and `false` if the connection is known to be closed or marked to be closed.
+* `open?` **MUST** return `false` **if** the connection was never open, is known to be closed or marked to be closed. Otherwise `true` **MUST** be returned.
 
 * `pending` **MUST** return -1 if the connection is closed. Otherwise, `pending` **SHOULD** return the number of pending writes (messages in the `write` queue\*) that need to be processed before the next time the `on_drained` callback is called.
 
-    Servers **MAY** choose to always return the value `0` if they never call the `on_drained` callback and the connection is open.
+    Servers **MAY** choose to always return the value `0` **ONLY IF** they never call the `on_drained` callback and the connection is open.
+
+    Servers that return a positive number **MUST** call the `on_drained` callback when a call to `pending` would return the value `0`.
+
+    \*Servers that divide large messages into a number of smaller messages (implement message fragmentation) **MAY** count each fragment separately, as if the fragmentation was performed by the user and `write` was called more than once per message.
+
+* `pubsub?` **MUST** return `false` **unless** the pub/sub extension is supported.
+
+   Pub/Sub patterns are idiomatic for WebSockets and EventSource connections but their API is out of scope for this extension.
 
-    Servers that return a positive number MUST call the `on_drained` callback when a call to `pending` would return the value `0`.
+* `class` **MUST** return the client's Class, allowing it be extended with additional features (such as Pub/Sub, etc').
 
-    \*Servers that divide large messages into a number of smaller messages (implement message fragmentation) MAY count each fragment separately, as if the fragmentation was performed by the user and `write` was called more than once per message.
+    **Note**: Ruby adds this method automatically to every class, no need to do a thing.
 
-* `env` returns the Rack `env` hash related to the originating HTTP request. Some changes to the `env` hash (such as removal of the IO hijacking support) MAY be implemented by the server.
+The server **MAY** support the following (optional) methods for the `client` object:
 
-WebSocket `ping` / `pong`, timeouts and network considerations should be implemented by the server. It is **RECOMMENDED** (but not required) that the server send `ping`s to prevent connection timeouts and detect network failure.
+* `handler` if implemented, **MUST** return the callback object linked to the `client` object.
 
-Server settings **MAY** (not required) be provided to allow for customization and adaptation for different network environments or WebSocket extensions. It is **RECOMMENDED** that any settings be available as command line arguments and **not** incorporated into the application's logic.
+* `handler=` if implemented, **MUST** set a new Callback Object for `client`.
+
+    This allows applications to switch from one callback object to another (i.e., in case of credential upgrades).
+
+    Once a new Callback Object was set, the server **MUST** call the old handler's `on_close` callback and **afterwards** call the new handler's `on_open` callback.
+
+    It is **RECOMMENDED** (but not required) that this also updates the value for `env['rack.upgrade']`.
+
+* `timeout` / `timeout=` allows applications to get / set connection timeouts dynamically and separately for each connection. Servers **SHOULD** provide a global setting for the default connection timeout. It is **RECOMMENDED** (but not required) that a global / default timeout setting be available from the command line (CLI).
+
+* `protocol` if implemented, **MUST** return the same value that was originally set by `env['rack.upgrade?']`.
+
+
+WebSocket `ping` / `pong`, timeouts and network considerations **SHOULD** be implemented by the server. It is **RECOMMENDED** (but not required) that the server send `ping`s to prevent connection timeouts and to detect network failure. Clients **SHOULD** also consider sending `ping`s to detect network errors (dropped connections).
+
+Server settings **MAY** be provided to allow for customization and adaptation for different network environments or WebSocket extensions. It is **RECOMMENDED** that any settings be available as command line arguments and **not** incorporated into the application's logic.
 
 ---
 
@@ -100,62 +135,65 @@ Server settings **MAY** (not required) be provided to allow for customization an
 
 * **Server**:
 
+    When a regular HTTP request arrives (non-upgradeable), the server will set the `env['rack.upgrade?']` flag to `false`, indicating that: 1. this specific request is NOT upgradable; and 2. the server supports this specification for either WebSocket and/or EventSource connections.
+
     When a WebSocket upgrade request arrives, the server will set the `env['rack.upgrade?']` flag to `:websocket`, indicating that: 1. this specific request is upgradable; and 2. the server supports this specification for WebSocket connections.
 
-    When an EventSource request arrives, the server will set the `env['rack.upgrade?']` flag to `:sse`, indicating that: 1. this specific request is an EventSource request; and 2. the server supports this specification.
+    When an EventSource request arrives, the server will set the `env['rack.upgrade?']` flag to `:sse`, indicating that: 1. this specific request is an EventSource request; and 2. the server supports this specification for EventSource connections.
 
 * **Client**:
 
-    When a client decides to upgrade a request, they will place an appropriate Callback Object in the `env['rack.upgrade']` Hash key.
+    If a client decides to upgrade a request, they will place an appropriate Callback Object in the `env['rack.upgrade']` Hash key.
 
 * **Server**:
 
-    1. The server will review the `env` Hash *before* sending the response. If the `env['rack.upgrade']` was set, the server will perform the upgrade.
+    1. If the application's response status indicates an error or a redirection (status code `>= 300`), the server shall ignore the Callback Object and/or remove it from the `env` Hash, ignoring the rest of the steps that follow.
 
-    2. The server will send the correct response status and headers, as well as any headers present in the response object. The server will also perform any required housekeeping, such as closing the response body, if it exists.
+    2. The server will review the `env` Hash *before* sending the response. If the `env['rack.upgrade']` was set, the server will perform the upgrade.
 
-         The response status provided by the response object shall be ignored and the correct response status shall be set by the server.
+    3. The server will send the correct response status and headers, as well as any headers present in the response object. The server will also perform any required housekeeping, such as closing the response body, if it exists.
 
-         If the application's response status indicates an error or a redirection (status code >= 300), the server shall ignore the Callback Object.
+         The response status provided by the response object shall be ignored and the correct response status shall be set by the server.
 
-    3. Once the upgrade had completed, the server will call the `on_open` callback.
+    4. Once the upgrade had completed, the server will call the `on_open` callback.
 
         No other callbacks shall be called until the `on_open` callback had returned.
 
-        WebSocket messages shall be handled by the `on_message` callback in the same order in which they arrive and the `on_message` will **not** be executed concurrently for the same connection.
+        WebSocket messages shall be handled by the `on_message` callback in the same order in which they arrive and the `on_message` **SHOULD NOT** be executed concurrently for the same connection.
 
-        The `on_close` callback will **not** be called while any other callback is running (`on_open`, `on_message`, `on_drained`, etc').
+        The `on_close` callback **MUST NOT** be called while any other callback is running (`on_open`, `on_message`, `on_drained`, etc').
 
-        The `on_drained` callback might be called concurrently with the `on_message` callback, allowing data to be sent even while other data is being processed. Multi-threading considerations may apply.
+        The `on_drained` callback **MAY** be called concurrently with the `on_message` callback, allowing data to be sent even while incoming data is being processed. Multi-threading considerations apply.
 
 ## Example Usage
 
 The following is an example WebSocket echo server implemented using this specification:
 
 ```ruby
-class WSConnection
+module WSConnection
     def on_open(client)
-        puts "WebSocket connection established."
+        puts "WebSocket connection established (#{client.object_id})."
     end
     def on_message(client, data)
-        client.write data
+        client.write data # echo the data back
         puts "on_drained MUST be implemented if #{ pending } != 0."
     end
     def on_drained(client)
-        puts "Yap,on_drained is implemented."
+        puts "If this line prints out, on_drained is supported by the server."
     end
     def on_shutdown(client)
         client.write "The server is going away. Goodbye."
     end
     def on_close(client)
-        puts "WebSocket connection closed."
+        puts "WebSocket connection closed (#{client.object_id})."
     end
+    extend self
 end
 
 module App
    def self.call(env)
        if(env['rack.upgrade?'.freeze] == :websocket)
-           env['rack.upgrade'.freeze] = WSConnection.new
+           env['rack.upgrade'.freeze] = WSConnection
            return [0, {}, []]
        end
        return [200, {"Content-Length" => "12", "Content-Type" => "text/plain"}, ["Hello World!"]]
@@ -165,23 +203,22 @@ end
 run App
 ```
 
-The following example uses Push notifications for both WebSocket and SSE connections. The Pub/Sub API isn't part of this specification but it is supported by iodine:
+The following example uses Push notifications for both WebSocket and SSE connections. The Pub/Sub API is subject to a separate Pub/Sub API extension and isn't part of this specification (it is, however, supported by iodine):
 
 ```ruby
-class Chat
-    def initialize(nickname)
-        @nickname = nickname
-    end
+module Chat
     def on_open(client)
+        client.class.prepend MyPubSubModule unless client.pubsub?
         client.subscribe "chat"
-        client.publish "chat", "#{@nickname} joined the chat."
+        client.publish "chat", "#{env[:nickname]} joined the chat."
     end
     def on_message(client, data)
-        client.publish "chat", "#{@nickname}: #{data}"
+        client.publish "chat", "#{env[:nickname]}: #{data}"
     end
     def on_close(client)
-        client.publish "chat", "#{@nickname}: left the chat."
+        client.publish "chat", "#{env[:nickname]}: left the chat."
     end
+    extend self
 end
 
 module App
@@ -189,7 +226,7 @@ module App
        if(env['rack.upgrade?'.freeze])
            nickname = env['PATH_INFO'][1..-1]
            nickname = "Someone" if nickname == "".freeze
-           env['rack.upgrade'.freeze] = Chat.new(nickname)
+           env[:nickname] = nickname
            return [0, {}, []]
        end
        return [200, {"Content-Length" => "12", "Content-Type" => "text/plain"}, ["Hello World!"]]

data/examples/async_task.ru

@@ -0,0 +1,92 @@
+# This is a task scheduling WebSocket push example application.
+#
+# Benchmark HTTPP with `ab` or `wrk` (a 5 seconds benchmark with 2000 concurrent clients):
+#
+#      ab -c 2000 -t 5 -n 1000000 -k http://127.0.0.1:3000/
+#      wrk -c2000 -d5 -t12 http://localhost:3000/
+#
+# Test websocket tasks using the browser. For example:
+#      ws = new WebSocket("ws://localhost:3000/userID"); ws.onmessage = function(e) {console.log(e.data);}; ws.onclose = function(e) {console.log("closed")};
+#      ws.onopen = function(e) {ws.send(JSON.stringify({'task': 'echo', 'data': 'Hello!'}));};
+require 'iodine'
+require 'json'
+
+TASK_PUBLISHING_ENGINE = Iodine::PubSub::PROCESS
+
+# This module handles tasks and send them back to the frontend
+module TaskHandler
+  def echo msg
+    msg = Iodine::JSON.parse(msg, symbolize_names: true)
+    publish_to = msg.delete(:from)
+    Iodine.publish(publish_to, msg.to_json, TASK_PUBLISHING_ENGINE) if publish_to
+    puts "performed 'echo' task"
+  rescue => e
+    puts "JSON task message error? #{e.message} - under attack?"
+  end
+
+  def add msg
+    msg = Iodine::JSON.parse(msg, symbolize_names: true)
+    raise "addition task requires an array of numbers" unless msg[:data].is_a?(Array)
+    msg[:data] = msg[:data].inject(0){|sum,x| sum + x }
+    publish_to = msg.delete(:from)
+    Iodine.publish(publish_to, msg.to_json, TASK_PUBLISHING_ENGINE) if publish_to
+    puts "performed 'add' task"
+  rescue => e
+    puts
+     "JSON task message error? #{e.message} - under attack?"
+  end
+
+  def listen2tasks
+    Iodine.subscribe(:echo) {|ch,msg| TaskHandler.echo(msg) }
+    Iodine.subscribe(:add) {|ch,msg| TaskHandler.add(msg) }
+  end
+
+  extend self
+end
+
+module WebsocketClient
+  def on_open client
+    # Pub/Sub directly to the client (or use a block to process the messages)
+    client.subscribe client.env['PATH_INFO'.freeze]
+  end
+  def on_message client, data
+    # Strings and symbol channel names are equivalent.
+    msg = Iodine::JSON.parse(data, symbolize_names: true)
+    raise "no valid task" unless ["echo".freeze, "add".freeze].include? msg[:task]
+    msg[:from] = client.env['PATH_INFO'.freeze]
+    client.publish msg[:task], msg.to_json, TASK_PUBLISHING_ENGINE
+  rescue => e
+      puts "JSON message error? #{e.message}\n\t#{data}\n\t#{msg}"
+  end
+  extend self
+end
+
+APP = Proc.new do |env|
+  if env['rack.upgrade?'.freeze] == :websocket 
+    env['rack.upgrade'.freeze] = WebsocketClient 
+    [0,{}, []] # It's possible to set cookies for the response.
+  elsif env['rack.upgrade?'.freeze] == :sse
+    puts "SSE connections can only receive data from the server, the can't write." 
+    env['rack.upgrade'.freeze] = WebsocketClient
+    [0,{}, []] # It's possible to set cookies for the response.
+  else
+    [200, {"Content-Type" => "text/plain"}, ["Send messages with WebSockets using JSON.\ni.e.: {\"task\":\"add\", \"data\":[1,2]}"]]
+  end
+end
+
+# test automatically for Redis extensions.
+if(Iodine::PubSub.default.is_a? Iodine::PubSub::Redis)
+  TASK_PUBLISHING_ENGINE = Iodine::PubSub.default
+  if(ARGV.include? "worker")
+    TaskHandler.listen2tasks
+    Iodine.workers = 1
+    Iodine.threads = 16 if Iodine.threads == 0
+    Iodine.start
+    exit(0)
+  end
+else
+  TaskHandler.listen2tasks
+end
+
+# # or in config.ru
+run APP

data/ext/iodine/iodine.c

@@ -33,6 +33,7 @@ VALUE IodineBaseModule;
 VALUE iodine_default_args;
 
 ID iodine_call_id;
+ID iodine_to_s_id;
 
 static VALUE address_sym;
 static VALUE app_sym;
@@ -1302,6 +1303,7 @@ void Init_iodine(void) {
   IodineBaseModule = rb_define_module_under(IodineModule, "Base");
   VALUE IodineCLIModule = rb_define_module_under(IodineBaseModule, "CLI");
   iodine_call_id = rb_intern2("call", 4);
+  iodine_to_s_id = rb_intern("to_s");
 
   // register core methods
   rb_define_module_function(IodineModule, "threads", iodine_threads_get, 0);

data/ext/iodine/iodine.h

@@ -53,6 +53,7 @@ extern VALUE IodineModule;
 extern VALUE IodineBaseModule;
 extern VALUE iodine_default_args;
 extern ID iodine_call_id;
+extern ID iodine_to_s_id;
 
 #define IODINE_RSTRINFO(rstr)                                                  \
   ((fio_str_info_s){.len = RSTRING_LEN(rstr), .data = RSTRING_PTR(rstr)})

data/ext/iodine/iodine_caller.c

@@ -5,7 +5,7 @@
 
 #include <fio.h>
 
-static __thread volatile uint8_t iodine_GVL_state;
+static __thread volatile uint8_t iodine_GVL_state = 1;
 
 /* *****************************************************************************
 Calling protected Ruby methods

data/ext/iodine/iodine_connection.c

@@ -159,7 +159,7 @@ Ruby Connection Methods - write, close open? pending
 ***************************************************************************** */
 
 /**
- * Writes data to the connection asynchronously.
+ * Writes data to the connection asynchronously. `data` MUST be a String.
  *
  * In effect, the `write` call does nothing, it only schedules the data to be
  * sent and marks the data as pending.
@@ -174,6 +174,16 @@ static VALUE iodine_connection_write(VALUE self, VALUE data) {
     return Qnil;
     // rb_raise(rb_eIOError, "Connection closed or invalid.");
   }
+  if (!RB_TYPE_P(data, T_STRING)) {
+    VALUE tmp = data;
+    data = IodineCaller.call(data, iodine_to_s_id);
+    if (!RB_TYPE_P(data, T_STRING))
+      Check_Type(tmp, T_STRING);
+    rb_backtrace();
+    FIO_LOG_WARNING(
+        "`Iodine::Connection#write` was called with a non-String object.");
+  }
+
   switch (c->info.type) {
   case IODINE_CONNECTION_WEBSOCKET:
     /* WebSockets*/
@@ -234,6 +244,14 @@ static VALUE iodine_connection_is_open(VALUE self) {
   }
   return Qfalse;
 }
+
+/**
+ * Always returns true, since Iodine connections support the pub/sub extension.
+ */
+static VALUE iodine_connection_is_pubsub(VALUE self) {
+  return INT2NUM(0);
+  (void)self;
+}
 /**
  * Returns the number of pending `write` operations that need to complete
  * before the next `on_drained` callback is called.
@@ -675,13 +693,6 @@ The method accepts an optional `engine` argument:
 
       publish(to, message, my_pubsub_engine)
 
-
-Alternatively, accepts the following named arguments:
-
-- `:to` - The channel to publish to (required).
-- `:message` - The message to be published (required).
-- `:engine` - If provided, the engine to use for pub/sub. Otherwise the default engine is used.
-
 */
 static VALUE iodine_pubsub_publish(int argc, VALUE *argv, VALUE self) {
   // clang-format on
@@ -903,6 +914,7 @@ void iodine_connection_init(void) {
                    0);
   rb_define_method(ConnectionKlass, "handler=", iodine_connection_handler_set,
                    1);
+  rb_define_method(ConnectionKlass, "pubsub?", iodine_connection_is_pubsub, 0);
   rb_define_method(ConnectionKlass, "subscribe", iodine_pubsub_subscribe, -1);
   rb_define_method(ConnectionKlass, "unsubscribe", iodine_pubsub_unsubscribe,
                    1);

data/ext/iodine/iodine_http.c

@@ -51,7 +51,6 @@ static VALUE hijack_func_sym;
 static ID close_method_id;
 static ID each_method_id;
 static ID attach_method_id;
-static ID iodine_to_s_method_id;
 static ID iodine_call_proc_id;
 
 static VALUE env_template_no_upgrade;
@@ -294,9 +293,9 @@ static int iodine_copy2env_task(FIOBJ o, void *env_) {
 
   } else {
     /* it's an array */
-    VALUE ary = rb_ary_new();
-    rb_hash_aset(env, hname, ary);
     size_t count = fiobj_ary_count(o);
+    VALUE ary = rb_ary_new2(count);
+    rb_hash_aset(env, hname, ary);
     for (size_t i = 0; i < count; ++i) {
       tmp = fiobj_obj2cstr(fiobj_ary_index(o, i));
       rb_ary_push(ary, rb_enc_str_new(tmp.data, tmp.len, IodineBinaryEncoding));
@@ -476,11 +475,11 @@ static int for_each_header_data(VALUE key, VALUE val, VALUE h_) {
   http_s *h = (http_s *)h_;
   // fprintf(stderr, "For_each - headers\n");
   if (TYPE(key) != T_STRING)
-    key = IodineCaller.call(key, iodine_to_s_method_id);
+    key = IodineCaller.call(key, iodine_to_s_id);
   if (TYPE(key) != T_STRING)
     return ST_CONTINUE;
   if (TYPE(val) != T_STRING) {
-    val = IodineCaller.call(val, iodine_to_s_method_id);
+    val = IodineCaller.call(val, iodine_to_s_id);
     if (TYPE(val) != T_STRING)
       return ST_STOP;
   }
@@ -1130,7 +1129,6 @@ void iodine_init_http(void) {
   close_method_id = rb_intern("close");
   each_method_id = rb_intern("each");
   attach_method_id = rb_intern("attach_fd");
-  iodine_to_s_method_id = rb_intern("to_s");
   iodine_call_proc_id = rb_intern("call");
 
   IodineUTF8Encoding = rb_enc_find("UTF-8");

data/ext/iodine/iodine_mustache.c

@@ -8,7 +8,6 @@
 #include <fio.h>
 
 static ID call_func_id;
-static ID to_s_func_id;
 static VALUE filename_id;
 static VALUE data_id;
 static VALUE template_id;
@@ -169,7 +168,7 @@ static int mustache_on_arg(mustache_section_s *section, const char *name,
     if (rb_respond_to(o, call_func_id))
       o = IodineCaller.call(o, call_func_id);
     if (!RB_TYPE_P(o, T_STRING))
-      o = IodineCaller.call(o, to_s_func_id);
+      o = IodineCaller.call(o, iodine_to_s_id);
   }
   if (!RB_TYPE_P(o, T_STRING) || !RSTRING_LEN(o))
     return 0;
@@ -220,7 +219,7 @@ static int32_t mustache_on_section_test(mustache_section_s *section,
     }
     o = IodineCaller.call2(o, call_func_id, 1, &str);
     if (!RB_TYPE_P(o, T_STRING))
-      o = rb_funcall2(o, to_s_func_id, 0, NULL);
+      o = rb_funcall2(o, iodine_to_s_id, 0, NULL);
     if (RB_TYPE_P(o, T_STRING) && RSTRING_LEN(o))
       mustache_write_text(section, RSTRING_PTR(o), RSTRING_LEN(o), 0);
     return 0;
@@ -553,7 +552,6 @@ Initialize Iodine::Mustache
 
 void iodine_init_mustache(void) {
   call_func_id = rb_intern2("call", 4);
-  to_s_func_id = rb_intern2("to_s", 4);
   filename_id = rb_id2sym(rb_intern2("filename", 8));
   data_id = rb_id2sym(rb_intern2("data", 4));
   template_id = rb_id2sym(rb_intern2("template", 8));

data/lib/iodine/version.rb

@@ -1,3 +1,3 @@
 module Iodine
-  VERSION = '0.7.42'.freeze
+  VERSION = '0.7.43'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: iodine
 version: !ruby/object:Gem::Version
-  version: 0.7.42
+  version: 0.7.43
 platform: ruby
 authors:
 - Boaz Segev
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-08-02 00:00:00.000000000 Z
+date: 2020-11-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -142,6 +142,7 @@ files:
 - bin/poc/config.ru
 - bin/poc/gemfile
 - bin/poc/www/index.html
+- examples/async_task.ru
 - examples/config.ru
 - examples/echo.ru
 - examples/hello.ru
@@ -243,7 +244,7 @@ licenses:
 metadata:
   allowed_push_host: https://rubygems.org
 post_install_message: |-
-  Thank you for installing Iodine 0.7.42.
+  Thank you for installing Iodine 0.7.43.
   Remember: if iodine supports your business, it's only fair to give value back (code contributions / donations).
 rdoc_options: []
 require_paths: