isomorfeus-iodine 0.7.44

data/Rakefile

@@ -0,0 +1,44 @@
+require "bundler/gem_tasks"
+require "rake/extensiontask"
+require_relative 'lib/iodine/version'
+
+begin
+  require 'rspec/core/rake_task'
+  RSpec::Core::RakeTask.new(:rspec)
+rescue LoadError
+end
+
+task :spec => [:compile, :rspec]
+
+task :push_packages do
+  Rake::Task['push_packages_to_rubygems'].invoke
+  Rake::Task['push_packages_to_github'].invoke
+end
+
+task :push_packages_to_rubygems do
+  system("gem push isomorfeus-iodine-#{Iodine::VERSION}.gem")
+end
+
+task :push_packages_to_github do
+  system("gem push --key github --host https://rubygems.pkg.github.com/isomorfeus isomorfeus-iodine-#{Iodine::VERSION}.gem")
+end
+
+task :push do
+  system("git push github")
+  system("git push gitlab")
+  system("git push bitbucket")
+  system("git push gitprep")
+end
+
+task :default => [:compile, :spec]
+
+Rake::ExtensionTask.new "iodine" do |ext|
+  ext.lib_dir = "lib/iodine"
+end
+
+# Rake::ExtensionTask.new "iodine_http" do |ext|
+#   ext.name = 'iodine_http'
+#   ext.lib_dir = "lib/iodine"
+#   ext.ext_dir = 'ext/iodine'
+#   ext.config_script = 'extconf-http.rb'
+# end

data/SPEC-PubSub-Draft.md

@@ -0,0 +1,159 @@
+# Ruby pub/sub API Specification Draft
+
+### Draft State
+
+This draft is under discussion and will be implemented by iodine starting with the 0.8.x versions.
+
+---
+
+## Purpose
+
+This document details a Rack specification extension for publish/subscribe (pub/sub) modules that can extend WebSocket / EventSource Rack servers.
+
+The purpose of this specification is:
+
+1. To keep separation of concerns by avoiding inter-process-communication logic (IPC) in the application code base.
+
+   This is important since IPC is often implemented using pipes / sockets, which could introduce network and IO concerns into the application code.
+
+2. To specify a common publish/subscribe (pub/sub) API for servers and pub/sub modules, allowing applications to easily switch between conforming implementations and servers.
+
+    Since pub/sub design is idiomatic to WebSocket and EventSource approaches, as well as other reactive programming techniques, it is better if applications aren't locked in to a specific server / implementation.
+
+3. To provide common considerations and guidelines for pub/sub implementors to consider when implementing their pub/sub modules.
+
+    Some concerns are common for pub/sub implementors, such as integrating third party message brokers (Redis, RabbitMQ, Cassandra)
+
+## Pub/Sub Instance Methods
+
+Conforming pub/sub implementations **MUST** implement the following pub/sub instance methods:
+
+* `pubsub?` **MUST** return the pub/sub API version as an integer number. Currently, set to `0` (development version).
+
+* `subscribe(to, is_pattern = false) { |from, message| optional_block }` where:
+
+    * `to` is a named **channel** (or **pattern**).
+
+        The implementation **MAY** support pattern matching for named channels (`to`). The pattern matching algorithm used, if any, **SHOULD** be documented.
+
+        If the implementation does **NOT** support pattern matching and `is_pattern` is truthful, the implementation **MUST** raise and exception.
+
+        `to` **SHOULD** be a String, but implementations **MAY** support Symbols as aliases to Strings (in which case `:my_channel` is the same `'my_channel'`).
+
+    * `block` is optional and accepts (if provided) two arguments (`from` which is equal to `to` and `message` which contains the data's content).
+
+        `block` (if provided) **MUST** be called when a publication was received by the named channel.
+
+        If no `block` is provided:
+
+        * If the pub/sub instance is an extended WebSocket / EventSource (SSE) `client` object (see the WebSocket / EventSource extension draft) data should be directly sent to the `client`.
+
+        * If the pub/sub isn't linked with a `client` / connection, an exception **MUST** be raised.
+
+    If a subscription to `to` already exists for the same pub/sub instance, it should be *replaced* by the new subscription (the old subscription should be canceled / unsubscribed).
+
+    If the pub/sub instance is an extended WebSocket / EventSource (SSE) `client` object (see the WebSocket / EventSource extension draft), the subscription **MUST** be closed automatically when the connection is closed (when `on_close` is called).
+
+    Otherwise, the subscription **MUST** be closed automatically when the pub/sub object goes out of scope and is garbage collected (if this ever happens).
+    
+    The `subscribe` method **MUST** return `nil` on a known failure (i.e., when the connection is already closed), or any truthful value on success.
+
+* `unsubscribe(from, is_pattern = false)` should cancel a subscription to the `from` named channel / pattern.
+
+* `publish(to, message, engine = nil)` where:
+
+    * `to` is a named channel, same as detailed in `subscribe`.
+
+        Implementations **MAY** support pattern based publishing. This **SHOULD** be documented as well as how patterns are detected (as opposed to named channels). Note that pattern publishing isn't supported by common backends (such as Redis) and introduces complex privacy and security concerns.
+
+    * `message` a String containing the data to be published.
+
+        If `message` is NOT a String, the implementation **MAY** convert the data silently to JSON. Otherwise, the implementation **MUST** raise an exception.
+
+        `message` encoding (binary / UTZ-8) **MAY** be altered during publication, but any change **MUST** result in valid encoding.
+
+    * `engine` routes the publish method to the specified pub/sub Engine (see later on). If none is specified, the default engine should be used. If `false` is specified, the message **MUST** be forwarded to all subscribed clients on the **same process**.
+
+    The `publish` method **MUST** return `true` if a publication was scheduled (not necessarily performed). If it's already known that the publication would fail, the method should return `false`.
+
+    An implementation **MUST** call the relevant PubSubEngine's `publish` method after performing any internal book keeping logic. If `engine` is `nil`, the default PubSubEngine should be called. If `engine` is `false`, the implementation **MUST** forward the published message to the actual clients (if any).
+
+    A global alias for this method (allowing it to be accessed from outside active connections) **MAY** be defined as `Rack::PubSub.publish`.
+
+## Integrating a Pub/Sub module into a WebSocket / EventSource (SSE) `client` object
+
+A conforming pub/sub module **MUST** be designed so that it can be integrated into WebSocket / EventSource (SSE) `client` objects be `include`-ing their class.
+
+This **MUST** result in a behavior where subscriptions are destroyed / canceled once the `client` object state changes to "closed" - i.e., either when the `on_close` callback is called, or the first time the method `client.open?` returns `false`.
+
+The idiomatic way to add a pub/sub module to a `client`'s class is:
+
+```ruby
+client.class.prepend MyPubSubModule unless client.pubsub?
+```
+
+The pub/sub module **MUST** expect and support this behavior.
+
+**Note**: the use of `prepend` (rather than `include`) is chosen so that it's possible to override the `client`'s instance method of `pubsub?`.
+
+## Connecting to External Backends (pub/sub Engines)
+
+It is common for scaling applications to require an external message broker backend such as Redis, RabbitMQ, etc'. The following requirements set a common interface for such "engine" implementation and integration.
+
+Pub/sub implementations **MUST** implement the following module / class methods in one of their public classes / modules (iodine implements these under `Iodine::PubSub`):
+
+* `attach(engine)` where `engine` is a `PubSubEngine` object, as described in this specification.
+
+    When a pub/sub engine is attached, the implementation **MUST** inform the engine of any existing or future subscriptions.
+
+    The implementation **MUST** call the engine's `subscribe` callback for each existing (and future) subscription.
+
+    The implementation **MUST** allow multiple "engines" to be attached when multiple calls to `attach` are made.
+
+* `detach(engine)` where `engine` is a PubSubEngine object as described in this specification.
+
+    The implementation **MUST** remove the engine from the attached engine list. The opposite of `attach`.
+
+* `default=(engine)` sets a default pub/sub engine, where `engine` is a PubSubEngine object as described in this specification.
+
+    Implementations **MUST** forward any `publish` method calls to the default pub/sub engine, unless an `engine` is specified in arguments passes to the `publish` method.
+
+* `default` returns the current default pub/sub engine, where the engine is a PubSubEngine object as described in this specification.
+
+* `reset(engine)` where `engine` is a PubSubEngine object as described in this specification.
+
+    Implementations **MUST** behave as if the engine was newly registered and (re)inform the engine of any existing subscriptions by calling engine's `subscribe` callback for each existing subscription.
+
+A `PubSubEngine` instance object **MUST** implement the following methods:
+
+* `subscribe(to, is_pattern = false)` this method informs the engine that a subscription to the specified channel / pattern exists for the calling the process. It **MUST ONLY** be called **once** for all existing and future subscriptions to that channel within the process.
+
+    The method **MUST** return `true` if a subscription was scheduled (or performed) or `false` if the subscription is known to fail.
+
+    This method will be called by the pub/sub implementation (for each registered engine). The engine **MAY** assume that the method would never be called directly by an application / client.
+
+    This method **MUST NOT** raise an exception.
+
+* `unsubscribe(from, is_pattern = false)` this method informs an engine that there are no more subscriptions to the named channel / pattern for the calling process.
+
+    The method's semantics are similar to `subscribe` only is performs the opposite action.
+
+    This method **MUST NOT** raise an exception.
+
+    This method will be called by the pub/sub implementation (for each registered engine). The engine **MAY** assume that the method would never be called directly by an application / client.
+
+* `publish(channel, message)` where both `channel` is either a Symbol or a String (both being equivalent) and `message` **MUST** be a String.
+
+    This method will be called by the server when a message is published using the engine.
+
+    This method will be called by the pub/sub implementation (for each registered engine).
+
+    The engine **MUST** assume that the method **MAY** be called directly by an application / client.
+
+In order for a PubSubEngine instance object to publish messages to all subscribed clients on a particular process, it **SHOULD** call the implementation's global `publish` method with the engine set to `false`.
+
+i.e., if the implementation's global `publish` method is in a class called `Iodine`:
+
+```ruby
+Iodine.publish channel, message, false
+```

data/SPEC-WebSocket-Draft.md

@@ -0,0 +1,239 @@
+### Draft State
+
+This draft is also implemented by [the Agoo server](https://github.com/ohler55/agoo) according to the specifications stated in [Rack PR#1272](https://github.com/rack/rack/pull/1272).
+
+---
+## Purpose
+
+This document details a Rack specification extension for WebSocket / EventSource servers.
+
+The purpose of this specification is:
+
+1. To improve application safety by phasing out the use of `hijack` and replacing it with the use of application object callbacks.
+
+    This should make it easer for applications to accept WebSocket and EventSource (SSE) connections without exposing themselves to risks and errors related to IO / network logic (such as slow client attacks, buffer flooding, etc').
+
+2. To improve separation of concerns between servers and applications, moving the IO / network logic related to WebSocket and EventSource (SSE) back to the server.
+
+    Simply put, when using a server that support this extension, the application / framework doesn’t need to have any knowledge about networking, transport protocols, IO streams, polling, etc'.
+
+## Rack WebSockets / EventSource
+
+Servers that publish WebSocket and/or EventSource (SSE) support using the `env['rack.upgrade?']` value **MUST** follow the requirements set in this document.
+
+This document reserves the Rack `env` Hash keys of `rack.upgrade?` and `rack.upgrade`.
+
+A conforming server **MUST** set `env['rack.upgrade?']` to `:websocket` for incoming WebSocket connections and `:sse` for incoming EventSource (SSE) connections. 
+
+If a connection is not "upgradeable", a conforming server **SHOULD** set `env['rack.upgrade?']` to either `nil` or `false`. Setting the `env['rack.upgrade?']` to either `false` or `nil` should make it easier for applications to test for server support during a normal HTTP request.
+
+If the connection is upgradeable and a client application set a value for `env['rack.upgrade']`:
+
+* the server **MUST** use that value as a WebSocket / EventSource Callback Object unless the response status code `>= 300` (redirection / error status code).
+
+* The response `body` **MUST NOT** be sent when switching to a Callback Object.
+
+If a connection is **NOT** upgradeable and a client application set a value for `env['rack.upgrade']`:
+
+* The server **SHOULD** ignore the Callback Object and process the response as if it did not exist.
+
+* A server **MAY** use the Callback Object to allow a client to "hijack" the data stream as raw data stream. Such behavior **MUST** be documented.
+
+### The WebSocket / EventSource Callback Object
+
+WebSocket and EventSource connection upgrade and handling is performed using a Callback Object.
+
+The Callback Object could be a any object which implements any (of none) of the following callbacks:
+
+* `on_open(client)` **MUST** be called once the connection had been established and/or the Callback Object had been linked to the `client` object.
+
+* `on_message(client, data)` **MUST** be called when incoming WebSocket data is received.
+
+    This callback is ignored for EventSource connections.
+
+    `data` **MUST** be a String with an encoding of UTF-8 for text messages and `binary` encoding for non-text messages (as specified by the WebSocket Protocol).
+
+    The *callback object* **MUST** assume that the `data` String will be a **recyclable buffer** and that it's content will be corrupted the moment the `on_message` callback returns.
+
+    Servers **MAY**, optionally, implement a **recyclable buffer** for the `on_message` callback. However, this is optional, is *not* required and might result in issues in cases where the client code is less than pristine.
+
+* `on_drained(client)` **MAY** be called when the `client.write` buffer becomes empty. **If** `client.pending` ever returns a non-zero value (see later on), the `on_drained` callback **MUST** be called once the write buffer becomes empty.
+
+* `on_shutdown(client)` **MAY** be called during the server's graceful shutdown process, _before_ the connection is closed and in addition to the `on_close` function (which is called _after_ the connection is closed.
+
+* `on_close(client)` **MUST** be called _after_ the connection was closed for whatever reason (socket errors, parsing errors, timeouts, client disconnection, `client.close` being called, etc') or the Callback Object was replaced by another Callback Object.
+
+
+The server **MUST** provide the Callback Object with a `client` object, that supports the following methods (this approach promises applications could be server agnostic):
+
+* `env` **MUST** return the Rack `env` hash related to the originating HTTP request. Some changes to the `env` hash (such as removal of the IO hijacking support) **MAY** be implemented by the server.
+
+* `write(data)` **MUST** schedule **all** the data to be sent. `data` **MUST** be a String. Servers **MAY** silently convert non-String objects to JSON if an application attempts to `write` a non-String value, otherwise servers **SHOULD** throw an exception.
+
+    A call to `write` only promises that the data is scheduled to be sent. Implementation details may differ across servers.
+
+    `write` shall return `true` on success and `false` if the connection is closed.
+
+    For WebSocket connections only (irrelevant for EventSource connections):
+
+    * If `data` is UTF-8 encoded, the data will be sent as text.
+
+    * If `data` is binary encoded it will be sent as non-text (as specified by the WebSocket Protocol).
+
+    A server **SHOULD** document its concurrency model, allowing developers to know whether `write` will block or not, whether buffered IO is implemented, etc'.
+
+    For example, evented servers are encouraged to avoid blocking and return immediately, deferring the actual `write` operation for later. However, (process/thread/fiber) per-connection based servers **MAY** choose to return only after all the data was sent. Documenting these differences will allows applications to choose the model that best fits their needs and environments.
+
+* `close` closes the connection once all the data scheduled using `write` was sent. If `close` is called while there is still data to be sent, `close` **SHOULD** return immediately and only take effect once the data was sent.
+
+    `close` shall always return `nil`.
+
+* `open?` **MUST** return `false` **if** the connection was never open, is known to be closed or marked to be closed. Otherwise `true` **MUST** be returned.
+
+* `pending` **MUST** return -1 if the connection is closed. Otherwise, `pending` **SHOULD** return the number of pending writes (messages in the `write` queue\*) that need to be processed before the next time the `on_drained` callback is called.
+
+    Servers **MAY** choose to always return the value `0` **ONLY IF** they never call the `on_drained` callback and the connection is open.
+
+    Servers that return a positive number **MUST** call the `on_drained` callback when a call to `pending` would return the value `0`.
+
+    \*Servers that divide large messages into a number of smaller messages (implement message fragmentation) **MAY** count each fragment separately, as if the fragmentation was performed by the user and `write` was called more than once per message.
+
+* `pubsub?` **MUST** return `false` **unless** the pub/sub extension is supported.
+
+   Pub/Sub patterns are idiomatic for WebSockets and EventSource connections but their API is out of scope for this extension.
+
+* `class` **MUST** return the client's Class, allowing it be extended with additional features (such as Pub/Sub, etc').
+
+    **Note**: Ruby adds this method automatically to every class, no need to do a thing.
+
+The server **MAY** support the following (optional) methods for the `client` object:
+
+* `handler` if implemented, **MUST** return the callback object linked to the `client` object.
+
+* `handler=` if implemented, **MUST** set a new Callback Object for `client`.
+
+    This allows applications to switch from one callback object to another (i.e., in case of credential upgrades).
+
+    Once a new Callback Object was set, the server **MUST** call the old handler's `on_close` callback and **afterwards** call the new handler's `on_open` callback.
+
+    It is **RECOMMENDED** (but not required) that this also updates the value for `env['rack.upgrade']`.
+
+* `timeout` / `timeout=` allows applications to get / set connection timeouts dynamically and separately for each connection. Servers **SHOULD** provide a global setting for the default connection timeout. It is **RECOMMENDED** (but not required) that a global / default timeout setting be available from the command line (CLI).
+
+* `protocol` if implemented, **MUST** return the same value that was originally set by `env['rack.upgrade?']`.
+
+
+WebSocket `ping` / `pong`, timeouts and network considerations **SHOULD** be implemented by the server. It is **RECOMMENDED** (but not required) that the server send `ping`s to prevent connection timeouts and to detect network failure. Clients **SHOULD** also consider sending `ping`s to detect network errors (dropped connections).
+
+Server settings **MAY** be provided to allow for customization and adaptation for different network environments or WebSocket extensions. It is **RECOMMENDED** that any settings be available as command line arguments and **not** incorporated into the application's logic.
+
+---
+
+## Implementation Examples
+
+### Server-Client Upgrade to WebSockets / EventSource
+
+* **Server**:
+
+    When a regular HTTP request arrives (non-upgradeable), the server will set the `env['rack.upgrade?']` flag to `false`, indicating that: 1. this specific request is NOT upgradable; and 2. the server supports this specification for either WebSocket and/or EventSource connections.
+
+    When a WebSocket upgrade request arrives, the server will set the `env['rack.upgrade?']` flag to `:websocket`, indicating that: 1. this specific request is upgradable; and 2. the server supports this specification for WebSocket connections.
+
+    When an EventSource request arrives, the server will set the `env['rack.upgrade?']` flag to `:sse`, indicating that: 1. this specific request is an EventSource request; and 2. the server supports this specification for EventSource connections.
+
+* **Client**:
+
+    If a client decides to upgrade a request, they will place an appropriate Callback Object in the `env['rack.upgrade']` Hash key.
+
+* **Server**:
+
+    1. If the application's response status indicates an error or a redirection (status code `>= 300`), the server shall ignore the Callback Object and/or remove it from the `env` Hash, ignoring the rest of the steps that follow.
+
+    2. The server will review the `env` Hash *before* sending the response. If the `env['rack.upgrade']` was set, the server will perform the upgrade.
+
+    3. The server will send the correct response status and headers, as well as any headers present in the response object. The server will also perform any required housekeeping, such as closing the response body, if it exists.
+
+         The response status provided by the response object shall be ignored and the correct response status shall be set by the server.
+
+    4. Once the upgrade had completed, the server will call the `on_open` callback.
+
+        No other callbacks shall be called until the `on_open` callback had returned.
+
+        WebSocket messages shall be handled by the `on_message` callback in the same order in which they arrive and the `on_message` **SHOULD NOT** be executed concurrently for the same connection.
+
+        The `on_close` callback **MUST NOT** be called while any other callback is running (`on_open`, `on_message`, `on_drained`, etc').
+
+        The `on_drained` callback **MAY** be called concurrently with the `on_message` callback, allowing data to be sent even while incoming data is being processed. Multi-threading considerations apply.
+
+## Example Usage
+
+The following is an example WebSocket echo server implemented using this specification:
+
+```ruby
+module WSConnection
+    def on_open(client)
+        puts "WebSocket connection established (#{client.object_id})."
+    end
+    def on_message(client, data)
+        client.write data # echo the data back
+        puts "on_drained MUST be implemented if #{ pending } != 0."
+    end
+    def on_drained(client)
+        puts "If this line prints out, on_drained is supported by the server."
+    end
+    def on_shutdown(client)
+        client.write "The server is going away. Goodbye."
+    end
+    def on_close(client)
+        puts "WebSocket connection closed (#{client.object_id})."
+    end
+    extend self
+end
+
+module App
+   def self.call(env)
+       if(env['rack.upgrade?'.freeze] == :websocket)
+           env['rack.upgrade'.freeze] = WSConnection
+           return [0, {}, []]
+       end
+       return [200, {"Content-Length" => "12", "Content-Type" => "text/plain"}, ["Hello World!"]]
+   end
+end
+
+run App
+```
+
+The following example uses Push notifications for both WebSocket and SSE connections. The Pub/Sub API is subject to a separate Pub/Sub API extension and isn't part of this specification (it is, however, supported by iodine):
+
+```ruby
+module Chat
+    def on_open(client)
+        client.class.prepend MyPubSubModule unless client.pubsub?
+        client.subscribe "chat"
+        client.publish "chat", "#{env[:nickname]} joined the chat."
+    end
+    def on_message(client, data)
+        client.publish "chat", "#{env[:nickname]}: #{data}"
+    end
+    def on_close(client)
+        client.publish "chat", "#{env[:nickname]}: left the chat."
+    end
+    extend self
+end
+
+module App
+   def self.call(env)
+       if(env['rack.upgrade?'.freeze])
+           nickname = env['PATH_INFO'][1..-1]
+           nickname = "Someone" if nickname == "".freeze
+           env[:nickname] = nickname
+           return [0, {}, []]
+       end
+       return [200, {"Content-Length" => "12", "Content-Type" => "text/plain"}, ["Hello World!"]]
+   end
+end
+
+run App
+```
+
+Note that SSE connections will only be able to receive messages (the `on_message` callback is never called).

data/bin/console

@@ -0,0 +1,22 @@
+#!/usr/bin/env ruby
+
+# this will compile Iodine and start a Ruby console with the Iodine gem loaded.
+
+Dir.chdir(File.expand_path(File.join('..', '..'), __FILE__))
+puts `rake clean`
+puts `rake compile`
+
+require 'benchmark'
+$LOAD_PATH.unshift File.expand_path(File.join('..', '..', 'lib'), __FILE__ )
+require "bundler/setup"
+require "iodine"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start