iodine 0.3.6 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of iodine might be problematic. Click here for more details.

Files changed (74) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +46 -0
  3. data/LIMITS.md +25 -0
  4. data/README.md +39 -80
  5. data/SPEC-Websocket-Draft.md +129 -4
  6. data/bin/echo +2 -2
  7. data/bin/http-hello +1 -0
  8. data/bin/updated api +113 -0
  9. data/bin/ws-echo +0 -1
  10. data/examples/broadcast.ru +56 -0
  11. data/examples/echo.ru +57 -0
  12. data/examples/hello.ru +30 -0
  13. data/examples/redis.ru +69 -0
  14. data/examples/shootout.ru +53 -0
  15. data/exe/iodine +2 -80
  16. data/ext/iodine/defer.c +11 -5
  17. data/ext/iodine/empty.h +26 -0
  18. data/ext/iodine/evio.h +1 -1
  19. data/ext/iodine/facil.c +103 -61
  20. data/ext/iodine/facil.h +20 -12
  21. data/ext/iodine/fio_dict.c +446 -0
  22. data/ext/iodine/fio_dict.h +90 -0
  23. data/ext/iodine/fio_hash_table.h +370 -0
  24. data/ext/iodine/fio_list.h +30 -3
  25. data/ext/iodine/http.c +169 -37
  26. data/ext/iodine/http.h +33 -10
  27. data/ext/iodine/http1.c +78 -42
  28. data/ext/iodine/http_request.c +6 -0
  29. data/ext/iodine/http_request.h +3 -0
  30. data/ext/iodine/http_response.c +43 -11
  31. data/ext/iodine/iodine.c +380 -0
  32. data/ext/iodine/iodine.h +62 -0
  33. data/ext/iodine/iodine_helpers.c +235 -0
  34. data/ext/iodine/iodine_helpers.h +13 -0
  35. data/ext/iodine/iodine_http.c +409 -241
  36. data/ext/iodine/iodine_http.h +7 -14
  37. data/ext/iodine/iodine_protocol.c +626 -0
  38. data/ext/iodine/iodine_protocol.h +13 -0
  39. data/ext/iodine/iodine_pubsub.c +646 -0
  40. data/ext/iodine/iodine_pubsub.h +27 -0
  41. data/ext/iodine/iodine_websockets.c +796 -0
  42. data/ext/iodine/iodine_websockets.h +19 -0
  43. data/ext/iodine/pubsub.c +544 -0
  44. data/ext/iodine/pubsub.h +215 -0
  45. data/ext/iodine/random.c +4 -4
  46. data/ext/iodine/rb-call.c +1 -5
  47. data/ext/iodine/rb-defer.c +3 -20
  48. data/ext/iodine/rb-rack-io.c +22 -22
  49. data/ext/iodine/rb-rack-io.h +3 -4
  50. data/ext/iodine/rb-registry.c +111 -118
  51. data/ext/iodine/redis_connection.c +277 -0
  52. data/ext/iodine/redis_connection.h +77 -0
  53. data/ext/iodine/redis_engine.c +398 -0
  54. data/ext/iodine/redis_engine.h +68 -0
  55. data/ext/iodine/resp.c +842 -0
  56. data/ext/iodine/resp.h +253 -0
  57. data/ext/iodine/sock.c +26 -12
  58. data/ext/iodine/sock.h +14 -3
  59. data/ext/iodine/spnlock.inc +19 -2
  60. data/ext/iodine/websockets.c +299 -11
  61. data/ext/iodine/websockets.h +159 -6
  62. data/lib/iodine.rb +104 -1
  63. data/lib/iodine/cli.rb +106 -0
  64. data/lib/iodine/monkeypatch.rb +40 -0
  65. data/lib/iodine/pubsub.rb +70 -0
  66. data/lib/iodine/version.rb +1 -1
  67. data/lib/iodine/websocket.rb +12 -0
  68. data/lib/rack/handler/iodine.rb +33 -7
  69. metadata +35 -7
  70. data/ext/iodine/iodine_core.c +0 -760
  71. data/ext/iodine/iodine_core.h +0 -79
  72. data/ext/iodine/iodine_websocket.c +0 -551
  73. data/ext/iodine/iodine_websocket.h +0 -22
  74. data/lib/iodine/http.rb +0 -4
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: efa1a3d3d054e8668d5e7f79bf04e61d5f0b169b
4
- data.tar.gz: fe096a9cc9f6e6a1196c3c3091228c6d3a72a539
3
+ metadata.gz: 9a37969c5d9707236f1d87fcc548a442908d01fd
4
+ data.tar.gz: 450f40d62e72cf9a575188ecd62509fc1429a771
5
5
  SHA512:
6
- metadata.gz: 6a6d75ec543937ff9f660cf643b442bd8adc8f4c31deac8d9233570385a6ae38a8743c4f1360b7bf3215401019d254e1fbb31c9dc57a1a197ee3c56aad97a1a6
7
- data.tar.gz: 980fa44bbba6a0ad6c43c5f8383482261540a6c602b52a98ad716d7bb51ff7635f8fd58aaeb0bfb60bdcbed74218fa98a9ea8edc3cb2122cb5ed1d88c104098e
6
+ metadata.gz: 6c3fb1a3627926510480b73a3468940c1e436c15e60c295c508003beeb8997120047a8d483df76e6bfdc0e295eb199e2222c5713c11b33da0ae97ef316e39560
7
+ data.tar.gz: 67291bdfbcf83e45f03aae739e17ff5b511746ef8e7aa4ba99622ecb5306d1a9c9558cef8867e93fd5469f00c031ed70a64355231f5b0f7baff0f4037c480d4e
@@ -8,6 +8,52 @@ Please notice that this change log contains changes for upcoming releases as wel
8
8
 
9
9
  ***
10
10
 
11
+ #### Change log v.0.4.0
12
+
13
+ **Braking change**: Some of the API was changed / updated, hopefully simplified.
14
+
15
+ **DEPRECTAION / Braking change**: The `websocket#write_each` function is gone. Future (planned) support for a Pub/Sub API would have caused confusion and since it's barely used (probably only there as a benchmarking proof of concept) it was removed.
16
+
17
+ **Update**: Now using `facil.io` v.0.5.0 The extended features include the following listed features.
18
+
19
+ **Fixes**: This was such a big rewrite, I suspect half the fixes I want to list are things I broke during the rewrite... but there were plenty of fixes.
20
+
21
+ **Feature**: Iodine now support native Websocket Pub/Sub, with [an example in the `examples` folder](./examples/redis.ru). i.e.:
22
+
23
+ ```ruby
24
+ # Within a Websocket connection:
25
+ subscribe "chat"
26
+ publish "chat", "Iodine is here!"
27
+ ```
28
+
29
+ **Feature**: Iodine's Pub/Sub API supports both direct client messages and server filtered messages. i.e.
30
+
31
+ ```ruby
32
+ # Within a Websocket connection:
33
+ subscribe("chat-server") {|msg| write "Notice: #{msg}" }
34
+ # v.s
35
+ subscribe("chat-client")
36
+
37
+ publish "chat-server", "Iodine is here!"
38
+ ```
39
+
40
+ **Feature**: Iodine's Pub/Sub API includes support for home made Pub/Sub Engines connecting Iodine to your Pub/Sub service of choice.
41
+
42
+ **Feature**: Iodine's Pub/Sub support includes a Process Cluster engine (pub/sub to all Websockets sharing the process cluster) as well as a Single Process engine (pub/sub to all websockets supporting a single process).
43
+
44
+ **Feature**: Iodine's Pub/Sub support includes a native Redis Pub/Sub engine. The parser is written from the ground up in C to keep the Iodine licensing as MIT. It's young, so keep your eyes pealed and submit any issues you encounter.
45
+
46
+ **Feature + Breaking Change**: Iodine now support multiple HTTP servers at once. i.e.:
47
+
48
+ ```ruby
49
+ # `Iodine::HTTP.listen` initializes an HTTP service in the C and system levels, so it can't be changed once initialized.
50
+ Iodine::HTTP.listen port: 3000, app: my_app1
51
+ Iodine::HTTP.listen port: 3000, app: my_app2, public: "./www"
52
+ Iodine.start
53
+ ```
54
+
55
+ ***
56
+
11
57
  #### Change log v.0.3.6
12
58
 
13
59
  **Update**: Now using `facil.io` v.0.4.3. This fixes some delays in websocket packet flushing (latency), as well as other internal polishes. It also promises some possible future feature extensions that could add a major performance boost.
@@ -0,0 +1,25 @@
1
+ # Iodine limits and settings
2
+
3
+ I will, at some point, document these... here's a few things:
4
+
5
+ ## HTTP/1.x limits
6
+
7
+ * HTTP Headers have a ~16Kb total size limit.
8
+
9
+ * Uploads are adjustable and limited to ~50Mib by default.
10
+
11
+ * HTTP keep-alive is adjustable and set to ~5 seconds by default.
12
+
13
+ ## Websocket
14
+
15
+ * Incoming Websocket message size limits are adjustable and limited to ~250Kib by default.
16
+
17
+ ## Pub/Sub
18
+
19
+ * Channel names are binary safe, but limited to 1024 characters (even though Redis doesn't seem to limit channel name length).
20
+
21
+ Iodine uses a [trie](https://en.wikipedia.org/wiki/Trie) to match channel names. A trie offers binary exact matching with a fast lookup (unlike hash lookups, this offers zero name collision risk). However, it also means that channel names are memory **expensive**.
22
+
23
+ As a side effect, sequencial channel names have an advantage over random channel names.
24
+
25
+ * Pub/sub is limited to the process cluster. To use pub/sub with an external service (such as Redis) an "Engine" is required (see YARD documentation).
data/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
  [![Inline docs](http://inch-ci.org/github/boazsegev/iodine.svg?branch=master)](http://www.rubydoc.info/github/boazsegev/iodine/master/frames)
7
7
  [![GitHub](https://img.shields.io/badge/GitHub-Open%20Source-blue.svg)](https://github.com/boazsegev/iodine)
8
8
 
9
- Iodine is a fast concurrent web server for real-time Ruby applications, with native support for Websockets, static file service and HTTP/1.1.
9
+ Iodine is a fast concurrent web server for real-time Ruby applications, with native support for Websockets, Pub/Sub, static file service, HTTP/1.1 and Redis Pub/Sub scaling (2 connections per Iodine process).
10
10
 
11
11
  Iodine also supports custom protocol authoring, making Object Oriented **Network Services** easy to write.
12
12
 
@@ -22,52 +22,41 @@ Iodine is an **evented** framework with a simple API that builds off the low lev
22
22
 
23
23
  Iodine is a C extension for Ruby, developed for Ruby MRI 2.2.2 and up... it should support the whole Ruby 2.0 MRI family, but Rack requires Ruby 2.2.2, and so Iodine matches this requirement.
24
24
 
25
- ## Iodine::Rack - an HTTP and Websockets server
25
+ ## Iodine::Rack == a fast and powerful HTTP + Websockets server with native Pub/Sub
26
26
 
27
27
  Iodine includes a light and fast HTTP and Websocket server written in C that was written according to the [Rack interface specifications](http://www.rubydoc.info/github/rack/rack/master/file/SPEC) and the [Websocket draft extension](./SPEC-Websocket-Draft.md).
28
28
 
29
+ With `Iodine.listen2http` it's possible to run multiple HTTP applications in addition to (or instead of) the default `Iodine::Rack` HTTP service.
30
+
31
+ Iodine also supports native process cluster Pub/Sub and a native RedisEngins to easily scale Iodine's Pub/Sub horizontally.
32
+
29
33
  ### Running the web server
30
34
 
31
35
  Using the Iodine server is easy, simply add Iodine as a gem to your Rack application:
32
36
 
33
37
  ```ruby
34
- gem 'iodine', '>=0.3'
38
+ gem 'iodine', '~>0.4'
35
39
  ```
36
40
 
37
- Iodine will calculate, when possible, a good enough default concurrency model for fast applications... this might not fit your application if you use database access or other blocking calls.
41
+ Iodine will calculate, when possible, a good enough default concurrency model for lightweight applications... this might not fit your application if you use heavier database access or other blocking calls.
38
42
 
39
43
  To get the most out of Iodine, consider the amount of CPU cores available and the concurrency level the application requires.
40
44
 
41
- Puma's model of 16 threads and 4 processes is easily adopted and proved to provide a good enough balance for most use-cases. Use:
45
+ The common model of 16 threads and 4 processes can be easily adopted:
42
46
 
43
47
  ```bash
44
48
  bundler exec iodine -p $PORT -t 16 -w 4
45
49
  ```
46
50
 
47
- It should be noted that automatic process scaling will cause issues with Websocket broadcast (`each`) support, since the `Websocket#each` method will be limited to the calling process (other clients might be connected to a different process).
48
-
49
- It is recommended that you consider using Redis to scale Websocket "events" across processes / machines.
50
- Look into [plezi.io](http://www.plezi.io) for automatic Websocket scaling with Redis and Iodine.
51
-
52
- ### Writing data to the network layer
53
-
54
- Iodine allows Ruby to write strings to the network layer. This includes HTTP and Websocket responses.
55
-
56
- Iodine will handle an internal buffer (~4 to ~16 Mb, version depending) so that `write` can return immediately (non-blocking).
57
-
58
- However, when the buffer is full, `write` will block until enough space in the buffer becomes available. Sending up to 16Kb of data (a single buffer "packet") is optimal. Sending a larger response might effect concurrency. Best Websocket response length is ~1Kb (1 TCP / IP packet) and allows for faster transmissions.
59
-
60
- When using the Iodine's web server (`Iodine::Rack`), the static file service offered by Iodine streams files (instead of using the buffer). Every file response will require up to 2 buffer "packets" (~32Kb), one for the header and the other for file streaming.
61
-
62
- This means that even a Gigabyte long response will use ~32Kb of memory, as long as it uses the static file service or the `X-Sendfile` extension (Iodine's static file service can be invoked by the Ruby application using the `X-Sendfile` header).
63
-
64
51
  ### Static file serving support
65
52
 
66
- Iodine supports static file serving that allows the server to serve static files directly, with no Ruby layer (all from C-land).
53
+ Iodine supports an internal static file service that bypasses the Ruby layer and serves static files directly from "C-land".
67
54
 
68
- This means that Iodine won't lock Ruby's GVL when sending static files. The files will be sent directly, allowing for true native concurrency. Since the Ruby layer is unaware of these requests, logging can be performed by turning iodine's logger on.
55
+ This means that Iodine won't lock Ruby's GVL when sending static files. The files will be sent directly, allowing for true native concurrency.
69
56
 
70
- To setup native static file service, setup the public folder's address **before** starting the server.
57
+ Since the Ruby layer is unaware of these requests, logging can be performed by turning iodine's logger on.
58
+
59
+ To use native static file service, setup the public folder's address **before** starting the server.
71
60
 
72
61
  This can be done when starting the server from the command line:
73
62
 
@@ -133,23 +122,38 @@ To "upgrade" the HTTP request to the Websockets protocol, simply provide Iodine
133
122
 
134
123
  Iodine will adopt the object, providing it with network functionality (methods such as `write`, `each`, `defer` and `close` will become available) and invoke it's callbacks on network events.
135
124
 
136
- Here is a simple example we can run in the terminal (`irb`) or easily paste into a `config.ru` file:
125
+ Here is a simple chatroom example we can run in the terminal (`irb`) or easily paste into a `config.ru` file:
137
126
 
138
127
  ```ruby
139
128
  require 'iodine'
140
129
  class WebsocketEcho
130
+ def on_open
131
+ # Pub/Sub directly to the client (or use a block to process the messages)
132
+ subscribe channel: :chat
133
+ # Writing directly to the socket
134
+ write "You're now in the chatroom."
135
+ end
141
136
  def on_message data
142
- write data
137
+ # Strings and symbol channel names are equivalent.
138
+ publish channel: "chat", message: data
143
139
  end
144
140
  end
145
141
  Iodine::Rack.app= Proc.new do |env|
146
142
  if env['upgrade.websocket?'.freeze] && env["HTTP_UPGRADE".freeze] =~ /websocket/i.freeze
147
- env['iodine.websocket'.freeze] = WebsocketEcho # or: WebsocketEcho.new
148
- [100,{}, []] # It's possible to set cookies for the response.
143
+ env['upgrade.websocket'.freeze] = WebsocketEcho # or: WebsocketEcho.new
144
+ [0,{}, []] # It's possible to set cookies for the response.
149
145
  else
150
146
  [200, {"Content-Length" => "12"}, ["Welcome Home"] ]
151
147
  end
152
148
  end
149
+ # Pus/Sub can be server oriented as well as connection bound
150
+ root_pid = Process.pid
151
+ Iodine.subscribe(channel: :chat) {|ch, msg| puts msg if Process.pid == root_pid }
152
+ # By default, Pub/Sub performs in process cluster mode.
153
+ Iodine.processes = 4
154
+ # static file serving can be set manually as well as using the command line:
155
+ Iodine::Rack.public = "www/public"
156
+ #
153
157
  Iodine.start
154
158
  ```
155
159
 
@@ -180,49 +184,10 @@ Iodine.start
180
184
 
181
185
  #### A few notes
182
186
 
183
- This design has a number of benefits, some of them related to better IO handling, resource optimization (no need for two IO polling systems), etc. This also allows us to use middleware without interfering with connection upgrades and provides backwards compatibility.
187
+ This design has a number of benefits, some of them related to better IO handling, resource optimization (no need for two IO polling systems), etc. This also allows us to use middleware without interfering with connection upgrades and provides backwards compatibility.
184
188
 
185
189
  Iodine::Rack imposes a few restrictions for performance and security reasons, such as that the headers (both sending and receiving) must be less than 8Kb in size. These restrictions shouldn't be an issue and are similar to limitations imposed by Apache.
186
190
 
187
- Here's a small HTTP and Websocket broadcast server with Iodine::Rack, which can be used directly from `irb`:
188
-
189
- ```ruby
190
- require 'iodine'
191
-
192
- # Our server controller and websockets handler
193
- class My_Broadcast
194
-
195
- # handle HTTP requests (a class callback, emulating a Proc)
196
- def self.call env
197
- if env["HTTP_UPGRADE".freeze] =~ /websocket/i.freeze
198
- env['upgrade.websocket'.freeze] = self.new(env)
199
- [0,{}, []]
200
- end
201
- [200, {"Content-Length" => "12".freeze}, ["Hello World!".freeze]]
202
- end
203
-
204
- def initialize env
205
- @env = env # allows us to access the HTTP request data during the Websocket session
206
- end
207
-
208
- # handles websocket data (an instance callback)
209
- def on_message data
210
- # data is the direct buffer and will be recycled once we leave this scope.
211
- # we'll copy it to prevent corruption when broadcasting the data asynchronously.
212
- data_copy = data.dup
213
- # We'll broadcast the data asynchronously to all open websocket connections.
214
- each {|ws| ws.write data_copy } # (limited to current process)
215
- close if data =~ /^bye[\r\n]/i
216
- end
217
- end
218
-
219
- # static file serving is as easy as (also supports simple byte serving):
220
- Iodine::Rack.public = "www/public"
221
-
222
- # start the server while setting the app at the same time
223
- Iodine::Rack.run My_Broadcast
224
- ```
225
-
226
191
  Of course, if you still want to use Rack's `hijack` API, Iodine will support you - but be aware that you will need to implement your own reactor and thread pool for any sockets you hijack, as well as a socket buffer for non-blocking `write` operations (why do that when you can write a protocol object and have the main reactor manage the socket?).
227
192
 
228
193
  ### Performance oriented design - but safety first
@@ -295,11 +260,13 @@ Then start comparing servers. Here are the settings I used to compare Iodine and
295
260
  $ RACK_ENV=production iodine -p 3000 -t 16 -w 4
296
261
  # vs.
297
262
  $ RACK_ENV=production puma -p 3000 -t 16 -w 4
263
+ # Review the `iodine -?` help for more command line options.
298
264
  ```
299
265
 
300
- Iodine performed almost twice as well, (~90K req/sec vs. ~44K req/sec) while keeping a memory foot print that was more then 20% lower (~65Mb vs. ~85Mb).
301
266
 
302
- Review the `iodine -?` help for more data.
267
+ When benchmarking with `wrk`, Iodine performed significantly better, (~62K req/sec vs. ~44K req/sec) while keeping a lower memory foot print (~60Mb vs. ~111Mb).
268
+
269
+ When benchmarking with `ab`, I got different results, where Iodine still performed significantly better, (~72K req/sec vs. ~36K req/sec and ~61Mb vs. ~81.6Mb). I suspect the difference between the two benchmarks has to do with system calls to `write`, but I have no real proof.
303
270
 
304
271
  Remember to compare the memory footprint after running some requests - it's not just speed that C is helping with, it's also memory management and object pooling (i.e., Iodine uses a buffer packet pool management).
305
272
 
@@ -355,21 +322,13 @@ Iodine.start
355
322
 
356
323
  ```
357
324
 
358
- ## I loved Iodine 0.1.x - is this an upgrade?
359
-
360
- This is **not** an upgrade, this is a **full rewrite**.
361
-
362
- Iodine 0.1.x was written in Ruby and had tons of bells and whistles and a somewhat different API. It also inherited the `IO.select` limit of 1024 concurrent connections.
363
-
364
- Iodine 0.2.x is written in C, doesn't have as many bells and whistles (i.e., no Websocket Client) and has a stripped down API (simpler to learn). The connection limit is calculated on startup, according to the system's limits. Connection overflows are terminated with an optional busy message, so the system won't crash.
365
-
366
325
  ## Why not EventMachine?
367
326
 
368
327
  You can go ahead and use EventMachine if you like. They're doing amazing work on that one and it's been used a lot in Ruby-land... really, tons of good developers and people on that project, I'm sure...
369
328
 
370
329
  But me, I prefer to make sure my development software runs the exact same code as my production software. So here we are.
371
330
 
372
- Also, I don't really understand all the minute details of EventMachine's API, it kept crashing my system every time I reached ~1024 active connections... I'm sure I just don't know how to use EventMachine, but that's just that.
331
+ Also, I don't really understand all the minute details of EventMachine's API, it kept crashing my system every time I reached 1K-2K active connections... I'm sure I just don't know how to use EventMachine, but that's just that.
373
332
 
374
333
  Besides, you're here - why not take Iodine out for a spin and see for yourself?
375
334
 
@@ -1,8 +1,10 @@
1
1
  ### Draft Inactivity Notice
2
2
 
3
- This proposed draft is only implemented by Iodine and hadn't seen external activity in the last four months. It is unlikely that this initiative will grow to become a community convention.
3
+ This proposed draft is only implemented by Iodine and hadn't seen external activity in a while.
4
4
 
5
- I still believe it's important to separate the Websocket server from the Websocket application (much like Rack did for HTTP).n I hope that in the future a community convention for this separation of concerns can be achieved.
5
+ Even though a number of other development teams (such as the teams for the Puma and Passenger server) mentioned that they plan to implements this draft, Iodine seems to be the only server currently implementing this draft and it is unlikely that this initiative will grow to become a community convention.
6
+
7
+ I still believe it's important to separate the Websocket server from the Websocket API used by application developers and frameworks, much like Rack did for HTTP. I hope that in the future a community convention for this separation of concerns can be achieved.
6
8
 
7
9
  ---
8
10
  ## Rack Websockets
@@ -67,13 +69,13 @@ Server settings **MAY** (not required) be provided to allow for customization an
67
69
 
68
70
  ## Upgrading
69
71
 
70
- * **Server**: When an upgrade request is received, the server will set the `env['upgrade.websocket?']` flag to `true`, indicating that: 1. this specific request is upgradable; and 2. this server supports specification.
72
+ * **Server**: When an upgrade request is received, the server will set the `env['upgrade.websocket?']` flag to `true`, indicating that: 1. this specific request is upgradable; and 2. this server supports this specification.
71
73
 
72
74
  * **Client**: When a client decides to upgrade a request, they will place a Websocket Callback Object (either a class or an instance) in the `env['upgrade.websocket']` Hash key.
73
75
 
74
76
  * **Server**: The server will review the `env` Hash *before* sending the response. If the `env['upgrade.websocket']` was set, the server will perform the upgrade.
75
77
 
76
- * **Server**: The server will send the correct response status and headers, as will as any headers present in the response. The server will also perform any required housekeeping, such as closing the response body, if exists.
78
+ * **Server**: The server will send the correct response status and headers, as well as any headers present in the response object. The server will also perform any required housekeeping, such as closing the response body, if exists.
77
79
 
78
80
  The response status provided by the response object shall be ignored and the correct response status shall be set by the server.
79
81
 
@@ -88,3 +90,126 @@ Server settings **MAY** (not required) be provided to allow for customization an
88
90
  The `on_close` callback will **not** be called while `on_message` or `on_open` callbacks are running.
89
91
 
90
92
  The `on_ready` callback might be called concurrently with the `on_message` callback, allowing data to be sent even while other data is being processed. Multi-threading considerations may apply.
93
+
94
+ ---
95
+
96
+ ## Iodine's Collection Extension to the Rack Websockets
97
+
98
+ The biggest benefit from Iodine's Collection Extension is that it allows the creation of pub/sub plugins and other similar extensions that require access to all the connected Websockets - no nee for the plugin to ask "where's the list" or "add this code to `on_open`" or anything at all, truly "plug and play".
99
+
100
+ This extension should be easy enough to implement using a loop or an array within each process context. These methods do **not** cross process boundaries and they are meant to help implement more advanced features (they aren't a feature as much as an "access point").
101
+
102
+ Internally, this extension also allows iodine to manage connection memory and resource allocation while relieving developers from rewriting the same workflow of connection storing (`on_open do ALL_WS << self; end `) and management. Knowing that the user doesn't keep Websocket object references, allows Iodine to safely optimize it's memory use.
103
+
104
+ * `Iodine::Websocket.each` (class method) will run a block of code for each connected Websocket Callback Object **belonging to the current process**. This can be called from outside of a Websocket Callback Object as well.
105
+
106
+ ```ruby
107
+ Iodine::Websocket.each {|ws| ws.write "You're connected to PID #{Process.pid}" }
108
+ ```
109
+
110
+ * `Iodine::Websocket.defer(conn_id)` Schedules a block of code to run for the specified connection at a later time, (*if* the connection is open) while preventing concurrent code from running for the same connection object.
111
+
112
+ ```ruby
113
+ Iodine::Websocket.defer(self.conn_id) {|ws| ws.write "still open" }
114
+ ```
115
+
116
+ * `#defer` (instance method) Schedules a block of code to run for the specified connection at a later time, (*if* the connection is still open) while preventing concurrent code from running for the same connection object.
117
+
118
+ ```ruby
119
+ defer { write "still open" }
120
+ ```
121
+
122
+ * `Iodine::Websocket.count` (instance method) Returns the number of active websocket connections (including connections that are in the process of closing down) **belonging to the current process**.
123
+
124
+ ```ruby
125
+ write "#{Iodine::Websocket.count} clients sharing this process"
126
+ ```
127
+
128
+ ## Iodine's Pub/Sub Extension to the Rack Websockets
129
+
130
+ This extension separates the websocket Pub/Sub semantics from the Pub/Sub engine (i.e. Redis, MongoDB, etc') or the Server. If the Collection Extension is implemented, than this isn't necessarily a big step forward (the big step forward is what can be done with is).
131
+
132
+ I don't personally believe this has a chance of being adopted by other server implementors, but it's a very powerful tool that Iodine supports.
133
+
134
+ Including these semantics, in the form of the described API, allows servers and Pub/Sub engines to be optimized in different ways without distracting the application (or framework implementor with environmental details.
135
+
136
+ For example, Iodine includes two default Pub/Sub engines `Iodine::PubSub::Engine::CLUSTER` (the default) and `Iodine::PubSub::Engine::SINGLE_PROCESS` that implement a localized Pub/Sub engine within a process cluster.
137
+
138
+ The Websocket module (i.e. `Iodine::Websocket`) includes the following singleton methods:
139
+
140
+ * `Iodine::Websocket.default_pubsub=` sets the default Pub/Sub engine.
141
+
142
+ * `Iodine::Websocket.default_pubsub` gets the default Pub/Sub engine.
143
+
144
+ Websocket Callback Objects inherit the following functions:
145
+
146
+ * `#subscribe` (instance method) Subscribes to a channel using a specific "engine" (or the default engine). *Returns a subscription object (success) or `nil` (error)*. Doesn't promise actual subscription, only that the subscription request was scheduled to be sent.
147
+
148
+ Messages from the subscription are sent directly to the client unless an optional block is provided.
149
+
150
+ If an optional block is provided, it should accept the channel and message couplet (i.e. `{|channel, message| } `) and call the websocket `write` manually.
151
+
152
+ All subscriptions (including server side subscriptions) MUST be automatically canceled **by the server** when the websocket closes.
153
+
154
+ * `#subscription?` (instance method) locates an existing subscription, if any. *Returns a subscription object (success) or `nil` (error)*.
155
+
156
+ * `#unsubscribe` (instance method) cancel / stop a subscription. Safely ignores `nil` subscription objects. Returns `nil`. Performance promise is similar to `subscribe` - the event was scheduled.
157
+
158
+ Notice: there's no need to call this function during the `on_close` callback, since the server will cancel all subscriptions automatically.
159
+
160
+ * `#publish` (instance method) publishes a message to engine's specified channel. Returns `true` on success or `false` on failure (i.e., engine error). Doesn't promise actual publishing, only that the message was scheduled to be published.
161
+
162
+ For example:
163
+
164
+ ```ruby
165
+ # client side subscription
166
+ s = subscribe(channel: "channel 1") # => subscription ID?
167
+ # client side unsubscribe
168
+ unsubscribe(s)
169
+
170
+ # client side subscription forcing binary encoding for Websocket protocol.
171
+ subscribe(channel: "channel 1", encoding: :binary) # => subscription ID?
172
+
173
+ # client side pattern subscription without saving reference
174
+ subscribe(pattern: "channel [0-9]") # => subscription ID?
175
+ # client side unsubscribe
176
+ unsubscribe( subscription?(pattern: "channel [0-9]"))
177
+
178
+ # server side anonymous block subscription
179
+ s = subscribe(channel: "channel ✨") do |channel, message|
180
+ puts message
181
+ end
182
+ # server side unsubscribe
183
+ unsubscribe(s)
184
+
185
+ # server side persistent block subscription without saving reference
186
+ block = proc {|channel, message| puts message }
187
+ subscribe({pattern: "*"}, &block)
188
+ # server side unsubscribe
189
+ unsubscribe subscription?({pattern: "*"}, &block)
190
+
191
+ # server side anonymous block subscription requires reference...
192
+ subscribe(pattern: "channel 5", force: text) do |channel, message|
193
+ puts message
194
+ end
195
+ # It's impossible to locate an anonymous block subscriptions!
196
+ subscription? (pattern: "channel 5", force: text) do |channel, message|
197
+ puts message
198
+ end
199
+ # => nil # ! can't locate
200
+
201
+ ```
202
+
203
+ Servers supporting this extension aren't required to implement any Pub/Sub engines, but they MUST implement a **bridge** between the server and Pub/Sub Engines that follows the following semantics:
204
+
205
+ Engines MUST inherit from a server specific Engine class and implement the following three methods that will be called by the server when required (servers might implement an internal distribution layer and call these functions at their discretion):
206
+
207
+ * `subscribe(channel, is_pattern)`: Subscribes to the channel. The `is_pattern` flag sets the type of subscription. Returns `true` / `false`.
208
+
209
+ * `unsubscribe(channel, is_pattern)`: Unsubscribes from the channel. The `is_pattern` flag sets the type of subscription. Returns `true` / `false`.
210
+
211
+ * `publish(channel, msg, is_pattern)`: Publishes to the channel (or all channels matching the pattern). Returns `true` / `false` (some engines, such as Redis, might not support pattern publishing, they should simply return `false`).
212
+
213
+ Engines inherit the following message from the server's Engine class and call it when messages were received:
214
+
215
+ * `distribute(channel, message, is_pattern = nil)`: If `channel` is a channel pattern rather than a channel name, the `is_pattern` should be set to `true` by the Engine. Engines that don't support pattern publishing, can simply ignore this flag.