iodine 0.4.14 → 0.4.15

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8110abd50ee8b728afb0b0bba32010636acae150
-  data.tar.gz: ccc3e4a8e25014e15bae41aadc343aa04c596586
+  metadata.gz: 8fae6d0a20c82608a04c0a8845ccc4992810ac6c
+  data.tar.gz: eec0b8600e70903a2433022c7d236c8a315f08d9
 SHA512:
-  metadata.gz: 73e41e508c1b66928402ff515890394ddf36b62bbc11c309d924306310ed72ff67d3c0e0f0562f97ee22b235a71a91d32ff2a46e2c7e839ec4f60b2f2a1431b4
-  data.tar.gz: 34415df07c8f045a4f287334e012d9678bb1ad44e8355872183ee06df01e0fcbd7a5c17bba5f8648fbf0bf86762f2c6cc67cd7b3fbe060a4318d0679f6f87a30
+  metadata.gz: c455f2313d471371eefacf262855f01c5efcffff54e46e06f966eee46cc3de1ea482a9e8547d40d7c04aa631cfd8889070fbd532d6372ea6a70f0b5b8f18d56b
+  data.tar.gz: bbf799a3051dac65311d12dbf287530b36ab873f53b256181a087235ee6f3fbe35fe52528d16364ca169105c3d3b076f3e17bedc960d238889f0395f6df4fbf0

data/CHANGELOG.md

@@ -6,7 +6,35 @@ Please notice that this change log contains changes for upcoming releases as wel
 
 ## Changes:
 
-#### Next
+#### Change log v.0.4.15
+
+**Update**: (`facil.io`) updating the facil.io library version to use the 0.5.8 released version.
+
+This includes the following changes (as well as other minor changes), as detailed in facil.io's CHANGELOG:
+
+**Fix**: (`defer`, `fiobj`) fix Linux compatibility concerns (when using GCC). Credit goes to @kotocom.
+
+**Fix**: (`defer`) fixes the non-debug version of the new defer, which didn't define some debug macros.
+
+**Updates**: minor updates to the boilerplate documentation and the "new application" creation process.
+
+**Fix**: Added `cmake_minimum_required` and related CMake fixes to the CMake file and generator. Credit to David Morán (@david-moran) for [PR #22](https://github.com/boazsegev/facil.io/pull/22) fixing the CMakelist.txt.
+
+**Compatibility**: (`websocket_parser`) removed unaligned memory access from the XOR logic in the parser, making it more compatible with older CPU systems that don't support unaligned memory access or 64 bit word lengths.
+
+**Optimization**: (`defer`) rewrote the data structure to use a hybrid cyclic buffer and linked list for the task queue (instead of a simple linked list), optimizing locality and minimizing memory allocations.
+
+**Misc**: minor updates and tweaks, such as adding the `fiobj_ary2prt` function for operations such as quick sort, updating some documentation etc'.
+
+**Fix**: (`fiobj`) fixed an where `gcc` would complain about overwriting the `fio_cstr_s` struct due to `const` members. Credit to @vit1251 for exposing this issue.
+
+**Fix**: (`fiobj`) fixed NULL pointer testing for `fiobj_free(NULL)`.
+
+**Compatibility**: (`gcc-6`) Fix some compatibility concerns with `gcc` version 6, as well as some warnings that were exposed when testing with `gcc`.
+
+**Optimization**: (`fiobj`) optimized the JSON parsing memory allocations as well as fixed some of the function declarations to add the `const` keyword where relevant.
+
+#### Change log v.0.4.14
 
 **Fix**: (`facil.io`) fixes an issue where timer monitoring would report failure when the timer exists and is being monitored.
 
@@ -92,7 +120,7 @@ Please notice that this change log contains changes for upcoming releases as wel
 
 **Fix**: fixed an issue where Websocket `ping` timeouts were being ignored for the default `Iodine::Rack` server, causing the default (40 seconds) to persist over specified valued.
 
-**Fix**: fixed a possible issue with highjacking which might cause the server to hang.
+**Fix**: fixed a possible issue with high-jacking which might cause the server to hang.
 
 ***
 
@@ -233,13 +261,13 @@ Iodine.start
 
 **Fix**: fixed another issue with `each_write` where a race condition review was performed outside the protected critical section, in some cases this would caused memory to be freed twice and crash the server. This issue is now resolved.
 
-**Deprecation**: In version 0.2.1 we have notified that the the websocket method `uuid` was deprecated in favor of `conn_id`, as suggested by the [Rack Websocket Draft](https://github.com/rack/rack/pull/1107). This deprecation is now enforced.
+**Deprecation**: In version 0.2.1 we have notified that the the Websocket method `uuid` was deprecated in favor of `conn_id`, as suggested by the [Rack Websocket Draft](https://github.com/rack/rack/pull/1107). This deprecation is now enforced.
 
 ***
 
 #### Change log v.0.2.13
 
-**Fix**: Fixed an issue presented in the C layer, where big fragmented websocket messages sent by the client could cause parsing errors and potentially, in some cases, cause a server thread to spin in a loop (DoS). Credit to @Filly for exposing the issue in the [`facil.io`](https://github.com/boazsegev/facil.io) layer. It should be noted that Chrome is the only browser where this issue could be invoked for testing.
+**Fix**: Fixed an issue presented in the C layer, where big fragmented Websocket messages sent by the client could cause parsing errors and potentially, in some cases, cause a server thread to spin in a loop (DoS). Credit to @Filly for exposing the issue in the [`facil.io`](https://github.com/boazsegev/facil.io) layer. It should be noted that Chrome is the only browser where this issue could be invoked for testing.
 
 **Credit**: credit to Elia Schito (@elia) and Augusts Bautra (@Epigene) for fixing parts of the documentation (PR #11 , #12).
 
@@ -358,7 +386,7 @@ Iodine.start
 
 **Update**: Websockets now support the `has_pending?` method and `on_ready` callback, as suggested by the [Rack Websocket Draft](https://github.com/rack/rack/pull/1107).
 
-**Update**: deprecated the websocket method `uuid` in favor of `conn_id`, as suggested by the [Rack Websocket Draft](https://github.com/rack/rack/pull/1107).
+**Update**: deprecated the Websocket method `uuid` in favor of `conn_id`, as suggested by the [Rack Websocket Draft](https://github.com/rack/rack/pull/1107).
 
 **Fix**: fixed an issue were the server would crash when attempting to send a long enough websocket message.
 
@@ -378,7 +406,7 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 **Optimization**: Minor optimizations. i.e. - creates 1 less Time object per request (The logging still creates a Time object unless disabled using `Iodine.logger = nil`).
 
-**Security**: Http/1 now reviews the Body's size as it grows (similar to Http/2), mitigating any potential attacks related to the size of the data sent.
+**Security**: HTTP/1 now reviews the Body's size as it grows (similar to HTTP/2), mitigating any potential attacks related to the size of the data sent.
 
 **Logs**: Log the number of threads utilized when starting up the server.
 
@@ -388,15 +416,15 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 **Update/Fix**: Updated the `x-forwarded-for` header recognition, to accommodate an Array formatting sometimes used (`["ip1", "ip2", ...]`).
 
-**Update**: native support for the `Forwarded` header Http.
+**Update**: native support for the `Forwarded` header HTTP.
 
-**API Changes**: `Iodine::Http.max_http_buffer` was replaced with `Iodine::Http.max_body_size`, for a better understanding of the method's result.
+**API Changes**: `Iodine::HTTP.max_http_buffer` was replaced with `Iodine::HTTP.max_body_size`, for a better understanding of the method's result.
 
 ***
 
 #### Change log v.0.1.19
 
-**Update**: added the `go_away` method to the Http/1 peorotocol, for seamless connection closeing across Http/2, Http/1 and Websockets.
+**Update**: added the `go_away` method to the HTTP/1 protocol, for seamless connection closeing across HTTP/2, HTTP/1 and Websockets.
 
 ***
 
@@ -408,11 +436,11 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 #### Change log v.0.1.17
 
-**Credit**: thanks you @frozenfoxx for going through the readme and fixing my broken grammer.
+**Credit**: thanks you @frozenfoxx for going through the readme and fixing my broken grammar.
 
 **Fix**: fixed an issue where multiple Pings might get sent when pinging takes time. Now pings are exclusive (run within their own Mutex).
 
-**Fix**: Http/2 is back... sorry about breaking it in the 0.1.16 version. When I updated the write buffer I forgot to write the status of the response, causing a protocol error related with the headers. It's now working again.
+**Fix**: HTTP/2 is back... sorry about breaking it in the 0.1.16 version. When I updated the write buffer I forgot to write the status of the response, causing a protocol error related with the headers. It's now working again.
 
 **Update**: by default and for security reasons, session id's created through a secure connection (SSL) will NOT be available on a non secure connection (SSL/TLS). However, while upgrading to the encrypted connection, the non_encrypted session storage is now available for review using the `Response#session_old` method.
 
@@ -422,13 +450,13 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 #### Change log v.0.1.16
 
-**Performance**: Http/1 and Http/2 connections now share and recycle their write buffer when while reading the response body and writing it to the IO. This (hopefuly) prevents excess `malloc` calls by the interperter.
+**Performance**: HTTP/1 and HTTP/2 connections now share and recycle their write buffer when while reading the response body and writing it to the IO. This (hopefully) prevents excess `malloc` calls by the interpreter.
 
 ***
 
 #### Change log v.0.1.15
 
-**Update**: IO reactor will now update IO status even when tasks are pending. IO will still be read only when there are no more tasks to handle, but this allows chained tasks to relate to the updated IO status. i.e. this should improve websocket availability for broadcasting (delay from connection to availability might occure until IO is registered).
+**Update**: IO reactor will now update IO status even when tasks are pending. IO will still be read only when there are no more tasks to handle, but this allows chained tasks to relate to the updated IO status. i.e. this should improve Websocket availability for broadcasting (delay from connection to availability might occur until IO is registered).
 
 **Update**: Websockets now support the `on_ping` callback, which will be called whenever a ping was sent without error.
 
@@ -450,7 +478,7 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 **Fix**: The `flash` cookie-jar will now actively prevent Symbol and String keys from overlapping.
 
-**Compatibility**: minor fixes and changes in preperation for Ruby 2.3.0. These may affect performance due to slower String initialization times.
+**Compatibility**: minor fixes and changes in preparation for Ruby 2.3.0. These may affect performance due to slower String initialization times.
 
 ***
 
@@ -458,7 +486,7 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 **Update**: Passing a hash as the cookie value will allow to set cookie parameters using the {Response#set_cookie} options. i.e.: `cookies['key']= {value: "lock", max_age: 20}`.
 
-**Security**: set the HttpOnly flag for session id cookies.
+**Security**: set the HTTPOnly flag for session id cookies.
 
 ***
 
@@ -472,7 +500,7 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 **Fix**: make sure the WebsocketClient doesn't automatically renew the connection when the connection was manually closed by the client.
 
-**Performance**: faster TimedEvent clearing when manually stopped. Minor improvements to direct big-file sending (recycle buffer to avoid malloc).
+**Performance**: faster TimedEvent clearing when manually stopped. Minor improvements to direct big-file sending (recycle buffer to avoid `malloc`).
 
 ***
 
@@ -488,17 +516,17 @@ Iodine is now written in C, as a C extension for Ruby. The little, if any, ruby
 
 #### Change log v.0.1.8
 
-**Fix**: Websocket broadcasts are now correctly executed within the IO's mutex locker. This maintains the idea that only one thread at a time should be executing code on behald of any given Protocol object ("yes" to concurrency between objects but "no" to concurrency within objects).
+**Fix**: Websocket broadcasts are now correctly executed within the IO's mutex locker. This maintains the idea that only one thread at a time should be executing code on behalf of any given Protocol object ("yes" to concurrency between objects but "no" to concurrency within objects).
 
 **Fix** fixed an issue where manually setting the number of threads for Rack applications (when using Iodine as a Rack server), the setting was mistakenly ignored.
 
-**Fix** fixed an issue where sometimes extractin the Http response's body would fail (if body is `nil`).
+**Fix** fixed an issue where sometimes extracting the HTTP response's body would fail (if body is `nil`).
 
-**Feature**: session objects are now aware of the session id. The seesion id is available by calling `response.session.id`
+**Feature**: session objects are now aware of the session id. The session id is available by calling `response.session.id`
 
-**Fix** fixed an issue where Http streaming wasn't chunk encoding after connection error handling update.
+**Fix** fixed an issue where HTTP streaming wasn't chunk encoding after connection error handling update.
 
-**Fix** fixed an issue where Http streaming would disconnect while still processing. Streaming timeout now extended to 15 seconds between response writes.
+**Fix** fixed an issue where HTTP streaming would disconnect while still processing. Streaming timeout now extended to 15 seconds between response writes.
 
 ***
 
@@ -514,7 +542,7 @@ Removed a deprecation notice for blocking API. Client API will remain blocking d
 
 **Fix**: fixed an issue where WebsocketClient wouldn't mask outgoing data, causing some servers to respond badly.
 
-**Performance**: minor performance improvements to the websocket parser, for unmasking messages.
+**Performance**: minor performance improvements to the Websocket parser, for unmasking messages.
 
 **Deprecation notice**:
 
@@ -524,7 +552,7 @@ Removed a deprecation notice for blocking API. Client API will remain blocking d
 
 #### Change log v.0.1.5
 
-**Feature**: The Response#body can now be set to a File object, allowing Iodine to preserve memory when serving large static files from disc. Limited Range requests are also supported - together, these changes allow Iodine to serve media files (such as movies) while suffering a smaller memory penalty and supporting a wider variaty of players (Safari requires Range request support for it's media player).
+**Feature**: The Response#body can now be set to a File object, allowing Iodine to preserve memory when serving large static files from disc. Limited Range requests are also supported - together, these changes allow Iodine to serve media files (such as movies) while suffering a smaller memory penalty and supporting a wider variety of players (Safari requires Range request support for it's media player).
 
 **Fix**: Fixed an issue where Iodine might take a long time to shut down after a Fatal Error during the server initialization.
 
@@ -536,33 +564,33 @@ Removed a deprecation notice for blocking API. Client API will remain blocking d
 
 **Fix**: fixed an issue where a protocol's #on_close callback wouldn't be called if the Iodine server receives a shutdown signal.
 
-**Fix**: fixed an issue where Http2 header size limit condition was not recognized by the Ruby parser (a double space issue, might be an issue with the 2.2.3 Ruby parser).
+**Fix**: fixed an issue where HTTP2 header size limit condition was not recognized by the Ruby parser (a double space issue, might be an issue with the 2.2.3 Ruby parser).
 
 ***
 
 #### Change log v.0.1.3
 
-**Fix**: fixed an issue with the new form/multipart parser, where the '+' sign would be converted to spaces on form fields (not uploaded files), causing inadvert potential change to the original POSTed data.
+**Fix**: fixed an issue with the new form/multipart parser, where the '+' sign would be converted to spaces on form fields (not uploaded files), causing in-advert potential change to the original POSTed data.
 
 ***
 
 #### Change log v.0.1.2
 
-**Fix**: fixed an issue where the default implementation of `ping` didn not reset the timeout if the connection wasn't being closed (the default implementation checks if the Protocol is working on existing data and either resets the timer allowing the work to complete or closes the connection if no work is being done).
+**Fix**: fixed an issue where the default implementation of `ping` did not reset the timeout if the connection wasn't being closed (the default implementation checks if the Protocol is working on existing data and either resets the timer allowing the work to complete or closes the connection if no work is being done).
 
 ***
 
 #### Change log v.0.1.1
 
-**Fix**: Fixed an issue where slow processing of Http/1 requests could cause timeout disconnections to occur while the request is being processed.
+**Fix**: Fixed an issue where slow processing of HTTP/1 requests could cause timeout disconnections to occur while the request is being processed.
 
 **Change/Security**: Uploads now use temporary files. Aceessing the data for file uploads should be done throught the `:file` property of the params hash (i.e. `params[:upload_field_name][:file]`). Using the `:data` property (old API) would cause the whole file to be dumped to the memory and the file's content will be returned as a String.
 
-**Change/Security**: Http upload limits are now enforced. The current default limit is about ~0.5GB.
+**Change/Security**: HTTP upload limits are now enforced. The current default limit is about ~0.5GB.
 
 **Feature**: WebsocketClient now supports both an auto-connection-renewal and a polling machanism built in to the `WebsocketClient.connect` API. The polling feature is mostly a handy helper for testing, as it is assumed that connection renewal and pub/sub offer a better design than polling.
 
-**Logging**: Better Http error logging and recognition.
+**Logging**: Better HTTP error logging and recognition.
 
 ***
 
@@ -572,17 +600,17 @@ Removed a deprecation notice for blocking API. Client API will remain blocking d
 
 We learn, we evolve, we change... but we remember our past and do our best to help with the transition and make it worth the toll it takes on our resources.
 
-I took much of the code used for GRHttp and GReactor, changed it, morphed it and united it into the singular Iodine gem. This includes Major API changes, refactoring of code, bug fixes and changes to the core approach of how a task/io based application should behave or be constructed.
+I took much of the code used for GRHTTP and GReactor, changed it, morphed it and united it into the singular Iodine gem. This includes Major API changes, refactoring of code, bug fixes and changes to the core approach of how a task/io based application should behave or be constructed.
 
 For example, Iodine kicks in automatically when the setup script is done, so that all code is run from within tasks and IO connections and no code is run in parallel to the Iodine engine.
 
 Another example, Iodine now favors Object Oriented code, so that some actions - such as writing a network service - require classes of objects to be declared or inherited (i.e. the Protocol class).
 
-This allows objects to manage their data as if they were in a single thread environment, unless the objects themselves are calling asynchronous code. For example, the Protocol class makes sure that the `on_open` and `on_message(data)` callbacks are excecuted within a Mutex (`on_close` is an exception to the rule since it is assumed that objects should be prepared to loose network connection at any moment).
+This allows objects to manage their data as if they were in a single thread environment, unless the objects themselves are calling asynchronous code. For example, the Protocol class makes sure that the `on_open` and `on_message(data)` callbacks are executed within a Mutex (`on_close` is an exception to the rule since it is assumed that objects should be prepared to loose network connection at any moment).
 
-Another example is that real-life deployemnt preferences were favored over adjustability or features. This means that some command-line arguments are automatically recognized (such as the `-p <port>` argument) and thet Iodine assumes a single web service per script/process (whereas GReactor and GRHttp allowed multiple listening sockets).
+Another example is that real-life deployment preferences were favored over adjustability or features. This means that some command-line arguments are automatically recognized (such as the `-p <port>` argument) and that Iodine assumes a single web service per script/process (whereas GReactor and GRHTTP allowed multiple listening sockets).
 
-I tested this new gem during the 0.0.x version releases, and I feel that version 0.1.0 is stable enough to work with. For instance, I left the Iodine server running all night under stress (repeatedly benchmarking it)... millions of requests later, under heavey load, a restart wasn't required and memory consumption didn't show any increase after the warmup period.
+I tested this new gem during the 0.0.x version releases, and I feel that version 0.1.0 is stable enough to work with. For instance, I left the Iodine server running all night under stress (repeatedly benchmarking it)... millions of requests later, under heavy load, a restart wasn't required and memory consumption didn't show any increase after the warmup period.
 
 
 

data/README.md

@@ -71,8 +71,10 @@ Or by adding a single line to the application. i.e. (a `config.ru` example):
 
 ```ruby
 require 'iodine'
+# static file service
 Iodine::Rack.public = '/my/public/folder'
-out = [404, {"Content-Length" => "10".freeze}.freeze, ["Not Found.".freeze].freeze].freeze
+# application
+out = [404, {"Content-Length" => "10"}, ["Not Found."]].freeze
 app = Proc.new { out }
 run app
 ```
@@ -99,8 +101,8 @@ app = proc do |env|
   elsif request.path_info == '/file'.freeze
     [200, { 'X-Header' => 'This was a Rack::Sendfile response sent as text.' }, File.open(__FILE__)]
   else
-    [200, { 'Content-Type'.freeze => 'text/html'.freeze,
-            'Content-Length'.freeze => request.path_info.length.to_s },
+    [200, { 'Content-Type' => 'text/html',
+            'Content-Length' => request.path_info.length.to_s },
      [request.path_info]]
  end
 end
@@ -276,7 +278,7 @@ def run_server
 end
 ```
 
-In pure Ruby (without using C extensions or Java), it's possible to do the same by using `select`... and although `select` has some issues, it works well for smaller concurrency levels.
+In pure Ruby (without using C extensions or Java), it's possible to do the same by using `select`... and although `select` has some issues, it works well for lighter loads.
 
 The server events are fairly fast and fragmented (longer code is fragmented across multiple events), so one thread is enough to run the server including it's static file service and everything... 
 
@@ -286,7 +288,7 @@ The thread pool is there to help slow user code.
 
 It's very common that the application's code will run slower and require external resources (i.e., databases, a custom pub/sub service, etc'). This slow code could "starve" the server, which is patiently waiting to run it's tasks on the same thread.
 
-The slower your application code, the more threads you will need to keep the server running smoothly.
+The slower your application code, the more threads you will need to keep the server running in a responsive manner (note that responsiveness and speed aren't always the same).
 
 ### How does it compare to other servers?
 

data/ext/iodine/base64.c

@@ -123,6 +123,10 @@ int bscrypt_base64_decode(char *target, char *encoded, int base64_len) {
     base64_len--;
     encoded++;
   }
+  while (encoded[base64_len - 1] == '\r' || encoded[base64_len - 1] == '\n' ||
+         encoded[base64_len - 1] == ' ' || encoded[base64_len - 1] == 0) {
+    base64_len--;
+  }
   while (base64_len >= 4) {
     tmp1 = *(encoded++);
     tmp2 = *(encoded++);
@@ -176,6 +180,14 @@ int bscrypt_base64_decode(char *target, char *encoded, int base64_len) {
     written += 3;
     break;
   }
+  if (encoded[-1] == '=') {
+    target--;
+    written--;
+    if (encoded[-2] == '=') {
+      target--;
+      written--;
+    }
+  }
   *target = 0;
   return written;
 }

data/ext/iodine/defer.c

@@ -19,12 +19,22 @@ Feel free to copy, use and enjoy according to the license provided.
 Compile time settings
 ***************************************************************************** */
 
-#ifndef DEFER_QUEUE_BUFFER
-#define DEFER_QUEUE_BUFFER 4096
-#endif
 #ifndef DEFER_THROTTLE
 #define DEFER_THROTTLE 524287UL
 #endif
+#ifndef DEFER_THROTTLE_LIMIT
+#define DEFER_THROTTLE_LIMIT 1572864UL
+#endif
+
+#ifndef DEFER_QUEUE_BLOCK_COUNT
+#if UINTPTR_MAX <= 0xFFFFFFFF
+/* Almost a page of memory on most 32 bit machines: ((4096/4)-4)/3 */
+#define DEFER_QUEUE_BLOCK_COUNT 340
+#else
+/* Almost a page of memory on most 64 bit machines: ((4096/8)-4)/3 */
+#define DEFER_QUEUE_BLOCK_COUNT 168
+#endif
+#endif
 
 /* *****************************************************************************
 Data Structures
@@ -37,32 +47,142 @@ typedef struct {
   void *arg2;
 } task_s;
 
-/* a single linked list for tasks */
-typedef struct task_node_s {
-  task_s task;
-  struct task_node_s *next;
-} task_node_s;
+/* task queue block */
+typedef struct queue_block_s {
+  task_s tasks[DEFER_QUEUE_BLOCK_COUNT];
+  struct queue_block_s *next;
+  size_t write;
+  size_t read;
+  unsigned char state;
+} queue_block_s;
 
-/* static memory allocation for a task node buffer */
-static task_node_s tasks_buffer[DEFER_QUEUE_BUFFER];
+static queue_block_s static_queue;
 
 /* the state machine - this holds all the data about the task queue and pool */
 static struct {
-  /* the next task to be performed */
-  task_node_s *first;
-  /* a pointer to the linked list's tail, where the next task will be stored */
-  task_node_s **last;
-  /* a linked list for task nodes (staticly allocated) */
-  task_node_s *pool;
   /* a lock for the state machine, used for multi-threading support */
   spn_lock_i lock;
-  /* a flag indicating whether the task pool was initialized */
-  unsigned char initialized;
-} deferred = {.first = NULL,
-              .last = &deferred.first,
-              .pool = NULL,
-              .lock = 0,
-              .initialized = 0};
+  /* current active block to pop tasks */
+  queue_block_s *reader;
+  /* current active block to push tasks */
+  queue_block_s *writer;
+} deferred = {.reader = &static_queue, .writer = &static_queue};
+
+/* *****************************************************************************
+Internal Data API
+***************************************************************************** */
+
+#if DEBUG
+static size_t count_alloc, count_dealloc;
+#define COUNT_ALLOC spn_add(&count_alloc, 1)
+#define COUNT_DEALLOC spn_add(&count_dealloc, 1)
+#define COUNT_RESET                                                            \
+  do {                                                                         \
+    count_alloc = count_dealloc = 0;                                           \
+  } while (0)
+#else
+#define COUNT_ALLOC
+#define COUNT_DEALLOC
+#define COUNT_RESET
+#endif
+
+static inline void push_task(task_s task) {
+  spn_lock(&deferred.lock);
+
+  /* test if full */
+  if (deferred.writer->state &&
+      deferred.writer->write == deferred.writer->read) {
+    /* return to static buffer or allocate new buffer */
+    if (static_queue.state == 2) {
+      deferred.writer->next = &static_queue;
+    } else {
+      deferred.writer->next = malloc(sizeof(*deferred.writer->next));
+      COUNT_ALLOC;
+      if (!deferred.writer->next)
+        goto critical_error;
+    }
+    deferred.writer = deferred.writer->next;
+    deferred.writer->write = 0;
+    deferred.writer->read = 0;
+    deferred.writer->state = 0;
+    deferred.writer->next = NULL;
+  }
+
+  /* place task and finish */
+  deferred.writer->tasks[deferred.writer->write++] = task;
+  /* cycle buffer */
+  if (deferred.writer->write == DEFER_QUEUE_BLOCK_COUNT) {
+    deferred.writer->write = 0;
+    deferred.writer->state = 1;
+  }
+  spn_unlock(&deferred.lock);
+  return;
+
+critical_error:
+  spn_unlock(&deferred.lock);
+  perror("ERROR CRITICAL: defer can't allocate task");
+  kill(0, SIGINT);
+  exit(errno);
+}
+
+static inline task_s pop_task(void) {
+  task_s ret = (task_s){NULL};
+  queue_block_s *to_free = NULL;
+  /* lock the state machine, to grab/create a task and place it at the tail
+  */ spn_lock(&deferred.lock);
+
+  /* empty? */
+  if (deferred.reader->write == deferred.reader->read &&
+      !deferred.reader->state)
+    goto finish;
+  /* collect task */
+  ret = deferred.reader->tasks[deferred.reader->read++];
+  /* cycle */
+  if (deferred.reader->read == DEFER_QUEUE_BLOCK_COUNT) {
+    deferred.reader->read = 0;
+    deferred.reader->state = 0;
+  }
+  /* did we finish the queue in the buffer? */
+  if (deferred.reader->write == deferred.reader->read) {
+    if (deferred.reader->next) {
+      to_free = deferred.reader;
+      deferred.reader = deferred.reader->next;
+    } else {
+      deferred.reader->write = deferred.reader->read = deferred.reader->state =
+          0;
+    }
+    goto finish;
+  }
+
+finish:
+  if (to_free == &static_queue) {
+    static_queue.state = 2;
+  }
+  spn_unlock(&deferred.lock);
+
+  if (to_free && to_free != &static_queue) {
+    free(to_free);
+    COUNT_DEALLOC;
+  }
+  return ret;
+}
+
+static inline void clear_tasks(void) {
+  spn_lock(&deferred.lock);
+  while (deferred.reader) {
+    queue_block_s *tmp = deferred.reader;
+    deferred.reader = deferred.reader->next;
+    if (tmp != &static_queue) {
+      COUNT_DEALLOC;
+      free(tmp);
+    }
+  }
+  static_queue = (queue_block_s){.next = NULL};
+  deferred.reader = deferred.writer = &static_queue;
+  spn_unlock(&deferred.lock);
+}
+
+#define push_task(...) push_task((task_s){__VA_ARGS__})
 
 /* *****************************************************************************
 API
@@ -73,79 +193,29 @@ int defer(void (*func)(void *, void *), void *arg1, void *arg2) {
   /* must have a task to defer */
   if (!func)
     goto call_error;
-  task_node_s *task;
-  /* lock the state machine, to grab/create a task and place it at the tail */
-  spn_lock(&deferred.lock);
-  if (deferred.pool) {
-    task = deferred.pool;
-    deferred.pool = deferred.pool->next;
-  } else if (deferred.initialized) {
-    task = malloc(sizeof(task_node_s));
-    if (!task)
-      goto error;
-  } else
-    goto initialize;
-
-schedule:
-  *deferred.last = task;
-  deferred.last = &task->next;
-  task->task.func = func;
-  task->task.arg1 = arg1;
-  task->task.arg2 = arg2;
-  task->next = NULL;
-  spn_unlock(&deferred.lock);
+  push_task(.func = func, .arg1 = arg1, .arg2 = arg2);
   return 0;
 
-error:
-  spn_unlock(&deferred.lock);
-  perror("ERROR CRITICAL: defer can't allocate task");
-  kill(0, SIGINT), exit(errno);
-
 call_error:
   return -1;
-
-initialize:
-  /* initialize the task pool using all the items in the static buffer */
-  /* also assign `task` one of the tasks from the pool and schedule the task */
-  deferred.initialized = 1;
-  task = tasks_buffer;
-  deferred.pool = tasks_buffer + 1;
-  for (size_t i = 1; i < (DEFER_QUEUE_BUFFER - 1); i++) {
-    tasks_buffer[i].next = &tasks_buffer[i + 1];
-  }
-  tasks_buffer[DEFER_QUEUE_BUFFER - 1].next = NULL;
-  goto schedule;
 }
 
 /** Performs all deferred functions until the queue had been depleted. */
 void defer_perform(void) {
-  task_node_s *tmp;
-  task_s task;
-restart:
-  spn_lock(&deferred.lock); /* remember never to perform tasks within a lock! */
-  tmp = deferred.first;
-  if (tmp) {
-    deferred.first = tmp->next;
-    if (!deferred.first)
-      deferred.last = &deferred.first;
-    task = tmp->task;
-    if (tmp >= tasks_buffer && tmp < tasks_buffer + DEFER_QUEUE_BUFFER) {
-      tmp->next = deferred.pool;
-      deferred.pool = tmp;
-    } else {
-      free(tmp);
-    }
-    spn_unlock(&deferred.lock);
-    /* perform the task outside the lock. */
+  task_s task = pop_task();
+  while (task.func) {
     task.func(task.arg1, task.arg2);
-    /* I used `goto` to optimize assembly instruction flow. maybe it helps. */
-    goto restart;
+    task = pop_task();
   }
-  spn_unlock(&deferred.lock);
 }
 
-/** returns true if there are deferred functions waiting for execution. */
-int defer_has_queue(void) { return deferred.first != NULL; }
+/** Returns true if there are deferred functions waiting for execution. */
+int defer_has_queue(void) {
+  return deferred.reader->read != deferred.reader->write;
+}
+
+/** Clears the queue. */
+void defer_clear_queue(void) { clear_tasks(); }
 
 /* *****************************************************************************
 Thread Pool Support
@@ -172,7 +242,7 @@ error:
 int defer_join_thread(void *p_thr) {
   if (!p_thr)
     return -1;
-  pthread_join(*(pthread_t *)p_thr, NULL);
+  pthread_join(*((pthread_t *)p_thr), NULL);
   free(p_thr);
   return 0;
 }
@@ -214,8 +284,8 @@ static void *defer_worker_thread(void *pool_) {
   signal(SIGPIPE, SIG_IGN);
   /* the throttle replaces conditional variables for better performance */
   size_t throttle = (pool->count) * DEFER_THROTTLE;
-  if (!throttle || throttle > 1572864UL)
-    throttle = 1572864UL;
+  if (!throttle || throttle > DEFER_THROTTLE_LIMIT)
+    throttle = DEFER_THROTTLE_LIMIT;
   /* perform any available tasks */
   defer_perform();
   /* as long as the flag is true, wait for and perform tasks. */
@@ -304,6 +374,7 @@ void reap_child_handler(int sig) {
   errno = old_errno;
 }
 
+#if !defined(NO_CHILD_REAPER) || NO_CHILD_REAPER == 0
 /* initializes zombie reaping for the process */
 inline static void reap_children(void) {
   struct sigaction sa;
@@ -315,6 +386,7 @@ inline static void reap_children(void) {
     kill(0, SIGINT), exit(errno);
   }
 }
+#endif
 
 /* a global process identifier (0 == root) */
 static int defer_fork_pid_id = 0;
@@ -443,6 +515,16 @@ static void sample_task(void *unused, void *unused2) {
   spn_unlock(&i_lock);
 }
 
+static void single_counter_task(void *unused, void *unused2) {
+  (void)(unused);
+  (void)(unused2);
+  spn_lock(&i_lock);
+  i_count++;
+  spn_unlock(&i_lock);
+  if (i_count < (1024 * 1024))
+    defer(single_counter_task, NULL, NULL);
+}
+
 static void sched_sample_task(void *unused, void *unused2) {
   (void)(unused);
   (void)(unused2);
@@ -490,10 +572,27 @@ void defer_test(void) {
   }
   end = clock();
   fprintf(stderr,
-          "Deferless (direct call) counter: %lu cycles with i_count = %lu\n",
-          end - start, i_count);
+          "Deferless (direct call) counter: %lu cycles with i_count = %lu, "
+          "%lu/%lu free/malloc\n",
+          end - start, i_count, count_dealloc, count_alloc);
 
   spn_lock(&i_lock);
+  COUNT_RESET;
+  i_count = 0;
+  spn_unlock(&i_lock);
+  start = clock();
+  defer(single_counter_task, NULL, NULL);
+  defer(single_counter_task, NULL, NULL);
+  defer_perform();
+  end = clock();
+  fprintf(stderr,
+          "Defer single thread, two tasks: "
+          "%lu cycles with i_count = %lu, %lu/%lu "
+          "free/malloc\n",
+          end - start, i_count, count_dealloc, count_alloc);
+
+  spn_lock(&i_lock);
+  COUNT_RESET;
   i_count = 0;
   spn_unlock(&i_lock);
   start = clock();
@@ -502,10 +601,13 @@ void defer_test(void) {
   }
   defer_perform();
   end = clock();
-  fprintf(stderr, "Defer single thread: %lu cycles with i_count = %lu\n",
-          end - start, i_count);
+  fprintf(stderr,
+          "Defer single thread: %lu cycles with i_count = %lu, %lu/%lu "
+          "free/malloc\n",
+          end - start, i_count, count_dealloc, count_alloc);
 
   spn_lock(&i_lock);
+  COUNT_RESET;
   i_count = 0;
   spn_unlock(&i_lock);
   start = clock();
@@ -519,12 +621,15 @@ void defer_test(void) {
     defer_pool_wait(pool);
     end = clock();
     fprintf(stderr,
-            "Defer multi-thread (%d threads): %lu cycles with i_count = %lu\n",
-            DEFER_TEST_THREAD_COUNT, end - start, i_count);
+            "Defer multi-thread (%d threads): %lu cycles with i_count = %lu, "
+            "%lu/%lu free/malloc\n",
+            DEFER_TEST_THREAD_COUNT, end - start, i_count, count_dealloc,
+            count_alloc);
   } else
     fprintf(stderr, "Defer multi-thread: FAILED!\n");
 
   spn_lock(&i_lock);
+  COUNT_RESET;
   i_count = 0;
   spn_unlock(&i_lock);
   start = clock();
@@ -533,31 +638,29 @@ void defer_test(void) {
   }
   defer_perform();
   end = clock();
-  fprintf(stderr, "Defer single thread (2): %lu cycles with i_count = %lu\n",
-          end - start, i_count);
+  fprintf(stderr,
+          "Defer single thread (2): %lu cycles with i_count = %lu, %lu/%lu "
+          "free/malloc\n",
+          end - start, i_count, count_dealloc, count_alloc);
 
   fprintf(stderr, "calling defer_perform.\n");
   defer(text_task, NULL, NULL);
   defer_perform();
-  fprintf(stderr, "defer_perform returned. i_count = %lu\n", i_count);
-  size_t pool_count = 0;
-  task_node_s *pos = deferred.pool;
-  while (pos) {
-    pool_count++;
-    pos = pos->next;
-  }
-  fprintf(stderr, "defer pool count %lu/%d (%s)\n", pool_count,
-          DEFER_QUEUE_BUFFER,
-          pool_count == DEFER_QUEUE_BUFFER ? "pass" : "FAILED");
+  fprintf(stderr,
+          "defer_perform returned. i_count = %lu, %lu/%lu free/malloc\n",
+          i_count, count_dealloc, count_alloc);
+
   fprintf(stderr, "press ^C to finish PID test\n");
   defer(pid_task, "pid test", NULL);
   if (defer_perform_in_fork(4, 64) > 0) {
     fprintf(stderr, "* %d finished\n", getpid());
     exit(0);
   };
-  fprintf(stderr,
-          "   === Defer pool memory footprint %lu X %d = %lu bytes ===\n",
-          sizeof(task_node_s), DEFER_QUEUE_BUFFER, sizeof(tasks_buffer));
+  fprintf(stderr, "* Defer queue %lu/%lu free/malloc\n", count_dealloc,
+          count_alloc);
+  defer_clear_queue();
+  fprintf(stderr, "* Defer queue %lu/%lu free/malloc\n", count_dealloc,
+          count_alloc);
 }
 
 #endif