dispatch_queue_rb 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: e177e6f49e44b8c2e8aa0e34ffdb8acf2dc2a555
+  data.tar.gz: ce6d7aed534ffe9ccb29f604ea28358b58c70cbb
+SHA512:
+  metadata.gz: feb77bdb4577eb050c0c0e9544b3bcba7eeda6774638a4bfe1beb7e8dd04b4a30a778310b0bb2d7b085a178d4623d7e38948b88ffe156c157172155d69758cfc
+  data.tar.gz: f2687609268da36638c6aabbdea5dcbda827dc82559701774b550881ee6e3dec2f33f1bbb175e29641e7016ce0f42c52285d30c4b47db7904f667c24b78bd0aa

data/Gemfile

@@ -0,0 +1,13 @@
+# =============================================================================
+#
+# MODULE      : Gemfile
+# PROJECT     : DispatchQueueRb
+# DESCRIPTION :
+#
+# Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+# =============================================================================
+
+
+
+source 'https://rubygems.org'
+gemspec

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,98 @@
+# DispatchQueueRb
+
+[![Gem Version](https://badge.fury.io/rb/dispatch_queue_rb.svg)](https://badge.fury.io/rb/dispatch_queue_rb)
+[![Build Status](https://travis-ci.org/marcus999/dispatch_queue_rb.svg?branch=master)](https://travis-ci.org/marcus999/dispatch_queue_rb)
+[![Code Climate](https://codeclimate.com/github/marcus999/dispatch_queue_rb/badges/gpa.svg)](https://codeclimate.com/github/marcus999/dispatch_queue_rb)
+[![Issue Count](https://codeclimate.com/github/marcus999/dispatch_queue_rb/badges/issue_count.svg)](https://codeclimate.com/github/marcus999/dispatch_queue_rb)
+
+
+
+## Overview
+
+DispatchQueueRb is a pure ruby implementation of the Grand Central Dispatch
+concurrency primitives, using Ruby threads and blocking synchronization primitives
+like Mutex and ConditionVariable.
+
+It implements serial and concurrent queues, with synchronous, asynchronous,
+barrier and delayed dispatch methods. All queues dispatch methods support an
+optional dispatch groups to synchronize on completion of a group of tasks.
+It also provides a thread pool based concurrent queue, scaled to the number
+of available cpu cores, and used by default to schedule the actual work.
+
+Beside a highly optimized lock-free C implementation with libdispatch,
+Grand Central Dispatch is a shift of paradigm that expresses concurrent
+programming concepts in terms of tasks that must be serialized with respect to
+each other and tasks that can be performed concurrently.
+
+Using threads, concurrency is usually defining with a fixed set of threads that
+perform specific tasks, and by passing data between threads using message queues
+and producer / consumer patterns. The flow of data and the sequence of
+operations is often very inflexible because what threads do and where they
+write their result to is often frozen at design time.
+
+With dispatch queues, concurrency is expressed by scheduling work items to
+serial and concurrent queues. The work that needs to be performed in the
+context of a queue is not frozen in a dedicated thread, but passed as a block
+of code to the dispatch method call. That block of code captures the data it
+needs to execute, knows how to access global immutable data, and defines what
+to do with the result.
+
+
+### Implements:
+  - SerialQueue and ConcurrentQueue
+  - dispatch_async() and dispatch_sync()
+  - dispatch_barrier_async() and dispatch_barrier_sync()
+  - dispatch_after()
+  - Dispatch.main_queue is a global serial queue attached to a single thread
+    (but not the main thread).
+  - Dispatch.default_queue is a global concurrent queue that is implemented as
+    a thread pool, scaled to the number of available cpu cores. It is the
+    default parent_queue for all private queues.
+
+### Key differences and not supported features:
+  - Implemented with Ruby threading primitives (Mutex, ConditionVariable) instead
+    of high-performance lock-free algorithms
+  - Aimed at MRI ruby 2.x, where ruby code cannot execute concurrently across
+    multiple cpu cores. Mostly useful to manage parallel sub-processes execution.
+  - Dispatch.default_queue does not monitor thread activity and does not spawn
+    more threads to compensate for blocked threads.
+
+### Version 1.0 limitations:
+  - Does not implement an equivalent for dispatch_source primitives.
+  - Does not support suspend / resume operations
+  - Does not provide multiple priority levels for global queues
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'dispatch_queue_rb'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install dispatch_queue_rb
+
+## Usage
+
+TODO: Write usage instructions here
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release` to create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+1. Fork it ( https://github.com/[my-github-username]/dispatch_queue_rb/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/dispatch_queue_rb.gemspec

@@ -0,0 +1,43 @@
+# =============================================================================
+#
+# MODULE      : dispatch_queue_rb.gemspec
+# PROJECT     : DispatchQueueRb
+# DESCRIPTION :
+#
+# Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+# =============================================================================
+
+
+require_relative 'lib/dispatch_queue_rb/version.rb'
+
+Gem::Specification.new do |spec|
+  spec.name         = 'dispatch_queue_rb'
+  spec.version      = DispatchQueue::VERSION
+  spec.authors      = ["Marc-Antoine Argenton"]
+  spec.email        = ["maargenton.dev@gmail.com"]
+  spec.summary      = "Pure ruby implementation of Grand Central Dispatch concurrency primitives."
+  spec.description  = %q{
+    DispatchQueueRb is a pure ruby implementation of Grand Central Dispatch concurrency primitives.
+    It implements serial and concurrent queues, with synchronous, asynchronous,
+    barrier and delayed dispatch methods. All queues dispatch methods support an
+    optional dispatch groups to synchronize on completion of a group of tasks.
+    It also provides a thread pool based concurrent queue, scaled to the number
+    of available cpu cores, and used by default to schedule the actual work.
+                      }.gsub( /\s+/, ' ').strip
+  spec.homepage     = "https://github.com/marcus999/dispatch_queue_rb"
+
+  spec.files        = Dir['[A-Z]*', 'rakefile.rb', '*.gemspec'].reject { |f| f =~ /.lock/ }
+  spec.files        += Dir['bin/**', 'lib/**/*.rb', 'test/**/*.rb', 'spec/**/*.rb', 'features/**/*.rb']
+  spec.executables  = spec.files.grep( %r{^bin/} ) { |f| File.basename(f) }
+  spec.test_files   = spec.files.grep( %r{^(test|spec|features)/} )
+
+  # spec.add_runtime_dependency 'facets', '~> 3.0'
+  # spec.add_runtime_dependency 'mustache', '~> 1.0'
+
+  spec.add_development_dependency 'bundler', '~> 1.7'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'watch', '~> 0.1'
+  spec.add_development_dependency 'rr', '~> 1.1'
+  spec.add_development_dependency 'minitest', '~> 5.3'
+  spec.add_development_dependency 'minitest-reporters', '~> 1.1'
+end

data/lib/dispatch_queue_rb.rb

@@ -0,0 +1,32 @@
+# =============================================================================
+#
+# MODULE      : lib/dispatch_queue_rb.rb
+# PROJECT     : DispatchQueueRb
+# DESCRIPTION :
+#
+# Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+# =============================================================================
+
+require 'set'
+
+require_relative 'dispatch_queue_rb/version.rb'
+
+require_relative 'dispatch_queue_rb/mixins/dispatch_sync_impl.rb'
+require_relative 'dispatch_queue_rb/mixins/dispatch_after_impl.rb'
+
+require_relative 'dispatch_queue_rb/internal/condition_variable_pool.rb'
+require_relative 'dispatch_queue_rb/internal/heap.rb'
+require_relative 'dispatch_queue_rb/internal/continuation.rb'
+require_relative 'dispatch_queue_rb/internal/thread_pool_queue.rb'
+require_relative 'dispatch_queue_rb/internal/timer_pool.rb'
+require_relative 'dispatch_queue_rb/internal/thread_queue.rb'
+
+require_relative 'dispatch_queue_rb/dispatch.rb'
+require_relative 'dispatch_queue_rb/serial_queue.rb'
+require_relative 'dispatch_queue_rb/concurrent_queue.rb'
+require_relative 'dispatch_queue_rb/dispatch_group.rb'
+
+Dispatch = DispatchQueue::Dispatch
+SerialQueue = DispatchQueue::SerialQueue
+ConcurrentQueue = DispatchQueue::ConcurrentQueue
+DispatchGroup = DispatchQueue::DispatchGroup

data/lib/dispatch_queue_rb/concurrent_queue.rb

@@ -0,0 +1,116 @@
+# =============================================================================
+#
+# MODULE      : lib/dispatch_queue_rb/concurrent_queue.rb
+# PROJECT     : DispatchQueue
+# DESCRIPTION :
+#
+# Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+# =============================================================================
+
+module DispatchQueue
+  class ConcurrentQueue
+
+    def initialize( parent_queue: nil )
+      @mutex = Mutex.new
+      @condition = ConditionVariable.new
+      @task_list = []
+      @parent_queue = parent_queue || Dispatch.default_queue
+      @scheduled_count = 0
+      @barrier_count = 0
+    end
+
+    def dispatch_async( group:nil, &task )
+      group.enter() if group
+      continuation = Continuation.new( target_queue:@parent_queue, group:group ) do
+        _run_task( task, false )
+      end
+
+      schedule_immediately = @mutex.synchronize do
+        if ( @barrier_count > 0)
+          @task_list << continuation
+          false
+        else
+          @scheduled_count += 1
+          true
+        end
+      end
+
+      continuation.run() if schedule_immediately
+      self
+    end
+
+    def dispatch_barrier_async( group:nil, &task )
+      group.enter() if group
+      continuation = Continuation.new( target_queue:@parent_queue, group:group, barrier:true ) do
+        _run_task( task, true )
+      end
+
+      barrier_task, tasks = @mutex.synchronize do
+        @barrier_count += 1
+        @task_list << continuation
+        resume_pending = ( @scheduled_count == 0 && @barrier_count == 1)
+        _sync_get_next_batch() if resume_pending
+      end
+
+      _schedule_next_batch( barrier_task, tasks )
+    end
+
+    include DispatchSyncImpl
+    include DispatchAfterImpl
+
+
+    # def _debug_trace_queue_state( prefix = "" )
+    #   puts "%-35s | scheduled: %3d, barrier: %3d, queued: %3d, barrier_head: %-5s" % [
+    #     prefix,
+    #     @scheduled_count,
+    #     @barrier_count,
+    #     @task_list.count,
+    #     !@task_list.empty? && @task_list.first[1],
+    #   ]
+    # end
+
+  private
+    def _run_task( task, barrier )
+      previous_queue = Thread.current[:current_queue]
+      Thread.current[:current_queue] = self
+
+      begin
+        task.call()
+      ensure
+        Thread.current[:current_queue] = previous_queue
+        _task_comleted( barrier )
+      end
+    end
+
+    def _task_comleted( barrier = false )
+      barrier_task, tasks = @mutex.synchronize do
+        resume_pending = barrier || ((@scheduled_count -= 1) == 0)
+        @barrier_count -= 1 if barrier
+        _sync_get_next_batch() if resume_pending
+      end
+
+      _schedule_next_batch( barrier_task, tasks )
+    end
+
+    def _sync_get_next_batch()
+      return nil if @task_list.empty?
+      return @task_list.shift if @task_list.first.barrier
+
+      tasks = []
+      tasks << @task_list.shift while !@task_list.first.barrier
+      @scheduled_count += tasks.count
+      return [nil, tasks]
+    end
+
+    def _schedule_next_batch( barrier_task, tasks )
+      if barrier_task
+        barrier_task.run()
+      elsif tasks
+        tasks.each do |t|
+          t.run()
+        end
+      end
+    end
+
+  end # class ConcurrentQueue
+end # module DispatchQueue

data/lib/dispatch_queue_rb/dispatch.rb

@@ -0,0 +1,66 @@
+# =============================================================================
+#
+# MODULE      : lib/dispatch_queue_rb/dispatch.rb
+# PROJECT     : DispatchQueue
+# DESCRIPTION :
+#
+# Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+# =============================================================================
+
+module DispatchQueue
+  module Dispatch
+
+    Result = Struct.new( :value )
+
+    class << self
+      def ncpu()
+        @@ncpu ||= `sysctl -n hw.ncpu`.to_i rescue 1
+      end
+
+      def default_queue
+        @@default_queue
+      end
+
+      def main_queue
+        @@main_queue
+      end
+
+      def synchronize()
+        mutex, condition = ConditionVariablePool.acquire()
+        result = nil
+        result_handler = Proc.new { |r|
+          result = r;
+          mutex.synchronize { condition.signal() }
+        }
+        mutex.synchronize do
+          yield result_handler
+          condition.wait( mutex )
+        end
+        ConditionVariablePool.release( mutex, condition )
+        result
+      end
+
+      def concurrent_map( input_array, target_queue:nil, &task )
+        group = DispatchGroup.new
+        target_queue ||= default_queue
+
+        output_results = input_array.map do |e|
+          result = Result.new
+          target_queue.dispatch_async( group:group ) do
+            result.value = task.call( e )
+          end
+          result
+        end
+
+        group.wait()
+        output_results.map { |result| result.value }
+      end
+
+
+    private
+      @@default_queue = ThreadPoolQueue.new()
+      @@main_queue = ThreadQueue.new()
+
+    end # class << self
+  end # class Dispatch
+end # module DispatchQueue

data/lib/dispatch_queue_rb/dispatch_group.rb

@@ -0,0 +1,72 @@
+# =============================================================================
+#
+# MODULE      : lib/dispatch_queue_rb/dispatch_group.rb
+# PROJECT     : DispatchQueue
+# DESCRIPTION :
+#
+# Copyright (c) 2016, Marc-Antoine Argenton.  All rights reserved.
+# =============================================================================
+
+module DispatchQueue
+  class DispatchGroup
+    def initialize()
+      @mutex = Mutex.new
+      @condition = ConditionVariable.new
+      @count = 0
+      @notify_list = []
+    end
+
+    def enter()
+      @mutex.synchronize { @count += 1 }
+      self
+    end
+
+    def leave()
+      notify_list = @mutex.synchronize do
+        @count -= 1
+        raise "Unbalanced calls to DispatchGroup.enter() / .leave()" if @count < 0
+        if @count == 0
+          @condition.broadcast()
+          _sync_swap_notify_list()
+        end
+      end
+
+      _schedule_notify_list( notify_list ) if notify_list
+      self
+    end
+
+    def notify( target_queue:nil, barrier:false, group:nil, &task )
+      continuation = Continuation.new( target_queue:target_queue,
+                                       barrier:barrier, group:group, &task )
+      @mutex.synchronize do
+        if @count == 0
+          continuation.run( default_target_queue:Dispatch.default_queue )
+        else
+          @notify_list << continuation
+        end
+      end
+      self
+    end
+
+    def wait( timeout:nil )
+      @mutex.synchronize do
+        return true if @count == 0
+        @condition.wait( @mutex, timeout )
+        return @count == 0
+      end
+    end
+
+  private
+    def _sync_swap_notify_list()
+      return nil if @notify_list.empty?
+      notify_list = @notify_list
+      @notify_list = []
+      return notify_list
+    end
+
+    def _schedule_notify_list( notify_list )
+      notify_list.each { |continuation| continuation.run() }
+    end
+
+  end # class DispatchGroup
+end # module DispatchQueue