em-pg-client 0.3.3 → 0.3.4

data/HISTORY.md

@@ -1,3 +1,8 @@
+0.3.4
+
+- spec: fix em_client_on_connect and em_connection_pool tests
+- asynchronous wait_for_notify and wait_for_notify_defer with specs
+
 0.3.3
 
 - rake: console for debugging

data/README.md

@@ -72,15 +72,16 @@ Features
   processing and transactions.
 * [Sequel Adapter](https://github.com/fl00r/em-pg-sequel) by Peter Yanovich.
 * Works on windows (requires ruby 2.0) ([issue #7][Issue 7]).
-* __New__ - supports asynchronous query data processing in single row mode
+* Supports asynchronous query data processing in single row mode
   ([issue #12][Issue 12]). See {file:BENCHMARKS.md BENCHMARKING}.
+* __New__ - asynchronous implementation of wait_for_notify
 
 Requirements
 ------------
 
 * ruby >= 1.9.2 (tested: 2.1.0, 2.0.0-p353, 1.9.3-p374, 1.9.2-p320)
 * https://bitbucket.org/ged/ruby-pg >= 0.17.0
-* [PostgreSQL](http://www.postgresql.org/ftp/source/) RDBMS >= 8.3
+* [PostgreSQL](http://www.postgresql.org/ftp/source/) RDBMS >= 8.4
 * http://rubyeventmachine.com >= 1.0.0
 *  [EM-Synchrony](https://github.com/igrigorik/em-synchrony)
   (optional - not needed for any of the client functionality,
@@ -96,7 +97,7 @@ Install
 #### Gemfile
 
 ```ruby
-  gem "em-pg-client", "~> 0.3.3"
+  gem "em-pg-client", "~> 0.3.4"
 ```
 
 #### Github
@@ -261,6 +262,52 @@ Additionally the `on_autoreconnect` callback may be set on the connection.
 It's being invoked after successfull connection restart, just before the
 pending command is sent again to the server.
 
+### Server-sent notifications - async style
+
+Not surprisingly, there are two possible ways to wait for notifications,
+one with a deferrable:
+
+```ruby
+  pg = PG::EM::Client.new
+  EM.run do
+    pg.wait_for_notify_defer(7).callback do |notify|
+      if notify
+        puts "Someone spoke to us on channel: #{notify[:relname]} from #{notify[:be_pid]}"
+      else
+        puts "Too late, 7 seconds passed"
+      end
+    end.errback do |ex|
+      puts "Connection to deep space lost..."
+    end
+    pg.query_defer("LISTEN deep_space") do
+      pg.query_defer("NOTIFY deep_space") do
+        puts "Reaching out... to the other worlds"
+      end
+    end
+  end
+```
+
+and the other, using fibers:
+
+```ruby
+  EM.synchrony do
+    pg = PG::EM::Client.new
+    EM::Synchrony.next_tick do
+      pg.query('LISTEN "some channel"')
+      pg.query('SELECT pg_notify($1::text,$2::text)', ['some channel', 'with some message'])
+    end
+    pg.wait_for_notify(10) do |channel, pid, payload|
+      puts "I've got notification on #{channel} #{payload}."
+    end.tap do |name|
+      puts "Whatever, I've been waiting too long already" if name.nil?
+    end
+  end
+```
+
+As you might have noticed, one does not simply wait for notifications,
+but one can also run some queries on the same connection at the same time,
+if one wishes so.
+
 ### Connection Pool
 
 Forever alone? Not anymore! There is a dedicated {PG::EM::ConnectionPool}
@@ -360,16 +407,14 @@ The other reason was to get rid of the ugly em / em-synchrony duality.
 Bugs/Limitations
 ----------------
 
-* no async support for: COPY commands (`get_copy_data`,  `put_copy_data`),
-  `wait_for_notify`
+* no async support for COPY commands (`get_copy_data`,  `put_copy_data`)
 * actually no ActiveRecord support (you are welcome to contribute).
 
 TODO:
 -----
 
 * more convenient streaming API
-* implement EM adapted version of `get_copy_data`, `put_copy_data`,
-  `wait_for_notify` and `transaction`
+* implement EM adapted version of `get_copy_data`, `put_copy_data`
 * ORM (ActiveRecord and maybe Datamapper) support as separate projects
 
 More Info
@@ -398,13 +443,10 @@ The greetz go to:
 * Andrew Rudenko [prepor](https://github.com/prepor) for the implicit idea
   of the re-usable watcher from his [em-pg](https://github.com/prepor/em-pg).
 
-[![Bitdeli Badge][BB img]][Bitdeli Badge]
-
 [Gem Version]: https://rubygems.org/gems/em-pg-client
 [Dependency Status]: https://gemnasium.com/royaltm/ruby-em-pg-client
 [Coverage Status]: https://coveralls.io/r/royaltm/ruby-em-pg-client
 [Build Status]: https://travis-ci.org/royaltm/ruby-em-pg-client
-[Bitdeli Badge]: https://bitdeli.com/free
 [Issue 7]: https://github.com/royaltm/ruby-em-pg-client/issues/7
 [Issue 12]: https://github.com/royaltm/ruby-em-pg-client/issues/12
 [GV img]: https://badge.fury.io/rb/em-pg-client.png

data/lib/pg/em-version.rb

@@ -1,5 +1,5 @@
 module PG
   module EM
-    VERSION = '0.3.3'
+    VERSION = '0.3.4'
   end
 end

data/lib/pg/em.rb

@@ -352,7 +352,7 @@ module PG
       #
       # @see http://deveiate.org/code/pg/PG/Connection.html#method-c-new PG::Connection.new
       def self.connect_defer(*args, &blk)
-        df = PG::EM::FeaturedDeferrable.new(&blk)
+        df = FeaturedDeferrable.new(&blk)
         async_args = parse_async_options(args)
         conn = df.protect { connect_start(*args) }
         if conn
@@ -523,70 +523,77 @@ module PG
 
       # @!visibility private
       # Perform auto re-connect. Used internally.
-      def async_autoreconnect!(deferrable, error, &send_proc)
+      def async_autoreconnect!(deferrable, error, send_proc = nil, &on_connection_bad)
         # reconnect only if connection is bad and flag is set
-        if self.status == CONNECTION_BAD && async_autoreconnect
-          # check if transaction was active
-          was_in_transaction = case @last_transaction_status
-          when PQTRANS_IDLE, PQTRANS_UNKNOWN
-            false
-          else
-            true
-          end
-          # reset asynchronously
-          reset_df = reset_defer
-          # just fail on reset failure
-          reset_df.errback { |ex| deferrable.fail ex }
-          # reset succeeds
-          reset_df.callback do
-            # handle on_autoreconnect
-            if on_autoreconnect
-              # wrap in a fiber, so on_autoreconnect code may yield from it
-              Fiber.new do
-                # call on_autoreconnect handler and fail if it raises an error
-                returned_df = begin
-                  on_autoreconnect.call(self, error)
-                rescue => ex
-                  ex
-                end
-                if returned_df.respond_to?(:callback) && returned_df.respond_to?(:errback)
-                  # the handler returned a deferrable
-                  returned_df.callback do
-                    if was_in_transaction || !send_proc
-                      # there was a transaction in progress, fail anyway
-                      deferrable.fail error
-                    else
-                      # try to call failed query command again
-                      deferrable.protect(&send_proc)
+        if self.status == CONNECTION_BAD
+
+          yield if block_given?
+
+          if async_autoreconnect
+            # check if transaction was active
+            was_in_transaction = case @last_transaction_status
+            when PQTRANS_IDLE, PQTRANS_UNKNOWN
+              false
+            else
+              true
+            end
+            # reset asynchronously
+            reset_df = reset_defer
+            # just fail on reset failure
+            reset_df.errback { |ex| deferrable.fail ex }
+            # reset succeeds
+            reset_df.callback do
+              # handle on_autoreconnect
+              if on_autoreconnect
+                # wrap in a fiber, so on_autoreconnect code may yield from it
+                Fiber.new do
+                  # call on_autoreconnect handler and fail if it raises an error
+                  returned_df = begin
+                    on_autoreconnect.call(self, error)
+                  rescue => ex
+                    ex
+                  end
+                  if returned_df.respond_to?(:callback) && returned_df.respond_to?(:errback)
+                    # the handler returned a deferrable
+                    returned_df.callback do
+                      if was_in_transaction || !send_proc
+                        # fail anyway, there was a transaction in progress or in single result mode
+                        deferrable.fail error
+                      else
+                        # try to call failed query command again
+                        deferrable.protect(&send_proc)
+                      end
                     end
+                    # fail when handler's deferrable fails
+                    returned_df.errback { |ex| deferrable.fail ex }
+                  elsif returned_df.is_a?(Exception)
+                    # tha handler returned an exception object, so fail with it
+                    deferrable.fail returned_df
+                  elsif returned_df == false || !send_proc || (was_in_transaction && returned_df != true)
+                    # tha handler returned false or in single result mode
+                    # or there was an active transaction and handler didn't return true
+                    deferrable.fail error
+                  else
+                    # try to call failed query command again
+                    deferrable.protect(&send_proc)
                   end
-                  # fail when handler's deferrable fails
-                  returned_df.errback { |ex| deferrable.fail ex }
-                elsif returned_df.is_a?(Exception)
-                  # tha handler returned an exception object, so fail with it
-                  deferrable.fail returned_df
-                elsif returned_df == false || !send_proc || (was_in_transaction && returned_df != true)
-                  # tha handler returned false or raised an exception
-                  # or there was an active transaction and handler didn't return true
-                  deferrable.fail error
-                else
-                  # try to call failed query command again
-                  deferrable.protect(&send_proc)
-                end
-              end.resume
-            elsif was_in_transaction || !send_proc
-              # there was a transaction in progress, fail anyway
-              deferrable.fail error
-            else
-              # no on_autoreconnect handler, no transaction, then
-              # try to call failed query command again
-              deferrable.protect(&send_proc)
+                end.resume
+              elsif was_in_transaction || !send_proc
+                # there was a transaction in progress or in single result mode;
+                # fail anyway
+                deferrable.fail error
+              else
+                # no on_autoreconnect handler, no transaction
+                # try to call failed query command again
+                deferrable.protect(&send_proc)
+              end
             end
+            # connection is bad, reset in progress, all done
+            return
           end
-        else
-          # connection is good, or the async_autoreconnect is not set
-          deferrable.fail error
         end
+        # connection is either good or bad, the async_autoreconnect is not set
+        deferrable.fail error
       end
 
       # @!macro deferrable_api
@@ -653,7 +660,7 @@ module PG
 
         class_eval <<-EOD, __FILE__, __LINE__
         def #{defer_name}(*args, &blk)
-          df = FeaturedDeferrable.new(&blk)
+          df = FeaturedDeferrable.new
           send_proc = proc do
             #{send_name}(*args)
             setup_emio_watcher.watch_results(df, send_proc)
@@ -663,10 +670,11 @@ module PG
             @last_transaction_status = transaction_status
             send_proc.call
           rescue Error => e
-            ::EM.next_tick { async_autoreconnect!(df, e, &send_proc) }
+            ::EM.next_tick { async_autoreconnect!(df, e, send_proc) }
           rescue Exception => e
             ::EM.next_tick { df.fail(e) }
           end
+          df.completion(&blk) if block_given?
           df
         end
         EOD
@@ -678,6 +686,45 @@ module PG
       alias_method :async_exec_defer,  :exec_defer
       alias_method :exec_params_defer, :exec_defer
 
+      # Asynchronously waits for notification or until the optional
+      # +timeout+ is reached, whichever comes first. +timeout+ is
+      # measured in seconds and can be fractional.
+      # Returns immediately with a Deferrable.
+      #
+      # Pass the block to the returned deferrable's +callback+ to obtain notification
+      # hash. In case of connection error +errback+ hook is called with an error object.
+      # If the +timeout+ is reached +nil+ is passed to deferrable's +callback+.
+      # If the block is provided it's bound to both the +callback+ and +errback+ hooks
+      # of the returned deferrable.
+      # If another call is made to this method before the notification is received
+      # (or before reaching timeout) the previous deferrable's +errback+ will be called
+      # with +nil+ argument.
+      #
+      # @return [FeaturedDeferrable]
+      # @yieldparam notification [Hash|nil|Error] notification hash or a PG::Error instance on error
+      #                                       or nil when timeout is reached or canceled.
+      #
+      # @see http://deveiate.org/code/pg/PG/Connection.html#method-i-notifies PG::Connection#notifies
+      def wait_for_notify_defer(timeout = nil, &blk)
+        df = FeaturedDeferrable.new
+        begin
+          check_async_command_aborted!
+          if status == CONNECTION_OK
+            setup_emio_watcher.watch_notify(df, timeout)
+          else
+            raise_error ConnectionBad
+          end
+        rescue Error => e
+          ::EM.next_tick { async_autoreconnect!(df, e) }
+        rescue Exception => e
+          ::EM.next_tick { df.fail(e) }
+        end
+        df.completion(&blk) if block_given?
+        df
+      end
+
+      alias_method :notifies_wait_defer, :wait_for_notify_defer
+
       # Asynchronously retrieves the next result from a call to
       # #send_query (or another asynchronous command) and immediately
       # returns with a Deferrable.
@@ -692,11 +739,10 @@ module PG
       # @see http://deveiate.org/code/pg/PG/Connection.html#method-i-get_result PG::Connection#get_result
       #
       def get_result_defer(&blk)
+        df = FeaturedDeferrable.new
         begin
-          df = FeaturedDeferrable.new(&blk)
           if status == CONNECTION_OK
             if is_busy
-              check_async_command_aborted!
               setup_emio_watcher.watch_results(df, nil, true)
             else
               df.succeed blocking_get_result
@@ -709,6 +755,7 @@ module PG
         rescue Exception => e
           ::EM.next_tick { df.fail(e) }
         end
+        df.completion(&blk) if block_given?
         df
       end
 
@@ -726,10 +773,9 @@ module PG
       # @see http://deveiate.org/code/pg/PG/Connection.html#method-i-get_last_result PG::Connection#get_last_result
       #
       def get_last_result_defer(&blk)
+        df = FeaturedDeferrable.new
         begin
-          df = FeaturedDeferrable.new(&blk)
           if status == CONNECTION_OK
-            check_async_command_aborted!
             setup_emio_watcher.watch_results(df)
           else
             df.succeed
@@ -739,11 +785,13 @@ module PG
         rescue Exception => e
           ::EM.next_tick { df.fail(e) }
         end
+        df.completion(&blk) if block_given?
         df
       end
 
       # @!endgroup
 
+      alias_method :blocking_wait_for_notify, :wait_for_notify
       alias_method :blocking_get_result, :get_result
 
       def raise_error(klass=Error, message=error_message)
@@ -892,6 +940,44 @@ module PG
       alias_method :async_query, :exec
       alias_method :async_exec,  :exec
 
+      # Blocks while waiting for notification(s), or until the optional
+      # +timeout+ is reached, whichever comes first.
+      # Returns +nil+ if +timeout+ is reached, the name of the +NOTIFY+
+      # event otherwise.
+      #
+      # If EventMachine reactor is running and the current fiber isn't the
+      # root fiber this method performs command asynchronously yielding
+      # current fiber. Other fibers can process while the current one is
+      # waiting for notifications.
+      #
+      # Otherwise performs a blocking call to a parent method.
+      # @return [String|nil]
+      # @yieldparam name [String] the name of the +NOTIFY+ event
+      # @yieldparam pid [Number] the generating pid
+      # @yieldparam payload [String] the optional payload
+      # @raise [PG::Error]
+      #
+      # @see http://deveiate.org/code/pg/PG/Connection.html#method-i-wait_for_notify PG::Connection#wait_for_notify
+      def wait_for_notify(timeout = nil)
+        if ::EM.reactor_running? && !(f = Fiber.current).equal?(ROOT_FIBER)
+          unless notify_hash = notifies
+            if (notify_hash = fiber_sync wait_for_notify_defer(timeout), f).is_a?(::Exception)
+              raise notify_hash
+            end
+          end
+          if notify_hash
+            if block_given?
+              yield notify_hash.values_at(:relname, :be_pid, :extra)
+            end
+            notify_hash[:relname]
+          end
+        else
+          super
+        end
+      end
+
+      alias_method :notifies_wait, :wait_for_notify
+
       # Retrieves the next result from a call to #send_query (or another
       # asynchronous command). If no more results are available returns
       # +nil+ and the block (if given) is never called.

data/lib/pg/em/client/watcher.rb

@@ -11,6 +11,11 @@ module PG
         def initialize(client)
           @client = client
           @is_connected = true
+          @one_result_mode = false
+          @deferrable = nil
+          @notify_deferrable = nil
+          @timer = nil
+          @notify_timer = nil
         end
 
         def watching?
@@ -21,28 +26,49 @@ module PG
           @one_result_mode
         end
 
-        def watch_results(deferrable, send_proc=nil, one_result_mode=false)
+        def watch_results(deferrable, send_proc = nil, one_result_mode = false)
           @one_result_mode = one_result_mode
           @last_result = nil
           @deferrable = deferrable
           @send_proc = send_proc
           cancel_timer
-          self.notify_readable = true
+          self.notify_readable = true unless notify_readable?
           if (timeout = @client.query_timeout) > 0
-            @notify_timestamp = Time.now
+            @readable_timestamp = Time.now
             setup_timer timeout
           end
           fetch_results
         end
 
+        def watch_notify(deferrable, timeout = nil)
+          notify_df = @notify_deferrable
+          @notify_deferrable = deferrable
+          cancel_notify_timer
+          self.notify_readable = true unless notify_readable?
+          if timeout
+            @notify_timer = ::EM::Timer.new(timeout) do
+              @notify_timer = nil
+              succeed_notify
+            end
+          end
+          notify_df.fail nil if notify_df
+          check_notify
+        end
+
         def setup_timer(timeout, adjustment = 0)
           @timer = ::EM::Timer.new(timeout - adjustment) do
-            if (last_interval = Time.now - @notify_timestamp) >= timeout
+            if (last_interval = Time.now - @readable_timestamp) >= timeout
               @timer = nil
+              cancel_notify_timer
               self.notify_readable = false
               @client.async_command_aborted = true
-              @deferrable.protect do
+              @send_proc = nil
+              begin
                 @client.raise_error ConnectionBad, "query timeout expired (async)"
+              rescue Exception => e
+                fail_result e
+                # notify should also fail: query timeout is like connection error
+                fail_notify e
               end
             else
               setup_timer timeout, last_interval
@@ -50,6 +76,13 @@ module PG
           end
         end
 
+        def cancel_notify_timer
+          if @notify_timer
+            @notify_timer.cancel
+            @notify_timer = nil
+          end
+        end
+
         def cancel_timer
           if @timer
             @timer.cancel
@@ -60,9 +93,17 @@ module PG
         def notify_readable
           @client.consume_input
         rescue Exception => e
-          handle_error(e)
+          handle_error e
         else
-          fetch_results
+          fetch_results if @deferrable
+          check_notify if @notify_deferrable
+        end
+
+        def check_notify
+          if notify_hash = @client.notifies
+            cancel_notify_timer
+            succeed_notify notify_hash
+          end
         end
 
         # Carefully extract results without
@@ -84,40 +125,75 @@ module PG
             @last_result = single_result
           end
         rescue Exception => e
-          handle_error(e)
+          handle_error e
         else
           if result == false
-            @notify_timestamp = Time.now if @timer
+            @readable_timestamp = Time.now if @timer
           else
-            self.notify_readable = false
             cancel_timer
-            @send_proc = nil
-            @deferrable.succeed(result)
+            self.notify_readable = false unless @notify_deferrable
+            df = @deferrable
+            @deferrable = @send_proc = nil
+            df.succeed result
           end
         end
 
         def unbind
           @is_connected = false
-          @deferrable.protect do
-            cancel_timer
+          cancel_timer
+          cancel_notify_timer
+          if @deferrable || @notify_deferrable
             @client.raise_error ConnectionBad, "connection reset"
-          end if @deferrable
+          end
+        rescue Exception => e
+          fail_result e
+          fail_notify e
         end
 
         private
 
+        def fail_result(e)
+          df = @deferrable
+          @deferrable = nil
+          df.fail e if df
+        end
+
+        def succeed_notify(notify_hash = nil)
+          self.notify_readable = false unless @deferrable
+          df = @notify_deferrable
+          @notify_deferrable = nil
+          df.succeed notify_hash
+        end
+
+        def fail_notify(e)
+          df = @notify_deferrable
+          @notify_deferrable = nil
+          df.fail e if df
+        end
+
         def handle_error(e)
-          self.notify_readable = false
           cancel_timer
           send_proc = @send_proc
           @send_proc = nil
-          df = @deferrable
+          df = @deferrable || FeaturedDeferrable.new
           # prevent unbind error on auto re-connect
-          @deferrable = false
+          @deferrable = nil
+          notify_df = @notify_deferrable
+          self.notify_readable = false unless notify_df
           if e.is_a?(PG::Error)
-            @client.async_autoreconnect!(df, e, &send_proc)
+            @client.async_autoreconnect!(df, e, send_proc) do
+              # there was a connection error so stop any remaining activity
+              if notify_df
+                @notify_deferrable = nil
+                cancel_notify_timer
+                self.notify_readable = false
+                # fail notify_df after deferrable completes
+                # handler might setup listen again then immediately
+                df.completion { notify_df.fail e }
+              end
+            end
           else
-            df.fail(e)
+            df.fail e
           end
         end