em-pg-client 0.1.1 → 0.2.0.pre.1

data/HISTORY.rdoc

@@ -1,9 +1,16 @@
-0.1.1
-- added on_reconnect callback
-- docs updated
-- added development dependency for eventmachine >= 1.0.0.beta.1
-- added separate client specs for eventmachine = 0.12.10
-- added error checking to eventmachine specs
-
-0.1.0
-- first release
+next release
+- added connect_timeout property with for async connect/reset
+- fix: async_autoreconnect for tcp/ip connections
+- fix: async_* does not raise errors; errors handled by deferrable
+- rework async_autoreconnect in fully async manner
+- added async_connect() and #async_reset()
+- API change: on_reconnect -> on_autoreconnect
+0.1.1
+- added on_reconnect callback
+- docs updated
+- added development dependency for eventmachine >= 1.0.0.beta.1
+- added separate client specs for eventmachine = 0.12.10
+- added error checking to eventmachine specs
+
+0.1.0
+- first release

data/README.rdoc

@@ -1,258 +1,294 @@
-= em-pg-client
-
-Author::    Rafał Michalski  (mailto:rafal@yeondir.com)
-
-* http://github.com/royaltm/ruby-em-pg-client
-
-== DESCRIPTION
-
-*em-pg-client* is a PostgreSQL EventMachine client wrapper for Ruby
-based on ruby-pg[https://bitbucket.org/ged/ruby-pg]
-
-== FEATURES
-
-* minimal changes to PG::Connection API
-* auto reconnects on socket connection loss (like server restarts)
-* true non-blocking asynchronous processing
-* EM-Synchrony[https://github.com/igrigorik/em-synchrony] support
-
-== BUGS/LIMITATIONS
-
-* actually no ActiveRecord or Sequel support (you are welcome to contribute).
-* connecting/reconnecting operation is blocking EM
-
-== TODO:
-
-* em-synchrony ORM (ActiveRecord, Sequel and maybe Datamapper) support
-* full async connection process (low priority)
-
-== REQUIREMENTS
-
-* ruby >= 1.9
-* https://bitbucket.org/ged/ruby-pg (>= 0.13)
-* http://rubyeventmachine.com
-* (optional) EM-Synchrony[https://github.com/igrigorik/em-synchrony]
-
-== INSTALL
-
-  $ sudo gem install em-pg-client
-
-== WHY?
-
-Because until now nobody did it to fit my needs.
-I've found at least 3 other implementations of EM postgres client:
-
-* https://github.com/jzimmek/em-postgresql-sequel
-* https://github.com/leftbee/em-postgresql-adapter
-* https://github.com/jtoy/em-postgres
-
-and (except the bundled one which uses no longer maintained postgres-pr library)
-all of them have similiar flaws:
-
-* 2 of them are designed to support some ORM (ActiveRecord or Sequel)
-  so they are EM-Synchrony only,
-* non-standard API method names,
-* no (nonexistent or non-working) autoreconnect implementation,
-* not fully supporting asynchronous PG::Connection API.
-
-The last one is worth some comment:
-
-They all use blocking methods to retrieve whole result from server
-(PGConn#block() or PGConn#get_result() which also
-blocks when there is not enough buffered data on socket).
-
-This implementation makes use of non-blocking: PGConn#is_busy and PGConn#consume_input methods.
-Depending on size of result sets and concurrency level the gain in overall speed and responsiveness of your
-application might be actually quite huge. I already have done some tests[link:BENCHMARKS.rdoc].
-
-== Thanks
-
-The greetz go to:
-- Authors[https://bitbucket.org/ged/ruby-pg/wiki/Home#!copying] of +pg+ driver (especially for its async-api)
-- Francis Cianfrocca for great reactor framework (EventMachine[https://github.com/eventmachine/eventmachine])
-- Ilya Grigorik (igrigorik[https://github.com/igrigorik]) for (untangling)[http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/] EM with Fibers
-
-== USAGE
-
-=== BASIC
-
-  require 'pg/em'
-  
-  # no async
-  pg = PG::EM::Client.new dbname: 'test'
-  pq.query('select * from foo') do |result|
-    puts Array(result).inspect
-  end
-  
-  # asynchronous
-  EM.run do
-    pq.query('select * from foo') do |result|
-      raise result if result.is_a? ::Exception
-      puts Array(result).inspect
-      EM.stop
-    end
-    puts "sent"
-  end
-
-=== AUTORECONNECTING IN ASYNC MODE
-
-  EM.run do
-    pg = PG::EM::Client.new dbname: 'test'
-    try_query = lambda do |&blk|
-      pg.query('select * from foo') do |result|
-        raise result if result.is_a? ::Exception
-        puts Array(result).inspect
-        blk.call
-      end
-    end
-    try_query.call {
-      system 'pg_ctl stop -m fast'
-      system 'pg_ctl start -w'
-      EM.add_timer(1) do
-        try_query.call { EM.stop }
-      end
-    }
-  end
-
-to disable this feature call:
-
-  pg.async_autoreconnect = false
-
-or
-
-  pg = PG::EM::Client.new dbname: 'test',
-    async_autoreconnect: false
-
-It's also possible to define +on_reconnect+ callback to be invoked
-while the connection has been reset. It's called just before the send query
-command is executed:
-
-  EM.run do
-    pg = PG::EM::Client.new dbname: 'test'
-    pg.prepare('bar', 'select * from foo order by cdate desc') do
-      pg.on_reconnect = proc { |c, e|
-        c.prepare('bar', 'select * from foo order by cdate desc')
-      }
-      try_query = lambda do |&blk|
-        pg.exec_prepared('bar') do |result|
-          raise result if result.is_a? ::Exception
-          puts Array(result).inspect
-          blk.call
-        end
-      end
-      try_query.call {
-        system 'pg_ctl stop -m fast'
-        system 'pg_ctl start -w'
-        EM.add_timer(1) do
-          try_query.call { EM.stop }
-        end
-      }
-    end
-  end
-
-As you can see it's possible to send async query from inside on_reconnect
-proc. However you have to pass +Deferrable+ from the async callback to the
-caller. See +#on_reconnect+ docs for details.
-
-=== TRUE ASYNC
-
-  EM.run do
-    pool = (1..10).map { PG::EM::Client.new dbname: 'alpha' }
-
-    togo = pool.length
-
-    pool.each_with_index do |pg, i|
-      pg.query("select * from foo") do |result|
-        puts "recv: #{i}"
-        EM.stop if (togo-=1).zero?
-      end
-      puts "sent: #{i}"
-    end
-  end
-
-=== EM-Synchrony
-
-  require 'em-synchrony/pg'
-
-  EM.synchrony do
-    pg = PG::EM::Client.new dbname: 'test'
-    pg.query('select * from foo') do |result|
-      puts Array(result).inspect
-    end
-    EM.stop
-  end
-
-==== Handling errors
-
-  EM.synchrony do
-    begin
-      pg.query('select * from foo') do |result|
-        puts result
-      end
-    rescue PG::Error => e
-      puts "PSQL error: #{e.inspect}"
-    end
-    EM.stop
-  end
-
-==== Parallel async queries
-
-  EM.synchrony do
-    pg = EM::Synchrony::ConnectionPool.new(size: 2) do  
-      PG::EM::Client.new :dbname => 'alpha'
-    end
-    multi = EventMachine::Synchrony::Multi.new
-    multi.add :foo, pg.aquery('select * from foo') # or #async_query()
-    multi.add :bar, pg.aquery('select * from bar') # #aquery() is just an alias
-    res = multi.perform
-    p res
-    EM.stop
-  end
-
-==== Fiber Concurrency
-
-  EM.synchrony do
-    # use ConnectionPool when more Fibers will be querying at the same time!
-    pg = EM::Synchrony::ConnectionPool.new(size: 5) do  
-      PG::EM::Client.new :dbname => 'alpha'
-    end
-    counter = 0
-    EM::Synchrony::FiberIterator.new(['select * from foo']*10, 5) do |query|
-      i = counter
-      pg.query(query) do |result|
-        puts "recv: #{i}"
-      end
-      puts "sent: #{i}"
-      counter += 1
-    end
-    EM.stop
-  end
-
-==== Async reconnect with on_reconnect callback
-
-    EM.synchrony do
-    pg = EM::Synchrony::ConnectionPool.new(size: 5) do
-      p = PG::EM::Client.new dbname: 'test'
-      p.on_reconnect = proc do |c, e|
-        c.prepare('bar', 'select * from foo order by cdate desc')
-      end
-      p.on_reconnect.call p
-      p
-    end
-    try_query = lambda do
-      pg.exec_prepared('bar') do |result|
-        raise result if result.is_a? ::Exception
-        puts Array(result).inspect
-      end
-    end
-    try_query.call
-    system 'pg_ctl stop -m fast'
-    system 'pg_ctl start -w'
-    EM::Synchrony.sleep(1)
-    try_query.call
-    EM.stop
-  end
-
-== LICENCE
-
-The MIT License - Copyright (c) 2012 Rafał Michalski
+= em-pg-client
+
+Author::    Rafał Michalski  (mailto:rafal@yeondir.com)
+
+* http://github.com/royaltm/ruby-em-pg-client
+
+== DESCRIPTION
+
+*em-pg-client* is a PostgreSQL EventMachine client wrapper for Ruby
+based on ruby-pg[https://bitbucket.org/ged/ruby-pg]
+
+== FEATURES
+
+* minimal changes to PG::Connection API
+* fully async auto reconnects on socket connection loss (like server restarts)
+* true non-blocking asynchronous processing
+* EM-Synchrony[https://github.com/igrigorik/em-synchrony] support
+
+== BUGS/LIMITATIONS
+
+* actually no ActiveRecord nor Sequel support (you are welcome to contribute).
+
+== API Changes
+
+* +on_reconnect+ renamed to more accurate +on_autoreconnect+
+  (well, it's not used by +#reset+ call)
+
+== TODO:
+
+* em-synchrony ORM (ActiveRecord, Sequel and maybe Datamapper) support
+  as separate projects
+
+== REQUIREMENTS
+
+* ruby >= 1.9
+* https://bitbucket.org/ged/ruby-pg (>= 0.13)
+* http://rubyeventmachine.com
+* (optional) EM-Synchrony[https://github.com/igrigorik/em-synchrony]
+
+== INSTALL
+
+=== Legacy
+
+  $ [sudo] gem install em-pg-client
+  
+==== Gemfile
+
+  # eventmachine
+  gem "em-pg-client", "~> 0.1.1", :require => 'pg/em'
+  # em-synchrony
+  gem "em-pg-client", "~> 0.1.1", :require => ['pg/em', 'em-synchrony/pg']
+
+=== Latest branch (fully-async)
+
+  $ [sudo] gem install em-pg-client --prerelease
+
+==== Gemfile
+
+  # eventmachine
+  gem "em-pg-client", "~> 0.2.0.pre", :require => 'pg/em'
+  # em-synchrony
+  gem "em-pg-client", "~> 0.2.0.pre", :require => ['pg/em', 'em-synchrony/pg']
+
+==== Github
+
+  git clone git://github.com/royaltm/ruby-em-pg-client.git
+  git checkout fully-async
+
+== WHY?
+
+Because until now nobody did it to fit my needs.
+I've found at least 3 other implementations of EM postgres client:
+
+* https://github.com/jzimmek/em-postgresql-sequel
+* https://github.com/leftbee/em-postgresql-adapter
+* https://github.com/jtoy/em-postgres
+
+and (except the bundled one which uses no longer maintained postgres-pr library)
+all of them have similiar flaws:
+
+* 2 of them are designed to support some ORM (ActiveRecord or Sequel)
+  so they are EM-Synchrony only,
+* non-standard API method names,
+* no (nonexistent or non-working) autoreconnect implementation,
+* not fully supporting asynchronous PG::Connection API.
+
+The last one is worth some comment:
+
+They all use blocking methods to retrieve whole result from server
+(PGConn#block() or PGConn#get_result() which also
+blocks when there is not enough buffered data on socket).
+
+This implementation makes use of non-blocking: PGConn#is_busy and PGConn#consume_input methods.
+Depending on size of result sets and concurrency level the gain in overall speed and responsiveness of your
+application might be actually quite huge. I've done some tests[link:BENCHMARKS.rdoc] already.
+
+== Thanks
+
+The greetz go to:
+- Authors[https://bitbucket.org/ged/ruby-pg/wiki/Home#!copying] of +pg+ driver (especially for its async-api)
+- Francis Cianfrocca for great reactor framework (EventMachine[https://github.com/eventmachine/eventmachine])
+- Ilya Grigorik (igrigorik[https://github.com/igrigorik]) for (untangling)[http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/] EM with Fibers
+
+== USAGE
+
+=== BASIC
+
+  require 'pg/em'
+  
+  # no async
+  pg = PG::EM::Client.new dbname: 'test'
+  pq.query('select * from foo') do |result|
+    puts Array(result).inspect
+  end
+  
+  # asynchronous
+  EM.run do
+    pq.query('select * from foo') do |result|
+      raise result if result.is_a? ::Exception
+      puts Array(result).inspect
+      EM.stop
+    end
+    puts "sent"
+  end
+
+=== AUTORECONNECTING IN ASYNC MODE
+Autoreconnecting is done in non-blocking manner using #async_reset internally.
+
+  EM.run do
+    pg = PG::EM::Client.new dbname: 'test',
+          connect_timeout: 5, query_timeout: 50
+    try_query = lambda do |&blk|
+      pg.query('select * from foo') do |result|
+        raise result if result.is_a? ::Exception
+        puts Array(result).inspect
+        blk.call
+      end
+    end
+    try_query.call {
+      system 'pg_ctl stop -m fast'
+      system 'pg_ctl start -w'
+      try_query.call { EM.stop }
+    }
+  end
+
+to disable this feature call:
+
+  pg.async_autoreconnect = false
+
+or
+
+  pg = PG::EM::Client.new dbname: 'test',
+    async_autoreconnect: false
+
+It's also possible to define +on_autoreconnect+ callback to be invoked
+while the connection has been reset. It's called just before the send query
+command is executed:
+
+  EM.run do
+    pg = PG::EM::Client.new dbname: 'test'
+    pg.prepare('bar', 'select * from foo order by cdate desc') do
+      pg.on_autoreconnect = proc { |c, e|
+        c.prepare('bar', 'select * from foo order by cdate desc')
+      }
+      try_query = lambda do |&blk|
+        pg.exec_prepared('bar') do |result|
+          raise result if result.is_a? ::Exception
+          puts Array(result).inspect
+          blk.call
+        end
+      end
+      try_query.call {
+        system 'pg_ctl stop -m fast'
+        system 'pg_ctl start -w'
+        try_query.call { EM.stop }
+      }
+    end
+  end
+
+As you can see it's possible to send async query from inside on_autoreconnect
+proc. However you have to pass +Deferrable+ from the async callback to the
+caller. See +#on_autoreconnect+ docs for details.
+
+=== TRUE ASYNC
+For non-blocking connect use PG::EM::Client.async_connect and #async_reset for 
+asynchronous connection reset. Like other async methods they return deferrable object.
+Use #callback to obtain already connected Client.
+
+  EM.run do
+    pool = (1..10).map {
+      PG::EM::Client.async_connect dbname: 'test',
+          connect_timeout: 5, query_timeout: 50 }
+
+    togo = pool.length
+
+    pool.each_with_index do |df, i|
+      df.callback do |pg|
+        pg.query("select * from foo") do |result|
+          puts "recv: #{i}"
+          EM.stop if (togo-=1).zero?
+        end
+        puts "sent: #{i}"
+      end
+      df.errback { |ex| raise ex }
+    end
+  end
+
+=== EM-Synchrony
+Under +em-synchrony+ PG::EM::Client.new is fully asynchronous and blocks only current fiber.
+This also applies to #reset.
+
+  require 'em-synchrony/pg'
+
+  EM.synchrony do
+    pg = PG::EM::Client.new dbname: 'test'
+    pg.query('select * from foo') do |result|
+      puts Array(result).inspect
+    end
+    EM.stop
+  end
+
+==== Handling errors
+
+  EM.synchrony do
+    begin
+      pg.query('select * from foo') do |result|
+        puts result
+      end
+    rescue PG::Error => e
+      puts "PSQL error: #{e.inspect}"
+    end
+    EM.stop
+  end
+
+==== Parallel async queries
+
+  EM.synchrony do
+    pg = EM::Synchrony::ConnectionPool.new(size: 2) do  
+      PG::EM::Client.new :dbname => 'test'
+    end
+    multi = EventMachine::Synchrony::Multi.new
+    multi.add :foo, pg.aquery('select * from foo') # or #async_query()
+    multi.add :bar, pg.aquery('select * from bar') # #aquery() is just an alias
+    res = multi.perform
+    p res
+    EM.stop
+  end
+
+==== Fiber Concurrency
+
+  EM.synchrony do
+    # use ConnectionPool when more Fibers will be querying at the same time!
+    pg = EM::Synchrony::ConnectionPool.new(size: 5) do  
+      PG::EM::Client.new :dbname => 'test'
+    end
+    counter = 0
+    EM::Synchrony::FiberIterator.new(['select * from foo']*10, 5) do |query|
+      i = counter
+      pg.query(query) do |result|
+        puts "recv: #{i}"
+      end
+      puts "sent: #{i}"
+      counter += 1
+    end
+    EM.stop
+  end
+
+==== Async reconnect with on_autoreconnect callback
+
+  EM.synchrony do
+    on_autoreconnect = proc do |c, e|
+      c.prepare('bar', 'select * from foo order by cdate desc')
+    end
+    pg = EM::Synchrony::ConnectionPool.new(size: 5) do
+      p = PG::EM::Client.new dbname: 'test', on_autoreconnect: on_autoreconnect
+      on_autoreconnect.call p
+      p
+    end
+    try_query = lambda do
+      pg.exec_prepared('bar') do |result|
+        raise result if result.is_a? ::Exception
+        puts Array(result).inspect
+      end
+    end
+    try_query.call
+    system 'pg_ctl stop -m fast'
+    system 'pg_ctl start -w'
+    try_query.call
+    EM.stop
+  end
+
+== LICENCE
+
+The MIT License - Copyright (c) 2012 Rafał Michalski