em-pg-client-12 0.3.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.rspec +1 -0
- data/.travis.yml +32 -0
- data/.yardopts +1 -0
- data/BENCHMARKS.md +91 -0
- data/Gemfile +5 -0
- data/HISTORY.md +107 -0
- data/LICENSE +21 -0
- data/README.md +456 -0
- data/Rakefile +157 -0
- data/benchmarks/em_pg.rb +96 -0
- data/benchmarks/single_row_mode.rb +88 -0
- data/em-pg-client.gemspec +34 -0
- data/examples/single_row_mode.rb +57 -0
- data/lib/em-pg-client.rb +1 -0
- data/lib/em-synchrony/pg.rb +3 -0
- data/lib/pg/em-version.rb +5 -0
- data/lib/pg/em.rb +1129 -0
- data/lib/pg/em/client/connect_watcher.rb +89 -0
- data/lib/pg/em/client/watcher.rb +204 -0
- data/lib/pg/em/connection_pool.rb +480 -0
- data/lib/pg/em/featured_deferrable.rb +43 -0
- data/spec/connection_pool_helpers.rb +89 -0
- data/spec/em_client.rb +33 -0
- data/spec/em_client_autoreconnect.rb +672 -0
- data/spec/em_client_common.rb +619 -0
- data/spec/em_client_on_connect.rb +171 -0
- data/spec/em_connection_pool.rb +200 -0
- data/spec/em_synchrony_client.rb +787 -0
- data/spec/em_synchrony_client_autoreconnect.rb +560 -0
- data/spec/pg_em_client_connect_finish.rb +54 -0
- data/spec/pg_em_client_connect_timeout.rb +91 -0
- data/spec/pg_em_client_options.rb +133 -0
- data/spec/pg_em_connection_pool.rb +679 -0
- data/spec/pg_em_featured_deferrable.rb +125 -0
- data/spec/spec_helper.rb +9 -0
- metadata +187 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 93743ff68b0f04bdcff75b78165fbcbeafc20b96
|
4
|
+
data.tar.gz: a05e6cb106bcc02666bc57b9ee98c301d70839c0
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 73384a9e88c54ffbc721b00aadded48d82697c128d23bb24679b5196da71a4b29e6432ab19242706dc0ab42d5cc53147ffaf08fe58189181e0d31e3a0166b636
|
7
|
+
data.tar.gz: 47a95833438b11d1e142956125b5bed1c22446f7ecb307d9d82ad8231466c96bb30f06326b227ac2a2484cfbc04faaeb627cd6773dd9b92a2641febdb1b558e6
|
data/.rspec
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
--require spec_helper
|
data/.travis.yml
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
language: ruby
|
2
|
+
rvm:
|
3
|
+
- 1.9.2
|
4
|
+
- 1.9.3
|
5
|
+
- 2.0.0
|
6
|
+
- 2.1.0
|
7
|
+
env:
|
8
|
+
- PGVERSION=9.3 PGBUILD=9.3.4-1-linux-x64
|
9
|
+
- PGVERSION=9.0 PGBUILD=9.0.17-1-linux-x64
|
10
|
+
- PGVERSION=8.4 PGBUILD=8.4.21-1-linux-x64
|
11
|
+
before_install:
|
12
|
+
- sudo /etc/init.d/postgresql stop
|
13
|
+
- export PGPREFIX=/opt/PostgreSQL/$PGVERSION
|
14
|
+
- export PGDATA=$PGPREFIX/data
|
15
|
+
- export PATH="$PGPREFIX/bin:$PATH"
|
16
|
+
- wget http://get.enterprisedb.com/postgresql/postgresql-$PGBUILD.run
|
17
|
+
- chmod +x postgresql-$PGBUILD.run
|
18
|
+
- sudo ./postgresql-$PGBUILD.run --mode unattended --unattendedmodeui minimal --prefix
|
19
|
+
$PGPREFIX --datadir $PGDATA;
|
20
|
+
- sudo sed s/md5\$/trust/g $PGDATA/pg_hba.conf >/tmp/pg_hba.conf.$$
|
21
|
+
- sudo mv /tmp/pg_hba.conf.$$ $PGDATA/pg_hba.conf
|
22
|
+
- sudo -i -u postgres $PGPREFIX/bin/pg_ctl -D $PGDATA reload
|
23
|
+
- PGHOST_UNIX=`netstat -l --protocol=unix|grep PGSQL|awk '{print $NF}'`
|
24
|
+
- export PGHOST_UNIX="`dirname $PGHOST_UNIX`"
|
25
|
+
- test -n "$PGHOST_UNIX"
|
26
|
+
- export PG_CTL_STOP_CMD="sudo -i -u postgres $PGPREFIX/bin/pg_ctl -D $PGDATA stop
|
27
|
+
-s -m fast"
|
28
|
+
- export PG_CTL_START_CMD="sudo -i -u postgres $PGPREFIX/bin/pg_ctl -D $PGDATA start
|
29
|
+
-l $PGDATA/postgresql.log -s -w"
|
30
|
+
- export PGUSER=postgres
|
31
|
+
- psql -c 'create database test;' -h "$PGHOST_UNIX"
|
32
|
+
script: COVERAGE=1 bundle exec rake test_with_coveralls
|
data/.yardopts
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
lib/pg/em/client/*.rb lib/pg/em/*.rb lib/pg/*.rb - benchmarks/*.rb BENCHMARKS.md LICENSE HISTORY.md
|
data/BENCHMARKS.md
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
Benchmarking
|
2
|
+
============
|
3
|
+
|
4
|
+
Environment
|
5
|
+
-----------
|
6
|
+
|
7
|
+
The machine running tests is Linux CentOS 2.6.18-194.32.1.el5xen #1 SMP with
|
8
|
+
Quad Core Xeon X3360 @ 2.83GHz, 4GB RAM.
|
9
|
+
|
10
|
+
Postgres server version: 9.0.3
|
11
|
+
Postgres pqlib version: 9.3
|
12
|
+
|
13
|
+
Fully Asynchronous vs. poor man's async
|
14
|
+
---------------------------------------
|
15
|
+
|
16
|
+
The following {file:benchmarks/em_pg.rb benchmark} compares fully
|
17
|
+
asynchronous implementation (`em-pg-client`) versus blocking em-pg drivers.
|
18
|
+
|
19
|
+
The goal of the test is to retrieve (~80000) rows from the same table with
|
20
|
+
a lot of text data, in chunks, using parallel connections.
|
21
|
+
|
22
|
+
The parallel method uses synchrony for simplicity.
|
23
|
+
|
24
|
+
* `single` is (eventmachine-less) job for retrieving a whole data table in
|
25
|
+
one simple query "select * from resources"
|
26
|
+
* `parallel` chunk_row_count / concurrency] uses em-pg-client for retrieving
|
27
|
+
result in chunks by `chunk_row_count` rows and using `concurrency` parallel
|
28
|
+
connections
|
29
|
+
* `blocking` chunk_row_count / concurrency is similiar to `parallel` except
|
30
|
+
that it uses special patched version of library that uses blocking
|
31
|
+
PGConnection methods
|
32
|
+
|
33
|
+
|
34
|
+
```
|
35
|
+
>> benchmark 1000
|
36
|
+
user system total real
|
37
|
+
single: 80.970000 0.350000 81.320000 (205.592592)
|
38
|
+
|
39
|
+
parallel 90000/1: 87.380000 0.710000 88.090000 (208.171564)
|
40
|
+
parallel 5000/5: 84.250000 3.760000 88.010000 (141.031289)
|
41
|
+
parallel 2000/10: 90.190000 4.970000 95.160000 (152.844950)
|
42
|
+
parallel 1000/20: 97.070000 5.390000 102.460000 (212.358631)
|
43
|
+
|
44
|
+
blocking 90000/1: 93.590000 0.610000 94.200000 (230.190776)
|
45
|
+
blocking 5000/5: 79.930000 1.810000 81.740000 (223.342432)
|
46
|
+
blocking 2000/10: 76.990000 2.820000 79.810000 (225.347169)
|
47
|
+
blocking 1000/20: 78.790000 3.230000 82.020000 (225.949107)
|
48
|
+
```
|
49
|
+
|
50
|
+
As we can see the gain from using asynchronous em-pg-client while
|
51
|
+
using `parallel` queries is noticeable (up to ~30%).
|
52
|
+
|
53
|
+
The `blocking` client however doesn't gain much from parallel execution.
|
54
|
+
This was expected because it freezes eventmachine until the whole
|
55
|
+
dataset is consumed by the client.
|
56
|
+
|
57
|
+
|
58
|
+
Threads vs. Fibers Streaming Benchmark
|
59
|
+
--------------------------------------
|
60
|
+
|
61
|
+
The following {file:benchmarks/single_row_mode.rb benchmark} compares
|
62
|
+
performance of parallel running threads using vanilla PG::Connection driver
|
63
|
+
versus EventMachine driven parallel Fibers using PG::EM::Client v0.3.2.
|
64
|
+
|
65
|
+
Each thread/fiber retrieves first 5000 rows from the same table with
|
66
|
+
a lot of text data in a `single_row_mode`. After 5000 rows is retrieved
|
67
|
+
the connection is being reset. The process is repeated after all parallel
|
68
|
+
running threads/fibers finish their task.
|
69
|
+
|
70
|
+
Both Thread and Fiber versions use the same chunk of code to retrieve rows.
|
71
|
+
|
72
|
+
```
|
73
|
+
>> benchmark 400
|
74
|
+
user system total real
|
75
|
+
threads 400x1: 24.970000 1.090000 26.060000 ( 30.683818)
|
76
|
+
threads 80x5: 24.730000 7.020000 31.750000 ( 51.402710)
|
77
|
+
threads 40x10: 22.880000 7.460000 30.340000 ( 52.548910)
|
78
|
+
threads 20x20: 22.220000 7.130000 29.350000 ( 53.911111)
|
79
|
+
threads 10x40: 22.570000 7.620000 30.190000 ( 54.111841)
|
80
|
+
|
81
|
+
fibers 400x1: 26.040000 1.060000 27.100000 ( 31.619598)
|
82
|
+
fibers 80x5: 28.690000 1.140000 29.830000 ( 33.025573)
|
83
|
+
fibers 40x10: 28.790000 1.280000 30.070000 ( 33.498418)
|
84
|
+
fibers 20x20: 29.100000 1.210000 30.310000 ( 33.289344)
|
85
|
+
fibers 10x40: 29.220000 1.340000 30.560000 ( 33.691188)
|
86
|
+
```
|
87
|
+
|
88
|
+
```
|
89
|
+
AxB - repeat A times running B parallel threads/fibers.
|
90
|
+
```
|
91
|
+
|
data/Gemfile
ADDED
data/HISTORY.md
ADDED
@@ -0,0 +1,107 @@
|
|
1
|
+
0.3.4
|
2
|
+
|
3
|
+
- spec: fix em_client_on_connect and em_connection_pool tests
|
4
|
+
- asynchronous wait_for_notify and wait_for_notify_defer with specs
|
5
|
+
|
6
|
+
0.3.3
|
7
|
+
|
8
|
+
- rake: console for debugging
|
9
|
+
- added on_connect callback
|
10
|
+
- given block to ConnectionPool.new set as on_connect option fallback
|
11
|
+
- spec: connection pool database tests
|
12
|
+
- spec: on_connect tests
|
13
|
+
- spec: on_connect + on_autoreconnect auto re-connect tests
|
14
|
+
- spec: added travis rvm 2.1.0 and updated postgres server versions
|
15
|
+
- sugar: on_connect and on_autoreconnect block setters
|
16
|
+
|
17
|
+
0.3.2
|
18
|
+
|
19
|
+
- fix: asynchronous get_result performance
|
20
|
+
- fix+specs: query_timeout timer is canceled on connection breakdown
|
21
|
+
- comply with pg+specs: asynchronous get_result and get_last_result return nil
|
22
|
+
when connection status is not ok
|
23
|
+
|
24
|
+
0.3.1
|
25
|
+
|
26
|
+
- support for asynchronous data streaming in single row mode -
|
27
|
+
asynchronous versions of get_result, get_last_result and
|
28
|
+
their deferrable variants
|
29
|
+
- watcher improvements allowing to reset pending commands
|
30
|
+
- minor DRY improvements in code and specs
|
31
|
+
- single_row_mode? helper
|
32
|
+
- spec: Travis CI and Coverage
|
33
|
+
|
34
|
+
0.3.0
|
35
|
+
|
36
|
+
- dedicated asynchronous connection pool
|
37
|
+
- works on windows (with ruby 2.0+): uses PGConn#socket_io object instead of
|
38
|
+
#socket file descriptor
|
39
|
+
- socket watch handler is not being detached between command calls
|
40
|
+
- no more separate em and em-synchrony client
|
41
|
+
- api changes: async_exec and async_query command are now fiber-synchronized
|
42
|
+
- api changes: other async_* methods are removed or deprecated
|
43
|
+
- api changes: asynchronous methods renamed to *_defer
|
44
|
+
- transaction() helper method that can be called recursively
|
45
|
+
- requirements updated: eventmachine >~ 1.0.0, pg >= 0.17.0, ruby >= 1.9.2
|
46
|
+
- spec: more auto re-connect test cases
|
47
|
+
- spec: more tests for connection establishing
|
48
|
+
- comply with pg: do not close the client on connection failure
|
49
|
+
- comply with pg: asynchronous connect_timeout fallbacks to environment variable
|
50
|
+
- fix: auto re-connect raises an error if the failed connection had unfinished
|
51
|
+
transaction state
|
52
|
+
- yardoc docs
|
53
|
+
|
54
|
+
0.2.1
|
55
|
+
|
56
|
+
- support for pg >= 0.14 native PG::Result#check
|
57
|
+
- support for pg >= 0.14 native PG::Connection#set_default_encoding
|
58
|
+
- fix: connection option Hash argument was modified by Client.new and Client.async_connect
|
59
|
+
|
60
|
+
0.2.0
|
61
|
+
|
62
|
+
- disabled async_autoreconnect by default unless on_autoreconnect is set
|
63
|
+
- async_connect sets #internal_encoding to Encoding.default_internal
|
64
|
+
- fix: finish connection on async connect_timeout
|
65
|
+
- nice errors generated on missing dependencies
|
66
|
+
- blocking #reset() should clear async_command_aborted flag
|
67
|
+
- less calls to #is_busy in Watcher#notify_readable
|
68
|
+
- async_describe_portal() + specs
|
69
|
+
- async_describe_prepared() + specs
|
70
|
+
|
71
|
+
0.2.0.pre.3
|
72
|
+
|
73
|
+
- status() returns CONNECTION_BAD for connections with expired query
|
74
|
+
- spec: query timeout expiration
|
75
|
+
- non-blocking result processing for multiple data query statements sent at once
|
76
|
+
- refine code in em-synchrony/pg
|
77
|
+
- spec: more detailed tests
|
78
|
+
- spec: async_connect
|
79
|
+
- spec: autoreconnect
|
80
|
+
|
81
|
+
0.2.0.pre.2
|
82
|
+
|
83
|
+
- errors from consume_input fails deferrable
|
84
|
+
- query_timeout now measures only network response timeout,
|
85
|
+
so it's not fired for large datasets
|
86
|
+
|
87
|
+
0.2.0.pre.1
|
88
|
+
|
89
|
+
- added query_timeout feature for async query commands
|
90
|
+
- added connect_timeout property for async connect/reset
|
91
|
+
- fix: async_autoreconnect for tcp/ip connections
|
92
|
+
- fix: async_* does not raise errors; errors handled by deferrable
|
93
|
+
- rework async_autoreconnect in fully async manner
|
94
|
+
- added async_connect() and async_reset()
|
95
|
+
- API change: on_reconnect -> on_autoreconnect
|
96
|
+
|
97
|
+
0.1.1
|
98
|
+
|
99
|
+
- added on_reconnect callback
|
100
|
+
- docs updated
|
101
|
+
- added development dependency for eventmachine >= 1.0.0.beta.1
|
102
|
+
- added separate client specs for eventmachine = 0.12.10
|
103
|
+
- added error checking to eventmachine specs
|
104
|
+
|
105
|
+
0.1.0
|
106
|
+
|
107
|
+
- first release
|
data/LICENSE
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
Copyright (c) 2013 Rafal Michalski (rafal at yeondir dot com)
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,456 @@
|
|
1
|
+
em-pg-client
|
2
|
+
============
|
3
|
+
|
4
|
+
The Ruby EventMachine driver interface to the PostgreSQL RDBMS. It is based on
|
5
|
+
[ruby-pg](https://bitbucket.org/ged/ruby-pg).
|
6
|
+
|
7
|
+
[![Gem Version][GV img]][Gem Version]
|
8
|
+
[![Dependency Status][DS img]][Dependency Status]
|
9
|
+
[![Coverage Status][CS img]][Coverage Status]
|
10
|
+
[![Build Status][BS img]][Build Status]
|
11
|
+
|
12
|
+
Author: Rafał Michalski (rafal at yeondir dot com)
|
13
|
+
|
14
|
+
* http://github.com/royaltm/ruby-em-pg-client
|
15
|
+
|
16
|
+
Description
|
17
|
+
-----------
|
18
|
+
|
19
|
+
__em-pg-client__ provides {PG::EM::Client} class which inherits
|
20
|
+
[PG::Connection](http://deveiate.org/code/pg/PG/Connection.html).
|
21
|
+
You can work with {PG::EM::Client} almost the same way you would work
|
22
|
+
with PG::Connection.
|
23
|
+
|
24
|
+
The real difference begins when you turn the EventMachine reactor on.
|
25
|
+
|
26
|
+
```ruby
|
27
|
+
require 'pg/em'
|
28
|
+
|
29
|
+
pg = PG::EM::Client.new dbname: 'test'
|
30
|
+
|
31
|
+
# no async
|
32
|
+
pg.query('select * from foo') do |result|
|
33
|
+
puts Array(result).inspect
|
34
|
+
end
|
35
|
+
|
36
|
+
# asynchronous
|
37
|
+
EM.run do
|
38
|
+
Fiber.new do
|
39
|
+
pg.query('select * from foo') do |result|
|
40
|
+
puts Array(result).inspect
|
41
|
+
end
|
42
|
+
EM.stop
|
43
|
+
end.resume
|
44
|
+
end
|
45
|
+
|
46
|
+
# asynchronous + deferrable
|
47
|
+
EM.run do
|
48
|
+
df = pg.query_defer('select * from foo')
|
49
|
+
df.callback { |result|
|
50
|
+
puts Array(result).inspect
|
51
|
+
EM.stop
|
52
|
+
}
|
53
|
+
df.errback {|ex|
|
54
|
+
raise ex
|
55
|
+
}
|
56
|
+
puts "sent"
|
57
|
+
end
|
58
|
+
```
|
59
|
+
|
60
|
+
Features
|
61
|
+
--------
|
62
|
+
|
63
|
+
* Non-blocking / fully asynchronous processing with EventMachine.
|
64
|
+
* Event reactor auto-detecting, asynchronous fiber-synchronized command methods
|
65
|
+
(the same code can be used regardless of the EventMachine reactor state)
|
66
|
+
* Asynchronous EM-style (Deferrable returning) command methods.
|
67
|
+
* Fully asynchronous automatic re-connects on connection failures
|
68
|
+
(e.g.: RDBMS restarts, network failures).
|
69
|
+
* Minimal changes to [PG::Connection](http://deveiate.org/code/pg/PG/Connection.html) API.
|
70
|
+
* Configurable timeouts (connect or execute) of asynchronous processing.
|
71
|
+
* Dedicated connection pool with dynamic size, supporting asynchronous
|
72
|
+
processing and transactions.
|
73
|
+
* [Sequel Adapter](https://github.com/fl00r/em-pg-sequel) by Peter Yanovich.
|
74
|
+
* Works on windows (requires ruby 2.0) ([issue #7][Issue 7]).
|
75
|
+
* Supports asynchronous query data processing in single row mode
|
76
|
+
([issue #12][Issue 12]). See {file:BENCHMARKS.md BENCHMARKING}.
|
77
|
+
* __New__ - asynchronous implementation of wait_for_notify
|
78
|
+
|
79
|
+
Requirements
|
80
|
+
------------
|
81
|
+
|
82
|
+
* ruby >= 1.9.2 (tested: 2.1.0, 2.0.0-p353, 1.9.3-p374, 1.9.2-p320)
|
83
|
+
* https://bitbucket.org/ged/ruby-pg >= 0.17.0
|
84
|
+
* [PostgreSQL](http://www.postgresql.org/ftp/source/) RDBMS >= 8.4
|
85
|
+
* http://rubyeventmachine.com >= 1.0.0
|
86
|
+
* [EM-Synchrony](https://github.com/igrigorik/em-synchrony)
|
87
|
+
(optional - not needed for any of the client functionality,
|
88
|
+
just wrap your code in a fiber)
|
89
|
+
|
90
|
+
Install
|
91
|
+
-------
|
92
|
+
|
93
|
+
```sh
|
94
|
+
$ [sudo] gem install em-pg-client
|
95
|
+
```
|
96
|
+
|
97
|
+
#### Gemfile
|
98
|
+
|
99
|
+
```ruby
|
100
|
+
gem "em-pg-client", "~> 0.3.4"
|
101
|
+
```
|
102
|
+
|
103
|
+
#### Github
|
104
|
+
|
105
|
+
```
|
106
|
+
git clone git://github.com/royaltm/ruby-em-pg-client.git
|
107
|
+
```
|
108
|
+
|
109
|
+
Usage
|
110
|
+
-----
|
111
|
+
|
112
|
+
### PG::Connection commands adapted to the EventMachine
|
113
|
+
|
114
|
+
#### Asynchronous, the EventMachine style:
|
115
|
+
|
116
|
+
* `Client.connect_defer` (singleton method)
|
117
|
+
* `reset_defer`
|
118
|
+
* `exec_defer` (alias: `query_defer`)
|
119
|
+
* `prepare_defer`
|
120
|
+
* `exec_prepared_defer`
|
121
|
+
* `describe_prepared_defer`
|
122
|
+
* `describe_portal_defer`
|
123
|
+
|
124
|
+
For arguments of these methods consult their original (without the `_defer`
|
125
|
+
suffix) counterparts in the
|
126
|
+
[PG::Connection](http://deveiate.org/code/pg/PG/Connection.html) manual.
|
127
|
+
|
128
|
+
Use `callback` with a block on the returned deferrable object to receive the
|
129
|
+
result. In case of `connect_defer` and `reset_defer` the result is an instance
|
130
|
+
of the {PG::EM::Client}. The received client is in connected state and ready
|
131
|
+
for the queries. Otherwise an instance of the
|
132
|
+
[PG::Result](http://deveiate.org/code/pg/PG/Result.html) is received. You may
|
133
|
+
`clear` the obtained result object or leave it to `gc`.
|
134
|
+
|
135
|
+
To detect an error in the executed command call `errback` on the deferrable
|
136
|
+
with a block. You should expect an instance of the raised `Exception`
|
137
|
+
(usually PG::Error) as the block argument.
|
138
|
+
|
139
|
+
#### Reactor sensing methods, EM-Synchrony style:
|
140
|
+
|
141
|
+
* `Client.new` (singleton, alias: `connect`, `open`, `setdb`, `setdblogin`)
|
142
|
+
* `reset`
|
143
|
+
* `exec` (alias: `query`, `async_exec`, `async_query`)
|
144
|
+
* `prepare`
|
145
|
+
* `exec_prepared`
|
146
|
+
* `describe_prepared`
|
147
|
+
* `describe_portal`
|
148
|
+
|
149
|
+
The above methods call `*_defer` counterparts of themselves and `yield`
|
150
|
+
from the current fiber awaiting for the result. The PG::Result instance
|
151
|
+
(or PG::EM::Client for `new`) is then returned to the caller.
|
152
|
+
If a code block is given, it will be passed the result as an argument.
|
153
|
+
In that case the value of the block is returned instead and the result is
|
154
|
+
being cleared (or in case of `new` - client is being closed) after block
|
155
|
+
terminates.
|
156
|
+
|
157
|
+
These methods check if EventMachine's reactor is running and the current fiber
|
158
|
+
is not a root fiber. Otherwise the parent (thread-blocking) PG::Connection
|
159
|
+
methods are being called.
|
160
|
+
|
161
|
+
You can call asynchronous, fiber aware and blocking methods without finishing
|
162
|
+
the connection. You only need to start/stop EventMachine in between the
|
163
|
+
asynchronous calls.
|
164
|
+
|
165
|
+
Although the [em-synchrony](https://github.com/igrigorik/em-synchrony/)
|
166
|
+
provides very nice set of tools for the untangled EventMachine, you don't
|
167
|
+
really require it to fully benefit from the PG::EM::Client. Just wrap your
|
168
|
+
asynchronous code in a fiber:
|
169
|
+
|
170
|
+
Fiber.new { ... }.resume
|
171
|
+
|
172
|
+
#### Special options
|
173
|
+
|
174
|
+
There are four special connection options and one of them is a standard `pg`
|
175
|
+
option used by the async methods. You may pass them as one of the __hash__
|
176
|
+
options to {PG::EM::Client.new} or {PG::EM::Client.connect_defer} or simply
|
177
|
+
use the accessor methods to change them on the fly.
|
178
|
+
|
179
|
+
The options are:
|
180
|
+
|
181
|
+
* `connect_timeout`
|
182
|
+
* `query_timeout`
|
183
|
+
* `async_autoreconnect`
|
184
|
+
* `on_autoreconnect`
|
185
|
+
* `on_connect`
|
186
|
+
|
187
|
+
Only `connect_timeout` is a standard `libpq` option, although changing it with
|
188
|
+
the accessor method affects asynchronous functions only.
|
189
|
+
See {PG::EM::Client} for more details.
|
190
|
+
|
191
|
+
#### Handling errors
|
192
|
+
|
193
|
+
Exactly like in `pg`:
|
194
|
+
|
195
|
+
```ruby
|
196
|
+
EM.synchrony do
|
197
|
+
begin
|
198
|
+
pg.query('smellect 1')
|
199
|
+
rescue => e
|
200
|
+
puts "error: #{e.inspect}"
|
201
|
+
end
|
202
|
+
EM.stop
|
203
|
+
end
|
204
|
+
```
|
205
|
+
|
206
|
+
with *_defer methods:
|
207
|
+
|
208
|
+
```ruby
|
209
|
+
EM.run do
|
210
|
+
pg.query_defer('smellect 1') do |ret|
|
211
|
+
if ret.is_a?(Exception)
|
212
|
+
puts "PSQL error: #{ret.inspect}"
|
213
|
+
end
|
214
|
+
end
|
215
|
+
end
|
216
|
+
```
|
217
|
+
|
218
|
+
or
|
219
|
+
|
220
|
+
```ruby
|
221
|
+
EM.run do
|
222
|
+
pg.query_defer('smellect 1').callback do |ret|
|
223
|
+
puts "do something with #{ret}"
|
224
|
+
end.errback do |err|
|
225
|
+
puts "PSQL error: #{err.inspect}"
|
226
|
+
end
|
227
|
+
end
|
228
|
+
```
|
229
|
+
|
230
|
+
### Auto re-connecting in asynchronous mode
|
231
|
+
|
232
|
+
Connection reset is done in a non-blocking manner using `reset_defer` internally.
|
233
|
+
|
234
|
+
```ruby
|
235
|
+
EM.run do
|
236
|
+
Fiber.new do
|
237
|
+
pg = PG::EM::Client.new async_autoreconnect: true
|
238
|
+
|
239
|
+
try_query = lambda do
|
240
|
+
pg.query('select * from foo') do |result|
|
241
|
+
puts Array(result).inspect
|
242
|
+
end
|
243
|
+
end
|
244
|
+
|
245
|
+
try_query.call
|
246
|
+
system 'pg_ctl stop -m fast'
|
247
|
+
system 'pg_ctl start -w'
|
248
|
+
try_query.call
|
249
|
+
|
250
|
+
EM.stop
|
251
|
+
end.resume
|
252
|
+
end
|
253
|
+
```
|
254
|
+
|
255
|
+
to enable this feature call:
|
256
|
+
|
257
|
+
```ruby
|
258
|
+
pg.async_autoreconnect = true
|
259
|
+
```
|
260
|
+
|
261
|
+
Additionally the `on_autoreconnect` callback may be set on the connection.
|
262
|
+
It's being invoked after successfull connection restart, just before the
|
263
|
+
pending command is sent again to the server.
|
264
|
+
|
265
|
+
### Server-sent notifications - async style
|
266
|
+
|
267
|
+
Not surprisingly, there are two possible ways to wait for notifications,
|
268
|
+
one with a deferrable:
|
269
|
+
|
270
|
+
```ruby
|
271
|
+
pg = PG::EM::Client.new
|
272
|
+
EM.run do
|
273
|
+
pg.wait_for_notify_defer(7).callback do |notify|
|
274
|
+
if notify
|
275
|
+
puts "Someone spoke to us on channel: #{notify[:relname]} from #{notify[:be_pid]}"
|
276
|
+
else
|
277
|
+
puts "Too late, 7 seconds passed"
|
278
|
+
end
|
279
|
+
end.errback do |ex|
|
280
|
+
puts "Connection to deep space lost..."
|
281
|
+
end
|
282
|
+
pg.query_defer("LISTEN deep_space") do
|
283
|
+
pg.query_defer("NOTIFY deep_space") do
|
284
|
+
puts "Reaching out... to the other worlds"
|
285
|
+
end
|
286
|
+
end
|
287
|
+
end
|
288
|
+
```
|
289
|
+
|
290
|
+
and the other, using fibers:
|
291
|
+
|
292
|
+
```ruby
|
293
|
+
EM.synchrony do
|
294
|
+
pg = PG::EM::Client.new
|
295
|
+
EM::Synchrony.next_tick do
|
296
|
+
pg.query('LISTEN "some channel"')
|
297
|
+
pg.query('SELECT pg_notify($1::text,$2::text)', ['some channel', 'with some message'])
|
298
|
+
end
|
299
|
+
pg.wait_for_notify(10) do |channel, pid, payload|
|
300
|
+
puts "I've got notification on #{channel} #{payload}."
|
301
|
+
end.tap do |name|
|
302
|
+
puts "Whatever, I've been waiting too long already" if name.nil?
|
303
|
+
end
|
304
|
+
end
|
305
|
+
```
|
306
|
+
|
307
|
+
As you might have noticed, one does not simply wait for notifications,
|
308
|
+
but one can also run some queries on the same connection at the same time,
|
309
|
+
if one wishes so.
|
310
|
+
|
311
|
+
### Connection Pool
|
312
|
+
|
313
|
+
Forever alone? Not anymore! There is a dedicated {PG::EM::ConnectionPool}
|
314
|
+
class with dynamic pool for both types of asynchronous commands (deferral
|
315
|
+
and fiber-synchronized).
|
316
|
+
|
317
|
+
It also provides a #transaction method which locks the in-transaction
|
318
|
+
connection to the calling fiber and allows to execute commands
|
319
|
+
on the same connection within a transaction block. The transactions may
|
320
|
+
be nested. See also docs for the {PG::EM::Client#transaction} method.
|
321
|
+
|
322
|
+
#### Parallel async queries
|
323
|
+
|
324
|
+
```ruby
|
325
|
+
require 'pg/em/connection_pool'
|
326
|
+
require 'em-synchrony'
|
327
|
+
|
328
|
+
EM.synchrony do
|
329
|
+
pg = PG::EM::ConnectionPool.new(size: 2, dbname: 'test')
|
330
|
+
|
331
|
+
multi = EM::Synchrony::Multi.new
|
332
|
+
multi.add :foo, pg.query_defer('select pg_sleep(1)')
|
333
|
+
multi.add :bar, pg.query_defer('select pg_sleep(1)')
|
334
|
+
|
335
|
+
start = Time.now
|
336
|
+
res = multi.perform
|
337
|
+
# around 1 sec.
|
338
|
+
puts Time.now - start
|
339
|
+
|
340
|
+
EM.stop
|
341
|
+
end
|
342
|
+
```
|
343
|
+
|
344
|
+
#### Fiber Concurrency
|
345
|
+
|
346
|
+
```ruby
|
347
|
+
require 'pg/em/connection_pool'
|
348
|
+
require 'em-synchrony'
|
349
|
+
require "em-synchrony/fiber_iterator"
|
350
|
+
|
351
|
+
EM.synchrony do
|
352
|
+
concurrency = 5
|
353
|
+
queries = (1..10).map {|i| "select pg_sleep(1); select #{i}" }
|
354
|
+
|
355
|
+
pg = PG::EM::ConnectionPool.new(size: concurrency, dbname: 'test')
|
356
|
+
|
357
|
+
start = Time.now
|
358
|
+
EM::Synchrony::FiberIterator.new(queries, concurrency).each do |query|
|
359
|
+
pg.query(query) do |result|
|
360
|
+
puts "recv: #{result.getvalue(0,0)}"
|
361
|
+
end
|
362
|
+
end
|
363
|
+
# around 2 secs.
|
364
|
+
puts Time.now - start
|
365
|
+
|
366
|
+
EM.stop
|
367
|
+
end
|
368
|
+
```
|
369
|
+
|
370
|
+
API Changes
|
371
|
+
-----------
|
372
|
+
|
373
|
+
### 0.2.x -> 0.3.x
|
374
|
+
|
375
|
+
There is a substantial difference in the API between this and the previous
|
376
|
+
releases. The idea behind it was to make this implementation as much
|
377
|
+
compatible as possible with the threaded `pg` interface.
|
378
|
+
E.g. the `#async_exec` is now an alias to `#exec`.
|
379
|
+
|
380
|
+
The other reason was to get rid of the ugly em / em-synchrony duality.
|
381
|
+
|
382
|
+
* There is no separate em-synchrony client version anymore.
|
383
|
+
* The methods returning Deferrable have now the `*_defer` suffix.
|
384
|
+
* The `#async_exec` and `#async_query` (in <= 0.2 they were deferrable methods)
|
385
|
+
are now aliases to `#exec`.
|
386
|
+
* The command methods `#exec`, `#query`, `#exec_*`, `#describe_*` are now
|
387
|
+
em-synchrony style methods (fiber-synchronized).
|
388
|
+
* The following methods were removed:
|
389
|
+
|
390
|
+
- `#async_prepare`,
|
391
|
+
- `#async_exec_prepared`,
|
392
|
+
- `#async_describe_prepared`,
|
393
|
+
- `#async_describe_portal`
|
394
|
+
|
395
|
+
as their names were confusing due to the unfortunate `#async_exec`.
|
396
|
+
|
397
|
+
* The `async_connect` and `#async_reset` are renamed to `connect_defer` and `#reset_defer`
|
398
|
+
respectively.
|
399
|
+
|
400
|
+
### 0.1.x -> 0.2.x
|
401
|
+
|
402
|
+
* `on_reconnect` renamed to more accurate `on_autoreconnect`
|
403
|
+
(well, it's not used by PG::EM::Client#reset call).
|
404
|
+
* `async_autoreconnect` is `false` by default if `on_autoreconnect`
|
405
|
+
is __not__ specified as initialization option.
|
406
|
+
|
407
|
+
Bugs/Limitations
|
408
|
+
----------------
|
409
|
+
|
410
|
+
* no async support for COPY commands (`get_copy_data`, `put_copy_data`)
|
411
|
+
* actually no ActiveRecord support (you are welcome to contribute).
|
412
|
+
|
413
|
+
TODO:
|
414
|
+
-----
|
415
|
+
|
416
|
+
* more convenient streaming API
|
417
|
+
* implement EM adapted version of `get_copy_data`, `put_copy_data`
|
418
|
+
* ORM (ActiveRecord and maybe Datamapper) support as separate projects
|
419
|
+
|
420
|
+
More Info
|
421
|
+
---------
|
422
|
+
|
423
|
+
This implementation makes use of non-blocking:
|
424
|
+
[PGConn#is_busy](http://deveiate.org/code/pg/PG/Connection.html#method-i-is_busy) and
|
425
|
+
[PGConn#consume_input](http://deveiate.org/code/pg/PG/Connection.html#method-i-consume_input) methods.
|
426
|
+
Depending on the size of queried results and the concurrency level, the gain
|
427
|
+
in overall speed and responsiveness of your application might be actually quite huge.
|
428
|
+
See {file:BENCHMARKS.md BENCHMARKING}.
|
429
|
+
|
430
|
+
Thanks
|
431
|
+
------
|
432
|
+
|
433
|
+
The greetz go to:
|
434
|
+
|
435
|
+
* [Authors](https://bitbucket.org/ged/ruby-pg/wiki/Home#!copying) of __pg__
|
436
|
+
driver (especially for its async-api)
|
437
|
+
* Francis Cianfrocca for great reactor framework
|
438
|
+
[EventMachine](https://github.com/eventmachine/eventmachine)
|
439
|
+
* Ilya Grigorik [igrigorik](https://github.com/igrigorik) for
|
440
|
+
[untangling EM with Fibers](http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/)
|
441
|
+
* Peter Yanovich [fl00r](https://github.com/fl00r) for the
|
442
|
+
[em-pg-sequel](https://github.com/fl00r/em-pg-sequel)
|
443
|
+
* Andrew Rudenko [prepor](https://github.com/prepor) for the implicit idea
|
444
|
+
of the re-usable watcher from his [em-pg](https://github.com/prepor/em-pg).
|
445
|
+
|
446
|
+
[Gem Version]: https://rubygems.org/gems/em-pg-client
|
447
|
+
[Dependency Status]: https://gemnasium.com/royaltm/ruby-em-pg-client
|
448
|
+
[Coverage Status]: https://coveralls.io/r/royaltm/ruby-em-pg-client
|
449
|
+
[Build Status]: https://travis-ci.org/royaltm/ruby-em-pg-client
|
450
|
+
[Issue 7]: https://github.com/royaltm/ruby-em-pg-client/issues/7
|
451
|
+
[Issue 12]: https://github.com/royaltm/ruby-em-pg-client/issues/12
|
452
|
+
[GV img]: https://badge.fury.io/rb/em-pg-client.png
|
453
|
+
[DS img]: https://gemnasium.com/royaltm/ruby-em-pg-client.png
|
454
|
+
[CS img]: https://coveralls.io/repos/royaltm/ruby-em-pg-client/badge.png
|
455
|
+
[BS img]: https://travis-ci.org/royaltm/ruby-em-pg-client.png
|
456
|
+
[BB img]: https://d2weczhvl823v0.cloudfront.net/royaltm/ruby-em-pg-client/trend.png
|