async-postgres 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +11 -0
- data/.rspec +4 -0
- data/.travis.yml +21 -0
- data/Gemfile +14 -0
- data/README.md +171 -0
- data/Rakefile +14 -0
- data/async-postgres.gemspec +27 -0
- data/examples/rack/config.ru +12 -0
- data/examples/rack/schema.rb +12 -0
- data/lib/async/postgres.rb +31 -0
- data/lib/async/postgres/connection.rb +85 -0
- data/lib/async/postgres/pool.rb +112 -0
- data/lib/async/postgres/version.rb +25 -0
- data/spec/async/activerecord_spec.rb +31 -0
- data/spec/async/postgres_spec.rb +19 -0
- data/spec/spec_helper.rb +54 -0
- metadata +161 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 0bd8fd443302461c5594198ce65582187884cf853adcde1bdcb09c1148bf5120
|
4
|
+
data.tar.gz: 32993929caa96e255153eb8d5e188902a8b9b4460ec5a6b56f6c5b72baf7a4f0
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 4bd4b7a88abbaf3dacdba492ab6594934405c3e6f1a818ce29dbbd663997406d487c47ed47e28ceab63716945cc81e900adbdaf369449bd0c76ef565fdbd4f6a
|
7
|
+
data.tar.gz: d78701201ce8465b9f09f7b01876cb090f15326fa745b7166829442689cf8a95c06156388d7fb4d538bcdec67eb46dfb55d5a10cb63defb9c50f176b204df84a
|
data/.gitignore
ADDED
data/.rspec
ADDED
data/.travis.yml
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
language: ruby
|
2
|
+
sudo: false
|
3
|
+
dist: trusty
|
4
|
+
cache: bundler
|
5
|
+
services:
|
6
|
+
- postgresql
|
7
|
+
before_script:
|
8
|
+
- psql -c 'create database test;' -U postgres
|
9
|
+
rvm:
|
10
|
+
- 2.2
|
11
|
+
- 2.3
|
12
|
+
- 2.4
|
13
|
+
- 2.5
|
14
|
+
- jruby-head
|
15
|
+
- ruby-head
|
16
|
+
- rbx-3
|
17
|
+
matrix:
|
18
|
+
allow_failures:
|
19
|
+
- rvm: ruby-head
|
20
|
+
- rvm: jruby-head
|
21
|
+
- rvm: rbx-3
|
data/Gemfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,171 @@
|
|
1
|
+
# Async::Postgres
|
2
|
+
|
3
|
+
This is an experimental drop in wrapper to make Postgres work asynchronously.
|
4
|
+
|
5
|
+
## Motivation
|
6
|
+
|
7
|
+
We have some IO bound web APIs generating statistics and we sometimes have issues when using [passenger] due to thread/process exhaustion. In addition, we make a lot of upstream HTTP RPCs and these are also IO bound.
|
8
|
+
|
9
|
+
This library, in combination with [async-http], ensure that we don't become IO bound in many cases. In addition, we don't need to tune the intermediate server as it will simply scale according to backend resource availability and IO throughput.
|
10
|
+
|
11
|
+
[passenger]: https://github.com/phusion/passenger
|
12
|
+
[async-http]: https://github.com/socketry/async-http
|
13
|
+
|
14
|
+
## Installation
|
15
|
+
|
16
|
+
Add this line to your application's Gemfile:
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
gem 'async-postgres'
|
20
|
+
```
|
21
|
+
|
22
|
+
And then execute:
|
23
|
+
|
24
|
+
$ bundle
|
25
|
+
|
26
|
+
Or install it yourself as:
|
27
|
+
|
28
|
+
$ gem install async-postgres
|
29
|
+
|
30
|
+
## Performance
|
31
|
+
|
32
|
+
For database-bound workloads, this approach yields significant improvements to throughput and ultimately latency.
|
33
|
+
|
34
|
+
Using the example provided, which sleeps in the database for 10x10ms, we expect 10 sequential requests/second:
|
35
|
+
|
36
|
+
```ruby
|
37
|
+
run lambda {|env|
|
38
|
+
10.times do
|
39
|
+
ActiveRecord::Base.connection.execute("SELECT pg_sleep(0.01)")
|
40
|
+
end
|
41
|
+
|
42
|
+
ActiveRecord::Base.clear_active_connections!
|
43
|
+
|
44
|
+
[200, {}, []]
|
45
|
+
}
|
46
|
+
```
|
47
|
+
|
48
|
+
When running on [puma], with 16 threads, we could expect roughly 16 threads * 10 sequential requests/second.
|
49
|
+
|
50
|
+
```
|
51
|
+
% puma
|
52
|
+
Puma starting in single mode...
|
53
|
+
* Version 3.11.2 (ruby 2.5.0-p0), codename: Love Song
|
54
|
+
* Min threads: 0, max threads: 16
|
55
|
+
* Environment: development
|
56
|
+
* Listening on tcp://0.0.0.0:9292
|
57
|
+
Use Ctrl-C to stop
|
58
|
+
|
59
|
+
% wrk -c 512 -t 128 -d 30 http://localhost:9292
|
60
|
+
Running 30s test @ http://localhost:9292
|
61
|
+
128 threads and 512 connections
|
62
|
+
Thread Stats Avg Stdev Max +/- Stdev
|
63
|
+
Latency 105.77ms 4.05ms 176.61ms 98.61%
|
64
|
+
Req/Sec 37.83 5.64 40.00 84.64%
|
65
|
+
4544 requests in 30.09s, 230.75KB read
|
66
|
+
Requests/sec: 151.00
|
67
|
+
Transfer/sec: 7.67KB
|
68
|
+
```
|
69
|
+
|
70
|
+
We can see we get close to the theoretical throughput given the number of available threads.
|
71
|
+
|
72
|
+
If we start [puma] with more threads, we get increased throughput.
|
73
|
+
|
74
|
+
```
|
75
|
+
% puma -t 128:128
|
76
|
+
Puma starting in single mode...
|
77
|
+
* Version 3.11.2 (ruby 2.5.0-p0), codename: Love Song
|
78
|
+
* Min threads: 128, max threads: 128
|
79
|
+
* Environment: development
|
80
|
+
* Listening on tcp://0.0.0.0:9292
|
81
|
+
Use Ctrl-C to stop
|
82
|
+
|
83
|
+
% wrk -c 512 -t 128 -d 30 http://localhost:9292
|
84
|
+
Running 30s test @ http://localhost:9292
|
85
|
+
128 threads and 512 connections
|
86
|
+
Thread Stats Avg Stdev Max +/- Stdev
|
87
|
+
Latency 153.92ms 27.06ms 597.36ms 95.11%
|
88
|
+
Req/Sec 26.34 9.14 40.00 70.04%
|
89
|
+
24985 requests in 30.10s, 1.24MB read
|
90
|
+
Socket errors: connect 0, read 49, write 0, timeout 0
|
91
|
+
Requests/sec: 830.06
|
92
|
+
Transfer/sec: 42.15KB
|
93
|
+
```
|
94
|
+
|
95
|
+
The theoretical throughput in this case is 128 threads * 10 sequential requests/second. Unfortunately, [puma] in it's current configuration quickly becomes CPU bound:
|
96
|
+
|
97
|
+
```
|
98
|
+
% puma -t 512:512
|
99
|
+
Puma starting in single mode...
|
100
|
+
* Version 3.11.2 (ruby 2.5.0-p0), codename: Love Song
|
101
|
+
* Min threads: 512, max threads: 512
|
102
|
+
* Environment: development
|
103
|
+
* Listening on tcp://0.0.0.0:9292
|
104
|
+
Use Ctrl-C to stop
|
105
|
+
|
106
|
+
% wrk -c 512 -t 128 -d 30 http://localhost:9292
|
107
|
+
Running 30s test @ http://localhost:9292
|
108
|
+
128 threads and 512 connections
|
109
|
+
Thread Stats Avg Stdev Max +/- Stdev
|
110
|
+
Latency 452.46ms 96.97ms 1.71s 80.84%
|
111
|
+
Req/Sec 10.12 5.52 40.00 77.06%
|
112
|
+
23343 requests in 30.10s, 1.16MB read
|
113
|
+
Requests/sec: 775.49
|
114
|
+
Transfer/sec: 39.38KB
|
115
|
+
```
|
116
|
+
|
117
|
+
When running with [falcon], we are limited by the database. Postgres has been configured with up to 1024 connections, and falcon runs one process per available (hyper-)core, 8 in this case. With up to 1024 connections, we could expect an upper bound of 512 connections * 10 sequential requests/second.
|
118
|
+
|
119
|
+
```
|
120
|
+
% falcon --quiet serve --forked
|
121
|
+
|
122
|
+
% wrk -c 512 -t 128 -d 30 http://localhost:9292
|
123
|
+
Running 30s test @ http://localhost:9292
|
124
|
+
128 threads and 512 connections
|
125
|
+
Thread Stats Avg Stdev Max +/- Stdev
|
126
|
+
Latency 158.85ms 20.33ms 263.33ms 68.57%
|
127
|
+
Req/Sec 25.25 8.97 40.00 72.16%
|
128
|
+
96641 requests in 30.10s, 4.52MB read
|
129
|
+
Requests/sec: 3210.90
|
130
|
+
Transfer/sec: 153.65KB
|
131
|
+
```
|
132
|
+
|
133
|
+
We get close to the theoretical 5120 requests/second limit, but at this point the entire test becomes CPU bound within Ruby/Falcon.
|
134
|
+
|
135
|
+
## Usage
|
136
|
+
|
137
|
+
In theory, this is a drop-in replacement for ActiveRecord. But, it must be used with an [async] capable server like [falcon].
|
138
|
+
|
139
|
+
[async]: https://github.com/socketry/async
|
140
|
+
[falcon]: https://github.com/socketry/falcon
|
141
|
+
[puma]: https://github.com/puma/puma
|
142
|
+
|
143
|
+
## Contributing
|
144
|
+
|
145
|
+
1. Fork it
|
146
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
147
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
148
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
149
|
+
5. Create new Pull Request
|
150
|
+
|
151
|
+
## License
|
152
|
+
|
153
|
+
Copyright, 2018, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
154
|
+
|
155
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
156
|
+
of this software and associated documentation files (the "Software"), to deal
|
157
|
+
in the Software without restriction, including without limitation the rights
|
158
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
159
|
+
copies of the Software, and to permit persons to whom the Software is
|
160
|
+
furnished to do so, subject to the following conditions:
|
161
|
+
|
162
|
+
The above copyright notice and this permission notice shall be included in
|
163
|
+
all copies or substantial portions of the Software.
|
164
|
+
|
165
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
166
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
167
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
168
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
169
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
170
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
171
|
+
THE SOFTWARE.
|
data/Rakefile
ADDED
@@ -0,0 +1,27 @@
|
|
1
|
+
|
2
|
+
require_relative 'lib/async/postgres/version'
|
3
|
+
|
4
|
+
Gem::Specification.new do |spec|
|
5
|
+
spec.name = "async-postgres"
|
6
|
+
spec.version = Async::Postgres::VERSION
|
7
|
+
spec.authors = ["Samuel Williams"]
|
8
|
+
spec.email = ["samuel.williams@oriontransfer.co.nz"]
|
9
|
+
spec.summary = %q{Access postgres without blocking.}
|
10
|
+
spec.homepage = "https://github.com/socketry/async-postgres"
|
11
|
+
spec.license = "MIT"
|
12
|
+
|
13
|
+
spec.files = `git ls-files`.split($/)
|
14
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
15
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
16
|
+
spec.require_paths = ["lib"]
|
17
|
+
|
18
|
+
spec.add_dependency "async", "~> 1.3"
|
19
|
+
spec.add_dependency "pg"
|
20
|
+
|
21
|
+
spec.add_development_dependency "async-rspec"
|
22
|
+
spec.add_development_dependency "activerecord"
|
23
|
+
|
24
|
+
spec.add_development_dependency "bundler", "~> 1.3"
|
25
|
+
spec.add_development_dependency "rspec", "~> 3.6"
|
26
|
+
spec.add_development_dependency "rake"
|
27
|
+
end
|
@@ -0,0 +1,12 @@
|
|
1
|
+
|
2
|
+
gem 'pg', '~> 0.18'
|
3
|
+
|
4
|
+
if defined?(Falcon)
|
5
|
+
$stderr.puts "Loading async/postgres"
|
6
|
+
require_relative '../lib/async/postgres'
|
7
|
+
end
|
8
|
+
|
9
|
+
require 'active_record'
|
10
|
+
|
11
|
+
ActiveRecord::Base.establish_connection(adapter: "postgresql", database: "test", pool: 1024)
|
12
|
+
# ActiveRecord::Base.logger = Logger.new(STDOUT)
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# Copyright, 2018, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
+
# of this software and associated documentation files (the "Software"), to deal
|
5
|
+
# in the Software without restriction, including without limitation the rights
|
6
|
+
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
7
|
+
# copies of the Software, and to permit persons to whom the Software is
|
8
|
+
# furnished to do so, subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in
|
11
|
+
# all copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
16
|
+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
17
|
+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
18
|
+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
19
|
+
# THE SOFTWARE.
|
20
|
+
|
21
|
+
require_relative "postgres/version"
|
22
|
+
require_relative "postgres/connection"
|
23
|
+
require_relative "postgres/pool"
|
24
|
+
|
25
|
+
require 'pg'
|
26
|
+
|
27
|
+
module PG
|
28
|
+
def self.connect(connection_string)
|
29
|
+
Async::Postgres::Proxy.new(connection_string)
|
30
|
+
end
|
31
|
+
end
|
@@ -0,0 +1,85 @@
|
|
1
|
+
# Copyright, 2018, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
+
# of this software and associated documentation files (the "Software"), to deal
|
5
|
+
# in the Software without restriction, including without limitation the rights
|
6
|
+
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
7
|
+
# copies of the Software, and to permit persons to whom the Software is
|
8
|
+
# furnished to do so, subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in
|
11
|
+
# all copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
16
|
+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
17
|
+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
18
|
+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
19
|
+
# THE SOFTWARE.
|
20
|
+
|
21
|
+
require 'async/wrapper'
|
22
|
+
|
23
|
+
require 'pg/connection'
|
24
|
+
|
25
|
+
module Async
|
26
|
+
module Postgres
|
27
|
+
class Connection < Wrapper
|
28
|
+
def initialize(connection_string, reactor = nil)
|
29
|
+
@connection = PG::Connection.connect_start(connection_string)
|
30
|
+
|
31
|
+
super(@connection.socket_io, reactor)
|
32
|
+
|
33
|
+
status = @connection.connect_poll
|
34
|
+
|
35
|
+
while true
|
36
|
+
if status == PG::PGRES_POLLING_FAILED
|
37
|
+
raise PG::Error.new(@connection.error_message)
|
38
|
+
elsif status == PG::PGRES_POLLING_READING
|
39
|
+
self.wait_readable
|
40
|
+
elsif(status == PG::PGRES_POLLING_WRITING)
|
41
|
+
self.wait_writable
|
42
|
+
elsif status == PG::PGRES_POLLING_OK
|
43
|
+
break
|
44
|
+
end
|
45
|
+
|
46
|
+
status = @connection.connect_poll
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
def async_exec(*args)
|
51
|
+
@connection.send_query(*args)
|
52
|
+
last_result = result = true
|
53
|
+
|
54
|
+
while true
|
55
|
+
wait_readable
|
56
|
+
|
57
|
+
@connection.consume_input
|
58
|
+
|
59
|
+
while @connection.is_busy == false
|
60
|
+
if result = @connection.get_result
|
61
|
+
last_result = result
|
62
|
+
|
63
|
+
yield result if block_given?
|
64
|
+
else
|
65
|
+
return last_result
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
69
|
+
ensure
|
70
|
+
@connection.get_result until result.nil?
|
71
|
+
end
|
72
|
+
|
73
|
+
alias exec async_exec
|
74
|
+
alias exec_params exec
|
75
|
+
|
76
|
+
def respond_to?(*args)
|
77
|
+
@connection.respond_to(*args)
|
78
|
+
end
|
79
|
+
|
80
|
+
def method_missing(*args)
|
81
|
+
@connection.send(*args)
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
end
|
@@ -0,0 +1,112 @@
|
|
1
|
+
# Copyright, 2018, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
+
# of this software and associated documentation files (the "Software"), to deal
|
5
|
+
# in the Software without restriction, including without limitation the rights
|
6
|
+
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
7
|
+
# copies of the Software, and to permit persons to whom the Software is
|
8
|
+
# furnished to do so, subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in
|
11
|
+
# all copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
16
|
+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
17
|
+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
18
|
+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
19
|
+
# THE SOFTWARE.
|
20
|
+
|
21
|
+
require 'async/reactor'
|
22
|
+
|
23
|
+
module Async
|
24
|
+
class Reactor < Node
|
25
|
+
attr_accessor :postgres_pools
|
26
|
+
end
|
27
|
+
|
28
|
+
module Postgres
|
29
|
+
class Proxy
|
30
|
+
def initialize(connection_string, task: Task.current)
|
31
|
+
@connection_string = connection_string
|
32
|
+
|
33
|
+
pools = task.reactor.postgres_pools ||= {}
|
34
|
+
|
35
|
+
@pool = pools[@connection_string] ||= Pool.new do
|
36
|
+
Connection.new(@connection_string)
|
37
|
+
end
|
38
|
+
end
|
39
|
+
|
40
|
+
def close
|
41
|
+
@pool.close
|
42
|
+
end
|
43
|
+
|
44
|
+
def async_exec(*args)
|
45
|
+
@pool.acquire do |connection|
|
46
|
+
connection.async_exec(*args)
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
def respond_to?(*args)
|
51
|
+
@pool.acquire do |connection|
|
52
|
+
connection.respond_to?(*args)
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
def method_missing(*args, &block)
|
57
|
+
@pool.acquire do |connection|
|
58
|
+
connection.send(*args, &block)
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
62
|
+
|
63
|
+
# This pool doesn't impose a maximum number of open resources, but it WILL block if there are no available resources and trying to allocate another one fails.
|
64
|
+
class Pool
|
65
|
+
def initialize(&block)
|
66
|
+
@available = []
|
67
|
+
@waiting = []
|
68
|
+
|
69
|
+
@constructor = block
|
70
|
+
end
|
71
|
+
|
72
|
+
def acquire
|
73
|
+
resource = wait_for_next_available
|
74
|
+
|
75
|
+
begin
|
76
|
+
yield resource
|
77
|
+
ensure
|
78
|
+
@available << resource
|
79
|
+
|
80
|
+
if task = @waiting.pop
|
81
|
+
task.resume
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
|
86
|
+
def close
|
87
|
+
@available.each(&:close)
|
88
|
+
@available.clear
|
89
|
+
end
|
90
|
+
|
91
|
+
def wait_for_next_available
|
92
|
+
until resource = next_available
|
93
|
+
@waiting << Fiber.current
|
94
|
+
Task.yield
|
95
|
+
end
|
96
|
+
|
97
|
+
return resource
|
98
|
+
end
|
99
|
+
|
100
|
+
def next_available
|
101
|
+
if @available.empty?
|
102
|
+
return @constructor.call # This might fail, which is okay :)
|
103
|
+
else
|
104
|
+
return @available.pop
|
105
|
+
end
|
106
|
+
rescue StandardError
|
107
|
+
$stderr.puts $!.inspect
|
108
|
+
return nil
|
109
|
+
end
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end
|
@@ -0,0 +1,25 @@
|
|
1
|
+
# Copyright, 2018, by Samuel G. D. Williams. <http://www.codeotaku.com>
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
+
# of this software and associated documentation files (the "Software"), to deal
|
5
|
+
# in the Software without restriction, including without limitation the rights
|
6
|
+
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
7
|
+
# copies of the Software, and to permit persons to whom the Software is
|
8
|
+
# furnished to do so, subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in
|
11
|
+
# all copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
16
|
+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
17
|
+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
18
|
+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
19
|
+
# THE SOFTWARE.
|
20
|
+
|
21
|
+
module Async
|
22
|
+
module Postgres
|
23
|
+
VERSION = "0.1.0"
|
24
|
+
end
|
25
|
+
end
|
@@ -0,0 +1,31 @@
|
|
1
|
+
|
2
|
+
require 'active_record'
|
3
|
+
|
4
|
+
class Book < ActiveRecord::Base
|
5
|
+
end
|
6
|
+
|
7
|
+
# Good ol' ActiveRecord and it's global state
|
8
|
+
# https://tenderlovemaking.com/2011/10/20/connection-management-in-activerecord.html
|
9
|
+
|
10
|
+
RSpec.describe ActiveRecord do
|
11
|
+
include_context Async::RSpec::Reactor
|
12
|
+
|
13
|
+
it "should work using async adapter" do
|
14
|
+
reactor.async do
|
15
|
+
ActiveRecord::Base.establish_connection(adapter: "postgresql", database: "test")
|
16
|
+
ActiveRecord::Base.logger = Logger.new(STDOUT)
|
17
|
+
|
18
|
+
ActiveRecord::Schema.define do
|
19
|
+
create_table :books, force: true do |t|
|
20
|
+
t.string :name
|
21
|
+
t.timestamps
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
Book.create(name: "How to use a fork")
|
26
|
+
|
27
|
+
# This closes the underlying connection.
|
28
|
+
ActiveRecord::Base.remove_connection
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
@@ -0,0 +1,19 @@
|
|
1
|
+
|
2
|
+
require 'async/postgres/connection'
|
3
|
+
|
4
|
+
RSpec.describe Async::Postgres::Connection do
|
5
|
+
include_context Async::RSpec::Reactor
|
6
|
+
|
7
|
+
let(:connection_string) {"host=localhost dbname=test"}
|
8
|
+
let(:connection) {Async::Postgres::Connection.new(connection_string)}
|
9
|
+
|
10
|
+
it "should execute query" do
|
11
|
+
reactor.async do
|
12
|
+
results = connection.async_exec("SELECT 42 AS LIFE")
|
13
|
+
|
14
|
+
expect(results.each.to_a).to be == [{"life" => "42"}]
|
15
|
+
|
16
|
+
connection.close
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1,54 @@
|
|
1
|
+
|
2
|
+
if ENV['COVERAGE']
|
3
|
+
begin
|
4
|
+
require 'simplecov'
|
5
|
+
|
6
|
+
SimpleCov.start do
|
7
|
+
add_filter "/spec/"
|
8
|
+
end
|
9
|
+
|
10
|
+
if ENV['TRAVIS']
|
11
|
+
require 'coveralls'
|
12
|
+
Coveralls.wear!
|
13
|
+
end
|
14
|
+
rescue LoadError
|
15
|
+
warn "Could not load simplecov: #{$!}"
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
require "bundler/setup"
|
20
|
+
require "async/postgres"
|
21
|
+
require 'async/rspec'
|
22
|
+
|
23
|
+
begin
|
24
|
+
require 'ruby-prof'
|
25
|
+
|
26
|
+
RSpec.shared_context "profile" do
|
27
|
+
before(:all) do
|
28
|
+
RubyProf.start
|
29
|
+
end
|
30
|
+
|
31
|
+
after(:all) do
|
32
|
+
result = RubyProf.stop
|
33
|
+
|
34
|
+
# Print a flat profile to text
|
35
|
+
printer = RubyProf::FlatPrinter.new(result)
|
36
|
+
printer.print(STDOUT)
|
37
|
+
end
|
38
|
+
end
|
39
|
+
rescue LoadError
|
40
|
+
RSpec.shared_context "profile" do
|
41
|
+
before(:all) do
|
42
|
+
puts "Profiling not supported on this platform."
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
RSpec.configure do |config|
|
48
|
+
# Enable flags like --only-failures and --next-failure
|
49
|
+
config.example_status_persistence_file_path = ".rspec_status"
|
50
|
+
|
51
|
+
config.expect_with :rspec do |c|
|
52
|
+
c.syntax = :expect
|
53
|
+
end
|
54
|
+
end
|
metadata
ADDED
@@ -0,0 +1,161 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: async-postgres
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Samuel Williams
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2018-10-22 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: async
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '1.3'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '1.3'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: pg
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: async-rspec
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: activerecord
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - ">="
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ">="
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: bundler
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '1.3'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '1.3'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: rspec
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - "~>"
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '3.6'
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - "~>"
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '3.6'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: rake
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - ">="
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '0'
|
104
|
+
type: :development
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - ">="
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '0'
|
111
|
+
description:
|
112
|
+
email:
|
113
|
+
- samuel.williams@oriontransfer.co.nz
|
114
|
+
executables: []
|
115
|
+
extensions: []
|
116
|
+
extra_rdoc_files: []
|
117
|
+
files:
|
118
|
+
- ".gitignore"
|
119
|
+
- ".rspec"
|
120
|
+
- ".travis.yml"
|
121
|
+
- Gemfile
|
122
|
+
- README.md
|
123
|
+
- Rakefile
|
124
|
+
- async-postgres.gemspec
|
125
|
+
- examples/rack/config.ru
|
126
|
+
- examples/rack/schema.rb
|
127
|
+
- lib/async/postgres.rb
|
128
|
+
- lib/async/postgres/connection.rb
|
129
|
+
- lib/async/postgres/pool.rb
|
130
|
+
- lib/async/postgres/version.rb
|
131
|
+
- spec/async/activerecord_spec.rb
|
132
|
+
- spec/async/postgres_spec.rb
|
133
|
+
- spec/spec_helper.rb
|
134
|
+
homepage: https://github.com/socketry/async-postgres
|
135
|
+
licenses:
|
136
|
+
- MIT
|
137
|
+
metadata: {}
|
138
|
+
post_install_message:
|
139
|
+
rdoc_options: []
|
140
|
+
require_paths:
|
141
|
+
- lib
|
142
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
143
|
+
requirements:
|
144
|
+
- - ">="
|
145
|
+
- !ruby/object:Gem::Version
|
146
|
+
version: '0'
|
147
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
148
|
+
requirements:
|
149
|
+
- - ">="
|
150
|
+
- !ruby/object:Gem::Version
|
151
|
+
version: '0'
|
152
|
+
requirements: []
|
153
|
+
rubyforge_project:
|
154
|
+
rubygems_version: 2.7.7
|
155
|
+
signing_key:
|
156
|
+
specification_version: 4
|
157
|
+
summary: Access postgres without blocking.
|
158
|
+
test_files:
|
159
|
+
- spec/async/activerecord_spec.rb
|
160
|
+
- spec/async/postgres_spec.rb
|
161
|
+
- spec/spec_helper.rb
|