activerecord-futures 0.0.1 → 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (35) hide show
  1. data/.travis.yml +20 -0
  2. data/Gemfile +11 -1
  3. data/README.md +43 -23
  4. data/Rakefile +17 -0
  5. data/activerecord-futures.gemspec +3 -1
  6. data/lib/active_record/connection_adapters/future_enabled.rb +34 -0
  7. data/lib/active_record/connection_adapters/future_enabled_mysql2_adapter.rb +23 -22
  8. data/lib/active_record/connection_adapters/future_enabled_postgresql_adapter.rb +52 -0
  9. data/lib/active_record/futures.rb +10 -9
  10. data/lib/active_record/futures/delegation.rb +1 -0
  11. data/lib/active_record/futures/future.rb +22 -6
  12. data/lib/active_record/futures/future_calculation.rb +4 -12
  13. data/lib/active_record/futures/future_calculation_array.rb +8 -0
  14. data/lib/active_record/futures/future_calculation_value.rb +11 -0
  15. data/lib/active_record/futures/future_relation.rb +12 -11
  16. data/lib/active_record/futures/proxy.rb +37 -0
  17. data/lib/active_record/futures/query_recording.rb +12 -26
  18. data/lib/activerecord-futures.rb +3 -0
  19. data/lib/activerecord-futures/version.rb +1 -1
  20. data/spec/active_record/futures/future_relation_spec.rb +9 -1
  21. data/spec/active_record/futures/proxy_spec.rb +51 -0
  22. data/spec/db/schema.rb +15 -6
  23. data/spec/in_action/combination_of_futures_spec.rb +153 -0
  24. data/spec/in_action/future_count_execution_spec.rb +98 -0
  25. data/spec/in_action/future_fulfillment_spec.rb +10 -7
  26. data/spec/in_action/future_pluck_execution_spec.rb +44 -0
  27. data/spec/in_action/future_relation_execution_spec.rb +52 -0
  28. data/spec/models/comment.rb +4 -0
  29. data/spec/models/post.rb +3 -0
  30. data/spec/models/user.rb +1 -0
  31. data/spec/spec_helper.rb +45 -9
  32. data/spec/support/matchers/exec.rb +66 -0
  33. data/spec/support/matchers/exec_query.rb +23 -0
  34. metadata +60 -6
  35. data/spec/models/country.rb +0 -2
data/.travis.yml ADDED
@@ -0,0 +1,20 @@
1
+ language: ruby
2
+ rvm:
3
+ - 1.9.3
4
+ env:
5
+ - ADAPTER=future_enabled_mysql2 activerecord=3.2.11
6
+ - ADAPTER=future_enabled_mysql2 activerecord=3.2.12
7
+ - ADAPTER=future_enabled_mysql2 activerecord=3.2.13
8
+ - ADAPTER=mysql2 activerecord=3.2.11
9
+ - ADAPTER=mysql2 activerecord=3.2.12
10
+ - ADAPTER=mysql2 activerecord=3.2.13
11
+ - ADAPTER=future_enabled_postgresql activerecord=3.2.11
12
+ - ADAPTER=future_enabled_postgresql activerecord=3.2.12
13
+ - ADAPTER=future_enabled_postgresql activerecord=3.2.13
14
+ - ADAPTER=postgresql activerecord=3.2.11
15
+ - ADAPTER=postgresql activerecord=3.2.12
16
+ - ADAPTER=postgresql activerecord=3.2.13
17
+
18
+ before_script:
19
+ - mysql -e 'create database activerecord_futures_test;'
20
+ - psql -c 'create database activerecord_futures_test;' -U postgres
data/Gemfile CHANGED
@@ -1,4 +1,14 @@
1
- source 'https://rubygems.org'
1
+ source 'http://rubygems.org'
2
2
 
3
3
  # Specify your gem's dependencies in activerecord-futures.gemspec
4
4
  gemspec
5
+
6
+ group :test do
7
+ gem 'rake'
8
+ end
9
+
10
+ gem 'coveralls', require: false
11
+
12
+ if ENV['activerecord']
13
+ gem "activerecord", ENV['activerecord']
14
+ end
data/README.md CHANGED
@@ -1,7 +1,21 @@
1
1
  # ActiveRecord::Futures
2
2
 
3
+ [![Build Status](https://travis-ci.org/leoasis/activerecord-futures.png)](https://travis-ci.org/leoasis/activerecord-futures)
4
+ [![Code Climate](https://codeclimate.com/github/leoasis/activerecord-futures.png)](https://codeclimate.com/github/leoasis/activerecord-futures)
5
+ [![Coverage Status](https://coveralls.io/repos/leoasis/activerecord-futures/badge.png?branch=master)](https://coveralls.io/r/leoasis/activerecord-futures)
6
+
7
+
3
8
  Define future queries in ActiveRecord that will get executed in a single round trip to the database.
4
9
 
10
+ This gem allows to easily optimize an application using activerecord. All
11
+ independent queries can be marked as futures, so that when you execute any of
12
+ them at a later time, all the other ones will be executed as well, but the query
13
+ of all of them will be executed in a single round trip to the database. That way,
14
+ when you access the other results, they'll already be there, not needing to go
15
+ to the database again.
16
+
17
+ The idea is heavily inspired from [NHibernate's future queries](http://ayende.com/blog/3979/nhibernate-futures)
18
+
5
19
  ## Installation
6
20
 
7
21
  Add this line to your application's Gemfile:
@@ -18,13 +32,11 @@ Or install it yourself as:
18
32
 
19
33
  ## Usage
20
34
 
21
- Currently, the only database supported is MySQL, and with a special adapter, provided by the gem.
22
-
23
- Set your config/database.yml file to use the given adapter:
35
+ Once the gem is installed, set your config/database.yml file to use a future enabled adapter:
24
36
 
25
37
  ```yml
26
38
  development: &development
27
- adapter: future_enabled_mysql2 # set this adapter for futures to work!
39
+ adapter: future_enabled_mysql2 # or "future_enabled_postgresql"
28
40
  username: your_username
29
41
  password: your_password
30
42
  database: your_database
@@ -34,19 +46,24 @@ development: &development
34
46
  Now let's see what this does, consider a model `User`, with a `:name` attribute:
35
47
 
36
48
  ```ruby
49
+
37
50
  # Build the queries and mark them as futures
38
- users = User.where("name like 'John%'").future # becomes a future relation, does not execute the query.
39
- count = User.where("name like 'John%'").future_count # becomes a future calculation, does not execute the query.
51
+ users = User.where("name like 'John%'")
52
+ user_list = users.future # becomes a future relation, does not execute the query.
53
+ count = users.future_count # becomes a future calculation, does not execute the query.
40
54
 
41
55
  # Execute any of the futures
42
56
  count = count.value # trigger the future execution, both queries will get executed in one round trip!
43
57
  #=> User Load (fetching Futures) (0.6ms) SELECT `users`.* FROM `users` WHERE (name like 'John%');SELECT COUNT(*) FROM `users` WHERE (name like 'John%')
44
58
 
45
59
  # Access the other results
46
- users = users.to_a # does not execute the query, results from previous query get loaded
60
+ user_list.to_a # does not execute the query, results from previous query get loaded
47
61
  ```
48
62
 
49
- Any amount of futures can be prepared, and the will get executed as soon as one of them needs to be evaluated.
63
+ Any amount of futures can be prepared, and they will get executed as soon as one of them needs to be evaluated.
64
+
65
+ This makes this especially useful for pagination queries, since you can execute
66
+ both count and page queries at once.
50
67
 
51
68
  ### Methods
52
69
 
@@ -55,28 +72,37 @@ executed whenever `#to_a` gets executed. Note that, as ActiveRecord does, enumer
55
72
  so things like `#each`, `#map`, `#collect` all trigger the future.
56
73
 
57
74
  Also, ActiveRecord::Relation instances get all the calculation methods provided by the ActiveRecord::Calculations module
58
- "futurized", that means, for `#count` you get `#future_count`, for `#sum` you get `#future_sum` and so on. These future
59
- calculations are triggered by executing the `#value` method, which also return the actual result of the calculation.
75
+ "futurized", that means, for `#count` you get `#future_count`, for `#sum` you get `#future_sum` and so on. If the calculation
76
+ returns a list of values, for example with a `#future_pluck` or a grouped `#future_count`, the future will be triggered with
77
+ the `#to_a` method (or any of the methods that delegate to `#to_a`). If it returns a single value, the future will be
78
+ triggered when you execute the `#value` method.
60
79
 
61
80
  ## Database support
62
81
 
63
82
  ### SQlite
64
83
 
65
- SQlite doesn't support multiple statement queries. Currently this gem doesn't fall back to the normal behavior if the
66
- adapter does not support futures, but this is in the road map :)
84
+ SQlite doesn't support multiple statement queries. ActiveRecord::Futures will fall back to normal query execution, that is,
85
+ it will execute the future's query whenever the future is triggered, but it will not execute the other futures' queries.
67
86
 
68
87
  ### MySQL
69
88
 
70
89
  Multi statement queries are supported by the mysql2 gem since version 0.3.12b1, so you'll need to use that one or a newer
71
90
  one.
72
- Currently the adapter provided is the same as the built-in in Rails, but it also sets the MULTI_STATEMENTS flag to allow
73
- multiple queries in a single command. It also has a special way to
74
- execute the queries in order to fetch the results correctly. You
75
- can check the code if you're curious!
91
+ Currently the adapter provided inherits the built-in one in Rails, and it also sets the MULTI_STATEMENTS flag to allow
92
+ multiple queries in a single command.
93
+ If you have an older version of the gem, ActiveRecord::Futures will fall back to normal query execution.
76
94
 
77
95
  ### Postgres
78
96
 
79
- Coming soon!
97
+ The pg gem supports multiple statement queries by using the `send_query` method
98
+ and retrieving the results via `get_result`.
99
+
100
+ ### Other databases
101
+
102
+ In general, ActiveRecord::Futures will look for a method `#supports_futures?` in the adapter. So any adapter that returns
103
+ false when calling the method, or does not respond to it, will fall back to normal query execution.
104
+ If you want to have support for ActiveRecord::Futures with your database, feel free to create a pull request with it, or
105
+ create your own gem, or just create an issue.
80
106
 
81
107
  ## Contributing
82
108
 
@@ -85,9 +111,3 @@ Coming soon!
85
111
  3. Commit your changes (`git commit -am 'Add some feature'`)
86
112
  4. Push to the branch (`git push origin my-new-feature`)
87
113
  5. Create new Pull Request
88
-
89
- ## Roadmap
90
-
91
- 1. Support for postgres
92
- 2. Fallback to normal queries when adapter does not support futures
93
- 3. Think of a way to use the normal adapters
data/Rakefile CHANGED
@@ -1 +1,18 @@
1
1
  require "bundler/gem_tasks"
2
+ require 'rspec/core/rake_task'
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
7
+
8
+ ADAPTERS = %w(future_enabled_mysql2 future_enabled_postgresql postgresql mysql2)
9
+
10
+ desc "Runs the specs with all databases"
11
+ task :all do
12
+ success = true
13
+ ADAPTERS.each do |adapter|
14
+ status = system({ "ADAPTER" => adapter }, "bundle exec rspec")
15
+ success &&= status
16
+ end
17
+ abort unless success
18
+ end
@@ -17,7 +17,9 @@ Gem::Specification.new do |gem|
17
17
  gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
18
18
  gem.require_paths = ["lib"]
19
19
 
20
- gem.add_dependency 'activerecord', '>= 3.2.13'
20
+ gem.add_dependency 'activerecord', '>= 3.2.11'
21
21
  gem.add_development_dependency 'rspec'
22
22
  gem.add_development_dependency 'rspec-spies'
23
+ gem.add_development_dependency 'mysql2', '>= 0.3.12.b1'
24
+ gem.add_development_dependency 'pg'
23
25
  end
@@ -0,0 +1,34 @@
1
+ module ActiveRecord
2
+ module ConnectionAdapters
3
+ module FutureEnabled
4
+ def supports_futures?
5
+ true
6
+ end
7
+
8
+ def exec_query(sql, name = 'SQL', binds = [])
9
+ my_future = Futures::Future.current
10
+
11
+ # default behavior if not a current future or not executing
12
+ # the current future's sql (some adapters like PostgreSQL
13
+ # may execute some attribute queries during a relation evaluation)
14
+ return super unless my_future && my_future.to_sql == sql
15
+
16
+ # return fulfilled result, if exists, to load the relation
17
+ return my_future.result if my_future.fulfilled?
18
+
19
+ futures = Futures::Future.all
20
+ futures_sql = futures.map(&:to_sql).join(';')
21
+ name = "#{name} (fetching Futures)"
22
+
23
+ result = future_execute(futures_sql, name)
24
+
25
+ futures.each do |future|
26
+ future.fulfill(build_active_record_result(result))
27
+ result = next_result
28
+ end
29
+
30
+ my_future.result
31
+ end
32
+ end
33
+ end
34
+ end
@@ -1,5 +1,5 @@
1
1
  require "active_record/connection_adapters/mysql2_adapter"
2
-
2
+ require "active_record/connection_adapters/future_enabled"
3
3
  module ActiveRecord
4
4
  class Base
5
5
  def self.future_enabled_mysql2_connection(config)
@@ -18,33 +18,34 @@ module ActiveRecord
18
18
 
19
19
  module ConnectionAdapters
20
20
  class FutureEnabledMysql2Adapter < Mysql2Adapter
21
+ include FutureEnabled
22
+
23
+ def initialize(*args)
24
+ super
25
+ unless supports_futures?
26
+ logger.warn("ActiveRecord::Futures - You're using the mysql2 future "\
27
+ "enabled adapter with an old version of the mysql2 gem. You must "\
28
+ "use a mysql2 gem version higher than or equal to 0.3.12b1 to take "\
29
+ "advantage of futures.\nFalling back to normal query execution behavior.")
30
+ end
31
+ end
21
32
 
22
33
  def supports_futures?
23
- true
34
+ # Support only if the mysql client allows fetching multiple statements
35
+ # results
36
+ @connection.respond_to?(:store_result)
24
37
  end
25
38
 
26
- def exec_query(sql, name = 'SQL', binds = [])
27
- my_future = Futures::Future.current
28
-
29
- # default behavior if not a current future
30
- return super unless my_future
31
-
32
- # return fulfilled result, if exists, to load the relation
33
- return my_future.result if my_future.fulfilled?
34
-
35
- futures = Futures::Future.all
36
-
37
- futures_sql = futures.map(&:to_sql).join(';')
38
- name = "#{name} (fetching Futures)"
39
-
40
- result = execute(futures_sql, name)
39
+ def future_execute(sql, name)
40
+ execute(sql, name)
41
+ end
41
42
 
42
- futures.each do |future|
43
- future.fulfill(ActiveRecord::Result.new(result.fields, result.to_a))
44
- result = @connection.store_result if @connection.next_result
45
- end
43
+ def build_active_record_result(raw_result)
44
+ ActiveRecord::Result.new(raw_result.fields, raw_result.to_a)
45
+ end
46
46
 
47
- my_future.result
47
+ def next_result
48
+ @connection.store_result if @connection.next_result
48
49
  end
49
50
  end
50
51
  end
@@ -0,0 +1,52 @@
1
+ require 'active_record/connection_adapters/postgresql_adapter'
2
+ require "active_record/connection_adapters/future_enabled"
3
+
4
+ module ActiveRecord
5
+ class Base
6
+ # Establishes a connection to the database that's used by all Active Record objects
7
+ def self.future_enabled_postgresql_connection(config) # :nodoc:
8
+ config = config.symbolize_keys
9
+ host = config[:host]
10
+ port = config[:port] || 5432
11
+ username = config[:username].to_s if config[:username]
12
+ password = config[:password].to_s if config[:password]
13
+
14
+ if config.key?(:database)
15
+ database = config[:database]
16
+ else
17
+ raise ArgumentError, "No database specified. Missing argument: database."
18
+ end
19
+
20
+ # The postgres drivers don't allow the creation of an unconnected PGconn object,
21
+ # so just pass a nil connection object for the time being.
22
+ ConnectionAdapters::FutureEnabledPostgreSQLAdapter.new(nil, logger, [host, port, nil, nil, database, username, password], config)
23
+ end
24
+ end
25
+
26
+ module ConnectionAdapters
27
+ class FutureEnabledPostgreSQLAdapter < PostgreSQLAdapter
28
+ include FutureEnabled
29
+
30
+ def future_execute(sql, name)
31
+ log(sql, name) do
32
+ # Clear the queue
33
+ @connection.get_last_result
34
+ @connection.send_query(sql)
35
+ @connection.block
36
+ @connection.get_result
37
+ end
38
+ end
39
+
40
+ def build_active_record_result(raw_result)
41
+ return if raw_result.nil?
42
+ result = ActiveRecord::Result.new(raw_result.fields, result_as_array(raw_result))
43
+ raw_result.clear
44
+ result
45
+ end
46
+
47
+ def next_result
48
+ @connection.get_result
49
+ end
50
+ end
51
+ end
52
+ end
@@ -3,7 +3,7 @@ module ActiveRecord
3
3
  include QueryRecording
4
4
 
5
5
  def self.original_calculation_methods
6
- ActiveRecord::Calculations.public_instance_methods
6
+ [:count, :average, :minimum, :maximum, :sum, :calculate, :pluck]
7
7
  end
8
8
 
9
9
  def self.future_calculation_methods
@@ -11,12 +11,7 @@ module ActiveRecord
11
11
  end
12
12
 
13
13
  def future
14
- supports_futures = connection.respond_to?(:supports_futures?) &&
15
- connection.supports_futures?
16
-
17
- # simply pass through if the connection adapter does not support
18
- # futures
19
- supports_futures ? FutureRelation.new(self) : self
14
+ FutureRelation.new(self)
20
15
  end
21
16
 
22
17
  method_table = Hash[future_calculation_methods.zip(original_calculation_methods)]
@@ -26,8 +21,14 @@ module ActiveRecord
26
21
  method_table.each do |future_method, method|
27
22
  define_method(future_method) do |*args, &block|
28
23
  exec = lambda { send(method, *args, &block) }
29
- query = record_query(&exec)
30
- FutureCalculation.new(query, exec)
24
+ query, type = record_query(&exec)
25
+
26
+ case type
27
+ when :value
28
+ FutureCalculationValue.new(self, query, exec)
29
+ when :all
30
+ FutureCalculationArray.new(self, query, exec)
31
+ end
31
32
  end
32
33
  end
33
34
  end
@@ -2,6 +2,7 @@ module ActiveRecord
2
2
  module Futures
3
3
  module Delegation
4
4
  delegate :future, to: :scoped
5
+ delegate :future_pluck, to: :scoped
5
6
  delegate *Futures.future_calculation_methods, to: :scoped
6
7
  end
7
8
  end
@@ -27,12 +27,23 @@ module ActiveRecord
27
27
  self.futures.each(&:load)
28
28
  clear
29
29
  end
30
+
31
+ private
32
+ def fetch_with(method)
33
+ define_method(method) do
34
+ # Flush all the futures upon first attempt to exec a future
35
+ Future.flush unless executed?
36
+ execute
37
+ end
38
+ end
30
39
  end
31
40
 
32
41
 
33
- attr_reader :result
42
+ attr_reader :result, :relation
43
+ private :relation
34
44
 
35
- def initialize
45
+ def initialize(relation)
46
+ @relation = relation
36
47
  Future.register(self)
37
48
  end
38
49
 
@@ -45,15 +56,15 @@ module ActiveRecord
45
56
  end
46
57
 
47
58
  def load
59
+ # Only perform a load if the adapter supports futures.
60
+ # This allows to fallback to normal query execution in futures
61
+ # when the adapter does not support futures.
62
+ return unless connection_supports_futures?
48
63
  Future.current = self
49
64
  execute
50
65
  Future.current = nil
51
66
  end
52
67
 
53
- def inspect
54
- to_a.inspect
55
- end
56
-
57
68
  def to_sql
58
69
  end
59
70
  undef_method :to_sql
@@ -67,6 +78,11 @@ module ActiveRecord
67
78
  end
68
79
  undef_method :executed?
69
80
 
81
+ def connection_supports_futures?
82
+ conn = relation.connection
83
+ conn.respond_to?(:supports_futures?) && conn.supports_futures?
84
+ end
85
+
70
86
  end
71
87
  end
72
88
  end