preload_pluck 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +88 -49
- data/lib/preload_pluck/version.rb +1 -1
- data/lib/preload_pluck.rb +147 -118
- metadata +15 -15
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 8dcb60367b14df2613159448fec6c4b5cbfbf655
|
|
4
|
+
data.tar.gz: a716b978a51994de1ab2c587946a8edfd7efc860
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 95384d11406759a070fd841474c0f32e42a7f34987aa0990e1ee372ed57c8c6d397372e43afa76d9610361948455d7a260883dc15147fafbaa0db72edb08c670
|
|
7
|
+
data.tar.gz: 8a3ac28dc9621347ce23e217cd36131c3c0b34e96ac99bb98261cd801f89d5aad23b20f7e6d18b4ac030a3521fb577d1c77568ce3ce04280f9370a816c90a90e
|
data/README.md
CHANGED
|
@@ -1,49 +1,88 @@
|
|
|
1
|
-
# Preload Pluck
|
|
2
|
-
|
|
3
|
-
[](https://travis-ci.org/assetricity/preload_pluck)
|
|
4
|
-
[](https://travis-ci.org/assetricity/preload_pluck)
|
|
4
|
+
[](https://coveralls.io/r/assetricity/preload_pluck?branch=master)
|
|
5
|
+
[](https://gemnasium.com/assetricity/preload_pluck)
|
|
6
|
+
|
|
7
|
+
Adds a `preload_pluck` method to ActiveRecord that allows querying using Rails 4 eager loading-style for joined tables (`preload`), and returns a 2-dimensional array without ActiveRecord model creation overhead (`pluck`).
|
|
8
|
+
|
|
9
|
+
The typical use case is for querying and displaying tabular data, such as on an index page, without any further manipulation needed by the involved ActiveRecord models. Data may originate from immediate attributes on the current model or from attributes from other models associated via `belongs_to`.
|
|
10
|
+
|
|
11
|
+
Note: Preload Pluck may not always increase query performance - always benchmark with your own queries and production data.
|
|
12
|
+
|
|
13
|
+
## Install
|
|
14
|
+
|
|
15
|
+
Add to the preload_pluck gem to your Gemfile:
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
gem 'preload_pluck'
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Usage
|
|
22
|
+
|
|
23
|
+
Call `preload_pluck` on an ActiveRecord model:
|
|
24
|
+
|
|
25
|
+
```ruby
|
|
26
|
+
Comment.order(:created_at).preload_pluck(:text, 'user.name')
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
This will return a 2-dimensional array where columns correspond to the passed arguments:
|
|
30
|
+
|
|
31
|
+
```ruby
|
|
32
|
+
[
|
|
33
|
+
['That was an interesting post', 'Alice']
|
|
34
|
+
['I thought so too', 'Bob']
|
|
35
|
+
]
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Attributes on the current model can be supplied by name:
|
|
39
|
+
|
|
40
|
+
```ruby
|
|
41
|
+
Comment.preload_pluck(:title, :text)
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Nested attributes should be separated by a period:
|
|
45
|
+
|
|
46
|
+
```ruby
|
|
47
|
+
Comment.preload_pluck('post.title', 'post.text')
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Both immediate and nested attributes can be mixed:
|
|
51
|
+
|
|
52
|
+
```ruby
|
|
53
|
+
Comment.preload_pluck(:title, :text, 'post.title', 'post.text')
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Any SQL conditions (e.g. where clauses, scopes, orders, limits) should be set before `preload_pluck` is called:
|
|
57
|
+
|
|
58
|
+
```ruby
|
|
59
|
+
Comment.order(:created_at)
|
|
60
|
+
.joins(:user)
|
|
61
|
+
.where(user: {name: 'Alice'))
|
|
62
|
+
.preload_pluck(:title, :text, 'post.title', 'post.text')
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
See `spec/preload_pluck_spec.rb` for more examples.
|
|
66
|
+
|
|
67
|
+
## Running Tests
|
|
68
|
+
|
|
69
|
+
SQLite must be installed before running tests.
|
|
70
|
+
|
|
71
|
+
To run tests:
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
bundle
|
|
75
|
+
rspec
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
By default, performance tests are disabled as it takes several minutes to insert data. To run performance tests:
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
rspec --tag performance
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## License
|
|
85
|
+
|
|
86
|
+
Copyright [Assetricity, LLC](http://assetricity.com)
|
|
87
|
+
|
|
88
|
+
Preload Pluck is released under the MIT License. See [LICENSE](https://github.com/assetricity/preload_pluck/blob/master/LICENSE) for details.
|
data/lib/preload_pluck.rb
CHANGED
|
@@ -1,118 +1,147 @@
|
|
|
1
|
-
require 'preload_pluck/version'
|
|
2
|
-
require 'active_record'
|
|
3
|
-
|
|
4
|
-
module PreloadPluck
|
|
5
|
-
class Field < Struct.new(:base_class, :path)
|
|
6
|
-
def nested?(level)
|
|
7
|
-
level + 2 <= path.length
|
|
8
|
-
end
|
|
9
|
-
|
|
10
|
-
def level?(level)
|
|
11
|
-
!!path[level]
|
|
12
|
-
end
|
|
13
|
-
|
|
14
|
-
def path_upto(level)
|
|
15
|
-
path[0..level].join('.')
|
|
16
|
-
end
|
|
17
|
-
|
|
18
|
-
def assoc(level)
|
|
19
|
-
return nil if level >= path.length
|
|
20
|
-
current_class = base_class
|
|
21
|
-
assoc = nil
|
|
22
|
-
(level + 1).times do |l|
|
|
23
|
-
assoc = current_class.reflect_on_association(path[l])
|
|
24
|
-
raise 'preload_pluck only supports belongs_to associations' unless assoc.macro == :belongs_to
|
|
25
|
-
current_class = assoc.class_name.constantize
|
|
26
|
-
end
|
|
27
|
-
assoc
|
|
28
|
-
end
|
|
29
|
-
end
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
1
|
+
require 'preload_pluck/version'
|
|
2
|
+
require 'active_record'
|
|
3
|
+
|
|
4
|
+
module PreloadPluck
|
|
5
|
+
class Field < Struct.new(:base_class, :path)
|
|
6
|
+
def nested?(level)
|
|
7
|
+
level + 2 <= path.length
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
def level?(level)
|
|
11
|
+
!!path[level]
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def path_upto(level)
|
|
15
|
+
path[0..level].join('.')
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def assoc(level)
|
|
19
|
+
return nil if level >= path.length
|
|
20
|
+
current_class = base_class
|
|
21
|
+
assoc = nil
|
|
22
|
+
(level + 1).times do |l|
|
|
23
|
+
assoc = current_class.reflect_on_association(path[l])
|
|
24
|
+
raise 'preload_pluck only supports belongs_to associations' unless assoc.macro == :belongs_to
|
|
25
|
+
current_class = assoc.class_name.constantize
|
|
26
|
+
end
|
|
27
|
+
assoc
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
# Return a 2-dimensional array of values where columns correspond to supplied arguments. Data from associations is
|
|
32
|
+
# eager loaded.
|
|
33
|
+
#
|
|
34
|
+
# Attributes on the current model can be supplied by name:
|
|
35
|
+
#
|
|
36
|
+
# Comment.preload_pluck(:title, :text)
|
|
37
|
+
#
|
|
38
|
+
# Nested attributes should be separated by a period:
|
|
39
|
+
#
|
|
40
|
+
# Comment.preload_pluck('post.title', 'post.text')
|
|
41
|
+
#
|
|
42
|
+
# Both immediate and nested attributes can be mixed:
|
|
43
|
+
#
|
|
44
|
+
# Comment.preload_pluck(:title, :text, 'post.title', 'post.text')
|
|
45
|
+
#
|
|
46
|
+
# Any SQL conditions should be set before `preload_pluck` is called:
|
|
47
|
+
#
|
|
48
|
+
# Comment.order(:created_at)
|
|
49
|
+
# .joins(:user)
|
|
50
|
+
# .where(user: {name: 'Alice'))
|
|
51
|
+
# .preload_pluck(:title, :text, 'post.title', 'post.text')
|
|
52
|
+
#
|
|
53
|
+
# @param args [Array<Symbol or String>] list of immediate and/or nested model attributes.
|
|
54
|
+
# @return [Array<Array>] 2-dimensional array where columns correspond to supplied arguments.
|
|
55
|
+
def preload_pluck(*args)
|
|
56
|
+
fields = args.map {|arg| Field.new(self, arg.to_s.split('.'))}
|
|
57
|
+
|
|
58
|
+
plucked_cols = fields.map do |field|
|
|
59
|
+
if field.nested?(0)
|
|
60
|
+
field.assoc(0).foreign_key
|
|
61
|
+
else
|
|
62
|
+
field.path.last
|
|
63
|
+
end
|
|
64
|
+
end.uniq
|
|
65
|
+
data = pluck(*plucked_cols)
|
|
66
|
+
if fields.length == 1
|
|
67
|
+
# Pluck returns a flat array if only one value, so use a consistent structure if there is one or multiple fields
|
|
68
|
+
data.map! {|val| [val]}
|
|
69
|
+
end
|
|
70
|
+
data_headers = __preload_pluck_to_header_lookup(plucked_cols)
|
|
71
|
+
|
|
72
|
+
# A cache of records that we populate then join to later based on foreign keys
|
|
73
|
+
nested_data = {}
|
|
74
|
+
|
|
75
|
+
# Incrementally process nested fields by level
|
|
76
|
+
max_level = fields.map {|f| f.path.length - 1}.max
|
|
77
|
+
max_level.times do |level|
|
|
78
|
+
fields.select {|f| f.nested?(level)}
|
|
79
|
+
.group_by {|f| f.path_upto(level)}
|
|
80
|
+
.each do |current_path, group|
|
|
81
|
+
# Just use the first item - could use any item in the group
|
|
82
|
+
assoc = group.first.assoc(level)
|
|
83
|
+
klass = assoc.class_name.constantize
|
|
84
|
+
|
|
85
|
+
# List of ids that are related to the previous objects (the IN clause in SQL preload statement)
|
|
86
|
+
if level == 0 # Level 0 is different as data is stored in a different structure
|
|
87
|
+
index = data_headers[assoc.foreign_key]
|
|
88
|
+
collection = data
|
|
89
|
+
else
|
|
90
|
+
prev_path = group.first.path_upto(level - 1)
|
|
91
|
+
index = nested_data[prev_path][:header][assoc.foreign_key]
|
|
92
|
+
collection = nested_data[prev_path][:data].values
|
|
93
|
+
end
|
|
94
|
+
ids = collection.map {|d| d[index]}.uniq
|
|
95
|
+
|
|
96
|
+
# Select id and other fields at the next level
|
|
97
|
+
cols = group.map do |f|
|
|
98
|
+
if f.nested?(level + 1)
|
|
99
|
+
f.assoc(level + 1).foreign_key
|
|
100
|
+
else
|
|
101
|
+
f.path[level + 1]
|
|
102
|
+
end
|
|
103
|
+
end.uniq
|
|
104
|
+
plucked_cols = [klass.primary_key, *cols]
|
|
105
|
+
indexed_data = klass.where(klass.primary_key => ids)
|
|
106
|
+
.pluck(*plucked_cols)
|
|
107
|
+
.index_by {|d| d[0]} # Index to quickly search on id
|
|
108
|
+
nested_data[current_path] = {
|
|
109
|
+
header: __preload_pluck_to_header_lookup(plucked_cols),
|
|
110
|
+
data: indexed_data
|
|
111
|
+
}
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
data.map do |attr|
|
|
116
|
+
fields.map do |field|
|
|
117
|
+
if field.nested?(0)
|
|
118
|
+
assoc = field.assoc(0)
|
|
119
|
+
val = attr[data_headers[assoc.foreign_key]]
|
|
120
|
+
(field.path.length - 1).times do |level|
|
|
121
|
+
current_path = field.path_upto(level)
|
|
122
|
+
if field.nested?(level + 1)
|
|
123
|
+
col = field.assoc(level + 1).foreign_key
|
|
124
|
+
else
|
|
125
|
+
col = field.path.last
|
|
126
|
+
end
|
|
127
|
+
current_data = nested_data[current_path]
|
|
128
|
+
current_row = current_data[:data][val]
|
|
129
|
+
if current_row
|
|
130
|
+
index = current_data[:header][col]
|
|
131
|
+
val = current_row[index]
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
val
|
|
135
|
+
else
|
|
136
|
+
attr[data_headers[field.path.last]]
|
|
137
|
+
end
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
def __preload_pluck_to_header_lookup(array)
|
|
143
|
+
Hash[array.map.with_index {|x, i| [x, i]}]
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
ActiveRecord::Base.extend(PreloadPluck)
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: preload_pluck
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Assetricity
|
|
@@ -38,6 +38,20 @@ dependencies:
|
|
|
38
38
|
- - ~>
|
|
39
39
|
- !ruby/object:Gem::Version
|
|
40
40
|
version: 0.7.0
|
|
41
|
+
- !ruby/object:Gem::Dependency
|
|
42
|
+
name: database_cleaner
|
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
|
44
|
+
requirements:
|
|
45
|
+
- - ~>
|
|
46
|
+
- !ruby/object:Gem::Version
|
|
47
|
+
version: 1.4.0
|
|
48
|
+
type: :development
|
|
49
|
+
prerelease: false
|
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
51
|
+
requirements:
|
|
52
|
+
- - ~>
|
|
53
|
+
- !ruby/object:Gem::Version
|
|
54
|
+
version: 1.4.0
|
|
41
55
|
- !ruby/object:Gem::Dependency
|
|
42
56
|
name: factory_girl
|
|
43
57
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -94,20 +108,6 @@ dependencies:
|
|
|
94
108
|
- - ~>
|
|
95
109
|
- !ruby/object:Gem::Version
|
|
96
110
|
version: 1.3.10
|
|
97
|
-
- !ruby/object:Gem::Dependency
|
|
98
|
-
name: database_cleaner
|
|
99
|
-
requirement: !ruby/object:Gem::Requirement
|
|
100
|
-
requirements:
|
|
101
|
-
- - ~>
|
|
102
|
-
- !ruby/object:Gem::Version
|
|
103
|
-
version: 1.4.0
|
|
104
|
-
type: :development
|
|
105
|
-
prerelease: false
|
|
106
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
107
|
-
requirements:
|
|
108
|
-
- - ~>
|
|
109
|
-
- !ruby/object:Gem::Version
|
|
110
|
-
version: 1.4.0
|
|
111
111
|
description: Adds a preload_pluck method to ActiveRecord that allows querying using
|
|
112
112
|
Rails 4 preload-style eager loading, and return a 2-dimensional array without ActiveRecord
|
|
113
113
|
model creation overhead.
|