n1_loader 1.4.1 → 1.4.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +2 -1
- data/.rubocop.yml +2 -0
- data/CHANGELOG.md +13 -0
- data/README.md +90 -238
- data/examples/active_record_integration.rb +33 -0
- data/examples/ar_lazy_integration.rb +37 -0
- data/examples/arguments_support.rb +67 -0
- data/examples/context/service.rb +20 -0
- data/examples/context/setup_ar_lazy.rb +15 -0
- data/examples/context/setup_database.rb +26 -0
- data/examples/core.rb +39 -0
- data/examples/graphql.rb +63 -0
- data/examples/isolated_loader.rb +13 -0
- data/examples/lazy_loading.rb +26 -0
- data/examples/reloading.rb +32 -0
- data/examples/shared_loader.rb +34 -0
- data/examples/single_case.rb +34 -0
- data/lib/n1_loader/active_record/base.rb +18 -0
- data/lib/n1_loader/active_record.rb +2 -0
- data/lib/n1_loader/core/loadable.rb +12 -0
- data/lib/n1_loader/core/loader.rb +4 -1
- data/lib/n1_loader/version.rb +1 -1
- data/n1_loader.gemspec +2 -1
- metadata +37 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3e0025899b1fbd0329159dcd2e1ddf1d28318fe76354fd2e1378b803e49a9d96
|
4
|
+
data.tar.gz: 2f0f222cd9298bb56e2c3a9ae8c1f6559c7ec2bec3653c5a9a2c7c172b49b7fd
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 48a55dd01386410a58a5fbf8ea9d556ac6fa2db33411a469b2c9f3fc99b9ccec2ad36eb482e0adba39a84af4ffd4adba6a51a8db23eba7869c03aca51add1d39
|
7
|
+
data.tar.gz: b4c275204251a8e6908e34af67beacd6bbc05a1fcd1168350ab538ecf48362c840c671d1223dfd3659b2fe87f1428ee0e012144c3c52fa11451a595a0b5a11d9
|
data/.gitignore
CHANGED
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,16 @@
|
|
1
|
+
## [1.4.4] - 2022/04/29
|
2
|
+
|
3
|
+
- Inject `N1Loader::Loadable` to `ActiveRecord::Base` automatically
|
4
|
+
- Make `reload` to call `n1_clear_cache`
|
5
|
+
|
6
|
+
## [1.4.3] - 2022-04-13
|
7
|
+
|
8
|
+
- Add `default` support to arguments
|
9
|
+
|
10
|
+
## [1.4.2] - 2022-03-01
|
11
|
+
|
12
|
+
- Add n1_clear_cache method which is useful for cases like reload in ActiveRecord
|
13
|
+
|
1
14
|
## [1.4.1] - 2022-02-24
|
2
15
|
|
3
16
|
- Fix preloading of invalid objects
|
data/README.md
CHANGED
@@ -3,295 +3,147 @@
|
|
3
3
|
[![CircleCI][1]][2]
|
4
4
|
[![Gem Version][3]][4]
|
5
5
|
|
6
|
-
|
7
|
-
|
6
|
+
N1Loader is designed to provide a way for avoiding [N+1 issues][7] of any kind.
|
7
|
+
For example, it can help with resolving N+1 for:
|
8
|
+
- database querying (most common case)
|
9
|
+
- 3rd party service calls
|
10
|
+
- complex calculations
|
11
|
+
- and many more
|
8
12
|
|
9
|
-
[
|
13
|
+
> [Toptal](https://www.toptal.com#snag-only-shrewd-web-development-experts) is hiring! [Join](https://www.toptal.com#snag-only-shrewd-web-development-experts) as Freelancer or [write me](mailto:lawliet.djez@gmail.com) if you want to join Core team.
|
10
14
|
|
11
|
-
|
12
|
-
- it can be [isolated](#isolated-loaders)
|
13
|
-
- it loads data [lazily](#lazy-loading)
|
14
|
-
- it supports [shareable loaders](#shareable-loaders) between multiple classes
|
15
|
-
- it supports [reloading](#reloading)
|
16
|
-
- it supports optimized [single object loading](#optimized-single-case)
|
17
|
-
- it supports [arguments](#arguments)
|
18
|
-
- it has an integration with [ActiveRecord][5] which makes it brilliant ([example](#activerecord))
|
19
|
-
- it has an integration with [ArLazyPreload][6] which makes it excellent ([example](#arlazypreload))
|
15
|
+
## Killer feature for GraphQL API
|
20
16
|
|
21
|
-
|
17
|
+
N1Loader in combination with [ArLazyPreload][6] is a killer feature for your GraphQL API.
|
18
|
+
Give it a try now and see incredible results instantly! Check out the [example](examples/graphql.rb) and start benefiting from it in your projects!
|
22
19
|
|
23
|
-
## Installation
|
24
|
-
|
25
|
-
Add this line to your application's Gemfile:
|
26
|
-
|
27
|
-
```ruby
|
28
|
-
gem 'n1_loader'
|
29
|
-
```
|
30
|
-
|
31
|
-
You can add integration with [ActiveRecord][5] by:
|
32
|
-
```ruby
|
33
|
-
gem 'n1_loader', require: 'n1_loader/active_record'
|
34
|
-
```
|
35
|
-
|
36
|
-
You can add the integration with [ActiveRecord][5] and [ArLazyPreload][6] by:
|
37
20
|
```ruby
|
38
21
|
gem 'n1_loader', require: 'n1_loader/ar_lazy_preload'
|
39
22
|
```
|
40
23
|
|
41
|
-
##
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
class User
|
45
|
-
include N1Loader::Loadable
|
46
|
-
|
47
|
-
# with inline loader
|
48
|
-
n1_optimized :orders_count do |users|
|
49
|
-
orders_per_user = Order.where(user: users).group(:user_id).count
|
50
|
-
|
51
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
52
|
-
end
|
53
|
-
end
|
24
|
+
## Enhance [ActiveRecord][5]
|
54
25
|
|
55
|
-
|
56
|
-
user = User.new
|
57
|
-
user.orders_count
|
26
|
+
Are you working with well-known Rails application? Try it out how well N1Loader fulfills missing gaps!
|
58
27
|
|
59
|
-
|
60
|
-
|
61
|
-
N1Loader::Preloader.new(users).preload(:orders_count)
|
62
|
-
users.map(&:orders_count)
|
28
|
+
```ruby
|
29
|
+
gem 'n1_loader', require: 'n1_loader/active_record'
|
63
30
|
```
|
64
31
|
|
65
|
-
|
32
|
+
Are you ready to forget about N+1 once and for all? Install [ArLazyPreload][6] and see dreams come true!
|
66
33
|
|
67
34
|
```ruby
|
68
|
-
|
69
|
-
include N1Loader::Loadable
|
70
|
-
|
71
|
-
# with inline loader
|
72
|
-
n1_optimized :orders_count do |users|
|
73
|
-
orders_per_user = Order.where(user: users).group(:user_id).count
|
74
|
-
|
75
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
76
|
-
end
|
77
|
-
end
|
78
|
-
|
79
|
-
user = User.new # => nothing was done for loading
|
80
|
-
user.orders_count # => first time loading
|
81
|
-
|
82
|
-
users = [User.new, User.new] # => nothing was done for loading
|
83
|
-
N1Loader::Preloader.new([users]).preload(:orders_count) # => we only initialized loader but didn't perform it yet
|
84
|
-
users.map(&:orders_count) # => loading has happen for the first time (without N+1)
|
35
|
+
gem 'n1_loader', require: 'n1_loader/ar_lazy_preload'
|
85
36
|
```
|
86
37
|
|
38
|
+
## Standalone mode
|
87
39
|
|
88
|
-
|
40
|
+
Are you not working with [ActiveRecord][5]? N1Loader is ready to be used as standalone solution! ([full snippet](examples/core.rb))
|
89
41
|
|
90
42
|
```ruby
|
91
|
-
|
92
|
-
def perform(users)
|
93
|
-
orders_per_user = Order.where(user: users).group(:user_id).count
|
94
|
-
|
95
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
96
|
-
end
|
97
|
-
end
|
98
|
-
|
99
|
-
class User
|
100
|
-
include N1Loader::Loadable
|
101
|
-
|
102
|
-
n1_optimized :orders_count, OrdersCountLoader
|
103
|
-
end
|
104
|
-
|
105
|
-
class Customer
|
106
|
-
include N1Loader::Loadable
|
107
|
-
|
108
|
-
n1_optimized :orders_count, OrdersCountLoader
|
109
|
-
end
|
110
|
-
|
111
|
-
User.new.orders_count # => works
|
112
|
-
Customer.new.orders_count # => works
|
43
|
+
gem 'n1_loader'
|
113
44
|
```
|
114
45
|
|
115
|
-
|
116
|
-
|
117
|
-
```ruby
|
118
|
-
class User
|
119
|
-
include N1Loader::Loadable
|
120
|
-
|
121
|
-
# with inline loader
|
122
|
-
n1_optimized :orders_count do |users|
|
123
|
-
orders_per_user = Order.where(user: users).group(:user_id).count
|
124
|
-
|
125
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
126
|
-
end
|
127
|
-
end
|
128
|
-
|
129
|
-
user = User.new
|
130
|
-
user.orders_count # => loader is executed first time and value was cached
|
131
|
-
user.orders_count(reload: true) # => loader is executed again and a new value was cached
|
46
|
+
## How to use it?
|
132
47
|
|
133
|
-
|
134
|
-
|
135
|
-
users.map(&:orders_count) # => loader was executed first time without N+1 issue and values were cached
|
136
|
-
|
137
|
-
N1Loader::Preloader.new(users).preload(:orders_count) # => loader was initialized again but not yet executed
|
138
|
-
users.map(&:orders_count) # => new loader was executed first time without N+1 issue and new values were cached
|
139
|
-
```
|
48
|
+
N1Loader provides DSL that allows you to define N+1 ready loaders that can
|
49
|
+
be injected into your objects in a way that you can avoid N+1 issues.
|
140
50
|
|
141
|
-
|
51
|
+
> _Disclaimer_: examples below are working but designed to show N1Loader potentials only.
|
52
|
+
In real live applications, N1Loader can be applied anywhere and in more [elegant way](examples/isolated_loader.rb).
|
142
53
|
|
54
|
+
Let's look at simple example below ([full snippet](examples/active_record_integration.rb)):
|
143
55
|
```ruby
|
144
|
-
class
|
145
|
-
|
146
|
-
|
56
|
+
class User < ActiveRecord::Base
|
57
|
+
has_many :payments
|
58
|
+
|
59
|
+
n1_optimized :payments_total do |users|
|
60
|
+
total_per_user =
|
61
|
+
Payment.group(:user_id)
|
62
|
+
.where(user: users)
|
63
|
+
.sum(:amount)
|
64
|
+
.tap { |h| h.default = 0 }
|
65
|
+
|
66
|
+
users.each do |user|
|
67
|
+
total = total_per_user[user.id]
|
68
|
+
fulfill(user, total)
|
69
|
+
end
|
147
70
|
end
|
148
71
|
end
|
149
72
|
|
150
|
-
|
151
|
-
|
152
|
-
|
153
|
-
|
73
|
+
class Payment < ActiveRecord::Base
|
74
|
+
belongs_to :user
|
75
|
+
|
76
|
+
validates :amount, presence: true
|
154
77
|
end
|
155
|
-
```
|
156
78
|
|
157
|
-
|
79
|
+
# A user has many payments.
|
80
|
+
# Assuming, we want to know for group of users, what is a total of their payments, we can do the following:
|
158
81
|
|
159
|
-
|
160
|
-
|
161
|
-
include N1Loader::Loadable
|
82
|
+
# Has N+1 issue
|
83
|
+
p User.all.map { |user| user.payments.sum(&:amount) }
|
162
84
|
|
163
|
-
|
164
|
-
|
165
|
-
orders_per_user = Order.where(user: users).group(:user_id).count
|
166
|
-
|
167
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
168
|
-
end
|
169
|
-
|
170
|
-
# Optimized for single object loading
|
171
|
-
def single(user)
|
172
|
-
user.orders.count
|
173
|
-
end
|
174
|
-
end
|
175
|
-
end
|
85
|
+
# Has no N+1 but we load too many data that we don't actually need
|
86
|
+
p User.all.includes(:payments).map { |user| user.payments.sum(&:amount) }
|
176
87
|
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
users = [User.new, User.new]
|
181
|
-
N1Loader::Preloader.new(users).preload(:orders_count)
|
182
|
-
users.map(&:orders_count) # perform will be used once without N+1
|
88
|
+
# Has no N+1 and we load only what we need
|
89
|
+
p User.all.includes(:payments_total).map { |user| user.payments_total }
|
183
90
|
```
|
184
91
|
|
185
|
-
|
92
|
+
Let's assume now, that we want to calculate the total of payments for the given period for a group of users.
|
93
|
+
N1Loader can do that as well! ([full snippet](examples/arguments_support.rb))
|
186
94
|
|
187
95
|
```ruby
|
188
|
-
class User
|
189
|
-
|
190
|
-
|
191
|
-
n1_optimized :orders_count do
|
192
|
-
argument :type
|
193
|
-
|
194
|
-
def perform(users)
|
195
|
-
orders_per_user = Order.where(type: type, user: users).group(:user_id).count
|
196
|
-
|
197
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
198
|
-
end
|
199
|
-
end
|
200
|
-
end
|
201
|
-
|
202
|
-
user = User.new
|
203
|
-
user.orders_count(type: :gifts) # The loader will be performed first time for this argument
|
204
|
-
user.orders_count(type: :sales) # The loader will be performed first time for this argument
|
205
|
-
user.orders_count(type: :gifts) # The cached value will be used
|
206
|
-
|
207
|
-
users = [User.new, User.new]
|
208
|
-
N1Loader::Preloader.new(users).preload(:orders_count)
|
209
|
-
users.map { |user| user.orders_count(type: :gifts) } # No N+1 here
|
210
|
-
```
|
96
|
+
class User < ActiveRecord::Base
|
97
|
+
has_many :payments
|
211
98
|
|
212
|
-
|
213
|
-
|
99
|
+
n1_optimized :payments_total do
|
100
|
+
argument :from
|
101
|
+
argument :to
|
214
102
|
|
215
|
-
```ruby
|
216
|
-
class User
|
217
|
-
include N1Loader::Loadable
|
218
|
-
|
219
|
-
n1_optimized :orders_count do
|
220
|
-
argument :sale
|
221
|
-
|
222
|
-
cache_key { sale.id }
|
223
|
-
|
224
103
|
def perform(users)
|
225
|
-
|
226
|
-
|
227
|
-
|
104
|
+
total_per_user =
|
105
|
+
Payment
|
106
|
+
.group(:user_id)
|
107
|
+
.where(created_at: from..to)
|
108
|
+
.where(user: users)
|
109
|
+
.sum(:amount)
|
110
|
+
.tap { |h| h.default = 0 }
|
111
|
+
|
112
|
+
users.each do |user|
|
113
|
+
total = total_per_user[user.id]
|
114
|
+
fulfill(user, total)
|
115
|
+
end
|
228
116
|
end
|
229
117
|
end
|
230
118
|
end
|
231
119
|
|
232
|
-
|
233
|
-
user
|
234
|
-
user.orders_count(sale: Sale.first) # the cached value will be returned
|
235
|
-
```
|
236
|
-
|
237
|
-
|
238
|
-
## Integrations
|
239
|
-
|
240
|
-
### [ActiveRecord][5]
|
241
|
-
|
242
|
-
_Note_: Rails 7 support is coming soon! Stay tuned!
|
120
|
+
class Payment < ActiveRecord::Base
|
121
|
+
belongs_to :user
|
243
122
|
|
244
|
-
|
245
|
-
class User < ActiveRecord::Base
|
246
|
-
include N1Loader::Loadable
|
247
|
-
|
248
|
-
n1_optimized :orders_count do |users|
|
249
|
-
orders_per_user = Order.where(user: users).group(:user_id).count
|
250
|
-
|
251
|
-
users.each { |user| fulfill(user, orders_per_user[user.id]) }
|
252
|
-
end
|
123
|
+
validates :amount, presence: true
|
253
124
|
end
|
254
125
|
|
255
|
-
#
|
256
|
-
user
|
257
|
-
user.orders_count
|
126
|
+
# Has N+1
|
127
|
+
p User.all.map { |user| user.payments.select { |payment| payment.created_at >= from && payment.created_at <= to }.sum(&:amount) }
|
258
128
|
|
259
|
-
#
|
260
|
-
User.
|
129
|
+
# Has no N+1 but we load too many data that we don't need
|
130
|
+
p User.all.includes(:payments).map { |user| user.payments.select { |payment| payment.created_at >= from && payment.created_at <= to }.sum(&:amount) }
|
261
131
|
|
262
|
-
#
|
263
|
-
|
264
|
-
N1Loader::Preloader.new(users).preload(:orders_count)
|
265
|
-
|
266
|
-
# No N+1 here
|
267
|
-
users.map(&:orders_count)
|
132
|
+
# Has no N+1 and calculation is the most efficient
|
133
|
+
p User.all.includes(:payments_total).map { |user| user.payments_total(from: from, to: to) }
|
268
134
|
```
|
269
135
|
|
270
|
-
|
271
|
-
|
272
|
-
|
273
|
-
|
274
|
-
|
275
|
-
|
276
|
-
|
277
|
-
|
278
|
-
|
279
|
-
|
280
|
-
|
281
|
-
end
|
282
|
-
|
283
|
-
# For single user
|
284
|
-
user = User.first
|
285
|
-
user.orders_count
|
286
|
-
|
287
|
-
# For many users without N+1
|
288
|
-
User.lazy_preload(:orders_count).all.map(&:orders_count)
|
289
|
-
# or
|
290
|
-
User.preload_associations_lazily.all.map(&:orders_count)
|
291
|
-
# or
|
292
|
-
ArLazyPreload.config.auto_preload = true
|
293
|
-
User.all.map(:orders_count)
|
294
|
-
```
|
136
|
+
## Features and benefits
|
137
|
+
|
138
|
+
- N1Loader doesn't use Promises which means it's easy to debug
|
139
|
+
- Doesn't require injection to objects, can be used in [isolation](examples/isolated_loader.rb)
|
140
|
+
- Loads data [lazily](examples/lazy_loading.rb)
|
141
|
+
- Loaders can be [shared](examples/shared_loader.rb) between multiple classes
|
142
|
+
- Loaded data can be [re-fetched](examples/reloading.rb)
|
143
|
+
- Loader can be optimized for [single cases](examples/single_case.rb)
|
144
|
+
- Loader support [arguments](examples/arguments_support.rb)
|
145
|
+
- Has [integration](examples/active_record_integration.rb) with [ActiveRecord][5] which makes it brilliant
|
146
|
+
- Has [integration](examples/ar_lazy_integration.rb) with [ArLazyPreload][6] which makes it excellent
|
295
147
|
|
296
148
|
## Contributing
|
297
149
|
|
@@ -0,0 +1,33 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "n1_loader/active_record"
|
4
|
+
|
5
|
+
require_relative 'context/setup_database'
|
6
|
+
|
7
|
+
class User < ActiveRecord::Base
|
8
|
+
has_many :payments
|
9
|
+
|
10
|
+
n1_optimized :payments_total do |users|
|
11
|
+
total_per_user = Payment.group(:user_id).where(user: users).sum(:amount).tap { |h| h.default = 0 }
|
12
|
+
|
13
|
+
users.each do |user|
|
14
|
+
total = total_per_user[user.id]
|
15
|
+
fulfill(user, total)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
class Payment < ActiveRecord::Base
|
21
|
+
belongs_to :user
|
22
|
+
|
23
|
+
validates :amount, presence: true
|
24
|
+
end
|
25
|
+
|
26
|
+
fill_database
|
27
|
+
|
28
|
+
# Has N+1
|
29
|
+
p User.all.map { |user| user.payments.sum(&:amount) }
|
30
|
+
# Has no N+1 but we load too many data that we don't need
|
31
|
+
p User.all.includes(:payments).map { |user| user.payments.sum(&:amount) }
|
32
|
+
# Has no N+1 and calculation is the most efficient
|
33
|
+
p User.all.includes(:payments_total).map(&:payments_total)
|
@@ -0,0 +1,37 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "n1_loader/ar_lazy_preload"
|
4
|
+
|
5
|
+
require_relative 'context/setup_ar_lazy'
|
6
|
+
require_relative 'context/setup_database'
|
7
|
+
|
8
|
+
class User < ActiveRecord::Base
|
9
|
+
has_many :payments
|
10
|
+
|
11
|
+
n1_optimized :payments_total do |users|
|
12
|
+
total_per_user = Payment.group(:user_id).where(user: users).sum(:amount).tap { |h| h.default = 0 }
|
13
|
+
|
14
|
+
users.each do |user|
|
15
|
+
total = total_per_user[user.id]
|
16
|
+
fulfill(user, total)
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
class Payment < ActiveRecord::Base
|
22
|
+
belongs_to :user
|
23
|
+
|
24
|
+
validates :amount, presence: true
|
25
|
+
end
|
26
|
+
|
27
|
+
fill_database
|
28
|
+
|
29
|
+
# Has N+1
|
30
|
+
p User.all.map { |user| user.payments.sum(&:amount) }
|
31
|
+
|
32
|
+
# Has no N+1 but we load too many data that we don't need
|
33
|
+
p User.preload_associations_lazily.map(&:payments_total)
|
34
|
+
|
35
|
+
# Has no N+1 and calculation is the most efficient
|
36
|
+
ArLazyPreload.config.auto_preload = true
|
37
|
+
User.all.map(&:payments_total)
|
@@ -0,0 +1,67 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "n1_loader/active_record"
|
4
|
+
|
5
|
+
require_relative 'context/setup_database'
|
6
|
+
|
7
|
+
class User < ActiveRecord::Base
|
8
|
+
has_many :payments
|
9
|
+
|
10
|
+
n1_optimized :payments_total do
|
11
|
+
# Arguments can be:
|
12
|
+
# argument :something, optional: true
|
13
|
+
# argument :something, default: -> { 100 }
|
14
|
+
#
|
15
|
+
# Note: do not use mutable (mostly timing related) defaults like:
|
16
|
+
# argument :from, default -> { 2.minutes.from_now }
|
17
|
+
# because such values will be unique for every loader call which will make N+1 issue stay
|
18
|
+
argument :from
|
19
|
+
argument :to
|
20
|
+
|
21
|
+
# This is used to define logic how loaders are compared to each other
|
22
|
+
# default is:
|
23
|
+
# cache_key { *arguments.map(&:object_id) }
|
24
|
+
cache_key { [from, to] }
|
25
|
+
|
26
|
+
def perform(users)
|
27
|
+
total_per_user =
|
28
|
+
Payment
|
29
|
+
.group(:user_id)
|
30
|
+
.where(created_at: from..to)
|
31
|
+
.where(user: users)
|
32
|
+
.sum(:amount)
|
33
|
+
.tap { |h| h.default = 0 }
|
34
|
+
|
35
|
+
users.each do |user|
|
36
|
+
total = total_per_user[user.id]
|
37
|
+
fulfill(user, total)
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
class Payment < ActiveRecord::Base
|
44
|
+
belongs_to :user
|
45
|
+
|
46
|
+
validates :amount, presence: true
|
47
|
+
end
|
48
|
+
|
49
|
+
fill_database
|
50
|
+
|
51
|
+
from = 2.days.ago
|
52
|
+
to = 1.day.ago
|
53
|
+
|
54
|
+
# Has N+1
|
55
|
+
p User.all.map { |user|
|
56
|
+
user.payments.select do |payment|
|
57
|
+
payment.created_at >= from && payment.created_at <= to
|
58
|
+
end.sum(&:amount)
|
59
|
+
}
|
60
|
+
# Has no N+1 but we load too many data that we don't need
|
61
|
+
p User.all.includes(:payments).map { |user|
|
62
|
+
user.payments.select do |payment|
|
63
|
+
payment.created_at >= from && payment.created_at <= to
|
64
|
+
end.sum(&:amount)
|
65
|
+
}
|
66
|
+
# Has no N+1 and calculation is the most efficient
|
67
|
+
p User.all.includes(:payments_total).map { |user| user.payments_total(from: from, to: to) }
|
@@ -0,0 +1,20 @@
|
|
1
|
+
# 3rd party service, or database, or anything else that can perform in batches
|
2
|
+
class Service
|
3
|
+
def self.count
|
4
|
+
@count ||= 0
|
5
|
+
end
|
6
|
+
|
7
|
+
def self.increase!
|
8
|
+
@count = (@count || 0) + 1
|
9
|
+
end
|
10
|
+
|
11
|
+
def self.receive(*users)
|
12
|
+
increase!
|
13
|
+
|
14
|
+
users.flatten.map(&:object_id)
|
15
|
+
end
|
16
|
+
|
17
|
+
def self.single(user)
|
18
|
+
user.object_id
|
19
|
+
end
|
20
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
ActiveSupport.on_load(:active_record) do
|
2
|
+
ActiveRecord::Base.include(ArLazyPreload::Base)
|
3
|
+
|
4
|
+
ActiveRecord::Relation.prepend(ArLazyPreload::Relation)
|
5
|
+
ActiveRecord::AssociationRelation.prepend(ArLazyPreload::AssociationRelation)
|
6
|
+
ActiveRecord::Relation::Merger.prepend(ArLazyPreload::Merger)
|
7
|
+
|
8
|
+
[
|
9
|
+
ActiveRecord::Associations::CollectionAssociation,
|
10
|
+
ActiveRecord::Associations::Association
|
11
|
+
].each { |klass| klass.prepend(ArLazyPreload::Association) }
|
12
|
+
|
13
|
+
ActiveRecord::Associations::CollectionAssociation.prepend(ArLazyPreload::CollectionAssociation)
|
14
|
+
ActiveRecord::Associations::CollectionProxy.prepend(ArLazyPreload::CollectionProxy)
|
15
|
+
end
|
@@ -0,0 +1,26 @@
|
|
1
|
+
require "sqlite3"
|
2
|
+
|
3
|
+
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
|
4
|
+
ActiveRecord::Base.connection.tables.each do |table|
|
5
|
+
ActiveRecord::Base.connection.drop_table(table, force: :cascade)
|
6
|
+
end
|
7
|
+
ActiveRecord::Schema.verbose = false
|
8
|
+
ActiveRecord::Base.logger = Logger.new($stdout)
|
9
|
+
|
10
|
+
ActiveRecord::Schema.define(version: 1) do
|
11
|
+
create_table(:payments) do |t|
|
12
|
+
t.belongs_to :user
|
13
|
+
t.integer :amount
|
14
|
+
t.timestamps
|
15
|
+
end
|
16
|
+
create_table(:users)
|
17
|
+
end
|
18
|
+
|
19
|
+
def fill_database
|
20
|
+
10.times do
|
21
|
+
user = User.create!
|
22
|
+
10.times do
|
23
|
+
Payment.create!(user: user, amount: rand(1000))
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
data/examples/core.rb
ADDED
@@ -0,0 +1,39 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "n1_loader"
|
4
|
+
|
5
|
+
require_relative 'context/service'
|
6
|
+
|
7
|
+
# Class that wants to request 3rd party service without N+1
|
8
|
+
class User
|
9
|
+
include N1Loader::Loadable
|
10
|
+
|
11
|
+
def unoptimized_call
|
12
|
+
Service.receive(self)[0]
|
13
|
+
end
|
14
|
+
|
15
|
+
n1_optimized :optimized_call do |users|
|
16
|
+
data = Service.receive(users)
|
17
|
+
|
18
|
+
users.each_with_index do |user, index|
|
19
|
+
fulfill(user, data[index])
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
# works fine for single case
|
25
|
+
user = User.new
|
26
|
+
p "Works correctly: #{user.unoptimized_call == user.optimized_call}"
|
27
|
+
|
28
|
+
users = [User.new, User.new]
|
29
|
+
|
30
|
+
# Has N+1
|
31
|
+
count_before = Service.count
|
32
|
+
p users.map(&:unoptimized_call)
|
33
|
+
p "Has N+1 #{Service.count == count_before + users.count}"
|
34
|
+
|
35
|
+
# Has no N+1
|
36
|
+
count_before = Service.count
|
37
|
+
N1Loader::Preloader.new(users).preload(:optimized_call)
|
38
|
+
p users.map(&:optimized_call)
|
39
|
+
p "Has no N+1: #{Service.count == count_before + 1}"
|
data/examples/graphql.rb
ADDED
@@ -0,0 +1,63 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "n1_loader/ar_lazy_preload"
|
4
|
+
require 'graphql'
|
5
|
+
|
6
|
+
require_relative 'context/setup_database'
|
7
|
+
require_relative 'context/setup_ar_lazy'
|
8
|
+
|
9
|
+
class User < ActiveRecord::Base
|
10
|
+
has_many :payments
|
11
|
+
|
12
|
+
n1_optimized :payments_total do |users|
|
13
|
+
total_per_user = Payment.group(:user_id).where(user: users).sum(:amount).tap { |h| h.default = 0 }
|
14
|
+
|
15
|
+
users.each do |user|
|
16
|
+
total = total_per_user[user.id]
|
17
|
+
fulfill(user, total)
|
18
|
+
end
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
class Payment < ActiveRecord::Base
|
23
|
+
belongs_to :user
|
24
|
+
|
25
|
+
validates :amount, presence: true
|
26
|
+
end
|
27
|
+
|
28
|
+
10.times do
|
29
|
+
user = User.create!
|
30
|
+
10.times do
|
31
|
+
Payment.create!(user: user, amount: rand(1000))
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
ArLazyPreload.config.auto_preload = true
|
36
|
+
# Or use +preload_associations_lazily+ when loading objects from database
|
37
|
+
|
38
|
+
class UserType < GraphQL::Schema::Object
|
39
|
+
field :payments_total, Integer
|
40
|
+
end
|
41
|
+
|
42
|
+
class QueryType < GraphQL::Schema::Object
|
43
|
+
field :users, [UserType]
|
44
|
+
|
45
|
+
def users
|
46
|
+
User.all
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
class Schema < GraphQL::Schema
|
51
|
+
query QueryType
|
52
|
+
end
|
53
|
+
|
54
|
+
query_string = <<~GQL
|
55
|
+
{
|
56
|
+
users {
|
57
|
+
paymentsTotal
|
58
|
+
}
|
59
|
+
}
|
60
|
+
GQL
|
61
|
+
|
62
|
+
# No N+1. And never will be!
|
63
|
+
p Schema.execute(query_string)['data']
|
@@ -0,0 +1,13 @@
|
|
1
|
+
require 'n1_loader'
|
2
|
+
|
3
|
+
class IsolatedLoader < N1Loader::Loader
|
4
|
+
def perform(elements)
|
5
|
+
elements.each { |element| fulfill(element, [element]) }
|
6
|
+
end
|
7
|
+
end
|
8
|
+
|
9
|
+
objects = [1, 2, 3, 4]
|
10
|
+
loader = IsolatedLoader.new(objects)
|
11
|
+
objects.each do |object|
|
12
|
+
loader.for(object) # => it has no N+1 and it doesn't require to be injected in the class
|
13
|
+
end
|
@@ -0,0 +1,26 @@
|
|
1
|
+
require 'n1_loader'
|
2
|
+
|
3
|
+
require_relative 'context/service'
|
4
|
+
|
5
|
+
# Class that wants to request 3rd party service without N+1
|
6
|
+
class User
|
7
|
+
include N1Loader::Loadable
|
8
|
+
|
9
|
+
n1_optimized :optimized_call do |users|
|
10
|
+
data = Service.receive(users)
|
11
|
+
|
12
|
+
users.each_with_index do |user, index|
|
13
|
+
fulfill(user, data[index])
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
users = [User.new, User.new, User.new]
|
19
|
+
|
20
|
+
# Initialized loader but didn't perform it yet
|
21
|
+
N1Loader::Preloader.new(users).preload(:optimized_call)
|
22
|
+
p "No calls yet: #{Service.count == 0}"
|
23
|
+
|
24
|
+
# First time loading
|
25
|
+
users.map(&:optimized_call)
|
26
|
+
p "First time loaded: #{Service.count == 1}"
|
@@ -0,0 +1,32 @@
|
|
1
|
+
require 'n1_loader'
|
2
|
+
|
3
|
+
require_relative 'context/service'
|
4
|
+
|
5
|
+
class User
|
6
|
+
include N1Loader::Loadable
|
7
|
+
|
8
|
+
n1_optimized :optimized_call do |users|
|
9
|
+
data = Service.receive(users)
|
10
|
+
|
11
|
+
users.each_with_index do |user, index|
|
12
|
+
fulfill(user, data[index])
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
users = [User.new, User.new, User.new]
|
18
|
+
|
19
|
+
# Initialized loader but didn't perform it yet
|
20
|
+
N1Loader::Preloader.new(users).preload(:optimized_call)
|
21
|
+
p "No calls yet: #{Service.count == 0}"
|
22
|
+
|
23
|
+
# First time loading
|
24
|
+
users.map(&:optimized_call)
|
25
|
+
p "First time loaded: #{Service.count == 1}"
|
26
|
+
|
27
|
+
users.first.optimized_call(reload: true)
|
28
|
+
p "Reloaded for this object only: #{Service.count == 2}"
|
29
|
+
|
30
|
+
users.first.n1_clear_cache
|
31
|
+
users.first.optimized_call
|
32
|
+
p "Reloaded for this object only: #{Service.count == 3}"
|
@@ -0,0 +1,34 @@
|
|
1
|
+
require 'n1_loader'
|
2
|
+
|
3
|
+
require_relative 'context/service'
|
4
|
+
|
5
|
+
# Loader that will be shared between multiple classes
|
6
|
+
class SharedLoader < N1Loader::Loader
|
7
|
+
def perform(objects)
|
8
|
+
data = Service.receive(objects)
|
9
|
+
|
10
|
+
objects.each_with_index do |user, index|
|
11
|
+
fulfill(user, data[index])
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
class User
|
17
|
+
include N1Loader::Loadable
|
18
|
+
|
19
|
+
n1_optimized :optimized_call, SharedLoader
|
20
|
+
end
|
21
|
+
|
22
|
+
class Payment
|
23
|
+
include N1Loader::Loadable
|
24
|
+
|
25
|
+
n1_optimized :optimized_call, SharedLoader
|
26
|
+
end
|
27
|
+
|
28
|
+
objects = [User.new, Payment.new, User.new, Payment.new]
|
29
|
+
|
30
|
+
N1Loader::Preloader.new(objects).preload(:optimized_call)
|
31
|
+
|
32
|
+
# First time loading for all objects
|
33
|
+
objects.map(&:optimized_call)
|
34
|
+
p "Loaded for all once: #{Service.count == 1}"
|
@@ -0,0 +1,34 @@
|
|
1
|
+
require 'n1_loader'
|
2
|
+
|
3
|
+
require_relative 'context/service'
|
4
|
+
|
5
|
+
# Loader that will be shared between multiple classes
|
6
|
+
class OptimizedLoader < N1Loader::Loader
|
7
|
+
def perform(objects)
|
8
|
+
data = Service.receive(objects)
|
9
|
+
|
10
|
+
objects.each_with_index do |user, index|
|
11
|
+
fulfill(user, data[index])
|
12
|
+
end
|
13
|
+
end
|
14
|
+
|
15
|
+
def single(object)
|
16
|
+
Service.single(object)
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
class User
|
21
|
+
include N1Loader::Loadable
|
22
|
+
|
23
|
+
n1_optimized :optimized_call, OptimizedLoader
|
24
|
+
end
|
25
|
+
|
26
|
+
objects = [User.new, User.new]
|
27
|
+
|
28
|
+
N1Loader::Preloader.new(objects).preload(:optimized_call)
|
29
|
+
|
30
|
+
objects.map(&:optimized_call)
|
31
|
+
p "Used multi-case perform: #{Service.count == 1}"
|
32
|
+
|
33
|
+
User.new.optimized_call
|
34
|
+
p "Used single-case perform: #{Service.count == 1}"
|
@@ -0,0 +1,18 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module N1Loader
|
4
|
+
module ActiveRecord
|
5
|
+
# Extension module for ActiveRecord::Base
|
6
|
+
module Base
|
7
|
+
extend ActiveSupport::Concern
|
8
|
+
|
9
|
+
include N1Loader::Loadable
|
10
|
+
|
11
|
+
# Clear N1Loader cache on reloading the object
|
12
|
+
def reload(*)
|
13
|
+
n1_clear_cache
|
14
|
+
super
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
@@ -16,6 +16,7 @@ end
|
|
16
16
|
ActiveSupport.on_load(:active_record) do
|
17
17
|
require_relative "active_record/loader"
|
18
18
|
require_relative "active_record/loader_collection"
|
19
|
+
require_relative "active_record/base"
|
19
20
|
|
20
21
|
case ActiveRecord::VERSION::MAJOR
|
21
22
|
when 6
|
@@ -25,4 +26,5 @@ ActiveSupport.on_load(:active_record) do
|
|
25
26
|
end
|
26
27
|
|
27
28
|
ActiveRecord::Associations::Preloader.prepend(N1Loader::ActiveRecord::Associations::Preloader)
|
29
|
+
ActiveRecord::Base.include(N1Loader::ActiveRecord::Base)
|
28
30
|
end
|
@@ -32,6 +32,12 @@ module N1Loader
|
|
32
32
|
send("#{name}_loader=", loader_collection)
|
33
33
|
end
|
34
34
|
|
35
|
+
def n1_clear_cache
|
36
|
+
self.class.n1_loaders.each do |name|
|
37
|
+
n1_loader_set(name, nil)
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
35
41
|
def self.included(base)
|
36
42
|
base.extend(ClassMethods)
|
37
43
|
end
|
@@ -45,6 +51,10 @@ module N1Loader
|
|
45
51
|
respond_to?("#{name}_loader")
|
46
52
|
end
|
47
53
|
|
54
|
+
def n1_loaders
|
55
|
+
@n1_loaders ||= superclass.respond_to?(:n1_loaders) ? superclass.n1_loaders.dup : []
|
56
|
+
end
|
57
|
+
|
48
58
|
def n1_optimized(name, loader = nil, &block) # rubocop:disable Metrics/MethodLength, Metrics/AbcSize
|
49
59
|
loader ||= Class.new(N1Loader::Loader) do
|
50
60
|
if block.arity == 1
|
@@ -56,6 +66,8 @@ module N1Loader
|
|
56
66
|
loader_name = "#{name}_loader"
|
57
67
|
loader_variable_name = "@#{loader_name}"
|
58
68
|
|
69
|
+
n1_loaders << name
|
70
|
+
|
59
71
|
define_singleton_method(loader_name) do
|
60
72
|
loader
|
61
73
|
end
|
@@ -17,10 +17,13 @@ module N1Loader
|
|
17
17
|
# @param name [Symbol]
|
18
18
|
# @param opts [Hash]
|
19
19
|
# @option opts [Boolean] optional false by default
|
20
|
+
# @option opts [Proc] default
|
20
21
|
def argument(name, **opts)
|
22
|
+
opts[:optional] = true if opts[:default]
|
23
|
+
|
21
24
|
@arguments ||= []
|
22
25
|
|
23
|
-
define_method(name) { args[name] }
|
26
|
+
define_method(name) { args[name] ||= opts[:default]&.call }
|
24
27
|
|
25
28
|
@arguments << opts.merge(name: name)
|
26
29
|
end
|
data/lib/n1_loader/version.rb
CHANGED
data/n1_loader.gemspec
CHANGED
@@ -22,9 +22,10 @@ Gem::Specification.new do |spec|
|
|
22
22
|
end
|
23
23
|
spec.require_paths = ["lib"]
|
24
24
|
|
25
|
-
spec.add_development_dependency "activerecord", ">= 5"
|
25
|
+
spec.add_development_dependency "activerecord", ">= 5", "< 7"
|
26
26
|
spec.add_development_dependency "ar_lazy_preload", ">= 0.6"
|
27
27
|
spec.add_development_dependency "db-query-matchers", "~> 0.10"
|
28
|
+
spec.add_development_dependency "graphql", "~> 2.0"
|
28
29
|
spec.add_development_dependency "rails", ">= 5"
|
29
30
|
spec.add_development_dependency "rspec", "~> 3.0"
|
30
31
|
spec.add_development_dependency "rspec_junit_formatter", "~> 0.4"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: n1_loader
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.4.
|
4
|
+
version: 1.4.4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Evgeniy Demin
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-
|
11
|
+
date: 2022-04-29 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -17,6 +17,9 @@ dependencies:
|
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '5'
|
20
|
+
- - "<"
|
21
|
+
- !ruby/object:Gem::Version
|
22
|
+
version: '7'
|
20
23
|
type: :development
|
21
24
|
prerelease: false
|
22
25
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -24,6 +27,9 @@ dependencies:
|
|
24
27
|
- - ">="
|
25
28
|
- !ruby/object:Gem::Version
|
26
29
|
version: '5'
|
30
|
+
- - "<"
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '7'
|
27
33
|
- !ruby/object:Gem::Dependency
|
28
34
|
name: ar_lazy_preload
|
29
35
|
requirement: !ruby/object:Gem::Requirement
|
@@ -52,6 +58,20 @@ dependencies:
|
|
52
58
|
- - "~>"
|
53
59
|
- !ruby/object:Gem::Version
|
54
60
|
version: '0.10'
|
61
|
+
- !ruby/object:Gem::Dependency
|
62
|
+
name: graphql
|
63
|
+
requirement: !ruby/object:Gem::Requirement
|
64
|
+
requirements:
|
65
|
+
- - "~>"
|
66
|
+
- !ruby/object:Gem::Version
|
67
|
+
version: '2.0'
|
68
|
+
type: :development
|
69
|
+
prerelease: false
|
70
|
+
version_requirements: !ruby/object:Gem::Requirement
|
71
|
+
requirements:
|
72
|
+
- - "~>"
|
73
|
+
- !ruby/object:Gem::Version
|
74
|
+
version: '2.0'
|
55
75
|
- !ruby/object:Gem::Dependency
|
56
76
|
name: rails
|
57
77
|
requirement: !ruby/object:Gem::Requirement
|
@@ -145,10 +165,24 @@ files:
|
|
145
165
|
- ar_lazy_preload-gemfiles/ar_lazy_preload_master.gemfile
|
146
166
|
- bin/console
|
147
167
|
- bin/setup
|
168
|
+
- examples/active_record_integration.rb
|
169
|
+
- examples/ar_lazy_integration.rb
|
170
|
+
- examples/arguments_support.rb
|
171
|
+
- examples/context/service.rb
|
172
|
+
- examples/context/setup_ar_lazy.rb
|
173
|
+
- examples/context/setup_database.rb
|
174
|
+
- examples/core.rb
|
175
|
+
- examples/graphql.rb
|
176
|
+
- examples/isolated_loader.rb
|
177
|
+
- examples/lazy_loading.rb
|
178
|
+
- examples/reloading.rb
|
179
|
+
- examples/shared_loader.rb
|
180
|
+
- examples/single_case.rb
|
148
181
|
- lib/n1_loader.rb
|
149
182
|
- lib/n1_loader/active_record.rb
|
150
183
|
- lib/n1_loader/active_record/associations_preloader_v5.rb
|
151
184
|
- lib/n1_loader/active_record/associations_preloader_v6.rb
|
185
|
+
- lib/n1_loader/active_record/base.rb
|
152
186
|
- lib/n1_loader/active_record/loader.rb
|
153
187
|
- lib/n1_loader/active_record/loader_collection.rb
|
154
188
|
- lib/n1_loader/ar_lazy_preload.rb
|
@@ -186,7 +220,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
186
220
|
- !ruby/object:Gem::Version
|
187
221
|
version: '0'
|
188
222
|
requirements: []
|
189
|
-
rubygems_version: 3.
|
223
|
+
rubygems_version: 3.1.6
|
190
224
|
signing_key:
|
191
225
|
specification_version: 4
|
192
226
|
summary: Loader to solve N+1 issue for good.
|