simple_query 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +454 -123
- data/lib/simple_query/builder.rb +240 -23
- data/lib/simple_query/clauses/aggregation_clause.rb +158 -0
- data/lib/simple_query/clauses/group_having_clause.rb +6 -6
- data/lib/simple_query/clauses/set_clause.rb +4 -3
- data/lib/simple_query/clauses/where_clause.rb +34 -9
- data/lib/simple_query/stream/mysql_stream.rb +9 -4
- data/lib/simple_query/stream/postgres_stream.rb +45 -11
- data/lib/simple_query/version.rb +1 -1
- data/lib/simple_query.rb +11 -4
- metadata +7 -6
data/README.md
CHANGED
|
@@ -1,39 +1,76 @@
|
|
|
1
1
|
# SimpleQuery
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://badge.fury.io/rb/simple_query)
|
|
4
|
+
[](https://github.com/kholdrex/simple_query/actions/workflows/ci.yml)
|
|
5
|
+
|
|
6
|
+
SimpleQuery is a lightweight query builder for ActiveRecord. It gives Rails applications an explicit, chainable API for read-heavy queries, joins, aggregations, read models, bulk updates, and adapter-aware streaming without replacing ActiveRecord.
|
|
7
|
+
|
|
8
|
+
Use it when you want ActiveRecord-backed SQL with less object-instantiation overhead, predictable result shapes, or a small query/read-model layer around reporting and data-processing code.
|
|
9
|
+
|
|
10
|
+
## Features
|
|
11
|
+
|
|
12
|
+
- Chainable query builder for ActiveRecord models
|
|
13
|
+
- Struct results by default for lightweight reads
|
|
14
|
+
- Optional custom read models with explicit attributes
|
|
15
|
+
- Hash, Arel, raw SQL, and ActiveRecord-style placeholder conditions
|
|
16
|
+
- Inner, left, right, and full joins
|
|
17
|
+
- `distinct`, `group`, `having`, `order`, `limit`, and `offset`
|
|
18
|
+
- Aggregation helpers: `count`, `sum`, `avg`, `min`, `max`, `variance`, `stddev`, `group_concat`, `stats`, and custom aggregations
|
|
19
|
+
- Named `simple_scope` definitions for reusable SimpleQuery chains
|
|
20
|
+
- `bulk_update` for set-based updates
|
|
21
|
+
- `stream_each` for large result sets on PostgreSQL and MySQL
|
|
4
22
|
|
|
5
23
|
## Installation
|
|
6
24
|
|
|
7
|
-
Add
|
|
25
|
+
Add SimpleQuery to your application's Gemfile:
|
|
8
26
|
|
|
9
27
|
```ruby
|
|
10
|
-
gem
|
|
28
|
+
gem "simple_query"
|
|
11
29
|
```
|
|
12
30
|
|
|
13
|
-
|
|
31
|
+
Then install:
|
|
32
|
+
|
|
14
33
|
```bash
|
|
15
34
|
bundle install
|
|
16
35
|
```
|
|
17
36
|
|
|
18
|
-
Or install it
|
|
37
|
+
Or install it directly:
|
|
38
|
+
|
|
19
39
|
```bash
|
|
20
40
|
gem install simple_query
|
|
21
41
|
```
|
|
22
42
|
|
|
43
|
+
## Compatibility
|
|
44
|
+
|
|
45
|
+
SimpleQuery currently declares support for:
|
|
46
|
+
|
|
47
|
+
- Ruby `>= 2.7`
|
|
48
|
+
- ActiveRecord `>= 7.0`, `< 8.2`
|
|
49
|
+
|
|
50
|
+
Queries run on the model's own connection, so models that use `connects_to` or `establish_connection` for another database work as expected.
|
|
51
|
+
|
|
52
|
+
The CI matrix covers ActiveRecord 7.0, 7.1, 7.2, 8.0, and 8.1 across PostgreSQL and MySQL. ActiveRecord 7.0 is covered through Ruby 3.2; ActiveRecord 8.0 and 8.1 require Ruby 3.2 or newer; otherwise each Ruby is exercised on the ActiveRecord versions that support it.
|
|
53
|
+
|
|
23
54
|
## Configuration
|
|
24
55
|
|
|
25
|
-
|
|
56
|
+
SimpleQuery does not patch every ActiveRecord model by default. Include it in the models that should expose `.simple_query`:
|
|
26
57
|
|
|
27
58
|
```ruby
|
|
28
|
-
# Manual include (per model)
|
|
29
59
|
class User < ActiveRecord::Base
|
|
30
60
|
include SimpleQuery
|
|
31
61
|
end
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Or include it globally if you want every ActiveRecord model to have `.simple_query`:
|
|
65
|
+
|
|
66
|
+
```ruby
|
|
67
|
+
# config/initializers/simple_query.rb
|
|
68
|
+
require "simple_query"
|
|
32
69
|
|
|
33
|
-
# or do it globally
|
|
34
70
|
ActiveRecord::Base.include(SimpleQuery)
|
|
35
71
|
```
|
|
36
|
-
|
|
72
|
+
|
|
73
|
+
You can also opt into global inclusion through SimpleQuery's configuration hook:
|
|
37
74
|
|
|
38
75
|
```ruby
|
|
39
76
|
# config/initializers/simple_query.rb
|
|
@@ -42,28 +79,144 @@ SimpleQuery.configure do |config|
|
|
|
42
79
|
end
|
|
43
80
|
```
|
|
44
81
|
|
|
45
|
-
|
|
82
|
+
## Basic usage
|
|
83
|
+
|
|
84
|
+
```ruby
|
|
85
|
+
users = User.simple_query
|
|
86
|
+
.select(:name, :email)
|
|
87
|
+
.where(active: true)
|
|
88
|
+
.order(name: :asc)
|
|
89
|
+
.limit(50)
|
|
90
|
+
.execute
|
|
91
|
+
|
|
92
|
+
users.first
|
|
93
|
+
# => #<struct name="Jane Doe", email="jane@example.com">
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
`execute` returns an array of lightweight `Struct` objects unless you map the query to a custom read model.
|
|
97
|
+
|
|
98
|
+
## Conditions
|
|
46
99
|
|
|
47
|
-
|
|
100
|
+
SimpleQuery accepts several condition styles.
|
|
48
101
|
|
|
49
|
-
|
|
102
|
+
### Hash conditions
|
|
50
103
|
|
|
51
|
-
Basic query
|
|
52
104
|
```ruby
|
|
53
|
-
User.simple_query
|
|
105
|
+
User.simple_query
|
|
106
|
+
.select(:name, :email)
|
|
107
|
+
.where(active: true, admin: false)
|
|
108
|
+
.execute
|
|
54
109
|
```
|
|
55
110
|
|
|
56
|
-
|
|
111
|
+
Hash conditions are predicates against the query's base table. Arrays become `IN (...)` (a `nil` element adds `OR ... IS NULL`), ranges become `BETWEEN`, and other values become equality checks. PostgreSQL array and range columns compare by equality, as in ActiveRecord.
|
|
112
|
+
|
|
113
|
+
### Arel conditions
|
|
57
114
|
|
|
58
|
-
SimpleQuery now supports **all major SQL join types** — including LEFT, RIGHT, and FULL — through the following DSL methods:
|
|
59
115
|
```ruby
|
|
116
|
+
users = User.arel_table
|
|
117
|
+
|
|
60
118
|
User.simple_query
|
|
61
|
-
.
|
|
62
|
-
.
|
|
119
|
+
.select(:name)
|
|
120
|
+
.where(users[:created_at].gteq(30.days.ago))
|
|
121
|
+
.execute
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### Placeholder conditions
|
|
125
|
+
|
|
126
|
+
Array conditions are sanitized through ActiveRecord's `sanitize_sql_array`, so they are the preferred way to use SQL fragments with external or user-provided values:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
escaped_search = ActiveRecord::Base.sanitize_sql_like(params[:search].to_s)
|
|
130
|
+
search_pattern = "%#{escaped_search}%"
|
|
131
|
+
|
|
132
|
+
User.simple_query
|
|
133
|
+
.where(["name LIKE ?", search_pattern])
|
|
134
|
+
.execute
|
|
135
|
+
|
|
136
|
+
User.simple_query
|
|
137
|
+
.where(["email = :email", { email: params[:email] }])
|
|
63
138
|
.execute
|
|
64
139
|
```
|
|
65
140
|
|
|
66
|
-
|
|
141
|
+
Keep the SQL fragment itself static and pass dynamic values as positional or named placeholders so ActiveRecord quotes them as SQL data literals. Do not interpolate request parameters, form values, or other untrusted input into the SQL string.
|
|
142
|
+
|
|
143
|
+
### Raw SQL conditions
|
|
144
|
+
|
|
145
|
+
Plain strings are treated as raw SQL fragments:
|
|
146
|
+
|
|
147
|
+
```ruby
|
|
148
|
+
User.simple_query
|
|
149
|
+
.where("active = TRUE")
|
|
150
|
+
.execute
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Raw SQL strings are an escape hatch for trusted, application-owned SQL only. If any value comes from a user, request, file, or external service, use hash, Arel, or placeholder conditions instead. The checklist below shows the safer patterns to prefer when query inputs are dynamic.
|
|
154
|
+
|
|
155
|
+
### SQL fragment safety checklist
|
|
156
|
+
|
|
157
|
+
SimpleQuery intentionally accepts SQL fragments in places where ActiveRecord also leaves judgment to the application, such as raw `where` strings, string `select` values, and `custom_aggregation` expressions. Treat those fragments as trusted code, not as a templating surface for request data.
|
|
158
|
+
|
|
159
|
+
Prefer these patterns for dynamic values:
|
|
160
|
+
|
|
161
|
+
```ruby
|
|
162
|
+
# Good: read dynamic values, then pass them separately so ActiveRecord quotes them.
|
|
163
|
+
email = params.fetch(:email)
|
|
164
|
+
User.simple_query
|
|
165
|
+
.where(["email = :email", { email: email }])
|
|
166
|
+
.execute
|
|
167
|
+
|
|
168
|
+
# Good: quote LIKE values with placeholders. Escape wildcard characters when
|
|
169
|
+
# `%` and `_` should be treated as literal search text instead of wildcards.
|
|
170
|
+
search = params.fetch(:search)
|
|
171
|
+
escaped_search = ActiveRecord::Base.sanitize_sql_like(search)
|
|
172
|
+
User.simple_query
|
|
173
|
+
.where(["name LIKE ?", "%#{escaped_search}%"])
|
|
174
|
+
.execute
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Avoid interpolating untrusted values into SQL fragments:
|
|
178
|
+
|
|
179
|
+
```ruby
|
|
180
|
+
# Unsafe: request data is interpolated into a raw SQL string.
|
|
181
|
+
User.simple_query
|
|
182
|
+
.where("email = '#{params[:email]}'")
|
|
183
|
+
.execute
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
Even with quotes around the value, interpolation is unsafe because an apostrophe or SQL fragment in the input can break out of the literal. Keep the SQL string static and pass values through placeholders so ActiveRecord quotes them as SQL data literals instead. For `LIKE` queries, `sanitize_sql_like` only changes matching semantics by escaping `%`, `_`, and the escape character itself; it is useful when those characters should be searched literally, but it does not replace placeholder quoting.
|
|
187
|
+
|
|
188
|
+
The same boundary applies outside `where`: keep selected expressions, ordering fragments, and custom aggregation SQL static and application-owned. When a user chooses a column, sort direction, or metric, map that input to a small allowlist before building the query:
|
|
189
|
+
|
|
190
|
+
```ruby
|
|
191
|
+
sort_columns = {
|
|
192
|
+
"name" => :name,
|
|
193
|
+
"created_at" => :created_at
|
|
194
|
+
}
|
|
195
|
+
sort_directions = {
|
|
196
|
+
"asc" => :asc,
|
|
197
|
+
"desc" => :desc
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
User.simple_query
|
|
201
|
+
.select(:id, :name, :email)
|
|
202
|
+
.order(sort_columns.fetch(params[:sort], :name) => sort_directions.fetch(params[:direction], :asc))
|
|
203
|
+
.execute
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
## Joins
|
|
207
|
+
|
|
208
|
+
Joins are explicit: pass the left table, right table, foreign key, and primary key.
|
|
209
|
+
|
|
210
|
+
```ruby
|
|
211
|
+
User.simple_query
|
|
212
|
+
.select(:name, :email)
|
|
213
|
+
.join(:users, :companies, foreign_key: :user_id, primary_key: :id)
|
|
214
|
+
.where(Company.arel_table[:name].eq("TechCorp"))
|
|
215
|
+
.execute
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Multiple joins can be chained:
|
|
219
|
+
|
|
67
220
|
```ruby
|
|
68
221
|
User.simple_query
|
|
69
222
|
.select(:name)
|
|
@@ -71,67 +224,125 @@ User.simple_query
|
|
|
71
224
|
.join(:companies, :projects, foreign_key: :company_id, primary_key: :id)
|
|
72
225
|
.where(Company.arel_table[:industry].eq("Technology"))
|
|
73
226
|
.where(Project.arel_table[:status].eq("active"))
|
|
74
|
-
.where(User.arel_table[:admin].eq(true))
|
|
75
227
|
.execute
|
|
76
228
|
```
|
|
77
229
|
|
|
78
|
-
|
|
230
|
+
Join helpers are also available:
|
|
231
|
+
|
|
79
232
|
```ruby
|
|
80
233
|
User.simple_query
|
|
234
|
+
.left_join(:users, :companies, foreign_key: :user_id, primary_key: :id)
|
|
81
235
|
.select(:name)
|
|
82
|
-
.
|
|
83
|
-
.lazy_execute
|
|
236
|
+
.execute
|
|
84
237
|
```
|
|
85
238
|
|
|
86
|
-
|
|
239
|
+
Supported join types:
|
|
240
|
+
|
|
241
|
+
- `join(..., type: :inner)` / `join(...)`
|
|
242
|
+
- `left_join(...)`
|
|
243
|
+
- `right_join(...)`
|
|
244
|
+
- `full_join(...)`
|
|
245
|
+
|
|
246
|
+
Database support for right and full outer joins depends on the adapter and database version.
|
|
87
247
|
|
|
88
|
-
|
|
248
|
+
## Selecting, ordering, and paging
|
|
89
249
|
|
|
90
250
|
```ruby
|
|
91
|
-
# Positional placeholders:
|
|
92
251
|
User.simple_query
|
|
93
|
-
.
|
|
252
|
+
.select(:id, :name, "LOWER(email) AS normalized_email")
|
|
253
|
+
.distinct
|
|
254
|
+
.where(active: true)
|
|
255
|
+
.order(name: :asc)
|
|
256
|
+
.limit(25)
|
|
257
|
+
.offset(50)
|
|
94
258
|
.execute
|
|
259
|
+
```
|
|
95
260
|
|
|
96
|
-
|
|
97
|
-
User.simple_query
|
|
98
|
-
.where(["email = :email", { email: "alice@example.com" }])
|
|
99
|
-
.execute
|
|
261
|
+
`select` accepts symbols, SQL strings, and Arel nodes. SQL strings are inserted as SQL fragments, so keep them trusted.
|
|
100
262
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
263
|
+
`order` accepts a hash of column names to directions, such as `order(created_at: :desc)`.
|
|
264
|
+
|
|
265
|
+
## Aggregations
|
|
266
|
+
|
|
267
|
+
SimpleQuery can build common aggregate expressions without hand-writing each SQL expression.
|
|
268
|
+
|
|
269
|
+
```ruby
|
|
270
|
+
Company.simple_query
|
|
271
|
+
.count
|
|
272
|
+
.sum(:annual_revenue)
|
|
273
|
+
.avg(:annual_revenue)
|
|
274
|
+
.min(:annual_revenue)
|
|
275
|
+
.max(:annual_revenue)
|
|
276
|
+
.execute
|
|
105
277
|
```
|
|
106
278
|
|
|
107
|
-
|
|
108
|
-
By default, SimpleQuery returns results as `Struct` objects for maximum speed. However, you can also define a lightweight model class for more explicit attribute handling or custom logic.
|
|
279
|
+
Custom aliases are supported:
|
|
109
280
|
|
|
110
|
-
**Create a read model** inheriting from `SimpleQuery::ReadModel`:
|
|
111
281
|
```ruby
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
282
|
+
Company.simple_query
|
|
283
|
+
.count(:id, alias_name: "company_count")
|
|
284
|
+
.sum(:annual_revenue, alias_name: "total_revenue")
|
|
285
|
+
.execute
|
|
116
286
|
```
|
|
117
287
|
|
|
118
|
-
|
|
288
|
+
Grouped aggregations work with selected fields:
|
|
289
|
+
|
|
119
290
|
```ruby
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
291
|
+
Company.simple_query
|
|
292
|
+
.select(:industry)
|
|
293
|
+
.count(alias_name: "company_count")
|
|
294
|
+
.sum(:annual_revenue, alias_name: "total_revenue")
|
|
295
|
+
.group(:industry)
|
|
296
|
+
.execute
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
Convenience helpers include:
|
|
300
|
+
|
|
301
|
+
```ruby
|
|
302
|
+
Company.simple_query.stats(:annual_revenue).execute
|
|
303
|
+
Company.simple_query.total_count.execute
|
|
304
|
+
User.simple_query.variance(:score).stddev(:score).execute
|
|
305
|
+
User.simple_query.group_concat(:name, separator: ", ").execute
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
For database-specific expressions, use `custom_aggregation` with trusted SQL:
|
|
309
|
+
|
|
310
|
+
```ruby
|
|
311
|
+
Company.simple_query
|
|
312
|
+
.custom_aggregation("COUNT(DISTINCT industry)", "unique_industries")
|
|
313
|
+
.execute
|
|
314
|
+
```
|
|
125
315
|
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
316
|
+
## Custom read models
|
|
317
|
+
|
|
318
|
+
By default, query results are returned as `Struct` objects. For named methods and domain-specific result objects, define a read model:
|
|
319
|
+
|
|
320
|
+
```ruby
|
|
321
|
+
class UserSummary < SimpleQuery::ReadModel
|
|
322
|
+
attribute :identifier, column: :id
|
|
323
|
+
attribute :full_name, column: :name
|
|
129
324
|
end
|
|
130
325
|
```
|
|
131
|
-
This custom read model approach provides more clarity or domain-specific logic while still being faster than typical ActiveRecord instantiation.
|
|
132
326
|
|
|
133
|
-
|
|
134
|
-
|
|
327
|
+
Map query results to the read model:
|
|
328
|
+
|
|
329
|
+
```ruby
|
|
330
|
+
users = User.simple_query
|
|
331
|
+
.select("users.id AS id", "users.name AS name")
|
|
332
|
+
.where(active: true)
|
|
333
|
+
.map_to(UserSummary)
|
|
334
|
+
.execute
|
|
335
|
+
|
|
336
|
+
users.first.identifier
|
|
337
|
+
users.first.full_name
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
The selected SQL aliases must match the read model's configured column names.
|
|
341
|
+
|
|
342
|
+
## Named scopes
|
|
343
|
+
|
|
344
|
+
Use `simple_scope` to define reusable query fragments on a model that includes SimpleQuery:
|
|
345
|
+
|
|
135
346
|
```ruby
|
|
136
347
|
class User < ActiveRecord::Base
|
|
137
348
|
include SimpleQuery
|
|
@@ -140,119 +351,239 @@ class User < ActiveRecord::Base
|
|
|
140
351
|
where(active: true)
|
|
141
352
|
end
|
|
142
353
|
|
|
143
|
-
simple_scope :admins
|
|
144
|
-
where(admin: true)
|
|
145
|
-
end
|
|
354
|
+
simple_scope :admins, -> { where(admin: true) }
|
|
146
355
|
|
|
147
|
-
# Block-based scope with parameter
|
|
148
356
|
simple_scope :by_name do |name|
|
|
149
357
|
where(name: name)
|
|
150
358
|
end
|
|
151
|
-
|
|
152
|
-
# Lambda-based scope with parameter
|
|
153
|
-
simple_scope :by_name, ->(name) { where(name: name) }
|
|
154
359
|
end
|
|
155
360
|
```
|
|
156
|
-
|
|
361
|
+
|
|
362
|
+
Scopes are evaluated in the context of the SimpleQuery builder and can be chained with the normal DSL:
|
|
157
363
|
|
|
158
364
|
```ruby
|
|
159
|
-
|
|
160
|
-
|
|
365
|
+
User.simple_query
|
|
366
|
+
.active
|
|
367
|
+
.admins
|
|
368
|
+
.by_name("Jane Doe")
|
|
369
|
+
.select(:id, :name)
|
|
370
|
+
.execute
|
|
371
|
+
```
|
|
161
372
|
|
|
162
|
-
|
|
163
|
-
results = User.simple_query.by_name("Jane Doe").execute
|
|
373
|
+
SimpleQuery validates positional scope arguments before invoking the scope body. If a scope is called with too few or too many positional arguments, it raises `ArgumentError` with the scope name and expected argument count. For scopes that declare Ruby keyword parameters, SimpleQuery forwards keywords to the scope body and lets Ruby raise its native keyword-argument errors.
|
|
164
374
|
|
|
165
|
-
|
|
166
|
-
results = User.simple_query
|
|
167
|
-
.by_name("John")
|
|
168
|
-
.active
|
|
169
|
-
.select(:id, :name)
|
|
170
|
-
.order(name: :asc)
|
|
171
|
-
.execute
|
|
172
|
-
```
|
|
173
|
-
### How It Works
|
|
375
|
+
## Lazy execution and streaming
|
|
174
376
|
|
|
175
|
-
|
|
176
|
-
Parameterized scopes accept arguments — passed directly to the block (e.g. |name| above).
|
|
177
|
-
Scopes return self, so you can chain multiple scopes or mix them with standard query methods.
|
|
377
|
+
`lazy_execute` returns an `Enumerator` over the query results:
|
|
178
378
|
|
|
179
|
-
|
|
379
|
+
```ruby
|
|
380
|
+
enumerator = User.simple_query
|
|
381
|
+
.select(:name)
|
|
382
|
+
.where(active: true)
|
|
383
|
+
.lazy_execute
|
|
384
|
+
|
|
385
|
+
enumerator.each do |user|
|
|
386
|
+
puts user.name
|
|
387
|
+
end
|
|
388
|
+
```
|
|
180
389
|
|
|
181
|
-
|
|
390
|
+
`lazy_execute` is useful when you want enumerator-style consumption, but it still uses ActiveRecord's `select_all` internally.
|
|
182
391
|
|
|
183
|
-
|
|
184
|
-
- **MySQL**: Uses `mysql2` gem’s **streaming** (`stream: true, cache_rows: false, as: :hash`).
|
|
392
|
+
For large PostgreSQL or MySQL result sets, use `stream_each`:
|
|
185
393
|
|
|
186
394
|
```ruby
|
|
187
|
-
# Example usage:
|
|
188
395
|
User.simple_query
|
|
396
|
+
.select(:id, :email)
|
|
189
397
|
.where(active: true)
|
|
190
|
-
.stream_each(batch_size: 10_000) do |
|
|
191
|
-
|
|
192
|
-
puts row.name
|
|
398
|
+
.stream_each(batch_size: 10_000) do |user|
|
|
399
|
+
puts user.email
|
|
193
400
|
end
|
|
194
401
|
```
|
|
195
402
|
|
|
403
|
+
Adapter behavior:
|
|
404
|
+
|
|
405
|
+
- PostgreSQL: uses a server-side cursor with `DECLARE` / `FETCH`; `batch_size` must be a positive integer and controls the number of rows fetched per cursor read
|
|
406
|
+
- MySQL: uses `mysql2` streaming with `stream: true`, `cache_rows: false`, and `as: :hash`; `batch_size` is validated as a positive integer for API symmetry, but mysql2 streams rows directly and does not expose a SimpleQuery-controlled batch size
|
|
407
|
+
- Other adapters: `stream_each` raises `SimpleQuery::UnsupportedAdapterError` with the current adapter name
|
|
408
|
+
|
|
409
|
+
Failure behavior:
|
|
410
|
+
|
|
411
|
+
- `batch_size` must be a positive integer; invalid values raise `ArgumentError` before any adapter-specific SQL runs
|
|
412
|
+
- PostgreSQL streaming wraps the cursor in a transaction, closes a declared cursor before rollback when row processing fails, attempts cursor cleanup before rollback on fetch failures, and propagates the original error
|
|
413
|
+
- MySQL streaming uses the mysql2 driver's streaming result handling; query and row processing errors are propagated to the caller
|
|
414
|
+
|
|
415
|
+
## Bulk updates
|
|
416
|
+
|
|
417
|
+
`bulk_update` builds and executes a set-based `UPDATE` for the current query conditions:
|
|
418
|
+
|
|
419
|
+
```ruby
|
|
420
|
+
User.simple_query
|
|
421
|
+
.where(active: false)
|
|
422
|
+
.bulk_update(set: { status: 0 })
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
`bulk_update` sends SQL directly through the ActiveRecord connection and does not instantiate models or run ActiveRecord callbacks. If no `where` conditions are present, it updates the entire table. Joins, `limit`/`offset`, and `group`/`having` are not applied to the `UPDATE`, so `bulk_update` raises `ArgumentError` when any of them are present.
|
|
426
|
+
|
|
427
|
+
## Subqueries
|
|
428
|
+
|
|
429
|
+
You can use `build_query` to embed a SimpleQuery query as an Arel subquery:
|
|
430
|
+
|
|
431
|
+
```ruby
|
|
432
|
+
company_users = Company.simple_query
|
|
433
|
+
.select(:user_id)
|
|
434
|
+
.where(industry: "Technology")
|
|
435
|
+
.build_query
|
|
436
|
+
|
|
437
|
+
User.simple_query
|
|
438
|
+
.select(:name)
|
|
439
|
+
.where(User.arel_table[:id].in(company_users))
|
|
440
|
+
.execute
|
|
441
|
+
```
|
|
442
|
+
|
|
196
443
|
## Performance
|
|
197
444
|
|
|
198
|
-
SimpleQuery
|
|
445
|
+
SimpleQuery is designed for read paths where ActiveRecord model instantiation is unnecessary overhead. Returning structs or read models can reduce allocation costs for large reporting-style queries.
|
|
446
|
+
|
|
447
|
+
The repository includes an optional benchmark harness comparing ActiveRecord object loading and `update_all` with SimpleQuery structs, read models, and `bulk_update`. Treat benchmark numbers as workload-specific: validate them against your database, indexes, adapter, Ruby version, and query shape before making production claims.
|
|
448
|
+
|
|
449
|
+
Run the reproducible local benchmark with:
|
|
450
|
+
|
|
451
|
+
```sh
|
|
452
|
+
bundle exec rake benchmark:reproducible
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
The harness uses SQLite by default, seeds deterministic data, warms up and repeats each timing measurement, and prints JSON with environment metadata plus timing results. If `memory_profiler` is available in your development bundle, the JSON also includes single-run allocation profiles for the read-path scenarios. Tune the workload with environment variables:
|
|
456
|
+
|
|
457
|
+
```sh
|
|
458
|
+
BENCHMARK_ROWS=50000 BENCHMARK_WARMUP=3 BENCHMARK_RUNS=10 \
|
|
459
|
+
bundle exec rake benchmark:reproducible
|
|
460
|
+
```
|
|
461
|
+
|
|
462
|
+
Available variables:
|
|
463
|
+
|
|
464
|
+
- `BENCHMARK_ROWS` (default: `10000`) controls the number of users and matching companies seeded.
|
|
465
|
+
- `BENCHMARK_WARMUP` (default: `2`) controls warmup iterations before samples are collected.
|
|
466
|
+
- `BENCHMARK_RUNS` (default: `5`) controls the number of timed samples per scenario.
|
|
467
|
+
- `BENCHMARK_DATABASE` (default: `:memory:`) can point at a SQLite database file if you want to inspect the generated benchmark-specific tables. The harness drops and recreates `simple_query_benchmark_users` and `simple_query_benchmark_companies` in that database on every run.
|
|
468
|
+
|
|
469
|
+
The benchmark task writes JSON to standard output so it can be redirected into an artifact file. Keep progress messages or diagnostics on standard error if you extend the harness later.
|
|
470
|
+
|
|
471
|
+
To preserve a run as a reviewable artifact, capture the JSON output to a file, validate that the artifact is parseable JSON, and keep the workload settings alongside it:
|
|
472
|
+
|
|
473
|
+
```sh
|
|
474
|
+
set -e
|
|
475
|
+
mkdir -p tmp/benchmarks
|
|
476
|
+
export BENCHMARK_ROWS=50000
|
|
477
|
+
export BENCHMARK_WARMUP=3
|
|
478
|
+
export BENCHMARK_RUNS=10
|
|
479
|
+
bundle exec rake benchmark:reproducible > tmp/benchmarks/simple_query-reproducible.json
|
|
480
|
+
ruby -rjson -e 'JSON.parse(File.read(ARGV.fetch(0)))' tmp/benchmarks/simple_query-reproducible.json
|
|
481
|
+
```
|
|
199
482
|
|
|
200
|
-
|
|
483
|
+
Before sharing or comparing a result, record enough environment metadata to recreate the run. The benchmark JSON includes runtime details, key dependency versions, and exported `BENCHMARK_*` values. When the task runs from a Git checkout, it also includes the current Git revision and dirty-checkout status; otherwise those Git fields are `null`. It is still useful to save companion hardware notes and the short Git status from the same shell session:
|
|
484
|
+
|
|
485
|
+
```sh
|
|
486
|
+
set -e
|
|
487
|
+
mkdir -p tmp/benchmarks
|
|
488
|
+
{
|
|
489
|
+
echo "git_revision=$(git describe --always --dirty)"
|
|
490
|
+
echo "git_status_short_start"
|
|
491
|
+
git status --short
|
|
492
|
+
echo "git_status_short_end"
|
|
493
|
+
echo "hardware=$(uname -a)"
|
|
494
|
+
} > tmp/benchmarks/simple_query-reproducible.metadata
|
|
201
495
|
```
|
|
202
|
-
🚀 Performance Results (1000,000 records):
|
|
203
|
-
ActiveRecord Query: 10.36932 seconds
|
|
204
|
-
SimpleQuery Execution (Struct): 3.46136 seconds
|
|
205
|
-
SimpleQuery Execution (Read model): 2.20905 seconds
|
|
206
496
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
497
|
+
Use captured benchmark output only as a reproducibility artifact. Do not compare runs from different hardware, databases, Ruby versions, dependency sets, dataset sizes, warmup counts, run counts, or benchmark databases as if they were equivalent. Prefer comparing changes by rerunning the same command on the same machine from a clean checkout of each revision, and treat non-empty `git status --short` metadata as a signal that the run may not be reproducible from the recorded revision alone. Scrub sensitive environment values before sharing artifacts if any `BENCHMARK_*` setting contains credentials, private paths, or hostnames.
|
|
498
|
+
|
|
499
|
+
## Safety notes
|
|
500
|
+
|
|
501
|
+
SimpleQuery is a query-building library, not an authorization or SQL-injection protection layer. It exposes both parameterized condition helpers and raw SQL escape hatches; choose the narrowest API that fits the value source.
|
|
502
|
+
|
|
503
|
+
Recommended usage:
|
|
504
|
+
|
|
505
|
+
- Prefer hash, Arel, or placeholder conditions for values derived from users or external systems.
|
|
506
|
+
- Treat raw SQL strings in `select`, `where`, and `custom_aggregation` as trusted-only escape hatches.
|
|
507
|
+
- Keep `group_concat` separators static and trusted; they are interpolated into database-specific SQL.
|
|
508
|
+
- `having` accepts the same condition styles as `where`; raw strings are trusted-only there too.
|
|
509
|
+
- Keep tenant, authorization, and visibility constraints explicit in your query code.
|
|
510
|
+
- Remember that `bulk_update` bypasses model callbacks and validations, like any direct SQL update.
|
|
511
|
+
|
|
512
|
+
### Parameterized values
|
|
210
513
|
|
|
211
|
-
|
|
212
|
-
Total allocated: 1.98 GB (16,001,659 objects)
|
|
213
|
-
Retained: ~2 KB
|
|
514
|
+
Use hash conditions for simple equality predicates and array placeholder conditions for SQL fragments that include external values:
|
|
214
515
|
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
516
|
+
```ruby
|
|
517
|
+
# Equality predicates are built through Arel.
|
|
518
|
+
User.simple_query.where(email: params[:email]).execute
|
|
519
|
+
|
|
520
|
+
# SQL fragments with external values should use ActiveRecord-style placeholders.
|
|
521
|
+
query = params[:query].to_s
|
|
522
|
+
escaped_query = ActiveRecord::Base.sanitize_sql_like(query)
|
|
523
|
+
|
|
524
|
+
User.simple_query
|
|
525
|
+
.where(["name ILIKE ?", "%#{escaped_query}%"])
|
|
526
|
+
.execute
|
|
527
|
+
|
|
528
|
+
User.simple_query
|
|
529
|
+
.where(["email = :email", { email: params[:email] }])
|
|
530
|
+
.execute
|
|
218
531
|
```
|
|
219
|
-
- **Struct-based** approach remains the fastest, skipping model overhead.
|
|
220
|
-
- **Read model** approach is still significantly faster than standard ActiveRecord while allowing domain-specific logic.
|
|
221
532
|
|
|
222
|
-
|
|
533
|
+
Placeholders quote values as SQL data literals, while `sanitize_sql_like` makes `%` and `_` match literally in `LIKE` patterns; escaping the pattern does not replace placeholder quoting.
|
|
534
|
+
|
|
535
|
+
Do not interpolate external values into raw strings:
|
|
536
|
+
|
|
537
|
+
```ruby
|
|
538
|
+
# Unsafe: params[:query] is part of the SQL string itself.
|
|
539
|
+
User.simple_query.where("name ILIKE '%#{params[:query]}%'").execute
|
|
223
540
|
```
|
|
224
|
-
🚀 Performance Results (1000,000 records):
|
|
225
|
-
ActiveRecord Query: 10.45833 seconds
|
|
226
|
-
SimpleQuery Execution (Struct): 3.04655 seconds
|
|
227
|
-
SimpleQuery Execution (Read model): 3.69052 seconds
|
|
228
541
|
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
SimpleQuery
|
|
542
|
+
### Trusted SQL escape hatches
|
|
543
|
+
|
|
544
|
+
Some APIs intentionally accept SQL fragments because they are meant for expressions that Arel or SimpleQuery helpers do not model:
|
|
232
545
|
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
546
|
+
```ruby
|
|
547
|
+
# Trusted expression: fixed by the application, not assembled from request input.
|
|
548
|
+
User.simple_query.select("LOWER(email) AS normalized_email").execute
|
|
236
549
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
550
|
+
Company.simple_query
|
|
551
|
+
.custom_aggregation("COUNT(DISTINCT industry)", "unique_industries")
|
|
552
|
+
.execute
|
|
240
553
|
```
|
|
241
|
-
|
|
242
|
-
|
|
554
|
+
|
|
555
|
+
Keep aliases, custom aggregation expressions, raw `select` strings, raw `where` strings, `having` expressions, and `group_concat` separators static or otherwise trusted. If a condition must include external values, express those values with hash, Arel, or placeholder `where` syntax instead of interpolating them into raw SQL fragments.
|
|
556
|
+
|
|
557
|
+
### Direct updates
|
|
558
|
+
|
|
559
|
+
`bulk_update` is a set-based SQL update. It is useful for maintenance jobs and data workflows where callbacks are intentionally skipped, but it will not run ActiveRecord validations, callbacks, authorization checks, or per-record business logic. Always add the intended `where` conditions before calling it unless the whole table is the target.
|
|
243
560
|
|
|
244
561
|
## Development
|
|
245
562
|
|
|
246
|
-
After checking out the
|
|
563
|
+
After checking out the repository:
|
|
564
|
+
|
|
565
|
+
```bash
|
|
566
|
+
bin/setup
|
|
567
|
+
bundle exec rspec
|
|
568
|
+
bundle exec rubocop
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
Database-sensitive behavior is tested against PostgreSQL and MySQL in CI.
|
|
572
|
+
|
|
573
|
+
You can also open a console:
|
|
574
|
+
|
|
575
|
+
```bash
|
|
576
|
+
bin/console
|
|
577
|
+
```
|
|
247
578
|
|
|
248
579
|
## Contributing
|
|
249
580
|
|
|
250
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/kholdrex/simple_query.
|
|
581
|
+
Bug reports and pull requests are welcome on GitHub at [kholdrex/simple_query](https://github.com/kholdrex/simple_query). Contributors are expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|
|
251
582
|
|
|
252
583
|
## License
|
|
253
584
|
|
|
254
|
-
The gem is available as open source under the terms of the [MIT License](
|
|
585
|
+
The gem is available as open source under the terms of the [MIT License](LICENSE.txt).
|
|
255
586
|
|
|
256
587
|
## Code of Conduct
|
|
257
588
|
|
|
258
|
-
Everyone interacting in the SimpleQuery project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [
|
|
589
|
+
Everyone interacting in the SimpleQuery project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|