simple_query 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +428 -185
- data/lib/simple_query/builder.rb +115 -23
- data/lib/simple_query/clauses/aggregation_clause.rb +8 -5
- data/lib/simple_query/clauses/group_having_clause.rb +6 -6
- data/lib/simple_query/clauses/set_clause.rb +4 -3
- data/lib/simple_query/clauses/where_clause.rb +34 -9
- data/lib/simple_query/stream/mysql_stream.rb +9 -4
- data/lib/simple_query/stream/postgres_stream.rb +45 -11
- data/lib/simple_query/version.rb +1 -1
- data/lib/simple_query.rb +10 -4
- metadata +6 -6
data/README.md
CHANGED
|
@@ -1,39 +1,76 @@
|
|
|
1
1
|
# SimpleQuery
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://badge.fury.io/rb/simple_query)
|
|
4
|
+
[](https://github.com/kholdrex/simple_query/actions/workflows/ci.yml)
|
|
5
|
+
|
|
6
|
+
SimpleQuery is a lightweight query builder for ActiveRecord. It gives Rails applications an explicit, chainable API for read-heavy queries, joins, aggregations, read models, bulk updates, and adapter-aware streaming without replacing ActiveRecord.
|
|
7
|
+
|
|
8
|
+
Use it when you want ActiveRecord-backed SQL with less object-instantiation overhead, predictable result shapes, or a small query/read-model layer around reporting and data-processing code.
|
|
9
|
+
|
|
10
|
+
## Features
|
|
11
|
+
|
|
12
|
+
- Chainable query builder for ActiveRecord models
|
|
13
|
+
- Struct results by default for lightweight reads
|
|
14
|
+
- Optional custom read models with explicit attributes
|
|
15
|
+
- Hash, Arel, raw SQL, and ActiveRecord-style placeholder conditions
|
|
16
|
+
- Inner, left, right, and full joins
|
|
17
|
+
- `distinct`, `group`, `having`, `order`, `limit`, and `offset`
|
|
18
|
+
- Aggregation helpers: `count`, `sum`, `avg`, `min`, `max`, `variance`, `stddev`, `group_concat`, `stats`, and custom aggregations
|
|
19
|
+
- Named `simple_scope` definitions for reusable SimpleQuery chains
|
|
20
|
+
- `bulk_update` for set-based updates
|
|
21
|
+
- `stream_each` for large result sets on PostgreSQL and MySQL
|
|
4
22
|
|
|
5
23
|
## Installation
|
|
6
24
|
|
|
7
|
-
Add
|
|
25
|
+
Add SimpleQuery to your application's Gemfile:
|
|
8
26
|
|
|
9
27
|
```ruby
|
|
10
|
-
gem
|
|
28
|
+
gem "simple_query"
|
|
11
29
|
```
|
|
12
30
|
|
|
13
|
-
|
|
31
|
+
Then install:
|
|
32
|
+
|
|
14
33
|
```bash
|
|
15
34
|
bundle install
|
|
16
35
|
```
|
|
17
36
|
|
|
18
|
-
Or install it
|
|
37
|
+
Or install it directly:
|
|
38
|
+
|
|
19
39
|
```bash
|
|
20
40
|
gem install simple_query
|
|
21
41
|
```
|
|
22
42
|
|
|
43
|
+
## Compatibility
|
|
44
|
+
|
|
45
|
+
SimpleQuery currently declares support for:
|
|
46
|
+
|
|
47
|
+
- Ruby `>= 2.7`
|
|
48
|
+
- ActiveRecord `>= 7.0`, `< 8.2`
|
|
49
|
+
|
|
50
|
+
Queries run on the model's own connection, so models that use `connects_to` or `establish_connection` for another database work as expected.
|
|
51
|
+
|
|
52
|
+
The CI matrix covers ActiveRecord 7.0, 7.1, 7.2, 8.0, and 8.1 across PostgreSQL and MySQL. ActiveRecord 7.0 is covered through Ruby 3.2; ActiveRecord 8.0 and 8.1 require Ruby 3.2 or newer; otherwise each Ruby is exercised on the ActiveRecord versions that support it.
|
|
53
|
+
|
|
23
54
|
## Configuration
|
|
24
55
|
|
|
25
|
-
|
|
56
|
+
SimpleQuery does not patch every ActiveRecord model by default. Include it in the models that should expose `.simple_query`:
|
|
26
57
|
|
|
27
58
|
```ruby
|
|
28
|
-
# Manual include (per model)
|
|
29
59
|
class User < ActiveRecord::Base
|
|
30
60
|
include SimpleQuery
|
|
31
61
|
end
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Or include it globally if you want every ActiveRecord model to have `.simple_query`:
|
|
65
|
+
|
|
66
|
+
```ruby
|
|
67
|
+
# config/initializers/simple_query.rb
|
|
68
|
+
require "simple_query"
|
|
32
69
|
|
|
33
|
-
# or do it globally
|
|
34
70
|
ActiveRecord::Base.include(SimpleQuery)
|
|
35
71
|
```
|
|
36
|
-
|
|
72
|
+
|
|
73
|
+
You can also opt into global inclusion through SimpleQuery's configuration hook:
|
|
37
74
|
|
|
38
75
|
```ruby
|
|
39
76
|
# config/initializers/simple_query.rb
|
|
@@ -42,184 +79,270 @@ SimpleQuery.configure do |config|
|
|
|
42
79
|
end
|
|
43
80
|
```
|
|
44
81
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
## Usage
|
|
82
|
+
## Basic usage
|
|
48
83
|
|
|
49
|
-
SimpleQuery offers an intuitive interface for building queries with joins, conditions, and aggregations. Here are some examples:
|
|
50
|
-
|
|
51
|
-
Basic query
|
|
52
84
|
```ruby
|
|
53
|
-
User.simple_query
|
|
85
|
+
users = User.simple_query
|
|
86
|
+
.select(:name, :email)
|
|
87
|
+
.where(active: true)
|
|
88
|
+
.order(name: :asc)
|
|
89
|
+
.limit(50)
|
|
90
|
+
.execute
|
|
91
|
+
|
|
92
|
+
users.first
|
|
93
|
+
# => #<struct name="Jane Doe", email="jane@example.com">
|
|
54
94
|
```
|
|
55
95
|
|
|
56
|
-
|
|
96
|
+
`execute` returns an array of lightweight `Struct` objects unless you map the query to a custom read model.
|
|
97
|
+
|
|
98
|
+
## Conditions
|
|
99
|
+
|
|
100
|
+
SimpleQuery accepts several condition styles.
|
|
101
|
+
|
|
102
|
+
### Hash conditions
|
|
57
103
|
|
|
58
|
-
SimpleQuery now supports **all major SQL join types** — including LEFT, RIGHT, and FULL — through the following DSL methods:
|
|
59
104
|
```ruby
|
|
60
105
|
User.simple_query
|
|
61
|
-
.
|
|
62
|
-
.
|
|
106
|
+
.select(:name, :email)
|
|
107
|
+
.where(active: true, admin: false)
|
|
63
108
|
.execute
|
|
64
109
|
```
|
|
65
110
|
|
|
66
|
-
|
|
111
|
+
Hash conditions are predicates against the query's base table. Arrays become `IN (...)` (a `nil` element adds `OR ... IS NULL`), ranges become `BETWEEN`, and other values become equality checks. PostgreSQL array and range columns compare by equality, as in ActiveRecord.
|
|
112
|
+
|
|
113
|
+
### Arel conditions
|
|
114
|
+
|
|
67
115
|
```ruby
|
|
116
|
+
users = User.arel_table
|
|
117
|
+
|
|
68
118
|
User.simple_query
|
|
69
119
|
.select(:name)
|
|
70
|
-
.
|
|
71
|
-
.join(:companies, :projects, foreign_key: :company_id, primary_key: :id)
|
|
72
|
-
.where(Company.arel_table[:industry].eq("Technology"))
|
|
73
|
-
.where(Project.arel_table[:status].eq("active"))
|
|
74
|
-
.where(User.arel_table[:admin].eq(true))
|
|
120
|
+
.where(users[:created_at].gteq(30.days.ago))
|
|
75
121
|
.execute
|
|
76
122
|
```
|
|
77
123
|
|
|
78
|
-
|
|
124
|
+
### Placeholder conditions
|
|
125
|
+
|
|
126
|
+
Array conditions are sanitized through ActiveRecord's `sanitize_sql_array`, so they are the preferred way to use SQL fragments with external or user-provided values:
|
|
127
|
+
|
|
79
128
|
```ruby
|
|
129
|
+
escaped_search = ActiveRecord::Base.sanitize_sql_like(params[:search].to_s)
|
|
130
|
+
search_pattern = "%#{escaped_search}%"
|
|
131
|
+
|
|
80
132
|
User.simple_query
|
|
81
|
-
.
|
|
82
|
-
.
|
|
83
|
-
|
|
133
|
+
.where(["name LIKE ?", search_pattern])
|
|
134
|
+
.execute
|
|
135
|
+
|
|
136
|
+
User.simple_query
|
|
137
|
+
.where(["email = :email", { email: params[:email] }])
|
|
138
|
+
.execute
|
|
84
139
|
```
|
|
85
140
|
|
|
86
|
-
|
|
141
|
+
Keep the SQL fragment itself static and pass dynamic values as positional or named placeholders so ActiveRecord quotes them as SQL data literals. Do not interpolate request parameters, form values, or other untrusted input into the SQL string.
|
|
142
|
+
|
|
143
|
+
### Raw SQL conditions
|
|
87
144
|
|
|
88
|
-
|
|
145
|
+
Plain strings are treated as raw SQL fragments:
|
|
89
146
|
|
|
90
147
|
```ruby
|
|
91
|
-
# Positional placeholders:
|
|
92
148
|
User.simple_query
|
|
93
|
-
.where(
|
|
149
|
+
.where("active = TRUE")
|
|
94
150
|
.execute
|
|
151
|
+
```
|
|
95
152
|
|
|
96
|
-
|
|
153
|
+
Raw SQL strings are an escape hatch for trusted, application-owned SQL only. If any value comes from a user, request, file, or external service, use hash, Arel, or placeholder conditions instead. The checklist below shows the safer patterns to prefer when query inputs are dynamic.
|
|
154
|
+
|
|
155
|
+
### SQL fragment safety checklist
|
|
156
|
+
|
|
157
|
+
SimpleQuery intentionally accepts SQL fragments in places where ActiveRecord also leaves judgment to the application, such as raw `where` strings, string `select` values, and `custom_aggregation` expressions. Treat those fragments as trusted code, not as a templating surface for request data.
|
|
158
|
+
|
|
159
|
+
Prefer these patterns for dynamic values:
|
|
160
|
+
|
|
161
|
+
```ruby
|
|
162
|
+
# Good: read dynamic values, then pass them separately so ActiveRecord quotes them.
|
|
163
|
+
email = params.fetch(:email)
|
|
97
164
|
User.simple_query
|
|
98
|
-
.where(["email = :email", { email:
|
|
165
|
+
.where(["email = :email", { email: email }])
|
|
99
166
|
.execute
|
|
100
167
|
|
|
101
|
-
#
|
|
168
|
+
# Good: quote LIKE values with placeholders. Escape wildcard characters when
|
|
169
|
+
# `%` and `_` should be treated as literal search text instead of wildcards.
|
|
170
|
+
search = params.fetch(:search)
|
|
171
|
+
escaped_search = ActiveRecord::Base.sanitize_sql_like(search)
|
|
102
172
|
User.simple_query
|
|
103
|
-
.where(["
|
|
173
|
+
.where(["name LIKE ?", "%#{escaped_search}%"])
|
|
104
174
|
.execute
|
|
105
175
|
```
|
|
106
176
|
|
|
107
|
-
|
|
177
|
+
Avoid interpolating untrusted values into SQL fragments:
|
|
108
178
|
|
|
109
|
-
|
|
179
|
+
```ruby
|
|
180
|
+
# Unsafe: request data is interpolated into a raw SQL string.
|
|
181
|
+
User.simple_query
|
|
182
|
+
.where("email = '#{params[:email]}'")
|
|
183
|
+
.execute
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
Even with quotes around the value, interpolation is unsafe because an apostrophe or SQL fragment in the input can break out of the literal. Keep the SQL string static and pass values through placeholders so ActiveRecord quotes them as SQL data literals instead. For `LIKE` queries, `sanitize_sql_like` only changes matching semantics by escaping `%`, `_`, and the escape character itself; it is useful when those characters should be searched literally, but it does not replace placeholder quoting.
|
|
110
187
|
|
|
111
|
-
|
|
188
|
+
The same boundary applies outside `where`: keep selected expressions, ordering fragments, and custom aggregation SQL static and application-owned. When a user chooses a column, sort direction, or metric, map that input to a small allowlist before building the query:
|
|
112
189
|
|
|
113
190
|
```ruby
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
191
|
+
sort_columns = {
|
|
192
|
+
"name" => :name,
|
|
193
|
+
"created_at" => :created_at
|
|
194
|
+
}
|
|
195
|
+
sort_directions = {
|
|
196
|
+
"asc" => :asc,
|
|
197
|
+
"desc" => :desc
|
|
198
|
+
}
|
|
117
199
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
200
|
+
User.simple_query
|
|
201
|
+
.select(:id, :name, :email)
|
|
202
|
+
.order(sort_columns.fetch(params[:sort], :name) => sort_directions.fetch(params[:direction], :asc))
|
|
203
|
+
.execute
|
|
204
|
+
```
|
|
121
205
|
|
|
122
|
-
|
|
123
|
-
Company.simple_query.sum(:annual_revenue).execute
|
|
124
|
-
# => #<struct sum_annual_revenue=50000000>
|
|
206
|
+
## Joins
|
|
125
207
|
|
|
126
|
-
|
|
127
|
-
Company.simple_query.avg(:annual_revenue).execute
|
|
128
|
-
# => #<struct avg_annual_revenue=1000000.5>
|
|
208
|
+
Joins are explicit: pass the left table, right table, foreign key, and primary key.
|
|
129
209
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
210
|
+
```ruby
|
|
211
|
+
User.simple_query
|
|
212
|
+
.select(:name, :email)
|
|
213
|
+
.join(:users, :companies, foreign_key: :user_id, primary_key: :id)
|
|
214
|
+
.where(Company.arel_table[:name].eq("TechCorp"))
|
|
215
|
+
.execute
|
|
133
216
|
```
|
|
134
217
|
|
|
135
|
-
|
|
218
|
+
Multiple joins can be chained:
|
|
136
219
|
|
|
137
220
|
```ruby
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
221
|
+
User.simple_query
|
|
222
|
+
.select(:name)
|
|
223
|
+
.join(:users, :companies, foreign_key: :user_id, primary_key: :id)
|
|
224
|
+
.join(:companies, :projects, foreign_key: :company_id, primary_key: :id)
|
|
225
|
+
.where(Company.arel_table[:industry].eq("Technology"))
|
|
226
|
+
.where(Project.arel_table[:status].eq("active"))
|
|
227
|
+
.execute
|
|
228
|
+
```
|
|
141
229
|
|
|
142
|
-
|
|
230
|
+
Join helpers are also available:
|
|
231
|
+
|
|
232
|
+
```ruby
|
|
143
233
|
User.simple_query
|
|
144
|
-
.
|
|
145
|
-
.
|
|
146
|
-
.group(:department)
|
|
234
|
+
.left_join(:users, :companies, foreign_key: :user_id, primary_key: :id)
|
|
235
|
+
.select(:name)
|
|
147
236
|
.execute
|
|
148
|
-
# => #<struct department="Engineering", group_concat_name="Alice, Bob, Charlie">
|
|
149
237
|
```
|
|
150
238
|
|
|
151
|
-
|
|
239
|
+
Supported join types:
|
|
240
|
+
|
|
241
|
+
- `join(..., type: :inner)` / `join(...)`
|
|
242
|
+
- `left_join(...)`
|
|
243
|
+
- `right_join(...)`
|
|
244
|
+
- `full_join(...)`
|
|
245
|
+
|
|
246
|
+
Database support for right and full outer joins depends on the adapter and database version.
|
|
247
|
+
|
|
248
|
+
## Selecting, ordering, and paging
|
|
249
|
+
|
|
250
|
+
```ruby
|
|
251
|
+
User.simple_query
|
|
252
|
+
.select(:id, :name, "LOWER(email) AS normalized_email")
|
|
253
|
+
.distinct
|
|
254
|
+
.where(active: true)
|
|
255
|
+
.order(name: :asc)
|
|
256
|
+
.limit(25)
|
|
257
|
+
.offset(50)
|
|
258
|
+
.execute
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
`select` accepts symbols, SQL strings, and Arel nodes. SQL strings are inserted as SQL fragments, so keep them trusted.
|
|
262
|
+
|
|
263
|
+
`order` accepts a hash of column names to directions, such as `order(created_at: :desc)`.
|
|
264
|
+
|
|
265
|
+
## Aggregations
|
|
266
|
+
|
|
267
|
+
SimpleQuery can build common aggregate expressions without hand-writing each SQL expression.
|
|
152
268
|
|
|
153
269
|
```ruby
|
|
154
|
-
# Get comprehensive statistics for a column
|
|
155
|
-
Company.simple_query.stats(:annual_revenue).execute
|
|
156
|
-
# => #<struct
|
|
157
|
-
# annual_revenue_count=100,
|
|
158
|
-
# annual_revenue_sum=50000000,
|
|
159
|
-
# annual_revenue_avg=500000,
|
|
160
|
-
# annual_revenue_min=100000,
|
|
161
|
-
# annual_revenue_max=2000000
|
|
162
|
-
# >
|
|
163
|
-
|
|
164
|
-
# Custom aggregations
|
|
165
270
|
Company.simple_query
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
271
|
+
.count
|
|
272
|
+
.sum(:annual_revenue)
|
|
273
|
+
.avg(:annual_revenue)
|
|
274
|
+
.min(:annual_revenue)
|
|
275
|
+
.max(:annual_revenue)
|
|
276
|
+
.execute
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
Custom aliases are supported:
|
|
280
|
+
|
|
281
|
+
```ruby
|
|
282
|
+
Company.simple_query
|
|
283
|
+
.count(:id, alias_name: "company_count")
|
|
284
|
+
.sum(:annual_revenue, alias_name: "total_revenue")
|
|
285
|
+
.execute
|
|
286
|
+
```
|
|
169
287
|
|
|
170
|
-
|
|
288
|
+
Grouped aggregations work with selected fields:
|
|
289
|
+
|
|
290
|
+
```ruby
|
|
171
291
|
Company.simple_query
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
# => [
|
|
178
|
-
# #<struct industry="Technology", count=50, sum_annual_revenue=25000000>,
|
|
179
|
-
# #<struct industry="Finance", count=30, sum_annual_revenue=20000000>
|
|
180
|
-
# ]
|
|
292
|
+
.select(:industry)
|
|
293
|
+
.count(alias_name: "company_count")
|
|
294
|
+
.sum(:annual_revenue, alias_name: "total_revenue")
|
|
295
|
+
.group(:industry)
|
|
296
|
+
.execute
|
|
181
297
|
```
|
|
182
298
|
|
|
183
|
-
|
|
299
|
+
Convenience helpers include:
|
|
300
|
+
|
|
301
|
+
```ruby
|
|
302
|
+
Company.simple_query.stats(:annual_revenue).execute
|
|
303
|
+
Company.simple_query.total_count.execute
|
|
304
|
+
User.simple_query.variance(:score).stddev(:score).execute
|
|
305
|
+
User.simple_query.group_concat(:name, separator: ", ").execute
|
|
306
|
+
```
|
|
184
307
|
|
|
185
|
-
|
|
308
|
+
For database-specific expressions, use `custom_aggregation` with trusted SQL:
|
|
186
309
|
|
|
187
310
|
```ruby
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
.execute
|
|
192
|
-
# => #<struct total_users=1000, total_score=85000>
|
|
311
|
+
Company.simple_query
|
|
312
|
+
.custom_aggregation("COUNT(DISTINCT industry)", "unique_industries")
|
|
313
|
+
.execute
|
|
193
314
|
```
|
|
194
315
|
|
|
195
|
-
## Custom
|
|
196
|
-
|
|
316
|
+
## Custom read models
|
|
317
|
+
|
|
318
|
+
By default, query results are returned as `Struct` objects. For named methods and domain-specific result objects, define a read model:
|
|
197
319
|
|
|
198
|
-
**Create a read model** inheriting from `SimpleQuery::ReadModel`:
|
|
199
320
|
```ruby
|
|
200
|
-
class
|
|
321
|
+
class UserSummary < SimpleQuery::ReadModel
|
|
201
322
|
attribute :identifier, column: :id
|
|
202
|
-
attribute :full_name,
|
|
323
|
+
attribute :full_name, column: :name
|
|
203
324
|
end
|
|
204
325
|
```
|
|
205
326
|
|
|
206
|
-
|
|
207
|
-
```ruby
|
|
208
|
-
results = User.simple_query
|
|
209
|
-
.select("users.id AS id", "users.name AS name")
|
|
210
|
-
.where(active: true)
|
|
211
|
-
.map_to(MyUserReadModel)
|
|
212
|
-
.execute
|
|
327
|
+
Map query results to the read model:
|
|
213
328
|
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
329
|
+
```ruby
|
|
330
|
+
users = User.simple_query
|
|
331
|
+
.select("users.id AS id", "users.name AS name")
|
|
332
|
+
.where(active: true)
|
|
333
|
+
.map_to(UserSummary)
|
|
334
|
+
.execute
|
|
335
|
+
|
|
336
|
+
users.first.identifier
|
|
337
|
+
users.first.full_name
|
|
218
338
|
```
|
|
219
|
-
This custom read model approach provides more clarity or domain-specific logic while still being faster than typical ActiveRecord instantiation.
|
|
220
339
|
|
|
221
|
-
|
|
222
|
-
|
|
340
|
+
The selected SQL aliases must match the read model's configured column names.
|
|
341
|
+
|
|
342
|
+
## Named scopes
|
|
343
|
+
|
|
344
|
+
Use `simple_scope` to define reusable query fragments on a model that includes SimpleQuery:
|
|
345
|
+
|
|
223
346
|
```ruby
|
|
224
347
|
class User < ActiveRecord::Base
|
|
225
348
|
include SimpleQuery
|
|
@@ -228,119 +351,239 @@ class User < ActiveRecord::Base
|
|
|
228
351
|
where(active: true)
|
|
229
352
|
end
|
|
230
353
|
|
|
231
|
-
simple_scope :admins
|
|
232
|
-
where(admin: true)
|
|
233
|
-
end
|
|
354
|
+
simple_scope :admins, -> { where(admin: true) }
|
|
234
355
|
|
|
235
|
-
# Block-based scope with parameter
|
|
236
356
|
simple_scope :by_name do |name|
|
|
237
357
|
where(name: name)
|
|
238
358
|
end
|
|
239
|
-
|
|
240
|
-
# Lambda-based scope with parameter
|
|
241
|
-
simple_scope :by_name, ->(name) { where(name: name) }
|
|
242
359
|
end
|
|
243
360
|
```
|
|
244
|
-
|
|
361
|
+
|
|
362
|
+
Scopes are evaluated in the context of the SimpleQuery builder and can be chained with the normal DSL:
|
|
245
363
|
|
|
246
364
|
```ruby
|
|
247
|
-
|
|
248
|
-
|
|
365
|
+
User.simple_query
|
|
366
|
+
.active
|
|
367
|
+
.admins
|
|
368
|
+
.by_name("Jane Doe")
|
|
369
|
+
.select(:id, :name)
|
|
370
|
+
.execute
|
|
371
|
+
```
|
|
249
372
|
|
|
250
|
-
|
|
251
|
-
results = User.simple_query.by_name("Jane Doe").execute
|
|
373
|
+
SimpleQuery validates positional scope arguments before invoking the scope body. If a scope is called with too few or too many positional arguments, it raises `ArgumentError` with the scope name and expected argument count. For scopes that declare Ruby keyword parameters, SimpleQuery forwards keywords to the scope body and lets Ruby raise its native keyword-argument errors.
|
|
252
374
|
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
.active
|
|
257
|
-
.select(:id, :name)
|
|
258
|
-
.order(name: :asc)
|
|
259
|
-
.execute
|
|
260
|
-
```
|
|
261
|
-
### How It Works
|
|
375
|
+
## Lazy execution and streaming
|
|
376
|
+
|
|
377
|
+
`lazy_execute` returns an `Enumerator` over the query results:
|
|
262
378
|
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
379
|
+
```ruby
|
|
380
|
+
enumerator = User.simple_query
|
|
381
|
+
.select(:name)
|
|
382
|
+
.where(active: true)
|
|
383
|
+
.lazy_execute
|
|
266
384
|
|
|
267
|
-
|
|
385
|
+
enumerator.each do |user|
|
|
386
|
+
puts user.name
|
|
387
|
+
end
|
|
388
|
+
```
|
|
268
389
|
|
|
269
|
-
|
|
390
|
+
`lazy_execute` is useful when you want enumerator-style consumption, but it still uses ActiveRecord's `select_all` internally.
|
|
270
391
|
|
|
271
|
-
|
|
272
|
-
- **MySQL**: Uses `mysql2` gem’s **streaming** (`stream: true, cache_rows: false, as: :hash`).
|
|
392
|
+
For large PostgreSQL or MySQL result sets, use `stream_each`:
|
|
273
393
|
|
|
274
394
|
```ruby
|
|
275
|
-
# Example usage:
|
|
276
395
|
User.simple_query
|
|
396
|
+
.select(:id, :email)
|
|
277
397
|
.where(active: true)
|
|
278
|
-
.stream_each(batch_size: 10_000) do |
|
|
279
|
-
|
|
280
|
-
puts row.name
|
|
398
|
+
.stream_each(batch_size: 10_000) do |user|
|
|
399
|
+
puts user.email
|
|
281
400
|
end
|
|
282
401
|
```
|
|
283
402
|
|
|
403
|
+
Adapter behavior:
|
|
404
|
+
|
|
405
|
+
- PostgreSQL: uses a server-side cursor with `DECLARE` / `FETCH`; `batch_size` must be a positive integer and controls the number of rows fetched per cursor read
|
|
406
|
+
- MySQL: uses `mysql2` streaming with `stream: true`, `cache_rows: false`, and `as: :hash`; `batch_size` is validated as a positive integer for API symmetry, but mysql2 streams rows directly and does not expose a SimpleQuery-controlled batch size
|
|
407
|
+
- Other adapters: `stream_each` raises `SimpleQuery::UnsupportedAdapterError` with the current adapter name
|
|
408
|
+
|
|
409
|
+
Failure behavior:
|
|
410
|
+
|
|
411
|
+
- `batch_size` must be a positive integer; invalid values raise `ArgumentError` before any adapter-specific SQL runs
|
|
412
|
+
- PostgreSQL streaming wraps the cursor in a transaction, closes a declared cursor before rollback when row processing fails, attempts cursor cleanup before rollback on fetch failures, and propagates the original error
|
|
413
|
+
- MySQL streaming uses the mysql2 driver's streaming result handling; query and row processing errors are propagated to the caller
|
|
414
|
+
|
|
415
|
+
## Bulk updates
|
|
416
|
+
|
|
417
|
+
`bulk_update` builds and executes a set-based `UPDATE` for the current query conditions:
|
|
418
|
+
|
|
419
|
+
```ruby
|
|
420
|
+
User.simple_query
|
|
421
|
+
.where(active: false)
|
|
422
|
+
.bulk_update(set: { status: 0 })
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
`bulk_update` sends SQL directly through the ActiveRecord connection and does not instantiate models or run ActiveRecord callbacks. If no `where` conditions are present, it updates the entire table. Joins, `limit`/`offset`, and `group`/`having` are not applied to the `UPDATE`, so `bulk_update` raises `ArgumentError` when any of them are present.
|
|
426
|
+
|
|
427
|
+
## Subqueries
|
|
428
|
+
|
|
429
|
+
You can use `build_query` to embed a SimpleQuery query as an Arel subquery:
|
|
430
|
+
|
|
431
|
+
```ruby
|
|
432
|
+
company_users = Company.simple_query
|
|
433
|
+
.select(:user_id)
|
|
434
|
+
.where(industry: "Technology")
|
|
435
|
+
.build_query
|
|
436
|
+
|
|
437
|
+
User.simple_query
|
|
438
|
+
.select(:name)
|
|
439
|
+
.where(User.arel_table[:id].in(company_users))
|
|
440
|
+
.execute
|
|
441
|
+
```
|
|
442
|
+
|
|
284
443
|
## Performance
|
|
285
444
|
|
|
286
|
-
SimpleQuery
|
|
445
|
+
SimpleQuery is designed for read paths where ActiveRecord model instantiation is unnecessary overhead. Returning structs or read models can reduce allocation costs for large reporting-style queries.
|
|
446
|
+
|
|
447
|
+
The repository includes an optional benchmark harness comparing ActiveRecord object loading and `update_all` with SimpleQuery structs, read models, and `bulk_update`. Treat benchmark numbers as workload-specific: validate them against your database, indexes, adapter, Ruby version, and query shape before making production claims.
|
|
448
|
+
|
|
449
|
+
Run the reproducible local benchmark with:
|
|
450
|
+
|
|
451
|
+
```sh
|
|
452
|
+
bundle exec rake benchmark:reproducible
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
The harness uses SQLite by default, seeds deterministic data, warms up and repeats each timing measurement, and prints JSON with environment metadata plus timing results. If `memory_profiler` is available in your development bundle, the JSON also includes single-run allocation profiles for the read-path scenarios. Tune the workload with environment variables:
|
|
287
456
|
|
|
288
|
-
|
|
457
|
+
```sh
|
|
458
|
+
BENCHMARK_ROWS=50000 BENCHMARK_WARMUP=3 BENCHMARK_RUNS=10 \
|
|
459
|
+
bundle exec rake benchmark:reproducible
|
|
289
460
|
```
|
|
290
|
-
🚀 Performance Results (1000,000 records):
|
|
291
|
-
ActiveRecord Query: 10.36932 seconds
|
|
292
|
-
SimpleQuery Execution (Struct): 3.46136 seconds
|
|
293
|
-
SimpleQuery Execution (Read model): 2.20905 seconds
|
|
294
461
|
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
462
|
+
Available variables:
|
|
463
|
+
|
|
464
|
+
- `BENCHMARK_ROWS` (default: `10000`) controls the number of users and matching companies seeded.
|
|
465
|
+
- `BENCHMARK_WARMUP` (default: `2`) controls warmup iterations before samples are collected.
|
|
466
|
+
- `BENCHMARK_RUNS` (default: `5`) controls the number of timed samples per scenario.
|
|
467
|
+
- `BENCHMARK_DATABASE` (default: `:memory:`) can point at a SQLite database file if you want to inspect the generated benchmark-specific tables. The harness drops and recreates `simple_query_benchmark_users` and `simple_query_benchmark_companies` in that database on every run.
|
|
468
|
+
|
|
469
|
+
The benchmark task writes JSON to standard output so it can be redirected into an artifact file. Keep progress messages or diagnostics on standard error if you extend the harness later.
|
|
298
470
|
|
|
299
|
-
|
|
300
|
-
Total allocated: 1.98 GB (16,001,659 objects)
|
|
301
|
-
Retained: ~2 KB
|
|
471
|
+
To preserve a run as a reviewable artifact, capture the JSON output to a file, validate that the artifact is parseable JSON, and keep the workload settings alongside it:
|
|
302
472
|
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
473
|
+
```sh
|
|
474
|
+
set -e
|
|
475
|
+
mkdir -p tmp/benchmarks
|
|
476
|
+
export BENCHMARK_ROWS=50000
|
|
477
|
+
export BENCHMARK_WARMUP=3
|
|
478
|
+
export BENCHMARK_RUNS=10
|
|
479
|
+
bundle exec rake benchmark:reproducible > tmp/benchmarks/simple_query-reproducible.json
|
|
480
|
+
ruby -rjson -e 'JSON.parse(File.read(ARGV.fetch(0)))' tmp/benchmarks/simple_query-reproducible.json
|
|
306
481
|
```
|
|
307
|
-
- **Struct-based** approach remains the fastest, skipping model overhead.
|
|
308
|
-
- **Read model** approach is still significantly faster than standard ActiveRecord while allowing domain-specific logic.
|
|
309
482
|
|
|
310
|
-
|
|
483
|
+
Before sharing or comparing a result, record enough environment metadata to recreate the run. The benchmark JSON includes runtime details, key dependency versions, and exported `BENCHMARK_*` values. When the task runs from a Git checkout, it also includes the current Git revision and dirty-checkout status; otherwise those Git fields are `null`. It is still useful to save companion hardware notes and the short Git status from the same shell session:
|
|
484
|
+
|
|
485
|
+
```sh
|
|
486
|
+
set -e
|
|
487
|
+
mkdir -p tmp/benchmarks
|
|
488
|
+
{
|
|
489
|
+
echo "git_revision=$(git describe --always --dirty)"
|
|
490
|
+
echo "git_status_short_start"
|
|
491
|
+
git status --short
|
|
492
|
+
echo "git_status_short_end"
|
|
493
|
+
echo "hardware=$(uname -a)"
|
|
494
|
+
} > tmp/benchmarks/simple_query-reproducible.metadata
|
|
311
495
|
```
|
|
312
|
-
🚀 Performance Results (1000,000 records):
|
|
313
|
-
ActiveRecord Query: 10.45833 seconds
|
|
314
|
-
SimpleQuery Execution (Struct): 3.04655 seconds
|
|
315
|
-
SimpleQuery Execution (Read model): 3.69052 seconds
|
|
316
496
|
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
497
|
+
Use captured benchmark output only as a reproducibility artifact. Do not compare runs from different hardware, databases, Ruby versions, dependency sets, dataset sizes, warmup counts, run counts, or benchmark databases as if they were equivalent. Prefer comparing changes by rerunning the same command on the same machine from a clean checkout of each revision, and treat non-empty `git status --short` metadata as a signal that the run may not be reproducible from the recorded revision alone. Scrub sensitive environment values before sharing artifacts if any `BENCHMARK_*` setting contains credentials, private paths, or hostnames.
|
|
498
|
+
|
|
499
|
+
## Safety notes
|
|
500
|
+
|
|
501
|
+
SimpleQuery is a query-building library, not an authorization or SQL-injection protection layer. It exposes both parameterized condition helpers and raw SQL escape hatches; choose the narrowest API that fits the value source.
|
|
502
|
+
|
|
503
|
+
Recommended usage:
|
|
320
504
|
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
505
|
+
- Prefer hash, Arel, or placeholder conditions for values derived from users or external systems.
|
|
506
|
+
- Treat raw SQL strings in `select`, `where`, and `custom_aggregation` as trusted-only escape hatches.
|
|
507
|
+
- Keep `group_concat` separators static and trusted; they are interpolated into database-specific SQL.
|
|
508
|
+
- `having` accepts the same condition styles as `where`; raw strings are trusted-only there too.
|
|
509
|
+
- Keep tenant, authorization, and visibility constraints explicit in your query code.
|
|
510
|
+
- Remember that `bulk_update` bypasses model callbacks and validations, like any direct SQL update.
|
|
324
511
|
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
512
|
+
### Parameterized values
|
|
513
|
+
|
|
514
|
+
Use hash conditions for simple equality predicates and array placeholder conditions for SQL fragments that include external values:
|
|
515
|
+
|
|
516
|
+
```ruby
|
|
517
|
+
# Equality predicates are built through Arel.
|
|
518
|
+
User.simple_query.where(email: params[:email]).execute
|
|
519
|
+
|
|
520
|
+
# SQL fragments with external values should use ActiveRecord-style placeholders.
|
|
521
|
+
query = params[:query].to_s
|
|
522
|
+
escaped_query = ActiveRecord::Base.sanitize_sql_like(query)
|
|
523
|
+
|
|
524
|
+
User.simple_query
|
|
525
|
+
.where(["name ILIKE ?", "%#{escaped_query}%"])
|
|
526
|
+
.execute
|
|
527
|
+
|
|
528
|
+
User.simple_query
|
|
529
|
+
.where(["email = :email", { email: params[:email] }])
|
|
530
|
+
.execute
|
|
328
531
|
```
|
|
329
|
-
|
|
330
|
-
|
|
532
|
+
|
|
533
|
+
Placeholders quote values as SQL data literals, while `sanitize_sql_like` makes `%` and `_` match literally in `LIKE` patterns; escaping the pattern does not replace placeholder quoting.
|
|
534
|
+
|
|
535
|
+
Do not interpolate external values into raw strings:
|
|
536
|
+
|
|
537
|
+
```ruby
|
|
538
|
+
# Unsafe: params[:query] is part of the SQL string itself.
|
|
539
|
+
User.simple_query.where("name ILIKE '%#{params[:query]}%'").execute
|
|
540
|
+
```
|
|
541
|
+
|
|
542
|
+
### Trusted SQL escape hatches
|
|
543
|
+
|
|
544
|
+
Some APIs intentionally accept SQL fragments because they are meant for expressions that Arel or SimpleQuery helpers do not model:
|
|
545
|
+
|
|
546
|
+
```ruby
|
|
547
|
+
# Trusted expression: fixed by the application, not assembled from request input.
|
|
548
|
+
User.simple_query.select("LOWER(email) AS normalized_email").execute
|
|
549
|
+
|
|
550
|
+
Company.simple_query
|
|
551
|
+
.custom_aggregation("COUNT(DISTINCT industry)", "unique_industries")
|
|
552
|
+
.execute
|
|
553
|
+
```
|
|
554
|
+
|
|
555
|
+
Keep aliases, custom aggregation expressions, raw `select` strings, raw `where` strings, `having` expressions, and `group_concat` separators static or otherwise trusted. If a condition must include external values, express those values with hash, Arel, or placeholder `where` syntax instead of interpolating them into raw SQL fragments.
|
|
556
|
+
|
|
557
|
+
### Direct updates
|
|
558
|
+
|
|
559
|
+
`bulk_update` is a set-based SQL update. It is useful for maintenance jobs and data workflows where callbacks are intentionally skipped, but it will not run ActiveRecord validations, callbacks, authorization checks, or per-record business logic. Always add the intended `where` conditions before calling it unless the whole table is the target.
|
|
331
560
|
|
|
332
561
|
## Development
|
|
333
562
|
|
|
334
|
-
After checking out the
|
|
563
|
+
After checking out the repository:
|
|
564
|
+
|
|
565
|
+
```bash
|
|
566
|
+
bin/setup
|
|
567
|
+
bundle exec rspec
|
|
568
|
+
bundle exec rubocop
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
Database-sensitive behavior is tested against PostgreSQL and MySQL in CI.
|
|
572
|
+
|
|
573
|
+
You can also open a console:
|
|
574
|
+
|
|
575
|
+
```bash
|
|
576
|
+
bin/console
|
|
577
|
+
```
|
|
335
578
|
|
|
336
579
|
## Contributing
|
|
337
580
|
|
|
338
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/kholdrex/simple_query.
|
|
581
|
+
Bug reports and pull requests are welcome on GitHub at [kholdrex/simple_query](https://github.com/kholdrex/simple_query). Contributors are expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|
|
339
582
|
|
|
340
583
|
## License
|
|
341
584
|
|
|
342
|
-
The gem is available as open source under the terms of the [MIT License](
|
|
585
|
+
The gem is available as open source under the terms of the [MIT License](LICENSE.txt).
|
|
343
586
|
|
|
344
587
|
## Code of Conduct
|
|
345
588
|
|
|
346
|
-
Everyone interacting in the SimpleQuery project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [
|
|
589
|
+
Everyone interacting in the SimpleQuery project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|