activerecord_cursor_paginate 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +5 -0
- data/LICENSE.txt +21 -0
- data/README.md +374 -0
- data/lib/activerecord_cursor_paginate/config.rb +11 -0
- data/lib/activerecord_cursor_paginate/cursor.rb +73 -0
- data/lib/activerecord_cursor_paginate/extension.rb +18 -0
- data/lib/activerecord_cursor_paginate/page.rb +85 -0
- data/lib/activerecord_cursor_paginate/paginator.rb +214 -0
- data/lib/activerecord_cursor_paginate/version.rb +5 -0
- data/lib/activerecord_cursor_paginate.rb +32 -0
- metadata +70 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 6a96b7bf3c40469653bf87dcf68cc159b30c389ed734047ef165b12d6ff7ef24
|
4
|
+
data.tar.gz: 2dea9c8b3847b5a72d6a915e9051e8b61891940e9feaa855edc822d49a393750
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: f54a87c366b7cfd08e59530b29fab0e0d5736cc323c4bab81ba0c3873ae5faa1254a2a9c94563e7aa9751debdb8802f28454a5781ddccf6811fe912c6808afca
|
7
|
+
data.tar.gz: c5689804eb34fa24e4e38f80994c9e3f16f45ab2594fe1fe9482397d2260b7d6db3453168a61fcb5f7e614ad97d8dfb6d8a1a42b4e560d780cebda759fd9f26d
|
data/CHANGELOG.md
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2024 fatkodima
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,374 @@
|
|
1
|
+
# ActiveRecordCursorPaginate
|
2
|
+
|
3
|
+
This library allows to paginate through an `ActiveRecord` relation using cursor pagination.
|
4
|
+
It also supports ordering by any column on the relation in either ascending or descending order.
|
5
|
+
|
6
|
+
Cursor pagination allows to paginate results and gracefully deal with deletions / additions on previous pages.
|
7
|
+
Where a regular limit / offset pagination would jump in results if a record on a previous page gets deleted or added while requesting the next page, cursor pagination just returns the records following the one identified in the request.
|
8
|
+
|
9
|
+
To learn more about cursor pagination, check out the _"How does it work?"_ section below.
|
10
|
+
|
11
|
+
[](https://github.com/fatkodima/activerecord_cursor_paginate/actions/workflows/test.yml)
|
12
|
+
|
13
|
+
## Requirements
|
14
|
+
|
15
|
+
- Ruby 2.7+
|
16
|
+
- Rails (ActiveRecord) 7.0+
|
17
|
+
|
18
|
+
If you need support for older ruby and rails, please open an issue.
|
19
|
+
|
20
|
+
## Installation
|
21
|
+
|
22
|
+
Add this line to your application's Gemfile:
|
23
|
+
|
24
|
+
```ruby
|
25
|
+
gem "activerecord_cursor_paginate"
|
26
|
+
```
|
27
|
+
|
28
|
+
And then run:
|
29
|
+
|
30
|
+
```sh
|
31
|
+
$ bundle install
|
32
|
+
```
|
33
|
+
|
34
|
+
## Usage
|
35
|
+
|
36
|
+
Let's assume we have a `Post` model of which we want to fetch some data and then paginate through it.
|
37
|
+
Therefore, we first apply our scopes, `where` clauses or other functionality as usual:
|
38
|
+
|
39
|
+
```ruby
|
40
|
+
posts = Post.where(author: "Jane")
|
41
|
+
```
|
42
|
+
|
43
|
+
And then we create our paginator to fetch the first response page:
|
44
|
+
|
45
|
+
```ruby
|
46
|
+
paginator = posts.cursor_paginate
|
47
|
+
page = paginator.fetch
|
48
|
+
page.records # => [#<Post:0x00007fd7071b2ea8 @id=1>, #<Post:0x00007fd7071bb738 @id=2>, ..., #<Post:0x00007fd707238260 @id=10>]
|
49
|
+
page.count # => 10
|
50
|
+
page.empty? # => false
|
51
|
+
page.cursors # => ["MQ==", "Mg==", ..., "MTA="]
|
52
|
+
| |
|
53
|
+
| |
|
54
|
+
page.previous_cursor # => "MQ==" |
|
55
|
+
page.next_cursor # => "MTA=" ------------------|
|
56
|
+
page.has_previous? # => false
|
57
|
+
page.has_next? # => true
|
58
|
+
```
|
59
|
+
|
60
|
+
Note that any ordering of the relation at this stage will be ignored by the gem.
|
61
|
+
Take a look at the next section _"Ordering"_ to see how you can have an order different than ascending IDs.
|
62
|
+
|
63
|
+
To then get the next result page, you simply need to pass the last cursor of the returned page item via:
|
64
|
+
|
65
|
+
```ruby
|
66
|
+
paginator = posts.cursor_paginate(after: "MTA=")
|
67
|
+
```
|
68
|
+
|
69
|
+
This will then fetch the next result page.
|
70
|
+
You can also just as easily paginate to previous pages by using `before` instead of `after` and using the first cursor of the current page.
|
71
|
+
|
72
|
+
```ruby
|
73
|
+
paginator = posts.cursor_paginate(before: "MQ==")
|
74
|
+
```
|
75
|
+
|
76
|
+
By default, this will always return up to 10 results. But you can also specify how many records should be returned via `limit` parameter.
|
77
|
+
|
78
|
+
```ruby
|
79
|
+
paginator = posts.cursor_paginate(after: "MTA=", limit: 2)
|
80
|
+
```
|
81
|
+
|
82
|
+
```ruby
|
83
|
+
paginator = posts.cursor_paginate(before: "MQ==", limit: 2)
|
84
|
+
```
|
85
|
+
|
86
|
+
You can also easily iterate over the whole relation:
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
paginator = posts.cursor_paginate
|
90
|
+
|
91
|
+
# Will lazily iterate over the pages.
|
92
|
+
paginator.pages.each do |page|
|
93
|
+
# do something with the page
|
94
|
+
end
|
95
|
+
```
|
96
|
+
|
97
|
+
### Ordering
|
98
|
+
|
99
|
+
As said, this gem ignores any previous ordering added to the passed relation.
|
100
|
+
But you can still paginate through relations with an order different than by ascending IDs.
|
101
|
+
|
102
|
+
You can specify a different column and direction to order the results by via an `order` parameter.
|
103
|
+
|
104
|
+
```ruby
|
105
|
+
# Order records ascending by the `:author` field.
|
106
|
+
paginator = posts.cursor_paginate(order: :author)
|
107
|
+
|
108
|
+
# Order records descending by the `:author` field.
|
109
|
+
paginator = posts.cursor_paginate(order: { author: :desc })
|
110
|
+
|
111
|
+
# Order records ascending by the `:author` and `:title` fields.
|
112
|
+
paginator = posts.cursor_paginate(order: [:author, :title])
|
113
|
+
|
114
|
+
# Order records ascending by the `:author` and descending by the `:title` fields.
|
115
|
+
paginator = posts.cursor_paginate(order: { author: :asc, title: :desc })
|
116
|
+
```
|
117
|
+
|
118
|
+
**Important:**
|
119
|
+
If your app regularly orders by another column, you might want to add a database index for this.
|
120
|
+
Say that your order column is `author` then you'll want to add a compound index on `(author, id)`.
|
121
|
+
If your table is called `posts` you can use a query like this in MySQL or PostgreSQL:
|
122
|
+
|
123
|
+
```sql
|
124
|
+
CREATE INDEX index_posts_on_author_and_id ON posts (author, id)
|
125
|
+
```
|
126
|
+
|
127
|
+
Or you can just do it via an `ActiveRecord::Migration`:
|
128
|
+
|
129
|
+
```ruby
|
130
|
+
class AddAuthorAndIdIndexToPosts < ActiveRecord::Migration[7.1]
|
131
|
+
def change
|
132
|
+
add_index :posts, [:author, :id]
|
133
|
+
end
|
134
|
+
end
|
135
|
+
```
|
136
|
+
|
137
|
+
Take a look at the _"How does it work?"_ to find out more why this is necessary.
|
138
|
+
|
139
|
+
#### Ordering and `JOIN`s
|
140
|
+
|
141
|
+
To order by a column from the `JOIN`ed table, you need to explicitly specify and fully qualify the column name for the `:order` parameter:
|
142
|
+
|
143
|
+
```ruby
|
144
|
+
paginator = User.joins(:projects).cursor_paginate(order: [Arel.sql("projects.id"), :id])
|
145
|
+
page = paginator.fetch
|
146
|
+
```
|
147
|
+
|
148
|
+
**Note**: Make sure to wrap custom SQL expressions by `Arel.sql`.
|
149
|
+
|
150
|
+
#### Order by more complex logic
|
151
|
+
|
152
|
+
Sometimes you might not only want to oder by a column ascending or descending, but need more complex logic.
|
153
|
+
Imagine you would also store the post's `category` on the `posts` table (as a plain string for simplicity's sake).
|
154
|
+
And the category could be `pinned`, `announcement`, or `general`.
|
155
|
+
Then you might want to show all `pinned` posts first, followed by the `announcement` ones and lastly show the `general` posts.
|
156
|
+
|
157
|
+
In MySQL you could e.g. use a `FIELD(category, 'pinned', 'announcement', 'general')` query in the `ORDER BY` clause to achieve this.
|
158
|
+
However, you cannot add an index to such a statement. And therefore, the performance of this is pretty dismal.
|
159
|
+
|
160
|
+
The gem supports ordering by custom SQL expressions, but make sure the performance will not suffer.
|
161
|
+
|
162
|
+
What is recommended if you _do_ need to order by more complex logic is to have a separate column that you only use for ordering.
|
163
|
+
You can use `ActiveRecord` hooks to automatically update this column whenever you change your data.
|
164
|
+
Or, for example in MySQL, you can also use a [generated column](https://dev.mysql.com/doc/refman/5.7/en/create-table-generated-columns.html) that is automatically being updated by the database based on some stored logic.
|
165
|
+
|
166
|
+
For example, if you want paginate `users` by a lowercased `email`, you can use the following:
|
167
|
+
|
168
|
+
```ruby
|
169
|
+
paginator = User.cursor_paginate(order: Arel.sql("lower(email)"))
|
170
|
+
page = paginator.fetch
|
171
|
+
```
|
172
|
+
|
173
|
+
**Note**: Make sure to wrap custom SQL expressions by `Arel.sql`.
|
174
|
+
|
175
|
+
### Configuration
|
176
|
+
|
177
|
+
You can change the default page size to a value that better fits the needs of your application.
|
178
|
+
So if a user doesn't request a given `:limit` value, the default amount of records is being returned.
|
179
|
+
|
180
|
+
To change the default, simply add an initializer to your app that does the following:
|
181
|
+
|
182
|
+
```ruby
|
183
|
+
# config/initializers/activerecord_cursor_paginate.rb
|
184
|
+
ActiveRecordCursorPaginate.configure do |config|
|
185
|
+
config.default_page_size = 50
|
186
|
+
end
|
187
|
+
```
|
188
|
+
|
189
|
+
This would set the default page size to 50.
|
190
|
+
|
191
|
+
You can also set a global `max_page_size` to prevent a client from requesting too large pages.
|
192
|
+
|
193
|
+
```ruby
|
194
|
+
ActiveRecordCursorPaginate.configure do |config|
|
195
|
+
config.max_page_size = 100
|
196
|
+
end
|
197
|
+
```
|
198
|
+
|
199
|
+
## How does it work?
|
200
|
+
|
201
|
+
This library allows to paginate through a passed relation using a cursor
|
202
|
+
for before or after parameters. It also supports ordering by any column
|
203
|
+
on the relation in either ascending or descending order.
|
204
|
+
|
205
|
+
Cursor pagination allows to paginate results and gracefully deal with
|
206
|
+
deletions / additions on previous pages. Where a regular limit / offset
|
207
|
+
pagination would jump in results if a record on a previous page gets deleted
|
208
|
+
or added while requesting the next page, cursor pagination just returns the
|
209
|
+
records following the one identified in the request.
|
210
|
+
|
211
|
+
How this works is that it uses a "cursor", which is an encoded value that
|
212
|
+
uniquely identifies a given row for the requested order. Then, based on
|
213
|
+
this cursor, you can request the "n records AFTER the cursor"
|
214
|
+
(forward-pagination) or the "n records BEFORE the cursor" (backward-pagination).
|
215
|
+
|
216
|
+
As an example, assume we have a table called "posts" with this data:
|
217
|
+
|
218
|
+
| id | author |
|
219
|
+
|----|--------|
|
220
|
+
| 1 | Jane |
|
221
|
+
| 2 | John |
|
222
|
+
| 3 | John |
|
223
|
+
| 4 | Jane |
|
224
|
+
| 5 | Jane |
|
225
|
+
| 6 | John |
|
226
|
+
| 7 | John |
|
227
|
+
|
228
|
+
Now if we make a basic request without any `before`, `after`, custom `order` column,
|
229
|
+
this will just request the first page of this relation.
|
230
|
+
|
231
|
+
```ruby
|
232
|
+
paginator = relation.cursor_paginate
|
233
|
+
page = paginator.fetch
|
234
|
+
```
|
235
|
+
|
236
|
+
Assume that our default page size here is 2 and we would get a query like this:
|
237
|
+
|
238
|
+
```sql
|
239
|
+
SELECT *
|
240
|
+
FROM "posts"
|
241
|
+
ORDER BY "posts"."id" ASC
|
242
|
+
LIMIT 2
|
243
|
+
```
|
244
|
+
|
245
|
+
This will return the first page of results, containing post #1 and #2. Since
|
246
|
+
no custom order is defined, each item in the returned collection will have a
|
247
|
+
cursor that only encodes the record's ID.
|
248
|
+
|
249
|
+
If we want to now request the next page, we can pass in the cursor of record
|
250
|
+
#2 which would be `"Mg=="` (can get via `page.cursor`). So now we can request
|
251
|
+
the next page by calling:
|
252
|
+
|
253
|
+
```ruby
|
254
|
+
paginator = relation.cursor_paginate(limit: 2, after: "Mg==")
|
255
|
+
page = paginator.fetch
|
256
|
+
```
|
257
|
+
|
258
|
+
And this will decode the given cursor and issue a query like:
|
259
|
+
|
260
|
+
```sql
|
261
|
+
SELECT *
|
262
|
+
FROM "posts"
|
263
|
+
WHERE "posts"."id" > 2
|
264
|
+
ORDER BY "posts"."id" ASC
|
265
|
+
LIMIT 2
|
266
|
+
```
|
267
|
+
|
268
|
+
Which would return posts #3 and #4. If we now want to paginate back, we can
|
269
|
+
request the posts that came before the first post, whose cursor would be
|
270
|
+
`"Mw=="` (can get via `page.previous_cursor`):
|
271
|
+
|
272
|
+
```ruby
|
273
|
+
paginator = relation.cursor_paginate(limit: 2, before: "Mw==")
|
274
|
+
page = paginator.fetch
|
275
|
+
```
|
276
|
+
|
277
|
+
Since we now paginate backward, the resulting SQL query needs to be flipped
|
278
|
+
around to get the last two records that have an ID smaller than the given one:
|
279
|
+
|
280
|
+
```sql
|
281
|
+
SELECT *
|
282
|
+
FROM "posts"
|
283
|
+
WHERE "posts"."id" < 3
|
284
|
+
ORDER BY "posts"."id" DESC
|
285
|
+
LIMIT 2
|
286
|
+
```
|
287
|
+
|
288
|
+
This would return posts #2 and #1. Since we still requested them in
|
289
|
+
ascending order, the result will be reversed before it is returned.
|
290
|
+
|
291
|
+
Now, in case that the user wants to order by a column different than the ID,
|
292
|
+
we require this information in our cursor. Therefore, when requesting the
|
293
|
+
first page like this:
|
294
|
+
|
295
|
+
```ruby
|
296
|
+
paginator = relation.cursor_paginate(order: :author)
|
297
|
+
page = paginator.fetch
|
298
|
+
```
|
299
|
+
|
300
|
+
This will issue the following SQL query:
|
301
|
+
|
302
|
+
```sql
|
303
|
+
SELECT *
|
304
|
+
FROM "posts"
|
305
|
+
ORDER BY "posts"."author" ASC, "posts"."id" ASC
|
306
|
+
LIMIT 2
|
307
|
+
```
|
308
|
+
|
309
|
+
As you can see, this will now order by the author first, and if two records
|
310
|
+
have the same author it will order them by ID. Ordering only the author is not
|
311
|
+
enough since we cannot know if the custom column only has unique values.
|
312
|
+
And we need to guarantee the correct order of ambiguous records independent
|
313
|
+
of the direction of ordering. This unique order is the basis of being able
|
314
|
+
to paginate forward and backward repeatedly and getting the correct records.
|
315
|
+
|
316
|
+
The query will then return records #1 and #4. But the cursor for these
|
317
|
+
records will also be different to the previous query where we ordered by ID
|
318
|
+
only. It is important that the cursor encodes all the data we need to
|
319
|
+
uniquely identify a row and filter based upon it. Therefore, we need to
|
320
|
+
encode the same information as we used for the ordering in our SQL query.
|
321
|
+
Hence, the cursor for pagination with a custom column contains a tuple of
|
322
|
+
data, the first record being the custom order column followed by the
|
323
|
+
record's ID.
|
324
|
+
|
325
|
+
Therefore, the cursor of record #4 will encode `['Jane', 4]`, which yields
|
326
|
+
this cursor: `"WyJKYW5lIiw0XQ=="`.
|
327
|
+
|
328
|
+
If we now want to request the next page via:
|
329
|
+
|
330
|
+
```ruby
|
331
|
+
paginator = relation.cursor_paginate(order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
|
332
|
+
page = paginator.fetch
|
333
|
+
```
|
334
|
+
|
335
|
+
We get this SQL query:
|
336
|
+
|
337
|
+
```sql
|
338
|
+
SELECT *
|
339
|
+
FROM "posts"
|
340
|
+
WHERE (author > 'Jane' OR (author = 'Jane') AND ("posts"."id" > 4))
|
341
|
+
ORDER BY "posts"."author" ASC, "posts"."id" ASC
|
342
|
+
LIMIT 2
|
343
|
+
```
|
344
|
+
|
345
|
+
You can see how the cursor is being used by the WHERE clause to uniquely
|
346
|
+
identify the row and properly filter based on this. We only want to get
|
347
|
+
records that either have a name that is alphabetically after `"Jane"` or
|
348
|
+
another `"Jane"` record with an ID that is higher than `4`. We will get the
|
349
|
+
records #5 and #2 as response.
|
350
|
+
|
351
|
+
When using a custom `order`, this affects both filtering as well as
|
352
|
+
ordering. Therefore, it is recommended to add an index for columns that are
|
353
|
+
frequently used for ordering. In our test case we would want to add a compound
|
354
|
+
index for the `(author, id)` column combination. Databases like MySQL and
|
355
|
+
PostgreSQL are able to then use the leftmost part of the index, in our case
|
356
|
+
`author`, by its own or can use it combined with the `id` index.
|
357
|
+
|
358
|
+
## Credits
|
359
|
+
|
360
|
+
Thanks to [rails_cursor_pagination gem](https://github.com/xing/rails_cursor_pagination) for the original ideas.
|
361
|
+
|
362
|
+
## Development
|
363
|
+
|
364
|
+
After checking out the repo, run `bundle install` to install dependencies. Then, run `bundle exec rake` to run the linter and tests. This project uses multiple Gemfiles to test against multiple versions of Active Record; you can run the tests against the specific version with `BUNDLE_GEMFILE=gemfiles/activerecord_71.gemfile bundle exec rake test`.
|
365
|
+
|
366
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
367
|
+
|
368
|
+
## Contributing
|
369
|
+
|
370
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/activerecord_cursor_paginate.
|
371
|
+
|
372
|
+
## License
|
373
|
+
|
374
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
@@ -0,0 +1,73 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "base64"
|
4
|
+
require "json"
|
5
|
+
|
6
|
+
module ActiveRecordCursorPaginate
|
7
|
+
# @private
|
8
|
+
class Cursor
|
9
|
+
class << self
|
10
|
+
def from_record(record, columns:)
|
11
|
+
columns = columns.map { |column| column.to_s.split(".").last }
|
12
|
+
values = columns.map { |column| record[column] }
|
13
|
+
new(columns: columns, values: values)
|
14
|
+
end
|
15
|
+
|
16
|
+
def decode(cursor_string:, columns:)
|
17
|
+
decoded = JSON.parse(Base64.strict_decode64(cursor_string))
|
18
|
+
|
19
|
+
if (columns.size == 1 && decoded.is_a?(Array)) ||
|
20
|
+
(decoded.is_a?(Array) && decoded.size != columns.size)
|
21
|
+
raise InvalidCursorError,
|
22
|
+
"The given cursor `#{cursor_string}` was decoded as `#{decoded}` but could not be parsed"
|
23
|
+
end
|
24
|
+
|
25
|
+
decoded =
|
26
|
+
if decoded.is_a?(Array)
|
27
|
+
decoded.map { |value| deserialize_time_if_needed(value) }
|
28
|
+
else
|
29
|
+
deserialize_time_if_needed(decoded)
|
30
|
+
end
|
31
|
+
|
32
|
+
new(columns: columns, values: decoded)
|
33
|
+
rescue ArgumentError, JSON::ParserError # ArgumentError is raised by strict_decode64
|
34
|
+
raise InvalidCursorError, "The given cursor `#{cursor_string}` could not be decoded"
|
35
|
+
end
|
36
|
+
|
37
|
+
private
|
38
|
+
def deserialize_time_if_needed(value)
|
39
|
+
if value.is_a?(String) && value.start_with?(TIMESTAMP_PREFIX)
|
40
|
+
seconds_with_frac = value.delete_prefix(TIMESTAMP_PREFIX).to_r / (10**6)
|
41
|
+
Time.at(seconds_with_frac).utc
|
42
|
+
else
|
43
|
+
value
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
attr_reader :columns, :values
|
49
|
+
|
50
|
+
def initialize(columns:, values:)
|
51
|
+
@columns = Array(columns)
|
52
|
+
@values = Array(values)
|
53
|
+
|
54
|
+
raise ArgumentError, "Cursor values can not be nil" if @values.any?(nil)
|
55
|
+
raise ArgumentError, ":columns and :values have different sizes" if @columns.size != @values.size
|
56
|
+
end
|
57
|
+
|
58
|
+
def encode
|
59
|
+
serialized_values = values.map do |value|
|
60
|
+
if value.is_a?(Time)
|
61
|
+
TIMESTAMP_PREFIX + value.strftime("%s%6N")
|
62
|
+
else
|
63
|
+
value
|
64
|
+
end
|
65
|
+
end
|
66
|
+
unencoded_cursor = (serialized_values.size == 1 ? serialized_values.first : serialized_values)
|
67
|
+
Base64.strict_encode64(unencoded_cursor.to_json)
|
68
|
+
end
|
69
|
+
|
70
|
+
TIMESTAMP_PREFIX = "0aIX2_" # something random
|
71
|
+
private_constant :TIMESTAMP_PREFIX
|
72
|
+
end
|
73
|
+
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ActiveRecordCursorPaginate
|
4
|
+
module Extension
|
5
|
+
# Convenient method to use on ActiveRecord::Relation to get a paginator.
|
6
|
+
# @return [ActiveRecordCursorPaginate::Paginator]
|
7
|
+
#
|
8
|
+
# @example
|
9
|
+
# paginator = Post.all.cursor_paginate(limit: 2, after: "Mg==")
|
10
|
+
# page = paginator.fetch
|
11
|
+
#
|
12
|
+
def cursor_paginate(after: nil, before: nil, limit: nil, order: nil)
|
13
|
+
relation = (is_a?(ActiveRecord::Relation) ? self : all)
|
14
|
+
Paginator.new(relation, after: after, before: before, limit: limit, order: order)
|
15
|
+
end
|
16
|
+
alias cursor_pagination cursor_paginate
|
17
|
+
end
|
18
|
+
end
|
@@ -0,0 +1,85 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ActiveRecordCursorPaginate
|
4
|
+
# Represents a batch of records retrieved via a single iteration of
|
5
|
+
# cursor-based pagination.
|
6
|
+
#
|
7
|
+
class Page
|
8
|
+
# Records this page contains.
|
9
|
+
# @return [ActiveRecord::Base]
|
10
|
+
#
|
11
|
+
attr_reader :records
|
12
|
+
|
13
|
+
def initialize(records, order_columns:, has_previous: false, has_next: false)
|
14
|
+
@records = records
|
15
|
+
@order_columns = order_columns
|
16
|
+
@has_previous = has_previous
|
17
|
+
@has_next = has_next
|
18
|
+
end
|
19
|
+
|
20
|
+
# Number of records in this page.
|
21
|
+
# @return [Integer]
|
22
|
+
#
|
23
|
+
def count
|
24
|
+
records.size
|
25
|
+
end
|
26
|
+
|
27
|
+
# Whether this page is empty.
|
28
|
+
# @return [Boolean]
|
29
|
+
#
|
30
|
+
def empty?
|
31
|
+
count == 0
|
32
|
+
end
|
33
|
+
|
34
|
+
# Returns the cursor, which can be used to retrieve the next page.
|
35
|
+
# @return [String]
|
36
|
+
#
|
37
|
+
def next_cursor
|
38
|
+
cursor_for_record(records.last)
|
39
|
+
end
|
40
|
+
alias cursor next_cursor
|
41
|
+
|
42
|
+
# Returns the cursor, which can be used to retrieve the previous page.
|
43
|
+
# @return [String]
|
44
|
+
#
|
45
|
+
def previous_cursor
|
46
|
+
cursor_for_record(records.first)
|
47
|
+
end
|
48
|
+
|
49
|
+
# Whether this page has a previous page.
|
50
|
+
# @return [Boolean]
|
51
|
+
#
|
52
|
+
def has_previous?
|
53
|
+
@has_previous
|
54
|
+
end
|
55
|
+
|
56
|
+
# Whether this page has a next page.
|
57
|
+
# @return [Boolean]
|
58
|
+
#
|
59
|
+
def has_next?
|
60
|
+
@has_next
|
61
|
+
end
|
62
|
+
|
63
|
+
# Returns cursor for a specific record.
|
64
|
+
#
|
65
|
+
# @param record [ActiveRecord::Base]
|
66
|
+
# @return [String]
|
67
|
+
#
|
68
|
+
def cursor_for(record)
|
69
|
+
cursor_for_record(record)
|
70
|
+
end
|
71
|
+
|
72
|
+
# Returns cursors for all the records on this page.
|
73
|
+
# @return [Array<String>]
|
74
|
+
#
|
75
|
+
def cursors
|
76
|
+
records.map { |record| cursor_for_record(record) }
|
77
|
+
end
|
78
|
+
|
79
|
+
private
|
80
|
+
def cursor_for_record(record)
|
81
|
+
cursor = Cursor.from_record(record, columns: @order_columns)
|
82
|
+
cursor.encode
|
83
|
+
end
|
84
|
+
end
|
85
|
+
end
|
@@ -0,0 +1,214 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ActiveRecordCursorPaginate
|
4
|
+
# Use this Paginator class to effortlessly paginate through ActiveRecord
|
5
|
+
# relations using cursor pagination.
|
6
|
+
#
|
7
|
+
# @example Iterating one page at a time
|
8
|
+
# ActiveRecordCursorPaginate::Paginator
|
9
|
+
# .new(relation, order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
|
10
|
+
# .fetch
|
11
|
+
#
|
12
|
+
# @example Iterating over the whole relation
|
13
|
+
# paginator = ActiveRecordCursorPaginate::Paginator
|
14
|
+
# .new(relation, order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
|
15
|
+
#
|
16
|
+
# # Will lazily iterate over the pages.
|
17
|
+
# paginator.pages.each do |page|
|
18
|
+
# # do something with the page
|
19
|
+
# end
|
20
|
+
#
|
21
|
+
class Paginator
|
22
|
+
# Create a new instance of the `ActiveRecordCursorPaginate::Paginator`
|
23
|
+
#
|
24
|
+
# @param relation [ActiveRecord::Relation] Relation that will be paginated.
|
25
|
+
# @param before [String, nil] Cursor to paginate upto (excluding).
|
26
|
+
# @param after [String, nil] Cursor to paginate forward from.
|
27
|
+
# @param limit [Integer, nil] Number of records to return in pagination.
|
28
|
+
# @param order [Symbol, String, nil, Array<Symbol, String>, Hash]
|
29
|
+
# Column(s) to order by, optionally with directions (either `:asc` or `:desc`,
|
30
|
+
# defaults to `:asc`). If none is provided, will default to ID column.
|
31
|
+
# NOTE: this will cause the query to filter on both the given column as
|
32
|
+
# well as the ID column. So you might want to add a compound index to your
|
33
|
+
# database similar to:
|
34
|
+
# ```sql
|
35
|
+
# CREATE INDEX <index_name> ON <table_name> (<order_fields>..., id)
|
36
|
+
# ```
|
37
|
+
# @raise [ArgumentError] If any parameter is not valid
|
38
|
+
#
|
39
|
+
def initialize(relation, before: nil, after: nil, limit: nil, order: nil)
|
40
|
+
unless relation.is_a?(ActiveRecord::Relation)
|
41
|
+
raise ArgumentError, "relation is not an ActiveRecord::Relation"
|
42
|
+
end
|
43
|
+
|
44
|
+
if before.present? && after.present?
|
45
|
+
raise ArgumentError, "Only one of :before and :after can be provided"
|
46
|
+
end
|
47
|
+
|
48
|
+
@relation = relation
|
49
|
+
@primary_key = @relation.primary_key
|
50
|
+
@cursor = before || after
|
51
|
+
@is_forward_pagination = before.blank?
|
52
|
+
|
53
|
+
config = ActiveRecordCursorPaginate.config
|
54
|
+
@page_size = limit || config.default_page_size
|
55
|
+
@page_size = [@page_size, config.max_page_size].min if config.max_page_size
|
56
|
+
|
57
|
+
order = normalize_order(order)
|
58
|
+
@columns = order.keys
|
59
|
+
@directions = order.values
|
60
|
+
end
|
61
|
+
|
62
|
+
# Get the paginated result.
|
63
|
+
# @return [ActiveRecordCursorPaginate::Page]
|
64
|
+
#
|
65
|
+
# @note Calling this method advances the paginator.
|
66
|
+
#
|
67
|
+
def fetch
|
68
|
+
relation = @relation
|
69
|
+
|
70
|
+
# Non trivial columns (expressions or joined tables columns).
|
71
|
+
if @columns.any?(/\W/)
|
72
|
+
arel_columns = @columns.map.with_index do |column, i|
|
73
|
+
arel_column(column).as("cursor_column_#{i + 1}")
|
74
|
+
end
|
75
|
+
cursor_column_names = 1.upto(@columns.size).map { |i| "cursor_column_#{i}" }
|
76
|
+
|
77
|
+
relation =
|
78
|
+
if relation.select_values.empty?
|
79
|
+
relation.select(Arel.star, arel_columns)
|
80
|
+
else
|
81
|
+
relation.select(arel_columns)
|
82
|
+
end
|
83
|
+
else
|
84
|
+
cursor_column_names = @columns
|
85
|
+
end
|
86
|
+
|
87
|
+
pagination_directions = @directions.map { |direction| pagination_direction(direction) }
|
88
|
+
relation = relation.reorder(cursor_column_names.zip(pagination_directions).to_h)
|
89
|
+
|
90
|
+
if @cursor
|
91
|
+
decoded_cursor = Cursor.decode(cursor_string: @cursor, columns: @columns)
|
92
|
+
relation = apply_cursor(relation, decoded_cursor)
|
93
|
+
end
|
94
|
+
|
95
|
+
relation = relation.limit(@page_size + 1)
|
96
|
+
records_plus_one = relation.to_a
|
97
|
+
has_additional = records_plus_one.size > @page_size
|
98
|
+
|
99
|
+
records = records_plus_one.take(@page_size)
|
100
|
+
records.reverse! unless @is_forward_pagination
|
101
|
+
|
102
|
+
if @is_forward_pagination
|
103
|
+
has_next_page = has_additional
|
104
|
+
has_previous_page = @cursor.present?
|
105
|
+
else
|
106
|
+
has_next_page = @cursor.present?
|
107
|
+
has_previous_page = has_additional
|
108
|
+
end
|
109
|
+
|
110
|
+
page = Page.new(
|
111
|
+
records,
|
112
|
+
order_columns: cursor_column_names,
|
113
|
+
has_next: has_next_page,
|
114
|
+
has_previous: has_previous_page
|
115
|
+
)
|
116
|
+
|
117
|
+
advance_by_page(page) unless page.empty?
|
118
|
+
|
119
|
+
page
|
120
|
+
end
|
121
|
+
alias page fetch
|
122
|
+
|
123
|
+
# Returns an enumerator that can be used to iterate over the whole relation.
|
124
|
+
# @return [Enumerator]
|
125
|
+
#
|
126
|
+
def pages
|
127
|
+
Enumerator.new do |yielder|
|
128
|
+
loop do
|
129
|
+
page = fetch
|
130
|
+
break if page.empty?
|
131
|
+
|
132
|
+
yielder.yield(page)
|
133
|
+
end
|
134
|
+
end
|
135
|
+
end
|
136
|
+
|
137
|
+
private
|
138
|
+
def normalize_order(order)
|
139
|
+
order ||= {}
|
140
|
+
default_direction = :asc
|
141
|
+
|
142
|
+
result =
|
143
|
+
case order
|
144
|
+
when String, Symbol
|
145
|
+
{ order => default_direction }
|
146
|
+
when Hash
|
147
|
+
order
|
148
|
+
when Array
|
149
|
+
order.to_h { |column| [column, default_direction] }
|
150
|
+
else
|
151
|
+
raise ArgumentError, "Invalid order: #{order.inspect}"
|
152
|
+
end
|
153
|
+
|
154
|
+
result = result.with_indifferent_access
|
155
|
+
result.transform_values! { |direction| direction.downcase.to_sym }
|
156
|
+
Array(@primary_key).each { |column| result[column] ||= default_direction }
|
157
|
+
result
|
158
|
+
end
|
159
|
+
|
160
|
+
def apply_cursor(relation, cursor)
|
161
|
+
operators = @directions.map { |direction| pagination_operator(direction) }
|
162
|
+
cursor_positions = cursor.columns.zip(cursor.values, operators)
|
163
|
+
|
164
|
+
where_clause = nil
|
165
|
+
cursor_positions.reverse_each.with_index do |(column, value, operator), index|
|
166
|
+
where_clause =
|
167
|
+
if index == 0
|
168
|
+
arel_column(column).public_send(operator, value)
|
169
|
+
else
|
170
|
+
arel_column(column).public_send(operator, value).or(
|
171
|
+
arel_column(column).eq(value).and(where_clause)
|
172
|
+
)
|
173
|
+
end
|
174
|
+
end
|
175
|
+
|
176
|
+
relation.where(where_clause)
|
177
|
+
end
|
178
|
+
|
179
|
+
def arel_column(column)
|
180
|
+
if Arel.arel_node?(column)
|
181
|
+
column
|
182
|
+
elsif column.match?(/\A\w+\.\w+\z/)
|
183
|
+
Arel.sql(column)
|
184
|
+
else
|
185
|
+
@relation.arel_table[column]
|
186
|
+
end
|
187
|
+
end
|
188
|
+
|
189
|
+
def pagination_direction(direction)
|
190
|
+
if @is_forward_pagination
|
191
|
+
direction
|
192
|
+
else
|
193
|
+
direction == :asc ? :desc : :asc
|
194
|
+
end
|
195
|
+
end
|
196
|
+
|
197
|
+
def pagination_operator(direction)
|
198
|
+
if @is_forward_pagination
|
199
|
+
direction == :asc ? :gt : :lt
|
200
|
+
else
|
201
|
+
direction == :asc ? :lt : :gt
|
202
|
+
end
|
203
|
+
end
|
204
|
+
|
205
|
+
def advance_by_page(page)
|
206
|
+
@cursor =
|
207
|
+
if @is_forward_pagination
|
208
|
+
page.next_cursor
|
209
|
+
else
|
210
|
+
page.previous_cursor
|
211
|
+
end
|
212
|
+
end
|
213
|
+
end
|
214
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "active_record"
|
4
|
+
|
5
|
+
require_relative "activerecord_cursor_paginate/version"
|
6
|
+
require_relative "activerecord_cursor_paginate/cursor"
|
7
|
+
require_relative "activerecord_cursor_paginate/page"
|
8
|
+
require_relative "activerecord_cursor_paginate/paginator"
|
9
|
+
require_relative "activerecord_cursor_paginate/extension"
|
10
|
+
require_relative "activerecord_cursor_paginate/config"
|
11
|
+
|
12
|
+
module ActiveRecordCursorPaginate
|
13
|
+
class Error < StandardError; end
|
14
|
+
|
15
|
+
# Error that gets raised if a cursor given as `before` or `after` cannot be
|
16
|
+
# properly parsed.
|
17
|
+
class InvalidCursorError < Error; end
|
18
|
+
|
19
|
+
class << self
|
20
|
+
def configure
|
21
|
+
yield config
|
22
|
+
end
|
23
|
+
|
24
|
+
def config
|
25
|
+
@config ||= Config.new
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
ActiveSupport.on_load(:active_record) do
|
31
|
+
extend ActiveRecordCursorPaginate::Extension
|
32
|
+
end
|
metadata
ADDED
@@ -0,0 +1,70 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: activerecord_cursor_paginate
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- fatkodima
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2024-03-08 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: activerecord
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '7.0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '7.0'
|
27
|
+
description:
|
28
|
+
email:
|
29
|
+
- fatkodima123@gmail.com
|
30
|
+
executables: []
|
31
|
+
extensions: []
|
32
|
+
extra_rdoc_files: []
|
33
|
+
files:
|
34
|
+
- CHANGELOG.md
|
35
|
+
- LICENSE.txt
|
36
|
+
- README.md
|
37
|
+
- lib/activerecord_cursor_paginate.rb
|
38
|
+
- lib/activerecord_cursor_paginate/config.rb
|
39
|
+
- lib/activerecord_cursor_paginate/cursor.rb
|
40
|
+
- lib/activerecord_cursor_paginate/extension.rb
|
41
|
+
- lib/activerecord_cursor_paginate/page.rb
|
42
|
+
- lib/activerecord_cursor_paginate/paginator.rb
|
43
|
+
- lib/activerecord_cursor_paginate/version.rb
|
44
|
+
homepage: https://github.com/fatkodima/activerecord_cursor_paginate
|
45
|
+
licenses:
|
46
|
+
- MIT
|
47
|
+
metadata:
|
48
|
+
homepage_uri: https://github.com/fatkodima/activerecord_cursor_paginate
|
49
|
+
source_code_uri: https://github.com/fatkodima/activerecord_cursor_paginate
|
50
|
+
changelog_uri: https://github.com/fatkodima/activerecord_cursor_paginate/blob/master/CHANGELOG.md
|
51
|
+
post_install_message:
|
52
|
+
rdoc_options: []
|
53
|
+
require_paths:
|
54
|
+
- lib
|
55
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
56
|
+
requirements:
|
57
|
+
- - ">="
|
58
|
+
- !ruby/object:Gem::Version
|
59
|
+
version: '2.7'
|
60
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
61
|
+
requirements:
|
62
|
+
- - ">="
|
63
|
+
- !ruby/object:Gem::Version
|
64
|
+
version: '0'
|
65
|
+
requirements: []
|
66
|
+
rubygems_version: 3.5.4
|
67
|
+
signing_key:
|
68
|
+
specification_version: 4
|
69
|
+
summary: Cursor-based pagination for ActiveRecord.
|
70
|
+
test_files: []
|