activerecord_cursor_paginate 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 6a96b7bf3c40469653bf87dcf68cc159b30c389ed734047ef165b12d6ff7ef24
4
+ data.tar.gz: 2dea9c8b3847b5a72d6a915e9051e8b61891940e9feaa855edc822d49a393750
5
+ SHA512:
6
+ metadata.gz: f54a87c366b7cfd08e59530b29fab0e0d5736cc323c4bab81ba0c3873ae5faa1254a2a9c94563e7aa9751debdb8802f28454a5781ddccf6811fe912c6808afca
7
+ data.tar.gz: c5689804eb34fa24e4e38f80994c9e3f16f45ab2594fe1fe9482397d2260b7d6db3453168a61fcb5f7e614ad97d8dfb6d8a1a42b4e560d780cebda759fd9f26d
data/CHANGELOG.md ADDED
@@ -0,0 +1,5 @@
1
+ ## master (unreleased)
2
+
3
+ ## 0.1.0 (2024-03-08)
4
+
5
+ - First release
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2024 fatkodima
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,374 @@
1
+ # ActiveRecordCursorPaginate
2
+
3
+ This library allows to paginate through an `ActiveRecord` relation using cursor pagination.
4
+ It also supports ordering by any column on the relation in either ascending or descending order.
5
+
6
+ Cursor pagination allows to paginate results and gracefully deal with deletions / additions on previous pages.
7
+ Where a regular limit / offset pagination would jump in results if a record on a previous page gets deleted or added while requesting the next page, cursor pagination just returns the records following the one identified in the request.
8
+
9
+ To learn more about cursor pagination, check out the _"How does it work?"_ section below.
10
+
11
+ [![Build Status](https://github.com/fatkodima/activerecord_cursor_paginate/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/fatkodima/activerecord_cursor_paginate/actions/workflows/test.yml)
12
+
13
+ ## Requirements
14
+
15
+ - Ruby 2.7+
16
+ - Rails (ActiveRecord) 7.0+
17
+
18
+ If you need support for older ruby and rails, please open an issue.
19
+
20
+ ## Installation
21
+
22
+ Add this line to your application's Gemfile:
23
+
24
+ ```ruby
25
+ gem "activerecord_cursor_paginate"
26
+ ```
27
+
28
+ And then run:
29
+
30
+ ```sh
31
+ $ bundle install
32
+ ```
33
+
34
+ ## Usage
35
+
36
+ Let's assume we have a `Post` model of which we want to fetch some data and then paginate through it.
37
+ Therefore, we first apply our scopes, `where` clauses or other functionality as usual:
38
+
39
+ ```ruby
40
+ posts = Post.where(author: "Jane")
41
+ ```
42
+
43
+ And then we create our paginator to fetch the first response page:
44
+
45
+ ```ruby
46
+ paginator = posts.cursor_paginate
47
+ page = paginator.fetch
48
+ page.records # => [#<Post:0x00007fd7071b2ea8 @id=1>, #<Post:0x00007fd7071bb738 @id=2>, ..., #<Post:0x00007fd707238260 @id=10>]
49
+ page.count # => 10
50
+ page.empty? # => false
51
+ page.cursors # => ["MQ==", "Mg==", ..., "MTA="]
52
+ | |
53
+ | |
54
+ page.previous_cursor # => "MQ==" |
55
+ page.next_cursor # => "MTA=" ------------------|
56
+ page.has_previous? # => false
57
+ page.has_next? # => true
58
+ ```
59
+
60
+ Note that any ordering of the relation at this stage will be ignored by the gem.
61
+ Take a look at the next section _"Ordering"_ to see how you can have an order different than ascending IDs.
62
+
63
+ To then get the next result page, you simply need to pass the last cursor of the returned page item via:
64
+
65
+ ```ruby
66
+ paginator = posts.cursor_paginate(after: "MTA=")
67
+ ```
68
+
69
+ This will then fetch the next result page.
70
+ You can also just as easily paginate to previous pages by using `before` instead of `after` and using the first cursor of the current page.
71
+
72
+ ```ruby
73
+ paginator = posts.cursor_paginate(before: "MQ==")
74
+ ```
75
+
76
+ By default, this will always return up to 10 results. But you can also specify how many records should be returned via `limit` parameter.
77
+
78
+ ```ruby
79
+ paginator = posts.cursor_paginate(after: "MTA=", limit: 2)
80
+ ```
81
+
82
+ ```ruby
83
+ paginator = posts.cursor_paginate(before: "MQ==", limit: 2)
84
+ ```
85
+
86
+ You can also easily iterate over the whole relation:
87
+
88
+ ```ruby
89
+ paginator = posts.cursor_paginate
90
+
91
+ # Will lazily iterate over the pages.
92
+ paginator.pages.each do |page|
93
+ # do something with the page
94
+ end
95
+ ```
96
+
97
+ ### Ordering
98
+
99
+ As said, this gem ignores any previous ordering added to the passed relation.
100
+ But you can still paginate through relations with an order different than by ascending IDs.
101
+
102
+ You can specify a different column and direction to order the results by via an `order` parameter.
103
+
104
+ ```ruby
105
+ # Order records ascending by the `:author` field.
106
+ paginator = posts.cursor_paginate(order: :author)
107
+
108
+ # Order records descending by the `:author` field.
109
+ paginator = posts.cursor_paginate(order: { author: :desc })
110
+
111
+ # Order records ascending by the `:author` and `:title` fields.
112
+ paginator = posts.cursor_paginate(order: [:author, :title])
113
+
114
+ # Order records ascending by the `:author` and descending by the `:title` fields.
115
+ paginator = posts.cursor_paginate(order: { author: :asc, title: :desc })
116
+ ```
117
+
118
+ **Important:**
119
+ If your app regularly orders by another column, you might want to add a database index for this.
120
+ Say that your order column is `author` then you'll want to add a compound index on `(author, id)`.
121
+ If your table is called `posts` you can use a query like this in MySQL or PostgreSQL:
122
+
123
+ ```sql
124
+ CREATE INDEX index_posts_on_author_and_id ON posts (author, id)
125
+ ```
126
+
127
+ Or you can just do it via an `ActiveRecord::Migration`:
128
+
129
+ ```ruby
130
+ class AddAuthorAndIdIndexToPosts < ActiveRecord::Migration[7.1]
131
+ def change
132
+ add_index :posts, [:author, :id]
133
+ end
134
+ end
135
+ ```
136
+
137
+ Take a look at the _"How does it work?"_ to find out more why this is necessary.
138
+
139
+ #### Ordering and `JOIN`s
140
+
141
+ To order by a column from the `JOIN`ed table, you need to explicitly specify and fully qualify the column name for the `:order` parameter:
142
+
143
+ ```ruby
144
+ paginator = User.joins(:projects).cursor_paginate(order: [Arel.sql("projects.id"), :id])
145
+ page = paginator.fetch
146
+ ```
147
+
148
+ **Note**: Make sure to wrap custom SQL expressions by `Arel.sql`.
149
+
150
+ #### Order by more complex logic
151
+
152
+ Sometimes you might not only want to oder by a column ascending or descending, but need more complex logic.
153
+ Imagine you would also store the post's `category` on the `posts` table (as a plain string for simplicity's sake).
154
+ And the category could be `pinned`, `announcement`, or `general`.
155
+ Then you might want to show all `pinned` posts first, followed by the `announcement` ones and lastly show the `general` posts.
156
+
157
+ In MySQL you could e.g. use a `FIELD(category, 'pinned', 'announcement', 'general')` query in the `ORDER BY` clause to achieve this.
158
+ However, you cannot add an index to such a statement. And therefore, the performance of this is pretty dismal.
159
+
160
+ The gem supports ordering by custom SQL expressions, but make sure the performance will not suffer.
161
+
162
+ What is recommended if you _do_ need to order by more complex logic is to have a separate column that you only use for ordering.
163
+ You can use `ActiveRecord` hooks to automatically update this column whenever you change your data.
164
+ Or, for example in MySQL, you can also use a [generated column](https://dev.mysql.com/doc/refman/5.7/en/create-table-generated-columns.html) that is automatically being updated by the database based on some stored logic.
165
+
166
+ For example, if you want paginate `users` by a lowercased `email`, you can use the following:
167
+
168
+ ```ruby
169
+ paginator = User.cursor_paginate(order: Arel.sql("lower(email)"))
170
+ page = paginator.fetch
171
+ ```
172
+
173
+ **Note**: Make sure to wrap custom SQL expressions by `Arel.sql`.
174
+
175
+ ### Configuration
176
+
177
+ You can change the default page size to a value that better fits the needs of your application.
178
+ So if a user doesn't request a given `:limit` value, the default amount of records is being returned.
179
+
180
+ To change the default, simply add an initializer to your app that does the following:
181
+
182
+ ```ruby
183
+ # config/initializers/activerecord_cursor_paginate.rb
184
+ ActiveRecordCursorPaginate.configure do |config|
185
+ config.default_page_size = 50
186
+ end
187
+ ```
188
+
189
+ This would set the default page size to 50.
190
+
191
+ You can also set a global `max_page_size` to prevent a client from requesting too large pages.
192
+
193
+ ```ruby
194
+ ActiveRecordCursorPaginate.configure do |config|
195
+ config.max_page_size = 100
196
+ end
197
+ ```
198
+
199
+ ## How does it work?
200
+
201
+ This library allows to paginate through a passed relation using a cursor
202
+ for before or after parameters. It also supports ordering by any column
203
+ on the relation in either ascending or descending order.
204
+
205
+ Cursor pagination allows to paginate results and gracefully deal with
206
+ deletions / additions on previous pages. Where a regular limit / offset
207
+ pagination would jump in results if a record on a previous page gets deleted
208
+ or added while requesting the next page, cursor pagination just returns the
209
+ records following the one identified in the request.
210
+
211
+ How this works is that it uses a "cursor", which is an encoded value that
212
+ uniquely identifies a given row for the requested order. Then, based on
213
+ this cursor, you can request the "n records AFTER the cursor"
214
+ (forward-pagination) or the "n records BEFORE the cursor" (backward-pagination).
215
+
216
+ As an example, assume we have a table called "posts" with this data:
217
+
218
+ | id | author |
219
+ |----|--------|
220
+ | 1 | Jane |
221
+ | 2 | John |
222
+ | 3 | John |
223
+ | 4 | Jane |
224
+ | 5 | Jane |
225
+ | 6 | John |
226
+ | 7 | John |
227
+
228
+ Now if we make a basic request without any `before`, `after`, custom `order` column,
229
+ this will just request the first page of this relation.
230
+
231
+ ```ruby
232
+ paginator = relation.cursor_paginate
233
+ page = paginator.fetch
234
+ ```
235
+
236
+ Assume that our default page size here is 2 and we would get a query like this:
237
+
238
+ ```sql
239
+ SELECT *
240
+ FROM "posts"
241
+ ORDER BY "posts"."id" ASC
242
+ LIMIT 2
243
+ ```
244
+
245
+ This will return the first page of results, containing post #1 and #2. Since
246
+ no custom order is defined, each item in the returned collection will have a
247
+ cursor that only encodes the record's ID.
248
+
249
+ If we want to now request the next page, we can pass in the cursor of record
250
+ #2 which would be `"Mg=="` (can get via `page.cursor`). So now we can request
251
+ the next page by calling:
252
+
253
+ ```ruby
254
+ paginator = relation.cursor_paginate(limit: 2, after: "Mg==")
255
+ page = paginator.fetch
256
+ ```
257
+
258
+ And this will decode the given cursor and issue a query like:
259
+
260
+ ```sql
261
+ SELECT *
262
+ FROM "posts"
263
+ WHERE "posts"."id" > 2
264
+ ORDER BY "posts"."id" ASC
265
+ LIMIT 2
266
+ ```
267
+
268
+ Which would return posts #3 and #4. If we now want to paginate back, we can
269
+ request the posts that came before the first post, whose cursor would be
270
+ `"Mw=="` (can get via `page.previous_cursor`):
271
+
272
+ ```ruby
273
+ paginator = relation.cursor_paginate(limit: 2, before: "Mw==")
274
+ page = paginator.fetch
275
+ ```
276
+
277
+ Since we now paginate backward, the resulting SQL query needs to be flipped
278
+ around to get the last two records that have an ID smaller than the given one:
279
+
280
+ ```sql
281
+ SELECT *
282
+ FROM "posts"
283
+ WHERE "posts"."id" < 3
284
+ ORDER BY "posts"."id" DESC
285
+ LIMIT 2
286
+ ```
287
+
288
+ This would return posts #2 and #1. Since we still requested them in
289
+ ascending order, the result will be reversed before it is returned.
290
+
291
+ Now, in case that the user wants to order by a column different than the ID,
292
+ we require this information in our cursor. Therefore, when requesting the
293
+ first page like this:
294
+
295
+ ```ruby
296
+ paginator = relation.cursor_paginate(order: :author)
297
+ page = paginator.fetch
298
+ ```
299
+
300
+ This will issue the following SQL query:
301
+
302
+ ```sql
303
+ SELECT *
304
+ FROM "posts"
305
+ ORDER BY "posts"."author" ASC, "posts"."id" ASC
306
+ LIMIT 2
307
+ ```
308
+
309
+ As you can see, this will now order by the author first, and if two records
310
+ have the same author it will order them by ID. Ordering only the author is not
311
+ enough since we cannot know if the custom column only has unique values.
312
+ And we need to guarantee the correct order of ambiguous records independent
313
+ of the direction of ordering. This unique order is the basis of being able
314
+ to paginate forward and backward repeatedly and getting the correct records.
315
+
316
+ The query will then return records #1 and #4. But the cursor for these
317
+ records will also be different to the previous query where we ordered by ID
318
+ only. It is important that the cursor encodes all the data we need to
319
+ uniquely identify a row and filter based upon it. Therefore, we need to
320
+ encode the same information as we used for the ordering in our SQL query.
321
+ Hence, the cursor for pagination with a custom column contains a tuple of
322
+ data, the first record being the custom order column followed by the
323
+ record's ID.
324
+
325
+ Therefore, the cursor of record #4 will encode `['Jane', 4]`, which yields
326
+ this cursor: `"WyJKYW5lIiw0XQ=="`.
327
+
328
+ If we now want to request the next page via:
329
+
330
+ ```ruby
331
+ paginator = relation.cursor_paginate(order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
332
+ page = paginator.fetch
333
+ ```
334
+
335
+ We get this SQL query:
336
+
337
+ ```sql
338
+ SELECT *
339
+ FROM "posts"
340
+ WHERE (author > 'Jane' OR (author = 'Jane') AND ("posts"."id" > 4))
341
+ ORDER BY "posts"."author" ASC, "posts"."id" ASC
342
+ LIMIT 2
343
+ ```
344
+
345
+ You can see how the cursor is being used by the WHERE clause to uniquely
346
+ identify the row and properly filter based on this. We only want to get
347
+ records that either have a name that is alphabetically after `"Jane"` or
348
+ another `"Jane"` record with an ID that is higher than `4`. We will get the
349
+ records #5 and #2 as response.
350
+
351
+ When using a custom `order`, this affects both filtering as well as
352
+ ordering. Therefore, it is recommended to add an index for columns that are
353
+ frequently used for ordering. In our test case we would want to add a compound
354
+ index for the `(author, id)` column combination. Databases like MySQL and
355
+ PostgreSQL are able to then use the leftmost part of the index, in our case
356
+ `author`, by its own or can use it combined with the `id` index.
357
+
358
+ ## Credits
359
+
360
+ Thanks to [rails_cursor_pagination gem](https://github.com/xing/rails_cursor_pagination) for the original ideas.
361
+
362
+ ## Development
363
+
364
+ After checking out the repo, run `bundle install` to install dependencies. Then, run `bundle exec rake` to run the linter and tests. This project uses multiple Gemfiles to test against multiple versions of Active Record; you can run the tests against the specific version with `BUNDLE_GEMFILE=gemfiles/activerecord_71.gemfile bundle exec rake test`.
365
+
366
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
367
+
368
+ ## Contributing
369
+
370
+ Bug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/activerecord_cursor_paginate.
371
+
372
+ ## License
373
+
374
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
@@ -0,0 +1,11 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ class Config
5
+ attr_accessor :default_page_size, :max_page_size
6
+
7
+ def initialize
8
+ @default_page_size = 10
9
+ end
10
+ end
11
+ end
@@ -0,0 +1,73 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "base64"
4
+ require "json"
5
+
6
+ module ActiveRecordCursorPaginate
7
+ # @private
8
+ class Cursor
9
+ class << self
10
+ def from_record(record, columns:)
11
+ columns = columns.map { |column| column.to_s.split(".").last }
12
+ values = columns.map { |column| record[column] }
13
+ new(columns: columns, values: values)
14
+ end
15
+
16
+ def decode(cursor_string:, columns:)
17
+ decoded = JSON.parse(Base64.strict_decode64(cursor_string))
18
+
19
+ if (columns.size == 1 && decoded.is_a?(Array)) ||
20
+ (decoded.is_a?(Array) && decoded.size != columns.size)
21
+ raise InvalidCursorError,
22
+ "The given cursor `#{cursor_string}` was decoded as `#{decoded}` but could not be parsed"
23
+ end
24
+
25
+ decoded =
26
+ if decoded.is_a?(Array)
27
+ decoded.map { |value| deserialize_time_if_needed(value) }
28
+ else
29
+ deserialize_time_if_needed(decoded)
30
+ end
31
+
32
+ new(columns: columns, values: decoded)
33
+ rescue ArgumentError, JSON::ParserError # ArgumentError is raised by strict_decode64
34
+ raise InvalidCursorError, "The given cursor `#{cursor_string}` could not be decoded"
35
+ end
36
+
37
+ private
38
+ def deserialize_time_if_needed(value)
39
+ if value.is_a?(String) && value.start_with?(TIMESTAMP_PREFIX)
40
+ seconds_with_frac = value.delete_prefix(TIMESTAMP_PREFIX).to_r / (10**6)
41
+ Time.at(seconds_with_frac).utc
42
+ else
43
+ value
44
+ end
45
+ end
46
+ end
47
+
48
+ attr_reader :columns, :values
49
+
50
+ def initialize(columns:, values:)
51
+ @columns = Array(columns)
52
+ @values = Array(values)
53
+
54
+ raise ArgumentError, "Cursor values can not be nil" if @values.any?(nil)
55
+ raise ArgumentError, ":columns and :values have different sizes" if @columns.size != @values.size
56
+ end
57
+
58
+ def encode
59
+ serialized_values = values.map do |value|
60
+ if value.is_a?(Time)
61
+ TIMESTAMP_PREFIX + value.strftime("%s%6N")
62
+ else
63
+ value
64
+ end
65
+ end
66
+ unencoded_cursor = (serialized_values.size == 1 ? serialized_values.first : serialized_values)
67
+ Base64.strict_encode64(unencoded_cursor.to_json)
68
+ end
69
+
70
+ TIMESTAMP_PREFIX = "0aIX2_" # something random
71
+ private_constant :TIMESTAMP_PREFIX
72
+ end
73
+ end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ module Extension
5
+ # Convenient method to use on ActiveRecord::Relation to get a paginator.
6
+ # @return [ActiveRecordCursorPaginate::Paginator]
7
+ #
8
+ # @example
9
+ # paginator = Post.all.cursor_paginate(limit: 2, after: "Mg==")
10
+ # page = paginator.fetch
11
+ #
12
+ def cursor_paginate(after: nil, before: nil, limit: nil, order: nil)
13
+ relation = (is_a?(ActiveRecord::Relation) ? self : all)
14
+ Paginator.new(relation, after: after, before: before, limit: limit, order: order)
15
+ end
16
+ alias cursor_pagination cursor_paginate
17
+ end
18
+ end
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ # Represents a batch of records retrieved via a single iteration of
5
+ # cursor-based pagination.
6
+ #
7
+ class Page
8
+ # Records this page contains.
9
+ # @return [ActiveRecord::Base]
10
+ #
11
+ attr_reader :records
12
+
13
+ def initialize(records, order_columns:, has_previous: false, has_next: false)
14
+ @records = records
15
+ @order_columns = order_columns
16
+ @has_previous = has_previous
17
+ @has_next = has_next
18
+ end
19
+
20
+ # Number of records in this page.
21
+ # @return [Integer]
22
+ #
23
+ def count
24
+ records.size
25
+ end
26
+
27
+ # Whether this page is empty.
28
+ # @return [Boolean]
29
+ #
30
+ def empty?
31
+ count == 0
32
+ end
33
+
34
+ # Returns the cursor, which can be used to retrieve the next page.
35
+ # @return [String]
36
+ #
37
+ def next_cursor
38
+ cursor_for_record(records.last)
39
+ end
40
+ alias cursor next_cursor
41
+
42
+ # Returns the cursor, which can be used to retrieve the previous page.
43
+ # @return [String]
44
+ #
45
+ def previous_cursor
46
+ cursor_for_record(records.first)
47
+ end
48
+
49
+ # Whether this page has a previous page.
50
+ # @return [Boolean]
51
+ #
52
+ def has_previous?
53
+ @has_previous
54
+ end
55
+
56
+ # Whether this page has a next page.
57
+ # @return [Boolean]
58
+ #
59
+ def has_next?
60
+ @has_next
61
+ end
62
+
63
+ # Returns cursor for a specific record.
64
+ #
65
+ # @param record [ActiveRecord::Base]
66
+ # @return [String]
67
+ #
68
+ def cursor_for(record)
69
+ cursor_for_record(record)
70
+ end
71
+
72
+ # Returns cursors for all the records on this page.
73
+ # @return [Array<String>]
74
+ #
75
+ def cursors
76
+ records.map { |record| cursor_for_record(record) }
77
+ end
78
+
79
+ private
80
+ def cursor_for_record(record)
81
+ cursor = Cursor.from_record(record, columns: @order_columns)
82
+ cursor.encode
83
+ end
84
+ end
85
+ end
@@ -0,0 +1,214 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ # Use this Paginator class to effortlessly paginate through ActiveRecord
5
+ # relations using cursor pagination.
6
+ #
7
+ # @example Iterating one page at a time
8
+ # ActiveRecordCursorPaginate::Paginator
9
+ # .new(relation, order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
10
+ # .fetch
11
+ #
12
+ # @example Iterating over the whole relation
13
+ # paginator = ActiveRecordCursorPaginate::Paginator
14
+ # .new(relation, order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
15
+ #
16
+ # # Will lazily iterate over the pages.
17
+ # paginator.pages.each do |page|
18
+ # # do something with the page
19
+ # end
20
+ #
21
+ class Paginator
22
+ # Create a new instance of the `ActiveRecordCursorPaginate::Paginator`
23
+ #
24
+ # @param relation [ActiveRecord::Relation] Relation that will be paginated.
25
+ # @param before [String, nil] Cursor to paginate upto (excluding).
26
+ # @param after [String, nil] Cursor to paginate forward from.
27
+ # @param limit [Integer, nil] Number of records to return in pagination.
28
+ # @param order [Symbol, String, nil, Array<Symbol, String>, Hash]
29
+ # Column(s) to order by, optionally with directions (either `:asc` or `:desc`,
30
+ # defaults to `:asc`). If none is provided, will default to ID column.
31
+ # NOTE: this will cause the query to filter on both the given column as
32
+ # well as the ID column. So you might want to add a compound index to your
33
+ # database similar to:
34
+ # ```sql
35
+ # CREATE INDEX <index_name> ON <table_name> (<order_fields>..., id)
36
+ # ```
37
+ # @raise [ArgumentError] If any parameter is not valid
38
+ #
39
+ def initialize(relation, before: nil, after: nil, limit: nil, order: nil)
40
+ unless relation.is_a?(ActiveRecord::Relation)
41
+ raise ArgumentError, "relation is not an ActiveRecord::Relation"
42
+ end
43
+
44
+ if before.present? && after.present?
45
+ raise ArgumentError, "Only one of :before and :after can be provided"
46
+ end
47
+
48
+ @relation = relation
49
+ @primary_key = @relation.primary_key
50
+ @cursor = before || after
51
+ @is_forward_pagination = before.blank?
52
+
53
+ config = ActiveRecordCursorPaginate.config
54
+ @page_size = limit || config.default_page_size
55
+ @page_size = [@page_size, config.max_page_size].min if config.max_page_size
56
+
57
+ order = normalize_order(order)
58
+ @columns = order.keys
59
+ @directions = order.values
60
+ end
61
+
62
+ # Get the paginated result.
63
+ # @return [ActiveRecordCursorPaginate::Page]
64
+ #
65
+ # @note Calling this method advances the paginator.
66
+ #
67
+ def fetch
68
+ relation = @relation
69
+
70
+ # Non trivial columns (expressions or joined tables columns).
71
+ if @columns.any?(/\W/)
72
+ arel_columns = @columns.map.with_index do |column, i|
73
+ arel_column(column).as("cursor_column_#{i + 1}")
74
+ end
75
+ cursor_column_names = 1.upto(@columns.size).map { |i| "cursor_column_#{i}" }
76
+
77
+ relation =
78
+ if relation.select_values.empty?
79
+ relation.select(Arel.star, arel_columns)
80
+ else
81
+ relation.select(arel_columns)
82
+ end
83
+ else
84
+ cursor_column_names = @columns
85
+ end
86
+
87
+ pagination_directions = @directions.map { |direction| pagination_direction(direction) }
88
+ relation = relation.reorder(cursor_column_names.zip(pagination_directions).to_h)
89
+
90
+ if @cursor
91
+ decoded_cursor = Cursor.decode(cursor_string: @cursor, columns: @columns)
92
+ relation = apply_cursor(relation, decoded_cursor)
93
+ end
94
+
95
+ relation = relation.limit(@page_size + 1)
96
+ records_plus_one = relation.to_a
97
+ has_additional = records_plus_one.size > @page_size
98
+
99
+ records = records_plus_one.take(@page_size)
100
+ records.reverse! unless @is_forward_pagination
101
+
102
+ if @is_forward_pagination
103
+ has_next_page = has_additional
104
+ has_previous_page = @cursor.present?
105
+ else
106
+ has_next_page = @cursor.present?
107
+ has_previous_page = has_additional
108
+ end
109
+
110
+ page = Page.new(
111
+ records,
112
+ order_columns: cursor_column_names,
113
+ has_next: has_next_page,
114
+ has_previous: has_previous_page
115
+ )
116
+
117
+ advance_by_page(page) unless page.empty?
118
+
119
+ page
120
+ end
121
+ alias page fetch
122
+
123
+ # Returns an enumerator that can be used to iterate over the whole relation.
124
+ # @return [Enumerator]
125
+ #
126
+ def pages
127
+ Enumerator.new do |yielder|
128
+ loop do
129
+ page = fetch
130
+ break if page.empty?
131
+
132
+ yielder.yield(page)
133
+ end
134
+ end
135
+ end
136
+
137
+ private
138
+ def normalize_order(order)
139
+ order ||= {}
140
+ default_direction = :asc
141
+
142
+ result =
143
+ case order
144
+ when String, Symbol
145
+ { order => default_direction }
146
+ when Hash
147
+ order
148
+ when Array
149
+ order.to_h { |column| [column, default_direction] }
150
+ else
151
+ raise ArgumentError, "Invalid order: #{order.inspect}"
152
+ end
153
+
154
+ result = result.with_indifferent_access
155
+ result.transform_values! { |direction| direction.downcase.to_sym }
156
+ Array(@primary_key).each { |column| result[column] ||= default_direction }
157
+ result
158
+ end
159
+
160
+ def apply_cursor(relation, cursor)
161
+ operators = @directions.map { |direction| pagination_operator(direction) }
162
+ cursor_positions = cursor.columns.zip(cursor.values, operators)
163
+
164
+ where_clause = nil
165
+ cursor_positions.reverse_each.with_index do |(column, value, operator), index|
166
+ where_clause =
167
+ if index == 0
168
+ arel_column(column).public_send(operator, value)
169
+ else
170
+ arel_column(column).public_send(operator, value).or(
171
+ arel_column(column).eq(value).and(where_clause)
172
+ )
173
+ end
174
+ end
175
+
176
+ relation.where(where_clause)
177
+ end
178
+
179
+ def arel_column(column)
180
+ if Arel.arel_node?(column)
181
+ column
182
+ elsif column.match?(/\A\w+\.\w+\z/)
183
+ Arel.sql(column)
184
+ else
185
+ @relation.arel_table[column]
186
+ end
187
+ end
188
+
189
+ def pagination_direction(direction)
190
+ if @is_forward_pagination
191
+ direction
192
+ else
193
+ direction == :asc ? :desc : :asc
194
+ end
195
+ end
196
+
197
+ def pagination_operator(direction)
198
+ if @is_forward_pagination
199
+ direction == :asc ? :gt : :lt
200
+ else
201
+ direction == :asc ? :lt : :gt
202
+ end
203
+ end
204
+
205
+ def advance_by_page(page)
206
+ @cursor =
207
+ if @is_forward_pagination
208
+ page.next_cursor
209
+ else
210
+ page.previous_cursor
211
+ end
212
+ end
213
+ end
214
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ VERSION = "0.1.0"
5
+ end
@@ -0,0 +1,32 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record"
4
+
5
+ require_relative "activerecord_cursor_paginate/version"
6
+ require_relative "activerecord_cursor_paginate/cursor"
7
+ require_relative "activerecord_cursor_paginate/page"
8
+ require_relative "activerecord_cursor_paginate/paginator"
9
+ require_relative "activerecord_cursor_paginate/extension"
10
+ require_relative "activerecord_cursor_paginate/config"
11
+
12
+ module ActiveRecordCursorPaginate
13
+ class Error < StandardError; end
14
+
15
+ # Error that gets raised if a cursor given as `before` or `after` cannot be
16
+ # properly parsed.
17
+ class InvalidCursorError < Error; end
18
+
19
+ class << self
20
+ def configure
21
+ yield config
22
+ end
23
+
24
+ def config
25
+ @config ||= Config.new
26
+ end
27
+ end
28
+ end
29
+
30
+ ActiveSupport.on_load(:active_record) do
31
+ extend ActiveRecordCursorPaginate::Extension
32
+ end
metadata ADDED
@@ -0,0 +1,70 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: activerecord_cursor_paginate
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - fatkodima
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2024-03-08 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: activerecord
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '7.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '7.0'
27
+ description:
28
+ email:
29
+ - fatkodima123@gmail.com
30
+ executables: []
31
+ extensions: []
32
+ extra_rdoc_files: []
33
+ files:
34
+ - CHANGELOG.md
35
+ - LICENSE.txt
36
+ - README.md
37
+ - lib/activerecord_cursor_paginate.rb
38
+ - lib/activerecord_cursor_paginate/config.rb
39
+ - lib/activerecord_cursor_paginate/cursor.rb
40
+ - lib/activerecord_cursor_paginate/extension.rb
41
+ - lib/activerecord_cursor_paginate/page.rb
42
+ - lib/activerecord_cursor_paginate/paginator.rb
43
+ - lib/activerecord_cursor_paginate/version.rb
44
+ homepage: https://github.com/fatkodima/activerecord_cursor_paginate
45
+ licenses:
46
+ - MIT
47
+ metadata:
48
+ homepage_uri: https://github.com/fatkodima/activerecord_cursor_paginate
49
+ source_code_uri: https://github.com/fatkodima/activerecord_cursor_paginate
50
+ changelog_uri: https://github.com/fatkodima/activerecord_cursor_paginate/blob/master/CHANGELOG.md
51
+ post_install_message:
52
+ rdoc_options: []
53
+ require_paths:
54
+ - lib
55
+ required_ruby_version: !ruby/object:Gem::Requirement
56
+ requirements:
57
+ - - ">="
58
+ - !ruby/object:Gem::Version
59
+ version: '2.7'
60
+ required_rubygems_version: !ruby/object:Gem::Requirement
61
+ requirements:
62
+ - - ">="
63
+ - !ruby/object:Gem::Version
64
+ version: '0'
65
+ requirements: []
66
+ rubygems_version: 3.5.4
67
+ signing_key:
68
+ specification_version: 4
69
+ summary: Cursor-based pagination for ActiveRecord.
70
+ test_files: []