activerecord_cursor_paginate 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 6a96b7bf3c40469653bf87dcf68cc159b30c389ed734047ef165b12d6ff7ef24
4
+ data.tar.gz: 2dea9c8b3847b5a72d6a915e9051e8b61891940e9feaa855edc822d49a393750
5
+ SHA512:
6
+ metadata.gz: f54a87c366b7cfd08e59530b29fab0e0d5736cc323c4bab81ba0c3873ae5faa1254a2a9c94563e7aa9751debdb8802f28454a5781ddccf6811fe912c6808afca
7
+ data.tar.gz: c5689804eb34fa24e4e38f80994c9e3f16f45ab2594fe1fe9482397d2260b7d6db3453168a61fcb5f7e614ad97d8dfb6d8a1a42b4e560d780cebda759fd9f26d
data/CHANGELOG.md ADDED
@@ -0,0 +1,5 @@
1
+ ## master (unreleased)
2
+
3
+ ## 0.1.0 (2024-03-08)
4
+
5
+ - First release
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2024 fatkodima
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,374 @@
1
+ # ActiveRecordCursorPaginate
2
+
3
+ This library allows to paginate through an `ActiveRecord` relation using cursor pagination.
4
+ It also supports ordering by any column on the relation in either ascending or descending order.
5
+
6
+ Cursor pagination allows to paginate results and gracefully deal with deletions / additions on previous pages.
7
+ Where a regular limit / offset pagination would jump in results if a record on a previous page gets deleted or added while requesting the next page, cursor pagination just returns the records following the one identified in the request.
8
+
9
+ To learn more about cursor pagination, check out the _"How does it work?"_ section below.
10
+
11
+ [![Build Status](https://github.com/fatkodima/activerecord_cursor_paginate/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/fatkodima/activerecord_cursor_paginate/actions/workflows/test.yml)
12
+
13
+ ## Requirements
14
+
15
+ - Ruby 2.7+
16
+ - Rails (ActiveRecord) 7.0+
17
+
18
+ If you need support for older ruby and rails, please open an issue.
19
+
20
+ ## Installation
21
+
22
+ Add this line to your application's Gemfile:
23
+
24
+ ```ruby
25
+ gem "activerecord_cursor_paginate"
26
+ ```
27
+
28
+ And then run:
29
+
30
+ ```sh
31
+ $ bundle install
32
+ ```
33
+
34
+ ## Usage
35
+
36
+ Let's assume we have a `Post` model of which we want to fetch some data and then paginate through it.
37
+ Therefore, we first apply our scopes, `where` clauses or other functionality as usual:
38
+
39
+ ```ruby
40
+ posts = Post.where(author: "Jane")
41
+ ```
42
+
43
+ And then we create our paginator to fetch the first response page:
44
+
45
+ ```ruby
46
+ paginator = posts.cursor_paginate
47
+ page = paginator.fetch
48
+ page.records # => [#<Post:0x00007fd7071b2ea8 @id=1>, #<Post:0x00007fd7071bb738 @id=2>, ..., #<Post:0x00007fd707238260 @id=10>]
49
+ page.count # => 10
50
+ page.empty? # => false
51
+ page.cursors # => ["MQ==", "Mg==", ..., "MTA="]
52
+ | |
53
+ | |
54
+ page.previous_cursor # => "MQ==" |
55
+ page.next_cursor # => "MTA=" ------------------|
56
+ page.has_previous? # => false
57
+ page.has_next? # => true
58
+ ```
59
+
60
+ Note that any ordering of the relation at this stage will be ignored by the gem.
61
+ Take a look at the next section _"Ordering"_ to see how you can have an order different than ascending IDs.
62
+
63
+ To then get the next result page, you simply need to pass the last cursor of the returned page item via:
64
+
65
+ ```ruby
66
+ paginator = posts.cursor_paginate(after: "MTA=")
67
+ ```
68
+
69
+ This will then fetch the next result page.
70
+ You can also just as easily paginate to previous pages by using `before` instead of `after` and using the first cursor of the current page.
71
+
72
+ ```ruby
73
+ paginator = posts.cursor_paginate(before: "MQ==")
74
+ ```
75
+
76
+ By default, this will always return up to 10 results. But you can also specify how many records should be returned via `limit` parameter.
77
+
78
+ ```ruby
79
+ paginator = posts.cursor_paginate(after: "MTA=", limit: 2)
80
+ ```
81
+
82
+ ```ruby
83
+ paginator = posts.cursor_paginate(before: "MQ==", limit: 2)
84
+ ```
85
+
86
+ You can also easily iterate over the whole relation:
87
+
88
+ ```ruby
89
+ paginator = posts.cursor_paginate
90
+
91
+ # Will lazily iterate over the pages.
92
+ paginator.pages.each do |page|
93
+ # do something with the page
94
+ end
95
+ ```
96
+
97
+ ### Ordering
98
+
99
+ As said, this gem ignores any previous ordering added to the passed relation.
100
+ But you can still paginate through relations with an order different than by ascending IDs.
101
+
102
+ You can specify a different column and direction to order the results by via an `order` parameter.
103
+
104
+ ```ruby
105
+ # Order records ascending by the `:author` field.
106
+ paginator = posts.cursor_paginate(order: :author)
107
+
108
+ # Order records descending by the `:author` field.
109
+ paginator = posts.cursor_paginate(order: { author: :desc })
110
+
111
+ # Order records ascending by the `:author` and `:title` fields.
112
+ paginator = posts.cursor_paginate(order: [:author, :title])
113
+
114
+ # Order records ascending by the `:author` and descending by the `:title` fields.
115
+ paginator = posts.cursor_paginate(order: { author: :asc, title: :desc })
116
+ ```
117
+
118
+ **Important:**
119
+ If your app regularly orders by another column, you might want to add a database index for this.
120
+ Say that your order column is `author` then you'll want to add a compound index on `(author, id)`.
121
+ If your table is called `posts` you can use a query like this in MySQL or PostgreSQL:
122
+
123
+ ```sql
124
+ CREATE INDEX index_posts_on_author_and_id ON posts (author, id)
125
+ ```
126
+
127
+ Or you can just do it via an `ActiveRecord::Migration`:
128
+
129
+ ```ruby
130
+ class AddAuthorAndIdIndexToPosts < ActiveRecord::Migration[7.1]
131
+ def change
132
+ add_index :posts, [:author, :id]
133
+ end
134
+ end
135
+ ```
136
+
137
+ Take a look at the _"How does it work?"_ to find out more why this is necessary.
138
+
139
+ #### Ordering and `JOIN`s
140
+
141
+ To order by a column from the `JOIN`ed table, you need to explicitly specify and fully qualify the column name for the `:order` parameter:
142
+
143
+ ```ruby
144
+ paginator = User.joins(:projects).cursor_paginate(order: [Arel.sql("projects.id"), :id])
145
+ page = paginator.fetch
146
+ ```
147
+
148
+ **Note**: Make sure to wrap custom SQL expressions by `Arel.sql`.
149
+
150
+ #### Order by more complex logic
151
+
152
+ Sometimes you might not only want to oder by a column ascending or descending, but need more complex logic.
153
+ Imagine you would also store the post's `category` on the `posts` table (as a plain string for simplicity's sake).
154
+ And the category could be `pinned`, `announcement`, or `general`.
155
+ Then you might want to show all `pinned` posts first, followed by the `announcement` ones and lastly show the `general` posts.
156
+
157
+ In MySQL you could e.g. use a `FIELD(category, 'pinned', 'announcement', 'general')` query in the `ORDER BY` clause to achieve this.
158
+ However, you cannot add an index to such a statement. And therefore, the performance of this is pretty dismal.
159
+
160
+ The gem supports ordering by custom SQL expressions, but make sure the performance will not suffer.
161
+
162
+ What is recommended if you _do_ need to order by more complex logic is to have a separate column that you only use for ordering.
163
+ You can use `ActiveRecord` hooks to automatically update this column whenever you change your data.
164
+ Or, for example in MySQL, you can also use a [generated column](https://dev.mysql.com/doc/refman/5.7/en/create-table-generated-columns.html) that is automatically being updated by the database based on some stored logic.
165
+
166
+ For example, if you want paginate `users` by a lowercased `email`, you can use the following:
167
+
168
+ ```ruby
169
+ paginator = User.cursor_paginate(order: Arel.sql("lower(email)"))
170
+ page = paginator.fetch
171
+ ```
172
+
173
+ **Note**: Make sure to wrap custom SQL expressions by `Arel.sql`.
174
+
175
+ ### Configuration
176
+
177
+ You can change the default page size to a value that better fits the needs of your application.
178
+ So if a user doesn't request a given `:limit` value, the default amount of records is being returned.
179
+
180
+ To change the default, simply add an initializer to your app that does the following:
181
+
182
+ ```ruby
183
+ # config/initializers/activerecord_cursor_paginate.rb
184
+ ActiveRecordCursorPaginate.configure do |config|
185
+ config.default_page_size = 50
186
+ end
187
+ ```
188
+
189
+ This would set the default page size to 50.
190
+
191
+ You can also set a global `max_page_size` to prevent a client from requesting too large pages.
192
+
193
+ ```ruby
194
+ ActiveRecordCursorPaginate.configure do |config|
195
+ config.max_page_size = 100
196
+ end
197
+ ```
198
+
199
+ ## How does it work?
200
+
201
+ This library allows to paginate through a passed relation using a cursor
202
+ for before or after parameters. It also supports ordering by any column
203
+ on the relation in either ascending or descending order.
204
+
205
+ Cursor pagination allows to paginate results and gracefully deal with
206
+ deletions / additions on previous pages. Where a regular limit / offset
207
+ pagination would jump in results if a record on a previous page gets deleted
208
+ or added while requesting the next page, cursor pagination just returns the
209
+ records following the one identified in the request.
210
+
211
+ How this works is that it uses a "cursor", which is an encoded value that
212
+ uniquely identifies a given row for the requested order. Then, based on
213
+ this cursor, you can request the "n records AFTER the cursor"
214
+ (forward-pagination) or the "n records BEFORE the cursor" (backward-pagination).
215
+
216
+ As an example, assume we have a table called "posts" with this data:
217
+
218
+ | id | author |
219
+ |----|--------|
220
+ | 1 | Jane |
221
+ | 2 | John |
222
+ | 3 | John |
223
+ | 4 | Jane |
224
+ | 5 | Jane |
225
+ | 6 | John |
226
+ | 7 | John |
227
+
228
+ Now if we make a basic request without any `before`, `after`, custom `order` column,
229
+ this will just request the first page of this relation.
230
+
231
+ ```ruby
232
+ paginator = relation.cursor_paginate
233
+ page = paginator.fetch
234
+ ```
235
+
236
+ Assume that our default page size here is 2 and we would get a query like this:
237
+
238
+ ```sql
239
+ SELECT *
240
+ FROM "posts"
241
+ ORDER BY "posts"."id" ASC
242
+ LIMIT 2
243
+ ```
244
+
245
+ This will return the first page of results, containing post #1 and #2. Since
246
+ no custom order is defined, each item in the returned collection will have a
247
+ cursor that only encodes the record's ID.
248
+
249
+ If we want to now request the next page, we can pass in the cursor of record
250
+ #2 which would be `"Mg=="` (can get via `page.cursor`). So now we can request
251
+ the next page by calling:
252
+
253
+ ```ruby
254
+ paginator = relation.cursor_paginate(limit: 2, after: "Mg==")
255
+ page = paginator.fetch
256
+ ```
257
+
258
+ And this will decode the given cursor and issue a query like:
259
+
260
+ ```sql
261
+ SELECT *
262
+ FROM "posts"
263
+ WHERE "posts"."id" > 2
264
+ ORDER BY "posts"."id" ASC
265
+ LIMIT 2
266
+ ```
267
+
268
+ Which would return posts #3 and #4. If we now want to paginate back, we can
269
+ request the posts that came before the first post, whose cursor would be
270
+ `"Mw=="` (can get via `page.previous_cursor`):
271
+
272
+ ```ruby
273
+ paginator = relation.cursor_paginate(limit: 2, before: "Mw==")
274
+ page = paginator.fetch
275
+ ```
276
+
277
+ Since we now paginate backward, the resulting SQL query needs to be flipped
278
+ around to get the last two records that have an ID smaller than the given one:
279
+
280
+ ```sql
281
+ SELECT *
282
+ FROM "posts"
283
+ WHERE "posts"."id" < 3
284
+ ORDER BY "posts"."id" DESC
285
+ LIMIT 2
286
+ ```
287
+
288
+ This would return posts #2 and #1. Since we still requested them in
289
+ ascending order, the result will be reversed before it is returned.
290
+
291
+ Now, in case that the user wants to order by a column different than the ID,
292
+ we require this information in our cursor. Therefore, when requesting the
293
+ first page like this:
294
+
295
+ ```ruby
296
+ paginator = relation.cursor_paginate(order: :author)
297
+ page = paginator.fetch
298
+ ```
299
+
300
+ This will issue the following SQL query:
301
+
302
+ ```sql
303
+ SELECT *
304
+ FROM "posts"
305
+ ORDER BY "posts"."author" ASC, "posts"."id" ASC
306
+ LIMIT 2
307
+ ```
308
+
309
+ As you can see, this will now order by the author first, and if two records
310
+ have the same author it will order them by ID. Ordering only the author is not
311
+ enough since we cannot know if the custom column only has unique values.
312
+ And we need to guarantee the correct order of ambiguous records independent
313
+ of the direction of ordering. This unique order is the basis of being able
314
+ to paginate forward and backward repeatedly and getting the correct records.
315
+
316
+ The query will then return records #1 and #4. But the cursor for these
317
+ records will also be different to the previous query where we ordered by ID
318
+ only. It is important that the cursor encodes all the data we need to
319
+ uniquely identify a row and filter based upon it. Therefore, we need to
320
+ encode the same information as we used for the ordering in our SQL query.
321
+ Hence, the cursor for pagination with a custom column contains a tuple of
322
+ data, the first record being the custom order column followed by the
323
+ record's ID.
324
+
325
+ Therefore, the cursor of record #4 will encode `['Jane', 4]`, which yields
326
+ this cursor: `"WyJKYW5lIiw0XQ=="`.
327
+
328
+ If we now want to request the next page via:
329
+
330
+ ```ruby
331
+ paginator = relation.cursor_paginate(order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
332
+ page = paginator.fetch
333
+ ```
334
+
335
+ We get this SQL query:
336
+
337
+ ```sql
338
+ SELECT *
339
+ FROM "posts"
340
+ WHERE (author > 'Jane' OR (author = 'Jane') AND ("posts"."id" > 4))
341
+ ORDER BY "posts"."author" ASC, "posts"."id" ASC
342
+ LIMIT 2
343
+ ```
344
+
345
+ You can see how the cursor is being used by the WHERE clause to uniquely
346
+ identify the row and properly filter based on this. We only want to get
347
+ records that either have a name that is alphabetically after `"Jane"` or
348
+ another `"Jane"` record with an ID that is higher than `4`. We will get the
349
+ records #5 and #2 as response.
350
+
351
+ When using a custom `order`, this affects both filtering as well as
352
+ ordering. Therefore, it is recommended to add an index for columns that are
353
+ frequently used for ordering. In our test case we would want to add a compound
354
+ index for the `(author, id)` column combination. Databases like MySQL and
355
+ PostgreSQL are able to then use the leftmost part of the index, in our case
356
+ `author`, by its own or can use it combined with the `id` index.
357
+
358
+ ## Credits
359
+
360
+ Thanks to [rails_cursor_pagination gem](https://github.com/xing/rails_cursor_pagination) for the original ideas.
361
+
362
+ ## Development
363
+
364
+ After checking out the repo, run `bundle install` to install dependencies. Then, run `bundle exec rake` to run the linter and tests. This project uses multiple Gemfiles to test against multiple versions of Active Record; you can run the tests against the specific version with `BUNDLE_GEMFILE=gemfiles/activerecord_71.gemfile bundle exec rake test`.
365
+
366
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
367
+
368
+ ## Contributing
369
+
370
+ Bug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/activerecord_cursor_paginate.
371
+
372
+ ## License
373
+
374
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
@@ -0,0 +1,11 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ class Config
5
+ attr_accessor :default_page_size, :max_page_size
6
+
7
+ def initialize
8
+ @default_page_size = 10
9
+ end
10
+ end
11
+ end
@@ -0,0 +1,73 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "base64"
4
+ require "json"
5
+
6
+ module ActiveRecordCursorPaginate
7
+ # @private
8
+ class Cursor
9
+ class << self
10
+ def from_record(record, columns:)
11
+ columns = columns.map { |column| column.to_s.split(".").last }
12
+ values = columns.map { |column| record[column] }
13
+ new(columns: columns, values: values)
14
+ end
15
+
16
+ def decode(cursor_string:, columns:)
17
+ decoded = JSON.parse(Base64.strict_decode64(cursor_string))
18
+
19
+ if (columns.size == 1 && decoded.is_a?(Array)) ||
20
+ (decoded.is_a?(Array) && decoded.size != columns.size)
21
+ raise InvalidCursorError,
22
+ "The given cursor `#{cursor_string}` was decoded as `#{decoded}` but could not be parsed"
23
+ end
24
+
25
+ decoded =
26
+ if decoded.is_a?(Array)
27
+ decoded.map { |value| deserialize_time_if_needed(value) }
28
+ else
29
+ deserialize_time_if_needed(decoded)
30
+ end
31
+
32
+ new(columns: columns, values: decoded)
33
+ rescue ArgumentError, JSON::ParserError # ArgumentError is raised by strict_decode64
34
+ raise InvalidCursorError, "The given cursor `#{cursor_string}` could not be decoded"
35
+ end
36
+
37
+ private
38
+ def deserialize_time_if_needed(value)
39
+ if value.is_a?(String) && value.start_with?(TIMESTAMP_PREFIX)
40
+ seconds_with_frac = value.delete_prefix(TIMESTAMP_PREFIX).to_r / (10**6)
41
+ Time.at(seconds_with_frac).utc
42
+ else
43
+ value
44
+ end
45
+ end
46
+ end
47
+
48
+ attr_reader :columns, :values
49
+
50
+ def initialize(columns:, values:)
51
+ @columns = Array(columns)
52
+ @values = Array(values)
53
+
54
+ raise ArgumentError, "Cursor values can not be nil" if @values.any?(nil)
55
+ raise ArgumentError, ":columns and :values have different sizes" if @columns.size != @values.size
56
+ end
57
+
58
+ def encode
59
+ serialized_values = values.map do |value|
60
+ if value.is_a?(Time)
61
+ TIMESTAMP_PREFIX + value.strftime("%s%6N")
62
+ else
63
+ value
64
+ end
65
+ end
66
+ unencoded_cursor = (serialized_values.size == 1 ? serialized_values.first : serialized_values)
67
+ Base64.strict_encode64(unencoded_cursor.to_json)
68
+ end
69
+
70
+ TIMESTAMP_PREFIX = "0aIX2_" # something random
71
+ private_constant :TIMESTAMP_PREFIX
72
+ end
73
+ end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ module Extension
5
+ # Convenient method to use on ActiveRecord::Relation to get a paginator.
6
+ # @return [ActiveRecordCursorPaginate::Paginator]
7
+ #
8
+ # @example
9
+ # paginator = Post.all.cursor_paginate(limit: 2, after: "Mg==")
10
+ # page = paginator.fetch
11
+ #
12
+ def cursor_paginate(after: nil, before: nil, limit: nil, order: nil)
13
+ relation = (is_a?(ActiveRecord::Relation) ? self : all)
14
+ Paginator.new(relation, after: after, before: before, limit: limit, order: order)
15
+ end
16
+ alias cursor_pagination cursor_paginate
17
+ end
18
+ end
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ # Represents a batch of records retrieved via a single iteration of
5
+ # cursor-based pagination.
6
+ #
7
+ class Page
8
+ # Records this page contains.
9
+ # @return [ActiveRecord::Base]
10
+ #
11
+ attr_reader :records
12
+
13
+ def initialize(records, order_columns:, has_previous: false, has_next: false)
14
+ @records = records
15
+ @order_columns = order_columns
16
+ @has_previous = has_previous
17
+ @has_next = has_next
18
+ end
19
+
20
+ # Number of records in this page.
21
+ # @return [Integer]
22
+ #
23
+ def count
24
+ records.size
25
+ end
26
+
27
+ # Whether this page is empty.
28
+ # @return [Boolean]
29
+ #
30
+ def empty?
31
+ count == 0
32
+ end
33
+
34
+ # Returns the cursor, which can be used to retrieve the next page.
35
+ # @return [String]
36
+ #
37
+ def next_cursor
38
+ cursor_for_record(records.last)
39
+ end
40
+ alias cursor next_cursor
41
+
42
+ # Returns the cursor, which can be used to retrieve the previous page.
43
+ # @return [String]
44
+ #
45
+ def previous_cursor
46
+ cursor_for_record(records.first)
47
+ end
48
+
49
+ # Whether this page has a previous page.
50
+ # @return [Boolean]
51
+ #
52
+ def has_previous?
53
+ @has_previous
54
+ end
55
+
56
+ # Whether this page has a next page.
57
+ # @return [Boolean]
58
+ #
59
+ def has_next?
60
+ @has_next
61
+ end
62
+
63
+ # Returns cursor for a specific record.
64
+ #
65
+ # @param record [ActiveRecord::Base]
66
+ # @return [String]
67
+ #
68
+ def cursor_for(record)
69
+ cursor_for_record(record)
70
+ end
71
+
72
+ # Returns cursors for all the records on this page.
73
+ # @return [Array<String>]
74
+ #
75
+ def cursors
76
+ records.map { |record| cursor_for_record(record) }
77
+ end
78
+
79
+ private
80
+ def cursor_for_record(record)
81
+ cursor = Cursor.from_record(record, columns: @order_columns)
82
+ cursor.encode
83
+ end
84
+ end
85
+ end
@@ -0,0 +1,214 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ # Use this Paginator class to effortlessly paginate through ActiveRecord
5
+ # relations using cursor pagination.
6
+ #
7
+ # @example Iterating one page at a time
8
+ # ActiveRecordCursorPaginate::Paginator
9
+ # .new(relation, order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
10
+ # .fetch
11
+ #
12
+ # @example Iterating over the whole relation
13
+ # paginator = ActiveRecordCursorPaginate::Paginator
14
+ # .new(relation, order: :author, limit: 2, after: "WyJKYW5lIiw0XQ==")
15
+ #
16
+ # # Will lazily iterate over the pages.
17
+ # paginator.pages.each do |page|
18
+ # # do something with the page
19
+ # end
20
+ #
21
+ class Paginator
22
+ # Create a new instance of the `ActiveRecordCursorPaginate::Paginator`
23
+ #
24
+ # @param relation [ActiveRecord::Relation] Relation that will be paginated.
25
+ # @param before [String, nil] Cursor to paginate upto (excluding).
26
+ # @param after [String, nil] Cursor to paginate forward from.
27
+ # @param limit [Integer, nil] Number of records to return in pagination.
28
+ # @param order [Symbol, String, nil, Array<Symbol, String>, Hash]
29
+ # Column(s) to order by, optionally with directions (either `:asc` or `:desc`,
30
+ # defaults to `:asc`). If none is provided, will default to ID column.
31
+ # NOTE: this will cause the query to filter on both the given column as
32
+ # well as the ID column. So you might want to add a compound index to your
33
+ # database similar to:
34
+ # ```sql
35
+ # CREATE INDEX <index_name> ON <table_name> (<order_fields>..., id)
36
+ # ```
37
+ # @raise [ArgumentError] If any parameter is not valid
38
+ #
39
+ def initialize(relation, before: nil, after: nil, limit: nil, order: nil)
40
+ unless relation.is_a?(ActiveRecord::Relation)
41
+ raise ArgumentError, "relation is not an ActiveRecord::Relation"
42
+ end
43
+
44
+ if before.present? && after.present?
45
+ raise ArgumentError, "Only one of :before and :after can be provided"
46
+ end
47
+
48
+ @relation = relation
49
+ @primary_key = @relation.primary_key
50
+ @cursor = before || after
51
+ @is_forward_pagination = before.blank?
52
+
53
+ config = ActiveRecordCursorPaginate.config
54
+ @page_size = limit || config.default_page_size
55
+ @page_size = [@page_size, config.max_page_size].min if config.max_page_size
56
+
57
+ order = normalize_order(order)
58
+ @columns = order.keys
59
+ @directions = order.values
60
+ end
61
+
62
+ # Get the paginated result.
63
+ # @return [ActiveRecordCursorPaginate::Page]
64
+ #
65
+ # @note Calling this method advances the paginator.
66
+ #
67
+ def fetch
68
+ relation = @relation
69
+
70
+ # Non trivial columns (expressions or joined tables columns).
71
+ if @columns.any?(/\W/)
72
+ arel_columns = @columns.map.with_index do |column, i|
73
+ arel_column(column).as("cursor_column_#{i + 1}")
74
+ end
75
+ cursor_column_names = 1.upto(@columns.size).map { |i| "cursor_column_#{i}" }
76
+
77
+ relation =
78
+ if relation.select_values.empty?
79
+ relation.select(Arel.star, arel_columns)
80
+ else
81
+ relation.select(arel_columns)
82
+ end
83
+ else
84
+ cursor_column_names = @columns
85
+ end
86
+
87
+ pagination_directions = @directions.map { |direction| pagination_direction(direction) }
88
+ relation = relation.reorder(cursor_column_names.zip(pagination_directions).to_h)
89
+
90
+ if @cursor
91
+ decoded_cursor = Cursor.decode(cursor_string: @cursor, columns: @columns)
92
+ relation = apply_cursor(relation, decoded_cursor)
93
+ end
94
+
95
+ relation = relation.limit(@page_size + 1)
96
+ records_plus_one = relation.to_a
97
+ has_additional = records_plus_one.size > @page_size
98
+
99
+ records = records_plus_one.take(@page_size)
100
+ records.reverse! unless @is_forward_pagination
101
+
102
+ if @is_forward_pagination
103
+ has_next_page = has_additional
104
+ has_previous_page = @cursor.present?
105
+ else
106
+ has_next_page = @cursor.present?
107
+ has_previous_page = has_additional
108
+ end
109
+
110
+ page = Page.new(
111
+ records,
112
+ order_columns: cursor_column_names,
113
+ has_next: has_next_page,
114
+ has_previous: has_previous_page
115
+ )
116
+
117
+ advance_by_page(page) unless page.empty?
118
+
119
+ page
120
+ end
121
+ alias page fetch
122
+
123
+ # Returns an enumerator that can be used to iterate over the whole relation.
124
+ # @return [Enumerator]
125
+ #
126
+ def pages
127
+ Enumerator.new do |yielder|
128
+ loop do
129
+ page = fetch
130
+ break if page.empty?
131
+
132
+ yielder.yield(page)
133
+ end
134
+ end
135
+ end
136
+
137
+ private
138
+ def normalize_order(order)
139
+ order ||= {}
140
+ default_direction = :asc
141
+
142
+ result =
143
+ case order
144
+ when String, Symbol
145
+ { order => default_direction }
146
+ when Hash
147
+ order
148
+ when Array
149
+ order.to_h { |column| [column, default_direction] }
150
+ else
151
+ raise ArgumentError, "Invalid order: #{order.inspect}"
152
+ end
153
+
154
+ result = result.with_indifferent_access
155
+ result.transform_values! { |direction| direction.downcase.to_sym }
156
+ Array(@primary_key).each { |column| result[column] ||= default_direction }
157
+ result
158
+ end
159
+
160
+ def apply_cursor(relation, cursor)
161
+ operators = @directions.map { |direction| pagination_operator(direction) }
162
+ cursor_positions = cursor.columns.zip(cursor.values, operators)
163
+
164
+ where_clause = nil
165
+ cursor_positions.reverse_each.with_index do |(column, value, operator), index|
166
+ where_clause =
167
+ if index == 0
168
+ arel_column(column).public_send(operator, value)
169
+ else
170
+ arel_column(column).public_send(operator, value).or(
171
+ arel_column(column).eq(value).and(where_clause)
172
+ )
173
+ end
174
+ end
175
+
176
+ relation.where(where_clause)
177
+ end
178
+
179
+ def arel_column(column)
180
+ if Arel.arel_node?(column)
181
+ column
182
+ elsif column.match?(/\A\w+\.\w+\z/)
183
+ Arel.sql(column)
184
+ else
185
+ @relation.arel_table[column]
186
+ end
187
+ end
188
+
189
+ def pagination_direction(direction)
190
+ if @is_forward_pagination
191
+ direction
192
+ else
193
+ direction == :asc ? :desc : :asc
194
+ end
195
+ end
196
+
197
+ def pagination_operator(direction)
198
+ if @is_forward_pagination
199
+ direction == :asc ? :gt : :lt
200
+ else
201
+ direction == :asc ? :lt : :gt
202
+ end
203
+ end
204
+
205
+ def advance_by_page(page)
206
+ @cursor =
207
+ if @is_forward_pagination
208
+ page.next_cursor
209
+ else
210
+ page.previous_cursor
211
+ end
212
+ end
213
+ end
214
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordCursorPaginate
4
+ VERSION = "0.1.0"
5
+ end
@@ -0,0 +1,32 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record"
4
+
5
+ require_relative "activerecord_cursor_paginate/version"
6
+ require_relative "activerecord_cursor_paginate/cursor"
7
+ require_relative "activerecord_cursor_paginate/page"
8
+ require_relative "activerecord_cursor_paginate/paginator"
9
+ require_relative "activerecord_cursor_paginate/extension"
10
+ require_relative "activerecord_cursor_paginate/config"
11
+
12
+ module ActiveRecordCursorPaginate
13
+ class Error < StandardError; end
14
+
15
+ # Error that gets raised if a cursor given as `before` or `after` cannot be
16
+ # properly parsed.
17
+ class InvalidCursorError < Error; end
18
+
19
+ class << self
20
+ def configure
21
+ yield config
22
+ end
23
+
24
+ def config
25
+ @config ||= Config.new
26
+ end
27
+ end
28
+ end
29
+
30
+ ActiveSupport.on_load(:active_record) do
31
+ extend ActiveRecordCursorPaginate::Extension
32
+ end
metadata ADDED
@@ -0,0 +1,70 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: activerecord_cursor_paginate
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - fatkodima
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2024-03-08 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: activerecord
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '7.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '7.0'
27
+ description:
28
+ email:
29
+ - fatkodima123@gmail.com
30
+ executables: []
31
+ extensions: []
32
+ extra_rdoc_files: []
33
+ files:
34
+ - CHANGELOG.md
35
+ - LICENSE.txt
36
+ - README.md
37
+ - lib/activerecord_cursor_paginate.rb
38
+ - lib/activerecord_cursor_paginate/config.rb
39
+ - lib/activerecord_cursor_paginate/cursor.rb
40
+ - lib/activerecord_cursor_paginate/extension.rb
41
+ - lib/activerecord_cursor_paginate/page.rb
42
+ - lib/activerecord_cursor_paginate/paginator.rb
43
+ - lib/activerecord_cursor_paginate/version.rb
44
+ homepage: https://github.com/fatkodima/activerecord_cursor_paginate
45
+ licenses:
46
+ - MIT
47
+ metadata:
48
+ homepage_uri: https://github.com/fatkodima/activerecord_cursor_paginate
49
+ source_code_uri: https://github.com/fatkodima/activerecord_cursor_paginate
50
+ changelog_uri: https://github.com/fatkodima/activerecord_cursor_paginate/blob/master/CHANGELOG.md
51
+ post_install_message:
52
+ rdoc_options: []
53
+ require_paths:
54
+ - lib
55
+ required_ruby_version: !ruby/object:Gem::Requirement
56
+ requirements:
57
+ - - ">="
58
+ - !ruby/object:Gem::Version
59
+ version: '2.7'
60
+ required_rubygems_version: !ruby/object:Gem::Requirement
61
+ requirements:
62
+ - - ">="
63
+ - !ruby/object:Gem::Version
64
+ version: '0'
65
+ requirements: []
66
+ rubygems_version: 3.5.4
67
+ signing_key:
68
+ specification_version: 4
69
+ summary: Cursor-based pagination for ActiveRecord.
70
+ test_files: []