custom_counter_cache 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +112 -0
- data/LICENSE +9 -0
- data/README.rdoc +261 -0
- data/lib/custom_counter_cache/dispatcher.rb +101 -0
- data/lib/custom_counter_cache/model.rb +197 -62
- data/lib/custom_counter_cache/railtie.rb +3 -0
- data/lib/custom_counter_cache/recount_job.rb +16 -0
- data/lib/custom_counter_cache/version.rb +1 -1
- data/lib/custom_counter_cache.rb +35 -1
- data/lib/tasks/custom_counter_cache.rake +12 -0
- metadata +126 -19
- data/test/counter_test.rb +0 -282
- data/test/test_helper.rb +0 -138
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 56544a2488e48dc35c3ad64e7eb7cb0bc2c0be1bc3f5aa8ee5a52a349de2fcb8
|
|
4
|
+
data.tar.gz: 5d40812f87a2d9ce720e77558ac1cec17c81a607170d2a9e690b56bcc9481e29
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 287db2bc86cca62e838cc32849eb1bb43fe66f9d40e771981bb56df221617e6bdf4392950808cafa031afb8a90fe9682ca68053bc0867ad5df802a76ca47f073
|
|
7
|
+
data.tar.gz: e096528b08c44673508784b4faa7bdaa500d5f5f30b5844d4d2a68a1dcf77ff241df30d4805ad58eb22f126f6fb821fc9214c5114fdf4ee520eca44e15ee3981
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.4.0
|
|
4
|
+
|
|
5
|
+
### Breaking
|
|
6
|
+
|
|
7
|
+
- Requires Ruby >= 3.3 and Rails (Active Record) >= 8.0.
|
|
8
|
+
- `:if`/`:unless` on `update_counter_cache` now only gate the create and update callbacks; a
|
|
9
|
+
destroy always recounts. Use `except: [:destroy]` to skip it.
|
|
10
|
+
- Column counters are written with `update_column`: recounts no longer save other unsaved
|
|
11
|
+
changes on the owner, touch its `updated_at`, run its callbacks or bump `lock_version`.
|
|
12
|
+
- `update_counter_cache` raises `ArgumentError` for associations other than `belongs_to`
|
|
13
|
+
(they have never worked).
|
|
14
|
+
- Every model that calls `define_counter_cache` gets the `counters` association, not only models
|
|
15
|
+
with a counter that has no column.
|
|
16
|
+
- Defining counters on a table that doesn't exist no longer silently defines nothing.
|
|
17
|
+
- Recounts triggered by `update_counter_cache` callbacks run after the saving transaction commits
|
|
18
|
+
instead of inside it: once per owner and counter per transaction, not at all on rollback, and
|
|
19
|
+
under a short lock on the owner's row. Counting inside the transaction couldn't see a concurrent
|
|
20
|
+
save's uncommitted child, so concurrent saves to the same owner left a stale count (on
|
|
21
|
+
PostgreSQL, 399 of 400 concurrent test runs). The counter now changes at commit rather than at
|
|
22
|
+
save; call `update_<name>` directly to recount immediately inside a transaction. Rails'
|
|
23
|
+
transactional tests commit each save within the test transaction, so tests see the recount.
|
|
24
|
+
- A counter stored in the counters table raises `ArgumentError` for a value that isn't a whole
|
|
25
|
+
number (e.g. `2.5`, `'many'`, infinity) instead of silently truncating it with `to_i`. `nil`
|
|
26
|
+
still means 0, and whole values like `4.0` or `BigDecimal('4')` are still accepted. Use a column
|
|
27
|
+
of a suitable type or `store: :cache` for fractional values.
|
|
28
|
+
|
|
29
|
+
### Changed
|
|
30
|
+
|
|
31
|
+
- Whether a counter uses a column or the counters table is decided when it's read or written, so
|
|
32
|
+
loading models never queries the database, and a column added after the model has loaded is
|
|
33
|
+
picked up. The Heroku `DATABASE_URL` workaround is gone.
|
|
34
|
+
- Counter rows are deleted on destroy only when the model has a counter without a column, so
|
|
35
|
+
column-only models never touch the counters table.
|
|
36
|
+
- A recount that raises after commit is passed to `ActiveSupport.error_reporter.unexpected` (with
|
|
37
|
+
the owner and counter as context) and logged, instead of propagating: the save has already
|
|
38
|
+
committed, and one failure no longer skips the other recounts queued for that commit. In
|
|
39
|
+
development and test Rails' debug mode still raises it. Direct `update_<name>` calls,
|
|
40
|
+
`recount_counter_caches` and `CustomCounterCache::RecountJob` still raise.
|
|
41
|
+
- Paranoia's `restore` now recounts the owner for callbacks declared after `acts_as_paranoid`; it
|
|
42
|
+
didn't before. Add `except: [:restore]` to keep the old behavior.
|
|
43
|
+
|
|
44
|
+
### Added
|
|
45
|
+
|
|
46
|
+
- `CustomCounterCache.counter_class_name` (default `'Counter'`) sets the model behind the
|
|
47
|
+
counters table.
|
|
48
|
+
- `define_counter_cache :name, touch: true` (or `touch: :column`) sets a timestamp on the owner
|
|
49
|
+
whenever the counter is recounted. A column counter is written in the same UPDATE, and no
|
|
50
|
+
callbacks run.
|
|
51
|
+
- `define_counter_cache :name, store: :cache, expires_in:` keeps a counter in
|
|
52
|
+
`CustomCounterCache.cache_store` (defaults to `Rails.cache`). It is computed on a miss and
|
|
53
|
+
deleted after commit when a child changes, instead of being recounted. Destroying the owner
|
|
54
|
+
deletes its keys.
|
|
55
|
+
- `update_counter_cache ..., on_change: [:attr]` recounts on update only when a listed attribute
|
|
56
|
+
or the association's key columns changed. Create and destroy always recount.
|
|
57
|
+
- `update_counter_cache [:article, :user], :comments_count` follows a path of `belongs_to`
|
|
58
|
+
associations and recounts both the old and the new owner at the end of it (grandparent counters).
|
|
59
|
+
- `update_counter_cache ..., recount: :later` recounts in `CustomCounterCache::RecountJob`
|
|
60
|
+
(Active Job), enqueued after commit, once per owner and counter per transaction.
|
|
61
|
+
- `CustomCounterCache.batch { }` recounts each owner and counter once after the block ends.
|
|
62
|
+
- `CustomCounterCache.skip { }` drops recounts inside the block.
|
|
63
|
+
- `Model.recount_counter_caches(*names, scope:, batch_size:)` recounts every record (or a scope) in
|
|
64
|
+
batches, repairing drift from `update_all`, `delete_all` and imports.
|
|
65
|
+
- The `custom_counter_cache:recount` rake task (`MODEL=`, `COUNTERS=`, `BATCH_SIZE=`) wraps
|
|
66
|
+
`recount_counter_caches`.
|
|
67
|
+
- Composite primary keys: owners and `belongs_to` associations with composite keys work with
|
|
68
|
+
column and cache counters, reassignment (including a change to only some key columns),
|
|
69
|
+
`on_change:`, `recount: :later` and `recount_counter_caches`. The counters table can't hold a
|
|
70
|
+
composite key, so a counter there raises `ArgumentError` naming the alternatives.
|
|
71
|
+
- `only:` and `except:` on `update_counter_cache` accept `:restore`, the event fired by Paranoia's
|
|
72
|
+
`restore`.
|
|
73
|
+
|
|
74
|
+
### Upgrading
|
|
75
|
+
|
|
76
|
+
- If an `:if`/`:unless` condition was meant to skip destroys, add `except: [:destroy]`.
|
|
77
|
+
- If you relied on a recount updating the owner's `updated_at`, pass `touch: true` to
|
|
78
|
+
`define_counter_cache`. Owner callbacks no longer run on a recount; call them explicitly if needed.
|
|
79
|
+
- If a migration adds a counter column and backfills it, call `reset_column_information` on the
|
|
80
|
+
model before backfilling.
|
|
81
|
+
- Counters now change when the saving transaction commits. Code that saves a child and reads the
|
|
82
|
+
owner's counter inside the same transaction should call `update_<name>` first.
|
|
83
|
+
- A counters-table counter whose block can return a fractional value (an average, a sum of
|
|
84
|
+
decimals) now raises: move it to a column of a suitable type or to `store: :cache`.
|
|
85
|
+
- On a model with a string or UUID primary key, the counters table's `countable_id` must be that
|
|
86
|
+
type (`t.references :countable, polymorphic: true, type: :uuid`).
|
|
87
|
+
|
|
88
|
+
## 0.3.2
|
|
89
|
+
|
|
90
|
+
### Fixed
|
|
91
|
+
|
|
92
|
+
- Moving a record between owners now recounts the old owner when `belongs_to` uses a custom
|
|
93
|
+
`primary_key:`.
|
|
94
|
+
- Polymorphic `belongs_to` with custom `foreign_key:`/`foreign_type:` no longer raises
|
|
95
|
+
`NoMethodError` on save.
|
|
96
|
+
- Moving a record away from a polymorphic type whose class no longer exists no longer raises
|
|
97
|
+
`NameError`; the new owner is still recounted.
|
|
98
|
+
- With `includes(:counters)`, counters stay current after an update, and a missing counter
|
|
99
|
+
returns 0 without a query.
|
|
100
|
+
- Two saves creating the same virtual counter at once no longer fail with
|
|
101
|
+
`ActiveRecord::RecordNotUnique`; the second updates the row instead.
|
|
102
|
+
- `update_counter_cache` without its `belongs_to` raises a clear `ArgumentError` instead of
|
|
103
|
+
`NoMethodError ... for nil`.
|
|
104
|
+
|
|
105
|
+
### Changed
|
|
106
|
+
|
|
107
|
+
- Depend on `activerecord` and `activesupport` instead of all of `rails`.
|
|
108
|
+
- Require Ruby >= 3.1 in the gemspec, matching the README and Rails 7.2.
|
|
109
|
+
- Test against every non-EOL Ruby (3.3, 3.4, 4.0) and Rails (8.0, 8.1).
|
|
110
|
+
- Expand the test suite to cover all branches.
|
|
111
|
+
- README: fix the `:if` example (`state_changed?` is always false in after callbacks)
|
|
112
|
+
and note that `:if` also applies to `after_destroy`.
|
data/LICENSE
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright © 2011-2026 Cedric Howe
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
|
6
|
+
|
|
7
|
+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
|
8
|
+
|
|
9
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.rdoc
ADDED
|
@@ -0,0 +1,261 @@
|
|
|
1
|
+
{rdoc-image:https://github.com/cedric/custom_counter_cache/actions/workflows/ci.yml/badge.svg}[https://github.com/cedric/custom_counter_cache/actions/workflows/ci.yml]
|
|
2
|
+
|
|
3
|
+
== Custom Counter Cache
|
|
4
|
+
|
|
5
|
+
This is a simple approach to creating a custom counter cache in Rails that can be used across multiple models.
|
|
6
|
+
|
|
7
|
+
=== Requirements
|
|
8
|
+
|
|
9
|
+
* Ruby >= 3.3
|
|
10
|
+
* Rails (Active Record) >= 8.0
|
|
11
|
+
|
|
12
|
+
CI covers the Ruby and Rails versions that are not yet end-of-life.
|
|
13
|
+
|
|
14
|
+
=== Installation
|
|
15
|
+
|
|
16
|
+
Add the following to your Gemfile:
|
|
17
|
+
|
|
18
|
+
gem 'custom_counter_cache'
|
|
19
|
+
|
|
20
|
+
== Defining a counter
|
|
21
|
+
|
|
22
|
+
The block calculates the counter's value from its owner. It is called whenever a model that
|
|
23
|
+
declares +update_counter_cache+ (see below) changes.
|
|
24
|
+
|
|
25
|
+
include CustomCounterCache::Model
|
|
26
|
+
define_counter_cache :articles_count do |user|
|
|
27
|
+
user.articles.where(state: 'published').count
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
Pass <tt>touch: true</tt> (or a column name, e.g. <tt>touch: :counted_at</tt>) to also set that
|
|
31
|
+
timestamp, <tt>updated_at</tt> for +true+, on every recount. A column counter is written in the same
|
|
32
|
+
UPDATE. No callbacks run.
|
|
33
|
+
|
|
34
|
+
Where the value is kept depends on the counter: a column of the same name, a row in a shared
|
|
35
|
+
counters table, or a cache store (<tt>store: :cache</tt>). See Choosing storage below.
|
|
36
|
+
|
|
37
|
+
== Triggering recounts
|
|
38
|
+
|
|
39
|
+
Declare which models change the counter with +update_counter_cache+, naming a +belongs_to+
|
|
40
|
+
association and the counter on its owner. It defines the after_create, after_update and
|
|
41
|
+
after_destroy callbacks (plus after_restore for Paranoia, see Soft deletes).
|
|
42
|
+
|
|
43
|
+
include CustomCounterCache::Model
|
|
44
|
+
update_counter_cache :user, :articles_count, if: -> (article) { article.saved_change_to_state? }
|
|
45
|
+
|
|
46
|
+
These callbacks can be added to any number of models that might need to change the counter.
|
|
47
|
+
When a record moves to another owner, both the old and the new owner are recounted.
|
|
48
|
+
|
|
49
|
+
Options:
|
|
50
|
+
|
|
51
|
+
* <tt>:if</tt>, <tt>:unless</tt>: limit when the create and update callbacks run. The callbacks
|
|
52
|
+
run after the record is saved, so use <tt>saved_change_to_state?</tt> rather than
|
|
53
|
+
<tt>state_changed?</tt>, which is always false by then. They don't apply to destroy or restore:
|
|
54
|
+
those have no saved changes, and an extra recount is never wrong.
|
|
55
|
+
* <tt>on_change:</tt>: an attribute or an array. Recount on update only when one of them changed;
|
|
56
|
+
prefer it to +:if+ for plain attribute checks. Create and destroy always recount. It combines
|
|
57
|
+
with +:if+ and +:unless+ (both must allow the update).
|
|
58
|
+
* <tt>only:</tt>, <tt>except:</tt>: limit the events, any of <tt>:create</tt>, <tt>:update</tt>,
|
|
59
|
+
<tt>:destroy</tt> and <tt>:restore</tt>, e.g. <tt>only: [:create, :destroy]</tt> or
|
|
60
|
+
<tt>except: [:destroy]</tt>.
|
|
61
|
+
* <tt>:prepend</tt>: prepend the callbacks instead of appending them.
|
|
62
|
+
* <tt>recount: :later</tt>: recount in a background job instead of right after commit. See
|
|
63
|
+
When recounts run.
|
|
64
|
+
|
|
65
|
+
For example, to recount only when the state changes:
|
|
66
|
+
|
|
67
|
+
update_counter_cache :user, :articles_count, on_change: [:state]
|
|
68
|
+
|
|
69
|
+
The association's key columns (plus the type column for a polymorphic one) are always watched
|
|
70
|
+
by +on_change+, so moving the record to another owner recounts both owners.
|
|
71
|
+
|
|
72
|
+
== Grandparent counters
|
|
73
|
+
|
|
74
|
+
Pass a path of +belongs_to+ associations to recount an owner further up. For a
|
|
75
|
+
<tt>comments_count</tt> on User across all of a user's articles:
|
|
76
|
+
|
|
77
|
+
class User < ApplicationRecord
|
|
78
|
+
include CustomCounterCache::Model
|
|
79
|
+
has_many :articles
|
|
80
|
+
has_many :comments, through: :articles
|
|
81
|
+
define_counter_cache(:comments_count) { |user| user.comments.count }
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
class Comment < ApplicationRecord
|
|
85
|
+
include CustomCounterCache::Model
|
|
86
|
+
belongs_to :article
|
|
87
|
+
update_counter_cache [:article, :user], :comments_count
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
class Article < ApplicationRecord
|
|
91
|
+
include CustomCounterCache::Model
|
|
92
|
+
belongs_to :user
|
|
93
|
+
update_counter_cache :user, :comments_count # an article moving users changes both counts
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
Moving a comment recounts the old and the new user, once if they're the same user. The second
|
|
97
|
+
declaration is needed because an Article changing user also changes both users' counts, and only
|
|
98
|
+
Article's own callbacks see that. A step after a polymorphic one is skipped for records whose
|
|
99
|
+
class has no such association; any other missing or non-+belongs_to+ step raises ArgumentError.
|
|
100
|
+
|
|
101
|
+
== Soft deletes
|
|
102
|
+
|
|
103
|
+
Discard soft-deletes with an ordinary update, so the callbacks already fire: filter with
|
|
104
|
+
<tt>.kept</tt> in the counter block, and if you use <tt>on_change:</tt> include
|
|
105
|
+
<tt>:discarded_at</tt> in it, or discarding won't recount.
|
|
106
|
+
|
|
107
|
+
define_counter_cache(:pages_count) { |notebook| notebook.pages.kept.count }
|
|
108
|
+
update_counter_cache :notebook, :pages_count, on_change: [:state, :discarded_at]
|
|
109
|
+
|
|
110
|
+
Paranoia's +destroy+ runs the destroy callbacks, and +restore+ is recounted too, provided
|
|
111
|
+
+acts_as_paranoid+ is declared before +update_counter_cache+. Skip it with <tt>except: [:restore]</tt>
|
|
112
|
+
(<tt>:restore</tt> is also accepted by <tt>only:</tt>).
|
|
113
|
+
|
|
114
|
+
class Parcel < ApplicationRecord
|
|
115
|
+
acts_as_paranoid
|
|
116
|
+
belongs_to :crate
|
|
117
|
+
update_counter_cache :crate, :parcels_count
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
== Batching and skipping
|
|
121
|
+
|
|
122
|
+
Every triggering save recounts its owner. For bulk work, batch the recounts so each owner and
|
|
123
|
+
counter is recounted once, when the block ends (or when its transaction commits):
|
|
124
|
+
|
|
125
|
+
CustomCounterCache.batch do
|
|
126
|
+
rows.each { |row| article.comments.create!(row) } # one recount of article.comments_count
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
Batches nest (the outermost one flushes) and still flush if the block raises, since a recount
|
|
130
|
+
reflects whatever is in the database. To skip recounts entirely, e.g. for an import you'll
|
|
131
|
+
recount afterwards:
|
|
132
|
+
|
|
133
|
+
CustomCounterCache.skip { import_comments }
|
|
134
|
+
|
|
135
|
+
Both are per thread (and per fiber).
|
|
136
|
+
|
|
137
|
+
== When recounts run
|
|
138
|
+
|
|
139
|
+
Recounts triggered by these callbacks run after the saving transaction commits, once per owner
|
|
140
|
+
and counter per transaction, and not at all if it rolls back. Each takes a short lock on the
|
|
141
|
+
owner's row while it counts, so two saves committing at once can't leave a stale value: the
|
|
142
|
+
later recount always counts after the earlier commit. (Counting inside the saving transaction
|
|
143
|
+
can't see a concurrent save's uncommitted child, so one of them would overwrite the other.)
|
|
144
|
+
|
|
145
|
+
The counter therefore changes when the transaction commits, not when the child is saved. Inside
|
|
146
|
+
a transaction, call the update method yourself if you need the new value straight away:
|
|
147
|
+
|
|
148
|
+
Article.transaction do
|
|
149
|
+
user.articles.create!(attrs)
|
|
150
|
+
user.update_articles_count # recounts now; update_* and recount_counter_caches never wait
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
If a recount raises after commit, the save has already succeeded and the count can be rebuilt, so
|
|
154
|
+
the error doesn't propagate to the caller or stop the other recounts queued for that commit. It's
|
|
155
|
+
passed to <tt>Rails.error.unexpected</tt>, which raises in development and test (with
|
|
156
|
+
<tt>consider_all_requests_local</tt>, as by default) and in production reports it to your error
|
|
157
|
+
tracker with the owner and counter in the context. It's also logged. Calling
|
|
158
|
+
<tt>update_<name></tt> or <tt>recount_counter_caches</tt> yourself raises as usual, and so does
|
|
159
|
+
<tt>CustomCounterCache::RecountJob</tt>, so your queue's retries apply.
|
|
160
|
+
|
|
161
|
+
To recount in a background job instead, enqueued after commit:
|
|
162
|
+
|
|
163
|
+
update_counter_cache :user, :articles_count, recount: :later # CustomCounterCache::RecountJob (Active Job)
|
|
164
|
+
|
|
165
|
+
To pick a queue, <tt>CustomCounterCache::RecountJob.queue_as :low</tt>. Rails' transactional
|
|
166
|
+
tests commit each save within the test transaction, so recounts happen there as they do in
|
|
167
|
+
production.
|
|
168
|
+
|
|
169
|
+
== Choosing storage
|
|
170
|
+
|
|
171
|
+
[Column] Add a column with the counter's name. Fastest to read, and you can sort and
|
|
172
|
+
filter by it in SQL. Needs a migration per counter.
|
|
173
|
+
[Counters table] Used automatically when there's no column. One shared table, no migration per
|
|
174
|
+
counter; preload with <tt>includes(:counters)</tt>. See The counters table.
|
|
175
|
+
[Cache] <tt>define_counter_cache :x, store: :cache, expires_in: 12.hours</tt>. Kept in
|
|
176
|
+
<tt>CustomCounterCache.cache_store</tt> (defaults to Rails.cache: Redis,
|
|
177
|
+
Memcached, Solid Cache...). Reads compute on a miss; a child change deletes the
|
|
178
|
+
key after commit instead of recounting, so writes are cheap and never lock the
|
|
179
|
+
owner's row. Destroying the owner deletes its keys. Not visible to SQL, and a
|
|
180
|
+
delete racing a concurrent read can leave a stale value for up to +expires_in+.
|
|
181
|
+
|
|
182
|
+
Column or counters table is decided when the counter is read or written, not when the model
|
|
183
|
+
loads, so defining counters never touches the database, and a column added later is picked up.
|
|
184
|
+
|
|
185
|
+
To use a column, add one:
|
|
186
|
+
|
|
187
|
+
def change
|
|
188
|
+
add_column :users, :articles_count, :integer, default: 0, null: false
|
|
189
|
+
end
|
|
190
|
+
|
|
191
|
+
== The counters table
|
|
192
|
+
|
|
193
|
+
To store counters in a single shared table instead, use this migration:
|
|
194
|
+
|
|
195
|
+
create_table :counters do |t|
|
|
196
|
+
t.references :countable, polymorphic: true
|
|
197
|
+
t.string :key, null: false
|
|
198
|
+
t.integer :value, null: false, default: 0
|
|
199
|
+
t.timestamps
|
|
200
|
+
end
|
|
201
|
+
add_index :counters, [ :countable_id, :countable_type, :key ], unique: true
|
|
202
|
+
|
|
203
|
+
The table holds whole numbers. A block result of +nil+ is stored as 0, and whole values such as
|
|
204
|
+
<tt>4.0</tt> or <tt>BigDecimal('4')</tt> are converted, but a fractional result such as an average
|
|
205
|
+
raises ArgumentError rather than being truncated: give that counter a column of a suitable type
|
|
206
|
+
(e.g. decimal) or use <tt>store: :cache</tt>.
|
|
207
|
+
|
|
208
|
+
+countable_id+ must match your models' primary key type: for string or UUID keys, use
|
|
209
|
+
<tt>t.references :countable, polymorphic: true, type: :uuid</tt> (or <tt>:string</tt>). A model
|
|
210
|
+
with a composite primary key can't use this table, since +countable_id+ holds one value: give its
|
|
211
|
+
counters a column or <tt>store: :cache</tt> (it raises ArgumentError otherwise). Composite keys
|
|
212
|
+
work everywhere else, including composite foreign keys on the +belongs_to+ side.
|
|
213
|
+
|
|
214
|
+
Here is the example model to go with:
|
|
215
|
+
|
|
216
|
+
class Counter < ActiveRecord::Base
|
|
217
|
+
belongs_to :countable, polymorphic: true
|
|
218
|
+
validates :countable, presence: true
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
To use a different model name (for example if +Counter+ is already taken), set
|
|
222
|
+
<tt>CustomCounterCache.counter_class_name</tt> (see Configuration).
|
|
223
|
+
|
|
224
|
+
When a record is destroyed, its counter rows are removed with a single DELETE, without loading
|
|
225
|
+
them or running Counter's callbacks. Do not add <tt>dependent: :destroy</tt> to the +belongs_to+
|
|
226
|
+
above: on a +belongs_to+ it means "destroying this Counter also destroys its owner".
|
|
227
|
+
|
|
228
|
+
== Backfilling and repairing
|
|
229
|
+
|
|
230
|
+
To backfill your counters, or repair them later, recount every record from the console or a migration:
|
|
231
|
+
|
|
232
|
+
User.recount_counter_caches
|
|
233
|
+
|
|
234
|
+
or only some counters and records:
|
|
235
|
+
|
|
236
|
+
User.recount_counter_caches(:articles_count, scope: User.where(id: 1..1000), batch_size: 500)
|
|
237
|
+
|
|
238
|
+
It returns the number of records processed. The same is available as a rake task:
|
|
239
|
+
|
|
240
|
+
bin/rails custom_counter_cache:recount MODEL=User COUNTERS=articles_count
|
|
241
|
+
|
|
242
|
+
+COUNTERS+ (comma-separated, default all) and +BATCH_SIZE+ (default 1000) are optional.
|
|
243
|
+
|
|
244
|
+
Callbacks can't see +update_all+, +delete_all+, +insert_all+ or SQL imports, so counts drift after
|
|
245
|
+
them. Run the recount periodically, or after an import, to repair that. It calls +update_<name>+
|
|
246
|
+
directly, so it also works inside <tt>CustomCounterCache.skip { }</tt>.
|
|
247
|
+
|
|
248
|
+
In a migration that also adds the column, call <tt>User.reset_column_information</tt> first so the
|
|
249
|
+
backfill writes to the new column.
|
|
250
|
+
|
|
251
|
+
== Configuration
|
|
252
|
+
|
|
253
|
+
Set these in an initializer (<tt>config/initializers/custom_counter_cache.rb</tt>):
|
|
254
|
+
|
|
255
|
+
* <tt>CustomCounterCache.counter_class_name</tt>: the model behind the counters table. Default
|
|
256
|
+
<tt>'Counter'</tt>. Must be set before your models load.
|
|
257
|
+
* <tt>CustomCounterCache.cache_store</tt>: the store for <tt>store: :cache</tt> counters. Defaults
|
|
258
|
+
to Rails.cache.
|
|
259
|
+
|
|
260
|
+
CustomCounterCache.counter_class_name = 'CounterCache'
|
|
261
|
+
CustomCounterCache.cache_store = ActiveSupport::Cache::MemoryStore.new
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
require 'active_support/isolated_execution_state'
|
|
2
|
+
|
|
3
|
+
module CustomCounterCache
|
|
4
|
+
# Recounts each owner and counter once after the outermost batch ends (after commit, if inside a transaction).
|
|
5
|
+
def self.batch(&block)
|
|
6
|
+
Dispatcher.batch(&block)
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
# Drops recounts inside the block, e.g. for an import followed by recount_counter_caches.
|
|
10
|
+
def self.skip(&block)
|
|
11
|
+
Dispatcher.skip(&block)
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
# Single path from update_counter_cache's callbacks to an owner's recount.
|
|
15
|
+
module Dispatcher
|
|
16
|
+
# Always after commit: inside the saving transaction a recount can't see a concurrent save's
|
|
17
|
+
# uncommitted child, and the later of the two overwrites the count with a stale one.
|
|
18
|
+
def self.recount(owner, name, timing = :after_commit)
|
|
19
|
+
return if state[:skip]
|
|
20
|
+
# A cached counter is invalidated after commit, never recounted in place.
|
|
21
|
+
timing = :invalidate if owner.class.custom_counter_cache_storage(name) == :cache
|
|
22
|
+
# Keep the first instance seen, so loaded counters on it stay current.
|
|
23
|
+
return state[:batch][[owner.class, owner.id, name]] ||= [owner, timing] if state[:batch]
|
|
24
|
+
defer(owner, name, timing)
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# Serializes recounts of one owner on its row, so each counts after any concurrent commit.
|
|
28
|
+
# Locks by query rather than with_lock, which would reload the owner and drop its unsaved changes.
|
|
29
|
+
def self.locked_recount(owner, name)
|
|
30
|
+
klass = owner.class
|
|
31
|
+
klass.transaction do
|
|
32
|
+
# Owner deleted meanwhile: nothing left to recount.
|
|
33
|
+
next unless klass.where(CustomCounterCache.key_conditions(klass.primary_key, owner.id)).lock.pick(*Array(klass.primary_key).first(1))
|
|
34
|
+
owner.public_send("update_#{name}")
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def self.batch
|
|
39
|
+
outermost = state[:batch].nil?
|
|
40
|
+
state[:batch] ||= {}
|
|
41
|
+
yield
|
|
42
|
+
ensure
|
|
43
|
+
# A recount reflects whatever is in the database, so flushing after an exception is still correct.
|
|
44
|
+
if outermost
|
|
45
|
+
pending = state.delete(:batch)
|
|
46
|
+
pending.each { |(_, _, name), (owner, timing)| recount(owner, name, timing) }
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def self.skip
|
|
51
|
+
skipping = state[:skip]
|
|
52
|
+
state[:skip] = true
|
|
53
|
+
yield
|
|
54
|
+
ensure
|
|
55
|
+
state[:skip] = skipping
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
# Once per owner and counter per transaction. A recount reads committed state, so running it late is still correct.
|
|
59
|
+
def self.defer(owner, name, timing)
|
|
60
|
+
key = [owner.class, owner.id, name, timing]
|
|
61
|
+
deferred = state[:deferred] ||= {}
|
|
62
|
+
return if deferred.key?(key)
|
|
63
|
+
|
|
64
|
+
transaction = owner.class.current_transaction
|
|
65
|
+
deferred[key] = true if transaction.open?
|
|
66
|
+
# A rolled-back savepoint drops its after_commit, so forget the key and let a later save schedule it again.
|
|
67
|
+
transaction.after_rollback { deferred.delete(key) }
|
|
68
|
+
transaction.after_commit do
|
|
69
|
+
deferred.delete(key)
|
|
70
|
+
perform(owner, name, timing)
|
|
71
|
+
rescue StandardError => error
|
|
72
|
+
failed(error, owner, name)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# The save has committed and the count can be rebuilt, so don't make the save look failed or
|
|
77
|
+
# stop the other recounts queued for this commit. unexpected still raises in development and test.
|
|
78
|
+
def self.failed(error, owner, name)
|
|
79
|
+
context = { owner: owner.class.name, owner_id: owner.id, counter: name.to_s }
|
|
80
|
+
owner.class.logger&.error("custom_counter_cache: recount of #{context[:owner]} #{context[:owner_id].inspect} " \
|
|
81
|
+
"#{context[:counter]} failed: #{error.class}: #{error.message}")
|
|
82
|
+
ActiveSupport.error_reporter.unexpected(error, context: context, source: 'custom_counter_cache')
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
def self.perform(owner, name, timing)
|
|
86
|
+
case timing
|
|
87
|
+
when :later then RecountJob.perform_later(owner.class.name, owner.id, name.to_s)
|
|
88
|
+
when :invalidate
|
|
89
|
+
CustomCounterCache.cache_store.delete(CustomCounterCache.cache_key(owner, name))
|
|
90
|
+
touch_column = owner.class.custom_counter_cache_touch_column(name)
|
|
91
|
+
# A cascade destroy invalidates after the owner is gone; there's nothing left to touch.
|
|
92
|
+
owner.update_column(touch_column, Time.current) if touch_column && !owner.destroyed?
|
|
93
|
+
else locked_recount(owner, name)
|
|
94
|
+
end
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
def self.state
|
|
98
|
+
ActiveSupport::IsolatedExecutionState[:custom_counter_cache] ||= {}
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
end
|