moderate 1.0.0.beta1 → 1.0.0.beta2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +39 -1
- data/README.md +2 -0
- data/context7.json +4 -0
- data/docs/images/moderate-user-report-block-actions.webp +0 -0
- data/lib/moderate/configuration.rb +14 -0
- data/lib/moderate/jobs/classify_job.rb +4 -0
- data/lib/moderate/models/concerns/content_filterable.rb +84 -25
- data/lib/moderate/models/flag.rb +33 -0
- data/lib/moderate/version.rb +1 -1
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 20519d26bf4bc96f23e6f679b3b175376700280e36f21fffa409230688526b25
|
|
4
|
+
data.tar.gz: 8abc344dcf72c577de12ba70dbf96895f008a041355a75308f815d8b267f5dd1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 65f2ad5ae6dae9a4a1269900e67205cfedaa5f82ff3b87d218ac34dee0e05bdb417fd77289126c47c9cf372b0dfd710fc1fa5d455d1f1f256395d2ec4bc7e79c
|
|
7
|
+
data.tar.gz: 5d3a60adaef05e13b448c1563d3104423c2d94d63ae0b10c7f2432268188a745849ca29df9e99fdb4860670a4c4f3f6fe7eda8fa07a4dd4a8f47169eca97d3f8
|
data/CHANGELOG.md
CHANGED
|
@@ -4,12 +4,50 @@ All notable changes to this project are documented here.
|
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
5
5
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [1.0.0.beta2] - 2026-07-10
|
|
8
|
+
|
|
9
|
+
Second beta on the road to 1.0. Fixes beta1's async-adapter routing bug (the one
|
|
10
|
+
that forced hosts to hand-roll their own enqueue), makes Active Storage
|
|
11
|
+
attachments filterable with zero wiring, and gives `Flag` first-class close
|
|
12
|
+
methods. Drop-in upgrade from beta1: no migrations, no breaking API changes —
|
|
13
|
+
and hosts that worked around the ClassifyJob bug can now delete the workaround.
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
- **Async adapters now actually run in `Moderate::ClassifyJob`.** beta1's `:flag`
|
|
18
|
+
after_commit called `Moderate.classify` inline for *every* adapter — including
|
|
19
|
+
ones declaring `synchronous? == false` — so a remote moderation API ran its
|
|
20
|
+
network call inside the request that saved the content, and `ClassifyJob`
|
|
21
|
+
(whose docs promised this routing) was never enqueued by anything. The concern
|
|
22
|
+
now checks `config.adapter_async?(policy.adapter)` (new public helper, same
|
|
23
|
+
probe as the `:block` validator) and enqueues the job instead; the job re-reads
|
|
24
|
+
the persisted value and files the Flag itself. Hosts that worked around this by
|
|
25
|
+
enqueuing `ClassifyJob` themselves and short-circuiting
|
|
26
|
+
`moderation_field_changed_for_commit?` can delete both workarounds. (#3)
|
|
27
|
+
|
|
28
|
+
### Added
|
|
29
|
+
|
|
30
|
+
- **Active Storage attachments filter out of the box.** `moderates :avatar, with:
|
|
31
|
+
:your_image_adapter, mode: :flag` on a `has_one_attached` model needs zero extra
|
|
32
|
+
wiring now: the concern snapshots `attachment_changes` in a `before_save` (AR
|
|
33
|
+
dirty tracking can't see attachment writes, and Active Storage clears the
|
|
34
|
+
changes before after_commit), consumes the snapshot at commit time, and both
|
|
35
|
+
the concern and `ClassifyJob` treat an unattached `ActiveStorage::Attached`
|
|
36
|
+
proxy as blank (nothing to classify — covers purge-between-enqueue-and-run).
|
|
37
|
+
The three `moderation_field_*` seam overrides remain for richer cases. (#3)
|
|
38
|
+
- **Flag close sugar.** `Flag#action!(note:, by: nil)` / `Flag#dismiss!(note:, by: nil)` —
|
|
39
|
+
model-level closes mirroring `Report#resolve!`/`#dismiss!`, so hosts stop hand-writing
|
|
40
|
+
status updates. `by:` stays nil for automated closes (don't fake a human in the audit
|
|
41
|
+
trail). Canonical automated use: dismiss a pending flag whose flagged content was
|
|
42
|
+
**superseded** (text edited, photo replaced/reverted) — left pending it keeps `flagged?`
|
|
43
|
+
true and mislabels the NEW content in any host UI keyed on it. (#4)
|
|
44
|
+
|
|
7
45
|
## [1.0.0] - unreleased
|
|
8
46
|
|
|
9
47
|
A complete, ground-up rewrite. `moderate` graduates from a single-purpose profanity
|
|
10
48
|
validator (0.1.0) into a full **Trust & Safety** engine for Rails apps with user-generated
|
|
11
49
|
content: report, block, filter, a moderation queue, appeals, and EU DSA / App Store / Google
|
|
12
|
-
Play **aligned** primitives. (First cut
|
|
50
|
+
Play **aligned** primitives. (First cut shipped as `1.0.0.beta1`; second as `1.0.0.beta2`.)
|
|
13
51
|
|
|
14
52
|
> **Breaking:** 1.0 keeps the gem name but is an entirely new API. The 0.x profanity
|
|
15
53
|
> validator (`validates :field, moderate: true`) still loads for backward compatibility
|
data/README.md
CHANGED
|
@@ -11,6 +11,8 @@ Trust & Safety (T&S) is the system within an app that lets users **report** abus
|
|
|
11
11
|
|
|
12
12
|
If you have an app where users can upload / generate content or send messages to each other, you probably need a Trust & Safety system.
|
|
13
13
|
|
|
14
|
+

|
|
15
|
+
|
|
14
16
|
`moderate` ships with mechanisms aligned with the **DSA** (EU Digital Services Act), and also aligned with the **Apple App Store** and Android's **Google Play** directives for User-Generated Content (UGC) in their app stores.
|
|
15
17
|
|
|
16
18
|
## 👨💻 Example
|
data/context7.json
ADDED
|
Binary file
|
|
@@ -230,6 +230,20 @@ module Moderate
|
|
|
230
230
|
@adapters.key?(normalize_name(name))
|
|
231
231
|
end
|
|
232
232
|
|
|
233
|
+
# Is the adapter registered under `name` background-only? Same probe the
|
|
234
|
+
# :block validator uses (`validate_block_mode_adapter!`): an adapter is
|
|
235
|
+
# async ONLY if it explicitly answers `synchronous? == false`; adapters
|
|
236
|
+
# that don't expose the predicate are assumed synchronous — the
|
|
237
|
+
# conservative default that keeps simple adapters working. The :flag
|
|
238
|
+
# enforcement path reads this to decide inline-classify vs routing the
|
|
239
|
+
# work through Moderate::ClassifyJob (see ContentFilterable).
|
|
240
|
+
def adapter_async?(name)
|
|
241
|
+
adapter = adapter_for(name)
|
|
242
|
+
return false if adapter.nil?
|
|
243
|
+
|
|
244
|
+
adapter.respond_to?(:synchronous?) && !adapter.synchronous?
|
|
245
|
+
end
|
|
246
|
+
|
|
233
247
|
# --- Filters --------------------------------------------------------------
|
|
234
248
|
|
|
235
249
|
# Declare a per-field filter policy in the initializer — the twin of
|
|
@@ -149,6 +149,10 @@ module Moderate
|
|
|
149
149
|
# easy to reproduce and keeps the job self-contained).
|
|
150
150
|
def blank?(value)
|
|
151
151
|
return true if value.nil?
|
|
152
|
+
# An ActiveStorage::Attached proxy with nothing attached is "blank" — the
|
|
153
|
+
# attachment may have been purged between enqueue and run, and a plain
|
|
154
|
+
# #empty?/#nil? probe can't see through the proxy.
|
|
155
|
+
return !value.attached? if value.respond_to?(:attached?)
|
|
152
156
|
return value.strip.empty? if value.is_a?(String)
|
|
153
157
|
return value.empty? if value.respond_to?(:empty?)
|
|
154
158
|
|
|
@@ -16,16 +16,30 @@ module Moderate
|
|
|
16
16
|
# Configuration validates that invariant (README: ":block requires a
|
|
17
17
|
# synchronous adapter").
|
|
18
18
|
# :flag — an AFTER_COMMIT side effect. The save SUCCEEDS, then (only if the
|
|
19
|
-
# field actually changed
|
|
20
|
-
# `Moderate::Flag` is filed
|
|
19
|
+
# field actually changed) the value is classified and a
|
|
20
|
+
# `Moderate::Flag` is filed when it trips. HOW it's classified
|
|
21
|
+
# depends on the adapter: a synchronous adapter (the wordlist)
|
|
22
|
+
# classifies inline right here; an ASYNC adapter (`synchronous? ==
|
|
23
|
+
# false` — any remote moderation API) is routed through
|
|
24
|
+
# `Moderate::ClassifyJob`, because blocking network I/O must never
|
|
25
|
+
# run inside the request that saved the content. The job re-reads
|
|
26
|
+
# the current value and files the Flag itself.
|
|
21
27
|
#
|
|
22
28
|
# WHY :flag lives in after_commit and not in a validator (this is the whole
|
|
23
29
|
# reason `:flag` is a `moderates` mode you can't hand-roll with `validates`):
|
|
24
30
|
# validators must be side-effect-free, and a Flag created inside a transaction
|
|
25
31
|
# that later rolls back would silently vanish — you'd think you flagged something
|
|
26
32
|
# you didn't. `after_commit` guarantees the surrounding transaction committed
|
|
27
|
-
# before we write the Flag
|
|
28
|
-
# validator").
|
|
33
|
+
# before we write the Flag (and that a ClassifyJob never races a rollback). See
|
|
34
|
+
# docs/configuration.md ("`:flag` never lives in a validator").
|
|
35
|
+
#
|
|
36
|
+
# ACTIVE STORAGE ATTACHMENTS work out of the box: `moderates :avatar, with:
|
|
37
|
+
# :your_image_adapter, mode: :flag` on a `has_one_attached :avatar` model needs
|
|
38
|
+
# no extra wiring. AR dirty tracking can't see attachment writes — and Active
|
|
39
|
+
# Storage clears `attachment_changes` before after_commit — so the concern
|
|
40
|
+
# snapshots "these filtered attachments changed" in a before_save and consumes
|
|
41
|
+
# the snapshot at commit time. The overridable seam below still exists for
|
|
42
|
+
# richer cases (derived values, non-AS blobs).
|
|
29
43
|
module ContentFilterable
|
|
30
44
|
extend ActiveSupport::Concern
|
|
31
45
|
|
|
@@ -36,6 +50,10 @@ module Moderate
|
|
|
36
50
|
class_attribute :moderation_filtered_fields, instance_writer: false, default: [].freeze
|
|
37
51
|
|
|
38
52
|
validate :moderate_blocked_fields_must_be_allowed
|
|
53
|
+
# Snapshot attachment writes BEFORE Active Storage's own save callbacks
|
|
54
|
+
# clear `attachment_changes` — by after_commit they're gone (see
|
|
55
|
+
# activestorage's Attached::Model). One-shot; consumed + cleared below.
|
|
56
|
+
before_save :moderate_snapshot_attachment_changes
|
|
39
57
|
after_commit :moderate_flag_filtered_fields
|
|
40
58
|
end
|
|
41
59
|
|
|
@@ -61,20 +79,21 @@ module Moderate
|
|
|
61
79
|
next unless policy.block?
|
|
62
80
|
|
|
63
81
|
value = moderation_field_value(field)
|
|
64
|
-
next if value
|
|
82
|
+
next if moderation_value_blank?(value)
|
|
65
83
|
|
|
66
84
|
result = Moderate.classify(value, policy: policy)
|
|
67
85
|
errors.add(field, :objectionable_content) if result.flagged?
|
|
68
86
|
end
|
|
69
87
|
end
|
|
70
88
|
|
|
71
|
-
# :flag enforcement — an after_commit side effect that files a Moderate::Flag
|
|
89
|
+
# :flag enforcement — an after_commit side effect that files a Moderate::Flag
|
|
90
|
+
# (inline for synchronous adapters; via Moderate::ClassifyJob for async ones).
|
|
72
91
|
#
|
|
73
92
|
# We only act when the field actually CHANGED on this commit (re-saving an
|
|
74
93
|
# untouched record must not re-flag it and spam the queue), and we wrap each
|
|
75
|
-
# field in `begin/ensure` so
|
|
76
|
-
# always runs even if classification raises —
|
|
77
|
-
#
|
|
94
|
+
# field in `begin/ensure` so the clean-up hook (`moderation_field_committed`)
|
|
95
|
+
# always runs even if classification raises — that hook is what clears the
|
|
96
|
+
# one-shot attachment snapshot.
|
|
78
97
|
def moderate_flag_filtered_fields
|
|
79
98
|
moderation_filtered_fields.each do |field|
|
|
80
99
|
policy = Moderate.filter_policy_for(self, field)
|
|
@@ -82,8 +101,18 @@ module Moderate
|
|
|
82
101
|
next unless moderation_field_changed_for_commit?(field)
|
|
83
102
|
|
|
84
103
|
begin
|
|
104
|
+
# ASYNC adapters (remote moderation APIs) never classify inline —
|
|
105
|
+
# a network call in the request's after_commit would stall the
|
|
106
|
+
# response for as long as the provider takes. ClassifyJob re-reads
|
|
107
|
+
# the value at run time (so it always classifies what's actually
|
|
108
|
+
# persisted) and files the Flag through the same Flag.flag! builder.
|
|
109
|
+
if Moderate.config.adapter_async?(policy.adapter)
|
|
110
|
+
Moderate::ClassifyJob.perform_later(self, field)
|
|
111
|
+
next
|
|
112
|
+
end
|
|
113
|
+
|
|
85
114
|
value = moderation_field_value(field)
|
|
86
|
-
next if value
|
|
115
|
+
next if moderation_value_blank?(value)
|
|
87
116
|
|
|
88
117
|
result = Moderate.classify(value, policy: policy)
|
|
89
118
|
next unless result.flagged?
|
|
@@ -119,23 +148,29 @@ module Moderate
|
|
|
119
148
|
# --- Overridable field seam -----------------------------------------------
|
|
120
149
|
#
|
|
121
150
|
# These three methods are the seam that lets one concern filter BOTH plain text
|
|
122
|
-
# columns AND non-column content
|
|
123
|
-
#
|
|
124
|
-
#
|
|
125
|
-
|
|
126
|
-
# The value to classify for `field`. Default: the attribute reader
|
|
127
|
-
#
|
|
128
|
-
#
|
|
151
|
+
# columns AND non-column content, without hosts having to re-plumb the common
|
|
152
|
+
# cases. Defaults handle text attributes AND Active Storage attachments; a
|
|
153
|
+
# host overrides them for anything richer (derived values, external blobs).
|
|
154
|
+
|
|
155
|
+
# The value to classify for `field`. Default: the attribute reader — which for
|
|
156
|
+
# a `has_one_attached` field returns the `ActiveStorage::Attached` proxy, the
|
|
157
|
+
# natural input for an image adapter (it can read `.record`, `.blob`,
|
|
158
|
+
# `.variant(...)`, or download bytes as it sees fit).
|
|
129
159
|
def moderation_field_value(field)
|
|
130
160
|
public_send(field)
|
|
131
161
|
end
|
|
132
162
|
|
|
133
|
-
# Did `field` change on the just-committed save?
|
|
134
|
-
#
|
|
135
|
-
#
|
|
136
|
-
#
|
|
137
|
-
#
|
|
163
|
+
# Did `field` change on the just-committed save? Defaults, in order:
|
|
164
|
+
# 1. the attachment snapshot taken in before_save (AR dirty tracking can't
|
|
165
|
+
# see attachment writes, and Active Storage clears `attachment_changes`
|
|
166
|
+
# before after_commit — hence the one-shot snapshot),
|
|
167
|
+
# 2. ActiveRecord's own dirty tracking (`saved_change_to_attribute?`),
|
|
168
|
+
# 3. `true` for a PORO/ActiveModel object with no dirty tracking at all —
|
|
169
|
+
# the conservative assumption.
|
|
170
|
+
# Override for non-attribute content the defaults can't see.
|
|
138
171
|
def moderation_field_changed_for_commit?(field)
|
|
172
|
+
return true if @moderate_changed_attachment_fields&.include?(field.to_s)
|
|
173
|
+
|
|
139
174
|
if respond_to?(:saved_change_to_attribute?)
|
|
140
175
|
saved_change_to_attribute?(field)
|
|
141
176
|
elsif respond_to?(:"saved_change_to_#{field}?")
|
|
@@ -146,10 +181,34 @@ module Moderate
|
|
|
146
181
|
end
|
|
147
182
|
|
|
148
183
|
# Per-field clean-up after the commit-time flag attempt (success OR failure).
|
|
149
|
-
#
|
|
150
|
-
#
|
|
151
|
-
def moderation_field_committed(
|
|
184
|
+
# Consumes the one-shot attachment snapshot; override-and-super if you track
|
|
185
|
+
# extra per-field state of your own.
|
|
186
|
+
def moderation_field_committed(field)
|
|
187
|
+
@moderate_changed_attachment_fields&.delete(field.to_s)
|
|
152
188
|
nil
|
|
153
189
|
end
|
|
190
|
+
|
|
191
|
+
# before_save: record which FILTERED fields have a pending attachment write
|
|
192
|
+
# on this save. `attachment_changes` only exists on Active Storage models —
|
|
193
|
+
# plain models skip straight through.
|
|
194
|
+
def moderate_snapshot_attachment_changes
|
|
195
|
+
return true unless respond_to?(:attachment_changes)
|
|
196
|
+
|
|
197
|
+
moderation_filtered_fields.each do |field|
|
|
198
|
+
next unless attachment_changes.key?(field)
|
|
199
|
+
|
|
200
|
+
(@moderate_changed_attachment_fields ||= Set.new) << field
|
|
201
|
+
end
|
|
202
|
+
true # never halt the save chain
|
|
203
|
+
end
|
|
204
|
+
|
|
205
|
+
# Blank check that can see through an `ActiveStorage::Attached` proxy — an
|
|
206
|
+
# attachment with nothing attached must read as "nothing to classify"
|
|
207
|
+
# (Object#blank? can't tell: the proxy is truthy and has no #empty?).
|
|
208
|
+
def moderation_value_blank?(value)
|
|
209
|
+
return !value.attached? if value.respond_to?(:attached?)
|
|
210
|
+
|
|
211
|
+
value.blank?
|
|
212
|
+
end
|
|
154
213
|
end
|
|
155
214
|
end
|
data/lib/moderate/models/flag.rb
CHANGED
|
@@ -97,6 +97,26 @@ module Moderate
|
|
|
97
97
|
status.in?(%w[actioned dismissed])
|
|
98
98
|
end
|
|
99
99
|
|
|
100
|
+
# Close this flag as ACTIONED — someone (or an automated policy) took
|
|
101
|
+
# action on the flagged content. Model-level sugar mirroring
|
|
102
|
+
# Report#resolve!/#dismiss! so hosts stop hand-writing status updates.
|
|
103
|
+
# `by:` is the reviewing user when a human decided; leave it nil for
|
|
104
|
+
# automated closes — recording a human who never looked would poison the
|
|
105
|
+
# audit trail (DSA statements of reasons read these rows).
|
|
106
|
+
def action!(note:, by: nil)
|
|
107
|
+
close!(new_status: "actioned", note: note, by: by)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
# Close this flag with NO action taken. The canonical automated use is
|
|
111
|
+
# SUPERSEDED content: when the flagged field changes (text edited, photo
|
|
112
|
+
# replaced/reverted), a still-pending flag is about content that is no
|
|
113
|
+
# longer live — leaving it pending keeps `flagged?` true and mislabels
|
|
114
|
+
# the NEW content in any host UI keyed on it. Hosts should dismiss with a
|
|
115
|
+
# note saying what superseded it.
|
|
116
|
+
def dismiss!(note:, by: nil)
|
|
117
|
+
close!(new_status: "dismissed", note: note, by: by)
|
|
118
|
+
end
|
|
119
|
+
|
|
100
120
|
# A label for the flagged thing in the queue. Asks the flaggable for its own
|
|
101
121
|
# `moderation_label` (Moderate::Reportable interface); falls back to a generic
|
|
102
122
|
# "Type id" string for content that doesn't implement it.
|
|
@@ -108,6 +128,19 @@ module Moderate
|
|
|
108
128
|
|
|
109
129
|
private
|
|
110
130
|
|
|
131
|
+
# Shared close path for action!/dismiss!. update! (not update_columns) on
|
|
132
|
+
# purpose: the `resolution_note presence if closed?` validation is the
|
|
133
|
+
# guard that every closed flag explains itself — sugar that skipped it
|
|
134
|
+
# would defeat the reason the sugar exists.
|
|
135
|
+
def close!(new_status:, note:, by:)
|
|
136
|
+
update!(
|
|
137
|
+
status: new_status,
|
|
138
|
+
resolution_note: note,
|
|
139
|
+
reviewed_by: by,
|
|
140
|
+
reviewed_at: Time.current
|
|
141
|
+
)
|
|
142
|
+
end
|
|
143
|
+
|
|
111
144
|
# See the before_save comment: keep the NOT-NULL JSON columns non-null on MySQL.
|
|
112
145
|
def default_json_columns
|
|
113
146
|
self.categories ||= []
|
data/lib/moderate/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: moderate
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.0.
|
|
4
|
+
version: 1.0.0.beta2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- rameerez
|
|
8
8
|
bindir: exe
|
|
9
9
|
cert_chain: []
|
|
10
|
-
date: 2026-
|
|
10
|
+
date: 2026-07-10 00:00:00.000000000 Z
|
|
11
11
|
dependencies:
|
|
12
12
|
- !ruby/object:Gem::Dependency
|
|
13
13
|
name: activerecord
|
|
@@ -136,9 +136,11 @@ files:
|
|
|
136
136
|
- config/moderate/blocklists/en.yml
|
|
137
137
|
- config/moderate/blocklists/es.yml
|
|
138
138
|
- config/routes.rb
|
|
139
|
+
- context7.json
|
|
139
140
|
- docs/compliance.md
|
|
140
141
|
- docs/configuration.md
|
|
141
142
|
- docs/dsa-notice-form.md
|
|
143
|
+
- docs/images/moderate-user-report-block-actions.webp
|
|
142
144
|
- docs/madmin.md
|
|
143
145
|
- docs/notifications.md
|
|
144
146
|
- examples/aws_rekognition_adapter.rb
|