moderate 0.1.0 → 1.0.0.beta2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (67) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +8 -0
  3. data/.simplecov +62 -0
  4. data/AGENTS.md +7 -0
  5. data/Appraisals +16 -0
  6. data/CHANGELOG.md +109 -1
  7. data/CLAUDE.md +7 -0
  8. data/README.md +378 -29
  9. data/Rakefile +28 -2
  10. data/app/controllers/concerns/moderate/moderation.rb +161 -0
  11. data/app/controllers/moderate/appeals_controller.rb +190 -0
  12. data/app/controllers/moderate/application_controller.rb +45 -0
  13. data/app/controllers/moderate/notices_controller.rb +382 -0
  14. data/app/controllers/moderate/transparency_reports_controller.rb +30 -0
  15. data/app/helpers/moderate/engine_helper.rb +151 -0
  16. data/app/views/moderate/appeals/new.html.erb +78 -0
  17. data/app/views/moderate/notices/new.html.erb +255 -0
  18. data/app/views/moderate/transparency_reports/_summary_card.html.erb +20 -0
  19. data/app/views/moderate/transparency_reports/show.html.erb +52 -0
  20. data/config/moderate/blocklists/en.yml +81 -0
  21. data/config/moderate/blocklists/es.yml +40 -0
  22. data/config/routes.rb +36 -0
  23. data/context7.json +4 -0
  24. data/docs/compliance.md +178 -0
  25. data/docs/configuration.md +326 -0
  26. data/docs/dsa-notice-form.md +371 -0
  27. data/docs/images/moderate-user-report-block-actions.webp +0 -0
  28. data/docs/madmin.md +490 -0
  29. data/docs/notifications.md +363 -0
  30. data/examples/aws_rekognition_adapter.rb +140 -0
  31. data/examples/openai_moderation_adapter.rb +111 -0
  32. data/gemfiles/rails_7.1.gemfile +36 -0
  33. data/gemfiles/rails_7.2.gemfile +36 -0
  34. data/gemfiles/rails_8.1.gemfile +36 -0
  35. data/lib/generators/moderate/install_generator.rb +56 -0
  36. data/lib/generators/moderate/templates/create_moderate_tables.rb.erb +237 -0
  37. data/lib/generators/moderate/templates/initializer.rb +198 -0
  38. data/lib/generators/moderate/views_generator.rb +63 -0
  39. data/lib/moderate/configuration.rb +355 -0
  40. data/lib/moderate/engine.rb +138 -0
  41. data/lib/moderate/errors.rb +26 -0
  42. data/lib/moderate/event.rb +75 -0
  43. data/lib/moderate/filters/base.rb +126 -0
  44. data/lib/moderate/filters/wordlist.rb +255 -0
  45. data/lib/moderate/jobs/classify_job.rb +162 -0
  46. data/lib/moderate/label.rb +111 -0
  47. data/lib/moderate/macros.rb +90 -0
  48. data/lib/moderate/models/appeal.rb +154 -0
  49. data/lib/moderate/models/application_record.rb +31 -0
  50. data/lib/moderate/models/block.rb +203 -0
  51. data/lib/moderate/models/concerns/actor.rb +174 -0
  52. data/lib/moderate/models/concerns/content_filterable.rb +214 -0
  53. data/lib/moderate/models/concerns/reportable.rb +282 -0
  54. data/lib/moderate/models/flag.rb +169 -0
  55. data/lib/moderate/models/report.rb +620 -0
  56. data/lib/moderate/result.rb +176 -0
  57. data/lib/moderate/services/intake_appeal.rb +89 -0
  58. data/lib/moderate/services/intake_notice.rb +132 -0
  59. data/lib/moderate/services/intake_report.rb +132 -0
  60. data/lib/moderate/services/resolve_appeal.rb +134 -0
  61. data/lib/moderate/services/resolve_flag.rb +101 -0
  62. data/lib/moderate/services/resolve_report.rb +291 -0
  63. data/lib/moderate/version.rb +1 -1
  64. data/lib/moderate.rb +365 -18
  65. data/log/development.log +0 -0
  66. data/log/test.log +0 -0
  67. metadata +156 -15
data/README.md CHANGED
@@ -1,71 +1,420 @@
1
- # 👮‍♂️ `moderate` - Moderate and block bad words from your Rails app
1
+ # 🛡️ `moderate` - Let your Rails users report content and block each other (Trust & Safety)
2
2
 
3
- `moderate` is a Ruby gem that moderates user-generated text content by adding a simple validation to block bad words in any text field.
3
+ [![Gem Version](https://badge.fury.io/rb/moderate.svg)](https://badge.fury.io/rb/moderate) [![Build Status](https://github.com/rameerez/moderate/workflows/Tests/badge.svg)](https://github.com/rameerez/moderate/actions)
4
4
 
5
- Simply add this to your model:
5
+ > [!TIP]
6
+ > **🚀 Ship your next Rails app 10x faster!** I've built **[RailsFast](https://railsfast.com/?ref=moderate)**, a production-ready Rails boilerplate template that comes with everything you need to launch a software business in days, not weeks. Go [check it out](https://railsfast.com/?ref=moderate)!
7
+
8
+ `moderate` gives your Rails app a complete **Trust & Safety** system.
9
+
10
+ Trust & Safety (T&S) is the system within an app that lets users **report** abusive content, **block** each other, **filter** objectionable text and images before they're posted (profanity, bad words, NSFW / nudity, etc.), and run a **moderation queue** your admins actually use. It also allows you to easily plug in automated AI moderation systems like **OpenAI Moderation** or **AWS Rekognition** to quickly filter, flag and/or automatically block harmful content (text or image).
11
+
12
+ If you have an app where users can upload / generate content or send messages to each other, you probably need a Trust & Safety system.
13
+
14
+ ![Moderate gem sample use case for reporting profiles and blocking users](/docs/images/moderate-user-report-block-actions.webp)
15
+
16
+ `moderate` ships with mechanisms aligned with the **DSA** (EU Digital Services Act), and also aligned with the **Apple App Store** and Android's **Google Play** directives for User-Generated Content (UGC) in their app stores.
17
+
18
+ ## 👨‍💻 Example
19
+
20
+ `moderate` reads like plain English. Make any model reportable:
21
+
22
+ ```ruby
23
+ class Comment < ApplicationRecord
24
+ has_reportable_content
25
+ end
26
+ ```
27
+
28
+ Let any user block another:
29
+
30
+ ```ruby
31
+ current_user.block!(@other_user)
32
+ current_user.blocks?(@other_user) # => true
33
+ ```
34
+
35
+ Filter content before it's posted — the zero-setup wordlist, or a real classifier like OpenAI moderation:
6
36
 
7
37
  ```ruby
8
- validates :text_field, moderate: true
38
+ # config/initializers/moderate.rb — wire up OpenAI moderation once (text AND images)
39
+ config.register_adapter :openai, OpenAIModerationAdapter.new
9
40
  ```
10
41
 
11
- That's it! You're done. `moderate` will work seamlessly with your existing validations and error messages.
42
+ ```ruby
43
+ class Message < ApplicationRecord
44
+ # Run every DM through OpenAI, but never block mid-conversation: `:flag` lets the
45
+ # message send, then classifies it in a background job and drops anything harmful
46
+ # into the moderation queue for review.
47
+ moderates :body, mode: :flag, with: :openai
48
+ end
49
+ ```
12
50
 
13
- > [!WARNING]
14
- > This gem is under development. It currently only supports a limited set of English profanity words. Word matching is very basic now, and it may be prone to false positives, and false negatives. I use it for very simple things like preventing new submissions if they contain bad words, but the gem can be improved for more complex use cases and sophisticated matching and content moderation. Please consider contributing if you can improve the gem, or have good ideas for additional features.
51
+ No API keys to start? Drop the `with:` and you get the built-in, zero-dependency `:wordlist` (a fast, multilingual profanity block) — same one-line API.
15
52
 
16
- # Why
53
+ And give admins a real queue to act on:
17
54
 
18
- Any text field where users can input text may be a place where bad words can be used. This gem blocks records from being created if they contain bad words, profanity, naughty / obscene words, etc.
55
+ ```ruby
56
+ Moderate::Report.pending # everything awaiting a decision
57
+ report.resolve!(by: current_user, remove_content: true, ban_user: true, note: "Hate speech")
58
+ ```
19
59
 
20
- It's good for Rails applications where you need to maintain a clean and respectful environment in comments, posts, or any other user input.
60
+ That's the whole idea: **the messy, legally-loaded plumbing every social/UGC app needs (report, block, filter, moderate, appeal, comply) as one coherent Ruby gem** instead of scattered, half-finished, store-rejecting DIY code.
21
61
 
22
- # How
62
+ > [!NOTE]
63
+ > `moderate` is **UI-agnostic by design**: most of a Trust & Safety system lives in *admin* surfaces, so the gem ships the **primitives** (models, services, helpers, controller concerns) and lets you **build your own UI**. It plugs into [`madmin`](https://github.com/excid3/madmin) (or any admin system) in minutes; see [Admin & moderation queue](#-admin--the-moderation-queue).
23
64
 
24
- `moderate` currently downloads a list of ~1k English profanity words from the [google-profanity-words](https://github.com/coffee-and-fun/google-profanity-words) repository and caches it in your Rails app's tmp directory.
65
+ ---
25
66
 
26
- ## Installation
67
+ ## Quickstart
27
68
 
28
- Add this line to your application's Gemfile:
69
+ Add the gem:
29
70
 
30
71
  ```ruby
31
- gem 'moderate'
72
+ gem "moderate"
32
73
  ```
33
74
 
34
- And then execute:
75
+ Install it (creates the migration + an initializer):
35
76
 
36
77
  ```bash
37
78
  bundle install
79
+ rails generate moderate:install
80
+ rails db:migrate
81
+ ```
82
+
83
+ Tell `moderate` who your users are, and make a model reportable:
84
+
85
+ ```ruby
86
+ # config/initializers/moderate.rb
87
+ Moderate.configure do |config|
88
+ config.user_class = "User"
89
+ end
90
+ ```
91
+
92
+ ```ruby
93
+ class User < ApplicationRecord
94
+ has_reporting_and_blocking # can report, block, be blocked, be banned
95
+ end
96
+
97
+ class Post < ApplicationRecord
98
+ has_reportable_content # users can report it
99
+ moderates :body, mode: :block # …and profanity is rejected on save — zero-setup built-in wordlist
100
+ end
101
+ ```
102
+
103
+ That's it. You now have reporting, blocking, filtering, and a moderation queue. Everything below is detail.
104
+
105
+ ---
106
+
107
+ ## Why this gem exists
108
+
109
+ Every app with user-generated content eventually faces the same wall. A user posts something vile, another user wants them gone, Apple rejects your build for "no way to report objectionable content," and a Spanish lawyer emails you about the Digital Services Act. So you start bolting on a `reports` table, a `blocks` table, a profanity regex, an admin page, a "notify the reporter" email… and it's suddenly a sprawling, half-correct subsystem entangled with your core app.
110
+
111
+ It's the kind of plumbing nobody wants to build, everybody rebuilds, and almost everybody ships *incomplete* — which is exactly what gets apps rejected from the stores and exposed under the DSA. `moderate` is the single, opinionated, batteries-included source of truth for it:
112
+
113
+ - **Report** users and content (in-app), with evidence snapshots and a real decision workflow.
114
+ - **Block** users (bidirectional), enforced everywhere a blocked pair could reconnect.
115
+ - **Filter** text and images before they're posted (`:off` / `:block` / `:flag`), with pluggable backends — a built-in offline wordlist, plus ready-to-copy reference adapters in `examples/` (OpenAI, AWS Rekognition) or your own.
116
+ - **Moderate** from a queue: remove content, ban users, dismiss, all audited.
117
+ - **Align** with the core DSA / store-review mechanisms: notice-and-action (Art. 16), statement of reasons (Art. 17), internal appeals (Art. 20), transparency counters (Art. 24); Apple Guideline 1.2 and Google Play UGC requirements.
118
+
119
+ Typical offending content include categories like these, all covered by the `moderate` gem: `harassment`, `hate`, `threats`, `sexual_content`, `spam`, `fraud`, `unsafe_behavior`, `illegal_content`, `privacy`, `child_safety`, `other`, `hate_abuse_harassment`, `violent_speech`, `graphic_violent_media`, `illegal_regulated_behaviors`, `impersonation`, `adult_sexual_content`, `private_non_consensual_content`, `suicide_self_harm`, `terrorism_violent_extremism`, `scam_fraud`
120
+
121
+ > [!IMPORTANT]
122
+ > The `moderate` gem is not a compliance certificate. You still own your policies, legal review, published contact information, jurisdiction-specific obligations, and day-to-day moderation operations. For example, EU DSA Article 19/24 complaint-handling and transparency duties have size/tier carve-outs (including micro/small enterprise exemptions); `moderate` just gives you the mechanisms when you need them, not a legal conclusion that every app must use every surface.
123
+
124
+ ## What `moderate` does and doesn't do
125
+
126
+ **Does:**
127
+ - User & content **reporting** (in-app) + a public **DSA legal-notice** intake form.
128
+ - **Blocking** with a single source-of-truth query you enforce in search, messaging, profiles, anywhere.
129
+ - **Pre-publication content filtering** with three modes and pluggable adapters — the built-in offline wordlist (text), plus image/LLM moderation via reference adapters you register (see `examples/`).
130
+ - A **moderation queue** with audited resolve / dismiss / remove-content / ban actions.
131
+ - **Appeals**, **statement-of-reasons** notifications, and **transparency** aggregation for the DSA.
132
+ - Optional **audit** and **notification** hooks that fan out to your mailer / admin alerts / push.
133
+
134
+ **Doesn't** (on purpose — these are other tools' jobs):
135
+ - Authentication / current-user (that's Devise — you tell `moderate` your user class).
136
+ - Sending the actual emails/push (that's [`goodmail`](https://github.com/rameerez/goodmail) / [`noticed`](https://github.com/excid3/noticed) — `moderate` just emits events).
137
+ - The admin UI chrome (that's [`madmin`](https://github.com/excid3/madmin) / your app — `moderate` gives you the data + primitives).
138
+ - A bulletproof ML classifier out of the box (the default text filter is a fast, multilingual wordlist; bring an LLM/image adapter when you want one).
139
+
140
+ ---
141
+
142
+ ## 🧑‍🤝‍🧑 Actors: report & block
143
+
144
+ Add `has_reporting_and_blocking` to your user model (or any model that acts on behalf of a person):
145
+
146
+ ```ruby
147
+ class User < ApplicationRecord
148
+ has_reporting_and_blocking
149
+ end
150
+ ```
151
+
152
+ *(Prefer an explicit include? `include Moderate::Actor` is the exact equivalent — the macro just lazily includes it.)*
153
+
154
+ **Blocking** is a bidirectional safety edge — once either side blocks, neither should see or reach the other:
155
+
156
+ ```ruby
157
+ current_user.block!(@other) # idempotent; audited; fires your on_block hook
158
+ current_user.unblock!(@other)
159
+ current_user.blocks?(@other) # I blocked them
160
+ current_user.blocked_by?(@other)
161
+ current_user.blocked_with?(@other) # either direction — the one you check in features
162
+ ```
163
+
164
+ Enforce it anywhere with the single source-of-truth query (no hand-rolled block SQL ever again):
165
+
166
+ ```ruby
167
+ # Hide blocked people from a marketplace / search / inbox:
168
+ Post.where.not(user_id: Moderate.blocked_ids_for(current_user))
169
+ ```
170
+
171
+ **Reporting** content or a person:
172
+
173
+ ```ruby
174
+ current_user.report!(@message, category: :harassment, details: "Won't stop messaging me")
175
+ current_user.report!(@user, category: :impersonation)
176
+ ```
177
+
178
+ `moderate` snapshots the offending content at report time (so evidence survives edits/deletes), infers who's responsible, sends the reporter a receipt, and drops it in the queue.
179
+
180
+ ## 🚩 Reportable content
181
+
182
+ Declare what can be reported with one `has_reportable_content` line — the fields are optional (omit them to report the whole record):
183
+
184
+ ```ruby
185
+ class Listing < ApplicationRecord
186
+ has_reportable_content :title, :description
187
+
188
+ # Tell moderate how to present & clean this content when a moderator acts:
189
+ def moderation_label = "Listing #{id}"
190
+ def reported_owner = user # who's responsible (defaults sensibly)
191
+ end
192
+ ```
193
+
194
+ *(Explicit-include equivalent: `include Moderate::Reportable` + `reportable_fields :title, :description`.)*
195
+
196
+ You get:
197
+
198
+ ```ruby
199
+ listing.reports # reports filed against this record
200
+ listing.reported? # any open report?
201
+ listing.flagged? # any pending flag (auto-filter OR manual)?
202
+ listing.flagged?(:description) # field-level pending flag?
203
+ ```
204
+
205
+ Drop a report link into any view with the helper (it renders nothing if the viewer can't report the content):
206
+
207
+ ```erb
208
+ <%= moderate_report_link(@listing, field: :description) %>
38
209
  ```
39
210
 
40
- Then, just add the `moderate` validation to any model with a text field:
211
+ Because `moderate` is UI-agnostic, it does not render a built-in "under review" badge. Use `flagged?` / `flagged?(:field)` to render copy that fits your product when `:flag` mode lets content through but queues it for review.
212
+
213
+ If your app runs inside Hotwire Native / Turbo Native, remember that native path configuration is host-owned. Add rules for the in-app report routes you mount (for example `/reports/new` **and** the form action `/reports`, so validation errors stay in the same modal stack) and for the engine's public legal routes **and their form actions** such as `<mount>/notices/new`, `<mount>/notices`, `<mount>/appeals/new`, `<mount>/appeals`, and `<mount>/transparency` — where `<mount>` is wherever you mounted `Moderate::Engine` in your routes (it is host-chosen, not fixed). `moderate` can provide the Rails routes; your native shell still decides whether they push, present modally, use a sheet, and which Android `uri` maps to the destination.
214
+
215
+ Adding a new reportable type is one `has_reportable_content` line — the intake, queue, snapshot, and admin code never change.
216
+
217
+ ## 🧪 Content filtering: `:off` / `:block` / `:flag`
218
+
219
+ Filtering is one declaration per field, with three modes:
220
+
221
+ ```ruby
222
+ class Message < ApplicationRecord
223
+ moderates :body # uses the default mode (see config)
224
+ end
225
+
226
+ class Profile < ApplicationRecord
227
+ moderates :bio, mode: :block # reject the save if it trips the filter
228
+ moderates :avatar, mode: :flag, with: :image # `:image` = a registered adapter (see examples/); only :wordlist ships built in
229
+ end
230
+ ```
231
+
232
+ - **`:off`** — no check.
233
+ - **`:block`** — the write is rejected with a validation error (great for public, high-trust fields).
234
+ - **`:flag`** — the write **succeeds**, and a `Moderate::Flag` is created **after commit** for human or automated review (great for DMs, where you don't want to block mid-conversation).
235
+
236
+ Why this matters: `:flag` never lives in a validator (validators must be side-effect-free, and a flag created inside a rolled-back transaction would silently vanish) — `moderate` handles that correctly for you.
237
+
238
+ Check content directly anywhere:
239
+
240
+ ```ruby
241
+ result = Moderate.classify("some sketchy text")
242
+ result.allowed? # => false
243
+ result.categories # => [:hate, :"hate/threatening"]
244
+ result.scores # => { "hate" => 0.97, "hate/threatening" => 0.81 } (0..1 for service adapters)
245
+ result.labels # => [#<Label category: :hate, subcategory: :threatening, score: 0.81, input: :text>, …]
246
+ ```
247
+
248
+ ### Filter adapters (the built-in wordlist, reference adapters, your own — one interface)
249
+
250
+ Every backend implements the same tiny contract — `classify(value) → Moderate::Result` — so they're interchangeable per field. `moderate` ships exactly **one** built-in adapter, the offline `:wordlist`; OpenAI, AWS Rekognition, and anything else are **bring-your-own** — copy a ready-made reference adapter from [`examples/`](examples/), add its gem to *your* Gemfile, and `register_adapter` it:
251
+
252
+ | Adapter | Use it for | Notes |
253
+ | --- | --- | --- |
254
+ | `:wordlist` (built-in, default) | text | Fast offline baseline, multilingual, zero-dependency. Includes Unicode normalization and common substitution handling, but it is not a contextual classifier. Ships `en`/`es` lists; add your own. The only adapter the gem ships. |
255
+ | OpenAI (reference adapter — [`examples/openai_moderation_adapter.rb`](examples/openai_moderation_adapter.rb)) | **text *and* image** | OpenAI `omni-moderation-latest` via the `ruby_llm` gem — **free**, multimodal, its category set IS the canonical taxonomy + `0..1` scores. Copy it in, `gem "ruby_llm"`, `register_adapter(:openai, …)`. Runs **async** (`Moderate::ClassifyJob`) in `:flag` mode. |
256
+ | AWS Rekognition (reference adapter — [`examples/aws_rekognition_adapter.rb`](examples/aws_rekognition_adapter.rb)) | images / avatars | `detect_moderation_labels` via `aws-sdk-rekognition`, with its taxonomy mapped onto the canonical labels. Copy it in, `gem "aws-sdk-rekognition"`, `register_adapter(:rekognition, …)`. Async, `:flag` mode. |
257
+ | *your own* | anything | `register_adapter(:replicate, …)` / Perspective / a self-hosted model — any object responding to `classify`. No built-in pretends the backend must be an "LLM". |
258
+
259
+ All adapters map their provider labels onto **one canonical taxonomy** (OpenAI's: `harassment[/threatening]`, `hate[/threatening]`, `sexual[/minors]`, `self-harm[/intent|/instructions]`, `violence[/graphic]`, `illicit[/violent]`), so `Moderate::Flag`, the DSA statement of reasons, and the transparency counters all speak one vocabulary.
260
+
261
+ ```ruby
262
+ Moderate.configure do |config|
263
+ config.default_filter_mode = :block
264
+ config.filter_adapter = :wordlist
265
+
266
+ # Bring an external classifier: copy examples/openai_moderation_adapter.rb into
267
+ # your app, add `gem "ruby_llm"`, then register and use it by name.
268
+ config.register_adapter :openai, OpenAIModerationAdapter.new
269
+
270
+ config.filter "Message", :body, with: :wordlist, mode: :flag
271
+ config.filter "Profile", :avatar, with: :openai, mode: :flag # one adapter moderates text AND images
272
+ end
273
+ ```
274
+
275
+ > **`:block` requires a synchronous adapter** (`:wordlist`) — you can't reject a save on a background result. The async reference adapters (the OpenAI/Rekognition examples) declare `synchronous? == false`, so they run in `:flag` mode (allow the write, classify in a job, file a `Moderate::Flag`). `moderate` validates this for you and says so.
276
+
277
+ Bring your own adapter — it's just an object that responds to `classify`:
278
+
279
+ ```ruby
280
+ class MyAdapter
281
+ def classify(value) = Moderate::Result.new(allowed: ..., categories: [...], scores: {...})
282
+ end
283
+ Moderate.register_adapter(:my_adapter, MyAdapter.new)
284
+ ```
285
+
286
+ > The original `moderate` (≤ 0.1) was *only* a profanity validator. That `validates :field, moderate: true` one-liner still works — it's now the `:wordlist` adapter in `:block` mode. See [Upgrading from 0.x](#upgrading-from-0x).
287
+
288
+ ## 🛠️ Admin & the moderation queue
289
+
290
+ Most of Trust & Safety happens in admin. `moderate` gives you the primitives; you bring the UI.
291
+
292
+ ```ruby
293
+ Moderate::Report.pending # the report queue
294
+ Moderate::Flag.pending # the auto-filter queue (human OR ML consumer reads the same scope)
295
+ Moderate::Appeal.pending # appeals awaiting a human
296
+
297
+ report.resolve!(by: admin, remove_content: true, ban_user: false, note: "Removed: hate speech")
298
+ report.dismiss!(by: admin, note: "No violation")
299
+ appeal.uphold!(by: admin, note: "...") # overturns the decision
300
+ appeal.reject!(by: admin, note: "...")
301
+ ```
302
+
303
+ Every action is atomic, requires a moderator + a note, runs your enforcement (content removal via the reportable's own `remove_reported_field!`, bans via your `ban_handler`), and writes to your audit log.
304
+
305
+ ### Use it from a controller (BYOUI)
306
+
307
+ ```ruby
308
+ class Admin::ReportsController < ApplicationController
309
+ include Moderate::Moderation # resolve!/dismiss! actions, strong params, redirects
310
+
311
+ before_action :require_admin
312
+ end
313
+ ```
314
+
315
+ ### Integrate with [`madmin`](https://github.com/excid3/madmin)
316
+
317
+ `moderate`'s models are plain ActiveRecord, so they show up in `madmin` like anything else. Generate a resource and point it at the model:
318
+
319
+ ```bash
320
+ rails generate madmin:resource Moderate::Report
321
+ ```
322
+
323
+ Then wire the resolve/dismiss actions to `Moderate::Report#resolve!`/`#dismiss!` from a custom member action (full walkthrough in [`docs/madmin.md`](docs/madmin.md)). The same pattern works for `Moderate::Flag` and `Moderate::Appeal`.
324
+
325
+ ## 🔔 Notifications & 🧾 audit — one hook each
326
+
327
+ `moderate` never sends an email or writes to *your* audit log directly. It **emits events** through two hooks you wire once — so notifications fan out wherever you want, and important actions are recorded however you want.
41
328
 
42
329
  ```ruby
43
- validates :text_field, moderate: true
330
+ Moderate.configure do |config|
331
+ # Called for every important action — wire it to your audit system (or leave it; it no-ops):
332
+ config.audit = ->(event) { AuditLog.record!(event_type: event.name, data: event.payload) }
333
+
334
+ # Called for every notifiable event — fan out to email / admin Telegram / push / in-app:
335
+ config.notify = ->(event) do
336
+ case event.name
337
+ when :report_received, :report_decision, :affected_user_decision
338
+ ModerationMailer.with(event:).public_send(event.name).deliver_later # goodmail
339
+ when :content_flagged, :report_received
340
+ Telegrama.send_message("🚩 #{event.payload[:summary]}") # admin alert
341
+ end
342
+ end
343
+
344
+ # Optional side effects when a block happens (e.g. tear down a pending invite):
345
+ config.on_block = ->(blocker:, blocked:, at:) { CancelPendingInvites.call(blocker, blocked, at: at) }
346
+ end
44
347
  ```
45
348
 
46
- `moderate` will raise an error if a bad word is found in the text field, preventing the record from being saved.
349
+ Events carry a stable envelope (`event.name`, `event.subject`, `event.recipients`, `event.actor`, `event.payload`), so a single `notify` hook can drive **goodmail** (user emails), **telegrama** (admin alerts), and **noticed** (in-app feed + push) at once. Notify users via email/in-app **and** ping admins on Telegram from the same place. (Recipes in [`docs/notifications.md`](docs/notifications.md).)
350
+
351
+ The full event vocabulary: `report_received`, `report_decision`, `affected_user_decision`, `appeal_received`, `appeal_decision`, `user_blocked`, `user_unblocked`, `user_banned`, `content_flagged`, `content_removed`.
47
352
 
48
- It works seamlessly with your existing validations and error messages.
353
+ ## ⚖️ DSA & app-store compliance, out of the box
49
354
 
50
- ## Configuration
355
+ `moderate` is built around the rules so you don't have to read the regulation:
356
+
357
+ - **DSA Art. 16 (notice & action):** a public, electronic notice form — a mountable engine you place at the path of your choosing (`mount Moderate::Engine => "/trust"`, no hardcoded `/legal`) — capturing the substantiated reason, exact URL, notifier name+email, good-faith statement, the EU **statement-of-reasons taxonomy**, and the member-state selector, with an automatic confirmation of receipt. A notice is a `Moderate::Report` with `intake_kind: "dsa"` (no separate model), built via `Moderate::Services::IntakeNotice`. The form prefills the reported-content fields from query params (editable) and a signed-in notifier's identity (locked), and auto-integrates [`rails_cloudflare_turnstile`](https://github.com/instrumentl/rails-cloudflare-turnstile) when present (falling back to a `config.notice_guard` proc, with an optional per-request skip hook for clients that cannot render a browser challenge). See [`docs/dsa-notice-form.md`](docs/dsa-notice-form.md).
358
+ - **DSA Art. 17 (statement of reasons):** decision notices state the action, the legal/contractual ground, whether automated means were used, and the redress path.
359
+ - **DSA Art. 20 (appeals):** a free, electronic internal complaint mechanism, open ≥ 6 months, decided by a human.
360
+ - **DSA Art. 24 (transparency):** counters you can publish (notices received, actions taken, median handling time, appeal outcomes). The public transparency page is **opt-in** (`config.transparency_report_enabled = true`, off by default) — a live portal isn't itself required (the duty is to *publish* a report, and micro/small enterprises are exempt), so you turn it on only when you want it.
361
+ - **Apple Guideline 1.2 & Google Play UGC:** filter-before-post, in-app report **and** block, ongoing moderation, published contact — `moderate` covers all four. See the mapped checklist in [`docs/compliance.md`](docs/compliance.md).
362
+
363
+ > Two taxonomies, on purpose: an in-app **community report** category set (harassment, spam, …) and a separate, regulator-aligned **DSA legal-reason** taxonomy for public notices. `moderate` ships both. The community set is host-customizable via `config.report_categories`; the DSA legal-reason taxonomy is regulator-defined and fixed.
364
+
365
+ ## 🤓 Why the models?
366
+
367
+ `rails generate moderate:install` creates four tables:
368
+
369
+ - **`moderate_reports`** — a report/notice + an immutable evidence snapshot + decision metadata + the appeal window. Serves both in-app reports and public DSA notices (distinguished by `intake_kind`).
370
+ - **`moderate_blocks`** — the bidirectional `blocker`/`blocked` edge, with a self-block check and the SSOT relation behind `Moderate.blocked_ids_for`.
371
+ - **`moderate_flags`** — system/auto-filter flags (source: `text_filter` / `image_filter` / `external_classifier` / `manual`), with the classifier's labels + scores; the queue both human admins and ML consumers read via `pending`.
372
+ - **`moderate_appeals`** — DSA Art. 20 internal complaints against a decision.
373
+
374
+ > The value-list taxonomies (community `category`, `status`, `content_type`, the DSA `legal_reason`/`legal_country_code`, `resolution_basis`, plus `Flag` source/mode/status and `Appeal` source/status) are validated **in the models** — frozen constants + ActiveModel `inclusion` validations — **not** by database `CHECK` constraints. That means **adding or customizing a label never requires a migration**: the community category list is host-overridable via `config.report_categories` (defaults to `Moderate::Report::DEFAULT_CATEGORIES`), and the gem can grow its own taxonomies in a point release without touching your schema. The only value guard kept at the DB level is a cheap message-length backstop; everything else the migration adds is structural (NOT NULLs, FKs, the unique block edge, and the self-block CHECK).
375
+
376
+ The migration is **adaptive**: it matches your app's primary-key type (UUID or bigint) and JSON column type (`jsonb` / `json`) automatically, so it drops cleanly into any Rails 7.1+ schema.
377
+
378
+ ## Configuration reference
51
379
 
52
- You can configure the `moderate` gem behavior by adding a `config/initializers/moderate.rb` file:
53
380
  ```ruby
54
381
  Moderate.configure do |config|
55
- # Custom error message when bad words are found
56
- config.error_message = "contains inappropriate language"
382
+ config.user_class = "User" # who reports/blocks/gets banned
383
+ config.default_filter_mode = :block # :off / :block / :flag
384
+ config.filter_adapter = :wordlist # default text adapter
57
385
 
58
- # Add your own words to the blacklist
59
- config.additional_words = ["badword1", "badword2"]
386
+ config.audit = ->(event) { ... } # optional; no-op by default
387
+ config.notify = ->(event) { ... } # optional; no-op by default
388
+ config.on_block = ->(blocker:, blocked:, at:) { ... } # optional
389
+ config.ban_handler = ->(user:, by:, reason:) { user.suspend! } # how a "ban" is applied in your app
60
390
 
61
- # Exclude words from the default list (false positives)
62
- config.excluded_words = ["good"]
391
+ config.filter "Message", :body, with: :wordlist, mode: :flag
63
392
  end
64
393
  ```
65
394
 
395
+ Reportable classes are auto-discovered from the `has_reportable_content` macro (or `include Moderate::Reportable`) — no manual registry.
396
+
397
+ ## Upgrading from 0.x
398
+
399
+ `moderate` 1.0 is a ground-up rewrite: the old gem was a profanity validator; 1.0 is a full Trust & Safety system. The one piece of the old API that remains is the validator, now backed by the `:wordlist` adapter:
400
+
401
+ ```ruby
402
+ validates :body, moderate: true # still works — equivalent to `moderates :body, mode: :block`
403
+ ```
404
+
405
+ Everything else is new. There's no automated data migration (0.x stored nothing). See [`CHANGELOG.md`](CHANGELOG.md).
406
+
407
+ ## Testing
408
+
409
+ We use Minitest. Run the suite (a dummy Rails app under `test/dummy`, against SQLite/PostgreSQL/MySQL via Appraisals):
410
+
411
+ ```bash
412
+ bundle exec rake test
413
+ ```
414
+
66
415
  ## Development
67
416
 
68
- After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
417
+ After checking out the repo, run `bin/setup` to install dependencies. Then run `rake test`. You can also run `bin/console` for an interactive prompt.
69
418
 
70
419
  To install this gem onto your local machine, run `bundle exec rake install`.
71
420
 
data/Rakefile CHANGED
@@ -1,4 +1,30 @@
1
- # frozen_string_literal: true
1
+ begin
2
+ require "bundler/setup"
3
+ rescue LoadError
4
+ puts "You must `gem install bundler` and `bundle install` to run rake tasks"
5
+ end
2
6
 
3
7
  require "bundler/gem_tasks"
4
- task default: %i[]
8
+
9
+ require "rdoc/task"
10
+
11
+ RDoc::Task.new(:rdoc) do |rdoc|
12
+ rdoc.rdoc_dir = "rdoc"
13
+ rdoc.title = "Moderate"
14
+ rdoc.options << "--line-numbers"
15
+ rdoc.rdoc_files.include("README.md")
16
+ rdoc.rdoc_files.include("lib/**/*.rb")
17
+ end
18
+
19
+ APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
20
+ load "rails/tasks/engine.rake"
21
+
22
+ require "rake/testtask"
23
+
24
+ Rake::TestTask.new(:test) do |t|
25
+ t.libs << "test"
26
+ t.pattern = "test/**/*_test.rb"
27
+ t.verbose = false
28
+ end
29
+
30
+ task default: :test
@@ -0,0 +1,161 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Moderate
4
+ # Drop-in admin moderation actions for a host's admin controller (BYOUI).
5
+ #
6
+ # class Admin::ReportsController < ApplicationController
7
+ # include Moderate::Moderation # resolve/dismiss (+ uphold/reject) actions
8
+ # before_action :require_admin # you bring auth
9
+ # end
10
+ #
11
+ # `moderate` deliberately ships NO admin UI — Trust & Safety chrome (branding,
12
+ # auth, layout) is the part every app wants to own. What it DOES own is the
13
+ # decision *logic*: every status change must go through the model's atomic
14
+ # decision method (`resolve!`/`dismiss!`/`uphold!`/`reject!`) so content removal,
15
+ # bans, the notify/audit hooks, the DSA Art. 17 statement-of-reasons, and the
16
+ # Art. 20 appeal window all happen together-or-not-at-all. This concern is the
17
+ # thin HTTP glue that calls those methods, so a host gets the standard wiring for
18
+ # free and never hand-rolls a raw `status = "..."` update (which would skip every
19
+ # one of those guarantees). See docs/madmin.md.
20
+ #
21
+ # The actions assume `@record` is already loaded (madmin's ResourceController and
22
+ # most admin frameworks set it from the member route). If yours doesn't, override
23
+ # `moderation_record` below.
24
+ module Moderation
25
+ extend ActiveSupport::Concern
26
+
27
+ # --- Report / Flag decisions ---------------------------------------------
28
+ # Both `Moderate::Report` and `Moderate::Flag` expose `resolve!`/`dismiss!` with
29
+ # the same keyword contract, so the SAME two actions drive either resource —
30
+ # the host just includes this concern in whichever controller.
31
+
32
+ # Resolve (action) a report/flag: optionally remove the offending content and/or
33
+ # ban the responsible user, always with a moderator + a note.
34
+ #
35
+ # `remove_content`/`ban_user` come from the form as checkboxes ("1"/"0"); the
36
+ # model casts them, but we pass them through untouched so the model stays the one
37
+ # place that interprets them. `note` is required by the model (it feeds the
38
+ # statement of reasons) — we let the model raise and turn that into a flash.
39
+ def resolve
40
+ record = moderation_record
41
+ record.resolve!(**moderation_decision_params)
42
+ redirect_after_moderation(record, notice: moderation_t(:resolved))
43
+ rescue => error
44
+ redirect_after_moderation(record, alert: moderation_error(:resolve, error))
45
+ end
46
+
47
+ # Dismiss (action) a report/flag: no violation found. Note still required.
48
+ def dismiss
49
+ record = moderation_record
50
+ record.dismiss!(by: moderation_actor, note: moderation_note)
51
+ redirect_after_moderation(record, notice: moderation_t(:dismissed))
52
+ rescue => error
53
+ redirect_after_moderation(record, alert: moderation_error(:dismiss, error))
54
+ end
55
+
56
+ # --- Appeal decisions (DSA Art. 20) --------------------------------------
57
+ # An appeal is a free, electronic, human-decided internal complaint against a
58
+ # decision. `uphold!` OVERTURNS the original decision; `reject!` CONFIRMS it.
59
+ # https://eur-lex.europa.eu/eli/reg/2022/2065/oj (Article 20)
60
+
61
+ def uphold
62
+ record = moderation_record
63
+ record.uphold!(by: moderation_actor, note: moderation_note)
64
+ redirect_after_moderation(record, notice: moderation_t(:upheld))
65
+ rescue => error
66
+ redirect_after_moderation(record, alert: moderation_error(:uphold, error))
67
+ end
68
+
69
+ def reject
70
+ record = moderation_record
71
+ record.reject!(by: moderation_actor, note: moderation_note)
72
+ redirect_after_moderation(record, notice: moderation_t(:rejected))
73
+ rescue => error
74
+ redirect_after_moderation(record, alert: moderation_error(:reject, error))
75
+ end
76
+
77
+ private
78
+
79
+ # The record being decided on. Override if your admin framework names it
80
+ # differently — madmin uses `@record`, many hand-rolled admins do too.
81
+ def moderation_record
82
+ @record
83
+ end
84
+
85
+ # The moderator making the call. `current_user` is the near-universal accessor;
86
+ # a host whose admin uses a different actor (e.g. `current_admin`) overrides
87
+ # this single method. Required by every decision method (the decision must be
88
+ # attributable to a human — a DSA Art. 17/20 requirement).
89
+ def moderation_actor
90
+ current_user
91
+ end
92
+
93
+ # The mandatory decision rationale. Required by the model too (belt and
94
+ # suspenders) — it's what populates the statement of reasons sent to the parties.
95
+ def moderation_note
96
+ params[:note]
97
+ end
98
+
99
+ # Strong params for the richest decision (`resolve` on a report). We don't use
100
+ # `params.require(:report)` because the decision form is a flat panel of
101
+ # checkboxes + a note, not a nested model form — so we read top-level params and
102
+ # hand the model exactly the keyword args it documents. `ban_user`/`remove_content`
103
+ # default to off so a missing checkbox never accidentally bans someone.
104
+ def moderation_decision_params
105
+ {
106
+ by: moderation_actor,
107
+ note: moderation_note,
108
+ remove_content: params[:remove_content],
109
+ ban_user: params[:ban_user]
110
+ }
111
+ end
112
+
113
+ # Redirect back to the record after a decision.
114
+ #
115
+ # `status: :see_other` (303) is REQUIRED, not stylistic: the decision actions are
116
+ # POSTs, and Turbo Drive only follows a redirect after a non-GET when the status
117
+ # is 303 — without it the redirect is swallowed and the page appears to hang.
118
+ # This is the same convention Rails' own scaffold create/update use.
119
+ # https://turbo.hotwired.dev/handbook/drive#redirecting-after-a-form-submission
120
+ #
121
+ # We fall back to `:back` when we can't build a path for the record, so the
122
+ # concern works regardless of the host's route names (BYOUI — we don't know
123
+ # them). The host can override `moderation_redirect_path` for an exact target.
124
+ def redirect_after_moderation(record, **flash)
125
+ path = moderation_redirect_path(record)
126
+ if path
127
+ redirect_to(path, status: :see_other, **flash)
128
+ else
129
+ redirect_back(fallback_location: "/", status: :see_other, **flash)
130
+ end
131
+ end
132
+
133
+ # Where to go after a decision. Returns nil so `redirect_after_moderation` falls
134
+ # back to `redirect_back` — the safe default for an admin we know nothing about.
135
+ # Override in the host controller to land on the record's show page, e.g.
136
+ # def moderation_redirect_path(record) = main_app.madmin_report_path(record)
137
+ def moderation_redirect_path(_record)
138
+ nil
139
+ end
140
+
141
+ # A flash message for the failure case. We surface the model's own error message
142
+ # (e.g. "report already closed", "note required") so the moderator sees WHY the
143
+ # decision didn't apply, not a generic "something went wrong".
144
+ def moderation_error(action, error)
145
+ message = error.respond_to?(:message) ? error.message : error.to_s
146
+ moderation_t(:"#{action}_failed", default: "Could not #{action}: #{message}", error: message)
147
+ end
148
+
149
+ # Translated flash copy, with a plain-English default so the concern works even
150
+ # before the host loads the gem's locale file.
151
+ def moderation_t(key, **options)
152
+ defaults = {
153
+ resolved: "Report resolved.",
154
+ dismissed: "Report dismissed.",
155
+ upheld: "Appeal upheld.",
156
+ rejected: "Appeal rejected."
157
+ }
158
+ I18n.t("moderate.moderation.#{key}", default: options.delete(:default) || defaults[key] || key.to_s, **options)
159
+ end
160
+ end
161
+ end