moderate 0.1.0 → 1.0.0.beta1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (65) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +8 -0
  3. data/.simplecov +62 -0
  4. data/AGENTS.md +7 -0
  5. data/Appraisals +16 -0
  6. data/CHANGELOG.md +71 -1
  7. data/CLAUDE.md +7 -0
  8. data/README.md +376 -29
  9. data/Rakefile +28 -2
  10. data/app/controllers/concerns/moderate/moderation.rb +161 -0
  11. data/app/controllers/moderate/appeals_controller.rb +190 -0
  12. data/app/controllers/moderate/application_controller.rb +45 -0
  13. data/app/controllers/moderate/notices_controller.rb +382 -0
  14. data/app/controllers/moderate/transparency_reports_controller.rb +30 -0
  15. data/app/helpers/moderate/engine_helper.rb +151 -0
  16. data/app/views/moderate/appeals/new.html.erb +78 -0
  17. data/app/views/moderate/notices/new.html.erb +255 -0
  18. data/app/views/moderate/transparency_reports/_summary_card.html.erb +20 -0
  19. data/app/views/moderate/transparency_reports/show.html.erb +52 -0
  20. data/config/moderate/blocklists/en.yml +81 -0
  21. data/config/moderate/blocklists/es.yml +40 -0
  22. data/config/routes.rb +36 -0
  23. data/docs/compliance.md +178 -0
  24. data/docs/configuration.md +326 -0
  25. data/docs/dsa-notice-form.md +371 -0
  26. data/docs/madmin.md +490 -0
  27. data/docs/notifications.md +363 -0
  28. data/examples/aws_rekognition_adapter.rb +140 -0
  29. data/examples/openai_moderation_adapter.rb +111 -0
  30. data/gemfiles/rails_7.1.gemfile +36 -0
  31. data/gemfiles/rails_7.2.gemfile +36 -0
  32. data/gemfiles/rails_8.1.gemfile +36 -0
  33. data/lib/generators/moderate/install_generator.rb +56 -0
  34. data/lib/generators/moderate/templates/create_moderate_tables.rb.erb +237 -0
  35. data/lib/generators/moderate/templates/initializer.rb +198 -0
  36. data/lib/generators/moderate/views_generator.rb +63 -0
  37. data/lib/moderate/configuration.rb +341 -0
  38. data/lib/moderate/engine.rb +138 -0
  39. data/lib/moderate/errors.rb +26 -0
  40. data/lib/moderate/event.rb +75 -0
  41. data/lib/moderate/filters/base.rb +126 -0
  42. data/lib/moderate/filters/wordlist.rb +255 -0
  43. data/lib/moderate/jobs/classify_job.rb +158 -0
  44. data/lib/moderate/label.rb +111 -0
  45. data/lib/moderate/macros.rb +90 -0
  46. data/lib/moderate/models/appeal.rb +154 -0
  47. data/lib/moderate/models/application_record.rb +31 -0
  48. data/lib/moderate/models/block.rb +203 -0
  49. data/lib/moderate/models/concerns/actor.rb +174 -0
  50. data/lib/moderate/models/concerns/content_filterable.rb +155 -0
  51. data/lib/moderate/models/concerns/reportable.rb +282 -0
  52. data/lib/moderate/models/flag.rb +136 -0
  53. data/lib/moderate/models/report.rb +620 -0
  54. data/lib/moderate/result.rb +176 -0
  55. data/lib/moderate/services/intake_appeal.rb +89 -0
  56. data/lib/moderate/services/intake_notice.rb +132 -0
  57. data/lib/moderate/services/intake_report.rb +132 -0
  58. data/lib/moderate/services/resolve_appeal.rb +134 -0
  59. data/lib/moderate/services/resolve_flag.rb +101 -0
  60. data/lib/moderate/services/resolve_report.rb +291 -0
  61. data/lib/moderate/version.rb +1 -1
  62. data/lib/moderate.rb +365 -18
  63. data/log/development.log +0 -0
  64. data/log/test.log +0 -0
  65. metadata +154 -15
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c7c7f6fd1cfa1c2ba6b28e2d0b75ce803d0ae2eec002011b18a9a25b2b98ce6f
4
- data.tar.gz: 71bab2651af6c3ff8520360281811b53d21a711496a9ef392334e8b2a59646b5
3
+ metadata.gz: 446ba3a97f77b4a88ebbd1d6e45e7935d6002be573835ef750829f54adaa47a7
4
+ data.tar.gz: 799dd6c0e21cac2b0b8038e268cf02baec331717c427ddf1d51bafa482b88776
5
5
  SHA512:
6
- metadata.gz: adfb034df21d5ecfa8fec498f34df2d6595d33033f05e7fb2832ec52f7c0012fa43806950b9c9d723cc0508563159972e57ceb051e3263ca0cfff11583bb9959
7
- data.tar.gz: 2dd06af75442e89d39e18f44c7dc61e37030dfbee6406f5be096903418489f9061dea5262aeaa874431ec4c3ac82f3c93933036d7715e3255e6431715c6923ff
6
+ metadata.gz: 00f2800700b563559a9d8d40dc2a3fea24d487d24d2f660059908a2b377cce5627354961f3bfeff0d7f1da2c5666cc9aac45c1603a48c15ff20c9ad4b2502d33
7
+ data.tar.gz: c047552f0534a7df8642d2615d112723b19ecaeb62d870261c13c4fb8b114833dc0ba72ae9a646cb8e4cacba658bd1e705724c451c58a2b0348506aece0c4fbd
data/.rubocop.yml ADDED
@@ -0,0 +1,8 @@
1
+ AllCops:
2
+ TargetRubyVersion: 3.2
3
+
4
+ Style/StringLiterals:
5
+ EnforcedStyle: double_quotes
6
+
7
+ Style/StringLiteralsInInterpolation:
8
+ EnforcedStyle: double_quotes
data/.simplecov ADDED
@@ -0,0 +1,62 @@
1
+ # frozen_string_literal: true
2
+
3
+ # SimpleCov configuration file (auto-loaded before test suite)
4
+ # This keeps test_helper.rb clean and follows best practices.
5
+ # Coherent with the rest of the gem ecosystem (usage_credits, pricing_plans, …).
6
+
7
+ SimpleCov.start do
8
+ # Use SimpleFormatter for terminal-only output (no HTML generation)
9
+ formatter SimpleCov::Formatter::SimpleFormatter
10
+
11
+ # Don't count the test suite itself toward coverage
12
+ add_filter "/test/"
13
+
14
+ # Don't count code that ISN'T unit-testable by this suite and would only distort
15
+ # the numbers:
16
+ # - Generators + their templates: these run via `rails generate moderate:install`
17
+ # / `moderate:views` in a real host, not in the engine's own unit suite. (The
18
+ # migration template IS exercised indirectly — the dummy migrates a copy of it —
19
+ # but the .erb itself is never loaded as Ruby here.)
20
+ # - version.rb: a single constant; nothing to cover.
21
+ # - The 0.x compatibility shims (text / text_validator / word_list): legacy
22
+ # profanity-validator code kept only so `validates :field, moderate: true` from
23
+ # 0.x still loads (see README "Upgrading from 0.x"). They are NOT part of the
24
+ # 1.0 Trust & Safety surface this suite tests, so they shouldn't pull the 1.0
25
+ # coverage number down.
26
+ add_filter "/lib/generators/"
27
+ add_filter "/lib/moderate/version.rb"
28
+ add_filter "/lib/moderate/text.rb"
29
+ add_filter "/lib/moderate/text_validator.rb"
30
+ add_filter "/lib/moderate/word_list.rb"
31
+
32
+ # Track Ruby files in the lib directory (gem source code)
33
+ track_files "lib/**/*.rb"
34
+
35
+ # Enable branch coverage for more detailed metrics
36
+ enable_coverage :branch
37
+
38
+ # Minimum coverage thresholds to prevent coverage REGRESSION. These reflect what
39
+ # the current shipped suite actually exercises (line ~86%, branch ~65%): the
40
+ # primitives — models, concerns, services, adapters, the facade, the value objects —
41
+ # are thoroughly covered; the lower branch number is driven by the engine's
42
+ # CONTROLLERS (the public DSA notice form + the BYOUI moderation concern) and the
43
+ # async ClassifyJob, whose request/job paths the unit suite doesn't drive. The
44
+ # thresholds sit just under the current floor so the gate catches a real regression
45
+ # without failing on the existing baseline; raise them as request/job coverage grows.
46
+ minimum_coverage line: 80, branch: 60
47
+
48
+ # Disambiguate parallel test runs
49
+ command_name "Job #{ENV['TEST_ENV_NUMBER']}" if ENV["TEST_ENV_NUMBER"]
50
+ end
51
+
52
+ # Print coverage summary to terminal after tests complete
53
+ SimpleCov.at_exit do
54
+ SimpleCov.result.format!
55
+ puts "\n" + "=" * 60
56
+ puts "COVERAGE SUMMARY"
57
+ puts "=" * 60
58
+ puts "Line Coverage: #{SimpleCov.result.covered_percent.round(2)}%"
59
+ branch_coverage = SimpleCov.result.coverage_statistics[:branch]&.percent&.round(2) || "N/A"
60
+ puts "Branch Coverage: #{branch_coverage}%"
61
+ puts "=" * 60
62
+ end
data/AGENTS.md ADDED
@@ -0,0 +1,7 @@
1
+ # AGENTS.md
2
+
3
+ This file provides guidance to AI Agents (like OpenAI's Codex, Cursor Agent, Claude Code, etc) when working with code in this repository.
4
+
5
+ Please read the `README.md` for a full overview of the gem's API and philosophy, and the `docs/` directory (`docs/configuration.md`, `docs/notifications.md`, `docs/compliance.md`, `docs/madmin.md`, `docs/dsa-notice-form.md`) for the detailed integration guides.
6
+
7
+ This gem is part of a coherent ecosystem (`railsfast`, `goodmail`, `telegrama`, `usage_credits`, `pricing_plans`, `wallets`, `api_keys`). Match the ecosystem conventions exactly: a single `Moderate.configure do |config| … end` block, `has_*`/verb-style class macros, adapter objects + no-op-default hook procs, string class names constantized lazily, adaptive install migrations, Minitest with a `test/dummy` app, SimpleCov, and the README/docs voice.
data/Appraisals ADDED
@@ -0,0 +1,16 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Test the minimum supported Rails version (matches the gemspec floor and the
4
+ # README's "Rails 7.1+ schema" claim — the adaptive migration must work here).
5
+ appraise "rails-7.1" do
6
+ gem "rails", "~> 7.1.0"
7
+ end
8
+
9
+ appraise "rails-7.2" do
10
+ gem "rails", "~> 7.2.0"
11
+ end
12
+
13
+ # Test the latest Rails version — this is the default/main Gemfile anyway.
14
+ appraise "rails-8.1" do
15
+ gem "rails", "~> 8.1.0"
16
+ end
data/CHANGELOG.md CHANGED
@@ -1,3 +1,73 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here.
4
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
5
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
+
7
+ ## [1.0.0] - unreleased
8
+
9
+ A complete, ground-up rewrite. `moderate` graduates from a single-purpose profanity
10
+ validator (0.1.0) into a full **Trust & Safety** engine for Rails apps with user-generated
11
+ content: report, block, filter, a moderation queue, appeals, and EU DSA / App Store / Google
12
+ Play **aligned** primitives. (First cut ships as `1.0.0.beta1`.)
13
+
14
+ > **Breaking:** 1.0 keeps the gem name but is an entirely new API. The 0.x profanity
15
+ > validator (`validates :field, moderate: true`) still loads for backward compatibility
16
+ > (see _Upgrading from 0.x_), but everything else is new. Pin `~> 0.1` if you relied on the
17
+ > old behavior and are not ready to adopt the new surface.
18
+
19
+ ### Added
20
+
21
+ - **Reporting.** `Moderate::Report` plus the `has_reportable_content :fields` macro and
22
+ `Actor#report!(reportable, category:, details:)`. Reports and DSA notices share one model
23
+ and one queue (`intake_kind: "community" | "dsa"`).
24
+ - **Blocking.** `Moderate::Block`, the `has_reporting_and_blocking` actor macro, and
25
+ `block!` / `unblock!` / `blocks?` / `blocked_by?` / `blocked_with?`. `Moderate.blocked_ids_for(user)`
26
+ is the bidirectional single source of truth you compose into feed/search/inbox queries.
27
+ Optional `config.on_block` teardown hook runs inside the block transaction.
28
+ - **Content filtering.** The `moderates :field, mode: :off|:block|:flag, with: :adapter` macro
29
+ (and the equivalent `config.filter`), the offline multilingual `:wordlist` adapter (the only
30
+ built-in), the `classify(value) => Moderate::Result` adapter contract with
31
+ `config.register_adapter`, asynchronous classification via `Moderate::ClassifyJob`, and
32
+ ready-to-copy reference adapters for OpenAI omni-moderation and AWS Rekognition under
33
+ `examples/` (bring-your-own, never a dependency).
34
+ - **Moderation queue & decisions.** `Moderate::Flag` and the service objects
35
+ `Moderate::Services::{IntakeReport, ResolveReport, ResolveFlag, IntakeAppeal, ResolveAppeal,
36
+ IntakeNotice}`. Decisions are taken under a row lock, re-check open state, apply enforcement
37
+ (remove content / ban) inside the transaction, and fire notifications outside it; the appeal
38
+ window and statement-of-reasons fields are stamped automatically.
39
+ - **DSA-aligned primitives.** A mountable public **notice-and-action** form (Art. 16) you mount
40
+ at any path, **statement of reasons** (Art. 17), internal **appeals** (Art. 20), and
41
+ **transparency** counters (Art. 24). The notice form prefills from query params + the signed-in
42
+ user, locks auto-filled identity fields, and auto-detects `rails_cloudflare_turnstile`.
43
+ - **Hooks (all no-op by default).** `config.audit`, `config.notify` (returns a delivery boolean
44
+ used to gate `decision_notified_at`), `config.on_block`, `config.ban_handler`, the
45
+ host-overridable `config.report_categories`, and `config.notice_human_verification_skip_if` /
46
+ `config.appeal_human_verification_skip_if` for native-app bot-gate carve-outs.
47
+ - **Optional integrations**, all auto-detected at runtime via `defined?`/`respond_to?` and never
48
+ hard dependencies: madmin, goodmail, telegrama, noticed, rails_cloudflare_turnstile.
49
+ - **Install tooling.** `rails generate moderate:install` writes a documented initializer and an
50
+ adaptive migration (uuid/bigint primary keys, jsonb/json/MySQL JSON columns); `moderate:views`
51
+ ejects the notice form for customization.
52
+
53
+ ### Changed
54
+
55
+ - Taxonomies (report categories, DSA legal reasons, country codes) are now frozen model
56
+ constants with inclusion validations instead of DB `CHECK` constraints — adding or
57
+ customizing a category needs no migration.
58
+ - External classifiers (OpenAI, image moderation) are reference adapters in `examples/`, not
59
+ shipped or loaded code — the gem core forces no service dependency on apps that never use it.
60
+ - All "DSA-compliant" / "App Store compliant" language reframed to **DSA-aligned primitives**:
61
+ the gem ships the mechanisms the law and the stores require; your policies, response times,
62
+ and operations are still yours.
63
+
64
+ ### Upgrading from 0.x
65
+
66
+ - The 0.x profanity validator still loads: `validates :field, moderate: true` continues to work
67
+ via compatibility shims, so existing apps keep validating. To adopt 1.0, add
68
+ `has_reporting_and_blocking` to your user model and `has_reportable_content` / `moderates` to your
69
+ content models, run `rails generate moderate:install`, and migrate.
70
+
1
71
  ## [0.1.0] - 2024-11-03
2
72
 
3
- - Initial release
73
+ - Initial release (profanity validator).
data/CLAUDE.md ADDED
@@ -0,0 +1,7 @@
1
+ # CLAUDE.md
2
+
3
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+ Please read the `README.md` for a full overview of the gem's API and philosophy, and the `docs/` directory (`docs/configuration.md`, `docs/notifications.md`, `docs/compliance.md`, `docs/madmin.md`, `docs/dsa-notice-form.md`) for the detailed integration guides.
6
+
7
+ This gem is part of a coherent ecosystem (`railsfast`, `goodmail`, `telegrama`, `usage_credits`, `pricing_plans`, `wallets`, `api_keys`). Match the ecosystem conventions exactly: a single `Moderate.configure do |config| … end` block, `has_*`/verb-style class macros, adapter objects + no-op-default hook procs, string class names constantized lazily, adaptive install migrations, Minitest with a `test/dummy` app, SimpleCov, and the README/docs voice.
data/README.md CHANGED
@@ -1,71 +1,418 @@
1
- # 👮‍♂️ `moderate` - Moderate and block bad words from your Rails app
1
+ # 🛡️ `moderate` - Let your Rails users report content and block each other (Trust & Safety)
2
2
 
3
- `moderate` is a Ruby gem that moderates user-generated text content by adding a simple validation to block bad words in any text field.
3
+ [![Gem Version](https://badge.fury.io/rb/moderate.svg)](https://badge.fury.io/rb/moderate) [![Build Status](https://github.com/rameerez/moderate/workflows/Tests/badge.svg)](https://github.com/rameerez/moderate/actions)
4
4
 
5
- Simply add this to your model:
5
+ > [!TIP]
6
+ > **🚀 Ship your next Rails app 10x faster!** I've built **[RailsFast](https://railsfast.com/?ref=moderate)**, a production-ready Rails boilerplate template that comes with everything you need to launch a software business in days, not weeks. Go [check it out](https://railsfast.com/?ref=moderate)!
7
+
8
+ `moderate` gives your Rails app a complete **Trust & Safety** system.
9
+
10
+ Trust & Safety (T&S) is the system within an app that lets users **report** abusive content, **block** each other, **filter** objectionable text and images before they're posted (profanity, bad words, NSFW / nudity, etc.), and run a **moderation queue** your admins actually use. It also allows you to easily plug in automated AI moderation systems like **OpenAI Moderation** or **AWS Rekognition** to quickly filter, flag and/or automatically block harmful content (text or image).
11
+
12
+ If you have an app where users can upload / generate content or send messages to each other, you probably need a Trust & Safety system.
13
+
14
+ `moderate` ships with mechanisms aligned with the **DSA** (EU Digital Services Act), and also aligned with the **Apple App Store** and Android's **Google Play** directives for User-Generated Content (UGC) in their app stores.
15
+
16
+ ## 👨‍💻 Example
17
+
18
+ `moderate` reads like plain English. Make any model reportable:
19
+
20
+ ```ruby
21
+ class Comment < ApplicationRecord
22
+ has_reportable_content
23
+ end
24
+ ```
25
+
26
+ Let any user block another:
27
+
28
+ ```ruby
29
+ current_user.block!(@other_user)
30
+ current_user.blocks?(@other_user) # => true
31
+ ```
32
+
33
+ Filter content before it's posted — the zero-setup wordlist, or a real classifier like OpenAI moderation:
6
34
 
7
35
  ```ruby
8
- validates :text_field, moderate: true
36
+ # config/initializers/moderate.rb — wire up OpenAI moderation once (text AND images)
37
+ config.register_adapter :openai, OpenAIModerationAdapter.new
9
38
  ```
10
39
 
11
- That's it! You're done. `moderate` will work seamlessly with your existing validations and error messages.
40
+ ```ruby
41
+ class Message < ApplicationRecord
42
+ # Run every DM through OpenAI, but never block mid-conversation: `:flag` lets the
43
+ # message send, then classifies it in a background job and drops anything harmful
44
+ # into the moderation queue for review.
45
+ moderates :body, mode: :flag, with: :openai
46
+ end
47
+ ```
12
48
 
13
- > [!WARNING]
14
- > This gem is under development. It currently only supports a limited set of English profanity words. Word matching is very basic now, and it may be prone to false positives, and false negatives. I use it for very simple things like preventing new submissions if they contain bad words, but the gem can be improved for more complex use cases and sophisticated matching and content moderation. Please consider contributing if you can improve the gem, or have good ideas for additional features.
49
+ No API keys to start? Drop the `with:` and you get the built-in, zero-dependency `:wordlist` (a fast, multilingual profanity block) — same one-line API.
15
50
 
16
- # Why
51
+ And give admins a real queue to act on:
17
52
 
18
- Any text field where users can input text may be a place where bad words can be used. This gem blocks records from being created if they contain bad words, profanity, naughty / obscene words, etc.
53
+ ```ruby
54
+ Moderate::Report.pending # everything awaiting a decision
55
+ report.resolve!(by: current_user, remove_content: true, ban_user: true, note: "Hate speech")
56
+ ```
19
57
 
20
- It's good for Rails applications where you need to maintain a clean and respectful environment in comments, posts, or any other user input.
58
+ That's the whole idea: **the messy, legally-loaded plumbing every social/UGC app needs (report, block, filter, moderate, appeal, comply) as one coherent Ruby gem** instead of scattered, half-finished, store-rejecting DIY code.
21
59
 
22
- # How
60
+ > [!NOTE]
61
+ > `moderate` is **UI-agnostic by design**: most of a Trust & Safety system lives in *admin* surfaces, so the gem ships the **primitives** (models, services, helpers, controller concerns) and lets you **build your own UI**. It plugs into [`madmin`](https://github.com/excid3/madmin) (or any admin system) in minutes; see [Admin & moderation queue](#-admin--the-moderation-queue).
23
62
 
24
- `moderate` currently downloads a list of ~1k English profanity words from the [google-profanity-words](https://github.com/coffee-and-fun/google-profanity-words) repository and caches it in your Rails app's tmp directory.
63
+ ---
25
64
 
26
- ## Installation
65
+ ## Quickstart
27
66
 
28
- Add this line to your application's Gemfile:
67
+ Add the gem:
29
68
 
30
69
  ```ruby
31
- gem 'moderate'
70
+ gem "moderate"
32
71
  ```
33
72
 
34
- And then execute:
73
+ Install it (creates the migration + an initializer):
35
74
 
36
75
  ```bash
37
76
  bundle install
77
+ rails generate moderate:install
78
+ rails db:migrate
79
+ ```
80
+
81
+ Tell `moderate` who your users are, and make a model reportable:
82
+
83
+ ```ruby
84
+ # config/initializers/moderate.rb
85
+ Moderate.configure do |config|
86
+ config.user_class = "User"
87
+ end
88
+ ```
89
+
90
+ ```ruby
91
+ class User < ApplicationRecord
92
+ has_reporting_and_blocking # can report, block, be blocked, be banned
93
+ end
94
+
95
+ class Post < ApplicationRecord
96
+ has_reportable_content # users can report it
97
+ moderates :body, mode: :block # …and profanity is rejected on save — zero-setup built-in wordlist
98
+ end
99
+ ```
100
+
101
+ That's it. You now have reporting, blocking, filtering, and a moderation queue. Everything below is detail.
102
+
103
+ ---
104
+
105
+ ## Why this gem exists
106
+
107
+ Every app with user-generated content eventually faces the same wall. A user posts something vile, another user wants them gone, Apple rejects your build for "no way to report objectionable content," and a Spanish lawyer emails you about the Digital Services Act. So you start bolting on a `reports` table, a `blocks` table, a profanity regex, an admin page, a "notify the reporter" email… and it's suddenly a sprawling, half-correct subsystem entangled with your core app.
108
+
109
+ It's the kind of plumbing nobody wants to build, everybody rebuilds, and almost everybody ships *incomplete* — which is exactly what gets apps rejected from the stores and exposed under the DSA. `moderate` is the single, opinionated, batteries-included source of truth for it:
110
+
111
+ - **Report** users and content (in-app), with evidence snapshots and a real decision workflow.
112
+ - **Block** users (bidirectional), enforced everywhere a blocked pair could reconnect.
113
+ - **Filter** text and images before they're posted (`:off` / `:block` / `:flag`), with pluggable backends — a built-in offline wordlist, plus ready-to-copy reference adapters in `examples/` (OpenAI, AWS Rekognition) or your own.
114
+ - **Moderate** from a queue: remove content, ban users, dismiss, all audited.
115
+ - **Align** with the core DSA / store-review mechanisms: notice-and-action (Art. 16), statement of reasons (Art. 17), internal appeals (Art. 20), transparency counters (Art. 24); Apple Guideline 1.2 and Google Play UGC requirements.
116
+
117
+ Typical offending content include categories like these, all covered by the `moderate` gem: `harassment`, `hate`, `threats`, `sexual_content`, `spam`, `fraud`, `unsafe_behavior`, `illegal_content`, `privacy`, `child_safety`, `other`, `hate_abuse_harassment`, `violent_speech`, `graphic_violent_media`, `illegal_regulated_behaviors`, `impersonation`, `adult_sexual_content`, `private_non_consensual_content`, `suicide_self_harm`, `terrorism_violent_extremism`, `scam_fraud`
118
+
119
+ > [!IMPORTANT]
120
+ > The `moderate` gem is not a compliance certificate. You still own your policies, legal review, published contact information, jurisdiction-specific obligations, and day-to-day moderation operations. For example, EU DSA Article 19/24 complaint-handling and transparency duties have size/tier carve-outs (including micro/small enterprise exemptions); `moderate` just gives you the mechanisms when you need them, not a legal conclusion that every app must use every surface.
121
+
122
+ ## What `moderate` does and doesn't do
123
+
124
+ **Does:**
125
+ - User & content **reporting** (in-app) + a public **DSA legal-notice** intake form.
126
+ - **Blocking** with a single source-of-truth query you enforce in search, messaging, profiles, anywhere.
127
+ - **Pre-publication content filtering** with three modes and pluggable adapters — the built-in offline wordlist (text), plus image/LLM moderation via reference adapters you register (see `examples/`).
128
+ - A **moderation queue** with audited resolve / dismiss / remove-content / ban actions.
129
+ - **Appeals**, **statement-of-reasons** notifications, and **transparency** aggregation for the DSA.
130
+ - Optional **audit** and **notification** hooks that fan out to your mailer / admin alerts / push.
131
+
132
+ **Doesn't** (on purpose — these are other tools' jobs):
133
+ - Authentication / current-user (that's Devise — you tell `moderate` your user class).
134
+ - Sending the actual emails/push (that's [`goodmail`](https://github.com/rameerez/goodmail) / [`noticed`](https://github.com/excid3/noticed) — `moderate` just emits events).
135
+ - The admin UI chrome (that's [`madmin`](https://github.com/excid3/madmin) / your app — `moderate` gives you the data + primitives).
136
+ - A bulletproof ML classifier out of the box (the default text filter is a fast, multilingual wordlist; bring an LLM/image adapter when you want one).
137
+
138
+ ---
139
+
140
+ ## 🧑‍🤝‍🧑 Actors: report & block
141
+
142
+ Add `has_reporting_and_blocking` to your user model (or any model that acts on behalf of a person):
143
+
144
+ ```ruby
145
+ class User < ApplicationRecord
146
+ has_reporting_and_blocking
147
+ end
148
+ ```
149
+
150
+ *(Prefer an explicit include? `include Moderate::Actor` is the exact equivalent — the macro just lazily includes it.)*
151
+
152
+ **Blocking** is a bidirectional safety edge — once either side blocks, neither should see or reach the other:
153
+
154
+ ```ruby
155
+ current_user.block!(@other) # idempotent; audited; fires your on_block hook
156
+ current_user.unblock!(@other)
157
+ current_user.blocks?(@other) # I blocked them
158
+ current_user.blocked_by?(@other)
159
+ current_user.blocked_with?(@other) # either direction — the one you check in features
160
+ ```
161
+
162
+ Enforce it anywhere with the single source-of-truth query (no hand-rolled block SQL ever again):
163
+
164
+ ```ruby
165
+ # Hide blocked people from a marketplace / search / inbox:
166
+ Post.where.not(user_id: Moderate.blocked_ids_for(current_user))
167
+ ```
168
+
169
+ **Reporting** content or a person:
170
+
171
+ ```ruby
172
+ current_user.report!(@message, category: :harassment, details: "Won't stop messaging me")
173
+ current_user.report!(@user, category: :impersonation)
174
+ ```
175
+
176
+ `moderate` snapshots the offending content at report time (so evidence survives edits/deletes), infers who's responsible, sends the reporter a receipt, and drops it in the queue.
177
+
178
+ ## 🚩 Reportable content
179
+
180
+ Declare what can be reported with one `has_reportable_content` line — the fields are optional (omit them to report the whole record):
181
+
182
+ ```ruby
183
+ class Listing < ApplicationRecord
184
+ has_reportable_content :title, :description
185
+
186
+ # Tell moderate how to present & clean this content when a moderator acts:
187
+ def moderation_label = "Listing #{id}"
188
+ def reported_owner = user # who's responsible (defaults sensibly)
189
+ end
38
190
  ```
39
191
 
40
- Then, just add the `moderate` validation to any model with a text field:
192
+ *(Explicit-include equivalent: `include Moderate::Reportable` + `reportable_fields :title, :description`.)*
193
+
194
+ You get:
41
195
 
42
196
  ```ruby
43
- validates :text_field, moderate: true
197
+ listing.reports # reports filed against this record
198
+ listing.reported? # any open report?
199
+ listing.flagged? # any pending flag (auto-filter OR manual)?
200
+ listing.flagged?(:description) # field-level pending flag?
201
+ ```
202
+
203
+ Drop a report link into any view with the helper (it renders nothing if the viewer can't report the content):
204
+
205
+ ```erb
206
+ <%= moderate_report_link(@listing, field: :description) %>
44
207
  ```
45
208
 
46
- `moderate` will raise an error if a bad word is found in the text field, preventing the record from being saved.
209
+ Because `moderate` is UI-agnostic, it does not render a built-in "under review" badge. Use `flagged?` / `flagged?(:field)` to render copy that fits your product when `:flag` mode lets content through but queues it for review.
47
210
 
48
- It works seamlessly with your existing validations and error messages.
211
+ If your app runs inside Hotwire Native / Turbo Native, remember that native path configuration is host-owned. Add rules for the in-app report routes you mount (for example `/reports/new` **and** the form action `/reports`, so validation errors stay in the same modal stack) and for the engine's public legal routes **and their form actions** such as `<mount>/notices/new`, `<mount>/notices`, `<mount>/appeals/new`, `<mount>/appeals`, and `<mount>/transparency` — where `<mount>` is wherever you mounted `Moderate::Engine` in your routes (it is host-chosen, not fixed). `moderate` can provide the Rails routes; your native shell still decides whether they push, present modally, use a sheet, and which Android `uri` maps to the destination.
49
212
 
50
- ## Configuration
213
+ Adding a new reportable type is one `has_reportable_content` line — the intake, queue, snapshot, and admin code never change.
214
+
215
+ ## 🧪 Content filtering: `:off` / `:block` / `:flag`
216
+
217
+ Filtering is one declaration per field, with three modes:
218
+
219
+ ```ruby
220
+ class Message < ApplicationRecord
221
+ moderates :body # uses the default mode (see config)
222
+ end
223
+
224
+ class Profile < ApplicationRecord
225
+ moderates :bio, mode: :block # reject the save if it trips the filter
226
+ moderates :avatar, mode: :flag, with: :image # `:image` = a registered adapter (see examples/); only :wordlist ships built in
227
+ end
228
+ ```
229
+
230
+ - **`:off`** — no check.
231
+ - **`:block`** — the write is rejected with a validation error (great for public, high-trust fields).
232
+ - **`:flag`** — the write **succeeds**, and a `Moderate::Flag` is created **after commit** for human or automated review (great for DMs, where you don't want to block mid-conversation).
233
+
234
+ Why this matters: `:flag` never lives in a validator (validators must be side-effect-free, and a flag created inside a rolled-back transaction would silently vanish) — `moderate` handles that correctly for you.
235
+
236
+ Check content directly anywhere:
237
+
238
+ ```ruby
239
+ result = Moderate.classify("some sketchy text")
240
+ result.allowed? # => false
241
+ result.categories # => [:hate, :"hate/threatening"]
242
+ result.scores # => { "hate" => 0.97, "hate/threatening" => 0.81 } (0..1 for service adapters)
243
+ result.labels # => [#<Label category: :hate, subcategory: :threatening, score: 0.81, input: :text>, …]
244
+ ```
245
+
246
+ ### Filter adapters (the built-in wordlist, reference adapters, your own — one interface)
247
+
248
+ Every backend implements the same tiny contract — `classify(value) → Moderate::Result` — so they're interchangeable per field. `moderate` ships exactly **one** built-in adapter, the offline `:wordlist`; OpenAI, AWS Rekognition, and anything else are **bring-your-own** — copy a ready-made reference adapter from [`examples/`](examples/), add its gem to *your* Gemfile, and `register_adapter` it:
249
+
250
+ | Adapter | Use it for | Notes |
251
+ | --- | --- | --- |
252
+ | `:wordlist` (built-in, default) | text | Fast offline baseline, multilingual, zero-dependency. Includes Unicode normalization and common substitution handling, but it is not a contextual classifier. Ships `en`/`es` lists; add your own. The only adapter the gem ships. |
253
+ | OpenAI (reference adapter — [`examples/openai_moderation_adapter.rb`](examples/openai_moderation_adapter.rb)) | **text *and* image** | OpenAI `omni-moderation-latest` via the `ruby_llm` gem — **free**, multimodal, its category set IS the canonical taxonomy + `0..1` scores. Copy it in, `gem "ruby_llm"`, `register_adapter(:openai, …)`. Runs **async** (`Moderate::ClassifyJob`) in `:flag` mode. |
254
+ | AWS Rekognition (reference adapter — [`examples/aws_rekognition_adapter.rb`](examples/aws_rekognition_adapter.rb)) | images / avatars | `detect_moderation_labels` via `aws-sdk-rekognition`, with its taxonomy mapped onto the canonical labels. Copy it in, `gem "aws-sdk-rekognition"`, `register_adapter(:rekognition, …)`. Async, `:flag` mode. |
255
+ | *your own* | anything | `register_adapter(:replicate, …)` / Perspective / a self-hosted model — any object responding to `classify`. No built-in pretends the backend must be an "LLM". |
256
+
257
+ All adapters map their provider labels onto **one canonical taxonomy** (OpenAI's: `harassment[/threatening]`, `hate[/threatening]`, `sexual[/minors]`, `self-harm[/intent|/instructions]`, `violence[/graphic]`, `illicit[/violent]`), so `Moderate::Flag`, the DSA statement of reasons, and the transparency counters all speak one vocabulary.
51
258
 
52
- You can configure the `moderate` gem behavior by adding a `config/initializers/moderate.rb` file:
53
259
  ```ruby
54
260
  Moderate.configure do |config|
55
- # Custom error message when bad words are found
56
- config.error_message = "contains inappropriate language"
261
+ config.default_filter_mode = :block
262
+ config.filter_adapter = :wordlist
263
+
264
+ # Bring an external classifier: copy examples/openai_moderation_adapter.rb into
265
+ # your app, add `gem "ruby_llm"`, then register and use it by name.
266
+ config.register_adapter :openai, OpenAIModerationAdapter.new
267
+
268
+ config.filter "Message", :body, with: :wordlist, mode: :flag
269
+ config.filter "Profile", :avatar, with: :openai, mode: :flag # one adapter moderates text AND images
270
+ end
271
+ ```
272
+
273
+ > **`:block` requires a synchronous adapter** (`:wordlist`) — you can't reject a save on a background result. The async reference adapters (the OpenAI/Rekognition examples) declare `synchronous? == false`, so they run in `:flag` mode (allow the write, classify in a job, file a `Moderate::Flag`). `moderate` validates this for you and says so.
274
+
275
+ Bring your own adapter — it's just an object that responds to `classify`:
276
+
277
+ ```ruby
278
+ class MyAdapter
279
+ def classify(value) = Moderate::Result.new(allowed: ..., categories: [...], scores: {...})
280
+ end
281
+ Moderate.register_adapter(:my_adapter, MyAdapter.new)
282
+ ```
283
+
284
+ > The original `moderate` (≤ 0.1) was *only* a profanity validator. That `validates :field, moderate: true` one-liner still works — it's now the `:wordlist` adapter in `:block` mode. See [Upgrading from 0.x](#upgrading-from-0x).
285
+
286
+ ## 🛠️ Admin & the moderation queue
287
+
288
+ Most of Trust & Safety happens in admin. `moderate` gives you the primitives; you bring the UI.
289
+
290
+ ```ruby
291
+ Moderate::Report.pending # the report queue
292
+ Moderate::Flag.pending # the auto-filter queue (human OR ML consumer reads the same scope)
293
+ Moderate::Appeal.pending # appeals awaiting a human
294
+
295
+ report.resolve!(by: admin, remove_content: true, ban_user: false, note: "Removed: hate speech")
296
+ report.dismiss!(by: admin, note: "No violation")
297
+ appeal.uphold!(by: admin, note: "...") # overturns the decision
298
+ appeal.reject!(by: admin, note: "...")
299
+ ```
57
300
 
58
- # Add your own words to the blacklist
59
- config.additional_words = ["badword1", "badword2"]
301
+ Every action is atomic, requires a moderator + a note, runs your enforcement (content removal via the reportable's own `remove_reported_field!`, bans via your `ban_handler`), and writes to your audit log.
60
302
 
61
- # Exclude words from the default list (false positives)
62
- config.excluded_words = ["good"]
303
+ ### Use it from a controller (BYOUI)
304
+
305
+ ```ruby
306
+ class Admin::ReportsController < ApplicationController
307
+ include Moderate::Moderation # resolve!/dismiss! actions, strong params, redirects
308
+
309
+ before_action :require_admin
63
310
  end
64
311
  ```
65
312
 
313
+ ### Integrate with [`madmin`](https://github.com/excid3/madmin)
314
+
315
+ `moderate`'s models are plain ActiveRecord, so they show up in `madmin` like anything else. Generate a resource and point it at the model:
316
+
317
+ ```bash
318
+ rails generate madmin:resource Moderate::Report
319
+ ```
320
+
321
+ Then wire the resolve/dismiss actions to `Moderate::Report#resolve!`/`#dismiss!` from a custom member action (full walkthrough in [`docs/madmin.md`](docs/madmin.md)). The same pattern works for `Moderate::Flag` and `Moderate::Appeal`.
322
+
323
+ ## 🔔 Notifications & 🧾 audit — one hook each
324
+
325
+ `moderate` never sends an email or writes to *your* audit log directly. It **emits events** through two hooks you wire once — so notifications fan out wherever you want, and important actions are recorded however you want.
326
+
327
+ ```ruby
328
+ Moderate.configure do |config|
329
+ # Called for every important action — wire it to your audit system (or leave it; it no-ops):
330
+ config.audit = ->(event) { AuditLog.record!(event_type: event.name, data: event.payload) }
331
+
332
+ # Called for every notifiable event — fan out to email / admin Telegram / push / in-app:
333
+ config.notify = ->(event) do
334
+ case event.name
335
+ when :report_received, :report_decision, :affected_user_decision
336
+ ModerationMailer.with(event:).public_send(event.name).deliver_later # goodmail
337
+ when :content_flagged, :report_received
338
+ Telegrama.send_message("🚩 #{event.payload[:summary]}") # admin alert
339
+ end
340
+ end
341
+
342
+ # Optional side effects when a block happens (e.g. tear down a pending invite):
343
+ config.on_block = ->(blocker:, blocked:, at:) { CancelPendingInvites.call(blocker, blocked, at: at) }
344
+ end
345
+ ```
346
+
347
+ Events carry a stable envelope (`event.name`, `event.subject`, `event.recipients`, `event.actor`, `event.payload`), so a single `notify` hook can drive **goodmail** (user emails), **telegrama** (admin alerts), and **noticed** (in-app feed + push) at once. Notify users via email/in-app **and** ping admins on Telegram from the same place. (Recipes in [`docs/notifications.md`](docs/notifications.md).)
348
+
349
+ The full event vocabulary: `report_received`, `report_decision`, `affected_user_decision`, `appeal_received`, `appeal_decision`, `user_blocked`, `user_unblocked`, `user_banned`, `content_flagged`, `content_removed`.
350
+
351
+ ## ⚖️ DSA & app-store compliance, out of the box
352
+
353
+ `moderate` is built around the rules so you don't have to read the regulation:
354
+
355
+ - **DSA Art. 16 (notice & action):** a public, electronic notice form — a mountable engine you place at the path of your choosing (`mount Moderate::Engine => "/trust"`, no hardcoded `/legal`) — capturing the substantiated reason, exact URL, notifier name+email, good-faith statement, the EU **statement-of-reasons taxonomy**, and the member-state selector, with an automatic confirmation of receipt. A notice is a `Moderate::Report` with `intake_kind: "dsa"` (no separate model), built via `Moderate::Services::IntakeNotice`. The form prefills the reported-content fields from query params (editable) and a signed-in notifier's identity (locked), and auto-integrates [`rails_cloudflare_turnstile`](https://github.com/instrumentl/rails-cloudflare-turnstile) when present (falling back to a `config.notice_guard` proc, with an optional per-request skip hook for clients that cannot render a browser challenge). See [`docs/dsa-notice-form.md`](docs/dsa-notice-form.md).
356
+ - **DSA Art. 17 (statement of reasons):** decision notices state the action, the legal/contractual ground, whether automated means were used, and the redress path.
357
+ - **DSA Art. 20 (appeals):** a free, electronic internal complaint mechanism, open ≥ 6 months, decided by a human.
358
+ - **DSA Art. 24 (transparency):** counters you can publish (notices received, actions taken, median handling time, appeal outcomes). The public transparency page is **opt-in** (`config.transparency_report_enabled = true`, off by default) — a live portal isn't itself required (the duty is to *publish* a report, and micro/small enterprises are exempt), so you turn it on only when you want it.
359
+ - **Apple Guideline 1.2 & Google Play UGC:** filter-before-post, in-app report **and** block, ongoing moderation, published contact — `moderate` covers all four. See the mapped checklist in [`docs/compliance.md`](docs/compliance.md).
360
+
361
+ > Two taxonomies, on purpose: an in-app **community report** category set (harassment, spam, …) and a separate, regulator-aligned **DSA legal-reason** taxonomy for public notices. `moderate` ships both. The community set is host-customizable via `config.report_categories`; the DSA legal-reason taxonomy is regulator-defined and fixed.
362
+
363
+ ## 🤓 Why the models?
364
+
365
+ `rails generate moderate:install` creates four tables:
366
+
367
+ - **`moderate_reports`** — a report/notice + an immutable evidence snapshot + decision metadata + the appeal window. Serves both in-app reports and public DSA notices (distinguished by `intake_kind`).
368
+ - **`moderate_blocks`** — the bidirectional `blocker`/`blocked` edge, with a self-block check and the SSOT relation behind `Moderate.blocked_ids_for`.
369
+ - **`moderate_flags`** — system/auto-filter flags (source: `text_filter` / `image_filter` / `external_classifier` / `manual`), with the classifier's labels + scores; the queue both human admins and ML consumers read via `pending`.
370
+ - **`moderate_appeals`** — DSA Art. 20 internal complaints against a decision.
371
+
372
+ > The value-list taxonomies (community `category`, `status`, `content_type`, the DSA `legal_reason`/`legal_country_code`, `resolution_basis`, plus `Flag` source/mode/status and `Appeal` source/status) are validated **in the models** — frozen constants + ActiveModel `inclusion` validations — **not** by database `CHECK` constraints. That means **adding or customizing a label never requires a migration**: the community category list is host-overridable via `config.report_categories` (defaults to `Moderate::Report::DEFAULT_CATEGORIES`), and the gem can grow its own taxonomies in a point release without touching your schema. The only value guard kept at the DB level is a cheap message-length backstop; everything else the migration adds is structural (NOT NULLs, FKs, the unique block edge, and the self-block CHECK).
373
+
374
+ The migration is **adaptive**: it matches your app's primary-key type (UUID or bigint) and JSON column type (`jsonb` / `json`) automatically, so it drops cleanly into any Rails 7.1+ schema.
375
+
376
+ ## Configuration reference
377
+
378
+ ```ruby
379
+ Moderate.configure do |config|
380
+ config.user_class = "User" # who reports/blocks/gets banned
381
+ config.default_filter_mode = :block # :off / :block / :flag
382
+ config.filter_adapter = :wordlist # default text adapter
383
+
384
+ config.audit = ->(event) { ... } # optional; no-op by default
385
+ config.notify = ->(event) { ... } # optional; no-op by default
386
+ config.on_block = ->(blocker:, blocked:, at:) { ... } # optional
387
+ config.ban_handler = ->(user:, by:, reason:) { user.suspend! } # how a "ban" is applied in your app
388
+
389
+ config.filter "Message", :body, with: :wordlist, mode: :flag
390
+ end
391
+ ```
392
+
393
+ Reportable classes are auto-discovered from the `has_reportable_content` macro (or `include Moderate::Reportable`) — no manual registry.
394
+
395
+ ## Upgrading from 0.x
396
+
397
+ `moderate` 1.0 is a ground-up rewrite: the old gem was a profanity validator; 1.0 is a full Trust & Safety system. The one piece of the old API that remains is the validator, now backed by the `:wordlist` adapter:
398
+
399
+ ```ruby
400
+ validates :body, moderate: true # still works — equivalent to `moderates :body, mode: :block`
401
+ ```
402
+
403
+ Everything else is new. There's no automated data migration (0.x stored nothing). See [`CHANGELOG.md`](CHANGELOG.md).
404
+
405
+ ## Testing
406
+
407
+ We use Minitest. Run the suite (a dummy Rails app under `test/dummy`, against SQLite/PostgreSQL/MySQL via Appraisals):
408
+
409
+ ```bash
410
+ bundle exec rake test
411
+ ```
412
+
66
413
  ## Development
67
414
 
68
- After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
415
+ After checking out the repo, run `bin/setup` to install dependencies. Then run `rake test`. You can also run `bin/console` for an interactive prompt.
69
416
 
70
417
  To install this gem onto your local machine, run `bundle exec rake install`.
71
418