decidim 0.29.2 → 0.30.0.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cfd0fd311aafc5b4ba94457a9f19a5f8cf6bf4a7ca9d53b76f13631104c87da7
-  data.tar.gz: 8d5340c17877bf7ce67940bcbbd88a1bf3cb3f6eeb2a55a6d73466378003f528
+  metadata.gz: b0b6d4745bd4402702b958b586f5ceae765d0aeb700679c76188a99153e2baf7
+  data.tar.gz: b7e81f9856aa1dce039bc61fd41b526b5afa2b36c768cb35041b7a5f88acc91e
 SHA512:
-  metadata.gz: e52b078d48d1ce5128005863d1d9256a28a441b2c1a6a9bee365a0b8d256a987bdee753f40d4a7f258b138276b3b4e662223618325bc2a1257852dac5d146bad
-  data.tar.gz: 29c0a84e18b6e5577e8440f141c380377ccd7da117e10e9af44614b3422bcbd1098735d0736fe6a0d1de374966d422204dbec7199b4f368ccd1ecd63449b444d
+  metadata.gz: 2da4f1073487e3f9549de150f87f159bda559dedf1f7b79bc39702686585782e3ddf5e690898dee631bb74a338829a7771f266da9939dd571d1b562a517291e0
+  data.tar.gz: d9e0d83a5e3b79be2f51b78f1c9dae8818b1867b353cda1a184f5aa974d02f4ada20f448e9bf6065b05375e94409e7b126e65a6cfb3bc39b5a2a39b84476145a

data/decidim.gemspec

@@ -18,7 +18,7 @@ Gem::Specification.new do |s|
     "homepage_uri" => "https://decidim.org",
     "source_code_uri" => "https://github.com/decidim/decidim"
   }
-  s.required_ruby_version = "~> 3.2.0"
+  s.required_ruby_version = "~> 3.3.0"
 
   s.name = "decidim"
 

data/docs/antora.yml

@@ -1,6 +1,6 @@
 name: en
 title: "Decidim Documentation"
-version: v0.29
+version: v0.30
 asciidoc:
   attributes:
     page-lang: en@

data/docs/modules/configure/pages/environment_variables.adoc

@@ -628,6 +628,30 @@ Additional context: This has been revealed as an issue during a security audit o
 |false
 |No
 
+|*DECIDIM_ENABLE_ETIQUETTE_VALIDATOR*
+|Enable/Disable Etiquette validator. The etiquette validator is applied to the create and edit forms of Proposals, Meetings, and Debates for both regular and admin users.
+|true
+|No
+
+|*DECIDIM_SPAM_DETECTION_BACKEND_USER*
+| The adapter type needed for user classifier. (for CI purposes, you can set "memory")
+| redis
+| No
+
+|*DECIDIM_SPAM_DETECTION_BACKEND_USER_REDIS_URL*
+| The redis connection url used by the user classifier
+| redis://localhost:6379/3
+| No
+
+|*DECIDIM_SPAM_DETECTION_BACKEND_RESOURCE*
+| The adapter type needed for resource classifier. (for CI purposes, you can set "memory")
+| redis
+| No
+
+|*DECIDIM_SPAM_DETECTION_BACKEND_RESOURCE_REDIS_URL*
+| The redis connection url used by the resource classifier
+| redis://localhost:6379/2
+| No
 
 |===
 

data/docs/modules/configure/pages/initializer.adoc

@@ -508,3 +508,29 @@ Additional Reading:
 
 * https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy[Content-Security-Policy in MDN web docs]
 * xref:customize:content_security_policy.adoc[Customize Content Security Policy]
+
+== Etiquette validator configuration
+
+This configuration option is used to enable or disable the etiquette validator in your Ruby on Rails application. The etiquette validator is responsible for ensuring that user-generated content meets specific formatting guidelines.
+
+=== Description
+
+The etiquette validator is applied to the create and edit forms of Proposals, Meetings, and Debates for both regular and admin users. Admin users can use the admin panel to use these forms. The validator checks for the following conditions:
+
+* `too_much_caps`: Ensures that the content does not contain an excessive amount of capital letters.
+* `too_many_marks`: Ensures that the content does not contain too many punctuation marks.
+* `must_start_with_caps`: Ensures that the content starts with a capital letter.
+
+=== Configuration
+
+To disable the etiquette validator, set the following in your application's configuration file:
+[source,ruby]
+....
+config.enable_etiquette_validator = false
+....
+
+Alternatively, you can use an environment variable:
+[source,ruby]
+....
+DECIDIM_ENABLE_ETIQUETTE_VALIDATOR
+....

data/docs/modules/configure/pages/system.adoc

@@ -0,0 +1,145 @@
+= System
+
+Every Decidim is multi-tenant by default. That means that you can have multiple organizations inside a same installation.
+This is being used for instance for the Barcelona and Girona Provincial Governments, for offering Decidim installations
+to many small cities, or by the City of Barcelona, for giving service to lots of organizations (associations,
+federations, cooperatives, etc).
+
+Even if you only have one organization you need to create it through the System Panel.
+It is available after you have installed Decidim, at `/system`. For instance if your domain is `example.org`,
+then it should be available at `http://example.org/system`.
+
+After you have freshly installed Decidim, you are redirected to the System panel log-in page:
+
+image::system-log_in.png[system panel log in]
+
+== 1. Create a system admin account
+
+For logging in to this dashboard, you need to create a system admin account from your terminal:
+
+[source, console]
+....
+bin/rails decidim_system:create_admin
+....
+
+You will be asked for an email and a password.
+For security, the password will not get displayed back at you and you need to confirm it.
+
+== 2. Access the Dashboard
+
+With the email and password that you have added in the past step, you are able to log in.
+After you have logged in, you should see an empty dashboard:
+
+image::system-dashboard.png[system empty dashboard]
+
+== 3. Create a new organization
+
+Inside of the system panel, you need to create your Organization through the form:
+
+TIP: Please mind that some of these fields cannot change through the System Form, as changing them could
+potentially leave your database in a broken state. You can change it through the console, but you will be on your own.
+
+=== General settings
+
+.System - New Organization form general settings
+[cols="20h,10h,~"]
+|===
+|Field |Type |Description
+
+|Name
+|Required
+|Name of your organization. For instance, "Decidim Barcelona".
+
+|Reference prefix
+|Required
+|The reference prefix is used to uniquely identify resources across all organizations. For instance, "BCN". You can change how
+it is created through xref:configure:initializer.adoc[Decidim's initializer] (see **Custom resource reference**).
+
+|Host
+|Required
+|Fully qualified domain name. For instance, "example.org" or "decidim.barcelona".
+
+|Secondary hosts
+|Optional
+|Others domains or subdomains that point to this same installation. Normally this should not be used.
+Enter each one of them in a new line. For instance, "example.com".
+
+|Organization admin name
+|Required
+|Name of the admin account. For instance, "Admin"
+
+|Organization admin email
+|Required
+|Email for the admin account. For instance, "admin@example.org"
+
+|Organization locales
+|Required
+|You need to configure at least one language. Decidim is multilingual, meaning that you can have most of the admin
+contents with any number of languages. For seeing other languages you need to configure first the
+xref:configure:initializer.adoc[Decidim's initializer].
+
+|Force authentication
+|Optional
+|Check if you want to **Force users to authenticate before access organization**.
+That means visitor they do not see any published content and they only have access to the log-in form.
+
+|Users registration mode
+|Optional
+|Control how you want the login / participant register or signup to work. See below.
+
+|Available authorizations
+|Optional
+|How you want to authorize and verify that your participants are who they say they are.
+It depends a lot on your Organization, country, etc. See xref:customize:authorizations.adoc[Authorizations customization].
+
+|===
+
+=== Advanced settings
+
+.System - New Organization form advanced settings
+[cols="20h,10h,~"]
+|===
+|Field |Type |Description
+
+|SMTP settings
+|Optional
+|Allows you to configure different SMTP servers for every organization in this installation.
+See xref:services:smtp.adoc[SMTP configuration].
+
+|Omniauth settings
+|Optional
+|Allows you to configure through external OAUTH providers for every Organization in this installation.
+See xref:services:social_providers.adoc[Social providers configuration].
+
+|File upload settings
+|Optional
+|Allows you to configure the formats accepted on your instance, as well as default file sizes for file uploads
+
+|Content security policy
+|Optional
+|Allows you to configure organization based rules, but also to configure system wide policies in case your decidim instance is running
+in multi tenant mode. To learn more, check this https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP[CSP header documentation].
+
+|===
+
+=== Users registration mode
+
+==== Allow participants to register and login
+
+Most of the installations use this setting. Means that everyone can register a participant account and login, without restrictions, only with an email account confirmation.
+You can disable that confirmation also if you want to through xref:configure:initializer.adoc[Decidim's initializer], although we do not recommend doing that,
+as it is a measure for stopping spam accounts.
+
+==== Do not allow participants to register, but allow existing participants to login
+
+This is for when you make a manual import of participants' accounts.
+
+==== Access only can be done with external accounts
+
+In the case where you have an external provider configured (like the `Omniauth settings`) and you want to have accounts from only this provider.
+
+== Edit an organization
+
+Finally, you can also Edit an organization, although some fields cannot be changed as some contents in the database depends on them.
+
+For accessing your newly created Organization, you can do it by going to the domain or subdomain that you have configured in `Host`.

data/docs/modules/customize/pages/menu.adoc

@@ -57,7 +57,7 @@ After applying the below initializer, the menu rendering will be modified to:
 # config/initializers/decidim.user_menu.rb
 
 Decidim.menu :user_menu do |menu|
-  menu.remove_item :user_interests
+  menu.remove_item :authorizations
   menu.move :own_user_groups, after: :account
   menu.move :authorizations, before: :notifications_settings
 end

data/docs/modules/develop/pages/ai_tools/lang_detection_formatter.adoc

@@ -0,0 +1,9 @@
+= Language Detection Formatter
+
+```ruby
+class Formatter
+  def cleanup(text)
+    strip_tags(text)
+  end
+end
+```

data/docs/modules/develop/pages/ai_tools/spam_detection_analyzer.adoc

@@ -0,0 +1,20 @@
+= Spam detection Analyzer
+
+```ruby
+class ResourceAnalyzer < Decidim::Ai::SpamDetection::Resource::Base
+  def fields
+    [:title, :body, :etc]
+  end
+
+ protected
+
+  def query
+    Your::Model.including()
+  end
+
+  def resource_hidden?(resource); end
+  def batch_train; end
+  def train; end #  delegated to classifier
+  def untrain; end #  delegated to classifier
+end
+```

data/docs/modules/develop/pages/ai_tools/spam_detection_service.adoc

@@ -0,0 +1,25 @@
+= Spam detection Service
+
+```ruby
+class SpamDetection::Service
+  def initialize
+    @registry = Decidim::Ai.spam_detection_registry
+  end
+
+  def train(category, text)
+    # train the strategy
+  end
+
+  def classify(text)
+    # classify the text
+  end
+
+  def untrain(category, text)
+    # untrain the strategy
+  end
+
+  def classification_log
+    # return the classification log
+  end
+end
+```

data/docs/modules/develop/pages/ai_tools/spam_detection_strategy.adoc

@@ -0,0 +1,43 @@
+= Spam detection Strategy
+
+```ruby
+module SpamDetection::Strategy
+  class Bayes < Base
+    def initialize(options = {})
+      # Add here your configuration, assign your variables
+    end
+
+    def log
+      return unless category
+
+      "The Classification engine marked this as #{category}"
+    end
+
+    def train(category, text)
+      # some call to the original backend
+    end
+
+    # Calling this method without any trained categories will throw an error
+    def untrain(category, content)
+      # some call to the original backend
+    end
+
+    def reset
+      # some call that actually resets the backend data
+    end
+
+    def classify(content)
+      @category, @internal_score = backend.classify_with_score(content)
+      category
+    end
+
+    def score
+      category.presence == "spam" ? 1 : 0
+    end
+
+    private
+
+    # your implementation
+  end
+end
+```

data/docs/modules/develop/pages/ai_tools/spam_detection_trainer.adoc

@@ -0,0 +1,83 @@
+= Real-time training
+
+In some instances, there may be the need to add Real-time training of the spam data. To achieve that, Decidim allows you to subscribe to 2 different events. Take into account that if you follow this strategy, you could have lots of false positives, as some spammers are using relevant content provided by participants to "cheat" the system.
+
+== Resource training
+
+In order to enable realtime training for based on your data, you will need to subscribe to the `decidim.admin.hide_resource:after` event by adding to your `config/initializers/ai_tools.rb` file the following content:
+
+```ruby
+Decidim::EventsManager.subscribe("decidim.admin.hide_resource:after") do |_event_name, data|
+  Decidim::MyModule::TrainHiddenResourceDataJob.perform_later(data[:resource])
+end
+```
+
+In this case `data[:resource]` is the actual resource that is being hidden.
+A class that would train / untrain that may look like:
+
+```ruby
+# frozen_string_literal: true
+
+module Decidim
+  module MyModule
+    class TrainHiddenResourceDataJob < ApplicationJob
+      include Decidim::TranslatableAttributes
+
+      def perform(resource)
+        return unless resource.respond_to?(:hidden?)
+
+        resource.reload
+
+        wrapped = Decidim::Ai::SpamDetection.resource_models[resource.class.name].constantize.new
+
+        if resource.hidden?
+          wrapped.fields.each do |field|
+            wrapped.untrain :ham, translated_attribute(resource.send(field))
+            wrapped.train :spam, translated_attribute(resource.send(field))
+          end
+        end
+      end
+    end
+  end
+end
+```
+
+== User training
+As you can do the same as above for the user.
+
+```ruby
+Decidim::EventsManager.subscribe("decidim.admin.block_user:after") do |_event_name, data|
+  Decidim::MyModule::TrainUserDataJob.perform_later(data[:resource])
+end
+```
+
+In this case `data[:resource]` is the actual user that is being hidden.
+A class that would train / untrain that may look like:
+
+```ruby
+# frozen_string_literal: true
+
+module Decidim
+  module MyModule
+    class TrainUserDataJob < ApplicationJob
+      def perform(user)
+        user.reload
+
+        if user.blocked?
+          classifier.untrain :ham, user.about
+          classifier.train :spam, user.about
+        else
+          classifier.untrain :spam, user.about
+          classifier.train :ham, user.about
+        end
+      end
+
+      protected
+
+      def classifier
+        @classifier ||= Decidim::Ai::SpamDetection.user_classifier
+      end
+    end
+  end
+end
+```

data/docs/modules/develop/pages/ai_tools.adoc

@@ -0,0 +1,12 @@
+= AI tools for decidim
+
+== Configuration reference
+* xref:services:aitools.adoc[AI Tools configuration]
+
+== Class Specific reference
+
+* xref:develop:ai_tools/lang_detection_formatter.adoc[Language Detection Formatter]
+* xref:develop:ai_tools/spam_detection_strategy.adoc[Spam detection Strategy]
+* xref:develop:ai_tools/spam_detection_service.adoc[Spam detection Service]
+* xref:develop:ai_tools/spam_detection_analyzer.adoc[Spam detection Analyzer]
+* xref:develop:ai_tools/spam_detection_trainer.adoc[Real-time training]

data/docs/modules/develop/pages/classes/models.adoc

@@ -73,6 +73,7 @@ Most commonly used concerns are:
 - `Decidim::ScopableParticipatorySpace`
 - `Decidim::ScopableResource`
 - `Decidim::Searchable`
+- `xref:develop:taxonomies.adoc[Decidim::Taxonomizable]`
 - `xref:develop:share_tokens.adoc[Decidim::ShareableWithToken]`
 - `xref:develop:traceable.adoc[Decidim::Traceable]`
 - `xref:develop:machine_translations.adoc[Decidim::TranslatableAttributes]`
@@ -80,9 +81,9 @@ Most commonly used concerns are:
 
 === Module specific concerns
 
-- `Decidim::Comments::Commentable`
-- `Decidim::Comments::CommentableWithComponent`
-- `Decidim::Comments::HasAvailabilityAttributes`
+- `xref:develop:commentable.adoc[Decidim::Comments::Commentable]`
+- `xref:develop:commentable.adoc[Decidim::Comments::CommentableWithComponent]`
+- `xref:develop:commentable.adoc[Decidim::Comments::HasAvailabilityAttributes]`
 - `Decidim::Forms::HasQuestionnaire`
 - `Decidim::Initiatives::HasArea`
 - `Decidim::Initiatives::InitiativeSlug`

data/docs/modules/develop/pages/commentable.adoc

@@ -0,0 +1,127 @@
+= Commentable
+
+== Things can be commentable
+
+`Commentable` is a feature to allow participants to comment on resources. This feature is used in many places in Decidim, like proposals, meetings, debates, etc.
+
+When commenting a resource, the comments counter is increased and a notification to all the followers of the participant, the participatory space and/or the resource is sent.
+
+Participants can comment with their own identity or with the identify of the `user_groups` they belong to.
+
+== Data model
+
+The `decidim_comments` table stores all the comments given to resources that are commentable. This is, one commentable has many comments, and each comment belongs to one commentable. Also, a comment can be a commentable itself, so comments can be nested.
+For performance, a commentable has a counter cache of comments.
+
+[source,ascii]
+----
++----------------------+
+|  Decidim::Commentable |
+|   ((Proposal,...))   |                                  +-------------+
++----------------------+  0..N +-------------------+   +--+Decidim::User|
+|-has_many comments    |-------+Decidim::Comment   |   |  +-------------+
+|#counter cache column |       +-------------------+   |
+|-comments_counter     |       |-author: may be a  |<--+
++----------------------+       |         user or a |   |
+                               |         user_group|   |  +------------------+
+                               +-------------------+   +--+Decidim::UserGroup|
+                                                          +------------------+
+----
+
+Thus, each commentable must have the comments counter cache column.
+This is an example migration to add the comments counter cache column to a resource:
+
+[source,ruby]
+----
+class AddCommentableCounterCacheToMeetings < ActiveRecord::Migration[5.2]
+  def change
+    add_column :decidim_meetings_meetings, :comments_count, :integer, null: false, default: 0
+    Decidim::Meetings::Meeting.reset_column_information
+    Decidim::Meetings::Meeting.find_each(&:update_comments_count)
+  end
+end
+----
+
+== Public view
+
+=== The "Comment" cell
+
+The `Comment` cell is used to render a comment in the public view. Usually, it is used in the `Comments` cell to render a list of comments.
+
+[source,ruby]
+----
+cell("decidim/comments", resource)
+----
+
+This will render all the comments for the given resource.
+
+=== Comments actions
+
+Each comment has a set of actions that can be performed by the user. By default, these actions are available:
+
+- A user can edit and delete their own comments.
+- A user can report a comment.
+- A user can copy the link to a comment.
+
+These actions are available through a dropdown menu in the comment.
+
+=== Resource-specific extra actions
+
+Each resource can define extra actions that can be performed on comments. These actions must be returned by the resource object through the `actions_for_comment` method. If the resource does not define this method, or returns empty, no extra actions will be available.
+
+This method will receive two parameters, the comment itself and the current user that wants to interact with the comment.
+
+It must return an array of hashes, where each hash has the following keys:
+
+- `label`: The text for the link for the action.
+- `url`: The URL where the action will be performed.
+- `icon`: The icon to be displayed for the action (optional).
+- `method`: The HTTP method for the action, usually `:post` or `:delete` (optional as it defaults to `get`).
+- `data`: A hash with the data attributes for the link (optional).
+
+All these extra actions will be displayed in the dropdown menu of the comment after the default ones.
+
+For example, this is how the `Proposal` model defines extra actions for comments:
+
+[source,ruby]
+----
+def user_has_actions?(user)
+  return false if authors.include?(user)
+  return false if user&.blocked?
+  return false if user&.deleted?
+  return false unless user&.confirmed?
+
+  true
+end
+
+def actions_for_comment(comment, current_user)
+  return if comment.commentable != self
+  return unless authors.include?(current_user)
+  return unless user_has_actions?(comment.author)
+
+  if coauthor_invitations_for(comment.author).any?
+    [
+      {
+        label: I18n.t("decidim.proposals.actions.cancel_coauthor_invitation"),
+        url: EngineRouter.main_proxy(component).cancel_proposal_invite_coauthors_path(proposal_id: id, id: comment.author.id),
+        icon: "user-forbid-line",
+        method: :delete,
+        data: { confirm: I18n.t("decidim.proposals.actions.cancel_coauthor_invitation_confirm") }
+      }
+    ]
+  else
+    [
+      {
+        label: I18n.t("decidim.proposals.actions.mark_as_coauthor"),
+        url: EngineRouter.main_proxy(component).proposal_invite_coauthors_path(proposal_id: id, id: comment.author.id),
+        icon: "user-add-line",
+        method: :post,
+        data: { confirm: I18n.t("decidim.proposals.actions.mark_as_coauthor_confirm") }
+      }
+    ]
+  end
+end
+----
+
+This will render a new menu item with the text "Mark as coauthor" or "Cancel coauthor invitation" depending on the state of the comment author that will allow to add the comment's author as a co-author of the proposal.
+

data/docs/modules/develop/pages/endorsable.adoc

@@ -70,7 +70,7 @@ Given that some settings have been defined in the Administration Panel, for the
 
 === The "Endorse" buttons cell
 
-It normally appears in the resource detail view (show). At the action card, in right-side of the view.
+It normally appears in the resource detail view (show), at the bottom of the resource content, but above the comments when comments are enabled.
 It allows users to endorse with any of their identities, the personal one, and/or their user_groups', if any.
 It also shows the current number of endorsements for this resource.
 
@@ -81,34 +81,25 @@ To render this button, `decidim-core` offers the `decidim/endorsement_buttons` c
 cell("decidim/endorsement_buttons", resource)
 ----
 
-This cell, renders the endorsements counter and the endorsement button by default. But it has the possibility to be invoked to render elements separately.
+This cell will render the endorsement buttons depending on whether user endorsed the resource or not. The endorsements are labeled as *Likes*.
 
 [source,ruby]
 ----
 # By default the `show` method is invoked as usual
 # Renders `render_endorsements_count` and `render_endorsements_button` in a block.
-cell("decidim/endorsement_buttons", resource)
-# It is recommended to use the `endorsement_buttons_cell` helper method
-endorsement_buttons_cell(resource)
-
-# Renders the "Endorse" button
+#
 # It takes into account:
 # - if endorsements are enabled
 # - if users are logged in
 # - if users can endorse with many identities (of their user_groups)
 # - if users require verification
-endorsement_buttons_cell(resource).render_endorsements_button
-
-# Renders the counter of endorsements that appears in card.
-endorsement_buttons_cell(resource).render_endorsements_count
-
-# Renders a button to perform the endorse action, but only with the personal identity of the user. It does not take into account if the user belongs to any user group.
-endorsement_buttons_cell(resource).render_user_identity_endorse_button
+ #
+cell("decidim/endorsement_buttons", resource)
 ----
 
 === The list of endorsers
 
-The `Decidim::EndorsersListCell` renders the list of endorsers of a resource. It is usually rendered in the show of the resource, just upside the comments.
+The `Decidim::EndorsersListCell` renders the list of endorsers of a resource. It is usually rendered in the show page of the resource, just upside the comments. Additionally, this cell also renders the pop-up required to view the endorsers of a certain resource.
 
 [source,ruby]
 ----