decidim-file_authorization_handler 0.27.1.5

data/README.md

@@ -0,0 +1,137 @@
+# Decidim File Authorization Handler
+
+> A plugin to add a csv based authorization handler to the Decidim platform
+
+Allows admin users to upload a CSV file containing Document IDs and birthdates
+to a given organization.
+This information is used by a Decidim authorization handler to authorize real
+users.
+
+## Usage
+
+This module provides a model `Decidim::FileAuthorizationHandler::CensusDatum`
+to store census information (identity document and birth date) in a hashed way
+(real data is never stored).
+
+It has an admin controller to upload CSV files with the information. When
+importing files all records are inserted and the duplicates are removed in a
+background job for performance reasons.
+
+To keep the plugin simple the uploaded file is processed when the file is
+uploaded.
+Uploading the file to a temporary storage system and processing it in
+background is kept out of the scope for the first release.
+
+### CSV file format
+
+The CSV file format is not configurable, but is extendable. The plugin expects a comma separated
+CSV with headers:
+
+```console
+ID_NUMBER,BIRTH_DATE
+00000000Z,07/03/2014
+```
+
+- ID_NUMBER: No format restrictions
+- BIRTHDATE: `dd/mm/YYYY` format
+
+The CSV separator can be overriden at installation level by using an
+initializer:
+
+```ruby
+# config/initializers/file_authorization_handler.rb
+Decidim::FileAuthorizationHandler::CsvData.col_sep= ";"
+```
+
+#### Extra columns
+
+Sometimes extra fields must be provided in order for Authorizers to do some census segmentation. This, for example,
+happens when a specific authorizer checks for the district of the citizen.
+
+Since v0.26.2.5 this module supports a variable number of extra columns for each user. These columns are then persisted
+in the `CensusDatum#extras` column as a hash of fields.
+
+Take, for example, the following CSV file
+
+```console
+ID_NUMBER,BIRTH_DATE,DISTRICT
+00000000Z,07/03/2014,17600
+```
+
+it will be imported into the `CensusDatum` as `{"district" => "17600"}`.
+
+Then, once the user is verified, all `CensusDaum#extras` will be added to the `Authorization#metadata` together with the `birthdate`. This way, custom authorizers will be able to make use of this information.
+
+Implementors and administrators must respect User Privacy and be aware of the corresponding Rules.
+
+### Overriding Authorization Handler name
+
+The authorization handler name is a default one not suitable for end users.
+Depending on the use case of the gem needs to be overriden accordingly.
+
+Override the following keys to modify the name of the authorization handler and
+its description:
+
+`decidim.authorization_handlers.file_authorization_handler.name`
+`decidim.authorization_handlers.file_authorization_handler.description`
+
+You can also override the name of the ID document and birthdate fields:
+
+`activemodel.attributes.file_authorization_handler.id_document`
+`activemodel.attributes.file_authorization_handler.birthdate`
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'decidim-file_authorization_handler'
+```
+
+And then execute:
+
+```bash
+bundle
+bin/rails decidim_file_authorization_handler:install:migrations
+bin/rails db:migrate
+```
+
+Finally, add the following line to your `config/routes.rb` file:
+
+```ruby
+mount Decidim::FileAuthorizationHandler::AdminEngine => '/admin'
+```
+
+## Run tests
+
+Node 16.9.1 is required!
+
+Create a dummy app in your application (if not present):
+
+```bash
+bin/rails decidim:generate_external_test_app
+cd spec/decidim_dummy_app/
+bundle exec rails decidim_file_authorization_handler:install:migrations
+RAILS_ENV=test bundle exec rails db:migrate
+```
+
+And run tests:
+
+```bash
+bundle exec rspec spec
+```
+
+## Troubleshooting
+
+If you find the following error after you have installed the engine:
+
+```
+undefined method 'decidim_file_authorization_handler_admin_path'
+for module#<Module:0x00007fa2aa4e2a10>
+```
+
+review if you have mounted the Engine routes into your application routes.
+
+## License
+
+AGPLv3 (same as Decidim)

data/Rakefile

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require "decidim/dev/common_rake"

data/app/controllers/decidim/file_authorization_handler/admin/censuses_controller.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    module Admin
+      class CensusesController < Decidim::Admin::ApplicationController
+        def show
+          enforce_permission_to :show, :authorization
+          @status = Status.new(current_organization)
+        end
+
+        def create
+          enforce_permission_to :create, :authorization
+          if params[:file]
+            data = CsvData.new(params[:file].path)
+            # rubocop: disable Rails/SkipsModelValidations
+            CensusDatum.insert_all(current_organization, data.values, data.headers[2..])
+            # rubocop: enable Rails/SkipsModelValidations
+            RemoveDuplicatesJob.perform_later(current_organization)
+            flash[:notice] = t(".success", count: data.values.count,
+                                           errors: data.errors.count)
+          end
+          redirect_to censuses_path
+        end
+
+        def destroy
+          enforce_permission_to :destroy, :authorization, organization: current_organization
+          CensusDatum.clear(current_organization)
+          redirect_to censuses_path, notice: t(".success")
+        end
+      end
+    end
+  end
+end

data/app/jobs/decidim/file_authorization_handler/application_job.rb

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    class ApplicationJob < ActiveJob::Base
+    end
+  end
+end

data/app/jobs/decidim/file_authorization_handler/remove_duplicates_job.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    class RemoveDuplicatesJob < ApplicationJob
+      queue_as :default
+
+      # rubocop:disable Style/HashSyntax
+      def perform(organization)
+        duplicated_census(organization).pluck(:id_document).each do |id_document|
+          CensusDatum.inside(organization)
+                     .where(id_document: id_document)
+                     .order(id: :desc)
+                     .all[1..]
+                     .each(&:delete)
+        end
+      end
+      # rubocop:enable Style/HashSyntax
+
+      private
+
+      def duplicated_census(organization)
+        CensusDatum.inside(organization)
+                   .select(:id_document)
+                   .group(:id_document)
+                   .having("count(id_document)>1")
+      end
+    end
+  end
+end

data/app/models/decidim/file_authorization_handler/application_record.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    class ApplicationRecord < ActiveRecord::Base
+      self.abstract_class = true
+    end
+  end
+end

data/app/models/decidim/file_authorization_handler/census_datum.rb

@@ -0,0 +1,70 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    class CensusDatum < ApplicationRecord
+      belongs_to :organization, foreign_key: :decidim_organization_id,
+                                class_name: "Decidim::Organization"
+      # An organzation scope
+      def self.inside(organization)
+        where(decidim_organization_id: organization.id)
+      end
+
+      # Search for a specific document id inside a organization
+      def self.search_id_document(organization, id_document)
+        CensusDatum.inside(organization)
+                   .where(id_document: normalize_and_encode_id_document(id_document))
+                   .order(created_at: :desc, id: :desc)
+                   .first
+      end
+
+      # Normalizes a id document string (remove invalid characters) and encode it
+      # to conform with Decidim privacy guidelines.
+      def self.normalize_and_encode_id_document(id_document)
+        return "" unless id_document
+
+        id_document = id_document.gsub(/[^A-z0-9]/, "").upcase
+        return "" if id_document.blank?
+
+        Digest::SHA256.hexdigest(
+          "#{id_document}-#{Rails.application.secrets.secret_key_base}"
+        )
+      end
+
+      # Convert a date from string to a Date object
+      def self.parse_date(string)
+        Date.strptime((string || "").strip, "%d/%m/%Y")
+      rescue StandardError
+        nil
+      end
+
+      # Insert a collection of values
+      def self.insert_all(organization, values, extra_headers = nil)
+        return if values.empty?
+
+        table_name = CensusDatum.table_name
+        columns = %w(id_document birthdate decidim_organization_id created_at).join(",")
+        columns = "#{columns},extras" if extra_headers.present?
+        now = Time.current
+        values = values.map do |row|
+          vals = "('#{row[0]}', '#{row[1]}', '#{organization.id}', '#{now}'"
+          if extra_headers.present?
+            extras = {}
+            extra_headers.present? && extra_headers.each_with_index do |header, idx|
+              extras[header.downcase] = row[2 + idx]
+            end
+            vals += ", '#{extras.to_json}'"
+          end
+          "#{vals})"
+        end
+        sql = "INSERT INTO #{table_name} (#{columns}) VALUES #{values.join(",")}"
+        ActiveRecord::Base.connection.execute(sql)
+      end
+
+      # Clear all census data for a given organization
+      def self.clear(organization)
+        CensusDatum.inside(organization).delete_all
+      end
+    end
+  end
+end

data/app/models/decidim/file_authorization_handler/csv_data.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+require "csv"
+
+module Decidim
+  module FileAuthorizationHandler
+    class CsvData
+      @col_sep = ","
+      class << self; attr_accessor :col_sep end
+
+      attr_reader :errors, :values, :headers
+
+      def initialize(file)
+        @file = file
+        @errors = []
+        @values = []
+
+        Rails.logger.info "CsvData.col_sep: #{CsvData.col_sep}"
+        CSV.foreach(@file, headers: true, col_sep: CsvData.col_sep) do |row|
+          @headers = row.headers
+          process_row(row)
+        end
+      end
+
+      private
+
+      def process_row(row)
+        id_document = CensusDatum.normalize_and_encode_id_document(row[0])
+        date = CensusDatum.parse_date(row[1])
+
+        if id_document.present? && !date.nil?
+          data = [id_document, date] + row[2..]
+          values << data
+        else
+          errors << row
+        end
+      end
+    end
+  end
+end

data/app/models/decidim/file_authorization_handler/status.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    # Provides information about the current status of the census data
+    # for a given organization
+    class Status
+      def initialize(organization)
+        @organization = organization
+      end
+
+      # Returns the date of the last import
+      def last_import_at
+        @last ||= CensusDatum.inside(@organization)
+                             .order(created_at: :desc).first
+        @last ? @last.created_at : nil
+      end
+
+      # Returns the number of unique census
+      def count
+        @count ||= CensusDatum.inside(@organization)
+                              .distinct.count(:id_document)
+        @count
+      end
+    end
+  end
+end

data/app/permissions/decidim/file_authorization_handler/admin/permissions.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Decidim
+  module FileAuthorizationHandler
+    module Admin
+      # Defines the abilities related to surveys for a logged in admin user.
+      class Permissions < Decidim::DefaultPermissions
+        def permissions
+          return permission_action if permission_action.scope != :admin
+
+          if user.organization.available_authorizations.include?("file_authorization_handler") &&
+             permission_action_in?(:show, :create, :destroy) &&
+             permission_action.subject == Decidim::FileAuthorizationHandler::CensusDatum
+            allow!
+          end
+          permission_action
+        end
+
+        private
+
+        def permission_action_in?(*actions)
+          actions.any? { |action| permission_action.action == action }
+        end
+      end
+    end
+  end
+end

data/app/services/file_authorization_handler.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+# An AuthorizationHandler that uses information uploaded from a CSV file
+# to authorize against the age of the user
+class FileAuthorizationHandler < Decidim::AuthorizationHandler
+  # This is the input (from the user) to validate against
+  attribute :id_document, String
+  attribute :birthdate, Decidim::Attributes::LocalizedDate
+
+  # This is the validation to perform
+  # If passed, an authorization is created
+  validates :id_document, presence: true
+  validates :birthdate, presence: true
+  validate :censed
+
+  def metadata
+    @metadata ||= begin
+      meta = { birthdate: census_for_user&.birthdate&.strftime("%Y/%m/%d") }
+      census_for_user&.extras&.each_pair do |key, value|
+        meta[key.to_sym] = value
+      end
+      meta
+    end
+  end
+
+  # This is required in new 0.8.4 version of decicim
+  # however, there's a bug and this doesn't work
+  def handler_name
+    +"file_authorization_handler"
+  end
+
+  # Checks if the id_document belongs to the census
+  def censed
+    return if census_for_user&.birthdate == birthdate
+
+    errors.add(:id_document, I18n.t("decidim.file_authorization_handler.errors.messages.not_censed"))
+  end
+
+  def authorized?
+    return true if census_for_user
+  end
+
+  def unique_id
+    return nil unless organization
+
+    Digest::SHA256.hexdigest("#{census_for_user&.id_document}-#{organization.id}-#{Rails.application.secrets.secret_key_base}")
+  end
+
+  def census_for_user
+    return unless organization
+
+    @census_for_user ||= Decidim::FileAuthorizationHandler::CensusDatum
+                         .search_id_document(organization, id_document)
+  end
+
+  def organization
+    current_organization || user&.organization
+  end
+end

data/app/views/decidim/file_authorization_handler/admin/censuses/show.html.erb

@@ -0,0 +1,39 @@
+
+<div class="card">
+  <div class="card-divider">
+    <h2 class="card-title">
+      <%= t('admin.show.title', scope: 'decidim.file_authorization_handler') %>
+    </h2>
+  </div>
+  <div class="card-section">
+    <% if @status.count > 0 %>
+      <p><%= t('decidim.file_authorization_handler.admin.show.data', count: @status.count,
+        due_date: l(@status.last_import_at, format: :long)) %>
+      </p>
+      <%= link_to t('decidim.file_authorization_handler.admin.destroy.title'),
+        censuses_path,
+        method: :delete,
+        class: 'button alert',
+        data: { confirm: t('decidim.file_authorization_handler.admin.destroy.confirm') } %>
+      <% else %>
+        <p><%= t('admin.show.empty', scope: 'decidim.file_authorization_handler') %></p>
+    <% end %>
+  </div>
+</div>
+
+
+<div class="card">
+  <div class="card-divider">
+    <h2 class="card-title">
+      <%= t('admin.new.title', scope: 'decidim.file_authorization_handler') %>
+    </h2>
+  </div>
+  <div class="card-section">
+    <p><%= t('decidim.file_authorization_handler.admin.new.info') %></p>
+    <%= form_tag censuses_path, multipart: true, class: 'form' do %>
+      <%= label_tag :name, t('admin.new.file', scope: 'decidim.file_authorization_handler') %>
+      <%= file_field_tag :file %>
+      <%= submit_tag t('admin.new.submit', scope: 'decidim.file_authorization_handler'), class: 'button' %>
+    <% end %>
+  </div>
+</div>

data/app/views/layouts/file_authorization_handler/application.html.erb

@@ -0,0 +1,14 @@
+<!DOCTYPE html>
+<html>
+<head>
+  <title>Census</title>
+  <%= stylesheet_link_tag    "census/application", media: "all" %>
+  <%= javascript_include_tag "census/application" %>
+  <%= csrf_meta_tags %>
+</head>
+<body>
+
+<%= yield %>
+
+</body>
+</html>

data/config/locales/ca.yml

@@ -0,0 +1,50 @@
+ca:
+  activemodel:
+    attributes:
+      file_authorization_handler:
+        id_document: Document d'identitat (DNI, NIF, Passaport o Targeta de Residència)
+        birthdate: Data de naixement
+  census_authorization:
+    form:
+      date_select:
+        day: Dia
+        month: Mes
+        year: Any
+  decidim:
+    authorization_handlers:
+      file_authorization_handler:
+        name: Cens Organització
+        explanation: Autoritza el teu compte d'usuari contra el cens de la Organització
+        type: CSV
+        fields:
+          birthdate: Data de naixement
+    file_authorization_handler:
+      errors:
+        messages:
+          not_censed: No hem pogut trobar el teu document d'identitat amb aquesta data de naixement al padró municipal. Si les dades són correctes i el problema persisteix, siusplau, posa't en contacte amb un administrador.
+          younger_than_minimum_age: Hauries de ser major de %{age} anys
+      admin:
+        destroy:
+          title: Esborrar totes les dades del cens
+          confirm: Aquesta acció no es pot desfer. Estàs segur/a que voleu continuar?
+        censuses:
+          create:
+            success: S'han importat amb èxit %{count} elements (%{errors} errors)
+          destroy:
+            success: S'han esborrat totes les dades censals
+        menu:
+          census: Pujar cens
+        show:
+          title: Dades del cens carregat
+          data: Hi ha un total de %{count} registres carregats. La última càrrega va ser el dia %{due_date}
+          empty: Encara no hi ha dades censals carregades. Utilitza el següent formulari per importar-lo utilitzant un fitxer CSV.
+        new:
+          info: "Ha de ser un fitxer generat en excel i exportat en CSV amb dues columnes: document d'identitat i data de naixement"
+          title: Pujar un nou cens
+          file: Arxiu excel .csv amb les dades del cens
+          submit: Carrega
+    verifications:
+      authorizations:
+        first_login:
+          actions:
+            file_authorization_handler: Verifica't amb el cens de l'organització

data/config/locales/en.yml

@@ -0,0 +1,50 @@
+en:
+  activemodel:
+    attributes:
+      file_authorization_handler:
+        id_document: Identification document (DNI, NIF, Password or Residence Card)
+        birthdate: Date of birth
+  census_authorization:
+    form:
+      date_select:
+        day: Day
+        month: Month
+        year: Year
+  decidim:
+    authorization_handlers:
+      file_authorization_handler:
+        name: Organization's Census
+        explanation: Authorize your user account against Organization's Census
+        type: CSV
+        fields:
+          birthdate: Birthdate
+    file_authorization_handler:
+      errors:
+        messages:
+          not_censed: We could not find your document ID matching with this birthdate in our Census. If the data entered is correct and the problem persists, please, contact an administrator.
+          younger_than_minimum_age: You should be older than %{age} years
+      admin:
+        destroy:
+          title: Delete all census data
+          confirm: Delete all the census can not be undone. Are you sure you want to continue?
+        censuses:
+          create:
+            success: Successfully imported %{count} items (%{errors} errors)
+          destroy:
+            success: All census data have been deleted
+        menu:
+          census: Upload census
+        show:
+          title: Current census data
+          data: There are %{count} records loaded in total. Last upload date was on %{due_date}
+          empty: There are no census data. Use the form below to import it using a CSV file.
+        new:
+          info: 'Must be a file generated by excel and exported with CSV format with two columns: identity document and date of birth'
+          title: Upload a new census
+          file: Excel .csv file with census data
+          submit: Upload file
+    verifications:
+      authorizations:
+        first_login:
+          actions:
+            file_authorization_handler: Get verified by organization's census

data/config/locales/es.yml

@@ -0,0 +1,50 @@
+es:
+  activemodel:
+    attributes:
+      file_authorization_handler:
+        id_document: Documento de identidad (DNI, NIF, Pasaporte o Targeta de Residencia)
+        birthdate: Fecha de nacimiento
+  census_authorization:
+    form:
+      date_select:
+        day: Día
+        month: Mes
+        year: Año
+  decidim:
+    authorization_handlers:
+      file_authorization_handler:
+        name: Censo de la Organización
+        explanation: Autoriza tu cuenta de usuario contra el Censo de la Organización
+        type: CSV
+        fields:
+          birthdate: Fecha de nacimiento
+    file_authorization_handler:
+      errors:
+        messages:
+          not_censed: No hemos podido encontrar tu documento de identidad con esta fecha de nacimiento en el padrón municipal. Si tus datos son correctos y el problema persiste, por favor, ponte en contacto con un administrador
+          younger_than_minimum_age: Deberías ser mayor de %{age} años
+      admin:
+        destroy:
+          title: Borrar todos los datos de censo
+          confirm: Borrar el censo no se puede deshacer. ¿Estás seguro/a que deseas continuar?
+        censuses:
+          create:
+            success: Se han importado con éxito %{count} elementos (%{errors} errores)
+          destroy:
+            success: Se han borrado todos los datos censales
+        menu:
+          census: Subir censo
+        show:
+          title: Datos de censo actuales
+          data: Hay un total de %{count} registros cargados. La última carga fue el día %{due_date}
+          empty: No hay datos censales. Usa el formulario a continuación para importar usando un fichero CSV.
+        new:
+          info: 'Debe ser un fichero generado por excel y exportado en formato CSV con dos columnas: documento de identidad y fecha de nacimiento'
+          title: Subir un nuevo censo
+          file: Archivo excel .csv con los datos del censo
+          submit: Subir archivo
+    verifications:
+      authorizations:
+        first_login:
+          actions:
+            file_authorization_handler: Verifícate con el censo de la organización

data/db/migrate/20171110120821_create_decidim_file_authorization_handler_census_datum.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+class CreateDecidimFileAuthorizationHandlerCensusDatum < ActiveRecord::Migration[5.1]
+  def change
+    create_table :decidim_file_authorization_handler_census_data do |t|
+      t.references :decidim_organization, index: { name: "decidim_census_data_org_id_index" }
+      t.string :id_document
+      t.date :birthdate
+
+      # The rows in this table are immutable (insert or delete, not update)
+      # To explicitly reflect this fact there is no `updated_at` column
+      t.datetime "created_at", null: false
+    end
+  end
+end