decidim-cdtb 0.1.2 → 0.1.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e29cddba26f79365ef28173d20a58ee1ae85d6a5cd8bc247c6652812497e96dd
-  data.tar.gz: 655282c8af0fa533ddf28ce34a913ca3e4e1ff49787cf266ded32054f2d7045b
+  metadata.gz: d908d1cc4021da9869dd10cacb1fe06b360301d4e13f1a70a55fdc90a2712b6a
+  data.tar.gz: e7a0c808803bbbd6dc75e5bdfaee7b3acfb1311c085f17eb41a9de32fc7795fa
 SHA512:
-  metadata.gz: b79a03dd6ffe799944966da2d357abb75ebe99d543aa9ad916c45999a2bc07bf8632c732565d1501e7572055d7172f90f85efdf640737165baa940ecc8724446
-  data.tar.gz: e59ad9f9295e1c4db656a2793e4ce4f002f1c25e75e2dc37d99c8bd819a9dc29f0497438bd6c32d1f41f4af654d8094f39c3876825b1bc0216a621751b5460d6
+  metadata.gz: 20ddde6eddf59ab12b7ca8d6f489692a04183331f0c1ef496b199b4350cdd103d2b23ca94329199d36f2a9e7222ea0c3269da25648ec0acb47bdde6b38666b92
+  data.tar.gz: 2dd41a8a1f79ffd4506c6b18d58393d3a88d8a021e0acf05cfbd4e4ac1dcc22250294c8de29d7168c2fce29bf2d2df71c0b9905414a3df1e28991aa665be8d36

data/CHANGELOG.md

@@ -1,6 +1,10 @@
 ## [Unreleased]
 
-## [0.1.2] - 2023-06-02
+## [0.1.3] - 2023-06-23 (Tan iguals com especials)
+
+- Validate migrations task
+
+## [0.1.2] - 2023-06-02 (Empastifada amb melmelada)
 
 - Add anonymize rake task
 

data/Gemfile

@@ -25,4 +25,5 @@ end
 
 group :test do
   gem "rspec", "~> 3.0"
+  gem "sqlite3"
 end

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    decidim-cdtb (0.1.2)
+    decidim-cdtb (0.1.3)
       decidim (>= 0.26.2)
       rails (>= 6)
       ruby-progressbar
@@ -700,6 +700,7 @@ GEM
       actionpack (>= 5.2)
       activesupport (>= 5.2)
       sprockets (>= 3.0.0)
+    sqlite3 (1.6.3-x86_64-linux)
     ssrf_filter (1.1.1)
     system_test_html_screenshots (0.2.0)
       actionpack (>= 5.2, < 6.1.a)
@@ -771,6 +772,7 @@ DEPENDENCIES
   rake (~> 13.0)
   rspec (~> 3.0)
   rubocop
+  sqlite3
 
 BUNDLED WITH
    2.3.6

data/README.md

@@ -8,16 +8,16 @@ This is CodiTramuntana's Decidim Toolbelt (cdtb), a gem to help managing Decidim
 Install the gem and add to the application's Gemfile by executing:
 
     $ bundle add decidim-cdtb
+    $ bundle install
 
-If bundler is not being used to manage dependencies, install the gem by executing:
+Install CI tests on your app with:
 
-    $ gem install decidim-cdtb
+    $ bin/rails generate cdtb:validate_migrations_ci
 
-## Usage
 
-### Rake tasks
+## Usage
 
-#### Organizations information
+### Organizations information
 
 Returns information regarding the organizations in a multitenant installation that match a search term ignorecase.
 
@@ -36,7 +36,7 @@ bin/rake cdtb:org_by_host_like[vila]
 host: localhost, time_zone: Madrid, locales: ca + [ca, es, oc], available authorizations: [postal_letter, members_picker_authorization_handler]
 ```
 
-#### Fix nicknames
+### Fix nicknames
 
 In a previous version than Decidim v0.25 a validation to the `Decidim::User.nickname` was added with a migration to fix existing nicknames. But the migration was only taking into acocunt managed (impersonated) users.
 
@@ -48,27 +48,21 @@ To execute the task run:
 bin/rake cdtb:fix_nicknames
 ```
 
-#### Anonymize production dump
+### Anonymize production dump
 
 Anonymize rake task was taken from https://github.com/AjuntamentdeBarcelona/decidim-barcelona
 
 Available rake tasks:
 
-```bin/rake cdtb:anonymize:check``` allows you to check if you can anonymize production dump
-
-```bin/rake cdtb:anonymize:all``` anonymizes whole production dump (without proposals)
-
-```bin/rake cdtb:anonymize:users``` anonymizes users
-
-```bin/rake cdtb:anonymize:proposals``` anonymizes proposals
+- `bin/rake cdtb:anonymize:check` allows you to check if you can anonymize production dump
+- `bin/rake cdtb:anonymize:all` anonymizes whole production dump (without proposals)
+- `bin/rake cdtb:anonymize:users` anonymizes users
+- `bin/rake cdtb:anonymize:proposals` anonymizes proposals
+- `bin/rake cdtb:anonymize:user_groups` anonymizes user groups
+- `bin/rake cdtb:anonymize:system_admins` anonymizes system admins
+- `bin/rake cdtb:anonymize:paper_trail` anonymizes paper trails
 
-```bin/rake cdtb:anonymize:user_groups``` anonymizes user groups
-
-```bin/rake cdtb:anonymize:system_admins``` anonymizes system admins
-
-```bin/rake cdtb:anonymize:paper_trail``` anonymizes paper trails
-
-#### Migrate ActiveStorage service from S3 to local
+### Migrate ActiveStorage service from S3 to local
 
 To migrate from S3 to local storage, the identified steps will be:
 
@@ -82,6 +76,26 @@ To migrate from S3 to local storage, the identified steps will be:
     `bin/rake cache:clear`
 5. Restart the Rails server
 
+### Upgrades:
+
+#### Upgrade modules
+
+Upgrades the gems with engines in them. All, Decidim modules and standard Rails engines.
+
+TO-DO To be finished
+
+#### Validate migrations
+
+Validates that migrations from all gems in the Gemfile have already been installed.
+
+```
+bin/rake cdtb:upgrades:validate_migrations
+```
+
+See the [Installation](#installation) chapter to install a GitHub Action on your app that will run this validation on your CI.
+
+TO-DO also check that all migrations have been executed and the schema.rb does not change
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -107,7 +121,7 @@ bundle exec rspec spec
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/decidim-cdtb. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/decidim-cdtb/blob/master/CODE_OF_CONDUCT.md).
+Bug reports and pull requests are welcome on GitHub at https://github.com/CodiTramuntana/decidim-cdtb. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/CodiTramuntana/decidim-cdtb/blob/master/CODE_OF_CONDUCT.md).
 
 ## License
 
@@ -115,4 +129,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 ## Code of Conduct
 
-Everyone interacting in the Decidim::Cdtb project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/decidim-cdtb/blob/master/CODE_OF_CONDUCT.md).
+Everyone interacting in the Decidim::Cdtb project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/CodiTramuntana/decidim-cdtb/blob/master/CODE_OF_CONDUCT.md).

data/lib/decidim/cdtb/engine.rb

@@ -2,7 +2,7 @@
 
 module Decidim
   module Cdtb
-    # Census have no public app (see AdminEngine)
+    # This module's engine
     class Engine < ::Rails::Engine
       isolate_namespace Decidim::Cdtb
 

data/lib/decidim/cdtb/multitenants/org_by_host_like.rb

@@ -30,9 +30,9 @@ module Decidim
                 locales: "#{org.default_locale} + [#{org.available_locales&.join(", ")}]",
                 available_authorizations: org.available_authorizations&.join(", ")
               }
-              do_log(h.to_yaml)
+              do_log_info(h.to_yaml)
             end
-            do_log("---------------------------------------------------------")
+            do_log_info("---------------------------------------------------------")
           end
         end
 
@@ -47,7 +47,7 @@ module Decidim
         end
 
         def show_full_info(org)
-          do_log(org.attributes.to_yaml)
+          do_log_info(org.attributes.to_yaml)
         end
 
         def show_summary_info(org)
@@ -57,7 +57,7 @@ module Decidim
             locales: "#{org.default_locale} + [#{org.available_locales&.join(", ")}]",
             available_authorizations: org.available_authorizations&.join(", ")
           }
-          do_log(h.to_yaml)
+          do_log_info(h.to_yaml)
         end
       end
     end

data/lib/decidim/cdtb/task.rb

@@ -22,21 +22,21 @@ module Decidim
       def init
         log_task_title(@title)
         @start_time= Time.zone.now
-        do_log("▶️  Starting at #{@start_time}")
+        do_log_info("▶️  Starting at #{@start_time}")
       end
 
       def execute!
         init
         ctx= {}
-        ctx[:progress_bar]= ProgressBar.create(total: total_items, title: title) if has_progress?
         prepare_execution(ctx)
+        ctx[:progress_bar]= ProgressBar.create(total: total_items, title: title) if has_progress?
         do_execution(ctx)
         end_execution(ctx)
         finish
       end
 
       def finish
-        do_log("⏱️  Took #{Time.zone.now - @start_time} seconds")
+        do_log_info("⏱️  Took #{Time.zone.now - @start_time} seconds")
         log_task_end
       end
 

data/lib/decidim/cdtb/tasks.rb

@@ -6,3 +6,4 @@ require "decidim/cdtb/fixes/nickname_fixer"
 require "decidim/cdtb/multitenants/org_by_host_like"
 require "decidim/cdtb/storage/local_sharding"
 require "decidim/cdtb/storage/set_local_on_blobs"
+require "decidim/cdtb/upgrades/validate_migrations_task"

data/lib/decidim/cdtb/tasks_utils.rb

@@ -10,26 +10,40 @@ module Decidim
         Rails.logger
       end
 
-      def do_log(msg)
+      def do_log_info(msg)
         puts msg
         logger.info(msg)
       end
 
+      def do_log_error(msg)
+        puts msg
+        logger.error(msg)
+      end
+
       def log_task_title(title)
-        do_log("⚙️  #{title}")
+        do_log_info("⚙️  #{title}")
       end
 
       def log_task_step(description)
-        do_log("➡️  #{description}")
+        do_log_info("➡️  #{description}")
       end
 
       def log_task_info(info)
-        do_log("ℹ️  #{info}")
+        do_log_info("ℹ️  #{info}")
+      end
+
+      def log_task_failure(msg)
+        @failed= true
+        do_log_error("⚠️  #{msg}")
       end
 
       def log_task_end
-        end_comment= "✅ Done."
-        do_log(end_comment)
+        end_comment= if defined?(@failed) && @failed
+                       "❌  Ended with errors!"
+                     else
+                       "✅ Done."
+                     end
+        do_log_info(end_comment)
       end
     end
   end

data/lib/decidim/cdtb/upgrades/install_gem_migrations_step.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+module Decidim
+  module Cdtb
+    module Upgrades
+      # Invokes rails to install gem migrations
+      #
+      class InstallGemMigrationsStep
+        # Pass +gem_names+ to define from which gems to install migrations.
+        def install!(gem_names)
+          cmd= "#{Rails.root.join("bin/rails")} railties:install:migrations"
+          env_vars= "FROM=#{gem_names.join(",")}"
+          `#{cmd} #{env_vars}`
+        end
+      end
+    end
+  end
+end

data/lib/decidim/cdtb/upgrades/upgrade_modules_task.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module Decidim
+  module Cdtb
+    module Upgrades
+      # Upgrades the gems with engines in them. All, Decidim modules and standard Rails engines.
+      #
+      class UpgradeModulesTask < ::Decidim::Cdtb::Task
+        def initialize
+          progress_bar= { title: "Modules" }
+          super("UPGRADE MODULES", progress_bar: progress_bar)
+        end
+
+        def prepare_execution(_ctx)
+          log_task_info("Have you updated the versions of your modules in the Gemfile (y/n)?")
+          response= $stdin.gets
+          if response&.downcase == "y"
+            get the number of modules
+          else
+            @exit= true
+          end
+        end
+
+        def total_items
+          @num_users
+        end
+
+        def do_execution(context)
+          # progress_bar= context[:progress_bar]
+
+          # system("bundle update decidim")
+          # system("bin/rails decidim:upgrade")
+          # system("bin/rails db:migrate")
+        end
+
+        def end_execution(_ctx)
+          log_task_step("#{@num_applied} users nicknamized")
+        end
+      end
+    end
+  end
+end

data/lib/decidim/cdtb/upgrades/validate_migrations_task.rb

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+require_relative "install_gem_migrations_step"
+
+module Decidim
+  module Cdtb
+    module Upgrades
+      # Validates that all Decidim modules have the migrations installed.
+      #
+      class ValidateMigrationsTask < ::Decidim::Cdtb::Task
+        STEPS_IN_DO_EXECUTION= 2
+
+        def initialize
+          progress_bar= { title: "Modules" }
+          super("VALIDATE MODULES MIGRATIONS", progress_bar: progress_bar)
+        end
+
+        def prepare_execution(_ctx)
+          all_railties= Rails.application.migration_railties
+          railties_w_migrations= all_railties.select do |railtie|
+            railtie.respond_to?(:paths) && railtie.paths["db/migrate"].first.present?
+          end
+          @gem_names= railties_w_migrations.map(&:railtie_name)
+
+          log_task_info("Found #{@gem_names.size} gems with migrations. Validating.....")
+        end
+
+        def total_items
+          STEPS_IN_DO_EXECUTION
+        end
+
+        def pending_migrations?
+          @pending_migrations.present?
+        end
+
+        def do_execution(context)
+          progress_bar= context[:progress_bar]
+
+          output= install_gem_migrations
+
+          progress_bar.increment
+
+          @pending_migrations= output.lines.select { |l| l.include?("Copied migration") }
+
+          progress_bar.increment
+        end
+
+        def end_execution(_ctx)
+          log_task_step("#{@gem_names.size} gems validated")
+          log_task_failure(@pending_migrations.join("\n")) if pending_migrations?
+        end
+
+        def install_gem_migrations
+          install_step= InstallGemMigrationsStep.new
+          install_step.install!(@gem_names)
+        end
+      end
+    end
+  end
+end

data/lib/decidim/cdtb/version.rb

@@ -2,7 +2,7 @@
 
 module Decidim
   module Cdtb
-    VERSION = "0.1.2"
+    VERSION = "0.1.3"
     DECIDIM_MIN_VERSION = ">= 0.26.2"
   end
 end

data/lib/decidim/cdtb.rb

@@ -2,6 +2,7 @@
 
 require_relative "cdtb/version"
 require_relative "cdtb/engine"
+require_relative "cdtb/tasks"
 
 module Decidim
   module Cdtb

data/lib/generators/cdtb/USAGE

@@ -0,0 +1,8 @@
+Description:
+    Installs a GitHub workflow to validate that all migrations from all engines have been installed
+
+Example:
+    rails generate validate_migrations
+
+    This will create:
+        .github/workflows/validate_migrations.yml

data/lib/generators/cdtb/templates/validate_migrations.yml

@@ -0,0 +1,52 @@
+name: "[CI] Validate migrations"
+
+on:
+  push:
+    branches:
+      - master
+      - release/*
+      - "*-stable"
+  pull_request:
+
+env:
+  DB_DATABASE: app
+  DB_USERNAME: postgres
+  DB_PASSWORD: postgres
+  RUBY_VERSION: 3.1.3
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    env:
+      SECRET_KEY_BASE: "secret_key_base"
+
+    steps:
+      - uses: actions/checkout@v2.0.0
+        with:
+          fetch-depth: 1
+      - uses: ruby/setup-ruby@master
+        with:
+          ruby-version: ${{ env.RUBY_VERSION }}
+          bundler-cache: true
+      - name: Recover Ruby dependency cache
+        uses: actions/cache@v1
+        with:
+          path: ./vendor/bundle
+          key: ${{ runner.OS }}-app-rubydeps-${{ hashFiles('Gemfile.lock') }}
+          restore-keys: |
+            ${{ runner.OS }}-app-rubydeps-${{ env.cache-name }}-
+            ${{ runner.OS }}-app-rubydeps-
+
+      - name: Set bundle local config vendor/bundle path
+        run: bundle config set --local path 'vendor/bundle'
+
+      - name: Install Ruby deps
+        uses: nick-invision/retry@v2
+        with:
+          timeout_minutes: 10
+          max_attempts: 3
+          retry_on: error
+          command: bundle install --jobs 4 --retry 3
+
+      - name: Run specs
+        run: bin/rails cdtb:upgrades:validate_migrations

data/lib/generators/cdtb/validate_migrations_ci_generator.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module Cdtb
+  # Generates the GitHub workflow that validates that the app has the migrations from all gems already installed
+  class ValidateMigrationsCiGenerator < Rails::Generators::Base
+    source_root File.expand_path("templates", __dir__)
+
+    def copy_github_workflow_file
+      copy_file "validate_migrations.yml", ".github/workflows/validate_migrations.yml"
+    end
+  end
+end

data/lib/tasks/upgrade.rake

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+#
+# A set of utils to manage Decidim Organizations.
+#
+namespace :cdtb do
+  namespace :upgrades do
+    require "decidim/cdtb/tasks_utils"
+    include ::Decidim::Cdtb::TasksUtils
+
+    desc <<~EODESC
+      Validates that migrations from all gems in the Gemfile have been installed.
+    EODESC
+    task validate_migrations: [:environment] do
+      task= ::Decidim::Cdtb::Upgrades::ValidateMigrationsTask.new
+      task.execute!
+      raise("There are pending migrations") if task.pending_migrations?
+    end
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: decidim-cdtb
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.1.3
 platform: ruby
 authors:
 - Oliver Valls
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-06-30 00:00:00.000000000 Z
+date: 2023-07-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: decidim
@@ -110,11 +110,18 @@ files:
 - lib/decidim/cdtb/task.rb
 - lib/decidim/cdtb/tasks.rb
 - lib/decidim/cdtb/tasks_utils.rb
+- lib/decidim/cdtb/upgrades/install_gem_migrations_step.rb
+- lib/decidim/cdtb/upgrades/upgrade_modules_task.rb
+- lib/decidim/cdtb/upgrades/validate_migrations_task.rb
 - lib/decidim/cdtb/version.rb
+- lib/generators/cdtb/USAGE
+- lib/generators/cdtb/templates/validate_migrations.yml
+- lib/generators/cdtb/validate_migrations_ci_generator.rb
 - lib/tasks/anonymize.rake
 - lib/tasks/cdtb.rake
 - lib/tasks/multitenants.rake
 - lib/tasks/storage.rake
+- lib/tasks/upgrade.rake
 - sig/decidim/cdtb.rbs
 homepage: http://github.com/CodiTramunana/cdtb
 licenses: