foreman_maintain 0.6.5 → 0.6.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 78da2357707f4caa6c3a4aa295c75a07abfff9370800b51b0f03647b2f3abb7f
-  data.tar.gz: 592c818c90870337f4fdc866a29e09e9f9b483f712603d18b84cc5533563d615
+  metadata.gz: e382b3653ec4332ef6bc891b83e92263874dfc079fb84c1f3db951696804832e
+  data.tar.gz: 03b8467289528688c03ed6ad7b8361befee931c9eb0d51a897aa38a755b9ad8a
 SHA512:
-  metadata.gz: 64cf4d1dee1a471a0082fb6daf6ec23812b50ecfb7a32620966c790c2bf976d75fb1b86ae14e73fbda169393abea261fba30c07c00fad589c19ea5ea9902584b
-  data.tar.gz: 94611d55110abfcb196b9bd758af79352930aa977d36a1deb2c08f412730f596fbfeea19585f03ed671797028e2f7a3e6e4484ecec6e77541c73645260fa1a4c
+  metadata.gz: 0f9b4071adbda3fb4f39c84a6c02891aa4e54c0138d8245dfb2da93844bbc6b51e8db88a0424a006fde8563b7b4806b833176caeacaaa9c4b3101ef23824c4f0
+  data.tar.gz: a00abc28ccc812b76de1ca75849f471bd16944fb3cd8bc7ead833f0c077c17e1860626ba6e9c52af834fa1f21c2a6c6b0c331551bc435d921926fdc694cda045

data/README.md

@@ -1,6 +1,6 @@
 # Foreman Maintenance [![Build Status](https://travis-ci.org/theforeman/foreman_maintain.svg?branch=master)](https://travis-ci.org/theforeman/foreman_maintain) <a href="https://codeclimate.com/github/theforeman/foreman_maintain"><img src="https://codeclimate.com/github/theforeman/foreman_maintain/badges/gpa.svg" /></a>
 
-`foreman_maintain` aims to provide various features that helps keep the
+The `foreman_maintain` aims to provide various features that helps keep the
 Foreman/Satellite up and running. It supports multiple versions and subparts
 of the Foreman infrastructure, including server or smart proxy and is smart
 enough to provide the right tools for the specific version.
@@ -19,8 +19,10 @@ Subcommands:
     upgrade                       Upgrade related commands
       list-versions                 List versions this system is upgradable to
       check --target-version TARGET_VERSION   Run pre-upgrade checks for upgrading to specified version
+            --disable-self-upgrade            Disable automatic self upgrade (default: false)
       run --target-version TARGET_VERSION     Run the full upgrade
           [--phase=phase TARGET_VERSION]      Run just a specific phase of the upgrade
+          --disable-self-upgrade              Disable automatic self upgrade (default: false)
 
     advanced                      Advanced tools for server maintenance
       procedure                     Run maintain procedures manually
@@ -59,8 +61,8 @@ Subcommands:
 Foreman-maintain implements upgrade tooling that helps the administrator to go
 through the upgrade process.
 
-Foreman-maintain scans the system to know, what version are available
-to upgrade to for the particular system. To see what versions are available
+Foreman-maintain scans the system to know, what versions are available
+for upgrade on the particular system. To see what versions are available
 for upgrade, run:
 
 ```
@@ -88,13 +90,13 @@ of the system:
 
   * **pre-migrations** - these steps perform changes on the system before
     the actual upgrade starts. An example is disabling access to the system from
-    external sources, a.k.a. maintenance mode or disabling sync plans during the run.
+    external sources, a.k.a. maintenance mode or disabling Katello sync plans during the run.
 
     After this phase ends, the system is still running the old version, and it's possible
     to revert the changes by running the post-migrations steps.
 
   * **migrations** - this phase performs the actual migrations, starting with
-    configuring new repositories, updated the packages and running the installer.
+    configuring new repositories, updating the packages and running the installer.
 
     At the end of this phase, the system should be fully migrated to the new version.
     However, the system is not fully operational yet, as the post-migrations steps
@@ -103,7 +105,7 @@ of the system:
   * **post-migrations** - these steps revert the changes made in pre-migrations phase,
     turning the system into fully-operational again.
 
-  * **post-upgrade checks** - this steps should perform sanity check of the system
+  * **post-upgrade checks** - these steps should perform sanity check of the system
     to ensure the system is valid and ready to be used again.
 
 
@@ -114,6 +116,40 @@ the *pre-upgrade check* phase. In case the upgrade failed before **migrations**
 phase made some modifying changes, the tool tries to rollback to the previous
 state of the system.
 
+#### Self-upgrade for rubygem-foreman_maintain package
+
+**Note:** This feature is available from `rubygem-foreman_maintain` version 0.6.4 and newer.
+
+When a user runs any `foreman-maintain upgrade` sub commands (e.g. `foreman-maintain upgrade check` or `foreman-maintain upgrade run`) then,
+
+  * If update available for `rubygem-foreman_maintain` package, the sub command tries to update this package. After successful package update, it returns the exit code 75 and requests user to re-run with the updated source code.
+
+    Here, exit code (value 75) is to indicate that it can not continue with further execution & needs re-run. e.g.,
+
+    ~~~
+    # foreman-maintain upgrade check --target-version TARGET_VERSION
+    Checking for new version of foreman-maintain...
+    rubygem-foreman_maintain.noarch   repository
+
+    Updating foreman-maintain package.
+
+    The foreman-maintain package successfully updated.
+
+    Re-run foreman-maintain with required options!
+
+    # echo $?
+    75
+    ~~~
+
+  * If update is not available for `rubygem-foreman_maintain` package, then sub command simply executes the further steps without halt.
+
+  * If user wants to skip self-update mechanism then `--disable-self-upgrade` flag can be used with upgrade sub commands. e.g.,
+
+  ~~~
+  # foreman-maintain upgrade check --target-version TARGET_VERSION --disable-self-upgrade
+  # foreman-maintain upgrade run --target-version TARGET_VERSION --disable-self-upgrade
+  ~~~
+
 #### Satellite notes
 
 To use custom organization/activation key for configuring repositories during
@@ -132,7 +168,7 @@ export FOREMAN_MAINTAIN_USE_BETA='1'
 
 ## Implementation
 
-`foreman_maintain` maps the CLI commands into definitions. This allows to keep the set
+The `foreman_maintain` maps the CLI commands into definitions. This allows to keep the set
 of the commands the user needs to know immutable from version-specific changes. The mapping
 between the CLI commands and definitions is made by defining various metadata.
 
@@ -329,8 +365,8 @@ end
 ```
 
 Before executing the command the feature checks if it has valid hammer configuration to run the command.
-Foreman maintain always use the 'admin' account to run the commands. The password is taken either form
-the Hammer config or installer answer files or asked from the user interactively (in this order).
+Foreman maintain always use the 'admin' account to run the commands. The password is taken from
+the hammer config or installer answer files or asked from the user interactively (in this order).
 The valid credentials are stored and reused next time if still valid.
 
 Usually we want to do the user interaction at the beginning of our scenario.
@@ -449,9 +485,18 @@ Possible options for the `:completion` attribute are:
 * `maintenance-mode is-enabled` returns `0 or 1` output depending upon the maintenance-mode status.
 Here, 0=ON & 1=OFF.
 
-If User would like to check whether maintenance-mode is ON/OFF on system in their external script then
+If users would like to check whether maintenance-mode is ON/OFF on system in their external script then
 they can use subcommand `foreman-maintain maintenance-mode is-enabled`.
 
+## Exit codes with special meanings -
+
+Every command returns an exit code. Any other exit status than 0 indicates a failure of some kind. Foreman Maintain uses following exit codes with special meaning.
+
+| Exit Code  | Description                        |
+| -----------| -----------------------------------|
+| 75         | Temporary failure and needs re-run |
+| 78         | Command executed with warning(s)   |
+
 ## How to contribute?
 
 Generally, follow the [Foreman guidelines](https://theforeman.org/contribute.html). For code-related contributions, fork this project and send a pull request with all changes. Some things to keep in mind:

data/bin/foreman-maintain-complete

@@ -13,7 +13,8 @@ require 'foreman_maintain/utils/bash'
 # rubocop:disable Lint/RescueWithoutErrorClass, Lint/HandleExceptions
 def cache_file(config_file)
   config = YAML.load(File.open(config_file))
-  config[:completion_cache_file]
+  config.fetch(:completion_cache_file,
+               "#{ENV['HOME'] || '/root'}/.cache/foreman_maintain_completion.yml")
 rescue
 end
 # rubocop:enable Lint/RescueWithoutErrorClass, Lint/HandleExceptions
@@ -23,7 +24,7 @@ config_file = if File.exist?(CONFIG_FILE)
               else
                 File.join(project_root, 'config/foreman_maintain.yml')
               end
-completion_cache_file = cache_file(config_file) || '~/.cache/foreman_maintain_completion.yml'
+completion_cache_file = cache_file(config_file)
 completion_cache_file = File.expand_path(completion_cache_file)
 
 # build the cache if it does not exist

data/definitions/checks/env_proxy.rb

@@ -0,0 +1,13 @@
+class Checks::EnvProxy < ForemanMaintain::Check
+  metadata do
+    label :env_proxy
+    tags :env_proxy
+    description 'Check to make sure no HTTP(S) proxy set in ENV'
+  end
+
+  def run
+    variables = %w[http_proxy https_proxy HTTP_PROXY HTTPS_PROXY]
+    has_proxy_set = true if variables.map { |variable| ENV[variable] }.compact.any?
+    assert(!has_proxy_set, 'Global HTTP(S) proxy in environment (env) is set. Please unset first!')
+  end
+end

data/definitions/checks/foreman/check_checkpoint_segments.rb

@@ -0,0 +1,51 @@
+module Checks
+  module Foreman
+    class CheckpointSegments < ForemanMaintain::Check
+      metadata do
+        label :check_postgresql_checkpoint_segments
+        description 'Check if checkpoint_segments configuration exists on the system'
+        confine do
+          feature(:foreman) && feature(:installer) &&
+            File.exist?(feature(:installer).custom_hiera_file)
+        end
+      end
+
+      def run
+        failure_message = check_custom_hiera
+        fail! failure_message if failure_message
+      end
+
+      # rubocop:disable Metrics/MethodLength
+      def check_custom_hiera
+        hiera_file = feature(:installer).custom_hiera_file
+        if (config = YAML.load_file(hiera_file)) &&
+           config.key?('postgresql::server::config_entries')
+          if config['postgresql::server::config_entries'].nil?
+            return <<-MESSAGE.strip_heredoc
+            ERROR: 'postgresql::server::config_entries' cannot be null.
+            Please remove it from following file and re-run the command.
+            - #{hiera_file}
+            MESSAGE
+          elsif config['postgresql::server::config_entries'].key?('checkpoint_segments')
+            message = <<-MESSAGE.strip_heredoc
+            ERROR: Tuning option 'checkpoint_segments' found.
+            This option is no longer valid for PostgreSQL 9.5 or newer.
+            Please remove it from following file and re-run the command.
+            - #{hiera_file}
+            MESSAGE
+            if feature(:katello)
+              message += <<-MESSAGE.strip_heredoc
+              The presence of checkpoint_segments in #{hiera_file} indicates manual tuning.
+              Manual tuning can override values provided by the --tuning parameter.
+              Review #{hiera_file} for values that are already provided by the built in tuning profiles.
+              Built in tuning profiles also provide a supported upgrade path.
+              MESSAGE
+            end
+            return message
+          end
+        end
+      end
+      # rubocop:enable Metrics/MethodLength
+    end
+  end
+end

data/definitions/checks/foreman/check_duplicate_permission.rb

@@ -0,0 +1,33 @@
+module Checks
+  module Foreman
+    class CheckDuplicatePermissions < ForemanMaintain::Check
+      metadata do
+        label :duplicate_permissions
+        for_feature :foreman_database
+        description 'Check for duplicate permissions from database'
+        tags :pre_upgrade
+      end
+
+      def run
+        duplicate_permissions = find_duplicate_permissions
+        assert(
+          duplicate_permissions.empty?,
+          'Duplicate permissions in your database',
+          :next_steps => [
+            Procedures::Foreman::RemoveDuplicatePermissions.new
+          ]
+        )
+      end
+
+      def find_duplicate_permissions
+        feature(:foreman_database).query(self.class.query_to_get_duplicate_permission)
+      end
+
+      def self.query_to_get_duplicate_permission
+        <<-SQL
+          SELECT id,name FROM permissions p WHERE (SELECT count(name) FROM permissions pr WHERE p.name =pr.name) > 1
+        SQL
+      end
+    end
+  end
+end

data/definitions/checks/foreman_proxy/check_tftp_storage.rb

@@ -0,0 +1,51 @@
+module Checks::ForemanProxy
+  class CheckTftpStorage < ForemanMaintain::Check
+    metadata do
+      label :check_tftp_storage
+      description 'Clean old Kernel and initramfs files from tftp-boot'
+      tags :default
+      confine do
+        feature(:satellite) && feature(:foreman_proxy) &&
+          feature(:foreman_proxy).features.include?('tftp') && non_zero_token_duration?
+      end
+    end
+
+    def run
+      if Dir.exist?(tftp_boot_directory)
+        files = old_files_from_tftp_boot
+        assert(files.empty?,
+               'There are old initrd and vmlinuz files present in tftp',
+               :next_steps => Procedures::Files::Remove.new(:files => files))
+      else
+        skip "TFTP #{tftp_boot_directory} directory doesn't exist."
+      end
+    end
+
+    def old_files_from_tftp_boot
+      Dir.glob("#{tftp_boot_directory}*-{vmlinuz,initrd.img}").map do |file|
+        unless File.directory?(file)
+          file if File.mtime(file) + (token_duration * 60) < Time.now
+        end
+      end.compact
+    end
+
+    def self.non_zero_token_duration?
+      lookup_token_duration != 0
+    end
+
+    def tftp_boot_directory
+      @tftp_boot_directory ||= "#{feature(:foreman_proxy).tftp_root_directory}/boot/"
+    end
+
+    def token_duration
+      @token_duration ||= self.class.lookup_token_duration
+    end
+
+    def self.lookup_token_duration
+      data = feature(:foreman_database). \
+             query("select s.value, s.default from settings s \
+                    where category = 'Setting::Provisioning' and name = 'token_duration'")
+      YAML.load(data[0]['value'] || data[0]['default'])
+    end
+  end
+end

data/definitions/checks/foreman_proxy/verify_dhcp_config_syntax.rb

@@ -4,20 +4,18 @@ module Checks::ForemanProxy
       description 'Check for verifying syntax for ISP DHCP configurations'
       tags :default
       confine do
-        feature(:foreman_proxy)
+        feature(:foreman_proxy) &&
+          feature(:foreman_proxy).features.include?('dhcp') &&
+          feature(:foreman_proxy).dhcp_isc_provider?
       end
     end
 
     def run
-      if feature(:foreman_proxy).features.include?('dhcp')
-        if feature(:foreman_proxy).dhcpd_conf_exist?
-          success = feature(:foreman_proxy).valid_dhcp_configs?
-          assert(success, 'Please check and verify DHCP configurations.')
-        else
-          fail! "Couldn't find configuration file at #{feature(:foreman_proxy).dhcpd_config_file}"
-        end
+      if feature(:foreman_proxy).dhcpd_conf_exist?
+        success = feature(:foreman_proxy).valid_dhcp_configs?
+        assert(success, 'Please check and verify DHCP configurations.')
       else
-        skip 'DHCP feature is not enabled'
+        fail! "Couldn't find configuration file at #{feature(:foreman_proxy).dhcpd_config_file}"
       end
     end
   end

data/definitions/checks/repositories/check_non_rh_repository.rb

@@ -2,7 +2,7 @@ module Checks::Repositories
   class CheckNonRhRepository < ForemanMaintain::Check
     metadata do
       label :check_non_redhat_repository
-      description "Check whether system don't have any non Red Hat repositories(Eg: EPEL) enabled"
+      description 'Check whether system has any non Red Hat repositories (e.g.: EPEL) enabled'
       tags :pre_upgrade
       confine do
         feature(:instance).downstream
@@ -11,7 +11,7 @@ module Checks::Repositories
 
     def run
       with_spinner('Checking repositories enabled on the system') do
-        assert(!epel_enabled?, 'System is subscribed to non Red Hat repositories(Eg: EPEL)')
+        assert(!epel_enabled?, 'System is subscribed to non Red Hat repositories')
       end
     end
 

data/definitions/features/capsule.rb

@@ -5,7 +5,8 @@ class Features::Capsule < ForemanMaintain::Feature
     label :capsule
 
     confine do
-      package_manager.installed?(['satellite-capsule']) ||
+      !package_manager.installed?(['satellite']) &&
+        package_manager.installed?(['satellite-capsule']) ||
         package_manager.installed?(['capsule-installer'])
     end
   end

data/definitions/features/foreman_proxy.rb

@@ -12,6 +12,9 @@ class Features::ForemanProxy < ForemanMaintain::Feature
   FOREMAN_PROXY_DHCP_YML_PATHS = ['/etc/foreman-proxy/settings.d/dhcp.yml',
                                   '/usr/local/etc/foreman-proxy/settings.d/dhcp.yml'].freeze
 
+  FOREMAN_PROXY_TFTP_YML_PATHS = ['/etc/foreman-proxy/settings.d/tftp.yml',
+                                  '/usr/local/etc/foreman-proxy/settings.d/tftp.yml'].freeze
+
   def valid_dhcp_configs?
     dhcp_req_pass? && !syntax_error_exists?
   end
@@ -66,7 +69,7 @@ class Features::ForemanProxy < ForemanMaintain::Feature
 
     configs.push('/var/lib/tftpboot') if backup_features.include?('tftp')
     configs += ['/var/named/', '/etc/named*'] if backup_features.include?('dns')
-    if backup_features.include?('dhcp')
+    if backup_features.include?('dhcp') && dhcp_isc_provider?
       configs += ['/var/lib/dhcpd', File.dirname(dhcpd_config_file)]
     end
     configs.push('/usr/share/xml/scap') if backup_features.include?('openscap')
@@ -98,7 +101,9 @@ class Features::ForemanProxy < ForemanMaintain::Feature
   end
 
   def certs_tar
-    feature(:installer).answers[content_module][certs_param_name[:param_section]] if content_module
+    if content_module
+      feature(:installer).answers.fetch(content_module, {})[certs_param_name[:param_key]]
+    end
   end
 
   def settings_file
@@ -113,6 +118,14 @@ class Features::ForemanProxy < ForemanMaintain::Feature
     @dhcpd_config_file ||= lookup_dhcpd_config_file
   end
 
+  def tftp_root_directory
+    @tftp_root_directory ||= lookup_tftp_root_directory
+  end
+
+  def dhcp_isc_provider?
+    configs_from_dhcp_yml[:use_provider] == 'dhcp_isc'
+  end
+
   private
 
   def backup_features(for_features)
@@ -209,11 +222,18 @@ class Features::ForemanProxy < ForemanMaintain::Feature
     dhcpd_config_file
   end
 
-  def lookup_using_dhcp_yml
-    dhcp_yml_path = lookup_into(FOREMAN_PROXY_DHCP_YML_PATHS)
-    raise "Couldn't find dhcp.yml file under foreman-proxy" unless dhcp_yml_path
+  def dhcp_yml_path
+    dhcp_path = lookup_into(FOREMAN_PROXY_DHCP_YML_PATHS)
+    raise "Couldn't find dhcp.yml file under foreman-proxy" unless dhcp_path
 
-    configs_from_dhcp_yml = yaml_load(dhcp_yml_path)
+    dhcp_path
+  end
+
+  def configs_from_dhcp_yml
+    @configs_from_dhcp_yml ||= yaml_load(dhcp_yml_path)
+  end
+
+  def lookup_using_dhcp_yml
     if configs_from_dhcp_yml.key?(:dhcp_config)
       return configs_from_dhcp_yml[:dhcp_config]
     elsif configs_from_dhcp_yml.key?(:use_provider)
@@ -226,6 +246,13 @@ class Features::ForemanProxy < ForemanMaintain::Feature
     end
   end
 
+  def lookup_tftp_root_directory
+    tftp_yml_path = lookup_into(FOREMAN_PROXY_TFTP_YML_PATHS)
+    raise "Couldn't find tftp.yml file under foreman-proxy" unless tftp_yml_path
+
+    yaml_load(tftp_yml_path)[:tftproot]
+  end
+
   def yaml_load(path)
     YAML.load_file(path) || {}
   end

data/definitions/features/foreman_server.rb

@@ -28,7 +28,7 @@ module ForemanMaintain
       def config_files
         [
           '/etc/httpd',
-          '/var/www/html/pub',
+          '/var/www/html/pub/katello-*',
           '/etc/squid',
           '/etc/foreman',
           '/etc/selinux/targeted/contexts/files/file_contexts.subs',

data/definitions/features/installer.rb

@@ -53,6 +53,10 @@ class Features::Installer < ForemanMaintain::Feature
     end
   end
 
+  def custom_hiera_file
+    @custom_hiera_file ||= File.join(config_directory, 'custom-hiera.yaml')
+  end
+
   def can_upgrade?
     @installer_type == :scenarios || @installer_type == :legacy_katello
   end

data/definitions/procedures/backup/prepare_directory.rb

@@ -8,6 +8,7 @@ module Procedures::Backup
       param :incremental_dir, 'Changes since specified backup only'
     end
 
+    # rubocop:disable Metrics/MethodLength
     def run
       puts "Creating backup folder #{@backup_dir}"
 
@@ -22,8 +23,14 @@ module Procedures::Backup
 
       FileUtils.rm(Dir.glob(File.join(@backup_dir, '.*.snar'))) if @preserve_dir
       if @incremental_dir
-        FileUtils.cp(Dir.glob(File.join(@incremental_dir, '.*.snar')), @backup_dir)
+        if (snar_files = Dir.glob(File.join(@incremental_dir, '.*.snar'))).empty?
+          raise "#{@incremental_dir}/*.snar files unavailable. "\
+                'Provide a valid previous backup directory'
+        else
+          FileUtils.cp(snar_files, @backup_dir)
+        end
       end
     end
+    # rubocop:enable Metrics/MethodLength
   end
 end

data/definitions/procedures/backup/snapshot/logical_volume_confirmation.rb

@@ -12,7 +12,7 @@ module Procedures::Backup
         backup_lv = get_lv_info(@backup_dir)
 
         dbs = {}
-        dbs[:pulp] = 'Pulp' if feature(:pulp2) && !@skip_pulp
+        dbs[:pulp2] = 'Pulp' if feature(:pulp2) && !@skip_pulp
         dbs[:mongo] = 'Mongo' if db_local?(:mongo)
         dbs[:candlepin_database] = 'Candlepin' if db_local?(:candlepin_database)
         dbs[:foreman_database] = 'Foreman' if db_local?(:foreman_database)

data/definitions/procedures/content/prepare.rb

@@ -3,6 +3,11 @@ module Procedures::Content
     metadata do
       description 'Prepare content for Pulp 3'
       for_feature :pulpcore
+
+      confine do
+        # FIXME: remove this condition on next downstream upgrade scenario
+        !feature(:instance).downstream
+      end
     end
 
     def run

data/definitions/procedures/content/switchover.rb

@@ -3,6 +3,11 @@ module Procedures::Content
     metadata do
       description 'Switch support for certain content from Pulp 2 to Pulp 3'
       for_feature :pulpcore
+
+      confine do
+        # FIXME: remove this condition on next downstream upgrade scenario
+        !feature(:instance).downstream
+      end
     end
 
     def run
@@ -15,7 +20,8 @@ module Procedures::Content
       puts 'Re-running the installer to switch specified content over to pulp3'
       args = ['--foreman-proxy-content-proxy-pulp-isos-to-pulpcore=true',
               '--katello-use-pulp-2-for-file=false',
-              '--katello-use-pulp-2-for-docker=false']
+              '--katello-use-pulp-2-for-docker=false',
+              '--katello-use-pulp-2-for-yum=false']
       feature(:installer).run(args.join(' '))
     end
   end

data/definitions/procedures/foreman/remove_duplicate_permissions.rb

@@ -0,0 +1,70 @@
+module Procedures::Foreman
+  class RemoveDuplicatePermissions < ForemanMaintain::Procedure
+    metadata do
+      for_feature :foreman_database
+      description 'Remove duplicate permissions from database'
+    end
+
+    def run
+      duplicate_permissions = feature(:foreman_database).query(
+        Checks::Foreman::CheckDuplicatePermissions.query_to_get_duplicate_permission
+      ).group_by { |permission| permission['name'] }
+      unassigned_permissions = []
+      duplicate_permissions.each_value do |permissions|
+        permission_ids = permissions.map { |i| i['id'] }
+        filterings = check_permissions_assign_to_filter(permission_ids)
+        assigned_permissions = filterings.keys
+        unassigned_permissions << permission_ids - assigned_permissions
+        fix_permissions(assigned_permissions) if assigned_permissions.length > 1
+      end
+      delete_permission(unassigned_permissions.flatten) unless unassigned_permissions.empty?
+    end
+
+    private
+
+    def check_permissions_assign_to_filter(permission_ids)
+      sql = <<-SQL
+        SELECT id, filter_id, permission_id FROM filterings WHERE permission_id IN (#{permission_ids.join(',')})
+      SQL
+      feature(:foreman_database).query(sql).group_by { |filtering| filtering['permission_id'] }
+    end
+
+    def fix_permissions(assigned_permissions)
+      persist_permission = assigned_permissions.shift
+      filter_ids = filters_for_permission(persist_permission)
+      update_filtering(assigned_permissions, persist_permission, filter_ids)
+      delete_filtering(assigned_permissions)
+      delete_permission(assigned_permissions)
+    end
+
+    def filters_for_permission(permission)
+      feature(:foreman_database).query(
+        "SELECT filter_id FROM filterings WHERE permission_id = #{permission.to_i}"
+      ).map { |filter| filter['filter_id'] }
+    end
+
+    def update_filtering(old_ids, new_id, filter_ids)
+      sql = <<-SQL
+      WITH rows AS (
+        UPDATE filterings SET permission_id = '#{new_id}' WHERE permission_id IN (#{old_ids.join(',')}) AND filter_id NOT IN (#{filter_ids.join(',')})
+        RETURNING id
+      )
+      SELECT id
+      FROM rows
+      SQL
+      feature(:foreman_database).query(sql)
+    end
+
+    def delete_filtering(permission_ids)
+      feature(:foreman_database).psql(
+        "DELETE FROM filterings where permission_id IN (#{permission_ids.join(',')})"
+      )
+    end
+
+    def delete_permission(permission_ids)
+      feature(:foreman_database).psql(
+        "DELETE FROM permissions where id IN (#{permission_ids.join(',')})"
+      )
+    end
+  end
+end

data/definitions/procedures/installer/upgrade.rb

@@ -6,7 +6,7 @@ module Procedures::Installer
 
     def run
       assumeyes_val = @assumeyes.nil? ? assumeyes? : @assumeyes
-      feature(:installer).run(@arguments, :interactive => !assumeyes_val)
+      feature(:installer).upgrade(:interactive => !assumeyes_val)
     end
   end
 end

data/definitions/procedures/packages/locking_status.rb

@@ -18,7 +18,7 @@ module Procedures::Packages
       else
         puts '  Packages are not locked.'
         puts "  WARNING: When locking is disabled there is a risk of unwanted update\n" \
-             "  of #{feature(:instance).product_name}' and its components and possible " \
+             "  of #{feature(:instance).product_name} and its components and possible " \
              'data inconsistency'
       end
     end

data/definitions/scenarios/content.rb

@@ -8,7 +8,10 @@ module ForemanMaintain::Scenarios
       end
 
       def compose
-        add_step(Procedures::Content::Prepare)
+        # FIXME: remove this condition on next downstream upgrade scenario
+        if Procedures::Content::Prepare.present?
+          add_step(Procedures::Content::Prepare)
+        end
       end
     end
 
@@ -20,7 +23,10 @@ module ForemanMaintain::Scenarios
       end
 
       def compose
-        add_step(Procedures::Content::Switchover)
+        # FIXME: remove this condition on next downstream upgrade scenario
+        if Procedures::Content::Switchover.present?
+          add_step(Procedures::Content::Switchover)
+        end
       end
     end
   end

data/definitions/scenarios/packages.rb

@@ -1,7 +1,10 @@
 module ForemanMaintain::Scenarios
   module Packages
     def self.skip_installer_run?(packages_list)
-      packages_list = packages_list.split(',').map(&:strip) if packages_list.is_a?(String)
+      if packages_list.is_a?(String)
+        packages_list = packages_list.split(',').map(&:strip)
+      end
+      packages_list ||= []
 
       return false unless packages_list.any? { |p| p.include?('foreman_maintain') }
       return true if packages_list.length == 1

data/definitions/scenarios/restore.rb

@@ -20,6 +20,7 @@ module ForemanMaintain::Scenarios
                              Checks::Restore::ValidateHostname,
                              Procedures::Selinux::SetFileSecurity,
                              Procedures::Restore::Configs)
+      add_step_with_context(Procedures::Crond::Stop) if feature(:cron)
       unless backup.incremental?
         add_steps_with_context(Procedures::Restore::EnsureMongoEngineMatches,
                                Procedures::Restore::InstallerReset)
@@ -41,6 +42,7 @@ module ForemanMaintain::Scenarios
       add_steps_with_context(Procedures::Restore::RegenerateQueues) if backup.online_backup?
       add_steps_with_context(Procedures::Service::Start,
                              Procedures::Service::DaemonReload)
+      add_step_with_context(Procedures::Crond::Start) if feature(:cron)
     end
     # rubocop:enable Metrics/MethodLength,Metrics/AbcSize
 
@@ -98,4 +100,15 @@ module ForemanMaintain::Scenarios
                   Procedures::Selinux::SetFileSecurity => :incremental_backup)
     end
   end
+
+  class RestoreRescue < ForemanMaintain::Scenario
+    metadata do
+      description 'Resuce Restore backup'
+      manual_detection
+    end
+
+    def compose
+      add_step_with_context(Procedures::Crond::Stop) if feature(:cron)
+    end
+  end
 end

data/definitions/scenarios/upgrade_to_satellite_6_8.rb

@@ -27,6 +27,7 @@ module Scenarios::Satellite_6_8
     def compose
       add_steps(find_checks(:default))
       add_steps(find_checks(:pre_upgrade))
+      add_step(Checks::Foreman::CheckpointSegments)
       add_step(Checks::Repositories::Validate.new(:version => '6.8'))
     end
   end

data/extras/foreman_protector/foreman-protector.py

@@ -26,7 +26,7 @@ def _load_whitelist():
                 continue
             _package_whitelist.add(line.rstrip().lower())
         llfile.close()
-    except urlgrabber.grabber.URLGrabError, e:
+    except urlgrabber.grabber.URLGrabError as e:
         raise PluginYumExit('Unable to read Foreman protector"s configuration: %s' % e)
 
 def _add_obsoletes(conduit):

data/lib/foreman_maintain.rb

@@ -53,6 +53,8 @@ module ForemanMaintain
     }.freeze
 
     def setup(options = {})
+      set_home_environment
+
       # using a queue, we can log the messages which are generated before initializing logger
       self.config = Config.new(options)
       load_definitions
@@ -60,6 +62,10 @@ module ForemanMaintain
       update_path
     end
 
+    def set_home_environment
+      ENV['HOME'] ||= '/root'
+    end
+
     # Appending PATH with expected paths needed for commands we run
     def update_path
       paths = ['/sbin']
@@ -162,7 +168,7 @@ module ForemanMaintain
         ForemanMaintain.package_manager.update(main_package_name, :assumeyes => true)
         puts "\nThe #{package_name} package successfully updated."\
              "\nRe-run #{command} with required options!"
-        exit 0
+        exit 75
       end
       puts "Nothing to update, can't find new version of #{package_name}."
     end

data/lib/foreman_maintain/cli/packages_command.rb

@@ -38,7 +38,7 @@ module ForemanMaintain
 
       subcommand 'update', 'Update packages in an unlocked session' do
         interactive_option
-        parameter '[PACKAGES ...]', 'packages to update', :attribute_name => :packages
+        parameter '[PACKAGES] ...', 'packages to update', :attribute_name => :packages
 
         def execute
           run_scenarios_and_exit(

data/lib/foreman_maintain/cli/restore_command.rb

@@ -13,7 +13,8 @@ module ForemanMaintain
           :backup_dir => @backup_dir,
           :incremental_backup => @incremental || incremental_backup?
         )
-        run_scenario(scenario)
+        rescue_scenario = Scenarios::RestoreRescue.new
+        run_scenario(scenario, rescue_scenario)
         exit runner.exit_code
       end
 

data/lib/foreman_maintain/cli/upgrade_command.rb

@@ -53,7 +53,10 @@ module ForemanMaintain
       end
 
       subcommand 'list-versions', 'List versions this system is upgradable to' do
+        disable_self_upgrade_option
+
         def execute
+          ForemanMaintain.perform_self_upgrade unless disable_self_upgrade?
           print_versions(UpgradeRunner.available_targets)
         end
       end

data/lib/foreman_maintain/concerns/base_database.rb

@@ -82,7 +82,7 @@ module ForemanMaintain
           tar_options = {
             :archive => backup_file,
             :command => 'create',
-            :transform => 's,^,var/lib/pgsql/data/,S',
+            :transform => "s,^,#{data_dir[1..-1]},S",
             :files => '*'
           }.merge(extra_tar_options)
           feature(:tar).run(tar_options)

data/lib/foreman_maintain/concerns/downstream.rb

@@ -72,7 +72,9 @@ module ForemanMaintain
       end
 
       def ansible_repo(server_version, rh_version_major)
-        if server_version >= version('6.6')
+        if server_version >= version('6.8')
+          "rhel-#{rh_version_major}-server-ansible-2.9-rpms"
+        elsif server_version >= version('6.6')
           "rhel-#{rh_version_major}-server-ansible-2.8-rpms"
         elsif server_version >= version('6.4')
           "rhel-#{rh_version_major}-server-ansible-2.6-rpms"
@@ -97,21 +99,17 @@ module ForemanMaintain
       end
 
       def common_repos(rh_version_major, full_version)
-        repos_arrary = common_repos_array(rh_version_major, full_version)
-        return repos_arrary.first(1) if feature(:capsule)
+        repos_arrary = if ENV['FOREMAN_MAINTAIN_USE_BETA'] == '1'
+                         ["rhel-#{rh_version_major}-server-satellite-maintenance-6-beta-rpms",
+                          "rhel-#{rh_version_major}-server-satellite-tools-6-beta-rpms"]
+                       else
+                         ["rhel-#{rh_version_major}-server-satellite-maintenance-6-rpms",
+                          "rhel-#{rh_version_major}-server-satellite-tools-#{full_version}-rpms"]
+                       end
 
-        repos_arrary
-      end
-
-      def common_repos_array(rh_version_major, full_version)
-        ["rhel-#{rh_version_major}-server-satellite-maintenance-6#{use_beta}-rpms",
-         "rhel-#{rh_version_major}-server-satellite-tools-#{full_version}#{use_beta}-rpms"]
-      end
-
-      def use_beta
-        return '-beta' if ENV['FOREMAN_MAINTAIN_USE_BETA'] == '1'
+        return repos_arrary.first(1) if feature(:satellite)
 
-        nil
+        repos_arrary
       end
 
       def main_rh_repos(rh_version_major)

data/lib/foreman_maintain/reporter/cli_reporter.rb

@@ -276,7 +276,7 @@ module ForemanMaintain
 
       # rubocop:disable Metrics/MethodLength, Metrics/AbcSize
       def scenario_failure_message(scenario)
-        return if scenario.passed?
+        return if scenario.passed? && !scenario.warning?
         message = []
         message << <<-MESSAGE.strip_heredoc
           Scenario [#{scenario.description}] failed.

data/lib/foreman_maintain/runner.rb

@@ -54,6 +54,7 @@ module ForemanMaintain
         @last_scenario = scenario
         @last_scenario_continuation_confirmed = false
       end
+      @exit_code = 78 if scenario.warning?
       @exit_code = 1 if scenario.failed?
     end
 

data/lib/foreman_maintain/scenario.rb

@@ -129,8 +129,11 @@ module ForemanMaintain
 
     def passed?
       (steps_with_abort(:whitelisted => false) +
-        steps_with_error(:whitelisted => false) +
-        steps_with_warning(:whitelisted => false)).empty?
+        steps_with_error(:whitelisted => false)).empty?
+    end
+
+    def warning?
+      !steps_with_warning(:whitelisted => false).empty?
     end
 
     def failed?

data/lib/foreman_maintain/version.rb

@@ -1,3 +1,3 @@
 module ForemanMaintain
-  VERSION = '0.6.5'.freeze
+  VERSION = '0.6.10'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: foreman_maintain
 version: !ruby/object:Gem::Version
-  version: 0.6.5
+  version: 0.6.10
 platform: ruby
 authors:
 - Ivan Nečas
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-06-03 00:00:00.000000000 Z
+date: 2020-09-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: clamp
@@ -126,13 +126,17 @@ files:
 - definitions/checks/check_tmout.rb
 - definitions/checks/disk/available_space.rb
 - definitions/checks/disk/performance.rb
+- definitions/checks/env_proxy.rb
+- definitions/checks/foreman/check_checkpoint_segments.rb
 - definitions/checks/foreman/check_corrupted_roles.rb
+- definitions/checks/foreman/check_duplicate_permission.rb
 - definitions/checks/foreman/check_duplicate_roles.rb
 - definitions/checks/foreman/db_up.rb
 - definitions/checks/foreman/facts_names.rb
 - definitions/checks/foreman/puppet_class_duplicates.rb
 - definitions/checks/foreman/validate_external_db_version.rb
 - definitions/checks/foreman_openscap/invalid_report_associations.rb
+- definitions/checks/foreman_proxy/check_tftp_storage.rb
 - definitions/checks/foreman_proxy/verify_dhcp_config_syntax.rb
 - definitions/checks/foreman_tasks/invalid/check_old.rb
 - definitions/checks/foreman_tasks/invalid/check_pending_state.rb
@@ -224,6 +228,7 @@ files:
 - definitions/procedures/foreman/apipie_cache.rb
 - definitions/procedures/foreman/fix_corrupted_roles.rb
 - definitions/procedures/foreman/remove_duplicate_obsolete_roles.rb
+- definitions/procedures/foreman/remove_duplicate_permissions.rb
 - definitions/procedures/foreman_docker/remove_foreman_docker.rb
 - definitions/procedures/foreman_openscap/invalid_report_associations.rb
 - definitions/procedures/foreman_proxy/features.rb
@@ -284,8 +289,6 @@ files:
 - definitions/scenarios/packages.rb
 - definitions/scenarios/restore.rb
 - definitions/scenarios/services.rb
-- definitions/scenarios/upgrade_to_capsule_6_7.rb
-- definitions/scenarios/upgrade_to_capsule_6_7_z.rb
 - definitions/scenarios/upgrade_to_capsule_6_8.rb
 - definitions/scenarios/upgrade_to_capsule_6_8_z.rb
 - definitions/scenarios/upgrade_to_satellite_6_2.rb
@@ -401,7 +404,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.8
+rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
 summary: Foreman maintenance tool belt

data/definitions/scenarios/upgrade_to_capsule_6_7.rb

@@ -1,88 +0,0 @@
-module Scenarios::Capsule_6_7
-  class Abstract < ForemanMaintain::Scenario
-    def self.upgrade_metadata(&block)
-      metadata do
-        tags :upgrade_scenario
-        confine do
-          feature(:capsule) &&
-            (feature(:capsule).current_minor_version == '6.6' || \
-            ForemanMaintain.upgrade_in_progress == '6.7')
-        end
-        instance_eval(&block)
-      end
-    end
-
-    def target_version
-      '6.7'
-    end
-  end
-
-  class PreUpgradeCheck < Abstract
-    upgrade_metadata do
-      description 'Checks before upgrading to Capsule 6.7'
-      tags :pre_upgrade_checks
-      run_strategy :fail_slow
-    end
-
-    def compose
-      add_steps(find_checks(:default))
-      add_steps(find_checks(:pre_upgrade))
-      add_step(Checks::Repositories::Validate.new(:version => '6.7'))
-    end
-  end
-
-  class PreMigrations < Abstract
-    upgrade_metadata do
-      description 'Procedures before migrating to Capsule 6.7'
-      tags :pre_migrations
-    end
-
-    def compose
-      add_steps(find_procedures(:pre_migrations))
-      add_step(Procedures::Service::Stop.new)
-    end
-  end
-
-  class Migrations < Abstract
-    upgrade_metadata do
-      description 'Migration scripts to Capsule 6.7'
-      tags :migrations
-    end
-
-    def set_context_mapping
-      context.map(:assumeyes, Procedures::Installer::Upgrade => :assumeyes)
-    end
-
-    def compose
-      add_step(Procedures::Repositories::Setup.new(:version => '6.7'))
-      add_step(Procedures::Packages::UnlockVersions.new)
-      add_step(Procedures::Packages::Update.new(:assumeyes => true))
-      add_step_with_context(Procedures::Installer::Upgrade)
-    end
-  end
-
-  class PostMigrations < Abstract
-    upgrade_metadata do
-      description 'Procedures after migrating to Capsule 6.7'
-      tags :post_migrations
-    end
-
-    def compose
-      add_step(Procedures::Service::Start.new)
-      add_steps(find_procedures(:post_migrations))
-    end
-  end
-
-  class PostUpgradeChecks < Abstract
-    upgrade_metadata do
-      description 'Checks after upgrading to Capsule 6.7'
-      tags :post_upgrade_checks
-      run_strategy :fail_slow
-    end
-
-    def compose
-      add_steps(find_checks(:default))
-      add_steps(find_checks(:post_upgrade))
-    end
-  end
-end

data/definitions/scenarios/upgrade_to_capsule_6_7_z.rb

@@ -1,88 +0,0 @@
-module Scenarios::Capsule_6_7_z
-  class Abstract < ForemanMaintain::Scenario
-    def self.upgrade_metadata(&block)
-      metadata do
-        tags :upgrade_scenario
-        confine do
-          feature(:capsule) &&
-            (feature(:capsule).current_minor_version == '6.7' || \
-              ForemanMaintain.upgrade_in_progress == '6.7.z')
-        end
-        instance_eval(&block)
-      end
-    end
-
-    def target_version
-      '6.7.z'
-    end
-  end
-
-  class PreUpgradeCheck < Abstract
-    upgrade_metadata do
-      description 'Checks before upgrading to Capsule 6.7.z'
-      tags :pre_upgrade_checks
-      run_strategy :fail_slow
-    end
-
-    def compose
-      add_steps(find_checks(:default))
-      add_steps(find_checks(:pre_upgrade))
-      add_step(Checks::Repositories::Validate.new(:version => '6.7'))
-    end
-  end
-
-  class PreMigrations < Abstract
-    upgrade_metadata do
-      description 'Procedures before migrating to Capsule 6.7.z'
-      tags :pre_migrations
-    end
-
-    def compose
-      add_steps(find_procedures(:pre_migrations))
-      add_step(Procedures::Service::Stop.new)
-    end
-  end
-
-  class Migrations < Abstract
-    upgrade_metadata do
-      description 'Migration scripts to Capsule 6.7.z'
-      tags :migrations
-    end
-
-    def set_context_mapping
-      context.map(:assumeyes, Procedures::Installer::Upgrade => :assumeyes)
-    end
-
-    def compose
-      add_step(Procedures::Repositories::Setup.new(:version => '6.7'))
-      add_step(Procedures::Packages::UnlockVersions.new)
-      add_step(Procedures::Packages::Update.new(:assumeyes => true))
-      add_step_with_context(Procedures::Installer::Upgrade)
-    end
-  end
-
-  class PostMigrations < Abstract
-    upgrade_metadata do
-      description 'Procedures after migrating to Capsule 6.7.z'
-      tags :post_migrations
-    end
-
-    def compose
-      add_step(Procedures::Service::Start.new)
-      add_steps(find_procedures(:post_migrations))
-    end
-  end
-
-  class PostUpgradeChecks < Abstract
-    upgrade_metadata do
-      description 'Checks after upgrading to Capsule 6.7.z'
-      tags :post_upgrade_checks
-      run_strategy :fail_slow
-    end
-
-    def compose
-      add_steps(find_checks(:default))
-      add_steps(find_checks(:post_upgrade))
-    end
-  end
-end