foreman_acd 0.2.1 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ef0b36ad161877bede4fe9a9df532a8201fc31710b1f42ab988152aeba863cac
-  data.tar.gz: a629384ff81f18c21883ea4a09a76828988bff02e6a709a7f2151e77fff46836
+  metadata.gz: 7b616c7ba21eea339af8509f618585dcd949ddead6ecdec57278a62c751e4fd2
+  data.tar.gz: 75c7d2e2c07d306dee127bc50c6b300934f90066bc4a491e49f17f65ac6b7c24
 SHA512:
-  metadata.gz: 8c8843594e0cbad7c6b85398c8b4d2c8fe18703ef7dc5c2ebd4fd46ccd0690a3cf30588e13dcef66e9d568fa4c5ca97fd272ebb5f5107e3f2ad1258f9fb9d35f
-  data.tar.gz: 99181b0218e89485fffedfdd1374b095c1830614718332246f72238af50adb65610e03fdaa0a035f6cd68208d8f72679b141eac2b558dc1824abb30b21120b56
+  metadata.gz: bba62637508bbbd4e35772a55ad1126fea7a8cb3c1658c6f734b02ccf03d44048db40fa621bb51e796220668e7a49e8b03c3169a77b818f6dbbde117d09edfe2
+  data.tar.gz: a6e4ba1bb97250e8bcebe6180abcf4755859b73828febda64353b47021acadcc33fd99e2ab4b45ac965e5d643c368bbfb47aee421a80979c7a4c69354a18b297

data/README.md

@@ -2,88 +2,139 @@
 
 # Foreman Application Centric Deployment
 
-A plugin to bring an user self service portal and application centric deployment to Foreman.
+A Foreman plugin providing application centric deployment and a self-service portal.
 
-# Description
+# Introduction
 
-The target of this plugin is, to deploy whole applications which include multiple hosts 
-and an Ansible Playbook / saltstack state to configure the application. 
+The target of this plugin is to deploy whole applications which include multiple hosts and configure them using an Ansible playbook.
 
 This plugin follows the idea of different user types working together.
-The administrative user creates application definitions including multiple servers and
-configuration management items (Ansible Playbook, saltstack state).
-The user can create and deploy new application instances with an easy to use self service portal. 
+The _administrative user_ creates Application Definitions including multiple servers and configuration management items (Ansible Playbooks).
+The _user_ can create and deploy new Application Instances with an easy to use self-service portal.
 
-*Example Application Definition:*
-To run a complex web application, a loadbalancer is required.
-The loadbalancer routes the requests to 3 different web servers.
-The web servers are using a database which is in high availability mode on 2 hosts.
-=> 6 hosts are required.
+*Example Application Definition*
+To run a complex web application, a load balancer is required.
+The load balancer routes the requests to three different web servers.
+The web servers are using a database which runs in high availability mode on two hosts.
+Therefore, six hosts in total are required.
 
-This plugin aims to setup all 6 hosts and to deploy the application.
+This plugin aims to setup all six hosts and to deploy the application.
 
-# Current State
-In the current state the plugin can be used to provide an easy to use self service user portal 
-to deploy new servers. 
+# Architecture
 
-# Road Map
-- Self service portal for single host deployments (current version)
-- Add application deployment with single host requirements 
-- Add application deployment with multi host requirements 
+## Ansible Playbooks
 
-## WARNING
+* Specify the path on your Foreman server to the Ansible playbook and playfile
+* Read groups configured in the Ansible playbook
 
-This plugin is in development. 
-In the current state, a self service portal to deploy single servers can be created.
+## Application Definitions
+
+* Use the configured Ansible playbook in an Application Definition
+* Overwrite group variables of the Ansible playbook for the Application Definition
+* Set Foreman parameters in the Application Definition
+* Setup various services like web servers, database servers, etc.
+
+## Application Instances
+
+* Use an Application Definition for your Application Instance
+* Configure specific hosts which use the Application Definition's services
+* Deploy these hosts
+* Configure the hosts using the configured Ansible playbook
+
+# How It Works
+
+* Configure an Ansible playbook, an Application Definition, and create an Application Instance.
+* All hosts are created when deploying the Application Instance.
+* After provisioning, the hosts are configured with the linked Ansible playbook.
+* This uses the [Smart Proxy ACD](https://github.com/ATIX-AG/smart_proxy_acd) component.
+* The job to configure the hosts will be send to the Smart Proxy ACD component which will
+    * download the Ansible playbook from your Foreman server (provided by an foreman_acd API);
+    * extract the Ansible playbook on the Smart Proxy;
+    * and run the Ansible playbook on the Smart Proxy.
+* You can see the output of the Ansible playbook run on the *Monitor > Job* page.
+
+:warning: This plugin is still in development.
 
 ## Installation
 
-See [How_to_Install_a_Plugin](http://projects.theforeman.org/projects/foreman/wiki/How_to_Install_a_Plugin)
-for how to install Foreman plugins
+See the [installation](https://theforeman.org/plugins/#2.Installation) chapter of the Foreman plugins documentation on how to install Foreman plugins.
+
+### TL;DR: 
+
+    yum install tfm-rubygem-foreman_acd
+    foreman-maintain service restart
+
+In some cases, you need to manually run
+
+    foreman-rake db:migrate
+    foreman-rake db:seed
+
+### Smart Proxy Installation
+
+You will need to install [Smart Proxy ACD](https://github.com/ATIX-AG/smart_proxy_acd), too. 
 
-ATM, Katello plugin need to exist, too.
+    yum install tfm-rubygem-smart_proxy_acd tfm-rubygem-smart_proxy_acd_core
+    foreman-maintain service restart
+
+You need to refresh the smart proxy features in *Infrastructure > Smart Proxies > Your Smart-Proxy > Actions > Refresh* after the installation of the Smart Proxy ACD components.
+
+### Tips
+
+* Make sure you have the [Katello](https://theforeman.org/plugins/katello/) plugin installed.
+* Make sure the Job Template `Run ACD Ansible Playbook - ACD Default` is part of your organization/location context.
 
 ## Usage
 
-Application Definition (Admin)
-* Create an Application Definition (Configure -> Application Definition) first
-* Select the Hostgroup you want to use. 
-* Specifiy the values, a user could overwrite.
-  (you can set a default value, if you want)
-
-Application Instance (User)
-* Create an Application Instance (Configure -> Application Instrane) 
-* Select the Application Definition which should be used
-* Set the values.
-  Remember, all parameters need to have a value. 
-* Save the Application Instance
-* To Deploy the host, select "Deploy" in the Action selection dropdown field
-  on the Application Instance index site
+### Ansible Playbook
+
+* Copy (or checkout a git repository) an Ansible playbook.
+Store it in `/etc/foreman/plugins/foreman_acd/ansible-playbooks/` so that SELinux is able to read it.
+* Add a new Ansible Playbook via *Applications > Ansible Playbooks*.
+* Specify the path to the Ansible playbook and name of the playbook file. (e.g. `site.yml`).
+* Save it and press *Import group variables* for this newly created Ansible playbook.
+
+### Application Definition (for Admins)
+
+* Create an Application Definition via *Applications > Application Definitions*.
+* Select the Ansible Playbook you want to use. 
+* Add new services and specify the host group you want to use.
+* Specify any values a user will be allowed to overwrite.
+You may also set a default value.
+
+### Application Instance (for Users)
+
+* Create an Application Instance via *Applications > Application Instances*.
+* Select the Application Definition you want to use.
+* Overwrite desired values.
+All Foreman parameters require a value.
+* Save the Application Instance.
+
+### Deploy and Configure the Application Instance (for Users)
+
+* Select *Deploy* in the action drop down menu via *Applications > Application Instance* to deploy a host.
+* Verify if the hosts are deployed via the *Report* button from the action drop down menu.
+* Run the Ansible playbook via *Run Ansible playbook* from the action drop down menu after all hosts are deployed.
 
 ## TODO
 
-- Add ansible playbook / saltstack support to configure the application
-- Multi-host support
-- Add validation to the different parameter types
+* Add `git` support for the Ansible playbooks.
+* Provide application templates which contains application definition and the required Ansible-playbook.
+* Add Saltstack support to configure the application.
+* Extend the Foreman parameter and value validation.
 
 ## Contributing
 
-Fork and send a Pull Request. Thanks!
+Fork and send a Pull Request.
+Thanks!
 
 ## Copyright
 
-Copyright (c) 2019 ATIX AG 
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
+Copyright (c) 2021 ATIX AG 
 
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
+This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
+This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+See the GNU General Public License for more details.
 
+You should have received a copy of the GNU General Public License along with this program.
+If not, see <http://www.gnu.org/licenses/>.

data/app/controllers/foreman_acd/ansible_playbooks_controller.rb

@@ -0,0 +1,199 @@
+# frozen_string_literal: true
+
+module ForemanAcd
+  # Ansible Playbook Controller
+  class AnsiblePlaybooksController < ::ForemanAcd::ApplicationController
+    include Foreman::Controller::AutoCompleteSearch
+    include ::ForemanAcd::Concerns::AnsiblePlaybookParameters
+
+    before_action :find_resource, :only => [:edit, :update, :destroy, :import_vars]
+    after_action :delete_synced_repo, :only => [:new, :edit, :create, :update, :destroy, :index]
+
+    def index
+      @ansible_playbooks = resource_base.search_for(params[:search], :order => params[:order]).paginate(:page => params[:page])
+    end
+
+    def new
+      @ansible_playbook = AnsiblePlaybook.new
+    end
+
+    def create
+      @ansible_playbook = AnsiblePlaybook.new(ansible_playbook_params)
+      if session[:git_path]
+        @ansible_playbook.update(:path => ansible_playbook_full_path(ansible_playbook_rename(@ansible_playbook[:name])))
+        FileUtils.mv(session[:git_path], @ansible_playbook[:path])
+        session[:git_path] = nil
+      end
+      if @ansible_playbook.save
+        process_success :success_msg => _("Successfully created %s. You need to press the \"Import groups\" button
+                                           before this ansible playbook can be used in App Definitions!") % @ansible_playbook
+      else
+        process_error
+      end
+    end
+
+    def edit; end
+
+    def update
+      # Move synced repo to new path if ansible_playbook name is changed
+      if !session[:git_path].nil? && ansible_playbook_params[:name] != @ansible_playbook[:name]
+        FileUtils.mv(@ansible_playbook[:path], ansible_playbook_full_path(ansible_playbook_rename(ansible_playbook_params[:name])))
+        @ansible_playbook.update(:path => ansible_playbook_full_path(ansible_playbook_rename(ansible_playbook_params[:name])))
+        session[:git_path] = nil
+
+      # Remove old version and copy new version of synced repository
+      elsif !session[:git_path].nil? && ansible_playbook_params[:name] == @ansible_playbook.name
+        remove_ansible_dir(@ansible_playbook[:path]) if @ansible_playbook.path
+        @ansible_playbook.update(:path => ansible_playbook_full_path(ansible_playbook_rename(@ansible_playbook[:name])))
+        FileUtils.mv(session[:git_path], @ansible_playbook[:path])
+        session[:git_path] = nil
+      end
+
+      if @ansible_playbook.update(ansible_playbook_params)
+        process_success
+      else
+        process_error
+      end
+    end
+
+    def destroy
+      if @ansible_playbook.destroy
+        process_success
+      else
+        process_error
+      end
+    end
+
+    def sync_git_repo
+      @ansible_playbook = AnsiblePlaybook.new
+      sync_params = params[:ansible_playbook]
+      dir = Dir.mktmpdir
+
+      begin
+        git = Git.init(dir)
+
+        if ForemanAcd.proxy_setting.present?
+          git.config('http.proxy', ForemanAcd.proxy_setting)
+          logger.info("HTTP Proxy used: #{git.config['http.proxy']}")
+        end
+
+        git.add_remote('origin', sync_params[:git_url])
+        git.fetch
+        git.checkout(sync_params[:git_commit])
+
+        session[:git_path] = git.dir.path
+      rescue StandardError => e
+        logger.error("Failed to sync git repository: #{e}")
+        render :json => { :status => 'error', :message => e }, :status => :internal_server_error
+      end
+    end
+
+    # Remove abandoned synced git repositories
+    def delete_synced_repo
+      names = []
+      AnsiblePlaybook.all.each do |ansible_playbook|
+        names.push(ansible_playbook_rename(ansible_playbook.name))
+      end
+      names.push('.', '..')
+      return unless Dir.exist?(ForemanAcd.ansible_playbook_path)
+      Dir.foreach(ForemanAcd.ansible_playbook_path) do |dirname|
+        next if names.include? dirname
+        remove_ansible_dir(ansible_playbook_full_path(dirname))
+        logger.info("Successfully removed #{dirname}")
+      end
+    end
+
+    def action_permission
+      case params[:action]
+      when 'import_vars'
+        :import_vars
+      when 'sync_git_repo'
+        :sync_git_repo
+      when 'grab'
+        :grab
+      else
+        super
+      end
+    end
+
+    # We need to move these to a smart_proxy_acd
+    def extract_variables(playbook_path)
+      errors = []
+      vars = {}
+
+      unless File.directory?(playbook_path) || File.directory?("#{playbook_path}/group_vars")
+        errors << "Playbook path '#{playbook_path}' or '#{playbook_path}/group_vars' doesn't exist"
+        return vars, errors
+      end
+
+      everything_empty = true
+
+      vars_files = Dir.glob("#{playbook_path}/group_vars/**/*")
+      vars_files.each do |vars_file|
+        loaded_yaml = {}
+        next if File.directory?(vars_file)
+
+        begin
+          loaded_yaml = YAML.load_file(vars_file)
+        rescue Psych::SyntaxError
+          err = "#{vars_file} is not YAML file"
+          logger.error(err)
+          errors << err
+        end
+        unless loaded_yaml.is_a? Hash
+          err = "Could not parse YAML file #{vars_file}"
+          logger.error(err)
+          errors << err
+        end
+        everything_empty = false unless loaded_yaml.empty?
+
+        # We need to support: group_vars/group_file and group_vars/group_dir/yaml_files
+        dir_and_file = File.split(vars_file)
+
+        group_name = if File.basename(dir_and_file[0]) == 'group_vars' # in case of group_vars/group_file
+                       File.basename(dir_and_file[1], '.*')
+                     else # in case of group_vars/group_dir/yaml_files
+                       File.basename(dir_and_file[0])
+                     end
+
+        logger.debug("Add ansible vars from file #{vars_file} to group #{group_name}")
+
+        if vars.key?(group_name)
+          vars[group_name].merge!(loaded_yaml)
+        else
+          vars[group_name] = loaded_yaml
+        end
+      end
+
+      errors << "No ansible group variable in #{playbook_path} defined." if everything_empty
+
+      [vars, errors]
+    end
+
+    def import_vars
+      logger.debug("Load ansible group vars for #{@ansible_playbook} from #{@ansible_playbook.path}")
+      vars, errors = extract_variables(@ansible_playbook.path)
+      @ansible_playbook.vars = vars.to_json
+      @ansible_playbook.save
+      if errors.empty?
+        process_success :success_msg => _('Successfully loaded ansible group variables from %s') % @ansible_playbook.name, :redirect => ansible_playbooks_path
+      else
+        process_error :error_msg => _(errors.join(' ')), :redirect => ansible_playbooks_path
+      end
+    end
+
+    private
+
+    def ansible_playbook_rename(name)
+      name.split(/\W+/).join('_')
+    end
+
+    def remove_ansible_dir(dirpath)
+      FileUtils.remove_dir(dirpath) if Dir.exist?(dirpath)
+    end
+
+    def ansible_playbook_full_path(dirname)
+      File.join(ForemanAcd.ansible_playbook_path, dirname)
+    end
+  end
+end

data/app/controllers/foreman_acd/api/v2/ansible_playbooks_controller.rb

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+module ForemanAcd
+  module Api
+    module V2
+      # API controller for Ansible Playbooks
+      class AnsiblePlaybooksController < ::ForemanAcd::Api::V2::BaseController
+        include ::Foreman::Controller::SmartProxyAuth
+        include ::ForemanAcd::Concerns::AnsiblePlaybookParameters
+
+        before_action :find_resource, :except => [:index, :create, :grab]
+
+        add_smart_proxy_filters :grab, :features => 'ACD'
+
+        api :GET, '/ansible_playbooks/:id', N_('Show ansible playbook')
+        param :id, :identifier, :required => true
+        def show; end
+
+        api :GET, '/ansible_playbooks', N_('List ansible playbooks')
+        param_group :search_and_pagination, ::Api::V2::BaseController
+        add_scoped_search_description_for(AnsiblePlaybook)
+        def index
+          @ansible_playbooks = resource_scope_for_index
+        end
+
+        def_param_group :ansible_playbook do
+          param :ansible_playbook, Hash, :required => true, :action_aware => true do
+            param :name, String, :required => true
+            param_group :taxonomies, ::Api::V2::BaseController
+            param :description, String, :required => true
+            param :services, String, :required => true
+          end
+        end
+
+        api :POST, '/ansible_playbooks', N_('Create a ansible playbook')
+        param_group :ansible_playbook, :as => :create
+        def create
+          @ansible_playbook = AnsiblePlaybook.new(ansible_playbook_params)
+          process_response @ansible_playbook.save
+        end
+
+        api :DELETE, '/ansible_playbooks/:id', N_('Deletes ansible playbook')
+        param :id, :identifier, :required => true
+        def destroy
+          process_response @ansible_playbook.destroy
+        end
+
+        api :GET, '/ansible_playbooks/:id/grab', N_('Grab ansible playbook')
+        param :id, :identifier, :required => true
+        def grab
+          ap = resource_class.find(params['id'])
+          command = "tar cz -C #{ap.path} --exclude \".git\" . 2>/dev/null | base64"
+          result = `#{command}`
+          send_data result, :type => 'text/plain', :disposition => 'inline'
+        end
+
+        def action_permission
+          case params[:action]
+          when 'grab'
+            'grab'
+          else
+            super
+          end
+        end
+
+        def resource_class
+          ForemanAcd::AnsiblePlaybook
+        end
+      end
+    end
+  end
+end