foreman_acd 0.4.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 433f2c68da81192db3c98fc013ed2149490f2471e84d372db8241522a33b6630
-  data.tar.gz: aa46c833de3feb1cce0802de82dec78f1610331984e6df8df649fdadfffa2b2a
+  metadata.gz: ca6ffd04ba17f399f8ecbf17615fde7ba6391cf389e16d83e427adb2c15cff9f
+  data.tar.gz: 9de27331a51e5bb6887250b01e40d6557fb9e5f66f96005f16e8d5fb2e6999f8
 SHA512:
-  metadata.gz: 0accbf314dd5776a5b012507d97619be98beff2c11e523da4d27f69e9fdb1bf1d284c134dba6549719312a66ede8cb80c8a06e3851f01a14951b1752ef2e29e8
-  data.tar.gz: d003bc7a5de1c180b2e33ade4e85b2912776db564d59ff9d8eca4f9ccc00c38591fd061649df07bb89e76ad0b0ac5efe1c64dd506132148aaf087b758a18c078
+  metadata.gz: ee8e1c5f4b7d842ce1f153c282b74a07a83edd6405e0b0ba7476602bd0c225bf8fefdb7e19aacfd2d79033f0c5429bc675e9c0dd1d794b2f1cfbe9d91764e9ed
+  data.tar.gz: 1d74484d949e080478432b0d76cba89b87fde3eda56a1d6cf7c69dc2207cdca2308a73b9f11de6a25d5713b2de12a546423b3d251d793925f55c33440cad108c

data/README.md

@@ -1,142 +1,142 @@
-[![Build Status master](https://travis-ci.org/ATIX-AG/foreman_acd.svg?branch=master)](https://travis-ci.org/ATIX-AG/foreman_acd)
-
 # Foreman Application Centric Deployment
 
-A plugin to bring an user self service portal and application centric deployment to Foreman.
-
+A Foreman plugin providing application centric deployment and a self-service portal.
 
-# Description
+# Introduction
 
-The target of this plugin is, to deploy whole applications which include multiple hosts and 
-configure them using an Ansible Playbook / saltstack state.
+The target of this plugin is to deploy whole applications which include multiple hosts and configure them using an Ansible playbook.
 
 This plugin follows the idea of different user types working together.
-The administrative user creates Application Definition including multiple servers and
-configuration management items (Ansible Playbook, saltstack state).
-The user can create and deploy new Application Instaces with an easy to use self service portal. 
+The _administrative user_ creates Application Definitions including multiple servers and configuration management items (Ansible Playbooks).
+The _user_ can create and deploy new Application Instances with an easy to use self-service portal.
+
+*Example Application Definition*
+To run a complex web application, a load balancer is required.
+The load balancer routes the requests to three different web servers.
+The web servers are using a database which runs in high availability mode on two hosts.
+Therefore, six hosts in total are required.
+
+This plugin aims to setup all six hosts and to deploy the application.
 
-*Example Application Definition:* To run a complex web application, a loadbalancer is required.
-The loadbalancer routes the requests to 3 different web servers.
-The web servers are using a database which is in high availability mode on 2 hosts.
-=> 6 hosts are required.
+# Architecture
 
-This plugin aims to setup all 6 hosts and to deploy the application.
+## Ansible Playbooks
 
+* Specify the path on your Foreman server to the Ansible playbook and playfile
+* Read groups configured in the Ansible playbook
 
-# Current State
+## Application Definitions
 
-## Application Definition
+* Use the configured Ansible playbook in an Application Definition
+* Overwrite group variables of the Ansible playbook for the Application Definition
+* Set Foreman parameters in the Application Definition
+* Setup various services like web servers, database servers, etc.
 
-- Configure a ansible playbook 
-- Use the configured ansible playbook in a Application Definition
-- Overwrite ansible playbook's group variables for the Application Definition
-- Set foreman parameters in the Application Definition
-- Setup various services like webservers, database-servers, etc.
+## Application Instances
 
-## Application Instance
+* Use an Application Definition for your Application Instance
+* Configure specific hosts which use the Application Definition's services
+* Deploy these hosts
+* Configure the hosts using the configured Ansible playbook
 
-- Use an Application Definition for your Application Instance
-- Configure specific hosts which uses the Application Definitions services
-- Deploy these hosts
-- Configure the hosts using the configured ansible playbook
+# How It Works
 
+* Configure an Ansible playbook, an Application Definition, and create an Application Instance.
+* All hosts are created when deploying the Application Instance.
+* After provisioning, the hosts are configured with the linked Ansible playbook.
+* This uses the [Smart Proxy ACD](https://github.com/ATIX-AG/smart_proxy_acd) component.
+* The job to configure the hosts will be send to the Smart Proxy ACD component which will
+    * download the Ansible playbook from your Foreman server (provided by an foreman_acd API);
+    * extract the Ansible playbook on the Smart Proxy;
+    * and run the Ansible playbook on the Smart Proxy.
+* You can see the output of the Ansible playbook run on the *Monitor > Job* page.
 
-# Road Map
-- 
-- Add Application deployment with single host requirements 
-- Add Application deployment with multi host requirements 
+:warning: This plugin is still in development.
 
-## WARNING
+## Documentation
 
-This plugin is in development. 
+See [Application Centric Deployment Guide](https://docs.theforeman.org/nightly/Application_Centric_Deployment/index-foreman-el.html)
 
 ## Installation
 
-See [How_to_Install_a_Plugin](https://theforeman.org/plugins/#2.Installation)
-for how to install Foreman plugins. 
+See the [installation](https://theforeman.org/plugins/#2.Installation) chapter of the Foreman plugins documentation on how to install Foreman plugins.
 
 ### TL;DR: 
 
     yum install tfm-rubygem-foreman_acd
-    foreman-maintain service restart --only foreman
+    foreman-maintain service restart
 
-In some cases you need to do manally
+In some cases, you need to manually run
 
     foreman-rake db:migrate
     foreman-rake db:seed
 
-### Hints
+### Smart Proxy Installation
 
-Katello plugin need to exist, too.
+You will need to install [Smart Proxy ACD](https://github.com/ATIX-AG/smart_proxy_acd), too. 
 
-### Configuration
+    yum install tfm-rubygem-smart_proxy_acd tfm-rubygem-smart_proxy_acd_core
+    foreman-maintain service restart
 
-To get ansible playbooks running, you need to:
+You need to refresh the smart proxy features in *Infrastructure > Smart Proxies > Your Smart-Proxy > Actions > Refresh* after the installation of the Smart Proxy ACD components.
 
-    cat /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub >> /root/.ssh/authorized_keys
-    
-Make sure, that the job template 'Run ACD Ansible Playbook - SSH Default' is part of your organization / location.     
+### Tips
+
+* Make sure you have the [Katello](https://theforeman.org/plugins/katello/) plugin installed.
+* Make sure the Job Template `Run ACD Ansible Playbook - ACD Default` is part of your organization/location context.
 
 ## Usage
 
 ### Ansible Playbook
 
-* Copy (or checkout a git repository) to e.g. /opt/ansible-playbook
-* Add a new Ansible Playbook via Configure -> Ansible Playbook
-* Set the path to /opt/ansible-playbook and name the playbook file. (e.g. site.yml)
-* Save it and press "Import group variables" for this newly created ansible playbook.
+* Copy (or checkout a git repository) an Ansible playbook.
+Store it in `/var/lib/foreman/foreman_acd/ansible-playbooks/` so that SELinux is able to read it.
+* Add a new Ansible Playbook via *Applications > Ansible Playbooks*.
+* Specify the path to the Ansible playbook and name of the playbook file. (e.g. `site.yml`).
+* Save it and press *Import group variables* for this newly created Ansible playbook.
 
-### Application Definition (Admin)
+### Application Definition (for Admins)
 
-* Create an Application Definition (Configure -> Application Definition) first
+* Create an Application Definition via *Applications > Application Definitions*.
 * Select the Ansible Playbook you want to use. 
-* Add new services and specifiy the host group you want to use.
-* Specifiy the values, a user could overwrite.
-  (you can set a default value, if you want)
-
-### Application Instance (User)
+* Add new services and specify the host group you want to use.
+* Specify any values a user will be allowed to overwrite.
+You may also set a default value.
 
-* Create an Application Instance (Configure -> Application Instance) 
-* Select the Application Definition which should be used
-* Set the values.
-  Remember, all parameters need to have a value. 
-* Save the Application Instance
+### Application Instance (for Users)
 
-### Deploy & Configure the Application Instance (User)
+* Create an Application Instance via *Applications > Application Instances*.
+* Select the Application Definition you want to use.
+* Overwrite desired values.
+All Foreman parameters require a value.
+* Save the Application Instance.
 
-* To Deploy the host, select "Deploy" in the action selection dropdown field
-  on the Application Instance index site.
-* See if the hosts are deployed via action selection dropdown -> report.
-* After all hosts are deployed, run the ansible playbook via 
-  action selection dropdown -> Run ansible playbook
+### Deploy and Configure the Application Instance (for Users)
 
+* Select *Deploy* in the action drop down menu via *Applications > Application Instance* to deploy a host.
+* Verify if the hosts are deployed via the *Report* button from the action drop down menu.
+* Run the Ansible playbook via *Run Ansible playbook* from the action drop down menu after all hosts are deployed.
 
 ## TODO
 
-- Add saltstack support to configure the application
-- "git" support for the ansible playbooks
-- Automatically run the ansible playbook after all hosts are deployed
-- More parameter / value validation
-- Deliver ansible playbooks to the connected foreman smart proxies
-
+* Add `git` support for the Ansible playbooks.
+* Provide application templates which contains application definition and the required Ansible-playbook.
+* Add Saltstack support to configure the application.
+* Extend the Foreman parameter and value validation.
 
 ## Contributing
 
-Fork and send a Pull Request. Thanks!
+Fork and send a Pull Request.
+Thanks!
 
 ## Copyright
 
-Copyright (c) 2020 ATIX AG 
+Copyright (c) 2021 ATIX AG 
 
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
+This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
+This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+See the GNU General Public License for more details.
 
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
+You should have received a copy of the GNU General Public License along with this program.
+If not, see <http://www.gnu.org/licenses/>.

data/app/controllers/foreman_acd/ansible_playbooks_controller.rb

@@ -7,6 +7,7 @@ module ForemanAcd
     include ::ForemanAcd::Concerns::AnsiblePlaybookParameters
 
     before_action :find_resource, :only => [:edit, :update, :destroy, :import_vars]
+    after_action :delete_synced_repo, :only => [:new, :edit, :create, :update, :destroy, :index]
 
     def index
       @ansible_playbooks = resource_base.search_for(params[:search], :order => params[:order]).paginate(:page => params[:page])
@@ -18,17 +19,36 @@ module ForemanAcd
 
     def create
       @ansible_playbook = AnsiblePlaybook.new(ansible_playbook_params)
+      if session[:git_path]
+        @ansible_playbook.update(:path => ansible_playbook_full_path(ansible_playbook_rename(@ansible_playbook[:name])))
+        FileUtils.mv(session[:git_path], @ansible_playbook[:path])
+        session[:git_path] = nil
+      end
       if @ansible_playbook.save
-        process_success
+        process_success :success_msg => _("Successfully created %s. You need to press the \"Import groups\" button
+                                           before this ansible playbook can be used in App Definitions!") % @ansible_playbook
       else
         process_error
       end
     end
 
-    def edit
-    end
+    def edit; end
 
     def update
+      # Move synced repo to new path if ansible_playbook name is changed
+      if !session[:git_path].nil? && ansible_playbook_params[:name] != @ansible_playbook[:name]
+        FileUtils.mv(@ansible_playbook[:path], ansible_playbook_full_path(ansible_playbook_rename(ansible_playbook_params[:name])))
+        @ansible_playbook.update(:path => ansible_playbook_full_path(ansible_playbook_rename(ansible_playbook_params[:name])))
+        session[:git_path] = nil
+
+      # Remove old version and copy new version of synced repository
+      elsif !session[:git_path].nil? && ansible_playbook_params[:name] == @ansible_playbook.name
+        remove_ansible_dir(@ansible_playbook[:path]) if @ansible_playbook.path
+        @ansible_playbook.update(:path => ansible_playbook_full_path(ansible_playbook_rename(@ansible_playbook[:name])))
+        FileUtils.mv(session[:git_path], @ansible_playbook[:path])
+        session[:git_path] = nil
+      end
+
       if @ansible_playbook.update(ansible_playbook_params)
         process_success
       else
@@ -44,10 +64,64 @@ module ForemanAcd
       end
     end
 
+    def sync_git_repo
+      @ansible_playbook = AnsiblePlaybook.new
+      sync_params = params[:ansible_playbook]
+      dir = Dir.mktmpdir
+
+      begin
+        git = Git.init(dir)
+
+        if ForemanAcd.proxy_setting.present?
+          git.config('http.proxy', ForemanAcd.proxy_setting)
+          logger.info("HTTP Proxy used: #{git.config['http.proxy']}")
+        end
+
+        if sync_params[:git_commit].empty?
+          if ForemanAcd.proxy_setting.present?
+            err_msg = _('Please set the Git Branch/Tag/Commit. This setting is necessary if a HTTP proxy is used!')
+            raise StandardError.new err_msg
+          else
+            commit = Git.ls_remote(sync_params[:git_url])['head'][:sha]
+          end
+        else
+          commit = sync_params[:git_commit]
+        end
+
+        git.add_remote('origin', sync_params[:git_url])
+        git.fetch
+        git.checkout(commit)
+
+        session[:git_path] = git.dir.path
+      rescue StandardError => e
+        logger.error("Failed to sync git repository: #{e}")
+        render :json => { :status => 'error', :message => e }, :status => :internal_server_error
+      end
+    end
+
+    # Remove abandoned synced git repositories
+    def delete_synced_repo
+      names = []
+      AnsiblePlaybook.all.each do |ansible_playbook|
+        names.push(ansible_playbook_rename(ansible_playbook.name))
+      end
+      names.push('.', '..')
+      return unless Dir.exist?(ForemanAcd.ansible_playbook_path)
+      Dir.foreach(ForemanAcd.ansible_playbook_path) do |dirname|
+        next if names.include? dirname
+        remove_ansible_dir(ansible_playbook_full_path(dirname))
+        logger.info("Successfully removed #{dirname}")
+      end
+    end
+
     def action_permission
       case params[:action]
       when 'import_vars'
         :import_vars
+      when 'sync_git_repo'
+        :sync_git_repo
+      when 'grab'
+        :grab
       else
         super
       end
@@ -87,15 +161,15 @@ module ForemanAcd
         # We need to support: group_vars/group_file and group_vars/group_dir/yaml_files
         dir_and_file = File.split(vars_file)
 
-        if File.basename(dir_and_file[0]) == 'group_vars' # in case of group_vars/group_file
-          group_name = File.basename(dir_and_file[1], '.*')
-        else # in case of group_vars/group_dir/yaml_files
-          group_name = File.basename(dir_and_file[0])
-        end
+        group_name = if File.basename(dir_and_file[0]) == 'group_vars' # in case of group_vars/group_file
+                       File.basename(dir_and_file[1], '.*')
+                     else # in case of group_vars/group_dir/yaml_files
+                       File.basename(dir_and_file[0])
+                     end
 
         logger.debug("Add ansible vars from file #{vars_file} to group #{group_name}")
 
-        if vars.has_key?(group_name)
+        if vars.key?(group_name)
           vars[group_name].merge!(loaded_yaml)
         else
           vars[group_name] = loaded_yaml
@@ -104,7 +178,7 @@ module ForemanAcd
 
       errors << "No ansible group variable in #{playbook_path} defined." if everything_empty
 
-      return vars, errors
+      [vars, errors]
     end
 
     def import_vars
@@ -113,10 +187,28 @@ module ForemanAcd
       @ansible_playbook.vars = vars.to_json
       @ansible_playbook.save
       if errors.empty?
-        process_success :success_msg => _("Successfully loaded ansible group variables from %s") % @ansible_playbook.name, :redirect => ansible_playbooks_path
+        process_success :success_msg => _('Successfully loaded ansible group variables from %s') % @ansible_playbook.name, :redirect => ansible_playbooks_path
       else
         process_error :error_msg => _(errors.join(' ')), :redirect => ansible_playbooks_path
       end
     end
+
+    private
+
+    def ansible_playbook_rename(name)
+      name.split(/\W+/).join('_')
+    end
+
+    def remove_ansible_dir(dirpath)
+      unless dirpath.start_with? ForemanAcd.ansible_playbook_path
+        logger.error("Sorry, the directory #{dirpath} is not within #{ForemanAcd.ansible_playbook_path} and will therefore not be cleaned up!")
+        return false
+      end
+      FileUtils.rm_rf(dirpath) if Dir.exist?(dirpath)
+    end
+
+    def ansible_playbook_full_path(dirname)
+      File.join(ForemanAcd.ansible_playbook_path, dirname)
+    end
   end
 end

data/app/controllers/foreman_acd/api/v2/ansible_playbooks_controller.rb

@@ -5,9 +5,12 @@ module ForemanAcd
     module V2
       # API controller for Ansible Playbooks
       class AnsiblePlaybooksController < ::ForemanAcd::Api::V2::BaseController
+        include ::Foreman::Controller::SmartProxyAuth
         include ::ForemanAcd::Concerns::AnsiblePlaybookParameters
 
-        before_action :find_resource, :except => [:index, :create]
+        before_action :find_resource, :except => [:index, :create, :grab]
+
+        add_smart_proxy_filters :grab, :features => 'ACD'
 
         api :GET, '/ansible_playbooks/:id', N_('Show ansible playbook')
         param :id, :identifier, :required => true
@@ -23,6 +26,7 @@ module ForemanAcd
         def_param_group :ansible_playbook do
           param :ansible_playbook, Hash, :required => true, :action_aware => true do
             param :name, String, :required => true
+            param_group :taxonomies, ::Api::V2::BaseController
             param :description, String, :required => true
             param :services, String, :required => true
           end
@@ -41,8 +45,22 @@ module ForemanAcd
           process_response @ansible_playbook.destroy
         end
 
-        def controller_permission
-          'ansible_playbooks'
+        api :GET, '/ansible_playbooks/:id/grab', N_('Grab ansible playbook')
+        param :id, :identifier, :required => true
+        def grab
+          ap = resource_class.find(params['id'])
+          command = "tar cz -C #{ap.path} --exclude \".git\" . 2>/dev/null | base64"
+          result = `#{command}`
+          send_data result, :type => 'text/plain', :disposition => 'inline'
+        end
+
+        def action_permission
+          case params[:action]
+          when 'grab'
+            'grab'
+          else
+            super
+          end
         end
 
         def resource_class

data/app/controllers/foreman_acd/api/v2/app_definitions_controller.rb

@@ -23,6 +23,7 @@ module ForemanAcd
         def_param_group :app_definition do
           param :app_definition, Hash, :required => true, :action_aware => true do
             param :name, String, :required => true
+            param_group :taxonomies, ::Api::V2::BaseController
             param :description, String, :required => true
             param :services, String, :required => true
           end

data/app/controllers/foreman_acd/api/v2/app_instances_controller.rb

@@ -11,18 +11,26 @@ module ForemanAcd
 
         api :GET, '/app_instances/:id', N_('Show application instance')
         param :id, :identifier, :required => true
+        param :organization_id, :identifier, :required => true
+        param :location_id, :identifier, :required => true
         def show; end
 
         api :GET, '/app_instances', N_('List application instances')
+        param :organization_id, :identifier, :required => true
+        param :location_id, :identifier, :required => true
         param_group :search_and_pagination, ::Api::V2::BaseController
         add_scoped_search_description_for(AppInstance)
         def index
-          @app_instances = resource_scope_for_index
+          scope = resource_scope_for_index.where(:organization => params[:organization_id]) if params[:organization_id].present?
+          scope = scope.where(:location => params[:location_id]) if params[:location_id].present?
+          @app_instances = scope
         end
 
         def_param_group :app_instance do
           param :app_instance, Hash, :required => true, :action_aware => true do
             param :name, String, :required => true
+            param :organization_id, :identifier, :required => true
+            param :location_id, :identifier, :required => true
             param :description, String, :required => true
             param :services, String, :required => true
           end