foreman_acd 0.0.5 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d71f9888fdae570d63f3983bb8b0dc248feb364ff0d4c1703bedeb6b75208c16
-  data.tar.gz: e84d3d6d2002d2e9d151d27b965ffec73ea8a10c3f5c706d295310978600a1b0
+  metadata.gz: 0d211aae9186f8f78068084f48a57c33bcc12840c8c187345a195ce90b474b62
+  data.tar.gz: ffdefcc21d390b4cd379cae7f5884a1660303408824206bcede99dfc00cac23b
 SHA512:
-  metadata.gz: 2460f666269432d8f3f62b983d48927b4849933ce5be2172c2a9bfdd4ec8540cf9656cb70e93773da09e2e74bc4b5f9317556985b2552b60b5af6d0a8e8a49ca
-  data.tar.gz: a03268372b9c9f8a5831f566d2d269df7a96b38b0295bdf973b183f93843c023307091a42b96159871c4efdda485e040b1edabc260c33193369a56935a55cc5b
+  metadata.gz: 596cce7e2dc9f22f8e53da0060d327e3649963a03df3b89bc60e454698c8318f96d2500dc266ef3a430da959437601154ee00682a416baeaaa0db119f2027d36
+  data.tar.gz: 357f087227ff418cf2ed0cbcf18e0ec5487fdb55c985c35fe55dea387f93db7eb1ec4268a312e325f925621bdb2cf16fdad1d06b13e893bce672e8d1aa03d2c3

data/README.md

@@ -4,67 +4,123 @@
 
 A plugin to bring an user self service portal and application centric deployment to Foreman.
 
+
 # Description
 
-The target of this plugin is, to deploy whole applications which include multiple hosts 
-and an Ansible Playbook / saltstack state to configure the application. 
+The target of this plugin is, to deploy whole applications which include multiple hosts and 
+configure them using an Ansible Playbook / saltstack state.
 
 This plugin follows the idea of different user types working together.
-The administrative user creates application definitions including multiple servers and
+The administrative user creates Application Definition including multiple servers and
 configuration management items (Ansible Playbook, saltstack state).
-The user can create and deploy new application instances with an easy to use self service portal. 
+The user can create and deploy new Application Instaces with an easy to use self service portal. 
 
-*Example Application Definition:*
-To run a complex web application, a loadbalancer is required.
+*Example Application Definition:* To run a complex web application, a loadbalancer is required.
 The loadbalancer routes the requests to 3 different web servers.
 The web servers are using a database which is in high availability mode on 2 hosts.
 => 6 hosts are required.
 
 This plugin aims to setup all 6 hosts and to deploy the application.
 
+
 # Current State
-In the current state the plugin can be used to provide an easy to use self service user portal 
-to deploy new servers. 
 
-# Road Map
-- Self service portal for single host deployments (current version)
-- Add application deployment with single host requirements 
-- Add application deployment with multi host requirements 
+## Application Definition
+
+- Configure a ansible playbook 
+- Use the configured ansible playbook in a Application Definition
+- Overwrite ansible playbook's group variables for the Application Definition
+- Set foreman parameters in the Application Definition
+- Setup various services like webservers, database-servers, etc.
+
+## Application Instance
+
+- Use an Application Definition for your Application Instance
+- Configure specific hosts which uses the Application Definitions services
+- Deploy these hosts
+- Configure the hosts using the configured ansible playbook
+
+# How it works
+
+- Configure ansible playbook, application definition and create an instance.
+- If you deploy the application instance, all hosts are created.
+- When done, you can configure the hosts with the linked ansible-playbook.
+- To do so, the [Smart Proxy ACD](https://github.com/ATIX-AG/smart_proxy_acd) is used.
+- The job to configure the hosts will be send to the Smart Proxy ACD which will
+    - download the ansible playbook from foreman (provided by an foreman_acd API)
+    - extract the ansible playbook on the Smart Proxy
+    - run the ansible-playbook on the Smart Proxy
+- On the Monitor -> Job page you see the output of the ansible-playbook run.
+
 
 ## WARNING
 
 This plugin is in development. 
-In the current state, a self service portal to deploy single servers can be created.
 
 ## Installation
 
-See [How_to_Install_a_Plugin](http://projects.theforeman.org/projects/foreman/wiki/How_to_Install_a_Plugin)
-for how to install Foreman plugins
+See [How_to_Install_a_Plugin](https://theforeman.org/plugins/#2.Installation)
+for how to install Foreman plugins. 
+
+You will need to install [Smart Proxy ACD](https://github.com/ATIX-AG/smart_proxy_acd), too. 
+
+### TL;DR: 
+
+    yum install tfm-rubygem-foreman_acd tfm-rubygem-smart_proxy_acd tfm-rubygem-smart_proxy_acd_core
+    foreman-maintain service restart
+
+In some cases you need to do manally
 
-ATM, Katello plugin need to exist, too.
+    foreman-rake db:migrate
+    foreman-rake db:seed
+
+### Hints
+
+Katello plugin need to exist, too.
 
 ## Usage
 
-Application Definition (Admin)
+### Ansible Playbook
+
+* Copy (or checkout a git repository) to e.g. /opt/ansible-playbook
+* Add a new Ansible Playbook via Configure -> Ansible Playbook
+* Set the path to /opt/ansible-playbook and name the playbook file. (e.g. site.yml)
+* Save it and press "Import group variables" for this newly created ansible playbook.
+
+### Application Definition (Admin)
+
 * Create an Application Definition (Configure -> Application Definition) first
-* Select the Hostgroup you want to use. 
+* Select the Ansible Playbook you want to use. 
+* Add new services and specifiy the host group you want to use.
 * Specifiy the values, a user could overwrite.
   (you can set a default value, if you want)
 
-Application Instance (User)
-* Create an Application Instance (Configure -> Application Instrane) 
+### Application Instance (User)
+
+* Create an Application Instance (Configure -> Application Instance) 
 * Select the Application Definition which should be used
 * Set the values.
   Remember, all parameters need to have a value. 
 * Save the Application Instance
-* To Deploy the host, select "Deploy" in the Action selection dropdown field
-  on the Application Instance index site
+
+### Deploy & Configure the Application Instance (User)
+
+* To Deploy the host, select "Deploy" in the action selection dropdown field
+  on the Application Instance index site.
+* See if the hosts are deployed via action selection dropdown -> report.
+* After all hosts are deployed, run the ansible playbook via 
+  action selection dropdown -> Run ansible playbook
+
 
 ## TODO
 
-- Add ansible playbook / saltstack support to configure the application
-- Multi-host support
-- Add validation to the different parameter types
+- "git" support for the ansible playbooks
+- Provide application templates which contains application definition and 
+  the required ansible-playbook.
+- Add saltstack support to configure the application
+- Automatically run the ansible playbook after all hosts are deployed
+- More parameter / value validation
+
 
 ## Contributing
 
@@ -72,7 +128,7 @@ Fork and send a Pull Request. Thanks!
 
 ## Copyright
 
-Copyright (c) 2019 ATIX AG 
+Copyright (c) 2021 ATIX AG 
 
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
@@ -86,4 +142,3 @@ GNU General Public License for more details.
 
 You should have received a copy of the GNU General Public License
 along with this program.  If not, see <http://www.gnu.org/licenses/>.
-

data/app/controllers/foreman_acd/ansible_playbooks_controller.rb

@@ -0,0 +1,122 @@
+# frozen_string_literal: true
+
+module ForemanAcd
+  # Ansible Playbook Controller
+  class AnsiblePlaybooksController < ::ForemanAcd::ApplicationController
+    include Foreman::Controller::AutoCompleteSearch
+    include ::ForemanAcd::Concerns::AnsiblePlaybookParameters
+
+    before_action :find_resource, :only => [:edit, :update, :destroy, :import_vars]
+
+    def index
+      @ansible_playbooks = resource_base.search_for(params[:search], :order => params[:order]).paginate(:page => params[:page])
+    end
+
+    def new
+      @ansible_playbook = AnsiblePlaybook.new
+    end
+
+    def create
+      @ansible_playbook = AnsiblePlaybook.new(ansible_playbook_params)
+      if @ansible_playbook.save
+        process_success
+      else
+        process_error
+      end
+    end
+
+    def edit
+    end
+
+    def update
+      if @ansible_playbook.update(ansible_playbook_params)
+        process_success
+      else
+        process_error
+      end
+    end
+
+    def destroy
+      if @ansible_playbook.destroy
+        process_success
+      else
+        process_error
+      end
+    end
+
+    def action_permission
+      case params[:action]
+      when 'import_vars'
+        :import_vars
+      else
+        super
+      end
+    end
+
+    # We need to move these to a smart_proxy_acd
+    def extract_variables(playbook_path)
+      errors = []
+      vars = {}
+
+      unless File.directory?(playbook_path) || File.directory?("#{playbook_path}/group_vars")
+        errors << "Playbook path '#{playbook_path}' or '#{playbook_path}/group_vars' doesn't exist"
+        return vars, errors
+      end
+
+      everything_empty = true
+
+      vars_files = Dir.glob("#{playbook_path}/group_vars/**/*")
+      vars_files.each do |vars_file|
+        loaded_yaml = {}
+        next if File.directory?(vars_file)
+
+        begin
+          loaded_yaml = YAML.load_file(vars_file)
+        rescue Psych::SyntaxError
+          err = "#{vars_file} is not YAML file"
+          logger.error(err)
+          errors << err
+        end
+        unless loaded_yaml.is_a? Hash
+          err = "Could not parse YAML file #{vars_file}"
+          logger.error(err)
+          errors << err
+        end
+        everything_empty = false unless loaded_yaml.empty?
+
+        # We need to support: group_vars/group_file and group_vars/group_dir/yaml_files
+        dir_and_file = File.split(vars_file)
+
+        if File.basename(dir_and_file[0]) == 'group_vars' # in case of group_vars/group_file
+          group_name = File.basename(dir_and_file[1], '.*')
+        else # in case of group_vars/group_dir/yaml_files
+          group_name = File.basename(dir_and_file[0])
+        end
+
+        logger.debug("Add ansible vars from file #{vars_file} to group #{group_name}")
+
+        if vars.has_key?(group_name)
+          vars[group_name].merge!(loaded_yaml)
+        else
+          vars[group_name] = loaded_yaml
+        end
+      end
+
+      errors << "No ansible group variable in #{playbook_path} defined." if everything_empty
+
+      return vars, errors
+    end
+
+    def import_vars
+      logger.debug("Load ansible group vars for #{@ansible_playbook} from #{@ansible_playbook.path}")
+      vars, errors = extract_variables(@ansible_playbook.path)
+      @ansible_playbook.vars = vars.to_json
+      @ansible_playbook.save
+      if errors.empty?
+        process_success :success_msg => _("Successfully loaded ansible group variables from %s") % @ansible_playbook.name, :redirect => ansible_playbooks_path
+      else
+        process_error :error_msg => _(errors.join(' ')), :redirect => ansible_playbooks_path
+      end
+    end
+  end
+end

data/app/controllers/foreman_acd/api/v2/ansible_playbooks_controller.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+module ForemanAcd
+  module Api
+    module V2
+      # API controller for Ansible Playbooks
+      class AnsiblePlaybooksController < ::ForemanAcd::Api::V2::BaseController
+        include ::Foreman::Controller::SmartProxyAuth
+        include ::ForemanAcd::Concerns::AnsiblePlaybookParameters
+
+        before_action :find_resource, :except => [:index, :create, :grab]
+
+        add_smart_proxy_filters :grab, :features => 'ACD'
+
+        api :GET, '/ansible_playbooks/:id', N_('Show ansible playbook')
+        param :id, :identifier, :required => true
+        def show; end
+
+        api :GET, '/ansible_playbooks', N_('List ansible playbooks')
+        param_group :search_and_pagination, ::Api::V2::BaseController
+        add_scoped_search_description_for(AnsiblePlaybook)
+        def index
+          @ansible_playbooks = resource_scope_for_index
+        end
+
+        def_param_group :ansible_playbook do
+          param :ansible_playbook, Hash, :required => true, :action_aware => true do
+            param :name, String, :required => true
+            param :description, String, :required => true
+            param :services, String, :required => true
+          end
+        end
+
+        api :POST, '/ansible_playbooks', N_('Create a ansible playbook')
+        param_group :ansible_playbook, :as => :create
+        def create
+          @ansible_playbook = AnsiblePlaybook.new(ansible_playbook_params)
+          process_response @ansible_playbook.save
+        end
+
+        api :DELETE, '/ansible_playbooks/:id', N_('Deletes ansible playbook')
+        param :id, :identifier, :required => true
+        def destroy
+          process_response @ansible_playbook.destroy
+        end
+
+        api :GET, '/ansible_playbooks/:id/grab', N_('Grab ansible playbook')
+        param :id, :identifier, :required => true
+        def grab
+          ap = resource_class.find(params['id'])
+          command = "tar cz -C #{ap.path} --exclude \".git\" . 2>/dev/null | base64"
+          result = %x<#{command}>
+          send_data result, type: "text/plain", disposition: 'inline'
+        end
+
+        def action_permission
+          case params[:action]
+          when 'grab'
+            'grab'
+          else
+           super
+          end
+        end
+
+        def resource_class
+          ForemanAcd::AnsiblePlaybook
+        end
+      end
+    end
+  end
+end

data/app/controllers/foreman_acd/api/v2/app_definitions_controller.rb

@@ -24,8 +24,7 @@ module ForemanAcd
           param :app_definition, Hash, :required => true, :action_aware => true do
             param :name, String, :required => true
             param :description, String, :required => true
-            param :hostgroup_id, :number, :required => true
-            param :parameters, String, :required => true
+            param :services, String, :required => true
           end
         end
 

data/app/controllers/foreman_acd/api/v2/app_instances_controller.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+module ForemanAcd
+  module Api
+    module V2
+      # API controller for App Instances
+      class AppInstancesController < ::ForemanAcd::Api::V2::BaseController
+        include ::ForemanAcd::Concerns::AppInstanceParameters
+
+        before_action :find_resource, :except => [:index, :create]
+
+        api :GET, '/app_instances/:id', N_('Show application instance')
+        param :id, :identifier, :required => true
+        def show; end
+
+        api :GET, '/app_instances', N_('List application instances')
+        param_group :search_and_pagination, ::Api::V2::BaseController
+        add_scoped_search_description_for(AppInstance)
+        def index
+          @app_instances = resource_scope_for_index
+        end
+
+        def_param_group :app_instance do
+          param :app_instance, Hash, :required => true, :action_aware => true do
+            param :name, String, :required => true
+            param :description, String, :required => true
+            param :services, String, :required => true
+          end
+        end
+
+        api :POST, '/app_instances', N_('Create a application instance')
+        param_group :app_instance, :as => :create
+        def create
+          @app_instance = AppInstance.new(app_instance_params)
+          process_response @app_instance.save
+        end
+
+        api :DELETE, '/app_instances/:id', N_('Deletes application instance')
+        param :id, :identifier, :required => true
+        def destroy
+          process_response @app_instance.destroy
+        end
+
+        def controller_permission
+          'app_instances'
+        end
+
+        def resource_class
+          ForemanAcd::AppInstance
+        end
+      end
+    end
+  end
+end

data/app/controllers/foreman_acd/app_definitions_controller.rb

@@ -6,18 +6,25 @@ module ForemanAcd
     include Foreman::Controller::AutoCompleteSearch
     include ::ForemanAcd::Concerns::AppDefinitionParameters
 
-    before_action :find_resource, :only => [:edit, :update, :destroy]
+    before_action :find_resource, :only => [:edit, :update, :destroy, :export]
+    before_action :read_hostgroups, :only => [:edit, :new, :import]
+    before_action :read_ansible_playbooks, :only => [:edit, :new]
+    before_action :handle_file_upload, :only => [:create]
 
     def index
       @app_definitions = resource_base.search_for(params[:search], :order => params[:order]).paginate(:page => params[:page])
     end
 
+    def read_ansible_playbooks
+      # Only use ansible playbooks for which the user pressed import group vars once.
+      @ansible_playbooks = AnsiblePlaybook.where.not(vars: nil).map { |elem| { elem.id => elem.name } }.reduce({}) { |h, v| h.merge v }
+    end
+
     def read_hostgroups
       @hostgroups = Hostgroup.all.map { |elem| { elem.id => elem.name } }.reduce({}) { |h, v| h.merge v }
     end
 
     def new
-      read_hostgroups
       @app_definition = AppDefinition.new
     end
 
@@ -31,11 +38,10 @@ module ForemanAcd
     end
 
     def edit
-      read_hostgroups
     end
 
     def update
-      if @app_definition.update(app_definition_params)
+      if @app_definition.update_attributes(app_definition_params)
         process_success
       else
         process_error
@@ -49,5 +55,31 @@ module ForemanAcd
         process_error
       end
     end
+
+    def action_permission
+      case params[:action]
+      when 'export'
+        :export
+      else
+        super
+      end
+    end
+
+    def import
+      @app_definition = AppDefinition.new
+    end
+
+    def export
+      filename = "#{@app_definition.name}.yaml"
+      data = JSON.parse(@app_definition.services).to_yaml
+      send_data data, :type => 'text/yaml', :disposition => 'attachment', :filename => filename
+    end
+
+    private
+
+    def handle_file_upload
+      return unless params[:foreman_acd_app_definition] && raw_file = params[:foreman_acd_app_definition][:app_definition_file]
+      params[:foreman_acd_app_definition][:services] = YAML.load_file(raw_file.tempfile).to_json
+    end
   end
 end