ncedit 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 10ab95a17f8d56d2402fcaed7806eb747d188ac9
-  data.tar.gz: cd3418a0471e0182176afab8a84b04dc5cb2fa85
+  metadata.gz: 7d5513ec51dfcc0ec6713acd877b0c5778fd1a08
+  data.tar.gz: fcdc71622c410f6ce96095a362aabe2e4834b561
 SHA512:
-  metadata.gz: 724f3a28bf6580603125026f92b98cc04f110d20485a48c3df1b6e02c8345321b5f2e6cded1599a4777afd069f6333f15cf79ab0608c750ad835f9d5266aaf87
-  data.tar.gz: 8e1aeee6e7c1fd300954540fca8b2ae3eebe7bc9dc7adf5a06206c8d9501d792e46ed7f66d67e46365b839ae41fe23ac8cde70ae000470e41d36f597bac2af87
+  metadata.gz: 53213593a3ce04d985e37e08b06782c43ae6969a595edfd044fcb1c80887cc92e127b99ede09e832fdf759ac16b24b461da22d084591bd995fe054f225530f8d
+  data.tar.gz: 2a2877fa3d0eaf0f3c8f81a96a912d3e41855d06eb19da19553dd3a22b3f2637e75d5d4e4c23cdef309daac3392eaca3c4f6a50a2daf9a9d5b1ce62b9663caa7

data/.gitignore

@@ -10,3 +10,4 @@
 # rspec failure tracking
 .rspec_status
 *.gem
+/.idea

data/README.md

@@ -3,7 +3,7 @@
 
 ncedit is a small utility program that lets you edit the Puppet Enterprise Node Classifier rules from the command line.
 
-Why would you want to do this given that we have the excellent [node_manager](https://forge.puppet.com/WhatsARanjit/node_manager) module already on the forge?  Well... lots of reasons.  First off, using puppet code to drive the Node Classifier means that you have to have the `node_manager` module alread installed, which means that you must already have your [classification rules](https://docs.puppet.com/pe/latest/console_classes_groups_getting_started.html) in to reference the module through [Code Manager](https://docs.puppet.com/pe/latest/code_mgr.html).  You could in-theory use Puppet Enterprise's new idempotent installer (just reinstall puppet over the top of itself) to fix this exact issue but then you still have the problem of how to classify your master in order to activate any other new rules (eg node-ttl) you want to use, which are associated with a new [role](https://docs.puppet.com/pe/latest/r_n_p_intro.html) for the Puppet Master.
+Why would you want to do this given that we have the excellent [node_manager](https://forge.puppet.com/WhatsARanjit/node_manager) module already on the forge?  Well... lots of reasons.  First off, using puppet code to drive the Node Classifier means that you have to have the `node_manager` module already installed, which means that you must already have your [classification rules](https://docs.puppet.com/pe/latest/console_classes_groups_getting_started.html) in to reference the module through [Code Manager](https://docs.puppet.com/pe/latest/code_mgr.html).  You could in-theory use Puppet Enterprise's new idempotent installer (just reinstall puppet over the top of itself) to fix this exact issue but then you still have the problem of how to classify your master in order to activate any other new rules (eg node-ttl) you want to use, which are associated with a new [role](https://docs.puppet.com/pe/latest/r_n_p_intro.html) for the Puppet Master.
 
 That's where this tool comes in since all you need is root shell on the Puppet Master.
 
@@ -14,7 +14,7 @@ You can:
 * Add or remove class parameters
 * Add or remove rules
 
-...All from the convenience of the CLI.  This also allows this tool to be called from scripts and other systems in order to setup Puppet Enterprise the way you want and with the minimum of effort.
+...All from the convenience of the CLI.  This allows us to be called from scripts and other systems in order to setup Puppet Enterprise the way you want and with the minimum of effort.
 
 You have a choice of running `ncedit` for each change you wish to make or collecting all of your edits into either a JSON or YAML file for bulk processing.
 
@@ -98,38 +98,102 @@ In each case the file needs to be ordered as follows:
 
 #### Ensuring changes
 * `ncedit` is idempotent so you may run the command as often as you like
+* Classes used in rules *must* already exist for classification to succeed, `--smart-update` will do a code redeploy for you if specified.
 
 ##### YAML
 ```shell
-ncedit  batch --yaml-file /path/to/batch.yaml
+ncedit batch --smart-update --yaml-file /path/to/batch.yaml
 ```
 
 ##### JSON
 ```shell
-ncedit  batch --json-file /path/to/batch.json
+ncedit batch --smart-update --json-file /path/to/batch.json
 ```
 
 ## Making per-item changes
 If you don't want to go to the hassle of creating a batch file or you have small, dynamic edits you wish to perform, your able to perform individual edits on the command line:
 
+
+### Practical examples
+One of the main uses of this tool is to configure Puppet Code Manager, so here are the exact commands you need (alternatively make a `batch.yaml` file to do it all in one hit):
+
+#### Code Manager R10K checkout URL
+```shell
+ncedit classes --smart-update --group-name 'PE Master' \
+    --class-name puppet_enterprise::profile::master \
+    --param-name r10k_remote \
+    --param-value https://github.com/puppetlabs/control-repo
+```
+
+#### Code Manager SSH key
+```shell
+ncedit classes --smart-update --group-name 'PE Master' \
+    --class-name puppet_enterprise::profile::master \
+    --param-name r10k_ \
+    --param-value /etc/puppetlabs/puppetserver/ssh/id_rsa
+```
+
+#### Code Manager proxy server
+```shell
+ncedit classes --smart-update --group-name 'PE Master' \
+    --class-name puppet_enterprise::profile::master \
+    --param-name r10k_proxy \
+    --param-value http://proxy.megacorp.com:3128
+```
+
+
+#### Enable Code Manager
+```shell
+ncedit classes --smart-update --group-name 'PE Master' \
+    --class-name puppet_enterprise::profile::master \
+    --param-name code_manager_auto_configure \
+    --param-value true
+    
+ncedit classes --smart-update --group-name 'PE Master' \
+    --class-name puppet_enterprise::profile::master \
+    --param-name file_sync_enabled \
+    --param-value automatic
+```
+
+
+#### Add a rule to configure the puppet master with extra classes
+```shell
+ncedit classes --smart-update --group-name "Puppet Masters" \
+    --class-name role::my_puppet_master \
+    --rule '["and", ["=",["fact","fqdn"],"'$(facter fqdn)'"]]' \
+    --rule-mode replace
+```
+
+* Run puppet and then re-deploy puppet code
+* Create a new group called `Puppet Masters`
+* Make it load the role `role::my_puppet_master`
+* Have it match any host matching the FQDN of the host the command was run from and replace any existing rules for the group
+
+#### Refresh the list of classes in the console
+```shell
+ncedit update_classes
+```
+
 ### Add a class to group
 ```shell
-ncedit --group-name FOO --class-name BAR
+ncedit classes --group-name FOO --class-name BAR
 ```
 * Group will be created if it doesn't exist
 * Class MUST exist AND puppet must be aware of it (defeat caching) for the request to be accepted
+* Pass `--smart-update` to run puppet and re-deploy classes if desired
 
 ### Add a parameter to a class for a group
 ```shell
-ncedit --group-name FOO --class-name BAR --param-name BAZ --param-value BAS
+ncedit classes --group-name FOO --class-name BAR --param-name BAZ --param-value BAS
 ```
 * Group will be created if it doesn't exist
 * Class will be added if it isn't already
 * Class MUST exist AND puppet must be aware of it (defeat caching) AND the parameter must exist on the class for the request to be accepted
+* Pass `--smart-update` to run puppet and re-deploy classes if desired
 
 ### Delete a parameter from a class inside a group
 ```shell
-ncedit --group-name FOO --class-name BAR --param-name BAZ --delete-param
+ncedit classes --group-name FOO --class-name BAR --param-name BAZ --delete-param
 ```
 * Group will be created if it doesn't exist
 * Class will be added if it isn't already
@@ -137,14 +201,14 @@ ncedit --group-name FOO --class-name BAR --param-name BAZ --delete-param
 
 ### Delete a class inside a group
 ```shell
-ncedit --group-name FOO --class-name BAR --delete-class
+ncedit classes --group-name FOO --class-name BAR --delete-class
 ```
 * Group will be created if it doesn't exist
 * Ensures the class `BAR` is not defined
 
 ### Replace all rules for group
 ```shell
-ncedit --group-name FOO --rule 'JSON_FRAGMENT' --rule-mode replace
+ncedit classes --group-name FOO --rule 'JSON_FRAGMENT' --rule-mode replace
 ```
 * Group will be created if it doesn't exist
 * Completely replaces existing rules for this group
@@ -152,13 +216,22 @@ ncedit --group-name FOO --rule 'JSON_FRAGMENT' --rule-mode replace
 
 ### Append a rule to group
 ```shell
-ncedit --group-name FOO --rule 'JSON_FRAGMENT' --rule-mode append
+ncedit classes --group-name FOO --rule 'JSON_FRAGMENT' --rule-mode append
 ```
 * Group will be created if it doesn't exist
 * Appends supplied JSON_FRAGMENT to the group's rules
 * Example JSON_FRAGMENT: `["and",["=",["fact","osfamily"],"RedHat"]]`
 * Notice in our example JSON_FRAGMENT that we have *kept* the outer `and` rule.  If we supply a conjuctive different to the rule's current value we will change it for the whole rule
 
+## Smart updates
+The NC API will only accept rules for classes that currently exist on the system, so its likely that your rules will be rejected if they have not yet been deployed to the master.  To make this process as seamless as possible, the `--smart-update` option will perform this task for you as-and-when required.  This assumes you have already setup RBAC to grant access to Code Manager, this can be done in a single command using the [pe_rbac gem](https://github.com/declarativesystems/pe_rbac#setting-up-code-manager-on-the-command-line)
+
+Smart updates work as follows:
+
+* If any of the `r10k_*` or `code_manager_auto_configure` parameters are set, update them immediately
+* Run `puppet agent -t` then `puppet-code deploy --all --wait`
+* Process any remaining directives
+
 ## Troubleshooting
 * If you cannot install the `ncedit` gem and you are behind a corporate proxy, ensure that you have correctly set your `http_proxy` and `https_proxy` variables on the shell before running `gem install`.
 * Some corporate proxies will attempt to eavesdrop on all SSL connections which will cause downloads to fail.  This can be resolved by installing a CA bundle (be sure you understand the implications of doing so) or domain whitelisting on the proxy itself
@@ -180,4 +253,4 @@ bundle exec rake spec
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/GeoffWilliams/ncedit.
+Bug reports and pull requests are welcome on GitHub at https://github.com/declarativesystems/ncedit.

data/exe/ncedit

@@ -6,6 +6,11 @@ require 'ncedit/cmd'
 # display help if nothing specified
 ARGV.push('-h') if ARGV.empty?
 
+# Add the path to the puppet-code command as a fallback.  It's last to
+# allow user to override via the real PATH if necessary
+ENV['PATH'] = "#{ENV['PATH']}:/opt/puppetlabs/puppet/bin/:/opt/puppetlabs/client-tools/bin/"
+
+
 Escort::App.create do |app|
   app.version NCEdit::VERSION
   app.summary "ncedit"
@@ -59,7 +64,11 @@ Escort::App.create do |app|
         :long    => '--rule-mode',
         :type    => :string,
       )
-
+      opts.opt(:smart_update,
+        'Smart update',
+        :long => '--smart-update',
+        :type => :boolean,
+      )
     end
     command.action do |options, arguments|
       NCEdit::Cmd::classes(options[:global][:commands][:classes][:options])
@@ -80,12 +89,18 @@ Escort::App.create do |app|
         :long => '--json-file',
         :type => :string,
       )
+      opts.opt(:smart_update,
+        'Smart update',
+        :long => '--smart-update',
+        :type => :boolean,
+      )
     end
     command.action do |options, arguments|
       yaml_file = options[:global][:commands][:batch][:options][:yaml_file]
       json_file = options[:global][:commands][:batch][:options][:json_file]
+      smart_update = options[:global][:commands][:batch][:options][:smart_update]
 
-      NCEdit::Cmd::batch(yaml_file: yaml_file, json_file: json_file)
+      NCEdit::Cmd::batch(yaml_file: yaml_file, json_file: json_file, smart_update: smart_update)
     end
   end
 

data/lib/ncedit/cmd.rb

@@ -8,6 +8,15 @@ module NCEdit
   module Cmd
     DEFAULT_RULE = "or"
 
+    R10K_SETTINGS_CLASS = "puppet_enterprise::profile::master"
+    R10K_SETTINGS_PARAMS = [
+        "code_manager_auto_configure",
+        "r10k_remote",
+        "r10k_private_key",
+        "r10k_proxy",
+        "r10k_postrun",
+    ]
+
     def self.init(puppetclassify = nil)
       if puppetclassify
         # use passed in puppetclassify if present - allows injection for easy
@@ -213,14 +222,23 @@ module NCEdit
     #         'badparam'
     #
     # 'Puppet Masters':
-    #   'clases':
+    #   'classes':
     #     'role::puppet::master':
     #   'append_rules':
     #     - - "="
     #       - "name"
     #       - "vmpump02.puppet.com"
-    def self.batch(yaml_file: nil, json_file: nil)
+    def self.batch(yaml_file: nil, json_file: nil, smart_update: false)
       data = read_batch_data(yaml_file: yaml_file, json_file: json_file)
+
+      if smart_update
+        needs_reclassify = contains_r10k_settings(data)
+        if needs_reclassify
+          apply_r10k_settings_now(needs_reclassify)
+        end
+        puppet_code_deploy
+      end
+
       data.each { |group_name, data|
 
         Escort::Logger.output.puts "Processing #{group_name}"
@@ -342,7 +360,7 @@ module NCEdit
       updated
     end
 
-    # Ensure a partualar rule exists in the group["rule"] array
+    # Ensure a particular rule exists in the group["rule"] array
     # This affects only the items in the chain, eg:
     # [
     #  "or",
@@ -424,7 +442,6 @@ module NCEdit
       updated
     end
 
-
     def self.classes(options)
       group_name    = options[:group_name]
       class_name    = options[:class_name]
@@ -434,6 +451,7 @@ module NCEdit
       delete_param  = options[:delete_param]
       rule          = options[:rule]
       rule_mode     = options[:rule_mode]
+      smart_update  = options[:smart_update]
 
       rule_change   = false
       class_change  = false
@@ -460,10 +478,21 @@ module NCEdit
         class_change |= ensure_param(group, class_name, param_name, nil, delete:true)
       elsif class_name and param_name and param_value
         # set a value inside a class
+        if smart_update
+          if ! is_r10k_param(class_name, param_name)
+
+            # not an R10K parameter, do an immediate update to make sure any classes
+            # we need are in-place
+            puppet_code_deploy
+          end
+        end
         Escort::Logger.output.puts "Setting parameter #{param_name} to #{param_value} on #{class_name} in #{group_name}"
         class_change = ensure_class(group, class_name)
         class_change |= ensure_param(group, class_name, param_name, param_value)
       elsif class_name
+        if smart_update
+          puppet_code_deploy
+        end
         Escort::Logger.output.puts "Adding #{class_name} to #{group_name}"
         class_change = ensure_class(group, class_name)
       end
@@ -501,5 +530,69 @@ module NCEdit
       @puppet_https.delete("#{@puppet_url}/puppet-admin-api/v1/environment-cache")
       @puppet_https.post("#{@rest_api_url}/v1/update-classes")
     end
+
+    def self.puppet_code_deploy
+      Escort::Logger.output.puts "Running puppet and deploying code..."
+
+      # if puppet run fails, just deploy the code anyway as the fix needed might
+      # be in the update...
+      system("puppet agent -t ; puppet-code deploy --all --wait")
+    end
+
+    # Extract the r10k settings ONLY from the passed in data has and update
+    # NCAPI with them immediately
+    def self.apply_r10k_settings_now(data)
+      data.each { |group_name,opts|
+        group = nc_group(group_name)
+
+        if opts.key?("classes")
+          Escort::Logger.output.puts "Setting up rules for #{group_name} immediately"
+          if ensure_classes_and_params(group, opts["classes"])
+            update_group(group_name, classes: group["classes"])
+          end
+        end
+      }
+    end
+
+    # Check if this is an r10k parameter or not
+    #
+    # @return true if this is to do with R10K otherwise false
+    def self.is_r10k_param(class_name, param_name)
+      class_name == R10K_SETTINGS_CLASS and R10K_SETTINGS_PARAMS.include?(param_name)
+    end
+
+    # Evaluate whether change instructions contain R10K settings or not
+    #
+    # @param data Hash of data settings to look at
+    # @return false if not found otherwise hash of with just the settings that
+    #   need to be applied immediately
+    def self.contains_r10k_settings(data)
+      found = {}
+      data.each { |group_name, opts|
+        if opts.key?("classes") and opts["classes"].key?(R10K_SETTINGS_CLASS)
+          opts["classes"][R10K_SETTINGS_CLASS].each { |param_name,param_value|
+            if is_r10k_param(R10K_SETTINGS_CLASS, param_name)
+              # make the hash structure we need
+              if ! found.key?(group_name)
+                found[group_name] = {}
+              end
+
+              if ! found[group_name].key?("classes")
+                found[group_name]["classes"] = {}
+              end
+
+              if ! found[group_name]["classes"].key?(R10K_SETTINGS_CLASS)
+                found[group_name]["classes"][R10K_SETTINGS_CLASS] = {}
+              end
+
+              found[group_name]["classes"][R10K_SETTINGS_CLASS][param_name] = param_value
+            end
+          }
+        end
+      }
+
+      # Return the found elements if there were any, otherwise simplify to false
+      ! found.empty? ? found : false
+    end
   end
 end

data/lib/ncedit/version.rb

@@ -1,3 +1,3 @@
 module NCEdit
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

data/ncedit.gemspec

@@ -6,12 +6,13 @@ require 'ncedit/version'
 Gem::Specification.new do |spec|
   spec.name          = "ncedit"
   spec.version       = NCEdit::VERSION
-  spec.authors       = ["Geoff Williams"]
-  spec.email         = ["geoff@geoffwilliams.me.uk"]
+  spec.authors       = ["Declarative Systems"]
+  spec.email         = ["sales@declarativesystems.com"]
+  spec.license       = "Apache-2.0"
 
   spec.summary       = %q{Edit Puppet Enterprise Node Classifier rules}
   spec.description   = %q{Use the puppet-classify gem to create/edit NC rules}
-  spec.homepage      = "https://github.com/GeoffWilliams/ncedit"
+  spec.homepage      = "https://github.com/declarativesystems/ncedit"
 
 
   spec.files         = `git ls-files -z`.split("\x0").reject do |f|
@@ -26,6 +27,6 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "rspec", "~> 3.0"
 
   spec.add_runtime_dependency "escort", "0.4.0"
-  spec.add_runtime_dependency "json", "2.0.3"
-  spec.add_runtime_dependency "puppetclassify", "0.1.5"
+  spec.add_runtime_dependency "json_pure", "2.1.0"
+  spec.add_runtime_dependency "puppetclassify", "0.1.7"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ncedit
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
-- Geoff Williams
+- Declarative Systems
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-04-20 00:00:00.000000000 Z
+date: 2017-09-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -67,36 +67,36 @@ dependencies:
       - !ruby/object:Gem::Version
         version: 0.4.0
 - !ruby/object:Gem::Dependency
-  name: json
+  name: json_pure
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 2.0.3
+        version: 2.1.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 2.0.3
+        version: 2.1.0
 - !ruby/object:Gem::Dependency
   name: puppetclassify
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.1.5
+        version: 0.1.7
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.1.5
+        version: 0.1.7
 description: Use the puppet-classify gem to create/edit NC rules
 email:
-- geoff@geoffwilliams.me.uk
+- sales@declarativesystems.com
 executables:
 - ncedit
 extensions: []
@@ -118,8 +118,9 @@ files:
 - lib/ncedit/cmd.rb
 - lib/ncedit/version.rb
 - ncedit.gemspec
-homepage: https://github.com/GeoffWilliams/ncedit
-licenses: []
+homepage: https://github.com/declarativesystems/ncedit
+licenses:
+- Apache-2.0
 metadata: {}
 post_install_message: 
 rdoc_options: []