hieracles 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 6f625244beacbc27c040813044b27538be1cc238
-  data.tar.gz: 43ee5e5416918b571def1e2ff14283c51f0b5d3f
+  metadata.gz: c52c2ee117ba60358faf78e464d6530786ebeb79
+  data.tar.gz: 9a1c3bd0fc147a75d1d0e399d75acd84c480dfa5
 SHA512:
-  metadata.gz: 52464e962c93a456f3911e9800991293a5250d3e8a9c39570d26792e46ca7def773f6eac6f486d6fb0a2b27cc947cfcc914bd1ebf8734c9d5287dd7f650e978c
-  data.tar.gz: 0502cf9ed9ee1c641a521990d94b5af61fa775307621b659679b663629c6c4e5b4dbed6c50418cd65717f7bd38ac03fea9a24363fdd9cc348272d641b8694acd
+  metadata.gz: f1039911f488752aeb1324c5a24c1ba0679fd1c64705766532d0fe364c6a7de311cf3de4cce0e069f08bde3c75ee553d5c26684ad4c0ea2d5ba72f7a1c5a10fa
+  data.tar.gz: 91b8db1bbc0dfa06899a21542c6e172facff41a4a5fbf157870d9c6981afe17c90993d706be53129bdde5e5bb3db17179bfc7ac9a56cf87d52904e579d7cb3db

data/CHANGELOG.md

@@ -1,6 +1,16 @@
 Hieracles Changelog
 =======================
 
+### 0.2.1 - 2015-12-11
+- add configuration variables for connection to PuppetDB
+- make PuppetDB calls fill up the facts for the node queried
+- add a `facts` command to display the facts, provided locally 
+  or from PuppetDB directly
+- make a new CLI tool named ppdb for direct calls to PuppetDB
+- add optional filtering on ppdb commands
+- add a man page for ppdb
+- add zsh completion for ppdb
+
 ### 0.2.0 - 2015-11-24
 - fix merging for unsorted arrays
 - translate all paths to absolute paths,

data/Gemfile

@@ -1,4 +1,6 @@
 source 'https://rubygems.org'
 
+gem 'awesome_print'
+
 # Specify your gem's dependencies in hieracles.gemspec
 gemspec

data/README.md

@@ -4,8 +4,8 @@ Hieracles
 [![Gem Version](https://img.shields.io/gem/v/hieracles.svg)](http://rubygems.org/gems/hieracles)
 [![Downloads](http://img.shields.io/gem/dt/hieracles.svg)](https://rubygems.org/gems/hieracles)
 [![Build Status](https://img.shields.io/travis/Gandi/hieracles.svg)](https://travis-ci.org/Gandi/hieracles)
-[![Test Coverage](https://img.shields.io/codeclimate/coverage//github/Gandi/hieracles.svg)](https://codeclimate.com/github/Gandi/hieracles/coverage)
-[![Dependency Status](https://img.shields.io/gemnasium/Gandi/hieracles.svg)](https://gemnasium.com/Gandi/hieracles)
+[![Coverage Status](https://img.shields.io/coveralls/Gandi/hieracles.svg)](https://coveralls.io/github/Gandi/hieracles?branch=master)
+[![Dependency Status](https://gemnasium.com/Gandi/hieracles.svg)](https://gemnasium.com/Gandi/hieracles)
 [![Code Climate](https://img.shields.io/codeclimate/github/Gandi/hieracles.svg)](https://codeclimate.com/github/Gandi/hieracles)
 
 Hieracles is a command-line tool for analysis and deep examination of [Hiera][hiera] parameters in a [Puppet][puppet] setup. It can be used to quickly visualize, from a local puppet (typically on a developers environment), all the Hiera params related to a specific node.
@@ -55,7 +55,7 @@ Configuration variables are:
 - encpath  
   where to read information about each nodes
 - hierafile  
-  where to find a hierafile customised for your local puppet installation
+  where to find a hierafile customized for your local puppet installation
 - format  
   can be plain, console, csv, yaml, rawyaml, json
 - defaultscope  
@@ -73,6 +73,14 @@ Usage
     Available commands:
       info        provides the farm, datacenter, country
                   associated to the given fqdn
+                  An extra param can be added for filtering
+                  eg. hc <fqdn> info timestamp
+                  eg. hc <fqdn> info farm
+      facts       lists facts, either provided as a fact file
+                  or grabbed from puppetdb.
+                  An extra param can be added for filtering
+                  eg. hc <fqdn> facts architecture
+                  eg. hc <fqdn> facts 'memory.*mb'
       files       list all files containing params affecting this fqdn
                   (in more than commons)
       paths       list all file paths for files with params
@@ -97,13 +105,13 @@ Usage
       -y <fact_file> - facts in yaml format
       -j <fact_file> - facts in json format
       -i - interactive mode
+      -db - query puppetdb
+      -nodb - do not query puppetdb
 
 
 About facts aka. scope
 ------------------------
 
-_(work in progress)_
-
 Like with Hiera CLI you can use hieracles with defined top-scope variables. Those top-scope vars can be defined with:
 
 - `-p extraparam=what;anotherparam=this`
@@ -131,24 +139,62 @@ Note that if the scope var is not defined or if the file declared in hiera confi
 An option, `-i` enables the `interactive mode` in which you are prompted to fill up for undefined scope variables encountered in the hiera config file. This behavior can be made systematic by enabling `interactive: true` in hieracles configuration file.
 
 
+Optionnaly connecting to a puppetDB
+--------------------------------------
+
+#### Configuration
+
+When adding to the configuration file:
+
+    usedb: false
+    puppetdb:
+      usessl: false
+      host: puppetdb.example.com
+      port: 8080
+
+or for a ssl setup:
+
+    usedb: false
+    puppetdb:
+      usessl: true
+      host: puppetdb.example.com
+      port: 8081
+      key: path/to/key
+      key_password: somepassword
+      cert: path/to/cert
+      ca_file: path/to/ca_file
+      verify_peer: false
+
+#### Usage
+
+If you set `usedb: false` the `hc` commands will not query the puppetdb unless you pass the `-db` options.
+
+If you set `usedb: true` the `hc` command will query the puppetdb by default and display extra informations for the queried node. This default behavior can be changed by passing the `-nodb` option on the commandline.
+
+#### Impact
+
+When usedb is true, a call to puppetdb will be made for all commands to retrieve facts if they are present for the queried node.
+
+
 Completion
 -------------
 There is a simple zsh completion file in `tools/completion`. 
 
 If you use [oh-my-zsh][omz] put it in `~/.oh-my-zsh/completions`
 
-    wget -O ~/.oh-my-zsh/completions/_hc https://raw.githubusercontent.com/Gandi/hieracles/master/tools/completion/_hc
+    wget -O ~/.oh-my-zsh/completions/_hc https://raw.githubusercontent.com/Gandi/hieracles/master/tools/completion/zsh/_hc
     echo 'compdef _hc hc "bundle exec hc"' >> ~/.zshrc
     echo 'autoload -U _hc' >> ~/.zshrc
     
 Otherwise 
 
     mkdir ~/.zsh-completions
-    wget -O ~/.zsh-completions/_hc https://raw.githubusercontent.com/Gandi/hieracles/master/tools/completion/_hc
+    wget -O ~/.zsh-completions/_hc https://raw.githubusercontent.com/Gandi/hieracles/master/tools/completion/zsh/_hc
     echo 'fpath=(~/.zsh-completions $fpath)' >> ~/.zshrc
     echo 'compdef _hc hc "bundle exec hc"' >> ~/.zshrc
     echo 'autoload -U _hc' >> ~/.zshrc
 
+Note: `ppdb` also has a completion file https://raw.githubusercontent.com/Gandi/hieracles/master/tools/completion/zsh/_ppdb
 
 Debian packaging
 --------------------
@@ -195,7 +241,7 @@ Todo
 - detect unused params
 - create a repl, which at launch reads all data so the queries are blazing fast
 - adapt to other ENCs
-- adapt to puppetdb storage
+- adapt to PuppetDB storage
 
 
 Authors

data/bin/hc

@@ -3,8 +3,9 @@
 $LOAD_PATH << File.expand_path("../../lib", __FILE__)
 
 require 'hieracles'
+require 'hieracles/options/hc'
 
-opt = Hieracles::Optparse.new(ARGV)
+opt = Hieracles::Options::Hc.new(ARGV)
 
 if opt.options.has_key? :version
   puts Hieracles.version
@@ -16,22 +17,27 @@ command = opt.payload[1]
 args = opt.payload[2..-1]
 
 unless fqdn && command
-  puts Hieracles::Help.usage
+  puts Hieracles::Options::Hc.usage
   exit(1)
 end
  
 if Hieracles::Format.method_defined? command
-  node = Hieracles::Node.new fqdn, opt.options
+  begin
+    node = Hieracles::Node.new fqdn, opt.options
+  rescue Exception => e
+    puts "*** Error: #{e.message}"
+    exit(1)
+  end
   begin
     formatter = Object.const_get("Hieracles::Formats::#{Hieracles::Config.format}")
   rescue
-    puts "  Unknown format #{Hieracles::Config.format}"
+    puts "*** Unknown format #{Hieracles::Config.format}"
     exit(1)
   end
   dispatch = formatter.new node
   puts dispatch.send(command.to_sym, args)
 else
-  puts "  Unknown command: #{command}"
-  Hieracles::Help.usage
+  puts "*** Unknown command: #{command}"
+  Hieracles::Options::Hc.usage
   exit(1)
 end

data/bin/ppdb

@@ -0,0 +1,42 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH << File.expand_path("../../lib", __FILE__)
+
+require 'hieracles'
+require 'hieracles/options/ppdb'
+
+opt = Hieracles::Options::Ppdb.new(ARGV)
+
+if opt.options.has_key? :version
+  puts Hieracles.version
+  exit(0)
+end
+
+unless opt.payload[1]
+  puts Hieracles::Options::Ppdb.usage
+  exit(1)
+end
+
+command = "#{opt.payload[0]}_#{opt.payload[1]}"
+args = opt.payload[2..-1] || []
+
+Hieracles::Config.load opt.options
+
+if Hieracles::Puppetdb::Request.method_defined? command
+  request = Hieracles::Puppetdb::Request.new Hieracles::Config.puppetdb
+  meth = request.method(command.to_sym)
+  required = request.method(command.to_sym).parameters.select { |arg| arg[0] == :req }.map { |arg| arg[1] }
+  if args.length < required.length
+    args = required.join(', ')
+    puts "*** The #{command} method requires arguments: #{args}"
+    exit(1)
+  else
+    format = Hieracles::Formats::Console.new(nil)
+    resp = meth.call(*args)
+    puts format.build_list(resp.data, resp.notifications, [])
+  end
+else
+  puts "*** Unknown command: #{command}"
+  puts Hieracles::Options::Ppdb.usage
+  exit(1)
+end

data/hc.1

@@ -1,7 +1,7 @@
-.TH hc 1 "2015-11-08" "version 0.1.5" "Hieracles command manual"
+.TH hc 1 "2015-12-05" "version 0.2.1" "Hieracles command manual"
 
 .SH NAME
-hc \- Command tool for Hieracles
+hc \- Command line tool for Hieracles
 .SH SYNOPSIS
 .B hc 
 .I fqdn
@@ -23,24 +23,47 @@ time, a generic Hiera overlay visualization tool.
 .PP
 hc command takes at minimum 2 arguments:
 .RS 4
+
 .IP fqdn 10
 the fqdn under which the node is known in the ENC dir
+
 .IP action 
 the action needs to be one of the following
 .RS
+
 .IP info 10
 provides the farm, datacenter, country associated to the given fqdn.
+An extra filter can be added to limit the list.
+Use ruby regexp without the enclosing slashes.
+.RS
+eg. hc <fqdn> info timestamps
+.RE
+
+.IP facts 10
+lists facts for a given fqdn, from puppetdb or local facts (see configuration).
+An extra filter can be added to limit the list.
+use ruby regexp without the enclosing slashes.
+.RS
+eg. hc <fqdn> facts architecture
+.RE
+.RS
+eg. hc <fqdn> facts 'memory.*mb'
+.RE
+
 .IP files
 list all files containing params affecting this fqdn 
 (in more than commons).
+
 .IP paths
 list all file paths for files with params.
+
 .IP modules
 list modules included in the farm where the node is.
+
 .IP params
 list params for the node matching the fqdn.
-An extra filter string can be added to limit the list
-use ruby regexp without the enclosing slashes
+An extra filter can be added to limit the list.
+Use ruby regexp without the enclosing slashes.
 .RS
 eg. hc <fqdn> params postfix.*version
 .RE
@@ -50,7 +73,8 @@ eg. hc <fqdn> params '^postfix'
 .RS
 eg. hc <fqdn> params 'version$'
 .RE
-.IP all
+
+.IP allparams
 same as params but including the common.yaml params (huge)
 Also accepts a search string
 .RE
@@ -122,7 +146,6 @@ the basepath is prepended to the paths found in the configfile and the hierafile
 .B \-\-encdir \fRpath
 encdir is where we can find the ENC definitions. It's stated in the configfile but can be overridden by the -e option.
 
-
 .TP
 .PD 0
 .B \-y \fRfile
@@ -136,7 +159,6 @@ and test locally what would be the consequences of switching from
 one environment to another. It's also good for anticipation
 of how new node, that don't have facts yet, will behave.
 
-
 .TP
 .PD 0
 .B \-j \fRfile
@@ -150,7 +172,6 @@ flag but in json format, which can be obtained on the node with
 and this behave the same way. Note that the \-y has precedence on
 the \-j in this context.
 
-
 .TP
 .PD 0
 .B \-i
@@ -166,6 +187,24 @@ in which you are prompted to fill up for undefined scope variables encountered i
 in hieracles configuration file.
 .RE
 
+.TP
+.PD 0
+.B \-db
+if
+.I "usedb: false"
+is specified in the hieracles configuration file, it cancels the calls to puppetdb 
+and only display information according to local data.
+
+
+.TP
+.PD 0
+.B \-nodb
+if
+.I "usedb: true"
+is specified in the hieracles configuration file, it forces the calls to puppetdb 
+for gathering facts and node information.
+
+
 .SH FILES
 .I ~/.config/hieracles/config.yaml
 .RS
@@ -178,6 +217,9 @@ A typical config file would look like:
 ---
 .RE
 .RS
+basepath: /home/mose/git/puppet
+.RE
+.RS
 classpath: farm_modules/%s/manifests/init.pp
 .RE
 .RS

data/lib/hieracles.rb

@@ -1,9 +1,8 @@
+require 'hieracles/notification'
 require 'hieracles/interpolate'
-require 'hieracles/optparse'
 require 'hieracles/config'
 require 'hieracles/hiera'
 require 'hieracles/utils'
-require 'hieracles/help'
 require 'hieracles/node'
 require 'hieracles/format'
 require 'hieracles/registry'
@@ -13,6 +12,7 @@ require 'hieracles/formats/yaml'
 require 'hieracles/formats/plain'
 require 'hieracles/formats/console'
 require 'hieracles/formats/rawyaml'
+require 'hieracles/puppetdb'
 
 # https://github.com/Gandi/hieracles
 module Hieracles

data/lib/hieracles/config.rb

@@ -9,37 +9,47 @@ module Hieracles
     include Hieracles::Utils
     extend self
 
-    attr_reader :extraparams, :server, :classpath, :scope,
+    attr_reader :extraparams, :server, :classpath, :scope, :puppetdb, :usedb,
       :modulepath, :hierafile, :basepath, :encpath, :format, :interactive
 
     def load(options)
-      @optionfile = options[:config] || defaultconfig
+      @options = options
+      @optionfile = @options[:config] || defaultconfig
       @extraparams = extract_params(options[:params])
-      values = get_config(@optionfile)
-      @server = values['server']
-      @basepath = File.expand_path(options[:basepath] || values['basepath'] || values['localpath'] || '.')
-      @classpath = build_path(values['classpath'])
-      @modulepath = resolve_path(values['modulepath'] || 'modules')
-      @encpath = resolve_path(options[:encpath] || values['encpath'] || 'enc')
-      @hierafile = resolve_path(options[:hierafile] || values['hierafile'] || 'hiera.yaml')
-      @format = (options[:format] || values['format'] || 'console').capitalize
-      facts_file = options[:yaml_facts] || options[:json_facts]
-      facts_format = options[:json_facts] ? :json : :yaml
-      @scope = sym_keys((facts_file && load_facts(facts_file, facts_format)) || values['defaultscope'] || {})
-      @interactive = options[:interactive] || values['interactive']
+      @values = get_config(@optionfile)
+      @server = @values['server']
+      @usedb = if @options[:db]
+        true
+      elsif @options[:nodb]
+        false
+      else
+        @values['usedb']
+      end
+      @puppetdb = @values['puppetdb']
+      @values['basepath'] ||= @values['localpath']
+      @basepath = File.expand_path(pick_first(:basepath, '.'))
+      @classpath = build_path(@values['classpath'])
+      @modulepath = resolve_path(pick_first(:modulepath, 'modules'))
+      @encpath = resolve_path(pick_first(:encpath, 'enc'))
+      @hierafile = resolve_path(pick_first(:hierafile, 'hiera.yaml'))
+      @format = pick_first(:format, 'console').capitalize
+      @scope = load_scope(@values)
+      @interactive = pick_first(:interactive, false)
+    end
+
+    def pick_first(label, default)
+      @options[label] || @values[label.to_s] || default
     end
 
     def get_config(file)
       initconfig(file) unless File.exist? file
-      values = YAML.load_file(file)
+      YAML.load_file(file)
     end
 
     def initconfig(file)
       FileUtils.mkdir_p(File.dirname(file))
       File.open(file, 'w') do |f|
         f.puts '---'
-        f.puts '# uncomment if you use the CGI method for discovery'
-        f.puts '# server: puppetserver.example.com'
         f.puts 'classpath: manifests/classes/%s.pp'
         f.puts 'modulepath: modules'
         f.puts 'encpath: enc'
@@ -60,8 +70,10 @@ module Hieracles
       end
     end
 
-    def path(what)
-      send(what.to_sym)
+    def load_scope(values)
+      facts_file = @options[:yaml_facts] || @options[:json_facts]
+      facts_format = @options[:json_facts] ? :json : :yaml
+      sym_keys((facts_file && load_facts(facts_file, facts_format)) || values['defaultscope'] || {})
     end
 
     def load_facts(file, format)