hiera-eyaml 1.1.4 → 1.3.1

data/.gitignore

@@ -3,3 +3,5 @@
 *.gradle
 keys/*.pem
 pkg/
+tmp/
+.DS_Store

data/Gemfile

@@ -3,3 +3,10 @@ source 'https://rubygems.org/'
 gem 'highline'
 gem 'trollop'
 
+group :development do
+  gem "aruba"
+  gem "hiera-eyaml-plaintext"
+  gem "puppet"
+end
+
+

data/Gemfile.lock

@@ -1,12 +1,40 @@
 GEM
   remote: https://rubygems.org/
   specs:
+    aruba (0.5.3)
+      childprocess (>= 0.3.6)
+      cucumber (>= 1.1.1)
+      rspec-expectations (>= 2.7.0)
+    builder (3.2.2)
+    childprocess (0.3.9)
+      ffi (~> 1.0, >= 1.0.11)
+    cucumber (1.3.5)
+      builder (>= 2.1.2)
+      diff-lcs (>= 1.1.3)
+      gherkin (~> 2.12.0)
+      multi_json (~> 1.7.5)
+      multi_test (>= 0.0.2)
+    diff-lcs (1.2.4)
+    ffi (1.9.0)
+    gherkin (2.12.0)
+      multi_json (~> 1.3)
+    hiera-eyaml (1.2.0)
+      highline (>= 1.6.19)
+      trollop (>= 2.0)
+    hiera-eyaml-plaintext (0.1)
+      hiera-eyaml (>= 1.2.0)
     highline (1.6.19)
+    multi_json (1.7.8)
+    multi_test (0.0.2)
+    rspec-expectations (2.14.0)
+      diff-lcs (>= 1.1.3, < 2.0)
     trollop (2.0)
 
 PLATFORMS
   ruby
 
 DEPENDENCIES
+  aruba
+  hiera-eyaml-plaintext
   highline
   trollop

data/PLUGINS.md

@@ -0,0 +1,4 @@
+PLUGINS
+=======
+
+Take a look at the skeleton project hiera-eyaml-plaintext, for a bare-bones demo plugin that you can copy and make into your own encryption plugin for hiera-eyaml.

data/README.md

@@ -6,16 +6,17 @@ within yaml type files to be used by Puppet.
 
 More info can be found [in this corresponding post](http://themettlemonkey.wordpress.com/2013/07/15/hiera-eyaml-per-value-encrypted-backend-for-hiera-and-puppet/).
 
-The Hiera eYaml backend uses yaml formatted files with the .eyaml extension. Simply wrap your
-encrypted string with ENC[] and place it in an eyaml file. You can mix your plain values
-in as well or separate them into different files.
+The Hiera eYaml backend uses yaml formatted files with the .eyaml extension. Simply prefix your
+encrypted string with the encryption method (PKCS7,) wrap it with ENC[] and place it in an eyaml file. You can mix your plain values in as well or separate them into different files.
+
+Example:
 
 <pre>
 ---
 plain-property: You can see me
 
 encrypted-property: >
-    ENC[Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
+    ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
     NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh
     jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y
     l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd
@@ -23,10 +24,7 @@ encrypted-property: >
     IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==]
 </pre>
 
-eYaml also supports encrypted values within arrays, hashes, nested arrays and nested hashes 
-(see below for examples)
-
-N.B. when using the multi-line string syntax (i.e. >) **don't wrap encrypted strings with "" or ''**
+eYaml supports multiple encryption types, and encrypted values can occur within arrays, hashes, nested arrays and nested hashes 
 
 Setup
 =====
@@ -37,47 +35,58 @@ Setup
 
 ### Generate keys
 
-The first step is to create a pair of keys on the Puppet master
+The first step is to create a pair of keys:
 
     $ eyaml -c
 
-This creates a public and private key with default names in the default location. (/etc/hiera/keys directory)
+This creates a public and private key with default names in the default location. (./keys)
 
 ### Encryption
 
     To encrypt something, you only need the public_key, so distribute that to people creating hiera properties
 
-    $ eyaml -e filename               # Encrypt a file
-    $ eyaml -e -s text                # Encrypt some text
+    $ eyaml -e -f filename            # Encrypt a file
+    $ eyaml -e -s 'hello there'       # Encrypt a string
     $ eyaml -e -p                     # Encrypt a password (prompt for it)
 
-    You can also specify the name of the value and the output style (block or string) so that you can
-    redirect the output directly to a file:
+### Decryption
 
-    $ eyaml -e -n my-secret -s "Secret Text" -o block
+    To decrypt something, you need the public_key and the private_key.
 
-    returns:
+    To test decryption you can also use the eyaml tool if you have both keys
 
-    $ my-secret: >
-    $   ENC[...
-    $   .......
-    $   ......]
+    $ eyaml -d -f filename               # Decrypt a file
+    $ eyaml -d -s 'ENC[PKCS7,.....]'     # Decrypt a string
 
-### Decryption
+### EYaml files
 
-    To decrypt something, you need the public_key and the private_key on the puppet master.
+    Once you have created a few eyaml files, with a mixture of encrypted and non-encrypted properties, you can edit the encrypted values in place, using the special edit mode of the eyaml utility
 
-    To test decryption you can also use the eyaml tool if you have both keys
+    $ eyaml -i filename.eyaml         # Edit an eyaml file in place
 
-    $ eyaml -d filename                  # Decrypt a file (PEM format)
-    $ eyaml -d -s SOME-ENCRYPTED-TEXT    # Decrypt some text
+Multiple Encryption Types
+=========================
 
-Change the permissions so that the private key is only readable by the user that hiera (puppet) is
-running as.
+hiera-eyaml backend is pluggable, so that further encryption types can be added as separate gems to the general mechanism which hiera-eyaml uses. Hiera-eyaml ships with one default mechanism of 'pkcs7', the encryption type widely used to sign smime email messages. 
 
-### Configure Hiera
+Other encryption types (if the gems for them have been loaded) can be specified using the following formats:
 
-Next configure hiera.yaml to use the eyaml backend
+<pre>
+    ENC[PKCS7,SOME_ENCRYPTED_VALUE]         # a PKCS7 encrypted value
+    ENC[GPG,SOME_ENCRYPTED_VALUE]           # a GPG encrypted value (hiera-eyaml-gpg)
+    ... etc ...
+</pre>
+
+When editing eyaml files, you will see that the unencrypted plaintext is marked in such a way as to identify the encryption method. This is so that the eyaml tool knows to encrypt it back using the correct method afterwards:
+
+<pre>
+some_key: DEC::PKCS7[very secret password]!
+</pre>
+
+Hiera
+=====
+
+To use eyaml with hiera and puppet, first configure hiera.yaml to use the eyaml backend
 
 <pre>
 ---
@@ -94,24 +103,20 @@ Next configure hiera.yaml to use the eyaml backend
 :eyaml:
     :datadir: '/etc/puppet/hieradata'
 
-    # Optional. Default is /etc/hiera/keys/private_key.pem
-    :private_key: /new/path/to/key/private_key.pem
+    # If using the pkcs7 encryptor (default)
+    :pkcs7_private_key: /path/to/private_key_file.pem
+    :pkcs7_public_key:  /path/to/public_key_file.pem
 
-    # Optional. Default is /etc/hiera/keys/public_key.pem
-    :public_key:  /new/path/to/key/public_key.pem
 </pre>
 
-### YAML files
+Then, edit your hiera yaml files (renaming them with the .eyaml extension), and insert your encrypted values:
 
-  Once the value is encrypted, wrap it with ENC[] and place it in the .eyaml file.
-
-Usages:
 <pre>
 ---
 plain-property: You can see me
 
 cipher-property : >
-    ENC[Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
+    ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
     NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh
     jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y
     l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd
@@ -125,7 +130,7 @@ environments:
     production:
         host: prod.org.com
         password: >
-            ENC[Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
+            ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
             NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh
             jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y
             l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd
@@ -136,7 +141,7 @@ things:
     - thing 1
     -   - nested thing 1.0
         - >
-            ENC[Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
+            ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
             NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh
             jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y
             l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd
@@ -146,8 +151,13 @@ things:
         - nested thing 2.1
 </pre>
 
+Notes
+=====
+
+If you do not specify an encryption method within ENC[] tags, it will be assumed to be PKCS7
+
 Authors
 =======
 
 - [Tom Poulton](http://github.com/TomPoulton) - Initial author. eyaml backend.
-- [Geoff Meakin](http://github.com/gtmtech) - Major contributor. eyaml command.
+- [Geoff Meakin](http://github.com/gtmtech) - Major contributor. eyaml command.

data/bin/eyaml

@@ -1,361 +1,13 @@
 #!/usr/bin/env ruby
 
-require 'openssl'
-require 'base64'
-require 'trollop'
-require 'highline/import'
-require 'hiera/backend/version'
-require 'tempfile'
+require 'rubygems'
+require 'hiera/backend/eyaml/CLI'
+require 'hiera/backend/eyaml/plugins'
+require 'hiera/backend/eyaml/encryptors/pkcs7'
 
-ENCRYPTED_BLOCK = />\n( *)ENC\[([a-zA-Z0-9+\/ \n]+)\]/
-ENCRYPTED_STRING = /ENC\[([a-zA-Z0-9+\/]+)\]/
-DECRYPTED_BLOCK = />\n( *)ENC!\[(.+)\]!ENC/
-DECRYPTED_STRING = /ENC!\[(.+)\]!ENC/
+# Register all plugins
+Hiera::Backend::Eyaml::Encryptors::Pkcs7.register
+Hiera::Backend::Eyaml::Plugins.find
 
-def ensure_key_dir_exists(key_file)
-  key_dir = File.dirname(key_file)
-
-  if !File.directory?(key_dir)
-    Dir.mkdir(key_dir)
-    puts "Created #{key_dir} dir for #{key_file}."
-  end
-end
-
-def get_file_input(options)
-  if options[:file]
-    File.read options[:file]
-  else
-    STDIN.read
-  end
-end
-
-def get_input(options)
-  return options[:string] if options[:string]
-
-  if options[:password]
-    ask("Enter password: ") {|q| q.echo = "*" }
-  end
-
-  get_file_input(options)
-end
-
-def encrypt(public_key, plaintext)
-  cipher = OpenSSL::Cipher::AES.new(256, :CBC)
-  ciphertext_binary = OpenSSL::PKCS7::encrypt([public_key], plaintext, cipher, OpenSSL::PKCS7::BINARY).to_der
-  Base64.encode64(ciphertext_binary).strip
-end
-
-def decrypt(public_key, private_key, ciphertext)
-    ciphertext_decoded = Base64.decode64(ciphertext)
-    pkcs7 = OpenSSL::PKCS7.new( ciphertext_decoded )
-    pkcs7.decrypt(private_key, public_key)
-end
-
-def decrypt_eyaml(public_key, private_key, cipher_eyaml)
-  # decrypt blocks first
-  plain_block_eyaml = cipher_eyaml.gsub(ENCRYPTED_BLOCK) { |match|
-    indentation = $1
-    ciphertext = $2.gsub(/[ \n]/, '')
-    plaintext = decrypt(public_key, private_key, ciphertext)
-    ">\n"+indentation+"ENC![" + plaintext + "]!ENC"
-  }
-  # then decrypt strings
-  plain_block_eyaml.gsub(ENCRYPTED_STRING) { |match|
-    plaintext = decrypt(public_key, private_key, $1)
-    "ENC![" + plaintext + "]!ENC"
-  }
-end
-
-def encrypt_eyaml(public_key, plain_eyaml)
-  # encrypt blocks
-  cipher_block_eyaml = plain_eyaml.gsub(DECRYPTED_BLOCK) { |match|
-    indentation = $1
-    ciphertext = encrypt(public_key, $2).gsub(/\n/,"\n"+indentation)
-    ">\n" + indentation + "ENC[" + ciphertext + "]"
-  }
-  # encrypt strings
-  cipher_block_eyaml.gsub(DECRYPTED_STRING) { |match|
-    ciphertext = encrypt(public_key, $1).gsub(/\n/,'')
-    "ENC[" + ciphertext + "]"
-  }
-end
-
-def shred(file, clean_size)
-  [0xff, 0x55, 0xaa, 0x00].each do |byte|
-    file.seek(0, IO::SEEK_SET)
-    clean_size.times { file.print(byte.chr) }
-    file.fsync
-  end
-end
-
-options = Trollop::options do
-    
-  version "Hiera-eyaml version " + Hiera::Backend::Eyaml::VERSION.to_s
-  banner <<-EOS
-Hiera-eyaml is a backend for Hiera which provides OpenSSL encryption/decryption for Hiera properties
-
-Usage:
-  eyaml [options] [file-to-encrypt]
-  EOS
-  
-  opt :createkeys, "Create public and private keys for use encrypting properties", :short => 'c'
-  opt :encrypt, "Encrypt a string, password, file or stdin"
-  opt :decrypt, "Decrypt a string, file or stdin"
-  opt :eyaml, "Assume input is eyaml format"
-  opt :edit, "Edit an encrypted file inplace, implies --eyaml"
-  opt :password, "Encrypt a password entered on the terminal", :short => 'p'
-  opt :string, "Encrypt a string provided on the command line", :short => 's', :type => :string 
-  opt :private_key, "Filename of the private_key", :type => :string, :default => "/etc/hiera/keys/private_key.pem"
-  opt :public_key, "Filename of the public_key", :type => :string, :default => "/etc/hiera/keys/public_key.pem"
-  opt :output, "Output mode to use when encrypting (examples, block or string)", :short => 'o', :type => :string, :default => "examples"
-  opt :name, "Add the name for the eyaml key (example, myvar: )", :short => 'n', :type => :string
-end
-
-main_option_count = 0
-main_option_count += 1 if options[:createkeys]
-main_option_count += 1 if options[:encrypt]
-main_option_count += 1 if options[:decrypt]
-main_option_count += 1 if options[:edit]
-
-Trollop::die "You can only specify one main action" if main_option_count > 1
-Trollop::die "You cannot specify --password and --string" if options[:password] and options[:string]
-
-options[:file] = ARGV[0]
-if options[:file] and (options[:password] or options[:string])
-  $stderr.puts "WARN: file supplied but will be shadowed by string or password"
-end
-
-if options[:createkeys]
-
-  # Try to do equivalent of:
-  # openssl req -x509 -nodes -days 100000 -newkey rsa:2048 -keyout privatekey.pem -out publickey.pem -subj '/'
-
-  ensure_key_dir_exists(options[:private_key])
-  ensure_key_dir_exists(options[:public_key])
-
-  key = OpenSSL::PKey::RSA.new(2048)
-  open( options[:private_key], "w" ) do |io|
-    io.write(key.to_pem)
-  end
-
-  $stderr.puts "#{options[:private_key]} created."
-
-  name = OpenSSL::X509::Name.parse("/")
-  cert = OpenSSL::X509::Certificate.new()
-  cert.serial = 0
-  cert.version = 2
-  cert.not_before = Time.now
-  cert.not_after = Time.now + 50 * 365 * 24 * 60 * 60
-  cert.public_key = key.public_key
-
-  ef = OpenSSL::X509::ExtensionFactory.new
-  ef.subject_certificate = cert
-  ef.issuer_certificate = cert
-  cert.extensions = [
-    ef.create_extension("basicConstraints","CA:TRUE", true),
-    ef.create_extension("subjectKeyIdentifier", "hash"),
-    # ef.create_extension("keyUsage", "cRLSign,keyCertSign", true),
-  ]
-  cert.add_extension ef.create_extension("authorityKeyIdentifier",
-                                         "keyid:always,issuer:always")
-
-  cert.sign key, OpenSSL::Digest::SHA1.new
-
-  open( options[:public_key], "w" ) do |io|
-    io.write(cert.to_pem)
-  end
-  $stderr.puts "#{options[:public_key]} created."
-  exit
-end
-
-if options[:eyaml]
-
-  if options[:decrypt]
-    # prepare to decrypt blocks
-    private_key_pem = File.read( options[:private_key] )
-    private_key = OpenSSL::PKey::RSA.new( private_key_pem )
-
-    public_key_pem = File.read( options[:public_key] )
-    public_key = OpenSSL::X509::Certificate.new( public_key_pem )
-
-    eyaml = get_file_input options
-    plain_eyaml = decrypt_eyaml(public_key, private_key, eyaml)
-
-    puts plain_eyaml
-    exit
-  end
-
-  if options[:encrypt]
-    # prepare to encrypt blocks
-    public_key_pem = File.read( options[:public_key] )
-    public_key = OpenSSL::X509::Certificate.new( public_key_pem )
-
-    eyaml = get_file_input options
-    cipher_eyaml = encrypt_yaml(public_key, eyaml)
-
-    puts eyaml
-    exit
-  end
-
-else
-
-  if options[:encrypt]
-    plaintext = get_input options
-
-    if plaintext.nil? or plaintext.length == 0
-      $stderr.puts "Specify a string or --file to encrypt something. See --help for more usage instructions."
-      exit
-    end
-
-    public_key_pem = File.read( options[:public_key] )
-    public_key = OpenSSL::X509::Certificate.new( public_key_pem )
-    
-    ciphertext_as_block = encrypt(public_key, plaintext)
-    ciphertext_as_string = ciphertext_as_block.split("\n").join('')
-
-    case options[:output]
-    when "examples"
-      if options[:name].nil?
-      puts "string: ENC[#{ciphertext_as_string}]\n\nOR\n\n"
-      puts "block: >"
-      puts "    ENC[" + ciphertext_as_block.gsub(/\n/, "\n    ") + "]"
-      else
-      puts options[:name] + ": ENC[#{ciphertext_as_string}]\n\nOR\n\n"
-      puts options[:name] +": >"
-      puts "    ENC[" + ciphertext_as_block.gsub(/\n/, "\n    ") + "]" 
-      puts
-      end
-    when "block"
-      if options[:name].nil?
-      puts "ENC[" + ciphertext_as_block + "]"
-      else
-      puts options[:name] + ": >"
-      puts "    ENC[" + ciphertext_as_block.gsub(/\n/, "\n    ") + "]"
-      puts
-      end
-    when "string"
-      if options[:name].nil?
-      puts "ENC[#{ciphertext_as_string}]"
-      else
-      puts options[:name] + ": ENC[#{ciphertext_as_string}]"
-      puts
-      end
-    else
-      $stderr.puts "Unknown output option: " + options[:output]
-      exit 1
-    end
-    exit
-
-  end
-
-  if options[:decrypt]
-
-    ciphertext = get_input options
-    if ciphertext.nil? or ciphertext.length == 0
-      $stderr.puts "Specify a string or --file to decrypt something. See --help for more usage instructions."
-      exit 1
-    end
-
-    if ciphertext.start_with? "ENC["
-
-      ciphertext = ciphertext[4..-2]
-
-      private_key_pem = File.read( options[:private_key] )
-      private_key = OpenSSL::PKey::RSA.new( private_key_pem )
-
-      public_key_pem = File.read( options[:public_key] )
-      public_key = OpenSSL::X509::Certificate.new( public_key_pem )
-
-      plaintext = decrypt(public_key, private_key, ciphertext)
-      puts "#{plaintext}"
-      exit
-
-    else
-
-      $stderr.puts "Ciphertext is not an eyaml encrypted string (Does not start with ENC[...])"
-      exit 1
-
-    end
-
-    exit
-
-  end
-end
-
-
-if options[:edit]
-  $stderr.puts "Launching edit mode"
-  # prepare to edit blocks
-  private_key_pem = File.read( options[:private_key] )
-  private_key = OpenSSL::PKey::RSA.new( private_key_pem )
-
-  public_key_pem = File.read( options[:public_key] )
-  public_key = OpenSSL::X509::Certificate.new( public_key_pem )
-
-  original_eyaml = File.read( options[:file] )
-  original_size = original_eyaml.length
-  plain_eyaml = decrypt_eyaml(public_key, private_key, original_eyaml)
-
-  # write temp file
-  plain_file = Tempfile.open('eyaml_edit')
-  plain_file.puts plain_eyaml
-  plain_file.flush
-
-  # open editor
-  $editor = ENV['EDITOR']
-  if $editor == nil
-    %w{/usr/bin/sensible-editor /usr/bin/editor /usr/bin/vim /usr/bin/vi}.each do |editor|
-      if FileTest.executable?(editor)
-        $editor = editor
-        break
-      end
-    end
-  end
-  $stderr.puts "Opening decrypted file for editing in " + $editor
-  system($editor, plain_file.path)
-  status = $?
-
-  throw "Process has not exited!?" unless status.exited?
-
-  unless status.exitstatus == 0
-    $syserr.puts "Editor did not exit successfully (exit code #{status.exitstatus}), aborting"
-    exit 1
-  end
-
-  # some editors do not write new content in place, but instead
-  # make a new file and more it in the old file's place.
-  begin
-    reopened_plain_file = File.open(plain_file.path, "r+")
-  rescue Exception => e
-    STDERR.puts e
-    exit 1
-  end
-  new_eyaml = reopened_plain_file.read
-  
-  # make sure nothing is left on disk
-  new_size = new_eyaml.length
-  old_size = plain_eyaml.length
-  clear_size = (new_size > old_size) ? new_size : old_size
-  shred(plain_file, clear_size)
-  plain_file.close true
-  shred(reopened_plain_file, clear_size)
-  reopened_plain_file.close
-
-  if new_eyaml.length == 0
-    $stderr.puts "Replacement content is empty, aborting"
-    exit 1
-  end
-
-  if (new_eyaml == plain_eyaml)
-    $stderr.puts "No changes made, aborting"
-    exit 1
-  end
-
-  # re-encrypt the file again, currently we re-encrypt indiscriminately
-  cipher_eyaml = encrypt_eyaml(public_key, new_eyaml)
-
-  File.open(options[:file], 'w') { |file| 
-    file.write(cipher_eyaml)
-  }
-  exit
-end
+Hiera::Backend::Eyaml::CLI.parse
+Hiera::Backend::Eyaml::CLI.execute