chef-vault 2.9.2 → 3.0.0.rc1

data/README.md

@@ -1,8 +1,8 @@
 # Chef-Vault
 
-[![Gem Version](https://badge.fury.io/rb/chef-vault.png)](http://badge.fury.io/rb/chef-vault)
+[![Gem Version](https://badge.fury.io/rb/chef-vault.svg)](http://badge.fury.io/rb/chef-vault)
 
-[![Build Status](https://travis-ci.org/chef/chef-vault.png?branch=master)](https://travis-ci.org/chef/chef-vault)
+[![Build Status](https://travis-ci.org/chef/chef-vault.svg?branch=master)](https://travis-ci.org/chef/chef-vault)
 
 [![Inline docs](http://inch-ci.org/github/chef/chef-vault.svg?branch=master)](http://inch-ci.org/github/chef/chef-vault)
 
@@ -32,6 +32,69 @@ This plugin is distributed as a Ruby Gem. To install it, run:
 Depending on your system's configuration, you may need to run this command
 with root privileges.
 
+## DEVELOPMENT:
+
+### Git Hooks
+
+There is a git pre-commit hook to help you keep your chefstyle up to date.
+If you wish to use it, simply:
+
+```
+mv hooks/pre-commit .git/hooks/
+chmod +x .git/hooks/pre-commit
+```
+
+### Running Your Changes
+
+To run your changes locally:
+
+```
+bundle install
+bundle exec knife vault
+```
+
+### Testing
+
+#### Rspec Tests
+
+There are some unit tests that can be run with:
+
+```
+bundle exec rspec spec/
+```
+
+#### Cucumber Testing
+
+There are cucumber tests. Run the whole suite with:
+
+```
+bundle exec rake features
+```
+
+If you get any failures, you can run the specific feature that failed with:
+
+```
+bundle exec cucumber features/<failed>.feature
+```
+
+If you want to test things out directly, after a failure you can go into the test
+directory and try out the commands that failed:
+
+```
+cd tmp/aruba
+bundle exec knife <your command that failed from test with -c knife.rb>
+```
+
+Optionally add `-VV` to the above to get a full stacktrace.
+
+### Rubocop Errors
+
+If you are seeing rubocop errors in travis for your pull request, run:
+
+`bundle exec chefstyle -a`
+
+This will fix up your rubocop errors automatically, and warn you about any it can't.
+
 ## KNIFE COMMANDS:
 
 See KNIFE_EXAMPLES.md for examples of commands
@@ -73,12 +136,14 @@ Short | Long | Description | Default | Valid Values | Sub-Commands
 ------|------|-------------|---------|--------------|-------------
 -M MODE | --mode MODE | Chef mode to run in. Can be set in knife.rb | solo | solo, client | all
 -S SEARCH | --search SEARCH | Chef Server SOLR Search Of Nodes | | | create, remove , update
+-C CLIENTS | --clients CLIENTS | Chef clients to be added as clients, can be comma list | | | create, remove , update
 -A ADMINS | --admins ADMINS | Chef clients or users to be vault admins, can be comma list | | | create, remove, update
 -J FILE | --json FILE | JSON file to be used for values, will be merged with VALUES if VALUES is passed | | | create, update
 | --file FILE | File that chef-vault should encrypt.  It adds "file-content" & "file-name" keys to the vault item | | | create, update
 -p DATA | --print DATA | Print extra vault data | | search, clients, admins, all | show
 -F FORMAT | --format FORMAT | Format for decrypted output | summary | summary, json, yaml, pp | show
 | --clean-unknown-clients | Remove unknown clients during key rotation | | | refresh, remove, rotate
+| --clean | Clean clients list before performing search | | | refresh, update
 
 ## USAGE IN RECIPES
 
@@ -228,8 +293,9 @@ small pull requests are preferred to large omnibus patches, as the
 robustness pass is a multi-person effort and we don't want to create merge
 conflicts unnecessarily.
 
-We also have a [Gitter room](https://gitter.im/Nordstrom/chef-vault)
-where you can discuss chef-vault and the robustness improvements.
+## Contributing
+
+For information on contributing to this project see <https://github.com/chef/chef/blob/master/CONTRIBUTING.md>
 
 ## Authors
 
@@ -238,6 +304,7 @@ Author:: Eli Klein - @eliklein<br>
 Author:: Joey Geiger - @jgeiger<br>
 Author:: Joshua Timberman - @jtimberman<br>
 Author:: James FitzGibbon - @jf647<br>
+Author:: Thom May - @thommay<br>
 
 ## Contributors
 
@@ -248,8 +315,10 @@ Contributor:: Reto Hermann<br>
 ## License
 
 Copyright:: Copyright (c) 2013-15 Nordstrom, Inc.<br>
+Copyright:: Copyright (c) 2016 Chef Software, Inc.<br>
 License:: Apache License, Version 2.0
 
+```text
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
@@ -261,3 +330,4 @@ distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
+```

data/Rakefile

@@ -46,5 +46,5 @@ end
 
 # test or the default task runs spec, features, style
 desc "run all tests"
-task default: [:spec, :features, :style]
+task default: [:coverage, :features, :style]
 task test: :default

data/bin/chef-vault

@@ -86,12 +86,13 @@ require "rubygems"
 $:.unshift(File.join(File.dirname(__FILE__), "..", "lib"))
 require "chef-vault"
 
+ChefVault::Log.init(STDOUT)
 ChefVault.load_config(options[:chef])
 item = ChefVault::Item.load(options[:vault], options[:item])
 
-$stdout.puts "#{options[:vault]}/#{options[:item]}"
+ChefVault::Log.info "#{options[:vault]}/#{options[:item]}"
 
 options[:values].split(",").each do |value|
   value.strip! # remove white space
-  $stdout.puts("\t#{value}: #{item[value]}")
+  ChefVault::Log.info ("\t#{value}: #{item[value]}")
 end

data/chef-vault.gemspec

@@ -21,32 +21,30 @@ Gem::Specification.new do |s|
   s.name             = "chef-vault"
   s.version          = ChefVault::VERSION
   s.has_rdoc         = true
-  s.authors          = ["Kevin Moser", "James FitzGibbon"]
-  s.email            = ["techcheftm@nordstrom.com"]
+  s.authors          = ["Thom May"]
+  s.email            = ["thom@chef.io"]
   s.summary          = "Data encryption support for Chef using data bags"
   s.description      = s.summary
   s.homepage         = "https://github.com/chef/chef-vault"
-
   s.license          = "Apache License, v2.0"
-
   s.files            = `git ls-files`.split("\n")
   s.require_paths    = ["lib"]
   s.bindir           = "bin"
   s.executables      = %w{ chef-vault }
 
-  s.add_development_dependency "rake", "~> 10.4"
-  s.add_development_dependency "rspec", "~> 3.2"
+  s.required_ruby_version = ">= 2.2.0"
+
+  s.add_development_dependency "rake", "~> 11.0"
+  s.add_development_dependency "rspec", "~> 3.4"
+  s.add_development_dependency "mutant-rspec"
   s.add_development_dependency "aruba", "~> 0.6"
   s.add_development_dependency "simplecov", "~> 0.9"
   s.add_development_dependency "simplecov-console", "~> 0.2"
-  s.add_development_dependency "rubocop", "~> 0.30"
-  # Chef 12 and higher pull in Ohai 8, which needs Ruby v2
-  # so only in the case of a CI build on ruby v1, we constrain
-  # chef to 11 or lower so that we can maintain CI test coverage
-  # of older versions
-  if ENV.key?("TRAVIS_BUILD") && RUBY_VERSION =~ /^1/
-    s.add_development_dependency "chef", "~> 11.18"
-  else
-    s.add_development_dependency "chef", ">= 0.10.10"
+  if ENV.key?("TRAVIS_BUILD") && RUBY_VERSION == "2.1.9"
+    # Test version of Chef with Chef Zero before
+    # /orgs/org/users/user/keys endpoint was added.
+    s.add_development_dependency "chef", "12.8.1"
+  else # Test most current version of Chef on 2.2.2
+    s.add_dependency :chef, "~> 12"
   end
 end

data/features/clean.feature

@@ -1,5 +1,4 @@
 Feature: clean client keys
-
   When updating a vault item, chef-vault normally performs the
   saved or specified query and encrypts the item for all nodes
   returned.  It does not remove old client keys from the vault

data/features/clean_on_refresh.feature

@@ -1,5 +1,4 @@
 Feature: clean unknown clients on vault refresh
-
   When refreshing a vault, new clients may be added if they match
   the search query or client list, but old clients that no longer
   exist will never be removed.  The --clean-unknown-clients switch

data/features/clean_unknown_clients.feature

@@ -1,5 +1,4 @@
 Feature: clean unknown clients on key rotation
-
   When removing a client from a vault item, chef-vault normally
   removes the key and then rotates the key.  If a client has been
   deleted in the meantime from the Chef server but not the vault,

data/features/detect_and_warn_v1_vault.feature

@@ -1,5 +1,4 @@
 Feature: Detect and Warn for v1 Vaults
-
   chef-vault can read a v1 vault, but the management commands
   tend to break when they try to reference v2 fields like
   clients and admins.  They should detect and warn when trying

data/features/isvault.feature

@@ -1,5 +1,4 @@
 Feature: determine if a data bag item is a vault
-
   If a data bag item is a vault, 'knife vault isvault VAULTNAME ITEMNAME'
   should exit 0.  Otherwise it should exit 1.
 

data/features/itemtype.feature

@@ -1,5 +1,4 @@
 Feature: determine the type of a data bag item
-
   'knife vault itemtype VAULTNAME ITEMNAME' should output one of
   'normal', 'encrypted', or 'vault' depending on what type of item
   it detects

data/features/vault_create.feature

@@ -1,5 +1,4 @@
 Feature: knife vault create
-
   'knife vault create' creates two Chef data bag items: an
   encrypted data bag item encrypted with a randomized shared
   secret, and a side-along data bag item suffixed with _keys
@@ -51,4 +50,4 @@ Feature: knife vault create
     Given a local mode chef repo with nodes 'one,two'
     And I create a vault item 'test/item' containing the JSON '{"foo": "bar"}' encrypted for 'one,two,three' with 'alice' as admin
     Then the exit status should not be 0
-    And the output should contain "FATAL: Could not find alice in users or clients!"
+    And the output should contain "FATAL: Could not find default key for alice in users or clients!"

data/features/vault_list.feature

@@ -1,5 +1,4 @@
 Feature: list data bags that are vaults
-
   knife vault list should list all data bags that appear to
   be vaults.  This is not an exact science; we assume that
   any data bag containing an even number of items and for

data/features/vault_show.feature

@@ -1,5 +1,4 @@
 Feature: knife vault show
-
   'knife vault show' displays the contents of a Chef encrypted
   data bag by fetching the asymmetrically encrypted shared
   secret and decrypting it using the private key of the user

data/features/vault_show_vaultname.feature

@@ -1,5 +1,4 @@
 Feature: knife vault show [VAULTNAME]
-
   'knife vault show [VAULTNAME]' displays the keys of a vault
   (i.e. the items that are not suffixed with _keys)
 

data/features/vault_update.feature

@@ -1,5 +1,4 @@
 Feature: knife vault update
-
   'knife vault update' is used to add clients, or administrators
   and to re-run the search query and update the vault's item values.
 

data/features/verify_id_matches.feature

@@ -1,5 +1,4 @@
 Feature: knife vault create with mismatched ID
-
   'knife vault create' creates a vault.  A JSON file can be passed
   on the command line.  If the vault ID specified on the command line
   does not match the value of the 'id' key in the JSON file, knife

data/features/wrong_private_key.feature

@@ -1,5 +1,4 @@
 Feature: Wrong private key during decrypt
-
   https://github.com/Nordstrom/chef-vault/issues/43
   If a vault is encrypted for a node and then the node's private
   key is regenerated, the error that comes back from chef-vault

data/hooks/pre-commit

@@ -0,0 +1,43 @@
+#!/usr/bin/env ruby
+output = `bundle exec chefstyle -a`
+if !$?.success?
+  puts "pre-commit hook: Tried to run `bundle exec chefstyle -a` to autocleanup errors, but it failed with output:"
+  puts output
+end
+
+detected  = /(\d+) offenses detected/.match(output)
+corrected = /(\d+) offenses corrected/.match(output)
+
+# no errors detected by chefstyle
+exit 0 if detected.nil?
+
+# chefstyle found errors
+if !detected.nil?
+  # get the first result from the capture group that isn't the whole capture
+  num_detected = detected.to_a[1].to_i
+  num_corrected = if corrected.nil?
+                    0
+                  else
+                    corrected.to_a[1].to_i
+                  end
+  if num_detected == num_corrected
+    puts <<EOF
+pre-commit hook: Ran `bundle exec chefstyle -a` to autocleanup errors if any existed and
+#{num_detected} were detected, but all were cleaned up. `git add` all files that were
+autoupdated and try commiting again. New git status:
+
+EOF
+    puts `git status`
+  else
+    puts <<EOF
+pre-commit hook: Ran `bundle exec chefstyle -a` to autocleanup errors if any existed and
+#{num_detected} were detected, but #{num_detected - num_corrected} could not be cleaned up
+automatically. Run:
+
+bundle exec chefstyle -a
+
+to see remaining errors to clean up by hand, add all updated files, and try commiting again.
+EOF
+  end
+  exit 1
+end

data/lib/chef-vault.rb

@@ -29,8 +29,10 @@ require "chef-vault/item"
 require "chef-vault/item_keys"
 require "chef-vault/user"
 require "chef-vault/certificate"
-require "chef-vault/chef_patch/api_client"
-require "chef-vault/chef_patch/user"
+require "chef-vault/chef_api"
+require "chef-vault/actor"
+
+require "mixlib/log"
 
 class ChefVault
   attr_accessor :vault
@@ -55,4 +57,10 @@ class ChefVault
   def self.load_config(chef_config_file)
     Chef::Config.from_file(chef_config_file)
   end
+
+  class Log
+    extend Mixlib::Log
+  end
+
+  Log.level = :error
 end

data/lib/chef-vault/actor.rb

@@ -0,0 +1,149 @@
+# Author:: Tyler Cloke <tyler@chef.io>
+# Copyright:: Copyright 2016, Chef Software, Inc.
+# License:: Apache License, Version 2.0
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+require "json"
+
+class ChefVault
+  class Actor
+
+    attr_accessor :key_string
+    attr_reader :type
+    attr_reader :name
+
+    def initialize(actor_type, actor_name)
+      if actor_type != "clients" && actor_type != "admins"
+        raise "You must pass either 'clients' or 'admins' as the first argument to ChefVault::Actor.new."
+      end
+      @type = actor_type
+      @name = actor_name
+    end
+
+    def key
+      @key ||= is_admin? ? get_admin_key : get_client_key
+    end
+
+    def get_admin_key
+      # chef vault currently only supports using the default key
+      get_key("users")
+    rescue Net::HTTPServerException => http_error
+      # if we failed to find an admin key, attempt to load a client key by the same name
+      case http_error.response.code
+      when "403"
+        print_forbidden_error
+        raise http_error
+      when "404"
+        begin
+          ChefVault::Log.warn "The default key for #{name} not found in users, trying client keys."
+          get_key("clients")
+        rescue Net::HTTPServerException => http_error
+          case http_error.response.code
+          when "404"
+            raise ChefVault::Exceptions::AdminNotFound,
+                  "FATAL: Could not find default key for #{name} in users or clients!"
+          when "403"
+            print_forbidden_error
+            raise http_error
+          else
+            raise http_error
+          end
+        end
+      else
+        raise http_error
+      end
+    end
+
+    def get_client_key
+      begin
+        get_key("clients")
+      rescue Net::HTTPServerException => http_error
+        if http_error.response.code.eql?("403")
+          print_forbidden_error
+          raise http_error
+        elsif http_error.response.code.eql?("404")
+          raise ChefVault::Exceptions::ClientNotFound,
+                "#{name} is not a valid chef client and/or node"
+        else
+          raise http_error
+        end
+      end
+    end
+
+    def is_client?
+      type == "clients"
+    end
+
+    def is_admin?
+      type == "admins"
+    end
+
+    # @private
+
+    def api
+      @api ||= ChefVault::ChefApi.new
+    end
+
+    # Use API V0 to load the public_key directly from the user object
+    # using the chef-client code.
+    def chef_api_client
+      @chef_api_client ||= begin
+                             require "chef/api_client"
+                             Chef::ApiClient
+                           end
+    end
+
+    # Similar thing as above but for client.
+    def chef_user
+      @chef_user ||= begin
+                       require "chef/user"
+                       Chef::User
+                     end
+    end
+
+    def get_key(request_actor_type)
+      api.org_scoped_rest_v1.get("#{request_actor_type}/#{name}/keys/default").fetch("public_key")
+    # If the keys endpoint doesn't exist, try getting it directly from the V0 chef object.
+    rescue Net::HTTPServerException => http_error
+      raise http_error unless http_error.response.code.eql?("404")
+      if request_actor_type.eql?("clients")
+        chef_api_client.load(name).public_key
+      else
+        chef_user.load(name).public_key
+      end
+    end
+
+    def print_forbidden_error
+      ChefVault::Log.error <<EOF
+ERROR: You received a 403 FORBIDDEN while requesting an #{type} key for #{name}.
+
+If you are on Chef Server < 12.5:
+  Clients do not have access to all public keys within their org.
+  Either upgrade to Chef Server >= 12.5 or make this request using a user.
+
+If you are on Chef Server == 12.5.0
+  All clients and users have access to the public keys endpoint. Getting
+  this error on 12.5.0 is unexpected regardless of what your
+  public_key_read_access_group contains.
+
+If you are on Chef Server > 12.5.1
+  Has your public_key_read_access_group been modified? This group controls
+  read access on public keys within your org. It defaults to the users
+  and client groups, so all org actors should have permission unless
+  the defaults have been changed.
+
+EOF
+    end
+  end
+end