secure_data_bag 2.2.0 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1a9feeb1dfc414c24ff0245dd5942cef163e5d4c
-  data.tar.gz: 89258e6b058db11c62d7dbd39af9eba58eef2700
+  metadata.gz: b22224b0b2102fd71b07fdc51184af1897b5b74e
+  data.tar.gz: 0bf974e3ba411d700bc55ed5d42fc9a7246258dd
 SHA512:
-  metadata.gz: 0d2aceb5f872a953842a29d00351bd2474696925dad5dda15bf6b09844b09652196f8fcb9e0c8e79f2ec5cb4e07aac152a48813eac87edce6ffea53983995465
-  data.tar.gz: d07e7e41cad38f400f64a84892ee03b089a49c88328c812d73433a5d711b067b533de271ae06ffa389db96c4f777f1c8bc72bc68069ee42f76c8a874088c2516
+  metadata.gz: 894f9fbfcf7dd4c3ad755302a06f0fd2dd489681f7336ad07609cedb7c3f09158c3b7560c8e2ea0211fa5147681b34514185b4cc442a4824896c6518e8491afd
+  data.tar.gz: 0710828746350ae0988324d280618527290e44bab263a6c464faab86215de89da56ff50fd4664c82eed4cc00f80973a1596bc414bc9bcdc1e2721f5ab3e4ff66

data/CHANGELOG.md

@@ -0,0 +1,8 @@
+secure_data_bag
+======
+# 2.2.0
+* Add support for printing data bag items after edit via `knife secure bag edit --print-after`
+* Resolve problem where `knife secure bag edit` would not encode fields
+
+# 2.1.x
+* Alot of initial commits ;)

data/README.md

@@ -1,41 +1,105 @@
-# SecureDataBag
+# SecureDataBag / Knife Secure Bag
 
-Provides a mechanism to partially encrypt data bag items on a per-key basis which gives us the opportunity to still search for every other field.
+Knife Secure Bag provides a consistent interface to DataBagItem, EncryptedDataBagItem as well as the custom created SecureDataBagItem while also providing a few extra handy features to help in your DataBag workflows. 
 
-When specifying keys to encrypt, the library will recursively walk through the data bag content searching for encryption candidates.
+SecureDataBagItem, can not only manage your existing DataBagItems and EncryptedDataBagItems, but it also provides you with a DataBag type which enables you to selectively encrypt only some of the fields in your DataBag thus allowing you to be able to search for the remaining fields. 
 
 ## Installation
 
-Add this line to your application's Gemfile:
+To build and install the plugin add it your Gemfile or run: 
 
-    gem 'secure_data_bag'
+```shell
+gem install secure_data_bag
+```
 
-And then execute:
+## Configuration
+
+#### Knife Secure Bag
+
+Defaults for the Knife command may be provided in your _knife.rb_ file.
+
+```ruby
+knife[:secure_data_bag][:encrypted_keys] = %w(
+  password
+  ssh_keys
+  ssh_ids
+  public_keys
+  private_keys
+  keys
+  secret
+)
+knife[:secure_data_bag][:secret_file] = "#{local_dir}/secret.pem"
+knife[:secure_data_bag][:export_root] = "#{kitchen_dir}/data_bags"
+knife[:secure_data_bag][:export_on_upload] = true
+knife[:secure_data_bag][:defaults][:secrets][:export_format] = 'plain'
+```
 
-    $ bundle
+To break this up:
 
-Or install it yourself as:
+`knife[:secure_data_bag][:encrypted_keys] = []`
 
-    $ gem install secure_data_bag
+When Knife Secure Bag encrypts a hash with an _encryption format_ of *nested*, it will recursively walk through the hash from the bottom up and encrypt any key found within this array.
 
-## Usage
+`knife[:secure_data_bag][:secret_file]`
 
-For the most part, this behaves exactly like a standard DataBagItem would. Encryption and Decryption of attributes ought to be completely transparent.
+When encryption is required, the shared secret found at this location will be loaded. 
 
-SecureDataBagItem is also able to read either standard DataBagItem objects or EncryptedDataBagItem since the encryption mechanism is on a per-key basis and is entirely compatible with EncryptedDataBagItem's format. One caveat, however, is that SecureDataBagItem can not be read by EncryptedDataBagItem.
+`knife[:secure_data_bag][:export_root]`
 
-SecureDataBagItem is also built on Mash rather than Hash so you'll find it more compatible with symbol keys. 
+When exporting a data\_bag\_item, files will be created in below this root directory. Typically this would be the data\_bag folder located within your kitchen.
 
-```
-> secret_key = SecureDataBagItem.load_key("/path/to/secret")
-> secret_key = nil # Load default secret
+`knife[:secure_data_bag][:export_on_upload]`
+
+When a data\_bag\_item is edited using `knife secure bag edit`, it may be automatically exported to the _export\_root_.
+
+`knife[:secure_data_bag][:defaults][:secrets][:export_format]`
+
+The configuration file additionally supports the _defaults_ hash which provides default values for all _command line arguments_ that one might use. Of all of them only the _export\_format_ key is likely to be of much use.
+
+## Examples
+
+#### Chef cookbook recipe
+
+```ruby
+metadata = {}
+
+# Define the keys we wish to encrypt
+metadata[:encrypted_keys] = %w(encoded)
+
+# Optionally load a specific shared secret. Otherwise, the global 
+# encrypted\_data\_bag\_secret will be automatically used.
+secret_key = SecureDataBagItem.load_key("/path/to/secret")
+
+# Create a hash of data to use as an exampe
+raw_data = {
+	id: "item", 
+  data_bag: "data_bag",
+	encoded: "my string", 
+	unencoded: "other string"
+}
+
+# Instantiate a SecureDataBagItem from a hash
+item = SecureDataBagItem.from_hash(data, metadata)
 
-> data = { id:"databag", "encoded":"my string", "unencoded":"other string" }
+# Or more explicitely
+item = SecureDataBagItem.from_hash(data, encrypted_keys: %w(encoded))
 
-> item = SecureDataBagItem.from_hash(data, key: secret_key)
-> item.raw_data # Unencoded hash
+# Or load from server
+item = SecureDataBagItem.load("data_bag", "item")
+
+# Print the un-encrypted raw data
+pp item.raw_data
+
+# Print the un-encrypted `encoded` key
+pp item['encoded']
+
+# Print the encrypted hash as a data_bag_item hash
+pp item.to_hash
+
+=begin
 { 
-  id:         "databag", 
+  id:         "item", 
+  data_bag:   "data_bag",
   encoded:    {
     encrypted_data: "encoded",
     cipher: aes-256-cbc,
@@ -44,41 +108,96 @@ SecureDataBagItem is also built on Mash rather than Hash so you'll find it more
   }
   unencoded:  "other string",
 }
-
-> item.to_hash
-{ 
-  id:         "databag", 
-  chef_type:  "data_bag_item",
-  data_bag:   "",
-  encoded:    "my string",
-  unencoded:  "other string"
-}
+=end
 ```
 
-A few knife commands are also provided which allow you to edit / show / from file any DataBagItem or EncryptedDataBagItem and convert them to SecureDataBag::Item format.
+## Usage
 
-```
-knife secure bag --help
-** SECURE BAG COMMANDS **
-knife secure bag edit BAG [ITEM] (options)
-knife secure bag from file BAG FILE|FLDR [FILE|FLDR] (options)
-knife secure bag show BAG [ITEM] (options)
-```
+#### Knife commands
 
-Additionally it accepts the following command-line options :
+Print an DataBagItem, EncryptedDataBagItem or SecureDataBagItem, auto-detecting the encryption method used as plain text.
 
+```shell
+knife secure bag show -F js secrets secret_item
 ```
---secret-file /path/to/secret.pem
---secret passcode
---encoded-fields field_one,field_two,field_three
+
+Print an DataBagItem, EncryptedDataBagItem or SecureDataBagItem, auto-detecting the encryption method used as a SecureDataBagItem in encrypted format.
+
+```shell
+knife secure bag show -F js secrets secret_item --enc-format nested
 ```
 
-Which may also be configured in knife.rb:
+Edit an EncryptedDataBagItem, preserve it's encryption type, and export a copy to the _data\_bag_ folder in your kitchen.
 
+```shell
+knife secure bag edit secrets secret_item --export
 ```
-knife[:secure_data_bag] = {
-  fields: ["password","ssh_keys"],
-  secret_file: "/path/to/secret.pem"
-}
+
+## Knife SubCommands
+
+Most of the SubCommands support the following command-line options:
+
+`--enc-format [plain,encrypted,nested]`
+
+Ensure that, when displaying or uploading the data\_bag\_item, we forcibly encrypt the data\_bag\_item using the specified format instead of preserving the existing format. 
+
+In this case: 
+- plain: refers to a DataBagItem
+- encrypted: refers to an EnrytpedDataBagItem
+- nested: refers to a SecureDataBagItem
+
+`--dec-format [plain,encrypted,nested]`
+
+Attempt to decrypt the data\_bag\_item using the given format rather than the auto-detected one. The only real reason to use this is when you wish to specifically select _plain_ as the format so as to not decrypt the item.
+
+`--enc-keys key1,key2,key3`
+
+Provide a comma delimited list of hash keys which should be encrypted when encrypting the data\_bag\_item. This list will be concatenated with any key names listed in the configuration file or which were previously encrypted. 
+
+`--export`
+
+Export the data\_bag\_item to json file in either of _export-format_ or _enc-format_.
+
+`--export-format`
+
+Overrides the encryption format only for the _export_ feature.
+
+`--export-root`
+
+Root directly under which a folder should exist for each _data_bag_ into which to export _data_bag_items_ as json files.
+
+When displaying the content of the _data\_bag\_item_, an additional key of *_secure_metadata* will be added to the output which contains gem specific metadata such as the encryption formats and any encrypted keys found. This key will _not_ be saved with the item, however it may be manipulated to alter the behavior of the _edit_ or _export_ commands.
+
+#### knife secure bag show DATA_BAG ITEM
+
+This command functions just like `knife data bag show` and is used to print out the content of either a DataBagItem, EncryptedDataBagItem or SecureDataBagItem.
+
+By default, it will auto-detect the Item type, and print it's unencrypted version to the terminal. This behavior, however, may be altered using the previously mentioned command line options.
+
+#### knife secure bag open PATH
+
+This commands functions much like `knife secure bag show`, however it is designed to load a _data\_bag\_item_ from disk as opposed to loading it from Chef server. This may be of use when view the content of an exported encrypted file.
+
+#### knife secure bag edit DATA_BAG DATA_BAG_ITEM
+
+This command functions just like `knife data bag edit` and is used to edit either a DataBagItem, EncryptedDataBagItem or a SecureDataBagItem. It supports all of the same options as `knife secure bag show`. 
+
+#### knife secure bag from file DATA_BAG PATH
+
+This command functions just like `knife data bag from file` and is used to upload either a DataBagItem, EncryptedDataBagItem or a SecureDataBagItem. It supports all of the same options as `knife secure bag show`. 
+
+## Recipe DSL
+
+The gem additionally provides a few Recipe DSL methods which may be useful.
+
+```ruby
+load_secure_item = secure_data_bag_item(
+  data_bag_name, 
+  data_bag_item, 
+  cache: false
+)
+
+load_plain_item = data_bag_item(data_bag_name, data_bag_item)
+convert_plain_to_secure = secure_data_bag_item!(load_plain_item)
 ```
 

data/Rakefile

@@ -1,2 +1 @@
-require "bundler/gem_tasks"
-
+require 'bundler/gem_tasks'

data/lib/chef/config.rb

@@ -1,14 +1,12 @@
-
 require 'chef/config'
 
 class Chef
   class Config
     config_context :knife do
       config_context :secure_data_bag do
-        default :secret_file, nil
-        default :fields, nil
+        config_context :export_metadata do
+        end
       end
     end
   end
 end
-

data/lib/chef/dsl/data_query.rb

@@ -1,4 +1,3 @@
 
-require "chef/dsl/data_query"
+require 'chef/dsl/data_query'
 Chef::DSL::DataQuery.send(:include, SecureDataBag::DSL::DataQuery)
-

data/lib/chef/knife/secure_bag_edit.rb

@@ -1,60 +1,58 @@
-
-require 'chef/knife/secure_bag_base'
 require 'chef/knife/data_bag_edit'
+require_relative 'secure_data_bag/base_mixin'
+require_relative 'secure_data_bag/export_mixin'
+require_relative 'secure_data_bag/secrets_mixin'
+require_relative 'secure_data_bag/defaults_mixin'
 
 class Chef
   class Knife
     class SecureBagEdit < Knife::DataBagEdit
-      include Knife::SecureBagBase
-
-      banner "knife secure bag edit BAG [ITEM] (options)"
-      category "secure bag"
+      include SecureDataBag::BaseMixin
+      include SecureDataBag::ExportMixin
+      include SecureDataBag::SecretsMixin
+      include SecureDataBag::DefaultsMixin
 
-      def load_item(bag, item_name)
-        item = SecureDataBag::Item.load(bag, item_name)
-        hash = item.to_hash(encoded: false)
-        hash["_encoded_fields"] = item.encoded_fields
-        hash
-      end
+      banner 'knife secure bag edit BAG [ITEM] (options)'
+      category 'secure bag'
 
       def run
         if @name_args.length != 2
-          stdout.puts "You must supply the data bag and an item to edit!"
+          stdout.puts 'You must supply the data bag and an item to edit.'
           stdout.puts opt_parser
           exit 1
         end
 
+        config_defaults_for_data_bag!(@name_args[0])
+
         # Load the SecureBagItem, EncryptedDataBagItem or DataBagItem
-        item = load_item(@name_args[0], @name_args[1])
+        item = load_item(@name_args[0], @name_args[1], config_metadata)
+        item_metadata = item.metadata.dup
+        item.encryption_format = 'plain'
 
         # Allow the user to modify the content
-        edited_item = edit_hash(item)
-
-        # Fetch the fields that are to be encoded
-        fields_to_encode = edited_item.delete("_encoded_fields")
-        if fields_to_encode and not fields_to_encode.empty?
-          ui.info("Saving with secure fields: #{fields_to_encode.join(", ")}")
-        else
-          ui.info("Saving without any secure fields")
-        end
+        data = item.to_hash(metadata: true)
+        data[::SecureDataBag::METADATA_KEY] = item_metadata
+
+        # Edit the hash
+        edited_item = edit_hash(data)
+        item_metadata = edited_item.delete(::SecureDataBag::METADATA_KEY)
 
         # Generate a new SecureBagItem
-        item_to_save = SecureDataBag::Item.new(
-          data: edited_item,
-          fields: fields_to_encode
-        )
-        item_to_save.data_bag @name_args[0] # Set data_bag to match initial
-        item_to_save["id"] = @name_args[1]     # Ensure id was not changed
+        item_to_save = ::SecureDataBag::Item
+                       .from_hash(edited_item, item_metadata)
+        item_to_save.data_bag @name_args[0]
+        item_to_save['id'] = @name_args[1]
+
         item_to_save.save
+        stdout.puts("Saved as #{@name_args[0]}[#{@name_args[1]}]")
 
-        stdout.puts("Saved data_bag_item[#{@name_args[1]}]")
+        export!(@name_args[0], @name_args[1], item_to_save) if should_export?
 
         if config[:print_after]
-          data_to_print = item_to_save.to_hash(encoded: true)
-          ui.output(Chef::JSONCompat.to_json_pretty(data_to_print))
+          data_to_print = item_to_save.to_hash
+          stdout.puts(Chef::JSONCompat.to_json_pretty(data_to_print))
         end
       end
     end
   end
 end
-

data/lib/chef/knife/secure_bag_from_file.rb

@@ -1,42 +1,34 @@
-
-require 'chef/knife/secure_bag_base'
 require 'chef/knife/data_bag_from_file'
+require_relative 'secure_data_bag/base_mixin'
+require_relative 'secure_data_bag/secrets_mixin'
+require_relative 'secure_data_bag/defaults_mixin'
 
 class Chef
   class Knife
     class SecureBagFromFile < Knife::DataBagFromFile
-      include Knife::SecureBagBase
+      include SecureDataBag::BaseMixin
+      include SecureDataBag::SecretsMixin
+      include SecureDataBag::DefaultsMixin
 
       deps do
         require 'chef/data_bag'
         require 'chef/data_bag_item'
         require 'chef/knife/core/object_loader'
         require 'chef/json_compat'
-        require 'secure_data_bag'
+        require 'chef/encrypted_data_bag_item'
       end
 
-      banner "knife secure bag from file BAG FILE|FLDR [FILE|FLDR] (options)"
-      category "secure bag"
-
-      option :all,
-        short:  "-a",
-        long:   "--all",
-        description: "Upload all data bags or all items for specified databag"
+      banner 'knife secure bag from file BAG FILE|FLDR [FILE|FLDR] (options)'
+      category 'secure bag'
 
-      def load_data_bag_hash(hash)
-        item = SecureDataBag::Item.from_hash hash, 
-          fields: encoded_fields,
-          secret: secret
-        item
-      end
+      def load_data_bag_items(data_bag, items = nil)
+        config_defaults_for_data_bag!(data_bag)
 
-      def load_data_bag_items(data_bag, items=nil)
         items ||= find_all_data_bag_items(data_bag)
         item_paths = normalize_item_paths(items)
         item_paths.each do |item_path|
-          item = loader.load_from("#{data_bags_path}", data_bag, item_path)
-          item = load_data_bag_hash(item)
-          item.data_bag(data_bag)
+          item = loader.load_from(data_bags_path, data_bag, item_path)
+          item = create_item(data_bag, item_path, item)
           item.save
           ui.info("Updated data_bag_item[#{item.data_bag}::#{item.id}]")
         end
@@ -44,4 +36,3 @@ class Chef
     end
   end
 end
-