vault-rails 0.3.1 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 47998d084dd52dda1d47a5f83744814782d82c0d
-  data.tar.gz: 74419f45c8933e11f2ec4fef210c3b4fc876848c
+SHA256:
+  metadata.gz: 60cc2b942cbd53163fc7effc3b0f47afc268f203ce079c444635ca9b4f14cc3a
+  data.tar.gz: d1c4e963e55205851a0e866eae50067f0a6c4839b53327ae6bb190d6ba98dd7f
 SHA512:
-  metadata.gz: 27a4208b7a3c8a8b398a5b108b01c275bddfa73a8ec1747580227307ea5f0f87b0c2d184c73fd4740cd6be54f50f7be36afbda5590c5cd44d8d513a678d7741b
-  data.tar.gz: 0cbfe20010c58fcb42a9c39d4a1e5e5bc5543a3d7b48a93e22d62fb8db726a5d06acb6d3be974c15909ce1766216e83d4b7c40af1c1fced8754a3c4471d73e37
+  metadata.gz: d08dbab43552537a3380e798efc70ba83694ac6e41bbd3e564c22ae0aead476c9080cf57685b9ac3a4e0719769ff91ce93a72c5bdd64b1a7593700a4306530a8
+  data.tar.gz: 594a5b3017457cee3369553b118ab67a105cabc42d7d6b52c4130953d954ec8063b4d52392088ce67f31a07ed6800134d1a5819c18378b312ede905ce9caea69

data/README.md

@@ -1,16 +1,27 @@
-Vault Rails [![Build Status](https://secure.travis-ci.org/hashicorp/vault-rails.svg?branch=master)](http://travis-ci.org/hashicorp/vault-rails)
+Vault Rails [![Build Status](https://circleci.com/gh/hashicorp/vault-rails.svg?style=shield)](https://circleci.com/gh/hashicorp/vault-rails)
 ===========
 
 Vault is the official Rails plugin for interacting with [Vault](https://vaultproject.io) by HashiCorp.
 
-**The documentation in this README corresponds to the master branch of the Vault Rails plugin. It may contain unreleased features or different APIs than the most recently released version. Please see the Git tag that corresponds to your version of the Vault Rails plugin for the proper documentation.**
+**If you're viewing this README from GitHub on the `master` branch, know that it may contain unreleased features or
+different APIs than the most recently released version. Please see the Git tag that corresponds to your version of the
+Vault Rails plugin for the proper documentation.**
+
+## Table of Contents
+1. [Quick Start](#quick-start)
+1. [Advanced Configuration](#advanced-configuration)
+1. [Caveats](#caveats)
+1. [Development](#development)
+
 
 Quick Start
 -----------
+↥ [back to top](#table-of-contents)
+
 1. Add to your Gemfile:
 
     ```ruby
-    gem "vault-rails", "~> 0.1", require: false
+    gem "vault-rails", require: false
     ```
 
     and then run the `bundle` command to install.
@@ -25,13 +36,13 @@ Quick Start
       # disabled, vault-rails will encrypt data in-memory using a similar
       # algorithm to Vault. The in-memory store uses a predictable encryption
       # which is great for development and test, but should _never_ be used in
-      # production.
+      # production. Default: ENV["VAULT_RAILS_ENABLED"].
       vault.enabled = Rails.env.production?
 
       # The name of the application. All encrypted keys in Vault will be
       # prefixed with this application name. If you change the name of the
       # application, you will need to migrate the encrypted data to the new
-      # key namespace.
+      # key namespace. Default: ENV["VAULT_RAILS_APPLICATION"].
       vault.application = "my_app"
 
       # The address of the Vault server. Default: ENV["VAULT_ADDR"].
@@ -60,7 +71,7 @@ Quick Start
 
     ```ruby
     class AddEncryptedSSNToPerson < ActiveRecord::Migration
-      add_column :persons, :ssn_encrypted, :string
+      add_column :people, :ssn_encrypted, :string
     end
     ```
 
@@ -72,10 +83,13 @@ Quick Start
     person.save #=> true
     person.ssn_encrypted #=> "vault:v0:EE3EV8P5hyo9h..."
     ```
+- **Note** The unencrypted value will still be saved if the attribute referenced has a corresponding column. (i.e. `ssn` in the case above)
 
 
 Advanced Configuration
 ----------------------
+↥ [back to top](#table-of-contents)
+
 The following section details some of the more advanced configuration options for vault-rails. As a general rule, you should try to use vault-rails without these options until absolutely necessary.
 
 #### Specifying the encrypted column
@@ -90,7 +104,7 @@ vault_attribute :credit_card,
 - **Note** This value **cannot** be the same name as the vault attribute!
 
 #### Specifying a custom key
-By default, the name of the key in Vault is `#{app}_#{table}_#{column}`. This is customizable by setting the `:key` coption when declaring the attribute:
+By default, the name of the key in Vault is `#{app}_#{table}_#{column}`. This is customizable by setting the `:key` option when declaring the attribute:
 
 ```ruby
 vault_attribute :credit_card,
@@ -99,7 +113,66 @@ vault_attribute :credit_card,
 
 - **Note** Changing this value for an existing application will make existing values no longer decryptable!
 
+#### Specifying a context (key derivation)
+
+Vault Transit supports key derivation, which allows the same key to be used for multiple purposes by deriving a new key based on a context value.
+
+The context can be specified as a string, symbol, or proc. Symbols (an instance method on the model) and procs are called for each encryption or decryption request, and should return a string.
+
+- **Note** Changing the context or context generator for an attribute will make existing values no longer decryptable!
+
+##### String
+
+With a string, all records will use the same context for this attribute:
+
+```ruby
+vault_attribute :credit_card,
+  context: "user-cc"
+```
+
+##### Symbol
+
+When using a symbol, a method will be called on the record to compute the context:
+
+```ruby
+belongs_to :user
+
+vault_attribute :credit_card,
+  context: :encryption_context
+
+def encryption_context
+  "user_#{user.id}"
+end
+```
+
+##### Proc
+
+Given a proc, it will be called each time to compute the context:
+
+```ruby
+belongs_to :user
+
+vault_attribute :credit_card,
+  context: ->(record) { "user_#{record.user.id}" }
+```
+
+The proc must take a single argument for the record.
+
+#### Specifying a default value
+
+An attribute can specify a default value, which will be set on initialization (`.new`) or after loading the value from the database. The default will be set if the value is `nil`.
+
+```ruby
+vault_attribute :access_level,
+  default: "readonly"
+
+vault_attribute :metadata,
+  serialize: :json,
+  default: {}
+```
+
 #### Specifying a different Vault path
+
 By default, the path to the transit backend in Vault is `transit/`. This is customizable by setting the `:path` option when declaring the attribute:
 
 ```ruby
@@ -109,16 +182,72 @@ vault_attribute :credit_card,
 
 - **Note** Changing this value for an existing application will make existing values no longer decryptable!
 
-#### Automatic serializing
+#### Lazy attribute decryption
+By default, `vault-rails` will decrypt a record’s encrypted attributes on that record’s initialization. You can configure an encrypted model to decrypt attributes lazily, which will prevent communication with Vault until an encrypted attribute’s getter method is called, at which point all of the record’s encrypted attributes will be decrypted. This is useful if you do not always need access to encrypted attributes. For example:
+
+
+```ruby
+class Person < ActiveRecord::Base
+  include Vault::EncryptedModel
+  vault_lazy_decrypt!
+
+  vault_attribute :ssn
+end
+
+# Without vault_lazy_decrypt:
+person = Person.find(id) # Vault communication happens here
+person.ssn
+# => "123-45-6789"
+
+# With vault_lazy_decrypt:
+person = Person.find(id)
+person.ssn # Vault communication happens here
+# => "123-45-6789"
+```
+
+#### Single, lazy attribute decryption
+By default, `vault-rails` will decrypt all encrypted attributes on that record’s initialization on a class by class basis. You can configure an encrypted model to decrypt attributes lazily and and individually. This will prevent vault from loading all vault_attributes defined on a class the moment one attribute is requested.
+
+
+```ruby
+class Person < ActiveRecord::Base
+  include Vault::EncryptedModel
+  vault_lazy_decrypt!
+  vault_single_decrypt!
+
+  vault_attribute :ssn
+  vault_attribute :email
+end
+
+# Without vault_single_decrypt:
+person = Person.find(id) # Vault communication happens here
+person.ssn # Vault communication happens here, fetches both ssn and email
+# => "123-45-6789"
+
+# With vault_single_decrypt:
+person = Person.find(id)
+person.ssn # Vault communication happens here, fetches only ssn
+# => "123-45-6789"
+person.email # Vault communication happens here, fetches only email
+# => "foobar@baz.com"
+```
+
+#### Serialization
+
 By default, all values are assumed to be "text" fields in the database. Sometimes it is beneficial for your application to work with a more flexible data structure (such as a Hash or Array). Vault-rails can automatically serialize and deserialize these structures for you:
 
 ```ruby
-vault_attribute :details
-  serialize: :json
+vault_attribute :details,
+  serialize: :json,
+  default: {}
 ```
 
+It is recommended to set a default matching type that you're serializing.
+
 - **Note** You can view the source for the exact serialization and deserialization options, but they are intentionally not customizable and cannot be used for a full object marshal/unmarshal.
 
+##### Custom Serializers
+
 For customized solutions, you can also pass a module to the `:serializer` key. This module must have the following API:
 
 ```ruby
@@ -151,10 +280,12 @@ vault_attribute :address,
   decode: ->(raw) { raw.to_s }
 ```
 
-- **Note** Changing the algorithm for encoding/decoding for an existing application will probably make the application crash when attempting to retrive existing values!
+- **Note** Changing the algorithm for encoding/decoding for an existing application will probably make the application crash when attempting to retrieve existing values!
 
 Caveats
 -------
+↥ [back to top](#table-of-contents)
+
 
 ### Mounting/Creating Keys in Vault
 The Vault Rails plugin does not automatically mount a backend. It is assumed the proper backend is mounted and accessible by the given token. You can mount a transit backend like this:
@@ -217,6 +348,8 @@ the security model).
 
 Development
 -----------
+↥ [back to top](#table-of-contents)
+
 1. Clone the project on GitHub
 2. Create a feature branch
 3. Submit a Pull Request

data/Rakefile

@@ -11,6 +11,9 @@ Bundler::GemHelper.install_tasks
 APP_RAKEFILE = File.expand_path("../spec/dummy/Rakefile", __FILE__)
 load "rails/tasks/engine.rake"
 
-require "rspec/core/rake_task"
-RSpec::Core::RakeTask.new(:spec)
 task default: :spec
+
+require "rspec/core/rake_task"
+RSpec::Core::RakeTask.new(:spec) do |t|
+  puts "\n==> Testing with Rails #{Rails::VERSION::STRING} and Ruby #{RUBY_VERSION} <==\n"
+end

data/lib/vault/encrypted_model.rb

@@ -29,52 +29,55 @@ module Vault
       #   the path to the transit backend (default: +transit+)
       # @option options [String] :key
       #   the name of the encryption key (default: +#{app}_#{table}_#{column}+)
+      # @option options [String, Symbol, Proc] :context
+      #   either a string context, or a symbol or proc used to generate a
+      #   context for key generation
+      # @option options [Object] :default
+      #   a default value for this attribute to be set to if the underlying
+      #   value is nil
       # @option options [Symbol, Class] :serializer
       #   the name of the serializer to use (or a class)
       # @option options [Proc] :encode
       #   a proc to encode the value with
       # @option options [Proc] :decode
       #   a proc to decode the value with
+      # @option options [Hash] :transform_secret
+      #   a hash providing details about the transformation to use,
+      #   this includes the name, and the role to use
       def vault_attribute(attribute, options = {})
-        encrypted_column = options[:encrypted_column] || "#{attribute}_encrypted"
-        path = options[:path] || "transit"
-        key = options[:key] || "#{Vault::Rails.application}_#{table_name}_#{attribute}"
-
         # Sanity check options!
         _vault_validate_options!(options)
 
-        # Get the serializer if one was given.
-        serializer = options[:serialize]
+        parsed_opts = if options[:transform_secret]
+                        parse_transform_secret_attributes(attribute, options)
+                      else
+                        parse_transit_attributes(attribute, options)
+                      end
+        parsed_opts[:encrypted_column] = options[:encrypted_column] || "#{attribute}_encrypted"
 
-        # Unless a class or module was given, construct our serializer. (Slass
-        # is a subset of Module).
-        if serializer && !serializer.is_a?(Module)
-          serializer = Vault::Rails.serializer_for(serializer)
-        end
+        # Make a note of this attribute so we can use it in the future (maybe).
+        __vault_attributes[attribute.to_sym] = parsed_opts
 
-        # See if custom encoding or decoding options were given.
-        if options[:encode] && options[:decode]
-          serializer = Class.new
-          serializer.define_singleton_method(:encode, &options[:encode])
-          serializer.define_singleton_method(:decode, &options[:decode])
-        end
+        self.attribute attribute.to_s, ActiveRecord::Type::Value.new,
+          default: nil
 
         # Getter
         define_method("#{attribute}") do
-          self.__vault_load_attributes! unless @__vault_loaded
-          instance_variable_get("@#{attribute}")
+          self.__vault_load_attributes!(attribute) unless @__vault_loaded
+          super()
         end
 
         # Setter
         define_method("#{attribute}=") do |value|
-          self.__vault_load_attributes! unless @__vault_loaded
+          self.__vault_load_attributes!(attribute) unless @__vault_loaded
 
           # We always set it as changed without comparing with the current value
           # because we allow our held values to be mutated, so we need to assume
-          # that if you call attr=, you want it send back regardless.
+          # that if you call attr=, you want it sent back regardless.
 
           attribute_will_change!("#{attribute}")
           instance_variable_set("@#{attribute}", value)
+          super(value)
 
           # Return the value to be consistent with other AR methods.
           value
@@ -82,37 +85,10 @@ module Vault
 
         # Checker
         define_method("#{attribute}?") do
-          self.__vault_load_attributes! unless @__vault_loaded
+          self.__vault_load_attributes!(attribute) unless @__vault_loaded
           instance_variable_get("@#{attribute}").present?
         end
 
-        # Dirty method
-        define_method("#{attribute}_change") do
-          changes["#{attribute}"]
-        end
-
-        # Dirty method
-        define_method("#{attribute}_changed?") do
-          changed.include?("#{attribute}")
-        end
-
-        # Dirty method
-        define_method("#{attribute}_was") do
-          if changes["#{attribute}"]
-            changes["#{attribute}"][0]
-          else
-            public_send("#{attribute}")
-          end
-        end
-
-        # Make a note of this attribute so we can use it in the future (maybe).
-        __vault_attributes[attribute.to_sym] = {
-          key: key,
-          path: path,
-          serializer: serializer,
-          encrypted_column: encrypted_column,
-        }
-
         self
       end
 
@@ -131,6 +107,11 @@ module Vault
             raise Vault::Rails::ValidationFailedError, "Cannot use a " \
               "custom encoder/decoder if a `:serializer' is specified!"
           end
+
+          if options[:transform_secret]
+            raise Vault::Rails::ValidationFailedError, "Cannot use the " \
+              "transform secrets engine with a specified `:serializer'!"
+          end
         end
 
         if options[:encode] && !options[:decode]
@@ -142,6 +123,19 @@ module Vault
           raise Vault::Rails::ValidationFailedError, "Cannot specify " \
             "`:decode' without specifying `:encode' as well!"
         end
+
+        if context = options[:context]
+          if context.is_a?(Proc) && context.arity != 1
+            raise Vault::Rails::ValidationFailedError, "Proc passed to " \
+              "`:context' must take 1 argument!"
+          end
+        end
+        if transform_opts = options[:transform_secret]
+          if !transform_opts[:transformation]
+            raise Vault::Rails::VaildationFailedError, "Transform Secrets " \
+              "requires a transformation name!"
+          end
+        end
       end
 
       def vault_lazy_decrypt
@@ -151,6 +145,62 @@ module Vault
       def vault_lazy_decrypt!
         @vault_lazy_decrypt = true
       end
+
+      def vault_single_decrypt
+        @vault_single_decrypt ||= false
+      end
+
+      def vault_single_decrypt!
+        @vault_single_decrypt = true
+      end
+
+      private
+
+      def parse_transform_secret_attributes(attribute, options)
+        opts = {}
+        opts[:transform_secret] = true
+
+        serializer = Class.new
+        serializer.define_singleton_method(:encode) do |raw|
+          return if raw.nil?
+          resp = Vault::Rails.transform_encode(raw, options[:transform_secret])
+          resp.dig(:data, :encoded_value)
+        end
+        serializer.define_singleton_method(:decode) do |raw|
+          return if raw.nil?
+          resp = Vault::Rails.transform_decode(raw, options[:transform_secret])
+          resp.dig(:data, :decoded_value)
+        end
+        opts[:serializer] = serializer
+        opts
+      end
+
+      def parse_transit_attributes(attribute, options)
+        opts = {}
+        opts[:path] = options[:path] || "transit"
+        opts[:key] = options[:key] || "#{Vault::Rails.application}_#{table_name}_#{attribute}"
+        opts[:context] = options[:context]
+        opts[:default] = options[:default]
+
+        # Get the serializer if one was given.
+        serializer = options[:serialize]
+
+        # Unless a class or module was given, construct our serializer. (Slass
+        # is a subset of Module).
+        if serializer && !serializer.is_a?(Module)
+          serializer = Vault::Rails.serializer_for(serializer)
+        end
+
+        # See if custom encoding or decoding options were given.
+        if options[:encode] && options[:decode]
+          serializer = Class.new
+          serializer.define_singleton_method(:encode, &options[:encode])
+          serializer.define_singleton_method(:decode, &options[:decode])
+        end
+
+        opts[:serializer] = serializer
+        opts
+      end
     end
 
     included do
@@ -166,6 +216,12 @@ module Vault
       # before theirs, resulting in attributes that are not persisted.
       after_save :__vault_persist_attributes!
 
+      # Before destroying a record, ensure that all vault attributes have been
+      # decrypted. Otherwise, attempting to read a lazily decrypted attribute
+      # after a destroy will result in a "Can't modify frozen Hash" error when
+      # vault-rails attempts to set the plain-text attribute.
+      before_destroy :__vault_load_attributes!, unless: -> { @__vault_loaded }
+
       # Decrypt all the attributes from Vault.
       # @return [true]
       def __vault_initialize_attributes!
@@ -177,12 +233,16 @@ module Vault
         __vault_load_attributes!
       end
 
-      def __vault_load_attributes!
+      def __vault_load_attributes!(attribute_to_read = nil)
         self.class.__vault_attributes.each do |attribute, options|
+          # skip loading certain keys in one of two cases:
+          # 1- the attribute has already been loaded
+          # 2- the single decrypt option is set AND this is not the attribute we're requesting to decrypt
+          next if instance_variable_get("@#{attribute}") || (self.class.vault_single_decrypt && attribute_to_read != attribute)
           self.__vault_load_attribute!(attribute, options)
         end
 
-        @__vault_loaded = true
+        @__vault_loaded = self.class.__vault_attributes.all? { |attribute, __| instance_variable_defined?("@#{attribute}") }
 
         return true
       end
@@ -193,26 +253,48 @@ module Vault
         path       = options[:path]
         serializer = options[:serializer]
         column     = options[:encrypted_column]
+        context    = options[:context]
+        default    = options[:default]
+        transform  = options[:transform_secret]
 
         # Load the ciphertext
         ciphertext = read_attribute(column)
 
         # If the user provided a value for the attribute, do not try to load
         # it from Vault
-        if instance_variable_get("@#{attribute}")
+        if attributes[attribute.to_s]
           return
         end
 
-        # Load the plaintext value
-        plaintext = Vault::Rails.decrypt(path, key, ciphertext)
+        # Generate context if needed
+        generated_context = __vault_generate_context(context)
+
+        if transform
+          # If this is a secret encrypted with FPE, we do not need to decrypt with vault
+          # This prevents a double encryption via standard vault encryption and FPE.
+          # FPE is decrypted later as part of the serializer
+          plaintext = ciphertext
+        else
+          # Load the plaintext value
+          plaintext = Vault::Rails.decrypt(
+            path, key, ciphertext,
+            context: generated_context
+          )
+        end
 
         # Deserialize the plaintext value, if a serializer exists
         if serializer
           plaintext = serializer.decode(plaintext)
         end
 
+        # Set to default if needed
+        if default && plaintext == nil
+          plaintext = default
+        end
+
         # Write the virtual attribute with the plaintext value
         instance_variable_set("@#{attribute}", plaintext)
+        @attributes.write_from_database attribute.to_s, plaintext
       end
 
       # Encrypt all the attributes using Vault and set the encrypted values back
@@ -244,23 +326,44 @@ module Vault
         path       = options[:path]
         serializer = options[:serializer]
         column     = options[:encrypted_column]
+        context    = options[:context]
+        transform  = options[:transform_secret]
 
         # Only persist changed attributes to minimize requests - this helps
         # minimize the number of requests to Vault.
-        if !changed.include?("#{attribute}")
-          return
+        if ActiveRecord.gem_version >= Gem::Version.new("6.0")
+          return unless previous_changes.include?(attribute)
+        elsif ActiveRecord.gem_version >= Gem::Version.new("5.2")
+          return unless previous_changes_include?(attribute)
+        elsif ActiveRecord.gem_version >= Gem::Version.new("5.1")
+          return unless saved_change_to_attribute?(attribute.to_s)
+        else
+          return unless attribute_changed?(attribute)
         end
 
         # Get the current value of the plaintext attribute
-        plaintext = instance_variable_get("@#{attribute}")
+        plaintext = attributes[attribute.to_s]
 
         # Apply the serialize to the plaintext value, if one exists
         if serializer
           plaintext = serializer.encode(plaintext)
         end
 
-        # Generate the ciphertext and store it back as an attribute
-        ciphertext = Vault::Rails.encrypt(path, key, plaintext)
+        # Generate context if needed
+        generated_context = __vault_generate_context(context)
+
+        if transform
+          # If this is a secret encrypted with FPE, we should not encrypt it in vault
+          # This prevents a double encryption via standard vault encryption and FPE.
+          # FPE was performed earlier as part of the serialization process.
+          ciphertext = plaintext
+        else
+          # Generate the ciphertext and store it back as an attribute
+          ciphertext = Vault::Rails.encrypt(
+            path, key, plaintext,
+            context: generated_context
+          )
+        end
 
         # Write the attribute back, so that we don't have to reload the record
         # to get the ciphertext
@@ -270,6 +373,20 @@ module Vault
         { column => ciphertext }
       end
 
+      # Generates an Vault Transit encryption context for use on derived keys.
+      def __vault_generate_context(context)
+        case context
+        when String
+          context
+        when Symbol
+          send(context)
+        when Proc
+          context.call(self)
+        else
+          nil
+        end
+      end
+
       # Override the reload method to reload the Vault attributes. This will
       # ensure that we always have the most recent data from Vault when we
       # reload a record from the database.
@@ -279,6 +396,7 @@ module Vault
           # from Vault
           self.class.__vault_attributes.each do |attribute, _|
             self.instance_variable_set("@#{attribute}", nil)
+            @attributes.write_from_database attribute.to_s, nil
           end
 
           self.__vault_initialize_attributes!