kms_encrypted 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 81d18dbb1dbaa5711a97eef2a4ece83fa44e2c2c
-  data.tar.gz: f7f8beede414a85f9ca399d8f499ff0f48b2bf53
+SHA256:
+  metadata.gz: bec6f5799315832f67c01e6ffe79b58a5c5019e3ca8efa5a2c0795149a2b727c
+  data.tar.gz: 8cbea27c197f2da4dbd1cad3bbee59c36c2404cd8eddb379c07be7ebbfc72a05
 SHA512:
-  metadata.gz: 3db72cc8666e461acedc3842829cc17e7ab0d32820eb9729e8fc1682dded9bed440f6d1201a28dacf5fdbcd350c6eaaa0035fe13274b37102a749cc8dd2c1a9f
-  data.tar.gz: 2964f2377c884927a6dc31ea1d48aa24d3e9ac6f4b70924da5fec5d05176bb98c715357f1d4032001b6f49485c63ddcf1550e4b5d271b05507e295bc6f52b9a4
+  metadata.gz: ba30a61a50245321fcca825ce5d36a4f350edb26b832f033d029b6051578a1f6d3ac8efb81f01339762a1a6869f6e70ca4a4fc780d3341ea80b4d17ca57724e8
+  data.tar.gz: e511299714c5d035f000d7e85de6855be813fccc0a65fe7332a9418e0dc2733c6618ae42070f582877b48db98117aabc9b605ab1931537885180b20693fb7da0

data/.travis.yml

@@ -5,8 +5,6 @@ gemfile:
 sudo: false
 before_install: gem install bundler
 script: bundle exec rake test
-env:
-  - KMS_KEY_ID=insecure-test-key
 notifications:
   email:
     on_success: never

data/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.3.0
+
+- Added support for Vault
+- Removed `KmsEncrypted.kms_client` and `KmsEncrypted.client_options` in favor of `KmsEncrypted.aws_client`
+- Removed `KmsEncrypted::Google.kms_client` in favor of `KmsEncrypted.google_client`
+
 ## 0.2.0
 
 - Added support for Google KMS

data/README.md

@@ -11,7 +11,7 @@ The attr_encrypted gem is great for encryption, but:
 
 Key management services address all of these issues and it’s easy to use them together.
 
-Supports [Amazon KMS](https://aws.amazon.com/kms/) and [Google KMS](https://cloud.google.com/kms/)
+Supports [Amazon KMS](https://aws.amazon.com/kms/), [Google KMS](https://cloud.google.com/kms/), and [Vault](https://www.vaultproject.io/)
 
 [![Build Status](https://travis-ci.org/ankane/kms_encrypted.svg?branch=master)](https://travis-ci.org/ankane/kms_encrypted)
 
@@ -29,6 +29,11 @@ Follow the instructions for your key management service:
 
 - [Amazon KMS](guides/Amazon.md)
 - [Google KMS](guides/Google.md)
+- [Vault](guides/Vault.md)
+
+## Related Projects
+
+To securely search encrypted data, check out [Blind Index](https://github.com/ankane/blind_index).
 
 ## History
 

data/guides/Amazon.md

@@ -57,7 +57,7 @@ class User < ApplicationRecord
 end
 ```
 
-The context is used as part of the encryption and decryption process, so it must be a value that doesn’t change. Otherwise, you won’t be able to decrypt. Read more about [encryption context here](http://docs.aws.amazon.com/kms/latest/developerguide/encryption-context.html).
+The context is used as part of the encryption and decryption process, so it must be a value that doesn’t change. Otherwise, you won’t be able to decrypt. Read more about [encryption context here](https://docs.aws.amazon.com/kms/latest/developerguide/encryption-context.html).
 
 The primary key is a good choice, but auto-generated ids aren’t available until a record is created, and we need to encrypt before this. One solution is to preload the primary key. Here’s what it looks like with Postgres:
 
@@ -147,7 +147,7 @@ We recommend setting up alerts on suspicious behavior.
 
 ## Key Rotation
 
-KMS supports [automatic key rotation](http://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html). No action is required in this case.
+KMS supports [automatic key rotation](https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html). No action is required in this case.
 
 To manually rotate keys, replace the old KMS key id with the new key id in your model. Your app does not need the old key id to perform rotation (however, the key must still be enabled in your AWS account).
 
@@ -259,4 +259,4 @@ user.rotate_kms_key_phone!
 
 ## File Uploads
 
-While outside the scope of this gem, you can also use KMS for sensitive file uploads. Check out [this guide](https://github.com/ankane/shorts/blob/master/AWS-Client-Side-Encryption.md) to learn more.
+While outside the scope of this gem, you can also use KMS for sensitive file uploads. Check out [this guide](https://ankane.org/aws-client-side-encryption) to learn more.

data/guides/Google.md

@@ -4,7 +4,7 @@ Add this line to your application’s Gemfile:
 
 ```ruby
 gem 'google-api-client'
-gem 'kms_encrypted', github: 'ankane/kms_encrypted'
+gem 'kms_encrypted'
 ```
 
 Add columns for the encrypted data and the encrypted KMS data keys

data/guides/Vault.md

@@ -0,0 +1,143 @@
+# Vault
+
+Add this line to your application’s Gemfile:
+
+```ruby
+gem 'vault'
+gem 'kms_encrypted'
+```
+
+Add columns for the encrypted data and the encrypted KMS data keys
+
+```ruby
+add_column :users, :encrypted_email, :text
+add_column :users, :encrypted_email_iv, :text
+add_column :users, :encrypted_kms_key, :text
+```
+
+Enable the [transit](https://www.vaultproject.io/docs/secrets/transit/index.html) backend
+
+```sh
+vault secrets enable transit
+```
+
+And create a key
+
+```sh
+vault write -f transit/keys/my-key
+```
+
+Set it in your environment along with your Vault credentials ([dotenv](https://github.com/bkeepers/dotenv) is great for this)
+
+```sh
+KMS_KEY_ID=vault/my-key
+VAULT_ADDR=http://127.0.0.1:8200
+VAULT_TOKEN=secret
+```
+
+And update your model
+
+```ruby
+class User < ApplicationRecord
+  has_kms_key
+
+  attr_encrypted :email, key: :kms_key
+end
+```
+
+For each encrypted attribute, use the `kms_key` method for its key.
+
+## Auditing
+
+Follow the [instructions here](https://www.vaultproject.io/docs/audit/) to set up data access logging. To know what data is being decrypted, you’ll need to add context.
+
+Add a `kms_encryption_context` method to your model.
+
+```ruby
+class User < ApplicationRecord
+  def kms_encryption_context
+    # some hash
+  end
+end
+```
+
+The context is used as part of the encryption and decryption process, so it must be a value that doesn’t change. Otherwise, you won’t be able to decrypt. Read more about [encryption context here](https://docs.aws.amazon.com/kms/latest/developerguide/encryption-context.html).
+
+The primary key is a good choice, but auto-generated ids aren’t available until a record is created, and we need to encrypt before this. One solution is to preload the primary key. Here’s what it looks like with Postgres:
+
+```ruby
+class User < ApplicationRecord
+  def kms_encryption_context
+    self.id ||= self.class.connection.execute("select nextval('#{self.class.sequence_name}')").first["nextval"]
+    {"Record" => "#{model_name}/#{id}"}
+  end
+end
+```
+
+## Alerting
+
+We recommend setting up alerts on suspicious behavior.
+
+## Key Rotation
+
+To manually rotate keys, replace the old key id with the new key id in your model.
+
+```sh
+KMS_KEY_ID=...
+```
+
+and run
+
+```ruby
+User.find_each do |user|
+  user.rotate_kms_key!
+end
+```
+
+## Testing
+
+For testing, you can prevent network calls to KMS by setting:
+
+```sh
+KMS_KEY_ID=insecure-test-key
+```
+
+## Multiple Keys Per Record
+
+You may want to protect different columns with different data keys (or even master keys).
+
+To do this, add more columns
+
+```ruby
+add_column :users, :encrypted_phone, :text
+add_column :users, :encrypted_phone_iv, :text
+add_column :users, :encrypted_kms_key_phone, :text
+```
+
+And update your model
+
+```ruby
+class User < ApplicationRecord
+  has_kms_key
+  has_kms_key name: :phone, key_id: "..."
+
+  attr_encrypted :email, key: :kms_key
+  attr_encrypted :phone, key: :kms_key_phone
+end
+```
+
+For context, use:
+
+```ruby
+class User < ApplicationRecord
+  def kms_encryption_context_phone
+    # some hash
+  end
+end
+```
+
+To rotate keys, use:
+
+```ruby
+user.rotate_kms_key_phone!
+```

data/kms_encrypted.gemspec

@@ -30,4 +30,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "attr_encrypted"
   spec.add_development_dependency "aws-sdk-kms"
   spec.add_development_dependency "google-api-client"
+  spec.add_development_dependency "vault"
 end

data/lib/kms_encrypted/model.rb

@@ -53,22 +53,36 @@ module KmsEncrypted
                   plaintext_key = OpenSSL::Random.random_bytes(32)
 
                   # encrypt it
+                  # load client first to ensure namespace is loaded
+                  client = KmsEncrypted.google_client
                   request = ::Google::Apis::CloudkmsV1::EncryptRequest.new(
                     plaintext: plaintext_key,
                     additional_authenticated_data: context.to_json
                   )
-                  response = KmsEncrypted::Google.kms_client.encrypt_crypto_key(key_id, request)
+                  response = client.encrypt_crypto_key(key_id, request)
                   key_version = response.name
 
                   # shorten key to save space
-                  short_key_id = Base64.encode64(key_version.split("/").select.with_index { |p, i| i.odd? }.join("/"))
+                  short_key_id = Base64.encode64(key_version.split("/").select.with_index { |_, i| i.odd? }.join("/"))
 
                   # build encrypted key
                   # we reference the key in the field for easy rotation
                   encrypted_key = "$gc$#{short_key_id}$#{[response.ciphertext].pack(default_encoding)}"
+                elsif key_id.start_with?("vault/")
+                  # generate random AES-256 key
+                  plaintext_key = OpenSSL::Random.random_bytes(32)
+
+                  # encrypt it
+                  response = KmsEncrypted.vault_client.logical.write(
+                    "transit/encrypt/#{key_id.sub("vault/", "")}",
+                    plaintext: Base64.encode64(plaintext_key),
+                    context: Base64.encode64(context.to_json)
+                  )
+
+                  encrypted_key = response.data[:ciphertext]
                 else
                   # generate data key from API
-                  resp = KmsEncrypted.kms_client.generate_data_key(
+                  resp = KmsEncrypted.aws_client.generate_data_key(
                     key_id: key_id,
                     encryption_context: context,
                     key_spec: "AES_256"
@@ -92,7 +106,7 @@ module KmsEncrypted
               }
               ActiveSupport::Notifications.instrument("decrypt_data_key.kms_encrypted", event) do
                 if encrypted_key.start_with?("insecure-data-key-")
-                  plaintext_key = "00000000000000000000000000000000"
+                  plaintext_key = "00000000000000000000000000000000".encode("BINARY")
                 elsif encrypted_key.start_with?("$gc$")
                   _, _, short_key_id, ciphertext = encrypted_key.split("$", 4)
 
@@ -104,13 +118,23 @@ module KmsEncrypted
                   stored_key_id.insert(6, "cryptoKeys")
                   stored_key_id = stored_key_id.join("/")
 
+                  # load client first to ensure namespace is loaded
+                  client = KmsEncrypted.google_client
                   request = ::Google::Apis::CloudkmsV1::DecryptRequest.new(
                     ciphertext: ciphertext.unpack(default_encoding).first,
                     additional_authenticated_data: context.to_json
                   )
-                  plaintext_key = KmsEncrypted::Google.kms_client.decrypt_crypto_key(stored_key_id, request).plaintext
+                  plaintext_key = client.decrypt_crypto_key(stored_key_id, request).plaintext
+                elsif encrypted_key.start_with?("vault:")
+                  response = KmsEncrypted.vault_client.logical.write(
+                    "transit/decrypt/#{key_id.sub("vault/", "")}",
+                    ciphertext: encrypted_key,
+                    context: Base64.encode64(context.to_json)
+                  )
+
+                  plaintext_key = Base64.decode64(response.data[:plaintext])
                 else
-                  plaintext_key = KmsEncrypted.kms_client.decrypt(
+                  plaintext_key = KmsEncrypted.aws_client.decrypt(
                     ciphertext_blob: encrypted_key.unpack(default_encoding).first,
                     encryption_context: context
                   ).plaintext

data/lib/kms_encrypted/version.rb

@@ -1,3 +1,3 @@
 module KmsEncrypted
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/kms_encrypted.rb

@@ -1,24 +1,6 @@
 # dependencies
 require "active_support"
 
-begin
-  # aws-sdk v3
-  require "aws-sdk-kms"
-rescue LoadError
-  begin
-    # aws-sdk v2
-    require "aws-sdk"
-  rescue LoadError
-    # do nothing
-  end
-end
-
-begin
-  require "google/apis/cloudkms_v1"
-rescue LoadError
-  # do nothing
-end
-
 # modules
 require "kms_encrypted/log_subscriber"
 require "kms_encrypted/model"
@@ -26,43 +8,31 @@ require "kms_encrypted/version"
 
 module KmsEncrypted
   class << self
-    attr_writer :kms_client
-
-    def kms_client
-      @kms_client ||= Aws::KMS::Client.new(client_options)
+    attr_writer :aws_client
+    attr_writer :google_client
+    attr_writer :vault_client
+
+    def aws_client
+      @aws_client ||= Aws::KMS::Client.new(
+        retry_limit: 2,
+        http_open_timeout: 2,
+        http_read_timeout: 2
+      )
     end
-    alias_method :kms, :kms_client
 
-    # deprecated, use kms_client instead
-    attr_reader :client_options
-
-    # deprecated, use kms_client instead
-    def client_options=(value)
-      @client_options = value
-      @kms_client = nil
+    def google_client
+      @google_client ||= begin
+        require "google/apis/cloudkms_v1"
+        client = ::Google::Apis::CloudkmsV1::CloudKMSService.new
+        client.authorization = ::Google::Auth.get_application_default(
+          "https://www.googleapis.com/auth/cloud-platform"
+        )
+        client
+      end
     end
-  end
 
-  # deprecated, use kms_client instead
-  self.client_options = {
-    retry_limit: 2,
-    http_open_timeout: 2,
-    http_read_timeout: 2
-  }
-
-  module Google
-    class << self
-      attr_writer :kms_client
-
-      def kms_client
-        @kms_client ||= begin
-          client = ::Google::Apis::CloudkmsV1::CloudKMSService.new
-          client.authorization = ::Google::Auth.get_application_default(
-            "https://www.googleapis.com/auth/cloud-platform"
-          )
-          client
-        end
-      end
+    def vault_client
+      @vault_client ||= ::Vault
     end
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kms_encrypted
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Andrew Kane
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-02-23 00:00:00.000000000 Z
+date: 2018-11-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -136,6 +136,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: vault
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: 
 email:
 - andrew@chartkick.com
@@ -152,6 +166,7 @@ files:
 - Rakefile
 - guides/Amazon.md
 - guides/Google.md
+- guides/Vault.md
 - kms_encrypted.gemspec
 - lib/kms_encrypted.rb
 - lib/kms_encrypted/log_subscriber.rb
@@ -177,7 +192,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.13
+rubygems_version: 2.7.7
 signing_key: 
 specification_version: 4
 summary: Simple, secure key management for attr_encrypted