sidekiq-encrypted_args 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 3d478d88a4ebaca82ad336b50e60ceebbdf9da240c4c830b3d8bc8ddb6c972d1
+  data.tar.gz: 900d93f22766a04db7ab649b2ff7318a2dbb89cb76dd61a732f8e87807308473
+SHA512:
+  metadata.gz: ee85740768b20d08a6caf000f84bbe94c9badc46de5637b753f23420ed299b82cccb969d8a87195e9be8c4323d3dcecbe38a2c71a458eaf4864a4932e2b43362
+  data.tar.gz: 4f7c7a5894368925e2d59cdcea8eeba6da98f2bf6c3a2cfaf24e99fea3903e203e0fab3039a338236582ae74bef6d114b70869572f1fcb5df6f0eecdd1d1b45e

data/CHANGE_LOG.md

@@ -0,0 +1,3 @@
+# 1.0.0
+
+* Initial release

data/MIT_LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2020 Brian Durand
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,106 @@
+# Sidekiq Encrypted Args
+
+![Continuous Integration](https://github.com/bdurand/sidekiq-encrypted_args/workflows/Continuous%20Integration/badge.svg?branch=master)
+[![Maintainability](https://api.codeclimate.com/v1/badges/70ab3782e4d5285eb173/maintainability)](https://codeclimate.com/github/bdurand/sidekiq-encrypted_args/maintainability)
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+
+Support for encrypting arguments for [Sidekiq](https://github.com/mperham/sidekiq).
+
+## Problem
+
+Sidekiq stores the arguments for jobs as JSON in Redis. If your workers include sensitive information (API keys, passwords, personally identifiable information, etc.), you run the risk of accidentally exposing this information. Job arguments are visible in the Sidekiq web interface and your security will only be as good as your Redis server security.
+
+This can be an even bigger issue if you use scheduled jobs since sensitive data on those jobs will live in Redis until the job is run. Data written to Redis can also be persisted to disk and live on long after the data in Redis has been deleted.
+
+## Solution
+
+This gem adds Sidekiq middleware that allows you to specify job arguments for your workers that should be encrypted in Redis. You do this by adding `encrypted_args` to the `sidekiq_options` in the worker. Jobs for these workers will have their arguments encrypted in Redis and decrypted when passed to `perform` method.
+
+To use the gem, you will need to specify a secret that will be used to encrypt the arguments as well as add the middleware to your Sidekiq client and server middleware stacks. You can set that up by adding this to the end of your Sidekiq initialization:
+
+```ruby
+Sidekiq::EncryptedArgs.configure!(secret: "YourSecretKey")
+```
+
+If the secret is not set, the value of the `SIDEKIQ_ENCRYPTED_ARGS_SECRET` environment variable will be used as the secret. If this variable is not set, job arguments will not be encrypted.
+
+The call to `Sidekiq::EncryptedArgs.configure!` will append the encryption middleware to the end of the client and server middleware chains. You add the middlewares manually if you need more control over where they appear in the stacks.
+
+```ruby
+Sidekiq::EncryptedArgs.secret = "YourSecretKey"
+
+Sidekiq.configure_client do |config|
+  config.client_middleware do |chain|
+    chain.add Sidekiq::EncryptedArgs::ClientMiddleware
+  end
+end
+
+Sidekiq.configure_server do |config|
+  config.server_middleware do |chain|
+    chain.add Sidekiq::EncryptedArgs::ServerMiddleware
+  end
+end
+```
+
+## Worker Configuration
+
+To declare that a worker is using encrypted arguments, you must set the `encrypted_args` sidekiq option.
+
+Setting the option to `true` will encrypt all the arguments passed to the `perform` method.
+
+```ruby
+class SecretWorker
+  include Sidekiq::Worker
+
+  sidekiq_options encrypted_args: true
+
+  def perform(arg_1, arg_2, arg_3)
+  end
+end
+```
+
+You can also encrypt just specific arguments with a hash or an array. This can be useful to preserve visibility into non-sensitive arguments that might be useful for troubleshooting or other reasons. All of these examples will encrypt just the second argument to the `perform` method.
+
+```ruby
+  # Pass in a list of argument names that should be encrypted
+  sidekiq_options encrypted_args: [:arg_2]
+
+  def perform(arg_1, arg_2, arg_3)
+  end
+```
+
+```ruby
+  # Pass in a hash with values indicating which arguments should be encrypted
+  sidekiq_options encrypted_args: { arg_2: true, arg_1: false }
+
+  def perform(arg_1, arg_2, arg_3)
+  end
+```
+
+```ruby
+  # Pass in an array of boolean values indicating which argument positions should be encrypted
+  sidekiq_options encrypted_args: [false, true]
+
+  def perform(arg_1, arg_2, arg_3)
+  end
+```
+
+You don't need to change anything else about your workers. All of the arguments passed to the `perform` method will already be unencrypted when the method is called.
+
+## Rolling Secrets
+
+If you need to roll your secret, you can simply provide an array when setting the secret.
+
+```ruby
+Sidekiq::EncryptedArgs.secret = ["CurrentSecret", "OldSecret"]
+```
+
+The left most key will be considered the current key and will be used for encrypting arguments. However, all of the keys will be tried when decrypting. This allows you to switch you secret keys without breaking jobs already enqueued in Redis.
+
+If you are using the `SIDEKIQ_ENCRYPTED_ARGS_SECRET` envrionment variable to specify your secret, you can delimit multiple keys with a spaces.
+
+You can also safely add encryption to an existing worker. Any jobs that are already enqueued will still run even without having the arguments encrypted in Redis.
+
+## Encryption
+
+Encrypted arguments are stored using AES-256-GCM with a key derived from your secret using PBKDF2.

data/VERSION

@@ -0,0 +1 @@
+1.0.0

data/benchmark.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+
+require "benchmark"
+require "sidekiq"
+require_relative "lib/sidekiq/encrypted_args"
+
+class WorkerWithoutEncryption
+  include Sidekiq::Worker
+
+  def perform(arg_1, arg_2, arg_3)
+  end
+end
+
+class WorkerWithEncryption
+  include Sidekiq::Worker
+
+  sidekiq_options encrypted_args: [true, true, true]
+
+  def perform(arg_1, arg_2, arg_3)
+  end
+end
+
+middleware = ->(worker_class) {
+  job = {"args" => ["foo", "bar", "baz"]}
+  Sidekiq::EncryptedArgs::ClientMiddleware.new.call(worker_class, job, "default") do
+    worker = worker_class.new
+    Sidekiq::EncryptedArgs::ServerMiddleware.new.call(worker, job, "default") do
+      worker.perform(*job["args"])
+    end
+  end
+}
+
+Benchmark.bm do |benchmark|
+  benchmark.report("No Encryption:  ") { 10000.times { middleware.call(WorkerWithoutEncryption) } }
+  benchmark.report("With Encryption:") { 10000.times { middleware.call(WorkerWithEncryption) } }
+end

data/lib/sidekiq-encrypted_args.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require_relative "sidekiq/encrypted_args"

data/lib/sidekiq/encrypted_args.rb

@@ -0,0 +1,160 @@
+# frozen_string_literal: true
+
+require "json"
+require "secret_keys"
+require "sidekiq"
+
+module Sidekiq
+  module EncryptedArgs
+    # Error thrown when the
+    class InvalidSecretError < StandardError
+    end
+
+    class << self
+      # Set the secret key used for encrypting arguments. If this is not set,
+      # the value will be loaded from the `SIDEKIQ_ENCRYPTED_ARGS_SECRET` environment
+      # variable. If that value is not set, arguments will not be encypted.
+      #
+      # @param [String] value One or more secrets to use for encypting arguments.
+      #
+      # @note You can set multiple secrets by passing an array if you need to roll your secrets.
+      # The left most value in the array will be used as the encryption secret, but
+      # all the values will be tried when decrypting. That way if you have scheduled
+      # jobs that were encypted with a different secret, you can still make it available
+      # when decrypting the arguments when the job gets run. If you are using the
+      # envrionment variable, separate the keys with spaces.
+      def secret=(value)
+        @encryptors = make_encryptors(value)
+      end
+
+      # Calling this method will add the client and server middleware to the Sidekiq
+      # middleware chains. If you need to ensure the order of where the middleware is
+      # added, you can forgo this method and add it yourself.
+      def configure!(secret: nil)
+        self.secret = secret unless secret.nil?
+
+        Sidekiq.configure_client do |config|
+          config.client_middleware do |chain|
+            chain.add Sidekiq::EncryptedArgs::ClientMiddleware
+          end
+        end
+
+        Sidekiq.configure_server do |config|
+          config.server_middleware do |chain|
+            chain.add Sidekiq::EncryptedArgs::ServerMiddleware
+          end
+        end
+      end
+
+      # Encrypt a value.
+      #
+      # @param [Object] data Data to encrypt. You can pass any JSON compatible data types or structures.
+      #
+      # @return [String]
+      def encrypt(data)
+        return nil if data.nil?
+        json = JSON.dump(data)
+        encrypted = encrypt_string(json)
+        if encrypted == json
+          data
+        else
+          encrypted
+        end
+      end
+
+      # Decrypt data
+      #
+      # @param [String] encrypted_data Data that was previously encrypted. If the value passed in is
+      # an unencrypted string, then the string itself will be returned.
+      #
+      # @return [String]
+      def decrypt(encrypted_data)
+        return encrypted_data unless SecretKeys::Encryptor.encrypted?(encrypted_data)
+        json = decrypt_string(encrypted_data)
+        JSON.parse(json)
+      end
+
+      protected
+
+      # Helper method to get the encrypted args option from an options hash. The value of this option
+      # can be `true` or an array indicating if each positional argument should be encrypted, or a hash
+      # with keys for the argument position and true as the value.
+      def encrypted_args_option(worker_class)
+        sidekiq_options = worker_class.sidekiq_options
+        option = sidekiq_options.fetch(:encrypted_args, sidekiq_options["encrypted_args"])
+
+        return nil if option.nil?
+
+        return Hash.new(true) if option == true
+
+        return replace_argument_positions(worker_class, option) if option.is_a?(Hash)
+
+        hash = {}
+        Array(option).each_with_index do |val, position|
+          if val.is_a?(Symbol) || val.is_a?(String)
+            hash[val] = true
+          else
+            hash[position] = val
+          end
+        end
+        replace_argument_positions(worker_class, hash)
+      end
+
+      private
+
+      # Hard coded password salt used sent to the encryptor. Do no change.
+      SALT = "3270e054"
+      private_constant :SALT
+
+      def encrypt_string(value)
+        encryptor = encryptors.first
+        return value if encryptor.nil?
+        encryptor.encrypt(value)
+      end
+
+      def decrypt_string(value)
+        return value if encryptors == [nil]
+        encryptors.each do |encryptor|
+          begin
+            return encryptor.decrypt(value) if encryptor
+          rescue OpenSSL::Cipher::CipherError
+            # Not the right key, try the next one
+          end
+        end
+        raise InvalidSecretError
+      end
+
+      def encryptors
+        if !defined?(@encryptors) || @encryptors.empty?
+          @encryptors = make_encryptors(ENV["SIDEKIQ_ENCRYPTED_ARGS_SECRET"].to_s.split)
+          if @encryptors.empty? && Sidekiq.logger
+            Sidekiq.logger.warn("#{self}: Secret not set for encrypting Sidekiq arguments; arguments will not be encrypted.")
+          end
+        end
+        @encryptors
+      end
+
+      def make_encryptors(secrets)
+        Array(secrets).map { |val| val.nil? ? nil : SecretKeys::Encryptor.from_password(val, SALT) }
+      end
+
+      def replace_argument_positions(worker_class, encrypt_option)
+        updated = {}
+        encrypt_option.each do |key, value|
+          if key.is_a?(Symbol) || key.is_a?(String)
+            key = key.to_sym
+            position = worker_class.instance_method(:perform).parameters.find_index { |_, name| name == key }
+            updated[position] = value if position
+          elsif key.is_a?(Integer)
+            updated[key] = value
+          end
+        end
+        updated
+      end
+    end
+  end
+end
+
+require_relative "encrypted_args/client_middleware"
+require_relative "encrypted_args/server_middleware"
+require_relative "encrypted_args/version"

data/lib/sidekiq/encrypted_args/client_middleware.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module EncryptedArgs
+    # Sidekiq client middleware for encrypting arguments on jobs for workers
+    # with `encrypted_args` set in the `sidekiq_options`.
+    class ClientMiddleware
+      def call(worker_class, job, queue, redis_pool = nil)
+        encrypted_args = EncryptedArgs.send(:encrypted_args_option, worker_class)
+        if encrypted_args
+          new_args = []
+          job["args"].each_with_index do |value, position|
+            value = EncryptedArgs.encrypt(value) if encrypted_args[position]
+            new_args << value
+          end
+          job["args"] = new_args
+        end
+
+        yield
+      end
+    end
+  end
+end

data/lib/sidekiq/encrypted_args/server_middleware.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module EncryptedArgs
+    class ServerMiddleware
+      # Sidekiq server middleware for encrypting arguments on jobs for workers
+      # with `encrypted_args` set in the `sidekiq_options`.
+      def call(worker, job, queue)
+        encrypted_args = EncryptedArgs.send(:encrypted_args_option, worker.class)
+        if encrypted_args
+          new_args = []
+          job["args"].each_with_index do |value, position|
+            value = EncryptedArgs.decrypt(value) if encrypted_args[position]
+            new_args << value
+          end
+          job["args"] = new_args
+        end
+
+        yield
+      end
+    end
+  end
+end

data/lib/sidekiq/encrypted_args/version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module EncryptedArgs
+    VERSION = File.read(File.join(__dir__, "..", "..", "..", "VERSION")).chomp.freeze
+  end
+end

data/sidekiq-encrypted_args.gemspec

@@ -0,0 +1,36 @@
+Gem::Specification.new do |spec|
+  spec.name = "sidekiq-encrypted_args"
+  spec.version = File.read(File.join(__dir__, "VERSION")).strip
+  spec.authors = ["Brian Durand", "Winston Durand"]
+  spec.email = ["bbdurand@gmail.com", "me@winstondurand.com"]
+
+  spec.summary = "Support for encrypting arguments that contain sensitive information in sidekiq jobs."
+  spec.homepage = "https://github.com/bdurand/sidekiq-encrypted_args"
+  spec.license = "MIT"
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  ignore_files = %w[
+    .
+    Appraisals
+    Gemfile
+    Gemfile.lock
+    Rakefile
+    gemfiles/
+    spec/
+  ]
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject { |f| ignore_files.any? { |path| f.start_with?(path) } }
+  end
+
+  spec.require_paths = ["lib"]
+  spec.bindir = "bin"
+  spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+
+  spec.required_ruby_version = ">= 2.4"
+
+  spec.add_dependency "sidekiq", ">= 4.0"
+  spec.add_dependency "secret_keys"
+
+  spec.add_development_dependency "bundler", "~>2.0"
+end

metadata

@@ -0,0 +1,99 @@
+--- !ruby/object:Gem::Specification
+name: sidekiq-encrypted_args
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Brian Durand
+- Winston Durand
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2020-06-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '4.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '4.0'
+- !ruby/object:Gem::Dependency
+  name: secret_keys
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+description: 
+email:
+- bbdurand@gmail.com
+- me@winstondurand.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGE_LOG.md
+- MIT_LICENSE.txt
+- README.md
+- VERSION
+- benchmark.rb
+- lib/sidekiq-encrypted_args.rb
+- lib/sidekiq/encrypted_args.rb
+- lib/sidekiq/encrypted_args/client_middleware.rb
+- lib/sidekiq/encrypted_args/server_middleware.rb
+- lib/sidekiq/encrypted_args/version.rb
+- sidekiq-encrypted_args.gemspec
+homepage: https://github.com/bdurand/sidekiq-encrypted_args
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '2.4'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.3
+signing_key: 
+specification_version: 4
+summary: Support for encrypting arguments that contain sensitive information in sidekiq
+  jobs.
+test_files: []