rails_surrogate_key_logging 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: df6c120ebe49e9724f0c2324feefd62dc61c317b87c118e438d3735364193897
-  data.tar.gz: a33a587bdc49075fcf20917797a9a631702ff4b52bd54f021e0088f383e36d3a
+  metadata.gz: 4f209a8f18098917e60dbd9dbe93a1d6367eeaf16e2da104bbbba0351fb09956
+  data.tar.gz: 0d70d2d6ed37c4981c286fbed4f409f397babd3695a90141a886d24b19ba2634
 SHA512:
-  metadata.gz: f5020c84352f1dc337ae18b2a8e4f6c8a576bea91ecdaf710fe9d0402fc31f82c6c865a23dee02269e45f7e3faa8e72208f8f64764357b765591a97ae8d5157f
-  data.tar.gz: e7c60aeebd180e87480c3e5857d94cd3926cb945545cc8c0a56f4bc5735dcd4c40b075aae1373e098a8186c3a641d5a3041582f88d42ae5740ebee6ed11c7abc
+  metadata.gz: c12692d51a77d61a606193aee29f77016f2386076c6334e35f8dfeb324a52f3caed7860a3ea83c44f2a91c861cd379eb8d2ed21002ace9732f5e9b8acfebe6fb
+  data.tar.gz: 969985b95d785c04f0c90042e57ce3bf2826d529adf96aac611994fa061a7c254175e4bcaa13eee4f39f18e95000bbd2ddaa268547d1317f6893d3e24a43d5db

data/README.md

@@ -1 +1,134 @@
 # rails-surrogate-key-logging
+
+This gem enhances and uses Rails' built-in `ParameterFilter` to add "Surrogate Key" logging.
+
+## Installation
+
+- Add `gem :rails_surrogate_key_logging` to your Gemfile.
+- Run `bin/bundle install`
+- Add `include SurrogateKeyLogging::ActionController::Params` to your `ApplicationController`
+- Add `include SurrogateKeyLogging::ActiveRecord::Attributes` to your `ApplicationRecord`
+
+
+
+## Configuration
+
+In a new application initializer (`config/initializers/surrogate_key_logging.rb`) or in your `config/environments/*.rb`, use the following block:
+
+```ruby
+SurrogateKeyLogging.configure do |config|
+  config.key = value
+end
+```
+
+### Config
+
+Key | Type | Default | Description
+---|---|---|---
+`enabled` | Boolean | `Rails.env.production?` | Whether surrogate logging is injected into Rails.
+`debug` | Boolean | `false` | Whether to log a statement showing that a surrogate replacement happened and what the mapping from surrogate to value, and logs from the key store (Such as queries made by ActiveRecord to it's Surrogate model).
+`key_prefix` | String | `''` | This string will be prepended to generated surrogates. Can make it easier to identify a surrogate in logs.
+`key_for` | Proc \| Lambda \| `responds_to?(:call)` | `-> (value) { "#{config.key_prefix}#{SecureRandom.uuid}" }` | The method used to generate a surrogate for a given value. While the `value` is supplied to the method, it is generally considered insecure for the surrogate to be derivable from it's value.
+`cache` | Boolean | `true` | Should the key mananger maintain an in-memory cache of value -> surrogate map that have been used. When in a server context, this cache will last for the lifetime of a single request. The cache can also be manually busted at any time by calling `SurrogateKeyLogging.reset!`.
+`cache_key_for` | Proc \| Lambda \| `responds_to?(:call)` | `-> (value) { value }` | The method used to create the keys used in the cache. Typically this should be left to the default unless you expect to make many surrogates for very large values.
+`key_ttl` | Integer | `90.days` | Used by `bin/rails skl:clear:stale` to delete old surrogates.
+`key_store` | Symbol | None | The key store to use. See [Key Stores](#key-stores).
+
+
+
+## Key Stores
+
+Key Store | Config Value
+---|---
+[ActiveRecord](#active-record) | `:active_record`
+
+### Active Record
+
+This will use a `SurrogateKeyLogging::Surrogate` model to manage surrogates. This will require adding `surrogate_key_logging_#{Rails.env}` to your application's `config/database.yml` See [Example](#example-database-yml). After configuring your `config/database.yml` you will need to run `bin/rails skl:key_store:active_record:db:create` and `bin/rails skl:key_store:active_record:db:migrate`.
+
+#### Example database.yml
+```yml
+default: &default
+  adapter: mysql2
+  username: <%= Rails.application.credentials.database[:username] %>
+  password: <%= Rails.application.credentials.database[:password] %>
+  host: 127.0.0.1
+  port: 3306
+  database: myapp_<%= Rails.env %>
+  prepared_statements: true
+
+surrogate_key_logging_default: &surrogate_key_logging_default
+  <<: *default
+  database: surrogate_keys_<%= Rails.env %>
+
+
+
+development:
+  <<: *default
+
+test:
+  <<: *default
+
+production:
+  <<: *default
+
+
+
+surrogate_key_logging_development:
+  <<: *surrogate_key_logging_default
+
+surrogate_key_logging_test:
+  <<: *surrogate_key_logging_default
+
+surrogate_key_logging_production:
+  <<: *surrogate_key_logging_default
+```
+
+
+
+## Usage
+
+### Controllers
+
+In any controller including `SurrogateKeyLogging::ActionController::Params` you may use the `surrogate_params(*params, action: '*')` method. This method may be used multiple times. Pass the `action` argument to limit those `params` to only that `action` or omit it to apply those `params` to ALL actions in that controller.
+
+#### Params format
+Param | Examples | Output
+---|---|---
+`:foo` | `{ foo: 'bar1', another: {foo: 'baz1'}, foobar: 'barbaz' }` | `{foo: SURROGATE, another: { foo: SURROGATE }, foobar: 'barbaz' }`
+`'another.foo'` | `{ foo: 'bar1', another: { foo: 'baz1' }, foobar: { another: { foo: 'barbaz' } } }` | `{ foo: 'bar1', another: { foo: SURROGATE }, foobar: { another: { foo: SURROGATE } } }`
+`'another[foo]'` | `{ foo: 'bar1', another: { foo: 'baz1' }, foobar: { another: { foo: 'barbaz' } } }` | `{ foo: 'bar1', another: { foo: SURROGATE }, foobar: { another: { foo: 'barbaz' } } }`
+
+#### Example Controller
+```ruby
+class WidgetsController < ApplicationController
+  surrogate_params :name
+  surrogate_params :owner, action: :search
+
+  def name
+    ...
+  end
+
+  def search
+    ...
+  end
+end
+```
+
+In this example the `name` parameter will be surrogated in all requests to this controller, and the `owner` parameter will surrogated only in requests to the `search` action.
+
+
+
+### Models
+
+In any controller including `SurrogateKeyLogging::ActiveRecord::Attributes` you may use `surrogate_parent_names(*names)` and `surrogate_attributes(*attrs)`. All permutations of parent names to attributes will be used to create filters. By default `surrogate_parent_names` is initialized with the singular and plural names of the model.
+
+#### Example Model
+```ruby
+class Widget < ApplicationRecord
+  surrogate_parent_names :things
+  surrogate_attributes :name, :owner
+end
+```
+
+In this example, the following filters will be used to look for attributes to be surrogated: `widget.name`, `widget[name]`, `[widget][name]`, `widgets.name`, `widgets[name]`, `[widgets][name]`, `things.name`, `things[name]`, `[things][name]`

data/lib/surrogate_key_logging/action_controller/params.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+require 'active_support/concern'
+
+module SurrogateKeyLogging
+  module ActionController
+    module Params
+      extend ActiveSupport::Concern
+
+      class_methods do
+        def surrogate_params(*params, action: '*')
+          @surrogate_params ||= ActiveSupport::HashWithIndifferentAccess.new {|h,k| h[k] = [] }
+          params.each do |param|
+            param = param.to_s
+            @surrogate_params[action] << param
+            if param.include?('.')
+              dots = param.split('.')
+              @surrogate_params[action] << [dots.first, dots[1..-1].map{|p| "[#{p}]"}].compact.join('')
+              @surrogate_params[action] << URI.encode_www_form_component(@surrogate_params[action].last)
+              @surrogate_params[action] << dots.map{|p| "[#{p}]"}.join('')
+              @surrogate_params[action] << URI.encode_www_form_component(@surrogate_params[action].last)
+            elsif param.include?('[') && param.include?(']')
+              @surrogate_params[action] << URI.encode_www_form_component(param)
+            else
+              @surrogate_params[action] << param
+            end
+          end
+          @surrogate_params
+        end
+      end
+
+    end
+  end
+end

data/lib/surrogate_key_logging/action_dispatch/params_filter.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module SurrogateKeyLogging
+  module ActionDispatch
+    class ParamsFilter
+
+      class << self
+        def call(params, req = nil)
+          new(params, req).filtered
+        end
+      end
+
+      attr_reader :params, :req
+      
+      def initialize(params, req = nil)
+        @params = params
+        @req = req
+      end
+
+      def filterable_params
+        @filterable_params ||= if req.controller_class.respond_to?(:surrogate_params)
+          surrogate_params = req.controller_class.surrogate_params
+          surrogate_params[params[:action]] + surrogate_params['*']
+        else
+          []
+        end
+      end
+
+      def params_filter
+        return @params_filter if @params_filter
+        attrs = SurrogateKeyLogging.parameter_filter.instance_variable_get(:@filters).dup
+        attrs += filterable_params
+        @params_filter = SurrogateKeyLogging.filter_for_attributes(attrs)
+      end
+
+      def filtered
+        @filtered ||= params_filter.filter params
+      end
+
+    end
+  end
+end

data/lib/surrogate_key_logging/action_dispatch/query_string_filter.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+module SurrogateKeyLogging
+  module ActionDispatch
+    class QueryStringFilter
+
+      class << self
+        def call(qs, req = nil)
+          new(qs, req).filtered
+        end
+      end
+
+      attr_reader :qs, :req
+      
+      def initialize(qs, req = nil)
+        @qs = qs
+        @req = req
+      end
+
+      def path_params
+        @path_params ||= begin
+          req.routes.recognize_path_with_request(req, req.path, {})
+        rescue
+          {}
+        end
+      end
+  
+      def controller_class
+        @controller_class = req.controller_class_for(path_params[:controller])
+      end
+  
+      def filterable_params
+        @filterable_params ||= if controller_class.respond_to?(:surrogate_params)
+          surrogate_params = controller_class.surrogate_params
+          surrogate_params[path_params[:action]] + surrogate_params['*']
+        else
+          []
+        end
+      end
+
+      def params_filter
+        return @params_filter if @params_filter
+        attrs = SurrogateKeyLogging.parameter_filter.instance_variable_get(:@filters).dup
+        attrs += filterable_params
+        @params_filter = SurrogateKeyLogging.filter_for_attributes(attrs)
+      end
+
+      def filtered
+        @filtered ||= qs.gsub(::ActionDispatch::Request::PAIR_RE) do |_|
+          params_filter.filter(::Regexp.last_match(1) => ::Regexp.last_match(2)).first.join('=')
+        end
+      end
+
+    end
+  end
+end

data/lib/surrogate_key_logging/action_dispatch/request.rb

@@ -4,10 +4,12 @@ module SurrogateKeyLogging
   module ActionDispatch
     module Request
 
+      def filtered_parameters
+        @filtered_parameters ||= ParamsFilter.call(super, self)
+      end
+
       def filtered_query_string
-        super.gsub(::ActionDispatch::Request::PAIR_RE) do |_|
-          SurrogateKeyLogging.filter_parameters(::Regexp.last_match(1) => ::Regexp.last_match(2)).first.join('=')
-        end
+        QueryStringFilter.call(super, self)
       end
 
     end

data/lib/surrogate_key_logging/action_dispatch.rb

@@ -4,6 +4,8 @@ module SurrogateKeyLogging
   module ActionDispatch
     extend ActiveSupport::Autoload
 
+    autoload :ParamsFilter
+    autoload :QueryStringFilter
     autoload :Request
 
   end

data/lib/surrogate_key_logging/active_record/attributes.rb

@@ -7,13 +7,9 @@ module SurrogateKeyLogging
     module Attributes
       extend ActiveSupport::Concern
 
-      included do
-        surrogate_parent_names model_name.singular, model_name.plural
-      end
-
       class_methods do
         def surrogate_parent_names(*names)
-          @surrogate_parent_names ||= []
+          @surrogate_parent_names ||= [model_name.singular, model_name.plural]
           names.each do |name|
             @surrogate_parent_names << name.to_sym
             surrogate_attributes.each do |attr|

data/lib/surrogate_key_logging/engine.rb

@@ -25,7 +25,7 @@ module SurrogateKeyLogging
     initializer 'surrogate_key_logging.config' do |app|
       SurrogateKeyLogging.configure do |config|
         config.enabled = Rails.env.production? unless config.key?(:enabled)
-        config.debug = !Rails.env.production? unless config.key?(:debug)
+        config.debug = false unless config.key?(:debug)
         config.key_prefix = '' unless config.key?(:key_prefix)
         config.key_for ||= -> (value) { "#{config.key_prefix}#{SecureRandom.uuid}" }
         config.cache = true unless config.key?(:cache)
@@ -49,11 +49,5 @@ module SurrogateKeyLogging
       end
     end
 
-    initializer 'surrogate_key_logging.middleware' do
-      if SurrogateKeyLogging.config.enabled
-        Rails.application.config.middleware.insert_before 0, Middleware
-      end
-    end
-
   end
 end

data/lib/surrogate_key_logging/key_manager.rb

@@ -36,7 +36,7 @@ module SurrogateKeyLogging
     
     def call(_key, value, _parents = [], _original_params = nil)
       surrogate = get(value)
-      # Rails.logger.tagged('SurrogateKeyLogging') { Rails.logger.info "Surrogate: `#{surrogate}`, value: `#{value}`" } if SurrogateKeyLogging.config.debug
+      Rails.logger.tagged('SurrogateKeyLogging') { Rails.logger.info "Surrogate: `#{surrogate}`, value: `#{value}`" } if SurrogateKeyLogging.config.debug
       surrogate
     end
 

data/lib/surrogate_key_logging/version.rb

@@ -4,8 +4,8 @@ module SurrogateKeyLogging
 
   module Version
     MAJOR = 0
-    MINOR = 1
-    PATCH = 0
+    MINOR = 2
+    PATCH = 1
 
   end
 

data/lib/surrogate_key_logging.rb

@@ -49,7 +49,11 @@ module SurrogateKeyLogging
     end
 
     def parameter_filter
-      @parameter_filter ||= ::ActiveSupport::ParameterFilter.new(surrogate_attributes, mask: key_manager)
+      @parameter_filter ||= filter_for_attributes(surrogate_attributes)
+    end
+
+    def filter_for_attributes(attrs)
+      ::ActiveSupport::ParameterFilter.new(attrs, mask: key_manager)
     end
 
     def key_store

data/rails_surrogate_key_logging.gemspec

@@ -27,8 +27,8 @@ Gem::Specification.new do |s|
 
   s.required_ruby_version = '>= 2.7.0'
 
-  s.add_dependency('actionpack', '>= 6.0.0')
-  s.add_dependency('activerecord', '>= 6.0.0')
-  s.add_dependency('activesupport', '>= 6.0.0')
-  s.add_dependency('railties', '>= 6.0.0')
+  s.add_dependency('actionpack', '>= 6.0.0', '< 7.0.0')
+  s.add_dependency('activerecord', '>= 6.0.0', '< 7.0.0')
+  s.add_dependency('activesupport', '>= 6.0.0', '< 7.0.0')
+  s.add_dependency('railties', '>= 6.0.0', '< 7.0.0')
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rails_surrogate_key_logging
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Taylor Yelverton
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2023-01-20 00:00:00.000000000 Z
+date: 2023-01-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: actionpack
@@ -17,6 +17,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -24,6 +27,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
 - !ruby/object:Gem::Dependency
   name: activerecord
   requirement: !ruby/object:Gem::Requirement
@@ -31,6 +37,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -38,6 +47,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
 - !ruby/object:Gem::Dependency
   name: activesupport
   requirement: !ruby/object:Gem::Requirement
@@ -45,6 +57,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -52,6 +67,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
 - !ruby/object:Gem::Dependency
   name: railties
   requirement: !ruby/object:Gem::Requirement
@@ -59,6 +77,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -66,6 +87,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 6.0.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: 7.0.0
 description: ''
 email: rubygems@yelvert.io
 executables: []
@@ -77,7 +101,10 @@ files:
 - lib/surrogate_key_logging.rb
 - lib/surrogate_key_logging/action_controller.rb
 - lib/surrogate_key_logging/action_controller/log_subscriber.rb
+- lib/surrogate_key_logging/action_controller/params.rb
 - lib/surrogate_key_logging/action_dispatch.rb
+- lib/surrogate_key_logging/action_dispatch/params_filter.rb
+- lib/surrogate_key_logging/action_dispatch/query_string_filter.rb
 - lib/surrogate_key_logging/action_dispatch/request.rb
 - lib/surrogate_key_logging/active_record.rb
 - lib/surrogate_key_logging/active_record/attributes.rb
@@ -89,7 +116,6 @@ files:
 - lib/surrogate_key_logging/key_store.rb
 - lib/surrogate_key_logging/key_store/active_record.rb
 - lib/surrogate_key_logging/key_store/base.rb
-- lib/surrogate_key_logging/middleware.rb
 - lib/surrogate_key_logging/version.rb
 - lib/tasks/key_store/active_record.rake
 - lib/tasks/surrogate_key_logging.rake

data/lib/surrogate_key_logging/middleware.rb

@@ -1,17 +0,0 @@
-# frozen_string_literal: true
-
-module SurrogateKeyLogging
-  class Middleware
-    attr_reader :app
-
-    def initialize(app)
-      @app = app
-    end
-
-    def call(env)
-      SurrogateKeyLogging.reset
-      @app.call(env)
-    end
-
-  end
-end