elm_history_tools 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 9a30716f0cdb540ad8228ec16e82e2e42d1a000f
-  data.tar.gz: 3163afcb272e617cce11a7c4afdfea92db1d801d
+  metadata.gz: 280b6456a5473cc4af85ebe96dd303466bf1341d
+  data.tar.gz: d18f4ff0a50be7ddd539b0198b51d05415812151
 SHA512:
-  metadata.gz: d34ce51ff0aa23cbc2dae2d6dbd5ee5a7cd568e02cd222a43cdf93e049998c5015d1d2be3f3af68ba12701d4b0c7bbc3f0e4a4c6225e60613b36dc83680bed7e
-  data.tar.gz: b1d356d39c1434c69b5520006bd1be06c39efde94ee8ff0096d008f2334371ef683d6cb9d18817a84eef0aa63af69b32dc10cce668768c62da06f92564a42679
+  metadata.gz: b9e621128a84df9e1ee1c84369cff2bb46c6fe7800a664b40ac3f64dc58efa028b140229f2b20693ab7390e3e3e02c319d980416316c0acc0f832c52283a460e
+  data.tar.gz: 9e7c57afa830d958f2fcce36d9894227605d4afb810a32f6ff679211068327e4c29cb37dc99e60651e8139bacaeadd8912bf1655f316fc7e5c37bc5a8fc3bf75

data/Gemfile.lock

@@ -1,12 +1,11 @@
 PATH
   remote: .
   specs:
-    elm_history_tools (0.1.0)
+    elm_history_tools (0.2.0)
 
 GEM
   remote: https://rubygems.org/
   specs:
-    awesome_print (1.8.0)
     byebug (10.0.2)
     diff-lcs (1.3)
     rake (10.5.0)
@@ -28,7 +27,6 @@ PLATFORMS
   ruby
 
 DEPENDENCIES
-  awesome_print
   bundler (~> 1.16)
   byebug
   elm_history_tools!

data/README.md

@@ -38,17 +38,46 @@ _Sanitizing History_
 
 How do you actually do this?
 
-COMING SOON
+`ElmHistoryTools::HistorySanitizer` allows you to sanitize Elm records and objects. Because these
+objects are so individual to your application, there's no universal code that can sanitize the data
+without breaking the structure. (This is why we can't, for instance, just use something like Rails'
+ParameterFilters.)
+
+Instead, you tell the HistorySanitizer which terms to look for and provide a block that can do the
+sanitizing:
+
+```ruby
+# let's say you have an Elm object like
+# MySecretData "access_token" {password = "myP@ssw0rd", expiration = "tomorrow"}
+ElmHistoryTools::HistorySanitizer.sanitize_history(history_data, ["MySecretData", "password"]) do |elm_object_or_record|
+  if elm_object_or_record["ctor"] == "MySecretData"
+    # replace the access token
+    # make sure to return the updated object!
+    elm_object_or_record.merge("_0" =>"[FILTERED]")
+  else
+    # we have the record
+    elm_object_or_record.merge("password" => "[FILTERED]")
+  end
+end
+
+# before: {"ctor" => "MySecretData", "_0" => "access_token", "_1" => {"password" => "myP@ssw0rd", "expiration" => "tomorrow"}}
+# after: {"ctor" => "MySecretData", "_0" => "[FILTERED]", "_1" => {"password" => "[FILTERED]", "expiration" => "tomorrow"}}
+```
+
+As you can see, this is neither flexible nor elegant -- unfortunately, that's the nature of the
+game when you're manipulating the state generated by an Elm program. Elm types may store sensitive data
+as basic data in the object, which leaves no automated way to detect and replace that information.
 
 _Format a History into a Hash_
 
-One approach we've found useful for user issues at eSpark Learning has been to skim the
-history and see if anything looks amiss. Eyeballing the user's actions shows if a student get an unhappy response back from a
-server, if they were able to they answer any quiz questions, etc., providing a useful context for
-further debugging.
+One approach we've found useful for user issues at eSpark Learning has been to skim the history and
+see if anything looks amiss. Eyeballing the user's actions shows if a student get an unhappy
+response back from a server, if they were able to they answer any quiz questions, etc., providing a
+useful context for further debugging.
 
-In order to make sense of data, it has to be presented in a comprehensible format. That's what the
-`ElmHistoryTools::HistoryFormatter` object can do:
+In order to make sense of data, it has to be presented in a compact and (relatively) human-readable
+format, which the raw history export is not. (See below for more information on that.) Such cleanup
+is what the `ElmHistoryTools::HistoryFormatter` object can do:
 
 ```ruby
 # given a JSON history file like

data/lib/elm_history_tools.rb

@@ -1,5 +1,7 @@
 require "elm_history_tools/version"
+require "elm_history_tools/utils"
 require "elm_history_tools/history_formatter"
+require "elm_history_tools/history_sanitizer"
 
 module ElmHistoryTools
   # Your code goes here...

data/lib/elm_history_tools/history_formatter.rb

@@ -20,23 +20,23 @@ module ElmHistoryTools::HistoryFormatter
   # alternative approach would be to use nil. While that would clearly distinguish between those
   # cases, it would make working with the results more complicated.
   def self.simplify_history_entry(entry)
-    if !entry.is_a?(Hash)
-      entry
-    elsif entry["ctor"] == "::"
-      # Elm lists are represented as nested entries with the contructor ::. (See the readme for
-      # more detail.)
-      # We collapse those into a proper Ruby array via flatten.
-      # The last entry of the list will have no nested entry, so we use compact to remove the nil.
-      [simplify_history_entry(entry["_0"]), simplify_history_entry(entry["_1"])].compact.flatten
-    elsif entry["ctor"]
-      # we have an Elm type
-      {
-        entry["ctor"] => entry.reject {|k, _v| k == "ctor"}.values.map {|val| simplify_history_entry(val) }
-      }
-    else
-      entry.each_with_object({}) do |(key, value), hash|
-        hash[key] = simplify_history_entry(value)
+    ElmHistoryTools::Utils.transform_object(entry) do |object_hash|
+      if object_hash["ctor"] == "::"
+        # Elm lists are represented as nested entries with the contructor ::. (See the readme for
+        # more detail.)
+        # We collapse those into a proper Ruby array via flatten.
+        # The last entry of the list will have no nested entry, so we use compact to remove the nil.
+        [simplify_history_entry(object_hash["_0"]), simplify_history_entry(object_hash["_1"])].compact.flatten
+      elsif object_hash["ctor"]
+        # we have an Elm object type (we know this because non-objects aren't passed to the block)
+        {
+          object_hash["ctor"] => object_hash.reject {|k, _v| k == "ctor"}.values.map {|val| simplify_history_entry(val) }
+        }
       end
     end
   end
+
+  class << self
+    private :simplify_history_entry
+  end
 end

data/lib/elm_history_tools/history_sanitizer.rb

@@ -0,0 +1,68 @@
+module ElmHistoryTools::HistorySanitizer
+  # We don't include the object received because it might have sensitive information
+  # This might be reconsidered in the future.
+  class ElmObjectExpected < StandardError; end
+
+  # Given a raw Elm history file parsed to JSON, a set of key terms to watch for (password, token,
+  # etc.) , sanitize a
+  # history so it doesn't contain sensitive information.
+  #
+  # The message sanitizers should be in the format of
+  # "HistoryConstructor" => block_taking_history_entry_and_returning_sanitized_version
+  #
+  # You can optionally specify a set of watch words to ensure you don't accidentally
+  # introduce unsafe data. If specified and any unsanitized entry contains a constructor or record
+  # property matching one of those words, an error will be raised. (So for instance, if you add a
+  # User object with an Password property to a message, a watch word for "password" will catch
+  # that -- though it would also catch password_reset_requested.)
+  #
+  # Each app will have such different entry structures that there's no easy to generalize this
+  # further.
+  def self.sanitize_history(history_data, watch_words, &sanitizing_block)
+    matchers = watch_words.map {|word| word.is_a?(Regexp) ? word : Regexp.new(word, true)}
+    history_data.dup.merge(
+      "history" => history_data["history"].map {|entry| sanitize(entry, matchers, &sanitizing_block)}
+    )
+  end
+
+  # For each entry, we sanitize it if there's a matching handler. If there is no sanitizer, see if
+  # any data in the entry merit flagging against a watch word.
+  def self.sanitize(entry, matchers, &sanitizing_block)
+    ElmHistoryTools::Utils.transform_object(entry) do |elm_object|
+      constructor = elm_object["ctor"]
+      raise ElmObjectExpected unless constructor
+
+      # replace any records that need it
+      cleaned_object = elm_object.each_with_object({}) do |(key, value), hash|
+        if value.is_a?(Hash) && value["ctor"]
+          # we have an elm object -- sanitize that
+          hash[key] = sanitize(value, matchers, &sanitizing_block)
+        elsif value.is_a?(Hash)
+          # We have an Elm record, represented as a raw hash
+          # This is inefficient, but should hopefully be acceptable for the relatively low number of records
+          # being processed. If not, we can rewrite it.
+          if value.keys.any? {|field| matchers.any? {|matcher| field.match(matcher)}}
+            hash[key] = yield value
+          else
+            hash[key] = value
+          end
+        else
+          # raw values
+          hash[key] = value
+        end
+      end
+
+      # now that we've sanitized any stored records or objects, sanitize the overall record
+      if matchers.any? {|matcher| constructor.match(matcher)}
+        yield cleaned_object
+      else
+        cleaned_object
+      end
+    end
+  end
+
+  class << self
+    private :sanitize
+  end
+end
+

data/lib/elm_history_tools/utils.rb

@@ -0,0 +1,22 @@
+module ElmHistoryTools::Utils
+  # This method gives us a tool to walk down an Elm history entry, processing all the Elm objects contained within per the provided block. Whatever the block returns will become the value at that point.
+  #
+  # To simplify matters, this doesn't process primitive values. Additionally, while each value an
+  # Elm record (which are stored as plain JSON objects) is examined, the overall record itself is
+  # never passed to the block -- to examine/alter an Elm record, do so in the callback for
+  # whichever Elm object contains it. (All history entries are Message objects, so there's always
+  # an entry point.)
+  def self.transform_object(entry, &block)
+    if !entry.is_a?(Hash)
+      entry
+    elsif entry["ctor"]
+      # If you want to walk down sub-entries, call transform_object within your block.
+      yield entry
+    else
+      # Each Elm record should be
+      entry.each_with_object({}) do |(key, value), hash|
+        hash[key] = transform_object(value, &block)
+      end
+    end
+  end
+end

data/lib/elm_history_tools/version.rb

@@ -1,3 +1,3 @@
 module ElmHistoryTools
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: elm_history_tools
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Alex Koppel
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-04-21 00:00:00.000000000 Z
+date: 2018-06-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -57,6 +57,8 @@ files:
 - elm_history_tools.gemspec
 - lib/elm_history_tools.rb
 - lib/elm_history_tools/history_formatter.rb
+- lib/elm_history_tools/history_sanitizer.rb
+- lib/elm_history_tools/utils.rb
 - lib/elm_history_tools/version.rb
 - readme-images/elm-history-dashboard.png
 homepage: http://github.com/arsduo/elm-history-tools