greyscale_record 0.0.1 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: '04649514246f75d5ecd1d02fba8b1b773744d771'
-  data.tar.gz: de857e40fabfa2d27622f7f6957eed6b9602b5f3
+  metadata.gz: e182b351304fb45824511ff704bae0fb071aea01
+  data.tar.gz: 147b67b9aca64b01243585e57cd21d11303a83ca
 SHA512:
-  metadata.gz: eb2a45fc827f0e4e1f5ea48bfd0589c1d3e9c73cf605330bc8c50e18188723759425a72d6c88bedf8b9adc9b40c33f086d36fe1b58a923f47e26f87824e95b8f
-  data.tar.gz: d218efe8bda685f876c4526bc58bdafaaa8b56624901225abe9a3e74c0bccd75a9970579503869837dffc78e335413da269ab0081d0c89b8136868b7c9ced85d
+  metadata.gz: fcd11e5ee95ec0f0630e32b65b20c5e9b01af69b2903aeb481b583ed4687de64b350d14b28b6d6e6f9d8c0d67113115715c847a610546a1fa61bb3ce7a36cdf2
+  data.tar.gz: da33bc49c25147f838485b784a843bbba247243b2f20c5af1eac74cef47af4cd7af556a5a9541d983fa40d0b8f116e0f39ae4693ca4ae63ea3db11becc58e080

data/README.md

@@ -1,6 +1,6 @@
 # Greyscale Record
 
-`GreyscaleRecord` is a simple read-only implementation of [YamlRecord](https://github.com/nicotaing/yaml_record). It's designed for users whose data is perfectly static and is stored in a flat file format (e.g. yaml files). It is a clone of [YamlBSides](https://github.com/gaorlov/yaml_b_sides), but is extended to support multiple backend drivers (YAML files, Greyscale API)
+`GreyscaleRecord` is a flat-file ORM, designed for users whose data is perfectly static and is stored in a flat format (e.g. yaml files). It is a clone of [YamlBSides](https://github.com/gaorlov/yaml_b_sides), but is extended to support multiple backend drivers (YAML files, JSON APIs)
 
 ## Installation
 
@@ -46,12 +46,6 @@ Note: `Greyscale Record` expects your class names to match the fixture names (e.
 
 Your `Person` class now responds to 
 
-### Query Methods
-
-* `all` : will give you all of the records in the table
-* `first` : wil return the first record in the table
-* `find( id )` : will find a single record with the specified yaml key
-* `find_by( properties = {} )` : will find all the recored that match all the proerties in the hash
 
 ### Indexing
 
@@ -153,6 +147,82 @@ Associations have some of the standard ActiveRecord options. Namely:
       #...
   ```
 
+### Query Methods
+
+* `all` : will give you all of the records in the table
+* `first` : wil return the first record in the table
+* `find( id )` : will find a single record with the specified yaml key
+* `find_by( properties = {} )` : will find all the recored that match all the proerties in the hash
+
+#### Relation methods
+
+These act as you would expect them to in ActiveRecord. These can be chained and applied to associations
+* `where( params )` : will return a relation which can be interacted with as if it were the resulting array
+* `and( params )` : identical to `where`, but nicer to read
+
+```ruby
+  Person.where( url_slug: "greg" ) #=> [<Person>]
+  Person.where( url_slug: "greg", name: "Greg Orlov" ).and( id: "greg") #=> [<Person>]
+  Person.where( url_slug: "greg", name: "Greg Orlov" ).where( id: "greg") #=> [<Person>]
+  # from the code above
+  Person.first.images.where( some_property: "value" )
+```
+
+### Data Sourcing
+
+As mentioned in the summary above, `GreyscaleRecord` can be connected to different data sources. The structure that we use is a `Driver`.
+There is a built in driver to use as an example: Yaml Driver. This is the structure that controls data flow from the original data store,
+such as YAML files or an API, and formats it to be consumed by the internal `GreyscaleRecord::DataStore::Store` object, which expects the
+result set to look roughly like 
+
+```
+{ table_name: {
+    record_id: { attribute: value, ... },
+    ...
+  },
+  ...
+}
+```
+
+`GreyscaleRecord` will populate the id field for you from the record keys.
+
+### Data Patching
+
+If you have a data set that you want to temporarily augment, you can apply a [JSON patch](http://jsonpatch.com/) to the data store.
+This will apply the patch within the context of your current thread until it is removed
+
+```ruby
+    patch = ::Hana::Patch.new [ { 'op' => 'add', 'path' => '/people/mike', 'value' => { id: "mike", name: "Mike Uchman" } } ]
+
+    # this will stick around indefinitely in this thread
+    data_store.apply_patch patch
+
+    Person.find( 'mike' ).name # => "Mike Uchman"
+    Person.where( id: 'mike' ).first.name #=> "Mike Uchman"
+
+    # let's go back to the original set
+    data_store.remove_patch
+
+    Person.where( id: "mike" ) #=> []
+```
+
+Or, if you are doing something simple and can fir your code in a single block, you can do:
+
+```ruby
+    data_store.with_patch patch do
+
+      Person.find( 'mike' ).name # => "Mike Uchman"
+      Person.where( id: 'mike' ).first.name #=> "Mike Uchman"
+
+    end
+
+    Person.where( id: "mike" ) #=> []
+```
+
+__NOTE__: The patch interface that `Store` expects is that of [Hana](https://github.com/tenderlove/hana). You don't have to use it, but it has
+to respond to `patch.apply( doc )`. 
+
+
 ### Example
 
 To use the `People` class from earlier, a fully fleshed out model would look something like:
@@ -198,8 +268,8 @@ The setup is pretty straightforward. Greyscale Record wants a logger and a base
 ```ruby
 GreyscaleRecord::logger = Rails.logger
 # for now this is the only driver
-GreyscaleRecord::Base.driver = GreyscaleRecord::Drivers::Yaml
-GreyscaleRecord::Drivers::Yaml.root = Rails.root.join 'db', 'fixtures'
+yaml_driver = GreyscaleRecord::Drivers::Yaml.new File.expand_path("./db/fixtures", File.dirname(__FILE__))
+GreyscaleRecord::Base.data_store = GreyscaleRecord::DataStore::Engine.new(yaml_driver)
 
 
 # in development.rb

data/greyscale_record.gemspec

@@ -1,4 +1,4 @@
-# coding: utf-8
+  # coding: utf-8
 lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'greyscale_record/version'
@@ -30,5 +30,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "minitest", "~> 5.0"
   spec.add_development_dependency "simplecov"
   spec.add_development_dependency "m"
-
+  spec.add_development_dependency "hana"
 end

data/lib/greyscale_record/base.rb

@@ -8,40 +8,20 @@ module GreyscaleRecord
     include Instanceable
     include Queriable
 
-    class_attribute :data
-    class_attribute :driver
+    class_attribute :data_store
 
     class << self
 
       def load!
-        @data = driver.load!(_class_name)
-        return unless @data
-        
-        # let's preemptively index by id so that when we do a find_by id:, or a where id: it won't table scan
-        idify_data!
-        
-        index :id unless GreyscaleRecord.live_reload
+        data_store.add_table name
       end
 
       def inherited(subclass)
         subclass.load!
       end
 
-      protected
-
-      def _class_name
-        self.name.pluralize.downcase
-      end
-
-      def idify_data!
-        @data.each do |k, v|
-          v[:id] = k
-        end
-      end
-
-      def data
-        load! if GreyscaleRecord.live_reload
-        @data
+      def name
+        self.to_s.pluralize.downcase
       end
     end
   end

data/lib/greyscale_record/cacheable.rb

@@ -5,7 +5,7 @@ module GreyscaleRecord
     included do
       class << self
         def cache_key
-          @cache_key ||= Digest::SHA256.hexdigest(data.to_json)
+          @cache_key ||= Digest::SHA256.hexdigest(all.to_json)
         end
       end
 

data/lib/greyscale_record/data_store/engine.rb

@@ -0,0 +1,66 @@
+module GreyscaleRecord
+  module DataStore
+    class Engine
+
+      # A data store can only have one driver. 
+      # It's like a database connection. If you want models to connect to different
+      #   databases, you have them inherit from different base classes that specify 
+      #   a db to connect to.
+
+      # Pros:
+      #   * All the data in your store comes from the same place. Easy to reason about
+      #   * No chance of data source collisions. Can't populate tables from multiple sources
+      #   * Cleaner table-adding interface 
+      #       store.add_table( name )
+      #            vs
+      #       store.add_table( name, driver )
+      #       # where does driver get initialized? 
+      #   * Straight forward patching: one complete patch at a time
+      #       How do you patch across differntly driven tables?
+      #       Do you need a patch stack?
+      #   * Depending on how your remote is built, you can pull in all the data at once
+      #
+      # Cons:
+      #   * Limiting if you want to have some tables come from YAML and some from remote
+      #
+      # Yeah. OK.
+
+      delegate :apply_patch, :remove_patch, :with_patch, :table, to: :store 
+
+      def initialize( driver )
+        @driver = driver
+        @store  = Store.new
+      end
+
+      # Read only store. No writes allowed. 
+
+      def find( options = {} )
+        table = store.table( options.delete(:_table) )
+        if GreyscaleRecord.live_reload
+          load_table!( table )
+        end
+
+        # TODO: this is where all the meat is
+        table.find options
+      end
+
+      def add_table( name )
+        load_table! name
+      end
+
+      def add_index( name, column )
+        store.table( name ).add_index( column )
+      end
+
+      private
+
+      def store
+        @store
+      end
+
+      def load_table!( name )
+        store.init_table name, @driver.load!( name )
+      end
+    end
+  end
+end

data/lib/greyscale_record/data_store/index.rb

@@ -0,0 +1,25 @@
+module GreyscaleRecord
+  module DataStore
+    class Index
+      def initialize(field, data)
+        @indexed_data = {}
+        data.each do |id, datum|
+          key = datum[field]
+
+          # storing key => array of matching ids
+          @indexed_data[key] = Array(@indexed_data[key]) + [id]
+        end
+      end
+
+      # returns ids 
+      def find(values)
+        # find all the arrays of ids for the values, 
+        #   get rid of nils (value not present), 
+        #   and compact for a single array result
+        values.map do |value|
+          @indexed_data[value]
+        end.compact.flatten
+      end
+    end
+  end
+end

data/lib/greyscale_record/data_store/store.rb

@@ -0,0 +1,72 @@
+module GreyscaleRecord
+  module DataStore
+    class Store
+      def initialize
+        @data = {}
+        @tables = {}
+      end
+
+      def []( name )
+        data[ name ]
+      end
+
+      def table( name )
+        unless @tables[name]
+          raise GreyscaleRecord::Errors::DataStoreError, "Data Store error: table '#{name}' does not exist"
+        end
+
+        @tables[name]
+      end
+
+      def init_table( name, rows )
+        @data[name] = rows
+        @tables[name] = Table.new( name, self )
+      end
+
+      def with_patch( patch )
+        apply_patch patch
+        yield
+        remove_patch
+      end
+
+      # This only allows for one patch at a time. 
+      # Is there ever a case when we would need, like a stack of these things? 
+      # I don't think so? 
+
+      def apply_patch( patch )
+        Thread.current[patch_key] = patched_data patch
+      end
+
+      def remove_patch
+        Thread.current[patch_key] = nil
+      end
+
+      def patched?
+        Thread.current[patch_key].present?
+      end
+      
+      private
+
+      def patched_data(patch)
+        unless patch.respond_to? :apply
+          raise GreyscaleRecord::Errors::DataStoreError, "Data Store Error: apply_patch: patch must respond to 'apply(doc)'."
+        end
+
+        patch.apply( @data.deep_dup )
+      end
+
+      def patch_key
+        @key ||= "#{object_id}_patch"
+      end
+
+
+      def data
+        if patched?
+          Thread.current[patch_key]
+        else
+          @data
+        end
+      end
+    end
+  end
+end

data/lib/greyscale_record/data_store/table.rb

@@ -0,0 +1,80 @@
+module GreyscaleRecord
+  module DataStore
+    class Table
+
+      def initialize(name, store)
+        @name = name
+        @store = store
+
+        # initialize the index array for later use
+        @indices = {}
+
+        # generate IDs for the records based on YAML keys
+        generate_ids!
+
+        # preemptively index the IDs
+        add_index :id
+      end
+
+      def all
+        rows.values
+      end
+
+      def add_index( column )
+        return if @store.patched?
+        @indices = @indices.merge( { column => Index.new(column, rows) } )
+      end
+
+      def find( params = {} )
+        return all if params.empty?
+        sets = params.map do | column, values |
+          if !patched? && indexed?( column )
+            find_in_index column, values
+          else
+            GreyscaleRecord.logger.warn "You are running a query on #{@name}.#{column} which is not indexed. This will perform a table scan."
+            find_in_column column, values
+          end
+        end
+
+        sets.inject( sets.first ) do |result, subset|
+          result & subset
+        end
+      end
+
+      private
+
+      def rows
+        @store[@name]
+      end
+
+      def patched?
+        @store.patched?
+      end
+
+      def indexed?(column)
+        @indices[column].present?
+      end
+
+      def find_in_column( column, values )
+        rows.values.select do |datum|
+          Array( values ).include? datum[ column ]
+        end
+      end
+
+      def find_in_index( column, values )
+        keys = @indices[column].find( Array( values ) )
+        
+        keys.map do |id|
+          rows[id]
+        end
+      end
+
+      def generate_ids!
+        # init IDs
+        rows.each do |k, v|
+          v[:id] = k
+        end
+      end
+    end
+  end
+end

data/lib/greyscale_record/data_store.rb

@@ -0,0 +1,8 @@
+module GreyscaleRecord
+  module DataStore
+    autoload :Engine,     'greyscale_record/data_store/engine'
+    autoload :Index,      'greyscale_record/data_store/index'
+    autoload :Store,      'greyscale_record/data_store/store'
+    autoload :Table,      'greyscale_record/data_store/table'
+  end
+end

data/lib/greyscale_record/drivers/base.rb

@@ -2,29 +2,31 @@ module GreyscaleRecord
   module Drivers
     class Base
 
-      class_attribute :root
+      attr_reader :root
+
+      def initialize( root )
+        @root = root
+      end
       
-      class << self
-        def load!(class_name)
+      def load!(object)
 
-          raise GreyscaleRecord::DriverError "driver needs to define a `root`" unless root
-          
-          data = load_data(class_name)
-          
-          GreyscaleRecord.logger.info "#{class_name} successfully loaded data"
+        raise GreyscaleRecord::Errors::DriverError, "driver needs to define a `root`" unless root
+        
+        data = load_data(object)
+        
+        GreyscaleRecord.logger.info "#{object} successfully loaded data"
 
-          data
+        data
 
-        rescue => e
-          GreyscaleRecord.logger.error "#{self} failed to load data for #{class_name}: #{e}"
-          return nil
-        end
+      rescue => e
+        GreyscaleRecord.logger.error "#{self.class} failed to load data for #{object}: #{e}`"
+        {}
+      end
 
-        private 
+      private 
 
-        def load_data
-          raise NotImplementedError "load_data is not implemented"
-        end
+      def load_data
+        raise NotImplementedError, "load_data is not implemented"
       end
     end
   end

data/lib/greyscale_record/drivers/yaml.rb

@@ -2,17 +2,14 @@ module GreyscaleRecord
   module Drivers
     class Yaml < Base
 
-      class << self
+      private 
 
-        private 
-
-        def load_data( class_name )
-          YAML.load_file( data_file( class_name ) ).with_indifferent_access
-        end
+      def load_data( object )
+        YAML.load_file( data_file( object ) ).with_indifferent_access
+      end
 
-        def data_file( class_name )
-          [root, "#{class_name}.yml"].compact.join("/")
-        end
+      def data_file( object )
+        [root, "#{object}.yml"].compact.join("/")
       end
     end
   end

data/lib/greyscale_record/errors.rb

@@ -11,5 +11,8 @@ module GreyscaleRecord
 
     class DriverError < StandardError
     end
+
+    class DataStoreError < StandardError
+    end
   end
 end

data/lib/greyscale_record/indexable.rb

@@ -3,31 +3,12 @@ module GreyscaleRecord
     extend ActiveSupport::Concern
 
     included do
-      class_attribute :__indices
-      self.__indices = { }
-
       class << self
+        # DEPRICATED
+        # TODO: remove
         def index(field)
           return if GreyscaleRecord.live_reload
-          self.__indices = __indices.merge( { field => Index.new(field, data) } )
-        end
-
-        def find_in_index(field, values)
-          keys = Array(index_for(field).find(values))
-          
-          keys.map do |id|
-            data[id]
-          end
-        end
-
-        def indexed?(field)
-          __indices[field].present?
-        end
-
-        protected 
-
-        def index_for(field)
-          __indices[field]
+          data_store.add_index( name, field )
         end
       end
     end

data/lib/greyscale_record/queriable.rb

@@ -5,69 +5,27 @@ module GreyscaleRecord
     included do
       class << self
         def find(id)
-          record = data[id]
-          raise Errors::RecordNotFound, "#{self}: Record not found: #{id}" unless record
-          new record
+          records = where( id: id.to_s )
+          raise Errors::RecordNotFound, "#{ self }: Record not found: #{ id }" if records.empty?
+          records.first
         end
 
-        def find_by(params = {})
+        def find_by( params = { } )
           results = where params
-          raise Errors::RecordNotFound, "#{self}: Could not find record that matches: #{params.inspect}" if results.empty?
+          raise Errors::RecordNotFound, "#{ self }: Could not find record that matches: #{ params.inspect }" if results.empty?
           results.first
         end
 
         def all
-          data.values.map do |obj|
-            new obj
-          end
+          where
         end
 
         def first
-          new data.values.first
+          all.first
         end
 
-        # TODO: move this into scopes
-        def where(params)
-          if all_indexed?(params.keys)
-            results = find_by_indexed(params)
-          else
-            results = find_by_scan(params)
-          end
-          results.map do |result|
-            new result
-          end
-        end
-
-        private
-
-        def find_by_scan(params)
-          data.values.select do |datum|
-            params.all? do |param, expected_value|
-              val = Array(expected_value).include? datum[param]
-            end
-          end
-        end
-
-        def find_by_indexed(params)
-          sets = []
-          params.each do |index, values|
-            sets << find_in_index(index, Array(values))
-          end
-
-          # find the intersection of all the sets
-          sets.inject( sets.first ) do |result, subset|
-            result & subset
-          end
-        end
-
-        def all_indexed?(fields)
-          fields.all? do |field|
-            indexed = indexed? field
-            unless indexed
-              GreyscaleRecord.logger.warn "You are running a query on #{self}.#{field} which is not indexed. This will perform a table scan."
-            end
-            indexed
-          end
+        def where( params = {} )
+          Relation.new self, params 
         end
       end
     end

data/lib/greyscale_record/relation.rb

@@ -0,0 +1,29 @@
+module GreyscaleRecord
+  class Relation
+
+    delegate :present?, :empty?, :==, to: :all
+
+    def initialize( base, params )
+      @base   = base
+      @params = params.dup.merge!( _table: @base.name )
+    end
+
+    def where( params )
+      self.class.new @base, @params.merge( params )
+    end
+
+    def and( params )
+      self.class.new @base, @params.merge( params )
+    end
+
+    def all
+      @all ||= @base.data_store.find( @params.dup ).map do | result |
+        @base.new result
+      end
+    end
+
+    def method_missing( method, *args, &block )
+      all.send method, *args, &block
+    end
+  end
+end

data/lib/greyscale_record/version.rb

@@ -1,3 +1,3 @@
 module GreyscaleRecord
-  VERSION = "0.0.1"
+  VERSION = "1.0.0"
 end

data/lib/greyscale_record.rb

@@ -5,6 +5,8 @@ require 'active_support'
 require 'active_support/concern'
 require 'active_support/core_ext/class/attribute'
 require 'active_support/core_ext/hash'
+require 'active_support/core_ext/hash/keys'
+require 'active_support/core_ext/object/deep_dup'
 require 'yaml'
 
 module GreyscaleRecord
@@ -12,13 +14,14 @@ module GreyscaleRecord
   autoload :Associations,   'greyscale_record/associations'
   autoload :Base,           'greyscale_record/base'
   autoload :Cacheable,      'greyscale_record/cacheable'
+  autoload :DataStore,      'greyscale_record/data_store'
   autoload :Drivers,        'greyscale_record/drivers'
   autoload :Errors,         'greyscale_record/errors'
   autoload :Instanceable,   'greyscale_record/instanceable'
-  autoload :Index,          'greyscale_record/index'
   autoload :Indexable,      'greyscale_record/indexable'
   autoload :Propertiable,   'greyscale_record/propertiable'
   autoload :Queriable,      'greyscale_record/queriable'
+  autoload :Relation,       'greyscale_record/relation'
 
   class << self
     attr_accessor :live_reload

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: greyscale_record
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 1.0.0
 platform: ruby
 authors:
 - Greg Orlov
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-06-29 00:00:00.000000000 Z
+date: 2017-07-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -108,6 +108,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: hana
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: An ActiveRecord-like interface for reading from & queryeing flat data
   interfaces
 email:
@@ -136,15 +150,20 @@ files:
 - lib/greyscale_record/associations/hasable.rb
 - lib/greyscale_record/base.rb
 - lib/greyscale_record/cacheable.rb
+- lib/greyscale_record/data_store.rb
+- lib/greyscale_record/data_store/engine.rb
+- lib/greyscale_record/data_store/index.rb
+- lib/greyscale_record/data_store/store.rb
+- lib/greyscale_record/data_store/table.rb
 - lib/greyscale_record/drivers.rb
 - lib/greyscale_record/drivers/base.rb
 - lib/greyscale_record/drivers/yaml.rb
 - lib/greyscale_record/errors.rb
-- lib/greyscale_record/index.rb
 - lib/greyscale_record/indexable.rb
 - lib/greyscale_record/instanceable.rb
 - lib/greyscale_record/propertiable.rb
 - lib/greyscale_record/queriable.rb
+- lib/greyscale_record/relation.rb
 - lib/greyscale_record/version.rb
 homepage: https://github.com/greyscale-io/greyscale_record
 licenses:

data/lib/greyscale_record/index.rb

@@ -1,23 +0,0 @@
-module GreyscaleRecord
-  class Index
-    def initialize(field, data)
-      @indexed_data = {}
-      data.each do |id, datum|
-        key = datum[field]
-
-        # storing key => array of matching ids
-        @indexed_data[key] = Array(@indexed_data[key]) + [id]
-      end
-    end
-
-    # returns ids 
-    def find(values)
-      # find all the arrays of ids for the values, 
-      #   get rid of nils (value not present), 
-      #   and compact for a single array result
-      values.map do |value|
-        @indexed_data[value]
-      end.compact.flatten
-    end
-  end
-end