drive_time 0.0.3

data/.gitignore

@@ -0,0 +1,61 @@
+# Also see Global .gitignore:  ~/.gitignore_global
+# https://github.com/stationkeeping/osx/blob/master/git/.gitignore_global
+
+# Environmental Variables
+#########################
+
+.env
+
+# Rails
+#########################
+
+*.rbc
+*.sassc
+.sass-cache
+capybara-*.html
+.rspec
+.rvmrc
+/.bundle
+/vendor/bundle
+/log/*
+/tmp/*
+/db/*.sqlite3
+/public/system/*
+/coverage/
+/spec/tmp/*
+**.orig
+rerun.txt
+pickle-email-*.html
+.project
+
+# Ruby
+#########################
+
+*.gem
+*.rbc
+.bundle
+.config
+coverage
+InstalledFiles
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+
+# YARD artifacts
+.yardoc
+_yardoc
+doc/
+
+# Ruby Gems
+#########################
+
+Gemfile.lock
+
+# Project Specific
+#########################
+
+spec/fixtures/cached/**

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in drive_time.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Pedr Browne
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,240 @@
+# DriveTime
+
+Drive Time allows you to transform a one or more Google Spredsheet into a Rails model graph.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'drive_time'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install drive_time
+
+## Usage
+
+### Drive Time
+
+Drive Time allows you to map a Google Spreadsheet to a your Rails database. Each worksheet represents a different Model class; its columns represent the model's attributes and each row represents a different instance. It is designed to make as many allowances for the person authoring the spreadsheet as possible so it can be used as a bridge between a non-technical user and yourself. Unless they are an idiot, in which case you're still screwed.
+
+*I am using it sucessfully in two projects at the moment, however this is very much alpha and I am adding features as I need them. My main priority is to add more comprehansive tests.*
+
+#### Installation
+
+You know the drill. Add `gem drive_time` to your `Gemfile` and run `$ bundle`.
+
+#### Prerequisits
+
+Drive Time uses the [Google Drive](https://github.com/gimite/google-drive-ruby) gem, which handles the connection to Google Drive, the download and transforms the Spreadsheet and Worksheet into Ruby objects. It relies on the presence of two envs:
+
+```
+GOOGLE_USERNAME=your.email@gmail.com
+GOOGLE_PASSWORD=YourPassword
+```
+
+You will also need a Rails project with a migrated datbase, ready to recieve the generated models.
+
+#### Using Drive Time
+
+Drive Time needs access to your Rails project. You can run it as part of the seeding process, or directly using `$ rails runner`.
+
+An example of usage:
+
+```
+include DriveTime
+
+mappings_path = File.join(File.dirname(__FILE__),'mappings.yml')
+SpreadsheetsConverter.new.load mappings_path
+```
+
+The `mappings.yml` is crucial. It specifies the name of your spreadsheet and how you want to map your worksheets to your models.
+
+##### Mappings
+
+A bare-bones mapping might look like this:
+
+```
+spreadsheets:
+- title: Business
+  worksheets:
+    - title: Company
+      key: name
+      fields:
+        - name: name
+        - name: description
+      associations:
+        - name: learning_group
+```
+
+Drive Time will look for a spreadsheet called 'Business' and download it. It will then find a spreadsheet within called 'Company' and run through each row, generating a new model of class `Company` with attributes of `name` and `description`.
+
+The `key` is a unique identifier which is used internally to identify the models and is used within the spreadsheets to declare associations. In this case we are declaring that we want to use the name field as the key. The crucial thing is that the key is unique amongst all models of that type. If no attributes are guarenteed to be unique, one option is to add an explicit `key` column, containing a unique identifier to the spreadsheet, however this is unwealdy and easy to lose track of. A better option is to use multiple attributes to generate the key. This can be done using a builder:
+
+```
+  worksheets:
+    - title: Company
+      key:
+        builder: join
+        from_fields: [name, city]
+```
+
+This will result in a key made from the company name and city combined. This is great for giving models of the same type a unique key, but requires more thought if the id will be used by other models to declare associations.
+
+*Note: Internally, keys are just downcased, underscored and whitespace-stripped versions of whater value is ussed for the key.*
+
+*All fields values are run through a markdown parser before they are added to the model.*
+
+##### Associations
+
+Below an association is declared between the `Company` and the `Product`. It is declared as a singular relationship using the `singular` mapping attribute.
+
+Drive Time assumes that if a worksheet declares a singular attribute mapping, it will contain a column named after the model. This field should contain the key of the instance of the model that it will use to satisfy its dependency. In the example below it would contain the sku of the `Product` model.
+
+```
+spreadsheets:
+- title: Business
+  worksheets:
+    - title: Company
+     key: name
+     fields:
+       - name: name
+       - name: description
+     associations:
+       - name: Product
+       - singular: true
+    - title: Product
+     key: sku
+     fields:
+       - name: title
+       - name: sku
+       - name: price
+```
+
+One-to-many relationships can be declared in two ways.
+
+If there are only a few possible options, a column can be assigned to each option and named after the option. For example if a company had one or more Regions from either Europe, or Asia or America, and we were using the name field for the Region key, we could add three columns to the Company spreadsheet named 'Europe', 'Asia', and 'America'. If we want to add that region we would add a value of 'Yes' to the field:
+
+```
+associations:
+  - name: region
+    builder: use_fields
+    field_names: [europe, asia, america]
+```
+
+Another option, appropriate for situations where there are many possible values, is to declare an inverted relationship. for example, if you had a 'Team' that had many 'Players', rather than declaring the team's players in the team Spreadsheet, you could declare each Player's team in a column in the Player Spreadsheet. For this to work, you must declare the relationship inverse. So in the Player worksheet mapping:
+
+```
+associations:
+  - name: team
+    inverse: true
+```
+
+A final option is to add a comma separated list of keys and use the 'join' builder. A model wil be added for each key:
+
+```
+associations:
+  - name: member
+    builder: multi
+```
+
+A polymorphic association can be declared using:
+
+```
+associations:
+  - name: team
+    polymorphic:
+      association: contactable
+```
+
+It is OK to mix 'polymorphic' and 'singular' for the same association.
+
+Using Google Spreadsheet's data validations can make things much easier on the Spreadsheet end. You can effectively add dropdowns containing values from a fixed list or from another Worksheet column, making sure that only valid values can be added.
+
+##### Caching
+
+By setting an env called `CACHED_DIR` to a directory path, Drive Time will store downloaded spreadsheets there and use them on subsequent occasions. Just delete the contents of the folder or remove the env to reload new versions the next time it runs.
+
+
+```
+CACHED_DIR=/Users/me/path/to/cached/directory
+```
+
+##### File Expansion and Text Docs
+
+If Drive Time encounters a value surrounded by `{{` and `}}` within a database field, it will use this value to load another file from Google Drive and use its content in the place of the field value. Possible values are `expand_spreadsheet` and `expand_file` which can be used to load the content of a spreadsheet or a `.txt` file respectively:
+
+```
+{{expand_spreadsheet}}
+```
+
+And:
+
+```
+{{expand_file}}
+```
+
+In both cases, Drive Time will try to load a file named identically to the key for the model on which the field will be added. To specify a different filename, add it in hard brackets and without an extension:
+
+```
+{{expand_spreadsheet[some_spreadsheet_file_name]}}
+```
+
+And:
+
+```
+{{expand_spreadsheet[some_txt_file_name]}}
+```
+
+In the case of the text file, its contents will simply be added. In the case of a spreadsheet, it will be converted to a JSON object which will be used as the field value.
+
+##### Debugging
+
+By default, Drive Time outputs a minimal set of messages. To enable a much more verbose output, set the log level to DEBUG:
+
+```
+require "log4r"
+DriveTime::log_level = Log4r::DEBUG
+```
+
+##### Other features
+
+If you want to map a database field to a differently named field on a model you can use the following:
+
+```
+fields:
+  - name: name
+    map_to: title
+```
+
+This will map a Worksheet field named 'name' to a model attribute named 'title'.
+
+You can also map the Worksheet title to a differently named model:
+
+```
+worksheets:
+    - title: Staff
+      map_to_class: Employee
+```
+
+This will use the Worksheet named 'Staff' to a model called 'Employee'.
+
+It is sometimes useful to generate a UID for models to use in linked to an image or icon resource. You can map the model's key to a model attribute using:
+
+```
+- title: Example
+  key: title
+  key_to: uid
+```
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,7 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new
+
+task :default => :spec
+task :test => :spec

data/drive_time.gemspec

@@ -0,0 +1,30 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'drive_time/version'
+
+Gem::Specification.new do |gem|
+  gem.name          = "drive_time"
+  gem.version       = DriveTime::VERSION
+  gem.authors       = ["Pedr Browne"]
+  gem.email         = ["pedr.browne@gmail.com"]
+  gem.description   = %q{Convert Google Spreadsheets to Rails Models}
+  gem.summary       = %q{Map Worksheets to Models and their columns to model attributes. Seed your database directly from Google Drive.}
+  gem.homepage      = "https://github.com/stationkeeping/drive_time"
+
+  gem.files         = `git ls-files`.split($/)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+
+  gem.add_dependency "log4r"
+  gem.add_dependency "deep_end"
+  gem.add_dependency "google_drive"
+
+  gem.add_development_dependency 'rake'
+  gem.add_development_dependency 'rspec'
+  gem.add_development_dependency 'dotenv'
+  gem.add_development_dependency "activemodel", "3.2.13"
+  gem.add_development_dependency "activerecord", "3.2.13"
+  gem.add_development_dependency "activesupport", "3.2.13"
+end

data/lib/drive_time/bi_directional_hash.rb

@@ -0,0 +1,33 @@
+module DriveTime
+
+  class BiDirectionalHash
+
+    def initialize
+      @key_to_value = {}
+      @value_to_key = {}
+    end
+
+    def insert(key, value)
+      @key_to_value[key] = value;
+      @value_to_key[value] = key;
+    end
+
+    def value_for_key(key)
+      return @key_to_value[key]
+    end
+
+    def key_for_value(value)
+      return @value_to_key[value]
+    end
+
+    def has_key_for_value(value)
+      return !key_for_value(value).nil?
+    end
+
+    def has_value_for_key(key)
+      return !value_for_key(key).nil?
+    end
+
+  end
+
+end

data/lib/drive_time/builders/join_builder.rb

@@ -0,0 +1,39 @@
+module DriveTime
+
+  # Take a series of name fields. Look up their values and assemble them into a single id
+  # For example it might build a name from a model's title and its amount
+  class JoinBuilder
+
+    class MissingFieldError < StandardError; end
+    class NoFieldsError < StandardError; end
+
+    # Fields to use for names
+    def build(field_keys, row_map)
+      values = []
+
+      field_keys.each do |field_key|
+        raise MissingFieldError, "No field for key #{field_key}" if !row_map.has_key? field_key
+        values << DriveTime.underscore_from_text(row_map[field_key]) unless row_map[field_key].empty?
+      end
+
+      raise NoFieldsError, 'No fields matched' if values.empty?
+
+      values.each_with_index do |value, index|
+
+        result = self.process_value value
+        if result
+          values[index] = result
+        end
+      end
+      values.join('_').downcase
+    end
+
+    protected
+
+    def process_value(value)
+      return value
+    end
+
+  end
+
+end

data/lib/drive_time/builders/name_builder.rb

@@ -0,0 +1,19 @@
+module DriveTime
+
+  # Take a series of name fields. Look up their values and assemble them into a single id
+  class NameBuilder < JoinBuilder
+
+    protected
+
+      def process_value(value)
+
+        # Add a period to initials
+        if value.length == 1
+          return "#{value}."
+        end
+        return value
+      end
+
+  end
+
+end

data/lib/drive_time/class_name_map.rb

@@ -0,0 +1,41 @@
+module DriveTime
+
+  # This is a fluid two-way map. In both directions it will return a mapping if it exists, otherwise
+  # it will return the value passed to it. It does not raise an exception if a mapping is not found.
+  class ClassNameMap
+
+    def initialize
+      @map = BiDirectionalHash.new
+    end
+
+    # Check for mapped class
+    def resolve_original_from_mapped(className)
+      if @map.has_key_for_value className
+        @map.key_for_value className
+      else
+        className
+      end
+    end
+
+    def resolve_mapped_from_original(className)
+      if @map.has_value_for_key className
+        @map.value_for_key className
+      else
+        className
+      end
+    end
+
+    # Accepts String versions of class names eg. ExampleClass
+    def save_mapping(class_name, mapped_class_name=nil)
+      if mapped_class_name
+        # Save mapping so we can look it up from both directions
+        @map.insert(class_name, mapped_class_name)
+        mapped_class_name
+      else
+        class_name
+      end
+    end
+
+  end
+
+end

data/lib/drive_time/converters/spreadsheets_converter.rb

@@ -0,0 +1,125 @@
+module DriveTime
+
+  class SpreadsheetsConverter
+
+    def initialize()
+      @dependency_graph = DeepEnd::Graph.new
+      @loader = DriveTime::Loader.new
+      @model_store = ModelStore.new(DriveTime::log_level)
+      @class_name_map = ClassNameMap.new
+    end
+
+    # Load mappings YML file
+    def load(mappings_path)
+      @mappings = ActiveSupport::HashWithIndifferentAccess.new(YAML::load File.open(mappings_path))
+      @namespace = @mappings[:namespace]
+      spreadsheets = download_spreadsheets
+
+      worksheets = []
+      spreadsheets.each do |spreadsheet|
+        # Create a map containing any class mappings
+        build_class_map spreadsheet
+        # Download and combine worksheets into single Array
+        worksheets.concat download_worksheets(spreadsheet)
+      end
+
+      worksheets = order_worksheets_by_dependencies( worksheets )
+
+      # Convert the worksheets
+      worksheets.each do |worksheet|
+        WorksheetConverter.new(@model_store, @class_name_map, @loader, @namespace).convert(worksheet)
+      end
+
+      @model_store.save_all
+    end
+
+    protected
+
+    def download_spreadsheets
+      spreadsheets = []
+      # First download the spreadsheets
+      @mappings[:spreadsheets].each do |spreadsheet_mapping|
+        spreadsheet =  @loader.load_spreadsheet(spreadsheet_mapping[:title])
+        raise "No spreadsheet with a title: #{spreadsheet_mapping['title']} available" if spreadsheet.nil?
+        # Store mapping on the spreadsheet
+        spreadsheet.mapping = spreadsheet_mapping
+        spreadsheets << spreadsheet
+      end
+      return spreadsheets
+    end
+
+    def download_worksheets(spreadsheet)
+      worksheet_mappings = spreadsheet.mapping[:worksheets]
+      worksheets = []
+      if worksheet_mappings
+        worksheet_mappings.each do |worksheet_mapping|
+          title = worksheet_mapping[:title]
+          worksheet = @loader.load_worksheet_from_spreadsheet spreadsheet, title
+          raise "No worksheet with a title: #{worksheet_mapping['title']} available" if worksheet.nil?
+          worksheet.mapping = worksheet_mapping
+          worksheets << worksheet
+        end
+        return worksheets
+      end
+    end
+
+    def order_worksheets_by_dependencies(worksheets)
+      Logger.log_as_header "Calculating worksheet dependencies"
+      # Now sort their dependencies before converting them
+      worksheets.each do |worksheet|
+        Logger.debug "Worsheet named #{worksheet.title}"
+        associations = find_associations worksheet, worksheets
+        associations.each { |association| Logger.debug " - #{association.title}" }
+        @dependency_graph.add_dependency worksheet, associations
+      end
+      return @dependency_graph.resolved_dependencies
+    end
+
+    def find_associations(dependent_worksheet, worksheets)
+
+      dependent_associations_mapping = dependent_worksheet.mapping[:associations]
+
+      associations = []
+      if dependent_associations_mapping
+        # Run through each association
+        dependent_associations_mapping.each do |association_mapping|
+          # Handle possible polymorphic association
+          # If name is an array, we need to add each possibility as a dependency
+          names = []
+          if association_mapping[:name].is_a? Array
+            names = association_mapping[:name]
+          else
+            names << association_mapping[:name]
+          end
+          names.each do |name|
+            associations << worksheet_for_association(name, worksheets)
+          end
+        end
+      end
+      return associations
+    end
+
+    def worksheet_for_association(name, worksheets)
+      # And find the worksheet that satisfies it
+      worksheets.each do |worksheet|
+        class_name = DriveTime.class_name_from_title(worksheet.title)
+        resolved = @class_name_map.resolve_mapped_from_original class_name
+        # If the name matches we have a dependent relationship
+        if resolved.underscore == name
+          return worksheet
+        end
+      end
+      raise MissingAssociationError, "No worksheet #{name} to satisfy multi association"
+    end
+
+    def build_class_map(spreadsheet)
+      worksheets_mapping = spreadsheet.mapping[:worksheets]
+      worksheets_mapping.each do |worksheet_mapping|
+        class_name = DriveTime.class_name_from_title(worksheet_mapping[:title])
+        mapped_class_name = worksheet_mapping[:map_to_class]
+        @class_name_map.save_mapping(class_name, mapped_class_name)
+      end
+    end
+
+  end
+end