databasion 0.1.1 → 0.2.0

data/CHANGELOG.md

@@ -0,0 +1,7 @@
+### 0.2.0
+
+* [FIXED] Fields flagged with an index were not properly referenced.
+* [CHANGE] There is now an Environment spreadsheet to define environments, and each environment replaces the old Database spreadsheet.  Databasion data is also broken down by directory, named after the relevant environment.  Specific environments are called with the new -e switch.
+* [NEW] Version control is now handled in the Environment spreadsheet, and works via the -r switch.  Ideal for near-realtime updates using crontab.
+* [NEW] Multiple primary keys can now be assigned by adding a comma delimited _primary_ to a field.
+* [NEW] The _options_ field in a spreadsheet environment definition now works according to the Rails Migration rules.

data/README.md

@@ -2,10 +2,12 @@
 
 ## Google Spreadsheet/Excel -> YAML -> Ruby Migration -> Database Management Tool
 
-A database management tool.  The theory is that a designer/planner can edit application data and a programmer can setup the database and it's fields, all in one happy little place.  As tables are added and data is changed, if the script is run once again it will update the target database.
+A database management tool.  The theory is that a designer/planner can edit application data while a programmer setups up the database schema and it's fields, all in one happy little place.  As tables are added and data is changed, if the script is run once again it will update the target database.
 
 TODO: While this system uses Rails Migrations, it isn't taking full advantage of them (i.e. tracking changes, allowing for rollbacks, etc).
 
+NOTE: The system is currently undergoing major changes, and will not remain backwards compatible up until roughly a 0.9 release.
+
 ## Requirements
 
 ### Ruby
@@ -16,7 +18,7 @@ TODO: While this system uses Rails Migrations, it isn't taking full advantage of
 
 * ActiveRecord >= 2.3.5
 * ActiveSupport >= 2.3.5
-* Google Spreadsheet >= 0.1.1
+* Google Spreadsheet >= 0.1.2
 * Spreadsheet >= 0.6.4.1
 
 ## Installation
@@ -29,42 +31,67 @@ TODO: While this system uses Rails Migrations, it isn't taking full advantage of
 
 None of this would really work if there weren't some conventions in place.  The following explains how the worksheet needs to be formatted, how the data spreadsheets themselves needs to be formatted, what fields are required, and what fields can be ignored.
 
-At the highest level there needs to be a worksheet named _Database_.  This is simply a master list of related spreadsheets, and what database they correspond to (for split table designs).  The column names are required.
+### Environment and Version Control
+
+In order to manage your _environments_ and _versions_ of the data for each environment, an __Environments__ spreadsheet is required.
+
+#### Keywords
+
+Row0 is reserved for keywords, which are listed below.
+
+* environment - Defines your environment.  These follow the standard _development, test, production_ breakdown.  Anything outside of these three are considered special case environments.
+* version - The current version this environment is at in regards to the databasion spreadsheet.  This is also used for auto-updates via crontab.
+
+A typical spreadsheet might look like the following.
+
+| environment | version
+|:------------|:-------
+| development |      50
+| test        |      49
+| production  |      45
+| brian_test  |      55
+| bojo_test   |      50
+
+Versions are tracked by a locally written version file when databasion is ran with the _--cron_ switch.
+
+You will also need to create spreadsheets named after the typical _development, test, production_ keywords as listed in the Environments spreadsheet.  Within each spreadsheet, database descriptions need to be defined as written below.  The column names are required.
+
+Note: Non-standard environments can also be created, however, they are treated as special case and not part of the environment chain.
 
-### Database
+### Example 'Development' Spreadsheet
 
 | spreadsheet | dbname| database| username| password| adapter| host     | port| options
 |:------------|:------|:--------|:--------|:--------|:-------|:---------|:----|:-------
 | superheroes | db1   | db_test | dbuser  | dbuser  | mysql  | 127.0.0.1|     |        
 
-The options column currently support's _force_, which tells the database to drop and recreate the table.
+The options column currently supports any SQL commands that can typically be passed to a Rails Migration.
 
 Next we define the actual table spreadsheets.
 
-### Superheroes
-
-| column0  |             |              |                    
-|:---------|:------------|:-------------|:-------------------
-| ignore   |             |              |                    
-| comment  |             |              |                    
-| table    | superheroes |              |                    
-| index    | yes         |              |
-| field    | id          | name         | power              
-| type     | integer     | string, 20   | string, 20, Wimp   
-|          | 1           | Brian Jones  | Ruby Hacker        
-|          | 2           | Superman     | Invincible         
-|          | 3           | Batman       | Rich               
-
-### Keywords
-
-* ignore - Anything written in this column will cause this column and it's data to be ignored.
+### Example 'Superheroes' Table
+
+| column0  |             |              |                    | 
+|:---------|:------------|:-------------|:-------------------|:--------
+| ignore   |             |              |                    | testing
+| comment  |             |              |                    |
+| table    | superheroes |              |                    |
+| index    | yes         |              |                    |
+| field    | id          | name         | power              | cape 
+| type     | integer     | string, 20   | string, 20, Wimp   | boolean
+|          | 1           | Brian Jones  | Ruby Hacker        | false
+|          | 2           | Superman     | Invincible         | true
+|          | 3           | Batman       | Rich               | true
+| testing  | 4           | Hulk         | Huge               | false
+
+### Table Spreadsheet Keywords
+
+* ignore - Anything written in this column will cause this column and it's data to be ignored, with the exception of _environment names_.  See the Environment and Version Control section below for further usage.
+* comment - Ideally a description of the field, what the values means, etc.
 * table - The name of the table, and an optional comma delimited 'false' if the table name should not be auto-pluralized.
-* index - If something is written in a columns field here, it will get flagged as an index and created via add_index(table, [fields]).
-* field - The name of the table column.
+* index - This will create an index on the designated field.  If a multi-index is required, indices will be grouped by unique names.  Multiple multi-indices are possible.
+* field - The name of the table column, with an optional comma delimited 'auto' or 'primary' parameter.  Auto is strictly limited to an 'id' field, and enables auto incrementation.
 * type  - A comma delimited list giving the type of the column (using Ruby migration terms), optional size, and optional default value.
 
-Note: If an 'id' column is specified, then it is assumed the id's are supplied by hand.  Auto-incrementation is disabled, and 'id' is the primary key.
-
 __Ruby Migration Types__
 
 * binary
@@ -79,13 +106,13 @@ __Ruby Migration Types__
 * time
 * timestamp
 
-### Columns
+#### Columns
 
 Currently column0 is reserved for keywords and comments.
 
 If something besides a keyword is written in column0, that row is ignored and will not be used.  This is useful if you need to edit out some data.
 
-### Rows
+#### Rows
 
 Row0 isn't technically reserved, but should ideally be saved for use with the _ignore_ flag.  If any text is written in a column (with the exception of column0), that column will be ignored.  This is useful for editing out columns that one doesn't currently want in the database.
 
@@ -103,16 +130,18 @@ Edit _config/google.yml_.  Then run the scripts.
     databasion --load
     databasion --svn
     databasion --git
+    databasion --cron
+    databasion --env
     
 Or run them all in order.
 
-    databasion --google --migrate --load --git
+    databasion --google --migrate --load --git --env development
     
 You can supply a different config path as well.
 
     databasion -g -m -l -i --config config/my.other.config.yml
     
-Someone administrating a production database with this tool would definitely want to run each script sequentially by hand.
+The environment switch defaults to _development_.
     
 ### YAML Configuration
 
@@ -133,27 +162,51 @@ If the currently created databasion project is committed to SVN, running the _--
 
 Much like SVN, if the project is commited to a GIT repo, the _--git_ switch will auto-add and commit all the project files.  If there isn't a repository, it will also initialize a new one for you.
 
-## Version Control
+## Keyword Environment Management (Currently not implemented)
+
+Keywords are also supported in the _ignore_ columns and rows of table definitions.  This allows us to not inadvertently add columns or data to systems which aren't configured to use them yet.  The following is an example.
+
+ | column0      |             |              |                    |         |                       
+ |:-------------|:------------|:-------------|:-------------------|:--------|:----------------------
+ | ignore       |             |              |                    | test    | brian_test, bojo_test
+ | comment      |             |              |                    |         |                       
+ | table        | superheroes |              |                    |         |                       
+ | index        | yes         |              |                    |         |                       
+ | field        | id          | name         | power              | cape    | mask                  
+ | type         | integer     | string, 20   | string, 20, Wimp   | boolean | boolean               
+ |              | 1           | Brian Jones  | Ruby Hacker        | false   | false                 
+ |              | 2           | Superman     | Invincible         | true    | false                 
+ |              | 3           | Batman       | Rich               | true    | true                  
+ | brian_test   | 4           | Hulk         | Huge               | false   | false                 
+ | brian_test   | 5           | Spawn        | Demonic            | true    | true                  
+ 
+Environments are updated in the following order:  development -> test -> production.  Special case environments are ignored with the exception of themselves.
+ 
+The above definition states that the _cape_ field should only be deployed _up to test_, and no further.  The _mask_ field should *only* be deployed to both brian_test and bojo_test since they are not common environment names.  Everything else will be updated clear up to production.
+
+It also follows that the field with the id's 4 and 5 should only be deployed to brian_test.
+
+### Version Control
 
 There is now a system in place to do crontab driven auto-updates.  This allows the data to be updated without anyone having to access any systems.
 
-First update _config/google.yml_'s cron section to reflect your project settings.  The _options_ are your standard databasion switches.  Make sure to supply a location for the version file so that it can be read at a later date.
+First update _config/google.yml_'s environment section to reflect your project settings for each given environment.  The _options_ are standard databasion switches.
 
-Next, add a _Version_ spreadsheet to the master worksheet.  In the A1 field use whichever number works best to start versioning with.  When using this to manage the project, if the number is higher than before then the system will be updated.  Currently version controlled rollbacks are not implemented.
+Next, add an entry for your environment in the _Environments_ spreadsheet, along with the starting version number.  When using this to manage the project, if the number is higher than before then the system will be updated.  Currently version controlled rollbacks are not implemented.
 
 Finally, add the databasion script to crontab.
 
 Example crontab:
 
-    */1 * * * * cd /home/my_user/project && databasion -r
+    */1 * * * * cd /home/my_user/project && databasion -r -e test
     
-This checks the Version spreadsheet once a minute, and if the version has changed runs databasion with the supplied options.
+This checks the Environment spreadsheet once a minute, and if the version for the target system (test in this case) has changed, runs databasion with the supplied options.
 
 Note:  This could easily be used from the commandline as well, and not just crontab.
 
-## Roadmap
+## Planned Features
 
-Long and winding.
+Plugins - It would be nice to be able to build plugin support so people can do pre/post processing of the data, and/or export out to other formats besides YAML, etc.  I also plan on designing this in such a way so that it isn't strictly RDMBS centric.
 
 ## Testing
 
@@ -165,3 +218,5 @@ __Brian Jones__ - Server Engineer, [Istpika](http://www.istpika.com)
 
 * Work: <brian.jones@istpika.com>
 * Personal: <mojobojo@gmail.com>
+
+Twitter: [mojobojo](http://twitter.com/mojobojo) - If you are using databasion give me a shoutout, I'm curious to see who is using this system.

data/Rakefile

@@ -8,11 +8,11 @@ Jeweler::Tasks.new do |gem|
   gem.email       = "mojobojo@gmail.com"
   gem.homepage    = "http://github.com/boj/databasion"
   gem.authors     = ["Brian Jones", "Istpika"]
-  gem.version     = "0.1.1"
+  gem.version     = "0.2.0"
 
   gem.add_dependency('activerecord', '>= 2.3.5')
   gem.add_dependency('activesupport', '>= 2.3.5')
-  gem.add_dependency('google-spreadsheet-ruby', '>= 0.1.1')
+  gem.add_dependency('google-spreadsheet-ruby', '>= 0.1.2')
   gem.add_dependency('spreadsheet', '>= 0.6.4.1')
 end
 

data/config/example.google.yml

@@ -1,10 +1,12 @@
+project_base: /home/user/project
+
 login:
   username: fakeuser
   password: fakepass
   
 sheets:  
   - name: non_existant_spreadsheet
-    key: key_google_doc_key
+    key: google_doc_key
     
 output:
   yaml_path: 'data'
@@ -18,9 +20,11 @@ svn:
   
 git:
   bin: /usr/local/bin/git
-    
-cron:
-  project_base: /home/user/project
-  version:
-    file: /home/user/project/config/version
-  options: -g -m -l -i
+  
+environments:
+  development:
+    cron_options: -g -m -l -i
+  test:
+    cron_options: -l
+  production:
+    cron_options: -l

data/databasion.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = %q{databasion}
-  s.version = "0.1.1"
+  s.version = "0.2.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Brian Jones", "Istpika"]
-  s.date = %q{2010-11-04}
+  s.date = %q{2010-12-03}
   s.default_executable = %q{databasion}
   s.email = %q{mojobojo@gmail.com}
   s.executables = ["databasion"]
@@ -18,7 +18,8 @@ Gem::Specification.new do |s|
      "README.md"
   ]
   s.files = [
-    "LICENSE",
+    "CHANGELOG.md",
+     "LICENSE",
      "README.md",
      "Rakefile",
      "bin/databasion",
@@ -68,18 +69,18 @@ Gem::Specification.new do |s|
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
       s.add_runtime_dependency(%q<activerecord>, [">= 2.3.5"])
       s.add_runtime_dependency(%q<activesupport>, [">= 2.3.5"])
-      s.add_runtime_dependency(%q<google-spreadsheet-ruby>, [">= 0.1.1"])
+      s.add_runtime_dependency(%q<google-spreadsheet-ruby>, [">= 0.1.2"])
       s.add_runtime_dependency(%q<spreadsheet>, [">= 0.6.4.1"])
     else
       s.add_dependency(%q<activerecord>, [">= 2.3.5"])
       s.add_dependency(%q<activesupport>, [">= 2.3.5"])
-      s.add_dependency(%q<google-spreadsheet-ruby>, [">= 0.1.1"])
+      s.add_dependency(%q<google-spreadsheet-ruby>, [">= 0.1.2"])
       s.add_dependency(%q<spreadsheet>, [">= 0.6.4.1"])
     end
   else
     s.add_dependency(%q<activerecord>, [">= 2.3.5"])
     s.add_dependency(%q<activesupport>, [">= 2.3.5"])
-    s.add_dependency(%q<google-spreadsheet-ruby>, [">= 0.1.1"])
+    s.add_dependency(%q<google-spreadsheet-ruby>, [">= 0.1.2"])
     s.add_dependency(%q<spreadsheet>, [">= 0.6.4.1"])
   end
 end

data/features/cron_system.feature

@@ -1,7 +1,7 @@
 Feature: Crontab
     Allow a user to run databasion updates via crontab
     
-    Scenario: User configures the Version spreadsheet
-        Given a version spreadsheet in Google Docs
+    Scenario: User configures the Environments spreadsheet
+        Given an environment spreadsheet in Google Docs
         When the cron system is ran and the version changes
         Then databasion should have been ran

data/features/env/helper_methods.rb

@@ -2,11 +2,17 @@ def load_parse_data
   {
     'name'        => 'mock',
     'plural'      => true,
-    'indexes'     => ["id"],
+    'auto'        => false,
+    'indexes'     => {:default => [0]},
+    'primaries'   => ["id", "power"],
     'fields'      => ["id", "name", "power"],
-    'types'       => ["integer", "string, 20", "string, 40"],
+    'types'       => ["integer", "string, 20", "string, 40, Wuss"],
     'data'        => [[1, "Brian Jones", "Super Intelligence"], [2, "Superman", "Invincible"], [3, "Batman", "Strength"]],
     'ignore_cols' => [2],
     'connection'  => {'database' => 'moon', 'dbname' => 'db1'}
   }
 end
+
+def load_opts
+  { :env => 'development' }
+end

data/features/step_definitions/build_migration_steps.rb

@@ -2,21 +2,19 @@ require 'lib/databasion'
 
 Given /a set of YAML data/ do
   @config = YAML.load(File.open('config/google.yml'))
-
+  @opts = load_opts
   @parse_data = load_parse_data
-  Databasion::YamlBuilder.run(@parse_data, @config['output']['yaml_path'])
+  Databasion::YamlBuilder.run(@parse_data, @config, @opts)
 end
 
 When /we run the BuildMigration system/ do
-  Databasion::BuildMigration.run(Dir['%s/**.yml' % @config['output']['yaml_path']], @config)
+  Databasion::BuildMigration.run(Dir['%s/%s/**.yml' % [@opts[:env], @config['output']['yaml_path']]], @config, @opts)
 end
 
 Then /it should build a migration, model, and database file/ do
-  File.exist?('%s/%s/100_%s_migration.rb' % [@config['output']['migrations']['path'], @parse_data['connection']['dbname'], @parse_data['name']]).should == true
-  File.exist?('%s/%s.rb' % [@config['output']['migrations']['models'], @parse_data['name']]).should == true
-  File.exist?('config/database.yml').should == true
-  FileUtils.rm_rf @config['output']['migrations']['path']
-  FileUtils.rm_rf @config['output']['migrations']['models']
-  FileUtils.rm_rf @config['output']['yaml_path']
-  FileUtils.rm 'config/database.yml'
+  File.exist?('%s/%s/%s/100_%s_migration.rb' % [@opts[:env], @config['output']['migrations']['path'], @parse_data['connection']['dbname'], @parse_data['name']]).should == true
+  File.exist?('%s/%s/%s.rb' % [@opts[:env], @config['output']['migrations']['models'], @parse_data['name']]).should == true
+  File.exist?('config/database.%s.yml' % @opts[:env]).should == true
+  FileUtils.rm_rf @opts[:env]
+  FileUtils.rm_rf 'config/database.%s.yml' % @opts[:env]
 end

data/features/step_definitions/cron_system_steps.rb

@@ -1,17 +1,19 @@
 require 'lib/databasion'
 
-Given /a version spreadsheet in Google Docs/ do
+Given /an environment spreadsheet in Google Docs/ do
   @config = YAML.load(File.open('config/google.yml'))
+  @opts = load_opts
 end
 
 When /the cron system is ran and the version changes/ do
   Databasion::CronSystem.config = @config
-  Databasion::CronSystem.run
+  Databasion::CronSystem.run(@opts)
 end
 
 Then /databasion should have been ran/ do
   Databasion::GoogleLoader.config = @config
-  version = Databasion::GoogleLoader.run_version
-  File.open(@config['cron']['version']['file']).readline.strip.should == version
-  FileUtils.rm_rf @config['cron']['version']['file']
+  version = Databasion::GoogleLoader.run_version(@opts)
+  version_file = "%s/config/version_%s" % [@config['project_base'], @opts[:env]]
+  File.open(version_file).readline.strip.should == version
+  FileUtils.rm_rf version_file
 end

data/features/step_definitions/yaml_builder_steps.rb

@@ -2,15 +2,16 @@ require 'lib/databasion'
 
 Given /a complete set of YAML definitions/ do
   @config = YAML.load(File.open('config/google.yml'))
-
+  @opts = load_opts
   @parse_data = load_parse_data
 end
 
 When /the YAML files are parsed/ do
-  Databasion::YamlBuilder.run(@parse_data, @config['output']['yaml_path'])
+  Databasion::YamlBuilder.run(@parse_data, @config, @opts)
 end
 
 Then /the result is Ruby migration files/ do
-  File.exist?("%s/%s.yml" % [@config['output']['yaml_path'], @parse_data['name']]).should == true
-  FileUtils.rm_rf @config['output']['yaml_path']
+  path = @opts[:env] + "/" + @config['output']['yaml_path']
+  File.exist?("%s/%s.yml" % [path, @parse_data['name']]).should == true
+  FileUtils.rm_rf @opts[:env]
 end

data/lib/databasion.rb

@@ -14,37 +14,37 @@ module Databasion
   
   @@config = nil
   
-  def self.run(system, config=nil, opts=nil)
+  def self.run(system, opts=nil)
     LOGGER.level = Logger::INFO
     
-    raise DatabasionError, 'Databasion requires a YAML config file path.' if config.nil?
-    @@config = YAML.load(File.open(config))
+    raise DatabasionError, 'Databasion requires a YAML config file path.' if opts[:config].nil?
+    @@config = YAML.load(File.open(opts[:config]))
     
     case system
     when "google"
-      run_google
+      run_google(opts)
     when "migrate"
-      run_migrate
+      run_migrate(opts)
     when "load"
       run_load(opts)
     when "svn"
-      run_svn
+      run_svn(opts)
     when "git"
-      run_git
+      run_git(opts)
     when "cron"
-      run_cron
+      run_cron(opts)
     end
   end
   
   private
-  def self.run_google
+  def self.run_google(opts)
     Databasion::GoogleLoader.config = @@config
-    Databasion::GoogleLoader.run
+    Databasion::GoogleLoader.run(opts)
   end
   
-  def self.run_migrate
+  def self.run_migrate(opts)
     Databasion::Migrate.config = @@config
-    Databasion::Migrate.run
+    Databasion::Migrate.run(opts)
   end
   
   def self.run_load(opts)
@@ -52,19 +52,19 @@ module Databasion
     Databasion::LoadData.run(opts)
   end
   
-  def self.run_svn
+  def self.run_svn(opts)
     Databasion::SvnCommitter.config = @@config
     Databasion::SvnCommitter.commit
   end
   
-  def self.run_git
+  def self.run_git(opts)
     Databasion::GitCommitter.config = @@config
     Databasion::GitCommitter.commit
   end
   
-  def self.run_cron
+  def self.run_cron(opts)
     Databasion::CronSystem.config = @@config
-    Databasion::CronSystem.run
+    Databasion::CronSystem.run(opts)
   end
 
   def self.set_ar_logger

data/lib/databasion/application.rb

@@ -23,6 +23,7 @@ module Databasion
         opt :svn, "Auto commit the project files (assuming it has been committed to SVN)"
         opt :git, "Auto commit the project files (assuming a working git repo)"
         opt :cron, "Run the version control system via crontab and update on version changes"
+        opt :env, "Define the environment with which to run.  Default: development", :type => String
       end
       if opts[:config].nil? and opts[:create].nil?
         config = "config/google.yml"
@@ -33,6 +34,10 @@ module Databasion
           Trollop::die :config, "A YAML config must be specified"
         end
       end
+      
+      if !opts[:env]
+        opts[:env] = 'development'
+      end
 
       if opts[:create]
         create_project(opts)
@@ -43,22 +48,22 @@ module Databasion
     
     def self.execute_databasion(opts)
       if opts[:cron]
-        Databasion.run('cron', opts[:config], opts)
+        Databasion.run('cron', opts)
       end
       if opts[:google]
-        Databasion.run('google', opts[:config], opts)
+        Databasion.run('google', opts)
       end
       if opts[:migrate]
-        Databasion.run('migrate', opts[:config], opts)
+        Databasion.run('migrate', opts)
       end
       if opts[:load]
-        Databasion.run('load', opts[:config], opts)
+        Databasion.run('load', opts)
       end
       if opts[:svn]
-        Databasion.run('svn', opts[:config], opts)
+        Databasion.run('svn', opts)
       end
       if opts[:git]
-        Databasion.run('git', opts[:config], opts)
+        Databasion.run('git', opts)
       end
     end
     

data/lib/databasion/build_migration.rb

@@ -10,18 +10,18 @@ module Databasion
     
     @@migration_start = 100
     
-    def self.run(file_list=[], config=nil)
+    def self.run(file_list=[], config=nil, opts=nil)
       raise BuildMigrationError, 'Databasion::BuildMigration requires an array list of files.  Try GoogleLoading first.' if file_list.empty?
       raise BuildMigrationError, 'Databasion::BuildMigration requires a parsed YAML config.' if config.nil?
       @@config = config
-      
+      @@environment = opts[:env]
       parse(file_list)
     end
     
     private
     def self.configure_start(dbname)
       @@migration_start = 100
-      files = Dir[@@config['output']['migrations']['path'] + "/%s/*.rb" % dbname].collect { |file| file.split("/").pop }.sort
+      files = Dir[@@environment + "/" + @@config['output']['migrations']['path'] + "/%s/*.rb" % dbname].collect { |file| file.split("/").pop }.sort
       @@migration_start = files[files.size-1].split("_")[0].to_i + 1 if files.size > 0
     end
     
@@ -47,10 +47,13 @@ module Databasion
     def self.migration_class(meta)
       template = ''
       File.open(File.expand_path(File.dirname(__FILE__)) + '/templates/migration.erb', 'r') { |f| template = f.read }
-      class_name = meta['name'].camelize
-      table_name = meta['name']
-      indexes = meta['indexes']
-      fields = meta['fields']
+      class_name  = meta['name'].camelize
+      table_name  = meta['name']
+      options     = meta['connection']['options']
+      indexes     = meta['indexes'] ? meta['indexes'] : []
+      primaries   = meta['primaries'] ? meta['primaries'] : []
+      fields      = meta['fields']
+      auto        = meta['auto']
 
       migration = ERB.new(template, nil, ">")
       migration.result(binding)
@@ -85,7 +88,7 @@ module Databasion
     end
     
     def self.write_migration(migration, file_name, sub_path)
-      path = @@config['output']['migrations']['path'] + "/" + sub_path
+      path = @@environment + "/" + @@config['output']['migrations']['path'] + "/" + sub_path
       check_output_path(path)
       unless migration_exists?(file_name)
         f = File.new("%s/%s_%s_migration.rb" % [path, @@migration_start, file_name], 'w')
@@ -100,8 +103,9 @@ module Databasion
     end
     
     def self.write_ruby(model, file_name)
-      check_output_path(@@config['output']['migrations']['models'])
-      f = File.new("%s/%s.rb" % [@@config['output']['migrations']['models'], file_name], 'w')
+      path = @@environment + "/" + @@config['output']['migrations']['models']
+      check_output_path(path)
+      f = File.new("%s/%s.rb" % [path, file_name], 'w')
       f.write(model)
       f.close
     end
@@ -113,19 +117,18 @@ module Databasion
         config.delete('dbname')
         output[dbname] = config
       end
-      f = File.open("config/database.yml", 'w')
+      f = File.open("config/database.%s.yml" % @@environment, 'w')
       f.write(YAML.dump(output))
       f.close
       Databasion::LOGGER.info "Wrote database config..."
     end
     
     def self.migration_exists?(file_name)
-      return true if find_migration_file(file_name)
-      false
+      find_migration_file(file_name) ? true : false
     end
     
     def self.find_migration_file(file_name)
-      files = Dir[@@config['output']['migrations']['path'] + "/**/*.rb"]
+      files = Dir[@@environment + "/" + @@config['output']['migrations']['path'] + "/**/*.rb"]
       files.each do |file|
         chunks = file.split("/").pop.split(".")[0].split("_")
         return file if chunks[1..chunks.size-2].join("_") == file_name

data/lib/databasion/cron_system.rb

@@ -8,6 +8,7 @@ module Databasion
     
     def self.config?
       raise CronSystemError, 'CronSystem cannot load without a config.' unless defined?(@@config)
+      raise CronSystemError, 'CronSystem now requires an environments config section.' unless defined?(@@config['environments'])
       true
     end
     
@@ -20,15 +21,17 @@ module Databasion
       @@config
     end
     
-    def self.run
+    def self.run(opts)
       Databasion::GoogleLoader.config = @@config
-      version = Databasion::GoogleLoader.run_version
+      version = Databasion::GoogleLoader.run_version(opts)
       
-      if File.exist?(@@config['cron']['version']['file'])
-        old_version = File.open(@@config['cron']['version']['file']).readline.strip
+      version_file = "%s/config/version_%s" % [@@config['project_base'], opts[:env]]
+      
+      if File.exist?(version_file)
+        old_version = File.open(version_file).readline.strip
         if version > old_version
           Databasion::LOGGER.info "Version changed, running databasion."
-          system "cd %s && databasion %s" % [@@config['cron']['project_base'], @@config['cron']['options']]
+          system "cd %s && %s %s" % [@@config['project_base'], File.dirname(__FILE__) + '/../../bin/databasion', @@config['environments'][opts[:env]]['cron_options']]
           write_version(version)
         elsif version < old_version
           Databasion::LOGGER.info "Version rollback is currently not implemented."
@@ -37,13 +40,13 @@ module Databasion
         end
       else
         Databasion::LOGGER.info "CronSystem running for the first time."
-        system "cd %s && databasion %s" % [@@config['cron']['project_base'], @@config['cron']['options']]
-        write_version(version)
+        system "cd %s && %s %s" % [@@config['project_base'], File.dirname(__FILE__) + '/../../bin/databasion', @@config['environments'][opts[:env]]['cron_options']]
+        write_version(version, version_file)
       end
     end
     
-    def self.write_version(version)
-      File.open(@@config['cron']['version']['file'], 'w') do |file|
+    def self.write_version(version, version_file)
+      File.open(version_file, 'w') do |file|
         file.write version
       end
     end

data/lib/databasion/google_loader.rb

@@ -5,15 +5,16 @@ module Databasion
   class GoogleLoaderError < StandardError; end
   
   class GoogleLoader
+    
+    @@session = nil
 
-    @@master_sheet  = 'Database'
-    @@version_sheet = 'Version'
+    @@environment_sheet = 'Environments'
     
-    @@table_def  = 'table'
-    @@field_def  = 'field'
-    @@type_def   = 'type'
-    @@index_def  = 'index'
-    @@ignore_def = 'ignore'
+    @@table_def   = 'table'
+    @@field_def   = 'field'
+    @@type_def    = 'type'
+    @@index_def   = 'index'
+    @@ignore_def  = 'ignore'
     
     def self.config?
       raise GoogleLoaderError, 'GoogleLoader cannot load without a config.' unless defined?(@@config)
@@ -33,20 +34,12 @@ module Databasion
       @@session
     end
     
-    def self.master_sheet=(master)
-      @@master_sheet = master
-    end
-    
-    def self.master_sheet
-      @@master_sheet
-    end
-    
-    def self.version_sheet=(version)
-      @@version_sheet = version
+    def self.environment_sheet=(environment)
+      @@environment_sheet = environment
     end
     
-    def self.version_sheet
-      @@version_sheet
+    def self.environment_sheet
+      @@environment_sheet
     end
     
     def self.login
@@ -57,18 +50,19 @@ module Databasion
       end
     end
     
-    def self.run
+    def self.run(opts=nil)
+      @@environment = opts[:env]
       config?
       login
       process.each do |data_hash|
-        Databasion::YamlBuilder.run(data_hash, @@config['output']['yaml_path'])
+        Databasion::YamlBuilder.run(data_hash, @@config, opts)
       end
     end
     
-    def self.run_version
+    def self.run_version(opts=nil)
       config?
-      login
-      fetch_version
+      login if session.nil?
+      fetch_environment_version(opts)
     end
     
     private
@@ -93,7 +87,7 @@ module Databasion
       master_list = []
       header_info = nil
       spreadsheet.worksheets.each do |worksheet|
-        if worksheet.title == master_sheet
+        if worksheet.title == @@environment
           worksheet.rows.each_with_index do |row, index|
             if index == 0
               header_info = row
@@ -113,12 +107,14 @@ module Databasion
     end
     
     def self.parse(worksheet)
-      name    = ''
-      plural  = true
-      fields  = []
-      types   = []
-      indexes = []
-      data    = []
+      name      = ''
+      plural    = true
+      auto      = false
+      fields    = []
+      primaries = []
+      types     = []
+      indexes   = {}
+      data      = []
 
       ignore_cols = []
       
@@ -137,15 +133,34 @@ module Databasion
           end
         when @@field_def
           row.each do |field|
-            fields.push field unless field.empty?
+            begin
+              unless field.empty?
+                d = field.split(",")
+                case d[1].strip
+                when 'primary'
+                  primaries.push d[0]
+                when 'auto'
+                  if d[0] == 'id'
+                    auto = true
+                  else
+                    primaries.push d[0]  
+                  end
+                end
+                fields.push d[0]
+              end
+            rescue
+              fields.push field unless field.empty?
+            end
           end
         when @@type_def
           row.each do |type|
             types.push type unless type.empty?
           end
         when @@index_def
-          row.each_with_index do |index, i|
-            indexes.push i-1 unless index.empty? or i == 0
+          row.each_with_index do |field, i|
+            unless field.empty? or i == 0
+              indexes.include?(field) ? (indexes[field].push i-1) : (indexes[field] ||= [i-1])
+            end
           end
         when @@ignore_def
           row.each_with_index do |ignore, i|
@@ -162,6 +177,8 @@ module Databasion
         'name'        => name,
         'plural'      => plural,
         'fields'      => fields[1..fields.size],
+        'auto'        => auto,
+        'primaries'   => primaries,
         'types'       => types[1..types.size],
         'indexes'     => indexes,
         'data'        => data,
@@ -169,19 +186,20 @@ module Databasion
       }
     end
     
-    def self.fetch_version
+    def self.fetch_environment_version(opts)
       version = nil
       @@config['sheets'].each do |token|
         spreadsheet = @@session.spreadsheet_by_key(token['key'])
         spreadsheet.worksheets.each do |worksheet|
-          if worksheet.title == version_sheet
-            worksheet.rows.each do |row|
-              version = row[0]
+          if worksheet.title == environment_sheet
+            worksheet.rows.each_with_index do |row, index|
+              next if index == 0
+              version = row[1] if row[0] == opts[:env]
             end
           end
         end
       end
-      raise GoogleLoaderError, "A Version spreadsheet was not found in any of the Google Spreadsheets supplied in google.yml" if version.nil?
+      raise GoogleLoaderError, "An Environments spreadsheet was not found in any of the Google Spreadsheets supplied in google.yml" if version.nil?
       version
     end
     

data/lib/databasion/load_data.rb

@@ -16,7 +16,7 @@ module Databasion
       Databasion.set_ar_logger
       Databasion::LOGGER.info "Updating from YAML..."
 
-      models = Dir[@@config['output']['migrations']['models'] + "/*.rb"].each { |file| load file }
+      models = Dir[opts[:env] + "/" + @@config['output']['migrations']['models'] + "/*.rb"].each { |file| load file }
 
       models.each do |model|
         f = model.split('/')
@@ -27,9 +27,9 @@ module Databasion
         Databasion::LOGGER.info "Loading %s into database..." % camel_name
 
         begin
-          yaml_file = YAML.load_file('%s/%s.yml' % [@@config['output']['yaml_path'], plural_name])
+          yaml_file = YAML.load_file('%s/%s.yml' % [opts[:env] + "/" + @@config['output']['yaml_path'], plural_name])
         rescue
-          yaml_file = YAML.load_file('%s/%s.yml' % [@@config['output']['yaml_path'], normal_name])
+          yaml_file = YAML.load_file('%s/%s.yml' % [opts[:env] + "/" + @@config['output']['yaml_path'], normal_name])
         end
 
         for row in yaml_file['data']

data/lib/databasion/migrate.rb

@@ -8,18 +8,18 @@ module Databasion
       @@config = config
     end
     
-    def self.run
+    def self.run(opts)
       require 'migration_helpers/init'
 
-      files = Dir["%s/*.yml" % @@config['output']['yaml_path']]
-      Databasion::BuildMigration.run(files, @@config)
+      files = Dir["%s/%s/*.yml" % [opts[:env], @@config['output']['yaml_path']]]
+      Databasion::BuildMigration.run(files, @@config, opts)
 
       Databasion.set_ar_logger
       Databasion::LOGGER.info "Migrating..."
 
-      YAML.load_file('config/database.yml').each do |config|
+      YAML.load_file('config/database.%s.yml' % opts[:env]).each do |config|
         ActiveRecord::Base.establish_connection(config[1])
-        path = @@config['output']['migrations']['path'] + "/" + config[0]
+        path = opts[:env] + "/" + @@config['output']['migrations']['path'] + "/" + config[0]
         ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
         ActiveRecord::Migrator.migrate(path, ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
       end

data/lib/databasion/templates/migration.erb

@@ -1,17 +1,15 @@
 class <%= class_name %>Migration < ActiveRecord::Migration
   def self.up
-    create_table :<%= table_name %><%= ', :id => false' if fields.collect {|f| f['field']}.include?('id') %> do |t|
+    create_table :<%= table_name %><%= ', :id => false' if !auto %><%= ', :options => "%s"' % options if options %> do |t|
     <% for field in fields %>
-      t.<%= field['type'] %> :<%= field['field'] %><%= ', :limit => %s' % field['size'] if field['size'] %><%= "\n" %>
+      t.<%= field['type'] %> :<%= field['field'] %><%= ', :limit => %s' % field['size'] if field['size'] %><%= ', :default => "%s"' % field['default'] if field['default'] %><%= "\n" %>
     <% end %>
     end
-    <% if fields.collect {|f| f['field']}.include?('id') %>
-    execute "ALTER TABLE <%= table_name %> ADD PRIMARY KEY (id)"
+    <% if primaries.size > 0 and !auto %>
+    execute "ALTER TABLE <%= table_name %> ADD PRIMARY KEY (<%= primaries.collect {|p| "%s, " % p }.to_s.strip.chop %>)"
     <% end %>
-    <% if indexes.size > 1 %>
-    add_index :<%= table_name %>, [<%= indexes.collect {|i| ":%s" % i }.join(",") %>]
-    <% elsif indexes.size == 1 %>
-    add_index :<%= table_name %>, :<%= indexes[0] %><%= "\n" %>
+    <% indexes.each do |index| %>
+    add_index :<%= table_name %>, [<%= index['index'].collect {|i| ":%s" % i }.join(", ") %>]
     <% end %>
   end
   def self.down

data/lib/databasion/yaml_builder.rb

@@ -6,9 +6,10 @@ module Databasion
   
   class YamlBuilder
     
-    def self.run(data_hash, output_path=nil)
-      raise YamalizeError, 'Databasion::YamlBuilder requires an output path.' if output_path.nil?
-      @@output_path = output_path
+    def self.run(data_hash, config=nil, opts=nil)
+      raise YamalizeError, 'Databasion::YamlBuilder requires an output path.' if config['output']['yaml_path'].nil?
+      @@config = config
+      @@environment = opts[:env]
       
       Databasion::LOGGER.info "Yamlbating %s..." % data_hash['name']
 
@@ -31,14 +32,19 @@ module Databasion
         yaml_output += "      size: %s\n" % type_data[1] if type_data[1]
         yaml_output += "      default: %s\n" % type_data[2] if type_data[2]
       end
-      indexes = ''
-      data_hash['indexes'].each_with_index do |index, i|
-        indexes += data_hash['fields'][i] + ","
+      yaml_output += "  auto: %s\n" % data_hash['auto']
+      yaml_output += "  indexes:\n"
+      data_hash['indexes'].each do |key, indexes|
+        index_list = []
+        indexes.each do |index|
+          index_list.push data_hash['fields'][index] unless data_hash['ignore_cols'].include?(index)
+        end
+        yaml_output += "    - index: [%s]\n" % index_list.join(", ")
       end
-      yaml_output += "  indexes: [%s]\n" % indexes.chop
+      yaml_output += "  primaries: [%s]\n" % data_hash['primaries'].join(", ") unless data_hash['primaries'].empty?
       yaml_output += "  connection:\n"
       data_hash['connection'].each do |key, value|
-        yaml_output += "    %s: %s\n" % [key, value] unless ['spreadsheet', 'options'].include?(key)
+        yaml_output += "    %s: %s\n" % [key, value] unless ['spreadsheet'].include?(key)
       end
       yaml_output += "\n"
     end
@@ -61,15 +67,16 @@ module Databasion
     end
     
     def self.write(file_name, yaml_output)
-      check_output_path
-      f = File.new("%s/%s.yml" % [@@output_path, file_name], 'w')
+      path = @@environment + "/" + @@config['output']['yaml_path']
+      check_output_path(path)
+      f = File.new("%s/%s.yml" % [path, file_name], 'w')
       f.write(yaml_output)
       f.close
     end
     
-    def self.check_output_path
-      unless File.exist?(@@output_path)
-        FileUtils.mkdir_p(@@output_path)
+    def self.check_output_path(path)
+      unless File.exist?(path)
+        FileUtils.mkdir_p(path)
       end
     end
     

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: databasion
 version: !ruby/object:Gem::Version 
-  hash: 25
+  hash: 23
   prerelease: false
   segments: 
   - 0
-  - 1
-  - 1
-  version: 0.1.1
+  - 2
+  - 0
+  version: 0.2.0
 platform: ruby
 authors: 
 - Brian Jones
@@ -16,7 +16,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-11-04 00:00:00 +09:00
+date: 2010-12-03 00:00:00 +09:00
 default_executable: databasion
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -59,12 +59,12 @@ dependencies:
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
-        hash: 25
+        hash: 31
         segments: 
         - 0
         - 1
-        - 1
-        version: 0.1.1
+        - 2
+        version: 0.1.2
   type: :runtime
   version_requirements: *id003
 - !ruby/object:Gem::Dependency 
@@ -94,6 +94,7 @@ extra_rdoc_files:
 - LICENSE
 - README.md
 files: 
+- CHANGELOG.md
 - LICENSE
 - README.md
 - Rakefile