mode 0.0.13 → 0.0.14

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d466ae670864e77dff5bb5e3beb559d3d8ab560d
-  data.tar.gz: 56f2cbb1e281c9733400843f283ffc707a6741f7
+  metadata.gz: 306d6b69e36902cfcfbbb337bd5fcb2a9ab34885
+  data.tar.gz: 1e37f39443b33605cc75857b0cb29ea7a4d3f645
 SHA512:
-  metadata.gz: 265a8928190c529947bb8837e173c3a7521ea70fba06b7e62ddf03019ffa20f423a5b51a3ab6c7bdce1f1506fb765f711c1a700f5cc32c880b368f2a9e4f0341
-  data.tar.gz: f2983257bcb269ebcc4282f98cbc53ef5a503e65e909f229e707c08e05424f083bfaa7169f6040e8b75f0964ccebc5c4e6e3162c3fac7c4c4d1cb90d934a03c4
+  metadata.gz: cc645a00b69a3fe58d1af485d1e538c6e19cfbf36f7ce4b4964fde02125ca0abbf72b64edeb7b5a32a1083c02d2b6152d39e5fde7aba53a7396b7eece18af562
+  data.tar.gz: 09acba6c1e90aad0627c1f71008e11da836a0fc83c02e847cef543c4c7f9bc6ef8310d76d8b544456eac0ec4051883cc17e453238bd1f90065581e16ba87263c

data/.gitignore

@@ -8,7 +8,6 @@ Gemfile.lock
 InstalledFiles
 _yardoc
 coverage
-doc/
 lib/bundler/man
 pkg
 rdoc

data/README.md

@@ -1,187 +1,132 @@
 # Mode
 
-This package provides command line tools for managing datasets and connecting data to Mode including:
-
-* Cloud data warehouse management
-* Personal data warehouse connectivity
-* Dataset formatting and importing (CSV)
-
+This gem provides command line tools for interacting with the Mode Analytics APIs.
 
 ## Install Mode
 
-**Mode requires Ruby 1.9 or newer**
+**Mode requires Ruby 1.9 or JRuby 1.7 or newer**
 
-If you don't currently have Ruby 1.9 or aren't sure then follow the directions for installing it before continuing.
+For the best performance and compatibility we recommend JRuby.
+
+If you don't currently have a suitable ruby or aren't sure then follow [these directions](doc/install_ruby.md) to install it before continuing.
 
 ```
 gem install mode
 ```
 
-## Install Ruby 1.9+ if it's not currently installed.
-
-This package requires at least Ruby 1.9 and Ruby 2.0 is recommended.
+## Logging In
 
-If you don't have an up to date version of Ruby or you're not sure then use the instructions below to get going.
+Before you use the mode gem you'll need to login.
 
-### Mac OSX
-
-To install Ruby on OSX complete the 4 steps below.
-
-1\. Install Homebrew
+The login process allows you to authorize the machine to connect via
+the API with the use of an API access token. The first time you use this
+command the it will look as follows:
 
 ```
-ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
-```
+$ mode login
+Enter your Mode credentials:
+Username: besquared
+Password: 
 
-2\. Install rbenv
+No API tokens found, would you like to create a new token? [y/n] y
 
-```
-brew update
-brew install rbenv ruby-build rbenv-gem-rehash
-echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
-source ~/.bash_profile
+Choose a token name: Connector
+Updated configuration at /Users/josh/.mode/config.yml
 ```
 
-3\. Install Ruby
+## Adding Data Sources
 
-Note: This usually takes several minutes
+Open the Mode configuration file in your favorite text editor (I'm using sublime text)
 
 ```
-rbenv install 2.0.0-p353
-rbenv global 2.0.0-p353
+$ subl ~/.mode/config.yml
 ```
 
-### OSX Combined
-
-For convenience you can just copy and paste all the lines at once into your terminal
+A sample data sources configuration is listed which you can copy to the bottom of your config file and modify:
 
 ```
-ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
-brew update
-brew install rbenv ruby-build rbenv-gem-rehash
-echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
-source ~/.bash_profile
-rbenv install 2.0.0-p353
-rbenv global 2.0.0-p353
-```
-
-### Windows
+data_sources:
+  development:
+    adapter: jdbc:postgresql
+    host: localhost
+    database: development
+    username: postgres
+    password:
 
-To install a current version of Ruby on windows complete the 2 steps below.
-
-1\. Install Ruby
-
-You can download the latest ruby version from [RubyInstaller](http://rubyinstaller.org/downloads/).
+  warehouse:
+    adapter: jdbc:redshift
+    host: hostname.redshift.amazonaws.com
+    port: 5439
+    database: dev
+    username: 
+    password: 
+```
 
-## Setup
+## Database Drivers
 
-### Init
+If you're using JRuby then you can load JDBC drivers by copying the driver's jar files into Mode's drivers directory
 
-Initializes a new configuration file at the specified path which holds API credentials and other information
+Example for teradata:
 
 ```
-$ mode login
-
-Initializing configuration at /Users/josh/.mode.yml
-Mode username: besquared
-Your can view your access tokens at http://www.modeanalytics.com/accounts/besquared/access_tokens
-Access token for besquared: ...
-Wrote configuration to /Users/josh/.mode.yml
+cp terajdbc4.jar ~/mode/drivers
+cp tdgssconfig.jar ~/mode/drivers
 ```
 
-## Working with CSV Data
+For ruby1.9+ you need to install the correct driver gem and libraries
 
-### Analyze
+```
+# PostgreSQL
+gem install pg
 
-The analyze command performs two useful functions:
+# Redshift
+gem install sequel_redshift
 
-* Ensures that the csv is well formed with no syntax errors
-* Tells you how Mode will recognize data types and format data
+# MySQL
+gem install mysql2
 
-For better performance analyze only inspects a sampled subset of file.
+# Oracle 9i+
+gem install oci8
 
-You can optionally set the sampling rate by passing the --sample option with a number between 0 and 1.
+# SQL Server & Azure
+gem install tiny_tds
 
+# DB2
+gem install ibm_db
 ```
-$ mode analyze bikeshare_small.csv
- 
-#  Analyzing bikeshare_small.csv (Sampling 12.70%)...
-#  Analyzed 12040 of 99999 rows
-+------------+-----+----------+------------+-------------+------------+---------------+-------------+-----------+
-| Field      | Key | Type     | String (%) | Integer (%) | Number (%) | Date/Time (%) | Boolean (%) | Empty (%) |
-+------------+-----+----------+------------+-------------+------------+---------------+-------------+-----------+
-| start_time | No  | datetime |            |             |            |       100.00% |             |           |
-| type       | No  | string   |    100.00% |             |            |               |             |           |
-+------------+-----+----------+------------+-------------+------------+---------------+-------------+-----------+
-```
-
-### Import
 
-More to come!
+## Verifying your API and driver configuration
 
-### Package
-
-The package command will analyze your dataset and create a data package based on the field types that are recognized.
-
-The data package can then be imported into the data warehouse.
+You can verify that your data sources are correct by using the verify command:
 
 ```
+$ mode connect verify
 ```
 
-### Package Format
-
-Mode packages are inspired by and compatible with [Open Knowledge Foundation Standards](http://data.okfn.org/standards), and specifically:
-
-* [Data Packages](http://data.okfn.org/standards/data-package)
-* [Simple Data Format](http://data.okfn.org/standards/simple-data-format)
+## Controlling the connector
 
-The most basic Mode data packages contains a normalized csv data file as well as a file describing the data.
+Starting
 
-* data.csv
-* datapackage.json
-
-Additionally they may contain a README.md, scripts and other resources. 
-
-#### datapackage.json
+```
+$ mode connect start
+```
 
-The datapackage.json file contains descriptive information of the data available in the package including naming, formatting and schema information.
+Restarting
 
 ```
-{
-  "name" : "my-dataset",
-  "version": 0.0.1,
-  "title": "a human friendly title"
+$ mode connect restart
+```
 
-  "resources": [
-    {
-      "name": "data",
-      "format": "csv",
-      "path": "data.csv",
+Stopping
 
-      "dialect": {
-        "delimiter": ",",
-        "quoteChar": "\"",
-        "doubleQuote": false,
-        "lineTerminator": "\r\n",
-        "skipInitialSpace": false
-      },
+```
+$ mode connect stop
+```
 
-      "schema": {
-        "fields": [
-          {
-            "name": "name of field (e.g. column name)",
-            "title": "A nicer human readable label or title for the field",
-            "type": "A string specifying the type",
-            "format": "A string specifying a format",
-            "description": "A description for the field"
-            ...
-          },
-        ],
+## Connector logging
 
-        "primaryKey": ['field1', 'field2', ...]
-      }
-    }
-  ]
-}
+The connect logs all its activity and errors to a log file that rotates weekly by default.
 
+```
+tail -f ~/.mode/log/mode.log
 ```

data/doc/install_ruby.md

@@ -0,0 +1,65 @@
+
+## Install Ruby 1.9+ if it's not currently installed.
+
+This package requires at least Ruby 1.9 and Ruby 2.0 is recommended.
+
+If you don't have an up to date version of Ruby or you're not sure then use the instructions below to get going.
+
+### Mac OSX
+
+To install Ruby on OSX complete the 4 steps below.
+
+1\. Install Homebrew
+
+```
+ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
+```
+
+2\. Install rbenv
+
+```
+brew update
+brew install rbenv ruby-build rbenv-gem-rehash
+echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
+source ~/.bash_profile
+```
+
+3\. Install Ruby or JRuby
+
+Note: This usually takes several minutes
+
+Ruby
+
+```
+rbenv install 2.0.0-p353
+rbenv global 2.0.0-p353
+```
+
+JRuby
+
+```
+rbenv install jruby-1.7.9
+rbenv global jruby-1.7.9
+```
+
+### OSX Combined
+
+For convenience you can just copy and paste all the lines at once into your terminal
+
+```
+ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
+brew update
+brew install rbenv ruby-build rbenv-gem-rehash
+echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
+source ~/.bash_profile
+rbenv install 2.0.0-p353
+rbenv global 2.0.0-p353
+```
+
+### Windows
+
+To install a current version of Ruby on windows complete the 2 steps below.
+
+1\. Install Ruby
+
+You can download the latest ruby version from [RubyInstaller](http://rubyinstaller.org/downloads/).

data/doc/packages.md

@@ -0,0 +1,94 @@
+## Working with CSV Data
+
+### Analyze
+
+The analyze command performs two useful functions:
+
+* Ensures that the csv is well formed with no syntax errors
+* Tells you how Mode will recognize data types and format data
+
+For better performance analyze only inspects a sampled subset of file.
+
+You can optionally set the sampling rate by passing the --sample option with a number between 0 and 1.
+
+```
+$ mode analyze bikeshare_small.csv
+ 
+#  Analyzing bikeshare_small.csv (Sampling 12.70%)...
+#  Analyzed 12040 of 99999 rows
++------------+-----+----------+------------+-------------+------------+---------------+-------------+-----------+
+| Field      | Key | Type     | String (%) | Integer (%) | Number (%) | Date/Time (%) | Boolean (%) | Empty (%) |
++------------+-----+----------+------------+-------------+------------+---------------+-------------+-----------+
+| start_time | No  | datetime |            |             |            |       100.00% |             |           |
+| type       | No  | string   |    100.00% |             |            |               |             |           |
++------------+-----+----------+------------+-------------+------------+---------------+-------------+-----------+
+```
+
+### Import
+
+More to come!
+
+### Package
+
+The package command will analyze your dataset and create a data package based on the field types that are recognized.
+
+The data package can then be imported into the data warehouse.
+
+```
+```
+
+### Package Format
+
+Mode packages are inspired by and compatible with [Open Knowledge Foundation Standards](http://data.okfn.org/standards), and specifically:
+
+* [Data Packages](http://data.okfn.org/standards/data-package)
+* [Simple Data Format](http://data.okfn.org/standards/simple-data-format)
+
+The most basic Mode data packages contains a normalized csv data file as well as a file describing the data.
+
+* data.csv
+* datapackage.json
+
+Additionally they may contain a README.md, scripts and other resources. 
+
+#### datapackage.json
+
+The datapackage.json file contains descriptive information of the data available in the package including naming, formatting and schema information.
+
+```
+{
+  "name" : "my-dataset",
+  "version": 0.0.1,
+  "title": "a human friendly title"
+
+  "resources": [
+    {
+      "name": "data",
+      "format": "csv",
+      "path": "data.csv",
+
+      "dialect": {
+        "delimiter": ",",
+        "quoteChar": "\"",
+        "doubleQuote": false,
+        "lineTerminator": "\r\n",
+        "skipInitialSpace": false
+      },
+
+      "schema": {
+        "fields": [
+          {
+            "name": "name of field (e.g. column name)",
+            "title": "A nicer human readable label or title for the field",
+            "type": "A string specifying the type",
+            "format": "A string specifying a format",
+            "description": "A description for the field"
+            ...
+          },
+        ],
+
+        "primaryKey": ['field1', 'field2', ...]
+      }
+    }
+  ]
+}

data/lib/mode/config.rb

@@ -86,12 +86,20 @@ module Mode
       end
     end
 
+    def data_source_hash
+      hsh = {}
+      data_sources.each do |data_source|
+        hsh[data_source.name] = data_source.props
+      end
+      hsh
+    end
+
     def to_yaml
       {
         'username' => username,
         'access_token' => access_token,
         'environment' => environment,
-        'data_sources' => data_sources
+        'data_sources' => data_source_hash
       }.to_yaml
     end
   end

data/lib/mode/connector/data_source.rb

@@ -64,7 +64,7 @@ module Mode
       def redshift?
         ['redshift', 'jdbc:redshift'].include?(adapter)
       end
-
+      
       private
 
       def adapter_opts

data/lib/mode/version.rb

@@ -1,3 +1,3 @@
 module Mode
-  VERSION = "0.0.13"
+  VERSION = "0.0.14"
 end

data/spec/api/request_spec.rb

@@ -46,7 +46,7 @@ describe Mode::API::Request do
       'test'        => 'test',
       'development' => 'localhost',
       'staging'     => 'staging.modeanalytics.com',
-      'production'  => 'www.modeanalytics.com'
+      'production'  => 'stealth.modeanalytics.com'
     }
 
     hosts.each do |env, host|

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mode
 version: !ruby/object:Gem::Version
-  version: 0.0.13
+  version: 0.0.14
 platform: ruby
 authors:
 - Mode Analytics
@@ -236,6 +236,8 @@ files:
 - README.md
 - Rakefile
 - bin/mode
+- doc/install_ruby.md
+- doc/packages.md
 - lib/connect.rb
 - lib/mode.rb
 - lib/mode/api/form.rb