spout 0.1.0.rc2 → 0.1.0.rc3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 107e09d3629aadfb7fd0e06b27c115a2d0e87fbd
4
- data.tar.gz: 62ea6faceb8218158e870aebedb42ac3047a2a66
3
+ metadata.gz: eac7961655c4b0c02a08c9f5265be806f1ba8d29
4
+ data.tar.gz: 70a68bf828fb2fc2cfa9ab70fa2cdc49d0d06146
5
5
  SHA512:
6
- metadata.gz: 3d5710e58b74e66277fa1dd0c6a85fbf63edbea619e5b51c5b7720b00f786d7c43586e07f99733432f61c06ac6f83f8daabcf56d12b924a67acb553bc8bfae3b
7
- data.tar.gz: a94c8b3c1bfd03586f62079dd01d91cd4563f2fbed2fabaf34d900ea2bc1d5801c22a8ad61846948b6ca80bce0d535aea8a9d741f82b86a9e3fd2ab5d8aee8bd
6
+ metadata.gz: bd6fa9f07d148ccc176188be95f5d86a36d5bd0a0f333f5a8c06ab7c516ca8147b6b08f8fab3b2f5de7341251d8faeb7b519db96f5862c8ddfec43ebb54bffc1
7
+ data.tar.gz: 7fa9bb7618a1eccad4eee36ebc11cf44335c64ae951ef476245d865f1931b82fed90bd9e9f12ccb2fcbba2cacfe76e056d408a6fc92cc88e7d0d6a8162ca19f3
data/README.md CHANGED
@@ -28,6 +28,38 @@ cd my_data_dictionary
28
28
  spout import data_dictionary.csv
29
29
  ```
30
30
 
31
+ The CSV should contain at minimal the two column headers:
32
+
33
+ `id`: This column will give the variable it's name, and also be used to name the file, i.e. `<id>.json`
34
+ `folder`: This can be blank, however it is used to place variables into a folder hiearchy. The folder column can contain forward slashes `/` to place a variable into a subfolder. An example may be, `id`: `myvarid`, `folder`: `Demographics/Subfolder` would create a file `variables/Demographics/Subfolder/myvarid.json`
35
+
36
+ Other columns that will be interpreted include:
37
+
38
+ `display_name`: The variable name as it is presented to the user. The display name should be reasonable in length and not unreasonably long.
39
+
40
+ `description`: A longer description of the variable.
41
+
42
+ `type`: Should be a valid variable type, i.e.:
43
+ - `identifier`
44
+ - `choices`
45
+ - `integer`
46
+ - `numeric`
47
+ - `string`
48
+ - `text`
49
+ - `date`
50
+ - `time`
51
+ - `file`
52
+
53
+ `domain`: The name of the domain that is associated with the variable. Typically, only variable of type `choices` have domains. These domains then reside in `domains` folder.
54
+
55
+ `units`: A string of the associated that are appended to variable values, or added to coordinates in graphs representing the variable.
56
+
57
+ `calculation`: A calculation represented using algebraic expressions along with `id` of other variables.
58
+
59
+ `labels`: A series of different names for the variable that are semi-colon `;` separated. These labels are commonly synonyms, or related terms used primarily for searching.
60
+
61
+ All other columns get grouped into a hash labeled `other`.
62
+
31
63
  ### Test your repository
32
64
 
33
65
  If you created your data dictionary repository using `spout new`, you can go ahead and test using:
@@ -69,6 +101,12 @@ Provide an optional version parameter to name the folder the CSVs will be genera
69
101
  spout export
70
102
  ```
71
103
 
104
+ You can optionally provide a version string
105
+
106
+ ```
107
+ spout export [1.0.0]
108
+ ```
109
+
72
110
  or
73
111
 
74
112
  ```
data/bin/spout CHANGED
@@ -2,7 +2,4 @@
2
2
 
3
3
  require File.expand_path('../../lib/spout', __FILE__)
4
4
 
5
- if ARGV.size > 0
6
- Spout::Actions.new.interpret(ARGV)
7
- exit(0)
8
- end
5
+ Spout::Actions.new.interpret(ARGV)
data/lib/spout/actions.rb CHANGED
@@ -3,18 +3,18 @@ module Spout
3
3
 
4
4
  def interpret(argv)
5
5
  case argv.first
6
- when 'new'
6
+ when 'new', 'n', 'ne', '-new', '-n', '-ne'
7
7
  new_template_dictionary(argv)
8
- when '--version', '-v', 'version'
8
+ when '--version', '-v', '-ve', '-ver', 'version', 'v', 've', 'ver'
9
9
  puts "Spout #{Spout::VERSION::STRING}"
10
- when 'help', '--help', '-h'
11
- help
12
- when 'import'
10
+ when 'test', 't', 'te', 'tes', '--test', '-t', '-te', '-tes'
11
+ system "bundle exec rake"
12
+ when 'import', 'i', 'im', 'imp', '--import', '-i', '-im', '-imp'
13
13
  import_from_csv(argv)
14
- when 'export'
15
- system "bundle exec rake dd:create"
14
+ when 'export', 'e', 'ex', 'exp', '--export', '-e', '-ex', '-exp'
15
+ new_data_dictionary_export(argv)
16
16
  else
17
- system "bundle exec rake"
17
+ help
18
18
  end
19
19
  end
20
20
 
@@ -43,17 +43,25 @@ EOT
43
43
  Usage: spout COMMAND [ARGS]
44
44
 
45
45
  The most common spout commands are:
46
- new Create a new Spout dictionary. "spout new my_dd" creates a
46
+ [n]ew Create a new Spout dictionary. "spout new my_dd" creates a
47
47
  new data dictionary called MyDD in "./my_dd"
48
- test Running the test file (short-cut alias: "t")
49
- import Import a CSV file into the JSON repository
50
- export Export the JSON respository to a CSV
51
- version Returns the version of Spout
48
+ [t]est Running the test file
49
+ [i]mport Import a CSV file into the JSON repository
50
+ [e]xport Export the JSON respository to a CSV
51
+ [v]ersion Returns the version of Spout
52
+
53
+ Each command can be referenced by the first letter: Ex: `spout t`, for test
52
54
 
53
55
  EOT
54
56
  puts help_message
55
57
  end
56
58
 
59
+ def new_data_dictionary_export(argv)
60
+ version = argv[1].to_s.gsub(/[^a-zA-Z0-9\.-]/, '_').strip
61
+ version_string = (version == '' ? "" : "VERSION=#{version}")
62
+ system "bundle exec rake dd:create #{version_string}"
63
+ end
64
+
57
65
  def new_template_dictionary(argv)
58
66
  @full_path = File.join(argv[1].to_s.strip)
59
67
  usage = <<-EOT
@@ -64,7 +72,7 @@ The FOLDER must be empty or new.
64
72
 
65
73
  EOT
66
74
 
67
- if Dir.exists?(@full_path) and (Dir.entries(@full_path) - ['.', '..']).size > 0
75
+ if @full_path == '' or (Dir.exists?(@full_path) and (Dir.entries(@full_path) - ['.', '..']).size > 0)
68
76
  puts usage
69
77
  exit(0)
70
78
  end
@@ -81,7 +89,7 @@ EOT
81
89
  directory 'test'
82
90
  copy_file 'test/dictionary_test.rb'
83
91
  copy_file 'test/test_helper.rb'
84
- puts " run bundle install"
92
+ puts " run".colorize( :green ) + " bundle install".colorize( :blue )
85
93
  Dir.chdir(@full_path)
86
94
  system "bundle install"
87
95
  end
@@ -92,13 +100,13 @@ EOT
92
100
  file_name = template_file if file_name == ''
93
101
  file_path = File.join(@full_path, file_name)
94
102
  template_file_path = File.join(File.expand_path(File.dirname(__FILE__)), "templates", template_file)
95
- puts " create #{file_name}"
103
+ puts " create".colorize( :green ) + " #{file_name}"
96
104
  FileUtils.copy(template_file_path, file_path)
97
105
  end
98
106
 
99
107
  def directory(directory_name)
100
108
  directory_path = File.join(@full_path, directory_name)
101
- puts " create #{directory_name}"
109
+ puts " create".colorize( :green ) + " #{directory_name}"
102
110
  FileUtils.mkpath(directory_path)
103
111
  end
104
112
 
@@ -1,4 +1,5 @@
1
1
  require 'rake/testtask'
2
+ require 'colorize'
2
3
 
3
4
  Rake::TestTask.new do |t|
4
5
  t.libs << "test"
@@ -33,12 +34,13 @@ namespace :dd do
33
34
  end
34
35
  CSV.open("#{folder}/domains.csv", "wb") do |csv|
35
36
  keys = %w(value display_name description)
36
- csv << ['id'] + keys
37
+ csv << ['folder', 'id'] + keys
37
38
  Dir.glob("domains/**/*.json").each do |file|
38
39
  if json = JSON.parse(File.read(file)) rescue false
39
- domain_name = file.gsub(/domains\//, '')
40
+ domain_folder = file.gsub(/domains\//, '').split('/')[0..-2].join('/')
41
+ domain_name = file.gsub(/domains\//, '').split('/').last.to_s.gsub(/.json/, '')
40
42
  json.each do |hash|
41
- csv << [domain_name] + keys.collect{|key| hash[key]}
43
+ csv << [domain_folder, domain_name] + keys.collect{|key| hash[key]}
42
44
  end
43
45
  end
44
46
  end
@@ -49,12 +51,18 @@ namespace :dd do
49
51
 
50
52
  desc 'Initialize JSON repository from a CSV file: CSV=datadictionary.csv'
51
53
  task :import do
54
+ additional_csv_info = "\n\nFor additional information on specifying CSV column headers before import see:\n\n " + "https://github.com/sleepepi/spout#generate-a-new-repository-from-an-existing-csv-file".colorize( :blue ).underline + "\n\n"
55
+
52
56
  puts ENV['CSV'].inspect
53
57
  if File.exists?(ENV['CSV'].to_s)
54
58
  CSV.parse( File.open(ENV['CSV'].to_s, 'r:iso-8859-1:utf-8'){|f| f.read}, headers: true ) do |line|
55
59
  row = line.to_hash
60
+ if not row.keys.include?('id')
61
+ puts "\nMissing column header `id` in data dictionary.".colorize( :red ) + additional_csv_info
62
+ exit(1)
63
+ end
56
64
  next if row['id'] == ''
57
- folder = File.join('variables', row.delete('folder').to_s.gsub(':', '/'))
65
+ folder = File.join('variables', row.delete('folder').to_s)
58
66
  FileUtils.mkpath folder
59
67
  hash = {}
60
68
  id = row.delete('id')
@@ -70,16 +78,16 @@ namespace :dd do
70
78
  hash['calculation'] = calculation if calculation != ''
71
79
  labels = row.delete('labels').to_s.split(';')
72
80
  hash['labels'] = labels if labels.size > 0
73
- hash['other'] = row
81
+ hash['other'] = row unless row.empty?
74
82
 
75
83
  file_name = File.join(folder, id.downcase + '.json')
76
84
  File.open(file_name, 'w') do |file|
77
85
  file.write(JSON.pretty_generate(hash))
78
86
  end
79
- puts " create #{file_name}"
87
+ puts " create".colorize( :green ) + " #{file_name}"
80
88
  end
81
89
  else
82
- puts "Please specify a valid CSV file."
90
+ puts "\nPlease specify a valid CSV file.".colorize( :red ) + additional_csv_info
83
91
  end
84
92
  end
85
93
  end
data/lib/spout/tests.rb CHANGED
@@ -6,12 +6,14 @@ require 'json'
6
6
  require 'spout/tests/domain_existence_validation'
7
7
  require 'spout/tests/json_validation'
8
8
  require 'spout/tests/variable_type_validation'
9
+ require 'spout/tests/domain_format'
9
10
 
10
11
  module Spout
11
12
  module Tests
12
13
  include Spout::Tests::JsonValidation
13
14
  include Spout::Tests::VariableTypeValidation
14
15
  include Spout::Tests::DomainExistenceValidation
16
+ include Spout::Tests::DomainFormat
15
17
 
16
18
  Turn.config.trace = 1
17
19
  end
@@ -6,8 +6,7 @@ module Spout
6
6
  result = begin
7
7
  domain_name = JSON.parse(File.read(item))["domain"]+".json"
8
8
  File.exists?(File.join("domains", domain_name))
9
- rescue JSON::ParserError => e
10
- error = e
9
+ rescue JSON::ParserError
11
10
  false
12
11
  end
13
12
  full_message = build_message(msg, "The domain \"domains/#{domain_name}\" referenced by ? does not exist.", item)
@@ -0,0 +1,31 @@
1
+ module Spout
2
+ module Tests
3
+ module DomainFormat
4
+
5
+ def assert_domain_format(item, msg = nil)
6
+ result = begin
7
+ json = JSON.parse(File.read(item))
8
+ if json.kind_of?(Array)
9
+ json.empty? or json.select{|o| not o.kind_of?(Hash)}.size == 0
10
+ else
11
+ false
12
+ end
13
+ rescue JSON::ParserError
14
+ false
15
+ end
16
+
17
+ full_message = build_message(msg, "Must be an array of choice hashes. Ex:\n[\n {\n \"value\": \"1\",\n \"display_name\": \"Option 1\",\n \"description\": \"...\"\n },\n { ... },\n ...\n]")
18
+ assert_block(full_message) do
19
+ result
20
+ end
21
+ end
22
+
23
+ Dir.glob("domains/**/*.json").each do |file|
24
+ define_method("test_json: "+file) do
25
+ assert_domain_format file
26
+ end
27
+ end
28
+
29
+ end
30
+ end
31
+ end
@@ -1,7 +1,7 @@
1
1
  module Spout
2
2
  module Tests
3
3
  module VariableTypeValidation
4
- VALID_VARIABLE_TYPES = ['identifier', 'choices', 'integer', 'numeric'].sort
4
+ VALID_VARIABLE_TYPES = ['identifier', 'choices', 'integer', 'numeric', 'string', 'text', 'date', 'time', 'file'].sort
5
5
 
6
6
  def assert_variable_type(item, msg = nil)
7
7
  full_message = build_message(msg, "? invalid variable type. Valid types: #{VALID_VARIABLE_TYPES.join(', ')}", item)
data/lib/spout/version.rb CHANGED
@@ -3,7 +3,7 @@ module Spout
3
3
  MAJOR = 0
4
4
  MINOR = 1
5
5
  TINY = 0
6
- BUILD = "rc2" # nil, "pre", "rc", "rc2"
6
+ BUILD = "rc3" # nil, "pre", "rc", "rc2"
7
7
 
8
8
  STRING = [MAJOR, MINOR, TINY, BUILD].compact.join('.')
9
9
  end
data/spout.gemspec CHANGED
@@ -29,6 +29,7 @@ Gem::Specification.new do |spec|
29
29
  spec.add_dependency "rake"
30
30
  spec.add_dependency "turn"
31
31
  spec.add_dependency "json"
32
+ spec.add_dependency "colorize", "~> 0.5.8"
32
33
 
33
34
  spec.add_development_dependency "bundler", "~> 1.3"
34
35
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: spout
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0.rc2
4
+ version: 0.1.0.rc3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Remo Mueller
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2013-05-16 00:00:00.000000000 Z
11
+ date: 2013-05-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -52,6 +52,20 @@ dependencies:
52
52
  - - '>='
53
53
  - !ruby/object:Gem::Version
54
54
  version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: colorize
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ~>
60
+ - !ruby/object:Gem::Version
61
+ version: 0.5.8
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ~>
67
+ - !ruby/object:Gem::Version
68
+ version: 0.5.8
55
69
  - !ruby/object:Gem::Dependency
56
70
  name: bundler
57
71
  requirement: !ruby/object:Gem::Requirement
@@ -88,6 +102,7 @@ files:
88
102
  - lib/spout/templates/test/test_helper.rb
89
103
  - lib/spout/templates/travis.yml
90
104
  - lib/spout/tests/domain_existence_validation.rb
105
+ - lib/spout/tests/domain_format.rb
91
106
  - lib/spout/tests/json_validation.rb
92
107
  - lib/spout/tests/variable_type_validation.rb
93
108
  - lib/spout/tests.rb