datahunter 0.2.4 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 881a15306c3ad3eb1b348a3121ac63af9f78f1bd
4
- data.tar.gz: 07f03917a80eebe1bcbcec00f204154f902b2419
3
+ metadata.gz: e9919f3ad7ae09cbb6e8c15d69ad969b6a7d540f
4
+ data.tar.gz: 3ab09545715106aa731338edb0bbb5803d8956e5
5
5
  SHA512:
6
- metadata.gz: 8ef1c9279257884f89a2d2d8b25592ce57a42ac67d69d07fceb222498a71c01f8c56a0cfcdea05a04851b9f112675503b231f3d213baf8896a24992c10f92ec1
7
- data.tar.gz: 6e4d1753b61b845c0e946f124ae8b61b323dd4aaadc7ac54b5fca5d66202553a1118e250ae6c350d889e8959179e663a373e71a12ffcd0ab680bf4c1a94ecfba
6
+ metadata.gz: 06a00c3e9c22884397dcb87563ffeb603e0dd2c382f0e3723a9825d9dbe449b0f1e14d93856a14b72269c1030e24fb2880c81af42691e7aea748752868108296
7
+ data.tar.gz: 367e8b39398dd88bff9d00d6859ca1ef5aeec44b87393670dff8e1363f62c9b3ad53b18febbcb7acf21d18c8a8cb312313425a776f54b126b008ce9bb7c15078
data/README.md CHANGED
@@ -25,67 +25,56 @@ We believe that pre-processing data is a bad idea and that hosting them could cr
25
25
  ### $ hunter find
26
26
 
27
27
  $ hunter find consumer
28
- ### Response in 0.783677 seconds
29
- ### Looks like we've got something for you!
28
+ ### Response in 0.894806 seconds
29
+ ### We've found 34 datasets corresponding to your query consumer:
30
30
 
31
- title: Consumer Complaint Database
32
- description: These are complaints we’ve received about financial products and services.
33
- publisher: Consumer Financial Protection Bureau
34
- temporal: ["2011", "2012", "2013"]
35
- spatial: ["us", "usa", "america", "united states", "united-states", "united states of america", "united-states-of-america", "world", "countries", "all"]
36
- created: 2014-02-25T18:48:25.192Z
37
- updated: 2014-02-25T23:48:24.897Z
38
- score: 20.743
31
+ 1. Consumer Complaint Database id: 54de229aa82653913d1142ee
32
+ ["us", "usa", "america", "united states", "united-states"]
33
+ These are complaints we’ve received about financial products and services.
34
+ 2. Consumer Expenditure Survey id: 54de229ba82653913d1142f8
35
+ ["us", "usa", "america", "united states", "united-states"]
36
+ The Consumer Expenditure Survey (CE) program consists of two surveys, the quarterly Interview ...
37
+ 3. Farmers Markets Geographic Data id: 54de229ba82653913d1142fe
38
+ ["us", "usa", "america", "united states", "united-states"]
39
+ longitude and latitude, state, address, name, and zip code of Farmers Markets in the United ...
40
+ 4. Consumer Price Indices id: 54e8c7dfa8268b5c235ac352
41
+ ["england", "scotland", "wales", "northern_ireland"]
42
+ Contains price indices, percentage changes and weights for the Consumer Prices Index (CPI), Retail ...
43
+ 5. Organogram and staff pay data for the Consumer Council for Water id: 54e8c872a8268b5c235ac5d9
44
+ ["uk", "england", "scotland", "wales", "ireland"]
45
+ A list of most Senior Civil Service posts in the Consumer Council for Water including title, ...
39
46
 
40
- ### get the data? (y/n)
41
- y
47
+ ### GET A DATASET ? (1/..5), show next 5 datasets? (RET) or abort? (abort)
48
+ 1
42
49
  0. Consumer Complaint Database - CSV
43
50
  1. Consumer Complaint Database - JSON
44
51
  2. Consumer Complaint Database - XML
52
+ 3. Consumer Complaint Database - api
45
53
  ### which one? (0/1/...)
46
- 0
54
+ 1
55
+ Create/overwrite /Users/Terpolilli/views.json?(y/rename/n)
56
+ y
47
57
  Start downloading...
48
- Your file has been downloaded, try to $ ls ;D
58
+ Your file has been downloaded ;)
49
59
  If this is not the file you expected, it's maybe because publisher don't always keep the metadata up-to-date. We try to clean most of uri's and check the url. Anyway you may be able to download your file by hand here:
50
60
  https://data.consumerfinance.gov/api/views/x94z-ydhh/rows.csv?accessType=DOWNLOAD
51
61
 
52
- ![usage](./hunter.gif)
53
-
54
- ### $ hunter search
55
-
56
- $ hunter search consumer
57
- ### Response in 0.402504 seconds
58
- ### We've found 12 datasets corresponding to your query consumer:
59
-
60
- - Consumer Complaint Database id: 548c82a7a826dfe85070e5fa
61
- - Consumer Expenditure Survey id: 548c82a9a826dfe85070e607
62
- - Farmers Markets Geographic Data id: 548c82aaa826dfe85070e60e
63
- - Food Price Outlook id: 548c82aba826dfe85070e624
64
- - Consumer Price Index id: 548c82fca826dfe85070e79f
65
- - Foreign Labor Statistics id: 548c82ada826dfe85070e663
66
- - Farmers Markets Search id: 548c82b1a826dfe85070e6f2
67
- - International Macroeconomic Data Set id: 548c8301a826dfe85070e84e
68
- - Consumer Expenditure Survey id: 548c82fba826dfe85070e782
69
- - Consumer Price Indexes - Base id: 548c82fca826dfe85070e7b5
70
- - Consumer Price Index - Department Store Inventory Price Index id: 548c82fea826dfe85070e7f4
71
- - Southeast Information Office id: 548c8305a826dfe85070e8cf
72
-
73
- You can get more informations about a dataset with $ hunter info <id>
74
- You can download directly a dataset with $ hunter get <id>
75
-
62
+ ![demo](hunter.gif)
63
+
76
64
  ### $ hunter info
77
65
 
78
- $ hunter info 548c82a7a826dfe85070e5fa
66
+ $ hunter info 54de229aa82653913d1142ee
67
+
68
+ Consumer Complaint Database
69
+ These are complaints we’ve received about financial products and services.
79
70
 
80
- title: Consumer Complaint Database
81
- description: These are complaints we’ve received about financial products and services.
82
71
  publisher: Consumer Financial Protection Bureau
83
72
  temporal: ["2011", "2012", "2013"]
84
73
  spatial: ["us", "usa", "america", "united states", "united-states", "united states of america", "united-states-of-america", "world", "countries", "all"]
85
- created: 2014-02-25T18:48:25.192Z
86
- updated: 2014-02-25T23:48:24.897Z
87
- score: 20.743
88
-
74
+ created: 2014-02-25T18:48:25.000Z
75
+ updated: 2015-01-30T15:16:28.000Z
76
+ score: 26.683
77
+
89
78
  ### $ hunter get
90
79
 
91
80
  $ hunter get 548c82a7a826dfe85070e5fa
@@ -95,11 +84,11 @@ We believe that pre-processing data is a bad idea and that hosting them could cr
95
84
  2. Consumer Complaint Database - XML
96
85
  ### which one? (0/1/...)
97
86
  1
98
- Create/overwrite /Users/Terpolilli/views.json?(y/rename/n)
87
+ ### Create/overwrite /Users/Terpolilli/views.json?(y/rename/n)
99
88
  rename
100
89
  Path/to/filename: /Users/Terpolilli/Downloads/consumer-data.json
101
- Start downloading...
102
- Your file has been downloaded ;)
90
+ ### Start downloading...
91
+ ### Your file has been downloaded ;)
103
92
  If this is not the file you expected, it's maybe because publisher don't always keep the metadata up-to-date. We try to clean most of uri's and check the url. Anyway you may be able to download your file by hand here:
104
93
  https://data.consumerfinance.gov/api/views/x94z-ydhh/rows.csv?accessType=DOWNLOAD
105
94
 
@@ -108,13 +97,22 @@ Don't hesitate to [give us any feedback about you experience with Hunter!](https
108
97
 
109
98
  ## Update
110
99
 
111
- * datasets indexed: 8335
112
- * last datasets indexed: Canada open data, NETL's Energy Data eXchange, dati.gov.it
100
+ * datasets indexed: 8336
101
+ * last datasets indexed: Canada open data, NETL's Energy Data eXchange, dati.gov.it, complete french health DAMIR data.
102
+
103
+ ## Change Log
104
+
105
+ * 0.3.x - Merge `$ hunter find <keyword>` and `$ hunter search <keyword>` commands.
106
+ The new `$ hunter find` command displays the datasets corresponding to the query, 5 by 5,
107
+ sorted by popularity
108
+ * 0.2.x - New feature: download datasets directly from the CLI
109
+ * 0.1.x
113
110
 
114
111
  ## Roadmap
115
112
 
116
113
  * more datasets
117
114
  * improve the download of datasets
115
+ * searched and downloaded data management
118
116
  * more generic lib to make the API more easy to reuse
119
117
 
120
118
  ## Contributing
data/bin/hunter CHANGED
@@ -9,74 +9,65 @@ require 'launchy'
9
9
  require 'colorize'
10
10
  require 'downloadr'
11
11
 
12
+ program :name, 'Hunter'
12
13
  program :version, Datahunter::VERSION
13
14
  program :description, 'Find an download Open Datasets directly from your terminal'
14
15
  program :help_formatter, :compact
15
16
 
16
17
  command :find do |c|
17
18
  c.syntax = 'hunter find keyword [spatial-coverage] [temporal-coverage]'
18
- c.summary = '$ hunter find <keyword> <geo-coverage> <temporal-coverage>
19
- * Returns the 3 most popular (most reused, viewed, etc)
20
- datasets corresponding to the triple keyword,
21
- spatial-coverage and temporal-coverage
22
- * Allow you to either download the dataset or to open
23
- your favorite browser directly at the good page'.colorize(:blue)
19
+ c.summary = 'Displays the datasets corresponding to the query'.colorize(:blue)
24
20
  c.description = c.summary
25
- c.example 'Find open data for the U.S. population', 'hunter find population us'
26
21
  c.action do |args, options|
27
-
22
+
28
23
  start_time = Time.now
29
24
 
30
25
  if args[0].nil?
31
26
  puts "usage: $ hunter find keyword [geo] [temp]"
32
27
  elsif
33
- url = Datahunter.datasets_url(args[0], args[1], args[2])
34
28
 
29
+ url = Datahunter.datasets_url(args[0], args[1], args[2])
35
30
  response = RestClient.get(url, :content_type => :json) do |response, request, result, &block|
36
31
  elapsed_time = Time.now - start_time
37
32
 
38
33
  case response.code
39
34
  when 200
40
35
  datasets = JSON.parse(response.body).reverse
36
+ number_of_datasets = datasets.length
37
+
41
38
  puts "### Response in #{elapsed_time} seconds".colorize(:blue)
39
+ puts "### We've found #{datasets.size} datasets corresponding to your query #{args[0].downcase}:".colorize(:blue)
40
+ puts
42
41
 
43
- messages = [
44
- "### Looks like we've got something for you!".colorize(:blue),
45
- "### And this one?".colorize(:blue),
46
- "### Last try...".colorize(:blue)
47
- ]
42
+ for i in 1..(number_of_datasets/5 + 1)
43
+ sub_datasets = datasets[(5 * i - 5) .. (5 * i - 1)]
48
44
 
49
- 3.times do |index|
50
- puts messages[index]
51
- puts
52
- Datahunter.print_dataset_info datasets[index]
45
+ Datahunter.print_coll_of_datasets_info_light sub_datasets
53
46
 
54
- case ask "### get the data? (y/n)".colorize(:yellow)
55
- when 'y'
56
- if datasets[index].has_key?("resources") and datasets[index]["resources"].any?
57
- Datahunter.download_the_data datasets[index]
58
- else
59
- Datahunter.open_in_browser datasets[index]["uri"]
60
- end
47
+ case ask "### GET A DATASET ? (1/..5), show next 5 datasets? (RET) or abort? (abort)".colorize(:yellow)
48
+ when '1'
49
+ Datahunter.get_dataset sub_datasets[0]
50
+ break
51
+ when '2'
52
+ Datahunter.get_dataset sub_datasets[1]
53
+ break
54
+ when '3'
55
+ Datahunter.get_dataset sub_datasets[2]
56
+ break
57
+ when '4'
58
+ Datahunter.get_dataset sub_datasets[3]
59
+ break
60
+ when '5'
61
+ Datahunter.get_dataset sub_datasets[4]
61
62
  break
62
-
63
- else # When no
64
- if index == 2
65
- puts ("You may want to try: ".colorize(:red) + "'$ hunter search #{args[0].downcase}'".colorize(:blue))
66
- end
67
- if index > 2 or datasets[index+1].nil?
68
- Datahunter.print_excuse_message
69
- Datahunter.print_request_dataset_message
70
- Datahunter.print_feedback_request
71
- break
72
- end
63
+ when 'abort'
64
+ break
65
+ else
66
+ next
73
67
  end
74
68
  end
75
69
  when 204
76
- puts "We've found nothing for your query. "\
77
- "Remember, this is a first prototype, there will be a lot more "\
78
- "datasets indexed soon. If you want us to find a dataset for you, or "\
79
- "if you just want to give us a feedback, don't hesitate!".colorize(:red)
70
+ Datahunter.print_no_dataset_message
80
71
  Datahunter.print_request_dataset_message
81
72
  else
82
73
  response.return!(request, result, &block)
@@ -86,55 +77,9 @@ command :find do |c|
86
77
  end
87
78
  end
88
79
 
89
- command :search do |c|
90
- c.syntax = 'hunter search keyword [spatial-coverage] [temporal-coverage]'
91
- c.summary = '$ hunter search <keyword> <geo-coverage> <temporal-coverage>
92
- * Returns all the datasets corresponding to
93
- the query, sorted by popularity.
94
- * WARNING: this command is going to be modified soon!'.colorize(:blue)
95
- c.description = c.summary
96
- c.example 'Search open data for population', 'hunter search population'
97
- c.action do |args, options|
98
-
99
- start_time = Time.now
100
-
101
- url = Datahunter.datasets_url(args[0], args[1], args[2])
102
-
103
- response = RestClient.get(url, :content_type => :json) do |response, request, result, &block|
104
-
105
- elapsed_time = Time.now - start_time
106
-
107
- case response.code
108
- when 200
109
- datasets = JSON.parse(response.body).reverse
110
- puts "### Response in #{elapsed_time} seconds".colorize(:blue)
111
- puts "### We've found #{datasets.size} datasets corresponding to your query #{args[0].downcase}:".colorize(:blue)
112
- puts
113
- datasets.each do |result|
114
- puts ("- ".colorize(:yellow) +
115
- "#{result["title"]}".colorize(:green) +
116
- " id: ".colorize(:blue) +
117
- "#{result["_id"]}")
118
- end
119
- puts
120
- puts ("You can get more informations about a dataset with " +
121
- "$ hunter info <id>".colorize(:blue))
122
- puts ("You can download directly a dataset with " +
123
- "$ hunter get <id>".colorize(:blue))
124
- when 204
125
- puts "We found nothing for your query: as every good hunter... we are still learning".colorize(:red)
126
- Datahunter.print_excuse_message
127
- Datahunter.print_feedback_request
128
- else
129
- response.return!(request, result, &block)
130
- end
131
- end
132
- end
133
- end
134
-
135
80
  command :info do |c|
136
81
  c.syntax = 'hunter info <id>'
137
- c.summary = 'Get informations about a dataset from its ID'.colorize(:blue)
82
+ c.summary = 'Displays all the informations about a dataset given its id'.colorize(:blue)
138
83
  c.description = c.summary
139
84
  c.action do |args, options|
140
85
 
@@ -142,7 +87,6 @@ command :info do |c|
142
87
  puts "usage: $ hunter info <id>"
143
88
  else
144
89
  url = Datahunter.ds_url(args[0])
145
-
146
90
  response = RestClient.get(url, :content_type => :json) do |response, request, result, &block|
147
91
 
148
92
  case response.code
@@ -160,8 +104,7 @@ end
160
104
 
161
105
  command :get do |c|
162
106
  c.syntax = 'hunter get <id>'
163
- c.summary =
164
- 'Get a dataset (direct download if possible, web page if not) from its ID'.colorize(:blue)
107
+ c.summary = 'Downloads a dataset given its id'.colorize(:blue)
165
108
  c.description = c.summary
166
109
  c.action do |args, options|
167
110
 
@@ -169,18 +112,13 @@ command :get do |c|
169
112
  puts "usage: $ hunter get id"
170
113
  else
171
114
  url = Datahunter.ds_url(args[0])
172
-
173
115
  response = RestClient.get(url, :content_type => :json) do |response, request, result, &block|
174
-
116
+
175
117
  case response.code
176
118
  when 200
177
119
  ds = JSON.parse(response.body)
178
120
  puts
179
- if ds.has_key?("resources") and ds["resources"].any?
180
- Datahunter.download_the_data ds
181
- else
182
- Datahunter.open_in_browser ds["uri"]
183
- end
121
+ Datahunter.get_dataset ds
184
122
  when 500
185
123
  puts "Invalid ID".colorize(:red)
186
124
  end
@@ -191,43 +129,27 @@ end
191
129
 
192
130
  command :about do |c|
193
131
  c.syntax = 'hunter about'
194
- c.summary = 'About Hunter
195
- * Vision
196
- * Value Proposition
197
- * Stats
198
- * Last datasets indexed'.colorize(:blue)
132
+ c.summary = 'Gives Hunter\'s vision, the number of datasets indexed etc'.colorize(:blue)
199
133
  c.description = c.summary
200
134
  c.action do |args, options|
201
- puts "........8888888888888888888888..........
202
- .....88888888888888888888888888888......
203
- ....8888888888888888888888888888888.....
204
- ...888888888888888888888888888~88888?...
205
- ..88888...888888888888888888..+888888...
206
- .888888.=....................O78888888..
207
- ?888888.......................Z88888888.
208
- 8888888.......................888888888.
209
- 8888888.......................888888888.
210
- 8888888:......8O.......88.....888888888.
211
- 88......ZO.....8=.....Z8.....8888888888.
212
- $.........:8...............888888888888.
213
- ............8.............8888888888888.
214
- ..............O.........:88888888888888.
215
- ...............8........88888888888888..
216
- ................O......8.......888888...
217
- ................8.88.8.........=8887 ..
218
- . ................8888...........?8.....
219
- .. ...........................8888......
220
- ... .........................888........
221
- .... .....................:88..........
222
- ..... ...............8.............".colorize(:red)
223
- puts
224
- puts 'Find and download Open Datasets directly from the terminal!'.colorize(:red)
225
- puts
226
- puts 'We believe there will be more and more data available on the web and the main issue will be to find them and to manage each sources. So we try to do it for you: not by pre-processing them but by orienting you.'.colorize(:red)
227
- puts
228
- puts 'Number of datasets currently indexed: 8335'.colorize(:blue)
229
- puts
230
- puts 'Last dataset indexed: "Canada Open Data" "NETLs Energy Data eXchange", dati.gov.it'
135
+ puts "
136
+ Find and download Open Datasets ....8888888888888888888888888888888.....
137
+ directly from the terminal! ...888888888888888888888888888~88888?...
138
+ ..88888...888888888888888888..+888888...
139
+ We believe there will be more .888888.=....................O78888888..
140
+ and more data available on the web ?888888.......................Z88888888.
141
+ and the main issue will be to find 8888888.......................888888888.
142
+ them and to manage each source. 8888888.......................888888888.
143
+ 8888888:......8O.......88.....888888888.
144
+ 88......ZO.....8=.....Z8.....8888888888.
145
+ $.........:8...............888888888888.
146
+ So we try to show you the way! ............8.............8888888888888.
147
+ ..............O.........:88888888888888.
148
+ ...............8........88888888888888..
149
+ ................O......8.......888888...
150
+ ................8.88.8.........=8887 ..
151
+ indexed datasets: 8336 . ................8888...........?8.....
152
+ last dataset: DAMIR french health .. ...........................8888......".colorize(:red)
231
153
  puts
232
154
  puts "### Hunter is still a prototype, please don't hesitate to help us make open datasets workflow better!".colorize(:blue)
233
155
  Datahunter.print_feedback_request
@@ -235,21 +157,22 @@ $.........:8...............888888888888.
235
157
  end
236
158
 
237
159
  command :guide do |c|
238
- c.action {puts ('$ hunter find <keyword> <geo-coverage> <temporal-coverage> '.colorize(:blue) +
239
- '
240
- * Returns the 3 most popular (most reused, viewed, etc) datasets corresponding to the query
241
- * Allow you to either download the dataset or to open your favorite browser directly at the good page')
242
-
243
- puts ('$ hunter search <keyword> <geo-coverage> <temporal-coverage>'.colorize(:blue) +
244
- '
245
- * Returns all the datasets corresponding to the query, sorted by popularity.
246
- * WARNING: this command is going to be modified soon!')
247
-
248
- puts ('$ hunter about'.colorize(:blue) +
249
- '
250
- * Vision
251
- * Value Proposition
252
- * Stats
253
- * Last datasets indexed')}
160
+ c.action do
161
+ puts ('$ hunter find <keyword> <spatial coverage> <temporal coverage> '.colorize(:blue) + "\n" +
162
+ 'Datasets corresponding to your query, sorted by popularity, '\
163
+ 'displayed 5 by 5. Search ~> Find ~> Get the data!')
164
+ puts
165
+ puts ('$ hunter info <id>'.colorize(:blue) + "\n" +
166
+ 'Display all the informations about a dataset given its id: '\
167
+ 'complete description, publisher, last update, etc')
168
+ puts
169
+ puts ('$ hunter get <id>'.colorize(:blue) + "\n" +
170
+ 'Get a dataset given its id: '\
171
+ 'if possible download the data, if not, it opens the web page '\
172
+ 'from where it will be possible')
173
+ puts
174
+ puts ('$ hunter about'.colorize(:blue) + "\n" +
175
+ 'About Hunter, the vision, number of datasets indexed etc')
176
+ end
254
177
  end
255
178
  default_command :guide
data/datahunter.gemspec CHANGED
@@ -24,11 +24,12 @@ Gem::Specification.new do |spec|
24
24
  spec.add_development_dependency "rake", "~> 10.0"
25
25
  spec.add_development_dependency "rspec"
26
26
  spec.add_development_dependency "aruba-rspec"
27
-
27
+
28
28
  spec.add_runtime_dependency "json", "~> 1.8.1"
29
- spec.add_runtime_dependency "commander", "~> 4.2.1"
30
- spec.add_runtime_dependency "rest-client", "~> 1.7.2"
29
+ spec.add_runtime_dependency "commander", "~> 4.3.0"
30
+ spec.add_runtime_dependency "rest-client", "~> 1.7.3"
31
31
  spec.add_runtime_dependency "launchy", "~> 2.4.3"
32
- spec.add_runtime_dependency "colorize", "~> 0.7.3"
32
+ spec.add_runtime_dependency "colorize", "~> 0.7.5"
33
33
  spec.add_runtime_dependency "downloadr", "0.0.41"
34
+ spec.add_runtime_dependency "addressable", "~> 2.3.7"
34
35
  end
@@ -5,13 +5,12 @@ require 'addressable/uri'
5
5
 
6
6
  module Datahunter
7
7
 
8
- # DATASETS_URL = "http://localhost:3000/api/datasets/"
9
8
  DATASETS_URL = "http://shrouded-harbor-5877.herokuapp.com/api/datasets/"
10
9
  FEEDBACK_URL = "https://docs.google.com/forms/d/1yNzZjCCXvWHQCbWz4sx-nui3LafeeLcT7FF9T-vbKvw/viewform"
11
10
  REQUEST_URL =
12
11
  "https://docs.google.com/forms/d/1NRKWmb_mcpKJmrutXvZSZnysM_v0rfLhjD897H3Myrw/viewform?usp=send_form"
13
12
 
14
- @extensions = ["json", "csv", "xml", "zip", "gz", "xls", "xlsx", "tsv"]
13
+ @extensions = ["json", "csv", "xml", "zip", "gz", "xls", "xlsx", "shp", "docx", "doc", "pdf", "txt", "tsv"]
15
14
 
16
15
  def self.datasets_url tag, geo=nil, temp=nil
17
16
  tag = tag.downcase.split.first if tag
@@ -33,17 +32,29 @@ module Datahunter
33
32
  end
34
33
 
35
34
  def self.print_dataset_info dataset
36
- puts ("title: ".colorize(:green) + "#{dataset["title"]}")
37
- puts ("description: ".colorize(:green) + "#{dataset["description"]}")
38
- puts ("publisher: ".colorize(:green) + "#{dataset["publisher"]}")
39
- puts ("temporal: ".colorize(:green) + "#{dataset["temporal"]}")
40
- puts ("spatial: ".colorize(:green) + "#{dataset["spatial"]}")
41
- puts ("created: ".colorize(:green) + "#{dataset["created"]}")
42
- puts ("updated: ".colorize(:green) + "#{dataset["updated"]}")
43
- puts ("score: ".colorize(:green) + "#{dataset["huntscore"]}")
35
+ puts ("#{dataset["title"]}".colorize(:green))
36
+ puts ("#{dataset["description"]}".colorize(:blue))
44
37
  puts
38
+ puts ("publisher: ".colorize(:blue) + "#{dataset["publisher"]}")
39
+ puts ("temporal: ".colorize(:blue) + "#{dataset["temporal"]}")
40
+ puts ("spatial: ".colorize(:blue) + "#{dataset["spatial"]}")
41
+ puts ("created: ".colorize(:blue) + "#{dataset["created"]}")
42
+ puts ("updated: ".colorize(:blue) + "#{dataset["updated"]}")
43
+ puts ("score: ".colorize(:blue) + "#{dataset["huntscore"]}")
45
44
  end
46
-
45
+
46
+ def self.print_coll_of_datasets_info_light coll_of_datasets
47
+ coll_of_datasets.each_with_index do |ds, index|
48
+ puts ("#{index+1}. ".colorize(:yellow) +
49
+ "#{ds["title"]}".colorize(:green) +
50
+ " id: ".colorize(:blue) +
51
+ "#{ds["_id"]}")
52
+ puts ("#{ds["spatial"].take(5)}")
53
+ puts ("#{ds["description"][0..100].gsub(/\w+\s*$/,'...')}".colorize(:blue))
54
+ end
55
+ puts
56
+ end
57
+
47
58
  def self.print_downloadable_links resources
48
59
  resources.each_with_index do |dl, i|
49
60
  puts ("#{i}. ".colorize(:yellow) +
@@ -68,15 +79,15 @@ module Datahunter
68
79
  file_name = uri.basename
69
80
  loc = location + "/" + file_name
70
81
 
71
- case ask ("Create/overwrite #{loc}?(y/rename/n)".colorize(:yellow))
82
+ case ask ("### Create/overwrite #{loc}?(y/rename/n)".colorize(:yellow))
72
83
  when 'rename'
73
84
  loc = ask "Path/to/filename: ".colorize(:yellow)
74
85
  when 'n'
75
86
  abort("Ok then")
76
87
  end
77
- puts "Start downloading..."
88
+ puts "### Start downloading..."
78
89
  Downloadr::HTTP.download(url, loc)
79
- puts "Your file has been downloaded ;)".colorize(:green)
90
+ puts "### Your file has been downloaded ;)".colorize(:green)
80
91
  Datahunter.print_excuse_and_alternative_url_message alt_url
81
92
  end
82
93
  end
@@ -105,32 +116,27 @@ module Datahunter
105
116
  end
106
117
  end
107
118
 
108
- ## Messages: feedback and excuses
109
-
110
- def self.print_feedback_request
111
- case ask "### give feedback? (y/n)".colorize(:yellow)
112
- when 'y'
113
- Launchy.open(FEEDBACK_URL, options = {})
119
+ def self.get_dataset dataset
120
+ if dataset.has_key?("resources") and dataset["resources"].any?
121
+ Datahunter.download_the_data dataset
114
122
  else
115
- puts "Bye for now!"
123
+ Datahunter.open_in_browser dataset["uri"]
116
124
  end
117
125
  end
118
126
 
119
- def self.print_request_dataset_message
120
- case ask "### request a dataset? (y/n)".colorize(:yellow)
121
- when 'y'
122
- Launchy.open(REQUEST_URL, options = {})
123
- end
124
- end
125
-
126
- def self.print_excuse_message
127
- puts "Remember, this is a first prototype, there will surely be a lot more "\
127
+ ## Messages
128
+ def self.print_no_dataset_message
129
+ puts "We've found nothing for your query. "\
130
+ "Remember, this is a first prototype, there will be a lot more "\
128
131
  "datasets indexed soon. If you want us to find a dataset for you, or "\
129
132
  "if you just want to give us a feedback, don't hesitate!".colorize(:red)
130
133
  end
131
134
 
132
135
  def self.print_excuse_and_alternative_url_message alt_url=""
133
- puts "If this is not the file you expected, it's maybe because publisher don't always keep the metadata up-to-date. We try to clean most of uri's and check the url. Anyway you may be able to download your file by hand here:"
136
+ puts "If this is not the file you expected, it's maybe because publisher"\
137
+ " don't always keep the metadata up-to-date. We try to clean most of"\
138
+ " uri's and check the url. Anyway you may be able to download your"\
139
+ " file by hand here:"
134
140
  puts "#{alt_url}".colorize(:blue)
135
141
  end
136
142
 
@@ -138,4 +144,22 @@ module Datahunter
138
144
  puts "The URL given by the publisher is not valid. We'll try to find out why "\
139
145
  "as soon as we can!".colorize(:red)
140
146
  end
147
+
148
+
149
+ ## Feedback requests
150
+ def self.print_feedback_request
151
+ case ask "### give feedback? (y/n)".colorize(:yellow)
152
+ when 'y'
153
+ Launchy.open(FEEDBACK_URL, options = {})
154
+ else
155
+ puts "Bye for now!"
156
+ end
157
+ end
158
+
159
+ def self.print_request_dataset_message
160
+ case ask "### request a dataset? (y/n)".colorize(:yellow)
161
+ when 'y'
162
+ Launchy.open(REQUEST_URL, options = {})
163
+ end
164
+ end
141
165
  end
@@ -1,3 +1,3 @@
1
1
  module Datahunter
2
- VERSION = "0.2.4"
2
+ VERSION = "0.3.0"
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: datahunter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.4
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Terpo
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-02-25 00:00:00.000000000 Z
11
+ date: 2015-03-04 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -86,28 +86,28 @@ dependencies:
86
86
  requirements:
87
87
  - - "~>"
88
88
  - !ruby/object:Gem::Version
89
- version: 4.2.1
89
+ version: 4.3.0
90
90
  type: :runtime
91
91
  prerelease: false
92
92
  version_requirements: !ruby/object:Gem::Requirement
93
93
  requirements:
94
94
  - - "~>"
95
95
  - !ruby/object:Gem::Version
96
- version: 4.2.1
96
+ version: 4.3.0
97
97
  - !ruby/object:Gem::Dependency
98
98
  name: rest-client
99
99
  requirement: !ruby/object:Gem::Requirement
100
100
  requirements:
101
101
  - - "~>"
102
102
  - !ruby/object:Gem::Version
103
- version: 1.7.2
103
+ version: 1.7.3
104
104
  type: :runtime
105
105
  prerelease: false
106
106
  version_requirements: !ruby/object:Gem::Requirement
107
107
  requirements:
108
108
  - - "~>"
109
109
  - !ruby/object:Gem::Version
110
- version: 1.7.2
110
+ version: 1.7.3
111
111
  - !ruby/object:Gem::Dependency
112
112
  name: launchy
113
113
  requirement: !ruby/object:Gem::Requirement
@@ -128,14 +128,14 @@ dependencies:
128
128
  requirements:
129
129
  - - "~>"
130
130
  - !ruby/object:Gem::Version
131
- version: 0.7.3
131
+ version: 0.7.5
132
132
  type: :runtime
133
133
  prerelease: false
134
134
  version_requirements: !ruby/object:Gem::Requirement
135
135
  requirements:
136
136
  - - "~>"
137
137
  - !ruby/object:Gem::Version
138
- version: 0.7.3
138
+ version: 0.7.5
139
139
  - !ruby/object:Gem::Dependency
140
140
  name: downloadr
141
141
  requirement: !ruby/object:Gem::Requirement
@@ -150,6 +150,20 @@ dependencies:
150
150
  - - '='
151
151
  - !ruby/object:Gem::Version
152
152
  version: 0.0.41
153
+ - !ruby/object:Gem::Dependency
154
+ name: addressable
155
+ requirement: !ruby/object:Gem::Requirement
156
+ requirements:
157
+ - - "~>"
158
+ - !ruby/object:Gem::Version
159
+ version: 2.3.7
160
+ type: :runtime
161
+ prerelease: false
162
+ version_requirements: !ruby/object:Gem::Requirement
163
+ requirements:
164
+ - - "~>"
165
+ - !ruby/object:Gem::Version
166
+ version: 2.3.7
153
167
  description: Command line interface to find open datasets via the Hunter API
154
168
  email:
155
169
  - nicolas.terpolilli@gmail.com