datahunter 0.1.6 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +64 -63
  3. data/bin/hunter +8 -4
  4. data/lib/datahunter/version.rb +1 -1
  5. metadata +2 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 2f3c168c10533b56d7b9c37d45afb780f34a86ca
4
- data.tar.gz: 088ef418aed953bc729bd759b762e866a745e39c
3
+ metadata.gz: b40b63c4ce869e693eb78a76e12fb9b2230b5db5
4
+ data.tar.gz: 1964961dfcd6a833cb83e89a6b90d2283eb70834
5
5
  SHA512:
6
- metadata.gz: 36b3577e2d7cb46ae4e7020d8f2d2a0e0d463b5a92d0ec97c199b7d2bfd6d2be01087a53069e79d9e8d4958cdbbb7f452ed6771a785f945fbb840dae5c291dce
7
- data.tar.gz: e4401e26b3aac1e21c2863508ee6b59107a218a92121e4d0735b477e89c14efcadc2e7f9c48479c9259173c874c615f67fdec9bf032efd9ea83274ce439c7298
6
+ metadata.gz: 6b71e8539a2abc417bcb1c480292900458c635ae3077a74bf6ff12f77a96d2750507a5c1b23614db7e0b491df563ae07c54e466b6f1478cb75759d2def37a958
7
+ data.tar.gz: ff744d142797c2e350463933269ffa807829c5bfc13ea4f6d17f8dcd382f85c7a56a268a186c4d10de94d0beda47e93343a753f2f7c6d863895f6ffaa1800915
data/README.md CHANGED
@@ -6,7 +6,9 @@
6
6
 
7
7
  [![Gem Version](https://badge.fury.io/rb/datahunter.svg)](http://badge.fury.io/rb/datahunter) [![Build Status](https://travis-ci.org/NTerpo/datahunter.svg?branch=master)](https://travis-ci.org/NTerpo/datahunter)
8
8
 
9
- **Hunter allows you to find and download open datasets quickly and easily from your terminal**. We work hard to index and sort the most useful datasets available for you!
9
+ **Hunter allows you to find and download open datasets quickly and easily from your terminal**.
10
+
11
+ We work hard to index and sort the most useful datasets available for you!
10
12
 
11
13
  In a world where there are more and more data publishers and data users, the research of those data is still a pain.
12
14
  Our aim with Hunter is to help publishers seeing their data reused and to help data users to find them quicker than ever and to handle them easily.
@@ -20,84 +22,83 @@ We believe that pre-processing data is a bad idea and that hosting them could cr
20
22
 
21
23
  ## Usage
22
24
 
23
- ### Example1. $ hunter find ~> download
25
+ ### $ hunter find
24
26
 
25
- $ hunter find alimentaires
26
- ### Response in 0.430501 seconds
27
+ $ hunter find consumer
28
+ ### Response in 0.783677 seconds
27
29
  ### Looks like we've got something for you!
28
-
29
- title: Produits alimentaires : ingrédients, nutrition, labels
30
-
31
- description: [Open Food Facts](http://fr.openfoodfacts.org/) répertorie les informations sur les produits alimentaires : ingrédients, informations nutritionnelles, labels etc. Les données proviennent majoritairement de la collecte citoyenne (crowdsourcing) des informations.
32
-
33
- publisher: Open Food Facts
34
-
35
- temporal: all
36
-
37
- spatial: ["france", "fr", "europe", "schengen", "eu", "ue", "countries", "world", "all"]
38
-
39
- created: 2014-05-07T02:44:58.119Z
40
-
41
- updated: 2014-12-13T16:40:14.136Z
42
30
 
43
- score: 13.427
44
-
31
+ title: Consumer Complaint Database
32
+ description: These are complaints we’ve received about financial products and services.
33
+ publisher: Consumer Financial Protection Bureau
34
+ temporal: ["2011", "2012", "2013"]
35
+ spatial: ["us", "usa", "america", "united states", "united-states", "united states of america", "united-states-of-america", "world", "countries", "all"]
36
+ created: 2014-02-25T18:48:25.192Z
37
+ updated: 2014-02-25T23:48:24.897Z
38
+ score: 20.743
39
+
45
40
  ### get the data? (y/n)
46
41
  y
47
-
48
- 0. Ingrédients, informations nutritionnelles et données sur les produits alimentaires - CSV
49
- 1. Ingrédients, informations nutritionnelles et données sur les produits alimentaires - RDF
50
- 2. Description des données, exports et API de la base de produits Open Food Facts - HTML
51
-
42
+ 0. Consumer Complaint Database - CSV
43
+ 1. Consumer Complaint Database - JSON
44
+ 2. Consumer Complaint Database - XML
52
45
  ### which one? (0/1/...)
53
46
  0
54
-
55
47
  Start downloading...
56
48
  Your file has been downloaded, try to $ ls ;D
57
-
58
- ### Example2. $ hunter find ~> open browser
59
- $ hunter find population france
60
- ### Response in 0.569162 seconds
61
- ### Looks like we've got something for you!
62
-
63
- title: Population
64
-
65
- description: Ce jeu de données permet d'accéder aux résultats des recensements de la population, à des séries chronologiques de la Banque de Données Macro-économiques de l'Insee sur le thème de la population et à d'autres données issues notamment des statistiques de l'état civil. Le recensement de la population permet de connaître la diversité et l'évolution de la population de la France. L'Insee fournit ainsi des statistiques sur les habitants et les logements, leur nombre et leurs caractéristiques : répartition par sexe et âge, professions, diplômes-formation, conditions de logement, modes de transport, déplacements domicile-travail, etc. La BDM est la principale base de données de séries et indices sur l'ensemble des domaines économiques et sociaux.
66
-
67
- publisher: Institut National de la Statistique et des Etudes Economiques (INSEE)
68
-
69
- temporal: ["1962", "1963", "1964", "1965", "1966", "1967", "1968", "1969", "1970", "1971", "1972", "1973", "1974", "1975", "1976", "1977", "1978", "1979", "1980", "1981", "1982", "1983", "1984", "1985", "1986", "1987", "1988", "1989", "1990", "1991", "1992", "1993", "1994", "1995", "1996", "1997", "1998", "1999", "2000", "2001", "2002", "2003", "2004", "2005", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013"]
70
-
71
- spatial: ["france", "fr", "europe", "schengen", "eu", "ue", "countries", "world", "all"]
72
-
73
- created: 2014-05-07T02:40:14.478Z
74
-
75
- updated: 2014-11-07T01:32:48.201Z
76
-
77
- score: 25.893
78
-
79
- ### get the data? (y/n)
80
- => y: open your favorite browser at the page where you'll be able to preview and download the data
81
- => n: suggest the next most popular dataset corresponding to the query
49
+ If this is not the file you expected, it's maybe because publisher don't always keep the metadata up-to-date. We try to clean most of uri's and check the url. Anyway you may be able to download your file by hand here:
50
+ https://data.consumerfinance.gov/api/views/x94z-ydhh/rows.csv?accessType=DOWNLOAD
82
51
 
83
52
  ![usage](./hunter.gif)
84
53
 
85
54
  ### $ hunter search
86
55
 
87
- $ hunter search paris
88
- ### Response in 0.281749 seconds
89
- ### We've found 5 datasets corresponding to your query paris:
90
-
91
- Vélib' - Paris et communes limitrophes (https://www.data.gouv.fr/fr/datasets/velib-paris-et-communes-limitrophes-idf/)
92
-
93
- Liste des sites des hotspots Paris WiFi (https://www.data.gouv.fr/fr/datasets/liste-des-sites-des-hotspots-paris-wifi-prs/)
94
-
95
- Liste des casernes à Paris et dans les départements de la petite couronne (https://www.data.gouv.fr/fr/datasets/liste-des-casernes-a-paris-et-dans-les-departements-de-la-petite-couronne-551678/)
96
-
97
- Points d'accueil police (coordonnées) - Paris (https://www.data.gouv.fr/fr/datasets/points-d-accueil-police-coordonnees-paris-30379560/)
56
+ $ hunter search consumer
57
+ ### Response in 0.402504 seconds
58
+ ### We've found 12 datasets corresponding to your query consumer:
59
+
60
+ - Consumer Complaint Database id: 548c82a7a826dfe85070e5fa
61
+ - Consumer Expenditure Survey id: 548c82a9a826dfe85070e607
62
+ - Farmers Markets Geographic Data id: 548c82aaa826dfe85070e60e
63
+ - Food Price Outlook id: 548c82aba826dfe85070e624
64
+ - Consumer Price Index id: 548c82fca826dfe85070e79f
65
+ - Foreign Labor Statistics id: 548c82ada826dfe85070e663
66
+ - Farmers Markets Search id: 548c82b1a826dfe85070e6f2
67
+ - International Macroeconomic Data Set id: 548c8301a826dfe85070e84e
68
+ - Consumer Expenditure Survey id: 548c82fba826dfe85070e782
69
+ - Consumer Price Indexes - Base id: 548c82fca826dfe85070e7b5
70
+ - Consumer Price Index - Department Store Inventory Price Index id: 548c82fea826dfe85070e7f4
71
+ - Southeast Information Office id: 548c8305a826dfe85070e8cf
72
+
73
+ You can get more informations about a dataset with $ hunter info <id>
74
+ You can download directly a dataset with $ hunter get <id>
75
+
76
+ ### $ hunter info
98
77
 
99
- Lieux de tournage de films (long métrage) (https://www.data.gouv.fr/fr/datasets/lieux-de-tournage-de-films-long-metrage-prs/)
78
+ $ hunter info 548c82a7a826dfe85070e5fa
79
+
80
+ title: Consumer Complaint Database
81
+ description: These are complaints we’ve received about financial products and services.
82
+ publisher: Consumer Financial Protection Bureau
83
+ temporal: ["2011", "2012", "2013"]
84
+ spatial: ["us", "usa", "america", "united states", "united-states", "united states of america", "united-states-of-america", "world", "countries", "all"]
85
+ created: 2014-02-25T18:48:25.192Z
86
+ updated: 2014-02-25T23:48:24.897Z
87
+ score: 20.743
88
+
89
+ ### $ hunter get
100
90
 
91
+ $ hunter get 548c82a7a826dfe85070e5fa
92
+
93
+ 0. Consumer Complaint Database - CSV
94
+ 1. Consumer Complaint Database - JSON
95
+ 2. Consumer Complaint Database - XML
96
+ ### which one? (0/1/...)
97
+ 1
98
+ Start downloading...
99
+ Your file has been downloaded, try to $ ls ;D
100
+ If this is not the file you expected, it's maybe because publisher don't always keep the metadata up-to-date. We try to clean most of uri's and check the url. Anyway you may be able to download your file by hand here:
101
+ https://data.consumerfinance.gov/api/views/x94z-ydhh/rows.csv?accessType=DOWNLOAD
101
102
 
102
103
  ## Feedbacks
103
104
  Don't hesitate to [give us any feedback about you experience with Hunter!](https://docs.google.com/forms/d/1yNzZjCCXvWHQCbWz4sx-nui3LafeeLcT7FF9T-vbKvw/viewform?usp=send_form)
data/bin/hunter CHANGED
@@ -74,7 +74,10 @@ command :find do |c|
74
74
  end
75
75
  end
76
76
  when 204
77
- puts "We've found nothing for your query: as every good hunter... we are still learning".colorize(:red)
77
+ puts "We've found nothing for your query. "\
78
+ "Remember, this is a first prototype, there will be a lot more "\
79
+ "datasets indexed soon. If you want us to find a dataset for you, or "\
80
+ "if you just want to give us a feedback, don't hesitate!".colorize(:red)
78
81
  Datahunter.print_request_dataset_message
79
82
  else
80
83
  response.return!(request, result, &block)
@@ -108,10 +111,11 @@ command :search do |c|
108
111
  puts "### Response in #{elapsed_time} seconds".colorize(:blue)
109
112
  puts "### We've found #{datasets.size} datasets corresponding to your query #{args[0].downcase}:".colorize(:blue)
110
113
  puts
111
- datasets.each_with_index do |result, i|
112
- puts ("#{i}. ".colorize(:yellow) +
114
+ datasets.each do |result|
115
+ puts ("- ".colorize(:yellow) +
113
116
  "#{result["title"]}".colorize(:green) +
114
- " id: #{result["_id"]}")
117
+ " id: ".colorize(:blue) +
118
+ "#{result["_id"]}")
115
119
  end
116
120
  puts
117
121
  puts ("You can get more informations about a dataset with " +
data/lib/datahunter/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module Datahunter
2
- VERSION = "0.1.6"
2
+ VERSION = "0.1.7"
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: datahunter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.6
4
+ version: 0.1.7
5
5
  platform: ruby
6
6
  authors:
7
7
  - Terpo
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-12-28 00:00:00.000000000 Z
11
+ date: 2015-01-06 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler