earth-ruby19 0.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (187) hide show
  1. data/Gemfile +5 -0
  2. data/Gemfile.lock +138 -0
  3. data/LICENSE +20 -0
  4. data/README.markdown +38 -0
  5. data/lib/earth.rb +146 -0
  6. data/lib/earth/active_record_ext.rb +9 -0
  7. data/lib/earth/air.rb +13 -0
  8. data/lib/earth/air/aircraft.rb +21 -0
  9. data/lib/earth/air/aircraft/data_miner.rb +184 -0
  10. data/lib/earth/air/aircraft_class.rb +10 -0
  11. data/lib/earth/air/aircraft_class/data_miner.rb +42 -0
  12. data/lib/earth/air/aircraft_manufacturer.rb +9 -0
  13. data/lib/earth/air/aircraft_manufacturer/data_miner.rb +20 -0
  14. data/lib/earth/air/airline.rb +16 -0
  15. data/lib/earth/air/airline/data_miner.rb +57 -0
  16. data/lib/earth/air/airport.rb +44 -0
  17. data/lib/earth/air/airport/data_miner.rb +80 -0
  18. data/lib/earth/air/data_miner.rb +15 -0
  19. data/lib/earth/air/flight_configuration.rb +7 -0
  20. data/lib/earth/air/flight_configuration/data_miner.rb +16 -0
  21. data/lib/earth/air/flight_distance_class.rb +7 -0
  22. data/lib/earth/air/flight_distance_class/data_miner.rb +16 -0
  23. data/lib/earth/air/flight_domesticity.rb +6 -0
  24. data/lib/earth/air/flight_domesticity/data_miner.rb +57 -0
  25. data/lib/earth/air/flight_fuel_type.rb +12 -0
  26. data/lib/earth/air/flight_fuel_type/data_miner.rb +12 -0
  27. data/lib/earth/air/flight_propulsion.rb +7 -0
  28. data/lib/earth/air/flight_propulsion/data_miner.rb +16 -0
  29. data/lib/earth/air/flight_seat_class.rb +12 -0
  30. data/lib/earth/air/flight_seat_class/data_miner.rb +36 -0
  31. data/lib/earth/air/flight_segment.rb +29 -0
  32. data/lib/earth/air/flight_segment/data_miner.rb +330 -0
  33. data/lib/earth/air/flight_service.rb +7 -0
  34. data/lib/earth/air/flight_service/data_miner.rb +16 -0
  35. data/lib/earth/all.rb +11 -0
  36. data/lib/earth/automobile.rb +8 -0
  37. data/lib/earth/automobile/automobile_fuel_type.rb +18 -0
  38. data/lib/earth/automobile/automobile_fuel_type/data_miner.rb +45 -0
  39. data/lib/earth/automobile/automobile_make.rb +14 -0
  40. data/lib/earth/automobile/automobile_make/data_miner.rb +68 -0
  41. data/lib/earth/automobile/automobile_make_fleet_year.rb +15 -0
  42. data/lib/earth/automobile/automobile_make_fleet_year/data_miner.rb +29 -0
  43. data/lib/earth/automobile/automobile_make_year.rb +14 -0
  44. data/lib/earth/automobile/automobile_make_year/data_miner.rb +45 -0
  45. data/lib/earth/automobile/automobile_model.rb +14 -0
  46. data/lib/earth/automobile/automobile_model/data_miner.rb +38 -0
  47. data/lib/earth/automobile/automobile_model_year.rb +15 -0
  48. data/lib/earth/automobile/automobile_model_year/data_miner.rb +51 -0
  49. data/lib/earth/automobile/automobile_size_class.rb +14 -0
  50. data/lib/earth/automobile/automobile_size_class/data_miner.rb +43 -0
  51. data/lib/earth/automobile/automobile_variant.rb +17 -0
  52. data/lib/earth/automobile/automobile_variant/data_miner.rb +460 -0
  53. data/lib/earth/automobile/data_miner.rb +8 -0
  54. data/lib/earth/base.rb +7 -0
  55. data/lib/earth/bus.rb +1 -0
  56. data/lib/earth/bus/bus_class.rb +19 -0
  57. data/lib/earth/bus/bus_class/data_miner.rb +41 -0
  58. data/lib/earth/bus/data_miner.rb +1 -0
  59. data/lib/earth/conversions_ext.rb +45 -0
  60. data/lib/earth/data_miner.rb +10 -0
  61. data/lib/earth/diet.rb +2 -0
  62. data/lib/earth/diet/data_miner.rb +2 -0
  63. data/lib/earth/diet/diet_class.rb +15 -0
  64. data/lib/earth/diet/diet_class/data_miner.rb +36 -0
  65. data/lib/earth/diet/food_group.rb +17 -0
  66. data/lib/earth/diet/food_group/data_miner.rb +26 -0
  67. data/lib/earth/fuel.rb +2 -0
  68. data/lib/earth/fuel/data_miner.rb +2 -0
  69. data/lib/earth/fuel/fuel_price.rb +13 -0
  70. data/lib/earth/fuel/fuel_price/data_miner.rb +20 -0
  71. data/lib/earth/fuel/fuel_type.rb +18 -0
  72. data/lib/earth/fuel/fuel_type/data_miner.rb +37 -0
  73. data/lib/earth/hospitality.rb +1 -0
  74. data/lib/earth/hospitality/data_miner.rb +1 -0
  75. data/lib/earth/hospitality/lodging_class.rb +9 -0
  76. data/lib/earth/hospitality/lodging_class/data_miner.rb +30 -0
  77. data/lib/earth/industry.rb +10 -0
  78. data/lib/earth/industry/industry.rb +23 -0
  79. data/lib/earth/industry/industry_product.rb +22 -0
  80. data/lib/earth/industry/industry_product_line.rb +20 -0
  81. data/lib/earth/industry/industry_sector.rb +20 -0
  82. data/lib/earth/industry/merchant.rb +18 -0
  83. data/lib/earth/industry/merchant_category.rb +22 -0
  84. data/lib/earth/industry/merchant_category_industry.rb +20 -0
  85. data/lib/earth/industry/product_line.rb +22 -0
  86. data/lib/earth/industry/product_line_industry_product.rb +20 -0
  87. data/lib/earth/industry/sector.rb +19 -0
  88. data/lib/earth/inflectors.rb +9 -0
  89. data/lib/earth/locality.rb +10 -0
  90. data/lib/earth/locality/census_division.rb +22 -0
  91. data/lib/earth/locality/census_division/data_miner.rb +64 -0
  92. data/lib/earth/locality/census_region.rb +13 -0
  93. data/lib/earth/locality/census_region/data_miner.rb +17 -0
  94. data/lib/earth/locality/climate_division.rb +17 -0
  95. data/lib/earth/locality/climate_division/data_miner.rb +20 -0
  96. data/lib/earth/locality/country.rb +13 -0
  97. data/lib/earth/locality/country/data_miner.rb +19 -0
  98. data/lib/earth/locality/data_miner.rb +10 -0
  99. data/lib/earth/locality/egrid_region.rb +15 -0
  100. data/lib/earth/locality/egrid_region/data_miner.rb +35 -0
  101. data/lib/earth/locality/egrid_subregion.rb +16 -0
  102. data/lib/earth/locality/egrid_subregion/data_miner.rb +65 -0
  103. data/lib/earth/locality/petroleum_administration_for_defense_district.rb +13 -0
  104. data/lib/earth/locality/petroleum_administration_for_defense_district/data_miner.rb +21 -0
  105. data/lib/earth/locality/state.rb +22 -0
  106. data/lib/earth/locality/state/data_miner.rb +37 -0
  107. data/lib/earth/locality/urbanity.rb +10 -0
  108. data/lib/earth/locality/urbanity/data_miner.rb +15 -0
  109. data/lib/earth/locality/zip_code.rb +23 -0
  110. data/lib/earth/locality/zip_code/data_miner.rb +43 -0
  111. data/lib/earth/pet.rb +4 -0
  112. data/lib/earth/pet/breed.rb +15 -0
  113. data/lib/earth/pet/breed/data_miner.rb +25 -0
  114. data/lib/earth/pet/breed_gender.rb +14 -0
  115. data/lib/earth/pet/breed_gender/data_miner.rb +21 -0
  116. data/lib/earth/pet/data_miner.rb +4 -0
  117. data/lib/earth/pet/gender.rb +10 -0
  118. data/lib/earth/pet/gender/data_miner.rb +13 -0
  119. data/lib/earth/pet/species.rb +40 -0
  120. data/lib/earth/pet/species/data_miner.rb +42 -0
  121. data/lib/earth/rail.rb +1 -0
  122. data/lib/earth/rail/data_miner.rb +1 -0
  123. data/lib/earth/rail/rail_class.rb +16 -0
  124. data/lib/earth/rail/rail_class/data_miner.rb +36 -0
  125. data/lib/earth/residence.rb +8 -0
  126. data/lib/earth/residence/air_conditioner_use.rb +13 -0
  127. data/lib/earth/residence/air_conditioner_use/data_miner.rb +22 -0
  128. data/lib/earth/residence/clothes_machine_use.rb +10 -0
  129. data/lib/earth/residence/clothes_machine_use/data_miner.rb +28 -0
  130. data/lib/earth/residence/data_miner.rb +8 -0
  131. data/lib/earth/residence/dishwasher_use.rb +10 -0
  132. data/lib/earth/residence/dishwasher_use/data_miner.rb +28 -0
  133. data/lib/earth/residence/residence_appliance.rb +16 -0
  134. data/lib/earth/residence/residence_appliance/data_miner.rb +20 -0
  135. data/lib/earth/residence/residence_class.rb +16 -0
  136. data/lib/earth/residence/residence_class/data_miner.rb +15 -0
  137. data/lib/earth/residence/residence_fuel_price.rb +18 -0
  138. data/lib/earth/residence/residence_fuel_price/data_miner.rb +200 -0
  139. data/lib/earth/residence/residence_fuel_type.rb +32 -0
  140. data/lib/earth/residence/residence_fuel_type/data_miner.rb +18 -0
  141. data/lib/earth/residence/residential_energy_consumption_survey_response.rb +39 -0
  142. data/lib/earth/residence/residential_energy_consumption_survey_response/data_miner.rb +283 -0
  143. data/spec/lib/earth_spec.rb +25 -0
  144. data/spec/spec_helper.rb +11 -0
  145. data/vendor/geokit-rails/CHANGELOG.rdoc +46 -0
  146. data/vendor/geokit-rails/MIT-LICENSE +20 -0
  147. data/vendor/geokit-rails/README.markdown +561 -0
  148. data/vendor/geokit-rails/Rakefile +18 -0
  149. data/vendor/geokit-rails/about.yml +9 -0
  150. data/vendor/geokit-rails/assets/api_keys_template +61 -0
  151. data/vendor/geokit-rails/init.rb +1 -0
  152. data/vendor/geokit-rails/install.rb +14 -0
  153. data/vendor/geokit-rails/lib/geokit-rails.rb +24 -0
  154. data/vendor/geokit-rails/lib/geokit-rails/acts_as_mappable.rb +456 -0
  155. data/vendor/geokit-rails/lib/geokit-rails/adapters/abstract.rb +31 -0
  156. data/vendor/geokit-rails/lib/geokit-rails/adapters/mysql.rb +22 -0
  157. data/vendor/geokit-rails/lib/geokit-rails/adapters/postgresql.rb +22 -0
  158. data/vendor/geokit-rails/lib/geokit-rails/adapters/sqlserver.rb +43 -0
  159. data/vendor/geokit-rails/lib/geokit-rails/defaults.rb +22 -0
  160. data/vendor/geokit-rails/lib/geokit-rails/geocoder_control.rb +16 -0
  161. data/vendor/geokit-rails/lib/geokit-rails/ip_geocode_lookup.rb +46 -0
  162. data/vendor/geokit-rails/test/acts_as_mappable_test.rb +474 -0
  163. data/vendor/geokit-rails/test/boot.rb +25 -0
  164. data/vendor/geokit-rails/test/database.yml +20 -0
  165. data/vendor/geokit-rails/test/fixtures/companies.yml +7 -0
  166. data/vendor/geokit-rails/test/fixtures/custom_locations.yml +54 -0
  167. data/vendor/geokit-rails/test/fixtures/locations.yml +54 -0
  168. data/vendor/geokit-rails/test/fixtures/mock_addresses.yml +17 -0
  169. data/vendor/geokit-rails/test/fixtures/mock_families.yml +2 -0
  170. data/vendor/geokit-rails/test/fixtures/mock_houses.yml +9 -0
  171. data/vendor/geokit-rails/test/fixtures/mock_organizations.yml +5 -0
  172. data/vendor/geokit-rails/test/fixtures/mock_people.yml +5 -0
  173. data/vendor/geokit-rails/test/fixtures/stores.yml +0 -0
  174. data/vendor/geokit-rails/test/ip_geocode_lookup_test.rb +77 -0
  175. data/vendor/geokit-rails/test/models/company.rb +3 -0
  176. data/vendor/geokit-rails/test/models/custom_location.rb +12 -0
  177. data/vendor/geokit-rails/test/models/location.rb +4 -0
  178. data/vendor/geokit-rails/test/models/mock_address.rb +4 -0
  179. data/vendor/geokit-rails/test/models/mock_family.rb +3 -0
  180. data/vendor/geokit-rails/test/models/mock_house.rb +3 -0
  181. data/vendor/geokit-rails/test/models/mock_organization.rb +4 -0
  182. data/vendor/geokit-rails/test/models/mock_person.rb +4 -0
  183. data/vendor/geokit-rails/test/models/store.rb +3 -0
  184. data/vendor/geokit-rails/test/schema.rb +60 -0
  185. data/vendor/geokit-rails/test/tasks.rake +31 -0
  186. data/vendor/geokit-rails/test/test_helper.rb +23 -0
  187. metadata +476 -0
@@ -0,0 +1,32 @@
1
+ class ResidenceFuelType < ActiveRecord::Base
2
+ set_primary_key :name
3
+
4
+ has_many :prices, :class_name => 'ResidenceFuelPrice', :foreign_key => 'residence_fuel_type_name'
5
+
6
+ data_miner do
7
+ tap "Brighter Planet's residence fuel types data", Earth.taps_server
8
+ end
9
+
10
+ def price_per_unit(relaxations = [])
11
+ conditions = { :residence_fuel_type_name => self }
12
+ relaxations.push Hash.new
13
+ relaxations.grab do |relaxation|
14
+ relaxation_conditions = Hash.new
15
+ if timeframe = relaxation[:timeframe]
16
+ relaxation_conditions[:year] = timeframe.from.year
17
+ relaxation_conditions[:month] = timeframe.from.month..timeframe.to.yesterday.month
18
+ end
19
+ if location = relaxation[:location]
20
+ relaxation_conditions[:locatable_type] = location.class.to_s
21
+ relaxation_conditions[:locatable_id] = location.id
22
+ end
23
+ ResidenceFuelPrice.average :price, :conditions => conditions.merge(relaxation_conditions)
24
+ end
25
+ end
26
+
27
+ class << self
28
+ def [](fuel)
29
+ find_by_name fuel.to_s.humanize.downcase
30
+ end
31
+ end
32
+ end
@@ -0,0 +1,18 @@
1
+ ResidenceFuelType.class_eval do
2
+ data_miner do
3
+ schema Earth.database_options do
4
+ string 'name'
5
+ float 'emission_factor'
6
+ string 'emission_factor_units'
7
+ # float 'energy_content'
8
+ # string 'energy_content_units'
9
+ end
10
+
11
+ import "a list of residential fuels and their emissions factors",
12
+ :url => 'http://spreadsheets.google.com/pub?key=rukxnmuhhsOsrztTrUaFCXQ' do
13
+ key 'name'
14
+ store 'emission_factor', :field_name => 'emission_factor', :units_field_name => 'units'
15
+ end
16
+ end
17
+ end
18
+
@@ -0,0 +1,39 @@
1
+ class ResidentialEnergyConsumptionSurveyResponse < ActiveRecord::Base
2
+ set_primary_key :department_of_energy_identifier
3
+
4
+ belongs_to :census_division, :foreign_key => 'census_division_number'
5
+ belongs_to :census_region, :foreign_key => 'census_region_number'
6
+ # what follows are entirely derived here
7
+ belongs_to :residence_class
8
+ belongs_to :urbanity
9
+ belongs_to :dishwasher_use
10
+ belongs_to :air_conditioner_use
11
+ belongs_to :clothes_machine_use
12
+
13
+ extend CohortScope
14
+ self.minimum_cohort_size = 5
15
+ SUBCOHORT_THRESHOLD = 5 # per Matt
16
+
17
+ INPUT_CHARACTERISTICS = [
18
+ :census_region,
19
+ :heating_degree_days,
20
+ :cooling_degree_days,
21
+ :residence_class,
22
+ :rooms,
23
+ :bedrooms,
24
+ :bathrooms,
25
+ :floorspace,
26
+ :residents,
27
+ :urbanity,
28
+ :construction_year,
29
+ :ownership,
30
+ ]
31
+
32
+ data_miner do
33
+ tap "Brighter Planet's sanitized RECS 2005", Earth.taps_server
34
+
35
+ process "pull dependencies" do
36
+ run_data_miner_on_belongs_to_associations
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,283 @@
1
+ ResidentialEnergyConsumptionSurveyResponse.class_eval do
2
+ data_miner do
3
+ # sabshere 9/20/10 sorted with sort -d -t "'" -k 2 ~/Desktop/parts.txt
4
+ schema Earth.database_options do
5
+ integer 'department_of_energy_identifier'
6
+ integer 'air_conditioner_use_id'
7
+ float 'annual_energy_from_electricity_for_air_conditioners'
8
+ string 'annual_energy_from_electricity_for_air_conditioners_units'
9
+ float 'annual_energy_from_electricity_for_clothes_driers'
10
+ string 'annual_energy_from_electricity_for_clothes_driers_units'
11
+ float 'annual_energy_from_electricity_for_dishwashers'
12
+ string 'annual_energy_from_electricity_for_dishwashers_units'
13
+ float 'annual_energy_from_electricity_for_freezers'
14
+ string 'annual_energy_from_electricity_for_freezers_units'
15
+ float 'annual_energy_from_electricity_for_heating_space'
16
+ string 'annual_energy_from_electricity_for_heating_space_units'
17
+ float 'annual_energy_from_electricity_for_heating_water'
18
+ string 'annual_energy_from_electricity_for_heating_water_units'
19
+ float 'annual_energy_from_electricity_for_other_appliances'
20
+ string 'annual_energy_from_electricity_for_other_appliances_units'
21
+ float 'annual_energy_from_electricity_for_refrigerators'
22
+ string 'annual_energy_from_electricity_for_refrigerators_units'
23
+ float 'annual_energy_from_fuel_oil_for_appliances'
24
+ string 'annual_energy_from_fuel_oil_for_appliances_units'
25
+ float 'annual_energy_from_fuel_oil_for_heating_space'
26
+ string 'annual_energy_from_fuel_oil_for_heating_space_units'
27
+ float 'annual_energy_from_fuel_oil_for_heating_water'
28
+ string 'annual_energy_from_fuel_oil_for_heating_water_units'
29
+ float 'annual_energy_from_kerosene'
30
+ string 'annual_energy_from_kerosene_units'
31
+ float 'annual_energy_from_natural_gas_for_appliances'
32
+ string 'annual_energy_from_natural_gas_for_appliances_units'
33
+ float 'annual_energy_from_natural_gas_for_heating_space'
34
+ string 'annual_energy_from_natural_gas_for_heating_space_units'
35
+ float 'annual_energy_from_natural_gas_for_heating_water'
36
+ string 'annual_energy_from_natural_gas_for_heating_water_units'
37
+ float 'annual_energy_from_propane_for_appliances'
38
+ string 'annual_energy_from_propane_for_appliances_units'
39
+ float 'annual_energy_from_propane_for_heating_space'
40
+ string 'annual_energy_from_propane_for_heating_space_units'
41
+ float 'annual_energy_from_propane_for_heating_water'
42
+ string 'annual_energy_from_propane_for_heating_water_units'
43
+ float 'annual_energy_from_wood'
44
+ string 'annual_energy_from_wood_units'
45
+ boolean 'attached_1car_garage'
46
+ boolean 'attached_2car_garage'
47
+ boolean 'attached_3car_garage'
48
+ float 'bathrooms'
49
+ integer 'bedrooms'
50
+ string 'census_division_name'
51
+ integer 'census_division_number'
52
+ string 'census_region_name'
53
+ integer 'census_region_number'
54
+ string 'central_ac_use'
55
+ string 'clothes_dryer_use'
56
+ integer 'clothes_machine_use_id'
57
+ string 'clothes_washer_use'
58
+ string 'construction_period'
59
+ date 'construction_year'
60
+ integer 'cooling_degree_days'
61
+ string 'cooling_degree_days_units'
62
+ boolean 'detached_1car_garage'
63
+ boolean 'detached_2car_garage'
64
+ boolean 'detached_3car_garage'
65
+ string 'dishwasher_use_id'
66
+ integer 'efficient_lights_on_1_to_4_hours'
67
+ integer 'efficient_lights_on_4_to_12_hours'
68
+ integer 'efficient_lights_on_over_12_hours'
69
+ float 'floorspace'
70
+ string 'floorspace_units'
71
+ integer 'freezer_count'
72
+ integer 'full_bathrooms'
73
+ integer 'half_bathrooms'
74
+ boolean 'heated_garage'
75
+ integer 'heating_degree_days'
76
+ string 'heating_degree_days_units'
77
+ float 'lighting_efficiency'
78
+ float 'lighting_use'
79
+ string 'lighting_use_units'
80
+ integer 'lights_on_1_to_4_hours'
81
+ integer 'lights_on_4_to_12_hours'
82
+ integer 'lights_on_over_12_hours'
83
+ integer 'outdoor_all_night_gas_lights'
84
+ integer 'outdoor_all_night_lights'
85
+ boolean 'ownership'
86
+ integer 'refrigerator_count'
87
+ string 'residence_class_id'
88
+ integer 'residents'
89
+ float 'rooms'
90
+ boolean 'thermostat_programmability'
91
+ integer 'total_rooms'
92
+ string 'urbanity_id'
93
+ float 'weighting'
94
+ string 'window_ac_use'
95
+ end
96
+
97
+ process "Define some unit conversions" do
98
+ Conversions.register :kbtus, :joules, 1_000.0 * 1_055.05585
99
+ Conversions.register :square_feet, :square_metres, 0.09290304
100
+ end
101
+
102
+ # conversions are NOT performed here, since we first have to zero out legitimate skips
103
+ # otherwise you will get values like "999 pounds = 453.138778 kilograms" (where 999 is really a legit skip)
104
+ import 'the 2005 EIA Residential Energy Consumption Survey microdata',
105
+ :url => 'http://www.eia.doe.gov/emeu/recs/recspubuse05/datafiles/RECS05alldata.csv',
106
+ :headers => :upcase do
107
+ key 'department_of_energy_identifier', :field_name => 'DOEID'
108
+
109
+ store 'residence_class_id', :field_name => 'TYPEHUQ', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/typehuq/typehuq.csv' }
110
+ store 'construction_year', :field_name => 'YEARMADE', :dictionary => { :input => 'Code', :sprintf => '%02d', :output => 'Date in the middle (synthetic)', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/yearmade/yearmade.csv' }
111
+ store 'construction_period', :field_name => 'YEARMADE', :dictionary => { :input => 'Code', :sprintf => '%02d', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/yearmade/yearmade.csv' }
112
+ store 'urbanity_id', :field_name => 'URBRUR', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/urbrur/urbrur.csv' }
113
+ store 'dishwasher_use_id', :field_name => 'DWASHUSE', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/dwashuse/dwashuse.csv' }
114
+ store 'central_ac_use', :field_name => 'USECENAC', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/usecenac/usecenac.csv' }
115
+ store 'window_ac_use', :field_name => 'USEWWAC', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/usewwac/usewwac.csv' }
116
+ store 'clothes_washer_use', :field_name => 'WASHLOAD', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/washload/washload.csv' }
117
+ store 'clothes_dryer_use', :field_name => 'DRYRUSE', :dictionary => { :input => 'Code', :output => 'Description', :url => 'http://github.com/brighterplanet/manually_curated_data/raw/master/dryruse/dryruse.csv' }
118
+
119
+ store 'census_division_number', :field_name => 'DIVISION'
120
+ store 'census_division_name', :field_name => 'DIVISION', :dictionary => { :input => 'number', :output => 'name', :url => 'http://data.brighterplanet.com/census_divisions.csv' }
121
+ store 'census_region_number', :field_name => 'DIVISION', :dictionary => { :input => 'number', :output => 'census_region_number', :url => 'http://data.brighterplanet.com/census_divisions.csv' }
122
+ store 'census_region_name', :field_name => 'DIVISION', :dictionary => { :input => 'number', :output => 'census_region_name', :url => 'http://data.brighterplanet.com/census_divisions.csv' }
123
+
124
+ store 'floorspace', :field_name => 'TOTSQFT', :units => :square_metres
125
+ store 'residents', :field_name => 'NHSLDMEM'
126
+ store 'refrigerator_count', :field_name => 'NUMFRIG'
127
+ store 'freezer_count', :field_name => 'NUMFREEZ'
128
+ store 'heating_degree_days', :field_name => 'HD65', :units => :degrees_fahrenheit_days # FIXME imperial
129
+ store 'cooling_degree_days', :field_name => 'CD65', :units => :degrees_fahrenheit_days # FIXME imperial
130
+ store 'annual_energy_from_fuel_oil_for_heating_space', :field_name => 'BTUFOSPH', :units => :joules
131
+ store 'annual_energy_from_fuel_oil_for_heating_water', :field_name => 'BTUFOWTH', :units => :joules
132
+ store 'annual_energy_from_fuel_oil_for_appliances', :field_name => 'BTUFOAPL', :units => :joules
133
+ store 'annual_energy_from_natural_gas_for_heating_space', :field_name => 'BTUNGSPH', :units => :joules
134
+ store 'annual_energy_from_natural_gas_for_heating_water', :field_name => 'BTUNGWTH', :units => :joules
135
+ store 'annual_energy_from_natural_gas_for_appliances', :field_name => 'BTUNGAPL', :units => :joules
136
+ store 'annual_energy_from_propane_for_heating_space', :field_name => 'BTULPSPH', :units => :joules
137
+ store 'annual_energy_from_propane_for_heating_water', :field_name => 'BTULPWTH', :units => :joules
138
+ store 'annual_energy_from_propane_for_appliances', :field_name => 'BTULPAPL', :units => :joules
139
+ store 'annual_energy_from_wood', :field_name => 'BTUWOOD', :units => :joules
140
+ store 'annual_energy_from_kerosene', :field_name => 'BTUKER', :units => :joules
141
+ store 'annual_energy_from_electricity_for_clothes_driers', :field_name => 'BTUELCDR', :units => :joules
142
+ store 'annual_energy_from_electricity_for_dishwashers', :field_name => 'BTUELDWH', :units => :joules
143
+ store 'annual_energy_from_electricity_for_freezers', :field_name => 'BTUELFZZ', :units => :joules
144
+ store 'annual_energy_from_electricity_for_refrigerators', :field_name => 'BTUELRFG', :units => :joules
145
+ store 'annual_energy_from_electricity_for_air_conditioners', :field_name => 'BTUELCOL', :units => :joules
146
+ store 'annual_energy_from_electricity_for_heating_space', :field_name => 'BTUELSPH', :units => :joules
147
+ store 'annual_energy_from_electricity_for_heating_water', :field_name => 'BTUELWTH', :units => :joules
148
+ store 'annual_energy_from_electricity_for_other_appliances', :field_name => 'BTUELAPL', :units => :joules
149
+ store 'weighting', :field_name => 'NWEIGHT'
150
+ store 'lighting_use', :static => nil, :units => :hours
151
+ store 'total_rooms', :field_name => 'TOTROOMS'
152
+ store 'full_bathrooms', :field_name => 'NCOMBATH'
153
+ store 'half_bathrooms', :field_name => 'NHAFBATH'
154
+ store 'bedrooms', :field_name => 'BEDROOMS'
155
+ store 'lights_on_1_to_4_hours', :field_name => 'LGT1'
156
+ store 'efficient_lights_on_1_to_4_hours', :field_name => 'LGT1EE'
157
+ store 'lights_on_4_to_12_hours', :field_name => 'LGT4'
158
+ store 'efficient_lights_on_4_to_12_hours', :field_name => 'LGT4EE'
159
+ store 'lights_on_over_12_hours', :field_name => 'LGT12'
160
+ store 'efficient_lights_on_over_12_hours', :field_name => 'LGT12EE'
161
+ store 'outdoor_all_night_lights', :field_name => 'NOUTLGTNT'
162
+ store 'outdoor_all_night_gas_lights', :field_name => 'NGASLIGHT'
163
+ # booleans where we treat anything other than true (for example legitimate skip or "occupied without paying rent") as false
164
+ store 'ownership', :synthesize => lambda { |row| row['KOWNRENT'] == '1' }
165
+ store 'thermostat_programmability', :synthesize => lambda { |row| row['PROTHERM'] == '1' }
166
+ store 'heated_garage', :synthesize => lambda { |row| row['GARGHEAT'] == '1' }
167
+ store 'attached_1car_garage', :synthesize => lambda { |row| row['GARAGE1C'] == '1' }
168
+ store 'detached_1car_garage', :synthesize => lambda { |row| row['DGARG1C'] == '1' }
169
+ store 'attached_2car_garage', :synthesize => lambda { |row| row['GARAGE2C'] == '1' }
170
+ store 'detached_2car_garage', :synthesize => lambda { |row| row['DGARG2C'] == '1' }
171
+ store 'attached_3car_garage', :synthesize => lambda { |row| row['GARAGE3C'] == '1' }
172
+ store 'detached_3car_garage', :synthesize => lambda { |row| row['DGARG3C'] == '1' }
173
+ end
174
+
175
+ # Rather than nullify the continuous variables that EIA identifies as LEGITIMATE SKIPS, we convert them to zero
176
+ # This makes it easier to derive useful information like "how many rooms does the house have?"
177
+ process "Zero out what the EIA calls LEGITIMATE SKIPS" do
178
+ %w{
179
+ annual_energy_from_electricity_for_air_conditioners
180
+ annual_energy_from_electricity_for_clothes_driers
181
+ annual_energy_from_electricity_for_dishwashers
182
+ annual_energy_from_electricity_for_freezers
183
+ annual_energy_from_electricity_for_heating_space
184
+ annual_energy_from_electricity_for_heating_water
185
+ annual_energy_from_electricity_for_other_appliances
186
+ annual_energy_from_electricity_for_refrigerators
187
+ annual_energy_from_fuel_oil_for_appliances
188
+ annual_energy_from_fuel_oil_for_heating_space
189
+ annual_energy_from_fuel_oil_for_heating_water
190
+ annual_energy_from_kerosene
191
+ annual_energy_from_propane_for_appliances
192
+ annual_energy_from_propane_for_heating_space
193
+ annual_energy_from_propane_for_heating_water
194
+ annual_energy_from_natural_gas_for_appliances
195
+ annual_energy_from_natural_gas_for_heating_space
196
+ annual_energy_from_natural_gas_for_heating_water
197
+ annual_energy_from_wood
198
+ lights_on_1_to_4_hours
199
+ lights_on_over_12_hours
200
+ efficient_lights_on_over_12_hours
201
+ efficient_lights_on_1_to_4_hours
202
+ lights_on_4_to_12_hours
203
+ efficient_lights_on_4_to_12_hours
204
+ outdoor_all_night_gas_lights
205
+ outdoor_all_night_lights
206
+ }.each do |attr_name|
207
+ max = maximum attr_name, :select => "CONVERT(#{attr_name}, UNSIGNED INTEGER)"
208
+ # if the maximum value of a row is all 999's, then it's a LEGITIMATE SKIP and we should set it to zero
209
+ if /^9+$/.match(max.to_i.to_s)
210
+ update_all "#{attr_name} = 0", "#{attr_name} = #{max}"
211
+ end
212
+ end
213
+ end
214
+
215
+ process "Convert units to metric" do
216
+ [
217
+ [ 'floorspace', :square_feet, :square_metres ],
218
+ [ 'annual_energy_from_fuel_oil_for_heating_space', :kbtus, :joules ],
219
+ [ 'annual_energy_from_fuel_oil_for_heating_water', :kbtus, :joules ],
220
+ [ 'annual_energy_from_fuel_oil_for_appliances', :kbtus, :joules ],
221
+ [ 'annual_energy_from_natural_gas_for_heating_space', :kbtus, :joules ],
222
+ [ 'annual_energy_from_natural_gas_for_heating_water', :kbtus, :joules ],
223
+ [ 'annual_energy_from_natural_gas_for_appliances', :kbtus, :joules ],
224
+ [ 'annual_energy_from_propane_for_heating_space', :kbtus, :joules ],
225
+ [ 'annual_energy_from_propane_for_heating_water', :kbtus, :joules ],
226
+ [ 'annual_energy_from_propane_for_appliances', :kbtus, :joules ],
227
+ [ 'annual_energy_from_wood', :kbtus, :joules ],
228
+ [ 'annual_energy_from_kerosene', :kbtus, :joules ],
229
+ [ 'annual_energy_from_electricity_for_clothes_driers', :kbtus, :joules ],
230
+ [ 'annual_energy_from_electricity_for_dishwashers', :kbtus, :joules ],
231
+ [ 'annual_energy_from_electricity_for_freezers', :kbtus, :joules ],
232
+ [ 'annual_energy_from_electricity_for_refrigerators', :kbtus, :joules ],
233
+ [ 'annual_energy_from_electricity_for_air_conditioners', :kbtus, :joules ],
234
+ [ 'annual_energy_from_electricity_for_heating_space', :kbtus, :joules ],
235
+ [ 'annual_energy_from_electricity_for_heating_water', :kbtus, :joules ],
236
+ [ 'annual_energy_from_electricity_for_other_appliances', :kbtus, :joules ],
237
+ ].each do |attr_name, from_units, to_units|
238
+ update_all "#{attr_name} = #{attr_name} * #{Conversions::Unit.exchange_rate from_units, to_units}"
239
+ end
240
+ end
241
+
242
+ process 'Add a new field "rooms" that estimates how many rooms are in the house' do
243
+ update_all 'rooms = total_rooms + full_bathrooms/2 + half_bathrooms/4 + heated_garage*(attached_1car_garage + detached_1car_garage + 2*(attached_2car_garage + detached_2car_garage) + 3*(attached_3car_garage + detached_3car_garage))'
244
+ end
245
+
246
+ process 'Add a new field "bathrooms" that synthesizes half and full bathrooms into one number' do
247
+ update_all 'bathrooms = full_bathrooms + 0.5 * half_bathrooms'
248
+ end
249
+
250
+ process 'Add a new field "lighting_use" that estimates how many hours light bulbs are turned on in the house' do
251
+ update_all 'lighting_use = 2*(lights_on_1_to_4_hours + efficient_lights_on_1_to_4_hours) + 8*(lights_on_4_to_12_hours + efficient_lights_on_4_to_12_hours) + 16*(lights_on_over_12_hours + efficient_lights_on_over_12_hours) + 12*(outdoor_all_night_lights + outdoor_all_night_gas_lights)'
252
+ end
253
+
254
+ process 'Add a new field "lighting_efficiency" that estimates what percentage of light bulbs in a house are energy-efficient' do
255
+ update_all 'lighting_efficiency = (2*efficient_lights_on_1_to_4_hours + 8*efficient_lights_on_4_to_12_hours + 16*efficient_lights_on_over_12_hours) / lighting_use'
256
+ end
257
+
258
+ process "synthesize air conditioner use from central AC and window AC use" do
259
+ update_all "air_conditioner_use_id = 'Turned on just about all summer'", " central_ac_use = 'Turned on just about all summer' OR window_ac_use = 'Turned on just about all summer'"
260
+ update_all "air_conditioner_use_id = 'Turned on quite a bit'", "(central_ac_use = 'Turned on quite a bit' OR window_ac_use = 'Turned on quite a bit') AND air_conditioner_use_id IS NULL"
261
+ update_all "air_conditioner_use_id = 'Turned on only a few days or nights when really needed'", "(central_ac_use = 'Turned on only a few days or nights when really needed' OR window_ac_use = 'Turned on only a few days or nights when really needed') AND air_conditioner_use_id IS NULL"
262
+ update_all "air_conditioner_use_id = 'Not used at all'", "(central_ac_use = 'Not used at all' OR window_ac_use = 'Not used at all') AND air_conditioner_use_id IS NULL"
263
+ end
264
+
265
+ process "synthesize clothes machine use from washer and dryer use" do
266
+ update_all "clothes_machine_use_id = clothes_washer_use", " clothes_dryer_use = 'Use it every time you wash clothes'"
267
+ update_all "clothes_machine_use_id = NULL", "clothes_washer_use IS NULL AND clothes_dryer_use = 'Use it for some, but not all, loads of wash'"
268
+ update_all "clothes_machine_use_id = '1 load or less each week'", "clothes_washer_use = '1 load or less each week' AND clothes_dryer_use = 'Use it for some, but not all, loads of wash'"
269
+ update_all "clothes_machine_use_id = '1 load or less each week'", "clothes_washer_use = '2 to 4 loads' AND clothes_dryer_use = 'Use it for some, but not all, loads of wash'"
270
+ update_all "clothes_machine_use_id = '2 to 4 loads'", "clothes_washer_use = '5 to 9 loads' AND clothes_dryer_use = 'Use it for some, but not all, loads of wash'"
271
+ update_all "clothes_machine_use_id = '5 to 9 loads'", "clothes_washer_use = '10 to 15 loads' AND clothes_dryer_use = 'Use it for some, but not all, loads of wash'"
272
+ update_all "clothes_machine_use_id = '10 to 15 loads'", "clothes_washer_use = 'More than 15 loads' AND clothes_dryer_use = 'Use it for some, but not all, loads of wash'"
273
+ update_all "clothes_machine_use_id = NULL", "clothes_washer_use IS NULL AND clothes_dryer_use = 'Use it infrequently'"
274
+ update_all "clothes_machine_use_id = '1 load or less each week'", "clothes_washer_use = '1 load or less each week' AND clothes_dryer_use = 'Use it infrequently'"
275
+ update_all "clothes_machine_use_id = '1 load or less each week'", "clothes_washer_use = '5 to 9 loads' AND clothes_dryer_use = 'Use it infrequently'"
276
+ update_all "clothes_machine_use_id = '2 to 4 loads'", "clothes_washer_use = '10 to 15 loads' AND clothes_dryer_use = 'Use it infrequently'"
277
+ update_all "clothes_machine_use_id = '5 to 9 loads'", "clothes_washer_use = 'More than 15 loads' AND clothes_dryer_use = 'Use it infrequently'"
278
+ end
279
+
280
+ # FIXME add precalc bathrooms per https://github.com/brighterplanet/cm1/commit/77df97c50311f3c59aad891f018bf3d487afeb98
281
+ end
282
+ end
283
+
@@ -0,0 +1,25 @@
1
+ require 'spec_helper'
2
+
3
+ describe Earth do
4
+ before :all do
5
+ Earth.init :all, :apply_schemas => true
6
+ end
7
+
8
+ it 'should require all Earth models' do
9
+ lambda do
10
+ Earth.resource_names.each { |k| k.constantize }
11
+ end.should_not raise_error(NameError)
12
+ end
13
+
14
+ it 'should include data_miner definitions' do
15
+ lambda do
16
+ Earth.resource_names.each { |k| k.constantize.should_receive(:data_miner) }
17
+ end
18
+ require 'earth/data_miner'
19
+ end
20
+
21
+ it 'should create a fallbacks table' do
22
+ Fallback.should be_table_exists
23
+ end
24
+ end
25
+
@@ -0,0 +1,11 @@
1
+ require 'rubygems'
2
+ require 'bundler'
3
+ Bundler.setup
4
+
5
+ require 'active_record'
6
+ require 'sqlite3'
7
+ ActiveRecord::Base.establish_connection :adapter => 'sqlite3',
8
+ :database => ':memory:'
9
+
10
+ $:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
11
+ require 'earth' # we do require Earth to live
@@ -0,0 +1,46 @@
1
+ == 2009-10-02 / Version 1.2.0
2
+ * Overhaul the test suite to be independent of a Rails project
3
+ * Added concept of database adapter. Ported mysql/postgresql conditional code to their own adapter.
4
+ * Added SQL Server support. THANKS http://github.com/brennandunn for all the improvements in this release
5
+
6
+ == 2009-09-26 / Version 1.1.3
7
+ * documentation updates and updated to work with Geokit gem v1.5.0
8
+ * IMPORTANT: in the Geokit gem, Geokit::Geocoders::timeout became Geokit::Geocoders::request_timeout for jruby compatibility.
9
+ The plugin sets this in config/initializers/geokit_config.rb. So if you've upgraded the gem to 1.5.0, you need to
10
+ make the change manually from Geokit::Geocoders::timeout to Geokit::Geocoders::request_timeout in config/initializers/geokit_config.rb
11
+
12
+ == 2009-06-08 / Version 1.1.2
13
+ * Added support for hashes in :through. So you can do: acts_as_mappable :through => { :state => :country } (Thanks José Valim).
14
+
15
+ == 2009-05-22 / Version 1.1.1
16
+ * Support for multiple ip geocoders (Thanks dreamcat4)
17
+ * Now checks if either :origin OR :bounds is passed, and proceeds with geokit query if this is true (Thanks Glenn Powell)
18
+ * Raises a helpful error if someone uses through but the association does not exists or was not defined yet (Thanks José Valim)
19
+
20
+ == 2009-04-11 / Version 1.1.0
21
+ * Fixed :through usages so that the through model is only included in the query if there
22
+ is an :origin passed in (Only if it is a geokit search) (Thanks Glenn Powell)
23
+ * Move library initialisation into lib/geokit-rails. init.rb uses lib/geokit-rails now (thanks Alban Peignier)
24
+ * Handle the case where a user passes a hash to the :conditions Finder option (thanks Adam Greene)
25
+ * Added ability to specify domain-specific API keys (Thanks Glenn Powell)
26
+
27
+ == 2009-02-20
28
+ * More powerful assosciations in the Rails Plugin:You can now specify a model as mappable "through" an associated model.
29
+ In other words, that associated model is the actual mappable model with "lat" and "lng" attributes, but this "through" model
30
+ can still utilize all Geokit's "find by distance" finders. Also Rails 2.3 compatibility (thanks github/glennpow)
31
+
32
+ == 2008-12-18
33
+ * Split Rails plugin from geocoder gem
34
+ * updated for Rails 2.2.2
35
+
36
+ == 2008-08-20
37
+ * Further fix of distance calculation, this time in SQL. Now uses least() function, which is available in MySQL version 3.22.5+ and postgres versions 8.1+
38
+
39
+ == 2008-01-16
40
+ * fixed the "zero-distance" bug (calculating between two points that are the same)
41
+
42
+ == 2007-11-12
43
+ * fixed a small but with queries crossing meridian, and also fixed find(:closest)
44
+
45
+ == 2007-10-11
46
+ * Fixed Rails2/Edge compatability