statement 0.9.4 → 0.9.5

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -41,23 +41,29 @@ puts results.first
41
41
 
42
42
  The sites that require HTML scraping are detailed in individual methods, and can be called individually or in bulk:
43
43
 
44
- ```
44
+ ```ruby
45
45
  results = Statement::Scraper.billnelson
46
46
  members = Statement::Scraper.member_scrapers
47
47
  ```
48
48
 
49
49
  Using the `koala` gem, Statement can fetch Facebook status feeds, given a Facebook ID. You'll need to either set environment variables `APP_ID` and `APP_SECRET` or create a `config.yml` file containing `app_id` and `app_secret` keys and values.
50
50
 
51
- ```
51
+ ```ruby
52
52
  f = Statement::Facebook.new
53
53
  results = f.feed('RepFincherTN08')
54
54
  ```
55
55
 
56
- It also can process IDs in batches, with a `slice` argument to indicate how many ids are passed in each batch:
56
+ It also can process IDs in batches by passing an array of IDs and a `slice` argument to indicate how many ids in each batch:
57
57
 
58
- ```
58
+ ```ruby
59
59
  f = Statement::Facebook.new
60
- results = f.batch(array, 10)
60
+ results = f.batch(facebook_ids, 10)
61
+ ```
62
+
63
+ In all cases Statement strips out posts that are not by the ID, and returns a Hash containing attributes from the feed:
64
+
65
+ ```ruby
66
+ {:id=>"9307301412_10151632750071413", :body=>"This is Gold Star Mother Larraine McGee whose son, Christopher Everett, Army National Guard, was killed in action September 2005. Precious family.", :link=>"http://www.facebook.com/photo.php?fbid=10151632750021413&set=a.118418671412.133511.9307301412&type=1&relevant_count=1", :title=>nil, :type=>"photo", :status_type=>"added_photos", :created_time=>#<DateTime: 2013-05-28T14:49:08+00:00 ((2456441j,53348s,0n),+0s,2299161j)>, :updated_time=>#<DateTime: 2013-05-28T17:41:37+00:00 ((2456441j,63697s,0n),+0s,2299161j)>, :facebook_id=>"9307301412"}
61
67
  ```
62
68
 
63
69
  ## Tests
@@ -16,14 +16,28 @@ module Statement
16
16
  end
17
17
 
18
18
  def feed(member_id)
19
- graph.get_connection(member_id, 'feed')
19
+ results = graph.get_connection(member_id, 'feed')
20
+ process_results(results.select{|r| r['from']['id'] == r['link']['id'].split('_').first})
20
21
  end
21
22
 
22
- # given an array of congressional facebook ids, pulls feeds in batches of 50.
23
+ # given an array of congressional facebook ids, pulls feeds in slices.
23
24
  def batch(member_ids, slice)
24
25
  results = []
25
26
  member_ids.each_slice(slice) do |members|
26
- results << graph.batch {|batch_api| members.each {|member| batch_api.get_connection(member, 'feed')}}
27
+ results << graph.batch do |batch_api|
28
+ members.each do |member|
29
+ batch_api.get_connection(member, 'feed')
30
+ end
31
+ end
32
+ end
33
+ process_results(results.flatten.select{|r| r['from']['id'] == r['id'].split('_').first})
34
+ end
35
+
36
+ def process_results(links)
37
+ results = []
38
+ links.each do |link|
39
+ facebook_id = link['id'].split('_').first
40
+ results << { :id => link['id'], :body => link['message'], :link => link['link'], :title => link['name'], :type => link['type'], :status_type => link['status_type'], :created_time => DateTime.parse(link['created_time']), :updated_time => DateTime.parse(link['updated_time']), :facebook_id => facebook_id }
27
41
  end
28
42
  results
29
43
  end
@@ -1,3 +1,3 @@
1
1
  module Statement
2
- VERSION = "0.9.4"
2
+ VERSION = "0.9.5"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: statement
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.9.4
4
+ version: 0.9.5
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,11 +9,11 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2013-05-24 00:00:00.000000000 Z
12
+ date: 2013-05-29 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: bundler
16
- requirement: &2156038320 !ruby/object:Gem::Requirement
16
+ requirement: &2151957300 !ruby/object:Gem::Requirement
17
17
  none: false
18
18
  requirements:
19
19
  - - ~>
@@ -21,10 +21,10 @@ dependencies:
21
21
  version: '1.3'
22
22
  type: :development
23
23
  prerelease: false
24
- version_requirements: *2156038320
24
+ version_requirements: *2151957300
25
25
  - !ruby/object:Gem::Dependency
26
26
  name: rake
27
- requirement: &2156037740 !ruby/object:Gem::Requirement
27
+ requirement: &2151956680 !ruby/object:Gem::Requirement
28
28
  none: false
29
29
  requirements:
30
30
  - - ! '>='
@@ -32,10 +32,10 @@ dependencies:
32
32
  version: '0'
33
33
  type: :development
34
34
  prerelease: false
35
- version_requirements: *2156037740
35
+ version_requirements: *2151956680
36
36
  - !ruby/object:Gem::Dependency
37
37
  name: webmock
38
- requirement: &2156037140 !ruby/object:Gem::Requirement
38
+ requirement: &2151972300 !ruby/object:Gem::Requirement
39
39
  none: false
40
40
  requirements:
41
41
  - - ! '>='
@@ -43,10 +43,10 @@ dependencies:
43
43
  version: '0'
44
44
  type: :development
45
45
  prerelease: false
46
- version_requirements: *2156037140
46
+ version_requirements: *2151972300
47
47
  - !ruby/object:Gem::Dependency
48
48
  name: american_date
49
- requirement: &2156036600 !ruby/object:Gem::Requirement
49
+ requirement: &2151971780 !ruby/object:Gem::Requirement
50
50
  none: false
51
51
  requirements:
52
52
  - - ! '>='
@@ -54,10 +54,10 @@ dependencies:
54
54
  version: '0'
55
55
  type: :runtime
56
56
  prerelease: false
57
- version_requirements: *2156036600
57
+ version_requirements: *2151971780
58
58
  - !ruby/object:Gem::Dependency
59
59
  name: nokogiri
60
- requirement: &2156035940 !ruby/object:Gem::Requirement
60
+ requirement: &2151971020 !ruby/object:Gem::Requirement
61
61
  none: false
62
62
  requirements:
63
63
  - - ! '>='
@@ -65,10 +65,10 @@ dependencies:
65
65
  version: '0'
66
66
  type: :runtime
67
67
  prerelease: false
68
- version_requirements: *2156035940
68
+ version_requirements: *2151971020
69
69
  - !ruby/object:Gem::Dependency
70
70
  name: koala
71
- requirement: &2156035320 !ruby/object:Gem::Requirement
71
+ requirement: &2151969860 !ruby/object:Gem::Requirement
72
72
  none: false
73
73
  requirements:
74
74
  - - ! '>='
@@ -76,10 +76,10 @@ dependencies:
76
76
  version: '0'
77
77
  type: :runtime
78
78
  prerelease: false
79
- version_requirements: *2156035320
79
+ version_requirements: *2151969860
80
80
  - !ruby/object:Gem::Dependency
81
81
  name: oj
82
- requirement: &2156034620 !ruby/object:Gem::Requirement
82
+ requirement: &2151968320 !ruby/object:Gem::Requirement
83
83
  none: false
84
84
  requirements:
85
85
  - - ! '>='
@@ -87,7 +87,7 @@ dependencies:
87
87
  version: '0'
88
88
  type: :runtime
89
89
  prerelease: false
90
- version_requirements: *2156034620
90
+ version_requirements: *2151968320
91
91
  description: Crawls congressional websites for press releases.
92
92
  email:
93
93
  - dwillis@gmail.com