reddit_bot 1.4.3 → 1.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +34 -34
- data/examples/Gemfile +2 -2
- data/examples/Gemfile.lock +6 -6
- data/examples/devflairbot/Gemfile +1 -1
- data/examples/devflairbot/Gemfile.lock +6 -5
- data/examples/devflairbot/main.rb +5 -2
- data/examples/get_dimensions.rb +26 -8
- data/examples/largeimages/Gemfile +7 -4
- data/examples/largeimages/Gemfile.lock +39 -25
- data/examples/largeimages/main.rb +39 -17
- data/examples/mlgtv/Gemfile +1 -1
- data/examples/mlgtv/Gemfile.lock +9 -1
- data/examples/mlgtv/channels.txt +16 -1
- data/examples/mlgtv/main.rb +15 -3
- data/lib/reddit_bot.rb +5 -5
- data/lib/reddit_bot/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9c7611c475346de3c940c8da95bbea7e104f6077
|
|
4
|
+
data.tar.gz: cef886a4b2707cebcd9e03609abc0ac018128e11
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9d3dc611b2dafe208539cf2592fc1b8f389bbc0bdadf2149a9806b76750332bb091cd95b3f7f2b31823863407c7ed155fb451c11e6c25e2d0d12659b556b4d93
|
|
7
|
+
data.tar.gz: 4ffc1f2d59001f024e978912dcaea03a412cedea6a594be3bd7044579d2bf38c93e3c9eddbddda28fb62a79de35ac92b0f1b235d26ed50917089415e51bf691a
|
data/README.md
CHANGED
|
@@ -3,32 +3,32 @@
|
|
|
3
3
|
[](https://gitter.im/Nakilon/reddit_bot)
|
|
4
4
|
[](http://badge.fury.io/rb/reddit_bot)
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
### What
|
|
7
7
|
|
|
8
8
|
This library provides an easy way to run bots and scripts that use Reddit API.
|
|
9
|
-
I ([/u/nakilon](https://reddit.com/u/nakilon)) currently run near 10 bots with it.
|
|
9
|
+
I ([/u/nakilon](https://www.reddit.com/u/nakilon)) currently run near 10 bots with it.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
### Why
|
|
12
12
|
|
|
13
13
|
Python (and so PRAW) sucks.
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
### Examples
|
|
16
16
|
|
|
17
|
+
I'm not usually publish every program (since there are thousand of them and adequate programmer knows that publishing everything is like throwing trash on street) but I'll add bots here so you could see how to use the library.
|
|
17
18
|
The [examples folder](examples) includes:
|
|
18
19
|
|
|
19
|
-
* sexypizza -- bot that updates wiki page with current flairs statistics
|
|
20
|
-
* devflairbot -- bot that flairs posts when some specifically flaired user comments there
|
|
21
|
-
* mlgtv -- bot that updates sidebar with currently streaming twitch channels
|
|
22
|
-
* councilofricks -- bot that flairs users according to Google Spreadsheet
|
|
23
|
-
* wallpaper -- bot that reports images with dimensions being not the same as in title
|
|
24
|
-
* cptflairbot3 -- bot that sets user flair according to request submitted via web form
|
|
25
|
-
also publishes its activily log here http://nakilon.pro/log.htm via Google Cloud Platform automations
|
|
26
|
-
* oneplus -- bot that removes and modmails about links to 1080x1920 images
|
|
27
|
-
* yayornay -- bot that flairs posts according to voting in top level comments
|
|
28
|
-
* realtimeww2 -- bot that posts tweets to a subreddit from a Twitter user timeline
|
|
29
|
-
*
|
|
30
|
-
|
|
31
|
-
For example, this showed that it would be ok to ignore /r/pics from now:
|
|
20
|
+
* **sexypizza** -- bot that updates wiki page with current flairs statistics
|
|
21
|
+
* **devflairbot** -- bot that flairs posts when some specifically flaired user comments there
|
|
22
|
+
* **mlgtv** -- bot that updates sidebar with currently streaming twitch channels
|
|
23
|
+
* **councilofricks** -- bot that flairs users according to Google Spreadsheet
|
|
24
|
+
* **wallpaper** -- bot that reports images with dimensions being not the same as in title
|
|
25
|
+
* **cptflairbot3** -- bot that sets user flair according to request submitted via web form
|
|
26
|
+
also publishes its activily log here http://www.nakilon.pro/casualpokemontrades/log.htm via Google Cloud Platform automations (Apps Script and Functions)
|
|
27
|
+
* **oneplus** -- bot that removes and modmails about links to 1080x1920 images
|
|
28
|
+
* **yayornay** -- bot that flairs posts according to voting in top level comments
|
|
29
|
+
* **realtimeww2** -- bot that posts tweets to a subreddit from a Twitter user timeline
|
|
30
|
+
* **largeimages** -- this was my first bot -- it uses two approaches to track the most high resolution photos posted anywhere on Reddit to x-post them to [subreddit /r/largeimages](https://www.reddit.com/r/largeimages)
|
|
31
|
+
* **largeimagesreview** -- script that was used /r/largeimages to calculates quality of x-posts from different subreddits based on mods activity (remove/approve) so it showed that /r/pics and /r/foodporn should better be excluded:
|
|
32
32
|
|
|
33
33
|
pics Total: 98 Quality: 19%
|
|
34
34
|
wallpapers Total: 69 Quality: 52%
|
|
@@ -41,8 +41,6 @@ The [examples folder](examples) includes:
|
|
|
41
41
|
carporn Total: 11 Quality: 45%
|
|
42
42
|
InfrastructurePorn Total: 9 Quality: 77%
|
|
43
43
|
|
|
44
|
-
Later version of this script also shows remove/approve statuses sorted by linked image resolution:
|
|
45
|
-
|
|
46
44
|
EarthPorn Total: 23 Quality: 82% ✅⛔✅✅✅✅⛔✅✅✅✅✅⛔⛔✅✅✅✅✅✅✅✅✅
|
|
47
45
|
FoodPorn Total: 5 Quality: 0% ⛔⛔⛔⛔⛔
|
|
48
46
|
carporn Total: 4 Quality: 0% ⛔⛔⛔⛔
|
|
@@ -57,35 +55,37 @@ The [examples folder](examples) includes:
|
|
|
57
55
|
ArchitecturePorn Total: 2 Quality: 50% ✅⛔
|
|
58
56
|
AbandonedPorn Total: 2 Quality: 100% ✅✅
|
|
59
57
|
|
|
60
|
-
|
|
58
|
+
### Usage
|
|
61
59
|
|
|
62
|
-
|
|
63
|
-
:client_secret: Fqo.....................AFI
|
|
64
|
-
:password: mybotpassword
|
|
65
|
-
:login: MyBotUsername
|
|
60
|
+
$ gem install reddit_bot
|
|
66
61
|
|
|
67
|
-
|
|
62
|
+
or via Gemfile:
|
|
68
63
|
|
|
69
|
-
|
|
64
|
+
source "https://rubygems.org"
|
|
65
|
+
gem "reddit_bot"
|
|
70
66
|
|
|
71
67
|
helloworld.rb:
|
|
72
68
|
|
|
73
69
|
require "reddit_bot"
|
|
74
70
|
|
|
75
|
-
|
|
71
|
+
You obviously can't run these examples as is, because they use some dependencies that are not in git repo, like `secrets.yaml` file for Reddit authorization of the following format:
|
|
76
72
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
TODO: manual on how to create bots with Reddit web interface and run via bash console
|
|
82
|
-
TODO: about NetHTTPUtils dependency; and that get_rersponse returns String code not Integer like request_data
|
|
73
|
+
:client_id: Kb9.......6wBw
|
|
74
|
+
:client_secret: Fqo.....................AFI
|
|
75
|
+
:password: mybotpassword
|
|
76
|
+
:login: MyBotUsername
|
|
83
77
|
|
|
84
78
|
To update the gem version in Gemfile.lock when using Gemfile like this: `gem "reddit_bot", "~>1.1.0"`, do the:
|
|
85
79
|
|
|
86
80
|
$ bundle update reddit_bot
|
|
87
81
|
|
|
88
|
-
|
|
82
|
+
### Contributing and License
|
|
89
83
|
|
|
90
84
|
Bug reports and pull requests are welcome.
|
|
91
85
|
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|
|
86
|
+
|
|
87
|
+
### TODO
|
|
88
|
+
|
|
89
|
+
* write more usage instructions here
|
|
90
|
+
* manual on how to create bots with Reddit web interface and run via bash console
|
|
91
|
+
* about NetHTTPUtils dependency; and that get_response returns String code not Integer like request_data
|
data/examples/Gemfile
CHANGED
|
@@ -2,6 +2,6 @@ source "https://rubygems.org"
|
|
|
2
2
|
|
|
3
3
|
gem "json"
|
|
4
4
|
|
|
5
|
-
gem "nethttputils", git: "git@github.com:Nakilon/nethttputils.git", tag: "v0.0.
|
|
6
|
-
gem "imgur2array", git: "git@github.com:Nakilon/imgur2array.git", tag: "v0.0.2.
|
|
5
|
+
gem "nethttputils", git: "git@github.com:Nakilon/nethttputils.git", tag: "v0.0.4.1"
|
|
6
|
+
gem "imgur2array", git: "git@github.com:Nakilon/imgur2array.git", tag: "v0.0.2.7"
|
|
7
7
|
gem "fastimage", "1.7.0"
|
data/examples/Gemfile.lock
CHANGED
|
@@ -1,16 +1,16 @@
|
|
|
1
1
|
GIT
|
|
2
2
|
remote: git@github.com:Nakilon/imgur2array.git
|
|
3
|
-
revision:
|
|
4
|
-
tag: v0.0.2.
|
|
3
|
+
revision: eea2aa8a39d72726318aae8805b66bcddd7acbb6
|
|
4
|
+
tag: v0.0.2.7
|
|
5
5
|
specs:
|
|
6
|
-
imgur2array (0.0.2.
|
|
6
|
+
imgur2array (0.0.2.7)
|
|
7
7
|
|
|
8
8
|
GIT
|
|
9
9
|
remote: git@github.com:Nakilon/nethttputils.git
|
|
10
|
-
revision:
|
|
11
|
-
tag: v0.0.
|
|
10
|
+
revision: 84ac32db6d44b68f5bd6e2f1ce8698fc5e2ad968
|
|
11
|
+
tag: v0.0.4.1
|
|
12
12
|
specs:
|
|
13
|
-
nethttputils (0.0.
|
|
13
|
+
nethttputils (0.0.4.1)
|
|
14
14
|
|
|
15
15
|
GEM
|
|
16
16
|
remote: https://rubygems.org/
|
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
GIT
|
|
2
|
-
remote:
|
|
3
|
-
revision:
|
|
2
|
+
remote: git@github.com:Nakilon/nethttputils.git
|
|
3
|
+
revision: 856f3a44f48c314ed3fdcf4532f7a7dd85ec31ef
|
|
4
|
+
tag: v0.0.3.1
|
|
4
5
|
specs:
|
|
5
|
-
|
|
6
|
+
nethttputils (0.0.3.1)
|
|
6
7
|
|
|
7
8
|
GEM
|
|
8
9
|
remote: https://rubygems.org/
|
|
@@ -15,8 +16,8 @@ PLATFORMS
|
|
|
15
16
|
ruby
|
|
16
17
|
|
|
17
18
|
DEPENDENCIES
|
|
18
|
-
|
|
19
|
+
nethttputils!
|
|
19
20
|
reddit_bot (~> 1.3.0)
|
|
20
21
|
|
|
21
22
|
BUNDLED WITH
|
|
22
|
-
1.
|
|
23
|
+
1.16.0
|
|
@@ -12,9 +12,12 @@ loop do
|
|
|
12
12
|
].each do |subreddit, developer_class|
|
|
13
13
|
puts "sub: #{subreddit}"
|
|
14
14
|
|
|
15
|
-
JSON.parse(
|
|
15
|
+
JSON.parse( begin
|
|
16
16
|
NetHTTPUtils.request_data "https://www.reddit.com/r/#{subreddit}/comments.json", header: ["User-Agent", "ajsdjasdasd"]
|
|
17
|
-
|
|
17
|
+
rescue NetHTTPUtils::Error => e
|
|
18
|
+
raise unless e.code == 503
|
|
19
|
+
retry
|
|
20
|
+
end )["data"]["children"].each do |comment|
|
|
18
21
|
id = comment["data"]["link_id"][3..-1]
|
|
19
22
|
commenter_flair = comment["data"]["author_flair_css_class"]
|
|
20
23
|
# puts "flair: #{commenter_flair}" if commenter_flair
|
data/examples/get_dimensions.rb
CHANGED
|
@@ -10,13 +10,13 @@ module GetDimensions
|
|
|
10
10
|
class Error404 < RuntimeError
|
|
11
11
|
def initialize url
|
|
12
12
|
# Module.nesting[1].logger.error url
|
|
13
|
-
super "GetDimensions NotFound error for #{url}"
|
|
13
|
+
super "GetDimensions: NotFound error for #{url}"
|
|
14
14
|
end
|
|
15
15
|
end
|
|
16
16
|
class ErrorUnknown < RuntimeError
|
|
17
17
|
def initialize url
|
|
18
18
|
# Module.nesting[1].logger.error url
|
|
19
|
-
super "GetDimensions
|
|
19
|
+
super "GetDimensions: fastimage can't get dimensions for unknown url #{url}"
|
|
20
20
|
end
|
|
21
21
|
end
|
|
22
22
|
|
|
@@ -32,11 +32,14 @@ module GetDimensions
|
|
|
32
32
|
%r{^https?://www\.reddit\.com/},
|
|
33
33
|
%r{^http://vimeo\.com/},
|
|
34
34
|
].any?{ |r| r =~ url }
|
|
35
|
-
|
|
35
|
+
|
|
36
|
+
begin
|
|
36
37
|
URI url
|
|
37
38
|
rescue URI::InvalidURIError
|
|
38
39
|
return :skipped
|
|
39
|
-
end
|
|
40
|
+
end
|
|
41
|
+
# return :skipped if %w{ minus com } == .host.split(?.).last(2)
|
|
42
|
+
|
|
40
43
|
fi = lambda do |url|
|
|
41
44
|
_ = FastImage.size url
|
|
42
45
|
_ ? [*_, url] : fail
|
|
@@ -47,7 +50,13 @@ module GetDimensions
|
|
|
47
50
|
[*_, url] if _
|
|
48
51
|
},
|
|
49
52
|
->_{ if %w{ imgur com } == URI(_).host.split(?.).last(2)
|
|
50
|
-
dimensions =
|
|
53
|
+
dimensions = begin
|
|
54
|
+
Imgur::imgur_to_array _
|
|
55
|
+
rescue Imgur::Error => e
|
|
56
|
+
raise Error404.new _ if e.to_s.start_with? "Imgur error: bad link pattern"
|
|
57
|
+
raise
|
|
58
|
+
end
|
|
59
|
+
raise Error404.new _ if !dimensions || dimensions.empty?
|
|
51
60
|
[
|
|
52
61
|
*dimensions.max_by{ |u, x, y, t| x * y }.take(3).rotate(1),
|
|
53
62
|
*dimensions.map(&:first),
|
|
@@ -88,7 +97,13 @@ module GetDimensions
|
|
|
88
97
|
consumer_key: ENV["_500PX_CONSUMER_KEY"],
|
|
89
98
|
} )["photo"].values_at("width", "height", "image_url")
|
|
90
99
|
end },
|
|
91
|
-
->_{
|
|
100
|
+
->_{
|
|
101
|
+
raise Error404.new _ if "404" == begin
|
|
102
|
+
NetHTTPUtils.get_response _
|
|
103
|
+
rescue SocketError => e
|
|
104
|
+
raise Error404.new _
|
|
105
|
+
end.code
|
|
106
|
+
},
|
|
92
107
|
->_{ raise ErrorUnknown.new _ },
|
|
93
108
|
].lazy.map{ |_| _[url] }.find{ |_| _ }
|
|
94
109
|
end
|
|
@@ -99,7 +114,7 @@ if $0 == __FILE__
|
|
|
99
114
|
puts "self testing..."
|
|
100
115
|
|
|
101
116
|
[
|
|
102
|
-
["http://minus.com/lkP3hgRJd9npi",
|
|
117
|
+
["http://minus.com/lkP3hgRJd9npi", GetDimensions::Error404],
|
|
103
118
|
["http://example.com", GetDimensions::ErrorUnknown],
|
|
104
119
|
["http://i.imgur.com/7xcxxkR.gifv", :skipped],
|
|
105
120
|
["http://imgur.com/HQHBBBD", [1024, 768, "https://i.imgur.com/HQHBBBD.jpg",
|
|
@@ -127,11 +142,14 @@ if $0 == __FILE__
|
|
|
127
142
|
["http://commons.wikimedia.org/wiki/File:Eduard_Bohlen_anagoria.jpg", [4367, 2928, "https://upload.wikimedia.org/wikipedia/commons/0/0d/Eduard_Bohlen_anagoria.jpg"]],
|
|
128
143
|
["https://500px.com/photo/112134597/milky-way-by-tom-hall", [4928, 2888, "https://drscdn.500px.org/photo/112134597/m%3D2048_k%3D1_a%3D1/v2?client_application_id=18857&webp=true&sig=c0d31cf9395d7849fbcce612ca9909225ec16fd293a7f460ea15d9e6a6c34257"]],
|
|
129
144
|
["https://i.redd.it/si758zk7r5xz.jpg", GetDimensions::Error404],
|
|
145
|
+
["http://www.cutehalloweencostumeideas.org/wp-content/uploads/2017/10/Niagara-Falls_04.jpg", GetDimensions::Error404],
|
|
146
|
+
["https://imgur.com/gallery/YO49F.", GetDimensions::Error404], # expect "Imgur error: bad link pattern"
|
|
147
|
+
["https://imgur.com/mM4Dh7Z", GetDimensions::Error404], # redirects to https://i.imgur.com/removed.png
|
|
130
148
|
].each do |input, expectation|
|
|
131
149
|
puts "testing #{input}"
|
|
132
150
|
if expectation.is_a? Class
|
|
133
151
|
begin
|
|
134
|
-
GetDimensions::get_dimensions input
|
|
152
|
+
p GetDimensions::get_dimensions input
|
|
135
153
|
fail
|
|
136
154
|
rescue expectation
|
|
137
155
|
end
|
|
@@ -2,11 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
gem "json"
|
|
4
4
|
|
|
5
|
-
gem "nethttputils", git: "git@github.com:Nakilon/nethttputils.git", tag: "v0.0.
|
|
5
|
+
gem "nethttputils", git: "git@github.com:Nakilon/nethttputils.git", tag: "v0.1.0.0"
|
|
6
|
+
|
|
6
7
|
gem "reddit_bot", "~>1.4.0"
|
|
7
8
|
gem "nokogiri"
|
|
8
9
|
|
|
9
|
-
gem "imgur2array", git: "git@github.com:Nakilon/imgur2array.git", tag: "v0.0.2.
|
|
10
|
-
gem "fastimage", "1.7.0"
|
|
10
|
+
gem "imgur2array", git: "git@github.com:Nakilon/imgur2array.git", tag: "v0.0.2.8"
|
|
11
|
+
gem "fastimage", "~>1.7.0"
|
|
12
|
+
gem "url2dimensions", git: "git@github.com:Nakilon/url2dimensions.git", tag: "v0.0.0.4"
|
|
11
13
|
|
|
12
|
-
gem "gcplogger", git: "git@github.com:Nakilon/gcplogger.git", tag: "v0.0.
|
|
14
|
+
gem "gcplogger", git: "git@github.com:Nakilon/gcplogger.git", tag: "v0.0.2.0"
|
|
15
|
+
gem "google-cloud-error_reporting"
|
|
@@ -1,67 +1,78 @@
|
|
|
1
1
|
GIT
|
|
2
2
|
remote: git@github.com:Nakilon/gcplogger.git
|
|
3
|
-
revision:
|
|
4
|
-
tag: v0.0.
|
|
3
|
+
revision: 409838cfb0cff225bb1c1b7e686cd23f2be910d4
|
|
4
|
+
tag: v0.0.2.0
|
|
5
5
|
specs:
|
|
6
|
-
gcplogger (0.0.
|
|
6
|
+
gcplogger (0.0.2.0)
|
|
7
7
|
google-cloud-logging
|
|
8
8
|
|
|
9
9
|
GIT
|
|
10
10
|
remote: git@github.com:Nakilon/imgur2array.git
|
|
11
|
-
revision:
|
|
12
|
-
tag: v0.0.2.
|
|
11
|
+
revision: 78fdf98f04a1303430365eff47674c699eae3602
|
|
12
|
+
tag: v0.0.2.8
|
|
13
13
|
specs:
|
|
14
|
-
imgur2array (0.0.2.
|
|
14
|
+
imgur2array (0.0.2.8)
|
|
15
15
|
|
|
16
16
|
GIT
|
|
17
17
|
remote: git@github.com:Nakilon/nethttputils.git
|
|
18
|
-
revision:
|
|
19
|
-
tag: v0.0.
|
|
18
|
+
revision: 547dae5f3c6c893424b30498efa442e4d268b1b4
|
|
19
|
+
tag: v0.1.0.0
|
|
20
|
+
specs:
|
|
21
|
+
nethttputils (0.1.0.0)
|
|
22
|
+
|
|
23
|
+
GIT
|
|
24
|
+
remote: git@github.com:Nakilon/url2dimensions.git
|
|
25
|
+
revision: 7364b82617bcdb7fafd5dd2bee268fedefe24074
|
|
26
|
+
tag: v0.0.0.4
|
|
20
27
|
specs:
|
|
21
|
-
|
|
28
|
+
url2dimensions (0.0.0.4)
|
|
22
29
|
|
|
23
30
|
GEM
|
|
24
31
|
remote: https://rubygems.org/
|
|
25
32
|
specs:
|
|
26
|
-
addressable (2.
|
|
33
|
+
addressable (2.5.2)
|
|
34
|
+
public_suffix (>= 2.0.2, < 4.0)
|
|
27
35
|
faraday (0.13.1)
|
|
28
36
|
multipart-post (>= 1.2, < 3)
|
|
29
37
|
fastimage (1.7.0)
|
|
30
38
|
addressable (~> 2.3, >= 2.3.5)
|
|
31
|
-
google-cloud-core (1.
|
|
39
|
+
google-cloud-core (1.1.0)
|
|
32
40
|
google-cloud-env (~> 1.0)
|
|
33
|
-
googleauth (~> 0.5.1)
|
|
34
41
|
google-cloud-env (1.0.1)
|
|
35
42
|
faraday (~> 0.11)
|
|
36
|
-
google-cloud-
|
|
37
|
-
google-cloud-core (~> 1.
|
|
38
|
-
google-gax (~> 0.
|
|
43
|
+
google-cloud-error_reporting (0.28.1)
|
|
44
|
+
google-cloud-core (~> 1.1)
|
|
45
|
+
google-gax (~> 0.10.1)
|
|
46
|
+
stackdriver-core (~> 1.2)
|
|
47
|
+
google-cloud-logging (1.3.2)
|
|
48
|
+
google-cloud-core (~> 1.1)
|
|
49
|
+
google-gax (~> 0.10.1)
|
|
39
50
|
stackdriver-core (~> 1.2)
|
|
40
|
-
google-gax (0.
|
|
51
|
+
google-gax (0.10.2)
|
|
41
52
|
google-protobuf (~> 3.2)
|
|
42
|
-
googleapis-common-protos (
|
|
43
|
-
googleauth (~> 0.
|
|
44
|
-
grpc (
|
|
53
|
+
googleapis-common-protos (>= 1.3.5, < 2.0)
|
|
54
|
+
googleauth (~> 0.6.2)
|
|
55
|
+
grpc (>= 1.7.2, < 2.0)
|
|
45
56
|
rly (~> 0.2.3)
|
|
46
|
-
google-protobuf (3.
|
|
57
|
+
google-protobuf (3.5.0)
|
|
47
58
|
googleapis-common-protos (1.3.7)
|
|
48
59
|
google-protobuf (~> 3.0)
|
|
49
60
|
googleapis-common-protos-types (~> 1.0)
|
|
50
61
|
grpc (~> 1.0)
|
|
51
62
|
googleapis-common-protos-types (1.0.1)
|
|
52
63
|
google-protobuf (~> 3.0)
|
|
53
|
-
googleauth (0.
|
|
64
|
+
googleauth (0.6.2)
|
|
54
65
|
faraday (~> 0.12)
|
|
55
|
-
jwt (
|
|
66
|
+
jwt (>= 1.4, < 3.0)
|
|
56
67
|
logging (~> 2.0)
|
|
57
68
|
memoist (~> 0.12)
|
|
58
69
|
multi_json (~> 1.11)
|
|
59
70
|
os (~> 0.9)
|
|
60
71
|
signet (~> 0.7)
|
|
61
|
-
grpc (1.
|
|
72
|
+
grpc (1.7.3)
|
|
62
73
|
google-protobuf (~> 3.1)
|
|
63
74
|
googleapis-common-protos-types (~> 1.0.0)
|
|
64
|
-
googleauth (
|
|
75
|
+
googleauth (>= 0.5.1, < 0.7)
|
|
65
76
|
json (2.1.0)
|
|
66
77
|
jwt (1.5.6)
|
|
67
78
|
little-plugger (1.1.4)
|
|
@@ -75,6 +86,7 @@ GEM
|
|
|
75
86
|
nokogiri (1.6.8.1)
|
|
76
87
|
mini_portile2 (~> 2.1.0)
|
|
77
88
|
os (0.9.6)
|
|
89
|
+
public_suffix (2.0.5)
|
|
78
90
|
reddit_bot (1.4.0)
|
|
79
91
|
json
|
|
80
92
|
rly (0.2.3)
|
|
@@ -89,13 +101,15 @@ PLATFORMS
|
|
|
89
101
|
ruby
|
|
90
102
|
|
|
91
103
|
DEPENDENCIES
|
|
92
|
-
fastimage (
|
|
104
|
+
fastimage (~> 1.7.0)
|
|
93
105
|
gcplogger!
|
|
106
|
+
google-cloud-error_reporting
|
|
94
107
|
imgur2array!
|
|
95
108
|
json
|
|
96
109
|
nethttputils!
|
|
97
110
|
nokogiri
|
|
98
111
|
reddit_bot (~> 1.4.0)
|
|
112
|
+
url2dimensions!
|
|
99
113
|
|
|
100
114
|
BUNDLED WITH
|
|
101
115
|
1.16.0
|
|
@@ -1,14 +1,20 @@
|
|
|
1
1
|
### THIS WAS MY THE VERY FIRST REDDIT BOT
|
|
2
2
|
|
|
3
3
|
|
|
4
|
-
require "nokogiri"
|
|
5
|
-
|
|
6
4
|
require "gcplogger"
|
|
7
5
|
logger = GCPLogger.logger "largeimagesbot"
|
|
8
6
|
|
|
9
|
-
|
|
7
|
+
fail("no ENV['ERROR_REPORTING_KEYFILE'] specified") unless ENV["ERROR_REPORTING_KEYFILE"]
|
|
8
|
+
require "google/cloud/error_reporting"
|
|
9
|
+
Google::Cloud::ErrorReporting.configure do |config|
|
|
10
|
+
config.project_id = (JSON.load File.read ENV["ERROR_REPORTING_KEYFILE"])["project_id"]
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
require "url2dimensions"
|
|
10
15
|
Imgur.logger = logger
|
|
11
16
|
|
|
17
|
+
require "nokogiri"
|
|
12
18
|
|
|
13
19
|
require "../boilerplate"
|
|
14
20
|
BOT = RedditBot::Bot.new YAML.load File.read "secrets.yaml"
|
|
@@ -24,15 +30,30 @@ INCLUDE = %w{
|
|
|
24
30
|
r/WQHD_Wallpaper
|
|
25
31
|
|
|
26
32
|
r/pic
|
|
33
|
+
|
|
34
|
+
r/oldmaps
|
|
27
35
|
}
|
|
28
36
|
EXCLUDE = %w{ foodporn powerwashingporn }
|
|
29
37
|
|
|
30
38
|
checked = []
|
|
31
39
|
|
|
32
40
|
loop do
|
|
33
|
-
|
|
41
|
+
begin
|
|
42
|
+
logger.warn "LOOP #{Time.now}"
|
|
43
|
+
rescue => e
|
|
44
|
+
puts "oops"
|
|
45
|
+
Google::Cloud::ErrorReporting.report e
|
|
46
|
+
sleep 5
|
|
47
|
+
raise
|
|
48
|
+
end
|
|
34
49
|
|
|
35
|
-
[ [:source_ultireddit,
|
|
50
|
+
[ [:source_ultireddit, 10000000, ( Nokogiri::XML( begin
|
|
51
|
+
NetHTTPUtils.request_data ENV["FEEDPCBR_URL"]
|
|
52
|
+
rescue NetHTTPUtils::Error => e
|
|
53
|
+
raise unless [502, 504].include? e.code
|
|
54
|
+
sleep 60
|
|
55
|
+
retry
|
|
56
|
+
end ).remove_namespaces!.xpath("feed/entry").map do |entry|
|
|
36
57
|
[
|
|
37
58
|
entry.at_xpath("id").text,
|
|
38
59
|
entry.at_xpath("link[@rel='via']")["href"],
|
|
@@ -42,7 +63,7 @@ loop do
|
|
|
42
63
|
entry.at_xpath("link[@rel='alternate']")["href"],
|
|
43
64
|
]
|
|
44
65
|
end ) ],
|
|
45
|
-
[:source_reddit,
|
|
66
|
+
[:source_reddit, 30000000, ( INCLUDE.flat_map do |sortasub|
|
|
46
67
|
BOT.new_posts(sortasub).take(100).map do |child|
|
|
47
68
|
next if child["is_self"]
|
|
48
69
|
next if EXCLUDE.include? child["subreddit"].downcase
|
|
@@ -58,19 +79,20 @@ loop do
|
|
|
58
79
|
checked << id
|
|
59
80
|
# next if Gem::Platform.local.os == "darwin" # prevent concurrent posting
|
|
60
81
|
logger.debug "image url for #{id}: #{url}"
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
82
|
+
next logger.warn "skipped a post by /u/sjhill" if author == "sjhill"
|
|
83
|
+
|
|
84
|
+
next logger.warn "skipped (URL2Dimensions :skipped) #{url} from http://redd.it/#{id}" if :skipped == _ = begin
|
|
85
|
+
URL2Dimensions::get_dimensions CGI.unescape_html url
|
|
86
|
+
rescue URL2Dimensions::Error404
|
|
87
|
+
next logger.warn "skipped (URL2Dimensions::Error404) #{url} from http://redd.it/#{id}"
|
|
88
|
+
rescue URL2Dimensions::ErrorUnknown
|
|
89
|
+
next logger.warn "skipped (URL2Dimensions::ErrorUnknown) #{url} from http://redd.it/#{id}"
|
|
68
90
|
end
|
|
69
91
|
fail "unable #{url} from http://redd.it/#{id}" unless _
|
|
70
92
|
width, height, best_direct_url, *all_direct_urls = _
|
|
71
|
-
logger.info "
|
|
93
|
+
logger.info "URL2Dimensions: %p" % [[width, height, best_direct_url, all_direct_urls.size]]
|
|
72
94
|
unless min_resolution <= width * height
|
|
73
|
-
next logger.
|
|
95
|
+
next logger.info "skipped low resolution #{source}"
|
|
74
96
|
end
|
|
75
97
|
# next if Gem::Platform.local.os == "darwin" # prevent concurrent posting
|
|
76
98
|
# puts "https://www.reddit.com/r/LargeImages/search.json?q=url%3A#{CGI.escape url}&restrict_sr=on"
|
|
@@ -85,8 +107,8 @@ loop do
|
|
|
85
107
|
" [#{all_direct_urls.size} images]" if all_direct_urls.size > 1
|
|
86
108
|
} #{
|
|
87
109
|
title.sub(/\s*\[?#{width}\s*[*x×]\s*#{height}\]?\s*/i, " ").
|
|
88
|
-
gsub(/\s+/, " ").strip.
|
|
89
|
-
|
|
110
|
+
sub("[OC]", " ").gsub(/\s+/, " ").strip.
|
|
111
|
+
gsub(/(?<=.{190 - subreddit.size}).+/, "...")
|
|
90
112
|
} /r/#{subreddit}".
|
|
91
113
|
gsub(/\s+\(\s+\)\s+/, " ")
|
|
92
114
|
logger.warn "new post #{source}: #{url} #{title.inspect}"
|
data/examples/mlgtv/Gemfile
CHANGED
data/examples/mlgtv/Gemfile.lock
CHANGED
|
@@ -1,3 +1,10 @@
|
|
|
1
|
+
GIT
|
|
2
|
+
remote: git@github.com:Nakilon/nethttputils.git
|
|
3
|
+
revision: 84ac32db6d44b68f5bd6e2f1ce8698fc5e2ad968
|
|
4
|
+
tag: v0.0.4.1
|
|
5
|
+
specs:
|
|
6
|
+
nethttputils (0.0.4.1)
|
|
7
|
+
|
|
1
8
|
GEM
|
|
2
9
|
remote: https://rubygems.org/
|
|
3
10
|
specs:
|
|
@@ -9,7 +16,8 @@ PLATFORMS
|
|
|
9
16
|
ruby
|
|
10
17
|
|
|
11
18
|
DEPENDENCIES
|
|
19
|
+
nethttputils!
|
|
12
20
|
reddit_bot (~> 1.3.0)
|
|
13
21
|
|
|
14
22
|
BUNDLED WITH
|
|
15
|
-
1.
|
|
23
|
+
1.16.0
|
data/examples/mlgtv/channels.txt
CHANGED
|
@@ -28,7 +28,7 @@ SlasheRAL
|
|
|
28
28
|
Apathy
|
|
29
29
|
ColtHavok
|
|
30
30
|
MikeSwarley
|
|
31
|
-
|
|
31
|
+
Parasite
|
|
32
32
|
TyreeLegal
|
|
33
33
|
Silly
|
|
34
34
|
Blfire
|
|
@@ -69,6 +69,7 @@ Swanny
|
|
|
69
69
|
MadCatEU
|
|
70
70
|
Rated_EU1
|
|
71
71
|
BsportJoshh
|
|
72
|
+
VortexSA
|
|
72
73
|
Sy_Vortex
|
|
73
74
|
TheMarkyB
|
|
74
75
|
Peatie95
|
|
@@ -84,6 +85,11 @@ skrapzq
|
|
|
84
85
|
Reedy
|
|
85
86
|
fighta71
|
|
86
87
|
Swiftazor
|
|
88
|
+
NakeeeZe
|
|
89
|
+
BeastnGOD
|
|
90
|
+
Dizmull
|
|
91
|
+
MitchBuZZO
|
|
92
|
+
YKEminence
|
|
87
93
|
BacabecNZ
|
|
88
94
|
Zeuss_Gaming
|
|
89
95
|
Hopeyy
|
|
@@ -95,8 +101,16 @@ Revan
|
|
|
95
101
|
BriceyHD
|
|
96
102
|
Benson
|
|
97
103
|
PHILWHI7
|
|
104
|
+
CallofDuty
|
|
105
|
+
MLGcod
|
|
98
106
|
MLG
|
|
99
107
|
mlgbravo
|
|
108
|
+
MLGcharlie
|
|
109
|
+
MLG_Delta
|
|
110
|
+
MLG_FR
|
|
111
|
+
MLG_German
|
|
112
|
+
MLG_ITL
|
|
113
|
+
MES
|
|
100
114
|
Multiplay_CoD
|
|
101
115
|
UMGEvents
|
|
102
116
|
GfinityTV
|
|
@@ -110,3 +124,4 @@ Vilesyder
|
|
|
110
124
|
benbance
|
|
111
125
|
JordonGeneral
|
|
112
126
|
dreeall
|
|
127
|
+
CESNLive
|
data/examples/mlgtv/main.rb
CHANGED
|
@@ -10,14 +10,26 @@ loop do
|
|
|
10
10
|
|
|
11
11
|
text = " Live Streams\n\n" + [].tap do |list|
|
|
12
12
|
|
|
13
|
-
throw :loop unless statuses = JSON.parse(
|
|
13
|
+
throw :loop unless statuses = JSON.parse( begin
|
|
14
14
|
NetHTTPUtils.request_data("http://streamapi.majorleaguegaming.com/service/streams/all")[/\{.+\}/m]
|
|
15
|
-
|
|
15
|
+
rescue NetHTTPUtils::Error => e
|
|
16
|
+
fail unless e.code == 408
|
|
17
|
+
puts 408
|
|
18
|
+
sleep 60
|
|
19
|
+
retry
|
|
20
|
+
end )["data"]["items"]
|
|
16
21
|
games = JSON.parse(
|
|
17
22
|
NetHTTPUtils.request_data("http://www.majorleaguegaming.com/api/games/all")[/\{.+\}/m]
|
|
18
23
|
)["data"]["items"]
|
|
19
24
|
begin
|
|
20
|
-
JSON.parse
|
|
25
|
+
JSON.parse begin
|
|
26
|
+
NetHTTPUtils.request_data("http://www.majorleaguegaming.com/api/channels/all?fields=name,url,tags,stream_name,game_id")
|
|
27
|
+
rescue NetHTTPUtils::Error => e
|
|
28
|
+
fail unless e.code == 404
|
|
29
|
+
puts 404
|
|
30
|
+
sleep 60
|
|
31
|
+
retry
|
|
32
|
+
end
|
|
21
33
|
rescue JSON::ParserError
|
|
22
34
|
puts "JSON::ParserError"
|
|
23
35
|
sleep 60
|
data/lib/reddit_bot.rb
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
STDOUT.sync = true
|
|
2
|
-
require "pp"
|
|
2
|
+
# require "pp"
|
|
3
3
|
|
|
4
4
|
require "net/http"
|
|
5
5
|
require "openssl"
|
|
@@ -166,7 +166,7 @@ module RedditBot
|
|
|
166
166
|
username: @username = @secrets[3],
|
|
167
167
|
password: @secrets[2],
|
|
168
168
|
}, {
|
|
169
|
-
"User-Agent" => "bot/#{@username}
|
|
169
|
+
"User-Agent" => "bot/#{@username}/#{RedditBot::VERSION} by /u/nakilon",
|
|
170
170
|
}, [@secrets[0], @secrets[1]]
|
|
171
171
|
unless @token_cached = response["access_token"]
|
|
172
172
|
fail "bot #{@username} isn't a 'developer' of app at https://www.reddit.com/prefs/apps/" if response == {"error"=>"invalid_grant"}
|
|
@@ -192,7 +192,7 @@ module RedditBot
|
|
|
192
192
|
nil until _ = catch(:"401") do
|
|
193
193
|
reddit_resp mtd, "https://oauth.reddit.com" + path, form, {
|
|
194
194
|
"Authorization" => "bearer #{token}",
|
|
195
|
-
"User-Agent" => "bot/#{@username}
|
|
195
|
+
"User-Agent" => "bot/#{@username}/#{RedditBot::VERSION} by /u/nakilon",
|
|
196
196
|
}
|
|
197
197
|
end
|
|
198
198
|
_
|
|
@@ -205,12 +205,12 @@ module RedditBot
|
|
|
205
205
|
case response.code
|
|
206
206
|
when "502", "503", "520", "500", "521", "504", "400", "522"
|
|
207
207
|
puts "LOL #{response.code} at #{Time.now}?"
|
|
208
|
-
|
|
208
|
+
p args
|
|
209
209
|
sleep 5
|
|
210
210
|
redo
|
|
211
211
|
when "409"
|
|
212
212
|
puts "Conflict (409)? at #{Time.now}?"
|
|
213
|
-
|
|
213
|
+
p args
|
|
214
214
|
sleep 5
|
|
215
215
|
redo
|
|
216
216
|
when "401"
|
data/lib/reddit_bot/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: reddit_bot
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.5.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Victor Maslov
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2018-01-23 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: json
|