newscast 1.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/CHANGELOG.md +4 -0
- data/Manifest.txt +8 -0
- data/README.md +249 -0
- data/Rakefile +32 -0
- data/lib/newscast.rb +358 -0
- data/lib/newscast/version.rb +22 -0
- data/test/helper.rb +10 -0
- data/test/test_queries.rb +94 -0
- metadata +128 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 8ecb966380bc32d55480f8d0415c071a03578381
|
4
|
+
data.tar.gz: 20c3742e4bd41f188a323a21f23605613610a477
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 814d7a15c7abd29cb545602f008aad5ddaf282548255e703f78ac9578e15ae64a982babbbf38492e4c846c0af0f33916489e72cadb3213d9282ad0ce6c831820
|
7
|
+
data.tar.gz: 6fec147ef395ad06ae6cbb671c06e48d4e0049544304b3454c48b63f4b99d25df06309a387feba326201d879e58da84b052106bdcfa22a182209a3ee18c2f6a8
|
data/CHANGELOG.md
ADDED
data/Manifest.txt
ADDED
data/README.md
ADDED
@@ -0,0 +1,249 @@
|
|
1
|
+
# newscast gem - newsreader for easy (re)use - build your own facebook newsfeed in 1-2-3 steps in 5 minutes
|
2
|
+
|
3
|
+
|
4
|
+
* home :: [github.com/feedreader/pluto](https://github.com/feedreader/pluto)
|
5
|
+
* bugs :: [github.com/feedreader/pluto/issues](https://github.com/feedreader/pluto/issues)
|
6
|
+
* gem :: [rubygems.org/gems/newscast](https://rubygems.org/gems/newscast)
|
7
|
+
* rdoc :: [rubydoc.info/gems/newscast](http://rubydoc.info/gems/newscast)
|
8
|
+
* forum :: [groups.google.com/group/wwwmake](http://groups.google.com/group/wwwmake)
|
9
|
+
|
10
|
+
|
11
|
+
|
12
|
+
## Usage
|
13
|
+
|
14
|
+
The `News` module offers easy to (re)use "porcelain" helpers / methods
|
15
|
+
that hide the "plumbing" for building your own newsreader or newsfeeds in minutes.
|
16
|
+
|
17
|
+
|
18
|
+
### Step 1 - Subscribe
|
19
|
+
|
20
|
+
Use the `News.subscribe( *feeds )` method to subscribe to news feeds / channels.
|
21
|
+
Example:
|
22
|
+
|
23
|
+
```ruby
|
24
|
+
News.subscribe(
|
25
|
+
'http://www.ruby-lang.org/en/feeds/news.rss', # Ruby Lang News
|
26
|
+
'http://www.jruby.org/atom.xml', # JRuby Lang News
|
27
|
+
'http://blog.rubygems.org/atom.xml', # RubyGems News
|
28
|
+
'http://bundler.io/blog/feed.xml', # Bundler News
|
29
|
+
'http://weblog.rubyonrails.org/feed/atom.xml', # Ruby on Rails News
|
30
|
+
'http://sinatrarb.com/feed.xml', # Sinatra News
|
31
|
+
'https://hanamirb.org/atom.xml', # Hanami News
|
32
|
+
'http://jekyllrb.com/feed.xml', # Jekyll News
|
33
|
+
'http://feeds.feedburner.com/jetbrains_rubymine?format=xml', # RubyMine IDE News
|
34
|
+
'https://blog.phusion.nl/rss/', # Phusion News
|
35
|
+
'https://rubyinstaller.org/feed.xml', # Ruby Installer for Windows News
|
36
|
+
'http://planetruby.github.io/calendar/feed.xml', # Ruby Conferences & Camps News
|
37
|
+
'https://rubytogether.org/news.xml', # Ruby Together News
|
38
|
+
'https://foundation.travis-ci.org/feed.xml', # Travis Foundation News
|
39
|
+
'https://railsgirlssummerofcode.org/blog.xml', # Rails Girls Summer of Code News
|
40
|
+
|
41
|
+
'http://blog.zenspider.com/atom.xml', # Ryan Davis @ Seattle › Washington › United States
|
42
|
+
'http://tenderlovemaking.com/atom.xml', # Aaron Patterson @ Seattle › Washington › United States
|
43
|
+
'http://blog.headius.com/feed.xml', # Charles Nutter @ Richfield › Minnesota › United States
|
44
|
+
'http://www.schneems.com/feed.xml', # Richard Schneeman @ Austin › Texas › United States
|
45
|
+
'https://eregon.me/blog/feed.xml', # Benoit Daloze @ Zurich › Switzerland
|
46
|
+
'http://samsaffron.com/posts.rss', # Sam Saffron @ Sydney › Australia
|
47
|
+
)
|
48
|
+
```
|
49
|
+
|
50
|
+
|
51
|
+
Note: If you call `News.subscribe` the method will setup a single-file SQLite database,
|
52
|
+
that is, `./news.db` and auto-migrate the schema, that is, database tables and so on.
|
53
|
+
|
54
|
+
|
55
|
+
### Step 2 - Update
|
56
|
+
|
57
|
+
Use the `News.update` method to fetch from the internets all news feeds / channels
|
58
|
+
and update the `./news.db` database. Example:
|
59
|
+
|
60
|
+
```ruby
|
61
|
+
News.update
|
62
|
+
```
|
63
|
+
|
64
|
+
|
65
|
+
|
66
|
+
### Step 3 - Query and Remix the News Items
|
67
|
+
|
68
|
+
Profit! Now you can query the `./news.db` database with any of the many
|
69
|
+
pre-made / ready-to-use query helpers. Example:
|
70
|
+
|
71
|
+
``` ruby
|
72
|
+
puts ":::::::::::::::::::::::::::::::::::::::::::::::::::"
|
73
|
+
puts ":: #{News.items.count} news items from #{News.channels.count} channels:"
|
74
|
+
puts
|
75
|
+
```
|
76
|
+
|
77
|
+
resulting in:
|
78
|
+
|
79
|
+
```
|
80
|
+
:::::::::::::::::::::::::::::::::::::::::::::::::::
|
81
|
+
:: 793 news items from 21 channels:
|
82
|
+
```
|
83
|
+
|
84
|
+
and
|
85
|
+
|
86
|
+
``` ruby
|
87
|
+
puts "100 Latest News Items"
|
88
|
+
News.latest.limit( 100 ).each do |item|
|
89
|
+
print "%4dd " % (Date.today.jd-item.updated.to_date.jd)
|
90
|
+
print " #{item.updated}"
|
91
|
+
print " - #{item.title}"
|
92
|
+
print " - #{URI(item.feed.feed_url).host}"
|
93
|
+
print "\n"
|
94
|
+
end
|
95
|
+
```
|
96
|
+
|
97
|
+
resulting in:
|
98
|
+
|
99
|
+
```
|
100
|
+
100 Latest News Items
|
101
|
+
0d 2020-02-04 00:00:00 - RubyNess @ Inverness, Scotland, United Kingdom - planetruby.github.io
|
102
|
+
0d 2020-02-04 00:00:00 - Ruby by the Bay (Ruby for Good, West Coast Edition) @ Marin Headlands (near San Francisco), California, United States - planetruby.github.io
|
103
|
+
8d 2020-01-27 14:44:38 - RubyMine 2020.1 EAP is Open! - jetbrains.com
|
104
|
+
11d 2020-01-24 00:00:00 - Crowdfunding for 2020 scholarships has commenced - railsgirlssummerofcode.org
|
105
|
+
15d 2020-01-20 00:00:00 - Alumni Interview with Keziah Naggita - railsgirlssummerofcode.org
|
106
|
+
16d 2020-01-19 00:00:00 - This week in Rails - Rack 2.1 released, disallowed deprecations, and more! - weblog.rubyonrails.org
|
107
|
+
19d 2020-01-16 00:00:00 - RubyConf Belarus (BY) @ Minsk, Belarus - planetruby.github.io
|
108
|
+
22d 2020-01-13 14:00:39 - Guide to String Encoding in Ruby - tenderlovemaking.com
|
109
|
+
23d 2020-01-12 23:00:00 - A Migration Path to Bundler 2+ - eregon.me
|
110
|
+
24d 2020-01-11 00:00:00 - This week in Rails - Deprecations, bugfixes and improvements! - weblog.rubyonrails.org
|
111
|
+
30d 2020-01-05 00:00:00 - This week in Rails - The 2019 edition - weblog.rubyonrails.org
|
112
|
+
30d 2020-01-05 00:00:00 - RubyInstaller 2.7.0-1 released - rubyinstaller.org
|
113
|
+
32d 2020-01-03 00:00:00 - Submit Your Open Source Projects - railsgirlssummerofcode.org
|
114
|
+
34d 2020-01-01 00:00:00 - Happy new year & Sinatra 2.0.8! - sinatrarb.com
|
115
|
+
39d 2019-12-27 00:00:00 - Ruby 2.7.0, Rails 6.0.2.1 and more - weblog.rubyonrails.org
|
116
|
+
41d 2019-12-25 00:00:00 - Ruby 2.7.0 Released - ruby-lang.org
|
117
|
+
...
|
118
|
+
```
|
119
|
+
|
120
|
+
More queries include:
|
121
|
+
|
122
|
+
``` ruby
|
123
|
+
News.today
|
124
|
+
|
125
|
+
News.week
|
126
|
+
News.week( 1 )
|
127
|
+
News.week( 1, 2019 )
|
128
|
+
|
129
|
+
News.month
|
130
|
+
News.month( 1 )
|
131
|
+
News.month( 1, 2019 )
|
132
|
+
|
133
|
+
News.year
|
134
|
+
News.year( 2019 )
|
135
|
+
|
136
|
+
News.this_week
|
137
|
+
News.this_month
|
138
|
+
News.this_year
|
139
|
+
|
140
|
+
News.q1
|
141
|
+
News.q2
|
142
|
+
News.q3
|
143
|
+
News.q4
|
144
|
+
```
|
145
|
+
|
146
|
+
and some more.
|
147
|
+
|
148
|
+
|
149
|
+
|
150
|
+
### All Together Now - Let's Build A Newsfeed in 20 Lines
|
151
|
+
|
152
|
+
```ruby
|
153
|
+
require 'newscast'
|
154
|
+
|
155
|
+
# step 1) subscribe to a list of news feeds / channels
|
156
|
+
|
157
|
+
News.subscribe(
|
158
|
+
'http://www.ruby-lang.org/en/feeds/news.rss', # Ruby Lang News
|
159
|
+
'http://www.jruby.org/atom.xml', # JRuby Lang News
|
160
|
+
'http://blog.rubygems.org/atom.xml', # RubyGems News
|
161
|
+
'http://bundler.io/blog/feed.xml', # Bundler News
|
162
|
+
'http://weblog.rubyonrails.org/feed/atom.xml', # Ruby on Rails News
|
163
|
+
'http://sinatrarb.com/feed.xml', # Sinatra News
|
164
|
+
'https://hanamirb.org/atom.xml', # Hanami News
|
165
|
+
'http://jekyllrb.com/feed.xml', # Jekyll News
|
166
|
+
'http://feeds.feedburner.com/jetbrains_rubymine?format=xml', # RubyMine IDE News
|
167
|
+
'https://blog.phusion.nl/rss/', # Phusion News
|
168
|
+
'https://rubyinstaller.org/feed.xml', # Ruby Installer for Windows News
|
169
|
+
'http://planetruby.github.io/calendar/feed.xml', # Ruby Conferences & Camps News
|
170
|
+
'https://rubytogether.org/news.xml', # Ruby Together News
|
171
|
+
'https://foundation.travis-ci.org/feed.xml', # Travis Foundation News
|
172
|
+
'https://railsgirlssummerofcode.org/blog.xml', # Rails Girls Summer of Code News
|
173
|
+
|
174
|
+
'http://blog.zenspider.com/atom.xml', # Ryan Davis @ Seattle › Washington › United States
|
175
|
+
'http://tenderlovemaking.com/atom.xml', # Aaron Patterson @ Seattle › Washington › United States
|
176
|
+
'http://blog.headius.com/feed.xml', # Charles Nutter @ Richfield › Minnesota › United States
|
177
|
+
'http://www.schneems.com/feed.xml', # Richard Schneeman @ Austin › Texas › United States
|
178
|
+
'https://eregon.me/blog/feed.xml', # Benoit Daloze @ Zurich › Switzerland
|
179
|
+
'http://samsaffron.com/posts.rss', # Sam Saffron @ Sydney › Australia
|
180
|
+
)
|
181
|
+
|
182
|
+
|
183
|
+
# step 2) fetch and update news feeds / channels
|
184
|
+
|
185
|
+
News.update
|
186
|
+
|
187
|
+
|
188
|
+
# step 3) mix up all news items in a new page
|
189
|
+
|
190
|
+
puts News.render( <<TXT )
|
191
|
+
<% News.latest.limit(100).each do |item| %>
|
192
|
+
<div class="item">
|
193
|
+
<h2><a href="<%= item.url %>"><%= item.title %></a></h2>
|
194
|
+
<div><%= item.content %></div>
|
195
|
+
</div>
|
196
|
+
<% end %>
|
197
|
+
TXT
|
198
|
+
```
|
199
|
+
|
200
|
+
resulting in:
|
201
|
+
|
202
|
+
``` html
|
203
|
+
<div class="item">
|
204
|
+
<h2><a href="http://planetruby.github.io/calendar/2020#rubyness">RubyNess @ Inverness,
|
205
|
+
Scotland, United Kingdom - Ruby Conferences 'n' Camps Update</a></h2>
|
206
|
+
<div><p>What's News? What's Upcoming in 2020?</p>
|
207
|
+
<p>
|
208
|
+
<b><a href="https://rubyness.org/" id="rubyness">RubyNess</a></b><br>
|
209
|
+
Jul/16+17 (2d) Thu+Fri @ Inverness, Scotland, United Kingdom •
|
210
|
+
<a href="https://twitter.com/rubynessconf" title="@rubynessconf">(Updates)</a>
|
211
|
+
</p>
|
212
|
+
<p>
|
213
|
+
See all <a href="http://planetruby.github.io/calendar/2020">Conferences 'n' Camps in 2020»</a>.
|
214
|
+
</p></div>
|
215
|
+
</div>
|
216
|
+
|
217
|
+
<div class="item">
|
218
|
+
<h2><a href="http://feedproxy.google.com/~r/jetbrains_rubymine/~3/DiNxpQHKHrk/">RubyMine
|
219
|
+
2020.1 EAP is Open!</a></h2>
|
220
|
+
<div><p>Hello everyone,</p>
|
221
|
+
<p>Today we are happy to announce the opening of the Early Access Program (EAP)
|
222
|
+
for RubyMine 2020.1! You can get EAP builds...
|
223
|
+
</p>
|
224
|
+
<p>Happy Developing!<br>
|
225
|
+
Your RubyMine team
|
226
|
+
</p></div>
|
227
|
+
</div>
|
228
|
+
...
|
229
|
+
```
|
230
|
+
|
231
|
+
|
232
|
+
|
233
|
+
## More
|
234
|
+
|
235
|
+
- [news.rb quick starter script](https://github.com/feedreader/news.rb) - build your own facebook newsfeed in 1-2-3 steps in 5 minutes
|
236
|
+
|
237
|
+
|
238
|
+
|
239
|
+
## License
|
240
|
+
|
241
|
+
![](https://publicdomainworks.github.io/buttons/zero88x31.png)
|
242
|
+
|
243
|
+
The `newscast` scripts are dedicated to the public domain.
|
244
|
+
Use it as you please with no restrictions whatsoever.
|
245
|
+
|
246
|
+
## Questions? Comments?
|
247
|
+
|
248
|
+
Send them along to the [wwwmake Forum/Mailing List](http://groups.google.com/group/wwwmake).
|
249
|
+
Thanks!
|
data/Rakefile
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
require 'hoe'
|
2
|
+
require './lib/newscast/version.rb'
|
3
|
+
|
4
|
+
Hoe.spec 'newscast' do
|
5
|
+
|
6
|
+
self.version = Newscast::VERSION
|
7
|
+
|
8
|
+
self.summary = "newscast - newsreader for easy (re)use - build your own facebook newsfeed in 1-2-3 steps in 5 minutes"
|
9
|
+
self.description = summary
|
10
|
+
|
11
|
+
self.urls = ['https://github.com/feedreader/pluto']
|
12
|
+
|
13
|
+
self.author = 'Gerald Bauer'
|
14
|
+
self.email = 'wwwmake@googlegroups.com'
|
15
|
+
|
16
|
+
# switch extension to .markdown for gihub formatting
|
17
|
+
self.readme_file = 'README.md'
|
18
|
+
self.history_file = 'CHANGELOG.md'
|
19
|
+
|
20
|
+
self.extra_deps = [
|
21
|
+
['pluto-models', '>= 1.6.1'],
|
22
|
+
['pluto-update', '>= 1.6.3'],
|
23
|
+
['sqlite3'], # note: for easy installation include sqlite database library
|
24
|
+
]
|
25
|
+
|
26
|
+
self.licenses = ['Public Domain']
|
27
|
+
|
28
|
+
self.spec_extras = {
|
29
|
+
required_ruby_version: '>= 2.2.2'
|
30
|
+
}
|
31
|
+
|
32
|
+
end
|
data/lib/newscast.rb
ADDED
@@ -0,0 +1,358 @@
|
|
1
|
+
|
2
|
+
require 'pluto/update'
|
3
|
+
|
4
|
+
|
5
|
+
## todo/fix: include stdlibs in pluto/update or pluto/models upstream!!!
|
6
|
+
require 'date'
|
7
|
+
require 'time'
|
8
|
+
require 'cgi'
|
9
|
+
require 'uri'
|
10
|
+
require 'digest'
|
11
|
+
require 'yaml' ## check - already included upstream?
|
12
|
+
require 'json' ## check - already included upstream?
|
13
|
+
|
14
|
+
require 'erb'
|
15
|
+
require 'ostruct'
|
16
|
+
|
17
|
+
|
18
|
+
|
19
|
+
class Date
|
20
|
+
## def quarter() 1+(self.month-1)/3; end
|
21
|
+
|
22
|
+
def quarter
|
23
|
+
case self.month
|
24
|
+
when 1,2,3 then 1
|
25
|
+
when 4,5,6 then 2
|
26
|
+
when 7,8,9 then 3
|
27
|
+
when 10,11,12 then 4
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
# -- for testing:
|
33
|
+
# (1..12).each do |num|
|
34
|
+
# puts "#{num} -> #{Date.new(2020,num,1).quarter}"
|
35
|
+
# end
|
36
|
+
# -- prints:
|
37
|
+
# 1 -> 1
|
38
|
+
# 2 -> 1
|
39
|
+
# 3 -> 1
|
40
|
+
# 4 -> 2
|
41
|
+
# 5 -> 2
|
42
|
+
# 6 -> 2
|
43
|
+
# 7 -> 3
|
44
|
+
# 8 -> 3
|
45
|
+
# 9 -> 3
|
46
|
+
# 10 -> 4
|
47
|
+
# 11 -> 4
|
48
|
+
# 12 -> 4
|
49
|
+
|
50
|
+
|
51
|
+
|
52
|
+
## our own code
|
53
|
+
require 'newscast/version'
|
54
|
+
|
55
|
+
|
56
|
+
|
57
|
+
module News
|
58
|
+
|
59
|
+
##
|
60
|
+
## todo/check: allow (add) site = nil for no site "filter" (all items / feeds) - why? why not?
|
61
|
+
def self.site=(value) @site = value; end
|
62
|
+
def self.site() @site ||= 'news'; end ## note: defaults to news
|
63
|
+
|
64
|
+
|
65
|
+
####
|
66
|
+
# helpers
|
67
|
+
|
68
|
+
def self.autogen_feed_key( feed_url )
|
69
|
+
## note:
|
70
|
+
## use a "fingerprint" hash digest as key
|
71
|
+
## do NOT include scheme (e.g. https or http)
|
72
|
+
## do NOT include port
|
73
|
+
## so you can change it without "breaking" the key - why? why not?
|
74
|
+
##
|
75
|
+
## e.g. u = URI( 'https://example.com:333/a/b?f=xml'
|
76
|
+
## u.host+u.request_uri
|
77
|
+
## #=> example.com/a/b?f=xml
|
78
|
+
uri = URI( feed_url )
|
79
|
+
## note: add host in "plain" text - making errors and the key more readable
|
80
|
+
## note: cut-off www. if leading e.g. www.ruby-lang.org => ruby-lang.org
|
81
|
+
host = uri.host.downcase.sub( /^www\./, '' )
|
82
|
+
# use a differt separator e.g _ or ~ and NOT $ - why? why not?
|
83
|
+
key = "#{host}$#{Digest::MD5.hexdigest( uri.request_uri )}"
|
84
|
+
key
|
85
|
+
end
|
86
|
+
|
87
|
+
def self.norm_feed_hash( old_h ) ## todo/check: rename to normalize/unify_feeds or resolve_feedlist or something?
|
88
|
+
## unify feed list entries
|
89
|
+
## case 1) if section name is some thing like [Andrew Kane]
|
90
|
+
## and NO title/name key than assume it's the title/name
|
91
|
+
## and auto-generated key/id
|
92
|
+
|
93
|
+
## move all "global" settings to [planet] section - why? why not?
|
94
|
+
## e.g.
|
95
|
+
## title = Planet Open Data News
|
96
|
+
## becomes
|
97
|
+
## [planet]
|
98
|
+
## title = Planet Open Data News
|
99
|
+
|
100
|
+
h = {}
|
101
|
+
old_h.each do |k,v|
|
102
|
+
if v.is_a?( String )
|
103
|
+
h[ k ] = v ## pass along as-is (assume "top-level" / global setting)
|
104
|
+
elsif v.is_a?( Hash )
|
105
|
+
## puts "#{k}:"
|
106
|
+
## pp v
|
107
|
+
## todo/fix: use "proper" ident e.g. allow 0-9 and .-_ too? why? why not?
|
108
|
+
if k =~ /^[a-z_][a-z0-9$_.]*$/ ## all lower case; assume id - add 0-9 and .-_ - why? why not?
|
109
|
+
h[ k ] = v
|
110
|
+
else
|
111
|
+
## puts "bingo! section name shortcut - #{k}"
|
112
|
+
if k.start_with?( 'http' )
|
113
|
+
if v.has_key?( 'feed' ) then raise ArgumentError.new( "duplicate >feed< hash table entry in section >#{k}<; cannot autogen key" ); end
|
114
|
+
|
115
|
+
new_k = autogen_feed_key( k )
|
116
|
+
# note: use merge - why? why not? to NOT overwrite existing entry - why? why not?
|
117
|
+
h[ new_k ] = { 'feed' => k }.merge( v )
|
118
|
+
else
|
119
|
+
## transform key to title and auto-generate id (new key)
|
120
|
+
if v.has_key?( 'title' ) || v.has_key?( 'name' ) then raise ArgumentError.new( "duplicate >name< or >title< hash table entry in section >#{k}<; cannot autogen key" ); end
|
121
|
+
if v.has_key?( 'feed' ) == false then raise ArgumentError.new( "expected / required >feed< hash table entry missing for section >#{k}<"); end
|
122
|
+
|
123
|
+
new_k = autogen_feed_key( v['feed'] )
|
124
|
+
# note: use merge - why? why not? to NOT overwrite existing entry - why? why not?
|
125
|
+
h[ new_k ] = { 'title' => k }.merge( v )
|
126
|
+
end
|
127
|
+
end
|
128
|
+
else
|
129
|
+
raise ArgumentError.new( "expected String or Hash for value (in hash table) but got >#{v.class.name}< for key >#{k}<" )
|
130
|
+
end
|
131
|
+
end
|
132
|
+
|
133
|
+
## todo/check - auto-add required (?) missing title if missing - why? why not?
|
134
|
+
h['title'] = 'Untitled' if h.has_key?( 'title' ) == false
|
135
|
+
|
136
|
+
h
|
137
|
+
end
|
138
|
+
|
139
|
+
|
140
|
+
def self.subscribe( *feeds )
|
141
|
+
|
142
|
+
site_hash = if feeds.size == 1 && feeds[0].is_a?( Hash )
|
143
|
+
## puts "bingo! it's a hash"
|
144
|
+
norm_feed_hash( feeds[0] )
|
145
|
+
elsif feeds.size == 1 && feeds[0].is_a?( String ) && feeds[0] =~ /\n/
|
146
|
+
## string includes newline (e.g. more than a single line?)
|
147
|
+
## if yes, parse and assume ini (config) format
|
148
|
+
norm_feed_hash( INI.load( feeds[0] ))
|
149
|
+
else ## assume list / array of strings (feed_urls)
|
150
|
+
## puts "bingo! it's a string list"
|
151
|
+
## auto-build a (simple) site hash
|
152
|
+
feeds.reduce( { ## note: keys are strings (NOT symbols) for now
|
153
|
+
'title' => 'Untitled'
|
154
|
+
## todo/check: remove title? required? check in model update if missing?
|
155
|
+
} ) do |h, feed|
|
156
|
+
key = autogen_feed_key( feed )
|
157
|
+
|
158
|
+
h[ key ] = { 'feed' => feed }
|
159
|
+
h
|
160
|
+
end
|
161
|
+
end
|
162
|
+
|
163
|
+
connection ## make sure we have a database connection (and setup) up-and-running
|
164
|
+
## note: always use "multi-site" setup; defaults to 'news' site key
|
165
|
+
Pluto::Model::Site.deep_create_or_update_from_hash!( site, site_hash )
|
166
|
+
end
|
167
|
+
|
168
|
+
def self.update
|
169
|
+
connection ## make sure we have a database connection (and setup) up-and-running
|
170
|
+
Pluto.update_feeds
|
171
|
+
end
|
172
|
+
def self.refresh() update; end ## convenience alias for update
|
173
|
+
|
174
|
+
|
175
|
+
|
176
|
+
def self.feeds
|
177
|
+
connection
|
178
|
+
## note: always use "multi-site" setup; defaults to 'news' site key
|
179
|
+
## note: add "default" scope - orders (sorts) by latest / time
|
180
|
+
rec = Pluto::Model::Site.where(key: site).first
|
181
|
+
if rec.nil?
|
182
|
+
Pluto::Model::Feed.none ## use null (relation) pattern to avoid crash on nil - why? why not?
|
183
|
+
else
|
184
|
+
rec.feeds.order(
|
185
|
+
Arel.sql( "coalesce(feeds.updated,feeds.published,'1970-01-01') desc" )
|
186
|
+
)
|
187
|
+
end
|
188
|
+
end
|
189
|
+
def self.channels() feeds; end ## convenience alias for feeds
|
190
|
+
|
191
|
+
|
192
|
+
def self.items
|
193
|
+
connection ## make sure we have a database connection (and setup) up-and-running
|
194
|
+
## note: always use "multi-site" setup; defaults to 'news' site key
|
195
|
+
## note: add "default" scope - orders (sorts) by latest / time
|
196
|
+
rec = Pluto::Model::Site.where(key: site).first
|
197
|
+
if rec.nil?
|
198
|
+
Pluto::Model::Item.none ## use null (relation) pattern to avoid crash on nil - why? why not?
|
199
|
+
else
|
200
|
+
rec.items.order(
|
201
|
+
Arel.sql( "coalesce(items.updated,items.published,'1970-01-01') desc" )
|
202
|
+
)
|
203
|
+
end
|
204
|
+
end
|
205
|
+
def self.latest() items; end ## note: "default" scope orders (sorts) by latest / time
|
206
|
+
|
207
|
+
|
208
|
+
def self.today
|
209
|
+
## todo: order by time!! - possible - why? why not?
|
210
|
+
## note: use date() to cut-off hours etc. if present?
|
211
|
+
|
212
|
+
q = Date.today.strftime('%Y-%m-%d') # e.g. 2020-02-20
|
213
|
+
items.
|
214
|
+
where(
|
215
|
+
Arel.sql( "date(coalesce(items.updated,items.published)) = '#{q}'" )
|
216
|
+
)
|
217
|
+
end
|
218
|
+
|
219
|
+
|
220
|
+
def self.between( start_date, end_date ) ## helper for week, q1, q2, etc.
|
221
|
+
q_start = start_date.strftime('%Y-%m-%d')
|
222
|
+
q_end = end_date.strftime('%Y-%m-%d')
|
223
|
+
|
224
|
+
items.
|
225
|
+
where(
|
226
|
+
Arel.sql( "date(coalesce(items.updated,items.published,'1970-01-01')) BETWEEN '#{q_start}' AND '#{q_end}'" )
|
227
|
+
)
|
228
|
+
end
|
229
|
+
|
230
|
+
def self.week( week=Date.today.cweek, year=Date.today.year )
|
231
|
+
## note: SQLite only supports "classic" week of year (not ISO "commercial week" starting on monday - and not on sunday)
|
232
|
+
## %W - week of year: 00-53
|
233
|
+
## thus, needs to calculate start and end date!!!
|
234
|
+
|
235
|
+
start_date = Date.commercial(year, week, 1)
|
236
|
+
end_date = Date.commercial(year, week, 7)
|
237
|
+
|
238
|
+
between( start_date, end_date )
|
239
|
+
end
|
240
|
+
|
241
|
+
def self.month( month=Date.today.month, year=Date.today.year )
|
242
|
+
q = "%4d-%02d" % [year,month] # e.g. 2020-01 etc.
|
243
|
+
items.
|
244
|
+
where(
|
245
|
+
Arel.sql( "strftime('%Y-%m', coalesce(items.updated,items.published,'1970-01-01')) = '#{q}'" )
|
246
|
+
)
|
247
|
+
end
|
248
|
+
|
249
|
+
def self.quarter( quarter=Date.today.quarter, year=Date.today.year )
|
250
|
+
case quarter
|
251
|
+
when 1 then between( Date.new( year, 1, 1), Date.new( year, 3, 31) );
|
252
|
+
when 2 then between( Date.new( year, 4, 1), Date.new( year, 6, 30) );
|
253
|
+
when 3 then between( Date.new( year, 7, 1), Date.new( year, 9, 30) );
|
254
|
+
when 4 then between( Date.new( year, 10, 1), Date.new( year,12, 31) );
|
255
|
+
else raise ArgumentError
|
256
|
+
end
|
257
|
+
end
|
258
|
+
|
259
|
+
def self.quarter1( year=Date.today.year ) quarter(1, year); end
|
260
|
+
def self.quarter2( year=Date.today.year ) quarter(2, year); end
|
261
|
+
def self.quarter3( year=Date.today.year ) quarter(3, year); end
|
262
|
+
def self.quarter4( year=Date.today.year ) quarter(4, year); end
|
263
|
+
|
264
|
+
def self.q( quarter=Date.today.quarter, year=Date.today.year ) quarter( quarter, year ); end
|
265
|
+
def self.q1( year=Date.today.year ) quarter1( year ); end
|
266
|
+
def self.q2( year=Date.today.year ) quarter2( year ); end
|
267
|
+
def self.q3( year=Date.today.year ) quarter3( year ); end
|
268
|
+
def self.q4( year=Date.today.year ) quarter4( year ); end
|
269
|
+
|
270
|
+
|
271
|
+
def self.year( year=Date.today.year )
|
272
|
+
q = year
|
273
|
+
items.
|
274
|
+
where(
|
275
|
+
Arel.sql( "strftime('%Y', coalesce(items.updated,items.published,'1970-01-01')) = '#{q}'" )
|
276
|
+
)
|
277
|
+
end
|
278
|
+
|
279
|
+
|
280
|
+
|
281
|
+
|
282
|
+
def self.this_week() week; end ## convenience alias - keep - why? why not?
|
283
|
+
def self.this_month() month; end
|
284
|
+
def self.this_quarter() quarter; end
|
285
|
+
def self.this_year() year; end
|
286
|
+
|
287
|
+
|
288
|
+
|
289
|
+
class Template
|
290
|
+
class Context < OpenStruct
|
291
|
+
## use a different name - why? why not?
|
292
|
+
## e.g. to_h, to_hash, vars, locals, assigns, etc.
|
293
|
+
def get_binding() binding; end
|
294
|
+
|
295
|
+
## add builtin helpers / shortcuts
|
296
|
+
def h( text ) CGI.escapeHTML( text ); end
|
297
|
+
end # class Template::Context
|
298
|
+
|
299
|
+
|
300
|
+
def initialize( text )
|
301
|
+
@template = ERB.new( text )
|
302
|
+
end
|
303
|
+
|
304
|
+
## todo: use locals / assigns or something instead of **kwargs - why? why not?
|
305
|
+
## allow/support (extra) locals / assigns - why? why not?
|
306
|
+
def render( **kwargs )
|
307
|
+
## note: Ruby >= 2.5 has ERB#result_with_hash - use later - why? why not?
|
308
|
+
@template.result( Context.new( **kwargs ).get_binding )
|
309
|
+
end
|
310
|
+
end # class Template
|
311
|
+
|
312
|
+
def self.render( text, **kwargs )
|
313
|
+
template = Template.new( text )
|
314
|
+
template.render( **kwargs )
|
315
|
+
end
|
316
|
+
|
317
|
+
|
318
|
+
|
319
|
+
|
320
|
+
class Configuration
|
321
|
+
def database=(value) @database = value; end
|
322
|
+
def database() @database || { adapter: 'sqlite3', database: './news.db' }; end
|
323
|
+
end # class Configuration
|
324
|
+
|
325
|
+
def self.configure
|
326
|
+
yield( config )
|
327
|
+
end
|
328
|
+
|
329
|
+
def self.config
|
330
|
+
@config ||= Configuration.new
|
331
|
+
end
|
332
|
+
|
333
|
+
|
334
|
+
class Connection
|
335
|
+
def initialize # convenience shortcut w/ automigrate
|
336
|
+
config = if News.config.database.is_a?(Hash)
|
337
|
+
News.config.database
|
338
|
+
else ## assume a string (e.g. :memory:, or <path> AND sqlite3 adapter)
|
339
|
+
{ adapter: 'sqlite3', database: News.config.database }
|
340
|
+
end
|
341
|
+
|
342
|
+
Pluto.connect( config )
|
343
|
+
Pluto.auto_migrate!
|
344
|
+
end
|
345
|
+
end # class Connection
|
346
|
+
|
347
|
+
def self.connection ## use for "auto-magic" connection w/ automigrate
|
348
|
+
## todo/fix: check - "just simply" return ActiveRecord connection - possible - why? why not?
|
349
|
+
## do NOT use our own Connection class
|
350
|
+
@connection ||= Connection.new
|
351
|
+
end
|
352
|
+
end # module News
|
353
|
+
|
354
|
+
|
355
|
+
|
356
|
+
|
357
|
+
# say hello
|
358
|
+
puts Newscast.banner if $DEBUG || (defined?($RUBYLIBS_DEBUG) && $RUBYLIBS_DEBUG)
|
@@ -0,0 +1,22 @@
|
|
1
|
+
|
2
|
+
module Newscast
|
3
|
+
|
4
|
+
MAJOR = 1
|
5
|
+
MINOR = 1
|
6
|
+
PATCH = 1
|
7
|
+
VERSION = [MAJOR,MINOR,PATCH].join('.')
|
8
|
+
|
9
|
+
def self.version
|
10
|
+
VERSION
|
11
|
+
end
|
12
|
+
|
13
|
+
def self.banner
|
14
|
+
### todo: add RUBY_PATCHLEVEL or RUBY_PATCH_LEVEL e.g. -p124
|
15
|
+
"newscast/#{VERSION} on Ruby #{RUBY_VERSION} (#{RUBY_RELEASE_DATE}) [#{RUBY_PLATFORM}]"
|
16
|
+
end
|
17
|
+
|
18
|
+
def self.root
|
19
|
+
"#{File.expand_path( File.dirname(File.dirname(File.dirname(__FILE__))) )}"
|
20
|
+
end
|
21
|
+
|
22
|
+
end # module Newscast
|
data/test/helper.rb
ADDED
@@ -0,0 +1,94 @@
|
|
1
|
+
###
|
2
|
+
# to run use
|
3
|
+
# ruby -I ./lib -I ./test test/test_queries.rb
|
4
|
+
|
5
|
+
|
6
|
+
require 'helper'
|
7
|
+
|
8
|
+
class TestQueries < MiniTest::Test
|
9
|
+
|
10
|
+
def test_queries
|
11
|
+
puts News.channels.to_sql
|
12
|
+
puts News.feeds.to_sql
|
13
|
+
|
14
|
+
puts News.items.to_sql
|
15
|
+
|
16
|
+
|
17
|
+
puts News.latest.limit( 2 ).to_sql
|
18
|
+
puts News.today.to_sql
|
19
|
+
|
20
|
+
puts News.week.to_sql
|
21
|
+
puts News.week( 1 ).to_sql
|
22
|
+
puts News.week( 1, 2019 ).to_sql
|
23
|
+
|
24
|
+
puts News.month.to_sql
|
25
|
+
puts News.month( 1 ).to_sql
|
26
|
+
puts News.month( 1, 2019 ).to_sql
|
27
|
+
|
28
|
+
puts News.year.to_sql
|
29
|
+
puts News.year( 2019 ).to_sql
|
30
|
+
|
31
|
+
puts News.this_week.to_sql
|
32
|
+
puts News.this_month.to_sql
|
33
|
+
puts News.this_quarter.to_sql
|
34
|
+
puts News.this_year.to_sql
|
35
|
+
|
36
|
+
|
37
|
+
puts News.quarter.to_sql
|
38
|
+
puts News.q.to_sql
|
39
|
+
|
40
|
+
puts News.quarter( 1 ).to_sql
|
41
|
+
puts News.quarter( 2 ).to_sql
|
42
|
+
puts News.quarter( 3 ).to_sql
|
43
|
+
puts News.quarter( 4 ).to_sql
|
44
|
+
puts News.quarter1.to_sql
|
45
|
+
puts News.quarter2.to_sql
|
46
|
+
puts News.quarter3.to_sql
|
47
|
+
puts News.quarter4.to_sql
|
48
|
+
|
49
|
+
puts News.q( 1 ).to_sql
|
50
|
+
puts News.q( 2 ).to_sql
|
51
|
+
puts News.q( 3 ).to_sql
|
52
|
+
puts News.q( 4 ).to_sql
|
53
|
+
puts News.q1.to_sql
|
54
|
+
puts News.q2.to_sql
|
55
|
+
puts News.q3.to_sql
|
56
|
+
puts News.q4.to_sql
|
57
|
+
puts News.q( 1, 2019 ).to_sql
|
58
|
+
puts News.q( 2, 2019 ).to_sql
|
59
|
+
puts News.q( 3, 2019 ).to_sql
|
60
|
+
puts News.q( 4, 2019 ).to_sql
|
61
|
+
puts News.q1( 2019 ).to_sql
|
62
|
+
puts News.q2( 2019 ).to_sql
|
63
|
+
puts News.q3( 2019 ).to_sql
|
64
|
+
puts News.q4( 2019 ).to_sql
|
65
|
+
|
66
|
+
|
67
|
+
###### run queries
|
68
|
+
pp News.latest.limit( 2 ).to_a
|
69
|
+
pp News.today.to_a
|
70
|
+
|
71
|
+
pp News.week.to_a
|
72
|
+
pp News.week( 1 ).to_a
|
73
|
+
pp News.week( 1, 2019 ).to_a
|
74
|
+
|
75
|
+
pp News.month.to_a
|
76
|
+
pp News.month( 1 ).to_a
|
77
|
+
pp News.month( 1, 2019 ).to_a
|
78
|
+
|
79
|
+
pp News.year.to_a
|
80
|
+
pp News.year( 2019 ).to_a
|
81
|
+
|
82
|
+
pp News.this_week.to_a
|
83
|
+
pp News.this_month.to_a
|
84
|
+
pp News.this_year.to_a
|
85
|
+
|
86
|
+
pp News.q1.to_a
|
87
|
+
pp News.q2.to_a
|
88
|
+
pp News.q3.to_a
|
89
|
+
pp News.q4.to_a
|
90
|
+
|
91
|
+
assert true
|
92
|
+
end
|
93
|
+
|
94
|
+
end # class TestQueries
|
metadata
ADDED
@@ -0,0 +1,128 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: newscast
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.1.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Gerald Bauer
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2020-03-02 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: pluto-models
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 1.6.1
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 1.6.1
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: pluto-update
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 1.6.3
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 1.6.3
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: sqlite3
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: rdoc
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '4.0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '4.0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: hoe
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '3.16'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '3.16'
|
83
|
+
description: newscast - newsreader for easy (re)use - build your own facebook newsfeed
|
84
|
+
in 1-2-3 steps in 5 minutes
|
85
|
+
email: wwwmake@googlegroups.com
|
86
|
+
executables: []
|
87
|
+
extensions: []
|
88
|
+
extra_rdoc_files:
|
89
|
+
- CHANGELOG.md
|
90
|
+
- Manifest.txt
|
91
|
+
- README.md
|
92
|
+
files:
|
93
|
+
- CHANGELOG.md
|
94
|
+
- Manifest.txt
|
95
|
+
- README.md
|
96
|
+
- Rakefile
|
97
|
+
- lib/newscast.rb
|
98
|
+
- lib/newscast/version.rb
|
99
|
+
- test/helper.rb
|
100
|
+
- test/test_queries.rb
|
101
|
+
homepage: https://github.com/feedreader/pluto
|
102
|
+
licenses:
|
103
|
+
- Public Domain
|
104
|
+
metadata: {}
|
105
|
+
post_install_message:
|
106
|
+
rdoc_options:
|
107
|
+
- "--main"
|
108
|
+
- README.md
|
109
|
+
require_paths:
|
110
|
+
- lib
|
111
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
112
|
+
requirements:
|
113
|
+
- - ">="
|
114
|
+
- !ruby/object:Gem::Version
|
115
|
+
version: 2.2.2
|
116
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
117
|
+
requirements:
|
118
|
+
- - ">="
|
119
|
+
- !ruby/object:Gem::Version
|
120
|
+
version: '0'
|
121
|
+
requirements: []
|
122
|
+
rubyforge_project:
|
123
|
+
rubygems_version: 2.5.2
|
124
|
+
signing_key:
|
125
|
+
specification_version: 4
|
126
|
+
summary: newscast - newsreader for easy (re)use - build your own facebook newsfeed
|
127
|
+
in 1-2-3 steps in 5 minutes
|
128
|
+
test_files: []
|