hacker_term 0.0.4 → 0.0.5

Sign up to get free protection for your applications and to get access to all the features.
Files changed (13) hide show
  1. data/.travis.yml +8 -0
  2. data/Gemfile.lock +11 -1
  3. data/README.md +8 -1
  4. data/data/comments.json +706 -0
  5. data/data/lorem.txt +1 -0
  6. data/hacker_term.gemspec +1 -0
  7. data/lib/hacker_term/comments_data.rb +28 -0
  8. data/lib/hacker_term/page_data.rb +3 -1
  9. data/lib/hacker_term/ui.rb +50 -39
  10. data/lib/hacker_term/version.rb +1 -1
  11. data/spec/comments_data_spec.rb +58 -0
  12. data/spec/page_data_spec.rb +8 -4
  13. metadata +19 -2
data/.travis.yml ADDED
@@ -0,0 +1,8 @@
1
+ language: ruby
2
+ rvm:
3
+ - "1.9.2"
4
+ - "1.9.3"
5
+ - jruby-19mode # JRuby in 1.9 mode
6
+ - rbx-19mode
7
+ # uncomment this line if your project needs to run something other than `rake`:
8
+ script: bundle exec rspec spec
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- hacker_term (0.0.1)
4
+ hacker_term (0.0.4)
5
5
  clipboard
6
6
  launchy
7
7
  rest-client
@@ -11,14 +11,24 @@ GEM
11
11
  specs:
12
12
  addressable (2.3.2)
13
13
  clipboard (1.0.1)
14
+ diff-lcs (1.1.3)
14
15
  launchy (2.1.2)
15
16
  addressable (~> 2.3)
16
17
  mime-types (1.19)
17
18
  rest-client (1.6.7)
18
19
  mime-types (>= 1.16)
20
+ rspec (2.12.0)
21
+ rspec-core (~> 2.12.0)
22
+ rspec-expectations (~> 2.12.0)
23
+ rspec-mocks (~> 2.12.0)
24
+ rspec-core (2.12.0)
25
+ rspec-expectations (2.12.0)
26
+ diff-lcs (~> 1.1.3)
27
+ rspec-mocks (2.12.0)
19
28
 
20
29
  PLATFORMS
21
30
  ruby
22
31
 
23
32
  DEPENDENCIES
24
33
  hacker_term!
34
+ rspec
data/README.md CHANGED
@@ -4,10 +4,15 @@ Hacker News on the Terminal.
4
4
 
5
5
  ![Screenshot](http://flydillonfly.files.wordpress.com/2013/01/hacker_term_png1.png)
6
6
 
7
+ Requirements
8
+ ------------
9
+ * Ruby 1.9.3
10
+
7
11
  Installation
8
12
  ------------
9
13
  * Install with `gem install hacker_term`
10
14
  * Run using `hacker_term`
15
+ * Tests included; I run them using `rspec -fd` in the project directory
11
16
 
12
17
  Overview
13
18
  --------
@@ -16,7 +21,7 @@ See the front page of HN, use the arrow keys to browse and open particular items
16
21
  * Uses the Ruby `curses` library to create a terminal UI.
17
22
  * Captures keyboard events to allow browsing of the HN front page from the terminal.
18
23
  * Tested (and looks colourful) on OSX Mountain Lion, but some functionality may be lost on other flavours of Linux.
19
- * Ditto the above point if using something other than the basic OSX terminal application.
24
+ * Not tested and verified on anything other than the basic OSX terminal application - but may well work anyway.
20
25
  * Uses the HN feed available at http://hndroidapi.appspot.com - without that resource this project would not exist.
21
26
  * Sorting options included.
22
27
  * Some stats included.
@@ -27,3 +32,5 @@ Background
27
32
  This project was created to allow me to scratch a particular programming itch after reading about https://github.com/etsy/mctop. It brought me back to my days in college coding in C where everything was a terminal program!
28
33
 
29
34
  Please enjoy/contribute/ignore as you see fit.
35
+
36
+ [![Build Status](https://travis-ci.org/ciaranarcher/hacker_term.png)](https://travis-ci.org/ciaranarcher/hacker_term)
data/data/comments.json ADDED
@@ -0,0 +1,706 @@
1
+ {
2
+ "items":[
3
+ {
4
+ "username":"edw519",
5
+ "comment":"From Line 42536 of the 2008 CSV file:__BR__20090201_PIT@ARI,2,30,18,ARI,PIT,1,1,1,(:18) (Shotgun) K.Warner pass short middle intended for A.Boldin INTERCEPTED by J.Harrison at PIT 0. J.Harrison for 100 yards TOUCHDOWN. Super Bowl Record longest interception return yards. Penalty on ARZ-E.Brown Face Mask (15 Yards) declined. The Replay Assistant challenged the runner broke the plane ruling and the play was Upheld.,7,10,2008__BR__They forgot: for(i=0;i<92;i++){yell('edw519','GO!')}__BR__Seriously, I had plans for the next 4 days, but I just scrapped them. Funny how jazzed I get when it's data that I can really relate to...__BR__I've already structured my data warehouse and started the loads. (I'll probably need a whole day just to parse the text in Field 10.) Then I'm going to build a Business Intelligence system on top of it. I will finally have the proof I need that I, not the offensive coordinator, should be texting each play to Coach Tomlin.__BR__See you guys on Monday.__BR__EDIT: OK, I'm back, but not for long. I'm having way too much fun with this...__BR__fleaflicker: Cool website & domain name. Thanks for the tips. I expect shortcomings in the data, but it looks like it's in a lot better shape than the usual free form enterprise quality\/vendor\/customer comments I usually have to parse. We'll see...__BR__MattSayer & sjs382: I don't plan to do any analysis. I prefer to build an app that enables others to do their own analyses, answering questions that nobody else is asking. Like \\\"Which Steeler makes the most tackles on opposing runs or more than 5 yards when it's 3rd down and longer than 8 yards to go, the temperature is below 38, and edw519 is twirling his Terrible Towel clockwise?\\\"__BR__jerf: Nice thought. I've spent years trying to earn enough money to buy the Pittsburgh Steelers just to fire the slackers and fumblers and win the Super Bowl every year. Maybe I should just take an easier route and solve that problem like any self-respecting hacker should: with data & logic. No Steeler game this weekend; I may have found my destiny <\/sarcasm>",
6
+ "id":"5003343",
7
+ "grayedOutPercent":0,
8
+ "reply_id":"5003343&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
9
+ "time":"3 hours ago",
10
+ "children":[
11
+ {
12
+ "username":"jtchang",
13
+ "comment":"And here is that play :)__BR__http:\/\/www.youtube.com\/watch?v=oM1iXHY8s9o",
14
+ "id":"5004400",
15
+ "grayedOutPercent":0,
16
+ "reply_id":"5004400&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
17
+ "time":"8 minutes ago",
18
+ "children":[
19
+
20
+ ]
21
+ },
22
+ {
23
+ "username":"fleaflicker",
24
+ "comment":"You'll find that the text descriptions aren't consistently formatted. It's tough to extract structured data from all play descriptions.__BR__For example, first initial plus last name does does not uniquely identify a player. You'll need accurate roster data first, and even then there are clashes.__BR__We store play data by its structured components (players involved, play type, player roles, etc) and then derive the text description. This allows us to reassemble pbp data from different pro games to show a \\\"feed\\\" for your fantasy team.__BR__Baseball has a smaller set of play outcomes\/transitions so its easier to model this way. As your example from the Steelers Super Bowl shows, football plays can be very complex.",
25
+ "id":"5003544",
26
+ "grayedOutPercent":0,
27
+ "reply_id":"5003544&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
28
+ "time":"2 hours ago",
29
+ "children":[
30
+ {
31
+ "username":"fennecfoxen",
32
+ "comment":"\\\"It's tough to extract structured data from all play descriptions.\\\"__BR__Which means you can treat it a bit like a text mining program. NASA had a text mining contest in 2007 as part of the SIAM conference on data mining which was really similar - instead of football plays it was textual descriptions of aeronautics incident reports and their classification. There were several papers that came out of that (I was with a group that did one of them, using an approximate nonnegative matrix classification approach - got beat out by some ensemble approaches).__BR__Anyway - if you'd like to do something with unstructured football play descriptions, text mining might be able to empower you to some extent without going through a full manual analysis, and those papers could be a good starting point. I think some of them ended up in a volume titled _Survey of Text Mining II_.",
33
+ "id":"5003789",
34
+ "grayedOutPercent":0,
35
+ "reply_id":"5003789&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
36
+ "time":"1 hour ago",
37
+ "children":[
38
+
39
+ ]
40
+ },
41
+ {
42
+ "username":"JL2010",
43
+ "comment":"I had asked a question on stack-overflow a while ago asking for some guidance on parsing this exact kind of stuff. http:\/\/stackoverflow.com\/questions\/8198923\/natural-language-...",
44
+ "id":"5003894",
45
+ "grayedOutPercent":0,
46
+ "reply_id":"5003894&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
47
+ "time":"1 hour ago",
48
+ "children":[
49
+
50
+ ]
51
+ }
52
+ ]
53
+ },
54
+ {
55
+ "username":"jerf",
56
+ "comment":"I eagerly wait your next eBook, \\\"How I Pivoted Into The NFL\\\".",
57
+ "id":"5003384",
58
+ "grayedOutPercent":0,
59
+ "reply_id":"5003384&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
60
+ "time":"3 hours ago",
61
+ "children":[
62
+ {
63
+ "username":"iansinke",
64
+ "comment":"\\\"How I Pivot-tabled Into The NFL\\\"",
65
+ "id":"5003549",
66
+ "grayedOutPercent":0,
67
+ "reply_id":"5003549&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
68
+ "time":"2 hours ago",
69
+ "children":[
70
+
71
+ ]
72
+ }
73
+ ]
74
+ },
75
+ {
76
+ "username":"sjs382",
77
+ "comment":"Funny, I was looking for just that line to see how the more complicated plays were described.__BR__First thing that I noticed was that the Game ID matched CBS's website's URLs: 20090201_PIT@ARI == http:\/\/www.cbssports.com\/nfl\/gametracker\/playbyplay\/NFL_2009...__BR__Also, I went to compare this PBP to both ESPN and CBS and found that both have the exact PBP data, which is interesting because it seems that they got this data directly from the NFL (or from the same source, at least). I guess this makes sense, but it's something I hadn't considered.__BR__For reference, ESPN's PBP for the same game: http:\/\/espn.go.com\/nfl\/playbyplay?gameId=290201022&perio...",
78
+ "id":"5003412",
79
+ "grayedOutPercent":0,
80
+ "reply_id":"5003412&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
81
+ "time":"2 hours ago",
82
+ "children":[
83
+ {
84
+ "username":"sswezey",
85
+ "comment":"I think they all get their data for Elias Sports Bureau",
86
+ "id":"5003529",
87
+ "grayedOutPercent":0,
88
+ "reply_id":"5003529&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
89
+ "time":"2 hours ago",
90
+ "children":[
91
+
92
+ ]
93
+ },
94
+ {
95
+ "username":"anon987",
96
+ "comment":"It looks like the same format they use for NFL Game Rewind too. I would guess that there is an official syntax and the data is provided by the NFL because if not you would have all types of formats and opinions about the game baked into each team's data. I would also guess the same office that keeps records (game, individual, all time, etc) are the ones that keep the play by play too.__BR__Overall this is neat but it's hard to find real life context within this data. Was the QB pressured, was a coverage blown, was there a pre-snap audible or motion or change by the defense, what was the formation, how much sleep did the players get the night before, etc etc.",
97
+ "id":"5003473",
98
+ "grayedOutPercent":0,
99
+ "reply_id":"5003473&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
100
+ "time":"2 hours ago",
101
+ "children":[
102
+
103
+ ]
104
+ }
105
+ ]
106
+ },
107
+ {
108
+ "username":"MattSayar",
109
+ "comment":"Please post your results, I'd love to read your analysis.",
110
+ "id":"5003373",
111
+ "grayedOutPercent":0,
112
+ "reply_id":"5003373&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
113
+ "time":"3 hours ago",
114
+ "children":[
115
+ {
116
+ "username":"sjs382",
117
+ "comment":"Agreed. I'd love to learn about the process, too. Not just the results\/findings.",
118
+ "id":"5003439",
119
+ "grayedOutPercent":0,
120
+ "reply_id":"5003439&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
121
+ "time":"2 hours ago",
122
+ "children":[
123
+
124
+ ]
125
+ }
126
+ ]
127
+ },
128
+ {
129
+ "username":"mav3r1ck",
130
+ "comment":"@edw518, Awesome! Thank you for sharing what you're doing here with the data. I was thinking of playing with this dataset too and since I'm new to this field (data), look forward to learning from you if you post more info in the future!__BR__Thanks!",
131
+ "id":"5003681",
132
+ "grayedOutPercent":0,
133
+ "reply_id":"5003681&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
134
+ "time":"2 hours ago",
135
+ "children":[
136
+ {
137
+ "username":"larrydag",
138
+ "comment":"One of the \\\"new\\\" ways that Burke (data creator) et al are using with this type of data is finding the Expected Points Added for each plays. The EPA allows one to determine how valuable players are to a team's performance.__BR__http:\/\/www.advancednflstats.com\/2010\/01\/expected-points-ep-a...__BR__I've been trying to work at the college football level with this same strategy but I'm still trying to figure out how its calculated. It seems trivial but it takes a lot of data organizing.",
139
+ "id":"5004431",
140
+ "grayedOutPercent":0,
141
+ "reply_id":"5004431&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
142
+ "time":"4 minutes ago",
143
+ "children":[
144
+
145
+ ]
146
+ }
147
+ ]
148
+ },
149
+ {
150
+ "username":"tesmar2",
151
+ "comment":"It looks like a lot of the work you are hoping to do has already been done on http:\/\/statsheet.com\/nfl__BR__Though perhaps not as open...",
152
+ "id":"5003703",
153
+ "grayedOutPercent":0,
154
+ "reply_id":"5003703&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
155
+ "time":"2 hours ago",
156
+ "children":[
157
+ {
158
+ "username":"tvon",
159
+ "comment":"See also, http:\/\/www.pro-football-reference.com\/, but I'd love to see something on github.",
160
+ "id":"5004053",
161
+ "grayedOutPercent":0,
162
+ "reply_id":"5004053&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
163
+ "time":"1 hour ago",
164
+ "children":[
165
+
166
+ ]
167
+ }
168
+ ]
169
+ }
170
+ ]
171
+ },
172
+ {
173
+ "username":"tghw",
174
+ "comment":"Looking through the 2002 season, there's an oddity around touchdowns and extra points. It seems that the 6 points for the touchdown are bundled with the extra point, and the score is not updated until the extra point is complete.__BR__It seems this might result in bugs, as in the Oct 20, 2002 game between Dallas and Arizona. In the third quarter, with a score of Arizona 6 - Dallas 0, Dallas scored a touchdown (row 13900) but \\\"aborted\\\" the extra point (row 13901). The 6 points for the Cowboys are not recorded in the data.__BR__The game eventually went to overtime, with the Cardinals kicking a winning field goal in OT for a final score of Arizona 9 - Dallas 6, but the data here records it as Arizona 6 - Dallas 0.",
175
+ "id":"5003378",
176
+ "grayedOutPercent":0,
177
+ "reply_id":"5003378&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
178
+ "time":"3 hours ago",
179
+ "children":[
180
+
181
+ ]
182
+ },
183
+ {
184
+ "username":"danso",
185
+ "comment":"There's a FAQ for this data that is on the site's main nav:__BR__http:\/\/www.advancednflstats.com\/2007\/02\/contact.html__BR__Of particular interest:__BR__Where did you get your data?__BR__Most of my team data comes from open online sources such as espn.com, nfl.com, myway.com, and yahoo.com. It's easy for anyone to grab whatever they're interested in from those sites.__BR__My play-by-play data comes from a source that's not publicly available, and at this time I regret that I cannot share it. However, I am working hard to develop a way to spread the wealth. One of my biggest goals is to help create a larger, more open, and more collaborative community for football research.__BR__----__BR__There's no real terms of service so I'm curious as to the constraints in using this for commercial purposes. I most definitely want to use this for teaching purposes (how to text-mine, how to build a web app from data, etc) but want to know what terms the data can be redistributed.",
186
+ "id":"5003989",
187
+ "grayedOutPercent":0,
188
+ "reply_id":"5003989&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
189
+ "time":"1 hour ago",
190
+ "children":[
191
+
192
+ ]
193
+ },
194
+ {
195
+ "username":"dude_abides",
196
+ "comment":"Here is an idea: build a predictive model of an offensive coach that predicts the play he will call, given a game situation (and based on that, build a predictiveness quotient for a coach).",
197
+ "id":"5003533",
198
+ "grayedOutPercent":0,
199
+ "reply_id":"5003533&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
200
+ "time":"2 hours ago",
201
+ "children":[
202
+ {
203
+ "username":"fleaflicker",
204
+ "comment":"It doesn't work like that in practice. Football is very dependent on matchups. Coaches will vary gameplans from week-to-week to exploit weaknesses they see on film.",
205
+ "id":"5003557",
206
+ "grayedOutPercent":0,
207
+ "reply_id":"5003557&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
208
+ "time":"2 hours ago",
209
+ "children":[
210
+ {
211
+ "username":"dude_abides",
212
+ "comment":"Matchup would be a part of the model. My experience with predictive modeling in various domains has taught me that people tend to underestimate how predictive they are (NFL offensive\/defensive coaches are no exception).",
213
+ "id":"5003599",
214
+ "grayedOutPercent":0,
215
+ "reply_id":"5003599&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
216
+ "time":"2 hours ago",
217
+ "children":[
218
+
219
+ ]
220
+ },
221
+ {
222
+ "username":"jpeterson",
223
+ "comment":"It could still be useful for finding tendencies, though.",
224
+ "id":"5003628",
225
+ "grayedOutPercent":0,
226
+ "reply_id":"5003628&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
227
+ "time":"2 hours ago",
228
+ "children":[
229
+ {
230
+ "username":"zachgemignani",
231
+ "comment":"Visualization of rushing tendencies using this very data http:\/\/labs.juiceanalytics.com\/spider\/index.html",
232
+ "id":"5004211",
233
+ "grayedOutPercent":0,
234
+ "reply_id":"5004211&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
235
+ "time":"40 minutes ago",
236
+ "children":[
237
+
238
+ ]
239
+ }
240
+ ]
241
+ }
242
+ ]
243
+ },
244
+ {
245
+ "username":"nchuhoai",
246
+ "comment":"Here is a start: https:\/\/www.dropbox.com\/s\/cy04oxaq83mxvoz\/report.pdf",
247
+ "id":"5004309",
248
+ "grayedOutPercent":0,
249
+ "reply_id":"5004309&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
250
+ "time":"24 minutes ago",
251
+ "children":[
252
+
253
+ ]
254
+ },
255
+ {
256
+ "username":"zgohr",
257
+ "comment":"If only a single play call had a single potential outcome, and that outcome was always met. Using these stats for predictions would seem extremely difficult beyond answering, \\\"will it be a run or a pass?\\\"",
258
+ "id":"5003587",
259
+ "grayedOutPercent":0,
260
+ "reply_id":"5003587&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
261
+ "time":"2 hours ago",
262
+ "children":[
263
+
264
+ ]
265
+ },
266
+ {
267
+ "username":"sethist",
268
+ "comment":"There are too many missing variables. The most obvious being who made the actual play call, the head coach, the offensive coordinator, or the quarterback.",
269
+ "id":"5003654",
270
+ "grayedOutPercent":0,
271
+ "reply_id":"5003654&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
272
+ "time":"2 hours ago",
273
+ "children":[
274
+
275
+ ]
276
+ }
277
+ ]
278
+ },
279
+ {
280
+ "username":"evanjacobs",
281
+ "comment":"A bit OT, but I thought this might be a good opportunity to mention the upcoming SportsHackDay in Seattle from Feb 1-3 which culminates in a group viewing of the SuperBowl.http:\/\/sportshackday.com\/",
282
+ "id":"5004432",
283
+ "grayedOutPercent":0,
284
+ "reply_id":"5004432&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
285
+ "time":"4 minutes ago",
286
+ "children":[
287
+
288
+ ]
289
+ },
290
+ {
291
+ "username":"arscan",
292
+ "comment":"I'm frankly surprised that this information is allowed to be distributed. I spent awhile in the financial services industry, and while it was really easy to obtain \\\"public\\\" information like stock quote data, I recall that we weren't allowed to simply scrape data from public sites... we had to pay a license fee to get a feed of the data if we were planning on repackaging & distributing it.__BR__It seems to me that the NFL would want to have exclusive rights to distribute this data and charge people a fee for access to it. Clearly I'm no expert in these legal affairs though.",
293
+ "id":"5003277",
294
+ "grayedOutPercent":0,
295
+ "reply_id":"5003277&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
296
+ "time":"3 hours ago",
297
+ "children":[
298
+ {
299
+ "username":"saturdayplace",
300
+ "comment":"I'd love to believe that this data falls underneath the \\\"Facts are not copyrightable\\\" decision: http:\/\/chart.copyrightdata.com\/c16B.html",
301
+ "id":"5004253",
302
+ "grayedOutPercent":0,
303
+ "reply_id":"5004253&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
304
+ "time":"33 minutes ago",
305
+ "children":[
306
+ {
307
+ "username":"xentronium",
308
+ "comment":"Facts are not copyrightable, but databases (compilations of data) are.__BR__IANALTINALA.",
309
+ "id":"5004366",
310
+ "grayedOutPercent":0,
311
+ "reply_id":"5004366&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
312
+ "time":"14 minutes ago",
313
+ "children":[
314
+
315
+ ]
316
+ }
317
+ ]
318
+ },
319
+ {
320
+ "username":"aidenn0",
321
+ "comment":"IANAL, but I asked one about this while ago; let's see if I can remember: It's complicated. The NFL broadcasts are copyrighted, and come with a statement that (among other things) distributing descriptions of the game is not allowed. That could be considered a derivative work.__BR__On the other hand, a live performance is generally not protected from copyright, so if you attend a live game to collect the data, you may be in the clear.__BR__The data isn't owned by the NFL, but all recordings of the games are, and so any data obtained by watching recordings of the games could potentially be controlled by the NFL.",
322
+ "id":"5003327",
323
+ "grayedOutPercent":0,
324
+ "reply_id":"5003327&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
325
+ "time":"3 hours ago",
326
+ "children":[
327
+ {
328
+ "username":"_delirium",
329
+ "comment":"It might not even violate the NFL's copyright if extracted from tapes. For one thing, something is only a \\\"derived work\\\" for copyright purposes if it's a \\\"creative work\\\" subject to copyright at all, and in the U.S., data sets comprising factual information aren't typically considered \\\"creative\\\". For another, it's not clear whether data about a recording is derived from the recording for copyright purposes. For example, a re-edit or mash-up of a film is clearly a derived work, but is a count of how many minutes each character speaks a derived work? Or is a Spotify-style algorithmic analysis of a song's musical style a derived work of the song?__BR__I wouldn't want to put a large bet on where exactly those lines are drawn, though.",
330
+ "id":"5003422",
331
+ "grayedOutPercent":0,
332
+ "reply_id":"5003422&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
333
+ "time":"2 hours ago",
334
+ "children":[
335
+
336
+ ]
337
+ },
338
+ {
339
+ "username":"sethist",
340
+ "comment":"IANAL too, but the NFL lost a relatively recent court case regarding fantasy football that, as far as I am aware, made the leagues statistics public knowledge as long as you compile the information yourself. Therefore the main legal issue with this data would be the source.",
341
+ "id":"5003357",
342
+ "grayedOutPercent":0,
343
+ "reply_id":"5003357&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
344
+ "time":"3 hours ago",
345
+ "children":[
346
+
347
+ ]
348
+ }
349
+ ]
350
+ },
351
+ {
352
+ "username":"vsprabhakara1",
353
+ "comment":"Generally speaking, stats are public domain as they are a public event that occurred. Because a sports league may disagree with this position doesn't mean that it isn't true. However, its entirely possible to violate a given site's TOU by scraping the data, it doesn't mean the data itself isn't allowed to be compiled or distributed.__BR__IANAL, but I worked at ESPN and founded Fanvibe (YC S'10), and worked quite a bit with the leagues and lawyers on rights-related topics.",
354
+ "id":"5003440",
355
+ "grayedOutPercent":0,
356
+ "reply_id":"5003440&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
357
+ "time":"2 hours ago",
358
+ "children":[
359
+
360
+ ]
361
+ },
362
+ {
363
+ "username":"dkoch",
364
+ "comment":"IANAL, but there was a landmark case where the NBA sued Motorola and STATS Inc. for distributing live game statistics. The ruling ended up in favor of STATS, where the decision was pure facts could not be copyrighted.",
365
+ "id":"5003584",
366
+ "grayedOutPercent":0,
367
+ "reply_id":"5003584&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
368
+ "time":"2 hours ago",
369
+ "children":[
370
+
371
+ ]
372
+ },
373
+ {
374
+ "username":"kodablah",
375
+ "comment":"The NFL is not near as strict as MLB (note how you never see an MLB highlight on youtube?) yet MLB allows http:\/\/retrosheet.org\/ to exist. I don't believe it's technically \\\"dissemination\\\".__BR__Edit: I should note that even http:\/\/gdx.mlb.com\/components\/game\/mlb\/ contents are governed by http:\/\/gdx.mlb.com\/components\/copyright.txt which doesn't allow commercial use.",
376
+ "id":"5003339",
377
+ "grayedOutPercent":0,
378
+ "reply_id":"5003339&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
379
+ "time":"3 hours ago",
380
+ "children":[
381
+ {
382
+ "username":"corin_",
383
+ "comment":"MLB are extremely strict, but it's an exaggeration to say \\\"you never see highlights on youtube\\\", I've watched them there many times. Likely either before they got taken down or because they were to small to care about, but there'll still be plenty on there. 5 second search brought up https:\/\/www.youtube.com\/watch?v=OZW7448mh94 right away as a very quick example.",
384
+ "id":"5004127",
385
+ "grayedOutPercent":0,
386
+ "reply_id":"5004127&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
387
+ "time":"53 minutes ago",
388
+ "children":[
389
+
390
+ ]
391
+ }
392
+ ]
393
+ },
394
+ {
395
+ "username":"frozenport",
396
+ "comment":"Lets pretend he got it from memory.",
397
+ "id":"5003708",
398
+ "grayedOutPercent":0,
399
+ "reply_id":"5003708&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
400
+ "time":"2 hours ago",
401
+ "children":[
402
+
403
+ ]
404
+ }
405
+ ]
406
+ },
407
+ {
408
+ "username":"danso",
409
+ "comment":"This looks like great fun...Judging by some of the sample entries, it will also be an instructive example of the limitations of CSV and why serious analysts who want to work with unstructured data need to know a scripting language, or at least regexes.__BR__Sample description field:> 20020905_SF@NYG,1,59,20,NYG,SF,3,11,81,(14:20) (Shotgun) K.Collins pass intended for T.Barber INTERCEPTED by T.Parrish (M.Rumph) at NYG 29. T.Parrish to NYG 23 for 6 yards (T.Barber).,0,0,2002__BR__In the comments section of the OP, someone posted this sample Excel function:__BR__ =IF(ISNUMBER(SEARCH(\\\"right tackle\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"right guard\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"left guard\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"up the middle\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"left tackle\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"left end\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"right end\\\",J2)),\\\"rush\\\",IF(ISNUMBER(SEARCH(\\\"pass\\\",J2)),\\\"pass\\\",IF(ISNUMBER(SEARCH(\\\"kneel\\\",J2)),\\\"kneel\\\",IF(ISNUMBER(SEARCH(\\\"punt\\\",J2)),\\\"punt\\\",IF(ISNUMBER(SEARCH(\\\"kicks\\\",J2)),\\\"kickoff\\\",IF(ISNUMBER(SEARCH(\\\"extra point\\\",J2)),\\\"extrapoint\\\",IF(ISNUMBER(SEARCH(\\\"sacked\\\",J2)),\\\"sack\\\",IF(ISNUMBER(SEARCH(\\\"PENALTY\\\",J2)),\\\"penalty\\\",IF(ISNUMBER(SEARCH(\\\"field goal\\\",J2)),\\\"fieldgoal\\\",IF(ISNUMBER(SEARCH(\\\"FUMBLES\\\",J2)),\\\"fumble\\\",IF(ISNUMBER(SEARCH(\\\"spiked\\\",J2)),\\\"spike\\\",IF(ISNUMBER(SEARCH(\\\"scrambles\\\",J2)),\\\"rush\\\",\\\"rush\\\"))))))))))))))))))Dear god, at what point do people finally realize that it's worth learning some simple scripting to work with text files?",
410
+ "id":"5003931",
411
+ "grayedOutPercent":0,
412
+ "reply_id":"5003931&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
413
+ "time":"1 hour ago",
414
+ "children":[
415
+
416
+ ]
417
+ },
418
+ {
419
+ "username":"petersalas",
420
+ "comment":"This seems like as good a time as any to share something I've been working on which uses the same source data, even though it's pretty rough at the moment (slow, bad data, only currently goes through week 8 of 2012, etc.):__BR__http:\/\/nfl-query.herokuapp.com\/__BR__The basic syntax is [stats] [conditions] : [row] \/ [column].__BR__There's some autocompletion to try to make it possible to discover what is accepted.__BR__Examples:__BR__passing yards : team \/ season__BR__first downs \/ first down attempts : down \/ distance__BR__rushing yards min 100 rushing yards : player, game, quarter__BR__rushing yards \/ carries min 200 carries : player__BR__One of the biggest problems is that it's currently way too easy to shoot yourself in the foot by making a really slow query.",
421
+ "id":"5004259",
422
+ "grayedOutPercent":0,
423
+ "reply_id":"5004259&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
424
+ "time":"32 minutes ago",
425
+ "children":[
426
+
427
+ ]
428
+ },
429
+ {
430
+ "username":"ImJasonH",
431
+ "comment":"I've started uploading these CSVs to a public Google BigQuery dataset called [nfl], so you can run queries over them like this:__BR__ SELECT off, COUNT(off) AS count FROM [nfl.2012reg] WHERE description CONTAINS \\\"INTERCEPTED\\\" GROUP BY off ORDER BY count DESC(This counts the number of plays that resulted in an interception by the team that threw the interception, sorted from most to fewest INTs)",
432
+ "id":"5003674",
433
+ "grayedOutPercent":0,
434
+ "reply_id":"5003674&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
435
+ "time":"2 hours ago",
436
+ "children":[
437
+ {
438
+ "username":"danvoell",
439
+ "comment":"I'm new to BigQuery, how do I access a public dataset? I ran the query and got the error Not Found: Dataset 578707073226:nfl",
440
+ "id":"5003747",
441
+ "grayedOutPercent":0,
442
+ "reply_id":"5003747&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
443
+ "time":"1 hour ago",
444
+ "children":[
445
+ {
446
+ "username":"ImJasonH",
447
+ "comment":"Sorry, I didn't actually make it public it seems. Should work now.",
448
+ "id":"5003794",
449
+ "grayedOutPercent":0,
450
+ "reply_id":"5003794&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
451
+ "time":"1 hour ago",
452
+ "children":[
453
+ {
454
+ "username":"danvoell",
455
+ "comment":"thanks!",
456
+ "id":"5003843",
457
+ "grayedOutPercent":0,
458
+ "reply_id":"5003843&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
459
+ "time":"1 hour ago",
460
+ "children":[
461
+
462
+ ]
463
+ }
464
+ ]
465
+ }
466
+ ]
467
+ },
468
+ {
469
+ "username":"snake_plissken",
470
+ "comment":"mean reversion between ints for NFL live betting on next int? i smell greenbacks!",
471
+ "id":"5003947",
472
+ "grayedOutPercent":0,
473
+ "reply_id":"5003947&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
474
+ "time":"1 hour ago",
475
+ "children":[
476
+
477
+ ]
478
+ }
479
+ ]
480
+ },
481
+ {
482
+ "username":"nchuhoai",
483
+ "comment":"For my ML class, I used this dataset to play around with predicting plays. Im obviously no expert, but wanted to share my report anyways:__BR__https:\/\/www.dropbox.com\/s\/cy04oxaq83mxvoz\/report.pdf",
484
+ "id":"5004293",
485
+ "grayedOutPercent":0,
486
+ "reply_id":"5004293&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
487
+ "time":"27 minutes ago",
488
+ "children":[
489
+
490
+ ]
491
+ },
492
+ {
493
+ "username":"euroclydon",
494
+ "comment":"How many of you are thinking right now: I'm going to generate an HTML page for every game and throw ads on it? Be Honest!",
495
+ "id":"5003274",
496
+ "grayedOutPercent":0,
497
+ "reply_id":"5003274&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
498
+ "time":"3 hours ago",
499
+ "children":[
500
+ {
501
+ "username":"DanBC",
502
+ "comment":"Is anyone going to try a 'moneyball' style Fix_Your_Fantasy_League_LineUp site with ads?",
503
+ "id":"5003426",
504
+ "grayedOutPercent":0,
505
+ "reply_id":"5003426&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
506
+ "time":"2 hours ago",
507
+ "children":[
508
+ {
509
+ "username":"404error",
510
+ "comment":"I might try to use the data to create mock drafts.",
511
+ "id":"5004083",
512
+ "grayedOutPercent":0,
513
+ "reply_id":"5004083&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
514
+ "time":"1 hour ago",
515
+ "children":[
516
+
517
+ ]
518
+ }
519
+ ]
520
+ },
521
+ {
522
+ "username":"pinchyfingers",
523
+ "comment":"hush :)",
524
+ "id":"5003338",
525
+ "grayedOutPercent":0,
526
+ "reply_id":"5003338&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
527
+ "time":"3 hours ago",
528
+ "children":[
529
+
530
+ ]
531
+ }
532
+ ]
533
+ },
534
+ {
535
+ "username":"eel",
536
+ "comment":"FYI, for baseball fans, you can get similar data about each play in the MLB from http:\/\/www.retrosheet.org",
537
+ "id":"5004352",
538
+ "grayedOutPercent":0,
539
+ "reply_id":"5004352&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
540
+ "time":"16 minutes ago",
541
+ "children":[
542
+
543
+ ]
544
+ },
545
+ {
546
+ "username":"bdittmer",
547
+ "comment":"Now all we need is some historical line & associated movement data...",
548
+ "id":"5004298",
549
+ "grayedOutPercent":0,
550
+ "reply_id":"5004298&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
551
+ "time":"26 minutes ago",
552
+ "children":[
553
+
554
+ ]
555
+ },
556
+ {
557
+ "username":"kevinburke",
558
+ "comment":"I wrote a small wrapper around the 4th down calculator on that site, which should help you figure out if your team should go for it on 4th down:__BR__http:\/\/downanddistance.herokuapp.com\/",
559
+ "id":"5003511",
560
+ "grayedOutPercent":0,
561
+ "reply_id":"5003511&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
562
+ "time":"2 hours ago",
563
+ "children":[
564
+ {
565
+ "username":"stuff4ben",
566
+ "comment":"Need a bounds check or two in there. Tried setting the number of yards you need to 1 and the yards away from the endzone to 99 and it threw up a nice exception. Cool calculator though!",
567
+ "id":"5003698",
568
+ "grayedOutPercent":0,
569
+ "reply_id":"5003698&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
570
+ "time":"2 hours ago",
571
+ "children":[
572
+ {
573
+ "username":"brianbreslin",
574
+ "comment":"As a Madden (game) aficionado my first thought was \\\"ALWAYS go for it on 4th!\\\" but then again I play super recklessly...",
575
+ "id":"5003795",
576
+ "grayedOutPercent":0,
577
+ "reply_id":"5003795&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
578
+ "time":"1 hour ago",
579
+ "children":[
580
+
581
+ ]
582
+ }
583
+ ]
584
+ }
585
+ ]
586
+ },
587
+ {
588
+ "username":"gavinlynch",
589
+ "comment":"Amazing!!! Thanks to www.advancednflstats.com for doing all the leg-work. Highly recommend their site too. Their in-game win probability statistics are always a must-have for me on game-day ^_^",
590
+ "id":"5003379",
591
+ "grayedOutPercent":0,
592
+ "reply_id":"5003379&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
593
+ "time":"3 hours ago",
594
+ "children":[
595
+ {
596
+ "username":"tghw",
597
+ "comment":"I really like his 4th Down analysis:__BR__http:\/\/www.advancednflstats.com\/2009\/09\/4th-down-study-part-...__BR__The tl;dr version can be found at:__BR__http:\/\/www.advancednflstats.com\/2010\/05\/4th-down-briefs.html__BR__The conclusion is that teams should go for it on 4th down much more often than they currently do.__BR__He also has a calculator where you can get the exact values:__BR__http:\/\/wp.advancednflstats.com\/4thdncalc1.php",
598
+ "id":"5003429",
599
+ "grayedOutPercent":0,
600
+ "reply_id":"5003429&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
601
+ "time":"2 hours ago",
602
+ "children":[
603
+ {
604
+ "username":"yukoncornelius",
605
+ "comment":"The Patriots have used this analysis:__BR__http:\/\/www.math.toronto.edu\/mpugh\/Teaching\/Sci199_03\/Footbal...__BR__I also believe Belichek\/Adams have funded some football economics research.",
606
+ "id":"5003741",
607
+ "grayedOutPercent":0,
608
+ "reply_id":"5003741&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
609
+ "time":"1 hour ago",
610
+ "children":[
611
+
612
+ ]
613
+ }
614
+ ]
615
+ }
616
+ ]
617
+ },
618
+ {
619
+ "username":"p4bl0",
620
+ "comment":"As a French not interested in sports at all this would have made no sense at all to me before I watched the TV series The League [1]. Now I kind of enjoy the fact that these stats exist and are available in an open format, even if I don't really care myself.__BR__[1] http:\/\/www.imdb.com\/title\/tt1480684\/",
621
+ "id":"5003726",
622
+ "grayedOutPercent":0,
623
+ "reply_id":"5003726&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
624
+ "time":"2 hours ago",
625
+ "children":[
626
+
627
+ ]
628
+ },
629
+ {
630
+ "username":"jredwards",
631
+ "comment":"I've used data from Brian Burke's site before. I think it's the exact PBP data the NFL has, but you'll find that the structure and common phrasings change over the years. I had to write a lot of regular expressions and I was still catching edge cases for weeks.__BR__btw, pro-football-reference has pbp data now too, and it probably goes back a lot further, but I think they discourage mass scraping of their site.",
632
+ "id":"5003952",
633
+ "grayedOutPercent":0,
634
+ "reply_id":"5003952&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
635
+ "time":"1 hour ago",
636
+ "children":[
637
+
638
+ ]
639
+ },
640
+ {
641
+ "username":"ScottWhigham",
642
+ "comment":"The comments on that are awesome too - great advice for parsing, categorizing, and such. I couldn't download 2010 though - \\\"Sorry, we are unable to generate a view of the document at this time. Please try again later.\\\"",
643
+ "id":"5003201",
644
+ "grayedOutPercent":0,
645
+ "reply_id":"5003201&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
646
+ "time":"3 hours ago",
647
+ "children":[
648
+ {
649
+ "username":"ScottWhigham",
650
+ "comment":"If you click the little down arrow (top left), it will download the file. Just a heads-up in case others see this message as well.",
651
+ "id":"5003206",
652
+ "grayedOutPercent":0,
653
+ "reply_id":"5003206&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
654
+ "time":"3 hours ago",
655
+ "children":[
656
+
657
+ ]
658
+ }
659
+ ]
660
+ },
661
+ {
662
+ "username":"winstonian",
663
+ "comment":"Would be great if there were columns:__BR__Head Coach- Offensive Coordinator- Defensive Coordinator- Formation- Play",
664
+ "id":"5004014",
665
+ "grayedOutPercent":0,
666
+ "reply_id":"5004014&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
667
+ "time":"1 hour ago",
668
+ "children":[
669
+
670
+ ]
671
+ },
672
+ {
673
+ "username":"activus",
674
+ "comment":"It would be interesting to take this data and build an app around it for fantasy football. If you have all the tendencies, and how players like your player have played against certain teams, you could make better guesses on who to play.",
675
+ "id":"5004222",
676
+ "grayedOutPercent":0,
677
+ "reply_id":"5004222&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
678
+ "time":"38 minutes ago",
679
+ "children":[
680
+
681
+ ]
682
+ },
683
+ {
684
+ "username":"asc76",
685
+ "comment":"Wow. Simply. Wow.",
686
+ "id":"5004028",
687
+ "grayedOutPercent":36,
688
+ "reply_id":"5004028&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
689
+ "time":"1 hour ago",
690
+ "children":[
691
+
692
+ ]
693
+ },
694
+ {
695
+ "username":"mcs",
696
+ "comment":"wow",
697
+ "id":"5003933",
698
+ "grayedOutPercent":36,
699
+ "reply_id":"5003933&whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
700
+ "time":"1 hour ago",
701
+ "children":[
702
+
703
+ ]
704
+ }
705
+ ]
706
+ }
data/data/lorem.txt ADDED
@@ -0,0 +1 @@
1
+ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer tristique est in est gravida sed convallis arcu porttitor. Curabitur hendrerit bibendum mauris eu gravida. Aliquam erat volutpat. Donec lectus purus, dapibus et mattis a, interdum in justo. Fusce in ante tortor, non euismod mi. Sed vel sapien ac sapien condimentum accumsan. Proin eu ante ut odio interdum tempus non quis justo. Nulla sed augue purus. Integer felis ante, placerat a placerat egestas, hendrerit a felis. Nulla elementum cursus orci, vel pretium neque pulvinar nec. Praesent sodales risus in nibh hendrerit non sagittis odio euismod.
data/hacker_term.gemspec CHANGED
@@ -14,6 +14,7 @@ Gem::Specification.new do |gem|
14
14
  gem.add_dependency('rest-client')
15
15
  gem.add_dependency('launchy')
16
16
  gem.add_dependency('clipboard')
17
+ gem.add_development_dependency('rspec')
17
18
 
18
19
  gem.files = `git ls-files`.split($/)
19
20
  gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
data/lib/hacker_term/comments_data.rb ADDED
@@ -0,0 +1,28 @@
1
+ require 'json'
2
+ require 'cgi'
3
+
4
+ module HackerTerm
5
+ class CommentsData
6
+ attr_reader :data
7
+
8
+ def initialize(data)
9
+ unescaped = CGI.unescapeHTML data
10
+ @data = JSON.parse(unescaped)['items']
11
+ end
12
+
13
+ def data_as_text(max_width)
14
+ @data.each do |line|
15
+ # words = line['comment'].split ' '
16
+ # new_line = ''
17
+ # words.inject(0) do |length, word|
18
+ # length + word.length + 1
19
+ # wor
20
+ # if length >= max_width
21
+ # length = 0
22
+
23
+ # end
24
+ # end
25
+ end
26
+ end
27
+ end
28
+ end
data/lib/hacker_term/page_data.rb CHANGED
@@ -1,4 +1,5 @@
1
1
  require 'json'
2
+ require 'cgi'
2
3
 
3
4
  # Controversial monkeypatch of String class so it can tell us if a string is a number
4
5
  class String
@@ -12,7 +13,8 @@ module HackerTerm
12
13
  attr_reader :data, :mean_score, :median_score, :mode_score, :sorted_by, :line_pos
13
14
 
14
15
  def initialize(data)
15
- @data = JSON.parse(data)['items']
16
+ unescaped = CGI.unescapeHTML data
17
+ @data = JSON.parse(unescaped)['items']
16
18
 
17
19
  add_missing_keys!
18
20
  format_numbers!
data/lib/hacker_term/ui.rb CHANGED
@@ -48,56 +48,67 @@ module HackerTerm
48
48
  addstr('-' * @total_width)
49
49
  end
50
50
 
51
- def draw_header
52
- output_divider(next_line_num)
53
- attrset color_pair(1)
54
- output_line(next_line_num, "HACKER NEWS TERMINAL - thanks to http://hndroidapi.appspot.com")
55
- output_line(next_line_num, "CMDS: Select (Arrows), Open Item (O), Open Item Discussion (D), Refresh (A)")
56
- output_line(next_line_num, "CMDS CONT: Sort by Rank (R), Score (S), Comments (C), Title (T) | Quit (Q)")
51
+ def <<(str)
52
+ throw 'invalid type' unless str.is_a? String
53
+ output_line(next_line_num, str)
54
+ end
55
+
56
+ def divider
57
57
  output_divider(next_line_num)
58
+ end
58
59
 
59
- # Get width_excl_title, i.e. width of all columns + some extra for |'s and spacing.
60
- # Once obtained, pad out the title column with the any width remaining
61
- # A nicer way to do this is always put the title last, and assume last column gets
62
- # remaining width. That way we can just loop through our cols, rather than hardcoding
63
- # them as per example below. I'm sticking to this because I want the title listed second.
64
- width_excl_title = @cols.inject(0) do |width, col|
65
- width += (3 + col.length)
60
+ def output(&blk)
61
+ blk.call self if block_given?
62
+ end
63
+
64
+ def draw_header
65
+ output do |buff|
66
+ buff.divider
67
+ attrset color_pair(1)
68
+ buff << "HACKER NEWS TERMINAL - thanks to http://hndroidapi.appspot.com"
69
+ buff << "CMDS: Select (Arrows), Open Item (O), Open Item Discussion (D), Refresh (A)"
70
+ buff << "CMDS CONT: Sort by Rank (R), Score (S), Comments (C), Title (T) | Quit (Q)"
71
+ buff.divider
72
+
73
+ # Get width_excl_title, i.e. width of all columns + some extra for |'s and spacing.
74
+ # Once obtained, pad out the title column with the any width remaining
75
+ # A nicer way to do this is always put the title last, and assume last column gets
76
+ # remaining width. That way we can just loop through our cols, rather than hardcoding
77
+ # them as per example below. I'm sticking to this because I want the title listed second.
78
+ width_excl_title = @cols.inject(0) do |width, col|
79
+ width += (3 + col.length)
80
+ end
81
+ attrset color_pair(2)
82
+ @title_width = @total_width - width_excl_title + 'title'.length
83
+ buff << "RANK | TITLE " + " " * (@total_width - width_excl_title) + "| SCORE | COMMENTS"
84
+ buff.divider
66
85
  end
67
- attrset color_pair(2)
68
- @title_width = @total_width - width_excl_title + 'title'.length
69
- output_line(next_line_num, "RANK | TITLE " + " " * (@total_width - width_excl_title) + "| SCORE | COMMENTS")
70
- output_divider(next_line_num)
71
86
  end
72
87
 
73
88
  def draw_footer(sorted_by, mean, median, mode)
74
- output_divider(next_line_num)
75
- attrset color_pair(1)
76
- formatted = sprintf("Sorted by: %7s | Scores: Mean: %4.2f | Median: %4.2f | Mode: %4.2f",
77
- sorted_by, mean, median, mode)
78
- output_line(next_line_num, formatted)
79
- output_divider(next_line_num)
89
+ output do |buff|
90
+ buff.divider
91
+ attrset color_pair(1)
92
+ buff << sprintf("Sorted by: %7s | Scores: Mean: %4.2f | Median: %4.2f | Mode: %4.2f",
93
+ sorted_by, mean, median, mode)
94
+ buff.divider
95
+ end
80
96
  end
81
97
 
82
98
  def draw_item_line(rank, data, selected)
99
+ # Truncate if too long
100
+ title = truncate_line! data
83
101
 
84
- begin
85
- # Truncate if too long
86
- title = truncate_line! data
87
-
88
- # Format and output
89
- if selected
90
- rank = '> ' + rank
91
- attrset color_pair(3)
92
- else
93
- attrset color_pair(0)
94
- end
95
-
96
- formatted = sprintf("%4s | %-#{@title_width}s | %5s | %8s", rank, title, data['score'], data['comments'])
97
- output_line(next_line_num, formatted)
98
- rescue => ex
99
- p "error: #{ex.to_s}"
102
+ # Format and output
103
+ if selected
104
+ rank = '> ' + rank
105
+ attrset color_pair(3)
106
+ else
107
+ attrset color_pair(0)
100
108
  end
109
+
110
+ self << sprintf("%4s | %-#{@title_width}s | %5s | %8s",
111
+ rank, title, data['score'], data['comments'])
101
112
  end
102
113
 
103
114
  def truncate_line!(data)
data/lib/hacker_term/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module HackerTerm
2
- VERSION = "0.0.4"
2
+ VERSION = "0.0.5"
3
3
  end
data/spec/comments_data_spec.rb ADDED
@@ -0,0 +1,58 @@
1
+ require 'hacker_term/comments_data'
2
+
3
+ module HackerTerm
4
+ describe CommentsData do
5
+ describe 'format JSON' do
6
+ let(:many_comments) { File.read './data/comments.json' }
7
+ let(:lorem) { File.read './data/lorem.txt'}
8
+
9
+ it 'creates an array of comments' do
10
+ comments = CommentsData.new many_comments
11
+ comments.data.should be_instance_of Array
12
+ end
13
+
14
+ it 'removes HTML entities' do
15
+ data = '{
16
+ "items":[
17
+ {
18
+ "username":"tghw",
19
+ "comment":"Looking through &amp; hi there.",
20
+ "id":"5003378",
21
+ "grayedOutPercent":0,
22
+ "reply_id":"5003378&amp;whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
23
+ "time":"3 hours ago",
24
+ "children":[
25
+
26
+ ]
27
+ }
28
+ ]
29
+ }'
30
+
31
+ comments = CommentsData.new data
32
+ comments.data.first['comment'].should_not match /&amp;/
33
+ end
34
+
35
+ it 'splits a long line into fixed width lines preserving words' do
36
+ data = %{
37
+ {
38
+ "items":[
39
+ {
40
+ "username":"tghw",
41
+ "comment":"#{lorem}",
42
+ "id":"5003378",
43
+ "grayedOutPercent":0,
44
+ "reply_id":"5003378&amp;whence=%69%74%65%6d%3f%69%64%3d%35%30%30%32%39%37%34",
45
+ "time":"3 hours ago",
46
+ "children":[
47
+
48
+ ]
49
+ }
50
+ ]
51
+ }
52
+ }
53
+ comments = CommentsData.new data
54
+ p comments.data
55
+ end
56
+ end
57
+ end
58
+ end
data/spec/page_data_spec.rb CHANGED
@@ -2,7 +2,7 @@ require 'hacker_term/page_data'
2
2
 
3
3
  module HackerTerm
4
4
  describe PageData do
5
- describe 'replace missing nodes and format numbers' do
5
+ describe 'formatting JSON' do
6
6
  before(:each) do
7
7
  @data =
8
8
  '{"items":[
@@ -12,7 +12,7 @@ module HackerTerm
12
12
  "description":"hn next id news2 "
13
13
  },
14
14
  {
15
- "title":"Ray Kurzweil joins Google",
15
+ "title":"Ray Kurzweil joins Google &amp;",
16
16
  "url":"http://www.kurzweilai.net/kurzweil-joins-google-to-work-on-new-projects-involving-machine-learning-and-language-processing?utm_source=twitterfeed&utm_medium=twitter",
17
17
  "score":"260 points",
18
18
  "user":"dumitrue",
@@ -49,11 +49,15 @@ module HackerTerm
49
49
  @pd.data.last['comments'].should == '122'
50
50
  end
51
51
 
52
+ it 'unescapes HTML entities in title where present' do
53
+ @pd.data.last['title'].should_not match /&amp;/
54
+ end
55
+
52
56
  end
53
57
 
54
58
  describe 'calculating stats' do
55
59
  before(:each) do
56
- @page_data = HackerTerm::PageData.new File.read './data/data.json'
60
+ @page_data = PageData.new File.read './data/data.json'
57
61
  end
58
62
 
59
63
  it 'provides a mean' do
@@ -71,7 +75,7 @@ module HackerTerm
71
75
 
72
76
  describe 'formatting URLs' do
73
77
  before(:each) do
74
- @pg = HackerTerm::PageData.new File.read './data/data.json'
78
+ @pg = PageData.new File.read './data/data.json'
75
79
  end
76
80
 
77
81
  it 'provides a URL for actual article' do
metadata CHANGED
@@ -2,7 +2,7 @@
2
2
  name: hacker_term
3
3
  version: !ruby/object:Gem::Version
4
4
  prerelease:
5
- version: 0.0.4
5
+ version: 0.0.5
6
6
  platform: ruby
7
7
  authors:
8
8
  - Ciaran Archer
@@ -10,7 +10,7 @@ autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
12
 
13
- date: 2013-01-03 00:00:00 Z
13
+ date: 2013-01-07 00:00:00 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: rest-client
@@ -45,6 +45,17 @@ dependencies:
45
45
  version: "0"
46
46
  type: :runtime
47
47
  version_requirements: *id003
48
+ - !ruby/object:Gem::Dependency
49
+ name: rspec
50
+ prerelease: false
51
+ requirement: &id004 !ruby/object:Gem::Requirement
52
+ none: false
53
+ requirements:
54
+ - - ">="
55
+ - !ruby/object:Gem::Version
56
+ version: "0"
57
+ type: :development
58
+ version_requirements: *id004
48
59
  description: Read Hacker News on the Terminal
49
60
  email:
50
61
  - ciaran.archer@gmail.com
@@ -56,18 +67,23 @@ extra_rdoc_files: []
56
67
 
57
68
  files:
58
69
  - .gitignore
70
+ - .travis.yml
59
71
  - Gemfile
60
72
  - Gemfile.lock
61
73
  - LICENSE.txt
62
74
  - README.md
63
75
  - Rakefile
64
76
  - bin/hacker_term
77
+ - data/comments.json
65
78
  - data/data.json
79
+ - data/lorem.txt
66
80
  - hacker_term.gemspec
67
81
  - lib/hacker_term.rb
82
+ - lib/hacker_term/comments_data.rb
68
83
  - lib/hacker_term/page_data.rb
69
84
  - lib/hacker_term/ui.rb
70
85
  - lib/hacker_term/version.rb
86
+ - spec/comments_data_spec.rb
71
87
  - spec/page_data_spec.rb
72
88
  homepage: https://github.com/ciaranarcher/hacker_term
73
89
  licenses: []
@@ -97,4 +113,5 @@ signing_key:
97
113
  specification_version: 3
98
114
  summary: Allows the reading, sorting and opening of HN articles from the terminal.
99
115
  test_files:
116
+ - spec/comments_data_spec.rb
100
117
  - spec/page_data_spec.rb