bwkfanboy 0.1.1 → 0.1.2

data/README.rdoc

@@ -7,12 +7,16 @@ The converter is not a magick tool: you'll need to write a plugin (in
 Ruby) for each site you want to watch. bwkfanboy provides guidelines and
 general assistance.
 
+(Plugins included with bwkfanboy are usually updated more frequently
+than the whole gem on rubygems.org, so grab the source before
+struggling).
+
 = Architecture
 
 == Plugins
 
-bwkfanboy comes with 1 exmple plugin that parses a search page of
-dailyprincetonian.com looking for bwk's articles.
+bwkfanboy comes with several plugins. One of them, for example, parses a
+search page of dailyprincetonian.com looking for bwk's articles.
 
 The plugin is a Ruby class +Page+ that inherits Bwkfanboy::Parse
 parent, overriding 1 method.
@@ -86,3 +90,6 @@ There are 2 method to get an Atom feed via HTTP:
 2. Small *bwkfanboy_server* HTTP server. It can run from any user and
    thus is able to inherit env variables for discovering your HOME
    directory. Read bin/bwkfanboy_server to know how to operate it.
+
+= License
+:include: doc/LICENSE

data/Rakefile

@@ -9,7 +9,7 @@ require 'rake/testtask'
 spec = Gem::Specification.new() {|i|
   i.name = "bwkfanboy"
   i.summary = 'A converter from HTML to Atom feed that you can use to watch sites that do not provide its own feed.'
-  i.version = '0.1.1'
+  i.version = '0.1.2'
   i.author = 'Alexander Gromnitsky'
   i.email = 'alexander.gromnitsky@gmail.com'
   i.homepage = 'http://github.com/gromnitsky/bwkfanboy'

data/TODO

@@ -1,7 +1,4 @@
 -*-text-*-
 
-0.0.2
------
-
 - Add plugin listing to bwkfanboy_server.
 - More tests.

data/doc/NEWS.rdoc

@@ -1,3 +1,7 @@
+=== Current
+
+- See git log.
+
 === 0.1.1
 
 - Plugins can have user-supplied options in realtime.

data/doc/README.rdoc

@@ -7,12 +7,16 @@ The converter is not a magick tool: you'll need to write a plugin (in
 Ruby) for each site you want to watch. bwkfanboy provides guidelines and
 general assistance.
 
+(Plugins included with bwkfanboy are usually updated more frequently
+than the whole gem on rubygems.org, so grab the source before
+struggling).
+
 = Architecture
 
 == Plugins
 
-bwkfanboy comes with 1 exmple plugin that parses a search page of
-dailyprincetonian.com looking for bwk's articles.
+bwkfanboy comes with several plugins. One of them, for example, parses a
+search page of dailyprincetonian.com looking for bwk's articles.
 
 The plugin is a Ruby class +Page+ that inherits Bwkfanboy::Parse
 parent, overriding 1 method.
@@ -86,3 +90,6 @@ There are 2 method to get an Atom feed via HTTP:
 2. Small *bwkfanboy_server* HTTP server. It can run from any user and
    thus is able to inherit env variables for discovering your HOME
    directory. Read bin/bwkfanboy_server to know how to operate it.
+
+= License
+:include: doc/LICENSE

data/doc/plugin.rdoc

@@ -102,28 +102,28 @@ HTML you want to parse. The general idea:
 
 === Options
 
-Plugins can have _options_ which user should provide to the plugin in
-the real-time. For example, say you're scraping a site where many users
-are wasting their time. If you want to watch for several of them it is
-silly to write a new plugin every time for a new participant. Instead,
-you can write 1 plugin which have an _option_ to take a parameter (a
-user name, in this case).
+Plugins can have _options_ and a user should provide then to the plugin
+in the real-time. For example, say you're scraping a site where many
+users are wasting their time. If you want to watch for several of them
+it is silly to write a new plugin every time for a new
+participant. Instead, you can write 1 plugin which have an _option_ to
+take a parameter (a user name, in this case).
 
-Options (if any) are always accessible via \#opt method which is just
-attr_reader of a hash.
+Options (if any) are always accessible via \#opt method which is just an
+attr_reader of an array.
 
-The really interesting trick one can to play with Meta::URI constant. It
-is possible to make it dynamic, for example:
+One can play the really interesting trick with Meta::URI constant. It is
+possible to make it dynamic, for example:
 
   URI = 'http://www.quora.com/#{opt[0]}/answers'
 
-Then, if user will provide 1 option (say 'Mark-Suster')--it will appear
-in the final URI as follows:
+Then, if a user will provide 1 option (say 'Mark-Suster')--it will
+appear in the final URI as follows:
 
   http://www.quora.com/Mark-Suster/answers
 
-Such dynamic is possible only for Meta::URI constant and if it is not
-static, _option_ becomes mandatory for the end-user.
+Such dynamic is possible only for Meta::URI constant and in such case,
+_option_ becomes mandatory for the end-user.
 
 == How to test all this
 

data/lib/bwkfanboy/plugins/bwk.rb

@@ -9,7 +9,7 @@ class Page < Bwkfanboy::Parse
     URI_DEBUG = '/home/alex/lib/software/alex/bwkfanboy/test/semis/bwk.html'
     ENC = 'UTF-8'
     VERSION = 1
-    COPYRIGHT = '(c) 2010 Alexander Gromnitsky'
+    COPYRIGHT = "See bwkfanboy's LICENSE file"
     TITLE = "Brian Kernighan's articles from Daily Princetonian"
     CONTENT_TYPE = 'html'
   end

data/lib/bwkfanboy/plugins/freebsd-ports-update.rb

@@ -6,7 +6,7 @@ class Page < Bwkfanboy::Parse
     URI_DEBUG = URI
     ENC = 'ASCII'
     VERSION = 1
-    COPYRIGHT = '(c) 2010 Alexander Gromnitsky'
+    COPYRIGHT = "See bwkfanboy's LICENSE file"
     TITLE = "News from FreeBSD ports"
     CONTENT_TYPE = 'text'
   end

data/lib/bwkfanboy/plugins/quora.js

@@ -82,6 +82,9 @@ function prepare4eval(body) {
 "function LoginSignal(args) { return arr(arguments) }\n" +
 "function LiveLogin(args) { return arr(arguments) }\n" +
 "function PresencePageMonitor(args) { return arr(arguments) }\n" +
+"function UserSig(args) { return arr(arguments) }\n" +
+"function HeaderLogo(args) { return arr(arguments) }\n" +
+"function NavElement(args) { return arr(arguments) }\n" +
 	'';
 	var tail = "\n_components;\n";
 

data/lib/bwkfanboy/plugins/quora.rb

@@ -17,9 +17,9 @@ class Page < Bwkfanboy::Parse
     URI = 'http://www.quora.com/#{opt[0]}/answers'
     URI_DEBUG = '/home/alex/lib/software/alex/bwkfanboy/test/semis/quora.html'
     ENC = 'UTF-8'
-    VERSION = 1
+    VERSION = 3
     COPYRIGHT = "See bwkfanboy's LICENSE file"
-    TITLE = "Last n answers (per-user) from Quora."
+    TITLE = "Last n answers (per-user) from Quora; requires nodejs"
     CONTENT_TYPE = 'html'
   end
   
@@ -34,8 +34,11 @@ class Page < Bwkfanboy::Parse
     doc.xpath("//script").each {|i|
       js = i.text
       if js.include?('"epoch_us"')
+        if Bwkfanboy::Utils.cfg[:verbose] >= 3
+          File.open("#{File.basename(__FILE__)}-epoch.js.raw", "w+") {|i| i.puts js }
+        end
         r = Bwkfanboy::Utils.cmd_run("echo '#{js}' | #{File.dirname(__FILE__)}/quora.js")
-        fail 'evaluation in nodejs failed' if r[0] != 0
+        fail "evaluation in nodejs failed: #{r[1]}" if r[0] != 0
         tstp = JSON.parse(r[2])
         break
       end
@@ -51,6 +54,7 @@ class Page < Bwkfanboy::Parse
       l = clean(i.xpath("h2//a")[0].attributes['href'].value())
       next unless tstp.key?(l)  # ignore answers without timestamps
       u = date(Time.at(tstp[l]/1000/1000).to_s)
+#      u = DateTime.new.iso8601
       l = url + l + '/answer/' + profile
       
       c = i.xpath("../div[@class='hidden expanded_q_text']/div").inner_html(encoding: Meta::ENC)

data/lib/bwkfanboy/utils.rb

@@ -7,7 +7,7 @@ require 'active_support/core_ext/module/attribute_accessors'
 module Bwkfanboy
   module Meta
     NAME = 'bwkfanboy'
-    VERSION = '0.1.1'
+    VERSION = '0.1.2'
     USER_AGENT = "#{NAME}/#{VERSION} (#{RUBY_PLATFORM}; N; #{Encoding.default_external.name}; #{RUBY_ENGINE}; rv:#{RUBY_VERSION}.#{RUBY_PATCHLEVEL})"
     PLUGIN_CLASS = 'Page'
     DIR_TMP = "/tmp/#{Meta::NAME}/#{ENV['USER']}"
@@ -125,9 +125,12 @@ module Bwkfanboy
 
     # used in CGI and WEBrick examples
     def self.cmd_run(cmd)
-      pid, stdin, stdout, stderr = Open4::popen4(cmd)
-      ignored, status = Process::waitpid2(pid)
-      [status.exitstatus, stderr.read, stdout.read]
+      so = sr = ''
+      status = Open4::popen4(cmd) { |pid, stdin, stdout, stderr|
+        so = stdout.read
+        sr = stderr.read
+      }
+      [status.exitstatus, sr, so]
     end
 
     def self.gem_dir_system