feedjira 2.0.0 → 2.1.0

data/fixtures/vcr_cassettes/success.yml

@@ -0,0 +1,281 @@
+---
+http_interactions:
+- request:
+    method: get
+    uri: http://feedjira.com/blog/feed.xml
+    body:
+      encoding: US-ASCII
+      string: ''
+    headers:
+      User-Agent:
+      - Faraday v0.9.2
+  response:
+    status:
+      code: 200
+      message: 
+    headers:
+      date:
+      - Mon, 31 Oct 2016 02:29:13 GMT
+      server:
+      - Apache/2.4.18 (Ubuntu)
+      last-modified:
+      - Fri, 07 Oct 2016 14:37:00 GMT
+      etag:
+      - '"393e-53e4757c9db00-gzip"'
+      accept-ranges:
+      - bytes
+      vary:
+      - Accept-Encoding
+      content-length:
+      - '5051'
+      connection:
+      - close
+      content-type:
+      - application/xml
+    body:
+      encoding: ASCII-8BIT
+      string: |
+        <?xml version="1.0" encoding="UTF-8"?>
+        <feed xmlns="http://www.w3.org/2005/Atom">
+          <title>Feedjira Blog</title>
+          <subtitle>A Blog for Feedjira</subtitle>
+          <id>http://feedjira.com/blog</id>
+          <link href="http://feedjira.com/blog"/>
+          <link href="http://feedjira.com/blog/feed.xml" rel="self"/>
+          <updated>2015-06-05T00:00:00Z</updated>
+          <author>
+            <name>Jon Allured</name>
+          </author>
+          <entry>
+            <title>Feedjira Two-Point-Oh</title>
+            <link rel="alternate" href="http://feedjira.com/blog/2015/06/05/feedjira-two-point-oh.html"/>
+            <id>http://feedjira.com/blog/2015/06/05/feedjira-two-point-oh.html</id>
+            <published>2015-06-05T00:00:00Z</published>
+            <updated>2015-04-17T16:16:07-05:00</updated>
+            <author>
+              <name>Jon Allured</name>
+            </author>
+            <content type="html">&lt;p&gt;About a year ago, I took Feedjira to &lt;a href="http://feedjira.com/blog/2014/03/17/feedjira-goes-one-point-oh.html"&gt;version 1.0&lt;/a&gt; and today I&amp;rsquo;m releasing
+        version 2.0. Not much has changed actually - the biggest difference is that
+        &lt;code&gt;curb&lt;/code&gt; has been replaced with &lt;code&gt;faraday&lt;/code&gt;. For many users, upgrading to version
+        2.0 won&amp;rsquo;t require anything more than a &lt;code&gt;bundle update feedjira&lt;/code&gt;.&lt;/p&gt;
+
+        &lt;p&gt;The other big change is that the &lt;code&gt;Feedjira#update&lt;/code&gt; method was removed. It was
+        unreliable and mostly just caused issues. To read more about why this was
+        removed, check out &lt;a href="https://github.com/feedjira/feedjira/issues/218#issuecomment-40282091"&gt;this comment&lt;/a&gt; on a GitHub issue that was opened
+        about it not working as expected.&lt;/p&gt;
+
+        &lt;p&gt;I&amp;rsquo;ve also updated the site to reflect the changes in this version. I took the
+        opportunity to greatly simply things and got just about all the documentation to
+        fit on one page.&lt;/p&gt;
+
+        &lt;p&gt;I hope you like version 2.0 - feel free to hit me up &lt;a href="https://twitter.com/feedjira"&gt;on Twitter&lt;/a&gt; or if
+        you have any trouble, open a &lt;a href="https://github.com/feedjira/feedjira/issues"&gt;GitHub issue&lt;/a&gt; and I&amp;rsquo;ll give you a hand.&lt;/p&gt;
+
+        &lt;p&gt;Finally, thanks to those that gave me feedback on either my thoughts on version
+        2.0 or the release candidate - I really appreciate it!&lt;/p&gt;
+        </content>
+          </entry>
+          <entry>
+            <title>Thoughts on Version Two-Point-Oh</title>
+            <link rel="alternate" href="http://feedjira.com/blog/2014/04/14/thoughts-on-version-two-point-oh.html"/>
+            <id>http://feedjira.com/blog/2014/04/14/thoughts-on-version-two-point-oh.html</id>
+            <published>2014-04-14T00:00:00Z</published>
+            <updated>2014-09-05T12:15:28-05:00</updated>
+            <author>
+              <name>Jon Allured</name>
+            </author>
+            <content type="html">&lt;p&gt;TLDR: development of version 2.0 is being done on the &lt;a href="https://github.com/feedjira/feedjira/tree/two-point-oh"&gt;two-point-oh branch&lt;/a&gt;
+        and a &lt;a href="https://github.com/feedjira/feedjira/issues/221"&gt;GitHub issue&lt;/a&gt; is open for you to give feedback on its direction.&lt;/p&gt;
+
+        &lt;p&gt;I&amp;rsquo;ve starting work on a rewrite for Feedjira and I wanted to try using a &lt;a href="https://github.com/feedjira/feedjira/issues/221"&gt;GitHub
+        issue&lt;/a&gt; to discuss what I&amp;rsquo;d like to see in the new version. I&amp;rsquo;m very
+        interested in what users think, so please chime in!!&lt;/p&gt;
+
+        &lt;p&gt;In no particular order, here are some design goals for the new version:&lt;/p&gt;
+
+        &lt;h2&gt;Change HTTP Library&lt;/h2&gt;
+
+        &lt;p&gt;This gem depends on &lt;a href="https://github.com/taf2/curb"&gt;curb&lt;/a&gt;, but I&amp;rsquo;d like to switch that to something that will
+        allow me to support JRuby. I did some experimenting and really liked working
+        with &lt;a href="https://github.com/lostisland/faraday"&gt;Faraday&lt;/a&gt;, so that&amp;rsquo;s what I&amp;rsquo;m using at the moment.&lt;/p&gt;
+
+        &lt;h2&gt;Forget Concurrency&lt;/h2&gt;
+
+        &lt;p&gt;Throwing an array of feed urls at Feedjira and watching it concurrently fetch
+        and parse them extremely quickly is cool in READMEs and demos, but I don&amp;rsquo;t think
+        it&amp;rsquo;s reality. I think what&amp;rsquo;s much more likely is a single url being passed and
+        concurrency being dealt with in the application, workers being the likely
+        approach.&lt;/p&gt;
+
+        &lt;h2&gt;More Objects&lt;/h2&gt;
+
+        &lt;p&gt;Feedjira has a dirty little secret and it&amp;rsquo;s the &lt;code&gt;Feedjira::Feed&lt;/code&gt; class. If you
+        open that sucker up, you&amp;rsquo;ll see 19 class methods and 0 instance methods. These
+        class methods do everything from fetching and parsing to setting up curb and
+        unzipping raw xml. Not only is &lt;a href="http://en.wikipedia.org/wiki/Single_responsibility_principle"&gt;SRP&lt;/a&gt; off in the corner crying, its a mess to
+        test. I&amp;rsquo;d like to break this down into smaller objects that get instantiated and
+        passed around in normal OO fashion.&lt;/p&gt;
+
+        &lt;h2&gt;Fewer Parsers&lt;/h2&gt;
+
+        &lt;p&gt;One of the things I like best about Feedjira is the ability for a user to define
+        their own custom parsers and I think its under-used. What I&amp;rsquo;d like to see is
+        Feedjira ship with a parser for Atom and a parser for RSS - the others are
+        really just custom parsers.&lt;/p&gt;
+
+        &lt;h2&gt;Custom Parser Project&lt;/h2&gt;
+
+        &lt;p&gt;With version 1.2, I was able to spin off the benchmarks into &lt;a href="https://github.com/feedjira/feedjira-benchmarks"&gt;their own
+        project&lt;/a&gt; and I was very happy with how it turned out - I&amp;rsquo;d like to see a
+        similar thing happen with the custom parsers mentioned above. You should be able
+        to easily grab these parsers and throw them in your application. If you write a
+        particularly cool custom parser, then you could contribute it back and others
+        could benefit from it.&lt;/p&gt;
+
+        &lt;h2&gt;Raise Exceptions Instead of Callbacks&lt;/h2&gt;
+
+        &lt;p&gt;Feedjira almost never raises an exception and instead provides the ability to
+        register callbacks for success and failure situations, but this has proven
+        brittle and painful. With the new version, I want to see sensible exceptions
+        raised when something goes wrong and that pretty much removes the need for
+        callbacks, so those will be cut.&lt;/p&gt;
+
+        &lt;h2&gt;Feed Updating is Business Logic&lt;/h2&gt;
+
+        &lt;p&gt;The updating functionality of Feedjira leaves a lot to be desired - its
+        inconsistent and there are lots of edges. I tend to steer users away and
+        encourage them to write this kind of code in their application instead. See
+        &lt;a href="https://github.com/feedjira/feedjira/issues/218"&gt;this discussion&lt;/a&gt; for more, but needless to say, I&amp;rsquo;d like to see this entire
+        feature set removed.&lt;/p&gt;
+
+        &lt;h2&gt;Configuration&lt;/h2&gt;
+
+        &lt;p&gt;I want configuration for Feedjira done in a proper config block like most gems.
+        If you want to change the default parsers, that should be a config option. If
+        you want to set the user agent, that should be a configuration option.&lt;/p&gt;
+
+        &lt;h2&gt;Feedback Appreciated&lt;/h2&gt;
+
+        &lt;p&gt;Do you have any other ideas to make Feedjira better? Please check out the
+        &lt;a href="https://github.com/feedjira/feedjira/issues/221"&gt;GitHub issue&lt;/a&gt; and share what you&amp;rsquo;re thinking. I&amp;rsquo;m open to any feedback, so
+        if I just threw your favorite feature on the chopping block, please tell me!&lt;/p&gt;
+        </content>
+          </entry>
+          <entry>
+            <title>A Separate Project For Benchmarks</title>
+            <link rel="alternate" href="http://feedjira.com/blog/2014/04/11/a-separate-project-for-benchmarks.html"/>
+            <id>http://feedjira.com/blog/2014/04/11/a-separate-project-for-benchmarks.html</id>
+            <published>2014-04-11T00:00:00Z</published>
+            <updated>2014-09-05T12:15:28-05:00</updated>
+            <author>
+              <name>Jon Allured</name>
+            </author>
+            <content type="html">&lt;p&gt;With the release of version 1.2.0, the benchmarks for Feedjira have been moved
+        to a separate project: &lt;a href="https://github.com/feedjira/feedjira-benchmarks"&gt;https://github.com/feedjira/feedjira-benchmarks&lt;/a&gt;.&lt;/p&gt;
+
+        &lt;p&gt;What&amp;rsquo;s nice about this is that it keeps things much more organized and allowed
+        me to add a pretty neat new benchmark - more on that in a moment. I also rewrote
+        the &lt;a href="/benchmarks.html"&gt;Benchmarks page&lt;/a&gt; on this site so please check that out.&lt;/p&gt;
+
+        &lt;p&gt;One benchmark that I didn&amp;rsquo;t pull over to the new project is the fetching one. I
+        left it out because it relied on the consistency of your network connection and
+        that didn&amp;rsquo;t feel like it was scientific enough.&lt;/p&gt;
+
+        &lt;p&gt;In its place I&amp;rsquo;ve added a benchmark for parsing speed that&amp;rsquo;s run against each
+        version of the gem. Its pretty cool - when you clone down the
+        feedjira-benchmarks project, you also clone down feedjira itself and then the
+        script moves to each version tag in the git repo, runs the benchmark and records
+        the results. It was pretty fun to write and works like a charm.&lt;/p&gt;
+
+        &lt;p&gt;For the charts, I ended up learning some &lt;a href="http://www.r-project.org/"&gt;R&lt;/a&gt; so that I could have a little
+        more control.&lt;/p&gt;
+
+        &lt;p&gt;Benchmarks are really only valuable if they can be reproduced by others, so I&amp;rsquo;ve
+        written up instructions in the README for setting things up and running them. If
+        you&amp;rsquo;re interested, take a look and try running them yourself!&lt;/p&gt;
+
+        &lt;p&gt;I&amp;rsquo;ll continue to use the benchmarks to ensure Feedjira stays fast, so you wont
+        have to worry about your favorite Ruby feed library slowing down. If you have
+        any feedback for me on the benchmarks, feel free to hit me up on Twitter
+        (&lt;a href="https://twitter.com/feedjira"&gt;@feedjira&lt;/a&gt;) or &lt;a href="https://github.com/feedjira/feedjira-benchmarks/issues"&gt;open an issue&lt;/a&gt; on the project&amp;rsquo;s repo.&lt;/p&gt;
+        </content>
+          </entry>
+          <entry>
+            <title>Feedjira Goes One-Point-Oh</title>
+            <link rel="alternate" href="http://feedjira.com/blog/2014/03/17/feedjira-goes-one-point-oh.html"/>
+            <id>http://feedjira.com/blog/2014/03/17/feedjira-goes-one-point-oh.html</id>
+            <published>2014-03-17T00:00:00Z</published>
+            <updated>2014-09-05T12:15:28-05:00</updated>
+            <author>
+              <name>Jon Allured</name>
+            </author>
+            <content type="html">&lt;p&gt;Last fall, I asked &lt;a href="http://www.pauldix.net"&gt;Paul Dix&lt;/a&gt; if I could take over maintenance of his gem
+        Feedzirra. My request was totally out of the blue, so I was pretty pumped when
+        he got right back to me and said yes. He said that he didn&amp;rsquo;t have time to work
+        on it anymore and so I should feel free to do whatever I thought was best.&lt;/p&gt;
+
+        &lt;p&gt;Score!&lt;/p&gt;
+
+        &lt;p&gt;My first order of business was to go through the many open issues and pull
+        requests on GitHub. When I started there were over 60, a number that I&amp;rsquo;ve gotten
+        down to just a few. I thought it was important to ensure that users saw me treat
+        their issue as important and even if it was very old (which many were), I asked
+        if there was anything I could do to help.&lt;/p&gt;
+
+        &lt;p&gt;I was pleasantly surprised by the nice way many people responded and we got to
+        work addressing their questions and issues.&lt;/p&gt;
+
+        &lt;p&gt;As I was working through issues and pull requests, I kept &lt;a href="http://semver.org"&gt;SemVer&lt;/a&gt; in mind -
+        bug fixes in patch releases and backward-compatible changes in minor releases.
+        But I also realized that it was past time for this project to be at version 1.0.
+        In the SemVer FAQ, they talk about when to release version 1.0 and Feedzirra fit
+        the bill: it was being used in production, there was a stable API and I was
+        taking backwards compatibilty seriously.&lt;/p&gt;
+
+        &lt;p&gt;So I treated it as a project at 1.0 and I did my best to release versions that
+        were backward compatible and added deprecations for what I wanted to do in 1.0.
+        I saw things that I wanted to completely rewrite, but I resisted the urge to
+        burn it all down and start again.&lt;/p&gt;
+
+        &lt;p&gt;When I was close to being caught up on the backlog of issues and pull requests,
+        I started thinking about releasing version 1.0, and I knew I wanted to create a
+        website for the project. I worked with &lt;a href="http://danielariza.com"&gt;Daniel Ariza&lt;/a&gt; to make it happen. I
+        ripped apart the README and rewrote just about all the sections.&lt;/p&gt;
+
+        &lt;p&gt;There was an open issue on the project about renaming the Gem and I knew that
+        launching the website and releasing 1.0 would be the perfect opportunity, so I
+        went for it. There was a suggestion to change the name to Feedzilla, but since
+        that is already a thing, I went with Feedjira. I bought the domain and setup an
+        organization by that name on GitHub.&lt;/p&gt;
+
+        &lt;p&gt;With those things in place, I needed to actually update the code for these
+        changes. I wanted to make this transition as easy as possible and devised a
+        simple way to use &lt;a href="/versions.html"&gt;three versions&lt;/a&gt; to make the jump to 1.0.&lt;/p&gt;
+
+        &lt;p&gt;For most users, upgrading to 1.0 should be a breeze, but I have an &lt;a href="/upgrading.html"&gt;upgrade
+        page&lt;/a&gt; to help with a couple details. If you have any trouble upgrading,
+        please let me know by &lt;a href="https://github.com/feedjira/feedjira/issues"&gt;opening an issue&lt;/a&gt;.&lt;/p&gt;
+
+        &lt;p&gt;There are still lots of things I&amp;rsquo;d like to do with this Gem. I mentioned seeing
+        things that I wanted to completely rewrite, so that&amp;rsquo;ll be something that I work
+        on for a 2.0 release, but that&amp;rsquo;s a ways off. I&amp;rsquo;d like to officially support
+        JRuby. Many people use Feedjira with Rails, so a separate project that helps
+        those users get up and running quickly seems to have value.&lt;/p&gt;
+
+        &lt;p&gt;The list goes on.&lt;/p&gt;
+
+        &lt;p&gt;I do have a request before I finish this thing: I&amp;rsquo;d like to hear from users that
+        have apps in production using Feedjira. If you&amp;rsquo;re using Feedjira for a
+        commercial app, please &lt;a href="feedjira@gmail.com"&gt;email me&lt;/a&gt;!&lt;/p&gt;
+
+        &lt;p&gt;Thanks to everyone who has helped me accomplish this, but especially &lt;a href="http://www.pauldix.net"&gt;Paul
+        Dix&lt;/a&gt; for creating such a fun project to work on, &lt;a href="http://danielariza.com"&gt;Daniel Ariza&lt;/a&gt; for a
+        badass website design and the many people who opened issues or sent pull
+        requests. Open source is fun to work on because of people like you!! &amp;lt;3 &amp;lt;3 &amp;lt;3&lt;/p&gt;
+        </content>
+          </entry>
+        </feed>
+    http_version: 
+  recorded_at: Mon, 31 Oct 2016 02:29:13 GMT
+recorded_with: VCR 3.0.3

data/lib/feedjira.rb

@@ -5,11 +5,16 @@ require 'sax-machine'
 require 'loofah'
 
 require 'feedjira/core_ext'
+require 'feedjira/date_time_utilities/date_time_pattern_parser'
+require 'feedjira/date_time_utilities/date_time_language_parser'
+require 'feedjira/date_time_utilities'
+require 'feedjira/date_time_utilities'
 require 'feedjira/feed_entry_utilities'
 require 'feedjira/feed_utilities'
 require 'feedjira/feed'
 require 'feedjira/parser'
 require 'feedjira/parser/rss_entry'
+require 'feedjira/parser/rss_image'
 require 'feedjira/parser/rss'
 require 'feedjira/parser/atom_entry'
 require 'feedjira/parser/atom'
@@ -18,13 +23,17 @@ require 'feedjira/version'
 
 require 'feedjira/parser/rss_feed_burner_entry'
 require 'feedjira/parser/rss_feed_burner'
+require 'feedjira/parser/podlove_chapter'
 require 'feedjira/parser/itunes_rss_owner'
+require 'feedjira/parser/itunes_rss_category'
 require 'feedjira/parser/itunes_rss_item'
 require 'feedjira/parser/itunes_rss'
 require 'feedjira/parser/atom_feed_burner_entry'
 require 'feedjira/parser/atom_feed_burner'
 require 'feedjira/parser/google_docs_atom_entry'
 require 'feedjira/parser/google_docs_atom'
+require 'feedjira/parser/atom_youtube_entry'
+require 'feedjira/parser/atom_youtube'
 
 module Feedjira
   class NoParserAvailable < StandardError; end

data/lib/feedjira/core_ext.rb

@@ -1,3 +1,3 @@
-require "feedjira/core_ext/time"
-require "feedjira/core_ext/date"
-require "feedjira/core_ext/string"
+require 'feedjira/core_ext/time'
+require 'feedjira/core_ext/date'
+require 'feedjira/core_ext/string'

data/lib/feedjira/core_ext/date.rb

@@ -8,10 +8,11 @@ class Date
   end
 
   def feed_utils_to_local_time
-    feed_utils_to_time(new_offset(DateTime.now.offset-offset), :local)
+    feed_utils_to_time(new_offset(DateTime.now.offset - offset), :local)
   end
 
   private
+
   def feed_utils_to_time(dest, method)
     Time.send(method, dest.year, dest.month, dest.day, dest.hour, dest.min,
               dest.sec, dest.zone)

data/lib/feedjira/core_ext/string.rb

@@ -1,6 +1,6 @@
 class String
   def sanitize!
-    self.replace(sanitize)
+    replace(sanitize)
   end
 
   def sanitize

data/lib/feedjira/core_ext/time.rb

@@ -1,5 +1,5 @@
-require "time"
-require "date"
+require 'time'
+require 'date'
 
 class Time
   # Parse a time string and convert it to UTC without raising errors.
@@ -11,21 +11,24 @@ class Time
   # === Returns
   # A Time instance in UTC or nil if there were errors while parsing.
   def self.parse_safely(dt)
-    if dt
-      case
-      when dt.is_a?(Time)
-        dt.utc
-      when dt.respond_to?(:empty?) && dt.empty?
-        nil
-      when dt.respond_to?(:to_datetime)
-        dt.to_datetime.utc
-      when dt.to_s =~ /\A\d{14}\z/
-        parse("#{dt.to_s}Z", true)
-      else
-        parse(dt.to_s, true).utc
-      end
+    if dt.is_a?(Time)
+      dt.utc
+    elsif dt.respond_to?(:to_datetime)
+      dt.to_datetime.utc
+    elsif dt.respond_to? :to_s
+      parse_string_safely dt.to_s
     end
   rescue StandardError
     nil
-  end unless method_defined?(:parse_safely)
+  end
+
+  def self.parse_string_safely(string)
+    return nil if string.empty?
+
+    if string =~ /\A\d{14}\z/
+      parse("#{string}Z", true)
+    else
+      parse(string).utc
+    end
+  end
 end

data/lib/feedjira/date_time_utilities.rb

@@ -0,0 +1,24 @@
+module Feedjira
+  module DateTimeUtilities
+    # This is our date parsing heuristic.
+    # Date Parsers are attempted in order.
+    DATE_PARSERS = [
+      DateTimePatternParser,
+      DateTimeLanguageParser,
+      DateTime
+    ].freeze
+
+    # Parse the given string starting with the most common parser (default ruby)
+    # and going over all other available parsers
+    def parse_datetime(string)
+      DATE_PARSERS.find do |parser|
+        begin
+          return parser.parse(string).feed_utils_to_gm_time
+        rescue
+          nil
+        end
+      end
+      warn "Failed to parse date #{string.inspect}"
+    end
+  end
+end

data/lib/feedjira/date_time_utilities/date_time_language_parser.rb

@@ -0,0 +1,22 @@
+module Feedjira
+  module DateTimeUtilities
+    class DateTimeLanguageParser
+      MONTHS_ENGLISH =
+        %w(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec).freeze
+      MONTHS_SPANISH =
+        %w(Ene Feb Mar Abr May Jun Jul Ago Sep Oct Nov Dic).freeze
+
+      def self.parse(string)
+        DateTime.parse(translate(string))
+      end
+
+      def self.translate(string)
+        MONTHS_SPANISH.each_with_index do |m, i|
+          rgx = Regexp.new("\s#{m}\s", Regexp::IGNORECASE)
+          return string.gsub(rgx, MONTHS_ENGLISH[i]) if string =~ rgx
+        end
+        raise "No translation found for #{string}"
+      end
+    end
+  end
+end

data/lib/feedjira/date_time_utilities/date_time_pattern_parser.rb

@@ -0,0 +1,29 @@
+module Feedjira
+  module DateTimeUtilities
+    class DateTimePatternParser
+      # rubocop:disable Style/AsciiComments
+      # Japanese Symbols are required for strange Date Strings like
+      # '水, 31 8 2016 07:37:00 PDT'
+      JAPANESE_SYMBOLS = %w(日 月 火 水 木 金 土).freeze
+      PATTERNS = ['%m/%d/%Y %T %p', '%d %m %Y %T %Z'].freeze
+
+      def self.parse(string)
+        PATTERNS.each do |p|
+          begin
+            datetime = DateTime.strptime(prepare(string), p)
+            return datetime
+          rescue
+            nil
+          end
+        end
+        raise "No pattern matched #{string}"
+      end
+
+      def self.prepare(string)
+        rgx = Regexp.new("^(#{JAPANESE_SYMBOLS.join('|')}),\s")
+        string.gsub(rgx, '')
+      end
+      private_class_method :prepare
+    end
+  end
+end