feedjira 0.9.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 1579ca4b98963d52648b62e13ce1100d09744a1b
+  data.tar.gz: f3fee2d061ffb4ad9bf54b363963abd1ac313970
+SHA512:
+  metadata.gz: 78294bd172b74e4f9e784bbabd7bfee8243f22457f16ca6946eeb12dc762b90252d0289480c04aae42727346718a8b0d41d077f5867e7a381c71796245fc68aa
+  data.tar.gz: febf23842faa2a597f5fdcae9e0c470d8d948a00e6d592115af82dc0dbb10baacee12678c02567e7d2d2fb49e8a6d12613676c9de21d4f8d5b1213222aeae0e9

data/.gitignore

@@ -0,0 +1,14 @@
+.DS_Store
+.rvmrc
+.ruby-version
+.ruby-gemset
+TODO
+Gemfile.lock
+rdoc/
+doc/
+pkg/
+coverage/
+.rbx
+.bundle
+*.swp
+*.swo

data/.rspec

@@ -0,0 +1 @@
+--color

data/.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+bundler_args: --without tools
+
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - 2.0.0
+  - rbx

data/CHANGELOG.md

@@ -0,0 +1,162 @@
+# Feedjira Changelog
+
+## 0.9.0
+
+* Project renamed to Feedjira
+
+## 0.7.1
+
+* Bugfix
+  * Don't use entry id for updating when feed doesn't provide it [[#205][]]
+
+[#205]: https://github.com/pauldix/feedzirra/pull/205
+
+## 0.7.0
+
+* General
+  * README update for callback arity [[#202][]]
+
+* Enhancements
+  * Add error info to `on_failure` callback [[#194][]]
+  * On failure callbacks get curl and error as args
+  * Bugfix for parsing dates that are ISO 8601 with milliseconds [[#203][]]
+
+[#194]: https://github.com/pauldix/feedzirra/pull/194
+[#202]: https://github.com/pauldix/feedzirra/pull/202
+[#203]: https://github.com/pauldix/feedzirra/pull/203
+
+## 0.6.0
+
+* General
+  * Update expected parser classes in docs [[#200][]]
+  * Fix Rubinius issue with Travis
+
+* Enhancements
+  * Added content to `itunes_rss_item` [[#198][]]
+  * Allow user to pass a particular parser using `parse_with`
+  * Strip leading whitespace from XML [[#196][]]
+  * Parse out RSS version [[#172][]]
+  * Add generic preprocessing hook for Parsers
+  * Add preprocessing hook for Atom XHTML content [[#58][]] [[#130][]]
+
+[#58]: https://github.com/pauldix/feedzirra/pull/58
+[#130]: https://github.com/pauldix/feedzirra/issues/130
+[#172]: https://github.com/pauldix/feedzirra/issues/172
+[#196]: https://github.com/pauldix/feedzirra/pull/196
+[#198]: https://github.com/pauldix/feedzirra/pull/198
+[#200]: https://github.com/pauldix/feedzirra/pull/200
+
+## 0.5.0
+
+* General
+  * Lots of README cleanup
+  * Remove pending specs
+  * Rewrite benchmarks and move them out of the spec folder
+  * Upgrade to latest Rspec
+
+* Enhancements
+  * Allow spaces in rss tag when checking parse-ability [[#127][]]
+  * Compare `entry_id` and `url` for finding new entries [[#195][]]
+  * Add closed captioned and order tags for iTunesRSSItem [[#160][]]
+
+[#127]: https://github.com/pauldix/feedzirra/pull/127
+[#160]: https://github.com/pauldix/feedzirra/pull/160
+[#195]: https://github.com/pauldix/feedzirra/pull/195
+
+## 0.4.0
+
+* Enhancements
+  * Raise when parser invokes its failure callback [[#159][]]
+  * Add PubSubHubbub hub urls as feed element [[#138][]]
+  * Add support for iTunes image in iTunes RSS item [[#164][]]
+
+* Bug fixes
+  * Use curb callbacks rather than response codes [[#161][]]
+
+[#138]: https://github.com/pauldix/feedzirra/pull/138
+[#159]: https://github.com/pauldix/feedzirra/issues/159
+[#161]: https://github.com/pauldix/feedzirra/pull/161
+[#164]: https://github.com/pauldix/feedzirra/pull/164
+
+## 0.3.0
+
+* General
+  * Add CodeClimate badge [[#192][]]
+
+* Enhancements
+  * CURL SSL Version option [[#156][]]
+  * Cookie support for Curb [[#98][]]
+
+* Deprecations
+  * For `ITunesRSSItem`, use `id` instead of `guid` [[#169][]]
+
+[#98]: https://github.com/pauldix/feedzirra/pull/98
+[#156]: https://github.com/pauldix/feedzirra/pull/156
+[#169]: https://github.com/pauldix/feedzirra/pull/169
+[#192]: https://github.com/pauldix/feedzirra/pull/192
+
+## 0.2.2
+
+* General
+  * Switch to CHANGELOG
+  * Set LICENSE in gemspec
+  * Lots of whitespace cleaning
+  * README updates
+
+* Enhancements
+  * Also use dc:identifier for `entry_id` [[#182][]]
+
+* Bug fixes
+  * Don't try to sanitize non-existent elements [[#174][]]
+  * Fix Rspec deprecations [[#188][]]
+  * Fix Travis [[#191][]]
+
+[#174]: https://github.com/pauldix/feedzirra/pull/174
+[#182]: https://github.com/pauldix/feedzirra/pull/182
+[#188]: https://github.com/pauldix/feedzirra/pull/188
+[#191]: https://github.com/pauldix/feedzirra/pull/191
+
+## 0.2.1
+
+* Use `Time.parse_safely` in `Feed.last_modified_from_header` [[#129][]].
+* Added image to the RSS Entry Parser [[#103][]].
+* Compatibility fixes for Ruby 2.0 [[#136][]].
+* Remove gorillib dependency [[#113][]].
+
+[#103]: https://github.com/pauldix/feedzirra/pull/103
+[#113]: https://github.com/pauldix/feedzirra/pull/113
+[#129]: https://github.com/pauldix/feedzirra/pull/129
+[#136]: https://github.com/pauldix/feedzirra/pull/136
+
+## 0.2.0.rc2
+
+* Bump sax-machine to `v0.2.0.rc1`, fixes encoding issues [[#76][]].
+
+[#76]: https://github.com/pauldix/feedzirra/issues/76
+
+## 0.2.0.rc1
+
+* Remove ActiveSupport dependency
+  * No longer tethered to any version of Rails!
+* Update curb (v0.8.0) and rspec (v2.10.0)
+* Revert [3008ceb][]
+* Add Travis-CI integration
+* General repository and gem maintenance
+
+[3008ceb]: https://github.com/pauldix/feedzirra/commit/3008ceb338df1f4c37a211d0aab8a6ad4f584dbc
+
+## 0.1.3
+
+* ?
+
+## 0.1.2
+
+* ?
+
+## 0.1.1
+
+* make FeedEntries enumerable (patch by Daniel Gregoire)
+
+## 0.1.0
+
+* lower builder requirement to make it rails-3 friendly

data/Gemfile

@@ -0,0 +1,17 @@
+source 'https://rubygems.org/'
+
+gemspec
+
+group :development, :test do
+  gem 'rake'
+end
+
+group :tools do
+  gem 'guard-rspec'
+  gem 'simplecov', :require => false, :platforms => :mri_19
+end
+
+platforms :rbx do
+  gem 'racc'
+  gem 'rubysl'
+end

data/Guardfile

@@ -0,0 +1,5 @@
+guard 'rspec' do
+  watch(%r{^spec/.+_spec\.rb$})
+  watch(%r{^lib/(.+)\.rb$})     { |m| "spec/#{m[1]}_spec.rb" }
+  watch('spec/spec_helper.rb')  { "spec" }
+end

data/README.md

@@ -0,0 +1,242 @@
+# Feedjira [![Build Status][travis-badge]][travis] [![Code Climate][code-climate-badge]][code-climate]
+
+[travis-badge]: https://secure.travis-ci.org/feedjira/feedjira.png
+[travis]: http://travis-ci.org/feedjira/feedjira
+[code-climate-badge]: https://codeclimate.com/github/feedjira/feedjira.png
+[code-climate]: https://codeclimate.com/github/feedjira/feedjira
+
+I'd like feedback on the api and any bugs encountered on feeds in the wild. I've
+set up a [google group][].
+
+[google group]: http://groups.google.com/group/feedjira
+
+## Description
+
+Feedjira is a feed library that is designed to get and update many feeds as
+quickly as possible. This includes using libcurl-multi through the [curb][] gem
+for faster http gets, and libxml through [nokogiri][] and [sax-machine][] for
+faster parsing.  Feedjira requires at least Ruby 1.9.2.
+
+[curb]: https://github.com/taf2/curb
+[nokogiri]: https://github.com/sparklemotion/nokogiri
+[sax-machine]: https://github.com/pauldix/sax-machine
+
+Once you have fetched feeds using Feedjira, they can be updated using the feed
+objects. Feedjira automatically inserts etag and last-modified information from
+the http response headers to lower bandwidth usage, eliminate unnecessary
+parsing, and make things speedier in general.
+
+Another feature present in Feedjira is the ability to create callback functions
+that get called "on success" and "on failure" when getting a feed. This makes it
+easy to do things like log errors or update data stores.
+
+The fetching and parsing logic have been decoupled so that either of them can be
+used in isolation if you'd prefer not to use everything that Feedjira offers.
+However, the code examples below use helper methods in the Feed class that put
+everything together to make things as simple as possible.
+
+The final feature of Feedjira is the ability to define custom parsing classes.
+In truth, Feedjira could be used to parse much more than feeds. Microformats,
+page scraping, and almost anything else are fair game.
+
+## Speedup date parsing
+
+In MRI before 1.9.3 the date parsing code was written in Ruby and was optimized
+for readability over speed, to speed up this part you can install the
+[home_run][] gem to replace it with an optimized C version. In most cases, if
+you are using Ruby 1.9.3+, you will not need to use home\_run.
+
+[home_run]: https://github.com/jeremyevans/home_run
+
+## Usage
+
+[A gist of the following code](http://gist.github.com/57285)
+
+```ruby
+require 'feedjira'
+
+# fetching a single feed
+feed = Feedjira::Feed.fetch_and_parse("http://feeds.feedburner.com/PaulDixExplainsNothing")
+
+# feed and entries accessors
+feed.title          # => "Paul Dix Explains Nothing"
+feed.url            # => "http://www.pauldix.net"
+feed.feed_url       # => "http://feeds.feedburner.com/PaulDixExplainsNothing"
+feed.etag           # => "GunxqnEP4NeYhrqq9TyVKTuDnh0"
+feed.last_modified  # => Sat Jan 31 17:58:16 -0500 2009 # it's a Time object
+
+entry = feed.entries.first
+entry.title      # => "Ruby Http Client Library Performance"
+entry.url        # => "http://www.pauldix.net/2009/01/ruby-http-client-library-performance.html"
+entry.author     # => "Paul Dix"
+entry.summary    # => "..."
+entry.content    # => "..."
+entry.published  # => Thu Jan 29 17:00:19 UTC 2009 # it's a Time object
+entry.categories # => ["...", "..."]
+
+# sanitizing an entry's content
+entry.title.sanitize    # => returns the title with harmful stuff escaped
+entry.author.sanitize   # => returns the author with harmful stuff escaped
+entry.content.sanitize  # => returns the content with harmful stuff escaped
+entry.content.sanitize! # => returns content with harmful stuff escaped and replaces original (also exists for author and title)
+entry.sanitize!         # => sanitizes the entry's title, author, and content in place (as in, it changes the value to clean versions)
+feed.sanitize_entries!  # => sanitizes all entries in place
+
+# updating a single feed
+updated_feed = Feedjira::Feed.update(feed)
+
+# an updated feed has the following extra accessors
+updated_feed.updated?     # returns true if any of the feed attributes have been modified. will return false if no new entries
+updated_feed.new_entries  # a collection of the entry objects that are newer than the latest in the feed before update
+
+# fetching multiple feeds
+feed_urls = ["http://feeds.feedburner.com/PaulDixExplainsNothing", "http://feeds.feedburner.com/trottercashion"]
+feeds = Feedjira::Feed.fetch_and_parse(feed_urls)
+
+# feeds is now a hash with the feed_urls as keys and the parsed feed objects as values. If an error was thrown
+# there will be a Fixnum of the http response code instead of a feed object
+
+# updating multiple feeds. it expects a collection of feed objects
+updated_feeds = Feedjira::Feed.update(feeds.values)
+
+# defining custom behavior on failure or success. note that a return status of 304 (not updated) will call the on_success handler
+feed = Feedjira::Feed.fetch_and_parse("http://feeds.feedburner.com/PaulDixExplainsNothing",
+	:on_success => lambda {|url, feed| puts feed.title },
+	:on_failure => lambda {|curl, error| puts error })
+
+# if a collection was passed into fetch_and_parse, the handlers will be called for each one
+
+# the behavior for the handlers when using Feedjira::Feed.update is slightly different. The feed passed into on_success will be
+# the updated feed with the standard updated accessors. on failure it will be the original feed object passed into update
+
+# fetching a feed via a proxy (optional)
+feed = Feedjira::Feed.fetch_and_parse("http://feeds.feedburner.com/PaulDixExplainsNothing", {:proxy_url => '10.0.0.1', :proxy_port => 3084})
+```
+
+## Extending
+
+### Adding a feed parsing class
+
+When determining which parser to use for a given XML document, the following
+list of parser classes is used:
+
+* `Feedjira::Parser::RSSFeedBurner`
+* `Feedjira::Parser::GoogleDocsAtom`
+* `Feedjira::Parser::AtomFeedBurner`
+* `Feedjira::Parser::Atom`
+* `Feedjira::Parser::ITunesRSS`
+* `Feedjira::Parser::RSS`
+
+You can insert your own parser at the front of this stack by calling
+`add_feed_class`, like this:
+
+```ruby
+Feedjira::Feed.add_feed_class MyAwesomeParser
+```
+
+Now when you `fetch_and_parse`, `MyAwesomeParser` will be the first one to get a
+chance to parse the feed.
+
+If you have the XML and just want to provide a parser class for one parse, you
+can specify that using `parse_with`:
+
+```ruby
+Feedjira::Feed.parse_with MyAwesomeParser, xml
+```
+
+### Adding attributes to all feeds types / all entries types
+
+```ruby
+# Add the generator attribute to all feed types
+Feedjira::Feed.add_common_feed_element('generator')
+Feedjira::Feed.fetch_and_parse("href="http://www.pauldix.net/atom.xml").generator # => 'TypePad'
+
+# Add some GeoRss information
+Feedjira::Feed.add_common_feed_entry_element('geo:lat', :as => :lat)
+Feedjira::Feed.fetch_and_parse("http://www.earthpublisher.com/georss.php").entries.each do |e|
+  p "lat: #[e.lat}, long: #{e.long]"
+end
+```
+
+### Adding attributes to only one class
+
+If you want to add attributes for only one class you simply have to declare them
+in the class
+
+```ruby
+# Add some GeoRss information
+require 'lib/feedjira/parser/rss_entry'
+
+class Feedjira::Parser::RSSEntry
+  element 'geo:lat', :as => :lat
+  element 'geo:long', :as => :long
+end
+
+# Fetch a feed containing GeoRss info and print them
+Feedjira::Feed.fetch_and_parse("http://www.earthpublisher.com/georss.php").entries.each do |e|
+  p "lat: #{e.lat}, long: #{e.long}"
+end
+```
+
+## Testing
+
+Feedjira uses [curb][] to perform requests. `curb` provides bindings for
+[libcurl][] and supports numerous protocols, including FILE. To test Feedjira
+with local file use `file://` protocol:
+
+[libcurl]: http://curl.haxx.se/libcurl/
+
+```ruby
+feed = Feedjira::Feed.fetch_and_parse('file:///home/feedjira/examples/feed.rss')
+```
+
+## Benchmarks
+
+Since a major goal of Feedjira is speed, benchmarks are provided--see the
+[Benchmark README][benchmark_readme] for more details.
+
+[benchmark_readme]: https://github.com/feedjira/feedjira/blob/master/benchmarks/README.md
+
+## TODO
+
+This thing needs to hammer on many different feeds in the wild. I'm sure there
+will be bugs. I want to find them and crush them. I didn't bother using the test
+suite for feedparser. i wanted to start fresh.
+
+Here are some more specific TODOs.
+
+* Make a feedjira-rails gem to integrate feedjira seamlessly with Rails and ActiveRecord.
+* Add support for authenticated feeds.
+* Create a super sweet DSL for defining new parsers.
+* I'm not keeping track of modified on entries. Should I add this?
+* Clean up the fetching code inside feed.rb so it doesn't suck so hard.
+* Make the feed_spec actually mock stuff out so it doesn't hit the net.
+* Readdress how feeds determine if they can parse a document. Maybe I should use namespaces instead?
+
+## LICENSE
+
+(The MIT License)
+
+Copyright (c) 2009-2013:
+
+- [Paul Dix](http://pauldix.net)
+- [Julien Kirch](http://archiloque.net/)
+- [Ezekiel Templin](http://zeke.templ.in/)
+- [Jon Allured](http://jonallured.com/)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the 'Software'), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.