jsl-feedzirra 0.0.12.7 → 0.0.12.8

data/README.rdoc

@@ -3,50 +3,25 @@
 === Description
 
 Feedzirra is a feed library that is designed to get and update many feeds as quickly as possible. This includes using libcurl-multi through the 
-taf2-curb[link:http://github.com/taf2/curb/tree/master] gem for faster http gets, and libxml through nokogiri[link:http://github.com/tenderlove/nokogiri/tree/master] 
-and sax-machine[link:http://github.com/pauldix/sax-machine/tree/master] for faster parsing.
+taf2-curb[link:http://github.com/taf2/curb/tree/master] gem for faster http gets, and libxml through 
+nokogiri[link:http://github.com/tenderlove/nokogiri/tree/master] and sax-machine[link:http://github.com/pauldix/sax-machine/tree/master] for 
+faster parsing.
 
-It allows for easy customization of feed parsing options through the definition of custom parsing classes, and allows you to take as little or as much control as you want in updating feeds.  Feedzirra
-makes it easy to figure out which content in feeds is new by providing simple 'backends' so that Feedzirra can track the last contents fetched from a  particular feed.  Out of the box, Feedzirra can 
-store this information in the filesystem, Memcached or Tokyo Cabinet.  If you want to keep track of new or updated feeds on your own, just use the default backend which will will let you set options 
-for conditional fetching of feeds without the help of Feedzirra.
+It allows for easy customization of feed parsing options through the definition of custom parsing classes, and allows you to take as little or as 
+much control as you want in updating feeds.  Feedzirra makes it easy to figure out which content in feeds is new by storing the previous retrieval
+of a feed in a key-value store.  Feedzirra uses the the "moneta" gem, which is a unified interface to key-value storage systems, in order to provide
+access to many different types of stores depending on your requirements.
 
 === Installation
 
-For now Feedzirra exists only on github. It also has a few gem requirements that are only on github. Before you start you need to have libcurl[link:http://curl.haxx.se/] and 
-libxml[link:http://xmlsoft.org/] installed. If you're on Leopard you have both. Otherwise, you'll need to grab them. Once you've got those libraries, these are the gems that get 
-used: nokogiri, pauldix-sax-machine, taf2-curb (note that this is a fork that lives on github and not the Ruby Forge version of curb), and pauldix-feedzirra. The feedzirra gemspec has all 
-the dependencies so you should be able to get up and running with the standard github gem install routine:
+For now Feedzirra exists only on github. It also has a few gem requirements that are only on github. Before you start you need to have 
+libcurl[link:http://curl.haxx.se/] and libxml[link:http://xmlsoft.org/] installed. If you're on Leopard you have both. Otherwise, you'll need to 
+grab them. Once you've got those libraries, these are the gems that get used: nokogiri, pauldix-sax-machine, taf2-curb (note that this is a fork 
+that lives on github and not the Ruby Forge version of curb), and pauldix-feedzirra. The feedzirra gemspec has all the dependencies so you should 
+be able to get up and running with the standard github gem install routine:
 
-    gem sources -a http://gems.github.com # if you haven't already
-    gem install pauldix-feedzirra
-
-==== Troubleshooting Installation
-
-*NOTE:*Some people have been reporting a few issues related to installation. First, the Ruby Forge version of curb is not what you want. It will not work. Nor will the curl-multi gem that lives on 
-Ruby Forge. You have to get the taf2-curb[link:http://github.com/taf2/curb/tree/master] fork installed.
-
-If you see this error when doing a require: 
-
-    /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- curb_core (LoadError)
-
-It means that the taf2-curb gem didn't build correctly. To resolve this you can do a git clone git://github.com/taf2/curb.git then run rake gem in the curb directory, then sudo gem 
-install pkg/curb-0.2.4.0.gem. After that you should be good.
-
-If you see something like this when trying to run it:
-
-    NoMethodError: undefined method `on_success' for #<Curl::Easy:0x1182724>
-        from ./lib/feedzirra/feed.rb:88:in `add_url_to_multi'
-
-This means that you are requiring curl-multi or the Ruby Forge version of Curb somewhere. You can't use those and need to get the taf2 version up and running.
-
-If you're on Debian or Ubuntu and getting errors while trying to install the taf2-curb gem, it could be because you don't have the latest version of libcurl installed. Do this to fix:
-
-    sudo apt-get install libcurl4-gnutls-dev
-
-Another problem could be if you are running Mac Ports and you have libcurl installed through there. You need to uninstall it for curb to work! The version in Mac Ports is old and doesn't play nice with curb. If you're running Leopard, you can just uninstall and you should be golden. If you're on an older version of OS X, you'll then need to {download curl}[http://curl.haxx.se/download.html] and build from source. Then you'll have to install the taf2-curb gem again. You might have to perform the step above.
-
-If you're still having issues, please let me know on the mailing list. Also, {Todd Fisher (taf2)}[link:http://github.com/taf2] is working on fixing the gem install. Please send him a full error report.
+  gem sources -a http://gems.github.com # if you haven't already
+  gem install pauldix-feedzirra
 
 === Usage
 
@@ -64,7 +39,7 @@ allows configuration of the backend store, as well as fetching options for the l
 an example of configuration with the Memcache store connected to Tokyo Tyrant (the front-end for Tokyo Cabinet):
 
 	reader = Feedzirra::Reader.new('http://www.pauldix.net/atom.xml', :backend => 
-		{ :class => Feedzirra::Backend::Memcache, :port => 1978, :server => 'localhost' })
+		{ :moneta_klass => Moneta::Memcache, :port => 1978, :server => 'localhost' })
 
 Other options that may be put in the options hash follow the original API described below.
 
@@ -150,13 +125,43 @@ There's also a {benchmark that shows the results of using Feedzirra to perform u
 
 === Discussion
 
-I'd like feedback on the api and any bugs encountered on feeds in the wild. I've set up a {google group here}[http://groups.google.com/group/feedzirra].
+I'd like feedback on the api and any bugs encountered on feeds in the wild. I've set up a 
+{google group here}[http://groups.google.com/group/feedzirra].
+
+==== Troubleshooting Installation
+
+*NOTE:*Some people have been reporting a few issues related to installation. First, the Ruby Forge version of curb is not what you want. It will not work. Nor will the curl-multi gem that lives on 
+Ruby Forge. You have to get the taf2-curb[link:http://github.com/taf2/curb/tree/master] fork installed.
+
+If you see this error when doing a require: 
+
+    /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- curb_core (LoadError)
+
+It means that the taf2-curb gem didn't build correctly. To resolve this you can do a git clone git://github.com/taf2/curb.git then run rake gem in the curb directory, then sudo gem 
+install pkg/curb-0.2.4.0.gem. After that you should be good.
+
+If you see something like this when trying to run it:
+
+    NoMethodError: undefined method `on_success' for #<Curl::Easy:0x1182724>
+        from ./lib/feedzirra/feed.rb:88:in `add_url_to_multi'
+
+This means that you are requiring curl-multi or the Ruby Forge version of Curb somewhere. You can't use those and need to get the taf2 version up and running.
+
+If you're on Debian or Ubuntu and getting errors while trying to install the taf2-curb gem, it could be because you don't have the latest version of libcurl installed. Do this to fix:
+
+    sudo apt-get install libcurl4-gnutls-dev
+
+Another problem could be if you are running Mac Ports and you have libcurl installed through there. You need to uninstall it for curb to work! The version in Mac Ports is old and doesn't play nice with curb. If you're running Leopard, you can just uninstall and you should be golden. If you're on an older version of OS X, you'll then need to {download curl}[http://curl.haxx.se/download.html] and build from source. Then you'll have to install the taf2-curb gem again. You might have to perform the step above.
+
+If you're still having issues, please let me know on the mailing list. Also, {Todd Fisher (taf2)}[link:http://github.com/taf2] is working on fixing the gem install. Please send him a full error report.
 
 === TODO
 
-This thing needs to hammer on many different feeds in the wild. I'm sure there will be bugs. I want to find them and crush them. I didn't bother using the test suite for feedparser. i wanted to start fresh.
+This thing needs to hammer on many different feeds in the wild. I'm sure there will be bugs. I want to find them and crush them. I didn't bother 
+using the test suite for feedparser. i wanted to start fresh.
 
 Here are some more specific TODOs.
+
 * Make a feedzirra-rails gem to integrate feedzirra seamlessly with Rails and ActiveRecord.
 * Add support for authenticated feeds.
 * Create a super sweet DSL for defining new parsers.

data/lib/feedzirra.rb

@@ -7,20 +7,18 @@ require 'curb'
 require 'sax-machine'
 require 'dryopteris'
 require 'uri'
+
 require 'active_support/basic_object'
 require 'active_support/core_ext/object'
 require 'active_support/core_ext/time'
 
+require 'hashback'
+
 require 'core_ext/date'
 require 'core_ext/string'
 require 'core_ext/array'
 require 'core_ext/hash'
 
-require 'feedzirra/backend/dev_null'
-require 'feedzirra/backend/filesystem'
-require 'feedzirra/backend/memcache'
-require 'feedzirra/backend/memory'
-
 require 'feedzirra/http_multi'
 
 require 'feedzirra/parser/feed_utilities'

data/lib/feedzirra/http_multi.rb

@@ -8,16 +8,19 @@ module Feedzirra
 
     DEFAULTS = {
       :backend => {
-        :class => Feedzirra::Backend::Memory 
+        :moneta_klass => 'Moneta::Memory' 
       }  
     }
     
     def initialize(*args)
-      @options          = DEFAULTS.merge(args.extract_options!)
-      @retrievables     = args.flatten
-      @multi            = Curl::Multi.new
-      @responses        = { }
-      @backend          = @options[:backend][:class].new(@options[:backend].except(:class))
+      @options        = DEFAULTS.merge(args.extract_options!)
+      @retrievables   = args.flatten
+      @multi          = Curl::Multi.new
+      @responses      = { }
+      
+      @backend        = HashBack::Backend.new( 'Feedzirra', 
+                          @options[:backend][:moneta_klass], 
+                          @options[:backend].except(:moneta_klass) )
     end
 
     # Prepares the curl object and calls #perform
@@ -43,7 +46,7 @@ module Feedzirra
         url = retrievable.feed_url
       else
         url = retrievable
-        retrievable = @backend.get(url)  # Try to fetch the last retrieval from backend
+        retrievable = @backend[url]  # Try to fetch the last retrieval from backend
       end
 
       easy = build_curl_easy(url, retrievable, retrievable_queue)
@@ -97,7 +100,7 @@ module Feedzirra
           set_updated_feed_entries!(retrievable, updated_feed)
         end
 
-        @backend.set(url, updated_feed)          
+        @backend[url] = updated_feed
 
         responses[url] = updated_feed
         @options[:on_success].call(retrievable) if @options.has_key?(:on_success)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: jsl-feedzirra
 version: !ruby/object:Gem::Version 
-  version: 0.0.12.7
+  version: 0.0.12.8
 platform: ruby
 authors: 
 - Paul Dix
@@ -12,6 +12,16 @@ cert_chain: []
 date: 2009-04-29 00:00:00 -07:00
 default_executable: 
 dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: hashback
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">"
+      - !ruby/object:Gem::Version 
+        version: 0.0.0
+    version: 
 - !ruby/object:Gem::Dependency 
   name: nokogiri
   type: :runtime
@@ -111,10 +121,6 @@ files:
 - lib/feedzirra/parser/rss_entry.rb
 - lib/feedzirra/parser/feed_utilities.rb
 - lib/feedzirra/parser/feed_entry_utilities.rb
-- lib/feedzirra/backend/filesystem.rb
-- lib/feedzirra/backend/dev_null.rb
-- lib/feedzirra/backend/memcache.rb
-- lib/feedzirra/backend/memory.rb
 - README.rdoc
 - Rakefile
 - spec/spec.opts
@@ -131,10 +137,6 @@ files:
 - spec/feedzirra/parser/rss_entry_spec.rb
 - spec/feedzirra/feed_utilities_spec.rb
 - spec/feedzirra/feed_entry_utilities_spec.rb
-- spec/feedzirra/backend/dev_null_spec.rb
-- spec/feedzirra/backend/filesystem_spec.rb
-- spec/feedzirra/backend/memcache_spec.rb
-- spec/feedzirra/backend/memory_spec.rb
 has_rdoc: true
 homepage: http://github.com/pauldix/feedzirra
 post_install_message: 

data/lib/feedzirra/backend/dev_null.rb

@@ -1,21 +0,0 @@
-module Feedzirra
-  module Backend
-    
-    # Class exists for cases where you don't want Feedzirra to remember what 
-    # it has fetched.  If this backend is selected, user needs to pass arguments
-    # in the form of a Feed object to the Reader class to help Feedzirra know when
-    # a page hasn't changed, and which feed entries have been updated.
-    class DevNull
-      
-      def initialize(options = { })
-      end
-
-      def get(url)
-      end
-    
-      def set(url, result) 
-      end
-      
-    end
-  end
-end

data/lib/feedzirra/backend/filesystem.rb

@@ -1,37 +0,0 @@
-require 'md5'
-require 'uri'
-
-module Feedzirra
-  module Backend
-    class Filesystem
-      
-      DEFAULTS = {
-        :path => File.expand_path(File.join(%w[ ~ / .feedzirra ]))
-      }
-      
-      def initialize(options = { })
-        @options = DEFAULTS.merge(options)
-        initialize_store
-      end
-      
-      def get(url)
-        f = filename_for(url)
-        Marshal.load(File.read(f)) if File.exist?(f)
-      end
-    
-      def set(url, result)
-        File.open(filename_for(url), 'w') {|f| f.write(Marshal.dump(result)) }
-      end
-      
-      private
-      
-      def initialize_store
-        FileUtils.mkdir(@options[:path]) unless File.exist?(@options[:path])
-      end
-      
-      def filename_for(url)
-        File.join(@options[:path], MD5.hexdigest(URI.parse(url).normalize.to_s))
-      end
-    end
-  end
-end

data/lib/feedzirra/backend/memcache.rb

@@ -1,37 +0,0 @@
-require 'memcache'
-
-module Feedzirra
-  module Backend
-    
-    # Can be used to set up Memcache, or clients able to speak the Memcache protocol such as 
-    # Tokyo Tyrant, as a Feedzirra::Backend.
-    class Memcache
-      DEFAULTS = {
-        :server => 'localhost',
-        :port   => '11211'
-      }
-      
-      def initialize(options = { })
-        @options  = DEFAULTS.merge(options)
-        @cache    = MemCache.new([ @options[:server], @options[:port] ].join(':'), :namespace => 'Feedzirra')
-      end
-      
-      def get(url)
-        res = @cache.get(key_for(url))
-        Marshal.load(res) unless res.nil?
-      end
-    
-      def set(url, result)
-        @cache.set(key_for(url), Marshal.dump(result))
-      end
-      
-      private
-      
-      def key_for(url)
-        MD5.hexdigest(URI.parse(url).normalize.to_s)
-      end
-
-    end
-    
-  end
-end

data/lib/feedzirra/backend/memory.rb

@@ -1,23 +0,0 @@
-module Feedzirra
-  module Backend
-    
-    # Memory store uses a ruby Hash to store the results of feed fetches.
-    # It won't persist after the application exits.
-    class Memory
-      
-      def initialize(options = { })
-        @store   = { }
-        @options = options
-      end
-      
-      def get(url)
-        @store[url]
-      end
-    
-      def set(url, result)
-        @store[url] = result
-      end
-      
-    end
-  end
-end

data/spec/feedzirra/backend/dev_null_spec.rb

@@ -1,35 +0,0 @@
-require File.join(File.dirname(__FILE__), %w[.. .. spec_helper])
-
-describe Feedzirra::Backend::DevNull do
-  before do
-    @backend = Feedzirra::Backend::DevNull.new
-  end
-  
-  it_should_behave_like "all backends"
-  
-  it "should initialize properly" do
-    Feedzirra::Backend::DevNull.new
-  end
-  
-  describe "#set" do
-    it "should accept two arguments" do
-      lambda {
-        @backend.set('foo', 'nothing') 
-      }.should_not raise_error
-    end
-  end
-  
-  describe "#get" do
-    it "should accept one argument" do
-      lambda {
-        @backend.get('foo') 
-      }.should_not raise_error
-    end
-    
-    it "should return nil even after something is set" do
-      @backend.set('junk', 'something')
-      @backend.get('junk').should be_nil
-    end
-  end
-  
-end

data/spec/feedzirra/backend/filesystem_spec.rb

@@ -1,31 +0,0 @@
-require File.join(File.dirname(__FILE__), %w[.. .. spec_helper])
-
-describe Feedzirra::Backend::Filesystem do
-  before do
-    @backend = Feedzirra::Backend::Filesystem.new({:path => '/tmp'})
-  end
-  
-  it_should_behave_like "all backends"
-  
-  it "should accept a Hash of options for inialization" do
-    Feedzirra::Backend::Filesystem.new({:path => '/tmp'})
-  end
-  
-  describe "#filename_for" do
-    it "should default to the users' home path plus the MD5 of the normalized URL" do
-      MD5.expects(:hexdigest).returns('bah')
-      uri = mock('uri')
-      uri.expects(:normalize).returns('foo')
-      URI.expects(:parse).returns(uri)
-      Feedzirra::Backend::Filesystem.new({:path => '/tmp'}).__send__(:filename_for, 'hey').should == '/tmp/bah'
-    end
-  end
-  
-  describe "#initialize_store" do
-    it "should call File.mkdir if the directory does not exist" do
-      File.expects(:exist?).with('/tmp').returns(false)
-      FileUtils.expects(:mkdir).with('/tmp')
-      @backend.__send__(:initialize_store)
-    end
-  end
-end

data/spec/feedzirra/backend/memcache_spec.rb

@@ -1,25 +0,0 @@
-require File.join(File.dirname(__FILE__), %w[.. .. spec_helper])
-
-describe Feedzirra::Backend::Memcache do
-  before do
-    @backend = Feedzirra::Backend::Memcache.new({:server => 'localhost', :port => 11211})
-    @cache = mock('cache')
-    @backend.instance_variable_set(:@cache, @cache)
-  end
-  
-  it_should_behave_like "all backends"  
-
-  describe "#get" do
-    it "should call #get on the cache" do
-      @cache.expects(:get)
-      @backend.get('foo')
-    end    
-  end
-  
-  describe "#set" do  
-    it "should call #set on the cache" do
-      @cache.expects(:set)
-      @backend.set('foo', 'hey')
-    end
-  end
-end

data/spec/feedzirra/backend/memory_spec.rb

@@ -1,14 +0,0 @@
-require File.join(File.dirname(__FILE__), %w[.. .. spec_helper])
-
-describe Feedzirra::Backend::Memory do
-  before do
-    @backend = Feedzirra::Backend::Memory.new
-  end
-  
-  it_should_behave_like "all backends"  
-  
-  it "should initialize properly" do
-    Feedzirra::Backend::Memory.new
-  end
-  
-end