flying-sphinx 0.5.1 → 0.5.2

data/.travis.yml

@@ -0,0 +1,4 @@
+rvm:
+  - 1.8.7
+  - 1.9.2
+  - ree

data/HISTORY

@@ -0,0 +1,48 @@
+0.5.2 - 28th July 2011
+* Log SSH exceptions during indexing (when we're being verbose).
+* This history file now exists (pre-populated).
+* An actual README, courtesy of some prodding by Mislav.
+* Added support for wordforms (automatically sent through to the server).
+* Don't presume that ENV['DATABASE_URL'] exists - it won't in non-Heroku environments.
+
+0.5.1 - 23rd June 2011
+* Adding rake as a development dependency, just for Travis CI.
+* Have a default database port - Cedar stack doesn't set it by default.
+* Allow for newer net-ssh gem versions (including 2.1).
+* Sinatra loader (equivalent of a Railtie).
+* Fixed bug for handling JSON always as a Hash (sometimes it's an Array).
+* Better error checking when index requests don't get created as expected.
+
+0.5.0 - 12th May 2011
+* Rake tasks for latest actions and last index log.
+* Switching to version 2 of the API.
+* Adding default rake task for Travis CI.
+* Allow for Rails 2 versions of Delayed Job.
+* Set client_key for connections, configuration if supported by Riddle and Thinking Sphinx.
+* Allow server to automatically close SSH connections.
+* Adding logging on API calls (Josh Kalderimis).
+* Allow for non-tunnelled index requests (for RDS).
+* Don't use custom database adapter if the database is MySQL.
+* Fall back to environment variables for connection settings.
+* Switching from JSON and HTTParty to MultiJSON and Faraday (Josh Kalderimis).
+* Using Bundler as gem driver instead of Jeweler (Josh Kalderimis).
+* More flexible JSON dependency.
+
+0.4.4 - 7th February 2011
+* Using ActiveRecord's connection information, as we can't rely on heroku_env.yml to be around.
+
+0.4.3 - 4th February 2011
+* Wait until the SSH session and forward is prepared before making the index request.
+* If credentials are invalid, raise an appropriate error instead of letting the JSON parsing fail.
+
+0.4.2 - 29th January 2011
+* Set the Thinking Sphinx database adapter when loaded, instead of requiring the Rails dispatcher to setup.
+
+0.4.1 - 24th January 2011
+* Comparing against response bodies for consistency (and avoiding HTTParty magic).
+
+0.4.0 - 18th January 2011
+* Using HTTPS for API calls.
+* Using Flying Sphinx server's account identifier instead of the heroku id from the environment (the latter being unreliable and not part of any official Heroku add-on documentation).
+
+Note: Any releases before this were most definitely experimental and pre-alpha.

data/README.textile

@@ -1,3 +1,38 @@
 h1. Flying Sphinx
 
-Coming soon - well, with a bit of luck.
+This library is the code behind the @flying-sphinx@ gem. Its sole purpose is to connect Thinking Sphinx to the "Flying Sphinx Heroku addon":http://addons.heroku.com/flying_sphinx - and so you can just use Thinking Sphinx as normal on Heroku-hosted sites.
+
+h2. Installation and Usage
+
+This is all covered pretty well on "the Flying Sphinx site":http://flying-sphinx.com/docs, but if you're reading this on Github, you're probably keen on using this repository as the source instead of the gem. Here's the code snippet for your @Gemfile@:
+
+<pre><code>gem 'flying-sphinx',
+  :git => 'git://github.com/flying-sphinx/flying-sphinx.git'</code></pre>
+
+h2. Compatibility and Limitations
+
+Flying Sphinx is built with support for Rails 2.3.6 and beyond. Sadly, earlier versions of Rails aren't supported due to a requirement of a recent version of Rack deep in the dependency tree. It's not tested against Rails 3.1 at this time, but in theory should work fine. Let me know if I'm wrong!
+
+It works with Rails 1.8.7 and 1.9.2, and on both Heroku's Bamboo and Cedar stacks. You _do_ need to use it with Heroku at this point - allowing others hosting elsewhere to use Flying Sphinx is something I'm considering, but it's not possible right now.
+
+There are limitations with Heroku's setup and Sphinx/Thinking Sphinx functionality - again, the official documentation "covers this":http://flying-sphinx.com/docs#limitations, so have a read there.
+
+h2. Contributing
+
+Patches are indeed welcome - though I understand it's a little tricky to test them thoroughly. It's best to give your changes a spin on your Heroku app that's running Flying Sphinx (and if you don't have one of those, I'm a little confused as to why you're contributing).
+
+API documentation is required - consider this paragraph a placeholder for that.
+
+In the meantime, please keep in mind the following:
+
+* Write tests using RSpec
+* Keep your commits in a separate branch
+* Don't change the version or History file in your feature branch - makes merging changes much easier.
+
+h2. Contributors
+
+* "Josh Kalderimis":http://blog.cookiestack.com/ for switching the API calls to Faraday, improving the specs, a ton of debugging, and encouraging a versioned API.
+
+h2. Licence
+
+Copyright &copy; 2011 Pat Allan, released under an MIT licence.

data/flying-sphinx.gemspec

@@ -1,10 +1,9 @@
 # -*- encoding: utf-8 -*-
 Gem::Specification.new do |s|
   s.name        = 'flying-sphinx'
-  s.version     = '0.5.1'
+  s.version     = '0.5.2'
   s.authors     = ['Pat Allan']
   s.email       = 'pat@freelancing-gods.com'
-  s.date        = '2011-05-12'
   s.summary     = 'Sphinx in the Cloud'
   s.description = 'Hooks Thinking Sphinx into the Flying Sphinx service'
   s.homepage    = 'https://flying-sphinx.com'

data/lib/flying_sphinx/api.rb

@@ -74,6 +74,6 @@ class FlyingSphinx::API
   end
   
   def log?
-    ENV['VERBOSE_LOGGING'].present?
+    ENV['VERBOSE_LOGGING'] && ENV['VERBOSE_LOGGING'].length > 0
   end
 end

data/lib/flying_sphinx/configuration.rb

@@ -16,9 +16,20 @@ class FlyingSphinx::Configuration
   def sphinx_configuration
     thinking_sphinx.generate
     set_database_settings
+    set_wordforms
 
     riddle.render
   end
+  
+  def wordform_file_pairs
+    @wordform_file_pairs ||= begin
+      pairs = {}
+      wordform_sources.each_with_index do |source, index|
+        pairs[source] = "#{base_path}/wordforms/#{index}.txt"
+      end
+      pairs
+    end
+  end
 
   def start_sphinx
     api.post('start')
@@ -112,6 +123,19 @@ class FlyingSphinx::Configuration
       end
     end
   end
+  
+  def set_wordforms
+    riddle.indexes.each do |index|
+      next unless index.respond_to?(:wordforms)
+      index.wordforms = wordform_file_pairs[index.wordforms]
+    end
+  end
+  
+  def wordform_sources
+    @wordform_sources ||= riddle.indexes.collect { |index|
+      index.respond_to?(:wordforms) ? index.wordforms : nil
+    }.flatten.compact.uniq
+  end
 
   def identifier_from_env
     ENV['FLYING_SPHINX_IDENTIFIER']

data/lib/flying_sphinx/index_request.rb

@@ -30,6 +30,7 @@ class FlyingSphinx::IndexRequest
 
   def update_and_index
     update_sphinx_configuration
+    update_sphinx_reference_files
     index
   end
   
@@ -68,6 +69,15 @@ class FlyingSphinx::IndexRequest
   def update_sphinx_configuration
     api.put('/', :configuration => configuration.sphinx_configuration)
   end
+  
+  def update_sphinx_reference_files
+    configuration.wordform_file_pairs.each do |local, remote|
+      api.post '/add_file',
+        :setting   => 'wordforms',
+        :file_name => remote.split('/').last,
+        :content   => open(local).read
+    end
+  end
 
   def index
     if FlyingSphinx::Tunnel.required?
@@ -75,9 +85,10 @@ class FlyingSphinx::IndexRequest
     else
       direct_index
     end
-  rescue Net::SSH::Exception
+  rescue Net::SSH::Exception => err
     # Server closed the connection on us. That's (hopefully) expected, nothing
     # to worry about.
+    puts "SSH/Indexing Error: #{err.message}" if log?
   rescue RuntimeError => err
     puts err.message
   end
@@ -137,4 +148,8 @@ class FlyingSphinx::IndexRequest
   def api
     configuration.api
   end
+  
+  def log?
+    ENV['VERBOSE_LOGGING'] && ENV['VERBOSE_LOGGING'].length > 0
+  end
 end

data/lib/flying_sphinx/rails.rb

@@ -10,7 +10,7 @@ unless Rails.env.development? || Rails.env.test?
       config.client_key
   end
 
-  if ENV['DATABASE_URL'][/^mysql/].nil?
+  if ENV['DATABASE_URL'] && ENV['DATABASE_URL'][/^mysql/].nil?
     ThinkingSphinx.database_adapter = FlyingSphinx::HerokuSharedAdapter
   end
 end

data/lib/flying_sphinx/railtie.rb

@@ -11,7 +11,7 @@ class FlyingSphinx::Railtie < Rails::Railtie
     ThinkingSphinx::Configuration.instance.configuration.searchd.client_key =
       config.client_key
     
-    if ENV['DATABASE_URL'][/^mysql/].nil?
+    if ENV['DATABASE_URL'] && ENV['DATABASE_URL'][/^mysql/].nil?
       ThinkingSphinx.database_adapter = FlyingSphinx::HerokuSharedAdapter
     end
   end unless Rails.env.development? || Rails.env.test?

data/spec/specs/index_request_spec.rb

@@ -3,7 +3,8 @@ require 'spec_helper'
 describe FlyingSphinx::IndexRequest do
   let(:api)           { FlyingSphinx::API.new 'foo', 'bar' }
   let(:configuration) {
-    stub(:configuration, :api => api, :sphinx_configuration => 'foo {}')
+    stub(:configuration, :api => api, :sphinx_configuration => 'foo {}',
+      :wordform_file_pairs => {})
   }
   
   let(:index_response)    {
@@ -67,6 +68,32 @@ describe FlyingSphinx::IndexRequest do
       end
     end
     
+    context 'with wordforms' do
+      let(:file_params) {
+        {:setting => 'wordforms', :file_name => 'bar.txt', :content => 'baz'}
+      }
+      
+      before :each do
+        configuration.stub!(:wordform_file_pairs => {'foo.txt' => 'bar.txt'})
+        index_request.stub!(:open => double('file', :read => 'baz'))
+      end
+      
+      it "sends the wordform file" do
+        api.should_receive(:put).with('/', conf_params).and_return('ok')
+        api.should_receive(:post).with('/add_file', file_params).
+          and_return('ok')
+        api.should_receive(:post).
+          with('indices', index_params).and_return(index_response)
+        
+        begin
+          Timeout::timeout(0.2) {
+            index_request.update_and_index
+          }
+        rescue Timeout::Error
+        end
+      end
+    end
+    
     context 'delta request without delta support' do
       it "should explain why the request failed" do
         api.should_receive(:put).

metadata

@@ -2,7 +2,7 @@
 name: flying-sphinx
 version: !ruby/object:Gem::Version 
   prerelease: 
-  version: 0.5.1
+  version: 0.5.2
 platform: ruby
 authors: 
 - Pat Allan
@@ -10,7 +10,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-05-12 00:00:00 +10:00
+date: 2011-07-28 00:00:00 +10:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -166,7 +166,9 @@ extra_rdoc_files:
 - README.textile
 files: 
 - .gitignore
+- .travis.yml
 - Gemfile
+- HISTORY
 - LICENCE
 - README.textile
 - Rakefile