thinking-sphinx 1.2.13 → 6.0.0

data/Gemfile

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'logger'
+gem 'mysql2', '~> 0.5.0',  :platform => :ruby
+gem 'pg',     '~> 0.18.4', :platform => :ruby
+
+gem 'activerecord', '< 7' if RUBY_VERSION.to_f <= 2.4
+
+if RUBY_PLATFORM == 'java'
+  gem 'jdbc-mysql',                          '5.1.35',    :platform => :jruby
+  gem 'activerecord-jdbcmysql-adapter',      '>= 1.3.23', :platform => :jruby
+  gem 'activerecord-jdbcpostgresql-adapter', '>= 1.3.23', :platform => :jruby
+  gem 'activerecord', '>= 3.2.22'
+end

data/LICENCE

@@ -1,4 +1,4 @@
-Copyright (c) 2008 Pat Allan
+Copyright (c) 2011 Pat Allan
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/Procfile.support

@@ -0,0 +1,2 @@
+postgres: postgres -D data/postgres -p ${POSTGRES_PORT:-5432}
+mysql: $(brew --prefix mysql@5.7)/bin/mysqld --datadir=$(PWD)/data/mysql --port ${MYSQL_PORT:-3306} --socket=mysql.sock

data/README.textile

@@ -1,144 +1,99 @@
 h1. Thinking Sphinx
 
+Thinking Sphinx is a library for connecting ActiveRecord to the Sphinx full-text search tool, and integrates closely with Rails (but also works with other Ruby web frameworks). The current release is v6.0.0.
+
+h2. Upgrading
+
+Please refer to "the changelog":https://github.com/pat/thinking-sphinx/blob/develop/CHANGELOG.markdown and "release notes":https://github.com/pat/thinking-sphinx/releases for any changes you need to make when upgrading. The release notes in particular are quite good at covering breaking changes and more details for new features.
+
+The documentation also has more details on what's involved for upgrading from "v4 to v5":https://freelancing-gods.com/thinking-sphinx/v5/upgrading.html, "v3 to v4":https://freelancing-gods.com/thinking-sphinx/v4/upgrading.html, and "v1/v2 to v3":https://freelancing-gods.com/thinking-sphinx/v3/upgrading.html.
+
+h2. Installation
+
+It's a gem, so install it like you would any other gem. You will also need to specify the mysql2 gem if you're using MRI, or jdbc-mysql if you're using JRuby:
+
+<pre><code>gem 'mysql2',          '~> 0.4',    :platform => :ruby
+gem 'jdbc-mysql',      '~> 5.1.35', :platform => :jruby
+gem 'thinking-sphinx', '~> 6.0'</code></pre>
+
+The MySQL gems mentioned are required for connecting to Sphinx, so please include it even when you're using PostgreSQL for your database.
+
+You'll also need to install Sphinx - this is covered in "the extended documentation":https://freelancing-gods.com/thinking-sphinx/installing_sphinx.html.
+
 h2. Usage
 
-First, if you haven't done so already, check out the main "usage":http://ts.freelancing-gods.com/usage.html page. Once you've done that, the next place to look for information is the specific method docs - ThinkingSphinx::Search and ThinkingSphinx::Index::Builder in particular.
+Begin by reading the "quick-start guide":https://freelancing-gods.com/thinking-sphinx/quickstart.html, and beyond that, "the documentation":https://freelancing-gods.com/thinking-sphinx/ should serve you pretty well.
+
+h2. Requirements
+
+The current release of Thinking Sphinx works with the following versions of its dependencies:
+
+|_. Library |_. Minimum |_. Tested Against |
+| Ruby | v3.0 | v3.0..v4.0 |
+| Sphinx | v2.2.11 | v2.2.11, v3.4.1 |
+| Manticore | v2.8 | v4.0, v6.0 |
+| ActiveRecord | v6.1 | v6.1..v8.1 |
+
+It _might_ work with older versions of Ruby (back to v2.7) and Rails (back to v5.0), but it's highly recommended to update to a supported release.
+
+It should also work with JRuby, but the test environment for that in CI has been unreliable, hence that's not actively tested against at the moment.
+
+h3. Sphinx or Manticore
+
+If you're using Sphinx, v2.2.11 is recommended even though it's quite old, as it works well with PostgreSQL databases (but if you're using MySQL - or real-time indices - then v3.3.1 should also be fine).
+
+If you're opting for Manticore instead, v2.8 or newer works, but v4 or newer is recommended as that's what is actively tested against. The v4.2 and 5.0 releases had bugs with facet searching, but that's been fixed in Manticore v6.0.
 
-Keep in mind that while Thinking Sphinx works for ActiveRecord with Merb, it doesn't yet support DataMapper (although that is planned).
+h3. Rails and ActiveRecord
+
+Currently Thinking Sphinx is built to support Rails/ActiveRecord 6.1 or newer. If you're using Sinatra and ActiveRecord instead of Rails, that's fine - just make sure you add the @:require => 'thinking_sphinx/sinatra'@ option when listing @thinking-sphinx@ in your Gemfile.
+
+The last releases for older Rails versions:
+
+|_. Rails Version |_. Thinking Sphinx Version |
+| 6.0 | v5.6.0 |
+| 5.2 | v5.6.0 |
+| 5.1 | v5.6.0 |
+| 5.0 | v5.6.0 |
+| 4.2 | v5.6.0 |
+| 4.1 | v4.4.1 |
+| 4.0 | v4.4.1 |
+| 3.2 | v4.4.1 |
+| 3.1 | v3.0.6 |
+| 3.0 | v2.1.0 |
+| 2.3 | v1.5.0 |
+
+Please note that these older versions are no longer actively supported.
+
+h3. Ruby
+
+You'll need either the standard Ruby (v3.0 or newer) or JRuby (9.1 or newer). The last release to support Ruby 2.4-2.7 is v5.6.0.
+
+h3. Database Versions
+
+MySQL 5.x and Postgres 8.4 or better are supported.
 
 h2. Contributing
 
-Fork on GitHub and after you've committed tested patches, send a pull request.
-
-To quickly see if your system is ready to run the thinking sphinx specs, run the contribute.rb script found in the project root directory. Use the following instructions to install any missing requirements.
-
-To get the spec suite running, you will need to install the ginger gem:
-
-  sudo gem install ginger --source http://gemcutter.org
-  
-Then install the cucumber, yard, jeweler and rspec gems. Make sure you have a git install version 1.6.0.0 or higher, otherwise the jeweler gem won't install.
-
-  sudo gem install cucumber yard jeweler rspec
-
-Then set up your database:
-
-  cp spec/fixtures/database.yml.default spec/fixtures/database.yml
-  mysqladmin -u root create thinking_sphinx
-  
-This last step can be done automatically by the contribute.rb script if all dependencies are met.
-
-Make sure you don't have another Sphinx daemon (searchd) running. If you do, quit it with "rake ts:stop"
-in the app root.
-
-You should now have a passing test suite from which to build your patch on.
-
-  rake spec
-
-If you get the message "Failed to start searchd daemon", run the spec with sudo:
-
-  sudo rake spec
-
-If you quit the spec suite before it's completed, you may be left with data in the test
-database, causing the next run to have failures. Let that run complete and then try again.
-
-h2. Contributors
-
-Since I first released this library, there's been quite a few people who have submitted patches, to my immense gratitude. Others have suggested syntax changes and general improvements. So my thanks to the following people:
-
-* Joost Hietbrink
-* Jonathan Conway
-* Gregory Mirzayantz
-* Tung Nguyen
-* Sean Cribbs
-* Benoit Caccinolo
-* John Barton
-* Oliver Beddows
-* Arthur Zapparoli
-* Dusty Doris
-* Marcus Crafter
-* Patrick Lenz
-* Björn Andreasson
-* James Healy
-* Jae-Jun Hwang
-* Xavier Shay
-* Jason Rust
-* Gopal Patel
-* Chris Heald
-* Peter Vandenberk
-* Josh French
-* Andrew Bennett
-* Jordan Fowler
-* Seth Walker
-* Joe Noon
-* Wolfgang Postler
-* Rick Olson
-* Killian Murphy
-* Morten Primdahl
-* Ryan Bates
-* David Eisinger
-* Shay Arnett
-* Minh Tran
-* Jeremy Durham
-* Piotr Sarnacki
-* Matt Johnson
-* Nicolas Blanco
-* Max Lapshin
-* Josh Natanson
-* Philip Hallstrom
-* Christian Rishøj
-* Mike Flester
-* Jim Remsik
-* Kennon Ballou
-* Henrik Nyh
-* Emil Tin
-* Doug Cole
-* Ed Hickey
-* Evan Weaver
-* Thibaut Barrere
-* Kristopher Chambers
-* Dmitrij Smalko
-* Aleksey Yeschenko
-* Lachie Cox
-* Lourens Naude
-* Tom Davies
-* Dan Pickett
-* Alex Caudill
-* Jim Benton
-* John Aughey
-* Keith Pitty
-* Jeff Talbot
-* Dana Contreras
-* Menno van der Sman
-* Bill Harding
-* Isaac Feliu
-* Andrei Bocan
-* László Bácsi
-* Peter Wagenet
-* Max Lapshin
-* Martin Emde
-* David Wennergren
-* Mark Lane
-* Eric Lindvall
-* Lawrence Pit
-* Mike Bailey
-* Bill Leeper
-* Michael Reinsch
-* Anderson Dias
-* Jerome Riga
-* Tien Dung
-* Johannes Kaefer
-* Paul Campbell
-* Matthew Beale
-* Tom Simnett
-* Erik Ostrom
-* Ole Riesenberg
-* Josh Kalderimis
-* J.D. Hollis
-* Jeffrey Chupp
-* Rob Anderton
-* Zach Inglis
-* Ward Bekker
-* Brian Terlson
-* Christian Aust
-* Martin Sarasale
-* Édouard Brière
+Please note that this project has a "Contributor Code of Conduct":http://contributor-covenant.org/version/1/0/0/. By participating in this project you agree to abide by its terms.
+
+To contribute, clone this repository and have a good look through the specs - you'll notice the distinction between acceptance tests that actually use Sphinx and go through the full stack, and unit tests (everything else) which use liberal test doubles to ensure they're only testing the behaviour of the class in question. I've found this leads to far better code design.
+
+All development is done on the @develop@ branch; please base any pull requests off of that branch. Please write the tests and then the code to get them passing, and send through a pull request.
+
+In order to run the tests, you'll need to create a database named @thinking_sphinx@:
+
+<pre><code># Either fire up a MySQL console:
+mysql -u root
+# OR a PostgreSQL console:
+psql
+# In that console, create the database:
+CREATE DATABASE thinking_sphinx;</code></pre>
+
+You can then run the unit tests with @rake spec:unit@, the acceptance tests with @rake spec:acceptance@, or all of the tests with just @rake@. To run these with PostgreSQL, you'll need to set the @DATABASE@ environment variable accordingly:
+
+<pre><code>DATABASE=postgresql rake</code></pre>
+
+h2. Licence
+
+Copyright (c) 2007-2026, Thinking Sphinx is developed and maintained by Pat Allan, and is released under the open MIT Licence. Many thanks to "all who have contributed patches":https://github.com/pat/thinking-sphinx/contributors.

data/Rakefile

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+require 'bundler'
+require 'appraisal'
+
+Bundler::GemHelper.install_tasks
+
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new
+
+namespace :spec do
+  desc 'Run unit specs only'
+  RSpec::Core::RakeTask.new(:unit) do |task|
+    task.pattern    = 'spec'
+    task.rspec_opts = '--tag "~live"'
+  end
+
+  desc 'Run acceptance specs only'
+  RSpec::Core::RakeTask.new(:acceptance) do |task|
+    task.pattern    = 'spec'
+    task.rspec_opts = '--tag "live"'
+  end
+end
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,15 @@
+#! /usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "thinking_sphinx"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/loadsphinx

@@ -0,0 +1,99 @@
+#!/usr/bin/env bash
+
+version=$1
+engine=$2
+
+set -e
+
+load_sphinx () {
+  distro="xenial"
+
+  case $version in
+    2.1.9)
+      url="http://sphinxsearch.com/files/sphinxsearch_2.1.9-release-0ubuntu11~trusty_amd64.deb"
+      format="deb"
+      distro="trusty";;
+    2.2.11)
+      url="http://sphinxsearch.com/files/sphinxsearch_2.2.11-release-1~jessie_amd64.deb"
+      format="deb"
+      distro="trusty";;
+    3.0.3)
+      url="http://sphinxsearch.com/files/sphinx-3.0.3-facc3fb-linux-amd64.tar.gz"
+      format="gz";;
+    3.1.1)
+      url="http://sphinxsearch.com/files/sphinx-3.1.1-612d99f-linux-amd64.tar.gz"
+      format="gz";;
+    3.2.1)
+      url="http://sphinxsearch.com/files/sphinx-3.2.1-f152e0b-linux-amd64.tar.gz"
+      format="gz";;
+    3.3.1)
+      url="http://sphinxsearch.com/files/sphinx-3.3.1-b72d67b-linux-amd64.tar.gz"
+      format="gz";;
+    3.4.1)
+      url="http://sphinxsearch.com/files/sphinx-3.4.1-efbcc65-linux-amd64.tar.gz"
+      format="gz";;
+    *)
+      echo "No Sphinx version $version available"
+      exit 1;;
+  esac
+
+  if [ "$distro" == "trusty" ]; then
+    curl --location http://launchpadlibrarian.net/247512886/libmysqlclient18_5.6.28-1ubuntu3_amd64.deb -o libmysql.deb
+    sudo apt-get install ./libmysql.deb
+  fi
+
+  if [ "$format" == "deb" ]; then
+    curl --location $url -o sphinx.deb
+    sudo apt-get install libodbc1
+    sudo dpkg -i ./sphinx.deb
+    sudo apt-get install -f
+  else
+    curl $url -o sphinx.tar.gz
+    tar -zxvf sphinx.tar.gz
+    sudo mv sphinx-$version/bin/* /usr/local/bin/.
+  fi
+}
+
+load_manticore () {
+  url="https://github.com/manticoresoftware/manticore/releases/download/$version/manticore_$version.deb"
+
+  case $version in
+    2.6.4)
+      url="https://github.com/manticoresoftware/manticoresearch/releases/download/2.6.4/manticore_2.6.4-180503-37308c3-release-stemmer.xenial_amd64-bin.deb";;
+    2.7.5)
+      url="https://github.com/manticoresoftware/manticoresearch/releases/download/2.7.5/manticore_2.7.5-181204-4a31c54-release-stemmer.xenial_amd64-bin.deb";;
+    2.8.2)
+      url="https://github.com/manticoresoftware/manticoresearch/releases/download/2.8.2/manticore_2.8.2-190402-4e81114d-release-stemmer.stretch_amd64-bin.deb";;
+    3.4.2)
+      url="https://github.com/manticoresoftware/manticoresearch/releases/download/3.4.2/manticore_3.4.2-200410-6903305-release.xenial_amd64-bin.deb";;
+    3.5.4)
+      url="https://repo.manticoresearch.com/repository/manticoresearch_focal/dists/focal/main/binary-amd64/manticore_3.5.4-210107-f70faec5_amd64.deb";;
+    4.0.2)
+      url="https://repo.manticoresearch.com/repository/manticoresearch_focal/dists/focal/main/binary-amd64/manticore_4.0.2-210921-af497f245_amd64.deb";;
+    4.2.0)
+      url="https://repo.manticoresearch.com/repository/manticoresearch_focal/dists/focal/main/binary-amd64/manticore_4.2.0-211223-15e927b28_amd64.deb";;
+    6.0.0)
+      url="skipped";;
+    *)
+      echo "No Manticore version $version available"
+      exit 1;;
+  esac
+
+  if [ "$version" == "6.0.0" ]; then
+    curl --location https://repo.manticoresearch.com/manticore-repo.noarch.deb -o repo.deb
+    sudo dpkg -i repo.deb
+    sudo apt update
+    sudo apt install manticore
+  else
+    sudo apt-get install default-libmysqlclient-dev
+    curl --location $url -o manticore.deb
+    sudo dpkg -i ./manticore.deb
+    sudo apt-get install -f
+  fi
+}
+
+if [ "$engine" == "sphinx" ]; then
+  load_sphinx
+else
+  load_manticore
+fi

data/bin/testmatrix

@@ -0,0 +1,48 @@
+#! /usr/bin/env ruby
+# frozen_string_literal: true
+
+require "json"
+
+RUBIES = %w[ 3.0 3.1 3.2 3.3 3.4 4.0 ]
+DATABASES = %w[ mysql2 postgresql ]
+SPHINX_ENGINES = {
+  "sphinx" => %w[ 2.2.11 3.4.1 ],
+  "manticore" => %w[ 4.0.2 6.0.0 ]
+}.freeze
+RAILS_VERSIONS = [
+  { version: '6_1', min_ruby: 2.7, max_ruby: 3.3 },
+  { version: '7_0', min_ruby: 2.7, max_ruby: 4.0 },
+  { version: '7_1', min_ruby: 2.7, max_ruby: 4.0 },
+  { version: '7_2', min_ruby: 3.1, max_ruby: 4.0 },
+  { version: '8_0', min_ruby: 3.2, max_ruby: 4.0 },
+  { version: '8_1', min_ruby: 3.2, max_ruby: 4.0 },
+].freeze
+
+# Returns an array of hashes, with each hash being a valid testable
+# combination:
+output = []
+
+RUBIES.each do |ruby|
+  DATABASES.each do |database|
+    SPHINX_ENGINES.each do |sphinx_engine, sphinx_versions|
+      sphinx_versions.each do |sphinx_version|
+        # Sphinx 3.4.1 doesn't play nicely with Postgres.
+        next if database == 'postgresql' && sphinx_engine == 'sphinx' && sphinx_version == '3.4.1'
+
+        RAILS_VERSIONS.each do |rails|
+          next unless ruby.to_f >= rails[:min_ruby] && ruby.to_f <= rails[:max_ruby]
+
+          output << {
+            ruby: ruby,
+            rails: rails[:version],
+            database: database,
+            sphinx_engine: sphinx_engine,
+            sphinx_version: sphinx_version
+          }
+        end
+      end
+    end
+  end
+end
+
+puts JSON.generate(output.compact)

data/lib/thinking/sphinx.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require 'thinking_sphinx'

data/lib/thinking-sphinx.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require 'thinking_sphinx'

data/lib/thinking_sphinx/active_record/association.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+class ThinkingSphinx::ActiveRecord::Association
+  def initialize(column)
+    @column = column
+  end
+
+  def stack
+    @column.__stack + [@column.__name]
+  end
+
+  def string?
+    @column.is_a?(String)
+  end
+
+  def to_s
+    @column.to_s
+  end
+end

data/lib/thinking_sphinx/active_record/association_proxy/attribute_finder.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+class ThinkingSphinx::ActiveRecord::AssociationProxy::AttributeFinder
+  def initialize(association)
+    @association = association
+  end
+
+  def attribute
+    attributes.detect { |attribute|
+      ThinkingSphinx::ActiveRecord::AssociationProxy::AttributeMatcher.new(
+        attribute, foreign_key
+      ).matches?
+    } or raise "Missing Attribute for Foreign Key #{foreign_key}"
+  end
+
+  private
+
+  def attributes
+    indices.collect(&:attributes).flatten
+  end
+
+  def configuration
+    ThinkingSphinx::Configuration.instance
+  end
+
+  def foreign_key
+    @foreign_key ||= reflection_target.foreign_key
+  end
+
+  def indices
+    @indices ||= begin
+      configuration.preload_indices
+      configuration.indices_for_references(
+        *configuration.index_set_class.reference_name(@association.klass)
+      ).reject &:distributed?
+    end
+  end
+
+  def reflection_target
+    target = @association.reflection
+    target = target.through_reflection if target.through_reflection
+    target
+  end
+end

data/lib/thinking_sphinx/active_record/association_proxy/attribute_matcher.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+class ThinkingSphinx::ActiveRecord::AssociationProxy::AttributeMatcher
+    def initialize(attribute, foreign_key)
+      @attribute, @foreign_key = attribute, foreign_key.to_s
+    end
+
+    def matches?
+      return false if many?
+
+      column_name_matches? || attribute_name_matches? || multi_singular_match?
+    end
+
+    private
+
+    attr_reader :attribute, :foreign_key
+
+    delegate :name, :multi?, :to => :attribute
+
+    def attribute_name_matches?
+      name == foreign_key
+    end
+
+    def column_name_matches?
+      column.__name.to_s == foreign_key
+    end
+
+    def column
+      attribute.respond_to?(:columns) ? attribute.columns.first :
+        attribute.column
+    end
+
+    def many?
+      attribute.respond_to?(:columns) && attribute.columns.many?
+    end
+
+    def multi_singular_match?
+      multi? && name.singularize == foreign_key
+    end
+  end

data/lib/thinking_sphinx/active_record/association_proxy.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module ThinkingSphinx::ActiveRecord::AssociationProxy
+  def search(query = nil, options = {})
+    perform_search super(*normalise_search_arguments(query, options))
+  end
+
+  def search_for_ids(query = nil, options = {})
+    perform_search super(*normalise_search_arguments(query, options))
+  end
+
+  private
+
+  def normalise_search_arguments(query, options)
+    query, options = nil, query if query.is_a?(Hash)
+    options[:ignore_scopes] = true
+
+    [query, options]
+  end
+
+  def perform_search(searcher)
+    ThinkingSphinx::Search::Merger.new(searcher).merge! nil,
+      :with => association_filter
+  end
+
+  def association_filter
+    attribute = AttributeFinder.new(proxy_association).attribute
+
+    {attribute.name.to_sym => proxy_association.owner.id}
+  end
+end
+
+require 'thinking_sphinx/active_record/association_proxy/attribute_finder'
+require 'thinking_sphinx/active_record/association_proxy/attribute_matcher'

data/lib/thinking_sphinx/active_record/attribute/sphinx_presenter.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+class ThinkingSphinx::ActiveRecord::Attribute::SphinxPresenter
+  SPHINX_TYPES = {
+    :integer   => :uint,
+    :boolean   => :bool,
+    :timestamp => :uint,
+    :float     => :float,
+    :string    => :string,
+    :bigint    => :bigint,
+    :ordinal   => :str2ordinal,
+    :wordcount => :str2wordcount,
+    :json      => :json
+  }
+
+  def initialize(attribute, source)
+    @attribute, @source = attribute, source
+  end
+
+  def collection_type
+    @attribute.multi? ? :multi : sphinx_type
+  end
+
+  def declaration
+    if @attribute.multi?
+      multi_declaration
+    else
+      @attribute.name
+    end
+  end
+
+  def sphinx_type
+    SPHINX_TYPES[@attribute.type]
+  end
+
+  private
+
+  def multi_declaration
+    case @attribute.source_type
+    when :query, :ranged_query
+      query
+    else
+      "#{sphinx_type} #{@attribute.name} from field"
+    end
+  end
+
+  def query
+    ThinkingSphinx::ActiveRecord::PropertyQuery.new(
+      @attribute, @source, sphinx_type
+    ).to_s
+  end
+end