fast_fuzzy 0.1.0-java

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 9027df9eb5741a88f7cb6e86217bf10f15e3600a
+  data.tar.gz: 638da9aba144ab4cc92b721e07e7fffb23ba9fa4
+SHA512:
+  metadata.gz: b179f35244d348ea5e2476707e25cf091975e8fdbf3a955f48d8155e4937fc530e869f8d9373f9268368ec444302b3bc34b208eac2cb102b8a3f724bae9cac92
+  data.tar.gz: 193d92a94d5611c9b1d687eb3377ef058e99922dad99ac0e4b5a3a605b393942ba73005939fced7dc617296fef6c3cd068f83cce97dd28f729e8b9cf0ef4d03f

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2014-2016 Colin Surprenant colin.surprenant@gmail.com
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -0,0 +1,140 @@
+# FastFuzzy
+
+This gem only supports [JRuby](http://jruby.org/).
+
+FastFuzzy performs fast and fuzzy text pattern matching. It uses the Lucene analyzers to tokenize the text using a configurable analyzer chain and the extracted tokens
+are matched against the searched text using ngram matching and a resulting match score is computed.
+
+The original intent of this code was to perform on-the-fly matching of some specific categories of expressions or sentences in social media. Using an analyzer chain to tokenize, remove stop words, etc, allows performing the matching only on the relevant text tokens. Using ngram scoring provides the fuzzyness confidence score to find approximate matching text with typos or different spelling or any number of variations. With experimentation an "acceptable" score can be decided to establish if the searched text matches or not against what we are looking for.
+
+Note that this gem also include a custom Lucene Twitter tokenizer, see usage examples below.
+
+## Installation
+
+This gem only supports [JRuby](http://jruby.org/).
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'fast_fuzzy'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install fast_fuzzy
+
+## Usage
+
+### Percolator
+
+- First configure the `Percolator` with any number of text strings which represents what we are looking for, or the queries:
+
+```ruby
+p = FastFuzzy::Percolator.new
+
+p << "looking for a restaurant"
+p << "recommend a restaurant"
+```
+
+- Run the `Percolator` against some text. The result is the list of matching "queries" sorted in ascending score order.
+
+```ruby
+p.percolate("hey! anyone can recomment a good restaurant in montreal tonight?")
+=> [[0, 0.5294117647058824], [1, 0.8421052631578947]]
+```
+
+In this example the last query  "recommend a restaurant" matched with a 0.842 or ~84% score.
+
+```ruby
+p.percolate("I am looking for a good suchi restaurant")
+=> [[1, 0.47368421052631576], [0, 0.8823529411764706]]
+```
+
+In this example the first query  "looking for a restaurant" matched with a 0.882 or ~88% score.
+
+### Custom Analyzer Chain
+
+Included is a custom Twitter Tokenizer and can be used by defining an analyzer chain for the Percolator:
+
+```ruby
+p = FastFuzzy::Percolator.new(:analyzer_chain => [
+  [Lucene::TwitterTokenizer],
+  [Lucene::LowerCaseFilter],
+  [Lucene::StopFilter, Lucene::StandardAnalyzer::STOP_WORDS_SET],
+])
+
+p << "looking for a restaurant"
+p << "recommend a restaurant"
+```
+
+```ruby
+p.percolate("RT yo lookin for a good restaurant #montreal #foodie")
+=> [[1, 0.5263157894736842], [0, 0.7647058823529411]]
+```
+
+## Custom Twitter Tokenizer
+
+The included custom Twitter Tokenizer was created to classify more Twitter specific tokens and help only match on actual text words. YMMV.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+### Building
+
+- Install gradle-jflex-plugin
+
+this uses the gradle-jflex-plugin. The original version does not support jflex 1.6.1 so
+you can use my forked version at https://github.com/colinsurprenant/gradle-jflex-plugin
+or the original if/when https://github.com/thomaslee/gradle-jflex-plugin/pull/2 is merged.
+
+```sh
+$ git clone https://github.com/colinsurprenant/gradle-jflex-plugin
+#  or
+$ git clone https://github.com/thomaslee/gradle-jflex-plugin
+```
+
+```sh
+$ cd gradle-jflex-plugin
+$ gradle build
+$ gradle install
+# or use gradlew if you do not have Gradle installed
+$ ./gradlew build
+$ ./gradlew install
+````
+
+- Build Java sources
+
+```sh
+$ gradle build
+# or use gradlew if you do not have Gradle installed
+$ ./gradlew build
+```
+
+### Tests / Specs
+
+```sh
+$ bundle install
+```
+
+```sh
+$ bundle exec rspec
+```
+
+## Author
+
+**Colin Surprenant** on [GitHub](https://github.com/colinsurprenant) and [Twitter](https://twitter.com/colinsurprenant).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/colinsurprenant/fast_fuzzy.
+
+## License and Copyright
+
+*FastFuzzy* is released under the Apache License, Version 2.0.

data/fast_fuzzy.gemspec

@@ -0,0 +1,30 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'fast_fuzzy/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "fast_fuzzy"
+  spec.version       = FastFuzzy::VERSION
+  spec.authors       = ["Colin Surprenant"]
+  spec.email         = ["colin.surprenant@gmail.com"]
+
+  spec.summary       = "JRuby Fast and fuzzy text pattern matching using Lucene analyzers"
+  spec.description   = "JRuby Fast and fuzzy text pattern matching using Lucene analyzers"
+  spec.homepage      = "https://github.com/colinsurprenant/fast_fuzzy"
+  spec.licenses      = ["Apache-2.0"]
+
+  spec.files         = Dir.glob(["*.gemspec", "lib/**/*.jar", "lib/**/*.rb", "spec/**/*.rb", "LICENSE", "README.md"])
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+
+  spec.platform      = "java"
+
+  spec.add_runtime_dependency "jar-dependencies"
+
+  spec.requirements << "jar org.apache.lucene:lucene-core, 5.4.1"
+  spec.requirements << "jar org.apache.lucene:lucene-analyzers-common, 5.4.1"
+
+  spec.add_development_dependency "bundler", "~> 1.10"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec"
+end

data/lib/fast_fuzzy.rb

@@ -0,0 +1,27 @@
+# encoding: utf-8
+
+require "java"
+
+module FastFuzzy
+  # local dev setup
+  classes_dir = File.expand_path("../../build/classes/main", __FILE__)
+
+  if File.directory?(classes_dir)
+    # if in local dev setup, add target to classpath
+    $CLASSPATH << classes_dir unless $CLASSPATH.include?(classes_dir)
+  else
+    jar_path = "fast_fuzzy/fastfuzzy-#{VERSION}.jar"
+    begin
+      require jar_path
+    rescue Exception => e
+      raise("Error loading #{jar_path}, cause: #{e.message}")
+    end
+  end
+end
+
+require "fast_fuzzy_jars"
+require "fast_fuzzy/version"
+require "fast_fuzzy/lucene"
+require "fast_fuzzy/analyzer"
+require "fast_fuzzy/ngram"
+require "fast_fuzzy/percolator"

data/lib/fast_fuzzy/analyzer.rb

@@ -0,0 +1,49 @@
+module FastFuzzy
+
+  class Analyzer
+    include Enumerable
+
+    STANDARD_CHAIN = [
+      [Lucene::StandardTokenizer],
+      [Lucene::StandardFilter],
+      [Lucene::LowerCaseFilter],
+      [Lucene::StopFilter, Lucene::StandardAnalyzer::STOP_WORDS_SET],
+    ]
+
+    def initialize(str, chain_definition = STANDARD_CHAIN)
+      @str = str
+
+      # first chain element must be the tokenizer, initialize it and set its reader
+      definition = chain_definition.first
+      clazz = definition.first
+      params = definition[1..-1]
+      tokenizer = clazz.new(*params)
+      tokenizer.set_reader(java.io.StringReader.new(str))
+
+      # initialize the following filters and build the stream chain
+      stream = chain_definition[1..-1].inject(tokenizer) do |result, definition|
+        clazz = definition.first
+        params = definition[1..-1]
+        clazz.new(result, *params)
+      end
+
+      # use CachingTokenFilter to allow multiple stream traversing
+      @stream = Lucene::CachingTokenFilter.new(stream)
+      @term_attr = @stream.addAttribute(Lucene::CharTermAttribute.java_class);
+      @type_attr = @stream.addAttribute(Lucene::TypeAttribute.java_class);
+    end
+
+    # implement each for Enumerable
+    def each(&block)
+      @stream.reset
+      while @stream.incrementToken
+        yield [@term_attr.to_string, @type_attr.type]
+      end
+      @stream.end
+    end
+
+    def close
+      @stream.close
+    end
+  end
+end

data/lib/fast_fuzzy/lucene.rb

@@ -0,0 +1,42 @@
+require 'java'
+
+module Lucene
+  Version = org.apache.lucene.util.Version
+  VERSION = Version::LATEST.to_string
+
+  ALPHANUM_TYPE = "<ALPHANUM>".freeze
+
+  module Analysis
+    include_package "org.apache.lucene.analysis"
+
+    module Standard
+      include_package "org.apache.lucene.analysis.standard"
+    end
+
+    module TokenAttributes
+      include_package 'org.apache.lucene.analysis.tokenattributes'
+    end
+
+    module Core
+      include_package 'org.apache.lucene.analysis.core'
+    end
+  end
+
+  module FastFuzzy
+    # custom TwitterTokenizer
+    include_package "com.colinsurprenant.fastfuzzy"
+  end
+
+  StandardTokenizer = Analysis::Standard::StandardTokenizer
+  ClassicTokenizer = Analysis::Standard::ClassicTokenizer
+  StandardAnalyzer = Analysis::Standard::StandardAnalyzer
+  StandardFilter = Analysis::Standard::StandardFilter
+  LowerCaseFilter = Analysis::Core::LowerCaseFilter
+  StopFilter = Analysis::Core::StopFilter
+  CharTermAttribute = Analysis::TokenAttributes::CharTermAttribute
+  CachingTokenFilter = Analysis::CachingTokenFilter
+  TypeAttribute = Analysis::TokenAttributes::TypeAttribute
+  FlagsAttribute = Analysis::TokenAttributes::FlagsAttribute
+
+  TwitterTokenizer = FastFuzzy::TwitterTokenizer
+end

data/lib/fast_fuzzy/ngram.rb

@@ -0,0 +1,18 @@
+module FastFuzzy
+
+  class Ngram
+
+    # generate the ngram for either the caracter grams of a string or the items grams of an array
+    # example: generate("abcd") => ["ab", "bc", "cd"], generate([:a, :b, :c, :d]) => [[:a, :b], [:b, :c], [:c, :d]]
+    # @param str_or_ary [String|Array] input string or array of anything
+    # @return [Array] array or ngrams
+    def self.generate(str_or_ary, n = 2)
+      if (s = str_or_ary.size) <= n
+        return [] if s.zero?
+        return [str_or_ary]
+      end
+
+      (0..str_or_ary.size - n).map{|i| str_or_ary[i, n]}
+    end
+  end
+end

data/lib/fast_fuzzy/percolator.rb

@@ -0,0 +1,79 @@
+require 'fast_fuzzy/analyzer'
+require 'fast_fuzzy/ngram'
+
+module FastFuzzy
+
+  class Percolator
+    attr_reader :queries, :index
+    N = 4 # >= 3 is necessary to wheight consecutiveness of words (ex "this car" => thi, his, is_, s_c, _ca, car)
+
+    def initialize(options = {})
+      @queries = options.delete(:queries) || []
+      @analyzer_chain = options.delete(:analyzer_chain) || Analyzer::STANDARD_CHAIN
+
+      @index = Hash.new{|h, k| h[k] = []}
+      @query_terms_count = []
+
+      build_index unless queries.empty?
+    end
+
+    def <<(query)
+      @queries << query
+      build_index
+    end
+
+    def percolate(str)
+      analyser = Analyzer.new(str, @analyzer_chain)
+
+      terms_phrase = analyser.select{|term| term[1] == Lucene::ALPHANUM_TYPE}.map{|term| term[0]}.join(' ')
+      terms_phrase = " #{terms_phrase} "
+      grams = Ngram.generate(terms_phrase, N).uniq
+      score(lookup(grams))
+    end
+
+    private
+
+    # build inverted index into @index that list all queries that belong to each term
+    # {"term" => [query id, query id, ...]}
+    def build_index
+      @index.clear
+      @query_terms_count.clear
+
+      @queries.each_index do |query_id|
+        analyser = Analyzer.new(@queries[query_id], @analyzer_chain)
+
+        terms_phrase = analyser.select{|term| term[1] == Lucene::ALPHANUM_TYPE}.map{|term| term[0]}.join(' ')
+        terms_phrase = " #{terms_phrase} "
+        grams = Ngram.generate(terms_phrase, N).uniq
+        @query_terms_count[query_id] = grams.size
+        grams.each{|gram| @index[gram] << query_id}
+      end
+      @index.each_key{|k| @index[k] = @index[k].uniq}
+
+      self
+    end
+
+    # for each search term lookup the index and keep count of the number of terms that matched each query
+    # @param terms [Array<String>] search terms
+    # @return [Hash] number of terms that matched each query, {query id => terms hit count}
+    def lookup(terms)
+      results = Hash.new(0)
+
+      terms.each do |term|
+        # @index[term] has list of query_id matching this term
+        @index[term].each{|query_id| results[query_id] += 1}
+      end
+
+      results
+    end
+
+
+    # compute match score per query for given lookup result
+    # @param result [Hash] number of terms that matched (> 0.0 score) each query, {query id => terms hit count}
+    # @return [Array<Array>] sorted array by score of [[query id, score]]. will include only queries with score > 0, [] if no match
+    def score(results)
+      results.map{|k, v| [k, v.to_f / @query_terms_count[k]]}.sort_by{|r| r[1]}
+    end
+
+  end
+end

data/lib/fast_fuzzy/version.rb

@@ -0,0 +1,3 @@
+module FastFuzzy
+  VERSION = "0.1.0"
+end

data/lib/fast_fuzzy_jars.rb

@@ -0,0 +1,5 @@
+# this is a generated file, to avoid over-writing it just delete this comment
+require 'jar_dependencies'
+
+require_jar( 'org.apache.lucene', 'lucene-core', '5.4.1' )
+require_jar( 'org.apache.lucene', 'lucene-analyzers-common', '5.4.1' )

data/spec/fast_fuzzy/analyzer_spec.rb

@@ -0,0 +1,116 @@
+require_relative '../spec_helper'
+
+describe FastFuzzy::Analyzer do
+
+  it "should support to_a" do
+    analyzer = FastFuzzy::Analyzer.new("aa bb")
+    expect(analyzer.to_a).to be_a(Array)
+    expect(analyzer.to_a.size).to eq(2)
+  end
+
+  it "should support each" do
+    analyzer = FastFuzzy::Analyzer.new("aa bb")
+    result = []
+    analyzer.each{|token| result << token}
+    expect(result.size).to eq(2)
+  end
+
+  shared_examples :analyzer do
+    it "should analyse and extract correct type" do
+      tests.each do |test|
+        analyzer = FastFuzzy::Analyzer.new(test.first, chain)
+        expect(analyzer.to_a).to eq(test.last)
+      end
+    end
+
+    it "should support each" do
+      analyzer = FastFuzzy::Analyzer.new("aa bb cc", chain)
+      results = []
+      analyzer.each{|term| results << term}
+      expect(results).to eq([["aa", "<ALPHANUM>"], ["bb", "<ALPHANUM>"], ["cc", "<ALPHANUM>"]])
+    end
+
+    it "should support Enumerable" do
+      analyzer = FastFuzzy::Analyzer.new("aa bb 12", chain)
+      expect(analyzer.map{|term| term.first}).to eq(["aa", "bb", "12"])
+      expect(analyzer.map{|term| term.last}).to eq(["<ALPHANUM>", "<ALPHANUM>", "<NUM>"])
+    end
+  end
+
+  context "using default analyzer chain" do
+    it_behaves_like :analyzer do
+      let(:tests) {[
+        # the following tests are actually copied from the TwitterTokenizer tests below and
+        # ajusted to the StandardTokenizer chain results.
+        # TODO: make better tests for StandardTokenizer & default chain
+
+        ["2012 $80K/year", [["2012", "<NUM>"],  ["80k", "<ALPHANUM>"], ["year", "<ALPHANUM>"]]],
+        ["is @ 7.9% and 1% not 4th", [["7.9", "<NUM>"], ["1", "<NUM>"], ["4th", "<ALPHANUM>"]]],
+
+        ["it's o'reilly o'reilly's hey", [["it's", "<ALPHANUM>"], ["o'reilly", "<ALPHANUM>"], ["o'reilly's", "<ALPHANUM>"], ["hey", "<ALPHANUM>"]]],
+
+        ["at&t excite@home", [["t", "<ALPHANUM>"], ["excite", "<ALPHANUM>"], ["home", "<ALPHANUM>"]]],
+
+        ["I.B.M. is big", [["i.b.m", "<ALPHANUM>"], ["big", "<ALPHANUM>"]]],
+
+        ["see @colinsurprenant do #win yeah", [["see", "<ALPHANUM>"], ["colinsurprenant", "<ALPHANUM>"], ["do", "<ALPHANUM>"], ["win", "<ALPHANUM>"], ["yeah", "<ALPHANUM>"]]],
+
+        ["go http://colinsurprenant.com/ http://a go", [["go", "<ALPHANUM>"], ["http", "<ALPHANUM>"], ["colinsurprenant.com", "<ALPHANUM>"], ["http", "<ALPHANUM>"], ["go", "<ALPHANUM>"]]],
+
+        ["colin.surprenant@gmail.com", [["colin.surprenant", "<ALPHANUM>"], ["gmail.com", "<ALPHANUM>"]]],
+
+        ["Face-Lift - Graphic", [["face", "<ALPHANUM>"], ["lift", "<ALPHANUM>"], ["graphic", "<ALPHANUM>"]]],
+
+        ["DUDZ :) :P :( 8-(", [["dudz", "<ALPHANUM>"], ["p", "<ALPHANUM>"], ["8", "<NUM>"]]],
+
+        ["RT stoopid #FF ", [["rt", "<ALPHANUM>"], ["stoopid", "<ALPHANUM>"], ["ff", "<ALPHANUM>"]]],
+      ]}
+
+      let(:chain) { FastFuzzy::Analyzer::STANDARD_CHAIN }
+    end
+  end
+
+  context "using TwitterTokenizer chain" do
+    it_behaves_like :analyzer do
+      let(:tests) {[
+        # NUM
+        ["2012 $80K/year", [["2012", "<NUM>"],  ["$80k", "<NUM>"], ["year", "<ALPHANUM>"]]],
+        ["is @ 7.9% and 1% not 4th", [["7.9%", "<NUM>"], ["1%", "<NUM>"], ["4th", "<ALPHANUM>"]]],
+
+        # APOSTROPHE
+        ["it's o'reilly o'reilly's hey", [["it's", "<APOSTROPHE>"], ["o'reilly", "<APOSTROPHE>"], ["o'reilly's", "<APOSTROPHE>"], ["hey", "<ALPHANUM>"]]],
+
+        # COMPANY
+        ["at&t excite@home", [["at&t", "<COMPANY>"], ["excite@home", "<COMPANY>"]]],
+
+        # ACRONYM
+        ["I.B.M. is big", [["i.b.m.", "<ACRONYM>"], ["big", "<ALPHANUM>"]]],
+
+        # USERNAME & HASHTAG
+        ["see @colinsurprenant do #win yeah", [["see", "<ALPHANUM>"], ["@colinsurprenant", "<USERNAME>"], ["do", "<ALPHANUM>"], ["#win", "<HASHTAG>"], ["yeah", "<ALPHANUM>"]]],
+
+        # URL
+        ["go http://colinsurprenant.com/ http://a go", [["go", "<ALPHANUM>"], ["http://colinsurprenant.com/", "<URL>"], ["http://a", "<URL>"], ["go", "<ALPHANUM>"]]],
+
+        # EMAIL
+        ["colin.surprenant@gmail.com", [["colin.surprenant@gmail.com", "<EMAIL>"]]],
+
+        # DASH
+        ["Face-Lift - Graphic", [["face-lift", "<DASH>"], ["graphic", "<ALPHANUM>"]]],
+
+        # EMOTICON
+        ["DUDZ :) :P :( 8-(", [["dudz", "<ALPHANUM>"], [":)", "<EMOTICON>"], [":p", "<EMOTICON>"], [":(", "<EMOTICON>"], ["8-(", "<EMOTICON>"]]],
+
+        # HASHTAG & RT
+        ["RT stoopid #FF ", [["rt", "<RT>"], ["stoopid", "<ALPHANUM>"], ["#ff", "<HASHTAG>"]]],
+      ]}
+
+      let(:chain) {[
+        [Lucene::TwitterTokenizer],
+        [Lucene::LowerCaseFilter],
+        [Lucene::StopFilter, Lucene::StandardAnalyzer::STOP_WORDS_SET],
+      ]}
+    end
+  end
+
+end

data/spec/fast_fuzzy/lucene_spec.rb

@@ -0,0 +1,11 @@
+require_relative "../spec_helper"
+
+describe Lucene do
+  it 'has a version number' do
+    expect(Lucene::VERSION).not_to be nil
+  end
+
+  it 'has the expected version' do
+    expect(Lucene::VERSION).to eq("5.4.1")
+  end
+end

data/spec/fast_fuzzy/ngram_spec.rb

@@ -0,0 +1,25 @@
+require_relative '../spec_helper'
+
+describe FastFuzzy::Ngram do
+
+  it "should generate bigrams" do
+    expect(FastFuzzy::Ngram.generate("allo")).to eq(["al", "ll", "lo"])
+    expect(FastFuzzy::Ngram.generate("allo", 2)).to eq(["al", "ll", "lo"])
+    expect(FastFuzzy::Ngram.generate([:a, :b, :c, :d], 2)).to eq([[:a, :b], [:b, :c], [:c, :d]])
+  end
+
+  it "should generate trigrams" do
+    expect(FastFuzzy::Ngram.generate("allotoi", 3)).to eq(["all", "llo", "lot", "oto", "toi"])
+    expect(FastFuzzy::Ngram.generate([:a, :b, :c, :d], 3)).to eq([[:a, :b, :c], [:b, :c, :d]])
+  end
+
+  it "should generate gram <= n" do
+    expect(FastFuzzy::Ngram.generate("al", 2)).to eq(["al"])
+    expect(FastFuzzy::Ngram.generate([:a, :b], 2)).to eq([[:a, :b]])
+  end
+
+  it "should generate gram < n" do
+    expect(FastFuzzy::Ngram.generate("a", 2)).to eq(["a"])
+    expect(FastFuzzy::Ngram.generate([:a], 2)).to eq([[:a]])
+  end
+end

data/spec/fast_fuzzy/percolator_spec.rb

@@ -0,0 +1,41 @@
+require_relative '../spec_helper'
+
+describe FastFuzzy::Percolator do
+
+  round4 = lambda {|x| x.round(4)}
+
+  it "should add queries" do
+    p = FastFuzzy::Percolator.new
+    p << "test1"
+    p << "test2"
+    expect(p.queries).to eq(["test1", "test2"])
+
+    p = FastFuzzy::Percolator.new(:queries => ["test1", "test2"])
+    expect(p.queries).to eq(["test1", "test2"])
+  end
+
+  it "should percolate using standard chain by default" do
+    p = FastFuzzy::Percolator.new
+    p << "faire une chose"
+    p << "etre subtil"
+
+    expect(p.percolate("veux tu faire quelque chose?").map{|r| [r[0], r[1].round(4)]}).to eq([[0, 0.6429]])
+    expect(p.percolate("allo toi").map{|r| [r[0], r[1].round(4)]}).to be_empty
+    expect(p.percolate("il faut etre quelque peu subtil").map{|r| [r[0], r[1].round(4)]}).to eq([[1, 0.8]])
+
+    p = FastFuzzy::Percolator.new
+    p << "looking for a job"
+    expect(p.percolate("I hate looking for a job").map{|r| [r[0], r[1].round(4)]}).to eq([[0, 1.0]])
+  end
+
+  it "should support configurable chain" do
+    p = FastFuzzy::Percolator.new(:analyzer_chain => [[Lucene::StandardTokenizer], [Lucene::StandardFilter]])
+    p << "foo bar"
+    expect(p.percolate("FOO BAR")).to eq([])
+
+    p = FastFuzzy::Percolator.new(:analyzer_chain => [[Lucene::StandardTokenizer], [Lucene::StandardFilter], [Lucene::LowerCaseFilter]])
+    p << "foo bar"
+    expect(p.percolate("FOO BAR")).to eq([[0, 1.0]])
+  end
+
+end

data/spec/fast_fuzzy_spec.rb

@@ -0,0 +1,7 @@
+require 'spec_helper'
+
+describe FastFuzzy do
+  it 'has a version number' do
+    expect(FastFuzzy::VERSION).not_to be nil
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
+require 'fast_fuzzy'

metadata

@@ -0,0 +1,127 @@
+--- !ruby/object:Gem::Specification
+name: fast_fuzzy
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: java
+authors:
+- Colin Surprenant
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2016-03-08 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: jar-dependencies
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.10'
+  name: bundler
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.10'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  name: rake
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: rspec
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: JRuby Fast and fuzzy text pattern matching using Lucene analyzers
+email:
+- colin.surprenant@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE
+- README.md
+- fast_fuzzy.gemspec
+- lib/fast_fuzzy.rb
+- lib/fast_fuzzy/analyzer.rb
+- lib/fast_fuzzy/fastfuzzy-0.1.0.jar
+- lib/fast_fuzzy/lucene.rb
+- lib/fast_fuzzy/ngram.rb
+- lib/fast_fuzzy/percolator.rb
+- lib/fast_fuzzy/version.rb
+- lib/fast_fuzzy_jars.rb
+- lib/org/apache/lucene/lucene-analyzers-common/5.4.1/lucene-analyzers-common-5.4.1.jar
+- lib/org/apache/lucene/lucene-core/5.4.1/lucene-core-5.4.1.jar
+- spec/fast_fuzzy/analyzer_spec.rb
+- spec/fast_fuzzy/lucene_spec.rb
+- spec/fast_fuzzy/ngram_spec.rb
+- spec/fast_fuzzy/percolator_spec.rb
+- spec/fast_fuzzy_spec.rb
+- spec/spec_helper.rb
+homepage: https://github.com/colinsurprenant/fast_fuzzy
+licenses:
+- Apache-2.0
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements:
+- jar org.apache.lucene:lucene-core, 5.4.1
+- jar org.apache.lucene:lucene-analyzers-common, 5.4.1
+rubyforge_project:
+rubygems_version: 2.4.8
+signing_key:
+specification_version: 4
+summary: JRuby Fast and fuzzy text pattern matching using Lucene analyzers
+test_files:
+- spec/fast_fuzzy/analyzer_spec.rb
+- spec/fast_fuzzy/lucene_spec.rb
+- spec/fast_fuzzy/ngram_spec.rb
+- spec/fast_fuzzy/percolator_spec.rb
+- spec/fast_fuzzy_spec.rb
+- spec/spec_helper.rb