drtom-textacular 4.0.0.alpha.20160302

Sign up to get free protection for your applications and to get access to all the features.
Files changed (36) hide show
  1. checksums.yaml +7 -0
  2. data/CHANGELOG.md +199 -0
  3. data/Gemfile +3 -0
  4. data/README.md +196 -0
  5. data/Rakefile +61 -0
  6. data/lib/textacular.rb +160 -0
  7. data/lib/textacular/full_text_indexer.rb +66 -0
  8. data/lib/textacular/migration_generator.rb +31 -0
  9. data/lib/textacular/postgres_module_installer.rb +57 -0
  10. data/lib/textacular/rails.rb +14 -0
  11. data/lib/textacular/searchable.rb +20 -0
  12. data/lib/textacular/tasks.rb +23 -0
  13. data/lib/textacular/trigram_installer.rb +18 -0
  14. data/lib/textacular/version.rb +7 -0
  15. data/spec/config.travis.yml +8 -0
  16. data/spec/config.yml.example +5 -0
  17. data/spec/spec_helper.rb +104 -0
  18. data/spec/support/ar_stand_in.rb +4 -0
  19. data/spec/support/character.rb +7 -0
  20. data/spec/support/game.rb +5 -0
  21. data/spec/support/game_extended_with_textacular.rb +5 -0
  22. data/spec/support/game_extended_with_textacular_and_custom_language.rb +7 -0
  23. data/spec/support/game_fail.rb +3 -0
  24. data/spec/support/game_fail_extended_with_textacular.rb +5 -0
  25. data/spec/support/not_there.rb +3 -0
  26. data/spec/support/textacular_web_comic.rb +7 -0
  27. data/spec/support/web_comic.rb +7 -0
  28. data/spec/support/web_comic_with_searchable.rb +6 -0
  29. data/spec/support/web_comic_with_searchable_name.rb +6 -0
  30. data/spec/support/web_comic_with_searchable_name_and_author.rb +6 -0
  31. data/spec/textacular/full_text_indexer_spec.rb +69 -0
  32. data/spec/textacular/migration_generator_spec.rb +67 -0
  33. data/spec/textacular/searchable_spec.rb +194 -0
  34. data/spec/textacular/trigram_installer_spec.rb +24 -0
  35. data/spec/textacular_spec.rb +287 -0
  36. metadata +210 -0
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 42f77ab66144c7716a06bc4e88e1810752077744
4
+ data.tar.gz: 892309c1127a3ad16b84661c5b16f597d49950b7
5
+ SHA512:
6
+ metadata.gz: 96c3abd3b0e03a58450674c632f933df4cde4cf42f4ca38053ddf184c66a846315e322b09568c2f97ef4af3d42b291e4e770b602db0d9619880f54117b2a9f71
7
+ data.tar.gz: d53292f2c1777ec2322fa6225c715bfdec0ee1aba10d6903647a12aec308dec9e1d0e71099dc05c512125c7c116b93cec24d5f2c6dc3696df2b4068634b48cda
@@ -0,0 +1,199 @@
1
+ # Change Log
2
+
3
+ ## Unreleased
4
+
5
+ ## 3.2.2
6
+
7
+ * Fallback to zero similarity when trying to match null values
8
+ * Fix search term escaping
9
+ * Trigram installation migration only install extension if not installed yet.
10
+ * Update the README to illustrate how to change the similarity threshold.
11
+ * Remove deprecated dynamic search methods.
12
+
13
+ ## 3.2.1
14
+
15
+ * Fix trigram installation migration reversed filename and content.
16
+ * Rewrite all tests using RSpec.
17
+ * We're ActiveRecord 4.2+ compatible until tests prove otherwise.
18
+
19
+ ## 3.2.0
20
+
21
+ * Add generator for trigram installation migration.
22
+ * Expand gemspec to allow newest version of ActiveRecord.
23
+
24
+ ## 3.1.0
25
+
26
+ * Avoid Deprecation warnings from ActiveRecord 4.0.0.rc2.
27
+ * Fix `method_missing` in ActiveRecord 4.0.0.rc2.
28
+ * Remove unused `Textacular#normalize` method.
29
+ * Add `OR` example to the README.
30
+ * Fix tests for Ruby 2.0.0 & related improvements.
31
+ * Improve Rails integration.
32
+ * Fix dependency loading for textacular rake tasks.
33
+ * Fix ranking failures when rows had `NULL` column values.
34
+ * Clean up Rakefile; should make developing the gem nicer.
35
+ * DEPRECATION: The dynamic search helpers will be removed at the next major
36
+ release.
37
+
38
+ ## 3.0.0
39
+
40
+ * All deprecations have been resolved. This breaks backwards compatibility.
41
+ * Rename gem to Textacular.
42
+
43
+ ## 2.2.0
44
+
45
+ * 1 DEPRECATION
46
+ * The whole gem is being renamed and will no longer be maintained as Texticle.
47
+ The new name it Textacular.
48
+ * 1 new feature
49
+ * Expand gemspec to allow Rails 4.
50
+
51
+ ## 2.1.1
52
+
53
+ * 1 bugfix
54
+ * Include `lib/textacular/version.rb` in the gemspec so the gem will load. Sorry!
55
+
56
+
57
+ ## 2.1.0
58
+
59
+ * 1 DEPRECATION
60
+ * `search` aliases new `advanced_search` method (same functionality as before), but will
61
+ alias `basic_search` in 3.0! Should print warnings.
62
+ * 3 new features
63
+ * Generate full text search indexes from a rake task (sort of like in 1.x). Supply a specific
64
+ model name.
65
+ * New search methods: `basic_search`, `advanced_search` and `fuzzy_search`. Basic allows special
66
+ characters like &, and % in search terms. Fuzzy is based on Postgres's trigram matching extension
67
+ pg_trgm. Advanced is the same functionality from `search` previously.
68
+ * Rake task that installs pg_trgm now works on Postgres 9.1 and up.
69
+ * 2 dev improvements
70
+ * Test database configuration not automatically generated from a rake task and ignored by git.
71
+ * New interactive developer console (powered by pry).
72
+
73
+
74
+ ## 2.0.3
75
+
76
+ * 1 new feature
77
+ * Allow searching through relations. Model.join(:relation).search(:relation => {:column => "query"})
78
+ works, and reduces the need for multi-model tables. Huge thanks to Ben Hamill for the pull request.
79
+ * Allow searching through all model columns irrespective of the column's type; we cast all columns to text
80
+ in the search query. Performance may degrade when searching through anything but a string column.
81
+ * 2 bugfixes
82
+ * Fix exceptions when adding Textacular to a table-less model.
83
+ * Column names in a search query are now scoped to the current table.
84
+ * 1 dev improvement
85
+ * Running `rake` from the project root will setup the test environment by creating a test database
86
+ and running the necessary migrations. `rake` can also be used to run all the project tests.
87
+
88
+
89
+ ## 2.0.2
90
+
91
+ * 1 bugfix
92
+ * Our #respond_to? overwritten method was causing failures when a model didn't have
93
+ a table (e.g. if migrations hadn't been run yet). Not the case anymore.
94
+
95
+
96
+ ## 2.0.1
97
+
98
+ * 1 new feature
99
+ * Can now define #searchable_language to specify the language used for the query. This changes
100
+ what's considered a stop word on Postgres' side. 'english' is the default language.
101
+ * 1 bugfix
102
+ * We were only specifying a language in to_tsvector() and not in to_tsquery(), which could
103
+ cause queries to fail if the default database language wasn't set to 'english'.
104
+
105
+
106
+ ## 2.0.pre4
107
+
108
+ * 1 new feature
109
+ * Searchable is now available to specify which columns you want searched:
110
+
111
+ ```ruby
112
+ require 'textacular/searchable'
113
+ class Game
114
+ extend Searchable(:title)
115
+ end
116
+ ```
117
+
118
+ This also allows Textacular use in Rails without having #search available to all models:
119
+
120
+ ```
121
+ gem 'textacular', '~> 2.0.pre4', :require => 'textacular/searchable'
122
+ ```
123
+ * 1 bugfix
124
+ * ActiveRecord::Base.extend(Textacular) doesn't break #method_missing and #respond_to? anymore
125
+
126
+
127
+ ## 2.0.pre3
128
+
129
+ * 1 new feature
130
+ * #select calls now limit the columns that are searched
131
+ * 1 bugfix
132
+ * #search calls without an argument assume an empty string as a search term (it errored out previously)
133
+
134
+
135
+ ## 2.0.pre2
136
+
137
+ * 1 bugfix
138
+ * #respond_to? wasn't overwritten correctly
139
+
140
+ ## 2.0.pre
141
+
142
+ * Complete refactoring of Textacular
143
+ * For users:
144
+ * Textacular should only be used for its simplicity; if you need to deeply configure your text search, please give `gem install pg_search` a try.
145
+ * `#search` method is now included in all ActiveRecord models by default, and searches across a model's :string columns.
146
+ * `#search_by_<column>` dynamic methods are now available.
147
+ * `#search` can now be chained; `Game.search_by_title("Street Fighter").search_by_system("PS3")` works.
148
+ * `#search` now accepts a hash to specify columns to be searched, e.g. `Game.search(:name => "Mario")`
149
+ * No more access to `#rank` values for results (though they're still ordered by rank).
150
+ * No way to give different weights to different columns in this release.
151
+ * For devs:
152
+ * We now have actual tests to run against; this will make accepting pull requests much more enjoyable.
153
+
154
+
155
+ ## HEAD (unreleased)
156
+
157
+ * 1 minor bugfix
158
+ * Multiple named indices are now supported.
159
+
160
+
161
+ ## 1.0.4 / 2010-08-19
162
+
163
+ * 2 major enhancements
164
+ * use Rails.root instead of RAILS_ROOT
165
+ * refactored tasks to ease maintainance and patchability
166
+ * 3 minor enhancements
167
+ * fix timestamp for migrationfile
168
+ * fixed deprecation warning for rails3 (dropping rails2-support)
169
+ * prevented warning about defined constant
170
+
171
+
172
+ ## 1.0.3 / 2010-07-07
173
+
174
+ * 1 major enhancement
175
+ * Added Rails 3 support.
176
+ * 1 bugfix
177
+ * Model names that end in double 's's (like Address) don't choke the rake tasks anymore.
178
+
179
+
180
+ ## 1.0.2 / 2009-10-17
181
+
182
+ * 1 bugfix
183
+ * Generated migration now uses UTC time rather than local time.
184
+
185
+
186
+ ## 1.0.1 / 2009-04-14
187
+
188
+ * 1 minor enhancement
189
+ * Textical adds a rake task to generate FTS index migrations. Just run:
190
+
191
+ ```
192
+ rake textical:migration
193
+ ```
194
+
195
+
196
+ ## 1.0.0 / 2009-04-14
197
+
198
+ * 1 major enhancement
199
+ * Birthday!
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
@@ -0,0 +1,196 @@
1
+ # textacular
2
+ [![Gem Version](http://img.shields.io/gem/v/textacular.svg)][rubygems]
3
+ [![Build Status](https://img.shields.io/travis/textacular/textacular/master.svg)][travis]
4
+ [![Code Climate](https://img.shields.io/codeclimate/github/textacular/textacular.svg)][codeclimate]
5
+
6
+ [rubygems]: http://rubygems.org/gems/textacular
7
+ [travis]: https://travis-ci.org/textacular/textacular
8
+ [codeclimate]: https://codeclimate.com/github/textacular/textacular
9
+
10
+ Further documentation available at http://textacular.github.com/textacular.
11
+
12
+
13
+ ## DESCRIPTION:
14
+
15
+ Textacular exposes full text search capabilities from PostgreSQL,
16
+ extending ActiveRecord with scopes making search easy and fun!
17
+
18
+
19
+ ## FEATURES/PROBLEMS:
20
+
21
+ * Only works with PostgreSQL
22
+
23
+
24
+ ## SYNOPSIS:
25
+
26
+ ### Quick Start
27
+
28
+ #### Rails 3 (or 4!)
29
+
30
+ In the project's Gemfile add
31
+
32
+ ```ruby
33
+ gem 'textacular', '~> 3.0'
34
+ ```
35
+
36
+ #### ActiveRecord outside of Rails
37
+
38
+ ```ruby
39
+ require 'textacular'
40
+
41
+ ActiveRecord::Base.extend(Textacular)
42
+ ```
43
+
44
+
45
+ ### Usage
46
+
47
+ Your models now have access to search methods:
48
+
49
+ The `#basic_search` method is what you might expect: it looks literally for what
50
+ you send to it, doing nothing fancy with the input:
51
+
52
+ ```ruby
53
+ Game.basic_search('Sonic') # will search through the model's :string columns
54
+ Game.basic_search(title: 'Mario', system: 'Nintendo')
55
+ ```
56
+
57
+ The `#advanced_search` method lets you use Postgres's search syntax like '|',
58
+ '&' and '!' ('or', 'and', and 'not') as well as some other craziness. The ideal
59
+ use for advanced_search is to take a search DSL you make up for your users and
60
+ translate it to PG's syntax. If for some reason you want to put user input
61
+ directly into an advanced search, you should be sure to catch exceptions from
62
+ syntax errors. Check [the Postgres docs]
63
+ (http://www.postgresql.org/docs/9.2/static/datatype-textsearch.html) for more:
64
+
65
+ ```ruby
66
+ Game.advanced_search(title: 'Street|Fantasy')
67
+ Game.advanced_search(system: '!PS2')
68
+ ```
69
+
70
+ Finally, the `#fuzzy_search` method lets you use Postgres's trigram search
71
+ functionality.
72
+
73
+ In order to use this, you'll need to make sure your database has the `pg_trgm`
74
+ module installed. Create and run a migration to install the module:
75
+
76
+ ```
77
+ rake textacular:create_trigram_migration
78
+ rake db:migrate
79
+ ```
80
+
81
+ Once that's installed, you can use it like this:
82
+
83
+ ```ruby
84
+ Comic.fuzzy_search(title: 'Questio') # matches Questionable Content
85
+ ```
86
+
87
+ Note that fuzzy searches are subject to a similarity threshold imposed by the `pg_trgm` module. The default is 0.3, meaning that at least 30% of the total string must match your search content. For example:
88
+
89
+ ```ruby
90
+ Comic.fuzzy_search(title: 'Pearls') # matches Pearls Before Swine
91
+ Comic.fuzzy_search(title: 'Pear') # does not match Pearls Before Swine
92
+ ```
93
+
94
+ The similarity threshold is hardcoded in PostgreSQL and can be modified on a per-connection basis, for example:
95
+
96
+ ```ruby
97
+ ActiveRecord::Base.connection.execute("SELECT set_limit(0.9);")
98
+ ```
99
+
100
+ For more info, view the `pg_trgm` documentation, specifically [F.35.2. Functions and Operators](http://www.postgresql.org/docs/9.1/static/pgtrgm.html).
101
+
102
+ Searches are also chainable:
103
+
104
+ ```ruby
105
+ Game.fuzzy_search(title: 'tree').basic_search(system: 'SNES')
106
+ ```
107
+
108
+ If you want to search on two or more fields with the OR operator use a hash for
109
+ the conditions and pass false as the second parameter:
110
+
111
+ ```ruby
112
+ Game.basic_search({name: 'Mario', nickname: 'Mario'}, false)
113
+ ```
114
+
115
+
116
+ ### Setting Language
117
+
118
+ To set proper searching dictionary just override class method on your model:
119
+
120
+ ```ruby
121
+ def self.searchable_language
122
+ 'russian'
123
+ end
124
+ ```
125
+
126
+ And all your queries would go right! And don`t forget to change the migration for indexes, like shown below.
127
+
128
+
129
+ ### Creating Indexes for Super Speed
130
+ You can have Postgresql use an index for the full-text search. To declare a full-text index, in a
131
+ migration add code like the following:
132
+
133
+ ```ruby
134
+ execute "
135
+ create index on email_logs using gin(to_tsvector('english', subject));
136
+ create index on email_logs using gin(to_tsvector('english', email_address));"
137
+ ```
138
+
139
+ In the above example, the table email_logs has two text columns that we search against, subject and email_address.
140
+ You will need to add an index for every text/string column you query against, or else Postgresql will revert to a
141
+ full table scan instead of using the indexes.
142
+
143
+ If you create these indexes, you should also switch to sql for your schema_format in `config/application.rb`:
144
+
145
+ ```ruby
146
+ config.active_record.schema_format = :sql
147
+ ```
148
+
149
+
150
+ ## REQUIREMENTS:
151
+
152
+ * ActiveRecord
153
+ * Ruby 1.9.2
154
+
155
+
156
+ ## INSTALL:
157
+
158
+ ```
159
+ $ gem install textacular
160
+ ```
161
+
162
+ ## Contributing
163
+
164
+ If you'd like to contribute, please see the [contribution guidelines](CONTRIBUTING.md).
165
+
166
+
167
+ ## Releasing
168
+
169
+ Maintainers: Please make sure to follow the [release steps](RELEASING.md) when
170
+ it's time to cut a new release.
171
+
172
+
173
+ ## LICENSE:
174
+
175
+ (The MIT License)
176
+
177
+ Copyright (c) 2011 Aaron Patterson
178
+
179
+ Permission is hereby granted, free of charge, to any person obtaining
180
+ a copy of this software and associated documentation files (the
181
+ 'Software'), to deal in the Software without restriction, including
182
+ without limitation the rights to use, copy, modify, merge, publish,
183
+ distribute, sublicense, and/or sell copies of the Software, and to
184
+ permit persons to whom the Software is furnished to do so, subject to
185
+ the following conditions:
186
+
187
+ The above copyright notice and this permission notice shall be
188
+ included in all copies or substantial portions of the Software.
189
+
190
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
191
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
192
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
193
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
194
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
195
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
196
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,61 @@
1
+ require "bundler/gem_tasks"
2
+
3
+ require 'active_record'
4
+ require 'pry'
5
+
6
+ require 'rspec/core/rake_task'
7
+
8
+ RSpec::Core::RakeTask.new(:spec)
9
+
10
+ task :default => :spec
11
+
12
+ file 'spec/config.yml' do |t|
13
+ sh 'erb spec/config.yml.example > spec/config.yml'
14
+ end
15
+
16
+ desc 'Fire up an interactive terminal to play with'
17
+ task :console => :'db:connect' do
18
+ Pry.start
19
+ end
20
+
21
+ namespace :db do
22
+
23
+ task :connect => 'spec/config.yml' do |t|
24
+ ActiveRecord::Base.establish_connection \
25
+ YAML.load_file 'spec/config.yml'
26
+ end
27
+
28
+ task :disconnect do
29
+ ActiveRecord::Base.clear_all_connections!
30
+ end
31
+
32
+ desc 'Create the test database'
33
+ task :create do
34
+ sh 'createdb textacular'
35
+ end
36
+
37
+ desc 'Drop the test database'
38
+ task :drop => :disconnect do
39
+ sh 'dropdb textacular'
40
+ end
41
+
42
+ namespace :migrate do
43
+
44
+ desc 'Run the test database migrations'
45
+ task :up => :'db:connect' do
46
+ ActiveRecord::Migrator.up 'db/migrate'
47
+ end
48
+
49
+ desc 'Reverse the test database migrations'
50
+ task :down => :'db:connect' do
51
+ ActiveRecord::Migrator.down 'db/migrate'
52
+ end
53
+ end
54
+ task :migrate => :'migrate:up'
55
+
56
+ desc 'Create and configure the test database'
57
+ task :setup => [ :create, :migrate ]
58
+
59
+ desc 'Drop the test tables and database'
60
+ task :teardown => [ :'migrate:down', :drop ]
61
+ end