drtom-textacular 4.0.0.alpha.20160302

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. checksums.yaml +7 -0
  2. data/CHANGELOG.md +199 -0
  3. data/Gemfile +3 -0
  4. data/README.md +196 -0
  5. data/Rakefile +61 -0
  6. data/lib/textacular.rb +160 -0
  7. data/lib/textacular/full_text_indexer.rb +66 -0
  8. data/lib/textacular/migration_generator.rb +31 -0
  9. data/lib/textacular/postgres_module_installer.rb +57 -0
  10. data/lib/textacular/rails.rb +14 -0
  11. data/lib/textacular/searchable.rb +20 -0
  12. data/lib/textacular/tasks.rb +23 -0
  13. data/lib/textacular/trigram_installer.rb +18 -0
  14. data/lib/textacular/version.rb +7 -0
  15. data/spec/config.travis.yml +8 -0
  16. data/spec/config.yml.example +5 -0
  17. data/spec/spec_helper.rb +104 -0
  18. data/spec/support/ar_stand_in.rb +4 -0
  19. data/spec/support/character.rb +7 -0
  20. data/spec/support/game.rb +5 -0
  21. data/spec/support/game_extended_with_textacular.rb +5 -0
  22. data/spec/support/game_extended_with_textacular_and_custom_language.rb +7 -0
  23. data/spec/support/game_fail.rb +3 -0
  24. data/spec/support/game_fail_extended_with_textacular.rb +5 -0
  25. data/spec/support/not_there.rb +3 -0
  26. data/spec/support/textacular_web_comic.rb +7 -0
  27. data/spec/support/web_comic.rb +7 -0
  28. data/spec/support/web_comic_with_searchable.rb +6 -0
  29. data/spec/support/web_comic_with_searchable_name.rb +6 -0
  30. data/spec/support/web_comic_with_searchable_name_and_author.rb +6 -0
  31. data/spec/textacular/full_text_indexer_spec.rb +69 -0
  32. data/spec/textacular/migration_generator_spec.rb +67 -0
  33. data/spec/textacular/searchable_spec.rb +194 -0
  34. data/spec/textacular/trigram_installer_spec.rb +24 -0
  35. data/spec/textacular_spec.rb +287 -0
  36. metadata +210 -0
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 42f77ab66144c7716a06bc4e88e1810752077744
4
+ data.tar.gz: 892309c1127a3ad16b84661c5b16f597d49950b7
5
+ SHA512:
6
+ metadata.gz: 96c3abd3b0e03a58450674c632f933df4cde4cf42f4ca38053ddf184c66a846315e322b09568c2f97ef4af3d42b291e4e770b602db0d9619880f54117b2a9f71
7
+ data.tar.gz: d53292f2c1777ec2322fa6225c715bfdec0ee1aba10d6903647a12aec308dec9e1d0e71099dc05c512125c7c116b93cec24d5f2c6dc3696df2b4068634b48cda
@@ -0,0 +1,199 @@
1
+ # Change Log
2
+
3
+ ## Unreleased
4
+
5
+ ## 3.2.2
6
+
7
+ * Fallback to zero similarity when trying to match null values
8
+ * Fix search term escaping
9
+ * Trigram installation migration only install extension if not installed yet.
10
+ * Update the README to illustrate how to change the similarity threshold.
11
+ * Remove deprecated dynamic search methods.
12
+
13
+ ## 3.2.1
14
+
15
+ * Fix trigram installation migration reversed filename and content.
16
+ * Rewrite all tests using RSpec.
17
+ * We're ActiveRecord 4.2+ compatible until tests prove otherwise.
18
+
19
+ ## 3.2.0
20
+
21
+ * Add generator for trigram installation migration.
22
+ * Expand gemspec to allow newest version of ActiveRecord.
23
+
24
+ ## 3.1.0
25
+
26
+ * Avoid Deprecation warnings from ActiveRecord 4.0.0.rc2.
27
+ * Fix `method_missing` in ActiveRecord 4.0.0.rc2.
28
+ * Remove unused `Textacular#normalize` method.
29
+ * Add `OR` example to the README.
30
+ * Fix tests for Ruby 2.0.0 & related improvements.
31
+ * Improve Rails integration.
32
+ * Fix dependency loading for textacular rake tasks.
33
+ * Fix ranking failures when rows had `NULL` column values.
34
+ * Clean up Rakefile; should make developing the gem nicer.
35
+ * DEPRECATION: The dynamic search helpers will be removed at the next major
36
+ release.
37
+
38
+ ## 3.0.0
39
+
40
+ * All deprecations have been resolved. This breaks backwards compatibility.
41
+ * Rename gem to Textacular.
42
+
43
+ ## 2.2.0
44
+
45
+ * 1 DEPRECATION
46
+ * The whole gem is being renamed and will no longer be maintained as Texticle.
47
+ The new name it Textacular.
48
+ * 1 new feature
49
+ * Expand gemspec to allow Rails 4.
50
+
51
+ ## 2.1.1
52
+
53
+ * 1 bugfix
54
+ * Include `lib/textacular/version.rb` in the gemspec so the gem will load. Sorry!
55
+
56
+
57
+ ## 2.1.0
58
+
59
+ * 1 DEPRECATION
60
+ * `search` aliases new `advanced_search` method (same functionality as before), but will
61
+ alias `basic_search` in 3.0! Should print warnings.
62
+ * 3 new features
63
+ * Generate full text search indexes from a rake task (sort of like in 1.x). Supply a specific
64
+ model name.
65
+ * New search methods: `basic_search`, `advanced_search` and `fuzzy_search`. Basic allows special
66
+ characters like &, and % in search terms. Fuzzy is based on Postgres's trigram matching extension
67
+ pg_trgm. Advanced is the same functionality from `search` previously.
68
+ * Rake task that installs pg_trgm now works on Postgres 9.1 and up.
69
+ * 2 dev improvements
70
+ * Test database configuration not automatically generated from a rake task and ignored by git.
71
+ * New interactive developer console (powered by pry).
72
+
73
+
74
+ ## 2.0.3
75
+
76
+ * 1 new feature
77
+ * Allow searching through relations. Model.join(:relation).search(:relation => {:column => "query"})
78
+ works, and reduces the need for multi-model tables. Huge thanks to Ben Hamill for the pull request.
79
+ * Allow searching through all model columns irrespective of the column's type; we cast all columns to text
80
+ in the search query. Performance may degrade when searching through anything but a string column.
81
+ * 2 bugfixes
82
+ * Fix exceptions when adding Textacular to a table-less model.
83
+ * Column names in a search query are now scoped to the current table.
84
+ * 1 dev improvement
85
+ * Running `rake` from the project root will setup the test environment by creating a test database
86
+ and running the necessary migrations. `rake` can also be used to run all the project tests.
87
+
88
+
89
+ ## 2.0.2
90
+
91
+ * 1 bugfix
92
+ * Our #respond_to? overwritten method was causing failures when a model didn't have
93
+ a table (e.g. if migrations hadn't been run yet). Not the case anymore.
94
+
95
+
96
+ ## 2.0.1
97
+
98
+ * 1 new feature
99
+ * Can now define #searchable_language to specify the language used for the query. This changes
100
+ what's considered a stop word on Postgres' side. 'english' is the default language.
101
+ * 1 bugfix
102
+ * We were only specifying a language in to_tsvector() and not in to_tsquery(), which could
103
+ cause queries to fail if the default database language wasn't set to 'english'.
104
+
105
+
106
+ ## 2.0.pre4
107
+
108
+ * 1 new feature
109
+ * Searchable is now available to specify which columns you want searched:
110
+
111
+ ```ruby
112
+ require 'textacular/searchable'
113
+ class Game
114
+ extend Searchable(:title)
115
+ end
116
+ ```
117
+
118
+ This also allows Textacular use in Rails without having #search available to all models:
119
+
120
+ ```
121
+ gem 'textacular', '~> 2.0.pre4', :require => 'textacular/searchable'
122
+ ```
123
+ * 1 bugfix
124
+ * ActiveRecord::Base.extend(Textacular) doesn't break #method_missing and #respond_to? anymore
125
+
126
+
127
+ ## 2.0.pre3
128
+
129
+ * 1 new feature
130
+ * #select calls now limit the columns that are searched
131
+ * 1 bugfix
132
+ * #search calls without an argument assume an empty string as a search term (it errored out previously)
133
+
134
+
135
+ ## 2.0.pre2
136
+
137
+ * 1 bugfix
138
+ * #respond_to? wasn't overwritten correctly
139
+
140
+ ## 2.0.pre
141
+
142
+ * Complete refactoring of Textacular
143
+ * For users:
144
+ * Textacular should only be used for its simplicity; if you need to deeply configure your text search, please give `gem install pg_search` a try.
145
+ * `#search` method is now included in all ActiveRecord models by default, and searches across a model's :string columns.
146
+ * `#search_by_<column>` dynamic methods are now available.
147
+ * `#search` can now be chained; `Game.search_by_title("Street Fighter").search_by_system("PS3")` works.
148
+ * `#search` now accepts a hash to specify columns to be searched, e.g. `Game.search(:name => "Mario")`
149
+ * No more access to `#rank` values for results (though they're still ordered by rank).
150
+ * No way to give different weights to different columns in this release.
151
+ * For devs:
152
+ * We now have actual tests to run against; this will make accepting pull requests much more enjoyable.
153
+
154
+
155
+ ## HEAD (unreleased)
156
+
157
+ * 1 minor bugfix
158
+ * Multiple named indices are now supported.
159
+
160
+
161
+ ## 1.0.4 / 2010-08-19
162
+
163
+ * 2 major enhancements
164
+ * use Rails.root instead of RAILS_ROOT
165
+ * refactored tasks to ease maintainance and patchability
166
+ * 3 minor enhancements
167
+ * fix timestamp for migrationfile
168
+ * fixed deprecation warning for rails3 (dropping rails2-support)
169
+ * prevented warning about defined constant
170
+
171
+
172
+ ## 1.0.3 / 2010-07-07
173
+
174
+ * 1 major enhancement
175
+ * Added Rails 3 support.
176
+ * 1 bugfix
177
+ * Model names that end in double 's's (like Address) don't choke the rake tasks anymore.
178
+
179
+
180
+ ## 1.0.2 / 2009-10-17
181
+
182
+ * 1 bugfix
183
+ * Generated migration now uses UTC time rather than local time.
184
+
185
+
186
+ ## 1.0.1 / 2009-04-14
187
+
188
+ * 1 minor enhancement
189
+ * Textical adds a rake task to generate FTS index migrations. Just run:
190
+
191
+ ```
192
+ rake textical:migration
193
+ ```
194
+
195
+
196
+ ## 1.0.0 / 2009-04-14
197
+
198
+ * 1 major enhancement
199
+ * Birthday!
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
@@ -0,0 +1,196 @@
1
+ # textacular
2
+ [![Gem Version](http://img.shields.io/gem/v/textacular.svg)][rubygems]
3
+ [![Build Status](https://img.shields.io/travis/textacular/textacular/master.svg)][travis]
4
+ [![Code Climate](https://img.shields.io/codeclimate/github/textacular/textacular.svg)][codeclimate]
5
+
6
+ [rubygems]: http://rubygems.org/gems/textacular
7
+ [travis]: https://travis-ci.org/textacular/textacular
8
+ [codeclimate]: https://codeclimate.com/github/textacular/textacular
9
+
10
+ Further documentation available at http://textacular.github.com/textacular.
11
+
12
+
13
+ ## DESCRIPTION:
14
+
15
+ Textacular exposes full text search capabilities from PostgreSQL,
16
+ extending ActiveRecord with scopes making search easy and fun!
17
+
18
+
19
+ ## FEATURES/PROBLEMS:
20
+
21
+ * Only works with PostgreSQL
22
+
23
+
24
+ ## SYNOPSIS:
25
+
26
+ ### Quick Start
27
+
28
+ #### Rails 3 (or 4!)
29
+
30
+ In the project's Gemfile add
31
+
32
+ ```ruby
33
+ gem 'textacular', '~> 3.0'
34
+ ```
35
+
36
+ #### ActiveRecord outside of Rails
37
+
38
+ ```ruby
39
+ require 'textacular'
40
+
41
+ ActiveRecord::Base.extend(Textacular)
42
+ ```
43
+
44
+
45
+ ### Usage
46
+
47
+ Your models now have access to search methods:
48
+
49
+ The `#basic_search` method is what you might expect: it looks literally for what
50
+ you send to it, doing nothing fancy with the input:
51
+
52
+ ```ruby
53
+ Game.basic_search('Sonic') # will search through the model's :string columns
54
+ Game.basic_search(title: 'Mario', system: 'Nintendo')
55
+ ```
56
+
57
+ The `#advanced_search` method lets you use Postgres's search syntax like '|',
58
+ '&' and '!' ('or', 'and', and 'not') as well as some other craziness. The ideal
59
+ use for advanced_search is to take a search DSL you make up for your users and
60
+ translate it to PG's syntax. If for some reason you want to put user input
61
+ directly into an advanced search, you should be sure to catch exceptions from
62
+ syntax errors. Check [the Postgres docs]
63
+ (http://www.postgresql.org/docs/9.2/static/datatype-textsearch.html) for more:
64
+
65
+ ```ruby
66
+ Game.advanced_search(title: 'Street|Fantasy')
67
+ Game.advanced_search(system: '!PS2')
68
+ ```
69
+
70
+ Finally, the `#fuzzy_search` method lets you use Postgres's trigram search
71
+ functionality.
72
+
73
+ In order to use this, you'll need to make sure your database has the `pg_trgm`
74
+ module installed. Create and run a migration to install the module:
75
+
76
+ ```
77
+ rake textacular:create_trigram_migration
78
+ rake db:migrate
79
+ ```
80
+
81
+ Once that's installed, you can use it like this:
82
+
83
+ ```ruby
84
+ Comic.fuzzy_search(title: 'Questio') # matches Questionable Content
85
+ ```
86
+
87
+ Note that fuzzy searches are subject to a similarity threshold imposed by the `pg_trgm` module. The default is 0.3, meaning that at least 30% of the total string must match your search content. For example:
88
+
89
+ ```ruby
90
+ Comic.fuzzy_search(title: 'Pearls') # matches Pearls Before Swine
91
+ Comic.fuzzy_search(title: 'Pear') # does not match Pearls Before Swine
92
+ ```
93
+
94
+ The similarity threshold is hardcoded in PostgreSQL and can be modified on a per-connection basis, for example:
95
+
96
+ ```ruby
97
+ ActiveRecord::Base.connection.execute("SELECT set_limit(0.9);")
98
+ ```
99
+
100
+ For more info, view the `pg_trgm` documentation, specifically [F.35.2. Functions and Operators](http://www.postgresql.org/docs/9.1/static/pgtrgm.html).
101
+
102
+ Searches are also chainable:
103
+
104
+ ```ruby
105
+ Game.fuzzy_search(title: 'tree').basic_search(system: 'SNES')
106
+ ```
107
+
108
+ If you want to search on two or more fields with the OR operator use a hash for
109
+ the conditions and pass false as the second parameter:
110
+
111
+ ```ruby
112
+ Game.basic_search({name: 'Mario', nickname: 'Mario'}, false)
113
+ ```
114
+
115
+
116
+ ### Setting Language
117
+
118
+ To set proper searching dictionary just override class method on your model:
119
+
120
+ ```ruby
121
+ def self.searchable_language
122
+ 'russian'
123
+ end
124
+ ```
125
+
126
+ And all your queries would go right! And don`t forget to change the migration for indexes, like shown below.
127
+
128
+
129
+ ### Creating Indexes for Super Speed
130
+ You can have Postgresql use an index for the full-text search. To declare a full-text index, in a
131
+ migration add code like the following:
132
+
133
+ ```ruby
134
+ execute "
135
+ create index on email_logs using gin(to_tsvector('english', subject));
136
+ create index on email_logs using gin(to_tsvector('english', email_address));"
137
+ ```
138
+
139
+ In the above example, the table email_logs has two text columns that we search against, subject and email_address.
140
+ You will need to add an index for every text/string column you query against, or else Postgresql will revert to a
141
+ full table scan instead of using the indexes.
142
+
143
+ If you create these indexes, you should also switch to sql for your schema_format in `config/application.rb`:
144
+
145
+ ```ruby
146
+ config.active_record.schema_format = :sql
147
+ ```
148
+
149
+
150
+ ## REQUIREMENTS:
151
+
152
+ * ActiveRecord
153
+ * Ruby 1.9.2
154
+
155
+
156
+ ## INSTALL:
157
+
158
+ ```
159
+ $ gem install textacular
160
+ ```
161
+
162
+ ## Contributing
163
+
164
+ If you'd like to contribute, please see the [contribution guidelines](CONTRIBUTING.md).
165
+
166
+
167
+ ## Releasing
168
+
169
+ Maintainers: Please make sure to follow the [release steps](RELEASING.md) when
170
+ it's time to cut a new release.
171
+
172
+
173
+ ## LICENSE:
174
+
175
+ (The MIT License)
176
+
177
+ Copyright (c) 2011 Aaron Patterson
178
+
179
+ Permission is hereby granted, free of charge, to any person obtaining
180
+ a copy of this software and associated documentation files (the
181
+ 'Software'), to deal in the Software without restriction, including
182
+ without limitation the rights to use, copy, modify, merge, publish,
183
+ distribute, sublicense, and/or sell copies of the Software, and to
184
+ permit persons to whom the Software is furnished to do so, subject to
185
+ the following conditions:
186
+
187
+ The above copyright notice and this permission notice shall be
188
+ included in all copies or substantial portions of the Software.
189
+
190
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
191
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
192
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
193
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
194
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
195
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
196
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,61 @@
1
+ require "bundler/gem_tasks"
2
+
3
+ require 'active_record'
4
+ require 'pry'
5
+
6
+ require 'rspec/core/rake_task'
7
+
8
+ RSpec::Core::RakeTask.new(:spec)
9
+
10
+ task :default => :spec
11
+
12
+ file 'spec/config.yml' do |t|
13
+ sh 'erb spec/config.yml.example > spec/config.yml'
14
+ end
15
+
16
+ desc 'Fire up an interactive terminal to play with'
17
+ task :console => :'db:connect' do
18
+ Pry.start
19
+ end
20
+
21
+ namespace :db do
22
+
23
+ task :connect => 'spec/config.yml' do |t|
24
+ ActiveRecord::Base.establish_connection \
25
+ YAML.load_file 'spec/config.yml'
26
+ end
27
+
28
+ task :disconnect do
29
+ ActiveRecord::Base.clear_all_connections!
30
+ end
31
+
32
+ desc 'Create the test database'
33
+ task :create do
34
+ sh 'createdb textacular'
35
+ end
36
+
37
+ desc 'Drop the test database'
38
+ task :drop => :disconnect do
39
+ sh 'dropdb textacular'
40
+ end
41
+
42
+ namespace :migrate do
43
+
44
+ desc 'Run the test database migrations'
45
+ task :up => :'db:connect' do
46
+ ActiveRecord::Migrator.up 'db/migrate'
47
+ end
48
+
49
+ desc 'Reverse the test database migrations'
50
+ task :down => :'db:connect' do
51
+ ActiveRecord::Migrator.down 'db/migrate'
52
+ end
53
+ end
54
+ task :migrate => :'migrate:up'
55
+
56
+ desc 'Create and configure the test database'
57
+ task :setup => [ :create, :migrate ]
58
+
59
+ desc 'Drop the test tables and database'
60
+ task :teardown => [ :'migrate:down', :drop ]
61
+ end