drtom-textacular 4.0.0.alpha.20160302
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +199 -0
- data/Gemfile +3 -0
- data/README.md +196 -0
- data/Rakefile +61 -0
- data/lib/textacular.rb +160 -0
- data/lib/textacular/full_text_indexer.rb +66 -0
- data/lib/textacular/migration_generator.rb +31 -0
- data/lib/textacular/postgres_module_installer.rb +57 -0
- data/lib/textacular/rails.rb +14 -0
- data/lib/textacular/searchable.rb +20 -0
- data/lib/textacular/tasks.rb +23 -0
- data/lib/textacular/trigram_installer.rb +18 -0
- data/lib/textacular/version.rb +7 -0
- data/spec/config.travis.yml +8 -0
- data/spec/config.yml.example +5 -0
- data/spec/spec_helper.rb +104 -0
- data/spec/support/ar_stand_in.rb +4 -0
- data/spec/support/character.rb +7 -0
- data/spec/support/game.rb +5 -0
- data/spec/support/game_extended_with_textacular.rb +5 -0
- data/spec/support/game_extended_with_textacular_and_custom_language.rb +7 -0
- data/spec/support/game_fail.rb +3 -0
- data/spec/support/game_fail_extended_with_textacular.rb +5 -0
- data/spec/support/not_there.rb +3 -0
- data/spec/support/textacular_web_comic.rb +7 -0
- data/spec/support/web_comic.rb +7 -0
- data/spec/support/web_comic_with_searchable.rb +6 -0
- data/spec/support/web_comic_with_searchable_name.rb +6 -0
- data/spec/support/web_comic_with_searchable_name_and_author.rb +6 -0
- data/spec/textacular/full_text_indexer_spec.rb +69 -0
- data/spec/textacular/migration_generator_spec.rb +67 -0
- data/spec/textacular/searchable_spec.rb +194 -0
- data/spec/textacular/trigram_installer_spec.rb +24 -0
- data/spec/textacular_spec.rb +287 -0
- metadata +210 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 42f77ab66144c7716a06bc4e88e1810752077744
|
4
|
+
data.tar.gz: 892309c1127a3ad16b84661c5b16f597d49950b7
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 96c3abd3b0e03a58450674c632f933df4cde4cf42f4ca38053ddf184c66a846315e322b09568c2f97ef4af3d42b291e4e770b602db0d9619880f54117b2a9f71
|
7
|
+
data.tar.gz: d53292f2c1777ec2322fa6225c715bfdec0ee1aba10d6903647a12aec308dec9e1d0e71099dc05c512125c7c116b93cec24d5f2c6dc3696df2b4068634b48cda
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,199 @@
|
|
1
|
+
# Change Log
|
2
|
+
|
3
|
+
## Unreleased
|
4
|
+
|
5
|
+
## 3.2.2
|
6
|
+
|
7
|
+
* Fallback to zero similarity when trying to match null values
|
8
|
+
* Fix search term escaping
|
9
|
+
* Trigram installation migration only install extension if not installed yet.
|
10
|
+
* Update the README to illustrate how to change the similarity threshold.
|
11
|
+
* Remove deprecated dynamic search methods.
|
12
|
+
|
13
|
+
## 3.2.1
|
14
|
+
|
15
|
+
* Fix trigram installation migration reversed filename and content.
|
16
|
+
* Rewrite all tests using RSpec.
|
17
|
+
* We're ActiveRecord 4.2+ compatible until tests prove otherwise.
|
18
|
+
|
19
|
+
## 3.2.0
|
20
|
+
|
21
|
+
* Add generator for trigram installation migration.
|
22
|
+
* Expand gemspec to allow newest version of ActiveRecord.
|
23
|
+
|
24
|
+
## 3.1.0
|
25
|
+
|
26
|
+
* Avoid Deprecation warnings from ActiveRecord 4.0.0.rc2.
|
27
|
+
* Fix `method_missing` in ActiveRecord 4.0.0.rc2.
|
28
|
+
* Remove unused `Textacular#normalize` method.
|
29
|
+
* Add `OR` example to the README.
|
30
|
+
* Fix tests for Ruby 2.0.0 & related improvements.
|
31
|
+
* Improve Rails integration.
|
32
|
+
* Fix dependency loading for textacular rake tasks.
|
33
|
+
* Fix ranking failures when rows had `NULL` column values.
|
34
|
+
* Clean up Rakefile; should make developing the gem nicer.
|
35
|
+
* DEPRECATION: The dynamic search helpers will be removed at the next major
|
36
|
+
release.
|
37
|
+
|
38
|
+
## 3.0.0
|
39
|
+
|
40
|
+
* All deprecations have been resolved. This breaks backwards compatibility.
|
41
|
+
* Rename gem to Textacular.
|
42
|
+
|
43
|
+
## 2.2.0
|
44
|
+
|
45
|
+
* 1 DEPRECATION
|
46
|
+
* The whole gem is being renamed and will no longer be maintained as Texticle.
|
47
|
+
The new name it Textacular.
|
48
|
+
* 1 new feature
|
49
|
+
* Expand gemspec to allow Rails 4.
|
50
|
+
|
51
|
+
## 2.1.1
|
52
|
+
|
53
|
+
* 1 bugfix
|
54
|
+
* Include `lib/textacular/version.rb` in the gemspec so the gem will load. Sorry!
|
55
|
+
|
56
|
+
|
57
|
+
## 2.1.0
|
58
|
+
|
59
|
+
* 1 DEPRECATION
|
60
|
+
* `search` aliases new `advanced_search` method (same functionality as before), but will
|
61
|
+
alias `basic_search` in 3.0! Should print warnings.
|
62
|
+
* 3 new features
|
63
|
+
* Generate full text search indexes from a rake task (sort of like in 1.x). Supply a specific
|
64
|
+
model name.
|
65
|
+
* New search methods: `basic_search`, `advanced_search` and `fuzzy_search`. Basic allows special
|
66
|
+
characters like &, and % in search terms. Fuzzy is based on Postgres's trigram matching extension
|
67
|
+
pg_trgm. Advanced is the same functionality from `search` previously.
|
68
|
+
* Rake task that installs pg_trgm now works on Postgres 9.1 and up.
|
69
|
+
* 2 dev improvements
|
70
|
+
* Test database configuration not automatically generated from a rake task and ignored by git.
|
71
|
+
* New interactive developer console (powered by pry).
|
72
|
+
|
73
|
+
|
74
|
+
## 2.0.3
|
75
|
+
|
76
|
+
* 1 new feature
|
77
|
+
* Allow searching through relations. Model.join(:relation).search(:relation => {:column => "query"})
|
78
|
+
works, and reduces the need for multi-model tables. Huge thanks to Ben Hamill for the pull request.
|
79
|
+
* Allow searching through all model columns irrespective of the column's type; we cast all columns to text
|
80
|
+
in the search query. Performance may degrade when searching through anything but a string column.
|
81
|
+
* 2 bugfixes
|
82
|
+
* Fix exceptions when adding Textacular to a table-less model.
|
83
|
+
* Column names in a search query are now scoped to the current table.
|
84
|
+
* 1 dev improvement
|
85
|
+
* Running `rake` from the project root will setup the test environment by creating a test database
|
86
|
+
and running the necessary migrations. `rake` can also be used to run all the project tests.
|
87
|
+
|
88
|
+
|
89
|
+
## 2.0.2
|
90
|
+
|
91
|
+
* 1 bugfix
|
92
|
+
* Our #respond_to? overwritten method was causing failures when a model didn't have
|
93
|
+
a table (e.g. if migrations hadn't been run yet). Not the case anymore.
|
94
|
+
|
95
|
+
|
96
|
+
## 2.0.1
|
97
|
+
|
98
|
+
* 1 new feature
|
99
|
+
* Can now define #searchable_language to specify the language used for the query. This changes
|
100
|
+
what's considered a stop word on Postgres' side. 'english' is the default language.
|
101
|
+
* 1 bugfix
|
102
|
+
* We were only specifying a language in to_tsvector() and not in to_tsquery(), which could
|
103
|
+
cause queries to fail if the default database language wasn't set to 'english'.
|
104
|
+
|
105
|
+
|
106
|
+
## 2.0.pre4
|
107
|
+
|
108
|
+
* 1 new feature
|
109
|
+
* Searchable is now available to specify which columns you want searched:
|
110
|
+
|
111
|
+
```ruby
|
112
|
+
require 'textacular/searchable'
|
113
|
+
class Game
|
114
|
+
extend Searchable(:title)
|
115
|
+
end
|
116
|
+
```
|
117
|
+
|
118
|
+
This also allows Textacular use in Rails without having #search available to all models:
|
119
|
+
|
120
|
+
```
|
121
|
+
gem 'textacular', '~> 2.0.pre4', :require => 'textacular/searchable'
|
122
|
+
```
|
123
|
+
* 1 bugfix
|
124
|
+
* ActiveRecord::Base.extend(Textacular) doesn't break #method_missing and #respond_to? anymore
|
125
|
+
|
126
|
+
|
127
|
+
## 2.0.pre3
|
128
|
+
|
129
|
+
* 1 new feature
|
130
|
+
* #select calls now limit the columns that are searched
|
131
|
+
* 1 bugfix
|
132
|
+
* #search calls without an argument assume an empty string as a search term (it errored out previously)
|
133
|
+
|
134
|
+
|
135
|
+
## 2.0.pre2
|
136
|
+
|
137
|
+
* 1 bugfix
|
138
|
+
* #respond_to? wasn't overwritten correctly
|
139
|
+
|
140
|
+
## 2.0.pre
|
141
|
+
|
142
|
+
* Complete refactoring of Textacular
|
143
|
+
* For users:
|
144
|
+
* Textacular should only be used for its simplicity; if you need to deeply configure your text search, please give `gem install pg_search` a try.
|
145
|
+
* `#search` method is now included in all ActiveRecord models by default, and searches across a model's :string columns.
|
146
|
+
* `#search_by_<column>` dynamic methods are now available.
|
147
|
+
* `#search` can now be chained; `Game.search_by_title("Street Fighter").search_by_system("PS3")` works.
|
148
|
+
* `#search` now accepts a hash to specify columns to be searched, e.g. `Game.search(:name => "Mario")`
|
149
|
+
* No more access to `#rank` values for results (though they're still ordered by rank).
|
150
|
+
* No way to give different weights to different columns in this release.
|
151
|
+
* For devs:
|
152
|
+
* We now have actual tests to run against; this will make accepting pull requests much more enjoyable.
|
153
|
+
|
154
|
+
|
155
|
+
## HEAD (unreleased)
|
156
|
+
|
157
|
+
* 1 minor bugfix
|
158
|
+
* Multiple named indices are now supported.
|
159
|
+
|
160
|
+
|
161
|
+
## 1.0.4 / 2010-08-19
|
162
|
+
|
163
|
+
* 2 major enhancements
|
164
|
+
* use Rails.root instead of RAILS_ROOT
|
165
|
+
* refactored tasks to ease maintainance and patchability
|
166
|
+
* 3 minor enhancements
|
167
|
+
* fix timestamp for migrationfile
|
168
|
+
* fixed deprecation warning for rails3 (dropping rails2-support)
|
169
|
+
* prevented warning about defined constant
|
170
|
+
|
171
|
+
|
172
|
+
## 1.0.3 / 2010-07-07
|
173
|
+
|
174
|
+
* 1 major enhancement
|
175
|
+
* Added Rails 3 support.
|
176
|
+
* 1 bugfix
|
177
|
+
* Model names that end in double 's's (like Address) don't choke the rake tasks anymore.
|
178
|
+
|
179
|
+
|
180
|
+
## 1.0.2 / 2009-10-17
|
181
|
+
|
182
|
+
* 1 bugfix
|
183
|
+
* Generated migration now uses UTC time rather than local time.
|
184
|
+
|
185
|
+
|
186
|
+
## 1.0.1 / 2009-04-14
|
187
|
+
|
188
|
+
* 1 minor enhancement
|
189
|
+
* Textical adds a rake task to generate FTS index migrations. Just run:
|
190
|
+
|
191
|
+
```
|
192
|
+
rake textical:migration
|
193
|
+
```
|
194
|
+
|
195
|
+
|
196
|
+
## 1.0.0 / 2009-04-14
|
197
|
+
|
198
|
+
* 1 major enhancement
|
199
|
+
* Birthday!
|
data/Gemfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,196 @@
|
|
1
|
+
# textacular
|
2
|
+
[][rubygems]
|
3
|
+
[][travis]
|
4
|
+
[][codeclimate]
|
5
|
+
|
6
|
+
[rubygems]: http://rubygems.org/gems/textacular
|
7
|
+
[travis]: https://travis-ci.org/textacular/textacular
|
8
|
+
[codeclimate]: https://codeclimate.com/github/textacular/textacular
|
9
|
+
|
10
|
+
Further documentation available at http://textacular.github.com/textacular.
|
11
|
+
|
12
|
+
|
13
|
+
## DESCRIPTION:
|
14
|
+
|
15
|
+
Textacular exposes full text search capabilities from PostgreSQL,
|
16
|
+
extending ActiveRecord with scopes making search easy and fun!
|
17
|
+
|
18
|
+
|
19
|
+
## FEATURES/PROBLEMS:
|
20
|
+
|
21
|
+
* Only works with PostgreSQL
|
22
|
+
|
23
|
+
|
24
|
+
## SYNOPSIS:
|
25
|
+
|
26
|
+
### Quick Start
|
27
|
+
|
28
|
+
#### Rails 3 (or 4!)
|
29
|
+
|
30
|
+
In the project's Gemfile add
|
31
|
+
|
32
|
+
```ruby
|
33
|
+
gem 'textacular', '~> 3.0'
|
34
|
+
```
|
35
|
+
|
36
|
+
#### ActiveRecord outside of Rails
|
37
|
+
|
38
|
+
```ruby
|
39
|
+
require 'textacular'
|
40
|
+
|
41
|
+
ActiveRecord::Base.extend(Textacular)
|
42
|
+
```
|
43
|
+
|
44
|
+
|
45
|
+
### Usage
|
46
|
+
|
47
|
+
Your models now have access to search methods:
|
48
|
+
|
49
|
+
The `#basic_search` method is what you might expect: it looks literally for what
|
50
|
+
you send to it, doing nothing fancy with the input:
|
51
|
+
|
52
|
+
```ruby
|
53
|
+
Game.basic_search('Sonic') # will search through the model's :string columns
|
54
|
+
Game.basic_search(title: 'Mario', system: 'Nintendo')
|
55
|
+
```
|
56
|
+
|
57
|
+
The `#advanced_search` method lets you use Postgres's search syntax like '|',
|
58
|
+
'&' and '!' ('or', 'and', and 'not') as well as some other craziness. The ideal
|
59
|
+
use for advanced_search is to take a search DSL you make up for your users and
|
60
|
+
translate it to PG's syntax. If for some reason you want to put user input
|
61
|
+
directly into an advanced search, you should be sure to catch exceptions from
|
62
|
+
syntax errors. Check [the Postgres docs]
|
63
|
+
(http://www.postgresql.org/docs/9.2/static/datatype-textsearch.html) for more:
|
64
|
+
|
65
|
+
```ruby
|
66
|
+
Game.advanced_search(title: 'Street|Fantasy')
|
67
|
+
Game.advanced_search(system: '!PS2')
|
68
|
+
```
|
69
|
+
|
70
|
+
Finally, the `#fuzzy_search` method lets you use Postgres's trigram search
|
71
|
+
functionality.
|
72
|
+
|
73
|
+
In order to use this, you'll need to make sure your database has the `pg_trgm`
|
74
|
+
module installed. Create and run a migration to install the module:
|
75
|
+
|
76
|
+
```
|
77
|
+
rake textacular:create_trigram_migration
|
78
|
+
rake db:migrate
|
79
|
+
```
|
80
|
+
|
81
|
+
Once that's installed, you can use it like this:
|
82
|
+
|
83
|
+
```ruby
|
84
|
+
Comic.fuzzy_search(title: 'Questio') # matches Questionable Content
|
85
|
+
```
|
86
|
+
|
87
|
+
Note that fuzzy searches are subject to a similarity threshold imposed by the `pg_trgm` module. The default is 0.3, meaning that at least 30% of the total string must match your search content. For example:
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
Comic.fuzzy_search(title: 'Pearls') # matches Pearls Before Swine
|
91
|
+
Comic.fuzzy_search(title: 'Pear') # does not match Pearls Before Swine
|
92
|
+
```
|
93
|
+
|
94
|
+
The similarity threshold is hardcoded in PostgreSQL and can be modified on a per-connection basis, for example:
|
95
|
+
|
96
|
+
```ruby
|
97
|
+
ActiveRecord::Base.connection.execute("SELECT set_limit(0.9);")
|
98
|
+
```
|
99
|
+
|
100
|
+
For more info, view the `pg_trgm` documentation, specifically [F.35.2. Functions and Operators](http://www.postgresql.org/docs/9.1/static/pgtrgm.html).
|
101
|
+
|
102
|
+
Searches are also chainable:
|
103
|
+
|
104
|
+
```ruby
|
105
|
+
Game.fuzzy_search(title: 'tree').basic_search(system: 'SNES')
|
106
|
+
```
|
107
|
+
|
108
|
+
If you want to search on two or more fields with the OR operator use a hash for
|
109
|
+
the conditions and pass false as the second parameter:
|
110
|
+
|
111
|
+
```ruby
|
112
|
+
Game.basic_search({name: 'Mario', nickname: 'Mario'}, false)
|
113
|
+
```
|
114
|
+
|
115
|
+
|
116
|
+
### Setting Language
|
117
|
+
|
118
|
+
To set proper searching dictionary just override class method on your model:
|
119
|
+
|
120
|
+
```ruby
|
121
|
+
def self.searchable_language
|
122
|
+
'russian'
|
123
|
+
end
|
124
|
+
```
|
125
|
+
|
126
|
+
And all your queries would go right! And don`t forget to change the migration for indexes, like shown below.
|
127
|
+
|
128
|
+
|
129
|
+
### Creating Indexes for Super Speed
|
130
|
+
You can have Postgresql use an index for the full-text search. To declare a full-text index, in a
|
131
|
+
migration add code like the following:
|
132
|
+
|
133
|
+
```ruby
|
134
|
+
execute "
|
135
|
+
create index on email_logs using gin(to_tsvector('english', subject));
|
136
|
+
create index on email_logs using gin(to_tsvector('english', email_address));"
|
137
|
+
```
|
138
|
+
|
139
|
+
In the above example, the table email_logs has two text columns that we search against, subject and email_address.
|
140
|
+
You will need to add an index for every text/string column you query against, or else Postgresql will revert to a
|
141
|
+
full table scan instead of using the indexes.
|
142
|
+
|
143
|
+
If you create these indexes, you should also switch to sql for your schema_format in `config/application.rb`:
|
144
|
+
|
145
|
+
```ruby
|
146
|
+
config.active_record.schema_format = :sql
|
147
|
+
```
|
148
|
+
|
149
|
+
|
150
|
+
## REQUIREMENTS:
|
151
|
+
|
152
|
+
* ActiveRecord
|
153
|
+
* Ruby 1.9.2
|
154
|
+
|
155
|
+
|
156
|
+
## INSTALL:
|
157
|
+
|
158
|
+
```
|
159
|
+
$ gem install textacular
|
160
|
+
```
|
161
|
+
|
162
|
+
## Contributing
|
163
|
+
|
164
|
+
If you'd like to contribute, please see the [contribution guidelines](CONTRIBUTING.md).
|
165
|
+
|
166
|
+
|
167
|
+
## Releasing
|
168
|
+
|
169
|
+
Maintainers: Please make sure to follow the [release steps](RELEASING.md) when
|
170
|
+
it's time to cut a new release.
|
171
|
+
|
172
|
+
|
173
|
+
## LICENSE:
|
174
|
+
|
175
|
+
(The MIT License)
|
176
|
+
|
177
|
+
Copyright (c) 2011 Aaron Patterson
|
178
|
+
|
179
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
180
|
+
a copy of this software and associated documentation files (the
|
181
|
+
'Software'), to deal in the Software without restriction, including
|
182
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
183
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
184
|
+
permit persons to whom the Software is furnished to do so, subject to
|
185
|
+
the following conditions:
|
186
|
+
|
187
|
+
The above copyright notice and this permission notice shall be
|
188
|
+
included in all copies or substantial portions of the Software.
|
189
|
+
|
190
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
191
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
192
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
193
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
194
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
195
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
196
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/Rakefile
ADDED
@@ -0,0 +1,61 @@
|
|
1
|
+
require "bundler/gem_tasks"
|
2
|
+
|
3
|
+
require 'active_record'
|
4
|
+
require 'pry'
|
5
|
+
|
6
|
+
require 'rspec/core/rake_task'
|
7
|
+
|
8
|
+
RSpec::Core::RakeTask.new(:spec)
|
9
|
+
|
10
|
+
task :default => :spec
|
11
|
+
|
12
|
+
file 'spec/config.yml' do |t|
|
13
|
+
sh 'erb spec/config.yml.example > spec/config.yml'
|
14
|
+
end
|
15
|
+
|
16
|
+
desc 'Fire up an interactive terminal to play with'
|
17
|
+
task :console => :'db:connect' do
|
18
|
+
Pry.start
|
19
|
+
end
|
20
|
+
|
21
|
+
namespace :db do
|
22
|
+
|
23
|
+
task :connect => 'spec/config.yml' do |t|
|
24
|
+
ActiveRecord::Base.establish_connection \
|
25
|
+
YAML.load_file 'spec/config.yml'
|
26
|
+
end
|
27
|
+
|
28
|
+
task :disconnect do
|
29
|
+
ActiveRecord::Base.clear_all_connections!
|
30
|
+
end
|
31
|
+
|
32
|
+
desc 'Create the test database'
|
33
|
+
task :create do
|
34
|
+
sh 'createdb textacular'
|
35
|
+
end
|
36
|
+
|
37
|
+
desc 'Drop the test database'
|
38
|
+
task :drop => :disconnect do
|
39
|
+
sh 'dropdb textacular'
|
40
|
+
end
|
41
|
+
|
42
|
+
namespace :migrate do
|
43
|
+
|
44
|
+
desc 'Run the test database migrations'
|
45
|
+
task :up => :'db:connect' do
|
46
|
+
ActiveRecord::Migrator.up 'db/migrate'
|
47
|
+
end
|
48
|
+
|
49
|
+
desc 'Reverse the test database migrations'
|
50
|
+
task :down => :'db:connect' do
|
51
|
+
ActiveRecord::Migrator.down 'db/migrate'
|
52
|
+
end
|
53
|
+
end
|
54
|
+
task :migrate => :'migrate:up'
|
55
|
+
|
56
|
+
desc 'Create and configure the test database'
|
57
|
+
task :setup => [ :create, :migrate ]
|
58
|
+
|
59
|
+
desc 'Drop the test tables and database'
|
60
|
+
task :teardown => [ :'migrate:down', :drop ]
|
61
|
+
end
|