db_text_search 0.1.2 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGES.md +9 -0
- data/README.md +42 -11
- data/db_text_search.gemspec +15 -14
- data/lib/db_text_search.rb +32 -4
- data/lib/db_text_search/{case_insensitive_eq.rb → case_insensitive.rb} +45 -43
- data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/abstract_adapter.rb +11 -2
- data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/collate_nocase_adapter.rb +16 -4
- data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/insensitive_column_adapter.rb +11 -4
- data/lib/db_text_search/case_insensitive/lower_adapter.rb +38 -0
- data/lib/db_text_search/{full_text_search.rb → full_text.rb} +14 -17
- data/lib/db_text_search/{full_text_search → full_text}/abstract_adapter.rb +5 -3
- data/lib/db_text_search/{full_text_search → full_text}/mysql_adapter.rb +6 -4
- data/lib/db_text_search/{full_text_search → full_text}/postgres_adapter.rb +7 -5
- data/lib/db_text_search/{full_text_search → full_text}/sqlite_adapter.rb +8 -7
- data/lib/db_text_search/query_building.rb +8 -6
- data/lib/db_text_search/version.rb +3 -1
- metadata +18 -19
- data/lib/db_text_search/case_insensitive_eq/lower_adapter.rb +0 -35
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0b93fa5e6feaef447c7bd1e8580b42aafa2cca3d
|
4
|
+
data.tar.gz: d1ddab5bfe8ef8cb544c711ab1b4792882578eab
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ffe48a13416b0dd7d820584d411c5a4e22806738e1bc3c36fd2be0121f890b621269f62a058b3791e0322e5f51edc05c255ab0ee87958c8b4e0e1846dba41ca9
|
7
|
+
data.tar.gz: b07cce34cda731a695d59da849cb0e62a0f2c2247584c32a21cf23c75afeba4382cad4d3fe2cff945e4ccd05bedcd114af44ed5248c1775c2fe5e4c3e148a016
|
data/CHANGES.md
CHANGED
@@ -1,3 +1,12 @@
|
|
1
|
+
## v0.2.0
|
2
|
+
|
3
|
+
* **Feature** Prefix matching via the new `CaseInsensitive#prefix` method.
|
4
|
+
* PostgreSQL CI index now uses the `text_pattern_ops` [opclass] by default (for prefix matching).
|
5
|
+
* Renamed `CaseInsensitiveEq` to `CaseInsensitive`, and `#find` to `#in`.
|
6
|
+
* Renamed `FullTextSearch` to `FullText`, and `#find` to `#search`.
|
7
|
+
|
8
|
+
[opclass]: http://www.postgresql.org/docs/9.5/static/indexes-opclass.html
|
9
|
+
|
1
10
|
## v0.1.2
|
2
11
|
|
3
12
|
Tightened the API. Improved documentation.
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
Different relational databases treat text search very differently.
|
4
4
|
DbTextSearch provides a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL to do:
|
5
5
|
|
6
|
-
* Case-insensitive string-in-set querying, and
|
6
|
+
* Case-insensitive string-in-set querying, prefix querying, and case-insensitive index creation.
|
7
7
|
* Basic full-text search for a list of terms, and FTS index creation.
|
8
8
|
|
9
9
|
## Installation
|
@@ -11,33 +11,40 @@ DbTextSearch provides a unified interface on top of ActiveRecord for SQLite, MyS
|
|
11
11
|
Add this line to your application's Gemfile:
|
12
12
|
|
13
13
|
```ruby
|
14
|
-
gem 'db_text_search', '~> 0.
|
14
|
+
gem 'db_text_search', '~> 0.2.0'
|
15
15
|
```
|
16
16
|
|
17
17
|
## Usage
|
18
18
|
|
19
19
|
### Case-insensitive string matching
|
20
20
|
|
21
|
-
Add an index in a migration to an existing CI or CS column:
|
21
|
+
Add an index in a migration to an existing CI (case-insensitive) or CS (case-sensitive) column:
|
22
22
|
|
23
23
|
```ruby
|
24
|
-
DbTextSearch::
|
24
|
+
DbTextSearch::CaseInsensitive.add_index connection, :users, :username
|
25
25
|
# Options: name, unique
|
26
26
|
```
|
27
27
|
|
28
28
|
Or, create a new CI column:
|
29
29
|
|
30
30
|
```ruby
|
31
|
-
DbTextSearch::
|
31
|
+
DbTextSearch::CaseInsensitive.add_ci_text_column connection, :users, :username
|
32
32
|
```
|
33
33
|
|
34
34
|
Perform a search for records with column that case-insensitively equals to one of the strings in a given set:
|
35
35
|
|
36
36
|
```ruby
|
37
|
-
|
37
|
+
# Find all confirmed users that have either the username Alice or Bob (case-insensitively):
|
38
|
+
DbTextSearch::CaseInsensitive.new(User.confirmed, :username).in(%w(Alice Bob))
|
38
39
|
#=> ActiveRecord::Relation
|
39
40
|
```
|
40
41
|
|
42
|
+
Perform a case-insensitive prefix search:
|
43
|
+
|
44
|
+
```ruby
|
45
|
+
DbTextSearch::CaseInsensitive.new(User.confirmed, :username).prefix('Jo')
|
46
|
+
```
|
47
|
+
|
41
48
|
See also: [API documentation][api-docs].
|
42
49
|
|
43
50
|
### Full text search
|
@@ -45,23 +52,25 @@ See also: [API documentation][api-docs].
|
|
45
52
|
Add an index:
|
46
53
|
|
47
54
|
```ruby
|
48
|
-
DbTextSearch::
|
55
|
+
DbTextSearch::FullText.add_index connection, :posts, :content
|
49
56
|
# Options: name
|
50
57
|
```
|
51
58
|
|
52
59
|
Perform a full-text search:
|
53
60
|
|
54
61
|
```ruby
|
55
|
-
DbTextSearch::
|
56
|
-
DbTextSearch::
|
62
|
+
DbTextSearch::FullText.new(Post.published, :content).search('peace')
|
63
|
+
DbTextSearch::FullText.new(Post.published, :content).search(%w(love kaori))
|
57
64
|
```
|
58
65
|
|
59
66
|
## Under the hood
|
60
67
|
|
68
|
+
### Case-insensitive string matching
|
69
|
+
|
61
70
|
<table>
|
62
|
-
<caption>Case-insensitive
|
71
|
+
<caption>Case-insensitive equality methods</caption>
|
63
72
|
<thead>
|
64
|
-
<tr><th rowspan="2">Column type</th><th colspan="2">SQLite</th><th colspan="2">MySQL</th><th colspan="2">PostgreSQL</th>
|
73
|
+
<tr><th rowspan="2">Column type</th><th colspan="2">SQLite</th><th colspan="2">MySQL</th><th colspan="2">PostgreSQL</th></tr>
|
65
74
|
<tr><th>Detected types</th><th>Search / index</th><th>Detected types</th><th>Search / index</th><th>Detected types</th><th>Search / index</th></tr>
|
66
75
|
</thead>
|
67
76
|
<tbody style="text-align: center">
|
@@ -69,14 +78,36 @@ DbTextSearch::FullTextSearch.new(Post.published, :content).find(%w(love kaori))
|
|
69
78
|
<td rowspan="2">always treated as CS</td> <td rowspan="2"><code>COLLATE NOCASE</code></td>
|
70
79
|
<td><i>default</i></td> <td><i>default</i></td>
|
71
80
|
<td><code>CITEXT</code></td> <td><i>default</i></td>
|
81
|
+
</tr>
|
72
82
|
<tr><th>CS</th>
|
73
83
|
<td>non-<code>ci</code> collations</td> <td><code>LOWER</code><br><b>no index</b></td>
|
74
84
|
<td><i>default</i></td> <td><code>LOWER</code></td>
|
75
85
|
</tr>
|
86
|
+
</tbody>
|
87
|
+
</table>
|
88
|
+
|
89
|
+
<table>
|
90
|
+
<caption>Case-insensitive prefix matching (using <code>LIKE</code>)</caption>
|
91
|
+
<thead>
|
92
|
+
<tr><th>Column type</th><th>SQLite</th><th>MySQL</th><th>PostgreSQL</th></tr>
|
93
|
+
</thead>
|
94
|
+
<tbody style="text-align: center">
|
95
|
+
<tr><th>CI</th>
|
96
|
+
<td rowspan="2">
|
97
|
+
<i>default</i>, <a href="https://www.sqlite.org/optoverview.html#prefix_opt"><b>cannot always use an index</b></a>,<br>
|
98
|
+
even for prefix queries
|
99
|
+
</td>
|
100
|
+
<td><i>default</i></td>
|
101
|
+
<td><b>cannot use an index</b></td>
|
102
|
+
</tr>
|
103
|
+
<tr><th>CS</th>
|
104
|
+
<td><b>cannot use an index</b></td>
|
105
|
+
<td><code>LOWER(column text_pattern_ops)</code></td>
|
76
106
|
</tr>
|
77
107
|
</tbody>
|
78
108
|
</table>
|
79
109
|
|
110
|
+
|
80
111
|
### Full-text search
|
81
112
|
|
82
113
|
#### MySQL
|
data/db_text_search.gemspec
CHANGED
@@ -3,22 +3,23 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
3
3
|
require 'db_text_search/version'
|
4
4
|
|
5
5
|
Gem::Specification.new do |s|
|
6
|
-
s.name
|
7
|
-
s.version
|
8
|
-
s.authors
|
9
|
-
s.email
|
6
|
+
s.name = 'db_text_search'
|
7
|
+
s.version = DbTextSearch::VERSION
|
8
|
+
s.authors = ['Gleb Mazovetskiy']
|
9
|
+
s.email = ['glex.spb@gmail.com']
|
10
10
|
|
11
|
-
s.summary
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
s.homepage
|
18
|
-
s.license
|
11
|
+
s.summary = 'A unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL'\
|
12
|
+
'for case-insensitive string search and basic full-text search.'
|
13
|
+
s.description = 'Different relational databases treat text search very differently. DbTextSearch provides '\
|
14
|
+
'a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL to do '\
|
15
|
+
'case-insensitive string-in-set querying and CI index creation, and '\
|
16
|
+
'basic full-text search for a list of terms, and FTS index creation.'
|
17
|
+
s.homepage = 'https://github.com/thredded/db_text_search'
|
18
|
+
s.license = 'MIT'
|
19
19
|
|
20
|
-
s.files
|
21
|
-
|
20
|
+
s.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec|bin|script)/|^\.|Rakefile|Gemfile}) }
|
21
|
+
|
22
|
+
s.require_paths = ['lib']
|
22
23
|
s.required_ruby_version = '~> 2.1'
|
23
24
|
|
24
25
|
s.add_dependency 'activerecord', '>= 4.1.15', '< 6.0'
|
data/lib/db_text_search.rb
CHANGED
@@ -1,14 +1,42 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'active_record'
|
2
4
|
require 'active_support/core_ext/hash/keys'
|
3
5
|
|
4
6
|
require 'db_text_search/version'
|
5
|
-
require 'db_text_search/
|
6
|
-
require 'db_text_search/
|
7
|
+
require 'db_text_search/case_insensitive'
|
8
|
+
require 'db_text_search/full_text'
|
7
9
|
|
8
10
|
# DbTextSearch provides a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL to do:
|
9
11
|
# * Case-insensitive string-in-set querying, and CI index creation.
|
10
12
|
# * Basic full-text search for a list of terms, and FTS index creation.
|
11
|
-
# @see DbTextSearch::
|
12
|
-
# @see DbTextSearch::
|
13
|
+
# @see DbTextSearch::CaseInsensitive
|
14
|
+
# @see DbTextSearch::FullText
|
13
15
|
module DbTextSearch
|
16
|
+
# Call the appropriate proc based on the adapter name.
|
17
|
+
# @param connection [ActiveRecord::ConnectionAdapters::AbstractAdapter]
|
18
|
+
# @param mysql [Proc]
|
19
|
+
# @param postgres [Proc]
|
20
|
+
# @param sqlite [Proc]
|
21
|
+
# @return the called proc return value.
|
22
|
+
# @api private
|
23
|
+
def self.match_adapter(connection, mysql:, postgres:, sqlite:)
|
24
|
+
case connection.adapter_name
|
25
|
+
when /mysql/i
|
26
|
+
mysql.call
|
27
|
+
when /postgres/i
|
28
|
+
postgres.call
|
29
|
+
when /sqlite/i
|
30
|
+
sqlite.call
|
31
|
+
else
|
32
|
+
unsupported_adapter! connection
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
# Raises an ArgumentError with "Unsupported adapter #{connection.adapter_name}"
|
37
|
+
# @param connection [ActiveRecord::ConnectionAdapters::AbstractAdapter]
|
38
|
+
# @api private
|
39
|
+
def self.unsupported_adapter!(connection)
|
40
|
+
fail ArgumentError, "Unsupported adapter #{connection.adapter_name}"
|
41
|
+
end
|
14
42
|
end
|
@@ -1,10 +1,12 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require 'db_text_search/
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/insensitive_column_adapter'
|
4
|
+
require 'db_text_search/case_insensitive/lower_adapter'
|
5
|
+
require 'db_text_search/case_insensitive/collate_nocase_adapter'
|
4
6
|
|
5
7
|
module DbTextSearch
|
6
|
-
# Provides case-insensitive string-in-set querying, and CI index creation.
|
7
|
-
class
|
8
|
+
# Provides case-insensitive string-in-set querying, LIKE querying, and CI index creation.
|
9
|
+
class CaseInsensitive
|
8
10
|
# @param scope [ActiveRecord::Relation, Class<ActiveRecord::Base>]
|
9
11
|
# @param column [Symbol] name
|
10
12
|
def initialize(scope, column)
|
@@ -14,10 +16,17 @@ module DbTextSearch
|
|
14
16
|
|
15
17
|
# @param value_or_values [String, Array<String>]
|
16
18
|
# @return [ActiveRecord::Relation]
|
17
|
-
def
|
19
|
+
def in(value_or_values)
|
18
20
|
values = Array(value_or_values)
|
19
21
|
return @scope.none if values.empty?
|
20
|
-
@adapter.
|
22
|
+
@adapter.in(values)
|
23
|
+
end
|
24
|
+
|
25
|
+
# @param query [String]
|
26
|
+
# @return [ActiveRecord::Relation] the scope of records with matching prefix.
|
27
|
+
def prefix(query)
|
28
|
+
return @scope.none if query.empty?
|
29
|
+
@adapter.prefix(query)
|
21
30
|
end
|
22
31
|
|
23
32
|
# Adds a case-insensitive column to the given table.
|
@@ -26,25 +35,20 @@ module DbTextSearch
|
|
26
35
|
# @param column_name [String, Symbol]
|
27
36
|
# @param options [Hash] passed to ActiveRecord::ConnectionAdapters::SchemaStatements#add_index
|
28
37
|
def self.add_ci_text_column(connection, table_name, column_name, options = {})
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
connection.add_column(table_name, column_name, 'TEXT COLLATE NOCASE', options)
|
44
|
-
end
|
45
|
-
else
|
46
|
-
fail ArgumentError.new("Unsupported adapter #{connection.adapter_name}")
|
47
|
-
end
|
38
|
+
connection.add_column table_name, column_name, *DbTextSearch.match_adapter(
|
39
|
+
connection,
|
40
|
+
mysql: -> { [:text, options] },
|
41
|
+
postgres: -> {
|
42
|
+
connection.enable_extension 'citext'
|
43
|
+
[(ActiveRecord::VERSION::STRING >= '4.2.0' ? :citext : 'CITEXT'), options]
|
44
|
+
},
|
45
|
+
sqlite: -> {
|
46
|
+
if ActiveRecord::VERSION::MAJOR >= 5
|
47
|
+
[:text, options.merge(collation: 'NOCASE')]
|
48
|
+
else
|
49
|
+
['TEXT COLLATE NOCASE', options]
|
50
|
+
end
|
51
|
+
})
|
48
52
|
end
|
49
53
|
|
50
54
|
# Add an index for case-insensitive string search.
|
@@ -65,15 +69,16 @@ module DbTextSearch
|
|
65
69
|
# @return [Class<AbstractAdapter>]
|
66
70
|
# @api private
|
67
71
|
def self.adapter_class(connection, table_name, column_name)
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
72
|
+
lower_or_insensitive = -> {
|
73
|
+
column_case_sensitive?(connection, table_name, column_name) ? LowerAdapter : InsensitiveColumnAdapter
|
74
|
+
}
|
75
|
+
DbTextSearch.match_adapter(
|
76
|
+
connection,
|
77
|
+
mysql: lower_or_insensitive,
|
78
|
+
postgres: lower_or_insensitive,
|
79
|
+
# Always use COLLATE NOCASE for SQLite, as we can't check if the column is case-sensitive.
|
80
|
+
# It has no performance impact apart from slightly longer query strings for case-insensitive columns.
|
81
|
+
sqlite: -> { CollateNocaseAdapter })
|
77
82
|
end
|
78
83
|
|
79
84
|
# @param connection [ActiveRecord::ConnectionAdapters::AbstractAdapter]
|
@@ -84,14 +89,11 @@ module DbTextSearch
|
|
84
89
|
# @api private
|
85
90
|
def self.column_case_sensitive?(connection, table_name, column_name)
|
86
91
|
column = connection.schema_cache.columns(table_name).detect { |c| c.name == column_name.to_s }
|
87
|
-
|
88
|
-
|
89
|
-
column.case_sensitive?
|
90
|
-
|
91
|
-
|
92
|
-
else
|
93
|
-
fail ArgumentError.new("Unsupported adapter #{connection.adapter_name}")
|
94
|
-
end
|
92
|
+
DbTextSearch.match_adapter(
|
93
|
+
connection,
|
94
|
+
mysql: -> { column.case_sensitive? },
|
95
|
+
postgres: -> { column.sql_type !~ /citext/i },
|
96
|
+
sqlite: -> { DbTextSearch.unsupported_adapter! connection })
|
95
97
|
end
|
96
98
|
end
|
97
99
|
end
|
@@ -1,6 +1,8 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'db_text_search/query_building'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class CaseInsensitive
|
4
6
|
# A base class for CaseInsensitiveStringFinder adapters.
|
5
7
|
# @abstract
|
6
8
|
# @api private
|
@@ -17,7 +19,14 @@ module DbTextSearch
|
|
17
19
|
# @param values [Array<String>]
|
18
20
|
# @return [ActiveRecord::Relation]
|
19
21
|
# @abstract
|
20
|
-
def
|
22
|
+
def in(values)
|
23
|
+
fail 'abstract'
|
24
|
+
end
|
25
|
+
|
26
|
+
# @param query [String]
|
27
|
+
# @return [ActiveRecord::Relation]
|
28
|
+
# @abstract
|
29
|
+
def prefix(query)
|
21
30
|
fail 'abstract'
|
22
31
|
end
|
23
32
|
|
@@ -1,15 +1,27 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class CaseInsensitive
|
4
6
|
# Provides case-insensitive string-in-set querying via COLLATE NOCASE.
|
5
7
|
# @api private
|
6
8
|
class CollateNocaseAdapter < AbstractAdapter
|
7
|
-
# (see AbstractAdapter#
|
8
|
-
def
|
9
|
+
# (see AbstractAdapter#in)
|
10
|
+
def in(values)
|
9
11
|
conn = @scope.connection
|
10
12
|
@scope.where "#{quoted_scope_column} COLLATE NOCASE IN (#{values.map { |v| conn.quote(v.to_s) }.join(', ')})"
|
11
13
|
end
|
12
14
|
|
15
|
+
# (see AbstractAdapter#prefix)
|
16
|
+
def prefix(query)
|
17
|
+
escape = '\\'
|
18
|
+
escaped_query = "#{sanitize_sql_like(query, escape)}%"
|
19
|
+
# assuming case_sensitive_prefix mode to be disabled, prefix it is by default.
|
20
|
+
# this is to avoid adding COLLATE NOCASE here, which prevents index use in SQLite LIKE.
|
21
|
+
@scope.where "#{quoted_scope_column} LIKE ?#{" ESCAPE '#{escape}'" if escaped_query.include?(escape)}",
|
22
|
+
escaped_query
|
23
|
+
end
|
24
|
+
|
13
25
|
# (see AbstractAdapter.add_index)
|
14
26
|
def self.add_index(connection, table_name, column_name, options = {})
|
15
27
|
# TODO: Switch to the native Rails solution once it's landed, as the current one requires SQL dump format.
|
data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/insensitive_column_adapter.rb
RENAMED
@@ -1,14 +1,21 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class CaseInsensitive
|
4
6
|
# Provides case-insensitive string-in-set querying for case-insensitive columns.
|
5
7
|
# @api private
|
6
8
|
class InsensitiveColumnAdapter < AbstractAdapter
|
7
|
-
# (see AbstractAdapter#
|
8
|
-
def
|
9
|
+
# (see AbstractAdapter#in)
|
10
|
+
def in(values)
|
9
11
|
@scope.where(@column => values)
|
10
12
|
end
|
11
13
|
|
14
|
+
# (see AbstractAdapter#prefix)
|
15
|
+
def prefix(query)
|
16
|
+
@scope.where "#{quoted_scope_column} LIKE ?", "#{sanitize_sql_like(query)}%"
|
17
|
+
end
|
18
|
+
|
12
19
|
# (see AbstractAdapter.add_index)
|
13
20
|
def self.add_index(connection, table_name, column_name, options = {})
|
14
21
|
connection.add_index table_name, column_name, options
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/abstract_adapter'
|
4
|
+
module DbTextSearch
|
5
|
+
class CaseInsensitive
|
6
|
+
# Provides case-insensitive string-in-set querying by applying the database LOWER function.
|
7
|
+
# @api private
|
8
|
+
class LowerAdapter < AbstractAdapter
|
9
|
+
# (see AbstractAdapter#in)
|
10
|
+
def in(values)
|
11
|
+
conn = @scope.connection
|
12
|
+
@scope.where "LOWER(#{quoted_scope_column}) IN (#{values.map { |v| "LOWER(#{conn.quote(v)})" }.join(', ')})"
|
13
|
+
end
|
14
|
+
|
15
|
+
# (see AbstractAdapter#prefix)
|
16
|
+
def prefix(query)
|
17
|
+
@scope.where "LOWER(#{quoted_scope_column}) LIKE LOWER(?)", "#{sanitize_sql_like(query)}%"
|
18
|
+
end
|
19
|
+
|
20
|
+
# (see AbstractAdapter.add_index)
|
21
|
+
def self.add_index(connection, table_name, column_name, options = {})
|
22
|
+
unsupported = -> { DbTextSearch.unsupported_adapter! connection }
|
23
|
+
DbTextSearch.match_adapter(
|
24
|
+
connection,
|
25
|
+
# TODO: Switch to native Rails support once it lands.
|
26
|
+
# https://github.com/rails/rails/pull/18499
|
27
|
+
postgres: -> {
|
28
|
+
options = options.dup
|
29
|
+
options[:name] ||= "#{table_name}_#{column_name}_lower"
|
30
|
+
options[:expression] = "(LOWER(#{connection.quote_column_name(column_name)}) text_pattern_ops)"
|
31
|
+
connection.exec_query(quoted_create_index(connection, table_name, **options))
|
32
|
+
},
|
33
|
+
mysql: unsupported,
|
34
|
+
sqlite: unsupported)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -1,10 +1,12 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require 'db_text_search/
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/postgres_adapter'
|
4
|
+
require 'db_text_search/full_text/mysql_adapter'
|
5
|
+
require 'db_text_search/full_text/sqlite_adapter'
|
4
6
|
|
5
7
|
module DbTextSearch
|
6
8
|
# Provides basic full-text search for a list of terms, and FTS index creation.
|
7
|
-
class
|
9
|
+
class FullText
|
8
10
|
# The default Postgres text search config.
|
9
11
|
DEFAULT_PG_TS_CONFIG = %q('english')
|
10
12
|
|
@@ -18,10 +20,10 @@ module DbTextSearch
|
|
18
20
|
# @param term_or_terms [String, Array<String>]
|
19
21
|
# @param pg_ts_config [String] for Postgres, the TS config to use; ignored for non-postgres.
|
20
22
|
# @return [ActiveRecord::Relation]
|
21
|
-
def
|
22
|
-
values
|
23
|
+
def search(term_or_terms, pg_ts_config: DEFAULT_PG_TS_CONFIG)
|
24
|
+
values = Array(term_or_terms)
|
23
25
|
return @scope.none if values.empty?
|
24
|
-
@adapter.
|
26
|
+
@adapter.search(values, pg_ts_config: pg_ts_config)
|
25
27
|
end
|
26
28
|
|
27
29
|
# Add an index for full text search.
|
@@ -43,16 +45,11 @@ module DbTextSearch
|
|
43
45
|
# @return [Class<AbstractAdapter>]
|
44
46
|
# @api private
|
45
47
|
def self.adapter_class(connection, _table_name, _column_name)
|
46
|
-
|
47
|
-
|
48
|
-
MysqlAdapter
|
49
|
-
|
50
|
-
|
51
|
-
when /sqlite/i
|
52
|
-
SqliteAdapter
|
53
|
-
else
|
54
|
-
fail ArgumentError.new("Unsupported adapter #{connection.adapter_name}")
|
55
|
-
end
|
48
|
+
DbTextSearch.match_adapter(
|
49
|
+
connection,
|
50
|
+
mysql: -> { MysqlAdapter },
|
51
|
+
postgres: -> { PostgresAdapter },
|
52
|
+
sqlite: -> { SqliteAdapter })
|
56
53
|
end
|
57
54
|
end
|
58
55
|
end
|
@@ -1,6 +1,8 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module DbTextSearch
|
2
|
-
class
|
3
|
-
# A base class for
|
4
|
+
class FullText
|
5
|
+
# A base class for FullText adapters.
|
4
6
|
# @abstract
|
5
7
|
# @api private
|
6
8
|
class AbstractAdapter
|
@@ -17,7 +19,7 @@ module DbTextSearch
|
|
17
19
|
# @param pg_ts_config [String] a pg text search config
|
18
20
|
# @return [ActiveRecord::Relation]
|
19
21
|
# @abstract
|
20
|
-
def
|
22
|
+
def search(terms, pg_ts_config:)
|
21
23
|
fail 'abstract'
|
22
24
|
end
|
23
25
|
|
@@ -1,6 +1,8 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class FullText
|
4
6
|
# Provides basic FTS support for MySQL.
|
5
7
|
#
|
6
8
|
# Runs a `MATCH AGAINST` query against a `FULLTEXT` index.
|
@@ -8,8 +10,8 @@ module DbTextSearch
|
|
8
10
|
# @note MySQL v5.6.4+ is required.
|
9
11
|
# @api private
|
10
12
|
class MysqlAdapter < AbstractAdapter
|
11
|
-
# (see AbstractAdapter#
|
12
|
-
def
|
13
|
+
# (see AbstractAdapter#search)
|
14
|
+
def search(terms, pg_ts_config:)
|
13
15
|
@scope.where("MATCH (#{quoted_scope_column}) AGAINST (?)", terms.uniq.join(' '))
|
14
16
|
end
|
15
17
|
|
@@ -1,15 +1,17 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class FullText
|
4
6
|
# Provides basic FTS support for PostgreSQL.
|
5
7
|
#
|
6
8
|
# Runs a `@@ plainto_tsquery` query against a `gist(to_tsvector(...))` index.
|
7
9
|
#
|
8
|
-
# @see DbTextSearch::
|
10
|
+
# @see DbTextSearch::FullText::DEFAULT_PG_TS_CONFIG
|
9
11
|
# @api private
|
10
12
|
class PostgresAdapter < AbstractAdapter
|
11
|
-
# (see AbstractAdapter#
|
12
|
-
def
|
13
|
+
# (see AbstractAdapter#search)
|
14
|
+
def search(terms, pg_ts_config:)
|
13
15
|
@scope.where("to_tsvector(#{pg_ts_config}, #{quoted_scope_column}) @@ plainto_tsquery(#{pg_ts_config}, ?)",
|
14
16
|
terms.uniq.join(' '))
|
15
17
|
end
|
@@ -1,6 +1,8 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class FullText
|
4
6
|
# Provides very basic FTS support for SQLite.
|
5
7
|
#
|
6
8
|
# Runs a `LIKE %term%` query for each term, joined with `AND`.
|
@@ -9,17 +11,16 @@ module DbTextSearch
|
|
9
11
|
# @note .add_index is a no-op.
|
10
12
|
# @api private
|
11
13
|
class SqliteAdapter < AbstractAdapter
|
12
|
-
# (see AbstractAdapter#
|
13
|
-
def
|
14
|
+
# (see AbstractAdapter#search)
|
15
|
+
def search(terms, pg_ts_config:)
|
14
16
|
quoted_col = quoted_scope_column
|
15
17
|
terms.map(&:downcase).uniq.inject(@scope) do |scope, term|
|
16
18
|
scope.where("#{quoted_col} COLLATE NOCASE LIKE ?", "%#{sanitize_sql_like term}%")
|
17
19
|
end
|
18
20
|
end
|
19
21
|
|
20
|
-
#
|
21
|
-
def self.add_index(
|
22
|
-
# A no-op, as we just use LIKE for sqlite.
|
22
|
+
# A no-op, as we just use LIKE for sqlite.
|
23
|
+
def self.add_index(_connection, _table_name, _column_name, name:, pg_ts_config:)
|
23
24
|
end
|
24
25
|
end
|
25
26
|
end
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module DbTextSearch
|
2
4
|
# Common methods for building SQL that use @scope and @column instance variables.
|
3
5
|
# @api private
|
@@ -6,6 +8,12 @@ module DbTextSearch
|
|
6
8
|
base.extend ClassMethods
|
7
9
|
end
|
8
10
|
|
11
|
+
# @return [String] SQL-quoted string suitable for use in a LIKE statement, with % and _ escaped.
|
12
|
+
def sanitize_sql_like(string, escape_character = '\\')
|
13
|
+
pattern = Regexp.union(escape_character, '%', '_')
|
14
|
+
string.gsub(pattern) { |x| [escape_character, x].join }
|
15
|
+
end
|
16
|
+
|
9
17
|
protected
|
10
18
|
|
11
19
|
# @return [String] SQL-quoted scope table name.
|
@@ -23,12 +31,6 @@ module DbTextSearch
|
|
23
31
|
"#{quoted_scope_table}.#{quoted_column}"
|
24
32
|
end
|
25
33
|
|
26
|
-
# @return [String] SQL-quoted string suitable for use in a LIKE statement, with % and _ escaped.
|
27
|
-
def sanitize_sql_like(string, escape_character = "\\")
|
28
|
-
pattern = Regexp.union(escape_character, '%', '_')
|
29
|
-
string.gsub(pattern) { |x| [escape_character, x].join }
|
30
|
-
end
|
31
|
-
|
32
34
|
# Common methods for building SQL
|
33
35
|
# @api private
|
34
36
|
module ClassMethods
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: db_text_search
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Gleb Mazovetskiy
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-
|
11
|
+
date: 2016-04-06 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -114,11 +114,10 @@ dependencies:
|
|
114
114
|
- - "~>"
|
115
115
|
- !ruby/object:Gem::Version
|
116
116
|
version: '3.4'
|
117
|
-
description:
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
* Basic full-text search for a list of terms, and FTS index creation.
|
117
|
+
description: Different relational databases treat text search very differently. DbTextSearch
|
118
|
+
provides a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL
|
119
|
+
to do case-insensitive string-in-set querying and CI index creation, and basic full-text
|
120
|
+
search for a list of terms, and FTS index creation.
|
122
121
|
email:
|
123
122
|
- glex.spb@gmail.com
|
124
123
|
executables: []
|
@@ -131,16 +130,16 @@ files:
|
|
131
130
|
- README.md
|
132
131
|
- db_text_search.gemspec
|
133
132
|
- lib/db_text_search.rb
|
134
|
-
- lib/db_text_search/
|
135
|
-
- lib/db_text_search/
|
136
|
-
- lib/db_text_search/
|
137
|
-
- lib/db_text_search/
|
138
|
-
- lib/db_text_search/
|
139
|
-
- lib/db_text_search/
|
140
|
-
- lib/db_text_search/
|
141
|
-
- lib/db_text_search/
|
142
|
-
- lib/db_text_search/
|
143
|
-
- lib/db_text_search/
|
133
|
+
- lib/db_text_search/case_insensitive.rb
|
134
|
+
- lib/db_text_search/case_insensitive/abstract_adapter.rb
|
135
|
+
- lib/db_text_search/case_insensitive/collate_nocase_adapter.rb
|
136
|
+
- lib/db_text_search/case_insensitive/insensitive_column_adapter.rb
|
137
|
+
- lib/db_text_search/case_insensitive/lower_adapter.rb
|
138
|
+
- lib/db_text_search/full_text.rb
|
139
|
+
- lib/db_text_search/full_text/abstract_adapter.rb
|
140
|
+
- lib/db_text_search/full_text/mysql_adapter.rb
|
141
|
+
- lib/db_text_search/full_text/postgres_adapter.rb
|
142
|
+
- lib/db_text_search/full_text/sqlite_adapter.rb
|
144
143
|
- lib/db_text_search/query_building.rb
|
145
144
|
- lib/db_text_search/version.rb
|
146
145
|
- shared.gemfile
|
@@ -167,7 +166,7 @@ rubyforge_project:
|
|
167
166
|
rubygems_version: 2.5.1
|
168
167
|
signing_key:
|
169
168
|
specification_version: 4
|
170
|
-
summary: A unified interface on top of ActiveRecord for SQLite, MySQL, and
|
171
|
-
|
169
|
+
summary: A unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQLfor
|
170
|
+
case-insensitive string search and basic full-text search.
|
172
171
|
test_files: []
|
173
172
|
has_rdoc:
|
@@ -1,35 +0,0 @@
|
|
1
|
-
require 'db_text_search/case_insensitive_eq/abstract_adapter'
|
2
|
-
module DbTextSearch
|
3
|
-
class CaseInsensitiveEq
|
4
|
-
# Provides case-insensitive string-in-set querying by applying the database LOWER function.
|
5
|
-
# @api private
|
6
|
-
class LowerAdapter < AbstractAdapter
|
7
|
-
# (see AbstractAdapter#find)
|
8
|
-
def find(values)
|
9
|
-
conn = @scope.connection
|
10
|
-
@scope.where "LOWER(#{quoted_scope_column}) IN (#{values.map { |v| "LOWER(#{conn.quote(v.to_s)})" }.join(', ')})"
|
11
|
-
end
|
12
|
-
|
13
|
-
# (see AbstractAdapter.add_index)
|
14
|
-
def self.add_index(connection, table_name, column_name, options = {})
|
15
|
-
if connection.adapter_name =~ /postgres/i
|
16
|
-
# TODO: Switch to native Rails support once it lands.
|
17
|
-
# https://github.com/rails/rails/pull/18499
|
18
|
-
options = options.dup
|
19
|
-
options[:name] ||= "#{table_name}_#{column_name}_lower"
|
20
|
-
options[:expression] = "(LOWER(#{connection.quote_column_name(column_name)}))"
|
21
|
-
if defined?(::SchemaPlus)
|
22
|
-
connection.add_index(table_name, column_name, options)
|
23
|
-
else
|
24
|
-
connection.exec_query quoted_create_index(connection, table_name, **options)
|
25
|
-
end
|
26
|
-
elsif connection.adapter_name =~ /mysql/i
|
27
|
-
fail ArgumentError.new('MySQL case-insensitive index creation for case-sensitive columns is not supported.')
|
28
|
-
else
|
29
|
-
fail ArgumentError.new(
|
30
|
-
"Cannot create a case-insensitive index for case-sensitive column on #{connection.adapter_name}.")
|
31
|
-
end
|
32
|
-
end
|
33
|
-
end
|
34
|
-
end
|
35
|
-
end
|