db_text_search 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGES.md +9 -0
- data/README.md +42 -11
- data/db_text_search.gemspec +15 -14
- data/lib/db_text_search.rb +32 -4
- data/lib/db_text_search/{case_insensitive_eq.rb → case_insensitive.rb} +45 -43
- data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/abstract_adapter.rb +11 -2
- data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/collate_nocase_adapter.rb +16 -4
- data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/insensitive_column_adapter.rb +11 -4
- data/lib/db_text_search/case_insensitive/lower_adapter.rb +38 -0
- data/lib/db_text_search/{full_text_search.rb → full_text.rb} +14 -17
- data/lib/db_text_search/{full_text_search → full_text}/abstract_adapter.rb +5 -3
- data/lib/db_text_search/{full_text_search → full_text}/mysql_adapter.rb +6 -4
- data/lib/db_text_search/{full_text_search → full_text}/postgres_adapter.rb +7 -5
- data/lib/db_text_search/{full_text_search → full_text}/sqlite_adapter.rb +8 -7
- data/lib/db_text_search/query_building.rb +8 -6
- data/lib/db_text_search/version.rb +3 -1
- metadata +18 -19
- data/lib/db_text_search/case_insensitive_eq/lower_adapter.rb +0 -35
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0b93fa5e6feaef447c7bd1e8580b42aafa2cca3d
|
4
|
+
data.tar.gz: d1ddab5bfe8ef8cb544c711ab1b4792882578eab
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ffe48a13416b0dd7d820584d411c5a4e22806738e1bc3c36fd2be0121f890b621269f62a058b3791e0322e5f51edc05c255ab0ee87958c8b4e0e1846dba41ca9
|
7
|
+
data.tar.gz: b07cce34cda731a695d59da849cb0e62a0f2c2247584c32a21cf23c75afeba4382cad4d3fe2cff945e4ccd05bedcd114af44ed5248c1775c2fe5e4c3e148a016
|
data/CHANGES.md
CHANGED
@@ -1,3 +1,12 @@
|
|
1
|
+
## v0.2.0
|
2
|
+
|
3
|
+
* **Feature** Prefix matching via the new `CaseInsensitive#prefix` method.
|
4
|
+
* PostgreSQL CI index now uses the `text_pattern_ops` [opclass] by default (for prefix matching).
|
5
|
+
* Renamed `CaseInsensitiveEq` to `CaseInsensitive`, and `#find` to `#in`.
|
6
|
+
* Renamed `FullTextSearch` to `FullText`, and `#find` to `#search`.
|
7
|
+
|
8
|
+
[opclass]: http://www.postgresql.org/docs/9.5/static/indexes-opclass.html
|
9
|
+
|
1
10
|
## v0.1.2
|
2
11
|
|
3
12
|
Tightened the API. Improved documentation.
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
Different relational databases treat text search very differently.
|
4
4
|
DbTextSearch provides a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL to do:
|
5
5
|
|
6
|
-
* Case-insensitive string-in-set querying, and
|
6
|
+
* Case-insensitive string-in-set querying, prefix querying, and case-insensitive index creation.
|
7
7
|
* Basic full-text search for a list of terms, and FTS index creation.
|
8
8
|
|
9
9
|
## Installation
|
@@ -11,33 +11,40 @@ DbTextSearch provides a unified interface on top of ActiveRecord for SQLite, MyS
|
|
11
11
|
Add this line to your application's Gemfile:
|
12
12
|
|
13
13
|
```ruby
|
14
|
-
gem 'db_text_search', '~> 0.
|
14
|
+
gem 'db_text_search', '~> 0.2.0'
|
15
15
|
```
|
16
16
|
|
17
17
|
## Usage
|
18
18
|
|
19
19
|
### Case-insensitive string matching
|
20
20
|
|
21
|
-
Add an index in a migration to an existing CI or CS column:
|
21
|
+
Add an index in a migration to an existing CI (case-insensitive) or CS (case-sensitive) column:
|
22
22
|
|
23
23
|
```ruby
|
24
|
-
DbTextSearch::
|
24
|
+
DbTextSearch::CaseInsensitive.add_index connection, :users, :username
|
25
25
|
# Options: name, unique
|
26
26
|
```
|
27
27
|
|
28
28
|
Or, create a new CI column:
|
29
29
|
|
30
30
|
```ruby
|
31
|
-
DbTextSearch::
|
31
|
+
DbTextSearch::CaseInsensitive.add_ci_text_column connection, :users, :username
|
32
32
|
```
|
33
33
|
|
34
34
|
Perform a search for records with column that case-insensitively equals to one of the strings in a given set:
|
35
35
|
|
36
36
|
```ruby
|
37
|
-
|
37
|
+
# Find all confirmed users that have either the username Alice or Bob (case-insensitively):
|
38
|
+
DbTextSearch::CaseInsensitive.new(User.confirmed, :username).in(%w(Alice Bob))
|
38
39
|
#=> ActiveRecord::Relation
|
39
40
|
```
|
40
41
|
|
42
|
+
Perform a case-insensitive prefix search:
|
43
|
+
|
44
|
+
```ruby
|
45
|
+
DbTextSearch::CaseInsensitive.new(User.confirmed, :username).prefix('Jo')
|
46
|
+
```
|
47
|
+
|
41
48
|
See also: [API documentation][api-docs].
|
42
49
|
|
43
50
|
### Full text search
|
@@ -45,23 +52,25 @@ See also: [API documentation][api-docs].
|
|
45
52
|
Add an index:
|
46
53
|
|
47
54
|
```ruby
|
48
|
-
DbTextSearch::
|
55
|
+
DbTextSearch::FullText.add_index connection, :posts, :content
|
49
56
|
# Options: name
|
50
57
|
```
|
51
58
|
|
52
59
|
Perform a full-text search:
|
53
60
|
|
54
61
|
```ruby
|
55
|
-
DbTextSearch::
|
56
|
-
DbTextSearch::
|
62
|
+
DbTextSearch::FullText.new(Post.published, :content).search('peace')
|
63
|
+
DbTextSearch::FullText.new(Post.published, :content).search(%w(love kaori))
|
57
64
|
```
|
58
65
|
|
59
66
|
## Under the hood
|
60
67
|
|
68
|
+
### Case-insensitive string matching
|
69
|
+
|
61
70
|
<table>
|
62
|
-
<caption>Case-insensitive
|
71
|
+
<caption>Case-insensitive equality methods</caption>
|
63
72
|
<thead>
|
64
|
-
<tr><th rowspan="2">Column type</th><th colspan="2">SQLite</th><th colspan="2">MySQL</th><th colspan="2">PostgreSQL</th>
|
73
|
+
<tr><th rowspan="2">Column type</th><th colspan="2">SQLite</th><th colspan="2">MySQL</th><th colspan="2">PostgreSQL</th></tr>
|
65
74
|
<tr><th>Detected types</th><th>Search / index</th><th>Detected types</th><th>Search / index</th><th>Detected types</th><th>Search / index</th></tr>
|
66
75
|
</thead>
|
67
76
|
<tbody style="text-align: center">
|
@@ -69,14 +78,36 @@ DbTextSearch::FullTextSearch.new(Post.published, :content).find(%w(love kaori))
|
|
69
78
|
<td rowspan="2">always treated as CS</td> <td rowspan="2"><code>COLLATE NOCASE</code></td>
|
70
79
|
<td><i>default</i></td> <td><i>default</i></td>
|
71
80
|
<td><code>CITEXT</code></td> <td><i>default</i></td>
|
81
|
+
</tr>
|
72
82
|
<tr><th>CS</th>
|
73
83
|
<td>non-<code>ci</code> collations</td> <td><code>LOWER</code><br><b>no index</b></td>
|
74
84
|
<td><i>default</i></td> <td><code>LOWER</code></td>
|
75
85
|
</tr>
|
86
|
+
</tbody>
|
87
|
+
</table>
|
88
|
+
|
89
|
+
<table>
|
90
|
+
<caption>Case-insensitive prefix matching (using <code>LIKE</code>)</caption>
|
91
|
+
<thead>
|
92
|
+
<tr><th>Column type</th><th>SQLite</th><th>MySQL</th><th>PostgreSQL</th></tr>
|
93
|
+
</thead>
|
94
|
+
<tbody style="text-align: center">
|
95
|
+
<tr><th>CI</th>
|
96
|
+
<td rowspan="2">
|
97
|
+
<i>default</i>, <a href="https://www.sqlite.org/optoverview.html#prefix_opt"><b>cannot always use an index</b></a>,<br>
|
98
|
+
even for prefix queries
|
99
|
+
</td>
|
100
|
+
<td><i>default</i></td>
|
101
|
+
<td><b>cannot use an index</b></td>
|
102
|
+
</tr>
|
103
|
+
<tr><th>CS</th>
|
104
|
+
<td><b>cannot use an index</b></td>
|
105
|
+
<td><code>LOWER(column text_pattern_ops)</code></td>
|
76
106
|
</tr>
|
77
107
|
</tbody>
|
78
108
|
</table>
|
79
109
|
|
110
|
+
|
80
111
|
### Full-text search
|
81
112
|
|
82
113
|
#### MySQL
|
data/db_text_search.gemspec
CHANGED
@@ -3,22 +3,23 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
3
3
|
require 'db_text_search/version'
|
4
4
|
|
5
5
|
Gem::Specification.new do |s|
|
6
|
-
s.name
|
7
|
-
s.version
|
8
|
-
s.authors
|
9
|
-
s.email
|
6
|
+
s.name = 'db_text_search'
|
7
|
+
s.version = DbTextSearch::VERSION
|
8
|
+
s.authors = ['Gleb Mazovetskiy']
|
9
|
+
s.email = ['glex.spb@gmail.com']
|
10
10
|
|
11
|
-
s.summary
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
s.homepage
|
18
|
-
s.license
|
11
|
+
s.summary = 'A unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL'\
|
12
|
+
'for case-insensitive string search and basic full-text search.'
|
13
|
+
s.description = 'Different relational databases treat text search very differently. DbTextSearch provides '\
|
14
|
+
'a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL to do '\
|
15
|
+
'case-insensitive string-in-set querying and CI index creation, and '\
|
16
|
+
'basic full-text search for a list of terms, and FTS index creation.'
|
17
|
+
s.homepage = 'https://github.com/thredded/db_text_search'
|
18
|
+
s.license = 'MIT'
|
19
19
|
|
20
|
-
s.files
|
21
|
-
|
20
|
+
s.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec|bin|script)/|^\.|Rakefile|Gemfile}) }
|
21
|
+
|
22
|
+
s.require_paths = ['lib']
|
22
23
|
s.required_ruby_version = '~> 2.1'
|
23
24
|
|
24
25
|
s.add_dependency 'activerecord', '>= 4.1.15', '< 6.0'
|
data/lib/db_text_search.rb
CHANGED
@@ -1,14 +1,42 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'active_record'
|
2
4
|
require 'active_support/core_ext/hash/keys'
|
3
5
|
|
4
6
|
require 'db_text_search/version'
|
5
|
-
require 'db_text_search/
|
6
|
-
require 'db_text_search/
|
7
|
+
require 'db_text_search/case_insensitive'
|
8
|
+
require 'db_text_search/full_text'
|
7
9
|
|
8
10
|
# DbTextSearch provides a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL to do:
|
9
11
|
# * Case-insensitive string-in-set querying, and CI index creation.
|
10
12
|
# * Basic full-text search for a list of terms, and FTS index creation.
|
11
|
-
# @see DbTextSearch::
|
12
|
-
# @see DbTextSearch::
|
13
|
+
# @see DbTextSearch::CaseInsensitive
|
14
|
+
# @see DbTextSearch::FullText
|
13
15
|
module DbTextSearch
|
16
|
+
# Call the appropriate proc based on the adapter name.
|
17
|
+
# @param connection [ActiveRecord::ConnectionAdapters::AbstractAdapter]
|
18
|
+
# @param mysql [Proc]
|
19
|
+
# @param postgres [Proc]
|
20
|
+
# @param sqlite [Proc]
|
21
|
+
# @return the called proc return value.
|
22
|
+
# @api private
|
23
|
+
def self.match_adapter(connection, mysql:, postgres:, sqlite:)
|
24
|
+
case connection.adapter_name
|
25
|
+
when /mysql/i
|
26
|
+
mysql.call
|
27
|
+
when /postgres/i
|
28
|
+
postgres.call
|
29
|
+
when /sqlite/i
|
30
|
+
sqlite.call
|
31
|
+
else
|
32
|
+
unsupported_adapter! connection
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
# Raises an ArgumentError with "Unsupported adapter #{connection.adapter_name}"
|
37
|
+
# @param connection [ActiveRecord::ConnectionAdapters::AbstractAdapter]
|
38
|
+
# @api private
|
39
|
+
def self.unsupported_adapter!(connection)
|
40
|
+
fail ArgumentError, "Unsupported adapter #{connection.adapter_name}"
|
41
|
+
end
|
14
42
|
end
|
@@ -1,10 +1,12 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require 'db_text_search/
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/insensitive_column_adapter'
|
4
|
+
require 'db_text_search/case_insensitive/lower_adapter'
|
5
|
+
require 'db_text_search/case_insensitive/collate_nocase_adapter'
|
4
6
|
|
5
7
|
module DbTextSearch
|
6
|
-
# Provides case-insensitive string-in-set querying, and CI index creation.
|
7
|
-
class
|
8
|
+
# Provides case-insensitive string-in-set querying, LIKE querying, and CI index creation.
|
9
|
+
class CaseInsensitive
|
8
10
|
# @param scope [ActiveRecord::Relation, Class<ActiveRecord::Base>]
|
9
11
|
# @param column [Symbol] name
|
10
12
|
def initialize(scope, column)
|
@@ -14,10 +16,17 @@ module DbTextSearch
|
|
14
16
|
|
15
17
|
# @param value_or_values [String, Array<String>]
|
16
18
|
# @return [ActiveRecord::Relation]
|
17
|
-
def
|
19
|
+
def in(value_or_values)
|
18
20
|
values = Array(value_or_values)
|
19
21
|
return @scope.none if values.empty?
|
20
|
-
@adapter.
|
22
|
+
@adapter.in(values)
|
23
|
+
end
|
24
|
+
|
25
|
+
# @param query [String]
|
26
|
+
# @return [ActiveRecord::Relation] the scope of records with matching prefix.
|
27
|
+
def prefix(query)
|
28
|
+
return @scope.none if query.empty?
|
29
|
+
@adapter.prefix(query)
|
21
30
|
end
|
22
31
|
|
23
32
|
# Adds a case-insensitive column to the given table.
|
@@ -26,25 +35,20 @@ module DbTextSearch
|
|
26
35
|
# @param column_name [String, Symbol]
|
27
36
|
# @param options [Hash] passed to ActiveRecord::ConnectionAdapters::SchemaStatements#add_index
|
28
37
|
def self.add_ci_text_column(connection, table_name, column_name, options = {})
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
connection.add_column(table_name, column_name, 'TEXT COLLATE NOCASE', options)
|
44
|
-
end
|
45
|
-
else
|
46
|
-
fail ArgumentError.new("Unsupported adapter #{connection.adapter_name}")
|
47
|
-
end
|
38
|
+
connection.add_column table_name, column_name, *DbTextSearch.match_adapter(
|
39
|
+
connection,
|
40
|
+
mysql: -> { [:text, options] },
|
41
|
+
postgres: -> {
|
42
|
+
connection.enable_extension 'citext'
|
43
|
+
[(ActiveRecord::VERSION::STRING >= '4.2.0' ? :citext : 'CITEXT'), options]
|
44
|
+
},
|
45
|
+
sqlite: -> {
|
46
|
+
if ActiveRecord::VERSION::MAJOR >= 5
|
47
|
+
[:text, options.merge(collation: 'NOCASE')]
|
48
|
+
else
|
49
|
+
['TEXT COLLATE NOCASE', options]
|
50
|
+
end
|
51
|
+
})
|
48
52
|
end
|
49
53
|
|
50
54
|
# Add an index for case-insensitive string search.
|
@@ -65,15 +69,16 @@ module DbTextSearch
|
|
65
69
|
# @return [Class<AbstractAdapter>]
|
66
70
|
# @api private
|
67
71
|
def self.adapter_class(connection, table_name, column_name)
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
72
|
+
lower_or_insensitive = -> {
|
73
|
+
column_case_sensitive?(connection, table_name, column_name) ? LowerAdapter : InsensitiveColumnAdapter
|
74
|
+
}
|
75
|
+
DbTextSearch.match_adapter(
|
76
|
+
connection,
|
77
|
+
mysql: lower_or_insensitive,
|
78
|
+
postgres: lower_or_insensitive,
|
79
|
+
# Always use COLLATE NOCASE for SQLite, as we can't check if the column is case-sensitive.
|
80
|
+
# It has no performance impact apart from slightly longer query strings for case-insensitive columns.
|
81
|
+
sqlite: -> { CollateNocaseAdapter })
|
77
82
|
end
|
78
83
|
|
79
84
|
# @param connection [ActiveRecord::ConnectionAdapters::AbstractAdapter]
|
@@ -84,14 +89,11 @@ module DbTextSearch
|
|
84
89
|
# @api private
|
85
90
|
def self.column_case_sensitive?(connection, table_name, column_name)
|
86
91
|
column = connection.schema_cache.columns(table_name).detect { |c| c.name == column_name.to_s }
|
87
|
-
|
88
|
-
|
89
|
-
column.case_sensitive?
|
90
|
-
|
91
|
-
|
92
|
-
else
|
93
|
-
fail ArgumentError.new("Unsupported adapter #{connection.adapter_name}")
|
94
|
-
end
|
92
|
+
DbTextSearch.match_adapter(
|
93
|
+
connection,
|
94
|
+
mysql: -> { column.case_sensitive? },
|
95
|
+
postgres: -> { column.sql_type !~ /citext/i },
|
96
|
+
sqlite: -> { DbTextSearch.unsupported_adapter! connection })
|
95
97
|
end
|
96
98
|
end
|
97
99
|
end
|
@@ -1,6 +1,8 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
require 'db_text_search/query_building'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class CaseInsensitive
|
4
6
|
# A base class for CaseInsensitiveStringFinder adapters.
|
5
7
|
# @abstract
|
6
8
|
# @api private
|
@@ -17,7 +19,14 @@ module DbTextSearch
|
|
17
19
|
# @param values [Array<String>]
|
18
20
|
# @return [ActiveRecord::Relation]
|
19
21
|
# @abstract
|
20
|
-
def
|
22
|
+
def in(values)
|
23
|
+
fail 'abstract'
|
24
|
+
end
|
25
|
+
|
26
|
+
# @param query [String]
|
27
|
+
# @return [ActiveRecord::Relation]
|
28
|
+
# @abstract
|
29
|
+
def prefix(query)
|
21
30
|
fail 'abstract'
|
22
31
|
end
|
23
32
|
|
@@ -1,15 +1,27 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class CaseInsensitive
|
4
6
|
# Provides case-insensitive string-in-set querying via COLLATE NOCASE.
|
5
7
|
# @api private
|
6
8
|
class CollateNocaseAdapter < AbstractAdapter
|
7
|
-
# (see AbstractAdapter#
|
8
|
-
def
|
9
|
+
# (see AbstractAdapter#in)
|
10
|
+
def in(values)
|
9
11
|
conn = @scope.connection
|
10
12
|
@scope.where "#{quoted_scope_column} COLLATE NOCASE IN (#{values.map { |v| conn.quote(v.to_s) }.join(', ')})"
|
11
13
|
end
|
12
14
|
|
15
|
+
# (see AbstractAdapter#prefix)
|
16
|
+
def prefix(query)
|
17
|
+
escape = '\\'
|
18
|
+
escaped_query = "#{sanitize_sql_like(query, escape)}%"
|
19
|
+
# assuming case_sensitive_prefix mode to be disabled, prefix it is by default.
|
20
|
+
# this is to avoid adding COLLATE NOCASE here, which prevents index use in SQLite LIKE.
|
21
|
+
@scope.where "#{quoted_scope_column} LIKE ?#{" ESCAPE '#{escape}'" if escaped_query.include?(escape)}",
|
22
|
+
escaped_query
|
23
|
+
end
|
24
|
+
|
13
25
|
# (see AbstractAdapter.add_index)
|
14
26
|
def self.add_index(connection, table_name, column_name, options = {})
|
15
27
|
# TODO: Switch to the native Rails solution once it's landed, as the current one requires SQL dump format.
|
data/lib/db_text_search/{case_insensitive_eq → case_insensitive}/insensitive_column_adapter.rb
RENAMED
@@ -1,14 +1,21 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class CaseInsensitive
|
4
6
|
# Provides case-insensitive string-in-set querying for case-insensitive columns.
|
5
7
|
# @api private
|
6
8
|
class InsensitiveColumnAdapter < AbstractAdapter
|
7
|
-
# (see AbstractAdapter#
|
8
|
-
def
|
9
|
+
# (see AbstractAdapter#in)
|
10
|
+
def in(values)
|
9
11
|
@scope.where(@column => values)
|
10
12
|
end
|
11
13
|
|
14
|
+
# (see AbstractAdapter#prefix)
|
15
|
+
def prefix(query)
|
16
|
+
@scope.where "#{quoted_scope_column} LIKE ?", "#{sanitize_sql_like(query)}%"
|
17
|
+
end
|
18
|
+
|
12
19
|
# (see AbstractAdapter.add_index)
|
13
20
|
def self.add_index(connection, table_name, column_name, options = {})
|
14
21
|
connection.add_index table_name, column_name, options
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/case_insensitive/abstract_adapter'
|
4
|
+
module DbTextSearch
|
5
|
+
class CaseInsensitive
|
6
|
+
# Provides case-insensitive string-in-set querying by applying the database LOWER function.
|
7
|
+
# @api private
|
8
|
+
class LowerAdapter < AbstractAdapter
|
9
|
+
# (see AbstractAdapter#in)
|
10
|
+
def in(values)
|
11
|
+
conn = @scope.connection
|
12
|
+
@scope.where "LOWER(#{quoted_scope_column}) IN (#{values.map { |v| "LOWER(#{conn.quote(v)})" }.join(', ')})"
|
13
|
+
end
|
14
|
+
|
15
|
+
# (see AbstractAdapter#prefix)
|
16
|
+
def prefix(query)
|
17
|
+
@scope.where "LOWER(#{quoted_scope_column}) LIKE LOWER(?)", "#{sanitize_sql_like(query)}%"
|
18
|
+
end
|
19
|
+
|
20
|
+
# (see AbstractAdapter.add_index)
|
21
|
+
def self.add_index(connection, table_name, column_name, options = {})
|
22
|
+
unsupported = -> { DbTextSearch.unsupported_adapter! connection }
|
23
|
+
DbTextSearch.match_adapter(
|
24
|
+
connection,
|
25
|
+
# TODO: Switch to native Rails support once it lands.
|
26
|
+
# https://github.com/rails/rails/pull/18499
|
27
|
+
postgres: -> {
|
28
|
+
options = options.dup
|
29
|
+
options[:name] ||= "#{table_name}_#{column_name}_lower"
|
30
|
+
options[:expression] = "(LOWER(#{connection.quote_column_name(column_name)}) text_pattern_ops)"
|
31
|
+
connection.exec_query(quoted_create_index(connection, table_name, **options))
|
32
|
+
},
|
33
|
+
mysql: unsupported,
|
34
|
+
sqlite: unsupported)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -1,10 +1,12 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require 'db_text_search/
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/postgres_adapter'
|
4
|
+
require 'db_text_search/full_text/mysql_adapter'
|
5
|
+
require 'db_text_search/full_text/sqlite_adapter'
|
4
6
|
|
5
7
|
module DbTextSearch
|
6
8
|
# Provides basic full-text search for a list of terms, and FTS index creation.
|
7
|
-
class
|
9
|
+
class FullText
|
8
10
|
# The default Postgres text search config.
|
9
11
|
DEFAULT_PG_TS_CONFIG = %q('english')
|
10
12
|
|
@@ -18,10 +20,10 @@ module DbTextSearch
|
|
18
20
|
# @param term_or_terms [String, Array<String>]
|
19
21
|
# @param pg_ts_config [String] for Postgres, the TS config to use; ignored for non-postgres.
|
20
22
|
# @return [ActiveRecord::Relation]
|
21
|
-
def
|
22
|
-
values
|
23
|
+
def search(term_or_terms, pg_ts_config: DEFAULT_PG_TS_CONFIG)
|
24
|
+
values = Array(term_or_terms)
|
23
25
|
return @scope.none if values.empty?
|
24
|
-
@adapter.
|
26
|
+
@adapter.search(values, pg_ts_config: pg_ts_config)
|
25
27
|
end
|
26
28
|
|
27
29
|
# Add an index for full text search.
|
@@ -43,16 +45,11 @@ module DbTextSearch
|
|
43
45
|
# @return [Class<AbstractAdapter>]
|
44
46
|
# @api private
|
45
47
|
def self.adapter_class(connection, _table_name, _column_name)
|
46
|
-
|
47
|
-
|
48
|
-
MysqlAdapter
|
49
|
-
|
50
|
-
|
51
|
-
when /sqlite/i
|
52
|
-
SqliteAdapter
|
53
|
-
else
|
54
|
-
fail ArgumentError.new("Unsupported adapter #{connection.adapter_name}")
|
55
|
-
end
|
48
|
+
DbTextSearch.match_adapter(
|
49
|
+
connection,
|
50
|
+
mysql: -> { MysqlAdapter },
|
51
|
+
postgres: -> { PostgresAdapter },
|
52
|
+
sqlite: -> { SqliteAdapter })
|
56
53
|
end
|
57
54
|
end
|
58
55
|
end
|
@@ -1,6 +1,8 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module DbTextSearch
|
2
|
-
class
|
3
|
-
# A base class for
|
4
|
+
class FullText
|
5
|
+
# A base class for FullText adapters.
|
4
6
|
# @abstract
|
5
7
|
# @api private
|
6
8
|
class AbstractAdapter
|
@@ -17,7 +19,7 @@ module DbTextSearch
|
|
17
19
|
# @param pg_ts_config [String] a pg text search config
|
18
20
|
# @return [ActiveRecord::Relation]
|
19
21
|
# @abstract
|
20
|
-
def
|
22
|
+
def search(terms, pg_ts_config:)
|
21
23
|
fail 'abstract'
|
22
24
|
end
|
23
25
|
|
@@ -1,6 +1,8 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class FullText
|
4
6
|
# Provides basic FTS support for MySQL.
|
5
7
|
#
|
6
8
|
# Runs a `MATCH AGAINST` query against a `FULLTEXT` index.
|
@@ -8,8 +10,8 @@ module DbTextSearch
|
|
8
10
|
# @note MySQL v5.6.4+ is required.
|
9
11
|
# @api private
|
10
12
|
class MysqlAdapter < AbstractAdapter
|
11
|
-
# (see AbstractAdapter#
|
12
|
-
def
|
13
|
+
# (see AbstractAdapter#search)
|
14
|
+
def search(terms, pg_ts_config:)
|
13
15
|
@scope.where("MATCH (#{quoted_scope_column}) AGAINST (?)", terms.uniq.join(' '))
|
14
16
|
end
|
15
17
|
|
@@ -1,15 +1,17 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class FullText
|
4
6
|
# Provides basic FTS support for PostgreSQL.
|
5
7
|
#
|
6
8
|
# Runs a `@@ plainto_tsquery` query against a `gist(to_tsvector(...))` index.
|
7
9
|
#
|
8
|
-
# @see DbTextSearch::
|
10
|
+
# @see DbTextSearch::FullText::DEFAULT_PG_TS_CONFIG
|
9
11
|
# @api private
|
10
12
|
class PostgresAdapter < AbstractAdapter
|
11
|
-
# (see AbstractAdapter#
|
12
|
-
def
|
13
|
+
# (see AbstractAdapter#search)
|
14
|
+
def search(terms, pg_ts_config:)
|
13
15
|
@scope.where("to_tsvector(#{pg_ts_config}, #{quoted_scope_column}) @@ plainto_tsquery(#{pg_ts_config}, ?)",
|
14
16
|
terms.uniq.join(' '))
|
15
17
|
end
|
@@ -1,6 +1,8 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'db_text_search/full_text/abstract_adapter'
|
2
4
|
module DbTextSearch
|
3
|
-
class
|
5
|
+
class FullText
|
4
6
|
# Provides very basic FTS support for SQLite.
|
5
7
|
#
|
6
8
|
# Runs a `LIKE %term%` query for each term, joined with `AND`.
|
@@ -9,17 +11,16 @@ module DbTextSearch
|
|
9
11
|
# @note .add_index is a no-op.
|
10
12
|
# @api private
|
11
13
|
class SqliteAdapter < AbstractAdapter
|
12
|
-
# (see AbstractAdapter#
|
13
|
-
def
|
14
|
+
# (see AbstractAdapter#search)
|
15
|
+
def search(terms, pg_ts_config:)
|
14
16
|
quoted_col = quoted_scope_column
|
15
17
|
terms.map(&:downcase).uniq.inject(@scope) do |scope, term|
|
16
18
|
scope.where("#{quoted_col} COLLATE NOCASE LIKE ?", "%#{sanitize_sql_like term}%")
|
17
19
|
end
|
18
20
|
end
|
19
21
|
|
20
|
-
#
|
21
|
-
def self.add_index(
|
22
|
-
# A no-op, as we just use LIKE for sqlite.
|
22
|
+
# A no-op, as we just use LIKE for sqlite.
|
23
|
+
def self.add_index(_connection, _table_name, _column_name, name:, pg_ts_config:)
|
23
24
|
end
|
24
25
|
end
|
25
26
|
end
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
1
3
|
module DbTextSearch
|
2
4
|
# Common methods for building SQL that use @scope and @column instance variables.
|
3
5
|
# @api private
|
@@ -6,6 +8,12 @@ module DbTextSearch
|
|
6
8
|
base.extend ClassMethods
|
7
9
|
end
|
8
10
|
|
11
|
+
# @return [String] SQL-quoted string suitable for use in a LIKE statement, with % and _ escaped.
|
12
|
+
def sanitize_sql_like(string, escape_character = '\\')
|
13
|
+
pattern = Regexp.union(escape_character, '%', '_')
|
14
|
+
string.gsub(pattern) { |x| [escape_character, x].join }
|
15
|
+
end
|
16
|
+
|
9
17
|
protected
|
10
18
|
|
11
19
|
# @return [String] SQL-quoted scope table name.
|
@@ -23,12 +31,6 @@ module DbTextSearch
|
|
23
31
|
"#{quoted_scope_table}.#{quoted_column}"
|
24
32
|
end
|
25
33
|
|
26
|
-
# @return [String] SQL-quoted string suitable for use in a LIKE statement, with % and _ escaped.
|
27
|
-
def sanitize_sql_like(string, escape_character = "\\")
|
28
|
-
pattern = Regexp.union(escape_character, '%', '_')
|
29
|
-
string.gsub(pattern) { |x| [escape_character, x].join }
|
30
|
-
end
|
31
|
-
|
32
34
|
# Common methods for building SQL
|
33
35
|
# @api private
|
34
36
|
module ClassMethods
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: db_text_search
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Gleb Mazovetskiy
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-
|
11
|
+
date: 2016-04-06 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -114,11 +114,10 @@ dependencies:
|
|
114
114
|
- - "~>"
|
115
115
|
- !ruby/object:Gem::Version
|
116
116
|
version: '3.4'
|
117
|
-
description:
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
* Basic full-text search for a list of terms, and FTS index creation.
|
117
|
+
description: Different relational databases treat text search very differently. DbTextSearch
|
118
|
+
provides a unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQL
|
119
|
+
to do case-insensitive string-in-set querying and CI index creation, and basic full-text
|
120
|
+
search for a list of terms, and FTS index creation.
|
122
121
|
email:
|
123
122
|
- glex.spb@gmail.com
|
124
123
|
executables: []
|
@@ -131,16 +130,16 @@ files:
|
|
131
130
|
- README.md
|
132
131
|
- db_text_search.gemspec
|
133
132
|
- lib/db_text_search.rb
|
134
|
-
- lib/db_text_search/
|
135
|
-
- lib/db_text_search/
|
136
|
-
- lib/db_text_search/
|
137
|
-
- lib/db_text_search/
|
138
|
-
- lib/db_text_search/
|
139
|
-
- lib/db_text_search/
|
140
|
-
- lib/db_text_search/
|
141
|
-
- lib/db_text_search/
|
142
|
-
- lib/db_text_search/
|
143
|
-
- lib/db_text_search/
|
133
|
+
- lib/db_text_search/case_insensitive.rb
|
134
|
+
- lib/db_text_search/case_insensitive/abstract_adapter.rb
|
135
|
+
- lib/db_text_search/case_insensitive/collate_nocase_adapter.rb
|
136
|
+
- lib/db_text_search/case_insensitive/insensitive_column_adapter.rb
|
137
|
+
- lib/db_text_search/case_insensitive/lower_adapter.rb
|
138
|
+
- lib/db_text_search/full_text.rb
|
139
|
+
- lib/db_text_search/full_text/abstract_adapter.rb
|
140
|
+
- lib/db_text_search/full_text/mysql_adapter.rb
|
141
|
+
- lib/db_text_search/full_text/postgres_adapter.rb
|
142
|
+
- lib/db_text_search/full_text/sqlite_adapter.rb
|
144
143
|
- lib/db_text_search/query_building.rb
|
145
144
|
- lib/db_text_search/version.rb
|
146
145
|
- shared.gemfile
|
@@ -167,7 +166,7 @@ rubyforge_project:
|
|
167
166
|
rubygems_version: 2.5.1
|
168
167
|
signing_key:
|
169
168
|
specification_version: 4
|
170
|
-
summary: A unified interface on top of ActiveRecord for SQLite, MySQL, and
|
171
|
-
|
169
|
+
summary: A unified interface on top of ActiveRecord for SQLite, MySQL, and PostgreSQLfor
|
170
|
+
case-insensitive string search and basic full-text search.
|
172
171
|
test_files: []
|
173
172
|
has_rdoc:
|
@@ -1,35 +0,0 @@
|
|
1
|
-
require 'db_text_search/case_insensitive_eq/abstract_adapter'
|
2
|
-
module DbTextSearch
|
3
|
-
class CaseInsensitiveEq
|
4
|
-
# Provides case-insensitive string-in-set querying by applying the database LOWER function.
|
5
|
-
# @api private
|
6
|
-
class LowerAdapter < AbstractAdapter
|
7
|
-
# (see AbstractAdapter#find)
|
8
|
-
def find(values)
|
9
|
-
conn = @scope.connection
|
10
|
-
@scope.where "LOWER(#{quoted_scope_column}) IN (#{values.map { |v| "LOWER(#{conn.quote(v.to_s)})" }.join(', ')})"
|
11
|
-
end
|
12
|
-
|
13
|
-
# (see AbstractAdapter.add_index)
|
14
|
-
def self.add_index(connection, table_name, column_name, options = {})
|
15
|
-
if connection.adapter_name =~ /postgres/i
|
16
|
-
# TODO: Switch to native Rails support once it lands.
|
17
|
-
# https://github.com/rails/rails/pull/18499
|
18
|
-
options = options.dup
|
19
|
-
options[:name] ||= "#{table_name}_#{column_name}_lower"
|
20
|
-
options[:expression] = "(LOWER(#{connection.quote_column_name(column_name)}))"
|
21
|
-
if defined?(::SchemaPlus)
|
22
|
-
connection.add_index(table_name, column_name, options)
|
23
|
-
else
|
24
|
-
connection.exec_query quoted_create_index(connection, table_name, **options)
|
25
|
-
end
|
26
|
-
elsif connection.adapter_name =~ /mysql/i
|
27
|
-
fail ArgumentError.new('MySQL case-insensitive index creation for case-sensitive columns is not supported.')
|
28
|
-
else
|
29
|
-
fail ArgumentError.new(
|
30
|
-
"Cannot create a case-insensitive index for case-sensitive column on #{connection.adapter_name}.")
|
31
|
-
end
|
32
|
-
end
|
33
|
-
end
|
34
|
-
end
|
35
|
-
end
|