redi_search 2.0.0 → 4.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/Gemfile CHANGED
@@ -9,10 +9,11 @@ gemspec
9
9
  gem "appraisal", "~> 2.2"
10
10
  gem "faker"
11
11
  gem "mocha"
12
- gem "pry"
13
12
  gem "rubocop"
13
+ gem "rubocop-minitest"
14
14
  gem "rubocop-performance"
15
+ gem "rubocop-rake"
15
16
  gem "simplecov"
16
17
  gem "sqlite3"
17
18
 
18
- gem "activerecord", "6.0.0.rc2"
19
+ gem "activerecord", "~> 6.1"
data/README.md CHANGED
@@ -6,8 +6,7 @@
6
6
 
7
7
  # RediSearch
8
8
 
9
- [![Build Status](https://travis-ci.com/npezza93/redi_search.svg?branch=master)](https://travis-ci.com/npezza93/redi_search)
10
- [![Test Coverage](https://api.codeclimate.com/v1/badges/c6437acac5684de2549d/test_coverage)](https://codeclimate.com/github/npezza93/redi_search/test_coverage)
9
+ ![Build Status](https://github.com/npezza93/redi_search/workflows/tests/badge.svg)
11
10
  [![Maintainability](https://api.codeclimate.com/v1/badges/c6437acac5684de2549d/maintainability)](https://codeclimate.com/github/npezza93/redi_search/maintainability)
12
11
 
13
12
  A simple, but powerful, Ruby wrapper around RediSearch, a search engine on top of
@@ -244,13 +243,11 @@ With no options: `{ place: :geo }`
244
243
 
245
244
  ## Document
246
245
 
247
- A `Document` is the Ruby representation of a RediSearch document.
246
+ A `Document` is the Ruby representation of a Redis hash.
248
247
 
249
- You can fetch a `Document` using `.get` or `.mget` class methods.
248
+ You can fetch a `Document` using `.get` class methods.
250
249
  - `get(index, document_id)` fetches a single `Document` in an `Index` for a
251
250
  given `document_id`.
252
- - `mget(index, *document_ids)` fetches a collection of `Document`s
253
- in an `Index` for the given `document_ids`.
254
251
 
255
252
  You can also make a `Document` instance using the
256
253
  `.for_object(index, record, serializer: nil, only: [])` class method. It takes
@@ -269,10 +266,7 @@ to override each other. There is also a `#document_id_without_index` method
269
266
  which removes the prepended index name.
270
267
 
271
268
  Finally there is a `#del` method that will remove the `Document` from the
272
- `Index`. It optionally accepts a `delete_document` named argument that signifies
273
- whether the `Document` should be completely removed from the Redis instance vs
274
- just the `Index`.
275
-
269
+ `Index`.
276
270
 
277
271
  ## Index
278
272
 
@@ -314,42 +308,32 @@ RediSearch::Index.new(name_of_index, schema)
314
308
  - If set, we avoid saving the term frequencies in the index. This saves
315
309
  memory but does not allow sorting based on the frequencies of a given
316
310
  term within the document.
317
- - `drop`
311
+ - `drop(keep_docs: false)`
318
312
  - Drops the `Index` from the Redis instance, returns a boolean. Has an
319
313
  accompanying bang method that will raise an exception upon failure. Will
320
- return `false` if the `Index` has already been dropped.
314
+ return `false` if the `Index` has already been dropped. Takes an option
315
+ keyword arg, `keep_docs`, that will by default remove all the document
316
+ hashes in Redis.
321
317
  - `exist?`
322
318
  - Returns a boolean signifying `Index` existence.
323
319
  - `info`
324
320
  - Returns a struct object with all the information about the `Index`.
325
321
  - `fields`
326
322
  - Returns an array of the field names in the `Index`.
327
- - `add(document, score: 1.0, replace: {}, language: nil, no_save: false)`
328
- - Takes a `Document` object and options. Has an
323
+ - `add(document)`
324
+ - Takes a `Document` object. Has an
329
325
  accompanying bang method that will raise an exception upon failure.
330
- - `score` -> The `Document`'s rank, a value between 0.0 and 1.0
331
- - `language` -> Use a stemmer for the supplied language during indexing.
332
- - `no_save` -> Don't save the actual `Document` in the database and only index it.
333
- - `replace` -> Accepts a boolean or a hash. If a truthy value is passed, we
334
- will do an UPSERT style insertion - and delete an older version of the
335
- `Document` if it exists.
336
- - `replace: { partial: true }` -> Allows you to not have to specify all
337
- fields for reindexing. Fields not given to the command will be loaded from
338
- the current version of the `Document`.
339
- - `add_multiple(documents, score: 1.0, replace: {}, language: nil, no_save: false)`
326
+ - `add_multiple(documents)`
340
327
  - Takes an array of `Document` objects. This provides a more performant way to
341
328
  add multiple documents to the `Index`. Accepts the same options as `add`.
342
- - `del(document, delete_document: false)`
343
- - Removes a `Document` from the `Index`. `delete_document` signifies whether the
344
- `Document` should be completely removed from the Redis instance vs just the
345
- `Index`.
329
+ - `del(document)`
330
+ - Removes a `Document` from the `Index`.
346
331
  - `document_count`
347
332
  - Returns the number of `Document`s in the `Index`
348
333
  - `add_field(field_name, schema)`
349
334
  - Adds a new field to the `Index`. Ex: `index.add_field(:first_name, text: { phonetic: "dm:en" })`
350
- - `reindex(documents, recreate: false, **options)`
335
+ - `reindex(documents, recreate: false)`
351
336
  - If `recreate` is `true` the `Index` will be dropped and recreated
352
- - `options` accepts the same options as `add`
353
337
 
354
338
 
355
339
  ## Searching
@@ -521,11 +505,9 @@ end
521
505
  This will automatically add `User.search` and `User.spellcheck`
522
506
  methods which behave the same as if you called them on an `Index` instance.
523
507
 
524
- `User.reindex(only: [], **options)` is also added and behaves similarly to
525
- `RediSearch::Index#reindex`. Some of the differences include:
526
- - By default, does an upsert for all `Document`s added using the
527
- option `replace: { partial: true }`.
528
- - `Document`s do not to be passed as the first parameter. The `search_import`
508
+ `User.reindex(recreate: false, only: [])` is also added and behaves
509
+ similarly to `RediSearch::Index#reindex`. Some of the differences include:
510
+ - `Document`s do not need to be passed as the first parameter. The `search_import`
529
511
  scope is automatically called and all the records are converted
530
512
  to `Document`s.
531
513
  - Accepts an optional `only` parameter where you can specify a limited number
@@ -561,6 +543,10 @@ class User < ApplicationRecord
561
543
  end
562
544
  ```
563
545
 
546
+ When searching, by default a collection of `Document`s is returned. Calling
547
+ `#results` on the search query will execute the search, and then look up all the
548
+ found records in the database and return an ActiveRecord relation.
549
+
564
550
  The default `Index` name for model `Index`s is
565
551
  `#{model_name.plural}_#{RediSearch.env}`. The `redi_search` method takes an
566
552
  optional `index_prefix` argument which gets prepended to the index name:
data/Rakefile CHANGED
@@ -20,5 +20,6 @@ Rake::TestTask.new("test:integration") do |t|
20
20
  t.test_files = FileList["test/integration/**/*_test.rb"]
21
21
  end
22
22
 
23
+ desc "run all tests"
23
24
  task test: [:default]
24
25
  task default: ["test:integration", "test:unit", :rubocop]
data/bin/console CHANGED
@@ -5,7 +5,7 @@ require "bundler/setup"
5
5
  require "redi_search"
6
6
 
7
7
  require "faker"
8
- require "pry"
8
+ require "irb"
9
9
  require "active_support/logger"
10
10
  require "active_record"
11
11
 
@@ -35,10 +35,7 @@ def seed_users(count = 10_000)
35
35
  end
36
36
 
37
37
  def reload!
38
- Object.send :remove_const, :RediSearch
39
- files = $LOADED_FEATURES.select { |feat| feat =~ /\/redi_search\// }
40
- files.each { |file| load file }
41
- true
38
+ exec($0)
42
39
  end
43
40
 
44
- Pry.start
41
+ IRB.start
@@ -2,12 +2,14 @@
2
2
 
3
3
  source "https://rubygems.org"
4
4
 
5
+ gem "appraisal", "~> 2.2"
6
+ gem "faker"
5
7
  gem "mocha"
6
- gem "pry"
7
8
  gem "rubocop"
9
+ gem "rubocop-minitest"
8
10
  gem "rubocop-performance"
9
11
  gem "simplecov"
10
12
  gem "sqlite3"
11
- gem "activerecord", "< 5.2", ">= 5.1"
13
+ gem "activerecord", "< 6.1", ">= 6.0"
12
14
 
13
15
  gemspec path: "../"
@@ -2,12 +2,14 @@
2
2
 
3
3
  source "https://rubygems.org"
4
4
 
5
+ gem "appraisal", "~> 2.2"
6
+ gem "faker"
5
7
  gem "mocha"
6
- gem "pry"
7
8
  gem "rubocop"
9
+ gem "rubocop-minitest"
8
10
  gem "rubocop-performance"
9
11
  gem "simplecov"
10
12
  gem "sqlite3"
11
- gem "activerecord", "< 6.0", ">= 5.2"
13
+ gem "activerecord", "< 6.2", ">= 6.1"
12
14
 
13
15
  gemspec path: "../"
data/lib/redi_search.rb CHANGED
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "delegate"
4
+ require "forwardable"
3
5
  require "redis"
4
6
  require "active_support/lazy_load_hooks"
5
7
 
@@ -29,7 +29,7 @@ module RediSearch
29
29
  end
30
30
 
31
31
  def field_schema
32
- @field_schema ||= Schema.make_field(field_name, raw_schema)
32
+ @field_schema ||= Schema.make_field(field_name, raw_schema).to_a
33
33
  end
34
34
  end
35
35
  end
@@ -9,10 +9,12 @@ module RediSearch
9
9
  class Client
10
10
  def initialize(redis = Redis.new)
11
11
  @redis = redis
12
+ @pipeline = false
12
13
  end
13
14
 
14
- def call!(command, *params)
15
+ def call!(command, *params, skip_ft: false)
15
16
  instrument(command.downcase, query: [command, params]) do
17
+ command = "FT.#{command}" unless skip_ft
16
18
  send_command(command, *params)
17
19
  end
18
20
  end
@@ -20,24 +22,32 @@ module RediSearch
20
22
  def multi
21
23
  Response.new(redis.multi do
22
24
  instrument("pipeline", query: ["begin pipeline"])
23
- yield
25
+ capture_pipeline { yield }
24
26
  instrument("pipeline", query: ["finish pipeline"])
25
27
  end)
26
28
  end
27
29
 
28
30
  private
29
31
 
30
- attr_reader :redis
32
+ attr_reader :redis
33
+ attr_accessor :pipeline
34
+
35
+ def capture_pipeline
36
+ self.pipeline = true
37
+ yield
38
+ self.pipeline = false
39
+ end
31
40
 
32
41
  def send_command(command, *params)
33
- Response.new(redis.call("FT.#{command}", *params))
42
+ Response.new(redis.call(command, *params))
34
43
  end
35
44
 
36
45
  def instrument(action, payload, &block)
37
46
  ActiveSupport::Notifications.instrument(
38
47
  "action.redi_search",
39
- { name: "RediSearch", action: action }.merge(payload),
40
- &Proc.new(&(block || proc {}))
48
+ { name: "RediSearch", action: action, inside_pipeline: pipeline }.
49
+ merge(payload),
50
+ &Proc.new(&(block || proc {})) # rubocop:disable Lint/EmptyBlock
41
51
  )
42
52
  end
43
53
  end
@@ -6,7 +6,7 @@ module RediSearch
6
6
  def ok?
7
7
  case response
8
8
  when String then response == "OK"
9
- when Integer then response == 1
9
+ when Integer then response >= 1
10
10
  when Array then array_ok?
11
11
  else response
12
12
  end
@@ -31,7 +31,8 @@ module RediSearch
31
31
  attr_reader :index, :schema, :options
32
32
 
33
33
  def command
34
- ["CREATE", index.name, *extract_options.compact, "SCHEMA", schema.to_a]
34
+ ["CREATE", index.name, "ON", "HASH", "PREFIX", 1, index.name,
35
+ *extract_options.compact, "SCHEMA", schema.to_a]
35
36
  end
36
37
 
37
38
  def extract_options
@@ -11,8 +11,8 @@ module RediSearch
11
11
  def for_object(index, record, serializer: nil, only: [])
12
12
  object_to_serialize = serializer&.new(record) || record
13
13
 
14
- field_values = index.schema.fields.map do |field|
15
- next unless only.empty? || only.include?(field.to_sym)
14
+ field_values = index.schema.fields.map(&:name).map do |field|
15
+ next unless only.empty? || only.include?(field)
16
16
 
17
17
  [field.to_s, object_to_serialize.public_send(field)]
18
18
  end.compact.to_h
@@ -23,10 +23,6 @@ module RediSearch
23
23
  def get(index, document_id)
24
24
  Finder.new(index, document_id).find
25
25
  end
26
-
27
- def mget(index, *document_ids)
28
- Finder.new(index, *document_ids).find
29
- end
30
26
  end
31
27
 
32
28
  attr_reader :attributes, :score
@@ -40,17 +36,20 @@ module RediSearch
40
36
  load_attributes
41
37
  end
42
38
 
43
- def del(delete_document: false)
44
- command = ["DEL", index.name, document_id, ("DD" if delete_document)]
45
- call!(*command.compact).ok?
39
+ def del
40
+ RediSearch.client.call!("DEL", document_id, skip_ft: true).ok?
46
41
  end
47
42
 
48
43
  def schema_fields
49
- @schema_fields ||= index.schema.fields.map(&:to_s)
44
+ @schema_fields ||= index.schema.fields.map do |field|
45
+ field.name.to_s
46
+ end
50
47
  end
51
48
 
52
49
  def redis_attributes
53
- attributes.to_a.flatten
50
+ attributes.flat_map do |field, value|
51
+ [field, index.schema[field.to_sym].serialize(value)]
52
+ end
54
53
  end
55
54
 
56
55
  def document_id
@@ -73,10 +72,6 @@ module RediSearch
73
72
 
74
73
  attr_reader :index
75
74
 
76
- def call!(*command)
77
- RediSearch.client.call!(*command)
78
- end
79
-
80
75
  def load_attributes
81
76
  attributes.each do |field, value|
82
77
  next unless schema_fields.include? field.to_s
@@ -3,75 +3,37 @@
3
3
  module RediSearch
4
4
  class Document
5
5
  class Finder
6
- def initialize(index, *document_ids)
6
+ def initialize(index, document_id)
7
7
  @index = index
8
- @document_ids = [*document_ids]
8
+ @document_id = document_id
9
9
  end
10
10
 
11
11
  def find
12
- if multi?
13
- parse_multi_documents
14
- else
15
- parse_document(document_ids.first, response)
16
- end
12
+ Document.new(index, document_id, Hash[*response]) if response?
17
13
  end
18
14
 
19
15
  private
20
16
 
21
- attr_reader :index, :document_ids
17
+ attr_reader :index, :document_id
22
18
 
23
19
  def response
24
- @response ||= call!(get_command, index.name, *prepended_document_ids)
20
+ @response ||= call!("HGETALL", prepended_document_id)
25
21
  end
26
22
 
27
23
  def call!(*command)
28
- RediSearch.client.call!(*command)
29
- end
30
-
31
- def get_command
32
- if multi?
33
- "MGET"
34
- else
35
- "GET"
36
- end
37
- end
38
-
39
- def multi?
40
- document_ids.size > 1
41
- end
42
-
43
- def prepended_document_ids
44
- document_ids.map do |document_id|
45
- prepend_document_id(document_id)
46
- end
24
+ RediSearch.client.call!(*command, skip_ft: true)
47
25
  end
48
26
 
49
- def prepend_document_id(id)
50
- if id.to_s.start_with? index.name
51
- id
27
+ def prepended_document_id
28
+ if document_id.to_s.start_with? index.name
29
+ document_id
52
30
  else
53
- "#{index.name}#{id}"
31
+ "#{index.name}#{document_id}"
54
32
  end
55
33
  end
56
34
 
57
- def parse_multi_documents
58
- document_ids.map.with_index do |document_id, index|
59
- parse_document(document_id, response[index])
60
- end.compact
61
- end
62
-
63
- def parse_document(document_id, document_response)
64
- return unless document_response?(document_response)
65
-
66
- Document.new(index, document_id, Hash[*document_response])
67
- end
68
-
69
- def document_response?(document_response)
70
- if document_response.respond_to?(:empty?)
71
- !document_response.empty?
72
- else
73
- !document_response.nil?
74
- end
35
+ def response?
36
+ !response.to_a.empty?
75
37
  end
76
38
  end
77
39
  end
@@ -0,0 +1,28 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RediSearch
4
+ class Hset
5
+ def initialize(index, document)
6
+ @index = index
7
+ @document = document
8
+ end
9
+
10
+ def call!
11
+ RediSearch.client.call!(*command, skip_ft: true)
12
+ end
13
+
14
+ def call
15
+ call!
16
+ rescue Redis::CommandError
17
+ false
18
+ end
19
+
20
+ private
21
+
22
+ attr_reader :index, :document
23
+
24
+ def command
25
+ ["HSET", document.document_id, document.redis_attributes].compact
26
+ end
27
+ end
28
+ end