asari 0.10.4 → 0.11.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/.travis.yml +9 -0
- data/README.md +27 -12
- data/Rakefile +5 -0
- data/asari.gemspec +2 -0
- data/lib/asari.rb +12 -3
- data/lib/asari/active_record.rb +13 -8
- data/lib/asari/version.rb +1 -1
- data/spec/documents_spec.rb +6 -4
- data/spec/search_spec.rb +21 -1
- data/spec_helper.rb +2 -0
- metadata +40 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f674450079a0b564ad5b1623ecd78b81d8bbfbf8
|
4
|
+
data.tar.gz: 42ff0091617dd6975c82c7ef2f4328d2fa9bba74
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: a8d64c17e02e70be0d07559be80c98e0f03ad09a1869b0240516ed3e69d27aa5822586df36f45035e0a9aed26c2698ac6fb8c6ea63cdce2ba1633054e3594e70
|
7
|
+
data.tar.gz: f6187beea7a8de32c173d4f873248a01435b860d611c910f90a69c77d48ee7e4216372165e8d684fce4415f6fd7ca76fcb80dd80fed67d7a42c376191d873fd6
|
data/.gitignore
CHANGED
data/.travis.yml
ADDED
data/README.md
CHANGED
@@ -1,7 +1,15 @@
|
|
1
1
|
# Asari
|
2
2
|
|
3
|
+
## Note on 2013-01-01 Amazon API support
|
4
|
+
|
5
|
+
We are actively working on a 1.0 version with support for the latest api along with complete Active Asari support. It should be solid enough to use in production via
|
6
|
+
|
7
|
+
gem "asari", :git => "git@github.com:wellbredgrapefruit/asari.git", :branch => "1.0"
|
8
|
+
|
3
9
|
## Description
|
4
10
|
|
11
|
+
[![Build Status](https://travis-ci.org/wellbredgrapefruit/asari.svg?branch=master)](https://travis-ci.org/wellbredgrapefruit/asari)
|
12
|
+
|
5
13
|
Asari is a Ruby wrapper for AWS CloudSearch, with optional ActiveRecord support
|
6
14
|
for easy integration with your Rails apps.
|
7
15
|
|
@@ -28,21 +36,28 @@ Amazon Cloud Search will give you a Search Endpoint and Document Endpoint. When
|
|
28
36
|
#### Boolean Query Usage
|
29
37
|
|
30
38
|
asari.search(filter: { and: { title: "donut", type: "cruller" }})
|
31
|
-
asari.search("boston creme", filter: { and: { title: "donut", or: { type: "cruller
|
39
|
+
asari.search("boston creme", filter: { and: { title: "donut", or: { type: "cruller|twist" }}}) # Full text search and nested boolean logic
|
40
|
+
|
41
|
+
Boolean Queries can also be provided as strings that will be directly used in
|
42
|
+
the query request. This allows for more control over the query string as well as
|
43
|
+
multiple uses of the same field name (where Ruby hashes don't allow for
|
44
|
+
non-unique keys.)
|
45
|
+
|
46
|
+
asari.search(filter: "(or type:'donut' type:'bagel')")
|
32
47
|
|
33
48
|
For more information on how to use Cloudsearch boolean queries, [see the
|
34
49
|
documentation.](http://docs.aws.amazon.com/cloudsearch/latest/developerguide/booleansearch.html)
|
35
50
|
|
36
51
|
### Geospatial Query Usage
|
37
52
|
|
38
|
-
While Cloudsearch does not natively support location search, you can implement rudimentary location search by representing latitude and longitude as integers in your search domain. Asari has a Geography module you can use to simplify the conversion of latitude and longitude to cartesian coordinates as well as the generation of a coordinate box to search within. Asari's Boolean Query syntax can then be used to search within the area. Note that because Cloudsearch only supports 32-bit unsigned integers, it is only possible to store latitude and longitude to two place values. This means very precise search isn't possible using Asari and Cloudsearch.
|
53
|
+
While Cloudsearch does not natively support location search, you can implement rudimentary location search by representing latitude and longitude as integers in your search domain. Asari has a Geography module you can use to simplify the conversion of latitude and longitude to cartesian coordinates as well as the generation of a coordinate box to search within. Asari's Boolean Query syntax can then be used to search within the area. Note that because Cloudsearch only supports 32-bit unsigned integers, it is only possible to store latitude and longitude to two place values. This means very precise search isn't possible using Asari and Cloudsearch.
|
39
54
|
|
40
55
|
coordinates = Asari::Geography.degrees_to_int(lat: 45.52, lng: 122.68)
|
41
56
|
#=> { lat: 2506271416, lng: 111298648 }
|
42
57
|
asari.add_item("1", { name: "Tommy Morgan", lat: coordinates[:lat], lng: coordinates[:lng] })
|
43
58
|
#=> nil
|
44
59
|
coordinate_box = Asari::Geography.coordinate_box(lat: 45.2, lng: 122.85, meters: 7500)
|
45
|
-
#=> { lat: 2505521415..2507021417, lng: 111263231..111334065 }
|
60
|
+
#=> { lat: 2505521415..2507021417, lng: 111263231..111334065 }
|
46
61
|
asari.search("tommy", filter: { and: coordinate_box }
|
47
62
|
#=> ["1"] = a list of document IDs
|
48
63
|
|
@@ -62,7 +77,7 @@ you want to actually connect to the search index, just do the following:
|
|
62
77
|
|
63
78
|
You can turn the sandbox back on, if you like, by setting the mode to `:sandbox`
|
64
79
|
again.
|
65
|
-
|
80
|
+
|
66
81
|
#### Pagination
|
67
82
|
|
68
83
|
Asari defaults to a page size of 10 (because that's CloudSearch's default), but
|
@@ -73,7 +88,7 @@ it allows you to specify pagination parameters with any search:
|
|
73
88
|
The results you get back from Asari#search aren't actually Array objects,
|
74
89
|
either: they're Asari::Collection objects, which are (currently) API-compatible
|
75
90
|
with will\_paginate:
|
76
|
-
|
91
|
+
|
77
92
|
results = asari.search("tommy", :page_size => 30, :page => 10)
|
78
93
|
results.total_entries #=> 5000
|
79
94
|
results.total_pages #=> 167
|
@@ -83,9 +98,9 @@ with will\_paginate:
|
|
83
98
|
|
84
99
|
#### Retrieving Data From Index Fields
|
85
100
|
|
86
|
-
By default Asari only returns the document id's for any hits returned from a search.
|
101
|
+
By default Asari only returns the document id's for any hits returned from a search.
|
87
102
|
If you have result_enabled a index field you can have asari resturn that field in the
|
88
|
-
result set without having to hit a database to get the results. Simply pass the
|
103
|
+
result set without having to hit a database to get the results. Simply pass the
|
89
104
|
:return_fields option with an array of fields
|
90
105
|
|
91
106
|
results = asari.search "Beavis", :return_fields => ["name", "address"]
|
@@ -97,7 +112,7 @@ The result will look like this
|
|
97
112
|
|
98
113
|
#### ActiveRecord
|
99
114
|
|
100
|
-
By default the ActiveRecord module for Asari is not included in your project. To use it you will need to require it via
|
115
|
+
By default the ActiveRecord module for Asari is not included in your project. To use it you will need to require it via
|
101
116
|
|
102
117
|
require 'asari/active_record'
|
103
118
|
|
@@ -118,10 +133,10 @@ index, and can represent any function on your AR object. You can then interact
|
|
118
133
|
with your AR objects as follows:
|
119
134
|
|
120
135
|
# Klass.asari_find returns a list of model objects in an
|
121
|
-
# Asari::Collection...
|
136
|
+
# Asari::Collection...
|
122
137
|
User.asari_find("tommy") #=> [<User:...>, <User:...>, <User:...>]
|
123
138
|
User.asari_find("tommy", :rank => "name")
|
124
|
-
|
139
|
+
|
125
140
|
# or with a specific instance, if you need to manually do some index
|
126
141
|
# management...
|
127
142
|
@user.asari_add_to_index
|
@@ -134,7 +149,7 @@ You can also specify a :when option, like so:
|
|
134
149
|
:favorite_sweater], :when => :indexable)
|
135
150
|
|
136
151
|
or
|
137
|
-
|
152
|
+
|
138
153
|
asari_index("search-domain-for-users", [:name, :email, :twitter_handle,
|
139
154
|
:favorite_sweater], :when => Proc.new { |user| !user.admin && user.indexable })
|
140
155
|
|
@@ -224,6 +239,6 @@ without limitation the rights to use, copy, modify, merge, publish,
|
|
224
239
|
distribute, sublicense, and/or sell copies of the Software, and to
|
225
240
|
permit persons to whom the Software is furnished to do so, subject to
|
226
241
|
the following conditions:
|
227
|
-
|
242
|
+
|
228
243
|
The above copyright notice and this permission notice shall be
|
229
244
|
included in all copies or substantial portions of the Software.
|
data/Rakefile
CHANGED
data/asari.gemspec
CHANGED
data/lib/asari.rb
CHANGED
@@ -55,6 +55,7 @@ class Asari
|
|
55
55
|
#
|
56
56
|
# @asari.search("fritters") #=> ["13","28"]
|
57
57
|
# @asari.search(filter: { and: { type: 'donuts' }}) #=> ["13,"28","35","50"]
|
58
|
+
# @asari.search(filter: "(or type:'donut' type:'bagel')") #=> ["13,"28","35","50", "80"]
|
58
59
|
# @asari.search("fritters", filter: { and: { type: 'donuts' }}) #=> ["13"]
|
59
60
|
#
|
60
61
|
# Returns: An Asari::Collection containing all document IDs in the system that match the
|
@@ -67,7 +68,13 @@ class Asari
|
|
67
68
|
return Asari::Collection.sandbox_fake if self.class.mode == :sandbox
|
68
69
|
term,options = "",term if term.is_a?(Hash) and options.empty?
|
69
70
|
|
70
|
-
bq =
|
71
|
+
bq = if options[:filter]
|
72
|
+
if options[:filter].is_a?(String)
|
73
|
+
options[:filter]
|
74
|
+
else
|
75
|
+
boolean_query(options[:filter])
|
76
|
+
end
|
77
|
+
end
|
71
78
|
page_size = options[:page_size].nil? ? 10 : options[:page_size].to_i
|
72
79
|
|
73
80
|
url = "http://search-#{search_domain}.#{aws_region}.cloudsearch.amazonaws.com/#{api_version}/search"
|
@@ -204,8 +211,10 @@ class Asari
|
|
204
211
|
else
|
205
212
|
if value.is_a?(Range) || value.is_a?(Integer)
|
206
213
|
memo += " #{key}:#{value}"
|
207
|
-
|
208
|
-
memo += " #{key}
|
214
|
+
elsif value.is_a?(String) && value =~ /\A\d*\.\.\d*\Z/
|
215
|
+
memo += " #{key}:#{value}"
|
216
|
+
elsif !value.to_s.empty?
|
217
|
+
memo += " #{key}:'#{value.to_s}'"
|
209
218
|
end
|
210
219
|
end
|
211
220
|
memo
|
data/lib/asari/active_record.rb
CHANGED
@@ -86,10 +86,7 @@ class Asari
|
|
86
86
|
if self.asari_when
|
87
87
|
return unless asari_should_index?(obj)
|
88
88
|
end
|
89
|
-
data =
|
90
|
-
self.asari_fields.each do |field|
|
91
|
-
data[field] = obj.send(field) || ""
|
92
|
-
end
|
89
|
+
data = self.asari_data_item(obj)
|
93
90
|
self.asari_instance.add_item(obj.send(:id), data)
|
94
91
|
rescue Asari::DocumentUpdateException => e
|
95
92
|
self.asari_on_error(e)
|
@@ -104,15 +101,23 @@ class Asari
|
|
104
101
|
return
|
105
102
|
end
|
106
103
|
end
|
107
|
-
data =
|
108
|
-
self.asari_fields.each do |field|
|
109
|
-
data[field] = obj.send(field)
|
110
|
-
end
|
104
|
+
data = self.asari_data_item(obj)
|
111
105
|
self.asari_instance.update_item(obj.send(:id), data)
|
112
106
|
rescue Asari::DocumentUpdateException => e
|
113
107
|
self.asari_on_error(e)
|
114
108
|
end
|
115
109
|
|
110
|
+
# Gather all the data to send to the CloudSearch
|
111
|
+
# Can be overriden by the model to adapt to special cases.
|
112
|
+
# Returns a hash of the data to send to the CloudSearch
|
113
|
+
def asari_data_item(obj)
|
114
|
+
data = {}
|
115
|
+
self.asari_fields.each do |field|
|
116
|
+
data[field] = obj.send(field) || ""
|
117
|
+
end
|
118
|
+
data
|
119
|
+
end
|
120
|
+
|
116
121
|
# Internal: method for removing a soon-to-be deleted item from the CloudSearch
|
117
122
|
# index. Should probably only be called from asari_remove_from_index above.
|
118
123
|
def asari_remove_item(obj)
|
data/lib/asari/version.rb
CHANGED
data/spec/documents_spec.rb
CHANGED
@@ -29,9 +29,10 @@ describe Asari do
|
|
29
29
|
end
|
30
30
|
|
31
31
|
it "converts Time, DateTime, and Date fields to timestamp integers for rankability" do
|
32
|
-
|
32
|
+
date = Date.new(2012, 4, 1)
|
33
|
+
HTTParty.should_receive(:post).with("http://doc-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/documents/batch", { :body => [{ "type" => "add", "id" => "1", "version" => 1, "lang" => "en", "fields" => { :time => 1333263600, :datetime => 1333238400, :date => date.to_time.to_i }}].to_json, :headers => { "Content-Type" => "application/json"}})
|
33
34
|
|
34
|
-
expect(@asari.add_item("1", {:time => Time.at(1333263600), :datetime => DateTime.new(2012, 4, 1), :date =>
|
35
|
+
expect(@asari.add_item("1", {:time => Time.at(1333263600), :datetime => DateTime.new(2012, 4, 1), :date => date})).to eq(nil)
|
35
36
|
end
|
36
37
|
|
37
38
|
it "allows you to update an item to the index." do
|
@@ -41,9 +42,10 @@ describe Asari do
|
|
41
42
|
end
|
42
43
|
|
43
44
|
it "converts Time, DateTime, and Date fields to timestamp integers for rankability on update as well" do
|
44
|
-
|
45
|
+
date = Date.new(2012, 4, 1)
|
46
|
+
HTTParty.should_receive(:post).with("http://doc-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/documents/batch", { :body => [{ "type" => "add", "id" => "1", "version" => 1, "lang" => "en", "fields" => { :time => 1333263600, :datetime => 1333238400, :date => date.to_time.to_i }}].to_json, :headers => { "Content-Type" => "application/json"}})
|
45
47
|
|
46
|
-
expect(@asari.update_item("1", {:time => Time.at(1333263600), :datetime => DateTime.new(2012, 4, 1), :date =>
|
48
|
+
expect(@asari.update_item("1", {:time => Time.at(1333263600), :datetime => DateTime.new(2012, 4, 1), :date => date})).to eq(nil)
|
47
49
|
end
|
48
50
|
|
49
51
|
it "allows you to delete an item from the index." do
|
data/spec/search_spec.rb
CHANGED
@@ -112,6 +112,11 @@ describe Asari do
|
|
112
112
|
end
|
113
113
|
|
114
114
|
describe "boolean searching" do
|
115
|
+
it "supports a string as the filters value" do
|
116
|
+
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+foo%3A%27bar%27+baz%3A%27bug%27%29&size=10")
|
117
|
+
@asari.search(filter: "(and foo:'bar' baz:'bug')")
|
118
|
+
end
|
119
|
+
|
115
120
|
it "builds a query string from a passed hash" do
|
116
121
|
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+foo%3A%27bar%27+baz%3A%27bug%27%29&size=10")
|
117
122
|
@asari.search(filter: { and: { foo: "bar", baz: "bug" }})
|
@@ -129,6 +134,21 @@ describe Asari do
|
|
129
134
|
})
|
130
135
|
end
|
131
136
|
|
137
|
+
it "supports ruby native numeric ranges" do
|
138
|
+
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+bottles%3A1..99+on%3A%27wall%27%29&size=10")
|
139
|
+
@asari.search(filter: { and: { bottles: (1..99), on: "wall"} })
|
140
|
+
end
|
141
|
+
|
142
|
+
it "it supports numeric ranges where the lower bound is open-ended" do
|
143
|
+
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+bottles%3A..99+on%3A%27wall%27%29&size=10")
|
144
|
+
@asari.search(filter: { and: { bottles: '..99', on: "wall"} })
|
145
|
+
end
|
146
|
+
|
147
|
+
it "it supports numeric ranges where the upper bound is open-ended" do
|
148
|
+
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+bottles%3A1..+on%3A%27wall%27%29&size=10")
|
149
|
+
@asari.search(filter: { and: { bottles: '1..', on: "wall"} })
|
150
|
+
end
|
151
|
+
|
132
152
|
it "fails gracefully with empty params" do
|
133
153
|
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28or+is_donut%3A%27true%27%29&size=10")
|
134
154
|
@asari.search(filter: { or: { is_donut: true, and:
|
@@ -146,7 +166,7 @@ describe Asari do
|
|
146
166
|
|
147
167
|
describe "geography searching" do
|
148
168
|
it "builds a proper query string" do
|
149
|
-
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+lat%3A2505771415..2506771417+lng%
|
169
|
+
HTTParty.should_receive(:get).with("http://search-testdomain.us-east-1.cloudsearch.amazonaws.com/2011-02-01/search?q=&bq=%28and+lat%3A2505771415..2506771417+lng%3A2358260777..2359261578%29&size=10")
|
150
170
|
@asari.search filter: { and: Asari::Geography.coordinate_box(meters: 5000, lat: 45.52, lng: 122.6819) }
|
151
171
|
end
|
152
172
|
end
|
data/spec_helper.rb
CHANGED
metadata
CHANGED
@@ -1,41 +1,69 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: asari
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.11.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tommy Morgan
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-06-10 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: httparty
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- -
|
17
|
+
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '0'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- -
|
24
|
+
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '0'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: rspec
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
|
-
- -
|
31
|
+
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
33
|
version: '0'
|
34
34
|
type: :development
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
|
-
- -
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: simplecov
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: rake
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - ">="
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ">="
|
39
67
|
- !ruby/object:Gem::Version
|
40
68
|
version: '0'
|
41
69
|
description: Asari s a Ruby interface for AWS CloudSearch
|
@@ -45,7 +73,8 @@ executables: []
|
|
45
73
|
extensions: []
|
46
74
|
extra_rdoc_files: []
|
47
75
|
files:
|
48
|
-
- .gitignore
|
76
|
+
- ".gitignore"
|
77
|
+
- ".travis.yml"
|
49
78
|
- Gemfile
|
50
79
|
- README.md
|
51
80
|
- Rakefile
|
@@ -74,25 +103,18 @@ require_paths:
|
|
74
103
|
- lib
|
75
104
|
required_ruby_version: !ruby/object:Gem::Requirement
|
76
105
|
requirements:
|
77
|
-
- -
|
106
|
+
- - ">="
|
78
107
|
- !ruby/object:Gem::Version
|
79
108
|
version: '0'
|
80
109
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
81
110
|
requirements:
|
82
|
-
- -
|
111
|
+
- - ">="
|
83
112
|
- !ruby/object:Gem::Version
|
84
113
|
version: '0'
|
85
114
|
requirements: []
|
86
115
|
rubyforge_project: asari
|
87
|
-
rubygems_version: 2.
|
116
|
+
rubygems_version: 2.2.2
|
88
117
|
signing_key:
|
89
118
|
specification_version: 4
|
90
119
|
summary: Asari is a Ruby interface for AWS CloudSearch.
|
91
|
-
test_files:
|
92
|
-
- spec/active_record_spec.rb
|
93
|
-
- spec/asari_spec.rb
|
94
|
-
- spec/collection_spec.rb
|
95
|
-
- spec/conditionals_spec.rb
|
96
|
-
- spec/documents_spec.rb
|
97
|
-
- spec/geography_spec.rb
|
98
|
-
- spec/search_spec.rb
|
120
|
+
test_files: []
|