riagent 0.0.2 → 0.0.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 189f3608c683deba5f7d5396006eca9567270aef
4
- data.tar.gz: 92e10aef1130cbf20a18f5796659f1ea7e6ced30
3
+ metadata.gz: 27288ed232ba0816f1d3801edbb0d254f9afddf3
4
+ data.tar.gz: 734ec9dcfb6fc9816660d911e0cfd93a348b3e0a
5
5
  SHA512:
6
- metadata.gz: c6246e9dfff27679cd03f175fd0500fe67f8ad7b410eff03a3b421dccb5b589cf5554464ada2dffa5e3f8828657c1bc36ace4bc431c7a91360325672865fb7d8
7
- data.tar.gz: a028746246a003269eaf376248b64713415e25d1279a1f6c1ed2dfc6ca6a4d6dd2b52c6ec55a53e382d98540b747c30aaa0c296b21bf5fdcd0580633b5733c98
6
+ metadata.gz: dcc0206d55b66b3db10d0ed0fd1131232fd5979bbf61a60d177850198d30a20a8d6b3a658d7c72790bf054cf470e6aeb6d2b57c34d7abbb2d31d1d66fc56ce8e
7
+ data.tar.gz: e834529f89a4755b803b926055565e1345f8a57c4529973e5e7acef529825e321138662ec65c1f3e64ddc4d76d8c02fd42757e744016d78a771a984037fe8fd8
data/Gemfile.lock CHANGED
@@ -5,16 +5,16 @@ PATH
5
5
  activemodel (~> 4.0)
6
6
  activesupport (~> 4.0)
7
7
  riagent-document
8
- riak-client
8
+ riak-client (~> 2.0.0.rc1)
9
9
  riak_json
10
10
 
11
11
  GEM
12
12
  remote: https://rubygems.org/
13
13
  specs:
14
- activemodel (4.0.5)
15
- activesupport (= 4.0.5)
14
+ activemodel (4.0.9)
15
+ activesupport (= 4.0.9)
16
16
  builder (~> 3.1.0)
17
- activesupport (4.0.5)
17
+ activesupport (4.0.9)
18
18
  i18n (~> 0.6, >= 0.6.9)
19
19
  minitest (~> 4.2)
20
20
  multi_json (~> 1.3)
@@ -24,50 +24,55 @@ GEM
24
24
  descendants_tracker (~> 0.0.4)
25
25
  ice_nine (~> 0.11.0)
26
26
  thread_safe (~> 0.3, >= 0.3.1)
27
- beefcake (0.3.7)
27
+ beefcake (1.0.0.pre2)
28
28
  builder (3.1.4)
29
29
  coercible (1.0.0)
30
30
  descendants_tracker (~> 0.0.1)
31
31
  descendants_tracker (0.0.4)
32
32
  thread_safe (~> 0.3, >= 0.3.1)
33
33
  equalizer (0.0.9)
34
- i18n (0.6.9)
34
+ i18n (0.6.11)
35
35
  ice_nine (0.11.0)
36
36
  innertube (1.0.2)
37
37
  json (1.8.1)
38
38
  mime-types (2.3)
39
- minitest (4.7.5)
39
+ minitest (4.7.0)
40
40
  minitest-spec-context (0.0.3)
41
41
  multi_json (1.10.1)
42
+ netrc (0.7.7)
43
+ r509 (0.10.0)
44
+ r509-cert-validator (0.0.4)
45
+ r509 (~> 0.10.0)
42
46
  rake (10.3.2)
43
- rest-client (1.6.7)
44
- mime-types (>= 1.16)
47
+ rest-client (1.7.2)
48
+ mime-types (>= 1.16, < 3.0)
49
+ netrc (~> 0.7)
45
50
  riagent-document (0.0.1)
46
51
  activesupport (~> 4.0)
47
52
  virtus
48
- riak-client (1.4.4.1)
49
- beefcake (~> 0.3.7)
50
- builder (>= 2.1.2)
53
+ riak-client (2.0.0.rc1)
54
+ beefcake (>= 1.0.0.pre1)
51
55
  i18n (>= 0.4.0)
52
56
  innertube (~> 1.0.2)
53
57
  multi_json (~> 1.0)
58
+ r509-cert-validator (~> 0.0.4)
54
59
  riak_json (0.0.4)
55
60
  json
56
61
  rest-client
57
62
  thread_safe (0.3.4)
58
- tzinfo (0.3.39)
59
- virtus (1.0.2)
63
+ tzinfo (0.3.41)
64
+ virtus (1.0.3)
60
65
  axiom-types (~> 0.1)
61
66
  coercible (~> 1.0)
62
- descendants_tracker (~> 0.0.3)
63
- equalizer (~> 0.0.9)
67
+ descendants_tracker (~> 0.0, >= 0.0.3)
68
+ equalizer (~> 0.0, >= 0.0.9)
64
69
 
65
70
  PLATFORMS
66
71
  ruby
67
72
 
68
73
  DEPENDENCIES
69
74
  bundler
70
- minitest (~> 4.2)
75
+ minitest (~> 4.7)
71
76
  minitest-spec-context
72
77
  rake
73
78
  riagent!
data/README.md CHANGED
@@ -1,90 +1,111 @@
1
1
  # riagent
2
2
 
3
- Object Document Mapper for RiakJson (Riak + Solr), for use with Ruby on Rails 4 and Sinatra.
3
+ Object Document Mapper for [Riak](http://basho.com/products/riak-kv/)
4
+ (including integration with Solr search), for use with Ruby on Rails 4 and
5
+ Sinatra.
4
6
 
5
7
  ## Requirements
6
8
  - Ruby 1.9+
7
- - Riak build with [Riak Json](https://github.com/basho-labs/riak_json) and Solr/Yokozuna enabled
8
- (see [Setup](https://github.com/basho-labs/riak_json#setup) section for instructions)
9
- - [RiakJson](https://github.com/basho-labs/riak_json_ruby_client) Ruby Client gem installed locally
10
- - [riagent-document](https://github.com/dmitrizagidulin/riagent-document) gem installed locally
9
+ - [Riak](http://basho.com/products/riak-kv/) version 2.0 or later
10
+ - [RiakJson](https://github.com/basho-labs/riak_json_ruby_client) Ruby Client
11
+ gem installed locally
12
+ - [riagent-document](https://github.com/dmitrizagidulin/riagent-document) gem
13
+ installed locally
11
14
 
12
15
  ## Motivation
13
- *a.k.a. Why not just use a plain [riak_json client](https://github.com/basho-labs/riak_json_ruby_client) or a
16
+ *a.k.a. Why not just use a plain
17
+ [riak_json client](https://github.com/basho-labs/riak_json_ruby_client) or a
14
18
  [riak-ruby-client](https://github.com/basho/riak-ruby-client)?*
15
19
 
16
- A RiakJson or Riak client just answers the question "How do I store stuff in Riak?".
17
- In order to develop any non-toy application using Riak as a persistence layer, a developer must answer further questions:
20
+ A Riak client just answers the question "How do I store stuff in Riak?".
21
+ In order to develop any non-toy application using Riak as a persistence layer,
22
+ a developer must answer further questions:
18
23
 
19
- - How do I define my model objects, and how do I serialize them so I can store them in Riak?
20
- - What will my access patterns be? Plain Key/Value reads and writes, or will I need to perform queries?
21
- - How do I integrate painlessly with my framework of choice? (Ruby on Rails, Sinatra)
24
+ - How do I define my model objects, and how do I serialize them so I can store
25
+ them in Riak?
26
+ - What will my access patterns be? Plain Key/Value reads and writes, or will I
27
+ need to perform queries?
28
+ - How do I integrate painlessly with my framework of choice? (Ruby on Rails,
29
+ Sinatra)
22
30
 
23
- Riagent attempts to provide answers to those questions, to encode recommended best-practice Riak query patterns,
24
- and in general to aid rapid application development by working with Riak's strenghts while respecting its limitations.
25
- It is intended as a spiritual successor to [Ripple](https://github.com/basho-labs/ripple).
31
+ Riagent attempts to provide answers to those questions, to encode recommended
32
+ best-practice Riak query patterns, and in general to aid rapid application
33
+ development by working with Riak's strenghts while respecting its limitations.
34
+ It is intended as a spiritual successor to
35
+ [Ripple](https://github.com/basho-labs/ripple).
26
36
 
27
37
  #### Model Definition and Serialization
28
- Riagent provides a model definition language that will be familiar to most Rails developers, via
29
- the [riagent-document](https://github.com/dmitrizagidulin/riagent-document) gem.
38
+ Riagent provides a model definition language that will be familiar to most Rails
39
+ developers, via the
40
+ [riagent-document](https://github.com/dmitrizagidulin/riagent-document) gem.
30
41
 
31
42
  ```ruby
32
43
  include 'riagent-document'
33
44
 
34
45
  class User
35
46
  include Riagent::ActiveDocument
36
-
47
+
37
48
  attribute :username, String
38
49
  attribute :email, String
39
50
  attribute :country, String, default: 'USA'
40
51
  end
41
52
  ```
42
53
 
43
- Riagent::ActiveDocument instances provide a rich set of functionality including attribute defaults, type coercions,
44
- conversion to and from JSON, search schema creation, easy persistence to Riak,
45
- document embedding, and more. See the [riagent-document README](https://github.com/dmitrizagidulin/riagent-document)
54
+ Riagent::ActiveDocument instances provide a rich set of functionality including
55
+ attribute defaults, type coercions, conversion to and from JSON, search schema
56
+ creation, easy persistence to Riak, document embedding, and more. See the
57
+ [riagent-document README](https://github.com/dmitrizagidulin/riagent-document)
46
58
  for more details.
47
59
 
48
60
  #### Advanced Riak K/V and Query Support
49
- Reads and writes of single objects into Riak are easy (and have the benefit of being massively scalable, highly concurrent, and fault-tolerant).
50
- But what about listing and querying? Every developer that gets past a Hello World get/put tutorial on Riak is soon faced with questions
51
- about more advanced access patterns. How do I implement collections, and list things? How do I search or query on various attributes?
52
- Should I use Secondary Indexes? What about Search/Solr integration?
53
-
54
- Riagent provides a set of high-level notations and functionality that allows developers create collections and associations on Riak,
55
- either via plain K/V operations when possible, or via advanced mechanisms such as Solr/[RiakJson](https://github.com/basho-labs/riak_json)
56
- queries when necessary.
61
+ Reads and writes of single objects into Riak are easy (and have the benefit of
62
+ being massively scalable, highly concurrent, and fault-tolerant). But what about
63
+ listing and querying? Every developer that gets past a Hello World get/put
64
+ tutorial on Riak is soon faced with questions about more advanced access
65
+ patterns. How do I implement collections, and list things? How do I search or
66
+ query on various attributes? Should I use Secondary Indexes? What about
67
+ Search/Solr integration?
68
+
69
+ Riagent provides a set of high-level notations and functionality that allows
70
+ developers create collections and associations on Riak, either via plain K/V
71
+ operations when possible, or via advanced mechanisms such as
72
+ Solr/[RiakJson](https://github.com/basho-labs/riak_json) queries when necessary.
57
73
 
58
74
  ```ruby
59
75
  class User
60
76
  include Riagent::ActiveDocument
61
-
77
+
62
78
  collection_type :riak_json # Persist to a RiakJson::Collection
63
-
79
+
64
80
  attribute :username, String, search_index: { as: :text }
65
81
  attribute :email, String, search_index: { as: :string }
66
82
  attribute :language, String, default: 'en'
67
-
83
+
68
84
  # Associations
69
85
  has_one :email_preference, :class => EmailPreference
70
86
  has_many :posts, :class => BlogPost, :using => :solr
71
-
87
+
72
88
  # Validations
73
89
  validates_presence_of :username
74
90
  end
75
91
  ```
76
92
 
77
93
  #### Rails and Sinatra integration
78
- Riagent and ActiveDocuments are integrated into the usual Rails workflow.
94
+ Riagent and ActiveDocuments are integrated into the usual Rails workflow.
79
95
 
80
- - Riagent provides client configuration and initialization, via the ```config/riak.yml``` file
81
- - Provides a simple Key/Value persistence layer with ```save()```, ```find()```, ```update()``` and ```destroy()``` methods.
96
+ - Riagent provides client configuration and initialization, via the
97
+ ```config/riak.yml``` file
98
+ - Provides a simple Key/Value persistence layer with ```save()```,
99
+ ```find()```, ```update()``` and ```destroy()``` methods.
82
100
  - ActiveDocument implements the ActiveModel API.
83
- (Specifically, passes the [ActiveModel Lint Test suite](http://api.rubyonrails.org/classes/ActiveModel/Lint/Tests.html))
101
+ (Specifically, passes the [ActiveModel Lint Test
102
+ suite](http://api.rubyonrails.org/classes/ActiveModel/Lint/Tests.html))
84
103
  - Provides a full range of validations for each attribute.
85
104
  - Provides ```before_save``` / ```after_save``` type Callback functionality
86
- - Provides a custom Query capability (to Riak/Solr), for searches, range queries, aggregations and more
87
- - Derives RiakJson/Solr search schemas from annotated document attributes (see Schemas below)
105
+ - Provides a custom Query capability (to Riak/Solr), for searches, range
106
+ queries, aggregations and more
107
+ - Derives RiakJson/Solr search schemas from annotated document attributes (see
108
+ Schemas below)
88
109
 
89
110
  ## Usage
90
111
  ### Adding Riagent to a Rails App
@@ -99,11 +120,13 @@ Run the install generator:
99
120
  rails generate riagent:install
100
121
  ```
101
122
 
102
- This creates a ```config/riak.yml``` file. Edit it to point to your running Riak instance.
123
+ This creates a ```config/riak.yml``` file. Edit it to point to your running Riak
124
+ instance.
103
125
 
104
126
  ### Controller and View helpers
105
- Since they implement the ActiveModel API, when you use ActiveDocuments in
127
+ Since they implement the ActiveModel API, when you use ActiveDocuments in
106
128
  a Rails model, the usual ```link_to```/route-based helpers work:
129
+
107
130
  ```erb
108
131
  # In a user view file
109
132
  <%= link_to @user.username, @user %> # => <a href="/users/EmuVX4kFHxxvlUVJj5TmPGgGPjP">HieronymusBosch</a>
@@ -131,12 +154,13 @@ new_user.key # => 'EmuVX4kFHxxvlUVJj5TmPGgGPjP'
131
154
  ```
132
155
 
133
156
  ### Key/Value Persistence
134
- The usual array of CRUD type k/v operations is available to an ActiveDocument model.
157
+ The usual array of CRUD type k/v operations is available to an ActiveDocument
158
+ model.
135
159
 
136
160
  Create documents via ```save()``` and ```save!()```
137
161
  ```ruby
138
162
  user = User.new({username: 'John', email: 'john@doe.com'})
139
- # If you save without specifying a key, RiakJson generates a UUID type key automatically
163
+ # If you save without specifying a key, it generates a UUID key automatically
140
164
  user.save # => 'EmuVX4kFHxxvlUVJj5TmPGgGPjP'
141
165
  ```
142
166
 
@@ -154,7 +178,7 @@ user.destroy # deletes the document
154
178
  ```
155
179
 
156
180
  ### Callbacks
157
- ActiveDocument currently supports ```before_*``` and ```after_*``` [callbacks](http://api.rubyonrails.org/classes/ActiveSupport/Callbacks.html)
181
+ ActiveDocument currently supports ```before_*``` and ```after_*``` [callbacks](http://api.rubyonrails.org/classes/ActiveSupport/Callbacks.html)
158
182
  for the following events:
159
183
  ```[:create, :update, :save, :destroy]```
160
184
 
@@ -169,23 +193,28 @@ user = User.find_one({ username: 'HieronymusBosch' })
169
193
  ```
170
194
 
171
195
  ## Search Schema Definition
172
- RiakJson uses Solr/Yokozuna to provide indexing and search capability for its collections.
173
- If you do not specify a collection schema explicitly, RiakJson creates one when you insert the first document to that collection
174
- (it [infers the schema](https://github.com/basho-labs/riak_json/blob/master/docs/architecture.md#inferred-schemas)
175
- based on the basic data types of the field values in the JSON).
176
- However, if you do not want to use this default schema behavior (for example, because RJ tries to index all of the fields),
177
- you can define and set a collection schema yourself, using RJ Ruby Client's [schema administration](https://github.com/basho-labs/riak_json_ruby_client#schema-administration)
196
+ RiakJson uses Solr/Yokozuna to provide indexing and search capability for its
197
+ collections. If you do not specify a collection schema explicitly, RiakJson
198
+ creates one when you insert the first document to that collection (it [infers
199
+ the
200
+ schema](https://github.com/basho-labs/riak_json/blob/master/docs/architecture.md#inferred-schemas)
201
+ based on the basic data types of the field values in the JSON). However, if you
202
+ do not want to use this default schema behavior (for example, because RJ tries
203
+ to index all of the fields), you can define and set a collection schema
204
+ yourself, using RJ Ruby Client's [schema
205
+ administration](https://github.com/basho-labs/riak_json_ruby_client#schema-administration)
178
206
  API.
179
207
 
180
- To make the process of schema definition even easier for developers, ActiveDocument provides the ```search_index``` attribute
181
- option. This annotation allows you to specify which document fields you want added to your search schema, as well as the
182
- Solr field type that will be used to index it.
208
+ To make the process of schema definition even easier for developers,
209
+ ActiveDocument provides the ```search_index``` attribute option. This annotation
210
+ allows you to specify which document fields you want added to your search
211
+ schema, as well as the Solr field type that will be used to index it.
183
212
 
184
213
  For example, the following model:
185
214
  ```ruby
186
215
  class User
187
216
  include Riagent::ActiveDocument
188
-
217
+
189
218
  attribute :username, String, required: true, search_index: { :as => :text }
190
219
  attribute :email, String, search_index: { :as => :string }
191
220
  attribute :country, String, default: 'USA'
@@ -208,23 +237,32 @@ User.schema # =>
208
237
  ```
209
238
 
210
239
  ### Schema Administration
211
- Note that if you use the ```search_index``` attribute annotations above, you will have to explicitly
212
- notify RiakJson of your intent to save the schema. You will have to call the ```set_schema()``` collection
213
- method before you start inserting documents (for example, in a ```db:setup``` Rake task).
240
+ Note that if you use the ```search_index``` attribute annotations above, you
241
+ will have to explicitly notify RiakJson of your intent to save the schema. You
242
+ will have to call the ```set_schema()``` collection method before you start
243
+ inserting documents (for example, in a ```db:setup``` Rake task).
214
244
 
215
245
  ```ruby
216
246
  User.collection.set_schema(User.schema)
217
247
  ```
218
248
 
219
249
  ## Testing the Riagent gem
220
- First, set up the Riak config file for (and make sure it's pointing to a running Riak instance)
250
+ First, set up the Riak config file for (and make sure it's pointing to a running
251
+ Riak instance)
221
252
 
222
253
  ```
223
254
  cp test/config/riak.yml.example test/config/riak.yml
224
255
  ```
225
256
 
257
+ The integration tests require that a Set bucket type be created, named `sets`:
258
+
259
+ ```
260
+ riak-admin bucket-type create sets '{"props":{"datatype":"set"}}'
261
+ riak-admin bucket-type activate sets
262
+ ```
263
+
226
264
  To run the tests:
227
265
 
228
266
  ```
229
267
  bundle exec rake test
230
- ```
268
+ ```
@@ -55,7 +55,8 @@ module Riagent
55
55
 
56
56
  # @param [Hash] env_config Configuration hash for a given environment
57
57
  def init_riak_client(env_config)
58
- client = Riak::Client.new host: env_config['host'], pb_port: env_config['pb_port'], protocol: 'pbc'
58
+ # client = Riak::Client.new host: env_config['host'], pb_port: env_config['pb_port'], protocol: 'pbc'
59
+ client = Riak::Client.new host: env_config['host'], pb_port: env_config['pb_port']
59
60
  self.riak_client = client
60
61
  end
61
62
 
@@ -78,7 +79,11 @@ module Riagent
78
79
 
79
80
  # @return [Riak::Client] The Riak client for the current thread.
80
81
  def riak_client
81
- Thread.current[:riak_client] ||= nil
82
+ unless Thread.current[:riak_client]
83
+ # Re-initialize client
84
+ self.init_riak_client(self.config_for(Rails.env))
85
+ end
86
+ Thread.current[:riak_client]
82
87
  end
83
88
 
84
89
  # Sets the Riak client for the current thread.
@@ -89,7 +94,11 @@ module Riagent
89
94
 
90
95
  # @return [RiakJson::Client] The RiakJson client for the current thread.
91
96
  def riak_json_client
92
- Thread.current[:riak_json_client] ||= nil
97
+ unless Thread.current[:riak_json_client]
98
+ # Re-initialize client
99
+ self.init_riak_json_client(self.config_for(Rails.env))
100
+ end
101
+ Thread.current[:riak_json_client]
93
102
  end
94
103
 
95
104
  # Sets the RiakJson client for the current thread.
@@ -20,6 +20,7 @@
20
20
 
21
21
  require "active_support/concern"
22
22
  require "riagent/persistence/persistence_strategy"
23
+ require "riagent/persistence/riak_dt_set_strategy"
23
24
  require "riagent/persistence/riak_json_strategy"
24
25
  require "riagent/persistence/riak_no_index_strategy"
25
26
 
@@ -30,7 +31,11 @@ module Riagent
30
31
  module Persistence
31
32
  extend ActiveSupport::Concern
32
33
 
33
- COLLECTION_TYPES = [:riak_json, :riak_no_index]
34
+ COLLECTION_TYPES = [:riak_json, :riak_kv]
35
+
36
+ # Key Listing strategies for +:riak_kv+ collections
37
+ # (the :riak_json collection type uses an implied solr query strategy)
38
+ VALID_KEY_LISTS = [:streaming_list_keys, :riak_dt_set]
34
39
 
35
40
  included do
36
41
  extend ActiveModel::Callbacks
@@ -111,7 +116,7 @@ module Riagent
111
116
  # collection_type :riak_json # persist to a RiakJson::Collection
112
117
  # end
113
118
  # </code>
114
- def collection_type(coll_type)
119
+ def collection_type(coll_type, options={})
115
120
  unless COLLECTION_TYPES.include? coll_type
116
121
  raise ArgumentError, "Invalid collection type: #{coll_type.to_s}"
117
122
  end
@@ -119,8 +124,15 @@ module Riagent
119
124
  case @collection_type
120
125
  when :riak_json
121
126
  self.persistence = Riagent::Persistence::RiakJsonStrategy.new(self)
122
- when :riak_no_index
123
- self.persistence = Riagent::Persistence::RiakNoIndexStrategy.new(self)
127
+ when :riak_kv
128
+ self.persistence = Riagent::Persistence::RiakKVStrategy.new(self)
129
+ if options.has_key? :list_keys_using
130
+ if options[:list_keys_using] == :streaming_list_keys
131
+ self.persistence = Riagent::Persistence::RiakNoIndexStrategy.new(self)
132
+ elsif options[:list_keys_using] == :riak_dt_set
133
+ self.persistence = Riagent::Persistence::RiakDTSetStrategy.new(self)
134
+ end
135
+ end
124
136
  end
125
137
  end
126
138