synchronisable 1.1.5 → 1.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: c70006cc02972431afa0d306a1015738489b60c0
4
- data.tar.gz: f12f1145a0f3c594e66e689df0e85b333aff056b
3
+ metadata.gz: 70388e8c1eef4b5cb5e12223a4201d02073248ad
4
+ data.tar.gz: 829ab2f2ebf18374eaaab7e34b97ca084c494224
5
5
  SHA512:
6
- metadata.gz: 2f67edbe1fb55314de9f84ef6db56037865fb0a387c67538a35ce14191d9d5e3a59439b35a49587caed9107943b927a027524a6af39260ef87b99cdae49b4760
7
- data.tar.gz: 44a3768ad6af8890d1fcc0a2e77cab978196686e8c204b38f5b96cb3a2df4e85a865e64be47d47a5094d9e05043f06a3eb07301f8ff1d4ea116fccd5fcbab153
6
+ metadata.gz: 3c4b925a4fb1a39e8c7d440fea7803c2303f89b7fad7b951564aa800b18b21d48934691f686e9e112102e005df959792f67d08ba4529687b7e4a84977905e74e
7
+ data.tar.gz: 7f69d985173f10d17fb3e84474c7b1dc4c0d0d8e8f85eadf9d7cf42a6364aa3072606a0a275a09060a8ec28268998640664019ba2de87f9bae199a239cc58e14
data/README.md CHANGED
@@ -7,11 +7,21 @@
7
7
 
8
8
  # Synchronisable
9
9
 
10
- Provides base fuctionality (models, DSL) for AR synchronization
11
- with external resources (apis, services etc).
10
+ ### :construction: this README & docs are work in progress :construction:
12
11
 
13
12
  ## Overview
14
13
 
14
+ Provides base fuctionality for active record models synchronization
15
+ with external resources. The remote source could be anything you like:
16
+ apis, services, site that you gonna parse and steal some data from it.
17
+
18
+ ## Resources
19
+
20
+ * [Rubygems](https://rubygems.org/gems/synchronisable)
21
+ * [API](http://rdoc.info/github/vyorkin/synchronisable/master/frames)
22
+ * [Bugs](https://github.com/vyorkin/synchronisable/issues)
23
+ * [Development](https://travis-ci.org/vyorkin/synchronisable)
24
+
15
25
  ## Installation
16
26
 
17
27
  Add this line to your application's Gemfile:
@@ -26,58 +36,195 @@ Or install it yourself as:
26
36
 
27
37
  $ gem install synchronisable
28
38
 
39
+ Optionally, if you are using rails to run an initializer generator:
40
+
41
+ $ rails g synchronisable:install
42
+
43
+ ## Rationale
44
+
45
+ Sometimes we need to sync our domain models (or some part of them)
46
+ with some kind of remote source. Its great if you can consume a well-done RESTful api
47
+ that is pretty close to you local domain models.
48
+ But unfortunately the remote data source could be just anything.
49
+
50
+ Actually this gem was made to consume data coming from a site parser :crying_cat_face:
51
+
52
+ Examples of the usage patterns are shown below.
53
+ You can find more by looking at the [dummy app](https://github.com/vyorkin/synchronisable/tree/master/spec/dummy/app)
54
+ [models](https://github.com/vyorkin/synchronisable/tree/master/spec/dummy/app/models) and
55
+ [synchronizers](https://github.com/vyorkin/synchronisable/tree/master/spec/dummy/app/synchronizers).
56
+
57
+ ## Features
58
+
59
+ * Attribute mapping, `unique_id`
60
+ * Associations sync + `:includes` option to specify (restrict)
61
+ an association tree to be synchronized
62
+ * `before` and `after` callbacks to hook into sync process
63
+ * ???
64
+
65
+ ## Configuration
66
+
67
+ For rails users there is a well-documented initializer.
68
+ Just run `rails g synchronisable:install` and you'll be fine.
69
+
70
+ Non-rails users can do so by using provided
71
+ `ActiveSupport::Configurable` interface. So here is the default settings:
72
+
73
+ ```ruby
74
+ Synchronisable.configure do |config|
75
+ # Logging configuration
76
+ #
77
+ # Default logger fallbacks to `Rails.logger` if available, otherwise
78
+ # `STDOUT` will be used for output.
79
+ #
80
+ config.logging = {
81
+ :logger => defined?(Rails) ? Rails.logger : Logger.new(STDOUT)
82
+ :verbose => true,
83
+ :colorize => true
84
+ }
85
+
86
+ # If you want to restrict synchronized models.
87
+ # By default it will try to sync all models that have
88
+ # a `synchronisable` dsl instruction.
89
+ #
90
+ config.models = %w(Foo Bar)
91
+ end
92
+ ```
93
+
29
94
  ## Usage
30
95
 
31
- For examples we'll be using a well-known domain with posts & comments
96
+ Imagine a situation when you have to periodically get data from
97
+ some remote source and store it locally.
98
+ Basically the task is to create local records if they don't exist
99
+ and update their attributes otherwise.
100
+
101
+ ### Gateways
102
+
103
+ Thing that provides an access to an external system or resource
104
+ is called [gateway](http://martinfowler.com/eaaCatalog/gateway.html).
105
+ You can take a look at the base [gateway](https://github.com/vyorkin/synchronisable/blob/master/lib/synchronisable/gateway.rb)
106
+ class to get a clue what does it mean in terms of this gem
107
+ (btw fetching data from a remote source is not a purpose of this gem).
108
+
109
+ The main idea is that gateway implementation class has only 2 methods:
110
+
111
+ * `fetch(params = {})` – returns an array of hashes, each hash contains
112
+ an attributes that should be (somehow) mapped over your target model.
113
+ * `find(params)` – returns a single hash with remote attributes.
114
+ `params` here is only to have a choice between representing a single
115
+ or a composite identity.
116
+
117
+ ### Models and synchronizers
118
+
119
+ The first step is to declare that your active record model is synchronizable.
120
+ You can do so by using corresponding `synchronisable` dsl instruction,
121
+ that optionally takes a synchonizer class to be used.
122
+ You should only specify it when the name can't be figured out
123
+ by the following convention: `ModelSynchronizer`.
124
+ So for example here we have a Tournament that has many Stages:
32
125
 
33
126
  ```ruby
34
- class Post < ActiveRecord::Base
35
- has_many :comments
127
+ class Tournament < ActiveRecord::Base
128
+ has_many :stages
36
129
 
37
130
  synchronisable
38
131
  end
39
132
 
40
- class Comment < ActiveRecord::Base
41
- belongs_to :post
133
+ class Stage < ActiveRecord::Base
134
+ belongs_to :tournament
42
135
 
43
- synchronisable MyCommentSynchronizer
136
+ synchronisable
44
137
  end
45
138
  ```
46
139
 
47
- As you can see above the first step is to declare your models to be
48
- synchronisable. You can do so by using corresponding dsl instruction,
49
- that optionally takes a synchonizer class to be used. Actually,
50
- the only reason to specify it its when it has a name, that can't be figured out
51
- by the following convention: `ModelSynchronizer`.
52
-
53
- After that you should define your model synchronizers
140
+ Lets define synchronizers:
54
141
 
55
142
  ```ruby
56
- class PostSynchronizer < Synchronisable::Synchronizer
57
- remote_id :p_id
143
+ class TournamentSynchronizer < Synchronisable::Synchronizer
144
+ has_many :stages
58
145
 
146
+ remote_id :tour_id
147
+ unique_id { |attrs| attrs[:name] }
148
+
149
+ mappings(
150
+ :eman => :name,
151
+ :eman_trohs => :short_name,
152
+ :gninnigeb => :beginning,
153
+ :gnidge => :ending,
154
+ :tnerruc_si => :is_current
155
+ )
156
+
157
+ only :name, :beginning, :ending
158
+
159
+ gateway TournamentGateway
160
+ end
161
+
162
+ class StageSynchronizer < Synchronisable::Synchronizer
163
+ has_many :matches
164
+
165
+ remote_id :stage_id
166
+
167
+ mappings(
168
+ :tour_id => :tournament_id,
169
+ :gninnigeb => :beginning,
170
+ :gnidge => :ending,
171
+ :eman => :name,
172
+ :rebmun => :number
173
+ )
174
+
175
+ except :ignored_1, :ignored_2
176
+
177
+ gateway StageGateway
178
+
179
+ before_sync do |source|
180
+ source.local_attrs[:name] != 'ignored'
181
+ end
182
+ end
183
+ ```
184
+
185
+ ### Gateways vs `fetch` & `find` in synchronizers
186
+
187
+ TDOO: Blah blah blah... Need to describe the difference & use cases.
188
+
189
+ ### Blah blah
190
+
191
+ ### Blah blah
192
+
193
+ ```ruby
194
+ class TournamentSynchronizer < Synchronisable::Synchronizer
59
195
  mappings(
60
196
  :t => :title,
61
197
  :c => :content
62
198
  )
63
199
 
200
+ remote_id :p_id
201
+
202
+ # Local attributes to ignore.
203
+ # These will not be set on your local record.
64
204
  except :ignored_attr1, :ignored_attr42
65
205
 
206
+ # Declares that we want to sync comments after syncing this model.
207
+ # The resulting hash with remote attributes should contain `comment_ids`
66
208
  has_many :comments
67
209
 
210
+ # Method that will be used to fetch all of the remote entities
68
211
  fetch do
69
- # return array of hashes with
70
- # remote entity attributes
212
+ # Somehow get and return an array of hashes with remote entity attibutes
213
+ [
214
+ { t: 'first', c: 'i am the first post' },
215
+ { t: 'second', c: 'content of the second post' }
216
+ ]
71
217
  end
72
218
 
219
+ # This method should return only one hash for the given id
73
220
  find do |id|
74
- # return a hash with
75
- # with remote entity attributes
221
+ # return a hash with with remote entity attributes
222
+ # ...
76
223
  end
77
224
 
78
- # Hooks/callbacks
79
-
225
+ #
80
226
  before_record_sync do |source|
227
+ # return false if you want to skip syncing of this particular record
81
228
  # ...
82
229
  end
83
230
 
@@ -95,6 +242,7 @@ class PostSynchronizer < Synchronisable::Synchronizer
95
242
 
96
243
  before_sync do |source|
97
244
  # ...
245
+ # return false if you want to skip syncing of this particular record
98
246
  end
99
247
 
100
248
  after_sync do |source|
@@ -102,6 +250,8 @@ class PostSynchronizer < Synchronisable::Synchronizer
102
250
  end
103
251
  end
104
252
 
253
+
254
+
105
255
  class MyCommentSynchronizer < Synchronisable::Synchronizer
106
256
  remote_id :c_id
107
257
 
@@ -0,0 +1,21 @@
1
+ module Synchronisable
2
+ class Configuration
3
+ include ActiveSupport::Configurable
4
+
5
+ config_accessor :models do
6
+ {}
7
+ end
8
+ config_accessor :logging do
9
+ default_logger = -> { Logger.new(STDOUT) }
10
+ rails_logger = -> { Rails.logger || default_logger.() }
11
+
12
+ logger = defined?(Rails) ? rails_logger.() : default_logger.()
13
+
14
+ {
15
+ :logger => logger,
16
+ :verbose => true,
17
+ :colorize => true
18
+ }
19
+ end
20
+ end
21
+ end
@@ -4,7 +4,7 @@ module Synchronisable
4
4
  not_implemented :fetch
5
5
  end
6
6
 
7
- def find(id)
7
+ def find(params)
8
8
  not_implemented :find
9
9
  end
10
10
 
@@ -10,7 +10,7 @@ module Synchronisable
10
10
 
11
11
  %i(verbose colorize).each do |name|
12
12
  define_method("#{name}_logging?".to_sym) do
13
- Synchronisable.logging[name]
13
+ Synchronisable.config.logging[name]
14
14
  end
15
15
  end
16
16
 
@@ -33,6 +33,12 @@ module Synchronisable
33
33
  class_attribute :synchronizer
34
34
  has_one :import, as: :synchronisable, class_name: 'Synchronisable::Import'
35
35
 
36
+ scope :without_import, -> {
37
+ includes(:import)
38
+ .where(imports: { synchronisable_id: nil })
39
+ .references(:imports)
40
+ }
41
+
36
42
  set_defaults(args)
37
43
  end
38
44
 
@@ -1,5 +1,3 @@
1
- require 'pry-byebug'
2
-
3
1
  module Synchronisable
4
2
  # TODO: Massive refactoring needed
5
3
 
@@ -27,7 +27,7 @@ module Synchronisable
27
27
  source.try(:with_indifferent_access)
28
28
  }
29
29
 
30
- # Attributes that will be ignored.
30
+ # Attributes to ignored.
31
31
  attribute :except, converter: SYMBOL_ARRAY_CONVERTER
32
32
 
33
33
  # The only attributes that will be used.
@@ -39,7 +39,7 @@ module Synchronisable
39
39
 
40
40
  # Logger that will be used during synchronization
41
41
  # of this particular model.
42
- attribute :logger, default: -> { Synchronisable.logging[:logger] }
42
+ attribute :logger, default: -> { Synchronisable.config.logging[:logger] }
43
43
 
44
44
  # Gateway to be used to get the remote data
45
45
  #
@@ -2,7 +2,7 @@ module Synchronisable
2
2
  module VERSION
3
3
  MAJOR = 1
4
4
  MINOR = 1
5
- PATCH = 5
5
+ PATCH = 6
6
6
  SUFFIX = nil
7
7
 
8
8
  STRING = [MAJOR, MINOR, PATCH, SUFFIX].compact.join('.')
@@ -11,44 +11,39 @@ require 'active_support/concern'
11
11
  require 'synchronisable/bootstrap/i18n'
12
12
 
13
13
  require 'synchronisable/version'
14
+ require 'synchronisable/configuration'
14
15
  require 'synchronisable/models/import'
15
16
  require 'synchronisable/synchronizer'
16
17
  require 'synchronisable/model'
17
18
  require 'synchronisable/gateway'
18
19
 
19
-
20
20
  module Synchronisable
21
- include ActiveSupport::Configurable
22
-
23
- config_accessor :models do
24
- {}
21
+ def self.config
22
+ @configuration ||= Configuration.new
25
23
  end
26
- config_accessor :logging do
27
- default_logger = -> { Logger.new(STDOUT) }
28
- rails_logger = -> { Rails.logger || default_logger.() }
29
-
30
- logger = defined?(Rails) ? rails_logger.() : default_logger.()
31
24
 
32
- {
33
- :logger => logger,
34
- :verbose => true,
35
- :colorize => true
36
- }
25
+ def self.configure
26
+ yield config
37
27
  end
38
28
 
39
29
  # Syncs models that are defined in {Synchronisable#models}
40
30
  #
41
- # @param models [Array] array of models that should be synchronized.
42
- # This take a precedence over models defined in {Synchronisable#models}.
43
- # If this parameter is not specified and {Synchronisable#models} is empty,
44
- # than it will try to sync only those models which have a corresponding synchronizers
31
+ # @overload sync(models, options)
32
+ # @param models [Array] array of models that should be synchronized.
33
+ # This take a precedence over models defined in {Synchronisable#models}.
34
+ # If this parameter is not specified and {Synchronisable#models} is empty,
35
+ # than it will try to sync only those models which have a corresponding synchronizers
36
+ # @param options [Hash] options that will be passed to controller
37
+ # @overload sync(models)
38
+ # @overlaod sync(options)
45
39
  #
46
40
  # @return [Array<[Synchronisable::Context]>] array of synchronization contexts
47
41
  #
48
42
  # @see Synchronisable::Context
49
- def self.sync(*models)
50
- source = source_models(models)
51
- source.map(&:sync)
43
+ def self.sync(*args)
44
+ options = args.extract_options!
45
+ source = source_models(args)
46
+ source.map { |model| model.sync(options) }
52
47
  end
53
48
 
54
49
  private
@@ -59,7 +54,7 @@ module Synchronisable
59
54
  end
60
55
 
61
56
  def self.default_models
62
- models.map(&:safe_constantize).compact
57
+ config.models.map(&:safe_constantize).compact
63
58
  end
64
59
 
65
60
  def self.find_models
@@ -9,7 +9,7 @@ describe Synchronisable do
9
9
  describe 'models specified in configuration' do
10
10
  context 'only Team and Match' do
11
11
  before :all do
12
- Synchronisable.models = %w(Match Team)
12
+ Synchronisable.config.models = %w(Match Team)
13
13
  end
14
14
 
15
15
  it { is_expected.to change { Match.count }.by(1) }
@@ -22,7 +22,7 @@ describe Synchronisable do
22
22
 
23
23
  context 'all' do
24
24
  before :all do
25
- Synchronisable.models = %w(
25
+ Synchronisable.config.models = %w(
26
26
  Tournament Team
27
27
  Match MatchPlayer Player
28
28
  )
@@ -40,7 +40,7 @@ describe Synchronisable do
40
40
 
41
41
  context 'when models setting is overriden in method call' do
42
42
  before :all do
43
- Synchronisable.models = %w(Team Match)
43
+ Synchronisable.config.models = %w(Team Match)
44
44
  end
45
45
 
46
46
  subject do
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: synchronisable
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.5
4
+ version: 1.1.6
5
5
  platform: ruby
6
6
  authors:
7
7
  - Vasiliy Yorkin
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-08-22 00:00:00.000000000 Z
11
+ date: 2014-09-01 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord