mongoid_collection_snapshot 1.0.1 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rspec +2 -0
- data/.travis.yml +2 -0
- data/CHANGELOG.md +6 -0
- data/Gemfile +5 -5
- data/README.md +100 -47
- data/Rakefile +3 -3
- data/lib/mongoid_collection_snapshot.rb +71 -41
- data/lib/mongoid_collection_snapshot/version.rb +1 -3
- data/spec/models/artist.rb +7 -0
- data/spec/models/artwork.rb +2 -1
- data/spec/models/average_artist_price.rb +21 -11
- data/spec/models/custom_connection_snapshot.rb +2 -2
- data/spec/models/multi_collection_snapshot.rb +21 -7
- data/spec/mongoid/collection_snapshot_spec.rb +68 -44
- data/spec/spec_helper.rb +4 -2
- metadata +13 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 041a83b26ca092439e83e0dd525c73d08ece188a
|
4
|
+
data.tar.gz: 4a5e32a1d3f3809279fbb0747988bde7f254db3b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: b889d8da972a015996384bdf38f2bbe2b53b025efb716a95d55ab88a429e9268dd5cbbdd045edc84fbdbfd34ab694190bcb0554e57c0c3a48802d84393fed2b6
|
7
|
+
data.tar.gz: 2945f8661a3dd9eb70795d79cca5efae7a925a3454a6e94ffe607ce3dd81816055a90ecd4cf2a74b9b7d2f9079776aebaa03fa21d38ec529f944468f11354ada
|
data/.rspec
ADDED
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,12 @@
|
|
1
1
|
Next Release
|
2
2
|
------------
|
3
3
|
|
4
|
+
1.1.0
|
5
|
+
-----
|
6
|
+
|
7
|
+
* [#11](https://github.com/aaw/mongoid_collection_snapshot/pull/10): Added support for accessing snapshot collection documents via Mongoid::CollectionSnapshot#documents - [@dblock](https://github.com/dblock).
|
8
|
+
* [#11](https://github.com/aaw/mongoid_collection_snapshot/pull/11): Upgraded RSpec - [@dblock](https://github.com/dblock).
|
9
|
+
|
4
10
|
1.0.1
|
5
11
|
-----
|
6
12
|
|
data/Gemfile
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
source
|
1
|
+
source 'http://rubygems.org'
|
2
2
|
|
3
3
|
case version = ENV['MONGOID_VERSION'] || '~> 4.0'
|
4
4
|
when /4/
|
@@ -9,10 +9,10 @@ else
|
|
9
9
|
gem 'mongoid', version
|
10
10
|
end
|
11
11
|
|
12
|
-
gem
|
12
|
+
gem 'mongoid_slug'
|
13
13
|
|
14
14
|
group :development, :test do
|
15
|
-
gem
|
16
|
-
gem
|
15
|
+
gem 'rspec', '~> 3.1'
|
16
|
+
gem 'rake'
|
17
|
+
gem 'timecop'
|
17
18
|
end
|
18
|
-
|
data/README.md
CHANGED
@@ -8,30 +8,18 @@ Easy maintenance of collections of processed data in MongoDB with the Mongoid 3.
|
|
8
8
|
Quick example:
|
9
9
|
--------------
|
10
10
|
|
11
|
-
Suppose that you have a Mongoid model called `Artwork`, stored
|
12
|
-
in a MongoDB collection called `artworks` and the underlying documents
|
13
|
-
look something like:
|
11
|
+
Suppose that you have a Mongoid model called `Artwork`, stored in a MongoDB collection called `artworks` and the underlying documents look something like:
|
14
12
|
|
15
13
|
{ name: 'Flowers', artist: 'Andy Warhol', price: 3000000 }
|
16
14
|
|
17
|
-
From time to time, your system runs a map/reduce job to compute the
|
18
|
-
average price of each artist's works, resulting in a collection called
|
19
|
-
`artist_average_price` that contains documents that look like:
|
15
|
+
From time to time, your system runs a map/reduce job to compute the average price of each artist's works, resulting in a collection called `artist_average_price` that contains documents that look like:
|
20
16
|
|
21
|
-
{ _id: { artist: 'Andy Warhol'}, value: { price: 1500000 } }
|
17
|
+
{ _id: { artist: 'Andy Warhol' }, value: { price: 1500000 } }
|
22
18
|
|
23
|
-
If your system wants to maintain and use this average price data, it has
|
24
|
-
to
|
25
|
-
map/reduce result documents don't map well to models in Mongoid.
|
26
|
-
Furthermore, even though map/reduce jobs can take some time to run, you probably
|
27
|
-
want the entire `artist_average_price` collection populated atomically
|
28
|
-
from the point of view of your system, since otherwise you don't ever
|
29
|
-
know the state of the data in the collection - you could access it in
|
30
|
-
the middle of a map/reduce and get partial, incorrect results.
|
19
|
+
If your system wants to maintain and use this average price data, it has to do so at the level of raw MongoDB operations, since map/reduce result documents don't map well to models in Mongoid.
|
20
|
+
Furthermore, even though map/reduce jobs can take some time to run, you probably want the entire `artist_average_price` collection populated atomically from the point of view of your system, since otherwise you don't ever know the state of the data in the collection - you could access it in the middle of a map/reduce and get partial, incorrect results.
|
31
21
|
|
32
|
-
mongoid_collection_snapshot solves this problem by providing an atomic
|
33
|
-
view of collections of data like map/reduce results that live outside
|
34
|
-
of Mongoid.
|
22
|
+
A mongoid_collection_snapshot solves this problem by providing an atomic view of collections of data like map/reduce results that live outside of Mongoid.
|
35
23
|
|
36
24
|
In the example above, we'd set up our average artist price collection like:
|
37
25
|
|
@@ -40,9 +28,10 @@ class AverageArtistPrice
|
|
40
28
|
include Mongoid::CollectionSnapshot
|
41
29
|
|
42
30
|
def build
|
31
|
+
|
43
32
|
map = <<-EOS
|
44
33
|
function() {
|
45
|
-
emit({
|
34
|
+
emit({ artist_id: this['artist_id']}, { count: 1, sum: this['price'] })
|
46
35
|
}
|
47
36
|
EOS
|
48
37
|
|
@@ -51,49 +40,82 @@ class AverageArtistPrice
|
|
51
40
|
var sum = 0;
|
52
41
|
var count = 0;
|
53
42
|
values.forEach(function(value) {
|
54
|
-
sum += value['
|
43
|
+
sum += value['sum'];
|
55
44
|
count += value['count'];
|
56
45
|
});
|
57
|
-
return({count: count, sum: sum});
|
46
|
+
return({ count: count, sum: sum });
|
58
47
|
}
|
59
48
|
EOS
|
60
49
|
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
50
|
+
Artwork.map_reduce(map, reduce).out(inline: 1).each do |doc|
|
51
|
+
collection_snapshot.insert(
|
52
|
+
artist_id: doc['_id']['artist_id'],
|
53
|
+
count: doc['value']['count'],
|
54
|
+
sum: doc['value']['sum']
|
55
|
+
)
|
56
|
+
end
|
66
57
|
end
|
58
|
+
end
|
67
59
|
|
68
|
-
|
69
|
-
|
70
|
-
|
60
|
+
```
|
61
|
+
|
62
|
+
Now, if you want to schedule a recomputation, just call `AverageArtistPrice.create`. You can define other methods on collection snapshots.
|
63
|
+
|
64
|
+
```ruby
|
65
|
+
class AverageArtistPrice
|
66
|
+
...
|
67
|
+
|
68
|
+
def average_price(artist_name)
|
69
|
+
artist = Artist.where(name: artist_name).first
|
70
|
+
doc = collection_snapshot.where(artist_id: artist.id).first
|
71
|
+
doc['sum'] / doc['count']
|
71
72
|
end
|
72
73
|
end
|
73
74
|
```
|
74
75
|
|
75
|
-
|
76
|
-
to schedule a recomputation, just call `AverageArtistPrice.create`. The latest
|
77
|
-
snapshot is always available as `AverageArtistPrice.latest`, so you can write
|
78
|
-
code like:
|
76
|
+
The latest snapshot is always available as `AverageArtistPrice.latest`, so you can write code like:
|
79
77
|
|
80
|
-
```
|
78
|
+
```ruby
|
81
79
|
warhol_expected_price = AverageArtistPrice.latest.average_price('Andy Warhol')
|
82
80
|
```
|
83
81
|
|
84
|
-
And always be sure that you'll never be looking at partial results. The only
|
85
|
-
thing you need to do to hook into mongoid_collection_snapshot is implement the
|
86
|
-
method `build`, which populates the collection snapshot and any indexes you need.
|
82
|
+
And always be sure that you'll never be looking at partial results. The only thing you need to do to hook into mongoid_collection_snapshot is implement the method `build`, which populates the collection snapshot and any indexes you need.
|
87
83
|
|
88
|
-
By default, mongoid_collection_snapshot maintains the most recent two snapshots
|
89
|
-
|
84
|
+
By default, mongoid_collection_snapshot maintains the most recent two snapshots computed any given time.
|
85
|
+
|
86
|
+
Query Snapshot Data with Mongoid
|
87
|
+
--------------------------------
|
88
|
+
|
89
|
+
You can do better than the average price example above and define first-class models for your collection snapshot data, then access them as any other Mongoid collection via collection snapshot's `.documents` method.
|
90
|
+
|
91
|
+
```ruby
|
92
|
+
class AverageArtistPrice
|
93
|
+
document do
|
94
|
+
belongs_to :artist, inverse_of: nil
|
95
|
+
field :sum, type: Integer
|
96
|
+
field :count, type: Integer
|
97
|
+
end
|
98
|
+
|
99
|
+
def average_price(artist_name)
|
100
|
+
artist = Artist.where(name: artist_name).first
|
101
|
+
doc = documents.where(artist: artist).first
|
102
|
+
doc.sum / doc.count
|
103
|
+
end
|
104
|
+
end
|
105
|
+
```
|
106
|
+
|
107
|
+
Another example iterates through all latest artist price averages.
|
108
|
+
|
109
|
+
```ruby
|
110
|
+
AverageArtistPrice.latest.documents.each do |doc|
|
111
|
+
puts "#{doc.artist.name}: #{doc.sum / doc.count}"
|
112
|
+
end
|
113
|
+
```
|
90
114
|
|
91
115
|
Multi-collection snapshots
|
92
116
|
--------------------------
|
93
117
|
|
94
|
-
You can maintain multiple collections atomically within the same snapshot by
|
95
|
-
passing unique collection identifiers to ``collection_snaphot`` when you call it
|
96
|
-
in your build or query methods:
|
118
|
+
You can maintain multiple collections atomically within the same snapshot by passing unique collection identifiers to `collection_snaphot` when you call it in your build or query methods:
|
97
119
|
|
98
120
|
``` ruby
|
99
121
|
class ArtistStats
|
@@ -103,22 +125,53 @@ class ArtistStats
|
|
103
125
|
# ...
|
104
126
|
# define map/reduce for average and max aggregations
|
105
127
|
# ...
|
106
|
-
Mongoid.default_session.command(
|
107
|
-
Mongoid.default_session.command(
|
128
|
+
Mongoid.default_session.command('mapreduce' => 'artworks', map: map_avg, reduce: reduce_avg, out: collection_snapshot('average'))
|
129
|
+
Mongoid.default_session.command('mapreduce' => 'artworks', map: map_max, reduce: reduce_max, out: collection_snapshot('max'))
|
108
130
|
end
|
109
131
|
|
110
132
|
def average_price(artist)
|
111
|
-
doc = collection_snapshot('average').find(
|
112
|
-
doc['value']['sum']/doc['value']['count']
|
133
|
+
doc = collection_snapshot('average').find('_id.artist' => artist).first
|
134
|
+
doc['value']['sum'] / doc['value']['count']
|
113
135
|
end
|
114
136
|
|
115
137
|
def max_price(artist)
|
116
|
-
doc = collection_snapshot('max').find(
|
138
|
+
doc = collection_snapshot('max').find('_id.artist' => artist).first
|
117
139
|
doc['value']['max']
|
118
140
|
end
|
119
141
|
end
|
120
142
|
```
|
121
143
|
|
144
|
+
Specify the name of the collection to define first class Mongoid models.
|
145
|
+
|
146
|
+
```ruby
|
147
|
+
class ArtistStats
|
148
|
+
document('average') do
|
149
|
+
field :value, type: Hash
|
150
|
+
end
|
151
|
+
|
152
|
+
document('max') do
|
153
|
+
field :value, type: Hash
|
154
|
+
end
|
155
|
+
end
|
156
|
+
```
|
157
|
+
|
158
|
+
Access these by name.
|
159
|
+
|
160
|
+
```ruby
|
161
|
+
ArtistStats.latest.documents('average')
|
162
|
+
ArtistStats.latest.documents('max')
|
163
|
+
```
|
164
|
+
|
165
|
+
If fields across multiple collection snapshots are identical, a single default `document` is sufficient.
|
166
|
+
|
167
|
+
```ruby
|
168
|
+
class ArtistStats
|
169
|
+
document do
|
170
|
+
field :value, type: Hash
|
171
|
+
end
|
172
|
+
end
|
173
|
+
```
|
174
|
+
|
122
175
|
Custom database connections
|
123
176
|
---------------------------
|
124
177
|
|
data/Rakefile
CHANGED
@@ -9,7 +9,7 @@ begin
|
|
9
9
|
Bundler.setup(:default, :development)
|
10
10
|
rescue Bundler::BundlerError => e
|
11
11
|
$stderr.puts e.message
|
12
|
-
$stderr.puts
|
12
|
+
$stderr.puts 'Run `bundle install` to install missing gems'
|
13
13
|
exit e.status_code
|
14
14
|
end
|
15
15
|
|
@@ -19,7 +19,7 @@ require 'rspec/core'
|
|
19
19
|
require 'rspec/core/rake_task'
|
20
20
|
RSpec::Core::RakeTask.new(:spec) do |spec|
|
21
21
|
spec.pattern = FileList['spec/**/*_spec.rb']
|
22
|
-
spec.rspec_opts =
|
22
|
+
spec.rspec_opts = '--color --format progress'
|
23
23
|
end
|
24
24
|
|
25
|
-
task :
|
25
|
+
task default: :spec
|
@@ -1,59 +1,89 @@
|
|
1
1
|
require 'mongoid_collection_snapshot/version'
|
2
2
|
|
3
|
-
module Mongoid
|
4
|
-
|
3
|
+
module Mongoid
|
4
|
+
module CollectionSnapshot
|
5
|
+
extend ActiveSupport::Concern
|
5
6
|
|
6
|
-
|
7
|
-
require 'mongoid_slug'
|
7
|
+
DEFAULT_COLLECTION_KEY_NAME = '*'
|
8
8
|
|
9
|
-
|
10
|
-
|
11
|
-
include Mongoid::Slug
|
9
|
+
included do
|
10
|
+
require 'mongoid_slug'
|
12
11
|
|
13
|
-
|
14
|
-
|
12
|
+
include Mongoid::Document
|
13
|
+
include Mongoid::Timestamps::Created
|
14
|
+
include Mongoid::Slug
|
15
15
|
|
16
|
-
|
16
|
+
field :workspace_basename, default: 'snapshot'
|
17
|
+
slug :workspace_basename
|
17
18
|
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
19
|
+
field :max_collection_snapshot_instances, default: 2
|
20
|
+
|
21
|
+
before_create :build
|
22
|
+
after_create :ensure_at_most_two_instances_exist
|
23
|
+
before_destroy :drop_snapshot_collections
|
24
|
+
|
25
|
+
cattr_accessor :document_blocks
|
26
|
+
cattr_accessor :document_classes
|
22
27
|
|
23
|
-
|
24
|
-
|
25
|
-
|
28
|
+
# Mongoid documents on this snapshot.
|
29
|
+
def documents(name = nil)
|
30
|
+
self.document_classes ||= {}
|
31
|
+
class_name = "#{self.class.name}#{id}#{name}".underscore.camelize
|
32
|
+
key = "#{class_name}-#{name || DEFAULT_COLLECTION_KEY_NAME}"
|
33
|
+
self.document_classes[key] ||= begin
|
34
|
+
document_block = document_blocks[name || DEFAULT_COLLECTION_KEY_NAME] if document_blocks
|
35
|
+
collection_name = collection_snapshot(name).name
|
36
|
+
klass = Class.new do
|
37
|
+
include Mongoid::Document
|
38
|
+
cattr_accessor :mongo_session
|
39
|
+
instance_eval(&document_block) if document_block
|
40
|
+
store_in collection: collection_name
|
41
|
+
end
|
42
|
+
klass.mongo_session = snapshot_session
|
43
|
+
Object.const_set(class_name, klass)
|
44
|
+
klass
|
45
|
+
end
|
46
|
+
end
|
26
47
|
end
|
27
|
-
end
|
28
48
|
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
49
|
+
module ClassMethods
|
50
|
+
def latest
|
51
|
+
order_by([[:created_at, :desc]]).first
|
52
|
+
end
|
53
|
+
|
54
|
+
def document(name = nil, &block)
|
55
|
+
self.document_blocks ||= {}
|
56
|
+
self.document_blocks[name || DEFAULT_COLLECTION_KEY_NAME] = block
|
57
|
+
end
|
34
58
|
end
|
35
|
-
end
|
36
59
|
|
37
|
-
|
38
|
-
|
39
|
-
|
60
|
+
def collection_snapshot(name = nil)
|
61
|
+
if name
|
62
|
+
snapshot_session["#{collection.name}.#{name}.#{slug}"]
|
63
|
+
else
|
64
|
+
snapshot_session["#{collection.name}.#{slug}"]
|
65
|
+
end
|
40
66
|
end
|
41
|
-
end
|
42
67
|
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
def ensure_at_most_two_instances_exist
|
48
|
-
all_instances = self.class.order_by([[:created_at, :desc]]).to_a
|
49
|
-
if all_instances.length > self.max_collection_snapshot_instances
|
50
|
-
all_instances[self.max_collection_snapshot_instances..-1].each { |instance| instance.destroy }
|
68
|
+
def drop_snapshot_collections
|
69
|
+
snapshot_session.collections.each do |collection|
|
70
|
+
collection.drop if collection.name =~ /^#{self.collection.name}\.([^\.]+\.)?#{slug}$/
|
71
|
+
end
|
51
72
|
end
|
52
|
-
end
|
53
73
|
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
74
|
+
# Since we should always be using the latest instance of this class, this method is
|
75
|
+
# called after each save - making sure only at most two instances exists should be
|
76
|
+
# sufficient to ensure that this data can be rebuilt live without corrupting any
|
77
|
+
# existing computations that might have a handle to the previous "latest" instance.
|
78
|
+
def ensure_at_most_two_instances_exist
|
79
|
+
all_instances = self.class.order_by([[:created_at, :desc]]).to_a
|
80
|
+
return unless all_instances.length > max_collection_snapshot_instances
|
81
|
+
all_instances[max_collection_snapshot_instances..-1].each(&:destroy)
|
82
|
+
end
|
58
83
|
|
84
|
+
# Override to supply custom database connection for snapshots
|
85
|
+
def snapshot_session
|
86
|
+
Mongoid.default_session
|
87
|
+
end
|
88
|
+
end
|
59
89
|
end
|
data/spec/models/artwork.rb
CHANGED
@@ -1,10 +1,16 @@
|
|
1
1
|
class AverageArtistPrice
|
2
2
|
include Mongoid::CollectionSnapshot
|
3
3
|
|
4
|
+
document do
|
5
|
+
belongs_to :artist, inverse_of: nil
|
6
|
+
field :sum, type: Integer
|
7
|
+
field :count, type: Integer
|
8
|
+
end
|
9
|
+
|
4
10
|
def build
|
5
11
|
map = <<-EOS
|
6
12
|
function() {
|
7
|
-
emit({
|
13
|
+
emit({ artist_id: this['artist_id']}, { count: 1, sum: this['price'] })
|
8
14
|
}
|
9
15
|
EOS
|
10
16
|
|
@@ -16,20 +22,24 @@ class AverageArtistPrice
|
|
16
22
|
sum += value['sum'];
|
17
23
|
count += value['count'];
|
18
24
|
});
|
19
|
-
return({count: count, sum: sum});
|
25
|
+
return({ count: count, sum: sum });
|
20
26
|
}
|
21
27
|
EOS
|
22
28
|
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
29
|
+
Artwork.map_reduce(map, reduce).out(inline: 1).each do |doc|
|
30
|
+
collection_snapshot.insert(
|
31
|
+
artist_id: doc['_id']['artist_id'],
|
32
|
+
count: doc['value']['count'],
|
33
|
+
sum: doc['value']['sum']
|
34
|
+
)
|
35
|
+
end
|
28
36
|
end
|
29
37
|
|
30
|
-
def average_price(
|
31
|
-
|
32
|
-
|
38
|
+
def average_price(artist_name)
|
39
|
+
artist = Artist.where(name: artist_name).first
|
40
|
+
fail 'missing artist' unless artist
|
41
|
+
doc = documents.where(artist: artist).first
|
42
|
+
fail 'missing record' unless doc
|
43
|
+
doc.sum / doc.count
|
33
44
|
end
|
34
|
-
|
35
45
|
end
|
@@ -2,7 +2,7 @@ class CustomConnectionSnapshot
|
|
2
2
|
include Mongoid::CollectionSnapshot
|
3
3
|
|
4
4
|
def self.snapshot_session
|
5
|
-
|
5
|
+
@snapshot_session ||= Moped::Session.new(['127.0.0.1:27017']).tap do |session|
|
6
6
|
session.use :snapshot_test
|
7
7
|
end
|
8
8
|
end
|
@@ -13,6 +13,6 @@ class CustomConnectionSnapshot
|
|
13
13
|
|
14
14
|
def build
|
15
15
|
collection_snapshot.insert('name' => 'foo')
|
16
|
-
collection_snapshot('foo').insert(
|
16
|
+
collection_snapshot('foo').insert('name' => 'bar')
|
17
17
|
end
|
18
18
|
end
|
@@ -1,14 +1,28 @@
|
|
1
1
|
class MultiCollectionSnapshot
|
2
2
|
include Mongoid::CollectionSnapshot
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
3
|
+
|
4
|
+
document('foo') do
|
5
|
+
field :name, type: String
|
6
|
+
field :count, type: Integer
|
7
|
+
end
|
8
|
+
|
9
|
+
document('bar') do
|
10
|
+
field :name, type: String
|
11
|
+
field :number, type: Integer
|
8
12
|
end
|
9
13
|
|
10
|
-
|
11
|
-
|
14
|
+
document('baz') do
|
15
|
+
field :name, type: String
|
16
|
+
field :digit, type: Integer
|
12
17
|
end
|
13
18
|
|
19
|
+
def build
|
20
|
+
collection_snapshot('foo').insert('name' => 'foo!', count: 1)
|
21
|
+
collection_snapshot('bar').insert('name' => 'bar!', number: 2)
|
22
|
+
collection_snapshot('baz').insert('name' => 'baz!', digit: 3)
|
23
|
+
end
|
24
|
+
|
25
|
+
def names
|
26
|
+
%w(foo bar baz).map { |x| collection_snapshot(x).find.first['name'] }.join('')
|
27
|
+
end
|
14
28
|
end
|
@@ -2,104 +2,128 @@ require 'spec_helper'
|
|
2
2
|
|
3
3
|
module Mongoid
|
4
4
|
describe CollectionSnapshot do
|
5
|
-
|
6
|
-
|
7
|
-
Mongoid::CollectionSnapshot::VERSION.should_not be_nil
|
5
|
+
it 'has a version' do
|
6
|
+
expect(Mongoid::CollectionSnapshot::VERSION).not_to be_nil
|
8
7
|
end
|
9
8
|
|
10
|
-
context
|
11
|
-
|
12
|
-
let!(:
|
13
|
-
let!(:
|
14
|
-
let!(:
|
9
|
+
context 'creating a basic snapshot' do
|
10
|
+
let!(:andy_warhol) { Artist.create!(name: 'Andy Warhol') }
|
11
|
+
let!(:damien_hirst) { Artist.create!(name: 'Damien Hirst') }
|
12
|
+
let!(:flowers) { Artwork.create!(name: 'Flowers', artist: andy_warhol, price: 3_000_000) }
|
13
|
+
let!(:guns) { Artwork.create!(name: 'Guns', artist: andy_warhol, price: 1_000_000) }
|
14
|
+
let!(:vinblastine) { Artwork.create!(name: 'Vinblastine', artist: damien_hirst, price: 1_500_000) }
|
15
15
|
|
16
|
-
it
|
17
|
-
AverageArtistPrice.latest.
|
16
|
+
it 'returns nil if no snapshot has been created' do
|
17
|
+
expect(AverageArtistPrice.latest).to be_nil
|
18
18
|
end
|
19
19
|
|
20
|
-
it
|
20
|
+
it 'runs the build method on creation' do
|
21
21
|
snapshot = AverageArtistPrice.create
|
22
|
-
snapshot.average_price('Andy Warhol').
|
23
|
-
snapshot.average_price('Damien Hirst').
|
22
|
+
expect(snapshot.average_price('Andy Warhol')).to eq(2_000_000)
|
23
|
+
expect(snapshot.average_price('Damien Hirst')).to eq(1_500_000)
|
24
24
|
end
|
25
25
|
|
26
|
-
it
|
26
|
+
it 'returns the most recent snapshot through the latest methods' do
|
27
27
|
first = AverageArtistPrice.create
|
28
|
-
first.
|
28
|
+
expect(first).to eq(AverageArtistPrice.latest)
|
29
29
|
# "latest" only works up to a resolution of 1 second since it relies on Mongoid::Timestamp. But this
|
30
30
|
# module is meant to snapshot long-running collection creation, so if you need a resolution of less
|
31
31
|
# than a second for "latest" then you're probably using the wrong gem. In tests, sleeping for a second
|
32
32
|
# makes sure we get what we expect.
|
33
|
-
|
33
|
+
Timecop.travel(1.second.from_now)
|
34
34
|
second = AverageArtistPrice.create
|
35
|
-
AverageArtistPrice.latest.
|
36
|
-
|
35
|
+
expect(AverageArtistPrice.latest).to eq(second)
|
36
|
+
Timecop.travel(1.second.from_now)
|
37
37
|
third = AverageArtistPrice.create
|
38
|
-
AverageArtistPrice.latest.
|
38
|
+
expect(AverageArtistPrice.latest).to eq(third)
|
39
39
|
end
|
40
40
|
|
41
|
-
it
|
41
|
+
it 'maintains at most two of the latest snapshots to support its calculations' do
|
42
42
|
AverageArtistPrice.create
|
43
43
|
10.times do
|
44
44
|
AverageArtistPrice.create
|
45
|
-
AverageArtistPrice.count.
|
45
|
+
expect(AverageArtistPrice.count).to eq(2)
|
46
46
|
end
|
47
47
|
end
|
48
48
|
|
49
|
-
|
49
|
+
context '#documents' do
|
50
|
+
it 'provides access to a Mongoid collection' do
|
51
|
+
snapshot = AverageArtistPrice.create
|
52
|
+
expect(snapshot.documents.count).to eq 2
|
53
|
+
document = snapshot.documents.where(artist: andy_warhol).first
|
54
|
+
expect(document.artist).to eq andy_warhol
|
55
|
+
expect(document.count).to eq 2
|
56
|
+
expect(document.sum).to eq 4_000_000
|
57
|
+
end
|
50
58
|
|
51
|
-
|
59
|
+
it 'only creates one global class reference' do
|
60
|
+
3.times do
|
61
|
+
index = AverageArtistPrice.create
|
62
|
+
2.times { expect(index.documents.count).to eq 2 }
|
63
|
+
end
|
64
|
+
expect(AverageArtistPrice.document_classes.count).to be >= 3
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
52
68
|
|
53
|
-
|
54
|
-
|
69
|
+
context 'creating a snapshot containing multiple collections' do
|
70
|
+
it 'populates several collections and allows them to be queried' do
|
71
|
+
expect(MultiCollectionSnapshot.latest).to be_nil
|
55
72
|
10.times { MultiCollectionSnapshot.create }
|
56
|
-
MultiCollectionSnapshot.latest.
|
73
|
+
expect(MultiCollectionSnapshot.latest.names).to eq('foo!bar!baz!')
|
57
74
|
end
|
58
75
|
|
59
|
-
it
|
76
|
+
it 'safely cleans up all collections used by the snapshot' do
|
60
77
|
# Create some collections with names close to the snapshots we'll create
|
61
|
-
Mongoid.default_session["#{MultiCollectionSnapshot.collection.name}.do.not_delete"].insert(
|
62
|
-
Mongoid.default_session["#{MultiCollectionSnapshot.collection.name}.snapshorty"].insert(
|
63
|
-
Mongoid.default_session["#{MultiCollectionSnapshot.collection.name}.hello.1"].insert(
|
78
|
+
Mongoid.default_session["#{MultiCollectionSnapshot.collection.name}.do.not_delete"].insert('a' => 1)
|
79
|
+
Mongoid.default_session["#{MultiCollectionSnapshot.collection.name}.snapshorty"].insert('a' => 1)
|
80
|
+
Mongoid.default_session["#{MultiCollectionSnapshot.collection.name}.hello.1"].insert('a' => 1)
|
64
81
|
|
65
82
|
MultiCollectionSnapshot.create
|
66
|
-
before_create = Mongoid.default_session.collections.map
|
67
|
-
before_create.length.
|
83
|
+
before_create = Mongoid.default_session.collections.map(&:name)
|
84
|
+
expect(before_create.length).to be > 0
|
68
85
|
|
69
|
-
|
86
|
+
Timecop.travel(1.second.from_now)
|
70
87
|
MultiCollectionSnapshot.create
|
71
|
-
after_create = Mongoid.default_session.collections.map
|
88
|
+
after_create = Mongoid.default_session.collections.map(&:name)
|
72
89
|
collections_created = (after_create - before_create).sort
|
73
|
-
collections_created.length.
|
90
|
+
expect(collections_created.length).to eq(3)
|
74
91
|
|
75
92
|
MultiCollectionSnapshot.latest.destroy
|
76
|
-
after_destroy = Mongoid.default_session.collections.map
|
93
|
+
after_destroy = Mongoid.default_session.collections.map(&:name)
|
77
94
|
collections_destroyed = (after_create - after_destroy).sort
|
78
|
-
collections_created.
|
95
|
+
expect(collections_created).to eq(collections_destroyed)
|
79
96
|
end
|
80
|
-
|
81
97
|
end
|
82
98
|
|
83
|
-
context
|
84
|
-
|
99
|
+
context 'with a custom snapshot connection' do
|
85
100
|
around(:each) do |example|
|
86
101
|
CustomConnectionSnapshot.snapshot_session.drop
|
87
102
|
example.run
|
88
103
|
CustomConnectionSnapshot.snapshot_session.drop
|
89
104
|
end
|
90
105
|
|
91
|
-
it
|
106
|
+
it 'builds snapshot in custom database' do
|
92
107
|
snapshot = CustomConnectionSnapshot.create
|
93
108
|
[
|
94
109
|
"#{CustomConnectionSnapshot.collection.name}.foo.#{snapshot.slug}",
|
95
110
|
"#{CustomConnectionSnapshot.collection.name}.#{snapshot.slug}"
|
96
111
|
].each do |collection_name|
|
97
|
-
Mongoid.default_session[collection_name].find.count.
|
98
|
-
CustomConnectionSnapshot.snapshot_session[collection_name].find.count.
|
112
|
+
expect(Mongoid.default_session[collection_name].find.count).to eq(0)
|
113
|
+
expect(CustomConnectionSnapshot.snapshot_session[collection_name].find.count).to eq(1)
|
99
114
|
end
|
100
115
|
end
|
101
116
|
|
117
|
+
context '#documents' do
|
118
|
+
it 'uses the custom session' do
|
119
|
+
expect(CustomConnectionSnapshot.new.documents.mongo_session).to eq CustomConnectionSnapshot.snapshot_session
|
120
|
+
end
|
121
|
+
it 'provides access to a Mongoid collection' do
|
122
|
+
snapshot = CustomConnectionSnapshot.create
|
123
|
+
expect(snapshot.collection_snapshot.find.count).to eq 1
|
124
|
+
expect(snapshot.documents.count).to eq 1
|
125
|
+
end
|
126
|
+
end
|
102
127
|
end
|
103
|
-
|
104
128
|
end
|
105
129
|
end
|
data/spec/spec_helper.rb
CHANGED
@@ -3,12 +3,13 @@ require 'bundler/setup'
|
|
3
3
|
require 'rspec'
|
4
4
|
|
5
5
|
require 'mongoid'
|
6
|
+
require 'timecop'
|
6
7
|
|
7
8
|
Mongoid.configure do |config|
|
8
|
-
config.connect_to(
|
9
|
+
config.connect_to('mongoid_collection_snapshot_test')
|
9
10
|
end
|
10
11
|
|
11
|
-
require File.expand_path(
|
12
|
+
require File.expand_path('../../lib/mongoid_collection_snapshot', __FILE__)
|
12
13
|
Dir["#{File.dirname(__FILE__)}/models/**/*.rb"].each { |f| require f }
|
13
14
|
|
14
15
|
RSpec.configure do |c|
|
@@ -20,3 +21,4 @@ RSpec.configure do |c|
|
|
20
21
|
end
|
21
22
|
end
|
22
23
|
|
24
|
+
RSpec.configure(&:raise_errors_for_deprecations!)
|
metadata
CHANGED
@@ -1,41 +1,41 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: mongoid_collection_snapshot
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Aaron Windsor
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-01-27 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: mongoid
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- -
|
17
|
+
- - '>='
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '3.0'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- -
|
24
|
+
- - '>='
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '3.0'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: mongoid_slug
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
|
-
- -
|
31
|
+
- - '>='
|
32
32
|
- !ruby/object:Gem::Version
|
33
33
|
version: '0'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
|
-
- -
|
38
|
+
- - '>='
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '0'
|
41
41
|
description:
|
@@ -44,8 +44,9 @@ executables: []
|
|
44
44
|
extensions: []
|
45
45
|
extra_rdoc_files: []
|
46
46
|
files:
|
47
|
-
-
|
48
|
-
-
|
47
|
+
- .gitignore
|
48
|
+
- .rspec
|
49
|
+
- .travis.yml
|
49
50
|
- CHANGELOG.md
|
50
51
|
- Gemfile
|
51
52
|
- LICENSE.txt
|
@@ -54,6 +55,7 @@ files:
|
|
54
55
|
- lib/mongoid_collection_snapshot.rb
|
55
56
|
- lib/mongoid_collection_snapshot/version.rb
|
56
57
|
- mongoid_collection_snapshot.gemspec
|
58
|
+
- spec/models/artist.rb
|
57
59
|
- spec/models/artwork.rb
|
58
60
|
- spec/models/average_artist_price.rb
|
59
61
|
- spec/models/custom_connection_snapshot.rb
|
@@ -70,17 +72,17 @@ require_paths:
|
|
70
72
|
- lib
|
71
73
|
required_ruby_version: !ruby/object:Gem::Requirement
|
72
74
|
requirements:
|
73
|
-
- -
|
75
|
+
- - '>='
|
74
76
|
- !ruby/object:Gem::Version
|
75
77
|
version: '0'
|
76
78
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
77
79
|
requirements:
|
78
|
-
- -
|
80
|
+
- - '>='
|
79
81
|
- !ruby/object:Gem::Version
|
80
82
|
version: 1.3.6
|
81
83
|
requirements: []
|
82
84
|
rubyforge_project:
|
83
|
-
rubygems_version: 2.
|
85
|
+
rubygems_version: 2.0.14
|
84
86
|
signing_key:
|
85
87
|
specification_version: 4
|
86
88
|
summary: Easy maintenence of collections of processed data in MongoDB with the Mongoid
|