meangirls 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +17 -0
- data/Gemfile +3 -0
- data/README.markdown +243 -0
- data/Rakefile +1 -0
- data/lib/meangirls.rb +58 -0
- data/lib/meangirls/counter.rb +33 -0
- data/lib/meangirls/crdt.rb +15 -0
- data/lib/meangirls/g_counter.rb +82 -0
- data/lib/meangirls/lww_set.rb +168 -0
- data/lib/meangirls/or_set.rb +134 -0
- data/lib/meangirls/set.rb +63 -0
- data/lib/meangirls/two_phase_set.rb +92 -0
- data/lib/meangirls/version.rb +3 -0
- data/meangirls.gemspec +24 -0
- data/spec/counter.rb +58 -0
- data/spec/crdt.rb +16 -0
- data/spec/g_counter.rb +14 -0
- data/spec/init.rb +9 -0
- data/spec/lww_set.rb +65 -0
- data/spec/or_set.rb +41 -0
- data/spec/prob.rb +93 -0
- data/spec/run +21 -0
- data/spec/set.rb +186 -0
- data/spec/two_phase_set.rb +37 -0
- metadata +159 -0
data/.gitignore
ADDED
data/Gemfile
ADDED
data/README.markdown
ADDED
@@ -0,0 +1,243 @@
|
|
1
|
+
Meangirls
|
2
|
+
=========
|
3
|
+
|
4
|
+
Serializable data types for eventually consistent systems.
|
5
|
+
|
6
|
+
Installation
|
7
|
+
============
|
8
|
+
|
9
|
+
Install locally:
|
10
|
+
|
11
|
+
``` bash
|
12
|
+
$ bundle install
|
13
|
+
$ rake build
|
14
|
+
$ gem install pkg/meangirls-0.1.0.gem
|
15
|
+
```
|
16
|
+
|
17
|
+
Or, you can reference it from your `Gemfile`:
|
18
|
+
|
19
|
+
``` ruby
|
20
|
+
gem "meangirls", github: "aphyr/meangirls", branch: "master"
|
21
|
+
```
|
22
|
+
|
23
|
+
Data Types
|
24
|
+
==========
|
25
|
+
|
26
|
+
Sets
|
27
|
+
----
|
28
|
+
|
29
|
+
### G-Set
|
30
|
+
|
31
|
+
Set union is commutative and convergent; hence it is always safe to have
|
32
|
+
simultaneous writes to a set *which only allows addition*. You cannot remove an
|
33
|
+
element of a G-Set.
|
34
|
+
|
35
|
+
JSON:
|
36
|
+
|
37
|
+
``` javascript
|
38
|
+
{
|
39
|
+
'type': 'g-set',
|
40
|
+
'e': ['a', 'b', 'c']
|
41
|
+
}
|
42
|
+
```
|
43
|
+
|
44
|
+
### 2P-Set
|
45
|
+
|
46
|
+
2-phase sets consist of two g-sets: one for adding, and another for removing.
|
47
|
+
To add an element, add it to the add set A. To remove e, add e to the remove
|
48
|
+
set R. An element can only be added once, and only removed once. Elements can
|
49
|
+
only be removed if they are present in the set. Removes take precedence over
|
50
|
+
adds.
|
51
|
+
|
52
|
+
JSON:
|
53
|
+
|
54
|
+
``` javascript
|
55
|
+
{
|
56
|
+
'type': '2p-set',
|
57
|
+
'a': ['a', 'b'],
|
58
|
+
'r': ['b']
|
59
|
+
}
|
60
|
+
```
|
61
|
+
|
62
|
+
In this set, only 'a' is present.
|
63
|
+
|
64
|
+
### LWW-Element-Set
|
65
|
+
|
66
|
+
LWW-Element-Set is like 2P-Set: it comprises an add G-Set (A) and a remove
|
67
|
+
G-Set (R), with a timestamp for each element. To add an element e, add (e,
|
68
|
+
timestamp) to the add set A. To remove e, add (e, timestamp) to the remove set
|
69
|
+
R. An element is present iff it is in A, and no *newer* element exists in R.
|
70
|
+
Merging is accomplished by taking the union of all A and all R, respectively.
|
71
|
+
|
72
|
+
Since the last write wins, we can safely take only the largest add, and the
|
73
|
+
largest delete. All others can be pruned.
|
74
|
+
|
75
|
+
When A and R have equal timestamps, the direction of the inequality determines
|
76
|
+
whether adds or removes win. {'bias': 'a'} indicates that adds win. {'bias':
|
77
|
+
'r'} indicates that removes win. The default bias is 'a'.
|
78
|
+
|
79
|
+
Timestamps may be *any* ordered primitive: integers, floats, strings, etc. If a
|
80
|
+
coordinated unique timestamp service is used, LWW-Element-Set behaves like a
|
81
|
+
traditional consistent Set structure. If non-unique timestamps are used, the
|
82
|
+
resolution of the timestamp determines the window under which conflicts will be
|
83
|
+
resolved by the bias towards adds or deletes.
|
84
|
+
|
85
|
+
TODO: define sorting strategies for strings. By byte value, UTF-8 ordering,
|
86
|
+
numeric, etc...
|
87
|
+
|
88
|
+
In JSON, we write the set as a list of 2- or 3-tuples: [element, add-time] or
|
89
|
+
[element, add-time, delete-time]
|
90
|
+
|
91
|
+
JSON:
|
92
|
+
|
93
|
+
``` javascript
|
94
|
+
{
|
95
|
+
'type': 'lww-e-set',
|
96
|
+
'bias': 'a',
|
97
|
+
'e': [
|
98
|
+
['a', 0],
|
99
|
+
['b', 1, 2],
|
100
|
+
['c', 2, 1],
|
101
|
+
['d', 3, 3]
|
102
|
+
]
|
103
|
+
}
|
104
|
+
```
|
105
|
+
|
106
|
+
In this set:
|
107
|
+
|
108
|
+
- a was created at 0 and still exists.
|
109
|
+
- b was deleted after creation; it does not exist.
|
110
|
+
- c was created after deletion; it exists
|
111
|
+
- d was created and deleted at the same time. Bias a means we prefer adds, so it exists.
|
112
|
+
|
113
|
+
### OR-Set
|
114
|
+
|
115
|
+
Observed-Removed Sets support adding and removing elements in a causally
|
116
|
+
consistent manner. It resembles LWW-Set, except that instead of times, unique
|
117
|
+
tags are associated with each insertion or deletion. In the case of conflicting
|
118
|
+
add and delete, add wins. An element is a member of the set iff the set of
|
119
|
+
insertion tags less the set of deletion tags is nonempty.
|
120
|
+
|
121
|
+
We write the set as a list of 2- or 3- tuples: [element, [add-tags]] or
|
122
|
+
[element, [add-tags], [remove-tags]]
|
123
|
+
|
124
|
+
To insert e, generate a unique tag, and add it to the insertion tag set for e.
|
125
|
+
|
126
|
+
To remove e, take all insertion tags for e, and insert them into the deletion
|
127
|
+
tags for e.
|
128
|
+
|
129
|
+
To merge two OR-Sets, for each element in either set, take the union of the
|
130
|
+
insertion tags and the union of the deletion tags.
|
131
|
+
|
132
|
+
Tags may be any primitive: strings, ints, floats, etc.
|
133
|
+
|
134
|
+
JSON:
|
135
|
+
|
136
|
+
``` javascript
|
137
|
+
{
|
138
|
+
'type': 'or-set',
|
139
|
+
'e': [
|
140
|
+
['a', [1]],
|
141
|
+
['b', [1], [1]],
|
142
|
+
['c', [1, 2], [2, 3]]
|
143
|
+
]
|
144
|
+
}
|
145
|
+
```
|
146
|
+
|
147
|
+
- a exists.
|
148
|
+
- b's only insertion was deleted, so it does not exist.
|
149
|
+
- c has two insertions, only one of which was deleted. It exists.
|
150
|
+
|
151
|
+
### Max-Change-Sets
|
152
|
+
|
153
|
+
MC-Sets resolve divergent histories for an element by choosing the value which
|
154
|
+
has changed the most. You cannot delete an element which is not present, and
|
155
|
+
cannot add an element which is already present. MC-sets are compact and do the
|
156
|
+
right thing when changes to elements are infrequent compared to the conflict
|
157
|
+
resolution window, but behave arbitrarily when divergent histories each include
|
158
|
+
many changes.
|
159
|
+
|
160
|
+
Each element e is associated with an integer n, implicitly assumed to be zero.
|
161
|
+
When n is even, the element is absent from the set. When n is odd, the element
|
162
|
+
is present. To add an element to the set, increment n from an even value by
|
163
|
+
one; to remove an element, increment n from an odd value by one. To merge sets,
|
164
|
+
take each element and choose the maximum value of n from each history.
|
165
|
+
|
166
|
+
When n is limited to [0, 2], Max-Change-Sets collapse to 2P-Sets. Unlike
|
167
|
+
2P-Sets, however, one can add and remove an arbitrary number of times. The
|
168
|
+
disadvantage is that there is no bias towards preserving adds or removes.
|
169
|
+
Instead, whichever history has incremented further (undergone more changes) is
|
170
|
+
preferred.
|
171
|
+
|
172
|
+
In JSON, max-change sets are represented as a list of [element, n] tuples.
|
173
|
+
|
174
|
+
JSON:
|
175
|
+
|
176
|
+
``` javascript
|
177
|
+
{
|
178
|
+
'type': 'mc-set',
|
179
|
+
'e': [
|
180
|
+
['a', 1],
|
181
|
+
['b', 2],
|
182
|
+
['c', 3]
|
183
|
+
]
|
184
|
+
}
|
185
|
+
```
|
186
|
+
|
187
|
+
- a is present
|
188
|
+
- b is absent
|
189
|
+
- c is present
|
190
|
+
|
191
|
+
Counters
|
192
|
+
--------
|
193
|
+
|
194
|
+
### G-Counter
|
195
|
+
|
196
|
+
A G-Counter is a grow-only counter (inspired by vector clocks) in
|
197
|
+
which only increment and merge are possible. Incrementing the counter
|
198
|
+
adds 1 to the count for the current actor. Divergent histories are
|
199
|
+
resolved by taking the maximum count for each actor (like a vector
|
200
|
+
clock merge). The value of the counter is the sum of all actor
|
201
|
+
counts.
|
202
|
+
|
203
|
+
JSON:
|
204
|
+
|
205
|
+
``` javascript
|
206
|
+
{
|
207
|
+
'type': 'g-counter',
|
208
|
+
'e': {
|
209
|
+
'a': 1,
|
210
|
+
'b': 5,
|
211
|
+
'c': 2
|
212
|
+
}
|
213
|
+
}
|
214
|
+
```
|
215
|
+
|
216
|
+
- The counter value is 8.
|
217
|
+
|
218
|
+
### PN-Counter
|
219
|
+
|
220
|
+
PN-Counters allow the counter to be decreased by tracking the
|
221
|
+
increments (P) separate from the decrements (N), both represented as
|
222
|
+
internal G-Counters. Merge is handled by merging the internal P and N
|
223
|
+
counters. The value of the counter is the value of the P counter minus
|
224
|
+
the value of the N counter.
|
225
|
+
|
226
|
+
JSON:
|
227
|
+
|
228
|
+
``` javascript
|
229
|
+
{
|
230
|
+
'type': 'pn-counter',
|
231
|
+
'p': {
|
232
|
+
'a': 10,
|
233
|
+
'b': 2
|
234
|
+
},
|
235
|
+
'n': {
|
236
|
+
'c': 5,
|
237
|
+
'a': 1
|
238
|
+
}
|
239
|
+
}
|
240
|
+
```
|
241
|
+
|
242
|
+
- P=12, N=6, so the value is 6.
|
243
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "bundler/gem_tasks"
|
data/lib/meangirls.rb
ADDED
@@ -0,0 +1,58 @@
|
|
1
|
+
module Meangirls
|
2
|
+
$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__))
|
3
|
+
|
4
|
+
class ReinsertNotAllowed < RuntimeError; end
|
5
|
+
class DeleteNotAllowed < RuntimeError; end
|
6
|
+
|
7
|
+
require 'set'
|
8
|
+
require 'base64'
|
9
|
+
require 'securerandom'
|
10
|
+
require 'socket'
|
11
|
+
require 'meangirls/crdt'
|
12
|
+
|
13
|
+
# Transforms a JSON data structure into a CRDT datatype.
|
14
|
+
def parse(s)
|
15
|
+
case s['type']
|
16
|
+
when '2p-set'
|
17
|
+
TwoPhaseSet.new s
|
18
|
+
when 'lww-set'
|
19
|
+
LWWSet.new s
|
20
|
+
when 'or-set'
|
21
|
+
ORSet.new s
|
22
|
+
when 'g-counter'
|
23
|
+
GCounter.new s
|
24
|
+
else
|
25
|
+
raise ArgumentError, "unknown type #{s['type']}"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
module_function :parse
|
29
|
+
|
30
|
+
# The default node name.
|
31
|
+
def node
|
32
|
+
@node ||= Socket.gethostname
|
33
|
+
end
|
34
|
+
module_function :node
|
35
|
+
|
36
|
+
# Set the default node name.
|
37
|
+
def node=(node)
|
38
|
+
@node = node
|
39
|
+
end
|
40
|
+
module_function :node=
|
41
|
+
|
42
|
+
# Return a pseudounique tag.
|
43
|
+
def tag
|
44
|
+
SecureRandom.urlsafe_base64
|
45
|
+
end
|
46
|
+
module_function :tag
|
47
|
+
|
48
|
+
# An ISO8601 time as close to the current time as possible, with the
|
49
|
+
# additional constraint that successive calls to this method will return
|
50
|
+
# monotonically increasing values.
|
51
|
+
#
|
52
|
+
# TODO: fold counter inside of time fraction
|
53
|
+
def timestamp
|
54
|
+
@i ||= 0
|
55
|
+
"#{Time.now.utc.iso8601}.#{@i += 1}"
|
56
|
+
end
|
57
|
+
module_function :timestamp
|
58
|
+
end
|
@@ -0,0 +1,33 @@
|
|
1
|
+
class Meangirls::Counter < Meangirls::CRDT
|
2
|
+
require 'meangirls/g_counter'
|
3
|
+
|
4
|
+
def ===(other)
|
5
|
+
# Can only compare to numerics
|
6
|
+
unless other.kind_of? Meangirls::Counter or
|
7
|
+
other.kind_of? Numeric
|
8
|
+
return false
|
9
|
+
end
|
10
|
+
|
11
|
+
# TODO: This gets awkward when we exceed FP integer precision in aggregate
|
12
|
+
# over actors.
|
13
|
+
if float?
|
14
|
+
self.to_f == other.to_f
|
15
|
+
else
|
16
|
+
self.to_i == other.to_i
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
# Returns a copy of this counter, with the local Meangirls node's value
|
21
|
+
# incremented by delta.
|
22
|
+
def +(delta)
|
23
|
+
clone.increment(Meangirls.node, delta)
|
24
|
+
end
|
25
|
+
|
26
|
+
def -(delta)
|
27
|
+
clone.increment(Meangirls.node, -1 * delta)
|
28
|
+
end
|
29
|
+
|
30
|
+
def to_i
|
31
|
+
to_f.to_i
|
32
|
+
end
|
33
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
class Meangirls::CRDT
|
2
|
+
require 'meangirls/set'
|
3
|
+
require 'meangirls/counter'
|
4
|
+
|
5
|
+
# Merge a list of CRDTs by folding over merge.
|
6
|
+
def self.merge(*os)
|
7
|
+
[*os].inject do |o1, o2|
|
8
|
+
o1.merge o2
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
def to_json
|
13
|
+
as_json.to_json
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,82 @@
|
|
1
|
+
class Meangirls::GCounter < Meangirls::Counter
|
2
|
+
attr_accessor :e
|
3
|
+
def initialize(hash = nil)
|
4
|
+
@e = {}
|
5
|
+
|
6
|
+
if hash
|
7
|
+
raise ArgumentError, 'hash must contain e' unless hash['e']
|
8
|
+
@e = hash['e']
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
# Strict equality: all actor counts match.
|
13
|
+
def ==(other)
|
14
|
+
other.kind_of? self.class and
|
15
|
+
@e == other.e
|
16
|
+
end
|
17
|
+
|
18
|
+
def as_json
|
19
|
+
{
|
20
|
+
'type' => type,
|
21
|
+
'e' => @e
|
22
|
+
}
|
23
|
+
end
|
24
|
+
|
25
|
+
# Adds delta to this counter, as tracked by node. Returns self.
|
26
|
+
def increment(node = 1, delta = 1)
|
27
|
+
if delta < 0
|
28
|
+
raise ArgumentError, "Can't decrement a GCounter"
|
29
|
+
end
|
30
|
+
|
31
|
+
if @e[node]
|
32
|
+
@e[node] += delta
|
33
|
+
else
|
34
|
+
@e[node] = delta
|
35
|
+
end
|
36
|
+
|
37
|
+
self
|
38
|
+
end
|
39
|
+
|
40
|
+
# Adds delta to this counter, using the current Meangirls node ID.
|
41
|
+
def +(delta)
|
42
|
+
increment Meangirls.node, delta
|
43
|
+
end
|
44
|
+
|
45
|
+
def clone
|
46
|
+
c = super
|
47
|
+
c.e = @e.clone
|
48
|
+
end
|
49
|
+
|
50
|
+
# Are any sums floating-point?
|
51
|
+
def float?
|
52
|
+
@e.any? do |k, v|
|
53
|
+
v.is_a? Float
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
# Merge with another GCounter and return the merged copy.
|
58
|
+
def merge(other)
|
59
|
+
unless other.kind_of? self.class
|
60
|
+
raise ArgumentErrornew, "other must be a #{self.class}"
|
61
|
+
end
|
62
|
+
|
63
|
+
copy = clone
|
64
|
+
@e.each do |k, v|
|
65
|
+
if existing = copy.e[k]
|
66
|
+
copy.e[k] = v if v > existing
|
67
|
+
else
|
68
|
+
copy.e[k] = v
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
copy
|
73
|
+
end
|
74
|
+
|
75
|
+
def to_f
|
76
|
+
@e.values.inject(&:+) || 0.0
|
77
|
+
end
|
78
|
+
|
79
|
+
def type
|
80
|
+
'g-counter'
|
81
|
+
end
|
82
|
+
end
|