meangirls 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +17 -0
- data/Gemfile +3 -0
- data/README.markdown +243 -0
- data/Rakefile +1 -0
- data/lib/meangirls.rb +58 -0
- data/lib/meangirls/counter.rb +33 -0
- data/lib/meangirls/crdt.rb +15 -0
- data/lib/meangirls/g_counter.rb +82 -0
- data/lib/meangirls/lww_set.rb +168 -0
- data/lib/meangirls/or_set.rb +134 -0
- data/lib/meangirls/set.rb +63 -0
- data/lib/meangirls/two_phase_set.rb +92 -0
- data/lib/meangirls/version.rb +3 -0
- data/meangirls.gemspec +24 -0
- data/spec/counter.rb +58 -0
- data/spec/crdt.rb +16 -0
- data/spec/g_counter.rb +14 -0
- data/spec/init.rb +9 -0
- data/spec/lww_set.rb +65 -0
- data/spec/or_set.rb +41 -0
- data/spec/prob.rb +93 -0
- data/spec/run +21 -0
- data/spec/set.rb +186 -0
- data/spec/two_phase_set.rb +37 -0
- metadata +159 -0
data/.gitignore
ADDED
data/Gemfile
ADDED
data/README.markdown
ADDED
@@ -0,0 +1,243 @@
|
|
1
|
+
Meangirls
|
2
|
+
=========
|
3
|
+
|
4
|
+
Serializable data types for eventually consistent systems.
|
5
|
+
|
6
|
+
Installation
|
7
|
+
============
|
8
|
+
|
9
|
+
Install locally:
|
10
|
+
|
11
|
+
``` bash
|
12
|
+
$ bundle install
|
13
|
+
$ rake build
|
14
|
+
$ gem install pkg/meangirls-0.1.0.gem
|
15
|
+
```
|
16
|
+
|
17
|
+
Or, you can reference it from your `Gemfile`:
|
18
|
+
|
19
|
+
``` ruby
|
20
|
+
gem "meangirls", github: "aphyr/meangirls", branch: "master"
|
21
|
+
```
|
22
|
+
|
23
|
+
Data Types
|
24
|
+
==========
|
25
|
+
|
26
|
+
Sets
|
27
|
+
----
|
28
|
+
|
29
|
+
### G-Set
|
30
|
+
|
31
|
+
Set union is commutative and convergent; hence it is always safe to have
|
32
|
+
simultaneous writes to a set *which only allows addition*. You cannot remove an
|
33
|
+
element of a G-Set.
|
34
|
+
|
35
|
+
JSON:
|
36
|
+
|
37
|
+
``` javascript
|
38
|
+
{
|
39
|
+
'type': 'g-set',
|
40
|
+
'e': ['a', 'b', 'c']
|
41
|
+
}
|
42
|
+
```
|
43
|
+
|
44
|
+
### 2P-Set
|
45
|
+
|
46
|
+
2-phase sets consist of two g-sets: one for adding, and another for removing.
|
47
|
+
To add an element, add it to the add set A. To remove e, add e to the remove
|
48
|
+
set R. An element can only be added once, and only removed once. Elements can
|
49
|
+
only be removed if they are present in the set. Removes take precedence over
|
50
|
+
adds.
|
51
|
+
|
52
|
+
JSON:
|
53
|
+
|
54
|
+
``` javascript
|
55
|
+
{
|
56
|
+
'type': '2p-set',
|
57
|
+
'a': ['a', 'b'],
|
58
|
+
'r': ['b']
|
59
|
+
}
|
60
|
+
```
|
61
|
+
|
62
|
+
In this set, only 'a' is present.
|
63
|
+
|
64
|
+
### LWW-Element-Set
|
65
|
+
|
66
|
+
LWW-Element-Set is like 2P-Set: it comprises an add G-Set (A) and a remove
|
67
|
+
G-Set (R), with a timestamp for each element. To add an element e, add (e,
|
68
|
+
timestamp) to the add set A. To remove e, add (e, timestamp) to the remove set
|
69
|
+
R. An element is present iff it is in A, and no *newer* element exists in R.
|
70
|
+
Merging is accomplished by taking the union of all A and all R, respectively.
|
71
|
+
|
72
|
+
Since the last write wins, we can safely take only the largest add, and the
|
73
|
+
largest delete. All others can be pruned.
|
74
|
+
|
75
|
+
When A and R have equal timestamps, the direction of the inequality determines
|
76
|
+
whether adds or removes win. {'bias': 'a'} indicates that adds win. {'bias':
|
77
|
+
'r'} indicates that removes win. The default bias is 'a'.
|
78
|
+
|
79
|
+
Timestamps may be *any* ordered primitive: integers, floats, strings, etc. If a
|
80
|
+
coordinated unique timestamp service is used, LWW-Element-Set behaves like a
|
81
|
+
traditional consistent Set structure. If non-unique timestamps are used, the
|
82
|
+
resolution of the timestamp determines the window under which conflicts will be
|
83
|
+
resolved by the bias towards adds or deletes.
|
84
|
+
|
85
|
+
TODO: define sorting strategies for strings. By byte value, UTF-8 ordering,
|
86
|
+
numeric, etc...
|
87
|
+
|
88
|
+
In JSON, we write the set as a list of 2- or 3-tuples: [element, add-time] or
|
89
|
+
[element, add-time, delete-time]
|
90
|
+
|
91
|
+
JSON:
|
92
|
+
|
93
|
+
``` javascript
|
94
|
+
{
|
95
|
+
'type': 'lww-e-set',
|
96
|
+
'bias': 'a',
|
97
|
+
'e': [
|
98
|
+
['a', 0],
|
99
|
+
['b', 1, 2],
|
100
|
+
['c', 2, 1],
|
101
|
+
['d', 3, 3]
|
102
|
+
]
|
103
|
+
}
|
104
|
+
```
|
105
|
+
|
106
|
+
In this set:
|
107
|
+
|
108
|
+
- a was created at 0 and still exists.
|
109
|
+
- b was deleted after creation; it does not exist.
|
110
|
+
- c was created after deletion; it exists
|
111
|
+
- d was created and deleted at the same time. Bias a means we prefer adds, so it exists.
|
112
|
+
|
113
|
+
### OR-Set
|
114
|
+
|
115
|
+
Observed-Removed Sets support adding and removing elements in a causally
|
116
|
+
consistent manner. It resembles LWW-Set, except that instead of times, unique
|
117
|
+
tags are associated with each insertion or deletion. In the case of conflicting
|
118
|
+
add and delete, add wins. An element is a member of the set iff the set of
|
119
|
+
insertion tags less the set of deletion tags is nonempty.
|
120
|
+
|
121
|
+
We write the set as a list of 2- or 3- tuples: [element, [add-tags]] or
|
122
|
+
[element, [add-tags], [remove-tags]]
|
123
|
+
|
124
|
+
To insert e, generate a unique tag, and add it to the insertion tag set for e.
|
125
|
+
|
126
|
+
To remove e, take all insertion tags for e, and insert them into the deletion
|
127
|
+
tags for e.
|
128
|
+
|
129
|
+
To merge two OR-Sets, for each element in either set, take the union of the
|
130
|
+
insertion tags and the union of the deletion tags.
|
131
|
+
|
132
|
+
Tags may be any primitive: strings, ints, floats, etc.
|
133
|
+
|
134
|
+
JSON:
|
135
|
+
|
136
|
+
``` javascript
|
137
|
+
{
|
138
|
+
'type': 'or-set',
|
139
|
+
'e': [
|
140
|
+
['a', [1]],
|
141
|
+
['b', [1], [1]],
|
142
|
+
['c', [1, 2], [2, 3]]
|
143
|
+
]
|
144
|
+
}
|
145
|
+
```
|
146
|
+
|
147
|
+
- a exists.
|
148
|
+
- b's only insertion was deleted, so it does not exist.
|
149
|
+
- c has two insertions, only one of which was deleted. It exists.
|
150
|
+
|
151
|
+
### Max-Change-Sets
|
152
|
+
|
153
|
+
MC-Sets resolve divergent histories for an element by choosing the value which
|
154
|
+
has changed the most. You cannot delete an element which is not present, and
|
155
|
+
cannot add an element which is already present. MC-sets are compact and do the
|
156
|
+
right thing when changes to elements are infrequent compared to the conflict
|
157
|
+
resolution window, but behave arbitrarily when divergent histories each include
|
158
|
+
many changes.
|
159
|
+
|
160
|
+
Each element e is associated with an integer n, implicitly assumed to be zero.
|
161
|
+
When n is even, the element is absent from the set. When n is odd, the element
|
162
|
+
is present. To add an element to the set, increment n from an even value by
|
163
|
+
one; to remove an element, increment n from an odd value by one. To merge sets,
|
164
|
+
take each element and choose the maximum value of n from each history.
|
165
|
+
|
166
|
+
When n is limited to [0, 2], Max-Change-Sets collapse to 2P-Sets. Unlike
|
167
|
+
2P-Sets, however, one can add and remove an arbitrary number of times. The
|
168
|
+
disadvantage is that there is no bias towards preserving adds or removes.
|
169
|
+
Instead, whichever history has incremented further (undergone more changes) is
|
170
|
+
preferred.
|
171
|
+
|
172
|
+
In JSON, max-change sets are represented as a list of [element, n] tuples.
|
173
|
+
|
174
|
+
JSON:
|
175
|
+
|
176
|
+
``` javascript
|
177
|
+
{
|
178
|
+
'type': 'mc-set',
|
179
|
+
'e': [
|
180
|
+
['a', 1],
|
181
|
+
['b', 2],
|
182
|
+
['c', 3]
|
183
|
+
]
|
184
|
+
}
|
185
|
+
```
|
186
|
+
|
187
|
+
- a is present
|
188
|
+
- b is absent
|
189
|
+
- c is present
|
190
|
+
|
191
|
+
Counters
|
192
|
+
--------
|
193
|
+
|
194
|
+
### G-Counter
|
195
|
+
|
196
|
+
A G-Counter is a grow-only counter (inspired by vector clocks) in
|
197
|
+
which only increment and merge are possible. Incrementing the counter
|
198
|
+
adds 1 to the count for the current actor. Divergent histories are
|
199
|
+
resolved by taking the maximum count for each actor (like a vector
|
200
|
+
clock merge). The value of the counter is the sum of all actor
|
201
|
+
counts.
|
202
|
+
|
203
|
+
JSON:
|
204
|
+
|
205
|
+
``` javascript
|
206
|
+
{
|
207
|
+
'type': 'g-counter',
|
208
|
+
'e': {
|
209
|
+
'a': 1,
|
210
|
+
'b': 5,
|
211
|
+
'c': 2
|
212
|
+
}
|
213
|
+
}
|
214
|
+
```
|
215
|
+
|
216
|
+
- The counter value is 8.
|
217
|
+
|
218
|
+
### PN-Counter
|
219
|
+
|
220
|
+
PN-Counters allow the counter to be decreased by tracking the
|
221
|
+
increments (P) separate from the decrements (N), both represented as
|
222
|
+
internal G-Counters. Merge is handled by merging the internal P and N
|
223
|
+
counters. The value of the counter is the value of the P counter minus
|
224
|
+
the value of the N counter.
|
225
|
+
|
226
|
+
JSON:
|
227
|
+
|
228
|
+
``` javascript
|
229
|
+
{
|
230
|
+
'type': 'pn-counter',
|
231
|
+
'p': {
|
232
|
+
'a': 10,
|
233
|
+
'b': 2
|
234
|
+
},
|
235
|
+
'n': {
|
236
|
+
'c': 5,
|
237
|
+
'a': 1
|
238
|
+
}
|
239
|
+
}
|
240
|
+
```
|
241
|
+
|
242
|
+
- P=12, N=6, so the value is 6.
|
243
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "bundler/gem_tasks"
|
data/lib/meangirls.rb
ADDED
@@ -0,0 +1,58 @@
|
|
1
|
+
module Meangirls
|
2
|
+
$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__))
|
3
|
+
|
4
|
+
class ReinsertNotAllowed < RuntimeError; end
|
5
|
+
class DeleteNotAllowed < RuntimeError; end
|
6
|
+
|
7
|
+
require 'set'
|
8
|
+
require 'base64'
|
9
|
+
require 'securerandom'
|
10
|
+
require 'socket'
|
11
|
+
require 'meangirls/crdt'
|
12
|
+
|
13
|
+
# Transforms a JSON data structure into a CRDT datatype.
|
14
|
+
def parse(s)
|
15
|
+
case s['type']
|
16
|
+
when '2p-set'
|
17
|
+
TwoPhaseSet.new s
|
18
|
+
when 'lww-set'
|
19
|
+
LWWSet.new s
|
20
|
+
when 'or-set'
|
21
|
+
ORSet.new s
|
22
|
+
when 'g-counter'
|
23
|
+
GCounter.new s
|
24
|
+
else
|
25
|
+
raise ArgumentError, "unknown type #{s['type']}"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
module_function :parse
|
29
|
+
|
30
|
+
# The default node name.
|
31
|
+
def node
|
32
|
+
@node ||= Socket.gethostname
|
33
|
+
end
|
34
|
+
module_function :node
|
35
|
+
|
36
|
+
# Set the default node name.
|
37
|
+
def node=(node)
|
38
|
+
@node = node
|
39
|
+
end
|
40
|
+
module_function :node=
|
41
|
+
|
42
|
+
# Return a pseudounique tag.
|
43
|
+
def tag
|
44
|
+
SecureRandom.urlsafe_base64
|
45
|
+
end
|
46
|
+
module_function :tag
|
47
|
+
|
48
|
+
# An ISO8601 time as close to the current time as possible, with the
|
49
|
+
# additional constraint that successive calls to this method will return
|
50
|
+
# monotonically increasing values.
|
51
|
+
#
|
52
|
+
# TODO: fold counter inside of time fraction
|
53
|
+
def timestamp
|
54
|
+
@i ||= 0
|
55
|
+
"#{Time.now.utc.iso8601}.#{@i += 1}"
|
56
|
+
end
|
57
|
+
module_function :timestamp
|
58
|
+
end
|
@@ -0,0 +1,33 @@
|
|
1
|
+
class Meangirls::Counter < Meangirls::CRDT
|
2
|
+
require 'meangirls/g_counter'
|
3
|
+
|
4
|
+
def ===(other)
|
5
|
+
# Can only compare to numerics
|
6
|
+
unless other.kind_of? Meangirls::Counter or
|
7
|
+
other.kind_of? Numeric
|
8
|
+
return false
|
9
|
+
end
|
10
|
+
|
11
|
+
# TODO: This gets awkward when we exceed FP integer precision in aggregate
|
12
|
+
# over actors.
|
13
|
+
if float?
|
14
|
+
self.to_f == other.to_f
|
15
|
+
else
|
16
|
+
self.to_i == other.to_i
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
# Returns a copy of this counter, with the local Meangirls node's value
|
21
|
+
# incremented by delta.
|
22
|
+
def +(delta)
|
23
|
+
clone.increment(Meangirls.node, delta)
|
24
|
+
end
|
25
|
+
|
26
|
+
def -(delta)
|
27
|
+
clone.increment(Meangirls.node, -1 * delta)
|
28
|
+
end
|
29
|
+
|
30
|
+
def to_i
|
31
|
+
to_f.to_i
|
32
|
+
end
|
33
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
class Meangirls::CRDT
|
2
|
+
require 'meangirls/set'
|
3
|
+
require 'meangirls/counter'
|
4
|
+
|
5
|
+
# Merge a list of CRDTs by folding over merge.
|
6
|
+
def self.merge(*os)
|
7
|
+
[*os].inject do |o1, o2|
|
8
|
+
o1.merge o2
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
def to_json
|
13
|
+
as_json.to_json
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,82 @@
|
|
1
|
+
class Meangirls::GCounter < Meangirls::Counter
|
2
|
+
attr_accessor :e
|
3
|
+
def initialize(hash = nil)
|
4
|
+
@e = {}
|
5
|
+
|
6
|
+
if hash
|
7
|
+
raise ArgumentError, 'hash must contain e' unless hash['e']
|
8
|
+
@e = hash['e']
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
# Strict equality: all actor counts match.
|
13
|
+
def ==(other)
|
14
|
+
other.kind_of? self.class and
|
15
|
+
@e == other.e
|
16
|
+
end
|
17
|
+
|
18
|
+
def as_json
|
19
|
+
{
|
20
|
+
'type' => type,
|
21
|
+
'e' => @e
|
22
|
+
}
|
23
|
+
end
|
24
|
+
|
25
|
+
# Adds delta to this counter, as tracked by node. Returns self.
|
26
|
+
def increment(node = 1, delta = 1)
|
27
|
+
if delta < 0
|
28
|
+
raise ArgumentError, "Can't decrement a GCounter"
|
29
|
+
end
|
30
|
+
|
31
|
+
if @e[node]
|
32
|
+
@e[node] += delta
|
33
|
+
else
|
34
|
+
@e[node] = delta
|
35
|
+
end
|
36
|
+
|
37
|
+
self
|
38
|
+
end
|
39
|
+
|
40
|
+
# Adds delta to this counter, using the current Meangirls node ID.
|
41
|
+
def +(delta)
|
42
|
+
increment Meangirls.node, delta
|
43
|
+
end
|
44
|
+
|
45
|
+
def clone
|
46
|
+
c = super
|
47
|
+
c.e = @e.clone
|
48
|
+
end
|
49
|
+
|
50
|
+
# Are any sums floating-point?
|
51
|
+
def float?
|
52
|
+
@e.any? do |k, v|
|
53
|
+
v.is_a? Float
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
# Merge with another GCounter and return the merged copy.
|
58
|
+
def merge(other)
|
59
|
+
unless other.kind_of? self.class
|
60
|
+
raise ArgumentErrornew, "other must be a #{self.class}"
|
61
|
+
end
|
62
|
+
|
63
|
+
copy = clone
|
64
|
+
@e.each do |k, v|
|
65
|
+
if existing = copy.e[k]
|
66
|
+
copy.e[k] = v if v > existing
|
67
|
+
else
|
68
|
+
copy.e[k] = v
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
copy
|
73
|
+
end
|
74
|
+
|
75
|
+
def to_f
|
76
|
+
@e.values.inject(&:+) || 0.0
|
77
|
+
end
|
78
|
+
|
79
|
+
def type
|
80
|
+
'g-counter'
|
81
|
+
end
|
82
|
+
end
|