meangirls 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
data/README.markdown ADDED
@@ -0,0 +1,243 @@
1
+ Meangirls
2
+ =========
3
+
4
+ Serializable data types for eventually consistent systems.
5
+
6
+ Installation
7
+ ============
8
+
9
+ Install locally:
10
+
11
+ ``` bash
12
+ $ bundle install
13
+ $ rake build
14
+ $ gem install pkg/meangirls-0.1.0.gem
15
+ ```
16
+
17
+ Or, you can reference it from your `Gemfile`:
18
+
19
+ ``` ruby
20
+ gem "meangirls", github: "aphyr/meangirls", branch: "master"
21
+ ```
22
+
23
+ Data Types
24
+ ==========
25
+
26
+ Sets
27
+ ----
28
+
29
+ ### G-Set
30
+
31
+ Set union is commutative and convergent; hence it is always safe to have
32
+ simultaneous writes to a set *which only allows addition*. You cannot remove an
33
+ element of a G-Set.
34
+
35
+ JSON:
36
+
37
+ ``` javascript
38
+ {
39
+ 'type': 'g-set',
40
+ 'e': ['a', 'b', 'c']
41
+ }
42
+ ```
43
+
44
+ ### 2P-Set
45
+
46
+ 2-phase sets consist of two g-sets: one for adding, and another for removing.
47
+ To add an element, add it to the add set A. To remove e, add e to the remove
48
+ set R. An element can only be added once, and only removed once. Elements can
49
+ only be removed if they are present in the set. Removes take precedence over
50
+ adds.
51
+
52
+ JSON:
53
+
54
+ ``` javascript
55
+ {
56
+ 'type': '2p-set',
57
+ 'a': ['a', 'b'],
58
+ 'r': ['b']
59
+ }
60
+ ```
61
+
62
+ In this set, only 'a' is present.
63
+
64
+ ### LWW-Element-Set
65
+
66
+ LWW-Element-Set is like 2P-Set: it comprises an add G-Set (A) and a remove
67
+ G-Set (R), with a timestamp for each element. To add an element e, add (e,
68
+ timestamp) to the add set A. To remove e, add (e, timestamp) to the remove set
69
+ R. An element is present iff it is in A, and no *newer* element exists in R.
70
+ Merging is accomplished by taking the union of all A and all R, respectively.
71
+
72
+ Since the last write wins, we can safely take only the largest add, and the
73
+ largest delete. All others can be pruned.
74
+
75
+ When A and R have equal timestamps, the direction of the inequality determines
76
+ whether adds or removes win. {'bias': 'a'} indicates that adds win. {'bias':
77
+ 'r'} indicates that removes win. The default bias is 'a'.
78
+
79
+ Timestamps may be *any* ordered primitive: integers, floats, strings, etc. If a
80
+ coordinated unique timestamp service is used, LWW-Element-Set behaves like a
81
+ traditional consistent Set structure. If non-unique timestamps are used, the
82
+ resolution of the timestamp determines the window under which conflicts will be
83
+ resolved by the bias towards adds or deletes.
84
+
85
+ TODO: define sorting strategies for strings. By byte value, UTF-8 ordering,
86
+ numeric, etc...
87
+
88
+ In JSON, we write the set as a list of 2- or 3-tuples: [element, add-time] or
89
+ [element, add-time, delete-time]
90
+
91
+ JSON:
92
+
93
+ ``` javascript
94
+ {
95
+ 'type': 'lww-e-set',
96
+ 'bias': 'a',
97
+ 'e': [
98
+ ['a', 0],
99
+ ['b', 1, 2],
100
+ ['c', 2, 1],
101
+ ['d', 3, 3]
102
+ ]
103
+ }
104
+ ```
105
+
106
+ In this set:
107
+
108
+ - a was created at 0 and still exists.
109
+ - b was deleted after creation; it does not exist.
110
+ - c was created after deletion; it exists
111
+ - d was created and deleted at the same time. Bias a means we prefer adds, so it exists.
112
+
113
+ ### OR-Set
114
+
115
+ Observed-Removed Sets support adding and removing elements in a causally
116
+ consistent manner. It resembles LWW-Set, except that instead of times, unique
117
+ tags are associated with each insertion or deletion. In the case of conflicting
118
+ add and delete, add wins. An element is a member of the set iff the set of
119
+ insertion tags less the set of deletion tags is nonempty.
120
+
121
+ We write the set as a list of 2- or 3- tuples: [element, [add-tags]] or
122
+ [element, [add-tags], [remove-tags]]
123
+
124
+ To insert e, generate a unique tag, and add it to the insertion tag set for e.
125
+
126
+ To remove e, take all insertion tags for e, and insert them into the deletion
127
+ tags for e.
128
+
129
+ To merge two OR-Sets, for each element in either set, take the union of the
130
+ insertion tags and the union of the deletion tags.
131
+
132
+ Tags may be any primitive: strings, ints, floats, etc.
133
+
134
+ JSON:
135
+
136
+ ``` javascript
137
+ {
138
+ 'type': 'or-set',
139
+ 'e': [
140
+ ['a', [1]],
141
+ ['b', [1], [1]],
142
+ ['c', [1, 2], [2, 3]]
143
+ ]
144
+ }
145
+ ```
146
+
147
+ - a exists.
148
+ - b's only insertion was deleted, so it does not exist.
149
+ - c has two insertions, only one of which was deleted. It exists.
150
+
151
+ ### Max-Change-Sets
152
+
153
+ MC-Sets resolve divergent histories for an element by choosing the value which
154
+ has changed the most. You cannot delete an element which is not present, and
155
+ cannot add an element which is already present. MC-sets are compact and do the
156
+ right thing when changes to elements are infrequent compared to the conflict
157
+ resolution window, but behave arbitrarily when divergent histories each include
158
+ many changes.
159
+
160
+ Each element e is associated with an integer n, implicitly assumed to be zero.
161
+ When n is even, the element is absent from the set. When n is odd, the element
162
+ is present. To add an element to the set, increment n from an even value by
163
+ one; to remove an element, increment n from an odd value by one. To merge sets,
164
+ take each element and choose the maximum value of n from each history.
165
+
166
+ When n is limited to [0, 2], Max-Change-Sets collapse to 2P-Sets. Unlike
167
+ 2P-Sets, however, one can add and remove an arbitrary number of times. The
168
+ disadvantage is that there is no bias towards preserving adds or removes.
169
+ Instead, whichever history has incremented further (undergone more changes) is
170
+ preferred.
171
+
172
+ In JSON, max-change sets are represented as a list of [element, n] tuples.
173
+
174
+ JSON:
175
+
176
+ ``` javascript
177
+ {
178
+ 'type': 'mc-set',
179
+ 'e': [
180
+ ['a', 1],
181
+ ['b', 2],
182
+ ['c', 3]
183
+ ]
184
+ }
185
+ ```
186
+
187
+ - a is present
188
+ - b is absent
189
+ - c is present
190
+
191
+ Counters
192
+ --------
193
+
194
+ ### G-Counter
195
+
196
+ A G-Counter is a grow-only counter (inspired by vector clocks) in
197
+ which only increment and merge are possible. Incrementing the counter
198
+ adds 1 to the count for the current actor. Divergent histories are
199
+ resolved by taking the maximum count for each actor (like a vector
200
+ clock merge). The value of the counter is the sum of all actor
201
+ counts.
202
+
203
+ JSON:
204
+
205
+ ``` javascript
206
+ {
207
+ 'type': 'g-counter',
208
+ 'e': {
209
+ 'a': 1,
210
+ 'b': 5,
211
+ 'c': 2
212
+ }
213
+ }
214
+ ```
215
+
216
+ - The counter value is 8.
217
+
218
+ ### PN-Counter
219
+
220
+ PN-Counters allow the counter to be decreased by tracking the
221
+ increments (P) separate from the decrements (N), both represented as
222
+ internal G-Counters. Merge is handled by merging the internal P and N
223
+ counters. The value of the counter is the value of the P counter minus
224
+ the value of the N counter.
225
+
226
+ JSON:
227
+
228
+ ``` javascript
229
+ {
230
+ 'type': 'pn-counter',
231
+ 'p': {
232
+ 'a': 10,
233
+ 'b': 2
234
+ },
235
+ 'n': {
236
+ 'c': 5,
237
+ 'a': 1
238
+ }
239
+ }
240
+ ```
241
+
242
+ - P=12, N=6, so the value is 6.
243
+
data/Rakefile ADDED
@@ -0,0 +1 @@
1
+ require "bundler/gem_tasks"
data/lib/meangirls.rb ADDED
@@ -0,0 +1,58 @@
1
+ module Meangirls
2
+ $LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__))
3
+
4
+ class ReinsertNotAllowed < RuntimeError; end
5
+ class DeleteNotAllowed < RuntimeError; end
6
+
7
+ require 'set'
8
+ require 'base64'
9
+ require 'securerandom'
10
+ require 'socket'
11
+ require 'meangirls/crdt'
12
+
13
+ # Transforms a JSON data structure into a CRDT datatype.
14
+ def parse(s)
15
+ case s['type']
16
+ when '2p-set'
17
+ TwoPhaseSet.new s
18
+ when 'lww-set'
19
+ LWWSet.new s
20
+ when 'or-set'
21
+ ORSet.new s
22
+ when 'g-counter'
23
+ GCounter.new s
24
+ else
25
+ raise ArgumentError, "unknown type #{s['type']}"
26
+ end
27
+ end
28
+ module_function :parse
29
+
30
+ # The default node name.
31
+ def node
32
+ @node ||= Socket.gethostname
33
+ end
34
+ module_function :node
35
+
36
+ # Set the default node name.
37
+ def node=(node)
38
+ @node = node
39
+ end
40
+ module_function :node=
41
+
42
+ # Return a pseudounique tag.
43
+ def tag
44
+ SecureRandom.urlsafe_base64
45
+ end
46
+ module_function :tag
47
+
48
+ # An ISO8601 time as close to the current time as possible, with the
49
+ # additional constraint that successive calls to this method will return
50
+ # monotonically increasing values.
51
+ #
52
+ # TODO: fold counter inside of time fraction
53
+ def timestamp
54
+ @i ||= 0
55
+ "#{Time.now.utc.iso8601}.#{@i += 1}"
56
+ end
57
+ module_function :timestamp
58
+ end
@@ -0,0 +1,33 @@
1
+ class Meangirls::Counter < Meangirls::CRDT
2
+ require 'meangirls/g_counter'
3
+
4
+ def ===(other)
5
+ # Can only compare to numerics
6
+ unless other.kind_of? Meangirls::Counter or
7
+ other.kind_of? Numeric
8
+ return false
9
+ end
10
+
11
+ # TODO: This gets awkward when we exceed FP integer precision in aggregate
12
+ # over actors.
13
+ if float?
14
+ self.to_f == other.to_f
15
+ else
16
+ self.to_i == other.to_i
17
+ end
18
+ end
19
+
20
+ # Returns a copy of this counter, with the local Meangirls node's value
21
+ # incremented by delta.
22
+ def +(delta)
23
+ clone.increment(Meangirls.node, delta)
24
+ end
25
+
26
+ def -(delta)
27
+ clone.increment(Meangirls.node, -1 * delta)
28
+ end
29
+
30
+ def to_i
31
+ to_f.to_i
32
+ end
33
+ end
@@ -0,0 +1,15 @@
1
+ class Meangirls::CRDT
2
+ require 'meangirls/set'
3
+ require 'meangirls/counter'
4
+
5
+ # Merge a list of CRDTs by folding over merge.
6
+ def self.merge(*os)
7
+ [*os].inject do |o1, o2|
8
+ o1.merge o2
9
+ end
10
+ end
11
+
12
+ def to_json
13
+ as_json.to_json
14
+ end
15
+ end
@@ -0,0 +1,82 @@
1
+ class Meangirls::GCounter < Meangirls::Counter
2
+ attr_accessor :e
3
+ def initialize(hash = nil)
4
+ @e = {}
5
+
6
+ if hash
7
+ raise ArgumentError, 'hash must contain e' unless hash['e']
8
+ @e = hash['e']
9
+ end
10
+ end
11
+
12
+ # Strict equality: all actor counts match.
13
+ def ==(other)
14
+ other.kind_of? self.class and
15
+ @e == other.e
16
+ end
17
+
18
+ def as_json
19
+ {
20
+ 'type' => type,
21
+ 'e' => @e
22
+ }
23
+ end
24
+
25
+ # Adds delta to this counter, as tracked by node. Returns self.
26
+ def increment(node = 1, delta = 1)
27
+ if delta < 0
28
+ raise ArgumentError, "Can't decrement a GCounter"
29
+ end
30
+
31
+ if @e[node]
32
+ @e[node] += delta
33
+ else
34
+ @e[node] = delta
35
+ end
36
+
37
+ self
38
+ end
39
+
40
+ # Adds delta to this counter, using the current Meangirls node ID.
41
+ def +(delta)
42
+ increment Meangirls.node, delta
43
+ end
44
+
45
+ def clone
46
+ c = super
47
+ c.e = @e.clone
48
+ end
49
+
50
+ # Are any sums floating-point?
51
+ def float?
52
+ @e.any? do |k, v|
53
+ v.is_a? Float
54
+ end
55
+ end
56
+
57
+ # Merge with another GCounter and return the merged copy.
58
+ def merge(other)
59
+ unless other.kind_of? self.class
60
+ raise ArgumentErrornew, "other must be a #{self.class}"
61
+ end
62
+
63
+ copy = clone
64
+ @e.each do |k, v|
65
+ if existing = copy.e[k]
66
+ copy.e[k] = v if v > existing
67
+ else
68
+ copy.e[k] = v
69
+ end
70
+ end
71
+
72
+ copy
73
+ end
74
+
75
+ def to_f
76
+ @e.values.inject(&:+) || 0.0
77
+ end
78
+
79
+ def type
80
+ 'g-counter'
81
+ end
82
+ end