riak-ruby-ledger 0.0.4 → 0.0.5

data/docs/release_notes.md

@@ -0,0 +1,33 @@
+## Release notes
+
+##### Version 0.0.4 and Counter Drift
+
+In version 0.0.4 of this gem, counter drift is still a possibility. Take the following scenario into consideration:
+
+1. Actor 1 and Actor 2 both are somehow trying to write the same transaction id, possibly because the process writing the transaction took too long, and your application erroneously had a policy of retrying the same transaction before the first actor finished.
+    a. If the Actor 1 is successful in writing the transaction before Actor 2 begins, Actor 2 will see that the transaction id already exists, and will return successful before attempting to write.
+    b. Similarly, if Actor 2 finishes before Actor 1 starts, Actor 1 would disregard the request and report success.
+    c. If Actor 1 and Actor 2 simultaneously and successfully write the same transaction, a result of is two siblings.
+2. If 1a or 1b happen, there is no problem. If 1c occurs, the second line of defense happens during a merge (merges are triggered prior to every write, and after every read).
+    a. If Actor 1 merges before Actor 2, Actor 1 will remove it's own duplicate transaction in favor of leaving Actor 2's version, knowing it cannot modify any other actors' data.
+    b. Similarly, if Actor 2 merges before Actor 1, it will remove it's own duplicate transaction.
+    c. If Actor 1 and Actor 2 merge simultaneously and successfully, they would both remove their own duplicate (from their point of view) version of the transaction, meaning it would be lost causing negative counter drift (on increments) and positive drift (on decrements)
+
+This is an unlikely but possible scenario. Here are some ways to reduce or elimiate the possibility of 2c from happening:
+
+1. The precursor to the condition resulting from 2c can be avoided by serializing writes per transaction, like in the example of a game's application server knowing to only submit one unique transaction at a time. Submitting simultaneous transactions is ok, so long as the same transaction isn't active in more than one actor at the same time.
+    a. This is possible using this gem, it's just a matter of implemeting some control over who can write a single unique at the same time.
+2. Have a no duplicate delete policy, meaning that you could potentially have an infinitely growing list of duplicate transactions if your application causes this situation often.
+    a. This is unimplemented in this gem as of now, but depending on the thoughts of others, I may add it as an optional policy.
+3. Attach a microsecond epoch to each transaction so that during merges the the duplicate transaction with the highest epoch always wins.
+    a. This is unimplemented in this gem, and it would only lessen the statistical likelihood of 2c happening, it would still be possible. Because it only lowers the likelihood.
+4. Do a string compare on the actor ids, whichever has the highest string compare value always keeps it's version of the duplicate transaction.
+    a. This is now implemented in version 0.1.0, see below.
+
+##### Version 0.0.5 and Actor Naming [***Important***]
+
+Solution 4 has been implemented to the potential counter drift caused by two simultaneous writes and later merges of a duplicate transaction as described in the previous section.
+
+As a result, keep in mind that when naming actors, they will be compared for ordering purposes
+
+Example: "ACTOR2" is greater than "ACTOR1", so ACTOR1 will always remove it's version of a duplicate transaction during a merge, and "ACTOR2" will never remove it's version. Avoid using actor ids that could potentially result in string equality.

data/docs/riak_counter_drift.md

@@ -0,0 +1,56 @@
+## Riak Counters and Drift
+
+### Summary
+
+**Why shouldn't I use Riak Counters?**
+
+CRDT PNCounters (two plain GCounters) such as Riak Counters are non-idempotent and store nothing about a counter transaction other than the final value. This means that if an increment operation fails in any number of ways (500 response from server, process that made the call dies, network connection is interrupted, operation times out, etc), your application now has no idea whether or not the increment actually happened.
+
+**What is Counter Drift?**
+
+In the above situation of a failed increment operation, your application has two choices:
+
+1. Retry the operation: This could result in the operation occuring twice causing what is called **positive counter drift**
+2. Don't retry the operation: This could result in the operation never occuring at all causing **negative counter drift**
+
+As such it doesn't make sense to use plain GCounters or PNCounters to store any counter that needs to be accurate.
+
+### When to use Riak Counters
+
+Riak Counters are very well suited for certain problems:
+
+* Facebook likes
+* Youtube views
+* Reddit upvotes
+* Twitter followers
+* Any non-critical counts
+    * Counts that do not adversely affect applications or users when off by a few
+
+### When not to use Riak Counters
+
+* Currency (virtual or real) balances
+* Metrics that result in charging a customer
+    * Keeping track of how many calls are made to a paid API endpoint
+    * Storage used by a user
+* Real-time counts
+* Any critical counts
+    * Counts that must be accurate
+
+### Counter Drift
+
+Riak Counters (and GSets in general) as currently implemented are not ***idempotent***. This simply means that you cannot retry the same increment or decrement operation more than once.
+
+Take the following scenario into consideration:
+
+1. User buys an in-game item that costs 50 gold, and has a current balance of 100 gold
+2. Application server attempts to debit user's account 50 gold
+    a. If Riak successfully returns 200 response code, no problem!
+    b. If Riak returns 500 (or any other error code), we don't have any way of knowing whether or not the operation succeeded
+    c. If the application server fails at any point in the execution, we also don't have a good way of knowing whether or not the operation succeeded
+
+In the case of 2b and 2c, we have the following choices:
+
+* Retry the operation (Risk absolute positive drift)
+    * If the original counter decrement was successful, we have now debited the user's balance twice, thereby charging them 100 gold for a 50 gold item
+* Never retry (Risk absolute negative drift)
+    * If the original counter decrement was unsuccessful, we gave the user an item for free

data/docs/usage.md

@@ -0,0 +1,110 @@
+## Suggested Usage and Configuration
+
+### Summary
+
+Depending on your use case, you may want to tweak the configuration options `:history_length` and `:retry_count`.
+
+The default `:history_length` is 10. This means that if a transaction fails, but your application is unable to determine whether or not the counter was actually incremented, you have a buffer space or window of 9 additional transactions on that counter before you can no longer retry the original failed transaction without assuming counter drift is happening.
+
+The default `:retry_count` is also 10. This means that if a transaction fails, the actor that attempted the transaction will continue trying 9 more times. If the request to change the counter still fails after the 10th try, the operation will return `false` for failure. At this point your application can attempt to try the transaction again, or return a failure to the user with a note that the transaction will be retried in the future.
+
+An example of a failure might look like the following:
+
+1. transaction1 fails with actor1, and because of the nature of the failure, your application is unsure whether or not the counter was actually incremented.
+
+	1. If your `:retry_count` is low, you can quickly determine in your application that something went wrong, and inform the user that the transaction was unsuccessful for now, but will be attempted later
+	2. If your `:retry_count` is high, the user will be kept waiting longer, but the odds of the transaction eventually working are higher
+2. If after the initial retries, the transaction was still a failure, your application must decide what to do next
+
+	1. If your `:history_length` is low, your options are limited. You must continue to retry that same failed transaction for that user (using any available actor) until it is successful. If you allow additional transactions to take place on the same counter before retrying, you run a high risk of counter drift.
+	2. If your `:history_length` is medium-high, then you have an allowance of (`:history_length` - 1) additional transactions for that counter before you run the risk of counter drift.
+
+**Note**
+
+This gem cannot guarentee transaction idempotence of a counter for greater than `:history_length` number of transactions.
+
+### Tunable Transaction History
+By allowing clients to set how many transactions to keep in the counter object as well as set a retry policy on the Riak actions performed on the counter, a good balance can be achieved. The `Riak::Ledger` class in this gem can be instantiated with the following options:
+
+```
+:actor => Actor ID, one per thread or serialized writer
+:history_length => Number of transactions to store per actor per type (credit or debit)
+:retry_count => Number of times to retry Riak requests if they fail
+```
+
+Furthermore, each `#credit!` and `#debit!` action against the ledger takes an (assumed) globally unique `transaction` id that is determined by your application.
+
+These options combined give you reasonable guarentees that a single transaction can be retried per counter continually as long as less than X number of other transactions are applied to the same counter (where X is the `:history_length`).
+
+The gem will automatically retry `:retry_count` number of times, and if it still fails after that you can define a secondary retry or reconciliation policy within your application to deal with the failure, although if the actions are continually failing, it is possible that something is systematically wrong with your Riak cluster.
+
+##### Merging Siblings and Collapsing Old Transactions
+
+Prior to every write (`#credit!` and `#debit!`), and on every read (`#find!`), two merges happen: Sibling Merges and Transaction Collapse
+
+Sibling Merges are just combining the data from two Riak siblings into a single object, nothing extraordinary happening here.
+
+Transaction collapse happens based on the specified or default `:history_length`. In the following example, assume `:history_length` is equal to 2:
+
+Add 3 transactions
+
+```
+ledger = Riak::Ledger.new(client["ledgers"], "player_2", {:history_length => 2})
+
+ledger.credit!("txn1", 10)
+ledger.credit!("txn2", 10)
+ledger.credit!("txn3", 10)
+```
+
+Check transaction existence
+
+```
+ledger.has_transaction? "txn1" #true
+ledger.has_transaction? "txn2" #true
+ledger.has_transaction? "txn3" #true
+```
+
+Based on the above, you might expect "txn1" to have been collapsed; however, merges happen only before writes, and when reads happen. This is because prior to every write, a read occurs triggering a merge. Given those facts, after a read happens, a merge should occur
+
+```
+ledger = Riak::Ledger.find!(client["ledgers"], "player_2", {:history_length => 2})
+
+ledger.has_transaction? "txn1" #false
+```
+
+### :retry_count values
+
+A low `:retry_count` (1-9) might be appropriate for applications that would rather give immediate failed request feedback to their users so that they can continue performing other actions. This should be coupled with a higher `:history_length` if you intend to allow your user to initiate other transactions while waiting to retry the first one.
+
+A medium `:retry_count` (10-50) might be appropriate for applications that require a higher level of certainty about a specific transaction's success at all times. Allowing a single actor to attempt retries for as long as necessary also greatly reduces the chance that duplicate transactions will ever be created, but requests will take longer in that case. Even if duplicate transactions are created, they should be merged at a later time, but it is safer to have a 1 actor per transaction at a time policy.
+
+### :history_length values
+
+A low `:history_length` (1-9) is never really suggested, as it lowers the time window for idempotent operations to occur. The only time a low `:history_length` might be necessary is if your cluster is not big enough to handle the space consumed by the transaction list. Here is an example calculation to show how much space various transaction histories might consume.
+
+```
+# Riak's replication value
+n = 3
+actor_count = 5
+# a single transaction within counter json might
+# look like this: ["550e8400-e29b-41d4-a716-446655440000": 10],
+bytes_per_txn = 45
+# if you have 1 million users, and 1 ledger per user
+number_of_counters = 1,000,000
+```
+
+For a `:history_length` of 10:
+
+```
+(actor_count * number_of_counters * bytes_per_txn * history_length) * n = 6750000000 bytes or 6.28643 GB total raw disk storage
+```
+
+For a `:history_length` of 50:
+
+```
+(actor_count * number_of_counters * bytes_per_txn * history_length) * n = 33750000000 bytes or 31.4321 GB total raw disk storage
+```
+
+A medium `:history_length` (10-50) is a safe balance for most applications. Applications suited for this range of values are ones that do not have very high concurrent access requirements on a per counter basis. For example an application that only allows a user to have one transaction in flight at a time, but wants the option to let the user continue doing a few more transactions before the state of the failed transaction is known.
+
+A high `:history_length` (50+) might be suitable for applications whose primary function is to provide highly concurrent and frequent access to a limited number of counters. An example might be a service that needs to keep accurate track of a limited number of statistics like how much bandwidth is consumed for a series of endpoints for the purposes of billing a customer.

data/lib/crdt/tgcounter.rb

@@ -1,5 +1,3 @@
-require 'set'
-
 module Riak::CRDT
   class TGCounter
     attr_accessor :counts, :actor, :history_length
@@ -63,23 +61,52 @@ module Riak::CRDT
       self.counts[actor]["txns"][transaction] = value
     end
 
-    # Get unique list of all transactions and values across all known actors
-    # @param [String] ignore_actor
+    # Get unique list of all transactions and values across all known actors, or optionally for a single actor
+    # @param [String] for_actor
     # @return [Hash]
-    def unique_transactions(ignore_actor=nil)
+    def unique_transactions(for_actor=nil)
       txns = Hash.new()
 
       self.counts.each do |a, values|
-        unless a == ignore_actor
-          values["txns"].arr.each do |arr|
-              txns[arr[0]] = arr[1]
-          end
+        next if for_actor && a != for_actor
+        values["txns"].arr.each do |arr|
+          txns[arr[0]] = arr[1]
         end
       end
 
       txns
     end
 
+    # Get unique list of all duplicate transactions per actor other than self
+    # @return [Hash]
+    def duplicate_transactions_by_actor()
+      actor_txns = Hash.new()
+
+      my_transactions = self.unique_transactions(self.actor).keys
+
+      self.counts.keys.each do |a|
+        next if a == self.actor
+        uniques = self.unique_transactions(a).keys
+        actor_txns[a] = (my_transactions & uniques)
+      end
+
+      actor_txns
+    end
+
+    # Get unique list of all duplicate transactions for all actors other than self
+    # @return [Hash]
+    def duplicate_transactions()
+      duplicates = Hash.new()
+
+      self.duplicate_transactions_by_actor().each do |a, txns|
+        txns.each do |txn, val|
+          duplicates[txn] = val
+        end
+      end
+
+      duplicates
+    end
+
     def has_transaction?(transaction)
       self.unique_transactions().keys.member?(transaction)
     end
@@ -96,11 +123,18 @@ module Riak::CRDT
       total
     end
 
-    # Merge actor data from a sibling into self, additionally compress oldest
-    # transactions that exceed the :history_length param into actor's total
+    # Merge actor data from a sibling into self, additionally remove duplicate
+    # transactions and compress oldest transactions that exceed the
+    # :history_length param into actor's total
     # @param [TGCounter] other
     def merge(other)
-      # Combine all actors first
+      self.merge_actors(other)
+      self.remove_duplicates()
+      self.compress_history()
+    end
+
+    # Combine all actors' data
+    def merge_actors(other)
       other.counts.each do |other_actor, other_values|
         if self.counts[other_actor]
           # Max of totals
@@ -118,18 +152,30 @@ module Riak::CRDT
           self.counts[other_actor] = other_values
         end
       end
+    end
 
-      # Remove duplicate transactions if other actors have claimed them
-      self.unique_transactions(actor).keys.each do |txn|
-        self.counts[actor]["txns"].delete(txn)
+    # Remove duplicate transactions if other actors have claimed them
+    def remove_duplicates()
+      self.duplicate_transactions_by_actor().each do |a, txns|
+        # Spaceship operator, if my actor is of greater value than theirs, skip because they should remove the dupe
+        next if (self.actor <=> a) == 1
+        txns.each do |txn|
+          self.counts[self.actor]["txns"].delete(txn)
+        end
       end
+    end
 
-      # Merge this actor's data based on history_length
+    # Compress this actor's data based on history_length
+    def compress_history()
       total = 0
+
+      duplicates = self.duplicate_transactions()
+
       if self.counts[actor]["txns"].length > self.history_length
         to_delete = self.counts[actor]["txns"].length - self.history_length
         self.counts[actor]["txns"].arr.slice!(0..to_delete - 1).each do |arr|
-          total += arr[1]
+          txn, val = arr
+          total += val unless duplicates.member? txn
         end
       end
 
@@ -138,7 +184,7 @@ module Riak::CRDT
   end
 end
 
-# Ease of use class: Wraps an ordered array with some hash-like functions
+# Ease of use class - Wraps an ordered array with some hash-like functions
 class TransactionArray
   attr_accessor :arr
 

data/lib/ledger.rb

@@ -54,7 +54,7 @@ module Riak
     # @see update!(transaction, value)
     # @return [Boolean]
     def credit!(transaction, value)
-      update!(transaction, value)
+      self.update!(transaction, value)
     end
 
     # Decrement the counter, merge and save it
@@ -63,7 +63,7 @@ module Riak
     # @see update!(transaction, value)
     # @return [Boolean]
     def debit!(transaction, value)
-      update!(transaction, value * -1)
+      self.update!(transaction, value * -1)
     end
 
     # Update the counter, merge and save it. Retry if unsuccessful
@@ -78,10 +78,10 @@ module Riak
       end
 
       # Get the current merged state of this counter
-      vclock = refresh()
+      vclock = self.refresh()
 
 
-      if has_transaction?(transaction)
+      if self.has_transaction?(transaction)
         # If the transaction already exists in the counter, no problem
         return true
       else
@@ -92,10 +92,10 @@ module Riak
           self.counter.increment(transaction, value)
         end
 
-        unless save(vclock)
+        unless self.save(vclock)
           # If the save wasn't successful, retry
           current_retry = self.retry_count unless current_retry
-          update!(transaction, value, current_retry - 1)
+          self.update!(transaction, value, current_retry - 1)
         else
           # If the save succeeded, no problem
           return true
@@ -103,7 +103,7 @@ module Riak
       end
     end
 
-    # Create a new Ledger object
+    # Check if the counter has transaction
     # @param [String] transaction
     # @return [Boolean]
     def has_transaction?(transaction)
@@ -155,7 +155,7 @@ module Riak
       object = self.bucket.new(self.key)
       object.vclock = vclock if vclock
       object.content_type = 'application/json'
-      object.raw_data = to_json
+      object.raw_data = self.to_json()
 
       begin
         options = {:returnbody => false}

data/lib/ledger/version.rb

@@ -1,5 +1,5 @@
 module Riak
   class Ledger
-    VERSION = "0.0.4"
+    VERSION = "0.0.5"
   end
-end
+end

data/riak-ruby-ledger.gemspec

@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
   spec.version       = Riak::Ledger::VERSION
   spec.authors       = ["drewkerrigan"]
   spec.email         = ["dkerrigan@basho.com"]
-  spec.description   = %q{A PNCounter CRDT based ledger with support for transaction ids and tunable write idempotence}
-  spec.summary       = %q{This gem attempts to provide a tunable Counter option by combining non-idempotent GCounters and a partially idempotent GSet for calculating a running counter or ledger. By allowing clients to set how many transactions to keep in the counter object as well as set a retry policy on the Riak actions performed on the counter, a good balance can be achieved.}
+  spec.description   = %q{An alternative to Riak Counters with idempotent writes within a client defined window}
+  spec.summary       = %q{The data type implemented is a PNCounter CRDT with an ordered array of transactions for each GCounter actor. Transaction ids are stored with the GCounter, so operations against this counter are idempotent while the transaction remains in any actor's array.}
   spec.homepage      = "https://github.com/drewkerrigan/riak-ruby-ledger"
   spec.license       = "Apache2"
 

data/test/lib/{tgcounter_test.rb → crdt/tgcounter_test.rb}

@@ -1,4 +1,4 @@
-require_relative '../test_helper'
+require_relative '../../test_helper'
 
 describe Riak::CRDT::TGCounter do
   options1 = {:actor => "ACTOR1", :history_length => 5}

data/test/lib/{tpncounter_test.rb → crdt/tpncounter_test.rb}

@@ -1,4 +1,4 @@
-require_relative '../test_helper'
+require_relative '../../test_helper'
 
 describe Riak::CRDT::TPNCounter do
   options1 = {:actor => "ACTOR1", :history_length => 5}
@@ -59,39 +59,40 @@ describe Riak::CRDT::TPNCounter do
 
   it "must merge" do
     counter = Riak::CRDT::TPNCounter.new(options1)
-    counter.increment("txn1", 10)
-    counter.increment("txn2", 10)
-    counter.increment("txn1", 10)
-    counter.increment("txn1", 10)
-    counter.decrement("txn3", 5)
-
-    counter.increment("txn4", 10)
-    counter.increment("txn5", 10)
-    counter.increment("txn6", 10)
-    counter.increment("txn7", 10)
-    counter.decrement("txn8", 5)
+    counter.increment("txn1", 10) #ignore
+    counter.increment("txn2", 10) #ignore
+    counter.increment("txn1", 10) #ignore
+    counter.increment("txn1", 10) #ignore
+    counter.decrement("txn3", 5) #keep
+
+    counter.increment("txn4", 10) #ignore
+    counter.increment("txn5", 10) #keep
+    counter.increment("txn6", 10) #keep
+    counter.increment("txn7", 10) #keep
+    counter.decrement("txn8", 5) #keep
 
     counter2 = Riak::CRDT::TPNCounter.new(options2)
-    counter2.increment("txn1", 10)
-    counter2.increment("txn2", 10)
-    counter2.increment("txn4", 10)
-    counter2.increment("txn1", 10)
-    counter2.decrement("txn5", 1)
-
-    counter2.increment("txn9", 10)
-    counter2.increment("txn10", 10)
-    counter2.increment("txn11", 10)
-    counter2.increment("txn12", 10)
-    counter2.decrement("txn13", 1)
+    counter2.increment("txn1", 10) #ignore
+    counter2.increment("txn2", 10) #keep
+    counter2.increment("txn4", 10) #keep
+    counter2.increment("txn1", 10) #keep
+    counter2.decrement("txn14", 1) #keep
+
+    counter2.increment("txn9", 10) #keep
+    counter2.increment("txn10", 10) #keep
+    counter2.increment("txn11", 10) #keep
+    counter2.increment("txn12", 10) #keep
+    counter2.decrement("txn13", 1) #keep
 
     counter.merge(counter2)
 
     assert_equal(0, counter.p.counts["ACTOR1"]["total"])
     assert_equal(88, counter.value)
 
+    counter.increment("txn9", 10) #ignore, keep in actor 1 even though actor 2 would normally have it
     counter2.merge(counter)
 
-    assert_equal(20, counter2.p.counts["ACTOR2"]["total"])
+    assert_equal(10, counter2.p.counts["ACTOR2"]["total"])
     assert_equal(88, counter2.value)
   end
 end