fluidfeatures 0.4.4 → 0.5.0

data/README.md

@@ -1,7 +1,166 @@
 [![Build Status](https://secure.travis-ci.org/FluidFeatures/fluidfeatures-ruby.png)](http://travis-ci.org/FluidFeatures/fluidfeatures-ruby)
 
-fluidfeatures-ruby
-===================
+Ruby graceful feature rollout and simple A/B testing
+====================================================
 
-Ruby client for API of FluidFeatures.com
+`gem fluidfeatures-ruby` is a pure Ruby client for the API of FluidFeatures.com, which provides an elegant way to wrap new code so you have real-time control over rolling out new features to your user-base.
+
+Integration with Rails
+----------------------
+
+If you are looking to use FluidFeatures with Rails then see https://github.com/FluidFeatures/fluidfeatures-rails
+
+`gem fluidfeatures-rails` uses this `gem fluidfeatures-ruby` but integrates into Rails, which makes it quick and easy to use FluidFeatures with your Rails application.
+
+Ruby specific usage (non-Rails usage)
+=====================================
+
+This gem can be used in any Ruby application. This means a Sinatra application, offline processing or sending emails. Anything that touches your user-base can have a customized experience for each user and can be a vehicle for A/B testing. For instance, you can test different email formats and see which one results in more conversions on your website. You could even integrate into a [Map-Reduce](http://www.bigfastblog.com/map-reduce-with-ruby-using-hadoop) job.
+
+Installation
+------------
+
+```
+gem install fluidfeatures-ruby
+```
+
+"Application"
+-------------
+
+You create a FluidFeatures::App (application) object with the credentials for accessing the FluidFeatures API. These credentials can be found of the application page of your FluidFeatures dashboard.
+
+Call `app` on the `FluidFeatures` module to instantiate this object.
+
+```ruby
+require 'fluidfeatures'
+
+config = {
+  "baseuri" => "https://www.fluidfeatures.com/service"
+  "appid" => "1vu33ki6emqe3"
+  "secret" = "sssseeecrrreeetttt"
+}
+
+fluid_app = FluidFeatures.app(config)
+```
+
+It's also possible to pass a logger object, which will direct all logging to your logger.
+
+```ruby
+fluid_app = FluidFeatures.app(config.update("logger" => logger))
+```
+
+User Transactions
+-----------------
+
+Each interaction with a user is wrapped in a transaction. In fluidfeature-rails, a transaction wraps a single HTTP request.
+
+Transactions guarantee that the feature set for a user will be consistent during the transaction. It is possible that you might reconfigure which users see which features during a transaction, so this ensures that each request is processed atomically regardless of any changes that are occurring outside of the transaction.
+
+To create a new user transaction call `user_transaction` on the `FluidFeatures::App` object, which we assigned to `fluid_app` above.
+
+```ruby
+fluid_user_transaction = fluid_app.user_transaction(user_id, url, display_name, is_anonymous, unique_attrs, cohort_attrs)
+```
+
+An transaction takes a few parameters.
+
+`user_id` is the unique id that you generally refer to this user by. This can be a numeric or a string. If this is `nil` then the user is labeled anonymous and a unique id is generated to track this user. You can retrieve this unique id by using `anonymous_user_id = fluid_user_transaction.user.unique_id`. In HTTP context you should set a HTTP cookie with this `anonymous_user_id`, so that this user is consistently treated as the same user and experiences the same experience for each visit.
+
+`url` is a reference to where the transaction is taking place. This has more meaning in HTTP context.
+
+`display_name` is a human readable name of the user. eg. "Fred Flintstone". FluidFeatures will use this in the web dashboard for display purposes and also for searching for this user.
+
+`is_anonymous` is a boolean that indicates whether this is an anonymous user or not. If this is true then `user_id` can be `nil` if you are not creating and tracking your own user ids for anonymous users.
+
+`unique_attrs` is a `Hash` of other attributes that are unique to this user. For instance, their Twitter handle. Provide these are key-value pairs, which can be displayed and searched in the FluidFeatures dashboard. You can freely pass any key-value pairs you wish. For example, `:nickname => "The Hungry Bear"` or `:twitter => "@philwhln"`.
+
+`cohort_attrs` is similar to `unique_attrs` but are not unique to this user. This is a way that you can group users together when managing feature visibility. These can be anything. A common one is `:admin => true` or `:admin => false`. Other ones include `:coffee_drinker => true`, `:age_group => "25-33"` or `:month_joined => "2012-july"`.
+
+Within the FluidFeatures service, we plan to automatically add other cohorts, such as `:is_early_adopter`, `:active_user`, `:stale_user` to help you target your new features and engage the right users with the right features. This is outside the scope of this gem.
+
+Is this feature enabled for this user?
+--------------------------------------
+
+This is at the core of graceful feature rollout. When the answer is always "no" for a feature, then you are able to push that feature into production without anyone knowing. After that you can start enabling it for specific users and groups of users.
+
+To find out if the feature is enabled for the current user, call `feature_enabled?` on your `FluidFeatures::UserTransaction` object, which we assigned to `fluid_transaction` above.
+
+```ruby
+if fluid_user_transaction.feature_enabled? feature_name
+  # implement feature here
+end
+```
+
+In the above example it will use `"default"` for the version of the feature. If you wish to define your own version, you can pass `version_name` as shown below.
+
+```ruby
+if fluid_user_transaction.feature_enabled? feature_name, version_name
+  # implement feature here
+end
+```
+
+Real-world examples might be...
+
+```ruby
+if fluid_user_transaction.feature_enabled? "theme", "old-one"
+  # show the user the old theme
+end
+if fluid_user_transaction.feature_enabled? "theme", "new-one"
+  # show the user the new theme
+end
+```
+
+```ruby
+if fluid_user_transaction.feature_enabled? "email-sender", "postfix"
+  # send email directly using postfix
+end
+if fluid_user_transaction.feature_enabled? "email-sender", "sendgrid-smtp"
+  # send email using sendgrid's smtp interface
+end
+if fluid_user_transaction.feature_enabled? "email-sender", "sendgrid-api"
+  # send email using sendgrid's http api
+end
+```
+
+When FluidFeatures::UserTransaction gets a call to `feature_enabled?` it may not have seen this feature version before, so it will report this back to the FluidFeatures service as a new feature version. By default, new feature versions are disabled for all users until they are assigned to specific users, cohorts or percentage of users. This is not always ideal, since you may want to wrap an existing feature, such as the `["theme","old-one"]` above, in order to phase it out or test it against a newer version. It that case you can tell FluidFeatures explicitly what the default enabled state of this feature version should be, by passing `default_enabled` boolean as `true` or `false`. `true` will result is all users initially seeing this feature version.
+
+```ruby
+if fluid_user_transaction.feature_enabled? feature_name, version_name, default_enabled
+  # implement feature here
+end
+```
+
+It's a goal!
+------------
+
+The main motivation behind FluidFeatures is to make your site better, increase your conversion rate and ensure that the features you are developing are something that your customers want.
+
+Rolling out new features is fun and doing it gracefully is important, but rolling out the right features is even more important. With "goals" you can keep track of how well each feature is doing and even run short A/B or multi-variant tests to validate your hypothesis about which version of a feature you should rollout.
+
+As you start to rollout a newer version of a feature, you can watch how well your goals are performing for that version and for other versions. If early indications show that your newer version sucks and conversions are lagging, then you can rollback or pause rollout until you can understand the issue better.
+
+Common high-level goals are "signed-up-to-mailing-list", "clicked-buy-button" or "visited-at-least-5-pages-during-session". Lower-level ones might include, "page-loaded-in-under-one-second", "no-errors-written-to-log" or "cpu-stayed-below-80-percent". Facebook tracks spikes in user comments, which they have found usually results in a feature gone wrong and users complaining. Your goals do not have to positive.
+
+Goals are boolean and when one fires for a user then any feature enabled for that user at that time is held accountant, whether the goal is positive or negative.
+
+Flagging a goal is easy. You simply call `goal_hit` on the `FluidFeatures::UserTransaction` object, which in the example below is `fluid_user_transaction`.
+
+```ruby
+fluid_user_transaction.goal_hit goal_name
+```
+
+Might look something like this...
+
+```ruby
+def buy_button_clicked
+  @fluid_user_transaction.goal_hit "clicked-buy-button"
+end
+```
+
+This goal will automatically appear in your FluidFeatures dashboard after the first time `goal_hit` is called with this value.
+
+More info
+=========
+
+For more information visit http://www.fluidfeatures.com or email support@fluidfeatures.com
 

data/lib/fluidfeatures.rb

@@ -1,15 +1,24 @@
-
 require "logger"
 
+require "fluidfeatures/config"
+
+require "fluidfeatures/persistence/storage"
+require "fluidfeatures/persistence/buckets"
+require "fluidfeatures/persistence/features"
+
 require "fluidfeatures/client"
 require "fluidfeatures/app"
 
 module FluidFeatures
   
-  def self.app(base_uri, app_id, secret, logger=nil)
-    logger ||= ::Logger.new(STDERR)
-    client = ::FluidFeatures::Client.new(base_uri, logger)
-    ::FluidFeatures::App.new(client, app_id, secret, logger)
+  class << self
+    attr_accessor :config
   end
 
+  def self.app(config)
+    config["logger"] ||= ::Logger.new(STDERR)
+    self.config = config
+    client = ::FluidFeatures::Client.new(config["baseuri"], config["logger"])
+    ::FluidFeatures::App.new(client, config["appid"], config["secret"], config["logger"])
+  end
 end

data/lib/fluidfeatures/app/reporter.rb

@@ -6,7 +6,7 @@ module FluidFeatures
     
     attr_accessor :app
     
-    # Throw away oldest buckets when this limit reached.
+    # Throw oldest buckets away or offload to persistent storage when this limit reached.
     MAX_BUCKETS = 10
 
     # Max number of transactions we queue in a bucket.
@@ -29,22 +29,39 @@ module FluidFeatures
       raise "app invalid : #{app}" unless app.is_a? ::FluidFeatures::App
       configure(app)
       run_loop
+      at_exit do
+        buckets_storage.append(@buckets)
+      end
+    end
+
+    def buckets_storage
+      @buckets_storage ||= FluidFeatures::Persistence::Buckets.create(FluidFeatures.config["cache"])
+    end
+
+    def features_storage
+      @features_storage ||= FluidFeatures::Persistence::Features.create(FluidFeatures.config["cache"])
     end
 
     def configure(app)
       @app = app
 
-      @buckets = []
+      @buckets = buckets_storage.fetch(MAX_BUCKETS)
+
       @buckets_lock = ::Mutex.new
 
+      #maybe could get rid of @current_bucket concept
       @current_bucket = nil
       @current_bucket_lock = ::Mutex.new
-      @current_bucket = new_bucket
+      @current_bucket = last_or_new_bucket
 
-      @unknown_features = {}
+      @unknown_features = features_storage.list_unknown
       @unknown_features_lock = ::Mutex.new
     end
 
+    def last_or_new_bucket
+      @buckets.empty? || @buckets.last.size >= MAX_BUCKET_SIZE ? new_bucket : @buckets.last
+    end
+
     # Pass FluidFeatures::AppUserTransaction for reporting
     # back to the FluidFeatures service.
     def report_transaction(transaction)
@@ -76,6 +93,7 @@ module FluidFeatures
 
       if transaction.unknown_features.size > 0
         queue_unknown_features(transaction.unknown_features)
+        features_storage.replace_unknown(@unknown_features)
       end
 
     end
@@ -176,6 +194,8 @@ module FluidFeatures
         end
         # return unknown features to queue until the next attempt at sending
         queue_unknown_features(unknown_features)
+      else
+        features_storage.replace_unknown({})
       end
 
       # return whether we were able to send or not
@@ -194,16 +214,15 @@ module FluidFeatures
     @private
     def new_bucket
       bucket = []
-      discarded_bucket = nil
       @buckets_lock.synchronize do
         @buckets << bucket
         if @buckets.size > MAX_BUCKETS
-          discarded_bucket = @buckets.shift
+          #offload to storage
+          unless buckets_storage.append_one(@buckets.shift)
+            app.logger.warn "[FF] Discarded transactions due to reporter backlog. These will not be reported to FluidFeatures."
+          end
         end
       end
-      if discarded_bucket
-        app.logger.warn "[FF] Discarded #{discarded_bucket.size} transactions due to reporter backlog. These will not be reported to FluidFeatures."
-      end
       bucket
     end
 
@@ -211,6 +230,11 @@ module FluidFeatures
     def remove_bucket
       removed_bucket = nil
       @buckets_lock.synchronize do
+        #try to get buckets from storage first
+        if @buckets.empty? && !buckets_storage.empty?
+          @buckets = buckets_storage.fetch(MAX_BUCKETS)
+        end
+
         if @buckets.size > 0
           removed_bucket = @buckets.shift
         end
@@ -231,6 +255,8 @@ module FluidFeatures
         if @buckets.size <= MAX_BUCKETS
           @buckets.unshift bucket
           success = true
+        else
+          success = buckets_storage.append_one(bucket)
         end
       end
       success

data/lib/fluidfeatures/app/state.rb

@@ -35,10 +35,14 @@ module FluidFeatures
 
     def configure(app)
       @app = app
-      @features = {}
+      @features = features_storage.list
       @features_lock = ::Mutex.new
     end
 
+    def features_storage
+      @features_storage ||= FluidFeatures::Persistence::Features.create(FluidFeatures.config["cache"])
+    end
+
     def features
       f = nil
       @features_lock.synchronize do
@@ -50,6 +54,7 @@ module FluidFeatures
     def features= f
       return unless f.is_a? Hash
       @features_lock.synchronize do
+        features_storage.replace(f) unless @features == f
         @features = f
       end
     end

data/lib/fluidfeatures/app/transaction.rb

@@ -12,7 +12,7 @@ module FluidFeatures
       @url  = url
 
       # take a snap-shot of the features end at
-      # the beginning of the transactionapplication
+      # the beginning of the transaction
       @features = user.features
 
       @features_hit = {}
@@ -23,7 +23,7 @@ module FluidFeatures
 
     end
 
-    def feature_enabled?(feature_name, version_name, default_enabled)
+    def feature_enabled?(feature_name, version_name=nil, default_enabled=nil)
       raise "transaction ended" if ended
       raise "feature_name invalid : #{feature_name}" unless feature_name.is_a? String
       version_name ||= ::FluidFeatures::DEFAULT_VERSION_NAME
@@ -73,7 +73,7 @@ module FluidFeatures
       end
     end
 
-    def goal_hit(goal_name, goal_version_name)
+    def goal_hit(goal_name, goal_version_name=nil)
       raise "transaction ended" if ended
       raise "goal_name invalid : #{goal_name}" unless goal_name.is_a? String
       goal_version_name ||= ::FluidFeatures::DEFAULT_VERSION_NAME

data/lib/fluidfeatures/config.rb

@@ -0,0 +1,24 @@
+
+require 'yaml'
+require 'erb'
+
+module FluidFeatures
+  class Config
+    attr_accessor :vars
+    def initialize(path, environment, replacements = {})
+      self.vars = YAML.load(ERB.new(File.read(path)).result)
+      self.vars = vars["common"].update(vars[environment])
+      self.vars = vars.update(replacements)
+      self.vars["cache"]["limit"] = self.class.parse_file_size(vars["cache"]["limit"]) if vars["cache"]
+    end
+
+    def [](name)
+      @vars[name.to_s]
+    end
+
+    def self.parse_file_size(size)
+      return size if !size || size.is_a?(Numeric) || !/(\d+)\s*(kb|mb|gb)$/.match(size.downcase)
+      $1.to_i * 1024 ** (%w{kb mb gb}.index($2) + 1)
+    end
+  end
+end

data/lib/fluidfeatures/persistence/buckets.rb

@@ -0,0 +1,71 @@
+module FluidFeatures
+  module Persistence
+    class Buckets < Storage
+      attr_accessor :limit
+
+      def self.create(config)
+        return NullBuckets.new unless config && config["dir"] && config["enable"] && config["limit"] > 0
+        new(config)
+      end
+
+      def initialize(config)
+        self.limit = config["limit"]
+        super config
+      end
+
+      def fetch(n = 1)
+        ret = []
+        store.transaction do
+          return [] unless store && store["buckets"] && !store["buckets"].empty?
+          n = store["buckets"].size if n > store["buckets"].size
+          ret = store["buckets"].slice!(0, n)
+          store.commit
+        end
+        return ret
+      end
+
+      def append(buckets)
+        transaction do
+          store["buckets"] += buckets
+        end
+      end
+
+      def append_one(bucket)
+        transaction do
+          store["buckets"].push(bucket)
+        end
+      end
+
+      def empty?
+        store.transaction(true) do
+          return true unless store["buckets"]
+          return store["buckets"].empty?
+        end
+      end
+
+      private
+
+      def transaction
+        begin
+          return false if file_size > limit
+          store.transaction do
+            store["buckets"] ||= []
+            yield
+          end
+        rescue Exception => _
+          return false
+        end
+      end
+    end
+
+    class NullBuckets
+      def fetch(*args); [] end
+
+      def append(*args); false end
+
+      def append_one(*args); false end
+
+      def empty?; true end
+    end
+  end
+end

data/lib/fluidfeatures/persistence/features.rb

@@ -0,0 +1,62 @@
+module FluidFeatures
+  module Persistence
+    class Features < Storage
+
+      def self.create(config)
+        return NullFeatures.new unless config && config["dir"] && config["enable"]
+        new(config)
+      end
+
+      def initialize(config)
+        super config
+      end
+
+      def list
+        store.transaction(true) do
+          return {} unless store && store["features"]
+          store["features"]
+        end
+      end
+
+      def replace(features)
+        transaction do
+          store["features"] = features
+          !!store["features"]
+        end
+      end
+
+      def list_unknown
+        store.transaction(true) do
+          return {} unless store && store["unknown_features"]
+          store["unknown_features"]
+        end
+      end
+
+      def replace_unknown(features)
+        transaction do
+          store["unknown_features"] = features
+          !!store["unknown_features"]
+        end
+      end
+
+      private
+
+      def transaction
+        begin
+          store.transaction do
+            yield
+          end
+        rescue Exception => _
+          return false
+        end
+      end
+    end
+
+    class NullFeatures
+      def list; {} end
+      def list_unknown; {} end
+      def replace(*args); false end
+      def replace_unknown(*args); false end
+    end
+  end
+end

data/lib/fluidfeatures/persistence/storage.rb

@@ -0,0 +1,32 @@
+require "pstore"
+
+module FluidFeatures
+  module Persistence
+    class Storage
+      attr_accessor :dir, :file_name, :store, :logger
+
+      def initialize(config)
+        self.dir = config["dir"]
+        self.logger = config["logger"] || Logger.new(STDERR)
+        self.file_name = "#{self.class.to_s.split('::').last.downcase}.pstore"
+        FileUtils.mkpath(dir) unless Dir.exists?(dir)
+      end
+
+      def store
+        @store ||= PStore.new(path, true)
+      end
+
+      def path
+        File.join(dir, file_name)
+      end
+
+      def file_size
+        begin
+          File.size(path)
+        rescue Exception => _
+          return 0
+        end
+      end
+    end
+  end
+end

data/lib/fluidfeatures/version.rb

@@ -1,3 +1,3 @@
 module FluidFeatures
-  VERSION = '0.4.4'
+  VERSION = '0.5.0'
 end

data/spec/app/reporter_spec.rb

@@ -16,10 +16,102 @@ describe FluidFeatures::AppReporter do
 
     before(:each) do
       described_class.any_instance.stub(:run_loop)
+      FluidFeatures.stub(:config).and_return(config)
     end
 
-    describe "#report_transaction" do
+    describe "#features_storage" do
+      before(:each) do
+        described_class.any_instance.stub(:configure)
+        FluidFeatures::Persistence::Features.should_receive(:create).with(config["cache"]).twice
+        .and_return(FluidFeatures::Persistence::NullFeatures.new)
+      end
+
+      it "should create features storage" do
+        reporter.features_storage
+      end
+
+      it "should not call create twice" do
+        reporter.features_storage
+        reporter.features_storage
+      end
+    end
+
+    describe "#buckets_storage" do
+      before(:each) do
+        described_class.any_instance.stub(:configure)
+        FluidFeatures::Persistence::Buckets.should_receive(:create).with(config["cache"])
+        .and_return(FluidFeatures::Persistence::NullBuckets.new)
+      end
+
+      it "should create buckets storage" do
+        reporter.buckets_storage
+      end
+
+      it "should not call create twice" do
+        reporter.buckets_storage
+        reporter.buckets_storage
+      end
+    end
+
+    describe "#configure" do
+      let(:features_storage) { mock("features_storage") }
+      let(:buckets_storage) { mock("buckets_storage") }
+      let(:bucket) { mock("bucket") }
+
+      before(:each) do
+        reporter.stub!(:features_storage).and_return(features_storage)
+        reporter.stub!(:buckets_storage).and_return(buckets_storage)
+        reporter.stub!(:last_or_new_bucket)
+        features_storage.stub!(:list_unknown)
+        buckets_storage.stub!(:fetch)
+      end
+
+      it "should assign result of buckets_storage.fetch to @buckets" do
+        buckets = mock("buckets")
+        buckets_storage.should_receive(:fetch).with(described_class::MAX_BUCKETS).and_return(buckets)
+        reporter.configure(app)
+        reporter.instance_variable_get(:@buckets).should == buckets
+      end
+
+      it "should assign result of last_or_new_bucket to @current_bucket" do
+        reporter.should_receive(:last_or_new_bucket).and_return(bucket)
+        reporter.configure(app)
+        reporter.instance_variable_get(:@current_bucket).should == bucket
+      end
+
+      it "should assign result of features_storage.list_unknown to @unknown_features" do
+        features = mock("features")
+        features_storage.should_receive(:list_unknown).and_return(features)
+        reporter.configure(app)
+        reporter.instance_variable_get(:@unknown_features).should == features
+      end
+    end
+
+    describe "last_or_new_bucket" do
+      it "should return new bucket if buckets empty" do
+        reporter.instance_variable_set(:@buckets, [])
+        reporter.should_receive(:new_bucket).and_return([])
+        reporter.last_or_new_bucket.should == []
+      end
+
+      it "should return new bucket if last bucket is full" do
+        bucket = mock("bucket")
+        bucket.should_receive(:size).and_return(described_class::MAX_BUCKET_SIZE + 1)
+        reporter.instance_variable_set(:@buckets, [bucket])
+        reporter.should_receive(:new_bucket).and_return([])
+        reporter.last_or_new_bucket.should == []
+      end
+
+      it "should return last bucket of buckets" do
+        bucket = mock("bucket")
+        bucket.should_receive(:size).and_return(described_class::MAX_BUCKET_SIZE - 1)
+        reporter.instance_variable_set(:@buckets, [bucket])
+        reporter.should_not_receive(:new_bucket)
+        reporter.last_or_new_bucket.should == bucket
+      end
+    end
 
+    describe "#report_transaction" do
       before(:each) do
         reporter.stub!(:queue_transaction_payload)
         reporter.stub!(:queue_unknown_features)
@@ -41,7 +133,6 @@ describe FluidFeatures::AppReporter do
         reporter.should_receive(:queue_unknown_features).with(transaction.unknown_features)
         reporter.report_transaction(transaction)
       end
-
     end
 
     describe "#transactions_queued?" do
@@ -78,6 +169,7 @@ describe FluidFeatures::AppReporter do
 
     describe "#queue_unknown_features" do
       let(:unknown_features) { { "feature" => { "a" => true } } }
+
       it "should add unknown_features to instance variable" do
         reporter.queue_unknown_features(unknown_features)
         reporter.instance_variable_get(:@unknown_features).should == unknown_features
@@ -90,11 +182,11 @@ describe FluidFeatures::AppReporter do
       it "should raise error if versions is not Hash" do
         expect { reporter.queue_unknown_features("feature" => "not cool") }.to raise_error /should be a Hash/
       end
-
     end
 
     describe "#send_transactions" do
       let(:unknown_features) { { "feature" => { "a" => true } } }
+
       let(:app) { mock "FluidFeatures::App", client: mock("client", uuid: 'client uuid'), logger: mock('logger') }
 
       before(:each) do
@@ -120,8 +212,82 @@ describe FluidFeatures::AppReporter do
         reporter.send_transactions
       end
 
+      it "should reset unknown features in storage on success" do
+        app.stub!(:post).and_return(true)
+        reporter.features_storage.should_receive(:replace_unknown).with({})
+        reporter.send_transactions
+      end
     end
 
-  end
+    describe "#new_bucket" do
+      let(:buckets) { [] }
+
+      before(:each) do
+        reporter.instance_variable_set(:@buckets, buckets)
+      end
+
+      it "should append first bucket to storage if over the limit" do
+        buckets.stub!(:size).and_return(described_class::MAX_BUCKETS + 1)
+        reporter.buckets_storage.should_receive(:append_one).with([])
+        reporter.new_bucket
+      end
+
+      it "should append first bucket to storage if" do
+        buckets.stub!(:size).and_return(described_class::MAX_BUCKETS - 1)
+        reporter.buckets_storage.should_not_receive(:append)
+        reporter.new_bucket
+      end
+    end
+
+    describe "#remove_bucket" do
+      let(:buckets) { [] }
+
+      before(:each) do
+        reporter.instance_variable_set(:@buckets, buckets)
+      end
+
+      it "should load buckets from storage if buckets empty and something in storage" do
+        buckets.stub!(:empty?).and_return(true)
+        reporter.buckets_storage.stub!(:empty?).and_return(false)
+        reporter.buckets_storage.should_receive(:fetch).and_return(buckets)
+        reporter.remove_bucket
+      end
+
+      it "should not load buckets if buckets not empty" do
+        buckets.stub!(:empty?).and_return(false)
+        reporter.buckets_storage.stub!(:empty?).and_return(false)
+        reporter.buckets_storage.should_not_receive(:fetch)
+        reporter.remove_bucket
+      end
+
+      it "should not load buckets if storage empty" do
+        buckets.stub!(:empty?).and_return(true)
+        reporter.buckets_storage.stub!(:empty?).and_return(true)
+        reporter.buckets_storage.should_not_receive(:fetch)
+        reporter.remove_bucket
+      end
+    end
+
+    describe "#unremove_bucket" do
+      let(:buckets) { [] }
 
+      let(:bucket) { mock("bucket") }
+
+      before(:each) do
+        reporter.instance_variable_set(:@buckets, buckets)
+      end
+
+      it "should offload bucket to storage if buckets over limit" do
+        buckets.stub!(:size).and_return(described_class::MAX_BUCKETS + 1)
+        reporter.buckets_storage.should_receive(:append_one).with(bucket)
+        reporter.unremove_bucket(bucket)
+      end
+
+      it "should not offload bucket to storage if buckets under limit" do
+        buckets.stub!(:size).and_return(described_class::MAX_BUCKETS - 1)
+        reporter.buckets_storage.should_not_receive(:append)
+        reporter.unremove_bucket(bucket)
+      end
+    end
+  end
 end

data/spec/app/state_spec.rb

@@ -1,4 +1,6 @@
 require 'spec_helper'
+require "fluidfeatures/persistence/buckets"
+require "fluidfeatures/persistence/features"
 
 describe FluidFeatures::AppState do
 
@@ -15,10 +17,54 @@ describe FluidFeatures::AppState do
       app.stub!(:is_a?).and_return(false)
       app.stub!(:is_a?).with(FluidFeatures::App).and_return(true)
       described_class.any_instance.stub(:run_loop)
+      FluidFeatures.stub(:config).and_return(config)
     end
 
-    describe "#load_state" do
+    describe "#features_storage" do
+      before(:each) do
+        described_class.any_instance.stub(:configure)
+        FluidFeatures::Persistence::Features.should_receive(:create).with(config["cache"])
+        .and_return(FluidFeatures::Persistence::NullFeatures.new)
+      end
+
+      it "should create features storage" do
+        state.features_storage
+      end
+
+      it "should not call create twice" do
+        state.features_storage
+        state.features_storage
+      end
+    end
+
+    describe "#configure" do
+      let(:features_storage) { mock("features_storage") }
 
+      before(:each) do
+        state.stub!(:features_storage).and_return(features_storage)
+      end
+
+      it "should assign result of features_storage.list to @features" do
+        features = mock("features")
+        features_storage.should_receive(:list).and_return(features)
+        state.configure(app)
+        state.instance_variable_get(:@features).should == features
+      end
+    end
+
+    describe "#features=" do
+      it "should replace features if there are changes" do
+        state.features_storage.should_receive(:replace).with({foo: "bar"})
+        state.features = {foo: "bar"}
+      end
+
+      it "should not replace features if not amended" do
+        state.features_storage.should_not_receive(:replace)
+        state.features = {}
+      end
+    end
+
+    describe "#load_state" do
       before(:each) do
         app.stub!(:get)
       end
@@ -34,7 +80,6 @@ describe FluidFeatures::AppState do
         }])
         state.load_state.should == [true, "feature" => { "versions" => { "a" => { "parts" => Set.new([1, 2, 3]) } } }]
       end
-
     end
 
     describe "#feature_version_enabled_for_user" do
@@ -67,9 +112,6 @@ describe FluidFeatures::AppState do
       it "should return true if stated implicitly" do
         state.feature_version_enabled_for_user("Feature", "a", 5, { "key" => "id" }).should be_true
       end
-
     end
-
   end
-
 end

data/spec/config_spec.rb

@@ -0,0 +1,53 @@
+require "spec_helper"
+
+describe FluidFeatures::Config do
+  let(:path) { "#{File.dirname(__FILE__)}/fixtures/fluidfeatures.yml" }
+
+  let(:env) { "development" }
+
+  let(:replacements) { {} }
+
+  let(:config) { FluidFeatures::Config.new(path, env, replacements) }
+
+  context "parse file size" do
+    [nil, 1024, "foo"].each do |input|
+      it "should pass #{input.class.to_s} through" do
+        described_class.parse_file_size(input).should == input
+      end
+    end
+
+    { "1Kb" => 1024, "2mb" => 2097152, "2GB" => 2147483648}.each do |i, o|
+      it "should read '#{i}' as #{o}" do
+        described_class.parse_file_size(i).should == o
+      end
+    end
+  end
+
+  %w{test development production}.each do |env_name|
+    context "with #{env_name} environment" do
+      let(:env) { env_name }
+
+      it "should load environment configuration from yml file and merge common section" do
+        config.vars.should == {
+            "baseuri" => "baseuri",
+            "cache" => { "enable" => false, "dir" => "cache_dir", "limit" => 2097152 },
+            "appid" => "#{env_name}_appid",
+            "secret" => "#{env_name}_secret"
+        }
+      end
+
+      context "and replacements" do
+        let(:replacements) { { "baseuri" => "env_baseuri", "appid" => "env_appid", "secret" => "env_secret" } }
+
+        it "should update variables with passed hash" do
+          config.vars.should == {
+              "baseuri" => "env_baseuri",
+              "cache" => { "enable" => false, "dir" => "cache_dir", "limit" => 2097152 },
+              "appid" => "env_appid",
+              "secret" => "env_secret"
+          }
+        end
+      end
+    end
+  end
+end

data/spec/fixtures/fluidfeatures.yml

@@ -0,0 +1,18 @@
+common:
+  baseuri: baseuri
+  cache:
+    enable: false
+    dir: cache_dir
+    limit: 2mb
+
+development:
+  appid: development_appid
+  secret: development_secret
+
+test:
+  appid: test_appid
+  secret: test_secret
+
+production:
+  appid: production_appid
+  secret: production_secret

data/spec/{fluidfeatures_spec.rb → integration/fluidfeatures_spec.rb}

@@ -3,26 +3,30 @@ require "spec_helper"
 describe "FluidFeatures" do
 
   describe "App initialization" do
-
     it "should raise 'host not set' without valid base uri" do
-      expect { FluidFeatures.app(nil, api_credentials[1], api_credentials[2]) }.
+      expect { FluidFeatures.app(config.remove("baseuri")) }.
         to raise_error(StandardError, /host not set/)
     end
 
     it "should raise 'app_id invalid' without valid app id" do
-      expect { FluidFeatures.app(api_credentials[0], nil, api_credentials[2]) }.
+      expect { FluidFeatures.app(config.remove("appid")) }.
         to raise_error(StandardError, /app_id invalid/)
     end
 
     it "should raise 'secret invalid' without valid secret" do
-      expect { FluidFeatures.app(api_credentials[0], api_credentials[1], nil) }.
+      expect { FluidFeatures.app(config.remove("secret")) }.
           to raise_error(StandardError, /secret invalid/)
     end
 
+    it "should set @config class variable to passed config" do
+      FluidFeatures::Client.stub!(:new); FluidFeatures::App.stub!(:new)
+      config = mock("config", "[]" => nil, "[]=" => nil)
+      FluidFeatures.app(config)
+      FluidFeatures.config.should == config
+    end
   end
 
   describe "API methods" do
-
     #let(:feature_name) { "Feature-#{UUID.new.generate}" }
     let(:feature_name) { "Feature1" }
 
@@ -47,7 +51,5 @@ describe "FluidFeatures" do
         sleep abit
       end
     end
-
   end
-
 end

data/spec/persistence/buckets_spec.rb

@@ -0,0 +1,95 @@
+require "spec_helper"
+
+describe FluidFeatures::Persistence::Buckets do
+  context "instance" do
+    let(:storage) { described_class.create(config["cache"]) }
+
+    describe "#fetch" do
+      it "should return [] for empty storage" do
+        storage.fetch.should == []
+      end
+
+      it "should return [] for empty storage when called with limit" do
+        storage.fetch(10).should == []
+      end
+
+      it "should return limited set when called with limit" do
+        storage.append([["bucket0"], ["bucket1"], ["bucket2"]])
+        storage.fetch(2).should == [["bucket0"], ["bucket1"]]
+        storage.should_not be_empty
+        storage.fetch.should == [["bucket2"]]
+        storage.should be_empty
+
+      end
+
+      it "should return limited set when called with limit over of bounds" do
+        storage.append([["bucket0"], ["bucket1"], ["bucket2"]])
+        storage.fetch(10).should == [["bucket0"], ["bucket1"], ["bucket2"]]
+        storage.should be_empty
+      end
+    end
+
+    describe "#append" do
+      it "should append buckets to empty storage" do
+        storage.append([["bucket"]]).should be_true
+        storage.fetch(10).should == [["bucket"]]
+      end
+
+      it "should append buckets to existing buckets" do
+        storage.append_one(["bucket0"])
+        storage.append([["bucket1"], ["bucket2"]]).should be_true
+        storage.fetch(10).should == [["bucket0"], ["bucket1"], ["bucket2"]]
+      end
+
+      it "should not append and return false if storage size over the limit" do
+        storage.stub!(:file_size).and_return(2)
+        storage.stub!(:limit).and_return(1)
+        storage.append([["bucket0"], ["bucket1"]]).should be_false
+        storage.fetch.should == []
+      end
+    end
+
+    describe "#append_one" do
+      it "should append bucket to empty storage" do
+        storage.append_one(["bucket"]).should be_true
+        storage.fetch.should == [["bucket"]]
+      end
+
+      it "should append bucket to existing buckets" do
+        storage.append([["bucket0"], ["bucket1"]])
+        storage.append_one(["bucket2"]).should be_true
+        storage.fetch(10).should == [["bucket0"], ["bucket1"], ["bucket2"]]
+      end
+
+      it "should not append and return false if storage size over the limit" do
+        storage.stub!(:file_size).and_return(2)
+        storage.stub!(:limit).and_return(1)
+        storage.append_one(["bucket"]).should be_false
+        storage.fetch.should == []
+      end
+    end
+
+    describe "#empty?" do
+      it "should return true for fresh storage" do
+        storage.should be_empty
+      end
+
+      it "should return false when storage has buckets" do
+        storage.append_one(["bucket"])
+        storage.should_not be_empty
+      end
+    end
+  end
+
+  context "null instance" do
+    let(:storage) { described_class.create(nil) }
+
+    specify { storage.fetch.should == [] }
+
+    specify { storage.append([["bucket0"], ["bucket1"]]).should == false }
+
+    specify { storage.append_one(["bucket"]).should == false }
+
+    specify { storage.empty?.should == true }
+  end
+end

data/spec/persistence/features_spec.rb

@@ -0,0 +1,37 @@
+require "spec_helper"
+
+describe FluidFeatures::Persistence::Features do
+  context "instance" do
+    let(:storage) { described_class.create(config["cache"]) }
+
+    it "#list should return all features" do
+      storage.replace({ feature: "foo" })
+      storage.list.should == ({ feature: "foo" })
+    end
+
+    it "#replace should replace features with passed hash" do
+      storage.replace({ feature: "foo" }).should == true
+    end
+
+    it "#list_unknown should return unknown_features features" do
+      storage.replace_unknown({ feature: "foo" })
+      storage.list_unknown.should == ({ feature: "foo" })
+    end
+
+    it "#replace_unknown should replace unknown_features with passed hash" do
+      storage.replace_unknown({ feature: "foo" }).should == true
+    end
+  end
+
+  context "null instance" do
+    let(:storage) { described_class.create(nil) }
+
+    specify { storage.list.should == {} }
+
+    specify { storage.list_unknown.should == {} }
+
+    specify { storage.replace([["bucket0"], ["bucket1"]]).should == false }
+
+    specify { storage.replace_unknown([["bucket0"], ["bucket1"]]).should == false }
+  end
+end

data/spec/spec_helper.rb

@@ -1,9 +1,8 @@
-# This file was generated by the `rspec --init` command. Conventionally, all
-# specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
-# Require this file using `require "spec_helper"` to ensure that it is only
-# loaded once.
-#
-# See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
+
+unless ENV["FLUIDFEATURES_APPID"]
+  $stderr.puts "Expect ENV variabled FLUIDFEATURES_APPID FLUIDFEATURES_SECRET FLUIDFEATURES_BASEURI to run the tests"
+  exit!
+end
 
 require 'vcr'
 require 'fluidfeatures'
@@ -17,6 +16,7 @@ VCR.configure do |c|
 end
 
 RSpec.configure do |config|
+  config.fail_fast = true
   config.treat_symbols_as_metadata_keys_with_true_values = true
   config.run_all_when_everything_filtered = true
   config.filter_run :focus
@@ -25,4 +25,9 @@ RSpec.configure do |config|
 
   config.extend VCR::RSpec::Macros
   config.include FluidFeatures::ApiHelpers
+
+  config.before(:each) do
+    dir = File.join(File.dirname(__FILE__), "tmp")
+    FileUtils.rm_rf(dir)
+  end
 end

data/spec/support/api_helpers.rb

@@ -1,11 +1,17 @@
 module FluidFeatures
   module ApiHelpers
-    def api_credentials
-      [ ENV["FLUIDFEATURES_BASEURI"], ENV["FLUIDFEATURES_APPID"], ENV["FLUIDFEATURES_SECRET"] ]
+    def config
+      {
+        "cache" => { "enable" => true, "dir" => "spec/tmp", "limit" => 1024 ** 2 },
+        "baseuri" => ENV["FLUIDFEATURES_BASEURI"],
+        "appid" => ENV["FLUIDFEATURES_APPID"],
+        "secret" => ENV["FLUIDFEATURES_SECRET"],
+        "logger" => Logger.new("/dev/null")
+      }
     end
 
     def app
-      @app ||= FluidFeatures.app(*api_credentials.push( Logger.new("/dev/null") ))
+      @app ||= FluidFeatures.app(config)
     end
 
     def transaction
@@ -16,7 +22,6 @@ module FluidFeatures
     def commit(transaction)
       transaction.end_transaction
       @transaction = transaction = nil
-      #Thread.list.map &:join
     end
 
     #time to sleep waiting for thread

data/spec/support/hash_ext.rb

@@ -0,0 +1,11 @@
+class Hash
+  def remove!(*keys)
+    keys.each{|key| self.delete(key) }
+    self
+  end
+
+  def remove(*keys)
+    self.dup.remove!(*keys)
+  end
+end
+

data/spec/support/polling_loop_shared_examples.rb

@@ -1,14 +1,13 @@
 shared_examples "polling loop" do |payload_method|
-
   let(:app) { mock "FluidFeatures::App" }
 
   before(:each) do
     app.stub!(:is_a?).and_return(false)
     app.stub!(:is_a?).with(FluidFeatures::App).and_return(true)
+    FluidFeatures.stub(:config).and_return(config)
   end
 
   context "initialization" do
-
     before(:each) do
       described_class.any_instance.stub(:configure)
       described_class.any_instance.stub(:run_loop)
@@ -38,5 +37,4 @@ shared_examples "polling loop" do |payload_method|
   it "should initialize @app" do
     described_class.new(app).app.should == app
   end
-
 end

data/spec/support/string_ext.rb

@@ -0,0 +1,12 @@
+class String
+  def const
+    names = self.split('::')
+    names.shift if names.empty? || names.first.empty?
+
+    constant = Object
+    names.each do |name|
+      constant = constant.const_defined?(name) ? constant.const_get(name) : constant.const_missing(name)
+    end
+    constant
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fluidfeatures
 version: !ruby/object:Gem::Version
-  version: 0.4.4
+  version: 0.5.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-12-05 00:00:00.000000000 Z
+date: 2013-01-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: persistent_http
@@ -163,7 +163,11 @@ files:
 - lib/fluidfeatures/app/transaction.rb
 - lib/fluidfeatures/app/user.rb
 - lib/fluidfeatures/client.rb
+- lib/fluidfeatures/config.rb
 - lib/fluidfeatures/const.rb
+- lib/fluidfeatures/persistence/buckets.rb
+- lib/fluidfeatures/persistence/features.rb
+- lib/fluidfeatures/persistence/storage.rb
 - lib/fluidfeatures/version.rb
 - lib/pre_ruby192/uri.rb
 - spec/app/feature_spec.rb
@@ -174,10 +178,16 @@ files:
 - spec/app_spec.rb
 - spec/cassettes/feature.yml
 - spec/cassettes/goal.yml
-- spec/fluidfeatures_spec.rb
+- spec/config_spec.rb
+- spec/fixtures/fluidfeatures.yml
+- spec/integration/fluidfeatures_spec.rb
+- spec/persistence/buckets_spec.rb
+- spec/persistence/features_spec.rb
 - spec/spec_helper.rb
 - spec/support/api_helpers.rb
+- spec/support/hash_ext.rb
 - spec/support/polling_loop_shared_examples.rb
+- spec/support/string_ext.rb
 homepage: https://github.com/FluidFeatures/fluidfeatures-ruby
 licenses: []
 post_install_message: