vanity 0.3.1 → 0.4.0

data/CHANGELOG

@@ -1,41 +1,56 @@
-0.3.1 (2009-11-13)
-* Changed: Redis 1.0 is now vendored into Vanity. This means one less
-dependecy ... actually two, since Redis brings with it RSpec.
+== 1.0.0 (2009-11-19)
+This release changes the way you define a new experiment.  You can use a method suitable for the type of experiment you want to define, or call the generic define method (previously: experiment method).  For example:
 
-0.3.0 (2009-11-13)
+  ab_test "My A/B test" do
+	  alternatives :a, :b
+	end
+
+The experiment method is no longer overloaded: it looks up an experiment (loading its definition if necessary), but does not define an experiment.  The ab_test method is overloaded, though this may change in the future.
+
+In addition, the ab_goal! method is now track!.  This method may be used for other tests in the future.
+
+* Added: A/B test report now showing number of participants.
+* Added: AbTest.score method accepts minimum probability (default 90), and
+* Removed: Experiment.reset! method. Destroy and save have the same effect.
+* Changed: Playground.define now requires an experiment type, ab_test is not the default any more.
+* Changed: When you run Vanity in development mode (configuration.cache_classes = false), it will reload experiments on each request. You can also Vanity.playground.reload!.
+* Changed: Fancy AJAX trickery in Rails console.
+* Changed: You can break long experiment descriptions into multiple paragraphs using two consecutive newlines.
+* Changed: AbTest confidence becomes probability; only returns choice alternative with probability equal or higher than that.
+* Changed: ab_goal! becomes track!.
+* Changed: Console becomes Dashboard, which is less confusing with rails console (script/console).
+
+== 0.3.1 (2009-11-13)
+* Changed: Redis 1.0 is now vendored into Vanity. This means one less dependecy ... actually two, since Redis brings with it RSpec.
+
+== 0.3.0 (2009-11-13)
 * Added: score now includes least performing alternatives, names and values.
 * Added: shiny reports.
-* Added: Rails console shows current experiments status and also allows you to
-choose which alternative you want to see.
+* Added: Rails console shows current experiments status and also allows you to choose which alternative you want to see.
 * Changed: letters instead of numbers for options (option 1 => option A).
 * Changed: experiment.alternatives is now an immutable snapshot.
-* Changed: experiment.score returns populated alternative objects instead of
-structs.
-* Changed: experiment.chooses uses Redis to store state, better for (when we
-get to) browser integration.
+* Changed: experiment.score returns populated alternative objects instead of structs.
+* Changed: experiment.chooses uses Redis to store state, better for (when we get to) browser integration.
 * Changed: experiment.chooses skips recording participant or conversion.
 * Changed: to MIT license.
 
-0.2.2 (2009-11-12)
+== 0.2.2 (2009-11-12)
 * Added: vanity binary, with single command for generating a report.
 * Added: return alternative by value from experiment.alternative(val) method.
 * Added: reset an experiment by calling reset!.
 * Added: experiment alternative name (option 1, option 2, etc).
-* Added: new scoring algorithm: use experiment.score instead of
-alternative.z_score/confidence.
+* Added: new scoring algorithm: use experiment.score instead of alternative.z_score/confidence.
 * Added: experiment.conclusion for plain English results.
 
-0.2.1 (2009-11-11)
+== 0.2.1 (2009-11-11)
 * Added: z-score and confidence level for A/B test alternatives.
 * Added: test auto-completion and auto-outcome (complete_it, outcome_is).
-* Changed: default alternatives are now false/true, so if can't decide
-outcome, fall back on false.
+* Changed: default alternatives are now false/true, so if can't decide outcome, fall back on false.
 
-0.2.0 (2009-11-10)
+== 0.2.0 (2009-11-10)
 * Added: experiment method on object, used to define and access experiments.
 * Added: playground configuration (Vanity.playground.namespace = , etc).
 * Added: use_vanity now accepts block instead of symbol.
 * Changed: Vanity::Helpers becomes Vanity::Rails.
-* Changed: A/B test experiments alternatives now handled using Alternatives
-object.
+* Changed: A/B test experiments alternatives now handled using Alternatives object.
 * Removed: A/B test measure method no longer in use.

data/README.rdoc

@@ -1,36 +1,42 @@
 Vanity is an Experiment Driven Development framework for Rails.
 
-http://farm3.static.flickr.com/2540/4099665871_497f274f68_o.jpg
+* Documentation: http://vanity.labnotes.org
+* On github: http://github.com/assaf/vanity
+* Vanity requires Ruby 1.9.1 or later, Redis 1.0 or later.
 
-Requires Ruby 1.9.1 or later, Redis 1.0 or later.
+http://farm3.static.flickr.com/2540/4099665871_497f274f68_o.jpg
 
 
 == A/B Testing with Rails (in 5 easy steps)
 
-Add Vanity to your Rails app:
+<b>Step 1:</b> Start using Vanity in your Rails application:
+
+  gem.config "vanity"
+
+And:
 
   class ApplicationController < ActionController::Base
     use_vanity :current_user
   end
 
-Define an A/B test.  This test compares three pricing options:
+<b>Step 2:</b> Define your first A/B test.  This experiment goes in the file <code>experiments/price_options.rb</code>:
 
-  experiment "Price options" do
-    description "Mirror, mirror on the wall, who's the better price of them all?"
+  ab_test "Price options" do
+    description "Mirror, mirror on the wall, who's the better price of all?"
     alternatives 19, 25, 29
   end
 
-Present different options to the user:
+<b>Step 3:</b> Present the different options to your users:
 
-  <h2>Get started for only $<%= ab_test :pricing_options %> a month!</h2>
+  <h2>Get started for only $<%= ab_test :price_options %> a month!</h2>
 
-Measure conversion:
+<b>Step 4:</b> Measure conversion:
 
   class SignupController < ApplicationController
     def signup
       @account = Account.new(params[:account])
       if @account.save
-        ab_goal! :pricing_options # <- conversion
+        track! :pricing_options # <- here be conversion!
         redirect_to @acccount
       else
         render action: :offer
@@ -38,15 +44,13 @@ Measure conversion:
     end
   end
 
-Check the report:
+<b>Step 5:</b> Check the report:
 
   vanity --output vanity.html
 
-Learn more about Vanity: http://assaf.github.com/vanity
-
 
-== Credits
+== Credits/License
 
-Experiment Driven Development: Nathaniel Talbott (http://blog.talbott.ws).
+Idea behind Experiment Driven Development: Nathaniel Talbott (http://blog.talbott.ws).
 
 Copyright (C) 2009 Assaf Arkin, released under the MIT license.

data/lib/vanity/commands/report.rb

@@ -3,10 +3,11 @@ require "cgi"
 
 module Vanity
   
-  # Render method available in templates (when running outside Rails).
+  # Render method available to templates (when used by Vanity command line,
+  # outside Rails).
   module Render
     
-    # Render the named template.  Used for reporting and the console.
+    # Render the named template.  Used for reporting and the dashboard.
     def render(path, locals = {})
       locals[:playground] = self
       keys = locals.keys
@@ -18,14 +19,20 @@ module Vanity
       ERB.new(path, nil, '<').result(locals.instance_eval { binding })
     end
 
+    # Escape HTML.
+    def h(html)
+      CGI.escape_html(html)
+    end
+
   end
 
+  # Commands available when running Vanity from the command line (see bin/vanity).
   module Commands
     class << self
       include Render
 
-      # Generate a report with all available tests.  Outputs to the named file,
-      # or stdout with no arguments.
+      # Generate an HTML report.  Outputs to the named file, or stdout with no
+      # arguments.
       def report(output = nil)
         html = render(Vanity.template("report"))
         if output

data/lib/vanity/experiment/ab_test.rb

@@ -1,10 +1,10 @@
 module Vanity
   module Experiment
 
-    # Experiment alternative.  See AbTest#alternatives and AbTest#score.
+    # One of several alternatives in an A/B test (see AbTest#alternatives).
     class Alternative
 
-      def initialize(experiment, id, value, participants, converted, conversions) #:nodoc:
+      def initialize(experiment, id, value, participants, converted, conversions)
         @experiment = experiment
         @id = id
         @name = "option #{(@id + 65).chr}"
@@ -27,39 +27,45 @@ module Vanity
       # Number of participants who viewed this alternative.
       attr_reader :participants
 
-      # Number of participants who converted on this alternative.
+      # Number of participants who converted on this alternative (a participant is counted only once).
       attr_reader :converted
 
       # Number of conversions for this alternative (same participant may be counted more than once).
       attr_reader :conversions
 
-      # Z-score for this alternative. Populated by AbTest#score.
+      # Z-score for this alternative, related to 2nd-best performing alternative. Populated by AbTest#score.
       attr_accessor :z_score
 
-      # Confidence derived from z-score. Populated by AbTest#score.
-      attr_accessor :confidence
+      # Probability derived from z-score. Populated by AbTest#score.
+      attr_accessor :probability
     
       # Difference from least performing alternative. Populated by AbTest#score.
       attr_accessor :difference
 
       # Conversion rate calculated as converted/participants, rounded to 3 places.
       def conversion_rate
-        @rate ||= (participants > 0 ? (converted.to_f/participants.to_f).round(3) : 0.0)
+        @conversion_rate ||= (participants > 0 ? (converted.to_f/participants.to_f).round(3) : 0.0)
       end
 
-      def <=>(other) # sort by conversion rate
-        conversion_rate <=> other.conversion_rate 
+      # The measure we use to order (sort) alternatives and decide which one is better (by calculating z-score).
+      # Defaults to conversion rate.
+      def measure
+        conversion_rate
+      end
+
+      def <=>(other)
+        measure <=> other.measure 
       end
 
       def ==(other)
         other && id == other.id && experiment == other.experiment
       end
 
-      def to_s #:nodoc:
+      def to_s
         name
       end
 
-      def inspect #:nodoc:
+      def inspect
         "#{name}: #{value} #{converted}/#{participants}"
       end
 
@@ -70,10 +76,11 @@ module Vanity
     class AbTest < Base
       class << self
 
-        def confidence(score) #:nodoc:
+        # Convert z-score to probability.
+        def probability(score)
           score = score.abs
-          confidence = AbTest::Z_TO_CONFIDENCE.find { |z,p| score >= z }
-          confidence ? confidence.last : 0
+          probability = AbTest::Z_TO_PROBABILITY.find { |z,p| score >= z }
+          probability ? probability.last : 0
         end
 
         def friendly_name
@@ -82,7 +89,7 @@ module Vanity
 
       end
 
-      def initialize(*args) #:nodoc:
+      def initialize(*args)
         super
         @alternatives = [false, true]
       end
@@ -91,7 +98,7 @@ module Vanity
 
       # Call this method once to set alternative values for this experiment.
       # Require at least two values.  For example:
-      #   experiment "Background color" do
+      #   ab_test "Background color" do
       #     alternatives "red", "blue", "orange"
       #   end
       # 
@@ -99,18 +106,18 @@ module Vanity
       #   alts = experiment(:background_color).alternatives
       #   puts "#{alts.count} alternatives, with the colors: #{alts.map(&:value).join(", ")}"
       #
-      # If you want to know how well each alternative is faring, use #score.
+      # If you want to know how well each alternative is doing, use #score.
       def alternatives(*args)
         unless args.empty?
           @alternatives = args.clone
         end
         class << self
-          alias :alternatives :_alternatives
+          define_method :alternatives, instance_method(:_alternatives)
         end
         alternatives
       end
 
-      def _alternatives #:nodoc:
+      def _alternatives
         alts = []
         @alternatives.each_with_index do |value, i|
           participants = redis.scard(key("alts:#{i}:participants")).to_i
@@ -120,34 +127,44 @@ module Vanity
         end
         alts
       end
+      private :_alternatives
 
-      # Returns an Alternative with the specified value.
+      # Returns an Alternative with the specified value.  For example, given:
+      #   ab_test "Which color" do
+      #     alternatives :red, :green, :blue
+      #   end
+      # Then:
+      #   alternative(:red) == alternatives[0]
+      #   alternative(:blue) == alternatives[2]
       def alternative(value)
-        if index = @alternatives.index(value)
-          participants = redis.scard(key("alts:#{index}:participants")).to_i
-          converted = redis.scard(key("alts:#{index}:converted")).to_i
-          conversions = redis[key("alts:#{index}:conversions")].to_i
-          Alternative.new(self, index, value, participants, converted, conversions)
-        end
+        alternatives.find { |alt| alt.value == value }
       end
 
-      # Sets this test to two alternatives: false and true.
+      # Defines an A/B test with two alternatives: false and true.  For example:
+      #   ab_test "More bacon" do
+      #     false_true
+      #   end
+      #
+      # This is the default pair of alternatives, so just syntactic sugar for
+      # those who love being explicit.
       def false_true
         alternatives false, true
       end
       alias true_false false_true
 
-      # Chooses a value for this experiment.
+      # Chooses a value for this experiment.  You probably want to use the
+      # Rails helper method ab_test instead.
       #
-      # This method returns different values for different identity (see
-      # #identify), and consistenly the same value for the same
-      # expriment/identity pair.
+      # This method picks an alternative for the current identity and returns
+      # the alternative's value.  It will consistently choose the same
+      # alternative for the same identity, and randomly split alternatives
+      # between different identities.
       #
       # For example:
       #   color = experiment(:which_blue).choose
       def choose
         if active?
-          identity = identify
+          identity = identity()
           index = redis[key("participant:#{identity}:show")]
           unless index
             index = alternative_for(identity)
@@ -160,13 +177,14 @@ module Vanity
         @alternatives[index.to_i]
       end
 
-      # Records a conversion.
-      #
+      # Tracks a conversion.  You probably want to use the Rails helper method
+      # track! instead.
+      # 
       # For example:
-      #   experiment(:which_blue).conversion!
-      def conversion!
+      #   experiment(:which_blue).track!
+      def track!
         return unless active?
-        identity = identify
+        identity = identity()
         return if redis[key("participants:#{identity}:show")]
         index = alternative_for(identity)
         if redis.sismember(key("alts:#{index}:participants"), identity)
@@ -179,9 +197,9 @@ module Vanity
       
       # -- Testing --
      
-      # Forces this experiment to use a particular alternative. Useful for
-      # tests, e.g.
-      #
+      # Forces this experiment to use a particular alternative.  You'll want to
+      # use this from your test cases to test for the different alternatives.
+      # For example:
       #   setup do
       #     experiment(:green_button).select(true)
       #   end
@@ -195,102 +213,102 @@ module Vanity
       #     experiment(:green_button).select(nil)
       #   end
       def chooses(value)
-        index = @alternatives.index(value)
-        raise ArgumentError, "No alternative #{value.inspect} for #{name}" unless index
-        identity = identify
-        redis[key("participant:#{identity}:show")] = index
+        if value.nil?
+          redis.del key("participant:#{identity}:show")
+        else
+          index = @alternatives.index(value)
+          raise ArgumentError, "No alternative #{value.inspect} for #{name}" unless index
+          redis[key("participant:#{identity}:show")] = index
+        end
         self
       end
 
-      def chosen?(alternative) #:nodoc:
-        identity = identify
+      # True if this alternative is currently showing (see #chooses).
+      def showing?(alternative)
+        identity = identity()
         index = redis[key("participant:#{identity}:show")]
         index && index.to_i == alternative.id
       end
 
-      # Used for testing.
-      def count(identity, value, *what) #:nodoc:
-        index = @alternatives.index(value)
-        raise ArgumentError, "No alternative #{value.inspect} for #{name}" unless index
-        if what.empty? || what.include?(:participant)
-          redis.sadd key("alts:#{index}:participants"), identity
-        end
-        if what.empty? || what.include?(:conversion)
-          redis.sadd key("alts:#{index}:converted"), identity
-          redis.incr key("alts:#{index}:conversions")
-        end
-        self
-      end
-
 
       # -- Reporting --
 
-      # Returns an object with the following methods:
-      # [:alts]   List of Alternative populated with interesting statistics.
-      # [:best]   Best performing alternative.
+      # Scores alternatives based on the current tracking data.  This method
+      # returns a structure with the following attributes:
+      # [:alts]   Ordered list of alternatives, populated with scoring info.
       # [:base]   Second best performing alternative.
       # [:least]  Least performing alternative (but more than zero conversion).
-      # [:choice] Choice alterntive, either the outcome or best alternative (if confidence >= 90%).
+      # [:choice] Choice alterntive, either the outcome or best alternative.
       #
-      # Alternatives returned by this method are populated with the following attributes:
-      # [:z_score]    Z-score (relative to the base alternative).
-      # [:confidence] Confidence (z-score mapped to 0, 90, 95, 99 or 99.9%).
-      # [:difference] Difference from the least performant altenative.
-      def score
+      # Alternatives returned by this method are populated with the following
+      # attributes:
+      # [:z_score]      Z-score (relative to the base alternative).
+      # [:probability]  Probability (z-score mapped to 0, 90, 95, 99 or 99.9%).
+      # [:difference]   Difference from the least performant altenative.
+      # 
+      # The choice alternative is set only if its probability is higher or
+      # equal to the specified probability (default is 90%).
+      def score(probability = 90)
         alts = alternatives
         # sort by conversion rate to find second best and 2nd best
-        sorted = alts.sort_by(&:conversion_rate)
+        sorted = alts.sort_by(&:measure)
         base = sorted[-2]
         # calculate z-score
-        pc = base.conversion_rate
+        pc = base.measure
         nc = base.participants
         alts.each do |alt|
-          p = alt.conversion_rate
+          p = alt.measure
           n = alt.participants
           alt.z_score = (p - pc) / ((p * (1-p)/n) + (pc * (1-pc)/nc)).abs ** 0.5
-          alt.confidence = AbTest.confidence(alt.z_score)
+          alt.probability = AbTest.probability(alt.z_score)
         end
         # difference is measured from least performant
-        if least = sorted.find { |alt| alt.conversion_rate > 0 }
+        if least = sorted.find { |alt| alt.measure > 0 }
           alts.each do |alt|
-            if alt.conversion_rate > least.conversion_rate
-              alt.difference = (alt.conversion_rate - least.conversion_rate) / least.conversion_rate * 100
+            if alt.measure > least.measure
+              alt.difference = (alt.measure - least.measure) / least.measure * 100
             end
           end
         end
         # best alternative is one with highest conversion rate (best shot).
-        # choice alternative can only pick best if we have high confidence (>90%).
-        best = sorted.last if sorted.last.conversion_rate > 0.0
-        choice = outcome ? alts[outcome.id] : (best && best.confidence >= 90 ? best : nil)
+        # choice alternative can only pick best if we have high probability (>90%).
+        best = sorted.last if sorted.last.measure > 0.0
+        choice = outcome ? alts[outcome.id] : (best && best.probability >= probability ? best : nil)
         Struct.new(:alts, :best, :base, :least, :choice).new(alts, best, base, least, choice)
       end
 
-      # Use the score returned by #score to derive a conclusion.  Returns an
+      # Use the result of #score to derive a conclusion.  Returns an
       # array of claims.
       def conclusion(score = score)
         claims = []
+        participants = score.alts.inject(0) { |t,alt| t + alt.participants }
+        claims << case participants
+          when 0 ; "There are no participants in this experiment yet."
+          when 1 ; "There is one participant in this experiment."
+          else ; "There are #{participants} participants in this experiment."
+        end
         # only interested in sorted alternatives with conversion
-        sorted = score.alts.select { |alt| alt.conversion_rate > 0.0 }.sort_by(&:conversion_rate).reverse
+        sorted = score.alts.select { |alt| alt.measure > 0.0 }.sort_by(&:measure).reverse
         if sorted.size > 1
           # start with alternatives that have conversion, from best to worst,
           # then alternatives with no conversion.
           sorted |= score.alts
           # we want a result that's clearly better than 2nd best.
           best, second = sorted[0], sorted[1]
-          if best.conversion_rate > second.conversion_rate
-            diff = ((best.conversion_rate - second.conversion_rate) / second.conversion_rate * 100).round
+          if best.measure > second.measure
+            diff = ((best.measure - second.measure) / second.measure * 100).round
             better = " (%d%% better than %s)" % [diff, second.name] if diff > 0
-            claims << "The best choice is %s: it converted at %.1f%%%s." % [best.name, best.conversion_rate * 100, better]
-            if best.confidence >= 90
-              claims << "With %d%% probability this result is statistically significant." % score.best.confidence
+            claims << "The best choice is %s: it converted at %.1f%%%s." % [best.name, best.measure * 100, better]
+            if best.probability >= 90
+              claims << "With %d%% probability this result is statistically significant." % score.best.probability
             else
               claims << "This result is not statistically significant, suggest you continue this experiment."
             end
             sorted.delete best
           end
           sorted.each do |alt|
-            if alt.conversion_rate > 0.0
-              claims << "%s converted at %.1f%%." % [alt.name.gsub(/^o/, "O"), alt.conversion_rate * 100]
+            if alt.measure > 0.0
+              claims << "%s converted at %.1f%%." % [alt.name.gsub(/^o/, "O"), alt.measure * 100]
             else
               claims << "%s did not convert." % alt.name.gsub(/^o/, "O")
             end
@@ -306,16 +324,17 @@ module Vanity
       # -- Completion --
 
       # Defines how the experiment can choose the optimal outcome on completion.
-      #
-      # The default implementation looks for the best (highest conversion rate)
-      # alternative.  If it's certain (95% or more) that this alternative is
-      # better than the first alternative, it switches to that one.  If it has
-      # no such certainty, it starts using the first alternative exclusively.
+     
+      # By default, Vanity will take the best alternative (highest conversion
+      # rate) and use that as the outcome.  You experiment may have different
+      # needs, maybe you want the least performing alternative, or factor cost
+      # in the equation?
       #
       # The default implementation reads like this:
       #   outcome_is do
-      #     highest = alternatives.sort.last
-      #     highest.confidence >= 95 ? highest ? alternatives.first
+      #     a, b = alternatives
+      #     # a is expensive, only choose a if it performs 2x better than b
+      #     a.measure > b.measure * 2 ? a : b
       #   end
       def outcome_is(&block)
         raise ArgumentError, "Missing block" unless block
@@ -323,7 +342,7 @@ module Vanity
         @outcome_is = block
       end
 
-      # Alternative chosen when this experiment was completed.
+      # Alternative chosen when this experiment completed.
       def outcome
         outcome = redis[key("outcome")]
         outcome && alternatives[outcome.to_i]
@@ -350,12 +369,7 @@ module Vanity
       
       # -- Store/validate --
 
-      def save
-        fail "Experiment #{name} needs at least two alternatives" unless alternatives.count >= 2
-        super
-      end
-
-      def reset!
+      def destroy
         @alternatives.count.times do |i|
           redis.del key("alts:#{i}:participants")
           redis.del key("alts:#{i}:converted")
@@ -365,12 +379,12 @@ module Vanity
         super
       end
 
-      def destroy
-        reset
+      def save
+        fail "Experiment #{name} needs at least two alternatives" unless alternatives.count >= 2
         super
       end
 
-    private
+    protected
 
       # Chooses an alternative for the identity and returns its index. This
       # method always returns the same alternative for a given experiment and
@@ -380,14 +394,41 @@ module Vanity
         Digest::MD5.hexdigest("#{name}/#{identity}").to_i(17) % @alternatives.count
       end
 
+      # Used for testing Vanity.
+      def count_participant(identity, value)
+        index = @alternatives.index(value)
+        raise ArgumentError, "No alternative #{value.inspect} for #{name}" unless index
+        redis.sadd key("alts:#{index}:participants"), identity
+        self
+      end
+
+      # Used for testing Vanity.
+      def count_conversion(identity, value)
+        index = @alternatives.index(value)
+        raise ArgumentError, "No alternative #{value.inspect} for #{name}" unless index
+        redis.sadd key("alts:#{index}:converted"), identity
+        redis.incr key("alts:#{index}:conversions")
+        self
+      end
+
       begin
-        a = 0
+        a = 50.0
         # Returns array of [z-score, percentage]
-        norm_dist = (-5.0..3.1).step(0.01).map { |x| [x, a += 1 / Math.sqrt(2 * Math::PI) * Math::E ** (-x ** 2 / 2)] }
+        norm_dist = (0.0..3.1).step(0.01).map { |x| [x, a += 1 / Math.sqrt(2 * Math::PI) * Math::E ** (-x ** 2 / 2)] }
         # We're really only interested in 90%, 95%, 99% and 99.9%.
-        Z_TO_CONFIDENCE = [90, 95, 99, 99.9].map { |pct| [norm_dist.find { |x,a| a >= pct }.first, pct] }.reverse
+        Z_TO_PROBABILITY = [90, 95, 99, 99.9].map { |pct| [norm_dist.find { |x,a| a >= pct }.first, pct] }.reverse
       end
 
     end
   end
+
+  module Definition
+    # Define an A/B test with the given name.  For example:
+    #   ab_test "New Banner" do
+    #     alternatives :red, :green, :blue
+    #   end
+    def ab_test(name, &block)
+      define name, :ab_test, &block
+    end
+  end
 end