field_test 0.2.4 → 0.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 91cef396b68f0ed43abcce42c80290ba216fe77105a947f0317925f7abb4d0fc
-  data.tar.gz: d2119a7219f278d182bf5484f959f788b03f40c129f23a58211ff8c9bfcfaeb5
+  metadata.gz: 25a516eac00c5ded25aa203d9e3feb89196c7641868e5575938bf223ee8d20d2
+  data.tar.gz: 89dc335addf645088c5251ccd7a99b0d5348857a210c520350a3d39a9244fc3f
 SHA512:
-  metadata.gz: fce4404e2f2c90215f14414bc9cd5e8c8cb3fb023e1adaea5a116e63e096ad52f25b1a74130cbebbd2d97e80c294b41289249172ca0c02ffb44a6d0ccaddcfe5
-  data.tar.gz: 8dcd8402e676d67c15552b93277007f4cc2fc5210a4c7beef72a5b0a0acd9e6c1f4c171cc65d8300e810acb5fd524d19f67eabef4da3b1cb86aa83e4d0c5c943
+  metadata.gz: 97c5900ff54996e5b31250b882300104c991e93ab4582b30e1a2ada4fb0c8f191ddbac9240941fce9d2160bf01a66ca1ed73cd3e4a5a734de99e487b9c0c3772
+  data.tar.gz: 765b5fbb1928889ff40b576b710b93ffe14adc38d6153715b89d806cbcfa46021fb4d0cc780754a1e794d3ad2ccbcb8f939dff72a148392ae95a0b968cb7d443

data/CHANGELOG.md

@@ -1,33 +1,71 @@
-## 0.2.4
+## 0.4.1 (2020-09-07)
+
+- Use `datetime` type in migration
+
+## 0.4.0 (2020-08-04)
+
+- Fixed CSRF vulnerability with non-session based authentication - [more info](https://github.com/ankane/field_test/issues/28)
+- Fixed cache key for requests
+
+## 0.3.2 (2020-04-16)
+
+- Added support for excluding IP addresses
+
+## 0.3.1 (2019-07-01)
+
+- Added `closed` and `keep_variant`
+- Added `field_test_upgrade_memberships` method
+- Fixed API controller error
+- Fixed bug where conversions were recorded after winner
+
+Security
+
+- Fixed arbitrary variants via query parameters - [more info](https://github.com/ankane/field_test/issues/17)
+
+## 0.3.0 (2019-06-02)
+
+- Added support for native apps
+- Added `cookies` option
+- Added `precision` option
+- Fixed bug in results with multiple goals
+- Fixed issue where metrics disappeared from dashboard when moving to multiple goals
+- Dropped support for Rails < 5
+
+Breaking changes
+
+- Split out participant id and type
+- Changed participant logic for emails
+
+## 0.2.4 (2019-01-03)
 
 - Fixed `PG::AmbiguousColumn` error
 
-## 0.2.3
+## 0.2.3 (2018-01-28)
 
 - Fixed participant reporting for multiple goals
 
-## 0.2.2
+## 0.2.2 (2017-05-01)
 
 - Added support for Rails 5.1
 
-## 0.2.1
+## 0.2.1 (2016-12-18)
 
 - Added support for multiple goals
 
-## 0.2.0
+## 0.2.0 (2016-12-17)
 
 - Better web UI
 - Removed `cookie:` prefix for unknown participants
 
-## 0.1.2
+## 0.1.2 (2016-12-17)
 
 - Exclude bots
 - Mailer improvements
 
-## 0.1.1
+## 0.1.1 (2016-12-15)
 
 - Added basic web UI
 
-## 0.1.0
+## 0.1.0 (2016-12-14)
 
 - First release

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2016 Andrew Kane
+Copyright (c) 2016-2019 Andrew Kane
 
 MIT License
 

data/README.md

@@ -3,11 +3,14 @@
 :maple_leaf: A/B testing for Rails
 
 - Designed for web and email
-- Comes with a [handy dashboard](https://fieldtest.dokkuapp.com/)
+- Comes with a [dashboard](https://fieldtest.dokkuapp.com/) to view results and update variants
+- Uses your database for storage
 - Seamlessly handles the transition from anonymous visitor to logged in user
 
 Uses [Bayesian statistics](https://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
 
+[![Build Status](https://travis-ci.org/ankane/field_test.svg?branch=master)](https://travis-ci.org/ankane/field_test)
+
 ## Installation
 
 Add this line to your application’s Gemfile:
@@ -19,7 +22,8 @@ gem "field_test"
 Run:
 
 ```sh
-rails g field_test:install
+rails generate field_test:install
+rails db:migrate
 ```
 
 And mount the dashboard in your `config/routes.rb`:
@@ -28,7 +32,7 @@ And mount the dashboard in your `config/routes.rb`:
 mount FieldTest::Engine, at: "field_test"
 ```
 
-Be sure to [secure the dashboard](#security) in production.
+Be sure to [secure the dashboard](#dashboard-security) in production.
 
 ![Screenshot](https://ankane.github.io/field_test/screenshot6.png)
 
@@ -45,12 +49,18 @@ experiments:
       - blue
 ```
 
-Refer to it in views, controllers, and mailers.
+Refer to it in controllers, views, and mailers.
 
 ```ruby
 button_color = field_test(:button_color)
 ```
 
+To make testing easier, you can specify a variant with query parameters
+
+```
+http://localhost:3000/?field_test[button_color]=green
+```
+
 When someone converts, record it with:
 
 ```ruby
@@ -67,24 +77,90 @@ experiments:
 
 All calls to `field_test` will now return the winner, and metrics will stop being recorded.
 
-## Features
+You can keep returning the variant for existing participants after a winner is declared:
 
-You can specify a variant with query parameters to make testing easier
+```yml
+experiments:
+  button_color:
+    winner: green
+    keep_variant: true
+```
+
+You can also close an experiment to new participants without declaring a winner while still recording metrics for existing participants:
 
+```yml
+experiments:
+  button_color:
+    closed: true
 ```
-?field_test[button_color]=green
+
+Calls to `field_test` for new participants will return the control, and they won’t be added to the experiment.
+
+You can get the list of experiments and variants for a user with:
+
+```ruby
+field_test_experiments
 ```
 
-Assign a specific variant to a user with:
+## JavaScript and Native Apps
+
+For JavaScript and native apps, add calls to your normal endpoints.
 
 ```ruby
-experiment = FieldTest::Experiment.find(:button_color)
-experiment.variant(participant, variant: "green")
+class CheckoutController < ActionController::API
+  def start
+    render json: {button_color: field_test(:button_color)}
+  end
+
+  def finish
+    field_test_converted(:button_color)
+    # ...
+  end
+end
 ```
 
-You can also change a user’s variant from the dashboard.
+For anonymous visitors in native apps, pass a `Field-Test-Visitor` header with a unique identifier.
 
-## Config
+## Participants
+
+Any model or string can be a participant in an experiment.
+
+For web requests, it uses `current_user` (if it exists) and an anonymous visitor id to determine the participant. Set your own with:
+
+```ruby
+class ApplicationController < ActionController::Base
+  def field_test_participant
+    current_company
+  end
+end
+```
+
+For mailers, it tries `@user` then `params[:user]` to determine the participant. Set your own with:
+
+```ruby
+class ApplicationMailer < ActionMailer::Base
+  def field_test_participant
+    @company
+  end
+end
+```
+
+You can also manually pass a participant with:
+
+```ruby
+field_test(:button_color, participant: company)
+```
+
+## Jobs
+
+To get variants in jobs, models, and other contexts, use:
+
+```ruby
+experiment = FieldTest::Experiment.find(:button_color)
+button_color = experiment.variant(user)
+```
+
+## Exclusions
 
 By default, bots are returned the first variant and excluded from metrics. Change this with:
 
@@ -93,6 +169,23 @@ exclude:
   bots: false
 ```
 
+Exclude certain IP addresses with:
+
+```yml
+exclude:
+  ips:
+    - 127.0.0.1
+    - 10.0.0.0/8
+```
+
+You can also use custom logic:
+
+```ruby
+field_test(:button_color, exclude: request.user_agent == "Test")
+```
+
+## Config
+
 Keep track of when experiments started and ended. Use any format `Time.parse` accepts. Variants assigned outside this window are not included in metrics.
 
 ```yml
@@ -126,6 +219,14 @@ experiments:
       - 15
 ```
 
+To help with GDPR compliance, you can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set) for anonymous visitors. Visitors with the same IP mask and user agent are grouped together.
+
+```yml
+cookies: false
+```
+
+## Dashboard Config
+
 If the dashboard gets slow, you can make it faster with:
 
 ```yml
@@ -134,12 +235,19 @@ cache: true
 
 This will use the Rails cache to speed up winning probability calculations.
 
-## Funnels
+If you need more precision, set:
+
+```yml
+precision: 1
+```
+
+## Multiple Goals
 
 You can set multiple goals for an experiment to track conversions at different parts of the funnel. First, run:
 
 ```sh
-rails g field_test:events
+rails generate field_test:events
+rails db:migrate
 ```
 
 And add to your config:
@@ -162,20 +270,30 @@ The results for all goals will appear on the dashboard.
 
 ## Analytics Platforms
 
-You can also send experiment data to analytics platforms like [Google Analytics](https://www.google.com/analytics/), [Mixpanel](https://mixpanel.com/), and [Ahoy](https://github.com/ankane/ahoy). Use:
+You may also want to send experiment data as properties to other analytics platforms like [Segment](https://segment.com), [Amplitude](https://amplitude.com), and [Ahoy](https://github.com/ankane/ahoy). Get the list of experiments and variants with:
 
 ```ruby
 field_test_experiments
 ```
 
-to get all experiments and variants for a participant and pass them as properties.
+### Ahoy
+
+You can configure Field Test to use Ahoy’s visitor token instead of creating its own:
+
+```ruby
+class ApplicationController < ActionController::Base
+  def field_test_participant
+    [ahoy.user, ahoy.visitor_token]
+  end
+end
+```
 
-## Security
+## Dashboard Security
 
 #### Devise
 
 ```ruby
-authenticate :user, -> (user) { user.admin? } do
+authenticate :user, ->(user) { user.admin? } do
   mount FieldTest::Engine, at: "field_test"
 end
 ```
@@ -189,13 +307,106 @@ ENV["FIELD_TEST_USERNAME"] = "moonrise"
 ENV["FIELD_TEST_PASSWORD"] = "kingdom"
 ```
 
-## Credits
+## Updating Variants
 
-A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
+Assign a specific variant to a user with:
+
+```ruby
+experiment = FieldTest::Experiment.find(:button_color)
+experiment.variant(participant, variant: "green")
+```
+
+You can also change a user’s variant from the dashboard.
+
+## Associations
+
+To associate models with field test memberships, use:
+
+```ruby
+class User < ApplicationRecord
+  has_many :field_test_memberships, class_name: "FieldTest::Membership", as: :participant
+end
+```
+
+Now you can do:
+
+```ruby
+user.field_test_memberships
+```
+
+## Upgrading
+
+### 0.3.0
+
+Upgrade the gem and add to `config/field_test.yml`:
+
+```yml
+legacy_participants: true
+```
+
+Also, if you use Field Test in emails, know that the default way participants are determined has changed. Restore the previous way with:
+
+```ruby
+class ApplicationMailer < ActionMailer::Base
+  def field_test_participant
+    message.to.first
+  end
+end
+```
+
+We also recommend upgrading participants when you have time.
+
+#### Upgrading Participants
+
+Field Test 0.3.0 splits the `field_test_memberships.participant` column into `participant_type` and `participant_id`.
+
+To upgrade without downtime, create a migration:
+
+```sh
+rails generate migration upgrade_field_test_participants
+```
+
+with:
+
+```ruby
+class UpgradeFieldTestParticipants < ActiveRecord::Migration[6.0]
+  def change
+    add_column :field_test_memberships, :participant_type, :string
+    add_column :field_test_memberships, :participant_id, :string
+
+    add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
+      unique: true, name: "index_field_test_memberships_on_participant_and_experiment"
+  end
+end
+```
+
+After you run it, writes will go to both the old and new sets of columns.
+
+Next, backfill data:
+
+```ruby
+FieldTest::Membership.where(participant_id: nil).find_each do |membership|
+  participant = membership.participant
+
+  if participant.include?(":")
+    participant_type, _, participant_id = participant.rpartition(":")
+    participant_type = nil if participant_type == "cookie" # legacy
+  else
+    participant_id = participant
+  end
+
+  membership.update!(
+    participant_type: participant_type,
+    participant_id: participant_id
+  )
+end
+```
+
+Finally, remove `legacy_participants: true` from the config file. Once you confirm it’s working, you can drop the `participant` column (you can rename it first just to be extra safe).
 
-## TODO
+## Credits
 
-- Code samples for analytics platforms
+A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
 
 ## History
 
@@ -209,3 +420,12 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
 - Fix bugs and [submit pull requests](https://github.com/ankane/field_test/pulls)
 - Write, clarify, or fix documentation
 - Suggest or add new features
+
+To get started with development:
+
+```sh
+git clone https://github.com/ankane/field_test.git
+cd field_test
+bundle install
+bundle exec rake test
+```