field_test 0.2.3 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: bf0f3fac61545757a6431b740be8d781bc40916b
-  data.tar.gz: 7b83659f8ebc1d6b5bbe1d642a8b376191de0b74
+SHA256:
+  metadata.gz: 96def1c6805d880a03c141a421570a3e4b9f912e9427c385b132ffae646bc74b
+  data.tar.gz: b6eb3262d522544a40db7169364ebef0f6d9f4d2f1f4c82712c91fec5a617a4f
 SHA512:
-  metadata.gz: dddab4904f52e6c186a0a39a52a9cc3dbf88bff748ddf15cd56f3b356deea6660539eff75a6794a94ad537c66ba0e73590e50cf390994b3c64ed16f9b08a9870
-  data.tar.gz: 43f4ada96d3338c83baf0e99067e96f873fd18c3888ebbd0f86154a18b352e8803a6886b327d3ca9f6cb93eb553b475d1ea3f1ae98bc86b48d3a83070927b476
+  metadata.gz: 35765af19297887bae1e807aeff9f4f741687b4c3fc961b6b91a6aa9c3060098f42bb677d1055309b2698377dd6694c24770e8ab0418fa4c675146c1661559c0
+  data.tar.gz: 786390b24efece6a99b5db602b6b0162a2b398b319f42e61c8b0db4aca021fb4b80fba65ecfa5c5d94ebb451b7b1c852c25983e72246a3aa4640dc41613963a1

data/CHANGELOG.md

@@ -1,29 +1,67 @@
-## 0.2.3
+## 0.4.0 (2020-08-04)
+
+- Fixed CSRF vulnerability with non-session based authentication
+- Fixed cache key for requests
+
+## 0.3.2 (2020-04-16)
+
+- Added support for excluding IP addresses
+
+## 0.3.1 (2019-07-01)
+
+- Added `closed` and `keep_variant`
+- Added `field_test_upgrade_memberships` method
+- Fixed API controller error
+- Fixed bug where conversions were recorded after winner
+
+Security
+
+- Fixed arbitrary variants via query parameters - see [#17](https://github.com/ankane/field_test/issues/17)
+
+## 0.3.0 (2019-06-02)
+
+- Added support for native apps
+- Added `cookies` option
+- Added `precision` option
+- Fixed bug in results with multiple goals
+- Fixed issue where metrics disappeared from dashboard when moving to multiple goals
+- Dropped support for Rails < 5
+
+Breaking changes
+
+- Split out participant id and type
+- Changed participant logic for emails
+
+## 0.2.4 (2019-01-03)
+
+- Fixed `PG::AmbiguousColumn` error
+
+## 0.2.3 (2018-01-28)
 
 - Fixed participant reporting for multiple goals
 
-## 0.2.2
+## 0.2.2 (2017-05-01)
 
 - Added support for Rails 5.1
 
-## 0.2.1
+## 0.2.1 (2016-12-18)
 
 - Added support for multiple goals
 
-## 0.2.0
+## 0.2.0 (2016-12-17)
 
 - Better web UI
 - Removed `cookie:` prefix for unknown participants
 
-## 0.1.2
+## 0.1.2 (2016-12-17)
 
 - Exclude bots
 - Mailer improvements
 
-## 0.1.1
+## 0.1.1 (2016-12-15)
 
 - Added basic web UI
 
-## 0.1.0
+## 0.1.0 (2016-12-14)
 
 - First release

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2016 Andrew Kane
+Copyright (c) 2016-2019 Andrew Kane
 
 MIT License
 

data/README.md

@@ -3,10 +3,13 @@
 :maple_leaf: A/B testing for Rails
 
 - Designed for web and email
-- Comes with a [handy dashboard](https://fieldtest.dokkuapp.com/)
+- Comes with a [dashboard](https://fieldtest.dokkuapp.com/) to view results and update variants
+- Uses your database for storage
 - Seamlessly handles the transition from anonymous visitor to logged in user
 
-Uses [Bayesian statistics](http://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
+Uses [Bayesian statistics](https://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
+
+[![Build Status](https://travis-ci.org/ankane/field_test.svg?branch=master)](https://travis-ci.org/ankane/field_test)
 
 ## Installation
 
@@ -19,7 +22,8 @@ gem "field_test"
 Run:
 
 ```sh
-rails g field_test:install
+rails generate field_test:install
+rails db:migrate
 ```
 
 And mount the dashboard in your `config/routes.rb`:
@@ -28,7 +32,7 @@ And mount the dashboard in your `config/routes.rb`:
 mount FieldTest::Engine, at: "field_test"
 ```
 
-Be sure to [secure the dashboard](#security) in production.
+Be sure to [secure the dashboard](#dashboard-security) in production.
 
 ![Screenshot](https://ankane.github.io/field_test/screenshot6.png)
 
@@ -45,12 +49,18 @@ experiments:
       - blue
 ```
 
-Refer to it in views, controllers, and mailers.
+Refer to it in controllers, views, and mailers.
 
 ```ruby
 button_color = field_test(:button_color)
 ```
 
+To make testing easier, you can specify a variant with query parameters
+
+```
+http://localhost:3000/?field_test[button_color]=green
+```
+
 When someone converts, record it with:
 
 ```ruby
@@ -67,22 +77,88 @@ experiments:
 
 All calls to `field_test` will now return the winner, and metrics will stop being recorded.
 
-## Features
+You can keep returning the variant for existing participants after a winner is declared:
 
-You can specify a variant with query parameters to make testing easier
+```yml
+experiments:
+  button_color:
+    winner: green
+    keep_variant: true
+```
 
+You can also close an experiment to new participants without declaring a winner while still recording metrics for existing participants:
+
+```yml
+experiments:
+  button_color:
+    closed: true
 ```
-?field_test[button_color]=green
+
+Calls to `field_test` for new participants will return the control, and they won’t be added to the experiment.
+
+You can get the list of experiments and variants for a user with:
+
+```ruby
+field_test_experiments
 ```
 
-Assign a specific variant to a user with:
+## JavaScript and Native Apps
+
+For JavaScript and native apps, add calls to your normal endpoints.
 
 ```ruby
-experiment = FieldTest::Experiment.find(:button_color)
-experiment.variant(participant, variant: "green")
+class CheckoutController < ActionController::API
+  def start
+    render json: {button_color: field_test(:button_color)}
+  end
+
+  def finish
+    field_test_converted(:button_color)
+    # ...
+  end
+end
 ```
 
-You can also change a user’s variant from the dashboard.
+For anonymous visitors in native apps, pass a `Field-Test-Visitor` header with a unique identifier.
+
+## Participants
+
+Any model or string can be a participant in an experiment.
+
+For web requests, it uses `current_user` (if it exists) and an anonymous visitor id to determine the participant. Set your own with:
+
+```ruby
+class ApplicationController < ActionController::Base
+  def field_test_participant
+    current_company
+  end
+end
+```
+
+For mailers, it tries `@user` then `params[:user]` to determine the participant. Set your own with:
+
+```ruby
+class ApplicationMailer < ActionMailer::Base
+  def field_test_participant
+    @company
+  end
+end
+```
+
+You can also manually pass a participant with:
+
+```ruby
+field_test(:button_color, participant: company)
+```
+
+## Jobs
+
+To get variants in jobs, models, and other contexts, use:
+
+```ruby
+experiment = FieldTest::Experiment.find(:button_color)
+button_color = experiment.variant(user)
+```
 
 ## Config
 
@@ -93,6 +169,15 @@ exclude:
   bots: false
 ```
 
+Exclude certain IP addresses with:
+
+```yml
+exclude:
+  ips:
+    - 127.0.0.1
+    - 10.0.0.0/8
+```
+
 Keep track of when experiments started and ended. Use any format `Time.parse` accepts. Variants assigned outside this window are not included in metrics.
 
 ```yml
@@ -126,6 +211,14 @@ experiments:
       - 15
 ```
 
+To help with GDPR compliance, you can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set) for anonymous visitors. Visitors with the same IP mask and user agent are grouped together.
+
+```yml
+cookies: false
+```
+
+## Dashboard Config
+
 If the dashboard gets slow, you can make it faster with:
 
 ```yml
@@ -134,12 +227,19 @@ cache: true
 
 This will use the Rails cache to speed up winning probability calculations.
 
-## Funnels
+If you need more precision, set:
+
+```yml
+precision: 1
+```
+
+## Multiple Goals
 
 You can set multiple goals for an experiment to track conversions at different parts of the funnel. First, run:
 
 ```sh
-rails g field_test:events
+rails generate field_test:events
+rails db:migrate
 ```
 
 And add to your config:
@@ -162,20 +262,30 @@ The results for all goals will appear on the dashboard.
 
 ## Analytics Platforms
 
-You can also send experiment data to analytics platforms like [Google Analytics](https://www.google.com/analytics/), [Mixpanel](https://mixpanel.com/), and [Ahoy](https://github.com/ankane/ahoy). Use:
+You may also want to send experiment data as properties to other analytics platforms like [Segment](https://segment.com), [Amplitude](https://amplitude.com), and [Ahoy](https://github.com/ankane/ahoy). Get the list of experiments and variants with:
 
 ```ruby
 field_test_experiments
 ```
 
-to get all experiments and variants for a participant and pass them as properties.
+### Ahoy
+
+You can configure Field Test to use Ahoy’s visitor token instead of creating its own:
+
+```ruby
+class ApplicationController < ActionController::Base
+  def field_test_participant
+    [ahoy.user, ahoy.visitor_token]
+  end
+end
+```
 
-## Security
+## Dashboard Security
 
 #### Devise
 
 ```ruby
-authenticate :user, -> (user) { user.admin? } do
+authenticate :user, ->(user) { user.admin? } do
   mount FieldTest::Engine, at: "field_test"
 end
 ```
@@ -189,13 +299,106 @@ ENV["FIELD_TEST_USERNAME"] = "moonrise"
 ENV["FIELD_TEST_PASSWORD"] = "kingdom"
 ```
 
-## Credits
+## Updating Variants
+
+Assign a specific variant to a user with:
+
+```ruby
+experiment = FieldTest::Experiment.find(:button_color)
+experiment.variant(participant, variant: "green")
+```
+
+You can also change a user’s variant from the dashboard.
+
+## Associations
+
+To associate models with field test memberships, use:
+
+```ruby
+class User < ApplicationRecord
+  has_many :field_test_memberships, class_name: "FieldTest::Membership", as: :participant
+end
+```
+
+Now you can do:
+
+```ruby
+user.field_test_memberships
+```
+
+## Upgrading
+
+### 0.3.0
+
+Upgrade the gem and add to `config/field_test.yml`:
+
+```yml
+legacy_participants: true
+```
+
+Also, if you use Field Test in emails, know that the default way participants are determined has changed. Restore the previous way with:
+
+```ruby
+class ApplicationMailer < ActionMailer::Base
+  def field_test_participant
+    message.to.first
+  end
+end
+```
+
+We also recommend upgrading participants when you have time.
+
+#### Upgrading Participants
+
+Field Test 0.3.0 splits the `field_test_memberships.participant` column into `participant_type` and `participant_id`.
+
+To upgrade without downtime, create a migration:
+
+```sh
+rails generate migration upgrade_field_test_participants
+```
+
+with:
 
-A huge thanks to [Evan Miller](http://www.evanmiller.org/) for deriving the Bayesian formulas.
+```ruby
+class UpgradeFieldTestParticipants < ActiveRecord::Migration[6.0]
+  def change
+    add_column :field_test_memberships, :participant_type, :string
+    add_column :field_test_memberships, :participant_id, :string
+
+    add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
+      unique: true, name: "index_field_test_memberships_on_participant_and_experiment"
+  end
+end
+```
+
+After you run it, writes will go to both the old and new sets of columns.
 
-## TODO
+Next, backfill data:
+
+```ruby
+FieldTest::Membership.where(participant_id: nil).find_each do |membership|
+  participant = membership.participant
+
+  if participant.include?(":")
+    participant_type, _, participant_id = participant.rpartition(":")
+    participant_type = nil if participant_type == "cookie" # legacy
+  else
+    participant_id = participant
+  end
+
+  membership.update!(
+    participant_type: participant_type,
+    participant_id: participant_id
+  )
+end
+```
 
-- Code samples for analytics platforms
+Finally, remove `legacy_participants: true` from the config file. Once you confirm it’s working, you can drop the `participant` column (you can rename it first just to be extra safe).
+
+## Credits
+
+A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
 
 ## History
 
@@ -209,3 +412,12 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
 - Fix bugs and [submit pull requests](https://github.com/ankane/field_test/pulls)
 - Write, clarify, or fix documentation
 - Suggest or add new features
+
+To get started with development:
+
+```sh
+git clone https://github.com/ankane/field_test.git
+cd field_test
+bundle install
+bundle exec rake test
+```