moses-vanity 1.7.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (98) hide show
  1. data/.autotest +22 -0
  2. data/.gitignore +7 -0
  3. data/.rvmrc +3 -0
  4. data/.travis.yml +13 -0
  5. data/CHANGELOG +374 -0
  6. data/Gemfile +28 -0
  7. data/MIT-LICENSE +21 -0
  8. data/README.rdoc +108 -0
  9. data/Rakefile +189 -0
  10. data/bin/vanity +16 -0
  11. data/doc/_config.yml +2 -0
  12. data/doc/_layouts/_header.html +34 -0
  13. data/doc/_layouts/page.html +47 -0
  14. data/doc/_metrics.textile +12 -0
  15. data/doc/ab_testing.textile +210 -0
  16. data/doc/configuring.textile +45 -0
  17. data/doc/contributing.textile +93 -0
  18. data/doc/credits.textile +23 -0
  19. data/doc/css/page.css +83 -0
  20. data/doc/css/print.css +43 -0
  21. data/doc/css/syntax.css +7 -0
  22. data/doc/email.textile +129 -0
  23. data/doc/experimental.textile +31 -0
  24. data/doc/faq.textile +8 -0
  25. data/doc/identity.textile +43 -0
  26. data/doc/images/ab_in_dashboard.png +0 -0
  27. data/doc/images/clear_winner.png +0 -0
  28. data/doc/images/price_options.png +0 -0
  29. data/doc/images/sidebar_test.png +0 -0
  30. data/doc/images/signup_metric.png +0 -0
  31. data/doc/images/vanity.png +0 -0
  32. data/doc/index.textile +91 -0
  33. data/doc/metrics.textile +231 -0
  34. data/doc/rails.textile +89 -0
  35. data/doc/site.js +27 -0
  36. data/generators/templates/vanity_migration.rb +53 -0
  37. data/generators/vanity_generator.rb +8 -0
  38. data/lib/generators/templates/vanity_migration.rb +53 -0
  39. data/lib/generators/vanity_generator.rb +15 -0
  40. data/lib/vanity.rb +36 -0
  41. data/lib/vanity/adapters/abstract_adapter.rb +140 -0
  42. data/lib/vanity/adapters/active_record_adapter.rb +248 -0
  43. data/lib/vanity/adapters/mock_adapter.rb +157 -0
  44. data/lib/vanity/adapters/mongodb_adapter.rb +178 -0
  45. data/lib/vanity/adapters/redis_adapter.rb +160 -0
  46. data/lib/vanity/backport.rb +26 -0
  47. data/lib/vanity/commands/list.rb +21 -0
  48. data/lib/vanity/commands/report.rb +64 -0
  49. data/lib/vanity/commands/upgrade.rb +34 -0
  50. data/lib/vanity/experiment/ab_test.rb +507 -0
  51. data/lib/vanity/experiment/base.rb +214 -0
  52. data/lib/vanity/frameworks.rb +16 -0
  53. data/lib/vanity/frameworks/rails.rb +318 -0
  54. data/lib/vanity/helpers.rb +66 -0
  55. data/lib/vanity/images/x.gif +0 -0
  56. data/lib/vanity/metric/active_record.rb +85 -0
  57. data/lib/vanity/metric/base.rb +244 -0
  58. data/lib/vanity/metric/google_analytics.rb +83 -0
  59. data/lib/vanity/metric/remote.rb +53 -0
  60. data/lib/vanity/playground.rb +396 -0
  61. data/lib/vanity/templates/_ab_test.erb +28 -0
  62. data/lib/vanity/templates/_experiment.erb +5 -0
  63. data/lib/vanity/templates/_experiments.erb +7 -0
  64. data/lib/vanity/templates/_metric.erb +14 -0
  65. data/lib/vanity/templates/_metrics.erb +13 -0
  66. data/lib/vanity/templates/_report.erb +27 -0
  67. data/lib/vanity/templates/_vanity.js.erb +20 -0
  68. data/lib/vanity/templates/flot.min.js +1 -0
  69. data/lib/vanity/templates/jquery.min.js +19 -0
  70. data/lib/vanity/templates/vanity.css +26 -0
  71. data/lib/vanity/templates/vanity.js +82 -0
  72. data/lib/vanity/version.rb +11 -0
  73. data/test/adapters/redis_adapter_test.rb +17 -0
  74. data/test/experiment/ab_test.rb +771 -0
  75. data/test/experiment/base_test.rb +150 -0
  76. data/test/experiments/age_and_zipcode.rb +19 -0
  77. data/test/experiments/metrics/cheers.rb +3 -0
  78. data/test/experiments/metrics/signups.rb +2 -0
  79. data/test/experiments/metrics/yawns.rb +3 -0
  80. data/test/experiments/null_abc.rb +5 -0
  81. data/test/metric/active_record_test.rb +277 -0
  82. data/test/metric/base_test.rb +293 -0
  83. data/test/metric/google_analytics_test.rb +104 -0
  84. data/test/metric/remote_test.rb +109 -0
  85. data/test/myapp/app/controllers/application_controller.rb +2 -0
  86. data/test/myapp/app/controllers/main_controller.rb +7 -0
  87. data/test/myapp/config/boot.rb +110 -0
  88. data/test/myapp/config/environment.rb +10 -0
  89. data/test/myapp/config/environments/production.rb +0 -0
  90. data/test/myapp/config/routes.rb +3 -0
  91. data/test/passenger_test.rb +43 -0
  92. data/test/playground_test.rb +26 -0
  93. data/test/rails_dashboard_test.rb +37 -0
  94. data/test/rails_helper_test.rb +36 -0
  95. data/test/rails_test.rb +389 -0
  96. data/test/test_helper.rb +145 -0
  97. data/vanity.gemspec +26 -0
  98. metadata +202 -0
@@ -0,0 +1,45 @@
1
+ ---
2
+ layout: page
3
+ title: Configuring the Playground
4
+ ---
5
+
6
+ Vanity will work out of the box on a default configuration. Assuming you're using Redis on localhost, post 6379, there's nothing special to do.
7
+
8
+ Database connection information is loaded from @config/vanity.yml@, based on the current environment (@RACK_ENV@ or @RAILS_ENV@). Example:
9
+
10
+ <pre>
11
+ development:
12
+ adapter: redis
13
+ connection: redis://localhost:6379/0
14
+ production:
15
+ adapter: mongodb
16
+ database: analytics
17
+ </pre>
18
+
19
+ If there's no configuration file and the application does not create a connection explicitly, Vanity will default to the Redis instance running on @localhost@ at port 6379.
20
+
21
+ The available database adapters are:
22
+
23
+ * +redis+ -- This adapter is used by default. Available options are connection and password. host, port, database (defaults to 0) options are available, but deprecated.
24
+ * +mongodb+ -- Available options are host, port, database (defaults to "vanity"), username and password.
25
+ * +active_record+ -- Uses existing ActiveRecord configuration, by you can over-ride by supplying different options. To pick different underlying adapter, set +active_record_adapter+.
26
+
27
+ You want Vanity to collect information (metrics, experiments, etc) in production, but there's no point collecting data in other environments. You can turn data collection on and off by setting @Vanity.playground.collecting@. Under Rails, collection is turned off in all environments expect production.
28
+
29
+ You may want to turn data collection on for integration tests, depending on what you're testing. Also, you may need to turn it on if your development server runs more than one process, e.g. if you're using Passenger for development and want to use the Vanity console to pick a particular alternative in an A/B test.
30
+
31
+ Available configuration options are:
32
+
33
+ |_. name |_. Is all about ... |_. Default |
34
+ | load_path | Directory containing experiment files | experiments |
35
+ | logger | This should be obvious | default/Rails |
36
+ | collecting | False if you won't want data collected | true |
37
+
38
+ When "running under Rails":rails.html, Vanity defaults to using the Rails logger, locates the load_path relative to Rails root, uses the @config/vanity.yml@ configuration file (if present) and turns collection on only in production mode.
39
+
40
+ If you run a different setup, use the playground object to configure Vanity. For example:
41
+
42
+ <pre>
43
+ Vanity.playground.load_path = "exp"
44
+ Vanity.playground.establish_connection "redis://db.example.com"
45
+ </pre>
@@ -0,0 +1,93 @@
1
+ ---
2
+ layout: page
3
+ title: Contributing
4
+ ---
5
+
6
+ <div id="toc">
7
+ # "How To Contribute":#contributing
8
+ # "Building From Source":#building
9
+ # "Documentation":#doc
10
+ # "Open Issues":#open
11
+ </div>
12
+
13
+ By all means.
14
+
15
+
16
+ h3(#contributing). How To Contribute
17
+
18
+ Pick on an "open issue":http://github.com/assaf/vanity/issues, "experimental feature":experimental.html, suggestion from the "Google Group":http://groups.google.com/group/vanity-talk, or whatever you feel like contributing.
19
+
20
+ To contribute new code/changes:
21
+ # "Fork the project":http://github.com/assaf/vanity
22
+ # Please use a topic branch to make your changes, it's easier to test them that way
23
+ # Fix, patch, enhance, document, improve, sprinkle pixie dust
24
+ # Tests. Please. Run @rake@ and if possible CI (see below)
25
+ # Send a pull request on GitHub
26
+
27
+ Bonus points for helping improve the documentation, writing some examples, and adding more test coverage.
28
+
29
+
30
+ h3(#building). Building From Source
31
+
32
+ Vanity is tested against multiple Ruby implementations, and a variety of database engines. To make life easier, we use "RVM":http://rvm.beginrescueend.com/ and "Bundler":http://gembundler.com/ to set up the test/development environment.
33
+
34
+ To test Vanity for the first time under whichever Ruby implementation you're currently using:
35
+
36
+ <pre>
37
+ $ rake test:setup
38
+ $ rake
39
+ </pre>
40
+
41
+ To test Vanity with specific database adapter:
42
+
43
+ <pre>
44
+ $ rake DB=redis
45
+ $ rake DB=mongodb
46
+ $ rake DB=mysql
47
+ </pre>
48
+
49
+
50
+ Before making a release, we run the full test suite against multiple Ruby VMs
51
+ and using multiple database adapters. Doing this on your own is easier than it
52
+ sounds:
53
+
54
+ # "Fork the project":http://github.com/assaf/vanity
55
+ # Go to "Travis CI":http://travis-ci.org/, setup a new account if you don't already have one
56
+ # "In your profile page":http://travis-ci.org/profile, tell Travis to build your fork
57
+ # @git push@ your changes into your fork and "watch Travis":http://travis-ci.org/ run the tests
58
+
59
+
60
+ To package Vanity as a gem and install on your machine:
61
+
62
+ <pre>
63
+ $ rake install
64
+ </pre>
65
+
66
+
67
+ h3(#doc). Documentation
68
+
69
+ Documentation is written in "Textile":http://redcloth.org/textile/writing-paragraph-text/, and converted to HTML using "Jekyll":http://jekyllrb.com/.
70
+
71
+ API reference is "RDoc":http://rdoc.sourceforge.net/doc/index.html, converted to HTML using "Yardoc":http://yardoc.org/.
72
+
73
+ To build and view documentation:
74
+
75
+ <pre>
76
+ $ rake docs
77
+ $ open html/index.html
78
+ </pre>
79
+
80
+ To clean up after yourself:
81
+
82
+ <pre>
83
+ $ rake clobber
84
+ </pre>
85
+
86
+
87
+ h3(#open). Open Issues
88
+
89
+ <notextile>
90
+ <table id="issues">
91
+ <tbody></tbody>
92
+ </table>
93
+ </notextile>
@@ -0,0 +1,23 @@
1
+ ---
2
+ layout: page
3
+ title: Credits
4
+ ---
5
+
6
+ Made better by:
7
+ Adam Keys
8
+ Brian Leonard
9
+ JS Boulanger
10
+ Ryan Carver
11
+ Stephen Celis
12
+
13
+ Experiment Driven Development conceived by "Nathaniel Talbott":http://blog.talbott.ws.
14
+
15
+ A/B tests, influenced by Patrick McKenzie's awesome "A/Bingo":http://www.bingocardcreator.com/abingo.
16
+
17
+ "jQuery":http://jquery.com, copyright of John Resig, released under the "MIT license":http://dev.jquery.com/browser/trunk/jquery/MIT-LICENSE.txt.
18
+
19
+ "Flot":http://code.google.com/p/flot/, copyright of IOLA and Ole Laursen, released under the "MIT license":http://code.google.com/p/flot/source/browse/trunk/LICENSE.txt.
20
+
21
+ Original code, copyright of "Assaf Arkin":http://labnotes.org, released under the "MIT license":http://github.com/assaf/vanity/blob/master/MIT-LICENSE.
22
+
23
+ Documentation available under the "Creative Commons Attribution":http://creativecommons.org/licenses/by/3.0/ license.
@@ -0,0 +1,83 @@
1
+ body {
2
+ font: 13pt "Geneva", "Lucida Sans", "Lucida Grande", "Lucida Sans Unicode", Verdana, sans-serif;
3
+ line-height: 1.5em;
4
+ margin: 2em auto;
5
+ background-color: #fff;
6
+ color: #444;
7
+ }
8
+ a:link { color: #4488f7; text-decoration: none }
9
+ a:visited { color: #4488f7; text-decoration: none }
10
+ a:hover { text-decoration: underline }
11
+ img { border: 1px solid #eee; padding: 16px; margin: 0 auto; display: block }
12
+ blockquote { padding-left: 2em; padding-right: 2em; margin-left: 0.3em; margin-right: 0; font-style: italic }
13
+ ul { list-style-type: disc; padding-left: 1.2em }
14
+ ul ul { list-style-type: disc; padding-left: 1em }
15
+ table { border-spacing: 0; width: 100%; margin: 0.3em 0 0.3em 0 }
16
+ th, td { padding: 0.3em 0.5em 0.3em 0.5em; border-bottom: 1px solid #D8D8D8; vertical-align: top; background-color: #FFF }
17
+ th, thead td { border-bottom: none; background-color: #669966; color: #fff; text-align: left }
18
+
19
+ pre, code {
20
+ font-family: "Andale Mono", "DejaVu Sans Mono", "Courier New", "Courier", monospace;
21
+ font-size: 11pt;
22
+ color: #222;
23
+ }
24
+ pre {
25
+ line-height: 1.2em;
26
+ padding: 0.6em 0 0.6em 1em;
27
+ white-space: pre-wrap;
28
+ background: #eef;
29
+ white-space: -moz-pre-wrap !important;
30
+ }
31
+ pre br {
32
+ display: none;
33
+ }
34
+
35
+ h1, h2, h3, h3, #header, #links, #sidebar {
36
+ font-family: "Gill Sans", "Trebuchet MS", "Calibri", sans-serif;
37
+ margin: 0;
38
+ color: #222;
39
+ }
40
+ h2 { font-size: 1.5em }
41
+ h3 { font-size: 1.2em; margin-top: 2.5em }
42
+ h4 { font-size: 1.0em; margin-top: 1.5em }
43
+ /*line-height: 1.25em; letter-spacing: 0.03em; color: #87C72E; */
44
+
45
+ .title { margin: 12pt 0 0 0; font-size: 48pt; font-weight: bold; letter-spacing: -2px; line-height: 48pt }
46
+ .title a { color: #000 !important }
47
+ .tagline { margin: -34pt 0 1.5em 9em; font-size: 18pt; font-weight: normal; color: #aaa; line-height: 18pt }
48
+
49
+ .footnote { font-size: 90%; margin-top: 4em }
50
+ .footnote+.footnote { margin-top: .6em }
51
+ #links { position: fixed; right: 0; top: 0; padding: .6em; color: #ccc }
52
+ #header, #content, #footer { width: 40em; margin-left: 210px }
53
+ #sidebar { float: left; width: 140px; margin: 20px }
54
+ #sidebar ul { list-style: none; text-align: right; margin: 0 0 3em 0; padding: 0 }
55
+ #sidebar ul a img { border: none; margin; 0; padding: 0 }
56
+ #sidebar ul#stats { color: #666 }
57
+ table#issues .votes { color: #ccc; float: right }
58
+ #content { padding: 1.35em 0 }
59
+ #footer { clear: both; padding: 0; margin-top: 3em; border-top: 1px solid #ccc; text-align: right }
60
+
61
+ #toc ol {
62
+ margin: 1em 0 2em 0.6em;
63
+ list-style-type: disc;
64
+ padding-left: 1em;
65
+ font-size: 0.9em;
66
+ }
67
+ #toc li {
68
+ margin: .3em 0;
69
+ }
70
+
71
+ .experimental {
72
+ font-family: "Rough Draft", "Stencil", "Tahoma", "sans-serif";
73
+ content: "Experimental";
74
+ text-transform: uppercase;
75
+ color: rgba(255,0,0,0.65);
76
+ font-size: 36pt;
77
+ position: absolute;
78
+ margin-top: -4em;
79
+ margin-left: 2em;
80
+ -webkit-transform: rotate(-15deg);
81
+ -moz-transform: rotate(-15deg);
82
+ }
83
+
@@ -0,0 +1,43 @@
1
+ body {
2
+ color: #000;
3
+ background: #fff;
4
+ width: 220mm;
5
+ margin: 0;
6
+ text-rendering: optimizeLegibility;
7
+ font-size: 14pt;
8
+ }
9
+ .title, .tagline {
10
+ display: none;
11
+ }
12
+ h1 {
13
+ page-break-before: always;
14
+ }
15
+ h1:first-child {
16
+ page-break-before: avoid;
17
+ }
18
+ h1, h2, h3 {
19
+ color: black;
20
+ }
21
+ pre, p, blockquote, table {
22
+ page-break-inside: avoid;
23
+ }
24
+ pre, code {
25
+ margin: 0;
26
+ font-size: 13pt;
27
+ }
28
+ a:link, a:visited {
29
+ background: transparent;
30
+ text-decoration: none;
31
+ color: #0044b3;
32
+ }
33
+
34
+ #links, #sidebar, #footer {
35
+ display: none;
36
+ }
37
+ #content {
38
+ color: black;
39
+ padding: 0;
40
+ border: 0;
41
+ margin-left: 0;
42
+ width: 100% !important;
43
+ }
@@ -0,0 +1,7 @@
1
+ .highlight .k, .highlight .kp { color: blue } /* keyword */
2
+ .highlight .no { color: darkblue } /* constant */
3
+ .highlight .s1 { color: green } /* string */
4
+ .highlight .n { color: black } /* identifier */
5
+ .highlight .o, .highlight .p { color: darkblue } /* = + */
6
+ .highlight .ss { color: darkblue } /* symbol */
7
+ .highlight .c1 { color: gray } /* comment */
@@ -0,0 +1,129 @@
1
+ ---
2
+ layout: page
3
+ title: Testing Emails
4
+ ---
5
+
6
+ <div id="toc">
7
+ # "Configuring ActionMailer":#config
8
+ # "Testing email subject lines":#subject
9
+ # "Testing email content":#content
10
+ </div>
11
+
12
+ h3(#config). Configuring ActionMailer
13
+
14
+ First setup Rails to send email. For example, if you are using GMail you can setup your SMTP settings like this:
15
+
16
+ <pre>
17
+ ActionMailer::Base.smtp_settings = {
18
+ :address => "smtp.gmail.com",
19
+ :port => "587",
20
+ :domain => "gmail.com",
21
+ :authentication => :plain,
22
+ :user_name => "your-email@gmail.com",
23
+ :password => "your-pass"
24
+ }
25
+ </pre>
26
+
27
+
28
+ h3(#subject). Testing email subject lines
29
+
30
+ In your @RAILS_ROOT/experiments/@ folder create an experiment file. For example:
31
+
32
+ <pre>
33
+ ab_test "Invite subject" do
34
+ description "Optimize invite subject line"
35
+ alternatives "Join now!", "You're invited to an exclusive event."
36
+ metrics :open
37
+ end
38
+ </pre>
39
+
40
+ In your @RAILS_ROOT/experiments/metrics/@ folder create a metric file for the metric you are testing. For example:
41
+
42
+ <pre>
43
+ metric "Open (Activation)" do
44
+ description "Measures how many recipients opened an email."
45
+ end
46
+ </pre>
47
+
48
+ Create an ActionMailer class, for example:
49
+
50
+ <pre>
51
+ class UserMailer < ActionMailer::Base
52
+ def invite_email(user)
53
+ use_vanity_mailer user
54
+ mail :to => user.email, :subject =>ab_test(:invite_subject)
55
+ end
56
+ end
57
+ </pre>
58
+
59
+ We set the identity of the "user" in the @use_vanity_mailer@ method. This can take a string or an object that responds to id. If it's nil then it will set it as a random number. Setting the appropriate context is important to have each user consistently get the same alternative in our experiment.
60
+
61
+ Now we need to include a tracking image in the email content. We pass in the vanity identity which we set when we called @use_vanity_mailer(user)@ and the metric we are tracking.
62
+
63
+ <pre>
64
+ <!DOCTYPE html>
65
+ <html>
66
+ <head>
67
+ <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
68
+ </head>
69
+ <body>
70
+ <h1>Hey Joseph</h1>
71
+ <p>
72
+ <%= vanity_tracking_image(Vanity.context.vanity_identity, :open, :host => "127.0.0.1:3000") %>
73
+ </p>
74
+ </body>
75
+ </html>
76
+ </pre>
77
+
78
+ Last, we have to include the @TrackingImage@ module into our @VanityController@. This is the same place that you can include the @Dashboard@.
79
+
80
+ @Vanity::Rails::TrackingImage@ will add a image method that will render a blank image.
81
+
82
+ <pre>
83
+ class VanityController < ApplicationController
84
+ include Vanity::Rails::Dashboard
85
+ include Vanity::Rails::TrackingImage
86
+ end
87
+ </pre>
88
+
89
+
90
+ h3(#subject). Testing email content
91
+
92
+ In your @RAILS_ROOT/experiments/@ folder create a new experiment file:
93
+
94
+ <pre>
95
+ ab_test "Invite text" do
96
+ description "Optimize invite text"
97
+ alternatives "A friend of yours invited you to use Vanity", "Vanity is the latest and greatest in a/b testing technology"
98
+ metrics :click
99
+ end
100
+ </pre>
101
+
102
+ In your @RAILS_ROOT/experiments/metrics/@ folder create a metric file for the metric you are testing:
103
+
104
+ <pre>
105
+ metric "Click (Acquisition)" do
106
+ description "Measures clickthough on email."
107
+ end
108
+ </pre>
109
+
110
+ A/B test your email content:
111
+
112
+ <pre>
113
+ <!DOCTYPE html>
114
+ <html>
115
+ <head>
116
+ <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
117
+ </head>
118
+ <body>
119
+ <h1>Hi!</h1>
120
+ <p>
121
+ <%= link_to ab_test(:invite_text), vanity_track_url_for(Vanity.context.vanity_identity, :click, :controller => "home", :action => "index", :host => "127.0.0.1:3000") %>
122
+ </p>
123
+ </body>
124
+ </html>
125
+ </pre>
126
+
127
+ Here we use the text from the "invite_text" experiment and then use the @vanity_track_url_for@ helper to add the identity and the metric to track into the url so that Vanity can track the click-throughs.
128
+
129
+ Remember: By default, Vanity only collects data in production mode.
@@ -0,0 +1,31 @@
1
+ ---
2
+ layout: page
3
+ title: Experimental Features
4
+ ---
5
+
6
+ <div id="toc">
7
+ # "Multi-series Metrics":#multi-series
8
+ </div>
9
+
10
+ <div class="experimental">Experimental</div>
11
+
12
+ Everything on this page is work-in-progress towards some future release. Interested in picking up any of these features and developing them further?
13
+
14
+
15
+ h3(#multi-series). Multi-series Metrics
16
+
17
+ Vanity 1.4 introduces some changes to the API to allow support for multi-series metrics. The intent is to allow more data to be captured and displayed using less metrics (i.e. charts).
18
+
19
+ First change to the @track!@ method, which now accepts a single value, array of values, or hash. Since incrementing the metric by one is quite common, this is still the default behavior when calling @track!@ with no arguments.
20
+
21
+ Next proposed change would allow the metric to define multiple columns. Those can be addressed by name or index, for example:
22
+
23
+ <pre>
24
+ metric :purchase do
25
+ columns :count, :total
26
+ end
27
+ metric(:purchase).track! p.items.length. p.total
28
+ metric(:purchase).track! :total=>p.total, :count=>p.items.length
29
+ </pre>
30
+
31
+ The metric's @values@ method will have to be changed to return an array of arrays, or new method introduced to keep the API backward compatible. And, of course, UI modified to display multiple series in a single graph.