coverband 2.0.0.alpha1 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/README.md +286 -156
- data/Rakefile +5 -0
- data/changes.md +33 -2
- data/coverband.gemspec +6 -3
- data/docs/coverband_install.gif +0 -0
- data/lib/coverband.rb +1 -2
- data/lib/coverband/adapters/base.rb +31 -0
- data/lib/coverband/adapters/file_store.rb +1 -1
- data/lib/coverband/adapters/memory_cache_store.rb +5 -1
- data/lib/coverband/adapters/redis_store.rb +2 -2
- data/lib/coverband/collectors/base.rb +9 -23
- data/lib/coverband/collectors/coverage.rb +4 -35
- data/lib/coverband/collectors/trace.rb +21 -9
- data/lib/coverband/configuration.rb +17 -12
- data/lib/coverband/s3_web.rb +7 -1
- data/lib/coverband/version.rb +1 -1
- data/test/benchmarks/benchmark.rake +93 -55
- data/test/test_helper.rb +5 -3
- data/test/unit/baseline_test.rb +5 -7
- data/test/unit/collectors_base_test.rb +8 -2
- data/test/unit/collectors_coverage_test.rb +1 -1
- data/test/unit/collectors_trace_test.rb +8 -0
- data/test/unit/configuration_test.rb +0 -4
- data/test/unit/middleware_test.rb +10 -40
- data/test/unit/reports_base_test.rb +26 -0
- data/test/unit/s3_report_writer_test.rb +0 -2
- data/test/unit/s3_web_test.rb +1 -0
- metadata +20 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 104227eaef7c235ff16b13a0f80c5f29fe4117deac05acbed7553e3343be7ea9
|
4
|
+
data.tar.gz: dd5a621f881e955c52c1a2acf903f3bc5abd550bc015650851cb385ff76bf758
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 7a2c2041a887b5ac729a540489b20e48fba117e2d70e827f3b7c5fb6da21382fa6786a093e915232771abc7a0903f78b3484713e39394fdf3bbf6cda6f8f57c9
|
7
|
+
data.tar.gz: 38ed9c82a8a607d349f4edef901d7d348b128c9ff2f6ad6276df388f29ee25e98c6e9534f3e0d4d16d1902f958575f72aa6676c8499d81f6a44b72eaaef534fc
|
data/README.md
CHANGED
@@ -1,29 +1,33 @@
|
|
1
1
|
# Coverband
|
2
2
|
|
3
|
-
[![Build Status](https://travis-ci.org/danmayer/coverband.svg?branch=master)](https://travis-ci.org/danmayer/coverband)
|
3
|
+
Build Status: [![Build Status](https://travis-ci.org/danmayer/coverband.svg?branch=master)](https://travis-ci.org/danmayer/coverband)
|
4
4
|
|
5
|
-
|
5
|
+
<p align="center">
|
6
|
+
<a href="#key-features">Key Features</a> •
|
7
|
+
<a href="#how-to-use">How To Use</a> •
|
8
|
+
<a href="#installation">Installation</a> •
|
9
|
+
<a href="#configuration">Configuration</a> •
|
10
|
+
<a href="#usage">Usage</a> •
|
11
|
+
<a href="#license">License</a>
|
12
|
+
</p>
|
6
13
|
|
7
|
-
|
8
|
-
* Ignore directories to avoid overhead data collection on vendor, lib, etc.
|
9
|
-
* Take a baseline to get initial app execution during app initialization. (the baseline is important because some code is executed during app load, but might not be invoked during any requests, think prefetching, initial cache builds, setting constants, etc...)
|
10
|
-
* Development mode for additional code usage details (number of LOC execution during single request, etc).
|
11
|
-
* Coverband is not intended for test code coverage, for that just check out [SimpleCov](https://github.com/colszowka/simplecov).
|
12
|
-
|
13
|
-
__Notes:__ Latest versions of Coverband drop support for anything less than Ruby 2.0, and Ruby 2.1+ is recommended.
|
14
|
+
A gem to measure production code usage, showing each line of code that is executed. Coverband allows easy configuration to collect and report on production code usage. It can be used as Rack middleware, wrapping a block with sampling, or manually configured to meet any need (like usage during background jobs).
|
14
15
|
|
15
|
-
|
16
|
+
Note: Coverband is not intended for test code coverage, for that just check out [SimpleCov](https://github.com/colszowka/simplecov).
|
16
17
|
|
17
|
-
|
18
|
+
## Key Features
|
18
19
|
|
19
|
-
|
20
|
+
* Allows sampling to avoid the performance overhead on every request.
|
21
|
+
* Ignore directories to avoid overhead data collection on vendor, lib, etc.
|
22
|
+
* Take a baseline to get initial app execution during app initialization. (the baseline is important because some code is executed during app load, but might not be invoked during any requests, think prefetching, initial cache builds, setting constants, etc...)
|
23
|
+
* Development mode for additional code usage details (number of LOC execution during single request, etc).
|
20
24
|
|
21
|
-
|
25
|
+
# How To Use
|
22
26
|
|
23
|
-
|
27
|
+
Below is my Coverband workflow, which hopefully will help other best use this library.
|
24
28
|
|
25
29
|
* install coverband.
|
26
|
-
* take baseline: `rake coverband:baseline` (__note: never run baseline on production__)
|
30
|
+
* take baseline measurment: `rake coverband:baseline` (__note: never run baseline on production__)
|
27
31
|
* validate baseline with `rake coverband:coverage`
|
28
32
|
* test setup in development (hit endpoints and generate new report)
|
29
33
|
* deploy to staging and verify functionality
|
@@ -38,7 +42,29 @@ The performance impact on Ruby 2.1+ is fairly small and no longer requires a C-e
|
|
38
42
|
* there is a tradeoff on accuracy and data value
|
39
43
|
* I recommend clearing after major code changes, significant releases, or some regular schedule.
|
40
44
|
|
41
|
-
##
|
45
|
+
## Example Output
|
46
|
+
|
47
|
+
Since Coverband is [Simplecov](https://github.com/colszowka/simplecov) output compatible it should work with any of the `SimpleCov::Formatter`'s available. The output below is produced using the default Simplecov HTML formatter.
|
48
|
+
|
49
|
+
Index Page
|
50
|
+
![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_index.png)
|
51
|
+
|
52
|
+
Details on a example Sinatra app
|
53
|
+
![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_details.png)
|
54
|
+
|
55
|
+
|
56
|
+
# Installation
|
57
|
+
|
58
|
+
Follow the below section to install and configure Coverband.
|
59
|
+
|
60
|
+
![coverband installation](https://raw.githubusercontent.com/danmayer/coverband/feature/via_coverage/docs/coverband_install.gif?raw=true)
|
61
|
+
|
62
|
+
## Prerequisites
|
63
|
+
|
64
|
+
* Coverband requires Ruby 2.1 to run the older and slower implementation (TracePoint).
|
65
|
+
* Modern Coverband Ruby 2.3+ is recommended, this runs the (Coverage) implementation.
|
66
|
+
|
67
|
+
## Gem Installation
|
42
68
|
|
43
69
|
Add this line to your application's Gemfile:
|
44
70
|
|
@@ -58,86 +84,45 @@ Or install it yourself as:
|
|
58
84
|
$ gem install coverband
|
59
85
|
```
|
60
86
|
|
61
|
-
|
62
|
-
|
63
|
-
* [Coverband Configuration](https://github.com/danmayer/coverband#configuration)
|
64
|
-
* Rake integration
|
65
|
-
* Coverband config setup
|
66
|
-
* Require Coverband
|
67
|
-
* Insert middleware in stack
|
68
|
-
* run `bundle exec rake coverband:baseline` ([what is baseline?](https://github.com/danmayer/coverband#coverband-baseline)) (__note: never run baseline on production__)
|
69
|
-
* run `bundle exec rake coverband:coverage` this will show app initialization coverage
|
70
|
-
* run app and hit a controller (hit at least +1 time over your `config.startup_delay` setting default is 0)
|
71
|
-
* run `bundle exec rake coverband:coverage` and you should see coverage increasing for the endpoints you hit.
|
72
|
-
|
73
|
-
|
74
|
-
## Example Output
|
75
|
-
|
76
|
-
Since Coverband is [Simplecov](https://github.com/colszowka/simplecov) output compatible it should work with any of the `SimpleCov::Formatter`'s available. The output below is produced using the default Simplecov HTML formatter.
|
77
|
-
|
78
|
-
Index Page
|
79
|
-
![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_index.png)
|
80
|
-
|
81
|
-
Details on a example Sinatra app
|
82
|
-
![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_details.png)
|
87
|
+
# Configuration
|
83
88
|
|
84
|
-
|
89
|
+
After you have the gem, you will want to setup the needed integration:
|
85
90
|
|
86
|
-
|
91
|
+
1. Coverband Config File Setup
|
92
|
+
2. Configure Rake
|
93
|
+
3. Insert Coverband Rack Middleware
|
87
94
|
|
88
|
-
|
95
|
+
### 1. Coverband Config File Setup
|
89
96
|
|
90
|
-
|
91
|
-
|
92
|
-
The baseline seems to cause some confusion. Basically, when Coverband records code usage, it will not request initial startup code like method definition, it covers what it hit during run time. This would produce a fairly odd view of code usage. To cover things like defining routes, dynamic methods, and the like Coverband records a baseline. The baseline should capture coverage of app initialization and boot up, we don't want to do this on deploy as it can be slow. So we take a recording of boot up as a one time baseline Rake task `bundle exec rake coverband:baseline`.
|
93
|
-
|
94
|
-
## Configuration
|
95
|
-
|
96
|
-
### 1. Create Coverband config file
|
97
|
-
|
98
|
-
You need to configure cover band you can either do that passing in all configuration options to `Coverband.configure` in block format, or a simpler style is to call `Coverband.configure` with nothing while will load `config/coverband.rb` expecting it to configure the app correctly. Below is an example config file for a Sinatra app:
|
97
|
+
You need to configure cover band you can either do that passing in all configuration options to `Coverband.configure` in block format, or a simpler style is to call `Coverband.configure` with nothing while will load `config/coverband.rb` expecting it to configure the app correctly. Below is an example config file for a Rails 5 app:
|
99
98
|
|
100
99
|
```ruby
|
101
100
|
#config/coverband.rb
|
102
101
|
Coverband.configure do |config|
|
103
102
|
config.root = Dir.pwd
|
104
|
-
|
105
|
-
if defined? Redis
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
#
|
112
|
-
#
|
113
|
-
|
114
|
-
|
115
|
-
|
116
|
-
#config.root_paths = ['/server/apps/my_app/releases/\d+/']
|
117
|
-
config.ignore = ['vendor','lib/scrazy_i18n_patch_thats_hit_all_the_time.rb']
|
118
|
-
# Since rails and other frameworks lazy load code. I have found it is bad to allow
|
119
|
-
# initial requests to record with coverband. This ignores first 15 requests
|
120
|
-
# NOTE: If you are using a threaded webserver (example: Puma) this will ignore requests for each thread
|
121
|
-
config.startup_delay = Rails.env.production? ? 5 : 0
|
122
|
-
# Percentage of requests recorded
|
123
|
-
config.percentage = Rails.env.production? ? 30.0 : 100.0
|
124
|
-
|
103
|
+
config.collector = 'coverage'
|
104
|
+
config.redis = Redis.new(url: ENV['REDIS_URL']) if defined? Redis
|
105
|
+
config.ignore = %w[vendor .erb$ .slim$]
|
106
|
+
# add paths that you deploy to that might be different than your local dev root path
|
107
|
+
config.root_paths = []
|
108
|
+
|
109
|
+
# reporting frequency
|
110
|
+
# if you are debugging changes to coverband I recommend setting to 100.0
|
111
|
+
# otherwise it is find to report slowly over time with less performance impact
|
112
|
+
# with the Coverage collector coverage is ALWAYS captured this is just how frequently
|
113
|
+
# it is reported to your back end store.
|
114
|
+
config.percentage = Rails.env.production? ? 1.0 : 100.0
|
125
115
|
config.logger = Rails.logger
|
126
116
|
|
127
|
-
#stats help you collect how often you are sampling requests and other info
|
128
|
-
if defined? Statsd
|
129
|
-
config.stats = Statsd.new('statsd.host.com', 8125)
|
130
|
-
end
|
131
|
-
|
132
117
|
# config options false, true, or 'debug'. Always use false in production
|
133
118
|
# true and debug can give helpful and interesting code usage information
|
134
119
|
# they both increase the performance overhead of the gem a little.
|
135
120
|
# they can also help with initially debugging the installation.
|
136
|
-
config.verbose =
|
121
|
+
# config.verbose = 'debug'
|
137
122
|
end
|
138
123
|
```
|
139
124
|
|
140
|
-
### 2.
|
125
|
+
### 2. Configure Rake
|
141
126
|
|
142
127
|
Either add the below to your `Rakefile` or to a file included in your Rakefile such as `lib/tasks/coverband.rake` if you want to break it up that way.
|
143
128
|
|
@@ -146,7 +131,7 @@ require 'coverband'
|
|
146
131
|
Coverband.configure
|
147
132
|
require 'coverband/tasks'
|
148
133
|
```
|
149
|
-
This should give you access to a number of
|
134
|
+
This should give you access to a number of Coverband tasks
|
150
135
|
|
151
136
|
```bash
|
152
137
|
bundle exec rake -T coverband
|
@@ -155,27 +140,39 @@ rake coverband:clear # reset coverband coverage data
|
|
155
140
|
rake coverband:coverage # report runtime coverband code coverage
|
156
141
|
```
|
157
142
|
|
158
|
-
|
143
|
+
### 3. Configure Rack to use the Coverband middleware
|
144
|
+
|
145
|
+
The middleware is what makes Coverband gather metrics while a webapp app runs.
|
146
|
+
|
147
|
+
#### For Rails apps
|
148
|
+
|
149
|
+
There are a number of options for starting Coverband and when to insert it into your middleware stack. At the moment this is my recommendation, to ensure you capture usage data on any files required by other plugins.
|
150
|
+
|
151
|
+
Then add the middleware to your Rails rake middle ware stack:
|
159
152
|
|
160
153
|
```ruby
|
161
|
-
|
162
|
-
|
163
|
-
task :baseline_app do
|
164
|
-
Coverband::Reporter.baseline {
|
165
|
-
require 'sinatra'
|
166
|
-
require './app.rb'
|
167
|
-
}
|
168
|
-
end
|
169
|
-
end
|
170
|
-
```
|
154
|
+
# config/application.rb
|
155
|
+
[...]
|
171
156
|
|
172
|
-
|
173
|
-
|
157
|
+
module MyApplication
|
158
|
+
class Application < Rails::Application
|
159
|
+
[...]
|
174
160
|
|
175
|
-
|
161
|
+
# Coverband needs to be setup before any of the initializers to capture usage of them
|
162
|
+
require 'coverband'
|
163
|
+
Coverband.configure
|
164
|
+
config.middleware.use Coverband::Middleware
|
176
165
|
|
177
|
-
|
178
|
-
|
166
|
+
# if one uses before_eager_load as I did previously
|
167
|
+
# any files that get loaded as part of railties will have no coverage
|
168
|
+
config.before_initialize do
|
169
|
+
require 'coverage'
|
170
|
+
Coverband::Collectors::Base.instance.start
|
171
|
+
end
|
172
|
+
|
173
|
+
end
|
174
|
+
end
|
175
|
+
```
|
179
176
|
|
180
177
|
#### For Sinatra apps
|
181
178
|
|
@@ -191,51 +188,130 @@ use Coverband::Middleware
|
|
191
188
|
run ActionController::Dispatcher.new
|
192
189
|
```
|
193
190
|
|
194
|
-
|
191
|
+
# Verify Correct Installation
|
195
192
|
|
196
|
-
|
193
|
+
* Gather baseline metrics: run `bundle exec rake coverband:baseline` (__note: never run baseline on production__)
|
194
|
+
* run `bundle exec rake coverband:coverage` this will show app initialization coverage
|
195
|
+
* run app and hit a controller (hit at least +1 time over your `config.startup_delay` setting default is 0)
|
196
|
+
* run `bundle exec rake coverband:coverage` and you should see coverage increasing for the endpoints you hit.
|
197
197
|
|
198
|
-
|
199
|
-
# config/initializers/coverband_middleware.rb
|
198
|
+
## Installation Script
|
200
199
|
|
201
|
-
|
200
|
+
These are the steps show in setting up coverband in the gif in the readme.
|
201
|
+
|
202
|
+
```
|
203
|
+
rails new coverage_example
|
204
|
+
cd coverage_example
|
205
|
+
atom .
|
206
|
+
|
207
|
+
# open Gemfile, add lines
|
208
|
+
gem 'redis'
|
209
|
+
gem 'coverband', '>= 2.0.0.alpha1', require: false
|
210
|
+
|
211
|
+
bundle install
|
212
|
+
|
213
|
+
# create config/coverband.rb
|
214
|
+
# copy the config from the readme
|
215
|
+
# If you don't set REDIS_URL, remove that to use default localhose
|
216
|
+
|
217
|
+
rails generate scaffold blogs
|
218
|
+
bundle exec rake db:migrate
|
219
|
+
|
220
|
+
# open Rakefile, add lines
|
202
221
|
require 'coverband'
|
203
222
|
Coverband.configure
|
204
|
-
|
223
|
+
require 'coverband/tasks'
|
205
224
|
|
206
|
-
|
225
|
+
# verify rake
|
226
|
+
rake -T coverband
|
207
227
|
|
208
|
-
|
209
|
-
#
|
210
|
-
[...]
|
228
|
+
# configure config/application.rb
|
229
|
+
# copy lines from readme
|
211
230
|
|
212
|
-
module MyApplication
|
213
|
-
class Application < Rails::Application
|
214
|
-
[...]
|
215
231
|
|
216
|
-
|
217
|
-
|
232
|
+
rake coverband:baseline
|
233
|
+
rake coverband:coverage
|
234
|
+
# view baseline coverage
|
218
235
|
|
219
|
-
|
220
|
-
|
236
|
+
rails s
|
237
|
+
|
238
|
+
open http://localhost:3000/blogs
|
239
|
+
|
240
|
+
# click around some to trigger usage
|
241
|
+
|
242
|
+
# view updated coverage
|
243
|
+
rake coverband:coverage
|
221
244
|
```
|
222
245
|
|
223
|
-
|
246
|
+
# Usage
|
247
|
+
|
248
|
+
### Example apps
|
249
|
+
|
250
|
+
- [Rails 5.1.x App](https://github.com/danmayer/coverage_rails_benchmark)
|
251
|
+
- [Sinatra app](https://github.com/danmayer/churn-site)
|
252
|
+
- [Non Rack Ruby app](https://github.com/danmayer/coverband_examples)
|
253
|
+
|
254
|
+
### Coverband Baseline
|
224
255
|
|
225
|
-
|
256
|
+
__TLDR:__ Baseline is app initialization coverage, not captured during runtime.
|
257
|
+
|
258
|
+
Before starting a service verify your baseline by running `rake coverband:baseline` followed by `rake coverband:coverage` to view what your baseline coverage looks like before any runtime traffic has been recorded.
|
259
|
+
|
260
|
+
The baseline seems to cause some confusion. Basically, when Coverband records code usage, it will not request initial startup code like method definition, it covers what it hit during run time. This would produce a fairly odd view of code usage. To cover things like defining routes, dynamic methods, and the like Coverband records a baseline. The baseline should capture coverage of app initialization and boot up, we don't want to do this on deploy as it can be slow. So we take a recording of boot up as a one time baseline Rake task `bundle exec rake coverband:baseline`.
|
226
261
|
|
227
262
|
1. Start your server with `rails s` or `rackup config.ru`.
|
228
263
|
2. Hit your development server exercising the endpoints you want to verify Coverband is recording (you should see debug outputs in your server console)
|
229
264
|
3. Run `rake coverband:coverage` again, previously it should have only shown the baseline data of your app initializing. After using it in development it should show increased coverage from the actions you have exercised.
|
230
265
|
|
231
|
-
Note: if you use `rails s` and data
|
266
|
+
Note: if you use `rails s` and data isn't recorded, make sure it is using your `config.ru`.
|
232
267
|
|
233
|
-
|
268
|
+
### Baseline Missing data
|
234
269
|
|
235
|
-
|
236
|
-
|
237
|
-
|
238
|
-
|
270
|
+
The default Coverband baseline task will try to detect your app as either Rack or Rails environment. It will load the app to take a baseline reading. The baseline coverage is important because some code is executed during app load, but might not be invoked during any requests, think prefetching, initial cache builds, setting constants, etc. If the baseline task doesn't load your app well you can override the default baseline to create a better baseline yourself. Below for example is how I take a baseline on a pretty simple Sinatra app.
|
271
|
+
|
272
|
+
```ruby
|
273
|
+
namespace :coverband do
|
274
|
+
desc "get coverage baseline"
|
275
|
+
task :baseline_app do
|
276
|
+
Coverband::Reporter.baseline {
|
277
|
+
require 'sinatra'
|
278
|
+
require './app.rb'
|
279
|
+
}
|
280
|
+
end
|
281
|
+
end
|
282
|
+
```
|
283
|
+
|
284
|
+
### View Coverage
|
285
|
+
|
286
|
+
You can view the report different ways, but the easiest is the Rake task which opens the Simplecov formated HTML.
|
287
|
+
|
288
|
+
`bundle exec rake coverband:coverage`
|
289
|
+
|
290
|
+
This should auto-open in your browser, but if it doesn't the output file should be in `coverage/index.html`
|
291
|
+
|
292
|
+
### Clear Coverage
|
293
|
+
|
294
|
+
If your code has changed and your coverage line data doesn't seem to match run time. You probably need to clear your old line data... You will need to run this in the environment you wish to clear the data from.
|
295
|
+
|
296
|
+
`rake coverband:clear`
|
297
|
+
|
298
|
+
### Automated Clearing Line Coverage Data
|
299
|
+
|
300
|
+
After a deploy where code has changed significantly.
|
301
|
+
|
302
|
+
The line numbers previously recorded in Redis may no longer match the current state of the files.
|
303
|
+
If being slightly out of sync isn't as important as gathering data over a long period,
|
304
|
+
you can live with minor inconsistency for some files.
|
305
|
+
|
306
|
+
As often as you like or as part of a deploy hook you can clear the recorded Coverband data with the following command.
|
307
|
+
|
308
|
+
```ruby
|
309
|
+
# defaults to the currently configured Coverband.configuration.redis
|
310
|
+
Coverband::Reporter.clear_coverage
|
311
|
+
# or pass in the current target redis
|
312
|
+
Coverband::Reporter.clear_coverage(Redis.new(:host => 'target.com', :port => 6789))
|
313
|
+
```
|
314
|
+
You can also do this with the included rake tasks.
|
239
315
|
|
240
316
|
### Manual configuration (for example for background jobs)
|
241
317
|
|
@@ -284,26 +360,11 @@ coverband.sample {
|
|
284
360
|
}
|
285
361
|
```
|
286
362
|
|
287
|
-
###
|
363
|
+
### Verbose Debug / Development Mode
|
288
364
|
|
289
|
-
|
365
|
+
Note: To debug issues getting coverband working. I recommend running in development mode, by turning verbose logging on `config.verbose = true` and passing in the Rails.logger `config.logger = Rails.logger` to the Coverband config. This makes it easy to follow in development mode. Be careful to not leave these on in production as they will affect performance.
|
290
366
|
|
291
|
-
|
292
|
-
If being slightly out of sync isn't as important as gathering data over a long period,
|
293
|
-
you can live with minor inconsistency for some files.
|
294
|
-
|
295
|
-
As often as you like or as part of a deploy hook you can clear the recorded Coverband data with the following command.
|
296
|
-
|
297
|
-
```ruby
|
298
|
-
# defaults to the currently configured Coverband.configuration.redis
|
299
|
-
Coverband::Reporter.clear_coverage
|
300
|
-
# or pass in the current target redis
|
301
|
-
Coverband::Reporter.clear_coverage(Redis.new(:host => 'target.com', :port => 6789))
|
302
|
-
```
|
303
|
-
You can also do this with the included rake tasks.
|
304
|
-
|
305
|
-
|
306
|
-
### Verbose debug mode for development
|
367
|
+
---
|
307
368
|
|
308
369
|
If you are trying to debug locally wondering what code is being run during a request. The verbose modes `config.verbose = true` and `config.verbose = 'debug'` can be useful. With true set it will output the number of lines executed per file, to the passed in log. The files are sorted from least used file to most active file. I have even run that mode in production without much of a problem. The debug verbose mode outputs both file usage and provides the number of calls per line of code. For example if you see something like below which indicates that the `application_helper` has 43150 lines executed. That might seem odd. Then looking at the breakdown of `application_helper` we can see that line `516` was executed 38,577 times. That seems bad, and is likely worth investigating perhaps memoizing or cacheing is required.
|
309
370
|
|
@@ -339,24 +400,101 @@ Coverband::Reporter.report :additional_scov_data => [data]
|
|
339
400
|
|
340
401
|
You can also pass a `:additional_scov_data => [data]` option to `Coverband::Reporter.get_current_scov_data` to write out merged data.
|
341
402
|
|
342
|
-
|
403
|
+
# Contributing To Coverband
|
343
404
|
|
344
405
|
If you are working on adding features, PRs, or bugfixes to Coverband this section should help get you going.
|
345
406
|
|
407
|
+
1. Fork it
|
408
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
409
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
410
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
411
|
+
5. Make sure all tests are passing (run `bundle install`, make sure Redis is running, and then execute `bundle exec rake test`)
|
412
|
+
6. Create new Pull Request
|
413
|
+
|
414
|
+
### Tests & Benchmarks
|
415
|
+
|
416
|
+
If you submit a change please make sure the tests and benchmarks are passing.
|
417
|
+
|
346
418
|
* run tests: `bundle exec rake`
|
347
419
|
* view test coverage: `open coverage/index.html`
|
348
420
|
* run the benchmarks before and after your change to see impact
|
349
421
|
* `bundle exec rake benchmarks`
|
350
422
|
|
351
|
-
### Known
|
423
|
+
### Known Issues
|
352
424
|
|
353
425
|
* __total fail__ on front end code, because of the precompiled template step basically coverage doesn't work well for `erb`, `slim`, and the like.
|
354
|
-
* If you don't have a baseline recorded your coverage can look odd like you are missing a bunch of data. It would be good if Coverband gave a more actionable warning in this situation.
|
355
426
|
* If you have SimpleCov filters, you need to clear them prior to generating your coverage report. As the filters will be applied to Coverband as well and can often filter out everything we are recording.
|
356
427
|
* the line numbers reported for `ERB` files are often off and aren't considered useful. I recommend filtering out .erb using the `config.ignore` option.
|
357
428
|
* coverage doesn't show for Rails `config/application.rb` or `config/boot.rb` as they get loaded when loading the Rake environment prior to starting to record the baseline..
|
358
429
|
|
359
|
-
|
430
|
+
### Debugging Redis Store
|
431
|
+
|
432
|
+
What files have been synced to Redis?
|
433
|
+
|
434
|
+
`Coverband.configuration.store.covered_files`
|
435
|
+
|
436
|
+
What is the coverage data in Redis?
|
437
|
+
|
438
|
+
`Coverband.configuration.store.coverage`
|
439
|
+
|
440
|
+
### Internal Formats
|
441
|
+
|
442
|
+
If you are doing development having some documented examples of various internal data formats can be helpfu....
|
443
|
+
|
444
|
+
The format we get from TracePoint, Coverage, Internal Representations, and Used by SimpleCov for reporting have traditionally varied a bit. We can document the differences in formats here.
|
445
|
+
|
446
|
+
#### Coverage
|
447
|
+
|
448
|
+
```
|
449
|
+
>> require 'coverage'
|
450
|
+
=> true
|
451
|
+
>> Coverage.start
|
452
|
+
=> nil
|
453
|
+
>> require './test/unit/dog.rb'
|
454
|
+
=> true
|
455
|
+
>> 5.times { Dog.new.bark }
|
456
|
+
=> 5
|
457
|
+
>> Coverage.peek_result
|
458
|
+
=> {"/Users/danmayer/projects/coverband/test/unit/dog.rb"=>[nil, nil, 1, 1, 5, nil, nil]}
|
459
|
+
```
|
460
|
+
|
461
|
+
#### SimpleCov
|
462
|
+
|
463
|
+
The same format, but relative paths.
|
464
|
+
|
465
|
+
```
|
466
|
+
{"test/unit/dog.rb"=>[1, 2, nil, nil, nil, nil, nil]}
|
467
|
+
```
|
468
|
+
|
469
|
+
#### Redis Store
|
470
|
+
|
471
|
+
We store relative path in Redis, the Redis hash stores line numbers -> count (as strings).
|
472
|
+
|
473
|
+
```
|
474
|
+
# Array
|
475
|
+
["test/unit/dog.rb"]
|
476
|
+
|
477
|
+
# Hash
|
478
|
+
{"test/unit/dog.rb"=>{"1"=>"1", "2"=>"2"}}
|
479
|
+
```
|
480
|
+
|
481
|
+
#### File Store
|
482
|
+
|
483
|
+
Similar format to redis store, but array with integer values
|
484
|
+
|
485
|
+
```
|
486
|
+
{"test/unit/dog.rb"=>{"1"=>1, "2"=>2}}
|
487
|
+
```
|
488
|
+
|
489
|
+
# Future Coverband
|
490
|
+
|
491
|
+
### Alternative Redis formats
|
492
|
+
|
493
|
+
* Look at alternative storage formats for Redis
|
494
|
+
* [redis bitmaps](http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/)
|
495
|
+
* [redis bitfield](https://stackoverflow.com/questions/47100606/optimal-way-to-store-array-of-integers-in-redis-database)
|
496
|
+
|
497
|
+
### Todo
|
360
498
|
|
361
499
|
* graphite adapters (it would allow passing in date ranges on usage)
|
362
500
|
* perf test for array vs hash
|
@@ -371,25 +509,15 @@ If you are working on adding features, PRs, or bugfixes to Coverband this sectio
|
|
371
509
|
* ability to change the Coverband config at runtime by changing the config pushed to the Redis hash. In memory cache around the changes to only make that call periodically.
|
372
510
|
* Opposed to just showing code usage on a route allow 'tagging' events which would record line coverage for that tag (this would allow tagging all code that modified an ActiveRecord model for example
|
373
511
|
* mountable rack app to view coverage similar to flipper-ui
|
512
|
+
* support runner, active job, etc without needed extra config (improved railtie integration)
|
374
513
|
|
375
|
-
|
376
|
-
|
377
|
-
1. Fork it
|
378
|
-
2. Create your feature branch (`git checkout -b my-new-feature`)
|
379
|
-
3. Commit your changes (`git commit -am 'Add some feature'`)
|
380
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
381
|
-
5. Make sure all tests are passing (run `bundle install`, make sure Redis is running, and then execute `bundle exec rake test`)
|
382
|
-
6. Create new Pull Request
|
383
|
-
|
384
|
-
## Resources
|
514
|
+
# Resources
|
385
515
|
|
386
516
|
These notes of kind of for myself, but if anyone is seriously interested in contributing to the project, these resources might be helpful. I learned a lot looking at various existing projects and open source code.
|
387
517
|
|
388
518
|
##### Ruby Std-lib Coverage
|
389
519
|
|
390
|
-
* [
|
391
|
-
* [Current Coverage Bug causing issues on 2.1.1](https://bugs.ruby-lang.org/issues/9572)
|
392
|
-
* [Ruby Coverage docs](http://www.ruby-doc.org/stdlib-1.9.3/libdoc/coverage/rdoc/Coverage.html)
|
520
|
+
* [Ruby Coverage docs](https://ruby-doc.org/stdlib-2.5.0/libdoc/coverage/rdoc/Coverage.html)
|
393
521
|
|
394
522
|
##### Other
|
395
523
|
|
@@ -404,5 +532,7 @@ These notes of kind of for myself, but if anyone is seriously interested in cont
|
|
404
532
|
* [learn from stackprof](https://github.com/tmm1/stackprof#readme)
|
405
533
|
* I believe there are possible ways to get even better data using the new [Ruby2 TracePoint API](http://www.ruby-doc.org/core/TracePoint.html)
|
406
534
|
|
407
|
-
|
535
|
+
# License
|
536
|
+
|
537
|
+
This is a MIT License project...
|
408
538
|
See the file license.txt for copying permission.
|