counters 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +5 -0
- data/.rvmrc +1 -0
- data/Gemfile +4 -0
- data/LICENSE +20 -0
- data/README.textile +84 -0
- data/Rakefile +12 -0
- data/autotest/discover.rb +1 -0
- data/counters.gemspec +22 -0
- data/lib/counters.rb +14 -0
- data/lib/counters/base.rb +53 -0
- data/lib/counters/file.rb +34 -0
- data/lib/counters/memory.rb +30 -0
- data/lib/counters/redis.rb +30 -0
- data/lib/counters/version.rb +3 -0
- data/spec/file_counter_spec.rb +75 -0
- data/spec/memory_counter_spec.rb +47 -0
- data/spec/redis_counter_spec.rb +57 -0
- data/spec/spec_helper.rb +26 -0
- metadata +110 -0
data/.rvmrc
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
rvm use 1.9.2
|
data/Gemfile
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2008-2009 François Beausoleil (francois@teksol.info)
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
'Software'), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
17
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
18
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
19
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
20
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.textile
ADDED
@@ -0,0 +1,84 @@
|
|
1
|
+
h1. Counters
|
2
|
+
|
3
|
+
Easily record any metric from anywhere within your system. Metrics are recorded to Redis (using the provided backend), in a single Hash key. You can then extract the keys later and use them with Cacti to generate graphs about anything going on.
|
4
|
+
|
5
|
+
h2. Sample Usage
|
6
|
+
|
7
|
+
Let's say you have a crawler. You'd like to record the number of URLs you visit, the number of URLs you skipped due to 304 Not Modified responses, and the number of bytes you consumed, and the amount of time each page takes to process. Here's how you'd do that:
|
8
|
+
|
9
|
+
<pre><code>require "counters"
|
10
|
+
require "rest_client"
|
11
|
+
Counter = Counters::Redis.new(Redis.new, "counters")
|
12
|
+
|
13
|
+
while url = STDIN.gets
|
14
|
+
Counter.hit "crawler.urls"
|
15
|
+
|
16
|
+
response = RestClient.get(url)
|
17
|
+
Counter.magnitude "crawler.bytes.read", response.length
|
18
|
+
next Counter.hit "crawler.urls.skipped" if response.code == 304
|
19
|
+
|
20
|
+
Counter.latency "crawler.processing" do
|
21
|
+
# some long and complicated processing
|
22
|
+
end
|
23
|
+
end
|
24
|
+
</code></pre>
|
25
|
+
|
26
|
+
Redis will have a single key, named counters here, with the following keys and values in it (after 1 call with a 200 response code):
|
27
|
+
|
28
|
+
* hits.crawler.urls = 1
|
29
|
+
* magnitudes.crawler.bytes.read = 2041
|
30
|
+
* latencies.crawler.processing.count = 1
|
31
|
+
* latencies.crawler.processing.nanoseconds = 381000000
|
32
|
+
|
33
|
+
h2. Other Implementations
|
34
|
+
|
35
|
+
For testing purposes, there also exists a <code>Counters::Memory</code>. This would be good in test mode, for example. The counters are exposed through accessor methods returning a Hash.
|
36
|
+
|
37
|
+
For file logging, you should use <code>Counters::File</code>, which accepts a <code>String</code>, <code>IO</code> or <code>Logger</code> instance. All of these will be transformed to a <code>Logger</code> instance with a very strict format. All events will be logged to the file, one event per line.
|
38
|
+
|
39
|
+
<pre><code>$ irb -r counters
|
40
|
+
> Counter = Counters::File.new("counters.log")
|
41
|
+
=> #<Counters::File:0x00000101a18f18 @logger=#<Logger:0x00000101a18ef0 @progname=nil, @level=0, @default_formatter=#<Logger::Formatter:0x00000101a18ea0 @datetime_format=nil>, @formatter=#<Proc:0x00000101a18bd0@/Users/francois/Projects/counters/lib/counters/file.rb:15 (lambda)>, @logdev=#<Logger::LogDevice:0x00000101a18e28 @shift_size=1048576, @shift_age=0, @filename="counters.log", @dev=#<File:counters.log>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x00000101a18e00 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000101a18d88>>>>>
|
42
|
+
> Counter.hit "crawler.page_read"
|
43
|
+
=> true
|
44
|
+
> Counter.magnitude "crawler.bytes_in", 9_921
|
45
|
+
=> true
|
46
|
+
> Counter.latency "crawler.processing" do sleep 0.3 ; end
|
47
|
+
=> true
|
48
|
+
> Counter.ping "crawler.alive"
|
49
|
+
=> true
|
50
|
+
|
51
|
+
$ cat counters.log
|
52
|
+
2011-02-21T09:46:21.296326000 - hit: crawler.page_read
|
53
|
+
2011-02-21T09:46:24.280388000 - magnitude: crawler.bytes_in 9921
|
54
|
+
2011-02-21T09:46:27.989183000 - latency: crawler.processing 0.3001821041107178s
|
55
|
+
2011-02-21T09:46:31.031969000 - ping: crawler.alive
|
56
|
+
2011-02-21T09:46:21.296326000 - hit: crawler.page_read
|
57
|
+
2011-02-21T09:46:24.280388000 - magnitude: crawler.bytes_in 13291
|
58
|
+
2011-02-21T09:46:27.989183000 - latency: crawler.processing 0.3123122982101s
|
59
|
+
</code></pre>
|
60
|
+
|
61
|
+
h2. LICENSE
|
62
|
+
|
63
|
+
(The MIT License)
|
64
|
+
|
65
|
+
Copyright (c) 2008-2009 François Beausoleil (francois@teksol.info)
|
66
|
+
|
67
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
68
|
+
a copy of this software and associated documentation files (the
|
69
|
+
'Software'), to deal in the Software without restriction, including
|
70
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
71
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
72
|
+
permit persons to whom the Software is furnished to do so, subject to
|
73
|
+
the following conditions:
|
74
|
+
|
75
|
+
The above copyright notice and this permission notice shall be
|
76
|
+
included in all copies or substantial portions of the Software.
|
77
|
+
|
78
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
79
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
80
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
81
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
82
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
83
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
84
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/Rakefile
ADDED
@@ -0,0 +1,12 @@
|
|
1
|
+
require 'bundler'
|
2
|
+
Bundler::GemHelper.install_tasks
|
3
|
+
|
4
|
+
begin
|
5
|
+
require "rspec/core/rake_task"
|
6
|
+
RSpec::Core::RakeTask.new
|
7
|
+
|
8
|
+
task :default => :spec
|
9
|
+
rescue LoadError
|
10
|
+
warn "RSpec not available - Rake tasks not available"
|
11
|
+
warn "Install rspec using: gem install rspec"
|
12
|
+
end
|
@@ -0,0 +1 @@
|
|
1
|
+
Autotest.add_discovery { "rspec2" }
|
data/counters.gemspec
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
$:.push File.expand_path("../lib", __FILE__)
|
3
|
+
require "counters/version"
|
4
|
+
|
5
|
+
Gem::Specification.new do |s|
|
6
|
+
s.name = "counters"
|
7
|
+
s.version = Counters::VERSION
|
8
|
+
s.platform = Gem::Platform::RUBY
|
9
|
+
s.authors = ["François Beausoleil"]
|
10
|
+
s.email = ["francois@teksol.info"]
|
11
|
+
s.homepage = ""
|
12
|
+
s.summary = %q{Provides an API to record any kind of metrics within your system}
|
13
|
+
s.description = %q{Using the provided API, record metrics (such as number of hits to a particular controller, bytes in/out, compression ratio) within your system. Visualization is NOT provided within this gem.}
|
14
|
+
|
15
|
+
s.files = `git ls-files`.split("\n")
|
16
|
+
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
17
|
+
s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
|
18
|
+
s.require_paths = ["lib"]
|
19
|
+
|
20
|
+
s.add_development_dependency "rspec"
|
21
|
+
s.add_development_dependency "timecop"
|
22
|
+
end
|
data/lib/counters.rb
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
# A simple way to record performance counters / metrics.
|
2
|
+
#
|
3
|
+
# All performance counters can be broken down into a few categories:
|
4
|
+
#
|
5
|
+
# * +ping+ Just to know something is still alive. You'd probably graph "now() - ping".
|
6
|
+
# * +hit+ Increments a counter. In this case you'd graph the 1st derivative, to see how the slope of changes.
|
7
|
+
# * +latency+ Increments a counter representing time. You'd again graph the 1st derivative.
|
8
|
+
# * +magnitude+ Sets a counter representing a value (bytes, free RAM, etc). Here you'd graph min, max, avg and stdev.
|
9
|
+
module Counters
|
10
|
+
autoload :Base, "counters/base"
|
11
|
+
autoload :Redis, "counters/redis"
|
12
|
+
autoload :Memory, "counters/memory"
|
13
|
+
autoload :File, "counters/file"
|
14
|
+
end
|
@@ -0,0 +1,53 @@
|
|
1
|
+
module Counters
|
2
|
+
class Base
|
3
|
+
def hit(key)
|
4
|
+
validate(key)
|
5
|
+
record_hit(key)
|
6
|
+
end
|
7
|
+
|
8
|
+
def magnitude(key, value)
|
9
|
+
validate(key)
|
10
|
+
record_magnitude(key, value)
|
11
|
+
end
|
12
|
+
|
13
|
+
def latency(key, time_in_seconds=nil)
|
14
|
+
validate(key)
|
15
|
+
if block_given? then
|
16
|
+
raise ArgumentError, "Must pass either a latency or a block, not both: received #{time_in_seconds.inspect} in addition to a block" if time_in_seconds
|
17
|
+
time_in_seconds = Benchmark.measure { yield }.real
|
18
|
+
end
|
19
|
+
|
20
|
+
record_latency(key, time_in_seconds)
|
21
|
+
end
|
22
|
+
|
23
|
+
def ping(key)
|
24
|
+
validate(key)
|
25
|
+
record_ping(key)
|
26
|
+
end
|
27
|
+
|
28
|
+
def record_hit(key)
|
29
|
+
raise "Subclass Responsibility Error: must be implemented in instances of #{self.class} but isn't"
|
30
|
+
end
|
31
|
+
protected :record_hit
|
32
|
+
|
33
|
+
def record_magnitude(key)
|
34
|
+
raise "Subclass Responsibility Error: must be implemented in instances of #{self.class} but isn't"
|
35
|
+
end
|
36
|
+
protected :record_magnitude
|
37
|
+
|
38
|
+
def record_latency(key)
|
39
|
+
raise "Subclass Responsibility Error: must be implemented in instances of #{self.class} but isn't"
|
40
|
+
end
|
41
|
+
protected :record_latency
|
42
|
+
|
43
|
+
def record_ping(key)
|
44
|
+
raise "Subclass Responsibility Error: must be implemented in instances of #{self.class} but isn't"
|
45
|
+
end
|
46
|
+
protected :record_ping
|
47
|
+
|
48
|
+
def validate(key)
|
49
|
+
key.to_s =~ /\A[.\w]+\Z/i or raise ArgumentError, "Keys can contain only letters, numbers, the underscore (_) and fullstop (.), received #{key.inspect}"
|
50
|
+
end
|
51
|
+
private :validate
|
52
|
+
end
|
53
|
+
end
|
@@ -0,0 +1,34 @@
|
|
1
|
+
require "benchmark"
|
2
|
+
require "logger"
|
3
|
+
|
4
|
+
module Counters
|
5
|
+
class File < Counters::Base
|
6
|
+
def initialize(path_or_io_or_logger)
|
7
|
+
@logger = if path_or_io_or_logger.kind_of?(Logger) then
|
8
|
+
path_or_io_or_logger
|
9
|
+
elsif path_or_io_or_logger.respond_to?(:<<) then
|
10
|
+
Logger.new(path_or_io_or_logger)
|
11
|
+
else
|
12
|
+
raise ArgumentError, "Counters::File expects an object which is either a Logger or respond to #<<, received a #{path_or_io_or_logger.class}"
|
13
|
+
end
|
14
|
+
|
15
|
+
@logger.formatter = lambda {|severity, datetime, progname, msg| "#{datetime.strftime("%Y-%m-%dT%H:%M:%S.%N")} - #{msg}\n"}
|
16
|
+
end
|
17
|
+
|
18
|
+
def record_hit(key)
|
19
|
+
@logger.info "hit: #{key}"
|
20
|
+
end
|
21
|
+
|
22
|
+
def record_magnitude(key, magnitude)
|
23
|
+
@logger.info "magnitude: #{key} #{magnitude}"
|
24
|
+
end
|
25
|
+
|
26
|
+
def record_latency(key, time_in_seconds=nil)
|
27
|
+
@logger.info "latency: #{key} #{time_in_seconds}s"
|
28
|
+
end
|
29
|
+
|
30
|
+
def record_ping(key)
|
31
|
+
@logger.info "ping: #{key}"
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
require "counters"
|
2
|
+
|
3
|
+
module Counters
|
4
|
+
class Memory < Counters::Base
|
5
|
+
attr_reader :hits, :latencies, :magnitudes, :pings
|
6
|
+
|
7
|
+
def initialize
|
8
|
+
@hits = Hash.new {|h,k| h[k] = 0}
|
9
|
+
@magnitudes = Hash.new {|h,k| h[k] = 0}
|
10
|
+
@latencies = Hash.new {|h,k| h[k] = Array.new}
|
11
|
+
@pings = Hash.new
|
12
|
+
end
|
13
|
+
|
14
|
+
def record_hit(key)
|
15
|
+
@hits[key] += 1
|
16
|
+
end
|
17
|
+
|
18
|
+
def record_ping(key)
|
19
|
+
@pings[key] = Time.now
|
20
|
+
end
|
21
|
+
|
22
|
+
def record_latency(key, time_in_seconds)
|
23
|
+
@latencies[key] << time_in_seconds
|
24
|
+
end
|
25
|
+
|
26
|
+
def record_magnitude(key, value)
|
27
|
+
@magnitudes[key] = value
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
require "benchmark"
|
2
|
+
require "counters"
|
3
|
+
|
4
|
+
module Counters
|
5
|
+
class Redis < Counters::Base
|
6
|
+
def initialize(redis, base_key)
|
7
|
+
@redis, @base_key = redis, base_key
|
8
|
+
end
|
9
|
+
|
10
|
+
def record_hit(key)
|
11
|
+
@redis.hincrby(@base_key, "hits.#{key}", 1)
|
12
|
+
end
|
13
|
+
|
14
|
+
def record_magnitude(key, amount)
|
15
|
+
@redis.hset(@base_key, "magnitudes.#{key}", amount)
|
16
|
+
end
|
17
|
+
|
18
|
+
def ping(key)
|
19
|
+
@redis.hset(@base_key, "pings.#{key}", Time.now.to_i)
|
20
|
+
end
|
21
|
+
|
22
|
+
# Redis requires integer keys, thus we scale all latencies to the nanosecond precision
|
23
|
+
SCALING_FACTOR = 1_000_000_000
|
24
|
+
|
25
|
+
def record_latency(key, latency_in_seconds)
|
26
|
+
@redis.hincrby(@base_key, "latencies.#{key}.count", 1)
|
27
|
+
@redis.hincrby(@base_key, "latencies.#{key}.nanoseconds", (latency_in_seconds * SCALING_FACTOR).to_i)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
@@ -0,0 +1,75 @@
|
|
1
|
+
require "spec_helper"
|
2
|
+
require "logger"
|
3
|
+
require "tempfile"
|
4
|
+
|
5
|
+
describe Counters::File do
|
6
|
+
TIMESTAMP_RE = /\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3,}/
|
7
|
+
|
8
|
+
let :tempfile do
|
9
|
+
Tempfile.new("counters.log")
|
10
|
+
end
|
11
|
+
|
12
|
+
let :counter do
|
13
|
+
Counters::File.new(tempfile)
|
14
|
+
end
|
15
|
+
|
16
|
+
it_should_behave_like "all counters"
|
17
|
+
|
18
|
+
it "should log a message to the logfile when a hit is recorded" do
|
19
|
+
counter.hit "urls.visited"
|
20
|
+
tempfile.rewind
|
21
|
+
tempfile.read.should =~ /^#{TIMESTAMP_RE}\s-\shit.*urls\.visited$/
|
22
|
+
end
|
23
|
+
|
24
|
+
it "should log a message to the logfile when a magnitude is recorded" do
|
25
|
+
counter.magnitude "bytes.read", 2_013
|
26
|
+
tempfile.rewind
|
27
|
+
tempfile.read.should =~ /^#{TIMESTAMP_RE}\s-\smagnitude.*bytes\.read.*2013$/
|
28
|
+
end
|
29
|
+
|
30
|
+
it "should log a message to the logfile when a latency is recorded" do
|
31
|
+
counter.latency "processing", 0.132 # in seconds
|
32
|
+
tempfile.rewind
|
33
|
+
tempfile.read.should =~ /^#{TIMESTAMP_RE}\s-\slatency.*processing.*0.132s$/
|
34
|
+
end
|
35
|
+
|
36
|
+
it "should record a message in the logfile when a ping is recorded" do
|
37
|
+
counter.ping "crawler.alive"
|
38
|
+
tempfile.rewind
|
39
|
+
tempfile.read.should =~ /^#{TIMESTAMP_RE}\s-\sping.*crawler\.alive$/
|
40
|
+
end
|
41
|
+
|
42
|
+
it "should log a message to the logfile when a latency is recorded using a block" do
|
43
|
+
counter.latency "crawling" do
|
44
|
+
sleep 0.1
|
45
|
+
end
|
46
|
+
|
47
|
+
tempfile.rewind
|
48
|
+
tempfile.read.should =~ /latency.*crawling.*0.1\d+s/
|
49
|
+
end
|
50
|
+
|
51
|
+
it "should raise an ArgumentError when calling #latency with both a block and a latency" do
|
52
|
+
lambda { counter.latency("processing", 0.123) { sleep 0.1 } }.should raise_error(ArgumentError)
|
53
|
+
end
|
54
|
+
|
55
|
+
it "should accept a filename on instantiation" do
|
56
|
+
Counters::File.new("counters.log")
|
57
|
+
end
|
58
|
+
|
59
|
+
it "should accept a File instance on instantiation" do
|
60
|
+
Counters::File.new( File.open("counters.log", "w") )
|
61
|
+
end
|
62
|
+
|
63
|
+
it "should accept a Logger instance on instantiation" do
|
64
|
+
Counters::File.new( Logger.new("counters.log") )
|
65
|
+
end
|
66
|
+
|
67
|
+
it "should raise an ArgumentError when a bad type is used in the initializer" do
|
68
|
+
lambda { Counters::File.new(nil) }.should raise_error(ArgumentError)
|
69
|
+
end
|
70
|
+
|
71
|
+
after(:each) do
|
72
|
+
fname = File.dirname(__FILE__) + "/../counters.log"
|
73
|
+
File.unlink(fname) if File.exist?(fname)
|
74
|
+
end
|
75
|
+
end
|
@@ -0,0 +1,47 @@
|
|
1
|
+
require "spec_helper"
|
2
|
+
|
3
|
+
describe Counters::Memory do
|
4
|
+
let :counter do
|
5
|
+
Counters::Memory.new
|
6
|
+
end
|
7
|
+
|
8
|
+
it_should_behave_like "all counters"
|
9
|
+
|
10
|
+
it "should record a hit with key 'pages.read'" do
|
11
|
+
counter.hit "pages.read"
|
12
|
+
counter.hits.should have_key("pages.read")
|
13
|
+
counter.hits["pages.read"].should == 1
|
14
|
+
|
15
|
+
counter.hit "pages.read"
|
16
|
+
counter.hits["pages.read"].should == 2
|
17
|
+
end
|
18
|
+
|
19
|
+
it "should record a ping with key 'processor.alive'" do
|
20
|
+
Timecop.freeze do
|
21
|
+
counter.ping "processor.alive"
|
22
|
+
counter.pings.should have_key("processor.alive")
|
23
|
+
counter.pings["processor.alive"].strftime("%Y-%m-%d %H:%M:%S").should == Time.now.strftime("%Y-%m-%d %H:%M:%S")
|
24
|
+
end
|
25
|
+
|
26
|
+
target_time = Time.now + 9
|
27
|
+
Timecop.travel(target_time) do
|
28
|
+
counter.ping "processor.alive"
|
29
|
+
counter.pings.should have_key("processor.alive")
|
30
|
+
counter.pings["processor.alive"].strftime("%Y-%m-%d %H:%M:%S").should == target_time.strftime("%Y-%m-%d %H:%M:%S")
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
it "should record a latency with a passed value" do
|
35
|
+
counter.latency "processor.enqueue", 0.00012
|
36
|
+
counter.latencies["processor.enqueue"].should == [0.00012]
|
37
|
+
|
38
|
+
counter.latency "processor.enqueue", 0.00121
|
39
|
+
counter.latencies["processor.enqueue"].should == [0.00012, 0.00121]
|
40
|
+
end
|
41
|
+
|
42
|
+
it "should record a magnitude" do
|
43
|
+
counter.magnitude "processor.bytes_compressed", 9_312
|
44
|
+
counter.magnitude "processor.bytes_compressed", 8_271
|
45
|
+
counter.magnitudes["processor.bytes_compressed"].should == 8_271
|
46
|
+
end
|
47
|
+
end
|
@@ -0,0 +1,57 @@
|
|
1
|
+
require "spec_helper"
|
2
|
+
|
3
|
+
describe Counters::Redis do
|
4
|
+
let :redis do
|
5
|
+
double("redis")
|
6
|
+
end
|
7
|
+
|
8
|
+
let :counter do
|
9
|
+
Counters::Redis.new(redis, "counters")
|
10
|
+
end
|
11
|
+
|
12
|
+
it_should_behave_like "all counters"
|
13
|
+
|
14
|
+
it "should record a hit on 'pages.read' by HINCRBY counters/hits.pages.read" do
|
15
|
+
redis.should_receive(:hincrby).with("counters", "hits.pages.read", 1).twice
|
16
|
+
2.times { counter.hit "pages.read" }
|
17
|
+
end
|
18
|
+
|
19
|
+
it "should record a magnitude on 'bytes.in' by HSET counters/magnitudes.bytes.in" do
|
20
|
+
redis.should_receive(:hset).with("counters", "magnitudes.bytes.in", 309).once
|
21
|
+
redis.should_receive(:hset).with("counters", "magnitudes.bytes.in", 392).once
|
22
|
+
counter.magnitude "bytes.in", 309
|
23
|
+
counter.magnitude "bytes.in", 392
|
24
|
+
end
|
25
|
+
|
26
|
+
it "should record a ping on 'crawler' by HSET counters/pings.crawler with today's date/time as an int" do
|
27
|
+
Timecop.freeze do
|
28
|
+
redis.should_receive(:hset).with("counters", "pings.crawler", Time.now.utc.to_i).once
|
29
|
+
counter.ping "crawler"
|
30
|
+
end
|
31
|
+
|
32
|
+
target_time = Time.now + 9
|
33
|
+
Timecop.travel(target_time) do
|
34
|
+
redis.should_receive(:hset).with("counters", "pings.crawler", target_time.to_i).once
|
35
|
+
counter.ping "crawler"
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
it "should record latency on 'crawler.download' by HINCRBY counters/latencies.crawler.download.count by 1 and counters/latencies.crawler.download.nanoseconds by the latency" do
|
40
|
+
redis.should_receive(:hincrby).with("counters", "latencies.crawler.download.count", 1).twice
|
41
|
+
redis.should_receive(:hincrby).with("counters", "latencies.crawler.download.nanoseconds", 1.90 * 1_000_000_000).once
|
42
|
+
redis.should_receive(:hincrby).with("counters", "latencies.crawler.download.nanoseconds", 2.02 * 1_000_000_000).once
|
43
|
+
|
44
|
+
counter.latency "crawler.download", 1.9
|
45
|
+
counter.latency "crawler.download", 2.02
|
46
|
+
end
|
47
|
+
|
48
|
+
it "should record a block's latency" do
|
49
|
+
redis.should_receive(:hincrby).with("counters", "latencies.crawler.process.count", 1).once
|
50
|
+
redis.should_receive(:hincrby).once.with do |key, subkey, latency|
|
51
|
+
key == "counters" && subkey == "latencies.crawler.process.nanoseconds" && latency >= 0.2 * 1_000_000_000 && latency < 0.3 * 1_000_000_000
|
52
|
+
end
|
53
|
+
counter.latency "crawler.process" do
|
54
|
+
sleep 0.2
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1,26 @@
|
|
1
|
+
require "counters"
|
2
|
+
require "timecop"
|
3
|
+
|
4
|
+
shared_examples_for "all counters" do
|
5
|
+
it "should raise a ArgumentError when the key includes invalid chars" do
|
6
|
+
lambda { counter.hit "hit!" } .should raise_error(ArgumentError)
|
7
|
+
lambda { counter.hit "hit counter" } .should raise_error(ArgumentError)
|
8
|
+
lambda { counter.hit "boy.hit?" } .should raise_error(ArgumentError)
|
9
|
+
lambda { counter.hit "hit/a" } .should raise_error(ArgumentError)
|
10
|
+
lambda { counter.hit "hit-a" } .should raise_error(ArgumentError)
|
11
|
+
lambda { counter.hit "" } .should raise_error(ArgumentError)
|
12
|
+
lambda { counter.hit nil } .should raise_error(ArgumentError)
|
13
|
+
end
|
14
|
+
|
15
|
+
it "should not raise ArgumentError when the key includes a number" do
|
16
|
+
lambda { counter.hit "hit1" }.should_not raise_error(ArgumentError)
|
17
|
+
end
|
18
|
+
|
19
|
+
it "should not raise ArgumentError when the key includes a dot / fullstop" do
|
20
|
+
lambda { counter.hit "hit." }.should_not raise_error(ArgumentError)
|
21
|
+
end
|
22
|
+
|
23
|
+
it "should not raise ArgumentError when the key includes an underscore" do
|
24
|
+
lambda { counter.hit "hit_" }.should_not raise_error(ArgumentError)
|
25
|
+
end
|
26
|
+
end
|
metadata
ADDED
@@ -0,0 +1,110 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: counters
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
prerelease: false
|
5
|
+
segments:
|
6
|
+
- 1
|
7
|
+
- 0
|
8
|
+
- 0
|
9
|
+
version: 1.0.0
|
10
|
+
platform: ruby
|
11
|
+
authors:
|
12
|
+
- "Fran\xC3\xA7ois Beausoleil"
|
13
|
+
autorequire:
|
14
|
+
bindir: bin
|
15
|
+
cert_chain: []
|
16
|
+
|
17
|
+
date: 2011-02-26 00:00:00 -05:00
|
18
|
+
default_executable:
|
19
|
+
dependencies:
|
20
|
+
- !ruby/object:Gem::Dependency
|
21
|
+
name: rspec
|
22
|
+
prerelease: false
|
23
|
+
requirement: &id001 !ruby/object:Gem::Requirement
|
24
|
+
none: false
|
25
|
+
requirements:
|
26
|
+
- - ">="
|
27
|
+
- !ruby/object:Gem::Version
|
28
|
+
segments:
|
29
|
+
- 0
|
30
|
+
version: "0"
|
31
|
+
type: :development
|
32
|
+
version_requirements: *id001
|
33
|
+
- !ruby/object:Gem::Dependency
|
34
|
+
name: timecop
|
35
|
+
prerelease: false
|
36
|
+
requirement: &id002 !ruby/object:Gem::Requirement
|
37
|
+
none: false
|
38
|
+
requirements:
|
39
|
+
- - ">="
|
40
|
+
- !ruby/object:Gem::Version
|
41
|
+
segments:
|
42
|
+
- 0
|
43
|
+
version: "0"
|
44
|
+
type: :development
|
45
|
+
version_requirements: *id002
|
46
|
+
description: Using the provided API, record metrics (such as number of hits to a particular controller, bytes in/out, compression ratio) within your system. Visualization is NOT provided within this gem.
|
47
|
+
email:
|
48
|
+
- francois@teksol.info
|
49
|
+
executables: []
|
50
|
+
|
51
|
+
extensions: []
|
52
|
+
|
53
|
+
extra_rdoc_files: []
|
54
|
+
|
55
|
+
files:
|
56
|
+
- .gitignore
|
57
|
+
- .rvmrc
|
58
|
+
- Gemfile
|
59
|
+
- LICENSE
|
60
|
+
- README.textile
|
61
|
+
- Rakefile
|
62
|
+
- autotest/discover.rb
|
63
|
+
- counters.gemspec
|
64
|
+
- lib/counters.rb
|
65
|
+
- lib/counters/base.rb
|
66
|
+
- lib/counters/file.rb
|
67
|
+
- lib/counters/memory.rb
|
68
|
+
- lib/counters/redis.rb
|
69
|
+
- lib/counters/version.rb
|
70
|
+
- spec/file_counter_spec.rb
|
71
|
+
- spec/memory_counter_spec.rb
|
72
|
+
- spec/redis_counter_spec.rb
|
73
|
+
- spec/spec_helper.rb
|
74
|
+
has_rdoc: true
|
75
|
+
homepage: ""
|
76
|
+
licenses: []
|
77
|
+
|
78
|
+
post_install_message:
|
79
|
+
rdoc_options: []
|
80
|
+
|
81
|
+
require_paths:
|
82
|
+
- lib
|
83
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
84
|
+
none: false
|
85
|
+
requirements:
|
86
|
+
- - ">="
|
87
|
+
- !ruby/object:Gem::Version
|
88
|
+
segments:
|
89
|
+
- 0
|
90
|
+
version: "0"
|
91
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
92
|
+
none: false
|
93
|
+
requirements:
|
94
|
+
- - ">="
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
segments:
|
97
|
+
- 0
|
98
|
+
version: "0"
|
99
|
+
requirements: []
|
100
|
+
|
101
|
+
rubyforge_project:
|
102
|
+
rubygems_version: 1.3.7
|
103
|
+
signing_key:
|
104
|
+
specification_version: 3
|
105
|
+
summary: Provides an API to record any kind of metrics within your system
|
106
|
+
test_files:
|
107
|
+
- spec/file_counter_spec.rb
|
108
|
+
- spec/memory_counter_spec.rb
|
109
|
+
- spec/redis_counter_spec.rb
|
110
|
+
- spec/spec_helper.rb
|