all_seeing_eye 0.0.19 → 0.0.20

data/Gemfile

@@ -2,7 +2,6 @@ source "http://rubygems.org"
 
 gem 'redis'
 gem 'chronic'
-gem 'rbtree'
 gem 'sinatra'
 gem 'vegas'
 gem 'activesupport', '>=2.3.12'

data/Gemfile.lock

@@ -13,7 +13,6 @@ GEM
       rake
     rack (1.2.3)
     rake (0.9.2)
-    rbtree (0.3.0)
     rcov (0.9.10)
     redis (2.2.2)
     sinatra (1.2.6)
@@ -36,7 +35,6 @@ DEPENDENCIES
   chronic
   i18n
   jeweler (~> 1.6.4)
-  rbtree
   rcov
   redis
   sinatra

data/README.markdown

@@ -0,0 +1,120 @@
+# AllSeeingEye
+
+A highly configurable Gem to collect specific data from requests, and view that data in a cute little Sinatra app.
+
+AllSeeingEye writes information about each request it observes into Redis. What kind of information? Why, whatever 
+you want! From the relatively banal (IP address, URI, user agent) to complicated application-specific data 
+(user account, ordered product, money spent), anything can be recorded by AllSeeingEye, stored in Redis, and indexed to be 
+viewed quickly in Sinatra. With pretty Flot-based graphs, even.
+
+It comes with a Rails2 integration by default, but I think with a little work it'd be pretty easy to get it into Rails3. 
+
+## Background
+
+Our application receives about half a million requests every day. While we have Google Analytics, and it's totally awesome, 
+many of those requests come from our iPhone app, and those requests don't receive the level of tracking we need. We could load 
+reporting software onto a users' iOS device: but, ethical issues aside, why would we want to? Almost anything we'd be 
+interested in tracking requires server interaction anyway, so it's easier and faster to track it all on the server side.
+
+The obvious solution was to mine the Rails logfiles, but we have multiple servers and weaving all the logs was a pain -- and 
+even when we did, some of the data we wanted wasn't exposed. My next thought was to log requests to our database, but we 
+experience enough database traffic as is without writing every request. A MemCached-backed write-through buffer temporarily 
+worked but when the writes happened the database still got very unhappy... and the table got enormous very quickly.
+
+Redis was a good compromise though. We use it for the truly excellent Resque, so it's already part of our stack. It's fast 
+and, while it's persistent, it's not exactly a tragedy if the Redis DB gets hosed, as none of the request data is so important 
+we can't just start collecting it again. Combined with some code to store arbitrary data, we quickly had a winner on our hands.
+
+So, AllSeeingEye was born out of our need to track incoming requests accurately, quickly, and in a way that is easily 
+displayable to our business users.
+
+## Installation
+
+Get Redis via the amazing Homebrew: `brew install redis`
+
+Add AllSeeingEye to your project's Gemfile: `gem 'all\_seeing\_eye'`
+
+Install it with Bundler: `bundle install`
+
+## Setup
+
+  bundle exec ruby script/generate all\_seeing\_eye
+
+This puts the AllSeeingEye config file at config/all\_seeing\_eye.yml. The file is annotated with all the information you 
+need to get started, but some quick explanation:
+
+### Timeout
+AllSeeingEye's data collection is wrapped by a timeout to ensure speediness. I keep this value at 30 milliseconds (0.03), but 
+feel free to change it to suit your application's needs.
+
+### Round To Seconds
+One data point per request is too many to sensibly view on a graph. This rounds the created\_at time of requests 
+to whatever (I choose 10 minutes), so that all requests within 10 minutes are collected and displayed together.
+
+### Redis
+Environment and then location of the redis server. If you do not include this key, AllSeeingEye tries to look for a redis.yml 
+or a resque.yml file instead to find the server.
+
+### Models
+You can define a number of different models to collect data with. Each has its own separate namespace, so data won't be 
+shared between them. I only use one right now (I call it 'request'), but if you want more then go for it.
+
+Each model definition should be structured like this:
+
+  model\_name:
+    first\_field\_name:
+      object: object_name
+      method: .method_name
+    second\_field\_name:
+      object: hash
+      method: "['key']"
+
+Model names and field names are completely arbitrary, and I try to humanize them as best as I can.
+
+Object and method are the literal object and the actual method you want executed, with one exception: the object 
+'controller' is translated into 'self'. The result of this is stored by AllSeeingEye in redis. So, for example, an 
+object could be "request" and the method could be "request\_uri". I'm trying to do a little secure sanity-checking here 
+rather than just directly eval'ing a string, but you can still mess things up pretty badly if you choose a bad object 
+and/or bad method... so try not to do that.
+
+As a helpful hint, recording "request\_uri" isn't nearly as helpful as you might think. Try to save requests without any params 
+and without a leading slash by defining a method in ApplicationController instead:
+
+  protected
+  def clean_request
+    request.request_uri.split('?').first[1..-1]
+  end 
+
+Then, in your all\_seeing\_eye.yml:
+  
+  request:
+    uri:
+      object: controller
+      method: .clean_request
+
+## Running
+
+When your Rails application starts, you should see each request contain "+++ Request watched by AllSeeingEye." That means 
+AllSeeingEye is running successfully. Good job!
+
+Starting the server is simple. Just type `all-seeing-eye` from the commandline and Sinatra should start automatically.
+
+## Extras
+
+I included the Nginx config we use for AllSeeingEye as a small bonus. It's in examples/all\_seeing\_eye-nginx.conf.
+
+## Contributing to AllSeeingEye
+ 
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
+* Fork the project
+* Start a feature/bugfix branch
+* Commit and push until you are happy with your contribution
+* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
+* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
+
+## Copyright
+
+Copyright (c) 2011 Josh Symonds. See LICENSE.txt for
+further details.
+

data/VERSION

@@ -1 +1 @@
-0.0.19
+0.0.20

data/all_seeing_eye.gemspec

@@ -5,33 +5,35 @@
 
 Gem::Specification.new do |s|
   s.name = %q{all_seeing_eye}
-  s.version = "0.0.19"
+  s.version = "0.0.20"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Josh Symonds"]
-  s.date = %q{2011-08-15}
+  s.date = %q{2011-08-16}
   s.default_executable = %q{all-seeing-eye}
   s.description = %q{AllSeeingEye observes all requests, with parameters that you specify, to Redis. Then it composes them into graphs for you to see.}
   s.email = %q{veraticus@gmail.com}
   s.executables = ["all-seeing-eye"]
   s.extra_rdoc_files = [
     "LICENSE.txt",
-    "README.rdoc"
+    "README.markdown"
   ]
   s.files = [
     "Gemfile",
     "Gemfile.lock",
     "LICENSE.txt",
-    "README.rdoc",
+    "README.markdown",
     "Rakefile",
     "VERSION",
     "all_seeing_eye.gemspec",
     "bin/all-seeing-eye",
+    "examples/all_seeing_eye-nginx.conf",
     "init.rb",
     "lib/all_seeing_eye.rb",
+    "lib/all_seeing_eye/generators/all_seeing_eye.yml",
+    "lib/all_seeing_eye/generators/all_seeing_eye_generator.rb",
     "lib/all_seeing_eye/integrations/rails2.rb",
     "lib/all_seeing_eye/model.rb",
-    "lib/all_seeing_eye/models/request.rb",
     "lib/all_seeing_eye/server.rb",
     "lib/all_seeing_eye/server/public/application.js",
     "lib/all_seeing_eye/server/public/eye.png",
@@ -63,7 +65,6 @@ Gem::Specification.new do |s|
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
       s.add_runtime_dependency(%q<redis>, [">= 0"])
       s.add_runtime_dependency(%q<chronic>, [">= 0"])
-      s.add_runtime_dependency(%q<rbtree>, [">= 0"])
       s.add_runtime_dependency(%q<sinatra>, [">= 0"])
       s.add_runtime_dependency(%q<vegas>, [">= 0"])
       s.add_runtime_dependency(%q<activesupport>, [">= 2.3.12"])
@@ -75,7 +76,6 @@ Gem::Specification.new do |s|
     else
       s.add_dependency(%q<redis>, [">= 0"])
       s.add_dependency(%q<chronic>, [">= 0"])
-      s.add_dependency(%q<rbtree>, [">= 0"])
       s.add_dependency(%q<sinatra>, [">= 0"])
       s.add_dependency(%q<vegas>, [">= 0"])
       s.add_dependency(%q<activesupport>, [">= 2.3.12"])
@@ -88,7 +88,6 @@ Gem::Specification.new do |s|
   else
     s.add_dependency(%q<redis>, [">= 0"])
     s.add_dependency(%q<chronic>, [">= 0"])
-    s.add_dependency(%q<rbtree>, [">= 0"])
     s.add_dependency(%q<sinatra>, [">= 0"])
     s.add_dependency(%q<vegas>, [">= 0"])
     s.add_dependency(%q<activesupport>, [">= 2.3.12"])

data/examples/all_seeing_eye-nginx.conf

@@ -0,0 +1,12 @@
+location /all_seeing_eye
+{
+  rewrite /all_seeing_eye/(.*) /$1 break;
+
+  proxy_set_header Host $host:1234;
+  proxy_set_header X-Real-IP $remote_addr;
+  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+  proxy_set_header X-Forwarded-Ssl on;
+  proxy_set_header X-Forwarded-Proto https;
+  
+  proxy_pass http://all_seeing_eye_server:all_seeing_eye_port;
+}

data/lib/all_seeing_eye/generators/all_seeing_eye.yml

@@ -0,0 +1,45 @@
+## Timeout
+#  The length of time AllSeeingEye has to complete everything it does. So, at maximum,
+#  AllSeeingEye will add this amount of time to each of your requests.
+timeout: 0.03 
+
+## Round To Seconds
+#  A data point for every request is too much to display sensibly. 
+#  10 minutes makes more sense here.
+round_to_seconds: 600
+
+## Redis
+#  The host and port of the redis server. We use database 7 for AllSeeingEye, so make sure that 
+#  redis can accept up to that many databases.
+redis:
+  development: localhost:6379
+
+## Define each model you want to track here.
+#  Model names should be the root of the hash, with each attribute following.
+#  Attributes should have a unique name and a method of acquiring them -- specifically, 
+#  an object and a method on that object. A couple objects you're probably 
+#  interested in:
+#    request: the request object from the controller
+#    response: the response object from the controller
+#    params: the parameter hash
+#    controller: the controller itself ("self")
+#  But anything works. Be careful with what you put here.
+request:
+  ip:
+    object: request
+    method: .remote_ip
+  uri:
+    object: request
+    method: .request_uri
+  browser:
+    object: request
+    method: .env["HTTP_USER_AGENT"]
+  controller:
+    object: params
+    method: '[:controller]'
+  action:
+    object: params
+    method: '[:action]'
+  status:
+    object: response
+    method: .status

data/lib/all_seeing_eye/generators/all_seeing_eye_generator.rb

@@ -0,0 +1,7 @@
+class AllSeeingEyeGenerator < Rails::Generator::Base
+  def manifest
+    record do |m|
+      m.template('all_seeing_eye.yml', "config/initializers/all_seeing_eye.yml")
+    end
+  end
+end

data/lib/all_seeing_eye/model.rb

@@ -129,32 +129,6 @@ class AllSeeingEye::Model
     end
   end
   
-  def self.conglomerate(array, options = {})
-    return [] if array.empty?
-    
-    options = {:granularity => 100}.merge(options)
-    if !array.first.score.nil?
-      array = array.sort{|a, b| a.score <=> b.score} 
-    else
-      array = array.sort{|a, b| a.created_at <=> b.created_at}
-    end
-    start = array.first.created_at.utc.to_i
-    stop = array.last.created_at.utc.to_i
-    interval = (stop - start) / options[:granularity]
-    interval = 1 if interval == 0
-    
-    tree = RBTree.new
-    options[:granularity].times {|n| tree[start + (interval * n)] = 0}
-    
-    array.each do |o|
-      val = o.created_at.utc.to_i
-      bound = tree.lower_bound(val)
-      bound = tree.upper_bound(val) if bound.nil?
-      tree[bound.first] += 1
-    end
-    tree.to_a
-  end
-  
   def initialize
     self.class.field_names.each do |key|
       self.class.send(:attr_accessor, key.to_sym)
@@ -167,7 +141,7 @@ class AllSeeingEye::Model
   end
   
   def save
-    timestamp = (self.created_at.to_i / 60) * 60
+    timestamp = (self.created_at.to_i / AllSeeingEye.configuration[:all_seeing_eye]['round_to_seconds']) * AllSeeingEye.configuration[:all_seeing_eye]['round_to_seconds']
     
     AllSeeingEye.redis["allseeingeye:#{self.class.model_name}:#{id}"] = Marshal.dump(self)
     AllSeeingEye.redis.sadd("allseeingeye:#{self.class.model_name}:all", id)

data/lib/all_seeing_eye/server/views/total.erb

@@ -10,7 +10,7 @@
   <h1>Total Requests</h1>
 <% end %>
 
-<% if @counts.empty? %>
+<% if @counts.nil? || @counts.empty? %>
   <h2>No requests found.</h2>
 <% else %>
   <div id='graph'></div>

data/lib/all_seeing_eye/server.rb

@@ -83,13 +83,6 @@ class AllSeeingEye
       redirect url_path("/search/#{CGI::escape(params[:query])}")
     end
     
-    get %r{/search/(.*?)$} do
-      @field = 'total'
-      @query = params[:captures].first
-      @counts = AllSeeingEye::Request.search(@query, :count => true)
-      show :total
-    end
-    
     get "/fields" do
       redirect url_path("/fields/#{AllSeeingEye::Request.field_keys.first}")
     end
@@ -104,6 +97,7 @@ class AllSeeingEye
       end
       
       get %r{/search/(.*?)/#{field}$} do
+        @page = params[:page].blank? ? 1 : params[:page].to_i
         @field = field
         @query = params[:captures].first
         @requests = AllSeeingEye::Request.search(@query)
@@ -115,6 +109,7 @@ class AllSeeingEye
 
       (AllSeeingEye::Request.field_keys - [field]).each do |f|
         get %r{/fields/#{field}/(.*?)/#{f}$} do
+          @page = params[:page].blank? ? 1 : params[:page].to_i
           @field = field
           @id = params[:captures].first
           @view = f
@@ -133,6 +128,13 @@ class AllSeeingEye
         show :total
       end
     end
+    
+    get %r{/search/(.*?)$} do
+      @field = 'total'
+      @query = params[:captures].first
+      @counts = AllSeeingEye::Request.search(@query, :count => true)
+      show :total
+    end
 
     def tabs
       @tabs ||= AllSeeingEye::Request.field_keys

data/lib/all_seeing_eye.rb

@@ -6,7 +6,11 @@ class AllSeeingEye
   def self.configuration
     location = ENV['ALL_SEEING_EYE_CONFIG'] || 'config'
     @@configuration ||= Hash.new
+    @@configuration[:all_seeing_eye] ||= YAML::load_file("./#{location}/all_seeing_eye.yml")
     @@configuration[:redis] ||=
+    if @@configuration[:all_seeing_eye]['redis'] && @@configuration[:all_seeing_eye]['redis'][AllSeeingEye.environment]
+      @@configuration[:all_seeing_eye]['redis'][AllSeeingEye.environment]
+    else
       begin
         YAML::load_file("./#{location}/resque.yml")[AllSeeingEye.environment]
       rescue
@@ -16,23 +20,37 @@ class AllSeeingEye
           'localhost:6379'
         end
       end
-    @@configuration[:all_seeing_eye] ||= YAML::load_file("./#{location}/all_seeing_eye.yml")
+    end
+    @@configuration[:models] ||= []
+    @@configuration[:all_seeing_eye].each do |key, value|
+      next if self.default_keys.include?(key) || AllSeeingEye.const_defined?(key.capitalize.to_sym)
+      klass = Class.new(AllSeeingEye::Model)
+      AllSeeingEye.const_set(key.capitalize.to_sym, klass) 
+      @@configuration[:models] << klass
+    end
     @@configuration
   end
   
+  def self.default_keys
+    ['timeout', 'round_to_seconds', 'redis']
+  end
+  
   def self.redis
     @@redis ||= Redis.new(:host => AllSeeingEye.configuration[:redis].split(':').first,
                           :port => AllSeeingEye.configuration[:redis].split(':').last,
                           :db => 7)
   end
+  
+  require 'redis'
+  require 'yaml'
+  require 'chronic'
+  require 'active_support'
+  require 'active_support/all'
+  require 'all_seeing_eye/model'
+  if defined?(Rails)
+    require 'all_seeing_eye/integrations/rails2' 
+    require 'all_seeing_eye/generators/all_seeing_eye_generator'
+  end
+  
+  self.configuration
 end
-
-require 'redis'
-require 'yaml'
-require 'chronic'
-require 'rbtree'
-require 'active_support'
-require 'active_support/all'
-require 'all_seeing_eye/model'
-require 'all_seeing_eye/models/request'
-require 'all_seeing_eye/integrations/rails2' if defined?(Rails)

data/test/base_test.rb

@@ -6,10 +6,23 @@ context 'with the base class' do
   end
   
   test 'load the configuration' do
-    assert AllSeeingEye.configuration[:all_seeing_eye]['model']
+    assert AllSeeingEye.configuration[:all_seeing_eye]['request']
+    assert !AllSeeingEye.configuration[:models].empty?
+    assert AllSeeingEye.configuration[:redis]
   end
   
   test 'connect to Redis' do
     assert_equal '1', AllSeeingEye.redis.info["connected_clients"]
   end
+  
+  test 'loads new classes from YML file' do
+    assert AllSeeingEye.const_defined?(:Request)
+    assert AllSeeingEye.const_defined?(:Response)
+    assert AllSeeingEye.configuration[:models].include?(AllSeeingEye::Request)
+    assert AllSeeingEye.configuration[:models].include?(AllSeeingEye::Response)
+  end
+  
+  test 'do not load default keys as classes' do
+    assert !AllSeeingEye.const_defined?(:Timeout)
+  end
 end

data/test/fixtures/all_seeing_eye.yml

@@ -1,5 +1,6 @@
 timeout: 0.01
-model:
+round_to_seconds: 60
+request:
   ip:
     object: request
     method: .remote_ip
@@ -12,10 +13,16 @@ model:
   controller:
     object: params
     method: '[:controller]'
-    params: 
   action:
     object: params
     method: '[:action]'
   status:
     object: response
     method: .status
+response:
+  ip:
+    object: request
+    method: .remote_ip
+  uri:
+    object: request
+    method: .request_uri

data/test/model_test.rb

@@ -6,27 +6,27 @@ context 'class methods without setup' do
   end
   
   test 'models know their own name' do
-    assert_equal 'model', AllSeeingEye::Model.model_name
+    assert_equal 'request', AllSeeingEye::Request.model_name
   end
   
   test 'models correctly find their next ID' do
-    id = AllSeeingEye::Model.next_id
-    (id + 1..(id + 11)).each {|n|assert_equal AllSeeingEye::Model.next_id, n}
+    id = AllSeeingEye::Request.next_id
+    (id + 1..(id + 11)).each {|n|assert_equal AllSeeingEye::Request.next_id, n}
   end
   
   test 'models should get fields' do
-    assert_equal AllSeeingEye::Model.fields,
-      {"uri"=>{"method"=>".request_uri", "object"=>"request"}, "action"=>{"method"=>"[:action]", "object"=>"params"}, "ip"=>{"method"=>".remote_ip", "object"=>"request"}, "controller"=>{"method"=>"[:controller]", "params"=>nil, "object"=>"params"}, "browser"=>{"method"=>".env[\"HTTP_USER_AGENT\"]", "object"=>"request"}, "status"=>{"method"=>".status", "object"=>"response"}}
+    assert_equal AllSeeingEye::Request.fields,
+      {"uri"=>{"method"=>".request_uri", "object"=>"request"}, "action"=>{"method"=>"[:action]", "object"=>"params"}, "ip"=>{"method"=>".remote_ip", "object"=>"request"}, "controller"=>{"method"=>"[:controller]", "object"=>"params"}, "browser"=>{"method"=>".env[\"HTTP_USER_AGENT\"]", "object"=>"request"}, "status"=>{"method"=>".status", "object"=>"response"}}
   end
   
   test 'models should get field names' do
     assert_equal ["uri", "action", "ip", "controller", "browser", "status", "time_spent", "created_at"],
-      AllSeeingEye::Model.field_names
+      AllSeeingEye::Request.field_names
   end
   
   test 'return a new model object' do
     time = Time.now
-    obj = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => time)
+    obj = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => time)
     
     assert_equal '/test/', obj.uri
     assert_equal '127.0.0.1', obj.ip
@@ -35,22 +35,22 @@ context 'class methods without setup' do
   end
   
   test 'create a model object' do
-    obj = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1')
+    obj = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1')
     
     assert_equal '/test/', obj.uri
     assert_equal '127.0.0.1', obj.ip
-    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:model:#{AllSeeingEye::Model.next_id - 1}"]
-    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:model:fields:uri", '/test/')
-    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:model:fields:ip", '127.0.0.1')
-    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:model:fields:uri:/test/", :by => 'nosort')
-    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:model:fields:ip:127.0.0.1", :by => 'nosort')
+    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:request:#{AllSeeingEye::Request.next_id - 1}"]
+    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:request:fields:uri", '/test/')
+    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:request:fields:ip", '127.0.0.1')
+    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:request:fields:uri:/test/", :by => 'nosort')
+    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:request:fields:ip:127.0.0.1", :by => 'nosort')
   end
   
   test 'unmarshal an object with load' do
     time = Time.now
-    raw = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => time)
+    raw = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => time)
     dump = Marshal.dump(raw)
-    obj = AllSeeingEye::Model.load(dump)
+    obj = AllSeeingEye::Request.load(dump)
     
     assert_equal '/test/', obj.uri
     assert_equal '127.0.0.1', obj.ip
@@ -60,26 +60,26 @@ context 'class methods without setup' do
   
   test 'find an object by its ID' do
     time = Time.now
-    obj = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => time)
-    found = AllSeeingEye::Model.find(obj.id)
+    obj = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => time)
+    found = AllSeeingEye::Request.find(obj.id)
     
     [:uri, :ip, :created_at].each {|f| assert_equal obj.send(f), found.send(f)}
   end
   
   test 'find all objects' do
-    obj1 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
-    obj2 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
-    obj3 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
+    obj1 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
+    obj2 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
+    obj3 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
     
-    assert_equal [obj1.id, obj2.id, obj3.id], AllSeeingEye::Model.all.collect(&:id)
+    assert_equal [obj1.id, obj2.id, obj3.id], AllSeeingEye::Request.all.collect(&:id)
   end
   
   test 'find the count of all objects' do
-    obj1 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
-    obj2 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
-    obj3 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
+    obj1 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
+    obj2 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
+    obj3 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => Time.now)
     
-    assert_equal 3, AllSeeingEye::Model.count
+    assert_equal 3, AllSeeingEye::Request.count
   end
 end
 
@@ -87,12 +87,12 @@ context 'class methods with setup' do
   setup do
     @time = Time.now
     
-    @obj1 = AllSeeingEye::Model.create(:uri => '/', :ip => '127.0.0.1', :created_at => @time - 10 * 60)
-    @obj2 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => @time - 5 * 60)
-    @obj3 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => @time - 3 * 60)
-    @obj4 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '192.168.0.1', :created_at => @time - 1 * 60)
-    @obj5 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => @time + 2 * 60)
-    @obj6 = AllSeeingEye::Model.create(:uri => '/test/', :ip => '192.168.0.1', :created_at => @time + 2 * 60)
+    @obj1 = AllSeeingEye::Request.create(:uri => '/', :ip => '127.0.0.1', :created_at => @time - 10 * 60)
+    @obj2 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => @time - 5 * 60)
+    @obj3 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => @time - 3 * 60)
+    @obj4 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '192.168.0.1', :created_at => @time - 1 * 60)
+    @obj5 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '127.0.0.1', :created_at => @time + 2 * 60)
+    @obj6 = AllSeeingEye::Request.create(:uri => '/test/', :ip => '192.168.0.1', :created_at => @time + 2 * 60)
   end
   
   teardown do
@@ -100,7 +100,7 @@ context 'class methods with setup' do
   end
   
   test 'find all objects in a date range with a start and a stop' do
-    results = AllSeeingEye::Model.all(:start => @time - 11 * 60, :stop => @time - 4 * 60)
+    results = AllSeeingEye::Request.all(:start => @time - 11 * 60, :stop => @time - 4 * 60)
     
     assert_equal 2, results.count
     assert_equal @obj1.id, results[0].id
@@ -108,7 +108,7 @@ context 'class methods with setup' do
   end
   
   test 'find all objects in a date range with only a start' do
-    results = AllSeeingEye::Model.all(:start => @time)
+    results = AllSeeingEye::Request.all(:start => @time)
     
     assert_equal 2, results.count
     assert_equal @obj5.id, results[0].id
@@ -116,7 +116,7 @@ context 'class methods with setup' do
   end
   
   test 'find all objects in a date range with only a stop' do
-    results = AllSeeingEye::Model.all(:stop => @time)
+    results = AllSeeingEye::Request.all(:stop => @time)
     
     assert_equal 4, results.count
     assert_equal @obj1.id, results[0].id
@@ -126,7 +126,7 @@ context 'class methods with setup' do
   end
   
   test 'find model objects by a string field' do
-    results = AllSeeingEye::Model.find_by_field('uri', :value => '/test/')
+    results = AllSeeingEye::Request.find_by_field('uri', :value => '/test/')
 
     assert_equal 5, results.count
     assert results.collect(&:id).include?(@obj2.id)
@@ -137,7 +137,7 @@ context 'class methods with setup' do
   end
   
   test 'find model objects by a string field with a start' do
-    results = AllSeeingEye::Model.find_by_field('uri', :value => '/test/', :start => @time - 4 * 60)
+    results = AllSeeingEye::Request.find_by_field('uri', :value => '/test/', :start => @time - 4 * 60)
     
     assert_equal 4, results.count
     assert results.collect(&:id).include?(@obj3.id)
@@ -147,7 +147,7 @@ context 'class methods with setup' do
   end
   
   test 'find model objects by a string field with a stop' do
-    results = AllSeeingEye::Model.find_by_field('uri', :value => '/test/', :stop => @time)
+    results = AllSeeingEye::Request.find_by_field('uri', :value => '/test/', :stop => @time)
     
     assert_equal 3, results.count
     assert results.collect(&:id).include?(@obj2.id)
@@ -156,7 +156,7 @@ context 'class methods with setup' do
   end
   
   test 'find model objects by a string field with a start and a stop' do
-    results = AllSeeingEye::Model.find_by_field('uri', :value => '/test/', :start => @time - 4 * 60, :stop => @time)
+    results = AllSeeingEye::Request.find_by_field('uri', :value => '/test/', :start => @time - 4 * 60, :stop => @time)
     
     assert_equal 2, results.count
     assert results.collect(&:id).include?(@obj3.id)
@@ -168,7 +168,7 @@ context 'class methods with setup' do
   end
   
   test 'find model objects by a date field' do
-    results = AllSeeingEye::Model.find_by_field('created_at', :start => @obj1.created_at, :stop => @obj4.created_at)
+    results = AllSeeingEye::Request.find_by_field('created_at', :start => @obj1.created_at, :stop => @obj4.created_at)
     
     assert_equal 4, results.count
     assert_equal results.first.id, @obj1.id
@@ -176,19 +176,19 @@ context 'class methods with setup' do
   end
   
   test 'count model objects' do
-    assert_equal 6, AllSeeingEye::Model.count
+    assert_equal 6, AllSeeingEye::Request.count
   end
   
   test 'return the first object' do
-    assert_equal @obj1.id, AllSeeingEye::Model.first.id
+    assert_equal @obj1.id, AllSeeingEye::Request.first.id
   end
   
   test 'return the last object' do
-    assert_equal @obj6.id, AllSeeingEye::Model.last.id
+    assert_equal @obj6.id, AllSeeingEye::Request.last.id
   end
   
   test 'return count in a field' do
-    counts = AllSeeingEye::Model.list_by_field('uri')
+    counts = AllSeeingEye::Request.list_by_field('uri')
 
     assert_equal '/test/', counts.first.first
     assert_equal 5, counts.first.last
@@ -197,7 +197,7 @@ context 'class methods with setup' do
   end
   
   test 'search with Chronic' do
-    results = AllSeeingEye::Model.search('last friday to five minutes ago')
+    results = AllSeeingEye::Request.search('last friday to five minutes ago')
     
     assert_equal 2, results.size
     assert_equal @obj1.id, results[0].id
@@ -205,7 +205,7 @@ context 'class methods with setup' do
   end
   
   test 'search by a specific field' do
-    results = AllSeeingEye::Model.search('uri:/test/')
+    results = AllSeeingEye::Request.search('uri:/test/')
     
     assert_equal 5, results.count
     assert results.collect(&:id).include?(@obj2.id)
@@ -216,7 +216,7 @@ context 'class methods with setup' do
   end
   
   test 'search for a count on a specific field' do
-    results = AllSeeingEye::Model.search('uri:/test/', :count => true)
+    results = AllSeeingEye::Request.search('uri:/test/', :count => true)
     
     assert_equal 4, results.count
     assert_equal results.last, [((@obj6.created_at.to_i / 60) * 60).to_s, 2]
@@ -229,36 +229,27 @@ context 'many objects' do
     @time = Time.now
     
     2000.times do |n|
-      instance_variable_set("@obj#{n}".to_sym, AllSeeingEye::Model.create(:uri => '/', :ip => n % 2 == 0 ? '127.0.0.1' : '192.168.0.1', :created_at => @time - (n * (1999 - n)) * 60))
+      instance_variable_set("@obj#{n}".to_sym, AllSeeingEye::Request.create(:uri => '/', :ip => n % 2 == 0 ? '127.0.0.1' : '192.168.0.1', :created_at => @time - (n * (1999 - n)) * 60))
     end
   end
-  
-  test 'conglomerate an array successfully' do
-    results = AllSeeingEye::Model.all
-    counts = AllSeeingEye::Model.conglomerate(results)
     
-    assert_equal 100, counts.count
-    assert_equal 2000, counts.inject(0) {|sum, k| sum + k.last}
-    assert_equal @obj999.created_at.to_i, counts.first.first
-  end
-  
   test 'get the created count' do
-    counts = AllSeeingEye::Model.list_by_field('created_at')
+    counts = AllSeeingEye::Request.list_by_field('created_at')
     
-    assert_equal counts, AllSeeingEye::Model.search('ten years ago to one year from now', :count => true)
+    assert_equal counts, AllSeeingEye::Request.search('ten years ago to one year from now', :count => true)
     assert_equal (@obj999.created_at.to_i / 60) * 60, counts.first.first.to_i
     assert_equal (@obj0.created_at.to_i / 60) * 60, counts.last.first.to_i
     assert_equal 2000, counts.inject(0) {|sum, k| sum + k.last}
   end
   
   test 'get the created count for a value' do
-    counts = AllSeeingEye::Model.list_by_field('ip', :value => '192.168.0.1')
+    counts = AllSeeingEye::Request.list_by_field('ip', :value => '192.168.0.1')
     
     assert_equal 1000, counts.inject(0) {|sum, k| sum + k.last}
   end
   
   test 'get only the first page of the count' do
-    counts = AllSeeingEye::Model.list_by_field('created_at', :page => 1)
+    counts = AllSeeingEye::Request.list_by_field('created_at', :page => 1)
     
     assert_equal 10, counts.size
     assert_equal (@obj999.created_at.to_i / 60) * 60, counts.first.first.to_i
@@ -266,7 +257,7 @@ context 'many objects' do
   end
   
   test 'get only the second page of the count' do
-    counts = AllSeeingEye::Model.list_by_field('created_at', :page => 2)
+    counts = AllSeeingEye::Request.list_by_field('created_at', :page => 2)
     
     assert_equal 10, counts.size
     assert_equal (@obj989.created_at.to_i / 60) * 60, counts.first.first.to_i
@@ -274,13 +265,13 @@ context 'many objects' do
   end
   
   test 'get only the first page of the count with a value' do
-    counts = AllSeeingEye::Model.list_by_field('ip', :value => '192.168.0.1', :page => 1)
+    counts = AllSeeingEye::Request.list_by_field('ip', :value => '192.168.0.1', :page => 1)
     
     assert_equal 10, counts.size
   end
   
   test 'return the count for a field with a given value' do
-    assert_equal 1000, AllSeeingEye::Model.count_by_field('ip', :value => '192.168.0.1')
+    assert_equal 1000, AllSeeingEye::Request.count_by_field('ip', :value => '192.168.0.1')
   end
 end
 
@@ -290,13 +281,13 @@ context 'instance methods' do
   end
   
   test 'assign all attr_accessors from fields at initialization' do
-    obj = AllSeeingEye::Model.new
+    obj = AllSeeingEye::Request.new
     
-    AllSeeingEye::Model.field_names.each {|f| assert obj.respond_to?(f.to_sym); assert obj.respond_to?("#{f}=".to_sym)}
+    AllSeeingEye::Request.field_names.each {|f| assert obj.respond_to?(f.to_sym); assert obj.respond_to?("#{f}=".to_sym)}
   end
   
   test 'assign an object an ID if it does not have one' do
-    obj = AllSeeingEye::Model.new
+    obj = AllSeeingEye::Request.new
     
     assert_equal nil, obj.instance_variable_get(:@id)
     assert_equal 1, obj.id
@@ -304,44 +295,44 @@ context 'instance methods' do
   end
   
   test 'save a new model object' do
-    obj = AllSeeingEye::Model.new(:uri => '/test/', :ip => '127.0.0.1')
+    obj = AllSeeingEye::Request.new(:uri => '/test/', :ip => '127.0.0.1')
     
     assert obj.save
     assert_equal '/test/', obj.uri
     assert_equal '127.0.0.1', obj.ip
-    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:model:#{AllSeeingEye::Model.next_id - 1}"]
-    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:model:fields:uri", '/test/')
-    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:model:fields:ip", '127.0.0.1')
-    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:model:fields:uri:/test/", :by => 'nosort')
+    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:request:#{AllSeeingEye::Request.next_id - 1}"]
+    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:request:fields:uri", '/test/')
+    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:request:fields:ip", '127.0.0.1')
+    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:request:fields:uri:/test/", :by => 'nosort')
   end
   
   test 'save a new model object with date attributes' do
     time = Time.now
-    obj = AllSeeingEye::Model.new(:created_at => time)
+    obj = AllSeeingEye::Request.new(:created_at => time)
     
     assert obj.save
     assert_equal time, obj.created_at
-    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:model:#{AllSeeingEye::Model.next_id - 1}"]
-    assert_equal time.to_i.to_s, AllSeeingEye.redis.zscore("allseeingeye:model:fields:created_at", obj.id)
+    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:request:#{AllSeeingEye::Request.next_id - 1}"]
+    assert_equal time.to_i.to_s, AllSeeingEye.redis.zscore("allseeingeye:request:fields:created_at", obj.id)
   end
   
   test 'ignore empty and nil attributes in model object creation' do
     time = Time.now
-    obj = AllSeeingEye::Model.new(:created_at => time, :uri => '/test/', :ip => nil, :action => '')
+    obj = AllSeeingEye::Request.new(:created_at => time, :uri => '/test/', :ip => nil, :action => '')
     
     assert obj.save
     assert_equal time, obj.created_at
     assert_equal '/test/', obj.uri
     assert_equal nil, obj.ip
     assert_equal '', obj.action
-    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:model:#{AllSeeingEye::Model.next_id - 1}"]
-    assert_equal time.to_i.to_s, AllSeeingEye.redis.zscore("allseeingeye:model:fields:created_at", obj.id)
-    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:model:fields:uri", '/test/')
-    assert_equal nil, AllSeeingEye.redis.zscore("allseeingeye:model:fields:ip", nil)
-    assert_equal nil, AllSeeingEye.redis.zscore("allseeingeye:model:fields:action", '')
-    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:model:fields:uri:/test/", :by => 'nosort')
-    assert_equal [], AllSeeingEye.redis.sort("allseeingeye:model:fields:ip:", :by => 'nosort')
-    assert_equal [], AllSeeingEye.redis.sort("allseeingeye:model:fields:action:", :by => 'nosort')
+    assert_equal Marshal.dump(obj), AllSeeingEye.redis["allseeingeye:request:#{AllSeeingEye::Request.next_id - 1}"]
+    assert_equal time.to_i.to_s, AllSeeingEye.redis.zscore("allseeingeye:request:fields:created_at", obj.id)
+    assert_equal '1', AllSeeingEye.redis.zscore("allseeingeye:request:fields:uri", '/test/')
+    assert_equal nil, AllSeeingEye.redis.zscore("allseeingeye:request:fields:ip", nil)
+    assert_equal nil, AllSeeingEye.redis.zscore("allseeingeye:request:fields:action", '')
+    assert_equal [obj.id.to_s], AllSeeingEye.redis.sort("allseeingeye:request:fields:uri:/test/", :by => 'nosort')
+    assert_equal [], AllSeeingEye.redis.sort("allseeingeye:request:fields:ip:", :by => 'nosort')
+    assert_equal [], AllSeeingEye.redis.sort("allseeingeye:request:fields:action:", :by => 'nosort')
   end
   
 end

data/test/test_helper.rb

@@ -1,12 +1,12 @@
 require 'rubygems'
 
+ENV['ALL_SEEING_EYE_ENV'] = 'test'
+ENV['ALL_SEEING_EYE_CONFIG'] = 'test/fixtures'
 dir = File.dirname(File.expand_path(__FILE__))
 $LOAD_PATH.unshift dir + '/../lib'
 require 'test/unit'
 require 'yaml'
 require 'all_seeing_eye'
-ENV['ALL_SEEING_EYE_ENV'] = 'test'
-ENV['ALL_SEEING_EYE_CONFIG'] = 'test/fixtures'
 
 if !system("which redis-server")
   puts '', "** can't find `redis-server` in your path"

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: all_seeing_eye
 version: !ruby/object:Gem::Version 
-  hash: 57
+  hash: 55
   prerelease: 
   segments: 
   - 0
   - 0
-  - 19
-  version: 0.0.19
+  - 20
+  version: 0.0.20
 platform: ruby
 authors: 
 - Josh Symonds
@@ -15,7 +15,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-08-15 00:00:00 -05:00
+date: 2011-08-16 00:00:00 -05:00
 default_executable: all-seeing-eye
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -57,7 +57,7 @@ dependencies:
         segments: 
         - 0
         version: "0"
-  name: rbtree
+  name: sinatra
   version_requirements: *id003
   prerelease: false
 - !ruby/object:Gem::Dependency 
@@ -71,26 +71,12 @@ dependencies:
         segments: 
         - 0
         version: "0"
-  name: sinatra
+  name: vegas
   version_requirements: *id004
   prerelease: false
 - !ruby/object:Gem::Dependency 
   type: :runtime
   requirement: &id005 !ruby/object:Gem::Requirement 
-    none: false
-    requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        hash: 3
-        segments: 
-        - 0
-        version: "0"
-  name: vegas
-  version_requirements: *id005
-  prerelease: false
-- !ruby/object:Gem::Dependency 
-  type: :runtime
-  requirement: &id006 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - ">="
@@ -102,11 +88,11 @@ dependencies:
         - 12
         version: 2.3.12
   name: activesupport
-  version_requirements: *id006
+  version_requirements: *id005
   prerelease: false
 - !ruby/object:Gem::Dependency 
   type: :runtime
-  requirement: &id007 !ruby/object:Gem::Requirement 
+  requirement: &id006 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - ">="
@@ -116,11 +102,11 @@ dependencies:
         - 0
         version: "0"
   name: i18n
-  version_requirements: *id007
+  version_requirements: *id006
   prerelease: false
 - !ruby/object:Gem::Dependency 
   type: :runtime
-  requirement: &id008 !ruby/object:Gem::Requirement 
+  requirement: &id007 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - ">="
@@ -130,11 +116,11 @@ dependencies:
         - 0
         version: "0"
   name: thin
-  version_requirements: *id008
+  version_requirements: *id007
   prerelease: false
 - !ruby/object:Gem::Dependency 
   type: :development
-  requirement: &id009 !ruby/object:Gem::Requirement 
+  requirement: &id008 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - ~>
@@ -146,11 +132,11 @@ dependencies:
         - 0
         version: 1.0.0
   name: bundler
-  version_requirements: *id009
+  version_requirements: *id008
   prerelease: false
 - !ruby/object:Gem::Dependency 
   type: :development
-  requirement: &id010 !ruby/object:Gem::Requirement 
+  requirement: &id009 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - ~>
@@ -162,11 +148,11 @@ dependencies:
         - 4
         version: 1.6.4
   name: jeweler
-  version_requirements: *id010
+  version_requirements: *id009
   prerelease: false
 - !ruby/object:Gem::Dependency 
   type: :development
-  requirement: &id011 !ruby/object:Gem::Requirement 
+  requirement: &id010 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - ">="
@@ -176,7 +162,7 @@ dependencies:
         - 0
         version: "0"
   name: rcov
-  version_requirements: *id011
+  version_requirements: *id010
   prerelease: false
 description: AllSeeingEye observes all requests, with parameters that you specify, to Redis. Then it composes them into graphs for you to see.
 email: veraticus@gmail.com
@@ -186,21 +172,23 @@ extensions: []
 
 extra_rdoc_files: 
 - LICENSE.txt
-- README.rdoc
+- README.markdown
 files: 
 - Gemfile
 - Gemfile.lock
 - LICENSE.txt
-- README.rdoc
+- README.markdown
 - Rakefile
 - VERSION
 - all_seeing_eye.gemspec
 - bin/all-seeing-eye
+- examples/all_seeing_eye-nginx.conf
 - init.rb
 - lib/all_seeing_eye.rb
+- lib/all_seeing_eye/generators/all_seeing_eye.yml
+- lib/all_seeing_eye/generators/all_seeing_eye_generator.rb
 - lib/all_seeing_eye/integrations/rails2.rb
 - lib/all_seeing_eye/model.rb
-- lib/all_seeing_eye/models/request.rb
 - lib/all_seeing_eye/server.rb
 - lib/all_seeing_eye/server/public/application.js
 - lib/all_seeing_eye/server/public/eye.png

data/README.rdoc

@@ -1,19 +0,0 @@
-= all_seeing_eye
-
-A highly configurable Gem that lets you view requests as they occur, backed by Redis.
-
-== Contributing to all_seeing_eye
- 
-* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
-* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
-* Fork the project
-* Start a feature/bugfix branch
-* Commit and push until you are happy with your contribution
-* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
-* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
-
-== Copyright
-
-Copyright (c) 2011 Josh Symonds. See LICENSE.txt for
-further details.
-

data/lib/all_seeing_eye/models/request.rb

@@ -1,3 +0,0 @@
-class AllSeeingEye::Request < AllSeeingEye::Model
-  
-end