methodmissing-scrooge 1.0.4 → 2.0.0

data/README.textile

@@ -1,353 +1,58 @@
 h1. Scrooge
 
-A Framework and ORM agnostic Model / record attribute tracker to ensure production
-Ruby applications only fetch the database content needed to minimize wire traffic
-and reduce conversion overheads to native Ruby types.
+h4. This is a complete rewrite from the initial coverage at "igvita.com":http://www.igvita.com/2009/02/27/activerecord-optimization-with-scrooge/ - read on below
 
-This is mostly an experiment into unobtrusive tracking, respecting development workflows
-and understanding Rack internals better. 
+Many thanks to Stephen Sykes ( "pennysmalls.com":http://pennysmalls.com ) for his time spent on shaping, implementing and troubleshooting this release.
+
+An ActiveRecord attribute tracker to ensure production Ruby applications only fetch the database content needed to minimize wire traffic and reduce conversion overheads to native Ruby types.
 
 h2. Why bother ?
 
 * Object conversion and moving unnecessary data is both expensive and tax existing infrastructure in high load setups
 * Manually extracting and scoping SELECT clauses is not sustainable in a clean and painless manner with iterative development, even less so in large projects.
 
-h2. Suggested Use
-
-There's 3 basic modes of operation :
-
-* Track : Track attribute access to dump a representative scope profile.
-
-* Scope : Scope the process and related resources to a previously persisted scope profile.
-
-* Track then scope : A multi-stage strategy that tracks attribute access for a given warmup period, aggregates the results
-                     across multiple processes and enforce a scoping policy representative of the tracking phase.
-
-h2. Resources
-
-A resource is :
-
-* A controller and action endpoint ( inferred through framework specific routing )
-* A content type / format - a PDF representation may have different Model attribute requirements than a vanilla ERB view.
-* Request method - typically popular public facing GET requests  
-* Public or Private as a logged in / authenticated representation likely have different attribute requirements.   
-
-All Model to attribute mappings is tracked on a per Resource basis.Multiple Models per Resource is supported.
-
-h2. Strategies
-
-h4. Tracking
-
-In tracking mode Scrooge installs filters ( either through Rack middleware or framework specific hooks ) that track attribute access on a per Resource basis.
-
-A Kernel#at_exit callback dumps and timestamps this profile ( or scope ) to eg. *framework_configuration_directory/config/scopes/1234147851/scope.yml* 
-
-This typically works well with functional or integration testing and can yield a substantial birds eye view of attribute
-use.The accuracy is directly proportional to test coverage and the quality of the test suite.
-
-Example log output :
-
-<pre>
-<code>
-  Processing HotelsController#index (for 0.0.0.0 at 2009-02-09 02:55:55) [GET]
-    Parameters: {"action"=>"index", "controller"=>"hotels"}
-    Hotel Load (0.3ms)   SELECT * FROM `hotels` LIMIT 0, 15
-  Rendering template within layouts/application
-  Rendering hotels/index
-    Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 491) LIMIT 1
-  [Scrooge] read attribute updated_at
-  Rendered hotels/_hotel (2.7ms)
-  Rendered shared/_header (0.1ms)
-  Rendered shared/_navigation (0.3ms)
-  Missing template hotels/_index_sidebar.erb in view path app/views
-  Rendered shared/_sidebar (0.1ms)
-  Rendered shared/_footer (0.1ms)
-  Completed in 91ms (View: 90, DB: 1) | 200 OK [http://test.host/hotels]
-    SQL (0.3ms)   ROLLBACK
-    SQL (0.1ms)   BEGIN
-</code>
-</pre>
-
-An example scope / profile, saved to disk :
+h2. What it does
 
 <pre>
 <code>
---- 
-- hotels_show_get: 
-    :action: show
-    :controller: hotels
-    :method: :get
-    :is_public: true
-    :format: "*/*"
-    :models: 
-    - Address: 
-      - line1
-      - line2
-      - created_at
-      - postcode
-      - updated_at
-      - country_id
-      - county
-      - location_id
-      - town
-      - hotel_id
-    - Hotel: 
-      - important_notes
-      - location_id
-- locations_index_get: 
-    :action: index
-    :controller: locations
-    :method: :get
-    :is_public: true
-    :format: "*/*"
-    :models: 
-    - Location: 
-      - name
-      - created_at
-      - code
-      - updated_at
-      - level
-      - id
-- countries_index_get: 
-    :action: index
-    :controller: countries
-    :method: :get
-    :is_public: true
-    :format: "*/*"
-    :models: 
-    - Country: 
-      - name
-      - created_at
-      - code
-      - updated_at
-      - id
-      - location_id
-      - continent_id
-- hotels_index_get: 
-    :action: index
-    :controller: hotels
-    :method: :get
-    :is_public: true
-    :format: "*/*"
-    :models: 
-    - Hotel: 
-      - from_price
-      - narrative
-      - star_rating
-      - latitude
-      - created_at
-      - hotel_name
-      - updated_at
-      - important_notes
-      - id
-      - apt
-      - location_id
-      - nearest_tube
-      - longitude
-      - telephone
-      - nearest_rail
-      - location_name
-      - distance
-    - Image: 
-      - thumbnail_width
-      - created_at
-      - title
-      - updated_at
-      - url
-      - thumbnail_height
-      - height
-      - thumbnail_url
-      - has_thumbnail
-      - hotel_id
-      - width
+	Processing HotelsController#show (for 127.0.0.1 at 2009-03-12 14:32:45) [GET]
+	  Parameters: {"action"=>"show", "id"=>"8699-radisson-hotel-waterfront-cape-town", "controller"=>"hotels"}
+	  Hotel Load Scrooged (0.3ms)   SELECT `hotels`.id FROM `hotels` WHERE (`hotels`.`id` = 8699) 
+	Rendering template within layouts/application
+	Rendering hotels/show
+	  Hotel Load (0.2ms)   SELECT `hotels`.location_id,`hotels`.hotel_name,`hotels`.location,`hotels`.from_price,`hotels`.star_rating,`hotels`.apt,`hotels`.latitude,`hotels`.longitude,`hotels`.distance,`hotels`.narrative,`hotels`.telephone,`hotels`.important_notes,`hotels`.nearest_tube,`hotels`.nearest_rail,`hotels`.created_at,`hotels`.updated_at FROM `hotels` WHERE (`hotels`.`id` = 8699) 
+	  Image Load Scrooged (0.2ms)   SELECT `images`.id FROM `images` WHERE (`images`.hotel_id = 8699) LIMIT 1
+	  Image Load (0.2ms)   SELECT `images`.hotel_id,`images`.title,`images`.url,`images`.width,`images`.height,`images`.thumbnail_url,`images`.thumbnail_width,`images`.thumbnail_height,`images`.has_thumbnail,`images`.created_at,`images`.updated_at FROM `images` WHERE (`images`.`id` = 488) 
+	Rendered shared/_header (0.0ms)
+	Rendered shared/_navigation (0.2ms)
+	  Image Load Scrooged (0.2ms)   SELECT `images`.id FROM `images` WHERE (`images`.hotel_id = 8699) 
+	  CACHE (0.0ms)   SELECT `images`.hotel_id,`images`.title,`images`.url,`images`.width,`images`.height,`images`.thumbnail_url,`images`.thumbnail_width,`images`.thumbnail_height,`images`.has_thumbnail,`images`.created_at,`images`.updated_at FROM `images` WHERE (`images`.`id` = 488) 
+	  Address Columns (44.8ms)   SHOW FIELDS FROM `addresses`
+	  Address Load Scrooged (0.5ms)   SELECT `addresses`.id FROM `addresses` WHERE (`addresses`.hotel_id = 8699) LIMIT 1
+	Rendered hotels/_show_sidebar (49.4ms)
+	Rendered shared/_footer (0.1ms)
+	Completed in 56ms (View: 8, DB: 46) | 200 OK [http://localhost/hotels/8699-radisson-hotel-waterfront-cape-town]
+
+
+	Processing HotelsController#show (for 127.0.0.1 at 2009-03-12 14:32:48) [GET]
+	  Parameters: {"action"=>"show", "id"=>"8699-radisson-hotel-waterfront-cape-town", "controller"=>"hotels"}
+	  Hotel Load Scrooged (0.3ms)   SELECT `hotels`.narrative,`hotels`.from_price,`hotels`.star_rating,`hotels`.hotel_name,`hotels`.id FROM `hotels` WHERE (`hotels`.`id` = 8699) 
+	Rendering template within layouts/application
+	Rendering hotels/show
+	  Image Load Scrooged (0.3ms)   SELECT `images`.url,`images`.id,`images`.height,`images`.width FROM `images` WHERE (`images`.hotel_id = 8699) LIMIT 1
+	Rendered shared/_header (0.0ms)
+	Rendered shared/_navigation (0.2ms)
+	  Image Load Scrooged (0.3ms)   SELECT `images`.thumbnail_width,`images`.id,`images`.thumbnail_height,`images`.thumbnail_url FROM `images` WHERE (`images`.hotel_id = 8699) 
+	  Address Load Scrooged (0.2ms)   SELECT `addresses`.id FROM `addresses` WHERE (`addresses`.hotel_id = 8699) LIMIT 1
+	Rendered hotels/_show_sidebar (1.3ms)
+	Rendered shared/_footer (0.0ms)
+	Completed in 7ms (View: 5, DB: 1) | 200 OK [http://localhost/hotels/8699-radisson-hotel-waterfront-cape-town]	
 </code>
-</pre>
-
-h4. Scope
-
-A previously persisted scope / profile can be restored from disk and injected to the applicable Resources.Database content retrieved will match that of the given scope timestamp.
-
-This is typically pushed to production where a hybrid ( track then scope strategy) mode of operation is frowned upon and adjusted for each major release or deployment.
+</pre>		
 
-Example log output :
-
-<pre>
-<code>
-  Processing HotelsController#index (for 0.0.0.0 at 2009-02-09 02:59:41) [GET]
-    Parameters: {"action"=>"index", "controller"=>"hotels"}
-    Hotel Load (0.4ms)   SELECT hotels.narrative, hotels.from_price, hotels.created_at, hotels.latitude, hotels.star_rating, hotels.hotel_name, hotels.updated_at, hotels.important_notes, hotels.apt, hotels.id, hotels.nearest_tube, hotels.location_id, hotels.nearest_rail, hotels.telephone, hotels.longitude, hotels.distance, hotels.location_name FROM `hotels` LIMIT 0, 15
-  Rendering template within layouts/application
-  Rendering hotels/index
-    Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 491) LIMIT 1
-  Rendered hotels/_hotel (2.8ms)
-  Rendered shared/_header (0.1ms)
-  Rendered shared/_navigation (0.3ms)
-  Missing template hotels/_index_sidebar.erb in view path app/views
-  Rendered shared/_sidebar (0.1ms)
-  Rendered shared/_footer (0.1ms)
-  Completed in 90ms (View: 5, DB: 1) | 200 OK [http://test.host/hotels]
-    SQL (0.1ms)   ROLLBACK
-    SQL (0.1ms)   BEGIN
-</code>
-</pre>
-
-h4. Track then scope
-
-Multi-stage and self configuring strategy that tracks attribute access for a given warmup period, synchronize the results across n-1 processes, aggregate the results to be representative of the whole cluster ( or seamless fallback to a single process ), remove the tracking filters and install functionality that scopes database access to that of the tracking phase.
-
-Recommended for production use.
-
-Example log output :
+h2. Suggested Use
 
-<pre>
-<code>
-Processing HotelsController#index (for 127.0.0.1 at 2009-02-16 00:00:58) [GET]
-  Parameters: {"action"=>"index", "controller"=>"hotels"}
-  Hotel Load (0.5ms)   SELECT * FROM `hotels` LIMIT 0, 15
-  Hotel Columns (7.7ms)   SHOW FIELDS FROM `hotels`
-  SQL (3.9ms)   SELECT count(*) AS count_all FROM `hotels` 
-Rendering template within layouts/application
-Rendering hotels/index
-  Image Load (0.5ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 11381) LIMIT 1
-  Image Columns (3.6ms)   SHOW FIELDS FROM `images`
-Rendered hotels/_hotel (200.2ms)
-  Image Load (0.4ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 11382) LIMIT 1
-Rendered hotels/_hotel (2.4ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 11697) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 12693) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 12738) LIMIT 1
-Rendered hotels/_hotel (1.6ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 12886) LIMIT 1
-Rendered hotels/_hotel (1.9ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13007) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13074) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13077) LIMIT 1
-Rendered hotels/_hotel (1.6ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13078) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.3ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13079) LIMIT 1
-Rendered hotels/_hotel (2.4ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13080) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13082) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13085) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13105) LIMIT 1
-Rendered hotels/_hotel (1.6ms)
-Rendered shared/_header (0.4ms)
-Rendered shared/_navigation (0.8ms)
-Missing template hotels/_index_sidebar.erb in view path app/views
-Rendered shared/_sidebar (0.4ms)
-Rendered shared/_footer (0.3ms)
-Completed in 270ms (View: 243, DB: 20) | 200 OK [http://localhost/hotels]
-[Scrooge] Execute stage :synchronize ...
-[Scrooge] Uninstalling tracking middleware ... 
-[Scrooge] Stop tracking ... 
-[Scrooge] Synchronize results with other processes ...
-Cache write: 17619400_63223_756033
-Cache read: scrooge_tracker_aggregation
-Cache write: scrooge_tracker_aggregation
-[Scrooge] Execute stage :aggregate ...
-[Scrooge] Aggregate results from other processes ...
-
-
-Processing HotelsController#index (for 127.0.0.1 at 2009-02-16 00:01:37) [GET]
-  Parameters: {"action"=>"index", "controller"=>"hotels"}
-  Hotel Load (0.5ms)   SELECT * FROM `hotels` LIMIT 0, 15
-  SQL (0.2ms)   SELECT count(*) AS count_all FROM `hotels` 
-Rendering template within layouts/application
-Rendering hotels/index
-  Image Load (0.3ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 11381) LIMIT 1
-Rendered hotels/_hotel (2.0ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 11382) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 11697) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 12693) LIMIT 1
-Rendered hotels/_hotel (1.6ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 12738) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 12886) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13007) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13074) LIMIT 1
-Rendered hotels/_hotel (1.4ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13077) LIMIT 1
-Rendered hotels/_hotel (1.6ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13078) LIMIT 1
-Rendered hotels/_hotel (1.6ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13079) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13080) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13082) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13085) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT * FROM `images` WHERE (`images`.hotel_id = 13105) LIMIT 1
-Rendered hotels/_hotel (81.1ms)
-Rendered shared/_header (0.1ms)
-Rendered shared/_navigation (0.3ms)
-Missing template hotels/_index_sidebar.erb in view path app/views
-Rendered shared/_sidebar (0.1ms)
-Rendered shared/_footer (0.1ms)
-Completed in 113ms (View: 107, DB: 4) | 200 OK [http://localhost/hotels]
-Cache read: scrooge_tracker_aggregation
-Cache read: 17619400_63223_756033
-[Scrooge] Execute stage :scope ...
-[Scrooge] Scope ...
-
-
-Processing HotelsController#index (for 127.0.0.1 at 2009-02-16 00:01:53) [GET]
-  Parameters: {"action"=>"index", "controller"=>"hotels"}
-  Hotel Load (0.7ms)   SELECT hotels.narrative, hotels.from_price, hotels.created_at, hotels.latitude, hotels.star_rating, hotels.hotel_name, hotels.updated_at, hotels.important_notes, hotels.apt, hotels.id, hotels.nearest_tube, hotels.location_id, hotels.nearest_rail, hotels.telephone, hotels.longitude, hotels.distance, hotels.location_name FROM `hotels` LIMIT 0, 15
-  SQL (0.2ms)   SELECT count(*) AS count_all FROM `hotels` 
-Rendering template within layouts/application
-Rendering hotels/index
-  Image Load (0.3ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 11381) LIMIT 1
-Rendered hotels/_hotel (2.0ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 11382) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 11697) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 12693) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 12738) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 12886) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13007) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13074) LIMIT 1
-Rendered hotels/_hotel (1.4ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13077) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13078) LIMIT 1
-Rendered hotels/_hotel (1.8ms)
-  Image Load (0.3ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13079) LIMIT 1
-Rendered hotels/_hotel (1.9ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13080) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13082) LIMIT 1
-Rendered hotels/_hotel (1.5ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13085) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-  Image Load (0.2ms)   SELECT images.created_at, images.thumbnail_width, images.title, images.updated_at, images.url, images.id, images.thumbnail_height, images.height, images.thumbnail_url, images.has_thumbnail, images.width, images.hotel_id FROM `images` WHERE (`images`.hotel_id = 13105) LIMIT 1
-Rendered hotels/_hotel (1.7ms)
-Rendered shared/_header (0.1ms)
-Rendered shared/_navigation (0.2ms)
-Missing template hotels/_index_sidebar.erb in view path app/views
-Rendered shared/_sidebar (0.0ms)
-Rendered shared/_footer (0.0ms)
-Completed in 34ms (View: 27, DB: 4) | 200 OK [http://localhost/hotels]
-</code>
-</pre>
+Install, and you're off to the races!
 
 h2. Installation
 
@@ -359,177 +64,162 @@ h4. From Git
 
   git pull git://github.com/methodmissing/scrooge.git
 
-
 h4. As a Gem
 
   sudo gem install methodmissing-scrooge -s http://gems.github.com
 
-h2. Configuration
+h2. Stability
 
-Scrooge installs ( see recommended installation above ) a configuration file with the following format within *framework_configuration_directory/scrooge.yml ( RAILS_ROOT/config/scrooge.yml for a Rails setup ) :
+The whole ActiveRecord test suite passes with scrooge, except for 9 failures related to callsite augmentation (note the SQL reload snippets below).Thoughts on handling or circumventing this much appreciated. 
 
 <pre>
 <code>
-  production:
-    orm: :active_record
-    storage: :memory
-    strategy: :track_then_scope
-    warmup: 600 # warmup / track for 10 minutes
-    scope:
-    on_missing_attribute: :reload # or :raise
-    logged_in_session: :user_id # session key that represents the logged in user 
-    enabled: true     
- development:
-    orm: :active_record
-    storage: :memory
-    strategy: :track
-    warmup: 600 # warmup / track for 10 minutes
-    scope: 
-    on_missing_attribute: :reload # or :raise
-    logged_in_session: :user_id # session key that represents the logged in user 
-    enabled: true    
-  test:
-    orm: :active_record
-    storage: :memory  
-    strategy: :track
-    warmup: 600 # warmup / track for 10 minutes
-    scope:
-    on_missing_attribute: :reload # or :raise
-    logged_in_session: :user_id # session key that represents the logged in user 
-    enabled: true
+  2) Failure:
+test_finding_with_includes_on_belongs_to_association_with_same_include_includes_only_once(EagerAssociationTest)
+    [/opt/local/lib/ruby/gems/1.8/gems/activerecord-2.3.1/test/cases/../../lib/active_record/test_case.rb:31:in `assert_queries'
+     /opt/local/lib/ruby/gems/1.8/gems/activerecord-2.3.1/test/cases/associations/eager_test.rb:139:in `test_finding_with_includes_on_belongs_to_association_with_same_include_includes_only_once'
+     /opt/local/lib/ruby/gems/1.8/gems/activesupport-2.3.1/lib/active_support/testing/setup_and_teardown.rb:57:in `__send__'
+     /opt/local/lib/ruby/gems/1.8/gems/activesupport-2.3.1/lib/active_support/testing/setup_and_teardown.rb:57:in `run']:
+5 instead of 3 queries were executed.
+Queries:
+SELECT `posts`.id,`posts`.type FROM `posts` WHERE (`posts`.`id` = 1) 
+SELECT `posts`.author_id,`posts`.title,`posts`.body,`posts`.comments_count,`posts`.taggings_count FROM `posts` WHERE (`posts`.`id` = 1) 
+SELECT `authors`.name,`authors`.id FROM `authors` WHERE (`authors`.`id` = 1) 
+SELECT `authors`.author_address_id,`authors`.author_address_extra_id FROM `authors` WHERE (`authors`.`id` = 1) 
+SELECT `author_addresses`.id FROM `author_addresses` WHERE (`author_addresses`.`id` = 1) .
+<3> expected but was
+<5>.
 </code>
 </pre>
 
-h4. ORM 
+To run tests in your environment :
 
-Scrooge is ORM agnostic and ships with an ActiveRecord layer.
+* Configure to run the ActiveRecord test suite as per the "docs":http://github.com/rails/rails/blob/8a17fd1a65ab8e2fa6b36d79603fde0e6ffd083f/activerecord/RUNNING_UNIT_TESTS
+* 'rake test' from within the scrooge root directory
+* It'll attempt to find the path to the ActiveRecord test cases through rubygems
+* Known to work with both 2.2.2, 2.3.0 and the upcoming 2.3.1
+
+h2. Initial Benchmarks
+
+Passenger, Rails 2.2.2, remote DB :
 
 <pre>
 <code>
-  orm: :active_record
-</pre>
+Without scrooge:
+
+Concurrency Level:      1
+Time taken for tests:   68.279156 seconds
+Complete requests:      150
+Failed requests:        0
+Write errors:           0
+Total transferred:      13741201 bytes
+HTML transferred:       13679100 bytes
+Requests per second:    2.20 [#/sec] (mean)
+Time per request:       455.194 [ms] (mean)
+Time per request:       455.194 [ms] (mean, across all concurrent requests)
+Transfer rate:          196.53 [Kbytes/sec] received
+
+With scrooge:
+
+Concurrency Level:      1
+Time taken for tests:   58.162039 seconds
+Complete requests:      150
+Failed requests:        0
+Write errors:           0
+Total transferred:      13747200 bytes
+HTML transferred:       13685100 bytes
+Requests per second:    2.58 [#/sec] (mean)
+Time per request:       387.747 [ms] (mean)
+Time per request:       387.747 [ms] (mean, across all concurrent requests)
+Transfer rate:          230.82 [Kbytes/sec] received
 </code>
+</pre>
+
+h2. How it works
 
-h4. Storage backend
+h4. Callsites
 
-Tracking results can be persisted to a given backend or storage option.Ships with a memory store, but can be extended to file system, memcached etc. as all Tracker components is designed to be Marshal friendly.A stub for future functionality.
+Ruby allows introspection of the call tree through
 
 <pre>
 <code>
-  storage: :memory
-</pre>
-</code>
-
-h4. Verbose
+  Kernel#caller
+</code>	
+</pre>	
 
-Log all tracking interactions to the framework logger when enabled.Disabled for production.
+Scrooge analyzes the last 10 calltree elements that triggered 
 
 <pre>
 <code>
-  verbose: false
+  ActiveRecord::Base.find_by_sql
+</code>	
 </pre>
-</code>
 
-h4. Strategy
+Lets refer to that as a callsite, or signature.
 
-One of :track, :scope or :track_then_scope .Only the :track_then_scope strategy respects the :warmup configuration option.
-
-h4. Warmup
-
-The designated warmup period for the :track_then_scope strategy, given in seconds.Typically 600 to 3600.
+Thus given SQL such as 
 
 <pre>
 <code>
-  warmup: 600	
+"SELECT * FROM `images` WHERE (`images`.hotel_id = 11697)  LIMIT 1"
+</code>	
 </pre>
-</code>
 
-h4. Scope
+Called from our application helper
 
-A scope is a reference to a timestamped Scrooge run where access to Model attributes is tracked on a per Resource basis.
+<pre>
+<code>
+["/Users/lourens/projects/superbreak_app/vendor/plugins/scrooge/rails/../lib/scrooge.rb:27:in `find_by_sql'", "/Users/lourens/projects/superbreak_app/vendor/rails/activerecord/lib/active_record/base.rb:1557:in `find_every'", "/Users/lourens/projects/superbreak_app/vendor/rails/activerecord/lib/active_record/base.rb:1514:in `find_initial'", "/Users/lourens/projects/superbreak_app/vendor/rails/activerecord/lib/active_record/base.rb:613:in `find'", "/Users/lourens/projects/superbreak_app/vendor/rails/activerecord/lib/active_record/associations/association_collection.rb:60:in `find'", "/Users/lourens/projects/superbreak_app/vendor/rails/activerecord/lib/active_record/associations/association_collection.rb:67:in `first'", "/Users/lourens/projects/superbreak_app/app/helpers/application_helper.rb:60:in `hotel_image'", "/Users/lourens/projects/superbreak_app/app/views/hotels/_hotel.html.erb:4:in `_run_erb_app47views47hotels47_hotel46html46erb_locals_hotel_hotel_counter_object'", "/Users/lourens/projects/superbreak_app/vendor/rails/actionpack/lib/action_view/renderable.rb:36:in `send'", "/Users/lourens/projects/superbreak_app/vendor/rails/actionpack/lib/action_view/renderable.rb:36:in `render'", "/Users/lourens/projects/superbreak_app/vendor/rails/actionpack/lib/action_view/renderable_partial.rb:20:in `render'"]
+</code>	
+</pre>
 
+We can generate a unique callsite identifier with the following calculation :
 <pre>
 <code>
-  scope: 1234567891
+(The above calltree <<  "SELECT * FROM `images` ).hash " # cut off conditions etc.
+</code>	
 </pre>
-</code>
 
-If not scope is given in the configuration, ENV['scope'] would also be considered to facilitate configuration through Capistrano etc.
+Callsites are tracked on a per model ( table name ) basis. 
 
-h4. Handling Missing Attributes
+h4. Scope
 
-When the contents for a given Model attribute has not been retrieved from the database, most ORM frameworks raise an error by default.This is configurable to reloading the model with all it's columns or raise instead.
+Only SQL statements that meet the following criteria is considered for optimization :
 
-<pre>
-<code>
-  on_missing_attribute: :reload # or :raise
-</pre>
-</code>
+* A SELECT statement
 
-h4. Private and Public Resources
+* Not an INNER JOIN
 
-An authenticated or logged in resource likely has different model requirements than it's public counterpart.
+* The Model has a primary key defined
 
-<pre>
-<code>
-  logged_in_session: :user_id # session key that represents the logged in user 
-</pre>
-</code>
+h4. How it tracks
+
+The ActiveRecord attributes Hash is replaced with a proxy that automatically augments the callsite with any attributes referenced through the Hash lookup keys.
 
-h4. Status
+h4. Storage
 
-Scrooge can be disabled with :
+There's a slight memory hit for each model as the callsites is stored as a class level Hash, which is relatively lightweight and looks like this :
 
 <pre>
 <code>
-  enabled: false 
-</pre>
+{-113952497=>#<Set: {"User", "Password"}>}	
 </code>
+</pre>
 
-h2. Rails specific rake tasks.
+h4. Tracking and scoping ?
 
-Ships with tasks to assist in inspecting results.
+The tracking and scoping phases is superseded by this implementation - none of those hindrances anymore.
 
-<pre>
-<code>
-methodmissing:superbreak_app lourens$ rake scrooge:list
-(in /Users/lourens/projects/superbreak_app)
-- 1234735663
-- 1234735722
-- 1234735744
-- 1234735790
-- 1234738880
-methodmissing:superbreak_app lourens$ rake scope=1234735790 scrooge:inspect
-(in /Users/lourens/projects/superbreak_app)
-#<GET :hotels/show (*/*)
- - #<Hotel :important_notes, :location_id>
- - #<Address :line1, :created_at, :line2, :postcode, :updated_at, :country_id, :county, :location_id, :town, :hotel_id>
-
-#<GET :countries/index (*/*)
- - #<Country :name, :created_at, :code, :updated_at, :id, :location_id, :continent_id>
-
-#<GET :locations/index (*/*)
- - #<Location :name, :created_at, :code, :updated_at, :level, :id>
-
-#<GET :hotels/index (*/*)
- - #<Image :created_at, :thumbnail_width, :title, :updated_at, :url, :thumbnail_height, :height, :thumbnail_url, :has_thumbnail, :width, :hotel_id>
- - #<Hotel :narrative, :from_price, :created_at, :latitude, :star_rating, :hotel_name, :updated_at, :important_notes, :apt, :id, :nearest_tube, :location_id, :nearest_rail, :telephone, :longitude, :distance, :location_name>
-</code>	
-</pre>
+h2. Todo
 
-h2. Notes
+* Deeper coverage for Scrooge::AttributesProxy ( pending conversion to a subclass of Hash instead )
 
-This is an initial release, has not yet been battle tested in production and is pending Ruby 1.9.1 compatibility.
+* Extract Scrooge::Callsite
 
-Developed on and for Rails 2.3, known to work with Rails 2.2.2
+* Track both columns AND association invocations off Scrooge::Callsite
 
-h2. Known pending issues
+* Have invoking Model#attributes not associate all columns with the callsite
 
-* Do not track columns that isn't explicitly defined ( AR counter caching checks etc. ) : 
-<code>
-<pre>
-ActiveRecord::StatementInvalid (Mysql::Error: Unknown column 'orders.brochures_count' in 'field list': SELECT orders.brochures_count, orders.id FROM `orders` WHERE (`orders`.`id` = 1688139)  LIMIT 1)
-</code>	
-</pre>
+* Avoid possible missing attribute exceptions for destroyed objects
 
-* alias_attribute compatibility
+(c) 2009 Lourens Naudé (methodmissing) and Stephen Sykes (sdsykes)