occams-record 0.10.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 9c305e7ef50172b06abc65501fda3a2e323a4b59
-  data.tar.gz: 8b3148052f587c1987485527473e99764132130a
+SHA256:
+  metadata.gz: 68db1cbd50ea9da8301200e9d57ca2517fe28624663c3d3ea2e463c99825f8d4
+  data.tar.gz: 4d55fddeaf26d1b2bc6b84ca09ec1d7a6d4ea23e5847b537c98f9ff5827bbc24
 SHA512:
-  metadata.gz: 4907ef959db18d068d58007f6f1df7d2aac60ce69cf1b9af4c91e89c518b8b2eb103d5574b6d03a34964a1456c62f18b372e0c50595a1bc9c2cb3b9a967afbbe
-  data.tar.gz: 39cbe4f0a8bb3af17944aba9d6c7839a6751b9b19053adbceb8df924815718d9c9dfaa596733bb63f1b3fb873815059c79264127d99b31a9621ffa69bdb99d4e
+  metadata.gz: 5924cf19d1fa6923146b289a661d80a964aa2616bc09768e8967dbe3a849349bb7aaf6c6edcb9fbe9301bf88cbad6f5a6dfb7397f8287c61b332d316197a18cb
+  data.tar.gz: f186dbeb9ea6f15cc4eebcb5cb2e6b135e5ba5b967c12e1162c938253f22fda343b3e2cc0401b93ba591fa94ebd695e6535e3a1ea8b4447aba11759dda549d28

data/README.md

@@ -2,28 +2,32 @@
 
 > Do not multiply entities beyond necessity. -- Occam's Razor
 
-Occam's Record is a high-efficiency query API for ActiveRecord. When loading thousands of records, ActiveRecord wastes a lot of RAM and CPU cycles on *things you'll never use.* Additionally, eagerly-loaded associations are forced to load each and every column, each and every record, and all in a certain order.
+Occam's Record is a high-efficiency query API for ActiveRecord. It is **not** an ORM or an ActiveRecord replacement. Use it to solve pain points in your existing ActiveRecord app.
 
-For those stuck with ActiveRecord, OccamsRecord seeks to solve these issues by making some very specific trade-offs:
+* 3x-5x faster than ActiveRecord.
+* Uses 1/3 the memory of ActiveRecord.
+* Eliminates the N+1 query problem.
+* Allows custom SQL when eager loading associations (use `select`, `where`, `order`, etc).
+* `find_each`/`find_in_batches` respects `order` and `limit`.
+* Allows eager loading of associations when using raw SQL.
+* Allows `find_each`/`find_in_batches` when using raw SQL.
 
-* OccamsRecord results are **read-only**.
-* OccamsRecord objects are **purely database rows** - they don't have any instance methods from your Rails models.
-* OccamsRecord queries must specify each association that will be used. Otherwise they simply won't be availble.
-
-For more on the rational behind OccamsRecord, see the Rational section at the end of the README. But in short, OccamsRecord is 3x-5x faster, uses 1/3 of the memory, and eliminates the N+1 query problem.
+[Look over the speed and memory measurements yourself!](https://github.com/jhollinger/occams-record/wiki/Measurements). OccamsRecord achieves all of this by making some very specific trade-offs:
 
-**BREAKING CHANGE** to `eager_load` in version **0.10.0**. See the examples below or [HISTORY.md](https://github.com/jhollinger/occams-record/blob/v0.10.0/HISTORY.md) for the new usage.
+* OccamsRecord results are **read-only**.
+* OccamsRecord results are **purely database rows** - they don't have any instance methods from your Rails models.
+* OccamsRecord queries must eager load each association that will be used. Otherwise they simply won't be availble.
 
 ## Usage
 
+Full documentation is available at [rubydoc.info/gems/occams-record](http://www.rubydoc.info/gems/occams-record).
+
 **Add to your Gemfile**
 
 ```ruby
 gem 'occams-record'
 ```
 
-Full documentation is available at [rubydoc.info/gems/occams-record](http://www.rubydoc.info/gems/occams-record).
-
 **Simple example**
 
 ```ruby
@@ -62,7 +66,7 @@ widgets[1].splines.map { |s| s.description }
 
 **An insane example, but only half as insane as the one that prompted the creation of this library**
 
-Here we're eager loading several levels down. Notice the `Proc` given to `eager_load(:orders)`. The `select:` option is just for convenience; you may instead pass a `Proc` and customize the query with any of ActiveRecord's query builder helpers (`select`, `where`, `order`, etc).
+Here we're eager loading several levels down. Notice the `Proc` given to `eager_load(:orders)`. The `select:` option is just for convenience; you may instead pass a `Proc` and customize the query with any of ActiveRecord's query builder helpers (`select`, `where`, `order`, `limit`, etc).
 
 ```ruby
 widgets = OccamsRecord.
@@ -118,14 +122,14 @@ widgets[0].expensive?
 => false
 
 widgets[0].orders[0].description
-=> "O839SJZ98B 1/8/2017"
+=> "O839SJZ98B - 1/8/2017"
 ```
 
 ## Raw SQL queries
 
-If you have a complicated query to run, you may drop down to hand-written SQL while still taking advantage of eager loading and variable escaping. (Note the slightly different syntax for binding variables.)
+If you have a complicated query to run, you may drop down to hand-written SQL while still taking advantage of eager loading and variable escaping (not possible in ActiveRecord). Note the slightly different syntax for binding variables.
 
-NOTE this feature is quite new and might have some bugs. Issues and Pull Requests welcome.
+NOTE this feature is quite new and might have some bugs. Since we are not yet at 1.0, breaking changes may occur. Issues and Pull Requests welcome.
 
 ```ruby
 widgets = OccamsRecord.sql(%(
@@ -151,30 +155,20 @@ widgets = OccamsRecord.
   run
 ```
 
-## Rational
-
-**What does OccamsRecord buy you?**
-
-* OccamsRecord results are **one-third the size** of ActiveRecord results.
-* OccamsRecord queries run **three to five times faster** than ActiveRecord queries.
-* When eager loading associations you may specify which columns to `SELECT`. (This can be a significant performance boost to both your database and Rails app, on top of the above numbers.)
-* When eager loading associations you may completely customize the query (`WHERE`, `ORDER BY`, `LIMIT`, etc.)
-* By forcing eager loading of associations, OccamsRecord bypasses the primary cause of performance problems in Rails: N+1 queries.
-* Forced eager loading also makes you consider the "shape" of your data, which can help you identify areas that need refactored (e.g. redundant foreign keys, more denormalization, etc.)
-
-**What don't you give up?**
-
-* You can still write your queries using ActiveRecord's query builder, as well as your existing models' associations & scopes.
-* You can still use ActiveRecord for everything else - small queries, creating, updating, and deleting records.
-* You can still inject some instance methods into your results, if you must. See below.
+To use `find_each` or `find_in_batches` with raw SQL you must provide the `LIMIT` and `OFFSET` statements yourself.
 
-**Is there evidence to back any of this up?**
-
-Glad you asked. [Look over the results yourself.](https://github.com/jhollinger/occams-record/wiki/Measurements)
-
-**Why not use a different ORM?**
-
-That's a great idea; check out [sequel](https://rubygems.org/gems/sequel) or [rom](https://rubygems.org/gems/rom)! But for large, legacy codebases heavily invested in ActiveRecord, switching ORMs often isn't practical. OccamsRecord can help you get some of those wins without a rewrite.
+```ruby
+widgets = OccamsRecord.sql(%(
+  SELECT * FROM widgets
+  WHERE category_id = %{cat_id}
+  LIMIT %{batch_limit}
+  OFFSET %{batch_offset}
+), {
+  cat_id: 5
+}).find_each { |widget|
+  puts widget.name
+}
+```
 
 ## Unsupported features
 

data/lib/occams-record/eager_loaders/has_one.rb

@@ -12,7 +12,7 @@ module OccamsRecord
         return if rows.empty?
         ids = rows.map { |r| r.send @ref.active_record_primary_key }.compact.uniq
         q = base_scope.where(@ref.foreign_key => ids)
-        q.where!(@ref.type => rows[0].class.try!(:model_name)) if @ref.options[:as]
+        q.where!(@ref.type => rows[0].class&.model_name) if @ref.options[:as]
         yield q
       end
 

data/lib/occams-record/raw_query.rb

@@ -25,6 +25,10 @@ module OccamsRecord
   #     eager_load(:category).
   #     run
   #
+  # NOTE To use find_each/find_in_batches, your SQL string must include 'LIMIT %{batch_limit} OFFSET %{batch_offset}',
+  # and an ORDER BY is strongly recomended.
+  # OccamsRecord will provide the bind values for you.
+  #
   # @param sql [String] The SELECT statement to run. Binds should use Ruby's named string substitution.
   # @param binds [Hash] Bind values (Symbol keys)
   # @param use [Array<Module>] optional Module to include in the result class (single or array)
@@ -46,6 +50,7 @@ module OccamsRecord
     attr_reader :binds
 
     include EagerLoaders::Builder
+    include Batches
 
     #
     # Initialize a new query.
@@ -133,5 +138,33 @@ module OccamsRecord
         a
       }
     end
+
+    #
+    # Returns an Enumerator that yields batches of records, of size "of".
+    # The SQL string must include 'LIMIT %{batch_limit} OFFSET %{batch_offset}'.
+    # The bind values will be provided by OccamsRecord.
+    #
+    # @param of [Integer] batch size
+    # @return [Enumerator] yields batches
+    #
+    def batches(of:)
+      unless @sql =~ /LIMIT\s+%\{batch_limit\}/i and @sql =~ /OFFSET\s+%\{batch_offset\}/i
+        raise ArgumentError, "When using find_each/find_in_batches you must specify 'LIMIT %{batch_limit} OFFSET %{batch_offset}'. SQL statement: #{@sql}"
+      end
+
+      Enumerator.new do |y|
+        offset = 0
+        loop do
+          results = RawQuery.new(@sql, @binds.merge({
+            batch_limit: of,
+            batch_offset: offset,
+          }), use: @use, query_logger: @query_logger, eager_loaders: @eager_loaders).model(@model).run
+
+          y.yield results if results.any?
+          break if results.size < of
+          offset += results.size
+        end
+      end
+    end
   end
 end

data/lib/occams-record/version.rb

@@ -3,5 +3,5 @@
 #
 module OccamsRecord
   # Library version
-  VERSION = '0.10.0'.freeze
+  VERSION = '0.11.0'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: occams-record
 version: !ruby/object:Gem::Version
-  version: 0.10.0
+  version: 0.11.0
 platform: ruby
 authors:
 - Jordan Hollinger
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-01-21 00:00:00.000000000 Z
+date: 2018-03-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -72,7 +72,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.2
+rubygems_version: 2.7.3
 signing_key: 
 specification_version: 4
 summary: The missing high-efficiency query API for ActiveRecord