activerecord-summarize 0.3.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d491ee7730156f77105ec7df6bac79b6410fa97b3387816f103dee233b43df8d
-  data.tar.gz: f33be41270ab955fcf2bf9b227cc7c212ae3aa385786da3a5b424a3e1e707e47
+  metadata.gz: d3349ef226e79ac7b8182798fbe3f0966a94bd29a5ec31202fe14c74e681bc68
+  data.tar.gz: f257baecb4562c791d0d7648f45ec2549e1ae6faff2cfcd386d95a5e68fb233a
 SHA512:
-  metadata.gz: 0d1f5308da4fc8b781e8dd5a69a10c671106acdb9b565c056d23b33114fc96f8d4ff9a60f3887b861f9142b2564c3f33f17d9e8ea52c212c27abbdacc861e2d6
-  data.tar.gz: 318bad930ac53001068e6e3396d921f71ff62c8a0899abe7a96a55d9dab59aa7e6e45c2cdc6e7a5f5ae60b67f18a47be79cb73ee29a85e1fac25a988c43dc47a
+  metadata.gz: fdf0dece89a7d1db578414a1682adb956e8a973c7d575b7d589d0645ef906bdba2f9d849a1833fd1cc10a7b6f1e105c2c36b54532624005d67d8501e1e03aa13
+  data.tar.gz: a21a8e232e86706283954d4690b99c824c80b3521337c2dba10dc3f415f295844670f6d0c4ac94d7a0a8fa58c5fe408a07a64b3d31b65f20086a1d6920409382

data/CHANGELOG.md

@@ -1,3 +1,21 @@
+## [0.5.0] - 2023-05-14
+
+- **FEATURE:** Your `summarize` blocks won't need to accept the proc second argument as often, because `ChainableResult` methods will also resolve their arguments. E.g., `query.summarize {|q| @mult = q.sum(:a) * q.sum(:b) }` now works, where previously you would have needed to write `query.summarize {|q,with| @mult = with[q.sum(:a),q.sum(:b)] {|a,b| a * b } }`.
+
+- **IMPROVEMENT:** The conventional name of the proc provided as an optional second argument to `summarize` blocks is now `with_resolved` instead of `with`. Interactively teaching `activerecord-summarize` to some people showed that this was an improvement in clarity. The local name of the proc has always been under your control (it's your block!), so this doesn't affect anything besides documentation and tests, but if for some reason you accessed the proc at its internal name of `ChainableResult::WITH`, that will still work, too, even though we now refer to it as `ChainableResult::WITH_RESOLVED`.
+
+## [0.4.0] - 2023-02-27
+
+- **FEATURE:** Support for top-level .group(:belongs_to_association), returning hash with models as keys.
+
+  I didn't realize this until a few months ago, but in ActiveRecord, if `Foo belongs_to :bar`, you can do `Foo.group(:bar).count` and get back a hash with `Bar` records as keys and counts as values. (ActiveRecord executes two total queries to implement this, one for the counts grouped by the `bar_id` foreign key, then another to retrieve the `Bar` models.)
+
+  Now the same behavior works with `summarize`: you can still retrieve any number of counts and/or sums about `Foo`—including some with additional filters and even sub-grouping—in a single query, and then we'll execute one additional query to retrieve the records for the `Bar` model keys.
+
+- **IMPROVEMENT:** `bin/console` is now much more useful for developing `activerecord-summarize`
+
+- **IMPROVEMENT:** Added some tests for queries joining HABTM associations and (of course, supporting the new feature) `belongs_to` associations. `summarize` preceded by joins is already stable and documented, but it didn't have tests before. 
+
 ## [0.3.1] - 2022-06-23
 
 - **BUGFIX:** `with` didn't work correctly with a single argument. Embarassingly, both the time-traveling version of `with` and the trivial/fake one provided when `noop: true` is set had single argument bugs, and they were different bugs.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    activerecord-summarize (0.3.1)
+    activerecord-summarize (0.5.0)
       activerecord (>= 5.0)
 
 GEM
@@ -65,4 +65,4 @@ DEPENDENCIES
   standard (~> 1.3)
 
 BUNDLED WITH
-   2.3.3
+   2.4.13

data/README.md

@@ -59,7 +59,7 @@ Purchase.complete.left_joins(:region).summarize do |purchases|
 end
 ```
 
-Until the `summarize` block ends, the return value of your calculations are `ChainableResult::Future` instances, a bit like a Promise with a more convenient API. You can call any method you like on a `ChainableResult`, and you'll get back another `ChainableResult`, and they'll all turn out alright in the end—provided you called methods that would have worked if you had run that calculation without `summarize`. OTOH, using a `ChainableResult` as an argument to another method generally will not work.
+Until the `summarize` block ends, the return value of your calculations are `ChainableResult::Future` instances, a bit like a Promise with a more convenient API. You can call any method you like on a `ChainableResult`, and you'll get back another `ChainableResult`, and they'll all turn out alright in the end—provided you called methods that would have worked if you had run that calculation without `summarize`. OTOH, using a `ChainableResult` as an argument to a method of a non-`ChainableResult` generally will not work.
 
 ```ruby
 Purchase.last_quarter.complete.summarize do |purchases|
@@ -68,24 +68,28 @@ Purchase.last_quarter.complete.summarize do |purchases|
   @vc_projection = @sales * 3
   # And this won't:
   @vc_projection = 3 * @sales
+  # But this will work since v0.5.0...
+  @units_sold = purchases.sum(:units)
+  # ...because methods of `ChainableResult` now resolve their argument(s)
+  @avg_unit_price = @sales / @units_sold
 end
 ```
 
-If, within a `summarize` block, you want to combine data from more than one `ChainableResult`, you must use the otherwise-optional second argument yielded to the block, a `proc` I like to name `with`. Pass it all the results you want to combine and a block that combines them and returns the new result:
+If, within a `summarize` block, you want to combine data from more than one `ChainableResult`, you may need to use the otherwise-optional second argument yielded to the block, a `proc` I like to name `with_resolved`. Pass it all the results you want to combine and a block that combines them and returns the new result:
 
 ```ruby
-Purchase.complete.left_joins(:promotion).summarize do |purchases, with|
+Purchase.complete.left_joins(:promotion).summarize do |purchases, with_resolved|
   @all_revenue = purchases.sum(:amount)
   promotions = purchases.where.not(promotions: {id: nil})
   @promotion_sales = promotions.count
   @promotion_discounts = promotions.sum("promotions.discount_amount")
-  @avg_discount = with[@promotion_sales, @promotion_discounts] do |sales, discounts|
+  @avg_discount = with_resolved[@promotion_sales, @promotion_discounts] do |sales, discounts|
     sales.zero? ? 0 : discounts / sales
   end
 end
 ```
 
-Treat a `with` block as a pure function: i.e., return the value you care about, and don't set or change any other state within the block. Behavior in any other case is undefined.
+Treat a `with_resolved` block as a pure function: i.e., return the value you care about, and don't set or change any other state within the block. Behavior in any other case is undefined.
 
 ## Escape hatch
 
@@ -93,7 +97,7 @@ The query generated by `summarize` is often much faster than equivalent queries
 
 By design, every operation performed with `summarize` is correct and corresponds to normal `ActiveRecord` behavior, and any operations that can't be done correctly this way or aren't yet will raise exceptions. But only imperfect humans have worked on this gem, so you might also wonder if `summarize` is producing correct results.
 
-Fortunately, you can easily check both with `summarize(noop: true)`, which causes `summarize` to yield the original relation it was called on and a trivial `with` proc. The block will be executed as though `summarize` were not involved, with each calculation executing separately and immediately returning numbers or hashes.
+Fortunately, you can easily check both with `summarize(noop: true)`, which causes `summarize` to yield the original relation it was called on and a trivial `with_resolved` proc. The block will be executed as though `summarize` were not involved, with each calculation executing separately and immediately returning numbers or hashes.
 
 If you do find any case where you get different results with `summarize(noop: true)`, I'd be grateful if you filed an issue.
 
@@ -117,10 +121,10 @@ When the parent relation already has `.group` applied, `pure: true` is implied a
 Build even more complex queries by using `summarize` on a relation that already has `.group` applied. Results are grouped just like a standard `.group(*expressions).count`, but instead of single numbers, the values are whatever set of calculations you return from the block, including further `.group(*more).calculate(:sum|:count,*args)` calculations, in whatever `Array` or `Hash` shape you arrange them. For example:
 
 ```ruby
-puts Purchase.last_year.complete.group(:region_id).summarize do |purchases,with|
+puts Purchase.last_year.complete.group(:region_id).summarize do |purchases,with_resolved|
   total = purchases.count
   by_quarter = purchases.group(CREATED_TO_YEAR_SQL, CREATED_TO_QUARTER_SQL).count.sort.to_h
-  target = with[total / 4, by_quarter.values.max] {|avg_q, best_q| [avg_q * 1.25, best_q].max.round }
+  target = with_resolved[total / 4, by_quarter.values.max] {|avg_q, best_q| [avg_q * 1.25, best_q].max.round }
   {last_year: total, quarters: by_quarter, unit_target: target}
 end
 # Output:
@@ -150,13 +154,13 @@ When the relation already has `group` applied, for correct results, `summarize`
 
 ```ruby
 # A trivial example:
-Purchase.complete.group(:region_id).summarize {|purchases| purchases.sum(:amount) }
+Purchase.complete.group(:region).summarize {|purchases| purchases.sum(:amount) }
 
 # ...is exactly equivalent to:
-Purchase.complete.group(:region_id).sum(:amount)
+Purchase.complete.group(:region).sum(:amount)
 
 # But if there were three regions, what should the value of @target be in this case?
-region_targets = Purchase.last_quarter.complete.group(:region_id).summarize do |purchases|
+region_targets = Purchase.last_quarter.complete.group(:region).summarize do |purchases|
   @target = purchases.sum(:amount) * 1.25
 end
 ```

data/bin/console

@@ -2,10 +2,11 @@
 # frozen_string_literal: true
 
 require "bundler/setup"
+require "active_record"
 require "activerecord/summarize"
+require_relative "../test/test_data" # Test fixtures so there's something to play with
 
-# You can add fixtures and/or initialization code here to make experimenting
-# with your gem easier. You can also use a different console, if you like.
+# You can use a different console, if you like.
 
 # (If you use this, don't forget to add pry to your Gemfile!)
 # require "pry"

data/docs/use_case_moderator_dashboard.md

@@ -69,19 +69,19 @@ def dashboard
   # If you forget, `daily_posts.popular.count` will raise `Unsummarizable` with a helpful message.
   all_posts = Post.where(subreddit: @subreddits.select(:id)).where(created_at: 30.days.ago..)
                   .left_joins(:popularity_threshold_setting).order(:created_at)
-  @subreddit_stats = all_posts.group(:subreddit_id).summarize do |posts, with|
+  @subreddit_stats = all_posts.group(:subreddit_id).summarize do |posts, with_resolved|
     daily_posts = posts.group("posts.created_at::date")
     dow_not_burried = posts.where(karma: 0..).group("EXTRACT(DOW FROM posts.created_at)")
     {
       posts_created: posts.count,
       buried_posts: posts.where(karma: ...0).count,
-      daily_popular_rate: with[
+      daily_popular_rate: with_resolved[
           daily_posts.popular.count,
           daily_posts.count
         ] do |popular, total|
           total.map { |date, count| [date, (popular[date]||0).to_f / count] }.to_h
         end,
-      dow_avg_comments: with[
+      dow_avg_comments: with_resolved[
           dow_not_buried.sum(:comments_count),
           dow_not_buried.count
         ] do |comments, posts|
@@ -94,4 +94,4 @@ end
 
 Since `summarize` runs a single query that visits each relevant `posts` row just once, adding additional calculations is pretty close to free.
 
-Even with the mental overhead of needing to join outside the block and use `with` to combine calculations (see [README](../README.md) for details), I think this is still easy to read, write, and reason about, and it beats the heck out of walls of SQL. What do you think?
+Even with the mental overhead of needing to join outside the block and use `with_resolved` to combine calculations (see [README](../README.md) for details), I think this is still easy to read, write, and reason about, and it beats the heck out of walls of SQL. What do you think?

data/lib/activerecord/summarize/version.rb

@@ -2,6 +2,6 @@
 
 module ActiveRecord
   module Summarize
-    VERSION = "0.3.1"
+    VERSION = "0.5.0"
   end
 end

data/lib/activerecord/summarize.rb

@@ -7,7 +7,7 @@ module ActiveRecord::Summarize
   class Unsummarizable < StandardError; end
 
   class Summarize
-    attr_reader :current_result_row, :pure, :noop, :from_where
+    attr_reader :current_result_row, :base_groups, :base_association, :pure, :noop, :from_where
     alias_method :pure?, :pure
     alias_method :noop?, :noop
 
@@ -29,7 +29,18 @@ module ActiveRecord::Summarize
     def initialize(relation, pure: nil, noop: false)
       @relation = relation
       @noop = noop
-      has_base_groups = relation.group_values.any?
+      @base_groups, @base_association = relation.group_values.dup.then do |group_fields|
+        # Based upon a bit from ActiveRecord::Calculations.execute_grouped_calculation,
+        # if the base relation is grouped only by a belongs_to association, group by
+        # the association's foreign key.
+        if group_fields.size == 1 && group_fields.first.respond_to?(:to_sym)
+          association = relation.klass._reflect_on_association(group_fields.first)
+          # Like ActiveRecord's group(:association).count behavior, this only works with belongs_to associations
+          next [Array(association.foreign_key), association] if association&.belongs_to?
+        end
+        [group_fields, nil]
+      end
+      has_base_groups = base_groups.any?
       raise Unsummarizable, "`summarize` must be pure when called on a grouped relation" if pure == false && has_base_groups
       raise ArgumentError, "`summarize(noop: true)` is impossible on a grouped relation" if noop && has_base_groups
       @pure = has_base_groups || !!pure
@@ -37,8 +48,8 @@ module ActiveRecord::Summarize
     end
 
     def process(&block)
-      # For noop, just yield the original relation and a transparent `with` proc.
-      return yield(@relation, ChainableResult::SYNC_WITH) if noop?
+      # For noop, just yield the original relation and a transparent `with_resolved` proc.
+      return yield(@relation, ChainableResult::SYNC_WITH_RESOLVED) if noop?
       # Within the block, the relation and its future clones intercept calls to
       # `count` and `sum`, registering them and returning a ChainableResult via
       # summarize.add_calculation.
@@ -49,28 +60,42 @@ module ActiveRecord::Summarize
             include InstanceMethods
           end
         end,
-        ChainableResult::WITH
+        ChainableResult::WITH_RESOLVED
       ))
       ChainableResult.with_cache(!pure?) do
         # `resolve` builds the single query that answers all collected calculations,
-        # executes it, and aggregates the results by the values of
-        # `@relation.group_values``. In the common case of no `@relation.group_values`,
-        # the result is just `{[]=>[*final_value_for_each_calculation]}`
+        # executes it, and aggregates the results by the values of `base_groups`.
+        # In the common case of no `base_groups`, the resolve returns:
+        # `{[]=>[*final_value_for_each_calculation]}`
         result = resolve.transform_values! do |row|
           # Each row (in the common case, only one) is used to resolve any
           # ChainableResults returned by the block. These may be a one-to-one mapping,
-          # or the block return may have combined some results via `with` or chained
+          # or the block return may have combined some results via `with`, chained
           # additional methods on results, etc..
           @current_result_row = row
           future_block_result.value
         end.then do |result|
-          # Change ungrouped result from `{[]=>v}` to `v` and grouped-by-one-column
-          # result from `{[k1]=>v1,[k2]=>v2,...}` to `{k1=>v1,k2=>v2,...}`.
-          # (Those are both probably more common than multiple-column base grouping.)
-          case @relation.group_values.size
-          when 0 then result.values.first
-          when 1 then result.transform_keys! { |k| k.first }
-          else result
+          # Now unpack/fix-up the result keys to match shape of Relation.count or Relation.group(*cols).count return values
+          if base_groups.empty?
+            # Change ungrouped result from `{[]=>v}` to `v`, like Relation.count
+            result.values.first
+          elsif base_association
+            # Change grouped-by-one-belongs_to-association result from `{[id1]=>v1,[id2]=>v2,...}` to
+            # `{<AssociatedModel id:id1>=>v1,<AssociatedModel id:id2>=>v2,...}` like Relation.group(:association).count
+
+            # Loosely based on a bit from ActiveRecord::Calculations.execute_grouped_calculation,
+            # retrieve the records for the group association and replace the keys of our final result.
+            key_class = base_association.klass.base_class
+            key_records = key_class
+              .where(key_class.primary_key => result.keys.flatten)
+              .index_by(&:id)
+            result.transform_keys! { |k| key_records[k[0]] }
+          elsif base_groups.size == 1
+            # Change grouped-by-one-column result from `{[k1]=>v1,[k2]=>v2,...}` to `{k1=>v1,k2=>v2,...}`, like Relation.group(:column).count
+            result.transform_keys! { |k| k[0] }
+          else
+            # Multiple-column base grouping (though perhaps relatively rare) requires no change.
+            result
           end
         end
         if !pure?
@@ -166,7 +191,7 @@ module ActiveRecord::Summarize
       base_group_columns = (0...base_groups.size)
       data
         .group_by { |row| row[base_group_columns] }
-        .tap { |h| h[[]] = [] if h.empty? && base_groups.size.zero? }
+        .tap { |h| h[[]] = [] if h.empty? && base_groups.empty? }
         .transform_values! do |rows|
           values = starting_values.map(&:dup) # map(&:dup) since some are hashes and we don't want to mutate starting_values
           rows.each do |row|
@@ -201,14 +226,10 @@ module ActiveRecord::Summarize
       end
     end
 
-    def base_groups
-      @relation.group_values.dup
-    end
-
     def all_groups
       # keep all base groups, even if they did something silly like group by
       # the same key twice, but otherwise don't repeat any groups
-      groups = base_groups
+      groups = base_groups.dup
       groups_set = Set.new(groups)
       @calculations.map { |f| f.relation.group_values }.flatten.each do |k|
         next if groups_set.include? k

data/lib/chainable_result.rb

@@ -12,9 +12,19 @@ class ChainableResult
     if use_cache?
       return @value if @cached
       @cached = true
-      @value = resolve_source.send(@method, *@args, **@opts, &@block)
+      @value = resolve_source.send(
+        @method,
+        *@args.map(&RESOLVE_ITEM),
+        **@opts.transform_values(&RESOLVE_ITEM),
+        &@block
+      )
     else
-      resolve_source.send(@method, *@args, **@opts, &@block)
+      resolve_source.send(
+        @method,
+        *@args.map(&RESOLVE_ITEM),
+        **@opts.transform_values(&RESOLVE_ITEM),
+        &@block
+      )
     end
   end
 
@@ -86,8 +96,9 @@ class ChainableResult
     (results.size == 1 ? results.first : results).then(&block)
   end
 
-  WITH = method(:with)
-  SYNC_WITH = method(:sync_with)
+  # Shorter names are deprecated
+  WITH_RESOLVED = WITH = method(:with)
+  SYNC_WITH_RESOLVED = SYNC_WITH = method(:sync_with)
 
   def self.resolve_item(item)
     case item

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activerecord-summarize
 version: !ruby/object:Gem::Version
-  version: 0.3.1
+  version: 0.5.0
 platform: ruby
 authors:
 - Joshua Paine
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2022-06-23 00:00:00.000000000 Z
+date: 2023-05-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -84,7 +84,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.3.3
+rubygems_version: 3.3.7
 signing_key:
 specification_version: 4
 summary: Run many .count and/or .sum queries in a single efficient query with minimal