skinny_includes 0.4.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: '0089447d2acabe947ad968c3939496cfc4e11357f8e57d80357bc9c2b694a0d1'
+  data.tar.gz: 9f12f4d7685fd86f217165fd9ba3859c17ccb90ac4d334ffd25f82b33916032f
+SHA512:
+  metadata.gz: ec92a26e5ffe31ed3ce3dc5dbd3008eb00f94ba8939f6a5cdb0857724320610d385c31966582473be0898fc35b4c764b92cc02d1a4603b6fc9f784f6ae26fba9
+  data.tar.gz: 54437698943556267db6cef5caf536f206dd2386824827d5600f31afd200089dcac13b5e4598a132a2b994eb46ab3d8013301735466e4c5194764d2d3ee080f3

data/.github/workflows/ci.yml

@@ -0,0 +1,27 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        ruby: ['3.0', '3.1', '3.2', '3.3']
+        rails: ['7.0', '7.1', '7.2', '8.0']
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Ruby
+      uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: ${{ matrix.ruby }}
+        bundler-cache: true
+
+    - name: Run tests
+      run: ruby spec/skinny_includes_spec.rb

data/CHANGELOG.md

@@ -0,0 +1,75 @@
+# Changelog
+
+## 0.1
+
+### Added
+- **Scoped associations support**: Association scopes are now properly respected when using `with_columns` and `without_columns`
+  - Works with `has_many`, `has_one`, and `belongs_to` associations
+  - Scopes are applied via `instance_exec` on the base query
+  - Example: `Post.includes(:published_comments).with_columns(published_comments: [:body])`
+
+- **Nested/chained includes support**: You can now load nested associations with column selection
+  - Use hash syntax with `:columns` and `:include` keys
+  - Supports unlimited nesting depth
+  - Automatically includes necessary foreign keys
+  - Example:
+    ```ruby
+    Post.with_columns(
+      comments: {
+        columns: [:body],
+        include: { author: [:name] }
+      }
+    )
+    ```
+  - Works with `belongs_to`, `has_many`, and `has_one` at any nesting level
+  - Works with both `with_columns` and `without_columns`
+  - Respects scoped associations at all nesting levels
+
+### Fixed
+- **belongs_to foreign key bug**: Fixed issue where the gem tried to select the foreign key column from the associated table in `belongs_to` associations
+  - Foreign keys are now only included for `has_many` and `has_one` associations (where they exist on the associated table)
+  - For `belongs_to`, the foreign key exists on the source table, not the target
+
+### Testing
+- Added comprehensive test coverage for `belongs_to` associations (15 tests)
+- Added comprehensive test coverage for `has_one` associations (9 tests)
+- Added tests for scoped associations (5 tests)
+- Added tests for nested includes (6 tests)
+- Added tests for all loading strategies (`includes`, `preload`, `eager_load`)
+- Total: 68 tests, all passing
+
+## Implementation Details
+
+### Scoped Associations
+The implementation applies association scopes by calling `instance_exec` on the base query with the scope proc:
+
+```ruby
+base_query = assoc_class.where(fk => parent_ids)
+
+if reflection.scope
+  base_query = base_query.instance_exec(&reflection.scope)
+end
+
+associated_records = base_query.select(*columns).to_a
+```
+
+This ensures that scopes defined like `-> { where(published: true) }` are properly evaluated in the context of the relation.
+
+### Implementation Details
+
+#### Nested Includes
+The implementation recursively loads nested associations:
+
+1. Parses column specs to extract `:columns` and `:include` keys
+2. Automatically includes foreign keys needed for nested associations
+3. Recursively calls `load_nested_associations` after loading each level
+4. Supports STI (Single Table Inheritance) by grouping records by class
+5. Works with all association types (`belongs_to`, `has_many`, `has_one`)
+
+The hash syntax makes the structure explicit:
+```ruby
+{
+  columns: [:foo, :bar],     # What to select
+  include: { nested: [...] } # What to nest
+}
+```

data/README.md

@@ -0,0 +1,163 @@
+# skinny_includes
+
+Select specific columns when preloading associations. Prevents N+1 queries, reduces memory usage.
+
+## Why?
+
+### Big beautiful columns 
+
+I have a lot of columns that have associated JSON directly on the table.
+
+```ruby
+class ThingWithJsonDataColumn < ApplicationRecord 
+  # has a `data` column name with column type of `json` 
+end
+```
+
+Sometimes I load 5k of these ThingWithJsonDataColumn objects in an HTTP request. This is expensive.
+
+I could write a custom association that excludes the column I don't care about, but that's silly.
+
+## Use case
+
+The obvious use case is large JSON or text columns that slow queries and inflate memory. Instead of writing custom scoped associations, exclude them:
+
+```ruby
+post.without_columns(comments: [:metadata_json, :body])
+```
+
+Loads all columns except `metadata_json` and `body`.
+
+## Installation
+
+```bash
+bundle add skinny_includes
+```
+
+Or manually:
+
+```ruby
+gem 'skinny_includes'
+```
+
+## Usage
+
+Two methods:
+
+**with_columns** — whitelist columns:
+
+```ruby
+Post.with_columns(comments: [:author, :upvotes])
+Post.includes(:comments).with_columns(comments: [:author])
+```
+
+**without_columns** — blacklist columns:
+
+```ruby
+Post.without_columns(comments: [:body, :metadata])
+Post.includes(:comments).without_columns(comments: :body)
+```
+
+Both work with multiple associations:
+
+```ruby
+Post.with_columns(comments: [:author], tags: [:name])
+Post.without_columns(comments: [:body], tags: [:description])
+```
+
+Primary keys and foreign keys are always included, even if excluded.
+
+## Scoped Associations
+
+Scoped associations are fully supported:
+
+```ruby
+class Post < ApplicationRecord
+  has_many :published_comments, -> { where(published: true) }, class_name: 'Comment'
+end
+
+Post.includes(:published_comments).with_columns(published_comments: [:body, :author])
+```
+
+The scope is respected—only published comments are loaded, and only the specified columns are selected.
+
+## Loading Strategies
+
+Works with all ActiveRecord loading strategies:
+
+- `includes` - Recommended, lets Rails choose the strategy
+- `preload` - Always uses separate queries
+- `eager_load` - Converts to the gem's loading strategy automatically
+
+## Supported Association Types
+
+- `has_many` ✅
+- `has_one` ✅
+- `belongs_to` ✅
+
+All association types work with both `with_columns` and `without_columns`.
+
+## Nested Includes
+
+Nested/chained includes are fully supported with hash syntax:
+
+```ruby
+Post.with_columns(
+  comments: {
+    columns: [:body],
+    include: { author: [:name] }
+  }
+)
+```
+
+This loads posts with their comments (only `body` column) and each comment's author (only `name` column). Foreign keys are automatically included as needed.
+
+### Multi-level Nesting
+
+You can nest as deep as you want:
+
+```ruby
+Post.with_columns(
+  comments: {
+    columns: [:body],
+    include: {
+      author: {
+        columns: [:name],
+        include: { profile: [:website] }
+      }
+    }
+  }
+)
+```
+
+### Automatic Foreign Key Inclusion
+
+Foreign keys are automatically included when you use nested includes, even if you don't explicitly specify them:
+
+```ruby
+# author_id is automatically selected to load the nested author association
+Post.with_columns(
+  comments: {
+    columns: [:body],  # author_id NOT listed, but will be included
+    include: { author: [:name] }
+  }
+)
+```
+
+### Nested with `without_columns`
+
+Works the same way:
+
+```ruby
+Post.without_columns(
+  comments: {
+    columns: [:metadata],  # Exclude metadata from comments
+    include: { author: [:bio] }  # Exclude bio from authors
+  }
+)
+```
+
+## Requirements
+
+- Ruby 3.0+
+- ActiveRecord 7.0+

data/RELEASING.md

@@ -0,0 +1,62 @@
+# Release Checklist
+
+## Before Release
+
+1. **Update version** in `lib/skinny_includes/version.rb`
+2. **Update CHANGELOG.md** with version and date
+3. **Run tests**: `ruby spec/skinny_includes_spec.rb`
+4. **Commit all changes**
+5. **Push to GitHub**
+
+## Release
+
+Run the release script:
+
+```bash
+bin/release
+```
+
+This will:
+- ✅ Check git is clean
+- ✅ Check version tag doesn't exist
+- ✅ Run tests
+- ✅ Build gem
+- ✅ Show files included
+- ✅ Ask for confirmation
+- ✅ Push to rubygems.org
+- ✅ Create and push git tag
+
+## Manual Release (if needed)
+
+```bash
+# Build
+gem build skinny_includes.gemspec
+
+# Push
+gem push skinny_includes-0.1.0.gem
+
+# Tag
+git tag -a v0.1.0 -m "Release v0.1.0"
+git push origin v0.1.0
+```
+
+## After Release
+
+1. Verify on rubygems.org: https://rubygems.org/gems/skinny_includes
+2. Test installation: `gem install skinny_includes`
+3. Update any dependent projects
+
+## Troubleshooting
+
+### "Repushing of gem versions is not allowed"
+You've already pushed this version. Bump the version number.
+
+### "You are not authorized to push"
+Run: `gem signin` or check your rubygems.org credentials
+
+### "Git tag already exists"
+Either delete the tag or bump the version:
+```bash
+git tag -d v0.1.0
+git push origin :refs/tags/v0.1.0
+```

data/Rakefile

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+task default: %i[]

data/lib/skinny_includes/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module SkinnyIncludes
+  VERSION = "0.4.0"
+end

data/lib/skinny_includes.rb

@@ -0,0 +1,291 @@
+# frozen_string_literal: true
+
+require_relative "skinny_includes/version"
+
+module SkinnyIncludes
+  module Relation
+    def with_columns(config)
+      config.each_key do |assoc|
+        reflection = model.reflect_on_association(assoc)
+        raise ArgumentError, "Unknown association: #{assoc}" unless reflection
+      end
+
+      spawn.tap { |relation| relation.with_columns_config!(config) }
+    end
+
+    def without_columns(config)
+      config.each_key do |assoc|
+        reflection = model.reflect_on_association(assoc)
+        raise ArgumentError, "Unknown association: #{assoc}" unless reflection
+      end
+
+      spawn.tap { |relation| relation.without_columns_config!(config) }
+    end
+
+    def with_columns_config!(config)
+      @values[:with_columns] = config
+    end
+
+    def without_columns_config!(config)
+      @values[:without_columns] = config
+    end
+
+    def with_columns_config
+      @values[:with_columns]
+    end
+
+    def without_columns_config
+      @values[:without_columns]
+    end
+
+    def load
+      return super unless with_columns_config || without_columns_config
+      return self if @loaded
+
+      config = with_columns_config || without_columns_config
+      is_without = without_columns_config.present?
+
+      [:preload, :includes].each do |key|
+        if @values[key]&.any?
+          remaining = Array(@values[key]).reject do |val|
+            config.keys.include?(val) || (val.is_a?(Hash) && (val.keys & config.keys).any?)
+          end
+
+          @values[key] = remaining if remaining != @values[key]
+        end
+      end
+
+      super
+
+      config.each do |assoc, column_spec|
+        reflection = model.reflect_on_association(assoc)
+        raise ArgumentError, "Unknown association: #{assoc}" unless reflection
+
+        next if @records.empty?
+
+        assoc_class = reflection.klass
+
+        # Parse column spec - could be array, hash with :columns and :include, or nil
+        columns, nested_includes = parse_column_spec(column_spec, assoc_class, is_without)
+
+        pk = assoc_class.primary_key.to_sym
+        columns |= [pk]
+
+        # If there are nested includes, ensure we select their foreign keys
+        if nested_includes
+          nested_includes.each_key do |nested_assoc|
+            nested_reflection = assoc_class.reflect_on_association(nested_assoc)
+            if nested_reflection
+              nested_fk = nested_reflection.foreign_key.to_sym
+              # Only add FK if it's on the current table (has_many/has_one/belongs_to)
+              if nested_reflection.macro == :belongs_to ||
+                 nested_reflection.macro == :has_many ||
+                 nested_reflection.macro == :has_one
+                columns |= [nested_fk] if assoc_class.column_names.include?(nested_fk.to_s)
+              end
+            end
+          end
+        end
+
+        fk = reflection.foreign_key.to_sym
+        parent_ids = @records.map(&:id).uniq
+
+        case reflection.macro
+        when :has_many, :has_one
+          # For has_many/has_one, the foreign key is on the associated table
+          columns |= [fk] if fk
+
+          # Build base query
+          base_query = assoc_class.where(fk => parent_ids)
+
+          # Apply association scope if present
+          if reflection.scope
+            base_query = base_query.instance_exec(&reflection.scope)
+          end
+
+          associated_records = base_query
+            .select(*columns)
+            .to_a
+
+          grouped = associated_records.group_by(&fk)
+
+          @records.each do |record|
+            records_for_parent = grouped[record.id] || []
+
+            if reflection.macro == :has_many
+              record.association(assoc).target = records_for_parent
+            else
+              record.association(assoc).target = records_for_parent.first
+            end
+
+            record.association(assoc).loaded!
+          end
+
+          # Load nested associations recursively
+          load_nested_associations(associated_records, nested_includes, is_without) if nested_includes && associated_records.any?
+
+        when :belongs_to
+          fk_values = @records.map { |r| r.send(fk) }.compact.uniq
+
+          base_query = assoc_class.where(pk => fk_values)
+
+          if reflection.scope
+            base_query = base_query.instance_exec(&reflection.scope)
+          end
+
+          associated_records = base_query
+            .select(*columns)
+            .index_by(&pk)
+
+          @records.each do |record|
+            fk_value = record.send(fk)
+            if fk_value
+              record.association(assoc).target = associated_records[fk_value]
+              record.association(assoc).loaded!
+            end
+          end
+
+          # Load nested associations recursively
+          load_nested_associations(associated_records.values, nested_includes, is_without) if nested_includes && associated_records.any?
+        end
+
+        # Load nested associations for has_many/has_one (already done above in their case blocks)
+      end
+
+      self
+    end
+
+    private
+
+    def parse_column_spec(column_spec, assoc_class, is_without)
+      if column_spec.is_a?(Hash) && (column_spec.key?(:columns) || column_spec.key?(:include))
+        # New hash syntax: { columns: [:foo], include: { bar: [:baz] } }
+        columns_list = column_spec[:columns]
+        nested = column_spec[:include]
+      else
+        # Legacy array syntax: [:foo, :bar]
+        columns_list = column_spec
+        nested = nil
+      end
+
+      if is_without
+        excluded = Array(columns_list).map(&:to_s)
+        all_columns = assoc_class.column_names
+        columns = (all_columns - excluded).map(&:to_sym)
+      else
+        columns = Array(columns_list || [])
+      end
+
+      [columns, nested]
+    end
+
+    def load_nested_associations(records, nested_config, is_without)
+      return if records.empty? || nested_config.nil? || nested_config.empty?
+
+      # Group records by class (in case of STI)
+      records_by_class = records.group_by(&:class)
+
+      records_by_class.each do |klass, klass_records|
+        nested_config.each do |nested_assoc, nested_column_spec|
+          nested_reflection = klass.reflect_on_association(nested_assoc)
+          raise ArgumentError, "Unknown association: #{nested_assoc}" unless nested_reflection
+
+          nested_assoc_class = nested_reflection.klass
+
+          # Parse nested column spec
+          nested_columns, deeper_nested = parse_column_spec(nested_column_spec, nested_assoc_class, is_without)
+
+          pk = nested_assoc_class.primary_key.to_sym
+          nested_columns |= [pk]
+
+          if deeper_nested
+            deeper_nested.each_key do |deeper_assoc|
+              deeper_reflection = nested_assoc_class.reflect_on_association(deeper_assoc)
+              if deeper_reflection
+                deeper_fk = deeper_reflection.foreign_key.to_sym
+                if nested_assoc_class.column_names.include?(deeper_fk.to_s)
+                  nested_columns |= [deeper_fk]
+                end
+              end
+            end
+          end
+
+          nested_fk = nested_reflection.foreign_key.to_sym
+
+          case nested_reflection.macro
+          when :has_many, :has_one
+            nested_columns |= [nested_fk] if nested_fk
+
+            parent_ids = klass_records.map(&:id).uniq
+            base_query = nested_assoc_class.where(nested_fk => parent_ids)
+
+            if nested_reflection.scope
+              base_query = base_query.instance_exec(&nested_reflection.scope)
+            end
+
+            nested_records = base_query.select(*nested_columns).to_a
+            grouped = nested_records.group_by(&nested_fk)
+
+            klass_records.each do |record|
+              records_for_parent = grouped[record.id] || []
+
+              if nested_reflection.macro == :has_many
+                record.association(nested_assoc).target = records_for_parent
+              else
+                record.association(nested_assoc).target = records_for_parent.first
+              end
+
+              record.association(nested_assoc).loaded!
+            end
+
+            load_nested_associations(nested_records, deeper_nested, is_without) if deeper_nested && nested_records.any?
+
+          when :belongs_to
+            fk_values = klass_records.map { |r| r.send(nested_fk) }.compact.uniq
+
+            base_query = nested_assoc_class.where(pk => fk_values)
+
+            if nested_reflection.scope
+              base_query = base_query.instance_exec(&nested_reflection.scope)
+            end
+
+            nested_records = base_query.select(*nested_columns).index_by(&pk)
+
+            klass_records.each do |record|
+              fk_value = record.send(nested_fk)
+              if fk_value
+                record.association(nested_assoc).target = nested_records[fk_value]
+                record.association(nested_assoc).loaded!
+              end
+            end
+
+            # Recursively load deeper nested associations
+            load_nested_associations(nested_records.values, deeper_nested, is_without) if deeper_nested && nested_records.any?
+          end
+        end
+      end
+    end
+  end
+
+  module ModelMethods
+    extend ActiveSupport::Concern
+
+    class_methods do
+      def with_columns(config)
+        all.with_columns(config)
+      end
+
+      def without_columns(config)
+        all.without_columns(config)
+      end
+    end
+  end
+end
+
+if defined?(ActiveRecord)
+  ActiveRecord::Relation.prepend(SkinnyIncludes::Relation)
+
+  ActiveSupport.on_load(:active_record) do
+    include SkinnyIncludes::ModelMethods
+  end
+end