appquery 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c63ba7b9e16d2f5b7071f8dd9203695f3c46b956d0302ff27a5cab0970cc598f
-  data.tar.gz: 71bebdc43197fce0e8917a3e6f4be0615bcb28215ef004b2496ce0554241c54d
+  metadata.gz: 64e74167bbafa7217db5f9c0bab6efc8b55655abbbcb7e2fccefca3dfe1afae8
+  data.tar.gz: 16f6870106206b547ed307fb6cbcd8d9250610239ccf9dc046e6e6c9719d76dd
 SHA512:
-  metadata.gz: 4f372fb1b1e524e05a2cdf5338d1114f76e08df3b2ec2028e9ec42b515dcbfe4c4c6d9d318a367351e884cfc3ed8e0c670f215d2ccdb072753271fa248993eed
-  data.tar.gz: d91262c52aac1d7cb753eab122c2fbf7d352bd160f5112e3139cf1b5bda848e05456644d3e8e37b2c038ba0795fcfe707470c5bb903cc05e0d0d81a30725de80
+  metadata.gz: 74ce3c5a8d22b2b41bc477069e9720c7b91cad52909ee3e4db8533c0a6f357227ec7e9485f513b8306a0d4994b53556971ae09284c72c2c5370bee45a0c244e3
+  data.tar.gz: 61b01b05753f3a6f27194e47b40bb91aa822c5c263dbe78d0328f1b09aad02d95acbfc7ba191d4fb773b536c1b1ad03465e05c87c4ecf1dc279ed3e461ba97b2

data/Appraisals

@@ -0,0 +1,15 @@
+appraise "rails-70" do
+  gem "rails", "~> 7.0"
+end
+
+appraise "rails-71" do
+  gem "rails", "~> 7.1"
+end
+
+appraise "rails-72" do
+  gem "rails", "~> 7.2"
+end
+
+appraise "rails-80" do
+  gem "rails", "~> 8.0"
+end

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2024 Gert Goet
+Copyright (c) 2025 Gert Goet
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -2,7 +2,7 @@
 
 [![Gem Version](https://badge.fury.io/rb/appquery.svg)](https://badge.fury.io/rb/appquery)
 
-A Rubygem :gem: that makes working with raw SQL queries in Rails projects more convenient.  
+A Rubygem :gem: that makes working with raw SQL (READ) queries in Rails projects more convenient.  
 Specifically it provides:
 - **...a dedicated folder for queries**  
   e.g. `app/queries/reports/weekly.sql` is instantiated via `AppQuery["reports/weekly"]`.
@@ -13,6 +13,31 @@ Specifically it provides:
     invoke  rspec
     create    spec/queries/reports/weekly_query_spec.rb
   ```
+- **...ERB templating**  
+  Simple ERB templating with helper-functions:
+  ```sql
+  -- app/queries/contracts.sql.erb
+  SELECT * FROM contracts
+  <%= order_by(order) %>
+  ```
+  ```ruby
+  AppQuery["contracts.sql.erb"].render(order: {year: :desc, month: :desc}).select_all
+  ```
+- **...positional and named binds**  
+  Intuitive binds:
+  ```ruby
+  AppQuery(%{select now() - (:interval)::interval as some_date}).select_value(binds: {interval: '1 day'})
+  AppQuery(<<~SQL).select_all(binds: [2.day.ago, Time.now, '5 minutes']).column("series")
+    select generate_series($1::timestamp, $2::timestamp, $3::interval) as series
+  SQL
+  ```
+- **...casting**  
+  Automatic and custom casting:
+  ```ruby
+  AppQuery(%{select array[1,2]}).select_value #=> [1,2]
+  cast = {"data" => ActiveRecord::Type::Json.new}
+  AppQuery(%{select '{"a": 1}' as data}).select_value(cast:)
+  ```
 - **...helpers to rewrite a query for introspection during development and testing**  
   See what a CTE yields: `query.select_all(select: "SELECT * FROM some_cte")`.  
   Query the end result: `query.select_one(select: "SELECT COUNT(*) FROM _ WHERE ...")`.  
@@ -38,7 +63,12 @@ Specifically it provides:
 
 > [!IMPORTANT]  
 > **Status**: alpha. API might change. See the CHANGELOG for breaking changes when upgrading.
->  
+>
+
+## Rationale
+
+Sometimes ActiveRecord doesn't cut it, and you'd rather use raw SQL to get the right data out. That, however, introduces some new problems. First of all, you'll run into the not-so-intuitive use of [select_(all|one|value)](https://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/DatabaseStatements.html#method-i-select_all) — for example, how they differ with respect to type casting, and how their behavior can vary between ActiveRecord versions. Then there's the testability, introspection, and maintainability of the resulting SQL queries.  
+This library aims to alleviate all of these issues by providing a consistent interface across select_* methods and ActiveRecord versions. It should make inspecting and testing queries easier—especially when they're built from CTEs.
 
 ## Installation
 
@@ -53,7 +83,83 @@ bundle add appquery
 > [!NOTE]
 > The following (trivial) examples are not meant to convince you to ditch your ORM, but just to show how this gem handles raw SQL queries.
 
-### Create
+### ...from console
+
+Testdriving can be easily done from the console. Either by cloning this repository (recommended, see `Development`-section) or installing the gem in an existing Rails project.  
+<details>
+  <summary>Database setup (the `bin/console`-script does this for your)</summary>
+  
+  ```ruby
+  ActiveRecord::Base.logger = Logger.new(STDOUT)
+  ActiveRecord::Base.establish_connection(url: 'postgres://localhost:5432/some_db')
+  ```
+</details>
+
+The following examples assume PostgreSQL (SQLite where stated):
+
+```ruby
+# showing select_(all|one|value)
+> AppQuery(%{select date('now') as today}).select_all.to_a
+=> [{"today" => "2025-05-10"}]
+> AppQuery(%{select date('now') as today}).select_one
+=> {"today" => "2025-05-10"}
+> AppQuery(%{select date('now') as today}).select_value
+=> "2025-05-10"
+
+# binds
+# positional binds
+> AppQuery(%{select now() - ($1)::interval as date}).select_value(binds: ['2 days'])
+# named binds
+> AppQuery(%{select now() - (:interval)::interval as date}).select_value(binds: {interval: '2 days'})
+
+# casting
+> AppQuery(%{select date('now') as today}).select_all(cast: true).to_a
+=> [{"today" => Sat, 10 May 2025}]
+
+## SQLite doesn't have a notion of dates or timestamp's so casting won't do anything:
+sqlite> AppQuery(%{select date('now') as today}).select_one(cast: true)
+=> {"today" => "2025-05-12"}
+## Providing per-column-casts fixes this:
+casts = {"today" => ActiveRecord::Type::Date.new}
+sqlite> AppQuery(%{select date('now') as today}).select_one(cast: casts)
+=> {"today" => Mon, 12 May 2025}
+
+# rewriting queries (using CTEs)
+q = AppQuery(<<~SQL)
+  WITH articles(id,title,published_on) AS (
+    values(1, 'Some title', '2024-3-31'),
+          (2, 'Other title', '2024-10-31'),
+          (3, 'Same title?', '2024-3-31'))
+  select * from articles order by id DESC
+SQL
+
+## query the articles-CTE
+q.select_all(select: %{select * from articles where id < 2}).to_a
+
+## query the end-result (available as the CTE named '_')
+q.select_one(select: %{select * from _ limit 1})
+
+## ERB templating
+# Extract a query from q that can be sorted dynamically:
+q2 = q.with_select("select id,title,published_on::date from articles <%= order_by(order) %>")
+q2.render(order: {"published_on::date": :desc, 'lower(title)': "asc"}).select_all.entries
+# shows latest articles first, and titles sorted alphabetically
+# for articles published on the same date.
+# order_by raises when it's passed something that would result in just `ORDER BY`:
+q2.render(order: {})
+# doing a select using a query that should be rendered, a `AppQuery::UnrenderedQueryError` will be raised:
+q2.select_all.entries
+
+# NOTE you can use both `order` and `@order`: local variables like `order` are required,
+# while instance variables like `@order` are optional.
+# To skip the order-part when provided:
+<%= @order.presence && order_by(order) %>
+# or use a default when order-part is always wanted but not always provided:
+<%= order_by(@order || {id: :desc}) %>
+```
+
+
+### ...in a Rails project
 
 > [!NOTE]
 > The included [example Rails app](./examples/ror) contains all data and queries described below.
@@ -462,17 +568,29 @@ query.replace_cte("recent_articles as (select values(1, 'Some article'))")
 
 - 💾 tested with **SQLite** and **PostgreSQL**
 - 🚆 tested with Rails **v6.1**, **v7** and **v8.0**
-- 💎 requires Ruby **>v3.1**  
+- 💎 requires Ruby **>v3.2**  
   Goal is to support [maintained Ruby versions](https://www.ruby-lang.org/en/downloads/branches/).
 
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. **Make sure to check it exits with status code 0.**
 
-Using [direnv](https://direnv.net/) for env-vars recommended.
+Using [mise](https://mise.jdx.dev/) for env-vars recommended.
+
+### console
+
+The [console-script](./bin/console) is setup such that it's easy to connect with a database and experiment with the library:
+```bash
+$ ./bin/console sqlite3::memory:
+$ ./bin/console postgres://localhost:5432/some_db
+
+# more details
+$ ./bin/console -h
+```
 
+### various
 
-Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+Run `rake spec` to run the tests.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
@@ -483,4 +601,3 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/eval/a
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
-

data/lib/app_query/base.rb

@@ -0,0 +1,45 @@
+module AppQuery
+  class Base
+    class_attribute :_cast, default: true, instance_predicate: false
+    class_attribute :_default_binds, default: {}, instance_predicate: false
+
+    class << self
+      def run(build_only: false, binds: {}, vars: {}, cast: self.cast, select: nil, **)
+        _build(binds:, vars:, cast:, select:).then do
+          build_only ? _1 : _1.select_all
+        end
+      end
+
+      def build(**opts)
+        run(build_only: true, **opts)
+      end
+
+      def default_binds(v = nil)
+        return _default_binds if v.nil?
+        self._default_binds = v
+      end
+
+      def cast(v = nil)
+        return _cast if v.nil?
+        self._cast = v
+      end
+
+      def query_name
+        derive_query_name unless defined?(@query_name)
+        @query_name
+      end
+
+      attr_writer :query_name
+
+      private
+
+      def _build(cast:, binds: {}, select: nil, vars: {})
+        AppQuery[query_name, binds:, cast:].render(vars).with_select(select)
+      end
+
+      def derive_query_name
+        self.query_name = name.underscore.sub(/_query$/, "")
+      end
+    end
+  end
+end

data/lib/app_query/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AppQuery
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/app_query.rb

@@ -7,6 +7,8 @@ require "active_record"
 module AppQuery
   class Error < StandardError; end
 
+  class UnrenderedQueryError < StandardError; end
+
   Configuration = Struct.new(:query_path)
 
   def self.configuration
@@ -24,10 +26,14 @@ module AppQuery
   end
   reset_configuration!
 
-  def self.[](v)
-    query_name = v.to_s
-    full_path = (Pathname.new(configuration.query_path) / "#{query_name}.sql").expand_path
-    Q.new(full_path.read, name: "AppQuery #{query_name}")
+  # Examples:
+  #   AppQuery[:invoices] # looks for invoices.sql
+  #   AppQuery["reports/weekly"]
+  #   AppQuery["invoices.sql.erb"]
+  def self.[](query_name, **opts)
+    filename = File.extname(query_name.to_s).empty? ? "#{query_name}.sql" : query_name.to_s
+    full_path = (Pathname.new(configuration.query_path) / filename).expand_path
+    Q.new(full_path.read, name: "AppQuery #{query_name}", filename: full_path.to_s, **opts)
   end
 
   class Result < ActiveRecord::Result
@@ -50,6 +56,10 @@ module AppQuery
       rows.map { _1[ix] }
     end
 
+    def size
+      count
+    end
+
     def self.from_ar_result(r, cast = nil)
       if r.empty?
         EMPTY
@@ -73,7 +83,7 @@ module AppQuery
           # => [["{1,2}"]]
           # > ActiveRecord::Base.connection.select_all("select array[1,2]").cast_values
           # => [[1, 2]]
-          rows = rows.map { [_1] } if r.columns.one?
+          rows = rows.zip if r.columns.one?
           new(r.columns, rows, overrides, cast: true)
         end
       end
@@ -88,27 +98,102 @@ module AppQuery
   end
 
   class Q
-    attr_reader :name, :sql
+    attr_reader :name, :sql, :binds, :cast
 
-    def initialize(sql, name: nil)
+    def initialize(sql, name: nil, filename: nil, binds: [], cast: true)
       @sql = sql
       @name = name
+      @filename = filename
+      @binds = binds
+      @cast = cast
+    end
+
+    def deep_dup
+      super.send(:reset!)
     end
 
-    def select_all(binds: [], select: nil, cast: false)
-      with_select(select).then do |aq|
-        ActiveRecord::Base.connection.select_all(aq.to_s, name, binds).then do |result|
-          Result.from_ar_result(result, cast)
+    def reset!
+      (instance_variables - %i[@sql @filename @name @binds @cast]).each do
+        instance_variable_set(_1, nil)
+      end
+      self
+    end
+    private :reset!
+
+    def render(vars)
+      vars ||= {}
+      with_sql(to_erb.result(render_helper(vars).get_binding))
+    end
+
+    def to_erb
+      ERB.new(sql, trim_mode: "-").tap { _1.location = [@filename, 0] if @filename }
+    end
+    private :to_erb
+
+    def render_helper(vars)
+      Module.new do
+        extend self
+
+        vars.each do |k, v|
+          define_method(k) { v }
+          instance_variable_set(:"@#{k}", v)
+        end
+
+        # Examples
+        #   <%= order_by({year: :desc, month: :desc}) %>
+        #   #=> ORDER BY year DESC, month DESC
+        #
+        # Using variable:
+        #   <%= order_by(ordering) %>
+        # NOTE Raises when ordering not provided or when blank.
+        #
+        # Make it optional:
+        #   <%= @ordering.presence && order_by(ordering) %>
+        #
+        def order_by(hash)
+          raise ArgumentError, "Provide columns to sort by, e.g. order_by(id: :asc)  (got #{hash.inspect})." unless hash.present?
+          "ORDER BY " + hash.map do |k, v|
+            v.nil? ? k : [k, v.upcase].join(" ")
+          end.join(", ")
+        end
+
+        def get_binding
+          binding
         end
       end
     end
+    private :render_helper
+
+    def select_all(binds: [], select: nil, cast: self.cast)
+      binds = binds.presence || @binds
+      with_select(select).render({}).then do |aq|
+        if binds.is_a?(Hash)
+          sql = if ActiveRecord::VERSION::STRING.to_f >= 7.1
+            Arel.sql(aq.to_s, **binds)
+          else
+            ActiveRecord::Base.sanitize_sql_array([aq.to_s, **binds])
+          end
+          ActiveRecord::Base.connection.select_all(sql, name).then do |result|
+            Result.from_ar_result(result, cast)
+          end
+        else
+          ActiveRecord::Base.connection.select_all(aq.to_s, name, binds).then do |result|
+            Result.from_ar_result(result, cast)
+          end
+        end
+      end
+    rescue NameError => e
+      # Prevent any subclasses, e.g. NoMethodError
+      raise e unless e.instance_of?(NameError)
+      raise UnrenderedQueryError, "Query is ERB. Use #render before select-ing."
+    end
 
-    def select_one(binds: [], select: nil, cast: false)
-      select_all(binds:, select:, cast:).first || {}
+    def select_one(binds: [], select: nil, cast: self.cast)
+      select_all(binds:, select:, cast:).first
     end
 
-    def select_value(binds: [], select: nil, cast: false)
-      select_one(binds:, select:, cast:).values.first
+    def select_value(binds: [], select: nil, cast: self.cast)
+      select_one(binds:, select:, cast:)&.values&.first
     end
 
     def tokens
@@ -123,13 +208,31 @@ module AppQuery
       tokens.filter { _1[:t] == "CTE_IDENTIFIER" }.map { _1[:v] }
     end
 
+    def with_binds(binds)
+      deep_dup.tap do
+        _1.instance_variable_set(:@binds, binds)
+      end
+    end
+
+    def with_cast(cast)
+      deep_dup.tap do
+        _1.instance_variable_set(:@cast, cast)
+      end
+    end
+
+    def with_sql(sql)
+      deep_dup.tap do
+        _1.instance_variable_set(:@sql, sql)
+      end
+    end
+
     def with_select(sql)
-      return self unless sql
+      return self if sql.nil?
       if cte_names.include?("_")
-        self.class.new(tokens.each_with_object([]) do |token, acc|
+        with_sql(tokens.each_with_object([]) do |token, acc|
           v = (token[:t] == "SELECT") ? sql : token[:v]
           acc << v
-        end.join, name: name)
+        end.join)
       else
         append_cte("_ as (\n  #{select}\n)").with_select(sql)
       end
@@ -152,10 +255,10 @@ module AppQuery
       end
 
       if cte_names.none?
-        self.class.new("WITH #{cte}\n#{self}")
+        with_sql("WITH #{cte}\n#{self}")
       else
         split_at_type = recursive? ? "RECURSIVE" : "WITH"
-        self.class.new(tokens.map do |token|
+        with_sql(tokens.map do |token|
           if token[:t] == split_at_type
             token[:v] + to_append.map { _1[:v] }.join
           else
@@ -175,11 +278,11 @@ module AppQuery
       end
 
       if cte_names.none?
-        self.class.new("WITH #{cte}\n#{self}")
+        with_sql("WITH #{cte}\n#{self}")
       else
         nof_ctes = cte_names.size
 
-        self.class.new(tokens.map do |token|
+        with_sql(tokens.map do |token|
           nof_ctes -= 1 if token[:t] == "CTE_SELECT"
 
           if nof_ctes.zero?
@@ -212,7 +315,7 @@ module AppQuery
 
       cte_found = false
 
-      self.class.new(tokens.map do |token|
+      with_sql(tokens.map do |token|
         if cte_found ||= token[:t] == "CTE_IDENTIFIER" && token[:v] == cte_name
           unless (cte_found = (token[:t] != "CTE_SELECT"))
             next to_append.map { _1[:v] }.join
@@ -243,3 +346,5 @@ rescue LoadError
 end
 
 require_relative "app_query/rspec" if Object.const_defined? :RSpec
+
+require "app_query/base" if defined?(ActiveRecord::Base)

data/mise.local.toml.example

@@ -0,0 +1,5 @@
+[env]
+# used for tests
+PG_DATABASE_URL="postgres://localhost:5432/some_db
+# used from console
+DATABASE_URL="postgres://localhost:5432/some_db

data/mise.toml

@@ -0,0 +1,6 @@
+[tools]
+ruby = "3.4"
+
+[env]
+_.path = ["bin"]
+PROJECT_ROOT = "{{config_root}}"

metadata

@@ -1,15 +1,28 @@
 --- !ruby/object:Gem::Specification
 name: appquery
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Gert Goet
-autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-11-13 00:00:00.000000000 Z
-dependencies: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: appraisal
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: "Improving introspection and testability of raw SQL queries in Rails\nThis
   gem improves introspection and testability of raw SQL queries in Rails by:\n- ...providing
   a separate query-folder and easy instantiation  \n  A query like `AppQuery[:some_query]`
@@ -27,15 +40,15 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".envrc"
-- ".envrc.private.example"
 - ".rspec"
 - ".standard.yml"
+- Appraisals
 - CHANGELOG.md
 - LICENSE.txt
 - README.md
 - Rakefile
 - lib/app_query.rb
+- lib/app_query/base.rb
 - lib/app_query/rspec.rb
 - lib/app_query/rspec/helpers.rb
 - lib/app_query/tokenizer.rb
@@ -46,8 +59,9 @@ files:
 - lib/rails/generators/query/templates/query.sql.tt
 - lib/rails/generators/rspec/query_generator.rb
 - lib/rails/generators/rspec/templates/query_spec.rb.tt
+- mise.local.toml.example
+- mise.toml
 - sig/appquery.rbs
-- tmp/.gitkeep
 homepage: https://github.com/eval/appquery
 licenses:
 - MIT
@@ -55,7 +69,6 @@ metadata:
   homepage_uri: https://github.com/eval/appquery
   source_code_uri: https://github.com/eval/appquery
   changelog_uri: https://github.com/eval/gem-try/blob/main/CHANGELOG.md
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -63,15 +76,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 3.1.0
+      version: 3.2.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.22
-signing_key:
+rubygems_version: 3.6.7
 specification_version: 4
 summary: "raw SQL \U0001F966, cooked \U0001F372 or: make working with raw SQL queries
   in Rails convenient by improving their introspection and testability."

data/.envrc

@@ -1,6 +0,0 @@
-# direnv config
-PATH_add bin
-
-export APP_ROOT=$(pwd)
-
-source_env_if_exists .envrc.private

data/.envrc.private.example

@@ -1,2 +0,0 @@
-# copy this to .envrc.private to use with direnv
-export PG_DATABASE_URL=postgres://localhost:5432/some_db