quote-sql 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 13f64b3b81c62230c2814c487d3a5b2e56109b651297a599449a2b8edd91b3d8
+  data.tar.gz: 8fc4b373eb65ab1d6ba2e86a32db5d60caf3bef38096e9a098127d9f892a4abe
+SHA512:
+  metadata.gz: d58cc18c35110eef2ddb9c08f1e23f9b353ab7fc73939063fa3a576156c0fd96b05099e8fe8742fcb4c46261735fc47d547d24f6218ce9f6776c8e91be873211
+  data.tar.gz: 7facea8c215baf9c9196512b636b3ae866343eea0ca8137589b25f3cd6684a4fa815661c05059be58c11ace2c0fc6ac3962d96d02c3c9be1fdef628a00c6349d

data/README.md

@@ -0,0 +1,123 @@
+# QuoteSql - Tool to build and run SQL queries easier
+I've built this library as an addition to ActiveRecord and Arel, however you can use it with any sql database and plain Ruby.
+
+Creating SQL queries and proper quoting becomes complicated especially when you need advanced queries.
+
+I created this library while coding for different projects, and had lots of Heredoc SQL queries, which pretty quickly becomes the kind of: 
+> When I wrote these lines of code, just me and God knew what they mean. Now its just God.
+
+My strategy is to segment SQL Queries in readable junks, which can be individually tested and then combine their sql to the final query.
+
+QuoteSql is used in production, but is still evolving.
+
+If you think QuoteSql is interesting, let's chat!
+Also if you have problems using it, just drop me a note.
+
+Best Martin
+
+## Examples
+### Simple quoting
+`QuoteSql.new("SELECT %field").quote(field: "abc").to_sql`
+  => SELECT 'abc'
+
+`QuoteSql.new("SELECT %field__text").quote(field__text: 9).to_sql`
+=> SELECT 9::TEXT
+
+### Quoting of columns and table from a model - or an object responding to table_name and column_names or columns
+`QuoteSql.new("SELECT %columns FROM %table_name").quote(table: User).to_sql`
+  => SELECT "id",firstname","lastname",... FROM "users"
+### Injecting raw sql in a query
+`QuoteSql.new("SELECT a,b,%raw FROM table").quote(raw: "jsonb_build_object('a', 1)").to_sql`
+  => SELECT "a,b,jsonb_build_object('a', 1) FROM table
+
+### Injecting ActiveRecord, Arel.sql or QuoteSql
+`QuoteSql.new("SELECT %column_names FROM (%any_name) a").
+    quote(any_name: User.select("%column_names").where(id: 3), column_names: [:firstname, :lastname]).to_sql`
+  => SELECT firstname, lastname FROM (SELECT firstname, lastname FROM users where id = 3)
+
+### Insert of values quoted and sorted with columns
+Values are be ordered in sequence of columns. Missing value entries are substitured with DEFAULT.
+`QuoteSql.new("INSERT INTO %table (%columns) VALUES %values ON CONFLICT (%constraint) DO NOTHING").
+    quote(table: User, values: [
+      {firstname: "Albert", id: 1, lastname: "Müller"},
+      {lastname: "Schultz", firstname: "herbert"}
+    ], constraint: :id).to_sql`
+  => INSERT INTO "users" ("id", "firstname", "lastname", "created_at")
+      VALUES (1, 'Albert', 'Müller', CURRENT_TIMESTAMP), (DEFAULT, 'herbert', 'Schultz', CURRENT_TIMESTAMP)
+      ON CONFLICT ("id") DO NOTHING
+      
+### Columns from a list
+`QuoteSql.new("SELECT %columns").quote(columns: [:a, :"b.c", c: "jsonb_build_object('d', 1)"]).to_sql`
+  => SELECT "a","b"."c",jsonb_build_object('d', 1) AS c
+
+## Substitution of 
+  In the SQL matches of `%foo` or `%{foo}` or `%foo_4_bar` or `%{foo_4_bar}` the *"mixins"*
+  are substituted with quoted values
+  the values are looked up from the options given in the quotes method
+  the mixins can be recursive.
+  **Caution! You need to take care, no protection against infinite recursion **
+  
+### Special mixins
+- `%table` | `%table_name` | `%table_names`
+- `%column` | `%columns` | `%column_names`
+- `%ident` | `%constraint` | `%constraints` quoting for database columns
+- `%raw` | `%sql` inserting raw SQL
+- `%value` | `%values` creates value section for e.g. insert
+  - In the right order
+    - Single value => (2)
+    - +Array+ => (column, column, column) n.b. has to be the correct order
+    - +Array+ of +Array+ => (...),(...),(...),...
+  - if the columns option is given (or implicitely by setting table)
+    - +Hash+ values are ordered according to the columns option, missing values are replaced by `DEFAULT`
+    - +Array+ of +Hash+ multiple record insert
+- `%bind` is replaced with the current bind sequence.
+  Without appended number the first %bind => $1, the second => $2 etc.
+  - %bind\\d+ => $+Integer+ e.g. `%bind7` => $7
+  - `%bind__text` => $1 and it is registered as text - this is used in prepared statements (TO BE IMPLEMENTED)
+  - `%key_bind__text` => $1 and it is registered as text when using +Hash+ in the execute
+    $1 will be mapped to the key's value in the +Hash+ TODO
+
+All can be preceded by additional letters and underscore e.g. `%foo_bar_column`
+
+### Type casts
+A database typecast is added to fields ending with double underscore and a valid db data type
+with optional array dimension
+
+- `%field__jsonb` => adds a `::JSONB` typecast to the field
+- `%number_to__text` => adds a `::TEXT` typecast to the field
+- `%array__text1` => adds a `::TEXT[]` (TO BE IMPLEMENTED)
+- `%array__text2` => adds a `::TEXT[][]` (TO BE IMPLEMENTED)
+
+### Quoting
+- Any value of the standard mixins are quoted with these exceptions
+- +Array+ are quoted as DB Arrays unless a type cast is given e.g. __jsonb
+- +Hash+ are quoted as jsonb unless a type cast is given e.g. __json
+- When the value responds to :to_sql or is a +Arel::Nodes::SqlLiteral+ its added as raw SQL
+- +Proc+ are executed with the +QuoteSQL::Quoter+ object as parameter and added as raw SQL
+
+### Special quoting columns
+- +String+ or +Symbol+ without a dot  e.g. :firstname => "firstname"
+- +String+ or +Symbol+ containing a dot e.g. "users.firstname" or => "users"."firstname"
+- +Array+
+  - +String+ and +Symbols+ see above
+  - +Hash+ see below
+- +Hash+ or within the +Array+
+  - +Symbol+ value will become the column name e.g. {table: :column} => "table"."column"
+  - +String+ value will become the expression, the key the AS {result: "SUM(*)"} => SUM(*) AS result
+  - +Proc+ are executed with the +QuoteSQL::Quoter+ object as parameter and added as raw SQL
+
+## Installing
+`gem install quote-sql`
+or in Gemfile
+`gem 'quote-sql'`
+
+### Ruby on Rails
+Add this to config/initializers/quote_sql.rb
+
+    ActiveSupport.on_load(:active_record) do
+      require 'quote_sql'
+      QuoteSql.db_connector = ActiveRecord::Base
+      String.include QuoteSql::Extension
+      ActiveRecord::Relation.include QuoteSql::Extension
+    end  
+

data/lib/quote_sql/connector/active_record_base.rb

@@ -0,0 +1,37 @@
+class QuoteSql
+  module Connector
+    module ActiveRecordBase
+      module ClassMethods
+        def conn
+          ::ActiveRecord::Base.connection
+        end
+
+        def quote_column_name(name)
+          conn.quote_column_name(name)
+        end
+
+        def quote(name)
+          conn.quote(name)
+        end
+      end
+
+      def conn
+        self.class.conn
+      end
+
+      def _exec_query(sql, binds = [], prepare: false, async: false)
+        conn.exec_query(sql, "SQL", binds, prepare:, async:)
+      end
+
+      def _exec(sql, binds = [], prepare: false, async: false)
+        options = { prepare:, async: }
+        result = _exec_query(sql, binds, **options)
+        columns = result.columns.map(&:to_sym)
+        result.cast_values.map do |row|
+          row = [row] unless row.is_a? Array
+          [columns, row].transpose.to_h
+        end
+      end
+    end
+  end
+end

data/lib/quote_sql/connector.rb

@@ -0,0 +1,13 @@
+class QuoteSql
+  module Connector
+    def self.set(klass)
+      file = "/" + klass.to_s.underscore.tr("/", "_")
+      require __FILE__.sub(/\.rb$/, file)
+      const_set :CONNECTOR, (to_s + file.classify).constantize
+      QuoteSql.include CONNECTOR
+      class << QuoteSql
+        prepend CONNECTOR::ClassMethods
+      end
+    end
+  end
+end

data/lib/quote_sql/deprecated.rb

@@ -0,0 +1,162 @@
+class QuoteSql
+  module Deprecated
+    private def conn
+      ApplicationRecord.connection
+    end
+
+    private def quote_sql_values(sub, casts)
+      sub.map do |s|
+        casts.map do |k, column|
+          column.transform_keys(&:to_sym) => { sql_type:, default:, array: }
+          value = s.key?(k) ? s[k] : s[k.to_sym]
+          if value.nil?
+            value = default
+          else
+            value = value.to_json if sql_type[/^json/]
+          end
+          "#{conn.quote(value)}::#{sql_type}"
+        end.join(",")
+      end
+    end
+
+    def quote_sql(**options)
+      loop do
+        # keys = []
+        break unless gsub!(%r{(?<=^|\W)[:$](#{options.keys.join("|")})(?=\W|$)}) do |m|
+          key = m[1..].to_sym
+          # keys << key
+          next m unless options.key? key
+          sub = options[key]
+          case sub
+          when Arel::Nodes::SqlLiteral
+            next sub
+          when NilClass
+            next "NULL"
+          when TrueClass, FalseClass
+            next sub.to_s.upcase
+          when Time
+            sub = sub.strftime("%Y-%m-%d %H:%M:%S.%3N%z")
+          end
+          if sub.respond_to? :to_sql
+            next sub.to_sql
+          end
+          case m
+          when /^:(.+)_(FROM_CLAUSE)$/ # prefix (column,...) AS ( VALUES (data::CAST, ...), ...)
+            name = conn.quote_column_name($1)
+            casts = sub.shift.transform_keys(&:to_s)
+            rv = quote_sql_values(sub, casts)
+            column_names = casts.map { conn.quote_column_name(_2.key?(:as) ? _2[:as] : _1) }
+            next "(VALUES \n(#{rv.join("),\n(")})\n ) #{name} (#{column_names.join(",") })"
+
+          when /^:(.+)_(as_select)$/i # prefix (column,...) AS ( VALUES (data::CAST, ...), ...)
+            name = conn.quote_column_name($1)
+            casts = sub.shift.transform_keys(&:to_s)
+            rv = quote_sql_values(sub, casts)
+            next "SELECT * FROM (VALUES \n(#{rv.join("),\n(")})\n ) #{name} (#{casts.keys.map { conn.quote_column_name(_1) }.join(",") })"
+          when /^:(.+)_(as_values)$/i # prefix (column,...) AS ( VALUES (data::CAST, ...), ...)
+            name = conn.quote_column_name($1)
+            casts = sub.shift.transform_keys(&:to_s)
+            rv = quote_sql_values(sub, casts)
+            next "#{name} (#{casts.keys.map { conn.quote_column_name(_1) }.join(",") }) AS ( VALUES \n(#{rv.join("),\n(")})\n )"
+          when /^:(.+)_(values)$/i
+            casts = sub.shift.transform_keys(&:to_sym)
+            rv = quote_sql_values(sub, casts)
+            next "VALUES \n(#{rv.join("),\n(")})\n"
+          when /_(LIST)$/i
+            next sub.map { conn.quote _1 }.join(",")
+          when /_(args)$/i
+            next sub.join(',')
+          when /_(raw|sql)$/i
+            next sub
+          when /_(ident|column)$/i, /table_name$/, /_?columns?$/, /column_names$/
+            if sub.is_a? Array
+              next sub.map do
+                _1[/^"[^"]+"\."[^"]+"$/] ? _1 : conn.quote_column_name(_1)
+              end.join(',')
+            else
+              next conn.quote_column_name(sub)
+            end
+          when /(?<=_)jsonb?$/i
+            next conn.quote(sub.to_json) + "::#{$MATCH}"
+          when /(?<=_)(uuid|int|text)$/i
+            cast = "::#{$MATCH}"
+          end
+          case sub
+          when Regexp
+            sub.to_postgres
+          when Array
+            dims = 1 # todo more dimensional Arrays
+            dive = ->(ary) do
+              ary.map { |s| conn.quote s }.join(',')
+            end
+            sub = "[#{dive.call sub}]"
+            cast += "[]" * dims if cast.present?
+            "ARRAY#{sub}#{cast}"
+          else
+            "#{conn.quote(sub)}#{cast}"
+          end
+        end
+        # break if options.except!(*keys).blank?
+      end
+      Arel.sql self
+    end
+
+    def exec
+      result = conn.exec_query(self)
+      columns = result.columns.map(&:to_sym)
+      result.cast_values.map do |row|
+        row = [row] unless row.is_a? Array
+        [columns, row].transpose.to_h
+      end
+    end
+
+    def quote_exec(**)
+      quote_sql(**).exec
+    end
+
+    module Dsql
+
+      def dsql
+        IO.popen(PG_FORMAT_BIN, "r+", err: "/dev/null") do |f|
+          f.write self
+          f.close_write
+          puts f.read
+        end
+        self
+      rescue
+        self
+      end
+    end
+
+    include Dsql
+
+    module String
+      def self.included(other)
+        other.include Dsql
+      end
+
+      def quote_sql(**)
+        Arel.sql(self).quote_sql(**)
+      end
+    end
+
+    module Relation
+      def quote_sql(**)
+        Arel.sql(to_sql).quote_sql(**)
+      end
+
+      def dsql
+        to_sql.dsql
+        self
+      end
+
+      def result
+        result = ApplicationRecord.connection.exec_query(to_sql)
+        columns = result.columns.map(&:to_sym)
+        result.cast_values.map do |row|
+          [columns, Array(row)].transpose.to_h
+        end
+      end
+    end
+  end
+end

data/lib/quote_sql/extension.rb

@@ -0,0 +1,14 @@
+class QuoteSql
+  module Extension
+    def self.included(other)
+      other.include QuoteSql::Formater
+    end
+
+    def quote_sql(**)
+      QuoteSql.new(self).quote(**)
+    end
+
+    alias qsql quote_sql
+  end
+end
+

data/lib/quote_sql/formater.rb

@@ -0,0 +1,23 @@
+class QuoteSql
+  module Formater
+    PG_FORMAT_BIN = `which pg_format`.chomp.presence
+
+    def dsql
+      puts to_formatted_sql
+      nil
+    end
+
+    def to_formatted_sql
+      sql = respond_to?(:to_sql) ? to_sql : to_s
+      IO.popen(PG_FORMAT_BIN, "r+", err: "/dev/null") do |f|
+        f.write(sql)
+        f.close_write
+        f.read
+      end
+    rescue
+      sql
+    end
+
+    alias to_sqf to_formatted_sql
+  end
+end

data/lib/quote_sql/quoter.rb

@@ -0,0 +1,188 @@
+class QuoteSql
+  class Quoter
+    def initialize(qsql, key, quotable)
+      @qsql = qsql
+      @key, @quotable = key, quotable
+    end
+
+    attr_reader :key, :quotable
+
+    def to_sql
+      return @quotable.call(self) if @quotable.is_a? Proc
+      case key.to_s
+      when /(?:^|(.*)_)table$/i
+        table
+      when /(?:^|(.*)_)columns?$/i
+        column_names
+      when /(?:^|(.*)_)(table_name?s?)$/i
+        table_name
+      when /(?:^|(.*)_)(column_name?s?)$/i
+        ident_name
+      when /(?:^|(.*)_)(ident|args)$/i
+        ident_name
+      when /(?:^|(.*)_)constraints?$/i
+        quotable.to_s
+      when /(?:^|(.*)_)(raw|sql)$/
+        quotable.to_s
+      when /(?:^|(.*)_)(values?)$/
+        values
+      else
+        quote
+      end
+    end
+
+    private def value(ary)
+      column_names = @qsql.column_names
+      if ary.is_a?(Hash) and column_names.present?
+        ary = @qsql.column_names.map do |column_name|
+          if ary.key? column_name&.to_sym
+            ary[column_name.to_sym]
+          elsif column_name[/^(created|updated)_at$/]
+            :current_timestamp
+          else
+            :default
+          end
+        end
+      end
+      "(" + ary.map do |i|
+        case i
+        when :default, :current_timestamp
+          next i.to_s.upcase
+        when Hash, Array
+          i = i.to_json
+        end
+        _quote(i)
+      end.join(",") + ")"
+
+    end
+
+    def values(item = @quotable)
+      case item
+      when Arel::Nodes::SqlLiteral
+        item = Arel.sql("(#{item})") unless item[/^\s*\(/] and item[/\)\s*$/]
+        return item
+      when Array
+
+        differences = item.map { _1.is_a?(Array) && _1.length }.uniq
+        if differences.length == 1
+          item.compact.map { value(_1) }.join(", ")
+        else
+          value([item])
+        end
+      when Hash
+        value([item])
+      else
+        return item.to_sql if item.respond_to? :to_sql
+        "(" + _quote(item) + ")"
+      end
+    end
+
+    def cast
+      if m = key.to_s[CASTS]
+        m[2..].sub(CASTS) { _1.tr("_", " ") }
+      end
+    end
+
+    def json?
+      !!key[/(^|_)(jsonb?)$/]
+    end
+
+    private def _quote(item = @quotable, cast = self.cast)
+      rv = QuoteSql.quote(item)
+      if cast
+        rv << "::#{cast}"
+        rv << "[]" * rv.depth if rv[/^ARRAY/]
+      end
+      rv
+    end
+
+    private def _quote_column_name(name, column = nil)
+      name, column = name.to_s.split(".") if column.nil?
+      rv = QuoteSql.quote_column_name(name)
+      return rv unless column.present?
+      rv + "." + QuoteSql.quote_column_name(column)
+    end
+
+    def quote(item = @quotable)
+      case item
+      when Arel::Nodes::SqlLiteral
+        return item
+      when Array
+        return _quote(item.to_json) if json?
+        _quote(item)
+      when Hash
+        return _quote(item.to_json) if json?
+        item.map do |as, item|
+          "#{_quote(item)} AS #{as}"
+        end.join(",")
+      else
+        return item.to_sql if item.respond_to? :to_sql
+        _quote(item)
+      end
+    end
+
+    def column_names(item = @quotable)
+      if item.respond_to?(:column_names)
+        item = item.column_names
+      elsif item.class.respond_to?(:column_names)
+        item = item.class.column_names
+      elsif item.is_a?(Array) and item[0].respond_to?(:name)
+        item = item.map(&:name)
+      end
+      @qsql.column_names ||= item
+      ident_name(item)
+    end
+
+    def ident_name(item = @quotable)
+      case item
+      when Array
+        item.map do |item|
+          case item
+          when Hash
+            ident_name(item)
+          when String, Symbol
+            _quote_column_name(item)
+          when Proc
+            item.call(self)
+          end
+        end.join(",")
+      when Hash
+        item.map do
+          case _2
+          when Symbol
+            _quote_column_name(_1, _2)
+          when String
+            "#{_2} AS #{_1}"
+          when Proc
+            item.call(self)
+          end
+        end.join(",")
+      else
+        _quote_column_name(item)
+      end
+    end
+
+    def table(item = @quotable)
+      @qsql.table_name ||= if item.respond_to?(:table_name)
+                             item = item.table_name
+                           elsif item.class.respond_to?(:table_name)
+                             item = item.class.table_name
+                           end
+      table_name(item || @qsql.table_name)
+    end
+
+    def table_name(item = @quotable)
+      case item
+      when Array
+        item.map do |item|
+          item.is_a?(Hash) ? table_name(item) : _quote_column_name(item)
+        end.join(",")
+      when Hash
+        raise NotImplementedError, "table name is a Hash"
+        # perhaps as ...
+      else
+        _quote_column_name(item)
+      end
+    end
+  end
+end

data/lib/quote_sql/quoting.rb

@@ -0,0 +1,48 @@
+class QuoteSql
+  module Quoting
+    def escape(item)
+      case item
+      when Regexp
+        escape_regex(item)
+      when Array
+        escape_array(item)
+      else
+        quote(item)
+      end
+    end
+
+    def escape_array(ary)
+      type = nil
+      dive = ->(ary) do
+        ary.flat_map do |elem|
+          if elem.is_a? Array
+            dive[s]
+          elsif !elem.nil? and (type ||= elem.class.to_s) != elem.class.to_s
+            raise TypeError, "Array elements have to be the same kind"
+          else
+            quote elem
+          end
+        end.join(',')
+      end
+      ary = "[#{dive[ary]}]"
+      "ARRAY#{ary}"
+    end
+
+    # quote ruby regex with a postgres regex
+    # @argument regexp [Regex]
+    # @return String
+    def escape_regex(regexp)
+      # https://gist.github.com/glv/24bedd7d39f16a762528d7b30e366aa7
+      pregex = regexp.to_s.gsub(/^\(\?-?[mix]+:|\)$/, '')
+      if pregex[/[*+?}]\+|\(\?<|&&|\\k|\\g|\\p\{/]
+        raise RegexpError, "cant convert Regexp #{sub}"
+      end
+      pregex.gsub!(/\\h/, "[[:xdigit:]]")
+      pregex.gsub!(/\\H/, "[^[:xdigit:]]")
+      pregex.gsub!(/\?[^>]>/, '')
+      pregex.gsub!(/\{,/, "{0,")
+      pregex.gsub!(/\\z/, "\\Z")
+      quote(pregex)
+    end
+  end
+end

data/lib/quote_sql/test.rb

@@ -0,0 +1,112 @@
+module QuoteSql::Test
+  def self.all
+    methods(false).grep(/^test_/).each do |name|
+      run(name)
+      puts
+    end
+
+  end
+
+  def self.run(name)
+    name = name.to_s.sub(/^test_/, "")
+    @expected = nil
+    @test = send("test_#{name}")
+    if sql.gsub(/\s+/, "") == expected&.gsub(/\s+/, "")
+      STDOUT.puts name, @test.original, @test.quotes.inspect, "✅ #{expected}"
+    else
+      STDOUT.puts name, @test.inspect, sql, "❌ #{expected}"
+    end
+  end
+
+  def self.expected(v = nil)
+    @expected ||= v
+  end
+
+  def self.sql
+    @test.to_sql
+  end
+
+  class PseudoActiveRecord
+    def self.table_name
+      "pseudo_active_records"
+    end
+
+    def self.column_names
+      %w(id column1 column2)
+    end
+
+    def to_qsl
+      "SELECT * FROM #{self.class.table_name}"
+    end
+  end
+
+  class << self
+    def test_columns_and_table_name_simple
+      expected Arel.sql(%(SELECT "a","b"."c" FROM "my_table"))
+      QuoteSql.new("SELECT %columns FROM %table_name").quote(
+        columns: [:a, b: :c],
+        table_name: "my_table"
+      )
+    end
+
+    def test_columns_and_table_name_complex
+      expected Arel.sql(%(SELECT "a","b"."c" FROM "table1","table2"))
+      QuoteSql.new("SELECT %columns FROM %table_names").quote(
+        columns: [:a, b: :c],
+        table_names: ["table1", "table2"]
+      )
+    end
+
+    def test_recursive_injects
+      expected Arel.sql(%(SELECT TRUE FROM "table1"))
+      QuoteSql.new("SELECT %raw FROM %table_names").quote(
+        raw: "%recurse1_raw",
+        recurse1_raw: "%recurse2",
+        recurse2: true,
+        table_names: "table1"
+      )
+    end
+
+    def test_values
+      expected Arel.sql(%(SELECT 'a text', 123, 'text' AS abc FROM "my_table"))
+      QuoteSql.new("SELECT %text, %{number}, %aliased_with_hash FROM %table_name").quote(
+        text: "a text",
+        number: 123,
+        aliased_with_hash: {
+          abc: "text"
+        },
+        table_name: "my_table"
+      )
+    end
+
+    def test_binds
+      expected Arel.sql(%(SELECT $1, $2, $1 AS get_bind_1_again FROM "my_table"))
+      QuoteSql.new("SELECT %bind, %bind__uuid, %bind1 AS get_bind_1_again FROM %table_name").quote(
+        table_name: "my_table"
+      )
+    end
+
+    def test_q3
+      expected Arel.sql(<<-SQL)
+            INSERT INTO "responses" ("id","type","task_id","index","data","parts","value","created_at","updated_at") 
+            VALUES (NULL,TRUE,'A','[5,5]','{"a":1}'),
+                   (1,FALSE,'B','[]','{"a":2}'),
+                   (2,NULL,'c','[1,2,3]','{"a":3}')
+            ON CONFLICT (responses_task_id_index_unique) DO NOTHING;
+        SQL
+
+      QuoteSql.new(<<-SQL).
+          INSERT INTO %table (%columns) VALUES %values
+            ON CONFLICT (responses_task_id_index_unique) DO NOTHING;
+        SQL
+        quote(
+          table: Response,
+          values: [
+            [nil, true, "A", [5, 5], { a: 1 }],
+            [1, false, "B", [], { a: 2 }],
+            [2, nil, "c", [1, 2, 3], { a: 3 }]
+          ]
+        )
+    end
+  end
+end

data/lib/quote_sql.rb

@@ -0,0 +1,306 @@
+Dir.glob(__FILE__.sub(/\.rb$/, "/*.rb")).each { require(_1) unless _1[/test\.rb$/] }
+
+# Tool to build and run SQL queries easier
+#
+#   QuoteSql.new("SELECT %field").quote(field: "abc").to_sql
+#   => SELECT 'abc'
+#
+#   QuoteSql.new("SELECT %field__text").quote(field__text: 9).to_sql
+#   => SELECT 9::TEXT
+#
+#   QuoteSql.new("SELECT %columns FROM %table_name").quote(table: User).to_sql
+#   => SELECT "id",firstname","lastname",... FROM "users"
+#
+#   QuoteSql.new("SELECT a,b,%raw FROM table").quote(raw: "jsonb_build_object('a', 1)").to_sql
+#   => SELECT "a,b,jsonb_build_object('a', 1) FROM table
+#
+#   QuoteSql.new("SELECT %column_names FROM (%any_name) a").
+#     quote(any_name: User.select("%column_names").where(id: 3), column_names: [:firstname, :lastname]).to_sql
+#   => SELECT firstname, lastname FROM (SELECT firstname, lastname FROM users where id = 3)
+#
+#   QuoteSql.new("INSERT INTO %table (%columns) VALUES %values ON CONFLICT (%constraint) DO NOTHING").
+#     quote(table: User, values: [
+#       {firstname: "Albert", id: 1, lastname: "Müller"},
+#       {lastname: "Schultz", firstname: "herbert"}
+#     ], constraint: :id).to_sql
+#   => INSERT INTO "users" ("id", "firstname", "lastname", "created_at")
+#       VALUES (1, 'Albert', 'Müller', CURRENT_TIMESTAMP), (DEFAULT, 'herbert', 'Schultz', CURRENT_TIMESTAMP)
+#       ON CONFLICT ("id") DO NOTHING
+#
+#   QuoteSql.new("SELECT %columns").quote(columns: [:a, :"b.c", c: "jsonb_build_object('d', 1)"]).to_sql
+#   => SELECT "a","b"."c",jsonb_build_object('d', 1) AS c
+#
+# Substitution
+#   In the SQL matches of %foo or %{foo} or %foo_4_bar or %{foo_4_bar} the *"mixins"*
+#   are substituted with quoted values
+#   the values are looked up from the options given in the quotes method
+#   the mixins can be recursive, Caution! You need to take care, you can create infintive loops!
+#
+# Special mixins are
+# - %table | %table_name | %table_names
+# - %column | %columns | %column_names
+# - %ident | %constraint | %constraints quoting for database columns
+# - %raw | %sql inserting raw SQL
+# - %value | %values creates value section for e.g. insert
+#   - In the right order
+#     - Single value => (2)
+#     - +Array+ => (column, column, column) n.b. has to be the correct order
+#     - +Array+ of +Array+ => (...),(...),(...),...
+#   - if the columns option is given (or implicitely by setting table)
+#     - +Hash+ values are ordered according to the columns option, missing values are replaced by DEFAULT
+#     - +Array+ of +Hash+ multiple record insert
+# - %bind is replaced with the current bind sequence.
+#   Without appended number the first %bind => $1, the second => $2 etc.
+#   - %bind\\d+ => $+Integer+ e.g. %bind7 => $7
+#   - %bind__text => $1 and it is registered as text - this is used in prepared statements TODO
+#   - %key_bind__text => $1 and it is registered as text when using +Hash+ in the execute
+#     $1 will be mapped to the key's value in the +Hash+ TODO
+#
+# All can be preceded by additional letters and underscore e.g. %foo_bar_column
+#
+# A database typecast is added to fields ending with double underscore and a valid db data type
+# with optional array dimension
+#
+# - %field__jsonb => adds a ::JSONB typecast to the field
+# - %number_to__text => adds a ::TEXT typecast to the field
+# - %array__text1 => adds a ::TEXT[] TODO
+# - %array__text2 => adds a ::TEXT[][] TODO
+#
+# Quoting
+# - Any value of the standard mixins are quoted with these exceptions
+# - +Array+ are quoted as DB Arrays unless the type cast e.g. __jsonb is given
+# - +Hash+ are quoted as jsonb
+# - When the value responds to :to_sql or is a +Arel::Nodes::SqlLiteral+ its added as raw SQL
+# - +Proc+ are executed with the +QuoteSQL::Quoter+ object as parameter and added as raw SQL
+#
+# Special quoting columns
+# - +String+ or +Symbol+ without a dot  e.g. :firstname => "firstname"
+# - +String+ or +Symbol+ containing a dot e.g. "users.firstname" or => "users"."firstname"
+# - +Array+
+#   - +String+ and +Symbols+ see above
+#   - +Hash+ see below
+# - +Hash+ or within the +Array+
+#   - +Symbol+ value will become the column name e.g. {table: :column} => "table"."column"
+#   - +String+ value will become the expression, the key the AS {result: "SUM(*)"} => SUM(*) AS result
+#   - +Proc+ are executed with the +QuoteSQL::Quoter+ object as parameter and added as raw SQL
+#
+class QuoteSql
+  DATA_TYPES_RE = %w(
+(?:small|big)(?:int|serial)
+bit bool(?:ean)? box bytea cidr circle date
+(?:date|int[48]|num|ts(?:tz)?)(?:multi)?range
+macaddr8?
+jsonb?
+ts(?:query|vector)
+float[48] (?:int|serial)[248]?
+double_precision  inet
+integer  line lseg   money   path pg_lsn
+pg_snapshot point polygon real  text timestamptz timetz
+txid_snapshot uuid xml
+(bit_varying|varbit|character|char|character varying|varchar)(_\\(\\d+\\))?
+(numeric|decimal)(_\\(\d+_\d+\\))?
+interval(_(YEAR|MONTH|DAY|HOUR|MINUTE|SECOND|YEAR_TO_MONTH|DAY_TO_HOUR|DAY_TO_MINUTE|DAY_TO_SECOND|HOUR_TO_MINUTE|HOUR_TO_SECOND|MINUTE_TO_SECOND))?(_\\(\d+\\))?
+time(stamp)?(_\\(\d+\\))?(_with(out)?_time_zone)?
+    ).join("|")
+
+  CASTS = Regexp.new("__(#{DATA_TYPES_RE})$", "i")
+
+  def self.conn
+    raise ArgumentError, "You need to define a database connection function"
+  end
+
+  def self.db_connector=(conn)
+    Connector.set(conn)
+  end
+
+  def initialize(sql = nil)
+    @original = sql.respond_to?(:to_sql) ? sql.to_sql : sql.to_s
+    @sql = @original.dup
+    @quotes = {}
+    @resolved = {}
+    @binds = []
+  end
+
+  attr_reader :sql, :quotes, :original, :binds
+  attr_writer :table_name, :column_names
+
+  def table_name
+    return @table_name if @table_name
+    return unless table = @quote&.dig(:table)
+    @table_name = table.respond_to?(:table_name) ? table.table_name : table.to_s
+  end
+
+  def column_names
+    return @column_names if @column_names
+    return unless columns = @quote&.dig(:columns)
+    @column_names = if columns[0].is_a? String
+      columns
+    else
+      columns.map(&:name)
+    end.map(&:to_s)
+  end
+
+  # Add quotes keys are symbolized
+  def quote(quotes1 = {}, **quotes2)
+    quotes = @quotes.merge(quotes1, quotes2).transform_keys(&:to_sym)
+    if table = quotes.delete(:table)
+      columns = quotes.delete(:columns) || table.columns
+    end
+    @quotes = { table:, columns:, **quotes }
+    self
+  end
+
+  class Error < ::RuntimeError
+    def initialize(quote_sql)
+      @object = quote_sql
+    end
+
+    def message
+      super + %Q@<QuoteSql #{@object.original.inspect} #{@object.errors.inspect}>@
+    end
+  end
+
+  def to_sql
+    mixin!
+    raise Error.new(self) if errors?
+    return Arel.sql @sql if defined? Arel
+    @sql
+  end
+
+  def result(binds = [], prepare: false, async: false)
+    sql = to_sql
+    if binds.present? and sql.scan(/(?<=\$)\d+/).map(&:to_i).max + 1 != binds.length
+      raise ArgumentError, "Wrong number of binds"
+    end
+    _exec(sql, binds, prepare: false, async: false)
+  rescue => exc
+    STDERR.puts exc.sql
+    raise exc
+  end
+
+  alias exec result
+
+  def prepare(name)
+    sql = to_sql
+    raise ArguemntError, "binds not all casted e.g. %bind__CAST" if @binds.reject.any?
+    name = quote_column_name(name)
+    _exec_query("PREPARE #{name} (#{@binds.join(',')}) AS #{sql}")
+    @prepare_name = name
+  end
+
+
+  # Executes a prepared statement
+  # Processes in batches records
+  # returns the array of the results depending on RETURNING is in the query
+  #
+  #   execute([1, "a", true, nil], ...)
+  #
+  #   execute({ id: 1, text: "a", bool: true, know: nil}, ...)
+  #
+  #   execute([1, "a", true, nil], ... batch: 500)
+  #   # set the batch size of 500
+  #
+  #   execute([1, "a", true, nil], ... batch: falss)
+  #   # processes all at once
+  def execute(*records, batch: 1000)
+    sql = "EXECUTE #{@prepare_name}(#{(1..@binds.length).map { "$#{_1}" }.join(",")})"
+    records.map! do |record|
+      if record.is_a?(Hash)
+        raise NotImplementedError, "record hash not yet implemented"
+      else
+        record = Array(record)
+      end
+      if @binds.length != record.length
+        next RuntimeError.new("binds are not equal arguments, #{record.inspect}")
+      end
+      _exec(sql, record, prepare: false, async: false)
+    end
+  end
+
+  def reset
+    @sql = @original
+  end
+
+  def errors
+    @quotes.to_h do |k, v|
+      r = @resolved[k]
+      next [nil, nil] unless r.nil? or r.is_a?(Exception)
+      [k, "#{@quotes[k].inspect} => #{v.inspect}"]
+    end.compact
+  end
+
+  def errors?
+    @resolved.any? { _2.is_a? Exception }
+  end
+
+  MIXIN_RE = /(%\{?([a-z][a-z0-9_]*)}|%([a-z][a-z0-9_]*)\b)/im
+
+  def key_matches
+    @sql.scan(MIXIN_RE).map do |full, *key|
+      key = key.compact[0]
+      [full, key, @quotes.key?(key.to_sym)]
+    end
+  end
+
+  def mixin!
+    unresolved = Set.new(key_matches.map(&:second))
+    last_unresolved = Set.new
+    loop do
+      s = StringScanner.new(@sql)
+      sql = ""
+      key_matches.each do |key_match, key, has_quote|
+        s.scan_until(/(.*?)#{key_match}([a-z0-9_]*)/im)
+        matched, pre, post = s.matched, s[1], s[2]
+        if m = key.match(/^bind(\d+)?(?:#{CASTS})?$/im)
+          if m[2].present?
+            cast = m[2].tr("_", " ")
+          end
+          if m[1].present?
+            bind_num = m[1].to_i
+            @binds[bind_num - 1] ||= cast
+            raise "cast #{bind_num} already set to #{@binds[bind_num - 1]}" unless @binds[bind_num - 1] == cast
+          else
+            @binds << cast
+            bind_num = @binds.length
+          end
+          matched = "#{pre}$#{bind_num}#{post}"
+        elsif has_quote
+          quoted = quoter(key)
+          unresolved.delete key
+          if (i = quoted.scan MIXIN_RE).present?
+            unresolved += i.map(&:last)
+          end
+          matched = "#{pre}#{quoted}#{post}"
+        end
+      rescue TypeError
+      ensure
+        sql << matched.to_s
+      end
+      @sql = sql + s.rest
+      break if unresolved.empty?
+      break if unresolved == last_unresolved
+      last_unresolved = unresolved.dup
+    end
+    self
+  end
+
+  def quoter(key)
+    quoter = @resolved[key.to_sym] = Quoter.new(self, key, @quotes[key.to_sym])
+    quoter.to_sql
+  rescue TypeError => exc
+    @resolved[key.to_sym] = exc
+    raise exc
+  end
+
+  extend Quoting
+
+end
+
+QuoteSql.include QuoteSql::Formater
+
+class Array
+  def depth
+    select { _1.is_a?(Array) }.map { _1.depth.to_i + 1 }.max || 1
+  end
+end
+

metadata

@@ -0,0 +1,55 @@
+--- !ruby/object:Gem::Specification
+name: quote-sql
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Martin Kufner
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2024-02-23 00:00:00.000000000 Z
+dependencies: []
+description: 'QuoteSql helps you creating SQL queries and proper quoting especially
+  with advanced queries.
+
+  '
+email: martin.kufner@quiz.baby
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/quote_sql.rb
+- lib/quote_sql/connector.rb
+- lib/quote_sql/connector/active_record_base.rb
+- lib/quote_sql/deprecated.rb
+- lib/quote_sql/extension.rb
+- lib/quote_sql/formater.rb
+- lib/quote_sql/quoter.rb
+- lib/quote_sql/quoting.rb
+- lib/quote_sql/test.rb
+homepage: https://github.com/martin-kufner/quote-sql
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.4.19
+signing_key:
+specification_version: 4
+summary: Tool to build and run SQL queries easier
+test_files: []