activerecord-cockroachdb-adapter 0.2.3 → 6.0.0beta1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 96a284c08e5798cc8b45332be543b14f60bdb8c0
-  data.tar.gz: ffa757e8b581788ba8d1880842d37eca7058695e
+SHA256:
+  metadata.gz: a580dde40d074b1982c6642310513d79bbcb710032a476a64bb9fcb99bec72bb
+  data.tar.gz: 91b11b334235484e298704ae514d435a61451a02f3f759881141fd6354549ee4
 SHA512:
-  metadata.gz: 456d42bc6a8a973488a6c66a9631d8e78f7e4a35abb379b3958239b6fb474bce9a3503678be6a098d788bb09425c02d30732fb1ee546b3d4c2b8cb21b3a7fe57
-  data.tar.gz: 40ada5999c613d6d2f953ce7fc99b9fbb3793685d258fe9cef790e2421df96d218ee3684081a499571ed38cc129be6e65de6a87328d57297fdd62323430c61c6
+  metadata.gz: 77b98dff7d2069ba0cdcd78bccf5aedf9ac31192c0813d7893a5a6f1084537d9d0b03039be58fe83a5a0e0cd58d5d42ba567e43beee62507d8212242722fdb13
+  data.tar.gz: f1bf6e2fa1925d94133a98dcd03690a24da2631134cc6a5eb6dd860253392f398db88572942f409f13dec4d3d7aeb34f18317a5b8cd4b30c70e4d75d13599b47

data/.gitignore

@@ -7,3 +7,4 @@
 /pkg/
 /spec/reports/
 /tmp/
+*.gem

data/.gitmodules

@@ -1,3 +0,0 @@
-[submodule "rails"]
-	path = rails
-	url = https://github.com/lego/ruby-on-rails.git

data/CONTRIBUTING.md

@@ -11,15 +11,19 @@ CockroachDBAdapter to initialize ActiveRecord for their projects.
 This adapter extends the PostgreSQL ActiveRecord adapter in order to
 override and monkey-patch functionality.
 
-The other repository is a fork of [Rails]. The tests have been modified
-for the purposes of testing our CockroachDB adapter.
-
 [activerecord-cockroachdb-adapter]: https://github.com/cockroachdb/activerecord-cockroachdb-adapter/
-[Rails]: https://github.com/lego/ruby-on-rails
-
 
 ## Setup and running tests
 
+In CockroachDB, create two databases to be used by the ActiveRecord test suite:
+activerecord_unittest and activerecord_unittest2.
+
+```sql
+CREATE DATABASE activerecord_unittest;
+
+CREATE DATABASE activerecord_unittest2;
+```
+
 It is best to have a Ruby environment manager installed, such as
 [rvm](https://rvm.io/), as Rails has varying Ruby version requirements.
 If you are using rvm, you then install and use the required Ruby
@@ -37,65 +41,42 @@ rvm use 2.2.5
 ```
 
 Using [bundler](http://bundler.io/), install the dependancies of Rails.
-Additionally, make sure the Rails git submodule is loaded.
 
 ```bash
-# Ensure the rails fork is fetched.
-git submodule update
-# Install rails dependancies.
-(cd rails && bundle install)
+bundle install
 ```
 
-Then, to run the test with an active CockroachDB instance:
+Then, to run the full test suite with an active CockroachDB instance:
 
 ```bash
-cp build/config.teamcity.yml rails/activerecord/test/config.yml
-(cd rails/activerecord && BUNDLE_GEMFILE=../Gemfile bundle exec rake db:cockroachdb:rebuild)
-(cd rails/activerecord && BUNDLE_GEMFILE=../Gemfile bundle exec rake test:cockroachdb)
+bundle exec rake test
 ```
 
-### Test commands in detail
+To run specific ActiveRecord tests, set environemnt variable `TEST_FILES_AR`. For example, to run ActiveRecord tests `test/cases/associations_test.rb` and `test/cases/ar_schema_test.rb.rb`
 
 ```bash
-cp build/config.teamcity.yml rails/activerecord/test/config.yml
+TEST_FILES_AR="test/cases/associations_test.rb,test/cases/ar_schema_test.rb" bundle exec rake test
 ```
 
-This copies the TeamCity ActiveRecord configuration for the application.
-This configuration specifies:
-
-- CockroachDB port and host the test suite uses.
-- Database names used for the different test connections. (ActiveRecord
-  uses two separate connections for some tests.)
+To run specific CockroachDB Adapter tests, set environemnt variable `TEST_FILES`. For example, to run CockroachDB Adpater tests `test/cases/adapter_test.rb` and `test/cases/associations/left_outer_join_association_test.rb`
 
-```
-(cd rails/activerecord && BUNDLE_GEMFILE=../Gemfile bundle exec rake db:cockroachdb:rebuild)
+```bash
+TEST_FILES="test/cases/adapter_test.rb,test/cases/associations/left_outer_join_association_test.rb" bundle exec rake test
 ```
 
-This prepares CockroachDB for running tests. It only drops and
-re-creates all of the databases needed.
+To run a specific test case, use minitest's `-n` option to run tests that match a given pattern. All minitest options are set via the `TESTOPTS` environemnt variable. For example, to run `test_indexes` from CockroachDB's `test/cases/adapter_test.rb` file
 
-- This command needs to be run from activerecord folder in order to use
-  the ActiveRecord `Rakefile`. The `Rakefile` defines scripts (called
-  tasks) such as executing tests.
-- `BUNDLE_GEMFILE=../Gemfile` tells `bundle` to use the dependancies for
-  Rails that were previously installed.
-- `bundle exec rake` uses `bundle` to execute the Ruby package `rake`.
-- `rake db:cockroachdb:rebuild` runs the specified Rake task. All tasks
-  can be found in `rails/activerecord/Rakefile`.
-
-
-```
-(cd rails/activerecord && BUNDLE_GEMFILE=../Gemfile bundle exec rake test:cockroachdb)
+```bash
+TEST_FILES="test/cases/adapter_test.rb" TESTOPTS=`-n=/test_indexes/` bundle exec rake test
 ```
 
-This executes the CockroachDB tests.
+By default, tests will be run from the bundled version of Rails. To run against a local copy, set environemnt variable `RAILS_SOURCE`. Running against a local copy of Rails can be helpful when try to debug issues.
 
-- Like the previous command, this one uses the Activerecord Rakefile and
-  the Rails Gemfile. The task code can be found in the Rakefile.
-- Running specific test files can be done by appending
-  `TESTFILES=test/cases/attribute_methods.rb` to the command. Globs are
-  used. Multiple individual files cannot be specified.
+```bash
+RAILS_SOURCE="path/to/local_copy" bundle exec rake test
+```
 
+`test/config.yml` assumes CockroachDB will be running at localhost:26257 with a root user. Make changes to `test/config.yml` as needed.
 
 # Improvements
 
@@ -143,15 +124,6 @@ A possible way to approach this would be to add a shim to cause any
 tests that use it to fail, and grep the tests that pass and then skip
 them.
 
-# Cleanup
-
-One of the earlier commits to the Rails repo did a big grep of
-`PostgreSQLAdapter` -> `CockroachDBAdapter`. In order to better support
-changes upstream, this modification should be changed to instead only
-add `CockroachDBAdapter` alongside any `PostgreSQLAdapter`. The later
-test cleanup commit will conflict on any further changes (like adding
-back PostgreSQL, or removing CockroachDB for PostgreSQL).
-
 ## Publishing to Rubygems
 
 TODO: Expand on this. Jordan is likely the only person with publishing

data/Gemfile

@@ -1,11 +1,63 @@
+require 'openssl'
 source 'https://rubygems.org'
+gemspec
 
-# gem 'activerecord', git: 'https://github.com/lego/ruby-on-rails.git'
+if ENV['RAILS_SOURCE']
+  gemspec path: ENV['RAILS_SOURCE']
+else
+  def get_version_from_gemspec
+    gemspec = eval(File.read('activerecord-cockroachdb-adapter.gemspec'))
 
-git_source(:github) { |repo| "https://github.com/#{repo}.git" }
+    gem_version = gemspec.dependencies.
+      find { |dep| dep.name == 'activerecord' }.
+      requirement.
+      requirements.
+      first.
+      last
 
-# Specify your gem's dependencies in activerecord-cockroachdb-adapter.gemspec
-gemspec
+    major, minor, tiny, pre = gem_version.segments
+
+    if pre
+      gem_version.to_s
+    else
+      find_latest_matching_version(major, minor)
+    end
+  end
+
+  def find_latest_matching_version(gemspec_major, gemspec_minor)
+    all_activerecord_versions.
+      reject { |version| version["prerelease"] }.
+      map { |version| version["number"].split(".").map(&:to_i) }.
+      find { |major, minor|
+        major == gemspec_major && (minor == gemspec_minor || gemspec_minor.nil?)
+      }.join(".")
+  end
+
+  def all_activerecord_versions
+    require 'net/http'
+    require 'yaml'
+
+    uri = URI.parse "https://rubygems.org/api/v1/versions/activerecord.yaml"
+    http = Net::HTTP.new(uri.host, uri.port)
+    http.use_ssl = true
+    http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+
+    YAML.load(
+      http.request(Net::HTTP::Get.new(uri.request_uri)).body
+    )
+  end
+
+  # Get Rails from source beacause the gem doesn't include tests
+  version = ENV['RAILS_VERSION'] || get_version_from_gemspec
+  gem 'rails', git: "https://github.com/rails/rails.git", tag: "v#{version}"
+end
+
+group :development do
+  gem "byebug"
+  gem "minitest-excludes"
 
-# We need a newish Rake since Active Job sets its test tasks' descriptions.
-gem "rake", ">= 11.1"
+  # Gems used by the ActiveRecord test suite
+  gem "bcrypt"
+  gem "mocha"
+  gem "sqlite3"
+end

data/README.md

@@ -7,7 +7,7 @@ CockroachDB adapter for ActiveRecord 4 and 5. This is a lightweight extension of
 Add this line to your project's Gemfile:
 
 ```ruby
-gem 'activerecord-cockroachdb-adapter', '~> 0.2.2'
+gem 'activerecord-cockroachdb-adapter', '~> 5.2.0'
 ```
 
 If you're using Rails 4.x, use the `0.1.x` versions of this gem.
@@ -26,4 +26,4 @@ development:
 ## Modifying the adapter?
 
 See [CONTRIBUTING.md](/CONTRIBUTING.md) for more details on setting up
-the environment and making modifications.
+the environment and making modifications.

data/Rakefile

@@ -1,10 +1,22 @@
 require "bundler/gem_tasks"
 require "rake/testtask"
+require_relative 'test/support/paths_cockroachdb'
+require_relative 'test/support/rake_helpers'
 
-Rake::TestTask.new(:test) do |t|
-  t.libs << "test"
-  t.libs << "lib"
-  t.test_files = FileList['test/**/*_test.rb']
+task test: ["test:cockroachdb"]
+task default: [:test]
+
+namespace :test do
+  Rake::TestTask.new("cockroachdb") do |t|
+    t.libs = ARTest::CockroachDB.test_load_paths
+    t.test_files = test_files
+    t.warning = !!ENV["WARNING"]
+    t.verbose = false
+  end
+
+  task "cockroachdb:env" do
+    ENV["ARCONN"] = "cockroachdb"
+  end
 end
 
-task :default => :test
+task 'test:cockroachdb' => 'test:cockroachdb:env'

data/activerecord-cockroachdb-adapter.gemspec

@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name          = "activerecord-cockroachdb-adapter"
-  spec.version       = "0.2.3"
+  spec.version       = "6.0.0beta1"
   spec.licenses      = ["Apache-2.0"]
   spec.authors       = ["Cockroach Labs"]
   spec.email         = ["cockroach-db@googlegroups.com"]
@@ -13,8 +13,8 @@ Gem::Specification.new do |spec|
   spec.description   = "Allows the use of CockroachDB as a backend for ActiveRecord and Rails apps."
   spec.homepage      = "https://github.com/cockroachdb/activerecord-cockroachdb-adapter"
 
-  spec.add_dependency "activerecord", "~> 5.2"
-  spec.add_dependency "pg", ">= 0.20", "< 0.22"
+  spec.add_dependency "activerecord", "~> 6.0.3"
+  spec.add_dependency "pg", ">= 0.20"
 
   # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
   # to allow pushing to a single host or delete this section to allow pushing to any host.
@@ -31,8 +31,4 @@ Gem::Specification.new do |spec|
   spec.bindir        = "exe"
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
-
-  spec.add_development_dependency "bundler", "~> 1.14"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "minitest", "~> 5.0"
 end

data/build/Dockerfile

@@ -1,7 +1,7 @@
 # This Dockerfile extends the Examples-ORM testing image in order to
 # install specific dependencies required for ActiveRecord tests.
 
-FROM cockroachdb/postgres-test:20170308-1644
+FROM cockroachdb/example-orms-builder:20200413-1918
 
 # Native dependencies for libxml-ruby and sqlite3.
 RUN apt-get update -y && apt-get install -y \

data/build/teamcity-test.sh

@@ -1,16 +1,15 @@
 #!/usr/bin/env bash
 
-set -euo pipefail
+set -euox pipefail
 
-# Download CockroachDB. NB: currently this uses an alpha, due to feature
-# requirements.
-VERSION=v2.0-alpha.20171218
+# Download CockroachDB
+VERSION=v20.2.1
 wget -qO- https://binaries.cockroachdb.com/cockroach-$VERSION.linux-amd64.tgz | tar  xvz
 readonly COCKROACH=./cockroach-$VERSION.linux-amd64/cockroach
 
 # Make sure cockroach can be found on the path. This is required for the
 # ActiveRecord Rakefile that rebuilds the test database.
-export PATH=$(pwd)/cockroach-$VERSION.linux-amd64/:$PATH
+export PATH=./cockroach-$VERSION.linux-amd64/:$PATH
 readonly urlfile=cockroach-url
 
 run_cockroach() {
@@ -22,7 +21,7 @@ run_cockroach() {
   cockroach quit --insecure || true
   rm -rf cockroach-data
   # Start CockroachDB.
-  cockroach start --insecure --host=localhost --listening-url-file="$urlfile" >/dev/null 2>&1 &
+  cockroach start-single-node --max-sql-memory=25% --cache=25% --insecure --host=localhost --listening-url-file="$urlfile" >/dev/null 2>&1 &
   # Ensure CockroachDB is stopped on script exit.
   trap "echo 'Exit routine: Killing CockroachDB.' && kill -9 $! &> /dev/null" EXIT
   # Wait until CockroachDB has started.
@@ -32,45 +31,26 @@ run_cockroach() {
     echo "server not yet available; sleeping for $backoff seconds"
     sleep $backoff
   done
+  cockroach sql --insecure -e 'CREATE DATABASE activerecord_unittest;'
+  cockroach sql --insecure -e 'CREATE DATABASE activerecord_unittest2;'
+  cockroach sql --insecure -e 'SET CLUSTER SETTING sql.stats.automatic_collection.enabled = false;'
+  cockroach sql --insecure -e 'SET CLUSTER SETTING sql.stats.histogram_collection.enabled = false;'
+  cockroach sql --insecure -e "SET CLUSTER SETTING jobs.retention_time = '180s';"
 }
 
-# Target the Rails dependency file.
-export BUNDLE_GEMFILE=$(pwd)/rails/Gemfile
-
 # Install ruby dependencies.
+gem install bundler:2.1.4
 bundle install
 
-cp build/config.teamcity.yml rails/activerecord/test/config.yml
-
-# 'Install' our adapter. This involves symlinking it inside of
-# ActiveRecord. Normally the adapter will transitively install
-# ActiveRecord, but we need to execute tests from inside the Rails
-# context so we cannot rely on that. We also need previous links to make
-# tests idempotent.
-rm -f rails/activerecord/lib/active_record/connection_adapters/cockroachdb_adapter.rb
-ln -s $(pwd)/lib/active_record/connection_adapters/cockroachdb_adapter.rb rails/activerecord/lib/active_record/connection_adapters/cockroachdb_adapter.rb
-rm -rf rails/activerecord/lib/active_record/connection_adapters/cockroachdb
-ln -s $(pwd)/lib/active_record/connection_adapters/cockroachdb rails/activerecord/lib/active_record/connection_adapters/cockroachdb
+run_cockroach
 
-# Get the test files with "# FILE(OK)". These should pass.
-TESTS=$(cd rails/activerecord && find test/cases -type f \( -name "*_test.rb" \) -exec grep -l "# FILE(OK)" {} +)
-
-for TESTFILE in ${TESTS}; do
-  # Start CockroachDB
-  run_cockroach
-  # Run the tests.
-  echo "Rebuilding database"
-  (cd rails/activerecord && bundle exec rake db:cockroachdb:rebuild)
-  echo "Running test: $TESTFILE"
-  # Run the test. Continue testing even if this file fails.
-  if ! (cd rails/activerecord && bundle exec rake test:cockroachdb TESTFILES=$TESTFILE); then
-    echo "Test FAILED: $TESTFILE"
+if ! (RUBYOPT="-W0" TESTOPTS="-v" bundle exec rake test); then
+    echo "Tests failed"
     HAS_FAILED=1
-  else
-    echo "Test PASSED: $TESTFILE"
+else
+    echo "Tests passed"
     HAS_FAILED=0
-  fi
-done
+fi
 
 # Attempt a clean shutdown for good measure. We'll force-kill in the
 # exit trap if this script fails.

data/docker.sh

@@ -2,7 +2,7 @@
 #
 # This file is largely cargo-culted from cockroachdb/cockroach/build/builder.sh.
 
-set -euo pipefail
+set -euox pipefail
 
 DOCKER_IMAGE_TAG=activerecord_test_container
 

data/lib/active_record/connection_adapters/cockroachdb/attribute_methods.rb

@@ -0,0 +1,28 @@
+module ActiveRecord
+  module CockroachDB
+    module AttributeMethodsMonkeyPatch
+
+      private
+
+      # Filter out rowid so it doesn't get inserted by ActiveRecord. rowid is a
+      # column added by CockroachDB for tables that don't define primary keys.
+      # CockroachDB will automatically insert rowid values. See
+      # https://www.cockroachlabs.com/docs/v19.2/create-table.html#create-a-table.
+      def attributes_for_create(attribute_names)
+        super.reject { |name| name == ConnectionAdapters::CockroachDBAdapter::DEFAULT_PRIMARY_KEY }
+      end
+
+      # Filter out rowid so it doesn't get updated by ActiveRecord. rowid is a
+      # column added by CockroachDB for tables that don't define primary keys.
+      # CockroachDB will automatically insert rowid values. See
+      # https://www.cockroachlabs.com/docs/v19.2/create-table.html#create-a-table.
+      def attributes_for_update(attribute_names)
+        super.reject { |name| name == ConnectionAdapters::CockroachDBAdapter::DEFAULT_PRIMARY_KEY }
+      end
+    end
+  end
+
+  class Base
+    prepend CockroachDB::AttributeMethodsMonkeyPatch
+  end
+end

data/lib/active_record/connection_adapters/cockroachdb/column.rb

@@ -0,0 +1,15 @@
+module ActiveRecord
+  module ConnectionAdapters
+    module CockroachDB
+      module PostgreSQLColumnMonkeyPatch
+        def serial?
+          default_function == "unique_rowid()"
+        end
+      end
+    end
+
+    class PostgreSQLColumn
+      prepend CockroachDB::PostgreSQLColumnMonkeyPatch
+    end
+  end
+end

data/lib/active_record/connection_adapters/cockroachdb/database_statements.rb

@@ -0,0 +1,102 @@
+module ActiveRecord
+  module ConnectionAdapters
+    module CockroachDB
+      module DatabaseStatements
+        # Since CockroachDB will run all transactions with serializable isolation,
+        # READ UNCOMMITTED, READ COMMITTED, and REPEATABLE READ are all aliases
+        # for SERIALIZABLE. This lets the adapter support all isolation levels,
+        # but READ UNCOMMITTED has been removed from this list because the
+        # ActiveRecord transaction isolation test fails for READ UNCOMMITTED.
+        # See https://www.cockroachlabs.com/docs/v19.2/transactions.html#isolation-levels
+        def transaction_isolation_levels
+          {
+            read_committed:   "READ COMMITTED",
+            repeatable_read:  "REPEATABLE READ",
+            serializable:     "SERIALIZABLE"
+          }
+        end
+
+        # Overridden to avoid using transactions for schema creation.
+        def insert_fixtures_set(fixture_set, tables_to_delete = [])
+          fixture_inserts = build_fixture_statements(fixture_set)
+          table_deletes = tables_to_delete.map { |table| "DELETE FROM #{quote_table_name(table)}" }
+          statements = table_deletes + fixture_inserts
+
+          with_multi_statements do
+            disable_referential_integrity do
+              execute_batch(statements, "Fixtures Load")
+            end
+          end
+        end
+
+        private
+          def execute_batch(statements, name = nil)
+            statements.each do |statement|
+              execute(statement, name)
+            end
+          end
+
+          DEFAULT_INSERT_VALUE = Arel.sql("DEFAULT").freeze
+          private_constant :DEFAULT_INSERT_VALUE
+
+          def default_insert_value(column)
+            DEFAULT_INSERT_VALUE
+          end
+
+          def build_fixture_sql(fixtures, table_name)
+            columns = schema_cache.columns_hash(table_name)
+
+            values_list = fixtures.map do |fixture|
+              fixture = fixture.stringify_keys
+
+              unknown_columns = fixture.keys - columns.keys
+              if unknown_columns.any?
+                raise Fixture::FixtureError, %(table "#{table_name}" has no columns named #{unknown_columns.map(&:inspect).join(', ')}.)
+              end
+
+              columns.map do |name, column|
+                if fixture.key?(name)
+                  type = lookup_cast_type_from_column(column)
+                  with_yaml_fallback(type.serialize(fixture[name]))
+                else
+                  default_insert_value(column)
+                end
+              end
+            end
+
+            table = Arel::Table.new(table_name)
+            manager = Arel::InsertManager.new
+            manager.into(table)
+
+            if values_list.size == 1
+              values = values_list.shift
+              new_values = []
+              columns.each_key.with_index { |column, i|
+                unless values[i].equal?(DEFAULT_INSERT_VALUE)
+                  new_values << values[i]
+                  manager.columns << table[column]
+                end
+              }
+              values_list << new_values
+            else
+              columns.each_key { |column| manager.columns << table[column] }
+            end
+
+            manager.values = manager.create_values_list(values_list)
+            manager.to_sql
+          end
+
+          def build_fixture_statements(fixture_set)
+            fixture_set.map do |table_name, fixtures|
+              next if fixtures.empty?
+              build_fixture_sql(fixtures, table_name)
+            end.compact
+          end
+
+          def with_multi_statements
+            yield
+          end
+      end
+    end
+  end
+end

data/lib/active_record/connection_adapters/cockroachdb/quoting.rb

@@ -0,0 +1,28 @@
+module ActiveRecord
+  module ConnectionAdapters
+    module CockroachDB
+      module Quoting
+        private
+
+        # CockroachDB does not allow inserting integer values into string
+        # columns, but ActiveRecord expects this to work. CockroachDB will
+        # however allow inserting string values into integer columns. It will
+        # try to parse string values and convert them to integers so they can be
+        # inserted in integer columns.
+        #
+        # We take advantage of this behavior here by forcing numeric values to
+        # always be strings. Then, we won't have to make any additional changes
+        # to ActiveRecord to support inserting integer values into string
+        # columns.
+        def _quote(value)
+          case value
+          when Numeric
+            "'#{quote_string(value.to_s)}'"
+          else
+            super
+          end
+        end
+      end
+    end
+  end
+end

data/lib/active_record/connection_adapters/cockroachdb/referential_integrity.rb

@@ -1,51 +1,36 @@
 # frozen_string_literal: true
-# NOTE(joey): This is cradled from connection_adapters/postgresql/referential_integrity.rb
-# It is commonly used for setting up fixtures during tests.
+
+# The PostgresSQL Adapter's ReferentialIntegrity module can disable and
+# re-enable foreign key constraints by disabling all table triggers. Since
+# triggers are not available in CockroachDB, we have to remove foreign keys and
+# re-add them via the ActiveRecord API.
+#
+# This module is commonly used to load test fixture data without having to worry
+# about the order in which that data is loaded.
 module ActiveRecord
   module ConnectionAdapters
     module CockroachDB
-      module ReferentialIntegrity # :nodoc:
-        def disable_referential_integrity # :nodoc:
-          original_exception = nil
-          fkeys = nil
+      module ReferentialIntegrity
+        def disable_referential_integrity
+          foreign_keys = tables.map { |table| foreign_keys(table) }.flatten
 
-          begin
-            transaction do
-              tables.each do |table_name|
-                fkeys = foreign_keys(table_name)
-                fkeys.each do |fkey|
-                  remove_foreign_key table_name, name: fkey.options[:name]
-                end
-              end
-            end
-          rescue ActiveRecord::ActiveRecordError => e
-            original_exception = e
+          foreign_keys.each do |foreign_key|
+            remove_foreign_key(foreign_key.from_table, name: foreign_key.options[:name])
           end
 
-          begin
-            yield
-          rescue ActiveRecord::InvalidForeignKey => e
-            warn <<-WARNING
-WARNING: Rails was not able to disable referential integrity.
-
-Please go to https://github.com/cockroachdb/activerecord-cockroachdb-adapter
-and report this issue.
-
-    cause: #{original_exception.try(:message)}
-
-              WARNING
-            raise e
-          end
+          yield
 
-          begin
-            transaction do
-              if !fkeys.nil?
-                 fkeys.each do |fkey|
-                  add_foreign_key fkey.from_table, fkey.to_table, fkey.options
-                end
+          foreign_keys.each do |foreign_key|
+            begin
+              add_foreign_key(foreign_key.from_table, foreign_key.to_table, foreign_key.options)
+            rescue ActiveRecord::StatementInvalid => error
+              if error.cause.class == PG::DuplicateObject
+                # This error is safe to ignore because the yielded caller
+                # already re-added the foreign key constraint.
+              else
+                raise error
               end
             end
-          rescue ActiveRecord::ActiveRecordError
           end
         end
       end

data/lib/active_record/connection_adapters/cockroachdb/schema_statements.rb

@@ -1,17 +1,70 @@
-require 'active_record/connection_adapters/postgresql/schema_statements'
-
 module ActiveRecord
   module ConnectionAdapters
     module CockroachDB
       module SchemaStatements
         include ActiveRecord::ConnectionAdapters::PostgreSQL::SchemaStatements
-        # NOTE(joey): This was ripped from PostgresSQL::SchemaStatements, with a
-        # slight modification to change setval(string, int, bool) to just
-        # setval(string, int) for CockroachDB compatbility.
-        # See https://github.com/cockroachdb/cockroach/issues/19723
-        #
+
+        def add_index(table_name, column_name, options = {})
+          super
+        rescue ActiveRecord::StatementInvalid => error
+          if debugging? && error.cause.class == PG::FeatureNotSupported
+            warn "#{error}\n\nThis error will be ignored and the index will not be created.\n\n"
+          else
+            raise error
+          end
+        end
+
+        # ActiveRecord allows for tables to exist without primary keys.
+        # Databases like PostgreSQL support this behavior, but CockroachDB does
+        # not. If a table is created without a primary key, CockroachDB will add
+        # a rowid column to serve as its primary key. This breaks a lot of
+        # ActiveRecord's assumptions so we'll treat tables with rowid primary
+        # keys as if they didn't have primary keys at all.
+        # https://www.cockroachlabs.com/docs/v19.2/create-table.html#create-a-table
+        # https://api.rubyonrails.org/v5.2.4/classes/ActiveRecord/ConnectionAdapters/SchemaStatements.html#method-i-create_table
+        def primary_key(table_name)
+          pk = super
+
+          if pk == CockroachDBAdapter::DEFAULT_PRIMARY_KEY
+            nil
+          else
+            pk
+          end
+        end
+
+        # CockroachDB uses unique_rowid() for primary keys, not sequences. It's
+        # possible to force a table to use sequences, but since it's not the
+        # default behavior we'll always return nil for default_sequence_name.
+        def default_sequence_name(table_name, pk = "id")
+          nil
+        end
+
+        # CockroachDB will use INT8 if the SQL type is INTEGER, so we make it use
+        # INT4 explicitly when needed.
+        def type_to_sql(type, limit: nil, precision: nil, scale: nil, array: nil, **) # :nodoc:
+          sql = \
+            case type.to_s
+            when "integer"
+              case limit
+              when nil; "int"
+              when 1, 2; "int2"
+              when 3, 4; "int4"
+              when 5..8; "int8"
+              else super
+              end
+            else
+              super
+            end
+          # The call to super might have appeneded [] already.
+          if array && type != :primary_key && !sql.end_with?("[]")
+            sql = "#{sql}[]"
+          end
+          sql
+        end
+
+        # This overrides the method from PostegreSQL adapter
         # Resets the sequence of a table's primary key to the maximum value.
-        def reset_pk_sequence!(table, pk = nil, sequence = nil) #:nodoc:
+        def reset_pk_sequence!(table, pk = nil, sequence = nil)
           unless pk && sequence
             default_pk, default_sequence = pk_and_sequence_for(table)
 
@@ -27,29 +80,10 @@ module ActiveRecord
             quoted_sequence = quote_table_name(sequence)
             max_pk = query_value("SELECT MAX(#{quote_column_name pk}) FROM #{quote_table_name(table)}", "SCHEMA")
             if max_pk.nil?
-              if postgresql_version >= 100000
-                minvalue = query_value("SELECT seqmin FROM pg_sequence WHERE seqrelid = #{quote(quoted_sequence)}::regclass", "SCHEMA")
-              else
-                minvalue = query_value("SELECT min_value FROM #{quoted_sequence}", "SCHEMA")
-              end
+              minvalue = query_value("SELECT seqmin FROM pg_sequence WHERE seqrelid = #{quote(quoted_sequence)}::regclass", "SCHEMA")
             end
-            if max_pk
-              # NOTE(joey): This is done to replace the call:
-              #
-              #    SELECT setval(..., max_pk, false)
-              #
-              # with
-              #
-              #    SELECT setval(..., max_pk-1)
-              #
-              # These two statements are semantically equivilant, but
-              # setval(string, int, bool) is not supported by CockroachDB.
-              #
-              # FIXME(joey): This is incorrect if the sequence is not 1
-              # incremented. We would need to pull out the custom increment value.
-              max_pk - 1
-            end
-            query_value("SELECT setval(#{quote(quoted_sequence)}, #{max_pk ? max_pk : minvalue})", "SCHEMA")
+
+            query_value("SELECT setval(#{quote(quoted_sequence)}, #{max_pk ? max_pk : minvalue}, #{max_pk ? true : false})", "SCHEMA")
           end
         end
       end

data/lib/active_record/connection_adapters/cockroachdb/transaction_manager.rb

@@ -1,25 +1,23 @@
 # frozen_string_literal: true
 
-require 'active_record/connection_adapters/abstract/transaction'
-
 module ActiveRecord
   module ConnectionAdapters
-
-    # NOTE(joey): This is a very sad monkey patch. Unfortunately, it is
-    # required in order to prevent doing more than 2 nested transactions
-    # while still allowing a single nested transaction. This is because
-    # CockroachDB only supports a single savepoint at the beginning of a
-    # transaction. Allowing this works for the common case of testing.
     module CockroachDB
       module TransactionManagerMonkeyPatch
-        def begin_transaction(options={})
-          @connection.lock.synchronize do
-            # If the transaction nesting is already 2 deep, raise an error.
-            if @connection.adapter_name == "CockroachDB" && @stack.is_a?(ActiveRecord::ConnectionAdapters::SavepointTransaction)
-              raise(ArgumentError, "cannot nest more than 1 transaction at a time. this is a CockroachDB limitation")
-            end
-          end
-          super(options)
+        # Capture ActiveRecord::SerializationFailure errors caused by
+        # transactions that fail due to serialization errors. Failed
+        # transactions will be retried until they pass or the max retry limit is
+        # exceeded.
+        def within_new_transaction(options = {})
+          attempts = options.fetch(:attempts, 0)
+          super
+        rescue ActiveRecord::SerializationFailure => error
+          raise if attempts >= @connection.max_transaction_retries
+
+          attempts += 1
+          sleep_seconds = (2 ** attempts + rand) / 10
+          sleep(sleep_seconds)
+          within_new_transaction(options.merge(attempts: attempts)) { yield }
         end
       end
     end

data/lib/active_record/connection_adapters/cockroachdb/type.rb

@@ -0,0 +1,14 @@
+module ActiveRecord
+  module Type
+    class << self
+      private
+
+      # Return :postgresql instead of :cockroachdb for current_adapter_name so
+      # we can continue using the ActiveRecord::Types defined in
+      # PostgreSQLAdapter.
+      def current_adapter_name
+        :postgresql
+      end
+    end
+  end
+end

data/lib/active_record/connection_adapters/cockroachdb_adapter.rb

@@ -1,8 +1,12 @@
 require 'active_record/connection_adapters/postgresql_adapter'
-require "active_record/connection_adapters/postgresql/schema_statements"
 require "active_record/connection_adapters/cockroachdb/schema_statements"
 require "active_record/connection_adapters/cockroachdb/referential_integrity"
 require "active_record/connection_adapters/cockroachdb/transaction_manager"
+require "active_record/connection_adapters/cockroachdb/column"
+require "active_record/connection_adapters/cockroachdb/database_statements"
+require "active_record/connection_adapters/cockroachdb/quoting"
+require "active_record/connection_adapters/cockroachdb/type"
+require "active_record/connection_adapters/cockroachdb/attribute_methods"
 
 module ActiveRecord
   module ConnectionHandling
@@ -23,7 +27,14 @@ module ActiveRecord
       # The postgres drivers don't allow the creation of an unconnected
       # PG::Connection object, so just pass a nil connection object for the
       # time being.
-      ConnectionAdapters::CockroachDBAdapter.new(nil, logger, conn_params, config)
+      conn = PG.connect(conn_params)
+      ConnectionAdapters::CockroachDBAdapter.new(conn, logger, conn_params, config)
+    rescue ::PG::Error => error
+      if error.message.include?("does not exist")
+        raise ActiveRecord::NoDatabaseError
+      else
+        raise
+      end
     end
   end
 end
@@ -32,16 +43,29 @@ module ActiveRecord
   module ConnectionAdapters
     class CockroachDBAdapter < PostgreSQLAdapter
       ADAPTER_NAME = "CockroachDB".freeze
+      DEFAULT_PRIMARY_KEY = "rowid"
 
       include CockroachDB::SchemaStatements
       include CockroachDB::ReferentialIntegrity
+      include CockroachDB::DatabaseStatements
+      include CockroachDB::Quoting
+
+      def debugging?
+        !!ENV["DEBUG_COCKROACHDB_ADAPTER"]
+      end
 
+      def max_transaction_retries
+        @max_transaction_retries ||= @config.fetch(:max_transaction_retries, 3)
+      end
+
+      # CockroachDB 20.1 can run queries that work against PostgreSQL 10+.
+      def postgresql_version
+        100000
+      end
 
-      # Note that in the migration from ActiveRecord 5.0 to 5.1, the
-      # `extract_schema_qualified_name` method was aliased in the PostgreSQLAdapter.
-      # To ensure backward compatibility with both <5.1 and 5.1, we rename it here
-      # to use the same original `Utils` module.
-      Utils = PostgreSQL::Utils
+      def supports_bulk_alter?
+        false
+      end
 
       def supports_json?
         # FIXME(joey): Add a version check.
@@ -56,22 +80,12 @@ module ActiveRecord
         false
       end
 
-      def supports_ranges?
-        # See cockroachdb/cockroach#17022
-        false
-      end
-
       def supports_materialized_views?
         false
       end
 
-      def supports_pg_crypto_uuid?
-        false
-      end
-
       def supports_partial_index?
-        # See cockroachdb/cockroach#9683
-        false
+        @crdb_version >= 202
       end
 
       def supports_expression_index?
@@ -94,8 +108,7 @@ module ActiveRecord
       end
 
       def supports_advisory_locks?
-        # FIXME(joey): We may want to make this false.
-        true
+        false
       end
 
       def supports_virtual_columns?
@@ -103,114 +116,8 @@ module ActiveRecord
         false
       end
 
-      def supports_savepoints?
-        # See cockroachdb/cockroach#10735.
-        false
-      end
-
-      def transaction_isolation_levels
-        {
-          # Explicitly prevent READ UNCOMMITTED from being used. This
-          # was due to the READ UNCOMMITTED test failing.
-          # read_uncommitted: "READ UNCOMMITTED",
-          read_committed:   "READ COMMITTED",
-          repeatable_read:  "REPEATABLE READ",
-          serializable:     "SERIALIZABLE"
-        }
-      end
-
-
-      # Sadly, we can only do savepoints at the beginning of
-      # transactions. This means that we cannot use them for most cases
-      # of transaction, so we just pretend they're usable.
-      def create_savepoint(name = "COCKROACH_RESTART"); end
-
-      def exec_rollback_to_savepoint(name = "COCKROACH_RESTART"); end
-
-      def release_savepoint(name = "COCKROACH_RESTART"); end
-
-      def indexes(table_name, name = nil) # :nodoc:
-        # The PostgreSQL adapter uses a correlated subquery in the following query,
-        # which CockroachDB does not yet support. That portion of the query fetches
-        # any non-standard opclasses that each index uses. CockroachDB also doesn't
-        # support opclasses at this time, so the query is modified to just remove
-        # the section about opclasses entirely.
-        if name
-          ActiveSupport::Deprecation.warn(<<-MSG.squish)
-            Passing name to #indexes is deprecated without replacement.
-          MSG
-        end
-
-        table = Utils.extract_schema_qualified_name(table_name.to_s)
-
-        result = query(<<-SQL, "SCHEMA")
-          SELECT distinct i.relname, d.indisunique, d.indkey, pg_get_indexdef(d.indexrelid), t.oid,
-                          pg_catalog.obj_description(i.oid, 'pg_class') AS comment
-          FROM pg_class t
-          INNER JOIN pg_index d ON t.oid = d.indrelid
-          INNER JOIN pg_class i ON d.indexrelid = i.oid
-          LEFT JOIN pg_namespace n ON n.oid = i.relnamespace
-          WHERE i.relkind = 'i'
-            AND d.indisprimary = 'f'
-            AND t.relname = '#{table.identifier}'
-            AND n.nspname = #{table.schema ? "'#{table.schema}'" : 'ANY (current_schemas(false))'}
-          ORDER BY i.relname
-        SQL
-
-        result.map do |row|
-          index_name = row[0]
-          unique = row[1]
-          indkey = row[2].split(" ").map(&:to_i)
-          inddef = row[3]
-          oid = row[4]
-          comment = row[5]
-
-          expressions, where = inddef.scan(/\((.+?)\)(?: WHERE (.+))?\z/).flatten
-
-          if indkey.include?(0)
-            columns = expressions
-          else
-            columns = Hash[query(<<-SQL.strip_heredoc, "SCHEMA")].values_at(*indkey).compact
-              SELECT a.attnum, a.attname
-              FROM pg_attribute a
-              WHERE a.attrelid = #{oid}
-              AND a.attnum IN (#{indkey.join(",")})
-            SQL
-
-            # add info on sort order for columns (only desc order is explicitly specified, asc is the default)
-            orders = Hash[
-              expressions.scan(/(\w+) DESC/).flatten.map { |order_column| [order_column, :desc] }
-            ]
-          end
-
-          # FIXME(joey): This may be specific to ActiveRecord 5.2.
-          IndexDefinition.new(
-            table_name,
-            index_name,
-            unique,
-            columns,
-            orders: orders,
-            where: where,
-            comment: comment.presence
-          )
-        end.compact
-      end
-
-
-      def primary_keys(table_name)
-          name = Utils.extract_schema_qualified_name(table_name.to_s)
-          select_values(<<-SQL.strip_heredoc, "SCHEMA")
-          SELECT column_name
-              FROM information_schema.key_column_usage kcu
-              JOIN information_schema.table_constraints tc
-              ON kcu.table_name = tc.table_name
-              AND kcu.table_schema = tc.table_schema
-              AND kcu.constraint_name = tc.constraint_name
-              WHERE constraint_type = 'PRIMARY KEY'
-              AND kcu.table_name = #{quote(name.identifier)}
-              AND kcu.table_schema = #{name.schema ? quote(name.schema) : "ANY (current_schemas(false))"}
-              ORDER BY kcu.ordinal_position
-          SQL
+      def supports_string_to_array_coercion?
+        @crdb_version >= 202
       end
 
       # This is hardcoded to 63 (as previously was in ActiveRecord 5.0) to aid in
@@ -228,6 +135,25 @@ module ActiveRecord
       alias index_name_length max_identifier_length
       alias table_alias_length max_identifier_length
 
+      def initialize(connection, logger, conn_params, config)
+        super(connection, logger, conn_params, config)
+        crdb_version_string = query_value("SHOW crdb_version")
+        if crdb_version_string.include? "v1."
+          version_num = 1
+        elsif crdb_version_string.include? "v2."
+          version_num 2
+        elsif crdb_version_string.include? "v19.1."
+          version_num = 191
+        elsif crdb_version_string.include? "v19.2."
+          version_num = 192
+        elsif crdb_version_string.include? "v20.1."
+          version_num = 201
+        else
+          version_num = 202
+        end
+        @crdb_version = version_num
+      end
+
       private
 
         def initialize_type_map(m = type_map)
@@ -295,6 +221,62 @@ module ActiveRecord
           end
         end
 
+        # Override extract_value_from_default because the upstream definition
+        # doesn't handle the variations in CockroachDB's behavior.
+        def extract_value_from_default(default)
+          super ||
+            extract_escaped_string_from_default(default) ||
+            extract_time_from_default(default) ||
+            extract_empty_array_from_default(default)
+        end
+
+        # Both PostgreSQL and CockroachDB use C-style string escapes under the
+        # covers. PostgreSQL obscures this for us and unescapes the strings, but
+        # CockroachDB does not. Here we'll use Ruby to unescape the string.
+        # See https://github.com/cockroachdb/cockroach/issues/47497 and
+        # https://www.postgresql.org/docs/9.2/sql-syntax-lexical.html#SQL-SYNTAX-STRINGS-ESCAPE.
+        def extract_escaped_string_from_default(default)
+          # Escaped strings start with an e followed by the string in quotes (e'…')
+          return unless default =~ /\A[\(B]?e'(.*)'.*::"?([\w. ]+)"?(?:\[\])?\z/m
+
+          # String#undump doesn't account for escaped single quote characters
+          "\"#{$1}\"".undump.gsub("\\'".freeze, "'".freeze)
+        end
+
+        # This method exists to extract the correct time and date defaults for a
+        # couple of reasons.
+        # 1) There's a bug in CockroachDB where the date type is missing from
+        # the column info query.
+        # https://github.com/cockroachdb/cockroach/issues/47285
+        # 2) PostgreSQL's timestamp without time zone type maps to CockroachDB's
+        # TIMESTAMP type. TIMESTAMP includes a UTC time zone while timestamp
+        # without time zone doesn't.
+        # https://www.cockroachlabs.com/docs/v19.2/timestamp.html#variants
+        def extract_time_from_default(default)
+          return unless default =~ /\A'(.*)'\z/
+
+          # If default has a UTC time zone, we'll drop the time zone information
+          # so it acts like PostgreSQL's timestamp without time zone. Then, try
+          # to parse the resulting string to verify if it's a time.
+          time = if default =~ /\A'(.*)(\+00:00)'\z/
+            $1
+          else
+            default
+          end
+
+          Time.parse(time).to_s
+        rescue
+          nil
+        end
+
+        # CockroachDB stores default values for arrays in the `ARRAY[...]` format.
+        # In general, it is hard to parse that, but it is easy to handle the common
+        # case of an empty array.
+        def extract_empty_array_from_default(default)
+          return unless supports_string_to_array_coercion? 
+          return unless default =~ /\AARRAY\[\]\z/
+          return "{}"
+        end
 
       # end private
     end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activerecord-cockroachdb-adapter
 version: !ruby/object:Gem::Version
-  version: 0.2.3
+  version: 6.0.0beta1
 platform: ruby
 authors:
 - Cockroach Labs
-autorequire: 
+autorequire:
 bindir: exe
 cert_chain: []
-date: 2018-04-19 00:00:00.000000000 Z
+date: 2020-12-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -16,14 +16,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.2'
+        version: 6.0.3
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.2'
+        version: 6.0.3
 - !ruby/object:Gem::Dependency
   name: pg
   requirement: !ruby/object:Gem::Requirement
@@ -31,9 +31,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0.20'
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '0.22'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -41,51 +38,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0.20'
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '0.22'
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.14'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.14'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '10.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '10.0'
-- !ruby/object:Gem::Dependency
-  name: minitest
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '5.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '5.0'
 description: Allows the use of CockroachDB as a backend for ActiveRecord and Rails
   apps.
 email:
@@ -109,10 +61,15 @@ files:
 - build/local-test.sh
 - build/teamcity-test.sh
 - docker.sh
+- lib/active_record/connection_adapters/cockroachdb/attribute_methods.rb
+- lib/active_record/connection_adapters/cockroachdb/column.rb
+- lib/active_record/connection_adapters/cockroachdb/database_statements.rb
 - lib/active_record/connection_adapters/cockroachdb/database_tasks.rb
+- lib/active_record/connection_adapters/cockroachdb/quoting.rb
 - lib/active_record/connection_adapters/cockroachdb/referential_integrity.rb
 - lib/active_record/connection_adapters/cockroachdb/schema_statements.rb
 - lib/active_record/connection_adapters/cockroachdb/transaction_manager.rb
+- lib/active_record/connection_adapters/cockroachdb/type.rb
 - lib/active_record/connection_adapters/cockroachdb_adapter.rb
 - lib/activerecord-cockroachdb-adapter.rb
 homepage: https://github.com/cockroachdb/activerecord-cockroachdb-adapter
@@ -120,7 +77,7 @@ licenses:
 - Apache-2.0
 metadata:
   allowed_push_host: https://rubygems.org
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -131,13 +88,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.5.2
-signing_key: 
+rubygems_version: 3.1.4
+signing_key:
 specification_version: 4
 summary: CockroachDB adapter for ActiveRecord.
 test_files: []