activerecord-cockroachdb-adapter 0.2.2 → 5.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: c39398a79fef87dc0b0cde4cf0558ca1820005a5
-  data.tar.gz: e18cf876bd4be4886a9429b55c5a82704c80d957
+SHA256:
+  metadata.gz: b2292eb1fe9b5352ccb2a135ee1681b342403f086d38d5c78f354d97008097fe
+  data.tar.gz: 7b144a460690d0a6eab7d827eb1e297c67f94be4191d8c949f33d74e90050a88
 SHA512:
-  metadata.gz: 60bcfd758fad44e6f3922896301f2c750d53b310879d525b8f70274838a1639a722614a2b9ad5e6f2db2e9e429b0c2b1f80adfd9d3c69473e00b4c6b8dad481c
-  data.tar.gz: 3ef108048920f4708465cebda053a2681420bf77da6b0a7ab036ff6a3f1c42060620d6c53bb21a4df9384e6b4f0ee2f0b0a14fadfef57e884c25bc1fb86d21b9
+  metadata.gz: 6424288bb2dce9c2a530f3d2b0edd303c156fdd8dd85822756f9953ca7ebaff4cc32fed8cc233c580e570f561797d38ca741837edeccf5c4e95639b7e2dab816
+  data.tar.gz: 0d69e3fbf8e5809429608bbd292269affc21d88afcd7687a26443e6bbb55754bd623ba4d62da484a1ad4415f6633ba9f49eb57ba1e7d54ba632d3fc30e922eaa

data/.gitignore

@@ -7,3 +7,4 @@
 /pkg/
 /spec/reports/
 /tmp/
+*.gem

data/CONTRIBUTING.md

@@ -0,0 +1,220 @@
+# Getting started
+
+
+## ActiveRecord adapters and you
+
+There are two repositories for the ActiveRecord adapter. The one you're in
+currently, [activerecord-cockroachdb-adapter], is the CockroachDB specific
+ActiveRecord code. Users install this alongside ActiveRecord then use
+CockroachDBAdapter to initialize ActiveRecord for their projects.
+
+This adapter extends the PostgreSQL ActiveRecord adapter in order to
+override and monkey-patch functionality.
+
+[activerecord-cockroachdb-adapter]: https://github.com/cockroachdb/activerecord-cockroachdb-adapter/
+
+## Setup and running tests
+
+In CockroachDB, create two databases to be used by the ActiveRecord test suite:
+activerecord_unittest and activerecord_unittest2.
+
+```sql
+CREATE DATABASE activerecord_unittest;
+
+CREATE DATABASE activerecord_unittest2;
+```
+
+It is best to have a Ruby environment manager installed, such as
+[rvm](https://rvm.io/), as Rails has varying Ruby version requirements.
+If you are using rvm, you then install and use the required Ruby
+version.  The current tests use Rails 5.2.0 beta and Ruby >= 2.2.2.
+
+(Alternatively, one can use `./docker.sh build/teamcity-test.sh` to run
+tests similarily to TeamCity. The database is destroyed between each
+test file.)
+
+
+```bash
+rvm install 2.2.5
+# This only makes Ruby 2.2.5 active for the length of the terminal session.
+rvm use 2.2.5
+```
+
+Using [bundler](http://bundler.io/), install the dependancies of Rails.
+
+```bash
+bundle install
+```
+
+Then, to run the full test suite with an active CockroachDB instance:
+
+```bash
+bundle exec rake test
+```
+
+To run specific ActiveRecord tests, set environemnt variable `TEST_FILES_AR`. For example, to run ActiveRecord tests `test/cases/associations_test.rb` and `test/cases/ar_schema_test.rb.rb`
+
+```bash
+TEST_FILES_AR="test/cases/associations_test.rb,test/cases/ar_schema_test.rb" bundle exec rake test
+```
+
+To run specific CockroachDB Adapter tests, set environemnt variable `TEST_FILES`. For example, to run CockroachDB Adpater tests `test/cases/adapter_test.rb` and `test/cases/associations/left_outer_join_association_test.rb`
+
+```bash
+TEST_FILES="test/cases/adapter_test.rb,test/cases/associations/left_outer_join_association_test.rb" bundle exec rake test
+```
+
+To run a specific test case, use minitest's `-n` option to run tests that match a given pattern. All minitest options are set via the `TESTOPTS` environemnt variable. For example, to run `test_indexes` from CockroachDB's `test/cases/adapter_test.rb` file
+
+```bash
+TEST_FILES="test/cases/adapter_test.rb" TESTOPTS=`-n=/test_indexes/` bundle exec rake test
+```
+
+By default, tests will be run from the bundled version of Rails. To run against a local copy, set environemnt variable `RAILS_SOURCE`. Running against a local copy of Rails can be helpful when try to debug issues.
+
+```bash
+RAILS_SOURCE="path/to/local_copy" bundle exec rake test
+```
+
+`test/config.yml` assumes CockroachDB will be running at localhost:26257 with a root user. Make changes to `test/config.yml` as needed.
+
+# Improvements
+
+
+## Support past Rails versions
+
+Currently, only a beta version of Rails is tested. This means that the
+adapter has been modified in to accomodate unreleased changes. In order
+to run the tests for Rails 5.1 or 4.2, the test changes will need to be
+cherry-picked back. Conflicts are mostly only expected for tests that
+have not yet been added.
+
+Sadly, this does mean that we will have to have multiple versions of the
+driver for the multiple versions of Rails.
+
+A proposal for the CockroachDB adapter versioning would be to follow
+ActiveRecord minor versions. For example, if you use Rails 4.2.5, you
+would specify the CockroachDB version `~> 4.2.0`.
+
+
+## Running CI automatically
+
+Currently the fork is set up to run using TeamCity only on the current
+master branch, with an alpha build of CockroachDB. it would be even
+better to be able to test multiple versions of the adapter, and do so
+against different versions of CockroachDB.
+
+
+## Adding feature support
+
+As CockroachDB improves, so do the features that can be supported in
+ActiveRecord. Many of them are gated by conditions the
+CockroachDBAdapter has overrided. As these features are completed, these
+gates should be toggled. Something that would help this process would be
+linking those issues back to this adapter so that part of the feature
+completing includes updating the adapter.
+
+
+## Execute only tests that run with a connection
+
+I have not investigated if this is already possible, but I would assume
+no.
+
+A possible way to approach this would be to add a shim to cause any
+tests that use it to fail, and grep the tests that pass and then skip
+them.
+
+## Publishing to Rubygems
+
+TODO: Expand on this. Jordan is likely the only person with publishing
+credentials. I'm not sure if there is anything else other than:
+
+```
+gem build ...
+gem publish <output file>
+```
+
+
+# Notes
+
+When executing the test suite, each test file will reload fixtures. This
+drops and creates about 200 tables (2 databases, 100 tables each).
+Currently there are performance problems that rise from having lots of
+table descriptors around, [cockroachdb/cockroach#20753]. At best, we can
+run test files individually, clear out the CockroachDB data, and restart
+the node to alleviate this.
+
+Currently, annotations have been added to test files to indicate if it
+is failing, and some brief details on why. Any annotated failures have
+been skipped right now for further investigation. The pattern is the
+following:
+
+`# FILE(OK)` indicates that the file is currently passing, with no skips
+required.
+
+`# FILE(BAD)` indicates that there are failures that have been skipped.
+These skips will look like `skip(reason) if current_adapter?(:CockroachDBAdapter)`.
+
+`# FILE(BROKEN)` indicates that there are failures that have not been
+skipped. This is often done if the entirety of a test file is
+unsupported.
+
+`# FILE(NOT DONE)` indicates files that have not yet been executed,
+cleaned up, or skipped until passing.
+
+The purpose of these was to make the tests grep-able while going through
+all the failures.
+
+
+[cockroachdb/cockroach#20753]: https://github.com/cockroachdb/cockroach/issues/20753#issuecomment-352810425
+
+
+## Tracked test failures
+
+Some of the skipped failures are:
+
+- `default:` key is not working for columns in table schema
+  definitions. This causes tests to fail due to unexpected data.
+
+- `array:` key is not working for columns in table schema definitions.
+
+- `"Salary is not appearing in list"` is being raised in a number of
+  places. Likely during fixture setup.
+
+- `sum` function seems to result in a different type in ActiveRecord.
+  Instead of returning a Ruby `int`, it returns a Ruby `string`. It
+  appears that MySQL2 also does this. A suspected cause might be how
+  `Decimal` is handled if `sum` consumes integers and return a
+  decimal.
+
+- Potentially fork the PostgreSQL::SchemaDumper to handle anything
+  specific to CockroachDB, like primary keys or bigints.
+
+- You can call `@connection.create_table(..., id: :bigint)`, but this
+  will not changes things for CockroachDB (I think...), so it would be
+  not allowed. Even better, our adapter could interpret this and
+  generate the appropriate explicit pkey column. Not sure what string
+  pkeys look like...
+
+- `string` types are introspected to `text` types.
+
+- A user can do an update, delete, and insert on views.
+
+- Postgres specific bit strings are not properly supported.
+
+Grepping for `FIXME(joey)`, `TODO(joey)`, and `NOTE(joey)` will yeild
+most of the touchpoints including test failures and temporary monkey
+patches. Some monkey patches were made directly to Rails, which will
+need to be cleaned up.
+
+
+# Notes for the non-Rubyer
+
+rvm is an environment manager that lets you manage and swap between
+multiple verisons of Ruby and their dependancies.
+
+bundle is dependancy manager that uses a projects `Gemfile` (and often
+`<project>.gemspec`) to manage and load dependancies and their required
+versions. When using projects commands are prefixed with
+`bundle exec ...`. Bundle will ensure that all depenedncies are fetched
+and used.

data/Gemfile

@@ -1,4 +1,63 @@
+require 'openssl'
 source 'https://rubygems.org'
-
-# Specify your gem's dependencies in activerecord-cockroachdb-adapter.gemspec
 gemspec
+
+if ENV['RAILS_SOURCE']
+  gemspec path: ENV['RAILS_SOURCE']
+else
+  def get_version_from_gemspec
+    gemspec = eval(File.read('activerecord-cockroachdb-adapter.gemspec'))
+
+    gem_version = gemspec.dependencies.
+      find { |dep| dep.name == 'activerecord' }.
+      requirement.
+      requirements.
+      first.
+      last
+
+    major, minor, tiny, pre = gem_version.segments
+
+    if pre
+      gem_version.to_s
+    else
+      find_latest_matching_version(major, minor)
+    end
+  end
+
+  def find_latest_matching_version(gemspec_major, gemspec_minor)
+    all_activerecord_versions.
+      reject { |version| version["prerelease"] }.
+      map { |version| version["number"].split(".").map(&:to_i) }.
+      find { |major, minor|
+        major == gemspec_major && (minor == gemspec_minor || gemspec_minor.nil?)
+      }.join(".")
+  end
+
+  def all_activerecord_versions
+    require 'net/http'
+    require 'yaml'
+
+    uri = URI.parse "https://rubygems.org/api/v1/versions/activerecord.yaml"
+    http = Net::HTTP.new(uri.host, uri.port)
+    http.use_ssl = true
+    http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+
+    YAML.load(
+      http.request(Net::HTTP::Get.new(uri.request_uri)).body
+    )
+  end
+
+  # Get Rails from source beacause the gem doesn't include tests
+  version = ENV['RAILS_VERSION'] || get_version_from_gemspec
+  gem 'rails', git: "https://github.com/rails/rails.git", tag: "v#{version}"
+end
+
+group :development do
+  gem "byebug"
+  gem "minitest-excludes"
+
+  # Gems used by the ActiveRecord test suite
+  gem "bcrypt"
+  gem "mocha"
+  gem "sqlite3"
+end

data/README.md

@@ -1,19 +1,29 @@
 # ActiveRecord CockroachDB Adapter
 
-CockroachDB adapter for ActiveRecord 5. This is a lightweight extension of the PostgreSQL adapter that establishes compatibility with [CockroachDB](https://github.com/cockroachdb/cockroach).
+CockroachDB adapter for ActiveRecord 4 and 5. This is a lightweight extension of the PostgreSQL adapter that establishes compatibility with [CockroachDB](https://github.com/cockroachdb/cockroach).
 
 ## Installation
 
 Add this line to your project's Gemfile:
 
 ```ruby
-gem 'activerecord-cockroachdb-adapter', '~> 0.2.2'
+gem 'activerecord-cockroachdb-adapter', '~> 5.2.0'
 ```
 
+If you're using Rails 4.x, use the `0.1.x` versions of this gem.
+
 In `database.yml`, use the following adapter setting:
 
 ```
 development:
   adapter: cockroachdb
   port: 26257
+  host: <hostname>
+  user: <username>
 ```
+
+
+## Modifying the adapter?
+
+See [CONTRIBUTING.md](/CONTRIBUTING.md) for more details on setting up
+the environment and making modifications.

data/Rakefile

@@ -1,10 +1,22 @@
 require "bundler/gem_tasks"
 require "rake/testtask"
+require_relative 'test/support/paths_cockroachdb'
+require_relative 'test/support/rake_helpers'
 
-Rake::TestTask.new(:test) do |t|
-  t.libs << "test"
-  t.libs << "lib"
-  t.test_files = FileList['test/**/*_test.rb']
+task test: ["test:cockroachdb"]
+task default: [:test]
+
+namespace :test do
+  Rake::TestTask.new("cockroachdb") do |t|
+    t.libs = ARTest::CockroachDB.test_load_paths
+    t.test_files = test_files
+    t.warning = !!ENV["WARNING"]
+    t.verbose = false
+  end
+
+  task "cockroachdb:env" do
+    ENV["ARCONN"] = "cockroachdb"
+  end
 end
 
-task :default => :test
+task 'test:cockroachdb' => 'test:cockroachdb:env'

data/activerecord-cockroachdb-adapter.gemspec

@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name          = "activerecord-cockroachdb-adapter"
-  spec.version       = "0.2.2"
+  spec.version       = "5.2.1"
   spec.licenses      = ["Apache-2.0"]
   spec.authors       = ["Cockroach Labs"]
   spec.email         = ["cockroach-db@googlegroups.com"]
@@ -13,8 +13,8 @@ Gem::Specification.new do |spec|
   spec.description   = "Allows the use of CockroachDB as a backend for ActiveRecord and Rails apps."
   spec.homepage      = "https://github.com/cockroachdb/activerecord-cockroachdb-adapter"
 
-  spec.add_dependency "activerecord", "~> 5.1", ">= 5.1.1"
-  spec.add_dependency "pg", ">= 0.20", "< 0.22"
+  spec.add_dependency "activerecord", "~> 5.2"
+  spec.add_dependency "pg", ">= 0.20"
 
   # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
   # to allow pushing to a single host or delete this section to allow pushing to any host.
@@ -31,8 +31,4 @@ Gem::Specification.new do |spec|
   spec.bindir        = "exe"
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
-
-  spec.add_development_dependency "bundler", "~> 1.14"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "minitest", "~> 5.0"
 end

data/build/Dockerfile

@@ -0,0 +1,17 @@
+# This Dockerfile extends the Examples-ORM testing image in order to
+# install specific dependencies required for ActiveRecord tests.
+
+FROM cockroachdb/example-orms-builder:20200413-1918
+
+# Native dependencies for libxml-ruby and sqlite3.
+RUN apt-get update -y && apt-get install -y \
+  libxslt-dev \
+  libxml2-dev \
+  libsqlite3-dev \
+  && rm -rf /var/lib/apt/lists/*
+
+# Ruby testing dependencies.
+RUN gem install bundle rake
+
+# Add global Gem binaries to the path.
+ENV PATH /usr/local/lib/ruby/gems/2.4.0::$PATH

data/build/config.teamcity.yml

@@ -0,0 +1,28 @@
+default_connection: cockroachdb
+
+with_manual_interventions: false
+
+connections:
+  cockroachdb:
+    arunit:
+      database: activerecord_unittest
+      host: localhost
+      port: 26257
+      user: root
+      requiressl: disable
+      min_messages: warning
+    arunit_without_prepared_statements:
+      database: activerecord_unittest
+      host: localhost
+      port: 26257
+      user: root
+      requiressl: disable
+      min_messages: warning
+      prepared_statements: false
+    arunit2:
+      database: activerecord_unittest2
+      host: localhost
+      port: 26257
+      user: root
+      requiressl: disable
+      min_messages: warning

data/build/local-test.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+readonly urlfile=cockroach-url
+
+# Start a CockroachDB server, wait for it to become ready, and arrange for it to
+# be force-killed when the script exits.
+rm -f "$urlfile"
+# Clean out a past CockroachDB instance. This happens if a build was
+# canceled on an agent.
+rm -rf $HOME/tmp/rails &
+# Start CockroachDB.
+cockroach quit --insecure || true
+cockroach start --insecure --host=localhost --listening-url-file="$urlfile" --store=path=$HOME/tmp/rails &
+trap "echo 'Exit routine: Killing CockroachDB.' && kill -9 $! &> /dev/null" EXIT
+for i in {0..3}
+do
+  [[ -f "$urlfile" ]] && break
+  backoff=$((2 ** i))
+  echo "server not yet available; sleeping for $backoff seconds"
+  sleep $backoff
+done
+
+# Target the Rails dependency file.
+export BUNDLE_GEMFILE=$(pwd)/rails/Gemfile
+
+# Run the tests.
+cp build/config.teamcity.yml rails/activerecord/test/config.yml
+echo "Rebuilding database"
+(cd rails/activerecord && bundle exec rake db:cockroachdb:rebuild)
+echo "Starting tests"
+(cd rails/activerecord && bundle exec rake test:cockroachdb TESTFILES=$1)
+
+# Attempt a clean shutdown for good measure. We'll force-kill in the atexit
+# handler if this fails.
+cockroach quit --insecure
+trap - EXIT