active_record_proxy_adapters 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 265cb32e99a265497548122846934c46405747109fa3bee34d1e2e66203b14a3
4
+ data.tar.gz: d9de3faf7274a0a94166637032c0a967e083edf55fc275b2456123d3f9a7ee79
5
+ SHA512:
6
+ metadata.gz: 871504b4290e8304effa5e30962a8db9fdab6663c73b4f9d9e99edf403af630755ddaaafa557619a86430812b7e54e19931274ec71dd4f86a7b4dc32b73fc669
7
+ data.tar.gz: f7605b374452517ae3346d896d043e092539abd0a86445eb1a04defbd7516628a70bcd5157fca10c730b7ee6fe047733d1e8e4d65a016c49651cbb12f7e8dcee
data/.rspec ADDED
@@ -0,0 +1,4 @@
1
+ --format documentation
2
+ --color
3
+ --order random
4
+ --require spec_helper
data/.rubocop.yml ADDED
@@ -0,0 +1,16 @@
1
+ require:
2
+ - rubocop-rspec
3
+
4
+ AllCops:
5
+ TargetRubyVersion: 3.1
6
+ NewCops: enable
7
+
8
+ Style/StringLiterals:
9
+ EnforcedStyle: double_quotes
10
+
11
+ Style/StringLiteralsInInterpolation:
12
+ EnforcedStyle: double_quotes
13
+
14
+ RSpec/NestedGroups:
15
+ Enabled: true
16
+ Max: 5
data/CHANGELOG.md ADDED
@@ -0,0 +1,9 @@
1
+ ## [0.1.0] - 2024-11-19
2
+
3
+ - Add PostgreSQLProxyAdapter
4
+
5
+ ## [0.1.0.rc2] - 2024-10-28
6
+
7
+ - Add PostgreSQLProxyAdapter
8
+
9
+ ## [Unreleased]
@@ -0,0 +1,132 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ We as members, contributors, and leaders pledge to make participation in our
6
+ community a harassment-free experience for everyone, regardless of age, body
7
+ size, visible or invisible disability, ethnicity, sex characteristics, gender
8
+ identity and expression, level of experience, education, socio-economic status,
9
+ nationality, personal appearance, race, caste, color, religion, or sexual
10
+ identity and orientation.
11
+
12
+ We pledge to act and interact in ways that contribute to an open, welcoming,
13
+ diverse, inclusive, and healthy community.
14
+
15
+ ## Our Standards
16
+
17
+ Examples of behavior that contributes to a positive environment for our
18
+ community include:
19
+
20
+ * Demonstrating empathy and kindness toward other people
21
+ * Being respectful of differing opinions, viewpoints, and experiences
22
+ * Giving and gracefully accepting constructive feedback
23
+ * Accepting responsibility and apologizing to those affected by our mistakes,
24
+ and learning from the experience
25
+ * Focusing on what is best not just for us as individuals, but for the overall
26
+ community
27
+
28
+ Examples of unacceptable behavior include:
29
+
30
+ * The use of sexualized language or imagery, and sexual attention or advances of
31
+ any kind
32
+ * Trolling, insulting or derogatory comments, and personal or political attacks
33
+ * Public or private harassment
34
+ * Publishing others' private information, such as a physical or email address,
35
+ without their explicit permission
36
+ * Other conduct which could reasonably be considered inappropriate in a
37
+ professional setting
38
+
39
+ ## Enforcement Responsibilities
40
+
41
+ Community leaders are responsible for clarifying and enforcing our standards of
42
+ acceptable behavior and will take appropriate and fair corrective action in
43
+ response to any behavior that they deem inappropriate, threatening, offensive,
44
+ or harmful.
45
+
46
+ Community leaders have the right and responsibility to remove, edit, or reject
47
+ comments, commits, code, wiki edits, issues, and other contributions that are
48
+ not aligned to this Code of Conduct, and will communicate reasons for moderation
49
+ decisions when appropriate.
50
+
51
+ ## Scope
52
+
53
+ This Code of Conduct applies within all community spaces, and also applies when
54
+ an individual is officially representing the community in public spaces.
55
+ Examples of representing our community include using an official email address,
56
+ posting via an official social media account, or acting as an appointed
57
+ representative at an online or offline event.
58
+
59
+ ## Enforcement
60
+
61
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
62
+ reported to the community leaders responsible for enforcement at
63
+ [INSERT CONTACT METHOD].
64
+ All complaints will be reviewed and investigated promptly and fairly.
65
+
66
+ All community leaders are obligated to respect the privacy and security of the
67
+ reporter of any incident.
68
+
69
+ ## Enforcement Guidelines
70
+
71
+ Community leaders will follow these Community Impact Guidelines in determining
72
+ the consequences for any action they deem in violation of this Code of Conduct:
73
+
74
+ ### 1. Correction
75
+
76
+ **Community Impact**: Use of inappropriate language or other behavior deemed
77
+ unprofessional or unwelcome in the community.
78
+
79
+ **Consequence**: A private, written warning from community leaders, providing
80
+ clarity around the nature of the violation and an explanation of why the
81
+ behavior was inappropriate. A public apology may be requested.
82
+
83
+ ### 2. Warning
84
+
85
+ **Community Impact**: A violation through a single incident or series of
86
+ actions.
87
+
88
+ **Consequence**: A warning with consequences for continued behavior. No
89
+ interaction with the people involved, including unsolicited interaction with
90
+ those enforcing the Code of Conduct, for a specified period of time. This
91
+ includes avoiding interactions in community spaces as well as external channels
92
+ like social media. Violating these terms may lead to a temporary or permanent
93
+ ban.
94
+
95
+ ### 3. Temporary Ban
96
+
97
+ **Community Impact**: A serious violation of community standards, including
98
+ sustained inappropriate behavior.
99
+
100
+ **Consequence**: A temporary ban from any sort of interaction or public
101
+ communication with the community for a specified period of time. No public or
102
+ private interaction with the people involved, including unsolicited interaction
103
+ with those enforcing the Code of Conduct, is allowed during this period.
104
+ Violating these terms may lead to a permanent ban.
105
+
106
+ ### 4. Permanent Ban
107
+
108
+ **Community Impact**: Demonstrating a pattern of violation of community
109
+ standards, including sustained inappropriate behavior, harassment of an
110
+ individual, or aggression toward or disparagement of classes of individuals.
111
+
112
+ **Consequence**: A permanent ban from any sort of public interaction within the
113
+ community.
114
+
115
+ ## Attribution
116
+
117
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage],
118
+ version 2.1, available at
119
+ [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
120
+
121
+ Community Impact Guidelines were inspired by
122
+ [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
123
+
124
+ For answers to common questions about this code of conduct, see the FAQ at
125
+ [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
126
+ [https://www.contributor-covenant.org/translations][translations].
127
+
128
+ [homepage]: https://www.contributor-covenant.org
129
+ [v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
130
+ [Mozilla CoC]: https://github.com/mozilla/diversity
131
+ [FAQ]: https://www.contributor-covenant.org/faq
132
+ [translations]: https://www.contributor-covenant.org/translations
data/Dockerfile ADDED
@@ -0,0 +1,18 @@
1
+ ARG RUBY_VERSION=3.2.3
2
+ ARG DOCKER_REGISTRY=docker.io
3
+ FROM $DOCKER_REGISTRY/ruby:$RUBY_VERSION-alpine
4
+ ARG RAILS_VERSION="~> 6.1.0"
5
+ ENV RAILS_VERSION=$RAILS_VERSION
6
+
7
+ RUN apk --update add \
8
+ build-base \
9
+ git \
10
+ postgresql-dev \
11
+ postgresql-client
12
+ RUN gem install bundler -v 2.5.13
13
+
14
+ COPY . /app
15
+ WORKDIR /app
16
+
17
+ RUN bundle install
18
+
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2024 Nasdaq
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,112 @@
1
+ # ActiveRecordProxyAdapters
2
+
3
+ A set of ActiveRecord adapters that leverage Rails native multiple database setup to allow automatic connection switching from _one_ primary pool to _one_ replica pool at the database statement level.
4
+
5
+ ## Installation
6
+
7
+ Install the gem and add to the application's Gemfile by executing:
8
+
9
+ $ bundle add 'active_record_proxy_adapters'
10
+
11
+ If bundler is not being used to manage dependencies, install the gem by executing:
12
+
13
+ $ gem install active_record_proxy_adapters
14
+
15
+ ## Usage
16
+
17
+ ### On Rails
18
+
19
+ In `config/database.yml`, use `postgresql_proxy` as the adapter for the `primary` database, and keep `postgresql` for the replica database.
20
+
21
+ ```yaml
22
+ # config/database.yml
23
+ development:
24
+ primary:
25
+ adapter: postgresql_proxy
26
+ # your primary credentials here
27
+
28
+ primary_replica:
29
+ adapter: postgresql
30
+ replica: true
31
+ # your replica credentials here
32
+ ```
33
+
34
+ ### Off Rails
35
+
36
+ ```ruby
37
+ # In your application setup
38
+ require "active_record_proxy_adapters"
39
+
40
+ ActiveSupport.on_load :active_record do
41
+ require "active_record_proxy_adapters/connection_handling"
42
+ ActiveRecord::Base.extend(ActiveRecordProxyAdapters::ConnectionHandling)
43
+ end
44
+
45
+ # in your base model
46
+ class ApplicationRecord << ActiveRecord::Base
47
+ establish_connection(
48
+ {
49
+ adapter: 'postgresql_proxy',
50
+ # your primary credentials here
51
+ },
52
+ role: :writing
53
+ )
54
+
55
+ establish_connection(
56
+ {
57
+ adapter: 'postgresql',
58
+ # your replica credentials here
59
+ },
60
+ role: :reading
61
+ )
62
+ end
63
+ ```
64
+
65
+ ### Configuration
66
+
67
+ The gem comes preconfigured out of the box. However, if default configuration does not suit your needs, you can modify them by using a `.configure` block:
68
+
69
+ ```ruby
70
+ # config/initializers/active_record_proxy_adapters.rb
71
+ ActiveRecordProxyAdapters.configure do |config|
72
+ # How long proxy should reroute all read requests to primary after a write
73
+ config.proxy_delay = 5.seconds # defaults to 2.seconds
74
+
75
+ # How long proxy should wait for replica to connect.
76
+ config.checkout_timeout = 5.seconds # defaults to 2.seconds
77
+ end
78
+ ```
79
+
80
+ ### How it works
81
+
82
+ The proxy will analyze each SQL string, using pattern matching, to decide the appropriate connection for it (i.e. if it should go to the primary or replica).
83
+
84
+ - All queries inside a transaction go to the primary
85
+ - All `SET` queries go to all connections
86
+ - All `INSERT`, `UPDATE` and `DELETE` queries go to the primary
87
+ - All `SELECT FOR UPDATE` queries go to the primary
88
+ - All `lock` queries (e.g `get_lock`) go the primary
89
+ - All sequence methods (e.g `nextval`) go the primary
90
+ - Everything else goes to the replica
91
+
92
+ #### TL;DR
93
+
94
+ All `SELECT` queries go to the _replica_, everything else goes to _primary_.
95
+
96
+ ## Development
97
+
98
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
99
+
100
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
101
+
102
+ ## Contributing
103
+
104
+ Bug reports and pull requests are welcome on GitHub at https://github.com/nasdaq/active_record_proxy_adapters. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/nasdaq/active_record_proxy_adapters/blob/main/CODE_OF_CONDUCT.md).
105
+
106
+ ## License
107
+
108
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
109
+
110
+ ## Code of Conduct
111
+
112
+ Everyone interacting in the ActiveRecordProxyAdapters project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/nasdaq/active_record_proxy_adapters/blob/main/CODE_OF_CONDUCT.md).
data/Rakefile ADDED
@@ -0,0 +1,12 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "bundler/gem_tasks"
4
+ require "rspec/core/rake_task"
5
+
6
+ RSpec::Core::RakeTask.new(:spec)
7
+
8
+ require "rubocop/rake_task"
9
+
10
+ RuboCop::RakeTask.new
11
+
12
+ task default: %i[spec rubocop]
@@ -0,0 +1,77 @@
1
+ name: active_record_proxy_adapters
2
+
3
+ x-postgres-common: &postgres-common
4
+ restart: always
5
+ user: postgres
6
+ healthcheck:
7
+ test: 'pg_isready -U postgres_primary_test --dbname=postgres'
8
+ interval: 10s
9
+ timeout: 5s
10
+ retries: 5
11
+ networks:
12
+ - postgres
13
+
14
+ services:
15
+ app:
16
+ build:
17
+ args:
18
+ - RUBY_VERSION=${RUBY_VERSION:-3.2.3}
19
+ - RAILS_VERSION=${RAILS_VERSION:-~> 6.1.0}
20
+ container_name: app
21
+ image: active_record_proxy_adapters-app:${ENV_TAG:-latest}
22
+ tty: true
23
+ stdin_open: true
24
+ environment:
25
+ PGHOST: postgres_primary
26
+ PG_PRIMARY_USER: postgres_primary_test
27
+ PG_PRIMARY_PASSWORD: postgres_primary_test
28
+ PG_PRIMARY_HOST: postgres_primary
29
+ PG_PRIMARY_PORT: 5432
30
+ PG_REPLICA_USER: postgres_primary_test
31
+ PG_REPLICA_PASSWORD: postgres_primary_test
32
+ PG_REPLICA_HOST: postgres_replica
33
+ PG_REPLICA_PORT: 5432
34
+ depends_on:
35
+ - postgres_primary
36
+ - postgres_replica
37
+ networks:
38
+ - app
39
+ - postgres
40
+ volumes:
41
+ - .:/app
42
+ postgres_primary:
43
+ <<: *postgres-common
44
+ build:
45
+ context: .
46
+ dockerfile: postgres_primary.dockerfile
47
+ args:
48
+ - POSTGRES_LOGGING_COLLECTOR=${POSTGRES_LOGGING_COLLECTOR:-}
49
+ - POSTGRES_LOG_DESTINATION=${POSTGRES_LOG_DESTINATION:-}
50
+ - POSTGRES_LOG_STATEMENT=${POSTGRES_LOG_STATEMENT:-}
51
+ - REPLICA_USER=replicator
52
+ - REPLICA_PASSWORD=replicator
53
+ environment:
54
+ POSTGRES_DB: postgres
55
+ POSTGRES_USER: postgres_primary_test
56
+ POSTGRES_PASSWORD: postgres_primary_test
57
+ POSTGRES_HOST_AUTH_METHOD: "scram-sha-256\nhost replication all 0.0.0.0/0 md5"
58
+ POSTGRES_INITDB_ARGS: "--auth-host=scram-sha-256"
59
+
60
+ postgres_replica:
61
+ <<: *postgres-common
62
+ build:
63
+ context: .
64
+ dockerfile: postgres_replica.dockerfile
65
+ container_name: postgres_replica
66
+ environment:
67
+ PGUSER: replicator
68
+ PGPASSWORD: replicator
69
+ PRIMARY_DATABASE_HOST: postgres_primary
70
+ depends_on:
71
+ - postgres_primary
72
+ networks:
73
+ app:
74
+ postgres:
75
+
76
+ volumes:
77
+ postgres_primary:
@@ -0,0 +1,43 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record/tasks/postgresql_proxy_database_tasks"
4
+ require "active_record/connection_adapters/postgresql_adapter"
5
+ require "active_record_proxy_adapters/active_record_context"
6
+ require "active_record_proxy_adapters/hijackable"
7
+ require "active_record_proxy_adapters/postgresql_proxy"
8
+
9
+ module ActiveRecord
10
+ module ConnectionAdapters
11
+ # This adapter is a proxy to the original PostgreSQLAdapter, allowing the use of the
12
+ # ActiveRecordProxyAdapters::PrimaryReplicaProxy.
13
+ class PostgreSQLProxyAdapter < PostgreSQLAdapter
14
+ include ActiveRecordProxyAdapters::Hijackable
15
+
16
+ ADAPTER_NAME = "PostgreSQLProxy"
17
+
18
+ delegate_to_proxy :execute, :exec_query
19
+
20
+ unless ActiveRecordProxyAdapters::ActiveRecordContext.active_record_v8_0_or_greater?
21
+ delegate_to_proxy :exec_no_cache, :exec_cache
22
+ end
23
+
24
+ def initialize(...)
25
+ @proxy = ActiveRecordProxyAdapters::PostgreSQLProxy.new(self)
26
+
27
+ super
28
+ end
29
+
30
+ private
31
+
32
+ attr_reader :proxy
33
+ end
34
+ end
35
+ end
36
+
37
+ if ActiveRecordProxyAdapters::ActiveRecordContext.active_record_v7_2_or_greater?
38
+ ActiveRecord::ConnectionAdapters.register(
39
+ "postgresql_proxy",
40
+ "ActiveRecord::ConnectionAdapters::PostgreSQLProxyAdapter",
41
+ "active_record/connection_adapters/postgresql_proxy_adapter"
42
+ )
43
+ end
@@ -0,0 +1,45 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecord
4
+ module Tasks
5
+ # Defines the postgresql tasks for dropping, creating, loading schema and dumping schema.
6
+ # Bypasses all the proxy logic to send all requests to primary.
7
+ class PostgreSQLProxyDatabaseTasks < PostgreSQLDatabaseTasks
8
+ def create(...)
9
+ sticking_to_primary { super }
10
+ end
11
+
12
+ def drop(...)
13
+ sticking_to_primary { super }
14
+ end
15
+
16
+ def structure_dump(...)
17
+ sticking_to_primary { super }
18
+ end
19
+
20
+ def structure_load(...)
21
+ sticking_to_primary { super }
22
+ end
23
+
24
+ def purge(...)
25
+ sticking_to_primary { super }
26
+ end
27
+
28
+ private
29
+
30
+ def sticking_to_primary(&)
31
+ ActiveRecord::Base.connected_to(role: context.writing_role, &)
32
+ end
33
+
34
+ def context
35
+ ActiveRecordProxyAdapters::ActiveRecordContext.new
36
+ end
37
+ end
38
+ end
39
+ end
40
+
41
+ # Allow proxy adapter to run rake tasks, i.e. db:drop, db:create, db:schema:load db:migrate, etc...
42
+ ActiveRecord::Tasks::DatabaseTasks.register_task(
43
+ /postgresql_proxy/,
44
+ "ActiveRecord::Tasks::PostgreSQLProxyDatabaseTasks"
45
+ )
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordProxyAdapters
4
+ # Collection of helpers to handle common active record methods that are defined in different places in different
5
+ # versions of rails.
6
+ class ActiveRecordContext
7
+ delegate :reading_role, :reading_role=, :writing_role, :writing_role=, to: :ActiveRecord
8
+ delegate :legacy_connection_handling, :legacy_connection_handling=, to: :connection_handling_context
9
+ delegate :version, to: :ActiveRecord, prefix: :active_record
10
+
11
+ class << self
12
+ delegate_missing_to :new
13
+ end
14
+
15
+ NullConnectionHandlingContext = Class.new do
16
+ def legacy_connection_handling
17
+ false
18
+ end
19
+
20
+ def legacy_connection_handling=(_value)
21
+ nil
22
+ end
23
+ end
24
+
25
+ def connection_class_for(connection)
26
+ connection.connection_class || ActiveRecord::Base
27
+ end
28
+
29
+ def connection_handling_context
30
+ # This config option has been removed in Rails 7.1+
31
+ return NullConnectionHandlingContext.new if active_record_v7_1_or_greater?
32
+
33
+ ActiveRecord
34
+ end
35
+
36
+ def active_record_v7_1_or_greater?
37
+ active_record_version >= Gem::Version.new("7.1")
38
+ end
39
+
40
+ def active_record_v7_2_or_greater?
41
+ active_record_version >= Gem::Version.new("7.2")
42
+ end
43
+
44
+ def active_record_v8_0_or_greater?
45
+ active_record_version >= Gem::Version.new("8.0")
46
+ end
47
+ end
48
+ end
@@ -0,0 +1,23 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_support/core_ext/integer/time"
4
+
5
+ module ActiveRecordProxyAdapters
6
+ # Provides a global configuration object to configure how the proxy should behave.
7
+ class Configuration
8
+ PROXY_DELAY = 2.seconds.freeze
9
+ CHECKOUT_TIMEOUT = 2.seconds.freeze
10
+
11
+ # @return [ActiveSupport::Duration] How long the proxy should reroute all read requests to the primary database
12
+ # since the latest write. Defaults to PROXY_DELAY.
13
+ attr_accessor :proxy_delay
14
+ # @return [ActiveSupport::Duration] How long the proxy should wait for a connection from the replica pool.
15
+ # Defaults to CHECKOUT_TIMEOUT.
16
+ attr_accessor :checkout_timeout
17
+
18
+ def initialize
19
+ self.proxy_delay = PROXY_DELAY
20
+ self.checkout_timeout = CHECKOUT_TIMEOUT
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,35 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record/connection_adapters/postgresql_proxy_adapter"
4
+
5
+ module ActiveRecordProxyAdapters
6
+ # Module to extend ActiveRecord::Base with the connection handling methods.
7
+ # Required to make adapter work in ActiveRecord versions <= 7.2.x
8
+ module ConnectionHandling
9
+ def postgresql_proxy_adapter_class
10
+ ::ActiveRecord::ConnectionAdapters::PostgreSQLProxyAdapter
11
+ end
12
+
13
+ # This method is a copy and paste from Rails' postgresql_connection,
14
+ # replacing PostgreSQLAdapter by PostgreSQLProxyAdapter
15
+ # This is required by ActiveRecord versions <= 7.2.x to establish a connection using the adapter.
16
+ def postgresql_proxy_connection(config) # rubocop:disable Metrics/AbcSize,Metrics/MethodLength
17
+ conn_params = config.symbolize_keys.compact
18
+
19
+ # Map ActiveRecords param names to PGs.
20
+ conn_params[:user] = conn_params.delete(:username) if conn_params[:username]
21
+ conn_params[:dbname] = conn_params.delete(:database) if conn_params[:database]
22
+
23
+ # Forward only valid config params to PG::Connection.connect.
24
+ valid_conn_param_keys = PG::Connection.conndefaults_hash.keys + [:requiressl]
25
+ conn_params.slice!(*valid_conn_param_keys)
26
+
27
+ postgresql_proxy_adapter_class.new(
28
+ postgresql_proxy_adapter_class.new_client(conn_params),
29
+ logger,
30
+ conn_params,
31
+ config
32
+ )
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,42 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record/tasks/postgresql_proxy_database_tasks"
4
+ require "active_record/connection_adapters/postgresql_adapter"
5
+ require "active_record_proxy_adapters/primary_replica_proxy"
6
+
7
+ module ActiveRecordProxyAdapters
8
+ # Defines mixins to delegate specific methods from the proxy to the adapter.
9
+ module Hijackable
10
+ extend ActiveSupport::Concern
11
+
12
+ class_methods do
13
+ # Renames the methods from the original Adapter using the proxy suffix (_unproxied)
14
+ # and delegate the original method name to the proxy.
15
+ # Example: delegate_to_proxy(:execute) creates a method `execute_unproxied`,
16
+ # while delegating :execute to the proxy.
17
+ # @param method_name [Array<Symbol>] the names of methods to be redefined.
18
+ def delegate_to_proxy(*method_names)
19
+ method_names.each do |method_name|
20
+ proxy_method_name = proxy_method_name_for(method_name)
21
+ proxy_method_private = private_method_defined?(method_name)
22
+
23
+ # some adapter methods are private. We need to make them public before aliasing.
24
+ public method_name if proxy_method_private
25
+
26
+ alias_method proxy_method_name, method_name
27
+
28
+ # If adapter method was originally private. We now make them private again.
29
+ private method_name, proxy_method_name if proxy_method_private
30
+ end
31
+
32
+ delegate(*method_names, to: :proxy)
33
+ end
34
+
35
+ private
36
+
37
+ def proxy_method_name_for(method_name)
38
+ :"#{method_name}#{ActiveRecordProxyAdapters::PrimaryReplicaProxy::UNPROXIED_METHOD_SUFFIX}"
39
+ end
40
+ end
41
+ end
42
+ end
@@ -0,0 +1,14 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record_proxy_adapters/primary_replica_proxy"
4
+ require "active_record_proxy_adapters/active_record_context"
5
+
6
+ module ActiveRecordProxyAdapters
7
+ # Proxy to the original PostgreSQLAdapter, allowing the use of the ActiveRecordProxyAdapters::PrimaryReplicaProxy.
8
+ class PostgreSQLProxy < PrimaryReplicaProxy
9
+ # ActiveRecord::PostgreSQLAdapter methods that should be proxied.
10
+ hijack_method :execute, :exec_query
11
+
12
+ hijack_method :exec_no_cache, :exec_cache unless ActiveRecordContext.active_record_v8_0_or_greater?
13
+ end
14
+ end
@@ -0,0 +1,190 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record_proxy_adapters/configuration"
4
+ require "active_support/core_ext/module/delegation"
5
+ require "active_support/core_ext/object/blank"
6
+ require "concurrent-ruby"
7
+ require "active_record_proxy_adapters/active_record_context"
8
+
9
+ module ActiveRecordProxyAdapters
10
+ # This is the base class for all proxies. It defines the methods that should be proxied
11
+ # and the logic to determine which database to use.
12
+ class PrimaryReplicaProxy # rubocop:disable Metrics/ClassLength
13
+ # All queries that match these patterns should be sent to the primary database
14
+ SQL_PRIMARY_MATCHERS = [
15
+ /\A\s*select.+for update\Z/i, /select.+lock in share mode\Z/i,
16
+ /\A\s*select.+(nextval|currval|lastval|get_lock|release_lock|pg_advisory_lock|pg_advisory_unlock)\(/i
17
+ ].map(&:freeze).freeze
18
+ # All queries that match these patterns should be sent to the replica database
19
+ SQL_REPLICA_MATCHERS = [/\A\s*(select|with.+\)\s*select)\s/i].map(&:freeze).freeze
20
+ # All queries that match these patterns should be sent to all databases
21
+ SQL_ALL_MATCHERS = [/\A\s*set\s/i].map(&:freeze).freeze
22
+ # Local sets queries should not be sent to all datbases
23
+ SQL_SKIP_ALL_MATCHERS = [/\A\s*set\s+local\s/i].map(&:freeze).freeze
24
+ # These patterns define which database statments are considered write statments, so we can shortly re-route all
25
+ # requests to the primary database so the replica has time to replicate
26
+ WRITE_STATEMENT_MATCHERS = [/\ABEGIN/i, /\ACOMMIT/i, /INSERT\sINTO\s/i, /UPDATE\s/i, /DELETE\sFROM\s/i,
27
+ /DROP\s/i].map(&:freeze).freeze
28
+ UNPROXIED_METHOD_SUFFIX = "_unproxied"
29
+
30
+ # Defines which methods should be hijacked from the original adapter and use the proxy
31
+ # @param method_names [Array<Symbol>] the list of method names from the adapter
32
+ def self.hijack_method(*method_names) # rubocop:disable Metrics/MethodLength
33
+ @hijacked_methods ||= Set.new
34
+ @hijacked_methods += Set.new(method_names)
35
+
36
+ method_names.each do |method_name|
37
+ define_method(method_name) do |*args, **kwargs, &block|
38
+ proxy_bypass_method = "#{method_name}#{UNPROXIED_METHOD_SUFFIX}"
39
+ sql_string = coerce_query_to_string(args.first)
40
+
41
+ appropriate_connection(sql_string) do |conn|
42
+ method_to_call = conn == primary_connection ? proxy_bypass_method : method_name
43
+
44
+ conn.send(method_to_call, *args, **kwargs, &block)
45
+ end
46
+ end
47
+ end
48
+ end
49
+
50
+ def self.hijacked_methods
51
+ @hijacked_methods.to_a
52
+ end
53
+
54
+ # @param primary_connection [ActiveRecord::ConnectionAdatpers::AbstractAdapter]
55
+ def initialize(primary_connection)
56
+ @primary_connection = primary_connection
57
+ @last_write_at = 0
58
+ @active_record_context = ActiveRecordContext.new
59
+ end
60
+
61
+ private
62
+
63
+ attr_reader :primary_connection, :last_write_at, :active_record_context
64
+
65
+ delegate :connected_to_stack, to: :connection_class
66
+ delegate :reading_role, :writing_role, to: :active_record_context
67
+
68
+ def connection_class
69
+ active_record_context.connection_class_for(primary_connection)
70
+ end
71
+
72
+ def replica_pool
73
+ ActiveRecord::Base.connected_to(role: reading_role) { ActiveRecord::Base.connection_pool }
74
+ end
75
+
76
+ def coerce_query_to_string(sql_or_arel)
77
+ sql_or_arel.respond_to?(:to_sql) ? sql_or_arel.to_sql : sql_or_arel.to_s
78
+ end
79
+
80
+ def appropriate_connection(sql_string, &block)
81
+ roles_for(sql_string).map do |role|
82
+ connection_for(role, sql_string) do |connection|
83
+ block.call(connection)
84
+ end
85
+ end.last
86
+ end
87
+
88
+ def roles_for(sql_string)
89
+ return [top_of_connection_stack_role] if top_of_connection_stack_role.present?
90
+
91
+ if need_all?(sql_string)
92
+ [reading_role, writing_role]
93
+ elsif need_primary?(sql_string)
94
+ [writing_role]
95
+ else
96
+ [reading_role]
97
+ end
98
+ end
99
+
100
+ def top_of_connection_stack_role
101
+ return if connected_to_stack.empty?
102
+
103
+ top = connected_to_stack.last
104
+ role = top[:role]
105
+ return unless role.present?
106
+
107
+ [reading_role, writing_role].include?(role) ? role : nil
108
+ end
109
+
110
+ def connection_for(role, sql_string) # rubocop:disable Metrics/MethodLength
111
+ connection = if role == writing_role
112
+ primary_connection
113
+ else
114
+ begin
115
+ replica_pool.checkout(checkout_timeout)
116
+ # rescue NoDatabaseError to avoid crashing when running db:create rake task
117
+ # rescue ConnectionNotEstablished to handle connectivity issues in the replica
118
+ # (for example, replication delay)
119
+ rescue ActiveRecord::NoDatabaseError, ActiveRecord::ConnectionNotEstablished
120
+ primary_connection
121
+ end
122
+ end
123
+
124
+ result = yield(connection)
125
+ update_primary_latest_write_timestamp if !replica_connection?(connection) && write_statement?(sql_string)
126
+
127
+ result
128
+ ensure
129
+ replica_connection?(connection) && replica_pool.checkin(connection)
130
+ end
131
+
132
+ def replica_connection?(connection)
133
+ connection != primary_connection
134
+ end
135
+
136
+ # @return [TrueClass] if there has been a write within the last {#proxy_delay} seconds
137
+ # @return [TrueClass] if sql_string matches a write statement (i.e. INSERT, UPDATE, DELETE, SELECT FOR UPDATE)
138
+ # @return [FalseClass] if sql_string matches a read statement (i.e. SELECT)
139
+ def need_primary?(sql_string)
140
+ return true if recent_write_to_primary?
141
+
142
+ return true if in_transaction?
143
+ return true if SQL_PRIMARY_MATCHERS.any?(&match_sql?(sql_string))
144
+ return false if SQL_REPLICA_MATCHERS.any?(&match_sql?(sql_string))
145
+
146
+ true
147
+ end
148
+
149
+ def need_all?(sql_string)
150
+ return false if SQL_SKIP_ALL_MATCHERS.any?(&match_sql?(sql_string))
151
+
152
+ SQL_ALL_MATCHERS.any?(&match_sql?(sql_string))
153
+ end
154
+
155
+ def write_statement?(sql_string)
156
+ WRITE_STATEMENT_MATCHERS.any?(&match_sql?(sql_string))
157
+ end
158
+
159
+ def match_sql?(sql_string)
160
+ proc { |matcher| matcher.match?(sql_string) }
161
+ end
162
+
163
+ # TODO: implement a context API to re-route requests to the primary database if a recent query was sent to it to
164
+ # avoid replication delay issues
165
+ # @return Boolean
166
+ def recent_write_to_primary?
167
+ Concurrent.monotonic_time - last_write_at < proxy_delay
168
+ end
169
+
170
+ def in_transaction?
171
+ primary_connection.open_transactions.positive?
172
+ end
173
+
174
+ def update_primary_latest_write_timestamp
175
+ @last_write_at = Concurrent.monotonic_time
176
+ end
177
+
178
+ def proxy_delay
179
+ proxy_config.proxy_delay
180
+ end
181
+
182
+ def checkout_timeout
183
+ proxy_config.checkout_timeout
184
+ end
185
+
186
+ def proxy_config
187
+ ActiveRecordProxyAdapters.config
188
+ end
189
+ end
190
+ end
@@ -0,0 +1,21 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_support"
4
+
5
+ module ActiveRecordProxyAdapters
6
+ # Hooks into rails boot process to extend ActiveRecord with the proxy adapter.
7
+ class Railtie < Rails::Railtie
8
+ ActiveSupport.on_load :active_record do
9
+ require "active_record_proxy_adapters/connection_handling"
10
+ ActiveRecord::Base.extend(ActiveRecordProxyAdapters::ConnectionHandling)
11
+ end
12
+
13
+ config.to_prepare do
14
+ Rails.autoloaders.each do |autoloader|
15
+ autoloader.inflector.inflect(
16
+ "postgresql_proxy_adapter" => "PostgreSQLProxyAdapter"
17
+ )
18
+ end
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveRecordProxyAdapters
4
+ VERSION = "0.1.1"
5
+ end
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record"
4
+ require "active_record_proxy_adapters/version"
5
+ require "active_record_proxy_adapters/configuration"
6
+
7
+ # The gem namespace.
8
+ module ActiveRecordProxyAdapters
9
+ class Error < StandardError; end
10
+
11
+ module_function
12
+
13
+ def configure
14
+ yield(config)
15
+ end
16
+
17
+ def config
18
+ @config ||= Configuration.new
19
+ end
20
+ end
21
+
22
+ require_relative "active_record_proxy_adapters/railtie" if defined?(Rails::Railtie)
@@ -0,0 +1,34 @@
1
+ FROM docker.io/postgres:14-alpine
2
+
3
+ ARG REPLICA_USER=replicator
4
+ ARG REPLICA_PASSWORD=replicator
5
+ ARG REPLICATION_SLOT_NAME=replication_slot
6
+ ARG INIT_SQL=00_init.sql
7
+ ARG POSTGRES_LOGGING_COLLECTOR=
8
+ ARG POSTGRES_LOG_DESTINATION=
9
+ ARG POSTGRES_LOG_STATEMENT=
10
+ ENV CONF_SAMPLE="/usr/local/share/postgresql/postgresql.conf.sample"
11
+
12
+ WORKDIR /docker-entrypoint-initdb.d
13
+
14
+ USER root
15
+
16
+ RUN touch $INIT_SQL
17
+ RUN chown -R postgres:postgres $INIT_SQL
18
+ RUN echo "CREATE USER ${REPLICA_USER} WITH REPLICATION ENCRYPTED PASSWORD '${REPLICA_PASSWORD}';" > $INIT_SQL
19
+ RUN echo "SELECT pg_create_physical_replication_slot('${REPLICATION_SLOT_NAME}');" >> $INIT_SQL
20
+
21
+ # Enable logging collector if given
22
+ RUN if [[ ! -z "${POSTGRES_LOGGING_COLLECTOR}" ]]; then sed -i "s/#\(logging_collector = \)off\(.*\)/\1${POSTGRES_LOGGING_COLLECTOR}\2/" ${CONF_SAMPLE}; fi
23
+
24
+ # Override default log destination if given
25
+ RUN if [[ ! -z "${POSTGRES_LOG_DESTINATION}" ]]; then sed -i "s/#\(log_destination = \)'stderr'\(.*\)/\1'${POSTGRES_LOG_DESTINATION}'\2/" ${CONF_SAMPLE}; fi
26
+
27
+ # Override log statement if given
28
+ RUN if [[ ! -z "${POSTGRES_LOG_STATEMENT}" ]]; then sed -i "s/#\(log_statement = \)'none'\(.*\)/\1'${POSTGRES_LOG_STATEMENT}'\2/" ${CONF_SAMPLE}; fi
29
+
30
+ WORKDIR /
31
+
32
+ USER postgres
33
+
34
+ CMD ["postgres", "-c", "wal_level=replica", "-c", "hot_standby=on", "-c", "max_wal_senders=10", "-c", "max_replication_slots=10", "-c", "hot_standby_feedback=on" ]
@@ -0,0 +1,23 @@
1
+ FROM docker.io/postgres:14-alpine
2
+
3
+ ENV PRIMARY_DATABASE_HOST=localhost
4
+ ENV PRIMARY_DATABASE_PORT=5432
5
+ ENV PRIMARY_REPLICATION_SLOT=replication_slot
6
+
7
+ USER root
8
+ RUN printf '' > cmd.sh
9
+
10
+ RUN echo 'until pg_basebackup --pgdata=/var/lib/postgresql/data -R --slot=$PRIMARY_REPLICATION_SLOT --host=$PRIMARY_DATABASE_HOST --port=$PRIMARY_DATABASE_PORT' >> cmd.sh
11
+ RUN echo 'do' >> cmd.sh
12
+ RUN echo "echo 'Waiting for primary to connect...'" >> cmd.sh
13
+ RUN echo 'sleep 1s' >> cmd.sh
14
+ RUN echo 'done' >> cmd.sh
15
+ RUN echo "echo 'Backup done, starting replica...'" >> cmd.sh
16
+ RUN echo 'chmod 0700 /var/lib/postgresql/data' >> cmd.sh
17
+ RUN echo 'postgres' >> cmd.sh
18
+
19
+ RUN chown -R postgres:postgres cmd.sh
20
+ USER postgres
21
+ RUN chmod u+rwx cmd.sh
22
+
23
+ CMD [ "./cmd.sh" ]
@@ -0,0 +1,4 @@
1
+ module ActiveRecordProxyAdapters
2
+ VERSION: String
3
+ # See the writing guide of rbs: https://github.com/ruby/rbs#guides
4
+ end
metadata ADDED
@@ -0,0 +1,114 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: active_record_proxy_adapters
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.1
5
+ platform: ruby
6
+ authors:
7
+ - Matt Cruz
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2024-11-28 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: activerecord
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: 7.0.0
20
+ - - "<"
21
+ - !ruby/object:Gem::Version
22
+ version: '8.1'
23
+ type: :runtime
24
+ prerelease: false
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - ">="
28
+ - !ruby/object:Gem::Version
29
+ version: 7.0.0
30
+ - - "<"
31
+ - !ruby/object:Gem::Version
32
+ version: '8.1'
33
+ - !ruby/object:Gem::Dependency
34
+ name: activesupport
35
+ requirement: !ruby/object:Gem::Requirement
36
+ requirements:
37
+ - - ">="
38
+ - !ruby/object:Gem::Version
39
+ version: 7.0.0
40
+ - - "<"
41
+ - !ruby/object:Gem::Version
42
+ version: '8.1'
43
+ type: :runtime
44
+ prerelease: false
45
+ version_requirements: !ruby/object:Gem::Requirement
46
+ requirements:
47
+ - - ">="
48
+ - !ruby/object:Gem::Version
49
+ version: 7.0.0
50
+ - - "<"
51
+ - !ruby/object:Gem::Version
52
+ version: '8.1'
53
+ description: |-
54
+ This gem allows automatic connection switching between a primary and one read replica database in ActiveRecord.
55
+ It pattern matches the SQL statement being sent to decide whether it should go to the replica (SELECT) or the
56
+ primary (INSERT, UPDATE, DELETE).
57
+ email:
58
+ - matt.cruz@nasdaq.com
59
+ executables: []
60
+ extensions: []
61
+ extra_rdoc_files: []
62
+ files:
63
+ - ".rspec"
64
+ - ".rubocop.yml"
65
+ - CHANGELOG.md
66
+ - CODE_OF_CONDUCT.md
67
+ - Dockerfile
68
+ - LICENSE.txt
69
+ - README.md
70
+ - Rakefile
71
+ - docker-compose.yml
72
+ - lib/active_record/connection_adapters/postgresql_proxy_adapter.rb
73
+ - lib/active_record/tasks/postgresql_proxy_database_tasks.rb
74
+ - lib/active_record_proxy_adapters.rb
75
+ - lib/active_record_proxy_adapters/active_record_context.rb
76
+ - lib/active_record_proxy_adapters/configuration.rb
77
+ - lib/active_record_proxy_adapters/connection_handling.rb
78
+ - lib/active_record_proxy_adapters/hijackable.rb
79
+ - lib/active_record_proxy_adapters/postgresql_proxy.rb
80
+ - lib/active_record_proxy_adapters/primary_replica_proxy.rb
81
+ - lib/active_record_proxy_adapters/railtie.rb
82
+ - lib/active_record_proxy_adapters/version.rb
83
+ - postgres_primary.dockerfile
84
+ - postgres_replica.dockerfile
85
+ - sig/active_record_proxy_adapters.rbs
86
+ homepage: https://github.com/Nasdaq/active_record_proxy_adapters
87
+ licenses:
88
+ - MIT
89
+ metadata:
90
+ allowed_push_host: https://rubygems.org/
91
+ homepage_uri: https://github.com/Nasdaq/active_record_proxy_adapters
92
+ source_code_uri: https://github.com/Nasdaq/active_record_proxy_adapters
93
+ changelog_uri: https://github.com/Nasdaq/active_record_proxy_adapters/blob/main/CHANGELOG.md
94
+ rubygems_mfa_required: 'true'
95
+ post_install_message:
96
+ rdoc_options: []
97
+ require_paths:
98
+ - lib
99
+ required_ruby_version: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: 3.1.0
104
+ required_rubygems_version: !ruby/object:Gem::Requirement
105
+ requirements:
106
+ - - ">="
107
+ - !ruby/object:Gem::Version
108
+ version: '0'
109
+ requirements: []
110
+ rubygems_version: 3.5.11
111
+ signing_key:
112
+ specification_version: 4
113
+ summary: Read replica proxy adapters for ActiveRecord!
114
+ test_files: []