active_record_proxy_adapters 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +19 -8
- data/Dockerfile +3 -1
- data/README.md +38 -7
- data/db/mysql_structure.sql +41 -0
- data/db/postgresql_structure.sql +16 -6
- data/docker-compose.yml +23 -3
- data/lib/active_record/connection_adapters/mysql2_proxy_adapter.rb +39 -0
- data/lib/active_record/tasks/mysql2_proxy_database_tasks.rb +19 -0
- data/lib/active_record/tasks/postgresql_proxy_database_tasks.rb +3 -29
- data/lib/active_record_proxy_adapters/connection_handling.rb +26 -0
- data/lib/active_record_proxy_adapters/database_tasks.rb +39 -0
- data/lib/active_record_proxy_adapters/hijackable.rb +32 -3
- data/lib/active_record_proxy_adapters/mysql2_proxy.rb +9 -0
- data/lib/active_record_proxy_adapters/postgresql_proxy.rb +0 -2
- data/lib/active_record_proxy_adapters/primary_replica_proxy.rb +27 -22
- data/lib/active_record_proxy_adapters/version.rb +1 -1
- data/postgres_primary.dockerfile +1 -1
- data/postgres_replica.dockerfile +1 -1
- metadata +7 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1f614128aa280355a9bc8c83e29805fc36c81f8389c017489c848e075a5510c3
|
|
4
|
+
data.tar.gz: d28585b2d7d1f7645dfc89007890a2c35ee19c4acc6de1b850e4f3b792cc0c07
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b3e36b3fe27bce5b1bdb9035fc399362f99d25062d97abc9024c02a30a8c6a328a2de3bdacda5682622c812f38f8b4a1da48d76e687131d3a6e07e44be4c6995
|
|
7
|
+
data.tar.gz: 053d2cfdea1db5699cd18404e8b22524a826012534eb0e077b307102021adeccde565f9e7b6417dbbe053669464a7220ab3e1e2cb3b92201cc0b296e8f23f5ce
|
data/CHANGELOG.md
CHANGED
|
@@ -1,30 +1,41 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
- Add Mysql2ProxyAdapter
|
|
4
|
+
|
|
5
|
+
## [0.2.2, 0.1.5] - 2025-01-02
|
|
6
|
+
|
|
7
|
+
- Handle PendingMigrationConnection introduced by Rails 7.2 and backported to Rails 7.1 https://github.com/Nasdaq/active_record_proxy_adapters/commit/793562694c05d554bad6e14637b34e5f9ffd2fc5
|
|
8
|
+
- Stick to same connection throughout request span https://github.com/Nasdaq/active_record_proxy_adapters/commit/789742fd7a33ecd555a995e8a1e1336455caec75
|
|
9
|
+
|
|
3
10
|
## [0.2.1] - 2025-01-02
|
|
4
11
|
|
|
5
|
-
- Fix replica connection pool getter when specific connection name is not found
|
|
12
|
+
- Fix replica connection pool getter when specific connection name is not found https://github.com/Nasdaq/active_record_proxy_adapters/commit/847e150dd21c5bc619745ee1d9d8fcaa9b8f2eea
|
|
6
13
|
|
|
7
14
|
## [0.2.0] - 2024-12-24
|
|
8
15
|
|
|
9
|
-
- Add custom log subscriber to tag queries based on the adapter being used
|
|
16
|
+
- Add custom log subscriber to tag queries based on the adapter being used https://github.com/Nasdaq/active_record_proxy_adapters/commit/68b8c1f4191388eb957bf12e0f84289da667e940
|
|
17
|
+
|
|
18
|
+
## [0.1.4] - 2025-01-02
|
|
19
|
+
|
|
20
|
+
- Fix replica connection pool getter when specific connection name is not found https://github.com/Nasdaq/active_record_proxy_adapters/commit/88b32a282b54d420e652f638656dbcf063ac8796
|
|
10
21
|
|
|
11
22
|
## [0.1.3] - 2024-12-24
|
|
12
23
|
|
|
13
|
-
- Fix replica connection pool getter when database configurations have multiple replicas
|
|
14
|
-
- Retrieve replica pool without checking out a connection
|
|
24
|
+
- Fix replica connection pool getter when database configurations have multiple replicas https://github.com/Nasdaq/active_record_proxy_adapters/commit/ea5a33997da45ac073f166b3fbd2d12426053cd6
|
|
25
|
+
- Retrieve replica pool without checking out a connection https://github.com/Nasdaq/active_record_proxy_adapters/commit/6470ef58e851082ae1f7a860ecdb5b451ef903c8
|
|
15
26
|
|
|
16
27
|
## [0.1.2] - 2024-12-16
|
|
17
28
|
|
|
18
|
-
- Fix CTE regex matcher
|
|
29
|
+
- Fix CTE regex matcher https://github.com/Nasdaq/active_record_proxy_adapters/commit/4b1d10bfd952fb1f5b102de8cc1a5bd05d25f5e9
|
|
19
30
|
|
|
20
31
|
## [0.1.1] - 2024-11-27
|
|
21
32
|
|
|
22
|
-
- Enable RubyGems MFA
|
|
33
|
+
- Enable RubyGems MFA https://github.com/Nasdaq/active_record_proxy_adapters/commit/2a71b1f4354fb966cc0aa68231ca5837814e07ee
|
|
23
34
|
|
|
24
35
|
## [0.1.0] - 2024-11-19
|
|
25
36
|
|
|
26
|
-
- Add PostgreSQLProxyAdapter
|
|
37
|
+
- Add PostgreSQLProxyAdapter https://github.com/Nasdaq/active_record_proxy_adapters/commit/2b3bb9f7359139519b32af3018ceb07fed8c6b33
|
|
27
38
|
|
|
28
39
|
## [0.1.0.rc2] - 2024-10-28
|
|
29
40
|
|
|
30
|
-
- Add PostgreSQLProxyAdapter
|
|
41
|
+
- Add PostgreSQLProxyAdapter https://github.com/Nasdaq/active_record_proxy_adapters/commit/2b3bb9f7359139519b32af3018ceb07fed8c6b33
|
data/Dockerfile
CHANGED
data/README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# ActiveRecordProxyAdapters
|
|
2
2
|
|
|
3
|
+
[](https://github.com/Nasdaq/active_record_proxy_adapters/actions/workflows/test.yml)
|
|
4
|
+
|
|
3
5
|
A set of ActiveRecord adapters that leverage Rails native multiple database setup to allow automatic connection switching from _one_ primary pool to _one_ replica pool at the database statement level.
|
|
4
6
|
|
|
5
7
|
## Why do I need this?
|
|
@@ -31,8 +33,19 @@ If bundler is not being used to manage dependencies, install the gem by executin
|
|
|
31
33
|
|
|
32
34
|
### On Rails
|
|
33
35
|
|
|
34
|
-
In `config/database.yml`, use `
|
|
36
|
+
In `config/database.yml`, use `{your_database_adapter}_proxy` as the adapter for the `primary` database, and keep `{your_database_adapter}` for the replica database.
|
|
37
|
+
|
|
38
|
+
Currently supported adapters:
|
|
39
|
+
|
|
40
|
+
- `postgresql`
|
|
41
|
+
- `mysql2`
|
|
42
|
+
|
|
43
|
+
Coming soon:
|
|
44
|
+
- `trilogy`
|
|
45
|
+
- `sqlite`
|
|
46
|
+
|
|
35
47
|
|
|
48
|
+
#### PostgreSQL
|
|
36
49
|
```yaml
|
|
37
50
|
# config/database.yml
|
|
38
51
|
development:
|
|
@@ -46,6 +59,20 @@ development:
|
|
|
46
59
|
# your replica credentials here
|
|
47
60
|
```
|
|
48
61
|
|
|
62
|
+
#### MySQL
|
|
63
|
+
```yaml
|
|
64
|
+
# config/database.yml
|
|
65
|
+
development:
|
|
66
|
+
primary:
|
|
67
|
+
adapter: mysql2_proxy
|
|
68
|
+
# your primary credentials here
|
|
69
|
+
|
|
70
|
+
primary_replica:
|
|
71
|
+
adapter: mysql2
|
|
72
|
+
replica: true
|
|
73
|
+
# your replica credentials here
|
|
74
|
+
```
|
|
75
|
+
|
|
49
76
|
```ruby
|
|
50
77
|
# app/models/application_record.rb
|
|
51
78
|
class ApplicationRecord < ActiveRecord::Base
|
|
@@ -88,7 +115,7 @@ end
|
|
|
88
115
|
|
|
89
116
|
## Configuration
|
|
90
117
|
|
|
91
|
-
The gem comes preconfigured out of the box. However, if default configuration does not suit your needs, you can modify
|
|
118
|
+
The gem comes preconfigured out of the box. However, if default configuration does not suit your needs, you can modify it by using a `.configure` block:
|
|
92
119
|
|
|
93
120
|
```ruby
|
|
94
121
|
# config/initializers/active_record_proxy_adapters.rb
|
|
@@ -237,6 +264,7 @@ end
|
|
|
237
264
|
|
|
238
265
|
# app/models/portal.rb
|
|
239
266
|
class Portal < ApplicationRecord
|
|
267
|
+
validates :name, uniqueness: true
|
|
240
268
|
end
|
|
241
269
|
|
|
242
270
|
# in rails console -e test
|
|
@@ -244,13 +272,17 @@ ActiveRecord::Base.logger.formatter = proc do |_severity, _time, _progname, msg|
|
|
|
244
272
|
"[#{Time.current.iso8601} THREAD #{Thread.current[:name]}] #{msg}\n"
|
|
245
273
|
end
|
|
246
274
|
|
|
275
|
+
ActiveRecordProxyAdapters.configure do |config|
|
|
276
|
+
config.proxy_delay = 2.seconds
|
|
277
|
+
end
|
|
278
|
+
|
|
247
279
|
def read_your_own_writes
|
|
248
280
|
proc do
|
|
249
281
|
Portal.all.count # should go to the replica
|
|
250
|
-
|
|
282
|
+
Portal.create(name: 'Read your own write')
|
|
251
283
|
|
|
252
284
|
5.times do
|
|
253
|
-
Portal.all.count # first one goes the primary, last
|
|
285
|
+
Portal.all.count # first one goes the primary, last 4 should go to the replica
|
|
254
286
|
sleep(3)
|
|
255
287
|
end
|
|
256
288
|
end
|
|
@@ -295,9 +327,8 @@ irb(main):051:0> test_multithread_queries
|
|
|
295
327
|
[2024-12-24T13:52:40-05:00 THREAD USE REPLICA] [PostgreSQL Replica] Portal Count (1.4ms) SELECT COUNT(*) FROM "portals"
|
|
296
328
|
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQL Replica] Portal Count (0.4ms) SELECT COUNT(*) FROM "portals"
|
|
297
329
|
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] TRANSACTION (0.5ms) BEGIN
|
|
298
|
-
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] Portal Exists? (
|
|
299
|
-
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] Portal
|
|
300
|
-
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] Portal Create (0.8ms) INSERT INTO "portals" ("name", "slug", "logo", "created_at", "updated_at", "visible") VALUES ($1, $2, $3, $4, $5, $6) RETURNING "id" [["name", "Portal-e065948fbbee73d3b2c576b48c2b37e021115158edc6a92390d613640460e1d4"], ["slug", "portal-e065948fbbee73d3b2c576b48c2b37e021115158edc6a92390d613640460e1d4"], ["logo", nil], ["created_at", "2024-12-24 18:52:40.428383"], ["updated_at", "2024-12-24 18:52:40.428383"], ["visible", true]]
|
|
330
|
+
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] Portal Exists? (0.4ms) SELECT 1 AS one FROM "portals" WHERE "portals"."name" = $1 LIMIT $2 [["name", "Read your own write"], ["LIMIT", 1]]
|
|
331
|
+
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] Portal Create (0.8ms) INSERT INTO "portals" ("name", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["name", "Read your own write"], ["created_at", "2024-12-24 18:52:40.428383"], ["updated_at", "2024-12-24 18:52:40.428383"]]
|
|
301
332
|
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] TRANSACTION (0.7ms) COMMIT
|
|
302
333
|
[2024-12-24T13:52:40-05:00 THREAD READ YOUR OWN WRITES] [PostgreSQLProxy Primary] Portal Count (0.6ms) SELECT COUNT(*) FROM "portals"
|
|
303
334
|
[2024-12-24T13:52:41-05:00 THREAD USE REPLICA] [PostgreSQL Replica] Portal Count (4.4ms) SELECT COUNT(*) FROM "portals"
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
/*M!999999\- enable the sandbox mode */
|
|
2
|
+
|
|
3
|
+
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
|
|
4
|
+
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
|
|
5
|
+
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
|
|
6
|
+
/*!40101 SET NAMES utf8mb4 */;
|
|
7
|
+
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
|
|
8
|
+
/*!40103 SET TIME_ZONE='+00:00' */;
|
|
9
|
+
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
|
|
10
|
+
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
|
|
11
|
+
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
|
|
12
|
+
/*M!100616 SET @OLD_NOTE_VERBOSITY=@@NOTE_VERBOSITY, NOTE_VERBOSITY=0 */;
|
|
13
|
+
DROP TABLE IF EXISTS `schema_migrations`;
|
|
14
|
+
/*!40101 SET @saved_cs_client = @@character_set_client */;
|
|
15
|
+
/*!40101 SET character_set_client = utf8 */;
|
|
16
|
+
CREATE TABLE `schema_migrations` (
|
|
17
|
+
`version` varchar(255) NOT NULL
|
|
18
|
+
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci;
|
|
19
|
+
/*!40101 SET character_set_client = @saved_cs_client */;
|
|
20
|
+
DROP TABLE IF EXISTS `users`;
|
|
21
|
+
/*!40101 SET @saved_cs_client = @@character_set_client */;
|
|
22
|
+
/*!40101 SET character_set_client = utf8 */;
|
|
23
|
+
CREATE TABLE `users` (
|
|
24
|
+
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
|
|
25
|
+
`name` text NOT NULL,
|
|
26
|
+
`email` text NOT NULL,
|
|
27
|
+
`created_at` timestamp NULL DEFAULT current_timestamp(),
|
|
28
|
+
`updated_at` timestamp NULL DEFAULT current_timestamp(),
|
|
29
|
+
PRIMARY KEY (`id`)
|
|
30
|
+
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci;
|
|
31
|
+
/*!40101 SET character_set_client = @saved_cs_client */;
|
|
32
|
+
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
|
|
33
|
+
|
|
34
|
+
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
|
|
35
|
+
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
|
|
36
|
+
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
|
|
37
|
+
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
|
|
38
|
+
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
|
|
39
|
+
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
|
|
40
|
+
/*M!100616 SET NOTE_VERBOSITY=@OLD_NOTE_VERBOSITY */;
|
|
41
|
+
|
data/db/postgresql_structure.sql
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
SET statement_timeout = 0;
|
|
2
2
|
SET lock_timeout = 0;
|
|
3
3
|
SET idle_in_transaction_session_timeout = 0;
|
|
4
|
+
SET transaction_timeout = 0;
|
|
4
5
|
SET client_encoding = 'UTF8';
|
|
5
6
|
SET standard_conforming_strings = on;
|
|
6
7
|
SELECT pg_catalog.set_config('search_path', '', false);
|
|
@@ -9,16 +10,18 @@ SET xmloption = content;
|
|
|
9
10
|
SET client_min_messages = warning;
|
|
10
11
|
SET row_security = off;
|
|
11
12
|
|
|
12
|
-
|
|
13
|
-
-- Name: public; Type: SCHEMA; Schema: -; Owner: -
|
|
14
|
-
--
|
|
13
|
+
SET default_tablespace = '';
|
|
15
14
|
|
|
16
|
-
|
|
15
|
+
SET default_table_access_method = heap;
|
|
17
16
|
|
|
17
|
+
--
|
|
18
|
+
-- Name: schema_migrations; Type: TABLE; Schema: public; Owner: -
|
|
19
|
+
--
|
|
18
20
|
|
|
19
|
-
|
|
21
|
+
CREATE TABLE public.schema_migrations (
|
|
22
|
+
version character varying(255) NOT NULL
|
|
23
|
+
);
|
|
20
24
|
|
|
21
|
-
SET default_table_access_method = heap;
|
|
22
25
|
|
|
23
26
|
--
|
|
24
27
|
-- Name: users; Type: TABLE; Schema: public; Owner: -
|
|
@@ -68,6 +71,13 @@ ALTER TABLE ONLY public.users
|
|
|
68
71
|
ADD CONSTRAINT users_pkey PRIMARY KEY (id);
|
|
69
72
|
|
|
70
73
|
|
|
74
|
+
--
|
|
75
|
+
-- Name: unique_schema_migrations; Type: INDEX; Schema: public; Owner: -
|
|
76
|
+
--
|
|
77
|
+
|
|
78
|
+
CREATE UNIQUE INDEX unique_schema_migrations ON public.schema_migrations USING btree (version);
|
|
79
|
+
|
|
80
|
+
|
|
71
81
|
--
|
|
72
82
|
-- PostgreSQL database dump complete
|
|
73
83
|
--
|
data/docker-compose.yml
CHANGED
|
@@ -31,12 +31,22 @@ services:
|
|
|
31
31
|
PG_REPLICA_PASSWORD: postgres_primary_test
|
|
32
32
|
PG_REPLICA_HOST: postgres_replica
|
|
33
33
|
PG_REPLICA_PORT: 5432
|
|
34
|
+
MYSQL_PRIMARY_USER: root
|
|
35
|
+
MYSQL_PRIMARY_PASSWORD: mysql
|
|
36
|
+
MYSQL_PRIMARY_HOST: mysql_primary
|
|
37
|
+
MYSQL_PRIMARY_PORT: 3306
|
|
38
|
+
MYSQL_REPLICA_USER: root
|
|
39
|
+
MYSQL_REPLICA_PASSWORD: mysql
|
|
40
|
+
MYSQL_REPLICA_HOST: mysql_primary
|
|
41
|
+
MYSQL_REPLICA_PORT: 3306
|
|
34
42
|
depends_on:
|
|
35
43
|
- postgres_primary
|
|
36
44
|
- postgres_replica
|
|
45
|
+
- mariadb
|
|
37
46
|
networks:
|
|
38
47
|
- app
|
|
39
48
|
- postgres
|
|
49
|
+
- mariadb
|
|
40
50
|
volumes:
|
|
41
51
|
- .:/app
|
|
42
52
|
postgres_primary:
|
|
@@ -70,9 +80,19 @@ services:
|
|
|
70
80
|
PRIMARY_DATABASE_HOST: postgres_primary
|
|
71
81
|
depends_on:
|
|
72
82
|
- postgres_primary
|
|
83
|
+
# TODO: create mysql replica images
|
|
84
|
+
mariadb:
|
|
85
|
+
image: mariadb:11.4
|
|
86
|
+
container_name: mysql_primary
|
|
87
|
+
environment:
|
|
88
|
+
MARIADB_ROOT_PASSWORD: mysql
|
|
89
|
+
MARIADB_DATABASE: mysql
|
|
90
|
+
ports:
|
|
91
|
+
- "3306:3306"
|
|
92
|
+
networks:
|
|
93
|
+
- mariadb
|
|
94
|
+
|
|
73
95
|
networks:
|
|
74
96
|
app:
|
|
75
97
|
postgres:
|
|
76
|
-
|
|
77
|
-
volumes:
|
|
78
|
-
postgres_primary:
|
|
98
|
+
mariadb:
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_record/tasks/mysql2_proxy_database_tasks"
|
|
4
|
+
require "active_record/connection_adapters/mysql2_adapter"
|
|
5
|
+
require "active_record_proxy_adapters/active_record_context"
|
|
6
|
+
require "active_record_proxy_adapters/hijackable"
|
|
7
|
+
require "active_record_proxy_adapters/mysql2_proxy"
|
|
8
|
+
|
|
9
|
+
module ActiveRecord
|
|
10
|
+
module ConnectionAdapters
|
|
11
|
+
# This adapter is a proxy to the original Mysql2Adapter, allowing the use of the
|
|
12
|
+
# ActiveRecordProxyAdapters::PrimaryReplicaProxy.
|
|
13
|
+
class Mysql2ProxyAdapter < Mysql2Adapter
|
|
14
|
+
include ActiveRecordProxyAdapters::Hijackable
|
|
15
|
+
|
|
16
|
+
ADAPTER_NAME = "Mysql2Proxy"
|
|
17
|
+
|
|
18
|
+
delegate_to_proxy :execute, :exec_query
|
|
19
|
+
|
|
20
|
+
def initialize(...)
|
|
21
|
+
@proxy = ActiveRecordProxyAdapters::Mysql2Proxy.new(self)
|
|
22
|
+
|
|
23
|
+
super
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
private
|
|
27
|
+
|
|
28
|
+
attr_reader :proxy
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
if ActiveRecordProxyAdapters::ActiveRecordContext.active_record_v7_2_or_greater?
|
|
34
|
+
ActiveRecord::ConnectionAdapters.register(
|
|
35
|
+
"mysql2_proxy",
|
|
36
|
+
"ActiveRecord::ConnectionAdapters::Mysql2ProxyAdapter",
|
|
37
|
+
"active_record/connection_adapters/mysql2_proxy_adapter"
|
|
38
|
+
)
|
|
39
|
+
end
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_record_proxy_adapters/database_tasks"
|
|
4
|
+
|
|
5
|
+
module ActiveRecord
|
|
6
|
+
module Tasks
|
|
7
|
+
# Defines the mysql tasks for dropping, creating, loading schema and dumping schema.
|
|
8
|
+
# Bypasses all the proxy logic to send all requests to primary.
|
|
9
|
+
class Mysql2ProxyDatabaseTasks < MySQLDatabaseTasks
|
|
10
|
+
include ActiveRecordProxyAdapters::DatabaseTasks
|
|
11
|
+
end
|
|
12
|
+
end
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
# Allow proxy adapter to run rake tasks, i.e. db:drop, db:create, db:schema:load db:migrate, etc...
|
|
16
|
+
ActiveRecord::Tasks::DatabaseTasks.register_task(
|
|
17
|
+
/mysql2_proxy/,
|
|
18
|
+
"ActiveRecord::Tasks::Mysql2ProxyDatabaseTasks"
|
|
19
|
+
)
|
|
@@ -1,39 +1,13 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "active_record_proxy_adapters/database_tasks"
|
|
4
|
+
|
|
3
5
|
module ActiveRecord
|
|
4
6
|
module Tasks
|
|
5
7
|
# Defines the postgresql tasks for dropping, creating, loading schema and dumping schema.
|
|
6
8
|
# Bypasses all the proxy logic to send all requests to primary.
|
|
7
9
|
class PostgreSQLProxyDatabaseTasks < PostgreSQLDatabaseTasks
|
|
8
|
-
|
|
9
|
-
sticking_to_primary { super }
|
|
10
|
-
end
|
|
11
|
-
|
|
12
|
-
def drop(...)
|
|
13
|
-
sticking_to_primary { super }
|
|
14
|
-
end
|
|
15
|
-
|
|
16
|
-
def structure_dump(...)
|
|
17
|
-
sticking_to_primary { super }
|
|
18
|
-
end
|
|
19
|
-
|
|
20
|
-
def structure_load(...)
|
|
21
|
-
sticking_to_primary { super }
|
|
22
|
-
end
|
|
23
|
-
|
|
24
|
-
def purge(...)
|
|
25
|
-
sticking_to_primary { super }
|
|
26
|
-
end
|
|
27
|
-
|
|
28
|
-
private
|
|
29
|
-
|
|
30
|
-
def sticking_to_primary(&)
|
|
31
|
-
ActiveRecord::Base.connected_to(role: context.writing_role, &)
|
|
32
|
-
end
|
|
33
|
-
|
|
34
|
-
def context
|
|
35
|
-
ActiveRecordProxyAdapters::ActiveRecordContext.new
|
|
36
|
-
end
|
|
10
|
+
include ActiveRecordProxyAdapters::DatabaseTasks
|
|
37
11
|
end
|
|
38
12
|
end
|
|
39
13
|
end
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "active_record/connection_adapters/postgresql_proxy_adapter"
|
|
4
|
+
require "active_record/connection_adapters/mysql2_proxy_adapter"
|
|
4
5
|
|
|
5
6
|
module ActiveRecordProxyAdapters
|
|
6
7
|
# Module to extend ActiveRecord::Base with the connection handling methods.
|
|
@@ -31,5 +32,30 @@ module ActiveRecordProxyAdapters
|
|
|
31
32
|
config
|
|
32
33
|
)
|
|
33
34
|
end
|
|
35
|
+
|
|
36
|
+
def mysql2_proxy_adapter_class
|
|
37
|
+
::ActiveRecord::ConnectionAdapters::Mysql2ProxyAdapter
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# This method is a copy and paste from Rails' mysql2_connection,
|
|
41
|
+
# replacing Mysql2Adapter by Mysql2ProxyAdapter
|
|
42
|
+
# This is required by ActiveRecord versions <= 7.2.x to establish a connection using the adapter.
|
|
43
|
+
def mysql2_proxy_connection(config) # rubocop:disable Metrics/MethodLength
|
|
44
|
+
config = config.symbolize_keys
|
|
45
|
+
config[:flags] ||= 0
|
|
46
|
+
|
|
47
|
+
if config[:flags].is_a? Array
|
|
48
|
+
config[:flags].push "FOUND_ROWS"
|
|
49
|
+
else
|
|
50
|
+
config[:flags] |= Mysql2::Client::FOUND_ROWS
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
mysql2_proxy_adapter_class.new(
|
|
54
|
+
mysql2_proxy_adapter_class.new_client(config),
|
|
55
|
+
logger,
|
|
56
|
+
nil,
|
|
57
|
+
config
|
|
58
|
+
)
|
|
59
|
+
end
|
|
34
60
|
end
|
|
35
61
|
end
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ActiveRecordProxyAdapters
|
|
4
|
+
module DatabaseTasks # rubocop:disable Style/Documentation
|
|
5
|
+
extend ActiveSupport::Concern
|
|
6
|
+
|
|
7
|
+
included do
|
|
8
|
+
def create(...)
|
|
9
|
+
sticking_to_primary { super }
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def drop(...)
|
|
13
|
+
sticking_to_primary { super }
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def structure_dump(...)
|
|
17
|
+
sticking_to_primary { super }
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def structure_load(...)
|
|
21
|
+
sticking_to_primary { super }
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def purge(...)
|
|
25
|
+
sticking_to_primary { super }
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
private
|
|
29
|
+
|
|
30
|
+
def sticking_to_primary(&)
|
|
31
|
+
ActiveRecord::Base.connected_to(role: context.writing_role, &)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def context
|
|
35
|
+
ActiveRecordProxyAdapters::ActiveRecordContext.new
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
end
|
|
@@ -2,14 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
require "active_record/tasks/postgresql_proxy_database_tasks"
|
|
4
4
|
require "active_record/connection_adapters/postgresql_adapter"
|
|
5
|
-
require "active_record_proxy_adapters/primary_replica_proxy"
|
|
6
5
|
|
|
7
6
|
module ActiveRecordProxyAdapters
|
|
8
7
|
# Defines mixins to delegate specific methods from the proxy to the adapter.
|
|
9
8
|
module Hijackable
|
|
10
9
|
extend ActiveSupport::Concern
|
|
11
10
|
|
|
12
|
-
class_methods do
|
|
11
|
+
class_methods do # rubocop:disable Metrics/BlockLength
|
|
13
12
|
# Renames the methods from the original Adapter using the proxy suffix (_unproxied)
|
|
14
13
|
# and delegate the original method name to the proxy.
|
|
15
14
|
# Example: delegate_to_proxy(:execute) creates a method `execute_unproxied`,
|
|
@@ -32,10 +31,40 @@ module ActiveRecordProxyAdapters
|
|
|
32
31
|
delegate(*method_names, to: :proxy)
|
|
33
32
|
end
|
|
34
33
|
|
|
34
|
+
# Defines which methods should be hijacked from the original adapter and use the proxy
|
|
35
|
+
# @param method_names [Array<Symbol>] the list of method names from the adapter
|
|
36
|
+
def hijack_method(*method_names) # rubocop:disable Metrics/MethodLength
|
|
37
|
+
@hijacked_methods ||= Set.new
|
|
38
|
+
@hijacked_methods += Set.new(method_names)
|
|
39
|
+
|
|
40
|
+
method_names.each do |method_name|
|
|
41
|
+
define_method(method_name) do |*args, **kwargs, &block|
|
|
42
|
+
proxy_bypass_method = "#{method_name}#{unproxied_method_suffix}"
|
|
43
|
+
sql_string = coerce_query_to_string(args.first)
|
|
44
|
+
|
|
45
|
+
appropriate_connection(sql_string) do |conn|
|
|
46
|
+
method_to_call = conn == primary_connection ? proxy_bypass_method : method_name
|
|
47
|
+
|
|
48
|
+
conn.send(method_to_call, *args, **kwargs, &block)
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def unproxied_method_suffix
|
|
55
|
+
"_unproxied"
|
|
56
|
+
end
|
|
57
|
+
|
|
35
58
|
private
|
|
36
59
|
|
|
37
60
|
def proxy_method_name_for(method_name)
|
|
38
|
-
:"#{method_name}#{
|
|
61
|
+
:"#{method_name}#{unproxied_method_suffix}"
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
included do
|
|
66
|
+
def unproxied_method_suffix
|
|
67
|
+
self.class.unproxied_method_suffix
|
|
39
68
|
end
|
|
40
69
|
end
|
|
41
70
|
end
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_record_proxy_adapters/primary_replica_proxy"
|
|
4
|
+
|
|
5
|
+
module ActiveRecordProxyAdapters
|
|
6
|
+
# Proxy to the original Mysql2Adapter, allowing the use of the ActiveRecordProxyAdapters::PrimaryReplicaProxy.
|
|
7
|
+
class Mysql2Proxy < PrimaryReplicaProxy
|
|
8
|
+
end
|
|
9
|
+
end
|
|
@@ -7,8 +7,6 @@ module ActiveRecordProxyAdapters
|
|
|
7
7
|
# Proxy to the original PostgreSQLAdapter, allowing the use of the ActiveRecordProxyAdapters::PrimaryReplicaProxy.
|
|
8
8
|
class PostgreSQLProxy < PrimaryReplicaProxy
|
|
9
9
|
# ActiveRecord::PostgreSQLAdapter methods that should be proxied.
|
|
10
|
-
hijack_method :execute, :exec_query
|
|
11
|
-
|
|
12
10
|
hijack_method :exec_no_cache, :exec_cache unless ActiveRecordContext.active_record_v8_0_or_greater?
|
|
13
11
|
end
|
|
14
12
|
end
|
|
@@ -5,11 +5,14 @@ require "active_support/core_ext/module/delegation"
|
|
|
5
5
|
require "active_support/core_ext/object/blank"
|
|
6
6
|
require "concurrent-ruby"
|
|
7
7
|
require "active_record_proxy_adapters/active_record_context"
|
|
8
|
+
require "active_record_proxy_adapters/hijackable"
|
|
8
9
|
|
|
9
10
|
module ActiveRecordProxyAdapters
|
|
10
11
|
# This is the base class for all proxies. It defines the methods that should be proxied
|
|
11
12
|
# and the logic to determine which database to use.
|
|
12
13
|
class PrimaryReplicaProxy # rubocop:disable Metrics/ClassLength
|
|
14
|
+
include Hijackable
|
|
15
|
+
|
|
13
16
|
# All queries that match these patterns should be sent to the primary database
|
|
14
17
|
SQL_PRIMARY_MATCHERS = [
|
|
15
18
|
/\A\s*select.+for update\Z/i, /select.+lock in share mode\Z/i,
|
|
@@ -25,27 +28,9 @@ module ActiveRecordProxyAdapters
|
|
|
25
28
|
# requests to the primary database so the replica has time to replicate
|
|
26
29
|
WRITE_STATEMENT_MATCHERS = [/\ABEGIN/i, /\ACOMMIT/i, /INSERT\sINTO\s/i, /UPDATE\s/i, /DELETE\sFROM\s/i,
|
|
27
30
|
/DROP\s/i].map(&:freeze).freeze
|
|
28
|
-
UNPROXIED_METHOD_SUFFIX = "_unproxied"
|
|
29
|
-
|
|
30
|
-
# Defines which methods should be hijacked from the original adapter and use the proxy
|
|
31
|
-
# @param method_names [Array<Symbol>] the list of method names from the adapter
|
|
32
|
-
def self.hijack_method(*method_names) # rubocop:disable Metrics/MethodLength
|
|
33
|
-
@hijacked_methods ||= Set.new
|
|
34
|
-
@hijacked_methods += Set.new(method_names)
|
|
35
|
-
|
|
36
|
-
method_names.each do |method_name|
|
|
37
|
-
define_method(method_name) do |*args, **kwargs, &block|
|
|
38
|
-
proxy_bypass_method = "#{method_name}#{UNPROXIED_METHOD_SUFFIX}"
|
|
39
|
-
sql_string = coerce_query_to_string(args.first)
|
|
40
|
-
|
|
41
|
-
appropriate_connection(sql_string) do |conn|
|
|
42
|
-
method_to_call = conn == primary_connection ? proxy_bypass_method : method_name
|
|
43
31
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
end
|
|
47
|
-
end
|
|
48
|
-
end
|
|
32
|
+
# Abstract adapter methods that should be proxied.
|
|
33
|
+
hijack_method :execute, :exec_query
|
|
49
34
|
|
|
50
35
|
def self.hijacked_methods
|
|
51
36
|
@hijacked_methods.to_a
|
|
@@ -62,7 +47,7 @@ module ActiveRecordProxyAdapters
|
|
|
62
47
|
|
|
63
48
|
attr_reader :primary_connection, :last_write_at, :active_record_context
|
|
64
49
|
|
|
65
|
-
delegate :connection_handler,
|
|
50
|
+
delegate :connection_handler, to: :connection_class
|
|
66
51
|
delegate :reading_role, :writing_role, to: :active_record_context
|
|
67
52
|
|
|
68
53
|
def replica_pool_unavailable?
|
|
@@ -120,11 +105,25 @@ module ActiveRecordProxyAdapters
|
|
|
120
105
|
[reading_role, writing_role].include?(role) ? role : nil
|
|
121
106
|
end
|
|
122
107
|
|
|
108
|
+
def connected_to_stack
|
|
109
|
+
return connection_class.connected_to_stack if connection_class.respond_to?(:connected_to_stack)
|
|
110
|
+
|
|
111
|
+
# handle Rails 7.2+ pending migrations Connection
|
|
112
|
+
return [{ role: writing_role }] if pending_migration_connection?
|
|
113
|
+
|
|
114
|
+
[]
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def pending_migration_connection?
|
|
118
|
+
active_record_context.active_record_v7_1_or_greater? &&
|
|
119
|
+
connection_class.name == "ActiveRecord::PendingMigrationConnection"
|
|
120
|
+
end
|
|
121
|
+
|
|
123
122
|
def connection_for(role, sql_string)
|
|
124
123
|
connection = primary_connection if role == writing_role || replica_pool_unavailable?
|
|
125
124
|
connection ||= checkout_replica_connection
|
|
126
125
|
|
|
127
|
-
result = yield
|
|
126
|
+
result = connected_to(role:) { yield connection }
|
|
128
127
|
|
|
129
128
|
update_primary_latest_write_timestamp if !replica_connection?(connection) && write_statement?(sql_string)
|
|
130
129
|
|
|
@@ -133,6 +132,12 @@ module ActiveRecordProxyAdapters
|
|
|
133
132
|
replica_connection?(connection) && replica_pool.checkin(connection)
|
|
134
133
|
end
|
|
135
134
|
|
|
135
|
+
def connected_to(role:, &block)
|
|
136
|
+
return block.call unless connection_class.respond_to?(:connected_to)
|
|
137
|
+
|
|
138
|
+
connection_class.connected_to(role:, &block)
|
|
139
|
+
end
|
|
140
|
+
|
|
136
141
|
def replica_connection?(connection)
|
|
137
142
|
connection && connection != primary_connection
|
|
138
143
|
end
|
data/postgres_primary.dockerfile
CHANGED
data/postgres_replica.dockerfile
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: active_record_proxy_adapters
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Matt Cruz
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2025-01-
|
|
11
|
+
date: 2025-01-18 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -67,17 +67,22 @@ files:
|
|
|
67
67
|
- Dockerfile
|
|
68
68
|
- LICENSE.txt
|
|
69
69
|
- README.md
|
|
70
|
+
- db/mysql_structure.sql
|
|
70
71
|
- db/postgresql_structure.sql
|
|
71
72
|
- docker-compose.yml
|
|
72
73
|
- docker/postgres_replica/cmd.sh
|
|
74
|
+
- lib/active_record/connection_adapters/mysql2_proxy_adapter.rb
|
|
73
75
|
- lib/active_record/connection_adapters/postgresql_proxy_adapter.rb
|
|
76
|
+
- lib/active_record/tasks/mysql2_proxy_database_tasks.rb
|
|
74
77
|
- lib/active_record/tasks/postgresql_proxy_database_tasks.rb
|
|
75
78
|
- lib/active_record_proxy_adapters.rb
|
|
76
79
|
- lib/active_record_proxy_adapters/active_record_context.rb
|
|
77
80
|
- lib/active_record_proxy_adapters/configuration.rb
|
|
78
81
|
- lib/active_record_proxy_adapters/connection_handling.rb
|
|
82
|
+
- lib/active_record_proxy_adapters/database_tasks.rb
|
|
79
83
|
- lib/active_record_proxy_adapters/hijackable.rb
|
|
80
84
|
- lib/active_record_proxy_adapters/log_subscriber.rb
|
|
85
|
+
- lib/active_record_proxy_adapters/mysql2_proxy.rb
|
|
81
86
|
- lib/active_record_proxy_adapters/postgresql_proxy.rb
|
|
82
87
|
- lib/active_record_proxy_adapters/primary_replica_proxy.rb
|
|
83
88
|
- lib/active_record_proxy_adapters/railtie.rb
|