ros-apartment 4.0.0.alpha13 → 4.0.0.alpha14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +1 -1
- data/lib/apartment/CLAUDE.md +13 -2
- data/lib/apartment/migration_role.rb +14 -7
- data/lib/apartment/patches/connection_registry.rb +7 -4
- data/lib/apartment/patches/live_tenant_propagation.rb +1 -1
- data/lib/apartment/patches/postgresql_type_map.rb +308 -0
- data/lib/apartment/version.rb +1 -1
- data/lib/apartment.rb +8 -5
- data/ros-apartment.gemspec +1 -1
- metadata +4 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2bce6584d418fcb45322b27efd6b816c8e33b603866310d9ffabf96e1078a776
|
|
4
|
+
data.tar.gz: 77d334fc612ff87fa615b2079f2263b8505c84652e966b08b94386061621fcaa
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 3f73a95f17f83e8034f2a1db98938018d5d1b5e83ec7508720305269b61cdb847c356616d0aa39ea7ad65e42716838c8f243753a14eaf47539ca20fc043ec4c5
|
|
7
|
+
data.tar.gz: c7626e6a4dfad74516f56be92ee2703e9a19e0cc1b11c525d0803dcb36f918f7bce72d9c68b62c226a712d138dd18ee6e0746ac2333b6b8a05a57a6984e3ab60
|
data/README.md
CHANGED
data/lib/apartment/CLAUDE.md
CHANGED
|
@@ -22,7 +22,8 @@ lib/apartment/
|
|
|
22
22
|
├── patches/ # ActiveRecord patches for tenant-aware connections
|
|
23
23
|
│ ├── connection_handling.rb # Prepends on AR::Base — tenant-aware connection_pool
|
|
24
24
|
│ ├── connection_registry.rb # Prepends on AR's PoolManager + ConnectionHandler — serializes the pool registry
|
|
25
|
-
│
|
|
25
|
+
│ ├── postgresql_sequence_name.rb # Prepends on the PG adapter — schema-agnostic Model.sequence_name memoization
|
|
26
|
+
│ └── postgresql_type_map.rb # Prepends on the PG adapter — one OID type map per database, shared across tenant pools
|
|
26
27
|
├── privileges/ # Tenant privilege policy support
|
|
27
28
|
│ └── context.rb # Privileges::Context: what a tenant_privilege_policy receives, one per phase
|
|
28
29
|
├── tasks/ # Rake task utilities; v4.rake for apartment:create/drop/migrate/seed/rollback
|
|
@@ -77,6 +78,16 @@ The readers are Rails' own, all reaching the registry through `ConnectionHandler
|
|
|
77
78
|
|
|
78
79
|
`serialize!` **fails closed**: it resolves each guarded method with `method_defined?` / `private_method_defined?` (matching what `super` actually needs, so an upstream method merely *moved* to a superclass or module still counts) and raises `Apartment::ConfigurationError` at `activate!` when one is genuinely gone. It only *warns* when upstream has grown a public accessor we do not guard — a hole beats a boot failure. See `docs/designs/ar-connection-registry-thread-safety.md`.
|
|
79
80
|
|
|
81
|
+
### patches/postgresql_type_map.rb — Shared PostgreSQL Type Map
|
|
82
|
+
|
|
83
|
+
Rails builds the PostgreSQL OID type map **per connection**: `configure_connection` ends in `reload_type_map`, which clears `@type_map` and runs `initialize_type_map`, and that issues three `pg_type` queries. Two of them have no usable index and sequential-scan `pg_type`, which grows by two rows per table per tenant schema (a composite type and its array type). Pool-per-tenant pays that on every cold tenant pool, so a nightly sweep over hundreds of tenants rebuilds the same map hundreds of times against a catalog hundreds of times larger than a single-schema app's: measured 3.4 ms of type-map queries per connect at 5 schemas, 65 ms at 500 (design: `docs/designs/postgresql-type-map-sharing.md`).
|
|
84
|
+
|
|
85
|
+
The patch shares one map per **database** (OIDs are database-wide; `search_path`, role and tenant play no part) across every adapter in the process, through two public `:nodoc:` seams identical on Rails 8.1 and main. `clear_cache!(new_connection: true)` is Rails' own "the socket is being replaced" signal, called from `reconnect!`, `disconnect!` and `reset!`; the override nils `@type_map` there, so every path to a new physical connection reaches `reload_type_map` with a nil map. `reload_type_map` with a nil map **adopts** the shared instance from `REGISTRY`, building and publishing with `put_if_absent` if this is the first connection to that database; with a live map, which only the enum DDL helpers reach, it **rebuilds** into a fresh instance and republishes.
|
|
86
|
+
|
|
87
|
+
The key is `[[host, port, database], [database_oid, postmaster_start_time], default_timezone]` and everything but the timezone comes off the **live connection, never `@config`** — libpq resolves defaults the config cannot show (`dbname` from the user name, a service file, `PGDATABASE`, `hostaddr` without `host`), so two identical-looking configs can reach different databases. The catalog identity costs one query (`pg_database.oid` plus the *epoch* of `pg_postmaster_start_time()`, both world-readable, 0.096 ms; the epoch because a timestamptz renders through the session's `TimeZone`/`DateStyle` and would split one catalog's key) and exists because a name is not a catalog: the patch deliberately survives `clear_cache!`, which is what heals a replaced catalog today, so a different cluster behind an unchanged endpoint (blue/green on logical replication, a restore into the same name) would otherwise be adopted — and fresh catalogs restart type OIDs at 16384, so a stale entry describes a *different* type rather than going unused. The OID alone will not do — `postgres` is OID 5 on every cluster ever initdb'd, so an app on the RDS default database would have a constant check — which is why the start time is in the same SELECT; publishing then supersedes same-endpoint entries carrying a different identity, so restarts do not strand one map each. Within one cluster the OID counter is global and monotonic, so a drop and recreate leaves dead entries, not wrong ones; a promoted physical replica rebuilds once, the accepted cost of fail-closed identity. A failed probe warns, **detaches `@type_map` first** and falls back to upstream — detaching is load-bearing, since upstream's `reload_type_map` clears a live map in place and would blank the shared instance for every other holder. The timezone is in the key because `initialize_type_map` bakes it into the `time`/`timestamp` registrations. A published map is never cleared in place: holders keep resolving against it and adopt the rebuilt one on their next physical reconnect, learning any new OID lazily through `get_oid_type` meanwhile, exactly as a stale per-connection map does today.
|
|
88
|
+
|
|
89
|
+
Three details are load-bearing. The map is a `SharedTypeMap < HashLookupTypeMap` whose `@mapping` is a `Concurrent::Map`, so lazy OID registrations from one thread are serialized against reads from every other by the same primitive Rails uses for the other half of that object; on MRI a plain Hash would already be safe here, but the argument should not be implicit. No registry lock is held across I/O: the catalog queries run before `put_if_absent`, whose **return value is load-bearing** — the loser of a publish race adopts the winner, so every adapter converges on one instance. And the build writes into the adapter's own `@type_map` before publishing, because `initialize_type_map` loads through the private `type_map` reader rather than its argument; `apply!` **fails closed** (`ConfigurationError`) if that reader or either public seam is gone. Retention is measured rather than bounded: 88.5 KB per map, one key for the whole process under schema-per-tenant, ~49 MB for 570 tenant databases — no eviction ships because a cap tight enough to matter would thrash the nightly sweep it exists to speed up, and eviction stays free to add later. Applied from the gem-load `on_load(:active_record_postgresqladapter)` hook alongside the sequence-name patch, not from `activate!`, because boot opens connections before `activate!` runs and the map they build is the one tenant pools should adopt. `add_pg_decoders` still runs its one index-driven `pg_type` lookup per connection; deliberately untouched. **Rails main** (rails/rails#57013, not in 8.1.x) already ships the built-in OIDs statically and defers the one remaining scan to the first unknown OID per adapter; both seams survive there, `HashLookupTypeMap#initialize` lost its `parent` argument (so `SharedTypeMap` forwards `...`), and the patch's value shrinks to making that deferred scan per process. The integration spec feature-detects `OID::WellKnown` and pends its two connect-time examples on main.
|
|
90
|
+
|
|
80
91
|
### pool_reaper.rb — Pool Eviction + Admission
|
|
81
92
|
|
|
82
93
|
Background `Concurrent::TimerTask` that evicts idle and excess tenant pools, and also serves as the pool manager's synchronous admission controller (`admit!`) when a cap is configured — evicting the LRU idle pool inline before a new one is established, applying `pool_overflow_policy` (`:evict_idle` / `:raise`) when no idle pool can be freed. A single `evict_tenant` primitive backs the timer (idle/LRU) and admission paths. Reap cadence (`interval`/`reaper_interval`) is decoupled from the idle window (`idle_timeout`/`pool_idle_timeout`). Created by `Apartment.configure`, stored as `Apartment.pool_reaper`. Deregisters evicted pools from AR's ConnectionHandler. Default tenant is never evicted.
|
|
@@ -136,7 +147,7 @@ Three hooks in Rails boot order:
|
|
|
136
147
|
|
|
137
148
|
`Apartment::MigrationRole.wrap` runs a block inside `connected_to(role: config.ddl_role)`, or yields when no role is configured. It exists as its own module because both `Migrator` and the adapters need it and an adapter cannot depend on `Migrator`; `Migrator.with_migration_role` stays as the documented entry point and delegates here. All tenant DDL goes through it — migrations, `Tenant.create`, and `Tenant.drop`'s engine call — because PostgreSQL scopes an `ALTER DEFAULT PRIVILEGES` rule with no `FOR ROLE` to the role that executed it, and because `DROP SCHEMA` needs ownership of a container `ddl_role` owns. See `docs/designs/v4-rbac-contract.md`.
|
|
138
149
|
|
|
139
|
-
An unresolvable role is translated here: `ActiveRecord::ConnectionNotEstablished` becomes an `Apartment::ConfigurationError` naming `ddl_role` and the symbol given. `connected_to` resolves no pool — it pushes onto `connected_to_stack` and yields — so the failure arrives from *inside* the block and cannot be detected before it; a wrap whose block never touches the database stays silent by design. What discriminates is not the error class but a probe: `retrieve_connection_pool` for our role, nil meaning the failure is ours to explain and a pool meaning it belongs to the caller's block and re-raises untouched. The rescue names `ConnectionNotEstablished` and **only** that class,
|
|
150
|
+
An unresolvable role is translated here: `ActiveRecord::ConnectionNotEstablished` becomes an `Apartment::ConfigurationError` naming `ddl_role` and the symbol given. `connected_to` resolves no pool — it pushes onto `connected_to_stack` and yields — so the failure arrives from *inside* the block and cannot be detected before it; a wrap whose block never touches the database stays silent by design. What discriminates is not the error class but a probe: `retrieve_connection_pool` for our role, nil meaning the failure is ours to explain and a pool meaning it belongs to the caller's block and re-raises untouched. The rescue names `ConnectionNotEstablished` and **only** that class, even though every supported Rails now raises the `ConnectionNotDefined` subclass here. Narrowing to the subclass would buy nothing — the probe, not the class, decides — and would cost the case the method exists for: a bare `ConnectionNotEstablished` raised inside the caller's block while `ddl_role` is genuinely unregistered would stop getting the message that names `ddl_role`. (History, no longer binding: the subclass did not exist before Rails 8.0, and Ruby resolves rescue constants at raise time, so while 7.2 was supported naming it raised `NameError` and destroyed the error it was classifying.) No wrapped error still needs translating: `Patches::ConnectionHandling#connection_pool` scopes its relabelling rescue to the tenant-resolution path, so errors do still arrive wrapped from inside it — but those are genuine tenant-pool failures that belong to the tenant rather than to `ddl_role`, and translating them would misattribute. A ddl_role failure is not one of them: it surfaces from the default-path lookup outside that boundary, and the tenant path establishes a pool for the role itself so it never fails on an unregistered one. The `ApartmentError` clause and one-layer unwrap this rescue once carried belonged to the old method-level rescue and went with it. Checked at first use rather than at `activate!`, which runs in `after_initialize`, after the eager-load initializer: under lazy loading no model has run `connects_to` yet and a boot-time check would fail on every boot.
|
|
140
151
|
|
|
141
152
|
### privileges.rb / privileges/context.rb — Adopter-Owned Privilege Policy
|
|
142
153
|
|
|
@@ -38,13 +38,20 @@ module Apartment
|
|
|
38
38
|
# explain. Non-nil means the role is fine and the error belongs to something the
|
|
39
39
|
# caller did inside the block, so it re-raises untouched.
|
|
40
40
|
#
|
|
41
|
-
# The probe is the discriminator; the class is only a pre-filter
|
|
42
|
-
#
|
|
43
|
-
#
|
|
44
|
-
#
|
|
45
|
-
#
|
|
46
|
-
# +ConnectionNotEstablished+
|
|
47
|
-
#
|
|
41
|
+
# The probe is the discriminator; the class is only a pre-filter. The clause names
|
|
42
|
+
# the +ConnectionNotEstablished+ superclass even though every supported Rails now
|
|
43
|
+
# raises the +ActiveRecord::ConnectionNotDefined+ subclass here, and that is
|
|
44
|
+
# deliberate rather than leftover. Narrowing to the subclass would buy nothing —
|
|
45
|
+
# the probe, not the class, decides — and would cost the case below: a bare
|
|
46
|
+
# +ConnectionNotEstablished+ raised inside the caller's block while +ddl_role+ is
|
|
47
|
+
# genuinely unregistered would stop getting the message that names +ddl_role+,
|
|
48
|
+
# which is the one situation this method exists to explain. Do not narrow it.
|
|
49
|
+
#
|
|
50
|
+
# (History, no longer binding: the subclass did not exist before Rails 8.0 — absent
|
|
51
|
+
# on 7.2.3.1, present on 8.0.5 and 8.1.3 — and Ruby resolves a rescue clause's
|
|
52
|
+
# constants at raise time, so while 7.2 was supported naming it raised NameError
|
|
53
|
+
# and destroyed the error it was meant to classify. The 8.1 floor removed that
|
|
54
|
+
# constraint; it did not change the conclusion.)
|
|
48
55
|
#
|
|
49
56
|
# The cost of the wider clause is that a bare ConnectionNotEstablished from the
|
|
50
57
|
# caller's own code now reaches the probe instead of passing straight through. That
|
|
@@ -264,10 +264,13 @@ module Apartment
|
|
|
264
264
|
# silently taking any shard already registered in it with it. Serializing
|
|
265
265
|
# the whole method makes the upsert atomic.
|
|
266
266
|
#
|
|
267
|
-
# Wrapped with argument forwarding rather than a reimplementation because
|
|
268
|
-
#
|
|
269
|
-
#
|
|
270
|
-
#
|
|
267
|
+
# Wrapped with argument forwarding rather than a reimplementation because the
|
|
268
|
+
# key derivation is upstream's business, not ours. Across the supported matrix
|
|
269
|
+
# the signature is currently stable — 8.1 and main both take a
|
|
270
|
+
# ConnectionDescriptor (verified) — so forwarding buys nothing today. It is
|
|
271
|
+
# kept because upstream has already changed this argument once: 7.2 passed the
|
|
272
|
+
# connection-name String and 8.0 replaced it with the descriptor. Forwarding
|
|
273
|
+
# costs a token and survives the next one.
|
|
271
274
|
#
|
|
272
275
|
# Declared private to match upstream. A prepended method is public by
|
|
273
276
|
# default, and leaving it so would widen a Rails internal into public API
|
|
@@ -7,7 +7,7 @@ module Apartment
|
|
|
7
7
|
# Backports rails/rails#56902 ("Pass IsolatedExecutionState.context to
|
|
8
8
|
# share_with") to Rails versions where the fix has not landed.
|
|
9
9
|
#
|
|
10
|
-
# On Rails
|
|
10
|
+
# On Rails 8.1.x stable, ActionController::Live#process calls
|
|
11
11
|
# ActiveSupport::IsolatedExecutionState.share_with(Thread.current, ...)
|
|
12
12
|
# which reads from Thread.current.active_support_execution_state. Under
|
|
13
13
|
# :fiber isolation that's empty — the data lives on Fiber.current's
|
|
@@ -0,0 +1,308 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Referenced in a class body below. Rails requires it from the PostgreSQL
|
|
4
|
+
# adapter file, which an eager-loading host app may not have loaded yet.
|
|
5
|
+
require 'active_record/type/hash_lookup_type_map'
|
|
6
|
+
|
|
7
|
+
module Apartment
|
|
8
|
+
module Patches
|
|
9
|
+
# Shares one PostgreSQL OID type map per database across every adapter in
|
|
10
|
+
# the process.
|
|
11
|
+
#
|
|
12
|
+
# Rails builds the type map per connection: configure_connection ends in
|
|
13
|
+
# reload_type_map, which clears @type_map and runs initialize_type_map, and
|
|
14
|
+
# that issues three pg_type queries (load_types_queries). Two of them have
|
|
15
|
+
# no usable index and sequential-scan pg_type, which grows by two rows per
|
|
16
|
+
# table per tenant schema. Pool-per-tenant pays that on every cold tenant
|
|
17
|
+
# pool, so a nightly sweep over hundreds of tenants rebuilds the same map
|
|
18
|
+
# hundreds of times against a catalog hundreds of times larger than a
|
|
19
|
+
# single-schema app's. Measured at 65 ms per connect for 500 schemas of
|
|
20
|
+
# 220 tables. The map is database-scoped (OIDs are), so one instance can
|
|
21
|
+
# serve every tenant pool. Design: docs/designs/postgresql-type-map-sharing.md.
|
|
22
|
+
#
|
|
23
|
+
# Two public :nodoc: seams, identical on Rails 8.1 and main:
|
|
24
|
+
#
|
|
25
|
+
# * clear_cache!(new_connection: true) is Rails' own "the socket is being
|
|
26
|
+
# replaced, drop connection-derived caches" signal, called from reconnect!,
|
|
27
|
+
# disconnect! and reset!. We drop @type_map there, so every path to a new
|
|
28
|
+
# physical connection reaches reload_type_map with a nil map.
|
|
29
|
+
# * reload_type_map with a nil map ADOPTS the shared instance (building and
|
|
30
|
+
# publishing it if this is the first connection to that database); with a
|
|
31
|
+
# live map it REBUILDS into a fresh instance and republishes. The live-map
|
|
32
|
+
# callers upstream are the enum DDL helpers and disable_extension. A
|
|
33
|
+
# published map is
|
|
34
|
+
# never cleared in place: other holders keep resolving against it and pick
|
|
35
|
+
# up the rebuilt one on their next physical reconnect, learning any new
|
|
36
|
+
# OID lazily through get_oid_type meanwhile, exactly as a stale
|
|
37
|
+
# per-connection map does today.
|
|
38
|
+
#
|
|
39
|
+
# The build writes into this adapter's own @type_map before publishing
|
|
40
|
+
# because initialize_type_map loads through the private +type_map+ reader,
|
|
41
|
+
# not its argument (load_additional_types constructs
|
|
42
|
+
# TypeMapInitializer.new(type_map)). Both that reader and
|
|
43
|
+
# initialize_type_map are checked by apply!, which refuses to boot without
|
|
44
|
+
# either.
|
|
45
|
+
#
|
|
46
|
+
# RETENTION, measured rather than bounded. An entry is removed only when a
|
|
47
|
+
# later publish supersedes it (see #apartment_supersede_stale_identities),
|
|
48
|
+
# so a process retains one map per database it has connected to, times the
|
|
49
|
+
# timezone variants live against it. One built map
|
|
50
|
+
# holds ~138 registrations and 88.5 KB of RSS (measured against PostgreSQL
|
|
51
|
+
# 18 on Rails 8.1). Schema-per-tenant -- the common case, and the one this
|
|
52
|
+
# patch was written for -- has exactly ONE key for the whole process. Only
|
|
53
|
+
# database-per-tenant grows, and it grows with the tenants a process
|
|
54
|
+
# actually serves rather than with time: 570 tenant databases is ~49 MB. No
|
|
55
|
+
# eviction policy ships because a cap tight enough to bound that
|
|
56
|
+
# meaningfully is also tight enough to thrash the nightly sweep it exists to
|
|
57
|
+
# speed up, which is the deployment that has the problem in the first place.
|
|
58
|
+
# Eviction is semantically free if that ever changes -- dropping an entry
|
|
59
|
+
# only costs the next cold connection one rebuild, which is the unpatched
|
|
60
|
+
# behaviour -- so a bound can land later with no correctness migration.
|
|
61
|
+
# +reset!+ is the escape hatch in the meantime.
|
|
62
|
+
module PostgresqlTypeMap
|
|
63
|
+
# HashLookupTypeMap whose mapping is a Concurrent::Map.
|
|
64
|
+
#
|
|
65
|
+
# Upstream keeps @mapping in a plain Hash because one connection owns it.
|
|
66
|
+
# Shared, it sees lock-free reads from every thread and occasional writes
|
|
67
|
+
# from lazy OID registration. On MRI a plain Hash would already be safe
|
|
68
|
+
# here (single C calls, no Ruby-level iteration in this class, Integer and
|
|
69
|
+
# String keys), but the argument should not be implicit: Concurrent::Map
|
|
70
|
+
# is the primitive Rails already uses for the other half of this object,
|
|
71
|
+
# and every operation the adapter and TypeMapInitializer perform on the
|
|
72
|
+
# store ([]=, fetch(key, default), key?, keys, clear) exists on it with the
|
|
73
|
+
# same semantics.
|
|
74
|
+
class SharedTypeMap < ActiveRecord::Type::HashLookupTypeMap
|
|
75
|
+
# Forwards whatever it is given: 8.1 takes an optional parent, Rails main
|
|
76
|
+
# takes nothing, and the adapter passes nothing on either. The forwarding is
|
|
77
|
+
# what spans the two SUPPORTED lanes -- not a shim for a dropped Rails.
|
|
78
|
+
def initialize(...)
|
|
79
|
+
super
|
|
80
|
+
@mapping = Concurrent::Map.new
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# [[host, port, database], [database_oid, postmaster_start_time],
|
|
85
|
+
# default_timezone] => SharedTypeMap. Nested so the endpoint and the
|
|
86
|
+
# catalog incarnation stay separable: #apartment_supersede_stale_identities
|
|
87
|
+
# needs exactly that distinction. Everything but the timezone is read off
|
|
88
|
+
# the live connection rather than @config -- see #apartment_type_map_key.
|
|
89
|
+
# Timezone is part of the key because initialize_type_map bakes
|
|
90
|
+
# @default_timezone into the time and timestamp registrations (verified on
|
|
91
|
+
# 8.1 and main).
|
|
92
|
+
REGISTRY = Concurrent::Map.new
|
|
93
|
+
|
|
94
|
+
# Identity of the catalog behind the endpoint name, in one round trip.
|
|
95
|
+
#
|
|
96
|
+
# The database OID alone is NOT enough, which a probe settles rather than
|
|
97
|
+
# an argument: template1, template0 and postgres hold OIDs 1, 4 and 5 on
|
|
98
|
+
# every cluster ever initdb'd, so an app whose database is `postgres` --
|
|
99
|
+
# the RDS default -- would have an identity check that evaluates to a
|
|
100
|
+
# constant. Freshly provisioned clusters also hand the first user database
|
|
101
|
+
# the same OID, 16384, so identical provisioning collides too. Neither
|
|
102
|
+
# needs a pooler or any exotic topology; a plain endpoint swap is enough.
|
|
103
|
+
# pg_postmaster_start_time() closes both: two live clusters disagree on it
|
|
104
|
+
# with near-certainty.
|
|
105
|
+
#
|
|
106
|
+
# Both are world-readable and need no privilege, unlike
|
|
107
|
+
# pg_control_system() and pg_control_checkpoint(), which are superuser-only
|
|
108
|
+
# by default. Everything is schema-qualified because an unqualified name
|
|
109
|
+
# can resolve to a temporary relation or an earlier entry in search_path.
|
|
110
|
+
# Measured at 0.049 ms of execution, 0.096 ms including the round trip,
|
|
111
|
+
# against 1.9 ms for a full type-map load on a 647-row pg_type and ~24 ms
|
|
112
|
+
# on a 281,927-row one.
|
|
113
|
+
# EXTRACT(EPOCH FROM ...) rather than the timestamptz itself, because the
|
|
114
|
+
# value is read as text and a timestamptz renders through the session's
|
|
115
|
+
# TimeZone and DateStyle. Two connection classes to the SAME catalog that
|
|
116
|
+
# differ in those (a per-role `ALTER ROLE ... SET TimeZone` with no
|
|
117
|
+
# explicit `variables` entry, say) would produce different identity
|
|
118
|
+
# strings, split the key, and set the supersede rule below ping-ponging --
|
|
119
|
+
# sharing quietly switching itself off, with no wrong data and no warning
|
|
120
|
+
# to explain it. A numeric epoch depends on neither GUC.
|
|
121
|
+
DATABASE_IDENTITY_SQL = <<~SQL.squish
|
|
122
|
+
SELECT d.oid, EXTRACT(EPOCH FROM pg_catalog.pg_postmaster_start_time())
|
|
123
|
+
FROM pg_catalog.pg_database d
|
|
124
|
+
WHERE d.datname = pg_catalog.current_database()
|
|
125
|
+
SQL
|
|
126
|
+
|
|
127
|
+
PUBLIC_SEAMS = %i[reload_type_map clear_cache!].freeze
|
|
128
|
+
PRIVATE_SEAMS = %i[initialize_type_map type_map].freeze
|
|
129
|
+
|
|
130
|
+
class << self
|
|
131
|
+
# FAILS CLOSED on a shape we cannot patch, for the same reason
|
|
132
|
+
# ConnectionRegistry does: the failure this patch prevents is database
|
|
133
|
+
# saturation during the nightly sweep, which is far harder to attribute
|
|
134
|
+
# than a boot error naming the ActiveRecord version. Resolution uses
|
|
135
|
+
# method_defined? so a seam that merely moved to a superclass still
|
|
136
|
+
# counts. Prepend is idempotent, so apply! may run more than once.
|
|
137
|
+
def apply!(adapter_class)
|
|
138
|
+
missing = PUBLIC_SEAMS.reject { |name| adapter_class.method_defined?(name) } +
|
|
139
|
+
PRIVATE_SEAMS.reject { |name| adapter_class.private_method_defined?(name) }
|
|
140
|
+
|
|
141
|
+
unless missing.empty?
|
|
142
|
+
raise(Apartment::ConfigurationError,
|
|
143
|
+
'Apartment cannot share the PostgreSQL type map on ActiveRecord ' \
|
|
144
|
+
"#{ActiveRecord::VERSION::STRING}: expected method(s) #{missing.join(', ')} " \
|
|
145
|
+
'are gone. Without them every cold tenant pool reloads the OID type map ' \
|
|
146
|
+
'from pg_type. Upgrade ros-apartment to a version that supports this ' \
|
|
147
|
+
'ActiveRecord release.')
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
adapter_class.prepend(self)
|
|
151
|
+
nil
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# Forget every shared map. The next connection to each database rebuilds.
|
|
155
|
+
# A test hook for suites that need a cold start, and the escape hatch for
|
|
156
|
+
# an adopter who wants the retention described above reclaimed. Not a
|
|
157
|
+
# configuration knob.
|
|
158
|
+
def reset!
|
|
159
|
+
REGISTRY.clear
|
|
160
|
+
end
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
def clear_cache!(new_connection: false)
|
|
164
|
+
super
|
|
165
|
+
@type_map = nil if new_connection
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
def reload_type_map
|
|
169
|
+
@lock.synchronize do
|
|
170
|
+
key = apartment_type_map_key
|
|
171
|
+
|
|
172
|
+
# No identity we can trust, so fall back to upstream's per-connection
|
|
173
|
+
# map: sharing under an identity we could not confirm is the one thing
|
|
174
|
+
# this must never do. Detaching first is load-bearing rather than
|
|
175
|
+
# tidy -- upstream's reload_type_map CLEARS a live @type_map in place,
|
|
176
|
+
# and if this adapter had already adopted the shared instance that
|
|
177
|
+
# would blank the map every other holder is resolving against, which
|
|
178
|
+
# is the invariant the whole design rests on.
|
|
179
|
+
if key.nil?
|
|
180
|
+
@type_map = nil
|
|
181
|
+
return super
|
|
182
|
+
end
|
|
183
|
+
|
|
184
|
+
if @type_map.nil?
|
|
185
|
+
@type_map = REGISTRY[key] || apartment_publish_type_map(key)
|
|
186
|
+
else
|
|
187
|
+
REGISTRY[key] = apartment_build_type_map
|
|
188
|
+
end
|
|
189
|
+
end
|
|
190
|
+
end
|
|
191
|
+
|
|
192
|
+
private
|
|
193
|
+
|
|
194
|
+
# Ask the live connection, never @config, and ask the server which catalog
|
|
195
|
+
# it actually is.
|
|
196
|
+
#
|
|
197
|
+
# Two different failures make @config alone the wrong source. First, libpq
|
|
198
|
+
# fills in defaults the configuration hash cannot show -- dbname
|
|
199
|
+
# defaulting to the user name, a service file, PGHOST/PGPORT/PGDATABASE,
|
|
200
|
+
# hostaddr given without host -- so two adapters whose configs look
|
|
201
|
+
# identical can land on different databases. PQhost, PQport and PQdb are
|
|
202
|
+
# client-side reads of the RESOLVED parameters (measured ~0.06 us each)
|
|
203
|
+
# and close all of that; PQhost reports hostaddr when only hostaddr was
|
|
204
|
+
# given.
|
|
205
|
+
#
|
|
206
|
+
# Second, and the reason for the query: a name is not a catalog. This
|
|
207
|
+
# patch deliberately survives +clear_cache!(new_connection: true)+, which
|
|
208
|
+
# is Rails' "the socket is being replaced, drop what you cached" signal,
|
|
209
|
+
# and that signal is exactly what heals a replaced catalog today. Put a
|
|
210
|
+
# different cluster behind an unchanged endpoint -- an RDS blue/green
|
|
211
|
+
# cutover on logical replication, a restore into the same name -- and
|
|
212
|
+
# every connection drops, every adapter reconnects, and without this each
|
|
213
|
+
# one would adopt the old catalog's map. Type OIDs in a fresh catalog
|
|
214
|
+
# restart at 16384, so a stale registration does not merely go unused: it
|
|
215
|
+
# can describe a DIFFERENT type that now holds its OID, and an enum gets
|
|
216
|
+
# cast as a domain with nothing raised. (A promoted physical replica is
|
|
217
|
+
# not affected either way -- its catalog is byte-identical, so the OIDs
|
|
218
|
+
# still agree.) Within one cluster the OID counter is global and
|
|
219
|
+
# monotonic, verified by probe, so a same-cluster drop and recreate leaves
|
|
220
|
+
# dead entries rather than wrong ones; it is the cross-cluster case this
|
|
221
|
+
# closes.
|
|
222
|
+
#
|
|
223
|
+
# The identity read runs on the raw connection rather than through the
|
|
224
|
+
# adapter: it must not recurse into the type map it is about to resolve,
|
|
225
|
+
# and PQexec's arity does not drift across the Rails versions this gem
|
|
226
|
+
# supports. Any PG error means we could not establish identity, so we do
|
|
227
|
+
# not share.
|
|
228
|
+
#
|
|
229
|
+
# A pooler whose single alias fans out across several databases, or across
|
|
230
|
+
# clusters, IS caught: every resolved libpq parameter names the pooler, but
|
|
231
|
+
# the identity query travels to the backend. What no connect-time check of
|
|
232
|
+
# any kind can see is a pooler that moves a live client connection onto a
|
|
233
|
+
# different cluster without configure_connection running again.
|
|
234
|
+
def apartment_type_map_key
|
|
235
|
+
raw = @raw_connection
|
|
236
|
+
return nil unless raw
|
|
237
|
+
|
|
238
|
+
[[raw.host, raw.port, raw.db], apartment_database_identity(raw), @default_timezone]
|
|
239
|
+
rescue PG::Error => e
|
|
240
|
+
# Sharing silently switching itself off has no symptom beyond "the
|
|
241
|
+
# nightly got slow again", so leave evidence. Rescuing rather than
|
|
242
|
+
# raising keeps a probe failure from breaking a connection that is
|
|
243
|
+
# otherwise fine; the cost is one rebuilt map.
|
|
244
|
+
warn('[Apartment] could not identify the database for type-map sharing ' \
|
|
245
|
+
"(#{e.class}: #{e.message.lines.first&.strip}); this connection builds its own map.")
|
|
246
|
+
nil
|
|
247
|
+
end
|
|
248
|
+
|
|
249
|
+
# Block form so the PG::Result is freed at once instead of waiting for GC.
|
|
250
|
+
def apartment_database_identity(raw)
|
|
251
|
+
raw.exec(DATABASE_IDENTITY_SQL) { |result| [result.getvalue(0, 0), result.getvalue(0, 1)] }
|
|
252
|
+
end
|
|
253
|
+
|
|
254
|
+
# put_if_absent rather than compute_if_absent: the build runs catalog
|
|
255
|
+
# queries on this connection, and no registry lock is held across that
|
|
256
|
+
# I/O. A lost race costs one redundant build, which is today's cost once.
|
|
257
|
+
# Returning the WINNER is load-bearing -- the loser adopts it and drops its
|
|
258
|
+
# own build, so every adapter converges on one instance.
|
|
259
|
+
def apartment_publish_type_map(key)
|
|
260
|
+
built = apartment_build_type_map
|
|
261
|
+
published = REGISTRY.put_if_absent(key, built) || built
|
|
262
|
+
apartment_supersede_stale_identities(key)
|
|
263
|
+
published
|
|
264
|
+
end
|
|
265
|
+
|
|
266
|
+
# Drop entries for the same endpoint under a DIFFERENT catalog incarnation.
|
|
267
|
+
#
|
|
268
|
+
# Without this, putting the postmaster's start time in the key would trade
|
|
269
|
+
# one unbounded growth for another: every database restart would strand an
|
|
270
|
+
# entry per database, forever, which is growth in TIME rather than in
|
|
271
|
+
# tenants. A superseded entry can never be adopted again -- its identity
|
|
272
|
+
# will not recur -- so removing it costs nothing, and the worst case if an
|
|
273
|
+
# endpoint is flipped back is one rebuild, which is the unpatched
|
|
274
|
+
# behaviour. Timezone variants of the same live catalog share an endpoint
|
|
275
|
+
# AND an identity, so they survive; only the identity slice discriminates.
|
|
276
|
+
#
|
|
277
|
+
# Keys are collected before deleting rather than deleted mid-iteration.
|
|
278
|
+
#
|
|
279
|
+
# Two publishers for the same endpoint under DIFFERENT identities can each
|
|
280
|
+
# delete the other's entry, leaving the registry empty for that endpoint.
|
|
281
|
+
# Harmless, and deliberately uncoordinated: it takes both incarnations
|
|
282
|
+
# reachable at once (a cutover window), every live adapter holds its map by
|
|
283
|
+
# reference, adoption is keyed by the identity the adopting connection
|
|
284
|
+
# measured on its own socket, and losing costs one rebuild -- the unpatched
|
|
285
|
+
# behaviour. Superseding before the publish has the symmetric race, so
|
|
286
|
+
# there is nothing to buy.
|
|
287
|
+
def apartment_supersede_stale_identities(key)
|
|
288
|
+
endpoint, identity = key
|
|
289
|
+
stale = REGISTRY.keys.select { |other| other[0] == endpoint && other[1] != identity }
|
|
290
|
+
stale.each { |other| REGISTRY.delete(other) }
|
|
291
|
+
end
|
|
292
|
+
|
|
293
|
+
def apartment_build_type_map
|
|
294
|
+
@type_map = SharedTypeMap.new
|
|
295
|
+
# Parity with upstream's own reload, which clears this before
|
|
296
|
+
# reinitialising. It exists only on Rails main, where it records whether
|
|
297
|
+
# this adapter has run the deferred bulk pg_type query; the assignment is
|
|
298
|
+
# inert on 8.1, where nothing reads it. Without it a rebuilt
|
|
299
|
+
# map would skip the bulk half of the next deferred load -- still
|
|
300
|
+
# correct, since the specific OID is always requested, but less than
|
|
301
|
+
# upstream promises.
|
|
302
|
+
@type_map_queried = false
|
|
303
|
+
initialize_type_map
|
|
304
|
+
@type_map
|
|
305
|
+
end
|
|
306
|
+
end
|
|
307
|
+
end
|
|
308
|
+
end
|
data/lib/apartment/version.rb
CHANGED
data/lib/apartment.rb
CHANGED
|
@@ -545,14 +545,17 @@ module Apartment # rubocop:disable Metrics/ModuleLength
|
|
|
545
545
|
end
|
|
546
546
|
end
|
|
547
547
|
|
|
548
|
-
# Prepend the
|
|
548
|
+
# Prepend the PostgreSQL adapter patches whenever the adapter loads
|
|
549
549
|
# (immediately, if it already has). Registered at gem load rather than in
|
|
550
|
-
# activate! because
|
|
551
|
-
#
|
|
552
|
-
#
|
|
553
|
-
#
|
|
550
|
+
# activate! because both fire before activate! can: ActiveRecord memoizes
|
|
551
|
+
# Model.sequence_name at first touch, and boot opens connections (schema cache,
|
|
552
|
+
# pending-migration check) whose type map the tenant pools should then adopt.
|
|
553
|
+
# No-op for apps that never load the PostgreSQL adapter, so MySQL/SQLite
|
|
554
|
+
# consumers never pull in pg. See each patch file for the full rationale.
|
|
554
555
|
ActiveSupport.on_load(:active_record_postgresqladapter) do
|
|
555
556
|
prepend(Apartment::Patches::PostgresqlSequenceName)
|
|
557
|
+
|
|
558
|
+
Apartment::Patches::PostgresqlTypeMap.apply!(self)
|
|
556
559
|
end
|
|
557
560
|
|
|
558
561
|
# Load Railtie when Rails is present (standard gem convention).
|
data/ros-apartment.gemspec
CHANGED
|
@@ -29,7 +29,7 @@ Gem::Specification.new do |s|
|
|
|
29
29
|
|
|
30
30
|
s.required_ruby_version = '>= 3.3'
|
|
31
31
|
|
|
32
|
-
s.add_dependency('activerecord', '>=
|
|
32
|
+
s.add_dependency('activerecord', '>= 8.1.0', '< 8.2')
|
|
33
33
|
s.add_dependency('activesupport', '>= 7.2.0', '< 8.2')
|
|
34
34
|
s.add_dependency('concurrent-ruby', '>= 1.3.0')
|
|
35
35
|
s.add_dependency('parallel', '>= 1.26.0')
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ros-apartment
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 4.0.0.
|
|
4
|
+
version: 4.0.0.alpha14
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Ryan Brunner
|
|
@@ -18,7 +18,7 @@ dependencies:
|
|
|
18
18
|
requirements:
|
|
19
19
|
- - ">="
|
|
20
20
|
- !ruby/object:Gem::Version
|
|
21
|
-
version:
|
|
21
|
+
version: 8.1.0
|
|
22
22
|
- - "<"
|
|
23
23
|
- !ruby/object:Gem::Version
|
|
24
24
|
version: '8.2'
|
|
@@ -28,7 +28,7 @@ dependencies:
|
|
|
28
28
|
requirements:
|
|
29
29
|
- - ">="
|
|
30
30
|
- !ruby/object:Gem::Version
|
|
31
|
-
version:
|
|
31
|
+
version: 8.1.0
|
|
32
32
|
- - "<"
|
|
33
33
|
- !ruby/object:Gem::Version
|
|
34
34
|
version: '8.2'
|
|
@@ -199,6 +199,7 @@ files:
|
|
|
199
199
|
- lib/apartment/patches/connection_registry.rb
|
|
200
200
|
- lib/apartment/patches/live_tenant_propagation.rb
|
|
201
201
|
- lib/apartment/patches/postgresql_sequence_name.rb
|
|
202
|
+
- lib/apartment/patches/postgresql_type_map.rb
|
|
202
203
|
- lib/apartment/pool_manager.rb
|
|
203
204
|
- lib/apartment/pool_observer.rb
|
|
204
205
|
- lib/apartment/pool_reaper.rb
|