ros-apartment 4.0.0.alpha7 → 4.0.0.alpha9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: '093f09466c1e39d4344764096cec982394980fb08c8b72712c96bd8ad30fdcc7'
4
- data.tar.gz: 402f7ef009203bb95296a90d7c6ff3dd34eeebca2d70c7da86cdb398ecc6fa60
3
+ metadata.gz: 5b4b1a4532066ffd3ebfdfa8c43b0f0cdd4e304f31f9905af8a91550e71c44a6
4
+ data.tar.gz: 75868c2063131bc20022d35142d87aff2c15f7c8b65b7fa3d8cbee43be4f11d1
5
5
  SHA512:
6
- metadata.gz: 99343f0efe17d444cc23b244581c56135a293371c97ee3fc5d2d2931e51c688c50d9d3d93d8ae55489ec36d3a6a43a910cacdc85d22c43a6c7e80a1bad189faa
7
- data.tar.gz: 2ccd4c456cf1a5ce190d4b50dedfc131231cd1f02518c0444313987ef0a5fda8acc00c1376bfc5daac791768f1b13685d0bb47a08ae5ba755943a2cd6eb5d04b
6
+ metadata.gz: 1f3c98442bf8ea826918440071a3a702615494d8761ef20f990135a10697650e05b72926f28a6cf1053b2aa969b67aee1b8c43249efe73ff58e8802c916f92a3
7
+ data.tar.gz: f23fd1d357e9b5b598516960a0d480f22512b0c8b4481827bb137cc074589a73d4684e4879dc1b2355a12a108fe1b25eaf54eff283aa6c89557c823d91a3e13a
data/README.md CHANGED
@@ -118,15 +118,19 @@ All options are set in `config/initializers/apartment.rb` inside an `Apartment.c
118
118
 
119
119
  ### Pool Settings
120
120
 
121
- `tenant_pool_size`: max connections per tenant pool. Default `nil` — each tenant pool inherits the app's base pool size (`DB_POOL_SIZE` / `pool:` in `database.yml`). Set it to size tenant pools independently of the app pool (e.g. to bound total connections across many schema-per-tenant pools).
121
+ `tenant_pool_size`: max connections per tenant pool. Default `nil` — each tenant pool inherits the app's base pool size (`DB_POOL_SIZE` / `pool:` in `database.yml`). **Set it to at least the peak number of threads/fibers in one process that can touch the _same_ tenant at once** (e.g. a Sidekiq role's concurrency for a same-tenant job fan-out); a smaller pool makes those threads block on connection checkout. It is a lazy ceiling — connections are created on demand and idle pools are reaped — so sizing for peak does not hold that many connections open at steady state.
122
122
 
123
123
  `pool_idle_timeout`: seconds an idle tenant pool must exceed before it is eligible for reaping (default: 300).
124
124
 
125
125
  `reaper_interval`: seconds between background reap passes. Default `nil` — derives from `pool_idle_timeout`. Set it lower to reap more often without shrinking the idle window.
126
126
 
127
- `max_total_connections`: ceiling on the number of live tenant pools; `nil` for unlimited (default: `nil`). Enforced synchronously at pool-creation time (see `pool_overflow_policy`) and trimmed continuously by the background reaper. Total backend connections ≈ `max_total_connections × tenant_pool_size`.
127
+ `max_tenant_pools`: ceiling on the number of live tenant pools per process; `nil` for unlimited (default: `nil`). Enforced synchronously at pool-creation time (see `pool_overflow_policy`) and trimmed continuously by the background reaper.
128
128
 
129
- `pool_overflow_policy`: behavior when a new pool would breach `max_total_connections` and every existing pool is pinned or in use (no idle pool to evict). `:evict_idle` (default) allow the new pool, emit a `cap_unmet` notification (soft cap, prioritizes availability). `:raise` raise `Apartment::PoolCapacityReached` (hard cap, sheds load). When an idle pool *is* available it is always evicted inline regardless of policy. See `docs/designs/pool-admission-control.md`.
129
+ `max_tenant_connections`: ceiling on total *tenant-pool* connections per process (default: `nil`); requires `tenant_pool_size`. The admission controller derives the pool budget as `floor(max_tenant_connections / tenant_pool_size)`. Per-process tenant-pool connections `effective_pool_budget × tenant_pool_size`, where `effective_pool_budget = min(max_tenant_pools, floor(max_tenant_connections / tenant_pool_size))`; the default pool and any separate pinned pool are additional. A tenant using multiple roles (e.g. `writing` + `reading`) holds one pool per role (`tenant:role` keys), counting once per role against the budget. For a hard external-pooler budget, pair this with `pool_overflow_policy: :raise` (the ceiling is soft under the default `:evict_idle`).
130
+
131
+ `max_total_connections`: **deprecated** (removed in v5) — it capped tenant-pool *count*, not connections. It now aliases `max_tenant_pools`; rename it. For a true connection ceiling, set `max_tenant_connections`.
132
+
133
+ `pool_overflow_policy`: behavior when a new pool would breach the pool budget (`max_tenant_pools` / `max_tenant_connections`) and every existing pool is pinned or in use (no idle pool to evict). `:evict_idle` (default) — allow the new pool, emit a `cap_unmet` notification (soft cap, prioritizes availability). `:raise` — raise `Apartment::PoolCapacityReached` (hard cap, sheds load). When an idle pool *is* available it is always evicted inline regardless of policy. See `docs/designs/pool-admission-control.md`.
130
134
 
131
135
  `reap_in_test`: keep the background reaper running under `Rails.env.test?` (default `false` — the Railtie stops it in test, where fixture transactions make mid-example eviction a liability). Set `true` if a deployed process can run under test-env semantics and must keep reaping — that's cleaner than guarding `RAILS_ENV` at boot to avoid silently leaking connections. It applies to *every* `Rails.env.test?` process, including CI, so enable it only when a real deployment needs it.
132
136
 
@@ -334,6 +338,36 @@ Platform notes: parallel migrations use threads. On macOS, libpq has known fork-
334
338
 
335
339
  ## Known Limitations
336
340
 
341
+ ### Connection poolers in transaction mode (PgBouncer, RDS Proxy)
342
+
343
+ > [!WARNING]
344
+ > With the PostgreSQL `:schema` strategy, **PgBouncer in `transaction` pooling mode can
345
+ > silently serve one tenant another tenant's data.** No error is raised — queries return the
346
+ > wrong tenant's rows. It is safe only on **PostgreSQL 18+** with
347
+ > `track_extra_parameters = IntervalStyle,search_path`; on PostgreSQL 17 and older,
348
+ > transaction mode cannot be made safe and you must use `session` mode.
349
+
350
+ Tenant isolation under `:schema` rests on `search_path`, which is session-scoped state.
351
+ Transaction-mode pooling hands a different backend to each transaction, so a `search_path`
352
+ set by one client is not guaranteed to be the one in effect for the next query. This affects
353
+ v3 and v4 alike; v4 reduces the number of `SET search_path` statements to one per connection
354
+ but does not eliminate them, and one is enough.
355
+
356
+ **Which pooler to use:** PgBouncer on **PostgreSQL 18+** with
357
+ `track_extra_parameters = IntervalStyle,search_path` is the only setup that safely multiplexes
358
+ a schema-per-tenant app — and it needs no changes to Apartment or your application. **RDS Proxy
359
+ is safe but reduces no connections**: a Rails app pins on it *with or without Apartment* (three
360
+ unconditional `SET`s in `configure_connection`, plus the extended query protocol even at
361
+ `prepared_statements: false` — [rails/rails#40207](https://github.com/rails/rails/issues/40207)),
362
+ and AWS offers no session-pinning filters for PostgreSQL. Keep RDS Proxy if you use it for
363
+ failover or IAM; don't adopt it to cut connections.
364
+
365
+ Database-per-tenant (`:database_name`) is not affected.
366
+
367
+ **Read [Connection Poolers](docs/connection-poolers.md) before putting a pooler in front of a
368
+ schema-per-tenant app**, and verify your configuration rather than assuming it: the failure
369
+ is silent, so a working app proves nothing.
370
+
337
371
  ### `connects_to` with Separate Databases
338
372
 
339
373
  If a model (or its abstract base class) uses `connects_to` to point at a completely different database (not just different roles on the same DB), Apartment's `connection_pool` patch will attempt to create a tenant pool for it.
@@ -20,7 +20,8 @@ lib/apartment/
20
20
  │ └── mysql_config.rb # MysqlConfig: placeholder
21
21
  ├── elevators/ # Rack middleware for tenant detection (see CLAUDE.md); v4 uses constructor keyword args, no class-level state; Generic, Subdomain, FirstSubdomain, Domain, Host, HostHash, Header
22
22
  ├── patches/ # ActiveRecord patches for tenant-aware connections
23
- └── connection_handling.rb # Prepends on AR::Base — tenant-aware connection_pool
23
+ ├── connection_handling.rb # Prepends on AR::Base — tenant-aware connection_pool
24
+ │ └── postgresql_sequence_name.rb # Prepends on the PG adapter — schema-agnostic Model.sequence_name memoization
24
25
  ├── tasks/ # Rake task utilities; v4.rake for apartment:create/drop/migrate/seed/rollback
25
26
  ├── config.rb # Configuration with validate!/freeze!
26
27
  ├── current.rb # Fiber-safe tenant context (CurrentAttributes)
@@ -59,7 +60,7 @@ lib/apartment/
59
60
 
60
61
  ### pool_manager.rb — Pool Cache
61
62
 
62
- `Concurrent::Map` storing connection pools by tenant key. Monotonic clock timestamps for idle/LRU tracking. `stats_for` returns `{ seconds_idle: N }`. `clear` disconnects all pools before clearing. When `max_total_connections` is set, `Apartment.configure` wires an `admission_controller` (the reaper) so cold creates route through a serialized, capacity-bounded path; otherwise the lock-free `compute_if_absent` fast path is used. See `docs/designs/pool-admission-control.md`.
63
+ `Concurrent::Map` storing connection pools by tenant key. Monotonic clock timestamps for idle/LRU tracking. `stats_for` returns `{ seconds_idle: N }`. `clear` disconnects all pools before clearing. When a pool budget is configured (`max_tenant_pools` and/or `max_tenant_connections`, resolved by `Config#effective_pool_budget`; `max_total_connections` is the deprecated alias of `max_tenant_pools`), `Apartment.configure` wires an `admission_controller` (the reaper) so cold creates route through a serialized, capacity-bounded path; otherwise the lock-free `compute_if_absent` fast path is used. See `docs/designs/pool-connection-budget.md` and `docs/designs/pool-admission-control.md`.
63
64
 
64
65
  ### pool_reaper.rb — Pool Eviction + Admission
65
66
 
@@ -97,7 +98,7 @@ All inherit from `AbstractAdapter`. Override `resolve_connection_config`, `creat
97
98
 
98
99
  ### pool_observer.rb — Observability (opt-in)
99
100
 
100
- Sink-agnostic subscriber for the pool events (`create`/`evict`/`cap_unmet`/`skip_evict`/`reaper_stopped`) + an optional `Concurrent::TimerTask` gauge sampler (`tenant_pools_live`, optional adopter `backend_count`). Normalizes each to a `Sample` and forwards to a caller `sink`; ships no transport. Error-isolated — never raises into instrumentation. See `docs/observability.md`.
101
+ Sink-agnostic subscriber for the pool events (`create`/`evict`/`cap_unmet`/`skip_evict`/`reaper_stopped`) + an optional `Concurrent::TimerTask` gauge sampler. Gauges: `tenant_pools_live`, optional adopter `backend_count`, and per-pass checkout-pressure gauges from each tenant pool's AR `ConnectionPool#stat` — `pools_waiting` (blocked on checkout), `pools_saturated` (`busy >= size`), `max_checkout_waiting` (worst pool's `waiting`, tenant in payload not a dimension). Per-pool `#stat` is rescued so one tearing-down pool can't abort the pass. Normalizes each to a `Sample` via the shared `emit_gauge` helper and forwards to a caller `sink`; ships no transport. Error-isolated — never raises into instrumentation. See `docs/observability.md`.
101
102
 
102
103
  ### railtie.rb — v4 Rails Integration
103
104
 
@@ -68,15 +68,13 @@ module Apartment
68
68
  end
69
69
 
70
70
  # Drop a tenant.
71
- def drop(tenant) # rubocop:disable Metrics/CyclomaticComplexity
71
+ def drop(tenant)
72
72
  drop_tenant(tenant)
73
73
  removed_pools = Apartment.pool_manager&.remove_tenant(tenant) || []
74
74
  removed_pools.each do |pool_key, pool|
75
- begin
76
- pool&.disconnect! if pool.respond_to?(:disconnect!)
77
- rescue StandardError => e
78
- warn "[Apartment] Pool disconnect failed for '#{pool_key}': #{e.class}: #{e.message}"
79
- end
75
+ # remove_tenant already took these out of the manager, so deregister_shard's
76
+ # own removal returns nil and cannot disconnect them — we close them here.
77
+ Apartment.disconnect_removed_pool(pool, pool_key)
80
78
  begin
81
79
  deregister_shard_from_ar_handler(pool_key)
82
80
  rescue StandardError => e
@@ -118,6 +116,16 @@ module Apartment
118
116
  false
119
117
  end
120
118
 
119
+ # Whether +conn+ sits in an aborted-transaction state that every subsequent
120
+ # statement will fail against until the transaction ends. PostgreSQL is the
121
+ # only supported engine with such a state (PQTRANS_INERROR); MySQL fails the
122
+ # statement and leaves the transaction usable, and its raw connection has no
123
+ # transaction_status at all. Base is conservative: never reclassify.
124
+ # See docs/designs/transaction-taint-detection.md (Evidence E).
125
+ def aborted_transaction?(_conn)
126
+ false
127
+ end
128
+
121
129
  # Request-path fail-safe contract. The elevator wraps the
122
130
  # tenant switch; on one of these error classes it asks
123
131
  # #tenant_container_gone? whether the tenant's storage actually vanished (a
@@ -10,6 +10,8 @@ module Apartment
10
10
  # to the environmentified tenant name. Lifecycle operations (create/drop)
11
11
  # execute DDL against the default connection.
12
12
  class PostgresqlDatabaseAdapter < AbstractAdapter
13
+ include PostgresqlTransactionState
14
+
13
15
  def resolve_connection_config(tenant, base_config: nil)
14
16
  config = base_config || send(:base_config)
15
17
  config.merge('database' => environmentify(tenant))
@@ -12,6 +12,8 @@ module Apartment
12
12
  # Apartment.config.postgres_config. Lifecycle operations (create/drop)
13
13
  # execute DDL against the default connection.
14
14
  class PostgresqlSchemaAdapter < AbstractAdapter
15
+ include PostgresqlTransactionState
16
+
15
17
  def shared_pinned_connection?
16
18
  !Apartment.config.force_separate_pinned_pool
17
19
  end
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Apartment
4
+ module Adapters
5
+ # PostgreSQL's aborted-transaction state, shared by both PG adapters
6
+ # (schema-per-tenant and database-per-tenant). Their implementations are
7
+ # identical, so this is the DRY seam rather than two copies.
8
+ #
9
+ # A failed statement inside a transaction moves the connection to
10
+ # PQTRANS_INERROR, where every subsequent statement raises
11
+ # PG::InFailedSqlTransaction until the transaction ends. Apartment heals this
12
+ # at pool checkin; see docs/designs/transaction-taint-detection.md.
13
+ module PostgresqlTransactionState
14
+ def aborted_transaction?(conn)
15
+ # NOT conn.raw_connection. That is the escape hatch for code about to *use*
16
+ # the driver, so it materializes lazy transactions, marks the connection
17
+ # dirty, and CONNECTS one that was never connected — at every checkin, on
18
+ # every pooled connection, which defeats Rails' lazy connect and burns a
19
+ # backend per tenant pool. We only read a status flag, so we take the handle
20
+ # directly. `connected?` keeps us off the never-connected path.
21
+ return false unless conn.connected?
22
+
23
+ raw = conn.instance_variable_get(:@raw_connection)
24
+ return false unless raw.respond_to?(:transaction_status)
25
+
26
+ raw.transaction_status == ::PG::PQTRANS_INERROR
27
+ rescue StandardError
28
+ # A connection too broken to report its own status is not ours to classify;
29
+ # AR's own verify!/reconnect path owns that case.
30
+ false
31
+ end
32
+ end
33
+ end
34
+ end
@@ -17,7 +17,8 @@ module Apartment
17
17
  attr_accessor :tenants_provider, :default_tenant,
18
18
  :default_tenant_switch_allowed,
19
19
  :tenant_pool_size, :pool_idle_timeout, :reaper_interval,
20
- :max_total_connections, :pool_overflow_policy,
20
+ :max_total_connections, :max_tenant_pools, :max_tenant_connections,
21
+ :pool_overflow_policy,
21
22
  :seed_after_create, :seed_data_file,
22
23
  :schema_load_strategy, :schema_file,
23
24
  :parallel_migration_threads,
@@ -26,7 +27,8 @@ module Apartment
26
27
  :active_record_log, :sql_query_tags,
27
28
  :shard_key_prefix,
28
29
  :migration_role, :app_role, :schema_cache_per_tenant, :check_pending_migrations,
29
- :force_separate_pinned_pool, :test_fixture_cleanup, :reap_in_test
30
+ :force_separate_pinned_pool, :test_fixture_cleanup, :reap_in_test,
31
+ :heal_tainted_connections
30
32
 
31
33
  def initialize # rubocop:disable Metrics/AbcSize
32
34
  @tenant_strategy = nil
@@ -38,6 +40,8 @@ module Apartment
38
40
  @pool_idle_timeout = 300
39
41
  @reaper_interval = nil
40
42
  @max_total_connections = nil
43
+ @max_tenant_pools = nil
44
+ @max_tenant_connections = nil
41
45
  @pool_overflow_policy = :evict_idle
42
46
  @seed_after_create = false
43
47
  @seed_data_file = nil
@@ -60,6 +64,13 @@ module Apartment
60
64
  @check_pending_migrations = true
61
65
  @force_separate_pinned_pool = false
62
66
  @reap_in_test = false
67
+ # Reset a tenant connection left in an aborted transaction when it is checked
68
+ # back into its pool. On by default: without it the poisoned connection is
69
+ # served to the next caller, and under pool-per-tenant that connection is the
70
+ # tenant's ONLY connection, so the tenant is dead on that worker until the
71
+ # process restarts. ActiveRecord's active? cannot detect the state.
72
+ # PostgreSQL-only in effect. See docs/designs/transaction-taint-detection.md.
73
+ @heal_tainted_connections = true
63
74
  @test_fixture_cleanup = true
64
75
  end
65
76
 
@@ -124,6 +135,17 @@ module Apartment
124
135
  # Reap on the idle-timeout cadence unless an explicit interval decouples
125
136
  # the two (reap more often without shrinking the idle window).
126
137
  @reaper_interval = @pool_idle_timeout if @reaper_interval.nil?
138
+
139
+ # max_total_connections is deprecated: the name said "connections" but it
140
+ # always capped tenant-pool COUNT. Alias it to its true meaning without
141
+ # changing behavior. Only fill when max_tenant_pools was not set explicitly,
142
+ # so validate!'s both-set guard can still catch a genuine double-spec.
143
+ return unless @max_total_connections
144
+
145
+ warn '[Apartment] DEPRECATION: config.max_total_connections is deprecated and will be ' \
146
+ 'removed in v5. It caps tenant-pool COUNT, not connections; rename it to ' \
147
+ 'max_tenant_pools. For a true connection ceiling, set max_tenant_connections.'
148
+ @max_tenant_pools = @max_total_connections if @max_tenant_pools.nil?
127
149
  end
128
150
 
129
151
  # Validate configuration completeness and consistency.
@@ -171,6 +193,35 @@ module Apartment
171
193
  "max_total_connections must be a positive integer or nil, got: #{@max_total_connections.inspect}")
172
194
  end
173
195
 
196
+ if @max_tenant_pools && (!@max_tenant_pools.is_a?(Integer) || @max_tenant_pools < 1)
197
+ raise(ConfigurationError,
198
+ "max_tenant_pools must be a positive integer or nil, got: #{@max_tenant_pools.inspect}")
199
+ end
200
+
201
+ if @max_total_connections && @max_tenant_pools && @max_total_connections != @max_tenant_pools
202
+ raise(ConfigurationError,
203
+ 'max_total_connections and max_tenant_pools are the same setting under two names; ' \
204
+ "set only one. Got max_total_connections=#{@max_total_connections}, " \
205
+ "max_tenant_pools=#{@max_tenant_pools}")
206
+ end
207
+
208
+ if @max_tenant_connections && (!@max_tenant_connections.is_a?(Integer) || @max_tenant_connections < 1)
209
+ raise(ConfigurationError,
210
+ "max_tenant_connections must be a positive integer or nil, got: #{@max_tenant_connections.inspect}")
211
+ end
212
+
213
+ if @max_tenant_connections && @tenant_pool_size.nil?
214
+ raise(ConfigurationError,
215
+ 'max_tenant_connections requires tenant_pool_size to be set (the pool budget is ' \
216
+ 'derived as max_tenant_connections / tenant_pool_size)')
217
+ end
218
+
219
+ if @max_tenant_connections && @tenant_pool_size && @max_tenant_connections < @tenant_pool_size
220
+ raise(ConfigurationError,
221
+ "max_tenant_connections (#{@max_tenant_connections}) must be >= tenant_pool_size " \
222
+ "(#{@tenant_pool_size}); it cannot fit a single tenant pool")
223
+ end
224
+
174
225
  unless %i[evict_idle raise].include?(@pool_overflow_policy)
175
226
  raise(ConfigurationError,
176
227
  'pool_overflow_policy must be :evict_idle or :raise, ' \
@@ -215,6 +266,12 @@ module Apartment
215
266
  "reap_in_test must be true or false, got: #{@reap_in_test.inspect}")
216
267
  end
217
268
 
269
+ unless [true, false].include?(@heal_tainted_connections)
270
+ raise(ConfigurationError,
271
+ 'heal_tainted_connections must be true or false, ' \
272
+ "got: #{@heal_tainted_connections.inspect}")
273
+ end
274
+
218
275
  unless @tenant_validator.nil? || @tenant_validator == false || @tenant_validator.respond_to?(:call)
219
276
  raise(ConfigurationError,
220
277
  'tenant_validator must be nil, false, or a callable, ' \
@@ -238,5 +295,15 @@ module Apartment
238
295
  def rails_env_name
239
296
  (Rails.env if defined?(Rails.env)) || ENV['RAILS_ENV'] || ENV['RACK_ENV'] || 'default_env'
240
297
  end
298
+
299
+ # The pool-count bound the admission controller enforces: the stricter of the
300
+ # explicit pool cap (max_tenant_pools) and the pool budget derived from the
301
+ # connection ceiling (max_tenant_connections / tenant_pool_size, floored).
302
+ # Returns nil (uncapped) when neither knob is set. Relies on a global
303
+ # tenant_pool_size, so tenant-pool connections <= budget * tenant_pool_size.
304
+ def effective_pool_budget
305
+ derived = @max_tenant_connections && @tenant_pool_size ? @max_tenant_connections / @tenant_pool_size : nil
306
+ [@max_tenant_pools, derived].compact.min
307
+ end
241
308
  end
242
309
  end
@@ -33,22 +33,31 @@ module Apartment
33
33
  # Raised when the tenant connection pool is exhausted.
34
34
  class PoolExhausted < ApartmentError; end
35
35
 
36
- # Raised when admitting a new tenant pool would exceed max_total_connections
37
- # and no idle pool can be evicted to make room, under the :raise overflow
38
- # policy. Distinct from PoolExhausted (a single pool's connections) — this is
39
- # the process-wide pool-count ceiling. See docs/designs/pool-admission-control.md.
36
+ # Raised when admitting a new tenant pool would exceed the effective pool
37
+ # budget (Config#effective_pool_budget) and no idle pool can be evicted to make
38
+ # room, under the :raise overflow policy. Distinct from PoolExhausted (a single
39
+ # pool's connections) — this is the process-wide pool-COUNT ceiling. The message
40
+ # names the knobs that set it, never the deprecated max_total_connections: that
41
+ # name is what made the budget look like a connection count in the first place.
42
+ # See docs/designs/pool-connection-budget.md and pool-admission-control.md.
40
43
  class PoolCapacityReached < ApartmentError
44
+ # +max_total+ is the effective POOL budget, not a connection count. The
45
+ # reader keeps its name for compatibility with anything already rescuing this.
41
46
  attr_reader :max_total, :current
42
47
 
48
+ alias pool_budget max_total
49
+
43
50
  def initialize(max_total: nil, current: nil)
44
51
  @max_total = max_total
45
52
  @current = current
46
53
  super(
47
- "Tenant pool capacity reached: #{current.inspect} pools open, " \
48
- "max_total_connections is #{max_total.inspect}, and no idle pool could " \
49
- 'be evicted to admit another (all pinned or in use). Raise ' \
50
- 'max_total_connections, reduce concurrent tenants, or set ' \
51
- 'pool_overflow_policy to :evict_idle to allow soft overflow.'
54
+ "Tenant pool capacity reached: #{current.inspect} tenant pools open, " \
55
+ "the effective pool budget is #{max_total.inspect}, and no idle pool " \
56
+ 'could be evicted to admit another (all pinned or in use). Raise ' \
57
+ 'max_tenant_pools, or raise max_tenant_connections (which derives the ' \
58
+ 'pool budget as max_tenant_connections / tenant_pool_size), reduce ' \
59
+ 'concurrent tenants, or set pool_overflow_policy to :evict_idle to ' \
60
+ 'allow soft overflow.'
52
61
  )
53
62
  end
54
63
  end
@@ -222,7 +222,11 @@ module Apartment
222
222
  role = Apartment.config.migration_role
223
223
  return unless role && Apartment.pool_manager
224
224
 
225
- Apartment.pool_manager.evict_by_role(role).each do |pool_key, _pool| # rubocop:disable Style/HashEachMethods
225
+ Apartment.pool_manager.evict_by_role(role).each do |pool_key, pool|
226
+ # evict_by_role already removed it from the manager, so deregister_shard has
227
+ # nothing left to disconnect; close it here or a pool AR no longer registers
228
+ # leaks its backend. See docs/designs/out-of-band-tenant-ddl.md.
229
+ Apartment.disconnect_removed_pool(pool, pool_key)
226
230
  Apartment.deregister_shard(pool_key)
227
231
  end
228
232
  rescue StandardError => e
@@ -86,15 +86,34 @@ module Apartment
86
86
  # to the reaper and to max_total accounting (a connection leak that also
87
87
  # undercounts the cap). Deregister it before re-raising so AR and the
88
88
  # manager stay consistent. The next request re-establishes cleanly.
89
+ #
90
+ # deregister_ar_shard, NOT deregister_shard: we are running inside
91
+ # PoolManager's create block, and the full form removes from PoolManager's
92
+ # Concurrent::Map — whose MRI backend guards compute_if_absent and delete
93
+ # with the same non-reentrant mutex, so that would raise ThreadError
94
+ # ("deadlock; recursive locking") and skip this deregistration entirely,
95
+ # orphaning the very pool this rescue exists to reclaim. There is nothing
96
+ # to remove from the manager here regardless: the pool is not stored until
97
+ # this block returns.
98
+ #
99
+ # Reached with +send+ because the AR-only half is private: it is a
100
+ # half-operation, and leaving one publicly reachable is the exact footgun
101
+ # this seam exists to close. This is the one place it is correct.
89
102
  begin
90
103
  raise(Apartment::PendingMigrationError, tenant) if check_pending_migrations?(pool)
91
104
 
92
105
  load_tenant_schema_cache(tenant, pool) if cfg.schema_cache_per_tenant
93
106
  rescue StandardError
94
- Apartment.deregister_shard(pool_key)
107
+ Apartment.send(:deregister_ar_shard, pool_key)
95
108
  raise
96
109
  end
97
110
 
111
+ # After the post-establish checks (a pool that fails them is discarded, so
112
+ # extending it would be wasted), and before the pool is handed out (so the
113
+ # very first checkin is already covered).
114
+ # See docs/designs/transaction-taint-detection.md.
115
+ Apartment::TransactionTaint.install(pool, tenant: tenant, pool_key: pool_key)
116
+
98
117
  pool
99
118
  end
100
119
  rescue Apartment::ApartmentError
@@ -0,0 +1,62 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Apartment
4
+ module Patches
5
+ # Keeps ActiveRecord's class-level Model.sequence_name memoization
6
+ # schema-agnostic under pool-per-tenant.
7
+ #
8
+ # Rails resolves default_sequence_name via pg_get_serial_sequence with an
9
+ # unqualified table name, so PostgreSQL answers through the current
10
+ # connection's search_path and returns a schema-QUALIFIED name — the schema
11
+ # of whichever tenant's pool happened to resolve it first. ActiveRecord then
12
+ # memoizes that value once per model class, process-wide. Consumers that
13
+ # render it into SQL (activerecord-import prefetches primary keys as a
14
+ # literal nextval(Model.sequence_name)) would draw ids from the
15
+ # first-resolver tenant's sequence in every tenant: wrong-tenant ids, silent
16
+ # sequence drift, and eventual PG::UniqueViolation.
17
+ #
18
+ # Stripping the connection's own current_schema prefix makes the memoized
19
+ # value schema-agnostic: nextval('widgets_id_seq') re-resolves through each
20
+ # pool's search_path, per tenant — the same invariant v4 relies on for table
21
+ # names (see docs/designs/v4-connection-model-rationale.md). A prefix naming
22
+ # any OTHER schema is preserved on purpose: persistent-schema tables and
23
+ # pinned models (whose table names are default_tenant-qualified and may
24
+ # execute on a shared tenant connection) are only correct BECAUSE they stay
25
+ # qualified.
26
+ #
27
+ # v3 shipped this guarantee as PostgreSqlAdapterPatch; it was lost in the
28
+ # Phase 2.5 v3 deletion (PR #356). Reset-on-switch is not an alternative:
29
+ # v4 serves tenants concurrently in one process, so only a schema-agnostic
30
+ # memoized value can be correct.
31
+ module PostgresqlSequenceName
32
+ def default_sequence_name(table_name, primary_key = 'id')
33
+ res = super
34
+ return res if res.nil?
35
+ # A schema-qualified table name is a PINNED model: Apartment qualifies
36
+ # those to the default tenant, and they resolve on whatever connection is
37
+ # current (the schema strategy shares the tenant's connection). Its
38
+ # sequence must stay qualified — stripping here is the mirror-image bug,
39
+ # because an unqualified name would later re-resolve against a *tenant's*
40
+ # search_path and draw ids from that tenant's stale copy of the pinned
41
+ # table's sequence. That order is the common one, not a corner case:
42
+ # pinned models are typically first touched at boot, on the default pool.
43
+ # v3 guarded the same case by force-adding the default_tenant prefix.
44
+ return res if table_name.to_s.include?('.')
45
+
46
+ # Routed table: drop whatever schema PostgreSQL qualified the sequence
47
+ # with — not just the connection's own. The two differ when the tenant
48
+ # schema is missing the table and search_path falls through to a schema
49
+ # that has it: the resolved sequence then names the FALLBACK schema, and
50
+ # preserving that would memoize it process-wide, so every tenant —
51
+ # including correctly-migrated ones — would draw from the fallback's
52
+ # sequence forever. Unqualified is the only value that lets the sequence
53
+ # re-resolve per pool exactly like the table name does.
54
+ #
55
+ # Parsed with AR's own splitter (rather than by hand) because it is what
56
+ # produced `res`, so it round-trips quoting and dotted identifiers.
57
+ ActiveRecord::ConnectionAdapters::PostgreSQL::Utils
58
+ .extract_schema_qualified_name(res).identifier
59
+ end
60
+ end
61
+ end
62
+ end
@@ -40,12 +40,22 @@ module Apartment
40
40
 
41
41
  # Delete pool first, then timestamp. This ordering prevents a concurrent
42
42
  # #get from orphaning a timestamp (get checks @pools, skips touch if absent).
43
+ #
44
+ # @api private
45
+ # Forgetting a pool here is only HALF of discarding it — the pool stays
46
+ # registered in AR's ConnectionHandler, leaking the registration and a live
47
+ # backend if the tenant is never re-accessed. Callers outside the gem want
48
+ # Apartment.deregister_shard (one pool) or Apartment.reset_tenant_pools! (all).
49
+ # These mutators also bypass PoolReaper's in-use guard: they will drop a pool
50
+ # with a checked-out connection or an open transaction.
51
+ # See docs/designs/out-of-band-tenant-ddl.md.
43
52
  def remove(tenant_key)
44
53
  pool = @pools.delete(tenant_key)
45
54
  @timestamps.delete(tenant_key)
46
55
  pool
47
56
  end
48
57
 
58
+ # @api private — see #remove.
49
59
  def remove_tenant(tenant)
50
60
  prefix = "#{tenant}:"
51
61
  removed = []
@@ -58,6 +68,7 @@ module Apartment
58
68
  removed
59
69
  end
60
70
 
71
+ # @api private — see #remove.
61
72
  def evict_by_role(role)
62
73
  suffix = ":#{role}"
63
74
  removed = []
@@ -66,14 +66,15 @@ module Apartment
66
66
  # when supplied. Safe to call from start_sampler! or an external scheduler.
67
67
  def sample!
68
68
  total = Apartment.pool_manager&.stats&.fetch(:total_pools, 0) || 0
69
- emit(Sample.new(name: :tenant_pools_live, kind: :gauge, value: total, dimensions: {}, payload: {}))
69
+ emit_gauge(:tenant_pools_live, total)
70
+ emit_checkout_pressure!
70
71
 
71
72
  return unless @backend_count
72
73
 
73
74
  backends = @backend_count.call
74
75
  return if backends.nil?
75
76
 
76
- emit(Sample.new(name: :backend_connections, kind: :gauge, value: backends, dimensions: {}, payload: {}))
77
+ emit_gauge(:backend_connections, backends)
77
78
  rescue StandardError => e
78
79
  warn_failure('sample!', e)
79
80
  end
@@ -104,6 +105,43 @@ module Apartment
104
105
  @sampler = nil
105
106
  end
106
107
 
108
+ # Per-tenant-pool checkout pressure, aggregated to low cardinality. waiting>0
109
+ # means threads are blocked acquiring a connection (the same-tenant fan-out
110
+ # starvation signal). The worst tenant is carried in the Sample payload, NOT
111
+ # as a metric dimension, to avoid unbounded time-series churn. Per-pool #stat
112
+ # is rescued so a pool tearing down mid-sample can't abort the whole pass.
113
+ def emit_checkout_pressure!
114
+ manager = Apartment.pool_manager
115
+ return unless manager
116
+
117
+ stats = collect_pool_stats(manager)
118
+ worst = stats.max_by { |s| s[:pending] } || { tenant: nil, pending: 0 }
119
+ payload = worst[:pending].positive? ? { tenant: worst[:tenant] } : {}
120
+
121
+ emit_gauge(:pools_waiting, stats.count { |s| s[:pending].positive? })
122
+ emit_gauge(:pools_saturated, stats.count { |s| s[:saturated] })
123
+ emit_gauge(:max_checkout_waiting, worst[:pending], payload: payload)
124
+ end
125
+
126
+ def emit_gauge(name, value, payload: {})
127
+ emit(Sample.new(name: name, kind: :gauge, value: value, dimensions: {}, payload: payload))
128
+ end
129
+
130
+ # Read each tenant pool's checkout stats into [{ tenant:, pending:, saturated: }],
131
+ # skipping any pool whose #stat raises (mid-teardown) so one bad pool can't
132
+ # abort the whole pass.
133
+ def collect_pool_stats(manager)
134
+ stats = []
135
+ manager.each_pair do |tenant_key, pool|
136
+ stat = pool.stat
137
+ stats << { tenant: tenant_key, pending: stat[:waiting].to_i,
138
+ saturated: stat[:busy].to_i >= stat[:size].to_i }
139
+ rescue StandardError
140
+ next
141
+ end
142
+ stats
143
+ end
144
+
107
145
  def record_event(event, payload)
108
146
  # Copy the notification payload so a sink that mutates Sample#payload
109
147
  # can't corrupt it for other subscribers of the same event.
@@ -142,6 +142,11 @@ module Apartment
142
142
  # on_evict with a nil pool.
143
143
  return nil unless (pool = @pool_manager.remove(tenant))
144
144
 
145
+ # We removed the pool ourselves, so deregister_shard's own removal returns nil
146
+ # and it has nothing left to disconnect. AR disconnects only a pool it still
147
+ # finds registered, so a pool that has drifted out of AR's handler would leak
148
+ # its backend unless we close it here. See docs/designs/out-of-band-tenant-ddl.md.
149
+ Apartment.disconnect_removed_pool(pool, tenant)
145
150
  deregister_from_ar_handler(tenant)
146
151
  Instrumentation.instrument(:evict, tenant: tenant, reason: reason)
147
152
  @on_evict&.call(tenant, pool)
@@ -0,0 +1,123 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Apartment
4
+ # Heals a tenant connection left in an aborted transaction (PostgreSQL's
5
+ # PQTRANS_INERROR) at the moment it is checked back into its pool.
6
+ #
7
+ # WHY CHECKIN. ActiveRecord's +active?+ probes with an empty query, which does
8
+ # NOT error in an aborted transaction, so +verify!+ pronounces a poisoned
9
+ # connection healthy and +checkin+ does not reset it. The pool then serves it to
10
+ # the next caller. Under pool-per-tenant that connection is the ONLY connection
11
+ # for its tenant, so the tenant is dead on that worker until the process
12
+ # restarts, while every other tenant looks fine and nothing in the logs explains
13
+ # it. That amplification is what makes this ours to fix: the defect is generic
14
+ # Rails/PG, the consequence is specific to pool-per-tenant.
15
+ #
16
+ # Checkin is also the one seam that serves both populations correctly. A
17
+ # FIXTURE-PINNED connection must KEEP its transaction (teardown_fixtures owns the
18
+ # rollback) and never reaches ConnectionPool#checkin's body; a production
19
+ # connection must be reset before reuse. We still test +pinned+ explicitly,
20
+ # because this override runs BEFORE the early return in AR's own +checkin+.
21
+ #
22
+ # NEVER issue a raw ROLLBACK here. It destroys the ENCLOSING transaction while
23
+ # ActiveRecord still believes its stack is intact, raises nothing, and lets
24
+ # subsequent writes autocommit — turning a loud failure into silent, permanent
25
+ # database pollution. +reset!+ is ActiveRecord's own primitive: it rolls back,
26
+ # DISCARDs session state, and resets AR's transaction bookkeeping together. It
27
+ # also re-applies the pool's schema_search_path, so a healed tenant connection
28
+ # still points at its own schema (verified — the alternative would be a
29
+ # cross-tenant leak, which is strictly worse than the bug being fixed).
30
+ #
31
+ # Design: docs/designs/transaction-taint-detection.md
32
+ module TransactionTaint
33
+ # Extended onto Apartment-owned tenant pools only. The primary pool and every
34
+ # app-owned pool are left alone: the same Rails defect reaches them, but they
35
+ # are not ours, and the blast radius there is one connection of N rather than a
36
+ # dead tenant. That gap is reported upstream, not patched here.
37
+ module PoolHeal
38
+ attr_accessor :apartment_tenant, :apartment_pool_key, :apartment_taint_warned
39
+
40
+ def checkin(conn)
41
+ TransactionTaint.heal(conn, self)
42
+ super
43
+ end
44
+ end
45
+
46
+ class << self
47
+ # Extend +pool+ with the heal. Called once, at tenant-pool creation.
48
+ def install(pool, tenant:, pool_key:)
49
+ return pool unless Apartment.config&.heal_tainted_connections
50
+
51
+ pool.extend(PoolHeal)
52
+ pool.apartment_tenant = tenant.to_s
53
+ pool.apartment_pool_key = pool_key
54
+ pool
55
+ end
56
+
57
+ # Best-effort, and it must never raise: an exception here would abort
58
+ # ConnectionPool#checkin and leak the connection out of the pool permanently,
59
+ # which is worse than the taint we came to fix.
60
+ def heal(conn, pool)
61
+ return if conn.pinned
62
+ return unless Apartment.adapter&.aborted_transaction?(conn)
63
+
64
+ open_transactions = conn.open_transactions
65
+ conn.reset!
66
+ warn_once(pool)
67
+ instrument(pool, open_transactions: open_transactions, healed: true)
68
+ rescue StandardError => e
69
+ discard(conn, pool, e)
70
+ end
71
+
72
+ private
73
+
74
+ # reset! failed, and +super+ is about to put this connection back in the pool's
75
+ # available list. Checking a STILL-POISONED connection back in would rebuild the
76
+ # exact outage we came to fix — the next lease gets it and the tenant stays dead.
77
+ # So drop the handle instead: disconnect! sends no SQL and cannot fail to nil out
78
+ # @raw_connection, so the pool keeps a dead shell that Rails transparently
79
+ # reconnects, fresh, on the next checkout. Checking in a disconnected connection
80
+ # is normal Rails (it is what flush/reap leave behind).
81
+ def discard(conn, pool, error)
82
+ conn.disconnect!
83
+ warn('[Apartment] could not reset the tainted connection for tenant ' \
84
+ "'#{pool.apartment_tenant}' (#{error.class}: #{error.message}); the connection " \
85
+ 'was dropped and will be reopened on next use.')
86
+ instrument(pool, open_transactions: nil, healed: false, error: error.class.name)
87
+ rescue StandardError
88
+ nil # Nothing further is safe to attempt inside checkin.
89
+ end
90
+
91
+ def instrument(pool, open_transactions:, healed:, error: nil)
92
+ Instrumentation.instrument(
93
+ 'transaction_taint',
94
+ tenant: pool.apartment_tenant,
95
+ pool_key: pool.apartment_pool_key,
96
+ open_transactions: open_transactions,
97
+ healed: healed,
98
+ error: error
99
+ )
100
+ end
101
+
102
+ # Once per pool per process. Healing means each taint is a distinct incident
103
+ # rather than a cascade, but an app that poisons on every request would still
104
+ # warn on every request — flooding the log during the exact incident where the
105
+ # signal matters. The notification fires every time; it is the countable
106
+ # channel, and the one to alert on.
107
+ def warn_once(pool)
108
+ return if pool.apartment_taint_warned
109
+
110
+ pool.apartment_taint_warned = true
111
+ warn(
112
+ "[Apartment] the connection for tenant '#{pool.apartment_tenant}' was checked in " \
113
+ 'while in an aborted transaction (PostgreSQL PQTRANS_INERROR) and has been reset. ' \
114
+ 'Something ran a statement that failed inside a transaction Rails did not unwind. ' \
115
+ 'Contain it by wrapping the failing call in ' \
116
+ 'ActiveRecord::Base.transaction(requires_new: true); subscribe to ' \
117
+ 'transaction_taint.apartment to find the call site. Do NOT add a raw ROLLBACK loop — ' \
118
+ 'see docs/testing.md.'
119
+ )
120
+ end
121
+ end
122
+ end
123
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Apartment
4
- VERSION = '4.0.0.alpha7'
4
+ VERSION = '4.0.0.alpha9'
5
5
  end
data/lib/apartment.rb CHANGED
@@ -213,23 +213,81 @@ module Apartment # rubocop:disable Metrics/ModuleLength
213
213
  ActiveRecord::QueryLogs.tags = ActiveRecord::QueryLogs.tags + [:tenant]
214
214
  end
215
215
 
216
- # Deregister a single tenant's shard from AR's ConnectionHandler.
216
+ # Discard one tenant pool, whole: deregister its shard from AR's ConnectionHandler
217
+ # and forget it in PoolManager, disconnecting it either way.
217
218
  # Safe to call when AR is not loaded or config is not set (no-op).
218
219
  # Used by PoolReaper eviction, AbstractAdapter#drop, and teardown.
220
+ #
221
+ # Both registries are updated because either one alone is wrong: deregistering
222
+ # from AR while the manager still holds the pool wedges the tenant permanently
223
+ # (the manager keeps handing back a pool AR has forgotten), and forgetting it in
224
+ # the manager alone leaks the AR registration and a live backend when the tenant
225
+ # is never re-accessed. Every internal caller already removes from the manager
226
+ # first, so the removal here is a no-op for them.
227
+ #
228
+ # The pool is disconnected here rather than left to AR, which disconnects only a
229
+ # pool it actually finds registered (ConnectionHandler#disconnect_pool_from_pool_manager
230
+ # guards `pool_config.disconnect!` behind `if pool_config`). A manager-held pool
231
+ # with no matching AR registration is reachable — the integration suite swaps the
232
+ # ConnectionHandler per example — and since we have just removed it from the
233
+ # manager, no later `PoolManager#clear` will disconnect it either. Mirrors
234
+ # AbstractAdapter#drop, which already removes, disconnects, then deregisters.
235
+ # (Callers that remove the pool from the manager THEMSELVES must disconnect it
236
+ # themselves too — the removal here returns nil for them, so there is nothing left
237
+ # for us to disconnect. PoolReaper#evict_tenant and Migrator#evict_migration_pools
238
+ # do exactly that.)
239
+ # See docs/designs/out-of-band-tenant-ddl.md.
240
+ #
241
+ # NOT SAFE inside a PoolManager create block — use +deregister_ar_shard+ there.
242
+ # PoolManager's @pools is a Concurrent::Map whose MRI backend guards
243
+ # +compute_if_absent+ and +delete+ with the SAME non-reentrant mutex, so removing
244
+ # a pool from inside the create block raises ThreadError ("deadlock; recursive
245
+ # locking"). The manager removal below is exactly that call.
246
+ #
247
+ # ORDER: AR first, manager removal in an +ensure+. The manager removal is an
248
+ # in-memory delete that cannot meaningfully fail, so it is the step that may run
249
+ # unconditionally; AR's removal does IO and is the one that can raise. Running the
250
+ # fallible step first, with the infallible one guaranteed after, means the call
251
+ # always ends with BOTH registries clear.
252
+ #
253
+ # Manager-first would be a race: between the manager delete and AR's removal, a
254
+ # concurrent tenant switch misses the manager, calls establish_connection — which
255
+ # RETURNS THE STILL-REGISTERED OLD POOL (ConnectionHandler#establish_connection
256
+ # reuses a pool whose db_config is equal) — and stores that doomed pool back in
257
+ # the manager. AR then unregisters and disconnects it, leaving the manager holding
258
+ # a dead pool: the permanent wedge this method exists to prevent, reintroduced.
259
+ # In this order the same interleaving costs at most one failed request, and the
260
+ # ensure clears the manager so the next switch rebuilds cleanly.
261
+ #
262
+ # Deliberately un-rescued at this level: both steps rescue their own failures, and
263
+ # swallowing everything here is what once hid a ThreadError, silently orphaning the
264
+ # pool the caller asked us to discard. Misuse should be loud.
219
265
  def deregister_shard(pool_key)
220
266
  return unless @config && defined?(ActiveRecord::Base)
221
267
 
222
- _, separator, role_str = pool_key.to_s.rpartition(':')
223
- role = separator.empty? || role_str.empty? ? ActiveRecord.writing_role : role_str.to_sym
268
+ begin
269
+ deregister_ar_shard(pool_key)
270
+ ensure
271
+ disconnect_removed_pool(@pool_manager&.remove(pool_key), pool_key)
272
+ end
273
+ end
224
274
 
225
- shard_key = :"#{@config.shard_key_prefix}_#{pool_key}"
226
- ActiveRecord::Base.connection_handler.remove_connection_pool(
227
- 'ActiveRecord::Base',
228
- role: role,
229
- shard: shard_key
230
- )
275
+ # @api private
276
+ # Disconnect a pool that has been removed from PoolManager. Idempotent with AR's
277
+ # own disconnect on the happy path (ConnectionPool#disconnect! empties @connections,
278
+ # so a second call has nothing left to close); load-bearing only when AR has no
279
+ # matching registration to disconnect, in which case nothing else will close it.
280
+ #
281
+ # Public because the callers that remove a pool from the manager THEMSELVES — and
282
+ # therefore get nil back from deregister_shard's own removal — must disconnect what
283
+ # they removed: PoolReaper#evict_tenant, Migrator#evict_migration_pools,
284
+ # AbstractAdapter#drop. Rescued so one broken pool cannot abort the caller.
285
+ def disconnect_removed_pool(pool, pool_key)
286
+ return unless pool.respond_to?(:disconnect!)
287
+
288
+ pool.disconnect!
231
289
  rescue StandardError => e
232
- warn "[Apartment] Failed to deregister AR pool for #{pool_key}: #{e.class}: #{e.message}"
290
+ warn "[Apartment] Failed to disconnect pool for #{pool_key}: #{e.class}: #{e.message}"
233
291
  end
234
292
 
235
293
  # Deregister all tenant pools from AR's ConnectionHandler and clear the
@@ -255,6 +313,32 @@ module Apartment # rubocop:disable Metrics/ModuleLength
255
313
 
256
314
  private
257
315
 
316
+ # The ActiveRecord half of a discard: deregister the shard (which disconnects the
317
+ # pool AR holds), leaving PoolManager untouched. The ONLY form that is safe inside
318
+ # a PoolManager create block — see the re-entrancy note on +deregister_shard+ —
319
+ # and correct there anyway: compute_if_absent does not store the pool until the
320
+ # block returns, so there is no manager entry to remove.
321
+ #
322
+ # PRIVATE ON PURPOSE. This is a half-operation, and a reachable half-operation is
323
+ # the bug this whole seam exists to eliminate: called on its own, it leaves AR
324
+ # without the pool while PoolManager keeps handing it out, wedging the tenant for
325
+ # the life of the process. The one legitimate caller reaches it with +send+.
326
+ def deregister_ar_shard(pool_key)
327
+ return unless @config && defined?(ActiveRecord::Base)
328
+
329
+ _, separator, role_str = pool_key.to_s.rpartition(':')
330
+ role = separator.empty? || role_str.empty? ? ActiveRecord.writing_role : role_str.to_sym
331
+
332
+ shard_key = :"#{@config.shard_key_prefix}_#{pool_key}"
333
+ ActiveRecord::Base.connection_handler.remove_connection_pool(
334
+ 'ActiveRecord::Base',
335
+ role: role,
336
+ shard: shard_key
337
+ )
338
+ rescue StandardError => e
339
+ warn "[Apartment] Failed to deregister AR pool for #{pool_key}: #{e.class}: #{e.message}"
340
+ end
341
+
258
342
  # Double-checked locking: the common path (already built) skips the mutex;
259
343
  # concurrent first callers serialize so exactly one validator is built.
260
344
  # TenantValidator.new subscribes to ActiveSupport::Notifications, so a
@@ -265,21 +349,23 @@ module Apartment # rubocop:disable Metrics/ModuleLength
265
349
  end
266
350
 
267
351
  # Build the pool manager + reaper for a freshly-validated config and start
268
- # the background reaper. When max_total_connections is set, wire the reaper
269
- # as the pool manager's admission controller so cold creates are bounded
270
- # synchronously; otherwise the manager keeps its lock-free create path.
352
+ # the background reaper. When a pool budget is configured (max_tenant_pools
353
+ # and/or max_tenant_connections, via Config#effective_pool_budget), wire the
354
+ # reaper as the pool manager's admission controller so cold creates are
355
+ # bounded synchronously; otherwise the manager keeps its lock-free create path.
271
356
  def setup_pools!(new_config)
357
+ budget = new_config.effective_pool_budget
272
358
  @pool_manager = PoolManager.new
273
359
  @pool_reaper = PoolReaper.new(
274
360
  pool_manager: @pool_manager,
275
361
  interval: new_config.reaper_interval,
276
362
  idle_timeout: new_config.pool_idle_timeout,
277
- max_total: new_config.max_total_connections,
363
+ max_total: budget,
278
364
  default_tenant: new_config.default_tenant,
279
365
  shard_key_prefix: new_config.shard_key_prefix,
280
366
  overflow_policy: new_config.pool_overflow_policy
281
367
  )
282
- @pool_manager.admission_controller = @pool_reaper if new_config.max_total_connections
368
+ @pool_manager.admission_controller = @pool_reaper if budget
283
369
  @pool_reaper.start
284
370
  end
285
371
 
@@ -370,6 +456,16 @@ module Apartment # rubocop:disable Metrics/ModuleLength
370
456
  end
371
457
  end
372
458
 
459
+ # Prepend the sequence-name patch whenever the PostgreSQL adapter loads
460
+ # (immediately, if it already has). Registered at gem load rather than in
461
+ # activate! because ActiveRecord memoizes Model.sequence_name at first touch,
462
+ # which can happen during boot before Apartment.activate! runs. No-op for apps
463
+ # that never load the PostgreSQL adapter, so MySQL/SQLite consumers never pull
464
+ # in pg. See the patch file for the full rationale.
465
+ ActiveSupport.on_load(:active_record_postgresqladapter) do
466
+ prepend(Apartment::Patches::PostgresqlSequenceName)
467
+ end
468
+
373
469
  # Load Railtie when Rails is present (standard gem convention).
374
470
  # Railtie is Zeitwerk-ignored — this explicit require is the only load path.
375
471
  require_relative 'apartment/railtie' if defined?(Rails::Railtie)
@@ -37,9 +37,20 @@ Apartment.configure do |config|
37
37
 
38
38
  # == Connection Pool =====================================================
39
39
 
40
- # config.tenant_pool_size = 5
41
- # config.pool_idle_timeout = 300
42
- # config.max_total_connections = nil
40
+ # tenant_pool_size MUST be >= the peak number of threads/fibers in one process
41
+ # that can touch the SAME tenant at once (e.g. a Sidekiq role's concurrency for a
42
+ # same-tenant job fan-out); otherwise those threads block on connection checkout.
43
+ # It is a lazy ceiling — connections are created on demand and idle pools reaped.
44
+ # config.tenant_pool_size = 5
45
+ # config.pool_idle_timeout = 300
46
+ #
47
+ # Bound how many tenant pools exist per process:
48
+ # config.max_tenant_pools = nil
49
+ # ...or bound total tenant-pool CONNECTIONS (requires tenant_pool_size); the
50
+ # admission controller derives the pool budget as floor(value / tenant_pool_size):
51
+ # config.max_tenant_connections = nil
52
+ #
53
+ # config.max_total_connections = nil # DEPRECATED: alias of max_tenant_pools; removed in v5
43
54
 
44
55
  # == Elevator (Request Tenant Detection) =================================
45
56
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ros-apartment
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.0.0.alpha7
4
+ version: 4.0.0.alpha9
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ryan Brunner
@@ -168,6 +168,7 @@ files:
168
168
  - lib/apartment/adapters/mysql2_adapter.rb
169
169
  - lib/apartment/adapters/postgresql_database_adapter.rb
170
170
  - lib/apartment/adapters/postgresql_schema_adapter.rb
171
+ - lib/apartment/adapters/postgresql_transaction_state.rb
171
172
  - lib/apartment/adapters/sqlite3_adapter.rb
172
173
  - lib/apartment/adapters/trilogy_adapter.rb
173
174
  - lib/apartment/cli.rb
@@ -194,6 +195,7 @@ files:
194
195
  - lib/apartment/migrator.rb
195
196
  - lib/apartment/patches/connection_handling.rb
196
197
  - lib/apartment/patches/live_tenant_propagation.rb
198
+ - lib/apartment/patches/postgresql_sequence_name.rb
197
199
  - lib/apartment/pool_manager.rb
198
200
  - lib/apartment/pool_observer.rb
199
201
  - lib/apartment/pool_reaper.rb
@@ -206,6 +208,7 @@ files:
206
208
  - lib/apartment/tenant_name_validator.rb
207
209
  - lib/apartment/tenant_validator.rb
208
210
  - lib/apartment/test_fixtures.rb
211
+ - lib/apartment/transaction_taint.rb
209
212
  - lib/apartment/version.rb
210
213
  - lib/generators/apartment/install/USAGE
211
214
  - lib/generators/apartment/install/install_generator.rb