solid_objects 0.7.3 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +26 -0
- data/docs/benchmarks.md +15 -0
- data/docs/realtime.md +30 -0
- data/docs/roadmap.md +27 -19
- data/lib/generators/solid_objects/templates/solid_objects.rb +8 -0
- data/lib/solid_objects/configuration.rb +10 -0
- data/lib/solid_objects/database_adapters/sqlite.rb +6 -4
- data/lib/solid_objects/supervisor.rb +125 -6
- data/lib/solid_objects/version.rb +1 -1
- data/lib/solid_objects/wake_up_adapters/postgresql.rb +136 -0
- data/lib/solid_objects/wake_up_adapters.rb +23 -0
- data/lib/solid_objects.rb +2 -0
- data/sig/generated/lib/solid_objects/configuration.rbs +10 -2
- data/sig/generated/lib/solid_objects/database_adapters/sqlite.rbs +2 -0
- data/sig/generated/lib/solid_objects/supervisor.rbs +45 -2
- data/sig/generated/lib/solid_objects/wake_up_adapters/postgresql.rbs +75 -0
- data/sig/generated/lib/solid_objects/wake_up_adapters.rbs +17 -0
- metadata +6 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 17e88ea06322a5e262d146e3d96a0b0d54d2339d46c6681a2fdb0f8d244a3640
|
|
4
|
+
data.tar.gz: d33b76b4e741921b4e199f3f91af652c06217960606dc7e3e95a03b28d34d3db
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0a217bfdb8454acbf7caa7a153095455e8b421aab4ebc7a16be056efa49db78bc4e0c452f50e24c04f1aad5177aafdb16ebff67a974618a0134c302581edf85a
|
|
7
|
+
data.tar.gz: 284a15eb44417ebaad3ec0ccad3b180999ed94df9d7ca73b74fcc70ed136f3458613ecbadb99115f13e93310d684a0451a62f6887dd7b0c2bdcf1af14ade2fcf
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.0 - 2026-08-10
|
|
4
|
+
|
|
5
|
+
- Replace a supervised role whose thread died. A role that raised left its
|
|
6
|
+
thread dead while the process kept running and quietly did less work; the
|
|
7
|
+
supervisor now restarts it until shutdown is requested. Prune dead process
|
|
8
|
+
records on an interval as part of the same monitor. Both intervals are
|
|
9
|
+
configurable through `supervisor_monitor_interval` and
|
|
10
|
+
`dead_process_cleanup_interval`.
|
|
11
|
+
- Run compatibility CI across the span the gemspec advertises: Ruby 3.3 and 3.4
|
|
12
|
+
against Rails 8.0 and 8.1. The suite previously ran on one combination, so
|
|
13
|
+
`>= 8.0` was a claim rather than a tested guarantee. Set `RAILS_VERSION` to
|
|
14
|
+
pin a Rails line locally.
|
|
15
|
+
- Stop a synchronous lock retry from asking for a negative wait when its
|
|
16
|
+
deadline expires between the check and the wait, which raised
|
|
17
|
+
`ArgumentError: time interval must not be negative` instead of the timeout
|
|
18
|
+
the caller expected. Found by the new compatibility matrix.
|
|
19
|
+
- Add `SolidObjects::WakeUpAdapters::Postgresql`, an optional cross-process
|
|
20
|
+
wake-up using PostgreSQL notifications. In-process signalling cannot reach a
|
|
21
|
+
worker process, so reactive delivery waited out `polling_interval`. With the
|
|
22
|
+
adapter configured, measured cross-process wake-up latency drops from 103.7 ms
|
|
23
|
+
to 2.9 ms at p50. The polling interval remains the upper bound, and neither
|
|
24
|
+
signalling nor waiting raises into its caller. `WakeUpAdapters.for` selects
|
|
25
|
+
notifications on PostgreSQL and the in-process default elsewhere; it is not
|
|
26
|
+
the default, because the adapter opens a connection per waiting thread
|
|
27
|
+
outside the pool and `LISTEN` does not survive a transaction-pooling proxy.
|
|
28
|
+
|
|
3
29
|
## 0.7.3 - 2026-08-09
|
|
4
30
|
|
|
5
31
|
- Coordinate batched component refreshes by revision as well as scope and batch
|
data/docs/benchmarks.md
CHANGED
|
@@ -76,6 +76,21 @@ Cable delivery, or browser rendering, which dominate wall-clock time in a real
|
|
|
76
76
|
deployment and make the request-count difference matter more than it appears
|
|
77
77
|
here. End-to-end latency against a deployed application has not been measured.
|
|
78
78
|
|
|
79
|
+
## Cross-process wake-up
|
|
80
|
+
|
|
81
|
+
Measured 2026-08-09 against PostgreSQL 17, 30 samples, with `polling_interval`
|
|
82
|
+
at its 100 ms default and a signal sent 2 ms after the waiter began.
|
|
83
|
+
|
|
84
|
+
| Wake-up strategy | p50 | p95 |
|
|
85
|
+
| --- | ---: | ---: |
|
|
86
|
+
| In-process `WakeUp` | 103.7 ms | 105.1 ms |
|
|
87
|
+
| `WakeUpAdapters::Postgresql` | 2.9 ms | 5.1 ms |
|
|
88
|
+
|
|
89
|
+
The in-process wake-up cannot reach another process, so a worker waits out the
|
|
90
|
+
full polling interval no matter how quickly the web process committed. The
|
|
91
|
+
notification adapter removes that floor rather than shrinking it, and the
|
|
92
|
+
polling interval remains the upper bound if a notification is missed.
|
|
93
|
+
|
|
79
94
|
## Durable row growth
|
|
80
95
|
|
|
81
96
|
The storage cost is deterministic even when latency is not:
|
data/docs/realtime.md
CHANGED
|
@@ -110,6 +110,36 @@ applications discover the namespaced engine asset. Applications created with
|
|
|
110
110
|
explicitly serve the module. Turbo's normal morph rules still apply; use
|
|
111
111
|
`data-turbo-permanent` for elements that must never be changed.
|
|
112
112
|
|
|
113
|
+
## Cross-process wake-up
|
|
114
|
+
|
|
115
|
+
Runtime roles poll for work and are woken early by an in-process signal. That
|
|
116
|
+
signal cannot cross process boundaries, so a commit in a Puma process does not
|
|
117
|
+
wake a broadcast executor in a worker process, and delivery waits out
|
|
118
|
+
`polling_interval`, 100 ms by default.
|
|
119
|
+
|
|
120
|
+
On PostgreSQL, install the notification adapter to remove that delay:
|
|
121
|
+
|
|
122
|
+
```ruby
|
|
123
|
+
# config/initializers/solid_objects.rb
|
|
124
|
+
configuration.wake_up_adapter = SolidObjects::WakeUpAdapters.for
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
`WakeUpAdapters.for` returns notifications on PostgreSQL and the in-process
|
|
128
|
+
default on SQLite and MySQL, so the same line is safe across adapters. Name
|
|
129
|
+
`SolidObjects::WakeUpAdapters::Postgresql.new` directly to require it.
|
|
130
|
+
|
|
131
|
+
MySQL has no notification primitive, so MySQL applications keep polling and tune
|
|
132
|
+
`polling_interval`.
|
|
133
|
+
|
|
134
|
+
Measured latency for a cross-process wake-up drops from 103.7 ms to 2.9 ms at
|
|
135
|
+
p50. The adapter keeps `polling_interval` as the upper bound: a missed or failed
|
|
136
|
+
notification costs latency, never correctness, and signalling never raises into
|
|
137
|
+
the caller that committed. `LISTEN` needs its own connection, so the adapter
|
|
138
|
+
opens one outside the pool and releases it on `stop`.
|
|
139
|
+
|
|
140
|
+
Applications on SQLite or MySQL, or that do not configure the adapter, keep the
|
|
141
|
+
existing polling behaviour.
|
|
142
|
+
|
|
113
143
|
## Batched component refreshes
|
|
114
144
|
|
|
115
145
|
A component refresh costs one browser request. When one actor mutation changes
|
data/docs/roadmap.md
CHANGED
|
@@ -31,23 +31,32 @@
|
|
|
31
31
|
- Bounded message/process pruning, actor-type opt-in instance expiration,
|
|
32
32
|
graceful caller shutdown, committed state snapshots, and an opt-in Minitest
|
|
33
33
|
helper
|
|
34
|
+
- Supervisor role replacement: a role whose thread dies is restarted until
|
|
35
|
+
shutdown is requested, and dead process records are pruned on an interval
|
|
34
36
|
- SQLite, PostgreSQL, and MySQL integration suites
|
|
37
|
+
- Opt-in cross-process wake-up on PostgreSQL through `WakeUpAdapters.for`, with
|
|
38
|
+
a listening connection per waiting thread and release on supervisor shutdown
|
|
35
39
|
- Inline RBS generation/validation, Steep, Standard Ruby, Solid Queue's exact
|
|
36
40
|
RuboCop policy, and a warning-free Brakeman scan
|
|
41
|
+
- Compatibility CI across the supported span: Ruby 3.3 and 3.4 against Rails 8.0
|
|
42
|
+
and 8.1, pinned through `RAILS_VERSION` so the advertised range is verified
|
|
43
|
+
rather than assumed
|
|
37
44
|
- A JavaScript suite covering the state payload and batched refresh browser
|
|
38
45
|
modules, run in CI with Node's test runner and jsdom, with every GitHub
|
|
39
46
|
Actions reference pinned to a commit SHA
|
|
40
47
|
|
|
41
48
|
## Partially implemented
|
|
42
49
|
|
|
43
|
-
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
50
|
+
- Wake-up strategy: in-process signaling, durable polling, injection, and an
|
|
51
|
+
opt-in PostgreSQL notification adapter are implemented; a Redis adapter is
|
|
52
|
+
not. In-process signaling cannot cross process boundaries, so without the
|
|
53
|
+
adapter a commit in a web process does not wake a broadcast executor in a
|
|
54
|
+
worker process and that delivery waits up to `polling_interval`, 100 ms by
|
|
55
|
+
default. `WakeUpAdapters.for` removes that delay on PostgreSQL, measured at
|
|
56
|
+
103.7 ms to 2.9 ms at p50. It is opt-in rather than automatic: it opens a
|
|
57
|
+
connection per waiting thread outside the pool, and `LISTEN` does not survive
|
|
58
|
+
a transaction-pooling proxy such as PgBouncer. MySQL has no notification
|
|
59
|
+
primitive, so MySQL applications keep polling.
|
|
51
60
|
- Realtime: scalar and dependency-driven keyed ERB component replacement or
|
|
52
61
|
morphing, personalized refresh authorization, revision fencing, coalescing,
|
|
53
62
|
reconnect convergence, batched refreshes, and personalized state payloads are
|
|
@@ -67,19 +76,18 @@
|
|
|
67
76
|
|
|
68
77
|
## Next milestones
|
|
69
78
|
|
|
70
|
-
1. Add
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
3. Add
|
|
75
|
-
4. Add
|
|
76
|
-
5. Add
|
|
77
|
-
6. Add
|
|
78
|
-
7. Add distributed rate limits, global admission hooks, and cache-capacity
|
|
79
|
+
1. Add an optional Redis wake-up adapter, which is the remaining cross-process
|
|
80
|
+
option for MySQL. The PostgreSQL notification adapter, its latency
|
|
81
|
+
benchmark, and its concurrency tests are implemented.
|
|
82
|
+
2. Add result lookup by request ID and broader deadlock retry classification.
|
|
83
|
+
3. Add scheduled retention and stale-process maintenance.
|
|
84
|
+
4. Add database/server-version checks and MySQL InnoDB verification at boot.
|
|
85
|
+
5. Add Turbo append intents and expand reconnect coverage in a full browser.
|
|
86
|
+
6. Add distributed rate limits, global admission hooks, and cache-capacity
|
|
79
87
|
eviction.
|
|
80
|
-
|
|
88
|
+
7. Expand security scanning and run compatibility CI across supported Rails and
|
|
81
89
|
Ruby versions.
|
|
82
|
-
|
|
90
|
+
8. Benchmark all workloads under documented hardware/database settings and
|
|
83
91
|
publish adapter-specific adoption measurements. Throughput, synchronous
|
|
84
92
|
latency, query counts, and the three reactive delivery paths are measured on
|
|
85
93
|
SQLite; adapter-specific and end-to-end browser measurements are not.
|
|
@@ -58,4 +58,12 @@ SolidObjects.configure do |configuration|
|
|
|
58
58
|
# configuration.authorize_administration = lambda do |authorization_context:, **|
|
|
59
59
|
# authorization_context.is_a?(Hash) && authorization_context[:source] == "cli"
|
|
60
60
|
# end
|
|
61
|
+
|
|
62
|
+
# Runtime roles poll for work and are woken early by an in-process signal,
|
|
63
|
+
# which cannot reach another process. On PostgreSQL, notifications remove that
|
|
64
|
+
# delay. This opens a connection per waiting thread outside the pool and does
|
|
65
|
+
# not work through a transaction-pooling proxy such as PgBouncer, so it is
|
|
66
|
+
# opt-in:
|
|
67
|
+
#
|
|
68
|
+
# configuration.wake_up_adapter = SolidObjects::WakeUpAdapters.for
|
|
61
69
|
end
|
|
@@ -21,6 +21,8 @@ module SolidObjects
|
|
|
21
21
|
# @rbs @process_heartbeat_interval: Float
|
|
22
22
|
# @rbs @process_alive_threshold: Float
|
|
23
23
|
# @rbs @shutdown_timeout: Float
|
|
24
|
+
# @rbs @supervisor_monitor_interval: Float
|
|
25
|
+
# @rbs @dead_process_cleanup_interval: Float
|
|
24
26
|
# @rbs @message_retention: Numeric
|
|
25
27
|
# @rbs @message_retention_by_actor_type: Hash[String, Numeric]
|
|
26
28
|
# @rbs @instance_retention_by_actor_type: Hash[String, Numeric]
|
|
@@ -62,6 +64,8 @@ module SolidObjects
|
|
|
62
64
|
:process_heartbeat_interval,
|
|
63
65
|
:process_alive_threshold,
|
|
64
66
|
:shutdown_timeout,
|
|
67
|
+
:supervisor_monitor_interval,
|
|
68
|
+
:dead_process_cleanup_interval,
|
|
65
69
|
:message_retention,
|
|
66
70
|
:message_retention_by_actor_type,
|
|
67
71
|
:instance_retention_by_actor_type,
|
|
@@ -102,6 +106,8 @@ module SolidObjects
|
|
|
102
106
|
@max_attempts = 5
|
|
103
107
|
@retry_delay = ->(attempt) { [ 2**(attempt - 1), 60 ].min.to_f }
|
|
104
108
|
@lock_retry_attempts = 10
|
|
109
|
+
@supervisor_monitor_interval = 1.0
|
|
110
|
+
@dead_process_cleanup_interval = 60.0
|
|
105
111
|
@process_heartbeat_interval = 15.0
|
|
106
112
|
@process_alive_threshold = 60.0
|
|
107
113
|
@shutdown_timeout = 15.0
|
|
@@ -138,6 +144,10 @@ module SolidObjects
|
|
|
138
144
|
raise ArgumentError, "table_name_prefix must contain lowercase letters, digits, and underscores"
|
|
139
145
|
end
|
|
140
146
|
|
|
147
|
+
unless supervisor_monitor_interval.positive?
|
|
148
|
+
raise ArgumentError, "supervisor_monitor_interval must be positive"
|
|
149
|
+
end
|
|
150
|
+
|
|
141
151
|
unless lease_duration > lease_renewal_interval
|
|
142
152
|
raise ArgumentError, "lease_duration must be greater than lease_renewal_interval"
|
|
143
153
|
end
|
|
@@ -160,13 +160,15 @@ module SolidObjects
|
|
|
160
160
|
end
|
|
161
161
|
end
|
|
162
162
|
|
|
163
|
+
# The deadline can expire between the check above and this wait, which
|
|
164
|
+
# would otherwise ask for a negative interval.
|
|
163
165
|
# @rbs () -> void
|
|
164
166
|
def wait_before_retry
|
|
167
|
+
interval = [ LOCK_RETRY_INTERVAL, SyncDeadline.remaining ].min
|
|
168
|
+
return unless interval.positive?
|
|
169
|
+
|
|
165
170
|
LOCK_RETRY_MUTEX.synchronize do
|
|
166
|
-
LOCK_RETRY_CONDITION.wait(
|
|
167
|
-
LOCK_RETRY_MUTEX,
|
|
168
|
-
[ LOCK_RETRY_INTERVAL, SyncDeadline.remaining ].min
|
|
169
|
-
)
|
|
171
|
+
LOCK_RETRY_CONDITION.wait(LOCK_RETRY_MUTEX, interval)
|
|
170
172
|
end
|
|
171
173
|
end
|
|
172
174
|
end
|
|
@@ -4,7 +4,10 @@ module SolidObjects
|
|
|
4
4
|
class Supervisor
|
|
5
5
|
# @rbs @components: Array[Worker | EffectExecutor | ReminderScheduler | BroadcastExecutor]
|
|
6
6
|
# @rbs @threads: Array[Thread]
|
|
7
|
+
# @rbs @monitor: Thread?
|
|
7
8
|
# @rbs @started: bool
|
|
9
|
+
# @rbs @cleaned_up_at: Float
|
|
10
|
+
# @rbs @lifecycle: Thread::Mutex
|
|
8
11
|
|
|
9
12
|
# @rbs (?worker_count: Integer, ?effect_worker_count: Integer, ?broadcast_worker_count: Integer, ?reminder_scheduler_count: Integer) -> void
|
|
10
13
|
def initialize(
|
|
@@ -20,7 +23,10 @@ module SolidObjects
|
|
|
20
23
|
reminder_scheduler_count:
|
|
21
24
|
)
|
|
22
25
|
@threads = []
|
|
26
|
+
@monitor = nil
|
|
23
27
|
@started = false
|
|
28
|
+
@cleaned_up_at = nil
|
|
29
|
+
@lifecycle = Thread::Mutex.new
|
|
24
30
|
end
|
|
25
31
|
|
|
26
32
|
# @rbs () -> void
|
|
@@ -36,7 +42,8 @@ module SolidObjects
|
|
|
36
42
|
return if @started
|
|
37
43
|
|
|
38
44
|
@started = true
|
|
39
|
-
@threads = components.map { |component|
|
|
45
|
+
@threads = components.map { |component| supervise(component) }
|
|
46
|
+
@monitor = Thread.new { monitor_loop }
|
|
40
47
|
SolidObjects.instrument(:"supervisor.started", component_count: components.length)
|
|
41
48
|
end
|
|
42
49
|
|
|
@@ -44,17 +51,129 @@ module SolidObjects
|
|
|
44
51
|
def stop
|
|
45
52
|
return unless @started
|
|
46
53
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
54
|
+
begin
|
|
55
|
+
# Flipping the flag under the same lock replacement takes means a
|
|
56
|
+
# replacement either completes before shutdown reads the component
|
|
57
|
+
# list, or never starts.
|
|
58
|
+
@lifecycle.synchronize { @started = false }
|
|
59
|
+
stop_monitor
|
|
60
|
+
components.each(&:request_shutdown)
|
|
61
|
+
join_until_timeout
|
|
62
|
+
components.reject(&:stopped?).each(&:stop)
|
|
63
|
+
ensure
|
|
64
|
+
# Connections held outside the pool must be released even when a
|
|
65
|
+
# component fails to stop, or they accumulate across restarts.
|
|
66
|
+
release_wake_up
|
|
67
|
+
@monitor = nil
|
|
68
|
+
SolidObjects.instrument(:"supervisor.stopped", component_count: components.length)
|
|
69
|
+
end
|
|
52
70
|
end
|
|
53
71
|
|
|
54
72
|
private
|
|
55
73
|
|
|
56
74
|
attr_reader :components, :threads
|
|
57
75
|
|
|
76
|
+
# A role that raises leaves its thread dead. Without replacement the
|
|
77
|
+
# process keeps running while quietly doing less work, so the supervisor
|
|
78
|
+
# watches its threads and restarts any that stopped before shutdown.
|
|
79
|
+
# A failing pass must not stop supervision, and must not retry without
|
|
80
|
+
# pacing either: a persistently failing database would otherwise spin.
|
|
81
|
+
# @rbs () -> void
|
|
82
|
+
def monitor_loop
|
|
83
|
+
while @started
|
|
84
|
+
begin
|
|
85
|
+
replace_dead_roles
|
|
86
|
+
cleanup_dead_processes
|
|
87
|
+
rescue => error
|
|
88
|
+
SolidObjects.instrument(
|
|
89
|
+
:"supervisor.monitor_failed",
|
|
90
|
+
error_class: error.class.name,
|
|
91
|
+
error_message: error.message
|
|
92
|
+
)
|
|
93
|
+
end
|
|
94
|
+
sleep SolidObjects.configuration.supervisor_monitor_interval
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# A role that raises runs its own shutdown cleanup on the way out, so a
|
|
99
|
+
# crashed component reports itself stopped exactly like one that was asked
|
|
100
|
+
# to stop. While the supervisor is still running, a dead thread can only
|
|
101
|
+
# mean a crash, so replacement keys on the supervisor rather than on the
|
|
102
|
+
# component. The crashed instance has already released its process record,
|
|
103
|
+
# so a fresh one takes its place.
|
|
104
|
+
# @rbs () -> void
|
|
105
|
+
def replace_dead_roles
|
|
106
|
+
components.each_with_index do |component, index|
|
|
107
|
+
thread = threads[index]
|
|
108
|
+
next if thread&.alive?
|
|
109
|
+
|
|
110
|
+
replaced = @lifecycle.synchronize do
|
|
111
|
+
next false unless @started
|
|
112
|
+
|
|
113
|
+
replacement = component.class.new
|
|
114
|
+
components[index] = replacement
|
|
115
|
+
threads[index] = supervise(replacement)
|
|
116
|
+
replacement
|
|
117
|
+
end
|
|
118
|
+
break unless replaced
|
|
119
|
+
|
|
120
|
+
SolidObjects.instrument(
|
|
121
|
+
:"supervisor.role_replaced",
|
|
122
|
+
role: replaced.class.name,
|
|
123
|
+
error_class: thread_error(thread)
|
|
124
|
+
)
|
|
125
|
+
end
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
# @rbs (Thread?) -> String?
|
|
129
|
+
def thread_error(thread)
|
|
130
|
+
thread&.join
|
|
131
|
+
nil
|
|
132
|
+
rescue => error
|
|
133
|
+
error.class.name
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
# @rbs () -> void
|
|
137
|
+
def cleanup_dead_processes
|
|
138
|
+
interval = SolidObjects.configuration.dead_process_cleanup_interval
|
|
139
|
+
return unless interval.positive?
|
|
140
|
+
return if @cleaned_up_at && monotonic_now - @cleaned_up_at < interval
|
|
141
|
+
|
|
142
|
+
@cleaned_up_at = monotonic_now
|
|
143
|
+
ProcessRegistry.cleanup_dead
|
|
144
|
+
end
|
|
145
|
+
|
|
146
|
+
# @rbs (untyped) -> Thread
|
|
147
|
+
def supervise(component)
|
|
148
|
+
Thread.new { component.run }
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
# The monitor only performs maintenance, so shutdown must never return while
|
|
152
|
+
# it is still alive: a pass blocked on the database would otherwise outlive
|
|
153
|
+
# the supervisor that owns it.
|
|
154
|
+
# @rbs () -> void
|
|
155
|
+
def stop_monitor
|
|
156
|
+
monitor = @monitor
|
|
157
|
+
@monitor = nil
|
|
158
|
+
return unless monitor
|
|
159
|
+
|
|
160
|
+
monitor.join(SolidObjects.configuration.shutdown_timeout)
|
|
161
|
+
monitor.kill if monitor.alive?
|
|
162
|
+
monitor.join(SolidObjects.configuration.supervisor_monitor_interval)
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
# A wake-up adapter may hold connections outside the pool, which would
|
|
166
|
+
# otherwise accumulate across restarts in one process.
|
|
167
|
+
# @rbs () -> void
|
|
168
|
+
def release_wake_up
|
|
169
|
+
wake_up = SolidObjects.wake_up
|
|
170
|
+
return unless wake_up.respond_to?(:stop)
|
|
171
|
+
|
|
172
|
+
wake_up.stop
|
|
173
|
+
rescue
|
|
174
|
+
nil
|
|
175
|
+
end
|
|
176
|
+
|
|
58
177
|
# @rbs (worker_count: Integer, effect_worker_count: Integer, broadcast_worker_count: Integer, reminder_scheduler_count: Integer) -> Array[Worker | EffectExecutor | ReminderScheduler | BroadcastExecutor]
|
|
59
178
|
def build_components(
|
|
60
179
|
worker_count:,
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# rbs_inline: enabled
|
|
2
|
+
|
|
3
|
+
module SolidObjects
|
|
4
|
+
module WakeUpAdapters
|
|
5
|
+
# Wakes runtime roles across processes using PostgreSQL notifications.
|
|
6
|
+
#
|
|
7
|
+
# The in-process wake-up cannot reach another process, so a commit in a web
|
|
8
|
+
# process leaves a worker waiting out its polling interval. This adapter
|
|
9
|
+
# keeps that polling interval as the upper bound and delivers a notification
|
|
10
|
+
# when one is available, so a missed or failed notification costs latency
|
|
11
|
+
# rather than correctness.
|
|
12
|
+
class Postgresql
|
|
13
|
+
CHANNEL = "solid_objects_wake_up"
|
|
14
|
+
FAILED_WAIT_INTERVAL = 0.05
|
|
15
|
+
|
|
16
|
+
# @rbs @channel: String
|
|
17
|
+
# @rbs @mutex: Thread::Mutex
|
|
18
|
+
# @rbs @connections: Array[untyped]
|
|
19
|
+
|
|
20
|
+
attr_reader :channel
|
|
21
|
+
|
|
22
|
+
# @rbs (?channel: String) -> void
|
|
23
|
+
def initialize(channel: CHANNEL)
|
|
24
|
+
@channel = channel
|
|
25
|
+
@mutex = Thread::Mutex.new
|
|
26
|
+
@connections = []
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# @rbs () -> bool
|
|
30
|
+
def signal
|
|
31
|
+
notify_channel
|
|
32
|
+
true
|
|
33
|
+
rescue => error
|
|
34
|
+
instrument_failure(:signal, error)
|
|
35
|
+
false
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# @rbs (timeout: Numeric) -> bool
|
|
39
|
+
def wait(timeout:)
|
|
40
|
+
connection = listening_connection
|
|
41
|
+
!connection.raw_connection.wait_for_notify(timeout.to_f).nil?
|
|
42
|
+
rescue => error
|
|
43
|
+
instrument_failure(:wait, error)
|
|
44
|
+
pace_after_failure(timeout)
|
|
45
|
+
false
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# Starts listening before a caller blocks, so a notification sent between
|
|
49
|
+
# startup and the first wait is not missed.
|
|
50
|
+
# @rbs () -> bool
|
|
51
|
+
def listen
|
|
52
|
+
listening_connection
|
|
53
|
+
true
|
|
54
|
+
rescue => error
|
|
55
|
+
instrument_failure(:listen, error)
|
|
56
|
+
false
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# @rbs () -> bool
|
|
60
|
+
def stop
|
|
61
|
+
open = mutex.synchronize do
|
|
62
|
+
listening = connections.dup
|
|
63
|
+
connections.clear
|
|
64
|
+
listening
|
|
65
|
+
end
|
|
66
|
+
Thread.current[thread_key] = nil
|
|
67
|
+
open.each { |connection| disconnect(connection) }
|
|
68
|
+
open.any?
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
private
|
|
72
|
+
|
|
73
|
+
attr_reader :mutex, :connections
|
|
74
|
+
|
|
75
|
+
# @rbs () -> void
|
|
76
|
+
def notify_channel
|
|
77
|
+
Record.connection_pool.with_connection do |connection|
|
|
78
|
+
connection.execute("NOTIFY #{connection.quote_table_name(channel)}")
|
|
79
|
+
end
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# A listening connection is dedicated and per thread. `LISTEN` is per
|
|
83
|
+
# connection, a blocking wait must not hold a connection the rest of the
|
|
84
|
+
# runtime needs, and one connection cannot serve concurrent waiters: the
|
|
85
|
+
# supervisor shares one adapter across roles, and a notification consumed
|
|
86
|
+
# by one waiter would leave the others asleep until their poll expired.
|
|
87
|
+
# @rbs () -> untyped
|
|
88
|
+
def listening_connection
|
|
89
|
+
connection = Thread.current[thread_key]
|
|
90
|
+
return connection if connection&.active?
|
|
91
|
+
|
|
92
|
+
open_listening_connection
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
# @rbs () -> untyped
|
|
96
|
+
def open_listening_connection
|
|
97
|
+
connection = Record.connection_pool.send(:new_connection)
|
|
98
|
+
connection.execute("LISTEN #{connection.quote_table_name(channel)}")
|
|
99
|
+
Thread.current[thread_key] = connection
|
|
100
|
+
mutex.synchronize { connections << connection }
|
|
101
|
+
connection
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
# @rbs () -> Symbol
|
|
105
|
+
def thread_key
|
|
106
|
+
:"solid_objects_wake_up_#{object_id}"
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
# @rbs (untyped) -> void
|
|
110
|
+
def disconnect(connection)
|
|
111
|
+
connection.disconnect!
|
|
112
|
+
rescue
|
|
113
|
+
nil
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
# @rbs (Numeric) -> void
|
|
117
|
+
def pace_after_failure(timeout)
|
|
118
|
+
interval = [ timeout.to_f, FAILED_WAIT_INTERVAL ].min
|
|
119
|
+
return unless interval.positive?
|
|
120
|
+
|
|
121
|
+
sleep interval
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# @rbs (Symbol, Exception) -> void
|
|
125
|
+
def instrument_failure(operation, error)
|
|
126
|
+
SolidObjects.instrument(
|
|
127
|
+
:"wake_up.failed",
|
|
128
|
+
adapter: "postgresql",
|
|
129
|
+
operation: operation.to_s,
|
|
130
|
+
error_class: error.class.name,
|
|
131
|
+
error_message: error.message
|
|
132
|
+
)
|
|
133
|
+
end
|
|
134
|
+
end
|
|
135
|
+
end
|
|
136
|
+
end
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# rbs_inline: enabled
|
|
2
|
+
|
|
3
|
+
module SolidObjects
|
|
4
|
+
module WakeUpAdapters
|
|
5
|
+
module_function
|
|
6
|
+
|
|
7
|
+
# Returns the best wake-up strategy for a connection: cross-process
|
|
8
|
+
# notifications where the database provides them, and the in-process
|
|
9
|
+
# default everywhere else.
|
|
10
|
+
#
|
|
11
|
+
# This is deliberately not the default. A notification adapter opens a
|
|
12
|
+
# connection per waiting thread outside the pool, and `LISTEN` does not
|
|
13
|
+
# survive a transaction-pooling proxy such as PgBouncer, so adopting it is
|
|
14
|
+
# a deployment decision rather than an upgrade side effect.
|
|
15
|
+
#
|
|
16
|
+
# @rbs (?untyped) -> untyped
|
|
17
|
+
def for(connection = Record.connection)
|
|
18
|
+
return Postgresql.new if connection.adapter_name.match?(/postgres/i)
|
|
19
|
+
|
|
20
|
+
WakeUp.new
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
data/lib/solid_objects.rb
CHANGED
|
@@ -46,6 +46,8 @@ require "solid_objects/actor_view"
|
|
|
46
46
|
require "solid_objects/actor_channel"
|
|
47
47
|
require "solid_objects/action_cable_broadcast_adapter"
|
|
48
48
|
require "solid_objects/wake_up"
|
|
49
|
+
require "solid_objects/wake_up_adapters/postgresql"
|
|
50
|
+
require "solid_objects/wake_up_adapters"
|
|
49
51
|
require "solid_objects/effect_registry"
|
|
50
52
|
require "solid_objects/commit_action_registry"
|
|
51
53
|
require "solid_objects/lease"
|
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
module SolidObjects
|
|
4
4
|
class Configuration
|
|
5
|
-
@process_alive_threshold: Float
|
|
6
|
-
|
|
7
5
|
@shutdown_timeout: Float
|
|
8
6
|
|
|
7
|
+
@supervisor_monitor_interval: Float
|
|
8
|
+
|
|
9
|
+
@dead_process_cleanup_interval: Float
|
|
10
|
+
|
|
9
11
|
@message_retention: Numeric
|
|
10
12
|
|
|
11
13
|
@message_retention_by_actor_type: Hash[String, Numeric]
|
|
@@ -82,6 +84,8 @@ module SolidObjects
|
|
|
82
84
|
|
|
83
85
|
@process_heartbeat_interval: Float
|
|
84
86
|
|
|
87
|
+
@process_alive_threshold: Float
|
|
88
|
+
|
|
85
89
|
attr_accessor table_name_prefix: untyped
|
|
86
90
|
|
|
87
91
|
attr_accessor polling_interval: untyped
|
|
@@ -120,6 +124,10 @@ module SolidObjects
|
|
|
120
124
|
|
|
121
125
|
attr_accessor shutdown_timeout: untyped
|
|
122
126
|
|
|
127
|
+
attr_accessor supervisor_monitor_interval: untyped
|
|
128
|
+
|
|
129
|
+
attr_accessor dead_process_cleanup_interval: untyped
|
|
130
|
+
|
|
123
131
|
attr_accessor message_retention: untyped
|
|
124
132
|
|
|
125
133
|
attr_accessor message_retention_by_actor_type: untyped
|
|
@@ -52,6 +52,8 @@ module SolidObjects
|
|
|
52
52
|
# @rbs (Integer) -> void
|
|
53
53
|
def wait_before_busy_retry: (Integer) -> void
|
|
54
54
|
|
|
55
|
+
# The deadline can expire between the check above and this wait, which
|
|
56
|
+
# would otherwise ask for a negative interval.
|
|
55
57
|
# @rbs () -> void
|
|
56
58
|
def wait_before_retry: () -> void
|
|
57
59
|
end
|
|
@@ -2,12 +2,18 @@
|
|
|
2
2
|
|
|
3
3
|
module SolidObjects
|
|
4
4
|
class Supervisor
|
|
5
|
-
@
|
|
5
|
+
@lifecycle: Thread::Mutex
|
|
6
6
|
|
|
7
|
-
@
|
|
7
|
+
@cleaned_up_at: Float
|
|
8
8
|
|
|
9
9
|
@started: bool
|
|
10
10
|
|
|
11
|
+
@monitor: Thread?
|
|
12
|
+
|
|
13
|
+
@threads: Array[Thread]
|
|
14
|
+
|
|
15
|
+
@components: Array[Worker | EffectExecutor | ReminderScheduler | BroadcastExecutor]
|
|
16
|
+
|
|
11
17
|
# @rbs (?worker_count: Integer, ?effect_worker_count: Integer, ?broadcast_worker_count: Integer, ?reminder_scheduler_count: Integer) -> void
|
|
12
18
|
def initialize: (?worker_count: Integer, ?effect_worker_count: Integer, ?broadcast_worker_count: Integer, ?reminder_scheduler_count: Integer) -> void
|
|
13
19
|
|
|
@@ -26,6 +32,43 @@ module SolidObjects
|
|
|
26
32
|
|
|
27
33
|
attr_reader threads: untyped
|
|
28
34
|
|
|
35
|
+
# A role that raises leaves its thread dead. Without replacement the
|
|
36
|
+
# process keeps running while quietly doing less work, so the supervisor
|
|
37
|
+
# watches its threads and restarts any that stopped before shutdown.
|
|
38
|
+
# A failing pass must not stop supervision, and must not retry without
|
|
39
|
+
# pacing either: a persistently failing database would otherwise spin.
|
|
40
|
+
# @rbs () -> void
|
|
41
|
+
def monitor_loop: () -> void
|
|
42
|
+
|
|
43
|
+
# A role that raises runs its own shutdown cleanup on the way out, so a
|
|
44
|
+
# crashed component reports itself stopped exactly like one that was asked
|
|
45
|
+
# to stop. While the supervisor is still running, a dead thread can only
|
|
46
|
+
# mean a crash, so replacement keys on the supervisor rather than on the
|
|
47
|
+
# component. The crashed instance has already released its process record,
|
|
48
|
+
# so a fresh one takes its place.
|
|
49
|
+
# @rbs () -> void
|
|
50
|
+
def replace_dead_roles: () -> void
|
|
51
|
+
|
|
52
|
+
# @rbs (Thread?) -> String?
|
|
53
|
+
def thread_error: (Thread?) -> String?
|
|
54
|
+
|
|
55
|
+
# @rbs () -> void
|
|
56
|
+
def cleanup_dead_processes: () -> void
|
|
57
|
+
|
|
58
|
+
# @rbs (untyped) -> Thread
|
|
59
|
+
def supervise: (untyped) -> Thread
|
|
60
|
+
|
|
61
|
+
# The monitor only performs maintenance, so shutdown must never return while
|
|
62
|
+
# it is still alive: a pass blocked on the database would otherwise outlive
|
|
63
|
+
# the supervisor that owns it.
|
|
64
|
+
# @rbs () -> void
|
|
65
|
+
def stop_monitor: () -> void
|
|
66
|
+
|
|
67
|
+
# A wake-up adapter may hold connections outside the pool, which would
|
|
68
|
+
# otherwise accumulate across restarts in one process.
|
|
69
|
+
# @rbs () -> void
|
|
70
|
+
def release_wake_up: () -> void
|
|
71
|
+
|
|
29
72
|
# @rbs (worker_count: Integer, effect_worker_count: Integer, broadcast_worker_count: Integer, reminder_scheduler_count: Integer) -> Array[Worker | EffectExecutor | ReminderScheduler | BroadcastExecutor]
|
|
30
73
|
def build_components: (worker_count: Integer, effect_worker_count: Integer, broadcast_worker_count: Integer, reminder_scheduler_count: Integer) -> Array[Worker | EffectExecutor | ReminderScheduler | BroadcastExecutor]
|
|
31
74
|
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Generated from lib/solid_objects/wake_up_adapters/postgresql.rb with RBS::Inline
|
|
2
|
+
|
|
3
|
+
module SolidObjects
|
|
4
|
+
module WakeUpAdapters
|
|
5
|
+
# Wakes runtime roles across processes using PostgreSQL notifications.
|
|
6
|
+
#
|
|
7
|
+
# The in-process wake-up cannot reach another process, so a commit in a web
|
|
8
|
+
# process leaves a worker waiting out its polling interval. This adapter
|
|
9
|
+
# keeps that polling interval as the upper bound and delivers a notification
|
|
10
|
+
# when one is available, so a missed or failed notification costs latency
|
|
11
|
+
# rather than correctness.
|
|
12
|
+
class Postgresql
|
|
13
|
+
CHANNEL: ::String
|
|
14
|
+
|
|
15
|
+
FAILED_WAIT_INTERVAL: ::Float
|
|
16
|
+
|
|
17
|
+
@connections: Array[untyped]
|
|
18
|
+
|
|
19
|
+
@mutex: Thread::Mutex
|
|
20
|
+
|
|
21
|
+
@channel: String
|
|
22
|
+
|
|
23
|
+
attr_reader channel: untyped
|
|
24
|
+
|
|
25
|
+
# @rbs (?channel: String) -> void
|
|
26
|
+
def initialize: (?channel: String) -> void
|
|
27
|
+
|
|
28
|
+
# @rbs () -> bool
|
|
29
|
+
def signal: () -> bool
|
|
30
|
+
|
|
31
|
+
# @rbs (timeout: Numeric) -> bool
|
|
32
|
+
def wait: (timeout: Numeric) -> bool
|
|
33
|
+
|
|
34
|
+
# Starts listening before a caller blocks, so a notification sent between
|
|
35
|
+
# startup and the first wait is not missed.
|
|
36
|
+
# @rbs () -> bool
|
|
37
|
+
def listen: () -> bool
|
|
38
|
+
|
|
39
|
+
# @rbs () -> bool
|
|
40
|
+
def stop: () -> bool
|
|
41
|
+
|
|
42
|
+
private
|
|
43
|
+
|
|
44
|
+
attr_reader mutex: untyped
|
|
45
|
+
|
|
46
|
+
attr_reader connections: untyped
|
|
47
|
+
|
|
48
|
+
# @rbs () -> void
|
|
49
|
+
def notify_channel: () -> void
|
|
50
|
+
|
|
51
|
+
# A listening connection is dedicated and per thread. `LISTEN` is per
|
|
52
|
+
# connection, a blocking wait must not hold a connection the rest of the
|
|
53
|
+
# runtime needs, and one connection cannot serve concurrent waiters: the
|
|
54
|
+
# supervisor shares one adapter across roles, and a notification consumed
|
|
55
|
+
# by one waiter would leave the others asleep until their poll expired.
|
|
56
|
+
# @rbs () -> untyped
|
|
57
|
+
def listening_connection: () -> untyped
|
|
58
|
+
|
|
59
|
+
# @rbs () -> untyped
|
|
60
|
+
def open_listening_connection: () -> untyped
|
|
61
|
+
|
|
62
|
+
# @rbs () -> Symbol
|
|
63
|
+
def thread_key: () -> Symbol
|
|
64
|
+
|
|
65
|
+
# @rbs (untyped) -> void
|
|
66
|
+
def disconnect: (untyped) -> void
|
|
67
|
+
|
|
68
|
+
# @rbs (Numeric) -> void
|
|
69
|
+
def pace_after_failure: (Numeric) -> void
|
|
70
|
+
|
|
71
|
+
# @rbs (Symbol, Exception) -> void
|
|
72
|
+
def instrument_failure: (Symbol, Exception) -> void
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Generated from lib/solid_objects/wake_up_adapters.rb with RBS::Inline
|
|
2
|
+
|
|
3
|
+
module SolidObjects
|
|
4
|
+
module WakeUpAdapters
|
|
5
|
+
# Returns the best wake-up strategy for a connection: cross-process
|
|
6
|
+
# notifications where the database provides them, and the in-process
|
|
7
|
+
# default everywhere else.
|
|
8
|
+
#
|
|
9
|
+
# This is deliberately not the default. A notification adapter opens a
|
|
10
|
+
# connection per waiting thread outside the pool, and `LISTEN` does not
|
|
11
|
+
# survive a transaction-pooling proxy such as PgBouncer, so adopting it is
|
|
12
|
+
# a deployment decision rather than an upgrade side effect.
|
|
13
|
+
#
|
|
14
|
+
# @rbs (?untyped) -> untyped
|
|
15
|
+
def self?.for: (?untyped) -> untyped
|
|
16
|
+
end
|
|
17
|
+
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: solid_objects
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.8.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Lucas Carlson
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-08-
|
|
11
|
+
date: 2026-08-10 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: actioncable
|
|
@@ -403,6 +403,8 @@ files:
|
|
|
403
403
|
- lib/solid_objects/turbo_stream_renderer.rb
|
|
404
404
|
- lib/solid_objects/version.rb
|
|
405
405
|
- lib/solid_objects/wake_up.rb
|
|
406
|
+
- lib/solid_objects/wake_up_adapters.rb
|
|
407
|
+
- lib/solid_objects/wake_up_adapters/postgresql.rb
|
|
406
408
|
- lib/solid_objects/worker.rb
|
|
407
409
|
- lib/tasks/solid_objects_tasks.rake
|
|
408
410
|
- sig/generated/controllers/solid_objects/application_controller.rbs
|
|
@@ -474,6 +476,8 @@ files:
|
|
|
474
476
|
- sig/generated/lib/solid_objects/turbo_stream_renderer.rbs
|
|
475
477
|
- sig/generated/lib/solid_objects/version.rbs
|
|
476
478
|
- sig/generated/lib/solid_objects/wake_up.rbs
|
|
479
|
+
- sig/generated/lib/solid_objects/wake_up_adapters.rbs
|
|
480
|
+
- sig/generated/lib/solid_objects/wake_up_adapters/postgresql.rbs
|
|
477
481
|
- sig/generated/lib/solid_objects/worker.rbs
|
|
478
482
|
- sig/generated/models/solid_objects/broadcast.rbs
|
|
479
483
|
- sig/generated/models/solid_objects/claimed_message.rbs
|