otp-rails 0.1.1 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 23149af487fee17bd68538aa5af0d5d0fa4bafc4d652be8ebc75b36038a36b56
4
- data.tar.gz: a9569a4de603358e2b13b3e3a5f4e9b816436eefc7b5256d3553a58aa0d7ee25
3
+ metadata.gz: 4f67972e8849cabbbc0a9e5f3e77cbde0b8b977d02728a6eec0b5f58b99dae74
4
+ data.tar.gz: c63e94a36788699efa990cc47ea160f890a2b74352260b845a5c5e80a64c96b0
5
5
  SHA512:
6
- metadata.gz: feb5ffee5725414004061ec389d68cd78da9c0dafe9173da72a5bc7ba341adfa52667a59d279323a161bb49d17243c8d746ef124f6f46fb9497332dd603d0bf7
7
- data.tar.gz: 37d797974e06728dbb760b6e5685bfdb77681fc04237ae789363e26181d4fdc1ae854febbf0f411686edd82fa0498dd5c7cb6f821492f16aea573d6d9aded95b
6
+ metadata.gz: 10b3255ea24b2b4f5a9b592c74ae48de0db1fb50207066e4db8e64ee00dcc662d1ffb96450575f83dc46ed55915fa498f01aeba469af51c76e72bc899b8cbbfb
7
+ data.tar.gz: 11c4e0021b731061f2d9b4b72c510840371d29a1faa4e723bf4da96c8af9373fff7490b457cd88e9a1356e66c0db2c7848ebe7b6a4fafc4f66aed21f8fd16585
data/CHANGELOG.md CHANGED
@@ -1,5 +1,37 @@
1
1
  # Changelog
2
2
 
3
+ ## v0.2.0 — 2026-09-13
4
+
5
+ **Puma plugin (DESIGN §4.2 step 2)** — `plugin :otp_rails` in `config/puma.rb`: the master
6
+ heartbeats worker-level state over the §5 socket; a missing worker reports `"degraded"`
7
+ (⇒ `child.degraded` telemetry, `meta: {workers, booted, phase}`) while puma replaces it —
8
+ visibility only, no lifecycle change. The `:puma` and `:solid_queue` adapters now export
9
+ `OTP_RAILS_CHILD_ID=<id>` to their children (explicit `env:` wins).
10
+
11
+ **Hardening from a three-track QA pass** (adversarial review + soak/stress + a Ruby⇄Elixir
12
+ contract harness now permanent in the sidecar's CI):
13
+ - Drain signals the whole process group — shell-wrapped cmds (`"a && b"`) no longer leave
14
+ their real workload running after a "clean" shutdown, or duplicate it on restart (#24).
15
+ - Fan-out follows OTP: all affected children stop in reverse start order before any
16
+ restart; declaration-order dependencies hold during `rest_for_one`/`one_for_all` (#14).
17
+ - One spawn path everywhere (fork → setsid → exec): an unspawnable `cmd:` is a child crash
18
+ (exit 127 → strategy → escalation), not a supervisor crash; identical on macOS/Linux (#15).
19
+ - Socket hardening: 64 KiB line cap, string-typed `token`/`cmd`/`id`/`state`, unknown-id
20
+ heartbeats dropped at intake, 64-connection cap, listen backlog 128, connection threads
21
+ torn down on stop, unusable socket path ⇒ exit 78. A well-formed heartbeat naming a
22
+ subtree id no longer crashes the tree (#25, #27, #16, #17, #30, #31).
23
+ - `stop` is prompt during stuck starts, restart fan-outs, and backoff sleeps — no more
24
+ blowing platform grace periods (#28). One healthy interval resets the backoff ladder
25
+ (#19). Non-restarted children leave no stale state or corpse telemetry (#20).
26
+ - `Heartbeat`: the beat thread survives raising/unencodable `state:`/`meta:` lambdas
27
+ (falls back to last-good state / `{}`), and closes failed sockets — no fd growth while
28
+ the supervisor is away (#26, #29).
29
+ - DESIGN §5 wire rules documented (token in the example, string fields, line cap, `ts`
30
+ informational); the Elixir sidecar mirrors them byte-for-byte.
31
+
32
+ **Breaking:** `OtpRails::Heartbeat.start` returns the `Heartbeat` instance (so `#stop`
33
+ works) instead of the raw Thread; still `nil` when unsupervised (#23).
34
+
3
35
  ## v0.1.1 — 2026-09-12
4
36
 
5
37
  - Gemspec only: author listed as `timimsms`. No code changes.
data/README.md CHANGED
@@ -76,6 +76,18 @@ Adapters:
76
76
  - **`:solid_queue`** — wraps `bin/jobs` (`cmd:` overrides). Health = the active heartbeat
77
77
  below, never the `solid_queue_processes` table.
78
78
 
79
+ For worker-level visibility on a cluster-mode `:puma` child, add the plugin to
80
+ `config/puma.rb`:
81
+
82
+ ```ruby
83
+ plugin :otp_rails
84
+ ```
85
+
86
+ The master then heartbeats worker state over the socket: any missing worker is reported
87
+ `"degraded"` (⇒ `[:otp_rails, :child, :degraded]` telemetry, `meta: {workers:, booted:,
88
+ phase:}`) while puma replaces the worker itself — visibility only, no lifecycle change.
89
+ Both adapters export `OTP_RAILS_CHILD_ID` so plugins and hooks heartbeat under the right id.
90
+
79
91
  ## Health & heartbeats
80
92
 
81
93
  Passive children are probed (PID, TCP, HTTP). Active children report themselves: the
@@ -87,8 +99,10 @@ supervisor listens on a Unix socket (mode 0600) and exports `OTP_RAILS_SOCK` /
87
99
  ```
88
100
 
89
101
  A child that has heartbeated is judged by heartbeat freshness: 3 missed `health_interval`s
90
- ⇒ `:degraded`, 6 ⇒ `:dead` ⇒ the strategy applies. Wrong token the line is silently
91
- dropped. The same socket accepts `{"cmd":"restart","id":"jobs","token":"…"}`.
102
+ ⇒ `:degraded`, 6 ⇒ `:dead` ⇒ the strategy applies. Wrong token, non-string fields, or
103
+ lines over 64 KiB ⇒ silently dropped. The same socket accepts
104
+ `{"cmd":"restart","id":"jobs","token":"…"}` — a control restart is deliberate remediation
105
+ (DESIGN §7: restarting is a feature), so it does not count toward restart intensity.
92
106
 
93
107
  From any child process (a Rails initializer, a Solid Queue hook — no Rails required):
94
108
 
@@ -137,6 +151,11 @@ subscriber and a JSON-lines exporter ship by default (`Telemetry::Subscribers`).
137
151
  restarts the supervisor; each child runs in its own session/process group, so stale
138
152
  orphans are findable and killable by pgid.
139
153
  - macOS also caps Unix socket paths at ~104 bytes — keep `socket PATH` short.
154
+ - Compound commands (`cmd: "a && b"`) run under an `sh` wrapper. Drain and kill signal the
155
+ whole process group, so supervised shutdown covers the real workload — but SIGKILL-of-
156
+ the-supervisor orphan prevention (pdeathsig) arms only the wrapper: on Linux the wrapper
157
+ gets SIGTERM and its children are orphaned. Prefer single-exec commands for children
158
+ that must never outlive the supervisor.
140
159
 
141
160
  ## Development
142
161
 
@@ -9,21 +9,21 @@ module OtpRails
9
9
  cmd = spec.opts.fetch(:cmd) { raise ConfigError, "#{spec.id}: :command adapter requires cmd:" }
10
10
  env = spec.opts.fetch(:env, {}).transform_keys(&:to_s)
11
11
  spawn_opts = spec.opts.fetch(:spawn_opts, {})
12
- pid =
13
- if OrphanGuard.available?
14
- parent = Process.pid
15
- Process.fork do
16
- Process.setsid # own session own pgroup, same as pgroup: true below
17
- OrphanGuard.arm!(parent)
18
- begin
19
- Process.exec(env, cmd, **spawn_opts)
20
- rescue SystemCallError
21
- Process.exit!(127)
22
- end
23
- end
24
- else
25
- Process.spawn(env, cmd, pgroup: true, **spawn_opts)
12
+ # One spawn path on every platform (#15): fork → setsid → exec, with
13
+ # exec failure becoming exit 127 through the normal link → crash →
14
+ # strategy machinery. Process.spawn raised Errno::ENOENT into the
15
+ # supervisor loop for a bad cmd on macOS (whole tree crashed, exit 1)
16
+ # while the Linux fork path restart-looped to escalation (exit 70).
17
+ parent = Process.pid
18
+ pid = Process.fork do
19
+ Process.setsid # own session ⇒ own pgroup (drain/kill signal the group)
20
+ OrphanGuard.arm!(parent)
21
+ begin
22
+ Process.exec(env, cmd, **spawn_opts)
23
+ rescue StandardError
24
+ Process.exit!(127)
26
25
  end
26
+ end
27
27
  Handle.new(pid, spec, Process.clock_gettime(Process::CLOCK_MONOTONIC), nil, nil)
28
28
  end
29
29
 
@@ -53,7 +53,12 @@ module OtpRails
53
53
  end
54
54
 
55
55
  def drain(handle, timeout:)
56
- Process.kill("TERM", handle.pid)
56
+ # Signal the whole process group, not just handle.pid: children are
57
+ # group leaders (setsid / pgroup: true), and a shell-wrapped cmd
58
+ # ("a && b") is an sh wrapper whose real workload is a grandchild in
59
+ # that group — TERM to the wrapper alone orphans the workload while
60
+ # reporting a clean drain (issue #24).
61
+ group_signal("TERM", handle.pid)
57
62
  handle.waiter&.join(timeout)
58
63
  !handle.exit_status.nil?
59
64
  rescue Errno::ESRCH
@@ -66,6 +71,16 @@ module OtpRails
66
71
  rescue Errno::ESRCH
67
72
  nil
68
73
  end
74
+
75
+ private
76
+
77
+ # TERM the group; fall back to the pid alone if the group is already
78
+ # gone by the time we signal (pure pid death races to the outer ESRCH).
79
+ def group_signal(sig, pid)
80
+ Process.kill(sig, -pid)
81
+ rescue Errno::ESRCH
82
+ Process.kill(sig, pid)
83
+ end
69
84
  end
70
85
  end
71
86
  Adapter.register(:command, Adapters::Command)
@@ -36,7 +36,10 @@ module OtpRails
36
36
  shutdown: spec.shutdown, start_timeout: spec.start_timeout,
37
37
  opts: spec.opts.merge(
38
38
  cmd: "bundle exec puma -C #{config}",
39
- probe: { http: "http://127.0.0.1:#{port}#{HEALTH_PATH}" }
39
+ probe: { http: "http://127.0.0.1:#{port}#{HEALTH_PATH}" },
40
+ # Tag the child so the otp_rails puma plugin (and any app hook)
41
+ # heartbeats under the right id; explicit env still wins.
42
+ env: { "OTP_RAILS_CHILD_ID" => spec.id.to_s }.merge(spec.opts.fetch(:env, {}))
40
43
  )
41
44
  )
42
45
  end
@@ -31,7 +31,12 @@ module OtpRails
31
31
  ChildSpec.new(
32
32
  id: spec.id, adapter: spec.adapter, restart: spec.restart,
33
33
  shutdown: spec.shutdown, start_timeout: spec.start_timeout,
34
- opts: spec.opts.merge(cmd: spec.opts.fetch(:cmd, DEFAULT_CMD))
34
+ opts: spec.opts.merge(
35
+ cmd: spec.opts.fetch(:cmd, DEFAULT_CMD),
36
+ # Tag the child so the Heartbeat hook picks up its id from env;
37
+ # explicit env still wins.
38
+ env: { "OTP_RAILS_CHILD_ID" => spec.id.to_s }.merge(spec.opts.fetch(:env, {}))
39
+ )
35
40
  )
36
41
  end
37
42
  end
@@ -9,16 +9,25 @@ require "json"
9
9
  module OtpRails
10
10
  # Sends DESIGN §5 NDJSON heartbeats to the supervising socket:
11
11
  #
12
- # OtpRails::Heartbeat.start(id: "jobs")
12
+ # beat = OtpRails::Heartbeat.start(id: "jobs") # => Heartbeat or nil
13
+ # beat&.stop
14
+ #
13
15
  # OtpRails::Heartbeat.start(id: "jobs", interval: 2,
14
16
  # state: -> { queue_backlog_ok? ? "healthy" : "degraded" },
15
17
  # meta: -> { { backlog: backlog_size } })
16
18
  #
17
- # Silently a no-op when OTP_RAILS_SOCK / OTP_RAILS_TOKEN are absent — the
18
- # child is running unsupervised (plain `bin/jobs` in development) and that
19
- # must not be an error. Reconnects on socket loss; never raises into the
20
- # host process.
19
+ # Silently a no-op (returns nil) when OTP_RAILS_SOCK / OTP_RAILS_TOKEN are
20
+ # absent — the child is running unsupervised and that must not be an error.
21
+ #
22
+ # The beating thread is unkillable by bad input (issue #26): a raising
23
+ # state/meta lambda falls back to the last good state / empty meta, an
24
+ # unencodable payload (bad UTF-8) drops down to a bare healthy beat, and
25
+ # socket failures close the socket (issue #29) and retry next beat. Going
26
+ # silent is the one thing this thread must never do while its process is
27
+ # healthy — silence is what gets the child restarted.
21
28
  class Heartbeat
29
+ # Returns the Heartbeat (so #stop works — issue #23), or nil when
30
+ # unsupervised.
22
31
  def self.start(id:, interval: 2, state: -> { "healthy" }, meta: -> { {} })
23
32
  new(id: id, interval: interval, state: state, meta: meta).start
24
33
  end
@@ -26,26 +35,64 @@ module OtpRails
26
35
  def initialize(id:, interval: 2, state: -> { "healthy" }, meta: -> { {} })
27
36
  @id, @interval, @state, @meta = id.to_s, interval, state, meta
28
37
  @sock_path, @token = ENV["OTP_RAILS_SOCK"], ENV["OTP_RAILS_TOKEN"]
38
+ @last_state = "healthy"
29
39
  end
30
40
 
31
- # Returns the beating thread, or nil when unsupervised.
32
41
  def start
33
42
  return nil unless @sock_path && @token
34
- @thread = Thread.new do
35
- sock = nil
36
- loop do
37
- begin
38
- sock ||= UNIXSocket.new(@sock_path)
39
- sock.puts(JSON.generate(id: @id, state: @state.call, ts: Time.now.to_i,
40
- token: @token, meta: @meta.call))
41
- rescue IOError, SystemCallError
42
- sock = nil # supervisor gone or restarting; try again next beat
43
- end
44
- sleep @interval
43
+ @thread ||= Thread.new { run_loop }
44
+ self
45
+ end
46
+
47
+ def stop
48
+ @thread&.kill
49
+ close_socket
50
+ end
51
+
52
+ def alive? = !!@thread&.alive?
53
+
54
+ private
55
+
56
+ def run_loop
57
+ loop do
58
+ begin
59
+ payload = build_payload
60
+ @sock ||= UNIXSocket.new(@sock_path)
61
+ @sock.puts(payload)
62
+ rescue StandardError
63
+ close_socket # supervisor gone or restarting; reconnect next beat
45
64
  end
65
+ sleep @interval
46
66
  end
47
67
  end
48
68
 
49
- def stop = @thread&.kill
69
+ def build_payload
70
+ state = safe_state
71
+ meta = begin
72
+ @meta.call
73
+ rescue StandardError
74
+ {}
75
+ end
76
+ encode(state, meta) || encode(state, {}) || encode("healthy", {})
77
+ end
78
+
79
+ def safe_state
80
+ @last_state = @state.call.to_s
81
+ rescue StandardError
82
+ @last_state
83
+ end
84
+
85
+ def encode(state, meta)
86
+ JSON.generate(id: @id, state: state, ts: Time.now.to_i, token: @token, meta: meta)
87
+ rescue StandardError
88
+ nil
89
+ end
90
+
91
+ def close_socket
92
+ @sock&.close
93
+ @sock = nil
94
+ rescue IOError, SystemCallError
95
+ @sock = nil
96
+ end
50
97
  end
51
98
  end
@@ -14,26 +14,41 @@ module OtpRails
14
14
  # Control: {"cmd":"restart","id":"jobs","token":"…"}
15
15
  # Any line with a missing or wrong token is dropped without a reply.
16
16
  class SocketServer
17
+ MAX_LINE = 64 * 1024 # §5: longer lines are malformed and dropped
18
+ MAX_CONNS = 64 # excess connections are refused (closed immediately)
19
+
17
20
  attr_reader :path, :token
18
21
 
19
22
  def initialize(path:, on_heartbeat:, on_control:)
20
23
  @path, @on_heartbeat, @on_control = path, on_heartbeat, on_control
21
24
  @token = SecureRandom.hex(16)
25
+ @conns = []
22
26
  end
23
27
 
24
28
  # Binds, chmods, and exports OTP_RAILS_SOCK / OTP_RAILS_TOKEN so children
25
29
  # spawned afterwards inherit them (DESIGN §9). Call before starting children.
26
30
  def start
27
- FileUtils.mkdir_p(File.dirname(@path))
28
- File.unlink(@path) if File.exist?(@path) # stale socket from a dead boot
29
- @server = UNIXServer.new(@path)
31
+ begin
32
+ FileUtils.mkdir_p(File.dirname(@path))
33
+ File.unlink(@path) if File.exist?(@path) # stale socket from a dead boot
34
+ @server = UNIXServer.new(@path)
35
+ rescue ArgumentError, SystemCallError => e
36
+ # e.g. > ~104-byte path on macOS, unwritable dir: config problem, not a crash
37
+ raise ConfigError, "heartbeat socket #{@path.inspect}: #{e.message}"
38
+ end
39
+ @server.listen(128) # default backlog is 5 on macOS; bursts got ECONNREFUSED
30
40
  File.chmod(0o600, @path)
31
41
  ENV["OTP_RAILS_SOCK"] = @path
32
42
  ENV["OTP_RAILS_TOKEN"] = @token
33
43
  @acceptor = Thread.new do
34
44
  loop do
35
45
  conn = @server.accept
36
- Thread.new { serve(conn) }
46
+ @conns.reject! { |c| !c[:thread].alive? }
47
+ if @conns.size >= MAX_CONNS
48
+ close_quietly(conn)
49
+ next
50
+ end
51
+ @conns << { conn: conn, thread: Thread.new { serve(conn) } }
37
52
  rescue IOError, SystemCallError
38
53
  break # server closed during shutdown
39
54
  end
@@ -43,6 +58,11 @@ module OtpRails
43
58
  def stop
44
59
  @server&.close
45
60
  @acceptor&.kill
61
+ @conns.each do |c| # connection threads must not outlive the server
62
+ c[:thread].kill
63
+ close_quietly(c[:conn])
64
+ end
65
+ @conns.clear
46
66
  File.unlink(@path) if File.exist?(@path)
47
67
  rescue SystemCallError
48
68
  nil
@@ -51,27 +71,45 @@ module OtpRails
51
71
  private
52
72
 
53
73
  def serve(conn)
54
- conn.each_line do |line|
55
- msg = begin
56
- JSON.parse(line)
57
- rescue JSON::ParserError
74
+ loop do
75
+ line = conn.gets("\n", MAX_LINE)
76
+ break if line.nil?
77
+ unless line.end_with?("\n")
78
+ # over-long line: memory stays capped at MAX_LINE — discard the rest
79
+ # of the line, then resume at the next newline
80
+ line = conn.gets("\n", MAX_LINE) while !line.nil? && !line.end_with?("\n")
81
+ break if line.nil?
58
82
  next
59
83
  end
60
- next unless msg["token"] == @token # bad token ⇒ dropped
61
- if msg["cmd"]
62
- @on_control.call(msg)
63
- elsif msg["id"] && msg["state"]
64
- @on_heartbeat.call(msg)
65
- end
84
+ handle_line(line)
66
85
  end
67
86
  rescue IOError, SystemCallError
68
87
  nil
69
88
  ensure
70
- begin
71
- conn.close
72
- rescue IOError
73
- nil
89
+ close_quietly(conn)
90
+ end
91
+
92
+ # §5: token, cmd, id, and state are JSON strings; anything else in those
93
+ # fields is malformed and silently dropped — same as a bad token. A bad
94
+ # line must never take the connection (or the supervisor) down.
95
+ def handle_line(line)
96
+ msg = begin
97
+ JSON.parse(line)
98
+ rescue StandardError
99
+ return
100
+ end
101
+ return unless msg.is_a?(Hash) && msg["token"] == @token
102
+ if msg.key?("cmd")
103
+ @on_control.call(msg) if msg["cmd"].is_a?(String)
104
+ elsif msg["id"].is_a?(String) && msg["state"].is_a?(String)
105
+ @on_heartbeat.call(msg)
74
106
  end
75
107
  end
108
+
109
+ def close_quietly(conn)
110
+ conn.close
111
+ rescue IOError, SystemCallError
112
+ nil
113
+ end
76
114
  end
77
115
  end
@@ -14,11 +14,19 @@ module OtpRails
14
14
  @live = {} # id => { adapter:, handle:, attempts:, generation:, monitor: }
15
15
  @queue = Queue.new
16
16
  @stopping = false
17
+ @stop_requested = false
17
18
  @heartbeats = {} # id => { at: monotonic ts of last heartbeat, state: reported state }
18
19
  return unless socket_path
19
20
  @socket = SocketServer.new(
20
21
  path: socket_path,
21
- on_heartbeat: ->(msg) { @heartbeats[msg["id"].to_sym] = { at: mono_now, state: msg["state"] } },
22
+ # Heartbeats for ids that aren't children of THIS supervisor are
23
+ # dropped at intake: ghost ids must not grow the table unboundedly
24
+ # (issue #16/#27). Subtree-grandchild routing is the open flat-id
25
+ # question — until it's decided, their beats are dropped, not hoarded.
26
+ on_heartbeat: lambda { |msg|
27
+ id = msg["id"].to_sym
28
+ @heartbeats[id] = { at: mono_now, state: msg["state"] } if @children.any? { |c| c.id == id }
29
+ },
22
30
  on_control: ->(msg) { @queue << { type: :control, cmd: msg["cmd"], id: msg["id"].to_s.to_sym } }
23
31
  )
24
32
  end
@@ -37,7 +45,10 @@ module OtpRails
37
45
  def run
38
46
  Telemetry.emit(:"supervisor.start", {}, { strategy: strategy, children: ids })
39
47
  @socket&.start # before children, so they inherit OTP_RAILS_SOCK/_TOKEN
40
- @children.each { |spec| start_child(spec) }
48
+ @children.each do |spec|
49
+ break if @stop_requested
50
+ start_child(spec)
51
+ end
41
52
  loop do
42
53
  msg = @queue.pop
43
54
  case msg[:type]
@@ -53,7 +64,14 @@ module OtpRails
53
64
  Telemetry.emit(:"supervisor.stop")
54
65
  end
55
66
 
56
- def stop = @queue << { type: :stop }
67
+ # Sets the flag first: the main loop may be stuck in a wait_healthy poll
68
+ # or a backoff sleep for up to start_timeout/backoff seconds, and shutdown
69
+ # must not wait for those (issue #28 — platforms SIGKILL after their grace
70
+ # period, which resurrects the orphan problem).
71
+ def stop
72
+ @stop_requested = true
73
+ @queue << { type: :stop }
74
+ end
57
75
 
58
76
  # Public remediation API (DESIGN §7). Over IPC in the real thing; direct call here.
59
77
  def restart!(id)
@@ -100,6 +118,11 @@ module OtpRails
100
118
  case effective_health(spec, adapter, handle)
101
119
  when :healthy
102
120
  degraded = 0
121
+ # One healthy interval resets the backoff ladder (#19): a child
122
+ # that crashes rarely should not converge to permanent max
123
+ # backoff. Crash-looping children never reach a monitor, so flap
124
+ # damping is unaffected.
125
+ entry[:attempts] = 0
103
126
  when :degraded
104
127
  degraded += 1
105
128
  Telemetry.emit(:"child.degraded", { consecutive: degraded }, { id: spec.id })
@@ -118,6 +141,7 @@ module OtpRails
118
141
  def wait_healthy(spec, adapter, handle)
119
142
  deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + spec.start_timeout
120
143
  loop do
144
+ return :stopping if @stop_requested # shutdown must not wait out start_timeout (#28)
121
145
  case effective_health(spec, adapter, handle)
122
146
  when :healthy then Telemetry.emit(:"child.healthy", {}, { id: spec.id }); return :healthy
123
147
  when :dead then return :dead # the exit message arrives via link
@@ -142,7 +166,10 @@ module OtpRails
142
166
  # back to the adapter's passive probe.
143
167
  def effective_health(spec, adapter, handle)
144
168
  hb = @heartbeats[spec.id]
145
- return adapter.health(handle) unless hb
169
+ # No heartbeat ⇒ passive probe. A nil health_interval (subtree specs)
170
+ # also falls through: freshness aging needs an interval, and dividing
171
+ # by nil crashed the whole tree when a heartbeat named such an id (#25).
172
+ return adapter.health(handle) unless hb && spec.health_interval
146
173
  missed = (mono_now - hb[:at]) / spec.health_interval
147
174
  return :dead if missed >= 6
148
175
  return :degraded if missed >= 3
@@ -178,21 +205,46 @@ module OtpRails
178
205
 
179
206
  uptime = Process.clock_gettime(Process::CLOCK_MONOTONIC) - entry[:handle].started_at
180
207
  Telemetry.emit(:"child.exit", { exit_code: status&.exitstatus, uptime_ms: (uptime * 1000).round }, { id: id })
181
- return unless spec.restart?(status)
208
+ unless spec.restart?(status)
209
+ # The child is gone for good: keep no stale entry, or stop_all and the
210
+ # monitor emit spurious drains for a corpse later (issue #20).
211
+ entry[:monitor]&.kill
212
+ @live.delete(id)
213
+ return
214
+ end
182
215
 
183
216
  if intensity.record!
184
217
  Telemetry.emit(:"supervisor.escalate", { restarts: intensity.count }, { within: intensity.within })
185
218
  raise Escalation, "restart intensity exceeded (#{intensity.count} in #{intensity.within}s)"
186
219
  end
187
220
 
188
- Strategy.affected(strategy, ids, id).each do |aid|
189
- aspec = spec_for(aid)
190
- stop_child(aspec) unless aid == id
191
- attempts = (@live[aid][:attempts] += 1)
221
+ affected = Strategy.affected(strategy, ids, id)
222
+ # OTP semantics (#14): declaration order encodes dependency, so first
223
+ # terminate ALL affected children in reverse start order — a
224
+ # replacement :b must never boot while an old :c that depended on the
225
+ # dead :b is still running — then restart them in start order.
226
+ affected.reverse_each do |aid|
227
+ stop_child(spec_for(aid)) unless aid == id
228
+ end
229
+ affected.each do |aid|
230
+ break if @stop_requested # shutdown preempts the restart fan-out (#28)
231
+ entry = @live[aid]
232
+ next unless entry # a temporary/clean-transient sibling is gone for good (#20)
233
+ attempts = (entry[:attempts] += 1)
192
234
  delay = backoff.delay(attempts)
193
235
  Telemetry.emit(:"child.restart", { backoff_ms: (delay * 1000).round }, { id: aid, attempt: attempts, strategy: strategy })
194
- sleep delay
195
- start_child(aspec)
236
+ interruptible_sleep(delay)
237
+ break if @stop_requested
238
+ start_child(spec_for(aid))
239
+ end
240
+ end
241
+
242
+ # Backoff must not delay shutdown (#28): sleep in slices, bail on stop.
243
+ def interruptible_sleep(seconds)
244
+ deadline = mono_now + seconds
245
+ while mono_now < deadline
246
+ return if @stop_requested
247
+ sleep [0.1, deadline - mono_now].min
196
248
  end
197
249
  end
198
250
 
@@ -35,7 +35,15 @@ module OtpRails
35
35
  measurements: measurements, metadata: metadata,
36
36
  ts: Process.clock_gettime(Process::CLOCK_REALTIME) }
37
37
  subs = @mutex.synchronize { @subscribers.dup }
38
- subs.each { |s| s.call(event) }
38
+ subs.each do |s|
39
+ s.call(event)
40
+ rescue StandardError => e
41
+ # A subscriber must never break the bus: emit is called from the
42
+ # supervisor loop and monitor threads, and other subscribers (the
43
+ # Elixir sidecar exporter, the resilience bridge) must keep
44
+ # receiving events even when one subscriber raises (resilience#1).
45
+ warn "[otp-rails] telemetry subscriber raised: #{e.class}: #{e.message}"
46
+ end
39
47
  event
40
48
  end
41
49
  end
@@ -1,4 +1,4 @@
1
1
  # frozen_string_literal: true
2
2
  module OtpRails
3
- VERSION = "0.1.1"
3
+ VERSION = "0.2.1"
4
4
  end
@@ -0,0 +1,51 @@
1
+ # frozen_string_literal: true
2
+ # Puma plugin (DESIGN §4.2 step 2, PLAN 2.5): worker-level visibility for a
3
+ # :puma child, over the §5 heartbeat socket. Enable with `plugin :otp_rails`
4
+ # in config/puma.rb.
5
+ #
6
+ # Lives under lib/puma/ (NOT lib/otp_rails/) because it may require puma —
7
+ # it is only ever loaded BY puma, so hard rule 1 (the supervisor has zero
8
+ # dependencies) holds. It reuses the frozen heartbeat protocol untouched:
9
+ # worker detail travels in `state` and `meta`, never in new fields.
10
+ #
11
+ # The master heartbeats every OTP_RAILS_HEARTBEAT_INTERVAL (default 2s):
12
+ # state = "degraded" while any worker is missing (booted < configured),
13
+ # "healthy" otherwise (single mode is always "healthy"),
14
+ # meta = { workers:, booted:, phase: } (cluster) | { mode: "single" }.
15
+ # The supervisor turns a reported "degraded" into [:otp_rails, :child,
16
+ # :degraded] telemetry via its normal health loop — no event added, and no
17
+ # lifecycle change: puma still replaces its own workers.
18
+ require "puma/plugin"
19
+ require "otp_rails/heartbeat"
20
+
21
+ Puma::Plugin.create do
22
+ def start(launcher)
23
+ beat = OtpRails::Heartbeat.start(
24
+ id: ENV["OTP_RAILS_CHILD_ID"] || "web",
25
+ interval: Float(ENV.fetch("OTP_RAILS_HEARTBEAT_INTERVAL", 2)),
26
+ state: -> { otp_rails_state(launcher) },
27
+ meta: -> { otp_rails_meta(launcher) }
28
+ )
29
+ launcher.events.on_stopped { beat.stop } if beat
30
+ end
31
+
32
+ private
33
+
34
+ # Raising inside the heartbeat thread would silently kill it, so both
35
+ # lambdas degrade to a safe value instead (stats can raise mid-boot).
36
+ def otp_rails_state(launcher)
37
+ stats = launcher.stats
38
+ return "healthy" unless stats[:workers] # single mode
39
+ stats[:booted_workers] < stats[:workers] ? "degraded" : "healthy"
40
+ rescue StandardError
41
+ "starting"
42
+ end
43
+
44
+ def otp_rails_meta(launcher)
45
+ stats = launcher.stats
46
+ return { mode: "single" } unless stats[:workers]
47
+ { workers: stats[:workers], booted: stats[:booted_workers], phase: stats[:phase] }
48
+ rescue StandardError
49
+ {}
50
+ end
51
+ end
metadata CHANGED
@@ -1,17 +1,18 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: otp-rails
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.2.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - timimsms
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2026-09-13 00:00:00.000000000 Z
11
+ date: 2026-09-14 00:00:00.000000000 Z
12
12
  dependencies: []
13
- description: A slim supervisor that starts, links, health-checks, and restarts the
14
- processes of a Rails app (web, jobs, cable, cron) with OTP strategies.
13
+ description: 'RENAMED: this gem is now `odoshi` (https://rubygems.org/gems/odoshi).
14
+ otp-rails 0.2.1 is identical to 0.2.0 plus a telemetry hardening fix; all future
15
+ releases ship as odoshi. See https://github.com/shishi-odoshi/odoshi.'
15
16
  email:
16
17
  executables:
17
18
  - otp-rails
@@ -41,6 +42,7 @@ files:
41
42
  - lib/otp_rails/supervisor.rb
42
43
  - lib/otp_rails/telemetry.rb
43
44
  - lib/otp_rails/version.rb
45
+ - lib/puma/plugin/otp_rails.rb
44
46
  homepage: https://github.com/shishi-odoshi/otp-rails
45
47
  licenses:
46
48
  - MIT
@@ -49,7 +51,9 @@ metadata:
49
51
  source_code_uri: https://github.com/shishi-odoshi/otp-rails
50
52
  changelog_uri: https://github.com/shishi-odoshi/otp-rails/blob/main/CHANGELOG.md
51
53
  bug_tracker_uri: https://github.com/shishi-odoshi/otp-rails/issues
52
- post_install_message:
54
+ post_install_message: |-
55
+ otp-rails has been renamed to `odoshi` — https://rubygems.org/gems/odoshi
56
+ otp-rails will receive no further releases. Migration table: CHANGELOG 0.3.0 in the odoshi repo.
53
57
  rdoc_options: []
54
58
  require_paths:
55
59
  - lib