yamine 0.22.1 → 0.23.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1f20b021d3a702ab4aef776ac24e10683b12c1b4ba58b7ad4e5b11b91316eb78
-  data.tar.gz: 4ff8a2e59f075f38aa81d084d9aea45041346790aefd687ef2fc08669bc34737
+  metadata.gz: c868efafc13a1f8bf3aadad4cf4d0cacffe110e51e091e30b79eb1e81d2f4a24
+  data.tar.gz: f888229b7d618e2936108a27342f9227c8f986e46611ba22ea880052ca190a49
 SHA512:
-  metadata.gz: 2e2542be9656d87375b9ab29ac2862f33b2be289c7932aa99f2f29abf3b2a47f40e24623f2ab8b902021e6b42fe4ed48ebe7dfc4a86a3184ddf4f859cb359d40
-  data.tar.gz: 85d76ec111ee796d23b89879d414bf5a248504f58983a0064c3d43ffaa88d532a0f3b5d6542f21b09d2fba5acb57f20e673fcb5f7d13490108f321adbf1eb896
+  metadata.gz: 3ad556cdad8d143e9beb87ba0cdcfc2fa5a686ad04d10ad763e048f4ace90d084527ecc23064998ac37d9a65705e831b3216675ab5306a5da8b869680f302371
+  data.tar.gz: b1b11d113410b72139b9f8ea19592342b29b40edab39869861e2488f4c4c8f193de00ad564f2ba771dba5e2a33072abd41c5e0f10c649dd0ff55bceaaeab98ff

data/CHANGELOG.md

@@ -2,6 +2,63 @@
 
 ## [Unreleased]
 
+## [0.23.1] - 2026-10-02
+
+### Fixed
+
+- **A chunked response now ends at its terminator instead of at end of
+  file.** The relay had one way to end a body with no Content-Length —
+  copy until the backend hung up — and it read every such body as
+  close-delimited. But a chunked body ends at its terminating chunk, and
+  the backend holding the connection open past it is keep-alive doing
+  exactly what it promised. So the relay blocked for the whole idle
+  bound, sixty seconds by default, with the client's connection pinned
+  the entire time. Every streamed page, every ActionController::Live
+  response and every event stream is framed this way, and the request
+  that followed on that connection — typically the page's main
+  JavaScript chunk — got nothing back until the connection was reaped.
+  A load test could not see it, because every request in one opened its
+  own connection. Chunked framing is now read rather than assumed, bytes
+  still cross to the client exactly as the backend wrote them, and the
+  connection is free for the next request the moment the terminator
+  arrives.
+- `minitest/mock` is required by the test helper, so `Object#stub` is
+  available to every test rather than only the ones that load it.
+
+## [0.23.0] - 2026-10-02
+
+### Fixed
+
+- **A stalled response body closes the connection instead of appending a
+  502 into a live 200.** When a response relay went quiet past the idle
+  bound, the IdleTimeout escaped `forward`'s two rescues (it is neither
+  HeadTimeout nor EOFError), landed in `handle`'s generic IOError rescue,
+  and rendered a whole second response head into the body the browser was
+  already reading. The page died of the corruption with no error anywhere
+  naming a cause — DevTools even showed the 502 as if a server had sent
+  it. A response whose head has been sent can now only end one way: the
+  connection closes, the stall is logged with the request that died, and
+  no second head is ever written.
+- **`backend-refused` no longer lies about what failed.** `dial` mapped
+  every SystemCallError to :refused, so a proxy out of file descriptors
+  answered "Nothing is listening at 127.0.0.1:4486" and sent the reader
+  off to restart an app that was perfectly fine. EMFILE/ENFILE now read
+  as `proxy-exhausted` and blame the proxy's own limit; every other dial
+  error reads as `proxy-unreachable`; a backend that accepts and closes
+  without answering reads as `backend-gone` instead of masquerading as
+  refused. The 502 page says which of these it was, and carries the fix
+  for that failure rather than a start command for an app that is up.
+
+### Added
+
+- **The root daemon can finally be debugged.** The launchd plist and the
+  systemd unit carried no log paths, so the one proxy process serving
+  traffic discarded every error it ever reported — the mislabeled 502
+  above was invisible precisely because of this. Both streams now land
+  in `proxy.log` in the state dir, and the daemon rotates that log at
+  boot (one generation kept), the same rule the spawned proxy already
+  applied.
+
 ## [0.22.1] - 2026-09-29
 
 ### Fixed

data/lib/yamine/cli/system.rb

@@ -97,6 +97,11 @@ module Yamine
           if foreground
             write_tls_marker(ctx, tls)
             write_tlds_file(ctx, tlds)
+            # The daemon appends to this log for its whole life (launchd
+            # and systemd both redirect their streams here), so each boot
+            # rotates first — one generation kept, the same rule the
+            # spawned proxy's Log.open_append already applies.
+            Log.rotate(File.join(ctx.store.dir, "proxy.log"))
             # Record BEFORE serving. This process (the launchd/systemd
             # service, or a hand-run foreground proxy) IS the proxy, and
             # every reader trusts these files: doctor, Context's URL
@@ -468,8 +473,37 @@ module Yamine
         # the user-writable gem directory it was staged from. (The
         # interpreter stays RbConfig.ruby: no root-owned Ruby >= 3.2
         # exists, so that residual risk is reported, not hidden.)
+        plist = launchd_plist(state_dir: state_dir, home: home)
+        remove_legacy_launchd(dir)
+        FileUtils.mkdir_p(dir)
+        path = File.join(dir, "#{LAUNCHD_LABEL}.plist")
+        File.write(path, plist)
+        File.chmod(0o644, path)
+        # launchd requires /Library/LaunchDaemons plists to be
+        # root-owned; we are root here (sudo re-exec). Enforce it
+        # explicitly: File.write keeps an existing file's owner, so a
+        # stale user-owned plist from an older version would otherwise
+        # survive the overwrite and bootstrap fails with error 5.
+        # Trust the CA into the System keychain while elevated: silent
+        # (no GUI popup) and trusted for every user on the machine.
+        File.chown(0, 0, path) if Process.uid.zero?
+        ensure_system_ca_trust
+        puts "    Registering the launchd service on port 443..."
+        launchctl_bootstrap(path)
+        puts "Installed root LaunchDaemon on port 443 (state: #{state_dir})."
+        puts "    Payload: #{PrivilegedPayload.bin_path} (root-owned, version-independent)."
+        puts "    Log: #{File.join(state_dir, "proxy.log")}."
+      end
+
+      # Pure plist builder (testable without root). Both output streams
+      # point at the state dir's proxy.log: a root daemon whose errors
+      # launchd discards cannot be debugged at all, and one whole
+      # misdiagnosed failure class came from exactly that blindness —
+      # the proxy's own account of a dial failure never reached disk.
+      def launchd_plist(state_dir:, home:)
         payload_bin = PrivilegedPayload.bin_path
-        plist = <<~PLIST
+        log_path = File.join(state_dir, "proxy.log")
+        <<~PLIST
           <?xml version="1.0" encoding="UTF-8"?>
           <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
           <plist version="1.0">
@@ -489,27 +523,11 @@ module Yamine
             </dict>
             <key>KeepAlive</key><true/>
             <key>RunAtLoad</key><true/>
+            <key>StandardOutPath</key><string>#{log_path}</string>
+            <key>StandardErrorPath</key><string>#{log_path}</string>
           </dict>
           </plist>
         PLIST
-        remove_legacy_launchd(dir)
-        FileUtils.mkdir_p(dir)
-        path = File.join(dir, "#{LAUNCHD_LABEL}.plist")
-        File.write(path, plist)
-        File.chmod(0o644, path)
-        # launchd requires /Library/LaunchDaemons plists to be
-        # root-owned; we are root here (sudo re-exec). Enforce it
-        # explicitly: File.write keeps an existing file's owner, so a
-        # stale user-owned plist from an older version would otherwise
-        # survive the overwrite and bootstrap fails with error 5.
-        # Trust the CA into the System keychain while elevated: silent
-        # (no GUI popup) and trusted for every user on the machine.
-        File.chown(0, 0, path) if Process.uid.zero?
-        ensure_system_ca_trust
-        puts "    Registering the launchd service on port 443..."
-        launchctl_bootstrap(path)
-        puts "Installed root LaunchDaemon on port 443 (state: #{state_dir})."
-        puts "    Payload: #{payload_bin} (root-owned, version-independent)."
       end
 
       # True when the trust store covers the CA this machine should be
@@ -587,9 +605,12 @@ module Yamine
       # Pure unit-file builder (testable without root). Binds 80/443 at
       # boot; the proxy runs with the invoking user's state dir. The
       # payload is the root-owned staged path — never the gem directory.
-      def systemd_unit
-        home = user_home_for_service
-        state_dir = ENV["YAMINE_STATE_DIR"] || File.join(home, ".yamine")
+      # Both output streams append to the state dir's proxy.log, matching
+      # the launchd unit, so a daemon that misbehaves can be read.
+      def systemd_unit(state_dir: nil, home: nil)
+        home ||= user_home_for_service
+        state_dir ||= ENV["YAMINE_STATE_DIR"] || File.join(home, ".yamine")
+        log_path = File.join(state_dir, "proxy.log")
         <<~UNIT
           # /etc/systemd/system/yamine.service  (binds 80/443 at boot)
           [Unit]
@@ -599,6 +620,8 @@ module Yamine
           ExecStart=#{RbConfig.ruby} #{PrivilegedPayload.bin_path} proxy start --foreground --port #{ProxyControl::DEFAULT_TLS_PORT}
           Environment=YAMINE_STATE_DIR=#{state_dir}
           Environment=HOME=#{home}
+          StandardOutput=append:#{log_path}
+          StandardError=append:#{log_path}
 
           [Install]
           WantedBy=multi-user.target

data/lib/yamine/proxy.rb

@@ -64,6 +64,7 @@ module Yamine
     # which is also the only one a retry can be safe for.
     HeadTimeout = Class.new(IdleTimeout)
     CHUNK_BYTES = 16_384
+    CRLF = "\r\n"
     # Route cache: routes.json is the source of truth, but re-reading
     # and re-parsing it on every request is wasteful under HMR polling.
     # Keyed on file mtime, so a freshly registered route is visible on
@@ -329,6 +330,13 @@ module Yamine
           keep, reason = forward(sock, entry, method, target, headers, buf)
         end
 
+        if reason == :body_lost
+          # The response head already reached the client; there is no
+          # answer left to give, only a connection to close. A 502 here
+          # would be garbage appended to a live response.
+          break
+        end
+
         if reason
           render_bad_gateway(sock, entry, host, reason)
           break
@@ -344,7 +352,7 @@ module Yamine
 
     # One attempt at the backend: dial, forward the request, relay the
     # response. Returns [keep_alive?, reason], where reason is nil on
-    # success and otherwise names which backend failure this was. That one
+    # success and otherwise names which failure this was. That one
     # value decides both whether a retry is safe and what the 502 says, so
     # "refused" and "silent" can never drift apart again.
     #
@@ -355,8 +363,8 @@ module Yamine
       begin
         backend = dial(entry)
       rescue SystemCallError => e
-        @on_error.call("dial failed for #{entry["hostname"]}: #{e.message}")
-        return [:close, :refused]
+        @on_error.call("dial failed for #{entry["hostname"]}: #{e.class}: #{e.message}")
+        return [:close, failure_reason(e)]
       end
 
       begin
@@ -370,15 +378,29 @@ module Yamine
       end
     end
 
+    # What a failed dial means. ECONNREFUSED is the one failure that
+    # indicts the backend; the rest indict the path or the proxy itself,
+    # and reporting them as "Nothing is listening" sends the reader off to
+    # restart an app that is perfectly fine — an fd-exhausted proxy spent
+    # one whole debugging evening dressed as a dead backend.
+    def failure_reason(error)
+      case error
+      when Errno::ECONNREFUSED then :refused
+      when Errno::EMFILE, Errno::ENFILE then :exhausted
+      else :unreachable
+      end
+    end
+
     # A refused dial never reached the app, so any method replays safely —
     # the request body is still unread on the client socket. A head
-    # timeout means the head DID reach the app, so the only safe replays
-    # are the bodiless methods: the body has already been consumed off the
-    # client socket and cannot be faithfully resent, and a duplicated
-    # message is a worse outcome than a slow page.
+    # timeout or an exhausted proxy means the attempt died on our side of
+    # the wire, so the only safe replays are the bodiless methods: the
+    # body has already been consumed off the client socket and cannot be
+    # faithfully resent, and a duplicated message is a worse outcome than
+    # a slow page.
     def retriable?(reason, method, headers)
       return true if reason == :refused
-      return false unless reason == :silent
+      return false unless [:silent, :exhausted].include?(reason)
       return false unless %w[GET HEAD OPTIONS].include?(method.to_s.upcase)
 
       request_body_length(headers) == 0
@@ -413,17 +435,53 @@ module Yamine
       return [:close, :gone] if rhead.nil?
 
       _rm, _rt, rheaders = parse_head(rhead)
-      write_all(sock, rhead)
-      write_all(sock, rbuf) unless rbuf.empty?
+      relay_after_head(sock, backend, rhead, rbuf, rheaders, headers, target)
+    end
 
-      rlen = content_length(rheaders)
-      if rlen.nil?
-        # No Content-Length: close-delimited response.
-        copy_stream(backend, sock)
-        return [:close, nil]
+    # Everything from the response head onward. Once the head is on the
+    # wire the 502 vocabulary is spent: a failure here can only end the
+    # connection, never answer it, and rendering a 502 into the middle of
+    # a live 200 corrupts the stream the browser is already reading (the
+    # second head arrives as body bytes; the page dies of it with no
+    # error naming anything). [:close, :body_lost] tells handle to close
+    # quietly instead — the stall itself is logged with the request that
+    # died, since a relay that goes quiet mid-body is exactly the failure
+    # that is otherwise invisible.
+    def relay_after_head(sock, backend, rhead, rbuf, rheaders, headers, target)
+      begin
+        write_all(sock, rhead)
+
+        rlen = content_length(rheaders)
+        if chunked?(rheaders)
+          # The chunk relay reads the framing to find where the body ends,
+          # so it takes the read-ahead itself — handing it bytes already on
+          # the wire would send the body twice.
+          relay_chunked(backend, sock, rbuf)
+        else
+          write_all(sock, rbuf) unless rbuf.empty?
+
+          if rlen.nil?
+            # No Content-Length and no chunk framing: the body really does
+            # end at end of file, because the backend is closing to say so.
+            copy_stream(backend, sock)
+            return [:close, nil]
+          else
+            remaining = rlen - rbuf.bytesize
+            copy_stream(backend, sock, remaining) if remaining > 0
+          end
+        end
+      rescue EOFError
+        @on_error.call("backend for #{headers["host"]}#{target} closed mid-body, " \
+                       "before its Content-Length of #{rheaders["content-length"]} was met")
+        return [:close, :body_lost]
+      rescue IdleTimeout => e
+        @on_error.call("response for #{headers["host"]}#{target} died mid-body: #{e.message}")
+        return [:close, :body_lost]
+      rescue IOError, SystemCallError
+        # The client or the backend hung up mid-relay — ordinary web
+        # traffic (a cancelled fetch, a departed tab), not a failure.
+        return [:close, :body_lost]
       end
-      remaining = rlen - rbuf.bytesize
-      copy_stream(backend, sock, remaining) if remaining > 0
 
       if keep_alive?(headers) && keep_alive?(rheaders)
         # Any bytes beyond Content-Length on the backend are a second
@@ -439,8 +497,13 @@ module Yamine
       return if rhead.nil?
 
       write_all(sock, rhead)
-      write_all(sock, rbuf) unless rbuf.empty?
-      copy_stream(backend, sock)
+      _rm, _rt, rheaders = parse_head(rhead)
+      if chunked?(rheaders)
+        relay_chunked(backend, sock, rbuf)
+      else
+        write_all(sock, rbuf) unless rbuf.empty?
+        copy_stream(backend, sock)
+      end
     rescue IOError, SystemCallError
       nil
     end
@@ -605,6 +668,79 @@ module Yamine
       end
     end
 
+    # Chunked framing is the one body the proxy has to read rather than
+    # copy, because it is the only thing that says where the body ends. It
+    # ends at the terminating chunk — not at end of file, and the backend
+    # holding the connection open past it is keep-alive doing exactly what
+    # it promised. Every streamed page, every ActionController::Live
+    # response and every event stream is framed this way.
+    #
+    # Bytes cross to the client exactly as the backend wrote them; only the
+    # framing is read, never rewritten. `buf` is the read-ahead already
+    # past the response head.
+    def relay_chunked(src, dst, buf)
+      loop do
+        line = take_line(src, buf)
+        return if line.nil?
+
+        write_all(dst, line)
+        size = chunk_size(line)
+        if size.zero?
+          relay_trailers(src, dst, buf)
+          return
+        end
+
+        write_all(dst, take_bytes(src, buf, size) || +"")
+        write_all(dst, take_line(src, buf) || +"")
+      end
+    end
+
+    # Trailers run to the blank line that closes them.
+    def relay_trailers(src, dst, buf)
+      loop do
+        line = take_line(src, buf)
+        return if line.nil?
+
+        write_all(dst, line)
+        return if line == CRLF
+      end
+    end
+
+    # A chunk-size line, less the CRLF that ends it. Extensions after a
+    # semicolon are the sender's and are not ours to weigh.
+    def chunk_size(line)
+      line.split(";", 2).first.to_s.strip.to_i(16)
+    end
+
+    def take_line(sock, buf)
+      until (index = buf.index(CRLF))
+        return nil unless fill_buf(sock, buf)
+      end
+      buf.slice!(0, index + 2)
+    end
+
+    def take_bytes(sock, buf, count)
+      out = +""
+      while out.bytesize < count
+        return nil if buf.empty? && !fill_buf(sock, buf)
+
+        take = [count - out.bytesize, buf.bytesize].min
+        out << buf.slice!(0, take)
+      end
+      out
+    end
+
+    def fill_buf(sock, buf)
+      buf << read_chunk(sock, CHUNK_BYTES)
+      true
+    rescue EOFError
+      false
+    end
+
+    def chunked?(headers)
+      headers["transfer-encoding"].to_s.downcase.include?("chunked")
+    end
+
     # Block until the socket is ready for the direction a nonblocking op
     # asked for; a timeout when the bound passes with no progress. A
     # caller-supplied bound only changes how long we wait — what a timeout
@@ -783,8 +919,7 @@ module Yamine
     # command to run. Modeled on render_not_found, which already answers
     # "nobody knows what this hostname is" properly.
     def render_bad_gateway(sock, entry, host, reason = :refused)
-      kind = reason == :silent ? "backend-silent" : "backend-refused"
-      headers = { ERROR_HEADER => kind }
+      headers = { ERROR_HEADER => error_kind(reason) }
       bare = Hostname.strip_port(host)
       # Same DNS-rebinding boundary as render_not_found: a Host outside our
       # TLDs is a website that got us to answer, not the local developer
@@ -793,16 +928,31 @@ module Yamine
 
       target = entry ? entry["target"].to_s : ""
       body = "<h1>Bad Gateway</h1><p>#{what_happened(reason, target)}</p>" \
-             "#{bad_gateway_owner(entry, bare)}#{bad_gateway_fix(entry)}"
+             "#{bad_gateway_owner(entry, bare)}#{bad_gateway_fix(entry, reason)}"
       respond(sock, 502, body, headers: headers)
     rescue IOError, SystemCallError
       nil
     end
 
-    # Which of the two failures this was, said the way each one has to be
-    # read: not listening means start the app, silent means go find out why
-    # an app that is up is not answering. One "not responding" for both
-    # sends the reader to the wrong place either way.
+    # The machine-readable failure class. Agents branch on this without
+    # parsing the page, so each failure must carry its own name — a closed
+    # connection labeled "refused", or a proxy out of file descriptors
+    # labeled "refused", points the reader at the app when the app is fine.
+    def error_kind(reason)
+      case reason
+      when :silent then "backend-silent"
+      when :gone then "backend-gone"
+      when :exhausted then "proxy-exhausted"
+      when :unreachable then "proxy-unreachable"
+      else "backend-refused"
+      end
+    end
+
+    # Which of the failures this was, said the way each one has to be
+    # read: not listening means start the app, silent means go find out
+    # why an app that is up is not answering, exhausted means the trouble
+    # is the proxy's own resource limit. One "not responding" for all of
+    # them sends the reader to the wrong place every time.
     def what_happened(reason, target)
       at = target.empty? ? "" : " at <code>#{escape(target)}</code>"
       case reason
@@ -812,6 +962,12 @@ module Yamine
       when :gone
         "The backend#{at} accepted the connection, then closed it without " \
           "sending a response."
+      when :exhausted
+        "The proxy ran out of file descriptors dialing#{at} — the limit is the " \
+          "proxy process's own, not the app's, and the backend may be perfectly fine."
+      when :unreachable
+        "The proxy could not reach the backend#{at} — the address is wrong or the " \
+          "path there failed, which is not the same as nothing listening."
       else
         "Nothing is listening#{at}."
       end
@@ -832,7 +988,14 @@ module Yamine
       "<p>The backend for <strong>#{escape(host)}</strong> is registered#{by}#{where}.</p>"
     end
 
-    def bad_gateway_fix(entry)
+    def bad_gateway_fix(entry, reason)
+      if [:exhausted, :unreachable].include?(reason)
+        # The app is not the suspect here, so the start command would be
+        # the wrong advice on a page that already says so.
+        return "<p>The backend looks up — check the proxy's own log: " \
+               "<code>proxy.log</code> in the yamine state dir.</p>"
+      end
+
       dir = entry&.dig("spec", "dir")
       return "<p>Start it with <code>yamine start</code> in that app's directory.</p>" unless dir
 

data/lib/yamine/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Yamine
-  VERSION = "0.22.1"
+  VERSION = "0.23.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: yamine
 version: !ruby/object:Gem::Version
-  version: 0.22.1
+  version: 0.23.1
 platform: ruby
 authors:
 - Kaka Ruto