laravel-cloud-rails 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 5ef19c3543f84778c71e51537a0213808fbadd1207511bbad5692812dce5cdd2
+  data.tar.gz: efae967d66fc0eb612102f8a5bd89bb3616ecffa1b9be54821fc551991fbabaa
+SHA512:
+  metadata.gz: fb9895734bd22d3712c126424d41d0ace0bcd9ed2dbc8724f27ab1331e51bf87d172a5b1f09eac5f645a85dace9a06a776101043a9e8801b6779b8535153a05b
+  data.tar.gz: 9961f7167b4ed7ae350f53e86ae1dda35d4d93c3eec6f40b588a0a1ff96ee001ee87ec1ad52e76278838b0155b2bc62fae1713ce2d5d98008b38bd3bdd09fffc

data/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Taylor Otwell
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,234 @@
+# laravel-cloud-rails
+
+Laravel Cloud features for Ruby on Rails applications.
+
+- [Requirements](#requirements)
+- [Installation](#installation)
+- [Features](#features)
+- [Managed Queues](#managed-queues)
+- [WebSockets](#websockets)
+- [Development](#development)
+- [License](#license)
+
+## Requirements
+
+- Ruby 3.2+
+- Rails / Active Job 7.1+
+
+## Installation
+
+Install the gem from GitHub:
+
+```bash
+bundle add laravel-cloud-rails --git https://github.com/laravel/laravel-cloud-rails.git
+```
+
+## Features
+
+| Feature | Summary |
+| --- | --- |
+| [Managed Queues](#managed-queues) | Laravel Cloud managed queues as an Active Job backend. |
+| [WebSockets](#websockets) | Action Cable over Laravel Cloud managed WebSockets. |
+
+---
+
+## Managed Queues
+
+Run Active Job on Laravel Cloud managed queues. Laravel Cloud runs and scales
+the queue workers for you. Throughput, duration and failed jobs appear in the
+Laravel Cloud dashboard, and failed jobs can be retried from there.
+
+### Setup
+
+Use the `:cloud` adapter in production:
+
+```ruby
+# config/environments/production.rb
+config.active_job.queue_adapter = :cloud
+```
+
+That's all. Laravel Cloud provides the rest.
+
+### Dispatching jobs
+
+Use Active Job as usual. A job's queue name is the managed queue it goes to;
+jobs on Active Job's default queue go to your environment's default queue.
+
+```ruby
+class ProcessReportJob < ApplicationJob
+  queue_as :critical
+end
+
+ProcessReportJob.perform_later(report)
+ProcessReportJob.set(wait: 5.minutes).perform_later(report)
+ActiveJob.perform_all_later(jobs)
+```
+
+- `wait:` / `wait_until:` can be at most 15 minutes; longer raises
+  `LaravelCloud::Queue::InvalidDispatch`.
+- `priority` isn't supported and is ignored.
+
+### FIFO queues
+
+A queue whose name ends in `.fifo` delivers jobs in order and deduplicates
+them. It works without extra setup. To control the ordering group or the
+deduplication key:
+
+```ruby
+class ProcessOrderJob < ApplicationJob
+  include LaravelCloud::Queue::MessageAttributes
+
+  queue_as "orders.fifo"
+
+  message_group_id { "customer-#{arguments.first.customer_id}" }
+  message_deduplication_id { "order-#{arguments.first.id}" }
+end
+```
+
+FIFO queues don't support `wait:`.
+
+### Fair queues
+
+On a standard queue, a message group ID marks the tenant a job belongs to, so
+one busy tenant can't starve the others:
+
+```ruby
+class SyncAccountJob < ApplicationJob
+  include LaravelCloud::Queue::MessageAttributes
+
+  message_group_id { "account-#{arguments.first.id}" }
+end
+```
+
+### Retries and failures
+
+`retry_on` and `discard_on` work as usual. Retry waits can be up to 12 hours.
+
+```ruby
+class ProcessReportJob < ApplicationJob
+  retry_on Net::OpenTimeout, wait: :polynomially_longer, attempts: 5
+end
+```
+
+A job's attempts are counted together across all its `retry_on` rules, rather
+than per exception.
+
+An exception that isn't handled fails the job. Failed jobs show up in the
+Laravel Cloud dashboard, where you can retry them.
+
+### Configuration
+
+Optional, in an initializer:
+
+```ruby
+# config/initializers/laravel_cloud.rb
+LaravelCloud::Queue.configure do |queue|
+  # Only run these job classes (default: any Active Job class).
+  queue.allowed_job_classes = %w[ProcessReportJob SendInvoiceJob]
+
+  # Called when a job fails, e.g. to report it to an error tracker.
+  queue.on_failure = ->(error, job_data) { Rails.error.report(error, context: job_data) }
+end
+```
+
+### Local development
+
+Managed queues only exist on Laravel Cloud, so use another adapter locally:
+
+```ruby
+# config/environments/development.rb
+config.active_job.queue_adapter = :async
+```
+
+### Errors
+
+| Error | Raised when |
+| --- | --- |
+| `LaravelCloud::Queue::InvalidDispatch` | A dispatch the queue can't honour: a wait over 15 minutes, a wait on a FIFO queue, or a deduplication ID on a standard queue. |
+| `LaravelCloud::Queue::NotConfigured` | No managed queue is configured. |
+| `ActiveJob::EnqueueError` | The job couldn't be sent to the queue. `perform_later` returns `false`, as with any adapter. |
+
+---
+
+## WebSockets
+
+Run Action Cable on Laravel Cloud managed WebSockets. Your channels and
+`@rails/actioncable` client work unchanged, and broadcasts reach every Puma
+worker and replica.
+
+### Setup
+
+Attach a WebSocket application to your Laravel Cloud environment, then use the
+`cloud` adapter in production:
+
+```yaml
+# config/cable.yml
+production:
+  adapter: cloud
+```
+
+That's all. Laravel Cloud provides the connection settings, and the gem adds
+its browser transport to your HTML pages. No JavaScript changes are needed.
+
+### Usage
+
+Use Action Cable as usual:
+
+```ruby
+class RoomChannel < ApplicationCable::Channel
+  def subscribed
+    stream_from "room:demo"
+  end
+end
+
+ActionCable.server.broadcast("room:demo", { text: "Your report is ready" })
+```
+
+Connection authentication, subscription rejections, `stream_for`, Turbo Streams
+and `remote_connections.where(...).disconnect` all work as usual.
+
+### Limitations
+
+- Each message sent to a browser must stay under about 9.5 KB, Reverb's request
+  limit. A larger message closes that connection without reconnecting.
+- Messages broadcast while a client is reconnecting aren't replayed, as with
+  Action Cable's own adapters.
+- Clients outside HTML pages, such as Web Workers, use Rails' native `/cable`
+  endpoint.
+
+### Local development
+
+Managed WebSockets only exist on Laravel Cloud, so use another adapter locally:
+
+```yaml
+# config/cable.yml
+development:
+  adapter: async
+```
+
+To try the `cloud` adapter locally, run a Reverb server that both Rails
+and the browser can reach, and set `PUSHER_APP_ID`, `PUSHER_APP_KEY`,
+`PUSHER_APP_SECRET`, `PUSHER_HOST`, `PUSHER_PORT` and `PUSHER_SCHEME`.
+
+### Errors
+
+| Error | Raised when |
+| --- | --- |
+| `LaravelCloud::Websockets::NotConfigured` | The `cloud` adapter is used without the `PUSHER_*` variables. |
+| `LaravelCloud::Websockets::PayloadTooLarge` | A broadcast is too large to send. |
+| `LaravelCloud::Websockets::Unavailable` | The WebSocket server can't be reached. |
+
+If Rails logs `backend connection lost (Backend error: Origin not allowed)`,
+check that `APP_URL` matches an allowed origin of your WebSocket application.
+
+## Development
+
+```bash
+bundle install
+bundle exec rake test
+node --test test/*_test.mjs
+```
+
+## License
+
+`laravel-cloud-rails` is open-sourced software licensed under the [MIT license](LICENSE.md).

data/exe/laravel-cloud

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "laravel_cloud"
+require "laravel_cloud/cli"
+
+exit LaravelCloud::CLI.new(ARGV).run

data/lib/action_cable/subscription_adapter/cloud/middleware.rb

@@ -0,0 +1,129 @@
+# frozen_string_literal: true
+
+module ActionCable
+  module SubscriptionAdapter
+    class Cloud
+      # Serves the browser transport, injects it into HTML pages, and receives
+      # the transport's channel commands.
+      class Middleware
+        ENDPOINT = "/__laravel_cloud/cable"
+        COMMANDS = %w[connect ready lease disconnect subscribe unsubscribe message].freeze
+      MAX_BYTES = 8192
+
+        def initialize(app)
+          @app = app
+        end
+
+        def call(environment)
+          return @app.call(environment) unless active?
+          request = ActionDispatch::Request.new(environment)
+          if request.path == "#{ENDPOINT}.js" && request.get?
+            return with_owner(request, [200, { "content-type" => "text/javascript", "cache-control" => "no-store", "x-content-type-options" => "nosniff" }, [script]])
+          end
+          return command(request) if request.path == ENDPOINT && request.post?
+
+          status, headers, body = @app.call(environment)
+          return [status, headers, body] unless status == 200 && headers["content-type"].to_s.include?("text/html") && !headers["content-encoding"] && body.respond_to?(:each)
+          allow_backend(request)
+          nonce = request.content_security_policy_nonce
+          tag = "<script src=\"#{ENDPOINT}.js\"#{nonce ? " nonce=\"#{ERB::Util.html_escape(nonce)}\"" : ''}></script>"
+          with_owner(request, [status, headers.except("content-length", "etag"), HtmlBody.new(body, tag)])
+        end
+
+        private
+
+        # Decided on the first request, once cable.yml and initializers have
+        # configured Action Cable. Without the variables, Action Cable's native
+        # /cable endpoint is left to report the misconfiguration.
+        def active?
+          return @active unless @active.nil?
+          adapter = (ActionCable.server.config.cable || {}).with_indifferent_access[:adapter]
+          @active = adapter.to_s == "cloud" && Websockets.enabled?
+        end
+
+        def command(request)
+          return json(403, error: "Invalid origin") unless request.get_header("HTTP_ORIGIN") == request.base_url
+          return json(415, error: "JSON required") unless request.media_type == "application/json"
+          return json(413, error: "Command too large") if request.content_length.to_i > MAX_BYTES
+          raw = request.body.read(MAX_BYTES + 1)
+          return json(413, error: "Command too large") if raw.bytesize > MAX_BYTES
+          payload = JSON.parse(raw)
+          raise InvalidCommand, "Invalid command" unless payload.is_a?(Hash) && COMMANDS.include?(payload["command"])
+          unless payload["command"] == "connect"
+            raise InvalidCommand, "Request ID required" unless payload["request_id"].is_a?(String) && payload["request_id"].bytesize.between?(1, 64)
+          end
+          if %w[subscribe unsubscribe message].include?(payload["command"])
+            raise InvalidCommand, "Invalid identifier" unless payload["identifier"].is_a?(String) && JSON.parse(payload["identifier"]).is_a?(Hash)
+            raise InvalidCommand, "Invalid message" if payload["command"] == "message" && (!payload["data"].is_a?(String) || !JSON.parse(payload["data"]).is_a?(Hash))
+          end
+          return json(401, error: "Connection cookie required") unless Cloud.owner(request)
+          status, response = Cloud.runtime.execute(payload, request)
+          json(status, response)
+        rescue JSON::ParserError, KeyError, ArgumentError, URI::InvalidURIError, InvalidCommand
+          json(400, error: "Invalid channel command")
+        rescue Websockets::Unavailable
+          json(503, error: "WebSocket backend unavailable")
+        end
+
+        def allow_backend(request)
+          policy = request.content_security_policy
+          return unless policy
+          policy = policy.dup
+          # Without either directive, connections are unrestricted; adding one would restrict them.
+          sources = policy.directives["connect-src"] || policy.directives["default-src"]
+          return unless sources
+          config = Websockets.settings
+          origin = "#{config[:scheme] == 'https' ? 'wss' : 'ws'}://#{config[:host]}:#{config[:port]}"
+          policy.connect_src(*(sources + [origin]).uniq)
+          request.content_security_policy = policy
+        end
+
+        def with_owner(request, response)
+          unless Cloud.owner(request)
+            response[1]["cache-control"] = "private, no-store"
+            Rack::Utils.set_cookie_header!(response[1], "__laravel_cloud_owner", value: Cloud.owner_cookie, path: "/", httponly: true, same_site: :lax, secure: request.ssl?)
+          end
+          response
+        end
+
+        def json(status, payload)
+          [status, { "content-type" => "application/json", "cache-control" => "no-store" }, [JSON.generate(payload)]]
+        end
+
+        def script
+          "globalThis.__laravelCloudWebsockets = #{JSON.generate(Websockets.settings)};\n" + File.read(File.join(__dir__, "transport.js"))
+        end
+      end
+
+      # Inserts the transport right after <head>, before application scripts
+      # capture the WebSocket constructor, without buffering the whole page.
+      class HtmlBody
+        def initialize(body, tag)
+          @body, @tag = body, tag
+        end
+
+        def each
+          buffer = +""
+          pending = true
+          @body.each do |part|
+            unless pending
+              yield part
+              next
+            end
+            buffer << part
+            if buffer.sub!(/<head(?:\s[^>]*)?>/i) { |head| "#{head}#{@tag}" } || buffer.bytesize > 65536
+              yield buffer
+              buffer = +""
+              pending = false
+            end
+          end
+          yield buffer unless buffer.empty?
+        end
+
+        def close
+          @body.close if @body.respond_to?(:close)
+        end
+      end
+    end
+  end
+end

data/lib/action_cable/subscription_adapter/cloud/railtie.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module ActionCable
+  module SubscriptionAdapter
+    class Cloud
+      class Railtie < Rails::Railtie
+        # Always installed: the middleware only acts once cable.yml selects the
+        # cloud adapter, which isn't known until Action Cable loads.
+        initializer "laravel_cloud.action_cable" do |app|
+          app.middleware.use(Middleware)
+        end
+      end
+    end
+  end
+end

data/lib/action_cable/subscription_adapter/cloud/runtime.rb

@@ -0,0 +1,255 @@
+# frozen_string_literal: true
+
+module ActionCable
+  module SubscriptionAdapter
+    class Cloud
+      module BridgedConnection
+        # The runtime's shared heartbeat channel replaces per-connection pings,
+        # which would otherwise each cost a backend publish every 3 seconds.
+        def beat; end
+      end
+
+      # Stands in for the WebSocket of a virtual Action Cable connection, so the
+      # connection encodes, closes and rejects exactly as it would natively.
+      class Session
+        attr_reader :connection, :owner, :token, :last_seen
+
+        def initialize(runtime, token, owner, environment)
+          @runtime, @token, @owner = runtime, token, owner
+          @lock = Monitor.new
+          @closed = @active = false
+          @pending = []
+          @sequence = 0
+          @requests = {}
+          @last_seen = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+          @connection = ActionCable.server.config.connection_class.call.new(ActionCable.server, environment)
+          @connection.extend(BridgedConnection)
+          @connection.instance_variable_set(:@websocket, self)
+        end
+
+        def open
+          @connection.send(:handle_open)
+        end
+
+        def alive?
+          !@closed
+        end
+
+        def possible?
+          true
+        end
+
+        def protocol
+          "actioncable-v1-json"
+        end
+
+        def channel
+          "private-roc-client-#{Digest::SHA256.hexdigest(@token)}"
+        end
+
+        def transmit(data)
+          @lock.synchronize do
+            return if @closed
+            if @active
+              publish(data)
+            else
+              raise InvalidCommand, "Too many initial connection frames" if @pending.size >= 256
+              @pending << data
+            end
+          end
+        end
+
+        def execute(payload)
+          @lock.synchronize do
+            return [410, { error: "Connection expired" }] if @closed
+            @last_seen = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+            request_id = payload.fetch("request_id")
+            return @requests[request_id] if @requests.key?(request_id)
+            case payload.fetch("command")
+            when "ready"
+              @active = true
+              @pending.each { |data| transmit(data) }
+              @pending.clear
+            when "disconnect"
+              close
+            when "lease"
+            else
+              @connection.handle_channel_command(payload.except("token", "request_id"))
+            end
+            result = [200, { ok: true }]
+            @requests[request_id] = result
+            @requests.shift if @requests.size > 128
+            result
+          end
+        end
+
+        def disconnect(reason)
+          @connection.close(reason: reason)
+        rescue Websockets::Unavailable
+          close
+        end
+
+        def close
+          @lock.synchronize do
+            return if @closed
+            @closed = true
+            begin
+              @connection.send(:handle_close)
+            ensure
+              @runtime.remove(@token)
+            end
+          end
+        end
+
+        private
+
+        def publish(data)
+          @sequence += 1
+          # Re-parsed so the frame is published as JSON once, not as a string.
+          Websockets.publish(channel, "cloud-frame", { sequence: @sequence, frame: JSON.parse(data) })
+        rescue Websockets::PayloadTooLarge
+          @sequence -= 1
+          @connection.close(reason: "message_too_large", reconnect: false)
+        end
+      end
+
+      class Runtime
+        attr_reader :pid, :id
+
+        def initialize
+          @pid = Process.pid
+          @id = SecureRandom.hex(16)
+          @lock = Mutex.new
+          @sessions, @pending, @listeners = {}, {}, {}
+          @alive = true
+          @bus = Websockets::Client.new(logger: ActionCable.server.logger) { stop }
+          @bus.subscribe(control_channel, method(:receive))
+          @timer = ActionCable.server.event_loop.timer(3) { pulse }
+        end
+
+        def alive?
+          @alive
+        end
+
+        def control_channel
+          "private-roc-worker-#{@id}"
+        end
+
+        def heartbeat_channel
+          "private-roc-live-#{@id}"
+        end
+
+        def subscribe(stream, callback, success = nil)
+          wrapped = ->(event, data) { callback.call(data) if event == "cloud-broadcast" }
+          @lock.synchronize { @listeners[[stream, callback]] = wrapped }
+          @bus.subscribe(Cloud.stream_channel(stream), wrapped, success)
+        end
+
+        def unsubscribe(stream, callback)
+          wrapped = @lock.synchronize { @listeners.delete([stream, callback]) }
+          @bus.unsubscribe(Cloud.stream_channel(stream), wrapped) if wrapped
+        end
+
+        def execute(payload, request)
+          owner = Cloud.owner(request)
+          return connect(payload, request, owner) if payload.fetch("command") == "connect"
+          worker = verify_token(payload.fetch("token"))
+          return [404, { error: "Unknown connection" }] unless worker
+          return local(payload, owner) if worker == @id
+
+          request_id = SecureRandom.hex(16)
+          queue = ::Queue.new
+          @lock.synchronize { @pending[request_id] = queue }
+          Websockets.publish("private-roc-worker-#{worker}", "cloud-rpc", { id: request_id, reply_to: control_channel, owner: owner, payload: payload })
+          queue.pop(timeout: 5) || [410, { error: "Connection owner unavailable" }]
+        ensure
+          @lock.synchronize { @pending.delete(request_id) } if request_id
+        end
+
+        def remove(token)
+          @lock.synchronize { @sessions.delete(token) }
+        end
+
+        def stop
+          return unless @alive
+          @alive = false
+          @timer&.shutdown
+          sessions = @lock.synchronize { @sessions.values.dup }
+          sessions.each do |session|
+            Rails.application.executor.wrap { session.disconnect("server_restart") }
+          rescue StandardError => error
+            ActionCable.server.logger.error("laravel-cloud-rails cleanup failed (#{error.class})")
+          end
+          ActionCable.server.connections.dup.each do |connection|
+            connection.close(reason: "server_restart") unless connection.is_a?(BridgedConnection)
+          end
+          @bus.close
+        end
+
+        private
+
+        def verify_token(token)
+          return unless token.is_a?(String) && token.match?(/\A[0-9a-f]{32}\.[0-9a-f]{48}\.[0-9a-f]{64}\z/)
+          worker, nonce, signature = token.split(".")
+          worker if ActiveSupport::SecurityUtils.secure_compare(Websockets.sign("#{worker}.#{nonce}"), signature)
+        end
+
+        def connect(payload, request, owner)
+          socket_id = payload.fetch("socket_id")
+          raise InvalidCommand, "Invalid socket ID" unless socket_id.is_a?(String) && socket_id.match?(/\A\d+\.\d+\z/)
+          url = URI.parse(payload.fetch("url", "/cable"))
+          raise InvalidCommand, "Invalid connection URL" unless url.path&.start_with?("/") && url.path.length < 2048
+          value = "#{@id}.#{SecureRandom.hex(24)}"
+          token = "#{value}.#{Websockets.sign(value)}"
+          environment = request.env.merge("PATH_INFO" => url.path, "QUERY_STRING" => url.query.to_s)
+          session = Session.new(self, token, owner, environment)
+          @lock.synchronize do
+            raise Websockets::Unavailable, "Too many connections" if @sessions.size >= 10000 || @sessions.values.count { |s| s.owner == owner } >= 32
+            @sessions[token] = session
+          end
+          session.connection.worker_pool.work(session.connection) { session.open }
+          return [401, { error: "Unauthorized connection" }] unless session.alive?
+          [200, { token: token, channels: [session.channel, heartbeat_channel].map { |channel| { name: channel, auth: Websockets.authorize(socket_id, channel) } } }]
+        rescue StandardError
+          session&.close
+          raise
+        end
+
+        def local(payload, owner)
+          session = @lock.synchronize { @sessions[payload["token"]] }
+          return [410, { error: "Connection expired" }] unless session
+          return [404, { error: "Unknown connection" }] unless ActiveSupport::SecurityUtils.secure_compare(session.owner, owner)
+          session.connection.worker_pool.work(session.connection) { session.execute(payload) }
+        rescue InvalidCommand, KeyError
+          [400, { error: "Invalid channel command" }]
+        rescue Websockets::Unavailable
+          [503, { error: "WebSocket backend unavailable" }]
+        end
+
+        def receive(event, data)
+          if event == "cloud-reply"
+            @lock.synchronize { @pending[data["id"]] }&.push(data.fetch("result"))
+          elsif event == "cloud-rpc"
+            result = Rails.application.executor.wrap { local(data.fetch("payload"), data.fetch("owner")) }
+            Websockets.publish(data.fetch("reply_to"), "cloud-reply", { id: data.fetch("id"), result: result })
+          end
+        rescue StandardError => error
+          ActionCable.server.logger.error("laravel-cloud-rails command failed (#{error.class})")
+        end
+
+        def pulse
+          @bus.ping
+          sessions = @lock.synchronize { @sessions.values.dup }
+          return if sessions.empty?
+          cutoff = Process.clock_gettime(Process::CLOCK_MONOTONIC) - 90
+          sessions.each do |session|
+            Rails.application.executor.wrap { session.disconnect("connection_timeout") } if session.last_seen < cutoff
+          end
+          Websockets.publish(heartbeat_channel, "cloud-heartbeat", { time: Time.now.to_i })
+        rescue Websockets::Unavailable
+          stop
+        end
+      end
+    end
+  end
+end