foobara-aws 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 16a858ce4e2944711259dacbf844fd258a1715018aaa7df98364e45739ec33de
-  data.tar.gz: 995b9e3af507e6cb53e3fd581945d27b9251f9559f0151c9151e5fa5bb277191
+  metadata.gz: 3905338908c79d15be8eea5643596893e7604a35d91b78760de209f06822c230
+  data.tar.gz: 6a771a30dcc41281ee879b80c2536a6f63bf7a9755846be1685ab1d474034ba5
 SHA512:
-  metadata.gz: a328cae1d9959faa1efc89dc399f007005673493704ae20dbe9e02904b85949f7eb76debefeff09830f2a367315efac9bde77981c660690299aced88f5871464
-  data.tar.gz: a0cb8f25daa37b5ea097d5bf13e145c0a7df3a27b2d6faa801e426dafb3f8f00def81a55093d7c95818cd2caf32b2d7bbe7c6f3864248993b099f749d1218d52
+  metadata.gz: cfba5032f0d68511918ed2e2fca46bdf3842bca216c5361fcda23b1360cd6ea8708793ebf7d5063ae68d10d3e738d87337624a7a0c3d11fc8b598a65d3965107
+  data.tar.gz: 0fda844ba145042a870a580ef2c445fb8d5e5359866c0134c70a7fea0e67a58326cfbbf77ea6f837af7f2e1dcb64b30861278de84738ed75a64a5c7b85d478b8

data/CHANGELOG.md

@@ -1,5 +1,33 @@
 ## [Unreleased]
 
+## [0.4.0] - 2026-08-05
+
+- `Foobara::AWS::SQSConnector` and `Foobara::AWS::EventHandler` — commands invoked by a queue rather than by HTTP.
+  Modelled on foobara-resque-connector, and using the same envelope, because a background job is not a special
+  kind of object: it is a command reached by a different transport. `connect` generates a `<Command>Async` that
+  enqueues, so a malformed job is refused where it is created rather than by a worker later.
+- With no queue URL the connector runs the command inline, so local development and test suites need no SQS and
+  no fake. The queue is a deployment concern.
+- `EventHandler` returns `batchItemFailures`, so one poison record does not fail its nine neighbours. `Service`
+  creates the event source mapping with `report_batch_item_failures: true`, without which that return value is
+  ignored.
+- `Unit` gained a `trigger` (`:http` or `:sqs`), and `plan_from_connectors` plans several connectors at once. An
+  SQS unit gets a queue, a dead-letter queue and an event source mapping instead of a route, and its commands
+  stay out of the authorizer's anonymous list.
+
+- `Packager` takes `provenance:` — a file recording which revision each unit last **changed** at, and `Service`
+  takes `provenance:` to stamp it as `FOOBARA_REVISION`. Deliberately not the deploying revision: stamping HEAD
+  everywhere would change every function's configuration on every commit, so CloudFormation would update all of
+  them and `cdk diff` would show all of them changing whether or not any code did.
+- A unit's revision moves only when its artifact's content hash moves, so an untouched function keeps the honest
+  answer to "what is actually running here?" and its configuration does not churn.
+- A revision recorded from a dirty tree is marked `-dirty` and treated as provisional: it is replaced by the first
+  clean revision seen to contain the same content, rather than sticking until the unit next changes.
+- `Service` now has specs. It previously had one — that it raises without aws-cdk-lib — which let a real defect
+  through: a method defined and never called, so an environment variable silently vanished. There is now a fake
+  CDK (`spec/support/fake_cdk.rb`) exercising functions, sizing, routes, tracing and the authorizer, including
+  that the authorizer declares no identity source and no caching.
+
 ## [0.3.0] - 2026-08-04
 
 - `Foobara::AWS.plan_from_connector(connector)` — build a plan by inspecting a live connector instead of a

data/README.md

@@ -15,6 +15,28 @@ service = Foobara::AWS::CDK::Service.new(self, "Api", plan: plan, code_root: "bu
 posts_table.grant_read_write_data(service.function("posts"))
 ```
 
+## Installation
+
+```bash
+bundle add foobara-aws
+```
+
+**No runtime dependencies.** `aws-cdk-lib` is your CDK app's, `foobara` is your
+application's, and `jwt` belongs to the authorizer's own deployment unit — each
+is referenced lazily, only where it is needed. That matters more than usual
+here, because half this gem is loaded *inside a Lambda*, where every dependency
+is paid for on each cold start.
+
+Which half you need depends on where you are:
+
+| you are | require | you need |
+| --- | --- | --- |
+| planning or packaging, in a build step | `foobara/aws/packager` | `foobara` |
+| synthesising, in a CDK app | `foobara/aws/cdk/service` | `aws-cdk-lib` |
+| serving a command, in a Lambda | `foobara/aws/handler` | neither |
+| verifying a token, in the authorizer | `foobara/aws/authorizer` | `jwt` |
+| checking a deployment | `foobara/aws/check` | neither |
+
 ## Where a plan comes from
 
 ```ruby
@@ -31,23 +53,14 @@ alone cannot say which commands are public.
 
 All three produce the same `Plan`, and a spec asserts the first two agree.
 
-## Two halves, on purpose
-
-```ruby
-Foobara::AWS.plan(manifest)   # what to deploy. Plain data. No CDK, no Foobara.
-Foobara::AWS::CDK::Service.new     # the AWS resources. No application.
-```
-
-They are separable because they usually run in different places. The plan is
-produced where the app is — a running connector serves `/manifest` — and
-consumed where the infrastructure is, which is often a CDK app that should not
-have to load an application in order to deploy it.
+## Carrying a plan between processes
 
-A plan survives a JSON round trip, so it travels as a build artifact:
+A plan is plain data and survives a JSON round trip, so it can be produced in one
+process and consumed in another:
 
 ```ruby
-# in the build, against a running connector
-File.write("build/plan.json", JSON.pretty_generate(Foobara::AWS.plan(manifest).to_h))
+# in the build, where the app is loaded
+File.write("build/plan.json", JSON.pretty_generate(Foobara::AWS.plan_from_connector(connector).to_h))
 
 # in the CDK app — no Foobara, no ORM, no application gems
 Foobara::AWS::CDK::Service.new(self, "Api",
@@ -55,9 +68,21 @@ Foobara::AWS::CDK::Service.new(self, "Api",
   code_root: "build")
 ```
 
-That also gets you a property worth having: synthesis depends on what was
-actually built, so `cdk synth` cannot create a route to a Lambda whose code is
-missing. `Service` raises if a unit's artifact directory is not there.
+Worth understanding the trade rather than picking by default, because the two
+options fail in opposite directions:
+
+- **Read the connector at synth time** and the topology can never lag the code,
+  because it *is* the code. Synthesis then loads the application, so it is
+  slower and can fail for application reasons. If you do this, have the CDK
+  app's Gemfile inherit the application's rather than restating it — resolving a
+  second set of Foobara versions means the plan is computed by a different
+  Foobara than the one deployed.
+- **Read a file** and synthesis depends on what was actually *built*, needs none
+  of the application's gems, and stays deterministic and offline. But the file is
+  only as current as the last build.
+
+Either way `Service` raises when a unit has no artifact, so a command added but
+not packaged fails at synth rather than 404ing after deploy.
 
 ## What the plan reads
 

data/lib/foobara/aws/cdk/service.rb

@@ -26,12 +26,13 @@ module Foobara
       class Service
         DEFAULT_SIZING = { memory_size: 1024, timeout_seconds: 10 }.freeze
 
-        attr_reader :api, :functions, :plan
+        attr_reader :api, :functions, :plan, :queues
 
         # rubocop:disable Metrics/ParameterLists
         def initialize(scope, id, plan:, code_root:, sizing: {}, defaults: DEFAULT_SIZING,
                        environment: {}, runtime: nil, architecture: nil, authorizer: nil,
-                       api_props: {}, tracing: nil, log_retention: nil)
+                       api_props: {}, tracing: nil, log_retention: nil, provenance: {},
+                       batch_size: 10)
           # Checked first: everything below needs it, and a bare LoadError from
           # `constructs` would say nothing about which dependency is missing.
           cdk
@@ -46,8 +47,16 @@ module Foobara
           @architecture = architecture
           @tracing = tracing
           @log_retention = log_retention
+          # unit name => the revision its artifact last changed at, as the
+          # packager recorded it. Deliberately NOT the deploying revision: a
+          # function that has not changed keeps the answer to "what is actually
+          # running here?", and its configuration does not churn on every
+          # deploy.
+          @provenance = provenance.transform_keys(&:to_s)
+          @batch_size = batch_size
           @functions = {}
 
+          @queues = {}
           @api = cdk::APIGatewayv2::HttpAPI.new(@scope, "Api", api_props)
           @authorizer = build_authorizer(authorizer)
           plan.units.each { |unit| add_unit(unit) }
@@ -58,6 +67,10 @@ module Foobara
         #   table.grant_read_write_data(service.function("posts"))
         def function(name) = @functions.fetch(name.to_s)
 
+        # The queue an SQS unit consumes, so the units that ENQUEUE onto it can
+        # be granted send access and told its URL.
+        def queue(name) = @queues.fetch(name.to_s)
+
         def url = @api.url
 
         private
@@ -66,6 +79,8 @@ module Foobara
           fn = build_function(unit)
           @functions[unit.name] = fn
 
+          return add_queue(unit, fn) unless unit.trigger == :http
+
           props = {
             path: unit.route,
             methods: [cdk::APIGatewayv2::HttpMethod::ANY],
@@ -77,12 +92,44 @@ module Foobara
           @api.add_routes(props)
         end
 
+        # An SQS unit is reached by an event source mapping rather than a route,
+        # so it gets a queue, a dead-letter queue behind it, and no URL at all.
+        def add_queue(unit, function)
+          dead_letter = cdk::SQS::Queue.new(@scope, "#{logical(unit.name)}Dlq", {})
+
+          queue = cdk::SQS::Queue.new(@scope, "#{logical(unit.name)}Queue", {
+                                        # Longer than the function's own timeout, or SQS
+                                        # redelivers work that is still running — which looks
+                                        # like duplicate processing rather than a timeout.
+                                        visibility_timeout: cdk::Duration.seconds(
+                                          sizing_for(unit).fetch(:timeout_seconds) * 6
+                                        ),
+                                        dead_letter_queue: { max_receive_count: 5, queue: dead_letter }
+                                      })
+
+          function.add_event_source(
+            cdk::LambdaEventSources::SQSEventSource.new(queue, {
+                                                          batch_size: @batch_size,
+                                                          # Without this, returning batchItemFailures
+                                                          # is ignored and one bad record fails its
+                                                          # nine neighbours.
+                                                          report_batch_item_failures: true
+                                                        })
+          )
+
+          @queues[unit.name] = queue
+        end
+
+        def sizing_for(unit)
+          @defaults.merge(unit.sizing).merge(@sizing.fetch(unit.name, {}))
+        end
+
         def build_function(unit)
           # Precedence: an explicit sizing: override, then whatever the unit's
           # commands declared via aws_lambda, then the defaults. The override
           # exists for the case the declaration cannot know about — an
           # environment where everything should be small, say.
-          sizing = @defaults.merge(unit.sizing).merge(@sizing.fetch(unit.name, {}))
+          sizing = sizing_for(unit)
           asset = File.join(@code_root, unit.name)
           unless Dir.exist?(asset)
             raise Error, "no artifact for unit #{unit.name.inspect} at #{asset} — build it before synthesising"
@@ -97,7 +144,7 @@ module Foobara
             code: cdk::Lambda::Code.from_asset(asset),
             memory_size: sizing.fetch(:memory_size),
             timeout: cdk::Duration.seconds(sizing.fetch(:timeout_seconds)),
-            environment: @environment.merge("FOOBARA_UNIT" => unit.name)
+            environment: unit_environment(unit)
           }.merge(observability))
         end
 
@@ -151,6 +198,13 @@ module Foobara
           }.merge(observability))
         end
 
+        def unit_environment(unit)
+          env = @environment.merge("FOOBARA_UNIT" => unit.name)
+          revision = @provenance[unit.name]
+          env["FOOBARA_REVISION"] = revision if revision
+          env
+        end
+
         # Tracing and log retention, applied identically to every function
         # including the authorizer — a trace that stops at the authorizer would
         # hide the one invocation that happens on every single request.
@@ -185,7 +239,10 @@ module Foobara
         # anything at stack level — a "comments" function and a "Comments" table
         # both want the id "Comments" otherwise, and synthesis fails.
         def construct(scope, id)
-          require "constructs"
+          # Only if it is not already there: aws-cdk-lib loads it itself, and
+          # requiring it again is pointless work in the common case.
+          require "constructs" unless defined?(::Constructs)
+
           Constructs::Construct.new(scope, id)
         end
 

data/lib/foobara/aws/event_handler.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+# Running Foobara commands from an AWS event source — SQS, and anything else
+# that delivers records in batches.
+#
+# RUNTIME, not synth. Require this and nothing else from the gem.
+#
+#   require "foobara/aws/event_handler"
+#   require_relative "config/connector"
+#
+#   HANDLER = Foobara::AWS::EventHandler.new(QUEUE_CONNECTOR)
+#   def handle(event:, context:) = HANDLER.call(event, context)
+#
+# PARTIAL BATCH FAILURE is the whole reason this is not three lines. SQS hands a
+# Lambda up to ten records at once. Raising on one of them fails the entire
+# batch, so the nine that succeeded are delivered again — work is repeated, and a
+# single poison message can replay its batch until the queue's retry limit.
+#
+# Returning `batchItemFailures` tells SQS which records to redeliver and lets the
+# rest be deleted. It only works if the event source mapping was created with
+# `report_batch_item_failures: true`, which Foobara::AWS::CDK::Service does.
+
+require "json"
+
+require_relative "error"
+require_relative "version"
+
+module Foobara
+  module AWS
+    class EventHandler
+      # What an enqueued message carries. Deliberately the same envelope the
+      # Resque connector uses — a command's name and its inputs are all that is
+      # needed, and keeping the shape means a queue is just another transport.
+      COMMAND_KEY = "command_name"
+      INPUTS_KEY = "inputs"
+
+      # +decode+ turns one record into [command_name, inputs] for event sources
+      # that carry no envelope — an S3 notification or a schedule knows nothing
+      # about commands, so the application has to say what it means.
+      def initialize(connector, decode: nil)
+        @connector = connector
+        @decode = decode
+      end
+
+      def call(event, _context = nil)
+        failures = records(event).filter_map { |record| failure_for(record) }
+
+        { "batchItemFailures" => failures }
+      end
+
+      private
+
+      def records(event) = Array(event["Records"] || event[:Records])
+
+      # Returns nil when the record succeeded, or its identifier when it did not.
+      # Nothing is re-raised: one bad record must not fail its nine neighbours,
+      # which is the entire point.
+      def failure_for(record)
+        run(record)
+        nil
+      rescue StandardError => e
+        # The message id, because that is what SQS asks for. Logged as well as
+        # reported, or a failure that keeps being redelivered is invisible.
+        warn("foobara-aws: record #{identifier(record)} failed: #{e.class}: #{e.message}")
+        { "itemIdentifier" => identifier(record) }
+      end
+
+      def identifier(record) = record["messageId"] || record[:messageId]
+
+      def run(record)
+        name, inputs = command_and_inputs(record)
+        command_class = @connector.transformed_command_from_name(name)
+        raise Error, "no command #{name.inspect} is connected" unless command_class
+
+        # run!, not run: a command that returns an error outcome has failed, and
+        # this needs to raise so the record is reported and redelivered.
+        command_class.new(inputs || {}).run!
+      end
+
+      def command_and_inputs(record)
+        return @decode.call(record) if @decode
+
+        body = record["body"] || record[:body]
+        envelope = body.is_a?(String) ? JSON.parse(body) : body
+        [envelope[COMMAND_KEY], envelope[INPUTS_KEY]]
+      end
+    end
+  end
+end

data/lib/foobara/aws/packager.rb

@@ -37,7 +37,8 @@ module Foobara
       # rubocop:disable Metrics/ParameterLists
       def initialize(plan:, root: Dir.pwd, out: "build", sources: %w[app config],
                      gemfiles: "units", image: DEFAULT_IMAGE, env: {}, mounts: [],
-                     handler_template: nil, authorizer: nil, docker: nil)
+                     handler_template: nil, authorizer: nil, docker: nil,
+                     provenance: nil, revision: nil)
         @plan = plan
         @root = File.expand_path(root)
         @out = File.expand_path(out, @root)
@@ -51,6 +52,11 @@ module Foobara
         @mounts = mounts
         @handler_template = handler_template
         @authorizer = authorizer
+        # Where to keep the record of which revision each unit last CHANGED at.
+        # Outside `out` on purpose: it is only useful if it outlives a build and
+        # is committed, which a build directory is usually not.
+        @provenance_path = provenance
+        @revision = revision
         # Injectable so the layout can be tested without Docker, and so an
         # environment with a different container runtime can substitute one.
         @docker = docker || method(:run_docker)
@@ -64,13 +70,92 @@ module Foobara
         built << build_authorizer if @authorizer
 
         File.write(plan_path, JSON.pretty_generate(@plan.to_h))
+        record_provenance(built) if @provenance_path
         built
       end
 
+      # unit name => the revision at which its artifact last changed. Empty
+      # unless `provenance:` was given.
+      attr_reader :provenance
+
       def plan_path = File.join(@out, "plan.json")
 
       private
 
+      # WHY THIS IS NOT SIMPLY THE CURRENT REVISION.
+      #
+      # Stamping every unit with HEAD would change every function's environment
+      # on every commit, so CloudFormation would update all of them and `cdk
+      # diff` would show all of them changing — every time, whether or not any
+      # of their code did. That is exactly the signal worth keeping.
+      #
+      # So a unit's revision only moves when its CONTENT moves. An untouched
+      # unit keeps the revision it last changed at, its configuration does not
+      # change, and CloudFormation leaves it alone. The file is also a readable
+      # history: `git log -p` over it says when each Lambda actually changed.
+      def record_provenance(built)
+        previous = read_provenance
+        current = built.to_h { |unit| [unit.fetch(:name), entry_for(unit, previous)] }
+
+        File.write(@provenance_path, "#{JSON.pretty_generate(current.sort.to_h)}\n")
+        @provenance = current.transform_values { |entry| entry["revision"] }
+      end
+
+      # Unchanged content keeps its old revision. Only a real change adopts the
+      # current one.
+      #
+      # Except when the old revision is PROVISIONAL — recorded from a dirty tree,
+      # so it names a commit that did not contain this content. Those are
+      # replaced by the first clean revision seen to contain it, which is the
+      # best available answer and, unlike the alternative, self-healing: the
+      # marker would otherwise stick until the unit next changed, which could be
+      # never.
+      def entry_for(unit, previous)
+        content = content_hash(unit.fetch(:dir))
+        before = previous[unit.fetch(:name)]
+        keep = before && before["content"] == content && !provisional?(before["revision"])
+
+        { "revision" => keep ? before["revision"] : revision_now, "content" => content }
+      end
+
+      def provisional?(revision) = revision.to_s.end_with?("-dirty")
+
+      def read_provenance
+        return {} unless File.exist?(@provenance_path)
+
+        JSON.parse(File.read(@provenance_path))
+      rescue JSON::ParserError
+        {}
+      end
+
+      # The artifact's contents, path-sensitively. Sorted so it does not depend
+      # on directory order, and paths are hashed as well as bytes so a rename is
+      # a change.
+      def content_hash(dir)
+        digest = Digest::SHA256.new
+
+        Dir.glob("**/*", File::FNM_DOTMATCH, base: dir).sort.each do |relative|
+          path = File.join(dir, relative)
+          next if File.directory?(path)
+
+          digest << relative << "\0" << Digest::SHA256.file(path).hexdigest
+        end
+
+        digest.hexdigest[0, 16]
+      end
+
+      # `-dirty` because a revision recorded from an unclean tree does not
+      # identify the code that was built, and silently pretending otherwise is
+      # worse than an ugly string.
+      def revision_now
+        @revision_now ||= @revision || begin
+          sha = `git rev-parse --short HEAD 2>/dev/null`.strip
+          raise Error, "no revision available: pass revision: or run inside a git repository" if sha.empty?
+
+          `git status --porcelain 2>/dev/null`.strip.empty? ? sha : "#{sha}-dirty"
+        end
+      end
+
       def build_unit(unit)
         dir = prepare(unit.name)
         @sources.each { |s| FileUtils.cp_r(File.join(@root, s), dir) }
@@ -101,7 +186,13 @@ module Foobara
       end
 
       def handler_source(unit)
-        template = @handler_template ? File.read(@handler_template) : DEFAULT_HANDLER
+        template =
+          if @handler_template
+            File.read(@handler_template)
+          else
+            unit.trigger == :http ? DEFAULT_HANDLER : DEFAULT_EVENT_HANDLER
+          end
+
         render(template, unit)
       end
 
@@ -252,6 +343,27 @@ module Foobara
         end
       ERB
 
+      # For a unit consumed from an event source rather than routed. It connects
+      # nothing itself: the application's own connector already knows which
+      # commands it consumes, and re-deriving that here could disagree with it.
+      DEFAULT_EVENT_HANDLER = <<~ERB
+        # Generated by foobara-aws for unit <%= unit.name.inspect %>. Do not edit.
+        #
+        # Consumes: <%= unit.commands.join(", ") %>
+        #
+        # Triggered by <%= unit.trigger %>, so there is no route and no
+        # authorizer — a message is already inside the trust boundary.
+        require_relative "vendor/bundle/bundler/setup"
+        require "foobara/aws/event_handler"
+        require_relative "config/connector"
+
+        HANDLER = Foobara::AWS::EventHandler.new(FOOBARA_EVENT_CONNECTOR)
+
+        def handle(event:, context:)
+          HANDLER.call(event, context)
+        end
+      ERB
+
       DEFAULT_AUTHORIZER = <<~ERB
         # Generated by foobara-aws. Do not edit.
         #

data/lib/foobara/aws/plan.rb

@@ -12,7 +12,10 @@ module Foobara
     # +sizing+ is whatever the unit's commands declared with
     # {Foobara::AWS::Lambda}, reduced to one value per key — empty when none
     # of them said anything.
-    Unit = Data.define(:name, :group, :commands, :route, :public_commands, :sizing)
+    # +trigger+ is how the unit is invoked: :http for a routed command, :sqs for
+    # one consumed from an SQS queue. An SQS unit has no route — it is reached by an
+    # event source mapping, not a URL — so `route` is nil for it.
+    Unit = Data.define(:name, :group, :commands, :route, :public_commands, :sizing, :trigger)
 
     # The whole topology, read out of a Foobara manifest.
     #
@@ -38,7 +41,7 @@ module Foobara
           "units" => units.map do |u|
             { "name" => u.name, "group" => u.group, "commands" => u.commands,
               "route" => u.route, "public_commands" => u.public_commands,
-              "sizing" => u.sizing }
+              "sizing" => u.sizing, "trigger" => u.trigger.to_s }
           end
         }
       end
@@ -56,7 +59,8 @@ module Foobara
             u = u.transform_keys(&:to_s)
             Unit.new(name: u.fetch("name"), group: u["group"], commands: u.fetch("commands"),
                      route: u["route"], public_commands: u.fetch("public_commands", []),
-                     sizing: (u["sizing"] || {}).to_h { |k, v| [k.to_sym, v] })
+                     sizing: (u["sizing"] || {}).to_h { |k, v| [k.to_sym, v] },
+                     trigger: u.fetch("trigger", "http").to_sym)
           end
         )
       end
@@ -72,13 +76,13 @@ module Foobara
       # A connector's manifest, not Foobara.manifest: `requires_authentication`
       # only exists once commands are connected, and it is the field the public
       # list comes from.
-      def plan(manifest, mount: "/run", granularity: :domain, exclude: DEFAULT_EXCLUDE)
+      def plan(manifest, mount: "/run", granularity: :domain, exclude: DEFAULT_EXCLUDE, trigger: :http)
         commands = app_commands(manifest, exclude)
         units =
           case granularity
-          when :domain then group_units(commands, "domain", mount)
-          when :organization then group_units(commands, "organization", mount)
-          when :command then command_units(commands, mount)
+          when :domain then group_units(commands, "domain", mount, trigger)
+          when :organization then group_units(commands, "organization", mount, trigger)
+          when :command then command_units(commands, mount, trigger)
           else raise ArgumentError, "unknown granularity #{granularity.inspect}"
           end
 
@@ -99,9 +103,24 @@ module Foobara
       #
       # It adapts each command into the shape {plan} already reads, so there is
       # one implementation and the two routes cannot drift.
+      # Several connectors at once, keyed by how each is triggered:
+      #
+      #   Foobara::AWS.plan_from_connectors(http: rack_connector, sqs: sqs_connector)
+      #
+      # A domain can appear under both — Profiles serving HTTP commands and
+      # consuming a queue — so a non-http unit's name is suffixed. Otherwise two
+      # units would claim the same name and the same artifact directory.
+      def plan_from_connectors(connectors, **)
+        plans = connectors.map { |trigger, connector| plan_from_connector(connector, trigger:, **) }
+        first = plans.first or raise ArgumentError, "no connectors given"
+
+        Plan.new(mount: first.mount, granularity: first.granularity,
+                 units: plans.flat_map(&:units).sort_by(&:name))
+      end
+
       def plan_from_connector(connector, **)
         commands = connector.command_registry.all_transformed_command_classes.to_h do |transformed|
-          [transformed.command_class.full_command_name, describe_command(transformed)]
+          [command_class_of(transformed).full_command_name, describe_command(transformed)]
         end
 
         plan({ "command" => commands }, **)
@@ -123,19 +142,44 @@ module Foobara
       # planning reads — a manifest carries much more, and none of the rest
       # matters here.
       def describe_command(transformed)
-        command_class = transformed.command_class
+        command_class = command_class_of(transformed)
 
         {
           "domain" => domain_name(command_class),
           "organization" => organization_name(command_class),
           "scoped_full_path" => Array(command_class.scoped_full_path),
-          "requires_authentication" => transformed.requires_authentication,
+          # Only a connector that gates commands answers this. One that does
+          # not — a queue has no anonymous callers — leaves it nil, which reads
+          # as "not gated" and is right: the question does not apply.
+          "requires_authentication" => gated?(transformed),
           "depends_on" => Array(command_class.depends_on).map(&:to_s),
           # The AWSLambda DSL, when the command extends it.
           "aws_lambda" => (command_class.aws_lambda_manifest if command_class.respond_to?(:aws_lambda_manifest))
         }.compact
       end
 
+      # A connector wraps commands in a TransformedCommand only when it has
+      # something to transform. One with no transformers — a queue connector,
+      # say — registers the command class itself.
+      #
+      # Rescuing rather than asking respond_to?: Foobara namespaces answer
+      # respond_to? through method_missing, so the question comes back true and
+      # the call still raises.
+      def command_class_of(transformed)
+        transformed.command_class
+      rescue NoMethodError
+        transformed
+      end
+
+      # Only a connector that gates commands answers this. One that does not —
+      # a queue has no anonymous callers — leaves it nil, which reads as "not
+      # gated", and for a unit with no route the question does not arise.
+      def gated?(transformed)
+        transformed.requires_authentication
+      rescue NoMethodError
+        nil
+      end
+
       def domain_name(command_class)
         command_class.domain.scoped_full_name
       rescue StandardError
@@ -154,11 +198,11 @@ module Foobara
         end
       end
 
-      def group_units(commands, key, mount)
+      def group_units(commands, key, mount, trigger)
         commands.group_by { |_name, c| c[key].to_s }.map do |group, members|
           names = members.map(&:first)
           Unit.new(
-            name: slug(group),
+            name: unit_name(slug(group), trigger),
             group: group,
             commands: names.sort,
             # Derived from the commands' own paths, not from the group's name.
@@ -171,18 +215,24 @@ module Foobara
             # safe with a REQUEST authorizer, which sees the path and can decide
             # per command; a JWT authorizer attaches per route, so a unit mixing
             # public and authenticated commands would have to be split.
-            route: greedy_route(members, mount),
-            public_commands: public_of(members),
-            sizing: sizing_of(members)
+            # No route for anything but HTTP: an SQS unit is reached by an
+            # event source mapping.
+            route: (greedy_route(members, mount) if trigger == :http),
+            # Only meaningful for something with a route. A queue has no
+            # anonymous callers, and letting these through would put the command
+            # in the authorizer's anonymous list, where it does not belong.
+            public_commands: trigger == :http ? public_of(members) : [],
+            sizing: sizing_of(members),
+            trigger: trigger
           )
         end
       end
 
-      def command_units(commands, mount)
+      def command_units(commands, mount, trigger)
         commands.map do |name, command|
           path = Array(command["scoped_full_path"])
           Unit.new(
-            name: slug(name),
+            name: unit_name(slug(name), trigger),
             group: command["domain"].to_s,
             # Closed over `depends_on`, transitively. A command that invokes
             # another must be able to reach it, so a per-command artifact has to
@@ -196,12 +246,13 @@ module Foobara
             commands: closure(name, commands),
             # Exact, not greedy: one command per Lambda needs no wildcard, and an
             # exact route is what lets API Gateway 404 an unknown command itself.
-            route: "#{mount}/#{path.join("/")}",
+            route: ("#{mount}/#{path.join("/")}" if trigger == :http),
             # Only the ROUTED command decides this. A dependency is invoked in
             # process, never over HTTP, so whether it requires authentication
             # says nothing about whether this route may be called anonymously.
-            public_commands: public_of([[name, command]]),
-            sizing: sizing_of([[name, command]])
+            public_commands: trigger == :http ? public_of([[name, command]]) : [],
+            sizing: sizing_of([[name, command]]),
+            trigger: trigger
           )
         end
       end
@@ -263,6 +314,11 @@ module Foobara
         end
       end
 
+      # HTTP units keep their bare name, so nothing about existing deployments
+      # changes; anything else is suffixed to stay distinct from a same-named
+      # domain served over HTTP.
+      def unit_name(name, trigger) = trigger == :http ? name : "#{name}-#{trigger}"
+
       def symbolize(hash) = hash.to_h { |k, v| [k.to_sym, v] }
 
       # A CloudFormation-friendly, filesystem-friendly identifier: "Posts" and

data/lib/foobara/aws/sqs_connector.rb

@@ -0,0 +1,120 @@
+# frozen_string_literal: true
+
+# A connector that runs commands from an SQS queue instead of over HTTP.
+#
+# Named for the service rather than for the idea: it speaks SQS specifically —
+# its envelope, its client, its at-least-once delivery — and calling it a queue
+# connector would suggest it abstracts over queues, which it does not.
+#
+# Modelled on foobara-resque-connector, and on purpose: the envelope is the same
+# (`command_name` and `inputs`), so a background job is not a special kind of
+# object — it is a command reached by a different transport.
+#
+#   QUEUE = Foobara::AWS::SQSConnector.new(queue_url: ENV["RECONCILE_QUEUE_URL"])
+#   QUEUE.connect(Profiles::ReconcileAuthorSnapshot)
+#
+#   Profiles::ReconcileAuthorSnapshotAsync.run!(user_sub: sub)   # enqueues
+#
+# `connect` generates that `…Async` command, so enqueuing is itself a command:
+# it validates its inputs against the real command's input type, so a malformed
+# job is refused where it is enqueued rather than discovered by a worker later.
+#
+# WITH NO QUEUE URL IT RUNS INLINE. That is what makes local development and a
+# test suite work without SQS or a fake, and it means the queue is a deployment
+# concern rather than something the application has to know about.
+#
+# Requires Foobara. The consuming half — Foobara::AWS::EventHandler — does not.
+
+require "json"
+
+# Not merely "foobara": Foobara::CommandConnector exists after foobara/all but
+# WITHOUT its methods, so subclassing it succeeds and fails later at `connect`
+# with "no superclass method". Requiring the project explicitly is the
+# difference between an error at load and a confusing one at runtime.
+require "foobara/command_connectors"
+
+require_relative "error"
+
+module Foobara
+  module AWS
+    class SQSConnector < Foobara::CommandConnector
+      # Enqueues rather than executes. Generated per connected command by
+      # `connect`, the way ResqueConnector generates its own.
+      class EnqueueCommand < Foobara::Command
+        class << self
+          attr_accessor :target_command_class, :sqs_connector
+        end
+
+        def execute
+          self.class.sqs_connector.enqueue(
+            self.class.target_command_class.full_command_name, raw_inputs
+          )
+        end
+      end
+
+      attr_reader :queue_url
+
+      def initialize(*, queue_url: nil, sqs: nil, **, &)
+        @queue_url = queue_url
+        @sqs = sqs
+
+        super(*, **, &)
+      end
+
+      def connect(connectable, *, **, &)
+        exposed = Util.array(super)
+
+        exposed.each { |exposed_command| build_async_command(exposed_command) }
+
+        exposed
+      end
+
+      # Puts the envelope on the queue, or runs the command inline when there is
+      # no queue configured.
+      def enqueue(command_name, inputs)
+        # Checked in both paths: a message naming a command nothing consumes is
+        # a job that can only fail, and it should fail here rather than in a
+        # worker after a delivery delay.
+        command_class = transformed_command_from_name(command_name)
+        raise Error, "command #{command_name.inspect} is not connected" unless command_class
+
+        envelope = { EventHandler::COMMAND_KEY => command_name }
+        envelope[EventHandler::INPUTS_KEY] = inputs unless inputs.nil? || inputs.empty?
+
+        return command_class.new(inputs || {}).run! unless queue_url
+
+        sqs.send_message(queue_url: queue_url, message_body: JSON.dump(envelope))
+        envelope
+      end
+
+      private
+
+      def sqs
+        @sqs ||= begin
+          require "aws-sdk-sqs"
+          Aws::SQS::Client.new
+        rescue LoadError
+          raise Error,
+                "SQSConnector needs the `aws-sdk-sqs` gem to enqueue, which foobara-aws does not depend on. " \
+                "Add it to the unit that enqueues, or pass sqs: your own client."
+        end
+      end
+
+      def build_async_command(exposed_command)
+        command_class = exposed_command.command_class
+        async = Util.make_class("#{command_class.name}Async", EnqueueCommand)
+
+        # The real command's input type, so an invalid job is refused at the
+        # point of enqueuing rather than by a worker minutes later.
+        inputs_type = exposed_command.transformed_command_class.inputs_type
+        async.inputs(inputs_type) if inputs_type
+
+        async.sqs_connector = self
+        async.target_command_class = exposed_command
+        async
+      end
+    end
+  end
+end
+
+require_relative "event_handler"

data/lib/foobara/aws/version.rb

@@ -2,6 +2,6 @@
 
 module Foobara
   module AWS
-    VERSION = "0.3.0"
+    VERSION = "0.4.0"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: foobara-aws
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Omar Qureshi
@@ -28,10 +28,12 @@ files:
 - lib/foobara/aws/cdk/service.rb
 - lib/foobara/aws/check.rb
 - lib/foobara/aws/error.rb
+- lib/foobara/aws/event_handler.rb
 - lib/foobara/aws/handler.rb
 - lib/foobara/aws/lambda.rb
 - lib/foobara/aws/packager.rb
 - lib/foobara/aws/plan.rb
+- lib/foobara/aws/sqs_connector.rb
 - lib/foobara/aws/version.rb
 homepage: https://github.com/omarqureshi/foobara-aws
 licenses: