lambdakiq 1.0.3 → 2.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3f6c1f6c45eb8d1a8a5188e4e70e63ed36b05bbf334a807cef72c523d6661b55
-  data.tar.gz: 4a9683cd82caadeedbbba483686c9584a0c0591bf85e63198202b4af0b414bf7
+  metadata.gz: 907b219d7a132815c0f656a7e69296aba6556b15d1c8f031afcdebebca42bb96
+  data.tar.gz: cd7480cb9e54c0037674d7335a669d443954f8812686beeee7dcaa9e17efb43e
 SHA512:
-  metadata.gz: 812d898f39463401a4dc348213b75b40d1892618a575f58791edfd7d168304ead00bf562831d414533304ccb83e00e826e0a8c43653c19432c9a190100e1ca15
-  data.tar.gz: 144f18512e4db39eae43f6299fa81e48750e0438540ccd03a3e831a5efd914639cc20fec86ebb820ec5aeff1c6cb4ce5a4a2396b34d495f1f3788b486d55aa48
+  metadata.gz: 33766b628c71b3c6803b9884cf44a1ec18655b04d1bccd3a8641c37bde5c29b8464b47822b6e10be6d67739e71abd53d9891c5cefbefd53988418eed39b66be6
+  data.tar.gz: e69cc722782cfb724c8265dce20c2c57db42cecf54747620dae54ee3856d203dbd51ccc13a27d52705f9dfc52e134501f6192be69ce6bb26c508332966f4110a

data/CHANGELOG.md

@@ -2,11 +2,23 @@
 
 See this http://keepachangelog.com link for information on how we want this documented formatted.
 
-## v1.0.2, v1.0.3
+## v2.0.1
 
 #### Fixed
 
-- Ensure `active_job` events are safe for non-Lambdakiq jobs.
+- Rails 5.1 oddities for class attribute default not set.
+
+## v2.0.0
+
+#### Changed
+
+- Leverage new `ReportBatchItemFailures` feature of SQS.
+
+## v1.0.2, v1.0.3, v1.0.4
+
+#### Fixed
+
+- Rails v5.2 compatibility. Metrics logging is are safe for non-Lambdakiq jobs
 
 ## v1.0.0
 

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    lambdakiq (1.0.3)
+    lambdakiq (2.0.1)
       activejob
       aws-sdk-sqs
       concurrent-ruby

data/README.md

@@ -30,7 +30,7 @@ Add the Lambdakiq gem to your `Gemfile`.
 gem 'lambdakiq'
 ```
 
-Open `config/initializers/production.rb` and set Lambdakiq as your ActiveJob queue adapter.
+Open `config/environments/production.rb` and set Lambdakiq as your ActiveJob queue adapter.
 
 ```ruby
 config.active_job.queue_adapter = :lambdakiq
@@ -45,6 +45,17 @@ class ApplicationJob < ActiveJob::Base
 end
 ```
 
+Using ActionMailer's built-in deliver job with ActiveJob? Make sure to include the Lambdakiq worker and set the queue name depending on your Rails version. You can do this in a newly created `config/initializers//action_mailer.rb` or another initializer of your choice.
+
+```ruby
+# Rails 5.x
+ActionMailer::DeliveryJob.include Lambdakiq::Worker
+ActionMailer::DeliveryJob.queue_as ENV['JOBS_QUEUE_NAME']
+# Rails 6.x
+ActionMailer::MailDeliveryJob.include Lambdakiq::Worker
+ActionMailer::MailDeliveryJob.queue_as ENV['JOBS_QUEUE_NAME']
+```
+
 The same Docker image will be used for both your `web` and `jobs` functions (example setup in following sections) which means the same `app.rb` handler would be used. The [Lamby](https://lamby.custominktech.com) gem automatically detects if Lambdakiq is being used so the following handler works as is.
 
 ```ruby
@@ -133,6 +144,8 @@ JobsLambda:
         Properties:
           Queue: !GetAtt JobsQueue.Arn
           BatchSize: 1
+          FunctionResponseTypes:
+            - ReportBatchItemFailures
     MemorySize: 1792
     PackageType: Image
     Policies:
@@ -149,9 +162,9 @@ JobsLambda:
 Here are some key aspects of our `JobsLambda` resource above:
 
 - The `Events` property uses the [SQS Type](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-sqs.html).
-- Our [BatchSize](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-sqs.html#sam-function-sqs-batchsize) is set to one so we can handle retrys more easily without worrying about idempotency in larger batches.
+- The [BatchSize](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-sqs.html#sam-function-sqs-batchsize) can be any number you like. Less means more Lambda concurrency, more means some jobs could take longer. The jobs function `Timeout` must be lower than the `JobsQueue`'s `VisibilityTimeout` property. When the batch size is one, the queue's visibility is generally one second more.
+- You must use `ReportBatchItemFailures` response types. Lambdakiq assumes we are [reporting batch item failures](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting). This is a new feature of SQS introduced in [November 2021](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting).
 - The `Metadata`'s Docker properties must be the same as our web function except for the `DockerTag`. This is needed for the image to be shared. This works around a known [SAM issue](https://github.com/aws/aws-sam-cli/issues/2466) vs using the `ImageConfig` property.
-- The jobs function `Timeout` must be lower than the `JobsQueue`'s `VisibilityTimeout` property. When the batch size is one, the queue's visibility is generally one second more.
 
 🎉 Deploy your application and have fun with ActiveJob on SQS & Lambda.
 
@@ -181,12 +194,6 @@ end
 
 - `retry` - Overrides the default Lambdakiq `max_retries` for this one job.
 
-## Concurrency & Limits
-
-AWS SQS is highly scalable with [few limits](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-quotas.html). As your jobs in SQS increases so should your concurrent functions to process that work. However, as this article, ["Why isn't my Lambda function with an Amazon SQS event source scaling optimally?"](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-sqs-scaling/) describes it is possible that errors will effect your concurrency.
-
-To help keep your queue and workers scalable, reduce the errors raised by your jobs. You an also reduce the retry count.
-
 ## Observability with CloudWatch
 
 Get ready to gain way more insights into your ActiveJobs using AWS' [CloudWatch](https://aws.amazon.com/cloudwatch/) service. Every AWS service, including SQS & Lambda, publishes detailed [CloudWatch Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html). This gem leverages [CloudWatch Embedded Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html) to add detailed ActiveJob metrics to that system. You can mix and match these data points to build your own [CloudWatch Dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html). If needed, any combination can be used to trigger [CloudWatch Alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html). Much like Sumo Logic, you can search & query for data using [CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html).

data/lib/lambdakiq/error.rb

@@ -2,16 +2,6 @@ module Lambdakiq
   class Error < StandardError
   end
 
-  class JobError < Error
-    attr_reader :original_exception, :job
-
-    def initialize(error)
-      @original_exception = error
-      super(error.message)
-      set_backtrace Rails.backtrace_cleaner.clean(error.backtrace)
-    end
-  end
-
   class FifoDelayError < Error
     def initialize(error)
       super

data/lib/lambdakiq/job.rb

@@ -9,9 +9,9 @@ module Lambdakiq
         records = Event.records(event)
         jobs = records.map { |record| new(record) }
         jobs.each(&:perform)
-        jwerror = jobs.detect{ |j| j.error }
-        return unless jwerror
-        raise JobError.new(jwerror.error)
+        failed_jobs = jobs.select { |j| j.error }
+        item_failures = failed_jobs.map { |j| { itemIdentifier: j.provider_job_id } }
+        { batchItemFailures: item_failures }
       end
 
     end
@@ -40,10 +40,14 @@ module Lambdakiq
       active_job.executions
     end
 
+    def provider_job_id
+      active_job.provider_job_id
+    end
+
     def perform
       if fifo_delay?
         fifo_delay
-        raise FifoDelayError, active_job.job_id
+        return
       end
       execute
     end
@@ -104,6 +108,7 @@ module Lambdakiq
     end
 
     def fifo_delay
+      @error = FifoDelayError.new(active_job.job_id)
       params = client_params.merge visibility_timeout: record.fifo_delay_visibility_timeout
       client.change_message_visibility(params)
     end

data/lib/lambdakiq/metrics.rb

@@ -30,8 +30,14 @@ module Lambdakiq
       job.class.name
     end
 
+    def adapter_name
+      event.payload[:adapter].class.name
+    end
+
     def lambdakiq?
-      job.respond_to?(:lambdakiq?) && job.lambdakiq?
+      adapter_name.include?('Lambdakiq') && 
+        job.respond_to?(:lambdakiq?) && 
+        job.lambdakiq?
     end
 
     def logger
@@ -65,7 +71,7 @@ module Lambdakiq
       set_property 'QueueName', job.queue_name
       set_property 'MessageId', job.provider_job_id if job.provider_job_id
       set_property 'ExceptionName', exception_name if exception_name
-      set_property 'EnqueuedAt', job.enqueued_at if job.enqueued_at
+      set_property 'EnqueuedAt', job.enqueued_at if job.respond_to?(:enqueued_at) && job.enqueued_at
       set_property 'Executions', job.executions if job.executions
       job.arguments.each_with_index do |argument, index|
         set_property "JobArg#{index+1}", argument
@@ -113,4 +119,3 @@ module Lambdakiq
 
   end
 end
-

data/lib/lambdakiq/version.rb

@@ -1,3 +1,3 @@
 module Lambdakiq
-  VERSION = '1.0.3'
+  VERSION = '2.0.1'
 end

data/lib/lambdakiq/worker.rb

@@ -4,8 +4,8 @@ module Lambdakiq
 
     included do
       class_attribute :lambdakiq_options_hash,
-                      instance_predicate: false,
-                      default: Hash.new
+                      instance_predicate: false
+      self.lambdakiq_options_hash = Hash.new
     end
 
     class_methods do

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: lambdakiq
 version: !ruby/object:Gem::Version
-  version: 1.0.3
+  version: 2.0.1
 platform: ruby
 authors:
 - Ken Collins
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-10-04 00:00:00.000000000 Z
+date: 2022-08-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob