logstash-input-aws-s3-sns-sqs 1.1.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d021893574e85e66adfaf76e5204ad66420e7143
+  data.tar.gz: 4ef9767ee60fe866c9cb7b3537e0e2fec8a5f7ef
+SHA512:
+  metadata.gz: 40179397b04cdcd863485224e801c83e64b2038f5e898ab6518706cfc53c155b2a80c814f461ad9c4bc9651f6e0f0f5c027ef4b1f5fccbce57f12ea644e7e448
+  data.tar.gz: c115323d783a5cdd8c77861ed63a6bb77b4ac3d29ccc9a7dd4845b3ca53766810d1aaef67724d357f202cc532173ea4d28965c2b05e995013645141a82fd90ee

data/CHANGELOG.md

@@ -0,0 +1,22 @@
+## 1.1.2
+- Fix a Bug in the S3 Key generation
+- Enable shipping throug SNS Topic (needs another toJSON)
+## 1.1.1
+- Added the ability to remove objects from S3 after processing.
+- Workaround an issue with the Ruby autoload that causes "uninitialized constant `Aws::Client::Errors`" errors.
+
+## 1.1.0
+- Logstash 5 compatibility
+
+## 1.0.3
+- added some metadata to the event (bucket and object name as commited by joshuaspence)
+- also try to unzip files ending with ".gz" (ALB logs are zipped but not marked with proper Content-Encoding)
+
+## 1.0.2
+- fix for broken UTF-8 (so we won't lose a whole s3 log file because of a single invalid line, ruby's split will die on those)
+
+## 1.0.1
+- same (because of screwed up rubygems.org release)
+
+## 1.0.0
+- Initial Release

data/CONTRIBUTORS

@@ -0,0 +1,13 @@
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped logstash along its way.
+
+Contributors:
+* cherweg(this fork + some bugfixes)
+* joshuaspence (event metadata)
+* Heiko-san (initial contributor)
+* logstash-input-sqs plugin as code base
+
+Note: If you've sent us patches, bug reports, or otherwise contributed to
+Logstash, and you aren't on the list above and want to be, please let us know
+and we'll make sure you're here. Contributions from folks like you are what make
+open source awesome.

data/Gemfile

@@ -0,0 +1,2 @@
+source 'https://rubygems.org'
+gemspec

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2012–2015 Elasticsearch <http://www.elastic.co>
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/NOTICE.TXT

@@ -0,0 +1,5 @@
+Elasticsearch
+Copyright 2012-2015 Elasticsearch
+
+This product includes software developed by The Apache Software
+Foundation (http://www.apache.org/).

data/README.md

@@ -0,0 +1,147 @@
+# Logstash Plugin
+
+This is a plugin for [Logstash](https://github.com/elastic/logstash).
+
+It is fully free and fully open source. The license is Apache 2.0.
+
+## Documentation
+
+ Get logs from AWS s3 buckets as issued by an object-created event via sqs.
+
+ This plugin is based on the logstash-input-sqs plugin but doesn't log the sqs event itself.
+ Instead it assumes, that the event is an s3 object-created event and will then download
+ and process the given file.
+
+ Some issues of logstash-input-sqs, like logstash not shutting down properly, have been
+ fixed for this plugin.
+
+ In contrast to logstash-input-sqs this plugin uses the "Receive Message Wait Time"
+ configured for the sqs queue in question, a good value will be something like 10 seconds
+ to ensure a reasonable shutdown time of logstash.
+ Also use a "Default Visibility Timeout" that is high enough for log files to be downloaded
+ and processed (I think a good value should be 5-10 minutes for most use cases), the plugin will
+ avoid removing the event from the queue if the associated log file couldn't be correctly
+ passed to the processing level of logstash (e.g. downloaded content size doesn't match sqs event).
+
+ This plugin is meant for high availability setups, in contrast to logstash-input-s3 you can safely
+ use multiple logstash nodes, since the usage of sqs will ensure that each logfile is processed
+ only once and no file will get lost on node failure or downscaling for auto-scaling groups.
+ (You should use a "Message Retention Period" >= 4 days for your sqs to ensure you can survive
+ a weekend of faulty log file processing)
+ The plugin will not delete objects from s3 buckets, so make sure to have a reasonable "Lifecycle"
+ configured for your buckets, which should keep the files at least "Message Retention Period" days.
+
+ A typical setup will contain some s3 buckets containing elb, cloudtrail or other log files.
+ These will be configured to send object-created events to a sqs queue, which will be configured
+ as the source queue for this plugin.
+ (The plugin supports gzipped content if it is marked with "contend-encoding: gzip" as it is the
+ case for cloudtrail logs)
+
+ The logstash node therefore must have sqs permissions + the permissions to download objects
+ from the s3 buckets that send events to the queue.
+ (If logstash nodes are running on EC2 you should use a ServerRole to provide permissions)
+ [source,json]
+   {
+       "Version": "2012-10-17",
+       "Statement": [
+           {
+               "Effect": "Allow",
+               "Action": [
+                   "sqs:Get*",
+                   "sqs:List*",
+                   "sqs:ReceiveMessage",
+                   "sqs:ChangeMessageVisibility*",
+                   "sqs:DeleteMessage*"
+               ],
+               "Resource": [
+                   "arn:aws:sqs:us-east-1:123456789012:my-elb-log-queue"
+               ]
+           },
+           {
+               "Effect": "Allow",
+               "Action": [
+                   "s3:Get*",
+                   "s3:List*",
+                   "s3:DeleteObject"
+               ],
+               "Resource": [
+                   "arn:aws:s3:::my-elb-logs",
+                   "arn:aws:s3:::my-elb-logs/*"
+               ]
+           }
+       ]
+   }
+
+## Need Help?
+
+Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
+
+## Developing
+
+### 1. Plugin Developement and Testing
+
+#### Code
+- To get started, you'll need JRuby with the Bundler gem installed.
+
+- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
+
+- Install dependencies
+```sh
+bundle install
+```
+
+#### Test
+
+- Update your dependencies
+
+```sh
+bundle install
+```
+
+- Run tests
+
+```sh
+bundle exec rspec
+```
+
+### 2. Running your unpublished Plugin in Logstash
+
+#### 2.1 Run in a local Logstash clone
+
+- Edit Logstash `Gemfile` and add the local plugin path, for example:
+```ruby
+gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
+```
+- Install plugin
+```sh
+bin/plugin install --no-verify
+```
+- Run Logstash with your plugin
+```sh
+bin/logstash -e 'filter {awesome {}}'
+```
+At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
+
+#### 2.2 Run in an installed Logstash
+
+You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
+
+- Build your plugin gem
+```sh
+gem build logstash-filter-awesome.gemspec
+```
+- Install the plugin from the Logstash home
+```sh
+bin/plugin install /your/local/plugin/logstash-filter-awesome.gem
+```
+- Start Logstash and proceed to test the plugin
+
+## Contributing
+
+All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
+
+Programming is not a required skill. Whatever you've seen about open source and maintainers or community members  saying "send patches or die" - you will not see that here.
+
+It is more important to the community that you are able to contribute.
+
+For more information about contributing, see the [CONTRIBUTING](https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md) file.

data/lib/logstash/inputs/s3sqs.rb

@@ -0,0 +1,244 @@
+# encoding: utf-8
+#
+require "logstash/inputs/threadable"
+require "logstash/namespace"
+require "logstash/timestamp"
+require "logstash/plugin_mixins/aws_config"
+require "logstash/errors"
+require 'logstash/inputs/s3sqs/patch'
+
+Aws.eager_autoload!
+
+# Get logs from AWS s3 buckets as issued by an object-created event via sqs.
+#
+# This plugin is based on the logstash-input-sqs plugin but doesn't log the sqs event itself.
+# Instead it assumes, that the event is an s3 object-created event and will then download
+# and process the given file.
+#
+# Some issues of logstash-input-sqs, like logstash not shutting down properly, have been
+# fixed for this plugin.
+#
+# In contrast to logstash-input-sqs this plugin uses the "Receive Message Wait Time"
+# configured for the sqs queue in question, a good value will be something like 10 seconds
+# to ensure a reasonable shutdown time of logstash.
+# Also use a "Default Visibility Timeout" that is high enough for log files to be downloaded
+# and processed (I think a good value should be 5-10 minutes for most use cases), the plugin will
+# avoid removing the event from the queue if the associated log file couldn't be correctly
+# passed to the processing level of logstash (e.g. downloaded content size doesn't match sqs event).
+#
+# This plugin is meant for high availability setups, in contrast to logstash-input-s3 you can safely
+# use multiple logstash nodes, since the usage of sqs will ensure that each logfile is processed
+# only once and no file will get lost on node failure or downscaling for auto-scaling groups.
+# (You should use a "Message Retention Period" >= 4 days for your sqs to ensure you can survive
+# a weekend of faulty log file processing)
+# The plugin will not delete objects from s3 buckets, so make sure to have a reasonable "Lifecycle"
+# configured for your buckets, which should keep the files at least "Message Retention Period" days.
+#
+# A typical setup will contain some s3 buckets containing elb, cloudtrail or other log files.
+# These will be configured to send object-created events to a sqs queue, which will be configured
+# as the source queue for this plugin.
+# (The plugin supports gzipped content if it is marked with "contend-encoding: gzip" as it is the
+# case for cloudtrail logs)
+#
+# The logstash node therefore must have sqs permissions + the permissions to download objects
+# from the s3 buckets that send events to the queue.
+# (If logstash nodes are running on EC2 you should use a ServerRole to provide permissions)
+# [source,json]
+#   {
+#       "Version": "2012-10-17",
+#       "Statement": [
+#           {
+#               "Effect": "Allow",
+#               "Action": [
+#                   "sqs:Get*",
+#                   "sqs:List*",
+#                   "sqs:ReceiveMessage",
+#                   "sqs:ChangeMessageVisibility*",
+#                   "sqs:DeleteMessage*"
+#               ],
+#               "Resource": [
+#                   "arn:aws:sqs:us-east-1:123456789012:my-elb-log-queue"
+#               ]
+#           },
+#           {
+#               "Effect": "Allow",
+#               "Action": [
+#                   "s3:Get*",
+#                   "s3:List*",
+#                   "s3:DeleteObject"
+#               ],
+#               "Resource": [
+#                   "arn:aws:s3:::my-elb-logs",
+#                   "arn:aws:s3:::my-elb-logs/*"
+#               ]
+#           }
+#       ]
+#   }
+#
+class LogStash::Inputs::S3SQS < LogStash::Inputs::Threadable
+  include LogStash::PluginMixins::AwsConfig::V2
+
+  BACKOFF_SLEEP_TIME = 1
+  BACKOFF_FACTOR = 2
+  MAX_TIME_BEFORE_GIVING_UP = 60
+  EVENT_SOURCE = 'aws:s3'
+  EVENT_TYPE = 'ObjectCreated'
+
+  config_name "s3sqs"
+
+  default :codec, "plain"
+
+  # Name of the SQS Queue to pull messages from. Note that this is just the name of the queue, not the URL or ARN.
+  config :queue, :validate => :string, :required => true
+
+  # Whether to delete files from S3 after processing.
+  config :delete_on_success, :validate => :boolean, :default => false
+
+  attr_reader :poller
+  attr_reader :s3
+
+  def register
+    require "aws-sdk"
+    require 'cgi'
+    @logger.info("Registering SQS input", :queue => @queue)
+    setup_queue
+  end
+
+  def setup_queue
+    aws_sqs_client = Aws::SQS::Client.new(aws_options_hash)
+    queue_url = aws_sqs_client.get_queue_url(:queue_name =>  @queue)[:queue_url]
+    @poller = Aws::SQS::QueuePoller.new(queue_url, :client => aws_sqs_client)
+    @s3 = Aws::S3::Client.new(aws_options_hash)
+  rescue Aws::SQS::Errors::ServiceError => e
+    @logger.error("Cannot establish connection to Amazon SQS", :error => e)
+    raise LogStash::ConfigurationError, "Verify the SQS queue name and your credentials"
+  end
+
+  def polling_options
+    {
+      # we will query 1 message at a time, so we can ensure correct error handling if we can't download a single file correctly
+      # (we will throw :skip_delete if download size isn't correct to process the event again later
+      # -> set a reasonable "Default Visibility Timeout" for your queue, so that there's enough time to process the log files)
+      :max_number_of_messages => 1,
+      # we will use the queue's setting, a good value is 10 seconds
+      # (to ensure fast logstash shutdown on the one hand and few api calls on the other hand)
+      :wait_time_seconds => nil,
+    }
+  end
+
+  def handle_message(message, queue)
+    hash = JSON.parse message.body
+    @logger.debug("handle_message", :hash => hash, :message => message)
+    # there may be test events sent from the s3 bucket which won't contain a Records array,
+    # we will skip those events and remove them from queue
+    if hash['Message'] then
+      # typically there will be only 1 record per event, but since it is an array we will
+      # treat it as if there could be more records
+      JSON.parse(hash['Message'])['Records'].each do |record|
+        @logger.info("We found a record", :record => record)
+	# in case there are any events with Records that aren't s3 object-created events and can't therefore be
+        # processed by this plugin, we will skip them and remove them from queue
+        if record['eventSource'] == EVENT_SOURCE and record['eventName'].start_with?(EVENT_TYPE) then
+          @logger.debug("It is a valid record")
+	  bucket = CGI.unescape(record['s3']['bucket']['name'])
+	  key    = CGI.unescape(record['s3']['object']['key'])
+
+          # try download and :skip_delete if it fails
+          begin
+            response = @s3.get_object(
+              bucket: bucket,
+              key: key,
+            )
+          rescue => e
+            @logger.warn("issuing :skip_delete on failed download", :bucket => bucket, :object => key, :error => e)
+            throw :skip_delete
+          end
+
+          # verify downloaded content size
+          if response.content_length == record['s3']['object']['size'] then
+            body = response.body
+            # if necessary unzip
+            if response.content_encoding == "gzip" or record['s3']['object']['key'].end_with?(".gz") then
+              @logger.debug("Ohhh i´ll try to unzip")
+	      begin
+		            temp = Zlib::GzipReader.new(body)
+              rescue => e
+                @logger.warn("content is marked to be gzipped but can't unzip it, assuming plain text", :bucket => bucket, :object => key, :error => e)
+                temp = body
+              end
+              body = temp
+            end
+            # process the plain text content
+            begin
+              lines = body.read.encode('UTF-8', 'binary', invalid: :replace, undef: :replace, replace: "\u2370").split(/\n/)
+              lines.each do |line|
+                @logger.debug("Decorating the event")
+		  @codec.decode(line) do |event|
+                  decorate(event)
+
+                  event.set('[@metadata][s3_bucket_name]', record['s3']['bucket']['name'])
+                  event.set('[@metadata][s3_object_key]', record['s3']['object']['key'])
+
+                  queue << event
+                end
+              end
+            rescue => e
+              @logger.warn("issuing :skip_delete on failed plain text processing", :bucket => bucket, :object => key, :error => e)
+              throw :skip_delete
+            end
+
+            # Delete the files from S3
+            begin
+              @s3.delete_object(bucket: bucket, key: key) if @delete_on_success
+            rescue => e
+              @logger.warn("Failed to delete S3 object", :bucket => bucket, :object => key, :error => e)
+            end
+          # otherwise try again later
+          else
+            @logger.warn("issuing :skip_delete on wrong download content size", :bucket => bucket, :object => key,
+              :download_size => response.content_length, :expected => record['s3']['object']['size'])
+            throw :skip_delete
+          end
+        end
+      end
+    end
+  end
+
+  def run(queue)
+    # ensure we can stop logstash correctly
+    poller.before_request do |stats|
+      if stop? then
+        @logger.warn("issuing :stop_polling on stop?", :queue => @queue)
+        # this can take up to "Receive Message Wait Time" (of the sqs queue) seconds to be recognized
+        throw :stop_polling
+      end
+    end
+    # poll a message and process it
+    run_with_backoff do
+      poller.poll(polling_options) do |message|
+        handle_message(message, queue)
+      end
+    end
+  end
+
+  private
+  # Runs an AWS request inside a Ruby block with an exponential backoff in case
+  # we experience a ServiceError.
+  #
+  # @param [Integer] max_time maximum amount of time to sleep before giving up.
+  # @param [Integer] sleep_time the initial amount of time to sleep before retrying.
+  # @param [Block] block Ruby code block to execute.
+  def run_with_backoff(max_time = MAX_TIME_BEFORE_GIVING_UP, sleep_time = BACKOFF_SLEEP_TIME, &block)
+    next_sleep = sleep_time
+    begin
+      block.call
+      next_sleep = sleep_time
+    rescue Aws::SQS::Errors::ServiceError => e
+      @logger.warn("Aws::SQS::Errors::ServiceError ... retrying SQS request with exponential backoff", :queue => @queue, :sleep_time => sleep_time, :error => e)
+      sleep(next_sleep)
+      next_sleep =  next_sleep > max_time ? sleep_time : sleep_time * BACKOFF_FACTOR
+      retry
+    end
+  end
+
+end # class

data/lib/logstash/inputs/s3sqs/patch.rb

@@ -0,0 +1,22 @@
+# This patch was stolen from logstash-plugins/logstash-output-s3#102.
+#
+# This patch is a workaround for a JRuby issue which has been fixed in JRuby
+# 9000, but not in JRuby 1.7. See https://github.com/jruby/jruby/issues/3645
+# and https://github.com/jruby/jruby/issues/3920. This is necessary because the
+# `aws-sdk` is doing tricky name discovery to generate the correct error class.
+#
+# As per https://github.com/aws/aws-sdk-ruby/issues/1301#issuecomment-261115960,
+# this patch may be short-lived anyway.
+require 'aws-sdk'
+
+begin
+  old_stderr = $stderr
+  $stderr = StringIO.new
+
+  module Aws
+    const_set(:S3, Aws::S3)
+    const_set(:SQS, Aws::SQS)
+  end
+ensure
+  $stderr = old_stderr
+end

data/logstash-input-aws-s3-sns-sqs.gemspec

@@ -0,0 +1,29 @@
+Gem::Specification.new do |s|
+  s.name            = 'logstash-input-aws-s3-sns-sqs'
+  s.version         = '1.1.2'
+  s.licenses        = ['Apache License (2.0)']
+  s.summary         = "Get logs from AWS s3 buckets as issued by an object-created event via sns -> sqs."
+  s.description     = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
+  s.authors         = ["Christian Herweg"]
+  s.email           = 'christian.herweg@gmail.com'
+  s.homepage        = "https://github.com/cherweg/logstash-input-s3-sns-sqs"
+  s.require_paths = ["lib"]
+
+  # Files
+  s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
+
+  # Tests
+  s.test_files = s.files.grep(%r{^(test|spec|features)/})
+
+  # Special flag to let us know this is actually a logstash plugin
+  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "input" }
+
+  # Gem dependencies
+  s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
+
+  s.add_runtime_dependency 'logstash-codec-json'
+  s.add_runtime_dependency "logstash-mixin-aws", ">= 1.0.0"
+
+  s.add_development_dependency 'logstash-devutils'
+end
+

data/spec/inputs/s3sqs_spec.rb

@@ -0,0 +1,9 @@
+# encoding: utf-8
+require "logstash/devutils/rspec/spec_helper"
+require "logstash/inputs/s3sqs"
+
+describe LogStash::Inputs::S3SQS do
+
+  true.should be_true
+
+end

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+# encoding: utf-8
+require "logstash/devutils/rspec/spec_helper"

metadata

@@ -0,0 +1,120 @@
+--- !ruby/object:Gem::Specification
+name: logstash-input-aws-s3-sns-sqs
+version: !ruby/object:Gem::Version
+  version: 1.1.2
+platform: ruby
+authors:
+- Christian Herweg
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2017-11-10 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '1.60'
+    - - <=
+      - !ruby/object:Gem::Version
+        version: '2.99'
+  name: logstash-core-plugin-api
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '1.60'
+    - - <=
+      - !ruby/object:Gem::Version
+        version: '2.99'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: logstash-codec-json
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+  name: logstash-mixin-aws
+  prerelease: false
+  type: :runtime
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.0
+- !ruby/object:Gem::Dependency
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  name: logstash-devutils
+  prerelease: false
+  type: :development
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program
+email: christian.herweg@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- CONTRIBUTORS
+- Gemfile
+- LICENSE
+- NOTICE.TXT
+- README.md
+- lib/logstash/inputs/s3sqs.rb
+- lib/logstash/inputs/s3sqs/patch.rb
+- logstash-input-aws-s3-sns-sqs.gemspec
+- spec/inputs/s3sqs_spec.rb
+- spec/spec_helper.rb
+homepage: https://github.com/cherweg/logstash-input-s3-sns-sqs
+licenses:
+- Apache License (2.0)
+metadata:
+  logstash_plugin: 'true'
+  logstash_group: input
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project:
+rubygems_version: 2.4.5
+signing_key:
+specification_version: 4
+summary: Get logs from AWS s3 buckets as issued by an object-created event via sns -> sqs.
+test_files:
+- spec/inputs/s3sqs_spec.rb
+- spec/spec_helper.rb