fluent-plugin-cloudfront-log-optimized 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 41c311f36c1f68872f1ad935ea398c3f21b0eb2154ed2ded25934f2e70966ab6
+  data.tar.gz: 0a6997034d17b4b5c02c66ac3dd8017cc842b40b3025bdcbb18c68719641536c
+SHA512:
+  metadata.gz: 87aec8f7754aefe0a81b47a7beb255d6b1598213a73623117cad56206e5f7b263ac3118963db64e65b1906f947ab448fe2d3e70448d14a74f5aeb6e155b8fd13
+  data.tar.gz: f77b92a998259fbf4b3c778b4032c87e3860b37355e10631cc01dce3c099f89bf094cab113c7e9f62875d8701b1910c17b289236addd726245eed3b6c7c7c905

data/.gitignore

@@ -0,0 +1,42 @@
+# Created by https://www.gitignore.io/api/ruby
+
+### Ruby ###
+.bin
+*.gems
+*.gem
+*.rbc
+/.config
+/coverage/
+/InstalledFiles
+/pkg/
+/spec/reports/
+/spec/examples.txt
+/test/tmp/
+/test/version_tmp/
+/tmp/
+
+## Specific to RubyMotion:
+.dat*
+.repl_history
+build/
+
+## Documentation cache and generated files:
+/.yardoc/
+/_yardoc/
+/doc/
+/rdoc/
+
+## Environment normalization:
+/.bundle/
+/vendor/bundle
+/lib/bundler/man/
+
+# for a library or gem, you might want to ignore these files since the code is
+# intended to run in multiple environments; otherwise, check them in:
+Gemfile.lock
+.ruby-version
+.ruby-gemset
+
+# unless supporting rvm < 1.11.0 or doing something fancy, ignore this:
+.rvmrc
+

data/CHANGELOG.md

@@ -0,0 +1,23 @@
+## Release 0.2.0 (kjwierenga)
+Merge branch 'feature/enh/rename-to-optimized' into develop
+Merge branch 'feature/fix/process-log-files-in-last-modified-order' into develop
+- [fix] process log files in order of ascending last_modified time
+Merge branch 'feature/enh/reduce-memory-usage' into develop
+- [enh] process log files with (small) constant memory usage even for very large log files
+- [add] EnumerableInflater so we can inflate the .gz file in a streaming fashion (greatly reducing memory usage)
+- [enh] transpose.to_h is faster than Hash[....zip(line)]
+Merge branch 'feature/fix/unescape-once' into develop
+- [enh] URI.unescape is deprecated and CGI.unescape is faster
+- [fix] unescape once; CloudFront encodes once so we should unescape once
+Merge branch 'feature/enh/optional-parse-date-time' into develop
+- [enh] use Time.iso8601 which is much faster than the Time.parse
+- [add] option to make parsing of date time optional
+Merge branch 'feature/enh/modernize-tests' into develop
+Merge branch 'feature/fix/verbose-param' into develop
+- [fix] verbose flag; it was set to string causing it to be always enabled
+
+## Release 0.1.1 (packetloop)
+- Packetloop 2019 version
+
+## Release 0.0.5 (kubihie)
+- Original 2016 version

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in fluent-plugin-cloudfront-log-optimized.gemspec
+gemspec

data/README.md

@@ -0,0 +1,101 @@
+# Fluent::Plugin::Cloudfront::Log
+This plugin will connect to the S3 bucket that you store your cloudfront logs in. Once the plugin processes them and ships them to FluentD, it moves them to another location (either another bucket or sub directory).
+
+## Lineage
+This is a fork of [packetloop's v0.14 fix](https://github.com/packetloop/fluent-plugin-cloudfront-log-v0.14-fix)
+which is a fork of the original [kubihie version](https://github.com/kubihie/fluent-plugin-cloudfront-log)
+with contributions from [lenfree's version](https://github.com/lenfree/fluent-plugin-cloudfront-log).
+This fork has optimizations to process hundreds of large CloudFront log files (tens of MB)
+efficiently and with constant memory usage.
+
+I will publish this gem so it can be used in production assuming upstream
+repositories are unmaintained. I would be happy to merge these changes back into [kubihie's version](https://github.com/kubihie/fluent-plugin-cloudfront-log).
+## Example config
+```
+<source>
+@type       cloudfront_log
+log_bucket  cloudfront-logs
+log_prefix  production
+region      us-east-1
+interval    300
+aws_key_id  xxxxxx
+aws_sec_key xxxxxx
+tag         reverb.cloudfront
+verbose     true
+</source>
+```
+
+## Configuration options
+
+#### log_bucket
+This option tells the plugin where to look for the cloudfront logs
+
+#### log_prefix
+For example if your logs are stored in a folder called "production" under the "cloudfront-logs" bucket, your logs would be stored in cloudfront like "cloudfront-logs/production/log.gz".
+In this case, you'd want to use the prefix "production".
+
+#### moved_log_bucket
+Here you can specify where you'd like the log files to be moved after processing. If left blank this defaults to a folder called `_moved` under the bucket configured for `@log_bucket`.
+
+#### moved_log_prefix
+This specifices what the log files will be named once they're processed. This defaults to `_moved`.
+
+#### region
+The region where your cloudfront logs are stored.
+
+#### interval
+This is the rate in seconds at which we check the bucket for updated logs. This defaults to 300.
+#### aws_sec_id
+The ID of your AWS keypair. Note: Since this plugin uses aws-sdk under the hood you can leave these two aws fields blank if you have an IAM role applied to your FluentD instance.
+
+#### aws_sec_key
+The secret key portion of your AWS keypair
+
+#### tag
+This is a FluentD builtin.
+
+#### thread_num
+The number of threads to create to concurrently process the S3 objects. Defaults to 4.
+
+#### s3_get_max
+Control the size of the S3 fetched list on each iteration. Default to 200.
+
+#### delimiter
+You shouldn't have to specify delimiter at all but this option is provided and passed to the S3 client in the event that you have a weird delimiter in your log file names. Defaults to `nil`.
+
+#### verbose
+Turn this on if you'd like to see verbose information about the plugin and how it's processing your files.
+
+### parse_date_time
+Turn this off when you don't want the date and time to be parsed into the timestamp for the record.
+Used when timestamp parsing can be implemented faster downstream. Default is true.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'fluent-plugin-cloudfront-log-optimized'
+```
+
+And then execute:
+
+$ bundle
+
+Or install it yourself as:
+
+$ gem install 'fluent-plugin-cloudfront-log-optimized'
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/packetloop/fluent-plugin-cloudfront-log-optimized.
+
+## Credits
+
+kubihie

data/Rakefile

@@ -0,0 +1,12 @@
+require "bundler/gem_tasks"
+
+require "rake/testtask"
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'lib' << 'test'
+  test.pattern = 'test/**/test_*.rb'
+  test.verbose = true
+  test.warning = false
+end
+
+
+task :default => :test

data/fluent-plugin-cloudfront-log-optimized.gemspec

@@ -0,0 +1,26 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+Gem::Specification.new do |spec|
+  spec.name          = "fluent-plugin-cloudfront-log-optimized"
+  spec.version       = "0.2.0"
+  spec.authors       = ["kubihee", "lenfree", "kjwierenga"]
+  spec.email         = ["kubihie@gmail.com", "lenfree.yeung@gmail.com", "k.j.wierenga@gmail.com"]
+
+  spec.summary       = %q{AWS CloudFront log input plugin optimized for large log files. Credit to kubihie and lenfree.}
+  spec.description   = %q{AWS CloudFront log input plugin for fluentd. Upstream appears to be unmaintained.}
+  spec.homepage      = "https://github.com/kjwierenga/fluent-plugin-cloudfront-log-optimized"
+
+  spec.files         = `git ls-files`.split($/)
+  spec.executables   = spec.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "fluentd", ">= 0.14.0", "< 2"
+  spec.add_dependency "aws-sdk-s3", "~> 1"
+  spec.add_dependency "aws-sdk-sqs", "~> 1"
+  spec.add_development_dependency "bundler", "~> 1.7"
+  spec.add_development_dependency "rake", "~> 12"
+  spec.add_development_dependency 'test-unit', "~> 2"
+end

data/lib/fluent/plugin/enumerable_inflater.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+require 'zlib'
+
+class EnumerableInflater
+
+  CHUNKSIZE = 1024**2
+
+  def initialize(options = {})
+    @io = options[:io]
+  end
+
+  def lines
+    Enumerator.new { |main_enum| stream_lines(main_enum) }
+  end
+
+  private
+
+  attr_reader :io, :inflater
+
+  def stream_lines(main_enum)
+    init_gzip_inflater
+
+    split_lines.lazy.each { |line| main_enum << line }
+  ensure
+    inflater&.close
+  end
+
+  def split_lines
+    buffer = ""
+
+    Enumerator.new do |yielder|
+      ungzip.each do |decompressed_chunk|
+        buffer += decompressed_chunk
+        new_buffer = ""
+        buffer.each_line do |l|
+          l.end_with?("\n") ? yielder << l : new_buffer += l
+        end
+
+        buffer = new_buffer
+      end
+    end
+  end
+
+  def ungzip
+    Enumerator.new do |yielder|
+      stream_file.each do |compressed|
+        inflater.inflate(compressed) do |decompressed_chunk|
+          yielder << decompressed_chunk
+        end
+      end
+    end
+  end
+
+  def stream_file
+    Enumerator.new do |stream_enum|
+      io.each(nil, CHUNKSIZE) do |chunk|
+        stream_enum << chunk
+      end
+    end
+  end
+
+  def init_gzip_inflater
+    # Taken from examples in:
+    # https://docs.ruby-lang.org/en/2.0.0/Zlib/Inflate.html
+    @inflater = Zlib::Inflate.new(Zlib::MAX_WBITS + 32)
+  end
+
+end

data/lib/fluent/plugin/in_cloudfront_log.rb

@@ -0,0 +1,216 @@
+require 'fluent/input'
+require 'fluent/plugin/enumerable_inflater'
+require 'fileutils'
+
+class Fluent::Cloudfront_LogInput < Fluent::Input
+  Fluent::Plugin.register_input('cloudfront_log', self)
+
+  config_param :aws_key_id,        :string,  :default => nil, :secret => true
+  config_param :aws_sec_key,       :string,  :default => nil, :secret => true
+  config_param :log_bucket,        :string
+  config_param :log_prefix,        :string
+  config_param :moved_log_bucket,  :string,  :default => nil
+  config_param :moved_log_prefix,  :string,  :default => '_moved'
+  config_param :region,            :string
+
+  config_param :tag,               :string,  :default => 'cloudfront.access'
+  config_param :interval,          :integer, :default => 300
+  config_param :delimiter,         :string,  :default => nil
+  config_param :verbose,           :bool,    :default => false
+  config_param :thread_num,        :integer, :default => 4
+  config_param :s3_get_max,        :integer, :default => 200
+
+  config_param :parse_date_time,   :bool, :default => true
+
+  def initialize
+    super
+    require 'logger'
+    require 'zlib'
+    require 'aws-sdk-s3'
+    require 'time'
+    require 'uri'
+  end
+
+  def configure(conf)
+    super
+
+    raise Fluent::ConfigError.new unless @region
+    raise Fluent::ConfigError.new unless @log_bucket
+    raise Fluent::ConfigError.new unless @log_prefix
+
+    @moved_log_bucket = @log_bucket unless @moved_log_bucket
+    @moved_log_prefix = @log_prefix + '_moved' unless @moved_log_prefix
+
+    if @verbose
+      log.info("@log_bucket: #{@log_bucket}")
+      log.info("@moved_log_bucket: #{@moved_log_bucket}")
+      log.info("@log_prefix: #{@log_prefix}")
+      log.info("@moved_log_prefix: #{@moved_log_prefix}")
+      log.info("@thread_num: #{@thread_num}")
+      log.info("@parse_date_time: #{@parse_date_time}")
+    end
+  end
+
+  def start
+    super
+    log.info("Cloudfront verbose logging enabled") if @verbose
+    client
+
+    @tmp_dir = File.join(plugin_root_dir || '/', 'tmp')
+    FileUtils.mkdir_p @tmp_dir
+
+    @loop = Coolio::Loop.new
+    timer = TimerWatcher.new(@interval, true, log, &method(:input))
+
+    @loop.attach(timer)
+    @thread = Thread.new(&method(:run))
+  end
+
+  def shutdown
+    @loop.stop
+    @thread.join
+  end
+
+  def run
+    @loop.run
+  end
+
+  def client
+    begin
+      options = {:region => @region}
+      if @aws_key_id and @aws_sec_key
+        options[:access_key_id] = @aws_key_id
+        options[:secret_access_key] = @aws_sec_key
+      end
+      @client = Aws::S3::Client.new(options)
+    rescue => e
+      log.warn("S3 client error. #{e.message}")
+    end
+  end
+
+  def parse_header(line)
+    case line
+    when /^#Version:.+/i then
+      @version = line.sub(/^#Version:/i, '').strip
+    when /^#Fields:.+/i then
+      @fields = line.sub(/^#Fields:/i, '').strip.split("\s")
+    end
+  end
+
+  def purge(filename)
+    # Key is the name of the object without the bucket prefix, e.g: asdf/asdf.jpg
+    source_object_key       = [@log_prefix, filename].join('/')
+
+    # Full path includes bucket name in addition to object key, e.g: bucket/asdf/asdf.jpg
+    source_object_full_path = [@log_bucket, source_object_key].join('/')
+
+    dest_object_key         = [@moved_log_prefix, filename].join('/')
+    dest_object_full_path   = [@moved_log_bucket, dest_object_key].join('/')
+
+    log.info("Copying object: #{source_object_full_path} to #{dest_object_full_path}") if @verbose
+
+    begin
+      client.copy_object(:bucket => @moved_log_bucket, :copy_source => source_object_full_path, :key => dest_object_key)
+    rescue => e
+      log.warn("S3 Copy client error. #{e.message}")
+      return
+    end
+
+
+    log.info("Deleting object: #{source_object_key} from #{@log_bucket}") if @verbose
+    begin
+      client.delete_object(:bucket => @log_bucket, :key => source_object_key)
+    rescue => e
+      log.warn("S3 Delete client error. #{e.message}")
+      return
+    end
+  end
+
+  def process_line(line)
+    if line[0.1] == '#'
+      parse_header(line)
+      return
+    end
+
+    record = [
+      @fields,
+      CGI.unescape(line).strip.split("\t") # hoge%2520fuga -> hoge%20fuga
+    ].transpose.to_h
+
+    timestamp = if @parse_date_time
+                  Time.iso8601("#{record['date']}T#{record['time']}+00:00").to_i
+                else
+                  Time.now.to_i
+                end
+
+    router.emit(@tag, timestamp, record)
+  end
+
+  def process_content(content)
+    filename = content.key.sub(/^#{@log_prefix}\//, "")
+    log.info("CloudFront Currently processing: #{filename}") if @verbose
+    return if filename[-1] == '/'  #skip directory/
+    return unless filename[-2, 2] == 'gz'  #skip without gz file
+
+    tmp_file_name = File.join(@tmp_dir, content.key.split('/').last)
+    File.open(tmp_file_name, File::RDWR|File::CREAT, 0644) do |file|
+      # download file to local file system
+      client.get_object({bucket: @log_bucket, key: content.key}, target: file)
+
+      # inflate and process in chunks
+      file.rewind
+      EnumerableInflater.new(io: file).lines.each do |line|
+        process_line(line)
+      end
+      purge(filename)
+    rescue => e
+      log.warn("S3 GET client error. #{e.message}")
+      return
+    ensure
+      File.delete(file)
+    end
+  end
+
+  def input
+    log.info("CloudFront Begining input going to list S3")
+    begin
+      s3_list = client.list_objects_v2(:bucket => @log_bucket, :prefix => @log_prefix , :delimiter => @delimiter, :max_keys => @s3_get_max)
+    rescue => e
+      log.warn("S3 GET list error. #{e.message}")
+      return
+    end
+    log.info("Finished S3 get list")
+    queue = Queue.new
+    threads = []
+    log.debug("S3 List size: #{s3_list.contents.length}")
+    s3_list.contents.sort_by(&:last_modified).each do |content|
+      queue << content
+    end
+    # BEGINS THREADS
+    @thread_num.times do
+      threads << Thread.new do
+        until queue.empty?
+          work_unit = queue.pop(true) rescue nil
+          if work_unit
+            process_content(work_unit)
+          end
+        end
+       end
+     end
+     log.debug("CloudFront Waiting for Threads to finish...")
+     threads.each { |t| t.join }
+     log.debug("CloudFront Finished")
+  end
+
+  class TimerWatcher < Coolio::TimerWatcher
+    def initialize(interval, repeat, log, &callback)
+      @callback = callback
+      @log = log
+      super(interval, repeat)
+    end
+
+    def on_timer
+      @callback.call
+    end
+  end
+end

data/test/helper.rb

@@ -0,0 +1,28 @@
+require 'rubygems'
+require 'bundler'
+begin
+  Bundler.setup(:default, :development)
+rescue Bundler::BundlerError => e
+  $stderr.puts e.message
+  $stderr.puts "Run `bundle install` to install missing gems"
+  exit e.status_code
+end
+require 'test/unit'
+
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+$LOAD_PATH.unshift(File.dirname(__FILE__))
+require 'fluent/test'
+unless ENV.has_key?('VERBOSE')
+  nulllogger = Object.new
+  nulllogger.instance_eval {|obj|
+    def method_missing(method, *args)
+      # pass
+    end
+  }
+  $log = nulllogger
+end
+
+require 'fluent/plugin/in_cloudfront_log'
+
+class Test::Unit::TestCase
+end

data/test/plugin/test_in_cloudfrontlog.rb

@@ -0,0 +1,108 @@
+require_relative '../helper'
+require 'fluent/test'
+require 'fluent/test/driver/input'
+
+class Cloudfront_LogInputTest < Test::Unit::TestCase
+  setup do
+    Fluent::Test.setup
+  end
+
+  MINIMAL_CONFIG = %[
+    region     ap-northeast-1
+    log_bucket bucket-name
+    log_prefix a/b/c
+
+    # aws_key_id       AKIAZZZZZZZZZZZZZZZZ
+    # aws_sec_key      1234567890qwertyuiopasdfghjklzxcvbnm
+    # moved_log_bucket bucket-name-moved
+    # moved_log_prefix a/b/c_moved
+    # tag              cloudfront
+    # interval         500
+    # verbose          true
+    # thread_num 8
+    # parse_date_time false
+  ]
+
+  def create_driver(conf = MINIMAL_CONFIG)
+    Fluent::Test::Driver::Input.new(Fluent::Cloudfront_LogInput).configure(conf)
+  end
+
+  test "create_driver doesn't raise error" do
+    assert_nothing_raised { create_driver }
+  end
+
+  sub_test_case "required parameters" do
+    test "region is required" do
+      exception = assert_raise(Fluent::ConfigError) {
+        create_driver(MINIMAL_CONFIG.gsub(/region.*$/, ''))
+      }
+      assert_equal("'region' parameter is required", exception.message)
+    end
+
+    test "log_bucket is required" do
+      exception = assert_raise(Fluent::ConfigError) {
+        create_driver(MINIMAL_CONFIG.gsub(/log_bucket.*$/, ''))
+      }
+      assert_equal("'log_bucket' parameter is required", exception.message)
+    end
+
+    test "log_prefix is required" do
+      exception = assert_raise(Fluent::ConfigError) {
+        create_driver(MINIMAL_CONFIG.gsub(/log_prefix.*$/, ''))
+      }
+      assert_equal("'log_prefix' parameter is required", exception.message)
+    end
+  end
+
+  sub_test_case "default values" do
+    test "moved_log_bucket is set to log_bucket" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal(driver.instance.log_bucket, driver.instance.moved_log_bucket)
+    end
+
+    test "moved_log_prefix is set to '_moved'" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal('_moved', driver.instance.moved_log_prefix)
+    end
+
+    test "tag is set to 'cloudfront.access'" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal('cloudfront.access', driver.instance.tag)
+    end
+
+    test "verbose is set to false" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal(false, driver.instance.verbose)
+    end
+
+    test "interval is set to 300" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal(300, driver.instance.interval)
+    end
+
+    test "thread_num is set to 4" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal(4, driver.instance.thread_num)
+    end
+
+    test "s3_get_max is set to 200" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal(200, driver.instance.s3_get_max)
+    end
+
+    test "parse_date_time true" do
+      driver = create_driver(MINIMAL_CONFIG)
+      assert_equal(true, driver.instance.parse_date_time)
+    end
+  end
+
+  sub_test_case "set specific values" do
+    test "moved_log_prefix is set to 'my-prefix'" do
+      driver = create_driver(MINIMAL_CONFIG + %[
+        moved_log_prefix 'my-prefix'
+      ])
+      assert_equal('my-prefix', driver.instance.moved_log_prefix)
+    end
+  end
+
+end

metadata

@@ -0,0 +1,149 @@
+--- !ruby/object:Gem::Specification
+name: fluent-plugin-cloudfront-log-optimized
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- kubihee
+- lenfree
+- kjwierenga
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2021-02-06 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: fluentd
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.14.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.14.0
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2'
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-s3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-sqs
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '12'
+- !ruby/object:Gem::Dependency
+  name: test-unit
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+description: AWS CloudFront log input plugin for fluentd. Upstream appears to be unmaintained.
+email:
+- kubihie@gmail.com
+- lenfree.yeung@gmail.com
+- k.j.wierenga@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- CHANGELOG.md
+- Gemfile
+- README.md
+- Rakefile
+- fluent-plugin-cloudfront-log-optimized.gemspec
+- lib/fluent/plugin/enumerable_inflater.rb
+- lib/fluent/plugin/in_cloudfront_log.rb
+- test/helper.rb
+- test/plugin/test_in_cloudfrontlog.rb
+homepage: https://github.com/kjwierenga/fluent-plugin-cloudfront-log-optimized
+licenses: []
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.8
+signing_key:
+specification_version: 4
+summary: AWS CloudFront log input plugin optimized for large log files. Credit to
+  kubihie and lenfree.
+test_files:
+- test/helper.rb
+- test/plugin/test_in_cloudfrontlog.rb