fluentd-plugin-kinesis-intuit 2.1.1

data/.travis.yml

@@ -0,0 +1,14 @@
+language: ruby
+os: linux
+sudo: false
+
+matrix:
+  include:
+    - rvm: 2.4.1
+      gemfile: Gemfile
+    - rvm: 2.4.1
+      gemfile: gemfiles/Gemfile.fluentd-0.14.10
+    - rvm: 2.4.4 # https://github.com/treasure-data/omnibus-td-agent/blob/v3.2.0/config/projects/td-agent3.rb#L22
+      gemfile: gemfiles/Gemfile.td-agent-3.2.0
+
+script: bundle exec rake test

data/CHANGELOG.md

@@ -0,0 +1,115 @@
+# CHANGELOG
+
+## 2.1.1
+
+- Bug - Fix require aws-sdk-core before requiring the aws related libraries [#140](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/140)
+
+## 2.1.0
+
+- Feature - Added sts_http_proxy parameter to assume_role_credentials configuration [#136](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/136)
+
+## 2.0.1
+
+- Bug - Fix AWS SDK conflict with s3 plugin [#131](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/131)
+
+## 2.0.0
+
+- Feature - Add `kinesis_streams_aggregated` ouput plugin [#107](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/107)
+- Feature - Support fluentd worker model [#104](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/104)
+- Feature - Support AWS SDK for Ruby v3 [#102](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/102)
+- Enhancement - Refactor class design [#103](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/103)
+- Enhancement - More configuration for AssumeRole [#63](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/63)
+- Enhancement - Refactor credentials helper [#94](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/94)
+- Enhancement - Revisit backoff logic [#69](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/69)
+- Enhancement - Support compressing output [#98](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/98)
+- Enhancement - Support nanosecond time_key format [#124](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/124)
+- License - Move back to Apache License Version 2.0
+
+## 1.3.0
+
+- Feature - Log KPL stdout/err to logger [#129](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/129) [#130](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/130)
+
+## 1.2.0
+
+- Feature - Add reduce_max_size_error_message configuration [#127](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/127)
+- Bug fix - Fix empty batch error [#125](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/125)
+
+## 1.1.3
+
+- Bug fix - Fix issues with fluentd 0.14.12 [#99](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/99)
+
+## 1.1.2
+
+- Bug fix - Adjust credentials_provider for newer version of aws-ruby-sdk [#93](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/93)
+
+## 1.1.1
+
+- Bug fix - Fix incompatibility for AWS SDK 2.5 [#80](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/80)
+- Bug fix - Fix wrong logic for batch request [#81](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/81)
+
+## 1.1.0
+
+- Feature - Derive stream name from fluentd tag for KPL [#67](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/67)
+- Enhancement - Make http_proxy parameter secret [#64](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/64)
+- Bug fix - Plugin incompatible with new fluentd release 0.14 [#70](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/70)
+- Misc - Fix legacy test and reduce travis tests [#74](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/74)
+- Misc - Some test, benchmark improvement [#74](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/74), [#75](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/75), [#76](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/76)
+
+## 1.0.1
+
+- Bug fix - Instance profile credentials expiring [#58](https://github.com/awslabs/aws-fluent-plugin-kinesis/issues/58)
+
+## 1.0.0
+
+To support Firehose and KPL, this was refactored and added more tests.
+
+- Feature - Support Firehose
+- Feature - Support KPL
+- Feature - New parameters, such as formatter, data_key, reset_backoff_if_success, batch_request_max_count, batch_request_max_size, log_truncate_max_size
+
+## 0.4.0
+
+- Feature - Add option to ensure Kinesis Stream connection. [#35](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/35)
+- Feature - Add option to support zlib compression. [#39](https://github.com/awslabs/aws-fluent-plugin-kinesis/pull/39)
+
+Note: We introduced [Semantic Versioning](http://semver.org/) here.
+
+## 0.3.6
+
+- **Cross account access support**: Added support for cross account access for Amazon Kinesis stream. With this update, you can put reocrds to streams those are owned by other AWS account. This feature is achieved by [AssumeRole](http://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html).
+
+## 0.3.5
+
+- **1MB record size limit**: Increased record size limit from 50KB to 1MB due to [Amazon Kinesis improvement.](http://aws.amazon.com/jp/about-aws/whats-new/2015/06/amazon-kinesis-announces-put-pricing-change-1mb-record-support-and-the-kinesis-producer-library/)
+- **Switching IAM user support**: Added support for [shared credential file](http://docs.aws.amazon.com/ja_jp/AWSSdkDocsRuby/latest/DeveloperGuide/prog-basics-creds.html#creds-specify-provider).
+
+## 0.3.4
+
+- **Multi-byte UTF-8 support**: We now support multi-byte UTF-8 by using *use_yajl* option.
+
+## 0.3.3
+
+- **Security improvements**: Disabled logging `aws_key_id` and `aws_sec_key` into log file.
+
+## 0.3.2
+
+- **http_proxy support**: Added HTTP proxy support.
+
+## 0.3.1
+
+- **Fluentd v0.12 support**: We now support Fluentd v0.12.
+
+## 0.3.0
+
+- **Throughput improvement**: Added support for [PutRecords API](http://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecords.html) by default.
+- **Bug fix**: Removed redundant Base64 encoding of data for each Kinesis record emitted. Applications consuming these records will need to be updated accordingly.
+
+## 0.2.0
+
+- **Partition key randomization**: Added support for partition key randomization.
+- **Throughput improvements**: Added support for spawning additional processes and threads to increase throughput to Amazon Kinesis.
+- **AWS SDK for Ruby V2**: Added support for [AWS SDK for Ruby V2](https://github.com/aws/aws-sdk-core-ruby).
+
+## 0.1.0
+
+- Release on Github

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,4 @@
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+opensource-codeofconduct@amazon.com with any additional questions or comments.

data/COMMUNITY.md

@@ -0,0 +1,26 @@
+# Community
+
+
+## Maintainers
+Following are the maintainers of this project
+[Lakshmi Prasanna M S](https://github.intuit.com/api/v3/users/lms)
+
+
+
+## Trusted Committers
+
+Following are the Trusted Committers of this project
+[Lakshmi Prasanna M S](https://github.intuit.com/api/v3/users/lms)
+
+
+
+## Slack Channel
+
+ - [#pi-idc-shankar-team](https://intuit-teams.slack.com/messages/G98QRJW5Q)
+
+
+## Jira
+
+ - [One Intuit Logging Support]
+- Epic: Containerized Application Logging
+- Product Owner : [None](https://github.intuit.com/api/v3/users/cloud-ops)

data/CONTRIBUTING.md

@@ -0,0 +1,160 @@
+# Contributing Guidelines
+
+This document contains information and guidelines about contributing to this project.
+Please read it before you start participating.
+
+## Asking Questions
+
+You can ask questions through our Slack channel [#pi-idc-shankar-team](https://intuit-teams.slack.com/messages/G98QRJW5Q) or by adding a [github issue](/issues).
+
+## Coding conventions
+
+Please follow standard practice for the language you are adding.
+
+## Testing conventions
+
+* Write tests to cover any new code
+* Try to write code that is unit-testable, and write unit tests
+* If appropriate also create integration tests
+
+## Branching conventions
+
+* Keep branches short-lived
+* Merge to master in small increments
+* Delete branch after merging a PR to master
+
+## Commit-message conventions
+
+* Include JIRA story or GitHub issue in commit subject
+* Write a description of the changes made in the commit message
+
+## Submitting a Pull Request
+
+You can submit a pull request to this repository at any time. Follow these guidelines to make it easier for you and the project maintainers to merge the PR to mainline:
+
+If you are external to the team please start by forking the project. If you are on the team please coordinate with them to produce an appropriate branch.
+
+1. [Fork](http://help.github.com/fork-a-repo/) the project, clone your fork,
+   and configure the remotes:
+
+   ```bash
+   # Clone your fork of the repo into the current directory
+   git clone git@github.intuit.com/cloud-logging/aws-fluent-plugin-kinesis-intuit.git
+   # Navigate to the newly cloned directory
+   cd oil-federated-search
+   # Assign the original repo to a remote called "upstream"
+   git remote add upstream https://github.intuit.com/cloud-logging/aws-fluent-plugin-kinesis-intuit.git
+   ```
+
+2. Run the unit tests.  If you added new functionality, extend existing test cases or add new ones.
+
+3. If you cloned a while ago, get the latest changes from upstream:
+
+   ```bash
+   git checkout master
+   git pull upstream master
+   ```
+
+4. Create a new topic branch (off the main project development branch) to
+   contain your feature, change, or fix:
+
+   ```bash
+   git checkout -b <topic-branch-name>
+   ```
+
+5. Commit your changes in logical chunks. For any Git project, some good rules
+   for commit messages are
+   * the first line is commit summary, 50 characters or less,
+   * followed by an empty line
+   * followed by an explanation of the commit, wrapped to 72 characters.
+
+   The first line of a commit message becomes the **title** of a pull request on
+   GitHub, like the subject line of an email. Including the key info in the
+   first line will help us respond faster to your pull.
+
+   If your pull request has multiple commits which revise the same lines of
+   code, it is better to [squash](http://davidwalsh.name/squash-commits-git)
+   those commits together into one logical unit.
+
+   But you don't always have to squash &mdash; it is fine for a pull request to
+   contain multiple commits when there is a logical reason for the separation.
+
+6. Push your topic branch up to your fork:
+
+   ```bash
+   git push origin <topic-branch-name>
+   ```
+
+7. [Open a Pull Request](https://help.github.com/articles/using-pull-requests/)
+    with a clear title and description.
+
+8. After your Pull Request is away, you might want to get yourself back onto
+   `master` and delete the topic branch:
+
+   ```bash
+   git checkout master
+   git branch -D <topic-branch-name>
+   ```
+
+Please refer to  [this wiki](https://github.intuit.com/fabric-app-shells/mobile-shell-ios/wiki/How-to-develop-across-multiple-repositories) if you want to make changes across multiple repositories.
+
+<sub><sup>Pull request instructions from [Sergey Alexandrov](https://github.com/taketwo)</sup></sub>
+
+## Reporting Other Issues
+
+A great way to contribute to the project
+is to send a detailed issue when you encounter an problem.
+We always appreciate a well-written, thorough bug report.
+
+Check that the project issues database
+doesn't already include that problem or suggestion before submitting an issue.
+If you find a match, add a quick "+1" or "I have this problem too."
+Doing this helps prioritize the most common problems and requests.
+
+When reporting issues, please include the following:
+
+- Steps to reproduce
+- Expected Results
+- Actual Results
+- Version if available
+- Environment Setup
+
+Please be descriptive in each and be sure that the report can easily be reproduced if possible.
+
+This information will help us review and fix your issue faster.
+
+## Developer's Certificate of Origin
+
+By making a contribution to this project, I certify that:
+
+- (a) The contribution was created in whole or in part by me and I
+      have the right to submit it under the open source license
+      indicated in the file; or
+
+- (b) The contribution is based upon previous work that, to the best
+      of my knowledge, is covered under an appropriate open source
+      license and I have the right under that license to submit that
+      work with modifications, whether created in whole or in part
+      by me, under the same open source license (unless I am
+      permitted to submit under a different license), as indicated
+      in the file; or
+
+- (c) The contribution was provided directly to me by some other
+      person who certified (a), (b) or (c) and I have not modified
+      it.
+
+- (d) I understand and agree that this project and the contribution
+      are public and that a record of the contribution (including all
+      personal information I submit with it, including my sign-off) is
+      maintained indefinitely and may be redistributed consistent with
+      this project or the open source license(s) involved.
+
+## Code of Conduct
+
+The Code of Conduct governs how we behave in public or in private
+whenever the project will be judged by our actions.
+We expect it to be honored by everyone who contributes to this project.
+
+---
+
+<sub><sup>*Some of the ideas and wording for the statements above were based on work by the [AlamoFire](https://github.com/Alamofire/Alamofire/blob/master/CONTRIBUTING.md) community. We commend them for their efforts to facilitate collaboration in their projects.*</sup></sub>

data/CONTRIBUTORS.txt

@@ -0,0 +1,6 @@
+This file contains a list of people who have made large or regular contributions
+to the Fluent Plugin for Amazon Kinesis Streams and Amazon Kinesis Firehose. Give them a special thanks!
+
+Genki Sugawara
+sgwr_dts@yahoo.co.jp
+https://github.com/winebarrel

data/Gemfile

@@ -0,0 +1,18 @@
+#
+# Copyright 2014-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"). You
+# may not use this file except in compliance with the License. A copy of
+# the License is located at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# or in the "license" file accompanying this file. This file is
+# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+# ANY KIND, either express or implied. See the License for the specific
+# language governing permissions and limitations under the License.
+
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in fluent-plugin-kinesis.gemspec
+gemspec

data/Gemfile.lock

@@ -0,0 +1,112 @@
+PATH
+  remote: .
+  specs:
+    fluent-plugin-kinesis-intuit (2.1.1)
+      aws-sdk-firehose (~> 1)
+      aws-sdk-kinesis (~> 1)
+      fluentd (>= 0.14.0, < 2)
+      google-protobuf (~> 3)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    addressable (2.5.2)
+      public_suffix (>= 2.0.2, < 4.0)
+    aws-eventstream (1.0.1)
+    aws-partitions (1.128.0)
+    aws-sdk-core (3.44.2)
+      aws-eventstream (~> 1.0)
+      aws-partitions (~> 1.0)
+      aws-sigv4 (~> 1.0)
+      jmespath (~> 1.0)
+    aws-sdk-firehose (1.11.0)
+      aws-sdk-core (~> 3, >= 3.39.0)
+      aws-sigv4 (~> 1.0)
+    aws-sdk-kinesis (1.9.0)
+      aws-sdk-core (~> 3, >= 3.39.0)
+      aws-sigv4 (~> 1.0)
+    aws-sigv4 (1.0.3)
+    binding_of_caller (0.8.0)
+      debug_inspector (>= 0.0.1)
+    byebug (10.0.2)
+    coderay (1.1.2)
+    cool.io (1.5.3)
+    crack (0.4.3)
+      safe_yaml (~> 1.0.0)
+    debug_inspector (0.0.3)
+    dig_rb (1.0.1)
+    fakefs (0.18.0)
+    fluentd (1.3.3)
+      cool.io (>= 1.4.5, < 2.0.0)
+      dig_rb (~> 1.0.0)
+      http_parser.rb (>= 0.5.1, < 0.7.0)
+      msgpack (>= 0.7.0, < 2.0.0)
+      serverengine (>= 2.0.4, < 3.0.0)
+      sigdump (~> 0.2.2)
+      strptime (>= 0.2.2, < 1.0.0)
+      tzinfo (~> 1.0)
+      tzinfo-data (~> 1.0)
+      yajl-ruby (~> 1.0)
+    google-protobuf (3.6.1)
+    hashdiff (0.3.8)
+    http_parser.rb (0.6.0)
+    jmespath (1.4.0)
+    metaclass (0.0.4)
+    method_source (0.9.2)
+    mocha (1.7.0)
+      metaclass (~> 0.0.1)
+    msgpack (1.2.4)
+    net-empty_port (0.0.2)
+    power_assert (1.1.3)
+    pry (0.12.2)
+      coderay (~> 1.1.0)
+      method_source (~> 0.9.0)
+    pry-byebug (3.6.0)
+      byebug (~> 10.0)
+      pry (~> 0.10)
+    pry-stack_explorer (0.4.9.3)
+      binding_of_caller (>= 0.7)
+      pry (>= 0.9.11)
+    public_suffix (3.0.3)
+    rake (10.5.0)
+    rr (1.2.1)
+    safe_yaml (1.0.4)
+    serverengine (2.1.0)
+      sigdump (~> 0.2.2)
+    sigdump (0.2.4)
+    strptime (0.2.3)
+    test-unit (3.2.9)
+      power_assert
+    test-unit-rr (1.0.5)
+      rr (>= 1.1.1)
+      test-unit (>= 2.5.2)
+    thread_safe (0.3.6)
+    tzinfo (1.2.5)
+      thread_safe (~> 0.1)
+    tzinfo-data (1.2018.9)
+      tzinfo (>= 1.0.0)
+    webmock (3.5.1)
+      addressable (>= 2.3.6)
+      crack (>= 0.3.2)
+      hashdiff
+    yajl-ruby (1.4.1)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  bundler (~> 1.10)
+  fakefs (>= 0.8.1)
+  fluent-plugin-kinesis-intuit!
+  mocha (>= 1.1.0)
+  net-empty_port (>= 0.0.2)
+  pry (>= 0.10.1)
+  pry-byebug (>= 3.3.0)
+  pry-stack_explorer (>= 0.4.9.2)
+  rake (~> 10.0)
+  test-unit (>= 3.0.8)
+  test-unit-rr (>= 1.0.3)
+  webmock (>= 1.24.2)
+
+BUNDLED WITH
+   1.16.4