logstash-input-kinesis 1.4.3-java → 1.5.0-java

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. checksums.yaml +4 -4
  2. data/CONTRIBUTORS +14 -0
  3. data/README.md +5 -4
  4. data/lib/com/amazonaws/amazon-kinesis-client/1.6.1/amazon-kinesis-client-1.6.1.jar +0 -0
  5. data/lib/com/amazonaws/aws-java-sdk-cloudwatch/1.10.20/aws-java-sdk-cloudwatch-1.10.20.jar +0 -0
  6. data/{vendor/jar-dependencies/runtime-jars/aws-java-sdk-core-1.10.50.jar → lib/com/amazonaws/aws-java-sdk-core/1.10.20/aws-java-sdk-core-1.10.20.jar} +0 -0
  7. data/lib/com/amazonaws/aws-java-sdk-dynamodb/1.10.20/aws-java-sdk-dynamodb-1.10.20.jar +0 -0
  8. data/lib/com/amazonaws/aws-java-sdk-kinesis/1.10.20/aws-java-sdk-kinesis-1.10.20.jar +0 -0
  9. data/lib/com/amazonaws/aws-java-sdk-kms/1.10.20/aws-java-sdk-kms-1.10.20.jar +0 -0
  10. data/lib/com/amazonaws/aws-java-sdk-s3/1.10.20/aws-java-sdk-s3-1.10.20.jar +0 -0
  11. data/lib/com/fasterxml/jackson/core/jackson-annotations/2.5.0/jackson-annotations-2.5.0.jar +0 -0
  12. data/lib/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar +0 -0
  13. data/lib/com/fasterxml/jackson/core/jackson-databind/2.5.3/jackson-databind-2.5.3.jar +0 -0
  14. data/lib/com/google/guava/guava/18.0/guava-18.0.jar +0 -0
  15. data/lib/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar +0 -0
  16. data/lib/commons-codec/commons-codec/1.9/commons-codec-1.9.jar +0 -0
  17. data/lib/commons-lang/commons-lang/2.6/commons-lang-2.6.jar +0 -0
  18. data/lib/commons-logging/commons-logging/1.2/commons-logging-1.2.jar +0 -0
  19. data/lib/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar +0 -0
  20. data/lib/logstash/inputs/kinesis.rb +1 -1
  21. data/lib/logstash/inputs/kinesis/version.rb +2 -1
  22. data/lib/logstash/inputs/kinesis/worker.rb +1 -0
  23. data/lib/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar +0 -0
  24. data/lib/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar +0 -0
  25. data/logstash-input-kinesis.gemspec +8 -3
  26. data/spec/inputs/kinesis/worker_spec.rb +1 -0
  27. data/spec/inputs/kinesis_spec.rb +12 -1
  28. metadata +50 -22
  29. data/.gitignore +0 -14
  30. data/.rspec +0 -2
  31. data/.travis.yml +0 -3
  32. data/LICENSE.txt +0 -14
  33. data/Rakefile +0 -15
  34. data/pom.xml +0 -32
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: e2e3ea8c6324844f371148a1e1bf7743e0bb5cfe
4
- data.tar.gz: 7be846fcd88272b49885cf60b6256b46a3c8b68f
3
+ metadata.gz: 03bbacd8aa1c1ead7454666582fa2ec84cd79d86
4
+ data.tar.gz: ae48dab34705083645faff5c7b7c3674ef658a8e
5
5
  SHA512:
6
- metadata.gz: b1d06f224b074e9db2a8df3f0720610d416109ed39da7ab2bfff171453bbc5f5e11d740b008463d4f09f2d9bd5e88c721d49d813fe714b623c2d83855c959f38
7
- data.tar.gz: 20a09f5d185eb7a14f30fdf07d93256df689b6d25fd00284ee24cff4e08eee55a50cf89bec92d08f244369652293c3bf81510f75cb82c78b3ecc472ae4492c5e
6
+ metadata.gz: 04342e5d1a6e34138995d89cb894b732997d0d1b51378e46fd4db49823c7b0d8707fbe9bdbbb647b527240372a0ff59360f558866092cff849e91fca8a8b8fda
7
+ data.tar.gz: 86232000df97f16996c078caf62c6cd35f676c411ce1901966d4edb8da372267867a65739a75ab3a40267e4b6ac3c607702a8bdc69fa9e9e1288c4ca5a26b159
@@ -0,0 +1,14 @@
1
+ The following is a list of people who have contributed ideas, code, bug
2
+ reports, or in general have helped logstash along its way.
3
+
4
+ Maintainers:
5
+ * Brian Palmer (codekitchen)
6
+
7
+ Contributors:
8
+ * Brian Palmer (codekitchen)
9
+ * Samuel García Martínez (samuelgmartinez)
10
+
11
+ Note: If you've sent us patches, bug reports, or otherwise contributed to
12
+ Logstash, and you aren't on the list above and want to be, please let us know
13
+ and we'll make sure you're here. Contributions from folks like you are what make
14
+ open source awesome.
data/README.md CHANGED
@@ -1,12 +1,12 @@
1
1
  # Logstash AWS Kinesis Input Plugin
2
2
 
3
- [![Build Status](https://travis-ci.org/codekitchen/logstash-input-kinesis.svg)](https://travis-ci.org/codekitchen/logstash-input-kinesis)
3
+ [![Build Status](https://travis-ci.org/logstash-plugins/logstash-input-kinesis.svg)](https://travis-ci.org/logstash-plugins/logstash-input-kinesis)
4
4
 
5
5
  This is a [AWS Kinesis](http://docs.aws.amazon.com/kinesis/latest/dev/introduction.html) input plugin for [Logstash](https://github.com/elasticsearch/logstash). Under the hood uses the [Kinesis Client Library](http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-record-processor-implementation-app-java.html).
6
6
 
7
7
  ## Installation
8
8
 
9
- This plugin requires Logstash >= 1.5, and can be installed by Logstash
9
+ This plugin requires Logstash >= 2.0, and can be installed by Logstash
10
10
  itself.
11
11
 
12
12
  ```sh
@@ -39,7 +39,7 @@ This are the properties you can configure and what are the default values:
39
39
  * `checkpoint_interval_seconds`: How many seconds between worker checkpoints to DynamoDB. A low value ussually means lower message replay in case of node failure/restart but it increases CPU+network ussage (which increases the AWS costs).
40
40
  * **required**: false
41
41
  * **default value**: `60`
42
- * `metrics`: Worker metric tracking. By default this is disabled, set it to "cloudwatch" to enable the cloudwatch integration in the Kinesis Client Library.
42
+ * `metrics`: Worker metric tracking. By default this is disabled, set it to "cloudwatch" to enable the cloudwatch integration in the Kinesis Client Library.
43
43
  * **required**: false
44
44
  * **default value**: `nil`
45
45
 
@@ -61,7 +61,8 @@ Look at the [documentation](https://docs.aws.amazon.com/AWSJavaSDK/latest/javado
61
61
 
62
62
  ## Contributing
63
63
 
64
- 1. Fork it ( https://github.com/codekitchen/logstash-input-kinesis/fork )
64
+ 0. https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md#contribution-steps
65
+ 1. Fork it ( https://github.com/logstash-plugins/logstash-input-kinesis/fork )
65
66
  2. Create your feature branch (`git checkout -b my-new-feature`)
66
67
  3. Commit your changes (`git commit -am 'Add some feature'`)
67
68
  4. Push to the branch (`git push origin my-new-feature`)
@@ -85,7 +85,7 @@ class LogStash::Inputs::Kinesis < LogStash::Inputs::Base
85
85
  end
86
86
  end
87
87
 
88
- def teardown
88
+ def stop
89
89
  @kcl_worker.shutdown if @kcl_worker
90
90
  end
91
91
 
@@ -1,7 +1,8 @@
1
+ # encoding: utf-8
1
2
  module Logstash
2
3
  module Input
3
4
  module Kinesis
4
- VERSION = "1.4.3"
5
+ VERSION = "1.5.0"
5
6
  end
6
7
  end
7
8
  end
@@ -1,3 +1,4 @@
1
+ # encoding: utf-8
1
2
  class LogStash::Inputs::Kinesis::Worker
2
3
  include com.amazonaws.services.kinesis.clientlibrary.interfaces.v2::IRecordProcessor
3
4
 
@@ -10,10 +10,10 @@ Gem::Specification.new do |spec|
10
10
  spec.email = ["brian@codekitchen.net"]
11
11
  spec.summary = %q{Logstash plugin for Kinesis input}
12
12
  spec.description = %q{This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program}
13
- spec.homepage = "https://github.com/codekitchen/logstash-input-kinesis"
13
+ spec.homepage = "https://github.com/logstash-plugins/logstash-input-kinesis"
14
14
  spec.licenses = ['Apache License (2.0)']
15
15
 
16
- spec.files = `git ls-files`.split($\)
16
+ spec.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
17
17
  spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
18
  spec.require_paths = ["lib"]
19
19
 
@@ -22,9 +22,14 @@ Gem::Specification.new do |spec|
22
22
 
23
23
  spec.platform = 'java'
24
24
 
25
+ spec.requirements << "jar 'com.amazonaws:amazon-kinesis-client', '1.6.1'"
26
+ spec.requirements << "jar 'org.apache.httpcomponents:httpclient', '4.5.1'"
27
+
28
+ spec.add_development_dependency 'jar-dependencies', '0.3.2'
29
+
25
30
  spec.add_development_dependency "bundler", "~> 1.7"
26
31
  spec.add_development_dependency "rake", "~> 10.0"
27
32
  spec.add_development_dependency "rspec", "~> 3.2.0"
28
- spec.add_development_dependency "logstash-core", ">= 1.5.1"
33
+ spec.add_development_dependency "logstash-core", "~> 2.4.0"
29
34
  spec.add_development_dependency "logstash-codec-json"
30
35
  end
@@ -48,6 +48,7 @@ RSpec.describe "LogStash::Inputs::Kinesis::Worker" do
48
48
  describe "#processRecords" do
49
49
  it "decodes and queues each record with decoration" do
50
50
  worker.processRecords(process_input)
51
+ expect(queue.size).to eq(2)
51
52
  m1 = queue.pop
52
53
  m2 = queue.pop
53
54
  expect(m1).to be_kind_of(LogStash::Event)
@@ -43,11 +43,22 @@ RSpec.describe "inputs/kinesis" do
43
43
  end
44
44
  end
45
45
 
46
+ context "#stop" do
47
+ it "stops the KCL worker" do
48
+ expect(kinesis).to receive(:kcl_builder).with(queue).and_return(stub_builder)
49
+ expect(kcl_worker).to receive(:run).with(no_args)
50
+ expect(kcl_worker).to receive(:shutdown).with(no_args)
51
+ kinesis.run(queue)
52
+ kinesis.do_stop # do_stop calls stop internally
53
+ end
54
+ end
55
+
46
56
  context "#worker_factory" do
47
57
  it "clones the codec for each worker" do
48
- expect(codec).to receive(:clone).once
49
58
  worker = kinesis.worker_factory(queue).call()
50
59
  expect(worker).to be_kind_of(LogStash::Inputs::Kinesis::Worker)
60
+ expect(worker.codec).to_not eq(kinesis.codec)
61
+ expect(worker.codec).to be_kind_of(codec.class)
51
62
  end
52
63
 
53
64
  it "generates a valid worker" do
metadata CHANGED
@@ -1,15 +1,29 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-kinesis
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.4.3
4
+ version: 1.5.0
5
5
  platform: java
6
6
  authors:
7
7
  - Brian Palmer
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-02-08 00:00:00.000000000 Z
11
+ date: 2016-09-29 00:00:00.000000000 Z
12
12
  dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - '='
17
+ - !ruby/object:Gem::Version
18
+ version: 0.3.2
19
+ name: jar-dependencies
20
+ prerelease: false
21
+ type: :development
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - '='
25
+ - !ruby/object:Gem::Version
26
+ version: 0.3.2
13
27
  - !ruby/object:Gem::Dependency
14
28
  requirement: !ruby/object:Gem::Requirement
15
29
  requirements:
@@ -55,17 +69,17 @@ dependencies:
55
69
  - !ruby/object:Gem::Dependency
56
70
  requirement: !ruby/object:Gem::Requirement
57
71
  requirements:
58
- - - '>='
72
+ - - ~>
59
73
  - !ruby/object:Gem::Version
60
- version: 1.5.1
74
+ version: 2.4.0
61
75
  name: logstash-core
62
76
  prerelease: false
63
77
  type: :development
64
78
  version_requirements: !ruby/object:Gem::Requirement
65
79
  requirements:
66
- - - '>='
80
+ - - ~>
67
81
  - !ruby/object:Gem::Version
68
- version: 1.5.1
82
+ version: 2.4.0
69
83
  - !ruby/object:Gem::Dependency
70
84
  requirement: !ruby/object:Gem::Requirement
71
85
  requirements:
@@ -87,26 +101,34 @@ executables: []
87
101
  extensions: []
88
102
  extra_rdoc_files: []
89
103
  files:
90
- - .gitignore
91
- - .rspec
92
- - .travis.yml
93
- - Gemfile
94
- - LICENSE.txt
95
- - README.md
96
- - Rakefile
97
104
  - lib/logstash-input-kinesis_jars.rb
105
+ - lib/com/amazonaws/amazon-kinesis-client/1.6.1/amazon-kinesis-client-1.6.1.jar
106
+ - lib/com/amazonaws/aws-java-sdk-cloudwatch/1.10.20/aws-java-sdk-cloudwatch-1.10.20.jar
107
+ - lib/com/amazonaws/aws-java-sdk-core/1.10.20/aws-java-sdk-core-1.10.20.jar
108
+ - lib/com/amazonaws/aws-java-sdk-dynamodb/1.10.20/aws-java-sdk-dynamodb-1.10.20.jar
109
+ - lib/com/amazonaws/aws-java-sdk-kinesis/1.10.20/aws-java-sdk-kinesis-1.10.20.jar
110
+ - lib/com/amazonaws/aws-java-sdk-kms/1.10.20/aws-java-sdk-kms-1.10.20.jar
111
+ - lib/com/amazonaws/aws-java-sdk-s3/1.10.20/aws-java-sdk-s3-1.10.20.jar
112
+ - lib/com/fasterxml/jackson/core/jackson-annotations/2.5.0/jackson-annotations-2.5.0.jar
113
+ - lib/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar
114
+ - lib/com/fasterxml/jackson/core/jackson-databind/2.5.3/jackson-databind-2.5.3.jar
115
+ - lib/com/google/guava/guava/18.0/guava-18.0.jar
116
+ - lib/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar
117
+ - lib/commons-codec/commons-codec/1.9/commons-codec-1.9.jar
118
+ - lib/commons-lang/commons-lang/2.6/commons-lang-2.6.jar
119
+ - lib/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
120
+ - lib/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar
98
121
  - lib/logstash/inputs/kinesis.rb
99
122
  - lib/logstash/inputs/kinesis/version.rb
100
123
  - lib/logstash/inputs/kinesis/worker.rb
101
- - logstash-input-kinesis.gemspec
102
- - pom.xml
103
- - spec/inputs/kinesis/worker_spec.rb
104
- - spec/inputs/kinesis_spec.rb
124
+ - lib/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar
125
+ - lib/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar
105
126
  - spec/spec_helper.rb
127
+ - spec/inputs/kinesis_spec.rb
128
+ - spec/inputs/kinesis/worker_spec.rb
106
129
  - vendor/jar-dependencies/runtime-jars/amazon-kinesis-client-1.6.1.jar
107
130
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-cloudwatch-1.10.20.jar
108
131
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-core-1.10.20.jar
109
- - vendor/jar-dependencies/runtime-jars/aws-java-sdk-core-1.10.50.jar
110
132
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-dynamodb-1.10.20.jar
111
133
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-kinesis-1.10.20.jar
112
134
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-kms-1.10.20.jar
@@ -122,7 +144,11 @@ files:
122
144
  - vendor/jar-dependencies/runtime-jars/jackson-databind-2.5.3.jar
123
145
  - vendor/jar-dependencies/runtime-jars/joda-time-2.8.1.jar
124
146
  - vendor/jar-dependencies/runtime-jars/protobuf-java-2.6.1.jar
125
- homepage: https://github.com/codekitchen/logstash-input-kinesis
147
+ - logstash-input-kinesis.gemspec
148
+ - README.md
149
+ - CONTRIBUTORS
150
+ - Gemfile
151
+ homepage: https://github.com/logstash-plugins/logstash-input-kinesis
126
152
  licenses:
127
153
  - Apache License (2.0)
128
154
  metadata:
@@ -142,13 +168,15 @@ required_rubygems_version: !ruby/object:Gem::Requirement
142
168
  - - '>='
143
169
  - !ruby/object:Gem::Version
144
170
  version: '0'
145
- requirements: []
171
+ requirements:
172
+ - jar 'com.amazonaws:amazon-kinesis-client', '1.6.1'
173
+ - jar 'org.apache.httpcomponents:httpclient', '4.5.1'
146
174
  rubyforge_project:
147
175
  rubygems_version: 2.1.9
148
176
  signing_key:
149
177
  specification_version: 4
150
178
  summary: Logstash plugin for Kinesis input
151
179
  test_files:
152
- - spec/inputs/kinesis/worker_spec.rb
153
- - spec/inputs/kinesis_spec.rb
154
180
  - spec/spec_helper.rb
181
+ - spec/inputs/kinesis_spec.rb
182
+ - spec/inputs/kinesis/worker_spec.rb
data/.gitignore DELETED
@@ -1,14 +0,0 @@
1
- /.bundle/
2
- /.yardoc
3
- /Gemfile.lock
4
- /_yardoc/
5
- /coverage/
6
- /doc/
7
- /pkg/
8
- /spec/reports/
9
- /tmp/
10
- *.bundle
11
- *.so
12
- *.o
13
- *.a
14
- mkmf.log
data/.rspec DELETED
@@ -1,2 +0,0 @@
1
- --color
2
- --require spec_helper
@@ -1,3 +0,0 @@
1
- language: ruby
2
- rvm:
3
- - jruby-19mode
@@ -1,14 +0,0 @@
1
- Copyright (c) 2015 Brian Palmer
2
-
3
- Licensed under the Apache License, Version 2.0 (the "License");
4
- you may not use this file except in compliance with the License.
5
- You may obtain a copy of the License at
6
-
7
- http://www.apache.org/licenses/LICENSE-2.0
8
-
9
- Unless required by applicable law or agreed to in writing, software
10
- distributed under the License is distributed on an "AS IS" BASIS,
11
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
- See the License for the specific language governing permissions and
13
- limitations under the License.
14
-
data/Rakefile DELETED
@@ -1,15 +0,0 @@
1
- require "bundler/gem_tasks"
2
-
3
- begin
4
- require 'rspec/core/rake_task'
5
- RSpec::Core::RakeTask.new(:spec)
6
- rescue LoadError
7
- end
8
-
9
- task default: "spec"
10
-
11
- task "vendor_jars" do
12
- sh "mvn dependency:copy-dependencies -DoutputDirectory=vendor/jar-dependencies/runtime-jars/"
13
- end
14
-
15
- task build: "vendor_jars"
data/pom.xml DELETED
@@ -1,32 +0,0 @@
1
- <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
2
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3
- <modelVersion>4.0.0</modelVersion>
4
- <groupId>codekitchen</groupId>
5
- <artifactId>logstash-input-kinesis</artifactId>
6
- <version>dev</version>
7
- <dependencies>
8
- <dependency>
9
- <groupId>org.apache.httpcomponents</groupId>
10
- <artifactId>httpclient</artifactId>
11
- <version>4.5.1</version>
12
- </dependency>
13
- <dependency>
14
- <groupId>com.amazonaws</groupId>
15
- <artifactId>amazon-kinesis-client</artifactId>
16
- <version>1.6.1</version>
17
- </dependency>
18
- </dependencies>
19
- <build>
20
- <plugins>
21
- <plugin>
22
- <groupId>org.apache.maven.plugins</groupId>
23
- <artifactId>maven-dependency-plugin</artifactId>
24
- <executions>
25
- <execution>
26
- <id>copy-dependencies</id>
27
- </execution>
28
- </executions>
29
- </plugin>
30
- </plugins>
31
- </build>
32
- </project>