logstash-input-kinesis 1.4.3-java → 1.5.0-java

Sign up to get free protection for your applications and to get access to all the features.
Files changed (34) hide show
  1. checksums.yaml +4 -4
  2. data/CONTRIBUTORS +14 -0
  3. data/README.md +5 -4
  4. data/lib/com/amazonaws/amazon-kinesis-client/1.6.1/amazon-kinesis-client-1.6.1.jar +0 -0
  5. data/lib/com/amazonaws/aws-java-sdk-cloudwatch/1.10.20/aws-java-sdk-cloudwatch-1.10.20.jar +0 -0
  6. data/{vendor/jar-dependencies/runtime-jars/aws-java-sdk-core-1.10.50.jar → lib/com/amazonaws/aws-java-sdk-core/1.10.20/aws-java-sdk-core-1.10.20.jar} +0 -0
  7. data/lib/com/amazonaws/aws-java-sdk-dynamodb/1.10.20/aws-java-sdk-dynamodb-1.10.20.jar +0 -0
  8. data/lib/com/amazonaws/aws-java-sdk-kinesis/1.10.20/aws-java-sdk-kinesis-1.10.20.jar +0 -0
  9. data/lib/com/amazonaws/aws-java-sdk-kms/1.10.20/aws-java-sdk-kms-1.10.20.jar +0 -0
  10. data/lib/com/amazonaws/aws-java-sdk-s3/1.10.20/aws-java-sdk-s3-1.10.20.jar +0 -0
  11. data/lib/com/fasterxml/jackson/core/jackson-annotations/2.5.0/jackson-annotations-2.5.0.jar +0 -0
  12. data/lib/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar +0 -0
  13. data/lib/com/fasterxml/jackson/core/jackson-databind/2.5.3/jackson-databind-2.5.3.jar +0 -0
  14. data/lib/com/google/guava/guava/18.0/guava-18.0.jar +0 -0
  15. data/lib/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar +0 -0
  16. data/lib/commons-codec/commons-codec/1.9/commons-codec-1.9.jar +0 -0
  17. data/lib/commons-lang/commons-lang/2.6/commons-lang-2.6.jar +0 -0
  18. data/lib/commons-logging/commons-logging/1.2/commons-logging-1.2.jar +0 -0
  19. data/lib/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar +0 -0
  20. data/lib/logstash/inputs/kinesis.rb +1 -1
  21. data/lib/logstash/inputs/kinesis/version.rb +2 -1
  22. data/lib/logstash/inputs/kinesis/worker.rb +1 -0
  23. data/lib/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar +0 -0
  24. data/lib/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar +0 -0
  25. data/logstash-input-kinesis.gemspec +8 -3
  26. data/spec/inputs/kinesis/worker_spec.rb +1 -0
  27. data/spec/inputs/kinesis_spec.rb +12 -1
  28. metadata +50 -22
  29. data/.gitignore +0 -14
  30. data/.rspec +0 -2
  31. data/.travis.yml +0 -3
  32. data/LICENSE.txt +0 -14
  33. data/Rakefile +0 -15
  34. data/pom.xml +0 -32
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: e2e3ea8c6324844f371148a1e1bf7743e0bb5cfe
4
- data.tar.gz: 7be846fcd88272b49885cf60b6256b46a3c8b68f
3
+ metadata.gz: 03bbacd8aa1c1ead7454666582fa2ec84cd79d86
4
+ data.tar.gz: ae48dab34705083645faff5c7b7c3674ef658a8e
5
5
  SHA512:
6
- metadata.gz: b1d06f224b074e9db2a8df3f0720610d416109ed39da7ab2bfff171453bbc5f5e11d740b008463d4f09f2d9bd5e88c721d49d813fe714b623c2d83855c959f38
7
- data.tar.gz: 20a09f5d185eb7a14f30fdf07d93256df689b6d25fd00284ee24cff4e08eee55a50cf89bec92d08f244369652293c3bf81510f75cb82c78b3ecc472ae4492c5e
6
+ metadata.gz: 04342e5d1a6e34138995d89cb894b732997d0d1b51378e46fd4db49823c7b0d8707fbe9bdbbb647b527240372a0ff59360f558866092cff849e91fca8a8b8fda
7
+ data.tar.gz: 86232000df97f16996c078caf62c6cd35f676c411ce1901966d4edb8da372267867a65739a75ab3a40267e4b6ac3c607702a8bdc69fa9e9e1288c4ca5a26b159
@@ -0,0 +1,14 @@
1
+ The following is a list of people who have contributed ideas, code, bug
2
+ reports, or in general have helped logstash along its way.
3
+
4
+ Maintainers:
5
+ * Brian Palmer (codekitchen)
6
+
7
+ Contributors:
8
+ * Brian Palmer (codekitchen)
9
+ * Samuel García Martínez (samuelgmartinez)
10
+
11
+ Note: If you've sent us patches, bug reports, or otherwise contributed to
12
+ Logstash, and you aren't on the list above and want to be, please let us know
13
+ and we'll make sure you're here. Contributions from folks like you are what make
14
+ open source awesome.
data/README.md CHANGED
@@ -1,12 +1,12 @@
1
1
  # Logstash AWS Kinesis Input Plugin
2
2
 
3
- [![Build Status](https://travis-ci.org/codekitchen/logstash-input-kinesis.svg)](https://travis-ci.org/codekitchen/logstash-input-kinesis)
3
+ [![Build Status](https://travis-ci.org/logstash-plugins/logstash-input-kinesis.svg)](https://travis-ci.org/logstash-plugins/logstash-input-kinesis)
4
4
 
5
5
  This is a [AWS Kinesis](http://docs.aws.amazon.com/kinesis/latest/dev/introduction.html) input plugin for [Logstash](https://github.com/elasticsearch/logstash). Under the hood uses the [Kinesis Client Library](http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-record-processor-implementation-app-java.html).
6
6
 
7
7
  ## Installation
8
8
 
9
- This plugin requires Logstash >= 1.5, and can be installed by Logstash
9
+ This plugin requires Logstash >= 2.0, and can be installed by Logstash
10
10
  itself.
11
11
 
12
12
  ```sh
@@ -39,7 +39,7 @@ This are the properties you can configure and what are the default values:
39
39
  * `checkpoint_interval_seconds`: How many seconds between worker checkpoints to DynamoDB. A low value ussually means lower message replay in case of node failure/restart but it increases CPU+network ussage (which increases the AWS costs).
40
40
  * **required**: false
41
41
  * **default value**: `60`
42
- * `metrics`: Worker metric tracking. By default this is disabled, set it to "cloudwatch" to enable the cloudwatch integration in the Kinesis Client Library.
42
+ * `metrics`: Worker metric tracking. By default this is disabled, set it to "cloudwatch" to enable the cloudwatch integration in the Kinesis Client Library.
43
43
  * **required**: false
44
44
  * **default value**: `nil`
45
45
 
@@ -61,7 +61,8 @@ Look at the [documentation](https://docs.aws.amazon.com/AWSJavaSDK/latest/javado
61
61
 
62
62
  ## Contributing
63
63
 
64
- 1. Fork it ( https://github.com/codekitchen/logstash-input-kinesis/fork )
64
+ 0. https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md#contribution-steps
65
+ 1. Fork it ( https://github.com/logstash-plugins/logstash-input-kinesis/fork )
65
66
  2. Create your feature branch (`git checkout -b my-new-feature`)
66
67
  3. Commit your changes (`git commit -am 'Add some feature'`)
67
68
  4. Push to the branch (`git push origin my-new-feature`)
@@ -85,7 +85,7 @@ class LogStash::Inputs::Kinesis < LogStash::Inputs::Base
85
85
  end
86
86
  end
87
87
 
88
- def teardown
88
+ def stop
89
89
  @kcl_worker.shutdown if @kcl_worker
90
90
  end
91
91
 
@@ -1,7 +1,8 @@
1
+ # encoding: utf-8
1
2
  module Logstash
2
3
  module Input
3
4
  module Kinesis
4
- VERSION = "1.4.3"
5
+ VERSION = "1.5.0"
5
6
  end
6
7
  end
7
8
  end
@@ -1,3 +1,4 @@
1
+ # encoding: utf-8
1
2
  class LogStash::Inputs::Kinesis::Worker
2
3
  include com.amazonaws.services.kinesis.clientlibrary.interfaces.v2::IRecordProcessor
3
4
 
@@ -10,10 +10,10 @@ Gem::Specification.new do |spec|
10
10
  spec.email = ["brian@codekitchen.net"]
11
11
  spec.summary = %q{Logstash plugin for Kinesis input}
12
12
  spec.description = %q{This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program}
13
- spec.homepage = "https://github.com/codekitchen/logstash-input-kinesis"
13
+ spec.homepage = "https://github.com/logstash-plugins/logstash-input-kinesis"
14
14
  spec.licenses = ['Apache License (2.0)']
15
15
 
16
- spec.files = `git ls-files`.split($\)
16
+ spec.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
17
17
  spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
18
  spec.require_paths = ["lib"]
19
19
 
@@ -22,9 +22,14 @@ Gem::Specification.new do |spec|
22
22
 
23
23
  spec.platform = 'java'
24
24
 
25
+ spec.requirements << "jar 'com.amazonaws:amazon-kinesis-client', '1.6.1'"
26
+ spec.requirements << "jar 'org.apache.httpcomponents:httpclient', '4.5.1'"
27
+
28
+ spec.add_development_dependency 'jar-dependencies', '0.3.2'
29
+
25
30
  spec.add_development_dependency "bundler", "~> 1.7"
26
31
  spec.add_development_dependency "rake", "~> 10.0"
27
32
  spec.add_development_dependency "rspec", "~> 3.2.0"
28
- spec.add_development_dependency "logstash-core", ">= 1.5.1"
33
+ spec.add_development_dependency "logstash-core", "~> 2.4.0"
29
34
  spec.add_development_dependency "logstash-codec-json"
30
35
  end
@@ -48,6 +48,7 @@ RSpec.describe "LogStash::Inputs::Kinesis::Worker" do
48
48
  describe "#processRecords" do
49
49
  it "decodes and queues each record with decoration" do
50
50
  worker.processRecords(process_input)
51
+ expect(queue.size).to eq(2)
51
52
  m1 = queue.pop
52
53
  m2 = queue.pop
53
54
  expect(m1).to be_kind_of(LogStash::Event)
@@ -43,11 +43,22 @@ RSpec.describe "inputs/kinesis" do
43
43
  end
44
44
  end
45
45
 
46
+ context "#stop" do
47
+ it "stops the KCL worker" do
48
+ expect(kinesis).to receive(:kcl_builder).with(queue).and_return(stub_builder)
49
+ expect(kcl_worker).to receive(:run).with(no_args)
50
+ expect(kcl_worker).to receive(:shutdown).with(no_args)
51
+ kinesis.run(queue)
52
+ kinesis.do_stop # do_stop calls stop internally
53
+ end
54
+ end
55
+
46
56
  context "#worker_factory" do
47
57
  it "clones the codec for each worker" do
48
- expect(codec).to receive(:clone).once
49
58
  worker = kinesis.worker_factory(queue).call()
50
59
  expect(worker).to be_kind_of(LogStash::Inputs::Kinesis::Worker)
60
+ expect(worker.codec).to_not eq(kinesis.codec)
61
+ expect(worker.codec).to be_kind_of(codec.class)
51
62
  end
52
63
 
53
64
  it "generates a valid worker" do
metadata CHANGED
@@ -1,15 +1,29 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-kinesis
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.4.3
4
+ version: 1.5.0
5
5
  platform: java
6
6
  authors:
7
7
  - Brian Palmer
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-02-08 00:00:00.000000000 Z
11
+ date: 2016-09-29 00:00:00.000000000 Z
12
12
  dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - '='
17
+ - !ruby/object:Gem::Version
18
+ version: 0.3.2
19
+ name: jar-dependencies
20
+ prerelease: false
21
+ type: :development
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - '='
25
+ - !ruby/object:Gem::Version
26
+ version: 0.3.2
13
27
  - !ruby/object:Gem::Dependency
14
28
  requirement: !ruby/object:Gem::Requirement
15
29
  requirements:
@@ -55,17 +69,17 @@ dependencies:
55
69
  - !ruby/object:Gem::Dependency
56
70
  requirement: !ruby/object:Gem::Requirement
57
71
  requirements:
58
- - - '>='
72
+ - - ~>
59
73
  - !ruby/object:Gem::Version
60
- version: 1.5.1
74
+ version: 2.4.0
61
75
  name: logstash-core
62
76
  prerelease: false
63
77
  type: :development
64
78
  version_requirements: !ruby/object:Gem::Requirement
65
79
  requirements:
66
- - - '>='
80
+ - - ~>
67
81
  - !ruby/object:Gem::Version
68
- version: 1.5.1
82
+ version: 2.4.0
69
83
  - !ruby/object:Gem::Dependency
70
84
  requirement: !ruby/object:Gem::Requirement
71
85
  requirements:
@@ -87,26 +101,34 @@ executables: []
87
101
  extensions: []
88
102
  extra_rdoc_files: []
89
103
  files:
90
- - .gitignore
91
- - .rspec
92
- - .travis.yml
93
- - Gemfile
94
- - LICENSE.txt
95
- - README.md
96
- - Rakefile
97
104
  - lib/logstash-input-kinesis_jars.rb
105
+ - lib/com/amazonaws/amazon-kinesis-client/1.6.1/amazon-kinesis-client-1.6.1.jar
106
+ - lib/com/amazonaws/aws-java-sdk-cloudwatch/1.10.20/aws-java-sdk-cloudwatch-1.10.20.jar
107
+ - lib/com/amazonaws/aws-java-sdk-core/1.10.20/aws-java-sdk-core-1.10.20.jar
108
+ - lib/com/amazonaws/aws-java-sdk-dynamodb/1.10.20/aws-java-sdk-dynamodb-1.10.20.jar
109
+ - lib/com/amazonaws/aws-java-sdk-kinesis/1.10.20/aws-java-sdk-kinesis-1.10.20.jar
110
+ - lib/com/amazonaws/aws-java-sdk-kms/1.10.20/aws-java-sdk-kms-1.10.20.jar
111
+ - lib/com/amazonaws/aws-java-sdk-s3/1.10.20/aws-java-sdk-s3-1.10.20.jar
112
+ - lib/com/fasterxml/jackson/core/jackson-annotations/2.5.0/jackson-annotations-2.5.0.jar
113
+ - lib/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar
114
+ - lib/com/fasterxml/jackson/core/jackson-databind/2.5.3/jackson-databind-2.5.3.jar
115
+ - lib/com/google/guava/guava/18.0/guava-18.0.jar
116
+ - lib/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar
117
+ - lib/commons-codec/commons-codec/1.9/commons-codec-1.9.jar
118
+ - lib/commons-lang/commons-lang/2.6/commons-lang-2.6.jar
119
+ - lib/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
120
+ - lib/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar
98
121
  - lib/logstash/inputs/kinesis.rb
99
122
  - lib/logstash/inputs/kinesis/version.rb
100
123
  - lib/logstash/inputs/kinesis/worker.rb
101
- - logstash-input-kinesis.gemspec
102
- - pom.xml
103
- - spec/inputs/kinesis/worker_spec.rb
104
- - spec/inputs/kinesis_spec.rb
124
+ - lib/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar
125
+ - lib/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar
105
126
  - spec/spec_helper.rb
127
+ - spec/inputs/kinesis_spec.rb
128
+ - spec/inputs/kinesis/worker_spec.rb
106
129
  - vendor/jar-dependencies/runtime-jars/amazon-kinesis-client-1.6.1.jar
107
130
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-cloudwatch-1.10.20.jar
108
131
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-core-1.10.20.jar
109
- - vendor/jar-dependencies/runtime-jars/aws-java-sdk-core-1.10.50.jar
110
132
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-dynamodb-1.10.20.jar
111
133
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-kinesis-1.10.20.jar
112
134
  - vendor/jar-dependencies/runtime-jars/aws-java-sdk-kms-1.10.20.jar
@@ -122,7 +144,11 @@ files:
122
144
  - vendor/jar-dependencies/runtime-jars/jackson-databind-2.5.3.jar
123
145
  - vendor/jar-dependencies/runtime-jars/joda-time-2.8.1.jar
124
146
  - vendor/jar-dependencies/runtime-jars/protobuf-java-2.6.1.jar
125
- homepage: https://github.com/codekitchen/logstash-input-kinesis
147
+ - logstash-input-kinesis.gemspec
148
+ - README.md
149
+ - CONTRIBUTORS
150
+ - Gemfile
151
+ homepage: https://github.com/logstash-plugins/logstash-input-kinesis
126
152
  licenses:
127
153
  - Apache License (2.0)
128
154
  metadata:
@@ -142,13 +168,15 @@ required_rubygems_version: !ruby/object:Gem::Requirement
142
168
  - - '>='
143
169
  - !ruby/object:Gem::Version
144
170
  version: '0'
145
- requirements: []
171
+ requirements:
172
+ - jar 'com.amazonaws:amazon-kinesis-client', '1.6.1'
173
+ - jar 'org.apache.httpcomponents:httpclient', '4.5.1'
146
174
  rubyforge_project:
147
175
  rubygems_version: 2.1.9
148
176
  signing_key:
149
177
  specification_version: 4
150
178
  summary: Logstash plugin for Kinesis input
151
179
  test_files:
152
- - spec/inputs/kinesis/worker_spec.rb
153
- - spec/inputs/kinesis_spec.rb
154
180
  - spec/spec_helper.rb
181
+ - spec/inputs/kinesis_spec.rb
182
+ - spec/inputs/kinesis/worker_spec.rb
data/.gitignore DELETED
@@ -1,14 +0,0 @@
1
- /.bundle/
2
- /.yardoc
3
- /Gemfile.lock
4
- /_yardoc/
5
- /coverage/
6
- /doc/
7
- /pkg/
8
- /spec/reports/
9
- /tmp/
10
- *.bundle
11
- *.so
12
- *.o
13
- *.a
14
- mkmf.log
data/.rspec DELETED
@@ -1,2 +0,0 @@
1
- --color
2
- --require spec_helper
@@ -1,3 +0,0 @@
1
- language: ruby
2
- rvm:
3
- - jruby-19mode
@@ -1,14 +0,0 @@
1
- Copyright (c) 2015 Brian Palmer
2
-
3
- Licensed under the Apache License, Version 2.0 (the "License");
4
- you may not use this file except in compliance with the License.
5
- You may obtain a copy of the License at
6
-
7
- http://www.apache.org/licenses/LICENSE-2.0
8
-
9
- Unless required by applicable law or agreed to in writing, software
10
- distributed under the License is distributed on an "AS IS" BASIS,
11
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
- See the License for the specific language governing permissions and
13
- limitations under the License.
14
-
data/Rakefile DELETED
@@ -1,15 +0,0 @@
1
- require "bundler/gem_tasks"
2
-
3
- begin
4
- require 'rspec/core/rake_task'
5
- RSpec::Core::RakeTask.new(:spec)
6
- rescue LoadError
7
- end
8
-
9
- task default: "spec"
10
-
11
- task "vendor_jars" do
12
- sh "mvn dependency:copy-dependencies -DoutputDirectory=vendor/jar-dependencies/runtime-jars/"
13
- end
14
-
15
- task build: "vendor_jars"
data/pom.xml DELETED
@@ -1,32 +0,0 @@
1
- <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
2
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3
- <modelVersion>4.0.0</modelVersion>
4
- <groupId>codekitchen</groupId>
5
- <artifactId>logstash-input-kinesis</artifactId>
6
- <version>dev</version>
7
- <dependencies>
8
- <dependency>
9
- <groupId>org.apache.httpcomponents</groupId>
10
- <artifactId>httpclient</artifactId>
11
- <version>4.5.1</version>
12
- </dependency>
13
- <dependency>
14
- <groupId>com.amazonaws</groupId>
15
- <artifactId>amazon-kinesis-client</artifactId>
16
- <version>1.6.1</version>
17
- </dependency>
18
- </dependencies>
19
- <build>
20
- <plugins>
21
- <plugin>
22
- <groupId>org.apache.maven.plugins</groupId>
23
- <artifactId>maven-dependency-plugin</artifactId>
24
- <executions>
25
- <execution>
26
- <id>copy-dependencies</id>
27
- </execution>
28
- </executions>
29
- </plugin>
30
- </plugins>
31
- </build>
32
- </project>