logstash-input-azurewadeventhub 0.9.6-java → 0.9.7-java

Sign up to get free protection for your applications and to get access to all the features.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +7 -0
  3. data/Gemfile +2 -1
  4. data/README.md +53 -64
  5. data/lib/logstash/inputs/azurewadeventhub.rb +7 -6
  6. data/logstash-input-azurewadeventhub.gemspec +5 -4
  7. metadata +22 -8
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: dbfc3cd9d2beff615f1c7b120b1bc716b365b27f
4
- data.tar.gz: ee15cf028e3586a403addb0e06e6e5af2c445f26
3
+ metadata.gz: 512bc2bf7898872779fbf253b3d6ff7591847712
4
+ data.tar.gz: 52ea8d8c44bfd5a1bc5024f7d4a94ffbc871af2f
5
5
  SHA512:
6
- metadata.gz: 8929852f3a860c1e4daefad0f21bec315598ab5b52d0dffe6170c63c5d5fb236618c05f1c28713e6121cbbb103b1d15149169c9b9aa844c95074cfcfb0436f1d
7
- data.tar.gz: bbe3799b054d0af51f5df6436e81f60394451d67a66bf4f5fb2fd3ae0d031285566c4bf59f960515dd313f95da3205760612fc4049130608480d8fadcdb18a13
6
+ metadata.gz: dde5ba1ada8108dbc2d55d8533add703e5463e2dac9104902f9414cfcce067e13b4b5c8851c3724756fd4f6a3fe5875477432205e2f5f427fa74bf12e6aff483
7
+ data.tar.gz: 5fc56bd9c6db0feb76e3afa1b983a71c41b14a3c7d5bca3b3b17b5e54775f4cbb121065db418fa30b8467534d3f628d852828a9c55de59751c5f9be476bf5185
data/CHANGELOG.md CHANGED
@@ -0,0 +1,7 @@
1
+ ## 2016.04.28
2
+ * Fixed the jar dependency problem. Now installing the plugin by using the "plugin" or "logstash-plugin" (for newer versions of Logstash) should automatically download and jar files needed.
3
+ * Added missing json runtime dependency.
4
+ * Fixed the default value of *time_since_epoch_millis* to use UTC time to match the SelectorFilter.
5
+ * Made the plugin to respect Logstash shutdown signal.
6
+ * Updated the *logstash-core* runtime dependency requirement to '~> 2.0'.
7
+ * Updated the *logstash-devutils* development dependency requirement to '>= 0.0.16'
data/Gemfile CHANGED
@@ -2,4 +2,5 @@ source 'https://rubygems.org'
2
2
  gemspec
3
3
  gem 'azure', '~> 0.7.1'
4
4
  gem 'jar-dependencies', '~> 0.3.2'
5
- gem 'logstash-core', '>= 1.4.0'
5
+ gem 'logstash-core', '~> 2.0'
6
+ gem 'json', '~> 1.8.3'
data/README.md CHANGED
@@ -1,91 +1,80 @@
1
- # Notice
2
- This plugin is a part of [Microsoft Azure Diagnostics with ELK](https://github.com/Azure/azure-diagnostics-tools).
1
+ # Logstash input plugin for Azure diagnostics data from Event Hubs
3
2
 
4
- [See more documentation.](https://github.com/mspnp/semantic-logging/blob/v3/ELK/md/LogstashExtensions.md#azure-wad-table)
3
+ ## Summary
4
+ This plugin reads Azure diagnostics data from specified Azure Event Hubs and parses the data for output.
5
5
 
6
- # Logstash Plugin
6
+ ## Installation
7
+ You can install this plugin using the Logstash "plugin" or "logstash-plugin" (for newer versions of Logstash) command:
8
+ ```sh
9
+ logstash-plugin install logstash-input-azurewadeventhub
10
+ ```
11
+ For more information, see Logstash reference [Working with plugins](https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html).
7
12
 
8
- This is a plugin for [Logstash](https://github.com/elasticsearch/logstash).
13
+ ## Configuration
14
+ ### Required Parameters
15
+ *key*
9
16
 
10
- It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
17
+ The shared access key to the target event hub.
11
18
 
12
- ## Documentation
19
+ *username*
13
20
 
14
- Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one [central location](http://www.elasticsearch.org/guide/en/logstash/current/).
21
+ The name of the shared access policy.
15
22
 
16
- - For formatting code or config example, you can use the asciidoc `[source,ruby]` directive
17
- - For more asciidoc formatting tips, see the excellent reference here https://github.com/elasticsearch/docs#asciidoc-guide
23
+ *namespace*
18
24
 
19
- ## Need Help?
25
+ Event Hub namespace.
20
26
 
21
- Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.
27
+ *eventhub*
22
28
 
23
- ## Developing
29
+ Event Hub name.
24
30
 
25
- ### 1. Plugin Developement and Testing
31
+ *partitions*
26
32
 
27
- #### Code
28
- - To get started, you'll need JRuby with the Bundler gem installed.
33
+ Partition count of the target event hub.
29
34
 
30
- - Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
35
+ ### Optional Parameters
36
+ *domain*
31
37
 
32
- - Install dependencies
33
- ```sh
34
- bundle install
35
- ```
38
+ Domain of the target Event Hub. Default value is "servicebus.windows.net".
36
39
 
37
- #### Test
40
+ *port*
38
41
 
39
- - Update your dependencies
42
+ Port of the target Event Hub. Default value is 5671.
40
43
 
41
- ```sh
42
- bundle install
43
- ```
44
+ *receive_credits*
44
45
 
45
- - Run tests
46
+ The credit number to limit the number of messages to receive in a processing cycle. Default value is 1000.
46
47
 
47
- ```sh
48
- bundle exec rspec
49
- ```
48
+ *consumer_group*
50
49
 
51
- ### 2. Running your unpublished Plugin in Logstash
50
+ Name of the consumer group. Default value is "$default".
52
51
 
53
- #### 2.1 Run in a local Logstash clone
52
+ *time_since_epoch_millis*
54
53
 
55
- - Edit Logstash `Gemfile` and add the local plugin path, for example:
54
+ Specifies the point of time after which the messages are received. Default value is the time when this plugin is initialized:
56
55
  ```ruby
57
- gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"
56
+ Time.now.utc.to_i * 1000
58
57
  ```
59
- - Install plugin
60
- ```sh
61
- bin/plugin install --no-verify
62
- ```
63
- - Run Logstash with your plugin
64
- ```sh
65
- bin/logstash -e 'filter {awesome {}}'
58
+ *thread_wait_sec*
59
+
60
+ Specifies the time (in seconds) to wait before another try if no message was received.
61
+
62
+ ### Examples
63
+ ```json
64
+ input
65
+ {
66
+ azurewadeventhub
67
+ {
68
+ key => "VGhpcyBpcyBhIGZha2Uga2V5Lg=="
69
+ username => "receivepolicy"
70
+ namespace => "mysbns"
71
+ eventhub => "myeventhub"
72
+ partitions => 4
73
+ }
74
+ }
66
75
  ```
67
- At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
68
-
69
- #### 2.2 Run in an installed Logstash
70
-
71
- You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
72
-
73
- - Build your plugin gem
74
- ```sh
75
- gem build logstash-filter-awesome.gemspec
76
- ```
77
- - Install the plugin from the Logstash home
78
- ```sh
79
- bin/plugin install /your/local/plugin/logstash-filter-awesome.gem
80
- ```
81
- - Start Logstash and proceed to test the plugin
82
-
83
- ## Contributing
84
-
85
- All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
86
-
87
- Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.
88
76
 
89
- It is more important to the community that you are able to contribute.
77
+ ## More information
78
+ The source code of this plugin is hosted in GitHub repo [Microsoft Azure Diagnostics with ELK](https://github.com/Azure/azure-diagnostics-tools). We welcome you to provide feedback and/or contribute to the project.
90
79
 
91
- For more information about contributing, see the [CONTRIBUTING](https://github.com/elasticsearch/logstash/blob/master/CONTRIBUTING.md) file.
80
+ Please also see [Analyze Diagnostics Data with ELK template](https://github.com/Azure/azure-quickstart-templates/tree/master/diagnostics-with-elk) for quick deployment of ELK to Azure.
data/lib/logstash/inputs/azurewadeventhub.rb CHANGED
@@ -5,6 +5,7 @@ require "logstash/namespace"
5
5
  require "securerandom"
6
6
  require "open-uri"
7
7
  require "thread"
8
+ require "json"
8
9
 
9
10
  require Dir[ File.dirname(__FILE__) + "/../../*_jars.rb" ].first
10
11
 
@@ -12,10 +13,10 @@ require Dir[ File.dirname(__FILE__) + "/../../*_jars.rb" ].first
12
13
  class LogStash::Inputs::Azurewadeventhub < LogStash::Inputs::Base
13
14
 
14
15
  config_name "azurewadeventhub"
15
- milestone 0
16
+ milestone 1
16
17
 
17
18
  default :codec, "json"
18
-
19
+
19
20
  config :key, :validate => :string
20
21
  config :username, :validate => :string
21
22
  config :namespace, :validate => :string
@@ -27,7 +28,7 @@ class LogStash::Inputs::Azurewadeventhub < LogStash::Inputs::Base
27
28
  config :partitions, :validate => :number
28
29
  config :consumer_group, :validate => :string, :default => "$default"
29
30
 
30
- config :time_since_epoch_millis, :validate => :number, :default => Time.now.to_i * 1000
31
+ config :time_since_epoch_millis, :validate => :number, :default => Time.now.utc.to_i * 1000
31
32
  config :thread_wait_sec, :validate => :number, :default => 5
32
33
 
33
34
 
@@ -74,7 +75,7 @@ class LogStash::Inputs::Azurewadeventhub < LogStash::Inputs::Base
74
75
  end
75
76
 
76
77
  def process(output_queue, receiver, partition)
77
- while true
78
+ while !stop?
78
79
  begin
79
80
  msg = receiver.receive(10)
80
81
  if msg
@@ -98,7 +99,7 @@ class LogStash::Inputs::Azurewadeventhub < LogStash::Inputs::Base
98
99
  end # process
99
100
 
100
101
  def process_partition(output_queue, partition)
101
- while true
102
+ while !stop?
102
103
  begin
103
104
  filter = SelectorFilter.new "amqp.annotation.x-opt-enqueuedtimeutc > '" + @time_since_epoch_millis.to_s + "'"
104
105
  filters = { org::apache::qpid::amqp_1_0::type::Symbol.valueOf("apache.org:selector-filter:string") => filter }
@@ -111,7 +112,7 @@ class LogStash::Inputs::Azurewadeventhub < LogStash::Inputs::Base
111
112
  process(output_queue,receiver,partition)
112
113
  rescue org::apache::qpid::amqp_1_0::client::ConnectionErrorException => e
113
114
  @logger.debug(" " + partition.to_s.rjust(2,"0") + " --- " + "resetting connection")
114
- @time_since_epoch_millis = Time.now.to_i * 1000
115
+ @time_since_epoch_millis = Time.now.utc.to_i * 1000
115
116
  end
116
117
  end
117
118
  rescue LogStash::ShutdownSignal => e
data/logstash-input-azurewadeventhub.gemspec CHANGED
@@ -1,10 +1,10 @@
1
1
  Gem::Specification.new do |s|
2
2
  s.name = 'logstash-input-azurewadeventhub'
3
- s.version = '0.9.6'
3
+ s.version = '0.9.7'
4
4
  s.platform = "java"
5
5
  s.licenses = ['Apache License (2.0)']
6
6
  s.summary = "This plugin will collect Microsoft Azure Diagnostics data from Azure Event Hubs."
7
- s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program."
7
+ s.description = "This gem is a Logstash plugin. It reads and parses diagnostics data from Azure Event Hubs."
8
8
  s.authors = ["Microsoft Corporation"]
9
9
  s.email = 'azdiag@microsoft.com'
10
10
  s.homepage = "https://github.com/Azure/azure-diagnostics-tools"
@@ -19,9 +19,10 @@ Gem::Specification.new do |s|
19
19
  s.metadata = { "logstash_plugin" => "true", "logstash_group" => "input" }
20
20
 
21
21
  # Gem dependencies
22
- s.add_runtime_dependency 'logstash-core', '>= 1.4.0'
22
+ s.add_runtime_dependency 'logstash-core', '~> 2.0'
23
23
  s.add_runtime_dependency 'azure', '~> 0.7.1'
24
- s.add_development_dependency 'logstash-devutils'
24
+ s.add_runtime_dependency 'json', '~> 1.8.3'
25
+ s.add_development_dependency 'logstash-devutils', '>= 0.0.16'
25
26
 
26
27
  #Jar dependencies
27
28
  s.requirements << "jar 'org.apache.qpid:qpid-amqp-1-0-common', '0.32'"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-input-azurewadeventhub
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.9.6
4
+ version: 0.9.7
5
5
  platform: java
6
6
  authors:
7
7
  - Microsoft Corporation
@@ -13,17 +13,17 @@ dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  requirement: !ruby/object:Gem::Requirement
15
15
  requirements:
16
- - - '>='
16
+ - - ~>
17
17
  - !ruby/object:Gem::Version
18
- version: 1.4.0
18
+ version: '2.0'
19
19
  name: logstash-core
20
20
  prerelease: false
21
21
  type: :runtime
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
- - - '>='
24
+ - - ~>
25
25
  - !ruby/object:Gem::Version
26
- version: 1.4.0
26
+ version: '2.0'
27
27
  - !ruby/object:Gem::Dependency
28
28
  requirement: !ruby/object:Gem::Requirement
29
29
  requirements:
@@ -38,12 +38,26 @@ dependencies:
38
38
  - - ~>
39
39
  - !ruby/object:Gem::Version
40
40
  version: 0.7.1
41
+ - !ruby/object:Gem::Dependency
42
+ requirement: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - ~>
45
+ - !ruby/object:Gem::Version
46
+ version: 1.8.3
47
+ name: json
48
+ prerelease: false
49
+ type: :runtime
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ~>
53
+ - !ruby/object:Gem::Version
54
+ version: 1.8.3
41
55
  - !ruby/object:Gem::Dependency
42
56
  requirement: !ruby/object:Gem::Requirement
43
57
  requirements:
44
58
  - - '>='
45
59
  - !ruby/object:Gem::Version
46
- version: '0'
60
+ version: 0.0.16
47
61
  name: logstash-devutils
48
62
  prerelease: false
49
63
  type: :development
@@ -51,7 +65,7 @@ dependencies:
51
65
  requirements:
52
66
  - - '>='
53
67
  - !ruby/object:Gem::Version
54
- version: '0'
68
+ version: 0.0.16
55
69
  - !ruby/object:Gem::Dependency
56
70
  requirement: !ruby/object:Gem::Requirement
57
71
  requirements:
@@ -66,7 +80,7 @@ dependencies:
66
80
  - - ~>
67
81
  - !ruby/object:Gem::Version
68
82
  version: 0.3.2
69
- description: This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program.
83
+ description: This gem is a Logstash plugin. It reads and parses diagnostics data from Azure Event Hubs.
70
84
  email: azdiag@microsoft.com
71
85
  executables: []
72
86
  extensions: []