sentinelblue-logstash-output-azure-loganalytics 1.1.1 → 1.1.2.rc1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 41e20d6292a1fe6e9b70ce0860be3f92dd47b6a90ddbb6d5ee87f7335fb30270
4
- data.tar.gz: bbfd6a8f894905f0f2e2e04fdcb3daa034f04047c5f847e26817670ee8cf58f3
3
+ metadata.gz: 7ba5af5f1a409d3a92195e04621db3b1a94e3d316b195cd2cd22f5e8d932cc6c
4
+ data.tar.gz: 01fc4bc23a6e33f8ab04aeb574dd529bd22064e8b5577f27ce9dc8e548d5f103
5
5
  SHA512:
6
- metadata.gz: 269d4146de6a9bdbd206f76c2d8c2563093cfb71c43a5d1660e429816d5e6577e4610987ce9b2c63f545efe385beaa1b394b0f26f7017a6ca43e9ca6e95195fa
7
- data.tar.gz: 78984fe0392d030f17f1842738371c875fc7478db115fc1f0f8a42769210ce0a7e5f277aab641f63e06ab267db55b1016b6d280bcd9c21e6c2c90e887d88dc5d
6
+ metadata.gz: 97f2e8591b4af40cb2bb76f20be0a847d1e10d3ed005c683293a26e1b681b224e1a50d8c604266a0816afec46af81b1283251312ccc37ef33fdb6814af64864f
7
+ data.tar.gz: 57bbb6ca200bfd4af2d1f1049585540c4eec5c10957f3ff48a489db46c3a352849aa5369ee402ec35d4958550ef0608a842e141e7139d3775b92981ba11f652a
data/README.md CHANGED
@@ -1,30 +1,35 @@
1
1
  # Sentinel Blue Azure Log Analytics output plugin for Logstash
2
2
 
3
- Azure Sentinel provides an output plugin for Logstash. Using this output plugin, you will be able to send any log you want using Logstash to the Azure Sentinel/Log Analytics workspace
4
- Today you will be able to send messages to custom logs table that you will define in the output plugin.
5
- [Getting started with Logstash](<https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html>)
3
+ Sentinel Blue provides an updated output plugin for Logstash. Using this output plugin, you will be able to send any log you want using Logstash to the Azure Sentinel/Log Analytics workspace using dynamic custom table names.
4
+
5
+ This allows you to set your destination table in your filtering process and reference it in the output plugin. The original plugin functionality has been preserved as well.
6
6
 
7
7
  Azure Sentinel output plugin uses the rest API integration to Log Analytics, in order to ingest the logs into custom logs tables [What are custom logs tables](<https://docs.microsoft.com/azure/azure-monitor/platform/data-sources-custom-logs>)
8
8
 
9
9
  This plugin is based on the original provided by the Azure Sentinel team. View the original plugin here: <https://github.com/Azure/Azure-Sentinel/tree/master/DataConnectors/microsoft-logstash-output-azure-loganalytics>
10
10
 
11
11
  ```text
12
- Plugin version: v1.1.1
13
- Released on: 2022-10-20
12
+ Plugin version: v1.1.2.rc1
13
+ Released on: 2022-10-28
14
14
  ```
15
15
 
16
16
  This plugin is currently in development and is free to use. We welcome contributions from the open source community on this project, and we request and appreciate feedback from users.
17
17
 
18
+ <https://rubygems.org/gems/sentinelblue-logstash-output-azure-loganalytics>
19
+
18
20
  ## Support
19
21
 
20
22
  For issues regarding the output plugin please open a support issue here. Create a new issue describing the problem so that we can assist you.
21
23
 
22
24
  ## Installation
23
25
 
24
- Azure Sentinel provides Logstash an output plugin to Log analytics workspace.
25
26
  Install the sentinelblue-logstash-output-azure-loganalytics, use [Logstash Working with plugins](<https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html>) document.
26
27
  For offline setup follow [Logstash Offline Plugin Management instruction](<https://www.elastic.co/guide/en/logstash/current/offline-plugins.html>).
27
28
 
29
+ ```bash
30
+ logstash-plugin install sentinelblue-logstash-output-azure-loganalytics
31
+ ```
32
+
28
33
  Required Logstash version: between 7.0+
29
34
 
30
35
  ## Configuration
data/VERSION CHANGED
@@ -1 +1 @@
1
- 1.1.1
1
+ 1.1.2.rc1
@@ -63,7 +63,7 @@ class LogStashAutoResizeBuffer
63
63
  resend_message(documents_json, amount_of_documents, @logstashLoganalyticsConfiguration.retransmission_time)
64
64
  end
65
65
  rescue Exception => ex
66
- @logger.error("Exception in posting data to Azure Loganalytics.\n[Exception: '#{ex}]'")
66
+ @logger.error("Exception in posting data to Azure Loganalytics. [Exception: '#{ex}]'")
67
67
  @logger.trace("Exception in posting data to Azure Loganalytics.[amount_of_documents=#{amount_of_documents} documents=#{documents_json}]")
68
68
  resend_message(documents_json, amount_of_documents, @logstashLoganalyticsConfiguration.retransmission_time)
69
69
  end
@@ -80,11 +80,11 @@ class LogStashAutoResizeBuffer
80
80
  if is_successfully_posted(response)
81
81
  @logger.info("Successfully sent #{amount_of_documents} logs into custom log analytics table[#{@custom_log_table_name}] after resending.")
82
82
  else
83
- @logger.debug("Resending #{amount_of_documents} documents failed, will try to resend for #{(remaining_duration - @logstashLoganalyticsConfiguration.RETRANSMISSION_DELAY)}")
83
+ @logger.debug("Resending #{amount_of_documents} documents failed (error code #{response.code}), will try to resend for #{(remaining_duration - @logstashLoganalyticsConfiguration.RETRANSMISSION_DELAY)}")
84
84
  resend_message(documents_json, amount_of_documents, (remaining_duration - @logstashLoganalyticsConfiguration.RETRANSMISSION_DELAY))
85
85
  end
86
86
  rescue Exception => ex
87
- @logger.debug("Resending #{amount_of_documents} documents failed, will try to resend for #{(remaining_duration - @logstashLoganalyticsConfiguration.RETRANSMISSION_DELAY)}")
87
+ @logger.debug("Resending #{amount_of_documents} documents failed (Exception: '#{ex}'), will try to resend for #{(remaining_duration - @logstashLoganalyticsConfiguration.RETRANSMISSION_DELAY)}")
88
88
  resend_message(documents_json, amount_of_documents, (remaining_duration - @logstashLoganalyticsConfiguration.RETRANSMISSION_DELAY))
89
89
  end
90
90
  else
@@ -1,8 +1,9 @@
1
1
  # encoding: utf-8
2
2
  class LogstashLoganalyticsOutputConfiguration
3
- def initialize(workspace_id, workspace_key, logger)
3
+ def initialize(workspace_id, workspace_key, custom_log_table_name, logger)
4
4
  @workspace_id = workspace_id
5
5
  @workspace_key = workspace_key
6
+ @custom_log_table_name = custom_log_table_name
6
7
  @logger = logger
7
8
 
8
9
  # Delay between each resending of a message
@@ -24,8 +25,14 @@ class LogstashLoganalyticsOutputConfiguration
24
25
  elsif @max_items < @MIN_MESSAGE_AMOUNT
25
26
  raise ArgumentError, "Setting max_items to value must be greater then #{@MIN_MESSAGE_AMOUNT}."
26
27
 
27
- elsif @workspace_id.empty? or @workspace_key.empty?
28
- raise ArgumentError, "Malformed configuration , the following arguments can not be null or empty.[workspace_id=#{@workspace_id} , workspace_key=#{@workspace_key}]"
28
+ elsif @workspace_id.empty? or @workspace_key.empty? or @custom_log_table_name.empty?
29
+ raise ArgumentError, "Malformed configuration , the following arguments can not be null or empty.[workspace_id=#{@workspace_id} , workspace_key=#{@workspace_key} , custom_log_table_name=#{@custom_log_table_name}]"
30
+
31
+ elsif !@custom_log_table_name.match(/^[a-zA-Z][[:alpha:][:digit:]_]*$/) and !@custom_log_table_name.match(/^%{((\[[\w_\-@]*\])*)([\w_\-@]*)}$/)
32
+ raise ArgumentError, "custom_log_table_name must be either a static name starting with a letter and consisting only of numbers, letters, and underscores OR a dynamic table name of the format used by logstash (e.g. %{field_name}, %{[nested][field]}."
33
+
34
+ elsif @custom_log_table_name.match(/^[a-zA-Z][[:alpha:][:digit:]_]*$/) and @custom_log_table_name.length > 100
35
+ raise ArgumentError, "custom_log_table_name must not exceed 100 characters"
29
36
 
30
37
  elsif @key_names.length > 500
31
38
  raise ArgumentError, 'Azure Loganalytics limits the amount of columns to 500 in each table.'
@@ -77,6 +84,10 @@ class LogstashLoganalyticsOutputConfiguration
77
84
  @workspace_key
78
85
  end
79
86
 
87
+ def custom_log_table_name
88
+ @custom_log_table_name
89
+ end
90
+
80
91
  def endpoint
81
92
  @endpoint
82
93
  end
@@ -89,7 +89,7 @@ class LogStash::Outputs::AzureLogAnalytics < LogStash::Outputs::Base
89
89
  custom_table_name = ""
90
90
 
91
91
  # Check if the table name is static or dynamic
92
- if @custom_log_table_name.match(/^[[:alpha:][:digit:]_]+$/)
92
+ if @custom_log_table_name.match(/^[a-zA-Z][[:alpha:][:digit:]_]*$/)
93
93
  # Table name is static.
94
94
  custom_table_name = @custom_log_table_name
95
95
 
@@ -100,28 +100,26 @@ class LogStash::Outputs::AzureLogAnalytics < LogStash::Outputs::Base
100
100
 
101
101
  else
102
102
  # Incorrect format
103
- @logger.warn("custom_log_table_name must be either a static name consisting only of numbers, letters, and underscores OR a dynamic table name of the format used by logstash (e.g. %{field_name}, %{[nested][field]}.")
103
+ @logger.error("custom_log_table_name must be either a static name consisting only of numbers, letters, and underscores OR a dynamic table name of the format used by logstash (e.g. %{field_name}, %{[nested][field]}.")
104
104
  break
105
105
 
106
106
  end
107
107
 
108
108
  # Check that the table name is a string, exists, and is les than 100 characters
109
- if !custom_table_name.is_a?(String)
110
- @logger.warn("The custom table name must be a string. If you used a dynamic name from one of the log fields, make sure it is a string.")
111
- break
109
+ if !custom_table_name.match(/^[a-zA-Z][[:alpha:][:digit:]_]*$/)
110
+ @logger.error("The custom table name must start with a letter and only consist of letters, numbers, and/or underscores (_). Also check the field name used. If it doesn't exist, you will also receive this error.")
111
+ next
112
112
 
113
113
  elsif custom_table_name.empty? or custom_table_name.nil?
114
- @logger.warn("The custom table name is empty. If you used a dynamic name from one of the log fields, check that it exists.")
115
- break
114
+ @logger.error("The custom table name is empty. Make sure the field you used always returns a table name.")
115
+ next
116
116
 
117
117
  elsif custom_table_name.length > 100
118
- @logger.warn("The custom table name must not exceed 100 characters")
119
- break
118
+ @logger.error("The custom table name must not exceed 100 characters")
119
+ next
120
120
 
121
121
  end
122
122
 
123
- @logger.info("Custom table name #{custom_table_name} is valid")
124
-
125
123
  # Determine if there is a buffer for the given table
126
124
  if buffers.keys.include?(custom_table_name)
127
125
  @logger.trace("Adding event document - " + event.to_s)
@@ -167,7 +165,7 @@ class LogStash::Outputs::AzureLogAnalytics < LogStash::Outputs::Base
167
165
  # Building the logstash object configuration from the output configuration provided by the user
168
166
  # Return LogstashLoganalyticsOutputConfiguration populated with the configuration values
169
167
  def build_logstash_configuration()
170
- logstash_configuration= LogstashLoganalyticsOutputConfiguration::new(@workspace_id, @workspace_key, @logger)
168
+ logstash_configuration= LogstashLoganalyticsOutputConfiguration::new(@workspace_id, @workspace_key, @custom_log_table_name, @logger)
171
169
  logstash_configuration.endpoint = @endpoint
172
170
  logstash_configuration.time_generated_field = @time_generated_field
173
171
  logstash_configuration.key_names = @key_names
@@ -6,7 +6,7 @@ Gem::Specification.new do |s|
6
6
  s.summary = %q{Sentinel Blue provides a plugin outputing to Azure Sentinel for Logstash. Using this output plugin, you will be able to send any log you want using Logstash to the Azure Sentinel/Log Analytics workspace. You can utilize a dynamic table name during output to simplify complex table schemes.}
7
7
  s.description = s.summary
8
8
  s.homepage = "https://github.com/sentinelblue/sentinelblue-logstash-output-azure-loganalytics"
9
- s.licenses = ['Apache License (2.0)']
9
+ s.licenses = ['Apache-2.0']
10
10
  s.require_paths = ["lib"]
11
11
 
12
12
  # Files
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sentinelblue-logstash-output-azure-loganalytics
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.1
4
+ version: 1.1.2.rc1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sentinel Blue
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-10-20 00:00:00.000000000 Z
11
+ date: 2022-10-28 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rest-client
@@ -94,7 +94,7 @@ files:
94
94
  - spec/outputs/azure_loganalytics_spec.rb
95
95
  homepage: https://github.com/sentinelblue/sentinelblue-logstash-output-azure-loganalytics
96
96
  licenses:
97
- - Apache License (2.0)
97
+ - Apache-2.0
98
98
  metadata:
99
99
  logstash_plugin: 'true'
100
100
  logstash_group: output
@@ -109,9 +109,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
109
109
  version: '0'
110
110
  required_rubygems_version: !ruby/object:Gem::Requirement
111
111
  requirements:
112
- - - ">="
112
+ - - ">"
113
113
  - !ruby/object:Gem::Version
114
- version: '0'
114
+ version: 1.3.1
115
115
  requirements: []
116
116
  rubygems_version: 3.3.7
117
117
  signing_key: