microsoft-sentinel-log-analytics-logstash-output-plugin 1.1.0 → 1.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 38074bb5b8fb9211f87c27ccbd46051d6fff20ae2c24c0c398150c1e1087fa13
-  data.tar.gz: 9b14d8ba18d9eba8f0464e07e2351aa7eee9ea3356d410eeef56fdace03bfe42
+  metadata.gz: bc9cee055d5aa8f90acde6ed20eacccf0d9e9a36fa9a5ebd1c916d85a410e628
+  data.tar.gz: '081cd8f53b716eeed931c417ce6fc7a13e3d2d1fe9acf0050858e5b87918d4ff'
 SHA512:
-  metadata.gz: 19fd167e1dd6ab8874ca2fa4d11971f01e070238ef08afffe4e851eaf7cc7c27b9dc559257deb118eb1c4d82295e3f1a5a3530cdc77cbef5f98ff0926fe4b44b
-  data.tar.gz: b41a633f1f6a0af1f07bdd8fb12f784febbb219c165de1c0e92557644033b30b5b73d66f9d0e4b361825d1f8a7320609d38b25181496e3dfdd12ec718169ed90
+  metadata.gz: 77753b09f6c4631fb2e2eb1e0ba5dd4eb4c33840901226f1caa08043b39caa77497cdfd42adb7ac6a857b22e3bd98512fff8e3e1ba4d6b7916eb16e822f752ce
+  data.tar.gz: 9ae33cb6bb9c96c19011b4a21f9ac943f75df18b99761023cb4ba00c84584a6ad0c05d8b0d5c0819993b434e2f6d14da1e62f92ba2f44a11aa6eeed22cbe0b9c

data/CHANGELOG.md

@@ -1,13 +1,14 @@
 ## 1.0.0
 * Initial release for output plugin for logstash to Microsoft Sentinel. This is done with the Log Analytics DCR based API.
 
-## 1.0.2
-* Upgrade the rest-client dependency minimum version to 2.1.0
-* Allow setting different proxy values for api connections.
-
-## 1.0.6
+## 1.1.0 
 * Increase timeout for read/open connections to 120 seconds.
 * Add error handling for when connection timeout occurs.
+* Upgrade the rest-client dependency minimum version to 2.1.0.
+* Allow setting different proxy values for api connections.
+* Upgrade version for ingestion api to 2023-01-01.
+* Rename the plugin to microsoft-sentinel-log-analytics-logstash-output-plugin.
 
-## 1.1.0 
-* Rename the plugin to microsoft-sentinel-log-analytics-logstash-output-plugin
+## 1.1.1
+* Support China and US Government Azure sovereign clouds.
+* Increase timeout for read/open connections to 240 seconds.

data/README.md

@@ -3,8 +3,8 @@
 Microsoft Sentinel provides a new output plugin for Logstash. Use this output plugin to send any log via Logstash to the Microsoft Sentinel/Log Analytics workspace. This is done with the Log Analytics DCR-based API.
 You may send logs to custom or standard tables.
 
-Plugin version: v1.0.2  
-Released on: 2023-04-27
+Plugin version: v1.1.0  
+Released on: 2023-07-23
 
 This plugin is currently in development and is free to use. We welcome contributions from the open source community on this project, and we request and appreciate feedback from users.
 
@@ -23,8 +23,9 @@ Microsoft Sentinel provides Logstash output plugin to Log analytics workspace us
 Install the microsoft-sentinel-log-analytics-logstash-output-plugin, use [Logstash Offline Plugin Management instruction](<https://www.elastic.co/guide/en/logstash/current/offline-plugins.html>). 
 
 Microsoft Sentinel's Logstash output plugin supports the following versions
-- Logstash 7 Between 7.0 and 7.17.6
-- Logstash 8 Between 8.0 and 8.4.2
+- 7.0 - 7.17.13
+- 8.0 - 8.9
+- 8.11
 
 Please note that when using Logstash 8, it is recommended to disable ECS in the pipeline. For more information refer to [Logstash documentation.](<https://www.elastic.co/guide/en/logstash/8.4/ecs-ls.html>)
 
@@ -41,8 +42,8 @@ output {
 }
 ```
 Note: make sure that the path exists before creating the sample file.
-2) Start Logstash. The plugin will write up to 10 records to a sample file named "sampleFile<epoch seconds>.json" in the configured path  
-(for example: "c:\temp\sampleFile1648453501.json")
+2) Start Logstash. The plugin will collect up to 10 records to a sample.
+3) The file named "sampleFile<epoch seconds>.json" in the configured path will be created once there are 10 events to sample or when the Logstash process exited gracefully. (for example: "c:\temp\sampleFile1648453501.json").
 
 
 ### Configurations:
@@ -124,6 +125,7 @@ output {
 - **proxy** - String, Empty by default. Specify which proxy URL to use for API calls for all of the communications with Azure.
 - **proxy_aad** - String, Empty by default. Specify which proxy URL to use for API calls for the Azure Active Directory service. Overrides the proxy setting.
 - **proxy_endpoint** - String, Empty by default. Specify which proxy URL to use when sending log data to the endpoint. Overrides the proxy setting.
+- **azure_cloud** - String, Empty by default. Used to specify the name of the Azure cloud that is being used, AzureCloud is set as default. Available values are: AzureCloud, AzureChinaCloud and AzureUSGovernment.
 
 #### Note: When setting an empty string as a value for a proxy setting, it will unset any system wide proxy setting.
 

data/lib/logstash/outputs/microsoft-sentinel-log-analytics-logstash-output-plugin.rb

@@ -68,6 +68,11 @@ class LogStash::Outputs::MicrosoftSentinelOutput < LogStash::Outputs::Base
   # Path where to place the sample file created
   config :sample_file_path, :validate => :string
 
+  # Used to specify the name of the Azure cloud that is being used. By default, the value is set to "AzureCloud", which
+  # is the public Azure cloud. However, you can specify a different Azure cloud if you are
+  # using a different environment, such as Azure Government or Azure China.
+  config :azure_cloud, :validate => :string
+
   public
   def register
     @logstash_configuration= build_logstash_configuration()
@@ -103,6 +108,7 @@ class LogStash::Outputs::MicrosoftSentinelOutput < LogStash::Outputs::Base
     logstash_configuration.proxy_aad = @proxy_aad || @proxy || ENV['http_proxy']
     logstash_configuration.proxy_endpoint = @proxy_endpoint || @proxy || ENV['http_proxy']
     logstash_configuration.retransmission_time = @retransmission_time
+    logstash_configuration.azure_cloud = @azure_cloud || "AzureCloud"
     
     return logstash_configuration
   end # def build_logstash_configuration

data/lib/logstash/sentinel_la/logAnalyticsAadTokenProvider.rb

@@ -9,8 +9,8 @@ require 'time'
 module LogStash; module Outputs; class MicrosoftSentinelOutputInternal
 class LogAnalyticsAadTokenProvider
   def initialize (logstashLoganalyticsConfiguration)
-    scope = CGI.escape("https://monitor.azure.com//.default")
-    @aad_uri = "https://login.microsoftonline.com"
+    scope = CGI.escape("#{logstashLoganalyticsConfiguration.get_monitor_endpoint}//.default")
+    @aad_uri = logstashLoganalyticsConfiguration.get_aad_endpoint
     @token_request_body = sprintf("client_id=%s&scope=%s&client_secret=%s&grant_type=client_credentials", logstashLoganalyticsConfiguration.client_app_Id, scope, logstashLoganalyticsConfiguration.client_app_secret)
     @token_request_uri = sprintf("%s/%s/oauth2/v2.0/token",@aad_uri, logstashLoganalyticsConfiguration.tenant_id)
     @token_state = {

data/lib/logstash/sentinel_la/logAnalyticsClient.rb

@@ -14,7 +14,7 @@ require "logstash/sentinel_la/logstashLoganalyticsConfiguration"
 require "logstash/sentinel_la/logAnalyticsAadTokenProvider"
 
 
-  def initialize (logstashLoganalyticsConfiguration)
+  def initialize(logstashLoganalyticsConfiguration)
     @logstashLoganalyticsConfiguration = logstashLoganalyticsConfiguration
     @logger = @logstashLoganalyticsConfiguration.logger
 
@@ -34,7 +34,7 @@ require "logstash/sentinel_la/logAnalyticsAadTokenProvider"
     # Post REST request
 
     return RestClient::Request.execute(method: :post, url: @uri, payload: body, headers: headers,
-                                        proxy: @logstashLoganalyticsConfiguration.proxy_endpoint, timeout: 120)
+                                        proxy: @logstashLoganalyticsConfiguration.proxy_endpoint, timeout: 240)
   end # def post_data
 
   # Static function to return if the response is OK or else

data/lib/logstash/sentinel_la/logstashLoganalyticsConfiguration.rb

@@ -23,6 +23,12 @@ class LogstashLoganalyticsOutputConfiguration
 
         # Taking 4K safety buffer
         @MAX_SIZE_BYTES = @loganalytics_api_data_limit - 10000
+
+        @azure_clouds = {
+            "AzureCloud" => {"aad" => "https://login.microsoftonline.com", "monitor" => "https://monitor.azure.com"},
+            "AzureChinaCloud" => {"aad" => "https://login.chinacloudapi.cn", "monitor" => "https://monitor.azure.cn"},
+            "AzureUSGovernment" => {"aad" => "https://login.microsoftonline.us", "monitor" => "https://monitor.azure.us"}
+        }.freeze
     end
 	
 	def validate_configuration()
@@ -68,6 +74,9 @@ class LogstashLoganalyticsOutputConfiguration
           if @key_names.length > 500
               raise ArgumentError, 'There are over 500 key names listed to be included in the events sent to Azure Loganalytics, which exceeds the limit of columns that can be define in each table in log analytics.'
           end
+          if !@azure_clouds.key?(@azure_cloud)
+            raise ArgumentError, "The specified Azure cloud #{@azure_cloud} is not supported. Supported clouds are: #{@azure_clouds.keys.join(", ")}."
+          end
       end
         @logger.info("Azure Loganalytics configuration was found valid.")
         # If all validation pass then configuration is valid
@@ -159,10 +168,6 @@ class LogstashLoganalyticsOutputConfiguration
         @MIN_MESSAGE_AMOUNT
     end
     
-    def max_items=(new_max_items)
-        @max_items = new_max_items
-    end
-
     def key_names=(new_key_names)
         @key_names = new_key_names
     end
@@ -218,5 +223,21 @@ class LogstashLoganalyticsOutputConfiguration
     def sample_file_path=(new_sample_file_path)
         @sample_file_path = new_sample_file_path
     end
+
+    def azure_cloud
+        @azure_cloud
+    end
+
+    def azure_cloud=(new_azure_cloud)
+        @azure_cloud = new_azure_cloud
+    end
+
+    def get_aad_endpoint
+        @azure_clouds[@azure_cloud]["aad"]
+    end
+    
+    def get_monitor_endpoint
+        @azure_clouds[@azure_cloud]["monitor"]
+    end
 end
 end ;end ;end 

data/lib/logstash/sentinel_la/version.rb

@@ -1,6 +1,6 @@
 module LogStash; module Outputs;
 class MicrosoftSentinelOutputInternal
-  VERSION_INFO = [1, 1, 0].freeze
+  VERSION_INFO = [1, 1, 1].freeze
   VERSION = VERSION_INFO.map(&:to_s).join('.').freeze
 
   def self.version

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: microsoft-sentinel-log-analytics-logstash-output-plugin
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.1.1
 platform: ruby
 authors:
 - Microsoft Sentinel
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-07-23 00:00:00.000000000 Z
+date: 2024-01-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rest-client
@@ -118,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.6
+rubygems_version: 3.3.26
 signing_key:
 specification_version: 4
 summary: Microsoft Sentinel provides a new output plugin for Logstash. Use this output