fluent-plugin-kusto 0.0.1.beta

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 90834d37833dc31d3a1aaa3f3de064ee260f9c619753857be060c2e48ba4a79c
4
+ data.tar.gz: 854001eaf38c3065262d2fd4abdb20c14e95af65ba8f08418825ddf6ac56ad94
5
+ SHA512:
6
+ metadata.gz: 66e9089d652413b6a405dc49fafa2fb3fe5399461dd18fdd87025293790dc8d92c9fcf4b8122937f2c06ec766ea43a48716dce499a61c6d298ad03f23299d5c0
7
+ data.tar.gz: 93e5fb41d9e4a76a5a34bad266abf596161f24143b1763d27a3378cf7f22afb87601f0a23b80775bb5cb75dc52b9876f6077f35ba59ae6dd3f19182790e32e68
data/Gemfile ADDED
@@ -0,0 +1,8 @@
1
+ # frozen_string_literal: true
2
+
3
+ source 'https://rubygems.org'
4
+
5
+ gemspec
6
+ gem 'fiddle'
7
+ gem 'mocha'
8
+ gem 'ostruct'
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) Microsoft Corporation.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE
data/README.md ADDED
@@ -0,0 +1,201 @@
1
+ # fluent-plugin-kusto
2
+
3
+ [Fluentd](https://fluentd.org/) output plugin for ingesting logs and data into [Azure Data Explorer (Kusto)](https://azure.microsoft.com/en-us/services/data-explorer/).
4
+
5
+ ## Overview
6
+
7
+ This plugin allows you to send data from Fluentd to Azure Data Explorer (Kusto) using Azure Blob Storage and Queue for scalable, reliable ingestion. It supports both buffered and non-buffered modes, handles authentication via Azure AD or Managed Identity, and provides robust error handling and logging.
8
+
9
+ ## Requirements
10
+
11
+ - Ruby 2.5 or later
12
+ Check version (Windows/Linux):
13
+ ```bash
14
+ ruby --version
15
+ ```
16
+ Install on Ubuntu/Linux:
17
+ ```bash
18
+ sudo apt-get install ruby-full
19
+ ```
20
+ Install on Windows (using RubyInstaller):
21
+ [Download RubyInstaller](https://rubyinstaller.org/)
22
+ [Official Ruby installation guide](https://www.ruby-lang.org/en/documentation/installation/)
23
+
24
+ - Fluentd v1.0 or later
25
+ Check version (Windows/Linux):
26
+ ```bash
27
+ fluentd --version
28
+ ```
29
+ Install on Ubuntu/Linux:
30
+ ```bash
31
+ gem install fluentd
32
+ ```
33
+ Install on Windows (in Command Prompt after Ruby is installed):
34
+ ```cmd
35
+ gem install fluentd
36
+ ```
37
+ [Official Fluentd installation guide](https://docs.fluentd.org/installation)
38
+
39
+
40
+ ## Installation
41
+
42
+ ### RubyGems
43
+
44
+ ```sh
45
+ $ gem install fluent-plugin-kusto --pre
46
+ ```
47
+
48
+ ### Bundler
49
+
50
+ Add the following line to your Gemfile:
51
+
52
+ ```ruby
53
+ gem "fluent-plugin-kusto", "~> 0.0.1.beta"
54
+ ```
55
+
56
+ **Note:** This is a beta release. Use the `--pre` flag with gem install or specify the beta version in your Gemfile.
57
+
58
+ And then execute:
59
+
60
+ ```sh
61
+ $ bundle
62
+ ```
63
+
64
+ ## Azure Data Explorer (Kusto) Prerequisites
65
+ The _Kusto_ output plugin lets you ingest your logs into an [Azure Data Explorer](https://azure.microsoft.com/en-us/services/data-explorer/) cluster, using the [Queued Ingestion](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/netfx/about-kusto-ingest#queued-ingestion) mechanism.
66
+
67
+ ## Ingest into Azure Data Explorer: create a Kusto cluster and database
68
+
69
+ Create an Azure Data Explorer cluster in one of the following ways:
70
+
71
+ * [Create a free-tier cluster](https://dataexplorer.azure.com/freecluster)
72
+ * [Create a fully featured cluster](https://docs.microsoft.com/en-us/azure/data-explorer/create-cluster-database-portal)
73
+
74
+ ## Create an Azure registered application
75
+
76
+ FluentD uses the Azure application's credentials to ingest data into your cluster.
77
+
78
+ * [Register an application](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app#register-an-application)
79
+ * [Add a client secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app#add-a-client-secret)
80
+ * [Authorize the app in your database](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/access-control/principals-and-identity-providers#azure-ad-tenants)
81
+
82
+ ## Create a Managed Identity in Azure
83
+
84
+ - **System-assigned Managed Identity:**
85
+ - [Enable system-assigned managed identity](https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-enable-system-assigned-managed-identity)
86
+
87
+ - **User-assigned Managed Identity:**
88
+ - [Create and assign user-assigned managed identity](https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-create-user-assigned-managed-identity)
89
+
90
+ - **Grant Permissions:**
91
+ - [Assign permissions to managed identity for Azure Data Explorer](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/management/access-control/principals-and-identity-providers)
92
+
93
+ ## Workload Identity Authentication
94
+
95
+ - [Follow workload_identity.md](workload_identity.md)
96
+
97
+ ## Create a database
98
+
99
+ To create a new database in your Azure Data Explorer cluster, use the following KQL command:
100
+
101
+ ```kql
102
+ .create database <database_name>
103
+ ```
104
+
105
+ ## Create a table
106
+
107
+ Fluent Bit ingests the event data into Kusto in a JSON format. By default, the table includes 3 properties:
108
+
109
+ * `record` - the actual event payload.
110
+ * `tag` - the event tag.
111
+ * `timestamp` - the event timestamp.
112
+
113
+ A table with the expected schema must exist in order for data to be ingested properly.
114
+
115
+ ```kql
116
+ .create table <table_name> (tag:string, timestamp:datetime, record:dynamic)
117
+ ```
118
+
119
+ ## Configuration parameters
120
+
121
+ | Key | Description | Default |
122
+ | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
123
+ | `tenant_id` | The tenant/domain ID of the Azure Active Directory (AAD) registered application. Required if `managed_identity_client_id` isn't set. | _none_ |
124
+ | `client_id` | The client ID of the AAD registered application. Required if `managed_identity_client_id` isn't set. | _none_ |
125
+ | `client_secret` | The client secret of the AAD registered application ([App Secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret)). Required if `managed_identity_client_id` isn't set. | _none_ |
126
+ | `managed_identity_client_id` | The managed identity ID to authenticate with. Set to `SYSTEM` for system-assigned managed identity, or set to the MI client ID (`GUID`) for user-assigned managed identity. Required if `tenant_id`, `client_id`, and `client_secret` aren't set. | _none_ |
127
+ | `endpoint` | The cluster's endpoint, usually in the form `https://cluster_name.region.kusto.windows.net` | _none_ |
128
+ | `database_name` | The database name. | _none_ |
129
+ | `table_name` | The table name. | _none_ |
130
+ | `compression_enabled` | If enabled, sends compressed HTTP payload (gzip) to Kusto. | `true` |
131
+ | `workers` | The number of [workers](../../../administration/multithreading#outputs) to perform flush operations for this output. | `0` |
132
+ | `buffered` | Enable buffering into disk before ingesting into Azure Kusto. If `buffered` is `true`, buffered mode is activated. If `false`, non-buffered mode is used. | `true` |
133
+ | `delayed` | If `true`, enables delayed commit for buffer chunks. Only supported in buffered mode (`buffered` must be `true`). If `buffered` is `false`, delayed commit is not available. | `false` |
134
+ | `azure_cloud` | Azure cloud environment. E.g., `AzureCloud`, `AzureChinaCloud`, `AzureUSGovernmentCloud`, `AzureGermanCloud`. | `AzureCloud` |
135
+ | `chunk_keys` (buffer section) | Only in buffered mode. Keys to use for chunking the buffer. Possible values: `tag`, `time`, or a combination such as `["tag", "time"]`. Controls how data is grouped and flushed. | `["time"]` |
136
+ | `timekey` (buffer section) | Only in buffered mode. Time interval for buffer chunking. Possible values: integer seconds (e.g., `60`, `3600`, `86400`). | `86400` (1 day) |
137
+ | `timekey_wait` (buffer section) | Only in buffered mode. Wait time before flushing a timekey chunk after its time window closes. Possible values: duration string (e.g., `30s`, `5m`). | `30s` |
138
+ | `timekey_use_utc` (buffer section) | Only in buffered mode. Use UTC for timekey chunking. Possible values: `true`, `false`. | `true` |
139
+ | `flush_at_shutdown` (buffer section) | Only in buffered mode. Flush buffer at shutdown. Possible values: `true`, `false`. | `true` |
140
+ | `retry_max_times` (buffer section) | Only in buffered mode. Maximum number of retry attempts for buffer flush. Possible values: integer (e.g., `5`, `10`). | `5` |
141
+ | `retry_wait` (buffer section) | Only in buffered mode. Wait time between buffer flush retries. Possible values: duration string (e.g., `1s`, `10s`). | `1s` |
142
+ | `overflow_action` (buffer section) | Only in buffered mode. Action to take when buffer overflows. Possible values: `block`, `drop_oldest_chunk`, `throw_exception`. | `block` |
143
+ | `chunk_limit_size` (buffer section) | Only in buffered mode. Maximum size per buffer chunk. Possible values: size string (e.g., `256m`, `1g`). | `256m` |
144
+ | `total_limit_size` (buffer section) | Only in buffered mode. Maximum total buffer size. Possible values: size string (e.g., `2g`, `10g`). | `2g` |
145
+ | `flush_mode` (buffer section) | Only in buffered mode. Buffer flush mode. Possible values: `interval`, `immediate`, `lazy`. | `interval` |
146
+ | `flush_interval` (buffer section) | Only in buffered mode. Interval for buffer flush. Possible values: duration string (e.g., `10s`, `1m`). | `10s` |
147
+ | `logger_path` | Optional. File path for plugin log output. If not set, logs are written to stdout. | stdout(terminal) |
148
+ | `auth_type` | The authentication type to use. Possible values: `aad`, `user_managed_identity`, `system_managed_identity`,`workload_identity`. | `aad` |
149
+ | `workload_identity_client_id` | The client ID for Azure Workload Identity authentication. Required if using workload identity for authentication. | _none_ |
150
+ | `workload_identity_tenant_id` | The tenant ID for Azure Workload Identity authentication. Required if using workload identity for authentication. | _none_ |
151
+ | `workload_identity_token_file` | The file path to the token file for Azure Workload Identity authentication. Required if using workload identity for authentication. | `/var/run/secrets/azure/tokens/azure-identity-token` |
152
+
153
+ ## Sample Configuration
154
+
155
+ ```conf
156
+ <system>
157
+ workers 1
158
+ </system>
159
+ <match test.kusto>
160
+ @type kusto
161
+ @log_level debug
162
+ buffered true
163
+ delayed false
164
+ endpoint https://yourcluster.region.kusto.windows.net
165
+ database_name your-db
166
+ table_name your-table
167
+ tenant_id <your-tenant-id>
168
+ client_id <your-client-id>
169
+ managed_identity_client_id SYSTEM
170
+ compression_enabled true
171
+ azure_cloud AzureCloud
172
+ logger_path /var/log/azure-kusto-fluentd.log
173
+ <buffer>
174
+ @type memory
175
+ # To chunk by tag only:
176
+ # chunk_keys tag
177
+ # To chunk by tag and time:
178
+ # chunk_keys tag,time
179
+ timekey 1m
180
+ timekey_wait 30s
181
+ timekey_use_utc true
182
+ flush_at_shutdown true
183
+ retry_max_times 5
184
+ retry_wait 1s
185
+ overflow_action block
186
+ chunk_limit_size 256m
187
+ total_limit_size 2g
188
+ flush_mode interval
189
+ flush_interval 10s
190
+ </buffer>
191
+ </match>
192
+ ```
193
+
194
+ # Fluentd Azure Data Explorer (Kusto) Output Plugin Architecture
195
+ ![Architecture](architecture.png)
196
+
197
+ This diagram shows the main components and data flow for the plugin, including configuration, error handling, token management, and Azure resource interactions.
198
+
199
+ ## Copyright
200
+
201
+ * License: Apache License, Version 2.0
@@ -0,0 +1,105 @@
1
+ # frozen_string_literal: true
2
+
3
+ # AadTokenProvider handles acquiring and refreshing Azure Active Directory tokens for Kusto ingestion.
4
+ #
5
+ # Responsibilities:
6
+ # - Build and send token requests to Azure AD endpoint
7
+ # - Cache and refresh tokens as needed
8
+ # - Support client credentials flow for authentication
9
+ require 'json'
10
+ require 'openssl'
11
+ require 'base64'
12
+ require 'time'
13
+ require 'net/http'
14
+ require 'uri'
15
+ require_relative '../kusto_error_handler'
16
+ require_relative 'tokenprovider_base'
17
+
18
+ class AadTokenProvider < AbstractTokenProvider
19
+ def initialize(outconfiguration)
20
+ super(outconfiguration)
21
+ token_request_params_set
22
+ end
23
+
24
+ # Use get_token from base class for token retrieval
25
+
26
+ private
27
+
28
+ def setup_config(outconfiguration)
29
+ @client_id = outconfiguration.client_app_id
30
+ @client_secret = outconfiguration.client_app_secret
31
+ @tenant_id = outconfiguration.tenant_id
32
+ @aad_uri = outconfiguration.aad_endpoint
33
+ @resource = outconfiguration.kusto_endpoint
34
+ @database_name = outconfiguration.database_name
35
+ @table_name = outconfiguration.table_name
36
+ @azure_cloud = outconfiguration.azure_cloud
37
+ @managed_identity_client_id = outconfiguration.managed_identity_client_id
38
+ end
39
+
40
+ def token_request_params_set
41
+ @token_request_uri = "#{@aad_uri}/#{@tenant_id}/oauth2/v2.0/token"
42
+ @scope = "#{@resource}/.default"
43
+ end
44
+
45
+ def fetch_token
46
+ response = post_token_request
47
+ {
48
+ access_token: response['access_token'],
49
+ expires_in: response['expires_in']
50
+ }
51
+ end
52
+
53
+ def post_token_request
54
+ headers = header
55
+ max_retries = 10
56
+ retries = 0
57
+ uri = URI.parse(@token_request_uri)
58
+ form_data = URI.encode_www_form(
59
+ 'grant_type' => 'client_credentials',
60
+ 'client_id' => @client_id,
61
+ 'client_secret' => @client_secret,
62
+ 'scope' => @scope
63
+ )
64
+ while retries < max_retries
65
+ begin
66
+ http = Net::HTTP.new(uri.host, uri.port)
67
+ http.use_ssl = (uri.scheme == 'https')
68
+ request = Net::HTTP::Post.new(uri.request_uri, headers)
69
+ request.body = form_data
70
+
71
+ response = http.request(request)
72
+ return JSON.parse(response.body) if [200, 201].include?(response.code.to_i)
73
+
74
+ begin
75
+ error_json = JSON.parse(response.body)
76
+ kusto_error_type = KustoErrorHandler.extract_kusto_error_type(error_json)
77
+ error = KustoErrorHandler.from_kusto_error_type(
78
+ kusto_error_type,
79
+ error_json['error_description'] || error_json['message'] || response.body
80
+ )
81
+ if error.permanent_error?
82
+ @logger.error("Permanent error encountered, not retrying. #{error.message}")
83
+ raise error
84
+ end
85
+ rescue JSON::ParserError
86
+ @logger.error("Failed to parse error response: #{response.body}")
87
+ raise "Permanent error while authenticating with AAD: #{response.body}"
88
+ end
89
+ end
90
+ retries += 1
91
+ @logger.error(
92
+ "Error while authenticating with AAD ('#{@aad_uri}'), retrying in 10 seconds. " \
93
+ "Attempt #{retries}/#{max_retries}"
94
+ )
95
+ sleep 10
96
+ end
97
+ raise "Failed to authenticate with AAD after #{max_retries} attempts."
98
+ end
99
+
100
+ def header
101
+ {
102
+ 'Content-Type' => 'application/x-www-form-urlencoded'
103
+ }
104
+ end
105
+ end
@@ -0,0 +1,51 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'json'
4
+ require 'time'
5
+ require 'open3'
6
+ require 'shellwords'
7
+ require_relative 'tokenprovider_base'
8
+
9
+ class AzCliTokenProvider < AbstractTokenProvider
10
+ def initialize(outconfiguration)
11
+ super(outconfiguration)
12
+ @resource = outconfiguration.kusto_endpoint
13
+ end
14
+
15
+ # Use get_token from base class for token retrieval
16
+
17
+ def fetch_token
18
+ token = acquire_token(@resource)
19
+ raise "No valid Azure CLI token found for resource: #{@resource}" unless token
20
+
21
+ {
22
+ access_token: token['accessToken'],
23
+ expires_in: (Time.parse(token['expiresOn']) - Time.now).to_i
24
+ }
25
+ end
26
+
27
+ def acquire_token(resource)
28
+ az_cli = locate_azure_cli
29
+ # Properly escape the Azure CLI executable path to prevent command injection
30
+ escaped_az_cli = Shellwords.escape(az_cli)
31
+ escaped_resource = Shellwords.escape(resource)
32
+ cmd = [escaped_az_cli, 'account', 'get-access-token', '--resource', escaped_resource, '--output', 'json']
33
+ stdout, stderr, status = Open3.capture3(*cmd)
34
+ raise "Failed to acquire Azure CLI token: #{stderr.strip}" if !status.success? && !status.success?
35
+
36
+ JSON.parse(stdout)
37
+ rescue Errno::ENOENT
38
+ raise "Azure CLI not found. Please install Azure CLI and run 'az login'."
39
+ end
40
+
41
+ def locate_azure_cli
42
+ exts = ENV['PATHEXT'] ? ENV['PATHEXT'].split(';') : ['']
43
+ ENV['PATH'].split(File::PATH_SEPARATOR).each do |path|
44
+ exts.each do |ext|
45
+ exe = File.join(path, "az#{ext}")
46
+ return exe if File.executable?(exe) && !File.directory?(exe)
47
+ end
48
+ end
49
+ raise "Azure CLI executable 'az' not found in PATH."
50
+ end
51
+ end
@@ -0,0 +1,92 @@
1
+ # frozen_string_literal: true
2
+
3
+ # ManagedIdentityTokenProvider handles acquiring and refreshing Azure Managed Identity tokens for Kusto ingestion.
4
+ #
5
+ # Responsibilities:
6
+ # - Build and send token requests to Azure IMDS endpoint
7
+ # - Cache and refresh tokens as needed
8
+ # - Support both system-assigned and user-assigned managed identities
9
+
10
+ require 'net/http'
11
+ require 'uri'
12
+ require 'json'
13
+ require 'erb'
14
+ require_relative 'tokenprovider_base'
15
+
16
+ class ManagedIdentityTokenProvider < AbstractTokenProvider
17
+ IMDS_TOKEN_ACQUIRE_URL = 'http://169.254.169.254/metadata/identity/oauth2/token'
18
+
19
+ def initialize(outconfiguration)
20
+ super(outconfiguration)
21
+ token_request_params_set(outconfiguration)
22
+ end
23
+
24
+ # Use get_token from base class for token retrieval
25
+
26
+ private
27
+
28
+ def setup_config(outconfiguration)
29
+ @resource = outconfiguration.kusto_endpoint
30
+ @managed_identity_client_id = outconfiguration.managed_identity_client_id
31
+ val = @managed_identity_client_id.to_s.strip
32
+ @use_system_assigned = (val.upcase == 'SYSTEM')
33
+ @use_user_assigned = !val.empty? && val.upcase != 'SYSTEM'
34
+ end
35
+
36
+ def append_header(name, value)
37
+ "#{name}=#{value}"
38
+ end
39
+
40
+ def token_request_params_set(_outconfiguration)
41
+ token_acquire_url = IMDS_TOKEN_ACQUIRE_URL.dup + '?' + append_header('resource',
42
+ ERB::Util.url_encode(outconfiguration.kusto_endpoint)) + '&' + append_header(
43
+ 'api-version', '2018-02-01'
44
+ )
45
+ unless @object_id.nil?
46
+ token_acquire_url = (token_acquire_url + '&' + append_header('object_id',
47
+ ERB::Util.url_encode(@object_id)))
48
+ end
49
+ unless @msi_res_id.nil?
50
+ token_acquire_url = (token_acquire_url + '&' + append_header('msi_res_id',
51
+ ERB::Util.url_encode(@msi_res_id)))
52
+ end
53
+ URI.parse(token_acquire_url)
54
+ return unless @use_user_assigned
55
+
56
+ (token_acquire_url + '&' + append_header('client_id',
57
+ ERB::Util.url_encode(@managed_identity_client_id)))
58
+ end
59
+
60
+ def fetch_token
61
+ response = post_token_request
62
+ {
63
+ access_token: response['access_token'],
64
+ expires_in: response['expires_in']
65
+ }
66
+ end
67
+
68
+ def post_token_request
69
+ headers = { 'Metadata' => 'true' }
70
+ max_retries = 2
71
+ retries = 0
72
+ uri = URI.parse(@token_acquire_url)
73
+ while retries < max_retries
74
+ begin
75
+ http = Net::HTTP.new(uri.host, uri.port)
76
+ request = Net::HTTP::Get.new(uri.request_uri, headers)
77
+ response = http.request(request)
78
+ return JSON.parse(response.body) if response.code.to_i == 200
79
+
80
+ @logger.error("Failed to get managed identity token: #{response.code} #{response.body}")
81
+ rescue StandardError => e
82
+ @logger.error("Error while requesting managed identity token: #{e.message}")
83
+ end
84
+ retries += 1
85
+ @logger.error(
86
+ "Retrying managed identity token request in 10 seconds. Attempt #{retries}/#{max_retries}"
87
+ )
88
+ sleep 10
89
+ end
90
+ raise "Failed to get managed identity token after #{max_retries} attempts."
91
+ end
92
+ end
@@ -0,0 +1,57 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'logger'
4
+
5
+ # AbstractTokenProvider defines the interface and shared logic for all token providers.
6
+ class AbstractTokenProvider
7
+ def initialize(outconfiguration)
8
+ @logger = setup_logger(outconfiguration)
9
+ setup_config(outconfiguration)
10
+ @token_state = { access_token: nil, expiry_time: nil, token_details_mutex: Mutex.new }
11
+ end
12
+
13
+ # Abstract method: must be implemented by subclasses to fetch a new token.
14
+ def fetch_token
15
+ raise NotImplementedError, 'Subclasses must implement fetch_token'
16
+ end
17
+
18
+ # Public method to get a valid token, refreshing if needed.
19
+ def get_token
20
+ @token_state[:token_details_mutex].synchronize do
21
+ if saved_token_need_refresh?
22
+ @logger.info("Refreshing token. Previous expiry: #{@token_state[:expiry_time]}")
23
+ refresh_saved_token
24
+ @logger.info("New token expiry: #{@token_state[:expiry_time]}")
25
+ end
26
+ @token_state[:access_token]
27
+ end
28
+ end
29
+
30
+ private
31
+
32
+ def setup_logger(outconfiguration)
33
+ outconfiguration.logger || Logger.new($stdout)
34
+ end
35
+
36
+ def setup_config(_outconfiguration)
37
+ # To be optionally overridden by subclasses
38
+ end
39
+
40
+ def saved_token_need_refresh?
41
+ @token_state[:access_token].nil? || @token_state[:expiry_time].nil? || @token_state[:expiry_time] <= Time.now
42
+ end
43
+
44
+ def refresh_saved_token
45
+ token_response = fetch_token
46
+ @token_state[:access_token] = token_response[:access_token]
47
+ @token_state[:expiry_time] = get_token_expiry_time(token_response[:expires_in])
48
+ end
49
+
50
+ def get_token_expiry_time(expires_in_seconds)
51
+ if expires_in_seconds.nil? || expires_in_seconds.to_i <= 0
52
+ Time.now + 3540 # Default to 59 minutes if expires_in is not provided or invalid
53
+ else
54
+ Time.now + expires_in_seconds.to_i - 1
55
+ end
56
+ end
57
+ end
@@ -0,0 +1,50 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'net/http'
4
+ require 'uri'
5
+ require 'json'
6
+ require_relative 'tokenprovider_base'
7
+
8
+ class WorkloadIdentity < AbstractTokenProvider
9
+ DEFAULT_TOKEN_FILE = '/var/run/secrets/azure/tokens/azure-identity-token'
10
+ AZURE_OAUTH2_TOKEN_ENDPOINT = 'https://login.microsoftonline.com/%<tenant_id>s/oauth2/v2.0/token'
11
+
12
+ # Use get_token from base class for token retrieval
13
+
14
+ private
15
+
16
+ def setup_config(outconfiguration)
17
+ @client_id = outconfiguration.workload_identity_client_id
18
+ @tenant_id = outconfiguration.workload_identity_tenant_id
19
+ @token_file = outconfiguration.workload_identity_token_file_path || DEFAULT_TOKEN_FILE
20
+ @kusto_endpoint = outconfiguration.kusto_endpoint
21
+ @scope = "#{@kusto_endpoint}/.default"
22
+ end
23
+
24
+ def fetch_token
25
+ response = acquire_workload_identity_token
26
+ {
27
+ access_token: response['access_token'],
28
+ expires_in: response['expires_in']
29
+ }
30
+ end
31
+
32
+ def acquire_workload_identity_token
33
+ oidc_token = File.read(@token_file).strip
34
+ uri = URI.parse(format(AZURE_OAUTH2_TOKEN_ENDPOINT, tenant_id: @tenant_id))
35
+ req = Net::HTTP::Post.new(uri)
36
+ req.set_form_data(
37
+ 'grant_type' => 'client_credentials',
38
+ 'client_id' => @client_id,
39
+ 'scope' => @scope,
40
+ 'client_assertion_type' => 'urn:ietf:params:oauth:client-assertion-type:jwt-bearer',
41
+ 'client_assertion' => oidc_token
42
+ )
43
+ http = Net::HTTP.new(uri.host, uri.port)
44
+ http.use_ssl = true
45
+ res = http.request(req)
46
+ raise "Failed to get access token: #{res.code} #{res.body}" unless res.is_a?(Net::HTTPSuccess)
47
+
48
+ JSON.parse(res.body)
49
+ end
50
+ end