logstash-output-kusto 1.0.4-java → 1.0.5-java
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -1
- data/README.md +13 -0
- data/SECURITY.md +41 -0
- data/lib/com/microsoft/azure/kusto/kusto-data/3.1.3/kusto-data-3.1.3.jar +0 -0
- data/lib/com/microsoft/azure/kusto/kusto-ingest/3.1.3/kusto-ingest-3.1.3.jar +0 -0
- data/lib/logstash/outputs/kusto/ingestor.rb +27 -12
- data/lib/logstash/outputs/kusto.rb +11 -2
- data/lib/logstash-output-kusto_jars.rb +4 -4
- data/logstash-output-kusto.gemspec +2 -2
- data/spec/outputs/kusto/ingestor_spec.rb +18 -6
- data/spec/outputs/kusto_spec.rb +4 -1
- metadata +6 -5
- data/lib/com/microsoft/azure/kusto/kusto-data/3.1.1/kusto-data-3.1.1.jar +0 -0
- data/lib/com/microsoft/azure/kusto/kusto-ingest/3.1.1/kusto-ingest-3.1.1.jar +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 40d2d10177bc99b9da270d95a843ca5490689ac6a9bbdd548744d041e29a557c
|
|
4
|
+
data.tar.gz: b5f85499389d432eb0e60e5be23bb75ea7b9c5c43df6a1d1a7e07f3b5fd44c57
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 82e08571c9c68c67dd7e2c8f653cfce159b01902cee31982ddefa9cea59769510acb25ebf5d3ae2ed18354a7ccae1ba0dcc6b6135940a706447b017d80d520fb
|
|
7
|
+
data.tar.gz: b3c7c5860bfe0fea8e7da8d5e5356a56b6dd54733214f46c6492e452580c635da637be223a80723db05c2c496edda4d8b607ebe47138a582b0b1fc8be7e5bd4a
|
data/CHANGELOG.md
CHANGED
|
@@ -28,4 +28,10 @@
|
|
|
28
28
|
# 1.0.0
|
|
29
29
|
|
|
30
30
|
- Use stable (2.1.2) version of the java sdk, and retrieve it from maven with bundler.
|
|
31
|
-
- Renamed `mapping` to `json_mapping` in order to clarify usage. `mapping` still remains as a deprecated parameter.
|
|
31
|
+
- Renamed `mapping` to `json_mapping` in order to clarify usage. `mapping` still remains as a deprecated parameter.
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
# 1.0.5
|
|
35
|
+
|
|
36
|
+
- Use (3.1.3) version of the java sdk, and retrieve it from maven with bundler.
|
|
37
|
+
- Added support for `proxy_host` `proxy_port` `proxy_protocol` to support proxying ingestion to Kusto
|
data/README.md
CHANGED
|
@@ -39,6 +39,9 @@ output {
|
|
|
39
39
|
database => "<database name>"
|
|
40
40
|
table => "<target table>"
|
|
41
41
|
json_mapping => "<mapping name>"
|
|
42
|
+
proxy_host => "<proxy host>"
|
|
43
|
+
proxy_port => <proxy port>
|
|
44
|
+
proxy_protocol => <"http"|"https">
|
|
42
45
|
}
|
|
43
46
|
}
|
|
44
47
|
```
|
|
@@ -57,6 +60,16 @@ More information about configuring Logstash can be found in the [logstash config
|
|
|
57
60
|
| **recovery** | If set to true (default), plugin will attempt to resend pre-existing temp files found in the path upon startup | |
|
|
58
61
|
| **delete_temp_files** | Determines if temp files will be deleted after a successful upload (true is default; set false for debug purposes only)| |
|
|
59
62
|
| **flush_interval** | The time (in seconds) for flushing writes to temporary files. Default is 2 seconds, 0 will flush on every event. Increase this value to reduce IO calls but keep in mind that events in the buffer will be lost in case of abrupt failure.| |
|
|
63
|
+
| **proxy_host** | The proxy hostname for redirecting traffic to Kusto.| |
|
|
64
|
+
| **proxy_port** | The proxy port for the proxy. Defaults to 80.| |
|
|
65
|
+
| **proxy_protocol** | The proxy server protocol , is one of http or https.| |
|
|
66
|
+
|
|
67
|
+
> Note : LS_JAVA_OPTS can be used to set proxy parameters as well (using export or SET options)
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
export LS_JAVA_OPTS="-Dhttp.proxyHost=1.2.34 -Dhttp.proxyPort=8989 -Dhttps.proxyHost=1.2.3.4 -Dhttps.proxyPort=8989"
|
|
71
|
+
```
|
|
72
|
+
|
|
60
73
|
|
|
61
74
|
## Development Requirements
|
|
62
75
|
|
data/SECURITY.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
<!-- BEGIN MICROSOFT SECURITY.MD V0.0.7 BLOCK -->
|
|
2
|
+
|
|
3
|
+
## Security
|
|
4
|
+
|
|
5
|
+
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
|
|
6
|
+
|
|
7
|
+
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below.
|
|
8
|
+
|
|
9
|
+
## Reporting Security Issues
|
|
10
|
+
|
|
11
|
+
**Please do not report security vulnerabilities through public GitHub issues.**
|
|
12
|
+
|
|
13
|
+
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report).
|
|
14
|
+
|
|
15
|
+
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
|
|
16
|
+
|
|
17
|
+
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
|
|
18
|
+
|
|
19
|
+
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
|
|
20
|
+
|
|
21
|
+
* Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
|
|
22
|
+
* Full paths of source file(s) related to the manifestation of the issue
|
|
23
|
+
* The location of the affected source code (tag/branch/commit or direct URL)
|
|
24
|
+
* Any special configuration required to reproduce the issue
|
|
25
|
+
* Step-by-step instructions to reproduce the issue
|
|
26
|
+
* Proof-of-concept or exploit code (if possible)
|
|
27
|
+
* Impact of the issue, including how an attacker might exploit the issue
|
|
28
|
+
|
|
29
|
+
This information will help us triage your report more quickly.
|
|
30
|
+
|
|
31
|
+
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs.
|
|
32
|
+
|
|
33
|
+
## Preferred Languages
|
|
34
|
+
|
|
35
|
+
We prefer all communications to be in English.
|
|
36
|
+
|
|
37
|
+
## Policy
|
|
38
|
+
|
|
39
|
+
Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd).
|
|
40
|
+
|
|
41
|
+
<!-- END MICROSOFT SECURITY.MD BLOCK -->
|
|
Binary file
|
|
@@ -20,23 +20,30 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
20
20
|
LOW_QUEUE_LENGTH = 3
|
|
21
21
|
FIELD_REF = /%\{[^}]+\}/
|
|
22
22
|
|
|
23
|
-
def initialize(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, logger, threadpool = DEFAULT_THREADPOOL)
|
|
23
|
+
def initialize(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host , proxy_port , proxy_protocol,logger, threadpool = DEFAULT_THREADPOOL)
|
|
24
24
|
@workers_pool = threadpool
|
|
25
25
|
@logger = logger
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
@logger.debug('Preparing Kusto resources.')
|
|
26
|
+
validate_config(database, table, json_mapping,proxy_protocol)
|
|
27
|
+
@logger.info('Preparing Kusto resources.')
|
|
30
28
|
|
|
31
29
|
kusto_java = Java::com.microsoft.azure.kusto
|
|
30
|
+
apache_http = Java::org.apache.http
|
|
32
31
|
kusto_connection_string = kusto_java.data.auth.ConnectionStringBuilder.createWithAadApplicationCredentials(ingest_url, app_id, app_key.value, app_tenant)
|
|
32
|
+
#
|
|
33
33
|
@logger.debug(Gem.loaded_specs.to_s)
|
|
34
34
|
# Unfortunately there's no way to avoid using the gem/plugin name directly...
|
|
35
35
|
name_for_tracing = "logstash-output-kusto:#{Gem.loaded_specs['logstash-output-kusto']&.version || "unknown"}"
|
|
36
36
|
@logger.debug("Client name for tracing: #{name_for_tracing}")
|
|
37
37
|
kusto_connection_string.setClientVersionForTracing(name_for_tracing)
|
|
38
|
-
|
|
39
|
-
@kusto_client =
|
|
38
|
+
|
|
39
|
+
@kusto_client = begin
|
|
40
|
+
if proxy_host.nil? || proxy_host.empty?
|
|
41
|
+
kusto_java.ingest.IngestClientFactory.createClient(kusto_connection_string)
|
|
42
|
+
else
|
|
43
|
+
kusto_http_client_properties = kusto_java.data.HttpClientProperties.builder().proxy(apache_http.HttpHost.new(proxy_host,proxy_port,proxy_protocol)).build()
|
|
44
|
+
kusto_java.ingest.IngestClientFactory.createClient(kusto_connection_string, kusto_http_client_properties)
|
|
45
|
+
end
|
|
46
|
+
end
|
|
40
47
|
|
|
41
48
|
@ingestion_properties = kusto_java.ingest.IngestionProperties.new(database, table)
|
|
42
49
|
@ingestion_properties.setIngestionMapping(json_mapping, kusto_java.ingest.IngestionMapping::IngestionMappingKind::JSON)
|
|
@@ -46,7 +53,7 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
46
53
|
@logger.debug('Kusto resources are ready.')
|
|
47
54
|
end
|
|
48
55
|
|
|
49
|
-
def validate_config(database, table, json_mapping)
|
|
56
|
+
def validate_config(database, table, json_mapping,proxy_protocol)
|
|
50
57
|
if database =~ FIELD_REF
|
|
51
58
|
@logger.error('database config value should not be dynamic.', database)
|
|
52
59
|
raise LogStash::ConfigurationError.new('database config value should not be dynamic.')
|
|
@@ -61,6 +68,12 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
61
68
|
@logger.error('json_mapping config value should not be dynamic.', json_mapping)
|
|
62
69
|
raise LogStash::ConfigurationError.new('json_mapping config value should not be dynamic.')
|
|
63
70
|
end
|
|
71
|
+
|
|
72
|
+
if not(["https", "http"].include? proxy_protocol)
|
|
73
|
+
@logger.error('proxy_protocol has to be http or https.', proxy_protocol)
|
|
74
|
+
raise LogStash::ConfigurationError.new('proxy_protocol has to be http or https.')
|
|
75
|
+
end
|
|
76
|
+
|
|
64
77
|
end
|
|
65
78
|
|
|
66
79
|
def upload_async(path, delete_on_success)
|
|
@@ -95,11 +108,13 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
95
108
|
# local_ingestion_properties.addJsonMappingName(json_mapping)
|
|
96
109
|
# end
|
|
97
110
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
111
|
+
if file_size > 0
|
|
112
|
+
file_source_info = Java::com.microsoft.azure.kusto.ingest.source.FileSourceInfo.new(path, 0); # 0 - let the sdk figure out the size of the file
|
|
113
|
+
@kusto_client.ingestFromFile(file_source_info, @ingestion_properties)
|
|
114
|
+
else
|
|
115
|
+
@logger.warn("File #{path} is an empty file and is not ingested.")
|
|
116
|
+
end
|
|
101
117
|
File.delete(path) if delete_on_success
|
|
102
|
-
|
|
103
118
|
@logger.debug("File #{path} sent to kusto.")
|
|
104
119
|
rescue Errno::ENOENT => e
|
|
105
120
|
@logger.error("File doesn't exist! Unrecoverable error.", exception: e.class, message: e.message, path: path, backtrace: e.backtrace)
|
|
@@ -88,7 +88,7 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
88
88
|
# Note that this must be in JSON format, as this is the interface between Logstash and Kusto
|
|
89
89
|
config :json_mapping, validate: :string, required: true
|
|
90
90
|
|
|
91
|
-
#
|
|
91
|
+
# Mapping name - deprecated, use json_mapping
|
|
92
92
|
config :mapping, validate: :string, deprecated: true
|
|
93
93
|
|
|
94
94
|
|
|
@@ -106,6 +106,15 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
106
106
|
# starts processing them in the main thread (not healthy)
|
|
107
107
|
config :upload_queue_size, validate: :number, default: 30
|
|
108
108
|
|
|
109
|
+
# Host of the proxy , is an optional field. Can connect directly
|
|
110
|
+
config :proxy_host, validate: :string, required: false
|
|
111
|
+
|
|
112
|
+
# Port where the proxy runs , defaults to 80. Usually a value like 3128
|
|
113
|
+
config :proxy_port, validate: :number, required: false , default: 80
|
|
114
|
+
|
|
115
|
+
# Check Proxy URL can be over http or https. Dowe need it this way or ignore this & remove this
|
|
116
|
+
config :proxy_protocol, validate: :string, required: false , default: 'http'
|
|
117
|
+
|
|
109
118
|
default :codec, 'json_lines'
|
|
110
119
|
|
|
111
120
|
def register
|
|
@@ -141,7 +150,7 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
141
150
|
max_queue: upload_queue_size,
|
|
142
151
|
fallback_policy: :caller_runs)
|
|
143
152
|
|
|
144
|
-
@ingestor = Ingestor.new(ingest_url, app_id, app_key, app_tenant, database, table, final_mapping, delete_temp_files, @logger, executor)
|
|
153
|
+
@ingestor = Ingestor.new(ingest_url, app_id, app_key, app_tenant, database, table, final_mapping, delete_temp_files, proxy_host, proxy_port,proxy_protocol, @logger, executor)
|
|
145
154
|
|
|
146
155
|
# send existing files
|
|
147
156
|
recover_past_files if recovery
|
|
@@ -4,7 +4,9 @@ begin
|
|
|
4
4
|
rescue LoadError
|
|
5
5
|
require 'io/netty/netty-resolver-dns-native-macos/4.1.68.Final/netty-resolver-dns-native-macos-4.1.68.Final-osx-x86_64.jar'
|
|
6
6
|
require 'com/google/guava/guava/24.1.1-jre/guava-24.1.1-jre.jar'
|
|
7
|
+
require 'com/microsoft/azure/kusto/kusto-data/3.1.3/kusto-data-3.1.3.jar'
|
|
7
8
|
require 'com/microsoft/azure/azure-storage/8.6.6/azure-storage-8.6.6.jar'
|
|
9
|
+
require 'com/microsoft/azure/kusto/kusto-ingest/3.1.3/kusto-ingest-3.1.3.jar'
|
|
8
10
|
require 'io/projectreactor/netty/reactor-netty-http/1.0.11/reactor-netty-http-1.0.11.jar'
|
|
9
11
|
require 'net/java/dev/jna/jna/5.5.0/jna-5.5.0.jar'
|
|
10
12
|
require 'io/netty/netty-handler-proxy/4.1.68.Final/netty-handler-proxy-4.1.68.Final.jar'
|
|
@@ -31,7 +33,6 @@ rescue LoadError
|
|
|
31
33
|
require 'jakarta/activation/jakarta.activation-api/1.2.1/jakarta.activation-api-1.2.1.jar'
|
|
32
34
|
require 'commons-logging/commons-logging/1.2/commons-logging-1.2.jar'
|
|
33
35
|
require 'net/minidev/accessors-smart/2.4.7/accessors-smart-2.4.7.jar'
|
|
34
|
-
require 'com/microsoft/azure/kusto/kusto-ingest/3.1.1/kusto-ingest-3.1.1.jar'
|
|
35
36
|
require 'io/netty/netty-codec/4.1.68.Final/netty-codec-4.1.68.Final.jar'
|
|
36
37
|
require 'commons-codec/commons-codec/1.11/commons-codec-1.11.jar'
|
|
37
38
|
require 'com/microsoft/azure/msal4j-persistence-extension/1.1.0/msal4j-persistence-extension-1.1.0.jar'
|
|
@@ -43,7 +44,6 @@ rescue LoadError
|
|
|
43
44
|
require 'com/fasterxml/jackson/core/jackson-annotations/2.13.0/jackson-annotations-2.13.0.jar'
|
|
44
45
|
require 'com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar'
|
|
45
46
|
require 'org/checkerframework/checker-compat-qual/2.0.0/checker-compat-qual-2.0.0.jar'
|
|
46
|
-
require 'com/microsoft/azure/kusto/kusto-data/3.1.1/kusto-data-3.1.1.jar'
|
|
47
47
|
require 'org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar'
|
|
48
48
|
require 'com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar'
|
|
49
49
|
require 'io/netty/netty-codec-http/4.1.68.Final/netty-codec-http-4.1.68.Final.jar'
|
|
@@ -78,7 +78,9 @@ end
|
|
|
78
78
|
if defined? Jars
|
|
79
79
|
require_jar 'io.netty', 'netty-resolver-dns-native-macos', 'osx-x86_64', '4.1.68.Final'
|
|
80
80
|
require_jar 'com.google.guava', 'guava', '24.1.1-jre'
|
|
81
|
+
require_jar 'com.microsoft.azure.kusto', 'kusto-data', '3.1.3'
|
|
81
82
|
require_jar 'com.microsoft.azure', 'azure-storage', '8.6.6'
|
|
83
|
+
require_jar 'com.microsoft.azure.kusto', 'kusto-ingest', '3.1.3'
|
|
82
84
|
require_jar 'io.projectreactor.netty', 'reactor-netty-http', '1.0.11'
|
|
83
85
|
require_jar 'net.java.dev.jna', 'jna', '5.5.0'
|
|
84
86
|
require_jar 'io.netty', 'netty-handler-proxy', '4.1.68.Final'
|
|
@@ -105,7 +107,6 @@ if defined? Jars
|
|
|
105
107
|
require_jar 'jakarta.activation', 'jakarta.activation-api', '1.2.1'
|
|
106
108
|
require_jar 'commons-logging', 'commons-logging', '1.2'
|
|
107
109
|
require_jar 'net.minidev', 'accessors-smart', '2.4.7'
|
|
108
|
-
require_jar 'com.microsoft.azure.kusto', 'kusto-ingest', '3.1.1'
|
|
109
110
|
require_jar 'io.netty', 'netty-codec', '4.1.68.Final'
|
|
110
111
|
require_jar 'commons-codec', 'commons-codec', '1.11'
|
|
111
112
|
require_jar 'com.microsoft.azure', 'msal4j-persistence-extension', '1.1.0'
|
|
@@ -117,7 +118,6 @@ if defined? Jars
|
|
|
117
118
|
require_jar 'com.fasterxml.jackson.core', 'jackson-annotations', '2.13.0'
|
|
118
119
|
require_jar 'com.google.code.findbugs', 'jsr305', '1.3.9'
|
|
119
120
|
require_jar 'org.checkerframework', 'checker-compat-qual', '2.0.0'
|
|
120
|
-
require_jar 'com.microsoft.azure.kusto', 'kusto-data', '3.1.1'
|
|
121
121
|
require_jar 'org.apache.httpcomponents', 'httpclient', '4.5.13'
|
|
122
122
|
require_jar 'com.google.j2objc', 'j2objc-annotations', '1.1'
|
|
123
123
|
require_jar 'io.netty', 'netty-codec-http', '4.1.68.Final'
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Gem::Specification.new do |s|
|
|
2
2
|
s.name = 'logstash-output-kusto' #WATCH OUT: we hardcoded usage of this name in one of the classes.
|
|
3
|
-
s.version = '1.0.
|
|
3
|
+
s.version = '1.0.5'
|
|
4
4
|
s.licenses = ['Apache-2.0']
|
|
5
5
|
s.summary = 'Writes events to Azure Data Explorer (Kusto)'
|
|
6
6
|
s.description = 'This is a logstash output plugin used to write events to an Azure Data Explorer (a.k.a Kusto)'
|
|
@@ -31,6 +31,6 @@ Gem::Specification.new do |s|
|
|
|
31
31
|
s.add_development_dependency 'rspec_junit_formatter'
|
|
32
32
|
|
|
33
33
|
# Jar dependencies
|
|
34
|
-
s.requirements << "jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.
|
|
34
|
+
s.requirements << "jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.3'"
|
|
35
35
|
s.add_runtime_dependency 'jar-dependencies'
|
|
36
36
|
end
|
|
@@ -11,6 +11,9 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
11
11
|
let(:app_tenant) { "mytenant" }
|
|
12
12
|
let(:database) { "mydatabase" }
|
|
13
13
|
let(:table) { "mytable" }
|
|
14
|
+
let(:proxy_host) { "localhost" }
|
|
15
|
+
let(:proxy_port) { 80 }
|
|
16
|
+
let(:proxy_protocol) { "http" }
|
|
14
17
|
let(:json_mapping) { "mymapping" }
|
|
15
18
|
let(:delete_local) { false }
|
|
16
19
|
let(:logger) { spy('logger') }
|
|
@@ -21,7 +24,7 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
21
24
|
# note that this will cause an internal error since connection is being tried.
|
|
22
25
|
# however we still want to test that all the java stuff is working as expected
|
|
23
26
|
expect {
|
|
24
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, logger)
|
|
27
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol, logger)
|
|
25
28
|
ingestor.stop
|
|
26
29
|
}.not_to raise_error
|
|
27
30
|
end
|
|
@@ -32,7 +35,7 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
32
35
|
dynamic_name_array.each do |test_database|
|
|
33
36
|
it "with database: #{test_database}" do
|
|
34
37
|
expect {
|
|
35
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, test_database, table, json_mapping, delete_local, logger)
|
|
38
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, test_database, table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol,logger)
|
|
36
39
|
ingestor.stop
|
|
37
40
|
}.to raise_error(LogStash::ConfigurationError)
|
|
38
41
|
end
|
|
@@ -43,7 +46,7 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
43
46
|
dynamic_name_array.each do |test_table|
|
|
44
47
|
it "with database: #{test_table}" do
|
|
45
48
|
expect {
|
|
46
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, test_table, json_mapping, delete_local, logger)
|
|
49
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, test_table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol,logger)
|
|
47
50
|
ingestor.stop
|
|
48
51
|
}.to raise_error(LogStash::ConfigurationError)
|
|
49
52
|
end
|
|
@@ -51,16 +54,25 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
51
54
|
end
|
|
52
55
|
|
|
53
56
|
context 'doesnt allow mapping to have some dynamic part' do
|
|
54
|
-
dynamic_name_array.each do |
|
|
55
|
-
it "with database: #{
|
|
57
|
+
dynamic_name_array.each do |json_mapping|
|
|
58
|
+
it "with database: #{json_mapping}" do
|
|
56
59
|
expect {
|
|
57
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table,
|
|
60
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol,logger)
|
|
58
61
|
ingestor.stop
|
|
59
62
|
}.to raise_error(LogStash::ConfigurationError)
|
|
60
63
|
end
|
|
61
64
|
end
|
|
62
65
|
end
|
|
63
66
|
|
|
67
|
+
context 'proxy protocol has to be http or https' do
|
|
68
|
+
it "with proxy protocol: socks" do
|
|
69
|
+
expect {
|
|
70
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host, proxy_port,'socks',logger)
|
|
71
|
+
ingestor.stop
|
|
72
|
+
}.to raise_error(LogStash::ConfigurationError)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
|
|
64
76
|
end
|
|
65
77
|
|
|
66
78
|
# describe 'receiving events' do
|
data/spec/outputs/kusto_spec.rb
CHANGED
|
@@ -12,7 +12,10 @@ describe LogStash::Outputs::Kusto do
|
|
|
12
12
|
"app_tenant" => "mytenant",
|
|
13
13
|
"database" => "mydatabase",
|
|
14
14
|
"table" => "mytable",
|
|
15
|
-
"json_mapping" => "mymapping"
|
|
15
|
+
"json_mapping" => "mymapping",
|
|
16
|
+
"proxy_host" => "localhost",
|
|
17
|
+
"proxy_port" => 3128,
|
|
18
|
+
"proxy_protocol" => "https"
|
|
16
19
|
} }
|
|
17
20
|
|
|
18
21
|
describe '#register' do
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: logstash-output-kusto
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.
|
|
4
|
+
version: 1.0.5
|
|
5
5
|
platform: java
|
|
6
6
|
authors:
|
|
7
7
|
- Tamir Kamara
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: bin
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2022-
|
|
12
|
+
date: 2022-08-04 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -149,6 +149,7 @@ files:
|
|
|
149
149
|
- Gemfile
|
|
150
150
|
- LICENSE
|
|
151
151
|
- README.md
|
|
152
|
+
- SECURITY.md
|
|
152
153
|
- lib/com/azure/azure-core-http-netty/1.11.1/azure-core-http-netty-1.11.1.jar
|
|
153
154
|
- lib/com/azure/azure-core/1.21.0/azure-core-1.21.0.jar
|
|
154
155
|
- lib/com/azure/azure-identity/1.3.7/azure-identity-1.3.7.jar
|
|
@@ -166,8 +167,8 @@ files:
|
|
|
166
167
|
- lib/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar
|
|
167
168
|
- lib/com/microsoft/azure/azure-keyvault-core/1.2.4/azure-keyvault-core-1.2.4.jar
|
|
168
169
|
- lib/com/microsoft/azure/azure-storage/8.6.6/azure-storage-8.6.6.jar
|
|
169
|
-
- lib/com/microsoft/azure/kusto/kusto-data/3.1.
|
|
170
|
-
- lib/com/microsoft/azure/kusto/kusto-ingest/3.1.
|
|
170
|
+
- lib/com/microsoft/azure/kusto/kusto-data/3.1.3/kusto-data-3.1.3.jar
|
|
171
|
+
- lib/com/microsoft/azure/kusto/kusto-ingest/3.1.3/kusto-ingest-3.1.3.jar
|
|
171
172
|
- lib/com/microsoft/azure/msal4j-persistence-extension/1.1.0/msal4j-persistence-extension-1.1.0.jar
|
|
172
173
|
- lib/com/microsoft/azure/msal4j/1.11.0/msal4j-1.11.0.jar
|
|
173
174
|
- lib/com/nimbusds/content-type/2.1/content-type-2.1.jar
|
|
@@ -249,7 +250,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
249
250
|
- !ruby/object:Gem::Version
|
|
250
251
|
version: '0'
|
|
251
252
|
requirements:
|
|
252
|
-
- jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.
|
|
253
|
+
- jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.3'
|
|
253
254
|
rubygems_version: 3.2.29
|
|
254
255
|
signing_key:
|
|
255
256
|
specification_version: 4
|
|
Binary file
|
|
Binary file
|