logstash-output-kusto 1.0.4-java → 1.0.5-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -1
- data/README.md +13 -0
- data/SECURITY.md +41 -0
- data/lib/com/microsoft/azure/kusto/kusto-data/3.1.3/kusto-data-3.1.3.jar +0 -0
- data/lib/com/microsoft/azure/kusto/kusto-ingest/3.1.3/kusto-ingest-3.1.3.jar +0 -0
- data/lib/logstash/outputs/kusto/ingestor.rb +27 -12
- data/lib/logstash/outputs/kusto.rb +11 -2
- data/lib/logstash-output-kusto_jars.rb +4 -4
- data/logstash-output-kusto.gemspec +2 -2
- data/spec/outputs/kusto/ingestor_spec.rb +18 -6
- data/spec/outputs/kusto_spec.rb +4 -1
- metadata +6 -5
- data/lib/com/microsoft/azure/kusto/kusto-data/3.1.1/kusto-data-3.1.1.jar +0 -0
- data/lib/com/microsoft/azure/kusto/kusto-ingest/3.1.1/kusto-ingest-3.1.1.jar +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 40d2d10177bc99b9da270d95a843ca5490689ac6a9bbdd548744d041e29a557c
|
|
4
|
+
data.tar.gz: b5f85499389d432eb0e60e5be23bb75ea7b9c5c43df6a1d1a7e07f3b5fd44c57
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 82e08571c9c68c67dd7e2c8f653cfce159b01902cee31982ddefa9cea59769510acb25ebf5d3ae2ed18354a7ccae1ba0dcc6b6135940a706447b017d80d520fb
|
|
7
|
+
data.tar.gz: b3c7c5860bfe0fea8e7da8d5e5356a56b6dd54733214f46c6492e452580c635da637be223a80723db05c2c496edda4d8b607ebe47138a582b0b1fc8be7e5bd4a
|
data/CHANGELOG.md
CHANGED
|
@@ -28,4 +28,10 @@
|
|
|
28
28
|
# 1.0.0
|
|
29
29
|
|
|
30
30
|
- Use stable (2.1.2) version of the java sdk, and retrieve it from maven with bundler.
|
|
31
|
-
- Renamed `mapping` to `json_mapping` in order to clarify usage. `mapping` still remains as a deprecated parameter.
|
|
31
|
+
- Renamed `mapping` to `json_mapping` in order to clarify usage. `mapping` still remains as a deprecated parameter.
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
# 1.0.5
|
|
35
|
+
|
|
36
|
+
- Use (3.1.3) version of the java sdk, and retrieve it from maven with bundler.
|
|
37
|
+
- Added support for `proxy_host` `proxy_port` `proxy_protocol` to support proxying ingestion to Kusto
|
data/README.md
CHANGED
|
@@ -39,6 +39,9 @@ output {
|
|
|
39
39
|
database => "<database name>"
|
|
40
40
|
table => "<target table>"
|
|
41
41
|
json_mapping => "<mapping name>"
|
|
42
|
+
proxy_host => "<proxy host>"
|
|
43
|
+
proxy_port => <proxy port>
|
|
44
|
+
proxy_protocol => <"http"|"https">
|
|
42
45
|
}
|
|
43
46
|
}
|
|
44
47
|
```
|
|
@@ -57,6 +60,16 @@ More information about configuring Logstash can be found in the [logstash config
|
|
|
57
60
|
| **recovery** | If set to true (default), plugin will attempt to resend pre-existing temp files found in the path upon startup | |
|
|
58
61
|
| **delete_temp_files** | Determines if temp files will be deleted after a successful upload (true is default; set false for debug purposes only)| |
|
|
59
62
|
| **flush_interval** | The time (in seconds) for flushing writes to temporary files. Default is 2 seconds, 0 will flush on every event. Increase this value to reduce IO calls but keep in mind that events in the buffer will be lost in case of abrupt failure.| |
|
|
63
|
+
| **proxy_host** | The proxy hostname for redirecting traffic to Kusto.| |
|
|
64
|
+
| **proxy_port** | The proxy port for the proxy. Defaults to 80.| |
|
|
65
|
+
| **proxy_protocol** | The proxy server protocol , is one of http or https.| |
|
|
66
|
+
|
|
67
|
+
> Note : LS_JAVA_OPTS can be used to set proxy parameters as well (using export or SET options)
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
export LS_JAVA_OPTS="-Dhttp.proxyHost=1.2.34 -Dhttp.proxyPort=8989 -Dhttps.proxyHost=1.2.3.4 -Dhttps.proxyPort=8989"
|
|
71
|
+
```
|
|
72
|
+
|
|
60
73
|
|
|
61
74
|
## Development Requirements
|
|
62
75
|
|
data/SECURITY.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
<!-- BEGIN MICROSOFT SECURITY.MD V0.0.7 BLOCK -->
|
|
2
|
+
|
|
3
|
+
## Security
|
|
4
|
+
|
|
5
|
+
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
|
|
6
|
+
|
|
7
|
+
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below.
|
|
8
|
+
|
|
9
|
+
## Reporting Security Issues
|
|
10
|
+
|
|
11
|
+
**Please do not report security vulnerabilities through public GitHub issues.**
|
|
12
|
+
|
|
13
|
+
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report).
|
|
14
|
+
|
|
15
|
+
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
|
|
16
|
+
|
|
17
|
+
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
|
|
18
|
+
|
|
19
|
+
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
|
|
20
|
+
|
|
21
|
+
* Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
|
|
22
|
+
* Full paths of source file(s) related to the manifestation of the issue
|
|
23
|
+
* The location of the affected source code (tag/branch/commit or direct URL)
|
|
24
|
+
* Any special configuration required to reproduce the issue
|
|
25
|
+
* Step-by-step instructions to reproduce the issue
|
|
26
|
+
* Proof-of-concept or exploit code (if possible)
|
|
27
|
+
* Impact of the issue, including how an attacker might exploit the issue
|
|
28
|
+
|
|
29
|
+
This information will help us triage your report more quickly.
|
|
30
|
+
|
|
31
|
+
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs.
|
|
32
|
+
|
|
33
|
+
## Preferred Languages
|
|
34
|
+
|
|
35
|
+
We prefer all communications to be in English.
|
|
36
|
+
|
|
37
|
+
## Policy
|
|
38
|
+
|
|
39
|
+
Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd).
|
|
40
|
+
|
|
41
|
+
<!-- END MICROSOFT SECURITY.MD BLOCK -->
|
|
Binary file
|
|
@@ -20,23 +20,30 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
20
20
|
LOW_QUEUE_LENGTH = 3
|
|
21
21
|
FIELD_REF = /%\{[^}]+\}/
|
|
22
22
|
|
|
23
|
-
def initialize(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, logger, threadpool = DEFAULT_THREADPOOL)
|
|
23
|
+
def initialize(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host , proxy_port , proxy_protocol,logger, threadpool = DEFAULT_THREADPOOL)
|
|
24
24
|
@workers_pool = threadpool
|
|
25
25
|
@logger = logger
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
@logger.debug('Preparing Kusto resources.')
|
|
26
|
+
validate_config(database, table, json_mapping,proxy_protocol)
|
|
27
|
+
@logger.info('Preparing Kusto resources.')
|
|
30
28
|
|
|
31
29
|
kusto_java = Java::com.microsoft.azure.kusto
|
|
30
|
+
apache_http = Java::org.apache.http
|
|
32
31
|
kusto_connection_string = kusto_java.data.auth.ConnectionStringBuilder.createWithAadApplicationCredentials(ingest_url, app_id, app_key.value, app_tenant)
|
|
32
|
+
#
|
|
33
33
|
@logger.debug(Gem.loaded_specs.to_s)
|
|
34
34
|
# Unfortunately there's no way to avoid using the gem/plugin name directly...
|
|
35
35
|
name_for_tracing = "logstash-output-kusto:#{Gem.loaded_specs['logstash-output-kusto']&.version || "unknown"}"
|
|
36
36
|
@logger.debug("Client name for tracing: #{name_for_tracing}")
|
|
37
37
|
kusto_connection_string.setClientVersionForTracing(name_for_tracing)
|
|
38
|
-
|
|
39
|
-
@kusto_client =
|
|
38
|
+
|
|
39
|
+
@kusto_client = begin
|
|
40
|
+
if proxy_host.nil? || proxy_host.empty?
|
|
41
|
+
kusto_java.ingest.IngestClientFactory.createClient(kusto_connection_string)
|
|
42
|
+
else
|
|
43
|
+
kusto_http_client_properties = kusto_java.data.HttpClientProperties.builder().proxy(apache_http.HttpHost.new(proxy_host,proxy_port,proxy_protocol)).build()
|
|
44
|
+
kusto_java.ingest.IngestClientFactory.createClient(kusto_connection_string, kusto_http_client_properties)
|
|
45
|
+
end
|
|
46
|
+
end
|
|
40
47
|
|
|
41
48
|
@ingestion_properties = kusto_java.ingest.IngestionProperties.new(database, table)
|
|
42
49
|
@ingestion_properties.setIngestionMapping(json_mapping, kusto_java.ingest.IngestionMapping::IngestionMappingKind::JSON)
|
|
@@ -46,7 +53,7 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
46
53
|
@logger.debug('Kusto resources are ready.')
|
|
47
54
|
end
|
|
48
55
|
|
|
49
|
-
def validate_config(database, table, json_mapping)
|
|
56
|
+
def validate_config(database, table, json_mapping,proxy_protocol)
|
|
50
57
|
if database =~ FIELD_REF
|
|
51
58
|
@logger.error('database config value should not be dynamic.', database)
|
|
52
59
|
raise LogStash::ConfigurationError.new('database config value should not be dynamic.')
|
|
@@ -61,6 +68,12 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
61
68
|
@logger.error('json_mapping config value should not be dynamic.', json_mapping)
|
|
62
69
|
raise LogStash::ConfigurationError.new('json_mapping config value should not be dynamic.')
|
|
63
70
|
end
|
|
71
|
+
|
|
72
|
+
if not(["https", "http"].include? proxy_protocol)
|
|
73
|
+
@logger.error('proxy_protocol has to be http or https.', proxy_protocol)
|
|
74
|
+
raise LogStash::ConfigurationError.new('proxy_protocol has to be http or https.')
|
|
75
|
+
end
|
|
76
|
+
|
|
64
77
|
end
|
|
65
78
|
|
|
66
79
|
def upload_async(path, delete_on_success)
|
|
@@ -95,11 +108,13 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
95
108
|
# local_ingestion_properties.addJsonMappingName(json_mapping)
|
|
96
109
|
# end
|
|
97
110
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
111
|
+
if file_size > 0
|
|
112
|
+
file_source_info = Java::com.microsoft.azure.kusto.ingest.source.FileSourceInfo.new(path, 0); # 0 - let the sdk figure out the size of the file
|
|
113
|
+
@kusto_client.ingestFromFile(file_source_info, @ingestion_properties)
|
|
114
|
+
else
|
|
115
|
+
@logger.warn("File #{path} is an empty file and is not ingested.")
|
|
116
|
+
end
|
|
101
117
|
File.delete(path) if delete_on_success
|
|
102
|
-
|
|
103
118
|
@logger.debug("File #{path} sent to kusto.")
|
|
104
119
|
rescue Errno::ENOENT => e
|
|
105
120
|
@logger.error("File doesn't exist! Unrecoverable error.", exception: e.class, message: e.message, path: path, backtrace: e.backtrace)
|
|
@@ -88,7 +88,7 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
88
88
|
# Note that this must be in JSON format, as this is the interface between Logstash and Kusto
|
|
89
89
|
config :json_mapping, validate: :string, required: true
|
|
90
90
|
|
|
91
|
-
#
|
|
91
|
+
# Mapping name - deprecated, use json_mapping
|
|
92
92
|
config :mapping, validate: :string, deprecated: true
|
|
93
93
|
|
|
94
94
|
|
|
@@ -106,6 +106,15 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
106
106
|
# starts processing them in the main thread (not healthy)
|
|
107
107
|
config :upload_queue_size, validate: :number, default: 30
|
|
108
108
|
|
|
109
|
+
# Host of the proxy , is an optional field. Can connect directly
|
|
110
|
+
config :proxy_host, validate: :string, required: false
|
|
111
|
+
|
|
112
|
+
# Port where the proxy runs , defaults to 80. Usually a value like 3128
|
|
113
|
+
config :proxy_port, validate: :number, required: false , default: 80
|
|
114
|
+
|
|
115
|
+
# Check Proxy URL can be over http or https. Dowe need it this way or ignore this & remove this
|
|
116
|
+
config :proxy_protocol, validate: :string, required: false , default: 'http'
|
|
117
|
+
|
|
109
118
|
default :codec, 'json_lines'
|
|
110
119
|
|
|
111
120
|
def register
|
|
@@ -141,7 +150,7 @@ class LogStash::Outputs::Kusto < LogStash::Outputs::Base
|
|
|
141
150
|
max_queue: upload_queue_size,
|
|
142
151
|
fallback_policy: :caller_runs)
|
|
143
152
|
|
|
144
|
-
@ingestor = Ingestor.new(ingest_url, app_id, app_key, app_tenant, database, table, final_mapping, delete_temp_files, @logger, executor)
|
|
153
|
+
@ingestor = Ingestor.new(ingest_url, app_id, app_key, app_tenant, database, table, final_mapping, delete_temp_files, proxy_host, proxy_port,proxy_protocol, @logger, executor)
|
|
145
154
|
|
|
146
155
|
# send existing files
|
|
147
156
|
recover_past_files if recovery
|
|
@@ -4,7 +4,9 @@ begin
|
|
|
4
4
|
rescue LoadError
|
|
5
5
|
require 'io/netty/netty-resolver-dns-native-macos/4.1.68.Final/netty-resolver-dns-native-macos-4.1.68.Final-osx-x86_64.jar'
|
|
6
6
|
require 'com/google/guava/guava/24.1.1-jre/guava-24.1.1-jre.jar'
|
|
7
|
+
require 'com/microsoft/azure/kusto/kusto-data/3.1.3/kusto-data-3.1.3.jar'
|
|
7
8
|
require 'com/microsoft/azure/azure-storage/8.6.6/azure-storage-8.6.6.jar'
|
|
9
|
+
require 'com/microsoft/azure/kusto/kusto-ingest/3.1.3/kusto-ingest-3.1.3.jar'
|
|
8
10
|
require 'io/projectreactor/netty/reactor-netty-http/1.0.11/reactor-netty-http-1.0.11.jar'
|
|
9
11
|
require 'net/java/dev/jna/jna/5.5.0/jna-5.5.0.jar'
|
|
10
12
|
require 'io/netty/netty-handler-proxy/4.1.68.Final/netty-handler-proxy-4.1.68.Final.jar'
|
|
@@ -31,7 +33,6 @@ rescue LoadError
|
|
|
31
33
|
require 'jakarta/activation/jakarta.activation-api/1.2.1/jakarta.activation-api-1.2.1.jar'
|
|
32
34
|
require 'commons-logging/commons-logging/1.2/commons-logging-1.2.jar'
|
|
33
35
|
require 'net/minidev/accessors-smart/2.4.7/accessors-smart-2.4.7.jar'
|
|
34
|
-
require 'com/microsoft/azure/kusto/kusto-ingest/3.1.1/kusto-ingest-3.1.1.jar'
|
|
35
36
|
require 'io/netty/netty-codec/4.1.68.Final/netty-codec-4.1.68.Final.jar'
|
|
36
37
|
require 'commons-codec/commons-codec/1.11/commons-codec-1.11.jar'
|
|
37
38
|
require 'com/microsoft/azure/msal4j-persistence-extension/1.1.0/msal4j-persistence-extension-1.1.0.jar'
|
|
@@ -43,7 +44,6 @@ rescue LoadError
|
|
|
43
44
|
require 'com/fasterxml/jackson/core/jackson-annotations/2.13.0/jackson-annotations-2.13.0.jar'
|
|
44
45
|
require 'com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar'
|
|
45
46
|
require 'org/checkerframework/checker-compat-qual/2.0.0/checker-compat-qual-2.0.0.jar'
|
|
46
|
-
require 'com/microsoft/azure/kusto/kusto-data/3.1.1/kusto-data-3.1.1.jar'
|
|
47
47
|
require 'org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar'
|
|
48
48
|
require 'com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar'
|
|
49
49
|
require 'io/netty/netty-codec-http/4.1.68.Final/netty-codec-http-4.1.68.Final.jar'
|
|
@@ -78,7 +78,9 @@ end
|
|
|
78
78
|
if defined? Jars
|
|
79
79
|
require_jar 'io.netty', 'netty-resolver-dns-native-macos', 'osx-x86_64', '4.1.68.Final'
|
|
80
80
|
require_jar 'com.google.guava', 'guava', '24.1.1-jre'
|
|
81
|
+
require_jar 'com.microsoft.azure.kusto', 'kusto-data', '3.1.3'
|
|
81
82
|
require_jar 'com.microsoft.azure', 'azure-storage', '8.6.6'
|
|
83
|
+
require_jar 'com.microsoft.azure.kusto', 'kusto-ingest', '3.1.3'
|
|
82
84
|
require_jar 'io.projectreactor.netty', 'reactor-netty-http', '1.0.11'
|
|
83
85
|
require_jar 'net.java.dev.jna', 'jna', '5.5.0'
|
|
84
86
|
require_jar 'io.netty', 'netty-handler-proxy', '4.1.68.Final'
|
|
@@ -105,7 +107,6 @@ if defined? Jars
|
|
|
105
107
|
require_jar 'jakarta.activation', 'jakarta.activation-api', '1.2.1'
|
|
106
108
|
require_jar 'commons-logging', 'commons-logging', '1.2'
|
|
107
109
|
require_jar 'net.minidev', 'accessors-smart', '2.4.7'
|
|
108
|
-
require_jar 'com.microsoft.azure.kusto', 'kusto-ingest', '3.1.1'
|
|
109
110
|
require_jar 'io.netty', 'netty-codec', '4.1.68.Final'
|
|
110
111
|
require_jar 'commons-codec', 'commons-codec', '1.11'
|
|
111
112
|
require_jar 'com.microsoft.azure', 'msal4j-persistence-extension', '1.1.0'
|
|
@@ -117,7 +118,6 @@ if defined? Jars
|
|
|
117
118
|
require_jar 'com.fasterxml.jackson.core', 'jackson-annotations', '2.13.0'
|
|
118
119
|
require_jar 'com.google.code.findbugs', 'jsr305', '1.3.9'
|
|
119
120
|
require_jar 'org.checkerframework', 'checker-compat-qual', '2.0.0'
|
|
120
|
-
require_jar 'com.microsoft.azure.kusto', 'kusto-data', '3.1.1'
|
|
121
121
|
require_jar 'org.apache.httpcomponents', 'httpclient', '4.5.13'
|
|
122
122
|
require_jar 'com.google.j2objc', 'j2objc-annotations', '1.1'
|
|
123
123
|
require_jar 'io.netty', 'netty-codec-http', '4.1.68.Final'
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Gem::Specification.new do |s|
|
|
2
2
|
s.name = 'logstash-output-kusto' #WATCH OUT: we hardcoded usage of this name in one of the classes.
|
|
3
|
-
s.version = '1.0.
|
|
3
|
+
s.version = '1.0.5'
|
|
4
4
|
s.licenses = ['Apache-2.0']
|
|
5
5
|
s.summary = 'Writes events to Azure Data Explorer (Kusto)'
|
|
6
6
|
s.description = 'This is a logstash output plugin used to write events to an Azure Data Explorer (a.k.a Kusto)'
|
|
@@ -31,6 +31,6 @@ Gem::Specification.new do |s|
|
|
|
31
31
|
s.add_development_dependency 'rspec_junit_formatter'
|
|
32
32
|
|
|
33
33
|
# Jar dependencies
|
|
34
|
-
s.requirements << "jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.
|
|
34
|
+
s.requirements << "jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.3'"
|
|
35
35
|
s.add_runtime_dependency 'jar-dependencies'
|
|
36
36
|
end
|
|
@@ -11,6 +11,9 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
11
11
|
let(:app_tenant) { "mytenant" }
|
|
12
12
|
let(:database) { "mydatabase" }
|
|
13
13
|
let(:table) { "mytable" }
|
|
14
|
+
let(:proxy_host) { "localhost" }
|
|
15
|
+
let(:proxy_port) { 80 }
|
|
16
|
+
let(:proxy_protocol) { "http" }
|
|
14
17
|
let(:json_mapping) { "mymapping" }
|
|
15
18
|
let(:delete_local) { false }
|
|
16
19
|
let(:logger) { spy('logger') }
|
|
@@ -21,7 +24,7 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
21
24
|
# note that this will cause an internal error since connection is being tried.
|
|
22
25
|
# however we still want to test that all the java stuff is working as expected
|
|
23
26
|
expect {
|
|
24
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, logger)
|
|
27
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol, logger)
|
|
25
28
|
ingestor.stop
|
|
26
29
|
}.not_to raise_error
|
|
27
30
|
end
|
|
@@ -32,7 +35,7 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
32
35
|
dynamic_name_array.each do |test_database|
|
|
33
36
|
it "with database: #{test_database}" do
|
|
34
37
|
expect {
|
|
35
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, test_database, table, json_mapping, delete_local, logger)
|
|
38
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, test_database, table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol,logger)
|
|
36
39
|
ingestor.stop
|
|
37
40
|
}.to raise_error(LogStash::ConfigurationError)
|
|
38
41
|
end
|
|
@@ -43,7 +46,7 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
43
46
|
dynamic_name_array.each do |test_table|
|
|
44
47
|
it "with database: #{test_table}" do
|
|
45
48
|
expect {
|
|
46
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, test_table, json_mapping, delete_local, logger)
|
|
49
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, test_table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol,logger)
|
|
47
50
|
ingestor.stop
|
|
48
51
|
}.to raise_error(LogStash::ConfigurationError)
|
|
49
52
|
end
|
|
@@ -51,16 +54,25 @@ describe LogStash::Outputs::Kusto::Ingestor do
|
|
|
51
54
|
end
|
|
52
55
|
|
|
53
56
|
context 'doesnt allow mapping to have some dynamic part' do
|
|
54
|
-
dynamic_name_array.each do |
|
|
55
|
-
it "with database: #{
|
|
57
|
+
dynamic_name_array.each do |json_mapping|
|
|
58
|
+
it "with database: #{json_mapping}" do
|
|
56
59
|
expect {
|
|
57
|
-
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table,
|
|
60
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host, proxy_port,proxy_protocol,logger)
|
|
58
61
|
ingestor.stop
|
|
59
62
|
}.to raise_error(LogStash::ConfigurationError)
|
|
60
63
|
end
|
|
61
64
|
end
|
|
62
65
|
end
|
|
63
66
|
|
|
67
|
+
context 'proxy protocol has to be http or https' do
|
|
68
|
+
it "with proxy protocol: socks" do
|
|
69
|
+
expect {
|
|
70
|
+
ingestor = described_class.new(ingest_url, app_id, app_key, app_tenant, database, table, json_mapping, delete_local, proxy_host, proxy_port,'socks',logger)
|
|
71
|
+
ingestor.stop
|
|
72
|
+
}.to raise_error(LogStash::ConfigurationError)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
|
|
64
76
|
end
|
|
65
77
|
|
|
66
78
|
# describe 'receiving events' do
|
data/spec/outputs/kusto_spec.rb
CHANGED
|
@@ -12,7 +12,10 @@ describe LogStash::Outputs::Kusto do
|
|
|
12
12
|
"app_tenant" => "mytenant",
|
|
13
13
|
"database" => "mydatabase",
|
|
14
14
|
"table" => "mytable",
|
|
15
|
-
"json_mapping" => "mymapping"
|
|
15
|
+
"json_mapping" => "mymapping",
|
|
16
|
+
"proxy_host" => "localhost",
|
|
17
|
+
"proxy_port" => 3128,
|
|
18
|
+
"proxy_protocol" => "https"
|
|
16
19
|
} }
|
|
17
20
|
|
|
18
21
|
describe '#register' do
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: logstash-output-kusto
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.
|
|
4
|
+
version: 1.0.5
|
|
5
5
|
platform: java
|
|
6
6
|
authors:
|
|
7
7
|
- Tamir Kamara
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: bin
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2022-
|
|
12
|
+
date: 2022-08-04 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -149,6 +149,7 @@ files:
|
|
|
149
149
|
- Gemfile
|
|
150
150
|
- LICENSE
|
|
151
151
|
- README.md
|
|
152
|
+
- SECURITY.md
|
|
152
153
|
- lib/com/azure/azure-core-http-netty/1.11.1/azure-core-http-netty-1.11.1.jar
|
|
153
154
|
- lib/com/azure/azure-core/1.21.0/azure-core-1.21.0.jar
|
|
154
155
|
- lib/com/azure/azure-identity/1.3.7/azure-identity-1.3.7.jar
|
|
@@ -166,8 +167,8 @@ files:
|
|
|
166
167
|
- lib/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar
|
|
167
168
|
- lib/com/microsoft/azure/azure-keyvault-core/1.2.4/azure-keyvault-core-1.2.4.jar
|
|
168
169
|
- lib/com/microsoft/azure/azure-storage/8.6.6/azure-storage-8.6.6.jar
|
|
169
|
-
- lib/com/microsoft/azure/kusto/kusto-data/3.1.
|
|
170
|
-
- lib/com/microsoft/azure/kusto/kusto-ingest/3.1.
|
|
170
|
+
- lib/com/microsoft/azure/kusto/kusto-data/3.1.3/kusto-data-3.1.3.jar
|
|
171
|
+
- lib/com/microsoft/azure/kusto/kusto-ingest/3.1.3/kusto-ingest-3.1.3.jar
|
|
171
172
|
- lib/com/microsoft/azure/msal4j-persistence-extension/1.1.0/msal4j-persistence-extension-1.1.0.jar
|
|
172
173
|
- lib/com/microsoft/azure/msal4j/1.11.0/msal4j-1.11.0.jar
|
|
173
174
|
- lib/com/nimbusds/content-type/2.1/content-type-2.1.jar
|
|
@@ -249,7 +250,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
249
250
|
- !ruby/object:Gem::Version
|
|
250
251
|
version: '0'
|
|
251
252
|
requirements:
|
|
252
|
-
- jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.
|
|
253
|
+
- jar 'com.microsoft.azure.kusto, kusto-ingest, 3.1.3'
|
|
253
254
|
rubygems_version: 3.2.29
|
|
254
255
|
signing_key:
|
|
255
256
|
specification_version: 4
|
|
Binary file
|
|
Binary file
|