azure-loganalytics-datacollector-api 0.1.4 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: f69486484bc9d5e52f6ecaf48c049ff178f85594a1ff83a9c1922db1a30666bc
4
- data.tar.gz: c4b1c23ccc77e0273b1447ae5464279270f05ca3f96d2f0e8d2d9e516d198758
3
+ metadata.gz: '0499f95d897bc46dae9b39636ac2dda0c9aa5750b70bc2703c0d3dc41bfa34f1'
4
+ data.tar.gz: 057401456546b8ac6b23e75bbc0e6c61a8a4c9ddb534f54508018011d3268d85
5
5
  SHA512:
6
- metadata.gz: 1e5dd0c8db1d44a861d2506200d7f93b668ac3d8d8a444bea73bed0e464166aa4c194f123a7482e007ed4a67da01ce5aea60933aa95cd667b04a32bdcba3591e
7
- data.tar.gz: 72d70760c358307f6b1fb8520bdb30d42e30217174c35875d236dad9799245b9aff7f4b5708a6d329a5c49adc8c73063aeec989a27afa44104c0de8986c2147e
6
+ metadata.gz: ed73c860640a73f1d0d0176e482361ebb30a082634410c44b7d540fc444b56e1a6548d1f81b43f10f5aef9967e1e8d17b63379c377250ed00b050157c29d4f79
7
+ data.tar.gz: 5bcb8d45153164a92ddeeaae37544d7722806c8726e2653d2f3fefa07e65f93f854e26e4e657cb9fef1d1d1c26d68d69daa963266b28cb62bfcaef1cbde42b29
@@ -1,5 +1,23 @@
1
+ ## 0.4.0
2
+ * restclient retries request on the following status code - [issue #10](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/10)
3
+ * 429 - Too Many Requests
4
+ * 500 - Internal Server Error
5
+ * 503 - Service Unavailable
6
+
7
+ ## 0.3.0
8
+ * Enhance log type validation: check not only alpha but also numeric, underscore, and character length (may not exceed 100) - [issue #11](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/11)
9
+
10
+ ## 0.2.0
11
+ * Support setting the x-ms-AzureResourceId Header - [issue #8](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/8)
12
+
13
+ ## 0.1.6
14
+ * fix CVE-2020-8130 - [issue #7](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/7)
15
+
16
+ ## 0.1.5
17
+ * Add endpoint parameter with the default value for Azure public - [PR#5](https://github.com/yokawasa/azure-log-analytics-data-collector/pull/5)
18
+
1
19
  ## 0.1.4
2
- * Add `set_proxy` method to client - [issue#2](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/2)
20
+ * Add `set_proxy` method to client - [issue#3](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/3)
3
21
 
4
22
  ## 0.1.2
5
23
  * fixedup bug - [issue #1](https://github.com/yokawasa/azure-log-analytics-data-collector/issues/1)
data/README.md CHANGED
@@ -1,5 +1,23 @@
1
1
  # azure-log-analytics-data-collector Ruby Client
2
- Azure Log Analytics Data Collector API Ruby Client
2
+
3
+ [Azure Log Analytics Data Collector API](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-collector-api) Client Libraries for Ruby. The repository was originally created for multiple programming languages, but it was refactored as a dedicated one for Ruby client. Python and PHP client libraries were moved to [azure-log-analytics-data-colloector-python](https://github.com/yokawasa/azure-log-analytics-data-collector-python) and [azure-log-analytics-data-colloector-php](https://github.com/yokawasa/azure-log-analytics-data-collector-php) respectively.
4
+
5
+
6
+ ## Retry policy
7
+
8
+ The client internal leverage [rest-client] to send HTTP request to the API. The client library retries request using the rest-client on the following status code (which is [recommended action](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-collector-api)).
9
+ * `429` - Too Many Requests
10
+ * `500` - Internal Server Error
11
+ * `503` - Service Unavailable
12
+
13
+ By default, the client library retres for a total of `3` times, sleeping `5 sec` between retries. The number of retries and sleeping time between retries can be changed with `set_retries` in the client class.
14
+
15
+ ```ruby
16
+ def set_retries(max_retries, retry_sleep_period)
17
+ @max_retries = max_retries
18
+ @retry_sleep_period = retry_sleep_period
19
+ end
20
+ ```
3
21
 
4
22
  ## Installation
5
23
  ```bash
@@ -78,7 +96,51 @@ else
78
96
  end
79
97
  ```
80
98
 
81
- ### Sample3 - use proxy to access the API
99
+ ### Sample3 - With time_generated_field and azure_resource_id option
100
+ Supported setting azure_resource_id option from version [0.2.0](https://github.com/yokawasa/azure-log-analytics-data-collector/releases/tag/v0.2.0)
101
+ ```ruby
102
+ require "azure/loganalytics/datacollectorapi/client"
103
+
104
+ customer_id = '<Customer ID aka WorkspaceID String>'
105
+ shared_key = '<The primary or the secondary Connected Sources client authentication key>'
106
+ log_type = "MyCustomLog"
107
+
108
+ posting_records = []
109
+ record1= {
110
+ :string => "MyText1",
111
+ :boolean => true,
112
+ :number => 100,
113
+ :timegen => "2017-11-23T11:13:35.576Z" # YYYY-MM-DDThh:mm:ssZ
114
+ }
115
+ record2= {
116
+ :string => "MyText2",
117
+ :boolean => false,
118
+ :number => 200,
119
+ :timegen => "2017-11-23T12:13:35.576Z" # YYYY-MM-DDThh:mm:ssZ
120
+ }
121
+ posting_records.push(record1)
122
+ posting_records.push(record2)
123
+
124
+ time_generated_field = "timegen"
125
+
126
+ # Azure Resource ID
127
+ # [Azure Resource ID Format]
128
+ # /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}
129
+ azure_resource_id ="/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/otherResourceGroup/providers/Microsoft.Storage/storageAccounts/examplestorage"
130
+
131
+ client=Azure::Loganalytics::Datacollectorapi::Client::new( customer_id, shared_key)
132
+ res = client.post_data(log_type, posting_records, time_generated_field, azure_resource_id)
133
+ puts res
134
+ puts "res code=#{res.code}"
135
+
136
+ if Azure::Loganalytics::Datacollectorapi::Client.is_success(res)
137
+ puts "operation was succeeded!"
138
+ else
139
+ puts "operation was failured!"
140
+ end
141
+ ```
142
+
143
+ ### Sample4 - use proxy to access the API
82
144
  ```ruby
83
145
  require "azure/loganalytics/datacollectorapi/client"
84
146
 
@@ -114,4 +176,3 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/yokawa
114
176
  ## License
115
177
 
116
178
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
117
-
@@ -0,0 +1,12 @@
1
+ # Sample codes for azure-log-analytics-data-collector
2
+ [Azure Log Analytics HTTP Data Collecotr API](https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-collector-api) Client libraries
3
+
4
+ ## Client libraries and Sample code
5
+
6
+ | Language | Source Code | Project URL |
7
+ | ------------- | ------------- |------------- |
8
+ | Ruby | This repository | [Rubygem Package](https://rubygems.org/gems/azure-loganalytics-datacollector-api) |
9
+ | Python | [Source code](https://github.com/yokawasa/azure-log-analytics-data-collector-python) | [Python Package](https://pypi.python.org/pypi/azure-log-analytics-data-collector-api) |
10
+ | PHP | [Source code](https://github.com/yokawasa/azure-log-analytics-data-collector-php) | NONE |
11
+ | CSharp | [Sample Code](https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-collector-api#sample-requests) | NONE |
12
+ | PowerShell | [Sample Code](https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-collector-api#sample-requests) | NONE |
@@ -7,7 +7,7 @@ Gem::Specification.new do |spec|
7
7
  spec.name = "azure-loganalytics-datacollector-api"
8
8
  spec.version = Azure::Loganalytics::Datacollectorapi::VERSION
9
9
  spec.authors = ["Yoichi Kawasaki"]
10
- spec.email = ["yoichi.kawasaki@outlook.com"]
10
+ spec.email = ["yokawasa@gmail.com"]
11
11
  spec.summary = %q{Azure Log Analytics Data Collector API Ruby Client}
12
12
  spec.description = spec.summary
13
13
  spec.homepage = "https://github.com/yokawasa/azure-log-analytics-data-collector"
@@ -20,7 +20,6 @@ Gem::Specification.new do |spec|
20
20
  spec.require_paths = ["lib"]
21
21
 
22
22
  spec.add_dependency "rest-client"
23
- spec.add_development_dependency "bundler", "~> 1.13"
24
- spec.add_development_dependency "rake", "~> 10.0"
23
+ spec.add_development_dependency "rake", ">= 12.3.3"
25
24
  spec.add_development_dependency "rspec", "~> 3.0"
26
25
  end
@@ -8,7 +8,10 @@ module Azure
8
8
 
9
9
  class Client
10
10
 
11
- def initialize (customer_id, shared_key)
11
+ DEFAUT_MAX_RETRIES = 3.freeze
12
+ DEFAULT_RETRY_SLEEP_PERIOD = 5.freeze
13
+
14
+ def initialize (customer_id, shared_key, endpoint ='ods.opinsights.azure.com')
12
15
  require 'rest-client'
13
16
  require 'json'
14
17
  require 'openssl'
@@ -17,15 +20,20 @@ module Azure
17
20
 
18
21
  @customer_id = customer_id
19
22
  @shared_key = shared_key
23
+ @endpoint = endpoint
24
+ @default_azure_resource_id = ''
25
+
26
+ @max_retries = DEFAUT_MAX_RETRIES
27
+ @retry_sleep_period = DEFAULT_RETRY_SLEEP_PERIOD
20
28
  end
21
29
 
22
- def post_data(log_type, json_records, record_timestamp ='')
30
+ def post_data(log_type, json_records, record_timestamp ='', azure_resource_id ='' )
23
31
  raise ConfigError, 'no log_type' if log_type.empty?
24
- raise ConfigError, 'log_type must be only alpha characters' if not is_alpha(log_type)
32
+ raise ConfigError, 'log_type must only contain alpha numeric and _, and not exceed 100 chars' if not is_valid_log_type(log_type)
25
33
  raise ConfigError, 'no json_records' if json_records.empty?
26
34
  body = json_records.to_json
27
- uri = sprintf("https://%s.ods.opinsights.azure.com/api/logs?api-version=%s",
28
- @customer_id, API_VERSION)
35
+ uri = sprintf("https://%s.%s/api/logs?api-version=%s",
36
+ @customer_id, @endpoint, API_VERSION)
29
37
  date = rfc1123date()
30
38
  sig = signature(date, body.bytesize)
31
39
 
@@ -34,16 +42,39 @@ module Azure
34
42
  'Authorization' => sig,
35
43
  'Log-Type' => log_type,
36
44
  'x-ms-date' => date,
45
+ 'x-ms-AzureResourceId' => azure_resource_id.empty? ? @default_azure_resource_id : azure_resource_id,
37
46
  'time-generated-field' => record_timestamp
38
47
  }
39
48
 
40
- res = RestClient.post( uri, body, headers)
41
- res
49
+ retries = 0
50
+ begin
51
+ res = RestClient.post( uri, body, headers)
52
+ res
53
+ rescue => e
54
+ c = e.response.code.to_i
55
+ if c == 429 || c == 500 || c==503
56
+ if retries < @max_retries
57
+ retries += 1
58
+ sleep(@retry_sleep_period)
59
+ retry
60
+ end
61
+ end
62
+ raise e
63
+ end
42
64
  end
43
65
 
44
66
  def set_proxy(proxy='')
45
67
  RestClient.proxy = proxy.empty? ? ENV['http_proxy'] : proxy
46
68
  end
69
+
70
+ def set_default_azure_resoruce_id(azure_resource_id)
71
+ @default_azure_resource_id = azure_resource_id
72
+ end
73
+
74
+ def set_retres(max_retries, retry_sleep_period)
75
+ @max_retries = max_retries
76
+ @retry_sleep_period = retry_sleep_period
77
+ end
47
78
 
48
79
  def self.is_success(res)
49
80
  return (res.code == 200) ? true : false
@@ -51,8 +82,8 @@ module Azure
51
82
 
52
83
  private
53
84
 
54
- def is_alpha(s)
55
- return (s.match(/^[[:alpha:]]+$/)) ? true : false
85
+ def is_valid_log_type(s)
86
+ return ( s.match(/^[a-zA-Z0-9_]+$/) && s.length <= 100 ) ? true : false
56
87
  end
57
88
 
58
89
  def rfc1123date()
@@ -1,7 +1,7 @@
1
1
  module Azure
2
2
  module Loganalytics
3
3
  module Datacollectorapi
4
- VERSION = "0.1.4"
4
+ VERSION = "0.4.0"
5
5
  end
6
6
  end
7
7
  end
@@ -5,15 +5,36 @@ describe Azure::Loganalytics::Datacollectorapi::Client do
5
5
  expect(Azure::Loganalytics::Datacollectorapi::VERSION).not_to be nil
6
6
  end
7
7
 
8
- #it "does something useful" do
9
- # expect(false).to eq(true)
10
- #end
8
+ customer_id = '<Customer ID aka WorkspaceID String>'
9
+ shared_key = '<Primary Key String>'
10
+ log_type = "MyCustomLog"
11
11
 
12
- it "posting data to datacollector api" do
13
- customer_id = '<Customer ID aka WorkspaceID String>'
14
- shared_key = '<Primary Key String>'
15
- log_type = "MyCustomLog"
12
+ it "log type validation" do
13
+ json_records = []
14
+ dummy= {
15
+ :string => "dummy title",
16
+ :number => 10
17
+ }
18
+ valid_log_type = "abcedefghijklmnopqrstuvwxyz1234567890_"
19
+ invalid_log_type1 = "*-|[]//"
20
+ invalid_log_type2 = "abcde__xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
21
+ json_records.push(dummy)
22
+ client=Azure::Loganalytics::Datacollectorapi::Client::new( customer_id, shared_key)
23
+
24
+ res = client.post_data(valid_log_type, json_records)
25
+ expect(Azure::Loganalytics::Datacollectorapi::Client.is_success(res)).to eq(true)
16
26
 
27
+ expect{
28
+ client.post_data(invalid_log_type1, json_records)
29
+ }.to raise_error(Exception)
30
+
31
+ expect{
32
+ client.post_data(invalid_log_type2, json_records)
33
+ }.to raise_error(Exception)
34
+
35
+ end
36
+
37
+ it "posting data to datacollector api" do
17
38
  json_records = []
18
39
  record1= {
19
40
  :string => "MyText1",
@@ -33,4 +54,53 @@ describe Azure::Loganalytics::Datacollectorapi::Client do
33
54
  expect(Azure::Loganalytics::Datacollectorapi::Client.is_success(res)).to eq(true)
34
55
  end
35
56
 
57
+ it "posting data to datacollector api with time-generated-field" do
58
+ json_records = []
59
+ record1= {
60
+ :string => "MyText1",
61
+ :boolean => true,
62
+ :number => 100,
63
+ :timegen => "2020-05-05T11:13:35.576Z" # YYYY-MM-DDThh:mm:ssZ
64
+ }
65
+ record2= {
66
+ :string => "MyText2",
67
+ :boolean => false,
68
+ :number => 200,
69
+ :timegen => "2020-05-05T12:13:35.576Z" # YYYY-MM-DDThh:mm:ssZ
70
+ }
71
+ json_records.push(record1)
72
+ json_records.push(record2)
73
+
74
+ time_generated_field = "timegen"
75
+ client=Azure::Loganalytics::Datacollectorapi::Client::new( customer_id, shared_key)
76
+ res = client.post_data(log_type, json_records, time_generated_field)
77
+ expect(Azure::Loganalytics::Datacollectorapi::Client.is_success(res)).to eq(true)
78
+ end
79
+
80
+ it "posting data to datacollector api with azure-resource-id" do
81
+ json_records = []
82
+ record1= {
83
+ :string => "MyText1",
84
+ :boolean => true,
85
+ :number => 100
86
+ }
87
+ record2= {
88
+ :string => "MyText2",
89
+ :boolean => false,
90
+ :number => 200
91
+ }
92
+ json_records.push(record1)
93
+ json_records.push(record2)
94
+
95
+ time_generated_field = ""
96
+ # Azure Resource ID
97
+ # https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/template-functions-resource#resourceid
98
+ # /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}
99
+ azure_resource_id ="/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/otherResourceGroup/providers/Microsoft.Storage/storageAccounts/examplestorage"
100
+
101
+ client=Azure::Loganalytics::Datacollectorapi::Client::new( customer_id, shared_key)
102
+ res = client.post_data(log_type, json_records, time_generated_field, azure_resource_id)
103
+ expect(Azure::Loganalytics::Datacollectorapi::Client.is_success(res)).to eq(true)
104
+ end
105
+
36
106
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: azure-loganalytics-datacollector-api
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.4
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Yoichi Kawasaki
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-05-12 00:00:00.000000000 Z
11
+ date: 2020-07-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rest-client
@@ -24,34 +24,20 @@ dependencies:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
26
  version: '0'
27
- - !ruby/object:Gem::Dependency
28
- name: bundler
29
- requirement: !ruby/object:Gem::Requirement
30
- requirements:
31
- - - "~>"
32
- - !ruby/object:Gem::Version
33
- version: '1.13'
34
- type: :development
35
- prerelease: false
36
- version_requirements: !ruby/object:Gem::Requirement
37
- requirements:
38
- - - "~>"
39
- - !ruby/object:Gem::Version
40
- version: '1.13'
41
27
  - !ruby/object:Gem::Dependency
42
28
  name: rake
43
29
  requirement: !ruby/object:Gem::Requirement
44
30
  requirements:
45
- - - "~>"
31
+ - - ">="
46
32
  - !ruby/object:Gem::Version
47
- version: '10.0'
33
+ version: 12.3.3
48
34
  type: :development
49
35
  prerelease: false
50
36
  version_requirements: !ruby/object:Gem::Requirement
51
37
  requirements:
52
- - - "~>"
38
+ - - ">="
53
39
  - !ruby/object:Gem::Version
54
- version: '10.0'
40
+ version: 12.3.3
55
41
  - !ruby/object:Gem::Dependency
56
42
  name: rspec
57
43
  requirement: !ruby/object:Gem::Requirement
@@ -68,7 +54,7 @@ dependencies:
68
54
  version: '3.0'
69
55
  description: Azure Log Analytics Data Collector API Ruby Client
70
56
  email:
71
- - yoichi.kawasaki@outlook.com
57
+ - yokawasa@gmail.com
72
58
  executables:
73
59
  - console
74
60
  - setup
@@ -82,6 +68,7 @@ files:
82
68
  - LICENSE.txt
83
69
  - README.md
84
70
  - Rakefile
71
+ - SampleCodes.md
85
72
  - azure-loganalytics-datacollector-api.gemspec
86
73
  - bin/console
87
74
  - bin/setup
@@ -108,7 +95,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
108
95
  - !ruby/object:Gem::Version
109
96
  version: '0'
110
97
  requirements: []
111
- rubygems_version: 3.0.3
98
+ rubygems_version: 3.1.3
112
99
  signing_key:
113
100
  specification_version: 4
114
101
  summary: Azure Log Analytics Data Collector API Ruby Client