fluent-plugin-datadog 0.10.5 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. checksums.yaml +5 -5
  2. data/README.md +66 -9
  3. data/fluent-plugin-datadog.gemspec +9 -4
  4. metadata +72 -11
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 64fbcf3f2a58f4ae91945601d802bfcbbe27a411
4
- data.tar.gz: a80b933ca941bdcaaf15f64f0ff2d4900e7cf967
2
+ SHA256:
3
+ metadata.gz: c796cd3bf58664302c3522ede3ed1518c268a1a557cd00244ceef41dcb97b8a7
4
+ data.tar.gz: 00c4cd64362d94ba967c9239f15cfa830f7e8533345f379433709630754ae2cf
5
5
  SHA512:
6
- metadata.gz: 97682ce0db7b87f629be382e95d9489aec3df163a014f7b74f87642d6741a01cbc5f6ee44d321ea8917023b2499bd0cd958e1dee31a2aa0f0d3502b72d023bae
7
- data.tar.gz: 7ede13e8ae2b91ae16cef2ba7a0a2323acb432ecc69dab99d2d08e0a1884fe68c7fbce426fa3209deff25aaa895a21354912eef3d4141feaf86eb7efd394763a
6
+ metadata.gz: 58ba2c958571d7756c8cbc18c610a324195a2c0f96e1cc4c62c79dee8438e8f6f16df2f77c50169f84d541e6db39530e47ccd67d8c43b9f2e8c590b5d6c0c360
7
+ data.tar.gz: a92d2e8972aa49c15609cefcc5b70c75cf8f6a674734935c4795b7d393d3e5e63c669ae3d2503714ad94860e15dd81851009d250e16ed56d9d3c901b44d8372b
data/README.md CHANGED
@@ -1,11 +1,15 @@
1
1
  # Fluentd output plugin for Datadog
2
2
 
3
- It mainly contains a proper JSON formatter and a socket handler that
4
- streams logs directly to Datadog - so no need to use a log shipper
3
+ This output plugin allows sending logs directly from Fluentd to Datadog - so you don't have to use a separate log shipper
5
4
  if you don't wan't to.
6
5
 
7
6
  ## Pre-requirements
8
7
 
8
+ | fluent-plugin-datadog | Fluentd | Ruby |
9
+ |:--------------------------|:-----------|:-------|
10
+ | \>= 0.12.0 | \>= v1 | \>= 2.4 |
11
+ | < 0.12.0 | \>= v0.12.0 | \>= 2.1 |
12
+
9
13
  To add the plugin to your fluentd agent, use the following command:
10
14
 
11
15
  gem install fluent-plugin-datadog
@@ -19,7 +23,7 @@ If you installed the td-agent instead
19
23
 
20
24
  To match events and send them to Datadog, simply add the following code to your configuration file.
21
25
 
22
- TCP example:
26
+ HTTP example:
23
27
 
24
28
  ```xml
25
29
  # Match events tagged with "datadog.**" and
@@ -35,10 +39,21 @@ TCP example:
35
39
  tag_key 'tag'
36
40
 
37
41
  # Optional parameters
38
- dd_source '<INTEGRATION_NAME>'
39
- dd_tags '<KEY1:VALU1>,<KEY2:VALUE2>'
42
+ dd_source '<INTEGRATION_NAME>'
43
+ dd_tags '<KEY1:VALUE1>,<KEY2:VALUE2>'
40
44
  dd_sourcecategory '<MY_SOURCE_CATEGORY>'
41
45
 
46
+ # Optional http proxy
47
+ http_proxy 'http://my-proxy.example'
48
+
49
+ <buffer>
50
+ @type memory
51
+ flush_thread_count 4
52
+ flush_interval 3s
53
+ chunk_limit_size 5m
54
+ chunk_limit_records 500
55
+ </buffer>
56
+
42
57
  </match>
43
58
  ```
44
59
 
@@ -68,14 +83,25 @@ As fluent-plugin-datadog is an output_buffer, you can set all output_buffer prop
68
83
  |-------------|--------------------------------------------------------------------------|----------------|
69
84
  | **api_key** | This parameter is required in order to authenticate your fluent agent. | nil |
70
85
  | **use_json** | Event format, if true, the event is sent in json format. Othwerwise, in plain text. | true |
71
- | **include_tag_key** | Automatically include tags in the record. | false |
72
- | **tag_key** | Name of the tag attribute, if they are included. | "tag" |
86
+ | **include_tag_key** | Automatically include the Fluentd tag in the record. | false |
87
+ | **tag_key** | Where to store the Fluentd tag. | "tag" |
88
+ | **timestamp_key** | Name of the attribute which will contain timestamp of the log event. If nil, timestamp attribute is not added. | "@timestamp" |
73
89
  | **use_ssl** | If true, the agent initializes a secure connection to Datadog. In clear TCP otherwise. | true |
90
+ | **no_ssl_validation** | Disable SSL validation (useful for proxy forwarding) | false |
91
+ | **ssl_port** | Port used to send logs over a SSL encrypted connection to Datadog. If use_http is disabled, use 10516 for the US region and 443 for the EU region. | 443 |
74
92
  | **max_retries** | The number of retries before the output plugin stops. Set to -1 for unlimited retries | -1 |
75
- | **max_retries** | The number of retries before the output plugin stops. Set to -1 for unlimited retries | -1 |
93
+ | **max_backoff** | The maximum time waited between each retry in seconds | 30 |
94
+ | **use_http** | Enable HTTP forwarding. If you disable it, make sure to change the port to 10514 or ssl_port to 10516 | true |
95
+ | **use_compression** | Enable log compression for HTTP | true |
96
+ | **compression_level** | Set the log compression level for HTTP (1 to 9, 9 being the best ratio) | 6 |
76
97
  | **dd_source** | This tells Datadog what integration it is | nil |
77
- | **dd_sourcecategory** | Multiple value attribute. Can be used to refine the source attribtue | nil |
98
+ | **dd_sourcecategory** | Multiple value attribute. Can be used to refine the source attribute | nil |
78
99
  | **dd_tags** | Custom tags with the following format "key1:value1, key2:value2" | nil |
100
+ | **dd_hostname** | Used by Datadog to identify the host submitting the logs. | `hostname -f` |
101
+ | **service** | Used by Datadog to correlate between logs, traces and metrics. | nil |
102
+ | **port** | Proxy port when logs are not directly forwarded to Datadog and ssl is not used | 80 |
103
+ | **host** | Proxy endpoint when logs are not directly forwarded to Datadog | http-intake.logs.datadoghq.com |
104
+ | **http_proxy** | HTTP proxy, only takes effect if HTTP forwarding is enabled (`use_http`). Defaults to `HTTP_PROXY`/`http_proxy` env vars. | nil |
79
105
 
80
106
  ### Docker and Kubernetes tags
81
107
 
@@ -99,3 +125,34 @@ Configuration example:
99
125
  type kubernetes_metadata
100
126
  </filter>
101
127
  ```
128
+
129
+ ### Encoding
130
+
131
+ Datadog's API expects log messages to be encoded in UTF-8.
132
+ If some of your logs are encoded with a different encoding, we recommend using the [`record_modifier` filter plugin](https://github.com/repeatedly/fluent-plugin-record-modifier#char_encoding)
133
+ to encode these logs to UTF-8.
134
+
135
+ Configuration example:
136
+
137
+ ```
138
+ # Change encoding of logs tagged with "datadog.**"
139
+ <filter datadog.**>
140
+ @type record_modifier
141
+
142
+ # change the encoding from the '<SOURCE_ENCODING>' of your logs to 'utf-8'
143
+ char_encoding <SOURCE_ENCODING>:utf-8
144
+ </filter>
145
+ ```
146
+
147
+ ## Build
148
+
149
+ To build a new version of this plugin and push it to RubyGems:
150
+
151
+ - Update the version in the .gemspec file accordingly
152
+ - `rake build` to build the gem file
153
+ - `rake release` to push the new gem to RubyGems
154
+
155
+ **Note**: The latest command will fail without appropriate credentials configured. You can set those credentials by running the following command:
156
+
157
+ `curl -u <USERNAME> https://rubygems.org/api/v1/api_key.yaml > ~/.gem/credentials`, it will ask for your password.
158
+
data/fluent-plugin-datadog.gemspec CHANGED
@@ -9,19 +9,24 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
9
9
 
10
10
  Gem::Specification.new do |spec|
11
11
  spec.name = "fluent-plugin-datadog"
12
- spec.version = "0.10.5"
12
+ spec.version = "0.13.0"
13
13
  spec.authors = ["Datadog Solutions Team"]
14
14
  spec.email = ["support@datadoghq.com"]
15
15
  spec.summary = "Datadog output plugin for Fluent event collector"
16
16
  spec.homepage = "http://datadoghq.com"
17
- spec.license = "Apache License 2.0"
17
+ spec.license = "Apache-2.0"
18
18
 
19
19
  spec.files = [".gitignore", "Gemfile", "LICENSE", "README.md", "Rakefile", "fluent-plugin-datadog.gemspec", "lib/fluent/plugin/out_datadog.rb"]
20
20
  spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
21
21
  spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
22
22
  spec.require_paths = ["lib"]
23
23
 
24
- spec.add_development_dependency "bundler", "~> 1.5"
25
- spec.add_development_dependency "rake"
24
+ spec.add_runtime_dependency "fluentd", [">= 1", "< 2"]
25
+ spec.add_runtime_dependency "net-http-persistent", '~> 3.1'
26
+
27
+ spec.add_development_dependency "bundler", "~> 2.1"
28
+ spec.add_development_dependency "test-unit", '~> 3.1'
29
+ spec.add_development_dependency "rake", "~> 12.0"
26
30
  spec.add_development_dependency "yajl-ruby", "~> 1.2"
31
+ spec.add_development_dependency 'webmock', "~> 3.5.0"
27
32
  end
metadata CHANGED
@@ -1,43 +1,91 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-datadog
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.10.5
4
+ version: 0.13.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Datadog Solutions Team
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2018-11-26 00:00:00.000000000 Z
11
+ date: 2021-03-22 00:00:00.000000000 Z
12
12
  dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: fluentd
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '1'
20
+ - - "<"
21
+ - !ruby/object:Gem::Version
22
+ version: '2'
23
+ type: :runtime
24
+ prerelease: false
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - ">="
28
+ - !ruby/object:Gem::Version
29
+ version: '1'
30
+ - - "<"
31
+ - !ruby/object:Gem::Version
32
+ version: '2'
33
+ - !ruby/object:Gem::Dependency
34
+ name: net-http-persistent
35
+ requirement: !ruby/object:Gem::Requirement
36
+ requirements:
37
+ - - "~>"
38
+ - !ruby/object:Gem::Version
39
+ version: '3.1'
40
+ type: :runtime
41
+ prerelease: false
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - "~>"
45
+ - !ruby/object:Gem::Version
46
+ version: '3.1'
13
47
  - !ruby/object:Gem::Dependency
14
48
  name: bundler
15
49
  requirement: !ruby/object:Gem::Requirement
16
50
  requirements:
17
51
  - - "~>"
18
52
  - !ruby/object:Gem::Version
19
- version: '1.5'
53
+ version: '2.1'
54
+ type: :development
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - "~>"
59
+ - !ruby/object:Gem::Version
60
+ version: '2.1'
61
+ - !ruby/object:Gem::Dependency
62
+ name: test-unit
63
+ requirement: !ruby/object:Gem::Requirement
64
+ requirements:
65
+ - - "~>"
66
+ - !ruby/object:Gem::Version
67
+ version: '3.1'
20
68
  type: :development
21
69
  prerelease: false
22
70
  version_requirements: !ruby/object:Gem::Requirement
23
71
  requirements:
24
72
  - - "~>"
25
73
  - !ruby/object:Gem::Version
26
- version: '1.5'
74
+ version: '3.1'
27
75
  - !ruby/object:Gem::Dependency
28
76
  name: rake
29
77
  requirement: !ruby/object:Gem::Requirement
30
78
  requirements:
31
- - - ">="
79
+ - - "~>"
32
80
  - !ruby/object:Gem::Version
33
- version: '0'
81
+ version: '12.0'
34
82
  type: :development
35
83
  prerelease: false
36
84
  version_requirements: !ruby/object:Gem::Requirement
37
85
  requirements:
38
- - - ">="
86
+ - - "~>"
39
87
  - !ruby/object:Gem::Version
40
- version: '0'
88
+ version: '12.0'
41
89
  - !ruby/object:Gem::Dependency
42
90
  name: yajl-ruby
43
91
  requirement: !ruby/object:Gem::Requirement
@@ -52,6 +100,20 @@ dependencies:
52
100
  - - "~>"
53
101
  - !ruby/object:Gem::Version
54
102
  version: '1.2'
103
+ - !ruby/object:Gem::Dependency
104
+ name: webmock
105
+ requirement: !ruby/object:Gem::Requirement
106
+ requirements:
107
+ - - "~>"
108
+ - !ruby/object:Gem::Version
109
+ version: 3.5.0
110
+ type: :development
111
+ prerelease: false
112
+ version_requirements: !ruby/object:Gem::Requirement
113
+ requirements:
114
+ - - "~>"
115
+ - !ruby/object:Gem::Version
116
+ version: 3.5.0
55
117
  description:
56
118
  email:
57
119
  - support@datadoghq.com
@@ -68,7 +130,7 @@ files:
68
130
  - lib/fluent/plugin/out_datadog.rb
69
131
  homepage: http://datadoghq.com
70
132
  licenses:
71
- - Apache License 2.0
133
+ - Apache-2.0
72
134
  metadata: {}
73
135
  post_install_message:
74
136
  rdoc_options: []
@@ -85,8 +147,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
85
147
  - !ruby/object:Gem::Version
86
148
  version: '0'
87
149
  requirements: []
88
- rubyforge_project:
89
- rubygems_version: 2.5.2.3
150
+ rubygems_version: 3.0.3
90
151
  signing_key:
91
152
  specification_version: 4
92
153
  summary: Datadog output plugin for Fluent event collector