fluent-plugin-azurestorage 0.0.7 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +11 -11
- data/.travis.yml +19 -21
- data/Gemfile +3 -3
- data/LICENSE.txt +22 -22
- data/README.md +210 -210
- data/Rakefile +14 -14
- data/VERSION +1 -1
- data/fluent-plugin-azurestorage.gemspec +26 -26
- data/lib/fluent/plugin/azurestorage_compressor_gzip_command.rb +51 -51
- data/lib/fluent/plugin/azurestorage_compressor_lzma2.rb +34 -34
- data/lib/fluent/plugin/azurestorage_compressor_lzo.rb +34 -34
- data/lib/fluent/plugin/out_azurestorage.rb +248 -248
- data/lib/fluent/plugin/upload_service.rb +207 -207
- data/test/test_out_azurestorage.rb +224 -224
- metadata +32 -27
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 776476b29eb416de1bf483aaf7f2a7a864ed10fe
|
|
4
|
+
data.tar.gz: 95cdf6fa62d1f4e772179694a1a3f7a611ee3149
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cec4a7f65d722e23bb49e46211c482b10f7c3e9f15e0ed286b960599aac7034a5d110db122ab072ace1ea056a820b68d7e2f55a5b9033f6252a6ba71710718cc
|
|
7
|
+
data.tar.gz: b21756a7d13066e1155853ee3ed2bd9262ef24304483f0306c509f34538ae4ee9029efd0a1d161432f65520785c30e7033f35aeee9e10a68acaca710bc0cd6ad
|
data/.gitignore
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
~*
|
|
2
|
-
#*
|
|
3
|
-
*~
|
|
4
|
-
[._]*.s[a-w][a-z]
|
|
5
|
-
.DS_Store
|
|
6
|
-
|
|
7
|
-
*.gem
|
|
8
|
-
.bundle
|
|
9
|
-
Gemfile.lock
|
|
10
|
-
vendor
|
|
11
|
-
.ruby-version
|
|
1
|
+
~*
|
|
2
|
+
#*
|
|
3
|
+
*~
|
|
4
|
+
[._]*.s[a-w][a-z]
|
|
5
|
+
.DS_Store
|
|
6
|
+
|
|
7
|
+
*.gem
|
|
8
|
+
.bundle
|
|
9
|
+
Gemfile.lock
|
|
10
|
+
vendor
|
|
11
|
+
.ruby-version
|
data/.travis.yml
CHANGED
|
@@ -1,21 +1,19 @@
|
|
|
1
|
-
language: ruby
|
|
2
|
-
|
|
3
|
-
rvm:
|
|
4
|
-
- 1
|
|
5
|
-
- 2.0
|
|
6
|
-
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
allow_failures:
|
|
21
|
-
- rvm: rbx
|
|
1
|
+
language: ruby
|
|
2
|
+
|
|
3
|
+
rvm:
|
|
4
|
+
- 2.1
|
|
5
|
+
- 2.2.0
|
|
6
|
+
- rbx
|
|
7
|
+
|
|
8
|
+
gemfile:
|
|
9
|
+
- Gemfile
|
|
10
|
+
|
|
11
|
+
branches:
|
|
12
|
+
only:
|
|
13
|
+
- master
|
|
14
|
+
|
|
15
|
+
script: bundle exec rake test
|
|
16
|
+
|
|
17
|
+
matrix:
|
|
18
|
+
allow_failures:
|
|
19
|
+
- rvm: rbx
|
data/Gemfile
CHANGED
|
@@ -1,3 +1,3 @@
|
|
|
1
|
-
source "http://rubygems.org"
|
|
2
|
-
|
|
3
|
-
gemspec
|
|
1
|
+
source "http://rubygems.org"
|
|
2
|
+
|
|
3
|
+
gemspec
|
data/LICENSE.txt
CHANGED
|
@@ -1,22 +1,22 @@
|
|
|
1
|
-
Copyright (c) 2014 Hidemasa Togashi
|
|
2
|
-
|
|
3
|
-
MIT License
|
|
4
|
-
|
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining
|
|
6
|
-
a copy of this software and associated documentation files (the
|
|
7
|
-
"Software"), to deal in the Software without restriction, including
|
|
8
|
-
without limitation the rights to use, copy, modify, merge, publish,
|
|
9
|
-
distribute, sublicense, and/or sell copies of the Software, and to
|
|
10
|
-
permit persons to whom the Software is furnished to do so, subject to
|
|
11
|
-
the following conditions:
|
|
12
|
-
|
|
13
|
-
The above copyright notice and this permission notice shall be
|
|
14
|
-
included in all copies or substantial portions of the Software.
|
|
15
|
-
|
|
16
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
17
|
-
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
18
|
-
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
19
|
-
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
|
20
|
-
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
21
|
-
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
|
22
|
-
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
|
1
|
+
Copyright (c) 2014 Hidemasa Togashi
|
|
2
|
+
|
|
3
|
+
MIT License
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
|
6
|
+
a copy of this software and associated documentation files (the
|
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
|
11
|
+
the following conditions:
|
|
12
|
+
|
|
13
|
+
The above copyright notice and this permission notice shall be
|
|
14
|
+
included in all copies or substantial portions of the Software.
|
|
15
|
+
|
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
CHANGED
|
@@ -1,210 +1,210 @@
|
|
|
1
|
-
# Azure Storage output plugin for Fluentd
|
|
2
|
-
|
|
3
|
-
[](https://travis-ci.org/htgc/fluent-plugin-azurestorage)
|
|
4
|
-
|
|
5
|
-
## Overview
|
|
6
|
-
|
|
7
|
-
Azure Storate output plugin buffers logs in local file and upload them to Azure Storage periodically.
|
|
8
|
-
This plugin is porting from [fluent-plugin-s3](https://github.com/fluent/fluent-plugin-s3) for AzureStorage.
|
|
9
|
-
|
|
10
|
-
## Installation
|
|
11
|
-
|
|
12
|
-
Install from RubyGems:
|
|
13
|
-
```
|
|
14
|
-
$ gem install fluent-plugin-azurestorage
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
## Configuration
|
|
18
|
-
|
|
19
|
-
```
|
|
20
|
-
<match pattern>
|
|
21
|
-
type azurestorage
|
|
22
|
-
|
|
23
|
-
azure_storage_account <your azure storage account>
|
|
24
|
-
azure_storage_access_key <your azure storage access key>
|
|
25
|
-
azure_container <your azure storage container>
|
|
26
|
-
azure_storage_type blob
|
|
27
|
-
store_as gzip
|
|
28
|
-
auto_create_container true
|
|
29
|
-
path logs/
|
|
30
|
-
azure_object_key_format %{path}%{time_slice}_%{index}.%{file_extension}
|
|
31
|
-
buffer_path /var/log/fluent/azurestorage
|
|
32
|
-
|
|
33
|
-
time_slice_format %Y%m%d-%H
|
|
34
|
-
time_slice_wait 10m
|
|
35
|
-
utc
|
|
36
|
-
</match>
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
### azure_storage_account
|
|
40
|
-
|
|
41
|
-
Your Azure Storage Account Name. This can be got from Azure Management potal.
|
|
42
|
-
This parameter is required when environment variable 'AZURE_STORAGE_ACCOUNT' is not set.
|
|
43
|
-
|
|
44
|
-
### azure_storage_access_key
|
|
45
|
-
|
|
46
|
-
Your Azure Storage Access Key(Primary or Secondary). This also can be got from Azure Management potal.
|
|
47
|
-
This parameter is required when environment variable 'AZURE_STORAGE_ACCESS_KEY' is not set.
|
|
48
|
-
|
|
49
|
-
### azure_container (Required)
|
|
50
|
-
|
|
51
|
-
Azure Storage Container name
|
|
52
|
-
|
|
53
|
-
### auto_create_container
|
|
54
|
-
|
|
55
|
-
This plugin create container if not exist when you set 'auto_create_container' to true.
|
|
56
|
-
|
|
57
|
-
### azure_storage_type
|
|
58
|
-
|
|
59
|
-
Azure Storage type. Now supports only 'blob'(default). 'tables' and 'queues' are not implemented.
|
|
60
|
-
|
|
61
|
-
### azure_object_key_format
|
|
62
|
-
|
|
63
|
-
The format of Azure Storage object keys. You can use several built-in variables:
|
|
64
|
-
|
|
65
|
-
- %{path}
|
|
66
|
-
- %{time_slice}
|
|
67
|
-
- %{index}
|
|
68
|
-
- %{file_extension}
|
|
69
|
-
|
|
70
|
-
to decide keys dynamically.
|
|
71
|
-
|
|
72
|
-
%{path} is exactly the value of *path* configured in the configuration file. E.g., "logs/" in the example configuration above.
|
|
73
|
-
%{time_slice} is the time-slice in text that are formatted with *time_slice_format*.
|
|
74
|
-
%{index} is the sequential number starts from 0, increments when multiple files are uploaded to Azure Storage in the same time slice.
|
|
75
|
-
%{file_extention} is always "gz" for now.
|
|
76
|
-
|
|
77
|
-
The default format is "%{path}%{time_slice}_%{index}.%{file_extension}".
|
|
78
|
-
|
|
79
|
-
For instance, using the example configuration above, actual object keys on Azure Storage will be something like:
|
|
80
|
-
|
|
81
|
-
```
|
|
82
|
-
"logs/20130111-22_0.gz"
|
|
83
|
-
"logs/20130111-23_0.gz"
|
|
84
|
-
"logs/20130111-23_1.gz"
|
|
85
|
-
"logs/20130112-00_0.gz"
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
With the configuration:
|
|
89
|
-
|
|
90
|
-
```
|
|
91
|
-
azure_object_key_format %{path}/events/ts=%{time_slice}/events_%{index}.%{file_extension}
|
|
92
|
-
path log
|
|
93
|
-
time_slice_format %Y%m%d-%H
|
|
94
|
-
```
|
|
95
|
-
|
|
96
|
-
You get:
|
|
97
|
-
|
|
98
|
-
```
|
|
99
|
-
"log/events/ts=20130111-22/events_0.gz"
|
|
100
|
-
"log/events/ts=20130111-23/events_0.gz"
|
|
101
|
-
"log/events/ts=20130111-23/events_1.gz"
|
|
102
|
-
"log/events/ts=20130112-00/events_0.gz"
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
The [fluent-mixin-config-placeholders](https://github.com/tagomoris/fluent-mixin-config-placeholders) mixin is also incorporated, so additional variables such as %{hostname}, %{uuid}, etc. can be used in the azure_object_key_format. This could prove useful in preventing filename conflicts when writing from multiple servers.
|
|
106
|
-
|
|
107
|
-
```
|
|
108
|
-
azure_object_key_format %{path}/events/ts=%{time_slice}/events_%{index}-%{hostname}.%{file_extension}
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
### store_as
|
|
112
|
-
|
|
113
|
-
Archive format on Azure Storage. You can use following types:
|
|
114
|
-
|
|
115
|
-
- gzip (default)
|
|
116
|
-
- json
|
|
117
|
-
- text
|
|
118
|
-
- lzo (Need lzop command)
|
|
119
|
-
- lzma2 (Need xz command)
|
|
120
|
-
|
|
121
|
-
### format
|
|
122
|
-
|
|
123
|
-
Change one line format in the Azure Storage object. Supported formats are 'out_file', 'json', 'ltsv' and 'single_value'.
|
|
124
|
-
|
|
125
|
-
- out_file (default)
|
|
126
|
-
|
|
127
|
-
```
|
|
128
|
-
time\ttag\t{..json1..}
|
|
129
|
-
time\ttag\t{..json2..}
|
|
130
|
-
...
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
- json
|
|
134
|
-
|
|
135
|
-
```
|
|
136
|
-
{..json1..}
|
|
137
|
-
{..json2..}
|
|
138
|
-
...
|
|
139
|
-
```
|
|
140
|
-
|
|
141
|
-
At this format, "time" and "tag" are omitted.
|
|
142
|
-
But you can set these information to the record by setting "include_tag_key" / "tag_key" and "include_time_key" / "time_key" option.
|
|
143
|
-
If you set following configuration in AzureStorage output:
|
|
144
|
-
|
|
145
|
-
```
|
|
146
|
-
format json
|
|
147
|
-
include_time_key true
|
|
148
|
-
time_key log_time # default is time
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
then the record has log_time field.
|
|
152
|
-
|
|
153
|
-
```
|
|
154
|
-
{"log_time":"time string",...}
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
- ltsv
|
|
158
|
-
|
|
159
|
-
```
|
|
160
|
-
key1:value1\tkey2:value2
|
|
161
|
-
key1:value1\tkey2:value2
|
|
162
|
-
...
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
"ltsv" format also accepts "include_xxx" related options. See "json" section.
|
|
166
|
-
|
|
167
|
-
- single_value
|
|
168
|
-
|
|
169
|
-
Use specified value instead of entire recode. If you get '{"message":"my log"}', then contents are
|
|
170
|
-
|
|
171
|
-
```
|
|
172
|
-
my log1
|
|
173
|
-
my log2
|
|
174
|
-
...
|
|
175
|
-
```
|
|
176
|
-
|
|
177
|
-
You can change key name by "message_key" option.
|
|
178
|
-
|
|
179
|
-
### path
|
|
180
|
-
|
|
181
|
-
path prefix of the files on Azure Storage. Default is "" (no prefix).
|
|
182
|
-
|
|
183
|
-
### buffer_path (required)
|
|
184
|
-
|
|
185
|
-
path prefix of the files to buffer logs.
|
|
186
|
-
|
|
187
|
-
### time_slice_format
|
|
188
|
-
|
|
189
|
-
Format of the time used as the file name. Default is '%Y%m%d'. Use '%Y%m%d%H' to split files hourly.
|
|
190
|
-
|
|
191
|
-
### time_slice_wait
|
|
192
|
-
|
|
193
|
-
The time to wait old logs. Default is 10 minutes. Specify larger value if old logs may reache.
|
|
194
|
-
|
|
195
|
-
### utc
|
|
196
|
-
|
|
197
|
-
Use UTC instead of local time.
|
|
198
|
-
|
|
199
|
-
## License
|
|
200
|
-
Azure Storage output plugin is licensed according to the terms of the Apache License, Version 2.0.
|
|
201
|
-
|
|
202
|
-
The full version of this lisence can be found at http://www.apache.org/licenses/LICENSE-2.0 [TXT](http://www.apache.org/licenses/LICENSE-2.0.txt) or [HTML](http://www.apache.org/licenses/LICENSE-2.0.html)
|
|
203
|
-
|
|
204
|
-
## Contributing
|
|
205
|
-
|
|
206
|
-
1. Fork it
|
|
207
|
-
2. Create your feature branch (`git checkout -b my-new-feature`)
|
|
208
|
-
3. Commit your changes (`git commit -am 'Add some feature'`)
|
|
209
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
|
210
|
-
5. Create new Pull Request
|
|
1
|
+
# Azure Storage output plugin for Fluentd
|
|
2
|
+
|
|
3
|
+
[](https://travis-ci.org/htgc/fluent-plugin-azurestorage)
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
Azure Storate output plugin buffers logs in local file and upload them to Azure Storage periodically.
|
|
8
|
+
This plugin is porting from [fluent-plugin-s3](https://github.com/fluent/fluent-plugin-s3) for AzureStorage.
|
|
9
|
+
|
|
10
|
+
## Installation
|
|
11
|
+
|
|
12
|
+
Install from RubyGems:
|
|
13
|
+
```
|
|
14
|
+
$ gem install fluent-plugin-azurestorage
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Configuration
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
<match pattern>
|
|
21
|
+
type azurestorage
|
|
22
|
+
|
|
23
|
+
azure_storage_account <your azure storage account>
|
|
24
|
+
azure_storage_access_key <your azure storage access key>
|
|
25
|
+
azure_container <your azure storage container>
|
|
26
|
+
azure_storage_type blob
|
|
27
|
+
store_as gzip
|
|
28
|
+
auto_create_container true
|
|
29
|
+
path logs/
|
|
30
|
+
azure_object_key_format %{path}%{time_slice}_%{index}.%{file_extension}
|
|
31
|
+
buffer_path /var/log/fluent/azurestorage
|
|
32
|
+
|
|
33
|
+
time_slice_format %Y%m%d-%H
|
|
34
|
+
time_slice_wait 10m
|
|
35
|
+
utc
|
|
36
|
+
</match>
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### azure_storage_account
|
|
40
|
+
|
|
41
|
+
Your Azure Storage Account Name. This can be got from Azure Management potal.
|
|
42
|
+
This parameter is required when environment variable 'AZURE_STORAGE_ACCOUNT' is not set.
|
|
43
|
+
|
|
44
|
+
### azure_storage_access_key
|
|
45
|
+
|
|
46
|
+
Your Azure Storage Access Key(Primary or Secondary). This also can be got from Azure Management potal.
|
|
47
|
+
This parameter is required when environment variable 'AZURE_STORAGE_ACCESS_KEY' is not set.
|
|
48
|
+
|
|
49
|
+
### azure_container (Required)
|
|
50
|
+
|
|
51
|
+
Azure Storage Container name
|
|
52
|
+
|
|
53
|
+
### auto_create_container
|
|
54
|
+
|
|
55
|
+
This plugin create container if not exist when you set 'auto_create_container' to true.
|
|
56
|
+
|
|
57
|
+
### azure_storage_type
|
|
58
|
+
|
|
59
|
+
Azure Storage type. Now supports only 'blob'(default). 'tables' and 'queues' are not implemented.
|
|
60
|
+
|
|
61
|
+
### azure_object_key_format
|
|
62
|
+
|
|
63
|
+
The format of Azure Storage object keys. You can use several built-in variables:
|
|
64
|
+
|
|
65
|
+
- %{path}
|
|
66
|
+
- %{time_slice}
|
|
67
|
+
- %{index}
|
|
68
|
+
- %{file_extension}
|
|
69
|
+
|
|
70
|
+
to decide keys dynamically.
|
|
71
|
+
|
|
72
|
+
%{path} is exactly the value of *path* configured in the configuration file. E.g., "logs/" in the example configuration above.
|
|
73
|
+
%{time_slice} is the time-slice in text that are formatted with *time_slice_format*.
|
|
74
|
+
%{index} is the sequential number starts from 0, increments when multiple files are uploaded to Azure Storage in the same time slice.
|
|
75
|
+
%{file_extention} is always "gz" for now.
|
|
76
|
+
|
|
77
|
+
The default format is "%{path}%{time_slice}_%{index}.%{file_extension}".
|
|
78
|
+
|
|
79
|
+
For instance, using the example configuration above, actual object keys on Azure Storage will be something like:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
"logs/20130111-22_0.gz"
|
|
83
|
+
"logs/20130111-23_0.gz"
|
|
84
|
+
"logs/20130111-23_1.gz"
|
|
85
|
+
"logs/20130112-00_0.gz"
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
With the configuration:
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
azure_object_key_format %{path}/events/ts=%{time_slice}/events_%{index}.%{file_extension}
|
|
92
|
+
path log
|
|
93
|
+
time_slice_format %Y%m%d-%H
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
You get:
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
"log/events/ts=20130111-22/events_0.gz"
|
|
100
|
+
"log/events/ts=20130111-23/events_0.gz"
|
|
101
|
+
"log/events/ts=20130111-23/events_1.gz"
|
|
102
|
+
"log/events/ts=20130112-00/events_0.gz"
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
The [fluent-mixin-config-placeholders](https://github.com/tagomoris/fluent-mixin-config-placeholders) mixin is also incorporated, so additional variables such as %{hostname}, %{uuid}, etc. can be used in the azure_object_key_format. This could prove useful in preventing filename conflicts when writing from multiple servers.
|
|
106
|
+
|
|
107
|
+
```
|
|
108
|
+
azure_object_key_format %{path}/events/ts=%{time_slice}/events_%{index}-%{hostname}.%{file_extension}
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
### store_as
|
|
112
|
+
|
|
113
|
+
Archive format on Azure Storage. You can use following types:
|
|
114
|
+
|
|
115
|
+
- gzip (default)
|
|
116
|
+
- json
|
|
117
|
+
- text
|
|
118
|
+
- lzo (Need lzop command)
|
|
119
|
+
- lzma2 (Need xz command)
|
|
120
|
+
|
|
121
|
+
### format
|
|
122
|
+
|
|
123
|
+
Change one line format in the Azure Storage object. Supported formats are 'out_file', 'json', 'ltsv' and 'single_value'.
|
|
124
|
+
|
|
125
|
+
- out_file (default)
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
time\ttag\t{..json1..}
|
|
129
|
+
time\ttag\t{..json2..}
|
|
130
|
+
...
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
- json
|
|
134
|
+
|
|
135
|
+
```
|
|
136
|
+
{..json1..}
|
|
137
|
+
{..json2..}
|
|
138
|
+
...
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
At this format, "time" and "tag" are omitted.
|
|
142
|
+
But you can set these information to the record by setting "include_tag_key" / "tag_key" and "include_time_key" / "time_key" option.
|
|
143
|
+
If you set following configuration in AzureStorage output:
|
|
144
|
+
|
|
145
|
+
```
|
|
146
|
+
format json
|
|
147
|
+
include_time_key true
|
|
148
|
+
time_key log_time # default is time
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
then the record has log_time field.
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
{"log_time":"time string",...}
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
- ltsv
|
|
158
|
+
|
|
159
|
+
```
|
|
160
|
+
key1:value1\tkey2:value2
|
|
161
|
+
key1:value1\tkey2:value2
|
|
162
|
+
...
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
"ltsv" format also accepts "include_xxx" related options. See "json" section.
|
|
166
|
+
|
|
167
|
+
- single_value
|
|
168
|
+
|
|
169
|
+
Use specified value instead of entire recode. If you get '{"message":"my log"}', then contents are
|
|
170
|
+
|
|
171
|
+
```
|
|
172
|
+
my log1
|
|
173
|
+
my log2
|
|
174
|
+
...
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
You can change key name by "message_key" option.
|
|
178
|
+
|
|
179
|
+
### path
|
|
180
|
+
|
|
181
|
+
path prefix of the files on Azure Storage. Default is "" (no prefix).
|
|
182
|
+
|
|
183
|
+
### buffer_path (required)
|
|
184
|
+
|
|
185
|
+
path prefix of the files to buffer logs.
|
|
186
|
+
|
|
187
|
+
### time_slice_format
|
|
188
|
+
|
|
189
|
+
Format of the time used as the file name. Default is '%Y%m%d'. Use '%Y%m%d%H' to split files hourly.
|
|
190
|
+
|
|
191
|
+
### time_slice_wait
|
|
192
|
+
|
|
193
|
+
The time to wait old logs. Default is 10 minutes. Specify larger value if old logs may reache.
|
|
194
|
+
|
|
195
|
+
### utc
|
|
196
|
+
|
|
197
|
+
Use UTC instead of local time.
|
|
198
|
+
|
|
199
|
+
## License
|
|
200
|
+
Azure Storage output plugin is licensed according to the terms of the Apache License, Version 2.0.
|
|
201
|
+
|
|
202
|
+
The full version of this lisence can be found at http://www.apache.org/licenses/LICENSE-2.0 [TXT](http://www.apache.org/licenses/LICENSE-2.0.txt) or [HTML](http://www.apache.org/licenses/LICENSE-2.0.html)
|
|
203
|
+
|
|
204
|
+
## Contributing
|
|
205
|
+
|
|
206
|
+
1. Fork it
|
|
207
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
|
208
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
|
209
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
|
210
|
+
5. Create new Pull Request
|