fluent-diagtool 0.1.2 → 0.1.7
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +13 -0
- data/AUTHORS +1 -1
- data/Gemfile.lock +5 -1
- data/README.md +73 -16
- data/exe/diagtool +10 -2
- data/fluent-diagtool.gemspec +2 -2
- data/lib/fluent/diagtool/collectutils.rb +156 -42
- data/lib/fluent/diagtool/diagutils.rb +117 -30
- data/lib/fluent/diagtool/maskutils.rb +15 -5
- data/lib/fluent/diagtool/validutils.rb +2 -0
- data/lib/fluent/diagtool/version.rb +1 -1
- metadata +11 -11
- data/bin/diagtool.rb +0 -37
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: '08fde300371aa506f74a6b700b18840d9ccfa88059a5b2ddff78256e839c37be'
|
4
|
+
data.tar.gz: 890483b9fa25f5ee44454d0c8e527ecd7dab2f60e676b283fc64d433c48ac6eb
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0e4563b5db53d26fe79dc2a1d7f4c425f63a28bc4ebba62c9005796a258d69b92f1ecac2b8e22f952ff9ea672f5e0e6ff10aeaab49b52773ab0cf295b21a13a9
|
7
|
+
data.tar.gz: 5e26af31e777513a58067262e6ea89cad0d7c07f29d3e1e5ceca816162dc089325502f6b5c4a68470be548340c0e9b312e75c0490b1e8346a05526c95b79cd72
|
data/.gitignore
ADDED
data/AUTHORS
CHANGED
@@ -1 +1 @@
|
|
1
|
-
TOMONORI KUBOTA <
|
1
|
+
TOMONORI KUBOTA <tkubota _at_ ctc-america.com>
|
data/Gemfile.lock
CHANGED
@@ -1,12 +1,16 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
fluent-diagtool (0.1.
|
4
|
+
fluent-diagtool (0.1.7.pre.beta)
|
5
|
+
fileutils
|
6
|
+
json
|
5
7
|
|
6
8
|
GEM
|
7
9
|
remote: https://rubygems.org/
|
8
10
|
specs:
|
9
11
|
diff-lcs (1.3)
|
12
|
+
fileutils (1.4.1)
|
13
|
+
json (2.3.0)
|
10
14
|
rake (12.3.3)
|
11
15
|
rspec (3.9.0)
|
12
16
|
rspec-core (~> 3.9.0)
|
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
# Fluentd Diagnostic Tool
|
2
2
|
|
3
|
-
The diagtool enable users to automate the date collection which is required for trouble shooting. The data collected by diagtool include the configuration and log files of the td-agent and diagnostic information of operating system such as network and memory status and stats. In some cases, configuration and log files contains the security sensitive information, such as IP addresses and Hostname. The diagtool also provides the functions to generate mask on IP addresses, Hostname(in FQDN style) and user defined keywords described in the collected data
|
4
|
-
The scope of data collection
|
3
|
+
The diagtool enable users to automate the date collection which is required for trouble shooting. The data collected by diagtool include the configuration and log files of the td-agent and diagnostic information of operating system such as network and memory status and stats. In some cases, configuration and log files contains the security sensitive information, such as IP addresses and Hostname. The diagtool also provides the functions to generate mask on IP addresses, Hostname(in FQDN style) and user defined keywords described in the collected data.
|
4
|
+
The scope of data collection:
|
5
5
|
- TD Agent information
|
6
6
|
- configuration files (*)
|
7
7
|
- log files (*)
|
@@ -11,28 +11,85 @@ The scope of data collection:
|
|
11
11
|
- OS log file
|
12
12
|
- OS parameters
|
13
13
|
- OS and kernel version
|
14
|
-
- time/date information
|
15
|
-
- maximum number of file descriptor(ulimit)
|
14
|
+
- time/date information(ntp -q/chronyc sources)
|
15
|
+
- maximum number of file descriptor(ulimit -n)
|
16
16
|
- kernel network parameters(sysctl)
|
17
|
-
-
|
18
|
-
-
|
19
|
-
|
20
|
-
|
17
|
+
- snapshot of current process(ps)
|
18
|
+
- network conectivity status/stats(netstat -plan/netstat -s)
|
19
|
+
- memory information(/proc/meminfo)
|
20
|
+
<br>
|
21
21
|
|
22
22
|
## Prerequisite
|
23
|
+
The diagtool provides support for td-agent based installation running on Linux OS. The td-agent is a stable distribution package of Fluentd.
|
24
|
+
The differences between Fluentd and td-agent are described in followed url:
|
25
|
+
https://www.fluentd.org/faqs
|
23
26
|
|
27
|
+
## Diagtool Installation
|
28
|
+
|
29
|
+
```
|
30
|
+
# gem install fluent-diagtool
|
31
|
+
Fetching: fileutils-1.0.2.gem (100%)
|
32
|
+
Successfully installed fileutils-1.0.2
|
33
|
+
Fetching: json-2.1.0.gem (100%)
|
34
|
+
Building native extensions. This could take a while...
|
35
|
+
Successfully installed json-2.1.0
|
36
|
+
Fetching: fluent-diagtool-0.1.2.gem (100%)
|
37
|
+
Successfully installed fluent-diagtool-0.1.2
|
38
|
+
3 gems installed
|
39
|
+
```
|
24
40
|
|
25
41
|
## Usage
|
26
42
|
```
|
27
|
-
#
|
28
|
-
Usage: diagtool
|
43
|
+
# diagtool --help
|
44
|
+
Usage: /usr/local/bin/diagtool -o OUTPUT_DIR -m {yes | no} -w {word1,[word2...]} -f {listfile} -s {hash seed}
|
45
|
+
--precheck Run Precheck (Optional)
|
29
46
|
-o, --output DIR Output directory (Mandatory)
|
30
47
|
-m, --mask yes|no Enable mask function (Optional : Default=no)
|
31
48
|
-w, --word-list word1,word2 Provide a list of user-defined words which will to be masked (Optional : Default=None)
|
32
|
-
-f, --word-file
|
49
|
+
-f, --word-file list_file provide a file which describes a List of user-defined words (Optional : Default=None)
|
33
50
|
-s, --hash-seed seed provide a word which will be used when generate the mask (Optional : Default=None)
|
51
|
+
-c, --conf config_file provide a full path of td-agent configuration file (Optional : Default=None)
|
52
|
+
-l, --log log_file provide a full path of td-agent log file (Optional : Default=None)
|
53
|
+
```
|
54
|
+
### Pre-check
|
55
|
+
The diagtool automatically extract the path of td-agent configuration and log files from td-agent daemon and use them during data collection if the td-agent is managed as daemon. The precheck options provides the function to confirm if the diagtool could gather the td-agent information as expected.
|
56
|
+
The following command output shows the case when the diagtool successfully gather information from daemon.
|
57
|
+
```
|
58
|
+
# diagtool --precheck
|
59
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] Check OS parameters...
|
60
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] operating system = CentOS Linux 8 (Core)
|
61
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] kernel version = Linux 4.18.0-147.el8.x86_64
|
62
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] Check td-agent parameters...
|
63
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent conf path = /etc/td-agent/
|
64
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent conf file = td-agent.conf
|
65
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent log path = /var/log/td-agent/
|
66
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent log = td-agent.log
|
67
|
+
2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] Precheck completed. You can run diagtool command without -c and -l options
|
34
68
|
```
|
35
|
-
|
69
|
+
In some cases, users do not manage td-agent as daemon but use own script to run td-agent with command line options. In that cases, users need to speccify the path of td-agent configuration and log files with -c and -l options respectively.
|
70
|
+
The following example shows the precheck results when the diagtool is not able to extract the path of td-agent configuration and log files.
|
71
|
+
```
|
72
|
+
# diagtool --precheck
|
73
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] Check OS parameters...
|
74
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] operating system = CentOS Linux 8 (Core)
|
75
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] kernel version = Linux 4.18.0-147.5.1.el8_1.x86_64
|
76
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] Check td-agent parameters...
|
77
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] td-agent conf path =
|
78
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] td-agent conf file =
|
79
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] td-agent log path =
|
80
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] td-agent log =
|
81
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [WARN] [Precheck] can not find td-agent conf path: please run diagtool command with -c /path/to/<td-agent conf file>
|
82
|
+
2020-05-28 05:45:14 +0000: [Diagtool] [WARN] [Precheck] can not find td-agent log path: please run diagtool command with -l /path/to/<td-agent log file>
|
83
|
+
```
|
84
|
+
|
85
|
+
### Run diagtool
|
86
|
+
|
87
|
+
#### The "@include" directive in td-agent configuration file
|
88
|
+
The "@include" directive is a function to reuse configuration defined in another configuration files. The diagtool read the td-agent configuration and collect the files described in "@include" directive as well. The details of "@include" directive are described in followed url:
|
89
|
+
https://docs.fluentd.org/configuration/config-file#6-re-use-your-config-the-include-directive
|
90
|
+
|
91
|
+
#### User defined words to be masked
|
92
|
+
The user-defined words can be specified both -e option and -f option and the words are merged when both options are selected.
|
36
93
|
The format of user-defined words list file specified in -f option should be followed format.
|
37
94
|
```
|
38
95
|
# cat word_list_sample
|
@@ -43,7 +100,7 @@ NOTE: When user specified the keywork, only the exact match words will be masked
|
|
43
100
|
|
44
101
|
#### Command sample:
|
45
102
|
```
|
46
|
-
#
|
103
|
+
# diagtool -o /tmp/work1 -w passwd1,passwd2 -f word_list_sample -m yes
|
47
104
|
2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Parsing command options...
|
48
105
|
2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Option : Output directory = /tmp/work1
|
49
106
|
2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Option : Mask = yes
|
@@ -99,9 +156,9 @@ NOTE: When user specified the keywork, only the exact match words will be masked
|
|
99
156
|
2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Export mask log file : ./mask_20200512182119.json
|
100
157
|
2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] Generate tar file /tmp/work1/diagout-20200512182119.tar.gz
|
101
158
|
```
|
102
|
-
|
103
|
-
When run diagtool with mask option, the log of mask is also created in 'mask_{timestamp}.json' file. Users are able to confirm how the mask was generated on each files.
|
104
|
-
|
159
|
+
#### Mask Function
|
160
|
+
When run diagtool with mask option, the log of mask is also created in 'mask_{timestamp}.json' file. Users are able to confirm how the mask was generated on each files.
|
161
|
+
The diagtool provides hash-seed option with '-s'. When hash-seed is specified, the mask will be generated with original word and hash-seed so that users could use unique mask value.
|
105
162
|
#### Mask sample - IP address: IPv4_{md5hash}
|
106
163
|
```
|
107
164
|
"Line112-8": {
|
data/exe/diagtool
CHANGED
@@ -26,13 +26,21 @@ include Diagtool
|
|
26
26
|
params = {}
|
27
27
|
OptionParser.new do |opt|
|
28
28
|
opt.banner = "Usage: #{$0} -o OUTPUT_DIR -m {yes | no} -w {word1,[word2...]} -f {listfile} -s {hash seed}"
|
29
|
+
opt.on('--precheck', 'Run Precheck (Optional)')
|
29
30
|
opt.on('-o','--output DIR', String, 'Output directory (Mandatory)')
|
30
31
|
opt.on('-m','--mask yes|no', String, 'Enable mask function (Optional : Default=no)')
|
31
32
|
opt.on('-w','--word-list word1,word2', Array, 'Provide a list of user-defined words which will to be masked (Optional : Default=None)')
|
32
|
-
opt.on('-f','--word-file
|
33
|
+
opt.on('-f','--word-file list_file', String, 'provide a file which describes a List of user-defined words (Optional : Default=None)')
|
33
34
|
opt.on('-s','--hash-seed seed', String, 'provide a word which will be used when generate the mask (Optional : Default=None)')
|
35
|
+
opt.on('-c','--conf config_file', String, 'provide a full path of td-agent configuration file (Optional : Default=None)')
|
36
|
+
opt.on('-l','--log log_file', String, 'provide a full path of td-agent log file (Optional : Default=None)')
|
34
37
|
end.parse!(into: params)
|
38
|
+
|
35
39
|
diag = DiagUtils.new(params)
|
36
|
-
|
40
|
+
if params[:precheck]
|
41
|
+
diag.run_precheck()
|
42
|
+
else
|
43
|
+
diag.run_diagtool()
|
44
|
+
end
|
37
45
|
|
38
46
|
|
data/fluent-diagtool.gemspec
CHANGED
@@ -19,6 +19,6 @@ Gem::Specification.new do |spec|
|
|
19
19
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
20
|
spec.require_paths = ["lib"]
|
21
21
|
|
22
|
-
spec.add_runtime_dependency("fileutils"
|
23
|
-
spec.add_runtime_dependency("json"
|
22
|
+
spec.add_runtime_dependency("fileutils")
|
23
|
+
spec.add_runtime_dependency("json")
|
24
24
|
end
|
@@ -24,17 +24,36 @@ module Diagtool
|
|
24
24
|
@logger = Logger.new(STDOUT, level: log_level, formatter: proc {|severity, datetime, progname, msg|
|
25
25
|
"#{datetime}: [Diagutils] [#{severity}] #{msg}\n"
|
26
26
|
})
|
27
|
+
@precheck = conf[:precheck]
|
27
28
|
@time_format = conf[:time]
|
28
|
-
@
|
29
|
+
@basedir = conf[:basedir]
|
29
30
|
@workdir = conf[:workdir]
|
30
|
-
|
31
|
-
|
32
|
-
@
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
31
|
+
@outdir = conf[:outdir]
|
32
|
+
|
33
|
+
@tdenv = gen_tdenv()
|
34
|
+
if not conf[:tdconf].empty?
|
35
|
+
@tdconf = conf[:tdconf].split('/')[-1]
|
36
|
+
@tdconf_path = conf[:tdconf].gsub(@tdconf,'')
|
37
|
+
elsif
|
38
|
+
if not @tdenv['FLUENT_CONF'].empty?
|
39
|
+
@tdconf = @tdenv['FLUENT_CONF'].split('/')[-1]
|
40
|
+
@tdconf_path = @tdenv['FLUENT_CONF'].gsub(@tdconf,'')
|
41
|
+
else
|
42
|
+
raise "The path of td-agent configuration file need to be specified." if conf[:precheck] == false
|
43
|
+
end
|
44
|
+
end
|
45
|
+
if not conf[:tdlog].empty?
|
46
|
+
@tdlog = conf[:tdlog].split('/')[-1]
|
47
|
+
@tdlog_path = conf[:tdlog].gsub(@tdlog,'')
|
48
|
+
elsif
|
49
|
+
if not @tdenv['TD_AGENT_LOG_FILE'].empty?
|
50
|
+
@tdlog = @tdenv['TD_AGENT_LOG_FILE'].split('/')[-1]
|
51
|
+
@tdlog_path = @tdenv['TD_AGENT_LOG_FILE'].gsub(@tdlog,'')
|
52
|
+
else
|
53
|
+
raise "The path of td-agent log file need to be specified." if conf[:precheck] == false
|
54
|
+
end
|
55
|
+
end
|
56
|
+
@osenv = gen_osenv()
|
38
57
|
@oslog_path = '/var/log/'
|
39
58
|
@oslog = 'messages'
|
40
59
|
@sysctl_path = '/etc/'
|
@@ -48,31 +67,66 @@ module Diagtool
|
|
48
67
|
@logger.info(" td-agent log path = #{@tdlog_path}")
|
49
68
|
@logger.info(" td-agent log = #{@tdlog}")
|
50
69
|
end
|
51
|
-
|
70
|
+
|
71
|
+
def gen_osenv()
|
52
72
|
stdout, stderr, status = Open3.capture3('hostnamectl')
|
53
73
|
os_dict = {}
|
54
74
|
stdout.each_line { |l|
|
55
75
|
s = l.split(":")
|
56
76
|
os_dict[s[0].chomp.strip] = s[1].chomp.strip
|
57
77
|
}
|
58
|
-
|
59
|
-
|
78
|
+
if @precheck == false # SKip if precheck is true
|
79
|
+
File.open(@outdir+'/os_env.output', 'w') do |f|
|
80
|
+
f.puts(stdout)
|
81
|
+
end
|
60
82
|
end
|
61
83
|
return os_dict
|
62
84
|
end
|
63
|
-
|
85
|
+
|
86
|
+
def gen_tdenv()
|
64
87
|
stdout, stderr, status = Open3.capture3('systemctl cat td-agent')
|
65
88
|
env_dict = {}
|
66
|
-
|
67
|
-
|
68
|
-
|
89
|
+
if status.success?
|
90
|
+
if @precheck == false # SKip if precheck is true
|
91
|
+
File.open(@outdir+'/td-agent_env.output', 'w') do |f|
|
92
|
+
f.puts(stdout)
|
93
|
+
end
|
94
|
+
end
|
69
95
|
stdout.split().each do | l |
|
70
|
-
|
71
|
-
|
72
|
-
|
96
|
+
if l.include?('Environment')
|
97
|
+
env_dict[l.split('=')[1]] = l.split('=')[2]
|
98
|
+
end
|
99
|
+
end
|
100
|
+
else
|
101
|
+
exe = 'fluentd'
|
102
|
+
stdout, stderr, status = Open3.capture3("ps aux | grep #{exe} | grep -v grep")
|
103
|
+
line = stdout.split(/\n/)
|
104
|
+
log_path = ''
|
105
|
+
conf_path = ''
|
106
|
+
line.each { |l|
|
107
|
+
cmd = l.split.drop(10)
|
108
|
+
i = 0
|
109
|
+
log_pos = 0
|
110
|
+
conf_pos = 0
|
111
|
+
if cmd[-1] != '--under-supervisor'
|
112
|
+
cmd.each { |c|
|
113
|
+
if c.include?("--log") || c.include?("-l")
|
114
|
+
log_pos = i + 1
|
115
|
+
log_path = cmd[log_pos]
|
116
|
+
elsif c.include?("--conf") || c.include?("-c")
|
117
|
+
conf_pos = i + 1
|
118
|
+
conf_path = cmd[conf_pos]
|
119
|
+
end
|
120
|
+
i+=1
|
121
|
+
}
|
122
|
+
end
|
123
|
+
}
|
124
|
+
env_dict['FLUENT_CONF'] = conf_path
|
125
|
+
env_dict['TD_AGENT_LOG_FILE'] = log_path
|
73
126
|
end
|
74
127
|
return env_dict
|
75
128
|
end
|
129
|
+
|
76
130
|
def export_env()
|
77
131
|
env = {
|
78
132
|
:os => @osenv['Operating System'],
|
@@ -84,60 +138,118 @@ module Diagtool
|
|
84
138
|
}
|
85
139
|
return env
|
86
140
|
end
|
141
|
+
|
87
142
|
def collect_tdconf()
|
88
|
-
|
89
|
-
FileUtils.
|
90
|
-
|
143
|
+
target_dir = @workdir+@tdconf_path
|
144
|
+
FileUtils.mkdir_p(target_dir)
|
145
|
+
FileUtils.cp(@tdconf_path+@tdconf, target_dir)
|
146
|
+
conf = @workdir+@tdconf_path+@tdconf
|
147
|
+
conf_list = []
|
148
|
+
conf_list.push target_dir + @tdconf
|
149
|
+
File.readlines(conf).each { |line|
|
150
|
+
if line.include? '@include'
|
151
|
+
f = line.split()[1]
|
152
|
+
if f.start_with?(/\//) # /tmp/work1/b.conf
|
153
|
+
if f.include?('*')
|
154
|
+
Dir.glob(f).each { |ff|
|
155
|
+
conf_inc = target_dir + ff.gsub(/\//,'__')
|
156
|
+
FileUtils.cp(ff, conf_inc)
|
157
|
+
conf_list.push conf_inc
|
158
|
+
}
|
159
|
+
else
|
160
|
+
conf_inc = target_dir+f.gsub(/\//,'__')
|
161
|
+
FileUtils.cp(f, conf_inc)
|
162
|
+
conf_list.push conf_inc
|
163
|
+
end
|
164
|
+
else
|
165
|
+
f = f.gsub('./','') if f.include?('./')
|
166
|
+
if f.include?('*')
|
167
|
+
Dir.glob(@tdconf_path+f).each{ |ff|
|
168
|
+
conf_inc = target_dir + ff.gsub(@tdconf_path,'').gsub(/\//,'__')
|
169
|
+
FileUtils.cp(ff, conf_inc)
|
170
|
+
conf_list.push conf_inc
|
171
|
+
}
|
172
|
+
else
|
173
|
+
conf_inc = target_dir+f.gsub(/\//,'__')
|
174
|
+
FileUtils.cp(@tdconf_path+f, conf_inc)
|
175
|
+
conf_list.push conf_inc
|
176
|
+
end
|
177
|
+
end
|
178
|
+
end
|
179
|
+
}
|
180
|
+
return conf_list
|
91
181
|
end
|
182
|
+
|
92
183
|
def collect_tdlog()
|
93
|
-
|
94
|
-
FileUtils.
|
95
|
-
|
184
|
+
target_dir = @workdir+@tdlog_path
|
185
|
+
FileUtils.mkdir_p(target_dir)
|
186
|
+
Dir.glob(@tdlog_path+@tdlog+'*').each{ |f|
|
187
|
+
FileUtils.cp(f, target_dir)
|
188
|
+
}
|
189
|
+
return Dir.glob(target_dir+@tdlog+'*')
|
96
190
|
end
|
191
|
+
|
97
192
|
def collect_sysctl()
|
98
|
-
|
99
|
-
FileUtils.
|
100
|
-
|
193
|
+
target_dir = @workdir+@sysctl_path
|
194
|
+
FileUtils.mkdir_p(target_dir)
|
195
|
+
FileUtils.cp(@sysctl_path+@sysctl, target_dir)
|
196
|
+
return target_dir+@sysctl
|
101
197
|
end
|
198
|
+
|
102
199
|
def collect_oslog()
|
103
|
-
|
104
|
-
FileUtils.
|
105
|
-
|
200
|
+
target_dir = @workdir+@oslog_path
|
201
|
+
FileUtils.mkdir_p(target_dir)
|
202
|
+
FileUtils.cp(@oslog_path+@oslog, target_dir)
|
203
|
+
return target_dir+@oslog
|
106
204
|
end
|
205
|
+
|
107
206
|
def collect_ulimit()
|
108
|
-
output = @
|
109
|
-
stdout, stderr, status = Open3.capture3("ulimit -n")
|
207
|
+
output = @outdir+'/ulimit_n.output'
|
208
|
+
stdout, stderr, status = Open3.capture3("sh -c 'ulimit -n'")
|
110
209
|
File.open(output, 'w') do |f|
|
111
210
|
f.puts(stdout)
|
112
211
|
end
|
113
212
|
return output
|
114
213
|
end
|
214
|
+
|
215
|
+
def collect_ps_eo()
|
216
|
+
output = @outdir+'/ps_eo.output'
|
217
|
+
stdout, stderr, status = Open3.capture3("ps -eo pid,ppid,stime,time,%mem,%cpu,cmd")
|
218
|
+
File.open(output, 'w') do |f|
|
219
|
+
f.puts(stdout)
|
220
|
+
end
|
221
|
+
return output
|
222
|
+
end
|
223
|
+
|
115
224
|
def collect_meminfo()
|
116
|
-
output = @
|
225
|
+
output = @outdir+'/meminfo.output'
|
117
226
|
stdout, stderr, status = Open3.capture3("cat /proc/meminfo")
|
118
227
|
File.open(output, 'w') do |f|
|
119
228
|
f.puts(stdout)
|
120
229
|
end
|
121
230
|
return output
|
122
231
|
end
|
123
|
-
|
124
|
-
|
125
|
-
|
232
|
+
|
233
|
+
def collect_netstat_plan()
|
234
|
+
output = @outdir+'/netstat_plan.output'
|
235
|
+
stdout, stderr, status = Open3.capture3("netstat -plan")
|
126
236
|
File.open(output, 'w') do |f|
|
127
237
|
f.puts(stdout)
|
128
238
|
end
|
129
239
|
return output
|
130
240
|
end
|
241
|
+
|
131
242
|
def collect_netstat_s()
|
132
|
-
output = @
|
243
|
+
output = @outdir+'/netstat_s.output'
|
133
244
|
stdout, stderr, status = Open3.capture3("netstat -s")
|
134
245
|
File.open(output, 'w') do |f|
|
135
246
|
f.puts(stdout)
|
136
247
|
end
|
137
248
|
return output
|
138
249
|
end
|
250
|
+
|
139
251
|
def collect_ntp(command)
|
140
|
-
output = @
|
252
|
+
output = @outdir+'/ntp_info.output'
|
141
253
|
stdout_date, stderr_date, status_date = Open3.capture3("date")
|
142
254
|
stdout_ntp, stderr_ntp, status_ntp = Open3.capture3("chronyc sources") if command == "chrony"
|
143
255
|
stdout_ntp, stderr_ntp, status_ntp = Open3.capture3("ntpq -p") if command == "ntp"
|
@@ -147,19 +259,21 @@ module Diagtool
|
|
147
259
|
end
|
148
260
|
return output
|
149
261
|
end
|
262
|
+
|
150
263
|
def collect_tdgems()
|
151
|
-
output = @
|
264
|
+
output = @outdir+'/tdgem_list.output'
|
152
265
|
stdout, stderr, status = Open3.capture3("td-agent-gem list | grep fluent")
|
153
266
|
File.open(output, 'w') do |f|
|
154
267
|
f.puts(stdout)
|
155
268
|
end
|
156
269
|
return output
|
157
270
|
end
|
271
|
+
|
158
272
|
def compress_output()
|
159
|
-
Dir.chdir(@
|
273
|
+
Dir.chdir(@basedir)
|
160
274
|
tar_file = 'diagout-'+@time_format+'.tar.gz'
|
161
275
|
stdout, stderr, status = Open3.capture3("tar cvfz #{tar_file} #{@time_format}")
|
162
|
-
return @
|
276
|
+
return @basedir + '/' + tar_file
|
163
277
|
end
|
164
278
|
end
|
165
279
|
end
|
@@ -27,10 +27,62 @@ module Diagtool
|
|
27
27
|
time = Time.new
|
28
28
|
@time_format = time.strftime("%Y%m%d%0k%M%0S")
|
29
29
|
@conf = parse_diagconf(params)
|
30
|
+
#@conf[:time] = @time_format
|
31
|
+
#@conf[:workdir] = @conf[:basedir] + '/' + @time_format
|
32
|
+
#@conf[:outdir] = @conf[:workdir] + '/output'
|
33
|
+
|
34
|
+
#FileUtils.mkdir_p(@conf[:workdir])
|
35
|
+
#FileUtils.mkdir_p(@conf[:outdir])
|
36
|
+
|
37
|
+
#diaglog = @conf[:workdir] + '/diagtool.output'
|
38
|
+
#@masklog = './mask_' + @time_format + '.json'
|
39
|
+
#@logger = Logger.new(STDOUT, formatter: proc {|severity, datetime, progname, msg|
|
40
|
+
# "#{datetime}: [Diagtool] [#{severity}] #{msg}\n"
|
41
|
+
#})
|
42
|
+
#@logger_file = Logger.new(diaglog, formatter: proc {|severity, datetime, progname, msg|
|
43
|
+
# "#{datetime}: [Diagtool] [#{severity}] #{msg}\n"
|
44
|
+
#})
|
45
|
+
#diaglogger_info("Parsing command options...")
|
46
|
+
#diaglogger_info(" Option : Output directory = #{@conf[:basedir]}")
|
47
|
+
#diaglogger_info(" Option : Mask = #{@conf[:mask]}")
|
48
|
+
#diaglogger_info(" Option : Word list = #{@conf[:words]}")
|
49
|
+
#diaglogger_info(" Option : Hash Seed = #{@conf[:seed]}")
|
50
|
+
end
|
51
|
+
|
52
|
+
def run_precheck()
|
53
|
+
prechecklog = Logger.new(STDOUT, formatter: proc {|severity, datetime, progname, msg|
|
54
|
+
"#{datetime}: [Diagtool] [#{severity}] #{msg}\n"
|
55
|
+
})
|
56
|
+
loglevel = 'WARN'
|
57
|
+
c = CollectUtils.new(@conf, loglevel)
|
58
|
+
c_env = c.export_env()
|
59
|
+
prechecklog.info("[Precheck] Check OS parameters...")
|
60
|
+
prechecklog.info("[Precheck] operating system = #{c_env[:os]}")
|
61
|
+
prechecklog.info("[Precheck] kernel version = #{c_env[:kernel]}")
|
62
|
+
prechecklog.info("[Precheck] Check td-agent parameters...")
|
63
|
+
prechecklog.info("[Precheck] td-agent conf path = #{c_env[:tdconf_path]}")
|
64
|
+
prechecklog.info("[Precheck] td-agent conf file = #{c_env[:tdconf]}")
|
65
|
+
prechecklog.info("[Precheck] td-agent log path = #{c_env[:tdlog_path]}")
|
66
|
+
prechecklog.info("[Precheck] td-agent log = #{c_env[:tdlog]}")
|
67
|
+
if c_env[:tdconf_path] == nil || c_env[:tdconf] == nil
|
68
|
+
prechecklog.warn("[Precheck] can not find td-agent conf path: please run diagtool command with -c /path/to/<td-agent conf file>")
|
69
|
+
end
|
70
|
+
if c_env[:tdlog_path] == nil || c_env[:tdlog] == nil
|
71
|
+
prechecklog.warn("[Precheck] can not find td-agent log path: please run diagtool command with -l /path/to/<td-agent log file>")
|
72
|
+
end
|
73
|
+
if c_env[:tdconf_path] != nil && c_env[:tdconf] != nil && c_env[:tdlog_path] != nil && c_env[:tdlog] != nil
|
74
|
+
prechecklog.info("[Precheck] Precheck completed. You can run diagtool command without -c and -l options")
|
75
|
+
end
|
76
|
+
end
|
77
|
+
|
78
|
+
def run_diagtool()
|
30
79
|
@conf[:time] = @time_format
|
31
|
-
@conf[:workdir] = @conf[:
|
80
|
+
@conf[:workdir] = @conf[:basedir] + '/' + @time_format
|
81
|
+
@conf[:outdir] = @conf[:workdir] + '/output'
|
32
82
|
FileUtils.mkdir_p(@conf[:workdir])
|
83
|
+
FileUtils.mkdir_p(@conf[:outdir])
|
33
84
|
diaglog = @conf[:workdir] + '/diagtool.output'
|
85
|
+
|
34
86
|
@masklog = './mask_' + @time_format + '.json'
|
35
87
|
@logger = Logger.new(STDOUT, formatter: proc {|severity, datetime, progname, msg|
|
36
88
|
"#{datetime}: [Diagtool] [#{severity}] #{msg}\n"
|
@@ -39,12 +91,11 @@ module Diagtool
|
|
39
91
|
"#{datetime}: [Diagtool] [#{severity}] #{msg}\n"
|
40
92
|
})
|
41
93
|
diaglogger_info("Parsing command options...")
|
42
|
-
diaglogger_info(" Option : Output directory = #{@conf[:
|
94
|
+
diaglogger_info(" Option : Output directory = #{@conf[:basedir]}")
|
43
95
|
diaglogger_info(" Option : Mask = #{@conf[:mask]}")
|
44
96
|
diaglogger_info(" Option : Word list = #{@conf[:words]}")
|
45
97
|
diaglogger_info(" Option : Hash Seed = #{@conf[:seed]}")
|
46
|
-
|
47
|
-
def diagtool()
|
98
|
+
|
48
99
|
loglevel = 'WARN'
|
49
100
|
diaglogger_info("Initializing parameters...")
|
50
101
|
c = CollectUtils.new(@conf, loglevel)
|
@@ -78,10 +129,14 @@ module Diagtool
|
|
78
129
|
oslog = m.mask_tdlog(oslog, clean = true)
|
79
130
|
end
|
80
131
|
diaglogger_info("[Collect] config file is stored in #{oslog}")
|
81
|
-
|
132
|
+
|
133
|
+
diaglogger_info("[Collect] Collecting process information...")
|
134
|
+
meminfo = c.collect_ps_eo()
|
135
|
+
diaglogger_info("[Collect] process informationis stored in #{meminfo}")
|
136
|
+
|
82
137
|
diaglogger_info("[Collect] Collecting OS memory information...")
|
83
138
|
meminfo = c.collect_meminfo()
|
84
|
-
diaglogger_info("[Collect]
|
139
|
+
diaglogger_info("[Collect] OS memory information is stored in #{meminfo}")
|
85
140
|
|
86
141
|
diaglogger_info("[Collect] Collecting date/time information...")
|
87
142
|
if system('which chronyc > /dev/null 2>&1')
|
@@ -95,7 +150,7 @@ module Diagtool
|
|
95
150
|
|
96
151
|
diaglogger_info("[Collect] Collecting netstat information...")
|
97
152
|
if system('which netstat > /dev/null 2>&1')
|
98
|
-
netstat_n = c.
|
153
|
+
netstat_n = c.collect_netstat_plan()
|
99
154
|
netstat_s = c.collect_netstat_s()
|
100
155
|
if @conf[:mask] == 'yes'
|
101
156
|
diaglogger_info("[Mask] Masking netstat file : #{netstat_n}...")
|
@@ -121,22 +176,28 @@ module Diagtool
|
|
121
176
|
end
|
122
177
|
end
|
123
178
|
|
124
|
-
|
125
|
-
|
126
|
-
|
179
|
+
if system("sh -c 'ulimit -n'")
|
180
|
+
diaglogger_info("[Collect] Collecting ulimit information...")
|
181
|
+
ulimit = c.collect_ulimit()
|
182
|
+
diaglogger_info("[Collect] ulimit information is stored in #{ulimit}")
|
127
183
|
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
184
|
+
diaglogger_info("[Valid] Validating ulimit information...")
|
185
|
+
ret, rec, val = v.valid_ulimit(ulimit)
|
186
|
+
if ret == true
|
187
|
+
diaglogger_info("[Valid] ulimit => #{val} is correct (recommendation is >#{rec})")
|
188
|
+
else
|
189
|
+
diaglogger_warn("[Valid] ulimit => #{val} is incorrect (recommendation is >#{rec})")
|
190
|
+
end
|
132
191
|
else
|
133
|
-
diaglogger_warn("[
|
192
|
+
diaglogger_warn("[Collect] ulimit command does not exist. skip collectig ulimit")
|
134
193
|
end
|
135
194
|
|
136
195
|
if @conf[:mask] == 'yes'
|
137
|
-
|
138
|
-
|
139
|
-
|
196
|
+
tdconf.each { | file |
|
197
|
+
diaglogger_info("[Mask] Masking td-agent config file : #{file}...")
|
198
|
+
m.mask_tdlog(file, clean = true)
|
199
|
+
}
|
200
|
+
tdlog.each { | file |
|
140
201
|
diaglogger_info("[Mask] Masking td-agent log file : #{file}...")
|
141
202
|
filename = file.split("/")[-1]
|
142
203
|
if filename.include?(".gz")
|
@@ -144,7 +205,7 @@ module Diagtool
|
|
144
205
|
elsif
|
145
206
|
m.mask_tdlog(file, clean = true)
|
146
207
|
end
|
147
|
-
|
208
|
+
}
|
148
209
|
end
|
149
210
|
|
150
211
|
if @conf[:mask] == 'yes'
|
@@ -158,20 +219,23 @@ module Diagtool
|
|
158
219
|
|
159
220
|
def parse_diagconf(params)
|
160
221
|
options = {
|
161
|
-
:
|
162
|
-
:mask => 'no',
|
163
|
-
:words => [],
|
164
|
-
:wfile => '',
|
165
|
-
:seed => ''
|
222
|
+
:precheck => '', :basedir => '', :mask => '', :words => [], :wfile => '', :seed => '', :tdconf =>'', :tdlog => ''
|
166
223
|
}
|
167
|
-
if params[:
|
168
|
-
|
169
|
-
|
224
|
+
if params[:precheck]
|
225
|
+
options[:precheck] = params[:precheck]
|
226
|
+
else
|
227
|
+
options[:precheck] = false
|
228
|
+
end
|
229
|
+
if options[:precheck] == false
|
230
|
+
if params[:output] != nil
|
231
|
+
if Dir.exist?(params[:output])
|
232
|
+
options[:basedir] = params[:output]
|
233
|
+
else
|
234
|
+
raise "output directory '#{basedir}' does not exist"
|
235
|
+
end
|
170
236
|
else
|
171
|
-
raise "output directory '
|
237
|
+
raise "output directory '-o' must be specified"
|
172
238
|
end
|
173
|
-
else
|
174
|
-
raise "output directory '-o' must be specified"
|
175
239
|
end
|
176
240
|
if params[:mask] == nil
|
177
241
|
options[:mask] = 'no'
|
@@ -195,20 +259,43 @@ module Diagtool
|
|
195
259
|
end
|
196
260
|
options[:words] = options[:words].uniq
|
197
261
|
options[:seed] = params[:"hash-seed"] if params[:"hash-seed"] != nil
|
262
|
+
|
263
|
+
if params[:conf] != nil
|
264
|
+
f = params[:conf]
|
265
|
+
if File.exist?(f)
|
266
|
+
options[:tdconf] = params[:conf]
|
267
|
+
else
|
268
|
+
raise "#{params[:conf]} : No such file or directory"
|
269
|
+
end
|
270
|
+
end
|
271
|
+
|
272
|
+
if params[:log] != nil
|
273
|
+
f = params[:log]
|
274
|
+
if File.exist?(f)
|
275
|
+
options[:tdlog] = params[:log]
|
276
|
+
else
|
277
|
+
raise "#{params[:log]} : No such file or directory"
|
278
|
+
end
|
279
|
+
end
|
280
|
+
|
198
281
|
return options
|
199
282
|
end
|
283
|
+
|
200
284
|
def diaglogger_debug(str)
|
201
285
|
@logger.debug(str)
|
202
286
|
@logger_file.debug(str)
|
203
287
|
end
|
288
|
+
|
204
289
|
def diaglogger_info(str)
|
205
290
|
@logger.info(str)
|
206
291
|
@logger_file.info(str)
|
207
292
|
end
|
293
|
+
|
208
294
|
def diaglogger_warn(str)
|
209
295
|
@logger.warn(str)
|
210
296
|
@logger_file.warn(str)
|
211
297
|
end
|
298
|
+
|
212
299
|
def diaglogger_error(str)
|
213
300
|
@logger.error(str)
|
214
301
|
@logger_file.error(str)
|
@@ -29,13 +29,10 @@ module Diagtool
|
|
29
29
|
})
|
30
30
|
@logger.debug("Initialize Maskutils: sanitized word = #{conf[:words]}")
|
31
31
|
@hash_seed = conf[:seed]
|
32
|
-
@id = {
|
33
|
-
:fid =>'',
|
34
|
-
:lid =>'',
|
35
|
-
:cid =>''
|
36
|
-
}
|
32
|
+
@id = {}
|
37
33
|
@masklog = Hash.new { |h,k| h[k] = Hash.new(&h.default_proc) }
|
38
34
|
end
|
35
|
+
|
39
36
|
def mask_tdlog(input_file, clean)
|
40
37
|
line_id = 0
|
41
38
|
f = File.open(input_file+'.mask', 'w')
|
@@ -51,6 +48,7 @@ module Diagtool
|
|
51
48
|
FileUtils.rm(input_file) if clean == true
|
52
49
|
return input_file+'.mask'
|
53
50
|
end
|
51
|
+
|
54
52
|
def mask_tdlog_gz(input_file, clean)
|
55
53
|
line_id = 0
|
56
54
|
f = File.open(input_file+'.mask', 'w')
|
@@ -68,6 +66,7 @@ module Diagtool
|
|
68
66
|
FileUtils.rm(input_file) if clean == true
|
69
67
|
return input_file+'.mask'
|
70
68
|
end
|
69
|
+
|
71
70
|
def mask_tdlog_inspector(line)
|
72
71
|
i = 0
|
73
72
|
contents=[]
|
@@ -157,6 +156,7 @@ module Diagtool
|
|
157
156
|
@logger.debug("Masked Line: #{line_masked}")
|
158
157
|
return line_masked
|
159
158
|
end
|
159
|
+
|
160
160
|
def mask_direct_pattern(str)
|
161
161
|
is_mask = false
|
162
162
|
if str.include?(">")
|
@@ -173,6 +173,7 @@ module Diagtool
|
|
173
173
|
end
|
174
174
|
return is_mask, str_mask
|
175
175
|
end
|
176
|
+
|
176
177
|
def mask_url_pattern(str)
|
177
178
|
is_mask = false
|
178
179
|
url = str.split('://')
|
@@ -215,6 +216,7 @@ module Diagtool
|
|
215
216
|
str_mask << ":" if str.end_with?(':')
|
216
217
|
return is_mask, str_mask
|
217
218
|
end
|
219
|
+
|
218
220
|
def mask_equal_pattern(str)
|
219
221
|
is_mask = false
|
220
222
|
l = str.split('=') ## Mask host=<address:ip/hostname> or bind=<address: ip/hostname>
|
@@ -228,6 +230,7 @@ module Diagtool
|
|
228
230
|
str_mask = l.join('=')
|
229
231
|
return is_mask, str_mask
|
230
232
|
end
|
233
|
+
|
231
234
|
def mask_colon_pattern(str)
|
232
235
|
is_mask = false
|
233
236
|
l = str.split(':')
|
@@ -242,6 +245,7 @@ module Diagtool
|
|
242
245
|
str_mask << ":" if str.end_with?(':')
|
243
246
|
return is_mask, str_mask
|
244
247
|
end
|
248
|
+
|
245
249
|
def mask_slash_pattern(str)
|
246
250
|
is_mask = false
|
247
251
|
l = str.split('/')
|
@@ -256,14 +260,17 @@ module Diagtool
|
|
256
260
|
str_mask << ":" if str.end_with?(':')
|
257
261
|
return is_mask, str_mask
|
258
262
|
end
|
263
|
+
|
259
264
|
def is_ipv4?(str)
|
260
265
|
!!(str =~ /^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/)
|
261
266
|
end
|
267
|
+
|
262
268
|
def is_fqdn?(str)
|
263
269
|
#!!(str =~ /^\b((?=[a-z0-9-]{1,63}\.)[a-z0-9]+(-[a-z0-9]+)*\.)+([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$/)
|
264
270
|
!!(str =~ /^\b(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.){2,}([A-Za-z]|[A-Za-z][A-Za-z\-]*[A-Za-z]){2,}$/)
|
265
271
|
#!!(str =~ /^\b(?=^.{1,254}$)(^(?:(?!\d+\.)[a-zA-Z0-9_\-]{1,63}\.?)+(?:[a-zA-Z]{2,})$)/)
|
266
272
|
end
|
273
|
+
|
267
274
|
def is_words?(str)
|
268
275
|
value = false
|
269
276
|
@words.each do | l |
|
@@ -274,6 +281,7 @@ module Diagtool
|
|
274
281
|
end
|
275
282
|
return value
|
276
283
|
end
|
284
|
+
|
277
285
|
def mask_ipv4_fqdn_words(str)
|
278
286
|
str = str.to_s
|
279
287
|
mtype = ''
|
@@ -299,11 +307,13 @@ module Diagtool
|
|
299
307
|
end
|
300
308
|
return is_mask, str, str_mask
|
301
309
|
end
|
310
|
+
|
302
311
|
def put_masklog(str, str_mask)
|
303
312
|
uid = "Line#{@id[:lid]}-#{@id[:cid]}"
|
304
313
|
@masklog[@id[:fid]][uid]['original'] = str
|
305
314
|
@masklog[@id[:fid]][uid]['mask'] = str_mask
|
306
315
|
end
|
316
|
+
|
307
317
|
def export_masklog(output_file)
|
308
318
|
masklog_json = JSON.pretty_generate(@masklog)
|
309
319
|
File.open(output_file, 'w') do |f|
|
@@ -38,6 +38,7 @@ module Diagtool
|
|
38
38
|
@logger.debug(" Default ulimit: #{@def_ulimit}")
|
39
39
|
@logger.debug(" Default sysctl: #{@def_sysctl}")
|
40
40
|
end
|
41
|
+
|
41
42
|
def valid_ulimit(ulimit_file)
|
42
43
|
@logger.info("Loading ulimit file: #{ulimit_file}")
|
43
44
|
File.readlines(ulimit_file).each { |line|
|
@@ -50,6 +51,7 @@ module Diagtool
|
|
50
51
|
end
|
51
52
|
}
|
52
53
|
end
|
54
|
+
|
53
55
|
def valid_sysctl(sysctl_file)
|
54
56
|
h = Hash.new()
|
55
57
|
v = Hash.new { |i,j| i[j] = Hash.new(&h.default_proc) }
|
metadata
CHANGED
@@ -1,43 +1,43 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fluent-diagtool
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.7
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- kubotat
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-06-07 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: fileutils
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- - "
|
17
|
+
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version:
|
19
|
+
version: '0'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- - "
|
24
|
+
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version:
|
26
|
+
version: '0'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: json
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
|
-
- - "
|
31
|
+
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version:
|
33
|
+
version: '0'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
|
-
- - "
|
38
|
+
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version:
|
40
|
+
version: '0'
|
41
41
|
description: Bringing productivity of trouble shooting to the next level by automating
|
42
42
|
collection of Fluentd configurations, settings and OS parameters as well as masking
|
43
43
|
sensitive information in logs and configurations.
|
@@ -48,6 +48,7 @@ executables:
|
|
48
48
|
extensions: []
|
49
49
|
extra_rdoc_files: []
|
50
50
|
files:
|
51
|
+
- ".gitignore"
|
51
52
|
- AUTHORS
|
52
53
|
- Gemfile
|
53
54
|
- Gemfile.lock
|
@@ -55,7 +56,6 @@ files:
|
|
55
56
|
- README.md
|
56
57
|
- Rakefile
|
57
58
|
- bin/console
|
58
|
-
- bin/diagtool.rb
|
59
59
|
- bin/setup
|
60
60
|
- bin/word_list_sample
|
61
61
|
- exe/diagtool
|
data/bin/diagtool.rb
DELETED
@@ -1,37 +0,0 @@
|
|
1
|
-
#
|
2
|
-
# Fluentd
|
3
|
-
#
|
4
|
-
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
-
# you may not use this file except in compliance with the License.
|
6
|
-
# You may obtain a copy of the License at
|
7
|
-
#
|
8
|
-
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
-
#
|
10
|
-
# Unless required by applicable law or agreed to in writing, software
|
11
|
-
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
-
# See the License for the specific language governing permissions and
|
14
|
-
# limitations under the License.
|
15
|
-
#
|
16
|
-
|
17
|
-
#!/usr/bin/ruby
|
18
|
-
require 'optparse'
|
19
|
-
require '../lib/collectutils'
|
20
|
-
require '../lib/maskutils'
|
21
|
-
require '../lib/validutils'
|
22
|
-
require '../lib/diagutils'
|
23
|
-
include Diagtool
|
24
|
-
|
25
|
-
params = {}
|
26
|
-
OptionParser.new do |opt|
|
27
|
-
opt.banner = "Usage: #{$0} -o OUTPUT_DIR -m {yes | no} -w {word1,[word2...]} -f {listfile} -s {hash seed}"
|
28
|
-
opt.on('-o','--output DIR', String, 'Output directory (Mandatory)')
|
29
|
-
opt.on('-m','--mask yes|no', String, 'Enable mask function (Optional : Default=no)')
|
30
|
-
opt.on('-w','--word-list word1,word2', Array, 'Provide a list of user-defined words which will to be masked (Optional : Default=None)')
|
31
|
-
opt.on('-f','--word-file listfile', String, 'provide a file which describes a List of user-defined words (Optional : Default=None)')
|
32
|
-
opt.on('-s','--hash-seed seed', String, 'provide a word which will be used when generate the mask (Optional : Default=None)')
|
33
|
-
end.parse!(into: params)
|
34
|
-
diag = DiagUtils.new(params)
|
35
|
-
diag.diagtool()
|
36
|
-
|
37
|
-
|