fluent-diagtool 1.0.0 → 1.0.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (5) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +110 -105
  3. data/lib/fluent/diagtool/collectutils.rb +1 -1
  4. data/lib/fluent/diagtool/version.rb +1 -1
  5. metadata +4 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8c852c088190fa51d4232c45aa7afd4013473c8089e194511a0910a8ca4794e5
4
- data.tar.gz: 96b031de365ad1d47c71b7ffe25dfbd53f4909fe07be484944f7814cfead4939
3
+ metadata.gz: bfb1e90d1be9a7723371f6ea3cf72ea4eec822d732e2d940e54871cf20ce02d0
4
+ data.tar.gz: bbc10274d97eef4e0c79b5da843947389607519dd8f34e66ebc775ecf7171736
5
5
  SHA512:
6
- metadata.gz: c6c96dd92c4c8db2975c84395288817b5053b3cdb4b0897970a5e253fc2714886baf0799dd5bd4161439e043c4827a65c178adefde241e2e974de53265bebef8
7
- data.tar.gz: e7d24f7ca1d450e70bc037c74bf1ff8a88e2d174cc77c04211ba164bf7e7e97961b0d2970c8e3e00e0ab57f357c6c13aa8640afc1ffa8b9d7b63d5867630c9e4
6
+ metadata.gz: 1917964e68291c9a4cef64fca18139a1235f3a8bacc6eac8071881e4303cf4b18a0217fd0a8085641e4881a4847542b53b4b2390af8a50298f72259f3d6cba9b
7
+ data.tar.gz: d8c90fa8083e37510c55e1ad3059767585bc0b9c9adff3709e58d0e4225329c4ef4b5e5509a1323ab7af4e48101bf06384db82c56ad1dce99cbef6e512218a08
data/README.md CHANGED
@@ -1,10 +1,11 @@
1
1
  # Fluentd Diagnostic Tool
2
2
 
3
- The diagtool enables users to automate the date collection which is required for troubleshooting. The data collected by diagtool include the configuration and log files of the td-agent and diagnostic information from an operating system such as network and memory status and stats. In some cases, configuration and log files contain security sensitive information, such as IP addresses and Hostname. The diagtool also provides the functions to generate masks on IP addresses, Hostname(in FQDN style) and user defined keywords described in the collected data.
3
+ Diagtool enables users to automate the date collection which is required for troubleshooting. Diagtool gathers configuration and log files of Fluentd and diagnostic information from an operating system, such as process information and network status. In some cases, configuration and log files contain security sensitive information, such as IP addresses and Hostname. Diagtool has the functions to generate masks on IP addresses, Hostname(in FQDN style) and user defined keywords in the collected files.
4
+
4
5
  The scope of data collection:
5
- - TD Agent information
6
- - configuration files (*)
7
- - log files (*)
6
+ - Fluentd information
7
+ - configuration files
8
+ - log files
8
9
  - td-agent environment values
9
10
  - installed td-agent-gem list
10
11
  - OS information
@@ -20,27 +21,27 @@ The scope of data collection:
20
21
  <br>
21
22
 
22
23
  ## Prerequisite
23
- The diagtool provides support for td-agent based installation running on Linux OS. The td-agent is a stable distribution package of Fluentd.
24
- The differences between Fluentd and td-agent are described in followed url:
25
- https://www.fluentd.org/faqs
24
+ Diagtool has been developed for Fluentd(td-agent) and FluentBit(td-agent-bit) running on Linux OS and Diatool does not work for Windows.
25
+ Diagtool is written in Ruby and Ruby version should be higher than 2.3 for the installation.
26
+ The supported Linux OS is described in the following page:
27
+ https://docs.fluentd.org/quickstart/td-agent-v2-vs-v3-vs-v4
26
28
 
27
29
  ## Diagtool Installation
28
30
 
31
+ When you are using td-agent, you can install Diagtool easily with "/usr/sbin/td-agent-gem" command.
29
32
  ```
30
- # gem install fluent-diagtool
31
- Fetching: fileutils-1.0.2.gem (100%)
32
- Successfully installed fileutils-1.0.2
33
- Fetching: json-2.1.0.gem (100%)
34
- Building native extensions. This could take a while...
35
- Successfully installed json-2.1.0
36
- Fetching: fluent-diagtool-1.0.0.gem (100%)
33
+ # /usr/sbin/td-agent-gem install fluent-diagtool
37
34
  Successfully installed fluent-diagtool-1.0.0
38
- 3 gems installed
35
+ Parsing documentation for fluent-diagtool-1.0.0
36
+ Installing ri documentation for fluent-diagtool-1.0.0
37
+ Done installing documentation for fluent-diagtool after 0 seconds
38
+ 1 gem installed
39
39
  ```
40
- When you are using td-agent, fluent-adiagtool should be installed using /usr/sbin/td-agent-gem command instead of gem command.
40
+ When using /usr/sbin/td-agent-gem command, fluent-diagtool is installed under "/opt/td-agent/embedded/lib/ruby/gems/2.4.0/bin/" directory. You can add that directory to $PATH in .bash_profile.
41
+
42
+ Otherwise, you can install Diagtool with common gem command. In this case, Ruby version higher than 2.3 might be required to install.
41
43
  ```
42
- # /usr/sbin/td-agent-gem install fluent-diagtool
43
- Fetching fluent-diagtool-1.0.0.gem
44
+ # gem install fluent-diagtool
44
45
  Successfully installed fluent-diagtool-1.0.0
45
46
  Parsing documentation for fluent-diagtool-1.0.0
46
47
  Installing ri documentation for fluent-diagtool-1.0.0
@@ -48,11 +49,11 @@ Done installing documentation for fluent-diagtool after 0 seconds
48
49
  1 gem installed
49
50
  ```
50
51
 
52
+
51
53
  ## Usage
52
- There are a few options in Diagtool. You can check the options of Diagtool with "--help" options. Diagtool performs the validation function in the process by default but you can turn on/off the mask function depending on the use cases.
53
54
  ```
54
- # diagtool --help
55
- Usage: /usr/local/bin/diagtool -o OUTPUT_DIR -m {yes | no} -w {word1,[word2...]} -f {listfile} -s {hash seed}
55
+ # fluent-diagtool --help
56
+ Usage: fluent-diagtool -o OUTPUT_DIR -m {yes | no} -w {word1,[word2...]} -f {listfile} -s {hash seed}
56
57
  --precheck Run Precheck (Optional)
57
58
  -t, --type fluentd|fluentbit Select the type of Fluentd (Mandatory)
58
59
  -o, --output DIR Output directory (Mandatory)
@@ -63,26 +64,27 @@ Usage: /usr/local/bin/diagtool -o OUTPUT_DIR -m {yes | no} -w {word1,[word2...]}
63
64
  -c, --conf config_file provide a full path of td-agent configuration file (Optional : Default=None)
64
65
  -l, --log log_file provide a full path of td-agent log file (Optional : Default=None)
65
66
  ```
66
- ### Pre-check
67
- The diagtool automatically parses the path of Fluentd configuration and log files from running Fluentd processes and daemon. The precheck options provides the function to confirm if the diagtool could gather the fluentd information as expected.
68
- The following command output shows the case when the diagtool successfully gather information from daemon.
69
- You need to specify the type of Fluentd, "fluentd" or "fluentbit".
70
- ```
71
- # diagtool --precheck -t fluentd
72
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] Check OS parameters...
73
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] operating system = CentOS Linux 8 (Core)
74
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] kernel version = Linux 4.18.0-147.el8.x86_64
75
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] Check td-agent parameters...
76
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent conf path = /etc/td-agent/
77
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent conf file = td-agent.conf
78
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent log path = /var/log/td-agent/
79
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] td-agent log = td-agent.log
80
- 2020-05-28 00:39:02 -0400: [Diagtool] [INFO] [Precheck] Precheck completed. You can run diagtool command without -c and -l options
81
- ```
82
- In some cases, users do not manage td-agent as daemon but use their own scripts to run td-agent with command line options. In that cases, users need to specify the path of td-agent configuration and log files with -c and -l options respectively.
83
- The following example shows the precheck results when the diagtool is not able to extract the path of td-agent configuration and log files.
84
- ```
85
- # diagtool --precheck -t fluentd
67
+ ### Precheck
68
+ In order to run Diagtool correctly, it is required to ensure that Diagtool can obtain the fundamental information of Fluentd. Basically, Diagtool automatically parses the required information from the running Fluentd processes. The precheck option is useful to confirm if Diagtool certainly collects the information as expected.
69
+ The following output example shows the case where Diatool properly collects the required information.
70
+
71
+ ```
72
+ # fluent-diagtool --precheck -t fluentd
73
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] Fluentd Type = fluentd
74
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] Check OS parameters...
75
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] operating system = CentOS Linux 7 (Core)
76
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] kernel version = Linux 3.10.0-1127.10.1.el7.x86_64
77
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] Check td-agent parameters...
78
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] td-agent conf path = /etc/td-agent/
79
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] td-agent conf file = td-agent.conf
80
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] td-agent log path = /var/log/td-agent/
81
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] td-agent log = td-agent.log
82
+ 2020-10-07 21:20:33 +0000: [Diagtool] [INFO] [Precheck] Precheck completed. You can run diagtool command without -c and -l options
83
+ ```
84
+ In some cases, Dialtool, with custom command line options, may fail to identify the path of Fluentd configuration and log files. You need to specify this information manually with “-c” and “-l” options.
85
+ The following example shows pre-check returns failure resulting Diagtool is not able to extract the path of td-agent configuration and log files.
86
+ ```
87
+ # fluent-diagtool --precheck -t fluentd
86
88
  2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] Check OS parameters...
87
89
  2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] operating system = CentOS Linux 8 (Core)
88
90
  2020-05-28 05:45:14 +0000: [Diagtool] [INFO] [Precheck] kernel version = Linux 4.18.0-147.5.1.el8_1.x86_64
@@ -96,9 +98,72 @@ The following example shows the precheck results when the diagtool is not able t
96
98
  ```
97
99
 
98
100
  ### Run diagtool
101
+ Once the pre-check is completed, you are ready to run the tool. The “-o” is mandatory out of provided options and the output will be generated as a compressed file under the directory specified by “-o“ option.
102
+ (*) If the pre-check results mentioned that it is not able to find “td-agent conf path” and “td-agent log path“, you need to use “-c“ and “-l” respectively to specify the file path manually.
103
+
104
+ #### Command sample:
105
+ ```
106
+ # fluent-diagtool -t fluentd -o /tmp -w passwd1,passwd2 -m yes
107
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] Parsing command options...
108
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] Option : Output directory = /tmp
109
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] Option : Mask = yes
110
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] Option : Word list = ["passwd1", "passwd2"]
111
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] Option : Hash Seed =
112
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] Initializing parameters...
113
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] Loading the environment parameters...
114
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] operating system = CentOS Linux 7 (Core)
115
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] kernel version = Linux 3.10.0-1127.10.1.el7.x86_64
116
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] td-agent conf path = /etc/td-agent/
117
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] td-agent conf file = td-agent.conf
118
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] td-agent log path = /var/log/td-agent/
119
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] td-agent log = td-agent.log
120
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] Collecting log files of td-agent...
121
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] Collecting config file of td-agent...
122
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] config file is stored in ["/tmp/20201007212928/etc/td-agent/td-agent.conf", "/tmp/20201007212928/etc/td-agent/http_fld_system.conf"]
123
+ 2020-10-07 21:29:28 +0000: [Diagtool] [INFO] [Collect] Collecting td-agent gem information...
124
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] td-agent gem information is stored in /tmp/20201007212928/output/tdgem_list.output
125
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting config file of OS log...
126
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Mask] Masking OS log file : /tmp/20201007212928/var/log/messages...
127
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] config file is stored in /tmp/20201007212928/var/log/messages.mask
128
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting date/time information...
129
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] date/time information is stored in /tmp/20201007212928/output/chronyc_sources.txt
130
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting command output : command = ps -eo pid,ppid,stime,time,%mem,%cpu,cmd
131
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Mask] Masking command output file : /tmp/20201007212928/output/ps_-eo_pid_ppid_stime_time_%mem_%cpu_cmd.txt...
132
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting command output ps stored in /tmp/20201007212928/output/ps_-eo_pid_ppid_stime_time_%mem_%cpu_cmd.txt.mask
133
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting command output : command = cat /proc/meminfo
134
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Mask] Masking command output file : /tmp/20201007212928/output/cat_-proc-meminfo.txt...
135
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting command output cat stored in /tmp/20201007212928/output/cat_-proc-meminfo.txt.mask
136
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Collect] Collecting command output : command = netstat -plan
137
+ 2020-10-07 21:29:29 +0000: [Diagtool] [INFO] [Mask] Masking command output file : /tmp/20201007212928/output/netstat_-plan.txt...
138
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] Collecting command output netstat stored in /tmp/20201007212928/output/netstat_-plan.txt.mask
139
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] Collecting command output : command = netstat -s
140
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Mask] Masking command output file : /tmp/20201007212928/output/netstat_-s.txt...
141
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] Collecting command output netstat stored in /tmp/20201007212928/output/netstat_-s.txt.mask
142
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] Collecting systctl information...
143
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] sysctl information is stored in /tmp/20201007212928/output/sysctl_-a.txt
144
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Validating systctl information...
145
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_core_netdev_max_backlog => 5000 is correct (recommendation is 5000)
146
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_core_rmem_max => 16777216 is correct (recommendation is 16777216)
147
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_core_somaxconn => 1024 is correct (recommendation is 1024)
148
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_core_wmem_max => 16777216 is correct (recommendation is 16777216)
149
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_ip_local_port_range => ["10240", "65535"] is correct (recommendation is ["10240", "65535"])
150
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_max_syn_backlog => 8096 is correct (recommendation is 8096)
151
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_rmem => ["4096", "12582912", "16777216"] is correct (recommendation is ["4096", "12582912", "16777216"])
152
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_slow_start_after_idle => 0 is correct (recommendation is 0)
153
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_tw_reuse => 1 is correct (recommendation is 1)
154
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_wmem => ["4096", "12582912", "16777216"] is correct (recommendation is ["4096", "12582912", "16777216"])
155
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] Collecting ulimit information...
156
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] ulimit information is stored in /tmp/20201007212928/output/sh_-c_'ulimit_-n'.txt
157
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] Validating ulimit information...
158
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Valid] ulimit => 65536 is correct (recommendation is >65535)
159
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Mask] Masking td-agent config file : /tmp/20201007212928/etc/td-agent/td-agent.conf...
160
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Mask] Masking td-agent config file : /tmp/20201007212928/etc/td-agent/http_fld_system.conf...
161
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Mask] Export mask log file : ./mask_20201007212928.json
162
+ 2020-10-07 21:29:30 +0000: [Diagtool] [INFO] [Collect] Generate tar file /tmp/diagout-20201007212928.tar.gz
163
+ ```
99
164
 
100
165
  #### The "@include" directive in td-agent configuration file
101
- The "@include" directive is a function to reuse configuration defined in other configuration files. The diagtool reads the td-agent configuration and collects the files described in "@include" directive as well. The details of "@include" directive are described in followed url:
166
+ The "@include" directive is a function to reuse configuration defined in other configuration files. Diagtool reads Fluentd configuration and gathers the files described in "@include" directive as well. The details of "@include" directive are described in followed page:
102
167
  https://docs.fluentd.org/configuration/config-file#6-re-use-your-config-the-include-directive
103
168
 
104
169
  #### User defined words to be masked
@@ -111,66 +176,8 @@ centos8102
111
176
  ```
112
177
  NOTE: When user specified the keywork, only the exact match words will be masked. For instance, when users like to mask words like "nginx1" and "nginx2", users need to specify "nginx1" and "nginx2" respectively and "nginx*" should not work in the tool.
113
178
 
114
- #### Command sample:
115
- ```
116
- # diagtool -t fluentd -o /tmp/work1 -w passwd1,passwd2 -f word_list_sample -m yes
117
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Parsing command options...
118
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Option : Output directory = /tmp/work1
119
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Option : Mask = yes
120
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Option : Word list = ["passwd1", "passwd2", "centos8101", "centos8102"]
121
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Option : Hash Seed =
122
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] Initializing parameters...
123
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] Loading the environment parameters...
124
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] operating system = CentOS Linux 8 (Core)
125
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] kernel version = Linux 4.18.0-147.el8.x86_64
126
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] td-agent conf path = /etc/td-agent/
127
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] td-agent conf file = td-agent.conf
128
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] td-agent log path = /var/log/td-agent/
129
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] td-agent log = td-agent.log
130
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] Collecting log files of td-agent...
131
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] log files of td-agent are stored in ["/tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200508.gz", "/tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200509.gz", "/tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200507.gz", "/tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200512", "/tmp/work1/20200512182119/var/log/td-agent/td-agent.log"]
132
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] Collecting config file of td-agent...
133
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] config file is stored in /tmp/work1/20200512182119/etc/td-agent/td-agent.conf
134
- 2020-05-12 18:21:19 -0400: [Diagtool] [INFO] [Collect] Collecting td-agent gem information...
135
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] td-agent gem information is stored in /tmp/work1/20200512182119/etc/td-agent/tdgem_list.output
136
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] Collecting config file of OS log...
137
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Mask] Masking OS log file : /tmp/work1/20200512182119/var/log/messages...
138
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] config file is stored in /tmp/work1/20200512182119/var/log/messages.mask
139
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] Collecting OS memory information...
140
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] config file is stored in /tmp/work1/20200512182119/meminfo.output
141
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] Collecting date/time information...
142
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] date/time information is stored in /tmp/work1/20200512182119/ntp_info.output
143
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Collect] Collecting netstat information...
144
- 2020-05-12 18:21:20 -0400: [Diagtool] [INFO] [Mask] Masking netstat file : /tmp/work1/20200512182119/netstat_n.output...
145
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] netstat information is stored in /tmp/work1/20200512182119/netstat_n.output.mask and /tmp/work1/20200512182119/netstat_s.output
146
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] Collecting systctl information...
147
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] sysctl information is stored in /tmp/work1/20200512182119/etc/sysctl.conf
148
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Validating systctl information...
149
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_core_somaxconn => 1024 is correct (recommendation is 1024)
150
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_core_netdev_max_backlog => 5000 is correct (recommendation is 5000)
151
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_core_rmem_max => 16777216 is correct (recommendation is 16777216)
152
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_core_wmem_max => 16777216 is correct (recommendation is 16777216)
153
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_wmem => ["4096", "12582912", "16777216"] is correct (recommendation is ["4096", "12582912", "16777216"])
154
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_rmem => ["4096", "12582912", "16777216"] is correct (recommendation is ["4096", "12582912", "16777216"])
155
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_max_syn_backlog => 8096 is correct (recommendation is 8096)
156
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_slow_start_after_idle => 0 is correct (recommendation is 0)
157
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_tcp_tw_reuse => 1 is correct (recommendation is 1)
158
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Sysctl: net_ipv4_ip_local_port_range => ["10240", "65535"] is correct (recommendation is ["10240", "65535"])
159
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] Collecting ulimit information...
160
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] ulimit information is stored in /tmp/work1/20200512182119/ulimit_n.output
161
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] Validating ulimit information...
162
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Valid] ulimit => 65536 is correct (recommendation is >65535)
163
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Masking td-agent config file : /tmp/work1/20200512182119/etc/td-agent/td-agent.conf...
164
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Masking td-agent log file : /tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200508.gz...
165
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Masking td-agent log file : /tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200509.gz...
166
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Masking td-agent log file : /tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200507.gz...
167
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Masking td-agent log file : /tmp/work1/20200512182119/var/log/td-agent/td-agent.log-20200512...
168
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Masking td-agent log file : /tmp/work1/20200512182119/var/log/td-agent/td-agent.log...
169
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Mask] Export mask log file : ./mask_20200512182119.json
170
- 2020-05-12 18:21:22 -0400: [Diagtool] [INFO] [Collect] Generate tar file /tmp/work1/diagout-20200512182119.tar.gz
171
- ```
172
179
  #### Mask Function
173
- When run diagtool with the mask option, the log of mask is also created in 'mask_{timestamp}.json' file. Users are able to confirm how the mask was generated on each file.
180
+ When run Diagtool with the mask option, the log of mask is also created in 'mask_{timestamp}.json' file. Users are able to confirm how the mask was generated on each file.
174
181
  The diagtool provides a hash-seed option with '-s'. When hash-seed is specified, the mask will be generated with the original word and hash-seed so that users could use a unique mask value.
175
182
  #### Mask sample - IP address: IPv4_{md5hash}
176
183
  ```
@@ -196,8 +203,6 @@ The diagtool provides a hash-seed option with '-s'. When hash-seed is specified,
196
203
  ```
197
204
 
198
205
  ## Tested Environment
199
- - OS : CentOS 8.1
200
- - Fluentd : td-agent version 3/4
206
+ - OS : CentOS 8.1 / Ubuntu 20.04
207
+ - Fluentd : td-agent version 3/4
201
208
  https://docs.fluentd.org/quickstart/td-agent-v2-vs-v3
202
- - Fluentbit : td-agent-bit
203
-
data/lib/fluent/diagtool/collectutils.rb CHANGED
@@ -240,7 +240,7 @@ module Diagtool
240
240
  end
241
241
  inc_list.push inc_http
242
242
  else
243
- if l.start_with?(/\//) # /tmp/work1/b.conf
243
+ if l.start_with?('/') # /tmp/work1/b.conf
244
244
  if l.include?('*')
245
245
  Dir.glob(l).each { |ll|
246
246
  inc_conf = target_dir + ll.gsub(/\//,'-')
data/lib/fluent/diagtool/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  module Fluent
2
2
  module Diagtool
3
- VERSION = "1.0.0"
3
+ VERSION = "1.0.1"
4
4
  end
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-diagtool
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0
4
+ version: 1.0.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - kubotat
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2020-10-07 00:00:00.000000000 Z
11
+ date: 2020-10-29 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fileutils
@@ -85,7 +85,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
85
85
  - !ruby/object:Gem::Version
86
86
  version: '0'
87
87
  requirements: []
88
- rubygems_version: 3.1.2
88
+ rubyforge_project:
89
+ rubygems_version: 2.7.6.2
89
90
  signing_key:
90
91
  specification_version: 4
91
92
  summary: Diagnostic Tool for Fluentd