fluent-plugin-watch-process 0.0.1 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 2d114a31b165c3feca90dac92247785d9f757e8951ded1aac49bc30806e505a1
4
+ data.tar.gz: 9a2aab2afb95873873e3e3581b2f5f23b12bd0e7062d456b7eebe257c7f8473a
5
+ SHA512:
6
+ metadata.gz: ad4b0bffb25faaed2082c8bab5904100750ee265113097a0f643d5c13305c53731a9afcb6bd0b661fba1d012b45e74294c57defd530a774fbaf052135bcb6f43
7
+ data.tar.gz: 8f4761bfec54a36ecb2f702d7749ce4fd21bb69f35834aaeef274167456de74e0e8c4d7d09bdaaa90dc9bc3647d66eafbeb16d81ca24aede35fb4d273edd7e43
data/.travis.yml CHANGED
@@ -1,6 +1,13 @@
1
1
  language: ruby
2
2
 
3
3
  rvm:
4
- - 2.1.0
5
- - 2.0.0
6
- - 1.9.3
4
+ - 2.3.1
5
+ - 2.2
6
+ - 2.1
7
+
8
+ before_install:
9
+ - gem update bundler
10
+
11
+ gemfile:
12
+ - Gemfile
13
+ - gemfiles/fluentd_v0.14.gemfile
data/Appraisals ADDED
@@ -0,0 +1,7 @@
1
+ appraise "fluentd v0.12" do
2
+ gem "fluentd", "~> 0.12.0"
3
+ end
4
+
5
+ appraise "fluentd v0.14" do
6
+ gem "fluentd", "~> 0.14.0"
7
+ end
data/README.md CHANGED
@@ -3,17 +3,17 @@ fluent-plugin-watch-process [![Build Status](https://travis-ci.org/y-ken/fluent-
3
3
 
4
4
  ## Overview
5
5
 
6
- Fluentd Input plugin to collect process information via ps command.
6
+ Fluentd Input plugin to collect continual process information via ps command. It is useful for cron/barch process monitoring.
7
7
 
8
8
  ## Use Cases
9
9
 
10
- * collect cron/batch process for analysis.
10
+ * collect cron/batch process for long term analysis.
11
11
  * high cpu load time
12
12
  * high usage of memory
13
13
  * determine too long running task
14
14
 
15
15
  * output destination example
16
- * Elasticsearch + Kibana to visualize statistics. Example: [example1.conf](https://github.com/y-ken/fluent-plugin-watch-process/blob/master/example1.conf)
16
+ * Elasticsearch + Kibana to visualize cron/batch process statistics. Example: [example1.conf](https://github.com/y-ken/fluent-plugin-watch-process/blob/master/example1.conf)
17
17
  * save process information as audit log into AWS S3 which filename isolated by hostname. Example: [example2.conf](https://github.com/y-ken/fluent-plugin-watch-process/blob/master/example2.conf)
18
18
 
19
19
  ## Installation
@@ -25,7 +25,10 @@ install with gem or fluent-gem command as:
25
25
  $ gem install fluent-plugin-watch-process
26
26
 
27
27
  # for td-agent
28
- $ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-watch-process
28
+ $ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-watch-process -v 0.1.1
29
+
30
+ # for td-agent2 (recommend)
31
+ $ sudo td-agent-gem install fluent-plugin-watch-process -v 0.1.1
29
32
  ```
30
33
 
31
34
  ## Configuration
@@ -36,14 +39,14 @@ It is a quick sample to output log to `/var/log/td-agent/td-agent.log` with td-a
36
39
 
37
40
  `````
38
41
  <source>
39
- type watch_process
42
+ @type watch_process
40
43
  tag debug.batch.${hostname} # Required
41
44
  lookup_user batchuser # Optional
42
45
  interval 10s # Optional (default: 5s)
43
46
  </source>
44
47
 
45
48
  <match debug.**>
46
- type stdout
49
+ @type stdout
47
50
  </match>
48
51
  `````
49
52
 
@@ -51,41 +54,96 @@ After restarting td-agent, it will output process information to the td-agent.lo
51
54
 
52
55
  `````
53
56
  $ tail -f /var/log/td-agent/td-agent.log
57
+ ...snip...
54
58
  2014-01-16 14:21:34 +0900 debug.batch.localhost: {"start_time":"2014-01-16 14:21:13 +0900","user":"td-agent","pid":17486,"parent_pid":17483,"cpu_time":"00:00:00","cpu_percent":1.5,"memory_percent":3.5,"mem_rss":36068,"mem_size":60708,"state":"S","proc_name":"ruby","command":"/usr/lib64/fluent/ruby/bin/ruby /usr/sbin/td-agent --group td-agent --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid","elapsed_time":21}
55
59
  `````
56
60
 
57
61
  ### Syntax
58
62
 
59
- * tag # Required
63
+ * tag (Required)
60
64
  * record output destination
61
65
  * supported tag placeholders are `${hostname}` and `__HOSTNAME__`.
62
66
 
63
- * command # Optional
67
+ * command (Optional)
64
68
  * execute ps command with some options
65
69
  * [default] Linux: `LANG=en_US.UTF-8 && ps -ewwo lstart,user:20,pid,ppid,time,%cpu,%mem,rss,sz,s,comm,cmd`
66
70
  * [default] MacOSX: `LANG=en_US.UTF-8 && ps -ewwo lstart,user,pid,ppid,time,%cpu,%mem,rss,vsz,state,comm,command`
71
+ * [default] Windows: described in the next section, `About Windows`.
67
72
 
68
- * keys # Optional
73
+ * keys (Optional)
69
74
  * output record keys of the ps command results
70
- * [default] start_time user pid parent_pid cpu_time cpu_percent memory_percent mem_rss mem_size state proc_name command
75
+ * [default] Linux, MacOSX: `start_time,user,pid,parent_pid,cpu_time,cpu_percent,memory_percent,mem_rss,mem_size,state,proc_name,command`
76
+ * need to modify `command` too if you modify this value.
77
+ * [default] Windows: `StartTime,UserName,SessionId,Id,CPU,WorkingSet,VirtualMemorySize,HandleCount,ProcessName`
78
+ * in Windows only, you can fix this without fixing `command`. These keys can be specified from the properties of `System.Diagnostics.Process` object of `.NET`.
79
+ * `UserName` key needs administrator privilege. You can exclude this to avoid needing administrator privilege.
71
80
 
72
- * types # Optional
81
+ * types (Optional)
73
82
  * settings of converting types from string to integer/float.
74
- * [default] pid:integer parent_pid:integer cpu_percent:float memory_percent:float mem_rss:integer mem_size:integer
83
+ * [default] Linux, MacOSX: `pid:integer,parent_pid:integer,cpu_percent:float,memory_percent:float,mem_rss:integer,mem_size:integer`
84
+ * [default] Windows: `SessionId:integer,Id:integer,CPU:float,WorkingSet:integer,VirtualMemorySize:integer,HandleCount:integer`
75
85
 
76
- * interval # Optional
86
+ * interval (Optional)
77
87
  * execute interval time
78
88
  * [default] 5s
79
89
 
80
- * lookup_user # Optional
90
+ * lookup_user (Optional)
81
91
  * filter process owner username with comma delimited
82
92
  * [default] N/A
83
93
 
84
- * hostname_command # Optional
94
+ * hostname_command (Optional)
85
95
  * settings for tag placeholder, `${hostname}` and `__HOSTNAME__`. By default, it using long hostname.
86
96
  * to use short hostname, set `hostname -s` for this option on linux/mac.
87
97
  * [default] `hostname`
88
98
 
99
+ * powershell_command (Optional)
100
+ * settings for powershell command name. PowerShell Core had been renamed its command to `pwsh` and PowerShell 7 continues to use `pwsh` as its command name.
101
+ * [default] `powershell`
102
+ * [avaliables] `powershell`, `pwsh`
103
+
104
+ ### About Windows
105
+
106
+ Default `command` preset for Windows provides many of keys as below. Generally, you can pick up the columns with `keys` option.
107
+ If you need additional keys, consider to update `command` option.
108
+
109
+ `````powershell
110
+ powershell -command "Get-Process -IncludeUserName
111
+ | ?{$_.StartTime -ne $NULL -and $_.CPU -ne $NULL}
112
+ | Select-Object -Property StartTime,UserName,SessionId,Id,CPU,WorkingSet,VirtualMemorySize,HandleCount,ProcessName
113
+ | %{$_.StartTime = $_.StartTime.ToString('o'); return $_;}
114
+ | ConvertTo-Csv -NoTypeInformation"
115
+ `````
116
+
117
+ Confirmed versions are:
118
+
119
+ | Windows version | PowerShell version information | Note |
120
+ | ------------------------------ | ------------------------------------------------------------------------- |-----------------------------------------------|
121
+ | Windows 10 10.0.19042 (20H2) | PSVersion: 5.1.19041.906 (default installed version), PSEdition: Desktop | `powershell_command` as `powershell` (default)|
122
+ | Windows 10 10.0.19042 (20H2) | PSVersion: 7.1.2, PSEdition: Core | `powershell_command` as `pwsh` |
123
+
124
+
125
+ Here are details of this default command.
126
+
127
+ * `Get-Process -IncludeUserName`
128
+ * `Get-Process` powershell command takes `System.Diagnostics.Process` objects.
129
+ * `IncludeUserName` option is needed to take `UserName`.
130
+ * this needs administrator privilege.
131
+ * this will be omitted if `keys` does not contain `UserName`.
132
+ * ` | ?{$_.StartTime -ne $NULL -and $_.CPU -ne $NULL}`
133
+ * this exlcludes some special processes that don't have some properties, such as the "Idle" process in Windows.
134
+ * ` | Select-Object -Property ...`
135
+ * this takes the necessary parameters from `System.Diagnostics.Process` objects.
136
+ * `...` part will be automatically fixed by `keys`.
137
+ * ` | %{$_.StartTime = $_.StartTime.ToString('o'); return $_;}`
138
+ * this fixes the format of `StartTime` value.
139
+ * note: in Windows, setting the "$env:Lang" environment variable is not effective in changing the format of the output.
140
+ * ` | ConvertTo-Csv -NoTypeInformation`
141
+ * this formats objects to csv strings.
142
+ * currently, it is needed that `command` outputs the results in csv format.
143
+ * this is because white space delimiter is not suitable for Windows, in which empty values are often mixed.
144
+
145
+ **Note:** When using with PowerShell 7 which is previously known as PowerShell Core, you must specify `powershell_command` parameter as `pwsh`. Otherwise, this plugin does not work correctly on PowerShell 7 (pwsh). This is because PowerShell Core and PowerShell 7 use different command name which is `pwsh` not `powershell`.
146
+
89
147
  ## FAQ
90
148
 
91
149
  * I need hostname key in the record.
data/example1.conf CHANGED
@@ -6,10 +6,10 @@
6
6
  # * fluent-plugin-elasticsearch
7
7
 
8
8
  <source>
9
- type watch_process
9
+ @type watch_process
10
10
 
11
11
  # specify output tag
12
- tag batch_process
12
+ tag batch.process
13
13
 
14
14
  # filter specific user owned process. if no need to filter, delete this line.
15
15
  lookup_user batchuser
@@ -18,18 +18,16 @@
18
18
  interval 10s
19
19
  </source>
20
20
 
21
- <match batch_process>
22
- type record_reformer
23
- output_tag reformed.${tag}
24
- enable_ruby false
21
+ # add hostname key into record
22
+ <filter batch.process>
23
+ @type record_transformer
25
24
  <record>
26
- # add hostname key into record
27
- hostname ${hostname}
25
+ hostname ${hostname}
28
26
  </record>
29
- </match>
27
+ </filter>
30
28
 
31
- <match reformed.*>
32
- type elasticsearch
29
+ <match batch.*>
30
+ @type elasticsearch
33
31
  host localhost
34
32
  port 9200
35
33
  logstash_format true
data/example2.conf CHANGED
@@ -5,9 +5,10 @@
5
5
  # * fluent-plugin-s3
6
6
 
7
7
  <source>
8
- type watch_process
8
+ @type watch_process
9
9
 
10
10
  # use ${hostname} placeholder to use filename
11
+ # If you want to customize it, Use #{Socket.gethostname} or #{Socket.gethostname.split('.').first} snippet for tag.
11
12
  tag batch_process.${hostname}
12
13
 
13
14
  # filter specific user owned process. if no need to filter, delete this line.
@@ -18,7 +19,7 @@
18
19
  </source>
19
20
 
20
21
  <match batch_process.*>
21
- type s3
22
+ @type s3
22
23
 
23
24
  aws_key_id YOUR_AWS_KEY_ID
24
25
  aws_sec_key YOUR_AWS_SECRET/KEY
@@ -3,11 +3,11 @@ $:.push File.expand_path("../lib", __FILE__)
3
3
 
4
4
  Gem::Specification.new do |s|
5
5
  s.name = "fluent-plugin-watch-process"
6
- s.version = "0.0.1"
6
+ s.version = "0.2.0"
7
7
  s.authors = ["Kentaro Yoshida"]
8
8
  s.email = ["y.ken.studio@gmail.com"]
9
9
  s.homepage = "https://github.com/y-ken/fluent-plugin-watch-process"
10
- s.summary = %q{Fluentd Input plugin to collect process information via ps command.}
10
+ s.summary = %q{Fluentd Input plugin to collect continual process information via ps command or PowerShell pwsh command for Linux/osx/Windows. It is useful for cron/barch process monitoring.}
11
11
 
12
12
  s.files = `git ls-files`.split("\n")
13
13
  s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
@@ -16,5 +16,9 @@ Gem::Specification.new do |s|
16
16
 
17
17
  # specify any dependencies:
18
18
  s.add_development_dependency "rake"
19
- s.add_runtime_dependency "fluentd"
19
+ s.add_development_dependency "test-unit", ">= 3.1.0"
20
+ s.add_development_dependency "appraisal"
21
+ s.add_runtime_dependency "fluentd", [">= 0.14.0", "< 2"]
22
+ s.add_runtime_dependency "fluent-mixin-rewrite-tag-name"
23
+ s.add_runtime_dependency "fluent-mixin-type-converter", ">= 0.1.0"
20
24
  end
@@ -0,0 +1,7 @@
1
+ # This file was generated by Appraisal
2
+
3
+ source "http://rubygems.org"
4
+
5
+ gem "fluentd", "~> 0.14.0"
6
+
7
+ gemspec :path => "../"
@@ -1,116 +1,221 @@
1
- module Fluent
2
- class WatchProcessInput < Fluent::Input
3
- Plugin.register_input('watch_process', self)
1
+ require 'time'
2
+ require 'csv' if Fluent.windows?
3
+ require "fluent/plugin/input"
4
+ require 'fluent/mixin/rewrite_tag_name'
5
+ require 'fluent/mixin/type_converter'
6
+
7
+ module Fluent::Plugin
8
+ class WatchProcessInput < Fluent::Plugin::Input
9
+ Fluent::Plugin.register_input('watch_process', self)
10
+
11
+ helpers :timer
12
+
13
+ DEFAULT_KEYS = %w(start_time user pid parent_pid cpu_time cpu_percent memory_percent mem_rss mem_size state proc_name command)
14
+ DEFAULT_TYPES = %w(
15
+ pid:integer
16
+ parent_pid:integer
17
+ cpu_percent:float
18
+ memory_percent:float
19
+ mem_rss:integer
20
+ mem_size:integer
21
+ ).join(",")
4
22
 
5
23
  config_param :tag, :string
6
24
  config_param :command, :string, :default => nil
7
- config_param :keys, :string, :default => nil
8
- config_param :types, :string, :default => nil
9
- config_param :interval, :string, :default => '5s'
10
- config_param :lookup_user, :string, :default => nil
25
+ config_param :keys, :array, :default => nil
26
+ config_param :interval, :time, :default => '5s'
27
+ config_param :lookup_user, :array, :default => nil
11
28
  config_param :hostname_command, :string, :default => 'hostname'
29
+ config_param :powershell_command, :enum, list: [:powershell, :pwsh], :default => :powershell
12
30
 
13
- Converters = {
14
- 'string' => lambda { |v| v.to_s },
15
- 'integer' => lambda { |v| v.to_i },
16
- 'float' => lambda { |v| v.to_f },
17
- 'bool' => lambda { |v|
18
- case v.downcase
19
- when 'true', 'yes', '1'
20
- true
21
- else
22
- false
23
- end
24
- },
25
- 'time' => lambda { |v, time_parser|
26
- time_parser.parse(v)
27
- },
28
- 'array' => lambda { |v, delimiter|
29
- v.to_s.split(delimiter)
30
- }
31
- }
31
+ include Fluent::HandleTagNameMixin
32
+ include Fluent::Mixin::RewriteTagName
33
+ include Fluent::Mixin::TypeConverter
32
34
 
33
35
  def initialize
34
36
  super
35
- require 'time'
36
37
  end
37
38
 
38
39
  def configure(conf)
39
40
  super
40
41
 
41
- @command = @command || get_ps_command
42
- @keys = @keys || %w(start_time user pid parent_pid cpu_time cpu_percent memory_percent mem_rss mem_size state proc_name command)
43
- types = @types || %w(pid:integer parent_pid:integer cpu_percent:float memory_percent:float mem_rss:integer mem_size:integer)
44
- @types_map = Hash[types.map{|v| v.split(':')}]
45
- @lookup_user = @lookup_user.gsub(' ', '').split(',') unless @lookup_user.nil?
46
- @interval = Config.time_value(@interval)
47
- @hostname = `#{@hostname_command}`.chomp
48
- $log.info "watch_process: polling start. :tag=>#{@tag} :lookup_user=>#{@lookup_user} :interval=>#{@interval} :command=>#{@command}"
42
+ @windows_watcher = WindowsWatcher.new(@keys, @command, @lookup_user, @powershell_command) if Fluent.windows?
43
+ @keys ||= Fluent.windows? ? @windows_watcher.keys : DEFAULT_KEYS
44
+ @command ||= get_ps_command
45
+ apply_default_types
46
+ log.info "watch_process: polling start. :tag=>#{@tag} :lookup_user=>#{@lookup_user} :interval=>#{@interval} :command=>#{@command}"
49
47
  end
50
48
 
51
49
  def start
52
- @thread = Thread.new(&method(:run))
50
+ super
51
+ timer_execute(:in_watch_process, @interval, &method(:on_timer))
53
52
  end
54
53
 
55
54
  def shutdown
56
- Thread.kill(@thread)
55
+ super
56
+ end
57
+
58
+ def apply_default_types
59
+ return unless @types.nil?
60
+ @types = Fluent.windows? ? @windows_watcher.default_types : DEFAULT_TYPES
61
+ @type_converters = parse_types_parameter unless @types.nil?
57
62
  end
58
63
 
59
- def run
60
- loop do
61
- io = IO.popen(@command, 'r')
64
+ def on_timer
65
+ io = IO.popen(@command, 'r')
66
+ begin
62
67
  io.gets
63
68
  while result = io.gets
64
- values = result.chomp.strip.split(/\s+/, @keys.size + 4)
65
- time = Time.parse(values[0...5].join(' '))
66
- data = Hash[
67
- @keys.zip([time.to_s, values.values_at(5..15)].flatten).map do |k,v|
68
- v = Converters[@types_map[k]].call(v) if @types_map.include?(k)
69
- [k,v]
70
- end
71
- ]
72
- data['elapsed_time'] = (Time.now - Time.parse(data['start_time'])).to_i
73
- next unless @lookup_user.nil? || @lookup_user.include?(data['user'])
74
- tag = @tag.gsub(/(\${[a-z]+}|__[A-Z]+__)/, get_placeholder)
75
- Engine.emit(tag, Engine.now, data)
69
+ if Fluent.windows?
70
+ data = @windows_watcher.parse_line(result)
71
+ next unless @windows_watcher.match_look_up_user?(data)
72
+ else
73
+ data = parse_line(result)
74
+ next unless match_look_up_user?(data)
75
+ end
76
+ emit_tag = tag.dup
77
+ filter_record(emit_tag, Fluent::Engine.now, data)
78
+ router.emit(emit_tag, Fluent::Engine.now, data)
76
79
  end
80
+ ensure
77
81
  io.close
78
- sleep @interval
79
82
  end
83
+ rescue StandardError => e
84
+ log.error "watch_process: error has occured. #{e.message}"
85
+ end
86
+
87
+ def parse_line(line)
88
+ keys_size = @keys.size
89
+ if line =~ /(?<lstart>(^\w+\s+\w+\s+\d+\s+\d\d:\d\d:\d\d \d+))/
90
+ lstart = Time.parse($~[:lstart])
91
+ line = line.sub($~[:lstart], '')
92
+ keys_size -= 1
93
+ end
94
+ values = [lstart.to_s, line.chomp.strip.split(/\s+/, keys_size)]
95
+ data = Hash[@keys.zip(values.reject(&:empty?).flatten)]
96
+ data['elapsed_time'] = (Time.now - Time.parse(data['start_time'])).to_i if data['start_time']
97
+ data
98
+ end
99
+
100
+ def match_look_up_user?(data)
101
+ return true if @lookup_user.nil?
102
+
103
+ @lookup_user.include?(data['user'])
80
104
  end
81
105
 
82
106
  def get_ps_command
83
- if OS.linux?
84
- "LANG=en_US.UTF-8 && ps -ewwo lstart,user:20,pid,ppid,time,%cpu,%mem,rss,sz,s,comm,cmd"
85
- elsif OS.mac?
107
+ if mac?
86
108
  "LANG=en_US.UTF-8 && ps -ewwo lstart,user,pid,ppid,time,%cpu,%mem,rss,vsz,state,comm,command"
109
+ elsif Fluent.windows?
110
+ @windows_watcher.command
111
+ else
112
+ "LANG=en_US.UTF-8 && ps -ewwo lstart,user:20,pid,ppid,time,%cpu,%mem,rss,sz,s,comm,cmd"
87
113
  end
88
114
  end
89
115
 
90
- module OS
91
- # ref. http://stackoverflow.com/questions/170956/how-can-i-find-which-operating-system-my-ruby-program-is-running-on
92
- def OS.windows?
93
- (/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM) != nil
116
+ def mac?
117
+ (/darwin/ =~ RUBY_PLATFORM) != nil
118
+ end
119
+
120
+ class WindowsWatcher
121
+ # Keys are from the "System.Diagnostics.Process" object properties that can be taken by the "Get-Process" command.
122
+ # You can check the all properties by the "(Get-Process)[0] | Get-Member" command.
123
+ DEFAULT_KEYS = %w(StartTime UserName SessionId Id CPU WorkingSet VirtualMemorySize HandleCount ProcessName)
124
+
125
+ DEFAULT_TYPES = %w(
126
+ SessionId:integer
127
+ Id:integer
128
+ CPU:float
129
+ WorkingSet:integer
130
+ VirtualMemorySize:integer
131
+ HandleCount:integer
132
+ ).join(",")
133
+
134
+ attr_reader :keys
135
+ attr_reader :command
136
+
137
+ def initialize(keys, command, lookup_user, powershell_command)
138
+ @keys = keys || DEFAULT_KEYS
139
+ @powershell_command = powershell_command
140
+ @command = command || default_command
141
+ @lookup_user = lookup_user
94
142
  end
95
143
 
96
- def OS.mac?
97
- (/darwin/ =~ RUBY_PLATFORM) != nil
144
+ def default_types
145
+ DEFAULT_TYPES
98
146
  end
99
147
 
100
- def OS.unix?
101
- !OS.windows?
148
+ def parse_line(line)
149
+ values = line.chomp.strip.parse_csv.map { |e| e ? e : "" }
150
+ data = Hash[@keys.zip(values)]
151
+
152
+ unless data["StartTime"].nil?
153
+ start_time = Time.parse(data['StartTime'])
154
+ data['ElapsedTime'] = (Time.now - start_time).to_i
155
+ data["StartTime"] = start_time.to_s
156
+ end
157
+
158
+ data
159
+ end
160
+
161
+ def match_look_up_user?(data)
162
+ return true if @lookup_user.nil?
163
+
164
+ @lookup_user.include?(data["UserName"])
102
165
  end
103
166
 
104
- def OS.linux?
105
- OS.unix? and not OS.mac?
167
+ def default_command
168
+ command = [
169
+ command_ps,
170
+ pipe_filtering_normal_ps,
171
+ pipe_select_columns,
172
+ pipe_fixing_locale,
173
+ pipe_formatting_output,
174
+ ].join
175
+ "#{@powershell_command} -command \"#{command}\""
106
176
  end
107
- end
108
177
 
109
- def get_placeholder
110
- return {
111
- '__HOSTNAME__' => @hostname,
112
- '${hostname}' => @hostname,
113
- }
114
- end
178
+ def command_ps
179
+ if @keys.include?("UserName")
180
+ # The "IncludeUserName" option is needed to get the username, but this option requires Administrator privilege.
181
+ "Get-Process -IncludeUserName"
182
+ else
183
+ "Get-Process"
184
+ end
185
+ end
186
+
187
+ private
188
+
189
+ def pipe_filtering_normal_ps
190
+ # There are some special processes that don't have some properties, such as the "Idle" process.
191
+ # Normally, these are specific to the system and are not important, so exclude them.
192
+ # Note: The same situation can occur in some processes if there are no Administrator privilege.
193
+ " | ?{$_.StartTime -ne $NULL -and $_.CPU -ne $NULL}"
194
+ end
195
+
196
+ def pipe_select_columns
197
+ if @keys.nil? || @keys.empty?
198
+ raise "The 'keys' parameter is not specified correctly. [keys: #{@keys}]"
199
+ end
200
+
201
+ " | Select-Object -Property #{@keys.join(',')}"
202
+ end
203
+
204
+ def pipe_fixing_locale()
205
+ # In Windows, setting the "$env:Lang" environment variable is not effective in changing the format of the output.
206
+ # You can use "Datetime.ToString" method to change format of datetime values in for-each pipe.
207
+ # Note: About "DateTime.ToString" method: https://docs.microsoft.com/en-us/dotnet/api/system.datetime.tostring
208
+ return "" unless @keys.include?("StartTime")
209
+
210
+ " | %{$_.StartTime = $_.StartTime.ToString('o'); return $_;}"
211
+ end
212
+
213
+ def pipe_formatting_output
214
+ # In the "ConvertTo-Csv" command, there are 2 lines of type info and header info at the beginning in the outputs.
215
+ # By the "NoTypeInformation" option, the line of type info is excluded.
216
+ # This enables you to skip the just first line for parsing, like linux or mac.
217
+ " | ConvertTo-Csv -NoTypeInformation"
218
+ end
219
+ end
115
220
  end
116
221
  end
data/test/helper.rb CHANGED
@@ -12,15 +12,7 @@ require 'test/unit'
12
12
  $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
13
  $LOAD_PATH.unshift(File.dirname(__FILE__))
14
14
  require 'fluent/test'
15
- unless ENV.has_key?('VERBOSE')
16
- nulllogger = Object.new
17
- nulllogger.instance_eval {|obj|
18
- def method_missing(method, *args)
19
- # pass
20
- end
21
- }
22
- $log = nulllogger
23
- end
15
+ require 'fluent/test/driver/input'
24
16
 
25
17
  require 'fluent/plugin/in_watch_process'
26
18
 
@@ -7,11 +7,10 @@ class WatchProcessInputTest < Test::Unit::TestCase
7
7
 
8
8
  CONFIG = %[
9
9
  tag input.watch_process
10
- lookup_user apache, mycron
11
10
  ]
12
11
 
13
- def create_driver(conf=CONFIG,tag='test')
14
- Fluent::Test::OutputTestDriver.new(Fluent::WatchProcessInput, tag).configure(conf)
12
+ def create_driver(conf = CONFIG)
13
+ Fluent::Test::Driver::Input.new(Fluent::Plugin::WatchProcessInput).configure(conf)
15
14
  end
16
15
 
17
16
  def test_configure
@@ -22,9 +21,105 @@ class WatchProcessInputTest < Test::Unit::TestCase
22
21
  tag input.watch_process
23
22
  lookup_user apache, mycron
24
23
  ]
25
- d.instance.inspect
26
24
  assert_equal 'input.watch_process', d.instance.tag
27
25
  assert_equal ['apache', 'mycron'], d.instance.lookup_user
26
+ assert_equal :powershell, d.instance.powershell_command
28
27
  end
29
- end
30
28
 
29
+ def test_windows_configure
30
+ omit "Only for UNIX like." unless Fluent.windows?
31
+ d = create_driver %[
32
+ tag input.watch_process
33
+ lookup_user apache, mycron
34
+ powershell_command pwsh
35
+ ]
36
+ assert_equal 'input.watch_process', d.instance.tag
37
+ assert_equal ['apache', 'mycron'], d.instance.lookup_user
38
+ assert_equal :pwsh, d.instance.powershell_command
39
+ end
40
+
41
+ def test_unixlike
42
+ omit "Only for UNIX like." if Fluent.windows?
43
+ whoami = `whoami`
44
+ d = create_driver %[
45
+ tag input.watch_process
46
+ lookup_user #{whoami}
47
+ interval 1s
48
+ ]
49
+ d.run(expect_emits: 1, timeout: 3)
50
+ assert(d.events.size > 1)
51
+ end
52
+
53
+ sub_test_case "Windows" do
54
+ def test_windows_default
55
+ omit "Only for Windows." unless Fluent.windows?
56
+ d = create_driver %[
57
+ tag input.watch_process
58
+ interval 1s
59
+ ]
60
+ default_keys = Fluent::Plugin::WatchProcessInput::WindowsWatcher::DEFAULT_KEYS + ["ElapsedTime"]
61
+
62
+ d.run(expect_records: 1, timeout: 10);
63
+
64
+ assert d.events.size > 0
65
+
66
+ tag, time, record = d.events[0]
67
+
68
+ assert_equal "input.watch_process", tag
69
+ assert time.is_a?(Fluent::EventTime)
70
+ assert_equal default_keys, record.keys
71
+ end
72
+
73
+ def test_windows_customized
74
+ omit "Only for Windows." unless Fluent.windows?
75
+ custom_keys = ["PM", "Id", "UserProcessorTime", "Path"]
76
+ d = create_driver %[
77
+ tag input.watch_process
78
+ interval 1s
79
+ keys #{custom_keys.join(",")}
80
+ types Id:integer
81
+ ]
82
+
83
+ d.run(expect_records: 1, timeout: 10);
84
+
85
+ assert d.events.size > 0
86
+
87
+ tag, time, record = d.events[0]
88
+
89
+ assert_equal "input.watch_process", tag
90
+ assert time.is_a?(Fluent::EventTime)
91
+ assert_equal custom_keys, record.keys
92
+ assert record["PM"].is_a?(String)
93
+ assert record["Id"].is_a?(Integer)
94
+ end
95
+
96
+ def test_windows_lookup
97
+ omit "Only for Windows." unless Fluent.windows?
98
+ d = create_driver %[
99
+ tag input.watch_process
100
+ interval 1s
101
+ ]
102
+ d.run(expect_records: 1, timeout: 10);
103
+
104
+ assert d.events.size > 0
105
+
106
+ tag, time, record = d.events[0]
107
+ lookup_user = record["UserName"]
108
+
109
+ d = create_driver %[
110
+ tag input.watch_process
111
+ interval 1s
112
+ lookup_user #{lookup_user}
113
+ ]
114
+ d.run(expect_records: 1, timeout: 10);
115
+
116
+ assert d.events.size > 0
117
+
118
+ other_user_records = d.events.reject do |tag, time, record|
119
+ lookup_user.include?(record["UserName"])
120
+ end
121
+
122
+ assert other_user_records.size == 0
123
+ end
124
+ end
125
+ end
metadata CHANGED
@@ -1,48 +1,105 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-watch-process
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
5
- prerelease:
4
+ version: 0.2.0
6
5
  platform: ruby
7
6
  authors:
8
7
  - Kentaro Yoshida
9
8
  autorequire:
10
9
  bindir: bin
11
10
  cert_chain: []
12
- date: 2014-01-16 00:00:00.000000000 Z
11
+ date: 2021-05-30 00:00:00.000000000 Z
13
12
  dependencies:
14
13
  - !ruby/object:Gem::Dependency
15
14
  name: rake
16
15
  requirement: !ruby/object:Gem::Requirement
17
- none: false
18
16
  requirements:
19
- - - ! '>='
17
+ - - ">="
20
18
  - !ruby/object:Gem::Version
21
19
  version: '0'
22
20
  type: :development
23
21
  prerelease: false
24
22
  version_requirements: !ruby/object:Gem::Requirement
25
- none: false
26
23
  requirements:
27
- - - ! '>='
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: test-unit
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: 3.1.0
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: 3.1.0
41
+ - !ruby/object:Gem::Dependency
42
+ name: appraisal
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
28
53
  - !ruby/object:Gem::Version
29
54
  version: '0'
30
55
  - !ruby/object:Gem::Dependency
31
56
  name: fluentd
32
57
  requirement: !ruby/object:Gem::Requirement
33
- none: false
34
58
  requirements:
35
- - - ! '>='
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: 0.14.0
62
+ - - "<"
63
+ - !ruby/object:Gem::Version
64
+ version: '2'
65
+ type: :runtime
66
+ prerelease: false
67
+ version_requirements: !ruby/object:Gem::Requirement
68
+ requirements:
69
+ - - ">="
70
+ - !ruby/object:Gem::Version
71
+ version: 0.14.0
72
+ - - "<"
73
+ - !ruby/object:Gem::Version
74
+ version: '2'
75
+ - !ruby/object:Gem::Dependency
76
+ name: fluent-mixin-rewrite-tag-name
77
+ requirement: !ruby/object:Gem::Requirement
78
+ requirements:
79
+ - - ">="
36
80
  - !ruby/object:Gem::Version
37
81
  version: '0'
38
82
  type: :runtime
39
83
  prerelease: false
40
84
  version_requirements: !ruby/object:Gem::Requirement
41
- none: false
42
85
  requirements:
43
- - - ! '>='
86
+ - - ">="
44
87
  - !ruby/object:Gem::Version
45
88
  version: '0'
89
+ - !ruby/object:Gem::Dependency
90
+ name: fluent-mixin-type-converter
91
+ requirement: !ruby/object:Gem::Requirement
92
+ requirements:
93
+ - - ">="
94
+ - !ruby/object:Gem::Version
95
+ version: 0.1.0
96
+ type: :runtime
97
+ prerelease: false
98
+ version_requirements: !ruby/object:Gem::Requirement
99
+ requirements:
100
+ - - ">="
101
+ - !ruby/object:Gem::Version
102
+ version: 0.1.0
46
103
  description:
47
104
  email:
48
105
  - y.ken.studio@gmail.com
@@ -50,8 +107,9 @@ executables: []
50
107
  extensions: []
51
108
  extra_rdoc_files: []
52
109
  files:
53
- - .gitignore
54
- - .travis.yml
110
+ - ".gitignore"
111
+ - ".travis.yml"
112
+ - Appraisals
55
113
  - Gemfile
56
114
  - LICENSE.txt
57
115
  - README.md
@@ -59,33 +117,34 @@ files:
59
117
  - example1.conf
60
118
  - example2.conf
61
119
  - fluent-plugin-watch-process.gemspec
120
+ - gemfiles/fluentd_v0.14.gemfile
62
121
  - lib/fluent/plugin/in_watch_process.rb
63
122
  - test/helper.rb
64
123
  - test/plugin/test_in_watch_process.rb
65
124
  homepage: https://github.com/y-ken/fluent-plugin-watch-process
66
125
  licenses: []
126
+ metadata: {}
67
127
  post_install_message:
68
128
  rdoc_options: []
69
129
  require_paths:
70
130
  - lib
71
131
  required_ruby_version: !ruby/object:Gem::Requirement
72
- none: false
73
132
  requirements:
74
- - - ! '>='
133
+ - - ">="
75
134
  - !ruby/object:Gem::Version
76
135
  version: '0'
77
136
  required_rubygems_version: !ruby/object:Gem::Requirement
78
- none: false
79
137
  requirements:
80
- - - ! '>='
138
+ - - ">="
81
139
  - !ruby/object:Gem::Version
82
140
  version: '0'
83
141
  requirements: []
84
- rubyforge_project:
85
- rubygems_version: 1.8.23
142
+ rubygems_version: 3.1.6
86
143
  signing_key:
87
- specification_version: 3
88
- summary: Fluentd Input plugin to collect process information via ps command.
144
+ specification_version: 4
145
+ summary: Fluentd Input plugin to collect continual process information via ps command
146
+ or PowerShell pwsh command for Linux/osx/Windows. It is useful for cron/barch process
147
+ monitoring.
89
148
  test_files:
90
149
  - test/helper.rb
91
150
  - test/plugin/test_in_watch_process.rb