fluent-plugin-watch-process 0.0.1 → 0.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 2d114a31b165c3feca90dac92247785d9f757e8951ded1aac49bc30806e505a1
4
+ data.tar.gz: 9a2aab2afb95873873e3e3581b2f5f23b12bd0e7062d456b7eebe257c7f8473a
5
+ SHA512:
6
+ metadata.gz: ad4b0bffb25faaed2082c8bab5904100750ee265113097a0f643d5c13305c53731a9afcb6bd0b661fba1d012b45e74294c57defd530a774fbaf052135bcb6f43
7
+ data.tar.gz: 8f4761bfec54a36ecb2f702d7749ce4fd21bb69f35834aaeef274167456de74e0e8c4d7d09bdaaa90dc9bc3647d66eafbeb16d81ca24aede35fb4d273edd7e43
data/.travis.yml CHANGED
@@ -1,6 +1,13 @@
1
1
  language: ruby
2
2
 
3
3
  rvm:
4
- - 2.1.0
5
- - 2.0.0
6
- - 1.9.3
4
+ - 2.3.1
5
+ - 2.2
6
+ - 2.1
7
+
8
+ before_install:
9
+ - gem update bundler
10
+
11
+ gemfile:
12
+ - Gemfile
13
+ - gemfiles/fluentd_v0.14.gemfile
data/Appraisals ADDED
@@ -0,0 +1,7 @@
1
+ appraise "fluentd v0.12" do
2
+ gem "fluentd", "~> 0.12.0"
3
+ end
4
+
5
+ appraise "fluentd v0.14" do
6
+ gem "fluentd", "~> 0.14.0"
7
+ end
data/README.md CHANGED
@@ -3,17 +3,17 @@ fluent-plugin-watch-process [![Build Status](https://travis-ci.org/y-ken/fluent-
3
3
 
4
4
  ## Overview
5
5
 
6
- Fluentd Input plugin to collect process information via ps command.
6
+ Fluentd Input plugin to collect continual process information via ps command. It is useful for cron/barch process monitoring.
7
7
 
8
8
  ## Use Cases
9
9
 
10
- * collect cron/batch process for analysis.
10
+ * collect cron/batch process for long term analysis.
11
11
  * high cpu load time
12
12
  * high usage of memory
13
13
  * determine too long running task
14
14
 
15
15
  * output destination example
16
- * Elasticsearch + Kibana to visualize statistics. Example: [example1.conf](https://github.com/y-ken/fluent-plugin-watch-process/blob/master/example1.conf)
16
+ * Elasticsearch + Kibana to visualize cron/batch process statistics. Example: [example1.conf](https://github.com/y-ken/fluent-plugin-watch-process/blob/master/example1.conf)
17
17
  * save process information as audit log into AWS S3 which filename isolated by hostname. Example: [example2.conf](https://github.com/y-ken/fluent-plugin-watch-process/blob/master/example2.conf)
18
18
 
19
19
  ## Installation
@@ -25,7 +25,10 @@ install with gem or fluent-gem command as:
25
25
  $ gem install fluent-plugin-watch-process
26
26
 
27
27
  # for td-agent
28
- $ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-watch-process
28
+ $ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-watch-process -v 0.1.1
29
+
30
+ # for td-agent2 (recommend)
31
+ $ sudo td-agent-gem install fluent-plugin-watch-process -v 0.1.1
29
32
  ```
30
33
 
31
34
  ## Configuration
@@ -36,14 +39,14 @@ It is a quick sample to output log to `/var/log/td-agent/td-agent.log` with td-a
36
39
 
37
40
  `````
38
41
  <source>
39
- type watch_process
42
+ @type watch_process
40
43
  tag debug.batch.${hostname} # Required
41
44
  lookup_user batchuser # Optional
42
45
  interval 10s # Optional (default: 5s)
43
46
  </source>
44
47
 
45
48
  <match debug.**>
46
- type stdout
49
+ @type stdout
47
50
  </match>
48
51
  `````
49
52
 
@@ -51,41 +54,96 @@ After restarting td-agent, it will output process information to the td-agent.lo
51
54
 
52
55
  `````
53
56
  $ tail -f /var/log/td-agent/td-agent.log
57
+ ...snip...
54
58
  2014-01-16 14:21:34 +0900 debug.batch.localhost: {"start_time":"2014-01-16 14:21:13 +0900","user":"td-agent","pid":17486,"parent_pid":17483,"cpu_time":"00:00:00","cpu_percent":1.5,"memory_percent":3.5,"mem_rss":36068,"mem_size":60708,"state":"S","proc_name":"ruby","command":"/usr/lib64/fluent/ruby/bin/ruby /usr/sbin/td-agent --group td-agent --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid","elapsed_time":21}
55
59
  `````
56
60
 
57
61
  ### Syntax
58
62
 
59
- * tag # Required
63
+ * tag (Required)
60
64
  * record output destination
61
65
  * supported tag placeholders are `${hostname}` and `__HOSTNAME__`.
62
66
 
63
- * command # Optional
67
+ * command (Optional)
64
68
  * execute ps command with some options
65
69
  * [default] Linux: `LANG=en_US.UTF-8 && ps -ewwo lstart,user:20,pid,ppid,time,%cpu,%mem,rss,sz,s,comm,cmd`
66
70
  * [default] MacOSX: `LANG=en_US.UTF-8 && ps -ewwo lstart,user,pid,ppid,time,%cpu,%mem,rss,vsz,state,comm,command`
71
+ * [default] Windows: described in the next section, `About Windows`.
67
72
 
68
- * keys # Optional
73
+ * keys (Optional)
69
74
  * output record keys of the ps command results
70
- * [default] start_time user pid parent_pid cpu_time cpu_percent memory_percent mem_rss mem_size state proc_name command
75
+ * [default] Linux, MacOSX: `start_time,user,pid,parent_pid,cpu_time,cpu_percent,memory_percent,mem_rss,mem_size,state,proc_name,command`
76
+ * need to modify `command` too if you modify this value.
77
+ * [default] Windows: `StartTime,UserName,SessionId,Id,CPU,WorkingSet,VirtualMemorySize,HandleCount,ProcessName`
78
+ * in Windows only, you can fix this without fixing `command`. These keys can be specified from the properties of `System.Diagnostics.Process` object of `.NET`.
79
+ * `UserName` key needs administrator privilege. You can exclude this to avoid needing administrator privilege.
71
80
 
72
- * types # Optional
81
+ * types (Optional)
73
82
  * settings of converting types from string to integer/float.
74
- * [default] pid:integer parent_pid:integer cpu_percent:float memory_percent:float mem_rss:integer mem_size:integer
83
+ * [default] Linux, MacOSX: `pid:integer,parent_pid:integer,cpu_percent:float,memory_percent:float,mem_rss:integer,mem_size:integer`
84
+ * [default] Windows: `SessionId:integer,Id:integer,CPU:float,WorkingSet:integer,VirtualMemorySize:integer,HandleCount:integer`
75
85
 
76
- * interval # Optional
86
+ * interval (Optional)
77
87
  * execute interval time
78
88
  * [default] 5s
79
89
 
80
- * lookup_user # Optional
90
+ * lookup_user (Optional)
81
91
  * filter process owner username with comma delimited
82
92
  * [default] N/A
83
93
 
84
- * hostname_command # Optional
94
+ * hostname_command (Optional)
85
95
  * settings for tag placeholder, `${hostname}` and `__HOSTNAME__`. By default, it using long hostname.
86
96
  * to use short hostname, set `hostname -s` for this option on linux/mac.
87
97
  * [default] `hostname`
88
98
 
99
+ * powershell_command (Optional)
100
+ * settings for powershell command name. PowerShell Core had been renamed its command to `pwsh` and PowerShell 7 continues to use `pwsh` as its command name.
101
+ * [default] `powershell`
102
+ * [avaliables] `powershell`, `pwsh`
103
+
104
+ ### About Windows
105
+
106
+ Default `command` preset for Windows provides many of keys as below. Generally, you can pick up the columns with `keys` option.
107
+ If you need additional keys, consider to update `command` option.
108
+
109
+ `````powershell
110
+ powershell -command "Get-Process -IncludeUserName
111
+ | ?{$_.StartTime -ne $NULL -and $_.CPU -ne $NULL}
112
+ | Select-Object -Property StartTime,UserName,SessionId,Id,CPU,WorkingSet,VirtualMemorySize,HandleCount,ProcessName
113
+ | %{$_.StartTime = $_.StartTime.ToString('o'); return $_;}
114
+ | ConvertTo-Csv -NoTypeInformation"
115
+ `````
116
+
117
+ Confirmed versions are:
118
+
119
+ | Windows version | PowerShell version information | Note |
120
+ | ------------------------------ | ------------------------------------------------------------------------- |-----------------------------------------------|
121
+ | Windows 10 10.0.19042 (20H2) | PSVersion: 5.1.19041.906 (default installed version), PSEdition: Desktop | `powershell_command` as `powershell` (default)|
122
+ | Windows 10 10.0.19042 (20H2) | PSVersion: 7.1.2, PSEdition: Core | `powershell_command` as `pwsh` |
123
+
124
+
125
+ Here are details of this default command.
126
+
127
+ * `Get-Process -IncludeUserName`
128
+ * `Get-Process` powershell command takes `System.Diagnostics.Process` objects.
129
+ * `IncludeUserName` option is needed to take `UserName`.
130
+ * this needs administrator privilege.
131
+ * this will be omitted if `keys` does not contain `UserName`.
132
+ * ` | ?{$_.StartTime -ne $NULL -and $_.CPU -ne $NULL}`
133
+ * this exlcludes some special processes that don't have some properties, such as the "Idle" process in Windows.
134
+ * ` | Select-Object -Property ...`
135
+ * this takes the necessary parameters from `System.Diagnostics.Process` objects.
136
+ * `...` part will be automatically fixed by `keys`.
137
+ * ` | %{$_.StartTime = $_.StartTime.ToString('o'); return $_;}`
138
+ * this fixes the format of `StartTime` value.
139
+ * note: in Windows, setting the "$env:Lang" environment variable is not effective in changing the format of the output.
140
+ * ` | ConvertTo-Csv -NoTypeInformation`
141
+ * this formats objects to csv strings.
142
+ * currently, it is needed that `command` outputs the results in csv format.
143
+ * this is because white space delimiter is not suitable for Windows, in which empty values are often mixed.
144
+
145
+ **Note:** When using with PowerShell 7 which is previously known as PowerShell Core, you must specify `powershell_command` parameter as `pwsh`. Otherwise, this plugin does not work correctly on PowerShell 7 (pwsh). This is because PowerShell Core and PowerShell 7 use different command name which is `pwsh` not `powershell`.
146
+
89
147
  ## FAQ
90
148
 
91
149
  * I need hostname key in the record.
data/example1.conf CHANGED
@@ -6,10 +6,10 @@
6
6
  # * fluent-plugin-elasticsearch
7
7
 
8
8
  <source>
9
- type watch_process
9
+ @type watch_process
10
10
 
11
11
  # specify output tag
12
- tag batch_process
12
+ tag batch.process
13
13
 
14
14
  # filter specific user owned process. if no need to filter, delete this line.
15
15
  lookup_user batchuser
@@ -18,18 +18,16 @@
18
18
  interval 10s
19
19
  </source>
20
20
 
21
- <match batch_process>
22
- type record_reformer
23
- output_tag reformed.${tag}
24
- enable_ruby false
21
+ # add hostname key into record
22
+ <filter batch.process>
23
+ @type record_transformer
25
24
  <record>
26
- # add hostname key into record
27
- hostname ${hostname}
25
+ hostname ${hostname}
28
26
  </record>
29
- </match>
27
+ </filter>
30
28
 
31
- <match reformed.*>
32
- type elasticsearch
29
+ <match batch.*>
30
+ @type elasticsearch
33
31
  host localhost
34
32
  port 9200
35
33
  logstash_format true
data/example2.conf CHANGED
@@ -5,9 +5,10 @@
5
5
  # * fluent-plugin-s3
6
6
 
7
7
  <source>
8
- type watch_process
8
+ @type watch_process
9
9
 
10
10
  # use ${hostname} placeholder to use filename
11
+ # If you want to customize it, Use #{Socket.gethostname} or #{Socket.gethostname.split('.').first} snippet for tag.
11
12
  tag batch_process.${hostname}
12
13
 
13
14
  # filter specific user owned process. if no need to filter, delete this line.
@@ -18,7 +19,7 @@
18
19
  </source>
19
20
 
20
21
  <match batch_process.*>
21
- type s3
22
+ @type s3
22
23
 
23
24
  aws_key_id YOUR_AWS_KEY_ID
24
25
  aws_sec_key YOUR_AWS_SECRET/KEY
@@ -3,11 +3,11 @@ $:.push File.expand_path("../lib", __FILE__)
3
3
 
4
4
  Gem::Specification.new do |s|
5
5
  s.name = "fluent-plugin-watch-process"
6
- s.version = "0.0.1"
6
+ s.version = "0.2.0"
7
7
  s.authors = ["Kentaro Yoshida"]
8
8
  s.email = ["y.ken.studio@gmail.com"]
9
9
  s.homepage = "https://github.com/y-ken/fluent-plugin-watch-process"
10
- s.summary = %q{Fluentd Input plugin to collect process information via ps command.}
10
+ s.summary = %q{Fluentd Input plugin to collect continual process information via ps command or PowerShell pwsh command for Linux/osx/Windows. It is useful for cron/barch process monitoring.}
11
11
 
12
12
  s.files = `git ls-files`.split("\n")
13
13
  s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
@@ -16,5 +16,9 @@ Gem::Specification.new do |s|
16
16
 
17
17
  # specify any dependencies:
18
18
  s.add_development_dependency "rake"
19
- s.add_runtime_dependency "fluentd"
19
+ s.add_development_dependency "test-unit", ">= 3.1.0"
20
+ s.add_development_dependency "appraisal"
21
+ s.add_runtime_dependency "fluentd", [">= 0.14.0", "< 2"]
22
+ s.add_runtime_dependency "fluent-mixin-rewrite-tag-name"
23
+ s.add_runtime_dependency "fluent-mixin-type-converter", ">= 0.1.0"
20
24
  end
@@ -0,0 +1,7 @@
1
+ # This file was generated by Appraisal
2
+
3
+ source "http://rubygems.org"
4
+
5
+ gem "fluentd", "~> 0.14.0"
6
+
7
+ gemspec :path => "../"
@@ -1,116 +1,221 @@
1
- module Fluent
2
- class WatchProcessInput < Fluent::Input
3
- Plugin.register_input('watch_process', self)
1
+ require 'time'
2
+ require 'csv' if Fluent.windows?
3
+ require "fluent/plugin/input"
4
+ require 'fluent/mixin/rewrite_tag_name'
5
+ require 'fluent/mixin/type_converter'
6
+
7
+ module Fluent::Plugin
8
+ class WatchProcessInput < Fluent::Plugin::Input
9
+ Fluent::Plugin.register_input('watch_process', self)
10
+
11
+ helpers :timer
12
+
13
+ DEFAULT_KEYS = %w(start_time user pid parent_pid cpu_time cpu_percent memory_percent mem_rss mem_size state proc_name command)
14
+ DEFAULT_TYPES = %w(
15
+ pid:integer
16
+ parent_pid:integer
17
+ cpu_percent:float
18
+ memory_percent:float
19
+ mem_rss:integer
20
+ mem_size:integer
21
+ ).join(",")
4
22
 
5
23
  config_param :tag, :string
6
24
  config_param :command, :string, :default => nil
7
- config_param :keys, :string, :default => nil
8
- config_param :types, :string, :default => nil
9
- config_param :interval, :string, :default => '5s'
10
- config_param :lookup_user, :string, :default => nil
25
+ config_param :keys, :array, :default => nil
26
+ config_param :interval, :time, :default => '5s'
27
+ config_param :lookup_user, :array, :default => nil
11
28
  config_param :hostname_command, :string, :default => 'hostname'
29
+ config_param :powershell_command, :enum, list: [:powershell, :pwsh], :default => :powershell
12
30
 
13
- Converters = {
14
- 'string' => lambda { |v| v.to_s },
15
- 'integer' => lambda { |v| v.to_i },
16
- 'float' => lambda { |v| v.to_f },
17
- 'bool' => lambda { |v|
18
- case v.downcase
19
- when 'true', 'yes', '1'
20
- true
21
- else
22
- false
23
- end
24
- },
25
- 'time' => lambda { |v, time_parser|
26
- time_parser.parse(v)
27
- },
28
- 'array' => lambda { |v, delimiter|
29
- v.to_s.split(delimiter)
30
- }
31
- }
31
+ include Fluent::HandleTagNameMixin
32
+ include Fluent::Mixin::RewriteTagName
33
+ include Fluent::Mixin::TypeConverter
32
34
 
33
35
  def initialize
34
36
  super
35
- require 'time'
36
37
  end
37
38
 
38
39
  def configure(conf)
39
40
  super
40
41
 
41
- @command = @command || get_ps_command
42
- @keys = @keys || %w(start_time user pid parent_pid cpu_time cpu_percent memory_percent mem_rss mem_size state proc_name command)
43
- types = @types || %w(pid:integer parent_pid:integer cpu_percent:float memory_percent:float mem_rss:integer mem_size:integer)
44
- @types_map = Hash[types.map{|v| v.split(':')}]
45
- @lookup_user = @lookup_user.gsub(' ', '').split(',') unless @lookup_user.nil?
46
- @interval = Config.time_value(@interval)
47
- @hostname = `#{@hostname_command}`.chomp
48
- $log.info "watch_process: polling start. :tag=>#{@tag} :lookup_user=>#{@lookup_user} :interval=>#{@interval} :command=>#{@command}"
42
+ @windows_watcher = WindowsWatcher.new(@keys, @command, @lookup_user, @powershell_command) if Fluent.windows?
43
+ @keys ||= Fluent.windows? ? @windows_watcher.keys : DEFAULT_KEYS
44
+ @command ||= get_ps_command
45
+ apply_default_types
46
+ log.info "watch_process: polling start. :tag=>#{@tag} :lookup_user=>#{@lookup_user} :interval=>#{@interval} :command=>#{@command}"
49
47
  end
50
48
 
51
49
  def start
52
- @thread = Thread.new(&method(:run))
50
+ super
51
+ timer_execute(:in_watch_process, @interval, &method(:on_timer))
53
52
  end
54
53
 
55
54
  def shutdown
56
- Thread.kill(@thread)
55
+ super
56
+ end
57
+
58
+ def apply_default_types
59
+ return unless @types.nil?
60
+ @types = Fluent.windows? ? @windows_watcher.default_types : DEFAULT_TYPES
61
+ @type_converters = parse_types_parameter unless @types.nil?
57
62
  end
58
63
 
59
- def run
60
- loop do
61
- io = IO.popen(@command, 'r')
64
+ def on_timer
65
+ io = IO.popen(@command, 'r')
66
+ begin
62
67
  io.gets
63
68
  while result = io.gets
64
- values = result.chomp.strip.split(/\s+/, @keys.size + 4)
65
- time = Time.parse(values[0...5].join(' '))
66
- data = Hash[
67
- @keys.zip([time.to_s, values.values_at(5..15)].flatten).map do |k,v|
68
- v = Converters[@types_map[k]].call(v) if @types_map.include?(k)
69
- [k,v]
70
- end
71
- ]
72
- data['elapsed_time'] = (Time.now - Time.parse(data['start_time'])).to_i
73
- next unless @lookup_user.nil? || @lookup_user.include?(data['user'])
74
- tag = @tag.gsub(/(\${[a-z]+}|__[A-Z]+__)/, get_placeholder)
75
- Engine.emit(tag, Engine.now, data)
69
+ if Fluent.windows?
70
+ data = @windows_watcher.parse_line(result)
71
+ next unless @windows_watcher.match_look_up_user?(data)
72
+ else
73
+ data = parse_line(result)
74
+ next unless match_look_up_user?(data)
75
+ end
76
+ emit_tag = tag.dup
77
+ filter_record(emit_tag, Fluent::Engine.now, data)
78
+ router.emit(emit_tag, Fluent::Engine.now, data)
76
79
  end
80
+ ensure
77
81
  io.close
78
- sleep @interval
79
82
  end
83
+ rescue StandardError => e
84
+ log.error "watch_process: error has occured. #{e.message}"
85
+ end
86
+
87
+ def parse_line(line)
88
+ keys_size = @keys.size
89
+ if line =~ /(?<lstart>(^\w+\s+\w+\s+\d+\s+\d\d:\d\d:\d\d \d+))/
90
+ lstart = Time.parse($~[:lstart])
91
+ line = line.sub($~[:lstart], '')
92
+ keys_size -= 1
93
+ end
94
+ values = [lstart.to_s, line.chomp.strip.split(/\s+/, keys_size)]
95
+ data = Hash[@keys.zip(values.reject(&:empty?).flatten)]
96
+ data['elapsed_time'] = (Time.now - Time.parse(data['start_time'])).to_i if data['start_time']
97
+ data
98
+ end
99
+
100
+ def match_look_up_user?(data)
101
+ return true if @lookup_user.nil?
102
+
103
+ @lookup_user.include?(data['user'])
80
104
  end
81
105
 
82
106
  def get_ps_command
83
- if OS.linux?
84
- "LANG=en_US.UTF-8 && ps -ewwo lstart,user:20,pid,ppid,time,%cpu,%mem,rss,sz,s,comm,cmd"
85
- elsif OS.mac?
107
+ if mac?
86
108
  "LANG=en_US.UTF-8 && ps -ewwo lstart,user,pid,ppid,time,%cpu,%mem,rss,vsz,state,comm,command"
109
+ elsif Fluent.windows?
110
+ @windows_watcher.command
111
+ else
112
+ "LANG=en_US.UTF-8 && ps -ewwo lstart,user:20,pid,ppid,time,%cpu,%mem,rss,sz,s,comm,cmd"
87
113
  end
88
114
  end
89
115
 
90
- module OS
91
- # ref. http://stackoverflow.com/questions/170956/how-can-i-find-which-operating-system-my-ruby-program-is-running-on
92
- def OS.windows?
93
- (/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM) != nil
116
+ def mac?
117
+ (/darwin/ =~ RUBY_PLATFORM) != nil
118
+ end
119
+
120
+ class WindowsWatcher
121
+ # Keys are from the "System.Diagnostics.Process" object properties that can be taken by the "Get-Process" command.
122
+ # You can check the all properties by the "(Get-Process)[0] | Get-Member" command.
123
+ DEFAULT_KEYS = %w(StartTime UserName SessionId Id CPU WorkingSet VirtualMemorySize HandleCount ProcessName)
124
+
125
+ DEFAULT_TYPES = %w(
126
+ SessionId:integer
127
+ Id:integer
128
+ CPU:float
129
+ WorkingSet:integer
130
+ VirtualMemorySize:integer
131
+ HandleCount:integer
132
+ ).join(",")
133
+
134
+ attr_reader :keys
135
+ attr_reader :command
136
+
137
+ def initialize(keys, command, lookup_user, powershell_command)
138
+ @keys = keys || DEFAULT_KEYS
139
+ @powershell_command = powershell_command
140
+ @command = command || default_command
141
+ @lookup_user = lookup_user
94
142
  end
95
143
 
96
- def OS.mac?
97
- (/darwin/ =~ RUBY_PLATFORM) != nil
144
+ def default_types
145
+ DEFAULT_TYPES
98
146
  end
99
147
 
100
- def OS.unix?
101
- !OS.windows?
148
+ def parse_line(line)
149
+ values = line.chomp.strip.parse_csv.map { |e| e ? e : "" }
150
+ data = Hash[@keys.zip(values)]
151
+
152
+ unless data["StartTime"].nil?
153
+ start_time = Time.parse(data['StartTime'])
154
+ data['ElapsedTime'] = (Time.now - start_time).to_i
155
+ data["StartTime"] = start_time.to_s
156
+ end
157
+
158
+ data
159
+ end
160
+
161
+ def match_look_up_user?(data)
162
+ return true if @lookup_user.nil?
163
+
164
+ @lookup_user.include?(data["UserName"])
102
165
  end
103
166
 
104
- def OS.linux?
105
- OS.unix? and not OS.mac?
167
+ def default_command
168
+ command = [
169
+ command_ps,
170
+ pipe_filtering_normal_ps,
171
+ pipe_select_columns,
172
+ pipe_fixing_locale,
173
+ pipe_formatting_output,
174
+ ].join
175
+ "#{@powershell_command} -command \"#{command}\""
106
176
  end
107
- end
108
177
 
109
- def get_placeholder
110
- return {
111
- '__HOSTNAME__' => @hostname,
112
- '${hostname}' => @hostname,
113
- }
114
- end
178
+ def command_ps
179
+ if @keys.include?("UserName")
180
+ # The "IncludeUserName" option is needed to get the username, but this option requires Administrator privilege.
181
+ "Get-Process -IncludeUserName"
182
+ else
183
+ "Get-Process"
184
+ end
185
+ end
186
+
187
+ private
188
+
189
+ def pipe_filtering_normal_ps
190
+ # There are some special processes that don't have some properties, such as the "Idle" process.
191
+ # Normally, these are specific to the system and are not important, so exclude them.
192
+ # Note: The same situation can occur in some processes if there are no Administrator privilege.
193
+ " | ?{$_.StartTime -ne $NULL -and $_.CPU -ne $NULL}"
194
+ end
195
+
196
+ def pipe_select_columns
197
+ if @keys.nil? || @keys.empty?
198
+ raise "The 'keys' parameter is not specified correctly. [keys: #{@keys}]"
199
+ end
200
+
201
+ " | Select-Object -Property #{@keys.join(',')}"
202
+ end
203
+
204
+ def pipe_fixing_locale()
205
+ # In Windows, setting the "$env:Lang" environment variable is not effective in changing the format of the output.
206
+ # You can use "Datetime.ToString" method to change format of datetime values in for-each pipe.
207
+ # Note: About "DateTime.ToString" method: https://docs.microsoft.com/en-us/dotnet/api/system.datetime.tostring
208
+ return "" unless @keys.include?("StartTime")
209
+
210
+ " | %{$_.StartTime = $_.StartTime.ToString('o'); return $_;}"
211
+ end
212
+
213
+ def pipe_formatting_output
214
+ # In the "ConvertTo-Csv" command, there are 2 lines of type info and header info at the beginning in the outputs.
215
+ # By the "NoTypeInformation" option, the line of type info is excluded.
216
+ # This enables you to skip the just first line for parsing, like linux or mac.
217
+ " | ConvertTo-Csv -NoTypeInformation"
218
+ end
219
+ end
115
220
  end
116
221
  end
data/test/helper.rb CHANGED
@@ -12,15 +12,7 @@ require 'test/unit'
12
12
  $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
13
13
  $LOAD_PATH.unshift(File.dirname(__FILE__))
14
14
  require 'fluent/test'
15
- unless ENV.has_key?('VERBOSE')
16
- nulllogger = Object.new
17
- nulllogger.instance_eval {|obj|
18
- def method_missing(method, *args)
19
- # pass
20
- end
21
- }
22
- $log = nulllogger
23
- end
15
+ require 'fluent/test/driver/input'
24
16
 
25
17
  require 'fluent/plugin/in_watch_process'
26
18
 
@@ -7,11 +7,10 @@ class WatchProcessInputTest < Test::Unit::TestCase
7
7
 
8
8
  CONFIG = %[
9
9
  tag input.watch_process
10
- lookup_user apache, mycron
11
10
  ]
12
11
 
13
- def create_driver(conf=CONFIG,tag='test')
14
- Fluent::Test::OutputTestDriver.new(Fluent::WatchProcessInput, tag).configure(conf)
12
+ def create_driver(conf = CONFIG)
13
+ Fluent::Test::Driver::Input.new(Fluent::Plugin::WatchProcessInput).configure(conf)
15
14
  end
16
15
 
17
16
  def test_configure
@@ -22,9 +21,105 @@ class WatchProcessInputTest < Test::Unit::TestCase
22
21
  tag input.watch_process
23
22
  lookup_user apache, mycron
24
23
  ]
25
- d.instance.inspect
26
24
  assert_equal 'input.watch_process', d.instance.tag
27
25
  assert_equal ['apache', 'mycron'], d.instance.lookup_user
26
+ assert_equal :powershell, d.instance.powershell_command
28
27
  end
29
- end
30
28
 
29
+ def test_windows_configure
30
+ omit "Only for UNIX like." unless Fluent.windows?
31
+ d = create_driver %[
32
+ tag input.watch_process
33
+ lookup_user apache, mycron
34
+ powershell_command pwsh
35
+ ]
36
+ assert_equal 'input.watch_process', d.instance.tag
37
+ assert_equal ['apache', 'mycron'], d.instance.lookup_user
38
+ assert_equal :pwsh, d.instance.powershell_command
39
+ end
40
+
41
+ def test_unixlike
42
+ omit "Only for UNIX like." if Fluent.windows?
43
+ whoami = `whoami`
44
+ d = create_driver %[
45
+ tag input.watch_process
46
+ lookup_user #{whoami}
47
+ interval 1s
48
+ ]
49
+ d.run(expect_emits: 1, timeout: 3)
50
+ assert(d.events.size > 1)
51
+ end
52
+
53
+ sub_test_case "Windows" do
54
+ def test_windows_default
55
+ omit "Only for Windows." unless Fluent.windows?
56
+ d = create_driver %[
57
+ tag input.watch_process
58
+ interval 1s
59
+ ]
60
+ default_keys = Fluent::Plugin::WatchProcessInput::WindowsWatcher::DEFAULT_KEYS + ["ElapsedTime"]
61
+
62
+ d.run(expect_records: 1, timeout: 10);
63
+
64
+ assert d.events.size > 0
65
+
66
+ tag, time, record = d.events[0]
67
+
68
+ assert_equal "input.watch_process", tag
69
+ assert time.is_a?(Fluent::EventTime)
70
+ assert_equal default_keys, record.keys
71
+ end
72
+
73
+ def test_windows_customized
74
+ omit "Only for Windows." unless Fluent.windows?
75
+ custom_keys = ["PM", "Id", "UserProcessorTime", "Path"]
76
+ d = create_driver %[
77
+ tag input.watch_process
78
+ interval 1s
79
+ keys #{custom_keys.join(",")}
80
+ types Id:integer
81
+ ]
82
+
83
+ d.run(expect_records: 1, timeout: 10);
84
+
85
+ assert d.events.size > 0
86
+
87
+ tag, time, record = d.events[0]
88
+
89
+ assert_equal "input.watch_process", tag
90
+ assert time.is_a?(Fluent::EventTime)
91
+ assert_equal custom_keys, record.keys
92
+ assert record["PM"].is_a?(String)
93
+ assert record["Id"].is_a?(Integer)
94
+ end
95
+
96
+ def test_windows_lookup
97
+ omit "Only for Windows." unless Fluent.windows?
98
+ d = create_driver %[
99
+ tag input.watch_process
100
+ interval 1s
101
+ ]
102
+ d.run(expect_records: 1, timeout: 10);
103
+
104
+ assert d.events.size > 0
105
+
106
+ tag, time, record = d.events[0]
107
+ lookup_user = record["UserName"]
108
+
109
+ d = create_driver %[
110
+ tag input.watch_process
111
+ interval 1s
112
+ lookup_user #{lookup_user}
113
+ ]
114
+ d.run(expect_records: 1, timeout: 10);
115
+
116
+ assert d.events.size > 0
117
+
118
+ other_user_records = d.events.reject do |tag, time, record|
119
+ lookup_user.include?(record["UserName"])
120
+ end
121
+
122
+ assert other_user_records.size == 0
123
+ end
124
+ end
125
+ end
metadata CHANGED
@@ -1,48 +1,105 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-watch-process
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
5
- prerelease:
4
+ version: 0.2.0
6
5
  platform: ruby
7
6
  authors:
8
7
  - Kentaro Yoshida
9
8
  autorequire:
10
9
  bindir: bin
11
10
  cert_chain: []
12
- date: 2014-01-16 00:00:00.000000000 Z
11
+ date: 2021-05-30 00:00:00.000000000 Z
13
12
  dependencies:
14
13
  - !ruby/object:Gem::Dependency
15
14
  name: rake
16
15
  requirement: !ruby/object:Gem::Requirement
17
- none: false
18
16
  requirements:
19
- - - ! '>='
17
+ - - ">="
20
18
  - !ruby/object:Gem::Version
21
19
  version: '0'
22
20
  type: :development
23
21
  prerelease: false
24
22
  version_requirements: !ruby/object:Gem::Requirement
25
- none: false
26
23
  requirements:
27
- - - ! '>='
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: test-unit
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: 3.1.0
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: 3.1.0
41
+ - !ruby/object:Gem::Dependency
42
+ name: appraisal
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
28
53
  - !ruby/object:Gem::Version
29
54
  version: '0'
30
55
  - !ruby/object:Gem::Dependency
31
56
  name: fluentd
32
57
  requirement: !ruby/object:Gem::Requirement
33
- none: false
34
58
  requirements:
35
- - - ! '>='
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: 0.14.0
62
+ - - "<"
63
+ - !ruby/object:Gem::Version
64
+ version: '2'
65
+ type: :runtime
66
+ prerelease: false
67
+ version_requirements: !ruby/object:Gem::Requirement
68
+ requirements:
69
+ - - ">="
70
+ - !ruby/object:Gem::Version
71
+ version: 0.14.0
72
+ - - "<"
73
+ - !ruby/object:Gem::Version
74
+ version: '2'
75
+ - !ruby/object:Gem::Dependency
76
+ name: fluent-mixin-rewrite-tag-name
77
+ requirement: !ruby/object:Gem::Requirement
78
+ requirements:
79
+ - - ">="
36
80
  - !ruby/object:Gem::Version
37
81
  version: '0'
38
82
  type: :runtime
39
83
  prerelease: false
40
84
  version_requirements: !ruby/object:Gem::Requirement
41
- none: false
42
85
  requirements:
43
- - - ! '>='
86
+ - - ">="
44
87
  - !ruby/object:Gem::Version
45
88
  version: '0'
89
+ - !ruby/object:Gem::Dependency
90
+ name: fluent-mixin-type-converter
91
+ requirement: !ruby/object:Gem::Requirement
92
+ requirements:
93
+ - - ">="
94
+ - !ruby/object:Gem::Version
95
+ version: 0.1.0
96
+ type: :runtime
97
+ prerelease: false
98
+ version_requirements: !ruby/object:Gem::Requirement
99
+ requirements:
100
+ - - ">="
101
+ - !ruby/object:Gem::Version
102
+ version: 0.1.0
46
103
  description:
47
104
  email:
48
105
  - y.ken.studio@gmail.com
@@ -50,8 +107,9 @@ executables: []
50
107
  extensions: []
51
108
  extra_rdoc_files: []
52
109
  files:
53
- - .gitignore
54
- - .travis.yml
110
+ - ".gitignore"
111
+ - ".travis.yml"
112
+ - Appraisals
55
113
  - Gemfile
56
114
  - LICENSE.txt
57
115
  - README.md
@@ -59,33 +117,34 @@ files:
59
117
  - example1.conf
60
118
  - example2.conf
61
119
  - fluent-plugin-watch-process.gemspec
120
+ - gemfiles/fluentd_v0.14.gemfile
62
121
  - lib/fluent/plugin/in_watch_process.rb
63
122
  - test/helper.rb
64
123
  - test/plugin/test_in_watch_process.rb
65
124
  homepage: https://github.com/y-ken/fluent-plugin-watch-process
66
125
  licenses: []
126
+ metadata: {}
67
127
  post_install_message:
68
128
  rdoc_options: []
69
129
  require_paths:
70
130
  - lib
71
131
  required_ruby_version: !ruby/object:Gem::Requirement
72
- none: false
73
132
  requirements:
74
- - - ! '>='
133
+ - - ">="
75
134
  - !ruby/object:Gem::Version
76
135
  version: '0'
77
136
  required_rubygems_version: !ruby/object:Gem::Requirement
78
- none: false
79
137
  requirements:
80
- - - ! '>='
138
+ - - ">="
81
139
  - !ruby/object:Gem::Version
82
140
  version: '0'
83
141
  requirements: []
84
- rubyforge_project:
85
- rubygems_version: 1.8.23
142
+ rubygems_version: 3.1.6
86
143
  signing_key:
87
- specification_version: 3
88
- summary: Fluentd Input plugin to collect process information via ps command.
144
+ specification_version: 4
145
+ summary: Fluentd Input plugin to collect continual process information via ps command
146
+ or PowerShell pwsh command for Linux/osx/Windows. It is useful for cron/barch process
147
+ monitoring.
89
148
  test_files:
90
149
  - test/helper.rb
91
150
  - test/plugin/test_in_watch_process.rb