influxdb-process 0.1.0 → 0.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 1392cd9aa1f34ccf439ff3be4ff7e353b0a8a177
4
- data.tar.gz: 455bb5e9016eb0676137b946b7c1460a4a2a3a9a
2
+ SHA256:
3
+ metadata.gz: 76cf75aea6c095da14dea5172b6dc6215ecbae72f474e78c9c7ceefea6df7f94
4
+ data.tar.gz: 3328ebb06483a440173a27ae325006a25bfa83ff39e8b3d4fa41c9a39079dc72
5
5
  SHA512:
6
- metadata.gz: d00fe0f8b62f6f674f47d6ab2d4eea9dce08233a826480df24e796be9e1ffb889f2023ad8270db185d99febd26fd8ba745aa76d472ec199a4172be6fee0330a1
7
- data.tar.gz: 0d355f43aaf1344a266ed51c8f2da64eecb205d389588d8a0b9a1d96f40c1079f80f572301681ca152f0fca30b82211eaddf390198ca8f26d52afcef0349b243
6
+ metadata.gz: ef07b2213d9bf9c69e469c287d2bf0869b8254e697cf994265c67ce81e2e97d961b76d6c94a5a5476f6ad156421fcb8e4b78eed77a982c443ae73fbdde0d59e8
7
+ data.tar.gz: 04d7c58cd6e7b70ff61f1b209a16395d3ff84614ffa825e90d77444d4c24b201910e2f6009fdb24a8f7855bc3bbdd70433e93e6e28bea6fd69fcff90bc87c86e
@@ -0,0 +1 @@
1
+ 2.5.0
@@ -0,0 +1,7 @@
1
+ language: ruby
2
+
3
+ rvm:
4
+ - 2.2
5
+ - 2.3
6
+ - 2.4
7
+ - 2.5
data/Makefile CHANGED
@@ -1,2 +1,5 @@
1
+ test:
2
+ bundle exec rake
3
+
1
4
  points:
2
- bundle exec ruby test/process.rb
5
+ bundle exec ruby misc/process.rb
data/README.md CHANGED
@@ -1,3 +1,6 @@
1
+ [![Gem Version](https://badge.fury.io/rb/influxdb-process.svg)](https://badge.fury.io/rb/influxdb-process)
2
+ [![Build Status](https://travis-ci.org/vassilevsky/influxdb-process.svg?branch=master)](https://travis-ci.org/vassilevsky/influxdb-process)
3
+
1
4
  # InfluxDB::Process
2
5
 
3
6
  Gathers metrics from the Ruby process it is executed in and sends them to an InfluxDB database.
@@ -28,22 +31,37 @@ Let's say you have the client in the `influxdb` variable, as it says.
28
31
  Add this to any place in your Ruby program:
29
32
 
30
33
  ```ruby
31
- InfluxDB::Process::Instrumentation.new(influxdb)
34
+ InfluxDB::Process::Instrumentation.new(influxdb).start
32
35
  ```
33
36
 
34
37
  When you deploy this, the process where this code is executed will create a new thread.
35
38
  It will periodically collect process metrics and send them to InfluxDB via the provided client.
36
39
 
37
- By default, metrics will be written to the `process_metrics` series every *10* seconds.
38
- You can set your own series name and interval via additional keyword arguments:
40
+ Alternatively, you can instrument whenever you want (after each request / job / batch):
41
+
42
+ ```ruby
43
+ # in an initializer, once
44
+ p = InfluxDB::Process::Instrumentation.new(influxdb)
45
+
46
+ # later, in an appropriate place, as many times as needed
47
+ p.instrument
48
+ ```
49
+
50
+ You can pass additional options to customize the behavior of this gem. Here they are (with default values):
39
51
 
40
52
  ```ruby
41
- InfluxDB::Process::Instrumentation.new(influxdb, series: 'ruby_stats', interval: 42)
53
+ InfluxDB::Process::Instrumentation.new(
54
+ influxdb,
55
+ memory_series: 'process_memory',
56
+ object_series: 'process_objects',
57
+ interval: 10, # seconds
58
+ process: $PROGRAM_NAME
59
+ )
42
60
  ```
43
61
 
44
62
  Metrics will be tagged with process name.
45
63
  By default, `$0`/`$PROGRAM_NAME` will be used.
46
- You can set your own process name via an additional keyword argument:
64
+ You can set your own process name via an additional keyword argument in the constructor. For example:
47
65
 
48
66
  ```ruby
49
67
  InfluxDB::Process::Instrumentation.new(influxdb, process: 'report_generator')
@@ -55,11 +73,23 @@ You can also set the `INFLUXDB_PROCESS_NAME` environment variable:
55
73
 
56
74
  It will take precedence over the keyword argument.
57
75
 
76
+ ## Limitations
77
+
78
+ System memory metrics (total, resident, shared memory used by the process as seen from the OS) are read
79
+ for the `/proc` filesystem. They are not reported on systems where it is not available.
80
+
58
81
  ## Development
59
82
 
60
83
  After checking out the repo, run `bin/setup` to install dependencies.
61
84
  You can also run `bin/console` for an interactive prompt that will allow you to experiment.
62
85
 
86
+ How to test this gem:
87
+ * Install and run InfluxDB
88
+ * Create a `test` database in it
89
+ * Run `make`
90
+
91
+ It will generate metrics and push them to the `test` database.
92
+
63
93
  To install this gem onto your local machine, run `bundle exec rake install`.
64
94
  To release a new version:
65
95
  * Update the version number in `version.rb`
data/Rakefile CHANGED
@@ -1,2 +1,5 @@
1
1
  require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
2
5
  task :default => :spec
@@ -23,4 +23,5 @@ Gem::Specification.new do |spec|
23
23
 
24
24
  spec.add_development_dependency "bundler", "~> 1.16"
25
25
  spec.add_development_dependency "rake", "~> 10.0"
26
+ spec.add_development_dependency "rspec", "~> 3.0"
26
27
  end
@@ -1,18 +1,105 @@
1
+ require "objspace"
2
+
1
3
  require "influxdb"
2
4
  require "influxdb/process/version"
3
5
 
4
6
  module InfluxDB
5
7
  module Process
6
8
  class Instrumentation
7
- def initialize(influxdb, series: 'process_metrics', interval: 10, process: nil)
9
+ def initialize(influxdb, memory_series: nil, object_series: nil, interval: nil, process: nil)
10
+ @influxdb = influxdb
11
+ @memory_series = memory_series || 'process_memory'
12
+ @object_series = object_series || 'process_objects'
13
+ @interval = interval || 10
8
14
  @process = ENV['INFLUXDB_PROCESS_NAME'] || process || $PROGRAM_NAME
9
15
 
16
+ @tags = {process: @process}
17
+
18
+ @pid = ::Process.pid
19
+ @system_memory_file = "/proc/#{@pid}/statm"
20
+ @can_read_system_memory = File.exist?(@system_memory_file)
21
+ @page_size = `getconf PAGESIZE`.to_i rescue 4096
22
+
23
+ @memory = {}
24
+ @objects = {}
25
+ end
26
+
27
+ def instrument
28
+ update_memory
29
+ update_objects
30
+
31
+ @influxdb.write_point(@memory_series, tags: @tags, values: @memory)
32
+ @influxdb.write_point(@object_series, tags: @tags, values: @objects)
33
+ end
34
+
35
+ def start
10
36
  Thread.new do
11
37
  loop do
12
- influxdb.write_point(series, tags: {process: @process}, values: GC.stat)
13
- sleep(interval)
38
+ instrument
39
+ sleep(@interval)
40
+ end
41
+ end
42
+ end
43
+
44
+ private
45
+
46
+ TYPES = {
47
+ :T_OBJECT => :simple_objects,
48
+ :T_CLASS => :classes,
49
+ :T_MODULE => :modules,
50
+ :T_FLOAT => :floats,
51
+ :T_STRING => :strings,
52
+ :T_REGEXP => :regexes,
53
+ :T_ARRAY => :arrays,
54
+ :T_HASH => :hashes,
55
+ :T_STRUCT => :structs,
56
+ :T_BIGNUM => :bignums,
57
+ :T_FILE => :file_descriptors,
58
+ :T_DATA => :data_objects,
59
+ :T_MATCH => :matches,
60
+ :T_COMPLEX => :complex_numbers,
61
+ :T_RATIONAL => :rational_numbers,
62
+ :T_SYMBOL => :symbols,
63
+ :T_IMEMO => :memos,
64
+ :T_ICLASS => :included_modules,
65
+ :T_ZOMBIE => :zombies,
66
+ :T_NODE => :ast_nodes,
67
+ :TOTAL => :objects_total,
68
+ :FREE => :free_object_slots,
69
+ }.freeze
70
+
71
+ def update_memory
72
+ ObjectSpace.count_objects_size.each do |type, size|
73
+ @memory[TYPES.fetch(type)] = size
74
+ end
75
+
76
+ if defined?(ActiveRecord::Base)
77
+ @memory[:ar_models] = ObjectSpace.memsize_of_all(ActiveRecord::Base)
78
+ end
79
+
80
+ if @can_read_system_memory
81
+ size, resident, share, text, _lib, data, _dt = File.read(@system_memory_file).split(' ').map do |pages|
82
+ pages.to_i * @page_size
14
83
  end
84
+
85
+ @memory[:total] = size
86
+ @memory[:resident] = resident
87
+ @memory[:shared] = share
88
+ @memory[:program] = text
89
+ @memory[:data] = data
90
+ end
91
+ end
92
+
93
+ def update_objects
94
+ ObjectSpace.count_objects.each do |type, count|
95
+ @objects[TYPES.fetch(type)] = count
15
96
  end
97
+
98
+ gc = GC.stat
99
+
100
+ @objects[:heap_available_slots] = gc[:heap_available_slots]
101
+ @objects[:heap_live_slots] = gc[:heap_live_slots]
102
+ @objects[:heap_free_slots] = gc[:heap_free_slots]
16
103
  end
17
104
  end
18
105
  end
@@ -1,5 +1,5 @@
1
1
  module InfluxDB
2
2
  module Process
3
- VERSION = "0.1.0"
3
+ VERSION = "0.2.0"
4
4
  end
5
5
  end
@@ -0,0 +1,17 @@
1
+ require 'influxdb/process'
2
+
3
+ influxdb = InfluxDB::Client.new('test')
4
+ InfluxDB::Process::Instrumentation.new(influxdb, process: ENV['CUSTOM_PROCESS'])
5
+
6
+ OBJECTS = []
7
+
8
+ def create_many_objects
9
+ 1000.times do
10
+ OBJECTS << ' ' * 1000
11
+ end
12
+ end
13
+
14
+ 10.times do
15
+ create_many_objects
16
+ sleep 10
17
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: influxdb-process
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ilya Vassilevsky
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2017-12-08 00:00:00.000000000 Z
11
+ date: 2018-03-05 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: influxdb
@@ -52,6 +52,20 @@ dependencies:
52
52
  - - "~>"
53
53
  - !ruby/object:Gem::Version
54
54
  version: '10.0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '3.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '3.0'
55
69
  description:
56
70
  email:
57
71
  - vassilevsky@gmail.com
@@ -60,6 +74,8 @@ extensions: []
60
74
  extra_rdoc_files: []
61
75
  files:
62
76
  - ".gitignore"
77
+ - ".ruby-version"
78
+ - ".travis.yml"
63
79
  - Gemfile
64
80
  - LICENSE.txt
65
81
  - Makefile
@@ -68,6 +84,7 @@ files:
68
84
  - influxdb-process.gemspec
69
85
  - lib/influxdb/process.rb
70
86
  - lib/influxdb/process/version.rb
87
+ - misc/process.rb
71
88
  homepage: https://github.com/vassilevsky/influxdb-process
72
89
  licenses:
73
90
  - MIT
@@ -88,7 +105,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
88
105
  version: '0'
89
106
  requirements: []
90
107
  rubyforge_project:
91
- rubygems_version: 2.6.13
108
+ rubygems_version: 2.7.3
92
109
  signing_key:
93
110
  specification_version: 4
94
111
  summary: Ruby process instrumentation to an InfluxDB database