mysqlcollector 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +91 -0
- data/bin/mysqlcollector +7 -0
- data/lib/mysqlcollector.rb +24 -0
- data/lib/mysqlcollector/cli.rb +89 -0
- data/lib/mysqlcollector/collector.rb +225 -0
- data/lib/mysqlcollector/daemon.rb +26 -0
- data/lib/mysqlcollector/influxdb.rb +58 -0
- data/lib/mysqlcollector/log.rb +38 -0
- data/lib/mysqlcollector/mysql.rb +32 -0
- data/lib/mysqlcollector/template.rb +14 -0
- data/lib/mysqlcollector/version.rb +5 -0
- metadata +155 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 8e5f52afef7c7eb799b226dccdfbe1439edbe3df
|
4
|
+
data.tar.gz: bfdf804b019f4113a4bfeff54072affda55d228a
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: c62b5cedaa75bebb10b5e96ec104ca89a8f47a4959b7b3baec0a693a94cabc7379ce0407a9f05970c4b6dbc6d89d2f4634da317168c375281c446d165b2378fa
|
7
|
+
data.tar.gz: a75e825c55cb7e8347c8aee22743b6c22250912bb8d6d58728063ac7b265f1118ae77076c1a127f12a7f2d187fde7c215b08726c92f90c0dd816be426fc7d60e
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2016 Nicola Strappazzon
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
# MySQLCollector [](https://travis-ci.org/swapbytes/mysqlcollector) [](https://badge.fury.io/rb/mysqlcollector) [](https://coveralls.io/github/swapbytes/mysqlcollector?branch=master)
|
2
|
+
|
3
|
+
Ruby GEM to collect MySQL Metrics and send them to InfluxDB. It's a fast and easy
|
4
|
+
implementation.
|
5
|
+
|
6
|
+
## Requirements
|
7
|
+
|
8
|
+
### Client
|
9
|
+
|
10
|
+
Where it's run MySQLCollector.
|
11
|
+
|
12
|
+
- Ruby 1.9.3
|
13
|
+
- MySQL 5.x
|
14
|
+
|
15
|
+
### Server
|
16
|
+
|
17
|
+
- InfluxDB
|
18
|
+
- Grafana
|
19
|
+
|
20
|
+
## Install
|
21
|
+
|
22
|
+
**MySQLCollector** installation is pretty standard:
|
23
|
+
|
24
|
+
|
25
|
+
```bash
|
26
|
+
$ [sudo] gem install mysqlcollector
|
27
|
+
```
|
28
|
+
|
29
|
+
Create a user and database on MySQL:
|
30
|
+
|
31
|
+
```sql
|
32
|
+
mysql> GRANT REPLICATION CLIENT, PROCESS ON *.*
|
33
|
+
TO 'mysqlcollector'@'%' IDENTIFIED BY 'mysqlcollector';
|
34
|
+
mysql> GRANT SELECT ON performance_schema.* TO 'mysqlcollector'@'%';
|
35
|
+
```
|
36
|
+
|
37
|
+
Create a user and database on InfluxDB:
|
38
|
+
|
39
|
+
```sql
|
40
|
+
> CREATE USER mysqlcollector WITH PASSWORD 'mysqlcollector';
|
41
|
+
> CREATE DATABASE mysqlcollector;
|
42
|
+
```
|
43
|
+
|
44
|
+
To generate a JSON template for Grafana:
|
45
|
+
|
46
|
+
```bash
|
47
|
+
$ mysqlcollector --template > grafana_mysql.json
|
48
|
+
```
|
49
|
+
|
50
|
+
For more help as import template, please see the [doc of grafana](http://docs.grafana.org/reference/export_import/).
|
51
|
+
|
52
|
+
## Basic Usage
|
53
|
+
|
54
|
+
For this case, InfluxDB and Grafana running in your local docker, try:
|
55
|
+
|
56
|
+
```bash
|
57
|
+
$ mysqlcollector --mysql-host 192.168.99.100 \
|
58
|
+
--influx-host 192.168.99.100
|
59
|
+
```
|
60
|
+
|
61
|
+
#### Daemonize
|
62
|
+
|
63
|
+
You can daemonize this tool, and send new metric every 30 seconds:
|
64
|
+
|
65
|
+
```bash
|
66
|
+
$ mysqlcollector --mysql-host 192.168.99.100 \
|
67
|
+
--influx-host 192.168.99.100 \
|
68
|
+
--daemonize
|
69
|
+
```
|
70
|
+
|
71
|
+
#### Debuging
|
72
|
+
|
73
|
+
```bash
|
74
|
+
$ mysqlcollector --mysql-host 192.168.99.100 \
|
75
|
+
--influx-host 192.168.99.100 \
|
76
|
+
--debug
|
77
|
+
```
|
78
|
+
|
79
|
+
## Warning
|
80
|
+
|
81
|
+
1. Do not use this tool in production before testing it.
|
82
|
+
2. Please, use when do you need.
|
83
|
+
3. The author is NOT responsible for misuse use of this tool.
|
84
|
+
|
85
|
+
## Contributing
|
86
|
+
|
87
|
+
1. Fork it
|
88
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
89
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
90
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
91
|
+
5. Create new Pull Request
|
data/bin/mysqlcollector
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
require 'rubygems'
|
4
|
+
require 'logger'
|
5
|
+
require 'influxdb'
|
6
|
+
require 'mysql2'
|
7
|
+
require 'optparse'
|
8
|
+
|
9
|
+
class Object
|
10
|
+
def is_number?
|
11
|
+
true if Float(self) rescue false
|
12
|
+
end
|
13
|
+
end
|
14
|
+
|
15
|
+
module Mysqlcollector
|
16
|
+
LIBRARY_PATH = File.join(File.dirname(__FILE__), 'mysqlcollector')
|
17
|
+
|
18
|
+
# Autoload libraries:
|
19
|
+
Dir["#{LIBRARY_PATH}/**/*.rb"].select do |file|
|
20
|
+
if File.file? file
|
21
|
+
require file
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
@@ -0,0 +1,89 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Cli
|
5
|
+
attr_reader :config
|
6
|
+
|
7
|
+
$config = { alias: 'localhost',
|
8
|
+
mysql_host: '192.168.99.100',
|
9
|
+
mysql_port: 3306,
|
10
|
+
mysql_username: 'mysqlcollector',
|
11
|
+
mysql_password: 'mysqlcollector',
|
12
|
+
influx_host: '192.168.99.100',
|
13
|
+
influx_user: 'mysqlcollector',
|
14
|
+
influx_password: 'mysqlcollector',
|
15
|
+
influx_database: 'mysqlcollector' }
|
16
|
+
|
17
|
+
def initialize
|
18
|
+
options
|
19
|
+
end
|
20
|
+
|
21
|
+
def options
|
22
|
+
ARGV.options do |opt|
|
23
|
+
begin
|
24
|
+
opt.on('--alias ALIAS', String, 'Alias for instance') do |v|
|
25
|
+
$config[:influx_database] = v
|
26
|
+
end
|
27
|
+
opt.on('--mysql-host HOST', String, 'MySQL Host') do |v|
|
28
|
+
$config[:mysql_host] = v
|
29
|
+
end
|
30
|
+
opt.on('--mysql-port PORT', Integer, 'MySQL Port') do |v|
|
31
|
+
$config[:mysql_host] = v
|
32
|
+
end
|
33
|
+
opt.on('--mysql-username USER', String, 'MySQL User name') do |v|
|
34
|
+
$config[:mysql_username] = v
|
35
|
+
end
|
36
|
+
opt.on('--mysql-password PASSWORD', String, 'MySQL User password') do |v|
|
37
|
+
$config[:mysql_password] = v
|
38
|
+
end
|
39
|
+
opt.on('--influx-host HOST', String, 'InfluxDB Host') do |v|
|
40
|
+
$config[:influx_host] = v
|
41
|
+
end
|
42
|
+
opt.on('--influx-username USER', String, 'InfluxDB User') do |v|
|
43
|
+
$config[:influx_user] = v
|
44
|
+
end
|
45
|
+
opt.on('--influx-password PASSWORD', String, 'InfluxDB User password') do |v|
|
46
|
+
$config[:influx_password] = v
|
47
|
+
end
|
48
|
+
opt.on('--influx-database DATABASE', String, 'InfluxDB Database') do |v|
|
49
|
+
$config[:influx_database] = v
|
50
|
+
end
|
51
|
+
opt.on('--daemonize', 'Run this every 30 seconds') do |v|
|
52
|
+
$config[:daemonize] = v
|
53
|
+
end
|
54
|
+
opt.on('--template', 'Export JSON template for Grafana') do |v|
|
55
|
+
$config[:template] = v
|
56
|
+
end
|
57
|
+
opt.on('--debug', 'Debug this tool') do |v|
|
58
|
+
$config[:debug] = v
|
59
|
+
end
|
60
|
+
|
61
|
+
opt.on('-v', '--version', 'Show version') do
|
62
|
+
puts "MySQL Collector V-#{Mysqlcollector::VERSION}"
|
63
|
+
end
|
64
|
+
opt.on('-?', '--help', 'Show this help') { puts opt.help; }
|
65
|
+
opt.parse!
|
66
|
+
|
67
|
+
if $config.empty?
|
68
|
+
puts opt.help
|
69
|
+
exit 1
|
70
|
+
end
|
71
|
+
rescue => e
|
72
|
+
puts e
|
73
|
+
exit 1
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
77
|
+
|
78
|
+
def start
|
79
|
+
if $config[:daemonize]
|
80
|
+
Deamon.new('Collector.new.start').run!
|
81
|
+
elsif $config[:template]
|
82
|
+
Template.new.grafana
|
83
|
+
else
|
84
|
+
collector = Collector.new
|
85
|
+
collector.start
|
86
|
+
end
|
87
|
+
end
|
88
|
+
end
|
89
|
+
end
|
@@ -0,0 +1,225 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Collector
|
5
|
+
SQL_INNODB = 'SHOW /*!50000 ENGINE*/ INNODB STATUS'
|
6
|
+
SQL_MASTER = 'SHOW MASTER LOGS'
|
7
|
+
SQL_PROCESS = 'SHOW PROCESSLIST'
|
8
|
+
SQL_SLAVE = 'SHOW SLAVE STATUS'
|
9
|
+
SQL_STATUS = 'SHOW /*!50002 GLOBAL */ STATUS'
|
10
|
+
SQL_VARIABLES = 'SHOW GLOBAL VARIABLES'
|
11
|
+
SQL_LOCKS = 'SHOW OPEN TABLES WHERE in_use = 1'
|
12
|
+
|
13
|
+
def initialize
|
14
|
+
@mysql = Mysql.new
|
15
|
+
@influx = Influxdb.new
|
16
|
+
end
|
17
|
+
|
18
|
+
def innodb_parse(text)
|
19
|
+
parsing = [# SEMAPHORES:
|
20
|
+
'Mutex spin waits (?<spin_waits>\d+), rounds (?<spin_rounds>\d+), OS waits (?<os_waits>\d+)',
|
21
|
+
'RW-shared spins (?<spin_waits>\d+), rounds (\d+), OS waits (?<os_waits>\d+)',
|
22
|
+
'RW-excl spins (?<spin_waits>\d+), rounds (\d+), OS waits (?<os_waits>\d+)',
|
23
|
+
# File I/O:
|
24
|
+
'(?<file_reads>\d+) OS file reads, (?<file_writes>\d+) OS file writes, (?<file_fsyncs>\d+) OS fsyncs',
|
25
|
+
'Pending normal aio reads: (?<pending_normal_aio_reads>\d+)(?: \[(?:\d+, )*\d+\] )?, aio writes: (?<pending_normal_aio_writes>\d+)(?: \[(?:\d+, )*\d+\] )?',
|
26
|
+
'ibuf aio reads: (?<pending_ibuf_aio_reads>\d+), log i\/o\'s: (?<pending_aio_log_ios>\d+), sync i\/o\'s: (?<pending_aio_sync_ios>\d+)',
|
27
|
+
'Pending flushes \(fsync\) log: (?<pending_log_flushes>\d+); buffer pool: (?<pending_buf_pool_flushes>\d+)',
|
28
|
+
# INSERT BUFFER AND ADAPTIVE HASH INDEX:
|
29
|
+
'Ibuf: size (?<ibuf_used_cells>\d+), free list len (?<ibuf_free_cells>\d+), seg size (?<ibuf_cell_count>\d+), (?<ibuf_merges>\d+) merges',
|
30
|
+
'merged operations:\n insert (?<ibuf_inserts>\d+), delete mark \d+, delete \d+\ndiscarded operations:\n insert (?<ibuf_merged>\d+)',
|
31
|
+
'Hash table size (?<hash_index_cells_total>\d+), node heap has (?<hash_index_cells_used>\d+) buffer',
|
32
|
+
# LOG:
|
33
|
+
'(?<log_writes>\d+) log i\/o\'s done',
|
34
|
+
'(?<pending_log_writes>\d+) pending log writes, (?<pending_chkp_writes>\d+) pending chkp writes',
|
35
|
+
'Log sequence number\W+(?<log_bytes_written>\d+)',
|
36
|
+
'Log flushed up to\W+(?<log_bytes_flushed>\d+)',
|
37
|
+
'Last checkpoint at\W+(?<last_checkpoint>\d+)',
|
38
|
+
# BUFFER POOL AND MEMORY
|
39
|
+
'Total memory allocated (?<total_mem_alloc>\d+); in additional pool allocated (?<additional_pool_alloc>\d+)',
|
40
|
+
'Buffer pool size\W+(?<pool_size>\d+)',
|
41
|
+
'Free buffers\W+(?<free_pages>\d+)',
|
42
|
+
'Database pages\W+(?<database_pages>\d+)',
|
43
|
+
'Modified db pages\W+(?<modified_pages>\d+)',
|
44
|
+
'Pages read (?<pages_read>\d+), created (?<pages_created>\d+), written (?<pages_written>\d+)',
|
45
|
+
# ROW OPERATIONS
|
46
|
+
'Number of rows inserted (?<rows_inserted>\d+), updated (?<rows_updated>\d+), deleted (?<rows_deleted>\d+), read (?<rows_read>\d+)',
|
47
|
+
'(?<queries_inside>\d+) queries inside InnoDB, (?<queries_queued>\d+) queries in queue',
|
48
|
+
# TRANSACTIONS
|
49
|
+
'Trx id counter (?<innodb_transactions>\d+)',
|
50
|
+
'History list length (?<history_list>\d+)' ]
|
51
|
+
|
52
|
+
variables = { 'additional_pool_alloc' => 0,
|
53
|
+
'database_pages' => 0,
|
54
|
+
'file_fsyncs' => 0,
|
55
|
+
'file_reads' => 0,
|
56
|
+
'file_writes' => 0,
|
57
|
+
'free_pages' => 0,
|
58
|
+
'hash_index_cells_total' => 0,
|
59
|
+
'hash_index_cells_used' => 0,
|
60
|
+
'history_list' => 0,
|
61
|
+
'ibuf_cell_count' => 0,
|
62
|
+
'ibuf_free_cells' => 0,
|
63
|
+
'ibuf_inserts' => 0,
|
64
|
+
'ibuf_merged' => 0,
|
65
|
+
'ibuf_merges' => 0,
|
66
|
+
'ibuf_used_cells' => 0,
|
67
|
+
'innodb_transactions' => 0,
|
68
|
+
'last_checkpoint' => 0,
|
69
|
+
'log_bytes_flushed' => 0,
|
70
|
+
'log_bytes_written' => 0,
|
71
|
+
'log_writes' => 0,
|
72
|
+
'modified_pages' => 0,
|
73
|
+
'os_waits' => 0,
|
74
|
+
'pages_created' => 0,
|
75
|
+
'pages_read' => 0,
|
76
|
+
'pages_written' => 0,
|
77
|
+
'pending_aio_log_ios' => 0,
|
78
|
+
'pending_aio_sync_ios' => 0,
|
79
|
+
'pending_buf_pool_flushes' => 0,
|
80
|
+
'pending_chkp_writes' => 0,
|
81
|
+
'pending_ibuf_aio_reads' => 0,
|
82
|
+
'pending_log_flushes' => 0,
|
83
|
+
'pending_log_writes' => 0,
|
84
|
+
'pending_normal_aio_reads' => 0,
|
85
|
+
'pending_normal_aio_writes' => 0,
|
86
|
+
'pool_size' => 0,
|
87
|
+
'queries_inside' => 0,
|
88
|
+
'queries_queued' => 0,
|
89
|
+
'rows_deleted' => 0,
|
90
|
+
'rows_inserted' => 0,
|
91
|
+
'rows_read' => 0,
|
92
|
+
'rows_updated' => 0,
|
93
|
+
'spin_rounds' => 0,
|
94
|
+
'spin_waits' => 0,
|
95
|
+
'total_mem_alloc' => 0 }
|
96
|
+
|
97
|
+
parsing.each do |regular_expression|
|
98
|
+
matchs = text.match(/#{regular_expression}/i)
|
99
|
+
|
100
|
+
unless matchs.nil?
|
101
|
+
matchs.names.each do |variable_name|
|
102
|
+
variables[variable_name] += matchs[variable_name].to_i
|
103
|
+
end
|
104
|
+
end
|
105
|
+
end
|
106
|
+
|
107
|
+
metrics_innodb = variables.map do |variable_name, variable_value|
|
108
|
+
[variable_name, variable_value]
|
109
|
+
end
|
110
|
+
|
111
|
+
# Transactions:
|
112
|
+
active_transactions = text.scan(/---TRANSACTION \d+, ACTIVE \d+ sec/).count
|
113
|
+
current_transactions = text.scan(/---TRANSACTION \d+, not started/).count
|
114
|
+
|
115
|
+
innodb_lock_structs = text.scan(/---TRANSACTION \d+, ACTIVE \d+ sec\n(?<innodb_lock_structs>\d+) lock struct\(s\), heap size/)
|
116
|
+
locked_transactions = text.scan(/---TRANSACTION \d+, ACTIVE \d+ sec\nLOCK WAIT (?<locked_transactions>\d+) lock struct\(s\), heap size/)
|
117
|
+
|
118
|
+
innodb_lock_structs = innodb_lock_structs.inject{|x,y| x.first.to_i + y.first.to_i }
|
119
|
+
locked_transactions = locked_transactions.inject{|x,y| x.first.to_i + y.first.to_i }
|
120
|
+
|
121
|
+
innodb_lock_structs = innodb_lock_structs.first.to_i if innodb_lock_structs.kind_of?(Array)
|
122
|
+
locked_transactions = locked_transactions.first.to_i if locked_transactions.kind_of?(Array)
|
123
|
+
|
124
|
+
innodb_lock_structs = 0 if innodb_lock_structs.nil?
|
125
|
+
locked_transactions = 0 if locked_transactions.nil?
|
126
|
+
|
127
|
+
current_transactions = current_transactions + active_transactions
|
128
|
+
|
129
|
+
variables = { 'innodb_lock_structs' => innodb_lock_structs,
|
130
|
+
'current_transactions' => current_transactions,
|
131
|
+
'active_transactions' => active_transactions,
|
132
|
+
'locked_transactions' => locked_transactions }
|
133
|
+
|
134
|
+
metrics_transactions = variables.map do |variable_name, variable_value|
|
135
|
+
[variable_name, variable_value]
|
136
|
+
end
|
137
|
+
|
138
|
+
metrics_innodb + metrics_transactions
|
139
|
+
end
|
140
|
+
|
141
|
+
def innodb
|
142
|
+
metadata = @mysql.execute(SQL_INNODB).first['Status']
|
143
|
+
innodb_parse(metadata)
|
144
|
+
end
|
145
|
+
|
146
|
+
def status
|
147
|
+
@mysql.execute(SQL_STATUS).map do |stats|
|
148
|
+
[stats['Variable_name'], stats['Value']]
|
149
|
+
end
|
150
|
+
end
|
151
|
+
|
152
|
+
def variables
|
153
|
+
@mysql.execute(SQL_VARIABLES).map do |stats|
|
154
|
+
[stats['Variable_name'], stats['Value']]
|
155
|
+
end
|
156
|
+
end
|
157
|
+
|
158
|
+
def slave
|
159
|
+
metrics = @mysql.execute(SQL_SLAVE).map do |stats|
|
160
|
+
stats.map do |variable_name, value|
|
161
|
+
[variable_name, value]
|
162
|
+
end
|
163
|
+
end
|
164
|
+
|
165
|
+
metrics.first
|
166
|
+
end
|
167
|
+
|
168
|
+
def locks
|
169
|
+
@mysql.execute(SQL_LOCKS).map do |lock|
|
170
|
+
["#{lock['Database']}.#{lock['Table']}", lock['In_use']]
|
171
|
+
end
|
172
|
+
end
|
173
|
+
|
174
|
+
def processlist
|
175
|
+
metrics = []
|
176
|
+
states = { 'State_closing_tables' => 0,
|
177
|
+
'State_copying_to_tmp_table' => 0,
|
178
|
+
'State_end' => 0,
|
179
|
+
'State_freeing_items' => 0,
|
180
|
+
'State_init' => 0,
|
181
|
+
'State_locked' => 0,
|
182
|
+
'State_login' => 0,
|
183
|
+
'State_none' => 0,
|
184
|
+
'State_other' => 0,
|
185
|
+
'State_preparing' => 0,
|
186
|
+
'State_reading_from_net' => 0,
|
187
|
+
'State_sending_data' => 0,
|
188
|
+
'State_sorting_result' => 0,
|
189
|
+
'State_statistics' => 0,
|
190
|
+
'State_updating' => 0,
|
191
|
+
'State_writing_to_net' => 0 };
|
192
|
+
|
193
|
+
@mysql.execute(SQL_PROCESS).each do |process|
|
194
|
+
state = process['State']
|
195
|
+
state = 'none' if process['State'].nil?
|
196
|
+
state = state.gsub(/^(Table lock|Waiting for .*lock)$/, 'Locked')
|
197
|
+
state = state.downcase
|
198
|
+
state = state.gsub(/ /, '_')
|
199
|
+
state = "State_#{state}"
|
200
|
+
|
201
|
+
if states.has_key?(state)
|
202
|
+
states[state] += 1
|
203
|
+
else
|
204
|
+
states['State_other'] += 1
|
205
|
+
end
|
206
|
+
|
207
|
+
metrics = states.map do |variable_name, value|
|
208
|
+
[variable_name, value]
|
209
|
+
end
|
210
|
+
end
|
211
|
+
metrics
|
212
|
+
end
|
213
|
+
|
214
|
+
def start
|
215
|
+
@influx.send('innodb', innodb)
|
216
|
+
@influx.send('process', processlist)
|
217
|
+
@influx.send('slave', slave)
|
218
|
+
@influx.send('status', status)
|
219
|
+
@influx.send('variables', variables)
|
220
|
+
@influx.send('locks', locks)
|
221
|
+
|
222
|
+
@mysql.close
|
223
|
+
end
|
224
|
+
end
|
225
|
+
end
|
@@ -0,0 +1,26 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Deamon
|
5
|
+
def initialize(method)
|
6
|
+
@method = method
|
7
|
+
end
|
8
|
+
|
9
|
+
def run!
|
10
|
+
begin
|
11
|
+
puts "Press Ctrl-C to stop MySQL Collector"
|
12
|
+
|
13
|
+
loop do
|
14
|
+
eval(@method)
|
15
|
+
sleep(30)
|
16
|
+
end
|
17
|
+
rescue Interrupt
|
18
|
+
puts "\n"
|
19
|
+
|
20
|
+
# Appendix E. Exit Codes With Special Meanings
|
21
|
+
# http://tldp.org/LDP/abs/html/exitcodes.html
|
22
|
+
exit 130
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -0,0 +1,58 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Influxdb
|
5
|
+
def initialize
|
6
|
+
@influx = InfluxDB::Client.new(host: $config[:influx_host],
|
7
|
+
database: $config[:influx_database],
|
8
|
+
username: $config[:influx_user],
|
9
|
+
password: $config[:influx_password],
|
10
|
+
retry: 10)
|
11
|
+
end
|
12
|
+
|
13
|
+
def vacuum(values)
|
14
|
+
# Validate it is array and have values?
|
15
|
+
return nil if values.nil?
|
16
|
+
return nil unless values.is_a?(Array)
|
17
|
+
|
18
|
+
# Remove empty variable on array of array:
|
19
|
+
values.reject{ |a| a.all? {|e| e.nil? || e.strip.empty? }}
|
20
|
+
end
|
21
|
+
|
22
|
+
def data(type, metrics, instance)
|
23
|
+
# Validate a variable metrics has values on array?
|
24
|
+
return nil if type.nil?
|
25
|
+
return nil if instance.nil?
|
26
|
+
return nil if metrics.nil?
|
27
|
+
return nil if metrics.count == 0
|
28
|
+
return nil unless metrics.is_a?(Array)
|
29
|
+
|
30
|
+
# Send value:
|
31
|
+
values = []
|
32
|
+
metrics.each do |metric, value|
|
33
|
+
if value.is_number?
|
34
|
+
values << {
|
35
|
+
series: type,
|
36
|
+
values: { value: Float(value) },
|
37
|
+
tags: { metric: metric,
|
38
|
+
instance: instance }
|
39
|
+
}
|
40
|
+
|
41
|
+
Mysqlcollector::Log.new.debug("InfluxDB Values: #{values.last}");
|
42
|
+
end
|
43
|
+
end
|
44
|
+
values
|
45
|
+
end
|
46
|
+
|
47
|
+
def send(type, metrics)
|
48
|
+
metrics = vacuum(metrics)
|
49
|
+
metrics = data(type, metrics, $config[:alias])
|
50
|
+
|
51
|
+
return nil unless metrics.is_a?(Array)
|
52
|
+
|
53
|
+
@influx.write_points(metrics)
|
54
|
+
rescue Exception => error
|
55
|
+
Mysqlcollector::Log.new.error(error.message)
|
56
|
+
end
|
57
|
+
end
|
58
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Log
|
5
|
+
def initialize
|
6
|
+
@log = Logger.new(STDOUT)
|
7
|
+
end
|
8
|
+
|
9
|
+
def valid?(command)
|
10
|
+
return false if command.nil?
|
11
|
+
return false if command.empty?
|
12
|
+
return false unless command.is_a?(String)
|
13
|
+
true
|
14
|
+
end
|
15
|
+
|
16
|
+
def debug(command)
|
17
|
+
return false unless valid?(command)
|
18
|
+
|
19
|
+
@log.debug("#{command}") if $config[:debug]
|
20
|
+
end
|
21
|
+
|
22
|
+
def error(error)
|
23
|
+
return false unless valid?(error)
|
24
|
+
|
25
|
+
@log.error(error)
|
26
|
+
exit 1
|
27
|
+
end
|
28
|
+
|
29
|
+
def sql_error(error, sql)
|
30
|
+
return false unless valid?(error)
|
31
|
+
return false unless valid?(sql)
|
32
|
+
|
33
|
+
@log.error("MySQL Error: #{error}")
|
34
|
+
@log.error("MySQL SQL Statement: #{sql}")
|
35
|
+
exit 1
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Mysql
|
5
|
+
def initialize
|
6
|
+
@conn = Mysql2::Client.new(host: $config[:mysql_host],
|
7
|
+
port: $config[:mysql_port],
|
8
|
+
username: $config[:mysql_username],
|
9
|
+
password: $config[:mysql_password])
|
10
|
+
|
11
|
+
Mysqlcollector::Log.new.debug("Connected on MySQL server: #{$config[:mysql_host]}:#{$config[:mysql_port]}");
|
12
|
+
rescue Exception => error
|
13
|
+
Mysqlcollector::Log.new.error(error.message)
|
14
|
+
end
|
15
|
+
|
16
|
+
def execute(sql)
|
17
|
+
begin
|
18
|
+
Mysqlcollector::Log.new.debug("MySQL Query: #{sql}")
|
19
|
+
|
20
|
+
@conn.query(sql).each(:as => :hash)
|
21
|
+
rescue Exception => error
|
22
|
+
Mysqlcollector::Log.new.sql_error(error, sql);
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def close
|
27
|
+
@conn.close
|
28
|
+
rescue
|
29
|
+
exit 1
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
@@ -0,0 +1,14 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
module Mysqlcollector
|
4
|
+
class Template
|
5
|
+
def grafana
|
6
|
+
path = File.expand_path(File.join(Dir.pwd, '/assets/grafana_dashboard.json'))
|
7
|
+
file = File.open(path, 'r')
|
8
|
+
file.each_line do |line|
|
9
|
+
puts line
|
10
|
+
end
|
11
|
+
file.close
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
metadata
ADDED
@@ -0,0 +1,155 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: mysqlcollector
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Nicola Strappazzon C.
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2016-06-05 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: cause
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: mysql2
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - '='
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 0.3.18
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - '='
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 0.3.18
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: influxdb
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - '='
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: 0.3.0
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - '='
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: 0.3.0
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: minitest
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '5.0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '5.0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: bundler
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '1.12'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '1.12'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: rake
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - "~>"
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '10.1'
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - "~>"
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '10.1'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: coveralls
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - ">="
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '0'
|
104
|
+
type: :development
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - ">="
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '0'
|
111
|
+
description: MySQL Collector is a Ruby Gem and command line tools, written for UNIX-like
|
112
|
+
operating systems. Collect MySQL Metrics and send them to InfluxDB
|
113
|
+
email: nicola@swapbytes.com
|
114
|
+
executables:
|
115
|
+
- mysqlcollector
|
116
|
+
extensions: []
|
117
|
+
extra_rdoc_files: []
|
118
|
+
files:
|
119
|
+
- bin/mysqlcollector
|
120
|
+
- lib/mysqlcollector/cli.rb
|
121
|
+
- lib/mysqlcollector/collector.rb
|
122
|
+
- lib/mysqlcollector/daemon.rb
|
123
|
+
- lib/mysqlcollector/influxdb.rb
|
124
|
+
- lib/mysqlcollector/log.rb
|
125
|
+
- lib/mysqlcollector/mysql.rb
|
126
|
+
- lib/mysqlcollector/template.rb
|
127
|
+
- lib/mysqlcollector/version.rb
|
128
|
+
- lib/mysqlcollector.rb
|
129
|
+
- LICENSE.txt
|
130
|
+
- README.md
|
131
|
+
homepage: https://github.com/swapbytes/mysqlcollector
|
132
|
+
licenses:
|
133
|
+
- MIT
|
134
|
+
metadata: {}
|
135
|
+
post_install_message:
|
136
|
+
rdoc_options: []
|
137
|
+
require_paths:
|
138
|
+
- lib
|
139
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
140
|
+
requirements:
|
141
|
+
- - ">="
|
142
|
+
- !ruby/object:Gem::Version
|
143
|
+
version: 1.9.3
|
144
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
145
|
+
requirements:
|
146
|
+
- - ">="
|
147
|
+
- !ruby/object:Gem::Version
|
148
|
+
version: '0'
|
149
|
+
requirements: []
|
150
|
+
rubyforge_project:
|
151
|
+
rubygems_version: 2.0.14.1
|
152
|
+
signing_key:
|
153
|
+
specification_version: 4
|
154
|
+
summary: Collect MySQL Metrics and send them to InfluxDB
|
155
|
+
test_files: []
|