fluent-plugin-groonga-query-log 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: acf3ada166906b4c1583d559239d1af2e4917fb4
4
+ data.tar.gz: bb12b02153446679022f26e28ad299ca50c6b4fc
5
+ SHA512:
6
+ metadata.gz: d7292a34808e12ba99aa0f381fe42d77e570819f016452c738611d9f7b203d29ed951181c6a423ed561aefb956b4e3852d0a2e92b36fcd0222d0dbc87c41d5ab
7
+ data.tar.gz: 69d4bb292bf90fc2c8ea641aa02915bdc03b47dacbc34ba6738b03c53b11565e9380d868f86442c1e8255c5d2202429fd128be84e55fa79effce7b67568f9f90
data/.yardopts ADDED
@@ -0,0 +1,8 @@
1
+ --title fluent-plugin-groonga-query-log
2
+ --readme README.md
3
+ --markup markdown
4
+ --markup-provider kramdown
5
+ --files doc/text/**/*.*
6
+ --output-dir doc/reference/en
7
+ --charset utf-8
8
+ --use-cache
data/Gemfile ADDED
@@ -0,0 +1,20 @@
1
+ # -*- mode: ruby -*-
2
+ #
3
+ # Copyright (C) 2015 Kouhei Sutou <kou@clear-code.com>
4
+ #
5
+ # This library is free software: you can redistribute it and/or modify
6
+ # it under the terms of the GNU Lesser General Public License as published by
7
+ # the Free Software Foundation, either version 3 of the License, or
8
+ # (at your option) any later version.
9
+ #
10
+ # This library is distributed in the hope that it will be useful,
11
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
12
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
+ # GNU Lesser General Public License for more details.
14
+ #
15
+ # You should have received a copy of the GNU Lesser General Public License
16
+ # along with this library. If not, see <http://www.gnu.org/licenses/>.
17
+
18
+ source "https://rubygems.org/"
19
+
20
+ gemspec
data/README.md ADDED
@@ -0,0 +1,47 @@
1
+ # README
2
+
3
+ ## Name
4
+
5
+ fluent-plugin-groonga-query-log
6
+
7
+ ## Description
8
+
9
+ Fluent-plugin-groonga-query-log is a Fluentd plugin to parse
10
+ [Groonga](http://groonga.org/)'s
11
+ [query log](http://groonga.org/docs/reference/log.html#query-log) with
12
+ Fluentd.
13
+
14
+ You can detect slow query in real time by using this plugin.
15
+
16
+ ## Install
17
+
18
+ % gem install fluent-plugin-groonga-query-log
19
+
20
+ ## Usage
21
+
22
+ TODO
23
+
24
+ ## Authors
25
+
26
+ * Kouhei Sutou `<kou@clear-code.com>`
27
+
28
+ ## License
29
+
30
+ LGPL 3. See doc/text/lgpl-3.txt for details.
31
+
32
+ (Kouhei Sutou has a right to change the license including
33
+ contributed patches.)
34
+
35
+ ## Mailing list
36
+
37
+ * English: [groonga-talk](https://lists.sourceforge.net/lists/listinfo/groonga-talk)
38
+ * Japanese: [groonga-dev](http://lists.sourceforge.jp/mailman/listinfo/groonga-dev)
39
+
40
+ ## Source
41
+
42
+ The repository for fluent-plugin-groonga-query-log is on
43
+ [GitHub](https://github.com/groonga/fluent-plugin-groonga-query-log/).
44
+
45
+ ## Thanks
46
+
47
+ * ...
@@ -0,0 +1,165 @@
1
+ GNU LESSER GENERAL PUBLIC LICENSE
2
+ Version 3, 29 June 2007
3
+
4
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
5
+ Everyone is permitted to copy and distribute verbatim copies
6
+ of this license document, but changing it is not allowed.
7
+
8
+
9
+ This version of the GNU Lesser General Public License incorporates
10
+ the terms and conditions of version 3 of the GNU General Public
11
+ License, supplemented by the additional permissions listed below.
12
+
13
+ 0. Additional Definitions.
14
+
15
+ As used herein, "this License" refers to version 3 of the GNU Lesser
16
+ General Public License, and the "GNU GPL" refers to version 3 of the GNU
17
+ General Public License.
18
+
19
+ "The Library" refers to a covered work governed by this License,
20
+ other than an Application or a Combined Work as defined below.
21
+
22
+ An "Application" is any work that makes use of an interface provided
23
+ by the Library, but which is not otherwise based on the Library.
24
+ Defining a subclass of a class defined by the Library is deemed a mode
25
+ of using an interface provided by the Library.
26
+
27
+ A "Combined Work" is a work produced by combining or linking an
28
+ Application with the Library. The particular version of the Library
29
+ with which the Combined Work was made is also called the "Linked
30
+ Version".
31
+
32
+ The "Minimal Corresponding Source" for a Combined Work means the
33
+ Corresponding Source for the Combined Work, excluding any source code
34
+ for portions of the Combined Work that, considered in isolation, are
35
+ based on the Application, and not on the Linked Version.
36
+
37
+ The "Corresponding Application Code" for a Combined Work means the
38
+ object code and/or source code for the Application, including any data
39
+ and utility programs needed for reproducing the Combined Work from the
40
+ Application, but excluding the System Libraries of the Combined Work.
41
+
42
+ 1. Exception to Section 3 of the GNU GPL.
43
+
44
+ You may convey a covered work under sections 3 and 4 of this License
45
+ without being bound by section 3 of the GNU GPL.
46
+
47
+ 2. Conveying Modified Versions.
48
+
49
+ If you modify a copy of the Library, and, in your modifications, a
50
+ facility refers to a function or data to be supplied by an Application
51
+ that uses the facility (other than as an argument passed when the
52
+ facility is invoked), then you may convey a copy of the modified
53
+ version:
54
+
55
+ a) under this License, provided that you make a good faith effort to
56
+ ensure that, in the event an Application does not supply the
57
+ function or data, the facility still operates, and performs
58
+ whatever part of its purpose remains meaningful, or
59
+
60
+ b) under the GNU GPL, with none of the additional permissions of
61
+ this License applicable to that copy.
62
+
63
+ 3. Object Code Incorporating Material from Library Header Files.
64
+
65
+ The object code form of an Application may incorporate material from
66
+ a header file that is part of the Library. You may convey such object
67
+ code under terms of your choice, provided that, if the incorporated
68
+ material is not limited to numerical parameters, data structure
69
+ layouts and accessors, or small macros, inline functions and templates
70
+ (ten or fewer lines in length), you do both of the following:
71
+
72
+ a) Give prominent notice with each copy of the object code that the
73
+ Library is used in it and that the Library and its use are
74
+ covered by this License.
75
+
76
+ b) Accompany the object code with a copy of the GNU GPL and this license
77
+ document.
78
+
79
+ 4. Combined Works.
80
+
81
+ You may convey a Combined Work under terms of your choice that,
82
+ taken together, effectively do not restrict modification of the
83
+ portions of the Library contained in the Combined Work and reverse
84
+ engineering for debugging such modifications, if you also do each of
85
+ the following:
86
+
87
+ a) Give prominent notice with each copy of the Combined Work that
88
+ the Library is used in it and that the Library and its use are
89
+ covered by this License.
90
+
91
+ b) Accompany the Combined Work with a copy of the GNU GPL and this license
92
+ document.
93
+
94
+ c) For a Combined Work that displays copyright notices during
95
+ execution, include the copyright notice for the Library among
96
+ these notices, as well as a reference directing the user to the
97
+ copies of the GNU GPL and this license document.
98
+
99
+ d) Do one of the following:
100
+
101
+ 0) Convey the Minimal Corresponding Source under the terms of this
102
+ License, and the Corresponding Application Code in a form
103
+ suitable for, and under terms that permit, the user to
104
+ recombine or relink the Application with a modified version of
105
+ the Linked Version to produce a modified Combined Work, in the
106
+ manner specified by section 6 of the GNU GPL for conveying
107
+ Corresponding Source.
108
+
109
+ 1) Use a suitable shared library mechanism for linking with the
110
+ Library. A suitable mechanism is one that (a) uses at run time
111
+ a copy of the Library already present on the user's computer
112
+ system, and (b) will operate properly with a modified version
113
+ of the Library that is interface-compatible with the Linked
114
+ Version.
115
+
116
+ e) Provide Installation Information, but only if you would otherwise
117
+ be required to provide such information under section 6 of the
118
+ GNU GPL, and only to the extent that such information is
119
+ necessary to install and execute a modified version of the
120
+ Combined Work produced by recombining or relinking the
121
+ Application with a modified version of the Linked Version. (If
122
+ you use option 4d0, the Installation Information must accompany
123
+ the Minimal Corresponding Source and Corresponding Application
124
+ Code. If you use option 4d1, you must provide the Installation
125
+ Information in the manner specified by section 6 of the GNU GPL
126
+ for conveying Corresponding Source.)
127
+
128
+ 5. Combined Libraries.
129
+
130
+ You may place library facilities that are a work based on the
131
+ Library side by side in a single library together with other library
132
+ facilities that are not Applications and are not covered by this
133
+ License, and convey such a combined library under terms of your
134
+ choice, if you do both of the following:
135
+
136
+ a) Accompany the combined library with a copy of the same work based
137
+ on the Library, uncombined with any other library facilities,
138
+ conveyed under the terms of this License.
139
+
140
+ b) Give prominent notice with the combined library that part of it
141
+ is a work based on the Library, and explaining where to find the
142
+ accompanying uncombined form of the same work.
143
+
144
+ 6. Revised Versions of the GNU Lesser General Public License.
145
+
146
+ The Free Software Foundation may publish revised and/or new versions
147
+ of the GNU Lesser General Public License from time to time. Such new
148
+ versions will be similar in spirit to the present version, but may
149
+ differ in detail to address new problems or concerns.
150
+
151
+ Each version is given a distinguishing version number. If the
152
+ Library as you received it specifies that a certain numbered version
153
+ of the GNU Lesser General Public License "or any later version"
154
+ applies to it, you have the option of following the terms and
155
+ conditions either of that published version or of any later version
156
+ published by the Free Software Foundation. If the Library as you
157
+ received it does not specify a version number of the GNU Lesser
158
+ General Public License, you may choose any version of the GNU Lesser
159
+ General Public License ever published by the Free Software Foundation.
160
+
161
+ If the Library as you received it specifies that a proxy can decide
162
+ whether future versions of the GNU Lesser General Public License shall
163
+ apply, that proxy's public statement of acceptance of any version is
164
+ permanent authorization for you to choose that version for the
165
+ Library.
data/doc/text/news.md ADDED
@@ -0,0 +1,5 @@
1
+ # News
2
+
3
+ ## 1.0.0: 2015-08-12
4
+
5
+ The first release!!!
@@ -0,0 +1,43 @@
1
+ # -*- mode: ruby -*-
2
+ #
3
+ # Copyright (C) 2015 Kouhei Sutou <kou@clear-code.com>
4
+ #
5
+ # This library is free software: you can redistribute it and/or modify
6
+ # it under the terms of the GNU Lesser General Public License as published by
7
+ # the Free Software Foundation, either version 3 of the License, or
8
+ # (at your option) any later version.
9
+ #
10
+ # This library is distributed in the hope that it will be useful,
11
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
12
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
+ # GNU Lesser General Public License for more details.
14
+ #
15
+ # You should have received a copy of the GNU Lesser General Public License
16
+ # along with this library. If not, see <http://www.gnu.org/licenses/>.
17
+
18
+ Gem::Specification.new do |spec|
19
+ spec.name = "fluent-plugin-groonga-query-log"
20
+ spec.version = "1.0.0"
21
+ spec.authors = ["Kouhei Sutou"]
22
+ spec.email = ["kou@clear-code.com"]
23
+ spec.summary = "Fluentd plugin to parse Groonga's query log."
24
+ spec.description = "You can detect slow query in real time by using this plugin."
25
+ spec.homepage = "https://github.com/groonga/fluent-plugin-groonga-query-log"
26
+ spec.license = "LGPLv3"
27
+
28
+ spec.files = ["README.md", "Gemfile", "#{spec.name}.gemspec"]
29
+ spec.files += [".yardopts"]
30
+ spec.files += Dir.glob("lib/**/*.rb")
31
+ spec.files += Dir.glob("sample/**/*")
32
+ spec.files += Dir.glob("doc/text/**/*")
33
+ spec.test_files += Dir.glob("test/**/*")
34
+ spec.require_paths = ["lib"]
35
+
36
+ spec.add_runtime_dependency("fluentd")
37
+ spec.add_runtime_dependency("groonga-query-log")
38
+
39
+ spec.add_development_dependency("rake")
40
+ spec.add_development_dependency("bundler")
41
+ spec.add_development_dependency("packnga")
42
+ spec.add_development_dependency("test-unit")
43
+ end
@@ -0,0 +1,111 @@
1
+ # Copyright (C) 2015 Kouhei Sutou <kou@clear-code.com>
2
+ #
3
+ # This library is free software: you can redistribute it and/or modify
4
+ # it under the terms of the GNU Lesser General Public License as published by
5
+ # the Free Software Foundation, either version 3 of the License, or
6
+ # (at your option) any later version.
7
+ #
8
+ # This library is distributed in the hope that it will be useful,
9
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
10
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11
+ # GNU Lesser General Public License for more details.
12
+ #
13
+ # You should have received a copy of the GNU Lesser General Public License
14
+ # along with this library. If not, see <http://www.gnu.org/licenses/>.
15
+
16
+ require "time"
17
+
18
+ require "groonga/query-log"
19
+
20
+ module Fluent
21
+ class GroongaQueryLogFilter < Filter
22
+ Plugin.register_filter("groonga_query_log", self)
23
+
24
+ config_param :raw_data_column_name, :string, :default => "message"
25
+ config_param :slow_operation_threshold, :float, :default => 0.1
26
+ config_param :slow_response_threshold, :float, :default => 0.2
27
+ config_param :flatten, :bool, :default => false
28
+ config_param :flatten_separator, :string, :default => nil
29
+
30
+ def configure(conf)
31
+ super
32
+
33
+ @parser = Groonga::QueryLog::Parser.new
34
+ end
35
+
36
+ def filter_stream(tag, event_stream)
37
+ statistics_event_stream = MultiEventStream.new
38
+ event_stream.each do |time, record|
39
+ raw_data = record[@raw_data_column_name]
40
+ next if raw_data.nil?
41
+ @parser.parse(raw_data) do |statistic|
42
+ statistic_record = create_record(statistic)
43
+ statistics_event_stream.add(time, statistic_record)
44
+ end
45
+ end
46
+ statistics_event_stream
47
+ end
48
+
49
+ private
50
+ def create_record(statistic)
51
+ record = statistic.to_hash
52
+ record["start_time"] = format_time(statistic.start_time)
53
+ record["last_time"] = format_time(statistic.last_time)
54
+ if @flatten
55
+ flatten_record!(record)
56
+ end
57
+ record
58
+ end
59
+
60
+ def format_time(time)
61
+ time.utc.iso8601(6)
62
+ end
63
+
64
+ def flatten_record!(record)
65
+ record.keys.each do |key|
66
+ value = record[key]
67
+ case value
68
+ when Hash
69
+ flatten_record_value_hash!(record, key, value)
70
+ when Array
71
+ flatten_record_value_array!(record, key, value)
72
+ end
73
+ end
74
+ end
75
+
76
+ def flatten_record_value!(record, base_key, value)
77
+ case value
78
+ when Hash
79
+ flatten_record_value_hash!(record, base_key, value)
80
+ when Array
81
+ flatten_record_value_array!(record, base_key, value)
82
+ else
83
+ record[base_key] = value
84
+ end
85
+ end
86
+
87
+ def flatten_record_value_hash!(record, base_key, hash)
88
+ record.delete(base_key)
89
+ hash.each do |key, value|
90
+ if @flatten_separator
91
+ flat_key = "#{base_key}#{@flatten_separator}#{key}"
92
+ else
93
+ flat_key = "#{base_key}.#{key}"
94
+ end
95
+ flatten_record_value!(record, flat_key, value)
96
+ end
97
+ end
98
+
99
+ def flatten_record_value_array!(record, base_key, array)
100
+ record.delete(base_key)
101
+ array.each_with_index do |value, i|
102
+ if @flatten_separator
103
+ flat_key = "#{base_key}#{@flatten_separator}#{i}"
104
+ else
105
+ flat_key = "#{base_key}[#{i}]"
106
+ end
107
+ flatten_record_value!(record, flat_key, value)
108
+ end
109
+ end
110
+ end
111
+ end
data/sample/dump.conf ADDED
@@ -0,0 +1,15 @@
1
+ <source>
2
+ type tail
3
+ path "#{ENV['GROONGA_QUERY_LOG_PATH'] || '/var/log/groonga/query.log'}"
4
+ read_from_head true
5
+ tag groonga.query
6
+ format none
7
+ </source>
8
+
9
+ <filter groonga.query>
10
+ type groonga_query_log
11
+ </filter>
12
+
13
+ <match **>
14
+ type stdout
15
+ </match>
data/sample/store.conf ADDED
@@ -0,0 +1,26 @@
1
+ <source>
2
+ type tail
3
+ path "#{ENV['GROONGA_QUERY_LOG_PATH'] || '/var/log/groonga/query.log'}"
4
+ pos_file /var/log/td-agent/groonga-query-log.pos
5
+ read_from_head "#{ENV['GROONGA_QUERY_LOG_READ_FROM_HEAD'] || 'false'}"
6
+ tag groonga.query
7
+ format none
8
+ </source>
9
+
10
+ <filter groonga.query>
11
+ type groonga_query_log
12
+ flatten true
13
+ flatten_separator _
14
+ </filter>
15
+
16
+ <match groonga.query>
17
+ type groonga
18
+ store_table QueyLogs
19
+
20
+ protocol http
21
+ host 127.0.0.1
22
+
23
+ buffer_type file
24
+ buffer_path /tmp/buffer
25
+ flush_interval 1
26
+ </match>
data/test/run-test.rb ADDED
@@ -0,0 +1,31 @@
1
+ #!/usr/bin/env ruby
2
+ #
3
+ # Copyright (C) 2015 Kouhei Sutou <kou@clear-code.com>
4
+ #
5
+ # This library is free software: you can redistribute it and/or modify
6
+ # it under the terms of the GNU Lesser General Public License as published by
7
+ # the Free Software Foundation, either version 3 of the License, or
8
+ # (at your option) any later version.
9
+ #
10
+ # This library is distributed in the hope that it will be useful,
11
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
12
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
+ # GNU Lesser General Public License for more details.
14
+ #
15
+ # You should have received a copy of the GNU Lesser General Public License
16
+ # along with this library. If not, see <http://www.gnu.org/licenses/>.
17
+
18
+ # $VERBOSE = true
19
+
20
+ base_dir = File.expand_path(File.join(File.dirname(__FILE__), ".."))
21
+ lib_dir = File.join(base_dir, "lib")
22
+ test_dir = File.join(base_dir, "test")
23
+
24
+ require "test-unit"
25
+
26
+ $LOAD_PATH.unshift(lib_dir)
27
+
28
+ ENV["TZ"] = "Asia/Tokyo"
29
+ ENV["TEST_UNIT_MAX_DIFF_TARGET_STRING_SIZE"] ||= "5000"
30
+
31
+ exit(Test::Unit::AutoRunner.run(true, test_dir))
@@ -0,0 +1,235 @@
1
+ # Copyright (C) 2015 Kouhei Sutou <kou@clear-code.com>
2
+ #
3
+ # This library is free software: you can redistribute it and/or modify
4
+ # it under the terms of the GNU Lesser General Public License as published by
5
+ # the Free Software Foundation, either version 3 of the License, or
6
+ # (at your option) any later version.
7
+ #
8
+ # This library is distributed in the hope that it will be useful,
9
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
10
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11
+ # GNU Lesser General Public License for more details.
12
+ #
13
+ # You should have received a copy of the GNU Lesser General Public License
14
+ # along with this library. If not, see <http://www.gnu.org/licenses/>.
15
+
16
+ require "fluent/test"
17
+ require "fluent/plugin/filter_groonga_query_log"
18
+
19
+ class GroongaQueryLogFilterTest < Test::Unit::TestCase
20
+ setup do
21
+ Fluent::Test.setup
22
+ @now = Time.parse("2015-08-12T08:45:42Z").to_i
23
+ Fluent::Engine.now = @now
24
+ end
25
+
26
+ private
27
+ def create_driver(configuration)
28
+ driver = Fluent::Test::FilterTestDriver.new(Fluent::GroongaQueryLogFilter)
29
+ driver.configure(configuration, true)
30
+ driver
31
+ end
32
+
33
+ sub_test_case "configure" do
34
+ test "default" do
35
+ driver = create_driver("")
36
+ filter = driver.instance
37
+ assert_equal({
38
+ :raw_data_column_name => "message",
39
+ :slow_operation_threshold => 0.1,
40
+ :slow_response_threshold => 0.2,
41
+ :flatten => false,
42
+ :flatten_separator => nil,
43
+ },
44
+ {
45
+ :raw_data_column_name => filter.raw_data_column_name,
46
+ :slow_operation_threshold => filter.slow_operation_threshold,
47
+ :slow_response_threshold => filter.slow_response_threshold,
48
+ :flatten => filter.flatten,
49
+ :flatten_separator => filter.flatten_separator,
50
+ })
51
+ end
52
+
53
+ test "raw_data_column_name" do
54
+ driver = create_driver("raw_data_column_name data")
55
+ filter = driver.instance
56
+ assert_equal("data", filter.raw_data_column_name)
57
+ end
58
+
59
+ test "slow_operation_threshold" do
60
+ driver = create_driver("slow_operation_threshold 1.1")
61
+ filter = driver.instance
62
+ assert_equal(1.1, filter.slow_operation_threshold)
63
+ end
64
+
65
+ test "slow_response_threshold" do
66
+ driver = create_driver("slow_response_threshold 2.5")
67
+ filter = driver.instance
68
+ assert_equal(2.5, filter.slow_response_threshold)
69
+ end
70
+
71
+ test "flatten" do
72
+ driver = create_driver("flatten true")
73
+ filter = driver.instance
74
+ assert_equal(true, filter.flatten)
75
+ end
76
+
77
+ test "flatten_separator" do
78
+ driver = create_driver("flatten_separator _")
79
+ filter = driver.instance
80
+ assert_equal("_", filter.flatten_separator)
81
+ end
82
+ end
83
+
84
+ sub_test_case "filter_stream" do
85
+ def emit(configuration, messages)
86
+ driver = create_driver(configuration)
87
+ driver.run do
88
+ messages.each do |message|
89
+ driver.emit({"message" => message}, @now)
90
+ end
91
+ end
92
+ driver.filtered
93
+ end
94
+
95
+ test "partial" do
96
+ messages = [
97
+ "2015-08-12 15:50:40.130990|0x7fb07d113da0|>/d/select?table=Entries&match_columns=name&query=xml",
98
+ "2015-08-12 15:50:40.296165|0x7fb07d113da0|:000000165177838 filter(10)",
99
+ "2015-08-12 15:50:40.296172|0x7fb07d113da0|:000000165184723 select(10)",
100
+ "2015-08-12 15:50:41.228129|0x7fb07d113da0|:000001097153433 output(10)",
101
+ ]
102
+ event_stream = emit("", messages)
103
+ assert_equal([], event_stream.to_a)
104
+ end
105
+
106
+ test "one" do
107
+ messages = [
108
+ "2015-08-12 15:50:40.130990|0x7fb07d113da0|>/d/select?table=Entries&match_columns=name&query=xml",
109
+ "2015-08-12 15:50:40.296165|0x7fb07d113da0|:000000165177838 filter(10)",
110
+ "2015-08-12 15:50:40.296172|0x7fb07d113da0|:000000165184723 select(10)",
111
+ "2015-08-12 15:50:41.228129|0x7fb07d113da0|:000001097153433 output(10)",
112
+ "2015-08-12 15:50:41.228317|0x7fb07d113da0|<000001097334986 rc=0",
113
+ ]
114
+ statistic = {
115
+ "start_time" => "2015-08-12T06:50:40.130990Z",
116
+ "last_time" => "2015-08-12T06:50:41.228324Z",
117
+ "elapsed" => 1.0973349860000001,
118
+ "return_code" => 0,
119
+ "slow" => true,
120
+ "command" => {
121
+ "raw" => "/d/select?table=Entries&match_columns=name&query=xml",
122
+ "name" => "select",
123
+ "parameters" => [
124
+ {"key" => :table, "value" => "Entries"},
125
+ {"key" => :match_columns, "value" => "name"},
126
+ {"key" => :query, "value" => "xml"},
127
+ ],
128
+ },
129
+ "operations" => [
130
+ {
131
+ "context" => "query: xml",
132
+ "name" => "filter",
133
+ "relative_elapsed" => 0.165177838,
134
+ "slow" => true,
135
+ },
136
+ {
137
+ "context" => nil,
138
+ "name" => "select",
139
+ "relative_elapsed" => 6.884999999999999e-06,
140
+ "slow" => false,
141
+ },
142
+ {
143
+ "context" => nil,
144
+ "name" => "output",
145
+ "relative_elapsed" => 0.93196871,
146
+ "slow" => true,
147
+ },
148
+ ],
149
+ }
150
+ event_stream = emit("", messages)
151
+ assert_equal([[@now, statistic]], event_stream.to_a)
152
+ end
153
+
154
+ test "flatten" do
155
+ messages = [
156
+ "2015-08-12 15:50:40.130990|0x7fb07d113da0|>/d/select?table=Entries&match_columns=name&query=xml",
157
+ "2015-08-12 15:50:40.296165|0x7fb07d113da0|:000000165177838 filter(10)",
158
+ "2015-08-12 15:50:40.296172|0x7fb07d113da0|:000000165184723 select(10)",
159
+ "2015-08-12 15:50:41.228129|0x7fb07d113da0|:000001097153433 output(10)",
160
+ "2015-08-12 15:50:41.228317|0x7fb07d113da0|<000001097334986 rc=0",
161
+ ]
162
+ statistic = {
163
+ "start_time" => "2015-08-12T06:50:40.130990Z",
164
+ "last_time" => "2015-08-12T06:50:41.228324Z",
165
+ "elapsed" => 1.0973349860000001,
166
+ "return_code" => 0,
167
+ "slow" => true,
168
+ "command.raw" => "/d/select?table=Entries&match_columns=name&query=xml",
169
+ "command.name" => "select",
170
+ "command.parameters[0].key" => :table,
171
+ "command.parameters[0].value" => "Entries",
172
+ "command.parameters[1].key" => :match_columns,
173
+ "command.parameters[1].value" => "name",
174
+ "command.parameters[2].key" => :query,
175
+ "command.parameters[2].value" => "xml",
176
+ "operations[0].context" => "query: xml",
177
+ "operations[0].name" => "filter",
178
+ "operations[0].relative_elapsed" => 0.165177838,
179
+ "operations[0].slow" => true,
180
+ "operations[1].context" => nil,
181
+ "operations[1].name" => "select",
182
+ "operations[1].relative_elapsed" => 6.884999999999999e-06,
183
+ "operations[1].slow" => false,
184
+ "operations[2].context" => nil,
185
+ "operations[2].name" => "output",
186
+ "operations[2].relative_elapsed" => 0.93196871,
187
+ "operations[2].slow" => true,
188
+ }
189
+ event_stream = emit("flatten true", messages)
190
+ assert_equal([[@now, statistic]], event_stream.to_a)
191
+ end
192
+
193
+ test "flatten_separator" do
194
+ messages = [
195
+ "2015-08-12 15:50:40.130990|0x7fb07d113da0|>/d/select?table=Entries&match_columns=name&query=xml",
196
+ "2015-08-12 15:50:40.296165|0x7fb07d113da0|:000000165177838 filter(10)",
197
+ "2015-08-12 15:50:40.296172|0x7fb07d113da0|:000000165184723 select(10)",
198
+ "2015-08-12 15:50:41.228129|0x7fb07d113da0|:000001097153433 output(10)",
199
+ "2015-08-12 15:50:41.228317|0x7fb07d113da0|<000001097334986 rc=0",
200
+ ]
201
+ statistic = {
202
+ "start_time" => "2015-08-12T06:50:40.130990Z",
203
+ "last_time" => "2015-08-12T06:50:41.228324Z",
204
+ "elapsed" => 1.0973349860000001,
205
+ "return_code" => 0,
206
+ "slow" => true,
207
+ "command_raw" => "/d/select?table=Entries&match_columns=name&query=xml",
208
+ "command_name" => "select",
209
+ "command_parameters_0_key" => :table,
210
+ "command_parameters_0_value" => "Entries",
211
+ "command_parameters_1_key" => :match_columns,
212
+ "command_parameters_1_value" => "name",
213
+ "command_parameters_2_key" => :query,
214
+ "command_parameters_2_value" => "xml",
215
+ "operations_0_context" => "query: xml",
216
+ "operations_0_name" => "filter",
217
+ "operations_0_relative_elapsed" => 0.165177838,
218
+ "operations_0_slow" => true,
219
+ "operations_1_context" => nil,
220
+ "operations_1_name" => "select",
221
+ "operations_1_relative_elapsed" => 6.884999999999999e-06,
222
+ "operations_1_slow" => false,
223
+ "operations_2_context" => nil,
224
+ "operations_2_name" => "output",
225
+ "operations_2_relative_elapsed" => 0.93196871,
226
+ "operations_2_slow" => true,
227
+ }
228
+ event_stream = emit(<<-CONFIGURATION, messages)
229
+ flatten true
230
+ flatten_separator _
231
+ CONFIGURATION
232
+ assert_equal([[@now, statistic]], event_stream.to_a)
233
+ end
234
+ end
235
+ end
metadata ADDED
@@ -0,0 +1,142 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: fluent-plugin-groonga-query-log
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - Kouhei Sutou
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2015-08-12 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: fluentd
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: groonga-query-log
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rake
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: bundler
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: packnga
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: test-unit
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ description: You can detect slow query in real time by using this plugin.
98
+ email:
99
+ - kou@clear-code.com
100
+ executables: []
101
+ extensions: []
102
+ extra_rdoc_files: []
103
+ files:
104
+ - ".yardopts"
105
+ - Gemfile
106
+ - README.md
107
+ - doc/text/lgpl-3.txt
108
+ - doc/text/news.md
109
+ - fluent-plugin-groonga-query-log.gemspec
110
+ - lib/fluent/plugin/filter_groonga_query_log.rb
111
+ - sample/dump.conf
112
+ - sample/store.conf
113
+ - test/run-test.rb
114
+ - test/test_filter_groonga_query_log.rb
115
+ homepage: https://github.com/groonga/fluent-plugin-groonga-query-log
116
+ licenses:
117
+ - LGPLv3
118
+ metadata: {}
119
+ post_install_message:
120
+ rdoc_options: []
121
+ require_paths:
122
+ - lib
123
+ required_ruby_version: !ruby/object:Gem::Requirement
124
+ requirements:
125
+ - - ">="
126
+ - !ruby/object:Gem::Version
127
+ version: '0'
128
+ required_rubygems_version: !ruby/object:Gem::Requirement
129
+ requirements:
130
+ - - ">="
131
+ - !ruby/object:Gem::Version
132
+ version: '0'
133
+ requirements: []
134
+ rubyforge_project:
135
+ rubygems_version: 2.2.2
136
+ signing_key:
137
+ specification_version: 4
138
+ summary: Fluentd plugin to parse Groonga's query log.
139
+ test_files:
140
+ - test/run-test.rb
141
+ - test/test_filter_groonga_query_log.rb
142
+ has_rdoc: