norikra-client 1.3.0-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +18 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +22 -0
- data/README.md +186 -0
- data/Rakefile +1 -0
- data/bin/norikra-client +8 -0
- data/lib/norikra-client.rb +1 -0
- data/lib/norikra/client.rb +91 -0
- data/lib/norikra/client/cli.rb +302 -0
- data/lib/norikra/client/cli/formatter.rb +35 -0
- data/lib/norikra/client/cli/parser.rb +35 -0
- data/lib/norikra/client/version.rb +5 -0
- data/norikra-client.gemspec +32 -0
- metadata +129 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 3dbf7ff93daeb041efacdb5735068fb53ce9feda
|
4
|
+
data.tar.gz: db54c46353e386b05499a72f5e269f5e7271ea41
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 55de95f13e671b42af92ec4666bd4b2a8b2ccf80218697b3053248430fc179e2b542a885e8312fb7e1d05f36e6c93dc4a4ea3cafd29033da85146f095a0b3155
|
7
|
+
data.tar.gz: 3b672c76011fd42ee68bf09263ed204d9ae6e36bd731772a847775230340ad725dd5e10a078890b04b6de604ad633495bedec997a7daa5071335f2dc4a35310b
|
data/.gitignore
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2013 TAGOMORI Satoshi
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,186 @@
|
|
1
|
+
# Norikra::Client (norikra-client)
|
2
|
+
|
3
|
+
This is the client library implementation for Norikra, and its handy CLI commands.
|
4
|
+
|
5
|
+
`Norikra` is CEP server, based on Esper. You can install `gem install norikra` on your JRuby.
|
6
|
+
For more information, see http://norikra.github.io .
|
7
|
+
|
8
|
+
Command `norikra-client` and module `Norikra::Client` are provided for both of CRuby and JRuby.
|
9
|
+
|
10
|
+
* For CRuby: `gem install norikra-client`
|
11
|
+
* For JRuby: `gem install norikra-client-jruby`
|
12
|
+
|
13
|
+
Both have a same API and same data representaions. You can use which you want.
|
14
|
+
|
15
|
+
## Commands
|
16
|
+
|
17
|
+
Command `norikra-client` have some subcommands.
|
18
|
+
|
19
|
+
norikra-client -h
|
20
|
+
Commands:
|
21
|
+
norikra-client event CMD ...ARGS # send/fetch events
|
22
|
+
norikra-client query CMD ...ARGS # manage queries
|
23
|
+
norikra-client target CMD ...ARGS # manage targets
|
24
|
+
norikra-client field CMD ...ARGS # manage table field/datatype definitions
|
25
|
+
|
26
|
+
norikra-client help [COMMAND] # Describe available commands or one specific command
|
27
|
+
|
28
|
+
Options:
|
29
|
+
[--host=HOST]
|
30
|
+
# Default: localhost
|
31
|
+
[--port=N]
|
32
|
+
# Default: 26571
|
33
|
+
|
34
|
+
Of course, you can see helps of each subcommands by `norikra-client SUBCMD help`
|
35
|
+
|
36
|
+
## Client Library
|
37
|
+
|
38
|
+
In your programs, you can operate Norikra with Norikra::Client instances.
|
39
|
+
|
40
|
+
require 'norikra-client'
|
41
|
+
|
42
|
+
client = Norikra::Client.new # connect to localhost:26571
|
43
|
+
# client = Norikra::Client.new(hostname, portnum)
|
44
|
+
|
45
|
+
## API
|
46
|
+
|
47
|
+
### Norikra::Client.new(host, port, options={})
|
48
|
+
|
49
|
+
Instanciate with server's hostname and port number.
|
50
|
+
|
51
|
+
require 'norikra-client'
|
52
|
+
client = Norikra::Client.new('norikra.server.local', 26571)
|
53
|
+
|
54
|
+
Timeout options are available:
|
55
|
+
|
56
|
+
client = Norikra::Client.new(
|
57
|
+
'norikra.server.local',
|
58
|
+
26571,
|
59
|
+
connect_timeout: 3, send_timeout: 1, receive_timeout: 3
|
60
|
+
)
|
61
|
+
|
62
|
+
### Norikra::Client#targets
|
63
|
+
|
64
|
+
Returns list of hashes, which contains `name`(string) and `auto_fields`(true/false) of each targets.
|
65
|
+
|
66
|
+
client.targets #=> [{'name' => "demo", 'auto_field' => false}, {'name' => "sample", 'auto_field' => true}]
|
67
|
+
|
68
|
+
### Norikra::Client#open(target_name, fields=nil, auto_field=true)
|
69
|
+
|
70
|
+
Create new target on norikra server, to put events and queries. Returns true or false (successfully created, or already exists).
|
71
|
+
|
72
|
+
client.open('sample') #=> true
|
73
|
+
|
74
|
+
Without fields, the specified target will be opend as 'lazy' mode, and actually opend when first input event arrived. Default field set will be determined at that time.
|
75
|
+
|
76
|
+
With field definitions, `#open()` creates the target and actually opens it immediately.
|
77
|
+
|
78
|
+
client.open('sample', {id:'integer', name:'string', age:'integer', email:'string'})
|
79
|
+
|
80
|
+
Fiels specified on `#open()` are configured as default field set.
|
81
|
+
|
82
|
+
`auto_field` means which fields in arrived events are automatically added on field list, or not. Default is true.
|
83
|
+
`auto_field: true` helps you to know What fields input event streams have. You can watch fields list on norikra, and write queries. But in cases your input streams have great many field name patterns, norikra returns long list of field names. That is not understandable. In these cases, you should specify `auto_field false`.
|
84
|
+
|
85
|
+
In spite of `auto_field false`, norikra server adds fields which are used in registered queries automatically.
|
86
|
+
|
87
|
+
### Norikra::Client#close
|
88
|
+
|
89
|
+
Close the specified target. Norikra server stops all queries with that target.
|
90
|
+
|
91
|
+
client.close('sample') #=> true
|
92
|
+
|
93
|
+
### Norikra::Client#modify(target_name, auto_field)
|
94
|
+
|
95
|
+
Modify the specified target configuration of auto_field.
|
96
|
+
|
97
|
+
client.modify('sample', false) #=> true
|
98
|
+
|
99
|
+
### Norikra::Client#queries
|
100
|
+
|
101
|
+
Returns a list of hashes which contains `name`, `group`, `expression` and `targets` of registered queries.
|
102
|
+
|
103
|
+
client.queries #=> [{'name' => 'q1', 'group' => nil, 'expression' => 'select count(*) from sample ...', 'targets' => ['sample']}]
|
104
|
+
|
105
|
+
Attributes of query:
|
106
|
+
* `name`: the name of query, that must be unique on norikra server
|
107
|
+
* `group`: the group name of query to be used in `sweep` call to fetch all events of queries with same `group` name (nil: default)
|
108
|
+
* `expression`: SQL expression
|
109
|
+
* `targets`: list of targets, which are referred in this SQL (2 or more targets by JOINs and SubQueries)
|
110
|
+
|
111
|
+
### Norikra::Client#register(query_name, query_group, query_expression)
|
112
|
+
|
113
|
+
Add query to norikra server. Query will be started immediately if all required targets/fields exists. If not, query actually starts when events arrived with all required fields.
|
114
|
+
|
115
|
+
client.register('q1', nil, 'select count(*) from sample.win:time_batch(1 min) where age >= 20') #=> true
|
116
|
+
|
117
|
+
### Norikra::Client#deregister(query_name)
|
118
|
+
|
119
|
+
Stop and remove specified query immediately.
|
120
|
+
|
121
|
+
client.deregister('q1') #=> true
|
122
|
+
|
123
|
+
### Norikra::Client#suspend(query_name)
|
124
|
+
|
125
|
+
Stop but does NOT remove specified query. Suspended queries are not recorded in stats file.
|
126
|
+
|
127
|
+
client.suspend('q1') #=> true
|
128
|
+
|
129
|
+
### Norikra::Client#resume(query_name)
|
130
|
+
|
131
|
+
Re-run specified query, which is suspended previously.
|
132
|
+
|
133
|
+
client.resume('q1') #=> true
|
134
|
+
|
135
|
+
### Norikra::Client#fields(target)
|
136
|
+
|
137
|
+
Returns the list of fields definitions, which contains `name`(string), `type`(string) and `optional`(true/false).
|
138
|
+
|
139
|
+
client.fields('sample') #=> [{'name' => 'id', 'type' => 'integer', 'optional' => false}, ...]
|
140
|
+
|
141
|
+
NOTE: Hashes and arrays are just returned as 'hash' and 'array'. Nested fields and these types are not returned to client.
|
142
|
+
|
143
|
+
### Norikra::Client#reserve(target, field_name, type)
|
144
|
+
|
145
|
+
Specify the type of field, which not arrived yet.
|
146
|
+
|
147
|
+
client.reserve('sample', 'phone', 'string') #=> true
|
148
|
+
|
149
|
+
This api is useful for fields, which has int data at a time, and has string data at the other time. Norikra determines fields' type at the first time with that field, and format input data by determined type.
|
150
|
+
If you want to parse as int for a field but that field's value may be either String or Integer, you should do `reserve()` to determine it as `int`.
|
151
|
+
|
152
|
+
### Norikra::Client#send(target, events)
|
153
|
+
|
154
|
+
Send list of input events into norikra server.
|
155
|
+
|
156
|
+
client.send('sample', [record1, record2, record3])
|
157
|
+
|
158
|
+
Members of events should be hash instance, which has a field at least.
|
159
|
+
Field values must be able to be serialized with MessagePack. Safe with String, Integer, Float, true/false/nil, Hash and Array.
|
160
|
+
|
161
|
+
### Norikra::Client#event(query_name)
|
162
|
+
|
163
|
+
Fetch events of specified query's output. Returns a list of `[time, event]`.
|
164
|
+
|
165
|
+
client.event('q1') #=> [ [1383297109,{"count(*)":50}], [1383297169,{"count(*)":37}], ... ]
|
166
|
+
|
167
|
+
The first value of returned pairs is the time of output event as unix time (seconds from epoch).
|
168
|
+
The second value is output record of query as Hash.
|
169
|
+
|
170
|
+
### Norikra::Client#sweep(query_group=nil)
|
171
|
+
|
172
|
+
Fetch all output events of queries of specified group. `query_group: nil` means default query group. Returns Hash instance like `'query_name' => [list of output (same as #event)]`
|
173
|
+
|
174
|
+
client.sweep() #=> for default group
|
175
|
+
client.sweep('my_secret_group')
|
176
|
+
|
177
|
+
## Versions
|
178
|
+
|
179
|
+
* v0.1.0:
|
180
|
+
* First release for production
|
181
|
+
|
182
|
+
## Copyright
|
183
|
+
|
184
|
+
* Copyright (c) 2013- TAGOMORI Satoshi (tagomoris)
|
185
|
+
* License
|
186
|
+
* MIT License
|
data/Rakefile
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "bundler/gem_tasks"
|
data/bin/norikra-client
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require 'norikra/client'
|
@@ -0,0 +1,91 @@
|
|
1
|
+
require "norikra/client/version"
|
2
|
+
|
3
|
+
require 'msgpack-rpc-over-http'
|
4
|
+
|
5
|
+
module Norikra
|
6
|
+
module RPC
|
7
|
+
class ClientError < MessagePack::RPCOverHTTP::RemoteError; end
|
8
|
+
class ServerError < MessagePack::RPCOverHTTP::RemoteError; end
|
9
|
+
class ServiceUnavailableError < MessagePack::RPCOverHTTP::RemoteError; end
|
10
|
+
end
|
11
|
+
|
12
|
+
class Client
|
13
|
+
RPC_DEFAULT_PORT = 26571
|
14
|
+
TIMEOUT_OPTIONS = [:connect_timeout, :send_timeout, :receive_timeout]
|
15
|
+
|
16
|
+
def initialize(host='localhost', port=RPC_DEFAULT_PORT, opts={})
|
17
|
+
@client = MessagePack::RPCOverHTTP::Client.new("http://#{host}:#{port}/")
|
18
|
+
|
19
|
+
@client.connect_timeout = opts[:connect_timeout] if opts.has_key?(:connect_timeout) && @client.respond_to?('connect_timeout='.to_sym)
|
20
|
+
@client.send_timeout = opts[:send_timeout] if opts.has_key?(:send_timeout) && @client.respond_to?('send_timeout='.to_sym)
|
21
|
+
@client.receive_timeout = opts[:receive_timeout] if opts.has_key?(:receive_timeout) && @client.respond_to?('receive_timeout='.to_sym)
|
22
|
+
end
|
23
|
+
|
24
|
+
def targets
|
25
|
+
@client.call(:targets) #=> {:name => "name", :auto_field => true}
|
26
|
+
end
|
27
|
+
|
28
|
+
def open(target, fields=nil, auto_field=true)
|
29
|
+
@client.call(:open, target, fields, auto_field)
|
30
|
+
end
|
31
|
+
|
32
|
+
def close(target)
|
33
|
+
@client.call(:close, target)
|
34
|
+
end
|
35
|
+
|
36
|
+
def modify(target, auto_field)
|
37
|
+
@client.call(:modify, target, auto_field)
|
38
|
+
end
|
39
|
+
|
40
|
+
def queries
|
41
|
+
@client.call(:queries)
|
42
|
+
end
|
43
|
+
|
44
|
+
def register(query_name, query_group, query_expression)
|
45
|
+
@client.call(:register, query_name, query_group, query_expression)
|
46
|
+
end
|
47
|
+
|
48
|
+
def deregister(query_name)
|
49
|
+
@client.call(:deregister, query_name)
|
50
|
+
end
|
51
|
+
|
52
|
+
def suspend(query_name)
|
53
|
+
@client.call(:suspend, query_name)
|
54
|
+
end
|
55
|
+
|
56
|
+
def resume(query_name)
|
57
|
+
@client.call(:resume, query_name)
|
58
|
+
end
|
59
|
+
|
60
|
+
def fields(target)
|
61
|
+
@client.call(:fields, target)
|
62
|
+
end
|
63
|
+
|
64
|
+
def reserve(target, field, type)
|
65
|
+
@client.call(:reserve, target, field, type)
|
66
|
+
end
|
67
|
+
|
68
|
+
def send(target, events)
|
69
|
+
@client.call(:send, target, events)
|
70
|
+
end
|
71
|
+
|
72
|
+
# [ [time, event], ... ]
|
73
|
+
def event(query_name)
|
74
|
+
@client.call(:event, query_name)
|
75
|
+
end
|
76
|
+
|
77
|
+
# [ [time, event], ... ]
|
78
|
+
def see(query_name)
|
79
|
+
@client.call(:see, query_name)
|
80
|
+
end
|
81
|
+
|
82
|
+
# {'query_name' => [ [time, event], ... ]}
|
83
|
+
def sweep(query_group=nil)
|
84
|
+
@client.call(:sweep, query_group)
|
85
|
+
end
|
86
|
+
|
87
|
+
def logs
|
88
|
+
@client.call(:logs)
|
89
|
+
end
|
90
|
+
end
|
91
|
+
end
|
@@ -0,0 +1,302 @@
|
|
1
|
+
require 'thor'
|
2
|
+
require 'norikra/client'
|
3
|
+
|
4
|
+
require 'norikra/client/cli/parser'
|
5
|
+
require 'norikra/client/cli/formatter'
|
6
|
+
|
7
|
+
class Norikra::Client
|
8
|
+
module CLIUtil
|
9
|
+
def client(options)
|
10
|
+
Norikra::Client.new(options[:host], options[:port])
|
11
|
+
end
|
12
|
+
def wrap
|
13
|
+
begin
|
14
|
+
yield
|
15
|
+
rescue Norikra::RPC::ClientError => e
|
16
|
+
puts "Failed: " + e.message
|
17
|
+
rescue Norikra::RPC::ServerError => e
|
18
|
+
puts "ERROR on norikra server: " + e.message
|
19
|
+
puts " For more details, see norikra server's logs"
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
class Target < Thor
|
25
|
+
include Norikra::Client::CLIUtil
|
26
|
+
|
27
|
+
desc "list", "show list of targets"
|
28
|
+
option :simple, :type => :boolean, :default => false, :desc => "suppress header/footer", :aliases => "-s"
|
29
|
+
def list
|
30
|
+
wrap do
|
31
|
+
puts ["TARGET","AUTO_FIELD"].join("\t") unless options[:simple]
|
32
|
+
targets = client(parent_options).targets
|
33
|
+
targets.each do |t|
|
34
|
+
puts [t['name'], t['auto_field']].join("\t")
|
35
|
+
end
|
36
|
+
puts "#{targets.size} targets found." unless options[:simple]
|
37
|
+
end
|
38
|
+
end
|
39
|
+
|
40
|
+
desc "open TARGET [fieldname1:type1 [fieldname2:type2 [fieldname3:type3] ...]]", "create new target (and define its fields)"
|
41
|
+
option :suppress_auto_field, :type => :boolean, :default => false, :desc => "suppress to define fields automatically", :aliases => "-x"
|
42
|
+
def open(target, *field_defs)
|
43
|
+
fields = nil
|
44
|
+
if field_defs.size > 0
|
45
|
+
fields = {}
|
46
|
+
field_defs.each do |str|
|
47
|
+
fname,ftype = str.split(':')
|
48
|
+
fields[fname] = ftype
|
49
|
+
end
|
50
|
+
end
|
51
|
+
auto_field = (not options[:suppress_auto_field])
|
52
|
+
|
53
|
+
wrap do
|
54
|
+
client(parent_options).open(target, fields, auto_field)
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
desc "close TARGET", "close existing target and all its queries"
|
59
|
+
def close(target)
|
60
|
+
wrap do
|
61
|
+
client(parent_options).close(target)
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
desc "modify TARGET BOOL_VALUE", "modify target to do define fields automatically or not"
|
66
|
+
def modify(target, val)
|
67
|
+
auto_field = ['yes','true','auto'].include?(val.downcase)
|
68
|
+
wrap do
|
69
|
+
client(parent_options).modify(target, auto_field)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
class Query < Thor
|
75
|
+
include Norikra::Client::CLIUtil
|
76
|
+
|
77
|
+
desc "list", "show list of queries"
|
78
|
+
option :simple, :type => :boolean, :default => false, :desc => "suppress header/footer", :aliases => "-s"
|
79
|
+
def list
|
80
|
+
wrap do
|
81
|
+
puts ["NAME", "GROUP", "TARGETS", "SUSPENDED", "QUERY"].join("\t") unless options[:simple]
|
82
|
+
queries = client(parent_options).queries
|
83
|
+
queries.sort{|a,b| (a['targets'].first <=> b['targets'].first).nonzero? || a['name'] <=> b['name']}.each do |q|
|
84
|
+
puts [
|
85
|
+
q['name'],
|
86
|
+
(q['group'] || 'default'),
|
87
|
+
q['targets'].join(','),
|
88
|
+
q['suspended'].to_s,
|
89
|
+
q['expression'].split("\n").map(&:strip).join(" ")
|
90
|
+
].join("\t")
|
91
|
+
end
|
92
|
+
puts "#{queries.size} queries found." unless options[:simple]
|
93
|
+
end
|
94
|
+
end
|
95
|
+
|
96
|
+
desc "add QUERY_NAME QUERY_EXPRESSION", "register a query"
|
97
|
+
option :group, :type => :string, :default => nil, :desc => "query group for sweep/listen (default: null)", :aliases => "-g"
|
98
|
+
def add(query_name, expression)
|
99
|
+
wrap do
|
100
|
+
client(parent_options).register(query_name, options[:group], expression)
|
101
|
+
end
|
102
|
+
end
|
103
|
+
|
104
|
+
desc "remove QUERY_NAME", "deregister a query"
|
105
|
+
def remove(query_name)
|
106
|
+
wrap do
|
107
|
+
client(parent_options).deregister(query_name)
|
108
|
+
end
|
109
|
+
end
|
110
|
+
|
111
|
+
desc "suspend QUERY_NAME", "specify to stop (but not removed)"
|
112
|
+
def suspend(query_name)
|
113
|
+
wrap do
|
114
|
+
client(parent_options).suspend(query_name)
|
115
|
+
end
|
116
|
+
end
|
117
|
+
|
118
|
+
desc "resume QUERY_NAME", "specify to re-run query suspended before"
|
119
|
+
def resume(query_name)
|
120
|
+
wrap do
|
121
|
+
client(parent_options).resume(query_name)
|
122
|
+
end
|
123
|
+
end
|
124
|
+
end
|
125
|
+
|
126
|
+
class Field < Thor
|
127
|
+
include Norikra::Client::CLIUtil
|
128
|
+
|
129
|
+
desc "list TARGET", "show list of field definitions of specified target"
|
130
|
+
option :simple, :type => :boolean, :default => false, :desc => "suppress header/footer", :aliases => "-s"
|
131
|
+
def list(target)
|
132
|
+
wrap do
|
133
|
+
puts "FIELD\tTYPE\tOPTIONAL" unless options[:simple]
|
134
|
+
fields = client(parent_options).fields(target)
|
135
|
+
fields.each do |f|
|
136
|
+
puts "#{f['name']}\t#{f['type']}\t#{f['optional']}"
|
137
|
+
end
|
138
|
+
puts "#{fields.size} fields found." unless options[:simple]
|
139
|
+
end
|
140
|
+
end
|
141
|
+
|
142
|
+
desc "add TARGET FIELDNAME TYPE", "reserve fieldname and its type of target"
|
143
|
+
def add(target, field, type)
|
144
|
+
wrap do
|
145
|
+
client(parent_options).reserve(target, field, type)
|
146
|
+
end
|
147
|
+
end
|
148
|
+
end
|
149
|
+
|
150
|
+
class Event < Thor
|
151
|
+
include Norikra::Client::CLIUtil
|
152
|
+
|
153
|
+
desc "send TARGET", "send data into targets"
|
154
|
+
option :format, :type => :string, :default => 'json', :desc => "format of input data per line of stdin [json(default), ltsv]"
|
155
|
+
option :batch_size, :type => :numeric, :default => 10000, :desc => "records sent in once transferring (default: 10000)"
|
156
|
+
def send(target)
|
157
|
+
client = client(parent_options)
|
158
|
+
parser = parser(options[:format])
|
159
|
+
buffer = []
|
160
|
+
$stdin.each_line do |line|
|
161
|
+
buffer.push(parser.parse(line))
|
162
|
+
if buffer.size >= options[:batch_size]
|
163
|
+
client.send(target, buffer)
|
164
|
+
buffer = []
|
165
|
+
end
|
166
|
+
end
|
167
|
+
|
168
|
+
wrap do
|
169
|
+
client.send(target, buffer) if buffer.size > 0
|
170
|
+
end
|
171
|
+
end
|
172
|
+
|
173
|
+
desc "fetch QUERY_NAME", "fetch events from specified query"
|
174
|
+
option :format, :type => :string, :default => 'json', :desc => "format of output data per line of stdout [json(default), ltsv]"
|
175
|
+
option :time_key, :type => :string, :default => 'time', :desc => "output key name for event time (default: time)"
|
176
|
+
option :time_format, :type => :string, :default => '%Y/%m/%d %H:%M:%S', :desc => "output time format (default: '2013/05/14 17:57:59')"
|
177
|
+
def fetch(query_name)
|
178
|
+
formatter = formatter(options[:format])
|
179
|
+
time_formatter = lambda{|t| Time.at(t).strftime(options[:time_format])}
|
180
|
+
|
181
|
+
wrap do
|
182
|
+
client(parent_options).event(query_name).each do |time,event|
|
183
|
+
event = {options[:time_key] => Time.at(time).strftime(options[:time_format])}.merge(event)
|
184
|
+
puts formatter.format(event)
|
185
|
+
end
|
186
|
+
end
|
187
|
+
end
|
188
|
+
|
189
|
+
desc "see QUERY_NAME", "see events of specified query, but not delete"
|
190
|
+
option :format, :type => :string, :default => 'json', :desc => "format of output data per line of stdout [json(default), ltsv]"
|
191
|
+
option :time_key, :type => :string, :default => 'time', :desc => "output key name for event time (default: time)"
|
192
|
+
option :time_format, :type => :string, :default => '%Y/%m/%d %H:%M:%S', :desc => "output time format (default: '2013/05/14 17:57:59')"
|
193
|
+
def see(query_name)
|
194
|
+
formatter = formatter(options[:format])
|
195
|
+
time_formatter = lambda{|t| Time.at(t).strftime(options[:time_format])}
|
196
|
+
|
197
|
+
wrap do
|
198
|
+
client(parent_options).see(query_name).each do |time,event|
|
199
|
+
event = {options[:time_key] => Time.at(time).strftime(options[:time_format])}.merge(event)
|
200
|
+
puts formatter.format(event)
|
201
|
+
end
|
202
|
+
end
|
203
|
+
end
|
204
|
+
|
205
|
+
desc "sweep [query_group_name]", "fetch all output events of all queries of default (or specified) query group"
|
206
|
+
option :format, :type => :string, :default => 'json', :desc => "format of output data per line of stdout [json(default), ltsv]"
|
207
|
+
option :query_name_key, :type => :string, :default => 'query', :desc => "output key name for query name (default: query)"
|
208
|
+
option :time_key, :type => :string, :default => 'time', :desc => "output key name for event time (default: time)"
|
209
|
+
option :time_format, :type => :string, :default => '%Y/%m/%d %H:%M:%S', :desc => "output time format (default: '2013/05/14 17:57:59')"
|
210
|
+
def sweep(query_group=nil)
|
211
|
+
wrap do
|
212
|
+
formatter = formatter(options[:format])
|
213
|
+
time_formatter = lambda{|t| Time.at(t).strftime(options[:time_format])}
|
214
|
+
|
215
|
+
data = client(parent_options).sweep(query_group)
|
216
|
+
|
217
|
+
data.keys.sort.each do |queryname|
|
218
|
+
events = data[queryname]
|
219
|
+
events.each do |time,event|
|
220
|
+
event = {
|
221
|
+
options[:time_key] => Time.at(time).strftime(options[:time_format]),
|
222
|
+
options[:query_name_key] => queryname,
|
223
|
+
}.merge(event)
|
224
|
+
puts formatter.format(event)
|
225
|
+
end
|
226
|
+
end
|
227
|
+
end
|
228
|
+
end
|
229
|
+
end
|
230
|
+
|
231
|
+
class Admin < Thor
|
232
|
+
include Norikra::Client::CLIUtil
|
233
|
+
|
234
|
+
desc "stats", "dump stats json: same with norikra server's --stats option"
|
235
|
+
def stats
|
236
|
+
opt = parent_options
|
237
|
+
client = client(parent_options)
|
238
|
+
|
239
|
+
targets = []
|
240
|
+
queries = []
|
241
|
+
|
242
|
+
wrap do
|
243
|
+
queries = client.queries()
|
244
|
+
|
245
|
+
client.targets().each do |t|
|
246
|
+
fields = {}
|
247
|
+
client.fields(t['name']).each do |f|
|
248
|
+
next if f['type'] == 'hash' || f['type'] == 'array'
|
249
|
+
fields[f['name']] = f
|
250
|
+
end
|
251
|
+
targets.push( { "name" => t['name'], "fields" => fields, "auto_field" => t['auto_field'] } )
|
252
|
+
end
|
253
|
+
end
|
254
|
+
|
255
|
+
require 'json'
|
256
|
+
|
257
|
+
puts JSON.pretty_generate({
|
258
|
+
"threads" => {
|
259
|
+
"engine" => { "inbound" => {}, "outbound" => {}, "route_exec" => {}, "timer_exec" => {} },
|
260
|
+
"rpc" => {},
|
261
|
+
"web" => {},
|
262
|
+
},
|
263
|
+
"log" => {},
|
264
|
+
"targets" => targets,
|
265
|
+
"queries" => queries,
|
266
|
+
})
|
267
|
+
end
|
268
|
+
|
269
|
+
desc "logs", "get and print Norikra server logs"
|
270
|
+
def logs
|
271
|
+
opt = parent_options
|
272
|
+
client = client(parent_options)
|
273
|
+
wrap do
|
274
|
+
client.logs().each do |time, level, line|
|
275
|
+
puts "#{time} [#{level}] #{line}"
|
276
|
+
end
|
277
|
+
end
|
278
|
+
end
|
279
|
+
end
|
280
|
+
|
281
|
+
class CLI < Thor
|
282
|
+
include Norikra::Client::CLIUtil
|
283
|
+
|
284
|
+
class_option :host, :type => :string, :default => 'localhost'
|
285
|
+
class_option :port, :type => :numeric, :default => 26571
|
286
|
+
|
287
|
+
desc "target CMD ...ARGS", "manage targets"
|
288
|
+
subcommand "target", Target
|
289
|
+
|
290
|
+
desc "field CMD ...ARGS", "manage target field/datatype definitions"
|
291
|
+
subcommand "field", Field
|
292
|
+
|
293
|
+
desc "query CMD ...ARGS", "manage queries"
|
294
|
+
subcommand "query", Query
|
295
|
+
|
296
|
+
desc "event CMD ...ARGS", "send/fetch events"
|
297
|
+
subcommand "event", Event
|
298
|
+
|
299
|
+
desc "admin CMD ...ARGS", "norikra server administrations"
|
300
|
+
subcommand "admin", Admin
|
301
|
+
end
|
302
|
+
end
|
@@ -0,0 +1,35 @@
|
|
1
|
+
module Norikra::Client::CLIUtil
|
2
|
+
def formatter(format, *args)
|
3
|
+
format ||= 'json'
|
4
|
+
case format
|
5
|
+
when /^json$/i
|
6
|
+
require 'json'
|
7
|
+
Formatter::JSON.new(*args)
|
8
|
+
when /^ltsv$/i
|
9
|
+
require 'ltsv'
|
10
|
+
Formatter::LTSV.new(*args)
|
11
|
+
else
|
12
|
+
raise ArgumentError, "unknown format name: #{format}"
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
module Formatter
|
17
|
+
class JSON
|
18
|
+
def initialize(*args)
|
19
|
+
require 'json'
|
20
|
+
end
|
21
|
+
def format(obj)
|
22
|
+
::JSON.dump(obj)
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
class LTSV
|
27
|
+
def initialize(*args)
|
28
|
+
require 'ltsv'
|
29
|
+
end
|
30
|
+
def format(obj)
|
31
|
+
::LTSV.dump(obj)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
@@ -0,0 +1,35 @@
|
|
1
|
+
module Norikra::Client::CLIUtil
|
2
|
+
def parser(format, *args)
|
3
|
+
format ||= 'json'
|
4
|
+
case format
|
5
|
+
when /^json$/i
|
6
|
+
require 'json'
|
7
|
+
Parser::JSON.new(*args)
|
8
|
+
when /^ltsv$/i
|
9
|
+
require 'ltsv'
|
10
|
+
Parser::LTSV.new(*args)
|
11
|
+
else
|
12
|
+
raise ArgumentError, "unknown format name: #{format}"
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
module Parser
|
17
|
+
class JSON
|
18
|
+
def initialize(*args)
|
19
|
+
require 'json'
|
20
|
+
end
|
21
|
+
def parse(line)
|
22
|
+
::JSON.parse(line.chop)
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
class LTSV
|
27
|
+
def initialize(*args)
|
28
|
+
require 'ltsv'
|
29
|
+
end
|
30
|
+
def parse(line)
|
31
|
+
::LTSV.parse_line(line.chop, {:symbolize_keys => false})
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'norikra/client/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "norikra-client"
|
8
|
+
spec.version = Norikra::Client::VERSION
|
9
|
+
|
10
|
+
if /java/ =~ RUBY_PLATFORM
|
11
|
+
spec.platform = "java"
|
12
|
+
end
|
13
|
+
|
14
|
+
spec.authors = ["TAGOMORI Satoshi"]
|
15
|
+
spec.email = ["tagomoris@gmail.com"]
|
16
|
+
spec.description = %q{Client commands and libraries for Norikra}
|
17
|
+
spec.summary = %q{Client commands and libraries for Norikra}
|
18
|
+
spec.homepage = "https://github.com/norikra/norikra-client-ruby"
|
19
|
+
spec.license = "APLv2"
|
20
|
+
|
21
|
+
spec.files = `git ls-files`.split($/)
|
22
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
23
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
24
|
+
spec.require_paths = ["lib"]
|
25
|
+
|
26
|
+
spec.add_runtime_dependency "msgpack-rpc-over-http", ">= 0.0.7"
|
27
|
+
spec.add_runtime_dependency "thor"
|
28
|
+
spec.add_runtime_dependency "ltsv"
|
29
|
+
|
30
|
+
spec.add_development_dependency "bundler", "~> 1.3"
|
31
|
+
spec.add_development_dependency "rake"
|
32
|
+
end
|
metadata
ADDED
@@ -0,0 +1,129 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: norikra-client
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.3.0
|
5
|
+
platform: java
|
6
|
+
authors:
|
7
|
+
- TAGOMORI Satoshi
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2015-04-24 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
15
|
+
requirements:
|
16
|
+
- - '>='
|
17
|
+
- !ruby/object:Gem::Version
|
18
|
+
version: 0.0.7
|
19
|
+
name: msgpack-rpc-over-http
|
20
|
+
prerelease: false
|
21
|
+
type: :runtime
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - '>='
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 0.0.7
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
29
|
+
requirements:
|
30
|
+
- - '>='
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '0'
|
33
|
+
name: thor
|
34
|
+
prerelease: false
|
35
|
+
type: :runtime
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - '>='
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
43
|
+
requirements:
|
44
|
+
- - '>='
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
version: '0'
|
47
|
+
name: ltsv
|
48
|
+
prerelease: false
|
49
|
+
type: :runtime
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - '>='
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
requirement: !ruby/object:Gem::Requirement
|
57
|
+
requirements:
|
58
|
+
- - ~>
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: '1.3'
|
61
|
+
name: bundler
|
62
|
+
prerelease: false
|
63
|
+
type: :development
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ~>
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '1.3'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
requirement: !ruby/object:Gem::Requirement
|
71
|
+
requirements:
|
72
|
+
- - '>='
|
73
|
+
- !ruby/object:Gem::Version
|
74
|
+
version: '0'
|
75
|
+
name: rake
|
76
|
+
prerelease: false
|
77
|
+
type: :development
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - '>='
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0'
|
83
|
+
description: Client commands and libraries for Norikra
|
84
|
+
email:
|
85
|
+
- tagomoris@gmail.com
|
86
|
+
executables:
|
87
|
+
- norikra-client
|
88
|
+
extensions: []
|
89
|
+
extra_rdoc_files: []
|
90
|
+
files:
|
91
|
+
- .gitignore
|
92
|
+
- Gemfile
|
93
|
+
- LICENSE.txt
|
94
|
+
- README.md
|
95
|
+
- Rakefile
|
96
|
+
- bin/norikra-client
|
97
|
+
- lib/norikra-client.rb
|
98
|
+
- lib/norikra/client.rb
|
99
|
+
- lib/norikra/client/cli.rb
|
100
|
+
- lib/norikra/client/cli/formatter.rb
|
101
|
+
- lib/norikra/client/cli/parser.rb
|
102
|
+
- lib/norikra/client/version.rb
|
103
|
+
- norikra-client.gemspec
|
104
|
+
homepage: https://github.com/norikra/norikra-client-ruby
|
105
|
+
licenses:
|
106
|
+
- APLv2
|
107
|
+
metadata: {}
|
108
|
+
post_install_message:
|
109
|
+
rdoc_options: []
|
110
|
+
require_paths:
|
111
|
+
- lib
|
112
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
113
|
+
requirements:
|
114
|
+
- - '>='
|
115
|
+
- !ruby/object:Gem::Version
|
116
|
+
version: '0'
|
117
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
118
|
+
requirements:
|
119
|
+
- - '>='
|
120
|
+
- !ruby/object:Gem::Version
|
121
|
+
version: '0'
|
122
|
+
requirements: []
|
123
|
+
rubyforge_project:
|
124
|
+
rubygems_version: 2.4.5
|
125
|
+
signing_key:
|
126
|
+
specification_version: 4
|
127
|
+
summary: Client commands and libraries for Norikra
|
128
|
+
test_files: []
|
129
|
+
has_rdoc:
|