sherlog-holmes 0.3.2 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +27 -0
- data/bin/sherlog +14 -5
- data/changelog.md +11 -0
- data/conf/patterns/java.yml +9 -0
- data/lib/sherlog_holmes.rb +6 -5
- data/lib/sherlog_holmes/entry.rb +14 -7
- data/lib/sherlog_holmes/filter.rb +7 -0
- data/lib/sherlog_holmes/parser.rb +7 -7
- data/lib/sherlog_holmes/version.rb +1 -1
- metadata +3 -3
- data/conf/patterns/jboss.yml +0 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7de344c1bb4ad99a2e4cb1369d9781196242f877
|
4
|
+
data.tar.gz: d71b629b448d315815dd00d9fcc27cdc56c3b30c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5e8d2263e831c1f7cb06ae3101465e448594f8c5aa4e44651733a7dc73cefaae0edf35a8cefa89e8a05689f1215490f5c2bb06dde28405e62f775ef29f2f7df9
|
7
|
+
data.tar.gz: 02a4e999969b868d738056068b52fd67466fa8ed0560c95ce3c1fb0cb6f348fc2469fae1e8be5157eefcf3e7c75475b918de300b3a47ad5b6df1f091571caafe
|
data/README.md
CHANGED
@@ -51,6 +51,23 @@ jboss:
|
|
51
51
|
|
52
52
|
The configuration should contain a unique id and at least a pattern for the log **entry**. Place you configuration file in a `*.yml` file inside your `$HOME/.sherlog/patterns` directory and you're ready to go!
|
53
53
|
|
54
|
+
### Custom Attributes
|
55
|
+
|
56
|
+
Sherlog allows you to define custom attributes by using named capture groups. Any capture group name that differs from the main fields will be stored as a custom attribute in the entry object.
|
57
|
+
|
58
|
+
### Configuration Inheritance
|
59
|
+
|
60
|
+
You can create a base configuration and then override some values to create another one. In this case, you need to specify the parent configuration with the `from` key:
|
61
|
+
|
62
|
+
```yaml
|
63
|
+
base.java:
|
64
|
+
exception: (?<exception>\w+(\.\w+)+(Exception|Error))
|
65
|
+
stacktrace: ^(\s+at)|(Caused by\:)|(\s+\.{3}\s\d+\smore)
|
66
|
+
jboss:
|
67
|
+
from: base.java
|
68
|
+
entry: (?<time>[0-9,.:]+)\s+(?<level>\w+)\s+\[(?<category>\S+)\]\s\((?<origin>[^)]+)\)?\s?(?<message>.+)
|
69
|
+
```
|
70
|
+
|
54
71
|
## Usage
|
55
72
|
|
56
73
|
Shelog Holmes provides the command line tool `sherlog`. You can use this to pass a log, the filters you need to apply and the process that needs to be executed (like showing the filtered entries or counting the exceptions):
|
@@ -97,6 +114,14 @@ This will filter entries using the exception field. You can use the wildcard `*`
|
|
97
114
|
|
98
115
|
This will filter entries with exceptions, regardless the kind.
|
99
116
|
|
117
|
+
`-f NAME, --field NAME`
|
118
|
+
|
119
|
+
This will filter entries using custom attributes found in named capture groups. This parameter specifies the custom attribute name. Use it with `-v | --value` for defining the expression.
|
120
|
+
|
121
|
+
`-v EXPRESSION, --value EXPRESSION`
|
122
|
+
|
123
|
+
Specifies the expression to use with the last `-f | --field` parameter. The wildcard `*` is accepted here.
|
124
|
+
|
100
125
|
### Logical Options
|
101
126
|
|
102
127
|
`--and`
|
@@ -153,7 +178,9 @@ $ sherlog --count levels,categories log-file.log
|
|
153
178
|
|
154
179
|
Currently, Sherlog has the following patterns:
|
155
180
|
|
181
|
+
- `base.java`: base pattern for Java outputs (contains patterns for exceptions and stacktraces only)
|
156
182
|
- `jboss`: matches Wildfly | EAP logs
|
183
|
+
- `jboss.fuse`: matches JBoss Fuse logs
|
157
184
|
|
158
185
|
## License
|
159
186
|
|
data/bin/sherlog
CHANGED
@@ -58,7 +58,7 @@ def print_table(name, data)
|
|
58
58
|
puts
|
59
59
|
end
|
60
60
|
|
61
|
-
@opts.banner = 'Usage: sherlog [options]
|
61
|
+
@opts.banner = 'Usage: sherlog [options] [logfile ...]'
|
62
62
|
|
63
63
|
@opts.separator "\n Config Options\n".bold.white
|
64
64
|
|
@@ -100,6 +100,14 @@ end
|
|
100
100
|
add_filter Filter::exceptions
|
101
101
|
end
|
102
102
|
|
103
|
+
@opts.on '-f NAME', '--field NAME', 'Sets the custom attribute filter name' do |name|
|
104
|
+
@field = name
|
105
|
+
end
|
106
|
+
|
107
|
+
@opts.on '-v EXPRESSION', '--value EXPRESSION', 'Sets the custom attribute filter expression' do |expression|
|
108
|
+
add_filter Filter::custom_attribute(@field, expression)
|
109
|
+
end
|
110
|
+
|
103
111
|
@opts.separator "\n Logical Options\n".bold.white
|
104
112
|
|
105
113
|
@opts.on '--and', 'Sets the next filter to use the AND operator' do
|
@@ -147,7 +155,6 @@ end
|
|
147
155
|
|
148
156
|
@listeners[:print].hide_stacktrace if @no_stacktrace and @listeners[:print]
|
149
157
|
|
150
|
-
file = ARGV.first
|
151
158
|
parser = Sherlog.parser @type if @type
|
152
159
|
parser ||= Parser::new
|
153
160
|
parser.filter @filter
|
@@ -155,12 +162,14 @@ parser.filter @filter
|
|
155
162
|
parser.on_new_entry listener
|
156
163
|
end
|
157
164
|
begin
|
158
|
-
if
|
159
|
-
parser.parse file
|
160
|
-
else
|
165
|
+
if ARGV.empty?
|
161
166
|
ARGF.each_line do |line|
|
162
167
|
parser.parse line.chomp
|
163
168
|
end
|
169
|
+
else
|
170
|
+
ARGV.each do |file|
|
171
|
+
parser.parse file
|
172
|
+
end
|
164
173
|
end
|
165
174
|
@groups.each do |group|
|
166
175
|
print_table group, @listeners[:count].send(group).sort_by { |name, count| -count }
|
data/changelog.md
CHANGED
@@ -1,5 +1,16 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## v0.4.0
|
4
|
+
|
5
|
+
- Added support for multiple log files
|
6
|
+
- Added support for custom attributes using capture groups
|
7
|
+
- Added JBoss Fuse pattern
|
8
|
+
- Added support for inheritance of patterns
|
9
|
+
|
10
|
+
## v0.3.3
|
11
|
+
|
12
|
+
- Apply filters before the entire entry was parsed
|
13
|
+
|
3
14
|
## v0.3.2
|
4
15
|
|
5
16
|
- Recognize 'Fault' in JBoss exception pattern
|
@@ -0,0 +1,9 @@
|
|
1
|
+
base.java:
|
2
|
+
exception: (?<exception>\w+(\.\w+)+(Exception|Error|Fault))
|
3
|
+
stacktrace: ^(\s+at)|(Caused by\:)|(\s+\.{3}\s\d+\smore)
|
4
|
+
jboss:
|
5
|
+
from: base.java
|
6
|
+
entry: (?<time>(?<date>\d{2,4}-\d{2}-\d{2,4}\s)?(\d{2}:\d{2}:\d{2},\d{3}))\s+(?<level>\w+)\s+\[(?<category>\S+)\]\s\((?<origin>[^)]+)\)?\s?(?<message>.+)
|
7
|
+
jboss.fuse:
|
8
|
+
from: base.java
|
9
|
+
entry: (?<time>(?<date>\d{2,4}-\d{2}-\d{2,4}\s)?(\d{2}:\d{2}:\d{2},\d{3}))\s+\|\s*(?<level>\w+)\s*\|\s*(?<origin>[^|]+)\s*\|\s*(?<category>[^|]+)\s*\|\s*(?<bundle>(?<bundle_id>\d+) - (?<bundle_name>\S+) - (?<bundle_version>\S+))\s*\|\s*(?<message>.+)
|
data/lib/sherlog_holmes.rb
CHANGED
@@ -38,11 +38,12 @@ module Sherlog
|
|
38
38
|
def self.load_patterns(file)
|
39
39
|
patterns = YAML::load_file file
|
40
40
|
patterns.each do |id, config|
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
41
|
+
pattern_config = {}
|
42
|
+
pattern_config[:entry] = Regexp::new(config['entry']) if config['entry']
|
43
|
+
pattern_config[:exception] = Regexp::new(config['exception']) if config['exception']
|
44
|
+
pattern_config[:stacktrace] = Regexp::new(config['stacktrace']) if config['stacktrace']
|
45
|
+
pattern_config = PATTERNS[config['from'].to_sym].merge pattern_config if config['from']
|
46
|
+
PATTERNS[id.to_sym] = pattern_config
|
46
47
|
end
|
47
48
|
end
|
48
49
|
|
data/lib/sherlog_holmes/entry.rb
CHANGED
@@ -26,15 +26,18 @@ module Sherlog
|
|
26
26
|
attr_accessor :time, :level, :category, :origin, :message, :exceptions, :stacktrace, :raw_content
|
27
27
|
|
28
28
|
def initialize(params = {})
|
29
|
-
|
30
|
-
@
|
31
|
-
@
|
32
|
-
@
|
33
|
-
@
|
34
|
-
@
|
35
|
-
@
|
29
|
+
params = params.dup
|
30
|
+
@time = params.delete :time if params[:time]
|
31
|
+
@level = params.delete :level if params[:level]
|
32
|
+
@category = params.delete :category if params[:category]
|
33
|
+
@origin = params.delete :origin if params[:origin]
|
34
|
+
@message = params.delete :message if params[:message]
|
35
|
+
@raw_content = params.delete :raw_content if params[:raw_content]
|
36
|
+
@exceptions = [params.delete(:exception)] if params[:exception]
|
37
|
+
@exceptions ||= params.delete(:exceptions) if params[:exceptions]
|
36
38
|
@exceptions ||= []
|
37
39
|
@stacktrace = []
|
40
|
+
@custom_attributes = params
|
38
41
|
end
|
39
42
|
|
40
43
|
def exception?
|
@@ -49,6 +52,10 @@ module Sherlog
|
|
49
52
|
@message << $/ << line
|
50
53
|
end
|
51
54
|
|
55
|
+
def [](custom_attribute)
|
56
|
+
@custom_attributes[custom_attribute.to_s] or @custom_attributes[custom_attribute.to_sym]
|
57
|
+
end
|
58
|
+
|
52
59
|
def to_s
|
53
60
|
format = []
|
54
61
|
params = []
|
@@ -57,6 +57,7 @@ module Sherlog
|
|
57
57
|
end
|
58
58
|
|
59
59
|
def self.expression(expression)
|
60
|
+
expression = expression.to_s
|
60
61
|
Filter::new do |object|
|
61
62
|
wildcard_at_start = expression.start_with? '*'
|
62
63
|
wildcard_at_end = expression.end_with? '*'
|
@@ -110,6 +111,12 @@ module Sherlog
|
|
110
111
|
end
|
111
112
|
end
|
112
113
|
|
114
|
+
def self.custom_attribute(attribute_name, expression)
|
115
|
+
Filter::new do |entry|
|
116
|
+
expression(expression).accept? entry[attribute_name]
|
117
|
+
end
|
118
|
+
end
|
119
|
+
|
113
120
|
end
|
114
121
|
|
115
122
|
end
|
@@ -60,12 +60,12 @@ module Sherlog
|
|
60
60
|
foreach input do |line|
|
61
61
|
try_guess_pattern line unless @patterns[:entry]
|
62
62
|
if @patterns[:entry] =~ line
|
63
|
-
entry_data = Hash[Regexp.last_match.names.map { |k| [k.to_sym, Regexp.last_match[k]] }]
|
64
|
-
notify entry
|
63
|
+
entry_data = Hash[Regexp.last_match.names.map { |k| [k.to_sym, Regexp.last_match[k].to_s.strip] }]
|
64
|
+
# notify the last entry parsed
|
65
|
+
notify entry if entry and @filter.accept? entry
|
65
66
|
entry = Entry::new entry_data
|
66
67
|
entry.raw_content = line.chomp
|
67
68
|
entry.exceptions << Regexp.last_match[:exception] if @patterns[:exception] =~ entry.message
|
68
|
-
entry = nil unless @filter.accept? entry
|
69
69
|
else
|
70
70
|
if entry
|
71
71
|
if entry.exception? and @patterns[:stacktrace] =~ line
|
@@ -78,7 +78,8 @@ module Sherlog
|
|
78
78
|
end
|
79
79
|
end
|
80
80
|
end
|
81
|
-
notify entry
|
81
|
+
# notify the last entry parsed
|
82
|
+
notify entry if entry and @filter.accept? entry
|
82
83
|
end
|
83
84
|
|
84
85
|
private
|
@@ -100,10 +101,9 @@ module Sherlog
|
|
100
101
|
|
101
102
|
def try_guess_pattern(line)
|
102
103
|
key, patterns = Sherlog.loaded_patterns.find do |key, patterns|
|
103
|
-
patterns[:entry].match line
|
104
|
+
patterns[:entry].match line if patterns[:entry]
|
104
105
|
end
|
105
|
-
patterns
|
106
|
-
@patterns.merge! patterns
|
106
|
+
@patterns.merge! patterns if patterns
|
107
107
|
end
|
108
108
|
|
109
109
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sherlog-holmes
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ataxexe
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-12-
|
11
|
+
date: 2015-12-29 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: yummi
|
@@ -93,7 +93,7 @@ files:
|
|
93
93
|
- Rakefile
|
94
94
|
- bin/sherlog
|
95
95
|
- changelog.md
|
96
|
-
- conf/patterns/
|
96
|
+
- conf/patterns/java.yml
|
97
97
|
- lib/sherlog_holmes.rb
|
98
98
|
- lib/sherlog_holmes/entry.rb
|
99
99
|
- lib/sherlog_holmes/filter.rb
|
data/conf/patterns/jboss.yml
DELETED