RTALogger 2.0.2 → 2.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +141 -8
- data/lib/RTALogger/version.rb +1 -1
- data/lib/log_factory_filter.rb +33 -0
- data/lib/log_filter_base.rb +39 -0
- data/lib/log_filter_context.rb +13 -0
- data/lib/log_filter_message.rb +13 -0
- data/lib/log_filter_topic.rb +13 -0
- data/lib/log_manager.rb +14 -17
- data/lib/log_propagator.rb +45 -13
- data/lib/log_record.rb +4 -0
- data/lib/log_repository.rb +57 -1
- data/lib/log_repository_console.rb +1 -1
- data/lib/log_repository_file.rb +6 -1
- data/lib/log_repository_fluentd.rb +51 -8
- data/lib/rta_logger_config.json +18 -3
- data/lib/sample.rb +1 -1
- metadata +7 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c31a37c1f8f9ce88c8fae11d7df9fe88d397e9e21da9c8b13ddce48fe5e216c5
|
4
|
+
data.tar.gz: 920e2000e9cf216dea8251dd6e6b5b2e3e2c114533e7459cd698eb2131c69c65
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 626deeb001694ef40e87743bb61742f1db0cbeeb46125f1f7b3185a52d33494d811439c3bc9d669c15e60563c9dc71d305159d3c801c06cdfcce82c4450970d0
|
7
|
+
data.tar.gz: 2a3a39efce340c0e0e204ebf6390d17283f6804b3846b3b086e849c25d92e9aec9994beb2a06ab6c81cf10a9abc8af0ded19d37317adbbe5abe16c1744a43aff
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -105,7 +105,22 @@ To use log manager APIs, first step is to have a quick review on Log Data Struct
|
|
105
105
|
{
|
106
106
|
"type": "text",
|
107
107
|
"delimiter": "|"
|
108
|
-
}
|
108
|
+
},
|
109
|
+
"filters":
|
110
|
+
[
|
111
|
+
{
|
112
|
+
"type": "topic",
|
113
|
+
"title": "topic_filter_1",
|
114
|
+
"enable": true,
|
115
|
+
"default_regex" : "^test$"
|
116
|
+
},
|
117
|
+
{
|
118
|
+
"type": "message",
|
119
|
+
"title": "message_filter_1",
|
120
|
+
"enable": false,
|
121
|
+
"default_regex" : "error"
|
122
|
+
}
|
123
|
+
]
|
109
124
|
},
|
110
125
|
{
|
111
126
|
"type": "file",
|
@@ -121,8 +136,8 @@ To use log manager APIs, first step is to have a quick review on Log Data Struct
|
|
121
136
|
}
|
122
137
|
},
|
123
138
|
{
|
124
|
-
"enable":
|
125
|
-
"title": "
|
139
|
+
"enable": false,
|
140
|
+
"title": "fluentd_repo_1",
|
126
141
|
"type": "fluentd",
|
127
142
|
"host": "localhost",
|
128
143
|
"port": "8888",
|
@@ -202,7 +217,22 @@ the result will be:
|
|
202
217
|
{
|
203
218
|
"type": "text",
|
204
219
|
"delimiter": "|"
|
205
|
-
}
|
220
|
+
},
|
221
|
+
"filters":
|
222
|
+
[
|
223
|
+
{
|
224
|
+
"type": "topic",
|
225
|
+
"title": "topic_filter_1",
|
226
|
+
"enable": true,
|
227
|
+
"default_regex" : "^test$"
|
228
|
+
},
|
229
|
+
{
|
230
|
+
"type": "message",
|
231
|
+
"title": "message_filter_1",
|
232
|
+
"enable": false,
|
233
|
+
"default_regex" : "error"
|
234
|
+
}
|
235
|
+
]
|
206
236
|
},
|
207
237
|
{
|
208
238
|
"type": "file",
|
@@ -218,8 +248,8 @@ the result will be:
|
|
218
248
|
}
|
219
249
|
},
|
220
250
|
{
|
221
|
-
"enable":
|
222
|
-
"title": "
|
251
|
+
"enable": false,
|
252
|
+
"title": "fluentd_repo_1",
|
223
253
|
"type": "fluentd",
|
224
254
|
"host": "localhost",
|
225
255
|
"port": "8888",
|
@@ -302,6 +332,22 @@ the result will be:
|
|
302
332
|
- "type": ["text"/"json"] type of formatter
|
303
333
|
- "delimiter": [any text delimiter you need.(as an instance pipe line "|")]
|
304
334
|
if formatter not defined then the json formatter will be used
|
335
|
+
New Features:
|
336
|
+
* In RTALogger version 2.1.0 and above defineing filters is possible for each repository.
|
337
|
+
* Repositories filtes could restrict the amount of data stored by specific repository.
|
338
|
+
* Also you can add new repositories and filters at run time.
|
339
|
+
* filters configuration: It's possible to apply multiple filters on a repository. When repository will
|
340
|
+
flush a log_record only if the record pass all filter conditions.
|
341
|
+
* There are variaty of filter types, including:
|
342
|
+
'topic' which apply a regular expresion on log_record.topic_title to pass filtering condition.
|
343
|
+
'context' which apply a regular expresion on log_record.context_id to pass filtering condition.
|
344
|
+
'message' which apply a regular expression on log_record.full_message to pass filtering condition.
|
345
|
+
* Each filter has following attributes to counfigure:
|
346
|
+
"type": ["topic"/"context"/"message" or any customized filter] described in previous lines.
|
347
|
+
"title": "filte_title" a unique title to make filter run time configuration
|
348
|
+
"enable": [true/false] this will enable or disable filtering opration of current filter.
|
349
|
+
"default_regex": "valid regual expression" to compare with corresponding attribute.
|
350
|
+
* It's possible to implement customize filter classes and integerate with RTALogger filter factory.
|
305
351
|
- topics: This is an optional item. When you need to customize a specific topic severity level or
|
306
352
|
enable value, you can define the settings here.
|
307
353
|
- title: The topic title to customize. (mandatoy).
|
@@ -346,12 +392,14 @@ the result will be:
|
|
346
392
|
# topic.enable
|
347
393
|
# topic.severity_level
|
348
394
|
# Other attributes could only change via config file via manager config_use_json_file
|
395
|
+
# In RTALogger version 2.1.0 and above, it is possible to add new repositories or repository filters at run time
|
396
|
+
# using log_manager.apply_run_time_config method.
|
349
397
|
log_manager.apply_run_time_config(config_json)
|
350
398
|
```
|
351
399
|
### Implement and Expand
|
352
|
-
#### Implement new log repository
|
400
|
+
#### Implement new customized log repository
|
353
401
|
It is possible to implement new log repositories. There will be fue rules to implement and
|
354
|
-
integrate new customized log repository with RTALogger LogManager.
|
402
|
+
integrate new customized log repository with RTALogger LogManager factory.
|
355
403
|
|
356
404
|
1- Define you class inside RTALogger module.
|
357
405
|
|
@@ -396,6 +444,91 @@ module RTALogger
|
|
396
444
|
end
|
397
445
|
```
|
398
446
|
|
447
|
+
#### Implement new customized log repository filter
|
448
|
+
It is possible to implement new log repository filter. There will be fue rules to implement and
|
449
|
+
integrate new customized log repository filter with RTALogger LogManager factory.
|
450
|
+
|
451
|
+
1- Define you class inside RTALogger module.
|
452
|
+
|
453
|
+
2- The class should be inherited from 'RTALogger::LogFilterBase'.
|
454
|
+
|
455
|
+
3- Also appropriate naming convention is necessary.
|
456
|
+
As an example if you are implementing a Console Repo, your class name should be LogFilterTopic and
|
457
|
+
your source code should be placed in a ruby file and name it log_filter_topic.rb
|
458
|
+
|
459
|
+
4- After implementing your own log filter, you should register the class at run-time using the following syntax:
|
460
|
+
```ruby
|
461
|
+
RTALogger::LogFactory.register_log_filter :topic, 'log_filter_topic.rb'
|
462
|
+
```
|
463
|
+
Another example: LogFilterMyCustomizedFilter
|
464
|
+
|
465
|
+
```ruby
|
466
|
+
RTALogger::LogFactory.register_log_filter :my_customized_filter, 'log_repository_my_customized_filter.rb'
|
467
|
+
```
|
468
|
+
Here is 'LogRepositoryTopic' implementation:
|
469
|
+
```ruby
|
470
|
+
require_relative 'log_filter_base'
|
471
|
+
|
472
|
+
module RTALogger
|
473
|
+
class LogFilterTopic < LogFilterBase
|
474
|
+
def match_conditions(log_record)
|
475
|
+
return true if !@enable
|
476
|
+
result = super
|
477
|
+
return result unless result
|
478
|
+
|
479
|
+
return default_regex.present? ? (Regexp.new(@default_regex).match(log_record.topic_title)) : result
|
480
|
+
end
|
481
|
+
end
|
482
|
+
end
|
483
|
+
```
|
484
|
+
#### Implement new customized log formatter
|
485
|
+
|
486
|
+
It is possible to implement new log formatter. There will be fue rules to implement and
|
487
|
+
integrate new customized log formatter with RTALogger LogManager factory.
|
488
|
+
|
489
|
+
1- Define you class inside RTALogger module.
|
490
|
+
|
491
|
+
2- The class should be inherited from 'RTALogger::LogFormatter'.
|
492
|
+
|
493
|
+
3- Also appropriate naming convention is necessary.
|
494
|
+
As an example if you are implementing a Text formatter, your class name should be LogFormatterText and
|
495
|
+
your source code should be placed in a ruby file and name it log_formatter_text.rb
|
496
|
+
|
497
|
+
4- After implementing your own log formatter, you should register the class at run-time using the following syntax:
|
498
|
+
```ruby
|
499
|
+
RTALogger::LogFactory.register_log_formatter :text, 'log_formatter_text.rb'
|
500
|
+
```
|
501
|
+
Another example: LogFormatterMyCustomizedFormatter
|
502
|
+
|
503
|
+
```ruby
|
504
|
+
RTALogger::LogFactory.register_log_formatter :my_customized_formatter, 'log_formatter_my_customized_formatter.rb'
|
505
|
+
```
|
506
|
+
Here is 'LogFormatterText' implementation:
|
507
|
+
```ruby
|
508
|
+
require_relative 'log_formatter_base'
|
509
|
+
require_relative 'severity_level'
|
510
|
+
|
511
|
+
module RTALogger
|
512
|
+
# text formatter which receive log_record and
|
513
|
+
# returns it's data as delimited text string
|
514
|
+
class LogFormatterText < LogFormatterBase
|
515
|
+
include SeverityLevel
|
516
|
+
|
517
|
+
def format(log_record)
|
518
|
+
return '' unless log_record
|
519
|
+
|
520
|
+
result = log_record.occurred_at.strftime('%F %H:%M:%S:%3N')
|
521
|
+
result << @delimiter << log_record.app_name
|
522
|
+
result << @delimiter << log_record.topic_title
|
523
|
+
result << @delimiter << log_record.context_id.to_s
|
524
|
+
result << @delimiter << parse_severity_level_to_s(log_record.severity)
|
525
|
+
result << @delimiter << log_record.message.join(' ').gsub(delimiter, '$<$')
|
526
|
+
|
527
|
+
result
|
528
|
+
end
|
529
|
+
end
|
530
|
+
end
|
531
|
+
```
|
399
532
|
## Contributing
|
400
533
|
|
401
534
|
Bug reports and pull requests are welcome on GitHub at https://github.com/BBahrainy/RTALogger.
|
data/lib/RTALogger/version.rb
CHANGED
@@ -0,0 +1,33 @@
|
|
1
|
+
require_relative 'log_filter_base'
|
2
|
+
|
3
|
+
module RTALogger
|
4
|
+
# Log factory to get new instance of log filter
|
5
|
+
module LogFactory
|
6
|
+
def self.create_filter(type, config_json = '')
|
7
|
+
lib_file = @log_filters[type.to_sym]
|
8
|
+
raise "unregistered filter class: #{type.to_s}" if lib_file.nil? || lib_file.empty?
|
9
|
+
begin
|
10
|
+
load lib_file
|
11
|
+
rescue
|
12
|
+
raise "unable to load formatter class file: #{lib_file}"
|
13
|
+
end
|
14
|
+
|
15
|
+
filter_class_name = 'RTALogger::' + ('log_filter_' + type.to_s).split('_').map(&:capitalize).join
|
16
|
+
filter_class = Object.const_get(filter_class_name)
|
17
|
+
return nil unless filter_class
|
18
|
+
result = filter_class.new
|
19
|
+
|
20
|
+
return result if config_json.empty?
|
21
|
+
result.load_config(config_json) if result.present?
|
22
|
+
return result
|
23
|
+
end
|
24
|
+
|
25
|
+
def self.register_log_filter(type, class_file_name)
|
26
|
+
@log_filters[type.to_sym] = class_file_name
|
27
|
+
end
|
28
|
+
|
29
|
+
@log_filters = {:topic => 'log_filter_topic.rb',
|
30
|
+
:context => 'log_filter_context.rb',
|
31
|
+
:message => 'log_filter_message.rb'}
|
32
|
+
end
|
33
|
+
end
|
@@ -0,0 +1,39 @@
|
|
1
|
+
module RTALogger
|
2
|
+
class LogFilterBase
|
3
|
+
def initialize
|
4
|
+
@title = self.class.to_s.split('::').last.underscore
|
5
|
+
@enable = true
|
6
|
+
end
|
7
|
+
|
8
|
+
attr_accessor :title
|
9
|
+
attr_accessor :enable
|
10
|
+
attr_accessor :default_regex
|
11
|
+
|
12
|
+
def match_conditions(log_record)
|
13
|
+
return true if !@enable
|
14
|
+
return log_record.present?
|
15
|
+
end
|
16
|
+
|
17
|
+
def load_config(config_json)
|
18
|
+
@title = config_json['title'] if config_json['title'].present?
|
19
|
+
@enable = config_json['enable'].nil? ? true : config_json['enable'].present?
|
20
|
+
@default_regex = config_json['default_regex'] if config_json['default_regex'].present?
|
21
|
+
end
|
22
|
+
|
23
|
+
def apply_run_time_config(config_json)
|
24
|
+
@enable = config_json['enable'].nil? ? true : config_json['enable'].present?
|
25
|
+
@default_regex = config_json['default_regex'] if config_json['default_regex'].present?
|
26
|
+
end
|
27
|
+
|
28
|
+
def to_builder
|
29
|
+
jb = Jbuilder.new do |json|
|
30
|
+
json.type self.class.to_s.split('::').last.underscore.sub('log_filter_', '')
|
31
|
+
json.title @title
|
32
|
+
json.enable @enable
|
33
|
+
json.default_regex @default_regex
|
34
|
+
end
|
35
|
+
|
36
|
+
jb
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
@@ -0,0 +1,13 @@
|
|
1
|
+
require_relative 'log_filter_base'
|
2
|
+
|
3
|
+
module RTALogger
|
4
|
+
class LogFilterContext < LogFilterBase
|
5
|
+
def match_conditions(log_record)
|
6
|
+
return true if !@enable
|
7
|
+
result = super
|
8
|
+
return result unless result
|
9
|
+
|
10
|
+
return default_regex.present? ? (Regexp.new(@default_regex).match(log_record.context_id.to_s)) : result
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
@@ -0,0 +1,13 @@
|
|
1
|
+
require_relative 'log_filter_base'
|
2
|
+
|
3
|
+
module RTALogger
|
4
|
+
class LogFilterMessage < LogFilterBase
|
5
|
+
def match_conditions(log_record)
|
6
|
+
return true if !@enable
|
7
|
+
result = super
|
8
|
+
return result unless result
|
9
|
+
|
10
|
+
return default_regex.present? ? (Regexp.new(@default_regex).match(log_record.full_message)) : result
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
@@ -0,0 +1,13 @@
|
|
1
|
+
require_relative 'log_filter_base'
|
2
|
+
|
3
|
+
module RTALogger
|
4
|
+
class LogFilterTopic < LogFilterBase
|
5
|
+
def match_conditions(log_record)
|
6
|
+
return true if !@enable
|
7
|
+
result = super
|
8
|
+
return result unless result
|
9
|
+
|
10
|
+
return default_regex.present? ? (Regexp.new(@default_regex).match(log_record.topic_title)) : result
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
data/lib/log_manager.rb
CHANGED
@@ -76,7 +76,7 @@ module RTALogger
|
|
76
76
|
@propagator.add_log_repository(LogFactory.create_repository(:console))
|
77
77
|
end
|
78
78
|
|
79
|
-
def config_use_json_string(config_string,
|
79
|
+
def config_use_json_string(config_string, title = '')
|
80
80
|
config_json = load_config_from_json_string(config_string, title)
|
81
81
|
apply_config(config_json)
|
82
82
|
rescue StandardError => e
|
@@ -138,7 +138,8 @@ module RTALogger
|
|
138
138
|
json.flush_size flush_size
|
139
139
|
json.flush_wait_time flush_wait_time
|
140
140
|
json.repositories do
|
141
|
-
json.array! @propagator.repositories.collect { |repository| repository.to_builder.attributes! }
|
141
|
+
# json.array! @propagator.repositories.collect { |repository| repository.to_builder.attributes! }
|
142
|
+
json.array! @propagator.to_builder
|
142
143
|
end
|
143
144
|
json.topics do
|
144
145
|
json.array! topics.keys.collect { |topic_key| @topics[topic_key].to_builder.attributes! }
|
@@ -156,22 +157,23 @@ module RTALogger
|
|
156
157
|
def apply_run_time_config(config_json)
|
157
158
|
return unless config_json
|
158
159
|
@enable = config_json['enable'] unless config_json['enable'].nil?
|
159
|
-
@default_severity_level =
|
160
|
+
@default_severity_level = parse_severity_level_to_s(config_json['severity_level']) unless config_json['severity_level'].nil?
|
160
161
|
self.buffer_size = config_json['buffer_size'] unless config_json['buffer_size'].nil?
|
161
162
|
self.flush_wait_time = config_json['flush_wait_time'] unless config_json['flush_wait_time'].nil?
|
162
163
|
@propagator.apply_run_time_config(config_json)
|
163
|
-
|
164
|
-
if config_json['topics']
|
165
|
-
config_json['topics'].each do |topic_config|
|
166
|
-
next if topic_config['title'].nil?
|
167
|
-
topic = topic_by_title(topic_config['title'])
|
168
|
-
topic.apply_run_time_config(topic_config) if topic
|
169
|
-
end
|
170
|
-
end
|
164
|
+
apply_run_time_config_topics(config_json)
|
171
165
|
end
|
172
166
|
|
173
167
|
private
|
174
168
|
|
169
|
+
def apply_run_time_config_topics(config_json)
|
170
|
+
config_json['topics']&.each do |topic_config|
|
171
|
+
next if topic_config['title'].nil?
|
172
|
+
topic = topic_by_title(topic_config['title'])
|
173
|
+
topic.apply_run_time_config(topic_config) if topic.present?
|
174
|
+
end
|
175
|
+
end
|
176
|
+
|
175
177
|
def load_config_from_json_file(config_file_name, title = '')
|
176
178
|
config_file = File.open config_file_name
|
177
179
|
config_json = ::JSON.load(config_file)
|
@@ -207,15 +209,10 @@ module RTALogger
|
|
207
209
|
@default_severity_level = parse_severity_level_to_i(config_json['severity_level']) if config_json['severity_level']
|
208
210
|
self.buffer_size = config_json['buffer_size'] if config_json['buffer_size']
|
209
211
|
self.flush_wait_time = config_json['flush_wait_seconds'] if config_json['flush_wait_seconds']
|
210
|
-
@propagator.
|
211
|
-
apply_config_repos(config_json)
|
212
|
+
@propagator.load_repositories(config_json)
|
212
213
|
apply_config_topics(config_json)
|
213
214
|
end
|
214
215
|
|
215
|
-
def apply_config_repos(config_json)
|
216
|
-
config_json['repositories']&.each { |item| @propagator.load_log_repository(item) }
|
217
|
-
end
|
218
|
-
|
219
216
|
def apply_config_topics(config_json)
|
220
217
|
config_json['topics']&.each do |topic|
|
221
218
|
next unless topic['title']
|
data/lib/log_propagator.rb
CHANGED
@@ -6,7 +6,7 @@ module RTALogger
|
|
6
6
|
def initialize
|
7
7
|
@semaphore = Mutex.new
|
8
8
|
@records = []
|
9
|
-
@repositories =
|
9
|
+
@repositories = {}
|
10
10
|
end
|
11
11
|
|
12
12
|
attr_reader :repositories
|
@@ -16,20 +16,34 @@ module RTALogger
|
|
16
16
|
end
|
17
17
|
|
18
18
|
def add_log_repository(repository)
|
19
|
+
return if repository.nil? || repository.title.to_s.empty?
|
19
20
|
return unless repository.is_a? RTALogger::LogRepository
|
20
|
-
@repositories.
|
21
|
+
@semaphore.synchronize { @repositories[repository.title.to_sym] = repository unless @repositories[repository.title.to_sym].present? }
|
21
22
|
end
|
22
23
|
|
23
24
|
def load_log_repository(config_json)
|
24
25
|
type = config_json['type']
|
25
26
|
return if type.to_s.strip.empty?
|
26
|
-
enable = config_json['enable'].nil? ? true : config_json['enable']
|
27
|
-
return unless enable
|
28
27
|
|
29
28
|
repository = ::RTALogger::LogFactory.create_repository(type, config_json)
|
30
29
|
add_log_repository(repository)
|
31
30
|
end
|
32
31
|
|
32
|
+
def load_repositories(config_json)
|
33
|
+
return if config_json.nil?
|
34
|
+
|
35
|
+
@semaphore.synchronize do
|
36
|
+
@repositories.clear
|
37
|
+
config_json['repositories']&.each do |repository_config|
|
38
|
+
type = repository_config['type']
|
39
|
+
next if type.to_s.strip.empty?
|
40
|
+
|
41
|
+
repository = ::RTALogger::LogFactory.create_repository(type, repository_config)
|
42
|
+
@repositories[repository.title.to_sym] = repository unless @repositories[repository.title.to_sym].present?
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
33
47
|
def drop_all_repositories
|
34
48
|
@semaphore.synchronize { @repositories.clear }
|
35
49
|
end
|
@@ -48,23 +62,41 @@ module RTALogger
|
|
48
62
|
|
49
63
|
def apply_run_time_config(config_json)
|
50
64
|
return unless config_json
|
65
|
+
apply_run_time_config_repositories(config_json)
|
66
|
+
end
|
51
67
|
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
repository.apply_run_time_config(repository_config) if repository
|
68
|
+
def propagate
|
69
|
+
@semaphore.synchronize do
|
70
|
+
@repositories.keys.each do |repository_key|
|
71
|
+
@repositories[repository_key.to_sym].add_log_records(@records)
|
57
72
|
end
|
73
|
+
@records.clear
|
58
74
|
end
|
59
75
|
end
|
60
76
|
|
61
|
-
def
|
77
|
+
def to_builder
|
78
|
+
result = nil
|
62
79
|
@semaphore.synchronize do
|
63
|
-
@repositories.
|
64
|
-
|
80
|
+
result = @repositories&.keys.collect { |repository_key| @repositories[repository_key].to_builder.attributes! }
|
81
|
+
end
|
82
|
+
|
83
|
+
return result
|
84
|
+
end
|
85
|
+
|
86
|
+
private
|
87
|
+
|
88
|
+
def apply_run_time_config_repositories(config_json)
|
89
|
+
config_json['repositories']&.each do |repository_config|
|
90
|
+
next if repository_config['title'].nil?
|
91
|
+
repository = repository_by_title(repository_config['title'])
|
92
|
+
if repository.present?
|
93
|
+
repository.apply_run_time_config(repository_config)
|
94
|
+
else
|
95
|
+
repository = ::RTALogger::LogFactory.create_repository(repository_config['type'], config_json)
|
96
|
+
add_log_repository(repository)
|
65
97
|
end
|
66
|
-
@records.clear
|
67
98
|
end
|
68
99
|
end
|
100
|
+
|
69
101
|
end
|
70
102
|
end
|
data/lib/log_record.rb
CHANGED
data/lib/log_repository.rb
CHANGED
@@ -1,4 +1,5 @@
|
|
1
1
|
require_relative 'string'
|
2
|
+
require_relative 'log_factory_filter'
|
2
3
|
|
3
4
|
module RTALogger
|
4
5
|
# base log repository class
|
@@ -9,6 +10,7 @@ module RTALogger
|
|
9
10
|
@title = self.class.to_s.split('::').last.underscore
|
10
11
|
@enable = true
|
11
12
|
@formatter = RTALogger::LogFactory.log_formatter_default
|
13
|
+
@filters = {}
|
12
14
|
end
|
13
15
|
|
14
16
|
# @@sub_classes = {}
|
@@ -33,8 +35,11 @@ module RTALogger
|
|
33
35
|
def add_log_records(items)
|
34
36
|
return 0 unless @enable
|
35
37
|
@semaphore.synchronize do
|
36
|
-
items.each
|
38
|
+
items.each do |item|
|
39
|
+
@log_records.push(item.dup) if filters_accept(item)
|
40
|
+
end
|
37
41
|
end
|
42
|
+
|
38
43
|
flush_and_clear
|
39
44
|
end
|
40
45
|
|
@@ -45,6 +50,8 @@ module RTALogger
|
|
45
50
|
if formatter_config && formatter_config['type']
|
46
51
|
@formatter = LogFactory.create_formatter(formatter_config['type'], formatter_config)
|
47
52
|
end
|
53
|
+
|
54
|
+
apply_config_filters(config_json)
|
48
55
|
end
|
49
56
|
|
50
57
|
def to_builder
|
@@ -52,6 +59,9 @@ module RTALogger
|
|
52
59
|
json.type self.class.to_s.split('::').last.underscore.sub('log_repository_', '')
|
53
60
|
json.enable enable
|
54
61
|
json.formatter @formatter.to_builder.attributes!
|
62
|
+
json.filters do
|
63
|
+
json.array! @filters.keys.collect { |filter_key| @filters[filter_key].to_builder.attributes! }
|
64
|
+
end
|
55
65
|
end
|
56
66
|
|
57
67
|
jb
|
@@ -61,17 +71,63 @@ module RTALogger
|
|
61
71
|
to_builder.target!
|
62
72
|
end
|
63
73
|
|
74
|
+
def filter_by_title(title)
|
75
|
+
result = nil
|
76
|
+
@semaphore.synchronize do
|
77
|
+
@filters.keys.each do |filter_key|
|
78
|
+
result = @filters[filter_key.to_sym] if filter_key.to_s.casecmp(title).zero?
|
79
|
+
break if result
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
return result
|
84
|
+
end
|
85
|
+
|
64
86
|
def apply_run_time_config(config_json)
|
65
87
|
return unless config_json
|
66
88
|
@enable = config_json['enable'] unless config_json['enable'].nil?
|
89
|
+
apply_run_time_config_filters(config_json)
|
67
90
|
end
|
68
91
|
|
69
92
|
protected
|
70
93
|
|
94
|
+
def apply_config_filters(config_json)
|
95
|
+
config_json['filters']&.each do |filter_config|
|
96
|
+
next if filter_config['type'].nil? || filter_config['title'].nil?
|
97
|
+
filter = LogFactory.create_filter(filter_config['type'], filter_config)
|
98
|
+
@filters[filter_config['title'].to_sym] = filter if filter.present?
|
99
|
+
end
|
100
|
+
end
|
101
|
+
|
102
|
+
def apply_run_time_config_filters(config_json)
|
103
|
+
return unless config_json
|
104
|
+
|
105
|
+
config_json['filters']&.each do |filter_config|
|
106
|
+
next if filter_config['title'].nil?
|
107
|
+
filter = filter_by_title(filter_config['title'])
|
108
|
+
if filter.present?
|
109
|
+
filter.apply_run_time_config(filter_config)
|
110
|
+
else
|
111
|
+
filter = LogFactory.create_filter(filter_config['type'], filter_config)
|
112
|
+
@semaphore.synchronize { @filters[filter_config['title'].to_sym] = filter if filter.present? }
|
113
|
+
end
|
114
|
+
end
|
115
|
+
end
|
116
|
+
|
71
117
|
def flush_and_clear
|
72
118
|
@semaphore.synchronize { @log_records.clear }
|
73
119
|
end
|
74
120
|
|
121
|
+
def filters_accept(log_record)
|
122
|
+
result = true
|
123
|
+
@filters&.keys.each do |filter_key|
|
124
|
+
result = @filters[filter_key.to_sym]&.match_conditions(log_record)
|
125
|
+
break unless result
|
126
|
+
end
|
127
|
+
|
128
|
+
return result
|
129
|
+
end
|
130
|
+
|
75
131
|
attr_reader :log_records
|
76
132
|
attr_reader :semaphore
|
77
133
|
end
|
data/lib/log_repository_file.rb
CHANGED
@@ -31,8 +31,13 @@ module RTALogger
|
|
31
31
|
|
32
32
|
json
|
33
33
|
end
|
34
|
+
|
34
35
|
# register :file
|
35
36
|
|
37
|
+
def apply_run_time_config(config_json)
|
38
|
+
super config_json
|
39
|
+
end
|
40
|
+
|
36
41
|
protected
|
37
42
|
|
38
43
|
def create_ruby_logger(file_path, period, shift_size)
|
@@ -45,7 +50,7 @@ module RTALogger
|
|
45
50
|
end
|
46
51
|
|
47
52
|
def flush_and_clear
|
48
|
-
semaphore.synchronize do
|
53
|
+
@semaphore.synchronize do
|
49
54
|
@log_records.each { |log_record| @file_logger.debug(@formatter.format(log_record)) }
|
50
55
|
end
|
51
56
|
super
|
@@ -7,43 +7,85 @@ module RTALogger
|
|
7
7
|
super()
|
8
8
|
@host = host
|
9
9
|
@port = port
|
10
|
-
@
|
10
|
+
@tls_options = tls_options
|
11
|
+
@fluent_logger = nil
|
11
12
|
end
|
12
13
|
|
13
14
|
def load_config(config_json)
|
14
15
|
super
|
15
16
|
|
16
|
-
host = config_json['host'].to_s
|
17
|
-
port = config_json['port'].
|
18
|
-
tls_options = config_json['tls_options']
|
17
|
+
@host = config_json['host'].to_s
|
18
|
+
@port = config_json['port'].to_i
|
19
|
+
@tls_options = config_json['tls_options']
|
19
20
|
|
20
|
-
@fluent_logger =
|
21
|
+
@semaphore.synchronize { @fluent_logger = nil }
|
22
|
+
end
|
23
|
+
|
24
|
+
def host=(host)
|
25
|
+
if @host != host
|
26
|
+
@host = host ? host : 'localhost'
|
27
|
+
@semaphore.synchronize { @fluent_logger = nil }
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
def host
|
32
|
+
@host
|
33
|
+
end
|
34
|
+
|
35
|
+
def port=(port)
|
36
|
+
if @port != port
|
37
|
+
@port = port ? port : 24224
|
38
|
+
@semaphore.synchronize { @fluent_logger = nil }
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
def port
|
43
|
+
@port
|
21
44
|
end
|
22
45
|
|
23
46
|
def to_builder
|
24
47
|
json = super
|
25
|
-
json.enable enable
|
48
|
+
json.enable @enable
|
26
49
|
json.host @host
|
27
50
|
json.port @port
|
51
|
+
json.tls_options @tls_options
|
28
52
|
|
29
53
|
json
|
30
54
|
end
|
55
|
+
|
31
56
|
# register :fluentd
|
32
57
|
|
58
|
+
def apply_run_time_config(config_json)
|
59
|
+
super config_json
|
60
|
+
|
61
|
+
@host = config_json['host'] unless config_json['host'].nil?
|
62
|
+
@port = config_json['port'] unless config_json['port'].nil?
|
63
|
+
@tls_options = config_json['tls_options'] unless config_json['tls_options'].nil?
|
64
|
+
@semaphore.synchronize { @fluent_logger = nil }
|
65
|
+
end
|
66
|
+
|
33
67
|
protected
|
34
68
|
|
35
69
|
def create_fluentd_logger(host, port, tls_options)
|
70
|
+
return nil unless @enable
|
71
|
+
|
36
72
|
unless tls_options
|
37
73
|
fluent_logger = ::Fluent::Logger::FluentLogger.new(nil, :host => host, :port => port, :use_nonblock => true, :wait_writeable => false)
|
38
74
|
else
|
39
|
-
fluent_logger = ::Fluent::Logger::FluentLogger.new(nil, :host => host,
|
75
|
+
fluent_logger = ::Fluent::Logger::FluentLogger.new(nil, :host => host,
|
76
|
+
:port => port,
|
77
|
+
:tls_options => tls_options,
|
78
|
+
:use_nonblock => true,
|
79
|
+
:wait_writeable => false)
|
40
80
|
end
|
41
81
|
|
42
82
|
fluent_logger
|
43
83
|
end
|
44
84
|
|
45
85
|
def flush_and_clear
|
46
|
-
semaphore.synchronize do
|
86
|
+
@semaphore.synchronize do
|
87
|
+
@fluent_logger = create_fluentd_logger(@host, @port, @tls_options) unless @fluent_logger.present?
|
88
|
+
|
47
89
|
@log_records.each do |log_record|
|
48
90
|
fluent_tag = log_record.app_name + '.' + log_record.topic_title
|
49
91
|
log_json_string = @formatter.format(log_record)
|
@@ -52,6 +94,7 @@ module RTALogger
|
|
52
94
|
@fluent_logger.post(fluent_tag, log_json)
|
53
95
|
end
|
54
96
|
end
|
97
|
+
|
55
98
|
super
|
56
99
|
end
|
57
100
|
end
|
data/lib/rta_logger_config.json
CHANGED
@@ -21,7 +21,22 @@
|
|
21
21
|
{
|
22
22
|
"type": "text",
|
23
23
|
"delimiter": "|"
|
24
|
-
}
|
24
|
+
},
|
25
|
+
"filters":
|
26
|
+
[
|
27
|
+
{
|
28
|
+
"type": "topic",
|
29
|
+
"title": "topic_filter_1",
|
30
|
+
"enable": true,
|
31
|
+
"default_regex" : "^test$"
|
32
|
+
},
|
33
|
+
{
|
34
|
+
"type": "message",
|
35
|
+
"title": "message_filter_1",
|
36
|
+
"enable": false,
|
37
|
+
"default_regex" : "error"
|
38
|
+
}
|
39
|
+
]
|
25
40
|
},
|
26
41
|
{
|
27
42
|
"type": "file",
|
@@ -37,8 +52,8 @@
|
|
37
52
|
}
|
38
53
|
},
|
39
54
|
{
|
40
|
-
"enable":
|
41
|
-
"title": "
|
55
|
+
"enable": false,
|
56
|
+
"title": "fluentd_repo_1",
|
42
57
|
"type": "fluentd",
|
43
58
|
"host": "localhost",
|
44
59
|
"port": "8888",
|
data/lib/sample.rb
CHANGED
@@ -39,7 +39,7 @@ topic.unknown(userID, 'Controller Name=', controller_name, 'unknown')
|
|
39
39
|
test_topic.error(userID, 'test_topic', 'error')
|
40
40
|
test_topic.fatal(userID, 'test_topic', 'fatal')
|
41
41
|
|
42
|
-
puts log_manager.reveal_config
|
42
|
+
# puts log_manager.reveal_config
|
43
43
|
|
44
44
|
# update specific topic log level if necessary
|
45
45
|
# log_manager.update_topic_level(controller_name, RTALogger::SeverityLevel::INFO)
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: RTALogger
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0
|
4
|
+
version: 2.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Babak Bahreini, RTA Backend Team
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-12-
|
11
|
+
date: 2020-12-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: fluent-logger
|
@@ -68,12 +68,17 @@ files:
|
|
68
68
|
- bin/setup
|
69
69
|
- lib/RTALogger.rb
|
70
70
|
- lib/RTALogger/version.rb
|
71
|
+
- lib/log_factory_filter.rb
|
71
72
|
- lib/log_factory_log_formatter.rb
|
72
73
|
- lib/log_factory_manager.rb
|
73
74
|
- lib/log_factory_propagator.rb
|
74
75
|
- lib/log_factory_record.rb
|
75
76
|
- lib/log_factory_repository.rb
|
76
77
|
- lib/log_factory_topic.rb
|
78
|
+
- lib/log_filter_base.rb
|
79
|
+
- lib/log_filter_context.rb
|
80
|
+
- lib/log_filter_message.rb
|
81
|
+
- lib/log_filter_topic.rb
|
77
82
|
- lib/log_formatter_base.rb
|
78
83
|
- lib/log_formatter_json.rb
|
79
84
|
- lib/log_formatter_text.rb
|