semantic_logger 4.7.2 → 4.10.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +50 -20
- data/lib/semantic_logger/appender/async.rb +1 -1
- data/lib/semantic_logger/appender/async_batch.rb +3 -1
- data/lib/semantic_logger/appender/bugsnag.rb +36 -23
- data/lib/semantic_logger/appender/elasticsearch.rb +24 -6
- data/lib/semantic_logger/appender/file.rb +249 -68
- data/lib/semantic_logger/appender/io.rb +68 -0
- data/lib/semantic_logger/appender/kafka.rb +4 -0
- data/lib/semantic_logger/appender/sentry_ruby.rb +138 -0
- data/lib/semantic_logger/appender/splunk.rb +2 -1
- data/lib/semantic_logger/appender/splunk_http.rb +3 -2
- data/lib/semantic_logger/appender/syslog.rb +5 -3
- data/lib/semantic_logger/appender/wrapper.rb +3 -2
- data/lib/semantic_logger/appender.rb +13 -6
- data/lib/semantic_logger/appenders.rb +31 -27
- data/lib/semantic_logger/base.rb +16 -7
- data/lib/semantic_logger/formatters/base.rb +1 -0
- data/lib/semantic_logger/formatters/color.rb +3 -3
- data/lib/semantic_logger/formatters/logfmt.rb +72 -0
- data/lib/semantic_logger/formatters/syslog.rb +2 -1
- data/lib/semantic_logger/formatters/syslog_cee.rb +2 -1
- data/lib/semantic_logger/formatters.rb +10 -11
- data/lib/semantic_logger/log.rb +2 -4
- data/lib/semantic_logger/loggable.rb +8 -1
- data/lib/semantic_logger/logger.rb +16 -6
- data/lib/semantic_logger/processor.rb +3 -3
- data/lib/semantic_logger/semantic_logger.rb +18 -10
- data/lib/semantic_logger/subscriber.rb +10 -0
- data/lib/semantic_logger/sync_processor.rb +5 -5
- data/lib/semantic_logger/test/capture_log_events.rb +34 -0
- data/lib/semantic_logger/utils.rb +29 -10
- data/lib/semantic_logger/version.rb +1 -1
- data/lib/semantic_logger.rb +4 -0
- metadata +13 -10
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 2ff7d4bcb345581f6ba19f834bb1bf2f921e97904f4b6236142b87af75a4526d
|
4
|
+
data.tar.gz: b1069d5360296c61c9796f630b1ec065eb1a0a7dc549778c66c5e26c90d0eadb
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c5cfbac7dd8795b11cf79ed3fa9a4a086128d5ee030f0e9db40b75f30f58997c1674a4284b4b287939216bc8de83372e2b7fe00d73e7856c3cc6830310740f0d
|
7
|
+
data.tar.gz: 2150a809562247d771103a5d387ae5852ff5b0649a4f4288a7dd07a94420bafcfb6b36f3e21077fcc053c2d8bcaffe80a110ab65b52178fb612b7368fd79d87d
|
data/README.md
CHANGED
@@ -1,25 +1,13 @@
|
|
1
1
|
# Semantic Logger
|
2
|
-
[![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://
|
2
|
+
[![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://github.com/reidmorrison/semantic_logger/workflows/build/badge.svg)](https://github.com/reidmorrison/semantic_logger/actions?query=workflow%3Abuild) [![Downloads](https://img.shields.io/gem/dt/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg)
|
3
3
|
|
4
4
|
Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.
|
5
5
|
|
6
|
-
* https://rocketjob.
|
6
|
+
* https://logger.rocketjob.io/
|
7
7
|
|
8
8
|
## Documentation
|
9
9
|
|
10
|
-
[Semantic Logger Guide](
|
11
|
-
|
12
|
-
[Reference Documentation](http://www.rubydoc.info/gems/semantic_logger/)
|
13
|
-
|
14
|
-
## Upgrading to Semantic Logger v4.4
|
15
|
-
|
16
|
-
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
17
|
-
workaround for Ruby 2.5 crashes is no longer needed.
|
18
|
-
I.e. Please remove the following line if being called anywhere:
|
19
|
-
|
20
|
-
~~~ruby
|
21
|
-
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
22
|
-
~~~
|
10
|
+
[Semantic Logger Guide](https://logger.rocketjob.io/)
|
23
11
|
|
24
12
|
## Logging Destinations
|
25
13
|
|
@@ -34,7 +22,7 @@ Logging to the following destinations are all supported "out-of-the-box":
|
|
34
22
|
* Splunk
|
35
23
|
* MongoDB
|
36
24
|
* Honeybadger
|
37
|
-
* Sentry
|
25
|
+
* Sentry (both with legacy `sentry-raven` and modern `sentry-ruby` gem)
|
38
26
|
* HTTP
|
39
27
|
* TCP
|
40
28
|
* UDP
|
@@ -50,7 +38,7 @@ handles saving log information to multiple destinations / appenders.
|
|
50
38
|
|
51
39
|
## Rails
|
52
40
|
|
53
|
-
When running Rails, use [rails_semantic_logger](http://github.com/
|
41
|
+
When running Rails, use [rails_semantic_logger](http://github.com/reidmorrison/rails_semantic_logger)
|
54
42
|
instead of Semantic Logger directly since it will automatically replace the Rails default logger with Semantic Logger.
|
55
43
|
|
56
44
|
## Rocket Job
|
@@ -72,8 +60,50 @@ and are therefore not automatically included by this gem:
|
|
72
60
|
- Splunk Appender: gem 'splunk-sdk-ruby'
|
73
61
|
- Elasticsearch Appender: gem 'elasticsearch'
|
74
62
|
- Kafka Appender: gem 'ruby-kafka'
|
63
|
+
- Legacy Sentry Appender: gem 'sentry-raven' (deprecated)
|
64
|
+
- Sentry Appender: gem 'sentry-ruby'
|
65
|
+
|
66
|
+
## Upgrading to Semantic Logger v4.9
|
67
|
+
|
68
|
+
These changes should not be noticeable by the majority of users of Semantic Logger, since
|
69
|
+
they are to the internal API. It is possible that advanced users may be using these internal
|
70
|
+
API's directly.
|
71
|
+
|
72
|
+
This does not affect any calls to the public api `SemanticLogger.add_appender`.
|
73
|
+
|
74
|
+
File and IO are now separate appenders. When creating the File appender explicitly, its arguments
|
75
|
+
have changed. For example, when requesting an IO stream, it needs to be changed from:
|
76
|
+
|
77
|
+
~~~ruby
|
78
|
+
SemanticLogger::Appender::File.new(io: $stderr)
|
79
|
+
~~~
|
80
|
+
to:
|
81
|
+
~~~ruby
|
82
|
+
SemanticLogger::Appender::IO.new($stderr)
|
83
|
+
~~~
|
84
|
+
|
85
|
+
Additionally, this needs to be changed from:
|
86
|
+
~~~ruby
|
87
|
+
SemanticLogger::Appender::File.new(file_name: "file.log")
|
88
|
+
~~~
|
89
|
+
to:
|
90
|
+
~~~ruby
|
91
|
+
SemanticLogger::Appender::File.new("file.log")
|
92
|
+
~~~
|
93
|
+
|
94
|
+
Rails Semantic Logger, if used, needs to be upgraded to v4.9 when upgrading to Semantic Logger v4.9.
|
95
|
+
|
96
|
+
## Upgrading to Semantic Logger v4.4
|
97
|
+
|
98
|
+
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
99
|
+
workaround for Ruby 2.5 crashes is no longer needed.
|
100
|
+
I.e. Please remove the following line if being called anywhere:
|
101
|
+
|
102
|
+
~~~ruby
|
103
|
+
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
104
|
+
~~~
|
75
105
|
|
76
|
-
##
|
106
|
+
## Upgrading to Semantic Logger v4.0
|
77
107
|
|
78
108
|
The following changes need to be made when upgrading to V4:
|
79
109
|
- Ruby V2.3 / JRuby V9.1 is now the minimum runtime version.
|
@@ -129,13 +159,13 @@ SemanticLogger.default_level = :trace
|
|
129
159
|
SemanticLogger.add_appender(file_name: 'development.log', formatter: :color)
|
130
160
|
~~~
|
131
161
|
|
132
|
-
If running rails, see: [Semantic Logger Rails](
|
162
|
+
If running rails, see: [Semantic Logger Rails](https://logger.rocketjob.io/rails.html)
|
133
163
|
|
134
164
|
## Author
|
135
165
|
|
136
166
|
[Reid Morrison](https://github.com/reidmorrison)
|
137
167
|
|
138
|
-
[Contributors](https://github.com/
|
168
|
+
[Contributors](https://github.com/reidmorrison/semantic_logger/graphs/contributors)
|
139
169
|
|
140
170
|
## Versioning
|
141
171
|
|
@@ -53,7 +53,7 @@ module SemanticLogger
|
|
53
53
|
# Re-open appender after a fork
|
54
54
|
def reopen
|
55
55
|
# Workaround CRuby crash on fork by recreating queue on reopen
|
56
|
-
# https://github.com/
|
56
|
+
# https://github.com/reidmorrison/semantic_logger/issues/103
|
57
57
|
@queue&.close
|
58
58
|
create_queue
|
59
59
|
|
@@ -64,6 +64,7 @@ module SemanticLogger
|
|
64
64
|
signal.wait(batch_seconds)
|
65
65
|
|
66
66
|
logs = []
|
67
|
+
messages = []
|
67
68
|
first = true
|
68
69
|
message_count = queue.length
|
69
70
|
message_count.times do
|
@@ -76,10 +77,11 @@ module SemanticLogger
|
|
76
77
|
first = false
|
77
78
|
end
|
78
79
|
else
|
79
|
-
|
80
|
+
messages << message
|
80
81
|
end
|
81
82
|
end
|
82
83
|
appender.batch(logs) if logs.size.positive?
|
84
|
+
messages.each { |message| process_message(message) }
|
83
85
|
signal.reset unless queue.size >= batch_size
|
84
86
|
end
|
85
87
|
end
|
@@ -40,42 +40,55 @@ module SemanticLogger
|
|
40
40
|
|
41
41
|
# Returns [Hash] of parameters to send to Bugsnag.
|
42
42
|
def call(log, logger)
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
43
|
+
hash = SemanticLogger::Formatters::Raw.new.call(log, logger)
|
44
|
+
hash.delete(:message) if hash[:exception] && (hash[:message] == hash[:exception][:message])
|
45
|
+
hash.delete(:time)
|
46
|
+
hash.delete(:level_index)
|
47
|
+
hash.delete(:exception)
|
48
|
+
hash[:file] = "#{hash[:file]}:#{hash.delete(:line)}" if hash.key?(:file)
|
49
|
+
hash
|
49
50
|
end
|
50
51
|
|
51
|
-
# Send an error notification to Bugsnag
|
52
52
|
def log(log)
|
53
53
|
# Ignore logs coming from Bugsnag itself
|
54
54
|
return false if log.name == "Bugsnag"
|
55
55
|
|
56
56
|
# Send error messages as Runtime exceptions
|
57
|
-
exception =
|
58
|
-
|
59
|
-
|
60
|
-
log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
|
61
|
-
log.exception
|
62
|
-
else
|
63
|
-
error = RuntimeError.new(log.message)
|
64
|
-
error.set_backtrace(log.backtrace) if log.backtrace
|
65
|
-
error
|
66
|
-
end
|
67
|
-
|
68
|
-
# For more documentation on the Bugsnag.notify method see:
|
69
|
-
# https://bugsnag.com/docs/notifiers/ruby#sending-handled-exceptions
|
70
|
-
::Bugsnag.notify(exception, formatter.call(log, self))
|
57
|
+
exception = extract_exception(log)
|
58
|
+
hash = formatter.call(log, self)
|
59
|
+
bugsnag_notify(exception, hash, log_level(log.level))
|
71
60
|
true
|
72
61
|
end
|
73
62
|
|
74
63
|
private
|
75
64
|
|
65
|
+
def bugsnag_notify(exception, hash, level)
|
66
|
+
if ::Bugsnag::VERSION.to_i >= 6
|
67
|
+
::Bugsnag.notify(exception) do |report|
|
68
|
+
report.severity = level
|
69
|
+
hash.each_pair { |key, value| report.add_tab(key, value) }
|
70
|
+
end
|
71
|
+
else
|
72
|
+
hash[:severity] = level
|
73
|
+
::Bugsnag.notify(exception, hash)
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
def extract_exception(log)
|
78
|
+
if log.exception
|
79
|
+
# Manually constructed Exception, without a backtrace.
|
80
|
+
log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
|
81
|
+
return log.exception
|
82
|
+
end
|
83
|
+
|
84
|
+
error = RuntimeError.new(log.message)
|
85
|
+
error.set_backtrace(log.backtrace) if log.backtrace
|
86
|
+
error
|
87
|
+
end
|
88
|
+
|
76
89
|
# Bugsnag supports: error, warning or info
|
77
|
-
def log_level(
|
78
|
-
case
|
90
|
+
def log_level(level)
|
91
|
+
case level
|
79
92
|
when :error, :fatal
|
80
93
|
"error"
|
81
94
|
when :warn
|
@@ -1,7 +1,8 @@
|
|
1
1
|
begin
|
2
2
|
require "elasticsearch"
|
3
3
|
rescue LoadError
|
4
|
-
raise LoadError,
|
4
|
+
raise LoadError,
|
5
|
+
'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
|
5
6
|
end
|
6
7
|
|
7
8
|
require "date"
|
@@ -17,7 +18,8 @@ require "date"
|
|
17
18
|
module SemanticLogger
|
18
19
|
module Appender
|
19
20
|
class Elasticsearch < SemanticLogger::Subscriber
|
20
|
-
attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
|
21
|
+
attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
|
22
|
+
:elasticsearch_args
|
21
23
|
|
22
24
|
# Create Elasticsearch appender over persistent HTTP(S)
|
23
25
|
#
|
@@ -133,7 +135,7 @@ module SemanticLogger
|
|
133
135
|
application: nil,
|
134
136
|
environment: nil,
|
135
137
|
host: nil,
|
136
|
-
|
138
|
+
data_stream: false,
|
137
139
|
**elasticsearch_args,
|
138
140
|
&block)
|
139
141
|
|
@@ -144,6 +146,7 @@ module SemanticLogger
|
|
144
146
|
@elasticsearch_args = elasticsearch_args.dup
|
145
147
|
@elasticsearch_args[:url] = url if url && !elasticsearch_args[:hosts]
|
146
148
|
@elasticsearch_args[:logger] = logger
|
149
|
+
@data_stream = data_stream
|
147
150
|
|
148
151
|
super(level: level, formatter: formatter, filter: filter, application: application, environment: environment, host: host, metrics: false, &block)
|
149
152
|
reopen
|
@@ -173,7 +176,12 @@ module SemanticLogger
|
|
173
176
|
private
|
174
177
|
|
175
178
|
def write_to_elasticsearch(messages)
|
176
|
-
bulk_result = @
|
179
|
+
bulk_result = if @data_stream
|
180
|
+
@client.bulk(index: index, body: messages)
|
181
|
+
else
|
182
|
+
@client.bulk(body: messages)
|
183
|
+
end
|
184
|
+
|
177
185
|
return unless bulk_result["errors"]
|
178
186
|
|
179
187
|
failed = bulk_result["items"].reject { |x| x["status"] == 201 }
|
@@ -182,11 +190,21 @@ module SemanticLogger
|
|
182
190
|
|
183
191
|
def bulk_index(log)
|
184
192
|
expanded_index_name = log.time.strftime("#{index}-#{date_pattern}")
|
185
|
-
|
193
|
+
if @data_stream
|
194
|
+
{"create" => {}}
|
195
|
+
else
|
196
|
+
{"index" => {"_index" => expanded_index_name, "_type" => type}}
|
197
|
+
end
|
186
198
|
end
|
187
199
|
|
188
200
|
def default_formatter
|
189
|
-
|
201
|
+
time_key = if @data_stream
|
202
|
+
"@timestamp"
|
203
|
+
else
|
204
|
+
:timestamp
|
205
|
+
end
|
206
|
+
|
207
|
+
SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: time_key)
|
190
208
|
end
|
191
209
|
end
|
192
210
|
end
|
@@ -1,3 +1,4 @@
|
|
1
|
+
require "date"
|
1
2
|
# File appender
|
2
3
|
#
|
3
4
|
# Writes log messages to a file or open iostream
|
@@ -5,105 +6,285 @@
|
|
5
6
|
module SemanticLogger
|
6
7
|
module Appender
|
7
8
|
class File < SemanticLogger::Subscriber
|
8
|
-
|
9
|
+
attr_accessor :file_name, :retry_count, :append, :exclusive_lock, :encoding,
|
10
|
+
:reopen_period, :reopen_count, :reopen_size
|
11
|
+
attr_reader :log_count, :log_size, :current_file_name, :reopen_at
|
12
|
+
|
13
|
+
# Create an appender to log to a named file.
|
9
14
|
#
|
10
15
|
# Parameters
|
11
|
-
#
|
12
|
-
#
|
13
|
-
# Or,
|
14
|
-
# :io [IO]
|
15
|
-
# An IO stream to which to write the log messages to.
|
16
|
-
#
|
17
|
-
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
18
|
-
# Override the log level for this appender.
|
19
|
-
# Default: SemanticLogger.default_level
|
20
|
-
#
|
21
|
-
# :formatter: [Object|Proc]
|
22
|
-
# An instance of a class that implements #call, or a Proc to be used to format
|
23
|
-
# the output from this appender
|
24
|
-
# Default: Use the built-in formatter (See: #call)
|
25
|
-
#
|
26
|
-
# :filter [Regexp|Proc]
|
27
|
-
# RegExp: Only include log messages where the class name matches the supplied
|
28
|
-
# regular expression. All other messages will be ignored.
|
29
|
-
# Proc: Only include log messages where the supplied Proc returns true
|
30
|
-
# The Proc must return true or false.
|
16
|
+
# file_name [String]
|
17
|
+
# Name of the file to write to.
|
31
18
|
#
|
32
|
-
#
|
33
|
-
#
|
19
|
+
# File name format directives:
|
20
|
+
# %p - Process Id
|
21
|
+
# %n - Short hostname (SemanticLogger.host). Everything before the first period in the hostname.
|
22
|
+
# %N - Full hostname (SemanticLogger.host)
|
23
|
+
# %a - Application name (SemanticLogger.application)
|
24
|
+
# %e - Environment name (SemanticLogger.environment)
|
25
|
+
# %D - Current Date. Equivalent to "%Y%m%d"
|
26
|
+
# %T - Current Time. Equivalent to "%H%M%S"
|
27
|
+
# %% - Literal `%` character
|
34
28
|
#
|
35
|
-
#
|
36
|
-
#
|
29
|
+
# Date:
|
30
|
+
# %Y - Year with century
|
31
|
+
# %C - year / 100 (round down. 20 in 2009)
|
32
|
+
# %y - year % 100 (00..99)
|
33
|
+
# %m - Month of the year, zero-padded (01..12)
|
34
|
+
# %d - Day of the month, zero-padded (01..31)
|
35
|
+
# %j - Day of the year (001..366)
|
36
|
+
# %U - Week number of the year. The week starts with Sunday. (00..53)
|
37
|
+
# %W - Week number of the year. The week starts with Monday. (00..53)
|
37
38
|
#
|
38
|
-
#
|
39
|
-
#
|
39
|
+
# Time:
|
40
|
+
# %H - 24 Hour of the day, zero-padded (00..23)
|
41
|
+
# %M - Minute of the hour (00..59)
|
42
|
+
# %S - Second of the minute (00..60)
|
40
43
|
#
|
41
|
-
#
|
42
|
-
#
|
44
|
+
# Examples:
|
45
|
+
# Create a log file name consisting of the short host name, process id, date, and time.
|
46
|
+
# "log/production-%n-%p-%D-%T.log"
|
43
47
|
#
|
44
|
-
#
|
45
|
-
#
|
48
|
+
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
49
|
+
# Override the log level for this appender.
|
50
|
+
# Default: SemanticLogger.default_level
|
46
51
|
#
|
47
|
-
#
|
52
|
+
# :formatter: [Object|Proc]
|
53
|
+
# An instance of a class that implements #call, or a Proc to be used to format
|
54
|
+
# the output from this appender
|
55
|
+
# Default: Use the built-in formatter (See: #call)
|
48
56
|
#
|
49
|
-
#
|
57
|
+
# :filter [Regexp|Proc]
|
58
|
+
# RegExp: Only include log messages where the class name matches the supplied
|
59
|
+
# regular expression. All other messages will be ignored.
|
60
|
+
# Proc: Only include log messages where the supplied Proc returns true
|
61
|
+
# The Proc must return true or false.
|
50
62
|
#
|
51
|
-
#
|
52
|
-
#
|
63
|
+
# :append [true|false]
|
64
|
+
# Append to the log file if already present?
|
65
|
+
# Default: true
|
53
66
|
#
|
54
|
-
#
|
55
|
-
#
|
67
|
+
# :exclusive_lock [true|false]
|
68
|
+
# Obtain an exclusive lock on the file, for operating systems that support it.
|
69
|
+
# Prevents multiple processes from trying to write to the same log file.
|
70
|
+
# Default: false
|
56
71
|
#
|
57
|
-
#
|
58
|
-
#
|
72
|
+
# :encoding ["UTF-8", "UTF-16", etc.]
|
73
|
+
# Encoding to use when writing to the file.
|
74
|
+
# Default: Encoding::BINARY
|
59
75
|
#
|
60
|
-
#
|
61
|
-
#
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
76
|
+
# :retry_count [Integer]
|
77
|
+
# Number of times to attempt to re-open the file name when an error occurs trying to
|
78
|
+
# write to the file.
|
79
|
+
# Note: Set to 0 to disable retries.
|
80
|
+
# Default: 1
|
81
|
+
#
|
82
|
+
# :reopen_period [String]
|
83
|
+
# Specify a period after which to re-open the log file, specified in minutes, hours, or days.
|
84
|
+
# The format of the duration must start with an Integer or Float number,
|
85
|
+
# followed by the duration specified as:
|
86
|
+
# "m" : minutes
|
87
|
+
# "h" : hours
|
88
|
+
# "d" : days
|
89
|
+
# The time is rounded down to the specified time interval, so that:
|
90
|
+
# - "1h" will re-open every hour at the beginning of the hour.
|
91
|
+
# - "30m" will re-open every 30 minutes at the beginning of the 30th minute.
|
92
|
+
# - "1d" will re-open every day at midnight.
|
93
|
+
# Examples:
|
94
|
+
# "60m" : Every 60 minutes at the beginning of the minute: 10:24:00, 11:24:00, 12:24:00, ...
|
95
|
+
# "1h" : Every hour at the beginning of the hour: 10:00:00, 11:00:00, 12:00:00, ...
|
96
|
+
# "1d" : Every day at the beginning of the day: "20211008 00:00:00", "20211009 00:00:00", ...
|
97
|
+
# Default: nil (Disabled)
|
98
|
+
#
|
99
|
+
# :reopen_count [Integer]
|
100
|
+
# Close and re-open the log file after every `reopen_count` number of logged entries.
|
101
|
+
# Default: 0 (Disabled)
|
102
|
+
#
|
103
|
+
# :reopen_size [Integer]
|
104
|
+
# Approximate number of bytes to write to a log file by this process before closing and re-opening it.
|
105
|
+
# Notes:
|
106
|
+
# - When `append: true` and the file already exists, it reads the size of the current log file
|
107
|
+
# and starts with that size.
|
108
|
+
# - If the current log file size already exceeds the `reopen_size`, its current size is ignored.
|
109
|
+
# - The `reopen_size` is only the amount of bytes written by this process, it excludes data
|
110
|
+
# written by other processes. Use a unique filename to prevent multiple processes from writing to
|
111
|
+
# the same log file at the same time.
|
112
|
+
# Default: 0 (Disabled)
|
113
|
+
#
|
114
|
+
# Example
|
115
|
+
# require "semantic_logger"
|
116
|
+
#
|
117
|
+
# # Enable trace level logging
|
118
|
+
# SemanticLogger.default_level = :info
|
119
|
+
#
|
120
|
+
# # Log to a file
|
121
|
+
# SemanticLogger.add_appender(file_name: "application.log", formatter: :color)
|
122
|
+
#
|
123
|
+
# logger = SemanticLogger["test"]
|
124
|
+
# logger.info "Hello World"
|
125
|
+
def initialize(file_name, retry_count: 1, append: true, reopen_period: nil, reopen_count: 0, reopen_size: 0, encoding: Encoding::BINARY, exclusive_lock: false, **args, &block)
|
126
|
+
if !file_name.is_a?(String) || file_name.empty?
|
127
|
+
raise(ArgumentError, "SemanticLogging::Appender::File file_name must be a non-empty string")
|
70
128
|
end
|
71
129
|
|
130
|
+
@file_name = file_name
|
131
|
+
@retry_count = retry_count
|
132
|
+
@file = nil
|
133
|
+
@append = append
|
134
|
+
@reopen_period = reopen_period
|
135
|
+
@reopen_count = reopen_count
|
136
|
+
@reopen_size = reopen_size
|
137
|
+
@encoding = encoding
|
138
|
+
@exclusive_lock = exclusive_lock
|
139
|
+
@log_count = 0
|
140
|
+
@log_size = 0
|
141
|
+
@reopen_at = nil
|
142
|
+
|
72
143
|
super(**args, &block)
|
73
144
|
end
|
74
145
|
|
75
146
|
# After forking an active process call #reopen to re-open
|
76
|
-
# open the file handles etc to resources
|
77
|
-
#
|
78
|
-
# Note: This method will only work if :file_name was supplied
|
79
|
-
# on the initializer.
|
80
|
-
# If :io was supplied, it will need to be re-opened manually.
|
147
|
+
# open the file handles etc to resources.
|
81
148
|
def reopen
|
82
|
-
|
149
|
+
begin
|
150
|
+
@file&.close
|
151
|
+
rescue StandardError
|
152
|
+
nil
|
153
|
+
end
|
154
|
+
|
155
|
+
self.current_file_name = apply_format_directives(file_name)
|
156
|
+
if ::File.directory?(file_name)
|
157
|
+
raise(ArgumentError, "The supplied log file_name: #{current_file_name} is already a directory.")
|
158
|
+
end
|
159
|
+
|
160
|
+
self.log_count = 0
|
161
|
+
if append && reopen_size && ::File.exist?(current_file_name)
|
162
|
+
self.log_size = ::File.size(current_file_name)
|
163
|
+
self.log_size = 0 if log_size >= reopen_size
|
164
|
+
else
|
165
|
+
self.log_size = 0
|
166
|
+
end
|
167
|
+
|
168
|
+
self.reopen_at = reopen_period ? next_reopen_period(reopen_period) : nil
|
83
169
|
|
84
|
-
|
170
|
+
options = ::File::WRONLY | ::File::CREAT
|
171
|
+
options |= ::File::APPEND if append
|
172
|
+
@file = ::File.open(current_file_name, options)
|
85
173
|
# Force all log entries to write immediately without buffering
|
86
174
|
# Allows multiple processes to write to the same log file simultaneously
|
87
|
-
@
|
88
|
-
@
|
89
|
-
@
|
175
|
+
@file.sync = true
|
176
|
+
@file.set_encoding(encoding) if @file.respond_to?(:set_encoding)
|
177
|
+
@file.flock(::File::LOCK_EX) if exclusive_lock
|
178
|
+
@file
|
90
179
|
end
|
91
180
|
|
92
|
-
#
|
93
|
-
#
|
94
|
-
# Ruby or Rails Loggers
|
181
|
+
# Since only one appender thread will be writing to the file at a time
|
182
|
+
# it is not necessary to protect access to the file with a semaphore.
|
95
183
|
def log(log)
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
184
|
+
reopen if time_to_reopen?
|
185
|
+
|
186
|
+
count = 0
|
187
|
+
begin
|
188
|
+
message = formatter.call(log, self) << "\n"
|
189
|
+
@file.write(message)
|
190
|
+
@log_count += 1
|
191
|
+
@log_size += message.size
|
192
|
+
rescue StandardError => e
|
193
|
+
if count < retry_count
|
194
|
+
count += 1
|
195
|
+
reopen
|
196
|
+
retry
|
197
|
+
end
|
198
|
+
raise(e)
|
199
|
+
end
|
100
200
|
true
|
101
201
|
end
|
102
202
|
|
103
203
|
# Flush all pending logs to disk.
|
104
|
-
# Waits for all sent documents to be
|
204
|
+
# Waits for all sent documents to be written to disk
|
105
205
|
def flush
|
106
|
-
@
|
206
|
+
@file&.flush
|
207
|
+
end
|
208
|
+
|
209
|
+
private
|
210
|
+
|
211
|
+
attr_writer :log_count, :log_size, :current_file_name, :reopen_at
|
212
|
+
|
213
|
+
def time_to_reopen?
|
214
|
+
return true unless @file
|
215
|
+
|
216
|
+
(reopen_count.positive? && (log_count >= reopen_count)) ||
|
217
|
+
(reopen_size.positive? && (log_size >= reopen_size)) ||
|
218
|
+
(reopen_at && (Time.now > reopen_at))
|
219
|
+
end
|
220
|
+
|
221
|
+
def apply_format_directives(file_name)
|
222
|
+
return file_name unless file_name.include?("%")
|
223
|
+
|
224
|
+
file_name.gsub(/%(.)/) { format_directive(Regexp.last_match(1)) }
|
225
|
+
end
|
226
|
+
|
227
|
+
def format_directive(directive)
|
228
|
+
case directive
|
229
|
+
when "p"
|
230
|
+
$$
|
231
|
+
when "n"
|
232
|
+
SemanticLogger.host.split(".")[0]
|
233
|
+
when "N"
|
234
|
+
SemanticLogger.host
|
235
|
+
when "a"
|
236
|
+
SemanticLogger.application
|
237
|
+
when "e"
|
238
|
+
SemanticLogger.environment
|
239
|
+
when "D"
|
240
|
+
Date.today.strftime("%Y%m%d")
|
241
|
+
when "Y", "C", "y", "m", "d", "j", "U", "W"
|
242
|
+
Date.today.strftime("%#{directive}")
|
243
|
+
when "T"
|
244
|
+
Time.now.strftime("%H%M%S")
|
245
|
+
when "H", "M", "S"
|
246
|
+
Time.now.strftime("%#{directive}")
|
247
|
+
when "%"
|
248
|
+
"%"
|
249
|
+
else
|
250
|
+
raise(ArgumentError, "Format Directive '#{directive}' in file_name: #{file_name} is not supported.")
|
251
|
+
end
|
252
|
+
end
|
253
|
+
|
254
|
+
def next_reopen_period(period_string)
|
255
|
+
return unless period_string
|
256
|
+
|
257
|
+
duration, period = parse_period(period_string)
|
258
|
+
calculate_reopen_at(duration, period)
|
259
|
+
end
|
260
|
+
|
261
|
+
def parse_period(period_string)
|
262
|
+
match = period_string.to_s.downcase.gsub(/\s+/, "").match(/([\d.]+)([mhd])/)
|
263
|
+
unless match
|
264
|
+
raise(ArgumentError,
|
265
|
+
"Invalid period definition: #{period_string}, must begin with an integer, followed by m,h, or d.")
|
266
|
+
end
|
267
|
+
|
268
|
+
duration = match[1]
|
269
|
+
period = match[2]
|
270
|
+
raise(ArgumentError, "Invalid or missing duration in: #{period_string}, must begin with an integer.") unless duration
|
271
|
+
raise(ArgumentError, "Invalid or missing period in: #{period_string}, must end with m,h, or d.") unless period
|
272
|
+
|
273
|
+
[duration.to_i, period]
|
274
|
+
end
|
275
|
+
|
276
|
+
# Round down the current time based on the period, then add on the duration for that period
|
277
|
+
def calculate_reopen_at(duration, period, time = Time.now)
|
278
|
+
case period
|
279
|
+
when "m"
|
280
|
+
Time.new(time.year, time.month, time.day, time.hour, time.min, 0) + (duration * 60)
|
281
|
+
when "h"
|
282
|
+
Time.new(time.year, time.month, time.day, time.hour, 0, 0) + (duration * 60 * 60)
|
283
|
+
when "d"
|
284
|
+
Time.new(time.year, time.month, time.day, 0, 0, 0) + (duration * 24 * 60 * 60)
|
285
|
+
else
|
286
|
+
raise(ArgumentError, "Invalid or missing period in: #{reopen_period}, must end with m,h, or d.")
|
287
|
+
end
|
107
288
|
end
|
108
289
|
end
|
109
290
|
end
|