semantic_logger 4.7.2 → 4.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +50 -20
- data/lib/semantic_logger/appender/async.rb +1 -1
- data/lib/semantic_logger/appender/async_batch.rb +3 -1
- data/lib/semantic_logger/appender/bugsnag.rb +36 -23
- data/lib/semantic_logger/appender/elasticsearch.rb +24 -6
- data/lib/semantic_logger/appender/file.rb +249 -68
- data/lib/semantic_logger/appender/io.rb +68 -0
- data/lib/semantic_logger/appender/kafka.rb +4 -0
- data/lib/semantic_logger/appender/sentry_ruby.rb +138 -0
- data/lib/semantic_logger/appender/splunk.rb +2 -1
- data/lib/semantic_logger/appender/splunk_http.rb +3 -2
- data/lib/semantic_logger/appender/syslog.rb +5 -3
- data/lib/semantic_logger/appender/wrapper.rb +3 -2
- data/lib/semantic_logger/appender.rb +13 -6
- data/lib/semantic_logger/appenders.rb +31 -27
- data/lib/semantic_logger/base.rb +16 -7
- data/lib/semantic_logger/formatters/base.rb +1 -0
- data/lib/semantic_logger/formatters/color.rb +3 -3
- data/lib/semantic_logger/formatters/logfmt.rb +72 -0
- data/lib/semantic_logger/formatters/syslog.rb +2 -1
- data/lib/semantic_logger/formatters/syslog_cee.rb +2 -1
- data/lib/semantic_logger/formatters.rb +10 -11
- data/lib/semantic_logger/log.rb +2 -4
- data/lib/semantic_logger/loggable.rb +8 -1
- data/lib/semantic_logger/logger.rb +16 -6
- data/lib/semantic_logger/processor.rb +3 -3
- data/lib/semantic_logger/semantic_logger.rb +18 -10
- data/lib/semantic_logger/subscriber.rb +10 -0
- data/lib/semantic_logger/sync_processor.rb +5 -5
- data/lib/semantic_logger/test/capture_log_events.rb +34 -0
- data/lib/semantic_logger/utils.rb +29 -10
- data/lib/semantic_logger/version.rb +1 -1
- data/lib/semantic_logger.rb +4 -0
- metadata +13 -10
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2ff7d4bcb345581f6ba19f834bb1bf2f921e97904f4b6236142b87af75a4526d
|
|
4
|
+
data.tar.gz: b1069d5360296c61c9796f630b1ec065eb1a0a7dc549778c66c5e26c90d0eadb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c5cfbac7dd8795b11cf79ed3fa9a4a086128d5ee030f0e9db40b75f30f58997c1674a4284b4b287939216bc8de83372e2b7fe00d73e7856c3cc6830310740f0d
|
|
7
|
+
data.tar.gz: 2150a809562247d771103a5d387ae5852ff5b0649a4f4288a7dd07a94420bafcfb6b36f3e21077fcc053c2d8bcaffe80a110ab65b52178fb612b7368fd79d87d
|
data/README.md
CHANGED
|
@@ -1,25 +1,13 @@
|
|
|
1
1
|
# Semantic Logger
|
|
2
|
-
[](https://rubygems.org/gems/semantic_logger) [](https://rubygems.org/gems/semantic_logger) [](https://github.com/reidmorrison/semantic_logger/actions?query=workflow%3Abuild) [](https://rubygems.org/gems/semantic_logger) [](http://opensource.org/licenses/Apache-2.0) 
|
|
3
3
|
|
|
4
4
|
Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.
|
|
5
5
|
|
|
6
|
-
* https://rocketjob.
|
|
6
|
+
* https://logger.rocketjob.io/
|
|
7
7
|
|
|
8
8
|
## Documentation
|
|
9
9
|
|
|
10
|
-
[Semantic Logger Guide](
|
|
11
|
-
|
|
12
|
-
[Reference Documentation](http://www.rubydoc.info/gems/semantic_logger/)
|
|
13
|
-
|
|
14
|
-
## Upgrading to Semantic Logger v4.4
|
|
15
|
-
|
|
16
|
-
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
|
17
|
-
workaround for Ruby 2.5 crashes is no longer needed.
|
|
18
|
-
I.e. Please remove the following line if being called anywhere:
|
|
19
|
-
|
|
20
|
-
~~~ruby
|
|
21
|
-
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
|
22
|
-
~~~
|
|
10
|
+
[Semantic Logger Guide](https://logger.rocketjob.io/)
|
|
23
11
|
|
|
24
12
|
## Logging Destinations
|
|
25
13
|
|
|
@@ -34,7 +22,7 @@ Logging to the following destinations are all supported "out-of-the-box":
|
|
|
34
22
|
* Splunk
|
|
35
23
|
* MongoDB
|
|
36
24
|
* Honeybadger
|
|
37
|
-
* Sentry
|
|
25
|
+
* Sentry (both with legacy `sentry-raven` and modern `sentry-ruby` gem)
|
|
38
26
|
* HTTP
|
|
39
27
|
* TCP
|
|
40
28
|
* UDP
|
|
@@ -50,7 +38,7 @@ handles saving log information to multiple destinations / appenders.
|
|
|
50
38
|
|
|
51
39
|
## Rails
|
|
52
40
|
|
|
53
|
-
When running Rails, use [rails_semantic_logger](http://github.com/
|
|
41
|
+
When running Rails, use [rails_semantic_logger](http://github.com/reidmorrison/rails_semantic_logger)
|
|
54
42
|
instead of Semantic Logger directly since it will automatically replace the Rails default logger with Semantic Logger.
|
|
55
43
|
|
|
56
44
|
## Rocket Job
|
|
@@ -72,8 +60,50 @@ and are therefore not automatically included by this gem:
|
|
|
72
60
|
- Splunk Appender: gem 'splunk-sdk-ruby'
|
|
73
61
|
- Elasticsearch Appender: gem 'elasticsearch'
|
|
74
62
|
- Kafka Appender: gem 'ruby-kafka'
|
|
63
|
+
- Legacy Sentry Appender: gem 'sentry-raven' (deprecated)
|
|
64
|
+
- Sentry Appender: gem 'sentry-ruby'
|
|
65
|
+
|
|
66
|
+
## Upgrading to Semantic Logger v4.9
|
|
67
|
+
|
|
68
|
+
These changes should not be noticeable by the majority of users of Semantic Logger, since
|
|
69
|
+
they are to the internal API. It is possible that advanced users may be using these internal
|
|
70
|
+
API's directly.
|
|
71
|
+
|
|
72
|
+
This does not affect any calls to the public api `SemanticLogger.add_appender`.
|
|
73
|
+
|
|
74
|
+
File and IO are now separate appenders. When creating the File appender explicitly, its arguments
|
|
75
|
+
have changed. For example, when requesting an IO stream, it needs to be changed from:
|
|
76
|
+
|
|
77
|
+
~~~ruby
|
|
78
|
+
SemanticLogger::Appender::File.new(io: $stderr)
|
|
79
|
+
~~~
|
|
80
|
+
to:
|
|
81
|
+
~~~ruby
|
|
82
|
+
SemanticLogger::Appender::IO.new($stderr)
|
|
83
|
+
~~~
|
|
84
|
+
|
|
85
|
+
Additionally, this needs to be changed from:
|
|
86
|
+
~~~ruby
|
|
87
|
+
SemanticLogger::Appender::File.new(file_name: "file.log")
|
|
88
|
+
~~~
|
|
89
|
+
to:
|
|
90
|
+
~~~ruby
|
|
91
|
+
SemanticLogger::Appender::File.new("file.log")
|
|
92
|
+
~~~
|
|
93
|
+
|
|
94
|
+
Rails Semantic Logger, if used, needs to be upgraded to v4.9 when upgrading to Semantic Logger v4.9.
|
|
95
|
+
|
|
96
|
+
## Upgrading to Semantic Logger v4.4
|
|
97
|
+
|
|
98
|
+
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
|
99
|
+
workaround for Ruby 2.5 crashes is no longer needed.
|
|
100
|
+
I.e. Please remove the following line if being called anywhere:
|
|
101
|
+
|
|
102
|
+
~~~ruby
|
|
103
|
+
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
|
104
|
+
~~~
|
|
75
105
|
|
|
76
|
-
##
|
|
106
|
+
## Upgrading to Semantic Logger v4.0
|
|
77
107
|
|
|
78
108
|
The following changes need to be made when upgrading to V4:
|
|
79
109
|
- Ruby V2.3 / JRuby V9.1 is now the minimum runtime version.
|
|
@@ -129,13 +159,13 @@ SemanticLogger.default_level = :trace
|
|
|
129
159
|
SemanticLogger.add_appender(file_name: 'development.log', formatter: :color)
|
|
130
160
|
~~~
|
|
131
161
|
|
|
132
|
-
If running rails, see: [Semantic Logger Rails](
|
|
162
|
+
If running rails, see: [Semantic Logger Rails](https://logger.rocketjob.io/rails.html)
|
|
133
163
|
|
|
134
164
|
## Author
|
|
135
165
|
|
|
136
166
|
[Reid Morrison](https://github.com/reidmorrison)
|
|
137
167
|
|
|
138
|
-
[Contributors](https://github.com/
|
|
168
|
+
[Contributors](https://github.com/reidmorrison/semantic_logger/graphs/contributors)
|
|
139
169
|
|
|
140
170
|
## Versioning
|
|
141
171
|
|
|
@@ -53,7 +53,7 @@ module SemanticLogger
|
|
|
53
53
|
# Re-open appender after a fork
|
|
54
54
|
def reopen
|
|
55
55
|
# Workaround CRuby crash on fork by recreating queue on reopen
|
|
56
|
-
# https://github.com/
|
|
56
|
+
# https://github.com/reidmorrison/semantic_logger/issues/103
|
|
57
57
|
@queue&.close
|
|
58
58
|
create_queue
|
|
59
59
|
|
|
@@ -64,6 +64,7 @@ module SemanticLogger
|
|
|
64
64
|
signal.wait(batch_seconds)
|
|
65
65
|
|
|
66
66
|
logs = []
|
|
67
|
+
messages = []
|
|
67
68
|
first = true
|
|
68
69
|
message_count = queue.length
|
|
69
70
|
message_count.times do
|
|
@@ -76,10 +77,11 @@ module SemanticLogger
|
|
|
76
77
|
first = false
|
|
77
78
|
end
|
|
78
79
|
else
|
|
79
|
-
|
|
80
|
+
messages << message
|
|
80
81
|
end
|
|
81
82
|
end
|
|
82
83
|
appender.batch(logs) if logs.size.positive?
|
|
84
|
+
messages.each { |message| process_message(message) }
|
|
83
85
|
signal.reset unless queue.size >= batch_size
|
|
84
86
|
end
|
|
85
87
|
end
|
|
@@ -40,42 +40,55 @@ module SemanticLogger
|
|
|
40
40
|
|
|
41
41
|
# Returns [Hash] of parameters to send to Bugsnag.
|
|
42
42
|
def call(log, logger)
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
43
|
+
hash = SemanticLogger::Formatters::Raw.new.call(log, logger)
|
|
44
|
+
hash.delete(:message) if hash[:exception] && (hash[:message] == hash[:exception][:message])
|
|
45
|
+
hash.delete(:time)
|
|
46
|
+
hash.delete(:level_index)
|
|
47
|
+
hash.delete(:exception)
|
|
48
|
+
hash[:file] = "#{hash[:file]}:#{hash.delete(:line)}" if hash.key?(:file)
|
|
49
|
+
hash
|
|
49
50
|
end
|
|
50
51
|
|
|
51
|
-
# Send an error notification to Bugsnag
|
|
52
52
|
def log(log)
|
|
53
53
|
# Ignore logs coming from Bugsnag itself
|
|
54
54
|
return false if log.name == "Bugsnag"
|
|
55
55
|
|
|
56
56
|
# Send error messages as Runtime exceptions
|
|
57
|
-
exception =
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
|
|
61
|
-
log.exception
|
|
62
|
-
else
|
|
63
|
-
error = RuntimeError.new(log.message)
|
|
64
|
-
error.set_backtrace(log.backtrace) if log.backtrace
|
|
65
|
-
error
|
|
66
|
-
end
|
|
67
|
-
|
|
68
|
-
# For more documentation on the Bugsnag.notify method see:
|
|
69
|
-
# https://bugsnag.com/docs/notifiers/ruby#sending-handled-exceptions
|
|
70
|
-
::Bugsnag.notify(exception, formatter.call(log, self))
|
|
57
|
+
exception = extract_exception(log)
|
|
58
|
+
hash = formatter.call(log, self)
|
|
59
|
+
bugsnag_notify(exception, hash, log_level(log.level))
|
|
71
60
|
true
|
|
72
61
|
end
|
|
73
62
|
|
|
74
63
|
private
|
|
75
64
|
|
|
65
|
+
def bugsnag_notify(exception, hash, level)
|
|
66
|
+
if ::Bugsnag::VERSION.to_i >= 6
|
|
67
|
+
::Bugsnag.notify(exception) do |report|
|
|
68
|
+
report.severity = level
|
|
69
|
+
hash.each_pair { |key, value| report.add_tab(key, value) }
|
|
70
|
+
end
|
|
71
|
+
else
|
|
72
|
+
hash[:severity] = level
|
|
73
|
+
::Bugsnag.notify(exception, hash)
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def extract_exception(log)
|
|
78
|
+
if log.exception
|
|
79
|
+
# Manually constructed Exception, without a backtrace.
|
|
80
|
+
log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
|
|
81
|
+
return log.exception
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
error = RuntimeError.new(log.message)
|
|
85
|
+
error.set_backtrace(log.backtrace) if log.backtrace
|
|
86
|
+
error
|
|
87
|
+
end
|
|
88
|
+
|
|
76
89
|
# Bugsnag supports: error, warning or info
|
|
77
|
-
def log_level(
|
|
78
|
-
case
|
|
90
|
+
def log_level(level)
|
|
91
|
+
case level
|
|
79
92
|
when :error, :fatal
|
|
80
93
|
"error"
|
|
81
94
|
when :warn
|
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
begin
|
|
2
2
|
require "elasticsearch"
|
|
3
3
|
rescue LoadError
|
|
4
|
-
raise LoadError,
|
|
4
|
+
raise LoadError,
|
|
5
|
+
'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
|
|
5
6
|
end
|
|
6
7
|
|
|
7
8
|
require "date"
|
|
@@ -17,7 +18,8 @@ require "date"
|
|
|
17
18
|
module SemanticLogger
|
|
18
19
|
module Appender
|
|
19
20
|
class Elasticsearch < SemanticLogger::Subscriber
|
|
20
|
-
attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
|
|
21
|
+
attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
|
|
22
|
+
:elasticsearch_args
|
|
21
23
|
|
|
22
24
|
# Create Elasticsearch appender over persistent HTTP(S)
|
|
23
25
|
#
|
|
@@ -133,7 +135,7 @@ module SemanticLogger
|
|
|
133
135
|
application: nil,
|
|
134
136
|
environment: nil,
|
|
135
137
|
host: nil,
|
|
136
|
-
|
|
138
|
+
data_stream: false,
|
|
137
139
|
**elasticsearch_args,
|
|
138
140
|
&block)
|
|
139
141
|
|
|
@@ -144,6 +146,7 @@ module SemanticLogger
|
|
|
144
146
|
@elasticsearch_args = elasticsearch_args.dup
|
|
145
147
|
@elasticsearch_args[:url] = url if url && !elasticsearch_args[:hosts]
|
|
146
148
|
@elasticsearch_args[:logger] = logger
|
|
149
|
+
@data_stream = data_stream
|
|
147
150
|
|
|
148
151
|
super(level: level, formatter: formatter, filter: filter, application: application, environment: environment, host: host, metrics: false, &block)
|
|
149
152
|
reopen
|
|
@@ -173,7 +176,12 @@ module SemanticLogger
|
|
|
173
176
|
private
|
|
174
177
|
|
|
175
178
|
def write_to_elasticsearch(messages)
|
|
176
|
-
bulk_result = @
|
|
179
|
+
bulk_result = if @data_stream
|
|
180
|
+
@client.bulk(index: index, body: messages)
|
|
181
|
+
else
|
|
182
|
+
@client.bulk(body: messages)
|
|
183
|
+
end
|
|
184
|
+
|
|
177
185
|
return unless bulk_result["errors"]
|
|
178
186
|
|
|
179
187
|
failed = bulk_result["items"].reject { |x| x["status"] == 201 }
|
|
@@ -182,11 +190,21 @@ module SemanticLogger
|
|
|
182
190
|
|
|
183
191
|
def bulk_index(log)
|
|
184
192
|
expanded_index_name = log.time.strftime("#{index}-#{date_pattern}")
|
|
185
|
-
|
|
193
|
+
if @data_stream
|
|
194
|
+
{"create" => {}}
|
|
195
|
+
else
|
|
196
|
+
{"index" => {"_index" => expanded_index_name, "_type" => type}}
|
|
197
|
+
end
|
|
186
198
|
end
|
|
187
199
|
|
|
188
200
|
def default_formatter
|
|
189
|
-
|
|
201
|
+
time_key = if @data_stream
|
|
202
|
+
"@timestamp"
|
|
203
|
+
else
|
|
204
|
+
:timestamp
|
|
205
|
+
end
|
|
206
|
+
|
|
207
|
+
SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: time_key)
|
|
190
208
|
end
|
|
191
209
|
end
|
|
192
210
|
end
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
require "date"
|
|
1
2
|
# File appender
|
|
2
3
|
#
|
|
3
4
|
# Writes log messages to a file or open iostream
|
|
@@ -5,105 +6,285 @@
|
|
|
5
6
|
module SemanticLogger
|
|
6
7
|
module Appender
|
|
7
8
|
class File < SemanticLogger::Subscriber
|
|
8
|
-
|
|
9
|
+
attr_accessor :file_name, :retry_count, :append, :exclusive_lock, :encoding,
|
|
10
|
+
:reopen_period, :reopen_count, :reopen_size
|
|
11
|
+
attr_reader :log_count, :log_size, :current_file_name, :reopen_at
|
|
12
|
+
|
|
13
|
+
# Create an appender to log to a named file.
|
|
9
14
|
#
|
|
10
15
|
# Parameters
|
|
11
|
-
#
|
|
12
|
-
#
|
|
13
|
-
# Or,
|
|
14
|
-
# :io [IO]
|
|
15
|
-
# An IO stream to which to write the log messages to.
|
|
16
|
-
#
|
|
17
|
-
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
|
18
|
-
# Override the log level for this appender.
|
|
19
|
-
# Default: SemanticLogger.default_level
|
|
20
|
-
#
|
|
21
|
-
# :formatter: [Object|Proc]
|
|
22
|
-
# An instance of a class that implements #call, or a Proc to be used to format
|
|
23
|
-
# the output from this appender
|
|
24
|
-
# Default: Use the built-in formatter (See: #call)
|
|
25
|
-
#
|
|
26
|
-
# :filter [Regexp|Proc]
|
|
27
|
-
# RegExp: Only include log messages where the class name matches the supplied
|
|
28
|
-
# regular expression. All other messages will be ignored.
|
|
29
|
-
# Proc: Only include log messages where the supplied Proc returns true
|
|
30
|
-
# The Proc must return true or false.
|
|
16
|
+
# file_name [String]
|
|
17
|
+
# Name of the file to write to.
|
|
31
18
|
#
|
|
32
|
-
#
|
|
33
|
-
#
|
|
19
|
+
# File name format directives:
|
|
20
|
+
# %p - Process Id
|
|
21
|
+
# %n - Short hostname (SemanticLogger.host). Everything before the first period in the hostname.
|
|
22
|
+
# %N - Full hostname (SemanticLogger.host)
|
|
23
|
+
# %a - Application name (SemanticLogger.application)
|
|
24
|
+
# %e - Environment name (SemanticLogger.environment)
|
|
25
|
+
# %D - Current Date. Equivalent to "%Y%m%d"
|
|
26
|
+
# %T - Current Time. Equivalent to "%H%M%S"
|
|
27
|
+
# %% - Literal `%` character
|
|
34
28
|
#
|
|
35
|
-
#
|
|
36
|
-
#
|
|
29
|
+
# Date:
|
|
30
|
+
# %Y - Year with century
|
|
31
|
+
# %C - year / 100 (round down. 20 in 2009)
|
|
32
|
+
# %y - year % 100 (00..99)
|
|
33
|
+
# %m - Month of the year, zero-padded (01..12)
|
|
34
|
+
# %d - Day of the month, zero-padded (01..31)
|
|
35
|
+
# %j - Day of the year (001..366)
|
|
36
|
+
# %U - Week number of the year. The week starts with Sunday. (00..53)
|
|
37
|
+
# %W - Week number of the year. The week starts with Monday. (00..53)
|
|
37
38
|
#
|
|
38
|
-
#
|
|
39
|
-
#
|
|
39
|
+
# Time:
|
|
40
|
+
# %H - 24 Hour of the day, zero-padded (00..23)
|
|
41
|
+
# %M - Minute of the hour (00..59)
|
|
42
|
+
# %S - Second of the minute (00..60)
|
|
40
43
|
#
|
|
41
|
-
#
|
|
42
|
-
#
|
|
44
|
+
# Examples:
|
|
45
|
+
# Create a log file name consisting of the short host name, process id, date, and time.
|
|
46
|
+
# "log/production-%n-%p-%D-%T.log"
|
|
43
47
|
#
|
|
44
|
-
#
|
|
45
|
-
#
|
|
48
|
+
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
|
49
|
+
# Override the log level for this appender.
|
|
50
|
+
# Default: SemanticLogger.default_level
|
|
46
51
|
#
|
|
47
|
-
#
|
|
52
|
+
# :formatter: [Object|Proc]
|
|
53
|
+
# An instance of a class that implements #call, or a Proc to be used to format
|
|
54
|
+
# the output from this appender
|
|
55
|
+
# Default: Use the built-in formatter (See: #call)
|
|
48
56
|
#
|
|
49
|
-
#
|
|
57
|
+
# :filter [Regexp|Proc]
|
|
58
|
+
# RegExp: Only include log messages where the class name matches the supplied
|
|
59
|
+
# regular expression. All other messages will be ignored.
|
|
60
|
+
# Proc: Only include log messages where the supplied Proc returns true
|
|
61
|
+
# The Proc must return true or false.
|
|
50
62
|
#
|
|
51
|
-
#
|
|
52
|
-
#
|
|
63
|
+
# :append [true|false]
|
|
64
|
+
# Append to the log file if already present?
|
|
65
|
+
# Default: true
|
|
53
66
|
#
|
|
54
|
-
#
|
|
55
|
-
#
|
|
67
|
+
# :exclusive_lock [true|false]
|
|
68
|
+
# Obtain an exclusive lock on the file, for operating systems that support it.
|
|
69
|
+
# Prevents multiple processes from trying to write to the same log file.
|
|
70
|
+
# Default: false
|
|
56
71
|
#
|
|
57
|
-
#
|
|
58
|
-
#
|
|
72
|
+
# :encoding ["UTF-8", "UTF-16", etc.]
|
|
73
|
+
# Encoding to use when writing to the file.
|
|
74
|
+
# Default: Encoding::BINARY
|
|
59
75
|
#
|
|
60
|
-
#
|
|
61
|
-
#
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
76
|
+
# :retry_count [Integer]
|
|
77
|
+
# Number of times to attempt to re-open the file name when an error occurs trying to
|
|
78
|
+
# write to the file.
|
|
79
|
+
# Note: Set to 0 to disable retries.
|
|
80
|
+
# Default: 1
|
|
81
|
+
#
|
|
82
|
+
# :reopen_period [String]
|
|
83
|
+
# Specify a period after which to re-open the log file, specified in minutes, hours, or days.
|
|
84
|
+
# The format of the duration must start with an Integer or Float number,
|
|
85
|
+
# followed by the duration specified as:
|
|
86
|
+
# "m" : minutes
|
|
87
|
+
# "h" : hours
|
|
88
|
+
# "d" : days
|
|
89
|
+
# The time is rounded down to the specified time interval, so that:
|
|
90
|
+
# - "1h" will re-open every hour at the beginning of the hour.
|
|
91
|
+
# - "30m" will re-open every 30 minutes at the beginning of the 30th minute.
|
|
92
|
+
# - "1d" will re-open every day at midnight.
|
|
93
|
+
# Examples:
|
|
94
|
+
# "60m" : Every 60 minutes at the beginning of the minute: 10:24:00, 11:24:00, 12:24:00, ...
|
|
95
|
+
# "1h" : Every hour at the beginning of the hour: 10:00:00, 11:00:00, 12:00:00, ...
|
|
96
|
+
# "1d" : Every day at the beginning of the day: "20211008 00:00:00", "20211009 00:00:00", ...
|
|
97
|
+
# Default: nil (Disabled)
|
|
98
|
+
#
|
|
99
|
+
# :reopen_count [Integer]
|
|
100
|
+
# Close and re-open the log file after every `reopen_count` number of logged entries.
|
|
101
|
+
# Default: 0 (Disabled)
|
|
102
|
+
#
|
|
103
|
+
# :reopen_size [Integer]
|
|
104
|
+
# Approximate number of bytes to write to a log file by this process before closing and re-opening it.
|
|
105
|
+
# Notes:
|
|
106
|
+
# - When `append: true` and the file already exists, it reads the size of the current log file
|
|
107
|
+
# and starts with that size.
|
|
108
|
+
# - If the current log file size already exceeds the `reopen_size`, its current size is ignored.
|
|
109
|
+
# - The `reopen_size` is only the amount of bytes written by this process, it excludes data
|
|
110
|
+
# written by other processes. Use a unique filename to prevent multiple processes from writing to
|
|
111
|
+
# the same log file at the same time.
|
|
112
|
+
# Default: 0 (Disabled)
|
|
113
|
+
#
|
|
114
|
+
# Example
|
|
115
|
+
# require "semantic_logger"
|
|
116
|
+
#
|
|
117
|
+
# # Enable trace level logging
|
|
118
|
+
# SemanticLogger.default_level = :info
|
|
119
|
+
#
|
|
120
|
+
# # Log to a file
|
|
121
|
+
# SemanticLogger.add_appender(file_name: "application.log", formatter: :color)
|
|
122
|
+
#
|
|
123
|
+
# logger = SemanticLogger["test"]
|
|
124
|
+
# logger.info "Hello World"
|
|
125
|
+
def initialize(file_name, retry_count: 1, append: true, reopen_period: nil, reopen_count: 0, reopen_size: 0, encoding: Encoding::BINARY, exclusive_lock: false, **args, &block)
|
|
126
|
+
if !file_name.is_a?(String) || file_name.empty?
|
|
127
|
+
raise(ArgumentError, "SemanticLogging::Appender::File file_name must be a non-empty string")
|
|
70
128
|
end
|
|
71
129
|
|
|
130
|
+
@file_name = file_name
|
|
131
|
+
@retry_count = retry_count
|
|
132
|
+
@file = nil
|
|
133
|
+
@append = append
|
|
134
|
+
@reopen_period = reopen_period
|
|
135
|
+
@reopen_count = reopen_count
|
|
136
|
+
@reopen_size = reopen_size
|
|
137
|
+
@encoding = encoding
|
|
138
|
+
@exclusive_lock = exclusive_lock
|
|
139
|
+
@log_count = 0
|
|
140
|
+
@log_size = 0
|
|
141
|
+
@reopen_at = nil
|
|
142
|
+
|
|
72
143
|
super(**args, &block)
|
|
73
144
|
end
|
|
74
145
|
|
|
75
146
|
# After forking an active process call #reopen to re-open
|
|
76
|
-
# open the file handles etc to resources
|
|
77
|
-
#
|
|
78
|
-
# Note: This method will only work if :file_name was supplied
|
|
79
|
-
# on the initializer.
|
|
80
|
-
# If :io was supplied, it will need to be re-opened manually.
|
|
147
|
+
# open the file handles etc to resources.
|
|
81
148
|
def reopen
|
|
82
|
-
|
|
149
|
+
begin
|
|
150
|
+
@file&.close
|
|
151
|
+
rescue StandardError
|
|
152
|
+
nil
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
self.current_file_name = apply_format_directives(file_name)
|
|
156
|
+
if ::File.directory?(file_name)
|
|
157
|
+
raise(ArgumentError, "The supplied log file_name: #{current_file_name} is already a directory.")
|
|
158
|
+
end
|
|
159
|
+
|
|
160
|
+
self.log_count = 0
|
|
161
|
+
if append && reopen_size && ::File.exist?(current_file_name)
|
|
162
|
+
self.log_size = ::File.size(current_file_name)
|
|
163
|
+
self.log_size = 0 if log_size >= reopen_size
|
|
164
|
+
else
|
|
165
|
+
self.log_size = 0
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
self.reopen_at = reopen_period ? next_reopen_period(reopen_period) : nil
|
|
83
169
|
|
|
84
|
-
|
|
170
|
+
options = ::File::WRONLY | ::File::CREAT
|
|
171
|
+
options |= ::File::APPEND if append
|
|
172
|
+
@file = ::File.open(current_file_name, options)
|
|
85
173
|
# Force all log entries to write immediately without buffering
|
|
86
174
|
# Allows multiple processes to write to the same log file simultaneously
|
|
87
|
-
@
|
|
88
|
-
@
|
|
89
|
-
@
|
|
175
|
+
@file.sync = true
|
|
176
|
+
@file.set_encoding(encoding) if @file.respond_to?(:set_encoding)
|
|
177
|
+
@file.flock(::File::LOCK_EX) if exclusive_lock
|
|
178
|
+
@file
|
|
90
179
|
end
|
|
91
180
|
|
|
92
|
-
#
|
|
93
|
-
#
|
|
94
|
-
# Ruby or Rails Loggers
|
|
181
|
+
# Since only one appender thread will be writing to the file at a time
|
|
182
|
+
# it is not necessary to protect access to the file with a semaphore.
|
|
95
183
|
def log(log)
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
184
|
+
reopen if time_to_reopen?
|
|
185
|
+
|
|
186
|
+
count = 0
|
|
187
|
+
begin
|
|
188
|
+
message = formatter.call(log, self) << "\n"
|
|
189
|
+
@file.write(message)
|
|
190
|
+
@log_count += 1
|
|
191
|
+
@log_size += message.size
|
|
192
|
+
rescue StandardError => e
|
|
193
|
+
if count < retry_count
|
|
194
|
+
count += 1
|
|
195
|
+
reopen
|
|
196
|
+
retry
|
|
197
|
+
end
|
|
198
|
+
raise(e)
|
|
199
|
+
end
|
|
100
200
|
true
|
|
101
201
|
end
|
|
102
202
|
|
|
103
203
|
# Flush all pending logs to disk.
|
|
104
|
-
# Waits for all sent documents to be
|
|
204
|
+
# Waits for all sent documents to be written to disk
|
|
105
205
|
def flush
|
|
106
|
-
@
|
|
206
|
+
@file&.flush
|
|
207
|
+
end
|
|
208
|
+
|
|
209
|
+
private
|
|
210
|
+
|
|
211
|
+
attr_writer :log_count, :log_size, :current_file_name, :reopen_at
|
|
212
|
+
|
|
213
|
+
def time_to_reopen?
|
|
214
|
+
return true unless @file
|
|
215
|
+
|
|
216
|
+
(reopen_count.positive? && (log_count >= reopen_count)) ||
|
|
217
|
+
(reopen_size.positive? && (log_size >= reopen_size)) ||
|
|
218
|
+
(reopen_at && (Time.now > reopen_at))
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
def apply_format_directives(file_name)
|
|
222
|
+
return file_name unless file_name.include?("%")
|
|
223
|
+
|
|
224
|
+
file_name.gsub(/%(.)/) { format_directive(Regexp.last_match(1)) }
|
|
225
|
+
end
|
|
226
|
+
|
|
227
|
+
def format_directive(directive)
|
|
228
|
+
case directive
|
|
229
|
+
when "p"
|
|
230
|
+
$$
|
|
231
|
+
when "n"
|
|
232
|
+
SemanticLogger.host.split(".")[0]
|
|
233
|
+
when "N"
|
|
234
|
+
SemanticLogger.host
|
|
235
|
+
when "a"
|
|
236
|
+
SemanticLogger.application
|
|
237
|
+
when "e"
|
|
238
|
+
SemanticLogger.environment
|
|
239
|
+
when "D"
|
|
240
|
+
Date.today.strftime("%Y%m%d")
|
|
241
|
+
when "Y", "C", "y", "m", "d", "j", "U", "W"
|
|
242
|
+
Date.today.strftime("%#{directive}")
|
|
243
|
+
when "T"
|
|
244
|
+
Time.now.strftime("%H%M%S")
|
|
245
|
+
when "H", "M", "S"
|
|
246
|
+
Time.now.strftime("%#{directive}")
|
|
247
|
+
when "%"
|
|
248
|
+
"%"
|
|
249
|
+
else
|
|
250
|
+
raise(ArgumentError, "Format Directive '#{directive}' in file_name: #{file_name} is not supported.")
|
|
251
|
+
end
|
|
252
|
+
end
|
|
253
|
+
|
|
254
|
+
def next_reopen_period(period_string)
|
|
255
|
+
return unless period_string
|
|
256
|
+
|
|
257
|
+
duration, period = parse_period(period_string)
|
|
258
|
+
calculate_reopen_at(duration, period)
|
|
259
|
+
end
|
|
260
|
+
|
|
261
|
+
def parse_period(period_string)
|
|
262
|
+
match = period_string.to_s.downcase.gsub(/\s+/, "").match(/([\d.]+)([mhd])/)
|
|
263
|
+
unless match
|
|
264
|
+
raise(ArgumentError,
|
|
265
|
+
"Invalid period definition: #{period_string}, must begin with an integer, followed by m,h, or d.")
|
|
266
|
+
end
|
|
267
|
+
|
|
268
|
+
duration = match[1]
|
|
269
|
+
period = match[2]
|
|
270
|
+
raise(ArgumentError, "Invalid or missing duration in: #{period_string}, must begin with an integer.") unless duration
|
|
271
|
+
raise(ArgumentError, "Invalid or missing period in: #{period_string}, must end with m,h, or d.") unless period
|
|
272
|
+
|
|
273
|
+
[duration.to_i, period]
|
|
274
|
+
end
|
|
275
|
+
|
|
276
|
+
# Round down the current time based on the period, then add on the duration for that period
|
|
277
|
+
def calculate_reopen_at(duration, period, time = Time.now)
|
|
278
|
+
case period
|
|
279
|
+
when "m"
|
|
280
|
+
Time.new(time.year, time.month, time.day, time.hour, time.min, 0) + (duration * 60)
|
|
281
|
+
when "h"
|
|
282
|
+
Time.new(time.year, time.month, time.day, time.hour, 0, 0) + (duration * 60 * 60)
|
|
283
|
+
when "d"
|
|
284
|
+
Time.new(time.year, time.month, time.day, 0, 0, 0) + (duration * 24 * 60 * 60)
|
|
285
|
+
else
|
|
286
|
+
raise(ArgumentError, "Invalid or missing period in: #{reopen_period}, must end with m,h, or d.")
|
|
287
|
+
end
|
|
107
288
|
end
|
|
108
289
|
end
|
|
109
290
|
end
|