semantic_logger 4.7.4 → 4.9.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +47 -19
- data/lib/semantic_logger/appender/async.rb +1 -1
- data/lib/semantic_logger/appender/elasticsearch.rb +24 -6
- data/lib/semantic_logger/appender/file.rb +249 -68
- data/lib/semantic_logger/appender/io.rb +68 -0
- data/lib/semantic_logger/appender/kafka.rb +4 -0
- data/lib/semantic_logger/appender/splunk.rb +2 -1
- data/lib/semantic_logger/appender/splunk_http.rb +2 -2
- data/lib/semantic_logger/appender/syslog.rb +5 -3
- data/lib/semantic_logger/appender/wrapper.rb +3 -2
- data/lib/semantic_logger/appender.rb +12 -6
- data/lib/semantic_logger/appenders.rb +31 -27
- data/lib/semantic_logger/base.rb +5 -4
- data/lib/semantic_logger/formatters/base.rb +1 -0
- data/lib/semantic_logger/formatters/color.rb +3 -3
- data/lib/semantic_logger/formatters/logfmt.rb +72 -0
- data/lib/semantic_logger/formatters/syslog.rb +2 -1
- data/lib/semantic_logger/formatters/syslog_cee.rb +2 -1
- data/lib/semantic_logger/formatters.rb +10 -11
- data/lib/semantic_logger/log.rb +2 -4
- data/lib/semantic_logger/loggable.rb +8 -1
- data/lib/semantic_logger/logger.rb +16 -6
- data/lib/semantic_logger/processor.rb +3 -3
- data/lib/semantic_logger/semantic_logger.rb +18 -10
- data/lib/semantic_logger/subscriber.rb +10 -0
- data/lib/semantic_logger/sync_processor.rb +5 -5
- data/lib/semantic_logger/test/capture_log_events.rb +34 -0
- data/lib/semantic_logger/utils.rb +29 -10
- data/lib/semantic_logger/version.rb +1 -1
- data/lib/semantic_logger.rb +4 -0
- metadata +8 -6
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9e586c9e92e30d49d498a13ac996e1ac4f17f51f0da5817354866c64d9d5ddab
|
4
|
+
data.tar.gz: 27daca7b6f2af20d521c004aa64df670a6026b8e845e89f022924375fc85e70e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: aef0b209aadb844bfbd27c8fca2119172119037cf73c4b3254d273f66f382114890c81617536b3a29aa6b3d19e65ba855aa48658ea86ca9c6df630561cb3cadf
|
7
|
+
data.tar.gz: cd341bbb661bf9c3ece3c95d63feef49c18af016fe062d3bd03a1127586598d9f3c086137be456113448b5d4d310b5261dbf96bf42213c8ad1093323a7d321e0
|
data/README.md
CHANGED
@@ -1,25 +1,13 @@
|
|
1
1
|
# Semantic Logger
|
2
|
-
[![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://
|
2
|
+
[![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://github.com/reidmorrison/semantic_logger/workflows/build/badge.svg)](https://github.com/reidmorrison/semantic_logger/actions?query=workflow%3Abuild) [![Downloads](https://img.shields.io/gem/dt/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg)
|
3
3
|
|
4
4
|
Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.
|
5
5
|
|
6
|
-
* https://rocketjob.
|
6
|
+
* https://logger.rocketjob.io/
|
7
7
|
|
8
8
|
## Documentation
|
9
9
|
|
10
|
-
[Semantic Logger Guide](
|
11
|
-
|
12
|
-
[Reference Documentation](http://www.rubydoc.info/gems/semantic_logger/)
|
13
|
-
|
14
|
-
## Upgrading to Semantic Logger v4.4
|
15
|
-
|
16
|
-
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
17
|
-
workaround for Ruby 2.5 crashes is no longer needed.
|
18
|
-
I.e. Please remove the following line if being called anywhere:
|
19
|
-
|
20
|
-
~~~ruby
|
21
|
-
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
22
|
-
~~~
|
10
|
+
[Semantic Logger Guide](https://logger.rocketjob.io/)
|
23
11
|
|
24
12
|
## Logging Destinations
|
25
13
|
|
@@ -50,7 +38,7 @@ handles saving log information to multiple destinations / appenders.
|
|
50
38
|
|
51
39
|
## Rails
|
52
40
|
|
53
|
-
When running Rails, use [rails_semantic_logger](http://github.com/
|
41
|
+
When running Rails, use [rails_semantic_logger](http://github.com/reidmorrison/rails_semantic_logger)
|
54
42
|
instead of Semantic Logger directly since it will automatically replace the Rails default logger with Semantic Logger.
|
55
43
|
|
56
44
|
## Rocket Job
|
@@ -73,7 +61,47 @@ and are therefore not automatically included by this gem:
|
|
73
61
|
- Elasticsearch Appender: gem 'elasticsearch'
|
74
62
|
- Kafka Appender: gem 'ruby-kafka'
|
75
63
|
|
76
|
-
##
|
64
|
+
## Upgrading to Semantic Logger v4.9
|
65
|
+
|
66
|
+
These changes should not be noticeable by the majority of users of Semantic Logger, since
|
67
|
+
they are to the internal API. It is possible that advanced users may be using these internal
|
68
|
+
API's directly.
|
69
|
+
|
70
|
+
This does not affect any calls to the public api `SemanticLogger.add_appender`.
|
71
|
+
|
72
|
+
File and IO are now separate appenders. When creating the File appender explicitly, its arguments
|
73
|
+
have changed. For example, when requesting an IO stream, it needs to be changed from:
|
74
|
+
|
75
|
+
~~~ruby
|
76
|
+
SemanticLogger::Appender::File.new(io: $stderr)
|
77
|
+
~~~
|
78
|
+
to:
|
79
|
+
~~~ruby
|
80
|
+
SemanticLogger::Appender::IO.new($stderr)
|
81
|
+
~~~
|
82
|
+
|
83
|
+
Additionally, this needs to be changed from:
|
84
|
+
~~~ruby
|
85
|
+
SemanticLogger::Appender::File.new(file_name: "file.log")
|
86
|
+
~~~
|
87
|
+
to:
|
88
|
+
~~~ruby
|
89
|
+
SemanticLogger::Appender::File.new("file.log")
|
90
|
+
~~~
|
91
|
+
|
92
|
+
Rails Semantic Logger, if used, needs to be upgraded to v4.9 when upgrading to Semantic Logger v4.9.
|
93
|
+
|
94
|
+
## Upgrading to Semantic Logger v4.4
|
95
|
+
|
96
|
+
With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
|
97
|
+
workaround for Ruby 2.5 crashes is no longer needed.
|
98
|
+
I.e. Please remove the following line if being called anywhere:
|
99
|
+
|
100
|
+
~~~ruby
|
101
|
+
SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
|
102
|
+
~~~
|
103
|
+
|
104
|
+
## Upgrading to Semantic Logger v4.0
|
77
105
|
|
78
106
|
The following changes need to be made when upgrading to V4:
|
79
107
|
- Ruby V2.3 / JRuby V9.1 is now the minimum runtime version.
|
@@ -129,13 +157,13 @@ SemanticLogger.default_level = :trace
|
|
129
157
|
SemanticLogger.add_appender(file_name: 'development.log', formatter: :color)
|
130
158
|
~~~
|
131
159
|
|
132
|
-
If running rails, see: [Semantic Logger Rails](
|
160
|
+
If running rails, see: [Semantic Logger Rails](https://logger.rocketjob.io/rails.html)
|
133
161
|
|
134
162
|
## Author
|
135
163
|
|
136
164
|
[Reid Morrison](https://github.com/reidmorrison)
|
137
165
|
|
138
|
-
[Contributors](https://github.com/
|
166
|
+
[Contributors](https://github.com/reidmorrison/semantic_logger/graphs/contributors)
|
139
167
|
|
140
168
|
## Versioning
|
141
169
|
|
@@ -53,7 +53,7 @@ module SemanticLogger
|
|
53
53
|
# Re-open appender after a fork
|
54
54
|
def reopen
|
55
55
|
# Workaround CRuby crash on fork by recreating queue on reopen
|
56
|
-
# https://github.com/
|
56
|
+
# https://github.com/reidmorrison/semantic_logger/issues/103
|
57
57
|
@queue&.close
|
58
58
|
create_queue
|
59
59
|
|
@@ -1,7 +1,8 @@
|
|
1
1
|
begin
|
2
2
|
require "elasticsearch"
|
3
3
|
rescue LoadError
|
4
|
-
raise LoadError,
|
4
|
+
raise LoadError,
|
5
|
+
'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
|
5
6
|
end
|
6
7
|
|
7
8
|
require "date"
|
@@ -17,7 +18,8 @@ require "date"
|
|
17
18
|
module SemanticLogger
|
18
19
|
module Appender
|
19
20
|
class Elasticsearch < SemanticLogger::Subscriber
|
20
|
-
attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
|
21
|
+
attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
|
22
|
+
:elasticsearch_args
|
21
23
|
|
22
24
|
# Create Elasticsearch appender over persistent HTTP(S)
|
23
25
|
#
|
@@ -133,7 +135,7 @@ module SemanticLogger
|
|
133
135
|
application: nil,
|
134
136
|
environment: nil,
|
135
137
|
host: nil,
|
136
|
-
|
138
|
+
data_stream: false,
|
137
139
|
**elasticsearch_args,
|
138
140
|
&block)
|
139
141
|
|
@@ -144,6 +146,7 @@ module SemanticLogger
|
|
144
146
|
@elasticsearch_args = elasticsearch_args.dup
|
145
147
|
@elasticsearch_args[:url] = url if url && !elasticsearch_args[:hosts]
|
146
148
|
@elasticsearch_args[:logger] = logger
|
149
|
+
@data_stream = data_stream
|
147
150
|
|
148
151
|
super(level: level, formatter: formatter, filter: filter, application: application, environment: environment, host: host, metrics: false, &block)
|
149
152
|
reopen
|
@@ -173,7 +176,12 @@ module SemanticLogger
|
|
173
176
|
private
|
174
177
|
|
175
178
|
def write_to_elasticsearch(messages)
|
176
|
-
bulk_result = @
|
179
|
+
bulk_result = if @data_stream
|
180
|
+
@client.bulk(index: index, body: messages)
|
181
|
+
else
|
182
|
+
@client.bulk(body: messages)
|
183
|
+
end
|
184
|
+
|
177
185
|
return unless bulk_result["errors"]
|
178
186
|
|
179
187
|
failed = bulk_result["items"].reject { |x| x["status"] == 201 }
|
@@ -182,11 +190,21 @@ module SemanticLogger
|
|
182
190
|
|
183
191
|
def bulk_index(log)
|
184
192
|
expanded_index_name = log.time.strftime("#{index}-#{date_pattern}")
|
185
|
-
|
193
|
+
if @data_stream
|
194
|
+
{"create" => {}}
|
195
|
+
else
|
196
|
+
{"index" => {"_index" => expanded_index_name, "_type" => type}}
|
197
|
+
end
|
186
198
|
end
|
187
199
|
|
188
200
|
def default_formatter
|
189
|
-
|
201
|
+
time_key = if @data_stream
|
202
|
+
"@timestamp"
|
203
|
+
else
|
204
|
+
:timestamp
|
205
|
+
end
|
206
|
+
|
207
|
+
SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: time_key)
|
190
208
|
end
|
191
209
|
end
|
192
210
|
end
|
@@ -1,3 +1,4 @@
|
|
1
|
+
require "date"
|
1
2
|
# File appender
|
2
3
|
#
|
3
4
|
# Writes log messages to a file or open iostream
|
@@ -5,105 +6,285 @@
|
|
5
6
|
module SemanticLogger
|
6
7
|
module Appender
|
7
8
|
class File < SemanticLogger::Subscriber
|
8
|
-
|
9
|
+
attr_accessor :file_name, :retry_count, :append, :exclusive_lock, :encoding,
|
10
|
+
:reopen_period, :reopen_count, :reopen_size
|
11
|
+
attr_reader :log_count, :log_size, :current_file_name, :reopen_at
|
12
|
+
|
13
|
+
# Create an appender to log to a named file.
|
9
14
|
#
|
10
15
|
# Parameters
|
11
|
-
#
|
12
|
-
#
|
13
|
-
# Or,
|
14
|
-
# :io [IO]
|
15
|
-
# An IO stream to which to write the log messages to.
|
16
|
-
#
|
17
|
-
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
18
|
-
# Override the log level for this appender.
|
19
|
-
# Default: SemanticLogger.default_level
|
20
|
-
#
|
21
|
-
# :formatter: [Object|Proc]
|
22
|
-
# An instance of a class that implements #call, or a Proc to be used to format
|
23
|
-
# the output from this appender
|
24
|
-
# Default: Use the built-in formatter (See: #call)
|
25
|
-
#
|
26
|
-
# :filter [Regexp|Proc]
|
27
|
-
# RegExp: Only include log messages where the class name matches the supplied
|
28
|
-
# regular expression. All other messages will be ignored.
|
29
|
-
# Proc: Only include log messages where the supplied Proc returns true
|
30
|
-
# The Proc must return true or false.
|
16
|
+
# file_name [String]
|
17
|
+
# Name of the file to write to.
|
31
18
|
#
|
32
|
-
#
|
33
|
-
#
|
19
|
+
# File name format directives:
|
20
|
+
# %p - Process Id
|
21
|
+
# %n - Short hostname (SemanticLogger.host). Everything before the first period in the hostname.
|
22
|
+
# %N - Full hostname (SemanticLogger.host)
|
23
|
+
# %a - Application name (SemanticLogger.application)
|
24
|
+
# %e - Environment name (SemanticLogger.environment)
|
25
|
+
# %D - Current Date. Equivalent to "%Y%m%d"
|
26
|
+
# %T - Current Time. Equivalent to "%H%M%S"
|
27
|
+
# %% - Literal `%` character
|
34
28
|
#
|
35
|
-
#
|
36
|
-
#
|
29
|
+
# Date:
|
30
|
+
# %Y - Year with century
|
31
|
+
# %C - year / 100 (round down. 20 in 2009)
|
32
|
+
# %y - year % 100 (00..99)
|
33
|
+
# %m - Month of the year, zero-padded (01..12)
|
34
|
+
# %d - Day of the month, zero-padded (01..31)
|
35
|
+
# %j - Day of the year (001..366)
|
36
|
+
# %U - Week number of the year. The week starts with Sunday. (00..53)
|
37
|
+
# %W - Week number of the year. The week starts with Monday. (00..53)
|
37
38
|
#
|
38
|
-
#
|
39
|
-
#
|
39
|
+
# Time:
|
40
|
+
# %H - 24 Hour of the day, zero-padded (00..23)
|
41
|
+
# %M - Minute of the hour (00..59)
|
42
|
+
# %S - Second of the minute (00..60)
|
40
43
|
#
|
41
|
-
#
|
42
|
-
#
|
44
|
+
# Examples:
|
45
|
+
# Create a log file name consisting of the short host name, process id, date, and time.
|
46
|
+
# "log/production-%n-%p-%D-%T.log"
|
43
47
|
#
|
44
|
-
#
|
45
|
-
#
|
48
|
+
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
49
|
+
# Override the log level for this appender.
|
50
|
+
# Default: SemanticLogger.default_level
|
46
51
|
#
|
47
|
-
#
|
52
|
+
# :formatter: [Object|Proc]
|
53
|
+
# An instance of a class that implements #call, or a Proc to be used to format
|
54
|
+
# the output from this appender
|
55
|
+
# Default: Use the built-in formatter (See: #call)
|
48
56
|
#
|
49
|
-
#
|
57
|
+
# :filter [Regexp|Proc]
|
58
|
+
# RegExp: Only include log messages where the class name matches the supplied
|
59
|
+
# regular expression. All other messages will be ignored.
|
60
|
+
# Proc: Only include log messages where the supplied Proc returns true
|
61
|
+
# The Proc must return true or false.
|
50
62
|
#
|
51
|
-
#
|
52
|
-
#
|
63
|
+
# :append [true|false]
|
64
|
+
# Append to the log file if already present?
|
65
|
+
# Default: true
|
53
66
|
#
|
54
|
-
#
|
55
|
-
#
|
67
|
+
# :exclusive_lock [true|false]
|
68
|
+
# Obtain an exclusive lock on the file, for operating systems that support it.
|
69
|
+
# Prevents multiple processes from trying to write to the same log file.
|
70
|
+
# Default: false
|
56
71
|
#
|
57
|
-
#
|
58
|
-
#
|
72
|
+
# :encoding ["UTF-8", "UTF-16", etc.]
|
73
|
+
# Encoding to use when writing to the file.
|
74
|
+
# Default: Encoding::BINARY
|
59
75
|
#
|
60
|
-
#
|
61
|
-
#
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
76
|
+
# :retry_count [Integer]
|
77
|
+
# Number of times to attempt to re-open the file name when an error occurs trying to
|
78
|
+
# write to the file.
|
79
|
+
# Note: Set to 0 to disable retries.
|
80
|
+
# Default: 1
|
81
|
+
#
|
82
|
+
# :reopen_period [String]
|
83
|
+
# Specify a period after which to re-open the log file, specified in minutes, hours, or days.
|
84
|
+
# The format of the duration must start with an Integer or Float number,
|
85
|
+
# followed by the duration specified as:
|
86
|
+
# "m" : minutes
|
87
|
+
# "h" : hours
|
88
|
+
# "d" : days
|
89
|
+
# The time is rounded down to the specified time interval, so that:
|
90
|
+
# - "1h" will re-open every hour at the beginning of the hour.
|
91
|
+
# - "30m" will re-open every 30 minutes at the beginning of the 30th minute.
|
92
|
+
# - "1d" will re-open every day at midnight.
|
93
|
+
# Examples:
|
94
|
+
# "60m" : Every 60 minutes at the beginning of the minute: 10:24:00, 11:24:00, 12:24:00, ...
|
95
|
+
# "1h" : Every hour at the beginning of the hour: 10:00:00, 11:00:00, 12:00:00, ...
|
96
|
+
# "1d" : Every day at the beginning of the day: "20211008 00:00:00", "20211009 00:00:00", ...
|
97
|
+
# Default: nil (Disabled)
|
98
|
+
#
|
99
|
+
# :reopen_count [Integer]
|
100
|
+
# Close and re-open the log file after every `reopen_count` number of logged entries.
|
101
|
+
# Default: 0 (Disabled)
|
102
|
+
#
|
103
|
+
# :reopen_size [Integer]
|
104
|
+
# Approximate number of bytes to write to a log file by this process before closing and re-opening it.
|
105
|
+
# Notes:
|
106
|
+
# - When `append: true` and the file already exists, it reads the size of the current log file
|
107
|
+
# and starts with that size.
|
108
|
+
# - If the current log file size already exceeds the `reopen_size`, its current size is ignored.
|
109
|
+
# - The `reopen_size` is only the amount of bytes written by this process, it excludes data
|
110
|
+
# written by other processes. Use a unique filename to prevent multiple processes from writing to
|
111
|
+
# the same log file at the same time.
|
112
|
+
# Default: 0 (Disabled)
|
113
|
+
#
|
114
|
+
# Example
|
115
|
+
# require "semantic_logger"
|
116
|
+
#
|
117
|
+
# # Enable trace level logging
|
118
|
+
# SemanticLogger.default_level = :info
|
119
|
+
#
|
120
|
+
# # Log to a file
|
121
|
+
# SemanticLogger.add_appender(file_name: "application.log", formatter: :color)
|
122
|
+
#
|
123
|
+
# logger = SemanticLogger["test"]
|
124
|
+
# logger.info "Hello World"
|
125
|
+
def initialize(file_name, retry_count: 1, append: true, reopen_period: nil, reopen_count: 0, reopen_size: 0, encoding: Encoding::BINARY, exclusive_lock: false, **args, &block)
|
126
|
+
if !file_name.is_a?(String) || file_name.empty?
|
127
|
+
raise(ArgumentError, "SemanticLogging::Appender::File file_name must be a non-empty string")
|
70
128
|
end
|
71
129
|
|
130
|
+
@file_name = file_name
|
131
|
+
@retry_count = retry_count
|
132
|
+
@file = nil
|
133
|
+
@append = append
|
134
|
+
@reopen_period = reopen_period
|
135
|
+
@reopen_count = reopen_count
|
136
|
+
@reopen_size = reopen_size
|
137
|
+
@encoding = encoding
|
138
|
+
@exclusive_lock = exclusive_lock
|
139
|
+
@log_count = 0
|
140
|
+
@log_size = 0
|
141
|
+
@reopen_at = nil
|
142
|
+
|
72
143
|
super(**args, &block)
|
73
144
|
end
|
74
145
|
|
75
146
|
# After forking an active process call #reopen to re-open
|
76
|
-
# open the file handles etc to resources
|
77
|
-
#
|
78
|
-
# Note: This method will only work if :file_name was supplied
|
79
|
-
# on the initializer.
|
80
|
-
# If :io was supplied, it will need to be re-opened manually.
|
147
|
+
# open the file handles etc to resources.
|
81
148
|
def reopen
|
82
|
-
|
149
|
+
begin
|
150
|
+
@file&.close
|
151
|
+
rescue StandardError
|
152
|
+
nil
|
153
|
+
end
|
154
|
+
|
155
|
+
self.current_file_name = apply_format_directives(file_name)
|
156
|
+
if ::File.directory?(file_name)
|
157
|
+
raise(ArgumentError, "The supplied log file_name: #{current_file_name} is already a directory.")
|
158
|
+
end
|
159
|
+
|
160
|
+
self.log_count = 0
|
161
|
+
if append && reopen_size && ::File.exist?(current_file_name)
|
162
|
+
self.log_size = ::File.size(current_file_name)
|
163
|
+
self.log_size = 0 if log_size >= reopen_size
|
164
|
+
else
|
165
|
+
self.log_size = 0
|
166
|
+
end
|
167
|
+
|
168
|
+
self.reopen_at = reopen_period ? next_reopen_period(reopen_period) : nil
|
83
169
|
|
84
|
-
|
170
|
+
options = ::File::WRONLY | ::File::CREAT
|
171
|
+
options |= ::File::APPEND if append
|
172
|
+
@file = ::File.open(current_file_name, options)
|
85
173
|
# Force all log entries to write immediately without buffering
|
86
174
|
# Allows multiple processes to write to the same log file simultaneously
|
87
|
-
@
|
88
|
-
@
|
89
|
-
@
|
175
|
+
@file.sync = true
|
176
|
+
@file.set_encoding(encoding) if @file.respond_to?(:set_encoding)
|
177
|
+
@file.flock(::File::LOCK_EX) if exclusive_lock
|
178
|
+
@file
|
90
179
|
end
|
91
180
|
|
92
|
-
#
|
93
|
-
#
|
94
|
-
# Ruby or Rails Loggers
|
181
|
+
# Since only one appender thread will be writing to the file at a time
|
182
|
+
# it is not necessary to protect access to the file with a semaphore.
|
95
183
|
def log(log)
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
184
|
+
reopen if time_to_reopen?
|
185
|
+
|
186
|
+
count = 0
|
187
|
+
begin
|
188
|
+
message = formatter.call(log, self) << "\n"
|
189
|
+
@file.write(message)
|
190
|
+
@log_count += 1
|
191
|
+
@log_size += message.size
|
192
|
+
rescue StandardError => e
|
193
|
+
if count < retry_count
|
194
|
+
count += 1
|
195
|
+
reopen
|
196
|
+
retry
|
197
|
+
end
|
198
|
+
raise(e)
|
199
|
+
end
|
100
200
|
true
|
101
201
|
end
|
102
202
|
|
103
203
|
# Flush all pending logs to disk.
|
104
|
-
# Waits for all sent documents to be
|
204
|
+
# Waits for all sent documents to be written to disk
|
105
205
|
def flush
|
106
|
-
@
|
206
|
+
@file&.flush
|
207
|
+
end
|
208
|
+
|
209
|
+
private
|
210
|
+
|
211
|
+
attr_writer :log_count, :log_size, :current_file_name, :reopen_at
|
212
|
+
|
213
|
+
def time_to_reopen?
|
214
|
+
return true unless @file
|
215
|
+
|
216
|
+
(reopen_count.positive? && (log_count >= reopen_count)) ||
|
217
|
+
(reopen_size.positive? && (log_size >= reopen_size)) ||
|
218
|
+
(reopen_at && (Time.now > reopen_at))
|
219
|
+
end
|
220
|
+
|
221
|
+
def apply_format_directives(file_name)
|
222
|
+
return file_name unless file_name.include?("%")
|
223
|
+
|
224
|
+
file_name.gsub(/%(.)/) { format_directive(Regexp.last_match(1)) }
|
225
|
+
end
|
226
|
+
|
227
|
+
def format_directive(directive)
|
228
|
+
case directive
|
229
|
+
when "p"
|
230
|
+
$$
|
231
|
+
when "n"
|
232
|
+
SemanticLogger.host.split(".")[0]
|
233
|
+
when "N"
|
234
|
+
SemanticLogger.host
|
235
|
+
when "a"
|
236
|
+
SemanticLogger.application
|
237
|
+
when "e"
|
238
|
+
SemanticLogger.environment
|
239
|
+
when "D"
|
240
|
+
Date.today.strftime("%Y%m%d")
|
241
|
+
when "Y", "C", "y", "m", "d", "j", "U", "W"
|
242
|
+
Date.today.strftime("%#{directive}")
|
243
|
+
when "T"
|
244
|
+
Time.now.strftime("%H%M%S")
|
245
|
+
when "H", "M", "S"
|
246
|
+
Time.now.strftime("%#{directive}")
|
247
|
+
when "%"
|
248
|
+
"%"
|
249
|
+
else
|
250
|
+
raise(ArgumentError, "Format Directive '#{directive}' in file_name: #{file_name} is not supported.")
|
251
|
+
end
|
252
|
+
end
|
253
|
+
|
254
|
+
def next_reopen_period(period_string)
|
255
|
+
return unless period_string
|
256
|
+
|
257
|
+
duration, period = parse_period(period_string)
|
258
|
+
calculate_reopen_at(duration, period)
|
259
|
+
end
|
260
|
+
|
261
|
+
def parse_period(period_string)
|
262
|
+
match = period_string.to_s.downcase.gsub(/\s+/, "").match(/([\d.]+)([mhd])/)
|
263
|
+
unless match
|
264
|
+
raise(ArgumentError,
|
265
|
+
"Invalid period definition: #{period_string}, must begin with an integer, followed by m,h, or d.")
|
266
|
+
end
|
267
|
+
|
268
|
+
duration = match[1]
|
269
|
+
period = match[2]
|
270
|
+
raise(ArgumentError, "Invalid or missing duration in: #{period_string}, must begin with an integer.") unless duration
|
271
|
+
raise(ArgumentError, "Invalid or missing period in: #{period_string}, must end with m,h, or d.") unless period
|
272
|
+
|
273
|
+
[duration.to_i, period]
|
274
|
+
end
|
275
|
+
|
276
|
+
# Round down the current time based on the period, then add on the duration for that period
|
277
|
+
def calculate_reopen_at(duration, period, time = Time.now)
|
278
|
+
case period
|
279
|
+
when "m"
|
280
|
+
Time.new(time.year, time.month, time.day, time.hour, time.min, 0) + (duration * 60)
|
281
|
+
when "h"
|
282
|
+
Time.new(time.year, time.month, time.day, time.hour, 0, 0) + (duration * 60 * 60)
|
283
|
+
when "d"
|
284
|
+
Time.new(time.year, time.month, time.day, 0, 0, 0) + (duration * 24 * 60 * 60)
|
285
|
+
else
|
286
|
+
raise(ArgumentError, "Invalid or missing period in: #{reopen_period}, must end with m,h, or d.")
|
287
|
+
end
|
107
288
|
end
|
108
289
|
end
|
109
290
|
end
|
@@ -0,0 +1,68 @@
|
|
1
|
+
# File appender
|
2
|
+
#
|
3
|
+
# Writes log messages to a file or open iostream
|
4
|
+
#
|
5
|
+
module SemanticLogger
|
6
|
+
module Appender
|
7
|
+
class IO < SemanticLogger::Subscriber
|
8
|
+
# Create a Stream Logger appender instance.
|
9
|
+
#
|
10
|
+
# Parameters
|
11
|
+
# io [IO]
|
12
|
+
# An IO stream to which to write the log messages to.
|
13
|
+
#
|
14
|
+
# :level [:trace | :debug | :info | :warn | :error | :fatal]
|
15
|
+
# Override the log level for this appender.
|
16
|
+
# Default: SemanticLogger.default_level
|
17
|
+
#
|
18
|
+
# :formatter: [Object|Proc]
|
19
|
+
# An instance of a class that implements #call, or a Proc to be used to format
|
20
|
+
# the output from this appender
|
21
|
+
# Default: Use the built-in formatter (See: #call)
|
22
|
+
#
|
23
|
+
# :filter [Regexp|Proc]
|
24
|
+
# RegExp: Only include log messages where the class name matches the supplied
|
25
|
+
# regular expression. All other messages will be ignored.
|
26
|
+
# Proc: Only include log messages where the supplied Proc returns true
|
27
|
+
# The Proc must return true or false.
|
28
|
+
#
|
29
|
+
# Example
|
30
|
+
# require "semantic_logger"
|
31
|
+
#
|
32
|
+
# # Enable trace level logging
|
33
|
+
# SemanticLogger.default_level = :info
|
34
|
+
#
|
35
|
+
# # Log to screen
|
36
|
+
# SemanticLogger.add_appender(io: $stdout, formatter: :color)
|
37
|
+
#
|
38
|
+
# logger = SemanticLogger['test']
|
39
|
+
# logger.info 'Hello World'
|
40
|
+
def initialize(io, **args, &block)
|
41
|
+
@io = io
|
42
|
+
unless @io.respond_to?(:write)
|
43
|
+
raise(ArgumentError, "SemanticLogging::Appender::IO io is not a valid IO instance: #{io.inspect}")
|
44
|
+
end
|
45
|
+
|
46
|
+
super(**args, &block)
|
47
|
+
end
|
48
|
+
|
49
|
+
def log(log)
|
50
|
+
# Since only one appender thread will be writing to the file at a time
|
51
|
+
# it is not necessary to protect access to the file with a semaphore
|
52
|
+
# Allow this logger to filter out log levels lower than it's own
|
53
|
+
@io.write(formatter.call(log, self) << "\n")
|
54
|
+
true
|
55
|
+
end
|
56
|
+
|
57
|
+
# Flush all pending logs to disk.
|
58
|
+
# Waits for all sent documents to be written to disk
|
59
|
+
def flush
|
60
|
+
@io.flush if @io.respond_to?(:flush)
|
61
|
+
end
|
62
|
+
|
63
|
+
def console_output?
|
64
|
+
[$stderr, $stdout].include?(@io)
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
@@ -1,7 +1,8 @@
|
|
1
1
|
begin
|
2
2
|
require "splunk-sdk-ruby"
|
3
3
|
rescue LoadError
|
4
|
-
raise LoadError,
|
4
|
+
raise LoadError,
|
5
|
+
'Gem splunk-sdk-ruby is required for logging to Splunk. Please add the gem "splunk-sdk-ruby" to your Gemfile.'
|
5
6
|
end
|
6
7
|
|
7
8
|
# Splunk log appender.
|
@@ -89,9 +89,9 @@ module SemanticLogger
|
|
89
89
|
# For splunk format requirements see:
|
90
90
|
# https://docs.splunk.com/Documentation/Splunk/latest/Data/FormateventsforHTTPEventCollector
|
91
91
|
def call(log, logger)
|
92
|
-
h
|
92
|
+
h = SemanticLogger::Formatters::Raw.new(time_format: :seconds).call(log, logger)
|
93
93
|
h.delete(:host)
|
94
|
-
message
|
94
|
+
message = {
|
95
95
|
source: logger.application,
|
96
96
|
host: logger.host,
|
97
97
|
time: h.delete(:time),
|