semantic_logger 4.7.2 → 4.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +50 -20
  3. data/lib/semantic_logger/appender/async.rb +1 -1
  4. data/lib/semantic_logger/appender/async_batch.rb +3 -1
  5. data/lib/semantic_logger/appender/bugsnag.rb +36 -23
  6. data/lib/semantic_logger/appender/elasticsearch.rb +24 -6
  7. data/lib/semantic_logger/appender/file.rb +249 -68
  8. data/lib/semantic_logger/appender/io.rb +68 -0
  9. data/lib/semantic_logger/appender/kafka.rb +4 -0
  10. data/lib/semantic_logger/appender/sentry_ruby.rb +138 -0
  11. data/lib/semantic_logger/appender/splunk.rb +2 -1
  12. data/lib/semantic_logger/appender/splunk_http.rb +3 -2
  13. data/lib/semantic_logger/appender/syslog.rb +5 -3
  14. data/lib/semantic_logger/appender/wrapper.rb +3 -2
  15. data/lib/semantic_logger/appender.rb +13 -6
  16. data/lib/semantic_logger/appenders.rb +31 -27
  17. data/lib/semantic_logger/base.rb +16 -7
  18. data/lib/semantic_logger/formatters/base.rb +1 -0
  19. data/lib/semantic_logger/formatters/color.rb +3 -3
  20. data/lib/semantic_logger/formatters/logfmt.rb +72 -0
  21. data/lib/semantic_logger/formatters/syslog.rb +2 -1
  22. data/lib/semantic_logger/formatters/syslog_cee.rb +2 -1
  23. data/lib/semantic_logger/formatters.rb +10 -11
  24. data/lib/semantic_logger/log.rb +2 -4
  25. data/lib/semantic_logger/loggable.rb +8 -1
  26. data/lib/semantic_logger/logger.rb +16 -6
  27. data/lib/semantic_logger/processor.rb +3 -3
  28. data/lib/semantic_logger/semantic_logger.rb +18 -10
  29. data/lib/semantic_logger/subscriber.rb +10 -0
  30. data/lib/semantic_logger/sync_processor.rb +5 -5
  31. data/lib/semantic_logger/test/capture_log_events.rb +34 -0
  32. data/lib/semantic_logger/utils.rb +29 -10
  33. data/lib/semantic_logger/version.rb +1 -1
  34. data/lib/semantic_logger.rb +4 -0
  35. metadata +13 -10
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c3a735a9fbaff1f869e8beb81bf7196688a87f94bb40f8fb7861ce505f60d8a1
4
- data.tar.gz: 0e5f6075ca9cd65fed94330affc7180c5bc638832391e967b5d7493d26f53e90
3
+ metadata.gz: 2ff7d4bcb345581f6ba19f834bb1bf2f921e97904f4b6236142b87af75a4526d
4
+ data.tar.gz: b1069d5360296c61c9796f630b1ec065eb1a0a7dc549778c66c5e26c90d0eadb
5
5
  SHA512:
6
- metadata.gz: 27046a2405e9662215ac6ed66d6db4a68171301c65f9c3ff06500bfedb1bf3dc2a18c2e67bbedbe705db54b26ff412623556a9f24b1b69b2a2508f54413d1951
7
- data.tar.gz: 43db220b7e11cf718f73bcc8343c3f60f77ad63569fba509ebb1d47531055fae87c2233b2de95ed942216bb2e741f77c5ab8902ccf9686f6caf540c3a8efd27f
6
+ metadata.gz: c5cfbac7dd8795b11cf79ed3fa9a4a086128d5ee030f0e9db40b75f30f58997c1674a4284b4b287939216bc8de83372e2b7fe00d73e7856c3cc6830310740f0d
7
+ data.tar.gz: 2150a809562247d771103a5d387ae5852ff5b0649a4f4288a7dd07a94420bafcfb6b36f3e21077fcc053c2d8bcaffe80a110ab65b52178fb612b7368fd79d87d
data/README.md CHANGED
@@ -1,25 +1,13 @@
1
1
  # Semantic Logger
2
- [![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://travis-ci.org/rocketjob/semantic_logger.svg?branch=master)](https://travis-ci.org/rocketjob/semantic_logger) [![Downloads](https://img.shields.io/gem/dt/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg) [![Gitter chat](https://img.shields.io/badge/IRC%20(gitter)-Support-brightgreen.svg)](https://gitter.im/rocketjob/support)
2
+ [![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://github.com/reidmorrison/semantic_logger/workflows/build/badge.svg)](https://github.com/reidmorrison/semantic_logger/actions?query=workflow%3Abuild) [![Downloads](https://img.shields.io/gem/dt/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg)
3
3
 
4
4
  Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.
5
5
 
6
- * https://rocketjob.github.io/semantic_logger/
6
+ * https://logger.rocketjob.io/
7
7
 
8
8
  ## Documentation
9
9
 
10
- [Semantic Logger Guide](http://rocketjob.github.io/semantic_logger)
11
-
12
- [Reference Documentation](http://www.rubydoc.info/gems/semantic_logger/)
13
-
14
- ## Upgrading to Semantic Logger v4.4
15
-
16
- With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
17
- workaround for Ruby 2.5 crashes is no longer needed.
18
- I.e. Please remove the following line if being called anywhere:
19
-
20
- ~~~ruby
21
- SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
22
- ~~~
10
+ [Semantic Logger Guide](https://logger.rocketjob.io/)
23
11
 
24
12
  ## Logging Destinations
25
13
 
@@ -34,7 +22,7 @@ Logging to the following destinations are all supported "out-of-the-box":
34
22
  * Splunk
35
23
  * MongoDB
36
24
  * Honeybadger
37
- * Sentry
25
+ * Sentry (both with legacy `sentry-raven` and modern `sentry-ruby` gem)
38
26
  * HTTP
39
27
  * TCP
40
28
  * UDP
@@ -50,7 +38,7 @@ handles saving log information to multiple destinations / appenders.
50
38
 
51
39
  ## Rails
52
40
 
53
- When running Rails, use [rails_semantic_logger](http://github.com/rocketjob/rails_semantic_logger)
41
+ When running Rails, use [rails_semantic_logger](http://github.com/reidmorrison/rails_semantic_logger)
54
42
  instead of Semantic Logger directly since it will automatically replace the Rails default logger with Semantic Logger.
55
43
 
56
44
  ## Rocket Job
@@ -72,8 +60,50 @@ and are therefore not automatically included by this gem:
72
60
  - Splunk Appender: gem 'splunk-sdk-ruby'
73
61
  - Elasticsearch Appender: gem 'elasticsearch'
74
62
  - Kafka Appender: gem 'ruby-kafka'
63
+ - Legacy Sentry Appender: gem 'sentry-raven' (deprecated)
64
+ - Sentry Appender: gem 'sentry-ruby'
65
+
66
+ ## Upgrading to Semantic Logger v4.9
67
+
68
+ These changes should not be noticeable by the majority of users of Semantic Logger, since
69
+ they are to the internal API. It is possible that advanced users may be using these internal
70
+ API's directly.
71
+
72
+ This does not affect any calls to the public api `SemanticLogger.add_appender`.
73
+
74
+ File and IO are now separate appenders. When creating the File appender explicitly, its arguments
75
+ have changed. For example, when requesting an IO stream, it needs to be changed from:
76
+
77
+ ~~~ruby
78
+ SemanticLogger::Appender::File.new(io: $stderr)
79
+ ~~~
80
+ to:
81
+ ~~~ruby
82
+ SemanticLogger::Appender::IO.new($stderr)
83
+ ~~~
84
+
85
+ Additionally, this needs to be changed from:
86
+ ~~~ruby
87
+ SemanticLogger::Appender::File.new(file_name: "file.log")
88
+ ~~~
89
+ to:
90
+ ~~~ruby
91
+ SemanticLogger::Appender::File.new("file.log")
92
+ ~~~
93
+
94
+ Rails Semantic Logger, if used, needs to be upgraded to v4.9 when upgrading to Semantic Logger v4.9.
95
+
96
+ ## Upgrading to Semantic Logger v4.4
97
+
98
+ With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
99
+ workaround for Ruby 2.5 crashes is no longer needed.
100
+ I.e. Please remove the following line if being called anywhere:
101
+
102
+ ~~~ruby
103
+ SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
104
+ ~~~
75
105
 
76
- ## V4 Upgrade notes
106
+ ## Upgrading to Semantic Logger v4.0
77
107
 
78
108
  The following changes need to be made when upgrading to V4:
79
109
  - Ruby V2.3 / JRuby V9.1 is now the minimum runtime version.
@@ -129,13 +159,13 @@ SemanticLogger.default_level = :trace
129
159
  SemanticLogger.add_appender(file_name: 'development.log', formatter: :color)
130
160
  ~~~
131
161
 
132
- If running rails, see: [Semantic Logger Rails](http://rocketjob.github.io/semantic_logger/rails.html)
162
+ If running rails, see: [Semantic Logger Rails](https://logger.rocketjob.io/rails.html)
133
163
 
134
164
  ## Author
135
165
 
136
166
  [Reid Morrison](https://github.com/reidmorrison)
137
167
 
138
- [Contributors](https://github.com/rocketjob/semantic_logger/graphs/contributors)
168
+ [Contributors](https://github.com/reidmorrison/semantic_logger/graphs/contributors)
139
169
 
140
170
  ## Versioning
141
171
 
@@ -53,7 +53,7 @@ module SemanticLogger
53
53
  # Re-open appender after a fork
54
54
  def reopen
55
55
  # Workaround CRuby crash on fork by recreating queue on reopen
56
- # https://github.com/rocketjob/semantic_logger/issues/103
56
+ # https://github.com/reidmorrison/semantic_logger/issues/103
57
57
  @queue&.close
58
58
  create_queue
59
59
 
@@ -64,6 +64,7 @@ module SemanticLogger
64
64
  signal.wait(batch_seconds)
65
65
 
66
66
  logs = []
67
+ messages = []
67
68
  first = true
68
69
  message_count = queue.length
69
70
  message_count.times do
@@ -76,10 +77,11 @@ module SemanticLogger
76
77
  first = false
77
78
  end
78
79
  else
79
- process_message(message)
80
+ messages << message
80
81
  end
81
82
  end
82
83
  appender.batch(logs) if logs.size.positive?
84
+ messages.each { |message| process_message(message) }
83
85
  signal.reset unless queue.size >= batch_size
84
86
  end
85
87
  end
@@ -40,42 +40,55 @@ module SemanticLogger
40
40
 
41
41
  # Returns [Hash] of parameters to send to Bugsnag.
42
42
  def call(log, logger)
43
- h = SemanticLogger::Formatters::Raw.new.call(log, logger)
44
- h[:severity] = log_level(log)
45
- h.delete(:message) if h[:exception] && (h[:message] == h[:exception][:message])
46
- h.delete(:time)
47
- h.delete(:exception)
48
- h
43
+ hash = SemanticLogger::Formatters::Raw.new.call(log, logger)
44
+ hash.delete(:message) if hash[:exception] && (hash[:message] == hash[:exception][:message])
45
+ hash.delete(:time)
46
+ hash.delete(:level_index)
47
+ hash.delete(:exception)
48
+ hash[:file] = "#{hash[:file]}:#{hash.delete(:line)}" if hash.key?(:file)
49
+ hash
49
50
  end
50
51
 
51
- # Send an error notification to Bugsnag
52
52
  def log(log)
53
53
  # Ignore logs coming from Bugsnag itself
54
54
  return false if log.name == "Bugsnag"
55
55
 
56
56
  # Send error messages as Runtime exceptions
57
- exception =
58
- if log.exception
59
- # Manually constructed Exception, without a backtrace.
60
- log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
61
- log.exception
62
- else
63
- error = RuntimeError.new(log.message)
64
- error.set_backtrace(log.backtrace) if log.backtrace
65
- error
66
- end
67
-
68
- # For more documentation on the Bugsnag.notify method see:
69
- # https://bugsnag.com/docs/notifiers/ruby#sending-handled-exceptions
70
- ::Bugsnag.notify(exception, formatter.call(log, self))
57
+ exception = extract_exception(log)
58
+ hash = formatter.call(log, self)
59
+ bugsnag_notify(exception, hash, log_level(log.level))
71
60
  true
72
61
  end
73
62
 
74
63
  private
75
64
 
65
+ def bugsnag_notify(exception, hash, level)
66
+ if ::Bugsnag::VERSION.to_i >= 6
67
+ ::Bugsnag.notify(exception) do |report|
68
+ report.severity = level
69
+ hash.each_pair { |key, value| report.add_tab(key, value) }
70
+ end
71
+ else
72
+ hash[:severity] = level
73
+ ::Bugsnag.notify(exception, hash)
74
+ end
75
+ end
76
+
77
+ def extract_exception(log)
78
+ if log.exception
79
+ # Manually constructed Exception, without a backtrace.
80
+ log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
81
+ return log.exception
82
+ end
83
+
84
+ error = RuntimeError.new(log.message)
85
+ error.set_backtrace(log.backtrace) if log.backtrace
86
+ error
87
+ end
88
+
76
89
  # Bugsnag supports: error, warning or info
77
- def log_level(log)
78
- case log.level
90
+ def log_level(level)
91
+ case level
79
92
  when :error, :fatal
80
93
  "error"
81
94
  when :warn
@@ -1,7 +1,8 @@
1
1
  begin
2
2
  require "elasticsearch"
3
3
  rescue LoadError
4
- raise LoadError, 'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
4
+ raise LoadError,
5
+ 'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
5
6
  end
6
7
 
7
8
  require "date"
@@ -17,7 +18,8 @@ require "date"
17
18
  module SemanticLogger
18
19
  module Appender
19
20
  class Elasticsearch < SemanticLogger::Subscriber
20
- attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size, :elasticsearch_args
21
+ attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
22
+ :elasticsearch_args
21
23
 
22
24
  # Create Elasticsearch appender over persistent HTTP(S)
23
25
  #
@@ -133,7 +135,7 @@ module SemanticLogger
133
135
  application: nil,
134
136
  environment: nil,
135
137
  host: nil,
136
- metrics: false,
138
+ data_stream: false,
137
139
  **elasticsearch_args,
138
140
  &block)
139
141
 
@@ -144,6 +146,7 @@ module SemanticLogger
144
146
  @elasticsearch_args = elasticsearch_args.dup
145
147
  @elasticsearch_args[:url] = url if url && !elasticsearch_args[:hosts]
146
148
  @elasticsearch_args[:logger] = logger
149
+ @data_stream = data_stream
147
150
 
148
151
  super(level: level, formatter: formatter, filter: filter, application: application, environment: environment, host: host, metrics: false, &block)
149
152
  reopen
@@ -173,7 +176,12 @@ module SemanticLogger
173
176
  private
174
177
 
175
178
  def write_to_elasticsearch(messages)
176
- bulk_result = @client.bulk(body: messages)
179
+ bulk_result = if @data_stream
180
+ @client.bulk(index: index, body: messages)
181
+ else
182
+ @client.bulk(body: messages)
183
+ end
184
+
177
185
  return unless bulk_result["errors"]
178
186
 
179
187
  failed = bulk_result["items"].reject { |x| x["status"] == 201 }
@@ -182,11 +190,21 @@ module SemanticLogger
182
190
 
183
191
  def bulk_index(log)
184
192
  expanded_index_name = log.time.strftime("#{index}-#{date_pattern}")
185
- {"index" => {"_index" => expanded_index_name, "_type" => type}}
193
+ if @data_stream
194
+ {"create" => {}}
195
+ else
196
+ {"index" => {"_index" => expanded_index_name, "_type" => type}}
197
+ end
186
198
  end
187
199
 
188
200
  def default_formatter
189
- SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: :timestamp)
201
+ time_key = if @data_stream
202
+ "@timestamp"
203
+ else
204
+ :timestamp
205
+ end
206
+
207
+ SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: time_key)
190
208
  end
191
209
  end
192
210
  end
@@ -1,3 +1,4 @@
1
+ require "date"
1
2
  # File appender
2
3
  #
3
4
  # Writes log messages to a file or open iostream
@@ -5,105 +6,285 @@
5
6
  module SemanticLogger
6
7
  module Appender
7
8
  class File < SemanticLogger::Subscriber
8
- # Create a File Logger appender instance.
9
+ attr_accessor :file_name, :retry_count, :append, :exclusive_lock, :encoding,
10
+ :reopen_period, :reopen_count, :reopen_size
11
+ attr_reader :log_count, :log_size, :current_file_name, :reopen_at
12
+
13
+ # Create an appender to log to a named file.
9
14
  #
10
15
  # Parameters
11
- # :file_name [String]
12
- # Name of file to write to.
13
- # Or,
14
- # :io [IO]
15
- # An IO stream to which to write the log messages to.
16
- #
17
- # :level [:trace | :debug | :info | :warn | :error | :fatal]
18
- # Override the log level for this appender.
19
- # Default: SemanticLogger.default_level
20
- #
21
- # :formatter: [Object|Proc]
22
- # An instance of a class that implements #call, or a Proc to be used to format
23
- # the output from this appender
24
- # Default: Use the built-in formatter (See: #call)
25
- #
26
- # :filter [Regexp|Proc]
27
- # RegExp: Only include log messages where the class name matches the supplied
28
- # regular expression. All other messages will be ignored.
29
- # Proc: Only include log messages where the supplied Proc returns true
30
- # The Proc must return true or false.
16
+ # file_name [String]
17
+ # Name of the file to write to.
31
18
  #
32
- # Example
33
- # require 'semantic_logger'
19
+ # File name format directives:
20
+ # %p - Process Id
21
+ # %n - Short hostname (SemanticLogger.host). Everything before the first period in the hostname.
22
+ # %N - Full hostname (SemanticLogger.host)
23
+ # %a - Application name (SemanticLogger.application)
24
+ # %e - Environment name (SemanticLogger.environment)
25
+ # %D - Current Date. Equivalent to "%Y%m%d"
26
+ # %T - Current Time. Equivalent to "%H%M%S"
27
+ # %% - Literal `%` character
34
28
  #
35
- # # Enable trace level logging
36
- # SemanticLogger.default_level = :info
29
+ # Date:
30
+ # %Y - Year with century
31
+ # %C - year / 100 (round down. 20 in 2009)
32
+ # %y - year % 100 (00..99)
33
+ # %m - Month of the year, zero-padded (01..12)
34
+ # %d - Day of the month, zero-padded (01..31)
35
+ # %j - Day of the year (001..366)
36
+ # %U - Week number of the year. The week starts with Sunday. (00..53)
37
+ # %W - Week number of the year. The week starts with Monday. (00..53)
37
38
  #
38
- # # Log to screen
39
- # SemanticLogger.add_appender(io: STDOUT, formatter: :color)
39
+ # Time:
40
+ # %H - 24 Hour of the day, zero-padded (00..23)
41
+ # %M - Minute of the hour (00..59)
42
+ # %S - Second of the minute (00..60)
40
43
  #
41
- # # And log to a file at the same time
42
- # SemanticLogger.add_appender(file_name: 'application.log', formatter: :color)
44
+ # Examples:
45
+ # Create a log file name consisting of the short host name, process id, date, and time.
46
+ # "log/production-%n-%p-%D-%T.log"
43
47
  #
44
- # logger = SemanticLogger['test']
45
- # logger.info 'Hello World'
48
+ # :level [:trace | :debug | :info | :warn | :error | :fatal]
49
+ # Override the log level for this appender.
50
+ # Default: SemanticLogger.default_level
46
51
  #
47
- # Example 2. To log all levels to file and only :info and above to screen:
52
+ # :formatter: [Object|Proc]
53
+ # An instance of a class that implements #call, or a Proc to be used to format
54
+ # the output from this appender
55
+ # Default: Use the built-in formatter (See: #call)
48
56
  #
49
- # require 'semantic_logger'
57
+ # :filter [Regexp|Proc]
58
+ # RegExp: Only include log messages where the class name matches the supplied
59
+ # regular expression. All other messages will be ignored.
60
+ # Proc: Only include log messages where the supplied Proc returns true
61
+ # The Proc must return true or false.
50
62
  #
51
- # # Enable trace level logging
52
- # SemanticLogger.default_level = :trace
63
+ # :append [true|false]
64
+ # Append to the log file if already present?
65
+ # Default: true
53
66
  #
54
- # # Log to screen but only display :info and above
55
- # SemanticLogger.add_appender(io: STDOUT, level: :info)
67
+ # :exclusive_lock [true|false]
68
+ # Obtain an exclusive lock on the file, for operating systems that support it.
69
+ # Prevents multiple processes from trying to write to the same log file.
70
+ # Default: false
56
71
  #
57
- # # And log to a file at the same time, including all :trace level data
58
- # SemanticLogger.add_appender(file_name: 'application.log')
72
+ # :encoding ["UTF-8", "UTF-16", etc.]
73
+ # Encoding to use when writing to the file.
74
+ # Default: Encoding::BINARY
59
75
  #
60
- # logger = SemanticLogger['test']
61
- # logger.info 'Hello World'
62
- def initialize(io: nil, file_name: nil, **args, &block)
63
- if io
64
- @log = io
65
- else
66
- @file_name = file_name
67
- raise "SemanticLogging::Appender::File missing mandatory parameter :file_name or :io" unless file_name
68
-
69
- reopen
76
+ # :retry_count [Integer]
77
+ # Number of times to attempt to re-open the file name when an error occurs trying to
78
+ # write to the file.
79
+ # Note: Set to 0 to disable retries.
80
+ # Default: 1
81
+ #
82
+ # :reopen_period [String]
83
+ # Specify a period after which to re-open the log file, specified in minutes, hours, or days.
84
+ # The format of the duration must start with an Integer or Float number,
85
+ # followed by the duration specified as:
86
+ # "m" : minutes
87
+ # "h" : hours
88
+ # "d" : days
89
+ # The time is rounded down to the specified time interval, so that:
90
+ # - "1h" will re-open every hour at the beginning of the hour.
91
+ # - "30m" will re-open every 30 minutes at the beginning of the 30th minute.
92
+ # - "1d" will re-open every day at midnight.
93
+ # Examples:
94
+ # "60m" : Every 60 minutes at the beginning of the minute: 10:24:00, 11:24:00, 12:24:00, ...
95
+ # "1h" : Every hour at the beginning of the hour: 10:00:00, 11:00:00, 12:00:00, ...
96
+ # "1d" : Every day at the beginning of the day: "20211008 00:00:00", "20211009 00:00:00", ...
97
+ # Default: nil (Disabled)
98
+ #
99
+ # :reopen_count [Integer]
100
+ # Close and re-open the log file after every `reopen_count` number of logged entries.
101
+ # Default: 0 (Disabled)
102
+ #
103
+ # :reopen_size [Integer]
104
+ # Approximate number of bytes to write to a log file by this process before closing and re-opening it.
105
+ # Notes:
106
+ # - When `append: true` and the file already exists, it reads the size of the current log file
107
+ # and starts with that size.
108
+ # - If the current log file size already exceeds the `reopen_size`, its current size is ignored.
109
+ # - The `reopen_size` is only the amount of bytes written by this process, it excludes data
110
+ # written by other processes. Use a unique filename to prevent multiple processes from writing to
111
+ # the same log file at the same time.
112
+ # Default: 0 (Disabled)
113
+ #
114
+ # Example
115
+ # require "semantic_logger"
116
+ #
117
+ # # Enable trace level logging
118
+ # SemanticLogger.default_level = :info
119
+ #
120
+ # # Log to a file
121
+ # SemanticLogger.add_appender(file_name: "application.log", formatter: :color)
122
+ #
123
+ # logger = SemanticLogger["test"]
124
+ # logger.info "Hello World"
125
+ def initialize(file_name, retry_count: 1, append: true, reopen_period: nil, reopen_count: 0, reopen_size: 0, encoding: Encoding::BINARY, exclusive_lock: false, **args, &block)
126
+ if !file_name.is_a?(String) || file_name.empty?
127
+ raise(ArgumentError, "SemanticLogging::Appender::File file_name must be a non-empty string")
70
128
  end
71
129
 
130
+ @file_name = file_name
131
+ @retry_count = retry_count
132
+ @file = nil
133
+ @append = append
134
+ @reopen_period = reopen_period
135
+ @reopen_count = reopen_count
136
+ @reopen_size = reopen_size
137
+ @encoding = encoding
138
+ @exclusive_lock = exclusive_lock
139
+ @log_count = 0
140
+ @log_size = 0
141
+ @reopen_at = nil
142
+
72
143
  super(**args, &block)
73
144
  end
74
145
 
75
146
  # After forking an active process call #reopen to re-open
76
- # open the file handles etc to resources
77
- #
78
- # Note: This method will only work if :file_name was supplied
79
- # on the initializer.
80
- # If :io was supplied, it will need to be re-opened manually.
147
+ # open the file handles etc to resources.
81
148
  def reopen
82
- return unless @file_name
149
+ begin
150
+ @file&.close
151
+ rescue StandardError
152
+ nil
153
+ end
154
+
155
+ self.current_file_name = apply_format_directives(file_name)
156
+ if ::File.directory?(file_name)
157
+ raise(ArgumentError, "The supplied log file_name: #{current_file_name} is already a directory.")
158
+ end
159
+
160
+ self.log_count = 0
161
+ if append && reopen_size && ::File.exist?(current_file_name)
162
+ self.log_size = ::File.size(current_file_name)
163
+ self.log_size = 0 if log_size >= reopen_size
164
+ else
165
+ self.log_size = 0
166
+ end
167
+
168
+ self.reopen_at = reopen_period ? next_reopen_period(reopen_period) : nil
83
169
 
84
- @log = ::File.open(@file_name, ::File::WRONLY | ::File::APPEND | ::File::CREAT)
170
+ options = ::File::WRONLY | ::File::CREAT
171
+ options |= ::File::APPEND if append
172
+ @file = ::File.open(current_file_name, options)
85
173
  # Force all log entries to write immediately without buffering
86
174
  # Allows multiple processes to write to the same log file simultaneously
87
- @log.sync = true
88
- @log.set_encoding(Encoding::BINARY) if @log.respond_to?(:set_encoding)
89
- @log
175
+ @file.sync = true
176
+ @file.set_encoding(encoding) if @file.respond_to?(:set_encoding)
177
+ @file.flock(::File::LOCK_EX) if exclusive_lock
178
+ @file
90
179
  end
91
180
 
92
- # Pass log calls to the underlying Rails, log4j or Ruby logger
93
- # trace entries are mapped to debug since :trace is not supported by the
94
- # Ruby or Rails Loggers
181
+ # Since only one appender thread will be writing to the file at a time
182
+ # it is not necessary to protect access to the file with a semaphore.
95
183
  def log(log)
96
- # Since only one appender thread will be writing to the file at a time
97
- # it is not necessary to protect access to the file with a semaphore
98
- # Allow this logger to filter out log levels lower than it's own
99
- @log.write(formatter.call(log, self) << "\n")
184
+ reopen if time_to_reopen?
185
+
186
+ count = 0
187
+ begin
188
+ message = formatter.call(log, self) << "\n"
189
+ @file.write(message)
190
+ @log_count += 1
191
+ @log_size += message.size
192
+ rescue StandardError => e
193
+ if count < retry_count
194
+ count += 1
195
+ reopen
196
+ retry
197
+ end
198
+ raise(e)
199
+ end
100
200
  true
101
201
  end
102
202
 
103
203
  # Flush all pending logs to disk.
104
- # Waits for all sent documents to be writted to disk
204
+ # Waits for all sent documents to be written to disk
105
205
  def flush
106
- @log.flush if @log.respond_to?(:flush)
206
+ @file&.flush
207
+ end
208
+
209
+ private
210
+
211
+ attr_writer :log_count, :log_size, :current_file_name, :reopen_at
212
+
213
+ def time_to_reopen?
214
+ return true unless @file
215
+
216
+ (reopen_count.positive? && (log_count >= reopen_count)) ||
217
+ (reopen_size.positive? && (log_size >= reopen_size)) ||
218
+ (reopen_at && (Time.now > reopen_at))
219
+ end
220
+
221
+ def apply_format_directives(file_name)
222
+ return file_name unless file_name.include?("%")
223
+
224
+ file_name.gsub(/%(.)/) { format_directive(Regexp.last_match(1)) }
225
+ end
226
+
227
+ def format_directive(directive)
228
+ case directive
229
+ when "p"
230
+ $$
231
+ when "n"
232
+ SemanticLogger.host.split(".")[0]
233
+ when "N"
234
+ SemanticLogger.host
235
+ when "a"
236
+ SemanticLogger.application
237
+ when "e"
238
+ SemanticLogger.environment
239
+ when "D"
240
+ Date.today.strftime("%Y%m%d")
241
+ when "Y", "C", "y", "m", "d", "j", "U", "W"
242
+ Date.today.strftime("%#{directive}")
243
+ when "T"
244
+ Time.now.strftime("%H%M%S")
245
+ when "H", "M", "S"
246
+ Time.now.strftime("%#{directive}")
247
+ when "%"
248
+ "%"
249
+ else
250
+ raise(ArgumentError, "Format Directive '#{directive}' in file_name: #{file_name} is not supported.")
251
+ end
252
+ end
253
+
254
+ def next_reopen_period(period_string)
255
+ return unless period_string
256
+
257
+ duration, period = parse_period(period_string)
258
+ calculate_reopen_at(duration, period)
259
+ end
260
+
261
+ def parse_period(period_string)
262
+ match = period_string.to_s.downcase.gsub(/\s+/, "").match(/([\d.]+)([mhd])/)
263
+ unless match
264
+ raise(ArgumentError,
265
+ "Invalid period definition: #{period_string}, must begin with an integer, followed by m,h, or d.")
266
+ end
267
+
268
+ duration = match[1]
269
+ period = match[2]
270
+ raise(ArgumentError, "Invalid or missing duration in: #{period_string}, must begin with an integer.") unless duration
271
+ raise(ArgumentError, "Invalid or missing period in: #{period_string}, must end with m,h, or d.") unless period
272
+
273
+ [duration.to_i, period]
274
+ end
275
+
276
+ # Round down the current time based on the period, then add on the duration for that period
277
+ def calculate_reopen_at(duration, period, time = Time.now)
278
+ case period
279
+ when "m"
280
+ Time.new(time.year, time.month, time.day, time.hour, time.min, 0) + (duration * 60)
281
+ when "h"
282
+ Time.new(time.year, time.month, time.day, time.hour, 0, 0) + (duration * 60 * 60)
283
+ when "d"
284
+ Time.new(time.year, time.month, time.day, 0, 0, 0) + (duration * 24 * 60 * 60)
285
+ else
286
+ raise(ArgumentError, "Invalid or missing period in: #{reopen_period}, must end with m,h, or d.")
287
+ end
107
288
  end
108
289
  end
109
290
  end